[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 12:15:42.036551  5352 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.58.62:45205
I20260709 12:15:42.038152  5352 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 12:15:42.039249  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:42.057317  5359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:42.057523  5358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:42.058406  5352 server_base.cc:1061] running on GCE node
W20260709 12:15:42.057783  5361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:43.309126  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:15:43.309373  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:43.309538  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599343309521 us; error 0 us; skew 500 ppm
I20260709 12:15:43.310321  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:43.319195  5352 webserver.cc:533] Webserver started at http://127.5.58.62:40653/ using document root <none> and password file <none>
I20260709 12:15:43.321033  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:43.321295  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:43.321744  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:43.328085  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599341860723-5352-0/minicluster-data/master-0-root/instance:
uuid: "a585945ae7a64ca495b67d749f3374bf"
format_stamp: "Formatted at 2026-07-09 12:15:43 on dist-test-slave-xs2d"
I20260709 12:15:43.336845  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.007s	sys 0.001s
I20260709 12:15:43.342150  5367 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:43.343760  5352 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:15:43.344194  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599341860723-5352-0/minicluster-data/master-0-root
uuid: "a585945ae7a64ca495b67d749f3374bf"
format_stamp: "Formatted at 2026-07-09 12:15:43 on dist-test-slave-xs2d"
I20260709 12:15:43.344668  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599341860723-5352-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599341860723-5352-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599341860723-5352-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:43.404196  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:43.405833  5352 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 12:15:43.406332  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:43.484514  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.62:45205
I20260709 12:15:43.484629  5418 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.62:45205 every 8 connection(s)
I20260709 12:15:43.493033  5419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:43.516126  5419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a585945ae7a64ca495b67d749f3374bf: Bootstrap starting.
I20260709 12:15:43.522334  5419 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a585945ae7a64ca495b67d749f3374bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:43.524271  5419 log.cc:826] T 00000000000000000000000000000000 P a585945ae7a64ca495b67d749f3374bf: Log is configured to *not* fsync() on all Append() calls
I20260709 12:15:43.529091  5419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a585945ae7a64ca495b67d749f3374bf: No bootstrap required, opened a new log
I20260709 12:15:43.552362  5419 raft_consensus.cc:359] T 00000000000000000000000000000000 P a585945ae7a64ca495b67d749f3374bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a585945ae7a64ca495b67d749f3374bf" member_type: VOTER }
I20260709 12:15:43.552939  5419 raft_consensus.cc:385] T 00000000000000000000000000000000 P a585945ae7a64ca495b67d749f3374bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:43.553262  5419 raft_consensus.cc:740] T 00000000000000000000000000000000 P a585945ae7a64ca495b67d749f3374bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a585945ae7a64ca495b67d749f3374bf, State: Initialized, Role: FOLLOWER
I20260709 12:15:43.554200  5419 consensus_queue.cc:260] T 00000000000000000000000000000000 P a585945ae7a64ca495b67d749f3374bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a585945ae7a64ca495b67d749f3374bf" member_type: VOTER }
I20260709 12:15:43.554745  5419 raft_consensus.cc:399] T 00000000000000000000000000000000 P a585945ae7a64ca495b67d749f3374bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:15:43.555040  5419 raft_consensus.cc:493] T 00000000000000000000000000000000 P a585945ae7a64ca495b67d749f3374bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:15:43.555455  5419 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a585945ae7a64ca495b67d749f3374bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:43.559633  5419 raft_consensus.cc:515] T 00000000000000000000000000000000 P a585945ae7a64ca495b67d749f3374bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a585945ae7a64ca495b67d749f3374bf" member_type: VOTER }
I20260709 12:15:43.560509  5419 leader_election.cc:304] T 00000000000000000000000000000000 P a585945ae7a64ca495b67d749f3374bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a585945ae7a64ca495b67d749f3374bf; no voters: 
I20260709 12:15:43.562459  5419 leader_election.cc:290] T 00000000000000000000000000000000 P a585945ae7a64ca495b67d749f3374bf [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:15:43.562841  5422 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a585945ae7a64ca495b67d749f3374bf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:43.565526  5422 raft_consensus.cc:697] T 00000000000000000000000000000000 P a585945ae7a64ca495b67d749f3374bf [term 1 LEADER]: Becoming Leader. State: Replica: a585945ae7a64ca495b67d749f3374bf, State: Running, Role: LEADER
I20260709 12:15:43.566437  5422 consensus_queue.cc:237] T 00000000000000000000000000000000 P a585945ae7a64ca495b67d749f3374bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a585945ae7a64ca495b67d749f3374bf" member_type: VOTER }
I20260709 12:15:43.566979  5419 sys_catalog.cc:565] T 00000000000000000000000000000000 P a585945ae7a64ca495b67d749f3374bf [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:15:43.578287  5424 sys_catalog.cc:455] T 00000000000000000000000000000000 P a585945ae7a64ca495b67d749f3374bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader a585945ae7a64ca495b67d749f3374bf. Latest consensus state: current_term: 1 leader_uuid: "a585945ae7a64ca495b67d749f3374bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a585945ae7a64ca495b67d749f3374bf" member_type: VOTER } }
I20260709 12:15:43.578505  5423 sys_catalog.cc:455] T 00000000000000000000000000000000 P a585945ae7a64ca495b67d749f3374bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a585945ae7a64ca495b67d749f3374bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a585945ae7a64ca495b67d749f3374bf" member_type: VOTER } }
I20260709 12:15:43.579159  5424 sys_catalog.cc:458] T 00000000000000000000000000000000 P a585945ae7a64ca495b67d749f3374bf [sys.catalog]: This master's current role is: LEADER
I20260709 12:15:43.579286  5423 sys_catalog.cc:458] T 00000000000000000000000000000000 P a585945ae7a64ca495b67d749f3374bf [sys.catalog]: This master's current role is: LEADER
I20260709 12:15:43.582393  5431 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:15:43.597220  5431 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:15:43.599742  5352 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:15:43.615290  5431 catalog_manager.cc:1383] Generated new cluster ID: bb78e114bb2946659508e80469ba6cdc
I20260709 12:15:43.615633  5431 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:15:43.637719  5431 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:15:43.639333  5431 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:15:43.658551  5431 catalog_manager.cc:6070] T 00000000000000000000000000000000 P a585945ae7a64ca495b67d749f3374bf: Generated new TSK 0
I20260709 12:15:43.659605  5431 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:15:43.666868  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:43.676734  5441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:43.677958  5442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:43.682137  5352 server_base.cc:1061] running on GCE node
W20260709 12:15:43.682353  5444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:43.683560  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:15:43.683804  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:43.684024  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599343684002 us; error 0 us; skew 500 ppm
I20260709 12:15:43.684752  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:43.689754  5352 webserver.cc:533] Webserver started at http://127.5.58.1:46003/ using document root <none> and password file <none>
I20260709 12:15:43.690335  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:43.690572  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:43.690866  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:43.692008  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599341860723-5352-0/minicluster-data/ts-0-root/instance:
uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
format_stamp: "Formatted at 2026-07-09 12:15:43 on dist-test-slave-xs2d"
I20260709 12:15:43.698307  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.001s	sys 0.004s
I20260709 12:15:43.703130  5449 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:43.704113  5352 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.000s
I20260709 12:15:43.704495  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599341860723-5352-0/minicluster-data/ts-0-root
uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
format_stamp: "Formatted at 2026-07-09 12:15:43 on dist-test-slave-xs2d"
I20260709 12:15:43.704903  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599341860723-5352-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599341860723-5352-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599341860723-5352-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:43.729823  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:43.731613  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:43.734463  5352 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:15:43.739440  5352 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:15:43.739717  5352 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:43.739996  5352 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:15:43.740187  5352 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:43.849354  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.1:39021
I20260709 12:15:43.849457  5511 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.1:39021 every 8 connection(s)
I20260709 12:15:43.857492  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:43.875015  5516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:43.880060  5517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:43.897349  5519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:43.898602  5352 server_base.cc:1061] running on GCE node
I20260709 12:15:43.899645  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:15:43.899904  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:43.900117  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599343900097 us; error 0 us; skew 500 ppm
I20260709 12:15:43.900954  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:43.904981  5352 webserver.cc:533] Webserver started at http://127.5.58.2:45757/ using document root <none> and password file <none>
I20260709 12:15:43.905786  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:43.906054  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:43.906127  5512 heartbeater.cc:344] Connected to a master server at 127.5.58.62:45205
I20260709 12:15:43.906474  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:43.906657  5512 heartbeater.cc:461] Registering TS with master...
I20260709 12:15:43.907940  5512 heartbeater.cc:507] Master 127.5.58.62:45205 requested a full tablet report, sending...
I20260709 12:15:43.908139  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599341860723-5352-0/minicluster-data/ts-1-root/instance:
uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
format_stamp: "Formatted at 2026-07-09 12:15:43 on dist-test-slave-xs2d"
I20260709 12:15:43.912561  5384 ts_manager.cc:194] Registered new tserver with Master: d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021)
I20260709 12:15:43.916232  5384 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44384
I20260709 12:15:43.916576  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.007s	sys 0.001s
I20260709 12:15:43.923175  5524 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:43.924067  5352 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.003s
I20260709 12:15:43.924374  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599341860723-5352-0/minicluster-data/ts-1-root
uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
format_stamp: "Formatted at 2026-07-09 12:15:43 on dist-test-slave-xs2d"
I20260709 12:15:43.924664  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599341860723-5352-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599341860723-5352-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599341860723-5352-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:43.944706  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:43.946122  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:43.948941  5352 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:15:43.952797  5352 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:15:43.953027  5352 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:43.953325  5352 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:15:43.953526  5352 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:44.064100  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.2:36305
I20260709 12:15:44.064409  5586 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.2:36305 every 8 connection(s)
I20260709 12:15:44.073463  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260709 12:15:44.085659  5587 heartbeater.cc:344] Connected to a master server at 127.5.58.62:45205
W20260709 12:15:44.085743  5590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:44.086117  5587 heartbeater.cc:461] Registering TS with master...
I20260709 12:15:44.087110  5587 heartbeater.cc:507] Master 127.5.58.62:45205 requested a full tablet report, sending...
W20260709 12:15:44.089026  5591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:44.089481  5384 ts_manager.cc:194] Registered new tserver with Master: 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2:36305)
I20260709 12:15:44.091456  5384 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44386
W20260709 12:15:44.092334  5593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:44.098769  5352 server_base.cc:1061] running on GCE node
I20260709 12:15:44.099756  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:15:44.100006  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:44.100183  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599344100164 us; error 0 us; skew 500 ppm
I20260709 12:15:44.101166  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:44.103771  5352 webserver.cc:533] Webserver started at http://127.5.58.3:44747/ using document root <none> and password file <none>
I20260709 12:15:44.104310  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:44.104580  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:44.104839  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:44.106012  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599341860723-5352-0/minicluster-data/ts-2-root/instance:
uuid: "92adb80abb834eceaef035f370597703"
format_stamp: "Formatted at 2026-07-09 12:15:44 on dist-test-slave-xs2d"
I20260709 12:15:44.110864  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:15:44.114499  5598 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:44.115341  5352 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:15:44.115653  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599341860723-5352-0/minicluster-data/ts-2-root
uuid: "92adb80abb834eceaef035f370597703"
format_stamp: "Formatted at 2026-07-09 12:15:44 on dist-test-slave-xs2d"
I20260709 12:15:44.115942  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599341860723-5352-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599341860723-5352-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599341860723-5352-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:44.137297  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:44.138525  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:44.141111  5352 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:15:44.144063  5352 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:15:44.144253  5352 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:44.144614  5352 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:15:44.144788  5352 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:44.249123  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.3:41427
I20260709 12:15:44.249226  5660 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.3:41427 every 8 connection(s)
I20260709 12:15:44.269594  5661 heartbeater.cc:344] Connected to a master server at 127.5.58.62:45205
I20260709 12:15:44.270051  5661 heartbeater.cc:461] Registering TS with master...
I20260709 12:15:44.270947  5661 heartbeater.cc:507] Master 127.5.58.62:45205 requested a full tablet report, sending...
I20260709 12:15:44.273113  5384 ts_manager.cc:194] Registered new tserver with Master: 92adb80abb834eceaef035f370597703 (127.5.58.3:41427)
I20260709 12:15:44.273267  5352 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013225183s
I20260709 12:15:44.275090  5384 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44392
I20260709 12:15:44.334565  5384 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44408:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:15:44.338351  5384 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:15:44.474195  5626 tablet_service.cc:1511] Processing CreateTablet for tablet a89ab5b73a4a42f7b8b529758c61edf7 (DEFAULT_TABLE table=test-workload [id=3148f90a7bb040639fa3967ad04befa5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:15:44.475005  5625 tablet_service.cc:1511] Processing CreateTablet for tablet 4fd3cf9b75144e5c88eee1a8917e283e (DEFAULT_TABLE table=test-workload [id=3148f90a7bb040639fa3967ad04befa5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:15:44.476265  5477 tablet_service.cc:1511] Processing CreateTablet for tablet a89ab5b73a4a42f7b8b529758c61edf7 (DEFAULT_TABLE table=test-workload [id=3148f90a7bb040639fa3967ad04befa5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:15:44.477773  5477 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a89ab5b73a4a42f7b8b529758c61edf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:44.481796  5624 tablet_service.cc:1511] Processing CreateTablet for tablet 930c10c719284908b31b1b583ac8bc92 (DEFAULT_TABLE table=test-workload [id=3148f90a7bb040639fa3967ad04befa5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:15:44.483798  5623 tablet_service.cc:1511] Processing CreateTablet for tablet 09cfb685589d470ab66f97d1d81df47c (DEFAULT_TABLE table=test-workload [id=3148f90a7bb040639fa3967ad04befa5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:15:44.479782  5622 tablet_service.cc:1511] Processing CreateTablet for tablet f0f1dad5e1c540edbbba5708a7698085 (DEFAULT_TABLE table=test-workload [id=3148f90a7bb040639fa3967ad04befa5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:15:44.486886  5476 tablet_service.cc:1511] Processing CreateTablet for tablet 4fd3cf9b75144e5c88eee1a8917e283e (DEFAULT_TABLE table=test-workload [id=3148f90a7bb040639fa3967ad04befa5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:15:44.487231  5621 tablet_service.cc:1511] Processing CreateTablet for tablet 1126f26405b742a6a136656ca8c86696 (DEFAULT_TABLE table=test-workload [id=3148f90a7bb040639fa3967ad04befa5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:15:44.488530  5476 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fd3cf9b75144e5c88eee1a8917e283e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:44.489874  5623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09cfb685589d470ab66f97d1d81df47c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:44.490064  5620 tablet_service.cc:1511] Processing CreateTablet for tablet a00c228a31d744968a6b32e3601d51eb (DEFAULT_TABLE table=test-workload [id=3148f90a7bb040639fa3967ad04befa5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:15:44.491571  5620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a00c228a31d744968a6b32e3601d51eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:44.493046  5624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 930c10c719284908b31b1b583ac8bc92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:44.510741  5619 tablet_service.cc:1511] Processing CreateTablet for tablet 62d225e09ba14ac4bd3bcc68a7b3495f (DEFAULT_TABLE table=test-workload [id=3148f90a7bb040639fa3967ad04befa5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:15:44.512707  5626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a89ab5b73a4a42f7b8b529758c61edf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:44.514556  5622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0f1dad5e1c540edbbba5708a7698085. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:44.516150  5625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fd3cf9b75144e5c88eee1a8917e283e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:44.518530  5621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1126f26405b742a6a136656ca8c86696. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:44.525611  5475 tablet_service.cc:1511] Processing CreateTablet for tablet 930c10c719284908b31b1b583ac8bc92 (DEFAULT_TABLE table=test-workload [id=3148f90a7bb040639fa3967ad04befa5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:15:44.534883  5475 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 930c10c719284908b31b1b583ac8bc92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:44.542377  5618 tablet_service.cc:1511] Processing CreateTablet for tablet f66af11cf9374a31b237a393194eccae (DEFAULT_TABLE table=test-workload [id=3148f90a7bb040639fa3967ad04befa5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:15:44.552464  5618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f66af11cf9374a31b237a393194eccae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:44.549163  5474 tablet_service.cc:1511] Processing CreateTablet for tablet 09cfb685589d470ab66f97d1d81df47c (DEFAULT_TABLE table=test-workload [id=3148f90a7bb040639fa3967ad04befa5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:15:44.564109  5470 tablet_service.cc:1511] Processing CreateTablet for tablet 62d225e09ba14ac4bd3bcc68a7b3495f (DEFAULT_TABLE table=test-workload [id=3148f90a7bb040639fa3967ad04befa5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:15:44.551265  5472 tablet_service.cc:1511] Processing CreateTablet for tablet 1126f26405b742a6a136656ca8c86696 (DEFAULT_TABLE table=test-workload [id=3148f90a7bb040639fa3967ad04befa5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:15:44.561074  5471 tablet_service.cc:1511] Processing CreateTablet for tablet a00c228a31d744968a6b32e3601d51eb (DEFAULT_TABLE table=test-workload [id=3148f90a7bb040639fa3967ad04befa5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:15:44.556200  5473 tablet_service.cc:1511] Processing CreateTablet for tablet f0f1dad5e1c540edbbba5708a7698085 (DEFAULT_TABLE table=test-workload [id=3148f90a7bb040639fa3967ad04befa5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:15:44.572430  5470 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62d225e09ba14ac4bd3bcc68a7b3495f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:44.575153  5473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0f1dad5e1c540edbbba5708a7698085. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:44.577415  5471 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a00c228a31d744968a6b32e3601d51eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:44.586926  5469 tablet_service.cc:1511] Processing CreateTablet for tablet f66af11cf9374a31b237a393194eccae (DEFAULT_TABLE table=test-workload [id=3148f90a7bb040639fa3967ad04befa5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:15:44.589382  5472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1126f26405b742a6a136656ca8c86696. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:44.592720  5469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f66af11cf9374a31b237a393194eccae. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:15:44.592628  5432 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:15:44.590054  5619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62d225e09ba14ac4bd3bcc68a7b3495f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:44.605183  5474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09cfb685589d470ab66f97d1d81df47c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:44.604257  5552 tablet_service.cc:1511] Processing CreateTablet for tablet a89ab5b73a4a42f7b8b529758c61edf7 (DEFAULT_TABLE table=test-workload [id=3148f90a7bb040639fa3967ad04befa5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:15:44.608798  5551 tablet_service.cc:1511] Processing CreateTablet for tablet 4fd3cf9b75144e5c88eee1a8917e283e (DEFAULT_TABLE table=test-workload [id=3148f90a7bb040639fa3967ad04befa5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:15:44.608605  5550 tablet_service.cc:1511] Processing CreateTablet for tablet 930c10c719284908b31b1b583ac8bc92 (DEFAULT_TABLE table=test-workload [id=3148f90a7bb040639fa3967ad04befa5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:15:44.625160  5549 tablet_service.cc:1511] Processing CreateTablet for tablet 09cfb685589d470ab66f97d1d81df47c (DEFAULT_TABLE table=test-workload [id=3148f90a7bb040639fa3967ad04befa5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:15:44.633224  5547 tablet_service.cc:1511] Processing CreateTablet for tablet 1126f26405b742a6a136656ca8c86696 (DEFAULT_TABLE table=test-workload [id=3148f90a7bb040639fa3967ad04befa5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:15:44.637882  5548 tablet_service.cc:1511] Processing CreateTablet for tablet f0f1dad5e1c540edbbba5708a7698085 (DEFAULT_TABLE table=test-workload [id=3148f90a7bb040639fa3967ad04befa5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:15:44.643194  5552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a89ab5b73a4a42f7b8b529758c61edf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:44.645434  5549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09cfb685589d470ab66f97d1d81df47c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:44.661598  5551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fd3cf9b75144e5c88eee1a8917e283e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:44.667079  5547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1126f26405b742a6a136656ca8c86696. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:44.675381  5550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 930c10c719284908b31b1b583ac8bc92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:44.679989  5548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0f1dad5e1c540edbbba5708a7698085. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:44.714376  5546 tablet_service.cc:1511] Processing CreateTablet for tablet a00c228a31d744968a6b32e3601d51eb (DEFAULT_TABLE table=test-workload [id=3148f90a7bb040639fa3967ad04befa5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:15:44.716147  5546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a00c228a31d744968a6b32e3601d51eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:44.726352  5545 tablet_service.cc:1511] Processing CreateTablet for tablet 62d225e09ba14ac4bd3bcc68a7b3495f (DEFAULT_TABLE table=test-workload [id=3148f90a7bb040639fa3967ad04befa5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:15:44.728394  5545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62d225e09ba14ac4bd3bcc68a7b3495f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:44.738616  5544 tablet_service.cc:1511] Processing CreateTablet for tablet f66af11cf9374a31b237a393194eccae (DEFAULT_TABLE table=test-workload [id=3148f90a7bb040639fa3967ad04befa5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:15:44.740461  5544 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f66af11cf9374a31b237a393194eccae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:44.762790  5679 tablet_bootstrap.cc:492] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703: Bootstrap starting.
I20260709 12:15:44.791998  5680 tablet_bootstrap.cc:492] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c: Bootstrap starting.
I20260709 12:15:44.851150  5681 tablet_bootstrap.cc:492] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Bootstrap starting.
I20260709 12:15:44.864730  5680 tablet_bootstrap.cc:654] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.877185  5679 tablet_bootstrap.cc:654] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.920967  5512 heartbeater.cc:499] Master 127.5.58.62:45205 was elected leader, sending a full tablet report...
I20260709 12:15:44.923826  5679 tablet_bootstrap.cc:492] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703: No bootstrap required, opened a new log
I20260709 12:15:44.924029  5681 tablet_bootstrap.cc:654] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.924525  5679 ts_tablet_manager.cc:1403] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703: Time spent bootstrapping tablet: real 0.164s	user 0.020s	sys 0.006s
I20260709 12:15:44.927273  5679 raft_consensus.cc:359] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:44.927979  5679 raft_consensus.cc:385] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.928386  5679 raft_consensus.cc:740] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92adb80abb834eceaef035f370597703, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.929327  5679 consensus_queue.cc:260] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:44.932421  5679 ts_tablet_manager.cc:1434] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:15:44.932905  5661 heartbeater.cc:499] Master 127.5.58.62:45205 was elected leader, sending a full tablet report...
I20260709 12:15:44.933929  5679 tablet_bootstrap.cc:492] T 4fd3cf9b75144e5c88eee1a8917e283e P 92adb80abb834eceaef035f370597703: Bootstrap starting.
I20260709 12:15:44.941104  5679 tablet_bootstrap.cc:654] T 4fd3cf9b75144e5c88eee1a8917e283e P 92adb80abb834eceaef035f370597703: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.947115  5679 tablet_bootstrap.cc:492] T 4fd3cf9b75144e5c88eee1a8917e283e P 92adb80abb834eceaef035f370597703: No bootstrap required, opened a new log
I20260709 12:15:44.947605  5679 ts_tablet_manager.cc:1403] T 4fd3cf9b75144e5c88eee1a8917e283e P 92adb80abb834eceaef035f370597703: Time spent bootstrapping tablet: real 0.014s	user 0.012s	sys 0.000s
I20260709 12:15:44.951917  5680 tablet_bootstrap.cc:492] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c: No bootstrap required, opened a new log
I20260709 12:15:44.952435  5680 ts_tablet_manager.cc:1403] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c: Time spent bootstrapping tablet: real 0.162s	user 0.017s	sys 0.013s
I20260709 12:15:44.951726  5679 raft_consensus.cc:359] T 4fd3cf9b75144e5c88eee1a8917e283e P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:44.952901  5679 raft_consensus.cc:385] T 4fd3cf9b75144e5c88eee1a8917e283e P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.953378  5679 raft_consensus.cc:740] T 4fd3cf9b75144e5c88eee1a8917e283e P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92adb80abb834eceaef035f370597703, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.954502  5679 consensus_queue.cc:260] T 4fd3cf9b75144e5c88eee1a8917e283e P 92adb80abb834eceaef035f370597703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:44.955399  5680 raft_consensus.cc:359] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:44.957623  5680 raft_consensus.cc:385] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.958099  5680 raft_consensus.cc:740] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7f53a4ef21a4d8788fae6df6bb5f98c, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.958313  5681 tablet_bootstrap.cc:492] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1: No bootstrap required, opened a new log
I20260709 12:15:44.958475  5679 ts_tablet_manager.cc:1434] T 4fd3cf9b75144e5c88eee1a8917e283e P 92adb80abb834eceaef035f370597703: Time spent starting tablet: real 0.011s	user 0.011s	sys 0.000s
I20260709 12:15:44.958930  5681 ts_tablet_manager.cc:1403] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Time spent bootstrapping tablet: real 0.109s	user 0.014s	sys 0.011s
I20260709 12:15:44.959697  5679 tablet_bootstrap.cc:492] T f66af11cf9374a31b237a393194eccae P 92adb80abb834eceaef035f370597703: Bootstrap starting.
I20260709 12:15:44.961401  5680 consensus_queue.cc:260] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:44.962186  5681 raft_consensus.cc:359] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:44.967639  5679 tablet_bootstrap.cc:654] T f66af11cf9374a31b237a393194eccae P 92adb80abb834eceaef035f370597703: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.967922  5681 raft_consensus.cc:385] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.968376  5681 raft_consensus.cc:740] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ea2ad5b842b4bf3868b4e25cae3e9f1, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.969137  5681 consensus_queue.cc:260] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:44.976162  5680 ts_tablet_manager.cc:1434] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.003s
I20260709 12:15:44.977198  5680 tablet_bootstrap.cc:492] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c: Bootstrap starting.
I20260709 12:15:44.980571  5587 heartbeater.cc:499] Master 127.5.58.62:45205 was elected leader, sending a full tablet report...
I20260709 12:15:44.981519  5681 ts_tablet_manager.cc:1434] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
I20260709 12:15:44.981813  5679 tablet_bootstrap.cc:492] T f66af11cf9374a31b237a393194eccae P 92adb80abb834eceaef035f370597703: No bootstrap required, opened a new log
I20260709 12:15:44.982313  5679 ts_tablet_manager.cc:1403] T f66af11cf9374a31b237a393194eccae P 92adb80abb834eceaef035f370597703: Time spent bootstrapping tablet: real 0.023s	user 0.018s	sys 0.001s
I20260709 12:15:44.982388  5681 tablet_bootstrap.cc:492] T 62d225e09ba14ac4bd3bcc68a7b3495f P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Bootstrap starting.
I20260709 12:15:44.987986  5680 tablet_bootstrap.cc:654] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.992125  5679 raft_consensus.cc:359] T f66af11cf9374a31b237a393194eccae P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:44.993158  5679 raft_consensus.cc:385] T f66af11cf9374a31b237a393194eccae P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:44.993558  5679 raft_consensus.cc:740] T f66af11cf9374a31b237a393194eccae P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92adb80abb834eceaef035f370597703, State: Initialized, Role: FOLLOWER
I20260709 12:15:44.994856  5679 consensus_queue.cc:260] T f66af11cf9374a31b237a393194eccae P 92adb80abb834eceaef035f370597703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:44.998261  5679 ts_tablet_manager.cc:1434] T f66af11cf9374a31b237a393194eccae P 92adb80abb834eceaef035f370597703: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 12:15:44.998644  5681 tablet_bootstrap.cc:654] T 62d225e09ba14ac4bd3bcc68a7b3495f P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:44.999747  5679 tablet_bootstrap.cc:492] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703: Bootstrap starting.
I20260709 12:15:45.005162  5680 tablet_bootstrap.cc:492] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c: No bootstrap required, opened a new log
I20260709 12:15:45.005662  5680 ts_tablet_manager.cc:1403] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.000s
I20260709 12:15:45.009766  5680 raft_consensus.cc:359] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } }
I20260709 12:15:45.010433  5679 tablet_bootstrap.cc:654] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:45.010727  5680 raft_consensus.cc:385] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:45.011077  5680 raft_consensus.cc:740] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7f53a4ef21a4d8788fae6df6bb5f98c, State: Initialized, Role: FOLLOWER
I20260709 12:15:45.011855  5680 consensus_queue.cc:260] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } }
W20260709 12:15:45.013406  5662 tablet.cc:2406] T 4fd3cf9b75144e5c88eee1a8917e283e P 92adb80abb834eceaef035f370597703: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:15:45.018778  5681 tablet_bootstrap.cc:492] T 62d225e09ba14ac4bd3bcc68a7b3495f P 3ea2ad5b842b4bf3868b4e25cae3e9f1: No bootstrap required, opened a new log
I20260709 12:15:45.019611  5681 ts_tablet_manager.cc:1403] T 62d225e09ba14ac4bd3bcc68a7b3495f P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Time spent bootstrapping tablet: real 0.037s	user 0.011s	sys 0.001s
I20260709 12:15:45.022853  5681 raft_consensus.cc:359] T 62d225e09ba14ac4bd3bcc68a7b3495f P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.023660  5681 raft_consensus.cc:385] T 62d225e09ba14ac4bd3bcc68a7b3495f P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:45.023975  5681 raft_consensus.cc:740] T 62d225e09ba14ac4bd3bcc68a7b3495f P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ea2ad5b842b4bf3868b4e25cae3e9f1, State: Initialized, Role: FOLLOWER
I20260709 12:15:45.026175  5680 ts_tablet_manager.cc:1434] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 12:15:45.027753  5687 raft_consensus.cc:493] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:45.028242  5687 raft_consensus.cc:515] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.034595  5687 leader_election.cc:290] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2:36305), 92adb80abb834eceaef035f370597703 (127.5.58.3:41427)
I20260709 12:15:45.041226  5680 tablet_bootstrap.cc:492] T 62d225e09ba14ac4bd3bcc68a7b3495f P d7f53a4ef21a4d8788fae6df6bb5f98c: Bootstrap starting.
I20260709 12:15:45.039631  5681 consensus_queue.cc:260] T 62d225e09ba14ac4bd3bcc68a7b3495f P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.047231  5681 ts_tablet_manager.cc:1434] T 62d225e09ba14ac4bd3bcc68a7b3495f P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Time spent starting tablet: real 0.027s	user 0.000s	sys 0.009s
I20260709 12:15:45.057428  5681 tablet_bootstrap.cc:492] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Bootstrap starting.
I20260709 12:15:45.058617  5562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fd3cf9b75144e5c88eee1a8917e283e" candidate_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" is_pre_election: true
W20260709 12:15:45.060566  5452 leader_election.cc:343] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2:36305): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:45.062969  5680 tablet_bootstrap.cc:654] T 62d225e09ba14ac4bd3bcc68a7b3495f P d7f53a4ef21a4d8788fae6df6bb5f98c: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:45.065335  5681 tablet_bootstrap.cc:654] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:45.067628  5679 tablet_bootstrap.cc:492] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703: No bootstrap required, opened a new log
I20260709 12:15:45.068161  5679 ts_tablet_manager.cc:1403] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703: Time spent bootstrapping tablet: real 0.069s	user 0.022s	sys 0.024s
I20260709 12:15:45.071820  5679 raft_consensus.cc:359] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.072664  5679 raft_consensus.cc:385] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:45.072983  5679 raft_consensus.cc:740] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92adb80abb834eceaef035f370597703, State: Initialized, Role: FOLLOWER
I20260709 12:15:45.074298  5679 consensus_queue.cc:260] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.077600  5684 raft_consensus.cc:493] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:45.085800  5684 raft_consensus.cc:515] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.097673  5636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fd3cf9b75144e5c88eee1a8917e283e" candidate_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92adb80abb834eceaef035f370597703" is_pre_election: true
I20260709 12:15:45.098866  5636 raft_consensus.cc:2468] T 4fd3cf9b75144e5c88eee1a8917e283e P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7f53a4ef21a4d8788fae6df6bb5f98c in term 0.
I20260709 12:15:45.100442  5688 raft_consensus.cc:493] T 62d225e09ba14ac4bd3bcc68a7b3495f P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:45.101140  5688 raft_consensus.cc:515] T 62d225e09ba14ac4bd3bcc68a7b3495f P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.105197  5688 leader_election.cc:290] T 62d225e09ba14ac4bd3bcc68a7b3495f P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021), 92adb80abb834eceaef035f370597703 (127.5.58.3:41427)
I20260709 12:15:45.115890  5679 ts_tablet_manager.cc:1434] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.001s
I20260709 12:15:45.137023  5684 leader_election.cc:290] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021), 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2:36305)
I20260709 12:15:45.147312  5679 tablet_bootstrap.cc:492] T 62d225e09ba14ac4bd3bcc68a7b3495f P 92adb80abb834eceaef035f370597703: Bootstrap starting.
I20260709 12:15:45.152514  5681 tablet_bootstrap.cc:492] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1: No bootstrap required, opened a new log
I20260709 12:15:45.153175  5681 ts_tablet_manager.cc:1403] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Time spent bootstrapping tablet: real 0.105s	user 0.023s	sys 0.001s
I20260709 12:15:45.156922  5681 raft_consensus.cc:359] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.157764  5681 raft_consensus.cc:385] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:45.158115  5681 raft_consensus.cc:740] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ea2ad5b842b4bf3868b4e25cae3e9f1, State: Initialized, Role: FOLLOWER
I20260709 12:15:45.158917  5681 consensus_queue.cc:260] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.163772  5680 tablet_bootstrap.cc:492] T 62d225e09ba14ac4bd3bcc68a7b3495f P d7f53a4ef21a4d8788fae6df6bb5f98c: No bootstrap required, opened a new log
I20260709 12:15:45.166697  5679 tablet_bootstrap.cc:654] T 62d225e09ba14ac4bd3bcc68a7b3495f P 92adb80abb834eceaef035f370597703: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:45.166638  5681 ts_tablet_manager.cc:1434] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:15:45.168984  5680 ts_tablet_manager.cc:1403] T 62d225e09ba14ac4bd3bcc68a7b3495f P d7f53a4ef21a4d8788fae6df6bb5f98c: Time spent bootstrapping tablet: real 0.142s	user 0.023s	sys 0.013s
I20260709 12:15:45.178429  5681 tablet_bootstrap.cc:492] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Bootstrap starting.
I20260709 12:15:45.189025  5680 raft_consensus.cc:359] T 62d225e09ba14ac4bd3bcc68a7b3495f P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.190506  5680 raft_consensus.cc:385] T 62d225e09ba14ac4bd3bcc68a7b3495f P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:45.192283  5680 raft_consensus.cc:740] T 62d225e09ba14ac4bd3bcc68a7b3495f P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7f53a4ef21a4d8788fae6df6bb5f98c, State: Initialized, Role: FOLLOWER
I20260709 12:15:45.183606  5450 leader_election.cc:304] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92adb80abb834eceaef035f370597703, d7f53a4ef21a4d8788fae6df6bb5f98c; no voters: 3ea2ad5b842b4bf3868b4e25cae3e9f1
I20260709 12:15:45.193235  5680 consensus_queue.cc:260] T 62d225e09ba14ac4bd3bcc68a7b3495f P d7f53a4ef21a4d8788fae6df6bb5f98c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.197455  5687 raft_consensus.cc:2804] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:45.200456  5687 raft_consensus.cc:493] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:45.201766  5687 raft_consensus.cc:3060] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:45.207643  5681 tablet_bootstrap.cc:654] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:45.214251  5679 tablet_bootstrap.cc:492] T 62d225e09ba14ac4bd3bcc68a7b3495f P 92adb80abb834eceaef035f370597703: No bootstrap required, opened a new log
I20260709 12:15:45.214879  5679 ts_tablet_manager.cc:1403] T 62d225e09ba14ac4bd3bcc68a7b3495f P 92adb80abb834eceaef035f370597703: Time spent bootstrapping tablet: real 0.069s	user 0.016s	sys 0.006s
I20260709 12:15:45.223155  5487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d225e09ba14ac4bd3bcc68a7b3495f" candidate_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" is_pre_election: true
I20260709 12:15:45.222851  5687 raft_consensus.cc:515] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.224184  5487 raft_consensus.cc:2468] T 62d225e09ba14ac4bd3bcc68a7b3495f P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ea2ad5b842b4bf3868b4e25cae3e9f1 in term 0.
I20260709 12:15:45.225526  5636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d225e09ba14ac4bd3bcc68a7b3495f" candidate_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92adb80abb834eceaef035f370597703" is_pre_election: true
I20260709 12:15:45.226366  5527 leader_election.cc:304] T 62d225e09ba14ac4bd3bcc68a7b3495f P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea2ad5b842b4bf3868b4e25cae3e9f1, d7f53a4ef21a4d8788fae6df6bb5f98c; no voters: 
I20260709 12:15:45.227420  5688 raft_consensus.cc:2804] T 62d225e09ba14ac4bd3bcc68a7b3495f P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:45.227782  5688 raft_consensus.cc:493] T 62d225e09ba14ac4bd3bcc68a7b3495f P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:45.228135  5688 raft_consensus.cc:3060] T 62d225e09ba14ac4bd3bcc68a7b3495f P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:45.229424  5562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fd3cf9b75144e5c88eee1a8917e283e" candidate_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
I20260709 12:15:45.230659  5687 leader_election.cc:290] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 1 election: Requested vote from peers 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2:36305), 92adb80abb834eceaef035f370597703 (127.5.58.3:41427)
W20260709 12:15:45.231313  5452 leader_election.cc:343] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2:36305): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:45.231603  5635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fd3cf9b75144e5c88eee1a8917e283e" candidate_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92adb80abb834eceaef035f370597703"
I20260709 12:15:45.232453  5635 raft_consensus.cc:3060] T 4fd3cf9b75144e5c88eee1a8917e283e P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:45.239269  5487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a00c228a31d744968a6b32e3601d51eb" candidate_uuid: "92adb80abb834eceaef035f370597703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" is_pre_election: true
I20260709 12:15:45.240399  5688 raft_consensus.cc:515] T 62d225e09ba14ac4bd3bcc68a7b3495f P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
W20260709 12:15:45.244445  5601 leader_election.cc:343] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:45.244578  5487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d225e09ba14ac4bd3bcc68a7b3495f" candidate_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
I20260709 12:15:45.245162  5487 raft_consensus.cc:3060] T 62d225e09ba14ac4bd3bcc68a7b3495f P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:45.239730  5679 raft_consensus.cc:359] T 62d225e09ba14ac4bd3bcc68a7b3495f P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.250681  5679 raft_consensus.cc:385] T 62d225e09ba14ac4bd3bcc68a7b3495f P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:45.251659  5635 raft_consensus.cc:2468] T 4fd3cf9b75144e5c88eee1a8917e283e P 92adb80abb834eceaef035f370597703 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f53a4ef21a4d8788fae6df6bb5f98c in term 1.
I20260709 12:15:45.253499  5450 leader_election.cc:304] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92adb80abb834eceaef035f370597703, d7f53a4ef21a4d8788fae6df6bb5f98c; no voters: 3ea2ad5b842b4bf3868b4e25cae3e9f1
I20260709 12:15:45.251034  5679 raft_consensus.cc:740] T 62d225e09ba14ac4bd3bcc68a7b3495f P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92adb80abb834eceaef035f370597703, State: Initialized, Role: FOLLOWER
I20260709 12:15:45.254412  5680 ts_tablet_manager.cc:1434] T 62d225e09ba14ac4bd3bcc68a7b3495f P d7f53a4ef21a4d8788fae6df6bb5f98c: Time spent starting tablet: real 0.083s	user 0.007s	sys 0.004s
I20260709 12:15:45.255743  5705 raft_consensus.cc:2804] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:45.256374  5705 raft_consensus.cc:697] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 LEADER]: Becoming Leader. State: Replica: d7f53a4ef21a4d8788fae6df6bb5f98c, State: Running, Role: LEADER
I20260709 12:15:45.258374  5705 consensus_queue.cc:237] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.256973  5635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d225e09ba14ac4bd3bcc68a7b3495f" candidate_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92adb80abb834eceaef035f370597703"
I20260709 12:15:45.262303  5680 tablet_bootstrap.cc:492] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c: Bootstrap starting.
I20260709 12:15:45.257863  5679 consensus_queue.cc:260] T 62d225e09ba14ac4bd3bcc68a7b3495f P 92adb80abb834eceaef035f370597703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.263271  5636 raft_consensus.cc:2468] T 62d225e09ba14ac4bd3bcc68a7b3495f P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ea2ad5b842b4bf3868b4e25cae3e9f1 in term 0.
I20260709 12:15:45.256662  5688 leader_election.cc:290] T 62d225e09ba14ac4bd3bcc68a7b3495f P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 1 election: Requested vote from peers d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021), 92adb80abb834eceaef035f370597703 (127.5.58.3:41427)
I20260709 12:15:45.271324  5681 tablet_bootstrap.cc:492] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1: No bootstrap required, opened a new log
I20260709 12:15:45.271848  5681 ts_tablet_manager.cc:1403] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Time spent bootstrapping tablet: real 0.095s	user 0.013s	sys 0.010s
I20260709 12:15:45.273705  5562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a00c228a31d744968a6b32e3601d51eb" candidate_uuid: "92adb80abb834eceaef035f370597703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" is_pre_election: true
I20260709 12:15:45.274317  5562 raft_consensus.cc:2468] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92adb80abb834eceaef035f370597703 in term 0.
I20260709 12:15:45.276654  5601 leader_election.cc:304] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ea2ad5b842b4bf3868b4e25cae3e9f1, 92adb80abb834eceaef035f370597703; no voters: d7f53a4ef21a4d8788fae6df6bb5f98c
I20260709 12:15:45.277796  5684 raft_consensus.cc:2804] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:45.278242  5684 raft_consensus.cc:493] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:45.278795  5684 raft_consensus.cc:3060] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:45.279197  5679 ts_tablet_manager.cc:1434] T 62d225e09ba14ac4bd3bcc68a7b3495f P 92adb80abb834eceaef035f370597703: Time spent starting tablet: real 0.064s	user 0.002s	sys 0.006s
I20260709 12:15:45.279039  5681 raft_consensus.cc:359] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.281044  5681 raft_consensus.cc:385] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:45.280047  5680 tablet_bootstrap.cc:654] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:45.281158  5487 raft_consensus.cc:2468] T 62d225e09ba14ac4bd3bcc68a7b3495f P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea2ad5b842b4bf3868b4e25cae3e9f1 in term 1.
I20260709 12:15:45.281425  5681 raft_consensus.cc:740] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ea2ad5b842b4bf3868b4e25cae3e9f1, State: Initialized, Role: FOLLOWER
I20260709 12:15:45.283039  5527 leader_election.cc:304] T 62d225e09ba14ac4bd3bcc68a7b3495f P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ea2ad5b842b4bf3868b4e25cae3e9f1, d7f53a4ef21a4d8788fae6df6bb5f98c; no voters: 92adb80abb834eceaef035f370597703
I20260709 12:15:45.283506  5681 consensus_queue.cc:260] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.283941  5688 raft_consensus.cc:2804] T 62d225e09ba14ac4bd3bcc68a7b3495f P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:45.289572  5684 raft_consensus.cc:515] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.292783  5487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a00c228a31d744968a6b32e3601d51eb" candidate_uuid: "92adb80abb834eceaef035f370597703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
I20260709 12:15:45.297434  5679 tablet_bootstrap.cc:492] T f0f1dad5e1c540edbbba5708a7698085 P 92adb80abb834eceaef035f370597703: Bootstrap starting.
W20260709 12:15:45.298346  5601 leader_election.cc:343] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:45.304900  5562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a00c228a31d744968a6b32e3601d51eb" candidate_uuid: "92adb80abb834eceaef035f370597703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
I20260709 12:15:45.305698  5562 raft_consensus.cc:3060] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:45.303432  5684 leader_election.cc:290] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [CANDIDATE]: Term 1 election: Requested vote from peers d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021), 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2:36305)
I20260709 12:15:45.317664  5688 raft_consensus.cc:697] T 62d225e09ba14ac4bd3bcc68a7b3495f P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 LEADER]: Becoming Leader. State: Replica: 3ea2ad5b842b4bf3868b4e25cae3e9f1, State: Running, Role: LEADER
I20260709 12:15:45.321894  5384 catalog_manager.cc:5697] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c reported cstate change: term changed from 0 to 1, leader changed from <none> to d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1). New cstate: current_term: 1 leader_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:45.325280  5688 consensus_queue.cc:237] T 62d225e09ba14ac4bd3bcc68a7b3495f P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.335486  5679 tablet_bootstrap.cc:654] T f0f1dad5e1c540edbbba5708a7698085 P 92adb80abb834eceaef035f370597703: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:45.340085  5708 raft_consensus.cc:493] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:45.343257  5708 raft_consensus.cc:515] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.339686  5562 raft_consensus.cc:2468] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92adb80abb834eceaef035f370597703 in term 1.
I20260709 12:15:45.346520  5708 leader_election.cc:290] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021), 92adb80abb834eceaef035f370597703 (127.5.58.3:41427)
I20260709 12:15:45.347405  5487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66af11cf9374a31b237a393194eccae" candidate_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" is_pre_election: true
I20260709 12:15:45.347708  5635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66af11cf9374a31b237a393194eccae" candidate_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92adb80abb834eceaef035f370597703" is_pre_election: true
I20260709 12:15:45.348124  5601 leader_election.cc:304] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ea2ad5b842b4bf3868b4e25cae3e9f1, 92adb80abb834eceaef035f370597703; no voters: d7f53a4ef21a4d8788fae6df6bb5f98c
I20260709 12:15:45.348438  5635 raft_consensus.cc:2468] T f66af11cf9374a31b237a393194eccae P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ea2ad5b842b4bf3868b4e25cae3e9f1 in term 0.
W20260709 12:15:45.349084  5527 leader_election.cc:343] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:45.349390  5684 raft_consensus.cc:2804] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:45.352540  5525 leader_election.cc:304] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ea2ad5b842b4bf3868b4e25cae3e9f1, 92adb80abb834eceaef035f370597703; no voters: d7f53a4ef21a4d8788fae6df6bb5f98c
I20260709 12:15:45.354081  5705 raft_consensus.cc:493] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:45.354533  5705 raft_consensus.cc:515] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } }
I20260709 12:15:45.353771  5709 raft_consensus.cc:2804] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:45.356751  5705 leader_election.cc:290] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92adb80abb834eceaef035f370597703 (127.5.58.3:41427), 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2:36305)
I20260709 12:15:45.357029  5709 raft_consensus.cc:493] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:45.356748  5679 tablet_bootstrap.cc:492] T f0f1dad5e1c540edbbba5708a7698085 P 92adb80abb834eceaef035f370597703: No bootstrap required, opened a new log
I20260709 12:15:45.357468  5709 raft_consensus.cc:3060] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:45.357496  5635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a89ab5b73a4a42f7b8b529758c61edf7" candidate_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92adb80abb834eceaef035f370597703" is_pre_election: true
I20260709 12:15:45.357873  5679 ts_tablet_manager.cc:1403] T f0f1dad5e1c540edbbba5708a7698085 P 92adb80abb834eceaef035f370597703: Time spent bootstrapping tablet: real 0.061s	user 0.010s	sys 0.008s
I20260709 12:15:45.358239  5562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a89ab5b73a4a42f7b8b529758c61edf7" candidate_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" is_pre_election: true
W20260709 12:15:45.359133  5450 leader_election.cc:343] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 92adb80abb834eceaef035f370597703 (127.5.58.3:41427): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:15:45.360139  5452 leader_election.cc:343] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2:36305): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:45.360626  5452 leader_election.cc:304] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7f53a4ef21a4d8788fae6df6bb5f98c; no voters: 3ea2ad5b842b4bf3868b4e25cae3e9f1, 92adb80abb834eceaef035f370597703
I20260709 12:15:45.361604  5705 raft_consensus.cc:2749] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:45.361433  5679 raft_consensus.cc:359] T f0f1dad5e1c540edbbba5708a7698085 P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.363718  5679 raft_consensus.cc:385] T f0f1dad5e1c540edbbba5708a7698085 P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:45.364137  5679 raft_consensus.cc:740] T f0f1dad5e1c540edbbba5708a7698085 P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92adb80abb834eceaef035f370597703, State: Initialized, Role: FOLLOWER
I20260709 12:15:45.365515  5681 ts_tablet_manager.cc:1434] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Time spent starting tablet: real 0.093s	user 0.025s	sys 0.024s
I20260709 12:15:45.367823  5681 tablet_bootstrap.cc:492] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Bootstrap starting.
I20260709 12:15:45.371238  5680 tablet_bootstrap.cc:492] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c: No bootstrap required, opened a new log
I20260709 12:15:45.371840  5680 ts_tablet_manager.cc:1403] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c: Time spent bootstrapping tablet: real 0.110s	user 0.011s	sys 0.007s
I20260709 12:15:45.365109  5679 consensus_queue.cc:260] T f0f1dad5e1c540edbbba5708a7698085 P 92adb80abb834eceaef035f370597703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.374073  5679 ts_tablet_manager.cc:1434] T f0f1dad5e1c540edbbba5708a7698085 P 92adb80abb834eceaef035f370597703: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.003s
I20260709 12:15:45.375125  5679 tablet_bootstrap.cc:492] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703: Bootstrap starting.
I20260709 12:15:45.368198  5684 raft_consensus.cc:697] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [term 1 LEADER]: Becoming Leader. State: Replica: 92adb80abb834eceaef035f370597703, State: Running, Role: LEADER
I20260709 12:15:45.375895  5680 raft_consensus.cc:359] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.376854  5680 raft_consensus.cc:385] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:45.377269  5680 raft_consensus.cc:740] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7f53a4ef21a4d8788fae6df6bb5f98c, State: Initialized, Role: FOLLOWER
I20260709 12:15:45.377012  5684 consensus_queue.cc:237] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.378203  5680 consensus_queue.cc:260] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.381330  5680 ts_tablet_manager.cc:1434] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:15:45.384078  5709 raft_consensus.cc:515] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.387540  5705 raft_consensus.cc:493] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:45.389609  5705 raft_consensus.cc:515] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.393282  5680 tablet_bootstrap.cc:492] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c: Bootstrap starting.
I20260709 12:15:45.396041  5705 leader_election.cc:290] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2:36305), 92adb80abb834eceaef035f370597703 (127.5.58.3:41427)
I20260709 12:15:45.397264  5635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0f1dad5e1c540edbbba5708a7698085" candidate_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92adb80abb834eceaef035f370597703" is_pre_election: true
I20260709 12:15:45.397593  5562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0f1dad5e1c540edbbba5708a7698085" candidate_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" is_pre_election: true
I20260709 12:15:45.399957  5635 raft_consensus.cc:2468] T f0f1dad5e1c540edbbba5708a7698085 P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7f53a4ef21a4d8788fae6df6bb5f98c in term 0.
W20260709 12:15:45.401120  5452 leader_election.cc:343] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2:36305): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:45.404520  5709 leader_election.cc:290] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 1 election: Requested vote from peers d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021), 92adb80abb834eceaef035f370597703 (127.5.58.3:41427)
I20260709 12:15:45.404991  5450 leader_election.cc:304] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92adb80abb834eceaef035f370597703, d7f53a4ef21a4d8788fae6df6bb5f98c; no voters: 3ea2ad5b842b4bf3868b4e25cae3e9f1
I20260709 12:15:45.405550  5681 tablet_bootstrap.cc:654] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:45.405880  5705 raft_consensus.cc:2804] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:45.406261  5705 raft_consensus.cc:493] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:45.407361  5705 raft_consensus.cc:3060] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:45.407287  5487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66af11cf9374a31b237a393194eccae" candidate_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
I20260709 12:15:45.409261  5635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66af11cf9374a31b237a393194eccae" candidate_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92adb80abb834eceaef035f370597703"
W20260709 12:15:45.410102  5527 leader_election.cc:343] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:45.410228  5635 raft_consensus.cc:3060] T f66af11cf9374a31b237a393194eccae P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:45.409116  5381 catalog_manager.cc:5697] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92adb80abb834eceaef035f370597703 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "92adb80abb834eceaef035f370597703" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:45.415313  5679 tablet_bootstrap.cc:654] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:45.412119  5382 catalog_manager.cc:5697] T 62d225e09ba14ac4bd3bcc68a7b3495f P 3ea2ad5b842b4bf3868b4e25cae3e9f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2). New cstate: current_term: 1 leader_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:45.428794  5635 raft_consensus.cc:2468] T f66af11cf9374a31b237a393194eccae P 92adb80abb834eceaef035f370597703 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea2ad5b842b4bf3868b4e25cae3e9f1 in term 1.
I20260709 12:15:45.431945  5680 tablet_bootstrap.cc:654] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:45.432257  5525 leader_election.cc:304] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ea2ad5b842b4bf3868b4e25cae3e9f1, 92adb80abb834eceaef035f370597703; no voters: d7f53a4ef21a4d8788fae6df6bb5f98c
I20260709 12:15:45.434909  5709 raft_consensus.cc:2804] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:45.435658  5709 raft_consensus.cc:697] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 LEADER]: Becoming Leader. State: Replica: 3ea2ad5b842b4bf3868b4e25cae3e9f1, State: Running, Role: LEADER
I20260709 12:15:45.437052  5709 consensus_queue.cc:237] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.426745  5705 raft_consensus.cc:515] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.448053  5562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0f1dad5e1c540edbbba5708a7698085" candidate_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
W20260709 12:15:45.450008  5452 leader_election.cc:343] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2:36305): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:45.443910  5679 tablet_bootstrap.cc:492] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703: No bootstrap required, opened a new log
I20260709 12:15:45.452099  5679 ts_tablet_manager.cc:1403] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703: Time spent bootstrapping tablet: real 0.077s	user 0.017s	sys 0.001s
I20260709 12:15:45.461599  5705 leader_election.cc:290] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 1 election: Requested vote from peers 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2:36305), 92adb80abb834eceaef035f370597703 (127.5.58.3:41427)
I20260709 12:15:45.462200  5681 tablet_bootstrap.cc:492] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1: No bootstrap required, opened a new log
I20260709 12:15:45.462677  5681 ts_tablet_manager.cc:1403] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Time spent bootstrapping tablet: real 0.095s	user 0.009s	sys 0.010s
I20260709 12:15:45.463826  5636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0f1dad5e1c540edbbba5708a7698085" candidate_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92adb80abb834eceaef035f370597703"
I20260709 12:15:45.464521  5636 raft_consensus.cc:3060] T f0f1dad5e1c540edbbba5708a7698085 P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:45.467145  5681 raft_consensus.cc:359] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } }
I20260709 12:15:45.470217  5680 tablet_bootstrap.cc:492] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c: No bootstrap required, opened a new log
I20260709 12:15:45.470747  5680 ts_tablet_manager.cc:1403] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c: Time spent bootstrapping tablet: real 0.080s	user 0.014s	sys 0.006s
I20260709 12:15:45.467938  5681 raft_consensus.cc:385] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:45.471789  5681 raft_consensus.cc:740] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ea2ad5b842b4bf3868b4e25cae3e9f1, State: Initialized, Role: FOLLOWER
I20260709 12:15:45.472878  5681 consensus_queue.cc:260] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } }
I20260709 12:15:45.475275  5679 raft_consensus.cc:359] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.476468  5679 raft_consensus.cc:385] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:45.476855  5679 raft_consensus.cc:740] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92adb80abb834eceaef035f370597703, State: Initialized, Role: FOLLOWER
I20260709 12:15:45.477660  5679 consensus_queue.cc:260] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.480430  5679 ts_tablet_manager.cc:1434] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703: Time spent starting tablet: real 0.028s	user 0.005s	sys 0.002s
I20260709 12:15:45.481508  5679 tablet_bootstrap.cc:492] T a89ab5b73a4a42f7b8b529758c61edf7 P 92adb80abb834eceaef035f370597703: Bootstrap starting.
I20260709 12:15:45.481962  5709 raft_consensus.cc:493] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:45.482504  5709 raft_consensus.cc:515] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.484663  5709 leader_election.cc:290] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021), 92adb80abb834eceaef035f370597703 (127.5.58.3:41427)
I20260709 12:15:45.485639  5681 ts_tablet_manager.cc:1434] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.000s
I20260709 12:15:45.485390  5487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09cfb685589d470ab66f97d1d81df47c" candidate_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" is_pre_election: true
I20260709 12:15:45.487090  5681 tablet_bootstrap.cc:492] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Bootstrap starting.
W20260709 12:15:45.487999  5527 leader_election.cc:343] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:45.491107  5679 tablet_bootstrap.cc:654] T a89ab5b73a4a42f7b8b529758c61edf7 P 92adb80abb834eceaef035f370597703: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:45.490839  5680 raft_consensus.cc:359] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.491601  5680 raft_consensus.cc:385] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:45.491925  5680 raft_consensus.cc:740] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7f53a4ef21a4d8788fae6df6bb5f98c, State: Initialized, Role: FOLLOWER
I20260709 12:15:45.492780  5680 consensus_queue.cc:260] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.495174  5680 ts_tablet_manager.cc:1434] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 12:15:45.496358  5680 tablet_bootstrap.cc:492] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c: Bootstrap starting.
I20260709 12:15:45.496528  5684 raft_consensus.cc:493] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:45.497014  5684 raft_consensus.cc:515] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.499094  5684 leader_election.cc:290] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021), 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2:36305)
I20260709 12:15:45.501473  5562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930c10c719284908b31b1b583ac8bc92" candidate_uuid: "92adb80abb834eceaef035f370597703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" is_pre_election: true
W20260709 12:15:45.504132  5601 leader_election.cc:343] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2:36305): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:45.505901  5487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930c10c719284908b31b1b583ac8bc92" candidate_uuid: "92adb80abb834eceaef035f370597703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" is_pre_election: true
I20260709 12:15:45.506942  5487 raft_consensus.cc:2468] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92adb80abb834eceaef035f370597703 in term 0.
I20260709 12:15:45.509505  5601 leader_election.cc:304] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92adb80abb834eceaef035f370597703, d7f53a4ef21a4d8788fae6df6bb5f98c; no voters: 3ea2ad5b842b4bf3868b4e25cae3e9f1
I20260709 12:15:45.510535  5681 tablet_bootstrap.cc:654] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:45.510718  5684 raft_consensus.cc:2804] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:45.511193  5684 raft_consensus.cc:493] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:45.511356  5636 raft_consensus.cc:2468] T f0f1dad5e1c540edbbba5708a7698085 P 92adb80abb834eceaef035f370597703 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f53a4ef21a4d8788fae6df6bb5f98c in term 1.
I20260709 12:15:45.511636  5684 raft_consensus.cc:3060] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:45.505975  5635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09cfb685589d470ab66f97d1d81df47c" candidate_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92adb80abb834eceaef035f370597703" is_pre_election: true
I20260709 12:15:45.514664  5680 tablet_bootstrap.cc:654] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c: Neither blocks nor log segments found. Creating new log.
W20260709 12:15:45.514305  5525 leader_election.cc:343] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 92adb80abb834eceaef035f370597703 (127.5.58.3:41427): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:45.515915  5679 tablet_bootstrap.cc:492] T a89ab5b73a4a42f7b8b529758c61edf7 P 92adb80abb834eceaef035f370597703: No bootstrap required, opened a new log
I20260709 12:15:45.516016  5525 leader_election.cc:304] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ea2ad5b842b4bf3868b4e25cae3e9f1; no voters: 92adb80abb834eceaef035f370597703, d7f53a4ef21a4d8788fae6df6bb5f98c
I20260709 12:15:45.516710  5679 ts_tablet_manager.cc:1403] T a89ab5b73a4a42f7b8b529758c61edf7 P 92adb80abb834eceaef035f370597703: Time spent bootstrapping tablet: real 0.035s	user 0.010s	sys 0.008s
I20260709 12:15:45.517040  5709 raft_consensus.cc:2749] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:45.517294  5450 leader_election.cc:304] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92adb80abb834eceaef035f370597703, d7f53a4ef21a4d8788fae6df6bb5f98c; no voters: 3ea2ad5b842b4bf3868b4e25cae3e9f1
I20260709 12:15:45.519088  5705 raft_consensus.cc:2804] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:45.519688  5705 raft_consensus.cc:697] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 LEADER]: Becoming Leader. State: Replica: d7f53a4ef21a4d8788fae6df6bb5f98c, State: Running, Role: LEADER
I20260709 12:15:45.520805  5705 consensus_queue.cc:237] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.525868  5679 raft_consensus.cc:359] T a89ab5b73a4a42f7b8b529758c61edf7 P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } }
I20260709 12:15:45.526974  5679 raft_consensus.cc:385] T a89ab5b73a4a42f7b8b529758c61edf7 P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:45.527274  5679 raft_consensus.cc:740] T a89ab5b73a4a42f7b8b529758c61edf7 P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92adb80abb834eceaef035f370597703, State: Initialized, Role: FOLLOWER
I20260709 12:15:45.524725  5384 catalog_manager.cc:5697] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2). New cstate: current_term: 1 leader_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:45.530737  5681 tablet_bootstrap.cc:492] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1: No bootstrap required, opened a new log
I20260709 12:15:45.530448  5679 consensus_queue.cc:260] T a89ab5b73a4a42f7b8b529758c61edf7 P 92adb80abb834eceaef035f370597703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } }
I20260709 12:15:45.532039  5681 ts_tablet_manager.cc:1403] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Time spent bootstrapping tablet: real 0.045s	user 0.015s	sys 0.001s
I20260709 12:15:45.535516  5679 ts_tablet_manager.cc:1434] T a89ab5b73a4a42f7b8b529758c61edf7 P 92adb80abb834eceaef035f370597703: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.004s
I20260709 12:15:45.537104  5679 tablet_bootstrap.cc:492] T 09cfb685589d470ab66f97d1d81df47c P 92adb80abb834eceaef035f370597703: Bootstrap starting.
I20260709 12:15:45.536597  5681 raft_consensus.cc:359] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.538730  5681 raft_consensus.cc:385] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:45.539162  5681 raft_consensus.cc:740] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ea2ad5b842b4bf3868b4e25cae3e9f1, State: Initialized, Role: FOLLOWER
I20260709 12:15:45.540141  5681 consensus_queue.cc:260] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.543057  5680 tablet_bootstrap.cc:492] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c: No bootstrap required, opened a new log
I20260709 12:15:45.543660  5680 ts_tablet_manager.cc:1403] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c: Time spent bootstrapping tablet: real 0.048s	user 0.010s	sys 0.011s
I20260709 12:15:45.549160  5680 raft_consensus.cc:359] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.550136  5680 raft_consensus.cc:385] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:45.550549  5680 raft_consensus.cc:740] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7f53a4ef21a4d8788fae6df6bb5f98c, State: Initialized, Role: FOLLOWER
I20260709 12:15:45.545722  5681 ts_tablet_manager.cc:1434] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.002s
I20260709 12:15:45.551676  5679 tablet_bootstrap.cc:654] T 09cfb685589d470ab66f97d1d81df47c P 92adb80abb834eceaef035f370597703: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:45.551366  5684 raft_consensus.cc:515] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.551952  5681 tablet_bootstrap.cc:492] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Bootstrap starting.
I20260709 12:15:45.559533  5684 leader_election.cc:290] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [CANDIDATE]: Term 1 election: Requested vote from peers d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021), 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2:36305)
I20260709 12:15:45.552727  5680 consensus_queue.cc:260] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.560743  5487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930c10c719284908b31b1b583ac8bc92" candidate_uuid: "92adb80abb834eceaef035f370597703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
I20260709 12:15:45.561316  5487 raft_consensus.cc:3060] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:45.562999  5562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930c10c719284908b31b1b583ac8bc92" candidate_uuid: "92adb80abb834eceaef035f370597703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
I20260709 12:15:45.563665  5562 raft_consensus.cc:3060] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:45.563409  5383 catalog_manager.cc:5697] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c reported cstate change: term changed from 0 to 1, leader changed from <none> to d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1). New cstate: current_term: 1 leader_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:45.565165  5681 tablet_bootstrap.cc:654] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:45.568229  5487 raft_consensus.cc:2468] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92adb80abb834eceaef035f370597703 in term 1.
I20260709 12:15:45.569774  5601 leader_election.cc:304] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92adb80abb834eceaef035f370597703, d7f53a4ef21a4d8788fae6df6bb5f98c; no voters: 
I20260709 12:15:45.570029  5680 ts_tablet_manager.cc:1434] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 12:15:45.570698  5684 raft_consensus.cc:2804] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:45.571179  5680 tablet_bootstrap.cc:492] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c: Bootstrap starting.
I20260709 12:15:45.572033  5684 raft_consensus.cc:697] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 1 LEADER]: Becoming Leader. State: Replica: 92adb80abb834eceaef035f370597703, State: Running, Role: LEADER
I20260709 12:15:45.573189  5684 consensus_queue.cc:237] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.576910  5705 raft_consensus.cc:493] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:45.577032  5562 raft_consensus.cc:2468] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92adb80abb834eceaef035f370597703 in term 1.
I20260709 12:15:45.577513  5705 raft_consensus.cc:515] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.579867  5705 leader_election.cc:290] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2:36305), 92adb80abb834eceaef035f370597703 (127.5.58.3:41427)
I20260709 12:15:45.581609  5561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09cfb685589d470ab66f97d1d81df47c" candidate_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" is_pre_election: true
I20260709 12:15:45.582393  5680 tablet_bootstrap.cc:654] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:45.582275  5561 raft_consensus.cc:2468] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7f53a4ef21a4d8788fae6df6bb5f98c in term 0.
I20260709 12:15:45.583846  5452 leader_election.cc:304] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea2ad5b842b4bf3868b4e25cae3e9f1, d7f53a4ef21a4d8788fae6df6bb5f98c; no voters: 
I20260709 12:15:45.584743  5705 raft_consensus.cc:2804] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:45.584775  5635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09cfb685589d470ab66f97d1d81df47c" candidate_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92adb80abb834eceaef035f370597703" is_pre_election: true
I20260709 12:15:45.585302  5705 raft_consensus.cc:493] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:45.585719  5705 raft_consensus.cc:3060] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:15:45.586269  5450 leader_election.cc:343] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 92adb80abb834eceaef035f370597703 (127.5.58.3:41427): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:45.594990  5679 tablet_bootstrap.cc:492] T 09cfb685589d470ab66f97d1d81df47c P 92adb80abb834eceaef035f370597703: No bootstrap required, opened a new log
W20260709 12:15:45.595006  5432 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:15:45.595613  5679 ts_tablet_manager.cc:1403] T 09cfb685589d470ab66f97d1d81df47c P 92adb80abb834eceaef035f370597703: Time spent bootstrapping tablet: real 0.059s	user 0.007s	sys 0.004s
I20260709 12:15:45.596268  5681 tablet_bootstrap.cc:492] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1: No bootstrap required, opened a new log
I20260709 12:15:45.595893  5383 catalog_manager.cc:5697] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92adb80abb834eceaef035f370597703 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "92adb80abb834eceaef035f370597703" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:45.596817  5681 ts_tablet_manager.cc:1403] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Time spent bootstrapping tablet: real 0.045s	user 0.007s	sys 0.007s
I20260709 12:15:45.598500  5679 raft_consensus.cc:359] T 09cfb685589d470ab66f97d1d81df47c P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.599175  5679 raft_consensus.cc:385] T 09cfb685589d470ab66f97d1d81df47c P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:45.598827  5705 raft_consensus.cc:515] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.599468  5679 raft_consensus.cc:740] T 09cfb685589d470ab66f97d1d81df47c P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92adb80abb834eceaef035f370597703, State: Initialized, Role: FOLLOWER
I20260709 12:15:45.600015  5681 raft_consensus.cc:359] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.600220  5679 consensus_queue.cc:260] T 09cfb685589d470ab66f97d1d81df47c P 92adb80abb834eceaef035f370597703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.600860  5681 raft_consensus.cc:385] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:45.601230  5681 raft_consensus.cc:740] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ea2ad5b842b4bf3868b4e25cae3e9f1, State: Initialized, Role: FOLLOWER
I20260709 12:15:45.602903  5562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09cfb685589d470ab66f97d1d81df47c" candidate_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
I20260709 12:15:45.603299  5635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09cfb685589d470ab66f97d1d81df47c" candidate_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92adb80abb834eceaef035f370597703"
I20260709 12:15:45.603521  5562 raft_consensus.cc:3060] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:45.603909  5635 raft_consensus.cc:3060] T 09cfb685589d470ab66f97d1d81df47c P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:45.605077  5679 ts_tablet_manager.cc:1434] T 09cfb685589d470ab66f97d1d81df47c P 92adb80abb834eceaef035f370597703: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.004s
I20260709 12:15:45.602557  5681 consensus_queue.cc:260] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.609189  5681 ts_tablet_manager.cc:1434] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:15:45.610306  5681 tablet_bootstrap.cc:492] T 4fd3cf9b75144e5c88eee1a8917e283e P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Bootstrap starting.
I20260709 12:15:45.613291  5705 leader_election.cc:290] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 1 election: Requested vote from peers 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2:36305), 92adb80abb834eceaef035f370597703 (127.5.58.3:41427)
I20260709 12:15:45.611027  5562 raft_consensus.cc:2468] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f53a4ef21a4d8788fae6df6bb5f98c in term 1.
I20260709 12:15:45.611461  5635 raft_consensus.cc:2468] T 09cfb685589d470ab66f97d1d81df47c P 92adb80abb834eceaef035f370597703 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f53a4ef21a4d8788fae6df6bb5f98c in term 1.
I20260709 12:15:45.617529  5452 leader_election.cc:304] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea2ad5b842b4bf3868b4e25cae3e9f1, d7f53a4ef21a4d8788fae6df6bb5f98c; no voters: 
I20260709 12:15:45.618443  5705 raft_consensus.cc:2804] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:45.619051  5705 raft_consensus.cc:697] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 LEADER]: Becoming Leader. State: Replica: d7f53a4ef21a4d8788fae6df6bb5f98c, State: Running, Role: LEADER
I20260709 12:15:45.619935  5705 consensus_queue.cc:237] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.625339  5680 tablet_bootstrap.cc:492] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c: No bootstrap required, opened a new log
I20260709 12:15:45.625982  5680 ts_tablet_manager.cc:1403] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c: Time spent bootstrapping tablet: real 0.055s	user 0.016s	sys 0.000s
I20260709 12:15:45.628937  5680 raft_consensus.cc:359] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.629750  5680 raft_consensus.cc:385] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:45.630120  5680 raft_consensus.cc:740] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7f53a4ef21a4d8788fae6df6bb5f98c, State: Initialized, Role: FOLLOWER
I20260709 12:15:45.630897  5680 consensus_queue.cc:260] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.625373  5681 tablet_bootstrap.cc:654] T 4fd3cf9b75144e5c88eee1a8917e283e P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:45.633812  5680 ts_tablet_manager.cc:1434] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:15:45.634908  5680 tablet_bootstrap.cc:492] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c: Bootstrap starting.
I20260709 12:15:45.641573  5680 tablet_bootstrap.cc:654] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:45.645054  5384 catalog_manager.cc:5697] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c reported cstate change: term changed from 0 to 1, leader changed from <none> to d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1). New cstate: current_term: 1 leader_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:45.646278  5681 tablet_bootstrap.cc:492] T 4fd3cf9b75144e5c88eee1a8917e283e P 3ea2ad5b842b4bf3868b4e25cae3e9f1: No bootstrap required, opened a new log
I20260709 12:15:45.646840  5681 ts_tablet_manager.cc:1403] T 4fd3cf9b75144e5c88eee1a8917e283e P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Time spent bootstrapping tablet: real 0.037s	user 0.017s	sys 0.001s
I20260709 12:15:45.649483  5681 raft_consensus.cc:359] T 4fd3cf9b75144e5c88eee1a8917e283e P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.650202  5681 raft_consensus.cc:385] T 4fd3cf9b75144e5c88eee1a8917e283e P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:45.650513  5681 raft_consensus.cc:740] T 4fd3cf9b75144e5c88eee1a8917e283e P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ea2ad5b842b4bf3868b4e25cae3e9f1, State: Initialized, Role: FOLLOWER
I20260709 12:15:45.651173  5681 consensus_queue.cc:260] T 4fd3cf9b75144e5c88eee1a8917e283e P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.653494  5681 ts_tablet_manager.cc:1434] T 4fd3cf9b75144e5c88eee1a8917e283e P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:15:45.654393  5681 tablet_bootstrap.cc:492] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Bootstrap starting.
I20260709 12:15:45.657724  5680 tablet_bootstrap.cc:492] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c: No bootstrap required, opened a new log
I20260709 12:15:45.658262  5680 ts_tablet_manager.cc:1403] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.001s
I20260709 12:15:45.659992  5681 tablet_bootstrap.cc:654] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:45.661172  5680 raft_consensus.cc:359] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.661962  5680 raft_consensus.cc:385] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:45.662297  5680 raft_consensus.cc:740] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7f53a4ef21a4d8788fae6df6bb5f98c, State: Initialized, Role: FOLLOWER
I20260709 12:15:45.663149  5680 consensus_queue.cc:260] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.665808  5680 ts_tablet_manager.cc:1434] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:15:45.666985  5680 tablet_bootstrap.cc:492] T f66af11cf9374a31b237a393194eccae P d7f53a4ef21a4d8788fae6df6bb5f98c: Bootstrap starting.
I20260709 12:15:45.671321  5681 tablet_bootstrap.cc:492] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1: No bootstrap required, opened a new log
I20260709 12:15:45.672439  5681 ts_tablet_manager.cc:1403] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.006s
I20260709 12:15:45.673128  5680 tablet_bootstrap.cc:654] T f66af11cf9374a31b237a393194eccae P d7f53a4ef21a4d8788fae6df6bb5f98c: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:45.675539  5681 raft_consensus.cc:359] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.678002  5681 raft_consensus.cc:385] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:45.678395  5681 raft_consensus.cc:740] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ea2ad5b842b4bf3868b4e25cae3e9f1, State: Initialized, Role: FOLLOWER
I20260709 12:15:45.679231  5681 consensus_queue.cc:260] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.681828  5681 ts_tablet_manager.cc:1434] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.002s
I20260709 12:15:45.685554  5680 tablet_bootstrap.cc:492] T f66af11cf9374a31b237a393194eccae P d7f53a4ef21a4d8788fae6df6bb5f98c: No bootstrap required, opened a new log
I20260709 12:15:45.686162  5680 ts_tablet_manager.cc:1403] T f66af11cf9374a31b237a393194eccae P d7f53a4ef21a4d8788fae6df6bb5f98c: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.004s
I20260709 12:15:45.689132  5680 raft_consensus.cc:359] T f66af11cf9374a31b237a393194eccae P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.689882  5680 raft_consensus.cc:385] T f66af11cf9374a31b237a393194eccae P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:45.690184  5680 raft_consensus.cc:740] T f66af11cf9374a31b237a393194eccae P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7f53a4ef21a4d8788fae6df6bb5f98c, State: Initialized, Role: FOLLOWER
I20260709 12:15:45.690886  5680 consensus_queue.cc:260] T f66af11cf9374a31b237a393194eccae P d7f53a4ef21a4d8788fae6df6bb5f98c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.693513  5680 ts_tablet_manager.cc:1434] T f66af11cf9374a31b237a393194eccae P d7f53a4ef21a4d8788fae6df6bb5f98c: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:15:45.747932  5562 raft_consensus.cc:3060] T 4fd3cf9b75144e5c88eee1a8917e283e P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:45.757814  5709 raft_consensus.cc:493] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:45.757993  5705 consensus_queue.cc:1048] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:45.758543  5709 raft_consensus.cc:515] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } }
I20260709 12:15:45.762156  5487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a89ab5b73a4a42f7b8b529758c61edf7" candidate_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" is_pre_election: true
I20260709 12:15:45.762406  5635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a89ab5b73a4a42f7b8b529758c61edf7" candidate_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92adb80abb834eceaef035f370597703" is_pre_election: true
I20260709 12:15:45.762883  5487 raft_consensus.cc:2468] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ea2ad5b842b4bf3868b4e25cae3e9f1 in term 0.
I20260709 12:15:45.763827  5709 leader_election.cc:290] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021), 92adb80abb834eceaef035f370597703 (127.5.58.3:41427)
I20260709 12:15:45.763468  5635 raft_consensus.cc:2468] T a89ab5b73a4a42f7b8b529758c61edf7 P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ea2ad5b842b4bf3868b4e25cae3e9f1 in term 0.
I20260709 12:15:45.765580  5525 leader_election.cc:304] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea2ad5b842b4bf3868b4e25cae3e9f1, 92adb80abb834eceaef035f370597703; no voters: 
I20260709 12:15:45.766541  5688 raft_consensus.cc:2804] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:45.766927  5688 raft_consensus.cc:493] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:45.767314  5688 raft_consensus.cc:3060] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:45.780400  5688 raft_consensus.cc:515] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } }
I20260709 12:15:45.784040  5487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a89ab5b73a4a42f7b8b529758c61edf7" candidate_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
I20260709 12:15:45.784708  5487 raft_consensus.cc:3060] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:45.796243  5688 leader_election.cc:290] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 1 election: Requested vote from peers d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021), 92adb80abb834eceaef035f370597703 (127.5.58.3:41427)
I20260709 12:15:45.798125  5635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a89ab5b73a4a42f7b8b529758c61edf7" candidate_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92adb80abb834eceaef035f370597703"
I20260709 12:15:45.798705  5487 raft_consensus.cc:2468] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea2ad5b842b4bf3868b4e25cae3e9f1 in term 1.
I20260709 12:15:45.799324  5635 raft_consensus.cc:3060] T a89ab5b73a4a42f7b8b529758c61edf7 P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:45.800256  5527 leader_election.cc:304] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea2ad5b842b4bf3868b4e25cae3e9f1, d7f53a4ef21a4d8788fae6df6bb5f98c; no voters: 
I20260709 12:15:45.801612  5688 raft_consensus.cc:2804] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:45.802130  5688 raft_consensus.cc:697] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 LEADER]: Becoming Leader. State: Replica: 3ea2ad5b842b4bf3868b4e25cae3e9f1, State: Running, Role: LEADER
I20260709 12:15:45.803143  5740 consensus_queue.cc:1048] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c [LEADER]: Connected to new peer: Peer: permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:45.802976  5688 consensus_queue.cc:237] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } }
I20260709 12:15:45.816695  5709 consensus_queue.cc:1048] T 62d225e09ba14ac4bd3bcc68a7b3495f P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:45.827394  5635 raft_consensus.cc:2468] T a89ab5b73a4a42f7b8b529758c61edf7 P 92adb80abb834eceaef035f370597703 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea2ad5b842b4bf3868b4e25cae3e9f1 in term 1.
I20260709 12:15:45.848866  5636 raft_consensus.cc:3060] T 62d225e09ba14ac4bd3bcc68a7b3495f P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:45.851135  5383 catalog_manager.cc:5697] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2). New cstate: current_term: 1 leader_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:45.861306  5688 consensus_queue.cc:1048] T 62d225e09ba14ac4bd3bcc68a7b3495f P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:45.923378  5684 raft_consensus.cc:493] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:45.923911  5684 raft_consensus.cc:515] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.925753  5684 leader_election.cc:290] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021), 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2:36305)
I20260709 12:15:45.926928  5486 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1126f26405b742a6a136656ca8c86696" candidate_uuid: "92adb80abb834eceaef035f370597703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" is_pre_election: true
I20260709 12:15:45.927083  5562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1126f26405b742a6a136656ca8c86696" candidate_uuid: "92adb80abb834eceaef035f370597703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" is_pre_election: true
I20260709 12:15:45.927712  5486 raft_consensus.cc:2468] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92adb80abb834eceaef035f370597703 in term 0.
I20260709 12:15:45.927742  5562 raft_consensus.cc:2468] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92adb80abb834eceaef035f370597703 in term 0.
I20260709 12:15:45.929138  5601 leader_election.cc:304] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92adb80abb834eceaef035f370597703, d7f53a4ef21a4d8788fae6df6bb5f98c; no voters: 
I20260709 12:15:45.929867  5684 raft_consensus.cc:2804] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:45.930261  5684 raft_consensus.cc:493] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:45.930600  5684 raft_consensus.cc:3060] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:45.935499  5684 raft_consensus.cc:515] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.937484  5684 leader_election.cc:290] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [CANDIDATE]: Term 1 election: Requested vote from peers d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021), 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2:36305)
I20260709 12:15:45.938333  5486 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1126f26405b742a6a136656ca8c86696" candidate_uuid: "92adb80abb834eceaef035f370597703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
I20260709 12:15:45.938585  5562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1126f26405b742a6a136656ca8c86696" candidate_uuid: "92adb80abb834eceaef035f370597703" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
I20260709 12:15:45.939083  5486 raft_consensus.cc:3060] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:45.939814  5562 raft_consensus.cc:3060] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:45.946321  5562 raft_consensus.cc:2468] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92adb80abb834eceaef035f370597703 in term 1.
I20260709 12:15:45.946322  5486 raft_consensus.cc:2468] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92adb80abb834eceaef035f370597703 in term 1.
I20260709 12:15:45.947760  5601 leader_election.cc:304] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92adb80abb834eceaef035f370597703, d7f53a4ef21a4d8788fae6df6bb5f98c; no voters: 
I20260709 12:15:45.948546  5684 raft_consensus.cc:2804] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:45.949016  5684 raft_consensus.cc:697] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 1 LEADER]: Becoming Leader. State: Replica: 92adb80abb834eceaef035f370597703, State: Running, Role: LEADER
I20260709 12:15:45.949860  5684 consensus_queue.cc:237] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:45.958288  5384 catalog_manager.cc:5697] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92adb80abb834eceaef035f370597703 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "92adb80abb834eceaef035f370597703" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:45.969553  5684 consensus_queue.cc:1048] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:45.974480  5709 consensus_queue.cc:1048] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:45.975057  5486 raft_consensus.cc:3060] T f66af11cf9374a31b237a393194eccae P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:45.994165  5714 consensus_queue.cc:1048] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:45.995926  5708 consensus_queue.cc:1048] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:45.999902  5687 raft_consensus.cc:493] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:46.000631  5687 raft_consensus.cc:515] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:46.004012  5687 leader_election.cc:290] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2:36305), 92adb80abb834eceaef035f370597703 (127.5.58.3:41427)
I20260709 12:15:46.004480  5562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a00c228a31d744968a6b32e3601d51eb" candidate_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" is_pre_election: true
I20260709 12:15:46.004906  5684 consensus_queue.cc:1048] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:46.005559  5562 raft_consensus.cc:2393] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d7f53a4ef21a4d8788fae6df6bb5f98c in current term 1: Already voted for candidate 92adb80abb834eceaef035f370597703 in this term.
I20260709 12:15:46.004617  5635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a00c228a31d744968a6b32e3601d51eb" candidate_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92adb80abb834eceaef035f370597703" is_pre_election: true
I20260709 12:15:46.015206  5450 leader_election.cc:304] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7f53a4ef21a4d8788fae6df6bb5f98c; no voters: 3ea2ad5b842b4bf3868b4e25cae3e9f1, 92adb80abb834eceaef035f370597703
I20260709 12:15:46.016155  5687 raft_consensus.cc:3060] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:46.024628  5687 raft_consensus.cc:2749] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:46.027168  5562 raft_consensus.cc:3060] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:46.034667  5752 consensus_queue.cc:1048] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:46.040927  5687 consensus_queue.cc:1048] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:46.089884  5740 consensus_queue.cc:1048] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [LEADER]: Connected to new peer: Peer: permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:46.153769  5740 consensus_queue.cc:1048] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [LEADER]: Connected to new peer: Peer: permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:46.168732  5740 consensus_queue.cc:1048] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:46.308368  5688 consensus_queue.cc:1048] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:46.322355  5562 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a89ab5b73a4a42f7b8b529758c61edf7"
dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
mode: GRACEFUL
new_leader_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
 from {username='slave'} at 127.0.0.1:34048
I20260709 12:15:46.323194  5562 raft_consensus.cc:606] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 LEADER]: Received request to transfer leadership to TS d7f53a4ef21a4d8788fae6df6bb5f98c
I20260709 12:15:46.325167  5688 consensus_queue.cc:1048] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:46.337505  5708 raft_consensus.cc:993] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1: : Instructing follower d7f53a4ef21a4d8788fae6df6bb5f98c to start an election
I20260709 12:15:46.337988  5708 raft_consensus.cc:1081] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 LEADER]: Signalling peer d7f53a4ef21a4d8788fae6df6bb5f98c to start an election
I20260709 12:15:46.342181  5486 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a89ab5b73a4a42f7b8b529758c61edf7"
dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
 from {username='slave'} at 127.0.0.1:55134
I20260709 12:15:46.342877  5486 raft_consensus.cc:493] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:46.343273  5486 raft_consensus.cc:3060] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:46.352605  5486 raft_consensus.cc:515] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } }
I20260709 12:15:46.355615  5636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a89ab5b73a4a42f7b8b529758c61edf7" candidate_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "92adb80abb834eceaef035f370597703"
I20260709 12:15:46.356520  5636 raft_consensus.cc:3060] T a89ab5b73a4a42f7b8b529758c61edf7 P 92adb80abb834eceaef035f370597703 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:46.356721  5562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a89ab5b73a4a42f7b8b529758c61edf7" candidate_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
I20260709 12:15:46.358796  5562 raft_consensus.cc:3055] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:46.359210  5562 raft_consensus.cc:740] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ea2ad5b842b4bf3868b4e25cae3e9f1, State: Running, Role: LEADER
I20260709 12:15:46.357471  5486 leader_election.cc:290] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 2 election: Requested vote from peers 92adb80abb834eceaef035f370597703 (127.5.58.3:41427), 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2:36305)
I20260709 12:15:46.361828  5635 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "930c10c719284908b31b1b583ac8bc92"
dest_uuid: "92adb80abb834eceaef035f370597703"
mode: GRACEFUL
new_leader_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
 from {username='slave'} at 127.0.0.1:44090
I20260709 12:15:46.361919  5562 consensus_queue.cc:260] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } }
I20260709 12:15:46.362473  5635 raft_consensus.cc:606] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 1 LEADER]: Received request to transfer leadership to TS d7f53a4ef21a4d8788fae6df6bb5f98c
I20260709 12:15:46.363374  5562 raft_consensus.cc:3060] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:46.363765  5636 raft_consensus.cc:2468] T a89ab5b73a4a42f7b8b529758c61edf7 P 92adb80abb834eceaef035f370597703 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f53a4ef21a4d8788fae6df6bb5f98c in term 2.
I20260709 12:15:46.365243  5450 leader_election.cc:304] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92adb80abb834eceaef035f370597703, d7f53a4ef21a4d8788fae6df6bb5f98c; no voters: 
I20260709 12:15:46.366110  5705 raft_consensus.cc:2804] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:46.366559  5705 raft_consensus.cc:697] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 LEADER]: Becoming Leader. State: Replica: d7f53a4ef21a4d8788fae6df6bb5f98c, State: Running, Role: LEADER
I20260709 12:15:46.367344  5705 consensus_queue.cc:237] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } }
I20260709 12:15:46.370249  5562 raft_consensus.cc:2468] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f53a4ef21a4d8788fae6df6bb5f98c in term 2.
I20260709 12:15:46.377763  5752 consensus_queue.cc:1048] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:46.380896  5383 catalog_manager.cc:5697] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c reported cstate change: term changed from 1 to 2, leader changed from 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2) to d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1). New cstate: current_term: 2 leader_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:46.389272  5487 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f0f1dad5e1c540edbbba5708a7698085"
dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
mode: GRACEFUL
new_leader_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
 from {username='slave'} at 127.0.0.1:55150
I20260709 12:15:46.389994  5487 raft_consensus.cc:606] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 LEADER]: Received request to transfer leadership to TS 3ea2ad5b842b4bf3868b4e25cae3e9f1
I20260709 12:15:46.397145  5636 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1126f26405b742a6a136656ca8c86696"
dest_uuid: "92adb80abb834eceaef035f370597703"
mode: GRACEFUL
new_leader_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
 from {username='slave'} at 127.0.0.1:44090
I20260709 12:15:46.401464  5636 raft_consensus.cc:606] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 1 LEADER]: Received request to transfer leadership to TS 3ea2ad5b842b4bf3868b4e25cae3e9f1
I20260709 12:15:46.403789  5752 consensus_queue.cc:1048] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:46.409798  5636 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a00c228a31d744968a6b32e3601d51eb"
dest_uuid: "92adb80abb834eceaef035f370597703"
mode: GRACEFUL
new_leader_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
 from {username='slave'} at 127.0.0.1:44090
I20260709 12:15:46.410537  5636 raft_consensus.cc:606] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [term 1 LEADER]: Received request to transfer leadership to TS 3ea2ad5b842b4bf3868b4e25cae3e9f1
I20260709 12:15:46.420166  5561 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f66af11cf9374a31b237a393194eccae"
dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
mode: GRACEFUL
new_leader_uuid: "92adb80abb834eceaef035f370597703"
 from {username='slave'} at 127.0.0.1:34048
I20260709 12:15:46.420818  5561 raft_consensus.cc:606] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 LEADER]: Received request to transfer leadership to TS 92adb80abb834eceaef035f370597703
I20260709 12:15:46.421154  5752 raft_consensus.cc:993] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703: : Instructing follower 3ea2ad5b842b4bf3868b4e25cae3e9f1 to start an election
I20260709 12:15:46.421586  5752 raft_consensus.cc:1081] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 1 LEADER]: Signalling peer 3ea2ad5b842b4bf3868b4e25cae3e9f1 to start an election
I20260709 12:15:46.423614  5561 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1126f26405b742a6a136656ca8c86696"
dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
 from {username='slave'} at 127.0.0.1:34032
I20260709 12:15:46.424573  5561 raft_consensus.cc:493] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:46.424993  5561 raft_consensus.cc:3060] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:46.429747  5561 raft_consensus.cc:515] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:46.431526  5561 leader_election.cc:290] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 2 election: Requested vote from peers d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021), 92adb80abb834eceaef035f370597703 (127.5.58.3:41427)
I20260709 12:15:46.432709  5636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1126f26405b742a6a136656ca8c86696" candidate_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "92adb80abb834eceaef035f370597703"
I20260709 12:15:46.433329  5636 raft_consensus.cc:3055] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:46.433755  5636 raft_consensus.cc:740] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 92adb80abb834eceaef035f370597703, State: Running, Role: LEADER
I20260709 12:15:46.433750  5486 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1126f26405b742a6a136656ca8c86696" candidate_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
I20260709 12:15:46.434742  5486 raft_consensus.cc:3060] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:46.434661  5636 consensus_queue.cc:260] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:46.436367  5636 raft_consensus.cc:3060] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:46.442317  5486 raft_consensus.cc:2468] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea2ad5b842b4bf3868b4e25cae3e9f1 in term 2.
I20260709 12:15:46.443289  5636 raft_consensus.cc:2468] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea2ad5b842b4bf3868b4e25cae3e9f1 in term 2.
I20260709 12:15:46.443713  5527 leader_election.cc:304] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea2ad5b842b4bf3868b4e25cae3e9f1, d7f53a4ef21a4d8788fae6df6bb5f98c; no voters: 
I20260709 12:15:46.444526  5688 raft_consensus.cc:2804] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:46.445005  5688 raft_consensus.cc:697] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 LEADER]: Becoming Leader. State: Replica: 3ea2ad5b842b4bf3868b4e25cae3e9f1, State: Running, Role: LEADER
I20260709 12:15:46.445796  5688 consensus_queue.cc:237] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:46.452239  5684 raft_consensus.cc:993] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703: : Instructing follower d7f53a4ef21a4d8788fae6df6bb5f98c to start an election
I20260709 12:15:46.452723  5684 raft_consensus.cc:1081] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 1 LEADER]: Signalling peer d7f53a4ef21a4d8788fae6df6bb5f98c to start an election
I20260709 12:15:46.454505  5486 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "930c10c719284908b31b1b583ac8bc92"
dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
 from {username='slave'} at 127.0.0.1:55118
I20260709 12:15:46.455116  5486 raft_consensus.cc:493] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:46.455621  5486 raft_consensus.cc:3060] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:46.455076  5383 catalog_manager.cc:5697] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 reported cstate change: term changed from 1 to 2, leader changed from 92adb80abb834eceaef035f370597703 (127.5.58.3) to 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2). New cstate: current_term: 2 leader_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:46.461872  5486 raft_consensus.cc:515] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:46.464617  5561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930c10c719284908b31b1b583ac8bc92" candidate_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
I20260709 12:15:46.465245  5561 raft_consensus.cc:3060] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:46.465965  5486 leader_election.cc:290] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 2 election: Requested vote from peers 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2:36305), 92adb80abb834eceaef035f370597703 (127.5.58.3:41427)
I20260709 12:15:46.466572  5636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930c10c719284908b31b1b583ac8bc92" candidate_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "92adb80abb834eceaef035f370597703"
I20260709 12:15:46.467195  5636 raft_consensus.cc:3055] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:46.467540  5636 raft_consensus.cc:740] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 92adb80abb834eceaef035f370597703, State: Running, Role: LEADER
I20260709 12:15:46.468286  5636 consensus_queue.cc:260] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:46.469282  5636 raft_consensus.cc:3060] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:46.471776  5561 raft_consensus.cc:2468] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f53a4ef21a4d8788fae6df6bb5f98c in term 2.
I20260709 12:15:46.473140  5452 leader_election.cc:304] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea2ad5b842b4bf3868b4e25cae3e9f1, d7f53a4ef21a4d8788fae6df6bb5f98c; no voters: 
I20260709 12:15:46.473732  5636 raft_consensus.cc:2468] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f53a4ef21a4d8788fae6df6bb5f98c in term 2.
I20260709 12:15:46.473927  5705 raft_consensus.cc:2804] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:46.474407  5705 raft_consensus.cc:697] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 LEADER]: Becoming Leader. State: Replica: d7f53a4ef21a4d8788fae6df6bb5f98c, State: Running, Role: LEADER
I20260709 12:15:46.475206  5705 consensus_queue.cc:237] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:46.481750  5383 catalog_manager.cc:5697] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c reported cstate change: term changed from 1 to 2, leader changed from 92adb80abb834eceaef035f370597703 (127.5.58.3) to d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1). New cstate: current_term: 2 leader_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:46.569267  5757 raft_consensus.cc:993] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c: : Instructing follower 3ea2ad5b842b4bf3868b4e25cae3e9f1 to start an election
I20260709 12:15:46.570191  5757 raft_consensus.cc:1081] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 LEADER]: Signalling peer 3ea2ad5b842b4bf3868b4e25cae3e9f1 to start an election
I20260709 12:15:46.572795  5561 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f0f1dad5e1c540edbbba5708a7698085"
dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
 from {username='slave'} at 127.0.0.1:34028
I20260709 12:15:46.573844  5561 raft_consensus.cc:493] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:46.573971  5688 raft_consensus.cc:993] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1: : Instructing follower 92adb80abb834eceaef035f370597703 to start an election
I20260709 12:15:46.574388  5561 raft_consensus.cc:3060] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:46.574537  5688 raft_consensus.cc:1081] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 LEADER]: Signalling peer 92adb80abb834eceaef035f370597703 to start an election
I20260709 12:15:46.577016  5636 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f66af11cf9374a31b237a393194eccae"
dest_uuid: "92adb80abb834eceaef035f370597703"
 from {username='slave'} at 127.0.0.1:44078
I20260709 12:15:46.577791  5636 raft_consensus.cc:493] T f66af11cf9374a31b237a393194eccae P 92adb80abb834eceaef035f370597703 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:46.578267  5636 raft_consensus.cc:3060] T f66af11cf9374a31b237a393194eccae P 92adb80abb834eceaef035f370597703 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:46.581032  5561 raft_consensus.cc:515] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:46.583874  5561 leader_election.cc:290] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 2 election: Requested vote from peers d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021), 92adb80abb834eceaef035f370597703 (127.5.58.3:41427)
I20260709 12:15:46.584278  5486 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0f1dad5e1c540edbbba5708a7698085" candidate_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
I20260709 12:15:46.584564  5635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0f1dad5e1c540edbbba5708a7698085" candidate_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "92adb80abb834eceaef035f370597703"
I20260709 12:15:46.585191  5486 raft_consensus.cc:3055] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:46.585240  5635 raft_consensus.cc:3060] T f0f1dad5e1c540edbbba5708a7698085 P 92adb80abb834eceaef035f370597703 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:46.585633  5486 raft_consensus.cc:740] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d7f53a4ef21a4d8788fae6df6bb5f98c, State: Running, Role: LEADER
I20260709 12:15:46.586375  5486 consensus_queue.cc:260] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:46.587189  5486 raft_consensus.cc:3060] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:46.590997  5636 raft_consensus.cc:515] T f66af11cf9374a31b237a393194eccae P 92adb80abb834eceaef035f370597703 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:46.592624  5635 raft_consensus.cc:2468] T f0f1dad5e1c540edbbba5708a7698085 P 92adb80abb834eceaef035f370597703 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea2ad5b842b4bf3868b4e25cae3e9f1 in term 2.
I20260709 12:15:46.593425  5636 leader_election.cc:290] T f66af11cf9374a31b237a393194eccae P 92adb80abb834eceaef035f370597703 [CANDIDATE]: Term 2 election: Requested vote from peers d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021), 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2:36305)
I20260709 12:15:46.593955  5525 leader_election.cc:304] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea2ad5b842b4bf3868b4e25cae3e9f1, 92adb80abb834eceaef035f370597703; no voters: 
I20260709 12:15:46.594590  5487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66af11cf9374a31b237a393194eccae" candidate_uuid: "92adb80abb834eceaef035f370597703" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
I20260709 12:15:46.594592  5561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66af11cf9374a31b237a393194eccae" candidate_uuid: "92adb80abb834eceaef035f370597703" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
I20260709 12:15:46.595110  5708 raft_consensus.cc:2804] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:46.595324  5487 raft_consensus.cc:3060] T f66af11cf9374a31b237a393194eccae P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:46.595392  5561 raft_consensus.cc:3055] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:46.595634  5708 raft_consensus.cc:697] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 LEADER]: Becoming Leader. State: Replica: 3ea2ad5b842b4bf3868b4e25cae3e9f1, State: Running, Role: LEADER
I20260709 12:15:46.595849  5561 raft_consensus.cc:740] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ea2ad5b842b4bf3868b4e25cae3e9f1, State: Running, Role: LEADER
I20260709 12:15:46.596808  5561 consensus_queue.cc:260] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:46.597996  5561 raft_consensus.cc:3060] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:46.600823  5708 consensus_queue.cc:237] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:46.605226  5561 raft_consensus.cc:2468] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92adb80abb834eceaef035f370597703 in term 2.
I20260709 12:15:46.607267  5601 leader_election.cc:304] T f66af11cf9374a31b237a393194eccae P 92adb80abb834eceaef035f370597703 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea2ad5b842b4bf3868b4e25cae3e9f1, 92adb80abb834eceaef035f370597703; no voters: 
I20260709 12:15:46.607542  5487 raft_consensus.cc:2468] T f66af11cf9374a31b237a393194eccae P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92adb80abb834eceaef035f370597703 in term 2.
I20260709 12:15:46.608105  5752 raft_consensus.cc:2804] T f66af11cf9374a31b237a393194eccae P 92adb80abb834eceaef035f370597703 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:46.608687  5752 raft_consensus.cc:697] T f66af11cf9374a31b237a393194eccae P 92adb80abb834eceaef035f370597703 [term 2 LEADER]: Becoming Leader. State: Replica: 92adb80abb834eceaef035f370597703, State: Running, Role: LEADER
I20260709 12:15:46.608856  5486 raft_consensus.cc:2468] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea2ad5b842b4bf3868b4e25cae3e9f1 in term 2.
I20260709 12:15:46.609526  5752 consensus_queue.cc:237] T f66af11cf9374a31b237a393194eccae P 92adb80abb834eceaef035f370597703 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:46.617664  5383 catalog_manager.cc:5697] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 reported cstate change: term changed from 1 to 2, leader changed from d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1) to 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2). New cstate: current_term: 2 leader_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:46.625370  5382 catalog_manager.cc:5697] T f66af11cf9374a31b237a393194eccae P 92adb80abb834eceaef035f370597703 reported cstate change: term changed from 1 to 2, leader changed from 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2) to 92adb80abb834eceaef035f370597703 (127.5.58.3). New cstate: current_term: 2 leader_uuid: "92adb80abb834eceaef035f370597703" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:46.692101  5684 raft_consensus.cc:993] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703: : Instructing follower 3ea2ad5b842b4bf3868b4e25cae3e9f1 to start an election
I20260709 12:15:46.692603  5752 raft_consensus.cc:1081] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [term 1 LEADER]: Signalling peer 3ea2ad5b842b4bf3868b4e25cae3e9f1 to start an election
I20260709 12:15:46.694388  5561 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a00c228a31d744968a6b32e3601d51eb"
dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
 from {username='slave'} at 127.0.0.1:34032
I20260709 12:15:46.694942  5561 raft_consensus.cc:493] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:46.695286  5561 raft_consensus.cc:3060] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:46.701970  5561 raft_consensus.cc:515] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:46.705371  5561 leader_election.cc:290] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 2 election: Requested vote from peers d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021), 92adb80abb834eceaef035f370597703 (127.5.58.3:41427)
I20260709 12:15:46.705747  5487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a00c228a31d744968a6b32e3601d51eb" candidate_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
I20260709 12:15:46.706547  5487 raft_consensus.cc:3060] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:46.708150  5636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a00c228a31d744968a6b32e3601d51eb" candidate_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "92adb80abb834eceaef035f370597703"
I20260709 12:15:46.708858  5636 raft_consensus.cc:3055] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:46.709180  5636 raft_consensus.cc:740] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 92adb80abb834eceaef035f370597703, State: Running, Role: LEADER
I20260709 12:15:46.710202  5636 consensus_queue.cc:260] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:46.711722  5636 raft_consensus.cc:3060] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:46.720244  5636 raft_consensus.cc:2468] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea2ad5b842b4bf3868b4e25cae3e9f1 in term 2.
I20260709 12:15:46.721909  5525 leader_election.cc:304] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea2ad5b842b4bf3868b4e25cae3e9f1, 92adb80abb834eceaef035f370597703; no voters: 
I20260709 12:15:46.722539  5487 raft_consensus.cc:2468] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea2ad5b842b4bf3868b4e25cae3e9f1 in term 2.
I20260709 12:15:46.722903  5708 raft_consensus.cc:2804] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:46.723434  5708 raft_consensus.cc:697] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 LEADER]: Becoming Leader. State: Replica: 3ea2ad5b842b4bf3868b4e25cae3e9f1, State: Running, Role: LEADER
I20260709 12:15:46.724402  5708 consensus_queue.cc:237] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:46.733873  5382 catalog_manager.cc:5697] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1 reported cstate change: term changed from 1 to 2, leader changed from 92adb80abb834eceaef035f370597703 (127.5.58.3) to 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2). New cstate: current_term: 2 leader_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:46.866569  5561 raft_consensus.cc:1275] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 FOLLOWER]: Refusing update from remote peer d7f53a4ef21a4d8788fae6df6bb5f98c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:46.868014  5757 consensus_queue.cc:1048] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:46.882102  5636 raft_consensus.cc:1275] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 2 FOLLOWER]: Refusing update from remote peer d7f53a4ef21a4d8788fae6df6bb5f98c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:46.883831  5705 consensus_queue.cc:1048] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c [LEADER]: Connected to new peer: Peer: permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:46.892198  5636 raft_consensus.cc:1275] T a89ab5b73a4a42f7b8b529758c61edf7 P 92adb80abb834eceaef035f370597703 [term 2 FOLLOWER]: Refusing update from remote peer d7f53a4ef21a4d8788fae6df6bb5f98c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:46.893568  5757 consensus_queue.cc:1048] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [LEADER]: Connected to new peer: Peer: permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:46.901818  5561 raft_consensus.cc:1275] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 FOLLOWER]: Refusing update from remote peer d7f53a4ef21a4d8788fae6df6bb5f98c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:46.904186  5705 consensus_queue.cc:1048] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:46.934504  5487 raft_consensus.cc:1275] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 FOLLOWER]: Refusing update from remote peer 3ea2ad5b842b4bf3868b4e25cae3e9f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:46.936650  5708 consensus_queue.cc:1048] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:46.945885  5636 raft_consensus.cc:1275] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 2 FOLLOWER]: Refusing update from remote peer 3ea2ad5b842b4bf3868b4e25cae3e9f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:46.947739  5688 consensus_queue.cc:1048] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:46.986790  5636 raft_consensus.cc:1275] T f0f1dad5e1c540edbbba5708a7698085 P 92adb80abb834eceaef035f370597703 [term 2 FOLLOWER]: Refusing update from remote peer 3ea2ad5b842b4bf3868b4e25cae3e9f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:46.988094  5688 consensus_queue.cc:1048] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:46.995316  5487 raft_consensus.cc:1275] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 FOLLOWER]: Refusing update from remote peer 3ea2ad5b842b4bf3868b4e25cae3e9f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:46.996802  5688 consensus_queue.cc:1048] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:47.034544  5487 raft_consensus.cc:1275] T f66af11cf9374a31b237a393194eccae P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 FOLLOWER]: Refusing update from remote peer 92adb80abb834eceaef035f370597703: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:47.035949  5752 consensus_queue.cc:1048] T f66af11cf9374a31b237a393194eccae P 92adb80abb834eceaef035f370597703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:47.044025  5561 raft_consensus.cc:1275] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 FOLLOWER]: Refusing update from remote peer 92adb80abb834eceaef035f370597703: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:47.045684  5769 consensus_queue.cc:1048] T f66af11cf9374a31b237a393194eccae P 92adb80abb834eceaef035f370597703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:47.131806  5487 raft_consensus.cc:1275] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 FOLLOWER]: Refusing update from remote peer 3ea2ad5b842b4bf3868b4e25cae3e9f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:47.132964  5709 consensus_queue.cc:1048] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:47.142165  5636 raft_consensus.cc:1275] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [term 2 FOLLOWER]: Refusing update from remote peer 3ea2ad5b842b4bf3868b4e25cae3e9f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:47.144204  5688 consensus_queue.cc:1048] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:49.455050  5352 auto_leader_rebalancer-test.cc:294] The leader distribution is 
I20260709 12:15:49.456007  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:49.477142  5562 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1126f26405b742a6a136656ca8c86696"
dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
mode: GRACEFUL
new_leader_uuid: "92adb80abb834eceaef035f370597703"
 from {username='slave'} at 127.0.0.1:34052
I20260709 12:15:49.477638  5562 raft_consensus.cc:606] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 LEADER]: Received request to transfer leadership to TS 92adb80abb834eceaef035f370597703
I20260709 12:15:49.488883  5487 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "930c10c719284908b31b1b583ac8bc92"
dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
mode: GRACEFUL
new_leader_uuid: "92adb80abb834eceaef035f370597703"
 from {username='slave'} at 127.0.0.1:55152
I20260709 12:15:49.489503  5487 raft_consensus.cc:606] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 LEADER]: Received request to transfer leadership to TS 92adb80abb834eceaef035f370597703
I20260709 12:15:49.490706  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:15:49.491189  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:49.546337  5781 raft_consensus.cc:993] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1: : Instructing follower 92adb80abb834eceaef035f370597703 to start an election
I20260709 12:15:49.546942  5782 raft_consensus.cc:993] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c: : Instructing follower 92adb80abb834eceaef035f370597703 to start an election
I20260709 12:15:49.547194  5708 raft_consensus.cc:1081] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 LEADER]: Signalling peer 92adb80abb834eceaef035f370597703 to start an election
I20260709 12:15:49.547505  5776 raft_consensus.cc:1081] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 LEADER]: Signalling peer 92adb80abb834eceaef035f370597703 to start an election
I20260709 12:15:49.549422  5635 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "930c10c719284908b31b1b583ac8bc92"
dest_uuid: "92adb80abb834eceaef035f370597703"
 from {username='slave'} at 127.0.0.1:44070
I20260709 12:15:49.549721  5636 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1126f26405b742a6a136656ca8c86696"
dest_uuid: "92adb80abb834eceaef035f370597703"
 from {username='slave'} at 127.0.0.1:44078
I20260709 12:15:49.550037  5635 raft_consensus.cc:493] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:49.550195  5636 raft_consensus.cc:493] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:49.550444  5635 raft_consensus.cc:3060] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:49.550544  5636 raft_consensus.cc:3060] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:49.556764  5635 raft_consensus.cc:515] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:49.557190  5636 raft_consensus.cc:515] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:49.558909  5635 leader_election.cc:290] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [CANDIDATE]: Term 3 election: Requested vote from peers d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021), 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2:36305)
I20260709 12:15:49.559388  5636 leader_election.cc:290] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [CANDIDATE]: Term 3 election: Requested vote from peers d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021), 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2:36305)
I20260709 12:15:49.559496  5487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930c10c719284908b31b1b583ac8bc92" candidate_uuid: "92adb80abb834eceaef035f370597703" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
I20260709 12:15:49.560130  5487 raft_consensus.cc:3055] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:49.561007  5487 raft_consensus.cc:740] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d7f53a4ef21a4d8788fae6df6bb5f98c, State: Running, Role: LEADER
I20260709 12:15:49.561021  5561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1126f26405b742a6a136656ca8c86696" candidate_uuid: "92adb80abb834eceaef035f370597703" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
I20260709 12:15:49.561100  5486 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1126f26405b742a6a136656ca8c86696" candidate_uuid: "92adb80abb834eceaef035f370597703" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
I20260709 12:15:49.560966  5562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930c10c719284908b31b1b583ac8bc92" candidate_uuid: "92adb80abb834eceaef035f370597703" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
I20260709 12:15:49.561805  5561 raft_consensus.cc:3055] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:49.561965  5486 raft_consensus.cc:3060] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:49.561990  5562 raft_consensus.cc:3060] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:49.562196  5561 raft_consensus.cc:740] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ea2ad5b842b4bf3868b4e25cae3e9f1, State: Running, Role: LEADER
I20260709 12:15:49.562167  5487 consensus_queue.cc:260] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:49.563191  5561 consensus_queue.cc:260] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:49.563647  5487 raft_consensus.cc:3060] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:49.564441  5561 raft_consensus.cc:3060] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:49.569638  5562 raft_consensus.cc:2468] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92adb80abb834eceaef035f370597703 in term 3.
I20260709 12:15:49.570791  5487 raft_consensus.cc:2468] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92adb80abb834eceaef035f370597703 in term 3.
I20260709 12:15:49.571209  5601 leader_election.cc:304] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea2ad5b842b4bf3868b4e25cae3e9f1, 92adb80abb834eceaef035f370597703; no voters: 
I20260709 12:15:49.572388  5561 raft_consensus.cc:2468] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92adb80abb834eceaef035f370597703 in term 3.
I20260709 12:15:49.572626  5775 raft_consensus.cc:2804] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:49.569638  5486 raft_consensus.cc:2468] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92adb80abb834eceaef035f370597703 in term 3.
I20260709 12:15:49.573693  5601 leader_election.cc:304] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea2ad5b842b4bf3868b4e25cae3e9f1, 92adb80abb834eceaef035f370597703; no voters: 
I20260709 12:15:49.574462  5775 raft_consensus.cc:697] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 3 LEADER]: Becoming Leader. State: Replica: 92adb80abb834eceaef035f370597703, State: Running, Role: LEADER
I20260709 12:15:49.574921  5783 raft_consensus.cc:2804] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:49.575407  5783 raft_consensus.cc:697] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 3 LEADER]: Becoming Leader. State: Replica: 92adb80abb834eceaef035f370597703, State: Running, Role: LEADER
I20260709 12:15:49.575380  5775 consensus_queue.cc:237] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:49.576360  5783 consensus_queue.cc:237] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:49.585965  5382 catalog_manager.cc:5697] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 reported cstate change: term changed from 2 to 3, leader changed from d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1) to 92adb80abb834eceaef035f370597703 (127.5.58.3). New cstate: current_term: 3 leader_uuid: "92adb80abb834eceaef035f370597703" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:49.587105  5382 catalog_manager.cc:5697] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 reported cstate change: term changed from 2 to 3, leader changed from 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2) to 92adb80abb834eceaef035f370597703 (127.5.58.3). New cstate: current_term: 3 leader_uuid: "92adb80abb834eceaef035f370597703" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:50.036540  5486 raft_consensus.cc:1275] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 3 FOLLOWER]: Refusing update from remote peer 92adb80abb834eceaef035f370597703: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:50.038344  5783 consensus_queue.cc:1048] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:15:50.051141  5562 raft_consensus.cc:1275] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 3 FOLLOWER]: Refusing update from remote peer 92adb80abb834eceaef035f370597703: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:50.053326  5784 consensus_queue.cc:1048] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:15:50.057323  5561 raft_consensus.cc:1275] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 3 FOLLOWER]: Refusing update from remote peer 92adb80abb834eceaef035f370597703: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:50.060241  5775 consensus_queue.cc:1048] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:15:50.076620  5486 raft_consensus.cc:1275] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 3 FOLLOWER]: Refusing update from remote peer 92adb80abb834eceaef035f370597703: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:50.078882  5775 consensus_queue.cc:1048] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:50.492233  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:50.498869  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:50.499325  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:51.500386  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:51.505268  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:51.505712  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:52.507500  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:52.514678  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:52.515141  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:53.516247  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:53.523685  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:53.524103  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:54.525194  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:54.531312  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:54.531700  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:55.532718  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:55.537501  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:55.537868  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:56.538743  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:56.544353  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:56.544797  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:57.545674  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:57.550354  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:57.550743  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:58.551679  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:58.556313  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:58.556705  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:59.589672  5352 auto_leader_rebalancer-test.cc:313] The leader distribution is 
I20260709 12:15:59.594250  5486 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a89ab5b73a4a42f7b8b529758c61edf7"
dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
mode: GRACEFUL
new_leader_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
 from {username='slave'} at 127.0.0.1:55150
I20260709 12:15:59.594905  5486 raft_consensus.cc:606] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 LEADER]: Received request to transfer leadership to TS 3ea2ad5b842b4bf3868b4e25cae3e9f1
I20260709 12:15:59.600160  5486 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4fd3cf9b75144e5c88eee1a8917e283e"
dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
mode: GRACEFUL
new_leader_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
 from {username='slave'} at 127.0.0.1:55150
I20260709 12:15:59.600778  5486 raft_consensus.cc:606] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 LEADER]: Received request to transfer leadership to TS 3ea2ad5b842b4bf3868b4e25cae3e9f1
I20260709 12:15:59.605983  5636 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "930c10c719284908b31b1b583ac8bc92"
dest_uuid: "92adb80abb834eceaef035f370597703"
mode: GRACEFUL
new_leader_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
 from {username='slave'} at 127.0.0.1:44090
I20260709 12:15:59.606441  5636 raft_consensus.cc:606] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 3 LEADER]: Received request to transfer leadership to TS 3ea2ad5b842b4bf3868b4e25cae3e9f1
I20260709 12:15:59.612797  5486 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "09cfb685589d470ab66f97d1d81df47c"
dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
mode: GRACEFUL
new_leader_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
 from {username='slave'} at 127.0.0.1:55150
I20260709 12:15:59.613261  5486 raft_consensus.cc:606] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 LEADER]: Received request to transfer leadership to TS 3ea2ad5b842b4bf3868b4e25cae3e9f1
I20260709 12:15:59.621001  5636 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1126f26405b742a6a136656ca8c86696"
dest_uuid: "92adb80abb834eceaef035f370597703"
mode: GRACEFUL
new_leader_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
 from {username='slave'} at 127.0.0.1:44090
I20260709 12:15:59.621450  5636 raft_consensus.cc:606] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 3 LEADER]: Received request to transfer leadership to TS 3ea2ad5b842b4bf3868b4e25cae3e9f1
I20260709 12:15:59.653481  5806 raft_consensus.cc:993] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703: : Instructing follower 3ea2ad5b842b4bf3868b4e25cae3e9f1 to start an election
I20260709 12:15:59.654016  5801 raft_consensus.cc:1081] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 3 LEADER]: Signalling peer 3ea2ad5b842b4bf3868b4e25cae3e9f1 to start an election
I20260709 12:15:59.655500  5562 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1126f26405b742a6a136656ca8c86696"
dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
 from {username='slave'} at 127.0.0.1:34032
I20260709 12:15:59.656006  5562 raft_consensus.cc:493] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:59.656385  5562 raft_consensus.cc:3060] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:59.660907  5806 raft_consensus.cc:993] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703: : Instructing follower 3ea2ad5b842b4bf3868b4e25cae3e9f1 to start an election
I20260709 12:15:59.661463  5806 raft_consensus.cc:1081] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 3 LEADER]: Signalling peer 3ea2ad5b842b4bf3868b4e25cae3e9f1 to start an election
I20260709 12:15:59.663326  5561 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "930c10c719284908b31b1b583ac8bc92"
dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
 from {username='slave'} at 127.0.0.1:34032
I20260709 12:15:59.663838  5561 raft_consensus.cc:493] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:59.663689  5562 raft_consensus.cc:515] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:59.664187  5561 raft_consensus.cc:3060] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:59.666671  5562 leader_election.cc:290] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 4 election: Requested vote from peers d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021), 92adb80abb834eceaef035f370597703 (127.5.58.3:41427)
I20260709 12:15:59.666952  5486 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1126f26405b742a6a136656ca8c86696" candidate_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
I20260709 12:15:59.667292  5636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1126f26405b742a6a136656ca8c86696" candidate_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "92adb80abb834eceaef035f370597703"
I20260709 12:15:59.667742  5486 raft_consensus.cc:3060] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:59.667904  5636 raft_consensus.cc:3055] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:15:59.668291  5636 raft_consensus.cc:740] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 92adb80abb834eceaef035f370597703, State: Running, Role: LEADER
I20260709 12:15:59.669284  5636 consensus_queue.cc:260] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:59.670634  5636 raft_consensus.cc:3060] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:59.671437  5561 raft_consensus.cc:515] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:59.673281  5561 leader_election.cc:290] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 4 election: Requested vote from peers d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021), 92adb80abb834eceaef035f370597703 (127.5.58.3:41427)
I20260709 12:15:59.674289  5487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930c10c719284908b31b1b583ac8bc92" candidate_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
I20260709 12:15:59.674876  5487 raft_consensus.cc:3060] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:59.674996  5635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930c10c719284908b31b1b583ac8bc92" candidate_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "92adb80abb834eceaef035f370597703"
I20260709 12:15:59.675643  5635 raft_consensus.cc:3055] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:15:59.675990  5635 raft_consensus.cc:740] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 92adb80abb834eceaef035f370597703, State: Running, Role: LEADER
I20260709 12:15:59.676532  5486 raft_consensus.cc:2468] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea2ad5b842b4bf3868b4e25cae3e9f1 in term 4.
I20260709 12:15:59.676935  5635 consensus_queue.cc:260] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:59.677592  5636 raft_consensus.cc:2468] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea2ad5b842b4bf3868b4e25cae3e9f1 in term 4.
I20260709 12:15:59.678429  5635 raft_consensus.cc:3060] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:15:59.678465  5527 leader_election.cc:304] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea2ad5b842b4bf3868b4e25cae3e9f1, d7f53a4ef21a4d8788fae6df6bb5f98c; no voters: 
I20260709 12:15:59.679790  5804 raft_consensus.cc:2804] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:15:59.683347  5487 raft_consensus.cc:2468] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea2ad5b842b4bf3868b4e25cae3e9f1 in term 4.
I20260709 12:15:59.684516  5804 raft_consensus.cc:697] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 4 LEADER]: Becoming Leader. State: Replica: 3ea2ad5b842b4bf3868b4e25cae3e9f1, State: Running, Role: LEADER
I20260709 12:15:59.685045  5527 leader_election.cc:304] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea2ad5b842b4bf3868b4e25cae3e9f1, d7f53a4ef21a4d8788fae6df6bb5f98c; no voters: 
I20260709 12:15:59.685719  5635 raft_consensus.cc:2468] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea2ad5b842b4bf3868b4e25cae3e9f1 in term 4.
I20260709 12:15:59.686020  5807 raft_consensus.cc:2804] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:15:59.685633  5804 consensus_queue.cc:237] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:59.687269  5807 raft_consensus.cc:697] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 4 LEADER]: Becoming Leader. State: Replica: 3ea2ad5b842b4bf3868b4e25cae3e9f1, State: Running, Role: LEADER
I20260709 12:15:59.688292  5807 consensus_queue.cc:237] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:59.695405  5383 catalog_manager.cc:5697] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 reported cstate change: term changed from 3 to 4, leader changed from 92adb80abb834eceaef035f370597703 (127.5.58.3) to 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2). New cstate: current_term: 4 leader_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:59.712920  5383 catalog_manager.cc:5697] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 reported cstate change: term changed from 3 to 4, leader changed from 92adb80abb834eceaef035f370597703 (127.5.58.3) to 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2). New cstate: current_term: 4 leader_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:59.766077  5815 raft_consensus.cc:993] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c: : Instructing follower 3ea2ad5b842b4bf3868b4e25cae3e9f1 to start an election
I20260709 12:15:59.766723  5803 raft_consensus.cc:1081] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 LEADER]: Signalling peer 3ea2ad5b842b4bf3868b4e25cae3e9f1 to start an election
I20260709 12:15:59.768610  5561 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "09cfb685589d470ab66f97d1d81df47c"
dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
 from {username='slave'} at 127.0.0.1:34028
I20260709 12:15:59.769341  5561 raft_consensus.cc:493] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:59.769668  5561 raft_consensus.cc:3060] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:59.774750  5561 raft_consensus.cc:515] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:59.776515  5561 leader_election.cc:290] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 2 election: Requested vote from peers d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021), 92adb80abb834eceaef035f370597703 (127.5.58.3:41427)
I20260709 12:15:59.777163  5487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09cfb685589d470ab66f97d1d81df47c" candidate_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
I20260709 12:15:59.777626  5635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09cfb685589d470ab66f97d1d81df47c" candidate_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "92adb80abb834eceaef035f370597703"
I20260709 12:15:59.777794  5487 raft_consensus.cc:3055] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:59.778110  5487 raft_consensus.cc:740] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d7f53a4ef21a4d8788fae6df6bb5f98c, State: Running, Role: LEADER
I20260709 12:15:59.778189  5635 raft_consensus.cc:3060] T 09cfb685589d470ab66f97d1d81df47c P 92adb80abb834eceaef035f370597703 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:59.778905  5487 consensus_queue.cc:260] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:59.779938  5487 raft_consensus.cc:3060] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:59.783831  5487 raft_consensus.cc:2468] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea2ad5b842b4bf3868b4e25cae3e9f1 in term 2.
I20260709 12:15:59.784376  5635 raft_consensus.cc:2468] T 09cfb685589d470ab66f97d1d81df47c P 92adb80abb834eceaef035f370597703 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea2ad5b842b4bf3868b4e25cae3e9f1 in term 2.
I20260709 12:15:59.785063  5527 leader_election.cc:304] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea2ad5b842b4bf3868b4e25cae3e9f1, d7f53a4ef21a4d8788fae6df6bb5f98c; no voters: 
I20260709 12:15:59.786031  5807 raft_consensus.cc:2804] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:59.786509  5807 raft_consensus.cc:697] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 LEADER]: Becoming Leader. State: Replica: 3ea2ad5b842b4bf3868b4e25cae3e9f1, State: Running, Role: LEADER
I20260709 12:15:59.787338  5807 consensus_queue.cc:237] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:59.794070  5383 catalog_manager.cc:5697] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 reported cstate change: term changed from 1 to 2, leader changed from d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1) to 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2). New cstate: current_term: 2 leader_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:59.816992  5815 raft_consensus.cc:993] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c: : Instructing follower 3ea2ad5b842b4bf3868b4e25cae3e9f1 to start an election
I20260709 12:15:59.817474  5815 raft_consensus.cc:1081] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 LEADER]: Signalling peer 3ea2ad5b842b4bf3868b4e25cae3e9f1 to start an election
I20260709 12:15:59.818946  5561 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a89ab5b73a4a42f7b8b529758c61edf7"
dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
 from {username='slave'} at 127.0.0.1:34028
I20260709 12:15:59.819358  5561 raft_consensus.cc:493] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:59.819635  5561 raft_consensus.cc:3060] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:59.824189  5561 raft_consensus.cc:515] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } }
I20260709 12:15:59.825639  5561 leader_election.cc:290] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 3 election: Requested vote from peers d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021), 92adb80abb834eceaef035f370597703 (127.5.58.3:41427)
I20260709 12:15:59.826659  5487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a89ab5b73a4a42f7b8b529758c61edf7" candidate_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
I20260709 12:15:59.826725  5635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a89ab5b73a4a42f7b8b529758c61edf7" candidate_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "92adb80abb834eceaef035f370597703"
I20260709 12:15:59.827252  5487 raft_consensus.cc:3055] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:59.827382  5635 raft_consensus.cc:3060] T a89ab5b73a4a42f7b8b529758c61edf7 P 92adb80abb834eceaef035f370597703 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:59.827586  5487 raft_consensus.cc:740] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d7f53a4ef21a4d8788fae6df6bb5f98c, State: Running, Role: LEADER
I20260709 12:15:59.828322  5487 consensus_queue.cc:260] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } }
I20260709 12:15:59.829293  5487 raft_consensus.cc:3060] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:59.832593  5635 raft_consensus.cc:2468] T a89ab5b73a4a42f7b8b529758c61edf7 P 92adb80abb834eceaef035f370597703 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea2ad5b842b4bf3868b4e25cae3e9f1 in term 3.
I20260709 12:15:59.833678  5487 raft_consensus.cc:2468] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea2ad5b842b4bf3868b4e25cae3e9f1 in term 3.
I20260709 12:15:59.834038  5525 leader_election.cc:304] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea2ad5b842b4bf3868b4e25cae3e9f1, 92adb80abb834eceaef035f370597703; no voters: 
I20260709 12:15:59.834901  5807 raft_consensus.cc:2804] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:59.835336  5807 raft_consensus.cc:697] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 3 LEADER]: Becoming Leader. State: Replica: 3ea2ad5b842b4bf3868b4e25cae3e9f1, State: Running, Role: LEADER
I20260709 12:15:59.836138  5807 consensus_queue.cc:237] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } }
I20260709 12:15:59.842777  5383 catalog_manager.cc:5697] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 reported cstate change: term changed from 2 to 3, leader changed from d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1) to 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2). New cstate: current_term: 3 leader_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:59.950944  5803 raft_consensus.cc:993] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c: : Instructing follower 3ea2ad5b842b4bf3868b4e25cae3e9f1 to start an election
I20260709 12:15:59.951449  5803 raft_consensus.cc:1081] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 LEADER]: Signalling peer 3ea2ad5b842b4bf3868b4e25cae3e9f1 to start an election
I20260709 12:15:59.952965  5561 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4fd3cf9b75144e5c88eee1a8917e283e"
dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
 from {username='slave'} at 127.0.0.1:34028
I20260709 12:15:59.953440  5561 raft_consensus.cc:493] T 4fd3cf9b75144e5c88eee1a8917e283e P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:59.953677  5561 raft_consensus.cc:3060] T 4fd3cf9b75144e5c88eee1a8917e283e P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:59.957574  5561 raft_consensus.cc:515] T 4fd3cf9b75144e5c88eee1a8917e283e P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:59.959182  5561 leader_election.cc:290] T 4fd3cf9b75144e5c88eee1a8917e283e P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 2 election: Requested vote from peers d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021), 92adb80abb834eceaef035f370597703 (127.5.58.3:41427)
I20260709 12:15:59.959900  5487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fd3cf9b75144e5c88eee1a8917e283e" candidate_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
I20260709 12:15:59.960232  5635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fd3cf9b75144e5c88eee1a8917e283e" candidate_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "92adb80abb834eceaef035f370597703"
I20260709 12:15:59.960520  5487 raft_consensus.cc:3055] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:59.960803  5635 raft_consensus.cc:3060] T 4fd3cf9b75144e5c88eee1a8917e283e P 92adb80abb834eceaef035f370597703 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:59.960837  5487 raft_consensus.cc:740] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d7f53a4ef21a4d8788fae6df6bb5f98c, State: Running, Role: LEADER
I20260709 12:15:59.961745  5487 consensus_queue.cc:260] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:59.962838  5487 raft_consensus.cc:3060] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:59.967772  5635 raft_consensus.cc:2468] T 4fd3cf9b75144e5c88eee1a8917e283e P 92adb80abb834eceaef035f370597703 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea2ad5b842b4bf3868b4e25cae3e9f1 in term 2.
I20260709 12:15:59.969062  5525 leader_election.cc:304] T 4fd3cf9b75144e5c88eee1a8917e283e P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea2ad5b842b4bf3868b4e25cae3e9f1, 92adb80abb834eceaef035f370597703; no voters: 
I20260709 12:15:59.969244  5487 raft_consensus.cc:2468] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea2ad5b842b4bf3868b4e25cae3e9f1 in term 2.
I20260709 12:15:59.970049  5807 raft_consensus.cc:2804] T 4fd3cf9b75144e5c88eee1a8917e283e P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:59.970695  5807 raft_consensus.cc:697] T 4fd3cf9b75144e5c88eee1a8917e283e P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 LEADER]: Becoming Leader. State: Replica: 3ea2ad5b842b4bf3868b4e25cae3e9f1, State: Running, Role: LEADER
I20260709 12:15:59.971542  5807 consensus_queue.cc:237] T 4fd3cf9b75144e5c88eee1a8917e283e P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:15:59.978012  5383 catalog_manager.cc:5697] T 4fd3cf9b75144e5c88eee1a8917e283e P 3ea2ad5b842b4bf3868b4e25cae3e9f1 reported cstate change: term changed from 1 to 2, leader changed from d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1) to 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2). New cstate: current_term: 2 leader_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:00.133247  5635 raft_consensus.cc:1275] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 4 FOLLOWER]: Refusing update from remote peer 3ea2ad5b842b4bf3868b4e25cae3e9f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:00.134598  5807 consensus_queue.cc:1048] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:00.146291  5487 raft_consensus.cc:1275] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 4 FOLLOWER]: Refusing update from remote peer 3ea2ad5b842b4bf3868b4e25cae3e9f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:00.148145  5804 consensus_queue.cc:1048] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:00.243726  5635 raft_consensus.cc:1275] T 09cfb685589d470ab66f97d1d81df47c P 92adb80abb834eceaef035f370597703 [term 2 FOLLOWER]: Refusing update from remote peer 3ea2ad5b842b4bf3868b4e25cae3e9f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:00.245321  5804 consensus_queue.cc:1048] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:00.254751  5487 raft_consensus.cc:1275] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 FOLLOWER]: Refusing update from remote peer 3ea2ad5b842b4bf3868b4e25cae3e9f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:00.257973  5808 consensus_queue.cc:1048] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:00.261148  5636 raft_consensus.cc:1275] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 4 FOLLOWER]: Refusing update from remote peer 3ea2ad5b842b4bf3868b4e25cae3e9f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:00.263605  5808 consensus_queue.cc:1048] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:00.275015  5487 raft_consensus.cc:1275] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 4 FOLLOWER]: Refusing update from remote peer 3ea2ad5b842b4bf3868b4e25cae3e9f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:00.277801  5807 consensus_queue.cc:1048] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:00.418869  5636 raft_consensus.cc:1275] T a89ab5b73a4a42f7b8b529758c61edf7 P 92adb80abb834eceaef035f370597703 [term 3 FOLLOWER]: Refusing update from remote peer 3ea2ad5b842b4bf3868b4e25cae3e9f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:00.420189  5807 consensus_queue.cc:1048] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:00.430130  5487 raft_consensus.cc:1275] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 3 FOLLOWER]: Refusing update from remote peer 3ea2ad5b842b4bf3868b4e25cae3e9f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:00.432015  5808 consensus_queue.cc:1048] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:00.470017  5487 raft_consensus.cc:1275] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 FOLLOWER]: Refusing update from remote peer 3ea2ad5b842b4bf3868b4e25cae3e9f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:00.471781  5808 consensus_queue.cc:1048] T 4fd3cf9b75144e5c88eee1a8917e283e P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:00.482676  5636 raft_consensus.cc:1275] T 4fd3cf9b75144e5c88eee1a8917e283e P 92adb80abb834eceaef035f370597703 [term 2 FOLLOWER]: Refusing update from remote peer 3ea2ad5b842b4bf3868b4e25cae3e9f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:00.484830  5804 consensus_queue.cc:1048] T 4fd3cf9b75144e5c88eee1a8917e283e P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:02.663087  5352 auto_leader_rebalancer-test.cc:328] The leader distribution is 
I20260709 12:16:02.663962  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:02.671701  5561 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "09cfb685589d470ab66f97d1d81df47c"
dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
mode: GRACEFUL
new_leader_uuid: "92adb80abb834eceaef035f370597703"
 from {username='slave'} at 127.0.0.1:34052
I20260709 12:16:02.672626  5561 raft_consensus.cc:606] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 LEADER]: Received request to transfer leadership to TS 92adb80abb834eceaef035f370597703
I20260709 12:16:02.676098  5561 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1126f26405b742a6a136656ca8c86696"
dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
mode: GRACEFUL
new_leader_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
 from {username='slave'} at 127.0.0.1:34052
I20260709 12:16:02.676819  5561 raft_consensus.cc:606] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 4 LEADER]: Received request to transfer leadership to TS d7f53a4ef21a4d8788fae6df6bb5f98c
I20260709 12:16:02.680227  5561 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a00c228a31d744968a6b32e3601d51eb"
dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
mode: GRACEFUL
new_leader_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
 from {username='slave'} at 127.0.0.1:34052
I20260709 12:16:02.681007  5561 raft_consensus.cc:606] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 LEADER]: Received request to transfer leadership to TS d7f53a4ef21a4d8788fae6df6bb5f98c
I20260709 12:16:02.684703  5561 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a89ab5b73a4a42f7b8b529758c61edf7"
dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
mode: GRACEFUL
new_leader_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
 from {username='slave'} at 127.0.0.1:34052
I20260709 12:16:02.685354  5561 raft_consensus.cc:606] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 3 LEADER]: Received request to transfer leadership to TS d7f53a4ef21a4d8788fae6df6bb5f98c
I20260709 12:16:02.688607  5561 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f0f1dad5e1c540edbbba5708a7698085"
dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
mode: GRACEFUL
new_leader_uuid: "92adb80abb834eceaef035f370597703"
 from {username='slave'} at 127.0.0.1:34052
I20260709 12:16:02.689107  5561 raft_consensus.cc:606] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 LEADER]: Received request to transfer leadership to TS 92adb80abb834eceaef035f370597703
I20260709 12:16:02.690316  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:16:02.690798  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:02.724268  5835 raft_consensus.cc:993] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1: : Instructing follower d7f53a4ef21a4d8788fae6df6bb5f98c to start an election
I20260709 12:16:02.724879  5835 raft_consensus.cc:1081] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 LEADER]: Signalling peer d7f53a4ef21a4d8788fae6df6bb5f98c to start an election
I20260709 12:16:02.726687  5487 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a00c228a31d744968a6b32e3601d51eb"
dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
 from {username='slave'} at 127.0.0.1:55134
I20260709 12:16:02.727190  5487 raft_consensus.cc:493] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:02.727664  5487 raft_consensus.cc:3060] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:02.732476  5487 raft_consensus.cc:515] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:16:02.735176  5487 leader_election.cc:290] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 3 election: Requested vote from peers 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2:36305), 92adb80abb834eceaef035f370597703 (127.5.58.3:41427)
I20260709 12:16:02.736119  5561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a00c228a31d744968a6b32e3601d51eb" candidate_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
I20260709 12:16:02.736568  5636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a00c228a31d744968a6b32e3601d51eb" candidate_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "92adb80abb834eceaef035f370597703"
I20260709 12:16:02.736928  5561 raft_consensus.cc:3055] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:02.737124  5636 raft_consensus.cc:3060] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:02.737282  5561 raft_consensus.cc:740] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ea2ad5b842b4bf3868b4e25cae3e9f1, State: Running, Role: LEADER
I20260709 12:16:02.738250  5561 consensus_queue.cc:260] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:16:02.739214  5561 raft_consensus.cc:3060] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:02.742450  5636 raft_consensus.cc:2468] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f53a4ef21a4d8788fae6df6bb5f98c in term 3.
I20260709 12:16:02.743728  5450 leader_election.cc:304] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92adb80abb834eceaef035f370597703, d7f53a4ef21a4d8788fae6df6bb5f98c; no voters: 
I20260709 12:16:02.745285  5561 raft_consensus.cc:2468] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f53a4ef21a4d8788fae6df6bb5f98c in term 3.
I20260709 12:16:02.745779  5837 raft_consensus.cc:2804] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:02.747201  5837 raft_consensus.cc:697] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c [term 3 LEADER]: Becoming Leader. State: Replica: d7f53a4ef21a4d8788fae6df6bb5f98c, State: Running, Role: LEADER
I20260709 12:16:02.748100  5837 consensus_queue.cc:237] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:16:02.754678  5833 raft_consensus.cc:993] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1: : Instructing follower 92adb80abb834eceaef035f370597703 to start an election
I20260709 12:16:02.755123  5835 raft_consensus.cc:1081] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 LEADER]: Signalling peer 92adb80abb834eceaef035f370597703 to start an election
I20260709 12:16:02.756646  5636 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f0f1dad5e1c540edbbba5708a7698085"
dest_uuid: "92adb80abb834eceaef035f370597703"
 from {username='slave'} at 127.0.0.1:44078
I20260709 12:16:02.757097  5636 raft_consensus.cc:493] T f0f1dad5e1c540edbbba5708a7698085 P 92adb80abb834eceaef035f370597703 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:02.757364  5636 raft_consensus.cc:3060] T f0f1dad5e1c540edbbba5708a7698085 P 92adb80abb834eceaef035f370597703 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:02.757406  5383 catalog_manager.cc:5697] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c reported cstate change: term changed from 2 to 3, leader changed from 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2) to d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1). New cstate: current_term: 3 leader_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:02.762245  5636 raft_consensus.cc:515] T f0f1dad5e1c540edbbba5708a7698085 P 92adb80abb834eceaef035f370597703 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:16:02.764789  5636 leader_election.cc:290] T f0f1dad5e1c540edbbba5708a7698085 P 92adb80abb834eceaef035f370597703 [CANDIDATE]: Term 3 election: Requested vote from peers d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021), 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2:36305)
I20260709 12:16:02.765501  5487 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0f1dad5e1c540edbbba5708a7698085" candidate_uuid: "92adb80abb834eceaef035f370597703" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
I20260709 12:16:02.765964  5561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0f1dad5e1c540edbbba5708a7698085" candidate_uuid: "92adb80abb834eceaef035f370597703" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
I20260709 12:16:02.766101  5487 raft_consensus.cc:3060] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:02.766510  5561 raft_consensus.cc:3055] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:02.766825  5561 raft_consensus.cc:740] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ea2ad5b842b4bf3868b4e25cae3e9f1, State: Running, Role: LEADER
I20260709 12:16:02.767570  5561 consensus_queue.cc:260] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:16:02.768944  5561 raft_consensus.cc:3060] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:02.772631  5487 raft_consensus.cc:2468] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92adb80abb834eceaef035f370597703 in term 3.
I20260709 12:16:02.773918  5601 leader_election.cc:304] T f0f1dad5e1c540edbbba5708a7698085 P 92adb80abb834eceaef035f370597703 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92adb80abb834eceaef035f370597703, d7f53a4ef21a4d8788fae6df6bb5f98c; no voters: 
I20260709 12:16:02.774636  5561 raft_consensus.cc:2468] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92adb80abb834eceaef035f370597703 in term 3.
I20260709 12:16:02.774907  5806 raft_consensus.cc:2804] T f0f1dad5e1c540edbbba5708a7698085 P 92adb80abb834eceaef035f370597703 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:02.776201  5806 raft_consensus.cc:697] T f0f1dad5e1c540edbbba5708a7698085 P 92adb80abb834eceaef035f370597703 [term 3 LEADER]: Becoming Leader. State: Replica: 92adb80abb834eceaef035f370597703, State: Running, Role: LEADER
I20260709 12:16:02.777153  5806 consensus_queue.cc:237] T f0f1dad5e1c540edbbba5708a7698085 P 92adb80abb834eceaef035f370597703 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:16:02.785215  5383 catalog_manager.cc:5697] T f0f1dad5e1c540edbbba5708a7698085 P 92adb80abb834eceaef035f370597703 reported cstate change: term changed from 2 to 3, leader changed from 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2) to 92adb80abb834eceaef035f370597703 (127.5.58.3). New cstate: current_term: 3 leader_uuid: "92adb80abb834eceaef035f370597703" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:02.818923  5833 raft_consensus.cc:993] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1: : Instructing follower d7f53a4ef21a4d8788fae6df6bb5f98c to start an election
I20260709 12:16:02.819365  5835 raft_consensus.cc:1081] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 4 LEADER]: Signalling peer d7f53a4ef21a4d8788fae6df6bb5f98c to start an election
I20260709 12:16:02.820855  5486 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1126f26405b742a6a136656ca8c86696"
dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
 from {username='slave'} at 127.0.0.1:55134
I20260709 12:16:02.821228  5486 raft_consensus.cc:493] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:02.821439  5486 raft_consensus.cc:3060] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:16:02.826959  5486 raft_consensus.cc:515] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:16:02.828563  5486 leader_election.cc:290] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 5 election: Requested vote from peers 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2:36305), 92adb80abb834eceaef035f370597703 (127.5.58.3:41427)
I20260709 12:16:02.829370  5561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1126f26405b742a6a136656ca8c86696" candidate_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
I20260709 12:16:02.829397  5636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1126f26405b742a6a136656ca8c86696" candidate_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "92adb80abb834eceaef035f370597703"
I20260709 12:16:02.830065  5561 raft_consensus.cc:3055] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:16:02.830077  5636 raft_consensus.cc:3060] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:16:02.830482  5561 raft_consensus.cc:740] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3ea2ad5b842b4bf3868b4e25cae3e9f1, State: Running, Role: LEADER
I20260709 12:16:02.831224  5561 consensus_queue.cc:260] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:16:02.832309  5561 raft_consensus.cc:3060] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:16:02.836156  5636 raft_consensus.cc:2468] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f53a4ef21a4d8788fae6df6bb5f98c in term 5.
I20260709 12:16:02.837332  5450 leader_election.cc:304] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92adb80abb834eceaef035f370597703, d7f53a4ef21a4d8788fae6df6bb5f98c; no voters: 
I20260709 12:16:02.837795  5561 raft_consensus.cc:2468] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f53a4ef21a4d8788fae6df6bb5f98c in term 5.
I20260709 12:16:02.838079  5837 raft_consensus.cc:2804] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:16:02.838593  5837 raft_consensus.cc:697] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 5 LEADER]: Becoming Leader. State: Replica: d7f53a4ef21a4d8788fae6df6bb5f98c, State: Running, Role: LEADER
I20260709 12:16:02.839440  5837 consensus_queue.cc:237] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:16:02.847071  5383 catalog_manager.cc:5697] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c reported cstate change: term changed from 4 to 5, leader changed from 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2) to d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1). New cstate: current_term: 5 leader_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:02.915726  5833 raft_consensus.cc:993] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1: : Instructing follower d7f53a4ef21a4d8788fae6df6bb5f98c to start an election
I20260709 12:16:02.916178  5835 raft_consensus.cc:1081] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 3 LEADER]: Signalling peer d7f53a4ef21a4d8788fae6df6bb5f98c to start an election
I20260709 12:16:02.917613  5486 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a89ab5b73a4a42f7b8b529758c61edf7"
dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
 from {username='slave'} at 127.0.0.1:55134
I20260709 12:16:02.918089  5486 raft_consensus.cc:493] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:02.918334  5486 raft_consensus.cc:3060] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:02.922278  5486 raft_consensus.cc:515] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } }
I20260709 12:16:02.923736  5486 leader_election.cc:290] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 4 election: Requested vote from peers 92adb80abb834eceaef035f370597703 (127.5.58.3:41427), 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2:36305)
I20260709 12:16:02.924645  5636 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a89ab5b73a4a42f7b8b529758c61edf7" candidate_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "92adb80abb834eceaef035f370597703"
I20260709 12:16:02.924798  5561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a89ab5b73a4a42f7b8b529758c61edf7" candidate_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
I20260709 12:16:02.925168  5636 raft_consensus.cc:3060] T a89ab5b73a4a42f7b8b529758c61edf7 P 92adb80abb834eceaef035f370597703 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:02.925200  5561 raft_consensus.cc:3055] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:16:02.925666  5561 raft_consensus.cc:740] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3ea2ad5b842b4bf3868b4e25cae3e9f1, State: Running, Role: LEADER
I20260709 12:16:02.926417  5561 consensus_queue.cc:260] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } }
I20260709 12:16:02.927506  5561 raft_consensus.cc:3060] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:02.931411  5636 raft_consensus.cc:2468] T a89ab5b73a4a42f7b8b529758c61edf7 P 92adb80abb834eceaef035f370597703 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f53a4ef21a4d8788fae6df6bb5f98c in term 4.
I20260709 12:16:02.932426  5450 leader_election.cc:304] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92adb80abb834eceaef035f370597703, d7f53a4ef21a4d8788fae6df6bb5f98c; no voters: 
I20260709 12:16:02.933109  5837 raft_consensus.cc:2804] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:16:02.933693  5837 raft_consensus.cc:697] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 4 LEADER]: Becoming Leader. State: Replica: d7f53a4ef21a4d8788fae6df6bb5f98c, State: Running, Role: LEADER
I20260709 12:16:02.933760  5561 raft_consensus.cc:2468] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7f53a4ef21a4d8788fae6df6bb5f98c in term 4.
I20260709 12:16:02.935325  5837 consensus_queue.cc:237] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } }
I20260709 12:16:02.942837  5383 catalog_manager.cc:5697] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c reported cstate change: term changed from 3 to 4, leader changed from 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2) to d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1). New cstate: current_term: 4 leader_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:03.127776  5636 raft_consensus.cc:1275] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [term 3 FOLLOWER]: Refusing update from remote peer d7f53a4ef21a4d8788fae6df6bb5f98c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:03.129163  5837 consensus_queue.cc:1048] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c [LEADER]: Connected to new peer: Peer: permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:03.143113  5561 raft_consensus.cc:1275] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 3 FOLLOWER]: Refusing update from remote peer d7f53a4ef21a4d8788fae6df6bb5f98c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:03.144987  5838 consensus_queue.cc:1048] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:03.191035  5833 raft_consensus.cc:993] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1: : Instructing follower 92adb80abb834eceaef035f370597703 to start an election
I20260709 12:16:03.191437  5833 raft_consensus.cc:1081] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 LEADER]: Signalling peer 92adb80abb834eceaef035f370597703 to start an election
I20260709 12:16:03.193378  5636 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "09cfb685589d470ab66f97d1d81df47c"
dest_uuid: "92adb80abb834eceaef035f370597703"
 from {username='slave'} at 127.0.0.1:44078
I20260709 12:16:03.193856  5636 raft_consensus.cc:493] T 09cfb685589d470ab66f97d1d81df47c P 92adb80abb834eceaef035f370597703 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:03.194231  5636 raft_consensus.cc:3060] T 09cfb685589d470ab66f97d1d81df47c P 92adb80abb834eceaef035f370597703 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:03.198316  5636 raft_consensus.cc:515] T 09cfb685589d470ab66f97d1d81df47c P 92adb80abb834eceaef035f370597703 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:16:03.199746  5636 leader_election.cc:290] T 09cfb685589d470ab66f97d1d81df47c P 92adb80abb834eceaef035f370597703 [CANDIDATE]: Term 3 election: Requested vote from peers d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021), 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2:36305)
I20260709 12:16:03.200780  5486 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09cfb685589d470ab66f97d1d81df47c" candidate_uuid: "92adb80abb834eceaef035f370597703" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c"
I20260709 12:16:03.200995  5561 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09cfb685589d470ab66f97d1d81df47c" candidate_uuid: "92adb80abb834eceaef035f370597703" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1"
I20260709 12:16:03.201272  5486 raft_consensus.cc:3060] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:03.201572  5561 raft_consensus.cc:3055] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:03.201864  5561 raft_consensus.cc:740] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ea2ad5b842b4bf3868b4e25cae3e9f1, State: Running, Role: LEADER
I20260709 12:16:03.202592  5561 consensus_queue.cc:260] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:16:03.203612  5561 raft_consensus.cc:3060] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:03.206634  5486 raft_consensus.cc:2468] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92adb80abb834eceaef035f370597703 in term 3.
I20260709 12:16:03.207243  5487 raft_consensus.cc:1240] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [term 3 FOLLOWER]: Rejecting Update request from peer 3ea2ad5b842b4bf3868b4e25cae3e9f1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:16:03.207564  5601 leader_election.cc:304] T 09cfb685589d470ab66f97d1d81df47c P 92adb80abb834eceaef035f370597703 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92adb80abb834eceaef035f370597703, d7f53a4ef21a4d8788fae6df6bb5f98c; no voters: 
I20260709 12:16:03.208499  5806 raft_consensus.cc:2804] T 09cfb685589d470ab66f97d1d81df47c P 92adb80abb834eceaef035f370597703 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:03.209023  5806 raft_consensus.cc:697] T 09cfb685589d470ab66f97d1d81df47c P 92adb80abb834eceaef035f370597703 [term 3 LEADER]: Becoming Leader. State: Replica: 92adb80abb834eceaef035f370597703, State: Running, Role: LEADER
I20260709 12:16:03.209977  5806 consensus_queue.cc:237] T 09cfb685589d470ab66f97d1d81df47c P 92adb80abb834eceaef035f370597703 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } }
I20260709 12:16:03.210556  5561 raft_consensus.cc:2468] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92adb80abb834eceaef035f370597703 in term 3.
I20260709 12:16:03.216868  5383 catalog_manager.cc:5697] T 09cfb685589d470ab66f97d1d81df47c P 92adb80abb834eceaef035f370597703 reported cstate change: term changed from 2 to 3, leader changed from 3ea2ad5b842b4bf3868b4e25cae3e9f1 (127.5.58.2) to 92adb80abb834eceaef035f370597703 (127.5.58.3). New cstate: current_term: 3 leader_uuid: "92adb80abb834eceaef035f370597703" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:03.235139  5487 raft_consensus.cc:1275] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 3 FOLLOWER]: Refusing update from remote peer 92adb80abb834eceaef035f370597703: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:03.236380  5806 consensus_queue.cc:1048] T f0f1dad5e1c540edbbba5708a7698085 P 92adb80abb834eceaef035f370597703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:03.245661  5561 raft_consensus.cc:1275] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 3 FOLLOWER]: Refusing update from remote peer 92adb80abb834eceaef035f370597703: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:03.247396  5844 consensus_queue.cc:1048] T f0f1dad5e1c540edbbba5708a7698085 P 92adb80abb834eceaef035f370597703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:03.336887  5561 raft_consensus.cc:1275] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 4 FOLLOWER]: Refusing update from remote peer d7f53a4ef21a4d8788fae6df6bb5f98c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:03.338244  5851 consensus_queue.cc:1048] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:03.347209  5636 raft_consensus.cc:1275] T a89ab5b73a4a42f7b8b529758c61edf7 P 92adb80abb834eceaef035f370597703 [term 4 FOLLOWER]: Refusing update from remote peer d7f53a4ef21a4d8788fae6df6bb5f98c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:03.349350  5838 consensus_queue.cc:1048] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [LEADER]: Connected to new peer: Peer: permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:03.377344  5636 raft_consensus.cc:1275] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 5 FOLLOWER]: Refusing update from remote peer d7f53a4ef21a4d8788fae6df6bb5f98c: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:16:03.378450  5838 consensus_queue.cc:1048] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c [LEADER]: Connected to new peer: Peer: permanent_uuid: "92adb80abb834eceaef035f370597703" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 41427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:03.388641  5561 raft_consensus.cc:1275] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 5 FOLLOWER]: Refusing update from remote peer d7f53a4ef21a4d8788fae6df6bb5f98c: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:16:03.390897  5851 consensus_queue.cc:1048] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:03.692162  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:03.697616  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:03.698079  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:03.736864  5487 raft_consensus.cc:1275] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [term 3 FOLLOWER]: Refusing update from remote peer 92adb80abb834eceaef035f370597703: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:03.738097  5844 consensus_queue.cc:1048] T 09cfb685589d470ab66f97d1d81df47c P 92adb80abb834eceaef035f370597703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7f53a4ef21a4d8788fae6df6bb5f98c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:03.747273  5561 raft_consensus.cc:1275] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 3 FOLLOWER]: Refusing update from remote peer 92adb80abb834eceaef035f370597703: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:03.748904  5856 consensus_queue.cc:1048] T 09cfb685589d470ab66f97d1d81df47c P 92adb80abb834eceaef035f370597703 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea2ad5b842b4bf3868b4e25cae3e9f1" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 36305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:04.699230  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:04.704716  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:04.705093  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:05.706238  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:05.711959  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:05.712381  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:06.713438  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:06.718529  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:06.718880  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:07.719900  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:07.725306  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:07.725678  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:08.726783  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:08.731391  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:08.731737  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:09.732748  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:09.737325  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:09.737672  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:10.738651  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:10.743412  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:10.743796  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:11.745116  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:11.750188  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:11.750542  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:12.785390  5352 auto_leader_rebalancer-test.cc:340] The leader distribution is 
I20260709 12:16:12.786279  5352 tablet_server.cc:179] TabletServer@127.5.58.1:0 shutting down...
I20260709 12:16:12.814805  5352 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:12.815651  5352 tablet_replica.cc:333] T f66af11cf9374a31b237a393194eccae P d7f53a4ef21a4d8788fae6df6bb5f98c: stopping tablet replica
I20260709 12:16:12.816504  5352 raft_consensus.cc:2243] T f66af11cf9374a31b237a393194eccae P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:12.817286  5352 raft_consensus.cc:2272] T f66af11cf9374a31b237a393194eccae P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:12.819876  5352 tablet_replica.cc:333] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c: stopping tablet replica
I20260709 12:16:12.820544  5352 raft_consensus.cc:2243] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:16:12.821795  5352 raft_consensus.cc:2272] T a00c228a31d744968a6b32e3601d51eb P d7f53a4ef21a4d8788fae6df6bb5f98c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:12.823804  5352 tablet_replica.cc:333] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c: stopping tablet replica
I20260709 12:16:12.824546  5352 raft_consensus.cc:2243] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:12.825030  5352 raft_consensus.cc:2272] T 09cfb685589d470ab66f97d1d81df47c P d7f53a4ef21a4d8788fae6df6bb5f98c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:12.826864  5352 tablet_replica.cc:333] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c: stopping tablet replica
I20260709 12:16:12.827394  5352 raft_consensus.cc:2243] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:12.827862  5352 raft_consensus.cc:2272] T f0f1dad5e1c540edbbba5708a7698085 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:12.829595  5352 tablet_replica.cc:333] T 62d225e09ba14ac4bd3bcc68a7b3495f P d7f53a4ef21a4d8788fae6df6bb5f98c: stopping tablet replica
I20260709 12:16:12.830086  5352 raft_consensus.cc:2243] T 62d225e09ba14ac4bd3bcc68a7b3495f P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:12.830668  5352 raft_consensus.cc:2272] T 62d225e09ba14ac4bd3bcc68a7b3495f P d7f53a4ef21a4d8788fae6df6bb5f98c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:12.832523  5352 tablet_replica.cc:333] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c: stopping tablet replica
I20260709 12:16:12.833046  5352 raft_consensus.cc:2243] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:16:12.833832  5352 raft_consensus.cc:2272] T 1126f26405b742a6a136656ca8c86696 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:12.835436  5352 tablet_replica.cc:333] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c: stopping tablet replica
I20260709 12:16:12.835907  5352 raft_consensus.cc:2243] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:16:12.836771  5352 raft_consensus.cc:2272] T a89ab5b73a4a42f7b8b529758c61edf7 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:12.838491  5352 tablet_replica.cc:333] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c: stopping tablet replica
I20260709 12:16:12.838891  5352 raft_consensus.cc:2243] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:12.839243  5352 raft_consensus.cc:2272] T 930c10c719284908b31b1b583ac8bc92 P d7f53a4ef21a4d8788fae6df6bb5f98c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:12.840781  5352 tablet_replica.cc:333] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c: stopping tablet replica
I20260709 12:16:12.841182  5352 raft_consensus.cc:2243] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:12.841677  5352 raft_consensus.cc:2272] T 4fd3cf9b75144e5c88eee1a8917e283e P d7f53a4ef21a4d8788fae6df6bb5f98c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:12.935166  5352 tablet_server.cc:196] TabletServer@127.5.58.1:0 shutdown complete.
W20260709 12:16:12.952970  5527 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.58.1:39021: connect: Connection refused (error 111)
I20260709 12:16:12.957374  5352 tablet_server.cc:179] TabletServer@127.5.58.2:0 shutting down...
W20260709 12:16:12.957521  5527 consensus_peers.cc:597] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 -> Peer d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021): Couldn't send request to peer d7f53a4ef21a4d8788fae6df6bb5f98c. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:39021: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:12.973163  5527 consensus_peers.cc:597] T 62d225e09ba14ac4bd3bcc68a7b3495f P 3ea2ad5b842b4bf3868b4e25cae3e9f1 -> Peer d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021): Couldn't send request to peer d7f53a4ef21a4d8788fae6df6bb5f98c. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:39021: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:12.987514  5601 consensus_peers.cc:597] T f0f1dad5e1c540edbbba5708a7698085 P 92adb80abb834eceaef035f370597703 -> Peer d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021): Couldn't send request to peer d7f53a4ef21a4d8788fae6df6bb5f98c. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:39021: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:12.998689  5352 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:12.999493  5352 tablet_replica.cc:333] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1: stopping tablet replica
I20260709 12:16:13.000152  5352 raft_consensus.cc:2243] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:13.000910  5352 raft_consensus.cc:2272] T f0f1dad5e1c540edbbba5708a7698085 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:13.003160  5352 tablet_replica.cc:333] T 4fd3cf9b75144e5c88eee1a8917e283e P 3ea2ad5b842b4bf3868b4e25cae3e9f1: stopping tablet replica
I20260709 12:16:13.004051  5352 raft_consensus.cc:2243] T 4fd3cf9b75144e5c88eee1a8917e283e P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:13.004889  5352 raft_consensus.cc:2272] T 4fd3cf9b75144e5c88eee1a8917e283e P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:13.006537  5352 tablet_replica.cc:333] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1: stopping tablet replica
I20260709 12:16:13.007097  5352 raft_consensus.cc:2243] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:13.007472  5352 raft_consensus.cc:2272] T a00c228a31d744968a6b32e3601d51eb P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:13.009325  5352 tablet_replica.cc:333] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1: stopping tablet replica
I20260709 12:16:13.009822  5352 raft_consensus.cc:2243] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:13.010231  5352 raft_consensus.cc:2272] T 09cfb685589d470ab66f97d1d81df47c P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:13.011884  5352 tablet_replica.cc:333] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1: stopping tablet replica
I20260709 12:16:13.012454  5352 raft_consensus.cc:2243] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:13.012841  5352 raft_consensus.cc:2272] T f66af11cf9374a31b237a393194eccae P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:13.014348  5352 tablet_replica.cc:333] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1: stopping tablet replica
I20260709 12:16:13.014827  5352 raft_consensus.cc:2243] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:13.015429  5352 raft_consensus.cc:2272] T 1126f26405b742a6a136656ca8c86696 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:13.017069  5352 tablet_replica.cc:333] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1: stopping tablet replica
I20260709 12:16:13.017565  5352 raft_consensus.cc:2243] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:16:13.018409  5352 raft_consensus.cc:2272] T 930c10c719284908b31b1b583ac8bc92 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:13.019956  5352 tablet_replica.cc:333] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1: stopping tablet replica
I20260709 12:16:13.020442  5352 raft_consensus.cc:2243] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:13.020866  5352 raft_consensus.cc:2272] T a89ab5b73a4a42f7b8b529758c61edf7 P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:13.022356  5352 tablet_replica.cc:333] T 62d225e09ba14ac4bd3bcc68a7b3495f P 3ea2ad5b842b4bf3868b4e25cae3e9f1: stopping tablet replica
I20260709 12:16:13.022804  5352 raft_consensus.cc:2243] T 62d225e09ba14ac4bd3bcc68a7b3495f P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:13.023538  5352 raft_consensus.cc:2272] T 62d225e09ba14ac4bd3bcc68a7b3495f P 3ea2ad5b842b4bf3868b4e25cae3e9f1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:13.032619  5601 consensus_peers.cc:597] T 09cfb685589d470ab66f97d1d81df47c P 92adb80abb834eceaef035f370597703 -> Peer d7f53a4ef21a4d8788fae6df6bb5f98c (127.5.58.1:39021): Couldn't send request to peer d7f53a4ef21a4d8788fae6df6bb5f98c. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:39021: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:13.059128  5352 tablet_server.cc:196] TabletServer@127.5.58.2:0 shutdown complete.
I20260709 12:16:13.080101  5352 tablet_server.cc:179] TabletServer@127.5.58.3:0 shutting down...
I20260709 12:16:13.109524  5352 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:13.110198  5352 tablet_replica.cc:333] T f0f1dad5e1c540edbbba5708a7698085 P 92adb80abb834eceaef035f370597703: stopping tablet replica
I20260709 12:16:13.110777  5352 raft_consensus.cc:2243] T f0f1dad5e1c540edbbba5708a7698085 P 92adb80abb834eceaef035f370597703 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:16:13.111568  5352 raft_consensus.cc:2272] T f0f1dad5e1c540edbbba5708a7698085 P 92adb80abb834eceaef035f370597703 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:13.113339  5352 tablet_replica.cc:333] T 62d225e09ba14ac4bd3bcc68a7b3495f P 92adb80abb834eceaef035f370597703: stopping tablet replica
I20260709 12:16:13.113818  5352 raft_consensus.cc:2243] T 62d225e09ba14ac4bd3bcc68a7b3495f P 92adb80abb834eceaef035f370597703 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:13.114200  5352 raft_consensus.cc:2272] T 62d225e09ba14ac4bd3bcc68a7b3495f P 92adb80abb834eceaef035f370597703 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:13.115721  5352 tablet_replica.cc:333] T f66af11cf9374a31b237a393194eccae P 92adb80abb834eceaef035f370597703: stopping tablet replica
I20260709 12:16:13.116187  5352 raft_consensus.cc:2243] T f66af11cf9374a31b237a393194eccae P 92adb80abb834eceaef035f370597703 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:13.117002  5352 raft_consensus.cc:2272] T f66af11cf9374a31b237a393194eccae P 92adb80abb834eceaef035f370597703 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:13.118714  5352 tablet_replica.cc:333] T a89ab5b73a4a42f7b8b529758c61edf7 P 92adb80abb834eceaef035f370597703: stopping tablet replica
I20260709 12:16:13.119187  5352 raft_consensus.cc:2243] T a89ab5b73a4a42f7b8b529758c61edf7 P 92adb80abb834eceaef035f370597703 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:13.119647  5352 raft_consensus.cc:2272] T a89ab5b73a4a42f7b8b529758c61edf7 P 92adb80abb834eceaef035f370597703 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:13.121163  5352 tablet_replica.cc:333] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703: stopping tablet replica
I20260709 12:16:13.121628  5352 raft_consensus.cc:2243] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:13.122038  5352 raft_consensus.cc:2272] T 1126f26405b742a6a136656ca8c86696 P 92adb80abb834eceaef035f370597703 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:13.123483  5352 tablet_replica.cc:333] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703: stopping tablet replica
I20260709 12:16:13.123940  5352 raft_consensus.cc:2243] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:13.124322  5352 raft_consensus.cc:2272] T 930c10c719284908b31b1b583ac8bc92 P 92adb80abb834eceaef035f370597703 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:13.125880  5352 tablet_replica.cc:333] T 4fd3cf9b75144e5c88eee1a8917e283e P 92adb80abb834eceaef035f370597703: stopping tablet replica
I20260709 12:16:13.126340  5352 raft_consensus.cc:2243] T 4fd3cf9b75144e5c88eee1a8917e283e P 92adb80abb834eceaef035f370597703 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:13.126735  5352 raft_consensus.cc:2272] T 4fd3cf9b75144e5c88eee1a8917e283e P 92adb80abb834eceaef035f370597703 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:13.128378  5352 tablet_replica.cc:333] T 09cfb685589d470ab66f97d1d81df47c P 92adb80abb834eceaef035f370597703: stopping tablet replica
I20260709 12:16:13.128867  5352 raft_consensus.cc:2243] T 09cfb685589d470ab66f97d1d81df47c P 92adb80abb834eceaef035f370597703 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:16:13.129737  5352 raft_consensus.cc:2272] T 09cfb685589d470ab66f97d1d81df47c P 92adb80abb834eceaef035f370597703 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:13.131274  5352 tablet_replica.cc:333] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703: stopping tablet replica
I20260709 12:16:13.131724  5352 raft_consensus.cc:2243] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:13.132148  5352 raft_consensus.cc:2272] T a00c228a31d744968a6b32e3601d51eb P 92adb80abb834eceaef035f370597703 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:13.162211  5352 tablet_server.cc:196] TabletServer@127.5.58.3:0 shutdown complete.
I20260709 12:16:13.181504  5352 master.cc:562] Master@127.5.58.62:45205 shutting down...
I20260709 12:16:13.391258  5352 raft_consensus.cc:2243] T 00000000000000000000000000000000 P a585945ae7a64ca495b67d749f3374bf [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:13.391862  5352 raft_consensus.cc:2272] T 00000000000000000000000000000000 P a585945ae7a64ca495b67d749f3374bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:13.392167  5352 tablet_replica.cc:333] T 00000000000000000000000000000000 P a585945ae7a64ca495b67d749f3374bf: stopping tablet replica
I20260709 12:16:13.413322  5352 master.cc:584] Master@127.5.58.62:45205 shutdown complete.
3ea2ad5b842b4bf3868b4e25cae3e9f1  4
92adb80abb834eceaef035f370597703  1
d7f53a4ef21a4d8788fae6df6bb5f98c  4
3ea2ad5b842b4bf3868b4e25cae3e9f1  3
92adb80abb834eceaef035f370597703  3
d7f53a4ef21a4d8788fae6df6bb5f98c  3
3ea2ad5b842b4bf3868b4e25cae3e9f1  8
92adb80abb834eceaef035f370597703  1
d7f53a4ef21a4d8788fae6df6bb5f98c  0
3ea2ad5b842b4bf3868b4e25cae3e9f1  3
92adb80abb834eceaef035f370597703  3
d7f53a4ef21a4d8788fae6df6bb5f98c  3
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (31428 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 12:16:13.461880  5352 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.58.62:43011
I20260709 12:16:13.463037  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:13.468469  5874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:13.469944  5875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:13.470384  5877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:13.471187  5352 server_base.cc:1061] running on GCE node
I20260709 12:16:13.472332  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:13.472626  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:13.472808  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599373472792 us; error 0 us; skew 500 ppm
I20260709 12:16:13.473371  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:13.476068  5352 webserver.cc:533] Webserver started at http://127.5.58.62:40287/ using document root <none> and password file <none>
I20260709 12:16:13.476646  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:13.476861  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:13.477141  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:13.478303  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599341860723-5352-0/minicluster-data/master-0-root/instance:
uuid: "4ba9cab5cfc54b7b92e51215c4d90ac2"
format_stamp: "Formatted at 2026-07-09 12:16:13 on dist-test-slave-xs2d"
I20260709 12:16:13.482656  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:16:13.486003  5882 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:13.486876  5352 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:16:13.487167  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599341860723-5352-0/minicluster-data/master-0-root
uuid: "4ba9cab5cfc54b7b92e51215c4d90ac2"
format_stamp: "Formatted at 2026-07-09 12:16:13 on dist-test-slave-xs2d"
I20260709 12:16:13.487406  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599341860723-5352-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599341860723-5352-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599341860723-5352-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:13.517977  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:13.519167  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:13.560640  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.62:43011
I20260709 12:16:13.560776  5933 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.62:43011 every 8 connection(s)
I20260709 12:16:13.564776  5934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:13.574483  5934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ba9cab5cfc54b7b92e51215c4d90ac2: Bootstrap starting.
I20260709 12:16:13.579596  5934 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ba9cab5cfc54b7b92e51215c4d90ac2: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:13.584424  5934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ba9cab5cfc54b7b92e51215c4d90ac2: No bootstrap required, opened a new log
I20260709 12:16:13.587473  5934 raft_consensus.cc:359] T 00000000000000000000000000000000 P 4ba9cab5cfc54b7b92e51215c4d90ac2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba9cab5cfc54b7b92e51215c4d90ac2" member_type: VOTER }
I20260709 12:16:13.588310  5934 raft_consensus.cc:385] T 00000000000000000000000000000000 P 4ba9cab5cfc54b7b92e51215c4d90ac2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:13.588686  5934 raft_consensus.cc:740] T 00000000000000000000000000000000 P 4ba9cab5cfc54b7b92e51215c4d90ac2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ba9cab5cfc54b7b92e51215c4d90ac2, State: Initialized, Role: FOLLOWER
I20260709 12:16:13.589433  5934 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ba9cab5cfc54b7b92e51215c4d90ac2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba9cab5cfc54b7b92e51215c4d90ac2" member_type: VOTER }
I20260709 12:16:13.589944  5934 raft_consensus.cc:399] T 00000000000000000000000000000000 P 4ba9cab5cfc54b7b92e51215c4d90ac2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:16:13.590140  5934 raft_consensus.cc:493] T 00000000000000000000000000000000 P 4ba9cab5cfc54b7b92e51215c4d90ac2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:16:13.590435  5934 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 4ba9cab5cfc54b7b92e51215c4d90ac2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:13.594482  5934 raft_consensus.cc:515] T 00000000000000000000000000000000 P 4ba9cab5cfc54b7b92e51215c4d90ac2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba9cab5cfc54b7b92e51215c4d90ac2" member_type: VOTER }
I20260709 12:16:13.595064  5934 leader_election.cc:304] T 00000000000000000000000000000000 P 4ba9cab5cfc54b7b92e51215c4d90ac2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ba9cab5cfc54b7b92e51215c4d90ac2; no voters: 
I20260709 12:16:13.596242  5934 leader_election.cc:290] T 00000000000000000000000000000000 P 4ba9cab5cfc54b7b92e51215c4d90ac2 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:16:13.596802  5937 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 4ba9cab5cfc54b7b92e51215c4d90ac2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:13.598359  5937 raft_consensus.cc:697] T 00000000000000000000000000000000 P 4ba9cab5cfc54b7b92e51215c4d90ac2 [term 1 LEADER]: Becoming Leader. State: Replica: 4ba9cab5cfc54b7b92e51215c4d90ac2, State: Running, Role: LEADER
I20260709 12:16:13.599079  5934 sys_catalog.cc:565] T 00000000000000000000000000000000 P 4ba9cab5cfc54b7b92e51215c4d90ac2 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:16:13.599113  5937 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ba9cab5cfc54b7b92e51215c4d90ac2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba9cab5cfc54b7b92e51215c4d90ac2" member_type: VOTER }
I20260709 12:16:13.605598  5939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ba9cab5cfc54b7b92e51215c4d90ac2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ba9cab5cfc54b7b92e51215c4d90ac2. Latest consensus state: current_term: 1 leader_uuid: "4ba9cab5cfc54b7b92e51215c4d90ac2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba9cab5cfc54b7b92e51215c4d90ac2" member_type: VOTER } }
I20260709 12:16:13.605720  5938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ba9cab5cfc54b7b92e51215c4d90ac2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ba9cab5cfc54b7b92e51215c4d90ac2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba9cab5cfc54b7b92e51215c4d90ac2" member_type: VOTER } }
I20260709 12:16:13.606693  5938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ba9cab5cfc54b7b92e51215c4d90ac2 [sys.catalog]: This master's current role is: LEADER
I20260709 12:16:13.606642  5939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ba9cab5cfc54b7b92e51215c4d90ac2 [sys.catalog]: This master's current role is: LEADER
I20260709 12:16:13.610841  5945 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:16:13.617262  5945 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:16:13.618275  5352 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:16:13.627259  5945 catalog_manager.cc:1383] Generated new cluster ID: 3f48b8edcf03406eae773b60a4d1ec1a
I20260709 12:16:13.627626  5945 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:16:13.677840  5945 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:16:13.679189  5945 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:16:13.696952  5945 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 4ba9cab5cfc54b7b92e51215c4d90ac2: Generated new TSK 0
I20260709 12:16:13.697566  5945 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:16:13.750360  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:13.756863  5955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:13.758363  5956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:13.759640  5958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:13.760553  5352 server_base.cc:1061] running on GCE node
I20260709 12:16:13.761610  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:13.761813  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:13.761938  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599373761926 us; error 0 us; skew 500 ppm
I20260709 12:16:13.762509  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:13.764994  5352 webserver.cc:533] Webserver started at http://127.5.58.1:39663/ using document root <none> and password file <none>
I20260709 12:16:13.765455  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:13.765614  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:13.765836  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:13.766814  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599341860723-5352-0/minicluster-data/ts-0-root/instance:
uuid: "bd8c877d092949e693c13e9103581f82"
format_stamp: "Formatted at 2026-07-09 12:16:13 on dist-test-slave-xs2d"
I20260709 12:16:13.771186  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:16:13.774536  5963 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:13.775333  5352 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:16:13.775614  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599341860723-5352-0/minicluster-data/ts-0-root
uuid: "bd8c877d092949e693c13e9103581f82"
format_stamp: "Formatted at 2026-07-09 12:16:13 on dist-test-slave-xs2d"
I20260709 12:16:13.775947  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599341860723-5352-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599341860723-5352-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599341860723-5352-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:13.798867  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:13.800241  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:13.802017  5352 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:13.804520  5352 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:13.804760  5352 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:13.805028  5352 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:13.805243  5352 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:13.847517  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.1:35557
I20260709 12:16:13.847610  6025 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.1:35557 every 8 connection(s)
I20260709 12:16:13.852985  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:13.860706  6030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:13.862762  6031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:13.863715  6033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:13.865136  5352 server_base.cc:1061] running on GCE node
I20260709 12:16:13.866902  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:13.867152  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:13.867426  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599373867408 us; error 0 us; skew 500 ppm
I20260709 12:16:13.868248  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:13.870186  6026 heartbeater.cc:344] Connected to a master server at 127.5.58.62:43011
I20260709 12:16:13.870585  6026 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:13.871048  5352 webserver.cc:533] Webserver started at http://127.5.58.2:41093/ using document root <none> and password file <none>
I20260709 12:16:13.871538  6026 heartbeater.cc:507] Master 127.5.58.62:43011 requested a full tablet report, sending...
I20260709 12:16:13.871665  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:13.871964  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:13.872293  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:13.873692  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599341860723-5352-0/minicluster-data/ts-1-root/instance:
uuid: "0ffd67a5e76e46a7b0089718369740bc"
format_stamp: "Formatted at 2026-07-09 12:16:13 on dist-test-slave-xs2d"
I20260709 12:16:13.874193  5899 ts_manager.cc:194] Registered new tserver with Master: bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557)
I20260709 12:16:13.876654  5899 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41370
I20260709 12:16:13.879588  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:16:13.883067  6038 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:13.884019  5352 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:16:13.884385  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599341860723-5352-0/minicluster-data/ts-1-root
uuid: "0ffd67a5e76e46a7b0089718369740bc"
format_stamp: "Formatted at 2026-07-09 12:16:13 on dist-test-slave-xs2d"
I20260709 12:16:13.884665  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599341860723-5352-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599341860723-5352-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599341860723-5352-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:13.896035  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:13.897353  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:13.898798  5352 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:13.900979  5352 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:13.901239  5352 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:13.901546  5352 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:13.901685  5352 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:13.945240  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.2:38019
I20260709 12:16:13.945338  6100 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.2:38019 every 8 connection(s)
I20260709 12:16:13.951061  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:13.958750  6104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:13.959978  6105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:13.962543  6101 heartbeater.cc:344] Connected to a master server at 127.5.58.62:43011
I20260709 12:16:13.962982  6101 heartbeater.cc:461] Registering TS with master...
W20260709 12:16:13.963160  6107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:13.963882  6101 heartbeater.cc:507] Master 127.5.58.62:43011 requested a full tablet report, sending...
I20260709 12:16:13.964057  5352 server_base.cc:1061] running on GCE node
I20260709 12:16:13.964946  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:13.965171  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:13.965375  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599373965356 us; error 0 us; skew 500 ppm
I20260709 12:16:13.965948  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:13.966241  5899 ts_manager.cc:194] Registered new tserver with Master: 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019)
I20260709 12:16:13.968019  5899 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41376
I20260709 12:16:13.968863  5352 webserver.cc:533] Webserver started at http://127.5.58.3:46531/ using document root <none> and password file <none>
I20260709 12:16:13.969470  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:13.969663  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:13.969939  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:13.971078  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599341860723-5352-0/minicluster-data/ts-2-root/instance:
uuid: "9468a996c51241cab3d90392e5864642"
format_stamp: "Formatted at 2026-07-09 12:16:13 on dist-test-slave-xs2d"
I20260709 12:16:13.975975  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.003s
I20260709 12:16:13.979615  6112 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:13.980690  5352 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:16:13.980994  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599341860723-5352-0/minicluster-data/ts-2-root
uuid: "9468a996c51241cab3d90392e5864642"
format_stamp: "Formatted at 2026-07-09 12:16:13 on dist-test-slave-xs2d"
I20260709 12:16:13.981303  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599341860723-5352-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599341860723-5352-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599341860723-5352-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:14.007911  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:14.009337  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:14.010630  5352 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:14.013036  5352 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:14.013293  5352 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:14.013542  5352 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:14.013724  5352 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:14.058614  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.3:44821
I20260709 12:16:14.058763  6174 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.3:44821 every 8 connection(s)
I20260709 12:16:14.072978  6175 heartbeater.cc:344] Connected to a master server at 127.5.58.62:43011
I20260709 12:16:14.073343  6175 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:14.074118  6175 heartbeater.cc:507] Master 127.5.58.62:43011 requested a full tablet report, sending...
I20260709 12:16:14.076030  5899 ts_manager.cc:194] Registered new tserver with Master: 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:14.076511  5352 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014713766s
I20260709 12:16:14.077399  5899 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41380
I20260709 12:16:14.103518  5899 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41390:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:16:14.106506  5899 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:16:14.185222  6065 tablet_service.cc:1511] Processing CreateTablet for tablet cb9f68ad1dc24fc29ed8ad07197a68ac (DEFAULT_TABLE table=test-workload [id=6d51fac7588f4d68a3824c437f8973b2]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:16:14.186293  6063 tablet_service.cc:1511] Processing CreateTablet for tablet a275014900ca4f32977e32bb6fc509fd (DEFAULT_TABLE table=test-workload [id=6d51fac7588f4d68a3824c437f8973b2]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:16:14.186786  6065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb9f68ad1dc24fc29ed8ad07197a68ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.187635  6063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a275014900ca4f32977e32bb6fc509fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.188529  6066 tablet_service.cc:1511] Processing CreateTablet for tablet 46526944508c443c9c12315136ae866f (DEFAULT_TABLE table=test-workload [id=6d51fac7588f4d68a3824c437f8973b2]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:16:14.189913  6066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46526944508c443c9c12315136ae866f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.190675  6062 tablet_service.cc:1511] Processing CreateTablet for tablet b40ad7108aec4ad2a5aa42acd343c6ac (DEFAULT_TABLE table=test-workload [id=6d51fac7588f4d68a3824c437f8973b2]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:16:14.192014  6062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b40ad7108aec4ad2a5aa42acd343c6ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.185222  6064 tablet_service.cc:1511] Processing CreateTablet for tablet 71c10f56e0ac401b875d85211fc6ae9f (DEFAULT_TABLE table=test-workload [id=6d51fac7588f4d68a3824c437f8973b2]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:16:14.193554  6061 tablet_service.cc:1511] Processing CreateTablet for tablet 8ff9114868e646a4967bf3e65485393c (DEFAULT_TABLE table=test-workload [id=6d51fac7588f4d68a3824c437f8973b2]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:16:14.196290  6060 tablet_service.cc:1511] Processing CreateTablet for tablet 3f858fa3ce674075aabe68fc8228266d (DEFAULT_TABLE table=test-workload [id=6d51fac7588f4d68a3824c437f8973b2]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:16:14.198318  6064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71c10f56e0ac401b875d85211fc6ae9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.200696  6140 tablet_service.cc:1511] Processing CreateTablet for tablet 46526944508c443c9c12315136ae866f (DEFAULT_TABLE table=test-workload [id=6d51fac7588f4d68a3824c437f8973b2]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:16:14.200977  6059 tablet_service.cc:1511] Processing CreateTablet for tablet 800b87c938ae4f1596d89f00bb34399d (DEFAULT_TABLE table=test-workload [id=6d51fac7588f4d68a3824c437f8973b2]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:16:14.202214  6140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46526944508c443c9c12315136ae866f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.201802  6139 tablet_service.cc:1511] Processing CreateTablet for tablet cb9f68ad1dc24fc29ed8ad07197a68ac (DEFAULT_TABLE table=test-workload [id=6d51fac7588f4d68a3824c437f8973b2]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:16:14.203305  6139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb9f68ad1dc24fc29ed8ad07197a68ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.204910  6137 tablet_service.cc:1511] Processing CreateTablet for tablet a275014900ca4f32977e32bb6fc509fd (DEFAULT_TABLE table=test-workload [id=6d51fac7588f4d68a3824c437f8973b2]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:16:14.206418  6137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a275014900ca4f32977e32bb6fc509fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.209416  6138 tablet_service.cc:1511] Processing CreateTablet for tablet 71c10f56e0ac401b875d85211fc6ae9f (DEFAULT_TABLE table=test-workload [id=6d51fac7588f4d68a3824c437f8973b2]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:16:14.210897  6138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71c10f56e0ac401b875d85211fc6ae9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.214466  6058 tablet_service.cc:1511] Processing CreateTablet for tablet a3442b7a72a34bb3971f1dae9e473e38 (DEFAULT_TABLE table=test-workload [id=6d51fac7588f4d68a3824c437f8973b2]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:16:14.214774  6136 tablet_service.cc:1511] Processing CreateTablet for tablet b40ad7108aec4ad2a5aa42acd343c6ac (DEFAULT_TABLE table=test-workload [id=6d51fac7588f4d68a3824c437f8973b2]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:16:14.216089  6135 tablet_service.cc:1511] Processing CreateTablet for tablet 8ff9114868e646a4967bf3e65485393c (DEFAULT_TABLE table=test-workload [id=6d51fac7588f4d68a3824c437f8973b2]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:16:14.217800  6134 tablet_service.cc:1511] Processing CreateTablet for tablet 3f858fa3ce674075aabe68fc8228266d (DEFAULT_TABLE table=test-workload [id=6d51fac7588f4d68a3824c437f8973b2]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:16:14.218400  6132 tablet_service.cc:1511] Processing CreateTablet for tablet a3442b7a72a34bb3971f1dae9e473e38 (DEFAULT_TABLE table=test-workload [id=6d51fac7588f4d68a3824c437f8973b2]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:16:14.220501  6133 tablet_service.cc:1511] Processing CreateTablet for tablet 800b87c938ae4f1596d89f00bb34399d (DEFAULT_TABLE table=test-workload [id=6d51fac7588f4d68a3824c437f8973b2]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:16:14.221649  6057 tablet_service.cc:1511] Processing CreateTablet for tablet d1ec21d99cc2496d84a013cadd31fbc6 (DEFAULT_TABLE table=test-workload [id=6d51fac7588f4d68a3824c437f8973b2]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:16:14.223158  6131 tablet_service.cc:1511] Processing CreateTablet for tablet d1ec21d99cc2496d84a013cadd31fbc6 (DEFAULT_TABLE table=test-workload [id=6d51fac7588f4d68a3824c437f8973b2]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:16:14.224713  6059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 800b87c938ae4f1596d89f00bb34399d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.225446  6061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ff9114868e646a4967bf3e65485393c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.226218  6058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3442b7a72a34bb3971f1dae9e473e38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.226866  6060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f858fa3ce674075aabe68fc8228266d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.216203  6136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b40ad7108aec4ad2a5aa42acd343c6ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.236721  6131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1ec21d99cc2496d84a013cadd31fbc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.240839  6133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 800b87c938ae4f1596d89f00bb34399d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.244861  6132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3442b7a72a34bb3971f1dae9e473e38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.248903  6135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ff9114868e646a4967bf3e65485393c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.256865  6057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1ec21d99cc2496d84a013cadd31fbc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.265399  6134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f858fa3ce674075aabe68fc8228266d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.329623  5989 tablet_service.cc:1511] Processing CreateTablet for tablet 71c10f56e0ac401b875d85211fc6ae9f (DEFAULT_TABLE table=test-workload [id=6d51fac7588f4d68a3824c437f8973b2]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:16:14.331125  5989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71c10f56e0ac401b875d85211fc6ae9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.330998  5991 tablet_service.cc:1511] Processing CreateTablet for tablet 46526944508c443c9c12315136ae866f (DEFAULT_TABLE table=test-workload [id=6d51fac7588f4d68a3824c437f8973b2]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:16:14.332420  5991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46526944508c443c9c12315136ae866f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.333763  6190 tablet_bootstrap.cc:492] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642: Bootstrap starting.
I20260709 12:16:14.333890  5986 tablet_service.cc:1511] Processing CreateTablet for tablet 8ff9114868e646a4967bf3e65485393c (DEFAULT_TABLE table=test-workload [id=6d51fac7588f4d68a3824c437f8973b2]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:16:14.336473  6191 tablet_bootstrap.cc:492] T 71c10f56e0ac401b875d85211fc6ae9f P 0ffd67a5e76e46a7b0089718369740bc: Bootstrap starting.
I20260709 12:16:14.336796  5985 tablet_service.cc:1511] Processing CreateTablet for tablet 3f858fa3ce674075aabe68fc8228266d (DEFAULT_TABLE table=test-workload [id=6d51fac7588f4d68a3824c437f8973b2]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:16:14.337033  5983 tablet_service.cc:1511] Processing CreateTablet for tablet a3442b7a72a34bb3971f1dae9e473e38 (DEFAULT_TABLE table=test-workload [id=6d51fac7588f4d68a3824c437f8973b2]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:16:14.337445  5988 tablet_service.cc:1511] Processing CreateTablet for tablet a275014900ca4f32977e32bb6fc509fd (DEFAULT_TABLE table=test-workload [id=6d51fac7588f4d68a3824c437f8973b2]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:16:14.339138  5990 tablet_service.cc:1511] Processing CreateTablet for tablet cb9f68ad1dc24fc29ed8ad07197a68ac (DEFAULT_TABLE table=test-workload [id=6d51fac7588f4d68a3824c437f8973b2]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:16:14.347259  5987 tablet_service.cc:1511] Processing CreateTablet for tablet b40ad7108aec4ad2a5aa42acd343c6ac (DEFAULT_TABLE table=test-workload [id=6d51fac7588f4d68a3824c437f8973b2]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:16:14.349805  5986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ff9114868e646a4967bf3e65485393c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.350459  5983 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3442b7a72a34bb3971f1dae9e473e38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.345628  5982 tablet_service.cc:1511] Processing CreateTablet for tablet d1ec21d99cc2496d84a013cadd31fbc6 (DEFAULT_TABLE table=test-workload [id=6d51fac7588f4d68a3824c437f8973b2]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:16:14.345633  5984 tablet_service.cc:1511] Processing CreateTablet for tablet 800b87c938ae4f1596d89f00bb34399d (DEFAULT_TABLE table=test-workload [id=6d51fac7588f4d68a3824c437f8973b2]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:16:14.352620  5982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1ec21d99cc2496d84a013cadd31fbc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.353328  5988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a275014900ca4f32977e32bb6fc509fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.353993  5990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb9f68ad1dc24fc29ed8ad07197a68ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.356664  5984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 800b87c938ae4f1596d89f00bb34399d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.361042  5987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b40ad7108aec4ad2a5aa42acd343c6ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.365974  5985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f858fa3ce674075aabe68fc8228266d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:14.366210  6190 tablet_bootstrap.cc:654] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.392856  6191 tablet_bootstrap.cc:654] T 71c10f56e0ac401b875d85211fc6ae9f P 0ffd67a5e76e46a7b0089718369740bc: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.415675  6190 tablet_bootstrap.cc:492] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642: No bootstrap required, opened a new log
I20260709 12:16:14.416188  6190 ts_tablet_manager.cc:1403] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642: Time spent bootstrapping tablet: real 0.083s	user 0.011s	sys 0.003s
I20260709 12:16:14.416819  6193 tablet_bootstrap.cc:492] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82: Bootstrap starting.
I20260709 12:16:14.419107  6190 raft_consensus.cc:359] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.419827  6190 raft_consensus.cc:385] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.420147  6190 raft_consensus.cc:740] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9468a996c51241cab3d90392e5864642, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.421665  6190 consensus_queue.cc:260] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.425266  6191 tablet_bootstrap.cc:492] T 71c10f56e0ac401b875d85211fc6ae9f P 0ffd67a5e76e46a7b0089718369740bc: No bootstrap required, opened a new log
I20260709 12:16:14.425784  6191 ts_tablet_manager.cc:1403] T 71c10f56e0ac401b875d85211fc6ae9f P 0ffd67a5e76e46a7b0089718369740bc: Time spent bootstrapping tablet: real 0.090s	user 0.023s	sys 0.010s
I20260709 12:16:14.430401  6191 raft_consensus.cc:359] T 71c10f56e0ac401b875d85211fc6ae9f P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.432214  6190 ts_tablet_manager.cc:1434] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:16:14.433133  6191 raft_consensus.cc:385] T 71c10f56e0ac401b875d85211fc6ae9f P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.433136  6175 heartbeater.cc:499] Master 127.5.58.62:43011 was elected leader, sending a full tablet report...
I20260709 12:16:14.433912  6190 tablet_bootstrap.cc:492] T d1ec21d99cc2496d84a013cadd31fbc6 P 9468a996c51241cab3d90392e5864642: Bootstrap starting.
I20260709 12:16:14.433804  6191 raft_consensus.cc:740] T 71c10f56e0ac401b875d85211fc6ae9f P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ffd67a5e76e46a7b0089718369740bc, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.434834  6191 consensus_queue.cc:260] T 71c10f56e0ac401b875d85211fc6ae9f P 0ffd67a5e76e46a7b0089718369740bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.432682  6193 tablet_bootstrap.cc:654] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.438025  6195 raft_consensus.cc:493] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.438704  6195 raft_consensus.cc:515] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.447472  6195 leader_election.cc:290] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557), 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019)
I20260709 12:16:14.467373  6191 ts_tablet_manager.cc:1434] T 71c10f56e0ac401b875d85211fc6ae9f P 0ffd67a5e76e46a7b0089718369740bc: Time spent starting tablet: real 0.041s	user 0.017s	sys 0.019s
I20260709 12:16:14.468621  6190 tablet_bootstrap.cc:654] T d1ec21d99cc2496d84a013cadd31fbc6 P 9468a996c51241cab3d90392e5864642: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.468822  6001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71c10f56e0ac401b875d85211fc6ae9f" candidate_uuid: "9468a996c51241cab3d90392e5864642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c877d092949e693c13e9103581f82" is_pre_election: true
I20260709 12:16:14.468199  6101 heartbeater.cc:499] Master 127.5.58.62:43011 was elected leader, sending a full tablet report...
W20260709 12:16:14.472050  6116 leader_election.cc:343] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:14.477957  6191 tablet_bootstrap.cc:492] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc: Bootstrap starting.
I20260709 12:16:14.491338  6191 tablet_bootstrap.cc:654] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.492048  6076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71c10f56e0ac401b875d85211fc6ae9f" candidate_uuid: "9468a996c51241cab3d90392e5864642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffd67a5e76e46a7b0089718369740bc" is_pre_election: true
I20260709 12:16:14.492753  6076 raft_consensus.cc:2468] T 71c10f56e0ac401b875d85211fc6ae9f P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9468a996c51241cab3d90392e5864642 in term 0.
I20260709 12:16:14.494117  6113 leader_election.cc:304] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc, 9468a996c51241cab3d90392e5864642; no voters: bd8c877d092949e693c13e9103581f82
I20260709 12:16:14.495091  6195 raft_consensus.cc:2804] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:14.495743  6195 raft_consensus.cc:493] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:14.496162  6195 raft_consensus.cc:3060] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.498519  6193 tablet_bootstrap.cc:492] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82: No bootstrap required, opened a new log
I20260709 12:16:14.499241  6193 ts_tablet_manager.cc:1403] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82: Time spent bootstrapping tablet: real 0.083s	user 0.013s	sys 0.015s
I20260709 12:16:14.502683  6190 tablet_bootstrap.cc:492] T d1ec21d99cc2496d84a013cadd31fbc6 P 9468a996c51241cab3d90392e5864642: No bootstrap required, opened a new log
I20260709 12:16:14.503352  6190 ts_tablet_manager.cc:1403] T d1ec21d99cc2496d84a013cadd31fbc6 P 9468a996c51241cab3d90392e5864642: Time spent bootstrapping tablet: real 0.070s	user 0.016s	sys 0.000s
I20260709 12:16:14.504349  6195 raft_consensus.cc:515] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.507629  6001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71c10f56e0ac401b875d85211fc6ae9f" candidate_uuid: "9468a996c51241cab3d90392e5864642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c877d092949e693c13e9103581f82"
W20260709 12:16:14.509671  6116 leader_election.cc:343] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:14.510953  6190 raft_consensus.cc:359] T d1ec21d99cc2496d84a013cadd31fbc6 P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.511948  6190 raft_consensus.cc:385] T d1ec21d99cc2496d84a013cadd31fbc6 P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.511507  6193 raft_consensus.cc:359] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.512413  6190 raft_consensus.cc:740] T d1ec21d99cc2496d84a013cadd31fbc6 P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9468a996c51241cab3d90392e5864642, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.512704  6193 raft_consensus.cc:385] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.512467  6076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71c10f56e0ac401b875d85211fc6ae9f" candidate_uuid: "9468a996c51241cab3d90392e5864642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
I20260709 12:16:14.513154  6193 raft_consensus.cc:740] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c877d092949e693c13e9103581f82, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.513644  6076 raft_consensus.cc:3060] T 71c10f56e0ac401b875d85211fc6ae9f P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.513304  6190 consensus_queue.cc:260] T d1ec21d99cc2496d84a013cadd31fbc6 P 9468a996c51241cab3d90392e5864642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.514308  6193 consensus_queue.cc:260] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.511006  6195 leader_election.cc:290] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 1 election: Requested vote from peers bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557), 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019)
I20260709 12:16:14.522219  6191 tablet_bootstrap.cc:492] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc: No bootstrap required, opened a new log
I20260709 12:16:14.522362  6076 raft_consensus.cc:2468] T 71c10f56e0ac401b875d85211fc6ae9f P 0ffd67a5e76e46a7b0089718369740bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9468a996c51241cab3d90392e5864642 in term 1.
I20260709 12:16:14.522845  6191 ts_tablet_manager.cc:1403] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc: Time spent bootstrapping tablet: real 0.045s	user 0.020s	sys 0.012s
I20260709 12:16:14.523502  6190 ts_tablet_manager.cc:1434] T d1ec21d99cc2496d84a013cadd31fbc6 P 9468a996c51241cab3d90392e5864642: Time spent starting tablet: real 0.020s	user 0.017s	sys 0.000s
I20260709 12:16:14.523801  6113 leader_election.cc:304] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc, 9468a996c51241cab3d90392e5864642; no voters: bd8c877d092949e693c13e9103581f82
I20260709 12:16:14.525584  6201 raft_consensus.cc:2804] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:14.526201  6191 raft_consensus.cc:359] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.526983  6191 raft_consensus.cc:385] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.527309  6191 raft_consensus.cc:740] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ffd67a5e76e46a7b0089718369740bc, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.527987  6190 tablet_bootstrap.cc:492] T a275014900ca4f32977e32bb6fc509fd P 9468a996c51241cab3d90392e5864642: Bootstrap starting.
I20260709 12:16:14.528092  6191 consensus_queue.cc:260] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.530529  6191 ts_tablet_manager.cc:1434] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:16:14.531523  6191 tablet_bootstrap.cc:492] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc: Bootstrap starting.
I20260709 12:16:14.538650  6191 tablet_bootstrap.cc:654] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.540122  6190 tablet_bootstrap.cc:654] T a275014900ca4f32977e32bb6fc509fd P 9468a996c51241cab3d90392e5864642: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.548559  6026 heartbeater.cc:499] Master 127.5.58.62:43011 was elected leader, sending a full tablet report...
I20260709 12:16:14.549182  6193 ts_tablet_manager.cc:1434] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82: Time spent starting tablet: real 0.050s	user 0.011s	sys 0.011s
I20260709 12:16:14.550151  6193 tablet_bootstrap.cc:492] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82: Bootstrap starting.
I20260709 12:16:14.553788  6190 tablet_bootstrap.cc:492] T a275014900ca4f32977e32bb6fc509fd P 9468a996c51241cab3d90392e5864642: No bootstrap required, opened a new log
I20260709 12:16:14.554474  6190 ts_tablet_manager.cc:1403] T a275014900ca4f32977e32bb6fc509fd P 9468a996c51241cab3d90392e5864642: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.008s
I20260709 12:16:14.556464  6193 tablet_bootstrap.cc:654] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.558753  6190 raft_consensus.cc:359] T a275014900ca4f32977e32bb6fc509fd P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.559971  6190 raft_consensus.cc:385] T a275014900ca4f32977e32bb6fc509fd P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.560886  6190 raft_consensus.cc:740] T a275014900ca4f32977e32bb6fc509fd P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9468a996c51241cab3d90392e5864642, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.562459  6190 consensus_queue.cc:260] T a275014900ca4f32977e32bb6fc509fd P 9468a996c51241cab3d90392e5864642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.565618  6201 raft_consensus.cc:697] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 1 LEADER]: Becoming Leader. State: Replica: 9468a996c51241cab3d90392e5864642, State: Running, Role: LEADER
I20260709 12:16:14.566589  6201 consensus_queue.cc:237] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.567369  6190 ts_tablet_manager.cc:1434] T a275014900ca4f32977e32bb6fc509fd P 9468a996c51241cab3d90392e5864642: Time spent starting tablet: real 0.012s	user 0.011s	sys 0.000s
I20260709 12:16:14.568691  6190 tablet_bootstrap.cc:492] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642: Bootstrap starting.
I20260709 12:16:14.571616  6191 tablet_bootstrap.cc:492] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc: No bootstrap required, opened a new log
I20260709 12:16:14.572139  6191 ts_tablet_manager.cc:1403] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc: Time spent bootstrapping tablet: real 0.041s	user 0.019s	sys 0.019s
I20260709 12:16:14.573355  6193 tablet_bootstrap.cc:492] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82: No bootstrap required, opened a new log
I20260709 12:16:14.573799  6193 ts_tablet_manager.cc:1403] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.000s
I20260709 12:16:14.574895  6190 tablet_bootstrap.cc:654] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.574994  6191 raft_consensus.cc:359] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.575762  6191 raft_consensus.cc:385] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.576064  6191 raft_consensus.cc:740] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ffd67a5e76e46a7b0089718369740bc, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.577436  6191 consensus_queue.cc:260] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.576743  6193 raft_consensus.cc:359] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.581559  6193 raft_consensus.cc:385] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.581907  6193 raft_consensus.cc:740] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c877d092949e693c13e9103581f82, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.582962  6193 consensus_queue.cc:260] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.583659  6191 ts_tablet_manager.cc:1434] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:16:14.584863  6191 tablet_bootstrap.cc:492] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc: Bootstrap starting.
I20260709 12:16:14.585196  6193 ts_tablet_manager.cc:1434] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 12:16:14.586354  6193 tablet_bootstrap.cc:492] T d1ec21d99cc2496d84a013cadd31fbc6 P bd8c877d092949e693c13e9103581f82: Bootstrap starting.
I20260709 12:16:14.587117  6190 tablet_bootstrap.cc:492] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642: No bootstrap required, opened a new log
I20260709 12:16:14.587580  6190 ts_tablet_manager.cc:1403] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.008s
I20260709 12:16:14.587944  5896 catalog_manager.cc:5697] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9468a996c51241cab3d90392e5864642 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "9468a996c51241cab3d90392e5864642" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:14.590535  6190 raft_consensus.cc:359] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.591251  6190 raft_consensus.cc:385] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.591528  6190 raft_consensus.cc:740] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9468a996c51241cab3d90392e5864642, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.592857  6193 tablet_bootstrap.cc:654] T d1ec21d99cc2496d84a013cadd31fbc6 P bd8c877d092949e693c13e9103581f82: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.592460  6190 consensus_queue.cc:260] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.595490  6190 ts_tablet_manager.cc:1434] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:16:14.596460  6190 tablet_bootstrap.cc:492] T cb9f68ad1dc24fc29ed8ad07197a68ac P 9468a996c51241cab3d90392e5864642: Bootstrap starting.
I20260709 12:16:14.602622  6190 tablet_bootstrap.cc:654] T cb9f68ad1dc24fc29ed8ad07197a68ac P 9468a996c51241cab3d90392e5864642: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.604357  6193 tablet_bootstrap.cc:492] T d1ec21d99cc2496d84a013cadd31fbc6 P bd8c877d092949e693c13e9103581f82: No bootstrap required, opened a new log
I20260709 12:16:14.604869  6193 ts_tablet_manager.cc:1403] T d1ec21d99cc2496d84a013cadd31fbc6 P bd8c877d092949e693c13e9103581f82: Time spent bootstrapping tablet: real 0.019s	user 0.005s	sys 0.012s
I20260709 12:16:14.607945  6193 raft_consensus.cc:359] T d1ec21d99cc2496d84a013cadd31fbc6 P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.608758  6193 raft_consensus.cc:385] T d1ec21d99cc2496d84a013cadd31fbc6 P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.609073  6193 raft_consensus.cc:740] T d1ec21d99cc2496d84a013cadd31fbc6 P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c877d092949e693c13e9103581f82, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.609903  6193 consensus_queue.cc:260] T d1ec21d99cc2496d84a013cadd31fbc6 P bd8c877d092949e693c13e9103581f82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
W20260709 12:16:14.614344  5947 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:14.615355  6193 ts_tablet_manager.cc:1434] T d1ec21d99cc2496d84a013cadd31fbc6 P bd8c877d092949e693c13e9103581f82: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:16:14.616444  6193 tablet_bootstrap.cc:492] T b40ad7108aec4ad2a5aa42acd343c6ac P bd8c877d092949e693c13e9103581f82: Bootstrap starting.
I20260709 12:16:14.617722  6191 tablet_bootstrap.cc:654] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.624802  6193 tablet_bootstrap.cc:654] T b40ad7108aec4ad2a5aa42acd343c6ac P bd8c877d092949e693c13e9103581f82: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.624739  6190 tablet_bootstrap.cc:492] T cb9f68ad1dc24fc29ed8ad07197a68ac P 9468a996c51241cab3d90392e5864642: No bootstrap required, opened a new log
I20260709 12:16:14.625828  6190 ts_tablet_manager.cc:1403] T cb9f68ad1dc24fc29ed8ad07197a68ac P 9468a996c51241cab3d90392e5864642: Time spent bootstrapping tablet: real 0.030s	user 0.024s	sys 0.003s
I20260709 12:16:14.629817  6191 tablet_bootstrap.cc:492] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc: No bootstrap required, opened a new log
I20260709 12:16:14.629398  6190 raft_consensus.cc:359] T cb9f68ad1dc24fc29ed8ad07197a68ac P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.630182  6190 raft_consensus.cc:385] T cb9f68ad1dc24fc29ed8ad07197a68ac P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.630317  6191 ts_tablet_manager.cc:1403] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc: Time spent bootstrapping tablet: real 0.046s	user 0.014s	sys 0.004s
I20260709 12:16:14.630594  6193 tablet_bootstrap.cc:492] T b40ad7108aec4ad2a5aa42acd343c6ac P bd8c877d092949e693c13e9103581f82: No bootstrap required, opened a new log
I20260709 12:16:14.630544  6190 raft_consensus.cc:740] T cb9f68ad1dc24fc29ed8ad07197a68ac P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9468a996c51241cab3d90392e5864642, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.631155  6193 ts_tablet_manager.cc:1403] T b40ad7108aec4ad2a5aa42acd343c6ac P bd8c877d092949e693c13e9103581f82: Time spent bootstrapping tablet: real 0.015s	user 0.002s	sys 0.011s
I20260709 12:16:14.631500  6190 consensus_queue.cc:260] T cb9f68ad1dc24fc29ed8ad07197a68ac P 9468a996c51241cab3d90392e5864642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.633566  6191 raft_consensus.cc:359] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.633988  6190 ts_tablet_manager.cc:1434] T cb9f68ad1dc24fc29ed8ad07197a68ac P 9468a996c51241cab3d90392e5864642: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:16:14.634359  6191 raft_consensus.cc:385] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.634081  6193 raft_consensus.cc:359] T b40ad7108aec4ad2a5aa42acd343c6ac P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.634737  6191 raft_consensus.cc:740] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ffd67a5e76e46a7b0089718369740bc, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.634943  6193 raft_consensus.cc:385] T b40ad7108aec4ad2a5aa42acd343c6ac P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.635277  6193 raft_consensus.cc:740] T b40ad7108aec4ad2a5aa42acd343c6ac P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c877d092949e693c13e9103581f82, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.635680  6190 tablet_bootstrap.cc:492] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642: Bootstrap starting.
I20260709 12:16:14.635711  6191 consensus_queue.cc:260] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.636051  6193 consensus_queue.cc:260] T b40ad7108aec4ad2a5aa42acd343c6ac P bd8c877d092949e693c13e9103581f82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.643467  6193 ts_tablet_manager.cc:1434] T b40ad7108aec4ad2a5aa42acd343c6ac P bd8c877d092949e693c13e9103581f82: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:16:14.643702  6190 tablet_bootstrap.cc:654] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.644546  6193 tablet_bootstrap.cc:492] T 8ff9114868e646a4967bf3e65485393c P bd8c877d092949e693c13e9103581f82: Bootstrap starting.
I20260709 12:16:14.646093  6191 ts_tablet_manager.cc:1434] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 12:16:14.647233  6191 tablet_bootstrap.cc:492] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc: Bootstrap starting.
I20260709 12:16:14.651324  6193 tablet_bootstrap.cc:654] T 8ff9114868e646a4967bf3e65485393c P bd8c877d092949e693c13e9103581f82: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.655074  6191 tablet_bootstrap.cc:654] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.655398  6190 tablet_bootstrap.cc:492] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642: No bootstrap required, opened a new log
I20260709 12:16:14.655849  6190 ts_tablet_manager.cc:1403] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.004s
I20260709 12:16:14.660189  6190 raft_consensus.cc:359] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.661247  6190 raft_consensus.cc:385] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.661635  6190 raft_consensus.cc:740] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9468a996c51241cab3d90392e5864642, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.662711  6190 consensus_queue.cc:260] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.665875  6193 tablet_bootstrap.cc:492] T 8ff9114868e646a4967bf3e65485393c P bd8c877d092949e693c13e9103581f82: No bootstrap required, opened a new log
I20260709 12:16:14.666309  6193 ts_tablet_manager.cc:1403] T 8ff9114868e646a4967bf3e65485393c P bd8c877d092949e693c13e9103581f82: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.006s
I20260709 12:16:14.668429  6190 ts_tablet_manager.cc:1434] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.002s
I20260709 12:16:14.669210  6193 raft_consensus.cc:359] T 8ff9114868e646a4967bf3e65485393c P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.669826  6190 tablet_bootstrap.cc:492] T a3442b7a72a34bb3971f1dae9e473e38 P 9468a996c51241cab3d90392e5864642: Bootstrap starting.
I20260709 12:16:14.669988  6193 raft_consensus.cc:385] T 8ff9114868e646a4967bf3e65485393c P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.670423  6193 raft_consensus.cc:740] T 8ff9114868e646a4967bf3e65485393c P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c877d092949e693c13e9103581f82, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.671365  6193 consensus_queue.cc:260] T 8ff9114868e646a4967bf3e65485393c P bd8c877d092949e693c13e9103581f82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.679000  6193 ts_tablet_manager.cc:1434] T 8ff9114868e646a4967bf3e65485393c P bd8c877d092949e693c13e9103581f82: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:16:14.679477  6190 tablet_bootstrap.cc:654] T a3442b7a72a34bb3971f1dae9e473e38 P 9468a996c51241cab3d90392e5864642: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.682179  6193 tablet_bootstrap.cc:492] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82: Bootstrap starting.
I20260709 12:16:14.687904  6191 tablet_bootstrap.cc:492] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc: No bootstrap required, opened a new log
I20260709 12:16:14.688700  6191 ts_tablet_manager.cc:1403] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc: Time spent bootstrapping tablet: real 0.042s	user 0.017s	sys 0.015s
I20260709 12:16:14.688804  6193 tablet_bootstrap.cc:654] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.691365  6190 tablet_bootstrap.cc:492] T a3442b7a72a34bb3971f1dae9e473e38 P 9468a996c51241cab3d90392e5864642: No bootstrap required, opened a new log
I20260709 12:16:14.692107  6190 ts_tablet_manager.cc:1403] T a3442b7a72a34bb3971f1dae9e473e38 P 9468a996c51241cab3d90392e5864642: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.005s
I20260709 12:16:14.691394  6191 raft_consensus.cc:359] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.692524  6191 raft_consensus.cc:385] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.692902  6191 raft_consensus.cc:740] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ffd67a5e76e46a7b0089718369740bc, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.693804  6202 raft_consensus.cc:493] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.693635  6191 consensus_queue.cc:260] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.694285  6202 raft_consensus.cc:515] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.694980  6193 tablet_bootstrap.cc:492] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82: No bootstrap required, opened a new log
I20260709 12:16:14.695554  6193 ts_tablet_manager.cc:1403] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82: Time spent bootstrapping tablet: real 0.014s	user 0.006s	sys 0.006s
I20260709 12:16:14.695693  6190 raft_consensus.cc:359] T a3442b7a72a34bb3971f1dae9e473e38 P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.696555  6190 raft_consensus.cc:385] T a3442b7a72a34bb3971f1dae9e473e38 P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.696885  6190 raft_consensus.cc:740] T a3442b7a72a34bb3971f1dae9e473e38 P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9468a996c51241cab3d90392e5864642, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.697870  6190 consensus_queue.cc:260] T a3442b7a72a34bb3971f1dae9e473e38 P 9468a996c51241cab3d90392e5864642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.699482  6193 raft_consensus.cc:359] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.700383  6193 raft_consensus.cc:385] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.700778  6193 raft_consensus.cc:740] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c877d092949e693c13e9103581f82, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.701447  6190 ts_tablet_manager.cc:1434] T a3442b7a72a34bb3971f1dae9e473e38 P 9468a996c51241cab3d90392e5864642: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.005s
I20260709 12:16:14.701859  6193 consensus_queue.cc:260] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.702507  6190 tablet_bootstrap.cc:492] T b40ad7108aec4ad2a5aa42acd343c6ac P 9468a996c51241cab3d90392e5864642: Bootstrap starting.
I20260709 12:16:14.710654  6202 leader_election.cc:290] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:14.711819  6193 ts_tablet_manager.cc:1434] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.007s
I20260709 12:16:14.714690  6190 tablet_bootstrap.cc:654] T b40ad7108aec4ad2a5aa42acd343c6ac P 9468a996c51241cab3d90392e5864642: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.719877  6193 tablet_bootstrap.cc:492] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82: Bootstrap starting.
I20260709 12:16:14.729615  6196 raft_consensus.cc:493] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.730105  6196 raft_consensus.cc:515] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.731716  6193 tablet_bootstrap.cc:654] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.720504  6191 ts_tablet_manager.cc:1434] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.003s
I20260709 12:16:14.733476  6191 tablet_bootstrap.cc:492] T a3442b7a72a34bb3971f1dae9e473e38 P 0ffd67a5e76e46a7b0089718369740bc: Bootstrap starting.
I20260709 12:16:14.732172  6196 leader_election.cc:290] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:14.745540  6191 tablet_bootstrap.cc:654] T a3442b7a72a34bb3971f1dae9e473e38 P 0ffd67a5e76e46a7b0089718369740bc: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.750274  6190 tablet_bootstrap.cc:492] T b40ad7108aec4ad2a5aa42acd343c6ac P 9468a996c51241cab3d90392e5864642: No bootstrap required, opened a new log
I20260709 12:16:14.750876  6190 ts_tablet_manager.cc:1403] T b40ad7108aec4ad2a5aa42acd343c6ac P 9468a996c51241cab3d90392e5864642: Time spent bootstrapping tablet: real 0.049s	user 0.023s	sys 0.016s
I20260709 12:16:14.755256  6190 raft_consensus.cc:359] T b40ad7108aec4ad2a5aa42acd343c6ac P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.756134  6190 raft_consensus.cc:385] T b40ad7108aec4ad2a5aa42acd343c6ac P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.756496  6190 raft_consensus.cc:740] T b40ad7108aec4ad2a5aa42acd343c6ac P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9468a996c51241cab3d90392e5864642, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.757323  6190 consensus_queue.cc:260] T b40ad7108aec4ad2a5aa42acd343c6ac P 9468a996c51241cab3d90392e5864642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.760074  6190 ts_tablet_manager.cc:1434] T b40ad7108aec4ad2a5aa42acd343c6ac P 9468a996c51241cab3d90392e5864642: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:16:14.762476  6193 tablet_bootstrap.cc:492] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82: No bootstrap required, opened a new log
I20260709 12:16:14.772734  6193 ts_tablet_manager.cc:1403] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82: Time spent bootstrapping tablet: real 0.053s	user 0.012s	sys 0.011s
I20260709 12:16:14.772888  6191 tablet_bootstrap.cc:492] T a3442b7a72a34bb3971f1dae9e473e38 P 0ffd67a5e76e46a7b0089718369740bc: No bootstrap required, opened a new log
I20260709 12:16:14.773337  6191 ts_tablet_manager.cc:1403] T a3442b7a72a34bb3971f1dae9e473e38 P 0ffd67a5e76e46a7b0089718369740bc: Time spent bootstrapping tablet: real 0.040s	user 0.016s	sys 0.019s
I20260709 12:16:14.775826  6193 raft_consensus.cc:359] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.776072  6191 raft_consensus.cc:359] T a3442b7a72a34bb3971f1dae9e473e38 P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.776628  6193 raft_consensus.cc:385] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.776949  6193 raft_consensus.cc:740] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c877d092949e693c13e9103581f82, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.776734  6191 raft_consensus.cc:385] T a3442b7a72a34bb3971f1dae9e473e38 P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.777805  6191 raft_consensus.cc:740] T a3442b7a72a34bb3971f1dae9e473e38 P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ffd67a5e76e46a7b0089718369740bc, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.777673  6193 consensus_queue.cc:260] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.779939  6193 ts_tablet_manager.cc:1434] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.001s
I20260709 12:16:14.781051  6190 tablet_bootstrap.cc:492] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642: Bootstrap starting.
I20260709 12:16:14.782282  6193 tablet_bootstrap.cc:492] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82: Bootstrap starting.
I20260709 12:16:14.786340  6191 consensus_queue.cc:260] T a3442b7a72a34bb3971f1dae9e473e38 P 0ffd67a5e76e46a7b0089718369740bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.788760  6193 tablet_bootstrap.cc:654] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.794039  6190 tablet_bootstrap.cc:654] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.795559  6191 ts_tablet_manager.cc:1434] T a3442b7a72a34bb3971f1dae9e473e38 P 0ffd67a5e76e46a7b0089718369740bc: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 12:16:14.793051  6196 raft_consensus.cc:493] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.796113  6196 raft_consensus.cc:515] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.798416  6196 leader_election.cc:290] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:14.799902  6191 tablet_bootstrap.cc:492] T b40ad7108aec4ad2a5aa42acd343c6ac P 0ffd67a5e76e46a7b0089718369740bc: Bootstrap starting.
I20260709 12:16:14.805721  6201 raft_consensus.cc:493] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.806423  6201 raft_consensus.cc:515] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.810343  6201 leader_election.cc:290] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019), bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557)
I20260709 12:16:14.810379  6196 raft_consensus.cc:493] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.811086  6196 raft_consensus.cc:515] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.813841  6075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46526944508c443c9c12315136ae866f" candidate_uuid: "9468a996c51241cab3d90392e5864642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffd67a5e76e46a7b0089718369740bc" is_pre_election: true
I20260709 12:16:14.814546  6075 raft_consensus.cc:2468] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9468a996c51241cab3d90392e5864642 in term 0.
I20260709 12:16:14.815779  6113 leader_election.cc:304] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc, 9468a996c51241cab3d90392e5864642; no voters: 
I20260709 12:16:14.816216  6001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46526944508c443c9c12315136ae866f" candidate_uuid: "9468a996c51241cab3d90392e5864642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c877d092949e693c13e9103581f82" is_pre_election: true
I20260709 12:16:14.816620  6201 raft_consensus.cc:2804] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:14.816854  6001 raft_consensus.cc:2468] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9468a996c51241cab3d90392e5864642 in term 0.
I20260709 12:16:14.816998  6201 raft_consensus.cc:493] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:14.817371  6201 raft_consensus.cc:3060] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.818154  6076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3442b7a72a34bb3971f1dae9e473e38" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffd67a5e76e46a7b0089718369740bc" is_pre_election: true
I20260709 12:16:14.819653  6076 raft_consensus.cc:2468] T a3442b7a72a34bb3971f1dae9e473e38 P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 0.
I20260709 12:16:14.825981  6001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a275014900ca4f32977e32bb6fc509fd" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c877d092949e693c13e9103581f82" is_pre_election: true
I20260709 12:16:14.826926  6001 raft_consensus.cc:2468] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ffd67a5e76e46a7b0089718369740bc in term 0.
I20260709 12:16:14.828941  5999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46526944508c443c9c12315136ae866f" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c877d092949e693c13e9103581f82" is_pre_election: true
I20260709 12:16:14.829332  6196 leader_election.cc:290] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:14.829847  5999 raft_consensus.cc:2468] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ffd67a5e76e46a7b0089718369740bc in term 0.
I20260709 12:16:14.830415  5964 leader_election.cc:304] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc, bd8c877d092949e693c13e9103581f82; no voters: 
I20260709 12:16:14.830842  6000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f858fa3ce674075aabe68fc8228266d" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c877d092949e693c13e9103581f82" is_pre_election: true
I20260709 12:16:14.831640  6221 raft_consensus.cc:2804] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:14.832144  6221 raft_consensus.cc:493] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:14.832705  6221 raft_consensus.cc:3060] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.834345  6042 leader_election.cc:304] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc, bd8c877d092949e693c13e9103581f82; no voters: 
I20260709 12:16:14.835678  6042 leader_election.cc:304] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc, bd8c877d092949e693c13e9103581f82; no voters: 
I20260709 12:16:14.841446  6191 tablet_bootstrap.cc:654] T b40ad7108aec4ad2a5aa42acd343c6ac P 0ffd67a5e76e46a7b0089718369740bc: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.842684  6196 raft_consensus.cc:2804] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:14.843072  6196 raft_consensus.cc:493] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:14.843411  6196 raft_consensus.cc:3060] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.849843  6150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a275014900ca4f32977e32bb6fc509fd" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9468a996c51241cab3d90392e5864642" is_pre_election: true
I20260709 12:16:14.850569  6150 raft_consensus.cc:2468] T a275014900ca4f32977e32bb6fc509fd P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ffd67a5e76e46a7b0089718369740bc in term 0.
I20260709 12:16:14.853479  6149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f858fa3ce674075aabe68fc8228266d" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9468a996c51241cab3d90392e5864642" is_pre_election: true
I20260709 12:16:14.855005  6201 raft_consensus.cc:515] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.855639  6148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46526944508c443c9c12315136ae866f" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9468a996c51241cab3d90392e5864642" is_pre_election: true
I20260709 12:16:14.856424  6190 tablet_bootstrap.cc:492] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642: No bootstrap required, opened a new log
I20260709 12:16:14.856833  6148 raft_consensus.cc:2393] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ffd67a5e76e46a7b0089718369740bc in current term 1: Already voted for candidate 9468a996c51241cab3d90392e5864642 in this term.
I20260709 12:16:14.856889  6190 ts_tablet_manager.cc:1403] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642: Time spent bootstrapping tablet: real 0.076s	user 0.026s	sys 0.010s
I20260709 12:16:14.858150  6076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46526944508c443c9c12315136ae866f" candidate_uuid: "9468a996c51241cab3d90392e5864642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
I20260709 12:16:14.858719  6076 raft_consensus.cc:3060] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:14.862082  6041 leader_election.cc:343] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:14.862989  6190 raft_consensus.cc:359] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.863692  6190 raft_consensus.cc:385] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.863976  6190 raft_consensus.cc:740] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9468a996c51241cab3d90392e5864642, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.864717  6190 consensus_queue.cc:260] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.866225  6076 raft_consensus.cc:2468] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9468a996c51241cab3d90392e5864642 in term 1.
I20260709 12:16:14.867897  6113 leader_election.cc:304] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc, 9468a996c51241cab3d90392e5864642; no voters: 
I20260709 12:16:14.869140  6203 raft_consensus.cc:2804] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:14.869701  6203 raft_consensus.cc:697] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 1 LEADER]: Becoming Leader. State: Replica: 9468a996c51241cab3d90392e5864642, State: Running, Role: LEADER
I20260709 12:16:14.870641  6203 consensus_queue.cc:237] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.870978  6221 raft_consensus.cc:515] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.874191  6195 raft_consensus.cc:493] T cb9f68ad1dc24fc29ed8ad07197a68ac P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.874910  6195 raft_consensus.cc:515] T cb9f68ad1dc24fc29ed8ad07197a68ac P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.875999  6193 tablet_bootstrap.cc:492] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82: No bootstrap required, opened a new log
I20260709 12:16:14.876588  6193 ts_tablet_manager.cc:1403] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82: Time spent bootstrapping tablet: real 0.095s	user 0.029s	sys 0.022s
I20260709 12:16:14.878136  6190 ts_tablet_manager.cc:1434] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.000s
I20260709 12:16:14.878239  6000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb9f68ad1dc24fc29ed8ad07197a68ac" candidate_uuid: "9468a996c51241cab3d90392e5864642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c877d092949e693c13e9103581f82" is_pre_election: true
I20260709 12:16:14.879208  6190 tablet_bootstrap.cc:492] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642: Bootstrap starting.
I20260709 12:16:14.879356  6193 raft_consensus.cc:359] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
W20260709 12:16:14.879771  6116 leader_election.cc:343] T cb9f68ad1dc24fc29ed8ad07197a68ac P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:14.880048  6193 raft_consensus.cc:385] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.880381  6193 raft_consensus.cc:740] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c877d092949e693c13e9103581f82, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.880774  6195 leader_election.cc:290] T cb9f68ad1dc24fc29ed8ad07197a68ac P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557), 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019)
W20260709 12:16:14.881681  6042 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:35557, user_credentials={real_user=slave}} blocked reactor thread for 46090.5us
W20260709 12:16:14.882894  6042 leader_election.cc:343] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:14.883599  6042 leader_election.cc:304] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc; no voters: 9468a996c51241cab3d90392e5864642, bd8c877d092949e693c13e9103581f82
I20260709 12:16:14.885169  6193 consensus_queue.cc:260] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.889518  6193 ts_tablet_manager.cc:1434] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:16:14.890920  6193 tablet_bootstrap.cc:492] T cb9f68ad1dc24fc29ed8ad07197a68ac P bd8c877d092949e693c13e9103581f82: Bootstrap starting.
I20260709 12:16:14.886871  6076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb9f68ad1dc24fc29ed8ad07197a68ac" candidate_uuid: "9468a996c51241cab3d90392e5864642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffd67a5e76e46a7b0089718369740bc" is_pre_election: true
I20260709 12:16:14.890223  6201 leader_election.cc:290] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 1 election: Requested vote from peers 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019), bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557)
I20260709 12:16:14.896291  6000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46526944508c443c9c12315136ae866f" candidate_uuid: "9468a996c51241cab3d90392e5864642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c877d092949e693c13e9103581f82"
I20260709 12:16:14.897128  6000 raft_consensus.cc:3060] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.897171  6076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3442b7a72a34bb3971f1dae9e473e38" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
I20260709 12:16:14.895713  6221 leader_election.cc:290] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 1 election: Requested vote from peers 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
W20260709 12:16:14.898991  6113 leader_election.cc:343] T cb9f68ad1dc24fc29ed8ad07197a68ac P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:14.901495  6113 leader_election.cc:304] T cb9f68ad1dc24fc29ed8ad07197a68ac P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9468a996c51241cab3d90392e5864642; no voters: 0ffd67a5e76e46a7b0089718369740bc, bd8c877d092949e693c13e9103581f82
I20260709 12:16:14.898916  6076 raft_consensus.cc:3060] T a3442b7a72a34bb3971f1dae9e473e38 P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.902499  6201 raft_consensus.cc:2749] T cb9f68ad1dc24fc29ed8ad07197a68ac P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:14.909171  6191 tablet_bootstrap.cc:492] T b40ad7108aec4ad2a5aa42acd343c6ac P 0ffd67a5e76e46a7b0089718369740bc: No bootstrap required, opened a new log
I20260709 12:16:14.909710  6191 ts_tablet_manager.cc:1403] T b40ad7108aec4ad2a5aa42acd343c6ac P 0ffd67a5e76e46a7b0089718369740bc: Time spent bootstrapping tablet: real 0.110s	user 0.014s	sys 0.000s
I20260709 12:16:14.911768  6000 raft_consensus.cc:2468] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9468a996c51241cab3d90392e5864642 in term 1.
I20260709 12:16:14.899796  6196 raft_consensus.cc:515] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.912868  6191 raft_consensus.cc:359] T b40ad7108aec4ad2a5aa42acd343c6ac P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.913640  6191 raft_consensus.cc:385] T b40ad7108aec4ad2a5aa42acd343c6ac P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.913787  6232 raft_consensus.cc:2749] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:14.913990  6191 raft_consensus.cc:740] T b40ad7108aec4ad2a5aa42acd343c6ac P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ffd67a5e76e46a7b0089718369740bc, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.914604  6196 leader_election.cc:290] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 1 election: Requested vote from peers bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:14.914824  6191 consensus_queue.cc:260] T b40ad7108aec4ad2a5aa42acd343c6ac P 0ffd67a5e76e46a7b0089718369740bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.915916  6190 tablet_bootstrap.cc:654] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.915799  6148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a275014900ca4f32977e32bb6fc509fd" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9468a996c51241cab3d90392e5864642"
I20260709 12:16:14.916452  6148 raft_consensus.cc:3060] T a275014900ca4f32977e32bb6fc509fd P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.915184  6230 raft_consensus.cc:2764] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:16:14.916811  6149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3442b7a72a34bb3971f1dae9e473e38" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9468a996c51241cab3d90392e5864642" is_pre_election: true
I20260709 12:16:14.918531  6149 raft_consensus.cc:2468] T a3442b7a72a34bb3971f1dae9e473e38 P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 0.
I20260709 12:16:14.920318  6150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3442b7a72a34bb3971f1dae9e473e38" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9468a996c51241cab3d90392e5864642"
I20260709 12:16:14.919232  6076 raft_consensus.cc:2468] T a3442b7a72a34bb3971f1dae9e473e38 P 0ffd67a5e76e46a7b0089718369740bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 1.
I20260709 12:16:14.921072  6150 raft_consensus.cc:3060] T a3442b7a72a34bb3971f1dae9e473e38 P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.917637  6232 raft_consensus.cc:493] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.924119  5964 leader_election.cc:304] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc, bd8c877d092949e693c13e9103581f82; no voters: 
I20260709 12:16:14.925040  6221 raft_consensus.cc:2804] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:14.925737  6221 raft_consensus.cc:697] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c877d092949e693c13e9103581f82, State: Running, Role: LEADER
I20260709 12:16:14.927477  6221 consensus_queue.cc:237] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.933964  6191 ts_tablet_manager.cc:1434] T b40ad7108aec4ad2a5aa42acd343c6ac P 0ffd67a5e76e46a7b0089718369740bc: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 12:16:14.931857  6232 raft_consensus.cc:515] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.939139  6232 leader_election.cc:290] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:14.941412  6149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "800b87c938ae4f1596d89f00bb34399d" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9468a996c51241cab3d90392e5864642" is_pre_election: true
I20260709 12:16:14.942137  6149 raft_consensus.cc:2468] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ffd67a5e76e46a7b0089718369740bc in term 0.
I20260709 12:16:14.939193  6202 raft_consensus.cc:493] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.943260  6202 raft_consensus.cc:515] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.945324  6202 leader_election.cc:290] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:14.946768  6150 raft_consensus.cc:2468] T a3442b7a72a34bb3971f1dae9e473e38 P 9468a996c51241cab3d90392e5864642 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 1.
I20260709 12:16:14.947425  6041 leader_election.cc:304] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc, 9468a996c51241cab3d90392e5864642; no voters: 
I20260709 12:16:14.950100  6076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "800b87c938ae4f1596d89f00bb34399d" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffd67a5e76e46a7b0089718369740bc" is_pre_election: true
I20260709 12:16:14.950703  6076 raft_consensus.cc:2468] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 0.
I20260709 12:16:14.951637  6149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "800b87c938ae4f1596d89f00bb34399d" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9468a996c51241cab3d90392e5864642" is_pre_election: true
I20260709 12:16:14.951815  5964 leader_election.cc:304] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc, bd8c877d092949e693c13e9103581f82; no voters: 
I20260709 12:16:14.952222  6149 raft_consensus.cc:2468] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 0.
I20260709 12:16:14.952622  6202 raft_consensus.cc:2804] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:14.952986  6202 raft_consensus.cc:493] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:14.953328  6202 raft_consensus.cc:3060] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.958968  6148 raft_consensus.cc:2468] T a275014900ca4f32977e32bb6fc509fd P 9468a996c51241cab3d90392e5864642 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ffd67a5e76e46a7b0089718369740bc in term 1.
I20260709 12:16:14.960197  6041 leader_election.cc:304] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc, 9468a996c51241cab3d90392e5864642; no voters: 
I20260709 12:16:14.943706  6193 tablet_bootstrap.cc:654] T cb9f68ad1dc24fc29ed8ad07197a68ac P bd8c877d092949e693c13e9103581f82: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.960749  6190 tablet_bootstrap.cc:492] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642: No bootstrap required, opened a new log
I20260709 12:16:14.961040  6196 raft_consensus.cc:2804] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:14.961468  6196 raft_consensus.cc:493] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:14.961588  6190 ts_tablet_manager.cc:1403] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642: Time spent bootstrapping tablet: real 0.083s	user 0.015s	sys 0.000s
I20260709 12:16:14.961822  6196 raft_consensus.cc:3060] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.962491  6202 raft_consensus.cc:515] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.965325  6076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "800b87c938ae4f1596d89f00bb34399d" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
I20260709 12:16:14.960716  6191 tablet_bootstrap.cc:492] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc: Bootstrap starting.
I20260709 12:16:14.965723  5896 catalog_manager.cc:5697] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9468a996c51241cab3d90392e5864642 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "9468a996c51241cab3d90392e5864642" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:14.968909  6196 raft_consensus.cc:515] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.965235  6190 raft_consensus.cc:359] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.970265  6076 raft_consensus.cc:2393] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bd8c877d092949e693c13e9103581f82 in current term 1: Already voted for candidate 0ffd67a5e76e46a7b0089718369740bc in this term.
I20260709 12:16:14.970327  6190 raft_consensus.cc:385] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:14.970743  6190 raft_consensus.cc:740] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9468a996c51241cab3d90392e5864642, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.971493  6190 consensus_queue.cc:260] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.975464  6234 raft_consensus.cc:2804] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:14.975787  6191 tablet_bootstrap.cc:654] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:14.976971  6234 raft_consensus.cc:697] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [term 1 LEADER]: Becoming Leader. State: Replica: 0ffd67a5e76e46a7b0089718369740bc, State: Running, Role: LEADER
I20260709 12:16:14.978259  6234 consensus_queue.cc:237] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.981341  5999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "800b87c938ae4f1596d89f00bb34399d" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c877d092949e693c13e9103581f82" is_pre_election: true
I20260709 12:16:14.981972  6190 ts_tablet_manager.cc:1434] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 12:16:14.982199  5999 raft_consensus.cc:2393] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ffd67a5e76e46a7b0089718369740bc in current term 1: Already voted for candidate bd8c877d092949e693c13e9103581f82 in this term.
I20260709 12:16:14.982429  6202 leader_election.cc:290] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 1 election: Requested vote from peers 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:14.982949  6202 raft_consensus.cc:493] T b40ad7108aec4ad2a5aa42acd343c6ac P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.983259  6001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "800b87c938ae4f1596d89f00bb34399d" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c877d092949e693c13e9103581f82"
I20260709 12:16:14.983471  6202 raft_consensus.cc:515] T b40ad7108aec4ad2a5aa42acd343c6ac P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.984130  6001 raft_consensus.cc:2393] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ffd67a5e76e46a7b0089718369740bc in current term 1: Already voted for candidate bd8c877d092949e693c13e9103581f82 in this term.
I20260709 12:16:14.982668  6000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a275014900ca4f32977e32bb6fc509fd" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c877d092949e693c13e9103581f82"
I20260709 12:16:14.984730  6000 raft_consensus.cc:3060] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:14.990226  6203 consensus_queue.cc:1048] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:14.992877  6148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "800b87c938ae4f1596d89f00bb34399d" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9468a996c51241cab3d90392e5864642"
I20260709 12:16:14.994580  6203 raft_consensus.cc:493] T d1ec21d99cc2496d84a013cadd31fbc6 P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.995178  6203 raft_consensus.cc:515] T d1ec21d99cc2496d84a013cadd31fbc6 P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:14.998314  6001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ec21d99cc2496d84a013cadd31fbc6" candidate_uuid: "9468a996c51241cab3d90392e5864642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c877d092949e693c13e9103581f82" is_pre_election: true
I20260709 12:16:14.999025  6001 raft_consensus.cc:2468] T d1ec21d99cc2496d84a013cadd31fbc6 P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9468a996c51241cab3d90392e5864642 in term 0.
I20260709 12:16:15.000295  6116 leader_election.cc:304] T d1ec21d99cc2496d84a013cadd31fbc6 P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9468a996c51241cab3d90392e5864642, bd8c877d092949e693c13e9103581f82; no voters: 
I20260709 12:16:15.001345  6201 raft_consensus.cc:2804] T d1ec21d99cc2496d84a013cadd31fbc6 P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:15.001757  6201 raft_consensus.cc:493] T d1ec21d99cc2496d84a013cadd31fbc6 P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:15.002204  6201 raft_consensus.cc:3060] T d1ec21d99cc2496d84a013cadd31fbc6 P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.001081  6202 leader_election.cc:290] T b40ad7108aec4ad2a5aa42acd343c6ac P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:15.004300  6193 tablet_bootstrap.cc:492] T cb9f68ad1dc24fc29ed8ad07197a68ac P bd8c877d092949e693c13e9103581f82: No bootstrap required, opened a new log
I20260709 12:16:15.005034  6193 ts_tablet_manager.cc:1403] T cb9f68ad1dc24fc29ed8ad07197a68ac P bd8c877d092949e693c13e9103581f82: Time spent bootstrapping tablet: real 0.114s	user 0.014s	sys 0.000s
I20260709 12:16:15.005825  6075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40ad7108aec4ad2a5aa42acd343c6ac" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffd67a5e76e46a7b0089718369740bc" is_pre_election: true
I20260709 12:16:15.006472  6075 raft_consensus.cc:2468] T b40ad7108aec4ad2a5aa42acd343c6ac P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 0.
I20260709 12:16:15.008538  5964 leader_election.cc:304] T b40ad7108aec4ad2a5aa42acd343c6ac P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc, bd8c877d092949e693c13e9103581f82; no voters: 
I20260709 12:16:15.009737  6202 raft_consensus.cc:2804] T b40ad7108aec4ad2a5aa42acd343c6ac P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:15.010087  6202 raft_consensus.cc:493] T b40ad7108aec4ad2a5aa42acd343c6ac P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:15.010452  6202 raft_consensus.cc:3060] T b40ad7108aec4ad2a5aa42acd343c6ac P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.013983  6203 leader_election.cc:290] T d1ec21d99cc2496d84a013cadd31fbc6 P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557), 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019)
I20260709 12:16:15.014654  6075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ec21d99cc2496d84a013cadd31fbc6" candidate_uuid: "9468a996c51241cab3d90392e5864642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffd67a5e76e46a7b0089718369740bc" is_pre_election: true
W20260709 12:16:15.016456  6113 leader_election.cc:343] T d1ec21d99cc2496d84a013cadd31fbc6 P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:15.011905  6193 raft_consensus.cc:359] T cb9f68ad1dc24fc29ed8ad07197a68ac P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.017831  6193 raft_consensus.cc:385] T cb9f68ad1dc24fc29ed8ad07197a68ac P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:15.018249  6193 raft_consensus.cc:740] T cb9f68ad1dc24fc29ed8ad07197a68ac P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c877d092949e693c13e9103581f82, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.017838  6201 raft_consensus.cc:515] T d1ec21d99cc2496d84a013cadd31fbc6 P 9468a996c51241cab3d90392e5864642 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.019033  6149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40ad7108aec4ad2a5aa42acd343c6ac" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9468a996c51241cab3d90392e5864642" is_pre_election: true
I20260709 12:16:15.019738  6149 raft_consensus.cc:2468] T b40ad7108aec4ad2a5aa42acd343c6ac P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 0.
I20260709 12:16:15.020038  6193 consensus_queue.cc:260] T cb9f68ad1dc24fc29ed8ad07197a68ac P bd8c877d092949e693c13e9103581f82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.021066  6201 leader_election.cc:290] T d1ec21d99cc2496d84a013cadd31fbc6 P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 1 election: Requested vote from peers bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557), 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019)
I20260709 12:16:15.020310  5899 catalog_manager.cc:5697] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2). New cstate: current_term: 1 leader_uuid: "0ffd67a5e76e46a7b0089718369740bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:15.022423  6193 ts_tablet_manager.cc:1434] T cb9f68ad1dc24fc29ed8ad07197a68ac P bd8c877d092949e693c13e9103581f82: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:16:15.022534  6076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ec21d99cc2496d84a013cadd31fbc6" candidate_uuid: "9468a996c51241cab3d90392e5864642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
I20260709 12:16:15.022694  6001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ec21d99cc2496d84a013cadd31fbc6" candidate_uuid: "9468a996c51241cab3d90392e5864642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c877d092949e693c13e9103581f82"
I20260709 12:16:15.023373  6193 tablet_bootstrap.cc:492] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82: Bootstrap starting.
I20260709 12:16:15.025890  6202 raft_consensus.cc:515] T b40ad7108aec4ad2a5aa42acd343c6ac P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.029520  6193 tablet_bootstrap.cc:654] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:15.030395  6075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40ad7108aec4ad2a5aa42acd343c6ac" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
I20260709 12:16:15.031006  6075 raft_consensus.cc:3060] T b40ad7108aec4ad2a5aa42acd343c6ac P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.032246  6202 leader_election.cc:290] T b40ad7108aec4ad2a5aa42acd343c6ac P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 1 election: Requested vote from peers 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:15.035648  5898 catalog_manager.cc:5697] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c877d092949e693c13e9103581f82 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "bd8c877d092949e693c13e9103581f82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:15.023293  6001 raft_consensus.cc:3060] T d1ec21d99cc2496d84a013cadd31fbc6 P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.051239  5999 raft_consensus.cc:3060] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.053407  6150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "800b87c938ae4f1596d89f00bb34399d" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9468a996c51241cab3d90392e5864642"
I20260709 12:16:15.055744  6041 leader_election.cc:304] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc; no voters: 9468a996c51241cab3d90392e5864642, bd8c877d092949e693c13e9103581f82
I20260709 12:16:15.058810  6232 raft_consensus.cc:2749] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:15.051131  6149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40ad7108aec4ad2a5aa42acd343c6ac" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9468a996c51241cab3d90392e5864642"
I20260709 12:16:15.061293  6149 raft_consensus.cc:3060] T b40ad7108aec4ad2a5aa42acd343c6ac P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.062680  6195 consensus_queue.cc:1048] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:15.065164  6075 raft_consensus.cc:2468] T b40ad7108aec4ad2a5aa42acd343c6ac P 0ffd67a5e76e46a7b0089718369740bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 1.
I20260709 12:16:15.067719  5964 leader_election.cc:304] T b40ad7108aec4ad2a5aa42acd343c6ac P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc, bd8c877d092949e693c13e9103581f82; no voters: 
I20260709 12:16:15.068157  6149 raft_consensus.cc:2468] T b40ad7108aec4ad2a5aa42acd343c6ac P 9468a996c51241cab3d90392e5864642 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 1.
I20260709 12:16:15.066562  6001 raft_consensus.cc:2468] T d1ec21d99cc2496d84a013cadd31fbc6 P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9468a996c51241cab3d90392e5864642 in term 1.
I20260709 12:16:15.071501  6202 raft_consensus.cc:2804] T b40ad7108aec4ad2a5aa42acd343c6ac P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:15.072041  6202 raft_consensus.cc:697] T b40ad7108aec4ad2a5aa42acd343c6ac P bd8c877d092949e693c13e9103581f82 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c877d092949e693c13e9103581f82, State: Running, Role: LEADER
W20260709 12:16:15.060628  6113 leader_election.cc:343] T d1ec21d99cc2496d84a013cadd31fbc6 P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:15.073081  6193 tablet_bootstrap.cc:492] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82: No bootstrap required, opened a new log
I20260709 12:16:15.072993  6202 consensus_queue.cc:237] T b40ad7108aec4ad2a5aa42acd343c6ac P bd8c877d092949e693c13e9103581f82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.073571  6193 ts_tablet_manager.cc:1403] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82: Time spent bootstrapping tablet: real 0.050s	user 0.014s	sys 0.016s
I20260709 12:16:15.073527  6116 leader_election.cc:304] T d1ec21d99cc2496d84a013cadd31fbc6 P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9468a996c51241cab3d90392e5864642, bd8c877d092949e693c13e9103581f82; no voters: 0ffd67a5e76e46a7b0089718369740bc
I20260709 12:16:15.076596  6193 raft_consensus.cc:359] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.077332  6193 raft_consensus.cc:385] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:15.077621  6193 raft_consensus.cc:740] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8c877d092949e693c13e9103581f82, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.078357  6193 consensus_queue.cc:260] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.076617  6203 raft_consensus.cc:2804] T d1ec21d99cc2496d84a013cadd31fbc6 P 9468a996c51241cab3d90392e5864642 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:15.083657  6193 ts_tablet_manager.cc:1434] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.004s
W20260709 12:16:15.087658  6176 tablet.cc:2406] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:16:15.078737  6196 leader_election.cc:290] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 1 election: Requested vote from peers bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:14.993804  6148 raft_consensus.cc:3060] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.094115  6203 raft_consensus.cc:697] T d1ec21d99cc2496d84a013cadd31fbc6 P 9468a996c51241cab3d90392e5864642 [term 1 LEADER]: Becoming Leader. State: Replica: 9468a996c51241cab3d90392e5864642, State: Running, Role: LEADER
I20260709 12:16:15.105051  6203 consensus_queue.cc:237] T d1ec21d99cc2496d84a013cadd31fbc6 P 9468a996c51241cab3d90392e5864642 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.103514  6201 raft_consensus.cc:493] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.111081  6201 raft_consensus.cc:515] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.118352  5999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ff9114868e646a4967bf3e65485393c" candidate_uuid: "9468a996c51241cab3d90392e5864642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c877d092949e693c13e9103581f82" is_pre_election: true
I20260709 12:16:15.120229  5999 raft_consensus.cc:2468] T 8ff9114868e646a4967bf3e65485393c P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9468a996c51241cab3d90392e5864642 in term 0.
I20260709 12:16:15.123549  6116 leader_election.cc:304] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9468a996c51241cab3d90392e5864642, bd8c877d092949e693c13e9103581f82; no voters: 
I20260709 12:16:15.125464  6247 raft_consensus.cc:2804] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:15.125860  6247 raft_consensus.cc:493] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:15.126215  6247 raft_consensus.cc:3060] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.132148  6075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ff9114868e646a4967bf3e65485393c" candidate_uuid: "9468a996c51241cab3d90392e5864642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffd67a5e76e46a7b0089718369740bc" is_pre_election: true
W20260709 12:16:15.133621  6113 leader_election.cc:343] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:15.134693  6191 tablet_bootstrap.cc:492] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc: No bootstrap required, opened a new log
I20260709 12:16:15.134373  6247 raft_consensus.cc:515] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.135185  6191 ts_tablet_manager.cc:1403] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc: Time spent bootstrapping tablet: real 0.175s	user 0.034s	sys 0.063s
I20260709 12:16:15.137204  6247 leader_election.cc:290] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 1 election: Requested vote from peers bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557), 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019)
I20260709 12:16:15.137866  6191 raft_consensus.cc:359] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.137972  5899 catalog_manager.cc:5697] T b40ad7108aec4ad2a5aa42acd343c6ac P bd8c877d092949e693c13e9103581f82 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c877d092949e693c13e9103581f82 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "bd8c877d092949e693c13e9103581f82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:15.138563  6191 raft_consensus.cc:385] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:15.138757  6148 raft_consensus.cc:2468] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 1.
I20260709 12:16:15.138984  6191 raft_consensus.cc:740] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ffd67a5e76e46a7b0089718369740bc, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.139930  6191 consensus_queue.cc:260] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.141896  5966 leader_election.cc:304] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9468a996c51241cab3d90392e5864642, bd8c877d092949e693c13e9103581f82; no voters: 0ffd67a5e76e46a7b0089718369740bc
I20260709 12:16:15.142477  6191 ts_tablet_manager.cc:1434] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:16:15.143134  6202 raft_consensus.cc:2804] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:15.143412  6191 tablet_bootstrap.cc:492] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc: Bootstrap starting.
I20260709 12:16:15.143647  6202 raft_consensus.cc:697] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c877d092949e693c13e9103581f82, State: Running, Role: LEADER
I20260709 12:16:15.144491  6202 consensus_queue.cc:237] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.145143  5999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ff9114868e646a4967bf3e65485393c" candidate_uuid: "9468a996c51241cab3d90392e5864642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c877d092949e693c13e9103581f82"
I20260709 12:16:15.145740  5999 raft_consensus.cc:3060] T 8ff9114868e646a4967bf3e65485393c P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.146466  6075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ff9114868e646a4967bf3e65485393c" candidate_uuid: "9468a996c51241cab3d90392e5864642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
W20260709 12:16:15.147953  6113 leader_election.cc:343] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:15.149708  6191 tablet_bootstrap.cc:654] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:15.154240  6191 tablet_bootstrap.cc:492] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc: No bootstrap required, opened a new log
I20260709 12:16:15.154183  5999 raft_consensus.cc:2468] T 8ff9114868e646a4967bf3e65485393c P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9468a996c51241cab3d90392e5864642 in term 1.
I20260709 12:16:15.154771  6191 ts_tablet_manager.cc:1403] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc: Time spent bootstrapping tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 12:16:15.155468  6201 leader_election.cc:290] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557), 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019)
I20260709 12:16:15.155555  6116 leader_election.cc:304] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9468a996c51241cab3d90392e5864642, bd8c877d092949e693c13e9103581f82; no voters: 0ffd67a5e76e46a7b0089718369740bc
I20260709 12:16:15.156778  6201 raft_consensus.cc:2804] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:15.157275  6000 raft_consensus.cc:2468] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ffd67a5e76e46a7b0089718369740bc in term 1.
I20260709 12:16:15.157344  6191 raft_consensus.cc:359] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.158124  6191 raft_consensus.cc:385] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:15.157467  6201 raft_consensus.cc:697] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [term 1 LEADER]: Becoming Leader. State: Replica: 9468a996c51241cab3d90392e5864642, State: Running, Role: LEADER
I20260709 12:16:15.158381  6191 raft_consensus.cc:740] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ffd67a5e76e46a7b0089718369740bc, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.159108  6201 consensus_queue.cc:237] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.159062  6191 consensus_queue.cc:260] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.161227  6191 ts_tablet_manager.cc:1434] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:16:15.162191  6191 tablet_bootstrap.cc:492] T d1ec21d99cc2496d84a013cadd31fbc6 P 0ffd67a5e76e46a7b0089718369740bc: Bootstrap starting.
I20260709 12:16:15.162767  5898 catalog_manager.cc:5697] T d1ec21d99cc2496d84a013cadd31fbc6 P 9468a996c51241cab3d90392e5864642 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9468a996c51241cab3d90392e5864642 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "9468a996c51241cab3d90392e5864642" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:15.165154  6196 raft_consensus.cc:493] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.165644  6196 raft_consensus.cc:515] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.167378  6191 tablet_bootstrap.cc:654] T d1ec21d99cc2496d84a013cadd31fbc6 P 0ffd67a5e76e46a7b0089718369740bc: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:15.167788  6196 leader_election.cc:290] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:15.180948  6000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb9f68ad1dc24fc29ed8ad07197a68ac" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c877d092949e693c13e9103581f82" is_pre_election: true
I20260709 12:16:15.183432  6000 raft_consensus.cc:2468] T cb9f68ad1dc24fc29ed8ad07197a68ac P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ffd67a5e76e46a7b0089718369740bc in term 0.
I20260709 12:16:15.182070  6148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb9f68ad1dc24fc29ed8ad07197a68ac" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9468a996c51241cab3d90392e5864642" is_pre_election: true
I20260709 12:16:15.184893  6148 raft_consensus.cc:2468] T cb9f68ad1dc24fc29ed8ad07197a68ac P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ffd67a5e76e46a7b0089718369740bc in term 0.
I20260709 12:16:15.185554  6042 leader_election.cc:304] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc, bd8c877d092949e693c13e9103581f82; no voters: 
I20260709 12:16:15.186931  6196 raft_consensus.cc:2804] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:15.187376  6196 raft_consensus.cc:493] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:15.187703  6196 raft_consensus.cc:3060] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.190107  6191 tablet_bootstrap.cc:492] T d1ec21d99cc2496d84a013cadd31fbc6 P 0ffd67a5e76e46a7b0089718369740bc: No bootstrap required, opened a new log
I20260709 12:16:15.190663  6191 ts_tablet_manager.cc:1403] T d1ec21d99cc2496d84a013cadd31fbc6 P 0ffd67a5e76e46a7b0089718369740bc: Time spent bootstrapping tablet: real 0.029s	user 0.006s	sys 0.020s
I20260709 12:16:15.193126  6191 raft_consensus.cc:359] T d1ec21d99cc2496d84a013cadd31fbc6 P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.193687  6191 raft_consensus.cc:385] T d1ec21d99cc2496d84a013cadd31fbc6 P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:15.193931  6191 raft_consensus.cc:740] T d1ec21d99cc2496d84a013cadd31fbc6 P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ffd67a5e76e46a7b0089718369740bc, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.193312  5899 catalog_manager.cc:5697] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c877d092949e693c13e9103581f82 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "bd8c877d092949e693c13e9103581f82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:15.194531  6191 consensus_queue.cc:260] T d1ec21d99cc2496d84a013cadd31fbc6 P 0ffd67a5e76e46a7b0089718369740bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.196400  6191 ts_tablet_manager.cc:1434] T d1ec21d99cc2496d84a013cadd31fbc6 P 0ffd67a5e76e46a7b0089718369740bc: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:16:15.197315  6196 raft_consensus.cc:515] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.199373  6196 leader_election.cc:290] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 1 election: Requested vote from peers bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:15.199987  6000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb9f68ad1dc24fc29ed8ad07197a68ac" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c877d092949e693c13e9103581f82"
I20260709 12:16:15.200531  6000 raft_consensus.cc:3060] T cb9f68ad1dc24fc29ed8ad07197a68ac P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.200569  6148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb9f68ad1dc24fc29ed8ad07197a68ac" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9468a996c51241cab3d90392e5864642"
I20260709 12:16:15.201195  6148 raft_consensus.cc:3060] T cb9f68ad1dc24fc29ed8ad07197a68ac P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.202732  5898 catalog_manager.cc:5697] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9468a996c51241cab3d90392e5864642 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "9468a996c51241cab3d90392e5864642" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:15.208024  6000 raft_consensus.cc:2468] T cb9f68ad1dc24fc29ed8ad07197a68ac P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ffd67a5e76e46a7b0089718369740bc in term 1.
I20260709 12:16:15.208393  6202 raft_consensus.cc:493] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.208936  6202 raft_consensus.cc:515] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.209754  6042 leader_election.cc:304] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc, bd8c877d092949e693c13e9103581f82; no voters: 
I20260709 12:16:15.211503  6202 leader_election.cc:290] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:15.212324  6148 raft_consensus.cc:2468] T cb9f68ad1dc24fc29ed8ad07197a68ac P 9468a996c51241cab3d90392e5864642 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ffd67a5e76e46a7b0089718369740bc in term 1.
I20260709 12:16:15.212616  6149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f858fa3ce674075aabe68fc8228266d" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9468a996c51241cab3d90392e5864642" is_pre_election: true
I20260709 12:16:15.213294  6149 raft_consensus.cc:2468] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 0.
I20260709 12:16:15.214584  5966 leader_election.cc:304] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9468a996c51241cab3d90392e5864642, bd8c877d092949e693c13e9103581f82; no voters: 
I20260709 12:16:15.215612  6202 raft_consensus.cc:2804] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:15.216050  6202 raft_consensus.cc:493] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:15.216468  6202 raft_consensus.cc:3060] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.216499  6075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f858fa3ce674075aabe68fc8228266d" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffd67a5e76e46a7b0089718369740bc" is_pre_election: true
I20260709 12:16:15.217248  6075 raft_consensus.cc:2468] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 0.
I20260709 12:16:15.210961  6196 raft_consensus.cc:2804] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:15.219120  6196 raft_consensus.cc:697] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc [term 1 LEADER]: Becoming Leader. State: Replica: 0ffd67a5e76e46a7b0089718369740bc, State: Running, Role: LEADER
I20260709 12:16:15.219764  6196 consensus_queue.cc:237] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.223906  6202 raft_consensus.cc:515] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.225742  6202 leader_election.cc:290] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 1 election: Requested vote from peers 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:15.226553  6075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f858fa3ce674075aabe68fc8228266d" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
I20260709 12:16:15.227043  6075 raft_consensus.cc:3060] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.226876  6149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f858fa3ce674075aabe68fc8228266d" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9468a996c51241cab3d90392e5864642"
I20260709 12:16:15.227531  6149 raft_consensus.cc:3060] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.228175  5898 catalog_manager.cc:5697] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2). New cstate: current_term: 1 leader_uuid: "0ffd67a5e76e46a7b0089718369740bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:15.231948  6075 raft_consensus.cc:2468] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 1.
I20260709 12:16:15.233278  5964 leader_election.cc:304] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc, bd8c877d092949e693c13e9103581f82; no voters: 
I20260709 12:16:15.234179  6202 raft_consensus.cc:2804] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:15.234504  6149 raft_consensus.cc:2468] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 1.
I20260709 12:16:15.234619  6202 raft_consensus.cc:697] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 1 LEADER]: Becoming Leader. State: Replica: bd8c877d092949e693c13e9103581f82, State: Running, Role: LEADER
I20260709 12:16:15.236028  6202 consensus_queue.cc:237] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.243456  5898 catalog_manager.cc:5697] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8c877d092949e693c13e9103581f82 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "bd8c877d092949e693c13e9103581f82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:15.269305  6196 raft_consensus.cc:493] T d1ec21d99cc2496d84a013cadd31fbc6 P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.269853  6196 raft_consensus.cc:515] T d1ec21d99cc2496d84a013cadd31fbc6 P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.271919  6196 leader_election.cc:290] T d1ec21d99cc2496d84a013cadd31fbc6 P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:15.272648  5999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ec21d99cc2496d84a013cadd31fbc6" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c877d092949e693c13e9103581f82" is_pre_election: true
I20260709 12:16:15.272917  6149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1ec21d99cc2496d84a013cadd31fbc6" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9468a996c51241cab3d90392e5864642" is_pre_election: true
I20260709 12:16:15.273445  5999 raft_consensus.cc:2393] T d1ec21d99cc2496d84a013cadd31fbc6 P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ffd67a5e76e46a7b0089718369740bc in current term 1: Already voted for candidate 9468a996c51241cab3d90392e5864642 in this term.
I20260709 12:16:15.273813  6201 consensus_queue.cc:1048] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:15.274868  6041 leader_election.cc:304] T d1ec21d99cc2496d84a013cadd31fbc6 P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc; no voters: 9468a996c51241cab3d90392e5864642, bd8c877d092949e693c13e9103581f82
I20260709 12:16:15.275856  6196 raft_consensus.cc:3060] T d1ec21d99cc2496d84a013cadd31fbc6 P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.285857  6195 consensus_queue.cc:1048] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:15.294661  6196 raft_consensus.cc:2749] T d1ec21d99cc2496d84a013cadd31fbc6 P 0ffd67a5e76e46a7b0089718369740bc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:15.346874  6221 consensus_queue.cc:1048] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:15.347718  6202 consensus_queue.cc:1048] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:15.395648  6196 consensus_queue.cc:1048] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:15.409227  6230 consensus_queue.cc:1048] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:15.414629  6148 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "46526944508c443c9c12315136ae866f"
dest_uuid: "9468a996c51241cab3d90392e5864642"
mode: GRACEFUL
new_leader_uuid: "bd8c877d092949e693c13e9103581f82"
 from {username='slave'} at 127.0.0.1:35334
I20260709 12:16:15.415800  6148 raft_consensus.cc:606] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 1 LEADER]: Received request to transfer leadership to TS bd8c877d092949e693c13e9103581f82
I20260709 12:16:15.433434  6075 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cb9f68ad1dc24fc29ed8ad07197a68ac"
dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
mode: GRACEFUL
new_leader_uuid: "bd8c877d092949e693c13e9103581f82"
 from {username='slave'} at 127.0.0.1:43862
I20260709 12:16:15.433940  6075 raft_consensus.cc:606] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc [term 1 LEADER]: Received request to transfer leadership to TS bd8c877d092949e693c13e9103581f82
I20260709 12:16:15.439356  6149 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "71c10f56e0ac401b875d85211fc6ae9f"
dest_uuid: "9468a996c51241cab3d90392e5864642"
mode: GRACEFUL
new_leader_uuid: "bd8c877d092949e693c13e9103581f82"
 from {username='slave'} at 127.0.0.1:35334
I20260709 12:16:15.439913  6149 raft_consensus.cc:606] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 1 LEADER]: Received request to transfer leadership to TS bd8c877d092949e693c13e9103581f82
I20260709 12:16:15.445199  6075 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a275014900ca4f32977e32bb6fc509fd"
dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
mode: GRACEFUL
new_leader_uuid: "bd8c877d092949e693c13e9103581f82"
 from {username='slave'} at 127.0.0.1:43862
I20260709 12:16:15.445911  6075 raft_consensus.cc:606] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [term 1 LEADER]: Received request to transfer leadership to TS bd8c877d092949e693c13e9103581f82
I20260709 12:16:15.454202  6149 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ff9114868e646a4967bf3e65485393c"
dest_uuid: "9468a996c51241cab3d90392e5864642"
mode: GRACEFUL
new_leader_uuid: "0ffd67a5e76e46a7b0089718369740bc"
 from {username='slave'} at 127.0.0.1:35334
I20260709 12:16:15.454732  6149 raft_consensus.cc:606] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [term 1 LEADER]: Received request to transfer leadership to TS 0ffd67a5e76e46a7b0089718369740bc
I20260709 12:16:15.469223  5999 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3f858fa3ce674075aabe68fc8228266d"
dest_uuid: "bd8c877d092949e693c13e9103581f82"
mode: GRACEFUL
new_leader_uuid: "0ffd67a5e76e46a7b0089718369740bc"
 from {username='slave'} at 127.0.0.1:43094
I20260709 12:16:15.469724  5999 raft_consensus.cc:606] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 1 LEADER]: Received request to transfer leadership to TS 0ffd67a5e76e46a7b0089718369740bc
I20260709 12:16:15.475684  6221 consensus_queue.cc:1048] T b40ad7108aec4ad2a5aa42acd343c6ac P bd8c877d092949e693c13e9103581f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:15.476311  5999 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "800b87c938ae4f1596d89f00bb34399d"
dest_uuid: "bd8c877d092949e693c13e9103581f82"
mode: GRACEFUL
new_leader_uuid: "0ffd67a5e76e46a7b0089718369740bc"
 from {username='slave'} at 127.0.0.1:43094
I20260709 12:16:15.476931  5999 raft_consensus.cc:606] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 1 LEADER]: Received request to transfer leadership to TS 0ffd67a5e76e46a7b0089718369740bc
I20260709 12:16:15.477897  6221 consensus_queue.cc:1048] T b40ad7108aec4ad2a5aa42acd343c6ac P bd8c877d092949e693c13e9103581f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:15.483737  5999 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a3442b7a72a34bb3971f1dae9e473e38"
dest_uuid: "bd8c877d092949e693c13e9103581f82"
mode: GRACEFUL
new_leader_uuid: "0ffd67a5e76e46a7b0089718369740bc"
 from {username='slave'} at 127.0.0.1:43094
I20260709 12:16:15.484434  5999 raft_consensus.cc:606] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82 [term 1 LEADER]: Received request to transfer leadership to TS 0ffd67a5e76e46a7b0089718369740bc
I20260709 12:16:15.529196  6230 raft_consensus.cc:493] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.529608  6230 raft_consensus.cc:515] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.532251  6230 leader_election.cc:290] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:15.532941  5999 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ff9114868e646a4967bf3e65485393c" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd8c877d092949e693c13e9103581f82" is_pre_election: true
I20260709 12:16:15.533726  5999 raft_consensus.cc:2393] T 8ff9114868e646a4967bf3e65485393c P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ffd67a5e76e46a7b0089718369740bc in current term 1: Already voted for candidate 9468a996c51241cab3d90392e5864642 in this term.
I20260709 12:16:15.533972  6149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ff9114868e646a4967bf3e65485393c" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9468a996c51241cab3d90392e5864642" is_pre_election: true
I20260709 12:16:15.535375  6041 leader_election.cc:304] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc; no voters: 9468a996c51241cab3d90392e5864642, bd8c877d092949e693c13e9103581f82
I20260709 12:16:15.536458  6230 raft_consensus.cc:3060] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:15.542371  6230 raft_consensus.cc:2749] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:15.546962  6221 consensus_queue.cc:1048] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:15.559085  6221 consensus_queue.cc:1048] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:15.559193  6195 consensus_queue.cc:1048] T d1ec21d99cc2496d84a013cadd31fbc6 P 9468a996c51241cab3d90392e5864642 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:15.568583  6245 consensus_queue.cc:1048] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:15.572608  6221 raft_consensus.cc:993] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82: : Instructing follower 0ffd67a5e76e46a7b0089718369740bc to start an election
I20260709 12:16:15.573318  6221 raft_consensus.cc:1081] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 1 LEADER]: Signalling peer 0ffd67a5e76e46a7b0089718369740bc to start an election
I20260709 12:16:15.574986  6076 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "800b87c938ae4f1596d89f00bb34399d"
dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
 from {username='slave'} at 127.0.0.1:43850
I20260709 12:16:15.575856  6076 raft_consensus.cc:493] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:15.576267  6076 raft_consensus.cc:3060] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:15.576500  6247 consensus_queue.cc:1048] T d1ec21d99cc2496d84a013cadd31fbc6 P 9468a996c51241cab3d90392e5864642 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:15.585971  6076 raft_consensus.cc:515] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.590078  6001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "800b87c938ae4f1596d89f00bb34399d" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bd8c877d092949e693c13e9103581f82"
I20260709 12:16:15.590760  6001 raft_consensus.cc:3055] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:15.591111  6001 raft_consensus.cc:740] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bd8c877d092949e693c13e9103581f82, State: Running, Role: LEADER
I20260709 12:16:15.588899  6076 leader_election.cc:290] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 2 election: Requested vote from peers bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:15.592724  6149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "800b87c938ae4f1596d89f00bb34399d" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9468a996c51241cab3d90392e5864642"
I20260709 12:16:15.593355  6149 raft_consensus.cc:3060] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:15.591892  6001 consensus_queue.cc:260] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.601629  6001 raft_consensus.cc:3060] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:15.608520  6247 consensus_queue.cc:1048] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:15.609022  6149 raft_consensus.cc:2468] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ffd67a5e76e46a7b0089718369740bc in term 2.
I20260709 12:16:15.609359  6001 raft_consensus.cc:2468] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ffd67a5e76e46a7b0089718369740bc in term 2.
I20260709 12:16:15.610255  6041 leader_election.cc:304] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc, 9468a996c51241cab3d90392e5864642; no voters: 
I20260709 12:16:15.611110  6234 raft_consensus.cc:2804] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:15.611580  6234 raft_consensus.cc:697] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [term 2 LEADER]: Becoming Leader. State: Replica: 0ffd67a5e76e46a7b0089718369740bc, State: Running, Role: LEADER
I20260709 12:16:15.612421  6234 consensus_queue.cc:237] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.626207  6247 raft_consensus.cc:993] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642: : Instructing follower bd8c877d092949e693c13e9103581f82 to start an election
I20260709 12:16:15.626674  6247 raft_consensus.cc:1081] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 1 LEADER]: Signalling peer bd8c877d092949e693c13e9103581f82 to start an election
I20260709 12:16:15.629292  6000 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "71c10f56e0ac401b875d85211fc6ae9f"
dest_uuid: "bd8c877d092949e693c13e9103581f82"
 from {username='slave'} at 127.0.0.1:43084
I20260709 12:16:15.629849  6000 raft_consensus.cc:493] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:15.630200  6000 raft_consensus.cc:3060] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:15.637499  6000 raft_consensus.cc:515] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.639688  6000 leader_election.cc:290] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 2 election: Requested vote from peers 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:15.639854  6201 raft_consensus.cc:993] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642: : Instructing follower 0ffd67a5e76e46a7b0089718369740bc to start an election
I20260709 12:16:15.640383  6201 raft_consensus.cc:1081] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [term 1 LEADER]: Signalling peer 0ffd67a5e76e46a7b0089718369740bc to start an election
I20260709 12:16:15.640022  5898 catalog_manager.cc:5697] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc reported cstate change: term changed from 1 to 2, leader changed from bd8c877d092949e693c13e9103581f82 (127.5.58.1) to 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2). New cstate: current_term: 2 leader_uuid: "0ffd67a5e76e46a7b0089718369740bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:15.641984  6075 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71c10f56e0ac401b875d85211fc6ae9f" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
I20260709 12:16:15.642213  6149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71c10f56e0ac401b875d85211fc6ae9f" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9468a996c51241cab3d90392e5864642"
I20260709 12:16:15.642218  6076 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8ff9114868e646a4967bf3e65485393c"
dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
 from {username='slave'} at 127.0.0.1:43838
I20260709 12:16:15.642587  6075 raft_consensus.cc:3060] T 71c10f56e0ac401b875d85211fc6ae9f P 0ffd67a5e76e46a7b0089718369740bc [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:15.642777  6149 raft_consensus.cc:3055] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:15.642911  6076 raft_consensus.cc:493] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:15.643193  6149 raft_consensus.cc:740] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9468a996c51241cab3d90392e5864642, State: Running, Role: LEADER
I20260709 12:16:15.643352  6076 raft_consensus.cc:3060] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:15.644032  6149 consensus_queue.cc:260] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.645478  6149 raft_consensus.cc:3060] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:15.650019  6075 raft_consensus.cc:2468] T 71c10f56e0ac401b875d85211fc6ae9f P 0ffd67a5e76e46a7b0089718369740bc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 2.
I20260709 12:16:15.651355  5964 leader_election.cc:304] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc, bd8c877d092949e693c13e9103581f82; no voters: 
I20260709 12:16:15.651809  6076 raft_consensus.cc:515] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.652222  6221 raft_consensus.cc:2804] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:15.652763  6221 raft_consensus.cc:697] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [term 2 LEADER]: Becoming Leader. State: Replica: bd8c877d092949e693c13e9103581f82, State: Running, Role: LEADER
I20260709 12:16:15.652945  6149 raft_consensus.cc:2468] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 2.
I20260709 12:16:15.653777  6221 consensus_queue.cc:237] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.655076  6076 leader_election.cc:290] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 2 election: Requested vote from peers bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:15.655361  6000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ff9114868e646a4967bf3e65485393c" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bd8c877d092949e693c13e9103581f82"
I20260709 12:16:15.655815  6149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ff9114868e646a4967bf3e65485393c" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9468a996c51241cab3d90392e5864642"
I20260709 12:16:15.656023  6000 raft_consensus.cc:3060] T 8ff9114868e646a4967bf3e65485393c P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:15.656592  6149 raft_consensus.cc:3055] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:15.656932  6149 raft_consensus.cc:740] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9468a996c51241cab3d90392e5864642, State: Running, Role: LEADER
I20260709 12:16:15.657743  6149 consensus_queue.cc:260] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.658972  6149 raft_consensus.cc:3060] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:15.663281  6000 raft_consensus.cc:2468] T 8ff9114868e646a4967bf3e65485393c P bd8c877d092949e693c13e9103581f82 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ffd67a5e76e46a7b0089718369740bc in term 2.
I20260709 12:16:15.664734  6042 leader_election.cc:304] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc, bd8c877d092949e693c13e9103581f82; no voters: 
I20260709 12:16:15.665613  6234 raft_consensus.cc:2804] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:15.665997  6149 raft_consensus.cc:2468] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ffd67a5e76e46a7b0089718369740bc in term 2.
I20260709 12:16:15.666186  6234 raft_consensus.cc:697] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc [term 2 LEADER]: Becoming Leader. State: Replica: 0ffd67a5e76e46a7b0089718369740bc, State: Running, Role: LEADER
I20260709 12:16:15.667156  6234 consensus_queue.cc:237] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.667514  5897 catalog_manager.cc:5697] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 reported cstate change: term changed from 1 to 2, leader changed from 9468a996c51241cab3d90392e5864642 (127.5.58.3) to bd8c877d092949e693c13e9103581f82 (127.5.58.1). New cstate: current_term: 2 leader_uuid: "bd8c877d092949e693c13e9103581f82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:15.675372  5898 catalog_manager.cc:5697] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc reported cstate change: term changed from 1 to 2, leader changed from 9468a996c51241cab3d90392e5864642 (127.5.58.3) to 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2). New cstate: current_term: 2 leader_uuid: "0ffd67a5e76e46a7b0089718369740bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:15.706038  6234 consensus_queue.cc:1048] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:15.712802  6230 raft_consensus.cc:993] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc: : Instructing follower bd8c877d092949e693c13e9103581f82 to start an election
I20260709 12:16:15.713328  6196 raft_consensus.cc:1081] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc [term 1 LEADER]: Signalling peer bd8c877d092949e693c13e9103581f82 to start an election
I20260709 12:16:15.714977  6001 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cb9f68ad1dc24fc29ed8ad07197a68ac"
dest_uuid: "bd8c877d092949e693c13e9103581f82"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:16:15.715543  6001 raft_consensus.cc:493] T cb9f68ad1dc24fc29ed8ad07197a68ac P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:15.715916  6001 raft_consensus.cc:3060] T cb9f68ad1dc24fc29ed8ad07197a68ac P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:15.721035  6234 consensus_queue.cc:1048] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:15.725808  6001 raft_consensus.cc:515] T cb9f68ad1dc24fc29ed8ad07197a68ac P bd8c877d092949e693c13e9103581f82 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.729063  6076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb9f68ad1dc24fc29ed8ad07197a68ac" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
I20260709 12:16:15.729681  6076 raft_consensus.cc:3055] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:15.729990  6076 raft_consensus.cc:740] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ffd67a5e76e46a7b0089718369740bc, State: Running, Role: LEADER
I20260709 12:16:15.730664  6149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb9f68ad1dc24fc29ed8ad07197a68ac" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9468a996c51241cab3d90392e5864642"
I20260709 12:16:15.731244  6149 raft_consensus.cc:3060] T cb9f68ad1dc24fc29ed8ad07197a68ac P 9468a996c51241cab3d90392e5864642 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:15.730804  6076 consensus_queue.cc:260] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
W20260709 12:16:15.731782  6234 consensus_queue.cc:1175] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9468a996c51241cab3d90392e5864642" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260709 12:16:15.733605  6001 leader_election.cc:290] T cb9f68ad1dc24fc29ed8ad07197a68ac P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 2 election: Requested vote from peers 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:15.735716  6076 raft_consensus.cc:3060] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:15.738539  6149 raft_consensus.cc:2468] T cb9f68ad1dc24fc29ed8ad07197a68ac P 9468a996c51241cab3d90392e5864642 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 2.
I20260709 12:16:15.739996  5966 leader_election.cc:304] T cb9f68ad1dc24fc29ed8ad07197a68ac P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9468a996c51241cab3d90392e5864642, bd8c877d092949e693c13e9103581f82; no voters: 
I20260709 12:16:15.740906  6221 raft_consensus.cc:2804] T cb9f68ad1dc24fc29ed8ad07197a68ac P bd8c877d092949e693c13e9103581f82 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:15.741390  6221 raft_consensus.cc:697] T cb9f68ad1dc24fc29ed8ad07197a68ac P bd8c877d092949e693c13e9103581f82 [term 2 LEADER]: Becoming Leader. State: Replica: bd8c877d092949e693c13e9103581f82, State: Running, Role: LEADER
I20260709 12:16:15.742197  6221 consensus_queue.cc:237] T cb9f68ad1dc24fc29ed8ad07197a68ac P bd8c877d092949e693c13e9103581f82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.744019  6076 raft_consensus.cc:2468] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 2.
I20260709 12:16:15.751188  5897 catalog_manager.cc:5697] T cb9f68ad1dc24fc29ed8ad07197a68ac P bd8c877d092949e693c13e9103581f82 reported cstate change: term changed from 1 to 2, leader changed from 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2) to bd8c877d092949e693c13e9103581f82 (127.5.58.1). New cstate: current_term: 2 leader_uuid: "bd8c877d092949e693c13e9103581f82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:15.774937  6221 consensus_queue.cc:1048] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:15.787101  6202 consensus_queue.cc:1048] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:15.796232  6202 raft_consensus.cc:993] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82: : Instructing follower 0ffd67a5e76e46a7b0089718369740bc to start an election
I20260709 12:16:15.796865  6202 raft_consensus.cc:1081] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 1 LEADER]: Signalling peer 0ffd67a5e76e46a7b0089718369740bc to start an election
I20260709 12:16:15.798859  6076 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3f858fa3ce674075aabe68fc8228266d"
dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
 from {username='slave'} at 127.0.0.1:43850
I20260709 12:16:15.799510  6076 raft_consensus.cc:493] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:15.799852  6196 raft_consensus.cc:993] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc: : Instructing follower bd8c877d092949e693c13e9103581f82 to start an election
I20260709 12:16:15.800144  6076 raft_consensus.cc:3060] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:15.800493  6234 raft_consensus.cc:1081] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [term 1 LEADER]: Signalling peer bd8c877d092949e693c13e9103581f82 to start an election
I20260709 12:16:15.802367  6001 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a275014900ca4f32977e32bb6fc509fd"
dest_uuid: "bd8c877d092949e693c13e9103581f82"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:16:15.802942  6001 raft_consensus.cc:493] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:15.803253  6001 raft_consensus.cc:3060] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:15.805737  6076 raft_consensus.cc:515] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.807500  6076 leader_election.cc:290] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 2 election: Requested vote from peers bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:15.808342  6000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f858fa3ce674075aabe68fc8228266d" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bd8c877d092949e693c13e9103581f82"
I20260709 12:16:15.808583  6149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f858fa3ce674075aabe68fc8228266d" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9468a996c51241cab3d90392e5864642"
I20260709 12:16:15.808965  6000 raft_consensus.cc:3055] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:15.809093  6149 raft_consensus.cc:3060] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:15.809365  6000 raft_consensus.cc:740] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bd8c877d092949e693c13e9103581f82, State: Running, Role: LEADER
I20260709 12:16:15.810514  6001 raft_consensus.cc:515] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.810177  6000 consensus_queue.cc:260] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.812726  6001 leader_election.cc:290] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 2 election: Requested vote from peers 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:15.813733  6148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a275014900ca4f32977e32bb6fc509fd" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9468a996c51241cab3d90392e5864642"
I20260709 12:16:15.814308  6148 raft_consensus.cc:3060] T a275014900ca4f32977e32bb6fc509fd P 9468a996c51241cab3d90392e5864642 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:15.814621  6000 raft_consensus.cc:3060] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:15.817385  6149 raft_consensus.cc:2468] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ffd67a5e76e46a7b0089718369740bc in term 2.
I20260709 12:16:15.818630  6041 leader_election.cc:304] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc, 9468a996c51241cab3d90392e5864642; no voters: 
I20260709 12:16:15.819510  6234 raft_consensus.cc:2804] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:15.819999  6234 raft_consensus.cc:697] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [term 2 LEADER]: Becoming Leader. State: Replica: 0ffd67a5e76e46a7b0089718369740bc, State: Running, Role: LEADER
I20260709 12:16:15.821444  6000 raft_consensus.cc:2468] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ffd67a5e76e46a7b0089718369740bc in term 2.
I20260709 12:16:15.821565  6148 raft_consensus.cc:2468] T a275014900ca4f32977e32bb6fc509fd P 9468a996c51241cab3d90392e5864642 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 2.
I20260709 12:16:15.821035  6234 consensus_queue.cc:237] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.823316  6076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a275014900ca4f32977e32bb6fc509fd" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
I20260709 12:16:15.824112  6076 raft_consensus.cc:3055] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:15.824434  6076 raft_consensus.cc:740] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ffd67a5e76e46a7b0089718369740bc, State: Running, Role: LEADER
I20260709 12:16:15.825165  5966 leader_election.cc:304] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9468a996c51241cab3d90392e5864642, bd8c877d092949e693c13e9103581f82; no voters: 
I20260709 12:16:15.825202  6076 consensus_queue.cc:260] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.826156  6221 raft_consensus.cc:2804] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:15.826606  6221 raft_consensus.cc:697] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82 [term 2 LEADER]: Becoming Leader. State: Replica: bd8c877d092949e693c13e9103581f82, State: Running, Role: LEADER
I20260709 12:16:15.826958  6076 raft_consensus.cc:3060] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:15.827442  6221 consensus_queue.cc:237] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.830699  5898 catalog_manager.cc:5697] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc reported cstate change: term changed from 1 to 2, leader changed from bd8c877d092949e693c13e9103581f82 (127.5.58.1) to 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2). New cstate: current_term: 2 leader_uuid: "0ffd67a5e76e46a7b0089718369740bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:15.834005  6076 raft_consensus.cc:2468] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 2.
I20260709 12:16:15.834949  5897 catalog_manager.cc:5697] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82 reported cstate change: term changed from 1 to 2, leader changed from 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2) to bd8c877d092949e693c13e9103581f82 (127.5.58.1). New cstate: current_term: 2 leader_uuid: "bd8c877d092949e693c13e9103581f82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:15.898353  6202 raft_consensus.cc:993] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82: : Instructing follower 0ffd67a5e76e46a7b0089718369740bc to start an election
I20260709 12:16:15.899101  6221 raft_consensus.cc:1081] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82 [term 1 LEADER]: Signalling peer 0ffd67a5e76e46a7b0089718369740bc to start an election
I20260709 12:16:15.900849  6076 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a3442b7a72a34bb3971f1dae9e473e38"
dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
 from {username='slave'} at 127.0.0.1:43850
I20260709 12:16:15.901537  6076 raft_consensus.cc:493] T a3442b7a72a34bb3971f1dae9e473e38 P 0ffd67a5e76e46a7b0089718369740bc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:15.901924  6076 raft_consensus.cc:3060] T a3442b7a72a34bb3971f1dae9e473e38 P 0ffd67a5e76e46a7b0089718369740bc [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:15.902087  6201 raft_consensus.cc:993] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642: : Instructing follower bd8c877d092949e693c13e9103581f82 to start an election
I20260709 12:16:15.902638  6247 raft_consensus.cc:1081] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 1 LEADER]: Signalling peer bd8c877d092949e693c13e9103581f82 to start an election
I20260709 12:16:15.904369  6000 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "46526944508c443c9c12315136ae866f"
dest_uuid: "bd8c877d092949e693c13e9103581f82"
 from {username='slave'} at 127.0.0.1:43084
I20260709 12:16:15.904996  6000 raft_consensus.cc:493] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:15.905445  6000 raft_consensus.cc:3060] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:15.907913  6076 raft_consensus.cc:515] T a3442b7a72a34bb3971f1dae9e473e38 P 0ffd67a5e76e46a7b0089718369740bc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.909789  6076 leader_election.cc:290] T a3442b7a72a34bb3971f1dae9e473e38 P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 2 election: Requested vote from peers bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:15.910632  6001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3442b7a72a34bb3971f1dae9e473e38" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bd8c877d092949e693c13e9103581f82"
I20260709 12:16:15.911046  6148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3442b7a72a34bb3971f1dae9e473e38" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9468a996c51241cab3d90392e5864642"
I20260709 12:16:15.910854  6000 raft_consensus.cc:515] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.911340  6001 raft_consensus.cc:3055] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:15.911651  6148 raft_consensus.cc:3060] T a3442b7a72a34bb3971f1dae9e473e38 P 9468a996c51241cab3d90392e5864642 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:15.911743  6001 raft_consensus.cc:740] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bd8c877d092949e693c13e9103581f82, State: Running, Role: LEADER
I20260709 12:16:15.913092  6001 consensus_queue.cc:260] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.914253  6076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46526944508c443c9c12315136ae866f" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
I20260709 12:16:15.914348  6000 leader_election.cc:290] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 2 election: Requested vote from peers 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:15.914995  6076 raft_consensus.cc:3060] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:15.914379  6001 raft_consensus.cc:3060] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:15.915470  6149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46526944508c443c9c12315136ae866f" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9468a996c51241cab3d90392e5864642"
I20260709 12:16:15.916077  6149 raft_consensus.cc:3055] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:15.916400  6149 raft_consensus.cc:740] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9468a996c51241cab3d90392e5864642, State: Running, Role: LEADER
I20260709 12:16:15.918025  6149 consensus_queue.cc:260] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.919239  6149 raft_consensus.cc:3060] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:15.920876  6148 raft_consensus.cc:2468] T a3442b7a72a34bb3971f1dae9e473e38 P 9468a996c51241cab3d90392e5864642 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ffd67a5e76e46a7b0089718369740bc in term 2.
I20260709 12:16:15.922142  6001 raft_consensus.cc:2468] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ffd67a5e76e46a7b0089718369740bc in term 2.
I20260709 12:16:15.922195  6041 leader_election.cc:304] T a3442b7a72a34bb3971f1dae9e473e38 P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc, 9468a996c51241cab3d90392e5864642; no voters: 
I20260709 12:16:15.922668  6076 raft_consensus.cc:2468] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 2.
I20260709 12:16:15.923262  6234 raft_consensus.cc:2804] T a3442b7a72a34bb3971f1dae9e473e38 P 0ffd67a5e76e46a7b0089718369740bc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:15.923780  6234 raft_consensus.cc:697] T a3442b7a72a34bb3971f1dae9e473e38 P 0ffd67a5e76e46a7b0089718369740bc [term 2 LEADER]: Becoming Leader. State: Replica: 0ffd67a5e76e46a7b0089718369740bc, State: Running, Role: LEADER
I20260709 12:16:15.923874  5964 leader_election.cc:304] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc, bd8c877d092949e693c13e9103581f82; no voters: 
I20260709 12:16:15.924748  6221 raft_consensus.cc:2804] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:15.925364  6221 raft_consensus.cc:697] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 2 LEADER]: Becoming Leader. State: Replica: bd8c877d092949e693c13e9103581f82, State: Running, Role: LEADER
I20260709 12:16:15.924764  6234 consensus_queue.cc:237] T a3442b7a72a34bb3971f1dae9e473e38 P 0ffd67a5e76e46a7b0089718369740bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.927052  6149 raft_consensus.cc:2468] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 2.
I20260709 12:16:15.926617  6221 consensus_queue.cc:237] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:15.934103  5897 catalog_manager.cc:5697] T a3442b7a72a34bb3971f1dae9e473e38 P 0ffd67a5e76e46a7b0089718369740bc reported cstate change: term changed from 1 to 2, leader changed from bd8c877d092949e693c13e9103581f82 (127.5.58.1) to 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2). New cstate: current_term: 2 leader_uuid: "0ffd67a5e76e46a7b0089718369740bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:15.934931  5898 catalog_manager.cc:5697] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 reported cstate change: term changed from 1 to 2, leader changed from 9468a996c51241cab3d90392e5864642 (127.5.58.3) to bd8c877d092949e693c13e9103581f82 (127.5.58.1). New cstate: current_term: 2 leader_uuid: "bd8c877d092949e693c13e9103581f82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:15.999203  6001 raft_consensus.cc:1275] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 2 FOLLOWER]: Refusing update from remote peer 0ffd67a5e76e46a7b0089718369740bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:16.000638  6234 consensus_queue.cc:1048] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:16.008728  6148 raft_consensus.cc:1275] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642 [term 2 FOLLOWER]: Refusing update from remote peer 0ffd67a5e76e46a7b0089718369740bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:16.010281  6230 consensus_queue.cc:1048] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:16.060264  6076 raft_consensus.cc:1275] T 71c10f56e0ac401b875d85211fc6ae9f P 0ffd67a5e76e46a7b0089718369740bc [term 2 FOLLOWER]: Refusing update from remote peer bd8c877d092949e693c13e9103581f82: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:16.061169  6148 raft_consensus.cc:1275] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [term 2 FOLLOWER]: Refusing update from remote peer 0ffd67a5e76e46a7b0089718369740bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:16.062172  6221 consensus_queue.cc:1048] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:16.062449  6230 consensus_queue.cc:1048] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:16.070519  6001 raft_consensus.cc:1275] T 8ff9114868e646a4967bf3e65485393c P bd8c877d092949e693c13e9103581f82 [term 2 FOLLOWER]: Refusing update from remote peer 0ffd67a5e76e46a7b0089718369740bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:16.072013  6148 raft_consensus.cc:1275] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 2 FOLLOWER]: Refusing update from remote peer bd8c877d092949e693c13e9103581f82: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:16.073041  6232 consensus_queue.cc:1048] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:16.074894  6202 consensus_queue.cc:1048] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:16.185434  6076 raft_consensus.cc:1275] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc [term 2 FOLLOWER]: Refusing update from remote peer bd8c877d092949e693c13e9103581f82: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:16.186944  6279 consensus_queue.cc:1048] T cb9f68ad1dc24fc29ed8ad07197a68ac P bd8c877d092949e693c13e9103581f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:16.196453  6149 raft_consensus.cc:1275] T cb9f68ad1dc24fc29ed8ad07197a68ac P 9468a996c51241cab3d90392e5864642 [term 2 FOLLOWER]: Refusing update from remote peer bd8c877d092949e693c13e9103581f82: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:16.197913  6237 consensus_queue.cc:1048] T cb9f68ad1dc24fc29ed8ad07197a68ac P bd8c877d092949e693c13e9103581f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:16.209856  6001 raft_consensus.cc:1275] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 2 FOLLOWER]: Refusing update from remote peer 0ffd67a5e76e46a7b0089718369740bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:16.211153  6230 consensus_queue.cc:1048] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:16.218928  6149 raft_consensus.cc:1275] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642 [term 2 FOLLOWER]: Refusing update from remote peer 0ffd67a5e76e46a7b0089718369740bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:16.220455  6230 consensus_queue.cc:1048] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:16.308048  6149 raft_consensus.cc:1275] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 2 FOLLOWER]: Refusing update from remote peer bd8c877d092949e693c13e9103581f82: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:16.309450  6237 consensus_queue.cc:1048] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:16.314075  6148 raft_consensus.cc:1275] T a275014900ca4f32977e32bb6fc509fd P 9468a996c51241cab3d90392e5864642 [term 2 FOLLOWER]: Refusing update from remote peer bd8c877d092949e693c13e9103581f82: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:16.315739  6237 consensus_queue.cc:1048] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:16.319680  6076 raft_consensus.cc:1275] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [term 2 FOLLOWER]: Refusing update from remote peer bd8c877d092949e693c13e9103581f82: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:16.321538  6237 consensus_queue.cc:1048] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:16.327201  6075 raft_consensus.cc:1275] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [term 2 FOLLOWER]: Refusing update from remote peer bd8c877d092949e693c13e9103581f82: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:16.328998  6237 consensus_queue.cc:1048] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:16.387110  6001 raft_consensus.cc:1275] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82 [term 2 FOLLOWER]: Refusing update from remote peer 0ffd67a5e76e46a7b0089718369740bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:16.388697  6232 consensus_queue.cc:1048] T a3442b7a72a34bb3971f1dae9e473e38 P 0ffd67a5e76e46a7b0089718369740bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:16.398073  6149 raft_consensus.cc:1275] T a3442b7a72a34bb3971f1dae9e473e38 P 9468a996c51241cab3d90392e5864642 [term 2 FOLLOWER]: Refusing update from remote peer 0ffd67a5e76e46a7b0089718369740bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:16.399784  6196 consensus_queue.cc:1048] T a3442b7a72a34bb3971f1dae9e473e38 P 0ffd67a5e76e46a7b0089718369740bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:18.531656  5352 auto_leader_rebalancer-test.cc:365] The leader distribution is 
I20260709 12:16:18.532496  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:18.549012  6001 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "46526944508c443c9c12315136ae866f"
dest_uuid: "bd8c877d092949e693c13e9103581f82"
mode: GRACEFUL
new_leader_uuid: "9468a996c51241cab3d90392e5864642"
 from {username='slave'} at 127.0.0.1:43106
I20260709 12:16:18.549523  6001 raft_consensus.cc:606] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 2 LEADER]: Received request to transfer leadership to TS 9468a996c51241cab3d90392e5864642
I20260709 12:16:18.551927  6001 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "71c10f56e0ac401b875d85211fc6ae9f"
dest_uuid: "bd8c877d092949e693c13e9103581f82"
mode: GRACEFUL
new_leader_uuid: "9468a996c51241cab3d90392e5864642"
 from {username='slave'} at 127.0.0.1:43106
I20260709 12:16:18.552480  6001 raft_consensus.cc:606] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [term 2 LEADER]: Received request to transfer leadership to TS 9468a996c51241cab3d90392e5864642
I20260709 12:16:18.553606  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:16:18.554131  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:18.729032  6288 raft_consensus.cc:993] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82: : Instructing follower 9468a996c51241cab3d90392e5864642 to start an election
I20260709 12:16:18.729844  6288 raft_consensus.cc:1081] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [term 2 LEADER]: Signalling peer 9468a996c51241cab3d90392e5864642 to start an election
I20260709 12:16:18.731534  6148 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "71c10f56e0ac401b875d85211fc6ae9f"
dest_uuid: "9468a996c51241cab3d90392e5864642"
 from {username='slave'} at 127.0.0.1:35316
I20260709 12:16:18.732169  6148 raft_consensus.cc:493] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:18.732607  6148 raft_consensus.cc:3060] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:18.738482  6148 raft_consensus.cc:515] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:18.740247  6148 leader_election.cc:290] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 3 election: Requested vote from peers bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557), 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019)
I20260709 12:16:18.741328  6001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71c10f56e0ac401b875d85211fc6ae9f" candidate_uuid: "9468a996c51241cab3d90392e5864642" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bd8c877d092949e693c13e9103581f82"
I20260709 12:16:18.741539  6076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71c10f56e0ac401b875d85211fc6ae9f" candidate_uuid: "9468a996c51241cab3d90392e5864642" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
I20260709 12:16:18.741982  6001 raft_consensus.cc:3055] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:18.742080  6076 raft_consensus.cc:3060] T 71c10f56e0ac401b875d85211fc6ae9f P 0ffd67a5e76e46a7b0089718369740bc [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:18.742415  6001 raft_consensus.cc:740] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bd8c877d092949e693c13e9103581f82, State: Running, Role: LEADER
I20260709 12:16:18.743269  6001 consensus_queue.cc:260] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:18.744576  6001 raft_consensus.cc:3060] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:18.748917  6076 raft_consensus.cc:2468] T 71c10f56e0ac401b875d85211fc6ae9f P 0ffd67a5e76e46a7b0089718369740bc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9468a996c51241cab3d90392e5864642 in term 3.
I20260709 12:16:18.750289  6113 leader_election.cc:304] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc, 9468a996c51241cab3d90392e5864642; no voters: 
I20260709 12:16:18.751045  6285 raft_consensus.cc:2804] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:18.751241  6001 raft_consensus.cc:2468] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9468a996c51241cab3d90392e5864642 in term 3.
I20260709 12:16:18.752548  6285 raft_consensus.cc:697] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 3 LEADER]: Becoming Leader. State: Replica: 9468a996c51241cab3d90392e5864642, State: Running, Role: LEADER
I20260709 12:16:18.753405  6285 consensus_queue.cc:237] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:18.761659  5899 catalog_manager.cc:5697] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 reported cstate change: term changed from 2 to 3, leader changed from bd8c877d092949e693c13e9103581f82 (127.5.58.1) to 9468a996c51241cab3d90392e5864642 (127.5.58.3). New cstate: current_term: 3 leader_uuid: "9468a996c51241cab3d90392e5864642" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:18.870236  6237 raft_consensus.cc:993] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82: : Instructing follower 9468a996c51241cab3d90392e5864642 to start an election
I20260709 12:16:18.870755  6288 raft_consensus.cc:1081] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 2 LEADER]: Signalling peer 9468a996c51241cab3d90392e5864642 to start an election
I20260709 12:16:18.872220  6148 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "46526944508c443c9c12315136ae866f"
dest_uuid: "9468a996c51241cab3d90392e5864642"
 from {username='slave'} at 127.0.0.1:35316
I20260709 12:16:18.872803  6148 raft_consensus.cc:493] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:18.873157  6148 raft_consensus.cc:3060] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:18.878403  6148 raft_consensus.cc:515] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:18.880146  6148 leader_election.cc:290] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 3 election: Requested vote from peers 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019), bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557)
I20260709 12:16:18.881114  6076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46526944508c443c9c12315136ae866f" candidate_uuid: "9468a996c51241cab3d90392e5864642" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
I20260709 12:16:18.881208  6001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46526944508c443c9c12315136ae866f" candidate_uuid: "9468a996c51241cab3d90392e5864642" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bd8c877d092949e693c13e9103581f82"
I20260709 12:16:18.881878  6076 raft_consensus.cc:3060] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:18.881965  6001 raft_consensus.cc:3055] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:18.882383  6001 raft_consensus.cc:740] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bd8c877d092949e693c13e9103581f82, State: Running, Role: LEADER
I20260709 12:16:18.883085  6001 consensus_queue.cc:260] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:18.884089  6001 raft_consensus.cc:3060] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:18.886832  6076 raft_consensus.cc:2468] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9468a996c51241cab3d90392e5864642 in term 3.
I20260709 12:16:18.888104  6113 leader_election.cc:304] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc, 9468a996c51241cab3d90392e5864642; no voters: 
I20260709 12:16:18.888952  6285 raft_consensus.cc:2804] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:18.889623  6285 raft_consensus.cc:697] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 3 LEADER]: Becoming Leader. State: Replica: 9468a996c51241cab3d90392e5864642, State: Running, Role: LEADER
I20260709 12:16:18.890497  6001 raft_consensus.cc:2468] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9468a996c51241cab3d90392e5864642 in term 3.
I20260709 12:16:18.890516  6285 consensus_queue.cc:237] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:18.898159  5899 catalog_manager.cc:5697] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 reported cstate change: term changed from 2 to 3, leader changed from bd8c877d092949e693c13e9103581f82 (127.5.58.1) to 9468a996c51241cab3d90392e5864642 (127.5.58.3). New cstate: current_term: 3 leader_uuid: "9468a996c51241cab3d90392e5864642" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:19.166406  6076 raft_consensus.cc:1275] T 71c10f56e0ac401b875d85211fc6ae9f P 0ffd67a5e76e46a7b0089718369740bc [term 3 FOLLOWER]: Refusing update from remote peer 9468a996c51241cab3d90392e5864642: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:19.168068  6285 consensus_queue.cc:1048] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:19.178037  6001 raft_consensus.cc:1275] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [term 3 FOLLOWER]: Refusing update from remote peer 9468a996c51241cab3d90392e5864642: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:19.179469  6289 consensus_queue.cc:1048] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:19.305642  6076 raft_consensus.cc:1275] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [term 3 FOLLOWER]: Refusing update from remote peer 9468a996c51241cab3d90392e5864642: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:19.307029  6289 consensus_queue.cc:1048] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:19.315860  6001 raft_consensus.cc:1275] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 3 FOLLOWER]: Refusing update from remote peer 9468a996c51241cab3d90392e5864642: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:19.319733  6289 consensus_queue.cc:1048] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:19.555063  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:19.560437  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:19.560880  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:20.561836  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:20.567251  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:20.567699  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:21.568774  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:21.575167  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:21.575583  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:22.576900  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:22.583489  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:22.583920  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:23.585057  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:23.590523  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:23.590957  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:24.591948  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:24.598320  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:24.598775  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:25.599798  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:25.605697  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:25.606154  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:26.607236  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:26.613979  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:26.614454  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:27.615432  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:27.621515  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:27.621954  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:28.656116  5352 auto_leader_rebalancer-test.cc:384] The leader distribution is 
I20260709 12:16:28.660982  6149 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "46526944508c443c9c12315136ae866f"
dest_uuid: "9468a996c51241cab3d90392e5864642"
mode: GRACEFUL
new_leader_uuid: "bd8c877d092949e693c13e9103581f82"
 from {username='slave'} at 127.0.0.1:35334
I20260709 12:16:28.661648  6149 raft_consensus.cc:606] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 3 LEADER]: Received request to transfer leadership to TS bd8c877d092949e693c13e9103581f82
I20260709 12:16:28.670076  6149 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "71c10f56e0ac401b875d85211fc6ae9f"
dest_uuid: "9468a996c51241cab3d90392e5864642"
mode: GRACEFUL
new_leader_uuid: "bd8c877d092949e693c13e9103581f82"
 from {username='slave'} at 127.0.0.1:35334
I20260709 12:16:28.670660  6149 raft_consensus.cc:606] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 3 LEADER]: Received request to transfer leadership to TS bd8c877d092949e693c13e9103581f82
I20260709 12:16:28.682828  6076 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ff9114868e646a4967bf3e65485393c"
dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
mode: GRACEFUL
new_leader_uuid: "bd8c877d092949e693c13e9103581f82"
 from {username='slave'} at 127.0.0.1:43862
I20260709 12:16:28.683378  6076 raft_consensus.cc:606] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc [term 2 LEADER]: Received request to transfer leadership to TS bd8c877d092949e693c13e9103581f82
I20260709 12:16:28.688349  6076 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3f858fa3ce674075aabe68fc8228266d"
dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
mode: GRACEFUL
new_leader_uuid: "bd8c877d092949e693c13e9103581f82"
 from {username='slave'} at 127.0.0.1:43862
I20260709 12:16:28.688840  6076 raft_consensus.cc:606] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [term 2 LEADER]: Received request to transfer leadership to TS bd8c877d092949e693c13e9103581f82
I20260709 12:16:28.693632  6076 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "800b87c938ae4f1596d89f00bb34399d"
dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
mode: GRACEFUL
new_leader_uuid: "bd8c877d092949e693c13e9103581f82"
 from {username='slave'} at 127.0.0.1:43862
I20260709 12:16:28.694113  6076 raft_consensus.cc:606] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [term 2 LEADER]: Received request to transfer leadership to TS bd8c877d092949e693c13e9103581f82
I20260709 12:16:28.731492  6318 raft_consensus.cc:993] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc: : Instructing follower bd8c877d092949e693c13e9103581f82 to start an election
I20260709 12:16:28.731967  6318 raft_consensus.cc:1081] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [term 2 LEADER]: Signalling peer bd8c877d092949e693c13e9103581f82 to start an election
I20260709 12:16:28.733542  6000 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3f858fa3ce674075aabe68fc8228266d"
dest_uuid: "bd8c877d092949e693c13e9103581f82"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:16:28.734000  6000 raft_consensus.cc:493] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:28.734328  6000 raft_consensus.cc:3060] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:28.740005  6000 raft_consensus.cc:515] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:28.741834  6000 leader_election.cc:290] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 3 election: Requested vote from peers 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:28.742769  6076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f858fa3ce674075aabe68fc8228266d" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
I20260709 12:16:28.742988  6149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f858fa3ce674075aabe68fc8228266d" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9468a996c51241cab3d90392e5864642"
I20260709 12:16:28.743448  6076 raft_consensus.cc:3055] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:28.743518  6149 raft_consensus.cc:3060] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:28.743822  6076 raft_consensus.cc:740] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0ffd67a5e76e46a7b0089718369740bc, State: Running, Role: LEADER
I20260709 12:16:28.744683  6076 consensus_queue.cc:260] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:28.745626  6076 raft_consensus.cc:3060] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:28.749485  6149 raft_consensus.cc:2468] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 3.
I20260709 12:16:28.750092  6076 raft_consensus.cc:2468] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 3.
I20260709 12:16:28.750733  5966 leader_election.cc:304] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9468a996c51241cab3d90392e5864642, bd8c877d092949e693c13e9103581f82; no voters: 
I20260709 12:16:28.751619  6288 raft_consensus.cc:2804] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:28.752537  6288 raft_consensus.cc:697] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 3 LEADER]: Becoming Leader. State: Replica: bd8c877d092949e693c13e9103581f82, State: Running, Role: LEADER
I20260709 12:16:28.753257  6288 consensus_queue.cc:237] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:28.759616  5897 catalog_manager.cc:5697] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 reported cstate change: term changed from 2 to 3, leader changed from 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2) to bd8c877d092949e693c13e9103581f82 (127.5.58.1). New cstate: current_term: 3 leader_uuid: "bd8c877d092949e693c13e9103581f82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:28.765770  6313 raft_consensus.cc:993] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc: : Instructing follower bd8c877d092949e693c13e9103581f82 to start an election
I20260709 12:16:28.766320  6318 raft_consensus.cc:1081] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc [term 2 LEADER]: Signalling peer bd8c877d092949e693c13e9103581f82 to start an election
I20260709 12:16:28.768299  6000 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8ff9114868e646a4967bf3e65485393c"
dest_uuid: "bd8c877d092949e693c13e9103581f82"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:16:28.768908  6000 raft_consensus.cc:493] T 8ff9114868e646a4967bf3e65485393c P bd8c877d092949e693c13e9103581f82 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:28.769294  6000 raft_consensus.cc:3060] T 8ff9114868e646a4967bf3e65485393c P bd8c877d092949e693c13e9103581f82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:28.775490  6000 raft_consensus.cc:515] T 8ff9114868e646a4967bf3e65485393c P bd8c877d092949e693c13e9103581f82 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:28.777837  6000 leader_election.cc:290] T 8ff9114868e646a4967bf3e65485393c P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 3 election: Requested vote from peers 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:28.777956  6076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ff9114868e646a4967bf3e65485393c" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
I20260709 12:16:28.778337  6149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ff9114868e646a4967bf3e65485393c" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9468a996c51241cab3d90392e5864642"
I20260709 12:16:28.778620  6076 raft_consensus.cc:3055] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:28.778880  6149 raft_consensus.cc:3060] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:28.778973  6076 raft_consensus.cc:740] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0ffd67a5e76e46a7b0089718369740bc, State: Running, Role: LEADER
I20260709 12:16:28.779961  6076 consensus_queue.cc:260] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:28.781364  6076 raft_consensus.cc:3060] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:28.785801  6149 raft_consensus.cc:2468] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 3.
I20260709 12:16:28.787114  5966 leader_election.cc:304] T 8ff9114868e646a4967bf3e65485393c P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9468a996c51241cab3d90392e5864642, bd8c877d092949e693c13e9103581f82; no voters: 
I20260709 12:16:28.787755  6076 raft_consensus.cc:2468] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 3.
I20260709 12:16:28.787966  6288 raft_consensus.cc:2804] T 8ff9114868e646a4967bf3e65485393c P bd8c877d092949e693c13e9103581f82 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:28.788550  6288 raft_consensus.cc:697] T 8ff9114868e646a4967bf3e65485393c P bd8c877d092949e693c13e9103581f82 [term 3 LEADER]: Becoming Leader. State: Replica: bd8c877d092949e693c13e9103581f82, State: Running, Role: LEADER
I20260709 12:16:28.789541  6288 consensus_queue.cc:237] T 8ff9114868e646a4967bf3e65485393c P bd8c877d092949e693c13e9103581f82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:28.796103  5897 catalog_manager.cc:5697] T 8ff9114868e646a4967bf3e65485393c P bd8c877d092949e693c13e9103581f82 reported cstate change: term changed from 2 to 3, leader changed from 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2) to bd8c877d092949e693c13e9103581f82 (127.5.58.1). New cstate: current_term: 3 leader_uuid: "bd8c877d092949e693c13e9103581f82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:28.843868  6326 raft_consensus.cc:993] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642: : Instructing follower bd8c877d092949e693c13e9103581f82 to start an election
I20260709 12:16:28.844329  6326 raft_consensus.cc:1081] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 3 LEADER]: Signalling peer bd8c877d092949e693c13e9103581f82 to start an election
I20260709 12:16:28.845803  6000 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "71c10f56e0ac401b875d85211fc6ae9f"
dest_uuid: "bd8c877d092949e693c13e9103581f82"
 from {username='slave'} at 127.0.0.1:43084
I20260709 12:16:28.846290  6000 raft_consensus.cc:493] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:28.846606  6000 raft_consensus.cc:3060] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:28.851125  6000 raft_consensus.cc:515] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:28.853168  6000 leader_election.cc:290] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 4 election: Requested vote from peers 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:28.853657  6076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71c10f56e0ac401b875d85211fc6ae9f" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
I20260709 12:16:28.853787  6149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71c10f56e0ac401b875d85211fc6ae9f" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9468a996c51241cab3d90392e5864642"
I20260709 12:16:28.854660  6149 raft_consensus.cc:3055] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:16:28.854318  6076 raft_consensus.cc:3060] T 71c10f56e0ac401b875d85211fc6ae9f P 0ffd67a5e76e46a7b0089718369740bc [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:28.854909  6149 raft_consensus.cc:740] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9468a996c51241cab3d90392e5864642, State: Running, Role: LEADER
I20260709 12:16:28.856235  6149 consensus_queue.cc:260] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:28.857439  6149 raft_consensus.cc:3060] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:28.861502  6076 raft_consensus.cc:2468] T 71c10f56e0ac401b875d85211fc6ae9f P 0ffd67a5e76e46a7b0089718369740bc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 4.
I20260709 12:16:28.862813  5964 leader_election.cc:304] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc, bd8c877d092949e693c13e9103581f82; no voters: 
I20260709 12:16:28.863404  6149 raft_consensus.cc:2468] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 4.
I20260709 12:16:28.863572  6288 raft_consensus.cc:2804] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:16:28.864094  6288 raft_consensus.cc:697] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [term 4 LEADER]: Becoming Leader. State: Replica: bd8c877d092949e693c13e9103581f82, State: Running, Role: LEADER
I20260709 12:16:28.864879  6288 consensus_queue.cc:237] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:28.873016  6309 raft_consensus.cc:993] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642: : Instructing follower bd8c877d092949e693c13e9103581f82 to start an election
I20260709 12:16:28.872394  5897 catalog_manager.cc:5697] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 reported cstate change: term changed from 3 to 4, leader changed from 9468a996c51241cab3d90392e5864642 (127.5.58.3) to bd8c877d092949e693c13e9103581f82 (127.5.58.1). New cstate: current_term: 4 leader_uuid: "bd8c877d092949e693c13e9103581f82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:28.873590  6326 raft_consensus.cc:1081] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 3 LEADER]: Signalling peer bd8c877d092949e693c13e9103581f82 to start an election
I20260709 12:16:28.875036  6000 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "46526944508c443c9c12315136ae866f"
dest_uuid: "bd8c877d092949e693c13e9103581f82"
 from {username='slave'} at 127.0.0.1:43084
I20260709 12:16:28.875440  6000 raft_consensus.cc:493] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:28.875710  6000 raft_consensus.cc:3060] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:28.880787  6000 raft_consensus.cc:515] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:28.882272  6000 leader_election.cc:290] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 4 election: Requested vote from peers 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:28.883077  6076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46526944508c443c9c12315136ae866f" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
I20260709 12:16:28.883348  6149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46526944508c443c9c12315136ae866f" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9468a996c51241cab3d90392e5864642"
I20260709 12:16:28.883615  6076 raft_consensus.cc:3060] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:28.883944  6149 raft_consensus.cc:3055] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:16:28.884238  6149 raft_consensus.cc:740] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9468a996c51241cab3d90392e5864642, State: Running, Role: LEADER
I20260709 12:16:28.884980  6149 consensus_queue.cc:260] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:28.885906  6149 raft_consensus.cc:3060] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:28.888784  6076 raft_consensus.cc:2468] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 4.
I20260709 12:16:28.889760  5964 leader_election.cc:304] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc, bd8c877d092949e693c13e9103581f82; no voters: 
I20260709 12:16:28.890479  6288 raft_consensus.cc:2804] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:16:28.890846  6288 raft_consensus.cc:697] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 4 LEADER]: Becoming Leader. State: Replica: bd8c877d092949e693c13e9103581f82, State: Running, Role: LEADER
I20260709 12:16:28.891543  6288 consensus_queue.cc:237] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:28.892283  6149 raft_consensus.cc:2468] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 4.
I20260709 12:16:28.899070  5897 catalog_manager.cc:5697] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 reported cstate change: term changed from 3 to 4, leader changed from 9468a996c51241cab3d90392e5864642 (127.5.58.3) to bd8c877d092949e693c13e9103581f82 (127.5.58.1). New cstate: current_term: 4 leader_uuid: "bd8c877d092949e693c13e9103581f82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:28.990418  6313 raft_consensus.cc:993] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc: : Instructing follower bd8c877d092949e693c13e9103581f82 to start an election
I20260709 12:16:28.991387  6313 raft_consensus.cc:1081] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [term 2 LEADER]: Signalling peer bd8c877d092949e693c13e9103581f82 to start an election
I20260709 12:16:28.993507  6000 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "800b87c938ae4f1596d89f00bb34399d"
dest_uuid: "bd8c877d092949e693c13e9103581f82"
 from {username='slave'} at 127.0.0.1:43086
I20260709 12:16:28.994199  6000 raft_consensus.cc:493] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:28.994642  6000 raft_consensus.cc:3060] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:28.999738  6000 raft_consensus.cc:515] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:29.002658  6000 leader_election.cc:290] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 3 election: Requested vote from peers 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:29.002712  6076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "800b87c938ae4f1596d89f00bb34399d" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
I20260709 12:16:29.003109  6149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "800b87c938ae4f1596d89f00bb34399d" candidate_uuid: "bd8c877d092949e693c13e9103581f82" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9468a996c51241cab3d90392e5864642"
I20260709 12:16:29.003470  6076 raft_consensus.cc:3055] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:29.003687  6149 raft_consensus.cc:3060] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:29.003789  6076 raft_consensus.cc:740] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0ffd67a5e76e46a7b0089718369740bc, State: Running, Role: LEADER
I20260709 12:16:29.004736  6076 consensus_queue.cc:260] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:29.005946  6076 raft_consensus.cc:3060] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:29.011430  6149 raft_consensus.cc:2468] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 3.
I20260709 12:16:29.012979  5966 leader_election.cc:304] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9468a996c51241cab3d90392e5864642, bd8c877d092949e693c13e9103581f82; no voters: 
I20260709 12:16:29.014106  6288 raft_consensus.cc:2804] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:29.014277  6076 raft_consensus.cc:2468] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8c877d092949e693c13e9103581f82 in term 3.
I20260709 12:16:29.016752  6288 raft_consensus.cc:697] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 3 LEADER]: Becoming Leader. State: Replica: bd8c877d092949e693c13e9103581f82, State: Running, Role: LEADER
I20260709 12:16:29.018064  6288 consensus_queue.cc:237] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:29.026317  5897 catalog_manager.cc:5697] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 reported cstate change: term changed from 2 to 3, leader changed from 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2) to bd8c877d092949e693c13e9103581f82 (127.5.58.1). New cstate: current_term: 3 leader_uuid: "bd8c877d092949e693c13e9103581f82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:29.250557  6076 raft_consensus.cc:1275] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [term 3 FOLLOWER]: Refusing update from remote peer bd8c877d092949e693c13e9103581f82: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:29.251748  6288 consensus_queue.cc:1048] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:29.263530  6148 raft_consensus.cc:1275] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 4 FOLLOWER]: Refusing update from remote peer bd8c877d092949e693c13e9103581f82: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:29.264861  6149 raft_consensus.cc:1275] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642 [term 3 FOLLOWER]: Refusing update from remote peer bd8c877d092949e693c13e9103581f82: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:29.265606  6329 consensus_queue.cc:1048] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:29.266278  6319 consensus_queue.cc:1048] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:29.277084  6076 raft_consensus.cc:1275] T 71c10f56e0ac401b875d85211fc6ae9f P 0ffd67a5e76e46a7b0089718369740bc [term 4 FOLLOWER]: Refusing update from remote peer bd8c877d092949e693c13e9103581f82: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:29.278522  6288 consensus_queue.cc:1048] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:29.318897  6076 raft_consensus.cc:1275] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc [term 3 FOLLOWER]: Refusing update from remote peer bd8c877d092949e693c13e9103581f82: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:29.320142  6288 consensus_queue.cc:1048] T 8ff9114868e646a4967bf3e65485393c P bd8c877d092949e693c13e9103581f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:29.329434  6148 raft_consensus.cc:1275] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [term 3 FOLLOWER]: Refusing update from remote peer bd8c877d092949e693c13e9103581f82: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:29.330868  6319 consensus_queue.cc:1048] T 8ff9114868e646a4967bf3e65485393c P bd8c877d092949e693c13e9103581f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:29.403470  6148 raft_consensus.cc:1275] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 4 FOLLOWER]: Refusing update from remote peer bd8c877d092949e693c13e9103581f82: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:29.404908  6329 consensus_queue.cc:1048] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:29.415273  6076 raft_consensus.cc:1275] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [term 4 FOLLOWER]: Refusing update from remote peer bd8c877d092949e693c13e9103581f82: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:29.416620  6329 consensus_queue.cc:1048] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:29.517761  6076 raft_consensus.cc:1275] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [term 3 FOLLOWER]: Refusing update from remote peer bd8c877d092949e693c13e9103581f82: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:29.519326  6288 consensus_queue.cc:1048] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:29.532650  6148 raft_consensus.cc:1275] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642 [term 3 FOLLOWER]: Refusing update from remote peer bd8c877d092949e693c13e9103581f82: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:29.533955  6319 consensus_queue.cc:1048] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:31.743386  5352 auto_leader_rebalancer-test.cc:399] The leader distribution is 
I20260709 12:16:31.744189  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:31.755141  6000 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3f858fa3ce674075aabe68fc8228266d"
dest_uuid: "bd8c877d092949e693c13e9103581f82"
mode: GRACEFUL
new_leader_uuid: "9468a996c51241cab3d90392e5864642"
 from {username='slave'} at 127.0.0.1:43106
I20260709 12:16:31.755749  6000 raft_consensus.cc:606] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 3 LEADER]: Received request to transfer leadership to TS 9468a996c51241cab3d90392e5864642
I20260709 12:16:31.758966  6000 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "46526944508c443c9c12315136ae866f"
dest_uuid: "bd8c877d092949e693c13e9103581f82"
mode: GRACEFUL
new_leader_uuid: "0ffd67a5e76e46a7b0089718369740bc"
 from {username='slave'} at 127.0.0.1:43106
I20260709 12:16:31.759572  6000 raft_consensus.cc:606] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 4 LEADER]: Received request to transfer leadership to TS 0ffd67a5e76e46a7b0089718369740bc
I20260709 12:16:31.762835  6000 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "800b87c938ae4f1596d89f00bb34399d"
dest_uuid: "bd8c877d092949e693c13e9103581f82"
mode: GRACEFUL
new_leader_uuid: "9468a996c51241cab3d90392e5864642"
 from {username='slave'} at 127.0.0.1:43106
I20260709 12:16:31.763422  6000 raft_consensus.cc:606] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 3 LEADER]: Received request to transfer leadership to TS 9468a996c51241cab3d90392e5864642
I20260709 12:16:31.766353  6000 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a275014900ca4f32977e32bb6fc509fd"
dest_uuid: "bd8c877d092949e693c13e9103581f82"
mode: GRACEFUL
new_leader_uuid: "0ffd67a5e76e46a7b0089718369740bc"
 from {username='slave'} at 127.0.0.1:43106
I20260709 12:16:31.766875  6000 raft_consensus.cc:606] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82 [term 2 LEADER]: Received request to transfer leadership to TS 0ffd67a5e76e46a7b0089718369740bc
I20260709 12:16:31.767956  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:16:31.768555  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:32.064188  6345 raft_consensus.cc:993] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82: : Instructing follower 0ffd67a5e76e46a7b0089718369740bc to start an election
I20260709 12:16:32.064739  6319 raft_consensus.cc:1081] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 4 LEADER]: Signalling peer 0ffd67a5e76e46a7b0089718369740bc to start an election
I20260709 12:16:32.066331  6076 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "46526944508c443c9c12315136ae866f"
dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
 from {username='slave'} at 127.0.0.1:43850
I20260709 12:16:32.067045  6076 raft_consensus.cc:493] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:32.068104  6076 raft_consensus.cc:3060] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:16:32.075173  6076 raft_consensus.cc:515] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:32.077282  6076 leader_election.cc:290] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 5 election: Requested vote from peers bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:32.078256  6000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46526944508c443c9c12315136ae866f" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "bd8c877d092949e693c13e9103581f82"
I20260709 12:16:32.078459  6148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46526944508c443c9c12315136ae866f" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "9468a996c51241cab3d90392e5864642"
I20260709 12:16:32.078822  6000 raft_consensus.cc:3055] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:16:32.079018  6148 raft_consensus.cc:3060] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:16:32.079180  6000 raft_consensus.cc:740] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: bd8c877d092949e693c13e9103581f82, State: Running, Role: LEADER
I20260709 12:16:32.080031  6000 consensus_queue.cc:260] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:32.081207  6000 raft_consensus.cc:3060] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:16:32.084820  6148 raft_consensus.cc:2468] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ffd67a5e76e46a7b0089718369740bc in term 5.
I20260709 12:16:32.086490  6041 leader_election.cc:304] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc, 9468a996c51241cab3d90392e5864642; no voters: 
I20260709 12:16:32.087663  6313 raft_consensus.cc:2804] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:16:32.088078  6000 raft_consensus.cc:2468] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ffd67a5e76e46a7b0089718369740bc in term 5.
I20260709 12:16:32.090876  6313 raft_consensus.cc:697] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [term 5 LEADER]: Becoming Leader. State: Replica: 0ffd67a5e76e46a7b0089718369740bc, State: Running, Role: LEADER
I20260709 12:16:32.092123  6313 consensus_queue.cc:237] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:32.100575  5897 catalog_manager.cc:5697] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc reported cstate change: term changed from 4 to 5, leader changed from bd8c877d092949e693c13e9103581f82 (127.5.58.1) to 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2). New cstate: current_term: 5 leader_uuid: "0ffd67a5e76e46a7b0089718369740bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:32.135581  6345 raft_consensus.cc:993] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82: : Instructing follower 0ffd67a5e76e46a7b0089718369740bc to start an election
I20260709 12:16:32.136230  6319 raft_consensus.cc:1081] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82 [term 2 LEADER]: Signalling peer 0ffd67a5e76e46a7b0089718369740bc to start an election
I20260709 12:16:32.138290  6076 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a275014900ca4f32977e32bb6fc509fd"
dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
 from {username='slave'} at 127.0.0.1:43850
I20260709 12:16:32.138733  6076 raft_consensus.cc:493] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:32.138974  6076 raft_consensus.cc:3060] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:32.144587  6076 raft_consensus.cc:515] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:32.147637  6076 leader_election.cc:290] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 3 election: Requested vote from peers bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557), 9468a996c51241cab3d90392e5864642 (127.5.58.3:44821)
I20260709 12:16:32.149624  6000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a275014900ca4f32977e32bb6fc509fd" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bd8c877d092949e693c13e9103581f82"
I20260709 12:16:32.150388  6000 raft_consensus.cc:3055] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:32.150694  6000 raft_consensus.cc:740] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bd8c877d092949e693c13e9103581f82, State: Running, Role: LEADER
I20260709 12:16:32.149509  6148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a275014900ca4f32977e32bb6fc509fd" candidate_uuid: "0ffd67a5e76e46a7b0089718369740bc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9468a996c51241cab3d90392e5864642"
I20260709 12:16:32.151295  6000 consensus_queue.cc:260] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:32.151929  6148 raft_consensus.cc:3060] T a275014900ca4f32977e32bb6fc509fd P 9468a996c51241cab3d90392e5864642 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:32.152616  6000 raft_consensus.cc:3060] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:32.157956  6148 raft_consensus.cc:2468] T a275014900ca4f32977e32bb6fc509fd P 9468a996c51241cab3d90392e5864642 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ffd67a5e76e46a7b0089718369740bc in term 3.
I20260709 12:16:32.159132  6041 leader_election.cc:304] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc, 9468a996c51241cab3d90392e5864642; no voters: 
I20260709 12:16:32.159943  6313 raft_consensus.cc:2804] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:32.160401  6313 raft_consensus.cc:697] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [term 3 LEADER]: Becoming Leader. State: Replica: 0ffd67a5e76e46a7b0089718369740bc, State: Running, Role: LEADER
I20260709 12:16:32.161525  6313 consensus_queue.cc:237] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:32.162122  6000 raft_consensus.cc:2468] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ffd67a5e76e46a7b0089718369740bc in term 3.
I20260709 12:16:32.174082  5897 catalog_manager.cc:5697] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc reported cstate change: term changed from 2 to 3, leader changed from bd8c877d092949e693c13e9103581f82 (127.5.58.1) to 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2). New cstate: current_term: 3 leader_uuid: "0ffd67a5e76e46a7b0089718369740bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:32.198616  6345 raft_consensus.cc:993] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82: : Instructing follower 9468a996c51241cab3d90392e5864642 to start an election
I20260709 12:16:32.199069  6319 raft_consensus.cc:1081] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 3 LEADER]: Signalling peer 9468a996c51241cab3d90392e5864642 to start an election
I20260709 12:16:32.200552  6148 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3f858fa3ce674075aabe68fc8228266d"
dest_uuid: "9468a996c51241cab3d90392e5864642"
 from {username='slave'} at 127.0.0.1:35316
I20260709 12:16:32.201256  6148 raft_consensus.cc:493] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:32.201725  6148 raft_consensus.cc:3060] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:32.207242  6148 raft_consensus.cc:515] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:32.209841  6148 leader_election.cc:290] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 4 election: Requested vote from peers bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557), 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019)
I20260709 12:16:32.211058  6076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f858fa3ce674075aabe68fc8228266d" candidate_uuid: "9468a996c51241cab3d90392e5864642" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
I20260709 12:16:32.211097  6000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f858fa3ce674075aabe68fc8228266d" candidate_uuid: "9468a996c51241cab3d90392e5864642" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "bd8c877d092949e693c13e9103581f82"
I20260709 12:16:32.211712  6076 raft_consensus.cc:3060] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:32.212534  6000 raft_consensus.cc:3055] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:16:32.212821  6000 raft_consensus.cc:740] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bd8c877d092949e693c13e9103581f82, State: Running, Role: LEADER
I20260709 12:16:32.213510  6000 consensus_queue.cc:260] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:32.214319  6000 raft_consensus.cc:3060] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:32.216764  6076 raft_consensus.cc:2468] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9468a996c51241cab3d90392e5864642 in term 4.
I20260709 12:16:32.217857  6113 leader_election.cc:304] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc, 9468a996c51241cab3d90392e5864642; no voters: 
I20260709 12:16:32.218472  6326 raft_consensus.cc:2804] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:16:32.218514  6000 raft_consensus.cc:2468] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9468a996c51241cab3d90392e5864642 in term 4.
I20260709 12:16:32.220207  6326 raft_consensus.cc:697] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642 [term 4 LEADER]: Becoming Leader. State: Replica: 9468a996c51241cab3d90392e5864642, State: Running, Role: LEADER
I20260709 12:16:32.221199  6326 consensus_queue.cc:237] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:32.227905  6345 raft_consensus.cc:993] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82: : Instructing follower 9468a996c51241cab3d90392e5864642 to start an election
I20260709 12:16:32.228482  6319 raft_consensus.cc:1081] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 3 LEADER]: Signalling peer 9468a996c51241cab3d90392e5864642 to start an election
I20260709 12:16:32.267678  6148 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "800b87c938ae4f1596d89f00bb34399d"
dest_uuid: "9468a996c51241cab3d90392e5864642"
 from {username='slave'} at 127.0.0.1:35316
I20260709 12:16:32.269600  6148 raft_consensus.cc:493] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:32.268573  5897 catalog_manager.cc:5697] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642 reported cstate change: term changed from 3 to 4, leader changed from bd8c877d092949e693c13e9103581f82 (127.5.58.1) to 9468a996c51241cab3d90392e5864642 (127.5.58.3). New cstate: current_term: 4 leader_uuid: "9468a996c51241cab3d90392e5864642" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:32.271196  6148 raft_consensus.cc:3060] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:32.284409  6148 raft_consensus.cc:515] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:32.291144  6148 leader_election.cc:290] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 4 election: Requested vote from peers bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557), 0ffd67a5e76e46a7b0089718369740bc (127.5.58.2:38019)
I20260709 12:16:32.291858  6000 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "800b87c938ae4f1596d89f00bb34399d" candidate_uuid: "9468a996c51241cab3d90392e5864642" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "bd8c877d092949e693c13e9103581f82"
I20260709 12:16:32.293341  6000 raft_consensus.cc:3055] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:16:32.293751  6000 raft_consensus.cc:740] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bd8c877d092949e693c13e9103581f82, State: Running, Role: LEADER
I20260709 12:16:32.294004  6076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "800b87c938ae4f1596d89f00bb34399d" candidate_uuid: "9468a996c51241cab3d90392e5864642" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "0ffd67a5e76e46a7b0089718369740bc"
I20260709 12:16:32.295634  6076 raft_consensus.cc:3060] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:32.295557  6000 consensus_queue.cc:260] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:32.298477  6000 raft_consensus.cc:3060] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:32.302824  6076 raft_consensus.cc:2468] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9468a996c51241cab3d90392e5864642 in term 4.
I20260709 12:16:32.304589  6113 leader_election.cc:304] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffd67a5e76e46a7b0089718369740bc, 9468a996c51241cab3d90392e5864642; no voters: 
I20260709 12:16:32.305595  6326 raft_consensus.cc:2804] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:16:32.306005  6326 raft_consensus.cc:697] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642 [term 4 LEADER]: Becoming Leader. State: Replica: 9468a996c51241cab3d90392e5864642, State: Running, Role: LEADER
I20260709 12:16:32.307227  6326 consensus_queue.cc:237] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } }
I20260709 12:16:32.309059  6000 raft_consensus.cc:2468] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9468a996c51241cab3d90392e5864642 in term 4.
I20260709 12:16:32.318179  5897 catalog_manager.cc:5697] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642 reported cstate change: term changed from 3 to 4, leader changed from bd8c877d092949e693c13e9103581f82 (127.5.58.1) to 9468a996c51241cab3d90392e5864642 (127.5.58.3). New cstate: current_term: 4 leader_uuid: "9468a996c51241cab3d90392e5864642" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:32.569028  6148 raft_consensus.cc:1275] T a275014900ca4f32977e32bb6fc509fd P 9468a996c51241cab3d90392e5864642 [term 3 FOLLOWER]: Refusing update from remote peer 0ffd67a5e76e46a7b0089718369740bc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:32.570389  6313 consensus_queue.cc:1048] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:32.585558  6000 raft_consensus.cc:1275] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82 [term 3 FOLLOWER]: Refusing update from remote peer 0ffd67a5e76e46a7b0089718369740bc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:32.588444  6313 consensus_queue.cc:1048] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:32.604256  6000 raft_consensus.cc:1275] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 5 FOLLOWER]: Refusing update from remote peer 0ffd67a5e76e46a7b0089718369740bc: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:16:32.606297  6346 consensus_queue.cc:1048] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:16:32.612866  6148 raft_consensus.cc:1275] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 5 FOLLOWER]: Refusing update from remote peer 0ffd67a5e76e46a7b0089718369740bc: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:16:32.614291  6346 consensus_queue.cc:1048] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "9468a996c51241cab3d90392e5864642" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 44821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:32.634333  6076 raft_consensus.cc:1275] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [term 4 FOLLOWER]: Refusing update from remote peer 9468a996c51241cab3d90392e5864642: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:32.636770  6326 consensus_queue.cc:1048] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:32.649838  6000 raft_consensus.cc:1275] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 4 FOLLOWER]: Refusing update from remote peer 9468a996c51241cab3d90392e5864642: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:32.652005  6353 consensus_queue.cc:1048] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:32.770219  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:32.775856  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:32.776319  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:32.796615  6075 raft_consensus.cc:1275] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [term 4 FOLLOWER]: Refusing update from remote peer 9468a996c51241cab3d90392e5864642: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:32.798046  6326 consensus_queue.cc:1048] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ffd67a5e76e46a7b0089718369740bc" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:32.819443  6000 raft_consensus.cc:1275] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 4 FOLLOWER]: Refusing update from remote peer 9468a996c51241cab3d90392e5864642: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:32.822964  6326 consensus_queue.cc:1048] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd8c877d092949e693c13e9103581f82" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 35557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:33.777384  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:33.783643  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:33.784086  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:34.785161  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:34.792192  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:34.792690  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:35.793717  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:35.799670  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:35.800154  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:36.801229  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:36.807380  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:36.807873  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:37.808884  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:37.814886  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:37.815357  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:38.816282  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:38.821461  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:38.821902  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:39.823021  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:39.830137  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:39.830585  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:40.831657  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:40.837479  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:40.838012  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:41.871923  5352 auto_leader_rebalancer-test.cc:410] The leader distribution is 
I20260709 12:16:41.872895  5352 tablet_server.cc:179] TabletServer@127.5.58.1:0 shutting down...
I20260709 12:16:41.903752  5352 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:41.904596  5352 tablet_replica.cc:333] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82: stopping tablet replica
I20260709 12:16:41.905301  5352 raft_consensus.cc:2243] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:41.905823  5352 raft_consensus.cc:2272] T 3f858fa3ce674075aabe68fc8228266d P bd8c877d092949e693c13e9103581f82 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:41.908263  5352 tablet_replica.cc:333] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82: stopping tablet replica
I20260709 12:16:41.908918  5352 raft_consensus.cc:2243] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:41.909395  5352 raft_consensus.cc:2272] T 46526944508c443c9c12315136ae866f P bd8c877d092949e693c13e9103581f82 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:41.911319  5352 tablet_replica.cc:333] T 8ff9114868e646a4967bf3e65485393c P bd8c877d092949e693c13e9103581f82: stopping tablet replica
I20260709 12:16:41.911885  5352 raft_consensus.cc:2243] T 8ff9114868e646a4967bf3e65485393c P bd8c877d092949e693c13e9103581f82 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:16:41.912833  5352 raft_consensus.cc:2272] T 8ff9114868e646a4967bf3e65485393c P bd8c877d092949e693c13e9103581f82 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:41.915010  5352 tablet_replica.cc:333] T b40ad7108aec4ad2a5aa42acd343c6ac P bd8c877d092949e693c13e9103581f82: stopping tablet replica
I20260709 12:16:41.915618  5352 raft_consensus.cc:2243] T b40ad7108aec4ad2a5aa42acd343c6ac P bd8c877d092949e693c13e9103581f82 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:41.916602  5352 raft_consensus.cc:2272] T b40ad7108aec4ad2a5aa42acd343c6ac P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:41.918306  5352 tablet_replica.cc:333] T cb9f68ad1dc24fc29ed8ad07197a68ac P bd8c877d092949e693c13e9103581f82: stopping tablet replica
I20260709 12:16:41.918798  5352 raft_consensus.cc:2243] T cb9f68ad1dc24fc29ed8ad07197a68ac P bd8c877d092949e693c13e9103581f82 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:41.919535  5352 raft_consensus.cc:2272] T cb9f68ad1dc24fc29ed8ad07197a68ac P bd8c877d092949e693c13e9103581f82 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:41.921211  5352 tablet_replica.cc:333] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82: stopping tablet replica
I20260709 12:16:41.921697  5352 raft_consensus.cc:2243] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:41.922061  5352 raft_consensus.cc:2272] T a275014900ca4f32977e32bb6fc509fd P bd8c877d092949e693c13e9103581f82 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:41.923620  5352 tablet_replica.cc:333] T d1ec21d99cc2496d84a013cadd31fbc6 P bd8c877d092949e693c13e9103581f82: stopping tablet replica
I20260709 12:16:41.924068  5352 raft_consensus.cc:2243] T d1ec21d99cc2496d84a013cadd31fbc6 P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:41.924572  5352 raft_consensus.cc:2272] T d1ec21d99cc2496d84a013cadd31fbc6 P bd8c877d092949e693c13e9103581f82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:41.926357  5352 tablet_replica.cc:333] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82: stopping tablet replica
I20260709 12:16:41.926865  5352 raft_consensus.cc:2243] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:41.927254  5352 raft_consensus.cc:2272] T 800b87c938ae4f1596d89f00bb34399d P bd8c877d092949e693c13e9103581f82 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:41.929836  5352 tablet_replica.cc:333] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82: stopping tablet replica
I20260709 12:16:41.930472  5352 raft_consensus.cc:2243] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:16:41.931445  5352 raft_consensus.cc:2272] T 71c10f56e0ac401b875d85211fc6ae9f P bd8c877d092949e693c13e9103581f82 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:41.933784  5352 tablet_replica.cc:333] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82: stopping tablet replica
I20260709 12:16:41.934403  5352 raft_consensus.cc:2243] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:41.934945  5352 raft_consensus.cc:2272] T a3442b7a72a34bb3971f1dae9e473e38 P bd8c877d092949e693c13e9103581f82 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:41.961606  6042 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.58.1:35557: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260709 12:16:41.965993  6042 consensus_peers.cc:597] T a3442b7a72a34bb3971f1dae9e473e38 P 0ffd67a5e76e46a7b0089718369740bc -> Peer bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557): Couldn't send request to peer bd8c877d092949e693c13e9103581f82. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:35557: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:41.971709  5352 tablet_server.cc:196] TabletServer@127.5.58.1:0 shutdown complete.
I20260709 12:16:41.993526  5352 tablet_server.cc:179] TabletServer@127.5.58.2:0 shutting down...
I20260709 12:16:42.027983  5352 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:42.028856  5352 tablet_replica.cc:333] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc: stopping tablet replica
I20260709 12:16:42.029641  5352 raft_consensus.cc:2243] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 12:16:42.029733  6116 consensus_peers.cc:597] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642 -> Peer bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557): Couldn't send request to peer bd8c877d092949e693c13e9103581f82. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:35557: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:42.030328  5352 raft_consensus.cc:2272] T 8ff9114868e646a4967bf3e65485393c P 0ffd67a5e76e46a7b0089718369740bc [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:42.032152  6116 consensus_peers.cc:597] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642 -> Peer bd8c877d092949e693c13e9103581f82 (127.5.58.1:35557): Couldn't send request to peer bd8c877d092949e693c13e9103581f82. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:35557: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:42.033062  5352 tablet_replica.cc:333] T b40ad7108aec4ad2a5aa42acd343c6ac P 0ffd67a5e76e46a7b0089718369740bc: stopping tablet replica
I20260709 12:16:42.033681  5352 raft_consensus.cc:2243] T b40ad7108aec4ad2a5aa42acd343c6ac P 0ffd67a5e76e46a7b0089718369740bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.034154  5352 raft_consensus.cc:2272] T b40ad7108aec4ad2a5aa42acd343c6ac P 0ffd67a5e76e46a7b0089718369740bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.036078  5352 tablet_replica.cc:333] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc: stopping tablet replica
I20260709 12:16:42.036707  5352 raft_consensus.cc:2243] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.037173  5352 raft_consensus.cc:2272] T 3f858fa3ce674075aabe68fc8228266d P 0ffd67a5e76e46a7b0089718369740bc [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.038954  5352 tablet_replica.cc:333] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc: stopping tablet replica
I20260709 12:16:42.039511  5352 raft_consensus.cc:2243] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.039992  5352 raft_consensus.cc:2272] T 800b87c938ae4f1596d89f00bb34399d P 0ffd67a5e76e46a7b0089718369740bc [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.041842  5352 tablet_replica.cc:333] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc: stopping tablet replica
I20260709 12:16:42.042423  5352 raft_consensus.cc:2243] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:16:42.043255  5352 raft_consensus.cc:2272] T 46526944508c443c9c12315136ae866f P 0ffd67a5e76e46a7b0089718369740bc [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.044937  5352 tablet_replica.cc:333] T d1ec21d99cc2496d84a013cadd31fbc6 P 0ffd67a5e76e46a7b0089718369740bc: stopping tablet replica
I20260709 12:16:42.045428  5352 raft_consensus.cc:2243] T d1ec21d99cc2496d84a013cadd31fbc6 P 0ffd67a5e76e46a7b0089718369740bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.045827  5352 raft_consensus.cc:2272] T d1ec21d99cc2496d84a013cadd31fbc6 P 0ffd67a5e76e46a7b0089718369740bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.047554  5352 tablet_replica.cc:333] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc: stopping tablet replica
I20260709 12:16:42.048017  5352 raft_consensus.cc:2243] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.048462  5352 raft_consensus.cc:2272] T cb9f68ad1dc24fc29ed8ad07197a68ac P 0ffd67a5e76e46a7b0089718369740bc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.050001  5352 tablet_replica.cc:333] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc: stopping tablet replica
I20260709 12:16:42.050467  5352 raft_consensus.cc:2243] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:16:42.051154  5352 raft_consensus.cc:2272] T a275014900ca4f32977e32bb6fc509fd P 0ffd67a5e76e46a7b0089718369740bc [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.052727  5352 tablet_replica.cc:333] T a3442b7a72a34bb3971f1dae9e473e38 P 0ffd67a5e76e46a7b0089718369740bc: stopping tablet replica
I20260709 12:16:42.053169  5352 raft_consensus.cc:2243] T a3442b7a72a34bb3971f1dae9e473e38 P 0ffd67a5e76e46a7b0089718369740bc [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:42.053853  5352 raft_consensus.cc:2272] T a3442b7a72a34bb3971f1dae9e473e38 P 0ffd67a5e76e46a7b0089718369740bc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.055486  5352 tablet_replica.cc:333] T 71c10f56e0ac401b875d85211fc6ae9f P 0ffd67a5e76e46a7b0089718369740bc: stopping tablet replica
I20260709 12:16:42.055931  5352 raft_consensus.cc:2243] T 71c10f56e0ac401b875d85211fc6ae9f P 0ffd67a5e76e46a7b0089718369740bc [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.056394  5352 raft_consensus.cc:2272] T 71c10f56e0ac401b875d85211fc6ae9f P 0ffd67a5e76e46a7b0089718369740bc [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.087005  5352 tablet_server.cc:196] TabletServer@127.5.58.2:0 shutdown complete.
I20260709 12:16:42.108104  5352 tablet_server.cc:179] TabletServer@127.5.58.3:0 shutting down...
I20260709 12:16:42.142050  5352 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:42.142757  5352 tablet_replica.cc:333] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642: stopping tablet replica
I20260709 12:16:42.143307  5352 raft_consensus.cc:2243] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.143821  5352 raft_consensus.cc:2272] T 8ff9114868e646a4967bf3e65485393c P 9468a996c51241cab3d90392e5864642 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.145659  5352 tablet_replica.cc:333] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642: stopping tablet replica
I20260709 12:16:42.146170  5352 raft_consensus.cc:2243] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:16:42.146881  5352 raft_consensus.cc:2272] T 3f858fa3ce674075aabe68fc8228266d P 9468a996c51241cab3d90392e5864642 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.148528  5352 tablet_replica.cc:333] T b40ad7108aec4ad2a5aa42acd343c6ac P 9468a996c51241cab3d90392e5864642: stopping tablet replica
I20260709 12:16:42.149053  5352 raft_consensus.cc:2243] T b40ad7108aec4ad2a5aa42acd343c6ac P 9468a996c51241cab3d90392e5864642 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.149433  5352 raft_consensus.cc:2272] T b40ad7108aec4ad2a5aa42acd343c6ac P 9468a996c51241cab3d90392e5864642 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.151069  5352 tablet_replica.cc:333] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642: stopping tablet replica
I20260709 12:16:42.151582  5352 raft_consensus.cc:2243] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.151989  5352 raft_consensus.cc:2272] T 46526944508c443c9c12315136ae866f P 9468a996c51241cab3d90392e5864642 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.153580  5352 tablet_replica.cc:333] T a275014900ca4f32977e32bb6fc509fd P 9468a996c51241cab3d90392e5864642: stopping tablet replica
I20260709 12:16:42.154059  5352 raft_consensus.cc:2243] T a275014900ca4f32977e32bb6fc509fd P 9468a996c51241cab3d90392e5864642 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.154456  5352 raft_consensus.cc:2272] T a275014900ca4f32977e32bb6fc509fd P 9468a996c51241cab3d90392e5864642 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.155999  5352 tablet_replica.cc:333] T d1ec21d99cc2496d84a013cadd31fbc6 P 9468a996c51241cab3d90392e5864642: stopping tablet replica
I20260709 12:16:42.156602  5352 raft_consensus.cc:2243] T d1ec21d99cc2496d84a013cadd31fbc6 P 9468a996c51241cab3d90392e5864642 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:42.157290  5352 raft_consensus.cc:2272] T d1ec21d99cc2496d84a013cadd31fbc6 P 9468a996c51241cab3d90392e5864642 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.158895  5352 tablet_replica.cc:333] T cb9f68ad1dc24fc29ed8ad07197a68ac P 9468a996c51241cab3d90392e5864642: stopping tablet replica
I20260709 12:16:42.159382  5352 raft_consensus.cc:2243] T cb9f68ad1dc24fc29ed8ad07197a68ac P 9468a996c51241cab3d90392e5864642 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.159789  5352 raft_consensus.cc:2272] T cb9f68ad1dc24fc29ed8ad07197a68ac P 9468a996c51241cab3d90392e5864642 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.161348  5352 tablet_replica.cc:333] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642: stopping tablet replica
I20260709 12:16:42.161824  5352 raft_consensus.cc:2243] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:16:42.162490  5352 raft_consensus.cc:2272] T 800b87c938ae4f1596d89f00bb34399d P 9468a996c51241cab3d90392e5864642 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.163959  5352 tablet_replica.cc:333] T a3442b7a72a34bb3971f1dae9e473e38 P 9468a996c51241cab3d90392e5864642: stopping tablet replica
I20260709 12:16:42.164490  5352 raft_consensus.cc:2243] T a3442b7a72a34bb3971f1dae9e473e38 P 9468a996c51241cab3d90392e5864642 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.164902  5352 raft_consensus.cc:2272] T a3442b7a72a34bb3971f1dae9e473e38 P 9468a996c51241cab3d90392e5864642 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.166352  5352 tablet_replica.cc:333] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642: stopping tablet replica
I20260709 12:16:42.166823  5352 raft_consensus.cc:2243] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:42.167200  5352 raft_consensus.cc:2272] T 71c10f56e0ac401b875d85211fc6ae9f P 9468a996c51241cab3d90392e5864642 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.195330  5352 tablet_server.cc:196] TabletServer@127.5.58.3:0 shutdown complete.
I20260709 12:16:42.214628  5352 master.cc:562] Master@127.5.58.62:43011 shutting down...
I20260709 12:16:42.239253  5352 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 4ba9cab5cfc54b7b92e51215c4d90ac2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:42.239815  5352 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 4ba9cab5cfc54b7b92e51215c4d90ac2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:42.240510  5352 tablet_replica.cc:333] T 00000000000000000000000000000000 P 4ba9cab5cfc54b7b92e51215c4d90ac2: stopping tablet replica
I20260709 12:16:42.260876  5352 master.cc:584] Master@127.5.58.62:43011 shutdown complete.
0ffd67a5e76e46a7b0089718369740bc  4
9468a996c51241cab3d90392e5864642  1
bd8c877d092949e693c13e9103581f82  5
0ffd67a5e76e46a7b0089718369740bc  4
9468a996c51241cab3d90392e5864642  3
bd8c877d092949e693c13e9103581f82  3
0ffd67a5e76e46a7b0089718369740bc  1
9468a996c51241cab3d90392e5864642  1
bd8c877d092949e693c13e9103581f82  8
0ffd67a5e76e46a7b0089718369740bc  3
9468a996c51241cab3d90392e5864642  3
bd8c877d092949e693c13e9103581f82  4
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (28856 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 12:16:42.318128  5352 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.58.62:36381
I20260709 12:16:42.319353  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:42.326035  6382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:42.326717  6383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:42.327646  6385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:42.328101  5352 server_base.cc:1061] running on GCE node
I20260709 12:16:42.329269  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:42.329499  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:42.329689  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599402329670 us; error 0 us; skew 500 ppm
I20260709 12:16:42.330240  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:42.332747  5352 webserver.cc:533] Webserver started at http://127.5.58.62:41329/ using document root <none> and password file <none>
I20260709 12:16:42.333262  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:42.333462  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:42.333741  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:42.335008  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/master-0-root/instance:
uuid: "39c4be726d3d495181bfe7425c8c0bd8"
format_stamp: "Formatted at 2026-07-09 12:16:42 on dist-test-slave-xs2d"
I20260709 12:16:42.339669  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.003s
I20260709 12:16:42.343276  6390 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:42.344226  5352 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:16:42.344552  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/master-0-root
uuid: "39c4be726d3d495181bfe7425c8c0bd8"
format_stamp: "Formatted at 2026-07-09 12:16:42 on dist-test-slave-xs2d"
I20260709 12:16:42.344803  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:42.356894  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:42.358134  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:42.402683  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.62:36381
I20260709 12:16:42.402801  6441 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.62:36381 every 8 connection(s)
I20260709 12:16:42.407156  6442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:42.418146  6442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8: Bootstrap starting.
I20260709 12:16:42.423291  6442 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:42.427946  6442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8: No bootstrap required, opened a new log
I20260709 12:16:42.430229  6442 raft_consensus.cc:359] T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39c4be726d3d495181bfe7425c8c0bd8" member_type: VOTER }
I20260709 12:16:42.430714  6442 raft_consensus.cc:385] T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:42.430893  6442 raft_consensus.cc:740] T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39c4be726d3d495181bfe7425c8c0bd8, State: Initialized, Role: FOLLOWER
I20260709 12:16:42.431528  6442 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39c4be726d3d495181bfe7425c8c0bd8" member_type: VOTER }
I20260709 12:16:42.431953  6442 raft_consensus.cc:399] T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:16:42.432168  6442 raft_consensus.cc:493] T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:16:42.432415  6442 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:42.436439  6442 raft_consensus.cc:515] T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39c4be726d3d495181bfe7425c8c0bd8" member_type: VOTER }
I20260709 12:16:42.436895  6442 leader_election.cc:304] T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39c4be726d3d495181bfe7425c8c0bd8; no voters: 
I20260709 12:16:42.437986  6442 leader_election.cc:290] T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:16:42.438431  6445 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:42.439754  6445 raft_consensus.cc:697] T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8 [term 1 LEADER]: Becoming Leader. State: Replica: 39c4be726d3d495181bfe7425c8c0bd8, State: Running, Role: LEADER
I20260709 12:16:42.440557  6445 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39c4be726d3d495181bfe7425c8c0bd8" member_type: VOTER }
I20260709 12:16:42.440882  6442 sys_catalog.cc:565] T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:16:42.447829  6447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39c4be726d3d495181bfe7425c8c0bd8. Latest consensus state: current_term: 1 leader_uuid: "39c4be726d3d495181bfe7425c8c0bd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39c4be726d3d495181bfe7425c8c0bd8" member_type: VOTER } }
I20260709 12:16:42.448429  6446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39c4be726d3d495181bfe7425c8c0bd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39c4be726d3d495181bfe7425c8c0bd8" member_type: VOTER } }
I20260709 12:16:42.448729  6447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8 [sys.catalog]: This master's current role is: LEADER
I20260709 12:16:42.449080  6446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8 [sys.catalog]: This master's current role is: LEADER
I20260709 12:16:42.450912  6454 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:16:42.456256  6454 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:16:42.457958  5352 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:16:42.465835  6454 catalog_manager.cc:1383] Generated new cluster ID: a7a2d4336b5e48a593001ff27dfdc395
I20260709 12:16:42.466140  6454 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:16:42.487498  6454 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:16:42.488821  6454 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:16:42.506161  6454 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8: Generated new TSK 0
I20260709 12:16:42.506780  6454 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:16:42.525229  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:42.532382  6463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:42.533541  6464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:42.536031  5352 server_base.cc:1061] running on GCE node
W20260709 12:16:42.536067  6466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:42.537444  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:42.537693  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:42.537864  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599402537849 us; error 0 us; skew 500 ppm
I20260709 12:16:42.538556  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:42.541182  5352 webserver.cc:533] Webserver started at http://127.5.58.1:41715/ using document root <none> and password file <none>
I20260709 12:16:42.541742  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:42.542001  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:42.542692  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:42.544646  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-0-root/instance:
uuid: "e7554c39679a41bbbca4659b72ab5c65"
format_stamp: "Formatted at 2026-07-09 12:16:42 on dist-test-slave-xs2d"
I20260709 12:16:42.549604  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 12:16:42.553624  6471 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:42.554636  5352 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:16:42.554988  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-0-root
uuid: "e7554c39679a41bbbca4659b72ab5c65"
format_stamp: "Formatted at 2026-07-09 12:16:42 on dist-test-slave-xs2d"
I20260709 12:16:42.555310  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:42.590032  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:42.591436  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:42.593570  5352 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:42.596259  5352 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:42.596539  5352 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:16:42.596860  5352 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:42.597040  5352 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:42.644942  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.1:36643
I20260709 12:16:42.645020  6533 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.1:36643 every 8 connection(s)
I20260709 12:16:42.650938  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:42.657930  6538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:42.660121  6539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:42.663244  6541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:42.664671  5352 server_base.cc:1061] running on GCE node
I20260709 12:16:42.666373  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:16:42.666427  6534 heartbeater.cc:344] Connected to a master server at 127.5.58.62:36381
W20260709 12:16:42.666641  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:42.666900  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599402666886 us; error 0 us; skew 500 ppm
I20260709 12:16:42.666993  6534 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:42.667543  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:42.668179  6534 heartbeater.cc:507] Master 127.5.58.62:36381 requested a full tablet report, sending...
I20260709 12:16:42.670393  5352 webserver.cc:533] Webserver started at http://127.5.58.2:41811/ using document root <none> and password file <none>
I20260709 12:16:42.671073  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:42.670970  6407 ts_manager.cc:194] Registered new tserver with Master: e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643)
I20260709 12:16:42.671357  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:42.671742  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:42.673316  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-1-root/instance:
uuid: "496c7c3792cd4587b4dbeba435f4be1e"
format_stamp: "Formatted at 2026-07-09 12:16:42 on dist-test-slave-xs2d"
I20260709 12:16:42.673477  6407 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57888
I20260709 12:16:42.681106  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.004s	sys 0.004s
I20260709 12:16:42.685845  6546 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:42.686709  5352 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:16:42.687050  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-1-root
uuid: "496c7c3792cd4587b4dbeba435f4be1e"
format_stamp: "Formatted at 2026-07-09 12:16:42 on dist-test-slave-xs2d"
I20260709 12:16:42.687332  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:42.697795  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:42.699139  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:42.701081  5352 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:42.703716  5352 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:42.703938  5352 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:42.704286  5352 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:42.704468  5352 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:16:42.749046  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.2:45101
I20260709 12:16:42.749161  6608 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.2:45101 every 8 connection(s)
I20260709 12:16:42.754280  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:42.761413  6612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:42.762223  6613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:42.766342  6609 heartbeater.cc:344] Connected to a master server at 127.5.58.62:36381
I20260709 12:16:42.766717  6609 heartbeater.cc:461] Registering TS with master...
W20260709 12:16:42.767458  6616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:42.767863  6609 heartbeater.cc:507] Master 127.5.58.62:36381 requested a full tablet report, sending...
I20260709 12:16:42.768743  5352 server_base.cc:1061] running on GCE node
I20260709 12:16:42.769915  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:42.770112  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:42.770351  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599402770311 us; error 0 us; skew 500 ppm
I20260709 12:16:42.770371  6407 ts_manager.cc:194] Registered new tserver with Master: 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:42.771135  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:42.771958  6407 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57900
I20260709 12:16:42.774214  5352 webserver.cc:533] Webserver started at http://127.5.58.3:33017/ using document root <none> and password file <none>
I20260709 12:16:42.774771  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:42.775002  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:42.775307  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:42.776728  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-2-root/instance:
uuid: "f2b06aebe97249a4877d798b8118ece4"
format_stamp: "Formatted at 2026-07-09 12:16:42 on dist-test-slave-xs2d"
I20260709 12:16:42.781641  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:16:42.785327  6620 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:42.786232  5352 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.004s	sys 0.000s
I20260709 12:16:42.786531  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-2-root
uuid: "f2b06aebe97249a4877d798b8118ece4"
format_stamp: "Formatted at 2026-07-09 12:16:42 on dist-test-slave-xs2d"
I20260709 12:16:42.786947  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:42.800074  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:42.801986  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:42.803568  5352 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:42.805985  5352 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:42.806223  5352 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:42.806466  5352 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:42.806639  5352 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:42.852105  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.3:36107
I20260709 12:16:42.852262  6682 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.3:36107 every 8 connection(s)
I20260709 12:16:42.867942  6683 heartbeater.cc:344] Connected to a master server at 127.5.58.62:36381
I20260709 12:16:42.868458  6683 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:42.869330  6683 heartbeater.cc:507] Master 127.5.58.62:36381 requested a full tablet report, sending...
I20260709 12:16:42.871510  6407 ts_manager.cc:194] Registered new tserver with Master: f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:42.872449  5352 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017269352s
I20260709 12:16:42.873202  6407 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57914
I20260709 12:16:42.901027  6407 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57922:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:16:42.907341  6407 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:16:43.106705  6499 tablet_service.cc:1511] Processing CreateTablet for tablet e91732822ea14700aa3230fbb6b1cfac (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:16:43.108458  6499 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e91732822ea14700aa3230fbb6b1cfac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.108847  6497 tablet_service.cc:1511] Processing CreateTablet for tablet 3af4a543c38a49bfa8ece9e702577ea1 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:16:43.115142  6497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3af4a543c38a49bfa8ece9e702577ea1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.116289  6571 tablet_service.cc:1511] Processing CreateTablet for tablet 53dc690d5068460a820e0ba9e24494ec (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:16:43.116897  6495 tablet_service.cc:1511] Processing CreateTablet for tablet 72cc6298c451423d8fe8c456777eb890 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:16:43.117766  6571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53dc690d5068460a820e0ba9e24494ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.118263  6495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72cc6298c451423d8fe8c456777eb890. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.118969  6648 tablet_service.cc:1511] Processing CreateTablet for tablet e91732822ea14700aa3230fbb6b1cfac (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:16:43.110618  6498 tablet_service.cc:1511] Processing CreateTablet for tablet 1997e98759d0465ab0e3c607df127068 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:16:43.120271  6648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e91732822ea14700aa3230fbb6b1cfac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.121187  6498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1997e98759d0465ab0e3c607df127068. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.125626  6647 tablet_service.cc:1511] Processing CreateTablet for tablet 1997e98759d0465ab0e3c607df127068 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:16:43.126979  6647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1997e98759d0465ab0e3c607df127068. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.110728  6574 tablet_service.cc:1511] Processing CreateTablet for tablet e91732822ea14700aa3230fbb6b1cfac (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:16:43.111431  6573 tablet_service.cc:1511] Processing CreateTablet for tablet 1997e98759d0465ab0e3c607df127068 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:16:43.112603  6496 tablet_service.cc:1511] Processing CreateTablet for tablet 53dc690d5068460a820e0ba9e24494ec (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:16:43.129288  6574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e91732822ea14700aa3230fbb6b1cfac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.129513  6496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53dc690d5068460a820e0ba9e24494ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.129617  6646 tablet_service.cc:1511] Processing CreateTablet for tablet 3af4a543c38a49bfa8ece9e702577ea1 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:16:43.131081  6646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3af4a543c38a49bfa8ece9e702577ea1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.113271  6572 tablet_service.cc:1511] Processing CreateTablet for tablet 3af4a543c38a49bfa8ece9e702577ea1 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:16:43.133533  6572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3af4a543c38a49bfa8ece9e702577ea1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.133925  6645 tablet_service.cc:1511] Processing CreateTablet for tablet 53dc690d5068460a820e0ba9e24494ec (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:16:43.135290  6573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1997e98759d0465ab0e3c607df127068. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.135692  6645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53dc690d5068460a820e0ba9e24494ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.147055  6570 tablet_service.cc:1511] Processing CreateTablet for tablet 72cc6298c451423d8fe8c456777eb890 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:16:43.148729  6570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72cc6298c451423d8fe8c456777eb890. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.150072  6644 tablet_service.cc:1511] Processing CreateTablet for tablet 72cc6298c451423d8fe8c456777eb890 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:16:43.151415  6644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72cc6298c451423d8fe8c456777eb890. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.183118  6494 tablet_service.cc:1511] Processing CreateTablet for tablet d1778f67a2ba4fc2825ec2be5097d77a (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:16:43.190531  6698 tablet_bootstrap.cc:492] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:43.208690  6494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1778f67a2ba4fc2825ec2be5097d77a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.218411  6571 tablet_service.cc:1511] Processing CreateTablet for tablet d1778f67a2ba4fc2825ec2be5097d77a (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:16:43.224176  6571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1778f67a2ba4fc2825ec2be5097d77a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.221601  6644 tablet_service.cc:1511] Processing CreateTablet for tablet d1778f67a2ba4fc2825ec2be5097d77a (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:16:43.228039  6644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1778f67a2ba4fc2825ec2be5097d77a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.232756  6572 tablet_service.cc:1511] Processing CreateTablet for tablet 6462df23af154f06b34f32a1ef4a45b2 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:16:43.233237  6646 tablet_service.cc:1511] Processing CreateTablet for tablet 6462df23af154f06b34f32a1ef4a45b2 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:16:43.234068  6572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6462df23af154f06b34f32a1ef4a45b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.234536  6646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6462df23af154f06b34f32a1ef4a45b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.235639  6700 tablet_bootstrap.cc:492] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:43.235751  6699 tablet_bootstrap.cc:492] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:43.242492  6699 tablet_bootstrap.cc:654] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:43.245216  6498 tablet_service.cc:1511] Processing CreateTablet for tablet 6462df23af154f06b34f32a1ef4a45b2 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:16:43.246506  6498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6462df23af154f06b34f32a1ef4a45b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.250042  6573 tablet_service.cc:1511] Processing CreateTablet for tablet 3e07e97c48184f8e99a4fc5a580724a5 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:16:43.251358  6573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e07e97c48184f8e99a4fc5a580724a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.252653  6499 tablet_service.cc:1511] Processing CreateTablet for tablet 3e07e97c48184f8e99a4fc5a580724a5 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:16:43.254186  6499 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e07e97c48184f8e99a4fc5a580724a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.255563  6700 tablet_bootstrap.cc:654] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:43.255864  6495 tablet_service.cc:1511] Processing CreateTablet for tablet de5f3f59d0794f4c898c5e7f2b721abe (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:16:43.256695  6645 tablet_service.cc:1511] Processing CreateTablet for tablet 3e07e97c48184f8e99a4fc5a580724a5 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:16:43.265926  6495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de5f3f59d0794f4c898c5e7f2b721abe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.272989  6497 tablet_service.cc:1511] Processing CreateTablet for tablet cb7f6a796adb4b8c89579ba77274985f (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:16:43.274331  6497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb7f6a796adb4b8c89579ba77274985f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.266458  6698 tablet_bootstrap.cc:654] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:43.270131  6647 tablet_service.cc:1511] Processing CreateTablet for tablet de5f3f59d0794f4c898c5e7f2b721abe (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:16:43.277493  6647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de5f3f59d0794f4c898c5e7f2b721abe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.285243  6571 tablet_service.cc:1511] Processing CreateTablet for tablet de5f3f59d0794f4c898c5e7f2b721abe (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:16:43.286614  6700 tablet_bootstrap.cc:492] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:43.287133  6700 ts_tablet_manager.cc:1403] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.052s	user 0.011s	sys 0.000s
I20260709 12:16:43.287031  6572 tablet_service.cc:1511] Processing CreateTablet for tablet cb7f6a796adb4b8c89579ba77274985f (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:16:43.289036  6645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e07e97c48184f8e99a4fc5a580724a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.289338  6496 tablet_service.cc:1511] Processing CreateTablet for tablet c22b6d97ae844b7195c5e9c024203d48 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:16:43.293416  6572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb7f6a796adb4b8c89579ba77274985f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.293797  6496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c22b6d97ae844b7195c5e9c024203d48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.301474  6644 tablet_service.cc:1511] Processing CreateTablet for tablet cb7f6a796adb4b8c89579ba77274985f (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:16:43.309051  6644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb7f6a796adb4b8c89579ba77274985f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.310272  6574 tablet_service.cc:1511] Processing CreateTablet for tablet c22b6d97ae844b7195c5e9c024203d48 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:16:43.315828  6646 tablet_service.cc:1511] Processing CreateTablet for tablet c22b6d97ae844b7195c5e9c024203d48 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:16:43.316967  6571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de5f3f59d0794f4c898c5e7f2b721abe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.317199  6646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c22b6d97ae844b7195c5e9c024203d48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.321060  6574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c22b6d97ae844b7195c5e9c024203d48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.323112  6700 raft_consensus.cc:359] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:43.323837  6700 raft_consensus.cc:385] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:43.326754  6699 tablet_bootstrap.cc:492] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:43.327245  6699 ts_tablet_manager.cc:1403] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.092s	user 0.002s	sys 0.017s
I20260709 12:16:43.330132  6699 raft_consensus.cc:359] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:43.330807  6699 raft_consensus.cc:385] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:43.331102  6699 raft_consensus.cc:740] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:43.331825  6699 consensus_queue.cc:260] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:43.334957  6698 tablet_bootstrap.cc:492] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:43.335472  6698 ts_tablet_manager.cc:1403] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.145s	user 0.012s	sys 0.003s
I20260709 12:16:43.338335  6698 raft_consensus.cc:359] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:43.339058  6698 raft_consensus.cc:385] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:43.339404  6698 raft_consensus.cc:740] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:43.340369  6698 consensus_queue.cc:260] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:43.324172  6700 raft_consensus.cc:740] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:43.345345  6497 tablet_service.cc:1511] Processing CreateTablet for tablet fa09afefaa884dc5943b210cb23a07b3 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:16:43.346757  6497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa09afefaa884dc5943b210cb23a07b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.346357  6700 consensus_queue.cc:260] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:43.347834  6573 tablet_service.cc:1511] Processing CreateTablet for tablet fa09afefaa884dc5943b210cb23a07b3 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:16:43.349200  6534 heartbeater.cc:499] Master 127.5.58.62:36381 was elected leader, sending a full tablet report...
I20260709 12:16:43.353724  6496 tablet_service.cc:1511] Processing CreateTablet for tablet 7ba56fbb3caa417db1c8d78a9caa28da (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:16:43.355051  6496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ba56fbb3caa417db1c8d78a9caa28da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.350277  6698 ts_tablet_manager.cc:1434] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.015s	user 0.013s	sys 0.001s
I20260709 12:16:43.351541  6609 heartbeater.cc:499] Master 127.5.58.62:36381 was elected leader, sending a full tablet report...
I20260709 12:16:43.359454  6699 ts_tablet_manager.cc:1434] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.032s	user 0.009s	sys 0.000s
I20260709 12:16:43.360719  6699 tablet_bootstrap.cc:492] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:43.361670  6683 heartbeater.cc:499] Master 127.5.58.62:36381 was elected leader, sending a full tablet report...
I20260709 12:16:43.367901  6698 tablet_bootstrap.cc:492] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:43.370342  6574 tablet_service.cc:1511] Processing CreateTablet for tablet 7ba56fbb3caa417db1c8d78a9caa28da (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:16:43.371512  6700 ts_tablet_manager.cc:1434] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.066s	user 0.007s	sys 0.000s
I20260709 12:16:43.371740  6574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ba56fbb3caa417db1c8d78a9caa28da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.372594  6700 tablet_bootstrap.cc:492] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:43.353724  6647 tablet_service.cc:1511] Processing CreateTablet for tablet fa09afefaa884dc5943b210cb23a07b3 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:16:43.376382  6573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa09afefaa884dc5943b210cb23a07b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.386883  6700 tablet_bootstrap.cc:654] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:43.387471  6647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa09afefaa884dc5943b210cb23a07b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.378454  6698 tablet_bootstrap.cc:654] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:43.379199  6705 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:43.390512  6705 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:43.392820  6699 tablet_bootstrap.cc:654] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:43.394431  6705 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:43.443045  6700 tablet_bootstrap.cc:492] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:43.443657  6700 ts_tablet_manager.cc:1403] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.071s	user 0.012s	sys 0.045s
I20260709 12:16:43.444530  6698 tablet_bootstrap.cc:492] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:43.445199  6698 ts_tablet_manager.cc:1403] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.078s	user 0.016s	sys 0.000s
I20260709 12:16:43.446858  6700 raft_consensus.cc:359] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:43.447742  6700 raft_consensus.cc:385] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:43.448107  6700 raft_consensus.cc:740] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:43.449108  6700 consensus_queue.cc:260] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:43.451275  6644 tablet_service.cc:1511] Processing CreateTablet for tablet 7ba56fbb3caa417db1c8d78a9caa28da (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:16:43.451517  6574 tablet_service.cc:1511] Processing CreateTablet for tablet eebce9eca2a5496bb4a3ab473a243166 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:16:43.451530  6571 tablet_service.cc:1511] Processing CreateTablet for tablet 8ea6f08346334787a6b558ca10a70249 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:16:43.452777  6574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eebce9eca2a5496bb4a3ab473a243166. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.451597  6645 tablet_service.cc:1511] Processing CreateTablet for tablet eebce9eca2a5496bb4a3ab473a243166 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:16:43.455353  6571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ea6f08346334787a6b558ca10a70249. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.452777  6644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ba56fbb3caa417db1c8d78a9caa28da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.460752  6645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eebce9eca2a5496bb4a3ab473a243166. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.465988  6496 tablet_service.cc:1511] Processing CreateTablet for tablet eebce9eca2a5496bb4a3ab473a243166 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:16:43.467541  6496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eebce9eca2a5496bb4a3ab473a243166. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.469442  6497 tablet_service.cc:1511] Processing CreateTablet for tablet 8ea6f08346334787a6b558ca10a70249 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:16:43.470697  6497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ea6f08346334787a6b558ca10a70249. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:43.471215  6455 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:43.449193  6698 raft_consensus.cc:359] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:43.472967  6698 raft_consensus.cc:385] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:43.473345  6698 raft_consensus.cc:740] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:43.474349  6698 consensus_queue.cc:260] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:43.476825  6698 ts_tablet_manager.cc:1434] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.031s	user 0.009s	sys 0.000s
I20260709 12:16:43.477447  6646 tablet_service.cc:1511] Processing CreateTablet for tablet 8ea6f08346334787a6b558ca10a70249 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:16:43.482088  6648 tablet_service.cc:1511] Processing CreateTablet for tablet 1c226c3677f8446f837400fd0df3062e (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:16:43.483229  6698 tablet_bootstrap.cc:492] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:43.483232  6646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ea6f08346334787a6b558ca10a70249. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.483953  6648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c226c3677f8446f837400fd0df3062e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.477507  6700 ts_tablet_manager.cc:1434] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.001s
I20260709 12:16:43.485230  6700 tablet_bootstrap.cc:492] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:43.494948  6700 tablet_bootstrap.cc:654] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:43.499969  6699 tablet_bootstrap.cc:492] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:43.501747  6495 tablet_service.cc:1511] Processing CreateTablet for tablet 1c226c3677f8446f837400fd0df3062e (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:16:43.503386  6495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c226c3677f8446f837400fd0df3062e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.504632  6698 tablet_bootstrap.cc:654] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:43.503530  6571 tablet_service.cc:1511] Processing CreateTablet for tablet 1c226c3677f8446f837400fd0df3062e (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:16:43.510067  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:43.512701  6699 ts_tablet_manager.cc:1403] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.152s	user 0.014s	sys 0.051s
W20260709 12:16:43.517431  6475 leader_election.cc:343] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:43.519739  6699 raft_consensus.cc:359] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:43.520650  6699 raft_consensus.cc:385] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:43.522040  6699 raft_consensus.cc:740] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:43.522282  6573 tablet_service.cc:1511] Processing CreateTablet for tablet 3318181e3c0b433a9e1cee517969105c (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:16:43.523011  6699 consensus_queue.cc:260] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:43.525298  6699 ts_tablet_manager.cc:1434] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:16:43.525476  6499 tablet_service.cc:1511] Processing CreateTablet for tablet 3318181e3c0b433a9e1cee517969105c (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:16:43.521656  6571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c226c3677f8446f837400fd0df3062e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.526484  6700 tablet_bootstrap.cc:492] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:43.526746  6499 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3318181e3c0b433a9e1cee517969105c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.526937  6700 ts_tablet_manager.cc:1403] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.042s	user 0.004s	sys 0.012s
I20260709 12:16:43.529790  6700 raft_consensus.cc:359] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:16:43.530224  6706 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:43.530521  6700 raft_consensus.cc:385] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:43.530828  6700 raft_consensus.cc:740] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:43.530685  6706 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:43.531792  6700 consensus_queue.cc:260] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:16:43.533636  6573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3318181e3c0b433a9e1cee517969105c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.533782  6699 tablet_bootstrap.cc:492] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:43.534735  6706 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:43.544505  6704 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:43.545205  6704 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:43.561113  6704 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:43.566532  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:43.567268  6584 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 0.
I20260709 12:16:43.569501  6644 tablet_service.cc:1511] Processing CreateTablet for tablet 3318181e3c0b433a9e1cee517969105c (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:16:43.570767  6644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3318181e3c0b433a9e1cee517969105c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.580303  6574 tablet_service.cc:1511] Processing CreateTablet for tablet f5b42468eaf14fd49be100af079d3921 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:16:43.593835  6574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5b42468eaf14fd49be100af079d3921. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.593483  6648 tablet_service.cc:1511] Processing CreateTablet for tablet f5b42468eaf14fd49be100af079d3921 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:16:43.594280  6647 tablet_service.cc:1511] Processing CreateTablet for tablet 23f03345855041f8aabfa3e94a1867d9 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:16:43.594964  6648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5b42468eaf14fd49be100af079d3921. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.595554  6647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23f03345855041f8aabfa3e94a1867d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.602944  6572 tablet_service.cc:1511] Processing CreateTablet for tablet 23f03345855041f8aabfa3e94a1867d9 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:16:43.604825  6572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23f03345855041f8aabfa3e94a1867d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.614301  6698 tablet_bootstrap.cc:492] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:43.614821  6698 ts_tablet_manager.cc:1403] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.132s	user 0.012s	sys 0.012s
I20260709 12:16:43.618585  6699 tablet_bootstrap.cc:654] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:43.618717  6473 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:43.621160  6706 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:43.622769  6705 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:43.635990  6698 raft_consensus.cc:359] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:16:43.636835  6698 raft_consensus.cc:385] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:43.637137  6698 raft_consensus.cc:740] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:43.637867  6698 consensus_queue.cc:260] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:16:43.642849  6570 tablet_service.cc:1511] Processing CreateTablet for tablet c50fba48b82e4ea29ce3f314b054d4ec (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:16:43.644136  6570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c50fba48b82e4ea29ce3f314b054d4ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.643567  6706 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:43.647704  6700 ts_tablet_manager.cc:1434] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.120s	user 0.019s	sys 0.007s
I20260709 12:16:43.647578  6569 tablet_service.cc:1511] Processing CreateTablet for tablet 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:16:43.649025  6569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d80f18ce04d4d97b0b3c5dc2ed0e4a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.649083  6700 tablet_bootstrap.cc:492] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:43.652356  6647 tablet_service.cc:1511] Processing CreateTablet for tablet c50fba48b82e4ea29ce3f314b054d4ec (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:16:43.633963  6705 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:43.656947  6705 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:43.661935  6706 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:43.662920  6646 tablet_service.cc:1511] Processing CreateTablet for tablet 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:16:43.664201  6646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d80f18ce04d4d97b0b3c5dc2ed0e4a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.673417  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:43.673491  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:43.675081  6657 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:43.688210  6497 tablet_service.cc:1511] Processing CreateTablet for tablet 23f03345855041f8aabfa3e94a1867d9 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
W20260709 12:16:43.688897  6550 leader_election.cc:343] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:43.690121  6497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23f03345855041f8aabfa3e94a1867d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.690402  6550 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:43.691419  6706 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:43.691815  6706 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:43.693797  6706 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:43.713481  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:43.711848  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:16:43.716543  6548 leader_election.cc:343] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:43.720870  6647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c50fba48b82e4ea29ce3f314b054d4ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.725695  6700 tablet_bootstrap.cc:654] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:43.729781  6569 tablet_service.cc:1511] Processing CreateTablet for tablet e416f9423f964ba5a29e536e8160c627 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:16:43.731568  6569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e416f9423f964ba5a29e536e8160c627. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.731942  6498 tablet_service.cc:1511] Processing CreateTablet for tablet e416f9423f964ba5a29e536e8160c627 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:16:43.732820  6496 tablet_service.cc:1511] Processing CreateTablet for tablet 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:16:43.731257  6495 tablet_service.cc:1511] Processing CreateTablet for tablet c50fba48b82e4ea29ce3f314b054d4ec (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:16:43.733210  6498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e416f9423f964ba5a29e536e8160c627. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.735893  6648 tablet_service.cc:1511] Processing CreateTablet for tablet e416f9423f964ba5a29e536e8160c627 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:16:43.736442  6496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d80f18ce04d4d97b0b3c5dc2ed0e4a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.737108  6648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e416f9423f964ba5a29e536e8160c627. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.739229  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:43.740556  6495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c50fba48b82e4ea29ce3f314b054d4ec. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:43.744902  6622 leader_election.cc:343] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:43.743597  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:43.748889  6705 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:43.750119  6509 raft_consensus.cc:2393] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 1: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:16:43.750041  6718 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:43.750722  6718 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:16:43.751986  6705 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:43.753144  6718 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:16:43.748055  6622 leader_election.cc:343] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:43.753799  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:43.754891  6548 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:43.757273  6704 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:43.758787  6718 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:43.759044  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:16:43.760841  6550 leader_election.cc:343] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:43.762688  6644 tablet_service.cc:1511] Processing CreateTablet for tablet 6f94b1cd286648a9b82d5abf69bb86d5 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:16:43.764014  6644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f94b1cd286648a9b82d5abf69bb86d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.753684  6706 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:43.754621  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:43.757936  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:16:43.765323  6475 leader_election.cc:343] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:43.765889  6718 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:43.766486  6706 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:43.729813  6499 tablet_service.cc:1511] Processing CreateTablet for tablet f5b42468eaf14fd49be100af079d3921 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:16:43.769555  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:43.770156  6657 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:43.772516  6699 tablet_bootstrap.cc:492] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:43.772536  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:43.774721  6645 tablet_service.cc:1511] Processing CreateTablet for tablet c6b5198a4b6741cdb13e49cf4e90173c (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:16:43.778993  6699 ts_tablet_manager.cc:1403] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.245s	user 0.027s	sys 0.013s
W20260709 12:16:43.779769  6548 leader_election.cc:343] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:43.781198  6645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6b5198a4b6741cdb13e49cf4e90173c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.765893  6509 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:43.777943  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:43.799197  6548 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:43.800071  6718 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:43.800474  6718 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:43.800801  6718 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:43.802752  6493 tablet_service.cc:1511] Processing CreateTablet for tablet c6b5198a4b6741cdb13e49cf4e90173c (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:16:43.803488  6699 raft_consensus.cc:359] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:43.804266  6699 raft_consensus.cc:385] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:43.804314  6643 tablet_service.cc:1511] Processing CreateTablet for tablet 14cf1c3a63624d94aa6c9c356e6108e2 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:16:43.804870  6699 raft_consensus.cc:740] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:43.805332  6494 tablet_service.cc:1511] Processing CreateTablet for tablet 6f94b1cd286648a9b82d5abf69bb86d5 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:16:43.807585  6699 consensus_queue.cc:260] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:43.808269  6499 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5b42468eaf14fd49be100af079d3921. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.809973  6699 ts_tablet_manager.cc:1434] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.029s	user 0.002s	sys 0.005s
I20260709 12:16:43.807444  6643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14cf1c3a63624d94aa6c9c356e6108e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.813977  6699 tablet_bootstrap.cc:492] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:43.816478  6571 tablet_service.cc:1511] Processing CreateTablet for tablet c6b5198a4b6741cdb13e49cf4e90173c (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:16:43.817785  6571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6b5198a4b6741cdb13e49cf4e90173c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.823447  6699 tablet_bootstrap.cc:654] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:43.826979  6573 tablet_service.cc:1511] Processing CreateTablet for tablet 6f94b1cd286648a9b82d5abf69bb86d5 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:16:43.830361  6574 tablet_service.cc:1511] Processing CreateTablet for tablet 14cf1c3a63624d94aa6c9c356e6108e2 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:16:43.831086  6492 tablet_service.cc:1511] Processing CreateTablet for tablet 14cf1c3a63624d94aa6c9c356e6108e2 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:16:43.831812  6573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f94b1cd286648a9b82d5abf69bb86d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.832429  6492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14cf1c3a63624d94aa6c9c356e6108e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.834393  6718 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:16:43.837325  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:43.837910  6509 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:43.842933  6718 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:43.843953  6493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6b5198a4b6741cdb13e49cf4e90173c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.844030  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:16:43.845546  6550 leader_election.cc:343] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:43.846365  6494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f94b1cd286648a9b82d5abf69bb86d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.858546  6574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14cf1c3a63624d94aa6c9c356e6108e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.869704  6584 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:16:43.870873  6473 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:43.871851  6705 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:43.872254  6698 ts_tablet_manager.cc:1434] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.257s	user 0.055s	sys 0.075s
I20260709 12:16:43.872402  6705 raft_consensus.cc:697] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:16:43.873481  6698 tablet_bootstrap.cc:492] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:43.873351  6705 consensus_queue.cc:237] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:43.881923  6573 tablet_service.cc:1511] Processing CreateTablet for tablet c9d981014b2c4cda80bca766fe58fe42 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:16:43.882309  6496 tablet_service.cc:1511] Processing CreateTablet for tablet c9d981014b2c4cda80bca766fe58fe42 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:16:43.883284  6573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9d981014b2c4cda80bca766fe58fe42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.883400  6496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9d981014b2c4cda80bca766fe58fe42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.893942  6720 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:43.894497  6720 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:43.897447  6720 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:43.898069  6657 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:16:43.899607  6550 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:43.901046  6718 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:43.904433  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:43.902252  6718 raft_consensus.cc:697] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:16:43.908224  6475 leader_election.cc:343] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:43.908366  6718 consensus_queue.cc:237] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:43.899607  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:43.912204  6495 tablet_service.cc:1511] Processing CreateTablet for tablet e423f656ad70421f959a9a3b81dac708 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:16:43.913808  6509 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:16:43.913945  6495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e423f656ad70421f959a9a3b81dac708. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.918036  6643 tablet_service.cc:1511] Processing CreateTablet for tablet c9d981014b2c4cda80bca766fe58fe42 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
W20260709 12:16:43.919121  6473 leader_election.cc:343] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:43.923444  6473 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:43.925705  6644 tablet_service.cc:1511] Processing CreateTablet for tablet e423f656ad70421f959a9a3b81dac708 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:16:43.926123  6705 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:43.926951  6644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e423f656ad70421f959a9a3b81dac708. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.930630  6646 tablet_service.cc:1511] Processing CreateTablet for tablet 13ab2c6da5994ae2a955fc11d1761c41 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:16:43.931998  6646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13ab2c6da5994ae2a955fc11d1761c41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.934680  6407 catalog_manager.cc:5697] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 0 to 1, leader changed from <none> to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:43.921619  6570 tablet_service.cc:1511] Processing CreateTablet for tablet e423f656ad70421f959a9a3b81dac708 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:16:43.938004  6570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e423f656ad70421f959a9a3b81dac708. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.938349  6642 tablet_service.cc:1511] Processing CreateTablet for tablet ef43f1290994490e8ef291fa12012731 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:16:43.939812  6642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef43f1290994490e8ef291fa12012731. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.940991  6641 tablet_service.cc:1511] Processing CreateTablet for tablet 6d6c58127aec4100b761ec4b52b842a0 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:16:43.942329  6641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d6c58127aec4100b761ec4b52b842a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.946470  6640 tablet_service.cc:1511] Processing CreateTablet for tablet dbf5342782d14193986ec66c46ce646e (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:16:43.947880  6640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbf5342782d14193986ec66c46ce646e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.953861  6699 tablet_bootstrap.cc:492] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:43.954380  6699 ts_tablet_manager.cc:1403] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.141s	user 0.018s	sys 0.015s
I20260709 12:16:43.922009  6648 tablet_service.cc:1511] Processing CreateTablet for tablet 591037e9504e4348a9cdf8842f34be22 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:16:43.957803  6699 raft_consensus.cc:359] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:16:43.958606  6699 raft_consensus.cc:385] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:43.958986  6639 tablet_service.cc:1511] Processing CreateTablet for tablet 8066a9d7ccf34c89ad8d3df18c4e86c2 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:16:43.965617  6497 tablet_service.cc:1511] Processing CreateTablet for tablet 591037e9504e4348a9cdf8842f34be22 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:16:43.966755  6700 tablet_bootstrap.cc:492] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:43.966917  6497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 591037e9504e4348a9cdf8842f34be22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:43.967489  6700 ts_tablet_manager.cc:1403] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.319s	user 0.012s	sys 0.026s
I20260709 12:16:43.970429  6700 raft_consensus.cc:359] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:43.971168  6700 raft_consensus.cc:385] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:43.971443  6700 raft_consensus.cc:740] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:43.972390  6548 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:43.973922  6706 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:43.972321  6700 consensus_queue.cc:260] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:43.958977  6699 raft_consensus.cc:740] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:43.980994  6699 consensus_queue.cc:260] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:16:43.991365  6699 ts_tablet_manager.cc:1434] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.037s	user 0.005s	sys 0.001s
I20260709 12:16:43.964082  6698 tablet_bootstrap.cc:654] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:44.007915  6643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9d981014b2c4cda80bca766fe58fe42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.009205  6574 tablet_service.cc:1511] Processing CreateTablet for tablet 591037e9504e4348a9cdf8842f34be22 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:16:44.009264  6494 tablet_service.cc:1511] Processing CreateTablet for tablet 13ab2c6da5994ae2a955fc11d1761c41 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:16:44.010689  6494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13ab2c6da5994ae2a955fc11d1761c41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.012925  6639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8066a9d7ccf34c89ad8d3df18c4e86c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.010699  6574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 591037e9504e4348a9cdf8842f34be22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.017961  6571 tablet_service.cc:1511] Processing CreateTablet for tablet 13ab2c6da5994ae2a955fc11d1761c41 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:16:44.019368  6571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13ab2c6da5994ae2a955fc11d1761c41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.020147  6647 tablet_service.cc:1511] Processing CreateTablet for tablet 3e7223833e1f47b7b68c0080facbbe95 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:16:44.021472  6647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e7223833e1f47b7b68c0080facbbe95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.025609  6648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 591037e9504e4348a9cdf8842f34be22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.029098  6645 tablet_service.cc:1511] Processing CreateTablet for tablet 27473d665a69492b9d08ab7409de0b5f (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:16:44.030673  6645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27473d665a69492b9d08ab7409de0b5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.029165  6644 tablet_service.cc:1511] Processing CreateTablet for tablet c27974d342ea4901aaa3946871539818 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:16:44.032586  6644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c27974d342ea4901aaa3946871539818. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.046326  6493 tablet_service.cc:1511] Processing CreateTablet for tablet ef43f1290994490e8ef291fa12012731 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:16:44.046788  6569 tablet_service.cc:1511] Processing CreateTablet for tablet ef43f1290994490e8ef291fa12012731 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:16:44.047675  6493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef43f1290994490e8ef291fa12012731. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.048069  6569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef43f1290994490e8ef291fa12012731. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.061094  6699 tablet_bootstrap.cc:492] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:44.107081  6573 tablet_service.cc:1511] Processing CreateTablet for tablet 6d6c58127aec4100b761ec4b52b842a0 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:16:44.108584  6497 tablet_service.cc:1511] Processing CreateTablet for tablet 6d6c58127aec4100b761ec4b52b842a0 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:16:44.110343  6497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d6c58127aec4100b761ec4b52b842a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.120857  6718 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:44.121579  6718 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:44.125365  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:16:44.126668  6548 leader_election.cc:343] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:44.161387  6699 tablet_bootstrap.cc:654] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:44.154095  6718 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:44.147720  6405 catalog_manager.cc:5697] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:44.168087  6706 raft_consensus.cc:697] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:16:44.173377  6573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d6c58127aec4100b761ec4b52b842a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.171262  6704 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:44.181280  6496 tablet_service.cc:1511] Processing CreateTablet for tablet dbf5342782d14193986ec66c46ce646e (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:16:44.182608  6496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbf5342782d14193986ec66c46ce646e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.183560  6574 tablet_service.cc:1511] Processing CreateTablet for tablet dbf5342782d14193986ec66c46ce646e (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:16:44.185052  6706 consensus_queue.cc:237] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:16:44.186355  6640 tablet_service.cc:1511] Processing CreateTablet for tablet 83301542a88f46cea64bc905b3199975 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:16:44.187740  6640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83301542a88f46cea64bc905b3199975. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.189496  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:44.190392  6658 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:44.192298  6550 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:44.193861  6718 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:44.194337  6718 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:44.194787  6718 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:44.197876  6641 tablet_service.cc:1511] Processing CreateTablet for tablet 72269ad6925341548796bb34823b3e00 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:16:44.199630  6641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72269ad6925341548796bb34823b3e00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.199966  6494 tablet_service.cc:1511] Processing CreateTablet for tablet 8066a9d7ccf34c89ad8d3df18c4e86c2 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:16:44.200501  6704 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:44.201702  6494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8066a9d7ccf34c89ad8d3df18c4e86c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.221019  6644 tablet_service.cc:1511] Processing CreateTablet for tablet 2ebfdf7a730a407185ead1aedcf4b8c4 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:16:44.222271  6644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ebfdf7a730a407185ead1aedcf4b8c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.223320  6648 tablet_service.cc:1511] Processing CreateTablet for tablet 5b0b7ace18d848afa42e884bcd2475a4 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:16:44.226549  6698 tablet_bootstrap.cc:492] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:44.226119  6639 tablet_service.cc:1511] Processing CreateTablet for tablet 8e01550f5e994006a4031ca81edcc9aa (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:16:44.227033  6698 ts_tablet_manager.cc:1403] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.354s	user 0.042s	sys 0.049s
I20260709 12:16:44.203176  6646 tablet_service.cc:1511] Processing CreateTablet for tablet b7d79febb5824e78a295b2c993c14108 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:16:44.250031  6492 tablet_service.cc:1511] Processing CreateTablet for tablet 3e7223833e1f47b7b68c0080facbbe95 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:16:44.254642  6492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e7223833e1f47b7b68c0080facbbe95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.255174  6569 tablet_service.cc:1511] Processing CreateTablet for tablet 8066a9d7ccf34c89ad8d3df18c4e86c2 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:16:44.256175  6700 ts_tablet_manager.cc:1434] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.288s	user 0.030s	sys 0.026s
I20260709 12:16:44.257155  6704 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:44.246577  6574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbf5342782d14193986ec66c46ce646e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.259774  6644 tablet_service.cc:1511] Processing CreateTablet for tablet 3e0ab356fc074f589a6ff4f9b4996b09 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:16:44.257340  6700 tablet_bootstrap.cc:492] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:44.261591  6643 tablet_service.cc:1511] Processing CreateTablet for tablet f9f8f5d393744ebabbed2f3e0131cf40 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:16:44.261864  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:44.262626  6584 raft_consensus.cc:2393] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 1: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:16:44.267925  6700 tablet_bootstrap.cc:654] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:44.272243  6639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e01550f5e994006a4031ca81edcc9aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.278437  6698 raft_consensus.cc:359] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:44.287957  6698 raft_consensus.cc:385] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:44.288909  6704 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:44.289372  6704 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:16:44.291483  6704 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:44.292929  6572 tablet_service.cc:1511] Processing CreateTablet for tablet c27974d342ea4901aaa3946871539818 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:16:44.293435  6568 tablet_service.cc:1511] Processing CreateTablet for tablet b7d79febb5824e78a295b2c993c14108 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:16:44.295238  6567 tablet_service.cc:1511] Processing CreateTablet for tablet 5b0b7ace18d848afa42e884bcd2475a4 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:16:44.296651  6498 tablet_service.cc:1511] Processing CreateTablet for tablet b7d79febb5824e78a295b2c993c14108 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:16:44.296934  6499 tablet_service.cc:1511] Processing CreateTablet for tablet 5b0b7ace18d848afa42e884bcd2475a4 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:16:44.297433  6566 tablet_service.cc:1511] Processing CreateTablet for tablet 2ebfdf7a730a407185ead1aedcf4b8c4 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:16:44.297945  6498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7d79febb5824e78a295b2c993c14108. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.298614  6499 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b0b7ace18d848afa42e884bcd2475a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.298918  6493 tablet_service.cc:1511] Processing CreateTablet for tablet 27473d665a69492b9d08ab7409de0b5f (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:16:44.299675  6491 tablet_service.cc:1511] Processing CreateTablet for tablet 2ebfdf7a730a407185ead1aedcf4b8c4 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:16:44.285674  6569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8066a9d7ccf34c89ad8d3df18c4e86c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.286703  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:44.287619  6646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7d79febb5824e78a295b2c993c14108. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.300163  6493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27473d665a69492b9d08ab7409de0b5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.278298  6642 tablet_service.cc:1511] Processing CreateTablet for tablet cbb34f14626f4aa895bae63ea2466e00 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:16:44.309856  6565 tablet_service.cc:1511] Processing CreateTablet for tablet 8e01550f5e994006a4031ca81edcc9aa (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
W20260709 12:16:44.312681  6548 consensus_peers.cc:597] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:44.312942  6490 tablet_service.cc:1511] Processing CreateTablet for tablet 8e01550f5e994006a4031ca81edcc9aa (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:16:44.314409  6644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e0ab356fc074f589a6ff4f9b4996b09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.313879  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:44.316689  6706 consensus_queue.cc:1048] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:44.317332  6495 tablet_service.cc:1511] Processing CreateTablet for tablet c27974d342ea4901aaa3946871539818 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:16:44.317649  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:44.318383  6508 raft_consensus.cc:2393] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 1: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:16:44.321027  6622 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:44.323351  6704 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:44.302583  6718 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:44.329272  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:44.329591  6570 tablet_service.cc:1511] Processing CreateTablet for tablet 27473d665a69492b9d08ab7409de0b5f (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
W20260709 12:16:44.330713  6548 leader_election.cc:343] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:44.331444  6491 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ebfdf7a730a407185ead1aedcf4b8c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.332055  6490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e01550f5e994006a4031ca81edcc9aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.288419  6698 raft_consensus.cc:740] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:44.339208  6495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c27974d342ea4901aaa3946871539818. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.343541  6567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b0b7ace18d848afa42e884bcd2475a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.345472  6496 tablet_service.cc:1511] Processing CreateTablet for tablet f9f8f5d393744ebabbed2f3e0131cf40 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:16:44.347494  6574 tablet_service.cc:1511] Processing CreateTablet for tablet f9f8f5d393744ebabbed2f3e0131cf40 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:16:44.350247  6496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9f8f5d393744ebabbed2f3e0131cf40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.352739  6574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9f8f5d393744ebabbed2f3e0131cf40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.365988  6565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e01550f5e994006a4031ca81edcc9aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.366732  6698 consensus_queue.cc:260] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:44.367095  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:44.367746  6657 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:44.369205  6698 ts_tablet_manager.cc:1434] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.142s	user 0.009s	sys 0.000s
I20260709 12:16:44.370241  6698 tablet_bootstrap.cc:492] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:44.371404  6571 tablet_service.cc:1511] Processing CreateTablet for tablet 3e7223833e1f47b7b68c0080facbbe95 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:16:44.372681  6571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e7223833e1f47b7b68c0080facbbe95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.375553  6657 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:16:44.377308  6725 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:44.377816  6725 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:44.379174  6648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b0b7ace18d848afa42e884bcd2475a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.381031  6725 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:44.381083  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:44.382619  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:44.384155  6475 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:44.384289  6550 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:44.363778  6718 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:44.385598  6725 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:44.388938  6706 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:44.389413  6706 raft_consensus.cc:697] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:16:44.390223  6706 consensus_queue.cc:237] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:44.390760  6573 tablet_service.cc:1511] Processing CreateTablet for tablet 83301542a88f46cea64bc905b3199975 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:16:44.398655  6640 tablet_service.cc:1511] Processing CreateTablet for tablet c7346d993d414b4197cec453cd7bf127 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:16:44.408059  6643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9f8f5d393744ebabbed2f3e0131cf40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.408776  6640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7346d993d414b4197cec453cd7bf127. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.300886  6647 tablet_service.cc:1511] Processing CreateTablet for tablet 7492a9464009416aa4c3a11e145dec61 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:16:44.410545  6642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbb34f14626f4aa895bae63ea2466e00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.411187  6647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7492a9464009416aa4c3a11e145dec61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.414690  6572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c27974d342ea4901aaa3946871539818. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.466684  6574 tablet_service.cc:1511] Processing CreateTablet for tablet 72269ad6925341548796bb34823b3e00 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:16:44.469431  6725 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
W20260709 12:16:44.479925  6455 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:44.484408  6570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27473d665a69492b9d08ab7409de0b5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.485119  6566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ebfdf7a730a407185ead1aedcf4b8c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.502996  6571 tablet_service.cc:1511] Processing CreateTablet for tablet 3e0ab356fc074f589a6ff4f9b4996b09 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:16:44.507382  6704 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:44.508441  6490 tablet_service.cc:1511] Processing CreateTablet for tablet 83301542a88f46cea64bc905b3199975 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:16:44.509756  6490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83301542a88f46cea64bc905b3199975. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.509920  6657 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:44.522352  6699 tablet_bootstrap.cc:492] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:44.522461  6725 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:44.523056  6699 ts_tablet_manager.cc:1403] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.462s	user 0.019s	sys 0.008s
I20260709 12:16:44.479321  6645 tablet_service.cc:1511] Processing CreateTablet for tablet d6029ffeecb94e4da196c61c032b862b (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:16:44.525795  6645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6029ffeecb94e4da196c61c032b862b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.525849  6700 tablet_bootstrap.cc:492] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:44.526559  6700 ts_tablet_manager.cc:1403] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.269s	user 0.010s	sys 0.015s
W20260709 12:16:44.445402  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 123857us
I20260709 12:16:44.533310  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:44.536170  6490 tablet_service.cc:1511] Processing CreateTablet for tablet cbb34f14626f4aa895bae63ea2466e00 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:16:44.529873  6700 raft_consensus.cc:359] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:44.539285  6644 tablet_service.cc:1511] Processing CreateTablet for tablet bd6fb503c26a49bab14a18c9b5e2d792 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:16:44.544137  6644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd6fb503c26a49bab14a18c9b5e2d792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.498699  6698 tablet_bootstrap.cc:654] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:44.546236  6496 tablet_service.cc:1511] Processing CreateTablet for tablet 3e0ab356fc074f589a6ff4f9b4996b09 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:16:44.552353  6492 tablet_service.cc:1511] Processing CreateTablet for tablet 72269ad6925341548796bb34823b3e00 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:16:44.553943  6492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72269ad6925341548796bb34823b3e00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.548450  6704 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:44.545606  6573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83301542a88f46cea64bc905b3199975. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.566321  6571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e0ab356fc074f589a6ff4f9b4996b09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.542680  6700 raft_consensus.cc:385] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:44.566838  6700 raft_consensus.cc:740] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:44.568776  6568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7d79febb5824e78a295b2c993c14108. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.570160  6574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72269ad6925341548796bb34823b3e00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.571832  6700 consensus_queue.cc:260] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:16:44.582894  6392 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 84852.5us
I20260709 12:16:44.603051  6499 tablet_service.cc:1511] Processing CreateTablet for tablet 7492a9464009416aa4c3a11e145dec61 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:16:44.605209  6565 tablet_service.cc:1511] Processing CreateTablet for tablet 7492a9464009416aa4c3a11e145dec61 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:16:44.606380  6493 tablet_service.cc:1511] Processing CreateTablet for tablet 55df50d51529421789bb752f47dff41a (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:16:44.607024  6565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7492a9464009416aa4c3a11e145dec61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.580142  6490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbb34f14626f4aa895bae63ea2466e00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.609381  6496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e0ab356fc074f589a6ff4f9b4996b09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.612712  6494 tablet_service.cc:1511] Processing CreateTablet for tablet c7346d993d414b4197cec453cd7bf127 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:16:44.623661  6728 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:44.603050  6569 tablet_service.cc:1511] Processing CreateTablet for tablet cbb34f14626f4aa895bae63ea2466e00 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:16:44.652647  6569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbb34f14626f4aa895bae63ea2466e00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.656957  6700 ts_tablet_manager.cc:1434] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.130s	user 0.005s	sys 0.003s
I20260709 12:16:44.649766  6499 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7492a9464009416aa4c3a11e145dec61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.671042  6571 tablet_service.cc:1511] Processing CreateTablet for tablet c7346d993d414b4197cec453cd7bf127 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:16:44.673442  6498 tablet_service.cc:1511] Processing CreateTablet for tablet bd6fb503c26a49bab14a18c9b5e2d792 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:16:44.653088  6495 tablet_service.cc:1511] Processing CreateTablet for tablet f2592253d3214104865730480e057d09 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:16:44.652297  6699 raft_consensus.cc:359] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:44.686470  6699 raft_consensus.cc:385] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:44.686781  6699 raft_consensus.cc:740] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:44.678244  6700 tablet_bootstrap.cc:492] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:44.680720  6574 tablet_service.cc:1511] Processing CreateTablet for tablet bd6fb503c26a49bab14a18c9b5e2d792 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:16:44.671518  6496 tablet_service.cc:1511] Processing CreateTablet for tablet d6029ffeecb94e4da196c61c032b862b (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:16:44.681910  6571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7346d993d414b4197cec453cd7bf127. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.698530  6698 tablet_bootstrap.cc:492] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:44.698256  6497 tablet_service.cc:1511] Processing CreateTablet for tablet ab430742d4f44bf79c4591f613055cd6 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:16:44.699067  6698 ts_tablet_manager.cc:1403] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.329s	user 0.019s	sys 0.006s
I20260709 12:16:44.702121  6698 raft_consensus.cc:359] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:44.702558  6647 tablet_service.cc:1511] Processing CreateTablet for tablet 555356e0b31d45c1bc3963dfe39b2df3 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:16:44.702901  6698 raft_consensus.cc:385] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:44.707180  6644 tablet_service.cc:1511] Processing CreateTablet for tablet 55df50d51529421789bb752f47dff41a (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:16:44.708986  6644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55df50d51529421789bb752f47dff41a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.703267  6698 raft_consensus.cc:740] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:44.713778  6574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd6fb503c26a49bab14a18c9b5e2d792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.722411  6641 tablet_service.cc:1511] Processing CreateTablet for tablet ab430742d4f44bf79c4591f613055cd6 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:16:44.687520  6699 consensus_queue.cc:260] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:44.733534  6700 tablet_bootstrap.cc:654] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:44.734889  6728 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:44.736627  6495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2592253d3214104865730480e057d09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.736419  6643 tablet_service.cc:1511] Processing CreateTablet for tablet f2592253d3214104865730480e057d09 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:16:44.740835  6698 consensus_queue.cc:260] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:44.744009  6699 ts_tablet_manager.cc:1434] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.221s	user 0.006s	sys 0.006s
I20260709 12:16:44.746444  6497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab430742d4f44bf79c4591f613055cd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.745558  6491 tablet_service.cc:1511] Processing CreateTablet for tablet 555356e0b31d45c1bc3963dfe39b2df3 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:16:44.760839  6646 tablet_service.cc:1511] Processing CreateTablet for tablet 5b17fd01574f4601b33dd37e7c78bc8f (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:16:44.761021  6640 tablet_service.cc:1511] Processing CreateTablet for tablet 48e50fd265d348779de32dff1ac6db39 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:16:44.786639  6571 tablet_service.cc:1511] Processing CreateTablet for tablet f2592253d3214104865730480e057d09 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:16:44.787830  6571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2592253d3214104865730480e057d09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.760857  6699 tablet_bootstrap.cc:492] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:44.788378  6647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 555356e0b31d45c1bc3963dfe39b2df3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.796846  6574 tablet_service.cc:1511] Processing CreateTablet for tablet d6029ffeecb94e4da196c61c032b862b (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:16:44.802333  6573 tablet_service.cc:1511] Processing CreateTablet for tablet 555356e0b31d45c1bc3963dfe39b2df3 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:16:44.803313  6573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 555356e0b31d45c1bc3963dfe39b2df3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.804489  6642 tablet_service.cc:1511] Processing CreateTablet for tablet e6167e4ddf454440bfc602ffbcd6da3e (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:16:44.805410  6569 tablet_service.cc:1511] Processing CreateTablet for tablet 55df50d51529421789bb752f47dff41a (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:16:44.807587  6572 tablet_service.cc:1511] Processing CreateTablet for tablet ab430742d4f44bf79c4591f613055cd6 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:16:44.808183  6570 tablet_service.cc:1511] Processing CreateTablet for tablet 5b17fd01574f4601b33dd37e7c78bc8f (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:16:44.809798  6643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2592253d3214104865730480e057d09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.810528  6642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6167e4ddf454440bfc602ffbcd6da3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.799415  6498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd6fb503c26a49bab14a18c9b5e2d792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.814817  6639 tablet_service.cc:1511] Processing CreateTablet for tablet 3a47d71c34b541b9b2589b4e80413fa2 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:16:44.815711  6641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab430742d4f44bf79c4591f613055cd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.817889  6698 ts_tablet_manager.cc:1434] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.118s	user 0.017s	sys 0.007s
I20260709 12:16:44.818874  6698 tablet_bootstrap.cc:492] T d1778f67a2ba4fc2825ec2be5097d77a P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:44.819448  6567 tablet_service.cc:1511] Processing CreateTablet for tablet 99e5417fa1d54082a5690b657f407f70 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:16:44.816145  6491 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 555356e0b31d45c1bc3963dfe39b2df3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.822276  6568 tablet_service.cc:1511] Processing CreateTablet for tablet 48e50fd265d348779de32dff1ac6db39 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:16:44.824223  6574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6029ffeecb94e4da196c61c032b862b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.829847  6494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7346d993d414b4197cec453cd7bf127. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.831089  6648 tablet_service.cc:1511] Processing CreateTablet for tablet d0bd537a4ec844a69ca8227a47d1a8f3 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:16:44.832962  6496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6029ffeecb94e4da196c61c032b862b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.836691  6640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48e50fd265d348779de32dff1ac6db39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.837704  6570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b17fd01574f4601b33dd37e7c78bc8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.840658  6572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab430742d4f44bf79c4591f613055cd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.842587  6407 catalog_manager.cc:5697] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } }
I20260709 12:16:44.848675  6567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99e5417fa1d54082a5690b657f407f70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.871944  6493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55df50d51529421789bb752f47dff41a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.877301  6496 tablet_service.cc:1511] Processing CreateTablet for tablet d0bd537a4ec844a69ca8227a47d1a8f3 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:16:44.877343  6571 tablet_service.cc:1511] Processing CreateTablet for tablet e6167e4ddf454440bfc602ffbcd6da3e (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:16:44.874213  6568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48e50fd265d348779de32dff1ac6db39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.898779  6569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55df50d51529421789bb752f47dff41a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.901961  6566 tablet_service.cc:1511] Processing CreateTablet for tablet d0bd537a4ec844a69ca8227a47d1a8f3 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:16:44.903502  6566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0bd537a4ec844a69ca8227a47d1a8f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.909096  6571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6167e4ddf454440bfc602ffbcd6da3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.879556  6497 tablet_service.cc:1511] Processing CreateTablet for tablet 5b17fd01574f4601b33dd37e7c78bc8f (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
W20260709 12:16:44.880582  6394 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 49511.7us
I20260709 12:16:44.881886  6574 tablet_service.cc:1511] Processing CreateTablet for tablet 3a47d71c34b541b9b2589b4e80413fa2 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:16:44.882041  6498 tablet_service.cc:1511] Processing CreateTablet for tablet 48e50fd265d348779de32dff1ac6db39 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
W20260709 12:16:44.909945  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 144875us
I20260709 12:16:44.938290  6699 tablet_bootstrap.cc:654] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:44.940303  6508 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:44.940920  6507 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:44.942487  6498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48e50fd265d348779de32dff1ac6db39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.942847  6492 tablet_service.cc:1511] Processing CreateTablet for tablet e6167e4ddf454440bfc602ffbcd6da3e (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:16:44.871730  6646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b17fd01574f4601b33dd37e7c78bc8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.935832  6700 tablet_bootstrap.cc:492] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:44.968864  6700 ts_tablet_manager.cc:1403] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.291s	user 0.016s	sys 0.017s
I20260709 12:16:44.969249  6574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a47d71c34b541b9b2589b4e80413fa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.971110  6648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0bd537a4ec844a69ca8227a47d1a8f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.972724  6639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a47d71c34b541b9b2589b4e80413fa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.974341  6492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6167e4ddf454440bfc602ffbcd6da3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.977653  6496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0bd537a4ec844a69ca8227a47d1a8f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.979763  6497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b17fd01574f4601b33dd37e7c78bc8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:44.980398  6644 tablet_service.cc:1511] Processing CreateTablet for tablet 99e5417fa1d54082a5690b657f407f70 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:16:44.981688  6644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99e5417fa1d54082a5690b657f407f70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:45.000142  6728 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:45.001075  6728 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.004477  6704 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:45.005159  6704 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.010128  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:45.011507  6506 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:45.014814  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:45.019389  6704 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:45.020828  6704 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:45.021936  6704 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:45.022761  6704 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.033322  6728 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:45.035135  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:45.035976  6506 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:45.037331  6548 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:16:45.038656  6718 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:45.039054  6718 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:45.039390  6718 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:45.045560  6392 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 169835us
I20260709 12:16:45.046356  6718 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.066107  6700 raft_consensus.cc:359] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.067879  6698 tablet_bootstrap.cc:654] T d1778f67a2ba4fc2825ec2be5097d77a P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.070900  6645 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:58970 (request call id 47) took 1007 ms. Trace:
I20260709 12:16:45.074913  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:45.082952  6490 tablet_service.cc:1511] Processing CreateTablet for tablet 99e5417fa1d54082a5690b657f407f70 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:16:45.107838  6706 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:45.120451  6718 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:45.121006  6490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99e5417fa1d54082a5690b657f407f70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:45.117026  6700 raft_consensus.cc:385] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.123687  6700 raft_consensus.cc:740] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.123590  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:45.124248  6509 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.124526  6700 consensus_queue.cc:260] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.121050  6753 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:45.125734  6753 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.130046  6700 ts_tablet_manager.cc:1434] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.161s	user 0.004s	sys 0.011s
I20260709 12:16:45.131021  6753 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:45.131578  6700 tablet_bootstrap.cc:492] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:45.131814  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:45.134351  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:45.135056  6656 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:16:45.137235  6475 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:45.137426  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:45.139644  6718 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:45.143167  6753 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:45.117159  6645 rpcz_store.cc:276] 0709 12:16:44.063612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:44.478037 (+414425us) service_pool.cc:224] Handling call
0709 12:16:44.525274 (+ 47237us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:16:44.525386 (+   112us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:16:44.713207 (+187821us) tablet_metadata.cc:665] Metadata flushed
0709 12:16:45.070358 (+357151us) tablet_replica.cc:206] Creating consensus instance
0709 12:16:45.070815 (+   457us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:16:45.148520  6704 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.149996  6658 raft_consensus.cc:2393] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 1: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:16:45.151037  6699 tablet_bootstrap.cc:492] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:45.151671  6699 ts_tablet_manager.cc:1403] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.391s	user 0.018s	sys 0.006s
I20260709 12:16:45.155763  6509 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:16:45.157778  6548 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:45.159353  6728 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:45.160234  6728 raft_consensus.cc:697] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:16:45.161415  6728 consensus_queue.cc:237] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.148949  6753 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:45.112661  6494 tablet_service.cc:1511] Processing CreateTablet for tablet 3a47d71c34b541b9b2589b4e80413fa2 (DEFAULT_TABLE table=test-workload [id=f93362d0518e483fbe3a01f43a217e48]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
W20260709 12:16:45.141997  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 127321us
I20260709 12:16:45.173379  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:45.179862  6700 tablet_bootstrap.cc:654] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.183108  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:45.185835  6704 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:16:45.189306  6392 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 44004.4us
I20260709 12:16:45.217160  6407 catalog_manager.cc:5697] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:45.168598  6753 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:45.221575  6494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a47d71c34b541b9b2589b4e80413fa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:45.223910  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:45.224763  6507 raft_consensus.cc:2393] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 1: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:16:45.225839  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:45.226989  6704 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:45.254427  6699 raft_consensus.cc:359] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.255292  6699 raft_consensus.cc:385] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.255626  6699 raft_consensus.cc:740] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.261008  6699 consensus_queue.cc:260] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.261970  6753 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.263177  6509 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:16:45.265672  6753 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:45.266779  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:45.267230  6718 consensus_queue.cc:1059] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:16:45.269634  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:45.271055  6718 raft_consensus.cc:3055] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:45.271425  6718 raft_consensus.cc:740] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:16:45.272207  6473 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:45.270118  6698 tablet_bootstrap.cc:492] T d1778f67a2ba4fc2825ec2be5097d77a P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:45.274034  6698 ts_tablet_manager.cc:1403] T d1778f67a2ba4fc2825ec2be5097d77a P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.455s	user 0.031s	sys 0.049s
I20260709 12:16:45.278615  6698 raft_consensus.cc:359] T d1778f67a2ba4fc2825ec2be5097d77a P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.279532  6698 raft_consensus.cc:385] T d1778f67a2ba4fc2825ec2be5097d77a P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.279912  6698 raft_consensus.cc:740] T d1778f67a2ba4fc2825ec2be5097d77a P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.272234  6718 consensus_queue.cc:260] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.280675  6405 catalog_manager.cc:5697] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:45.283298  6728 consensus_queue.cc:1175] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f2b06aebe97249a4877d798b8118ece4" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20260709 12:16:45.282014  6718 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:45.294512  6699 ts_tablet_manager.cc:1434] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.142s	user 0.008s	sys 0.004s
I20260709 12:16:45.295639  6699 tablet_bootstrap.cc:492] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:45.309985  6700 tablet_bootstrap.cc:492] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:45.310613  6700 ts_tablet_manager.cc:1403] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.179s	user 0.022s	sys 0.006s
I20260709 12:16:45.312486  6698 consensus_queue.cc:260] T d1778f67a2ba4fc2825ec2be5097d77a P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.300762  6753 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:16:45.318874  6698 ts_tablet_manager.cc:1434] T d1778f67a2ba4fc2825ec2be5097d77a P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.044s	user 0.004s	sys 0.004s
I20260709 12:16:45.319914  6698 tablet_bootstrap.cc:492] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:45.325210  6700 raft_consensus.cc:359] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.326074  6700 raft_consensus.cc:385] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.326386  6699 tablet_bootstrap.cc:654] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.326395  6700 raft_consensus.cc:740] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.328264  6700 consensus_queue.cc:260] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.339946  6698 tablet_bootstrap.cc:654] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.346982  6700 ts_tablet_manager.cc:1434] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.002s
I20260709 12:16:45.348142  6700 tablet_bootstrap.cc:492] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
W20260709 12:16:45.352486  6610 tablet.cc:2406] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:16:45.359416  6698 tablet_bootstrap.cc:492] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:45.360024  6698 ts_tablet_manager.cc:1403] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.040s	user 0.007s	sys 0.015s
I20260709 12:16:45.363292  6698 raft_consensus.cc:359] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.364034  6698 raft_consensus.cc:385] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.364362  6698 raft_consensus.cc:740] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.366529  6700 tablet_bootstrap.cc:654] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.370306  6698 consensus_queue.cc:260] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.372431  6699 tablet_bootstrap.cc:492] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:45.372845  6698 ts_tablet_manager.cc:1434] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:16:45.373126  6699 ts_tablet_manager.cc:1403] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.078s	user 0.024s	sys 0.013s
I20260709 12:16:45.375260  6700 tablet_bootstrap.cc:492] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:45.375698  6700 ts_tablet_manager.cc:1403] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.028s	user 0.004s	sys 0.007s
I20260709 12:16:45.376184  6699 raft_consensus.cc:359] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.377067  6699 raft_consensus.cc:385] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.377403  6699 raft_consensus.cc:740] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.378196  6699 consensus_queue.cc:260] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.379652  6698 tablet_bootstrap.cc:492] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:45.378423  6700 raft_consensus.cc:359] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.383083  6700 raft_consensus.cc:385] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.385092  6700 raft_consensus.cc:740] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.385938  6700 consensus_queue.cc:260] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.386404  6698 tablet_bootstrap.cc:654] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.384769  6699 ts_tablet_manager.cc:1434] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:16:45.390216  6700 ts_tablet_manager.cc:1434] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.014s	user 0.002s	sys 0.006s
I20260709 12:16:45.391114  6700 tablet_bootstrap.cc:492] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:45.393191  6699 tablet_bootstrap.cc:492] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:45.396631  6700 tablet_bootstrap.cc:654] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.403075  6699 tablet_bootstrap.cc:654] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.413281  6700 tablet_bootstrap.cc:492] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:45.413754  6700 ts_tablet_manager.cc:1403] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.023s	user 0.008s	sys 0.007s
I20260709 12:16:45.416244  6700 raft_consensus.cc:359] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.416934  6700 raft_consensus.cc:385] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.417213  6700 raft_consensus.cc:740] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.417865  6700 consensus_queue.cc:260] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.420203  6698 tablet_bootstrap.cc:492] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:45.420469  6700 ts_tablet_manager.cc:1434] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:16:45.420854  6698 ts_tablet_manager.cc:1403] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.041s	user 0.010s	sys 0.006s
I20260709 12:16:45.421705  6700 tablet_bootstrap.cc:492] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:45.423789  6698 raft_consensus.cc:359] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.424706  6698 raft_consensus.cc:385] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.425060  6698 raft_consensus.cc:740] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.425894  6698 consensus_queue.cc:260] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.428779  6698 ts_tablet_manager.cc:1434] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:16:45.428936  6700 tablet_bootstrap.cc:654] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.434886  6699 tablet_bootstrap.cc:492] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:45.435400  6699 ts_tablet_manager.cc:1403] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.042s	user 0.014s	sys 0.001s
I20260709 12:16:45.438385  6699 raft_consensus.cc:359] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.439486  6699 raft_consensus.cc:385] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.439872  6699 raft_consensus.cc:740] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.440781  6699 consensus_queue.cc:260] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.451493  6699 ts_tablet_manager.cc:1434] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:16:45.457374  6698 tablet_bootstrap.cc:492] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:45.462203  6699 tablet_bootstrap.cc:492] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:45.466748  6698 tablet_bootstrap.cc:654] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.469661  6699 tablet_bootstrap.cc:654] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.481643  6699 tablet_bootstrap.cc:492] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:45.482229  6699 ts_tablet_manager.cc:1403] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.002s
W20260709 12:16:45.487000  6455 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:45.487195  6700 tablet_bootstrap.cc:492] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:45.487900  6700 ts_tablet_manager.cc:1403] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.066s	user 0.027s	sys 0.016s
I20260709 12:16:45.488252  6699 raft_consensus.cc:359] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.489323  6699 raft_consensus.cc:385] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.489670  6699 raft_consensus.cc:740] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.490581  6699 consensus_queue.cc:260] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.491946  6700 raft_consensus.cc:359] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.492990  6700 raft_consensus.cc:385] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.493357  6700 raft_consensus.cc:740] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.494414  6700 consensus_queue.cc:260] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.498719  6698 tablet_bootstrap.cc:492] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:45.500222  6700 ts_tablet_manager.cc:1434] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.001s
I20260709 12:16:45.500685  6756 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:45.503690  6698 ts_tablet_manager.cc:1403] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.047s	user 0.021s	sys 0.019s
I20260709 12:16:45.503728  6756 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.506219  6756 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:45.502790  6699 ts_tablet_manager.cc:1434] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.000s
I20260709 12:16:45.508081  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:45.508800  6584 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:45.508091  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:45.510241  6509 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:45.506942  6698 raft_consensus.cc:359] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.511562  6622 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:45.512035  6698 raft_consensus.cc:385] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.512372  6698 raft_consensus.cc:740] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.513235  6698 consensus_queue.cc:260] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.514895  6756 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:45.515295  6756 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:45.515657  6756 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.520496  6698 ts_tablet_manager.cc:1434] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.003s
I20260709 12:16:45.520993  6699 tablet_bootstrap.cc:492] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:45.521450  6698 tablet_bootstrap.cc:492] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:45.526007  6704 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:45.526507  6704 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.527362  6698 tablet_bootstrap.cc:654] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.528674  6704 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:45.533038  6700 tablet_bootstrap.cc:492] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:45.536409  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:45.537258  6509 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:45.538854  6756 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.540907  6756 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:45.541113  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:45.541831  6584 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:45.541366  6622 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:45.527375  6699 tablet_bootstrap.cc:654] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.539911  6700 tablet_bootstrap.cc:654] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.545845  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:45.546780  6509 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.551316  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:45.551862  6698 tablet_bootstrap.cc:492] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:45.551941  6584 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.552410  6698 ts_tablet_manager.cc:1403] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.031s	user 0.007s	sys 0.010s
I20260709 12:16:45.554941  6698 raft_consensus.cc:359] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.555593  6698 raft_consensus.cc:385] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.555855  6698 raft_consensus.cc:740] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.557337  6698 consensus_queue.cc:260] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.558914  6584 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:45.560076  6622 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:45.560881  6704 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:45.560948  6509 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:45.561313  6704 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:45.561774  6704 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.563203  6698 ts_tablet_manager.cc:1434] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.001s
I20260709 12:16:45.564117  6698 tablet_bootstrap.cc:492] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:45.568817  6704 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.570003  6698 tablet_bootstrap.cc:654] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.572165  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:45.572829  6507 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.578369  6756 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:45.579579  6718 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:45.585395  6718 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:45.586014  6507 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:45.585888  6718 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.586709  6698 tablet_bootstrap.cc:492] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:45.587231  6698 ts_tablet_manager.cc:1403] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.023s	user 0.008s	sys 0.008s
I20260709 12:16:45.587311  6622 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:45.591118  6704 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:45.591050  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:45.591358  6698 raft_consensus.cc:359] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.591704  6718 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:45.593262  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:16:45.592410  6550 leader_election.cc:343] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:45.601245  6718 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:45.603166  6718 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.594079  6700 tablet_bootstrap.cc:492] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:45.604627  6700 ts_tablet_manager.cc:1403] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.072s	user 0.009s	sys 0.004s
I20260709 12:16:45.606045  6718 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:45.592010  6698 raft_consensus.cc:385] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.606541  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:45.606715  6698 raft_consensus.cc:740] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.607215  6656 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:45.607880  6698 consensus_queue.cc:260] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.608547  6550 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:45.609299  6509 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:45.609654  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:45.610391  6507 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:45.595257  6704 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:45.611375  6704 raft_consensus.cc:697] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:16:45.611738  6548 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:45.612605  6728 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:45.612408  6704 consensus_queue.cc:237] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.612962  6728 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:45.613351  6728 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.614946  6700 raft_consensus.cc:359] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.615610  6700 raft_consensus.cc:385] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.615871  6700 raft_consensus.cc:740] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.609603  6718 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:45.616431  6698 ts_tablet_manager.cc:1434] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 12:16:45.616858  6700 consensus_queue.cc:260] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.617964  6718 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:45.618477  6718 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.619491  6698 tablet_bootstrap.cc:492] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:45.619786  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:45.620590  6584 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.621172  6728 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.634497  6700 ts_tablet_manager.cc:1434] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 12:16:45.640902  6584 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:45.650024  6718 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.650799  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:45.651479  6509 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.653072  6700 tablet_bootstrap.cc:492] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:45.654590  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:45.655598  6718 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:45.657326  6728 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:45.659067  6699 tablet_bootstrap.cc:492] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:45.655763  6656 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.661306  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:45.671895  6756 raft_consensus.cc:697] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:16:45.675465  6756 consensus_queue.cc:237] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.676039  6656 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:16:45.682940  6699 ts_tablet_manager.cc:1403] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.162s	user 0.013s	sys 0.009s
W20260709 12:16:45.684485  6550 leader_election.cc:343] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:45.685582  6718 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:45.685951  6550 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:45.687825  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:45.688644  6508 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.689527  6699 raft_consensus.cc:359] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.690244  6699 raft_consensus.cc:385] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.690547  6699 raft_consensus.cc:740] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.692320  6699 consensus_queue.cc:260] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.693037  6763 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:45.693563  6763 raft_consensus.cc:697] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:16:45.694614  6699 ts_tablet_manager.cc:1434] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.003s
I20260709 12:16:45.694446  6763 consensus_queue.cc:237] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.695564  6699 tablet_bootstrap.cc:492] T cb7f6a796adb4b8c89579ba77274985f P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:45.710773  6509 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:16:45.712092  6700 tablet_bootstrap.cc:654] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.711944  6548 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:45.715492  6728 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:45.715969  6728 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.718745  6698 tablet_bootstrap.cc:654] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.721220  6718 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:45.723202  6405 catalog_manager.cc:5697] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:45.730576  6728 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:45.731992  6718 raft_consensus.cc:697] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:16:45.739203  6508 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:16:45.739507  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:45.740375  6509 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:45.741015  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:45.741848  6657 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:45.742200  6548 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:16:45.742942  6700 tablet_bootstrap.cc:492] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:45.745914  6700 ts_tablet_manager.cc:1403] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.093s	user 0.016s	sys 0.007s
I20260709 12:16:45.749935  6777 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:45.750659  6699 tablet_bootstrap.cc:654] T cb7f6a796adb4b8c89579ba77274985f P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.751803  6728 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:45.750681  6777 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.753741  6700 raft_consensus.cc:359] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.754416  6700 raft_consensus.cc:385] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.754681  6700 raft_consensus.cc:740] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.754733  6777 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:45.752355  6728 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.755388  6700 consensus_queue.cc:260] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.748768  6786 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:45.755999  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:45.756578  6786 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:45.757071  6786 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.756930  6509 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:45.757150  6718 consensus_queue.cc:237] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.760691  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:45.761062  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:45.761461  6584 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:45.762086  6700 ts_tablet_manager.cc:1434] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 12:16:45.762034  6657 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:45.763060  6700 tablet_bootstrap.cc:492] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:45.763468  6550 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:45.764655  6728 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:45.764895  6763 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:45.767447  6763 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:45.768672  6763 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.772710  6622 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:45.778716  6786 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.774904  6704 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:45.780710  6704 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:45.781533  6704 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.775574  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:45.783146  6786 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:45.783061  6509 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:45.783069  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:45.784560  6656 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.790508  6704 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.797103  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:45.798053  6509 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.805122  6405 catalog_manager.cc:5697] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:45.811875  6509 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:16:45.813524  6763 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.822577  6763 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:45.825214  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:45.826040  6657 raft_consensus.cc:2393] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 1: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:16:45.825862  6548 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:16:45.825636  6407 catalog_manager.cc:5697] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:45.816008  6656 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:16:45.829541  6700 tablet_bootstrap.cc:654] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.830214  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:45.831388  6508 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.833666  6704 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:45.834869  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:45.835757  6584 raft_consensus.cc:2393] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 1: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:16:45.839975  6508 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:45.831336  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:45.843788  6509 raft_consensus.cc:2393] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 1: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:16:45.840739  6763 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:45.844408  6763 raft_consensus.cc:697] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:16:45.845197  6548 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:45.846207  6718 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:45.847128  6622 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:16:45.848423  6699 tablet_bootstrap.cc:492] T cb7f6a796adb4b8c89579ba77274985f P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:45.848544  6704 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:45.848996  6699 ts_tablet_manager.cc:1403] T cb7f6a796adb4b8c89579ba77274985f P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.154s	user 0.021s	sys 0.020s
I20260709 12:16:45.849124  6704 raft_consensus.cc:697] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:16:45.850060  6704 consensus_queue.cc:237] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.851850  6699 raft_consensus.cc:359] T cb7f6a796adb4b8c89579ba77274985f P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.852690  6699 raft_consensus.cc:385] T cb7f6a796adb4b8c89579ba77274985f P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.852968  6699 raft_consensus.cc:740] T cb7f6a796adb4b8c89579ba77274985f P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.853710  6698 tablet_bootstrap.cc:492] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:45.854537  6698 ts_tablet_manager.cc:1403] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.236s	user 0.025s	sys 0.020s
I20260709 12:16:45.854022  6699 consensus_queue.cc:260] T cb7f6a796adb4b8c89579ba77274985f P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.845299  6763 consensus_queue.cc:237] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.858006  6699 ts_tablet_manager.cc:1434] T cb7f6a796adb4b8c89579ba77274985f P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:16:45.858388  6700 tablet_bootstrap.cc:492] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:45.858877  6700 ts_tablet_manager.cc:1403] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.096s	user 0.015s	sys 0.002s
I20260709 12:16:45.859361  6699 tablet_bootstrap.cc:492] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:45.868674  6698 raft_consensus.cc:359] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.868081  6405 catalog_manager.cc:5697] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:45.869670  6698 raft_consensus.cc:385] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.870007  6698 raft_consensus.cc:740] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.871238  6699 tablet_bootstrap.cc:654] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.870802  6698 consensus_queue.cc:260] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.873070  6698 ts_tablet_manager.cc:1434] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:16:45.875963  6700 raft_consensus.cc:359] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.877035  6700 raft_consensus.cc:385] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.877290  6700 raft_consensus.cc:740] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.878098  6700 consensus_queue.cc:260] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.880420  6700 ts_tablet_manager.cc:1434] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.001s
I20260709 12:16:45.881443  6700 tablet_bootstrap.cc:492] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:45.881771  6698 tablet_bootstrap.cc:492] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:45.887377  6699 tablet_bootstrap.cc:492] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:45.887933  6699 ts_tablet_manager.cc:1403] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.029s	user 0.010s	sys 0.010s
I20260709 12:16:45.887255  6406 catalog_manager.cc:5697] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:45.892043  6700 tablet_bootstrap.cc:654] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.906579  6405 catalog_manager.cc:5697] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:45.907641  6699 raft_consensus.cc:359] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.908710  6699 raft_consensus.cc:385] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.909056  6699 raft_consensus.cc:740] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.909674  6698 tablet_bootstrap.cc:654] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.909812  6699 consensus_queue.cc:260] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.912249  6699 ts_tablet_manager.cc:1434] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.024s	user 0.004s	sys 0.000s
I20260709 12:16:45.917112  6700 tablet_bootstrap.cc:492] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:45.917762  6698 tablet_bootstrap.cc:492] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:45.917824  6700 ts_tablet_manager.cc:1403] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.037s	user 0.015s	sys 0.003s
I20260709 12:16:45.918341  6698 ts_tablet_manager.cc:1403] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.037s	user 0.010s	sys 0.009s
I20260709 12:16:45.918514  6699 tablet_bootstrap.cc:492] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:45.921663  6700 raft_consensus.cc:359] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.922140  6698 raft_consensus.cc:359] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.922717  6700 raft_consensus.cc:385] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.922959  6698 raft_consensus.cc:385] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.923054  6700 raft_consensus.cc:740] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.923364  6698 raft_consensus.cc:740] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.923818  6700 consensus_queue.cc:260] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.924134  6698 consensus_queue.cc:260] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.925938  6700 ts_tablet_manager.cc:1434] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:16:45.926411  6698 ts_tablet_manager.cc:1434] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:16:45.927155  6700 tablet_bootstrap.cc:492] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:45.927589  6698 tablet_bootstrap.cc:492] T eebce9eca2a5496bb4a3ab473a243166 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:45.933070  6700 tablet_bootstrap.cc:654] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.935936  6698 tablet_bootstrap.cc:654] T eebce9eca2a5496bb4a3ab473a243166 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.941087  6704 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:45.941781  6704 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.943706  6699 tablet_bootstrap.cc:654] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.944491  6704 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:45.945631  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:45.946448  6508 raft_consensus.cc:2393] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 1: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:16:45.946488  6698 tablet_bootstrap.cc:492] T eebce9eca2a5496bb4a3ab473a243166 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:45.947157  6698 ts_tablet_manager.cc:1403] T eebce9eca2a5496bb4a3ab473a243166 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.002s
I20260709 12:16:45.948103  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:45.949498  6622 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:45.950172  6704 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.950100  6698 raft_consensus.cc:359] T eebce9eca2a5496bb4a3ab473a243166 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.950700  6700 tablet_bootstrap.cc:492] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:45.950970  6698 raft_consensus.cc:385] T eebce9eca2a5496bb4a3ab473a243166 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.951236  6700 ts_tablet_manager.cc:1403] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.004s
I20260709 12:16:45.951330  6698 raft_consensus.cc:740] T eebce9eca2a5496bb4a3ab473a243166 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.952612  6698 consensus_queue.cc:260] T eebce9eca2a5496bb4a3ab473a243166 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.954617  6700 raft_consensus.cc:359] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.955271  6700 raft_consensus.cc:385] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.955554  6700 raft_consensus.cc:740] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.956275  6700 consensus_queue.cc:260] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.958560  6704 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:45.959792  6698 ts_tablet_manager.cc:1434] T eebce9eca2a5496bb4a3ab473a243166 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:16:45.960981  6698 tablet_bootstrap.cc:492] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:45.964190  6700 ts_tablet_manager.cc:1434] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:16:45.965276  6700 tablet_bootstrap.cc:492] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:45.967456  6698 tablet_bootstrap.cc:654] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.970396  6700 tablet_bootstrap.cc:654] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.978071  6699 tablet_bootstrap.cc:492] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:45.978538  6699 ts_tablet_manager.cc:1403] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.060s	user 0.012s	sys 0.008s
I20260709 12:16:45.982314  6700 tablet_bootstrap.cc:492] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:45.983448  6700 ts_tablet_manager.cc:1403] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.002s
I20260709 12:16:45.985273  6698 tablet_bootstrap.cc:492] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:45.985874  6698 ts_tablet_manager.cc:1403] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.025s	user 0.011s	sys 0.005s
I20260709 12:16:45.987058  6700 raft_consensus.cc:359] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.982326  6699 raft_consensus.cc:359] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.989035  6700 raft_consensus.cc:385] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.988780  6698 raft_consensus.cc:359] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.989210  6699 raft_consensus.cc:385] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.989426  6700 raft_consensus.cc:740] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.989635  6698 raft_consensus.cc:385] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.989894  6699 raft_consensus.cc:740] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.990001  6698 raft_consensus.cc:740] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.990314  6700 consensus_queue.cc:260] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.990996  6698 consensus_queue.cc:260] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.991325  6699 consensus_queue.cc:260] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:45.992635  6700 ts_tablet_manager.cc:1434] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.006s
I20260709 12:16:45.992830  6698 ts_tablet_manager.cc:1434] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 12:16:45.993496  6700 tablet_bootstrap.cc:492] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:45.994102  6698 tablet_bootstrap.cc:492] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:46.000485  6698 tablet_bootstrap.cc:654] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.005076  6700 tablet_bootstrap.cc:654] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.007431  6699 ts_tablet_manager.cc:1434] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.003s
I20260709 12:16:46.016165  6763 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:46.016986  6763 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.019733  6763 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:46.021529  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:46.022097  6699 tablet_bootstrap.cc:492] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:46.022656  6698 tablet_bootstrap.cc:492] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:46.022409  6704 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:46.022419  6656 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:46.023276  6698 ts_tablet_manager.cc:1403] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.029s	user 0.009s	sys 0.016s
I20260709 12:16:46.025264  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:46.024631  6763 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:46.027021  6763 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.028367  6550 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:46.029820  6718 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:46.030342  6718 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:46.031325  6718 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.031636  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:46.032876  6763 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:16:46.033320  6548 leader_election.cc:343] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:46.034132  6786 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:46.037283  6786 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.039970  6786 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:46.041347  6753 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:46.041992  6548 leader_election.cc:343] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:46.041792  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:46.042351  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:46.041792  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:46.041860  6753 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.044480  6508 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:46.044855  6698 raft_consensus.cc:359] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:16:46.045390  6550 leader_election.cc:343] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:46.045809  6753 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:16:46.046234  6550 leader_election.cc:343] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:46.048552  6550 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:46.049280  6699 tablet_bootstrap.cc:654] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.045531  6698 raft_consensus.cc:385] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.046177  6548 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:46.050333  6698 raft_consensus.cc:740] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.047189  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:46.047376  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:46.049813  6786 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:16:46.052982  6473 leader_election.cc:343] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:46.052817  6698 consensus_queue.cc:260] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:16:46.053966  6475 leader_election.cc:343] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:46.054536  6475 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:46.052016  6786 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:46.056882  6753 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:46.058301  6698 ts_tablet_manager.cc:1434] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.003s
I20260709 12:16:46.056944  6763 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:46.055696  6718 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.059844  6698 tablet_bootstrap.cc:492] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:46.056454  6786 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:46.062743  6786 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.065513  6704 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:46.068145  6753 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:46.059127  6763 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.068825  6753 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.070391  6763 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:46.071252  6753 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:46.071462  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:16:46.072598  6550 leader_election.cc:343] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:46.073139  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:46.075593  6584 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 0.
I20260709 12:16:46.077291  6473 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:16:46.078287  6753 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:46.080129  6753 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:46.080499  6753 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.080629  6700 tablet_bootstrap.cc:492] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:46.081100  6700 ts_tablet_manager.cc:1403] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.088s	user 0.013s	sys 0.006s
I20260709 12:16:46.085700  6700 raft_consensus.cc:359] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.086192  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:46.086534  6700 raft_consensus.cc:385] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.086947  6700 raft_consensus.cc:740] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.087812  6700 consensus_queue.cc:260] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:16:46.090171  6475 leader_election.cc:343] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:46.090400  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:46.090850  6700 ts_tablet_manager.cc:1434] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.002s
I20260709 12:16:46.092079  6700 tablet_bootstrap.cc:492] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:46.096127  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:46.098115  6656 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.099484  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:46.101488  6506 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.097136  6718 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:46.108453  6753 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.111847  6698 tablet_bootstrap.cc:654] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.113772  6507 raft_consensus.cc:2393] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 1: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:16:46.113729  6704 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:46.115339  6786 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.118858  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:46.120198  6728 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:46.120303  6548 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:16:46.120684  6550 leader_election.cc:343] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:46.120786  6728 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.121903  6656 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:16:46.122958  6699 tablet_bootstrap.cc:492] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:46.123078  6728 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:46.123469  6699 ts_tablet_manager.cc:1403] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.102s	user 0.012s	sys 0.014s
I20260709 12:16:46.126051  6706 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.127581  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:46.128297  6657 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:46.123950  6550 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:46.134358  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:46.125116  6786 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:46.138690  6505 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.141173  6506 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:16:46.145191  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:46.147465  6550 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:46.148895  6782 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:46.149242  6728 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:46.149389  6782 raft_consensus.cc:697] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:16:46.149641  6728 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:46.149986  6728 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.150285  6782 consensus_queue.cc:237] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.151289  6756 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:46.158645  6706 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:46.171265  6700 tablet_bootstrap.cc:654] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.170872  6699 raft_consensus.cc:359] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.172129  6699 raft_consensus.cc:385] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.172513  6699 raft_consensus.cc:740] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.173624  6699 consensus_queue.cc:260] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.175490  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:46.176167  6509 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:46.182252  6699 ts_tablet_manager.cc:1434] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.058s	user 0.006s	sys 0.002s
I20260709 12:16:46.183743  6784 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:16:46.172214  6505 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:16:46.194641  6706 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:46.197002  6699 tablet_bootstrap.cc:492] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:46.198342  6728 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.209848  6548 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:46.211103  6706 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:46.211639  6706 raft_consensus.cc:697] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:16:46.200743  6753 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:46.212743  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:46.212539  6706 consensus_queue.cc:237] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:16:46.223649  6475 leader_election.cc:343] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:46.238050  6718 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:46.239981  6718 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:46.240568  6718 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.259361  6699 tablet_bootstrap.cc:654] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.264357  6704 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:46.264964  6704 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.267160  6704 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:46.250887  6728 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:46.269986  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:46.289472  6509 raft_consensus.cc:2393] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 1: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:16:46.291224  6583 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:16:46.296921  6718 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:46.311002  6700 tablet_bootstrap.cc:492] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:46.298303  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:46.311654  6700 ts_tablet_manager.cc:1403] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.220s	user 0.017s	sys 0.027s
I20260709 12:16:46.312717  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:46.314675  6473 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:46.314857  6700 raft_consensus.cc:359] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.316715  6756 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.316061  6700 raft_consensus.cc:385] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.318287  6700 raft_consensus.cc:740] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.324180  6700 consensus_queue.cc:260] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.318361  6753 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:46.311046  6718 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 12:16:46.326053  6753 raft_consensus.cc:697] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:16:46.327106  6700 ts_tablet_manager.cc:1434] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.015s	user 0.010s	sys 0.000s
I20260709 12:16:46.326987  6753 consensus_queue.cc:237] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.328253  6700 tablet_bootstrap.cc:492] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
W20260709 12:16:46.317448  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 75688.7us
I20260709 12:16:46.298398  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:46.328801  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:46.337169  6509 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.333870  6406 catalog_manager.cc:5697] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:46.342023  6699 tablet_bootstrap.cc:492] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:46.369638  6699 ts_tablet_manager.cc:1403] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.173s	user 0.013s	sys 0.007s
I20260709 12:16:46.358073  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:46.342648  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:46.379395  6698 tablet_bootstrap.cc:492] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:46.379914  6698 ts_tablet_manager.cc:1403] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.320s	user 0.022s	sys 0.023s
I20260709 12:16:46.381466  6657 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.389010  6698 raft_consensus.cc:359] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:16:46.353543  6548 leader_election.cc:343] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:46.391319  6698 raft_consensus.cc:385] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.392068  6698 raft_consensus.cc:740] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.394201  6698 consensus_queue.cc:260] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.396755  6698 ts_tablet_manager.cc:1434] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.016s	user 0.010s	sys 0.000s
I20260709 12:16:46.397034  6756 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:16:46.399384  6550 leader_election.cc:343] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:46.399881  6550 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:46.402611  6700 tablet_bootstrap.cc:654] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.404812  6756 consensus_queue.cc:1048] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:46.407334  6698 tablet_bootstrap.cc:492] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:46.408521  6821 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:46.411273  6728 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:46.416458  6509 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:16:46.418545  6407 catalog_manager.cc:5697] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 0 to 1, leader changed from <none> to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:46.423171  6548 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:16:46.424712  6728 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:46.425433  6728 raft_consensus.cc:697] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:16:46.426391  6657 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:16:46.426472  6699 raft_consensus.cc:359] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.427363  6699 raft_consensus.cc:385] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.427661  6699 raft_consensus.cc:740] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.428676  6699 consensus_queue.cc:260] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.431169  6699 ts_tablet_manager.cc:1434] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.060s	user 0.005s	sys 0.001s
I20260709 12:16:46.432273  6699 tablet_bootstrap.cc:492] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:46.426391  6728 consensus_queue.cc:237] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.442373  6706 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:46.443015  6756 consensus_queue.cc:1048] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:46.447057  6698 tablet_bootstrap.cc:654] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.452430  6405 catalog_manager.cc:5697] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:46.468504  6753 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:46.469529  6753 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.471266  6698 tablet_bootstrap.cc:492] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:46.534049  6698 ts_tablet_manager.cc:1403] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.127s	user 0.030s	sys 0.050s
I20260709 12:16:46.535820  6700 tablet_bootstrap.cc:492] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:46.535737  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:46.536564  6584 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 0.
I20260709 12:16:46.537901  6473 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:16:46.538882  6815 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:46.539244  6815 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:46.539592  6815 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.542963  6698 raft_consensus.cc:359] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.543732  6698 raft_consensus.cc:385] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.544099  6698 raft_consensus.cc:740] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.545068  6699 tablet_bootstrap.cc:654] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.544775  6698 consensus_queue.cc:260] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:16:46.546510  6828 log.cc:927] Time spent T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:16:46.547495  6698 ts_tablet_manager.cc:1434] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.003s
I20260709 12:16:46.548391  6698 tablet_bootstrap.cc:492] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:46.554369  6698 tablet_bootstrap.cc:654] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.556272  6815 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.561293  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:46.561975  6583 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.561975  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:16:46.563977  6475 leader_election.cc:343] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:46.565153  6815 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:46.567600  6753 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:46.567852  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:16:46.569532  6475 leader_election.cc:343] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:46.574373  6405 catalog_manager.cc:5697] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:46.584143  6583 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:16:46.579425  6698 tablet_bootstrap.cc:492] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:46.586042  6473 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:46.586889  6700 ts_tablet_manager.cc:1403] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.259s	user 0.047s	sys 0.005s
I20260709 12:16:46.588303  6753 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:46.588944  6753 raft_consensus.cc:697] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:16:46.589838  6753 consensus_queue.cc:237] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.590131  6700 raft_consensus.cc:359] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.590870  6700 raft_consensus.cc:385] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.591176  6700 raft_consensus.cc:740] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.591894  6700 consensus_queue.cc:260] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.586056  6698 ts_tablet_manager.cc:1403] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.038s	user 0.012s	sys 0.004s
W20260709 12:16:46.579663  6455 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:46.594184  6728 consensus_queue.cc:1048] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:46.595263  6700 ts_tablet_manager.cc:1434] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:16:46.596299  6700 tablet_bootstrap.cc:492] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:46.597204  6698 raft_consensus.cc:359] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.598163  6698 raft_consensus.cc:385] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.598527  6728 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:46.598567  6698 raft_consensus.cc:740] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.599329  6728 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.600078  6698 consensus_queue.cc:260] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.603003  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:46.603518  6700 tablet_bootstrap.cc:654] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.603916  6506 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:46.604156  6728 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:46.604425  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:46.605407  6548 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:16:46.606128  6550 leader_election.cc:343] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:46.606637  6728 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:46.607059  6728 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:46.607671  6728 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.608898  6698 ts_tablet_manager.cc:1434] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.016s	user 0.010s	sys 0.000s
I20260709 12:16:46.610023  6698 tablet_bootstrap.cc:492] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:46.611230  6699 tablet_bootstrap.cc:492] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:46.611727  6699 ts_tablet_manager.cc:1403] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.180s	user 0.027s	sys 0.015s
I20260709 12:16:46.616554  6698 tablet_bootstrap.cc:654] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.616894  6728 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.620136  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:46.621038  6506 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.636869  6784 consensus_queue.cc:1048] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:46.638379  6506 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:16:46.641587  6756 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:16:46.642175  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:46.642174  6756 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.646939  6756 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:46.649556  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:46.650959  6507 raft_consensus.cc:2393] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 2: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:16:46.653239  6550 leader_election.cc:343] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:46.658666  6699 raft_consensus.cc:359] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.659482  6699 raft_consensus.cc:385] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.659801  6699 raft_consensus.cc:740] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.660744  6821 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:46.661482  6821 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.664788  6699 consensus_queue.cc:260] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.666143  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:46.667016  6583 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:16:46.668638  6753 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:46.669307  6753 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.673170  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:46.674763  6699 ts_tablet_manager.cc:1434] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.063s	user 0.007s	sys 0.000s
I20260709 12:16:46.679392  6699 tablet_bootstrap.cc:492] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
W20260709 12:16:46.680647  6473 leader_election.cc:343] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:46.678166  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:46.674860  6700 tablet_bootstrap.cc:492] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:46.682996  6700 ts_tablet_manager.cc:1403] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.087s	user 0.009s	sys 0.004s
I20260709 12:16:46.685534  6756 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:46.685966  6756 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:16:46.687593  6756 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:46.693228  6700 raft_consensus.cc:359] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.695284  6700 raft_consensus.cc:385] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.695634  6700 raft_consensus.cc:740] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.696359  6700 consensus_queue.cc:260] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.698709  6406 catalog_manager.cc:5697] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 0 to 1, leader changed from <none> to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:46.704778  6728 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:46.707499  6756 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.724257  6548 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:46.742851  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:46.743798  6506 raft_consensus.cc:2393] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 2: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:16:46.748649  6756 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:46.752489  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:16:46.754427  6475 leader_election.cc:343] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:46.755095  6475 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:46.756163  6815 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:46.757588  6699 tablet_bootstrap.cc:654] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.758031  6821 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:46.761842  6753 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:46.765213  6700 ts_tablet_manager.cc:1434] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.082s	user 0.006s	sys 0.006s
I20260709 12:16:46.766482  6700 tablet_bootstrap.cc:492] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:46.766950  6698 tablet_bootstrap.cc:492] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:46.767906  6698 ts_tablet_manager.cc:1403] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.158s	user 0.062s	sys 0.081s
W20260709 12:16:46.768853  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 44766.8us
I20260709 12:16:46.770596  6698 raft_consensus.cc:359] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:16:46.771296  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 61844.5us
I20260709 12:16:46.772265  6728 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:46.771519  6699 tablet_bootstrap.cc:492] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:46.774552  6699 ts_tablet_manager.cc:1403] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.095s	user 0.010s	sys 0.007s
I20260709 12:16:46.775406  6700 tablet_bootstrap.cc:654] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.776525  6698 raft_consensus.cc:385] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.777159  6699 raft_consensus.cc:359] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.777729  6698 raft_consensus.cc:740] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.777858  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:46.776667  6728 raft_consensus.cc:697] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:16:46.779006  6728 consensus_queue.cc:237] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.779083  6698 consensus_queue.cc:260] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.782356  6698 ts_tablet_manager.cc:1434] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:16:46.784480  6782 consensus_queue.cc:1048] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:46.786381  6583 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:16:46.786969  6700 tablet_bootstrap.cc:492] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:46.788051  6700 ts_tablet_manager.cc:1403] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.004s
I20260709 12:16:46.787946  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:46.788971  6756 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:46.789448  6756 raft_consensus.cc:697] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:16:46.790366  6756 consensus_queue.cc:237] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.789348  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:46.793706  6656 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:46.794795  6550 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:46.795621  6821 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:46.795996  6821 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:46.796324  6821 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.798046  6698 tablet_bootstrap.cc:492] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:46.778080  6699 raft_consensus.cc:385] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.801127  6699 raft_consensus.cc:740] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.804670  6699 consensus_queue.cc:260] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.807199  6699 ts_tablet_manager.cc:1434] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.001s
I20260709 12:16:46.809048  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:46.809772  6506 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:46.811036  6655 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.818657  6706 consensus_queue.cc:1048] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:46.825744  6699 tablet_bootstrap.cc:492] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:46.826689  6821 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.829221  6821 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:46.842324  6407 catalog_manager.cc:5697] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:46.851276  6700 raft_consensus.cc:359] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.852234  6700 raft_consensus.cc:385] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.852643  6700 raft_consensus.cc:740] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.853585  6700 consensus_queue.cc:260] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.856362  6815 consensus_queue.cc:1048] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:46.857172  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:46.857772  6655 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.859575  6698 tablet_bootstrap.cc:654] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.859750  6406 catalog_manager.cc:5697] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 1 to 2, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 2 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:46.863844  6699 tablet_bootstrap.cc:654] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.871481  6655 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:16:46.873904  6700 ts_tablet_manager.cc:1434] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.085s	user 0.008s	sys 0.000s
I20260709 12:16:46.874130  6550 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:46.875000  6700 tablet_bootstrap.cc:492] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:46.875072  6706 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:46.876305  6784 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:46.877933  6815 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:46.877151  6784 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.878676  6815 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.883291  6784 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:46.885475  6815 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:46.886796  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:46.876114  6706 raft_consensus.cc:697] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:16:46.888921  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:46.893149  6706 consensus_queue.cc:237] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.898357  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:46.898988  6506 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.900846  6583 raft_consensus.cc:2410] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 12:16:46.903232  6473 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:46.904034  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:46.904734  6507 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:46.905718  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:46.905776  6807 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:16:46.907347  6548 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:16:46.910048  6728 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:46.910475  6728 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:46.910555  6753 consensus_queue.cc:1048] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:46.910921  6728 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:46.911914  6550 leader_election.cc:343] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:46.914189  6700 tablet_bootstrap.cc:654] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.916383  6698 tablet_bootstrap.cc:492] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:46.916875  6698 ts_tablet_manager.cc:1403] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.119s	user 0.010s	sys 0.007s
I20260709 12:16:46.919612  6698 raft_consensus.cc:359] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.922381  6506 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:16:46.920320  6698 raft_consensus.cc:385] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.925784  6698 raft_consensus.cc:740] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.927007  6700 tablet_bootstrap.cc:492] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:46.927549  6700 ts_tablet_manager.cc:1403] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.053s	user 0.016s	sys 0.004s
I20260709 12:16:46.929787  6699 tablet_bootstrap.cc:492] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:46.929380  6698 consensus_queue.cc:260] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.930279  6699 ts_tablet_manager.cc:1403] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.118s	user 0.019s	sys 0.000s
I20260709 12:16:46.933984  6699 raft_consensus.cc:359] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.934593  6698 ts_tablet_manager.cc:1434] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 12:16:46.934813  6699 raft_consensus.cc:385] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.935276  6699 raft_consensus.cc:740] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.936237  6699 consensus_queue.cc:260] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.941570  6699 ts_tablet_manager.cc:1434] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:16:46.942761  6699 tablet_bootstrap.cc:492] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:46.939363  6700 raft_consensus.cc:359] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.944613  6700 raft_consensus.cc:385] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.945001  6700 raft_consensus.cc:740] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.946363  6700 consensus_queue.cc:260] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.948681  6699 tablet_bootstrap.cc:654] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.949018  6700 ts_tablet_manager.cc:1434] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.002s
I20260709 12:16:46.950067  6700 tablet_bootstrap.cc:492] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:46.946800  6728 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.954939  6698 tablet_bootstrap.cc:492] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:46.960428  6699 tablet_bootstrap.cc:492] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:46.960952  6699 ts_tablet_manager.cc:1403] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.004s
I20260709 12:16:46.960959  6700 tablet_bootstrap.cc:654] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.964476  6699 raft_consensus.cc:359] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.965574  6699 raft_consensus.cc:385] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.965966  6699 raft_consensus.cc:740] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.967092  6699 consensus_queue.cc:260] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:46.969334  6728 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:46.969818  6699 ts_tablet_manager.cc:1434] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:16:46.970678  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:16:46.972026  6550 leader_election.cc:343] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:46.972448  6699 tablet_bootstrap.cc:492] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:46.978201  6658 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.980239  6699 tablet_bootstrap.cc:654] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.981107  6407 catalog_manager.cc:5697] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:46.984323  6698 tablet_bootstrap.cc:654] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.990002  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:46.990667  6509 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.996042  6509 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:16:46.999749  6753 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:47.005048  6548 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:47.006163  6728 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:47.006686  6728 raft_consensus.cc:697] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:16:47.007714  6728 consensus_queue.cc:237] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.011571  6700 tablet_bootstrap.cc:492] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:47.012137  6700 ts_tablet_manager.cc:1403] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.062s	user 0.017s	sys 0.002s
I20260709 12:16:47.016845  6700 raft_consensus.cc:359] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.017521  6700 raft_consensus.cc:385] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.017786  6700 raft_consensus.cc:740] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.018555  6700 consensus_queue.cc:260] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.021523  6699 tablet_bootstrap.cc:492] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:47.022140  6699 ts_tablet_manager.cc:1403] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.050s	user 0.011s	sys 0.005s
I20260709 12:16:47.027217  6815 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:47.027701  6821 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:47.027180  6700 ts_tablet_manager.cc:1434] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 12:16:47.033849  6700 tablet_bootstrap.cc:492] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:47.042883  6706 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:47.048203  6699 raft_consensus.cc:359] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.049724  6699 raft_consensus.cc:385] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.050493  6699 raft_consensus.cc:740] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.051891  6700 tablet_bootstrap.cc:654] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.051709  6699 consensus_queue.cc:260] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.057938  6699 ts_tablet_manager.cc:1434] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.036s	user 0.005s	sys 0.004s
I20260709 12:16:47.056689  6704 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:47.062443  6706 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:47.063131  6706 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.066090  6706 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:47.062180  6699 tablet_bootstrap.cc:492] T e416f9423f964ba5a29e536e8160c627 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:47.067850  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:47.059880  6704 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.070089  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:47.070818  6507 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:47.070998  6704 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:47.071866  6656 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:47.073963  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:47.074206  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:47.074636  6507 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:47.075002  6584 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:47.076122  6622 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:47.076438  6550 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:47.077438  6784 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:47.077823  6784 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:47.078199  6784 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.076489  6407 catalog_manager.cc:5697] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:47.085907  6698 tablet_bootstrap.cc:492] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:47.086541  6698 ts_tablet_manager.cc:1403] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.132s	user 0.011s	sys 0.014s
I20260709 12:16:47.079212  6704 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:47.092600  6698 raft_consensus.cc:359] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.093287  6698 raft_consensus.cc:385] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.093547  6698 raft_consensus.cc:740] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.094157  6698 consensus_queue.cc:260] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.089663  6704 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:47.096860  6704 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.099488  6699 tablet_bootstrap.cc:654] T e416f9423f964ba5a29e536e8160c627 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.102051  6698 ts_tablet_manager.cc:1434] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.002s
I20260709 12:16:47.102882  6698 tablet_bootstrap.cc:492] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:47.119604  6784 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.120736  6704 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.133515  6704 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:47.127420  6718 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:47.134300  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:47.134227  6718 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.134961  6506 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.136418  6718 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:47.140156  6700 tablet_bootstrap.cc:492] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:47.140053  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:47.140659  6700 ts_tablet_manager.cc:1403] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.107s	user 0.012s	sys 0.001s
W20260709 12:16:47.141667  6550 leader_election.cc:343] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:47.142081  6784 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:47.131865  6698 tablet_bootstrap.cc:654] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.146672  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:47.147617  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:47.148548  6658 raft_consensus.cc:2393] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 1: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:16:47.149673  6506 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:16:47.151011  6548 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:47.151331  6699 tablet_bootstrap.cc:492] T e416f9423f964ba5a29e536e8160c627 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:47.154561  6699 ts_tablet_manager.cc:1403] T e416f9423f964ba5a29e536e8160c627 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.093s	user 0.019s	sys 0.009s
I20260709 12:16:47.154680  6718 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:47.155225  6718 raft_consensus.cc:697] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:16:47.144865  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:47.144847  6700 raft_consensus.cc:359] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.156131  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:47.157058  6700 raft_consensus.cc:385] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.157582  6700 raft_consensus.cc:740] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.156129  6718 consensus_queue.cc:237] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.157994  6699 raft_consensus.cc:359] T e416f9423f964ba5a29e536e8160c627 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.158771  6699 raft_consensus.cc:385] T e416f9423f964ba5a29e536e8160c627 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.158357  6700 consensus_queue.cc:260] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.159067  6699 raft_consensus.cc:740] T e416f9423f964ba5a29e536e8160c627 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.159829  6699 consensus_queue.cc:260] T e416f9423f964ba5a29e536e8160c627 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.160728  6700 ts_tablet_manager.cc:1434] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.001s
I20260709 12:16:47.160989  6622 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:47.161556  6700 tablet_bootstrap.cc:492] T c6b5198a4b6741cdb13e49cf4e90173c P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:47.162655  6779 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:16:47.164745  6548 leader_election.cc:343] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:47.165287  6548 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:47.165892  6699 ts_tablet_manager.cc:1434] T e416f9423f964ba5a29e536e8160c627 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:16:47.166941  6699 tablet_bootstrap.cc:492] T 14cf1c3a63624d94aa6c9c356e6108e2 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:47.169106  6784 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:47.175561  6699 tablet_bootstrap.cc:654] T 14cf1c3a63624d94aa6c9c356e6108e2 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.179724  6700 tablet_bootstrap.cc:654] T c6b5198a4b6741cdb13e49cf4e90173c P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.186614  6698 tablet_bootstrap.cc:492] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:47.187098  6698 ts_tablet_manager.cc:1403] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.084s	user 0.007s	sys 0.006s
I20260709 12:16:47.195240  6407 catalog_manager.cc:5697] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:47.191565  6698 raft_consensus.cc:359] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.197219  6698 raft_consensus.cc:385] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.197592  6698 raft_consensus.cc:740] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.198442  6698 consensus_queue.cc:260] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.200510  6583 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:47.200835  6699 tablet_bootstrap.cc:492] T 14cf1c3a63624d94aa6c9c356e6108e2 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:47.200618  6698 ts_tablet_manager.cc:1434] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.001s
I20260709 12:16:47.201406  6699 ts_tablet_manager.cc:1403] T 14cf1c3a63624d94aa6c9c356e6108e2 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.035s	user 0.016s	sys 0.007s
I20260709 12:16:47.203089  6698 tablet_bootstrap.cc:492] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:47.204807  6699 raft_consensus.cc:359] T 14cf1c3a63624d94aa6c9c356e6108e2 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.205569  6699 raft_consensus.cc:385] T 14cf1c3a63624d94aa6c9c356e6108e2 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.205874  6699 raft_consensus.cc:740] T 14cf1c3a63624d94aa6c9c356e6108e2 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.206023  6704 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:47.206806  6699 consensus_queue.cc:260] T 14cf1c3a63624d94aa6c9c356e6108e2 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.209297  6700 tablet_bootstrap.cc:492] T c6b5198a4b6741cdb13e49cf4e90173c P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:47.209640  6699 ts_tablet_manager.cc:1434] T 14cf1c3a63624d94aa6c9c356e6108e2 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:16:47.209785  6700 ts_tablet_manager.cc:1403] T c6b5198a4b6741cdb13e49cf4e90173c P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.048s	user 0.012s	sys 0.005s
I20260709 12:16:47.210994  6698 tablet_bootstrap.cc:654] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.212627  6700 raft_consensus.cc:359] T c6b5198a4b6741cdb13e49cf4e90173c P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.213455  6700 raft_consensus.cc:385] T c6b5198a4b6741cdb13e49cf4e90173c P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.213759  6700 raft_consensus.cc:740] T c6b5198a4b6741cdb13e49cf4e90173c P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.214658  6700 consensus_queue.cc:260] T c6b5198a4b6741cdb13e49cf4e90173c P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.216872  6700 ts_tablet_manager.cc:1434] T c6b5198a4b6741cdb13e49cf4e90173c P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 12:16:47.217767  6700 tablet_bootstrap.cc:492] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:47.232875  6700 tablet_bootstrap.cc:654] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.233578  6699 tablet_bootstrap.cc:492] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:47.247874  6699 tablet_bootstrap.cc:654] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.250883  6784 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:47.251228  6700 tablet_bootstrap.cc:492] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:47.251837  6700 ts_tablet_manager.cc:1403] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.034s	user 0.015s	sys 0.003s
I20260709 12:16:47.251466  6784 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.253186  6509 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:47.255089  6756 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:47.255198  6700 raft_consensus.cc:359] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.255951  6700 raft_consensus.cc:385] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.256258  6700 raft_consensus.cc:740] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.256757  6698 tablet_bootstrap.cc:492] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:47.257078  6700 consensus_queue.cc:260] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.257503  6698 ts_tablet_manager.cc:1403] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.055s	user 0.018s	sys 0.024s
I20260709 12:16:47.259703  6700 ts_tablet_manager.cc:1434] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:16:47.261816  6784 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:47.262387  6698 raft_consensus.cc:359] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.263516  6698 raft_consensus.cc:385] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.264535  6698 raft_consensus.cc:740] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.265823  6700 tablet_bootstrap.cc:492] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:47.266896  6698 consensus_queue.cc:260] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.267665  6699 tablet_bootstrap.cc:492] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:47.268163  6699 ts_tablet_manager.cc:1403] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.035s	user 0.017s	sys 0.011s
I20260709 12:16:47.270490  6699 raft_consensus.cc:359] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.271054  6699 raft_consensus.cc:385] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.271394  6699 raft_consensus.cc:740] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.272243  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:47.273020  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:47.274422  6700 tablet_bootstrap.cc:654] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:47.274737  6550 leader_election.cc:343] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:47.280237  6698 ts_tablet_manager.cc:1434] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 12:16:47.280174  6815 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:47.281322  6815 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.282099  6698 tablet_bootstrap.cc:492] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:47.283505  6699 consensus_queue.cc:260] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.285399  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:16:47.281747  6548 leader_election.cc:343] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:47.286243  6700 tablet_bootstrap.cc:492] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:47.286202  6583 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 0.
I20260709 12:16:47.286890  6700 ts_tablet_manager.cc:1403] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.001s
I20260709 12:16:47.288316  6699 ts_tablet_manager.cc:1434] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 12:16:47.288760  6473 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:16:47.289414  6699 tablet_bootstrap.cc:492] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:47.289712  6753 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:47.290097  6753 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:47.290445  6753 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.290271  6700 raft_consensus.cc:359] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.291234  6700 raft_consensus.cc:385] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.291528  6700 raft_consensus.cc:740] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.292634  6700 consensus_queue.cc:260] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.295185  6700 ts_tablet_manager.cc:1434] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 12:16:47.295437  6698 tablet_bootstrap.cc:654] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.295626  6699 tablet_bootstrap.cc:654] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.296766  6700 tablet_bootstrap.cc:492] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:47.298182  6784 consensus_queue.cc:1048] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:47.301342  6815 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:47.303128  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:47.305076  6656 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 0.
I20260709 12:16:47.305217  6700 tablet_bootstrap.cc:654] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.308158  6698 tablet_bootstrap.cc:492] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:47.308678  6698 ts_tablet_manager.cc:1403] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.027s	user 0.008s	sys 0.012s
I20260709 12:16:47.286139  6548 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:47.313519  6784 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:47.314191  6700 tablet_bootstrap.cc:492] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:47.314625  6700 ts_tablet_manager.cc:1403] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.004s
I20260709 12:16:47.315737  6698 raft_consensus.cc:359] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.316468  6698 raft_consensus.cc:385] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.316694  6698 raft_consensus.cc:740] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.317361  6698 consensus_queue.cc:260] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.319625  6698 ts_tablet_manager.cc:1434] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 12:16:47.319300  6700 raft_consensus.cc:359] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.320026  6700 raft_consensus.cc:385] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.320291  6700 raft_consensus.cc:740] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.320773  6807 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:47.321283  6807 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.322479  6698 tablet_bootstrap.cc:492] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:47.323465  6807 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:47.324321  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:47.328980  6583 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 0.
I20260709 12:16:47.330335  6473 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:16:47.331223  6807 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:47.331650  6807 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:47.332060  6807 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.325022  6753 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.325507  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:47.337608  6656 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 0.
I20260709 12:16:47.325538  6700 consensus_queue.cc:260] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.339955  6700 ts_tablet_manager.cc:1434] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 12:16:47.340934  6700 tablet_bootstrap.cc:492] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:47.340863  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:47.341522  6656 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:47.326164  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 44548.6us
I20260709 12:16:47.341974  6699 tablet_bootstrap.cc:492] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:47.342410  6699 ts_tablet_manager.cc:1403] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.053s	user 0.009s	sys 0.002s
I20260709 12:16:47.342891  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:47.343528  6583 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.345063  6699 raft_consensus.cc:359] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.345695  6699 raft_consensus.cc:385] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.345978  6699 raft_consensus.cc:740] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.336858  6698 tablet_bootstrap.cc:654] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.346866  6699 consensus_queue.cc:260] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.347205  6718 consensus_queue.cc:1048] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:47.348176  6779 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:47.348662  6779 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.349305  6699 ts_tablet_manager.cc:1434] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:16:47.350214  6699 tablet_bootstrap.cc:492] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:47.350823  6779 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:47.352845  6700 tablet_bootstrap.cc:654] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.351436  6583 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:16:47.356211  6807 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.364782  6753 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:47.366889  6807 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:47.368801  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:47.369756  6582 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.372365  6473 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:16:47.357107  6699 tablet_bootstrap.cc:654] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.356933  6821 consensus_queue.cc:1048] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:47.358553  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:47.363623  6656 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:16:47.355546  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:47.378329  6807 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:47.377142  6584 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:47.395772  6815 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:47.397461  6655 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.398940  6728 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:47.401726  6728 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.403760  6728 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:47.404311  6815 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.406302  6815 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:47.407845  6718 consensus_queue.cc:1048] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:47.408053  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:16:47.409878  6550 leader_election.cc:343] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:47.419271  6582 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:16:47.423087  6698 tablet_bootstrap.cc:492] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:47.423703  6698 ts_tablet_manager.cc:1403] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.101s	user 0.010s	sys 0.002s
W20260709 12:16:47.414450  6622 leader_election.cc:343] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:47.422343  6779 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:47.451655  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:47.453105  6622 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:47.454038  6704 raft_consensus.cc:2804] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:47.454634  6704 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:47.455139  6704 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.436134  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:47.457942  6656 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.464365  6779 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.467595  6779 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:16:47.469062  6475 leader_election.cc:343] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:47.480510  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:47.481307  6505 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:47.488410  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:47.489213  6582 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:47.489770  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:47.489244  6698 raft_consensus.cc:359] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.491078  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:47.491317  6698 raft_consensus.cc:385] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.492719  6870 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:47.493151  6870 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:47.493765  6870 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.492918  6698 raft_consensus.cc:740] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.495610  6698 consensus_queue.cc:260] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.490494  6505 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:47.499174  6807 raft_consensus.cc:697] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:16:47.515005  6704 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.545049  6698 ts_tablet_manager.cc:1434] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.121s	user 0.012s	sys 0.000s
I20260709 12:16:47.548161  6698 tablet_bootstrap.cc:492] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:47.532395  6548 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:47.548877  6656 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:16:47.562518  6699 tablet_bootstrap.cc:492] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:47.563011  6700 tablet_bootstrap.cc:492] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:47.563263  6699 ts_tablet_manager.cc:1403] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.213s	user 0.010s	sys 0.010s
I20260709 12:16:47.563464  6700 ts_tablet_manager.cc:1403] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.223s	user 0.062s	sys 0.073s
I20260709 12:16:47.566612  6704 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:47.557180  6821 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:47.567512  6821 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:47.567123  6699 raft_consensus.cc:359] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.567885  6821 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.567919  6699 raft_consensus.cc:385] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.568691  6699 raft_consensus.cc:740] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
W20260709 12:16:47.564239  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 62561us
I20260709 12:16:47.569409  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:47.570294  6582 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 0.
I20260709 12:16:47.572211  6718 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:47.569507  6699 consensus_queue.cc:260] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.572722  6718 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.574213  6699 ts_tablet_manager.cc:1434] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:16:47.575433  6699 tablet_bootstrap.cc:492] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:47.583222  6821 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.557729  6807 consensus_queue.cc:237] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.592036  6475 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:47.593324  6718 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:47.600739  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:47.602599  6655 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.602717  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:47.604429  6881 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:47.604970  6881 raft_consensus.cc:697] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:16:47.605803  6881 consensus_queue.cc:237] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:16:47.610661  6550 leader_election.cc:343] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:47.614188  6784 consensus_queue.cc:1048] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:16:47.614419  6548 leader_election.cc:343] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:47.615296  6548 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:47.617460  6784 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:47.623598  6728 consensus_queue.cc:1048] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:47.635540  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:47.635906  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:47.636471  6509 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:47.637408  6550 leader_election.cc:343] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:47.638060  6821 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:16:47.638468  6455 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:47.647746  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:47.648520  6582 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.643241  6700 raft_consensus.cc:359] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.650224  6700 raft_consensus.cc:385] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.654331  6700 raft_consensus.cc:740] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.653134  6473 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:47.660997  6807 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:47.665927  6807 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:47.675324  6698 tablet_bootstrap.cc:654] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:47.671433  6700 consensus_queue.cc:260] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.671904  6807 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.675803  6870 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.688304  6700 ts_tablet_manager.cc:1434] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.125s	user 0.013s	sys 0.000s
I20260709 12:16:47.691247  6700 tablet_bootstrap.cc:492] T 13ab2c6da5994ae2a955fc11d1761c41 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:47.696491  6718 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:47.696079  6728 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:16:47.692157  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:47.703599  6507 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.698343  6718 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.726495  6779 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:47.727062  6779 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.764586  6699 tablet_bootstrap.cc:654] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:47.761600  6888 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:16:47.775584  6779 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:47.788393  6718 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:47.792496  6870 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:47.796464  6698 tablet_bootstrap.cc:492] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:47.821120  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:47.842775  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:47.843719  6505 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.871663  6509 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:47.886592  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:47.887341  6583 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.887576  6622 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:47.888626  6779 raft_consensus.cc:2804] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:47.889199  6779 raft_consensus.cc:697] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:16:47.890067  6779 consensus_queue.cc:237] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.893047  6582 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:47.895819  6583 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:47.897279  6807 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.899729  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:47.900674  6655 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:47.923992  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:47.925239  6582 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.928072  6700 tablet_bootstrap.cc:654] T 13ab2c6da5994ae2a955fc11d1761c41 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:47.924045  6891 log.cc:927] Time spent T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 12:16:47.938941  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:16:47.931543  6548 leader_election.cc:343] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:47.934617  6698 ts_tablet_manager.cc:1403] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.387s	user 0.027s	sys 0.025s
I20260709 12:16:47.931636  6505 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:47.937427  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:47.900002  6807 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:47.947005  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:47.947974  6870 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:47.948498  6870 raft_consensus.cc:697] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:16:47.948791  6698 raft_consensus.cc:359] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.949514  6698 raft_consensus.cc:385] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:47.949389  6870 consensus_queue.cc:237] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.949836  6698 raft_consensus.cc:740] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:47.941840  6584 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:47.953812  6698 consensus_queue.cc:260] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:47.958988  6507 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:16:47.942541  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:16:47.963963  6622 leader_election.cc:343] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:47.968803  6548 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:47.971498  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:47.944821  6550 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:47.972591  6698 ts_tablet_manager.cc:1434] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.001s
I20260709 12:16:47.973909  6698 tablet_bootstrap.cc:492] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:47.976624  6699 tablet_bootstrap.cc:492] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:47.979521  6821 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:16:48.052011  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 83340.3us
I20260709 12:16:48.052286  6821 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:48.052851  6821 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.976650  6779 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:48.053793  6779 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:47.977490  6898 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:48.061141  6898 raft_consensus.cc:697] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:16:48.060390  6779 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:48.048655  6582 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:16:48.058491  6698 tablet_bootstrap.cc:654] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:48.045446  6699 ts_tablet_manager.cc:1403] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.470s	user 0.026s	sys 0.008s
I20260709 12:16:48.062134  6898 consensus_queue.cc:237] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:16:48.049348  6475 leader_election.cc:343] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:48.057024  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 115700us
I20260709 12:16:48.073845  6821 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.074360  6473 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:48.075340  6753 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:48.075819  6753 raft_consensus.cc:697] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:16:48.076692  6821 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:48.077019  6753 consensus_queue.cc:237] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.084784  6700 tablet_bootstrap.cc:492] T 13ab2c6da5994ae2a955fc11d1761c41 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:48.085289  6700 ts_tablet_manager.cc:1403] T 13ab2c6da5994ae2a955fc11d1761c41 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.396s	user 0.016s	sys 0.034s
I20260709 12:16:48.089622  6893 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:48.095058  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:48.092933  6700 raft_consensus.cc:359] T 13ab2c6da5994ae2a955fc11d1761c41 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.096553  6700 raft_consensus.cc:385] T 13ab2c6da5994ae2a955fc11d1761c41 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:48.096870  6700 raft_consensus.cc:740] T 13ab2c6da5994ae2a955fc11d1761c41 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:48.097981  6699 raft_consensus.cc:359] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.098949  6699 raft_consensus.cc:385] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:48.099375  6699 raft_consensus.cc:740] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:48.101347  6700 consensus_queue.cc:260] T 13ab2c6da5994ae2a955fc11d1761c41 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.104591  6699 consensus_queue.cc:260] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.109524  6699 ts_tablet_manager.cc:1434] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.040s	user 0.009s	sys 0.000s
I20260709 12:16:48.110710  6699 tablet_bootstrap.cc:492] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:48.112010  6807 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:48.112648  6700 ts_tablet_manager.cc:1434] T 13ab2c6da5994ae2a955fc11d1761c41 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.027s	user 0.004s	sys 0.006s
I20260709 12:16:48.113967  6700 tablet_bootstrap.cc:492] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:48.116286  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:16:48.127874  6548 leader_election.cc:343] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:48.142673  6753 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:48.148370  6698 tablet_bootstrap.cc:492] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:48.148942  6698 ts_tablet_manager.cc:1403] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.175s	user 0.009s	sys 0.015s
I20260709 12:16:48.153411  6699 tablet_bootstrap.cc:654] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:48.165639  6698 raft_consensus.cc:359] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.166599  6698 raft_consensus.cc:385] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:48.166936  6698 raft_consensus.cc:740] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:48.166976  6779 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.169495  6779 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:48.170912  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:48.172809  6655 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 1: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:16:48.174010  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:16:48.174340  6622 leader_election.cc:343] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:48.174808  6584 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 1: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:16:48.175897  6550 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:48.178138  6821 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:48.179026  6698 consensus_queue.cc:260] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.176112  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:48.182528  6870 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:48.182706  6407 catalog_manager.cc:5697] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 0 to 1, leader changed from <none> to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:48.185066  6407 catalog_manager.cc:5697] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 0 to 1, leader changed from <none> to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:48.191728  6698 ts_tablet_manager.cc:1434] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.001s
I20260709 12:16:48.192986  6698 tablet_bootstrap.cc:492] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:48.207504  6700 tablet_bootstrap.cc:654] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:48.221633  6698 tablet_bootstrap.cc:654] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:48.231834  6881 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:48.236145  6881 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:16:48.257264  6807 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:48.272279  6807 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.271509  6699 tablet_bootstrap.cc:492] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:48.275439  6699 ts_tablet_manager.cc:1403] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.165s	user 0.023s	sys 0.022s
I20260709 12:16:48.275910  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:48.276696  6582 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 0.
I20260709 12:16:48.277534  6807 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:48.277949  6473 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:16:48.283211  6881 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:48.283660  6881 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:48.283993  6881 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:48.279539  6699 raft_consensus.cc:359] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.281075  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:48.289608  6699 raft_consensus.cc:385] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:16:48.289552  6475 leader_election.cc:343] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:48.290691  6699 raft_consensus.cc:740] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:48.291595  6779 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:48.292207  6699 consensus_queue.cc:260] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.298595  6699 ts_tablet_manager.cc:1434] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 12:16:48.300489  6699 tablet_bootstrap.cc:492] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:48.301163  6700 tablet_bootstrap.cc:492] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:48.302248  6700 ts_tablet_manager.cc:1403] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.189s	user 0.013s	sys 0.012s
I20260709 12:16:48.306360  6881 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.310057  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:48.310808  6581 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:48.313328  6870 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:48.313887  6870 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.316228  6870 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:48.315336  6700 raft_consensus.cc:359] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.318147  6700 raft_consensus.cc:385] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:48.318249  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:48.318487  6700 raft_consensus.cc:740] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:48.319823  6881 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:16:48.320755  6475 leader_election.cc:343] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:48.320945  6699 tablet_bootstrap.cc:654] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:48.321724  6700 consensus_queue.cc:260] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.322939  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:48.323617  6505 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:48.324389  6700 ts_tablet_manager.cc:1434] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
I20260709 12:16:48.324597  6887 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:48.325773  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:48.326611  6584 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:48.327589  6700 tablet_bootstrap.cc:492] T 6d6c58127aec4100b761ec4b52b842a0 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:48.327461  6622 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:48.333352  6581 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:16:48.334472  6870 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:48.335038  6870 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:48.334909  6473 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:48.336468  6870 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:48.340054  6881 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:48.340832  6881 raft_consensus.cc:697] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:16:48.345084  6881 consensus_queue.cc:237] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.352408  6700 tablet_bootstrap.cc:654] T 6d6c58127aec4100b761ec4b52b842a0 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:48.356364  6698 tablet_bootstrap.cc:492] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:48.356900  6698 ts_tablet_manager.cc:1403] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.164s	user 0.015s	sys 0.010s
I20260709 12:16:48.358726  6887 consensus_queue.cc:1048] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:48.359661  6698 raft_consensus.cc:359] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.361418  6698 raft_consensus.cc:385] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:48.361739  6698 raft_consensus.cc:740] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:48.362484  6698 consensus_queue.cc:260] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.365561  6887 consensus_queue.cc:1048] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:48.369067  6698 ts_tablet_manager.cc:1434] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:16:48.370489  6700 tablet_bootstrap.cc:492] T 6d6c58127aec4100b761ec4b52b842a0 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:48.371146  6700 ts_tablet_manager.cc:1403] T 6d6c58127aec4100b761ec4b52b842a0 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.046s	user 0.008s	sys 0.009s
I20260709 12:16:48.374176  6700 raft_consensus.cc:359] T 6d6c58127aec4100b761ec4b52b842a0 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.375406  6700 raft_consensus.cc:385] T 6d6c58127aec4100b761ec4b52b842a0 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:48.375813  6700 raft_consensus.cc:740] T 6d6c58127aec4100b761ec4b52b842a0 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:48.377383  6700 consensus_queue.cc:260] T 6d6c58127aec4100b761ec4b52b842a0 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.381408  6700 ts_tablet_manager.cc:1434] T 6d6c58127aec4100b761ec4b52b842a0 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:16:48.382395  6700 tablet_bootstrap.cc:492] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:48.382965  6699 tablet_bootstrap.cc:492] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:48.383471  6699 ts_tablet_manager.cc:1403] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.083s	user 0.029s	sys 0.019s
I20260709 12:16:48.380234  6698 tablet_bootstrap.cc:492] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:48.387928  6918 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:48.388439  6918 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.390188  6821 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:48.394992  6821 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.385627  6870 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.399017  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:16:48.400671  6548 leader_election.cc:343] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:48.417532  6918 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:48.427065  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:16:48.429281  6550 leader_election.cc:343] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:48.429965  6550 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:48.431488  6898 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:48.433048  6699 raft_consensus.cc:359] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.434099  6699 raft_consensus.cc:385] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:48.435864  6699 raft_consensus.cc:740] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:48.423141  6821 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:48.440485  6700 tablet_bootstrap.cc:654] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:48.441296  6699 consensus_queue.cc:260] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.446043  6699 ts_tablet_manager.cc:1434] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.062s	user 0.017s	sys 0.019s
I20260709 12:16:48.440286  6870 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:48.448272  6699 tablet_bootstrap.cc:492] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
W20260709 12:16:48.441006  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 51202.3us
I20260709 12:16:48.451260  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:48.452878  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:48.454313  6507 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:48.456172  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:48.457063  6583 raft_consensus.cc:2393] T c6b5198a4b6741cdb13e49cf4e90173c P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 1: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:16:48.457996  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:48.458621  6581 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:48.459131  6699 tablet_bootstrap.cc:654] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:48.460897  6698 tablet_bootstrap.cc:654] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:48.461229  6507 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:48.465222  6622 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:48.465884  6779 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:48.466089  6622 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:48.466353  6779 raft_consensus.cc:697] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:16:48.466969  6887 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:48.473742  6581 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:48.477654  6779 consensus_queue.cc:237] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.477658  6881 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:16:48.480271  6700 tablet_bootstrap.cc:492] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:48.480777  6700 ts_tablet_manager.cc:1403] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.099s	user 0.012s	sys 0.005s
I20260709 12:16:48.483366  6887 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:48.483592  6700 raft_consensus.cc:359] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.484370  6700 raft_consensus.cc:385] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:48.484661  6700 raft_consensus.cc:740] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:48.485107  6406 catalog_manager.cc:5697] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:48.486614  6700 consensus_queue.cc:260] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.488521  6699 tablet_bootstrap.cc:492] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:48.489262  6699 ts_tablet_manager.cc:1403] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.042s	user 0.009s	sys 0.019s
I20260709 12:16:48.489441  6700 ts_tablet_manager.cc:1434] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.005s
I20260709 12:16:48.490458  6700 tablet_bootstrap.cc:492] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:48.493402  6699 raft_consensus.cc:359] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.494292  6699 raft_consensus.cc:385] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:48.494745  6699 raft_consensus.cc:740] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:48.495628  6699 consensus_queue.cc:260] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.496739  6700 tablet_bootstrap.cc:654] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:48.499055  6699 ts_tablet_manager.cc:1434] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:16:48.500589  6821 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:48.501215  6699 tablet_bootstrap.cc:492] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:48.510325  6699 tablet_bootstrap.cc:654] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:48.486842  6406 catalog_manager.cc:5697] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:48.523767  6658 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:48.549675  6881 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:48.557971  6821 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:48.577852  6404 catalog_manager.cc:5697] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:48.597240  6698 tablet_bootstrap.cc:492] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:48.598110  6698 ts_tablet_manager.cc:1403] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.222s	user 0.017s	sys 0.000s
I20260709 12:16:48.599283  6699 tablet_bootstrap.cc:492] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:48.599920  6699 ts_tablet_manager.cc:1403] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.099s	user 0.021s	sys 0.065s
I20260709 12:16:48.602159  6699 raft_consensus.cc:359] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.602833  6699 raft_consensus.cc:385] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:48.602675  6698 raft_consensus.cc:359] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.603183  6699 raft_consensus.cc:740] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:48.603350  6698 raft_consensus.cc:385] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:48.603614  6698 raft_consensus.cc:740] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:48.603852  6699 consensus_queue.cc:260] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.604187  6698 consensus_queue.cc:260] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.605904  6699 ts_tablet_manager.cc:1434] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:16:48.606623  6698 ts_tablet_manager.cc:1434] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:16:48.606727  6699 tablet_bootstrap.cc:492] T c9d981014b2c4cda80bca766fe58fe42 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:48.607473  6698 tablet_bootstrap.cc:492] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:48.613214  6699 tablet_bootstrap.cc:654] T c9d981014b2c4cda80bca766fe58fe42 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:48.613790  6698 tablet_bootstrap.cc:654] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:48.624521  6699 tablet_bootstrap.cc:492] T c9d981014b2c4cda80bca766fe58fe42 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:48.625180  6699 ts_tablet_manager.cc:1403] T c9d981014b2c4cda80bca766fe58fe42 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.004s
I20260709 12:16:48.626380  6407 catalog_manager.cc:5697] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 0 to 1, leader changed from <none> to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:48.628692  6699 raft_consensus.cc:359] T c9d981014b2c4cda80bca766fe58fe42 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.630232  6698 tablet_bootstrap.cc:492] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:48.632913  6699 raft_consensus.cc:385] T c9d981014b2c4cda80bca766fe58fe42 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:48.633285  6699 raft_consensus.cc:740] T c9d981014b2c4cda80bca766fe58fe42 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:48.633448  6698 ts_tablet_manager.cc:1403] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.005s
I20260709 12:16:48.634512  6699 consensus_queue.cc:260] T c9d981014b2c4cda80bca766fe58fe42 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.636602  6699 ts_tablet_manager.cc:1434] T c9d981014b2c4cda80bca766fe58fe42 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 12:16:48.637398  6699 tablet_bootstrap.cc:492] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:48.637560  6698 raft_consensus.cc:359] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.638409  6698 raft_consensus.cc:385] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:48.638653  6698 raft_consensus.cc:740] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:48.639264  6698 consensus_queue.cc:260] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.643087  6700 tablet_bootstrap.cc:492] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:48.643610  6700 ts_tablet_manager.cc:1403] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.153s	user 0.044s	sys 0.053s
I20260709 12:16:48.644227  6407 catalog_manager.cc:5697] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 0 to 1, leader changed from <none> to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:48.646446  6700 raft_consensus.cc:359] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.647269  6700 raft_consensus.cc:385] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:48.647583  6700 raft_consensus.cc:740] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:48.648452  6700 consensus_queue.cc:260] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.649765  6698 ts_tablet_manager.cc:1434] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.001s
I20260709 12:16:48.653254  6698 tablet_bootstrap.cc:492] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:48.654377  6699 tablet_bootstrap.cc:654] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:48.656646  6700 ts_tablet_manager.cc:1434] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.004s
I20260709 12:16:48.657904  6700 tablet_bootstrap.cc:492] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:48.661324  6698 tablet_bootstrap.cc:654] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:48.664217  6700 tablet_bootstrap.cc:654] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:48.691794  6699 tablet_bootstrap.cc:492] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:48.692279  6699 ts_tablet_manager.cc:1403] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.055s	user 0.006s	sys 0.004s
I20260709 12:16:48.692881  6881 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:48.693418  6881 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.695246  6699 raft_consensus.cc:359] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.696000  6699 raft_consensus.cc:385] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:48.696316  6699 raft_consensus.cc:740] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:48.697045  6699 consensus_queue.cc:260] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.699258  6699 ts_tablet_manager.cc:1434] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:16:48.699558  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:48.700227  6699 tablet_bootstrap.cc:492] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:48.700227  6581 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 0.
I20260709 12:16:48.703526  6881 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:48.706466  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:48.706804  6473 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:16:48.709026  6807 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:48.709424  6807 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:48.709796  6807 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:48.708006  6699 tablet_bootstrap.cc:654] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:48.715147  6475 leader_election.cc:343] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:48.717394  6807 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.720945  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:48.721612  6581 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:48.722747  6699 tablet_bootstrap.cc:492] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:48.723511  6699 ts_tablet_manager.cc:1403] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.002s
I20260709 12:16:48.729485  6699 raft_consensus.cc:359] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.730054  6881 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:48.730262  6699 raft_consensus.cc:385] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:48.730562  6581 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:16:48.732125  6473 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:16:48.733448  6883 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:48.734050  6883 raft_consensus.cc:697] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:16:48.735127  6883 consensus_queue.cc:237] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.741113  6807 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:48.730578  6699 raft_consensus.cc:740] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:48.742740  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:48.743189  6699 consensus_queue.cc:260] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.746021  6699 ts_tablet_manager.cc:1434] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
W20260709 12:16:48.746695  6475 leader_election.cc:343] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:48.748783  6699 tablet_bootstrap.cc:492] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:48.730609  6881 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.756765  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:48.757753  6581 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 1: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:16:48.759397  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:48.760617  6658 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 1: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:16:48.760723  6881 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:48.760615  6699 tablet_bootstrap.cc:654] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:48.762208  6475 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:48.763396  6807 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:48.774097  6699 tablet_bootstrap.cc:492] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:48.774698  6699 ts_tablet_manager.cc:1403] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.026s	user 0.008s	sys 0.010s
I20260709 12:16:48.779727  6699 raft_consensus.cc:359] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.780540  6699 raft_consensus.cc:385] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:48.780833  6699 raft_consensus.cc:740] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:48.781924  6807 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:48.781704  6699 consensus_queue.cc:260] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.783420  6698 tablet_bootstrap.cc:492] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:48.784111  6698 ts_tablet_manager.cc:1403] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.134s	user 0.021s	sys 0.067s
I20260709 12:16:48.784471  6699 ts_tablet_manager.cc:1434] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.009s	user 0.011s	sys 0.001s
I20260709 12:16:48.785400  6699 tablet_bootstrap.cc:492] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:48.786756  6698 raft_consensus.cc:359] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.787320  6698 raft_consensus.cc:385] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:16:48.787551  6455 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:48.787578  6698 raft_consensus.cc:740] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:48.789924  6698 consensus_queue.cc:260] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.791663  6699 tablet_bootstrap.cc:654] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:48.791973  6700 tablet_bootstrap.cc:492] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:48.792053  6698 ts_tablet_manager.cc:1434] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.003s
I20260709 12:16:48.792486  6700 ts_tablet_manager.cc:1403] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.135s	user 0.031s	sys 0.057s
I20260709 12:16:48.793601  6698 tablet_bootstrap.cc:492] T 3e7223833e1f47b7b68c0080facbbe95 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:48.795493  6700 raft_consensus.cc:359] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.796161  6700 raft_consensus.cc:385] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:48.796486  6700 raft_consensus.cc:740] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:48.797600  6779 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:48.797542  6700 consensus_queue.cc:260] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.798235  6779 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.800731  6700 ts_tablet_manager.cc:1434] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:16:48.801188  6698 tablet_bootstrap.cc:654] T 3e7223833e1f47b7b68c0080facbbe95 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:48.801764  6700 tablet_bootstrap.cc:492] T 8e01550f5e994006a4031ca81edcc9aa P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:48.801498  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:48.803555  6699 tablet_bootstrap.cc:492] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
W20260709 12:16:48.803705  6622 leader_election.cc:343] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:48.804000  6699 ts_tablet_manager.cc:1403] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.019s	user 0.006s	sys 0.009s
I20260709 12:16:48.805209  6779 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:48.806859  6699 raft_consensus.cc:359] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.811225  6699 raft_consensus.cc:385] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:48.811517  6699 raft_consensus.cc:740] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:48.808001  6700 tablet_bootstrap.cc:654] T 8e01550f5e994006a4031ca81edcc9aa P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:48.808538  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:48.814585  6698 tablet_bootstrap.cc:492] T 3e7223833e1f47b7b68c0080facbbe95 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:48.812152  6699 consensus_queue.cc:260] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.815536  6698 ts_tablet_manager.cc:1403] T 3e7223833e1f47b7b68c0080facbbe95 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.006s
W20260709 12:16:48.815779  6622 leader_election.cc:343] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:48.816365  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:48.824484  6779 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:48.824240  6881 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:48.825982  6405 catalog_manager.cc:5697] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:48.829579  6898 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:48.830166  6898 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.831123  6779 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:48.831632  6779 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.835716  6779 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:48.815976  6699 ts_tablet_manager.cc:1434] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:16:48.837984  6699 tablet_bootstrap.cc:492] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:48.844153  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:48.845494  6583 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:48.847523  6897 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:48.847604  6821 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:48.848153  6897 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.848376  6821 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.851508  6821 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:48.845770  6407 catalog_manager.cc:5697] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 0 to 1, leader changed from <none> to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:48.853415  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:48.857195  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:16:48.857412  6550 leader_election.cc:343] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:48.864775  6656 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:48.866413  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:48.867096  6658 raft_consensus.cc:2468] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:48.864638  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:48.868629  6550 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:48.869501  6821 raft_consensus.cc:2804] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:48.861068  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:48.869560  6622 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:48.869926  6821 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:48.870347  6821 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:48.869764  6548 leader_election.cc:343] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:48.871158  6622 leader_election.cc:343] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:48.871346  6548 leader_election.cc:343] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:48.871733  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:48.871765  6548 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:48.872977  6784 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:16:48.874017  6548 leader_election.cc:343] T 6d6c58127aec4100b761ec4b52b842a0 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:48.874691  6779 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:48.875070  6779 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:48.875392  6779 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:48.861565  6897 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:48.865518  6698 raft_consensus.cc:359] T 3e7223833e1f47b7b68c0080facbbe95 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.878979  6698 raft_consensus.cc:385] T 3e7223833e1f47b7b68c0080facbbe95 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:48.879248  6698 raft_consensus.cc:740] T 3e7223833e1f47b7b68c0080facbbe95 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:48.879794  6898 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:48.879913  6698 consensus_queue.cc:260] T 3e7223833e1f47b7b68c0080facbbe95 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.883715  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:48.877252  6821 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.885715  6698 ts_tablet_manager.cc:1434] T 3e7223833e1f47b7b68c0080facbbe95 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.070s	user 0.006s	sys 0.000s
I20260709 12:16:48.889936  6698 tablet_bootstrap.cc:492] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:48.891745  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:48.893425  6821 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:16:48.894178  6548 leader_election.cc:343] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:48.894562  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:48.895522  6657 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:48.898890  6887 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:48.899405  6700 tablet_bootstrap.cc:492] T 8e01550f5e994006a4031ca81edcc9aa P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:48.899847  6700 ts_tablet_manager.cc:1403] T 8e01550f5e994006a4031ca81edcc9aa P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.098s	user 0.018s	sys 0.036s
I20260709 12:16:48.902458  6700 raft_consensus.cc:359] T 8e01550f5e994006a4031ca81edcc9aa P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.903092  6700 raft_consensus.cc:385] T 8e01550f5e994006a4031ca81edcc9aa P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:48.903352  6700 raft_consensus.cc:740] T 8e01550f5e994006a4031ca81edcc9aa P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:48.904032  6700 consensus_queue.cc:260] T 8e01550f5e994006a4031ca81edcc9aa P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.905233  6779 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.906008  6700 ts_tablet_manager.cc:1434] T 8e01550f5e994006a4031ca81edcc9aa P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 12:16:48.906514  6658 raft_consensus.cc:2393] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 1: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:16:48.906848  6700 tablet_bootstrap.cc:492] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:48.907529  6779 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:48.908784  6550 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:48.911166  6821 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:48.911060  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:16:48.912644  6622 leader_election.cc:343] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:48.912755  6881 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:48.913237  6881 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.934118  6657 raft_consensus.cc:2468] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:16:48.934202  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:48.935034  6581 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 0.
I20260709 12:16:48.941457  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:48.935462  6550 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:48.943338  6784 raft_consensus.cc:2804] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:48.943893  6784 raft_consensus.cc:697] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:16:48.946961  6699 tablet_bootstrap.cc:654] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:48.948921  6881 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:48.954859  6821 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:48.949292  6784 consensus_queue.cc:237] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.949759  6883 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:16:48.947551  6473 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:16:48.962076  6887 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:48.964077  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:16:48.969339  6475 leader_election.cc:343] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:48.971242  6881 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:48.971760  6881 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:48.973703  6698 tablet_bootstrap.cc:654] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:48.974171  6881 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:48.978229  6700 tablet_bootstrap.cc:654] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:48.985533  6582 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:48.989950  6779 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:48.993944  6700 tablet_bootstrap.cc:492] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:48.994663  6700 ts_tablet_manager.cc:1403] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.088s	user 0.012s	sys 0.004s
I20260709 12:16:48.990562  6779 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:48.998961  6887 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:48.999300  6779 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:48.999465  6887 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.015602  6699 tablet_bootstrap.cc:492] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:49.016429  6699 ts_tablet_manager.cc:1403] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.179s	user 0.012s	sys 0.017s
I20260709 12:16:49.017570  6700 raft_consensus.cc:359] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.018455  6700 raft_consensus.cc:385] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.018764  6700 raft_consensus.cc:740] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.012848  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:49.028919  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:49.035600  6583 raft_consensus.cc:2393] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 1: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:16:49.023180  6753 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:49.038189  6753 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.043828  6887 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:49.044179  6753 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:49.046437  6699 raft_consensus.cc:359] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.048200  6699 raft_consensus.cc:385] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.049803  6699 raft_consensus.cc:740] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.053205  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:49.053985  6506 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:49.053664  6699 consensus_queue.cc:260] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.056280  6622 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:49.056632  6698 tablet_bootstrap.cc:492] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:49.057132  6698 ts_tablet_manager.cc:1403] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.167s	user 0.018s	sys 0.008s
I20260709 12:16:49.057207  6925 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:49.058123  6622 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:49.059049  6699 ts_tablet_manager.cc:1434] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.042s	user 0.011s	sys 0.001s
I20260709 12:16:49.053205  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:49.060904  6584 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 0.
I20260709 12:16:49.061223  6887 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.062897  6925 raft_consensus.cc:697] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:16:49.062741  6698 raft_consensus.cc:359] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.063652  6473 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:16:49.063832  6925 consensus_queue.cc:237] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.064647  6753 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:49.064976  6753 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:49.065340  6753 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.063652  6698 raft_consensus.cc:385] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.068871  6698 raft_consensus.cc:740] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.048144  6881 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.062741  6700 consensus_queue.cc:260] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.077251  6784 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:49.077648  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:49.077715  6784 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.078457  6584 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:49.079831  6784 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:49.085559  6698 consensus_queue.cc:260] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.084517  6622 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:49.087514  6700 ts_tablet_manager.cc:1434] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.092s	user 0.012s	sys 0.000s
I20260709 12:16:49.085191  6699 tablet_bootstrap.cc:492] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:49.100589  6700 tablet_bootstrap.cc:492] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:49.087706  6407 catalog_manager.cc:5697] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:49.091622  6698 ts_tablet_manager.cc:1434] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.034s	user 0.003s	sys 0.005s
I20260709 12:16:49.103392  6698 tablet_bootstrap.cc:492] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:49.105343  6883 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:49.105815  6883 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.108739  6883 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:49.088735  6779 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:49.109440  6779 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:49.109840  6779 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.094590  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:49.089709  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:49.090090  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:49.115057  6656 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:49.118160  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:49.118712  6508 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:49.119532  6584 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:49.119947  6473 leader_election.cc:343] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:49.119992  6548 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:16:49.120940  6897 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:49.121315  6897 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:49.121649  6897 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.095359  6881 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:49.129071  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:49.141836  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:49.142488  6925 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:49.173295  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:49.159862  6925 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:16:49.186127  6475 leader_election.cc:343] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:49.187114  6475 leader_election.cc:343] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:49.187616  6475 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:49.189579  6807 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:16:49.234984  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 103284us
I20260709 12:16:49.238332  6925 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:49.245348  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:49.246642  6698 tablet_bootstrap.cc:654] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:49.248991  6622 leader_election.cc:343] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:49.249378  6405 catalog_manager.cc:5697] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:49.256984  6700 tablet_bootstrap.cc:654] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.243836  6784 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:49.241577  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 54508.1us
I20260709 12:16:49.265570  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:49.267884  6584 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:16:49.271575  6887 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:49.271500  6473 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:49.272511  6807 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:49.272244  6779 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.272981  6807 raft_consensus.cc:697] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:16:49.273218  6881 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:49.273875  6807 consensus_queue.cc:237] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.274708  6656 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 1: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:16:49.274784  6779 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:49.276477  6753 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:16:49.267567  6622 leader_election.cc:343] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:49.278211  6622 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:49.264925  6784 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.282629  6897 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.307245  6897 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:49.300307  6887 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:49.319311  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:49.320158  6508 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 1: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:16:49.315945  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:49.325528  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:49.327040  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:49.327598  6656 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.324507  6584 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.336655  6700 tablet_bootstrap.cc:492] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:49.337144  6700 ts_tablet_manager.cc:1403] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.237s	user 0.011s	sys 0.028s
I20260709 12:16:49.324242  6753 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:49.341014  6699 tablet_bootstrap.cc:654] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.341954  6784 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:49.343478  6656 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:16:49.345840  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:49.347195  6405 catalog_manager.cc:5697] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 0 to 1, leader changed from <none> to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:49.333287  6807 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 12:16:49.356657  6550 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:49.357650  6960 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:49.358196  6960 raft_consensus.cc:697] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:16:49.359118  6960 consensus_queue.cc:237] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.363201  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:49.395205  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:49.399016  6656 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 1: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:16:49.410091  6475 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:49.411017  6807 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:16:49.422988  6966 log.cc:927] Time spent T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:16:49.444048  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:49.444574  6584 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
W20260709 12:16:49.445214  6550 leader_election.cc:343] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:49.444844  6505 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.444906  6700 raft_consensus.cc:359] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.473234  6700 raft_consensus.cc:385] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.473529  6700 raft_consensus.cc:740] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.474107  6700 consensus_queue.cc:260] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.448354  6698 tablet_bootstrap.cc:492] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
W20260709 12:16:49.453984  6548 leader_election.cc:343] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:49.475346  6698 ts_tablet_manager.cc:1403] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.372s	user 0.010s	sys 0.020s
I20260709 12:16:49.475360  6548 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:49.476253  6960 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:49.479206  6622 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:49.479984  6887 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:49.480501  6887 raft_consensus.cc:697] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:16:49.481504  6925 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:49.481397  6887 consensus_queue.cc:237] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.461935  6699 tablet_bootstrap.cc:492] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:49.484076  6699 ts_tablet_manager.cc:1403] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.424s	user 0.021s	sys 0.018s
I20260709 12:16:49.486112  6698 raft_consensus.cc:359] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.486799  6698 raft_consensus.cc:385] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.487067  6698 raft_consensus.cc:740] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.487138  6699 raft_consensus.cc:359] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.487722  6698 consensus_queue.cc:260] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.490638  6698 ts_tablet_manager.cc:1434] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 12:16:49.488188  6699 raft_consensus.cc:385] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.490945  6699 raft_consensus.cc:740] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.491274  6700 ts_tablet_manager.cc:1434] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.154s	user 0.007s	sys 0.009s
I20260709 12:16:49.491771  6698 tablet_bootstrap.cc:492] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:49.491585  6699 consensus_queue.cc:260] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.492486  6700 tablet_bootstrap.cc:492] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:49.493193  6699 ts_tablet_manager.cc:1434] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.002s
I20260709 12:16:49.494025  6699 tablet_bootstrap.cc:492] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:49.500758  6698 tablet_bootstrap.cc:654] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.506301  6407 catalog_manager.cc:5697] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
W20260709 12:16:49.509320  6548 consensus_peers.cc:597] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:49.516181  6622 consensus_peers.cc:597] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:49.511176  6505 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:16:49.509888  6404 catalog_manager.cc:5697] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:49.515722  6699 tablet_bootstrap.cc:654] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.522688  6700 tablet_bootstrap.cc:654] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.543200  6699 tablet_bootstrap.cc:492] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:49.543704  6699 ts_tablet_manager.cc:1403] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.050s	user 0.011s	sys 0.006s
I20260709 12:16:49.546557  6699 raft_consensus.cc:359] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.547262  6699 raft_consensus.cc:385] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.547538  6699 raft_consensus.cc:740] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.548143  6699 consensus_queue.cc:260] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.552229  6700 tablet_bootstrap.cc:492] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:49.552827  6700 ts_tablet_manager.cc:1403] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.061s	user 0.020s	sys 0.001s
I20260709 12:16:49.554774  6698 tablet_bootstrap.cc:492] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:49.555037  6699 ts_tablet_manager.cc:1434] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:16:49.555248  6698 ts_tablet_manager.cc:1403] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.064s	user 0.013s	sys 0.006s
I20260709 12:16:49.555963  6699 tablet_bootstrap.cc:492] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:49.560410  6700 raft_consensus.cc:359] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.560822  6784 consensus_queue.cc:1048] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:49.564574  6698 raft_consensus.cc:359] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.565212  6699 tablet_bootstrap.cc:654] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.565454  6698 raft_consensus.cc:385] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.566547  6700 raft_consensus.cc:385] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.566493  6698 raft_consensus.cc:740] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.567040  6700 raft_consensus.cc:740] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.567623  6698 consensus_queue.cc:260] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.568601  6700 consensus_queue.cc:260] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.576151  6700 ts_tablet_manager.cc:1434] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.003s
I20260709 12:16:49.577744  6700 tablet_bootstrap.cc:492] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:49.580063  6698 ts_tablet_manager.cc:1434] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 12:16:49.587808  6700 tablet_bootstrap.cc:654] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.588467  6698 tablet_bootstrap.cc:492] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:49.597708  6699 tablet_bootstrap.cc:492] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:49.598357  6699 ts_tablet_manager.cc:1403] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.043s	user 0.013s	sys 0.005s
I20260709 12:16:49.606671  6698 tablet_bootstrap.cc:654] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.602344  6699 raft_consensus.cc:359] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.610077  6699 raft_consensus.cc:385] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.610427  6699 raft_consensus.cc:740] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.612500  6699 consensus_queue.cc:260] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.613809  6700 tablet_bootstrap.cc:492] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:49.614352  6700 ts_tablet_manager.cc:1403] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.037s	user 0.011s	sys 0.008s
I20260709 12:16:49.617619  6698 tablet_bootstrap.cc:492] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:49.617233  6700 raft_consensus.cc:359] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.618098  6700 raft_consensus.cc:385] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.618106  6698 ts_tablet_manager.cc:1403] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.030s	user 0.005s	sys 0.011s
I20260709 12:16:49.618552  6700 raft_consensus.cc:740] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.619463  6700 consensus_queue.cc:260] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.621026  6698 raft_consensus.cc:359] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.621721  6698 raft_consensus.cc:385] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.622525  6700 ts_tablet_manager.cc:1434] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:16:49.622020  6698 raft_consensus.cc:740] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.623551  6700 tablet_bootstrap.cc:492] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:49.623335  6698 consensus_queue.cc:260] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.625231  6699 ts_tablet_manager.cc:1434] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 12:16:49.625371  6698 ts_tablet_manager.cc:1434] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:16:49.626233  6699 tablet_bootstrap.cc:492] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:49.626413  6698 tablet_bootstrap.cc:492] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:49.634567  6699 tablet_bootstrap.cc:654] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.634940  6700 tablet_bootstrap.cc:654] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.634953  6698 tablet_bootstrap.cc:654] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.649374  6698 tablet_bootstrap.cc:492] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:49.649858  6698 ts_tablet_manager.cc:1403] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.024s	user 0.009s	sys 0.003s
I20260709 12:16:49.652979  6698 raft_consensus.cc:359] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.653712  6698 raft_consensus.cc:385] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.654721  6698 raft_consensus.cc:740] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.655437  6698 consensus_queue.cc:260] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.658084  6698 ts_tablet_manager.cc:1434] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:16:49.660154  6698 tablet_bootstrap.cc:492] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:49.662377  6700 tablet_bootstrap.cc:492] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:49.663074  6700 ts_tablet_manager.cc:1403] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.040s	user 0.016s	sys 0.013s
I20260709 12:16:49.666304  6698 tablet_bootstrap.cc:654] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.667375  6700 raft_consensus.cc:359] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.668398  6700 raft_consensus.cc:385] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.668845  6700 raft_consensus.cc:740] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.684685  6881 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:49.669893  6700 consensus_queue.cc:260] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.693588  6700 ts_tablet_manager.cc:1434] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 12:16:49.694685  6700 tablet_bootstrap.cc:492] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:49.695171  6699 tablet_bootstrap.cc:492] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:49.695660  6699 ts_tablet_manager.cc:1403] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.070s	user 0.013s	sys 0.036s
I20260709 12:16:49.698689  6699 raft_consensus.cc:359] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.699458  6699 raft_consensus.cc:385] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.699765  6699 raft_consensus.cc:740] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.700515  6699 consensus_queue.cc:260] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.702675  6700 tablet_bootstrap.cc:654] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.704277  6698 tablet_bootstrap.cc:492] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:49.704836  6698 ts_tablet_manager.cc:1403] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.045s	user 0.017s	sys 0.019s
I20260709 12:16:49.707597  6698 raft_consensus.cc:359] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.708276  6698 raft_consensus.cc:385] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.708603  6698 raft_consensus.cc:740] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.709022  6699 ts_tablet_manager.cc:1434] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.004s
I20260709 12:16:49.709430  6698 consensus_queue.cc:260] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.710879  6959 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:49.711392  6959 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.712098  6699 tablet_bootstrap.cc:492] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
W20260709 12:16:49.713732  6475 consensus_peers.cc:597] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:49.713723  6959 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:49.715107  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:49.715778  6505 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:49.717299  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:49.717984  6581 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:49.718043  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:49.718722  6698 ts_tablet_manager.cc:1434] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:16:49.719138  6959 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:49.719542  6959 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:49.719739  6698 tablet_bootstrap.cc:492] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:49.719925  6959 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:49.726949  6698 tablet_bootstrap.cc:654] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.727248  6699 tablet_bootstrap.cc:654] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.738276  6959 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.741357  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:49.741895  6959 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:49.744457  6700 tablet_bootstrap.cc:492] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:49.745025  6700 ts_tablet_manager.cc:1403] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.051s	user 0.014s	sys 0.006s
I20260709 12:16:49.745522  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:49.742183  6505 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:49.746219  6582 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:49.748221  6700 raft_consensus.cc:359] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.749066  6700 raft_consensus.cc:385] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.749503  6700 raft_consensus.cc:740] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.750608  6700 consensus_queue.cc:260] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.752410  6700 ts_tablet_manager.cc:1434] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:16:49.759889  6700 tablet_bootstrap.cc:492] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:49.764583  6582 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:16:49.766062  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:49.766841  6959 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:49.767331  6959 raft_consensus.cc:697] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:16:49.768195  6959 consensus_queue.cc:237] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.781908  6699 tablet_bootstrap.cc:492] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:49.782648  6699 ts_tablet_manager.cc:1403] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.073s	user 0.011s	sys 0.019s
I20260709 12:16:49.790208  6505 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:16:49.798379  6698 tablet_bootstrap.cc:492] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:49.799443  6698 ts_tablet_manager.cc:1403] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.080s	user 0.029s	sys 0.012s
I20260709 12:16:49.802639  6699 raft_consensus.cc:359] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.803668  6699 raft_consensus.cc:385] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.803990  6699 raft_consensus.cc:740] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.804644  6699 consensus_queue.cc:260] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.806141  6699 ts_tablet_manager.cc:1434] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 12:16:49.807024  6699 tablet_bootstrap.cc:492] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:49.808113  6897 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:49.808781  6897 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.812016  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:49.812156  6897 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:49.814581  6967 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:49.812848  6658 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:49.815732  6967 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.812857  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:49.816864  6550 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:49.820212  6898 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:49.820447  6967 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:49.821609  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:49.822576  6582 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 0.
I20260709 12:16:49.820698  6898 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:49.824007  6898 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.824537  6473 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:16:49.826296  6967 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:16:49.826399  6548 leader_election.cc:343] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:49.827147  6967 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:49.828362  6967 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.829653  6404 catalog_manager.cc:5697] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 0 to 2, leader changed from <none> to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 2 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:49.830045  6698 raft_consensus.cc:359] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.830938  6698 raft_consensus.cc:385] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.831246  6698 raft_consensus.cc:740] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.831853  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:49.831900  6698 consensus_queue.cc:260] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:16:49.834462  6475 leader_election.cc:343] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:49.834581  6698 ts_tablet_manager.cc:1434] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.035s	user 0.005s	sys 0.001s
I20260709 12:16:49.835631  6698 tablet_bootstrap.cc:492] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:49.837152  6898 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.842070  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:16:49.844784  6548 leader_election.cc:343] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:49.846567  6699 tablet_bootstrap.cc:654] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.849830  6698 tablet_bootstrap.cc:654] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.850329  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:49.848619  6898 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:49.852254  6655 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.858846  6700 tablet_bootstrap.cc:654] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.858695  6963 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:49.859421  6963 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.871315  6963 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:49.864179  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:49.874642  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:49.874850  6505 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:49.863215  6655 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:16:49.875365  6581 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:49.875980  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:49.876729  6779 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:49.876902  6550 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:49.877151  6779 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:49.877558  6779 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.877681  6784 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:49.878206  6784 raft_consensus.cc:697] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:16:49.878996  6784 consensus_queue.cc:237] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.880653  6967 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.883921  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:49.884512  6582 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.885593  6779 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.887602  6779 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:49.888525  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:49.888749  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:49.889213  6505 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.889565  6581 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.891813  6582 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:16:49.892237  6898 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:49.892974  6473 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:16:49.892937  6898 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.893786  6883 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:49.894268  6883 raft_consensus.cc:697] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:16:49.895080  6883 consensus_queue.cc:237] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.897378  6505 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:49.900974  6898 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:49.901540  6960 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:49.902612  6960 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.903420  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:49.904222  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:16:49.908008  6548 leader_election.cc:343] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:49.912531  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:49.913151  6508 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:49.905092  6887 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:49.914740  6887 raft_consensus.cc:697] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:16:49.914369  6548 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:16:49.901284  6779 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:49.917281  6404 catalog_manager.cc:5697] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:49.921645  6898 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:49.916258  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:49.922176  6898 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:49.901608  6963 consensus_queue.cc:1048] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 12:16:49.910902  6967 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:49.922739  6960 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:49.929136  6897 consensus_queue.cc:1048] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:16:49.933090  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:49.915601  6887 consensus_queue.cc:237] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.921350  6779 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.938504  6698 tablet_bootstrap.cc:492] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:49.932432  6898 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.941022  6698 ts_tablet_manager.cc:1403] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.106s	user 0.013s	sys 0.008s
I20260709 12:16:49.935468  6967 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:49.942344  6967 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.943971  6881 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:49.948156  6475 leader_election.cc:343] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:49.949332  6699 tablet_bootstrap.cc:492] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:49.949779  6699 ts_tablet_manager.cc:1403] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.143s	user 0.014s	sys 0.001s
I20260709 12:16:49.955492  6925 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:49.946674  6881 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:50.028101  6779 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:49.973991  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:49.973999  6898 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:16:49.963514  6550 leader_election.cc:343] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:50.044920  6550 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:50.048118  6700 tablet_bootstrap.cc:492] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:50.049042  6700 ts_tablet_manager.cc:1403] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.289s	user 0.025s	sys 0.024s
I20260709 12:16:49.955955  6698 raft_consensus.cc:359] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:50.049660  6698 raft_consensus.cc:385] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:16:50.049471  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 85994.2us
I20260709 12:16:50.050105  6698 raft_consensus.cc:740] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:50.057240  6784 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:50.063210  6898 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:49.956202  6581 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:49.979357  6967 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:49.990034  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:50.059756  6698 consensus_queue.cc:260] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:49.997726  6807 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:49.957970  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:50.074898  6698 ts_tablet_manager.cc:1434] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.133s	user 0.006s	sys 0.002s
W20260709 12:16:50.067840  6455 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:50.064278  6699 raft_consensus.cc:359] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:50.084923  6699 raft_consensus.cc:385] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:50.085343  6699 raft_consensus.cc:740] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:50.086442  6699 consensus_queue.cc:260] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:50.066848  6996 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:49.956027  6925 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:50.070578  6881 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:50.092845  6658 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:50.101603  6407 catalog_manager.cc:5697] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 0 to 1, leader changed from <none> to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:50.106657  6698 tablet_bootstrap.cc:492] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:50.099362  6996 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:50.075001  6807 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:16:50.114413  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 62392.2us
I20260709 12:16:50.114506  6700 raft_consensus.cc:359] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:50.115890  6700 raft_consensus.cc:385] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:50.116161  6700 raft_consensus.cc:740] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:50.116781  6925 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:50.116878  6700 consensus_queue.cc:260] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:50.118862  6700 ts_tablet_manager.cc:1434] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.069s	user 0.005s	sys 0.000s
I20260709 12:16:50.125654  6699 ts_tablet_manager.cc:1434] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.176s	user 0.009s	sys 0.007s
I20260709 12:16:50.127321  6699 tablet_bootstrap.cc:492] T f9f8f5d393744ebabbed2f3e0131cf40 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:50.130434  6700 tablet_bootstrap.cc:492] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
W20260709 12:16:50.135371  6622 leader_election.cc:343] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:50.146648  6698 tablet_bootstrap.cc:654] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:50.146865  6699 tablet_bootstrap.cc:654] T f9f8f5d393744ebabbed2f3e0131cf40 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:50.158671  6473 leader_election.cc:343] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:50.140020  6779 consensus_queue.cc:1048] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260709 12:16:50.161702  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:50.167299  6699 tablet_bootstrap.cc:492] T f9f8f5d393744ebabbed2f3e0131cf40 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:50.168052  6699 ts_tablet_manager.cc:1403] T f9f8f5d393744ebabbed2f3e0131cf40 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.041s	user 0.013s	sys 0.009s
I20260709 12:16:50.161988  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:50.173996  6658 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:50.168157  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:50.185640  6881 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:50.186157  6881 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:50.187994  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:50.179673  6807 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:50.188138  6881 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:16:50.193202  6475 leader_election.cc:343] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:50.194191  6475 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:50.185643  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:50.200754  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:16:50.201193  6622 leader_election.cc:343] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:50.201957  6658 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:16:50.210745  6550 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:50.211475  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:50.215095  6898 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:50.216390  6699 raft_consensus.cc:359] T f9f8f5d393744ebabbed2f3e0131cf40 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:50.217170  6699 raft_consensus.cc:385] T f9f8f5d393744ebabbed2f3e0131cf40 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:50.217680  6699 raft_consensus.cc:740] T f9f8f5d393744ebabbed2f3e0131cf40 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
W20260709 12:16:50.218514  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 304262us
I20260709 12:16:50.222865  6699 consensus_queue.cc:260] T f9f8f5d393744ebabbed2f3e0131cf40 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:50.224705  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:50.221766  6404 catalog_manager.cc:5697] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:50.230544  6699 ts_tablet_manager.cc:1434] T f9f8f5d393744ebabbed2f3e0131cf40 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.062s	user 0.008s	sys 0.000s
I20260709 12:16:50.218003  6883 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:50.231613  6699 tablet_bootstrap.cc:492] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:50.232709  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:50.232738  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:16:50.232146  6473 leader_election.cc:343] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:50.226109  6898 raft_consensus.cc:697] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:16:50.239727  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:50.232632  7004 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:50.245431  6698 tablet_bootstrap.cc:492] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:50.245991  6698 ts_tablet_manager.cc:1403] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.140s	user 0.015s	sys 0.012s
I20260709 12:16:50.250190  6698 raft_consensus.cc:359] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:50.250893  6698 raft_consensus.cc:385] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:50.251174  6698 raft_consensus.cc:740] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:50.253255  6698 consensus_queue.cc:260] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:50.242650  6581 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:50.252723  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:50.270920  6505 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:50.275987  6698 ts_tablet_manager.cc:1434] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.002s
I20260709 12:16:50.277400  6698 tablet_bootstrap.cc:492] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:50.278321  6506 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:50.280723  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 45081.4us
I20260709 12:16:50.273633  6898 consensus_queue.cc:237] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:50.295923  6475 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:50.296608  6821 consensus_queue.cc:1048] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:16:50.297693  6475 leader_election.cc:343] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:50.298401  6475 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:50.300940  6475 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:50.304010  6967 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:50.325145  6700 tablet_bootstrap.cc:654] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:50.306377  6883 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:50.267129  7004 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:50.328121  6779 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:50.372254  7004 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:50.364526  6583 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:50.370885  6925 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:50.387109  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 94128.6us
I20260709 12:16:50.379616  6508 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:50.367070  6779 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:16:50.397217  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 98813.7us
I20260709 12:16:50.391690  6925 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:50.406069  6622 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:50.406754  6699 tablet_bootstrap.cc:654] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:50.427575  6698 tablet_bootstrap.cc:654] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:50.418913  6959 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:50.437638  6507 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:16:50.446532  6959 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:50.455374  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:50.494589  6959 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:50.508381  6652 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:50.512957  6925 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:50.524485  6779 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:50.536897  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:50.525974  6963 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 12:16:50.555428  6999 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:50.560503  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:50.561944  6999 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:50.566963  6999 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:50.556015  6505 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
W20260709 12:16:50.592935  6622 leader_election.cc:343] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:50.596968  6550 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:50.590633  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:50.594512  6999 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 12:16:50.617779  6698 tablet_bootstrap.cc:492] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:50.618356  6698 ts_tablet_manager.cc:1403] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.341s	user 0.023s	sys 0.019s
I20260709 12:16:50.617229  7017 consensus_queue.cc:1048] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 12:16:50.628851  6699 tablet_bootstrap.cc:492] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:50.697079  6700 tablet_bootstrap.cc:492] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:50.701115  6699 ts_tablet_manager.cc:1403] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.470s	user 0.044s	sys 0.015s
I20260709 12:16:50.701251  6700 ts_tablet_manager.cc:1403] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.571s	user 0.030s	sys 0.012s
I20260709 12:16:50.628743  7023 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:50.675444  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:16:50.689301  6548 consensus_peers.cc:597] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:50.690186  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:50.638226  7021 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:50.702097  7023 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:50.704525  6700 raft_consensus.cc:359] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:50.637624  7002 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:50.637517  6577 raft_consensus.cc:2393] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 1: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:16:50.694856  6698 raft_consensus.cc:359] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:50.705276  7023 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:50.705317  7021 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:16:50.660516  7019 log.cc:927] Time spent T 8066a9d7ccf34c89ad8d3df18c4e86c2 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.149s	user 0.003s	sys 0.000s
W20260709 12:16:50.708742  6473 leader_election.cc:343] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:50.709236  6700 raft_consensus.cc:385] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:50.709311  6698 raft_consensus.cc:385] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:50.730041  6698 raft_consensus.cc:740] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:50.694010  6406 catalog_manager.cc:5697] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:50.688961  6584 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:16:50.651019  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:50.729486  6700 raft_consensus.cc:740] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:50.733467  6698 consensus_queue.cc:260] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:50.713376  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:50.733680  6700 consensus_queue.cc:260] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:50.655978  6996 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:16:50.740202  6548 leader_election.cc:343] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:50.748235  6651 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:50.753221  6700 ts_tablet_manager.cc:1434] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.051s	user 0.006s	sys 0.001s
I20260709 12:16:50.755837  6700 tablet_bootstrap.cc:492] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:50.738385  6925 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:50.761286  7011 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:50.761885  7011 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:50.774204  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:50.775231  6576 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:50.777599  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:50.778330  6505 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:50.759603  6925 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:50.812249  6698 ts_tablet_manager.cc:1434] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.194s	user 0.010s	sys 0.002s
I20260709 12:16:50.813736  6698 tablet_bootstrap.cc:492] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:50.838176  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:50.839502  7026 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:50.839918  7026 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:50.840533  7026 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:50.809984  6925 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:50.847711  6959 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:50.899806  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:50.956641  7023 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:50.969440  6699 raft_consensus.cc:359] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:50.970188  6699 raft_consensus.cc:385] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:50.970466  6699 raft_consensus.cc:740] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:50.973133  6699 consensus_queue.cc:260] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:50.978048  6959 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:50.985818  6699 ts_tablet_manager.cc:1434] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.277s	user 0.014s	sys 0.010s
I20260709 12:16:51.001092  6698 tablet_bootstrap.cc:654] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:50.987073  7021 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:51.003476  6700 tablet_bootstrap.cc:654] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:51.002820  6883 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:16:51.006829  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:51.007424  6509 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:51.012148  6622 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:16:51.013248  6622 leader_election.cc:343] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:51.013686  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:51.014498  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:51.014806  6887 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:51.015074  6581 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:51.015290  6887 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.017856  6779 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:51.018771  6779 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:51.019392  7008 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:51.020805  7011 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:51.032608  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:51.033541  6651 raft_consensus.cc:2393] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 1: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:16:51.032935  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:51.039570  6581 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:51.041855  6887 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:16:51.044991  7032 log.cc:927] Time spent T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:16:51.048243  6959 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:51.052615  7023 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:51.053107  6699 tablet_bootstrap.cc:492] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
W20260709 12:16:51.054968  7035 log.cc:927] Time spent T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:16:51.051334  6925 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.058060  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:51.077391  6699 tablet_bootstrap.cc:654] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:51.078171  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 52406.8us
W20260709 12:16:51.078999  6622 leader_election.cc:343] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:51.077581  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:51.084771  6925 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:51.085007  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:51.080814  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:51.085923  6651 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:51.086261  6581 raft_consensus.cc:2393] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 1: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:16:51.086094  7026 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.087133  6652 raft_consensus.cc:2393] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 1: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:16:51.087395  6654 raft_consensus.cc:2393] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 1: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:16:51.087486  6473 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:51.094197  6475 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:51.100183  7016 consensus_queue.cc:1048] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:16:51.099227  7011 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:51.108600  6548 leader_election.cc:343] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:51.114632  6622 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:51.126672  7026 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:51.119455  6509 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:51.123373  6779 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:51.138360  6699 tablet_bootstrap.cc:492] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:51.123021  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:51.128295  6999 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:51.127615  7000 consensus_queue.cc:1048] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:16:51.138996  6699 ts_tablet_manager.cc:1403] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.086s	user 0.013s	sys 0.003s
I20260709 12:16:51.146915  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:51.148264  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:16:51.149127  7034 log.cc:927] Time spent T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 12:16:51.149322  7007 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:51.151031  6779 raft_consensus.cc:697] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:16:51.152004  6779 consensus_queue.cc:237] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.133425  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:16:51.157752  6622 leader_election.cc:343] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:51.161643  6699 raft_consensus.cc:359] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.166496  6699 raft_consensus.cc:385] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:51.166858  6699 raft_consensus.cc:740] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:51.164479  6584 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:51.172016  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:51.178141  6550 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:16:51.180945  6548 leader_election.cc:343] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:51.181319  6548 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:51.182009  6699 consensus_queue.cc:260] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.185429  7003 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:51.185997  7003 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:51.186470  7003 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:51.216300  6699 ts_tablet_manager.cc:1434] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.072s	user 0.008s	sys 0.000s
I20260709 12:16:51.222136  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:51.222769  7002 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:51.222932  6578 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:51.223485  6455 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:51.230741  6698 tablet_bootstrap.cc:492] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:51.231392  6698 ts_tablet_manager.cc:1403] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.418s	user 0.004s	sys 0.034s
I20260709 12:16:51.228600  6897 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:51.276216  6699 tablet_bootstrap.cc:492] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:51.223276  7002 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.304945  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:51.306942  6579 raft_consensus.cc:2393] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 1: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:16:51.307636  7025 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:51.308108  7025 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:51.318112  7011 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:51.308511  7025 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:51.355499  7025 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.386976  7025 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:51.397626  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:51.398962  6698 raft_consensus.cc:359] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.399718  6698 raft_consensus.cc:385] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:51.400023  6698 raft_consensus.cc:740] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:51.405786  6500 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:51.405471  6698 consensus_queue.cc:260] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.411500  6578 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:51.414580  6698 ts_tablet_manager.cc:1434] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.183s	user 0.022s	sys 0.048s
I20260709 12:16:51.426643  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:51.411469  7003 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.442265  6999 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:51.442889  6999 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:16:51.443347  7042 log.cc:927] Time spent T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.320s	user 0.004s	sys 0.000s
I20260709 12:16:51.446587  6700 tablet_bootstrap.cc:492] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:51.447240  6700 ts_tablet_manager.cc:1403] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.692s	user 0.027s	sys 0.020s
I20260709 12:16:51.449409  7002 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:51.452955  7003 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:51.454730  6698 tablet_bootstrap.cc:492] T d6029ffeecb94e4da196c61c032b862b P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:51.456261  7025 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:51.456377  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:51.457731  6959 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:51.458461  6959 raft_consensus.cc:697] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:16:51.461154  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 52228.2us
W20260709 12:16:51.467967  6473 leader_election.cc:343] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:51.459561  6959 consensus_queue.cc:237] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.482715  7025 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:51.486239  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:51.490257  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:51.490948  6699 tablet_bootstrap.cc:654] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:51.491235  6652 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 1: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:16:51.476722  6999 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:16:51.499784  6622 leader_election.cc:343] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:51.509546  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:51.494323  7025 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.513306  6655 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 0.
I20260709 12:16:51.514164  7025 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:51.527278  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:51.527961  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:51.528256  6575 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 1: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:16:51.534502  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:51.533203  6700 raft_consensus.cc:359] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.535334  6577 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:51.535517  6700 raft_consensus.cc:385] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:51.536062  6700 raft_consensus.cc:740] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:51.537230  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:51.537137  6700 consensus_queue.cc:260] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.546533  6700 ts_tablet_manager.cc:1434] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.099s	user 0.010s	sys 0.011s
I20260709 12:16:51.548435  6700 tablet_bootstrap.cc:492] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:51.551551  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:16:51.555410  6622 leader_election.cc:343] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:51.562038  6698 tablet_bootstrap.cc:654] T d6029ffeecb94e4da196c61c032b862b P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:51.559680  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:51.567543  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:51.567785  6475 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:16:51.575552  7057 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:51.576576  7007 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:51.577011  7007 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:51.577435  7007 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:51.587195  7056 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:51.587721  7056 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:16:51.588045  6548 leader_election.cc:343] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:51.588671  6548 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:51.589495  6897 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:51.588167  7056 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:51.593091  6622 leader_election.cc:343] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:51.592500  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:51.606861  6406 catalog_manager.cc:5697] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:51.610929  6406 catalog_manager.cc:5697] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:51.640517  6700 tablet_bootstrap.cc:654] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:51.640455  7057 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:51.637159  7007 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.646413  6698 tablet_bootstrap.cc:492] T d6029ffeecb94e4da196c61c032b862b P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:51.647044  6698 ts_tablet_manager.cc:1403] T d6029ffeecb94e4da196c61c032b862b P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.192s	user 0.020s	sys 0.001s
I20260709 12:16:51.647275  6504 raft_consensus.cc:2393] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 1: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:16:51.646440  6699 tablet_bootstrap.cc:492] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:51.649890  6699 ts_tablet_manager.cc:1403] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.374s	user 0.011s	sys 0.012s
I20260709 12:16:51.665642  6698 raft_consensus.cc:359] T d6029ffeecb94e4da196c61c032b862b P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.666957  6698 raft_consensus.cc:385] T d6029ffeecb94e4da196c61c032b862b P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:51.667552  6698 raft_consensus.cc:740] T d6029ffeecb94e4da196c61c032b862b P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:51.669360  6698 consensus_queue.cc:260] T d6029ffeecb94e4da196c61c032b862b P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.670455  6622 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:51.662451  7056 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.677656  6925 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:51.679391  6699 raft_consensus.cc:359] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.680444  6699 raft_consensus.cc:385] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:51.680807  6699 raft_consensus.cc:740] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:51.681645  6699 consensus_queue.cc:260] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.683023  7007 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:51.686808  6700 tablet_bootstrap.cc:492] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:51.687302  6700 ts_tablet_manager.cc:1403] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.139s	user 0.010s	sys 0.022s
I20260709 12:16:51.686568  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:51.687237  6699 ts_tablet_manager.cc:1434] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.001s
I20260709 12:16:51.688170  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:51.689381  6699 tablet_bootstrap.cc:492] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
W20260709 12:16:51.694195  6473 leader_election.cc:343] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:51.698899  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:16:51.700559  6622 leader_election.cc:343] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:51.701184  6925 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:51.701529  7056 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:51.702175  6698 ts_tablet_manager.cc:1434] T d6029ffeecb94e4da196c61c032b862b P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.055s	user 0.010s	sys 0.000s
I20260709 12:16:51.697026  6700 raft_consensus.cc:359] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.702646  6700 raft_consensus.cc:385] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:51.703042  6698 tablet_bootstrap.cc:492] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:51.703192  6700 raft_consensus.cc:740] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:51.704041  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:51.704130  6700 consensus_queue.cc:260] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.705772  6579 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:51.714988  6656 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:16:51.716315  6700 ts_tablet_manager.cc:1434] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.028s	user 0.004s	sys 0.004s
I20260709 12:16:51.716854  6475 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:16:51.717334  6700 tablet_bootstrap.cc:492] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:51.718052  7007 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:51.718772  7007 raft_consensus.cc:697] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:16:51.719916  7007 consensus_queue.cc:237] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.721660  6699 tablet_bootstrap.cc:654] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:51.730172  6579 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:51.731729  6700 tablet_bootstrap.cc:654] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:51.731830  6999 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:51.737452  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:51.740306  6698 tablet_bootstrap.cc:654] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:51.738185  7056 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:51.742622  7056 raft_consensus.cc:697] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:16:51.743559  7056 consensus_queue.cc:237] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.747159  6700 tablet_bootstrap.cc:492] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:51.747768  6700 ts_tablet_manager.cc:1403] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.031s	user 0.009s	sys 0.005s
I20260709 12:16:51.750202  6883 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:51.750718  6883 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.751003  6700 raft_consensus.cc:359] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.752527  6700 raft_consensus.cc:385] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:51.753089  6883 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:51.753160  6700 raft_consensus.cc:740] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:51.755074  6700 consensus_queue.cc:260] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.755975  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:16:51.757655  6475 leader_election.cc:343] T d6029ffeecb94e4da196c61c032b862b P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:51.759413  6700 ts_tablet_manager.cc:1434] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.011s	user 0.001s	sys 0.005s
I20260709 12:16:51.760386  6700 tablet_bootstrap.cc:492] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:51.770066  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:16:51.786592  6473 leader_election.cc:343] T d6029ffeecb94e4da196c61c032b862b P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:51.787065  6473 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:51.788450  7008 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:51.789701  6698 tablet_bootstrap.cc:492] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:51.790153  6698 ts_tablet_manager.cc:1403] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.087s	user 0.013s	sys 0.009s
I20260709 12:16:51.790560  6699 tablet_bootstrap.cc:492] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:51.790572  6700 tablet_bootstrap.cc:654] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:51.791091  6699 ts_tablet_manager.cc:1403] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.102s	user 0.016s	sys 0.005s
I20260709 12:16:51.797827  6699 raft_consensus.cc:359] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.798492  6699 raft_consensus.cc:385] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:51.798758  6699 raft_consensus.cc:740] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:51.799412  6699 consensus_queue.cc:260] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.801968  6699 ts_tablet_manager.cc:1434] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:16:51.802798  6699 tablet_bootstrap.cc:492] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:51.820487  6698 raft_consensus.cc:359] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.821424  6698 raft_consensus.cc:385] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:51.821718  6698 raft_consensus.cc:740] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:51.825698  6698 consensus_queue.cc:260] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.826454  6699 tablet_bootstrap.cc:654] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:51.822973  6404 catalog_manager.cc:5697] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 0 to 1, leader changed from <none> to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:51.843319  6700 tablet_bootstrap.cc:492] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:51.843751  6700 ts_tablet_manager.cc:1403] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.084s	user 0.009s	sys 0.013s
I20260709 12:16:51.846997  6700 raft_consensus.cc:359] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.847729  6700 raft_consensus.cc:385] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:51.848023  6700 raft_consensus.cc:740] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:51.845446  6698 ts_tablet_manager.cc:1434] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.055s	user 0.004s	sys 0.004s
I20260709 12:16:51.848867  6700 consensus_queue.cc:260] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.849689  6699 tablet_bootstrap.cc:492] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:51.850175  6699 ts_tablet_manager.cc:1403] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.048s	user 0.006s	sys 0.011s
I20260709 12:16:51.850306  6698 tablet_bootstrap.cc:492] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:51.851643  6700 ts_tablet_manager.cc:1434] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:16:51.852702  6700 tablet_bootstrap.cc:492] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:51.857782  6698 tablet_bootstrap.cc:654] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:51.859094  6700 tablet_bootstrap.cc:654] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:51.859138  6699 raft_consensus.cc:359] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.859967  6699 raft_consensus.cc:385] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:51.860276  6699 raft_consensus.cc:740] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:51.861063  6699 consensus_queue.cc:260] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.864696  6699 ts_tablet_manager.cc:1434] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.001s
I20260709 12:16:51.865638  6699 tablet_bootstrap.cc:492] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:51.859252  6407 catalog_manager.cc:5697] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:51.873993  6699 tablet_bootstrap.cc:654] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:51.878225  7051 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:51.879267  7051 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.883286  7051 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:51.883538  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:16:51.886474  6473 leader_election.cc:343] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:51.883585  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:16:51.893484  6475 leader_election.cc:343] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:51.893936  6475 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:51.894328  7021 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:51.894964  7021 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.896667  7051 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:51.901129  6699 tablet_bootstrap.cc:492] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:51.902014  6699 ts_tablet_manager.cc:1403] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.037s	user 0.011s	sys 0.007s
I20260709 12:16:51.902046  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:51.903264  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:16:51.905684  6550 leader_election.cc:343] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:51.906692  6548 leader_election.cc:343] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:51.906579  7056 consensus_queue.cc:1048] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:51.907160  6548 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:51.907963  7055 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:51.904266  7021 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:51.919401  6699 raft_consensus.cc:359] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.920284  6699 raft_consensus.cc:385] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:51.920636  6699 raft_consensus.cc:740] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:51.922348  7008 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:51.922904  7008 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.930119  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:51.932456  7008 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:51.937698  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:51.939210  6473 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:51.940850  6999 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:51.941325  6999 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.943264  6999 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:51.943924  6700 tablet_bootstrap.cc:492] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:51.945143  6700 ts_tablet_manager.cc:1403] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.093s	user 0.006s	sys 0.008s
I20260709 12:16:51.946625  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:51.948863  7052 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:51.951578  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:51.959789  6698 tablet_bootstrap.cc:492] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:51.961509  7057 consensus_queue.cc:1048] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:51.962682  6698 ts_tablet_manager.cc:1403] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.113s	user 0.032s	sys 0.052s
I20260709 12:16:51.964280  6699 consensus_queue.cc:260] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.961661  6700 raft_consensus.cc:359] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.965571  6700 raft_consensus.cc:385] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:51.965888  6700 raft_consensus.cc:740] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:51.965906  6622 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:51.966812  6700 consensus_queue.cc:260] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.969305  6700 ts_tablet_manager.cc:1434] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 12:16:51.969017  6698 raft_consensus.cc:359] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.969863  6698 raft_consensus.cc:385] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:51.970227  6698 raft_consensus.cc:740] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:51.971091  6698 consensus_queue.cc:260] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.971486  7056 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:51.973816  6698 ts_tablet_manager.cc:1434] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 12:16:51.965255  6999 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:51.976182  6999 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:51.978545  6999 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:51.981613  6698 tablet_bootstrap.cc:492] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:51.980623  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:51.982435  6504 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:51.984083  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:51.986599  6699 ts_tablet_manager.cc:1434] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.084s	user 0.003s	sys 0.003s
I20260709 12:16:51.987640  6699 tablet_bootstrap.cc:492] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:51.989107  6698 tablet_bootstrap.cc:654] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:51.995283  6998 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:51.995818  6998 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:51.996183  6998 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:51.998121  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:51.999470  6577 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:52.000849  6700 tablet_bootstrap.cc:492] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:52.009536  7015 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:52.010253  7015 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:52.013072  7015 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:52.013980  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:52.014478  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:52.014894  6656 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:52.015115  6504 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:52.016505  6548 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:16:52.017901  7015 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:52.018280  7015 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:52.018915  7015 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:52.034943  6700 tablet_bootstrap.cc:654] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:52.038645  6698 tablet_bootstrap.cc:492] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:52.039194  6698 ts_tablet_manager.cc:1403] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.058s	user 0.015s	sys 0.025s
I20260709 12:16:52.039848  6699 tablet_bootstrap.cc:654] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:52.043324  7015 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:52.045490  6698 raft_consensus.cc:359] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:52.046144  6698 raft_consensus.cc:385] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:52.046458  6698 raft_consensus.cc:740] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:52.046563  7015 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:52.047119  6698 consensus_queue.cc:260] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:52.061331  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:52.062125  6656 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:52.051620  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:52.065328  6698 ts_tablet_manager.cc:1434] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.026s	user 0.004s	sys 0.003s
I20260709 12:16:52.067121  6698 tablet_bootstrap.cc:492] T 5b17fd01574f4601b33dd37e7c78bc8f P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:52.064980  7008 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:52.077039  7008 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:52.066493  6504 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:52.081919  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:52.083133  6579 raft_consensus.cc:2393] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 1: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:16:52.086598  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:52.087802  6650 raft_consensus.cc:2393] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 1: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:16:52.080104  7008 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:52.094534  6475 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:52.099143  7008 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:52.105911  6656 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:16:52.107113  6550 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:52.113247  6897 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:52.113749  6897 raft_consensus.cc:697] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:16:52.114548  6897 consensus_queue.cc:237] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:52.118189  6998 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:52.122473  6699 tablet_bootstrap.cc:492] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:52.124361  6699 ts_tablet_manager.cc:1403] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.137s	user 0.023s	sys 0.034s
I20260709 12:16:52.125877  6698 tablet_bootstrap.cc:654] T 5b17fd01574f4601b33dd37e7c78bc8f P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:52.103825  6504 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:16:52.123411  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:52.131727  6509 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:52.132963  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:52.132092  7008 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:52.133601  6579 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:52.134189  6507 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:52.141417  6700 tablet_bootstrap.cc:492] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:52.142156  6700 ts_tablet_manager.cc:1403] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.142s	user 0.019s	sys 0.027s
I20260709 12:16:52.126485  6998 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:52.155193  6699 raft_consensus.cc:359] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:52.156773  6699 raft_consensus.cc:385] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:52.157171  6699 raft_consensus.cc:740] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:52.181321  6699 consensus_queue.cc:260] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:52.187845  6507 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:52.189735  6699 ts_tablet_manager.cc:1434] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.065s	user 0.006s	sys 0.002s
I20260709 12:16:52.189157  6404 catalog_manager.cc:5697] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:52.176669  7084 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:52.191861  7084 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:52.192660  6700 raft_consensus.cc:359] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:52.193310  6700 raft_consensus.cc:385] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:52.193609  6700 raft_consensus.cc:740] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:52.194219  6700 consensus_queue.cc:260] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:52.195437  6699 tablet_bootstrap.cc:492] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:52.205345  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:52.209241  6508 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:52.217273  6548 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:16:52.218565  7023 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:52.218955  7023 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:52.219291  7023 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:52.228554  6700 ts_tablet_manager.cc:1434] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.086s	user 0.004s	sys 0.004s
I20260709 12:16:52.229377  6700 tablet_bootstrap.cc:492] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:52.230786  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:52.241634  6959 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:52.241650  6652 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:52.258352  6698 tablet_bootstrap.cc:492] T 5b17fd01574f4601b33dd37e7c78bc8f P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:52.258904  6698 ts_tablet_manager.cc:1403] T 5b17fd01574f4601b33dd37e7c78bc8f P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.192s	user 0.027s	sys 0.015s
I20260709 12:16:52.244378  6959 raft_consensus.cc:697] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:16:52.244020  7084 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:52.259475  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:52.276994  6999 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:52.256383  6579 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
W20260709 12:16:52.280548  6455 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:52.281934  6999 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:52.263893  6959 consensus_queue.cc:237] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:52.295562  6650 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:52.306665  6698 raft_consensus.cc:359] T 5b17fd01574f4601b33dd37e7c78bc8f P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:52.307677  6698 raft_consensus.cc:385] T 5b17fd01574f4601b33dd37e7c78bc8f P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:52.308028  6698 raft_consensus.cc:740] T 5b17fd01574f4601b33dd37e7c78bc8f P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:52.310235  7023 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:52.313552  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:52.314144  6650 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:52.319326  6700 tablet_bootstrap.cc:654] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:52.320916  6622 consensus_peers.cc:597] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:52.326432  6699 tablet_bootstrap.cc:654] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:52.329509  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:16:52.395512  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 74852.4us
I20260709 12:16:52.397198  6509 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:52.319727  6698 consensus_queue.cc:260] T 5b17fd01574f4601b33dd37e7c78bc8f P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:52.408123  6509 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:16:52.351671  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:52.412678  6698 ts_tablet_manager.cc:1434] T 5b17fd01574f4601b33dd37e7c78bc8f P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.153s	user 0.007s	sys 0.010s
I20260709 12:16:52.318252  6999 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:52.312249  7023 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:52.354043  6650 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:16:52.331640  6998 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.087s
I20260709 12:16:52.414400  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:52.432621  6548 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:16:52.433570  7023 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:16:52.433866  6622 leader_election.cc:343] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:52.419251  7015 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:52.435966  6883 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:52.440730  7023 raft_consensus.cc:697] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:16:52.449930  7015 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:52.462674  7023 consensus_queue.cc:237] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:52.532750  7015 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:52.538789  6883 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:52.545284  6698 tablet_bootstrap.cc:492] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:52.553362  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:52.562901  6581 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:52.594852  6883 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:52.606457  7015 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:52.607460  7015 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:52.618124  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:52.622915  6699 tablet_bootstrap.cc:492] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:52.623935  6699 ts_tablet_manager.cc:1403] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.429s	user 0.019s	sys 0.009s
I20260709 12:16:52.627939  6883 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:52.634831  7015 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:52.637267  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:52.638376  6654 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 1: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:16:52.657778  6622 leader_election.cc:343] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:52.658438  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:52.660064  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:52.662536  6700 tablet_bootstrap.cc:492] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:52.663470  6700 ts_tablet_manager.cc:1403] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.434s	user 0.019s	sys 0.017s
W20260709 12:16:52.669297  6550 leader_election.cc:343] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:52.676087  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:52.677342  6580 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 1: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:16:52.712152  7025 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:52.714538  6699 raft_consensus.cc:359] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:52.715538  6699 raft_consensus.cc:385] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:52.717499  6699 raft_consensus.cc:740] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:52.720480  6548 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:52.720721  6699 consensus_queue.cc:260] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:52.722839  7007 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:52.723543  7007 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:52.723884  6473 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:52.725178  7051 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:52.734782  7007 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:52.739529  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:16:52.741487  6475 leader_election.cc:343] T 5b17fd01574f4601b33dd37e7c78bc8f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:52.727240  6700 raft_consensus.cc:359] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:52.744725  6700 raft_consensus.cc:385] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:52.745074  6700 raft_consensus.cc:740] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:52.745949  6699 ts_tablet_manager.cc:1434] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.121s	user 0.003s	sys 0.005s
I20260709 12:16:52.745878  6700 consensus_queue.cc:260] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:52.749311  6699 tablet_bootstrap.cc:492] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:52.750260  7102 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:52.753156  6700 ts_tablet_manager.cc:1434] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.089s	user 0.006s	sys 0.002s
I20260709 12:16:52.754571  6700 tablet_bootstrap.cc:492] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:52.759925  6407 catalog_manager.cc:5697] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:52.762569  6883 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:52.759963  6698 tablet_bootstrap.cc:654] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:52.780117  7101 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:52.781529  6699 tablet_bootstrap.cc:654] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:52.784940  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:52.810858  7051 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:52.816610  6700 tablet_bootstrap.cc:654] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:52.825080  6548 leader_election.cc:343] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:52.825754  6548 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:52.816861  6404 catalog_manager.cc:5697] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:52.829836  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:16:52.841509  6473 leader_election.cc:343] T 5b17fd01574f4601b33dd37e7c78bc8f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:52.841982  6473 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:52.843214  7051 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:52.854393  7101 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:52.854615  7084 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:16:52.912261  6700 tablet_bootstrap.cc:492] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:52.912786  6700 ts_tablet_manager.cc:1403] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.159s	user 0.035s	sys 0.027s
I20260709 12:16:52.918340  6698 tablet_bootstrap.cc:492] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:52.919603  6698 ts_tablet_manager.cc:1403] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.375s	user 0.026s	sys 0.018s
I20260709 12:16:52.943423  7116 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:52.960316  6700 raft_consensus.cc:359] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:52.961056  6700 raft_consensus.cc:385] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:52.961246  6700 raft_consensus.cc:740] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:52.979987  7056 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:52.980827  7056 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:52.984299  6699 tablet_bootstrap.cc:492] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:52.985005  6699 ts_tablet_manager.cc:1403] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.236s	user 0.019s	sys 0.025s
I20260709 12:16:52.999650  7056 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:53.003993  6698 raft_consensus.cc:359] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.006327  6698 raft_consensus.cc:385] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.006680  6698 raft_consensus.cc:740] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.002080  6700 consensus_queue.cc:260] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.028431  6698 consensus_queue.cc:260] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.031106  6698 ts_tablet_manager.cc:1434] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.111s	user 0.006s	sys 0.000s
I20260709 12:16:53.028750  7057 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:53.033470  6700 ts_tablet_manager.cc:1434] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.120s	user 0.012s	sys 0.011s
I20260709 12:16:53.035503  6700 tablet_bootstrap.cc:492] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:53.039767  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:53.043432  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:53.041425  6698 tablet_bootstrap.cc:492] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:53.047824  7102 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:53.048780  7102 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.055483  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:16:53.057158  6548 leader_election.cc:343] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:53.058058  6622 leader_election.cc:343] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:53.060724  7102 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:53.044142  6581 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:53.082454  6700 tablet_bootstrap.cc:654] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.084728  7007 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:53.085500  7007 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.084007  6699 raft_consensus.cc:359] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.089345  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:53.094208  6655 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
I20260709 12:16:53.091580  6699 raft_consensus.cc:385] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.095409  6699 raft_consensus.cc:740] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.098758  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:53.100039  6550 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:53.137051  6699 consensus_queue.cc:260] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.091107  7007 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:53.146641  7102 raft_consensus.cc:2804] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:53.147073  7102 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:53.147403  7102 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.098740  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:53.158594  6699 ts_tablet_manager.cc:1434] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.173s	user 0.018s	sys 0.005s
I20260709 12:16:53.159137  7091 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:53.163504  7091 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.164160  7021 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:16:53.166385  7090 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:53.167029  7090 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.167949  7091 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:53.169178  7090 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:53.174757  6698 tablet_bootstrap.cc:654] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:53.147981  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 48030.2us
I20260709 12:16:53.177110  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:53.185182  7090 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:53.185400  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:53.185685  7090 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:53.186208  7090 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.186235  6582 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:53.191411  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:53.192270  7057 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:53.192674  7057 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:53.194159  7057 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:53.200719  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:53.201496  6507 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:53.203506  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:53.203293  7057 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.204165  6508 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:53.207204  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:53.207942  6582 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:53.209208  7057 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:53.217057  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:53.181313  7102 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.217821  6508 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:53.217911  7090 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.221122  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:53.221732  6582 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:53.223402  6473 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:53.234386  7090 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:53.235236  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:53.238065  7097 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:53.236004  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:53.246326  6654 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.255517  6699 tablet_bootstrap.cc:492] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
W20260709 12:16:53.260119  6548 leader_election.cc:343] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:53.229141  7102 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:53.245883  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:53.244915  6508 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:16:53.281006  7023 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:16:53.285523  6698 tablet_bootstrap.cc:492] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:53.286188  6698 ts_tablet_manager.cc:1403] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.254s	user 0.022s	sys 0.000s
I20260709 12:16:53.275728  7090 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:53.294853  7090 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:53.295382  7090 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:53.300776  6698 raft_consensus.cc:359] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.301559  6698 raft_consensus.cc:385] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.301862  6698 raft_consensus.cc:740] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.303062  6654 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:16:53.304243  6698 consensus_queue.cc:260] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.299937  6582 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:16:53.332262  6698 ts_tablet_manager.cc:1434] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.046s	user 0.009s	sys 0.000s
I20260709 12:16:53.333356  6698 tablet_bootstrap.cc:492] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:53.306664  6700 tablet_bootstrap.cc:492] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:53.311132  6699 tablet_bootstrap.cc:654] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.336690  6700 ts_tablet_manager.cc:1403] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.301s	user 0.021s	sys 0.029s
I20260709 12:16:53.316920  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:53.337472  6577 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.339278  6700 raft_consensus.cc:359] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.339959  6700 raft_consensus.cc:385] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.340217  6700 raft_consensus.cc:740] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.319860  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:53.318171  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:53.342576  7118 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:53.344633  6550 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:53.345494  7084 raft_consensus.cc:2804] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:53.346786  7118 raft_consensus.cc:697] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:16:53.347018  7084 raft_consensus.cc:697] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:16:53.348155  7118 consensus_queue.cc:237] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.351002  6700 consensus_queue.cc:260] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.348199  7084 consensus_queue.cc:237] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.354620  6700 ts_tablet_manager.cc:1434] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.001s
I20260709 12:16:53.356997  6698 tablet_bootstrap.cc:654] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.354637  7090 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.365474  7090 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:53.368834  6700 tablet_bootstrap.cc:492] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
W20260709 12:16:53.369099  6622 leader_election.cc:343] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:53.370270  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:53.370958  6501 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:53.374348  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:53.375025  6581 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:53.376015  6577 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:53.379925  7097 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:53.380467  7097 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.390202  6699 tablet_bootstrap.cc:492] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:53.390784  6699 ts_tablet_manager.cc:1403] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.144s	user 0.014s	sys 0.003s
I20260709 12:16:53.393775  7097 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:53.395129  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:53.395875  6582 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 0.
I20260709 12:16:53.396180  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:53.395613  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:53.397135  6473 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:16:53.398056  6475 leader_election.cc:343] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:53.402788  7090 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:53.401844  7097 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:53.403545  7097 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:53.403466  7090 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.404101  7097 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.406577  6507 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.405128  7091 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:53.408931  7091 raft_consensus.cc:697] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:16:53.411089  7091 consensus_queue.cc:237] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.422556  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:53.421185  7090 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:53.440531  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:53.441046  6699 raft_consensus.cc:359] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.444926  6699 raft_consensus.cc:385] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.445240  6699 raft_consensus.cc:740] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.445900  6699 consensus_queue.cc:260] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.447566  6699 ts_tablet_manager.cc:1434] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.056s	user 0.007s	sys 0.002s
I20260709 12:16:53.429462  6582 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:53.451100  6699 tablet_bootstrap.cc:492] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
W20260709 12:16:53.454825  6622 leader_election.cc:343] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:53.456053  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:53.459465  6700 tablet_bootstrap.cc:654] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.461680  7090 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:53.462215  7090 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:53.462597  7090 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.454784  6501 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:16:53.463402  6581 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:16:53.492111  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:53.519611  7118 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:53.520216  7118 raft_consensus.cc:697] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:16:53.521162  7118 consensus_queue.cc:237] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.526093  6406 catalog_manager.cc:5697] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:53.548213  7090 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:16:53.582091  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 97924.5us
I20260709 12:16:53.582149  7090 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:16:53.587198  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 76806.7us
W20260709 12:16:53.521092  6455 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:53.555738  7097 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.591141  7097 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:53.577310  6698 tablet_bootstrap.cc:492] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:53.594259  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:53.595031  6698 ts_tablet_manager.cc:1403] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.262s	user 0.023s	sys 0.017s
I20260709 12:16:53.595242  6582 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.598127  6698 raft_consensus.cc:359] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.598892  6698 raft_consensus.cc:385] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.599361  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:53.602754  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:16:53.608549  6475 leader_election.cc:343] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:53.609186  6582 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:16:53.610937  6473 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:53.611955  7051 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:53.612241  6700 tablet_bootstrap.cc:492] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:53.612519  7051 raft_consensus.cc:697] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:16:53.613050  6700 ts_tablet_manager.cc:1403] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.245s	user 0.008s	sys 0.009s
I20260709 12:16:53.613526  7051 consensus_queue.cc:237] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.615304  6699 tablet_bootstrap.cc:654] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.615904  6700 raft_consensus.cc:359] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.619696  6700 raft_consensus.cc:385] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.620028  6700 raft_consensus.cc:740] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.616117  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:53.623163  6575 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.625211  6698 raft_consensus.cc:740] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.626055  6698 consensus_queue.cc:260] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.621240  6700 consensus_queue.cc:260] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.629570  6700 ts_tablet_manager.cc:1434] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.003s
I20260709 12:16:53.630609  6700 tablet_bootstrap.cc:492] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
W20260709 12:16:53.617823  6622 leader_election.cc:343] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:53.632280  6575 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:53.633630  6698 ts_tablet_manager.cc:1434] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.002s
I20260709 12:16:53.637912  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:53.638916  7056 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:53.639554  7056 raft_consensus.cc:697] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:16:53.640486  7056 consensus_queue.cc:237] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.644529  6404 catalog_manager.cc:5697] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 0 to 1, leader changed from <none> to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:53.653010  6698 tablet_bootstrap.cc:492] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:53.666602  6700 tablet_bootstrap.cc:654] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.676607  7021 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:53.677402  7021 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.683631  6698 tablet_bootstrap.cc:654] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.685603  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:53.684422  7021 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:53.687086  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:53.687791  6654 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 0.
W20260709 12:16:53.690878  6548 leader_election.cc:343] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:53.693380  6550 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:53.697086  7102 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:53.697466  7102 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:53.697781  7102 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.709348  7102 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.712085  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:16:53.713706  6548 leader_election.cc:343] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:53.721662  7051 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:53.723209  7051 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.726682  7051 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:53.733888  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:53.735539  6406 catalog_manager.cc:5697] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:53.737783  6406 catalog_manager.cc:5697] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 0 to 2, leader changed from <none> to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 2 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:53.740157  6406 catalog_manager.cc:5697] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
W20260709 12:16:53.744190  6473 leader_election.cc:343] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:53.745901  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:53.748040  6700 tablet_bootstrap.cc:492] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:53.748939  6700 ts_tablet_manager.cc:1403] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.119s	user 0.017s	sys 0.035s
W20260709 12:16:53.749234  6475 leader_election.cc:343] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:53.749637  6475 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:53.749878  6406 catalog_manager.cc:5697] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 0 to 2, leader changed from <none> to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 2 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:53.754398  7051 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:53.758126  6699 tablet_bootstrap.cc:492] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:53.758630  6699 ts_tablet_manager.cc:1403] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.310s	user 0.018s	sys 0.025s
I20260709 12:16:53.764586  6700 raft_consensus.cc:359] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.765240  6700 raft_consensus.cc:385] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.765499  6700 raft_consensus.cc:740] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.766148  6700 consensus_queue.cc:260] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.766600  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:53.767742  6653 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.772482  6699 raft_consensus.cc:359] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.773407  6699 raft_consensus.cc:385] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.779691  6699 raft_consensus.cc:740] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.780762  6699 consensus_queue.cc:260] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.783641  6700 ts_tablet_manager.cc:1434] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.034s	user 0.005s	sys 0.002s
I20260709 12:16:53.788025  6699 ts_tablet_manager.cc:1434] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 12:16:53.767688  7102 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:53.799656  6700 tablet_bootstrap.cc:492] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:53.801525  6698 tablet_bootstrap.cc:492] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:53.802280  6698 ts_tablet_manager.cc:1403] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.149s	user 0.017s	sys 0.019s
I20260709 12:16:53.802289  6699 tablet_bootstrap.cc:492] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:53.806874  6575 raft_consensus.cc:1275] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:16:53.810024  7091 consensus_queue.cc:1048] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:53.811498  6700 tablet_bootstrap.cc:654] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.812203  6698 raft_consensus.cc:359] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.812959  6698 raft_consensus.cc:385] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.813243  6698 raft_consensus.cc:740] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.821033  6501 raft_consensus.cc:1275] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:16:53.822470  6698 consensus_queue.cc:260] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.826133  7091 consensus_queue.cc:1048] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:53.826144  6653 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:16:53.829033  6550 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:53.829424  6698 ts_tablet_manager.cc:1434] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 12:16:53.830339  7102 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:53.829509  6699 tablet_bootstrap.cc:654] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.831133  7102 raft_consensus.cc:697] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:16:53.833668  6698 tablet_bootstrap.cc:492] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:53.840670  7101 consensus_queue.cc:1048] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:53.832223  7102 consensus_queue.cc:237] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.850463  6698 tablet_bootstrap.cc:654] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.862531  6699 tablet_bootstrap.cc:492] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:53.865362  6699 ts_tablet_manager.cc:1403] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.065s	user 0.017s	sys 0.002s
I20260709 12:16:53.865828  6883 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:53.866436  6883 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.869379  6699 raft_consensus.cc:359] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.870318  6699 raft_consensus.cc:385] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.872634  6699 raft_consensus.cc:740] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.875528  6699 consensus_queue.cc:260] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.878520  6883 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:53.874897  6698 tablet_bootstrap.cc:492] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:53.879283  6698 ts_tablet_manager.cc:1403] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.049s	user 0.008s	sys 0.004s
I20260709 12:16:53.881260  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:53.881933  6575 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 0.
I20260709 12:16:53.881995  6698 raft_consensus.cc:359] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.883893  6698 raft_consensus.cc:385] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.884057  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:53.884258  6698 raft_consensus.cc:740] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.884730  6652 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 0.
I20260709 12:16:53.878669  6699 ts_tablet_manager.cc:1434] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 12:16:53.883250  6473 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:16:53.887247  6699 tablet_bootstrap.cc:492] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:53.887542  7097 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:53.885031  6698 consensus_queue.cc:260] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.888137  7097 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:53.888559  7097 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.894416  6699 tablet_bootstrap.cc:654] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.895638  6698 ts_tablet_manager.cc:1434] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.000s
I20260709 12:16:53.897037  6698 tablet_bootstrap.cc:492] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:53.900728  7097 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:16:53.903532  6548 consensus_peers.cc:597] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:53.905345  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:53.906039  6581 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.906420  6698 tablet_bootstrap.cc:654] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.920002  6581 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:16:53.922180  7056 consensus_queue.cc:1048] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:53.923542  6473 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:16:53.924046  6700 tablet_bootstrap.cc:492] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:53.924538  6700 ts_tablet_manager.cc:1403] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.132s	user 0.007s	sys 0.010s
I20260709 12:16:53.924453  6883 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:53.922237  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:53.925143  6883 raft_consensus.cc:697] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:16:53.925681  6653 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.926188  6883 consensus_queue.cc:237] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.922030  7097 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:53.939046  6700 raft_consensus.cc:359] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.939941  6700 raft_consensus.cc:385] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.940275  6700 raft_consensus.cc:740] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.943181  6699 tablet_bootstrap.cc:492] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:53.943698  6699 ts_tablet_manager.cc:1403] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.057s	user 0.012s	sys 0.012s
I20260709 12:16:53.946326  6699 raft_consensus.cc:359] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.946977  6699 raft_consensus.cc:385] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.947247  6699 raft_consensus.cc:740] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.949895  6698 tablet_bootstrap.cc:492] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:53.950356  6698 ts_tablet_manager.cc:1403] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.054s	user 0.016s	sys 0.007s
I20260709 12:16:53.953843  6698 raft_consensus.cc:359] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.954684  6698 raft_consensus.cc:385] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.952837  6653 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:16:53.955001  6698 raft_consensus.cc:740] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.947814  6699 consensus_queue.cc:260] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.961184  6699 ts_tablet_manager.cc:1434] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:16:53.962190  6699 tablet_bootstrap.cc:492] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:53.964893  6406 catalog_manager.cc:5697] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 0 to 1, leader changed from <none> to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:53.955641  6698 consensus_queue.cc:260] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.970544  6698 ts_tablet_manager.cc:1434] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.004s
I20260709 12:16:53.970463  6407 catalog_manager.cc:5697] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:53.972236  6698 tablet_bootstrap.cc:492] T e6167e4ddf454440bfc602ffbcd6da3e P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:53.973945  6700 consensus_queue.cc:260] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:53.976467  6700 ts_tablet_manager.cc:1434] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.041s	user 0.008s	sys 0.000s
I20260709 12:16:53.977851  6699 tablet_bootstrap.cc:654] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.977694  6700 tablet_bootstrap.cc:492] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:53.985661  6698 tablet_bootstrap.cc:654] T e6167e4ddf454440bfc602ffbcd6da3e P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.991063  6575 raft_consensus.cc:1275] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:16:53.995286  7091 consensus_queue.cc:1048] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:16:54.004217  6475 consensus_peers.cc:597] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:54.007788  6699 tablet_bootstrap.cc:492] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:54.008533  6699 ts_tablet_manager.cc:1403] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.047s	user 0.014s	sys 0.000s
I20260709 12:16:54.011696  6699 raft_consensus.cc:359] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:54.012559  6699 raft_consensus.cc:385] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.012930  6699 raft_consensus.cc:740] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.013581  6699 consensus_queue.cc:260] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:54.016134  6699 ts_tablet_manager.cc:1434] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:16:54.018008  6699 tablet_bootstrap.cc:492] T f2592253d3214104865730480e057d09 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:54.007731  6700 tablet_bootstrap.cc:654] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.030622  6504 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.040141  7057 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.040921  7057 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:54.048456  7057 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:54.060634  6698 tablet_bootstrap.cc:492] T e6167e4ddf454440bfc602ffbcd6da3e P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:54.065474  6500 raft_consensus.cc:1275] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:16:54.080077  6698 ts_tablet_manager.cc:1403] T e6167e4ddf454440bfc602ffbcd6da3e P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.108s	user 0.019s	sys 0.014s
I20260709 12:16:54.083748  7056 consensus_queue.cc:1048] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:54.072662  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:54.072984  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:54.063598  6883 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.064745  6699 tablet_bootstrap.cc:654] T f2592253d3214104865730480e057d09 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.070024  7057 consensus_queue.cc:1048] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 12:16:54.086227  6883 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:16:54.086864  6622 leader_election.cc:343] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:54.069041  6700 tablet_bootstrap.cc:492] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:54.089264  6883 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:54.089434  6700 ts_tablet_manager.cc:1403] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.112s	user 0.013s	sys 0.001s
I20260709 12:16:54.090885  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:54.086026  6501 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:54.094528  6700 raft_consensus.cc:359] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:54.095394  6700 raft_consensus.cc:385] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.095709  6700 raft_consensus.cc:740] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.096448  6700 consensus_queue.cc:260] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:54.097482  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:54.099383  6700 ts_tablet_manager.cc:1434] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:16:54.100661  6473 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:54.108250  6700 tablet_bootstrap.cc:492] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:54.116302  7091 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.118304  7091 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:54.124184  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:16:54.125783  6622 leader_election.cc:343] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:54.123008  7091 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:54.136155  6698 raft_consensus.cc:359] T e6167e4ddf454440bfc602ffbcd6da3e P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:54.136986  6698 raft_consensus.cc:385] T e6167e4ddf454440bfc602ffbcd6da3e P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.137269  6698 raft_consensus.cc:740] T e6167e4ddf454440bfc602ffbcd6da3e P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.138592  6698 consensus_queue.cc:260] T e6167e4ddf454440bfc602ffbcd6da3e P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:54.140228  6698 ts_tablet_manager.cc:1434] T e6167e4ddf454440bfc602ffbcd6da3e P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.057s	user 0.005s	sys 0.000s
I20260709 12:16:54.141258  6698 tablet_bootstrap.cc:492] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:54.141882  6622 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:16:54.142906  7091 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:54.143289  7091 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:54.143661  7091 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.145628  7096 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.147220  7096 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:54.147974  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:54.148638  6501 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:54.150974  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:16:54.152815  7057 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:54.153286  7057 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:54.153692  7057 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.153777  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:54.152452  7051 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:54.159258  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:54.160368  6581 raft_consensus.cc:2393] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 1: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:16:54.161831  6473 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:54.162678  7051 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.152421  7096 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:54.185459  6698 tablet_bootstrap.cc:654] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.192811  7057 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:54.197965  7091 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:54.197865  7051 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:54.200117  7057 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:54.200300  7091 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:54.199239  7159 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:16:54.203727  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:54.204921  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:54.204368  6582 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.209162  6699 tablet_bootstrap.cc:492] T f2592253d3214104865730480e057d09 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:54.209674  6699 ts_tablet_manager.cc:1403] T f2592253d3214104865730480e057d09 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.192s	user 0.019s	sys 0.016s
W20260709 12:16:54.209059  6622 leader_election.cc:343] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:54.214778  6700 tablet_bootstrap.cc:654] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.212936  7159 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.216739  7159 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:54.216041  7091 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.221939  7091 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:54.221994  6699 raft_consensus.cc:359] T f2592253d3214104865730480e057d09 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:54.227118  6699 raft_consensus.cc:385] T f2592253d3214104865730480e057d09 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.227473  6699 raft_consensus.cc:740] T f2592253d3214104865730480e057d09 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.227771  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:54.228479  6501 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.228260  6699 consensus_queue.cc:260] T f2592253d3214104865730480e057d09 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:54.228742  7159 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:54.230137  6582 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:54.234448  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:54.235709  6508 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.238711  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:54.239830  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:54.239573  6577 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:54.244824  7051 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:54.250058  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:54.250720  6582 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:54.251570  6508 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:54.251945  6622 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:54.238479  7091 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:54.260903  7091 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:54.254942  6699 ts_tablet_manager.cc:1434] T f2592253d3214104865730480e057d09 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.045s	user 0.008s	sys 0.000s
I20260709 12:16:54.261552  7091 raft_consensus.cc:697] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:16:54.262301  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:54.262462  7091 consensus_queue.cc:237] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:54.261289  6501 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:54.262502  6699 tablet_bootstrap.cc:492] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4: Bootstrap starting.
I20260709 12:16:54.252501  7159 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:54.263197  6508 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:54.266000  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:54.268575  7057 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:54.269081  7057 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:54.269481  7057 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.277050  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:16:54.283727  6622 leader_election.cc:343] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:54.286487  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:16:54.283082  7159 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:16:54.303267  6548 consensus_peers.cc:597] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:54.290248  7056 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:54.309350  7056 raft_consensus.cc:697] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:16:54.311412  7056 consensus_queue.cc:237] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:54.301084  7159 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:54.302134  7057 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:54.317499  7159 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.329620  6698 tablet_bootstrap.cc:492] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:54.330227  6698 ts_tablet_manager.cc:1403] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.189s	user 0.013s	sys 0.004s
I20260709 12:16:54.347076  6698 raft_consensus.cc:359] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:54.347796  6698 raft_consensus.cc:385] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.348059  6698 raft_consensus.cc:740] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.351023  6700 tablet_bootstrap.cc:492] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:54.351464  6700 ts_tablet_manager.cc:1403] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.243s	user 0.023s	sys 0.008s
I20260709 12:16:54.354326  6698 consensus_queue.cc:260] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:54.365595  6699 tablet_bootstrap.cc:654] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.350114  7057 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:54.369398  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:54.376716  6407 catalog_manager.cc:5697] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
W20260709 12:16:54.384541  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 76869us
W20260709 12:16:54.385854  6622 leader_election.cc:343] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:54.389488  6698 ts_tablet_manager.cc:1434] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.059s	user 0.007s	sys 0.001s
I20260709 12:16:54.394799  6700 raft_consensus.cc:359] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:54.424974  6700 raft_consensus.cc:385] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.425283  6700 raft_consensus.cc:740] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.381294  6883 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.047s
I20260709 12:16:54.444096  6698 tablet_bootstrap.cc:492] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:54.455976  6700 consensus_queue.cc:260] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:54.433089  7051 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.464450  6700 ts_tablet_manager.cc:1434] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.113s	user 0.016s	sys 0.000s
W20260709 12:16:54.457448  7168 log.cc:927] Time spent T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
W20260709 12:16:54.499260  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 65820.4us
I20260709 12:16:54.466391  7051 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:16:54.544076  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 81674.5us
I20260709 12:16:54.548826  7159 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:54.574048  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:54.574784  6700 tablet_bootstrap.cc:492] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e: Bootstrap starting.
I20260709 12:16:54.574728  6577 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.578089  7159 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:54.621855  6699 tablet_bootstrap.cc:492] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4: No bootstrap required, opened a new log
I20260709 12:16:54.625175  6699 ts_tablet_manager.cc:1403] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4: Time spent bootstrapping tablet: real 0.363s	user 0.034s	sys 0.009s
I20260709 12:16:54.597626  6504 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.630838  6653 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.597630  7051 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:54.636885  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:54.639602  6577 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:54.644529  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:16:54.645782  6502 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.647924  7097 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.649544  7097 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:54.651763  7097 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:54.687489  6699 raft_consensus.cc:359] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:54.689801  6699 raft_consensus.cc:385] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.690677  6699 raft_consensus.cc:740] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.691740  6699 consensus_queue.cc:260] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:54.701372  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:54.693399  6502 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:54.729645  6699 ts_tablet_manager.cc:1434] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4: Time spent starting tablet: real 0.104s	user 0.004s	sys 0.004s
I20260709 12:16:54.760576  6700 tablet_bootstrap.cc:654] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.769590  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:16:54.771978  7159 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:54.772656  7159 raft_consensus.cc:697] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:16:54.773496  7159 consensus_queue.cc:237] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:54.775924  6698 tablet_bootstrap.cc:654] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.778151  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:54.778723  6581 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.785931  6475 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:54.783922  7174 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:16:54.793852  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:54.787972  7051 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:54.794625  6650 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 0.
I20260709 12:16:54.796885  7193 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.797308  7193 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:54.800104  7193 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:54.805802  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:54.826005  6581 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:16:54.837152  6700 tablet_bootstrap.cc:492] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e: No bootstrap required, opened a new log
I20260709 12:16:54.837661  6700 ts_tablet_manager.cc:1403] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent bootstrapping tablet: real 0.263s	user 0.028s	sys 0.018s
I20260709 12:16:54.851948  6475 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:54.839147  7196 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:54.858726  7101 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.862534  7101 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:54.863152  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:54.864979  7101 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:54.866818  7181 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:54.867494  7181 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:54.867395  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:54.867934  7181 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.875089  6700 raft_consensus.cc:359] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:54.877575  6700 raft_consensus.cc:385] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.877885  6700 raft_consensus.cc:740] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.879498  6579 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.909333  6883 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:54.913735  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:54.871591  7196 raft_consensus.cc:697] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:16:54.912154  6406 catalog_manager.cc:5697] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:54.931886  6700 consensus_queue.cc:260] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:16:54.938871  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 54823.9us
I20260709 12:16:54.951452  6700 ts_tablet_manager.cc:1434] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent starting tablet: real 0.113s	user 0.007s	sys 0.010s
I20260709 12:16:54.935689  7196 consensus_queue.cc:237] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:54.943133  6577 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 0.
I20260709 12:16:54.984694  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:16:54.985918  6503 raft_consensus.cc:2393] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 1: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:16:55.010404  6455 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:55.091018  6698 tablet_bootstrap.cc:492] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:55.091528  6698 ts_tablet_manager.cc:1403] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.658s	user 0.028s	sys 0.028s
W20260709 12:16:55.102133  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 100495us
I20260709 12:16:55.103137  6548 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:55.115945  7181 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:55.117463  6500 raft_consensus.cc:2393] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 1: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:16:55.118765  6698 raft_consensus.cc:359] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:55.119555  6698 raft_consensus.cc:385] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:55.119871  6698 raft_consensus.cc:740] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:55.117465  7102 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:55.121989  6698 consensus_queue.cc:260] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:55.123571  6698 ts_tablet_manager.cc:1434] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.000s
I20260709 12:16:55.124890  7181 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:55.147320  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:16:55.148005  6656 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:55.153504  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:55.154385  6580 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 0.
I20260709 12:16:55.156111  6698 tablet_bootstrap.cc:492] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65: Bootstrap starting.
I20260709 12:16:55.164707  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:16:55.166070  6580 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:55.166702  6656 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:16:55.175369  6475 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:55.176409  7184 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:55.176955  7184 raft_consensus.cc:697] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:16:55.177850  7184 consensus_queue.cc:237] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:55.199513  6698 tablet_bootstrap.cc:654] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:55.207433  6580 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:16:55.216678  7186 consensus_queue.cc:1048] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:55.219277  6622 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:16:55.222782  7186 raft_consensus.cc:2764] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:16:55.225880  7186 consensus_queue.cc:1048] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:55.233855  7118 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:16:55.269647  7190 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:55.271770  6698 tablet_bootstrap.cc:492] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65: No bootstrap required, opened a new log
I20260709 12:16:55.269590  6406 catalog_manager.cc:5697] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 0 to 1, leader changed from <none> to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:55.276728  6698 ts_tablet_manager.cc:1403] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65: Time spent bootstrapping tablet: real 0.116s	user 0.029s	sys 0.005s
I20260709 12:16:55.279539  6698 raft_consensus.cc:359] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:55.280223  6698 raft_consensus.cc:385] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:55.280519  6698 raft_consensus.cc:740] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Initialized, Role: FOLLOWER
I20260709 12:16:55.281116  6698 consensus_queue.cc:260] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:55.283277  6698 ts_tablet_manager.cc:1434] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:16:55.310984  7181 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:55.311480  7181 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:16:55.313926  7181 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:55.316021  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:16:55.319177  7196 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:55.321316  6406 catalog_manager.cc:5697] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:55.323621  6406 catalog_manager.cc:5697] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:55.328498  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:16:55.329993  6473 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:16:55.331969  7181 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:55.337041  6505 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:55.347512  7196 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:55.381487  7196 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:55.395565  7196 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:55.576833  7181 consensus_queue.cc:1048] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:55.587635  7184 consensus_queue.cc:1048] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:55.775323  6505 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:55.959323  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:55.994621  7227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:55.994717  7228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:56.001370  7230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:56.007154  5352 server_base.cc:1061] running on GCE node
I20260709 12:16:56.010365  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:56.010715  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:56.010962  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599416010939 us; error 0 us; skew 500 ppm
I20260709 12:16:56.011754  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:56.017647  5352 webserver.cc:533] Webserver started at http://127.5.58.4:35949/ using document root <none> and password file <none>
I20260709 12:16:56.018445  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:56.018771  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:56.019287  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:56.020948  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-3-root/instance:
uuid: "20a202c76244485fbc8f1610473db8b9"
format_stamp: "Formatted at 2026-07-09 12:16:56 on dist-test-slave-xs2d"
I20260709 12:16:56.026895  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:16:56.035400  7235 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:56.037530  5352 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.002s	sys 0.001s
I20260709 12:16:56.037946  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-3-root
uuid: "20a202c76244485fbc8f1610473db8b9"
format_stamp: "Formatted at 2026-07-09 12:16:56 on dist-test-slave-xs2d"
I20260709 12:16:56.038357  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:56.085536  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:56.087548  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:56.089053  5352 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:56.093118  5352 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:56.093459  5352 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:56.093832  5352 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:56.094115  5352 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:56.167829  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.4:44857
I20260709 12:16:56.167904  7298 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.4:44857 every 8 connection(s)
I20260709 12:16:56.173426  5352 auto_leader_rebalancer-test.cc:439] add a tserver: 20a202c76244485fbc8f1610473db8b9
I20260709 12:16:56.190658  7299 heartbeater.cc:344] Connected to a master server at 127.5.58.62:36381
I20260709 12:16:56.191152  7299 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:56.192163  7299 heartbeater.cc:507] Master 127.5.58.62:36381 requested a full tablet report, sending...
I20260709 12:16:56.194648  6404 ts_manager.cc:194] Registered new tserver with Master: 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:16:56.197086  6404 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46184
I20260709 12:16:57.173442  6656 consensus_queue.cc:237] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:16:57.180136  6506 raft_consensus.cc:1275] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:57.180176  6580 raft_consensus.cc:1275] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:57.182053  7308 consensus_queue.cc:1048] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:57.183080  7194 consensus_queue.cc:1048] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:57.193482  7169 raft_consensus.cc:2955] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:16:57.200169  7299 heartbeater.cc:499] Master 127.5.58.62:36381 was elected leader, sending a full tablet report...
W20260709 12:16:57.207873  6623 consensus_peers.cc:597] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e07e97c48184f8e99a4fc5a580724a5. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:57.208124  6578 raft_consensus.cc:2955] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:16:57.218583  6500 raft_consensus.cc:2955] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:16:57.219897  6404 catalog_manager.cc:5697] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:57.247682  6506 consensus_queue.cc:237] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:16:57.257243  6656 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:57.258975  7304 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:57.264364  6578 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:57.268878  7192 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:57.270813  7192 raft_consensus.cc:2955] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
W20260709 12:16:57.276700  6474 consensus_peers.cc:597] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f2592253d3214104865730480e057d09. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:57.278987  6656 raft_consensus.cc:2955] T f2592253d3214104865730480e057d09 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:16:57.278973  6578 raft_consensus.cc:2955] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:16:57.302655  6404 catalog_manager.cc:5697] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:57.339557  6578 consensus_queue.cc:237] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:16:57.360766  6656 raft_consensus.cc:1275] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:57.362825  7177 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:57.364312  6500 raft_consensus.cc:1275] T d1778f67a2ba4fc2825ec2be5097d77a P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:57.365828  7021 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:16:57.378039  6549 consensus_peers.cc:597] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d1778f67a2ba4fc2825ec2be5097d77a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:57.378986  6506 raft_consensus.cc:2955] T d1778f67a2ba4fc2825ec2be5097d77a P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:16:57.378718  7329 raft_consensus.cc:2955] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:16:57.381350  6656 raft_consensus.cc:2955] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:16:57.394574  6505 consensus_queue.cc:237] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:16:57.403380  6404 catalog_manager.cc:5697] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 reported cstate change: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:16:57.410562  6656 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:57.411746  6580 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:16:57.413002  7304 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:16:57.413092  6474 consensus_peers.cc:597] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e416f9423f964ba5a29e536e8160c627. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:57.415004  7322 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:57.430073  7192 raft_consensus.cc:2955] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:16:57.432993  6651 raft_consensus.cc:2955] T e416f9423f964ba5a29e536e8160c627 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:16:57.439024  6578 raft_consensus.cc:2955] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:16:57.452437  6406 catalog_manager.cc:5697] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:57.718760  7334 ts_tablet_manager.cc:933] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:16:57.721130  7334 tablet_copy_client.cc:323] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.3:36107
I20260709 12:16:57.738629  7338 ts_tablet_manager.cc:933] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:16:57.746373  7338 tablet_copy_client.cc:323] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.2:45101
I20260709 12:16:57.746618  6668 tablet_copy_service.cc:140] P f2b06aebe97249a4877d798b8118ece4: Received BeginTabletCopySession request for tablet 3e07e97c48184f8e99a4fc5a580724a5 from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:49792)
I20260709 12:16:57.747326  6668 tablet_copy_service.cc:161] P f2b06aebe97249a4877d798b8118ece4: Beginning new tablet copy session on tablet 3e07e97c48184f8e99a4fc5a580724a5 from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:49792: session id = 20a202c76244485fbc8f1610473db8b9-3e07e97c48184f8e99a4fc5a580724a5
I20260709 12:16:57.761940  6668 tablet_copy_source_session.cc:215] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:57.762568  6594 tablet_copy_service.cc:140] P 496c7c3792cd4587b4dbeba435f4be1e: Received BeginTabletCopySession request for tablet d1778f67a2ba4fc2825ec2be5097d77a from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:54204)
I20260709 12:16:57.763216  6594 tablet_copy_service.cc:161] P 496c7c3792cd4587b4dbeba435f4be1e: Beginning new tablet copy session on tablet d1778f67a2ba4fc2825ec2be5097d77a from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:54204: session id = 20a202c76244485fbc8f1610473db8b9-d1778f67a2ba4fc2825ec2be5097d77a
I20260709 12:16:57.767737  7334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e07e97c48184f8e99a4fc5a580724a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:57.791659  7334 tablet_copy_client.cc:806] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:57.792603  7334 tablet_copy_client.cc:670] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:57.791915  6594 tablet_copy_source_session.cc:215] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:57.798209  7338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1778f67a2ba4fc2825ec2be5097d77a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:57.818249  7338 tablet_copy_client.cc:806] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:57.818989  7338 tablet_copy_client.cc:670] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:57.833436  7340 ts_tablet_manager.cc:933] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643)
I20260709 12:16:57.842803  7338 tablet_copy_client.cc:538] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:57.843199  7334 tablet_copy_client.cc:538] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:57.854974  7334 tablet_bootstrap.cc:492] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
I20260709 12:16:57.864190  7343 ts_tablet_manager.cc:933] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643)
I20260709 12:16:57.869336  7340 tablet_copy_client.cc:323] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.1:36643
W20260709 12:16:57.879889  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 71858.9us
I20260709 12:16:57.881752  7343 tablet_copy_client.cc:323] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.1:36643
I20260709 12:16:57.910497  7334 tablet_bootstrap.cc:492] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:57.911856  7334 tablet_bootstrap.cc:492] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:16:57.912698  7334 ts_tablet_manager.cc:1403] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 0.058s	user 0.029s	sys 0.000s
I20260709 12:16:57.919762  7338 tablet_bootstrap.cc:492] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
I20260709 12:16:57.921396  7334 raft_consensus.cc:359] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:16:57.931723  7334 raft_consensus.cc:740] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
I20260709 12:16:57.933174  7334 consensus_queue.cc:260] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:16:57.964941  7338 tablet_bootstrap.cc:492] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:57.967480  6519 tablet_copy_service.cc:140] P e7554c39679a41bbbca4659b72ab5c65: Received BeginTabletCopySession request for tablet f2592253d3214104865730480e057d09 from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:41304)
I20260709 12:16:57.968112  6518 tablet_copy_service.cc:140] P e7554c39679a41bbbca4659b72ab5c65: Received BeginTabletCopySession request for tablet e416f9423f964ba5a29e536e8160c627 from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:41304)
I20260709 12:16:57.977717  6518 tablet_copy_service.cc:161] P e7554c39679a41bbbca4659b72ab5c65: Beginning new tablet copy session on tablet e416f9423f964ba5a29e536e8160c627 from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:41304: session id = 20a202c76244485fbc8f1610473db8b9-e416f9423f964ba5a29e536e8160c627
I20260709 12:16:57.983757  6519 tablet_copy_service.cc:161] P e7554c39679a41bbbca4659b72ab5c65: Beginning new tablet copy session on tablet f2592253d3214104865730480e057d09 from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:41304: session id = 20a202c76244485fbc8f1610473db8b9-f2592253d3214104865730480e057d09
I20260709 12:16:57.989584  7338 tablet_bootstrap.cc:492] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:16:57.990480  7338 ts_tablet_manager.cc:1403] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 0.071s	user 0.024s	sys 0.010s
I20260709 12:16:57.997236  7338 raft_consensus.cc:359] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:16:58.005128  7338 raft_consensus.cc:740] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
I20260709 12:16:58.007562  7334 ts_tablet_manager.cc:1434] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.094s	user 0.027s	sys 0.047s
I20260709 12:16:58.026198  7338 consensus_queue.cc:260] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:16:58.036315  7338 ts_tablet_manager.cc:1434] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.044s	user 0.004s	sys 0.004s
I20260709 12:16:58.039081  6594 tablet_copy_service.cc:342] P 496c7c3792cd4587b4dbeba435f4be1e: Request end of tablet copy session 20a202c76244485fbc8f1610473db8b9-d1778f67a2ba4fc2825ec2be5097d77a received from {username='slave'} at 127.0.0.1:54204
I20260709 12:16:58.039637  6594 tablet_copy_service.cc:434] P 496c7c3792cd4587b4dbeba435f4be1e: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-d1778f67a2ba4fc2825ec2be5097d77a on tablet d1778f67a2ba4fc2825ec2be5097d77a with peer 20a202c76244485fbc8f1610473db8b9
I20260709 12:16:58.016383  6519 tablet_copy_source_session.cc:215] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:58.011869  6518 tablet_copy_source_session.cc:215] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:58.028648  6668 tablet_copy_service.cc:342] P f2b06aebe97249a4877d798b8118ece4: Request end of tablet copy session 20a202c76244485fbc8f1610473db8b9-3e07e97c48184f8e99a4fc5a580724a5 received from {username='slave'} at 127.0.0.1:49792
I20260709 12:16:58.045794  6668 tablet_copy_service.cc:434] P f2b06aebe97249a4877d798b8118ece4: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-3e07e97c48184f8e99a4fc5a580724a5 on tablet 3e07e97c48184f8e99a4fc5a580724a5 with peer 20a202c76244485fbc8f1610473db8b9
I20260709 12:16:58.049105  7343 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e416f9423f964ba5a29e536e8160c627. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:58.052845  7340 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2592253d3214104865730480e057d09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:58.075605  7340 tablet_copy_client.cc:806] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:58.076215  7343 tablet_copy_client.cc:806] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:58.076640  7340 tablet_copy_client.cc:670] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:58.077096  7343 tablet_copy_client.cc:670] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:58.081632  7340 tablet_copy_client.cc:538] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:58.081578  7343 tablet_copy_client.cc:538] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:58.090153  7340 tablet_bootstrap.cc:492] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
I20260709 12:16:58.090601  7343 tablet_bootstrap.cc:492] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
I20260709 12:16:58.106489  7340 tablet_bootstrap.cc:492] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:58.107295  7340 tablet_bootstrap.cc:492] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:16:58.107919  7340 ts_tablet_manager.cc:1403] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 0.018s	user 0.018s	sys 0.000s
I20260709 12:16:58.110342  7340 raft_consensus.cc:359] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:16:58.111085  7340 raft_consensus.cc:740] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
I20260709 12:16:58.111649  7340 consensus_queue.cc:260] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:16:58.112455  7343 tablet_bootstrap.cc:492] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:58.115698  7343 tablet_bootstrap.cc:492] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:16:58.114591  7340 ts_tablet_manager.cc:1434] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:16:58.116616  7343 ts_tablet_manager.cc:1403] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.015s
I20260709 12:16:58.119267  6518 tablet_copy_service.cc:342] P e7554c39679a41bbbca4659b72ab5c65: Request end of tablet copy session 20a202c76244485fbc8f1610473db8b9-f2592253d3214104865730480e057d09 received from {username='slave'} at 127.0.0.1:41304
I20260709 12:16:58.119805  6518 tablet_copy_service.cc:434] P e7554c39679a41bbbca4659b72ab5c65: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-f2592253d3214104865730480e057d09 on tablet f2592253d3214104865730480e057d09 with peer 20a202c76244485fbc8f1610473db8b9
I20260709 12:16:58.122092  7343 raft_consensus.cc:359] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:16:58.123118  7343 raft_consensus.cc:740] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
I20260709 12:16:58.123895  7343 consensus_queue.cc:260] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:16:58.128101  7343 ts_tablet_manager.cc:1434] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.002s
I20260709 12:16:58.129937  6518 tablet_copy_service.cc:342] P e7554c39679a41bbbca4659b72ab5c65: Request end of tablet copy session 20a202c76244485fbc8f1610473db8b9-e416f9423f964ba5a29e536e8160c627 received from {username='slave'} at 127.0.0.1:41304
I20260709 12:16:58.130352  6518 tablet_copy_service.cc:434] P e7554c39679a41bbbca4659b72ab5c65: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-e416f9423f964ba5a29e536e8160c627 on tablet e416f9423f964ba5a29e536e8160c627 with peer 20a202c76244485fbc8f1610473db8b9
I20260709 12:16:58.145159  7273 raft_consensus.cc:1217] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:58.262946  7273 raft_consensus.cc:1217] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:58.325647  7273 raft_consensus.cc:1217] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:58.442925  7273 raft_consensus.cc:1217] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:16:58.618239  7330 raft_consensus.cc:1064] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:16:58.621687  7330 consensus_queue.cc:237] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:16:58.629299  7273 raft_consensus.cc:1275] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:58.630971  6500 raft_consensus.cc:1275] T d1778f67a2ba4fc2825ec2be5097d77a P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:58.631331  6651 raft_consensus.cc:1275] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:58.632156  7349 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:58.633260  7329 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:58.634083  7330 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:58.665578  7329 raft_consensus.cc:2955] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:16:58.674229  7273 raft_consensus.cc:2955] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:16:58.682035  6651 raft_consensus.cc:2955] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:16:58.687760  6500 raft_consensus.cc:2955] T d1778f67a2ba4fc2825ec2be5097d77a P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:16:58.715320  6406 catalog_manager.cc:5697] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:58.722321  7313 raft_consensus.cc:1064] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:16:58.727660  7313 consensus_queue.cc:237] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:16:58.738688  7273 raft_consensus.cc:1275] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:58.739118  6578 raft_consensus.cc:1275] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:58.741344  7353 consensus_queue.cc:1048] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:58.745694  6505 raft_consensus.cc:1275] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:58.753739  7313 consensus_queue.cc:1048] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:58.755736  7353 consensus_queue.cc:1048] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:58.770421  7194 raft_consensus.cc:2955] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:16:58.774866  6505 raft_consensus.cc:2955] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:16:58.796423  6580 raft_consensus.cc:2955] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:16:58.797483  7322 raft_consensus.cc:1064] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:16:58.802852  7274 raft_consensus.cc:2955] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:16:58.809059  7322 consensus_queue.cc:237] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:16:58.833897  6651 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:58.836652  7342 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:58.852597  7274 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:58.854622  6580 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:58.854568  7341 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:58.856679  7341 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:58.866365  6580 consensus_queue.cc:237] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:16:58.876147  7274 raft_consensus.cc:1275] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:58.877681  7349 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:58.879271  6651 raft_consensus.cc:1275] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:58.883412  7329 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:58.898705  7330 raft_consensus.cc:2955] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:16:58.914791  6651 raft_consensus.cc:2955] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:16:58.921715  6406 catalog_manager.cc:5697] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:16:58.929023  7274 raft_consensus.cc:2955] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:16:58.931363  6392 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d1778f67a2ba4fc2825ec2be5097d77a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:58.912357  7342 raft_consensus.cc:2955] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:16:58.947891  6654 consensus_queue.cc:237] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:16:58.950627  6403 catalog_manager.cc:5697] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: config changed from index 3 to 4, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:58.955250  6505 raft_consensus.cc:1275] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:58.964447  7169 consensus_queue.cc:1048] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:58.968310  6654 raft_consensus.cc:2955] T f2592253d3214104865730480e057d09 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:16:58.976351  7194 raft_consensus.cc:2955] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:16:58.995064  6504 raft_consensus.cc:2955] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:16:59.004357  6394 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3e07e97c48184f8e99a4fc5a580724a5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:16:59.024895  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 113826us
I20260709 12:16:59.030921  6578 raft_consensus.cc:2955] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:16:59.039284  6494 tablet_service.cc:1558] Processing DeleteTablet for tablet d1778f67a2ba4fc2825ec2be5097d77a with delete_type TABLET_DATA_TOMBSTONED (TS e7554c39679a41bbbca4659b72ab5c65 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45870
I20260709 12:16:59.044623  7273 raft_consensus.cc:2955] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:16:59.053552  7274 raft_consensus.cc:1275] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:59.055747  7313 consensus_queue.cc:1048] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:59.060670  7376 tablet_replica.cc:333] T d1778f67a2ba4fc2825ec2be5097d77a P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:16:59.061880  7376 raft_consensus.cc:2243] T d1778f67a2ba4fc2825ec2be5097d77a P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:59.062661  7376 raft_consensus.cc:2272] T d1778f67a2ba4fc2825ec2be5097d77a P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:59.064241  7342 raft_consensus.cc:1064] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:16:59.067566  6403 catalog_manager.cc:5697] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:59.068910  7376 ts_tablet_manager.cc:1916] T d1778f67a2ba4fc2825ec2be5097d77a P e7554c39679a41bbbca4659b72ab5c65: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:59.060693  7274 raft_consensus.cc:2955] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:16:59.087165  7376 ts_tablet_manager.cc:1929] T d1778f67a2ba4fc2825ec2be5097d77a P e7554c39679a41bbbca4659b72ab5c65: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:59.087687  7376 log.cc:1199] T d1778f67a2ba4fc2825ec2be5097d77a P e7554c39679a41bbbca4659b72ab5c65: Deleting WAL directory at /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-0-root/wals/d1778f67a2ba4fc2825ec2be5097d77a
I20260709 12:16:59.068082  7342 consensus_queue.cc:237] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:16:59.089952  6392 catalog_manager.cc:5028] TS e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): tablet d1778f67a2ba4fc2825ec2be5097d77a (table test-workload [id=f93362d0518e483fbe3a01f43a217e48]) successfully deleted
I20260709 12:16:59.093305  6504 consensus_queue.cc:237] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:16:59.105875  6580 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:59.105804  6406 catalog_manager.cc:5697] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: config changed from index 3 to 4, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:59.107718  7341 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:59.122699  7274 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 12:16:59.125316  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 61396.6us
I20260709 12:16:59.126972  7370 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:59.130475  7370 raft_consensus.cc:2955] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:16:59.137660  7272 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:59.140892  7322 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:59.145316  6651 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:59.146998  7272 raft_consensus.cc:2955] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:16:59.149717  6580 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:16:59.152431  7341 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:59.153124  7342 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:59.152626  6578 raft_consensus.cc:2955] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:16:59.158886  6568 tablet_service.cc:1558] Processing DeleteTablet for tablet 3e07e97c48184f8e99a4fc5a580724a5 with delete_type TABLET_DATA_TOMBSTONED (TS 496c7c3792cd4587b4dbeba435f4be1e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56112
I20260709 12:16:59.161409  6392 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f2592253d3214104865730480e057d09 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:59.173655  7375 raft_consensus.cc:2955] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:16:59.182922  6578 raft_consensus.cc:2955] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:16:59.188205  6651 raft_consensus.cc:2955] T e416f9423f964ba5a29e536e8160c627 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:16:59.190402  6403 catalog_manager.cc:5697] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: config changed from index 3 to 4, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:16:59.210836  7272 raft_consensus.cc:2955] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:16:59.215042  6640 tablet_service.cc:1558] Processing DeleteTablet for tablet f2592253d3214104865730480e057d09 with delete_type TABLET_DATA_TOMBSTONED (TS f2b06aebe97249a4877d798b8118ece4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58970
I20260709 12:16:59.175078  7380 tablet_replica.cc:333] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:16:59.218777  7380 raft_consensus.cc:2243] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:59.230088  6401 catalog_manager.cc:5697] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:59.220456  7380 raft_consensus.cc:2272] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:59.283595  6505 consensus_queue.cc:237] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:16:59.289186  7383 tablet_replica.cc:333] T f2592253d3214104865730480e057d09 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:16:59.290242  7383 raft_consensus.cc:2243] T f2592253d3214104865730480e057d09 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:59.290934  7383 raft_consensus.cc:2272] T f2592253d3214104865730480e057d09 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:59.297884  6576 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:59.299679  7375 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:59.303450  7272 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:16:59.304134  7383 ts_tablet_manager.cc:1916] T f2592253d3214104865730480e057d09 P f2b06aebe97249a4877d798b8118ece4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:59.313586  7380 ts_tablet_manager.cc:1916] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:59.314031  7370 raft_consensus.cc:2955] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:16:59.324636  7375 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:59.326771  6576 raft_consensus.cc:2955] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:16:59.336848  6392 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e416f9423f964ba5a29e536e8160c627 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:59.336687  7272 raft_consensus.cc:2955] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:16:59.348694  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 55254us
I20260709 12:16:59.354579  6403 catalog_manager.cc:5697] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: config changed from index 3 to 4, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:59.365856  7380 ts_tablet_manager.cc:1929] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:59.366464  7380 log.cc:1199] T 3e07e97c48184f8e99a4fc5a580724a5 P 496c7c3792cd4587b4dbeba435f4be1e: Deleting WAL directory at /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-1-root/wals/3e07e97c48184f8e99a4fc5a580724a5
I20260709 12:16:59.368193  6392 catalog_manager.cc:5028] TS 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): tablet 3e07e97c48184f8e99a4fc5a580724a5 (table test-workload [id=f93362d0518e483fbe3a01f43a217e48]) successfully deleted
I20260709 12:16:59.369448  7383 ts_tablet_manager.cc:1929] T f2592253d3214104865730480e057d09 P f2b06aebe97249a4877d798b8118ece4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:59.369935  7383 log.cc:1199] T f2592253d3214104865730480e057d09 P f2b06aebe97249a4877d798b8118ece4: Deleting WAL directory at /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-2-root/wals/f2592253d3214104865730480e057d09
I20260709 12:16:59.371873  6394 catalog_manager.cc:5028] TS f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): tablet f2592253d3214104865730480e057d09 (table test-workload [id=f93362d0518e483fbe3a01f43a217e48]) successfully deleted
I20260709 12:16:59.375677  6640 tablet_service.cc:1558] Processing DeleteTablet for tablet e416f9423f964ba5a29e536e8160c627 with delete_type TABLET_DATA_TOMBSTONED (TS f2b06aebe97249a4877d798b8118ece4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58970
I20260709 12:16:59.378536  7383 tablet_replica.cc:333] T e416f9423f964ba5a29e536e8160c627 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:16:59.380085  7383 raft_consensus.cc:2243] T e416f9423f964ba5a29e536e8160c627 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:59.380964  7383 raft_consensus.cc:2272] T e416f9423f964ba5a29e536e8160c627 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:59.384156  7383 ts_tablet_manager.cc:1916] T e416f9423f964ba5a29e536e8160c627 P f2b06aebe97249a4877d798b8118ece4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:59.394564  7383 ts_tablet_manager.cc:1929] T e416f9423f964ba5a29e536e8160c627 P f2b06aebe97249a4877d798b8118ece4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:16:59.394930  7383 log.cc:1199] T e416f9423f964ba5a29e536e8160c627 P f2b06aebe97249a4877d798b8118ece4: Deleting WAL directory at /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-2-root/wals/e416f9423f964ba5a29e536e8160c627
I20260709 12:16:59.396498  6394 catalog_manager.cc:5028] TS f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): tablet e416f9423f964ba5a29e536e8160c627 (table test-workload [id=f93362d0518e483fbe3a01f43a217e48]) successfully deleted
I20260709 12:17:00.441592  6576 consensus_queue.cc:237] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:00.450035  6506 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:00.450225  6549 consensus_peers.cc:597] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d6029ffeecb94e4da196c61c032b862b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:00.451472  7330 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:00.451917  6656 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:00.453599  7330 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:00.478925  7330 raft_consensus.cc:2955] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:00.480676  6654 raft_consensus.cc:2955] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:00.494717  6401 catalog_manager.cc:5697] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 reported cstate change: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:00.496613  6506 raft_consensus.cc:2955] T d6029ffeecb94e4da196c61c032b862b P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
W20260709 12:17:00.499457  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:00.500231  6505 consensus_queue.cc:237] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:00.513470  6654 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:00.515296  7375 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:00.530918  6576 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:00.535432  6474 consensus_peers.cc:597] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2ebfdf7a730a407185ead1aedcf4b8c4. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:00.538645  7370 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:17:00.584064  7396 log.cc:927] Time spent T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:17:00.613871  7397 raft_consensus.cc:2955] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:00.623303  6656 raft_consensus.cc:2955] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:00.623571  6580 raft_consensus.cc:2955] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:00.630625  6654 consensus_queue.cc:237] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:00.648422  6406 catalog_manager.cc:5697] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:00.652529  6505 raft_consensus.cc:1275] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:00.653903  6576 raft_consensus.cc:1275] T 13ab2c6da5994ae2a955fc11d1761c41 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:00.655547  7388 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:17:00.657790  6623 consensus_peers.cc:597] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 13ab2c6da5994ae2a955fc11d1761c41. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:00.659034  7169 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:00.679983  7169 raft_consensus.cc:2955] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:00.699054  6506 raft_consensus.cc:2955] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:00.703987  6577 raft_consensus.cc:2955] T 13ab2c6da5994ae2a955fc11d1761c41 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:00.709159  6581 consensus_queue.cc:237] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:00.718967  6401 catalog_manager.cc:5697] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:00.728133  6654 raft_consensus.cc:1275] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:00.728722  6505 raft_consensus.cc:1275] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:00.729661  6549 consensus_peers.cc:597] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8066a9d7ccf34c89ad8d3df18c4e86c2. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:00.729683  7402 consensus_queue.cc:1048] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:00.732102  7399 consensus_queue.cc:1048] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:00.753691  7373 raft_consensus.cc:2955] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:00.754940  6506 raft_consensus.cc:2955] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:00.779138  6654 raft_consensus.cc:2955] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:00.787508  6406 catalog_manager.cc:5697] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:00.871479  7410 ts_tablet_manager.cc:933] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:00.873489  7410 tablet_copy_client.cc:323] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.2:45101
I20260709 12:17:00.875094  6594 tablet_copy_service.cc:140] P 496c7c3792cd4587b4dbeba435f4be1e: Received BeginTabletCopySession request for tablet d6029ffeecb94e4da196c61c032b862b from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:54204)
I20260709 12:17:00.875648  6594 tablet_copy_service.cc:161] P 496c7c3792cd4587b4dbeba435f4be1e: Beginning new tablet copy session on tablet d6029ffeecb94e4da196c61c032b862b from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:54204: session id = 20a202c76244485fbc8f1610473db8b9-d6029ffeecb94e4da196c61c032b862b
I20260709 12:17:00.883635  6594 tablet_copy_source_session.cc:215] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:00.887931  7410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6029ffeecb94e4da196c61c032b862b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:00.906085  7410 tablet_copy_client.cc:806] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:00.906800  7410 tablet_copy_client.cc:670] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:00.912102  7410 tablet_copy_client.cc:538] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:00.919459  7410 tablet_bootstrap.cc:492] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
I20260709 12:17:00.935873  7410 tablet_bootstrap.cc:492] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:00.936661  7410 tablet_bootstrap.cc:492] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:17:00.937214  7410 ts_tablet_manager.cc:1403] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 0.018s	user 0.016s	sys 0.004s
I20260709 12:17:00.939344  7410 raft_consensus.cc:359] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:00.939957  7410 raft_consensus.cc:740] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
I20260709 12:17:00.940531  7410 consensus_queue.cc:260] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:00.943746  7410 ts_tablet_manager.cc:1434] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 12:17:00.945921  6594 tablet_copy_service.cc:342] P 496c7c3792cd4587b4dbeba435f4be1e: Request end of tablet copy session 20a202c76244485fbc8f1610473db8b9-d6029ffeecb94e4da196c61c032b862b received from {username='slave'} at 127.0.0.1:54204
I20260709 12:17:00.946326  6594 tablet_copy_service.cc:434] P 496c7c3792cd4587b4dbeba435f4be1e: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-d6029ffeecb94e4da196c61c032b862b on tablet d6029ffeecb94e4da196c61c032b862b with peer 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:00.999787  7410 ts_tablet_manager.cc:933] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643)
I20260709 12:17:01.001477  7410 tablet_copy_client.cc:323] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.1:36643
I20260709 12:17:01.002942  6518 tablet_copy_service.cc:140] P e7554c39679a41bbbca4659b72ab5c65: Received BeginTabletCopySession request for tablet 2ebfdf7a730a407185ead1aedcf4b8c4 from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:41304)
I20260709 12:17:01.003438  6518 tablet_copy_service.cc:161] P e7554c39679a41bbbca4659b72ab5c65: Beginning new tablet copy session on tablet 2ebfdf7a730a407185ead1aedcf4b8c4 from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:41304: session id = 20a202c76244485fbc8f1610473db8b9-2ebfdf7a730a407185ead1aedcf4b8c4
I20260709 12:17:01.008934  6518 tablet_copy_source_session.cc:215] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:01.011620  7410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ebfdf7a730a407185ead1aedcf4b8c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:01.026893  7410 tablet_copy_client.cc:806] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:01.027472  7410 tablet_copy_client.cc:670] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:01.033123  7410 tablet_copy_client.cc:538] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:01.042856  7410 tablet_bootstrap.cc:492] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
I20260709 12:17:01.061334  7410 tablet_bootstrap.cc:492] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:01.062037  7410 tablet_bootstrap.cc:492] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:17:01.062577  7410 ts_tablet_manager.cc:1403] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.003s
I20260709 12:17:01.064739  7410 raft_consensus.cc:359] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:01.065541  7410 raft_consensus.cc:740] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
I20260709 12:17:01.066155  7410 consensus_queue.cc:260] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:01.068810  7410 ts_tablet_manager.cc:1434] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.001s
I20260709 12:17:01.071007  6518 tablet_copy_service.cc:342] P e7554c39679a41bbbca4659b72ab5c65: Request end of tablet copy session 20a202c76244485fbc8f1610473db8b9-2ebfdf7a730a407185ead1aedcf4b8c4 received from {username='slave'} at 127.0.0.1:41304
I20260709 12:17:01.071514  6518 tablet_copy_service.cc:434] P e7554c39679a41bbbca4659b72ab5c65: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-2ebfdf7a730a407185ead1aedcf4b8c4 on tablet 2ebfdf7a730a407185ead1aedcf4b8c4 with peer 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:01.124271  7410 ts_tablet_manager.cc:933] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:01.127852  7410 tablet_copy_client.cc:323] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.2:45101
I20260709 12:17:01.129674  6594 tablet_copy_service.cc:140] P 496c7c3792cd4587b4dbeba435f4be1e: Received BeginTabletCopySession request for tablet 8066a9d7ccf34c89ad8d3df18c4e86c2 from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:54204)
I20260709 12:17:01.130204  6594 tablet_copy_service.cc:161] P 496c7c3792cd4587b4dbeba435f4be1e: Beginning new tablet copy session on tablet 8066a9d7ccf34c89ad8d3df18c4e86c2 from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:54204: session id = 20a202c76244485fbc8f1610473db8b9-8066a9d7ccf34c89ad8d3df18c4e86c2
I20260709 12:17:01.137243  6594 tablet_copy_source_session.cc:215] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:01.141041  7410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8066a9d7ccf34c89ad8d3df18c4e86c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:01.159281  7410 tablet_copy_client.cc:806] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:01.160530  7410 tablet_copy_client.cc:670] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:01.164626  7410 tablet_copy_client.cc:538] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:01.172302  7410 tablet_bootstrap.cc:492] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
I20260709 12:17:01.175052  7417 ts_tablet_manager.cc:933] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:01.182927  7417 tablet_copy_client.cc:323] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.3:36107
I20260709 12:17:01.184767  6668 tablet_copy_service.cc:140] P f2b06aebe97249a4877d798b8118ece4: Received BeginTabletCopySession request for tablet 13ab2c6da5994ae2a955fc11d1761c41 from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:49792)
I20260709 12:17:01.185389  6668 tablet_copy_service.cc:161] P f2b06aebe97249a4877d798b8118ece4: Beginning new tablet copy session on tablet 13ab2c6da5994ae2a955fc11d1761c41 from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:49792: session id = 20a202c76244485fbc8f1610473db8b9-13ab2c6da5994ae2a955fc11d1761c41
I20260709 12:17:01.195030  6668 tablet_copy_source_session.cc:215] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:01.198755  7417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13ab2c6da5994ae2a955fc11d1761c41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:01.203065  7410 tablet_bootstrap.cc:492] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:01.204092  7410 tablet_bootstrap.cc:492] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:17:01.204873  7410 ts_tablet_manager.cc:1403] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 0.033s	user 0.022s	sys 0.008s
I20260709 12:17:01.210723  7410 raft_consensus.cc:359] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:01.211582  7410 raft_consensus.cc:740] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
I20260709 12:17:01.212142  7410 consensus_queue.cc:260] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:01.214115  7410 ts_tablet_manager.cc:1434] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:17:01.215907  6594 tablet_copy_service.cc:342] P 496c7c3792cd4587b4dbeba435f4be1e: Request end of tablet copy session 20a202c76244485fbc8f1610473db8b9-8066a9d7ccf34c89ad8d3df18c4e86c2 received from {username='slave'} at 127.0.0.1:54204
I20260709 12:17:01.216398  6594 tablet_copy_service.cc:434] P 496c7c3792cd4587b4dbeba435f4be1e: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-8066a9d7ccf34c89ad8d3df18c4e86c2 on tablet 8066a9d7ccf34c89ad8d3df18c4e86c2 with peer 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:01.222877  7417 tablet_copy_client.cc:806] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:01.223657  7417 tablet_copy_client.cc:670] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:01.246248  7417 tablet_copy_client.cc:538] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:01.257052  7417 tablet_bootstrap.cc:492] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
I20260709 12:17:01.283216  7417 tablet_bootstrap.cc:492] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:01.284595  7417 tablet_bootstrap.cc:492] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:17:01.285672  7417 ts_tablet_manager.cc:1403] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 0.029s	user 0.022s	sys 0.004s
I20260709 12:17:01.290463  7417 raft_consensus.cc:359] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:01.291476  7417 raft_consensus.cc:740] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
I20260709 12:17:01.292177  7417 consensus_queue.cc:260] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:01.295143  7417 ts_tablet_manager.cc:1434] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.005s
I20260709 12:17:01.308051  6668 tablet_copy_service.cc:342] P f2b06aebe97249a4877d798b8118ece4: Request end of tablet copy session 20a202c76244485fbc8f1610473db8b9-13ab2c6da5994ae2a955fc11d1761c41 received from {username='slave'} at 127.0.0.1:49792
I20260709 12:17:01.308584  6668 tablet_copy_service.cc:434] P f2b06aebe97249a4877d798b8118ece4: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-13ab2c6da5994ae2a955fc11d1761c41 on tablet 13ab2c6da5994ae2a955fc11d1761c41 with peer 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:01.381093  7272 raft_consensus.cc:1217] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:01.428900  7272 raft_consensus.cc:1217] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:01.701900  7272 raft_consensus.cc:1217] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:01.743392  7272 raft_consensus.cc:1217] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:01.811125  7421 raft_consensus.cc:1064] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:17:01.813472  7421 consensus_queue.cc:237] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:01.820719  7272 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:01.821545  6506 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:01.822471  7399 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:01.822872  6654 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:01.823501  7421 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:01.825292  7387 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:01.840900  7399 raft_consensus.cc:2955] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:01.842901  6654 raft_consensus.cc:2955] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:01.852816  7272 raft_consensus.cc:2955] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:01.864382  6506 raft_consensus.cc:2955] T d6029ffeecb94e4da196c61c032b862b P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:01.866242  6406 catalog_manager.cc:5697] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 reported cstate change: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:01.909353  6581 consensus_queue.cc:237] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:01.919628  6654 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:01.921682  7272 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:01.921747  7399 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:17:01.925665  7399 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:01.941180  7387 raft_consensus.cc:2955] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:01.961476  6392 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d6029ffeecb94e4da196c61c032b862b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:01.962441  7272 raft_consensus.cc:2955] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:01.978422  6654 raft_consensus.cc:2955] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:02.059315  7397 raft_consensus.cc:1064] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:17:02.062907  7397 consensus_queue.cc:237] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:02.462602  6656 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:02.464061  7399 raft_consensus.cc:1064] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:17:02.464119  7436 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:02.464505  6401 catalog_manager.cc:5697] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 reported cstate change: config changed from index 3 to 4, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:02.470861  7406 raft_consensus.cc:1064] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:17:02.480858  7406 consensus_queue.cc:237] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:02.488301  7399 consensus_queue.cc:237] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:02.491457  6578 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:02.506989  7272 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:02.508746  7436 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:02.513592  7272 raft_consensus.cc:1275] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:02.520478  7384 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:02.535315  7436 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:17:02.537999  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:02.562177  7443 log.cc:927] Time spent T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:17:02.576891  7274 raft_consensus.cc:1275] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:17:02.578161  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 54448.7us
W20260709 12:17:02.600471  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 82830.9us
W20260709 12:17:02.607510  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 82067.6us
W20260709 12:17:02.659487  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 141689us
I20260709 12:17:02.716038  6503 raft_consensus.cc:1275] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:02.717555  6651 raft_consensus.cc:1275] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:02.726756  7444 consensus_queue.cc:1048] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:02.738528  7446 consensus_queue.cc:1048] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:02.756007  6494 tablet_service.cc:1558] Processing DeleteTablet for tablet d6029ffeecb94e4da196c61c032b862b with delete_type TABLET_DATA_TOMBSTONED (TS e7554c39679a41bbbca4659b72ab5c65 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45870
I20260709 12:17:02.802659  7458 tablet_replica.cc:333] T d6029ffeecb94e4da196c61c032b862b P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
W20260709 12:17:02.803718  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 49133.3us
I20260709 12:17:02.818343  7436 raft_consensus.cc:2955] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:02.823786  6656 raft_consensus.cc:2955] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:02.857158  6503 raft_consensus.cc:1275] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:02.843470  6580 raft_consensus.cc:2955] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:02.874737  7458 raft_consensus.cc:2243] T d6029ffeecb94e4da196c61c032b862b P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:02.875815  7458 raft_consensus.cc:2272] T d6029ffeecb94e4da196c61c032b862b P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:02.891539  7458 ts_tablet_manager.cc:1916] T d6029ffeecb94e4da196c61c032b862b P e7554c39679a41bbbca4659b72ab5c65: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:02.934383  7421 raft_consensus.cc:2955] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:02.936374  6654 raft_consensus.cc:2955] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:02.965785  7272 raft_consensus.cc:2955] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:03.001513  7424 consensus_queue.cc:1048] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:03.015455  6403 catalog_manager.cc:5697] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:17:03.028304  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:03.038925  7387 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:03.039595  7387 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:03.040982  7399 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:03.041625  7399 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:03.044219  7274 raft_consensus.cc:2955] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:03.053359  7387 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:03.056635  7399 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:03.057611  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:03.060133  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:03.079346  6578 raft_consensus.cc:1275] T 13ab2c6da5994ae2a955fc11d1761c41 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:03.062585  6402 catalog_manager.cc:5697] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:03.098922  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:03.100812  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:03.133136  7440 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:03.133600  7458 ts_tablet_manager.cc:1929] T d6029ffeecb94e4da196c61c032b862b P e7554c39679a41bbbca4659b72ab5c65: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:03.135794  7458 log.cc:1199] T d6029ffeecb94e4da196c61c032b862b P e7554c39679a41bbbca4659b72ab5c65: Deleting WAL directory at /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-0-root/wals/d6029ffeecb94e4da196c61c032b862b
I20260709 12:17:03.143117  6501 raft_consensus.cc:2955] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:03.160776  6548 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:03.166872  6576 consensus_queue.cc:237] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:03.168586  6392 catalog_manager.cc:5028] TS e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): tablet d6029ffeecb94e4da196c61c032b862b (table test-workload [id=f93362d0518e483fbe3a01f43a217e48]) successfully deleted
I20260709 12:17:03.176419  6656 raft_consensus.cc:1275] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:03.179109  7424 consensus_queue.cc:1048] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:03.191671  6656 raft_consensus.cc:2955] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:03.194715  7406 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:03.190013  7399 raft_consensus.cc:2955] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:03.198827  6501 consensus_queue.cc:237] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:03.195804  7424 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:03.203760  6548 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:03.191731  7274 raft_consensus.cc:1275] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:03.206509  7438 consensus_queue.cc:1048] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:03.211185  7272 raft_consensus.cc:2955] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:03.217188  7273 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:03.220449  6650 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:03.220605  7436 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:03.223196  7438 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:03.223486  7450 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:03.223798  7438 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:03.226280  7438 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:03.204903  7424 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:03.232594  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:03.231802  7449 raft_consensus.cc:2955] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:03.238281  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:03.242414  6650 raft_consensus.cc:2955] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:03.243624  7270 raft_consensus.cc:2955] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:03.243389  7440 raft_consensus.cc:2955] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:03.253854  6501 raft_consensus.cc:2955] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:03.265542  6548 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:03.266393  7424 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:03.274024  6392 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2ebfdf7a730a407185ead1aedcf4b8c4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:03.283185  6402 catalog_manager.cc:5697] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: config changed from index 3 to 4, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:03.287889  6401 catalog_manager.cc:5697] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: config changed from index 3 to 4, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:03.296490  6392 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8066a9d7ccf34c89ad8d3df18c4e86c2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:03.314337  6568 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ebfdf7a730a407185ead1aedcf4b8c4 with delete_type TABLET_DATA_TOMBSTONED (TS 496c7c3792cd4587b4dbeba435f4be1e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56112
I20260709 12:17:03.314489  6494 tablet_service.cc:1558] Processing DeleteTablet for tablet 8066a9d7ccf34c89ad8d3df18c4e86c2 with delete_type TABLET_DATA_TOMBSTONED (TS e7554c39679a41bbbca4659b72ab5c65 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45870
I20260709 12:17:03.316363  7458 tablet_replica.cc:333] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:17:03.317569  7458 raft_consensus.cc:2243] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:03.318312  7458 raft_consensus.cc:2272] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:03.321902  7270 raft_consensus.cc:2955] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:03.328984  7458 ts_tablet_manager.cc:1916] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P e7554c39679a41bbbca4659b72ab5c65: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:03.348923  6578 raft_consensus.cc:2955] T 13ab2c6da5994ae2a955fc11d1761c41 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:03.391670  7458 ts_tablet_manager.cc:1929] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P e7554c39679a41bbbca4659b72ab5c65: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:03.392277  7458 log.cc:1199] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P e7554c39679a41bbbca4659b72ab5c65: Deleting WAL directory at /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-0-root/wals/8066a9d7ccf34c89ad8d3df18c4e86c2
I20260709 12:17:03.396621  6392 catalog_manager.cc:5028] TS e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): tablet 8066a9d7ccf34c89ad8d3df18c4e86c2 (table test-workload [id=f93362d0518e483fbe3a01f43a217e48]) successfully deleted
I20260709 12:17:03.406270  6402 catalog_manager.cc:5697] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:03.408286  7465 tablet_replica.cc:333] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:17:03.439471  6654 consensus_queue.cc:237] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:03.451315  7465 raft_consensus.cc:2243] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:03.452381  7465 raft_consensus.cc:2272] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:03.458806  7465 ts_tablet_manager.cc:1916] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 496c7c3792cd4587b4dbeba435f4be1e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:03.468711  7270 raft_consensus.cc:1275] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:03.470750  6501 raft_consensus.cc:1275] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:03.474898  7440 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:03.476889  7440 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:03.485745  7384 raft_consensus.cc:2955] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:03.489792  7270 raft_consensus.cc:2955] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:03.494508  6501 raft_consensus.cc:2955] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:03.505493  6394 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 13ab2c6da5994ae2a955fc11d1761c41 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:03.508183  7465 ts_tablet_manager.cc:1929] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 496c7c3792cd4587b4dbeba435f4be1e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:03.508740  7465 log.cc:1199] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 496c7c3792cd4587b4dbeba435f4be1e: Deleting WAL directory at /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-1-root/wals/2ebfdf7a730a407185ead1aedcf4b8c4
I20260709 12:17:03.510653  6392 catalog_manager.cc:5028] TS 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): tablet 2ebfdf7a730a407185ead1aedcf4b8c4 (table test-workload [id=f93362d0518e483fbe3a01f43a217e48]) successfully deleted
I20260709 12:17:03.512426  6403 catalog_manager.cc:5697] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: config changed from index 3 to 4, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:03.523859  6568 tablet_service.cc:1558] Processing DeleteTablet for tablet 13ab2c6da5994ae2a955fc11d1761c41 with delete_type TABLET_DATA_TOMBSTONED (TS 496c7c3792cd4587b4dbeba435f4be1e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56112
I20260709 12:17:03.524879  7465 tablet_replica.cc:333] T 13ab2c6da5994ae2a955fc11d1761c41 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:17:03.526430  7465 raft_consensus.cc:2243] T 13ab2c6da5994ae2a955fc11d1761c41 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:03.527238  7465 raft_consensus.cc:2272] T 13ab2c6da5994ae2a955fc11d1761c41 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:03.531409  7465 ts_tablet_manager.cc:1916] T 13ab2c6da5994ae2a955fc11d1761c41 P 496c7c3792cd4587b4dbeba435f4be1e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:03.553022  7465 ts_tablet_manager.cc:1929] T 13ab2c6da5994ae2a955fc11d1761c41 P 496c7c3792cd4587b4dbeba435f4be1e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:03.553340  7465 log.cc:1199] T 13ab2c6da5994ae2a955fc11d1761c41 P 496c7c3792cd4587b4dbeba435f4be1e: Deleting WAL directory at /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-1-root/wals/13ab2c6da5994ae2a955fc11d1761c41
I20260709 12:17:03.554638  6392 catalog_manager.cc:5028] TS 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): tablet 13ab2c6da5994ae2a955fc11d1761c41 (table test-workload [id=f93362d0518e483fbe3a01f43a217e48]) successfully deleted
W20260709 12:17:03.611407  6605 debug-util.cc:398] Leaking SignalData structure 0x7b0800167700 after lost signal to thread 5355
W20260709 12:17:03.612511  6605 debug-util.cc:398] Leaking SignalData structure 0x7b0800167de0 after lost signal to thread 6441
W20260709 12:17:03.613830  6605 debug-util.cc:398] Leaking SignalData structure 0x7b0800303500 after lost signal to thread 6533
W20260709 12:17:03.615062  6605 debug-util.cc:398] Leaking SignalData structure 0x7b08003052c0 after lost signal to thread 6608
W20260709 12:17:03.616168  6605 debug-util.cc:398] Leaking SignalData structure 0x7b08002a4b00 after lost signal to thread 6682
W20260709 12:17:03.617276  6605 debug-util.cc:398] Leaking SignalData structure 0x7b0800307000 after lost signal to thread 7298
I20260709 12:17:04.629523  6578 consensus_queue.cc:237] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
W20260709 12:17:04.653546  6549 consensus_peers.cc:597] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cb7f6a796adb4b8c89579ba77274985f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:04.655232  6654 raft_consensus.cc:1275] T cb7f6a796adb4b8c89579ba77274985f P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:04.655967  6501 raft_consensus.cc:1275] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:04.656673  7446 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:04.657590  7438 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:04.667677  7446 raft_consensus.cc:2955] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:04.673486  6654 raft_consensus.cc:2955] T cb7f6a796adb4b8c89579ba77274985f P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:04.673923  6501 raft_consensus.cc:2955] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:04.694535  6402 catalog_manager.cc:5697] T cb7f6a796adb4b8c89579ba77274985f P f2b06aebe97249a4877d798b8118ece4 reported cstate change: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:04.702430  6578 consensus_queue.cc:237] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:04.720654  6654 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:04.722386  6549 consensus_peers.cc:597] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23f03345855041f8aabfa3e94a1867d9. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:04.722803  6509 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:04.723723  7446 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:04.725122  7438 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:04.771243  7467 raft_consensus.cc:2955] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:04.774307  6509 raft_consensus.cc:2955] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:04.774354  6654 raft_consensus.cc:2955] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:04.804184  6401 catalog_manager.cc:5697] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:04.809595  6580 consensus_queue.cc:237] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
W20260709 12:17:04.837673  6549 consensus_peers.cc:597] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e7223833e1f47b7b68c0080facbbe95. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:04.839140  6656 raft_consensus.cc:1275] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:04.840498  6509 raft_consensus.cc:1275] T 3e7223833e1f47b7b68c0080facbbe95 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:04.849356  7446 consensus_queue.cc:1048] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:04.851683  7438 consensus_queue.cc:1048] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:17:04.893086  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 47569.7us
I20260709 12:17:04.918774  7446 raft_consensus.cc:2955] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:04.930356  6654 raft_consensus.cc:2955] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:04.959196  6501 consensus_queue.cc:237] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:04.965201  6401 catalog_manager.cc:5697] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:04.975486  6505 raft_consensus.cc:2955] T 3e7223833e1f47b7b68c0080facbbe95 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:05.000895  6654 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:05.002319  7436 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:05.002517  6576 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:05.006419  6474 consensus_peers.cc:597] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c6b5198a4b6741cdb13e49cf4e90173c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:05.015805  7436 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:05.042882  7449 raft_consensus.cc:2955] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:05.046808  6576 raft_consensus.cc:2955] T c6b5198a4b6741cdb13e49cf4e90173c P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:05.049156  6650 raft_consensus.cc:2955] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:05.072638  6401 catalog_manager.cc:5697] T c6b5198a4b6741cdb13e49cf4e90173c P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:05.142163  7492 ts_tablet_manager.cc:933] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:05.156824  7492 tablet_copy_client.cc:323] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.2:45101
I20260709 12:17:05.161803  6594 tablet_copy_service.cc:140] P 496c7c3792cd4587b4dbeba435f4be1e: Received BeginTabletCopySession request for tablet 23f03345855041f8aabfa3e94a1867d9 from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:54204)
I20260709 12:17:05.161994  7494 ts_tablet_manager.cc:933] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:05.162387  6594 tablet_copy_service.cc:161] P 496c7c3792cd4587b4dbeba435f4be1e: Beginning new tablet copy session on tablet 23f03345855041f8aabfa3e94a1867d9 from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:54204: session id = 20a202c76244485fbc8f1610473db8b9-23f03345855041f8aabfa3e94a1867d9
I20260709 12:17:05.170526  6594 tablet_copy_source_session.cc:215] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:05.173319  7494 tablet_copy_client.cc:323] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.2:45101
I20260709 12:17:05.174355  7492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23f03345855041f8aabfa3e94a1867d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.184018  6594 tablet_copy_service.cc:140] P 496c7c3792cd4587b4dbeba435f4be1e: Received BeginTabletCopySession request for tablet cb7f6a796adb4b8c89579ba77274985f from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:54204)
I20260709 12:17:05.184616  6594 tablet_copy_service.cc:161] P 496c7c3792cd4587b4dbeba435f4be1e: Beginning new tablet copy session on tablet cb7f6a796adb4b8c89579ba77274985f from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:54204: session id = 20a202c76244485fbc8f1610473db8b9-cb7f6a796adb4b8c89579ba77274985f
I20260709 12:17:05.207278  7492 tablet_copy_client.cc:806] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:05.208026  7492 tablet_copy_client.cc:670] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:05.212788  7492 tablet_copy_client.cc:538] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:05.214963  6594 tablet_copy_source_session.cc:215] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:05.227981  7494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb7f6a796adb4b8c89579ba77274985f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.247218  7492 tablet_bootstrap.cc:492] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
I20260709 12:17:05.254017  7494 tablet_copy_client.cc:806] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:05.254698  7494 tablet_copy_client.cc:670] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:05.258862  7494 tablet_copy_client.cc:538] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:05.271612  7494 tablet_bootstrap.cc:492] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
I20260709 12:17:05.316828  7497 ts_tablet_manager.cc:933] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:05.328227  7492 tablet_bootstrap.cc:492] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:05.329360  7494 tablet_bootstrap.cc:492] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:05.329770  7492 tablet_bootstrap.cc:492] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:17:05.330896  7494 tablet_bootstrap.cc:492] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:17:05.330986  7492 ts_tablet_manager.cc:1403] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 0.088s	user 0.028s	sys 0.004s
I20260709 12:17:05.332024  7494 ts_tablet_manager.cc:1403] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 0.061s	user 0.024s	sys 0.007s
I20260709 12:17:05.335623  7492 raft_consensus.cc:359] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:05.337090  7497 tablet_copy_client.cc:323] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.2:45101
I20260709 12:17:05.338560  7492 raft_consensus.cc:740] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
I20260709 12:17:05.339455  7492 consensus_queue.cc:260] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:05.340358  6594 tablet_copy_service.cc:140] P 496c7c3792cd4587b4dbeba435f4be1e: Received BeginTabletCopySession request for tablet 3e7223833e1f47b7b68c0080facbbe95 from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:54204)
I20260709 12:17:05.340989  6594 tablet_copy_service.cc:161] P 496c7c3792cd4587b4dbeba435f4be1e: Beginning new tablet copy session on tablet 3e7223833e1f47b7b68c0080facbbe95 from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:54204: session id = 20a202c76244485fbc8f1610473db8b9-3e7223833e1f47b7b68c0080facbbe95
I20260709 12:17:05.337716  7494 raft_consensus.cc:359] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:05.351778  7494 raft_consensus.cc:740] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
I20260709 12:17:05.356720  7492 ts_tablet_manager.cc:1434] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.025s	user 0.010s	sys 0.005s
I20260709 12:17:05.361048  6593 tablet_copy_service.cc:342] P 496c7c3792cd4587b4dbeba435f4be1e: Request end of tablet copy session 20a202c76244485fbc8f1610473db8b9-23f03345855041f8aabfa3e94a1867d9 received from {username='slave'} at 127.0.0.1:54204
I20260709 12:17:05.361545  6593 tablet_copy_service.cc:434] P 496c7c3792cd4587b4dbeba435f4be1e: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-23f03345855041f8aabfa3e94a1867d9 on tablet 23f03345855041f8aabfa3e94a1867d9 with peer 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:05.361860  6594 tablet_copy_source_session.cc:215] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:05.352754  7494 consensus_queue.cc:260] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:05.365660  7497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e7223833e1f47b7b68c0080facbbe95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.366462  7494 ts_tablet_manager.cc:1434] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 12:17:05.370442  6594 tablet_copy_service.cc:342] P 496c7c3792cd4587b4dbeba435f4be1e: Request end of tablet copy session 20a202c76244485fbc8f1610473db8b9-cb7f6a796adb4b8c89579ba77274985f received from {username='slave'} at 127.0.0.1:54204
I20260709 12:17:05.371409  6594 tablet_copy_service.cc:434] P 496c7c3792cd4587b4dbeba435f4be1e: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-cb7f6a796adb4b8c89579ba77274985f on tablet cb7f6a796adb4b8c89579ba77274985f with peer 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:05.429081  7494 ts_tablet_manager.cc:933] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643)
I20260709 12:17:05.436730  7494 tablet_copy_client.cc:323] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.1:36643
I20260709 12:17:05.439961  6518 tablet_copy_service.cc:140] P e7554c39679a41bbbca4659b72ab5c65: Received BeginTabletCopySession request for tablet c6b5198a4b6741cdb13e49cf4e90173c from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:41304)
I20260709 12:17:05.440557  6518 tablet_copy_service.cc:161] P e7554c39679a41bbbca4659b72ab5c65: Beginning new tablet copy session on tablet c6b5198a4b6741cdb13e49cf4e90173c from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:41304: session id = 20a202c76244485fbc8f1610473db8b9-c6b5198a4b6741cdb13e49cf4e90173c
I20260709 12:17:05.447353  7497 tablet_copy_client.cc:806] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:05.448792  6518 tablet_copy_source_session.cc:215] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:05.449199  7497 tablet_copy_client.cc:670] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:05.452868  7494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6b5198a4b6741cdb13e49cf4e90173c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:05.457340  7497 tablet_copy_client.cc:538] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:05.476269  7494 tablet_copy_client.cc:806] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:05.476984  7494 tablet_copy_client.cc:670] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:05.479631  7497 tablet_bootstrap.cc:492] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
I20260709 12:17:05.494024  7494 tablet_copy_client.cc:538] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:05.516611  7494 tablet_bootstrap.cc:492] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
I20260709 12:17:05.590308  7272 raft_consensus.cc:1217] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:05.606491  7497 tablet_bootstrap.cc:492] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:05.607568  7497 tablet_bootstrap.cc:492] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:17:05.608253  7497 ts_tablet_manager.cc:1403] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 0.129s	user 0.025s	sys 0.009s
I20260709 12:17:05.611795  7494 tablet_bootstrap.cc:492] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:05.612845  7494 tablet_bootstrap.cc:492] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:17:05.613873  7494 ts_tablet_manager.cc:1403] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 0.097s	user 0.035s	sys 0.023s
I20260709 12:17:05.618201  7494 raft_consensus.cc:359] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:05.619350  7494 raft_consensus.cc:740] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
I20260709 12:17:05.620039  7494 consensus_queue.cc:260] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:05.623538  7494 ts_tablet_manager.cc:1434] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:17:05.611236  7497 raft_consensus.cc:359] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:05.625048  7497 raft_consensus.cc:740] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
I20260709 12:17:05.625324  6518 tablet_copy_service.cc:342] P e7554c39679a41bbbca4659b72ab5c65: Request end of tablet copy session 20a202c76244485fbc8f1610473db8b9-c6b5198a4b6741cdb13e49cf4e90173c received from {username='slave'} at 127.0.0.1:41304
I20260709 12:17:05.625852  6518 tablet_copy_service.cc:434] P e7554c39679a41bbbca4659b72ab5c65: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-c6b5198a4b6741cdb13e49cf4e90173c on tablet c6b5198a4b6741cdb13e49cf4e90173c with peer 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:05.625885  7497 consensus_queue.cc:260] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:05.628794  7497 ts_tablet_manager.cc:1434] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 12:17:05.632194  6594 tablet_copy_service.cc:342] P 496c7c3792cd4587b4dbeba435f4be1e: Request end of tablet copy session 20a202c76244485fbc8f1610473db8b9-3e7223833e1f47b7b68c0080facbbe95 received from {username='slave'} at 127.0.0.1:54204
I20260709 12:17:05.632684  6594 tablet_copy_service.cc:434] P 496c7c3792cd4587b4dbeba435f4be1e: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-3e7223833e1f47b7b68c0080facbbe95 on tablet 3e7223833e1f47b7b68c0080facbbe95 with peer 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:05.661152  7272 raft_consensus.cc:1217] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:05.696897  7272 raft_consensus.cc:1217] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:05.953274  7272 raft_consensus.cc:1217] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:06.079825  7481 raft_consensus.cc:1064] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:17:06.082396  7481 consensus_queue.cc:237] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:06.114300  7272 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:06.115808  7483 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:06.131445  7467 raft_consensus.cc:1064] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:17:06.134616  6509 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:06.141263  6650 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:06.144914  7481 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:06.154512  7481 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:06.156584  7481 raft_consensus.cc:1064] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:17:06.142031  7467 consensus_queue.cc:237] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:06.188676  7270 raft_consensus.cc:1275] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:06.190282  7446 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:06.196758  7481 consensus_queue.cc:237] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:06.198035  7483 raft_consensus.cc:2955] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:06.201936  6656 raft_consensus.cc:2955] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:06.222252  6509 raft_consensus.cc:2955] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:06.239636  7274 raft_consensus.cc:1275] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:06.239944  6509 raft_consensus.cc:1275] T 3e7223833e1f47b7b68c0080facbbe95 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:06.241278  7507 consensus_queue.cc:1048] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:06.244197  6401 catalog_manager.cc:5697] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:06.251148  7512 consensus_queue.cc:1048] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:06.251487  6654 raft_consensus.cc:1275] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:06.265091  7446 consensus_queue.cc:1048] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:06.267933  6650 raft_consensus.cc:1275] T cb7f6a796adb4b8c89579ba77274985f P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:06.273151  6505 raft_consensus.cc:1275] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:06.280486  7481 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:06.283517  7446 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:06.302568  7507 raft_consensus.cc:2955] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:06.310117  6656 raft_consensus.cc:2955] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:06.311904  7274 raft_consensus.cc:2955] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:06.320549  7446 raft_consensus.cc:2955] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:06.320854  6501 raft_consensus.cc:2955] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:06.398459  7272 raft_consensus.cc:2955] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:17:06.412870  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 43480.1us
W20260709 12:17:06.461284  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 56191us
I20260709 12:17:06.467505  6650 raft_consensus.cc:2955] T cb7f6a796adb4b8c89579ba77274985f P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:06.472374  6576 consensus_queue.cc:237] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:06.481024  7272 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:06.482180  6403 catalog_manager.cc:5697] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:06.484619  7525 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:06.489310  6402 catalog_manager.cc:5697] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:06.509876  6509 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:06.509513  7272 raft_consensus.cc:2955] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:06.512813  6501 raft_consensus.cc:2955] T 3e7223833e1f47b7b68c0080facbbe95 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:06.513741  7274 raft_consensus.cc:2955] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:06.511952  7527 raft_consensus.cc:2955] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:06.520040  7525 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:06.539863  6504 raft_consensus.cc:2955] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:17:06.557168  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 176913us
I20260709 12:17:06.558568  6401 catalog_manager.cc:5697] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: config changed from index 3 to 4, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:06.561133  6392 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 23f03345855041f8aabfa3e94a1867d9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:06.589963  7449 raft_consensus.cc:1064] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:17:06.601467  7449 consensus_queue.cc:237] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:06.633150  6640 tablet_service.cc:1558] Processing DeleteTablet for tablet 23f03345855041f8aabfa3e94a1867d9 with delete_type TABLET_DATA_TOMBSTONED (TS f2b06aebe97249a4877d798b8118ece4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58970
I20260709 12:17:06.650593  7532 tablet_replica.cc:333] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:17:06.667950  6580 consensus_queue.cc:237] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:06.669579  6578 consensus_queue.cc:237] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:06.674110  6650 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:06.675032  7274 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:06.675680  7449 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:06.680627  7274 raft_consensus.cc:1275] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:06.683606  6580 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:06.686425  7525 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:06.690171  6501 raft_consensus.cc:1275] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:06.690289  7436 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:06.693221  7449 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:06.698412  6654 raft_consensus.cc:1275] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:06.701510  7527 consensus_queue.cc:1048] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:06.702642  7507 raft_consensus.cc:2955] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:06.703987  7274 raft_consensus.cc:1275] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:06.720574  7525 consensus_queue.cc:1048] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:06.720920  7526 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:06.726428  6392 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet cb7f6a796adb4b8c89579ba77274985f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:06.729341  6656 raft_consensus.cc:2955] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:06.734628  7512 raft_consensus.cc:2955] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:06.711834  7274 raft_consensus.cc:2955] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:06.749123  7529 raft_consensus.cc:2955] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:06.767450  6403 catalog_manager.cc:5697] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: config changed from index 3 to 4, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:06.749476  6503 raft_consensus.cc:2955] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:06.752415  6654 raft_consensus.cc:2955] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:06.790686  6640 tablet_service.cc:1558] Processing DeleteTablet for tablet cb7f6a796adb4b8c89579ba77274985f with delete_type TABLET_DATA_TOMBSTONED (TS f2b06aebe97249a4877d798b8118ece4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58970
I20260709 12:17:06.794237  7272 raft_consensus.cc:2955] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:06.798760  6403 catalog_manager.cc:5697] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 reported cstate change: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:17:06.810660  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 46933.8us
W20260709 12:17:06.811389  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 44798.4us
I20260709 12:17:06.812208  6392 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3e7223833e1f47b7b68c0080facbbe95 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:06.837277  7270 raft_consensus.cc:2955] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:06.849177  6501 consensus_queue.cc:237] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:06.849843  6401 catalog_manager.cc:5697] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: config changed from index 3 to 4, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:06.858028  6654 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:06.860008  7449 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:06.878912  6494 tablet_service.cc:1558] Processing DeleteTablet for tablet 3e7223833e1f47b7b68c0080facbbe95 with delete_type TABLET_DATA_TOMBSTONED (TS e7554c39679a41bbbca4659b72ab5c65 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45870
I20260709 12:17:06.892446  7536 tablet_replica.cc:333] T 3e7223833e1f47b7b68c0080facbbe95 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:17:06.893813  7536 raft_consensus.cc:2243] T 3e7223833e1f47b7b68c0080facbbe95 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:06.898011  7449 raft_consensus.cc:2955] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:06.909827  7273 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:06.900938  7536 raft_consensus.cc:2272] T 3e7223833e1f47b7b68c0080facbbe95 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:06.904100  7532 raft_consensus.cc:2243] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:06.942587  7532 raft_consensus.cc:2272] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:06.950606  7536 ts_tablet_manager.cc:1916] T 3e7223833e1f47b7b68c0080facbbe95 P e7554c39679a41bbbca4659b72ab5c65: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:06.960511  7529 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:06.941196  6656 raft_consensus.cc:2955] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:06.976994  6392 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c6b5198a4b6741cdb13e49cf4e90173c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:06.983110  7272 raft_consensus.cc:2955] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:06.996459  7532 ts_tablet_manager.cc:1916] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:07.011482  6402 catalog_manager.cc:5697] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 reported cstate change: config changed from index 3 to 4, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:07.028640  6568 tablet_service.cc:1558] Processing DeleteTablet for tablet c6b5198a4b6741cdb13e49cf4e90173c with delete_type TABLET_DATA_TOMBSTONED (TS 496c7c3792cd4587b4dbeba435f4be1e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56112
I20260709 12:17:07.055835  7537 tablet_replica.cc:333] T c6b5198a4b6741cdb13e49cf4e90173c P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:17:07.057298  7537 raft_consensus.cc:2243] T c6b5198a4b6741cdb13e49cf4e90173c P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:07.058211  7537 pending_rounds.cc:70] T c6b5198a4b6741cdb13e49cf4e90173c P 496c7c3792cd4587b4dbeba435f4be1e: Trying to abort 1 pending ops.
I20260709 12:17:07.058655  7537 pending_rounds.cc:77] T c6b5198a4b6741cdb13e49cf4e90173c P 496c7c3792cd4587b4dbeba435f4be1e: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7305623251317612544 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" old_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } } new_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } } }
I20260709 12:17:07.060259  7537 raft_consensus.cc:2917] T c6b5198a4b6741cdb13e49cf4e90173c P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted
I20260709 12:17:07.060993  7537 raft_consensus.cc:2889] T c6b5198a4b6741cdb13e49cf4e90173c P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 12:17:07.066107  7537 raft_consensus.cc:2272] T c6b5198a4b6741cdb13e49cf4e90173c P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:07.075302  7536 ts_tablet_manager.cc:1929] T 3e7223833e1f47b7b68c0080facbbe95 P e7554c39679a41bbbca4659b72ab5c65: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:07.080960  7536 log.cc:1199] T 3e7223833e1f47b7b68c0080facbbe95 P e7554c39679a41bbbca4659b72ab5c65: Deleting WAL directory at /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-0-root/wals/3e7223833e1f47b7b68c0080facbbe95
I20260709 12:17:07.080376  7537 ts_tablet_manager.cc:1916] T c6b5198a4b6741cdb13e49cf4e90173c P 496c7c3792cd4587b4dbeba435f4be1e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:07.086766  6392 catalog_manager.cc:5028] TS e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): tablet 3e7223833e1f47b7b68c0080facbbe95 (table test-workload [id=f93362d0518e483fbe3a01f43a217e48]) successfully deleted
I20260709 12:17:07.150126  7532 ts_tablet_manager.cc:1929] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:07.150667  7532 log.cc:1199] T 23f03345855041f8aabfa3e94a1867d9 P f2b06aebe97249a4877d798b8118ece4: Deleting WAL directory at /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-2-root/wals/23f03345855041f8aabfa3e94a1867d9
I20260709 12:17:07.154362  7537 ts_tablet_manager.cc:1929] T c6b5198a4b6741cdb13e49cf4e90173c P 496c7c3792cd4587b4dbeba435f4be1e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:07.154877  7537 log.cc:1199] T c6b5198a4b6741cdb13e49cf4e90173c P 496c7c3792cd4587b4dbeba435f4be1e: Deleting WAL directory at /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-1-root/wals/c6b5198a4b6741cdb13e49cf4e90173c
I20260709 12:17:07.155624  6394 catalog_manager.cc:5028] TS f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): tablet 23f03345855041f8aabfa3e94a1867d9 (table test-workload [id=f93362d0518e483fbe3a01f43a217e48]) successfully deleted
I20260709 12:17:07.156669  6392 catalog_manager.cc:5028] TS 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): tablet c6b5198a4b6741cdb13e49cf4e90173c (table test-workload [id=f93362d0518e483fbe3a01f43a217e48]) successfully deleted
I20260709 12:17:07.157352  7532 tablet_replica.cc:333] T cb7f6a796adb4b8c89579ba77274985f P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:17:07.158914  7532 raft_consensus.cc:2243] T cb7f6a796adb4b8c89579ba77274985f P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:07.159667  7532 raft_consensus.cc:2272] T cb7f6a796adb4b8c89579ba77274985f P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:07.165499  7532 ts_tablet_manager.cc:1916] T cb7f6a796adb4b8c89579ba77274985f P f2b06aebe97249a4877d798b8118ece4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:07.185683  7532 ts_tablet_manager.cc:1929] T cb7f6a796adb4b8c89579ba77274985f P f2b06aebe97249a4877d798b8118ece4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:07.186223  7532 log.cc:1199] T cb7f6a796adb4b8c89579ba77274985f P f2b06aebe97249a4877d798b8118ece4: Deleting WAL directory at /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-2-root/wals/cb7f6a796adb4b8c89579ba77274985f
I20260709 12:17:07.188846  6394 catalog_manager.cc:5028] TS f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): tablet cb7f6a796adb4b8c89579ba77274985f (table test-workload [id=f93362d0518e483fbe3a01f43a217e48]) successfully deleted
I20260709 12:17:08.144032  6501 consensus_queue.cc:237] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:08.157330  6656 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:08.158890  7449 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:17:08.165716  6474 consensus_peers.cc:597] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 14cf1c3a63624d94aa6c9c356e6108e2. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:08.168834  6576 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:08.170709  7540 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:08.191298  7449 raft_consensus.cc:2955] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:08.197081  6656 raft_consensus.cc:2955] T 14cf1c3a63624d94aa6c9c356e6108e2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:08.196923  6578 raft_consensus.cc:2955] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:08.207911  6576 consensus_queue.cc:237] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
W20260709 12:17:08.221204  6549 consensus_peers.cc:597] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: eebce9eca2a5496bb4a3ab473a243166. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:08.221446  6650 raft_consensus.cc:1275] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:08.225818  6503 raft_consensus.cc:1275] T eebce9eca2a5496bb4a3ab473a243166 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:08.230104  7527 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:08.230955  6401 catalog_manager.cc:5697] T 14cf1c3a63624d94aa6c9c356e6108e2 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:08.234990  7527 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:08.286173  7527 raft_consensus.cc:2955] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:08.288028  6650 raft_consensus.cc:2955] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:08.289815  6503 raft_consensus.cc:2955] T eebce9eca2a5496bb4a3ab473a243166 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:08.319118  6509 consensus_queue.cc:237] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:08.334180  6650 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:08.336032  6578 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:08.338871  6474 consensus_peers.cc:597] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8e01550f5e994006a4031ca81edcc9aa. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:08.348600  7541 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:08.349609  7542 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:08.385141  7436 raft_consensus.cc:2955] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:08.364177  6402 catalog_manager.cc:5697] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:08.392822  6656 raft_consensus.cc:2955] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:08.387624  6579 raft_consensus.cc:2955] T 8e01550f5e994006a4031ca81edcc9aa P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:08.434458  6650 consensus_queue.cc:237] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:08.462003  6406 catalog_manager.cc:5697] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:08.529036  6623 consensus_peers.cc:597] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6d6c58127aec4100b761ec4b52b842a0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:08.534087  6503 raft_consensus.cc:1275] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:08.535772  7538 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:08.545528  7480 raft_consensus.cc:2955] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:08.547353  6503 raft_consensus.cc:2955] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:08.556061  6579 raft_consensus.cc:1275] T 6d6c58127aec4100b761ec4b52b842a0 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:08.567468  7523 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:08.588583  6402 catalog_manager.cc:5697] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:08.593103  6578 raft_consensus.cc:2955] T 6d6c58127aec4100b761ec4b52b842a0 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:08.631953  7560 ts_tablet_manager.cc:933] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:08.643075  7560 tablet_copy_client.cc:323] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.2:45101
I20260709 12:17:08.645187  6594 tablet_copy_service.cc:140] P 496c7c3792cd4587b4dbeba435f4be1e: Received BeginTabletCopySession request for tablet eebce9eca2a5496bb4a3ab473a243166 from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:54204)
I20260709 12:17:08.645965  6594 tablet_copy_service.cc:161] P 496c7c3792cd4587b4dbeba435f4be1e: Beginning new tablet copy session on tablet eebce9eca2a5496bb4a3ab473a243166 from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:54204: session id = 20a202c76244485fbc8f1610473db8b9-eebce9eca2a5496bb4a3ab473a243166
I20260709 12:17:08.651266  6594 tablet_copy_source_session.cc:215] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:08.654307  7560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eebce9eca2a5496bb4a3ab473a243166. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:08.684232  7560 tablet_copy_client.cc:806] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:08.685207  7560 tablet_copy_client.cc:670] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:08.693007  7560 tablet_copy_client.cc:538] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:08.701962  7560 tablet_bootstrap.cc:492] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
I20260709 12:17:08.723398  7560 tablet_bootstrap.cc:492] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:08.724395  7560 tablet_bootstrap.cc:492] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:17:08.725175  7560 ts_tablet_manager.cc:1403] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.007s
I20260709 12:17:08.728753  7560 raft_consensus.cc:359] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:08.729566  7560 raft_consensus.cc:740] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
I20260709 12:17:08.730201  7560 consensus_queue.cc:260] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:08.755676  7560 ts_tablet_manager.cc:1434] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.019s
I20260709 12:17:08.757951  6594 tablet_copy_service.cc:342] P 496c7c3792cd4587b4dbeba435f4be1e: Request end of tablet copy session 20a202c76244485fbc8f1610473db8b9-eebce9eca2a5496bb4a3ab473a243166 received from {username='slave'} at 127.0.0.1:54204
I20260709 12:17:08.758333  6594 tablet_copy_service.cc:434] P 496c7c3792cd4587b4dbeba435f4be1e: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-eebce9eca2a5496bb4a3ab473a243166 on tablet eebce9eca2a5496bb4a3ab473a243166 with peer 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:08.783800  7560 ts_tablet_manager.cc:933] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643)
I20260709 12:17:08.786135  7560 tablet_copy_client.cc:323] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.1:36643
I20260709 12:17:08.788400  6518 tablet_copy_service.cc:140] P e7554c39679a41bbbca4659b72ab5c65: Received BeginTabletCopySession request for tablet 14cf1c3a63624d94aa6c9c356e6108e2 from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:41304)
I20260709 12:17:08.789031  6518 tablet_copy_service.cc:161] P e7554c39679a41bbbca4659b72ab5c65: Beginning new tablet copy session on tablet 14cf1c3a63624d94aa6c9c356e6108e2 from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:41304: session id = 20a202c76244485fbc8f1610473db8b9-14cf1c3a63624d94aa6c9c356e6108e2
I20260709 12:17:08.803730  6518 tablet_copy_source_session.cc:215] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:08.808264  7560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14cf1c3a63624d94aa6c9c356e6108e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:08.828322  7560 tablet_copy_client.cc:806] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:08.829034  7560 tablet_copy_client.cc:670] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:08.844173  7560 tablet_copy_client.cc:538] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:08.844605  7565 ts_tablet_manager.cc:933] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643)
I20260709 12:17:08.847151  7565 tablet_copy_client.cc:323] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.1:36643
I20260709 12:17:08.849025  6518 tablet_copy_service.cc:140] P e7554c39679a41bbbca4659b72ab5c65: Received BeginTabletCopySession request for tablet 8e01550f5e994006a4031ca81edcc9aa from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:41304)
I20260709 12:17:08.849582  6518 tablet_copy_service.cc:161] P e7554c39679a41bbbca4659b72ab5c65: Beginning new tablet copy session on tablet 8e01550f5e994006a4031ca81edcc9aa from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:41304: session id = 20a202c76244485fbc8f1610473db8b9-8e01550f5e994006a4031ca81edcc9aa
I20260709 12:17:08.856178  7560 tablet_bootstrap.cc:492] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
I20260709 12:17:08.857466  6518 tablet_copy_source_session.cc:215] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:08.860960  7565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e01550f5e994006a4031ca81edcc9aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:08.890235  7565 tablet_copy_client.cc:806] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:08.890865  7565 tablet_copy_client.cc:670] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:08.900884  7565 tablet_copy_client.cc:538] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:08.910988  7565 tablet_bootstrap.cc:492] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
I20260709 12:17:08.924866  7560 tablet_bootstrap.cc:492] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:08.925907  7560 tablet_bootstrap.cc:492] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:17:08.926604  7560 ts_tablet_manager.cc:1403] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 0.071s	user 0.025s	sys 0.008s
I20260709 12:17:08.932175  7560 raft_consensus.cc:359] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:08.933092  7560 raft_consensus.cc:740] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
I20260709 12:17:08.933780  7560 consensus_queue.cc:260] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:08.936820  7560 ts_tablet_manager.cc:1434] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.001s
I20260709 12:17:08.942587  6518 tablet_copy_service.cc:342] P e7554c39679a41bbbca4659b72ab5c65: Request end of tablet copy session 20a202c76244485fbc8f1610473db8b9-14cf1c3a63624d94aa6c9c356e6108e2 received from {username='slave'} at 127.0.0.1:41304
I20260709 12:17:08.943090  6518 tablet_copy_service.cc:434] P e7554c39679a41bbbca4659b72ab5c65: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-14cf1c3a63624d94aa6c9c356e6108e2 on tablet 14cf1c3a63624d94aa6c9c356e6108e2 with peer 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:08.960772  7565 tablet_bootstrap.cc:492] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:08.961998  7565 tablet_bootstrap.cc:492] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:17:08.962831  7565 ts_tablet_manager.cc:1403] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 0.052s	user 0.021s	sys 0.001s
I20260709 12:17:08.966413  7565 raft_consensus.cc:359] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:08.967305  7565 raft_consensus.cc:740] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
I20260709 12:17:08.967978  7565 consensus_queue.cc:260] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:08.970916  7565 ts_tablet_manager.cc:1434] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 12:17:08.972690  6518 tablet_copy_service.cc:342] P e7554c39679a41bbbca4659b72ab5c65: Request end of tablet copy session 20a202c76244485fbc8f1610473db8b9-8e01550f5e994006a4031ca81edcc9aa received from {username='slave'} at 127.0.0.1:41304
I20260709 12:17:08.973204  6518 tablet_copy_service.cc:434] P e7554c39679a41bbbca4659b72ab5c65: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-8e01550f5e994006a4031ca81edcc9aa on tablet 8e01550f5e994006a4031ca81edcc9aa with peer 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:09.150812  7565 ts_tablet_manager.cc:933] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:09.165863  7565 tablet_copy_client.cc:323] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.3:36107
I20260709 12:17:09.174024  6668 tablet_copy_service.cc:140] P f2b06aebe97249a4877d798b8118ece4: Received BeginTabletCopySession request for tablet 6d6c58127aec4100b761ec4b52b842a0 from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:49792)
I20260709 12:17:09.174556  6668 tablet_copy_service.cc:161] P f2b06aebe97249a4877d798b8118ece4: Beginning new tablet copy session on tablet 6d6c58127aec4100b761ec4b52b842a0 from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:49792: session id = 20a202c76244485fbc8f1610473db8b9-6d6c58127aec4100b761ec4b52b842a0
I20260709 12:17:09.185956  6668 tablet_copy_source_session.cc:215] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:09.196264  7565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d6c58127aec4100b761ec4b52b842a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:09.220041  7565 tablet_copy_client.cc:806] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:09.220959  7565 tablet_copy_client.cc:670] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:09.223608  7274 raft_consensus.cc:1217] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:09.240998  7565 tablet_copy_client.cc:538] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:09.259027  7565 tablet_bootstrap.cc:492] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
I20260709 12:17:09.286073  7565 tablet_bootstrap.cc:492] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:09.287014  7565 tablet_bootstrap.cc:492] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:17:09.287710  7565 ts_tablet_manager.cc:1403] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 0.029s	user 0.017s	sys 0.010s
I20260709 12:17:09.290736  7565 raft_consensus.cc:359] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:09.291585  7565 raft_consensus.cc:740] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
I20260709 12:17:09.292271  7565 consensus_queue.cc:260] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:09.295250  7565 ts_tablet_manager.cc:1434] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 12:17:09.297621  6668 tablet_copy_service.cc:342] P f2b06aebe97249a4877d798b8118ece4: Request end of tablet copy session 20a202c76244485fbc8f1610473db8b9-6d6c58127aec4100b761ec4b52b842a0 received from {username='slave'} at 127.0.0.1:49792
I20260709 12:17:09.298082  6668 tablet_copy_service.cc:434] P f2b06aebe97249a4877d798b8118ece4: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-6d6c58127aec4100b761ec4b52b842a0 on tablet 6d6c58127aec4100b761ec4b52b842a0 with peer 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:09.331606  7274 raft_consensus.cc:1217] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:09.394452  7274 raft_consensus.cc:1217] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:09.530462  7274 raft_consensus.cc:1217] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:09.742777  7574 raft_consensus.cc:1064] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:17:09.745762  7574 consensus_queue.cc:237] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:09.765856  7274 raft_consensus.cc:1275] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:09.767314  7547 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:09.771170  6501 raft_consensus.cc:1275] T eebce9eca2a5496bb4a3ab473a243166 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:09.773762  6654 raft_consensus.cc:1275] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:09.773911  7507 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:09.779021  7575 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:09.799458  7547 raft_consensus.cc:2955] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:09.801666  6650 raft_consensus.cc:2955] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:09.814121  7274 raft_consensus.cc:2955] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:09.831552  6402 catalog_manager.cc:5697] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:09.857604  6503 raft_consensus.cc:2955] T eebce9eca2a5496bb4a3ab473a243166 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:09.871308  7573 raft_consensus.cc:1064] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:17:09.874418  7573 consensus_queue.cc:237] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:09.915006  7572 raft_consensus.cc:1064] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:17:09.917193  6654 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:09.918869  7583 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:09.922406  7572 consensus_queue.cc:237] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:09.925037  6578 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:09.926501  7571 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:09.944962  7274 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:09.956817  7583 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:09.984670  6578 consensus_queue.cc:237] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:09.987809  7571 raft_consensus.cc:2955] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:09.991322  7272 raft_consensus.cc:1275] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:09.994058  7507 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:10.020197  7574 raft_consensus.cc:2955] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:10.032644  7273 raft_consensus.cc:2955] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:10.057693  6654 raft_consensus.cc:2955] T 14cf1c3a63624d94aa6c9c356e6108e2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:10.069818  6656 raft_consensus.cc:1275] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:10.077196  7574 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:10.077590  6392 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet eebce9eca2a5496bb4a3ab473a243166 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:10.083020  6650 raft_consensus.cc:2955] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:10.110080  7274 raft_consensus.cc:2955] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:10.137770  6401 catalog_manager.cc:5697] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: config changed from index 3 to 4, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:10.170867  7479 raft_consensus.cc:1064] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:17:10.174100  7479 consensus_queue.cc:237] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:10.177042  6578 raft_consensus.cc:2955] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:10.193507  6403 catalog_manager.cc:5697] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:10.206288  6494 tablet_service.cc:1558] Processing DeleteTablet for tablet eebce9eca2a5496bb4a3ab473a243166 with delete_type TABLET_DATA_TOMBSTONED (TS e7554c39679a41bbbca4659b72ab5c65 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45870
I20260709 12:17:10.225605  7594 tablet_replica.cc:333] T eebce9eca2a5496bb4a3ab473a243166 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:17:10.256415  7594 raft_consensus.cc:2243] T eebce9eca2a5496bb4a3ab473a243166 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:10.257193  7594 raft_consensus.cc:2272] T eebce9eca2a5496bb4a3ab473a243166 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:10.262215  7594 ts_tablet_manager.cc:1916] T eebce9eca2a5496bb4a3ab473a243166 P e7554c39679a41bbbca4659b72ab5c65: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:10.278653  7274 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:10.280531  7571 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:10.291965  6578 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:10.295145  7592 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:10.303663  7594 ts_tablet_manager.cc:1929] T eebce9eca2a5496bb4a3ab473a243166 P e7554c39679a41bbbca4659b72ab5c65: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:10.305729  7274 raft_consensus.cc:1275] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:10.310528  6501 consensus_queue.cc:237] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:10.304201  7594 log.cc:1199] T eebce9eca2a5496bb4a3ab473a243166 P e7554c39679a41bbbca4659b72ab5c65: Deleting WAL directory at /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-0-root/wals/eebce9eca2a5496bb4a3ab473a243166
I20260709 12:17:10.328966  6392 catalog_manager.cc:5028] TS e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): tablet eebce9eca2a5496bb4a3ab473a243166 (table test-workload [id=f93362d0518e483fbe3a01f43a217e48]) successfully deleted
I20260709 12:17:10.331192  7274 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:10.333540  7583 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:10.341569  6576 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:10.345005  6503 raft_consensus.cc:1275] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:10.349037  7573 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:10.360091  7592 raft_consensus.cc:2955] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:10.371809  7586 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:10.362489  7274 raft_consensus.cc:2955] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:10.396283  7571 raft_consensus.cc:2955] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:10.397861  7273 raft_consensus.cc:2955] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:10.410140  6650 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:10.419816  7573 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:10.431646  6578 raft_consensus.cc:1275] T 6d6c58127aec4100b761ec4b52b842a0 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:17:10.443095  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 73852.6us
I20260709 12:17:10.455651  6650 raft_consensus.cc:2955] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:10.460007  6392 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 14cf1c3a63624d94aa6c9c356e6108e2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:10.471662  6579 raft_consensus.cc:2955] T 8e01550f5e994006a4031ca81edcc9aa P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:10.501174  7479 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:10.505738  6401 catalog_manager.cc:5697] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 reported cstate change: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:10.550400  7586 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:10.543354  6576 raft_consensus.cc:2955] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:10.582261  7586 raft_consensus.cc:2955] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:10.579505  6401 catalog_manager.cc:5697] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: config changed from index 3 to 4, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:10.605621  6503 raft_consensus.cc:2955] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:10.617107  6640 tablet_service.cc:1558] Processing DeleteTablet for tablet 14cf1c3a63624d94aa6c9c356e6108e2 with delete_type TABLET_DATA_TOMBSTONED (TS f2b06aebe97249a4877d798b8118ece4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58970
I20260709 12:17:10.624404  7273 raft_consensus.cc:2955] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:10.632298  7606 tablet_replica.cc:333] T 14cf1c3a63624d94aa6c9c356e6108e2 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:17:10.637037  7606 raft_consensus.cc:2243] T 14cf1c3a63624d94aa6c9c356e6108e2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:10.637952  7606 raft_consensus.cc:2272] T 14cf1c3a63624d94aa6c9c356e6108e2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:10.649912  6576 raft_consensus.cc:2955] T 6d6c58127aec4100b761ec4b52b842a0 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:10.654256  6406 catalog_manager.cc:5697] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:10.666857  7606 ts_tablet_manager.cc:1916] T 14cf1c3a63624d94aa6c9c356e6108e2 P f2b06aebe97249a4877d798b8118ece4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:10.699278  6654 consensus_queue.cc:237] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:10.721276  7273 raft_consensus.cc:1275] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:10.743441  6503 raft_consensus.cc:1275] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:10.747937  7586 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:10.757174  7587 raft_consensus.cc:2955] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:10.768473  6503 raft_consensus.cc:2955] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:10.773500  7479 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:10.787487  7273 raft_consensus.cc:2955] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:10.802738  6394 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6d6c58127aec4100b761ec4b52b842a0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:10.805822  6402 catalog_manager.cc:5697] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: config changed from index 3 to 4, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:10.810739  7606 ts_tablet_manager.cc:1929] T 14cf1c3a63624d94aa6c9c356e6108e2 P f2b06aebe97249a4877d798b8118ece4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:10.811378  7606 log.cc:1199] T 14cf1c3a63624d94aa6c9c356e6108e2 P f2b06aebe97249a4877d798b8118ece4: Deleting WAL directory at /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-2-root/wals/14cf1c3a63624d94aa6c9c356e6108e2
I20260709 12:17:10.814489  6394 catalog_manager.cc:5028] TS f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): tablet 14cf1c3a63624d94aa6c9c356e6108e2 (table test-workload [id=f93362d0518e483fbe3a01f43a217e48]) successfully deleted
I20260709 12:17:10.823078  6568 tablet_service.cc:1558] Processing DeleteTablet for tablet 6d6c58127aec4100b761ec4b52b842a0 with delete_type TABLET_DATA_TOMBSTONED (TS 496c7c3792cd4587b4dbeba435f4be1e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56112
I20260709 12:17:10.856913  7607 tablet_replica.cc:333] T 6d6c58127aec4100b761ec4b52b842a0 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:17:10.862886  7607 raft_consensus.cc:2243] T 6d6c58127aec4100b761ec4b52b842a0 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:10.864126  7607 raft_consensus.cc:2272] T 6d6c58127aec4100b761ec4b52b842a0 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:10.871336  7607 ts_tablet_manager.cc:1916] T 6d6c58127aec4100b761ec4b52b842a0 P 496c7c3792cd4587b4dbeba435f4be1e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:10.888903  7607 ts_tablet_manager.cc:1929] T 6d6c58127aec4100b761ec4b52b842a0 P 496c7c3792cd4587b4dbeba435f4be1e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:10.889307  7607 log.cc:1199] T 6d6c58127aec4100b761ec4b52b842a0 P 496c7c3792cd4587b4dbeba435f4be1e: Deleting WAL directory at /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-1-root/wals/6d6c58127aec4100b761ec4b52b842a0
I20260709 12:17:10.892220  6392 catalog_manager.cc:5028] TS 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): tablet 6d6c58127aec4100b761ec4b52b842a0 (table test-workload [id=f93362d0518e483fbe3a01f43a217e48]) successfully deleted
I20260709 12:17:10.988472  6501 consensus_queue.cc:237] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:10.993826  7273 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:10.993980  6654 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:10.995596  7602 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:10.997455  7602 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:11.003647  7608 raft_consensus.cc:2955] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:11.006220  6650 raft_consensus.cc:2955] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:11.014918  7273 raft_consensus.cc:2955] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:11.018455  6392 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8e01550f5e994006a4031ca81edcc9aa with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:11.025297  6402 catalog_manager.cc:5697] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 reported cstate change: config changed from index 3 to 4, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:11.041952  6568 tablet_service.cc:1558] Processing DeleteTablet for tablet 8e01550f5e994006a4031ca81edcc9aa with delete_type TABLET_DATA_TOMBSTONED (TS 496c7c3792cd4587b4dbeba435f4be1e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56112
I20260709 12:17:11.043078  7607 tablet_replica.cc:333] T 8e01550f5e994006a4031ca81edcc9aa P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:17:11.043880  7607 raft_consensus.cc:2243] T 8e01550f5e994006a4031ca81edcc9aa P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:11.044643  7607 raft_consensus.cc:2272] T 8e01550f5e994006a4031ca81edcc9aa P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:11.049170  7607 ts_tablet_manager.cc:1916] T 8e01550f5e994006a4031ca81edcc9aa P 496c7c3792cd4587b4dbeba435f4be1e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:11.064560  7607 ts_tablet_manager.cc:1929] T 8e01550f5e994006a4031ca81edcc9aa P 496c7c3792cd4587b4dbeba435f4be1e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:11.064965  7607 log.cc:1199] T 8e01550f5e994006a4031ca81edcc9aa P 496c7c3792cd4587b4dbeba435f4be1e: Deleting WAL directory at /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-1-root/wals/8e01550f5e994006a4031ca81edcc9aa
I20260709 12:17:11.067929  6392 catalog_manager.cc:5028] TS 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): tablet 8e01550f5e994006a4031ca81edcc9aa (table test-workload [id=f93362d0518e483fbe3a01f43a217e48]) successfully deleted
I20260709 12:17:12.110517  6503 consensus_queue.cc:237] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:12.116487  6654 raft_consensus.cc:1275] T f9f8f5d393744ebabbed2f3e0131cf40 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:12.116864  6474 consensus_peers.cc:597] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f9f8f5d393744ebabbed2f3e0131cf40. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:12.117970  7608 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:12.118253  6582 raft_consensus.cc:1275] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:12.119663  7592 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:12.133138  7592 raft_consensus.cc:2955] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:12.146806  6654 raft_consensus.cc:2955] T f9f8f5d393744ebabbed2f3e0131cf40 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:12.153931  6582 raft_consensus.cc:2955] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:12.164419  6401 catalog_manager.cc:5697] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:12.165652  6650 consensus_queue.cc:237] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:12.199998  6503 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:12.200105  6582 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:12.201151  6623 consensus_peers.cc:597] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5b17fd01574f4601b33dd37e7c78bc8f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:12.201820  7587 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:12.203715  7609 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:12.219524  7619 raft_consensus.cc:2955] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:12.225032  6582 raft_consensus.cc:2955] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:12.225806  6501 raft_consensus.cc:2955] T 5b17fd01574f4601b33dd37e7c78bc8f P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:12.244668  6576 consensus_queue.cc:237] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:12.256184  6406 catalog_manager.cc:5697] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 reported cstate change: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:12.257469  6650 raft_consensus.cc:1275] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:12.257818  6549 consensus_peers.cc:597] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e6167e4ddf454440bfc602ffbcd6da3e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:12.259406  7597 consensus_queue.cc:1048] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:12.267853  6503 raft_consensus.cc:1275] T e6167e4ddf454440bfc602ffbcd6da3e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:12.269534  7574 consensus_queue.cc:1048] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:12.280300  7574 raft_consensus.cc:2955] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:12.282300  6654 raft_consensus.cc:2955] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:12.286667  6503 raft_consensus.cc:2955] T e6167e4ddf454440bfc602ffbcd6da3e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:12.300390  6501 consensus_queue.cc:237] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:12.303306  6402 catalog_manager.cc:5697] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:12.309090  6654 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:12.317430  6474 consensus_peers.cc:597] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c9d981014b2c4cda80bca766fe58fe42. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:12.317878  6576 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:12.318837  7592 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:12.319957  7615 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:12.335075  7613 raft_consensus.cc:2955] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:12.337409  6576 raft_consensus.cc:2955] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:12.345234  6654 raft_consensus.cc:2955] T c9d981014b2c4cda80bca766fe58fe42 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:12.356890  6406 catalog_manager.cc:5697] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:12.678306  7631 ts_tablet_manager.cc:933] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:12.690320  7631 tablet_copy_client.cc:323] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.2:45101
I20260709 12:17:12.692399  6594 tablet_copy_service.cc:140] P 496c7c3792cd4587b4dbeba435f4be1e: Received BeginTabletCopySession request for tablet e6167e4ddf454440bfc602ffbcd6da3e from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:54204)
I20260709 12:17:12.693205  6594 tablet_copy_service.cc:161] P 496c7c3792cd4587b4dbeba435f4be1e: Beginning new tablet copy session on tablet e6167e4ddf454440bfc602ffbcd6da3e from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:54204: session id = 20a202c76244485fbc8f1610473db8b9-e6167e4ddf454440bfc602ffbcd6da3e
I20260709 12:17:12.727123  7634 ts_tablet_manager.cc:933] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:12.727632  6594 tablet_copy_source_session.cc:215] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:12.734573  7631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6167e4ddf454440bfc602ffbcd6da3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.786862  7631 tablet_copy_client.cc:806] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:12.787824  7631 tablet_copy_client.cc:670] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:12.794903  7631 tablet_copy_client.cc:538] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:12.797914  7633 ts_tablet_manager.cc:933] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643)
I20260709 12:17:12.814916  7634 tablet_copy_client.cc:323] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.3:36107
I20260709 12:17:12.814806  7638 ts_tablet_manager.cc:933] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643)
I20260709 12:17:12.816718  6668 tablet_copy_service.cc:140] P f2b06aebe97249a4877d798b8118ece4: Received BeginTabletCopySession request for tablet 5b17fd01574f4601b33dd37e7c78bc8f from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:49792)
I20260709 12:17:12.817257  6668 tablet_copy_service.cc:161] P f2b06aebe97249a4877d798b8118ece4: Beginning new tablet copy session on tablet 5b17fd01574f4601b33dd37e7c78bc8f from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:49792: session id = 20a202c76244485fbc8f1610473db8b9-5b17fd01574f4601b33dd37e7c78bc8f
I20260709 12:17:12.817797  7633 tablet_copy_client.cc:323] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.1:36643
I20260709 12:17:12.819557  7631 tablet_bootstrap.cc:492] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
I20260709 12:17:12.821604  6518 tablet_copy_service.cc:140] P e7554c39679a41bbbca4659b72ab5c65: Received BeginTabletCopySession request for tablet f9f8f5d393744ebabbed2f3e0131cf40 from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:41304)
I20260709 12:17:12.822211  6518 tablet_copy_service.cc:161] P e7554c39679a41bbbca4659b72ab5c65: Beginning new tablet copy session on tablet f9f8f5d393744ebabbed2f3e0131cf40 from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:41304: session id = 20a202c76244485fbc8f1610473db8b9-f9f8f5d393744ebabbed2f3e0131cf40
I20260709 12:17:12.831970  6518 tablet_copy_source_session.cc:215] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:12.831970  6668 tablet_copy_source_session.cc:215] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:12.834582  7638 tablet_copy_client.cc:323] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.1:36643
I20260709 12:17:12.835798  7634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b17fd01574f4601b33dd37e7c78bc8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.869724  6518 tablet_copy_service.cc:140] P e7554c39679a41bbbca4659b72ab5c65: Received BeginTabletCopySession request for tablet c9d981014b2c4cda80bca766fe58fe42 from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:41304)
I20260709 12:17:12.871122  6518 tablet_copy_service.cc:161] P e7554c39679a41bbbca4659b72ab5c65: Beginning new tablet copy session on tablet c9d981014b2c4cda80bca766fe58fe42 from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:41304: session id = 20a202c76244485fbc8f1610473db8b9-c9d981014b2c4cda80bca766fe58fe42
I20260709 12:17:12.884851  7633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9f8f5d393744ebabbed2f3e0131cf40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:12.949275  6518 tablet_copy_source_session.cc:215] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:12.972921  7633 tablet_copy_client.cc:806] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:12.973810  7633 tablet_copy_client.cc:670] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:13.003649  7638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9d981014b2c4cda80bca766fe58fe42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:13.020720  7631 tablet_bootstrap.cc:492] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:13.025955  7633 tablet_copy_client.cc:538] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:17:13.041379  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 54616us
I20260709 12:17:13.073649  7631 tablet_bootstrap.cc:492] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:17:13.082769  7631 ts_tablet_manager.cc:1403] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 0.263s	user 0.028s	sys 0.004s
I20260709 12:17:13.048379  7633 tablet_bootstrap.cc:492] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
I20260709 12:17:13.130824  7634 tablet_copy_client.cc:806] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:13.131474  7634 tablet_copy_client.cc:670] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:13.142819  7631 raft_consensus.cc:359] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:13.146638  7634 tablet_copy_client.cc:538] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:13.147118  7631 raft_consensus.cc:740] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
W20260709 12:17:13.156855  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 55771.6us
I20260709 12:17:13.180624  7631 consensus_queue.cc:260] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:13.187528  7634 tablet_bootstrap.cc:492] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
I20260709 12:17:13.206993  7633 tablet_bootstrap.cc:492] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:13.208166  7633 tablet_bootstrap.cc:492] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:17:13.200099  7638 tablet_copy_client.cc:806] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:13.209087  7638 tablet_copy_client.cc:670] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:13.211130  7633 ts_tablet_manager.cc:1403] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 0.163s	user 0.031s	sys 0.019s
I20260709 12:17:13.213698  7638 tablet_copy_client.cc:538] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:13.215476  7631 ts_tablet_manager.cc:1434] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.132s	user 0.015s	sys 0.000s
I20260709 12:17:13.224536  7638 tablet_bootstrap.cc:492] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
I20260709 12:17:13.230563  6594 tablet_copy_service.cc:342] P 496c7c3792cd4587b4dbeba435f4be1e: Request end of tablet copy session 20a202c76244485fbc8f1610473db8b9-e6167e4ddf454440bfc602ffbcd6da3e received from {username='slave'} at 127.0.0.1:54204
I20260709 12:17:13.231246  6594 tablet_copy_service.cc:434] P 496c7c3792cd4587b4dbeba435f4be1e: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-e6167e4ddf454440bfc602ffbcd6da3e on tablet e6167e4ddf454440bfc602ffbcd6da3e with peer 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:13.231594  7274 raft_consensus.cc:1217] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:13.233912  7633 raft_consensus.cc:359] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:13.234897  7633 raft_consensus.cc:740] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
I20260709 12:17:13.235685  7633 consensus_queue.cc:260] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:13.251042  7633 ts_tablet_manager.cc:1434] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.004s
I20260709 12:17:13.255911  6518 tablet_copy_service.cc:342] P e7554c39679a41bbbca4659b72ab5c65: Request end of tablet copy session 20a202c76244485fbc8f1610473db8b9-f9f8f5d393744ebabbed2f3e0131cf40 received from {username='slave'} at 127.0.0.1:41304
I20260709 12:17:13.256484  6518 tablet_copy_service.cc:434] P e7554c39679a41bbbca4659b72ab5c65: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-f9f8f5d393744ebabbed2f3e0131cf40 on tablet f9f8f5d393744ebabbed2f3e0131cf40 with peer 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:13.272697  7638 tablet_bootstrap.cc:492] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:13.273900  7638 tablet_bootstrap.cc:492] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:17:13.274713  7638 ts_tablet_manager.cc:1403] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 0.051s	user 0.027s	sys 0.008s
I20260709 12:17:13.277771  7638 raft_consensus.cc:359] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:13.278676  7638 raft_consensus.cc:740] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
I20260709 12:17:13.279389  7638 consensus_queue.cc:260] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:13.282063  7638 ts_tablet_manager.cc:1434] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:17:13.284170  6518 tablet_copy_service.cc:342] P e7554c39679a41bbbca4659b72ab5c65: Request end of tablet copy session 20a202c76244485fbc8f1610473db8b9-c9d981014b2c4cda80bca766fe58fe42 received from {username='slave'} at 127.0.0.1:41304
I20260709 12:17:13.284626  6518 tablet_copy_service.cc:434] P e7554c39679a41bbbca4659b72ab5c65: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-c9d981014b2c4cda80bca766fe58fe42 on tablet c9d981014b2c4cda80bca766fe58fe42 with peer 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:13.301924  7634 tablet_bootstrap.cc:492] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:13.302417  7270 raft_consensus.cc:1217] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:13.303261  7634 tablet_bootstrap.cc:492] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:17:13.308418  7634 ts_tablet_manager.cc:1403] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 0.128s	user 0.028s	sys 0.001s
I20260709 12:17:13.312398  7634 raft_consensus.cc:359] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:13.320925  7634 raft_consensus.cc:740] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
I20260709 12:17:13.330859  7634 consensus_queue.cc:260] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:13.336154  7634 ts_tablet_manager.cc:1434] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 12:17:13.338594  6668 tablet_copy_service.cc:342] P f2b06aebe97249a4877d798b8118ece4: Request end of tablet copy session 20a202c76244485fbc8f1610473db8b9-5b17fd01574f4601b33dd37e7c78bc8f received from {username='slave'} at 127.0.0.1:49792
I20260709 12:17:13.339043  6668 tablet_copy_service.cc:434] P f2b06aebe97249a4877d798b8118ece4: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-5b17fd01574f4601b33dd37e7c78bc8f on tablet 5b17fd01574f4601b33dd37e7c78bc8f with peer 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:13.667037  7274 raft_consensus.cc:1217] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:13.729326  7592 raft_consensus.cc:1064] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:17:13.742365  7274 raft_consensus.cc:1217] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:13.742316  7592 consensus_queue.cc:237] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:13.764009  6654 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:13.764550  7274 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:13.765810  7592 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:13.766716  6576 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:13.767812  7642 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:13.769969  7613 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:13.787192  7574 raft_consensus.cc:1064] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:17:13.790009  7574 consensus_queue.cc:237] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:13.791484  7613 raft_consensus.cc:2955] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:13.802096  6654 raft_consensus.cc:2955] T c9d981014b2c4cda80bca766fe58fe42 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:13.804497  7274 raft_consensus.cc:2955] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:13.833856  6406 catalog_manager.cc:5697] T c9d981014b2c4cda80bca766fe58fe42 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:13.844759  7274 raft_consensus.cc:1275] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:13.844588  6578 raft_consensus.cc:2955] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:13.849938  6650 raft_consensus.cc:1275] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:13.853008  7625 consensus_queue.cc:1048] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:13.855495  6501 raft_consensus.cc:1275] T e6167e4ddf454440bfc602ffbcd6da3e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:13.856927  7625 consensus_queue.cc:1048] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:13.871920  7643 consensus_queue.cc:1048] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:13.886363  7574 raft_consensus.cc:2955] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:13.892012  6650 raft_consensus.cc:2955] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:13.908300  7274 raft_consensus.cc:2955] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:13.909140  6501 raft_consensus.cc:2955] T e6167e4ddf454440bfc602ffbcd6da3e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:13.912001  6503 consensus_queue.cc:237] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:13.931429  6578 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:13.933741  7592 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:13.937947  7274 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:13.925261  6403 catalog_manager.cc:5697] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 reported cstate change: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:13.941741  7642 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:13.949306  7657 raft_consensus.cc:2955] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:13.952798  6578 raft_consensus.cc:2955] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:13.955804  6576 consensus_queue.cc:237] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:13.962461  7270 raft_consensus.cc:1275] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:13.961992  7274 raft_consensus.cc:2955] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:13.965137  7625 consensus_queue.cc:1048] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:13.971714  6650 raft_consensus.cc:1275] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:13.973558  7625 consensus_queue.cc:1048] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:13.976869  7645 raft_consensus.cc:2955] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:13.994130  6403 catalog_manager.cc:5697] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: config changed from index 3 to 4, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:13.996487  6650 raft_consensus.cc:2955] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:13.998910  6392 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c9d981014b2c4cda80bca766fe58fe42 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:14.003633  7270 raft_consensus.cc:2955] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:14.014112  6640 tablet_service.cc:1558] Processing DeleteTablet for tablet c9d981014b2c4cda80bca766fe58fe42 with delete_type TABLET_DATA_TOMBSTONED (TS f2b06aebe97249a4877d798b8118ece4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58970
I20260709 12:17:14.016016  6403 catalog_manager.cc:5697] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 reported cstate change: config changed from index 3 to 4, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:14.021847  7664 tablet_replica.cc:333] T c9d981014b2c4cda80bca766fe58fe42 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:17:14.023653  7664 raft_consensus.cc:2243] T c9d981014b2c4cda80bca766fe58fe42 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:14.024747  7664 raft_consensus.cc:2272] T c9d981014b2c4cda80bca766fe58fe42 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:14.030762  7664 ts_tablet_manager.cc:1916] T c9d981014b2c4cda80bca766fe58fe42 P f2b06aebe97249a4877d798b8118ece4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:14.031409  6392 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e6167e4ddf454440bfc602ffbcd6da3e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:14.038904  6494 tablet_service.cc:1558] Processing DeleteTablet for tablet e6167e4ddf454440bfc602ffbcd6da3e with delete_type TABLET_DATA_TOMBSTONED (TS e7554c39679a41bbbca4659b72ab5c65 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45870
I20260709 12:17:14.055554  7664 ts_tablet_manager.cc:1929] T c9d981014b2c4cda80bca766fe58fe42 P f2b06aebe97249a4877d798b8118ece4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:14.056392  7664 log.cc:1199] T c9d981014b2c4cda80bca766fe58fe42 P f2b06aebe97249a4877d798b8118ece4: Deleting WAL directory at /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-2-root/wals/c9d981014b2c4cda80bca766fe58fe42
I20260709 12:17:14.059301  6394 catalog_manager.cc:5028] TS f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): tablet c9d981014b2c4cda80bca766fe58fe42 (table test-workload [id=f93362d0518e483fbe3a01f43a217e48]) successfully deleted
I20260709 12:17:14.060169  7665 tablet_replica.cc:333] T e6167e4ddf454440bfc602ffbcd6da3e P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:17:14.061498  7665 raft_consensus.cc:2243] T e6167e4ddf454440bfc602ffbcd6da3e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:14.063361  7665 raft_consensus.cc:2272] T e6167e4ddf454440bfc602ffbcd6da3e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:14.067270  7665 ts_tablet_manager.cc:1916] T e6167e4ddf454440bfc602ffbcd6da3e P e7554c39679a41bbbca4659b72ab5c65: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:14.085455  7665 ts_tablet_manager.cc:1929] T e6167e4ddf454440bfc602ffbcd6da3e P e7554c39679a41bbbca4659b72ab5c65: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:14.085875  7665 log.cc:1199] T e6167e4ddf454440bfc602ffbcd6da3e P e7554c39679a41bbbca4659b72ab5c65: Deleting WAL directory at /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-0-root/wals/e6167e4ddf454440bfc602ffbcd6da3e
I20260709 12:17:14.087782  6392 catalog_manager.cc:5028] TS e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): tablet e6167e4ddf454440bfc602ffbcd6da3e (table test-workload [id=f93362d0518e483fbe3a01f43a217e48]) successfully deleted
I20260709 12:17:14.179577  7479 raft_consensus.cc:1064] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:17:14.181905  7479 consensus_queue.cc:237] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:14.201479  7270 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:14.202749  6582 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:14.202860  6503 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:14.203106  7619 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:14.204421  7479 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:14.207623  7609 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:14.213961  7657 raft_consensus.cc:1064] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:17:14.216632  7657 consensus_queue.cc:237] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:14.237133  7479 raft_consensus.cc:2955] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:14.239105  7270 raft_consensus.cc:2955] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:14.266666  6401 catalog_manager.cc:5697] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 reported cstate change: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:14.288056  6650 raft_consensus.cc:1275] T f9f8f5d393744ebabbed2f3e0131cf40 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:14.290169  7642 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:14.292285  7274 raft_consensus.cc:1275] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:14.293951  7592 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:17:14.297994  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 61036.8us
I20260709 12:17:14.303231  6582 raft_consensus.cc:1275] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:14.304114  6576 raft_consensus.cc:2955] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:14.310180  7657 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:14.319115  6651 consensus_queue.cc:237] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:14.323798  6504 raft_consensus.cc:2955] T 5b17fd01574f4601b33dd37e7c78bc8f P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:14.355602  7270 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:14.357585  7672 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:14.367899  6582 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:14.368556  7619 raft_consensus.cc:2955] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:14.370589  7270 raft_consensus.cc:2955] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:14.395859  7609 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:14.395768  7642 raft_consensus.cc:2955] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:14.400627  6394 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5b17fd01574f4601b33dd37e7c78bc8f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:14.403680  6401 catalog_manager.cc:5697] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 reported cstate change: config changed from index 3 to 4, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:14.408217  6578 raft_consensus.cc:2955] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:14.408258  6651 raft_consensus.cc:2955] T f9f8f5d393744ebabbed2f3e0131cf40 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:14.413582  6494 tablet_service.cc:1558] Processing DeleteTablet for tablet 5b17fd01574f4601b33dd37e7c78bc8f with delete_type TABLET_DATA_TOMBSTONED (TS e7554c39679a41bbbca4659b72ab5c65 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45870
I20260709 12:17:14.414594  7665 tablet_replica.cc:333] T 5b17fd01574f4601b33dd37e7c78bc8f P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:17:14.415589  7665 raft_consensus.cc:2243] T 5b17fd01574f4601b33dd37e7c78bc8f P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:14.417910  6582 raft_consensus.cc:2955] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:14.420789  7665 raft_consensus.cc:2272] T 5b17fd01574f4601b33dd37e7c78bc8f P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:14.424438  7665 ts_tablet_manager.cc:1916] T 5b17fd01574f4601b33dd37e7c78bc8f P e7554c39679a41bbbca4659b72ab5c65: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:14.433007  6402 catalog_manager.cc:5697] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:14.450105  7274 raft_consensus.cc:2955] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:14.493713  7665 ts_tablet_manager.cc:1929] T 5b17fd01574f4601b33dd37e7c78bc8f P e7554c39679a41bbbca4659b72ab5c65: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:14.498440  6504 consensus_queue.cc:237] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:14.505940  7665 log.cc:1199] T 5b17fd01574f4601b33dd37e7c78bc8f P e7554c39679a41bbbca4659b72ab5c65: Deleting WAL directory at /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-0-root/wals/5b17fd01574f4601b33dd37e7c78bc8f
I20260709 12:17:14.509932  6392 catalog_manager.cc:5028] TS e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): tablet 5b17fd01574f4601b33dd37e7c78bc8f (table test-workload [id=f93362d0518e483fbe3a01f43a217e48]) successfully deleted
I20260709 12:17:14.510682  6582 raft_consensus.cc:1275] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:14.513402  7642 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:14.515806  7273 raft_consensus.cc:1275] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:14.518846  7642 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:14.521085  7642 raft_consensus.cc:2955] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:14.522914  6582 raft_consensus.cc:2955] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:14.532001  7274 raft_consensus.cc:2955] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:14.550908  6403 catalog_manager.cc:5697] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: config changed from index 3 to 4, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New cstate: current_term: 1 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:14.569029  6640 tablet_service.cc:1558] Processing DeleteTablet for tablet f9f8f5d393744ebabbed2f3e0131cf40 with delete_type TABLET_DATA_TOMBSTONED (TS f2b06aebe97249a4877d798b8118ece4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58970
I20260709 12:17:14.572963  6392 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f9f8f5d393744ebabbed2f3e0131cf40 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:17:14.585186  7676 tablet_replica.cc:333] T f9f8f5d393744ebabbed2f3e0131cf40 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:17:14.586411  7676 raft_consensus.cc:2243] T f9f8f5d393744ebabbed2f3e0131cf40 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:14.587208  7676 raft_consensus.cc:2272] T f9f8f5d393744ebabbed2f3e0131cf40 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:14.592209  7676 ts_tablet_manager.cc:1916] T f9f8f5d393744ebabbed2f3e0131cf40 P f2b06aebe97249a4877d798b8118ece4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:17:14.609831  7676 ts_tablet_manager.cc:1929] T f9f8f5d393744ebabbed2f3e0131cf40 P f2b06aebe97249a4877d798b8118ece4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:17:14.610158  7676 log.cc:1199] T f9f8f5d393744ebabbed2f3e0131cf40 P f2b06aebe97249a4877d798b8118ece4: Deleting WAL directory at /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-2-root/wals/f9f8f5d393744ebabbed2f3e0131cf40
I20260709 12:17:14.611570  6394 catalog_manager.cc:5028] TS f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): tablet f9f8f5d393744ebabbed2f3e0131cf40 (table test-workload [id=f93362d0518e483fbe3a01f43a217e48]) successfully deleted
I20260709 12:17:15.665326  6654 consensus_queue.cc:237] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:15.671823  6582 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:15.671916  6623 consensus_peers.cc:597] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c22b6d97ae844b7195c5e9c024203d48. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:15.673373  7679 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:15.673738  6501 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:15.675320  7619 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:15.682133  7679 raft_consensus.cc:2955] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:15.686044  6582 raft_consensus.cc:2955] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:15.686875  6504 raft_consensus.cc:2955] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:15.726186  6403 catalog_manager.cc:5697] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:15.730822  6582 consensus_queue.cc:237] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:15.741276  6654 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:17:15.741390  6549 consensus_peers.cc:597] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a47d71c34b541b9b2589b4e80413fa2. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:15.743492  7574 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:15.744145  6504 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:15.745714  7574 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:15.752955  7666 raft_consensus.cc:2955] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:15.754802  6654 raft_consensus.cc:2955] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:15.765519  6504 raft_consensus.cc:2955] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:15.777797  6654 consensus_queue.cc:237] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:15.781891  6402 catalog_manager.cc:5697] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
W20260709 12:17:15.803340  6623 consensus_peers.cc:597] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6f94b1cd286648a9b82d5abf69bb86d5. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:15.805166  6504 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:15.806806  6582 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:15.808907  7619 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:15.809995  7609 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:15.822338  7684 raft_consensus.cc:2955] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:15.827136  6501 raft_consensus.cc:2955] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:15.844625  6654 consensus_queue.cc:237] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:15.844322  6582 raft_consensus.cc:2955] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:15.848016  6402 catalog_manager.cc:5697] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:15.864189  6623 consensus_peers.cc:597] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e0ab356fc074f589a6ff4f9b4996b09. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:15.865615  6578 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:15.867523  6504 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:17:15.868863  7684 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:15.870153  7609 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:15.889186  7619 raft_consensus.cc:2955] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:15.894999  6578 raft_consensus.cc:2955] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:15.895998  6504 raft_consensus.cc:2955] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:15.924589  6402 catalog_manager.cc:5697] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: config changed from index -1 to 2, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 1 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:16.146134  7700 ts_tablet_manager.cc:933] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:16.151422  7700 tablet_copy_client.cc:323] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.3:36107
I20260709 12:17:16.153337  6668 tablet_copy_service.cc:140] P f2b06aebe97249a4877d798b8118ece4: Received BeginTabletCopySession request for tablet c22b6d97ae844b7195c5e9c024203d48 from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:49792)
I20260709 12:17:16.153816  6668 tablet_copy_service.cc:161] P f2b06aebe97249a4877d798b8118ece4: Beginning new tablet copy session on tablet c22b6d97ae844b7195c5e9c024203d48 from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:49792: session id = 20a202c76244485fbc8f1610473db8b9-c22b6d97ae844b7195c5e9c024203d48
I20260709 12:17:16.159904  6668 tablet_copy_source_session.cc:215] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:16.163321  7700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c22b6d97ae844b7195c5e9c024203d48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:16.180032  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:16.184741  7700 tablet_copy_client.cc:806] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:16.185434  7700 tablet_copy_client.cc:670] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:16.191108  7700 tablet_copy_client.cc:538] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:16.214366  7700 tablet_bootstrap.cc:492] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
I20260709 12:17:16.223893  7702 ts_tablet_manager.cc:933] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:16.235116  7702 tablet_copy_client.cc:323] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.3:36107
I20260709 12:17:16.236903  6668 tablet_copy_service.cc:140] P f2b06aebe97249a4877d798b8118ece4: Received BeginTabletCopySession request for tablet 6f94b1cd286648a9b82d5abf69bb86d5 from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:49792)
I20260709 12:17:16.237536  6668 tablet_copy_service.cc:161] P f2b06aebe97249a4877d798b8118ece4: Beginning new tablet copy session on tablet 6f94b1cd286648a9b82d5abf69bb86d5 from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:49792: session id = 20a202c76244485fbc8f1610473db8b9-6f94b1cd286648a9b82d5abf69bb86d5
I20260709 12:17:16.246017  6668 tablet_copy_source_session.cc:215] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:16.250010  7702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f94b1cd286648a9b82d5abf69bb86d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:16.268867  7702 tablet_copy_client.cc:806] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:16.269572  7702 tablet_copy_client.cc:670] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:16.274433  7702 tablet_copy_client.cc:538] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:16.279685  7700 tablet_bootstrap.cc:492] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:16.280828  7700 tablet_bootstrap.cc:492] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:17:16.281872  7700 ts_tablet_manager.cc:1403] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 0.068s	user 0.013s	sys 0.016s
I20260709 12:17:16.285467  7702 tablet_bootstrap.cc:492] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
I20260709 12:17:16.287838  7700 raft_consensus.cc:359] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:16.288939  7700 raft_consensus.cc:740] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
I20260709 12:17:16.300921  7700 consensus_queue.cc:260] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:16.324358  7708 ts_tablet_manager.cc:933] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:16.329159  7700 ts_tablet_manager.cc:1434] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.047s	user 0.013s	sys 0.002s
I20260709 12:17:16.332576  6668 tablet_copy_service.cc:342] P f2b06aebe97249a4877d798b8118ece4: Request end of tablet copy session 20a202c76244485fbc8f1610473db8b9-c22b6d97ae844b7195c5e9c024203d48 received from {username='slave'} at 127.0.0.1:49792
I20260709 12:17:16.333210  6668 tablet_copy_service.cc:434] P f2b06aebe97249a4877d798b8118ece4: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-c22b6d97ae844b7195c5e9c024203d48 on tablet c22b6d97ae844b7195c5e9c024203d48 with peer 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:16.337227  7705 ts_tablet_manager.cc:933] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:16.344386  7708 tablet_copy_client.cc:323] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.2:45101
I20260709 12:17:16.346242  6594 tablet_copy_service.cc:140] P 496c7c3792cd4587b4dbeba435f4be1e: Received BeginTabletCopySession request for tablet 3a47d71c34b541b9b2589b4e80413fa2 from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:54204)
I20260709 12:17:16.346777  6594 tablet_copy_service.cc:161] P 496c7c3792cd4587b4dbeba435f4be1e: Beginning new tablet copy session on tablet 3a47d71c34b541b9b2589b4e80413fa2 from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:54204: session id = 20a202c76244485fbc8f1610473db8b9-3a47d71c34b541b9b2589b4e80413fa2
I20260709 12:17:16.348306  7702 tablet_bootstrap.cc:492] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:16.349332  7702 tablet_bootstrap.cc:492] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:17:16.350211  7702 ts_tablet_manager.cc:1403] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 0.065s	user 0.021s	sys 0.014s
I20260709 12:17:16.353420  7702 raft_consensus.cc:359] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:16.354269  7702 raft_consensus.cc:740] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
I20260709 12:17:16.354897  7702 consensus_queue.cc:260] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:16.357879  7702 ts_tablet_manager.cc:1434] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:17:16.364359  6668 tablet_copy_service.cc:342] P f2b06aebe97249a4877d798b8118ece4: Request end of tablet copy session 20a202c76244485fbc8f1610473db8b9-6f94b1cd286648a9b82d5abf69bb86d5 received from {username='slave'} at 127.0.0.1:49792
I20260709 12:17:16.364825  6668 tablet_copy_service.cc:434] P f2b06aebe97249a4877d798b8118ece4: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-6f94b1cd286648a9b82d5abf69bb86d5 on tablet 6f94b1cd286648a9b82d5abf69bb86d5 with peer 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:16.365468  6594 tablet_copy_source_session.cc:215] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:16.373672  7708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a47d71c34b541b9b2589b4e80413fa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:16.381639  6582 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:17:16.382511  6582 raft_consensus.cc:606] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:16.392004  6582 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "20a202c76244485fbc8f1610473db8b9"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:17:16.392819  6582 raft_consensus.cc:606] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Received request to transfer leadership to TS 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:16.407218  7708 tablet_copy_client.cc:806] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:16.407918  7708 tablet_copy_client.cc:670] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:16.412194  7705 tablet_copy_client.cc:323] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.3:36107
I20260709 12:17:16.414512  6668 tablet_copy_service.cc:140] P f2b06aebe97249a4877d798b8118ece4: Received BeginTabletCopySession request for tablet 3e0ab356fc074f589a6ff4f9b4996b09 from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:49792)
I20260709 12:17:16.415177  6668 tablet_copy_service.cc:161] P f2b06aebe97249a4877d798b8118ece4: Beginning new tablet copy session on tablet 3e0ab356fc074f589a6ff4f9b4996b09 from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:49792: session id = 20a202c76244485fbc8f1610473db8b9-3e0ab356fc074f589a6ff4f9b4996b09
I20260709 12:17:16.424079  7708 tablet_copy_client.cc:538] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:16.455725  6668 tablet_copy_source_session.cc:215] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:17:16.460119  7708 tablet_bootstrap.cc:492] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
I20260709 12:17:16.463593  7705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e0ab356fc074f589a6ff4f9b4996b09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:16.525506  6650 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c27974d342ea4901aaa3946871539818"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:55116
I20260709 12:17:16.526188  6650 raft_consensus.cc:606] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Received request to transfer leadership to TS 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:16.528813  7705 tablet_copy_client.cc:806] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
I20260709 12:17:16.529726  7705 tablet_copy_client.cc:670] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:17:16.534998  7705 tablet_copy_client.cc:538] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:17:16.776415  7705 tablet_bootstrap.cc:492] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
I20260709 12:17:16.799116  7694 raft_consensus.cc:993] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e: : Instructing follower 20a202c76244485fbc8f1610473db8b9 to start an election
I20260709 12:17:16.799962  7677 raft_consensus.cc:1081] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Signalling peer 20a202c76244485fbc8f1610473db8b9 to start an election
W20260709 12:17:16.804517  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 243962us
W20260709 12:17:17.591388  6548 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.770s	user 0.017s	sys 0.031s
W20260709 12:17:17.626574  6548 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.017s	sys 0.031s
W20260709 12:17:17.672762  6622 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.006s	sys 0.018s
W20260709 12:17:17.708082  6622 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.711s	user 0.006s	sys 0.018s
W20260709 12:17:17.802861  6472 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.010s	sys 0.005s
W20260709 12:17:17.811064  6475 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.012s	sys 0.005s
W20260709 12:17:17.811651  6475 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.013s	sys 0.005s
W20260709 12:17:18.008961  6621 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.008s	sys 0.007s
W20260709 12:17:18.009595  6621 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.030s	user 0.009s	sys 0.007s
W20260709 12:17:18.289876  6548 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.000s	sys 0.011s
W20260709 12:17:18.291258  6548 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.001s	sys 0.011s
W20260709 12:17:18.431576  6472 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.004s	sys 0.025s
W20260709 12:17:18.432768  6472 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.004s	sys 0.025s
W20260709 12:17:18.023712  6530 debug-util.cc:398] Leaking SignalData structure 0x7b08000cd200 after lost signal to thread 5355
W20260709 12:17:18.472539  6549 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.002s	sys 0.012s
W20260709 12:17:18.473090  6549 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.002s	sys 0.012s
I20260709 12:17:18.523267  7717 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:18.523958  7717 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.526204  7717 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:18.409499  7716 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:18.548121  7716 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.618405  7716 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:18.618880  7719 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:18.631572  6582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 795) took 1807 ms. Trace:
I20260709 12:17:18.632089  6582 rpcz_store.cc:276] 0709 12:17:16.824283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.630672 (+1806389us) service_pool.cc:224] Handling call
0709 12:17:18.631043 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.631236 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:18.631264 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:18.631301 (+    37us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:18.631328 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.631384 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:18.631506 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:18.707811  7715 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:17:18.710958  6473 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.822s	user 0.017s	sys 0.002s
W20260709 12:17:18.713135  6473 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.379s	user 0.018s	sys 0.002s
I20260709 12:17:18.708472  7715 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.719399  7715 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:18.720738  7717 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:18.721403  7717 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:18.730865  7715 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:18.728219  6401 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57914 (request call id 185) took 1751 ms. Trace:
I20260709 12:17:18.734190  6401 rpcz_store.cc:276] 0709 12:17:17.017971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:17.018315 (+   344us) service_pool.cc:224] Handling call
0709 12:17:18.728110 (+1709795us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20260709 12:17:18.739188  7735 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:18.716538  7716 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:18.735767  7726 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:17:18.740522  6624 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.003s	sys 0.004s
I20260709 12:17:18.740646  7722 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:18.742027  7740 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:18.747255  7740 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.751405  7740 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:18.765132  7723 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:18.765712  7723 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.742067  7716 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:18.742393  6624 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.944s	user 0.004s	sys 0.004s
W20260709 12:17:18.776626  6547 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.797s	user 0.005s	sys 0.011s
W20260709 12:17:18.777081  6547 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.063s	user 0.005s	sys 0.012s
I20260709 12:17:18.778297  7728 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:18.780239  7732 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:18.782599  7733 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:18.743973  7657 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:18.709643  7719 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.748950  7715 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.775298  7717 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:18.785614  7642 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:18.786186  7657 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.787712  7721 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:18.774681  7726 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.786378  7738 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:18.745106  7741 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:18.790745  7642 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
W20260709 12:17:18.792161  6622 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.049s	sys 0.103s
W20260709 12:17:18.805902  6622 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.050s	sys 0.103s
I20260709 12:17:18.778930  7728 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.810060  7642 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:18.783177  7733 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:17:18.813194  7737 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:18.813655  7657 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:18.764432  7735 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.746201  7722 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:18.801450  7738 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:18.827438  7716 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:18.820001  7717 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:18.813745  7737 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.831067  7716 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:18.835003  7716 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.835461  7719 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:18.835971  7719 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:18.836490  7719 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:18.836920  7716 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:18.837405  7716 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:18.837855  7716 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:18.838461  7719 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:18.829517  7740 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:18.823848  7731 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:18.840644  7733 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:18.841403  7737 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:18.841805  7720 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:18.841085  7734 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:18.790889  7732 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.802023  7721 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.839634  7723 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:18.871572  7716 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:18.815855  7741 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:18.879290  7741 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:18.855217  7740 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:18.859115  7722 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:18.839890  7717 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:18.887163  6530 debug-util.cc:398] Leaking SignalData structure 0x7b08000814c0 after lost signal to thread 6441
I20260709 12:17:18.846411  7726 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:18.895279  7732 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:18.897357  7716 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:18.897945  7716 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.901477  7738 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:18.903426  6576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 797) took 2078 ms. Trace:
I20260709 12:17:18.903939  6576 rpcz_store.cc:276] 0709 12:17:16.824903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.902423 (+2077520us) service_pool.cc:224] Handling call
0709 12:17:18.902795 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.902996 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:18.903037 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:18.903089 (+    52us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:18.903131 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.903201 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:18.903346 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:18.870417  7720 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.909548  7720 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:18.888978  7715 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:18.847988  7731 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.912009  7728 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:18.876497  7734 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.922214  7734 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:18.892359  7727 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:18.916822  7735 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:17:18.926026  6474 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.118s	user 0.018s	sys 0.039s
W20260709 12:17:18.926746  6474 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.121s	user 0.019s	sys 0.039s
I20260709 12:17:18.900141  7716 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:17:18.925875  6550 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.006s	sys 0.012s
I20260709 12:17:18.931221  7734 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:18.936008  7728 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:18.936553  7721 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:18.936956  7715 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:18.936836  7728 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.937558  7715 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.939697  7715 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:18.940178  7705 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 1.936s	user 0.012s	sys 0.013s
I20260709 12:17:18.940311  7728 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:18.943938  7705 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 1.940s	user 0.016s	sys 0.013s
I20260709 12:17:18.953473  7715 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:18.942540  7721 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:18.924247  7736 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:18.936592  7734 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.925675  7730 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:18.966743  7742 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:18.963348  7719 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:18.963093  7721 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.940675  7727 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:18.968539  6472 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.027s	sys 0.029s
I20260709 12:17:18.943579  7725 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:18.965684  7720 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:17:18.970750  6472 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.028s	sys 0.030s
I20260709 12:17:18.971521  7731 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:18.973841  7732 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:18.973198  7717 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:18.974663  7734 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:18.962682  7715 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.987294  7746 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:18.987393  7744 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:18.977977  7728 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:18.978700  7740 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:18.974097  7719 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.985956  7748 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:18.975658  7720 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:18.990325  7726 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:18.992027  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 801) took 2160 ms. Trace:
I20260709 12:17:18.992175  7751 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:18.992408  7738 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:18.988015  7746 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.993796  6576 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:17:18.978986  7725 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.994910  7715 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:18.994242  7657 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:18.996377  6581 rpcz_store.cc:276] 0709 12:17:16.826111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:18.978627 (+2152516us) service_pool.cc:224] Handling call
0709 12:17:18.985778 (+  7151us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:18.985964 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:18.985990 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:18.986030 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:18.986065 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:18.986127 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:18.986231 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:18.998472  6576 raft_consensus.cc:606] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:18.999516  7716 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:18.977075  7742 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.993578  7728 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.996605  7734 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:18.990458  7732 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.976929  7736 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.998927  7740 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:19.000099  7717 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:18.992390  7744 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:19.001886  7657 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:19.004015  7735 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:18.999861  7726 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.998016  7738 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.974515  7730 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:19.026907  7642 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:19.030022  7642 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:19.022398  7715 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:19.005075  7716 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:19.006297  7734 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:19.024729  7721 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.026788  7733 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:19.037315  7729 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:19.026873  7731 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:19.006886  7751 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:19.039161  7642 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.048020  7642 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:19.024883  7740 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:19.029254  7735 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:19.008589  7750 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:19.061474  7642 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:18.999017  7748 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:19.049329  7721 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:19.026731  7717 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:19.067865  7657 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.049258  7723 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:19.041857  7747 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:19.070660  7642 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.076229  7642 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:19.052587  7719 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.054508  7741 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:19.076747  7642 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:19.074514  7657 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:19.071195  7730 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.078073  7742 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.052443  7727 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.049293  7733 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:19.048970  7715 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:19.068657  7720 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.090065  7719 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:19.071197  7726 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.051852  7731 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:19.041925  7743 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:19.051505  7729 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:19.098310  7642 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:19.100194  7745 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:19.100414  7749 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:19.100811  7733 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:19.100973  7749 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:19.095845  7725 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:19.076426  7721 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:19.094444  7742 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:19.079723  7723 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:17:19.093459  7730 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:19.096664  7739 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:19.113539  7748 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.097446  7746 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.119771  7746 raft_consensus.cc:993] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e: : Instructing follower e7554c39679a41bbbca4659b72ab5c65 to start an election
I20260709 12:17:19.091493  7747 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:19.093014  7657 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:19.123095  7715 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.103408  7728 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.099438  7719 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:19.125536  7657 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.125631  7747 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:19.126179  7748 raft_consensus.cc:1081] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Signalling peer e7554c39679a41bbbca4659b72ab5c65 to start an election
I20260709 12:17:19.127092  7731 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:19.108536  7732 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.094373  7741 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:19.100821  7745 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:19.145452  7740 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:19.123818  7742 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:19.149192  7742 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.124065  7716 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:19.072283  7750 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:19.157446  7751 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:19.158327  7734 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.160418  7721 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:19.155965  7735 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:17:19.169598  6530 debug-util.cc:398] Leaking SignalData structure 0x7b080033d620 after lost signal to thread 6533
I20260709 12:17:19.142726  7738 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.106038  7743 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:19.176187  7717 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:19.165720  6403 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57888 (request call id 211) took 2166 ms. Trace:
I20260709 12:17:19.166491  7723 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:19.122351  7730 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:19.177111  6403 rpcz_store.cc:276] 0709 12:17:16.999811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:17.000130 (+   319us) service_pool.cc:224] Handling call
0709 12:17:19.162024 (+2161894us) spinlock_profiling.cc:243] Waited 74.9 ms on lock 0x7b540008ab28. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287c07b331 00007f287c0763fc 00007f287bc47e84 00007f287bd4604e 00007f287a2072a4 00007f287a207232 00007f287a207161 00007f287a2070dc 00007f287a206392 00007f28741c448c 00007f28741c38c6
0709 12:17:19.165549 (+  3525us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":0,"spinlock_wait_cycles":172181248}
I20260709 12:17:19.151448  7744 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.184607  7730 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.186138  7741 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:19.190078  7745 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:19.177124  7736 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.169658  7729 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.162705  7739 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:19.195055  7749 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:19.202333  7732 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:19.203194  7732 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:19.207940  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.207288  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.248417  6651 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:19.213712  7705 tablet_bootstrap.cc:492] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:19.208469  7728 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:19.202366  7729 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:19.261040  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.223959  7738 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:19.263834  6651 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:19.268213  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.269299  6654 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:19.258973  7744 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:19.266825  7738 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:19.266086  7747 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:19.273773  7739 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:19.273774  7732 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:19.274444  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 280654us
I20260709 12:17:19.271535  7657 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:19.278955  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.281697  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.272912  7754 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:19.266259  7729 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:19.283993  7726 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:19.284479  7755 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:19.266624  7728 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:19.278574  7705 tablet_bootstrap.cc:492] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:17:19.299775  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.285843  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.286424  6656 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:19.271368  7744 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:19.307608  7657 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:19.331156  7657 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.331552  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:19.331800  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:19.332170  6506 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:17:19.332473  6505 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:17:19.333012  7760 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:19.333379  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:19.333647  7728 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.334247  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:19.328591  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.341074  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:19.303520  7754 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:19.344236  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:19.337507  7705 ts_tablet_manager.cc:1403] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 2.780s	user 0.036s	sys 0.025s
I20260709 12:17:19.302798  7743 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.343637  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:19.309845  7750 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.350400  6581 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:19.358912  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.356626  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.358891  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:19.333582  7760 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:19.287060  7747 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:19.365927  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.365180  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:19.336881  6653 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:19.374621  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:19.375864  6503 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:17:19.369001  6578 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:19.369800  7759 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:19.372150  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:19.369823  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:19.369861  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:19.374701  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:19.382838  6581 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:19.380826  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:19.383668  6578 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:19.388981  6473 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:19.389303  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:19.390466  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:19.391600  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:19.310534  7726 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:19.392271  6576 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:19.382972  6582 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:19.394256  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:19.391695  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:19.349507  7728 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:19.394441  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:19.406971  7642 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:19.407428  7642 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:19.407888  7642 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.395450  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:19.396014  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:19.396452  7747 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:19.397001  7708 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 2.907s	user 0.075s	sys 0.146s
I20260709 12:17:19.374966  7743 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:19.388059  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:19.399513  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:19.327471  7755 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:19.422581  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:19.405014  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:19.415616  6581 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:19.442386  7743 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:19.443996  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:19.442838  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:19.402385  6582 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:19.450732  7642 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:19.414841  7760 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.445995  6579 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:19.454007  7760 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:19.454730  7760 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:19.455265  7762 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:19.459689  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:19.415200  7708 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 2.926s	user 0.076s	sys 0.146s
I20260709 12:17:19.461695  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:19.477479  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:19.453085  6583 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:19.405723  7733 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:19.457155  7762 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:19.404799  7754 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.506930  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:19.493232  6473 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:19.415817  7728 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:19.514143  7738 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.396996  7759 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:19.563143  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 70003us
I20260709 12:17:19.700722  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:19.702491  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:19.703292  6581 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:19.701428  6473 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:19.635371  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:19.725953  6473 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:17:19.613029  6530 debug-util.cc:398] Leaking SignalData structure 0x7b080033e9a0 after lost signal to thread 6608
W20260709 12:17:19.607708  7708 log.cc:927] Time spent T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 12:17:19.617302  7719 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.587287  7754 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:19.717660  7743 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.736721  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:19.799288  6581 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:19.505632  7733 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:19.539204  7744 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:19.771302  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:19.796958  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:19.643951  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:19.772266  7732 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:19.725253  7734 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:19.737233  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:19.797040  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:19.809852  7752 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:19.797878  7754 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:19.775063  7736 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:19.735075  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.816522  6507 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 1.
I20260709 12:17:19.817516  6508 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:17:19.608995  7705 raft_consensus.cc:359] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:19.831265  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:19.836799  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:19.837416  6578 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:19.832075  6504 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:17:19.811654  7732 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:19.841463  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.841543  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:19.844373  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:19.846903  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.847368  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.848240  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.848418  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.849503  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.849826  7763 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:19.850536  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.850314  7763 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:19.851804  7708 tablet_bootstrap.cc:492] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:19.852321  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:19.852700  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:19.853773  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.854857  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.855852  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.826215  7754 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.849514  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.860704  7765 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:19.860940  7763 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:19.830509  7752 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:19.836721  7705 raft_consensus.cc:740] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
I20260709 12:17:19.874944  6475 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:19.871563  7642 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.873948  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.837220  7755 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.845188  7732 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.836594  7766 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:19.878461  6475 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:17:19.882759  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 156903us
I20260709 12:17:19.818673  7734 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:19.860700  6530 debug-util.cc:398] Leaking SignalData structure 0x7b0800332640 after lost signal to thread 6682
I20260709 12:17:19.882988  6475 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:19.836896  7736 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:19.878304  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.883958  6473 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:19.877388  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.883608  7642 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:19.887993  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.874095  7764 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:19.878345  7752 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.897289  7642 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:19.905714  7642 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.889197  6475 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:19.881498  7708 tablet_bootstrap.cc:492] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:17:19.881202  7763 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:19.905993  6473 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:19.885147  7744 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:19.881717  7755 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:19.910955  6475 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:19.916925  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:19.911093  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.918807  6654 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:19.918984  7708 ts_tablet_manager.cc:1403] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 3.459s	user 0.108s	sys 0.162s
I20260709 12:17:19.919847  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.920301  6473 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:19.920588  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 769) took 1006 ms. Trace:
I20260709 12:17:19.921066  6655 rpcz_store.cc:276] 0709 12:17:18.878850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:19.543194 (+664344us) service_pool.cc:224] Handling call
0709 12:17:19.856997 (+313803us) spinlock_profiling.cc:243] Waited 43.9 ms on lock 0x7b5800015998. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:17:19.857240 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:19.857459 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:19.857487 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:19.857538 (+    51us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:19.864447 (+  6909us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:19.873960 (+  9513us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:19.881797 (+  7837us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":101037312}
I20260709 12:17:19.922642  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.922587  7708 raft_consensus.cc:359] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:19.923616  7708 raft_consensus.cc:740] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
I20260709 12:17:19.918958  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:19.896284  7743 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:19.925989  6473 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:19.914038  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.928527  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.873574  7765 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:19.909780  7727 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:19.924484  7708 consensus_queue.cc:260] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:19.938443  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:19.938702  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:19.939257  7733 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.939749  7238 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:19.940300  7726 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:19.940634  7708 ts_tablet_manager.cc:1434] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.003s
I20260709 12:17:19.940867  7727 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:19.941256  7727 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.932148  6654 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:19.911809  7744 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:19.887900  7766 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:19.924069  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.943389  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.938609  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:19.945986  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.931499  7761 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:19.931385  7743 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:17:19.942322  6530 debug-util.cc:398] Leaking SignalData structure 0x7b080025e200 after lost signal to thread 7298
I20260709 12:17:19.935473  7715 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:19.930544  7657 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:19.937613  7642 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:19.980895  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:19.924821  7763 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:19.958153  7762 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.954005  7733 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:19.958896  7744 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.913354  7764 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:19.953554  7768 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:19.997179  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.997519  7768 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:19.997972  7768 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:19.932638  7755 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:19.987277  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:19.998735  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.000108  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.000849  6578 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:17:19.967952  7743 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.974747  7657 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:17:19.988785  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 62830.6us
I20260709 12:17:19.975317  7759 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:19.988765  7642 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:19.982640  6530 debug-util.cc:398] Leaking SignalData structure 0x7b0800289c40 after lost signal to thread 7696
I20260709 12:17:19.968761  7761 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:19.970940  7715 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:19.996697  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.990541  6594 tablet_copy_service.cc:342] P 496c7c3792cd4587b4dbeba435f4be1e: Request end of tablet copy session 20a202c76244485fbc8f1610473db8b9-3a47d71c34b541b9b2589b4e80413fa2 received from {username='slave'} at 127.0.0.1:54204
I20260709 12:17:20.003631  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.971709  7767 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:19.910605  7705 consensus_queue.cc:260] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:20.006858  7657 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.007334  7772 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:20.007429  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.005841  6473 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:20.020861  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:19.990293  7733 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:20.044174  7733 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.996155  7762 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:17:20.021688  6530 debug-util.cc:398] Leaking SignalData structure 0x7b080032a960 after lost signal to thread 7709
I20260709 12:17:20.049827  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:20.050786  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 45030.6us
I20260709 12:17:20.054824  6473 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:20.056013  7642 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:20.056383  7642 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:20.056712  7642 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.057075  6473 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:20.063839  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.064100  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.027771  6594 tablet_copy_service.cc:434] P 496c7c3792cd4587b4dbeba435f4be1e: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-3a47d71c34b541b9b2589b4e80413fa2 on tablet 3a47d71c34b541b9b2589b4e80413fa2 with peer 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:20.064960  6503 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:17:20.016880  7761 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.017035  7715 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.078826  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.080379  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.026527  6652 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:20.088121  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.028070  7772 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:20.090544  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:20.091168  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.091292  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.047015  7771 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:17:20.093528  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 182640us
I20260709 12:17:20.045903  7762 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:20.101907  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.098439  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.103523  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.104456  6475 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:20.067919  7770 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:20.065744  7774 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:20.107976  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.029160  7767 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:20.092779  7734 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.121841  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.069351  7769 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:20.095274  7736 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:20.096784  7772 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.137315  7274 raft_consensus.cc:1217] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:20.137113  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.120682  7773 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:20.166203  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:20.128170  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.163003  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.111789  7774 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:20.183007  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.183708  6650 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:20.190475  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.130167  7769 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:20.106600  7771 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:20.198184  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.201722  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.203254  7273 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9"
dest_uuid: "20a202c76244485fbc8f1610473db8b9"
 from {username='slave'} at 127.0.0.1:38562
I20260709 12:17:20.215441  7273 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:20.215862  7273 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.203787  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.204511  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.206801  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.207842  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.163239  7776 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:20.168236  7729 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.128086  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.227090  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.216562  7777 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:20.188162  7774 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.197103  7780 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:20.233449  7755 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.202900  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.124027  7770 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:20.142369  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.191347  7765 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.214198  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.187886  7728 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.236903  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.238734  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.262849  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.239241  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.264566  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.265443  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.235906  7776 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:20.267848  7776 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:20.251652  7777 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:20.234088  7763 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:17:20.244239  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 139834us
I20260709 12:17:20.248080  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.199916  7769 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.253046  7760 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:20.254348  7705 ts_tablet_manager.cc:1434] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.893s	user 0.019s	sys 0.004s
I20260709 12:17:20.255004  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.256083  7771 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.194581  7775 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:20.225559  7766 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.239388  6473 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:20.281582  6475 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:20.284395  7765 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:17:20.293799  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 171699us
I20260709 12:17:20.280421  7273 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:20.298174  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 58853.4us
I20260709 12:17:20.306825  6473 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:20.296622  7642 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:20.294337  7705 ts_tablet_manager.cc:1467] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9: Tablet startup took 943ms
I20260709 12:17:20.319187  6475 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:20.325973  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:20.326618  7274 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:17:20.331086  7642 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.313938  7726 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.316094  7773 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:20.343705  7642 raft_consensus.cc:2804] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:20.346822  7642 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:20.347250  7642 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.320189  7770 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.348905  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.341075  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.296931  7775 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:20.359292  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.359717  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.361816  7269 raft_consensus.cc:1217] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:20.259171  7780 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:20.314239  7765 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:20.364857  7764 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:20.349756  7773 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:20.398787  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.366613  7762 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.399003  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:20.406988  7767 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.408610  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.408756  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.411609  7273 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:20.383970  7726 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:17:20.423374  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 89408.8us
I20260709 12:17:20.428169  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:20.430467  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 123781us
I20260709 12:17:20.432790  6473 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:20.395612  7773 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.439303  6473 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:20.435099  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 115969us
I20260709 12:17:20.441329  6475 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:20.318166  7705 ts_tablet_manager.cc:1469] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9: Trace:
0709 12:17:16.301858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:16.302004 (+   146us) service_pool.cc:224] Handling call
0709 12:17:16.337609 (+ 35605us) ts_tablet_manager.cc:934] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
0709 12:17:16.462237 (+124628us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:17:16.511589 (+ 49352us) tablet_metadata.cc:665] Metadata flushed
0709 12:17:16.511715 (+   126us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:17:16.520953 (+  9238us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:17:16.521542 (+   589us) tablet_replica.cc:206] Creating consensus instance
0709 12:17:16.521982 (+   440us) ts_tablet_manager.cc:989] Replica registered.
0709 12:17:16.522052 (+    70us) inbound_call.cc:177] Queueing success response
0709 12:17:16.557099 (+ 35047us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:17:19.360929 (+2803830us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:17:19.397841 (+ 36912us) tablet_replica.cc:261] Starting instrumentation
0709 12:17:19.398899 (+  1058us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":356,"mutex_wait_us":23262,"raft.queue_time_us":134545,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":25724,"tablet-copy.queue_time_us":33089,"tablet-download-pool.queue_time_us":275,"tablet-download-pool.run_cpu_time_us":1401,"tablet-download-pool.run_wall_time_us":4296,"thread_start_us":81609,"threads_started":2}
I20260709 12:17:20.445191  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.454799  6475 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:20.433662  7726 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:20.440943  7729 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:20.458562  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:20.459232  7274 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:17:20.466089  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:20.467435  7787 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.468946  6475 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:17:20.469676  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 151933us
I20260709 12:17:20.440495  7271 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:17:20.472975  6475 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:20.473882  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.475286  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.454938  7747 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:20.457309  7726 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.456125  7785 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.461150  7729 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:20.475150  7728 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:20.491451  7270 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:20.481600  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.485509  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.477751  7715 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:20.504949  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:20.502167  6622 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:20.483994  7729 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.483711  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:20.508121  7269 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:17:20.487057  7728 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:17:20.521073  7754 consensus_meta.cc:287] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.645s	user 0.014s	sys 0.005s
I20260709 12:17:20.521557  7754 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:20.523324  7780 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.523772  7754 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.525444  7657 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:20.526000  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.526569  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.527438  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.529162  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.529618  6503 raft_consensus.cc:1240] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:20.530244  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:20.522003  7744 consensus_meta.cc:287] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.517s	user 0.007s	sys 0.006s
I20260709 12:17:20.530731  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.532634  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:20.533653  7752 consensus_meta.cc:287] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.622s	user 0.013s	sys 0.000s
I20260709 12:17:20.538480  7777 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.541844  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.513798  7731 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:20.542750  7731 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:20.543044  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.543167  7731 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.545454  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:20.546084  6582 raft_consensus.cc:3055] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:20.546438  6582 raft_consensus.cc:740] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:20.548094  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.528644  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.517333  7728 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.553925  7272 raft_consensus.cc:1217] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:20.518078  7785 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.529232  7775 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.551944  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.553088  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:20.547972  7737 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:20.572854  7737 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:20.570782  7733 consensus_meta.cc:287] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.526s	user 0.010s	sys 0.013s
I20260709 12:17:20.573993  7733 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:20.575140  7737 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:20.553505  7774 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:20.548416  6582 consensus_queue.cc:260] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:20.556661  6549 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:17:20.560108  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:20.545748  7744 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:20.570850  7732 consensus_meta.cc:287] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.683s	user 0.015s	sys 0.006s
I20260709 12:17:20.560104  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.579670  6582 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.580698  7272 raft_consensus.cc:1217] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:17:20.577414  7642 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:20.592686  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.594555  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.596323  7744 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.587960  7732 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:20.599119  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:20.599731  7732 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.601665  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:20.548972  7752 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:20.603293  6549 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:17:20.588104  7734 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:20.605304  7642 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.584494  7765 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:20.604434  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:20.611631  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:17:20.613385  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 141134us
I20260709 12:17:20.616914  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.609048  7734 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:20.615203  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:20.626977  7774 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.627511  7786 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:20.644625  7786 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:20.645031  7786 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.648079  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:20.622830  6473 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:20.631281  6549 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:20.632251  7734 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:20.631435  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 158516us
I20260709 12:17:20.613372  7715 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.594589  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:20.625562  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.634117  7657 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.638648  7726 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:20.639087  7761 consensus_meta.cc:287] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.558s	user 0.001s	sys 0.012s
W20260709 12:17:20.640856  7768 consensus_meta.cc:287] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.642s	user 0.013s	sys 0.002s
I20260709 12:17:20.628365  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.652511  6475 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:17:20.666739  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 44010.8us
I20260709 12:17:20.654865  6582 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
I20260709 12:17:20.669384  6623 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:20.647549  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:20.656981  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:20.658605  7727 consensus_meta.cc:287] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.717s	user 0.010s	sys 0.005s
I20260709 12:17:20.646498  7733 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:20.665337  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 158279us
I20260709 12:17:20.651763  7736 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.662755  6655 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 1.
I20260709 12:17:20.667914  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.672183  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:20.677018  7272 raft_consensus.cc:2393] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 2: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:17:20.690480  6623 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:20.691066  6475 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:20.691847  7790 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:20.692806  7726 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.693794  7790 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:20.694247  7790 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.694391  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:17:20.696177  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:20.698726  6475 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:20.699317  7732 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:20.699414  7757 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:20.699960  7757 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:20.700189  7732 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.700399  7757 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.701273  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:20.701836  7274 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:17:20.702868  6475 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:20.704648  7732 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.706621  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.707868  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:20.670603  7761 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:17:20.671578  7768 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:20.673390  7727 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:20.712507  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:20.712576  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:20.700438  7752 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.713531  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:20.702508  7765 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:20.715406  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:20.716151  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:20.718868  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:20.716715  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:20.716202  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:20.723080  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:20.725270  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.723906  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:20.723778  7791 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.732064  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:20.694669  7769 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:20.725247  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.725673  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.726325  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.727392  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.727324  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:20.728118  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:20.734584  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.735221  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.735589  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.735828  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.736763  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.737033  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.737147  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.737737  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.738026  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.738440  6505 raft_consensus.cc:2393] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 2: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:20.728077  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:20.730180  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.724061  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:20.734969  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:20.732362  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:20.759302  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:20.762034  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.764039  7726 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:20.739068  7765 raft_consensus.cc:697] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 2 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:20.794314  7726 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:20.761307  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:20.795116  7765 consensus_queue.cc:237] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:20.765260  6509 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:20.796694  7726 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.765516  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.766855  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:20.773844  7768 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:20.774609  7772 consensus_meta.cc:287] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.605s	user 0.013s	sys 0.000s
I20260709 12:17:20.774268  7729 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:20.766208  7761 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:20.777962  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.778872  6507 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:45874
I20260709 12:17:20.801079  6507 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:20.801433  6507 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.802222  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:20.804180  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:20.779786  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.783602  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.738592  6502 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:20.802146  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.810775  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:20.784807  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.785281  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:20.790027  7769 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.792083  6549 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:17:20.792136  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:20.794708  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.765708  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.764199  7793 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:20.818511  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:20.805850  7772 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:20.820001  7270 raft_consensus.cc:1240] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:20.820148  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.821390  7793 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:20.821818  7793 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.818728  7729 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.824651  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.825278  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.834977  6506 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 1.
I20260709 12:17:20.825487  7728 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:20.835235  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 221031us
I20260709 12:17:20.837507  7728 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.838765  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.839468  6668 tablet_copy_service.cc:342] P f2b06aebe97249a4877d798b8118ece4: Request end of tablet copy session 20a202c76244485fbc8f1610473db8b9-3e0ab356fc074f589a6ff4f9b4996b09 received from {username='slave'} at 127.0.0.1:49792
I20260709 12:17:20.824703  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:17:20.826223  7743 consensus_meta.cc:287] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.809s	user 0.008s	sys 0.015s
I20260709 12:17:20.824703  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.838689  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.825271  7684 consensus_queue.cc:1059] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:20.840034  6668 tablet_copy_service.cc:434] P f2b06aebe97249a4877d798b8118ece4: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-3e0ab356fc074f589a6ff4f9b4996b09 on tablet 3e0ab356fc074f589a6ff4f9b4996b09 with peer 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:20.847263  6474 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:20.848652  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:20.851761  6548 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:20.852555  7574 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:20.852916  7574 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:20.852986  6548 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:20.853343  7574 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.845813  7773 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:20.844477  7743 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:20.849436  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.854178  6548 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:20.822615  7736 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:20.854024  6474 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:20.856696  7694 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:20.857955  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.859135  7240 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:20.860713  6474 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:20.863140  6474 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:20.864369  7240 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:20.868461  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:20.868820  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:20.869411  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.868137  7738 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.870062  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:20.871326  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:20.860913  7694 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:20.873240  7694 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.873273  7775 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.873590  7794 raft_consensus.cc:3055] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:20.873735  6548 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:20.875527  6548 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:20.877038  6507 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:20.877327  7240 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:20.857308  7790 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:20.874156  7768 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.878530  7719 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:20.874468  7769 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.878175  7738 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:20.879644  7738 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:20.880005  7738 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.880463  6651 raft_consensus.cc:1240] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:20.882948  7240 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:20.884192  7240 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:20.855135  7761 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.885203  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:20.884759  7574 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:20.860855  7736 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:20.874087  7794 raft_consensus.cc:740] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:20.886158  6548 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:20.911320  7574 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.889446  7777 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.928095  7574 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.907899  7767 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.908484  7736 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.885818  6475 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:20.891690  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 133716us
I20260709 12:17:20.874614  7752 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.868461  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:17:20.939054  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 53376.7us
I20260709 12:17:20.940999  6475 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:20.942026  6622 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:20.897944  6507 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.900990  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:20.945397  6583 raft_consensus.cc:3055] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:20.950738  6583 raft_consensus.cc:740] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:20.929033  7772 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.951795  6583 consensus_queue.cc:260] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:20.953238  6583 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.953959  7773 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.903648  7793 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:20.932473  7727 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.905572  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:20.885222  7775 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.888936  7780 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.890947  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:20.946910  7790 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:20.945232  7796 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.973033  7731 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:20.878364  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:20.974079  6652 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:20.904742  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:20.957468  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:20.894658  7240 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:20.977311  7757 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:20.977813  7780 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:20.978256  7780 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:20.978413  6656 raft_consensus.cc:1240] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:20.978611  7731 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:20.978680  7780 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.960237  6401 catalog_manager.cc:5697] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 1 to 2, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 2 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:20.962052  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:20.938285  7761 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.968077  7734 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:20.969331  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:20.969132  7694 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:20.971104  7793 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.980957  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:17:20.979755  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 85162.7us
I20260709 12:17:20.986763  7240 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:20.966131  7794 consensus_queue.cc:260] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:20.989101  7772 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.989255  7775 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.989784  7238 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.009043  7238 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:21.010151  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:21.010967  7767 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.011634  7767 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.012411  7767 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 2 LEADER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.013310  6583 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:21.013623  7786 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:21.014123  7790 raft_consensus.cc:1064] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
W20260709 12:17:20.987946  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 47013.8us
I20260709 12:17:20.993024  7743 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:20.979270  7757 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:20.994633  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:20.998346  7794 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:20.999043  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:20.993321  6582 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:20.998102  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:20.988835  7240 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.018077  6473 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:20.994493  7734 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:20.988262  7768 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.020876  7775 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:20.995898  7694 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:21.022346  7793 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:21.022812  7240 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.022913  7793 raft_consensus.cc:697] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 2 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:21.002378  7736 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:21.027513  6474 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.002820  7737 raft_consensus.cc:1064] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:17:21.011776  7780 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:21.036191  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 95944.8us
W20260709 12:17:21.036896  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 150938us
I20260709 12:17:21.030283  7775 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.038273  6475 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.043004  6578 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:21.043619  6575 raft_consensus.cc:2393] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 2: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:21.041867  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:21.044567  7793 consensus_queue.cc:237] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:21.037989  7727 consensus_queue.cc:1059] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 12:17:21.054292  7752 raft_consensus.cc:3055] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:21.054701  7752 raft_consensus.cc:740] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:21.037338  7734 consensus_queue.cc:1059] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 12:17:21.042178  7761 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.055538  7752 consensus_queue.cc:260] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:21.056614  7738 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:21.060863  7734 raft_consensus.cc:3055] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:21.061272  7734 raft_consensus.cc:740] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:21.059953  7761 consensus_queue.cc:1059] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:17:21.062282  7761 raft_consensus.cc:3055] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:21.062187  7734 consensus_queue.cc:260] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:21.049769  7773 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.063385  7734 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.064189  7738 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:21.052098  6577 raft_consensus.cc:1240] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:21.062664  7761 raft_consensus.cc:740] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:21.065510  7786 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:21.065310  7761 consensus_queue.cc:260] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:21.046547  6548 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.057027  7752 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.069885  6622 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:21.070992  6475 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.073247  6548 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.073877  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:21.076191  6548 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.071817  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:21.036260  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:21.071854  7694 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.103072  6475 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.105522  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:21.105719  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:21.107106  6548 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.076653  7743 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.036247  7790 consensus_queue.cc:237] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:21.076666  7803 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.100666  7780 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:21.113652  6548 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.101253  7761 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.096350  7804 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:21.119724  7804 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:21.073716  7727 consensus_queue.cc:1059] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 12:17:21.112491  6475 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.117022  7574 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.117609  6548 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.119602  7731 consensus_queue.cc:1059] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:17:21.128158  6501 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:21.128901  6505 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:21.103974  7798 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:21.110013  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:21.111698  7775 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:21.110582  7800 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:21.117393  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 81659.4us
I20260709 12:17:21.123525  7796 consensus_queue.cc:1059] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:21.123617  7759 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.115031  7785 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.124130  7736 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:21.124758  7728 raft_consensus.cc:3055] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:21.131974  6509 raft_consensus.cc:1240] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:21.132143  6502 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:21.132639  6500 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:21.133080  6507 raft_consensus.cc:1240] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:21.133561  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:21.133982  6504 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:21.161700  7757 raft_consensus.cc:3055] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:21.162122  7757 raft_consensus.cc:740] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:21.162416  6504 raft_consensus.cc:1240] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:21.162921  7757 consensus_queue.cc:260] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:21.164117  7799 raft_consensus.cc:1064] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:17:21.159708  7728 raft_consensus.cc:740] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:21.164109  7757 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.165340  7728 consensus_queue.cc:260] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:21.166621  7728 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.166667  7799 consensus_queue.cc:237] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:21.135035  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:21.115207  7738 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.137849  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:21.141215  6508 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 1. Current term is 2. Ops: []
W20260709 12:17:21.143071  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:21.120266  7804 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.145506  6475 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.173050  7785 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.173167  6505 raft_consensus.cc:2393] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 2: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:21.173288  6475 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.174261  7768 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.174664  6475 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.123028  6548 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.148619  6652 raft_consensus.cc:2393] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 2: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:21.149616  7798 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:21.176723  7798 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:21.177450  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 54498.2us
I20260709 12:17:21.178297  6548 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.179487  6548 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.180094  7795 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.180440  7800 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.141309  7775 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:21.181385  7775 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.154196  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:21.182911  6548 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.183465  6475 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.188125  6475 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.181398  7793 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.189922  7759 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:21.188788  7791 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.190586  6475 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.191064  7791 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.191288  7793 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.191720  7731 raft_consensus.cc:3055] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:21.192123  7731 raft_consensus.cc:740] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:21.192554  6475 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.194352  6475 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.195895  6548 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.200368  6548 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.202569  6548 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.203367  6475 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.184309  7773 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.207368  6475 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.199400  7729 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:21.212908  7802 raft_consensus.cc:1064] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:17:21.199586  7694 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:21.227891  6475 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.229146  6475 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.230221  6475 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.197379  7238 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.230556  7732 consensus_queue.cc:1059] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:21.205843  7801 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:21.231258  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 70741.7us
I20260709 12:17:21.231628  7732 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.232479  7727 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.232240  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:21.233103  7732 raft_consensus.cc:3055] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:21.233516  7732 raft_consensus.cc:740] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:21.234305  7732 consensus_queue.cc:260] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:21.235106  7772 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.199018  7769 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.215260  7798 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:21.197604  7793 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.222172  7775 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:21.229758  7802 consensus_queue.cc:237] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:21.235513  7732 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.218385  7737 consensus_queue.cc:237] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:21.235725  7803 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.219832  6622 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:21.236619  7780 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.193012  7731 consensus_queue.cc:260] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:21.236625  6548 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.230803  7785 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.244637  6548 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.248136  6548 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:21.204604  7738 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:21.251293  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:21.251423  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:21.251928  7757 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:21.252249  7801 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.252794  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:21.245883  7803 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.254029  7793 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.254385  7731 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.254563  7768 consensus_queue.cc:1059] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:21.255869  7238 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.274714  7757 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:21.280936  6649 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
I20260709 12:17:21.281566  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:21.259099  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:21.280929  7757 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.284677  7240 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:21.259765  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:21.280465  7694 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.286762  7767 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.287698  7767 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.290807  7804 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:21.292104  6652 raft_consensus.cc:2393] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 2: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:21.236079  7791 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.293629  7803 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.269694  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:21.268823  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:21.273478  6548 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.293736  7775 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:21.293078  7780 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:21.297606  6548 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.299093  6548 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.264578  6506 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
I20260709 12:17:21.300143  7759 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.297569  7780 raft_consensus.cc:697] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 2 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:21.302250  6622 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:21.304489  6475 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.308199  6622 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:21.312031  6577 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:17:21.312124  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:21.312913  6577 raft_consensus.cc:606] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:21.323047  6578 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
W20260709 12:17:21.324606  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:21.324370  6622 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:21.328264  6475 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.301061  7738 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.332315  6475 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.338055  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:21.339671  7270 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:21.332892  7761 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.340772  7761 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.340920  6504 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:21.341220  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:21.341771  6505 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.342159  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:21.342854  6503 raft_consensus.cc:2393] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 2: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:21.342442  7684 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:21.341796  6509 raft_consensus.cc:1240] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:21.333899  7788 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.344624  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:21.345163  6577 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:21.345206  6501 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:21.346519  6502 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:21.346684  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:21.347164  7804 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:21.348968  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:21.301607  7734 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.349664  6503 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 2: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:21.349779  6578 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:17:21.352918  6578 raft_consensus.cc:606] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:21.350704  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:21.351325  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:21.304287  7768 raft_consensus.cc:3055] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:21.304795  7781 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.354936  6500 raft_consensus.cc:2393] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 2: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:21.356644  6548 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.329895  7796 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.359407  7738 consensus_queue.cc:1059] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:17:21.330298  7728 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.404102  6507 raft_consensus.cc:1240] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:21.354628  7768 raft_consensus.cc:740] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:21.359493  7736 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.402549  7801 consensus_queue.cc:1059] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 12:17:21.416644  7768 consensus_queue.cc:260] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:21.418228  7768 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.411698  7780 consensus_queue.cc:237] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:21.416044  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:21.423092  6505 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
I20260709 12:17:21.423779  6504 raft_consensus.cc:1240] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:21.428035  6501 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:21.430651  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:21.431792  6653 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:21.432581  7798 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:21.407853  7765 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:17:21.401383  7757 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:21.412879  7269 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:21.435222  6650 raft_consensus.cc:1240] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:21.436254  6509 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:21.438513  7801 raft_consensus.cc:3055] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:21.438908  7801 raft_consensus.cc:740] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:21.441118  7757 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:21.442536  7765 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:21.440143  7801 consensus_queue.cc:260] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:17:21.443107  7765 raft_consensus.cc:697] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 2 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:21.443922  7801 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:21.445768  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 44371.4us
I20260709 12:17:21.446756  6548 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:21.463017  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 138782us
I20260709 12:17:21.463717  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:21.469630  6622 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:21.459816  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:17:21.444095  7765 consensus_queue.cc:237] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:21.472772  6622 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:21.474149  6548 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.475690  7794 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.473295  7798 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.478451  7798 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.479782  6622 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:21.481196  6622 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:21.481878  6508 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
I20260709 12:17:21.482511  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:21.484915  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:21.485491  7757 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.485886  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:21.487046  6622 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:21.449203  7804 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 12:17:21.493053  6548 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:21.453565  7738 consensus_queue.cc:1059] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 12:17:21.483093  7794 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.486266  7813 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.482940  7798 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.504366  7269 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:21.499531  6548 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:21.508172  7574 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:21.508647  7694 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:21.509263  7694 raft_consensus.cc:697] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:21.509421  6501 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:21.510227  7694 consensus_queue.cc:237] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:21.497586  6622 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:21.483134  7736 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.491699  7757 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.499311  7795 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:21.513408  6509 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:21.513918  7795 raft_consensus.cc:697] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:21.487421  7767 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.514810  6502 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:21.495496  7803 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.502554  7786 raft_consensus.cc:3055] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:21.499336  7815 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.516415  7786 raft_consensus.cc:740] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:21.516858  7815 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.517354  7786 consensus_queue.cc:260] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:21.514931  7795 consensus_queue.cc:237] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:21.519356  6622 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:21.519168  7786 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.524971  7734 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:21.525878  6649 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:21.526489  7757 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.529460  6622 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:21.527647  7759 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:17:21.535190  6548 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:21.535971  7736 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:21.532301  7719 consensus_queue.cc:1059] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 12:17:21.536562  7736 raft_consensus.cc:697] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:21.531677  6622 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:21.533573  7757 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.547132  6622 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:21.548843  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:21.549664  6503 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:21.550180  6622 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:21.547384  7815 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.552982  7804 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.553678  7798 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.555371  7800 raft_consensus.cc:3055] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:21.555791  7800 raft_consensus.cc:740] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:21.556772  7800 consensus_queue.cc:260] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:21.558584  7800 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.564517  6622 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:21.545027  7736 consensus_queue.cc:237] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:21.566681  7731 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:21.567217  7731 raft_consensus.cc:697] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:21.568190  7731 consensus_queue.cc:237] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:21.562808  7798 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.572468  7238 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:21.573614  7803 consensus_queue.cc:1059] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:21.573768  7767 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:21.574666  7767 raft_consensus.cc:697] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 2 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:21.575567  7767 consensus_queue.cc:237] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:21.577371  7274 raft_consensus.cc:1240] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 2 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:21.579042  7719 consensus_queue.cc:1059] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:21.580056  7719 consensus_queue.cc:1243] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer 20a202c76244485fbc8f1610473db8b9 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:17:21.580540  7574 raft_consensus.cc:3055] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:21.580937  7574 raft_consensus.cc:740] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:21.581796  7574 consensus_queue.cc:260] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:21.582736  7574 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.589030  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:21.588855  7803 raft_consensus.cc:3055] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:21.592674  7803 raft_consensus.cc:740] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:21.609155  7738 raft_consensus.cc:3055] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:21.609668  7738 raft_consensus.cc:740] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:21.610728  6475 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:21.610611  7738 consensus_queue.cc:260] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:21.610984  7715 consensus_queue.cc:1059] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:21.611478  6503 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:21.611914  7791 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:21.612076  7715 raft_consensus.cc:3055] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:21.612526  7791 raft_consensus.cc:697] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 2 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:21.612632  7715 raft_consensus.cc:740] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:17:21.618667  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:21.612105  7738 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.621842  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:21.618844  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:21.622997  6582 raft_consensus.cc:2393] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 2: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:21.625524  6658 raft_consensus.cc:1275] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:21.625200  7803 consensus_queue.cc:260] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:21.613793  7791 consensus_queue.cc:237] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:21.619045  7800 consensus_queue.cc:1059] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:17:21.637686  6507 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:21.642164  6578 raft_consensus.cc:1240] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:21.642823  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:21.644204  6500 raft_consensus.cc:1240] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:21.645287  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:21.653267  6503 raft_consensus.cc:1240] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:21.688832  7267 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 1 LEARNER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:21.654405  6575 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:21.688833  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:21.693096  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:17:21.702404  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 46648.1us
I20260709 12:17:21.654520  7801 raft_consensus.cc:3055] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:21.721354  6652 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:21.717718  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:17:21.733083  7819 log.cc:927] Time spent T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 12:17:21.660864  7715 consensus_queue.cc:260] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:21.703416  7804 consensus_queue.cc:1059] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 12:17:21.742395  7804 raft_consensus.cc:3055] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:21.742916  7804 raft_consensus.cc:740] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:21.647886  7785 consensus_queue.cc:1048] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.067s
I20260709 12:17:21.748878  7780 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:21.717309  7801 raft_consensus.cc:740] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:21.752969  7801 consensus_queue.cc:260] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:21.754807  7801 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.703100  7757 consensus_queue.cc:1059] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 12:17:21.757673  7757 raft_consensus.cc:3055] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:21.758164  7757 raft_consensus.cc:740] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:21.759269  7757 consensus_queue.cc:260] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:21.761093  7794 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:17:21.764573  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:21.769958  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:21.777426  6575 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:17:21.833834  7815 consensus_queue.cc:1059] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:21.834720  6622 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:21.835060  7815 consensus_queue.cc:1059] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:21.835330  7813 consensus_queue.cc:1059] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:21.834631  7684 consensus_queue.cc:1059] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:21.836169  7815 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.837797  7684 raft_consensus.cc:3055] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:21.836481  7790 raft_consensus.cc:3055] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:21.838375  7684 raft_consensus.cc:740] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:21.838486  7790 raft_consensus.cc:740] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:21.839388  7790 consensus_queue.cc:260] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:21.839388  7684 consensus_queue.cc:260] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:21.840592  7803 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.744056  7804 consensus_queue.cc:260] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:21.840662  7790 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.836059  7796 raft_consensus.cc:3055] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:21.845419  7796 raft_consensus.cc:740] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:21.846212  7796 consensus_queue.cc:260] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:21.847383  7796 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.836359  7813 raft_consensus.cc:3055] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:21.848904  7813 raft_consensus.cc:740] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:21.854525  6622 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:21.870095  7813 consensus_queue.cc:260] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:21.885388  6401 catalog_manager.cc:5697] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 1 to 2, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 2 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:21.898932  6502 raft_consensus.cc:1240] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:21.899389  7799 consensus_queue.cc:1059] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:21.902277  6508 raft_consensus.cc:2955] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:21.872200  7815 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:21.909746  6503 raft_consensus.cc:1240] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:21.890681  7684 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.901165  7813 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.872608  7781 consensus_queue.cc:1059] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 12:17:21.918668  7731 raft_consensus.cc:3055] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:21.891005  7802 consensus_queue.cc:1059] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 12:17:21.930238  7731 raft_consensus.cc:740] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:21.932924  6577 raft_consensus.cc:1275] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:21.933437  7731 consensus_queue.cc:260] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:21.910383  7832 consensus_queue.cc:1059] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 12:17:21.944280  7829 consensus_queue.cc:1059] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:21.953871  7804 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.973848  7829 raft_consensus.cc:3055] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:21.974327  7829 raft_consensus.cc:740] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:21.977464  7832 raft_consensus.cc:3055] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:21.928939  7694 consensus_queue.cc:1059] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 12:17:22.013743  6508 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.982226  7832 raft_consensus.cc:740] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:22.017501  7832 consensus_queue.cc:260] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:22.013180  7829 consensus_queue.cc:260] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:22.005838  7807 log.cc:927] Time spent T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:17:21.946319  7785 consensus_queue.cc:1048] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.073s
I20260709 12:17:22.043865  6508 raft_consensus.cc:1275] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:17:22.046226  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 174769us
W20260709 12:17:22.054517  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:22.059049  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 87704us
W20260709 12:17:22.059625  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 79293.5us
W20260709 12:17:22.061380  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:22.066143  6578 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:22.069096  6508 raft_consensus.cc:1240] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:22.046868  7832 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:22.055395  7829 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:22.056650  7757 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:22.075493  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:22.066129  7780 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 12:17:22.086766  6509 raft_consensus.cc:1275] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:22.087875  7694 raft_consensus.cc:3055] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:22.088371  7694 raft_consensus.cc:740] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:22.084712  7270 raft_consensus.cc:2955] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:21.978226  7828 raft_consensus.cc:2955] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:22.064682  7836 consensus_queue.cc:1059] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 12:17:22.122171  7836 raft_consensus.cc:3055] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:22.122562  7836 raft_consensus.cc:740] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:22.124081  7715 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:22.150463  6502 raft_consensus.cc:1275] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:22.157289  7270 raft_consensus.cc:1275] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:22.089308  7694 consensus_queue.cc:260] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:22.123382  7836 consensus_queue.cc:260] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:22.195111  7836 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:22.179553  7800 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:22.200843  7803 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:22.215533  6504 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:17:22.204433  7788 raft_consensus.cc:3055] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:22.229525  6622 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:22.234615  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:22.221067  7694 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:22.195894  7794 consensus_queue.cc:1175] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "20a202c76244485fbc8f1610473db8b9" responder_term: 1 status { last_received { term: 1 index: 3 } last_committed_idx: 2 last_received_current_leader { term: 1 index: 3 } }
W20260709 12:17:22.216945  7737 consensus_queue.cc:1175] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e7554c39679a41bbbca4659b72ab5c65" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260709 12:17:22.232182  7790 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:17:22.245985  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:22.248262  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:22.237732  7684 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:22.231130  7788 raft_consensus.cc:740] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:22.259665  6502 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:22.260411  7804 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 12:17:22.287871  7574 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:22.275171  7781 raft_consensus.cc:3055] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:17:22.292599  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:22.294358  6652 raft_consensus.cc:1275] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:22.297788  7781 raft_consensus.cc:740] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:17:22.346067  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 284136us
I20260709 12:17:22.339728  7732 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.312040  7780 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:17:22.372881  7781 consensus_queue.cc:260] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:22.377523  7781 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:22.381973  7767 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:22.382696  6577 raft_consensus.cc:1275] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:22.382561  7767 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:22.386960  7844 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:22.327303  7788 consensus_queue.cc:260] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:22.360705  7732 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
W20260709 12:17:22.405448  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:22.429703  6577 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:22.431512  6649 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:22.433935  7731 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:22.437726  6403 catalog_manager.cc:5697] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 1 to 2, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 2 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:22.439464  6403 catalog_manager.cc:5697] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 1 to 2, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 2 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:22.441654  7732 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:22.442204  7769 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:22.442339  6403 catalog_manager.cc:5697] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 1 to 2, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 2 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:22.450893  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:22.452052  7788 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:22.451833  7765 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:22.459908  7274 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:22.464861  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:22.452759  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:22.480929  6581 raft_consensus.cc:1275] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:22.413740  7767 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:22.491710  6577 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:22.493155  6509 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:22.499547  7274 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:17:22.552719  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 88535.4us
I20260709 12:17:22.555720  7803 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:22.568464  6622 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:22.570199  7274 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:22.571547  7802 consensus_queue.cc:1059] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:22.574661  6473 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:22.576352  6651 raft_consensus.cc:2410] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 12:17:22.584991  6508 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:22.589968  7769 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:22.590569  7769 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:22.591035  7769 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:22.592033  6504 raft_consensus.cc:1275] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:22.604851  7274 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:22.576314  7802 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:22.543882  7775 raft_consensus.cc:476] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 2 LEADER]: Not starting pre-election -- already a leader
I20260709 12:17:22.589311  7794 raft_consensus.cc:3055] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:22.620551  7798 consensus_queue.cc:1048] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 12:17:22.680744  7794 raft_consensus.cc:740] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:22.638582  6650 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:22.618597  6649 raft_consensus.cc:2955] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:22.665448  7832 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:17:22.666182  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 70157us
I20260709 12:17:22.668107  7786 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:17:22.668804  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 79222.9us
I20260709 12:17:22.681813  7794 consensus_queue.cc:260] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:22.692484  7731 consensus_queue.cc:1048] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 12:17:22.700403  7802 raft_consensus.cc:697] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:22.710204  7791 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:17:22.716976  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 335294us
I20260709 12:17:22.739619  7785 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.733832  7732 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.742856  7785 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:22.747773  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:22.757612  6504 raft_consensus.cc:2393] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 2: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:22.754038  6503 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:17:22.776926  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:22.785710  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:22.786424  6650 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:22.788198  6475 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:22.776530  7785 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:22.761900  7732 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:22.789877  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:22.790870  6584 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:22.791853  7732 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:22.795516  7802 consensus_queue.cc:237] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:22.805766  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:22.806241  7765 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.806804  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:22.808221  6650 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:22.769675  7775 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:22.810860  6475 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:22.822095  7775 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:22.823452  7274 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:22.824187  7726 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:22.813827  7785 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.809192  7715 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:22.796929  7794 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:22.784375  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:22.824790  7726 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:22.839022  7785 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:22.806811  7765 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:22.833916  7781 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.858855  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:22.865242  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:22.837121  7715 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:22.848300  7726 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:22.879179  7715 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:22.879904  7785 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:22.917181  7574 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.917755  7574 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:17:22.873075  7780 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:17:22.892894  7791 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.925772  7791 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:22.928277  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:22.929131  6654 raft_consensus.cc:2393] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 2: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:22.873622  7781 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:22.939543  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:22.940392  6654 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:22.941792  6475 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:22.922483  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:22.954860  6657 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:22.955811  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:22.956067  7732 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.956584  6577 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:22.956724  7732 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:22.932564  7853 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:17:22.918323  7775 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:22.959569  7732 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:22.961256  6475 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:22.961818  7769 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:22.966367  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:22.964294  7781 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:22.967082  6652 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:22.936081  7828 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.967466  7732 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:22.969172  7828 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:22.923529  7738 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.971660  7738 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:22.974217  7574 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:22.951476  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:22.976506  7732 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:22.977049  7732 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:22.968981  6475 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:22.961177  7765 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:22.950035  7785 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:22.941610  7791 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:22.991626  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:23.012486  6401 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57888 (request call id 212) took 1392 ms. Trace:
I20260709 12:17:22.995908  7828 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:22.996411  6503 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:22.997238  7773 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.997856  7793 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.032407  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:23.032615  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:23.032703  7793 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:23.033788  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:23.000782  6402 catalog_manager.cc:5697] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 1 to 2, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 2 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:22.967490  7769 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:23.017450  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:23.040758  6505 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:17:23.004011  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:23.008817  7785 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:23.027257  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:23.031251  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:23.012631  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:23.049898  6650 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:23.033159  6577 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:22.998301  7738 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:23.036466  6655 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:17:23.042662  6402 catalog_manager.cc:5697] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 1 to 2, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 2 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:23.047288  6548 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:17:23.048856  6500 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.006s	sys 0.011s
I20260709 12:17:23.008836  7743 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:23.082084  7743 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:23.082528  7743 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:23.051648  6550 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:23.085047  6550 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:23.055346  7799 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:17:23.085978  6500 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.007s	sys 0.011s
I20260709 12:17:23.063635  6402 catalog_manager.cc:5697] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 1 to 2, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 2 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:23.013147  6401 rpcz_store.cc:276] 0709 12:17:21.624583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:21.632530 (+  7947us) service_pool.cc:224] Handling call
0709 12:17:21.700120 (+ 67590us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b540008acd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df3090 00007f2875e91f3a 00007f287bc47e9c 00007f287bd4604e 00007f287a2072a4 00007f287a207232 00007f287a207161 00007f287a2070dc 00007f287a206392 00007f28741c448c 00007f28741c38c6
0709 12:17:22.976041 (+1275921us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:21.908436 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:21.923141 (+ 14705us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:21.923173 (+    32us) write_op.cc:435] Acquired schema lock
0709 12:17:21.923185 (+    12us) tablet.cc:662] Decoding operations
0709 12:17:21.923493 (+   308us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:21.923548 (+    55us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:21.923574 (+    26us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:17:21.923749 (+   175us) tablet.cc:701] Row locks acquired
0709 12:17:21.923775 (+    26us) write_op.cc:260] PREPARE: finished
0709 12:17:21.923908 (+   133us) write_op.cc:270] Start()
0709 12:17:21.924008 (+   100us) write_op.cc:276] Timestamp: P: 1783599441923864 usec, L: 0
0709 12:17:21.924025 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 12:17:21.924608 (+   583us) log.cc:844] Serialized 531 byte log entry
0709 12:17:22.386304 (+461696us) op_driver.cc:464] REPLICATION: finished
0709 12:17:22.761124 (+374820us) write_op.cc:301] APPLY: starting
0709 12:17:22.761215 (+    91us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:22.761322 (+   107us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:22.761470 (+   148us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:22.793624 (+ 32154us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:22.793805 (+   181us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:17:22.793861 (+    56us) write_op.cc:312] APPLY: finished
0709 12:17:22.794056 (+   195us) log.cc:844] Serialized 22 byte log entry
0709 12:17:22.794281 (+   225us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:22.794378 (+    97us) write_op.cc:454] Released schema lock
0709 12:17:22.794417 (+    39us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:22.794785 (+   368us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:22.794814 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":2,"spinlock_wait_cycles":26692864,"child_traces":[["op",{"apply.queue_time_us":374545,"apply.run_cpu_time_us":2061,"apply.run_wall_time_us":33645,"num_ops":1,"prepare.queue_time_us":5610,"prepare.run_cpu_time_us":2337,"prepare.run_wall_time_us":20824,"replication_time_us":462101,"thread_start_us":190530,"threads_started":1}]]}
I20260709 12:17:23.055251  6584 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:23.051214  6657 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:17:23.079993  6652 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:23.080798  7769 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.081696  7726 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:23.050366  7785 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:23.085085  7738 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:23.032428  7773 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:23.088054  7574 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:23.112015  7574 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:23.090701  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:23.112543  7574 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:23.112763  6579 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:17:23.093786  7832 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:23.115087  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:23.099128  7796 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.093670  7757 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:23.122424  6507 raft_consensus.cc:2368] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 2. Current term is 3.
W20260709 12:17:23.123521  7272 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.005s	sys 0.020s
I20260709 12:17:23.087414  7799 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:23.127537  7272 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.006s	sys 0.020s
W20260709 12:17:23.112797  7270 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.011s	sys 0.012s
I20260709 12:17:23.108377  7829 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:23.129594  6506 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
W20260709 12:17:23.130131  7270 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.011s	sys 0.013s
I20260709 12:17:23.110461  7835 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:23.111699  7738 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:23.111799  7731 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:23.148881  7574 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:23.123414  7793 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:23.150388  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:23.158197  7574 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:23.128324  7815 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:23.162417  7815 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:23.162730  7859 log.cc:927] Time spent T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:17:23.162899  7815 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:23.110531  7769 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:23.151444  7738 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:23.128605  7796 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:23.138681  7757 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:23.148342  7829 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:23.150439  7835 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:23.160923  7731 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:23.247090  7731 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:23.245669  7832 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:23.265578  7832 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:23.268070  7799 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:23.276984  7835 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:23.279314  7796 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:17:23.318338  7858 log.cc:927] Time spent T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
W20260709 12:17:23.320685  7857 log.cc:927] Time spent T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
W20260709 12:17:23.322608  7863 log.cc:927] Time spent T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
W20260709 12:17:23.324546  7867 log.cc:927] Time spent T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 12:17:23.375682  7829 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:23.404814  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:17:23.407539  6509 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.015s	sys 0.011s
W20260709 12:17:23.408304  6509 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.015s	sys 0.011s
W20260709 12:17:23.415884  6504 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.033s	sys 0.031s
W20260709 12:17:23.418119  6504 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.034s	sys 0.031s
I20260709 12:17:23.450693  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:23.452555  6504 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:23.468688  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:23.470337  6500 raft_consensus.cc:2393] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 2: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:23.473047  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:23.473953  6651 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:23.479982  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:23.482255  6658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 37) took 1082 ms. Trace:
I20260709 12:17:23.482738  6658 rpcz_store.cc:276] 0709 12:17:22.399576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:22.399725 (+   149us) service_pool.cc:224] Handling call
0709 12:17:22.400112 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:22.400573 (+   461us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:22.400613 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:22.413463 (+ 12850us) log.cc:844] Serialized 26 byte log entry
0709 12:17:22.885039 (+471576us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:22.885083 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:22.885365 (+   282us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:23.481783 (+596418us) raft_consensus.cc:1685] finished
0709 12:17:23.481816 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:23.482070 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2419,"raft.queue_time_us":2842,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":7382,"thread_start_us":431878,"threads_started":1,"wal-append.queue_time_us":684776}
I20260709 12:17:23.484807  6657 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:23.485391  6651 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:23.481374  6650 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
W20260709 12:17:23.497395  7238 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.027s	sys 0.016s
W20260709 12:17:23.555248  6474 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.000s	sys 0.005s
W20260709 12:17:23.584481  6474 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.000s	sys 0.006s
W20260709 12:17:23.590808  7238 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.028s	sys 0.016s
I20260709 12:17:23.597304  7815 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:23.602926  6473 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.004s	sys 0.036s
W20260709 12:17:23.603605  6473 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.005s	sys 0.036s
I20260709 12:17:23.603549  7775 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:17:23.604267  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 585358us
I20260709 12:17:23.605638  7801 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.606673  7801 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:23.610267  6504 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:23.613284  7272 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38562 (request call id 296) took 1043 ms. Trace:
I20260709 12:17:23.613912  7272 rpcz_store.cc:276] 0709 12:17:22.569638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:22.570927 (+  1289us) service_pool.cc:224] Handling call
0709 12:17:22.571321 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:22.571725 (+   404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:22.571767 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:22.572361 (+   594us) log.cc:844] Serialized 26 byte log entry
0709 12:17:23.129733 (+557372us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:23.129778 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:23.129841 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:23.612471 (+482630us) raft_consensus.cc:1685] finished
0709 12:17:23.612519 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:23.612816 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":797,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":127,"spinlock_wait_cycles":22912,"thread_start_us":556993,"threads_started":1,"wal-append.queue_time_us":659222}
I20260709 12:17:23.619576  7796 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:23.624513  7802 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.045s	sys 0.106s
W20260709 12:17:23.625137  7802 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.045s	sys 0.106s
I20260709 12:17:23.630234  7788 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:17:23.632548  6472 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.022s	sys 0.017s
W20260709 12:17:23.633036  6472 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.022s	sys 0.017s
I20260709 12:17:23.277961  7769 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:23.636830  7769 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:23.637670  7769 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:23.643139  6475 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:23.604971  7775 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:23.664809  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 777506us
W20260709 12:17:23.675935  6577 consensus_meta.cc:287] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.587s	user 0.003s	sys 0.014s
I20260709 12:17:23.631641  7788 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:23.604890  7800 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:23.603582  7786 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:23.693378  6577 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
W20260709 12:17:23.692711  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 49928.1us
I20260709 12:17:23.676316  7694 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.697132  7796 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:23.729844  7694 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:23.731305  7801 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:23.732687  7881 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:23.733345  6652 consensus_meta.cc:287] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.568s	user 0.003s	sys 0.019s
I20260709 12:17:23.733701  7890 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.733565  7881 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:23.734442  7890 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:23.735391  7715 consensus_meta.cc:287] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.788s	user 0.008s	sys 0.002s
I20260709 12:17:23.693303  7769 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:23.750705  7726 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:17:23.755529  7743 consensus_meta.cc:287] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.672s	user 0.007s	sys 0.006s
I20260709 12:17:23.756075  7743 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:23.748504  7815 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:23.773283  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 1428) took 1217 ms. Trace:
W20260709 12:17:23.780184  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 207822us
I20260709 12:17:23.736223  7715 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:23.784531  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 51960.7us
I20260709 12:17:23.767413  6652 raft_consensus.cc:2410] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 12:17:23.762079  7800 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:23.756954  7871 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:23.848454  6475 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:17:23.860512  7794 consensus_meta.cc:287] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 1.015s	user 0.007s	sys 0.021s
I20260709 12:17:23.742343  7796 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:23.800058  7801 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.875471  7881 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:23.888237  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:23.883101  7726 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:23.903221  6650 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:23.900902  7803 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:23.910382  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:23.913322  7757 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:23.907876  7801 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:23.913838  6501 raft_consensus.cc:2393] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 3: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:23.873382  7786 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:23.931301  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 188649us
I20260709 12:17:23.898242  7694 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:23.924281  7801 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:23.935210  7757 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.950670  6475 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:17:23.951634  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 138625us
I20260709 12:17:23.984414  7768 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.949137  7890 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:23.941658  7726 raft_consensus.cc:697] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:17:23.996773  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 57411.2us
I20260709 12:17:23.916756  7871 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:24.004468  7238 leader_election.cc:400] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 2. Current term is 3.
I20260709 12:17:24.004864  7238 leader_election.cc:403] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:24.004935  7898 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.010875  6650 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:24.010933  7715 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:23.984400  7891 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.027725  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:24.028800  6650 raft_consensus.cc:2393] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 3: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:24.005651  7795 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.028501  7731 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.012190  7788 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:24.008841  7890 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:23.852442  6649 rpcz_store.cc:276] 0709 12:17:22.512100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:22.515489 (+  3389us) service_pool.cc:224] Handling call
0709 12:17:22.578167 (+ 62678us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:22.607017 (+ 28850us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:17:23.003856 (+396839us) log.cc:844] Serialized 14 byte log entry
0709 12:17:23.339381 (+335525us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:17:23.594043 (+254662us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:23.594235 (+   192us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:23.675731 (+ 81496us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:23.685289 (+  9558us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:23.729073 (+ 43784us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12764,"raft.queue_time_us":180732,"raft.run_cpu_time_us":374,"raft.run_wall_time_us":26424,"spinlock_wait_cycles":24122752,"thread_start_us":589246,"threads_started":2,"wal-append.queue_time_us":313084}
I20260709 12:17:24.045960  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:24.047549  7574 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:24.034098  7743 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.007553  7757 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:24.030576  7775 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.045976  7891 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.034571  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:24.013921  7803 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.012013  7898 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:24.059940  6548 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:24.073036  7871 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.073966  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:24.046033  7880 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:24.031885  7731 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.063867  7268 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Advancing to term 3
W20260709 12:17:24.082633  7807 log.cc:927] Time spent T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8: Append to log took a long time: real 0.210s	user 0.001s	sys 0.001s
I20260709 12:17:24.082310  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:24.042096  7768 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:24.085950  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 64781.1us
W20260709 12:17:24.092599  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 88455.3us
I20260709 12:17:24.094970  6654 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:24.095741  7768 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.096485  7899 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:24.096937  6656 raft_consensus.cc:2393] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 2: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:24.097333  7899 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:24.099905  7899 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:24.061316  7795 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.071924  7890 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.122674  7793 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:24.122597  7574 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.123677  7876 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:24.123637  7793 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:24.124272  7876 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:24.124783  7876 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:24.124832  7888 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:24.125469  7738 consensus_meta.cc:287] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.899s	user 0.011s	sys 0.006s
I20260709 12:17:24.125658  7888 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.125998  7738 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:17:24.108626  6657 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:17:24.114516  7800 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.120699  7768 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.128732  7726 consensus_queue.cc:237] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:24.129040  7768 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.130487  7892 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.131141  7892 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.132481  7694 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:24.133138  7694 raft_consensus.cc:697] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:24.134341  7773 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.134487  7694 consensus_queue.cc:237] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.147858  6654 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:24.162166  7896 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:24.163193  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 55020.2us
I20260709 12:17:24.163014  7896 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:24.163776  7898 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.164206  7238 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:24.165985  7891 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.159677  7900 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:24.166940  7900 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:24.167330  7900 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:24.114893  7880 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.169242  7890 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.170552  7880 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.171810  7268 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
W20260709 12:17:24.173130  7732 consensus_meta.cc:287] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 1.144s	user 0.006s	sys 0.012s
I20260709 12:17:24.173555  7732 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.174772  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:24.175496  6577 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:24.175959  7732 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.176980  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:24.181298  7832 consensus_meta.cc:287] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.894s	user 0.010s	sys 0.009s
I20260709 12:17:24.181962  7903 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:24.171984  7785 consensus_meta.cc:287] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.998s	user 0.012s	sys 0.007s
I20260709 12:17:24.199025  7879 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.200590  7882 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.201315  7882 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:24.159451  7871 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:24.192170  7786 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.230706  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:24.195370  6583 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:24.238001  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:24.241956  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:24.212543  7832 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.239339  7803 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.241869  6473 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:24.250480  7843 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:24.251082  7737 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:24.211460  7785 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.238070  7786 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:24.259744  7829 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:24.260041  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:24.259876  7895 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 2. Current term is 3.
I20260709 12:17:24.263084  7732 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:24.263938  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:24.199930  7879 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.267712  7768 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.295069  7803 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:24.301700  7829 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.302503  7795 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.302603  7803 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.303843  7757 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:24.268074  7769 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:24.269002  7896 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:24.269024  6501 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:24.273819  7882 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:24.277298  7574 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.279664  7731 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:24.282480  7269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38562 (request call id 301) took 1473 ms. Trace:
I20260709 12:17:24.283303  7793 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:24.239929  7775 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:24.238670  7871 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:24.318907  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 199877us
I20260709 12:17:24.305312  7803 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.235091  7903 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.317203  6500 raft_consensus.cc:2410] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:24.298692  7732 raft_consensus.cc:697] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:24.323213  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:24.336190  6656 raft_consensus.cc:2368] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 2. Current term is 3.
I20260709 12:17:24.298843  7890 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:24.344058  7916 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:24.345093  7916 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:24.332019  7574 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:17:24.349862  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 185816us
I20260709 12:17:24.352248  6502 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:24.352557  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:24.353435  6652 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:24.315431  7829 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.335440  7879 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.365522  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:24.368295  7240 leader_election.cc:400] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 2. Current term is 3.
I20260709 12:17:24.368839  7240 leader_election.cc:403] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:24.368978  7574 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:24.374814  6654 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
I20260709 12:17:24.356515  7800 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:24.321364  7786 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.322690  7769 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:24.373176  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:24.380762  7901 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.381340  7909 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.318667  7737 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:24.371129  7829 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:24.391993  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:24.409024  6650 raft_consensus.cc:2410] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 12:17:24.391070  6656 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:24.386751  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:24.376116  7890 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.394228  7775 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:24.403103  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:24.403566  7785 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.403106  7911 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:24.405153  7732 consensus_queue.cc:237] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:24.406497  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:24.406911  7737 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:24.418287  7574 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:24.423825  6508 raft_consensus.cc:2410] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:24.373049  7876 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.440621  7872 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:24.443186  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:24.443542  7872 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:24.414062  6550 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:24.391618  7731 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:24.432982  7912 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:24.433676  7919 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.407335  7897 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:24.321539  7843 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.434846  6579 raft_consensus.cc:2393] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 3: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:24.434993  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:24.435998  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:24.390314  7768 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.438650  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:24.423535  6475 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:24.394421  7800 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.487051  7786 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.486898  7903 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.486403  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:24.425161  6583 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:24.506702  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:24.487684  7919 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.488297  7757 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:24.431282  7900 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.487775  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:24.417059  7879 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:24.427351  7901 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.444978  7829 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.443241  7909 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:24.613960  7781 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:24.614532  7837 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:24.701242  7837 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.702250  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:24.703019  6653 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:24.703712  7837 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:24.703876  7574 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:24.704730  7781 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.615073  7785 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:24.580412  7912 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.615388  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:24.618551  7903 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:24.675508  7901 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.633493  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:24.675726  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:24.638002  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:24.639503  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:24.585435  7914 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:24.669415  7802 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:24.461898  7911 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.585816  7768 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.683234  7829 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.685751  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:24.783766  6650 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:24.784106  7876 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.316828  7269 rpcz_store.cc:276] 0709 12:17:22.808793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:22.817948 (+  9155us) service_pool.cc:224] Handling call
0709 12:17:22.818322 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:23.682542 (+864220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:23.682619 (+    77us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:23.732447 (+ 49828us) log.cc:844] Serialized 26 byte log entry
0709 12:17:23.869399 (+136952us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:23.869488 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:24.281900 (+412412us) spinlock_profiling.cc:243] Waited 64.9 ms on lock 0x7b5400502654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:24.281961 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:24.282015 (+    54us) raft_consensus.cc:1685] finished
0709 12:17:24.282028 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:24.282276 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26962,"raft.queue_time_us":64987,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"spinlock_wait_cycles":149366144,"thread_start_us":392447,"threads_started":2,"wal-append.queue_time_us":299364}
I20260709 12:17:24.692611  6504 raft_consensus.cc:2393] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 3: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:24.693212  7888 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.600697  7832 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:24.600585  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 177166us
I20260709 12:17:24.696679  7900 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.696866  7916 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.697289  6580 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:17:24.514823  6581 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
W20260709 12:17:24.532362  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 83260.5us
I20260709 12:17:24.704828  7574 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:24.562778  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 152500us
I20260709 12:17:24.672961  6576 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:24.673175  6582 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:17:24.531747  7737 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:24.613580  7897 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:24.757786  7914 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:24.758855  7802 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:24.760778  7912 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.762794  7788 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:24.755393  7769 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:24.720525  7800 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:24.760141  7923 log.cc:927] Time spent T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
W20260709 12:17:24.765884  7925 log.cc:927] Time spent T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.079s	user 0.001s	sys 0.001s
I20260709 12:17:24.720917  6584 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:17:24.513698  7731 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.767884  7781 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:24.772607  6658 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:24.776546  7871 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:24.722167  7903 raft_consensus.cc:697] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:24.778326  7843 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.778388  7775 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 2. Current term is 3.
I20260709 12:17:24.713348  7786 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:24.781472  7919 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.712777  7879 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.781666  6652 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:24.716109  7757 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.793748  6475 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:24.784209  7890 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.799947  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:24.802325  7574 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:24.752707  7785 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.824672  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:24.826402  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:24.837080  6508 raft_consensus.cc:2393] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 2: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:24.830619  6650 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:24.834898  6580 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:24.840700  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:24.841058  6577 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:24.821888  7769 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:24.843195  6475 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:24.834029  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:24.847900  6502 raft_consensus.cc:2393] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 2: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:24.847611  7828 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:24.831354  7788 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:24.848472  7828 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.849783  7788 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:24.847633  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:24.850621  7828 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.850872  6501 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:17:24.851811  7900 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:24.852293  7900 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:24.852468  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:24.852741  7900 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:24.853154  6575 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:17:24.854168  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:24.854816  6575 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:24.826310  7898 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:24.856792  7898 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.858862  7898 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.859622  7929 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:24.860075  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:24.860787  6583 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:17:24.861279  7802 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:24.861687  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:24.861951  7802 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:24.862347  6583 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:17:24.862939  7757 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:24.862555  7802 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:24.824817  7930 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:24.861337  7731 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:24.846902  6475 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:24.863250  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:24.946955  6583 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:17:24.947423  7900 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.948294  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:24.949069  6583 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:17:24.949517  7872 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:24.950313  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:24.950567  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:24.930313  6576 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:24.914245  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:24.836678  7876 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:24.958066  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:24.958158  7919 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:24.961526  6500 raft_consensus.cc:1275] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:24.962517  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:24.963388  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:24.964879  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 118031us
I20260709 12:17:24.976020  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:24.994762  6503 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:17:24.976212  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:24.995123  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:24.995122  6475 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:24.977229  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:24.997045  6583 raft_consensus.cc:2410] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 12:17:24.998229  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:24.998895  6583 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:17:24.999841  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:24.884444  7738 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.996079  6576 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:24.860199  7929 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:24.979328  7715 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:24.885041  7879 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:24.979614  7901 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:24.883574  7903 consensus_queue.cc:237] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:25.001176  7715 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.980816  7930 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:25.006701  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:24.993491  6506 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:17:25.007457  6503 raft_consensus.cc:2368] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3.
I20260709 12:17:25.008002  6501 raft_consensus.cc:2393] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 2: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:25.008100  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.008383  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.008783  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:25.009032  6503 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:17:25.009533  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.010192  6506 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:17:25.010793  6508 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.011438  6475 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:25.007285  7900 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:25.014346  7918 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:25.014763  7900 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:25.014900  7918 raft_consensus.cc:697] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:24.995067  6578 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:24.982451  7898 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:25.016875  7898 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:25.017244  7898 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:24.955449  7911 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:25.014379  6507 raft_consensus.cc:2393] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 2: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:24.840055  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:25.025749  6584 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:17:25.025763  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:25.028258  6579 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:24.844267  7786 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:24.993281  7876 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:24.885694  7769 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:25.036890  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:17:25.051632  7935 log.cc:927] Time spent T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 12:17:25.052160  7907 log.cc:927] Time spent T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.570s	user 0.001s	sys 0.000s
I20260709 12:17:25.052523  7892 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:25.066170  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 246111us
I20260709 12:17:25.000752  6583 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:17:25.063354  6548 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:25.031240  7731 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.070531  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:25.070999  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:25.071841  6650 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.074242  7911 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:25.077428  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:25.077725  7911 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:25.078130  6655 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.078153  7911 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.080776  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:25.083272  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:25.083925  6658 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:25.084646  7872 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:25.084748  7934 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:25.085246  7934 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:25.085623  7934 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.074748  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:25.015959  7918 consensus_queue.cc:237] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:25.003232  7929 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:25.103920  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:25.076284  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:25.076906  7731 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:25.090945  6650 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:25.001418  7901 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:25.095700  6548 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:25.062255  7876 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.105595  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:25.120019  6508 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:25.120602  6654 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.065733  7897 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:25.045593  7737 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:25.119148  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:25.107664  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:25.127786  7731 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:25.127614  7911 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:25.129659  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.130977  6651 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.113199  6474 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:25.121264  7694 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:25.122099  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:25.124061  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:25.133117  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.134439  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:25.134775  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:25.135183  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:25.135913  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:25.136551  6584 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.134896  6507 raft_consensus.cc:2393] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 2: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:25.133788  7694 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:25.141733  7694 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.142779  7769 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:25.066208  7919 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:25.141306  6654 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:25.149883  6576 raft_consensus.cc:1275] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:25.123541  6652 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:25.160633  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.160679  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:25.172940  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:25.173210  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:25.133973  7892 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:25.174373  6580 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:25.178289  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.177929  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:25.179877  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.180096  6580 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.177747  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.179991  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:25.181305  6578 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:25.183411  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:25.186880  7922 log.cc:927] Time spent T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.571s	user 0.000s	sys 0.003s
I20260709 12:17:25.189136  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:25.190107  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 203483us
I20260709 12:17:25.182328  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:25.191275  6582 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.191586  6580 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:25.151271  7900 raft_consensus.cc:697] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 4 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:25.194033  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:25.172626  7769 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:25.201179  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:25.201500  7893 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:25.201987  7893 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:25.202378  6622 leader_election.cc:400] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3.
I20260709 12:17:25.202790  6622 leader_election.cc:403] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:25.202656  7781 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:25.203372  7781 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:25.203814  7781 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.202410  7893 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.207054  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:25.207660  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:25.207741  7272 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:17:25.214923  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.217602  6506 raft_consensus.cc:2393] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 2: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:25.181938  7802 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:25.190763  7911 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:25.222149  6584 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:25.228277  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.241793  6507 raft_consensus.cc:2393] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 2: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:25.230266  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.247093  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:25.247397  6506 raft_consensus.cc:2368] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3.
I20260709 12:17:25.247845  6580 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
W20260709 12:17:25.249230  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 47026.6us
I20260709 12:17:25.248651  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:25.253233  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.258227  7269 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:25.258503  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:25.253674  7900 consensus_queue.cc:237] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:25.264241  7786 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:25.223342  7898 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:25.221477  7769 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:25.273264  6508 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
I20260709 12:17:25.274665  7901 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:25.274755  7769 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:25.277673  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:25.278151  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:25.278312  6580 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.482523  6578 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:25.464123  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.484771  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:25.437304  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:25.311383  6655 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:25.431944  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:25.288918  6651 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:25.335145  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:25.308689  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:17:25.493929  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 239570us
I20260709 12:17:25.455190  6583 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:25.501092  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.485394  7272 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:17:25.313724  6473 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:25.475754  6582 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:25.504917  6658 raft_consensus.cc:2410] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 2 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
W20260709 12:17:25.511710  6575 consensus_meta.cc:287] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.656s	user 0.011s	sys 0.003s
I20260709 12:17:25.513613  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:25.514398  6657 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:17:25.228399  7802 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3.
I20260709 12:17:25.271703  7898 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:25.355562  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:25.519999  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.522675  6506 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:17:25.520831  6580 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
W20260709 12:17:25.528281  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 214654us
I20260709 12:17:25.470942  7919 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:25.455404  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:25.316897  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:25.532682  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.520277  7897 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:25.512195  6575 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:25.517483  7929 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:25.544727  7929 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:25.545145  7929 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:25.531222  7939 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.538745  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:25.566690  7939 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:17:25.539376  6473 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:25.568370  7726 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:25.568481  6473 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:25.568783  7726 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:25.569365  7726 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:25.570088  6473 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:25.572309  7892 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:25.572723  7892 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:25.573083  7892 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.543411  7271 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:17:25.574728  7942 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:25.523325  7788 consensus_queue.cc:1048] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 12:17:25.539547  6506 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:25.534504  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:25.511754  7934 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:25.577839  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:25.578715  7934 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:25.579144  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:25.579334  7934 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:25.579670  7934 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:25.579896  7788 raft_consensus.cc:2764] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:17:25.553040  7781 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:25.580292  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:25.581028  7788 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:25.581379  7788 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:25.555346  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.581568  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:25.581733  7788 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:25.582208  6500 raft_consensus.cc:2393] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 2: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:25.556828  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:25.557300  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.583451  6503 raft_consensus.cc:2368] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3.
I20260709 12:17:25.583840  7872 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:25.584403  7872 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:25.584437  7897 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:25.586571  7872 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:25.586673  7897 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:25.587217  7785 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:25.549603  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.551105  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:25.558931  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:25.537211  7732 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:25.562071  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:25.563766  7909 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:25.591486  6654 raft_consensus.cc:2393] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 3: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:25.565802  7694 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:25.532518  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.568859  7939 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:25.538905  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:25.576957  7882 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:25.557420  7893 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:25.580078  7934 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:25.589006  7757 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:25.565563  7876 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:25.593026  6622 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:25.581748  7781 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:25.604390  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.602751  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:25.600361  6501 raft_consensus.cc:2368] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3.
I20260709 12:17:25.600859  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:25.588407  6509 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:25.589305  6473 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:25.603055  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:25.595667  7268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38558 (request call id 255) took 1185 ms. Trace:
I20260709 12:17:25.606323  6655 raft_consensus.cc:2368] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3.
I20260709 12:17:25.608150  6581 raft_consensus.cc:2393] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 3: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:25.598690  7267 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:17:25.609807  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:25.611737  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:25.612317  6473 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:25.617293  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:25.598765  7882 raft_consensus.cc:697] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 3 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:25.619248  6473 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:25.621104  6622 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:25.627372  7876 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:25.607617  7802 raft_consensus.cc:2804] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:25.628226  7802 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:25.628738  7802 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.607903  7268 rpcz_store.cc:276] 0709 12:17:24.410787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.419899 (+  9112us) service_pool.cc:224] Handling call
0709 12:17:24.440605 (+ 20706us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:24.481819 (+ 41214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:24.483535 (+  1716us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:24.489541 (+  6006us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:24.517541 (+ 28000us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:25.595282 (+1077741us) spinlock_profiling.cc:243] Waited 4.4 ms on lock 0x7b540037ad54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:25.595329 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:25.595607 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10130176}
I20260709 12:17:25.632661  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:25.633489  6658 raft_consensus.cc:2368] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3.
I20260709 12:17:25.636209  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.602124  7757 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:25.637419  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:25.644942  7882 consensus_queue.cc:237] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:25.648757  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.635807  7897 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:25.649082  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.649616  7897 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:25.650089  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.649642  6505 raft_consensus.cc:2393] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 2: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:25.651593  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.638396  6473 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:25.654523  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.652262  6503 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:25.656064  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:25.634443  7951 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:25.657433  7951 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:25.657848  7951 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:25.643131  7781 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:25.660605  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.661001  7781 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:25.661434  7781 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:25.663007  7897 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:25.636003  7919 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:25.663771  7919 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:25.617044  7732 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:25.635866  7944 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:25.664623  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.665409  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:25.665879  6402 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57900 (request call id 185) took 3475 ms. Trace:
I20260709 12:17:25.666042  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:25.665279  7944 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:25.667359  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:25.667587  7944 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.669144  6502 raft_consensus.cc:2393] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 3: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:17:25.667587  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 144634us
I20260709 12:17:25.664608  7835 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:25.670197  7835 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:25.670806  6549 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:25.672140  7738 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:25.673254  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:25.673278  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.674075  6657 raft_consensus.cc:2393] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 3: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:25.674947  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:25.664163  7919 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.666886  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:25.683491  6658 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:17:25.651669  7757 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.654184  6473 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:25.683245  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.627415  7694 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:25.676136  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:25.700103  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:25.700387  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:25.700819  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:25.697892  6473 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:25.655356  7909 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:25.715421  6473 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:25.717041  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:25.717451  6473 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:25.715663  6550 leader_election.cc:400] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3.
I20260709 12:17:25.716776  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:25.718385  6550 leader_election.cc:403] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:25.719717  6579 raft_consensus.cc:2368] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 2. Current term is 3.
I20260709 12:17:25.666388  6402 rpcz_store.cc:276] 0709 12:17:22.072626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:22.073215 (+   589us) service_pool.cc:224] Handling call
0709 12:17:25.548243 (+3475028us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:23.350106 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:23.350926 (+   820us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:23.350977 (+    51us) write_op.cc:435] Acquired schema lock
0709 12:17:23.351001 (+    24us) tablet.cc:662] Decoding operations
0709 12:17:23.354217 (+  3216us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:23.354292 (+    75us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:23.354344 (+    52us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:17:23.354804 (+   460us) tablet.cc:701] Row locks acquired
0709 12:17:23.354827 (+    23us) write_op.cc:260] PREPARE: finished
0709 12:17:23.355054 (+   227us) write_op.cc:270] Start()
0709 12:17:23.355201 (+   147us) write_op.cc:276] Timestamp: P: 1783599443354974 usec, L: 0
0709 12:17:23.355228 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 12:17:23.356908 (+  1680us) log.cc:844] Serialized 1274 byte log entry
0709 12:17:24.281368 (+924460us) op_driver.cc:464] REPLICATION: finished
0709 12:17:24.295938 (+ 14570us) write_op.cc:301] APPLY: starting
0709 12:17:24.320370 (+ 24432us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:24.349338 (+ 28968us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:24.360302 (+ 10964us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:24.450549 (+ 90247us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:24.513735 (+ 63186us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 12:17:24.531944 (+ 18209us) write_op.cc:312] APPLY: finished
0709 12:17:24.604831 (+ 72887us) log.cc:844] Serialized 34 byte log entry
0709 12:17:24.668204 (+ 63373us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:24.706210 (+ 38006us) write_op.cc:454] Released schema lock
0709 12:17:24.711699 (+  5489us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:24.763475 (+ 51776us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:24.763552 (+    77us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":205305,"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":5429,"apply.run_cpu_time_us":4582,"apply.run_wall_time_us":454714,"num_ops":3,"prepare.queue_time_us":241608,"prepare.run_cpu_time_us":4309,"prepare.run_wall_time_us":7329,"replication_time_us":925915}]]}
I20260709 12:17:25.720547  7785 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:25.721108  7785 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:25.721544  7785 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.722374  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:25.723920  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:25.672272  7835 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:25.724593  6576 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 1.
I20260709 12:17:25.727146  6473 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:25.725422  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:25.728731  6503 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.716735  7876 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:25.714457  7732 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:25.730338  7238 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:17:25.722309  6504 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.015s	sys 0.027s
W20260709 12:17:25.752681  6504 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.016s	sys 0.027s
I20260709 12:17:25.732836  6548 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:25.754374  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:25.755281  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:25.756043  6651 raft_consensus.cc:2368] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3.
I20260709 12:17:25.757637  7872 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:25.757875  7238 leader_election.cc:400] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 2. Current term is 3.
I20260709 12:17:25.759162  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:25.760277  7238 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:25.761456  6548 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:25.694836  7929 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:25.770907  6475 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:25.713726  7726 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:25.731812  7793 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:25.734905  7900 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:25.756522  7876 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:25.733925  7909 raft_consensus.cc:697] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:25.779464  6475 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:25.790109  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:25.790899  6651 raft_consensus.cc:2393] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 3: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:25.763816  7872 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:25.773969  7775 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:25.798485  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:25.778937  7732 raft_consensus.cc:697] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:25.790899  7876 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:25.806324  7905 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.181s	user 0.014s	sys 0.025s
W20260709 12:17:25.806808  7905 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.182s	user 0.014s	sys 0.025s
W20260709 12:17:25.807466  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 88505.3us
I20260709 12:17:25.794000  7957 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.792886  7939 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:25.793030  7900 raft_consensus.cc:697] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:25.821538  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:25.812415  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 85354us
I20260709 12:17:25.824096  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:25.813993  7967 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:25.826246  6622 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:25.799942  7775 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:25.799577  7962 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:25.816954  7757 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:25.804262  6653 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:25.803530  7893 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:25.843233  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:25.814366  6475 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:25.827183  7882 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:25.838328  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:25.854915  6500 raft_consensus.cc:2393] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 3: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:25.820822  7737 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:25.845193  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.844924  7900 consensus_queue.cc:237] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:25.845984  6403 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46184 (request call id 111) took 3902 ms. Trace:
I20260709 12:17:25.845788  7909 consensus_queue.cc:237] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:25.846563  6406 catalog_manager.cc:5697] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 1 to 2, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 2 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:25.848474  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.849911  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.801956  7788 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:25.850427  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:25.832649  7775 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:25.831918  7954 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:25.843241  7793 raft_consensus.cc:697] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:25.874869  7793 consensus_queue.cc:237] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:25.838234  7936 consensus_queue.cc:1048] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
W20260709 12:17:25.860033  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 51128.1us
W20260709 12:17:25.861930  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 152044us
I20260709 12:17:25.858668  7882 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:17:25.863438  7959 log.cc:927] Time spent T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 12:17:25.863979  7966 log.cc:927] Time spent T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
W20260709 12:17:25.862316  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 100940us
I20260709 12:17:25.853304  7731 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:25.893208  6548 leader_election.cc:400] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3.
I20260709 12:17:25.878669  6475 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:25.889310  6550 leader_election.cc:400] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3.
I20260709 12:17:25.855784  6652 raft_consensus.cc:2368] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3.
I20260709 12:17:25.896154  6655 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:25.864145  7919 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:25.893587  6548 leader_election.cc:403] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:25.894079  6550 leader_election.cc:403] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:25.893289  7969 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:25.903545  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:25.904456  6500 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.908476  6548 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:25.898233  7939 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:25.915283  6550 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:25.898903  7726 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:25.929071  6548 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:25.853293  7732 consensus_queue.cc:237] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:25.902697  7768 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:25.930976  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:25.930167  6548 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:25.913017  7911 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:25.942996  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.934167  7929 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:25.925122  7939 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:25.902161  7882 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:25.908951  7731 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:17:25.952248  7788 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:25.952363  7897 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:25.939839  7971 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:25.859088  6403 rpcz_store.cc:276] 0709 12:17:21.943065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:21.943189 (+   124us) service_pool.cc:224] Handling call
0709 12:17:22.021913 (+ 78724us) spinlock_profiling.cc:243] Waited 26.4 ms on lock 0x7b5400002210. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d650b2 00007f287bcde759 00007f287bc76a76 00007f287bc47d58 00007f287bd4604e 00007f287a2072a4 00007f287a207232 00007f287a207161 00007f287a2070dc
0709 12:17:25.845782 (+3823869us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:23.156795 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:23.223008 (+ 66213us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:23.223068 (+    60us) write_op.cc:435] Acquired schema lock
0709 12:17:23.223101 (+    33us) tablet.cc:662] Decoding operations
0709 12:17:23.235148 (+ 12047us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:23.241635 (+  6487us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:23.243456 (+  1821us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:17:23.274932 (+ 31476us) tablet.cc:701] Row locks acquired
0709 12:17:23.276222 (+  1290us) write_op.cc:260] PREPARE: finished
0709 12:17:23.284194 (+  7972us) write_op.cc:270] Start()
0709 12:17:23.294798 (+ 10604us) write_op.cc:276] Timestamp: P: 1783599443282215 usec, L: 0
0709 12:17:23.312467 (+ 17669us) op_driver.cc:348] REPLICATION: starting
0709 12:17:23.349446 (+ 36979us) log.cc:844] Serialized 1274 byte log entry
0709 12:17:23.884484 (+535038us) op_driver.cc:464] REPLICATION: finished
0709 12:17:24.251663 (+367179us) write_op.cc:301] APPLY: starting
0709 12:17:24.251798 (+   135us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:24.263586 (+ 11788us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:24.263750 (+   164us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:24.264504 (+   754us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:24.264768 (+   264us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 12:17:24.264858 (+    90us) write_op.cc:312] APPLY: finished
0709 12:17:24.265281 (+   423us) log.cc:844] Serialized 34 byte log entry
0709 12:17:24.265492 (+   211us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:24.265685 (+   193us) write_op.cc:454] Released schema lock
0709 12:17:24.265741 (+    56us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:24.266212 (+   471us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:24.266243 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":565606,"reported_tablets":3,"spinlock_wait_cycles":60826496,"child_traces":[["op",{"apply.queue_time_us":342628,"apply.run_cpu_time_us":3112,"apply.run_wall_time_us":14580,"num_ops":3,"prepare.queue_time_us":670635,"prepare.run_cpu_time_us":6174,"prepare.run_wall_time_us":211933,"replication_time_us":534880,"thread_start_us":679079,"threads_started":2}]]}
I20260709 12:17:25.920324  7970 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3.
I20260709 12:17:25.939541  7694 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3.
I20260709 12:17:25.977881  7982 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:25.978530  7982 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:25.981048  7982 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:25.981539  7982 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.982005  7982 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:25.983969  7982 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:25.986016  6502 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 1372) took 1036 ms. Trace:
I20260709 12:17:25.957085  7939 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.990691  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:25.991300  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:25.991335  6501 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:25.994947  7980 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:25.986666  6502 rpcz_store.cc:276] 0709 12:17:24.949753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.949985 (+   232us) service_pool.cc:224] Handling call
0709 12:17:25.985731 (+1035746us) spinlock_profiling.cc:243] Waited 643 ms on lock 0x7b5400388454. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:25.985934 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1479409408}
I20260709 12:17:25.998133  7980 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:25.998672  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:25.934113  7978 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:25.947937  7892 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:25.971243  7905 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:25.972175  7872 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:25.988585  7757 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:26.020095  6503 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:25.968618  7897 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:25.995687  7911 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:26.002151  7957 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3.
I20260709 12:17:26.024551  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:26.027976  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:26.035555  6504 raft_consensus.cc:2393] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 3: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:26.036870  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:25.996317  7944 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:26.046003  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:26.045620  7980 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:26.053850  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:26.028968  7911 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:26.056246  6652 raft_consensus.cc:2393] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 3: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:26.017704  7872 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:26.062860  6503 raft_consensus.cc:2368] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3.
I20260709 12:17:26.063915  6504 raft_consensus.cc:1275] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:26.068984  7882 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:26.072880  7731 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:26.074488  7876 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:26.075563  6500 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:26.076529  7785 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:26.078553  7802 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:26.083118  6506 raft_consensus.cc:2393] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 3: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:26.081089  7775 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:26.084157  7785 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:26.085404  7897 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:26.086431  7775 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:26.087356  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:26.075989  7781 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:26.088625  6650 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:26.089401  7872 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:26.089829  6501 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:26.060952  7911 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.094695  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:26.091094  6622 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.927s (SENT) [suppressed 2 similar messages]
I20260709 12:17:26.100136  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:26.100971  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:26.103617  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:26.103724  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:26.104519  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:26.107203  6582 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:26.109534  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:26.110553  6580 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.109609  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:26.111837  6583 raft_consensus.cc:2393] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 3: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:26.113174  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:26.119792  6577 raft_consensus.cc:1275] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:26.108677  7876 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:26.134090  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 203976us
W20260709 12:17:26.140547  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 78901.4us
I20260709 12:17:26.141649  6580 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:26.142347  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:26.142710  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:26.143373  6500 raft_consensus.cc:2368] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3.
W20260709 12:17:26.214061  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 78501.3us
I20260709 12:17:26.214649  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:26.214969  6548 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:26.215334  6501 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.216648  7973 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:26.217093  7973 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:26.217546  7973 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.217597  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:26.218515  6500 raft_consensus.cc:2368] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3.
I20260709 12:17:26.249630  7892 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:17:26.256701  7951 consensus_meta.cc:287] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.598s	user 0.010s	sys 0.000s
I20260709 12:17:26.257299  7951 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:26.258615  6658 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.260618  6548 leader_election.cc:400] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3.
I20260709 12:17:26.261776  6548 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:26.259691  7973 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:26.265345  7944 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:26.266784  7802 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:26.267201  7973 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:26.273581  6548 leader_election.cc:400] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3.
I20260709 12:17:26.274559  6548 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:26.274098  7973 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:26.285054  6501 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:26.282034  6548 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:26.285007  6658 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:26.282773  7971 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:26.291694  7737 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:26.293985  7905 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:17:26.299412  7934 consensus_meta.cc:287] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.663s	user 0.008s	sys 0.004s
I20260709 12:17:26.283259  7781 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:26.302857  7951 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:26.309067  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:26.309580  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:26.311599  6575 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:26.307041  7911 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:26.430451  7911 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:26.430902  7918 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:26.431542  7934 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:26.432225  7802 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.432687  6651 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.432919  7802 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:26.434495  7905 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.435622  7802 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:26.311791  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:26.425822  6582 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 3
W20260709 12:17:26.330700  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 43565.2us
I20260709 12:17:26.313823  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:26.426663  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:26.441331  6651 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:26.426810  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:26.441843  6548 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:26.443234  6548 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:26.442979  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:26.445027  6548 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:26.431259  7837 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:26.362634  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:26.444207  7962 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:26.445467  6652 raft_consensus.cc:2393] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 3: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:26.427615  6623 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:17:26.447775  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 85106.7us
I20260709 12:17:26.425477  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:26.446846  6580 raft_consensus.cc:2368] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 2. Current term is 3.
I20260709 12:17:26.448611  6473 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:26.447288  6654 raft_consensus.cc:2368] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3.
I20260709 12:17:26.283746  7973 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:26.450310  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:26.309670  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:26.451972  6623 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:26.463341  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:26.464761  6654 raft_consensus.cc:2410] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 12:17:26.447705  7781 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.467585  6473 leader_election.cc:400] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 2. Current term is 3.
I20260709 12:17:26.469441  6473 leader_election.cc:403] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:17:26.462342  6550 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:26.314671  7986 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:26.454304  7973 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.450567  7911 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:26.510499  6579 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.511235  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:26.511251  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:26.520799  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 895) took 1884 ms. Trace:
I20260709 12:17:26.514667  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:26.484293  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:26.487210  6582 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:26.537804  6652 raft_consensus.cc:2368] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 2. Current term is 3.
I20260709 12:17:26.470165  6473 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:17:26.512198  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 49931.1us
W20260709 12:17:26.538429  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 68350.8us
I20260709 12:17:26.538879  6550 leader_election.cc:400] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3.
I20260709 12:17:26.539160  6550 leader_election.cc:403] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:26.462731  7951 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.450940  7981 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:26.453306  7732 consensus_queue.cc:1048] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.058s
I20260709 12:17:26.467336  7788 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:26.449462  7837 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:26.539273  6473 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:26.458627  7802 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:26.536514  6654 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:26.463038  7892 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:26.469080  7962 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:26.459167  7876 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:26.456346  7939 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:26.511828  7785 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 2. Current term is 3.
I20260709 12:17:26.513374  6649 raft_consensus.cc:2368] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 3. Current term is 4.
I20260709 12:17:26.520169  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:26.583554  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:26.521553  7694 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:17:26.601343  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 62129.1us
I20260709 12:17:26.585604  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:26.603686  6577 raft_consensus.cc:2393] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 3: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:26.536124  6656 rpcz_store.cc:276] 0709 12:17:24.636655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:24.638586 (+  1931us) service_pool.cc:224] Handling call
0709 12:17:24.638994 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:24.871977 (+232983us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:17:24.882719 (+ 10742us) log.cc:844] Serialized 14 byte log entry
0709 12:17:25.022889 (+140170us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:25.022963 (+    74us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:25.023024 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.520600 (+1497576us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b54000749d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:26.520639 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:26.520744 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":16368,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":2394,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":112,"spinlock_wait_cycles":452799232,"thread_start_us":139661,"threads_started":1,"wal-append.queue_time_us":121331,"wal-append.run_cpu_time_us":1284,"wal-append.run_wall_time_us":1004779}
I20260709 12:17:26.592118  7918 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:26.595572  6584 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
I20260709 12:17:26.597671  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:26.608239  6652 raft_consensus.cc:2393] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 3: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:26.512691  7781 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:26.543213  7944 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:26.581650  7980 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:26.546182  7936 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:26.541127  7954 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:26.549188  7882 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:26.601039  7694 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:26.595983  7802 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:26.589811  7892 raft_consensus.cc:697] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:26.617650  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:26.619041  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:26.546481  7951 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:26.597209  7911 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3.
I20260709 12:17:26.626601  6582 raft_consensus.cc:2393] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 3: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:26.607321  6657 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.633821  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:26.634938  6652 raft_consensus.cc:2393] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 3: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:26.612149  7715 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:26.593703  7876 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:26.639176  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:26.614886  7918 raft_consensus.cc:697] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:26.598618  7962 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:26.623615  7694 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:26.651345  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 112589us
I20260709 12:17:26.640847  6502 raft_consensus.cc:2393] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 3: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:26.652145  6550 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:26.629402  7802 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.654034  6550 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:26.655352  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:26.655941  6577 raft_consensus.cc:2393] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 3: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:26.626813  7882 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:26.645030  7934 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:26.658488  7911 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:26.658876  6550 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:26.663904  7939 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:26.624017  7980 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:26.659149  7892 consensus_queue.cc:237] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:26.667801  6579 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:26.668540  6652 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:26.670207  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:26.620725  7936 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:26.671095  6577 raft_consensus.cc:2393] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 3: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:26.659168  7781 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:26.669003  7986 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:26.672544  7986 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.677793  7980 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:26.678700  7914 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:26.655107  7971 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:26.680078  7918 consensus_queue.cc:237] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:26.660317  7882 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:26.684950  7951 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:26.669552  7879 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:26.660248  7876 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:26.623678  7954 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:26.661566  7962 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:26.692168  7934 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:26.693078  7954 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:26.693286  6656 raft_consensus.cc:1275] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:26.694675  7936 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:26.695618  6653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 1454) took 1293 ms. Trace:
I20260709 12:17:26.692799  7934 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:26.697587  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:26.707286  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:26.710469  6622 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:26.712733  6550 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:26.713655  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:26.715716  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:26.719578  7944 raft_consensus.cc:697] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:26.722571  6577 raft_consensus.cc:2368] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3.
I20260709 12:17:26.723627  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:26.724359  6503 raft_consensus.cc:2368] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3.
I20260709 12:17:26.724903  7930 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:26.727355  6652 raft_consensus.cc:1275] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:26.727715  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:26.729763  7934 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:26.730389  7962 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:26.733353  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:26.733741  7986 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:26.734570  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:26.735271  6582 raft_consensus.cc:2368] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3.
I20260709 12:17:26.729916  7971 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:26.735327  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:26.736368  6577 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:26.738855  6651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58980 (request call id 1486) took 1078 ms. Trace:
I20260709 12:17:26.739640  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:26.740623  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:26.739256  6651 rpcz_store.cc:276] 0709 12:17:25.677175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.831277 (+154102us) service_pool.cc:224] Handling call
0709 12:17:26.738717 (+907440us) spinlock_profiling.cc:243] Waited 588 ms on lock 0x7b54000758d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:26.738802 (+    85us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1353465088}
I20260709 12:17:26.741199  6580 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:17:26.741732  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:26.741997  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:26.742157  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:26.720161  7951 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:26.742614  6580 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:26.742730  6575 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.742995  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:26.695990  6653 rpcz_store.cc:276] 0709 12:17:25.402209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.421541 (+ 19332us) service_pool.cc:224] Handling call
0709 12:17:25.513001 (+ 91460us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:25.841462 (+328461us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:26.695375 (+853913us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b54003e8ad4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:26.695553 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":642977024}
I20260709 12:17:26.743554  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:26.744681  7994 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:26.745357  7994 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:26.747686  7994 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:26.747804  7970 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:26.729686  7905 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:26.748449  7970 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:26.749014  6658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58980 (request call id 1483) took 1089 ms. Trace:
I20260709 12:17:26.749395  6658 rpcz_store.cc:276] 0709 12:17:25.659479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.782303 (+122824us) service_pool.cc:224] Handling call
0709 12:17:26.748855 (+966552us) spinlock_profiling.cc:243] Waited 459 ms on lock 0x7b54002a3ad4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:26.748952 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1056521472}
I20260709 12:17:26.750535  7970 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:26.749281  7775 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:26.672560  7837 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:26.751132  7273 raft_consensus.cc:1240] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:26.752938  6550 leader_election.cc:400] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3.
I20260709 12:17:26.753288  6550 leader_election.cc:403] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:26.755029  6550 leader_election.cc:400] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 3. Current term is 4.
I20260709 12:17:26.755321  6550 leader_election.cc:403] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:26.749441  7240 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:26.756494  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:26.757896  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:26.758767  6651 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 3.
I20260709 12:17:26.761895  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:26.763851  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:26.764783  6580 raft_consensus.cc:2368] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3.
I20260709 12:17:26.769711  6577 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:26.751600  7775 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:26.775805  6550 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:26.776314  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:26.785676  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:26.786307  6550 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:26.736060  6657 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:26.786384  6507 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:17:26.777942  6548 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:26.795181  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:26.796689  6622 leader_election.cc:400] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3.
I20260709 12:17:26.796991  6622 leader_election.cc:403] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:17:26.797976  6622 leader_election.cc:400] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3.
I20260709 12:17:26.798213  6622 leader_election.cc:403] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:17:26.765352  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:26.799988  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:26.781275  7994 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 3. Current term is 4.
I20260709 12:17:26.804836  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:26.801352  7837 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3.
I20260709 12:17:26.807930  6657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58980 (request call id 1479) took 1145 ms. Trace:
I20260709 12:17:26.808961  7775 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:26.787942  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:26.760939  7971 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:26.810132  6653 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 1.
I20260709 12:17:26.810599  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:26.811124  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:26.811201  7973 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:26.804919  7835 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3.
I20260709 12:17:26.811864  7914 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:26.784006  7879 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 12:17:26.798944  7978 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:26.809355  7694 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:26.813954  7944 consensus_queue.cc:237] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:26.815739  7694 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:26.811367  6651 raft_consensus.cc:2393] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 3: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:26.811431  7971 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:26.817061  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:26.799751  7968 consensus_queue.cc:1059] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 12:17:26.817624  6651 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:26.817754  6653 raft_consensus.cc:2393] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 3: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:26.808465  6657 rpcz_store.cc:276] 0709 12:17:25.658232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.733385 (+ 75153us) service_pool.cc:224] Handling call
0709 12:17:26.520023 (+786638us) spinlock_profiling.cc:243] Waited 8.42 ms on lock 0x7b5800015998. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:17:26.799336 (+279313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19370368}
I20260709 12:17:26.802474  7951 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:26.818724  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:26.819700  7905 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:26.819857  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:26.816118  6575 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:26.822168  7970 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3.
I20260709 12:17:26.820732  6657 raft_consensus.cc:2368] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 2. Current term is 3.
I20260709 12:17:26.826400  7802 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:26.836400  7995 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:26.836843  6622 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:26.837024  7995 raft_consensus.cc:697] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:26.838102  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:26.838025  7995 consensus_queue.cc:237] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:26.839030  7936 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:26.839051  7837 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:26.839705  7936 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:26.839787  7837 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:26.840234  7936 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:26.840390  7837 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:26.840612  7957 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:17:26.836289  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 58439.9us
I20260709 12:17:26.857187  6622 leader_election.cc:400] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3.
I20260709 12:17:26.857692  6622 leader_election.cc:403] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:26.860602  6651 raft_consensus.cc:2410] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
W20260709 12:17:26.863834  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 114452us
I20260709 12:17:27.062129  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:27.072574  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:26.870326  6583 raft_consensus.cc:2393] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 3: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:27.058269  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:27.075234  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:27.106568  7240 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:27.069528  6650 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58980 (request call id 1473) took 1442 ms (client timeout 1756 ms). Trace:
I20260709 12:17:27.107496  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:27.068536  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 298555us
I20260709 12:17:26.886660  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:27.088558  6475 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:27.073751  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:27.074415  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:26.861902  7954 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:26.861735  7951 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:27.108990  6651 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 1.
I20260709 12:17:27.109598  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:17:27.104565  6506 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 1390) took 1231 ms (client timeout 1512 ms). Trace:
I20260709 12:17:27.189121  6507 raft_consensus.cc:2393] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 3: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:27.111107  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:26.862475  7835 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3.
W20260709 12:17:27.058569  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 305032us
I20260709 12:17:27.120102  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54204 (request call id 59) took 1405 ms. Trace:
I20260709 12:17:27.144112  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:27.196079  7270 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:17:27.159164  7240 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:27.110668  7273 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:17:27.196470  6622 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:27.118789  6575 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:27.109066  6581 raft_consensus.cc:2393] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 3: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:17:27.159958  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 71501.1us
I20260709 12:17:27.204387  6475 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:27.206780  6475 leader_election.cc:400] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 2. Current term is 3.
I20260709 12:17:27.207314  7270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38562 (request call id 318) took 1671 ms. Trace:
I20260709 12:17:26.861562  7978 raft_consensus.cc:697] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:26.875732  7973 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:27.108569  6650 rpcz_store.cc:269] 0709 12:17:25.627352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.655804 (+ 28452us) service_pool.cc:224] Handling call
0709 12:17:27.069311 (+1413507us) spinlock_profiling.cc:243] Waited 432 ms on lock 0x7b540044a7d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:27.069466 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":993275136}
I20260709 12:17:27.190907  7900 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:27.163914  7905 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:27.167133  8002 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:27.247329  7876 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:27.254633  8002 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:27.167888  7802 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:27.168292  7969 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:27.168793  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:27.257501  7919 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:27.170329  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:27.257699  7969 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:27.258543  7271 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:27.258771  7837 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:27.171695  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:27.260848  6654 raft_consensus.cc:2368] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3.
I20260709 12:17:27.261358  7837 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:27.263001  6654 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:27.169477  7238 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:17:27.264685  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 97283.5us
I20260709 12:17:27.256877  8002 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:27.103920  7957 raft_consensus.cc:697] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:27.176946  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:27.268173  7957 consensus_queue.cc:237] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:17:27.270385  7271 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:27.271020  7914 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:27.271279  7969 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:26.874452  7980 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:27.200742  7954 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:27.187537  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:27.272990  7980 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:27.189260  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:26.874541  7968 raft_consensus.cc:3055] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:27.274039  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:27.274410  7968 raft_consensus.cc:740] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:17:27.188927  6506 rpcz_store.cc:269] 0709 12:17:25.839888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.840547 (+   659us) service_pool.cc:224] Handling call
0709 12:17:27.071047 (+1230500us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b540039f654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:27.071177 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":496611584}
I20260709 12:17:27.198264  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:27.275216  7980 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:17:27.198915  6655 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58980 (request call id 1467) took 1587 ms (client timeout 1836 ms). Trace:
I20260709 12:17:27.275372  7968 consensus_queue.cc:260] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:27.199661  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:27.195253  6578 rpcz_store.cc:276] 0709 12:17:25.701470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.701595 (+   125us) service_pool.cc:224] Handling call
0709 12:17:27.075113 (+1373518us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b5800116198. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64e9c 00007f2872d64a5b 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:17:27.107000 (+ 31887us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26711680}
I20260709 12:17:27.201841  7930 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:27.207747  7270 rpcz_store.cc:276] 0709 12:17:25.524726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.524831 (+   105us) service_pool.cc:224] Handling call
0709 12:17:27.070094 (+1545263us) spinlock_profiling.cc:243] Waited 1 ms on lock 0x7b5800125198. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:17:27.196506 (+126412us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2310912}
I20260709 12:17:27.224836  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:27.267498  7936 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:27.207803  6475 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:17:27.225603  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 150675us
W20260709 12:17:27.225827  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 102122us
I20260709 12:17:27.238649  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:26.863025  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:27.248167  8006 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:27.252305  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:27.252483  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:27.253099  6575 raft_consensus.cc:1275] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:27.252668  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:27.287160  7273 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:17:27.225826  7977 consensus_queue.cc:1059] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
W20260709 12:17:27.291375  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 83631.9us
I20260709 12:17:27.256817  6509 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:27.202986  7882 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:27.171725  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:27.185971  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:27.297361  6580 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:17:27.299183  6580 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:27.273330  6649 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:27.187400  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:17:27.276036  6655 rpcz_store.cc:269] 0709 12:17:25.611151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.611496 (+   345us) service_pool.cc:224] Handling call
0709 12:17:27.198776 (+1587280us) spinlock_profiling.cc:243] Waited 284 ms on lock 0x7b54000753d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:27.198864 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":652483840}
I20260709 12:17:27.278632  6656 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:27.227294  7835 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:27.188189  7951 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:27.284917  6578 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:27.284397  7978 consensus_queue.cc:237] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:27.308661  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:27.295096  8007 raft_consensus.cc:2764] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:17:27.302623  7968 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:27.304833  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 100616us
I20260709 12:17:27.225994  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:27.294509  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 129970us
I20260709 12:17:27.312656  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:27.295408  6475 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:27.285755  7930 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:27.307248  7951 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.302532  7882 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:27.304467  7919 consensus_queue.cc:1048] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 12:17:27.312094  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:27.301880  7969 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 12:17:27.338989  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:27.339083  6473 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:17:27.643570  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 304548us
I20260709 12:17:27.499553  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:27.644486  6623 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:17:27.644871  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 349501us
I20260709 12:17:27.478255  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:27.568475  7939 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:27.646417  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:27.646646  7939 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:27.648517  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 269966us
I20260709 12:17:27.481946  8002 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:27.304716  7269 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:27.651332  7271 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:27.484644  7954 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:27.357878  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:27.351312  7914 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:27.486728  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 972) took 1110 ms. Trace:
I20260709 12:17:27.655305  7914 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:27.655711  6583 rpcz_store.cc:276] 0709 12:17:26.376393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.376699 (+   306us) service_pool.cc:224] Handling call
0709 12:17:27.486544 (+1109845us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b5400420754. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:27.486673 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":984354688}
I20260709 12:17:27.658377  7914 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:27.659276  7265 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 3. (term mismatch)
I20260709 12:17:27.659722  7265 pending_rounds.cc:85] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9: Aborting all ops after (but not including) 2
I20260709 12:17:27.660020  7265 pending_rounds.cc:107] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 1.3
I20260709 12:17:27.660522  7265 raft_consensus.cc:2917] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted by new leader
I20260709 12:17:27.661074  7265 raft_consensus.cc:2889] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 3 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:27.588915  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:27.650058  8002 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:27.680045  8002 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:27.697881  7939 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:27.714718  7951 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:27.717124  7951 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:27.718901  6656 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:27.720778  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:27.730001  6649 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:27.496017  7240 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:17:27.738076  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 242132us
I20260709 12:17:27.738343  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:27.739279  7240 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:27.737154  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 398263us
W20260709 12:17:27.737031  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 92624.1us
I20260709 12:17:27.740273  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:27.741145  7930 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:27.741494  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 1488) took 1194 ms. Trace:
I20260709 12:17:27.741060  6623 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:27.741925  6652 rpcz_store.cc:276] 0709 12:17:26.547852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.728188 (+180336us) service_pool.cc:224] Handling call
0709 12:17:26.735036 (+  6848us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:26.759998 (+ 24962us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:26.760047 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:26.760109 (+    62us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:26.760156 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.741267 (+981111us) spinlock_profiling.cc:243] Waited 24.7 ms on lock 0x7b54001b1554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:27.741311 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:27.741440 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56845568}
I20260709 12:17:27.742610  7882 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:27.743716  7835 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:27.744634  7835 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:27.745072  7835 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:27.741667  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:27.746637  7882 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:27.747154  7882 raft_consensus.cc:697] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 3 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:27.748056  7882 consensus_queue.cc:237] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:27.748770  6623 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:27.750844  8011 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:27.360731  8007 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:27.752449  7837 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:17:27.752965  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 100035us
I20260709 12:17:27.753084  7837 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:27.752820  8007 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:27.753574  7837 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:27.754681  7835 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:27.755087  8007 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:27.359081  7785 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:17:27.745270  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 99670.1us
I20260709 12:17:27.757674  6475 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:27.758515  7793 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:27.759433  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:27.759644  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:27.759797  7793 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:27.738230  7900 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:27.760773  7900 raft_consensus.cc:697] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:27.481122  7936 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:27.761875  7900 consensus_queue.cc:237] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:27.762487  6506 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:27.760814  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:27.764197  6503 raft_consensus.cc:2368] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 2. Current term is 3.
I20260709 12:17:27.752389  6622 leader_election.cc:400] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3.
I20260709 12:17:27.752502  7954 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:27.742609  7930 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:27.767067  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:27.768748  6508 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:27.769305  7240 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:17:27.768832  6505 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 1386) took 1934 ms (client timeout 2138 ms). Trace:
I20260709 12:17:27.766268  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:27.773228  6506 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 1.
W20260709 12:17:27.770016  6505 rpcz_store.cc:269] 0709 12:17:25.834699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:25.834817 (+   118us) service_pool.cc:224] Handling call
0709 12:17:27.768653 (+1933836us) spinlock_profiling.cc:243] Waited 926 ms on lock 0x7b54001e91d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:27.768775 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2129868160}
I20260709 12:17:27.776185  6622 leader_election.cc:400] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3.
I20260709 12:17:27.776651  6622 leader_election.cc:403] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:17:27.782321  6622 leader_election.cc:400] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3.
I20260709 12:17:27.782936  6622 leader_election.cc:403] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:17:27.780844  6406 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57914 (request call id 187) took 4715 ms. Trace:
I20260709 12:17:27.764096  7837 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:27.780299  7914 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:27.786280  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:27.786646  7914 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:27.787009  7914 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:27.787359  7914 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:27.787451  7837 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:27.760741  7793 raft_consensus.cc:697] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:27.789959  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:27.790373  7793 consensus_queue.cc:237] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:27.792089  6652 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:17:27.788746  7835 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:27.789253  6649 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:27.801069  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1296) took 1283 ms. Trace:
I20260709 12:17:27.796213  6652 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:27.812314  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:27.813895  7905 consensus_queue.cc:1059] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:27.814965  7905 raft_consensus.cc:3055] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:27.815135  7238 leader_election.cc:400] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 2. Current term is 3.
I20260709 12:17:27.815461  7238 leader_election.cc:403] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:17:27.816025  8006 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:27.816577  7238 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:27.818765  8007 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:27.819478  8007 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:27.820508  7715 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:27.821112  7715 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:27.822202  8007 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:27.823592  7715 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:27.824999  8007 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:27.815361  7905 raft_consensus.cc:740] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:27.826486  8007 consensus_queue.cc:1059] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:27.827486  7969 raft_consensus.cc:3055] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:27.801491  6584 rpcz_store.cc:276] 0709 12:17:26.517147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.791799 (+1274652us) service_pool.cc:224] Handling call
0709 12:17:27.792077 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.792233 (+   156us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:27.792258 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:27.792297 (+    39us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:27.792324 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:27.800873 (+  8549us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:27.801011 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:27.828698  6656 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:27.826648  7905 consensus_queue.cc:260] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:27.831571  6622 leader_election.cc:400] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3.
I20260709 12:17:27.829499  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:27.835587  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 86887.9us
I20260709 12:17:27.780040  8002 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:27.855803  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:27.800088  7802 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3.
I20260709 12:17:27.841575  6622 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:27.850482  6508 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 3.
I20260709 12:17:27.852429  6654 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:27.827827  7969 raft_consensus.cc:740] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:27.869701  6622 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:27.828183  7936 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3.
W20260709 12:17:27.872541  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 103308us
I20260709 12:17:27.873909  7240 leader_election.cc:400] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 2. Current term is 3.
I20260709 12:17:27.874954  7240 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:27.854580  8002 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:27.881609  7793 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 12:17:27.900007  7969 consensus_queue.cc:260] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:27.901391  7969 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.882315  7905 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:27.907128  8006 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 2. Current term is 3.
I20260709 12:17:27.907873  8006 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:27.884569  7802 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:27.910069  8027 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:27.914081  6656 raft_consensus.cc:1240] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:27.919555  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 46054.3us
I20260709 12:17:27.882758  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:27.932830  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:27.886592  8002 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:27.887442  7951 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:27.920478  7802 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:27.920553  6657 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 12:17:27.953671  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 95575.5us
I20260709 12:17:27.930100  8027 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:27.961057  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:27.888662  7892 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.066s
I20260709 12:17:27.964797  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:27.908265  8006 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:27.784273  6406 rpcz_store.cc:276] 0709 12:17:23.057721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:23.076432 (+ 18711us) service_pool.cc:224] Handling call
0709 12:17:27.768515 (+4692083us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:26.097458 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:26.098140 (+   682us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:26.098189 (+    49us) write_op.cc:435] Acquired schema lock
0709 12:17:26.098207 (+    18us) tablet.cc:662] Decoding operations
0709 12:17:26.098633 (+   426us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:26.098729 (+    96us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:26.098760 (+    31us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:17:26.099054 (+   294us) tablet.cc:701] Row locks acquired
0709 12:17:26.099105 (+    51us) write_op.cc:260] PREPARE: finished
0709 12:17:26.099267 (+   162us) write_op.cc:270] Start()
0709 12:17:26.099496 (+   229us) write_op.cc:276] Timestamp: P: 1783599446099200 usec, L: 0
0709 12:17:26.099520 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:17:26.115084 (+ 15564us) log.cc:844] Serialized 531 byte log entry
0709 12:17:26.295944 (+180860us) spinlock_profiling.cc:243] Waited 1.34 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2872d7f0ac 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b
0709 12:17:26.695771 (+399827us) op_driver.cc:464] REPLICATION: finished
0709 12:17:27.152818 (+457047us) write_op.cc:301] APPLY: starting
0709 12:17:27.164017 (+ 11199us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:27.168356 (+  4339us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:27.168382 (+    26us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:27.168670 (+   288us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:27.168823 (+   153us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:17:27.168896 (+    73us) write_op.cc:312] APPLY: finished
0709 12:17:27.169119 (+   223us) log.cc:844] Serialized 22 byte log entry
0709 12:17:27.169289 (+   170us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:27.169367 (+    78us) write_op.cc:454] Released schema lock
0709 12:17:27.169404 (+    37us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:27.169765 (+   361us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:27.169797 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":75531,"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":456746,"apply.run_cpu_time_us":2023,"apply.run_wall_time_us":16946,"num_ops":1,"prepare.queue_time_us":110522,"prepare.run_cpu_time_us":27835,"prepare.run_wall_time_us":312422,"raft.queue_time_us":179545,"raft.run_cpu_time_us":32295,"raft.run_wall_time_us":369320,"replication_time_us":596116,"spinlock_wait_cycles":3168000,"thread_start_us":875258,"threads_started":4,"wal-append.queue_time_us":332066}]]}
I20260709 12:17:27.978917  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:27.979200  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:27.962285  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:27.973933  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:27.885082  8016 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.066s
W20260709 12:17:27.978466  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 141058us
I20260709 12:17:27.991816  6623 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:27.986934  6505 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:27.994079  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:27.994913  6500 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:27.979761  8027 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:27.972397  7802 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:28.003896  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:27.996094  8006 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:27.974045  7732 consensus_queue.cc:1059] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
W20260709 12:17:28.045018  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 53284.4us
I20260709 12:17:28.046223  6623 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:28.050251  7882 raft_consensus.cc:2804] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:28.051010  7882 raft_consensus.cc:697] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 3 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:28.006699  8022 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:28.055119  6623 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:27.995853  7837 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 12:17:28.008371  8028 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:28.069864  6623 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:17:28.077184  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 75495.7us
I20260709 12:17:28.041724  8032 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.053690  7732 consensus_queue.cc:1243] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer f2b06aebe97249a4877d798b8118ece4 log is divergent from this leader: its last log entry 1.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:17:28.089262  6623 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:28.067967  8041 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:17:28.098467  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 223587us
I20260709 12:17:28.098177  6623 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:28.086515  8011 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:28.066285  8029 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:28.085305  8028 raft_consensus.cc:697] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:28.068893  8042 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:28.097796  8032 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:28.106815  8038 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:28.111491  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:28.052286  7882 consensus_queue.cc:237] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:28.098852  7951 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:28.096801  8041 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:28.079859  8022 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:28.117604  8038 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:28.120719  8041 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:28.120988  8038 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.119041  8002 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:28.106873  8031 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:28.123984  8028 consensus_queue.cc:237] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:28.127936  6507 raft_consensus.cc:2410] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:17:28.129190  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.114773  8042 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:28.141490  7986 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:28.145815  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 46480.7us
I20260709 12:17:28.147092  7240 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:28.149690  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:28.150518  6655 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 4: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:28.153944  7240 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:28.155542  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:28.156725  6653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 931) took 1382 ms. Trace:
I20260709 12:17:28.157272  6653 rpcz_store.cc:276] 0709 12:17:26.774615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.810812 (+ 36197us) service_pool.cc:224] Handling call
0709 12:17:28.156449 (+1345637us) spinlock_profiling.cc:243] Waited 5.76 ms on lock 0x7b54000758d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:28.156642 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13238144}
I20260709 12:17:28.158784  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:28.159389  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:28.163446  7785 raft_consensus.cc:3055] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:28.163590  8032 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:28.163895  7785 raft_consensus.cc:740] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:28.164842  7785 consensus_queue.cc:260] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:28.166163  7785 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.166584  7802 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:28.167547  6658 raft_consensus.cc:1240] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:28.169443  6650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 1490) took 1620 ms. Trace:
I20260709 12:17:28.169796  6650 rpcz_store.cc:276] 0709 12:17:26.549092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.250720 (+701628us) service_pool.cc:224] Handling call
0709 12:17:27.251044 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:27.358551 (+107507us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:27.358596 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:27.358654 (+    58us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:27.358696 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:28.169266 (+810570us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b54000b5c54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:28.169300 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:28.169395 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":246642048}
I20260709 12:17:28.171202  8043 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 3 LEARNER]: Advancing to term 4
I20260709 12:17:28.154623  7985 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:28.178766  8028 consensus_queue.cc:1059] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:17:28.184844  8028 raft_consensus.cc:3055] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:28.187296  7914 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:28.191252  7914 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:28.197402  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:28.197824  8006 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:28.198448  8027 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:28.180006  7802 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:28.200661  8042 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:28.202106  8006 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:28.185257  8028 raft_consensus.cc:740] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:28.206393  8002 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:28.212761  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:28.214669  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:28.214913  8038 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:28.216450  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:28.223901  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:28.224951  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:28.216099  7738 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 12:17:28.237527  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:28.235327  6403 catalog_manager.cc:5697] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 1 to 2, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 2 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
W20260709 12:17:28.241412  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 141802us
I20260709 12:17:28.243098  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:28.244004  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:28.244313  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:28.244417  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:28.244830  7270 raft_consensus.cc:2393] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 2: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:17:28.244946  7267 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:28.245105  7265 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
W20260709 12:17:28.249358  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 74849.4us
I20260709 12:17:28.217516  8038 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:28.206569  8028 consensus_queue.cc:260] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:28.253307  7273 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:28.253450  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:28.253655  8028 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:28.254129  6584 raft_consensus.cc:2393] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 3: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:28.254979  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:28.252846  6580 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:28.255868  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:28.256961  8043 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 4 LEARNER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:28.257285  6578 raft_consensus.cc:1240] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:28.258167  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:28.240958  6403 catalog_manager.cc:5697] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:28.208017  8041 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:28.265748  6403 catalog_manager.cc:5697] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:28.268904  6623 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:28.269531  6576 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:28.269575  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:28.271530  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:28.272078  6650 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:28.267184  7971 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:28.274565  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 1.42376e+06us
I20260709 12:17:28.273675  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:28.281172  7971 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:28.281868  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:28.283253  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:28.283625  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:28.283887  7273 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:17:28.284389  7265 raft_consensus.cc:2393] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 4: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:17:28.274150  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:28.276713  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:28.286237  6658 raft_consensus.cc:2393] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 3: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:28.288952  7930 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:28.297331  6623 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:28.309842  6650 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:28.294982  6578 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:28.291589  8027 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:28.295078  6549 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:17:28.314208  6549 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:28.291858  8048 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:28.323720  6579 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:28.318027  8048 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:28.295534  7882 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:28.331008  7918 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:28.293905  8041 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:28.333050  6473 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:28.316892  7802 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:28.334805  8048 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:28.331702  8007 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:17:28.318228  7977 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:28.335364  8012 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:28.339016  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:28.342645  7999 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:28.342916  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:28.297188  8038 consensus_queue.cc:1059] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:17:28.344226  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:28.362430  8038 raft_consensus.cc:3055] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:28.362818  8038 raft_consensus.cc:740] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:28.363692  8038 consensus_queue.cc:260] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:28.365000  8038 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:28.337507  7802 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:28.365890  7271 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:28.366357  7802 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:28.345945  7273 raft_consensus.cc:1240] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:28.368422  6622 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:28.352093  7892 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.369190  7892 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:28.370173  7876 consensus_queue.cc:1059] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:28.348726  8022 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:28.293428  7930 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:28.334442  7882 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:28.359203  8027 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:28.358179  7909 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:28.357304  7977 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:28.360720  8012 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:28.360812  7238 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:28.340215  7785 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 12:17:28.356163  7973 raft_consensus.cc:3055] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:28.371870  8018 raft_consensus.cc:3055] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:28.388581  6656 raft_consensus.cc:1275] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:17:28.392293  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 44123.4us
I20260709 12:17:28.394629  6474 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:28.395363  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:28.396080  6650 raft_consensus.cc:2393] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 3: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:28.396664  6655 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:28.386976  7882 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.387697  7995 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.399030  6652 raft_consensus.cc:1240] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:28.391215  7977 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:28.392736  8012 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:28.415526  6474 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:28.393800  8028 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:28.393741  7909 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:28.393523  7973 raft_consensus.cc:740] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:28.420874  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:28.424821  7238 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:28.423199  6656 raft_consensus.cc:1275] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:28.404556  8018 raft_consensus.cc:740] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:28.429836  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:28.407780  7892 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:28.395957  7786 consensus_queue.cc:1059] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:17:28.414295  8029 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:28.431706  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:28.435681  6652 raft_consensus.cc:1275] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:28.437740  6650 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:28.439244  7238 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:28.439970  6474 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:28.415489  7995 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:28.437189  8036 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:28.452410  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:28.457465  6652 raft_consensus.cc:1240] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:28.460588  7898 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:28.460824  8024 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:28.461437  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:28.460954  6653 raft_consensus.cc:2393] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 3: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:28.463320  6473 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:28.467464  8051 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:28.468060  8051 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:28.468570  7802 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:28.469411  8013 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.471930  6550 leader_election.cc:400] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3.
I20260709 12:17:28.472311  7793 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:28.472640  8052 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.473096  7944 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:28.473289  8052 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:28.472314  6550 leader_election.cc:403] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:28.469959  8013 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:28.475979  7879 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3.
I20260709 12:17:28.478386  6550 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:28.480319  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:28.466427  7973 consensus_queue.cc:260] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:28.485508  7240 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:28.520206  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 216761us
I20260709 12:17:28.524236  6475 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:28.470933  7981 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.522428  6548 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:28.529196  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:28.530108  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 44681.1us
I20260709 12:17:28.531845  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:28.567054  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 85825.3us
W20260709 12:17:28.567444  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 45167.3us
I20260709 12:17:28.567938  6473 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:28.481793  7930 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:28.479501  7999 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:28.568248  6548 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:28.576704  6548 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:28.569870  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 45701.2us
I20260709 12:17:28.484424  7879 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:28.580377  6652 raft_consensus.cc:2393] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 3: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:28.480022  7802 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:28.582712  7738 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:28.485324  7945 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:28.574978  7898 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:28.593272  7971 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:28.581452  6548 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:28.570509  7911 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:17:28.599495  8055 log.cc:927] Time spent T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:17:28.531956  8024 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:28.601923  6473 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:28.594058  7968 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:28.593328  7879 raft_consensus.cc:697] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 2 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:28.532028  8007 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.081s
I20260709 12:17:28.575224  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:28.613173  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:28.487057  7944 consensus_queue.cc:1059] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.112s
I20260709 12:17:28.594544  7973 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:28.575431  7900 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 12:17:28.646040  6652 raft_consensus.cc:2376] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3. Re-sending same reply.
I20260709 12:17:28.610896  7919 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:28.485033  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:28.528617  7793 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.117s
I20260709 12:17:28.595311  7876 consensus_queue.cc:1059] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 12:17:28.619489  7738 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:28.690804  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 250895us
I20260709 12:17:28.601178  8016 consensus_queue.cc:1048] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.063s
I20260709 12:17:28.693321  6474 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:28.643244  7962 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:28.657605  7898 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:28.661166  7994 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.628669  7892 consensus_queue.cc:1048] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 12:17:28.676180  7968 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.568001  7981 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:28.656672  7837 consensus_queue.cc:1059] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 12:17:28.643138  7945 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.058s
W20260709 12:17:28.738603  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 45331.3us
I20260709 12:17:28.739976  6474 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:28.697090  6653 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:28.707710  7909 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:28.664564  7980 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 12:17:28.725879  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:28.700379  7939 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 12:17:28.774104  6474 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:28.778548  8063 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:28.779944  6474 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:28.744215  7999 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:28.766105  7930 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.744997  8013 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:28.712108  7962 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:28.726428  7994 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:28.798606  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:28.800855  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:28.758011  7981 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:28.770169  8007 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:28.736378  7968 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:28.820119  6502 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:28.821718  7968 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:28.821920  8029 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:28.822525  8029 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.746505  7732 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.061s
I20260709 12:17:28.819172  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:28.785117  7909 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:28.828263  8052 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:28.828634  7898 raft_consensus.cc:3055] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:28.829167  7898 raft_consensus.cc:740] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:28.830461  8031 raft_consensus.cc:3055] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:28.830857  8031 raft_consensus.cc:740] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:28.805327  7793 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:28.815408  8021 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 12:17:28.831696  8031 consensus_queue.cc:260] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:28.834491  8031 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:28.841724  7962 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:28.847884  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:28.848729  6501 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:17:28.768057  7879 consensus_queue.cc:237] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:28.818423  7995 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:28.817426  7930 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:28.822762  7994 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:28.839708  7977 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:28.859048  7930 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:28.830237  7898 consensus_queue.cc:260] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:28.842907  8012 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:28.844223  7267 consensus_meta.cc:287] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.599s	user 0.007s	sys 0.006s
I20260709 12:17:28.872701  7267 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:17:28.857427  7882 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:28.864419  8038 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:28.874615  8038 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:28.875706  7882 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:28.878629  7270 raft_consensus.cc:1240] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:28.884362  6506 raft_consensus.cc:1275] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:28.877053  7837 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.885566  7837 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
W20260709 12:17:28.887563  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 285401us
I20260709 12:17:28.909271  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:28.916577  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:28.917615  6506 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.903512  6550 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:28.927104  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:28.929694  7898 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:28.930681  8012 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:17:28.932430  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 44252.5us
I20260709 12:17:28.932629  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:28.934338  8021 consensus_queue.cc:1059] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:17:28.935096  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:28.897146  7837 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:28.936484  7962 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:28.936913  7962 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:28.937258  7962 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.937386  7793 raft_consensus.cc:3055] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:28.937891  7793 raft_consensus.cc:740] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:28.938810  7793 consensus_queue.cc:260] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:28.940076  7793 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.928654  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:28.941171  6649 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.941517  7977 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:28.943346  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:28.944751  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:28.944756  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:28.946789  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:28.946682  8029 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:28.947752  6507 raft_consensus.cc:1275] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:28.949043  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:28.949795  7267 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:28.952270  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:28.954126  7957 raft_consensus.cc:3055] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:28.954627  7957 raft_consensus.cc:740] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:28.955770  6507 raft_consensus.cc:1240] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:28.960580  8038 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:28.962793  7238 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:28.965174  6654 raft_consensus.cc:1240] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:28.949891  6508 raft_consensus.cc:1275] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:28.968425  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:28.969599  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 323947us
I20260709 12:17:28.980558  6655 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:28.983176  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:28.983871  6508 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:28.987255  6652 raft_consensus.cc:1240] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:28.955591  7957 consensus_queue.cc:260] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:28.929167  7995 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:28.967335  8043 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:29.000895  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:28.969689  8024 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
W20260709 12:17:29.056061  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 43882.8us
I20260709 12:17:29.061921  7694 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:29.063095  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:29.062829  7694 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:29.063915  6575 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:29.065227  8062 consensus_queue.cc:1059] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:29.066282  8018 consensus_queue.cc:260] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:29.067449  8018 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:29.067507  6578 raft_consensus.cc:1275] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:29.068239  6583 raft_consensus.cc:1275] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:29.068969  6579 raft_consensus.cc:1240] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:29.069416  7977 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:29.070011  7694 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:29.069708  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:29.074023  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:29.074609  6580 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:29.050149  6622 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:29.078600  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:29.012202  7936 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:28.994810  8036 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:29.011308  7944 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:29.011036  7918 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:29.086719  6622 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:29.002096  7995 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:29.089468  6580 raft_consensus.cc:2410] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 12:17:29.070555  7911 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:29.091120  6504 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:29.049885  7939 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.092154  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:29.070258  7977 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:29.093783  6622 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:29.101768  8045 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:29.108407  7911 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:29.110487  7911 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:29.101763  7957 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:29.114675  6623 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:29.118010  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:17:29.098479  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:29.093454  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:29.096532  7918 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:29.116817  6623 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:29.071967  7980 consensus_queue.cc:1048] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 12:17:29.116832  8038 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:29.089838  8031 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:29.125573  6500 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:17:29.129638  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:29.131626  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:29.132130  6583 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:29.132229  6580 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:17:29.134001  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:29.118551  8029 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:29.123340  8045 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:29.090016  7936 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:29.147336  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:29.125021  7930 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:29.116824  8051 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:29.105576  7785 consensus_queue.cc:1059] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 12:17:29.129973  8008 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:29.134586  8063 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:29.099263  7944 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:29.085559  6579 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:17:29.169394  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:29.170070  6578 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:29.097587  8036 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:29.172436  7930 raft_consensus.cc:697] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:29.172809  7785 raft_consensus.cc:3055] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:29.173204  7785 raft_consensus.cc:740] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:29.173360  7930 consensus_queue.cc:237] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:29.174010  7785 consensus_queue.cc:260] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:29.174506  7918 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:29.175205  7785 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:29.176801  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:29.169574  6579 raft_consensus.cc:606] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:29.178197  6579 raft_consensus.cc:614] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:29.179100  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:29.179313  6506 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:29.179482  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:29.180874  7977 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:29.145848  7954 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 12:17:29.180085  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:29.116931  7939 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:29.190317  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:29.191568  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:29.192117  6579 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:29.192852  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:29.201030  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:29.154078  8045 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:29.151988  8038 raft_consensus.cc:697] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 3 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:29.203938  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:29.205811  6583 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:17:29.207020  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:29.209790  6622 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:29.214656  6548 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:29.222957  6548 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:29.193315  7995 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:29.203111  6649 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:29.260991  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:29.262282  6550 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:29.262420  7879 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:29.263448  6582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1289) took 2797 ms. Trace:
I20260709 12:17:29.263633  7995 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:29.205832  8045 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:29.263895  6582 rpcz_store.cc:276] 0709 12:17:26.465631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:27.148152 (+682521us) service_pool.cc:224] Handling call
0709 12:17:27.172396 (+ 24244us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:28.265223 (+1092827us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:28.265256 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:28.817625 (+552369us) spinlock_profiling.cc:243] Waited 2.39 ms on lock 0x7b2c00010c48. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:17:28.832661 (+ 15036us) log.cc:844] Serialized 26 byte log entry
0709 12:17:29.125427 (+292766us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:29.125470 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:29.125537 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:29.263134 (+137597us) raft_consensus.cc:1685] finished
0709 12:17:29.263154 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:29.263364 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1037220,"raft.queue_time_us":1045118,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":116,"spinlock_wait_cycles":5485952,"thread_start_us":325075,"threads_started":2,"wal-append.queue_time_us":406609}
I20260709 12:17:29.243238  7977 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:29.240621  7802 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:29.342031  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:29.346241  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:29.373441  6473 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:29.347039  6579 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:29.360872  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:29.194020  8063 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:29.185434  8008 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:29.243666  8065 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:29.365890  6656 raft_consensus.cc:1240] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:29.377209  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:29.395694  7882 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:29.398880  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:29.404270  7882 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:29.404939  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:29.405985  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:29.406530  6580 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:29.407269  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:29.407572  7882 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:29.399928  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:29.399873  6473 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:29.407887  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:29.408695  7273 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:17:29.408994  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:29.410303  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:29.410363  6623 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:29.418953  8024 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:29.419551  8024 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:29.439325  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:29.440850  7936 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:29.441519  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:29.442127  6503 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:29.443054  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:29.443984  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:29.445757  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 45950.8us
I20260709 12:17:29.446267  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:29.453229  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:29.454499  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:29.459595  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:29.397833  7995 raft_consensus.cc:697] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:29.418968  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:29.419013  7802 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:29.465330  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:29.461838  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:29.475658  6578 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:17:29.437674  7986 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.463343  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:29.422115  7900 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:29.401954  8043 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.456382  6622 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:29.486478  8051 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:29.486929  6548 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:29.453624  7876 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:17:29.464589  8058 log.cc:927] Time spent T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.764s	user 0.005s	sys 0.000s
I20260709 12:17:29.496369  6622 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:29.468022  8061 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.501152  6548 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:29.503453  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:29.445693  7267 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:29.448730  7919 consensus_queue.cc:1048] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 12:17:29.546379  6503 raft_consensus.cc:1275] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:17:29.547219  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 46182.6us
I20260709 12:17:29.451429  7898 consensus_queue.cc:1059] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 12:17:29.477421  7802 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:29.567385  6508 raft_consensus.cc:1275] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:29.567616  8065 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:29.517431  7837 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:29.582193  7944 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:29.499729  7986 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:29.586208  7986 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:29.587157  7898 raft_consensus.cc:3055] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:29.567178  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:29.588224  7802 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:29.589046  8075 log.cc:927] Time spent T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:17:29.516945  8051 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:29.449697  8038 consensus_queue.cc:237] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:29.489195  7900 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:29.594146  6578 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:29.460592  8068 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.100s
W20260709 12:17:29.597039  6655 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.013s	sys 0.046s
I20260709 12:17:29.519454  8031 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:29.617641  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:29.515990  8052 raft_consensus.cc:2764] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:17:29.620962  6622 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:29.622774  6622 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:29.515215  7876 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:29.547762  8024 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:29.608672  6655 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.013s	sys 0.046s
W20260709 12:17:29.609088  6658 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.004s	sys 0.015s
W20260709 12:17:29.609439  6652 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.009s	sys 0.011s
I20260709 12:17:29.614176  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:29.615423  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:29.634016  6652 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.009s	sys 0.011s
W20260709 12:17:29.605703  6657 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.043s	sys 0.044s
I20260709 12:17:29.598846  7898 raft_consensus.cc:740] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:17:29.634354  6658 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.004s	sys 0.016s
I20260709 12:17:29.640125  6622 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:29.609524  7900 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:29.504794  8043 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:29.642228  6622 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:29.644634  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:29.616971  7939 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:29.595317  8061 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:29.632386  6653 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.006s	sys 0.031s
W20260709 12:17:29.668362  6653 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.007s	sys 0.032s
I20260709 12:17:29.631538  7936 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:29.670109  8043 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:29.647341  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:29.643610  8041 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:29.671897  6622 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:29.673457  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:29.674904  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:29.676173  7936 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:29.638427  6657 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.044s	sys 0.044s
I20260709 12:17:29.667833  8061 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:29.639283  7882 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:29.639472  8031 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:29.645463  7957 consensus_meta.cc:287] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.534s	user 0.010s	sys 0.000s
I20260709 12:17:29.649662  7954 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:29.686918  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:29.678732  7930 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:29.692457  6578 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:29.649614  8051 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:29.715242  6581 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.003s	sys 0.046s
I20260709 12:17:29.717515  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:29.719350  6581 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.003s	sys 0.046s
I20260709 12:17:29.719231  6474 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:29.718446  6650 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 1.
I20260709 12:17:29.729914  7837 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:29.629325  7995 consensus_queue.cc:237] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:29.756611  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:29.691341  7936 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:29.743479  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:29.745217  8016 raft_consensus.cc:3055] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:29.745880  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:29.747087  8051 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:29.747543  8085 log.cc:927] Time spent T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:17:29.750761  7957 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:29.758579  8031 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.761533  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:29.627678  7876 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:29.692195  7898 consensus_queue.cc:260] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:29.736939  7265 raft_consensus.cc:1240] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:17:29.739599  7930 raft_consensus.cc:697] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:29.830885  8051 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:29.833897  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:29.834848  7962 consensus_meta.cc:287] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.897s	user 0.010s	sys 0.003s
I20260709 12:17:29.765751  8063 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:29.776368  8016 raft_consensus.cc:740] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:29.767190  7936 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:29.692461  7882 raft_consensus.cc:697] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:29.849496  6579 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:17:29.754170  7919 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:29.873072  8018 consensus_meta.cc:287] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.805s	user 0.009s	sys 0.008s
I20260709 12:17:29.873072  7970 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:29.877175  6577 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:17:29.898895  6577 raft_consensus.cc:606] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:29.878201  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:29.880879  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:29.881628  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:29.853407  8043 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:29.910096  6504 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:29.914915  6578 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 3.
I20260709 12:17:29.910259  6576 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:29.911824  6583 raft_consensus.cc:1275] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:29.880250  8052 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:29.894428  7936 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:29.876379  7957 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:29.935611  7237 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.017s	sys 0.033s
W20260709 12:17:29.935999  7237 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.017s	sys 0.033s
I20260709 12:17:29.877737  8031 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:29.959731  6576 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:29.924393  7267 raft_consensus.cc:1240] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:29.892992  7962 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:29.978659  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 303818us
I20260709 12:17:29.979017  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:29.937248  8021 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:29.913668  7970 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:29.987587  6622 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:29.993292  6653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 969) took 1029 ms. Trace:
I20260709 12:17:29.994582  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:17:29.995575  6582 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.000s	sys 0.035s
I20260709 12:17:29.975354  6657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 972) took 1007 ms. Trace:
I20260709 12:17:29.929109  8052 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:29.978052  8041 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:29.996054  6582 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.000s	sys 0.035s
W20260709 12:17:29.996289  6654 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.024s	sys 0.040s
W20260709 12:17:30.003644  6654 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.024s	sys 0.040s
I20260709 12:17:29.925062  8043 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:30.005445  6576 raft_consensus.cc:1240] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:30.007969  6580 raft_consensus.cc:2393] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 4: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:29.994488  7980 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:29.993943  6653 rpcz_store.cc:276] 0709 12:17:28.964120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.967426 (+  3306us) service_pool.cc:224] Handling call
0709 12:17:29.047748 (+ 80322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:29.080945 (+ 33197us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:17:29.668883 (+587938us) log.cc:844] Serialized 14 byte log entry
0709 12:17:29.991791 (+322908us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:29.991848 (+    57us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:29.991897 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:29.992207 (+   310us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:29.993213 (+  1006us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":581690,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"thread_start_us":869534,"threads_started":2}
W20260709 12:17:30.013379  8090 log.cc:927] Time spent T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:17:30.013607  7268 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:17:30.016939  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:30.022962  8029 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:30.022710  6576 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:17:30.027187  6576 raft_consensus.cc:606] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:30.027558  6576 raft_consensus.cc:614] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:30.027756  8051 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:30.029001  6576 raft_consensus.cc:1240] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:30.030050  8043 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:29.979701  7930 consensus_queue.cc:237] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:29.998370  6657 rpcz_store.cc:276] 0709 12:17:28.965125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.966982 (+  1857us) service_pool.cc:224] Handling call
0709 12:17:28.997709 (+ 30727us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:29.058373 (+ 60664us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:17:29.689772 (+631399us) log.cc:844] Serialized 14 byte log entry
0709 12:17:29.926339 (+236567us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:29.926424 (+    85us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:29.926500 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:29.936052 (+  9552us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:29.972059 (+ 36007us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":596614,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":70,"raft.queue_time_us":7162,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":4554,"thread_start_us":824932,"threads_started":2}
I20260709 12:17:30.038005  7957 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:30.039740  6549 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:30.011011  7882 consensus_queue.cc:237] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:30.045823  8051 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:29.995262  8021 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:30.054754  8021 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:30.068837  8029 raft_consensus.cc:697] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:30.070935  8102 consensus_queue.cc:1048] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:30.070823  8029 consensus_queue.cc:237] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:30.073431  8031 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:30.123306  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1325) took 1046 ms. Trace:
W20260709 12:17:30.130818  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 133532us
I20260709 12:17:30.123805  6580 rpcz_store.cc:276] 0709 12:17:29.081385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.407054 (+325669us) service_pool.cc:224] Handling call
0709 12:17:30.123081 (+716027us) spinlock_profiling.cc:243] Waited 600 ms on lock 0x7b540013f954. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:30.123240 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1379607936}
I20260709 12:17:30.141201  6657 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:55116
I20260709 12:17:30.120195  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:30.151142  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:30.121850  8094 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:17:30.014708  7962 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:30.111316  8008 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:30.169520  6580 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
I20260709 12:17:30.171375  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:30.182462  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:30.183149  6501 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:30.186434  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:30.142807  8043 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.187058  6501 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:30.200925  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1013) took 1066 ms. Trace:
I20260709 12:17:30.201483  7970 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:30.201361  6577 rpcz_store.cc:276] 0709 12:17:29.134970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.911343 (+776373us) service_pool.cc:224] Handling call
0709 12:17:29.911728 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:30.017634 (+105906us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:30.017675 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:30.017732 (+    57us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:30.017776 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:30.200692 (+182916us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b54001cbe54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:30.200733 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:30.200865 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":242977664}
I20260709 12:17:30.209803  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1015) took 1046 ms. Trace:
I20260709 12:17:30.209199  6654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 982) took 1228 ms. Trace:
W20260709 12:17:30.189306  8073 log.cc:927] Time spent T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.420s	user 0.001s	sys 0.001s
I20260709 12:17:30.215178  6657 raft_consensus.cc:606] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:30.215950  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1344) took 1044 ms. Trace:
I20260709 12:17:30.216598  6580 rpcz_store.cc:276] 0709 12:17:29.171836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:30.212787 (+1040951us) service_pool.cc:224] Handling call
0709 12:17:30.215436 (+  2649us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:30.215620 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:30.215645 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:30.215686 (+    41us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:30.215715 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:30.215771 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:30.215885 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:30.210655  8097 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:30.210358  8029 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.222587  8029 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:30.191205  8038 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.211500  6654 rpcz_store.cc:276] 0709 12:17:28.980888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.981371 (+   483us) service_pool.cc:224] Handling call
0709 12:17:28.981580 (+   209us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:28.981758 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:17:29.455269 (+473511us) log.cc:844] Serialized 14 byte log entry
0709 12:17:30.208717 (+753448us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:30.208769 (+    52us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:30.208810 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:30.208968 (+   158us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:30.209123 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49312,"prepare.queue_time_us":686701,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":90,"raft.queue_time_us":68491,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":84,"thread_start_us":1157992,"threads_started":3,"wal-append.queue_time_us":726480}
I20260709 12:17:30.215334  7271 raft_consensus.cc:2393] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 3: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:30.245833  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:30.255707  8029 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:30.256232  7909 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.262434  7270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38562 (request call id 337) took 1919 ms. Trace:
I20260709 12:17:30.262851  7270 rpcz_store.cc:276] 0709 12:17:28.343403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.343563 (+   160us) service_pool.cc:224] Handling call
0709 12:17:28.878455 (+534892us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:28.879032 (+   577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:30.262217 (+1383185us) spinlock_profiling.cc:243] Waited 535 ms on lock 0x7b54005ddc50. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:30.262374 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1229387008}
I20260709 12:17:30.265059  6579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1324) took 1188 ms. Trace:
I20260709 12:17:30.211304  6583 rpcz_store.cc:276] 0709 12:17:29.163898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.917997 (+754099us) service_pool.cc:224] Handling call
0709 12:17:29.918374 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:29.919031 (+   657us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:29.919075 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:29.934847 (+ 15772us) log.cc:844] Serialized 26 byte log entry
0709 12:17:30.209411 (+274564us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:30.209458 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:30.209531 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:30.209580 (+    49us) raft_consensus.cc:1685] finished
0709 12:17:30.209590 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:30.209718 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1416,"spinlock_wait_cycles":14162304,"thread_start_us":200415,"threads_started":1,"wal-append.queue_time_us":200385}
I20260709 12:17:30.265632  6579 rpcz_store.cc:276] 0709 12:17:29.076144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.376906 (+300762us) service_pool.cc:224] Handling call
0709 12:17:30.264829 (+887923us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b54001d10d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:30.264979 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":646972160}
I20260709 12:17:30.214426  7918 consensus_queue.cc:1059] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 12:17:30.278287  8052 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:30.280289  7876 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:30.288395  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1335) took 1133 ms. Trace:
I20260709 12:17:30.209240  8043 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:30.296608  6501 raft_consensus.cc:1240] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:30.298996  6583 raft_consensus.cc:1275] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:30.302348  6583 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:30.258230  8092 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.309216  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1011) took 1212 ms. Trace:
I20260709 12:17:30.320842  6584 rpcz_store.cc:276] 0709 12:17:29.096741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.444152 (+347411us) service_pool.cc:224] Handling call
0709 12:17:29.445032 (+   880us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:29.445334 (+   302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:29.445368 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:29.454178 (+  8810us) log.cc:844] Serialized 26 byte log entry
0709 12:17:29.853943 (+399765us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:29.854000 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:29.854063 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:30.308878 (+454815us) raft_consensus.cc:1685] finished
0709 12:17:30.308906 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:30.309129 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":510,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":15842,"thread_start_us":352983,"threads_started":1,"wal-append.queue_time_us":513401}
I20260709 12:17:30.313226  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:30.256817  7909 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:30.319998  8092 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:30.340549  8092 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:30.340651  7909 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:30.309026  7737 consensus_queue.cc:1059] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:17:30.325011  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:30.344102  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:30.344555  6500 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:17:30.345136  6657 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
I20260709 12:17:30.345714  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:30.346323  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:30.344169  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:30.346949  7240 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:30.288937  6581 rpcz_store.cc:276] 0709 12:17:29.154656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.913612 (+758956us) service_pool.cc:224] Handling call
0709 12:17:29.913944 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:29.914275 (+   331us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:29.914299 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:29.929486 (+ 15187us) log.cc:844] Serialized 26 byte log entry
0709 12:17:30.122201 (+192715us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:30.122237 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:30.122302 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:30.288143 (+165841us) raft_consensus.cc:1685] finished
0709 12:17:30.288170 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:30.288307 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":474,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":8488,"thread_start_us":138899,"threads_started":1,"wal-append.queue_time_us":310828}
I20260709 12:17:30.326886  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:30.327208  8100 log.cc:927] Time spent T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:17:30.327639  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:30.347242  6654 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
I20260709 12:17:30.308629  7785 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:30.320645  8021 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:30.283241  7900 consensus_meta.cc:287] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.630s	user 0.001s	sys 0.009s
I20260709 12:17:30.351404  6509 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 3.
I20260709 12:17:30.377885  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:30.337061  7876 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:30.338088  8045 consensus_meta.cc:287] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.997s	user 0.011s	sys 0.002s
I20260709 12:17:30.349834  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:30.342204  7981 raft_consensus.cc:3055] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:30.352474  7240 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:30.334662  8043 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:30.334834  7802 consensus_meta.cc:287] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.720s	user 0.006s	sys 0.006s
I20260709 12:17:30.380812  6506 raft_consensus.cc:2393] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 3: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:30.356706  7909 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.380021  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:30.324318  7936 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:30.381904  7981 raft_consensus.cc:740] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:30.395447  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:30.385411  8043 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:30.397495  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:30.399237  8122 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:30.399951  8122 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:30.400477  8122 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:30.401158  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 981) took 1420 ms. Trace:
W20260709 12:17:30.399111  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 80633.3us
I20260709 12:17:30.380354  7785 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:30.414367  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:30.381395  8021 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:30.382827  8045 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:17:30.381564  7900 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:30.384500  7802 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:30.392602  7909 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:30.395784  7271 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:30.415532  6509 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 1.
I20260709 12:17:30.481663  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1345) took 1309 ms. Trace:
I20260709 12:17:30.416607  6651 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:30.491994  7238 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:30.459939  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:30.518534  6577 rpcz_store.cc:276] 0709 12:17:29.172091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:30.480581 (+1308490us) service_pool.cc:224] Handling call
0709 12:17:30.481006 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:30.481228 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:30.481266 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:30.481320 (+    54us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:30.481357 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:30.481427 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:30.481588 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:30.513269  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:30.520632  6580 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
I20260709 12:17:30.401665  6655 rpcz_store.cc:276] 0709 12:17:28.980589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.981289 (+   700us) service_pool.cc:224] Handling call
0709 12:17:28.981508 (+   219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:28.981673 (+   165us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:17:30.079650 (+1097977us) spinlock_profiling.cc:243] Waited 4.33 ms on lock 0x7b18001a21e0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2875a314bc 00007f2872d7f48b 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d99455 00007f28798813c7 00007f2875e939a7 00007f2875ea30bf
0709 12:17:30.079837 (+   187us) log.cc:844] Serialized 14 byte log entry
0709 12:17:30.383403 (+303566us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:30.383475 (+    72us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:30.400868 (+ 17393us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:30.400945 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:30.401093 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":646481,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":4071,"raft.queue_time_us":7004,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"spinlock_wait_cycles":21697024,"thread_start_us":940699,"threads_started":3,"wal-append.queue_time_us":139532}
I20260709 12:17:30.404644  8067 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:17:30.541280  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 49428.5us
I20260709 12:17:30.542021  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:30.542621  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:30.465375  8062 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:30.404953  8043 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:30.549291  6658 raft_consensus.cc:2393] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 2: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:30.423039  8118 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:30.550720  6474 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:30.499279  8117 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:30.447227  8029 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.549073  8104 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:30.447544  7876 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:30.459772  7793 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:30.465675  7918 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:30.558727  8127 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:30.519606  8093 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:30.552145  8124 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.551932  8043 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:30.551648  7930 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:30.569453  8092 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:30.548943  8062 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:30.569736  8104 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:30.497373  8016 consensus_queue.cc:260] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:30.500746  7981 consensus_queue.cc:260] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:30.573006  8052 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:30.570953  8029 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:30.567736  8118 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:30.575642  7876 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:30.579386  7793 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:30.570192  8117 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:30.635162  8116 log.cc:927] Time spent T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 12:17:30.635824  8117 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:30.638265  7909 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:30.633492  7802 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:30.639642  8038 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:30.640393  8052 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:30.642632  8052 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:30.609270  7936 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:30.583295  7918 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:30.582300  8008 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:30.644476  8021 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:30.644866  8008 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:30.645610  8128 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.646092  8128 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:30.646991  8008 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:30.647671  8104 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.559703  8067 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:30.608701  8127 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:17:30.632894  8110 log.cc:927] Time spent T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.413s	user 0.005s	sys 0.000s
I20260709 12:17:30.633489  8134 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:30.633985  8045 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:30.657675  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:30.650067  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:30.628749  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 85259us
I20260709 12:17:30.658739  6650 raft_consensus.cc:2410] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:30.658888  6507 raft_consensus.cc:2410] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:30.636415  6651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 976) took 1665 ms. Trace:
I20260709 12:17:30.651647  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:30.657694  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:30.659474  6651 rpcz_store.cc:276] 0709 12:17:28.970674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.971459 (+   785us) service_pool.cc:224] Handling call
0709 12:17:30.080582 (+1109123us) spinlock_profiling.cc:243] Waited 1.67 ms on lock 0x7b5800015998. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:17:30.080854 (+   272us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:30.520185 (+439331us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:30.636136 (+115951us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b54002a58d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:30.636355 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":775102464}
I20260709 12:17:30.661165  6654 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:30.590266  7930 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:30.662266  6657 raft_consensus.cc:1240] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:30.591035  8092 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:30.663187  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:30.619472  7785 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:30.648274  8128 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:30.692543  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:30.657752  7936 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:30.657047  8129 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:30.693743  6653 raft_consensus.cc:2393] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 2: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:30.699666  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:30.702934  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:30.703696  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:30.659770  8134 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:30.705675  6507 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.700973  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:30.701159  6504 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:30.706471  6503 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:17:30.675604  7930 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.589820  8124 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:30.705159  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:30.717388  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:30.691112  8092 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.694903  8140 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:30.732080  6504 raft_consensus.cc:1275] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:30.702407  8128 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:30.774438  6504 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 12:17:30.774887  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:30.774721  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 79293.9us
I20260709 12:17:30.777298  6503 raft_consensus.cc:1240] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:30.708969  8134 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:30.716706  8117 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:30.587623  8093 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:30.711814  7882 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:30.709888  7936 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:30.790778  7918 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:30.791644  7936 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:17:30.791934  8130 log.cc:927] Time spent T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 12:17:30.794225  8118 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:30.775925  8139 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.821194  8118 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:30.821341  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:30.822034  7274 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:30.821872  8118 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:30.824287  8118 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:30.825069  7270 raft_consensus.cc:1240] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:30.693248  8127 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:30.713111  8129 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:30.806134  8135 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:30.807559  8139 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:30.817960  8062 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:30.798653  8117 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:30.786407  8128 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:30.800573  7882 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:30.849803  8138 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.850500  6502 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 1445) took 1986 ms. Trace:
I20260709 12:17:30.850409  8138 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:30.850997  6502 rpcz_store.cc:276] 0709 12:17:28.863856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.068776 (+204920us) service_pool.cc:224] Handling call
0709 12:17:29.069193 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:29.250879 (+181686us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:29.258662 (+  7783us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:29.273991 (+ 15329us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:29.317921 (+ 43930us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:30.850228 (+1532307us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b540039e754. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:30.850281 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:30.850416 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":340566912}
I20260709 12:17:30.853595  7999 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.854106  7999 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:30.856158  7999 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:30.856554  8136 consensus_queue.cc:1059] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:17:30.858702  8093 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:30.860980  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:30.863696  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:30.777290  8140 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:30.867820  6657 raft_consensus.cc:1240] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:30.841673  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:30.870821  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:30.872480  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:30.874440  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:30.875670  7876 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:17:30.902798  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 97857.1us
I20260709 12:17:30.859350  8093 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.936954  6657 raft_consensus.cc:2410] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:30.937044  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:30.897637  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:30.898924  6655 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:30.843850  8043 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:30.930095  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:30.939108  6651 raft_consensus.cc:2410] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:30.940814  6656 raft_consensus.cc:2410] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:30.941560  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:30.935946  7882 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:17:30.946858  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 72514.7us
I20260709 12:17:30.947360  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:30.947722  8052 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:30.868543  7936 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:30.951519  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:30.943197  8029 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:30.935321  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:30.950836  6653 raft_consensus.cc:2410] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:30.896698  8124 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:30.934111  8117 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:30.956529  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:30.846985  8135 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:30.958681  6503 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:30.962069  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:30.948484  7939 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:30.870133  7999 raft_consensus.cc:3055] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:30.898255  7271 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:30.959132  6475 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:30.957629  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:30.972221  7270 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:17:30.972797  8124 raft_consensus.cc:670] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:30.955956  7936 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:30.952713  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:30.978430  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:30.934553  8104 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:30.869285  8118 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:30.975802  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:30.961658  8029 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:17:30.987161  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:30.993266  6651 raft_consensus.cc:1240] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:30.993573  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:30.994181  7274 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:30.943202  8093 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:30.985946  6653 raft_consensus.cc:1240] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:30.985661  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:30.979146  8142 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:30.998795  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:30.953504  8052 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:30.970664  7999 raft_consensus.cc:740] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:30.985206  6649 raft_consensus.cc:2393] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 4: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:30.995337  8029 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:30.951877  7876 consensus_queue.cc:1059] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:17:31.005769  8138 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:30.962075  8045 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:17:31.004428  6650 raft_consensus.cc:1240] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:31.008252  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 59293.1us
I20260709 12:17:31.000634  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:31.007828  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:31.002767  6504 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 3.
I20260709 12:17:30.992028  6507 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:31.010843  8142 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:31.024855  8052 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:31.005914  8036 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:31.028888  8139 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:31.029543  7930 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:31.091653  8145 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.032691  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:31.092147  6652 raft_consensus.cc:2393] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 4: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:31.092510  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:31.092159  8145 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:31.093370  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:31.093951  6653 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:31.096120  6474 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:31.035470  8104 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:31.018028  8143 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.039390  8134 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:31.045719  8138 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:30.959179  8132 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.065s
I20260709 12:17:31.090016  6651 raft_consensus.cc:2393] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 4: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:31.089679  7274 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
W20260709 12:17:31.088263  8144 log.cc:927] Time spent T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:17:31.012609  8029 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:31.099915  6622 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:31.101477  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:31.084801  7936 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:31.002332  8118 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:31.094923  8145 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:31.102552  7876 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:31.109395  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:31.097612  8142 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:31.099550  7999 consensus_queue.cc:260] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:31.109632  8131 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 12:17:31.111572  7999 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:31.114750  6622 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:31.119027  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:31.120252  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:31.121538  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:31.122013  6507 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:31.122430  8142 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:31.122833  6655 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 4: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:17:31.123016  8096 log.cc:927] Time spent T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.971s	user 0.002s	sys 0.002s
I20260709 12:17:31.124817  6582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1009) took 2041 ms. Trace:
I20260709 12:17:31.125209  6582 rpcz_store.cc:276] 0709 12:17:29.083001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.414061 (+331060us) service_pool.cc:224] Handling call
0709 12:17:29.414545 (+   484us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:29.461291 (+ 46746us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:29.461323 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:29.468083 (+  6760us) log.cc:844] Serialized 26 byte log entry
0709 12:17:30.003629 (+535546us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:30.174529 (+170900us) log.cc:844] Serialized 14 byte log entry
0709 12:17:30.186714 (+ 12185us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:30.186823 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:31.001748 (+814925us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b24003d75b8. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872c229c4 00007f2872c20acb 00007f2872ddbdc9 00007f2872ddba55 00007f2872ddb09d 00007f2875a0d10e 00007f2875a2488c 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221
0709 12:17:31.124501 (+122753us) raft_consensus.cc:1685] finished
0709 12:17:31.124598 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:31.124742 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63721,"prepare.queue_time_us":196068,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":59973,"raft.run_cpu_time_us":341,"raft.run_wall_time_us":25233,"spinlock_wait_cycles":316674304,"thread_start_us":663293,"threads_started":2,"wal-append.queue_time_us":542028}
I20260709 12:17:31.127825  8045 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:31.128711  8016 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:31.128588  8045 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:31.131682  8045 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:31.133270  6550 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:31.135828  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:31.103688  8138 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:31.134578  7962 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:31.112108  7876 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:31.141171  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:31.139849  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:31.140753  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:31.144083  6653 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:31.120923  8117 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:31.145316  8069 consensus_queue.cc:1059] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:17:31.148981  6623 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:31.146953  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:31.150902  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:31.151542  6651 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:31.137784  8127 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:31.152386  6653 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:31.144012  7876 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:31.157505  6576 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:17:31.158377  7898 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:31.159566  6576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1341) took 1988 ms. Trace:
I20260709 12:17:31.146100  8117 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:31.160529  8117 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:31.153707  7936 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:31.166882  7876 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:31.167588  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:31.167658  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:31.169006  7876 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:31.154394  8097 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:31.154700  7930 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:31.156497  6658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49792 (request call id 67) took 1438 ms. Trace:
W20260709 12:17:31.157239  6578 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54204 (request call id 77) took 1456 ms (client timeout 1783 ms). Trace:
I20260709 12:17:31.154241  8135 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:31.184154  6658 rpcz_store.cc:276] 0709 12:17:29.717654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.745525 (+ 27871us) service_pool.cc:224] Handling call
0709 12:17:31.156266 (+1410741us) spinlock_profiling.cc:243] Waited 773 ms on lock 0x7b54002a4ed4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:31.156436 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1778102144}
W20260709 12:17:31.184465  6578 rpcz_store.cc:269] 0709 12:17:29.701099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:29.844284 (+143185us) service_pool.cc:224] Handling call
0709 12:17:31.157044 (+1312760us) spinlock_profiling.cc:243] Waited 4.94 ms on lock 0x7b540032a354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:31.157181 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11356544}
I20260709 12:17:31.170347  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:31.171191  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:31.186652  6509 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:31.140421  8093 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:31.159953  6576 rpcz_store.cc:276] 0709 12:17:29.171130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:30.030720 (+859590us) service_pool.cc:224] Handling call
0709 12:17:31.157172 (+1126452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:31.157879 (+   707us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:31.159371 (+  1492us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540032a350. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:31.159507 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2589814528}
I20260709 12:17:31.149895  8140 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:31.190347  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:31.190828  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:31.191532  6651 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:17:31.188284  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:31.192801  6505 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
I20260709 12:17:31.105476  8143 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:31.173645  8142 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.189533  6581 raft_consensus.cc:2393] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 2: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:31.195088  6507 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
W20260709 12:17:31.198565  7238 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 2.024s (SENT) [suppressed 14 similar messages]
W20260709 12:17:31.199331  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 60931.1us
I20260709 12:17:31.199273  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:31.202250  8143 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:31.202807  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:31.203498  6502 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
I20260709 12:17:31.204380  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:31.205086  8152 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:31.207357  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:31.221239  6550 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:31.205754  8147 consensus_queue.cc:1059] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.102s
I20260709 12:17:31.222856  6550 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:31.205560  8069 consensus_queue.cc:1059] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:31.219877  6507 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 1.
I20260709 12:17:31.193904  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:17:31.206918  8151 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 12:17:31.228158  8118 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:31.246501  8134 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:31.220818  8097 raft_consensus.cc:697] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:31.238513  8138 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:31.226318  8122 consensus_meta.cc:287] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.823s	user 0.008s	sys 0.006s
I20260709 12:17:31.248777  8122 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:31.250276  8064 raft_consensus.cc:3055] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:31.233667  8092 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:31.248674  8097 consensus_queue.cc:237] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:31.207921  8142 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:31.256882  6509 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 3.
W20260709 12:17:31.262161  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 63639.5us
I20260709 12:17:31.265038  6502 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
I20260709 12:17:31.266218  7971 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:31.266860  7971 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:31.269037  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:31.225744  8152 raft_consensus.cc:697] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:31.251485  7271 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:31.255775  8064 raft_consensus.cc:740] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:31.283255  7270 raft_consensus.cc:2393] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 4: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:17:31.298339  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:31.301512  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:31.301282  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:31.313902  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:31.321732  6578 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:31.318418  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:31.298802  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:31.295342  6505 raft_consensus.cc:1275] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:17:31.322990  6505 pending_rounds.cc:85] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 1
I20260709 12:17:31.323266  6505 pending_rounds.cc:107] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:17:31.283373  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:31.320747  6581 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:31.313479  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:31.323508  6505 raft_consensus.cc:2889] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:17:31.324668  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 101892us
I20260709 12:17:31.325610  6550 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:31.318760  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:31.299261  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:31.330936  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:31.331351  6658 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
I20260709 12:17:31.330345  6651 raft_consensus.cc:2393] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 2: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:31.332708  6509 raft_consensus.cc:2410] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 12:17:31.324440  6576 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:31.337312  8159 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:31.338755  6505 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:31.340183  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:31.338788  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:31.293222  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:31.337857  8159 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:31.350220  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:31.353545  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:31.355091  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:31.316794  8092 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:31.345853  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:31.329391  7936 raft_consensus.cc:3055] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:31.338852  8156 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.315814  8117 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:31.380863  8122 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:31.370074  7980 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:31.361352  7900 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:31.370255  8155 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.385074  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:31.354573  8152 consensus_queue.cc:237] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:17:31.411904  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:31.381371  8142 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:31.371573  7936 raft_consensus.cc:740] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:31.422662  8117 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:31.375835  8069 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:31.426327  7954 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.543663  6506 raft_consensus.cc:1275] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:17:31.545117  6506 pending_rounds.cc:85] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 1
I20260709 12:17:31.545432  6506 pending_rounds.cc:107] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:17:31.545725  6506 raft_consensus.cc:2889] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:31.424605  8092 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:31.539283  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:31.547298  8024 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:31.547605  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:31.384974  8156 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:31.549398  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:31.551554  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 176021us
I20260709 12:17:31.485744  7980 raft_consensus.cc:697] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:31.478116  8157 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.085s
I20260709 12:17:31.562834  8024 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:31.565331  8024 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:31.566176  7936 consensus_queue.cc:260] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:31.567567  7936 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:31.578563  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:31.561259  6576 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:31.583509  6657 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:31.584990  7238 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:31.593263  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:31.598760  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:31.551635  8159 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:31.599509  7238 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:31.605106  7238 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:31.606292  7238 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:31.607288  7238 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:31.607345  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:31.608110  6581 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:17:31.599812  6506 raft_consensus.cc:2393] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 4: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:31.608635  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:31.609246  6576 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:31.608183  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:31.610473  6582 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
W20260709 12:17:31.612208  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 276883us
I20260709 12:17:31.614773  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:31.615712  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:31.617295  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:31.625453  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 254073us
I20260709 12:17:31.625952  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:31.385735  8064 consensus_queue.cc:260] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:31.626454  6550 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:31.552254  7954 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:31.628273  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:31.559536  8092 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:31.600956  8134 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.635387  6577 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:31.632767  6473 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:31.636669  6550 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:31.637127  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:31.640858  6550 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:31.641223  6578 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:31.643074  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:31.607779  8036 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:31.651448  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:31.652503  6652 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:31.484886  8155 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:31.665886  8155 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:31.650271  7995 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:31.617540  8138 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:31.613735  8127 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:31.656538  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:31.617862  8140 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:31.617093  8093 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:31.669804  6473 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:31.677423  7273 raft_consensus.cc:1240] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:31.678016  6650 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:31.679838  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:31.683439  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:31.684419  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:31.689042  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:31.689831  6580 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 3.
I20260709 12:17:31.651262  8142 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:31.668280  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:31.709460  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:31.710314  6577 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:31.702796  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:31.703801  6583 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:17:31.712478  6583 raft_consensus.cc:606] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:31.658267  8036 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:31.668813  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:31.715667  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:31.672694  8138 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:31.679212  8093 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:31.679945  8063 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:31.677410  8127 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:31.655277  8134 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:31.684830  8162 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:31.683142  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:31.691114  8140 raft_consensus.cc:697] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:31.680120  8135 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:17:31.698462  8142 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:31.724803  8127 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:31.725030  8138 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:31.477555  8041 raft_consensus.cc:2955] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 3 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:31.695078  7785 consensus_queue.cc:1059] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:17:31.709239  8143 consensus_queue.cc:1059] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:17:31.739145  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:31.722899  8156 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:31.724403  7900 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:31.724970  8093 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:31.681322  7980 consensus_queue.cc:237] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:31.742801  8067 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:31.723191  6575 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:31.711233  7995 consensus_queue.cc:1059] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 12:17:31.717545  8036 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:31.724574  8063 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:31.752884  8142 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:31.741154  8163 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:31.723903  8159 consensus_queue.cc:1059] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:17:31.772655  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:31.813531  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:31.814322  6655 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:31.815359  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:31.774093  6654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 988) took 1123 ms. Trace:
W20260709 12:17:31.775394  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 134676us
I20260709 12:17:31.816761  6550 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:31.790220  6577 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 3.
I20260709 12:17:31.816325  6654 rpcz_store.cc:276] 0709 12:17:30.650914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:30.651044 (+   130us) service_pool.cc:224] Handling call
0709 12:17:30.651334 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:30.661548 (+ 10214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:31.773774 (+1112226us) spinlock_profiling.cc:243] Waited 9.55 ms on lock 0x7b54002a35d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:31.774021 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21954944}
I20260709 12:17:31.820832  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:31.821760  6654 raft_consensus.cc:2393] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 4: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:31.795146  6649 raft_consensus.cc:1240] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:31.802111  8063 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:31.765767  8162 consensus_queue.cc:1048] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 12:17:31.773511  7900 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:31.858879  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 136902us
I20260709 12:17:31.792692  8156 consensus_queue.cc:1059] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
W20260709 12:17:31.806834  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 154756us
I20260709 12:17:31.888842  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:31.841619  8029 consensus_meta.cc:287] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.733s	user 0.010s	sys 0.003s
I20260709 12:17:31.851955  8159 consensus_queue.cc:1243] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Peer f2b06aebe97249a4877d798b8118ece4 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:17:31.913014  7238 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:31.917164  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 225553us
I20260709 12:17:31.893095  8092 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:17:31.928612  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 68763us
I20260709 12:17:31.908445  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:31.902129  7954 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:31.901499  8171 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:17:31.933594  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 116918us
I20260709 12:17:31.944370  6550 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:31.930876  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:31.908097  6581 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:31.897957  8152 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:31.900179  8029 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:31.852609  8140 consensus_queue.cc:237] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:31.933460  8172 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:31.966569  6656 raft_consensus.cc:2393] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 3: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:31.960058  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:31.965714  6575 raft_consensus.cc:2393] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 2: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:31.950472  7785 raft_consensus.cc:3055] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:31.968616  8180 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:31.968876  7785 raft_consensus.cc:740] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:31.969211  8180 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:31.969924  8051 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:31.969722  7785 consensus_queue.cc:260] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:31.970942  7785 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:31.951139  8092 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:31.971355  8180 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:31.955096  8152 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:31.992352  8186 consensus_queue.cc:1059] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:31.992877  8152 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:17:31.993711  8168 log.cc:927] Time spent T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:17:31.965137  7995 raft_consensus.cc:3055] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:31.974403  7898 raft_consensus.cc:3055] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:32.001595  8174 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:32.002462  7793 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:32.003201  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 1462) took 1822 ms. Trace:
I20260709 12:17:31.948470  8171 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:32.005776  8029 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:32.015427  7995 raft_consensus.cc:740] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:32.021543  8152 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.022080  8152 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:32.007136  8063 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:32.030891  8171 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:32.033252  8152 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:32.033663  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:17:32.033958  7271 consensus_meta.cc:287] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.694s	user 0.008s	sys 0.003s
I20260709 12:17:32.032146  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:32.035704  6653 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:32.008903  6475 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:32.000829  8176 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:32.050871  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 117782us
I20260709 12:17:32.010123  8067 raft_consensus.cc:3055] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:32.010315  7240 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:32.016626  7898 raft_consensus.cc:740] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:17:32.013062  8170 log.cc:927] Time spent T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:17:32.013460  8159 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:32.070390  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:32.071976  6655 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:32.073342  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:32.075554  6653 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 3.
I20260709 12:17:31.967240  8143 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:32.016985  6501 rpcz_store.cc:276] 0709 12:17:30.181123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:30.202059 (+ 20936us) service_pool.cc:224] Handling call
0709 12:17:30.202357 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:30.309378 (+107021us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.002947 (+1693569us) spinlock_profiling.cc:243] Waited 83 ms on lock 0x7b5400387f54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:32.003127 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":190991232}
I20260709 12:17:31.988299  8038 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:31.966225  8134 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:32.034304  7271 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
W20260709 12:17:32.053856  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 43618.1us
I20260709 12:17:32.010869  8128 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:32.055641  7900 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:32.056417  8127 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:32.057806  7273 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:17:32.053227  8067 raft_consensus.cc:740] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:32.058614  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:32.059384  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:32.059738  8024 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:32.059955  8176 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:32.060479  8154 log.cc:927] Time spent T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.833s	user 0.000s	sys 0.003s
I20260709 12:17:32.067801  7971 raft_consensus.cc:3055] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:17:32.072950  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 64108us
W20260709 12:17:31.996232  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 73990.4us
W20260709 12:17:32.088028  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 143762us
I20260709 12:17:32.058454  7981 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:32.110450  6548 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:32.096043  8190 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:32.110886  6550 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:32.111099  7274 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:32.111801  6548 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:32.088732  8122 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:32.116431  6475 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:32.121613  6548 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:32.175228  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:32.123199  7265 raft_consensus.cc:1275] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:32.121613  6550 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:32.129036  8182 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:32.109720  7270 raft_consensus.cc:2955] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:32.127622  8159 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:32.147061  6657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 1522) took 1202 ms. Trace:
I20260709 12:17:32.181501  6657 rpcz_store.cc:276] 0709 12:17:30.944307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:30.944535 (+   228us) service_pool.cc:224] Handling call
0709 12:17:31.580462 (+635927us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:31.584074 (+  3612us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.146736 (+562662us) spinlock_profiling.cc:243] Waited 636 ms on lock 0x7b54002a3ad0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:32.146988 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1461781248}
I20260709 12:17:32.149274  6579 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
I20260709 12:17:32.152901  8051 raft_consensus.cc:3055] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:32.155342  8117 consensus_queue.cc:1048] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:17:32.159670  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:32.160013  7900 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:32.159366  7980 consensus_queue.cc:1059] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:32.160315  6504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41304 (request call id 90) took 1182 ms. Trace:
I20260709 12:17:32.086247  8143 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:32.089107  8038 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:32.166023  8129 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:32.167085  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:32.167392  8043 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:32.099150  7268 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 4 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:32.114568  7971 raft_consensus.cc:740] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:32.203722  7971 consensus_queue.cc:260] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:32.204900  7971 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:32.172325  6501 raft_consensus.cc:1275] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:32.173411  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:32.208683  8143 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:32.127170  7898 consensus_queue.cc:260] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:32.119230  7738 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:32.142549  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 48763.8us
I20260709 12:17:32.214675  8036 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:32.215410  6473 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:32.216512  7272 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:32.216746  8132 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:32.217212  6473 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:32.180490  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 58974.8us
I20260709 12:17:32.218480  6550 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:32.218659  8024 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:32.219635  6550 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:17:32.180689  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 93938.3us
I20260709 12:17:32.220280  8092 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:32.220993  7240 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:32.222448  8036 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:32.222664  8045 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:32.223536  8024 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:32.183611  7238 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:32.186446  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:32.186586  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:32.225077  6653 raft_consensus.cc:2393] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 4: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:32.225589  7238 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:32.225106  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:32.226163  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:32.227473  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:32.227632  7238 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:32.187282  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:32.228612  6657 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:32.134668  8190 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:32.180138  7981 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:32.189857  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:32.190663  6504 rpcz_store.cc:276] 0709 12:17:30.977984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:30.978123 (+   139us) service_pool.cc:224] Handling call
0709 12:17:32.160154 (+1182031us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b54002c56d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:32.160262 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54401408}
W20260709 12:17:32.232483  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:32.198432  8178 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:32.185021  8051 raft_consensus.cc:740] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:32.263535  8127 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:32.264604  8051 consensus_queue.cc:260] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:32.197769  8063 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:32.198014  8166 raft_consensus.cc:3055] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:32.184967  7962 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:32.265861  8132 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.266253  8166 raft_consensus.cc:740] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:32.266757  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:32.266590  8132 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:32.267266  8166 consensus_queue.cc:260] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:32.186812  8159 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:32.268512  8159 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:32.186940  8163 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:32.188439  8140 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:17:32.176291  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 54773.7us
I20260709 12:17:32.188973  7900 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:32.212468  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:32.301364  6501 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:32.191418  8182 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:32.230347  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:32.230568  6473 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:32.231117  8129 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:32.233244  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:32.231562  7785 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:32.128159  8008 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.060s
I20260709 12:17:32.235790  7737 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:32.286767  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:32.265851  8051 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:32.316447  6504 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:17:32.317293  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:32.317819  6504 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:32.286340  6548 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:32.263536  7265 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:32.224133  8024 raft_consensus.cc:697] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:32.312495  6508 raft_consensus.cc:2955] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:32.300652  7898 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:32.171957  8067 consensus_queue.cc:260] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:32.233716  6550 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:32.308197  6658 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:32.308938  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 78427.6us
I20260709 12:17:32.303857  8182 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:32.310192  8132 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:32.311678  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:32.328418  6507 raft_consensus.cc:2393] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 5: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:32.312464  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:32.231153  8180 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:32.233160  8190 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:32.200452  8093 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:32.223994  8142 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:32.322860  6548 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:32.335516  6403 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57888 (request call id 213) took 5866 ms. Trace:
I20260709 12:17:32.335908  6548 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:32.235611  8092 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:32.265924  8038 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:32.323045  6501 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:32.222468  8138 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:32.350373  8138 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:32.318509  7737 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:32.351004  7737 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:32.352542  8193 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:32.353003  8193 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:32.352821  8024 consensus_queue.cc:237] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:32.353819  8093 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:32.304049  8036 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:32.326910  6473 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:32.355521  6473 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:32.356531  7273 raft_consensus.cc:1240] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:32.304617  7900 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:32.357160  6505 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:32.357807  6473 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:32.358407  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:32.358927  6473 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:32.359396  8016 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:32.331928  6504 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:32.332511  7962 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:32.360735  7918 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:32.361234  7918 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:32.361794  7918 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:32.304721  7240 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:32.362810  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:32.363090  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 58469.6us
I20260709 12:17:32.363919  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:32.364624  6504 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:32.364545  6473 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:32.365613  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:32.366122  6474 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:32.366526  6501 raft_consensus.cc:2410] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:32.353410  8193 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:32.353943  7266 raft_consensus.cc:2393] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 3: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:17:32.355964  8142 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:32.360097  8016 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:32.337772  6548 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:32.447184  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 109471us
I20260709 12:17:32.334684  8180 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:32.316370  8143 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:32.314719  8166 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:32.326234  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 92597.9us
I20260709 12:17:32.367285  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:32.368685  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:32.335979  6403 rpcz_store.cc:276] 0709 12:17:26.514250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:26.514905 (+   655us) service_pool.cc:224] Handling call
0709 12:17:32.335430 (+5820525us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:28.397606 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:28.398191 (+   585us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:28.398222 (+    31us) write_op.cc:435] Acquired schema lock
0709 12:17:28.398238 (+    16us) tablet.cc:662] Decoding operations
0709 12:17:28.436485 (+ 38247us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:28.445121 (+  8636us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:28.455199 (+ 10078us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:17:28.580316 (+125117us) tablet.cc:701] Row locks acquired
0709 12:17:28.583725 (+  3409us) write_op.cc:260] PREPARE: finished
0709 12:17:28.609911 (+ 26186us) write_op.cc:270] Start()
0709 12:17:28.627083 (+ 17172us) write_op.cc:276] Timestamp: P: 1783599448601058 usec, L: 0
0709 12:17:28.628490 (+  1407us) op_driver.cc:348] REPLICATION: starting
0709 12:17:28.823265 (+194775us) log.cc:844] Serialized 1340 byte log entry
0709 12:17:30.205743 (+1382478us) op_driver.cc:464] REPLICATION: finished
0709 12:17:30.451092 (+245349us) write_op.cc:301] APPLY: starting
0709 12:17:30.459610 (+  8518us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:30.499574 (+ 39964us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:30.552409 (+ 52835us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:30.552929 (+   520us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:30.553117 (+   188us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 12:17:30.553163 (+    46us) write_op.cc:312] APPLY: finished
0709 12:17:30.556655 (+  3492us) log.cc:844] Serialized 35 byte log entry
0709 12:17:30.556873 (+   218us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:30.556985 (+   112us) write_op.cc:454] Released schema lock
0709 12:17:30.557037 (+    52us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:31.164543 (+607506us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b540008acd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e84e28 00007f28798bf581 00007f28798bed58 00007f28798c39f1 00007f28798c39a9 00007f28798c3939 00007f28798c3901 00007f28798c2b2d 0000000000388634 0000000000388479
0709 12:17:31.164811 (+   268us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:31.164853 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":11,"child_traces":[["op",{"apply.queue_time_us":245064,"apply.run_cpu_time_us":2807,"apply.run_wall_time_us":106334,"num_ops":3,"prepare.queue_time_us":119654,"prepare.run_cpu_time_us":70334,"prepare.run_wall_time_us":2767130,"raft.queue_time_us":537494,"raft.run_cpu_time_us":13651,"raft.run_wall_time_us":216255,"replication_time_us":1576972,"spinlock_wait_cycles":30489472,"thread_start_us":1068070,"threads_started":4,"wal-append.queue_time_us":812099,"wal-append.run_cpu_time_us":27504,"wal-append.run_wall_time_us":1909304}]]}
I20260709 12:17:32.369390  6473 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:32.307339  8178 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:32.519433  6550 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:17:32.520040  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 150724us
I20260709 12:17:32.521186  6473 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:32.519281  6653 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:32.367954  8036 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:32.449909  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 68807.3us
I20260709 12:17:32.530408  6474 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:32.346086  8162 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:32.523010  6473 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:32.327665  8163 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:32.528369  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 79023.2us
I20260709 12:17:32.448498  7785 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:32.361030  7962 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:32.449623  6658 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 3.
W20260709 12:17:32.531494  6579 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54204 (request call id 81) took 1570 ms (client timeout 1731 ms). Trace:
I20260709 12:17:32.446405  8117 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:32.373183  7999 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:32.520987  7793 raft_consensus.cc:2764] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
W20260709 12:17:32.545475  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 317931us
I20260709 12:17:32.452452  8180 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:32.442916  8134 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:17:32.542681  6656 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58980 (request call id 1530) took 1587 ms (client timeout 1534 ms). Trace:
I20260709 12:17:32.510892  8045 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:17:32.557964  6579 rpcz_store.cc:269] 0709 12:17:30.961603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:30.961752 (+   149us) service_pool.cc:224] Handling call
0709 12:17:32.531303 (+1569551us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b540034dfd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:32.531436 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2728265600}
W20260709 12:17:32.570880  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 51526.8us
I20260709 12:17:32.573067  6653 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 3.
I20260709 12:17:32.362615  8069 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:32.564558  6656 rpcz_store.cc:269] 0709 12:17:30.958142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:30.975469 (+ 17327us) service_pool.cc:224] Handling call
0709 12:17:32.542483 (+1567014us) spinlock_profiling.cc:243] Waited 982 ms on lock 0x7b54003e8ad4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:32.542618 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2258870144}
I20260709 12:17:32.522739  8143 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:32.526875  7268 raft_consensus.cc:1275] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:32.564788  7238 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:32.574530  7240 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:32.527354  8063 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:32.578303  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:32.547089  8176 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:32.597033  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 255989us
I20260709 12:17:32.597752  7238 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:32.643343  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 45717.7us
I20260709 12:17:32.559754  7962 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:32.646731  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 74917us
I20260709 12:17:32.547866  8142 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:32.450589  8181 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:32.559279  7999 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:32.595422  8143 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:32.546198  7879 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.064s
W20260709 12:17:32.662317  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:32.574862  8134 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:32.663264  6653 raft_consensus.cc:1275] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:17:32.663749  6658 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:32.664763  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:32.663267  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:32.602483  7793 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:32.598024  8043 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.596237  8008 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:17:32.666594  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 143623us
I20260709 12:17:32.651623  8138 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:17:32.673547  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 136930us
I20260709 12:17:32.646446  8093 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:32.696374  8093 raft_consensus.cc:697] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:32.674978  6656 raft_consensus.cc:1275] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:32.700698  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:32.701828  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:32.705036  8200 log.cc:927] Time spent T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
I20260709 12:17:32.705662  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:32.706306  6584 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:32.707319  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:32.707850  6584 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:32.711522  6650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 998) took 1096 ms. Trace:
I20260709 12:17:32.712004  6650 rpcz_store.cc:276] 0709 12:17:31.614712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:31.615500 (+   788us) service_pool.cc:224] Handling call
0709 12:17:32.711382 (+1095882us) spinlock_profiling.cc:243] Waited 61.7 ms on lock 0x7b54001b1054. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:32.711468 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":141931392}
I20260709 12:17:32.713506  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:32.714125  6581 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:32.672665  8142 raft_consensus.cc:697] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:32.696103  8138 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:32.715389  8142 consensus_queue.cc:237] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:32.715852  8138 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:32.718459  8043 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:32.719401  7900 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:32.721278  8043 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:32.721570  7274 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38562 (request call id 351) took 1073 ms. Trace:
I20260709 12:17:32.721817  8043 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:32.721931  7274 rpcz_store.cc:276] 0709 12:17:31.647988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:31.649245 (+  1257us) service_pool.cc:224] Handling call
0709 12:17:31.649599 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:32.111600 (+462001us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.721366 (+609766us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b54000a50d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:32.721505 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1060730240}
I20260709 12:17:32.668933  7999 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:32.594198  8045 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:32.646027  8162 consensus_queue.cc:1048] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 12:17:32.728639  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:32.574132  8117 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:32.673429  8134 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:32.659986  8069 consensus_queue.cc:1059] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 12:17:32.687350  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:32.699538  6475 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:32.748745  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:32.751643  8199 log.cc:927] Time spent T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.205s	user 0.000s	sys 0.000s
I20260709 12:17:32.665202  7738 consensus_queue.cc:1059] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.074s
I20260709 12:17:32.752774  8093 consensus_queue.cc:237] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:32.754164  8138 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:32.699365  7793 raft_consensus.cc:697] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 4 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:32.690690  8008 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:17:32.756923  8197 log.cc:927] Time spent T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.230s	user 0.000s	sys 0.000s
I20260709 12:17:32.759379  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:32.797189  7918 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:32.765400  6473 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:32.780733  7879 consensus_queue.cc:1059] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 12:17:32.789134  8182 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:32.815114  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 49799.3us
I20260709 12:17:32.815483  7971 raft_consensus.cc:3055] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:32.815896  7971 raft_consensus.cc:740] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:32.816144  6473 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:32.816725  8182 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:32.791088  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:32.819118  6584 raft_consensus.cc:2368] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 2. Current term is 3.
I20260709 12:17:32.780908  7981 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.801750  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:32.810628  8180 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:32.777710  8016 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:32.813963  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 267460us
W20260709 12:17:32.760463  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 61008.6us
I20260709 12:17:32.793896  8008 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:32.830435  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:32.831456  6651 raft_consensus.cc:1240] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:32.835543  6656 raft_consensus.cc:2393] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 3: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:17:32.822203  8041 consensus_meta.cc:287] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.923s	user 0.012s	sys 0.000s
I20260709 12:17:32.789604  7995 consensus_queue.cc:260] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:17:32.842432  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:32.814314  8138 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:32.849769  6501 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 12:17:32.823361  8177 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:32.823901  8207 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:32.802821  7737 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:32.816798  7971 consensus_queue.cc:260] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:32.825995  8069 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.848604  8178 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:32.849442  8117 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:32.859660  7995 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:32.855713  8177 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:32.826220  7981 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:32.860843  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:32.860018  8207 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:32.869580  8069 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:32.891000  7879 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:32.900902  7879 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:32.893708  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:32.908859  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:32.909456  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:32.918497  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 1532) took 1908 ms. Trace:
I20260709 12:17:32.888837  8177 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:32.880457  8185 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:32.863771  7793 consensus_queue.cc:237] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:32.899770  8207 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:32.906877  7900 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:32.905529  6575 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:32.943954  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 126408us
I20260709 12:17:32.947741  6473 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:32.882869  8210 consensus_queue.cc:1048] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.059s
W20260709 12:17:32.951094  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 123678us
I20260709 12:17:32.953222  8209 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:32.966235  7737 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:32.958576  8119 raft_consensus.cc:3055] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:32.920893  6649 rpcz_store.cc:276] 0709 12:17:31.010311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:31.094598 (+ 84287us) service_pool.cc:224] Handling call
0709 12:17:31.156423 (+ 61825us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:31.837532 (+681109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.738874 (+901342us) spinlock_profiling.cc:243] Waited 614 ms on lock 0x7b54002a4ed4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:32.918218 (+179344us) spinlock_profiling.cc:243] Waited 61.4 ms on lock 0x7b54002a4ed0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:32.918431 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1552644736}
I20260709 12:17:32.979343  6655 raft_consensus.cc:1275] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:32.929312  8185 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:32.964434  7918 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:32.990473  8122 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:32.991990  8063 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:32.965123  8209 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:17:32.999672  8218 log.cc:927] Time spent T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 12:17:33.000998  7981 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:33.012828  7962 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:33.012867  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:33.017025  7272 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38562 (request call id 356) took 1358 ms. Trace:
I20260709 12:17:33.019505  8209 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:33.029808  7898 consensus_meta.cc:287] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.701s	user 0.005s	sys 0.005s
I20260709 12:17:33.036175  8143 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:32.977943  8180 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:33.047200  7272 rpcz_store.cc:276] 0709 12:17:31.658823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:31.659795 (+   972us) service_pool.cc:224] Handling call
0709 12:17:31.660190 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:32.216894 (+556704us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:33.016714 (+799820us) spinlock_profiling.cc:243] Waited 556 ms on lock 0x7b5400502654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:33.016948 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1278950272}
W20260709 12:17:33.060765  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 232508us
I20260709 12:17:32.976198  8119 raft_consensus.cc:740] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:33.059680  6509 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
I20260709 12:17:33.061905  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:17:33.044914  8051 consensus_meta.cc:287] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.721s	user 0.011s	sys 0.002s
I20260709 12:17:33.062731  6580 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:33.063580  6583 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:33.075806  8169 consensus_queue.cc:1059] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:17:33.080744  6475 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:33.079883  8069 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:33.083173  6475 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:33.083999  8227 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:33.084558  8063 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:33.084667  7962 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:33.085037  8063 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:33.085505  8063 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:33.086612  8064 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:33.090715  6475 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:33.055274  8008 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:33.092396  7900 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:33.094128  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:33.096289  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:33.096397  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:17:33.096954  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:33.097311  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:33.076927  8210 raft_consensus.cc:3055] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:33.101306  6656 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:33.101002  6653 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:33.094313  7962 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:33.107378  8178 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:33.107453  7962 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:33.107942  8178 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:33.102979  8210 raft_consensus.cc:740] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:33.109462  7266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38546 (request call id 213) took 1052 ms. Trace:
I20260709 12:17:33.109277  8210 consensus_queue.cc:260] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:33.110138  8178 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:33.110471  8210 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:33.115365  7918 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:33.121088  7918 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:33.121488  8143 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:33.123272  7918 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:33.123764  7900 raft_consensus.cc:697] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 4 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:17:33.124106  6549 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:56116 dropped due to backpressure: service queue is full with 50 items
I20260709 12:17:33.124750  7900 consensus_queue.cc:237] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:33.127347  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:33.127482  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:33.128545  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:33.128818  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:33.130473  6508 raft_consensus.cc:1240] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:33.130817  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:33.131785  6507 raft_consensus.cc:2393] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 3: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:33.132861  6501 raft_consensus.cc:1240] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:33.134151  6581 raft_consensus.cc:1240] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:33.139302  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:33.141090  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:33.141582  6507 raft_consensus.cc:2393] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 4: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:33.141718  6582 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:33.141772  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:33.135178  8175 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:33.109850  7266 rpcz_store.cc:276] 0709 12:17:32.057346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.059161 (+  1815us) service_pool.cc:224] Handling call
0709 12:17:33.109197 (+1050036us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b54005de654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:33.109395 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":583926016}
I20260709 12:17:33.134956  8179 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:33.154330  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:33.159102  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 1481) took 2021 ms. Trace:
I20260709 12:17:33.159498  6503 rpcz_store.cc:276] 0709 12:17:31.149738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:31.151536 (+  1798us) service_pool.cc:224] Handling call
0709 12:17:32.160682 (+1009146us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:32.161017 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:32.161042 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:32.161470 (+   428us) log.cc:844] Serialized 26 byte log entry
0709 12:17:32.199521 (+ 38051us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:32.199545 (+    24us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:32.199589 (+    44us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:32.706502 (+506913us) raft_consensus.cc:1685] finished
0709 12:17:32.706518 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:33.156959 (+450441us) spinlock_profiling.cc:243] Waited 1e+03 ms on lock 0x7b54002c56d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:33.159017 (+  2058us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":253,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":100,"spinlock_wait_cycles":2299630848,"thread_start_us":37802,"threads_started":1,"wal-append.queue_time_us":166282}
I20260709 12:17:33.165115  8185 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:33.167049  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1005) took 1546 ms. Trace:
I20260709 12:17:33.167420  6652 rpcz_store.cc:276] 0709 12:17:31.620788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:31.980121 (+359333us) service_pool.cc:224] Handling call
0709 12:17:31.980548 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:31.993758 (+ 13210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:31.993789 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:31.993832 (+    43us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:31.993861 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:33.166835 (+1172974us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b54003ea3d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:33.166876 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:33.166994 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29873920}
I20260709 12:17:33.169878  8008 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:33.171337  6575 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 3.
I20260709 12:17:33.171466  6656 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:33.173170  7879 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:33.173656  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:33.174705  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:33.124497  6549 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:56116 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1374)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 1068)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1360)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1386)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1356)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1361)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1365)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1362)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1368)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 1069)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1367)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 1061)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 1065)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1382)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1363)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1366)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1347)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54204 (request call id 97)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1371)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1370)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1354)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1385)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1349)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1383)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1369)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1353)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1372)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1380)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1355)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:53972 (request call id 9)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1062)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 98)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1348)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1350)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1351)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1357)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1358)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1359)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1364)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1066)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 100)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1373)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1375)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1376)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1377)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1378)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1379)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1381)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1384)
I20260709 12:17:33.175194  6501 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:33.177944  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:33.178977  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:33.179842  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:33.180529  6582 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:33.180768  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:17:32.360934  8194 consensus_queue.cc:1175] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f2b06aebe97249a4877d798b8118ece4" responder_term: 4 status { last_received { term: 1 index: 1 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4. Ops: [3.2-3.2]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:17:33.181595  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:33.183118  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:33.183943  6507 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:33.187105  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:33.187942  6577 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:33.179905  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:33.191344  6580 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:33.191341  6501 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 3.
I20260709 12:17:33.197049  6548 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:33.199062  6548 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:17:33.199805  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:33.201200  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:33.203410  7238 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:33.203755  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:17:33.155606  8193 consensus_meta.cc:287] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.783s	user 0.013s	sys 0.001s
I20260709 12:17:33.189577  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:33.202181  7273 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:33.207113  6507 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 3.
I20260709 12:17:33.212119  6548 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:33.206915  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:33.213563  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:33.212205  6583 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:33.214102  6580 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:17:33.214841  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:33.215055  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:33.215651  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:33.208148  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:33.216056  6583 raft_consensus.cc:2393] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 3: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:33.222024  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:33.222651  6503 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:33.223104  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:33.223444  6577 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:33.223663  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:33.206630  7995 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:17:33.229674  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:33.225507  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:33.243428  6549 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:17:33.246752  6656 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:33.224709  6475 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:33.231645  6548 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:33.248026  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:33.248582  6580 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:33.151470  8175 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:33.211584  8138 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:33.253062  6582 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:33.206955  8112 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:33.188371  8179 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:33.240799  8162 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:33.261034  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:17:33.247275  8159 consensus_meta.cc:287] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.978s	user 0.000s	sys 0.013s
I20260709 12:17:33.267272  6582 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:33.254427  8161 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:33.106597  8119 consensus_queue.cc:260] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:33.221238  8052 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:33.246767  7986 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:33.279186  7986 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:33.283176  8134 consensus_meta.cc:287] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.515s	user 0.006s	sys 0.005s
I20260709 12:17:33.283742  8134 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:33.282969  6582 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
W20260709 12:17:33.286777  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:33.287412  7999 consensus_meta.cc:287] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.564s	user 0.007s	sys 0.005s
I20260709 12:17:33.287604  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:33.287729  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:33.287815  7999 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:33.288725  6582 raft_consensus.cc:2368] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3.
I20260709 12:17:33.252048  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:33.254343  7898 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:33.273907  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:33.254915  8051 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:33.255043  6475 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:33.272398  6509 raft_consensus.cc:1275] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:33.209347  8207 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:33.459705  8207 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:33.296030  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:33.289232  8092 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:33.289585  6508 raft_consensus.cc:2393] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 5: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:33.299443  6583 raft_consensus.cc:2393] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 3: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:33.223753  8193 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:33.498574  6502 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 1492) took 1301 ms. Trace:
W20260709 12:17:33.455471  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 200475us
I20260709 12:17:33.849763  6502 rpcz_store.cc:276] 0709 12:17:32.197279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.198205 (+   926us) service_pool.cc:224] Handling call
0709 12:17:32.198542 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:32.232159 (+ 33617us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:32.235989 (+  3830us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:32.285680 (+ 49691us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:32.302404 (+ 16724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:33.498278 (+1195874us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b540039fb54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:33.498359 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:33.498513 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27476352}
I20260709 12:17:33.624930  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:33.855275  6577 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 4.
I20260709 12:17:33.857075  6576 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:33.646555  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:33.385885  6622 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
W20260709 12:17:33.557541  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:33.864513  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:33.695087  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:33.868641  6575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1360) took 1036 ms. Trace:
I20260709 12:17:33.869240  6575 rpcz_store.cc:276] 0709 12:17:32.832478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.694829 (+862351us) service_pool.cc:224] Handling call
0709 12:17:33.868377 (+173548us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:33.297495  6578 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:33.297598  6580 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:33.353683  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:33.300726  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:33.896301  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1347) took 1067 ms. Trace:
I20260709 12:17:33.896822  6581 rpcz_store.cc:276] 0709 12:17:32.829094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.353450 (+524356us) service_pool.cc:224] Handling call
0709 12:17:33.896105 (+542655us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:33.897234  6582 raft_consensus.cc:2393] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 4: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:33.354311  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:33.461248  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:33.497949  6652 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:33.905191  8117 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:33.906520  8117 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:33.907506  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:33.351440  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:33.909914  6653 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:33.910323  6576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1052) took 1196 ms. Trace:
I20260709 12:17:33.351859  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:33.350878  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:33.341964  6655 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:33.345888  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:33.352593  6657 raft_consensus.cc:1240] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:33.380089  7268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38546 (request call id 218) took 1067 ms. Trace:
I20260709 12:17:33.451817  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:33.529059  7269 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:33.299448  7898 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:33.941708  7898 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:33.302158  7962 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:33.491112  7981 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:33.952592  7981 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:33.522753  7986 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:33.253655  7995 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:33.307430  7999 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:33.433888  7971 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:33.985011  7981 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:33.427268  8029 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:33.987375  8029 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:33.721374  8036 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:33.988900  8036 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:33.315407  8045 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:33.266000  8063 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:33.434392  8067 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:33.560946  8092 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:34.004475  8092 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:33.353426  8101 consensus_queue.cc:1059] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:34.011646  8101 raft_consensus.cc:3055] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:33.385962  8106 consensus_queue.cc:1048] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:17:34.013621  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:33.997335  8036 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:33.634748  8119 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:33.267141  8138 raft_consensus.cc:697] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:33.322295  8134 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:33.274809  8159 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:34.026453  6579 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:33.272964  8161 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:34.030915  8161 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:33.314996  8175 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:33.664963  8176 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:34.034412  8029 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:34.034132  8176 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:34.012132  8101 raft_consensus.cc:740] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:33.327916  8177 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:34.038964  8177 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:33.456007  8178 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:34.041458  8178 raft_consensus.cc:697] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:34.042681  8178 consensus_queue.cc:237] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:33.327221  8179 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:33.245745  8185 consensus_queue.cc:1059] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 12:17:33.387080  8181 raft_consensus.cc:3055] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:17:33.295163  8190 consensus_meta.cc:287] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.931s	user 0.001s	sys 0.012s
I20260709 12:17:34.051824  8190 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:34.052544  8138 consensus_queue.cc:237] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:34.053745  6505 raft_consensus.cc:1240] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:17:33.347499  8192 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:34.060442  8192 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:34.061023  8176 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:34.063535  8192 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:17:33.526285  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 304780us
I20260709 12:17:34.065310  6473 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:34.067559  6473 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:33.311388  8209 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:33.345085  8227 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:33.813766  7793 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:33.816825  8165 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:34.074128  6504 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:34.074195  8067 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:34.074718  8067 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:34.075176  8067 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:34.074384  8165 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:33.819981  7271 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:33.821969  6506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 100) took 1629 ms. Trace:
I20260709 12:17:34.077383  6506 rpcz_store.cc:276] 0709 12:17:32.192022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.192177 (+   155us) service_pool.cc:224] Handling call
0709 12:17:32.192588 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:32.552901 (+360313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:32.552948 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:32.553009 (+    61us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:32.553052 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:33.821246 (+1268194us) spinlock_profiling.cc:243] Waited 360 ms on lock 0x7b54001e82d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:33.821360 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:33.821706 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":828051328}
I20260709 12:17:33.820856  6402 catalog_manager.cc:5697] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 1 to 3, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 3 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:34.078256  8177 consensus_queue.cc:1059] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:34.079636  8179 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:33.841310  6406 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46184 (request call id 112) took 5806 ms. Trace:
I20260709 12:17:34.080240  8179 raft_consensus.cc:697] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:34.080529  6406 rpcz_store.cc:276] 0709 12:17:28.034218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.043497 (+  9279us) service_pool.cc:224] Handling call
0709 12:17:33.840940 (+5797443us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28898,"reported_tablets":8}
I20260709 12:17:34.081442  8179 consensus_queue.cc:237] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:33.845892  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:33.856143  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54204 (request call id 97) took 1098 ms. Trace:
I20260709 12:17:34.083069  6577 rpcz_store.cc:276] 0709 12:17:32.758095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.624604 (+866509us) service_pool.cc:224] Handling call
0709 12:17:33.856039 (+231435us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.083438  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1370) took 1103 ms. Trace:
I20260709 12:17:34.084023  6583 rpcz_store.cc:276] 0709 12:17:32.980108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.844183 (+864075us) service_pool.cc:224] Handling call
0709 12:17:34.083347 (+239164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.085455  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:34.086323  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1369) took 1120 ms. Trace:
I20260709 12:17:33.876191  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:34.088754  6649 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:34.087194  8209 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:33.886534  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:34.090307  6658 raft_consensus.cc:2393] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 4: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:33.857636  6475 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
W20260709 12:17:34.091924  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 238764us
I20260709 12:17:33.900166  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 1061) took 1168 ms. Trace:
I20260709 12:17:34.092572  6580 rpcz_store.cc:276] 0709 12:17:32.731540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.247791 (+516251us) service_pool.cc:224] Handling call
0709 12:17:33.899911 (+652120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.093731  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:34.094655  6580 raft_consensus.cc:2410] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:33.911147  8189 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:34.094863  8176 raft_consensus.cc:3055] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:34.080029  6402 catalog_manager.cc:5697] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:34.095528  8176 raft_consensus.cc:740] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:34.095482  8189 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:17:33.910861  6576 rpcz_store.cc:276] 0709 12:17:32.698485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.708861 (+ 10376us) service_pool.cc:224] Handling call
0709 12:17:32.709131 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:33.858558 (+1149427us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:33.874968 (+ 16410us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b540006e5d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:33.895050 (+ 20082us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2638663936}
I20260709 12:17:34.094936  8175 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:17:34.098277  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:34.099040  6576 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 5: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:34.099444  6576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1372) took 1068 ms. Trace:
I20260709 12:17:33.913033  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:34.092058  8177 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:34.101734  8177 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:33.915706  8093 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:33.913312  8117 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:33.920725  8166 raft_consensus.cc:2764] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:17:33.923707  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:33.915512  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 529519us
I20260709 12:17:33.934492  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:33.937244  6550 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:34.106062  6582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1349) took 1276 ms. Trace:
I20260709 12:17:34.106185  6581 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:34.106527  6582 rpcz_store.cc:276] 0709 12:17:32.829570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.923194 (+1093624us) service_pool.cc:224] Handling call
0709 12:17:34.105976 (+182782us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.106769  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1385) took 1045 ms. Trace:
I20260709 12:17:34.107234  6581 rpcz_store.cc:276] 0709 12:17:33.061123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.934028 (+872905us) service_pool.cc:224] Handling call
0709 12:17:34.106715 (+172687us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.107738  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:33.943727  8191 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:33.945097  7268 rpcz_store.cc:276] 0709 12:17:32.327310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.344995 (+ 17685us) service_pool.cc:224] Handling call
0709 12:17:32.349616 (+  4621us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:32.597456 (+247840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:33.379723 (+782267us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b54005db1d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:33.380006 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31625088}
I20260709 12:17:33.964655  6653 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:33.967092  6578 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:33.983001  7995 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:34.022557  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:34.095036  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1353) took 1264 ms. Trace:
I20260709 12:17:34.116232  6580 rpcz_store.cc:276] 0709 12:17:32.830466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.093344 (+1262878us) service_pool.cc:224] Handling call
0709 12:17:34.094990 (+  1646us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.099856  6576 rpcz_store.cc:276] 0709 12:17:33.030849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.097996 (+1067147us) service_pool.cc:224] Handling call
0709 12:17:34.099381 (+  1385us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.029505  8159 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:34.030052  6579 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 1059) took 1329 ms (client timeout 1764 ms). Trace:
I20260709 12:17:34.032702  8063 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:34.035985  8101 consensus_queue.cc:260] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:34.039482  8036 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:34.044253  6584 raft_consensus.cc:2410] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:34.046115  8180 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:34.049026  8181 raft_consensus.cc:740] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:34.056298  8190 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:34.063673  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:34.084175  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:33.861167  8193 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:33.883540  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:34.102089  8177 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:34.108235  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:34.111637  6578 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "20a202c76244485fbc8f1610473db8b9"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:17:34.154363  6578 raft_consensus.cc:606] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Received request to transfer leadership to TS 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:34.154505  6581 raft_consensus.cc:2410] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:34.086715  6583 rpcz_store.cc:276] 0709 12:17:32.966461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.085070 (+1118609us) service_pool.cc:224] Handling call
0709 12:17:34.086238 (+  1168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.154752  6578 raft_consensus.cc:614] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:17:34.131903  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 194942us
W20260709 12:17:34.133359  6579 rpcz_store.cc:269] 0709 12:17:32.700200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.701606 (+  1406us) service_pool.cc:224] Handling call
0709 12:17:34.029757 (+1328151us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b540034bcd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:34.029956 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3043865856}
I20260709 12:17:34.135495  6657 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
I20260709 12:17:34.149665  6582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1380) took 1101 ms. Trace:
I20260709 12:17:34.156371  6582 rpcz_store.cc:276] 0709 12:17:33.033453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.107402 (+1073949us) service_pool.cc:224] Handling call
0709 12:17:34.134951 (+ 27549us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:17:34.132431  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 65045.9us
I20260709 12:17:34.158140  6577 raft_consensus.cc:2410] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:34.164242  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1355) took 1328 ms. Trace:
I20260709 12:17:34.164752  6581 rpcz_store.cc:276] 0709 12:17:32.830945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.107903 (+1276958us) service_pool.cc:224] Handling call
0709 12:17:34.159339 (+ 51436us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.166642  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1383) took 1108 ms. Trace:
I20260709 12:17:34.163590  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:53972 (request call id 9) took 1054 ms. Trace:
I20260709 12:17:34.167112  6577 rpcz_store.cc:276] 0709 12:17:33.053660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.083880 (+1030220us) service_pool.cc:224] Handling call
0709 12:17:34.161111 (+ 77231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.169420  8036 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:34.169526  8176 consensus_queue.cc:260] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:34.170068  6402 catalog_manager.cc:5697] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:34.133271  7793 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:34.150799  8192 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:34.172700  6622 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:34.173509  7269 raft_consensus.cc:2393] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 2: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:17:34.173875  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1062) took 1442 ms. Trace:
I20260709 12:17:34.174275  7793 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:34.174688  8166 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:34.174930  6622 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:34.177109  8041 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:34.155422  8180 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:34.178258  8029 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:34.179083  8029 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:34.179961  7238 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:34.181120  6579 raft_consensus.cc:1240] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:34.181816  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:34.182698  8093 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:34.183104  6500 raft_consensus.cc:2410] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:34.177757  8041 raft_consensus.cc:697] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:34.181872  6579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1350) took 1352 ms. Trace:
I20260709 12:17:34.182096  8029 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:34.174310  6580 rpcz_store.cc:276] 0709 12:17:32.731953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.164033 (+1432080us) service_pool.cc:224] Handling call
0709 12:17:34.173233 (+  9200us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.173478 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:34.173507 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:34.173571 (+    64us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:34.173638 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.173706 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:34.173806 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.176267  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:34.168361  6578 rpcz_store.cc:276] 0709 12:17:33.102086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.111389 (+1009303us) service_pool.cc:224] Handling call
0709 12:17:34.155192 (+ 43803us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.184813  6579 rpcz_store.cc:276] 0709 12:17:32.829800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.160687 (+1330887us) service_pool.cc:224] Handling call
0709 12:17:34.180821 (+ 20134us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.181550 (+   729us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.181799 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.183880  8172 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:17:34.233831  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 57742.5us
I20260709 12:17:34.235534  6579 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:34.235998  7971 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:34.236260  6579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1358) took 1404 ms. Trace:
I20260709 12:17:34.236606  7971 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:34.236657  6579 rpcz_store.cc:276] 0709 12:17:32.831971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.221457 (+1389486us) service_pool.cc:224] Handling call
0709 12:17:34.229722 (+  8265us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:34.235988 (+  6266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.236210 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.238303  6579 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:34.238854  6579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1364) took 1352 ms. Trace:
I20260709 12:17:34.238958  7971 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:34.239176  6579 rpcz_store.cc:276] 0709 12:17:32.886923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.237724 (+1350801us) service_pool.cc:224] Handling call
0709 12:17:34.238082 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.238652 (+   570us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.238786 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.239631  8181 consensus_queue.cc:260] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:34.184230  6402 catalog_manager.cc:5697] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:34.245644  6402 catalog_manager.cc:5697] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:34.246431  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 64733.1us
I20260709 12:17:34.247525  6402 catalog_manager.cc:5697] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 1 to 3, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 3 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:17:34.248920  7267 raft_consensus.cc:1240] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:34.239277  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:34.249825  7793 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:34.252571  7273 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:34.253038  7793 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:34.227239  8036 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:34.255393  8101 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:34.255908  7793 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:34.265143  7971 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:34.266852  7971 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:34.268720  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 54423us
I20260709 12:17:34.272289  8069 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:34.272784  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:34.272986  8029 consensus_queue.cc:1048] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:34.273003  8069 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:34.274417  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:34.275570  8069 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:34.275942  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1351) took 1445 ms. Trace:
I20260709 12:17:34.276168  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:34.276539  8117 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:34.276432  6581 rpcz_store.cc:276] 0709 12:17:32.830007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.213831 (+1383824us) service_pool.cc:224] Handling call
0709 12:17:34.275239 (+ 61408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.275592 (+   353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:34.275617 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:34.275680 (+    63us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:34.275713 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.275769 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:34.275870 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.277288  8117 raft_consensus.cc:697] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 3 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:34.277366  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:34.278426  6581 raft_consensus.cc:1275] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:34.278966  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 100) took 1268 ms. Trace:
I20260709 12:17:34.278587  8117 consensus_queue.cc:237] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:34.275753  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:34.279359  6581 rpcz_store.cc:276] 0709 12:17:33.010167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.277725 (+1267558us) service_pool.cc:224] Handling call
0709 12:17:34.278058 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.278756 (+   698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.278918 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.276821  6579 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:34.239521  8176 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:34.257273  6580 raft_consensus.cc:1240] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:34.276919  6656 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:34.212025  7898 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:34.273269  8181 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:34.295243  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1357) took 1459 ms. Trace:
I20260709 12:17:34.298290  6580 rpcz_store.cc:276] 0709 12:17:32.831732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.220576 (+1388844us) service_pool.cc:224] Handling call
0709 12:17:34.227105 (+  6529us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.290892 (+ 63787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.291178 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.294342  6550 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:34.239864  8165 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:34.286593  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:34.342137  6548 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:17:34.343528  6500 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 1503) took 1982 ms (client timeout 1580 ms). Trace:
W20260709 12:17:34.343938  6500 rpcz_store.cc:269] 0709 12:17:32.361601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:32.362562 (+   961us) service_pool.cc:224] Handling call
0709 12:17:34.343250 (+1980688us) spinlock_profiling.cc:243] Waited 1.82 s on lock 0x7b540017bbd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:34.343436 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4183706368}
I20260709 12:17:34.345417  6473 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:34.356686  6622 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:34.369944  7267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38546 (request call id 224) took 1011 ms. Trace:
I20260709 12:17:34.362253  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1371) took 1332 ms. Trace:
W20260709 12:17:34.388244  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 95265.8us
I20260709 12:17:34.454843  6584 rpcz_store.cc:276] 0709 12:17:33.030415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.646201 (+615786us) service_pool.cc:224] Handling call
0709 12:17:34.347224 (+701023us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b54001d15d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:34.362184 (+ 14960us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":425164160}
I20260709 12:17:34.456557  6584 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:17:34.456650  8029 consensus_queue.cc:1059] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:34.457208  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1378) took 1424 ms. Trace:
I20260709 12:17:34.457561  6584 rpcz_store.cc:276] 0709 12:17:33.032888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.455778 (+1422890us) service_pool.cc:224] Handling call
0709 12:17:34.456184 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.456953 (+   769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.457158 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.458019  8029 raft_consensus.cc:3055] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:34.458418  8029 raft_consensus.cc:740] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:34.326004  7971 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:34.459385  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1379) took 1426 ms. Trace:
I20260709 12:17:34.459744  6584 rpcz_store.cc:276] 0709 12:17:33.033169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.458372 (+1425203us) service_pool.cc:224] Handling call
0709 12:17:34.458781 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.459075 (+   294us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:34.459099 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:34.459143 (+    44us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:34.459176 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.459233 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:34.459336 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.460933  8117 consensus_queue.cc:1059] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:17:34.446802  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 106187us
I20260709 12:17:34.462514  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1381) took 1428 ms. Trace:
I20260709 12:17:34.462996  7238 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:34.462903  6584 rpcz_store.cc:276] 0709 12:17:33.033743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.461530 (+1427787us) service_pool.cc:224] Handling call
0709 12:17:34.461917 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.462197 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:34.462224 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:34.462267 (+    43us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:34.462301 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.462349 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:34.462442 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.465700  7238 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:34.466269  6500 raft_consensus.cc:1275] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:17:34.467614  8106 raft_consensus.cc:3055] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:34.468024  8106 raft_consensus.cc:740] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:34.468503  6500 raft_consensus.cc:1275] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:17:34.469180  8138 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:34.469718  8138 raft_consensus.cc:697] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:34.470583  8190 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:34.471128  8190 raft_consensus.cc:697] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:34.470814  8138 consensus_queue.cc:237] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:34.472143  8190 consensus_queue.cc:237] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:34.365600  8041 consensus_queue.cc:237] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:34.473173  8172 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:34.447788  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 206516us
I20260709 12:17:34.449812  6575 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:17:34.450621  8165 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:34.475970  6575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1354) took 1645 ms. Trace:
I20260709 12:17:34.476116  6550 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:34.476454  6575 rpcz_store.cc:276] 0709 12:17:32.830728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.912571 (+1081843us) service_pool.cc:224] Handling call
0709 12:17:34.475903 (+563332us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.478160  6575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 101) took 1273 ms. Trace:
I20260709 12:17:34.478534  6575 rpcz_store.cc:276] 0709 12:17:33.205236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.477215 (+1271979us) service_pool.cc:224] Handling call
0709 12:17:34.477573 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.477792 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:34.477821 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:34.477905 (+    84us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:34.477938 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.477999 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:34.478105 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.453882  8177 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:34.460634  7267 rpcz_store.cc:276] 0709 12:17:33.358958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:33.359478 (+   520us) service_pool.cc:224] Handling call
0709 12:17:33.359818 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.295738 (+935920us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.369565 (+ 73827us) spinlock_profiling.cc:243] Waited 889 ms on lock 0x7b54005ddc54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:34.369852 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2043550592}
I20260709 12:17:34.472498  8036 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:34.475963  8165 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:34.480538  8165 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:34.481348  8180 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:34.486301  6584 raft_consensus.cc:1275] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:17:34.486109  8177 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:34.459403  8029 consensus_queue.cc:260] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:34.447001  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 189006us
I20260709 12:17:34.434361  6579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1066) took 1484 ms. Trace:
I20260709 12:17:34.452086  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:17:34.488485  6584 pending_rounds.cc:85] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 1
I20260709 12:17:34.488984  6582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1348) took 1598 ms. Trace:
I20260709 12:17:34.490343  7240 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:34.493753  8067 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:34.495050  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1373) took 1463 ms. Trace:
I20260709 12:17:34.495946  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1359) took 1663 ms. Trace:
I20260709 12:17:34.497957  8119 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:34.502403  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1375) took 1470 ms. Trace:
I20260709 12:17:34.498355  6502 raft_consensus.cc:1240] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 12:17:34.499509  6506 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:34.506412  6579 rpcz_store.cc:276] 0709 12:17:32.952957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.240483 (+1287526us) service_pool.cc:224] Handling call
0709 12:17:34.250861 (+ 10378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.285482 (+ 34621us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.434234 (+148752us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.507248  6578 rpcz_store.cc:276] 0709 12:17:32.832217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.225281 (+1393064us) service_pool.cc:224] Handling call
0709 12:17:34.489139 (+263858us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.495629 (+  6490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:34.495658 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:34.495702 (+    44us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:34.495733 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.495792 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:34.495895 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.507128  6581 rpcz_store.cc:276] 0709 12:17:33.031311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.280116 (+1248805us) service_pool.cc:224] Handling call
0709 12:17:34.347651 (+ 67535us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.488904 (+141253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:34.488950 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:34.494718 (+  5768us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:34.494776 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.494858 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:34.494991 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.512598  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:34.507879  6584 pending_rounds.cc:107] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:17:34.517848  6584 raft_consensus.cc:2889] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:34.512645  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:34.520763  6500 raft_consensus.cc:1240] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:34.529335  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:34.529986  6509 raft_consensus.cc:1240] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5. Ops: [1.3-4.4]
I20260709 12:17:34.531383  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:34.532440  7971 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:34.534583  8067 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:17:34.535331  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 350498us
I20260709 12:17:34.535638  8252 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:34.536195  8252 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:34.536521  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 60513.9us
I20260709 12:17:34.536903  8252 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:34.537853  6550 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:34.539810  6550 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:17:34.540482  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 268182us
I20260709 12:17:34.541988  6576 raft_consensus.cc:2955] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:34.506793  6582 rpcz_store.cc:276] 0709 12:17:32.829358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.166569 (+1337211us) service_pool.cc:224] Handling call
0709 12:17:34.229589 (+ 63020us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.240020 (+ 10431us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:17:34.240350 (+   330us) log.cc:844] Serialized 14 byte log entry
0709 12:17:34.427619 (+187269us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:34.427712 (+    93us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:34.427771 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.427864 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:34.428035 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19413,"prepare.queue_time_us":2724,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":169301,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":112,"thread_start_us":60913,"threads_started":1}
I20260709 12:17:34.502161  8180 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:34.496304  8189 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:34.563156  7995 consensus_queue.cc:1059] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:34.563854  6504 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
I20260709 12:17:34.502864  8245 consensus_queue.cc:1059] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 12:17:34.563156  6473 leader_election.cc:400] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 2. Current term is 3.
I20260709 12:17:34.577772  6473 leader_election.cc:403] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:17:34.569895  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:34.510007  6580 rpcz_store.cc:276] 0709 12:17:33.031973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.299300 (+1267327us) service_pool.cc:224] Handling call
0709 12:17:34.448658 (+149358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.449141 (+   483us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:17:34.449749 (+   608us) log.cc:844] Serialized 14 byte log entry
0709 12:17:34.501479 (+ 51730us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:34.501634 (+   155us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:34.501754 (+   120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.501914 (+   160us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:34.502353 (+   439us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41668,"prepare.queue_time_us":183,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":47782,"thread_start_us":3327,"threads_started":1}
W20260709 12:17:34.570828  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 73976us
I20260709 12:17:34.568002  7971 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:34.580570  7971 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:34.581138  7971 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:34.590481  6505 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:34.592935  8247 log.cc:927] Time spent T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.303s	user 0.001s	sys 0.000s
W20260709 12:17:34.593560  6605 debug-util.cc:398] Leaking SignalData structure 0x7b080028b400 after lost signal to thread 5355
I20260709 12:17:34.563428  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:34.594732  6605 debug-util.cc:398] Leaking SignalData structure 0x7b080014c420 after lost signal to thread 6441
I20260709 12:17:34.596544  7962 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:34.597013  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1384) took 1537 ms. Trace:
I20260709 12:17:34.597430  6584 rpcz_store.cc:276] 0709 12:17:33.059889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.463897 (+1404008us) service_pool.cc:224] Handling call
0709 12:17:34.464234 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.596743 (+132509us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.596955 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.597972  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:34.599987  8036 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:17:34.603855  8092 consensus_meta.cc:287] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.598s	user 0.007s	sys 0.003s
I20260709 12:17:34.604933  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:34.606182  8177 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:34.606827  8177 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:34.607663  6550 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:17:34.596292  6605 debug-util.cc:398] Leaking SignalData structure 0x7b08000ec800 after lost signal to thread 6533
I20260709 12:17:34.565696  8159 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:34.576117  8161 consensus_meta.cc:287] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.545s	user 0.012s	sys 0.000s
I20260709 12:17:34.614091  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:34.585265  8181 raft_consensus.cc:3055] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:34.592978  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:34.568701  8180 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:17:34.617791  8209 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 2. Current term is 3.
I20260709 12:17:34.624008  8192 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:34.654633  8064 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:34.633505  7962 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:34.669533  7962 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:34.671931  8253 log.cc:927] Time spent T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.073s	user 0.003s	sys 0.001s
I20260709 12:17:34.668038  8029 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:34.631311  7995 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:34.672060  7962 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:34.678601  8176 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:34.633697  8181 raft_consensus.cc:740] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:17:34.676795  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 113651us
W20260709 12:17:34.685247  8254 log.cc:927] Time spent T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.076s	user 0.000s	sys 0.002s
I20260709 12:17:34.685122  8036 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:34.629346  8245 consensus_queue.cc:1243] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer 20a202c76244485fbc8f1610473db8b9 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:17:34.637351  8172 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:34.701174  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:34.664695  8064 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:34.624872  8161 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:34.703096  6605 debug-util.cc:398] Leaking SignalData structure 0x7b0800167960 after lost signal to thread 6608
W20260709 12:17:34.713946  6605 debug-util.cc:398] Leaking SignalData structure 0x7b0800167940 after lost signal to thread 6609
I20260709 12:17:34.613379  8092 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:34.709088  8036 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:34.710439  6502 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 3: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:34.725437  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:34.726660  8161 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:34.726837  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:34.730307  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:17:34.732558  6605 debug-util.cc:398] Leaking SignalData structure 0x7b0800350140 after lost signal to thread 6682
W20260709 12:17:34.740070  6605 debug-util.cc:398] Leaking SignalData structure 0x7b0800350260 after lost signal to thread 6683
I20260709 12:17:34.679174  8192 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:34.737908  6657 raft_consensus.cc:1240] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:34.739176  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:34.739861  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:34.743561  6500 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:34.737705  6473 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:34.740797  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1376) took 1708 ms. Trace:
I20260709 12:17:34.744323  6577 rpcz_store.cc:276] 0709 12:17:33.032306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.361743 (+1329437us) service_pool.cc:224] Handling call
0709 12:17:34.716129 (+354386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:34.740187 (+ 24058us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:34.740218 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:34.740260 (+    42us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:34.740289 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.740630 (+   341us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:34.740747 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:34.710568  7995 raft_consensus.cc:697] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:34.760080  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:34.762588  6605 debug-util.cc:398] Leaking SignalData structure 0x7b080034fea0 after lost signal to thread 7298
I20260709 12:17:34.763103  6501 raft_consensus.cc:1240] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:34.763195  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:34.764652  6502 raft_consensus.cc:2368] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 4. Current term is 5.
I20260709 12:17:34.765937  6502 raft_consensus.cc:1240] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3. Ops: []
W20260709 12:17:34.767752  8259 log.cc:927] Time spent T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:17:34.767911  6500 raft_consensus.cc:1240] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:34.769832  8192 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:34.773290  6657 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:34.762404  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:34.777774  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:34.778206  8138 consensus_queue.cc:1059] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:34.779197  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:34.779373  8036 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:34.779934  8036 raft_consensus.cc:3055] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:34.780038  6584 raft_consensus.cc:2410] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:34.761480  8106 consensus_queue.cc:260] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:34.781840  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:34.755523  8181 consensus_queue.cc:260] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:34.782742  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:34.783123  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:34.783418  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:34.798247  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:34.761049  8101 consensus_meta.cc:287] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.505s	user 0.010s	sys 0.003s
I20260709 12:17:34.784754  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:34.821632  6579 raft_consensus.cc:1240] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:34.785508  8064 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:34.715667  8176 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:34.822769  8243 raft_consensus.cc:3055] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:34.823338  8243 raft_consensus.cc:740] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:34.824448  6575 raft_consensus.cc:1275] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:34.824774  8176 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:17:34.824983  8262 log.cc:927] Time spent T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:17:34.821539  8106 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:34.825788  6504 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:17:34.786617  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:34.786948  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:34.828311  7269 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:34.829553  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:34.834470  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 98) took 2076 ms. Trace:
I20260709 12:17:34.834910  6583 rpcz_store.cc:276] 0709 12:17:32.758402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.182961 (+1424559us) service_pool.cc:224] Handling call
0709 12:17:34.183421 (+   460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:34.213699 (+ 30278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:34.220520 (+  6821us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:34.242335 (+ 21815us) log.cc:844] Serialized 26 byte log entry
0709 12:17:34.453334 (+210999us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:34.453931 (+   597us) log.cc:844] Serialized 14 byte log entry
0709 12:17:34.501184 (+ 47253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.501262 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:34.600697 (+ 99435us) spinlock_profiling.cc:243] Waited 25.2 ms on lock 0x7b48000030d8. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287adb2aa1 00007f287ada9b37 00007f287ada99ca 00007f287aea126f 00007f287aea0f37 00007f287aeb555c 00007f287aeb5508 00007f287aeb5488 00007f287aeb5443 00007f287aeb46ec 00007f2879ddea96
0709 12:17:34.668270 (+ 67573us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b48000030d8. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287adb2aa1 00007f287ada9b37 00007f287ada99ca 00007f287aea126f 00007f287aea0f37 00007f287aeb555c 00007f287aeb5508 00007f287aeb5488 00007f287aeb5443 00007f287aeb46ec 00007f2879ddea96
0709 12:17:34.834234 (+165964us) raft_consensus.cc:1685] finished
0709 12:17:34.834266 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:34.834393 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47030,"prepare.queue_time_us":157,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":78780,"raft.run_cpu_time_us":4233,"raft.run_wall_time_us":540738,"spinlock_wait_cycles":86962816,"thread_start_us":147360,"threads_started":1,"wal-append.queue_time_us":460520}
I20260709 12:17:34.836592  7269 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:34.788759  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:34.789458  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:34.838196  7995 consensus_queue.cc:237] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:34.790129  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:34.839226  6652 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:34.790680  6505 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:34.760774  8069 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:34.791167  8192 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:34.791693  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:34.792047  8036 raft_consensus.cc:740] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:34.778494  6580 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:34.783438  8161 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:34.800846  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:34.791893  8165 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:34.801018  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:34.801178  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:34.801469  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:34.802044  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:34.803133  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:17:34.804106  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 83504.6us
I20260709 12:17:34.805852  6581 raft_consensus.cc:1275] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:17:34.817996  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:34.852872  6581 pending_rounds.cc:85] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 1
I20260709 12:17:34.853191  6581 pending_rounds.cc:107] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:17:34.853468  6581 raft_consensus.cc:2889] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:34.774145  8178 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:34.795756  6582 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
I20260709 12:17:34.825331  8185 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:34.761363  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:34.788113  7986 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:34.791885  8252 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:34.872293  6473 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:34.872990  6505 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:34.873356  8176 consensus_queue.cc:1059] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:34.873958  7793 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:34.874089  8179 consensus_queue.cc:1059] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:34.875638  6475 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:34.879703  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:34.855206  7272 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 4 LEARNER]: Advancing to term 5
I20260709 12:17:34.840524  7962 consensus_queue.cc:1059] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 12:17:34.859248  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:34.859762  6577 raft_consensus.cc:2393] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 3: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:34.859902  6657 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:34.859587  8185 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:34.860468  6650 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:34.861464  8036 consensus_queue.cc:260] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:34.862411  8161 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:34.862510  7971 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:34.911679  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:34.912946  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:34.855228  6649 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:34.863473  6548 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:34.865347  8192 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:34.865936  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:34.866417  8177 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:34.868435  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:34.869160  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:34.869643  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:34.870467  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:34.870821  6503 raft_consensus.cc:1240] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:17:34.845708  6549 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:17:34.857646  7240 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:34.880852  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:34.923645  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:34.906256  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:34.874929  8166 consensus_meta.cc:287] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.700s	user 0.004s	sys 0.007s
W20260709 12:17:34.910609  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 165086us
I20260709 12:17:34.910888  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:34.907914  6473 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:34.924882  6550 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:34.891049  6622 connection.cc:256] RPC call timeout handler was delayed by -1.6880883670000912s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18400000000000002s for a response
I20260709 12:17:34.920493  6658 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:34.926990  6500 raft_consensus.cc:2393] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 3: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:17:34.929159  6622 connection.cc:256] RPC call timeout handler was delayed by -1.6754883670000709s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1854s for a response
I20260709 12:17:34.931875  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:34.931871  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 78223.2us
W20260709 12:17:34.933359  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 75802.8us
I20260709 12:17:34.933629  6548 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:34.913066  8161 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:34.904394  8106 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:34.932569  6507 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:34.940668  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:34.947057  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 101421us
I20260709 12:17:34.952440  8045 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:34.955184  6549 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:34.957089  6653 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:34.957149  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:34.944145  8119 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:34.926673  7981 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:34.855165  8069 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:34.907925  8181 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:34.934605  8166 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:34.882645  8243 consensus_queue.cc:260] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:34.953137  8045 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:34.904196  7986 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:35.019814  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 64709.2us
I20260709 12:17:35.032051  6549 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:35.020571  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:34.967396  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:34.960817  8189 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:35.022322  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:34.931874  7971 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:34.936303  7793 raft_consensus.cc:697] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 4 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:34.944232  8269 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:17:35.033773  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 158649us
W20260709 12:17:35.038028  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 108253us
W20260709 12:17:35.040162  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 78981.5us
I20260709 12:17:34.962743  8172 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
I20260709 12:17:34.954548  8106 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:34.962359  8161 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:35.024494  8193 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:35.051419  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 89686.6us
I20260709 12:17:35.051649  6548 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:35.054076  6548 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:35.035403  8092 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:35.057168  6548 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:35.036895  8189 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:35.059088  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:35.043085  7269 raft_consensus.cc:1240] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5. Ops: []
W20260709 12:17:35.069969  6622 connection.cc:256] RPC call timeout handler was delayed by -1.6268883669999923s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1908s for a response
I20260709 12:17:35.073998  7272 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEARNER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:35.056434  7971 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:35.088351  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:35.088603  8252 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:35.089039  6651 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:35.088801  7793 consensus_queue.cc:237] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:35.087837  7238 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:35.062973  8189 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:35.093632  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 218054us
I20260709 12:17:35.083901  8243 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:35.100433  6654 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:35.101552  8045 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:35.102398  8165 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:35.103798  8106 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:35.107817  6576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1377) took 2075 ms. Trace:
I20260709 12:17:35.094753  8193 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:35.108764  8227 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:35.093308  8176 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 12:17:35.109472  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:35.109340  8227 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:35.110828  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:35.111385  6649 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:35.112592  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 52598.9us
I20260709 12:17:35.114605  8275 consensus_queue.cc:1059] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:35.094408  8161 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:35.115363  8161 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:35.103932  8273 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:35.115744  8161 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:35.116403  8273 raft_consensus.cc:697] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 2 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:35.117041  8192 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:35.117363  8273 consensus_queue.cc:237] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:35.118978  8159 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:35.119513  8159 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:35.115700  8275 raft_consensus.cc:3055] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:35.120564  8269 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:35.120817  8275 raft_consensus.cc:740] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:35.121256  8106 consensus_queue.cc:1059] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:35.122006  7898 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:35.122179  8106 raft_consensus.cc:3055] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:35.122546  8106 raft_consensus.cc:740] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:35.093806  7962 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:35.123672  8106 consensus_queue.cc:260] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:35.125444  8106 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:35.126678  6622 connection.cc:256] RPC call timeout handler was delayed by -1.5444884790001652s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24670000000000003s for a response
W20260709 12:17:35.127065  6622 connection.cc:256] RPC call timeout handler was delayed by -1.5273964130001332s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1525s for a response
W20260709 12:17:35.127344  6622 connection.cc:256] RPC call timeout handler was delayed by -1.4895296640002016s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3106s for a response
W20260709 12:17:35.127600  6622 connection.cc:256] RPC call timeout handler was delayed by -1.4162883670001065s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2142s for a response
W20260709 12:17:35.127874  6622 connection.cc:256] RPC call timeout handler was delayed by -1.3838883670000541s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2178s for a response
W20260709 12:17:35.128132  6622 connection.cc:256] RPC call timeout handler was delayed by -1.2256820850000167s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21760000000000002s for a response
W20260709 12:17:35.128408  6622 connection.cc:256] RPC call timeout handler was delayed by -1.1761820849999367s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2231s for a response
I20260709 12:17:35.128594  8275 consensus_queue.cc:260] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:35.130527  6656 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:35.130617  8275 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:35.131793  7981 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:35.129117  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:35.132918  8101 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:35.133394  8101 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:35.133796  8101 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:35.134171  8178 raft_consensus.cc:3055] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:35.134548  8178 raft_consensus.cc:740] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:35.130900  6657 raft_consensus.cc:2410] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:35.108217  6576 rpcz_store.cc:276] 0709 12:17:33.032614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.387390 (+1354776us) service_pool.cc:224] Handling call
0709 12:17:34.446281 (+ 58891us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:34.494821 (+ 48540us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:17:34.615269 (+120448us) log.cc:844] Serialized 14 byte log entry
0709 12:17:34.720027 (+104758us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:34.742343 (+ 22316us) log.cc:844] Serialized 26 byte log entry
0709 12:17:34.742463 (+   120us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:34.758113 (+ 15650us) log.cc:844] Serialized 14 byte log entry
0709 12:17:34.760664 (+  2551us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:34.760867 (+   203us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:35.107614 (+346747us) raft_consensus.cc:1685] finished
0709 12:17:35.107638 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:35.107764 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52244,"prepare.queue_time_us":15502,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":50897,"raft.run_cpu_time_us":401,"raft.run_wall_time_us":26596,"thread_start_us":104117,"threads_started":1,"wal-append.queue_time_us":184468}
I20260709 12:17:35.138428  6652 raft_consensus.cc:2410] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:35.148823  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:35.150770  6650 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:35.151471  6651 raft_consensus.cc:1240] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:35.157568  6655 raft_consensus.cc:1240] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:35.159296  6653 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:35.111666  8227 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:35.165957  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:17:35.174573  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 142561us
I20260709 12:17:35.150350  8177 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:35.224249  6507 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:35.220481  6549 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:17:35.121009  8269 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:35.222357  6652 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:35.164443  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:35.225260  6501 raft_consensus.cc:1240] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:35.229192  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 100171us
I20260709 12:17:35.227173  6508 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 3.
W20260709 12:17:35.232985  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 55168.8us
I20260709 12:17:35.234473  6650 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:35.162375  8117 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:35.234930  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:35.232925  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 72324.9us
I20260709 12:17:35.235530  6658 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:35.235083  6622 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:35.239009  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:35.243897  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:35.230685  8269 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:35.248247  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 188844us
I20260709 12:17:35.249305  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:35.249244  8178 consensus_queue.cc:260] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:35.250211  6581 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:35.237299  7995 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:35.247601  6475 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:35.260020  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:35.261138  6550 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:35.247916  8243 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:35.266368  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:35.268028  6507 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 4.
I20260709 12:17:35.270004  6577 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:35.270744  8036 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:35.271756  6577 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:35.272737  8185 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:35.273211  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:35.256893  8179 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:35.273504  8069 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:35.243278  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:35.273942  8179 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:35.275724  7268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38558 (request call id 303) took 1126 ms. Trace:
I20260709 12:17:35.273172  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:35.276157  7268 rpcz_store.cc:276] 0709 12:17:34.149481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.151100 (+  1619us) service_pool.cc:224] Handling call
0709 12:17:34.169100 (+ 18000us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:34.182660 (+ 13560us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:34.182693 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:34.183432 (+   739us) log.cc:844] Serialized 52 byte log entry
0709 12:17:34.347058 (+163626us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:34.671165 (+324107us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:17:34.718503 (+ 47338us) log.cc:844] Serialized 14 byte log entry
0709 12:17:34.747287 (+ 28784us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:35.275390 (+528103us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b54004d7e54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:35.275443 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:35.275488 (+    45us) raft_consensus.cc:1685] finished
0709 12:17:35.275500 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:35.275649 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3723,"prepare.queue_time_us":397240,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":54338,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":218,"spinlock_wait_cycles":60914176,"thread_start_us":294627,"threads_started":2,"wal-append.queue_time_us":414828}
I20260709 12:17:35.279014  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:35.278239  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:35.279527  8180 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:35.281292  8159 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:35.281646  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:35.281311  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:35.284756  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:35.285316  6577 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:35.287817  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:35.289487  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:35.290055  6580 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:35.290531  8064 raft_consensus.cc:3055] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:35.290963  8064 raft_consensus.cc:740] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:17:35.300455  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 57385.1us
W20260709 12:17:35.307840  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:35.309943  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:35.263908  8161 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:17:35.239248  8117 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:35.320152  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:35.320878  6582 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:35.316820  8274 raft_consensus.cc:3055] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:35.317458  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:35.335122  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:35.335994  7271 raft_consensus.cc:2393] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 3: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:35.317734  8178 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:35.336496  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:35.337217  7268 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:35.291846  8064 consensus_queue.cc:260] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:35.339637  8064 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:35.306763  7995 raft_consensus.cc:697] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:35.341399  7995 consensus_queue.cc:237] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:35.344056  8269 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:35.347685  8269 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:35.348407  6578 raft_consensus.cc:2410] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:17:35.349120  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:35.322816  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 92790.4us
I20260709 12:17:35.353510  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:35.354293  8161 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:35.354797  8243 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:35.304807  8067 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:35.356738  8067 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:35.357275  8067 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:35.325999  6649 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 3.
W20260709 12:17:35.326548  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 65437.4us
I20260709 12:17:35.365830  6550 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:17:35.366329  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 44388.9us
I20260709 12:17:35.276072  8179 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:35.327939  8274 raft_consensus.cc:740] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:17:35.358858  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:35.369133  6550 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:17:35.370870  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 313773us
I20260709 12:17:35.375586  6581 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:35.374339  6548 leader_election.cc:400] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 4. Current term is 5.
I20260709 12:17:35.376662  6548 leader_election.cc:403] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:17:35.330282  8192 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:35.377967  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:35.384382  6580 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:35.373745  7986 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:35.388197  8280 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:35.388794  8280 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:35.389178  8280 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:35.373965  8181 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:35.392431  8282 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:35.393080  8282 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:35.395306  8282 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:35.409049  6622 leader_election.cc:400] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3.
I20260709 12:17:35.410482  8287 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 4. Current term is 5.
I20260709 12:17:35.410326  8274 consensus_queue.cc:260] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:35.370200  8177 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:35.412067  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:35.412474  8285 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.167s
I20260709 12:17:35.412820  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:35.414342  6583 raft_consensus.cc:1240] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:35.414417  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:35.417382  6475 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:35.417783  8181 raft_consensus.cc:697] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:35.418021  6582 raft_consensus.cc:1275] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:35.418479  7898 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:35.420980  8101 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:35.403304  8189 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:35.426492  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 52348.6us
I20260709 12:17:35.437081  7240 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:35.421368  6622 leader_election.cc:403] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:17:35.379964  8192 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:35.425634  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:35.428014  6576 raft_consensus.cc:2393] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 5: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:35.430241  8181 consensus_queue.cc:237] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:35.447613  6475 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:35.430627  8101 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:35.431360  8177 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:35.431619  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:35.426955  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:35.450012  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:35.455389  6475 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:35.448482  8273 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:35.469584  6654 raft_consensus.cc:2393] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 5: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:35.452024  8274 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:35.451210  8192 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:35.478757  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:35.479341  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:35.480149  6649 raft_consensus.cc:2393] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 3: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:17:35.468730  7986 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:35.480948  7986 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:35.484968  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:35.454339  8243 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3.
I20260709 12:17:35.434150  8189 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:35.490583  8117 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:35.472836  6475 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:35.474728  8178 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:35.504323  8178 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:35.504748  8178 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:35.508737  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:35.494755  8177 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:35.496760  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:35.452400  8036 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:35.457965  8275 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:17:35.519657  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 46872.6us
I20260709 12:17:35.528249  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:35.539111  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
W20260709 12:17:35.515328  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:35.483763  7986 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:35.541896  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 153821us
I20260709 12:17:35.543095  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:35.507678  8280 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:35.551304  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:35.555308  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:35.556222  6584 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:35.520867  8273 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:35.477039  8067 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:35.560968  6582 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:35.557623  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:35.562402  6475 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:35.543123  8189 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:35.542400  8282 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:35.572131  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:35.574254  6580 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:35.576457  6581 raft_consensus.cc:2393] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 5: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:35.578320  6578 raft_consensus.cc:2393] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 3: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:35.550420  8036 raft_consensus.cc:697] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:35.579077  6475 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:35.558714  8275 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:35.591571  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:35.592170  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:35.602478  6509 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 3: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:35.606168  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:35.580075  7268 raft_consensus.cc:2393] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 3: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:17:35.609215  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:35.612097  6655 raft_consensus.cc:1275] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 2. (term mismatch)
I20260709 12:17:35.640661  6655 pending_rounds.cc:85] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 1
I20260709 12:17:35.640981  6655 pending_rounds.cc:107] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 4.2
I20260709 12:17:35.641273  6655 raft_consensus.cc:2889] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:35.604753  8273 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:35.611939  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:35.653685  8273 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:35.653955  6508 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:35.654158  8273 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:35.555239  8179 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 12:17:35.654945  8178 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:35.657663  8178 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:35.615839  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:35.613217  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:35.664300  6501 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:35.621433  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:35.669034  6576 raft_consensus.cc:2393] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 2: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:35.619210  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:35.670537  8192 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:35.585407  7238 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:35.616227  6473 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:35.618306  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:35.642266  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:35.679507  7273 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:35.565411  8209 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 12:17:35.606801  6506 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:35.644169  6654 raft_consensus.cc:3055] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:35.580379  8282 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:35.649245  8036 consensus_queue.cc:237] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:35.651291  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:35.651453  8280 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:35.607127  8278 consensus_queue.cc:1059] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:17:35.653209  6581 raft_consensus.cc:1275] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 2. (index mismatch)
I20260709 12:17:35.580180  8189 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:35.611734  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:35.582041  7898 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:35.676657  6501 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
W20260709 12:17:35.727695  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 142367us
I20260709 12:17:35.605589  8275 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:35.682323  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:35.684855  6654 raft_consensus.cc:740] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:35.721406  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:35.729152  6655 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:35.614658  7995 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:35.730929  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:35.733080  6506 raft_consensus.cc:2393] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 5: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:35.734004  6653 raft_consensus.cc:1275] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:35.748095  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:35.748750  6500 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:35.749033  8282 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:35.749575  8181 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:35.749879  7986 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:35.750059  8181 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:35.750576  7986 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:35.752492  8178 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:35.752971  8178 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:35.753191  7986 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:35.754333  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:35.754916  8178 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:35.755030  7268 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:35.755777  6580 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:35.755542  6654 consensus_queue.cc:260] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:35.756522  6654 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:35.757061  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 140904us
W20260709 12:17:35.763079  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:35.685978  8209 consensus_queue.cc:1243] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer f2b06aebe97249a4877d798b8118ece4 log is divergent from this leader: its last log entry 1.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 12:17:35.805534  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 47856us
I20260709 12:17:35.806823  6584 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:35.806783  6473 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:35.752092  8181 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:35.726984  8269 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:35.809088  6549 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:35.724895  8177 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:35.811228  6501 raft_consensus.cc:1275] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:35.745945  7238 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:35.815436  6500 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:17:35.732844  8275 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:35.816946  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 242196us
I20260709 12:17:35.733418  8029 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:35.820905  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:35.823634  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 77821us
I20260709 12:17:35.824118  6506 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:35.825230  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:35.736141  7898 raft_consensus.cc:697] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:35.837064  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:35.840020  8067 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:35.840920  6473 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:35.856199  6473 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:35.856876  8179 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:35.857564  6473 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:35.840812  7793 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:35.841843  8266 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:35.841951  7898 consensus_queue.cc:237] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:35.843297  8190 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:35.860508  8179 raft_consensus.cc:1064] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:17:35.807101  8282 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:17:35.860929  8179 raft_consensus.cc:1068] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65: Unable to promote non-voter 20a202c76244485fbc8f1610473db8b9: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:17:35.861264  8282 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:35.861625  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:35.862250  6501 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:35.863111  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:35.864044  6549 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:17:35.843554  8093 raft_consensus.cc:3055] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:35.843827  8189 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:35.845297  8192 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:35.870292  7793 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:35.844194  8287 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:17:35.846794  8296 log.cc:927] Time spent T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.202s	user 0.003s	sys 0.000s
I20260709 12:17:35.888792  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:35.889701  6503 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:35.841984  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:35.848847  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:35.828683  8278 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:35.852305  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:35.895169  6650 raft_consensus.cc:2393] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 5: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:35.852522  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:35.896168  6656 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:35.854235  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:35.809655  7995 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:35.859851  8180 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:35.868672  6509 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:35.869550  6473 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:35.870350  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:35.870546  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:35.903599  6508 raft_consensus.cc:2393] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 5: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:35.887768  6655 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:17:35.896870  6548 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:35.899873  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:35.904785  6652 raft_consensus.cc:2393] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 5: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:17:35.910130  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:35.912765  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:35.914369  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:35.915460  6652 raft_consensus.cc:2393] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 5: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:35.911700  6658 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:17:35.870275  7986 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:17:35.919186  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 49700.5us
I20260709 12:17:35.920769  6473 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:35.922858  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:35.868942  8093 raft_consensus.cc:740] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:35.928853  6550 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:35.869108  8227 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:35.831238  8252 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 12:17:35.868719  8181 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.054s
I20260709 12:17:35.945696  6473 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:35.919497  8064 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:35.956646  6549 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:35.957319  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 127514us
I20260709 12:17:35.918356  8243 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:35.923907  7986 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:35.938866  7793 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:17:35.962350  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:35.940217  8284 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:35.968899  8282 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:35.970717  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 51630.4us
I20260709 12:17:35.971441  7995 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:35.974926  8275 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:35.956454  8178 consensus_queue.cc:1048] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 12:17:35.976395  8192 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:35.963132  7986 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:35.981775  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:35.968211  6473 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:35.988411  6654 raft_consensus.cc:1275] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:17:35.988873  6654 pending_rounds.cc:85] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 1
I20260709 12:17:35.989151  6654 pending_rounds.cc:107] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:17:35.989449  6654 raft_consensus.cc:2889] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:35.991689  6503 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
I20260709 12:17:36.012411  6473 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:35.965551  8227 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:35.993327  8273 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:36.021438  6656 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:17:35.990720  8189 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:35.997052  8177 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:36.023936  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:36.024215  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 194239us
I20260709 12:17:36.028591  8275 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:35.996325  8192 raft_consensus.cc:697] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:36.032145  6505 raft_consensus.cc:2393] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 5: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:36.023934  8227 raft_consensus.cc:697] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:17:36.033152  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 65697.6us
I20260709 12:17:36.021015  8067 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:36.021270  8029 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:36.023125  8284 consensus_queue.cc:1048] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 12:17:36.052560  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:35.995760  8093 consensus_queue.cc:260] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:36.064193  8192 consensus_queue.cc:237] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:36.065758  8252 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:17:36.072572  8298 log.cc:927] Time spent T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.340s	user 0.003s	sys 0.000s
I20260709 12:17:36.072670  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:36.075212  8252 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.075738  8252 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:36.069837  7986 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:36.077374  6622 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:36.079314  7238 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:36.077262  8243 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:36.080567  6651 raft_consensus.cc:1275] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:36.081317  7986 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:36.050400  8269 raft_consensus.cc:697] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 4 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:36.084444  8284 raft_consensus.cc:1064] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:17:36.084890  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:36.101341  6654 raft_consensus.cc:2393] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 5: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:36.084939  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:17:36.060554  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 131758us
W20260709 12:17:36.079465  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 158017us
I20260709 12:17:36.081007  8227 consensus_queue.cc:237] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:36.111127  6548 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:36.068452  8093 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:36.102632  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 136900us
I20260709 12:17:36.053403  8273 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:36.101352  8117 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:36.104357  8284 raft_consensus.cc:1068] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65: Unable to promote non-voter 20a202c76244485fbc8f1610473db8b9: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:17:36.112484  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:36.100507  8305 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:36.124981  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:36.128155  6548 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:36.129627  6548 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:36.130750  6548 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:36.134035  7271 raft_consensus.cc:2393] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 2: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:17:36.131482  7986 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:36.134948  8305 raft_consensus.cc:697] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 2 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:36.135342  7986 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:36.135883  8305 consensus_queue.cc:237] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:36.137465  7986 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:36.138042  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 60760.4us
I20260709 12:17:36.139460  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:17:36.140427  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:36.125119  8280 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:36.141016  8117 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:36.141397  8280 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:36.142618  8252 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:36.142897  7266 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:36.143392  8280 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:36.144047  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:36.144665  7268 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:36.145347  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:36.148731  8269 consensus_queue.cc:237] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:36.149710  6548 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:36.153407  6548 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:36.154441  6550 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:36.155725  6548 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:36.160797  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 148470us
I20260709 12:17:36.164165  6473 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:36.165973  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:36.176163  8176 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:36.178112  6473 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:36.178963  8310 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:36.180372  8177 consensus_queue.cc:1048] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:36.181077  6473 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:36.181042  8310 consensus_queue.cc:1048] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:36.182185  7268 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:36.183146  8310 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:36.183509  8310 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:36.183859  8310 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:36.192024  8310 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:36.193982  8176 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:36.194985  8243 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.161005  6406 catalog_manager.cc:5697] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 2 to 3, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 3 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:17:36.203603  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 47960.3us
I20260709 12:17:36.224674  6548 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:36.199508  8181 raft_consensus.cc:2804] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:17:36.264863  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 52437.4us
I20260709 12:17:36.272547  8181 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:17:36.336786  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 169596us
I20260709 12:17:36.200002  7898 consensus_queue.cc:1048] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.093s
I20260709 12:17:36.208174  8243 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:36.430841  8243 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:36.432963  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:36.435767  8029 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:36.426889  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:36.437160  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:36.199826  7986 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:36.274153  7995 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.084s
I20260709 12:17:36.409948  8287 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:36.410303  8036 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.203086  8291 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.203359  8064 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:36.415925  8282 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:36.200911  8161 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:36.162959  8308 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:36.426241  8176 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:36.469931  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 44900.7us
I20260709 12:17:36.411671  8310 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:36.167903  8189 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:36.412374  6406 catalog_manager.cc:5697] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: config changed from index 2 to 3, term changed from 1 to 3, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:17:36.414218  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 189683us
I20260709 12:17:36.485888  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:36.486708  6581 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:36.165743  8307 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:36.420734  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:36.422310  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:36.424736  8181 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:36.426973  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:36.440158  7268 raft_consensus.cc:2368] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4.
I20260709 12:17:36.409058  6649 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:36.469215  8064 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:36.414891  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 260498us
I20260709 12:17:36.470582  8308 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:36.470058  8282 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:36.497617  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:36.499572  6550 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:17:36.487933  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:36.487639  6501 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 3.
I20260709 12:17:36.487975  6548 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:36.492635  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:36.502162  6576 raft_consensus.cc:2393] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 5: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:36.502801  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:36.493428  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:36.461796  8243 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:36.491258  6549 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:17:36.508630  8275 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.509513  8275 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:36.510732  8332 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:36.510869  6548 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:36.471611  8313 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.482024  8227 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.514969  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:17:36.521401  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:36.521880  6581 raft_consensus.cc:2368] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4.
I20260709 12:17:36.524024  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:36.525399  6503 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:36.526871  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:36.527428  6506 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:36.530256  6504 raft_consensus.cc:1275] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:36.532284  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:36.532907  6655 raft_consensus.cc:3055] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:36.533097  6507 raft_consensus.cc:2393] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 3: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:36.533397  6655 raft_consensus.cc:740] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:36.486840  8321 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:36.540985  8321 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:36.542444  6653 raft_consensus.cc:1275] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:36.543088  8321 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:36.490988  8287 raft_consensus.cc:697] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:36.540153  8307 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:36.494182  7986 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 12:17:36.545523  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:36.513382  8282 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:36.555156  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:36.512682  8064 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:36.482352  8036 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:36.559456  6657 pending_rounds.cc:85] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 2
I20260709 12:17:36.563076  6655 consensus_queue.cc:260] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:36.574345  6657 pending_rounds.cc:107] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 1.3
I20260709 12:17:36.537003  6406 catalog_manager.cc:5697] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 1 to 2, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 2 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:36.569543  6581 raft_consensus.cc:1275] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:36.562821  6505 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:36.520210  8189 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:36.487015  8176 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:36.513273  8308 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:36.471004  8291 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:36.577793  6657 raft_consensus.cc:2917] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted by new leader
I20260709 12:17:36.585801  8036 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:36.586459  8291 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:36.580585  8189 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:36.587378  8189 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:36.587813  8312 log.cc:927] Time spent T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
W20260709 12:17:36.590104  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 45278.1us
W20260709 12:17:36.590811  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 101755us
I20260709 12:17:36.591379  6657 raft_consensus.cc:2889] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:36.561566  8332 raft_consensus.cc:697] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:36.596407  6581 raft_consensus.cc:1275] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:36.597072  6655 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:36.608872  6509 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:36.578886  6406 catalog_manager.cc:5697] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 1 to 4, now has a pending config: VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1), VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2), VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3), VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 4 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:36.613667  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:36.616839  6654 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
I20260709 12:17:36.538298  8227 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:36.625172  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:36.628520  6506 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:36.630019  6652 raft_consensus.cc:3055] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:36.630421  6652 raft_consensus.cc:740] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:36.539307  8313 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:36.639854  6474 leader_election.cc:400] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4.
I20260709 12:17:36.640280  8313 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:36.642144  8161 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:36.643010  8313 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.626215  7995 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.612098  8117 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.623908  8331 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:17:36.637081  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 45459.8us
I20260709 12:17:36.643541  8313 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:36.661522  8321 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:36.662139  8321 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:36.618618  8307 raft_consensus.cc:697] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:36.640178  6474 leader_election.cc:403] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:17:36.621224  6406 catalog_manager.cc:5697] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 1 to 3, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 3 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:36.650632  6655 raft_consensus.cc:1275] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:36.635357  8332 consensus_queue.cc:237] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:36.653445  6652 consensus_queue.cc:260] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:36.637679  8176 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:36.665052  6473 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:36.708956  7272 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:17:36.667649  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:36.668773  6402 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57900 (request call id 186) took 7978 ms. Trace:
W20260709 12:17:36.709632  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 44694.4us
I20260709 12:17:36.702243  6550 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:36.708897  8280 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.712376  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:36.709376  7272 pending_rounds.cc:85] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9: Aborting all ops after (but not including) 2
I20260709 12:17:36.713086  7898 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:36.713505  7272 pending_rounds.cc:107] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 1.3
I20260709 12:17:36.713615  7898 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:36.713850  7272 raft_consensus.cc:2917] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted by new leader
I20260709 12:17:36.713764  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:36.714429  7272 raft_consensus.cc:2889] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:36.714781  7898 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:36.714915  7272 pending_rounds.cc:107] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:17:36.714984  6583 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:36.715227  7272 raft_consensus.cc:2889] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:36.712903  6473 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:36.717561  6473 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:36.718122  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:36.718132  6581 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:36.719015  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:36.721452  6575 raft_consensus.cc:1275] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:36.696928  6406 catalog_manager.cc:5697] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 1 to 3, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 3 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:17:36.719687  6583 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:36.729413  6652 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:36.729547  6473 leader_election.cc:400] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4.
I20260709 12:17:36.662752  8313 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:36.703233  8312 log.cc:927] Time spent T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:17:36.734329  6580 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:36.743702  6406 catalog_manager.cc:5697] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 1 to 3, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 3 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
W20260709 12:17:36.748823  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 136014us
I20260709 12:17:36.756603  6406 catalog_manager.cc:5697] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 1 to 3, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 3 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:17:36.701561  8177 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4.
I20260709 12:17:36.757942  6406 catalog_manager.cc:5697] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 1 to 2, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 2 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:36.759047  6654 raft_consensus.cc:1275] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:36.745260  8176 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:36.759889  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:36.841696  6581 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:36.709225  8291 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:36.784042  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 81861.8us
I20260709 12:17:36.765686  6406 catalog_manager.cc:5697] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 1 to 3, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 3 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:17:36.838918  8161 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:17:36.848562  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 87911.9us
I20260709 12:17:36.732509  8273 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:17:36.844133  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 87989.8us
I20260709 12:17:36.849014  6406 catalog_manager.cc:5697] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:36.712961  8280 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:36.759980  8335 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.083s
I20260709 12:17:36.848043  6652 raft_consensus.cc:1275] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:17:36.869593  6406 catalog_manager.cc:5697] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 2 to 4, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 4 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:17:36.671669  8117 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:36.720958  7995 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:36.874603  6406 catalog_manager.cc:5697] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 3 to 4, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 4 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:36.839586  8330 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:36.849216  8333 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:36.655023  8287 consensus_queue.cc:237] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:17:36.843181  8323 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:36.856304  8291 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:36.869472  8161 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:36.871960  8192 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:37.034930  6406 catalog_manager.cc:5697] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 1 to 2, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 2 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:37.036907  8117 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:37.040298  8189 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:37.043785  8181 consensus_meta.cc:287] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.531s	user 0.008s	sys 0.000s
I20260709 12:17:37.047912  8101 consensus_queue.cc:1048] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 12:17:37.059262  8181 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:36.671487  8331 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:37.060657  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:37.061789  8101 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:37.062369  8101 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:37.062763  8101 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:36.898754  8307 consensus_queue.cc:237] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:37.054590  8093 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:37.064888  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:37.065241  8093 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:37.061902  6622 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:36.853868  8305 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 12:17:37.067225  8189 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:36.748168  8284 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.123s
I20260709 12:17:37.067451  8282 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:37.068195  8338 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:37.069160  8338 consensus_queue.cc:1048] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:36.878005  7793 consensus_queue.cc:1048] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 12:17:36.864073  8209 consensus_queue.cc:1048] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.027s
W20260709 12:17:37.057921  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 183618us
I20260709 12:17:37.037467  8227 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:37.038641  8346 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:37.188663  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:37.188740  8346 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:37.190114  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:37.190562  6473 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:37.191223  8346 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:37.065670  6657 raft_consensus.cc:2393] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 5: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:37.191753  8209 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:37.192184  8209 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:37.192502  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:37.064220  7898 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:37.192817  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:37.193359  6657 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
W20260709 12:17:37.194005  8314 log.cc:927] Time spent T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.681s	user 0.000s	sys 0.019s
I20260709 12:17:37.194267  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:37.194695  7898 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:37.130420  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:37.056792  6584 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:37.067358  8093 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:17:37.127969  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:37.195991  6473 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:17:37.196668  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:37.196867  7898 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:37.196940  6473 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:37.197402  7898 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:37.197630  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:37.197975  7898 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:37.126402  8331 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:37.125618  8282 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:37.199308  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:37.200054  6657 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:37.202664  6657 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:17:37.202795  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:37.134379  6583 consensus_queue.cc:237] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:37.046655  8336 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.068s
I20260709 12:17:37.054774  8333 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:37.206732  8333 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:37.126266  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:37.040863  8334 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:17:37.136241  8064 consensus_meta.cc:287] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.553s	user 0.008s	sys 0.003s
I20260709 12:17:37.135149  8310 raft_consensus.cc:1064] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
W20260709 12:17:37.208797  8310 raft_consensus.cc:1068] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65: Unable to promote non-voter 20a202c76244485fbc8f1610473db8b9: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:17:37.141721  7995 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:37.128106  8308 consensus_meta.cc:287] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.535s	user 0.002s	sys 0.010s
I20260709 12:17:37.208724  8064 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:37.209619  8308 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:37.210137  6652 raft_consensus.cc:1275] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:37.211011  8064 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:37.138497  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:37.211694  8308 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:37.211834  6655 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:37.144682  7238 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:37.212558  6578 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:17:37.212704  6577 raft_consensus.cc:1240] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:17:37.058096  8192 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:37.146279  8323 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:37.213508  8323 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:37.152700  6582 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.046s	sys 0.097s
W20260709 12:17:37.214571  6582 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.047s	sys 0.097s
I20260709 12:17:37.145924  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:17:37.215147  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 69287us
I20260709 12:17:37.154445  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:37.159323  8280 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:37.151892  8101 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:37.216024  8192 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:37.163775  8190 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:37.223142  7898 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:37.223613  8190 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:37.164464  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:37.224938  6504 raft_consensus.cc:2393] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 3: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:37.157945  8181 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:37.165158  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:37.225494  6473 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:37.225906  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:37.226073  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:37.165349  8291 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:37.226230  6507 raft_consensus.cc:2393] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 5: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:37.226264  7793 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:37.166359  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:37.226622  6504 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
I20260709 12:17:37.226917  7271 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:37.167121  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:37.227655  6506 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:37.164983  6509 raft_consensus.cc:1275] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:37.228787  6506 raft_consensus.cc:1275] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:17:37.167160  8161 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:37.168352  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:37.168469  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:37.230794  6508 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:37.227656  7793 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:37.231462  7793 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:37.231674  8177 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:37.167503  8321 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:37.169493  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:37.176029  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:37.173029  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:37.232288  8177 raft_consensus.cc:697] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:37.176059  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:37.234447  6501 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 3.
W20260709 12:17:37.235003  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 62060.8us
I20260709 12:17:37.184132  8275 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:37.132226  7240 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:37.234333  8284 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:37.236110  8093 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:37.236563  8093 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:37.192579  8209 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:37.236670  8284 raft_consensus.cc:697] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 2 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:37.238286  8284 consensus_queue.cc:237] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:37.227227  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:37.228075  6504 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:37.239583  6507 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:17:37.191257  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:37.240372  6649 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:37.155220  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:37.216616  8323 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:37.240790  6505 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 2: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:17:37.236989  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 104868us
I20260709 12:17:37.241370  6651 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:37.217801  8101 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:17:37.163335  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:37.208196  6650 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
W20260709 12:17:37.220098  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 75504.9us
I20260709 12:17:37.242908  8269 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:17:37.220129  8335 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:37.243302  8308 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:37.203146  6657 pending_rounds.cc:85] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 2
I20260709 12:17:37.245433  6657 pending_rounds.cc:107] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 1.3
I20260709 12:17:37.245797  6657 raft_consensus.cc:2917] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted by new leader
I20260709 12:17:37.243876  8335 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:37.246534  6657 raft_consensus.cc:2889] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:37.246971  6657 pending_rounds.cc:107] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:17:37.247267  6550 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:37.166167  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:37.248072  6503 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:37.248790  6505 raft_consensus.cc:1240] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:37.250011  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:37.250797  6501 raft_consensus.cc:2393] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 5: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:37.227178  6500 raft_consensus.cc:2393] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 2: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:37.234598  8177 consensus_queue.cc:237] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:37.254309  6507 raft_consensus.cc:1240] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:37.238922  8190 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:37.236908  8093 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:37.257515  6503 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:37.221181  8336 raft_consensus.cc:2764] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:17:37.262295  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:37.262948  8093 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:37.246902  8335 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:37.263898  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:37.264833  7268 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:37.217658  8334 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:37.268476  8192 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:37.268433  6548 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:37.268730  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:37.269212  8323 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:37.269531  8291 consensus_queue.cc:1048] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:37.270300  8334 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:37.270731  8323 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:37.271229  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:37.271606  6548 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:37.271876  6506 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:17:37.272100  8266 consensus_queue.cc:1059] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:37.272507  8291 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:37.272981  8291 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:37.273422  8291 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:37.273648  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:37.277443  6548 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:37.277637  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:37.278400  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:37.278826  8161 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:37.279314  8161 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:37.279680  8161 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:37.181934  6406 catalog_manager.cc:5697] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 1 to 3, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 3 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:17:37.280836  6406 catalog_manager.cc:5697] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 1 to 2, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 2 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:37.282666  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:37.282902  6406 catalog_manager.cc:5697] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 1 to 3, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 3 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:17:37.285676  8192 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:37.286342  6508 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:37.287078  6504 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:36.709874  6402 rpcz_store.cc:276] 0709 12:17:28.690557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:28.698494 (+  7937us) service_pool.cc:224] Handling call
0709 12:17:30.664019 (+1965525us) spinlock_profiling.cc:243] Waited 531 ms on lock 0x7b540008acd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df3090 00007f2875e91f3a 00007f287bc47e9c 00007f287bd4604e 00007f287a2072a4 00007f287a207232 00007f287a207161 00007f287a2070dc 00007f287a206392 00007f28741c448c 00007f28741c38c6
0709 12:17:36.668697 (+6004678us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:34.683906 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:34.684792 (+   886us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:34.684862 (+    70us) write_op.cc:435] Acquired schema lock
0709 12:17:34.684904 (+    42us) tablet.cc:662] Decoding operations
0709 12:17:34.724367 (+ 39463us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:34.753095 (+ 28728us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:34.754207 (+  1112us) tablet.cc:685] Acquiring locks for 6 operations
0709 12:17:35.021778 (+267571us) tablet.cc:701] Row locks acquired
0709 12:17:35.021841 (+    63us) write_op.cc:260] PREPARE: finished
0709 12:17:35.022109 (+   268us) write_op.cc:270] Start()
0709 12:17:35.022286 (+   177us) write_op.cc:276] Timestamp: P: 1783599455022031 usec, L: 0
0709 12:17:35.022328 (+    42us) op_driver.cc:348] REPLICATION: starting
0709 12:17:35.054466 (+ 32138us) log.cc:844] Serialized 2479 byte log entry
0709 12:17:35.447344 (+392878us) op_driver.cc:464] REPLICATION: finished
0709 12:17:35.552978 (+105634us) write_op.cc:301] APPLY: starting
0709 12:17:35.566051 (+ 13073us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:35.580609 (+ 14558us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:35.618146 (+ 37537us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:35.683782 (+ 65636us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:35.684068 (+   286us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 12:17:35.684137 (+    69us) write_op.cc:312] APPLY: finished
0709 12:17:35.735411 (+ 51274us) log.cc:844] Serialized 53 byte log entry
0709 12:17:35.735628 (+   217us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:35.735756 (+   128us) write_op.cc:454] Released schema lock
0709 12:17:35.746968 (+ 11212us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:35.747367 (+   399us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:35.747401 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1750003,"reported_tablets":14,"spinlock_wait_cycles":1220551552,"child_traces":[["op",{"apply.queue_time_us":98238,"apply.run_cpu_time_us":4651,"apply.run_wall_time_us":194407,"num_ops":6,"prepare.queue_time_us":109164,"prepare.run_cpu_time_us":113634,"prepare.run_wall_time_us":554475,"raft.queue_time_us":104556,"raft.run_cpu_time_us":15931,"raft.run_wall_time_us":25435,"replication_time_us":424841,"thread_start_us":315211,"threads_started":4,"wal-append.queue_time_us":272430,"wal-append.run_cpu_time_us":13846,"wal-append.run_wall_time_us":1427888}]]}
I20260709 12:17:37.218930  6658 raft_consensus.cc:1240] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:37.290277  8161 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:37.291533  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:37.316056  6503 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:17:37.263784  8190 consensus_queue.cc:1048] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:37.335654  6549 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:37.337531  6549 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
W20260709 12:17:37.297881  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 55175.7us
I20260709 12:17:37.247486  6657 raft_consensus.cc:2889] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:17:37.297927  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 50725.3us
I20260709 12:17:37.309773  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:37.346484  6550 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:37.346590  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:37.347224  8305 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:17:37.347640  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:37.309456  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:37.349227  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:37.350052  6576 raft_consensus.cc:2393] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 3: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:37.350893  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:37.351533  6576 raft_consensus.cc:2393] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 5: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:37.314296  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:37.314296  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:37.353528  6575 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 5: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:17:37.318996  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 93560.4us
I20260709 12:17:37.359745  6580 raft_consensus.cc:2393] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 5: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:37.360904  6576 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:37.373320  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 75722.9us
I20260709 12:17:37.378335  6550 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:37.380378  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:37.381106  6580 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:37.382858  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:37.388851  6580 raft_consensus.cc:2393] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 5: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:37.396621  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:37.397848  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:37.396716  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:37.399076  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:37.404784  6578 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:37.411072  6473 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:37.334620  7986 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:17:37.396371  8362 log.cc:927] Time spent T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.058s	user 0.000s	sys 0.006s
I20260709 12:17:37.416675  6576 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:17:37.394881  7981 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:37.420825  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:37.421649  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:37.423084  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:37.398316  8181 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:37.398043  8064 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:37.432029  6575 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:17:37.496949  6583 raft_consensus.cc:1275] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:17:37.461184  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:37.441387  8332 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:37.508960  6583 raft_consensus.cc:1240] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
W20260709 12:17:37.525381  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 114418us
I20260709 12:17:37.397811  8067 consensus_queue.cc:1048] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 12:17:37.452104  6651 raft_consensus.cc:1275] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:17:37.541040  6622 connection.cc:256] RPC call timeout handler was delayed by -1.2562723910000386s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1665s for a response
I20260709 12:17:37.434736  7986 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:37.496671  8323 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:37.553422  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 139879us
I20260709 12:17:37.437965  8181 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:37.441035  8064 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:37.565166  6475 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:37.570863  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:37.435532  7271 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:37.527631  8177 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:37.526549  8332 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:37.438313  8209 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:37.584594  6653 raft_consensus.cc:2393] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 4: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:37.550433  8093 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:37.547132  7986 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:37.565032  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:37.590550  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:37.590837  8358 log.cc:927] Time spent T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
I20260709 12:17:37.591118  6503 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:17:37.592748  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:37.570564  8181 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:37.605590  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:37.607218  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:37.586392  8332 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:37.576156  8064 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:37.578308  8177 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:37.617395  8177 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:37.614054  8327 log.cc:927] Time spent T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 12:17:37.585605  8291 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:37.623864  8291 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:37.614926  8337 log.cc:927] Time spent T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:17:37.616834  8161 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:37.582626  8117 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:37.624619  6583 raft_consensus.cc:1240] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:37.605221  8227 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:37.614509  8272 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:37.687651  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:37.688617  6506 raft_consensus.cc:2393] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 4: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:37.588253  8313 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:37.630198  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:37.631165  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:37.654925  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:37.686959  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:37.591118  6649 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:37.612684  8179 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:37.696018  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:37.585371  8180 consensus_queue.cc:1059] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 12:17:37.702862  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:37.705580  6500 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
W20260709 12:17:37.735293  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 206381us
I20260709 12:17:37.639041  7898 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:37.679246  8192 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:37.747956  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 206154us
I20260709 12:17:37.703414  8313 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:37.751336  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:37.751325  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:37.752624  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:37.760835  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:37.769992  6653 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:37.738514  8179 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:37.695837  8117 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:37.705118  8285 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 12:17:37.755810  8313 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:37.701932  8272 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:37.782827  8305 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:37.808660  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:37.826228  8093 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.112s
I20260709 12:17:37.896054  8243 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 12:17:37.886317  8338 consensus_queue.cc:1059] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 12:17:37.897568  8305 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:17:37.948549  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 139692us
W20260709 12:17:37.951094  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 134183us
W20260709 12:17:37.998725  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:38.006855  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 54993.9us
I20260709 12:17:38.009850  6653 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
W20260709 12:17:38.014981  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 228929us
I20260709 12:17:37.957491  8305 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:38.012766  6649 raft_consensus.cc:2393] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 5: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:38.026610  8345 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.027642  8383 consensus_queue.cc:1059] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:17:38.026196  6473 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:38.030395  6622 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:38.001255  8209 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:38.011868  8093 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:37.941488  8367 consensus_queue.cc:1059] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.431s
I20260709 12:17:38.025149  8275 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.058138  8209 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:38.062433  8378 log.cc:927] Time spent T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
W20260709 12:17:38.063115  8335 consensus_meta.cc:287] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.798s	user 0.009s	sys 0.001s
I20260709 12:17:38.063548  8335 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:38.064390  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:38.002408  8372 consensus_queue.cc:1048] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.384s
I20260709 12:17:38.061666  8275 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:38.100536  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.060145  8093 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:38.106470  8342 log.cc:927] Time spent T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:17:38.043370  8345 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:38.106631  8399 consensus_queue.cc:1059] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 12:17:38.102231  8383 raft_consensus.cc:3055] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:38.132138  8180 consensus_queue.cc:1048] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:38.132426  8399 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:38.133158  8338 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:38.135540  8384 log.cc:927] Time spent T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.235s	user 0.001s	sys 0.000s
I20260709 12:17:38.136636  8093 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:38.143569  8064 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:38.134510  8272 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:38.153928  8036 raft_consensus.cc:3055] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:38.154469  8036 raft_consensus.cc:740] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:38.155452  8036 consensus_queue.cc:260] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:38.135182  8383 raft_consensus.cc:740] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:17:38.188057  8376 log.cc:927] Time spent T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 12:17:38.144910  8117 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:38.157876  8335 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:38.189361  8372 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:38.221583  8036 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:38.200837  8275 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:38.195062  8064 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:38.225809  8379 log.cc:927] Time spent T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.051s	user 0.003s	sys 0.000s
I20260709 12:17:38.228127  8399 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.228650  8117 raft_consensus.cc:3055] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:17:38.259704  8181 consensus_meta.cc:287] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.638s	user 0.008s	sys 0.003s
I20260709 12:17:38.260882  8390 raft_consensus.cc:3055] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:38.261276  8390 raft_consensus.cc:740] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:17:38.266517  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 267761us
I20260709 12:17:38.246402  8117 raft_consensus.cc:740] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:17:38.261678  8190 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.006s	sys 0.029s
W20260709 12:17:38.262045  8332 consensus_meta.cc:287] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.577s	user 0.009s	sys 0.001s
I20260709 12:17:38.277437  8209 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:38.278219  7793 consensus_meta.cc:287] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 1.046s	user 0.008s	sys 0.004s
I20260709 12:17:38.277999  8209 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:38.278637  7793 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:38.275064  8190 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.006s	sys 0.029s
I20260709 12:17:38.260186  8181 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:38.281620  8332 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:38.287832  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:38.245042  8372 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:38.288499  6507 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:38.289532  7793 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:38.290342  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:38.300917  6505 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 4.
I20260709 12:17:38.246935  8399 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:38.289903  6475 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:38.304706  8272 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:38.305245  8272 raft_consensus.cc:697] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:38.306097  8272 consensus_queue.cc:237] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:38.306492  8399 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:38.280201  8209 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:38.307924  6507 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 4.
I20260709 12:17:38.246551  8383 consensus_queue.cc:260] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:38.287964  8181 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:38.307092  8313 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:38.322968  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 258705us
W20260709 12:17:38.328663  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 232580us
I20260709 12:17:38.331151  8313 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:38.311565  6475 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:38.327570  8332 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:38.336386  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:38.318933  8345 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:38.329344  7986 consensus_meta.cc:287] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.718s	user 0.010s	sys 0.002s
I20260709 12:17:38.341424  6622 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:38.342609  8398 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:38.346098  8398 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:38.346486  8398 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:38.343151  8403 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:38.285988  8390 consensus_queue.cc:260] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:38.343663  8176 raft_consensus.cc:3055] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:38.313060  8117 consensus_queue.cc:260] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:38.353529  8117 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:38.354477  8400 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:38.342329  7986 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:38.361305  8372 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:38.368829  8176 raft_consensus.cc:740] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:17:38.374897  8177 consensus_meta.cc:287] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.755s	user 0.012s	sys 0.000s
I20260709 12:17:38.437506  8177 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:38.374568  8390 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:38.439183  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 66062.7us
W20260709 12:17:38.441138  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.420488  8305 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:38.442852  8177 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:38.448699  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:38.452318  8305 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:38.455947  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:38.447988  8252 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:38.463124  8274 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:38.463779  8274 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:38.465368  8408 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.466984  8274 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:38.467674  8274 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:38.459662  8252 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:38.470415  7986 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:38.441108  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:38.475936  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:38.478569  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:38.477903  8029 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:38.444988  8176 consensus_queue.cc:260] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:38.488505  6622 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:38.490917  8373 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.491611  8373 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:38.487043  8029 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:38.494520  8373 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:38.494810  8029 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:38.483121  8117 consensus_queue.cc:1059] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
W20260709 12:17:38.501077  8360 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.880s	user 0.000s	sys 0.008s
W20260709 12:17:38.501644  8352 log.cc:927] Time spent T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.880s	user 0.012s	sys 0.000s
W20260709 12:17:38.501844  8368 log.cc:927] Time spent T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.881s	user 0.000s	sys 0.010s
I20260709 12:17:38.467512  8408 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:38.503233  8351 log.cc:927] Time spent T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.888s	user 0.000s	sys 0.014s
I20260709 12:17:38.512641  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:38.513437  7267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38558 (request call id 324) took 1437 ms. Trace:
I20260709 12:17:38.507601  8252 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:38.513885  7267 rpcz_store.cc:276] 0709 12:17:37.076161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.132829 (+ 56668us) service_pool.cc:224] Handling call
0709 12:17:37.133155 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:37.133893 (+   738us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:37.133927 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:37.134352 (+   425us) log.cc:844] Serialized 26 byte log entry
0709 12:17:37.449048 (+314696us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:37.449132 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:37.449229 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:38.502424 (+1053195us) spinlock_profiling.cc:243] Waited 71.1 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:17:38.513109 (+ 10685us) raft_consensus.cc:1685] finished
0709 12:17:38.513138 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:38.513359 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":294,"raft.run_cpu_time_us":50,"raft.run_wall_time_us":50,"spinlock_wait_cycles":163416448,"thread_start_us":314303,"threads_started":1,"wal-append.queue_time_us":419338}
I20260709 12:17:38.516013  7269 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:38.468284  8274 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:38.506508  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1136) took 1124 ms. Trace:
W20260709 12:17:38.507690  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 973793us
I20260709 12:17:38.509963  6576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1139) took 1127 ms. Trace:
I20260709 12:17:38.521386  6576 rpcz_store.cc:276] 0709 12:17:37.382673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.451055 (+ 68382us) service_pool.cc:224] Handling call
0709 12:17:37.451356 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:37.451539 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:17:37.563356 (+111817us) log.cc:844] Serialized 14 byte log entry
0709 12:17:38.509662 (+946306us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:38.509712 (+    50us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:38.509752 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:38.509814 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:38.509921 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1027078,"prepare.queue_time_us":115986,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":950552,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":75}
I20260709 12:17:38.521015  6580 rpcz_store.cc:276] 0709 12:17:37.382003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.430802 (+ 48799us) service_pool.cc:224] Handling call
0709 12:17:37.431048 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:37.431230 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:17:37.527488 (+ 96258us) log.cc:844] Serialized 14 byte log entry
0709 12:17:38.506131 (+978643us) spinlock_profiling.cc:243] Waited 6.2 ms on lock 0x7b1800462760. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d16c0a 00007f287c456219 00007f2872d9ea40 00007f2872d994e4 00007f2872d99455 00007f2875e81060 00007f2875e93716 00007f2875ea4287 00007f2875ea4208
0709 12:17:38.506263 (+   132us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:38.506308 (+    45us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:38.506346 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:38.506389 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:38.506466 (+    77us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47279,"prepare.queue_time_us":101287,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":11057,"raft.queue_time_us":979861,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":8338,"spinlock_wait_cycles":14261632}
I20260709 12:17:38.511400  8406 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.526119  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:38.526599  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:38.527124  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:38.527554  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:38.528180  6651 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:17:38.530023  6579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1135) took 1148 ms. Trace:
I20260709 12:17:38.530195  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:38.532655  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:38.532963  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:38.533530  6649 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:38.515552  8398 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:38.536512  8408 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:38.536376  8406 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:38.543200  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:38.644616  6656 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:38.538311  8409 consensus_queue.cc:1059] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:38.564155  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:38.617285  6622 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:38.640156  8412 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:17:38.642767  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 88907.8us
I20260709 12:17:38.646023  6506 raft_consensus.cc:2393] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 5: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:38.530454  6579 rpcz_store.cc:276] 0709 12:17:37.381793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.429250 (+ 47457us) service_pool.cc:224] Handling call
0709 12:17:37.429537 (+   287us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:37.442336 (+ 12799us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:37.442371 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:37.506900 (+ 64529us) log.cc:844] Serialized 52 byte log entry
0709 12:17:37.747843 (+240943us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:37.767956 (+ 20113us) log.cc:844] Serialized 14 byte log entry
0709 12:17:38.529718 (+761762us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:38.529790 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:38.529831 (+    41us) raft_consensus.cc:1685] finished
0709 12:17:38.529845 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:38.529970 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1713367,"prepare.queue_time_us":19900,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":6729,"raft.queue_time_us":120554,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":20100,"thread_start_us":175629,"threads_started":1,"wal-append.queue_time_us":481867}
I20260709 12:17:38.649155  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:38.651562  6507 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:38.652684  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:38.658509  6509 raft_consensus.cc:2393] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 4: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:38.658962  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:38.659560  6504 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:38.659637  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:38.662171  6578 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:38.641402  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:38.665481  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:17:38.685555  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.688223  6577 raft_consensus.cc:1240] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:17:38.673705  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:38.692972  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:38.688987  7238 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:38.685622  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:38.686347  6500 raft_consensus.cc:1240] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:38.702785  6584 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:38.698639  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:38.700959  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:38.703560  6580 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:38.673285  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:38.704730  6505 raft_consensus.cc:2393] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 4: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:38.702353  6579 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:38.687395  6576 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:38.709718  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:38.703526  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:38.719269  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:38.722383  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:38.723368  6580 raft_consensus.cc:2393] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 5: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:38.725708  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:38.726924  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:38.720043  6582 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:38.729386  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:38.648535  8412 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:38.742815  6500 raft_consensus.cc:1240] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 12:17:38.745133  8402 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:38.730504  6582 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 6: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:38.748389  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:38.749425  6500 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:38.752534  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:17:38.760805  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 72507us
I20260709 12:17:38.688476  8287 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.769050  6550 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:38.774631  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 120) took 1474 ms. Trace:
I20260709 12:17:38.775143  6581 rpcz_store.cc:276] 0709 12:17:37.300581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.301050 (+   469us) service_pool.cc:224] Handling call
0709 12:17:37.301638 (+   588us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:37.334359 (+ 32721us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:37.334395 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:37.393810 (+ 59415us) log.cc:844] Serialized 26 byte log entry
0709 12:17:37.532438 (+138628us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:37.532500 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:37.532616 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:38.501126 (+968510us) spinlock_profiling.cc:243] Waited 59.5 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:17:38.767567 (+266441us) raft_consensus.cc:1685] finished
0709 12:17:38.767598 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:38.774536 (+  6938us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14471,"raft.queue_time_us":12001,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":61214,"spinlock_wait_cycles":136747904,"thread_start_us":95564,"threads_started":1,"wal-append.queue_time_us":213750}
I20260709 12:17:38.770643  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:38.771247  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:38.782636  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:38.783399  6650 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:17:38.665796  8335 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:38.784657  8335 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:38.787349  8335 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:38.727872  8400 raft_consensus.cc:3055] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:38.710022  8373 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:38.708912  8280 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:38.772603  6576 raft_consensus.cc:1240] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 12:17:38.720512  8093 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:38.765137  8412 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:38.741505  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:17:38.803697  8417 log.cc:927] Time spent T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:17:38.804108  8421 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.804571  8411 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.802088  6656 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 3.
I20260709 12:17:38.812620  6622 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:38.845323  6504 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
W20260709 12:17:38.815521  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 46544.7us
I20260709 12:17:38.976522  6475 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:38.941211  6548 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:38.976714  6650 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 4.
W20260709 12:17:38.916697  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:38.757844  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:38.814659  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:38.978837  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:38.972278  8419 log.cc:927] Time spent T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 12:17:39.004107  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:38.767401  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:39.000020  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:38.799928  8400 raft_consensus.cc:740] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:39.008783  6582 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:17:39.001866  6576 raft_consensus.cc:2393] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 4: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:17:39.009657  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 68571.4us
I20260709 12:17:38.926198  8335 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:38.964896  8390 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:38.832774  8338 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:38.863691  8409 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:38.980607  8402 raft_consensus.cc:697] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 4 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:39.099591  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:38.962496  8180 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:38.799276  8308 consensus_queue.cc:1048] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.218s
I20260709 12:17:39.082569  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:38.819130  8371 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:39.083438  6548 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:39.083992  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:39.081282  6581 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:39.087424  6507 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:38.862008  8181 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:17:39.089911  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 277343us
I20260709 12:17:39.074916  8405 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:38.982514  8029 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:39.097697  8274 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:38.999926  8064 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:39.108073  7240 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:39.108445  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:39.139837  6651 consensus_meta.cc:287] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.611s	user 0.009s	sys 0.001s
I20260709 12:17:39.089169  8161 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:39.010264  6658 raft_consensus.cc:2468] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 1.
I20260709 12:17:39.074515  8282 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:39.114089  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:39.113538  8181 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:38.984486  8421 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:39.148895  8181 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:39.138257  8029 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:39.098174  8390 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:39.121935  6548 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:39.171774  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 49945.3us
I20260709 12:17:39.172792  6548 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:39.174144  6548 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:39.007488  8413 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:39.106770  6502 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 6: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:38.983464  8411 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:17:39.118635  8371 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:39.091135  8398 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:38.982493  8280 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:39.143560  8344 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 12:17:39.127753  8252 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:17:39.137646  8418 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.263s	user 0.001s	sys 0.000s
I20260709 12:17:39.113258  8180 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:39.140202  6651 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:39.210876  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:39.143525  6581 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "20a202c76244485fbc8f1610473db8b9"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:17:39.117661  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:39.110704  8347 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:17:39.091871  8335 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:39.122287  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:39.179107  6548 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:39.127002  8331 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:39.147743  8274 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.114821  8338 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:39.190763  6501 raft_consensus.cc:1275] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:39.150457  8406 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:39.118300  8272 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:38.972941  8093 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:39.183081  8332 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:38.808130  8287 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:39.187500  8192 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:39.217108  6509 raft_consensus.cc:1240] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 12:17:39.216547  6581 raft_consensus.cc:606] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Received request to transfer leadership to TS 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:39.194868  8323 raft_consensus.cc:2804] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:39.147231  8064 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:39.158420  8161 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:39.226117  6579 consensus_meta.cc:287] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.504s	user 0.011s	sys 0.000s
I20260709 12:17:39.293414  6579 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:39.128057  8409 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:39.300351  8409 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:39.300908  8409 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.301371  8409 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:39.302024  8181 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:39.184731  8405 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:17:39.232683  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 53655.2us
I20260709 12:17:39.229863  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:39.304831  8181 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:39.305536  8181 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:39.305786  6548 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:39.227841  6576 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 4.
W20260709 12:17:39.242228  6584 consensus_meta.cc:287] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.518s	user 0.012s	sys 0.000s
I20260709 12:17:39.306703  6584 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:39.307027  6548 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:39.238992  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:39.233963  8398 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:39.297654  8161 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:39.297587  8064 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:39.305869  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:39.278906  8331 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:39.318205  6582 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:17:39.205442  8413 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:39.311550  6577 raft_consensus.cc:2410] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:39.216557  8252 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:39.314340  6651 raft_consensus.cc:2393] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 4: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:39.233965  8308 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:39.180078  8400 consensus_queue.cc:260] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:39.310266  6579 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:39.306013  8181 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:39.337721  8064 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:39.323508  8426 log.cc:927] Time spent T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 12:17:39.237000  8274 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:39.198130  8402 consensus_queue.cc:237] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:39.312650  8428 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:17:39.308756  8366 log.cc:927] Time spent T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 1.694s	user 0.013s	sys 0.011s
I20260709 12:17:39.392442  8428 raft_consensus.cc:697] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:39.286491  8433 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.304033  8409 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:39.400754  8243 raft_consensus.cc:3055] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:39.401384  8243 raft_consensus.cc:740] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:39.402546  8243 consensus_queue.cc:260] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:39.403306  8398 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:39.393862  8428 consensus_queue.cc:237] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:39.290964  8093 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:39.322465  8427 log.cc:927] Time spent T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:17:39.348953  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:39.407811  6651 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:39.409057  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:39.351383  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:39.358865  8308 raft_consensus.cc:697] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:17:39.359130  8374 log.cc:927] Time spent T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 1.733s	user 0.022s	sys 0.029s
I20260709 12:17:39.360280  8287 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:39.360545  8181 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:39.338032  8331 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:39.316250  8405 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:39.362267  8029 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:39.364344  8390 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:39.423753  8390 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.424451  8390 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:39.425825  8405 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:39.426965  8390 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:39.403954  8243 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:39.366257  8421 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:39.428951  8421 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:39.429657  8421 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:39.432963  8421 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:39.371116  8413 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:39.433310  8067 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:39.373237  8338 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:39.306911  8371 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:39.434019  8067 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:39.381214  8179 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:39.381193  8274 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:39.385306  8280 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:39.435469  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:39.388887  8252 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:39.389004  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:39.403887  8227 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:39.368011  8335 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:39.412945  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1412) took 1864 ms. Trace:
I20260709 12:17:39.438244  8227 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:39.438736  6583 rpcz_store.cc:276] 0709 12:17:37.548455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.549035 (+   580us) service_pool.cc:224] Handling call
0709 12:17:37.549457 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:37.680673 (+131216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:39.412649 (+1731976us) spinlock_profiling.cc:243] Waited 74.9 ms on lock 0x7b54003abe54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:39.412872 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":172189440}
I20260709 12:17:39.439711  7271 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:17:39.440280  6583 raft_consensus.cc:1240] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:39.420661  6654 raft_consensus.cc:1240] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 2. Current term is 3. Ops: []
W20260709 12:17:39.365948  6649 consensus_meta.cc:287] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.709s	user 0.011s	sys 0.000s
I20260709 12:17:39.444566  6649 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:17:39.445547  8331 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:39.445550  8117 raft_consensus.cc:2764] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:17:39.326417  8161 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:39.444873  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:39.415529  8308 consensus_queue.cc:237] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:39.388062  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:39.443143  8192 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:39.394285  8433 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:39.403887  8398 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:39.472672  6658 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:39.932982  8383 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:39.933651  6657 raft_consensus.cc:1275] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:39.935509  6658 raft_consensus.cc:1240] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:39.512550  8227 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:17:39.654693  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:39.867172  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 1481) took 1130 ms. Trace:
W20260709 12:17:39.638293  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 524255us
I20260709 12:17:39.852800  6575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1410) took 2422 ms. Trace:
W20260709 12:17:39.472553  8386 log.cc:927] Time spent T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 1.439s	user 0.022s	sys 0.001s
I20260709 12:17:39.573717  8412 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:39.940276  8344 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:39.450964  8067 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:17:39.638891  8305 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.020s	sys 0.029s
W20260709 12:17:39.941509  8305 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.024s	sys 0.036s
I20260709 12:17:39.941083  8344 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:39.829989  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 1529) took 1172 ms. Trace:
I20260709 12:17:39.942365  6507 rpcz_store.cc:276] 0709 12:17:38.657661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.659403 (+  1742us) service_pool.cc:224] Handling call
0709 12:17:39.829714 (+1170311us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b54002c47d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:39.829902 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":495541504}
I20260709 12:17:39.583700  6508 raft_consensus.cc:1275] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 2. (index mismatch)
I20260709 12:17:39.581571  8400 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:39.943924  8344 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:39.944643  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 1479) took 1285 ms. Trace:
I20260709 12:17:39.775112  8375 raft_consensus.cc:1064] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
W20260709 12:17:39.945545  8375 raft_consensus.cc:1068] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65: Unable to promote non-voter 20a202c76244485fbc8f1610473db8b9: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:17:39.946215  8176 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:39.574375  8243 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:17:39.768025  8303 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.063s	sys 0.055s
W20260709 12:17:39.754290  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:39.947141  8243 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:39.721915  8282 consensus_meta.cc:287] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.540s	user 0.011s	sys 0.000s
I20260709 12:17:39.523399  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1413) took 1952 ms. Trace:
I20260709 12:17:39.948474  8282 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:39.776988  8436 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:39.937184  6501 rpcz_store.cc:276] 0709 12:17:38.736474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.781183 (+ 44709us) service_pool.cc:224] Handling call
0709 12:17:38.781505 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:39.237906 (+456401us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:39.866849 (+628943us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b540046cdd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:39.867094 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":940119296}
W20260709 12:17:39.640446  6475 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.027s	sys 0.028s
W20260709 12:17:39.968297  6475 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.988s	user 0.032s	sys 0.028s
W20260709 12:17:39.969415  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 992840us
I20260709 12:17:39.948889  6578 rpcz_store.cc:276] 0709 12:17:37.571009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.571333 (+   324us) service_pool.cc:224] Handling call
0709 12:17:37.574632 (+  3299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:38.665272 (+1090640us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:39.523087 (+857815us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54000573d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:39.523323 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2500559104}
I20260709 12:17:39.512583  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:39.949787  6622 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:39.511921  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:39.973177  6656 raft_consensus.cc:2410] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:39.973609  6650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 1140) took 1241 ms. Trace:
W20260709 12:17:39.937651  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:39.975556  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:39.945122  6508 rpcz_store.cc:276] 0709 12:17:38.658931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.659187 (+   256us) service_pool.cc:224] Handling call
0709 12:17:39.583241 (+924054us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:39.943510 (+360269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:39.944453 (+   943us) spinlock_profiling.cc:243] Waited 894 ms on lock 0x7b540039e250. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:39.944586 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2055531776}
I20260709 12:17:39.977128  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:39.978246  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 1139) took 1250 ms. Trace:
I20260709 12:17:39.978543  6658 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 1.
W20260709 12:17:39.508452  8354 log.cc:927] Time spent T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 1.881s	user 0.005s	sys 0.019s
I20260709 12:17:39.824285  6649 raft_consensus.cc:3055] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:39.865394  8372 raft_consensus.cc:3055] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:39.938109  7240 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:39.473188  7267 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:17:39.778808  7898 raft_consensus.cc:1064] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
W20260709 12:17:39.979689  6621 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.781s	user 0.029s	sys 0.076s
W20260709 12:17:39.947265  8303 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.134s	user 0.064s	sys 0.061s
I20260709 12:17:39.978756  6656 rpcz_store.cc:276] 0709 12:17:38.727539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.975413 (+247874us) service_pool.cc:224] Handling call
0709 12:17:39.512454 (+537041us) spinlock_profiling.cc:243] Waited 2.68 ms on lock 0x7b5800015998. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:17:39.978185 (+465731us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6170880}
I20260709 12:17:39.983927  6649 raft_consensus.cc:740] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:39.939329  6575 rpcz_store.cc:276] 0709 12:17:37.430139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.441323 (+ 11184us) service_pool.cc:224] Handling call
0709 12:17:37.441610 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:37.568577 (+126967us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:37.568624 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:37.569196 (+   572us) log.cc:844] Serialized 26 byte log entry
0709 12:17:37.753717 (+184521us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:37.798681 (+ 44964us) log.cc:844] Serialized 14 byte log entry
0709 12:17:38.305949 (+507268us) spinlock_profiling.cc:243] Waited 1.36 ms on lock 0x7b18000ff520. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:17:38.531565 (+225616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:39.852310 (+1320745us) spinlock_profiling.cc:243] Waited 66.3 ms on lock 0x7b54004204d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:39.852412 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:39.852479 (+    67us) raft_consensus.cc:1685] finished
0709 12:17:39.852497 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:39.852687 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1790182,"prepare.queue_time_us":52761,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":932,"raft.queue_time_us":801664,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":113666,"spinlock_wait_cycles":155558400,"thread_start_us":145421,"threads_started":1,"wal-append.queue_time_us":342075,"wal-append.run_cpu_time_us":2337,"wal-append.run_wall_time_us":1043650}
W20260709 12:17:39.473052  7269 consensus_meta.cc:287] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.712s	user 0.010s	sys 0.002s
I20260709 12:17:39.518806  8411 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:17:39.988112  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 50092.2us
I20260709 12:17:39.974152  6650 rpcz_store.cc:276] 0709 12:17:38.732459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.977877 (+245418us) service_pool.cc:224] Handling call
0709 12:17:39.511749 (+533872us) spinlock_profiling.cc:243] Waited 19.8 ms on lock 0x7b5800015998. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:17:39.973514 (+461765us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":45607808}
I20260709 12:17:39.988704  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:39.855216  8180 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:17:39.991659  6621 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.793s	user 0.029s	sys 0.076s
W20260709 12:17:39.994060  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 44389.1us
I20260709 12:17:39.977753  8344 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:39.990207  8332 consensus_meta.cc:287] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.699s	user 0.010s	sys 0.002s
I20260709 12:17:40.006352  6622 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:17:40.016952  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:39.986860  8364 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:40.024816  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:40.030100  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:40.030947  6658 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:40.033847  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:40.036742  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:39.987609  8372 raft_consensus.cc:740] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:40.044610  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:40.000315  7898 raft_consensus.cc:1068] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65: Unable to promote non-voter 20a202c76244485fbc8f1610473db8b9: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:17:39.992219  8252 raft_consensus.cc:2804] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:40.051383  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:40.052968  6475 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:40.055951  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:40.056766  6650 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:39.999607  8375 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:40.054922  6657 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5.
I20260709 12:17:40.002719  7269 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
W20260709 12:17:40.063994  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 57752.7us
I20260709 12:17:40.022107  7271 raft_consensus.cc:1240] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:40.065016  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:40.047405  6649 consensus_queue.cc:260] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:17:39.981024  8436 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:40.046325  8332 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:40.075543  6649 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:40.078083  6622 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:40.080165  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:40.058655  8373 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:40.077252  7240 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:40.055588  8252 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:40.064028  8180 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:17:40.093976  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:40.087531  6475 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:17:40.101074  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:40.101706  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:40.072825  7269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38562 (request call id 403) took 1552 ms. Trace:
I20260709 12:17:40.106035  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:40.107633  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:40.108675  6657 raft_consensus.cc:2410] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:17:40.044835  8344 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:40.103379  7269 rpcz_store.cc:276] 0709 12:17:38.517772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.519638 (+  1866us) service_pool.cc:224] Handling call
0709 12:17:40.070397 (+1550759us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:40.082990  8399 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:40.112645  6622 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:40.112797  8399 consensus_queue.cc:1059] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:40.113201  8398 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:40.114333  6622 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:40.114709  8398 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:40.115633  6652 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:40.116871  8412 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:40.086705  8435 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:40.118076  8435 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:40.118593  8435 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:40.113699  6403 catalog_manager.cc:5697] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 1 to 3, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 3 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:40.083278  8117 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:40.064085  8067 consensus_queue.cc:1059] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 12:17:40.126590  6652 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
W20260709 12:17:40.128173  6548 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.820s	user 0.013s	sys 0.047s
W20260709 12:17:40.133904  6548 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.826s	user 0.014s	sys 0.047s
W20260709 12:17:40.134472  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 827421us
I20260709 12:17:40.104307  8252 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:40.095721  8180 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:40.110349  8036 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:40.094714  8178 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 12:17:40.102437  6649 raft_consensus.cc:1275] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 2. (term mismatch)
I20260709 12:17:40.149318  6649 pending_rounds.cc:85] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 1
I20260709 12:17:40.149717  6649 pending_rounds.cc:107] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 4.2
I20260709 12:17:40.150099  6649 raft_consensus.cc:2889] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:17:40.154682  6550 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.867s	user 0.002s	sys 0.040s
W20260709 12:17:40.155301  6550 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.867s	user 0.003s	sys 0.040s
I20260709 12:17:40.069285  8375 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:40.132834  8373 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:40.135658  6548 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:40.169242  6548 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:40.170457  6548 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:40.109364  8345 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:17:40.132642  6473 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.920s	user 0.015s	sys 0.054s
W20260709 12:17:40.173527  6473 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.963s	user 0.016s	sys 0.054s
W20260709 12:17:40.173962  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:40.110064  8364 consensus_queue.cc:1059] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
W20260709 12:17:40.156304  8093 consensus_meta.cc:287] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.733s	user 0.009s	sys 0.002s
I20260709 12:17:40.135138  6406 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57914 (request call id 188) took 5410 ms. Trace:
W20260709 12:17:40.193353  6653 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.914s	user 0.013s	sys 0.026s
I20260709 12:17:40.170639  8243 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:17:40.193795  6653 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.915s	user 0.013s	sys 0.026s
I20260709 12:17:40.161453  6550 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:40.138747  8180 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:40.139500  8367 raft_consensus.cc:3055] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:17:40.135747  7238 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.911s	user 0.012s	sys 0.032s
I20260709 12:17:40.139889  8117 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:40.127210  6403 catalog_manager.cc:5697] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:17:40.229712  6403 catalog_manager.cc:5697] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 2 to 3, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 3 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:40.176302  8345 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:40.173367  8332 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:40.176599  8282 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:17:40.176189  8266 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.245s	user 0.018s	sys 0.025s
I20260709 12:17:40.180859  8067 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:40.181404  6473 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:40.139761  8036 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:40.182955  8291 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:40.179966  6548 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
W20260709 12:17:40.244647  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 63342.9us
W20260709 12:17:40.244915  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 65020.4us
I20260709 12:17:40.195683  8398 consensus_queue.cc:1059] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:17:40.203361  8192 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:40.205135  8433 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:40.207628  8372 consensus_queue.cc:260] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:40.209534  8436 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:40.192850  6406 rpcz_store.cc:276] 0709 12:17:34.725239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:34.729939 (+  4700us) service_pool.cc:224] Handling call
0709 12:17:40.135035 (+5405096us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:37.615160 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:37.615832 (+   672us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:37.615872 (+    40us) write_op.cc:435] Acquired schema lock
0709 12:17:37.615884 (+    12us) tablet.cc:662] Decoding operations
0709 12:17:37.617452 (+  1568us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:37.617516 (+    64us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:37.617551 (+    35us) tablet.cc:685] Acquiring locks for 17 operations
0709 12:17:38.005269 (+387718us) tablet.cc:701] Row locks acquired
0709 12:17:38.012412 (+  7143us) write_op.cc:260] PREPARE: finished
0709 12:17:38.028819 (+ 16407us) write_op.cc:270] Start()
0709 12:17:38.042926 (+ 14107us) write_op.cc:276] Timestamp: P: 1783599458024430 usec, L: 0
0709 12:17:38.044467 (+  1541us) op_driver.cc:348] REPLICATION: starting
0709 12:17:38.064492 (+ 20025us) log.cc:844] Serialized 6923 byte log entry
0709 12:17:38.805438 (+740946us) op_driver.cc:464] REPLICATION: finished
0709 12:17:39.235473 (+430035us) write_op.cc:301] APPLY: starting
0709 12:17:39.235572 (+    99us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:39.310810 (+ 75238us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:39.349572 (+ 38762us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:39.381927 (+ 32355us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:39.382285 (+   358us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=17
0709 12:17:39.382360 (+    75us) write_op.cc:312] APPLY: finished
0709 12:17:39.382910 (+   550us) log.cc:844] Serialized 119 byte log entry
0709 12:17:39.383149 (+   239us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:39.383389 (+   240us) write_op.cc:454] Released schema lock
0709 12:17:39.383468 (+    79us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:39.383874 (+   406us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:39.383902 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":236028,"reported_tablets":24,"child_traces":[["op",{"apply.queue_time_us":419618,"apply.run_cpu_time_us":5950,"apply.run_wall_time_us":148402,"num_ops":17,"prepare.queue_time_us":167026,"prepare.run_cpu_time_us":29401,"prepare.run_wall_time_us":662119,"raft.queue_time_us":314763,"raft.run_cpu_time_us":38970,"raft.run_wall_time_us":398650,"replication_time_us":687633,"thread_start_us":701670,"threads_started":4,"wal-append.queue_time_us":279960}]]}
I20260709 12:17:40.197855  6550 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:40.214186  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:40.305721  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:40.218103  8243 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:40.222597  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:17:40.227257  7238 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.002s	user 0.012s	sys 0.032s
W20260709 12:17:40.307152  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 1.10554e+06us
I20260709 12:17:40.308558  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:40.232205  6403 catalog_manager.cc:5697] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 1 to 3, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 3 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:40.240025  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:40.246433  6473 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:40.311278  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 64922.1us
W20260709 12:17:40.295560  6622 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.625s (SENT) [suppressed 11 similar messages]
I20260709 12:17:40.312114  6473 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:40.313659  6473 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:40.190224  8093 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:40.248559  8036 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:40.297382  8291 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:40.238353  8345 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:40.222337  8375 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:40.279222  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58980 (request call id 1633) took 1703 ms. Trace:
W20260709 12:17:40.299562  6474 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.062s	user 0.017s	sys 0.032s
W20260709 12:17:40.241353  8266 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.314s	user 0.019s	sys 0.026s
I20260709 12:17:40.227150  8367 raft_consensus.cc:740] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:40.317135  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:40.318924  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:40.319936  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:40.336977  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:40.303197  8344 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:40.235865  8346 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:40.297806  8178 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:17:40.317636  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 119881us
I20260709 12:17:40.238688  8117 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:40.325069  6655 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.824s	user 0.033s	sys 0.013s
I20260709 12:17:40.325551  6473 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:40.353250  6473 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:40.354524  6473 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:40.300848  8067 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:40.356642  6579 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.940s	user 0.014s	sys 0.038s
I20260709 12:17:40.330590  6652 rpcz_store.cc:276] 0709 12:17:38.527103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.532418 (+  5315us) service_pool.cc:224] Handling call
0709 12:17:40.230266 (+1697848us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b540008f7d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:40.230445 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3638953728}
I20260709 12:17:40.315994  6403 catalog_manager.cc:5697] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 1 to 3, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 3 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:17:40.359843  6579 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.943s	user 0.015s	sys 0.038s
W20260709 12:17:40.323196  8307 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.875s	user 0.016s	sys 0.032s
I20260709 12:17:40.341006  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:40.341777  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:40.317819  8433 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:40.342664  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:40.343562  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:40.344717  8429 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:40.298364  8272 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:17:40.345412  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:40.366263  6657 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:40.323468  8372 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:40.346169  6651 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:40.348227  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:40.316936  8192 raft_consensus.cc:697] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:40.322485  8227 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:40.324920  8190 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:17:40.361217  8402 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.948s	user 0.010s	sys 0.043s
I20260709 12:17:40.350519  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:40.391137  6652 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:17:40.341032  7237 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.438s	user 0.012s	sys 0.049s
W20260709 12:17:40.330271  6474 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.093s	user 0.017s	sys 0.032s
W20260709 12:17:40.395206  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 1.15754e+06us
I20260709 12:17:40.333787  7898 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:40.360020  8375 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:40.316622  8243 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:40.358282  6473 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:40.357362  6655 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.859s	user 0.033s	sys 0.013s
I20260709 12:17:40.362727  6508 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 1.
I20260709 12:17:40.360958  6403 catalog_manager.cc:5697] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:40.352998  8178 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:40.398351  6403 catalog_manager.cc:5697] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 2 to 4, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 4 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:40.373342  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:40.327327  8291 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:40.301585  8323 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.994s	user 0.055s	sys 0.116s
I20260709 12:17:40.374404  8101 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:40.375659  8307 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.927s	user 0.016s	sys 0.033s
W20260709 12:17:40.398098  7237 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.495s	user 0.013s	sys 0.049s
I20260709 12:17:40.358372  8438 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:40.362739  8408 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:40.406834  6403 catalog_manager.cc:5697] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 2 to 3, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 3 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:40.353536  8346 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:40.363709  8428 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.957s	user 0.015s	sys 0.038s
W20260709 12:17:40.432510  8428 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.026s	user 0.015s	sys 0.038s
I20260709 12:17:40.433200  7268 raft_consensus.cc:1240] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:40.433640  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:40.435072  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:40.435770  7269 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 1.
I20260709 12:17:40.438261  8192 consensus_queue.cc:237] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:40.444538  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:40.446220  8445 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:40.446904  8445 raft_consensus.cc:697] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:40.447402  8117 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:40.435791  8252 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:40.449352  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:40.451834  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:40.452659  7272 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:40.389559  8190 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:40.392208  8402 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.979s	user 0.010s	sys 0.043s
I20260709 12:17:40.372399  8429 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:40.413597  6656 raft_consensus.cc:2410] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:17:40.417371  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:40.417098  6549 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:17:40.422034  6403 catalog_manager.cc:5697] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 2 to 3, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 3 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:40.418358  8101 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:40.425354  7266 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:40.407454  8178 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:40.455000  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:40.411967  8323 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.103s	user 0.055s	sys 0.116s
I20260709 12:17:40.471515  7272 raft_consensus.cc:1240] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 1. Current term is 2. Ops: []
W20260709 12:17:40.455372  7793 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.040s	user 0.025s	sys 0.029s
I20260709 12:17:40.417620  7898 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:40.481978  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:40.494587  8067 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:40.497645  8243 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:40.447886  8445 consensus_queue.cc:237] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:40.499372  8346 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:40.499931  8346 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:40.420437  8408 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:40.505337  8036 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:40.506294  8408 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:40.508589  7273 raft_consensus.cc:1240] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6. Ops: []
W20260709 12:17:40.508406  7793 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.093s	user 0.026s	sys 0.029s
I20260709 12:17:40.510896  8336 raft_consensus.cc:3055] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:40.511312  8336 raft_consensus.cc:740] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:40.512063  8336 consensus_queue.cc:260] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:40.513708  8291 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:40.514338  8291 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:40.515965  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:40.490401  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:40.518085  6509 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 4.
I20260709 12:17:40.462148  8190 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:40.469731  8429 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:40.463464  8428 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:40.517642  8176 consensus_meta.cc:287] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.540s	user 0.011s	sys 0.000s
I20260709 12:17:40.477522  8323 raft_consensus.cc:697] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:40.530881  8036 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:40.531261  8291 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:40.579121  8252 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:40.580773  7268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38558 (request call id 328) took 2052 ms. Trace:
I20260709 12:17:40.581233  7268 rpcz_store.cc:276] 0709 12:17:38.528822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.529419 (+   597us) service_pool.cc:224] Handling call
0709 12:17:39.821096 (+1291677us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b5800125198. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:17:39.821409 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:40.433592 (+612183us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:40.580500 (+146908us) spinlock_profiling.cc:243] Waited 611 ms on lock 0x7b54002e9fd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:40.580705 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1408678528}
W20260709 12:17:40.575233  8440 log.cc:927] Time spent T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 12:17:40.589905  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:40.596186  6650 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 4: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:40.504155  8180 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:40.589000  6654 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.015s	sys 0.035s
I20260709 12:17:40.534370  8411 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 12:17:40.537375  8428 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:40.615190  6654 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.016s	sys 0.035s
I20260709 12:17:40.574843  8093 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:17:40.614337  8308 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.017s	sys 0.025s
I20260709 12:17:40.575639  8176 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:40.531194  8367 consensus_queue.cc:260] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:40.608065  8291 raft_consensus.cc:697] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:17:40.643808  8308 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.017s	sys 0.026s
W20260709 12:17:40.673408  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:40.684922  6658 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.010s	sys 0.010s
W20260709 12:17:40.688529  6658 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.010s	sys 0.010s
I20260709 12:17:40.685117  8367 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:40.658243  8176 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:40.684454  8332 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:40.711551  8399 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.015s	sys 0.026s
W20260709 12:17:40.732220  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 486492us
I20260709 12:17:40.726501  8243 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:17:40.745528  8399 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.016s	sys 0.026s
I20260709 12:17:40.763386  6548 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:40.754710  8243 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:40.782323  8243 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:17:40.753552  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 401587us
W20260709 12:17:40.777942  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 482372us
I20260709 12:17:40.753712  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:40.687356  8323 consensus_queue.cc:237] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:40.783586  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 366546us
I20260709 12:17:40.800652  7238 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:40.804456  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:40.805248  6501 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:40.803807  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 402562us
I20260709 12:17:40.806599  6473 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:40.826921  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:40.827545  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:40.827903  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:40.828737  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:40.802227  8450 log.cc:927] Time spent T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
I20260709 12:17:40.760156  8373 consensus_queue.cc:1059] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 12:17:40.827302  8036 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:40.845970  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:40.751641  8291 consensus_queue.cc:237] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:40.833381  8180 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:40.850440  6473 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:40.848243  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1144) took 1935 ms. Trace:
I20260709 12:17:40.834930  8375 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:40.863132  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:40.863654  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:40.855675  6655 rpcz_store.cc:276] 0709 12:17:38.876219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:39.457117 (+580898us) service_pool.cc:224] Handling call
0709 12:17:39.478422 (+ 21305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:39.478685 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:17:40.397072 (+918387us) log.cc:844] Serialized 14 byte log entry
0709 12:17:40.804564 (+407492us) log.cc:844] Serialized 14 byte log entry
0709 12:17:40.804816 (+   252us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:40.804969 (+   153us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:40.805019 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:40.805079 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:40.805184 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1101970,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":1874,"raft.queue_time_us":4001,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":21692,"spinlock_wait_cycles":97200640,"thread_start_us":1302044,"threads_started":2}
I20260709 12:17:40.864120  6507 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 3.
W20260709 12:17:40.876044  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 68097.6us
I20260709 12:17:40.886337  6579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 130) took 1809 ms. Trace:
I20260709 12:17:40.886878  6579 rpcz_store.cc:276] 0709 12:17:39.077173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:39.374830 (+297657us) service_pool.cc:224] Handling call
0709 12:17:39.375115 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:39.378828 (+  3713us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:17:40.360982 (+982154us) log.cc:844] Serialized 14 byte log entry
0709 12:17:40.885255 (+524273us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:40.885314 (+    59us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:40.885375 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:40.886115 (+   740us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:40.886267 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6185,"prepare.queue_time_us":1209857,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":65,"thread_start_us":1458352,"threads_started":3}
I20260709 12:17:40.850258  8428 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:40.850756  8036 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:40.899330  8178 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:40.891559  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:40.900415  6650 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:40.871913  7793 consensus_queue.cc:1059] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 12:17:40.902035  7272 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:17:40.902311  8307 consensus_queue.cc:1059] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:40.903241  8307 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:40.906528  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:40.907248  7274 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:40.908561  8227 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:40.909633  8282 consensus_queue.cc:1059] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:40.910308  6651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 1137) took 2232 ms. Trace:
I20260709 12:17:40.910712  6651 rpcz_store.cc:276] 0709 12:17:38.677650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:39.408708 (+731058us) service_pool.cc:224] Handling call
0709 12:17:40.910147 (+1501439us) spinlock_profiling.cc:243] Waited 936 ms on lock 0x7b540044a2d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:40.910255 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2153248640}
W20260709 12:17:40.911700  8345 consensus_meta.cc:287] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.576s	user 0.008s	sys 0.001s
I20260709 12:17:40.912319  8346 raft_consensus.cc:3055] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:40.912719  8346 raft_consensus.cc:740] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:40.891140  6653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1128) took 2342 ms. Trace:
I20260709 12:17:40.899629  8036 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:40.911638  8398 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:40.915702  6650 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
W20260709 12:17:40.895151  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 91377.4us
I20260709 12:17:40.921762  6501 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5.
I20260709 12:17:40.913604  6653 rpcz_store.cc:276] 0709 12:17:38.548918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.549046 (+   128us) service_pool.cc:224] Handling call
0709 12:17:38.549430 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:39.205630 (+656200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:39.205676 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:39.207613 (+  1937us) log.cc:844] Serialized 26 byte log entry
0709 12:17:40.194345 (+986732us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:40.194401 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:40.246250 (+ 51849us) spinlock_profiling.cc:243] Waited 454 ms on lock 0x7b54001509d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:40.246302 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:40.872242 (+625940us) raft_consensus.cc:1685] finished
0709 12:17:40.872265 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:40.891034 (+ 18769us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8691,"raft.queue_time_us":189632,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":18325,"spinlock_wait_cycles":1045002112,"thread_start_us":1107842,"threads_started":2,"wal-append.queue_time_us":1237868}
W20260709 12:17:40.935609  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:40.954020  6654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1145) took 2031 ms. Trace:
I20260709 12:17:40.871004  8441 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:40.955894  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:40.956961  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:17:40.961584  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:40.962218  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:40.954463  6654 rpcz_store.cc:276] 0709 12:17:38.922772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:39.458072 (+535300us) service_pool.cc:224] Handling call
0709 12:17:39.933265 (+475193us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:39.933467 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:17:40.633154 (+699687us) log.cc:844] Serialized 14 byte log entry
0709 12:17:40.953676 (+320522us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:40.953729 (+    53us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:40.953769 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:40.953850 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:40.953964 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70308,"prepare.queue_time_us":708249,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":43,"thread_start_us":949062,"threads_started":2}
I20260709 12:17:40.914271  8344 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:40.903942  8429 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:40.913013  8400 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:40.889621  8470 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:40.853806  8451 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.111s
W20260709 12:17:40.991119  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 57510.3us
W20260709 12:17:40.998850  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 197367us
I20260709 12:17:40.999465  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.000207  6506 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:17:41.000762  6548 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:41.002017  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:40.970389  8455 log.cc:927] Time spent T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.261s	user 0.000s	sys 0.001s
I20260709 12:17:41.002806  6506 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:41.006188  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.011000  6501 raft_consensus.cc:1240] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 4. Ops: []
I20260709 12:17:41.008373  6548 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:40.960093  8398 raft_consensus.cc:697] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:40.960184  8176 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:40.908190  8408 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:40.995097  8452 log.cc:927] Time spent T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.263s	user 0.001s	sys 0.001s
I20260709 12:17:40.912272  8345 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:40.986696  8367 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:41.032727  6550 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:41.040740  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:41.047078  6507 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:41.033037  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.044147  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.011443  8291 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
W20260709 12:17:41.041836  8466 log.cc:927] Time spent T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:17:40.994614  8346 consensus_queue.cc:260] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:41.069115  8346 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:41.068955  8483 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:41.071799  8483 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:41.045251  8332 consensus_queue.cc:1059] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 12:17:41.040982  8117 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:41.072712  8117 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:41.073801  8408 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:41.074484  8408 raft_consensus.cc:3055] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:41.074872  8408 raft_consensus.cc:740] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:41.075289  8117 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:41.062160  6584 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:41.062592  8180 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.058270  8442 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:41.077759  7266 raft_consensus.cc:2393] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 5: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:17:41.077085  8180 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:41.062217  6502 raft_consensus.cc:1240] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:41.066004  8308 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:41.067529  6550 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:41.077196  8442 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:41.057978  8460 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:40.995868  8482 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:41.008934  8465 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:41.036134  8470 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:41.105190  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:41.104103  6550 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:41.102632  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:41.107362  6575 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:17:41.110589  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:41.111325  6575 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:41.063732  8367 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:41.005508  8445 consensus_queue.cc:1059] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.080s
I20260709 12:17:41.079681  8180 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:40.970664  8441 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:41.004410  8400 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:41.112250  6507 raft_consensus.cc:2410] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20260709 12:17:41.151130  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:41.169425  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:41.170140  6508 raft_consensus.cc:2368] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 5. Current term is 6.
I20260709 12:17:41.170009  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:41.116951  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.170663  6658 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
W20260709 12:17:41.173821  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 323389us
I20260709 12:17:41.178607  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.084232  8485 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:41.152204  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:41.180883  6657 raft_consensus.cc:2393] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 4: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:41.142046  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:41.132310  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 57884.1us
I20260709 12:17:41.122066  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:41.155861  8398 consensus_queue.cc:237] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:41.156481  8372 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.158133  8178 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:41.184316  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:41.185084  6575 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:17:41.158967  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.186002  6503 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
W20260709 12:17:41.162350  8491 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:17:41.080157  8411 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.167448  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:41.167636  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:41.168301  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:41.171779  6505 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:17:41.114023  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:41.164659  8470 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:41.143096  8493 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:41.113410  8308 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:41.190232  6576 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 6: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:41.191635  6657 raft_consensus.cc:1240] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:41.198616  6550 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:41.200984  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.201776  6508 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:17:41.203333  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.203948  6505 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 1.
I20260709 12:17:41.176599  8375 raft_consensus.cc:697] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:41.204977  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:41.205313  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:41.188469  8451 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.099151  8460 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:41.190222  8494 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:41.195348  8470 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:41.198734  8117 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:41.104455  8482 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:41.256075  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:41.256430  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.114383  8465 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:41.143710  8408 consensus_queue.cc:260] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:41.260934  8435 consensus_meta.cc:287] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 1.141s	user 0.010s	sys 0.006s
I20260709 12:17:41.270217  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:41.271934  6581 raft_consensus.cc:1240] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:41.190989  8372 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:41.249164  8488 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.260803  8470 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:41.273017  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:41.303494  6474 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:17:41.278887  8474 log.cc:927] Time spent T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 12:17:41.279213  8467 log.cc:927] Time spent T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:17:41.261462  8494 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:41.259925  8308 raft_consensus.cc:697] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:41.267299  8497 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:41.281699  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.273861  6576 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
W20260709 12:17:41.299141  8190 consensus_meta.cc:287] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.763s	user 0.013s	sys 0.000s
I20260709 12:17:41.280840  8429 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:41.292472  6578 raft_consensus.cc:2468] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
I20260709 12:17:41.295253  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:41.277155  6505 raft_consensus.cc:1240] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:41.302301  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 84324.7us
I20260709 12:17:41.278192  8435 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:41.271004  8483 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:41.259387  8291 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:17:41.267768  8117 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:41.205214  8493 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:41.196552  8411 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:41.308151  6474 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:17:41.306133  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 107565us
I20260709 12:17:41.282713  8345 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:41.312678  6579 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:41.309082  8307 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:41.317178  6622 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:41.280911  6582 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:41.324376  6656 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:41.259119  8451 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:41.307135  8408 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:41.317351  8468 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:41.322347  8445 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:41.326789  8460 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:41.327047  8435 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:41.288955  8488 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:41.337656  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:41.342108  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:41.338569  7238 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:41.313692  8484 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:41.354097  7238 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:41.317014  8483 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:41.304663  8498 consensus_queue.cc:1059] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 12:17:41.359894  6579 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:17:41.360526  6579 raft_consensus.cc:606] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
W20260709 12:17:41.344044  8477 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.282s	user 0.000s	sys 0.002s
I20260709 12:17:41.360888  6579 raft_consensus.cc:614] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:41.362928  6658 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:41.361172  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:41.343372  8451 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:41.344573  8460 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:41.317727  8190 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:41.364789  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:41.340288  8442 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:41.328645  8500 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:41.362462  8464 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:41.337005  8441 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:41.385389  8470 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:41.385774  8504 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:41.386195  8411 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:41.386420  8504 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:41.387161  8308 consensus_queue.cc:237] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:41.388948  8493 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:41.353454  8307 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:41.335029  8468 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:41.332350  8101 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:41.331897  8117 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:41.370854  8506 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:17:41.342790  8478 log.cc:927] Time spent T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.276s	user 0.000s	sys 0.002s
I20260709 12:17:41.359154  8483 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:41.380797  8346 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:41.380043  8367 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:41.383608  8190 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:41.387602  8470 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:41.371083  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 53976.3us
W20260709 12:17:41.390959  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:41.364229  8488 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:41.395730  8506 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:41.321094  8497 consensus_queue.cc:1059] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
W20260709 12:17:41.372484  8439 log.cc:927] Time spent T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.970s	user 0.009s	sys 0.006s
I20260709 12:17:41.376468  8446 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:41.410734  8506 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:41.343703  8445 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:41.411334  8497 raft_consensus.cc:3055] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:41.411741  8497 raft_consensus.cc:740] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:41.411861  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1153) took 2680 ms. Trace:
W20260709 12:17:41.412827  8486 log.cc:927] Time spent T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:17:41.412876  8445 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:41.377035  8460 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:41.413841  8460 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:41.406344  8346 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:41.418540  8346 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:41.394724  6649 consensus_queue.cc:237] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:41.322628  8375 consensus_queue.cc:237] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:41.421471  8503 log.cc:927] Time spent T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:17:41.382628  8499 consensus_queue.cc:1059] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 12:17:41.380236  8400 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:41.381230  8500 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:41.381680  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:41.393285  8468 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:41.407064  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:41.396466  8462 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:41.412273  6580 rpcz_store.cc:276] 0709 12:17:38.731399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:38.731848 (+   449us) service_pool.cc:224] Handling call
0709 12:17:38.732151 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:38.982721 (+250570us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:38.982771 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:39.005548 (+ 22777us) log.cc:844] Serialized 26 byte log entry
0709 12:17:39.611000 (+605452us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:39.611061 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:39.611159 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:41.304836 (+1693677us) spinlock_profiling.cc:243] Waited 6.91 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:17:41.411582 (+106746us) raft_consensus.cc:1685] finished
0709 12:17:41.411604 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:41.411800 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":116008,"raft.queue_time_us":242139,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":28339,"spinlock_wait_cycles":15891328,"thread_start_us":497495,"threads_started":1,"wal-append.queue_time_us":1247320}
I20260709 12:17:41.428211  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1163) took 1243 ms. Trace:
I20260709 12:17:41.440222  6652 rpcz_store.cc:276] 0709 12:17:40.184954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.394269 (+209315us) service_pool.cc:224] Handling call
0709 12:17:40.394597 (+   328us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:40.404358 (+  9761us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:40.404401 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:40.410789 (+  6388us) log.cc:844] Serialized 52 byte log entry
0709 12:17:40.883436 (+472647us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:40.883488 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:40.883570 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:41.343426 (+459856us) spinlock_profiling.cc:243] Waited 19.5 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:17:41.423845 (+ 80419us) raft_consensus.cc:1685] finished
0709 12:17:41.423875 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:41.428126 (+  4251us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26588,"raft.queue_time_us":10031,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":4182,"spinlock_wait_cycles":44844288,"thread_start_us":396110,"threads_started":1,"wal-append.queue_time_us":531525}
I20260709 12:17:41.437574  6622 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:41.443639  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:41.359351  8484 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:41.402988  8504 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:41.447391  6622 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:17:41.448736  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 88483.3us
I20260709 12:17:41.451076  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:41.451891  6654 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:41.453101  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:41.454012  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:41.458262  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.458994  6505 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:41.423839  8465 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:41.459601  8498 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:41.460036  8498 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:41.460294  8307 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:41.460489  8498 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:41.460731  8441 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:41.461189  8441 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:41.461555  8441 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:41.445338  8400 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:41.464022  8400 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:41.464380  8400 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:41.465806  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:41.466785  8036 consensus_meta.cc:287] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.505s	user 0.008s	sys 0.000s
I20260709 12:17:41.467094  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.467209  8036 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:41.447361  8117 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:41.468595  7268 raft_consensus.cc:1240] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 1. Current term is 2. Ops: [1.4-1.4]
I20260709 12:17:41.468667  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.447903  8462 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:41.469463  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.470867  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.471262  8462 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:41.471477  6505 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
I20260709 12:17:41.460399  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:41.469522  6502 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 3: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:41.476784  8494 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:41.478233  8483 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:41.478454  8441 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:41.476218  8400 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:41.479800  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.438916  8190 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.448092  8497 consensus_queue.cc:260] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:41.480475  8441 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:41.431777  8500 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:41.460036  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.481302  8465 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 12:17:41.487684  8117 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.492484  8441 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.494277  8117 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:41.494976  8441 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:41.497076  8441 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:41.497922  8484 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:41.500725  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:41.501360  6654 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:17:41.501670  8336 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:41.503401  6550 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:41.504278  8484 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:41.504719  8484 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:41.505101  8484 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:41.505033  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:41.506438  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:41.506649  8117 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:41.508261  8497 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:41.509321  7238 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:41.510236  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:41.510272  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:41.511010  6657 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
W20260709 12:17:41.514586  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:41.515221  6657 raft_consensus.cc:1240] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:41.504563  6653 raft_consensus.cc:1275] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:17:41.517277  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.517823  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:41.518643  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:41.480522  6652 raft_consensus.cc:2393] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 4: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:41.519017  6653 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:41.522639  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:41.524057  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:41.524703  6658 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:41.525107  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:41.516278  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:41.527114  6657 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:41.525991  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:41.527874  6550 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:41.533538  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:41.534198  6550 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:41.536371  8484 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:41.539266  8465 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:41.539719  8465 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:41.540120  8465 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:41.540372  8303 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:41.540913  8303 raft_consensus.cc:697] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:41.541937  8303 consensus_queue.cc:237] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:41.543987  8462 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:41.544453  8462 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:41.544512  8307 consensus_queue.cc:1048] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:41.544992  8462 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:41.492578  8190 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:41.548630  8190 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:41.552707  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.553589  6505 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:41.555820  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.558619  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.560218  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:41.560830  6658 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:41.561955  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:41.562697  6655 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:41.565934  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.567067  6506 raft_consensus.cc:2393] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 4: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:41.568993  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:41.575325  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.532634  8282 raft_consensus.cc:3055] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:41.549269  8432 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:17:41.578181  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:41.554232  8412 raft_consensus.cc:3055] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:17:41.580924  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:41.486958  8400 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:41.567128  8484 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:41.592564  6509 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:41.592789  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:41.589584  6622 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:41.572279  8470 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.599689  6507 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:41.603217  6502 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:41.588479  8117 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:41.579530  8282 raft_consensus.cc:740] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:41.587908  8494 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.606592  6622 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:41.580524  8432 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:41.589860  6657 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:41.546777  8500 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:41.614750  6622 leader_election.cc:400] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 5. Current term is 6.
I20260709 12:17:41.611003  6653 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:41.618541  6622 leader_election.cc:403] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:17:41.588588  8412 raft_consensus.cc:740] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:41.603870  8117 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:41.620229  6622 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:41.621446  6653 pending_rounds.cc:85] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 1
I20260709 12:17:41.623790  6653 pending_rounds.cc:107] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:17:41.624130  6653 raft_consensus.cc:2889] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:41.624662  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:41.608810  8497 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:41.627871  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:41.628690  8373 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:41.629105  8373 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:41.629495  8373 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:41.629644  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:41.630313  6582 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:17:41.631763  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:41.604646  8036 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:41.615638  7269 raft_consensus.cc:1240] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:41.633345  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:41.633495  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:41.633445  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:41.637296  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:41.617410  8432 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:41.635381  6582 raft_consensus.cc:2393] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 4: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:41.616686  8383 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:41.637033  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:41.637738  6580 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:41.634569  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:41.661084  6579 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:17:41.638186  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:41.638350  6622 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:41.638849  6658 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:41.639281  6575 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:17:41.617479  8483 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:41.608567  8470 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:41.678784  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:41.694872  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 56594us
I20260709 12:17:41.699692  6575 raft_consensus.cc:606] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:41.618094  8494 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:41.700556  6502 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:17:41.619469  8282 consensus_queue.cc:260] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:41.701671  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:41.659296  8492 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:41.727072  6622 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:41.622852  8400 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 5. Current term is 6.
I20260709 12:17:41.632499  8441 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:41.632529  8399 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:41.633239  8497 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:41.706810  8117 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:17:41.757263  8515 log.cc:927] Time spent T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 12:17:41.707336  8412 consensus_queue.cc:260] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:41.712103  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:41.713074  6575 raft_consensus.cc:614] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:41.751351  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.752157  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.743763  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:41.754217  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:41.770081  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:41.770741  6507 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:41.755244  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.731896  8492 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:41.755749  8282 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:41.743052  8383 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:41.699508  8493 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:41.744046  8399 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:41.766561  8373 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:41.777464  8399 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:41.778465  6654 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:41.766461  8412 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:41.767330  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:41.755831  8497 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:41.776695  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.782874  6501 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:41.783550  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:41.784310  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.785367  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.787431  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.770663  8435 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:41.790366  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:41.790392  6474 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:41.792842  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1170) took 1132 ms. Trace:
I20260709 12:17:41.793370  6656 rpcz_store.cc:276] 0709 12:17:40.660855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.911205 (+250350us) service_pool.cc:224] Handling call
0709 12:17:40.911469 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:41.328905 (+417436us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:41.792568 (+463663us) spinlock_profiling.cc:243] Waited 413 ms on lock 0x7b540010ed54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:41.792771 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":949064448}
W20260709 12:17:41.799093  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 72149us
I20260709 12:17:41.800602  6622 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:41.802047  6622 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:41.775043  8492 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:41.804219  6650 raft_consensus.cc:1275] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:17:41.805454  6650 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:41.805572  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:41.806340  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:41.809450  7240 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:41.810429  6475 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:41.811426  6507 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:41.812606  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:41.778481  8493 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:41.814875  6580 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
W20260709 12:17:41.766990  8243 consensus_queue.cc:1175] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "20a202c76244485fbc8f1610473db8b9" responder_term: 4 status { last_received { term: 3 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 3. Current term is 4. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:17:41.804733  7267 raft_consensus.cc:2393] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 4: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:17:41.803345  7269 raft_consensus.cc:2393] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 2: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:17:41.816978  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:41.815574  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:41.835994  6656 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:41.840293  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:41.841236  6504 raft_consensus.cc:2393] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 4: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:41.836403  7270 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:41.834388  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:41.803202  8272 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:41.845607  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.850459  6504 raft_consensus.cc:2393] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 4: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:41.817086  8493 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:41.786219  8498 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:41.847610  6507 raft_consensus.cc:1275] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:17:41.866815  6502 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
W20260709 12:17:41.834836  7267 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38558 (request call id 336) took 1385 ms (client timeout 1546 ms). Trace:
I20260709 12:17:41.834231  8303 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:41.806977  8438 consensus_queue.cc:1059] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:17:41.848930  8487 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:41.835582  8532 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:41.844472  8117 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:41.844424  8398 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:41.883256  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:17:41.868566  8516 log.cc:927] Time spent T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 12:17:41.858314  8451 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:41.915802  6656 raft_consensus.cc:2468] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
W20260709 12:17:41.920572  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 103864us
I20260709 12:17:41.867209  8372 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:17:41.923115  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:41.921767  6475 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:41.930173  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:41.930975  6507 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:17:41.873318  8500 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:41.862596  8462 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:41.844372  8468 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:41.917461  8373 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:41.917294  8470 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:41.956714  7240 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:41.923101  8383 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:41.922856  8487 raft_consensus.cc:697] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 4 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:41.966552  6584 raft_consensus.cc:2393] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 2: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:41.917495  8190 consensus_queue.cc:1059] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 12:17:41.969403  6658 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
W20260709 12:17:41.870814  7267 rpcz_store.cc:269] 0709 12:17:40.449462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.474344 (+ 24882us) service_pool.cc:224] Handling call
0709 12:17:41.754053 (+1279709us) spinlock_profiling.cc:243] Waited 22.3 ms on lock 0x7b5800125198. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:17:41.834766 (+ 80713us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51371776}
I20260709 12:17:41.972012  6507 raft_consensus.cc:1275] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:17:41.920550  8117 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:41.880522  8460 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:41.973269  6475 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:41.975816  6503 raft_consensus.cc:1240] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 1. Current term is 2. Ops: [1.4-1.4]
I20260709 12:17:41.957739  8484 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:41.928421  8451 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:42.006665  8484 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:41.923877  8465 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:42.009379  6576 raft_consensus.cc:2393] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 4: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:42.009253  8484 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:41.983451  8462 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:41.987109  8540 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:42.010576  8465 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:42.010697  8451 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:42.011014  8540 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:42.011502  8540 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:42.012104  8451 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:42.012362  6475 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:41.990696  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:41.988974  8117 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:41.991937  8460 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:41.954998  8266 consensus_queue.cc:1048] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 12:17:41.992213  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:41.992908  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:41.991501  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:41.923342  8398 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:42.004030  8543 raft_consensus.cc:670] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } [suppressed 3 similar messages]
I20260709 12:17:42.012635  8451 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:42.002990  6550 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:42.014034  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:42.014832  6583 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:17:41.970844  6502 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 12:17:42.057739  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 54897.7us
I20260709 12:17:42.059775  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:42.051553  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 103721us
I20260709 12:17:42.017270  8451 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:42.015499  6579 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:42.020292  8303 raft_consensus.cc:2804] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:42.054790  6580 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:42.063510  6548 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:41.998967  8101 raft_consensus.cc:2764] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:17:42.019047  8544 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:42.014979  6582 raft_consensus.cc:2393] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 4: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:42.017362  8398 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:17:42.086774  8525 log.cc:927] Time spent T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
I20260709 12:17:42.086862  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:41.971591  8383 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:42.059580  8503 log.cc:927] Time spent T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:17:42.020287  8494 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:42.068061  8462 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:42.083794  6548 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:42.088994  6507 raft_consensus.cc:1240] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:42.085116  8544 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:41.995373  8536 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.104s
I20260709 12:17:42.082015  6583 raft_consensus.cc:1240] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:42.101594  6548 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:42.103916  6507 pending_rounds.cc:85] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 4
I20260709 12:17:42.104809  6507 pending_rounds.cc:107] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:17:42.105119  6507 raft_consensus.cc:2889] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:42.089282  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:42.110167  6548 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:42.090011  8303 raft_consensus.cc:697] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 2 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:42.101027  8544 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:42.115166  6548 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:42.133862  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:42.106884  8488 raft_consensus.cc:3055] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:42.103953  8395 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:42.112815  8527 log.cc:927] Time spent T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.266s	user 0.001s	sys 0.000s
I20260709 12:17:42.105532  8192 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:42.143486  6548 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:42.145061  6548 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:42.139133  8488 raft_consensus.cc:740] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:42.119377  8512 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:42.114080  8498 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:42.159379  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:42.147704  6548 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:42.113804  8344 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:42.138199  8282 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:42.153072  6579 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
W20260709 12:17:42.148545  8534 log.cc:927] Time spent T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
W20260709 12:17:42.217301  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 69697.1us
W20260709 12:17:42.214073  7266 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38562 (request call id 409) took 1705 ms (client timeout 1977 ms). Trace:
I20260709 12:17:42.251952  8514 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:42.141527  8323 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:42.148481  8511 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:42.114658  8462 raft_consensus.cc:697] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:42.215680  6651 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:42.250705  7270 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:42.152658  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:42.254699  6548 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:42.215855  8448 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:42.148933  8441 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:42.252951  8323 raft_consensus.cc:697] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:42.100353  8487 consensus_queue.cc:237] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:42.226130  8344 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:42.245615  8484 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:42.259517  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:42.307202  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:42.268256  6548 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:42.146960  8332 consensus_queue.cc:1059] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.111s
W20260709 12:17:42.251957  7266 rpcz_store.cc:269] 0709 12:17:40.508966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.509321 (+   355us) service_pool.cc:224] Handling call
0709 12:17:40.746254 (+236933us) spinlock_profiling.cc:243] Waited 8.11 ms on lock 0x7b5800125198. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:17:42.141436 (+1395182us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b5400507dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:42.148727 (+  7291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":580121216}
I20260709 12:17:42.309377  6548 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:42.139430  8543 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.114s
I20260709 12:17:42.287544  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:42.313171  6653 raft_consensus.cc:2393] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 4: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:17:42.291005  6549 connection.cc:256] RPC call timeout handler was delayed by -1.0440575059999446s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15360000000000001s for a response
I20260709 12:17:42.255738  8511 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:42.294533  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:42.279078  8282 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:42.145205  8395 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:42.246001  8540 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:42.314540  6652 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
W20260709 12:17:42.301342  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 344649us
I20260709 12:17:42.278298  8470 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:42.321934  6548 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:42.323413  6652 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:42.299993  8514 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:42.327077  6474 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:42.330972  6474 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:42.332187  7272 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:42.316407  8448 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:17:42.337491  8538 log.cc:927] Time spent T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 12:17:42.340287  6651 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:42.340936  6649 raft_consensus.cc:1240] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:42.315090  8511 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:42.341854  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1422) took 3546 ms. Trace:
I20260709 12:17:42.342971  6651 raft_consensus.cc:1240] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:42.342451  6581 rpcz_store.cc:276] 0709 12:17:38.795115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:39.300509 (+505394us) service_pool.cc:224] Handling call
0709 12:17:39.300770 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:41.279100 (+1978330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:42.341608 (+1062508us) spinlock_profiling.cc:243] Waited 1.97 s on lock 0x7b540032a354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:42.341766 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4532570496}
I20260709 12:17:42.346469  6653 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:42.327581  8332 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:42.258978  8488 consensus_queue.cc:260] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:42.331251  8470 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:42.335632  8537 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:42.377918  6651 raft_consensus.cc:1275] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:17:42.380383  6651 pending_rounds.cc:85] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 2
I20260709 12:17:42.380973  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:42.380932  6651 pending_rounds.cc:107] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:17:42.383589  6651 raft_consensus.cc:2889] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:42.351819  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:42.353084  8383 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:17:42.375591  8412 consensus_meta.cc:287] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.582s	user 0.011s	sys 0.008s
I20260709 12:17:42.324669  8282 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:42.353875  8547 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:17:42.376451  8548 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
W20260709 12:17:42.398116  8535 log.cc:927] Time spent T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 12:17:42.392077  8332 raft_consensus.cc:697] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:42.374768  8500 consensus_queue.cc:1059] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:17:42.399343  8488 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:42.335340  8514 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:42.352007  8303 consensus_queue.cc:237] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:42.408772  8468 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:42.379027  8462 consensus_queue.cc:237] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:42.465747  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 80355.4us
I20260709 12:17:42.400581  8470 consensus_queue.cc:1048] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 12:17:42.378844  8323 consensus_queue.cc:237] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:42.470315  7898 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:42.470845  7898 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:42.473038  7898 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:42.450387  8559 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:42.456037  8332 consensus_queue.cc:237] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:42.488574  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:42.477197  8555 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:17:42.521488  8563 consensus_queue.cc:1048] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:42.521852  6501 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:17:42.517671  8554 log.cc:927] Time spent T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.052s	user 0.001s	sys 0.001s
I20260709 12:17:42.494558  8559 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:42.525156  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:42.525911  8559 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:42.526299  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:42.526543  8540 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:42.527860  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:17:42.535034  6578 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54204 (request call id 137) took 1580 ms (client timeout 1707 ms). Trace:
W20260709 12:17:42.535478  6578 rpcz_store.cc:269] 0709 12:17:40.954904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.955502 (+   598us) service_pool.cc:224] Handling call
0709 12:17:42.534858 (+1579356us) spinlock_profiling.cc:243] Waited 319 ms on lock 0x7b54002666d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:42.534963 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":733165440}
I20260709 12:17:42.545011  6622 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:42.562310  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:42.566895  6501 raft_consensus.cc:2393] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 5: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:42.570394  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:42.565790  8547 raft_consensus.cc:697] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:42.532908  8498 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:42.532747  8551 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:42.567178  8494 raft_consensus.cc:2764] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:17:42.570791  8383 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:42.573722  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:42.573920  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:42.582020  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 98550.3us
I20260709 12:17:42.577425  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:42.605645  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:42.603667  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 91997.1us
I20260709 12:17:42.610514  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:42.611536  6501 raft_consensus.cc:2393] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 3: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:42.583621  8565 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:42.613642  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:42.615477  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:17:42.603972  8553 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 12:17:42.618343  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:42.619215  6501 raft_consensus.cc:1275] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:42.616514  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:42.620100  6581 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:42.614601  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:42.585572  8551 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:42.574342  8398 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:42.590304  8282 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:42.584733  8498 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:42.620746  6584 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:42.622690  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:42.626677  6579 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 4.
I20260709 12:17:42.608992  8559 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:42.624109  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:42.628069  6583 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:42.633000  8559 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:42.634042  8398 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:42.610430  8494 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:42.635192  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:42.609222  8470 consensus_queue.cc:1059] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:17:42.635109  8498 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:42.627394  8565 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:42.605553  8558 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:42.621832  8551 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:42.659143  8192 consensus_meta.cc:287] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.511s	user 0.013s	sys 0.003s
I20260709 12:17:42.681430  8192 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:42.665225  8565 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:42.656286  6581 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
I20260709 12:17:42.684979  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:42.687417  8498 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:42.673662  8558 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:42.689961  8498 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:42.693491  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 1497) took 1058 ms. Trace:
I20260709 12:17:42.693974  6503 rpcz_store.cc:276] 0709 12:17:41.634864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:41.802967 (+168103us) service_pool.cc:224] Handling call
0709 12:17:41.825065 (+ 22098us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:42.006203 (+181138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:42.693137 (+686934us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54000ff354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:42.693420 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":324148608}
I20260709 12:17:42.670349  8494 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:42.690639  8558 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:42.691365  8505 raft_consensus.cc:3055] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:42.691782  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:17:42.658846  8493 consensus_meta.cc:287] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.792s	user 0.010s	sys 0.004s
I20260709 12:17:42.623718  8569 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:42.580554  8547 consensus_queue.cc:237] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:42.697451  6475 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:42.741896  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:17:42.742585  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 45230.8us
I20260709 12:17:42.743479  6475 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:42.741674  8505 raft_consensus.cc:740] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:42.747325  6578 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:42.747805  6579 raft_consensus.cc:3055] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:42.751314  6579 raft_consensus.cc:740] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:42.753901  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:42.754690  6580 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 4.
I20260709 12:17:42.673743  8344 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:42.756191  8567 consensus_queue.cc:1059] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 12:17:42.752262  6579 consensus_queue.cc:260] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:42.756973  6584 raft_consensus.cc:1240] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:42.758205  6579 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:42.755592  8551 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:42.756124  6549 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:42.748121  8557 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:17:42.754323  8399 consensus_meta.cc:287] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.976s	user 0.009s	sys 0.004s
W20260709 12:17:42.673631  8432 consensus_meta.cc:287] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.970s	user 0.008s	sys 0.018s
W20260709 12:17:42.671602  8497 consensus_meta.cc:287] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.872s	user 0.011s	sys 0.006s
I20260709 12:17:42.744710  8570 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:42.751060  6583 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5.
I20260709 12:17:42.745600  8493 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:42.794289  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:42.670258  8398 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:42.797610  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:42.786051  6578 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
I20260709 12:17:42.799446  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:42.800802  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:42.800087  8497 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:42.765486  8498 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:42.805828  6650 raft_consensus.cc:2393] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 2: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:42.806732  6578 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:42.802168  8432 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:42.811501  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:17:42.799752  8492 consensus_meta.cc:287] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.979s	user 0.012s	sys 0.001s
I20260709 12:17:42.808298  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:42.812641  6656 raft_consensus.cc:1240] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 1. Current term is 2. Ops: []
W20260709 12:17:42.815410  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 48445.8us
I20260709 12:17:42.812654  8498 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:42.817894  8498 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:42.812757  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:42.821090  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:42.806324  8570 raft_consensus.cc:697] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 4 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:42.783082  8558 consensus_queue.cc:1059] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 12:17:42.790638  8399 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:42.790359  8540 consensus_queue.cc:1059] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.023s
W20260709 12:17:42.827523  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 69736.2us
W20260709 12:17:42.812762  8451 consensus_meta.cc:287] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.730s	user 0.011s	sys 0.001s
I20260709 12:17:42.763478  8511 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:42.828989  6579 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:17:42.812523  8557 raft_consensus.cc:697] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:42.826326  8559 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:42.741323  8569 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:42.824163  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:42.831117  8566 consensus_queue.cc:1059] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.077s
I20260709 12:17:42.833545  8568 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:42.834700  8562 consensus_queue.cc:1048] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:42.835847  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:42.848245  6580 raft_consensus.cc:2393] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 4: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:42.841739  8561 raft_consensus.cc:3055] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:42.848717  8561 raft_consensus.cc:740] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:42.849207  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:42.835826  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:42.838513  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:42.838701  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:42.850687  6583 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:42.839939  6579 pending_rounds.cc:85] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 5
I20260709 12:17:42.789125  8505 consensus_queue.cc:260] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:42.828732  8383 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:42.851828  8505 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:42.851915  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:42.852665  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:42.819077  8492 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:42.857363  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:42.858661  6579 pending_rounds.cc:107] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:17:42.859045  6579 raft_consensus.cc:2889] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:42.840058  8551 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:42.861469  6582 raft_consensus.cc:1240] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:42.861394  6583 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:42.863547  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:42.831863  8567 consensus_queue.cc:1059] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 12:17:42.867161  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:42.856190  8492 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:42.850994  8498 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:42.869261  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:42.865211  6583 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:42.870117  6579 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:42.840811  8451 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:42.873769  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:42.849507  8561 consensus_queue.cc:260] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:42.881968  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:42.884848  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:42.889153  6651 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:42.881279  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:42.876149  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:42.890138  6580 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:42.887312  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:42.881614  6655 raft_consensus.cc:1240] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:42.890372  6578 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:42.890739  6584 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:42.894963  6501 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:42.896087  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:42.891995  6653 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:17:42.901041  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:42.870930  8498 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:42.872253  8544 consensus_meta.cc:287] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.730s	user 0.005s	sys 0.008s
I20260709 12:17:42.903329  8476 raft_consensus.cc:3055] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:42.904366  8561 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:42.904855  8291 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:42.908612  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:42.885963  8344 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:42.924592  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:42.912055  6578 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
W20260709 12:17:42.916743  8574 log.cc:927] Time spent T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 12:17:42.916993  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:42.954037  6578 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "20a202c76244485fbc8f1610473db8b9"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:17:42.954212  6503 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:42.876847  8435 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:42.886476  8395 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:42.919155  8569 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:42.920864  6580 raft_consensus.cc:1240] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:17:42.878329  7898 raft_consensus.cc:3055] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:42.874550  8494 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:42.922254  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:42.897606  8558 consensus_queue.cc:1059] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 12:17:42.902123  8570 consensus_queue.cc:237] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:42.934600  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:42.955261  6584 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:42.886356  8557 consensus_queue.cc:237] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:42.954654  6578 raft_consensus.cc:606] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Received request to transfer leadership to TS 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:43.015759  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:42.981716  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 152) took 1466 ms. Trace:
I20260709 12:17:43.017164  6578 raft_consensus.cc:614] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:43.016958  6508 rpcz_store.cc:276] 0709 12:17:41.515420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:41.515558 (+   138us) service_pool.cc:224] Handling call
0709 12:17:41.515809 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:41.515990 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:17:41.618246 (+102256us) log.cc:844] Serialized 14 byte log entry
0709 12:17:42.980966 (+1362720us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b180034e760. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2875a314bc 00007f2872d7f48b 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a2923a
0709 12:17:42.981224 (+   258us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:42.981292 (+    68us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:42.981382 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:42.981479 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:42.981637 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":124687,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":2029,"raft.queue_time_us":879,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":2112,"spinlock_wait_cycles":45401344,"thread_start_us":189796,"threads_started":2,"wal-append.queue_time_us":179486}
I20260709 12:17:42.976622  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:42.927263  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:43.021889  6651 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:43.015012  7898 raft_consensus.cc:740] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:43.006194  8465 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:42.929535  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:43.027992  6657 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:43.028697  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:43.026680  6501 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:43.027742  8465 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:43.030936  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:43.030774  7898 consensus_queue.cc:260] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:43.031724  6583 raft_consensus.cc:2393] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 3: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:42.930270  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:43.032687  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:42.925487  8575 log.cc:927] Time spent T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:17:43.027814  8494 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:43.044209  6584 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:42.999625  8514 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:43.047333  7898 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:43.049926  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:42.927747  8476 raft_consensus.cc:740] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:43.050294  8465 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:43.050379  6580 raft_consensus.cc:1240] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 4. Current term is 5. Ops: []
W20260709 12:17:43.051548  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.048668  8399 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:43.013311  8498 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:42.978129  8432 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:43.021718  8435 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:42.925372  8562 consensus_queue.cc:1059] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.128s
I20260709 12:17:43.080420  6502 raft_consensus.cc:1240] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:42.923779  8544 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:43.080966  8562 raft_consensus.cc:3055] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:43.081333  8562 raft_consensus.cc:740] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:43.054236  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:43.082082  6503 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:43.024444  6550 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:43.056317  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:43.057063  6501 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:43.057530  6507 raft_consensus.cc:2393] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 6: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:43.058163  6548 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:17:43.083165  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 58832.9us
I20260709 12:17:43.083506  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:43.084048  6548 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:43.043658  6579 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
I20260709 12:17:43.032049  8383 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:43.061873  6657 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:17:43.062734  8511 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:43.085815  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:43.086654  6657 raft_consensus.cc:2410] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:17:43.088325  6651 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 12:17:43.088773  6651 pending_rounds.cc:85] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 4
I20260709 12:17:43.088970  6651 pending_rounds.cc:107] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 3.5
I20260709 12:17:43.089210  6651 raft_consensus.cc:2917] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Aborting config change with OpId 3.5: Aborted: Op aborted by new leader
I20260709 12:17:43.089533  6651 raft_consensus.cc:2889] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:43.067142  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:43.084681  6550 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:43.090205  6584 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:17:43.091274  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:43.091601  8558 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:43.096462  8465 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:43.096958  8465 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:43.097666  8465 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:43.098052  8344 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:43.099004  8344 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:43.096858  8556 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:43.065806  8494 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:43.103483  8494 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:43.103511  8462 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:43.104760  8462 raft_consensus.cc:697] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 2 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:43.106544  6657 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:17:43.092567  8383 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:17:43.107612  6548 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:43.108675  6651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:58970 (request call id 70) took 2977 ms. Trace:
I20260709 12:17:43.109712  6548 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:43.104856  8476 consensus_queue.cc:260] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:43.114396  8476 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:43.116681  8544 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:43.116765  8282 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:43.117326  8282 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:43.117707  8282 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:43.118453  6576 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1443) took 1482 ms (client timeout 1938 ms). Trace:
I20260709 12:17:43.112427  8556 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:17:43.118851  6576 rpcz_store.cc:269] 0709 12:17:41.636548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:41.636758 (+   210us) service_pool.cc:224] Handling call
0709 12:17:43.118221 (+1481463us) spinlock_profiling.cc:243] Waited 349 ms on lock 0x7b540034dfd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:43.118385 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":802812160}
I20260709 12:17:43.119614  8556 raft_consensus.cc:697] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:43.120219  6581 raft_consensus.cc:1240] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:43.109064  6651 rpcz_store.cc:276] 0709 12:17:40.112191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.363569 (+251378us) service_pool.cc:224] Handling call
0709 12:17:41.387323 (+1023754us) spinlock_profiling.cc:243] Waited 478 ms on lock 0x7b540011c720. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e06520 00007f2875e8fb42 00007f2875e8e800 00007f2875e879fa 00007f287ae4f4c9 00007f28743cbc54 00007f28743cbbe2 00007f28743cbb11 00007f28743cba8c 00007f28743cad42
0709 12:17:41.422492 (+ 35169us) log.cc:844] Serialized 480 byte log entry
0709 12:17:43.089887 (+1667395us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12763,"raft.queue_time_us":22883,"raft.run_cpu_time_us":1205,"raft.run_wall_time_us":13025,"spinlock_wait_cycles":1098604032,"thread_start_us":127716,"threads_started":1,"wal-append.queue_time_us":171259,"wal-append.run_cpu_time_us":2540,"wal-append.run_wall_time_us":1186563}
I20260709 12:17:43.116712  8192 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 12:17:43.129964  8462 consensus_queue.cc:237] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:43.131100  7269 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:17:43.148959  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 1541) took 2151 ms. Trace:
I20260709 12:17:43.149559  6509 rpcz_store.cc:276] 0709 12:17:40.997188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:40.997676 (+   488us) service_pool.cc:224] Handling call
0709 12:17:40.998090 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:41.593052 (+594962us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:43.148597 (+1555545us) spinlock_profiling.cc:243] Waited 594 ms on lock 0x7b5400203d54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:43.148827 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1366660352}
I20260709 12:17:43.153759  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:43.145033  8512 raft_consensus.cc:3055] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:43.154928  8512 raft_consensus.cc:740] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:43.156366  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:43.157289  6654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 1673) took 1592 ms. Trace:
I20260709 12:17:43.157793  6654 rpcz_store.cc:276] 0709 12:17:41.564792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:41.615338 (+ 50546us) service_pool.cc:224] Handling call
0709 12:17:41.634359 (+ 19021us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:41.778918 (+144559us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:43.157008 (+1378090us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54002a3ad4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:43.157212 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":330953472}
W20260709 12:17:43.159480  6500 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 1488) took 1594 ms (client timeout 1530 ms). Trace:
I20260709 12:17:43.173544  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:43.175410  6583 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
W20260709 12:17:43.159909  6500 rpcz_store.cc:269] 0709 12:17:41.564901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:41.565048 (+   147us) service_pool.cc:224] Handling call
0709 12:17:43.156164 (+1591116us) spinlock_profiling.cc:243] Waited 7.76 ms on lock 0x7b5800002a98. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:17:43.159405 (+  3241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17845504}
I20260709 12:17:43.155720  8512 consensus_queue.cc:260] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:43.189718  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:43.190202  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:43.191119  6654 raft_consensus.cc:2410] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:43.193043  6548 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:43.194125  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.187919  8512 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:43.249433  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 56471.2us
I20260709 12:17:43.091336  8562 consensus_queue.cc:260] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:43.256399  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.259440  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:43.260666  6500 raft_consensus.cc:2410] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:17:43.173383  8443 raft_consensus.cc:3055] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:43.173532  8441 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:43.270079  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:43.270653  6509 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:17:43.271464  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:43.272038  6583 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:43.257678  8514 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:43.257441  8568 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:43.286854  6394 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 6f94b1cd286648a9b82d5abf69bb86d5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) due to error UNKNOWN_ERROR; will retry: Aborted: Op aborted by new leader
I20260709 12:17:43.291028  8451 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:43.293133  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:43.298285  6654 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:43.268992  8344 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:43.271584  8383 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:43.269137  8443 raft_consensus.cc:740] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:43.273785  8441 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:43.301047  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:43.311177  6501 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 7: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:17:43.302109  6394 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 103074us
I20260709 12:17:43.307149  6583 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:43.314265  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:43.315321  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:43.316553  6501 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:17:43.128449  8556 consensus_queue.cc:237] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:43.324965  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:43.276521  8282 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:43.329474  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:17:43.334645  8579 log.cc:927] Time spent T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 12:17:43.334764  8282 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:43.349570  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:43.376964  6650 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49792 (request call id 150) took 1571 ms (client timeout 1544 ms). Trace:
W20260709 12:17:43.377470  6650 rpcz_store.cc:269] 0709 12:17:41.805103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:41.806140 (+  1037us) service_pool.cc:224] Handling call
0709 12:17:43.376747 (+1570607us) spinlock_profiling.cc:243] Waited 998 ms on lock 0x7b54001b1554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:43.376886 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2296529792}
I20260709 12:17:43.378433  6549 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:43.387861  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:43.389364  6583 raft_consensus.cc:2393] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 5: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:43.358647  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:43.392556  6549 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:43.301560  8568 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:43.395300  8568 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:43.390833  6650 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:55116
I20260709 12:17:43.396740  6650 raft_consensus.cc:606] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Received request to transfer leadership to TS 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:43.392848  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:43.397727  6650 raft_consensus.cc:614] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:43.397858  6583 raft_consensus.cc:2393] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 4: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:43.353508  8435 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:43.389629  8192 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:43.408845  8192 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:43.414357  6654 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:17:43.407816  7267 raft_consensus.cc:2393] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 5: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:17:43.418255  6583 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:43.397768  8344 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:43.419727  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 61821.6us
I20260709 12:17:43.423483  6651 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:43.406817  8564 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:43.356598  8443 consensus_queue.cc:260] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:43.433264  6583 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:43.420611  8192 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:43.435880  6577 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1432) took 2268 ms (client timeout 2056 ms). Trace:
I20260709 12:17:43.403787  8465 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:43.444453  6577 rpcz_store.cc:269] 0709 12:17:41.167295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:41.167411 (+   116us) service_pool.cc:224] Handling call
0709 12:17:43.435645 (+2268234us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b54001cc854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:43.435787 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":595910528}
I20260709 12:17:43.450340  6577 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:43.453512  6576 raft_consensus.cc:2410] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:17:43.454031  6577 raft_consensus.cc:1275] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:17:43.461037  8561 consensus_meta.cc:287] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.538s	user 0.010s	sys 0.000s
I20260709 12:17:43.462244  6583 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "20a202c76244485fbc8f1610473db8b9"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:17:43.462901  6583 raft_consensus.cc:606] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Received request to transfer leadership to TS 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:43.463243  6583 raft_consensus.cc:614] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:43.464488  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:17:43.424747  8307 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 12:17:43.449213  8558 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:43.457546  8443 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:43.478281  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:43.478634  8593 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:43.478929  6583 raft_consensus.cc:2376] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2. Re-sending same reply.
I20260709 12:17:43.470119  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:43.480358  7267 raft_consensus.cc:2393] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 5: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
W20260709 12:17:43.480721  8485 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 12:17:43.479370  8593 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:43.482858  8523 log.cc:927] Time spent T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 1.635s	user 0.005s	sys 0.000s
I20260709 12:17:43.484082  8593 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:43.488668  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 1677) took 1671 ms. Trace:
I20260709 12:17:43.489135  6656 rpcz_store.cc:276] 0709 12:17:41.817582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:41.917202 (+ 99620us) service_pool.cc:224] Handling call
0709 12:17:41.924513 (+  7311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:42.815381 (+890868us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:43.488439 (+673058us) spinlock_profiling.cc:243] Waited 888 ms on lock 0x7b54000b5c54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:43.488603 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2041976576}
W20260709 12:17:43.492492  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:43.493310  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.493203  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:43.493971  6654 raft_consensus.cc:2393] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 2: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:43.469254  8093 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:43.473383  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:43.503969  6580 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:43.518446  8542 raft_consensus.cc:3055] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:43.519111  8542 raft_consensus.cc:740] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:43.528435  6506 raft_consensus.cc:1240] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:43.529198  6506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 1492) took 1947 ms. Trace:
I20260709 12:17:43.529522  6506 rpcz_store.cc:276] 0709 12:17:41.582076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:41.582225 (+   149us) service_pool.cc:224] Handling call
0709 12:17:43.527995 (+1945770us) spinlock_profiling.cc:243] Waited 6.11 ms on lock 0x7b5800002a98. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:17:43.528216 (+   221us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:43.528920 (+   704us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:43.529115 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14047104}
I20260709 12:17:43.530140  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:43.530390  6549 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:43.530800  6583 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:43.502544  8093 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:43.523989  8492 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:43.539184  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:43.540076  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:17:43.541040  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:43.542526  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:17:43.543632  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.534080  8558 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:43.567427  6583 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5.
W20260709 12:17:43.572763  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.538609  8435 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:43.576735  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:43.577883  6505 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 1491) took 2000 ms (client timeout 2221 ms). Trace:
I20260709 12:17:43.584256  8595 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:43.585973  8598 consensus_queue.cc:1059] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:43.520172  8542 consensus_queue.cc:260] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:43.588647  7238 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:43.547120  8543 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:17:43.578383  6505 rpcz_store.cc:269] 0709 12:17:41.577175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:41.577663 (+   488us) service_pool.cc:224] Handling call
0709 12:17:43.576528 (+1998865us) spinlock_profiling.cc:243] Waited 7.81 ms on lock 0x7b5800002a98. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:17:43.577799 (+  1271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17966976}
I20260709 12:17:43.593355  6505 raft_consensus.cc:3055] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:43.593947  6505 raft_consensus.cc:740] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:43.600165  7270 raft_consensus.cc:1240] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:43.568130  8432 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:43.538380  8492 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:43.567384  8375 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:43.573400  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:43.597847  8543 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:43.617090  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:43.598738  8595 raft_consensus.cc:3055] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:43.607020  8465 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:43.614398  8542 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:43.615235  8592 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:43.627686  7269 raft_consensus.cc:2368] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 4. Current term is 5.
W20260709 12:17:43.633045  8590 log.cc:927] Time spent T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
W20260709 12:17:43.633698  8589 log.cc:927] Time spent T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 12:17:43.620070  8543 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:43.645641  8514 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:43.595180  6505 consensus_queue.cc:260] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:43.667230  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.674489  6505 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:43.674590  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.673655  7270 raft_consensus.cc:2393] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 2: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:17:43.620405  8595 raft_consensus.cc:740] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
W20260709 12:17:43.689038  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.690869  8441 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:43.692449  6657 raft_consensus.cc:1240] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:43.694265  8492 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:43.695237  8595 consensus_queue.cc:260] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:43.696300  8595 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:43.701603  8441 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:43.706621  8488 raft_consensus.cc:3055] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:43.707100  8488 raft_consensus.cc:740] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:17:43.714389  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 123905us
I20260709 12:17:43.612630  8375 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:43.705313  8493 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:43.723301  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:43.723997  6576 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 1.
W20260709 12:17:43.725736  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:43.728868  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.732926  6505 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:43.733650  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:43.734383  6583 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
I20260709 12:17:43.727998  8432 raft_consensus.cc:697] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
W20260709 12:17:43.743784  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 118793us
I20260709 12:17:43.708030  8488 consensus_queue.cc:260] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:43.745738  6474 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:43.732092  8093 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:43.760839  8488 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:43.749650  8375 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:43.755123  6474 leader_election.cc:400] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 4. Current term is 5.
I20260709 12:17:43.769495  6474 leader_election.cc:403] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:17:43.735244  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:43.773244  7238 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:17:43.755280  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:43.765712  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:43.765808  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:43.775527  6651 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:43.746306  8603 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:43.776739  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:43.777382  6651 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:43.775640  6577 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:43.771670  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.779323  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:43.780694  6473 connection.cc:256] RPC call timeout handler was delayed by -1.2132263650000823s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15100000000000002s for a response
W20260709 12:17:43.781132  6473 connection.cc:256] RPC call timeout handler was delayed by -1.18082636500003s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
W20260709 12:17:43.781409  6473 connection.cc:256] RPC call timeout handler was delayed by -1.1754263650000212s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1552s for a response
I20260709 12:17:43.780601  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:43.782282  7238 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:17:43.782985  6584 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:17:43.749714  8482 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:43.787734  6473 connection.cc:256] RPC call timeout handler was delayed by -1.07732636500009s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16610000000000003s for a response
W20260709 12:17:43.790643  6473 connection.cc:256] RPC call timeout handler was delayed by -1.0215263649999997s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1723s for a response
I20260709 12:17:43.796582  6550 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:43.768671  8443 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:43.791749  6473 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:43.814661  6473 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:43.816769  6473 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:43.819021  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:43.818356  7238 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
W20260709 12:17:43.860924  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:43.872468  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 55786.1us
I20260709 12:17:43.873467  6473 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:43.874877  6473 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:43.778923  8411 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:43.876037  6575 raft_consensus.cc:1240] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 2 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:43.764094  8470 consensus_queue.cc:1059] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 12:17:43.880522  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:43.763840  8499 consensus_queue.cc:1059] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 12:17:43.768360  8558 consensus_queue.cc:1059] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 12:17:43.884902  8093 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:43.885219  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:43.820487  8243 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:43.878823  8272 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:43.886873  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.768435  8514 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 12:17:43.882033  6575 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:43.872159  8508 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:43.923430  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:43.932456  6654 raft_consensus.cc:2468] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:43.819680  8438 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:43.924798  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:43.871877  8448 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:43.872048  8468 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:43.921792  6651 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:43.915072  6473 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:43.920413  8557 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:43.878494  8556 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:43.929932  6575 raft_consensus.cc:1240] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:43.904009  6577 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:17:43.974069  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:43.944896  8367 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:43.945299  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1189) took 1851 ms. Trace:
W20260709 12:17:43.977773  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 159505us
I20260709 12:17:43.950806  7265 raft_consensus.cc:1240] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:43.958006  6583 raft_consensus.cc:2393] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 3: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:43.882205  8190 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:43.929803  8608 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:43.977880  6649 rpcz_store.cc:276] 0709 12:17:42.093559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:42.339585 (+246026us) service_pool.cc:224] Handling call
0709 12:17:42.340756 (+  1171us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:42.341286 (+   530us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:43.945011 (+1603725us) spinlock_profiling.cc:243] Waited 890 us on lock 0x7b540008f050. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:43.945208 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2046848}
I20260709 12:17:43.990391  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:43.972052  8536 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:43.977650  8367 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:43.993949  8367 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:43.932660  8243 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:43.926717  8432 consensus_queue.cc:237] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:43.996312  6656 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:43.941691  6651 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:43.935415  8093 raft_consensus.cc:697] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:43.932904  7269 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:43.957526  8468 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:43.999734  6654 raft_consensus.cc:1240] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3. Ops: []
W20260709 12:17:44.001047  8604 log.cc:927] Time spent T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:17:44.001734  8243 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:43.991179  8448 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:43.992022  8608 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:44.007401  8323 raft_consensus.cc:3055] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:44.009434  8323 raft_consensus.cc:740] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:43.995277  8536 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:44.001801  8093 consensus_queue.cc:237] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:43.997789  7268 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38558 (request call id 340) took 2195 ms (client timeout 2031 ms). Trace:
I20260709 12:17:43.972584  8508 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:44.003677  8468 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:44.012511  8606 log.cc:927] Time spent T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 12:17:44.007671  8448 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:43.978818  8556 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:44.017508  8511 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.915s	user 0.000s	sys 0.016s
I20260709 12:17:44.014483  8536 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:44.011411  8608 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:44.029928  6656 raft_consensus.cc:1275] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:44.021639  8605 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:44.026223  8511 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.925s	user 0.000s	sys 0.016s
I20260709 12:17:44.010351  8323 consensus_queue.cc:260] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:44.038793  8323 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:44.044389  8508 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:44.050213  8448 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:44.051020  8411 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 4. Current term is 5.
I20260709 12:17:44.057360  8537 raft_consensus.cc:1064] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:17:44.059804  8562 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:44.036188  8605 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:44.090034  8611 log.cc:927] Time spent T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:17:44.097826  8605 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:43.951333  6473 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:44.137027  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 185785us
I20260709 12:17:44.138917  6473 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:44.146868  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:44.147814  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.150063  6474 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:17:44.160418  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.162503  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 152) took 1833 ms. Trace:
I20260709 12:17:44.163316  8411 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:44.164584  8411 raft_consensus.cc:2764] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:17:44.167063  8411 raft_consensus.cc:2764] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 12:17:44.017078  7268 rpcz_store.cc:269] 0709 12:17:41.802136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:41.803287 (+  1151us) service_pool.cc:224] Handling call
0709 12:17:43.990218 (+2186931us) spinlock_profiling.cc:243] Waited 6.97 ms on lock 0x7b5800125198. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:17:43.997661 (+  7443us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16023808}
I20260709 12:17:44.182020  8537 consensus_queue.cc:237] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 2, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:44.163192  6652 rpcz_store.cc:276] 0709 12:17:42.328814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:42.382281 (+ 53467us) service_pool.cc:224] Handling call
0709 12:17:42.411847 (+ 29566us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:42.412701 (+   854us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:42.412733 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:42.449247 (+ 36514us) log.cc:844] Serialized 26 byte log entry
0709 12:17:43.955856 (+1506609us) spinlock_profiling.cc:243] Waited 1.17 ms on lock 0x7b18001d42e0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:17:44.012351 (+ 56495us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:44.012401 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.012479 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:44.161548 (+149069us) raft_consensus.cc:1685] finished
0709 12:17:44.161640 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:44.162158 (+   518us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3391,"raft.queue_time_us":441,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":34810,"spinlock_wait_cycles":2702080,"thread_start_us":56253,"threads_started":1,"wal-append.queue_time_us":1585458}
W20260709 12:17:44.189826  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.211139  6473 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:44.216750  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.220142  6473 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:44.222983  6473 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:44.230666  8542 consensus_meta.cc:287] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.563s	user 0.000s	sys 0.013s
W20260709 12:17:44.231117  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.232625  8451 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:44.237615  8556 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:44.240309  7268 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:44.249958  8570 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:44.252954  8550 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:44.251698  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.248462  8583 consensus_queue.cc:1059] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
W20260709 12:17:44.294116  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 71307.2us
I20260709 12:17:44.295907  6657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 1695) took 1103 ms. Trace:
I20260709 12:17:44.298439  7265 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEARNER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:44.302806  8537 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:44.296427  6657 rpcz_store.cc:276] 0709 12:17:43.192548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.192673 (+   125us) service_pool.cc:224] Handling call
0709 12:17:43.192909 (+   236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:43.692889 (+499980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.295356 (+602467us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b540044a7d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:44.295712 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1148352256}
I20260709 12:17:44.307138  8583 raft_consensus.cc:3055] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:44.307638  8583 raft_consensus.cc:740] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:44.308569  8367 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:44.309213  8583 consensus_queue.cc:260] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:44.311012  8583 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:44.311221  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:44.320847  8562 consensus_queue.cc:1048] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:44.323232  8497 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:44.314357  7273 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:17:44.329635  7273 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38562 (request call id 417) took 2007 ms. Trace:
I20260709 12:17:44.329881  7273 rpcz_store.cc:276] 0709 12:17:42.322287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:42.322442 (+   155us) service_pool.cc:224] Handling call
0709 12:17:44.313365 (+1990923us) spinlock_profiling.cc:243] Waited 2.76 ms on lock 0x7b5800125198. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:17:44.313734 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:44.329268 (+ 15534us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.329551 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6348160}
W20260709 12:17:44.317304  8620 log.cc:927] Time spent T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.238s	user 0.005s	sys 0.000s
I20260709 12:17:44.317934  8550 consensus_queue.cc:1059] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:44.318419  8484 raft_consensus.cc:3055] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:44.325409  7272 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38562 (request call id 424) took 1006 ms. Trace:
I20260709 12:17:44.332370  8484 raft_consensus.cc:740] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:44.332492  7272 rpcz_store.cc:276] 0709 12:17:43.319385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.319846 (+   461us) service_pool.cc:224] Handling call
0709 12:17:44.310772 (+990926us) spinlock_profiling.cc:243] Waited 7.69 ms on lock 0x7b5800125198. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:17:44.325333 (+ 14561us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17684736}
I20260709 12:17:44.311753  8367 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:44.326242  8496 raft_consensus.cc:3055] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:44.333943  8484 consensus_queue.cc:260] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:44.332190  8537 raft_consensus.cc:3055] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:44.335403  8537 raft_consensus.cc:740] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:44.335328  8484 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:44.334474  8496 raft_consensus.cc:740] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:17:44.337147  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 55313.9us
I20260709 12:17:44.341156  8496 consensus_queue.cc:260] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:44.342121  8511 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.342670  6502 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 1513) took 1448 ms. Trace:
I20260709 12:17:44.343576  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:44.343204  8511 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:44.343075  8496 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:44.350633  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 142) took 1664 ms. Trace:
I20260709 12:17:44.350888  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 1187) took 1265 ms. Trace:
I20260709 12:17:44.351909  8511 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:44.343184  6502 rpcz_store.cc:276] 0709 12:17:42.893670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:42.898288 (+  4618us) service_pool.cc:224] Handling call
0709 12:17:42.905054 (+  6766us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:43.080934 (+175880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.342053 (+1261119us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b54001e82d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:44.342390 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":390120704}
I20260709 12:17:44.353178  6582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 141) took 1728 ms. Trace:
I20260709 12:17:44.403164  6582 rpcz_store.cc:276] 0709 12:17:42.625007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:42.625174 (+   167us) service_pool.cc:224] Handling call
0709 12:17:42.632457 (+  7283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:42.862842 (+230385us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.352691 (+1489849us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b5400057dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:44.352995 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":526203776}
I20260709 12:17:44.353705  8556 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.404533  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:44.405251  8556 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:44.397377  6581 rpcz_store.cc:276] 0709 12:17:42.685631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:42.741249 (+ 55618us) service_pool.cc:224] Handling call
0709 12:17:43.120055 (+378806us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:43.120677 (+   622us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.350178 (+1229501us) spinlock_profiling.cc:243] Waited 378 ms on lock 0x7b540034dfd0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:44.350411 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":870341504}
I20260709 12:17:44.407085  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:44.397797  6580 rpcz_store.cc:276] 0709 12:17:43.085534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.085786 (+   252us) service_pool.cc:224] Handling call
0709 12:17:44.350482 (+1264696us) spinlock_profiling.cc:243] Waited 404 ms on lock 0x7b540034d5d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:44.350675 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":929235840}
I20260709 12:17:44.407691  8556 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:44.364679  8563 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:44.409155  8563 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:44.410218  8570 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:44.411671  8563 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:44.368618  8323 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:44.369643  8508 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:17:44.372215  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.372380  6657 raft_consensus.cc:1275] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:44.372776  8465 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.373308  6550 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:44.375247  6650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 1696) took 1121 ms. Trace:
I20260709 12:17:44.377095  8608 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:44.378494  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:44.368052  6549 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:44.379251  8448 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:44.379419  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:44.380455  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:44.381175  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:44.381295  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:17:44.382831  8500 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 12:17:44.386234  8550 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.339150  8537 consensus_queue.cc:260] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:44.437029  8550 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:44.438144  8537 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:44.439468  8550 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:44.388792  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:44.440459  8432 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.440699  7271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38562 (request call id 422) took 1131 ms. Trace:
I20260709 12:17:44.441210  7271 rpcz_store.cc:276] 0709 12:17:43.309586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.311244 (+  1658us) service_pool.cc:224] Handling call
0709 12:17:44.380209 (+1068965us) spinlock_profiling.cc:243] Waited 1.86 ms on lock 0x7b5800125198. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:17:44.440597 (+ 60388us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4275584}
W20260709 12:17:44.441880  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 68881.7us
I20260709 12:17:44.392611  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:44.395138  8536 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:44.444984  8536 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:44.445832  8608 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:44.434589  6473 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:44.446816  8448 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.447510  8448 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:44.435489  6506 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:17:44.448606  8272 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:44.413507  8497 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:44.448647  6473 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:44.450197  8448 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:44.417160  6650 rpcz_store.cc:276] 0709 12:17:43.253234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:43.410530 (+157296us) service_pool.cc:224] Handling call
0709 12:17:43.411591 (+  1061us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:43.411916 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:43.411944 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:43.419518 (+  7574us) log.cc:844] Serialized 26 byte log entry
0709 12:17:44.374261 (+954743us) spinlock_profiling.cc:243] Waited 2.97 ms on lock 0x7b18000edb20. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2875a314bc 00007f2872d7f48b 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1
0709 12:17:44.374552 (+   291us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:44.374618 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.374724 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:44.374766 (+    42us) raft_consensus.cc:1685] finished
0709 12:17:44.374779 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:44.375056 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":101,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":4177,"spinlock_wait_cycles":6837632,"thread_start_us":63422,"threads_started":1,"wal-append.queue_time_us":71646}
I20260709 12:17:44.450218  8497 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:44.456641  8367 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:44.457356  8367 raft_consensus.cc:697] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 6 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:44.458652  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:44.458724  8367 consensus_queue.cc:237] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:44.459714  6651 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5.
W20260709 12:17:44.464479  8543 consensus_meta.cc:287] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.753s	user 0.009s	sys 0.012s
I20260709 12:17:44.464378  8432 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:44.464988  8543 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:44.466115  7270 raft_consensus.cc:2368] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 4. Current term is 5.
I20260709 12:17:44.467142  8543 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:44.468984  8470 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.469632  8470 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:44.471158  6652 raft_consensus.cc:1240] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:44.470008  7273 raft_consensus.cc:1240] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:44.477924  8432 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:44.476687  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:44.477998  8497 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:44.478957  6474 leader_election.cc:400] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 4. Current term is 5.
I20260709 12:17:44.479332  6474 leader_election.cc:403] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:17:44.479773  6650 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:44.483803  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:44.486599  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:44.495111  8508 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:44.497282  8465 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:44.498042  6657 raft_consensus.cc:1240] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:17:44.499315  6500 raft_consensus.cc:2393] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 2: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:44.499776  8465 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:44.500628  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:44.502182  6658 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:44.504581  8583 raft_consensus.cc:2804] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:44.505131  8583 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:44.505592  8583 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:44.508596  8508 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:17:44.513249  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:44.514618  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.514791  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:44.444257  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:44.516111  8323 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:17:44.516502  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 148606us
I20260709 12:17:44.517474  6403 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46184 (request call id 113) took 8769 ms. Trace:
I20260709 12:17:44.516177  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:44.519642  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:44.520558  7274 raft_consensus.cc:2393] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 5: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:17:44.521761  6508 raft_consensus.cc:2368] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7.
I20260709 12:17:44.523077  7270 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:44.524849  8605 consensus_queue.cc:1059] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:44.525417  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:44.526175  8605 consensus_queue.cc:1059] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:17:44.527232  8598 raft_consensus.cc:3055] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:44.472047  8470 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:44.527645  8598 raft_consensus.cc:740] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:44.528800  8598 consensus_queue.cc:260] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:44.529541  8570 consensus_queue.cc:1059] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:44.530393  8570 consensus_queue.cc:1243] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer 496c7c3792cd4587b4dbeba435f4be1e log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:17:44.531002  8598 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:44.531481  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 68243.7us
I20260709 12:17:44.531088  8570 raft_consensus.cc:3055] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:44.531980  8570 raft_consensus.cc:740] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:44.533253  6473 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:44.533013  8570 consensus_queue.cc:260] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:44.534199  8375 consensus_queue.cc:1059] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 5.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:44.534948  8550 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:44.535688  8557 raft_consensus.cc:3055] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:44.536160  8557 raft_consensus.cc:740] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:44.536126  8570 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:44.536796  8176 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:44.537374  8557 consensus_queue.cc:260] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:44.538549  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 59826.4us
I20260709 12:17:44.538745  8557 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:44.541754  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:44.542367  8451 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:44.541973  8375 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:44.542992  8451 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:44.544503  8542 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.543725  8375 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:44.544906  8443 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.545135  8542 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:44.545557  8443 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:44.546926  8375 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:44.547322  8542 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:44.547667  8443 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:44.547869  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:44.548501  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:44.548969  8550 raft_consensus.cc:697] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:44.550151  8550 consensus_queue.cc:237] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:44.552942  8451 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:44.557472  6550 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:44.558729  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:44.559341  8323 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:44.559669  6507 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:44.561882  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:44.447741  8468 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:44.564590  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:44.565138  6658 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:44.566219  8468 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:44.567142  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:44.567821  6508 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:44.572986  6656 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:17:44.571497  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:44.581717  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 47243.9us
I20260709 12:17:44.584721  6508 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
I20260709 12:17:44.575193  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:44.594149  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:44.580649  8323 raft_consensus.cc:697] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:44.600570  6657 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:44.611501  8468 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.608078  8448 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 12:17:44.617496  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:44.603567  8508 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.623207  6658 raft_consensus.cc:1240] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:44.621019  6656 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:17:44.589743  8484 consensus_queue.cc:1059] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 12:17:44.624161  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:44.627862  6581 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:17:44.625838  6504 raft_consensus.cc:1240] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:17:44.639423  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:44.640103  6584 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:44.644147  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:44.644572  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:44.644361  8323 consensus_queue.cc:237] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:44.653445  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:44.654215  6577 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:44.660151  6580 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:17:44.632504  8176 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 4. Current term is 5.
I20260709 12:17:44.662765  6584 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 3.
I20260709 12:17:44.647078  6657 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:17:44.660863  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:44.667245  6622 connection.cc:256] RPC call timeout handler was delayed by -1.6759950359999038s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16290000000000002s for a response
I20260709 12:17:44.518013  6403 rpcz_store.cc:276] 0709 12:17:35.747736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:35.756372 (+  8636us) service_pool.cc:224] Handling call
0709 12:17:44.517171 (+8760799us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:41.162320 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:41.163037 (+   717us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:41.163083 (+    46us) write_op.cc:435] Acquired schema lock
0709 12:17:41.163102 (+    19us) tablet.cc:662] Decoding operations
0709 12:17:41.164273 (+  1171us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:41.165264 (+   991us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:41.166230 (+   966us) tablet.cc:685] Acquiring locks for 9 operations
0709 12:17:41.318250 (+152020us) tablet.cc:701] Row locks acquired
0709 12:17:41.319853 (+  1603us) write_op.cc:260] PREPARE: finished
0709 12:17:41.326980 (+  7127us) write_op.cc:270] Start()
0709 12:17:41.327099 (+   119us) write_op.cc:276] Timestamp: P: 1783599461326927 usec, L: 0
0709 12:17:41.327126 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 12:17:41.330605 (+  3479us) log.cc:844] Serialized 3580 byte log entry
0709 12:17:41.704737 (+374132us) op_driver.cc:464] REPLICATION: finished
0709 12:17:41.843672 (+138935us) write_op.cc:301] APPLY: starting
0709 12:17:41.853047 (+  9375us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:41.927482 (+ 74435us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:41.973886 (+ 46404us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:42.260670 (+286784us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:42.321231 (+ 60561us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0709 12:17:42.332647 (+ 11416us) write_op.cc:312] APPLY: finished
0709 12:17:42.340092 (+  7445us) log.cc:844] Serialized 71 byte log entry
0709 12:17:42.340313 (+   221us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:42.340463 (+   150us) write_op.cc:454] Released schema lock
0709 12:17:42.340506 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:42.340861 (+   355us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:42.340888 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":310518,"reported_tablets":13,"child_traces":[["op",{"apply.queue_time_us":112773,"apply.run_cpu_time_us":6456,"apply.run_wall_time_us":505460,"num_ops":9,"prepare.queue_time_us":486845,"prepare.run_cpu_time_us":23674,"prepare.run_wall_time_us":232055,"raft.queue_time_us":148848,"raft.run_cpu_time_us":33726,"raft.run_wall_time_us":49107,"replication_time_us":377464,"thread_start_us":395272,"threads_started":4,"wal-append.queue_time_us":216345,"wal-append.run_cpu_time_us":14195,"wal-append.run_wall_time_us":1804451}]]}
I20260709 12:17:44.671571  7240 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:44.672317  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:44.673211  6649 raft_consensus.cc:2410] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 12:17:44.673622  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:44.674937  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:44.676016  7273 raft_consensus.cc:2393] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 2: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:17:44.676465  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:44.677367  6583 raft_consensus.cc:2368] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7.
I20260709 12:17:44.677963  8443 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.678582  8443 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:44.681563  8176 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.682350  8176 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:44.668190  6622 connection.cc:256] RPC call timeout handler was delayed by -1.6066950360000192s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1706s for a response
I20260709 12:17:44.689309  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:44.623090  8468 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:44.638137  8451 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.699961  8176 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:44.700093  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:44.697132  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:44.702553  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:44.707718  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.705928  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:44.715404  6658 raft_consensus.cc:1240] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6. Ops: []
W20260709 12:17:44.717492  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:44.717844  6622 connection.cc:256] RPC call timeout handler was delayed by -1.3978950359999089s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1938s for a response
W20260709 12:17:44.718186  6622 connection.cc:256] RPC call timeout handler was delayed by -1.3872232430001077s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15280000000000002s for a response
I20260709 12:17:44.718564  6657 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:17:44.668876  8470 consensus_queue.cc:1048] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 12:17:44.664482  8508 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:44.727159  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:44.715962  8432 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:44.730777  6652 raft_consensus.cc:1275] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:44.731352  6649 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:17:44.732582  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.728133  6579 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:17:44.730021  6580 raft_consensus.cc:1240] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:44.739495  6656 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:44.712586  8583 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:44.720703  6502 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 6.
I20260709 12:17:44.711462  8608 consensus_queue.cc:1048] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 12:17:44.733932  8497 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.732937  8432 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:44.747570  6504 raft_consensus.cc:1275] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:44.753082  6503 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:44.723299  8451 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:44.744071  8443 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:44.755749  6509 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5.
I20260709 12:17:44.757292  8093 consensus_queue.cc:1059] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:17:44.755420  8432 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:44.757884  6394 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 6f94b1cd286648a9b82d5abf69bb86d5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica f2b06aebe97249a4877d798b8118ece4 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:44.740397  8448 consensus_queue.cc:1059] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 12:17:44.763562  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:17:44.769814  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 90033.5us
W20260709 12:17:44.810521  6394 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 70082.5us
I20260709 12:17:44.811632  6504 raft_consensus.cc:1240] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7. Ops: [4.2-6.3]
I20260709 12:17:44.766263  8176 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.816656  6503 raft_consensus.cc:1240] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 1. Current term is 2. Ops: [1.4-1.4]
I20260709 12:17:44.813995  8497 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:44.827154  8497 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:44.827706  8497 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:44.829128  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.795647  8511 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.819684  6654 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:44.835616  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:44.838937  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.840003  8635 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:44.840431  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:44.840675  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:44.849670  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:44.850958  6584 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
W20260709 12:17:44.872681  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:44.871555  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 46118.6us
I20260709 12:17:44.855419  6577 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
W20260709 12:17:44.821235  8631 log.cc:927] Time spent T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:17:44.828153  8497 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
W20260709 12:17:44.880868  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 128798us
I20260709 12:17:44.886214  6504 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
W20260709 12:17:44.889010  8628 log.cc:927] Time spent T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 12:17:44.919850  6622 connection.cc:256] RPC call timeout handler was delayed by -1.1280232430001433s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1816s for a response
W20260709 12:17:44.920603  6622 connection.cc:256] RPC call timeout handler was delayed by -1.1280232430001433s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1816s for a response
W20260709 12:17:44.921041  6622 connection.cc:256] RPC call timeout handler was delayed by -1.0263232429999789s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19290000000000002s for a response
W20260709 12:17:44.887328  8630 log.cc:927] Time spent T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:17:44.919000  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:44.927841  6579 raft_consensus.cc:1813] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Handling vote request from an unknown peer f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:44.888592  8468 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:44.887568  8451 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:44.836370  8176 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:44.885433  8552 raft_consensus.cc:3055] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:44.932183  6579 raft_consensus.cc:2410] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 2 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 12:17:44.928052  6500 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5.
I20260709 12:17:44.933454  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:44.935284  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.840585  8635 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:44.914263  6548 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:44.882581  8508 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:44.944793  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1203) took 2020 ms. Trace:
I20260709 12:17:44.945041  8508 consensus_queue.cc:1059] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:44.836546  8448 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.094s
I20260709 12:17:44.853171  8511 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:44.943557  6548 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:44.945256  6655 rpcz_store.cc:276] 0709 12:17:42.924240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:42.924417 (+   177us) service_pool.cc:224] Handling call
0709 12:17:42.924734 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:42.925126 (+   392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:42.925159 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:42.925658 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 12:17:44.409008 (+1483350us) spinlock_profiling.cc:243] Waited 1.25 ms on lock 0x7b18003f6fa0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:17:44.612007 (+202999us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:44.616943 (+  4936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:44.625648 (+  8705us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:44.944402 (+318754us) raft_consensus.cc:1685] finished
0709 12:17:44.944423 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:44.944631 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":133,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":29766,"spinlock_wait_cycles":2880768,"thread_start_us":173569,"threads_started":1,"wal-append.queue_time_us":1884217}
I20260709 12:17:44.947893  6548 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:44.948256  8511 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:44.948904  8511 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:44.949635  8511 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:44.950703  6548 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:44.951675  8511 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:44.943807  8488 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.952668  8636 consensus_queue.cc:1059] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 12:17:44.952780  8488 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:44.944444  8465 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:44.942047  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:44.922235  8537 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:44.958438  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 143921us
I20260709 12:17:44.967449  8488 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:44.967500  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:44.968813  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.969414  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:44.970012  6500 raft_consensus.cc:2393] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 3: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:44.971599  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:17:44.974095  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:44.978253  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:44.941784  8552 raft_consensus.cc:740] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:44.981074  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:44.979238  6503 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:44.959895  8465 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:44.989082  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:44.989926  6500 raft_consensus.cc:2410] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 12:17:44.990819  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:44.994050  6550 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:44.995378  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:44.996016  6500 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:17:44.996385  7266 raft_consensus.cc:1275] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:44.988704  8488 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 12:17:45.007174  8635 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:45.008499  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:45.009116  6650 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:45.012575  8323 consensus_queue.cc:1059] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:45.013739  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:45.016624  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:45.017961  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:44.987191  8465 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:44.983474  8583 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:44.970690  8537 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:45.025056  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:17:45.030895  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 57419.4us
I20260709 12:17:45.017915  8483 raft_consensus.cc:3055] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:45.036664  6550 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:45.021477  8323 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.012898  8637 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:45.037899  6504 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:17:45.039080  6503 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:17:45.039942  8641 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.044625  8583 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:45.045092  8583 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:45.040550  8629 log.cc:927] Time spent T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:17:45.043763  8641 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:45.031710  8488 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:17:45.062101  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:45.063386  6650 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
W20260709 12:17:45.064388  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:45.035005  8483 raft_consensus.cc:740] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:45.069366  6506 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:45.051898  8583 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:45.071866  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:45.053642  8488 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.072788  6475 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:17:45.068199  8570 consensus_meta.cc:287] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.531s	user 0.010s	sys 0.002s
I20260709 12:17:45.070437  8557 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:17:45.083722  6500 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:45.053005  8637 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:45.066808  8608 consensus_queue.cc:1059] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 12:17:45.085068  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:45.087730  7266 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEARNER]: Advancing to term 7
I20260709 12:17:45.088043  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:45.089500  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:45.090185  6654 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:17:45.090953  6649 raft_consensus.cc:2410] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:17:45.052554  8323 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:45.078742  8488 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:45.095113  8593 raft_consensus.cc:3055] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:45.083772  8451 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:45.084506  8497 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:45.088255  8484 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:17:45.117455  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 48115.1us
I20260709 12:17:45.095551  8537 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:45.097311  8593 raft_consensus.cc:740] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:45.126593  8593 consensus_queue.cc:260] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:45.127589  8641 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:45.128206  8593 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:45.106837  8451 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.132697  8451 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:45.134958  8598 consensus_meta.cc:287] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.603s	user 0.006s	sys 0.004s
I20260709 12:17:45.136830  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:45.102599  8176 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:45.140296  6550 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:45.142154  6503 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:45.142330  8323 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:45.153234  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 99760.6us
I20260709 12:17:45.154059  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:45.156808  6475 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:45.121474  8484 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:45.098559  8483 consensus_queue.cc:260] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:45.164358  8176 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:45.112556  8552 consensus_queue.cc:260] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:45.184240  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 44010.6us
I20260709 12:17:45.167057  8641 raft_consensus.cc:2764] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 12:17:45.192812  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 48354.3us
I20260709 12:17:45.194696  6550 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:45.193964  8176 raft_consensus.cc:697] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:45.195904  8552 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:45.196487  8483 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:45.210536  6550 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:45.195529  8583 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:45.237025  8484 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:45.238154  8646 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:45.238811  8646 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.239151  8646 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:45.239609  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:45.240428  6655 raft_consensus.cc:2393] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 7: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:45.240779  8637 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:45.241451  8648 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:45.241968  8648 raft_consensus.cc:697] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:45.242933  8648 consensus_queue.cc:237] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:45.243837  7266 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 7 LEARNER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:17:45.241984  6475 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:45.234547  6622 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:45.255491  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:45.255189  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:45.256163  6651 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:17:45.255254  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:45.257401  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:45.265118  6649 raft_consensus.cc:2393] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 3: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:45.263957  8570 raft_consensus.cc:2764] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:17:45.279369  8570 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:45.279817  8563 raft_consensus.cc:3055] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:45.280269  8563 raft_consensus.cc:740] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:45.280725  8176 consensus_queue.cc:237] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:45.281111  8563 consensus_queue.cc:260] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:45.281662  8488 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:45.282924  8563 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:45.279832  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 45371.4us
I20260709 12:17:45.284282  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:45.231299  8451 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:45.292315  8443 consensus_queue.cc:1048] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:45.292867  6622 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:45.286993  6655 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:45.256811  8536 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:45.295912  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:45.265691  8652 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:17:45.300681  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 91825.7us
I20260709 12:17:45.301854  6475 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:45.306910  8432 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:45.310505  8432 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:45.313356  6652 raft_consensus.cc:1275] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:17:45.321801  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:45.298316  8637 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:45.297986  8641 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:45.322794  7271 raft_consensus.cc:2393] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 7: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:17:45.324792  6651 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
W20260709 12:17:45.331928  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 64609.3us
I20260709 12:17:45.305186  8652 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.315783  8468 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:45.298688  8557 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:17:45.338611  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:45.339291  6652 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:45.328974  8488 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:45.344972  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:45.342052  8468 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.358970  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:45.362469  6652 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:17:45.337656  8652 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:45.329591  8484 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 12:17:45.315189  8536 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:45.365255  8451 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:45.368820  8465 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:45.374568  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:45.380808  6501 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:45.367096  8468 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:45.441320  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:45.422701  8637 consensus_queue.cc:1048] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 12:17:45.442296  6584 raft_consensus.cc:2410] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:17:45.443622  6577 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:45.443668  8642 raft_consensus.cc:3055] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:45.444693  8642 raft_consensus.cc:740] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:45.445180  6580 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:45.445549  8642 consensus_queue.cc:260] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:45.446431  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:45.446795  8642 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:45.447000  6576 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:45.447283  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:45.449117  6583 raft_consensus.cc:1240] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:45.452292  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 53501.4us
W20260709 12:17:45.458747  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 102168us
W20260709 12:17:45.458833  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 143997us
I20260709 12:17:45.417634  8583 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:45.468245  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:45.468288  6509 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:45.470808  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:45.472775  6503 raft_consensus.cc:2376] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5. Re-sending same reply.
I20260709 12:17:45.477766  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:45.478593  6649 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:45.481069  6583 raft_consensus.cc:1240] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:45.488447  6475 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:45.489802  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:45.496767  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:45.499341  6500 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:45.495072  6550 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:45.505031  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:45.508474  8656 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.509795  8656 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:45.510881  8465 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:45.511391  8443 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:45.520537  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 52286.4us
I20260709 12:17:45.524940  6401 catalog_manager.cc:5697] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 4 to 5, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 5 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:45.530354  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:17:45.531411  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 47293.3us
I20260709 12:17:45.535775  8552 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:45.497615  6656 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:45.495812  6475 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:45.547983  6507 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:17:45.555001  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:45.584128  6650 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:45.555505  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:45.552426  6655 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:17:45.582495  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:45.587495  6658 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:45.558185  6576 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5.
W20260709 12:17:45.560645  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 64943.4us
W20260709 12:17:45.582072  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 65815.9us
I20260709 12:17:45.555743  6549 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:45.536322  8552 raft_consensus.cc:697] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:45.586971  8557 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:45.608956  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:45.606215  6475 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:45.542424  8367 raft_consensus.cc:1064] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:17:45.613593  6475 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:45.536836  8665 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:45.591943  8637 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:45.619472  6650 raft_consensus.cc:2410] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:17:45.620935  6401 catalog_manager.cc:5697] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 2 to 3, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 3 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:45.606828  8598 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:45.615492  8557 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.612880  8483 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.619233  8537 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:45.617969  8665 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:45.619432  8367 raft_consensus.cc:1068] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65: Unable to promote non-voter 20a202c76244485fbc8f1610473db8b9: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:17:45.635277  6622 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:17:45.663300  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:45.625944  8656 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:17:45.652220  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:45.631616  8432 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.644424  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:17:45.668047  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:45.631457  8643 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:45.639163  8484 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:45.639066  8667 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:45.639696  8537 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.632951  8583 consensus_queue.cc:1048] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 12:17:45.637642  8557 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:45.629715  8637 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:45.663545  8665 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:45.665237  8605 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:45.693363  8511 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.679273  8643 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.691219  8537 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:45.696714  8666 consensus_queue.cc:1048] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.084s
I20260709 12:17:45.639776  8598 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:45.697553  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 210392us
I20260709 12:17:45.729347  8598 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:45.732218  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:17:45.698060  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 108102us
I20260709 12:17:45.699137  8497 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:45.699807  8559 raft_consensus.cc:3055] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:45.681854  8432 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:45.700989  8552 consensus_queue.cc:237] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:45.660848  8483 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:45.720500  8511 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:45.724381  8600 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.133s
I20260709 12:17:45.690062  8667 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.743748  8679 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:45.745828  8432 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:45.746333  8367 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:45.748111  6649 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:17:45.748636  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:45.772751  7273 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:45.712435  8643 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:45.754480  8646 consensus_meta.cc:287] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.515s	user 0.005s	sys 0.005s
I20260709 12:17:45.755388  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 167) took 1160 ms. Trace:
W20260709 12:17:45.760102  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 217273us
I20260709 12:17:45.775125  6401 catalog_manager.cc:5697] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 1 to 4, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 4 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:45.772562  8666 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:45.782606  8646 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:45.757190  8671 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.077s
I20260709 12:17:45.790901  8646 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:45.788489  8666 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:45.793721  8666 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:45.778271  6508 rpcz_store.cc:276] 0709 12:17:44.597814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:44.603203 (+  5389us) service_pool.cc:224] Handling call
0709 12:17:44.614199 (+ 10996us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:44.631003 (+ 16804us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:44.631051 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:44.638507 (+  7456us) log.cc:844] Serialized 26 byte log entry
0709 12:17:45.450001 (+811494us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b18004114c0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:17:45.593254 (+143253us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:45.593316 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:45.593411 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:45.755121 (+161710us) raft_consensus.cc:1685] finished
0709 12:17:45.755140 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:45.755331 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30822,"raft.queue_time_us":3353,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":29652,"spinlock_wait_cycles":34075904,"thread_start_us":142519,"threads_started":1,"wal-append.queue_time_us":1084413}
W20260709 12:17:45.757366  8660 log.cc:927] Time spent T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 12:17:45.788431  8679 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:45.797043  6576 raft_consensus.cc:1275] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 12:17:45.797432  6576 pending_rounds.cc:85] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 6
I20260709 12:17:45.797665  6576 pending_rounds.cc:107] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 12:17:45.797948  6576 raft_consensus.cc:2889] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:45.797976  8679 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:45.799197  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:45.799960  6580 raft_consensus.cc:2368] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 4. Current term is 5.
I20260709 12:17:45.737035  8559 raft_consensus.cc:740] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:45.802846  8559 consensus_queue.cc:260] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:45.803907  8559 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:45.809597  8483 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:45.809724  8557 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:45.812001  8557 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:45.813033  8652 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:45.822218  8652 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:45.781447  6652 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7.
I20260709 12:17:45.774366  8367 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:17:45.774045  8648 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.820883  8661 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 12:17:45.816139  8637 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:45.816983  6583 raft_consensus.cc:1240] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:45.769241  8443 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.818221  6548 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:45.814882  8511 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:45.819861  8468 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:45.825695  8367 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:17:45.827255  8563 consensus_meta.cc:287] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.544s	user 0.011s	sys 0.002s
W20260709 12:17:45.829010  8674 log.cc:927] Time spent T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 12:17:45.828612  8648 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:45.874810  8648 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:45.885560  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:45.887110  7274 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:45.890861  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:45.844094  6657 raft_consensus.cc:1275] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:17:45.855924  8635 raft_consensus.cc:3055] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:45.894038  8635 raft_consensus.cc:740] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:45.895208  8635 consensus_queue.cc:260] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:45.857847  8451 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:45.897377  8451 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:45.858175  8443 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:45.904537  8443 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:45.859478  6548 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:45.907171  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 47823.1us
I20260709 12:17:45.859818  8653 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:45.909385  8653 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:45.860093  8536 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:45.912030  6548 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:45.865048  6550 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:45.863240  8468 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:45.913270  8635 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:45.773311  8667 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:45.953891  8451 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:45.957350  6649 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7. Ops: []
W20260709 12:17:45.963104  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:45.963738  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 328558us
I20260709 12:17:46.003198  6622 leader_election.cc:400] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7.
I20260709 12:17:46.003593  6622 leader_election.cc:403] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:17:46.003804  7238 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:45.964236  8682 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:46.007608  8682 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:46.008209  7238 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:45.969074  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 299565us
I20260709 12:17:46.012679  8684 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:46.013552  8543 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:46.013314  8550 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:46.014194  8543 raft_consensus.cc:697] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 2 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:46.015355  8543 consensus_queue.cc:237] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:46.013321  8684 raft_consensus.cc:697] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:46.019814  8684 consensus_queue.cc:237] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:46.024178  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:46.025170  6655 raft_consensus.cc:2393] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 3: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:46.026005  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:46.026746  6653 raft_consensus.cc:2393] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 7: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:17:46.030237  6654 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:17:46.031164  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:46.031868  6654 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:46.000511  8665 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:46.038610  8637 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7.
I20260709 12:17:46.038815  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:46.040869  6655 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:46.042316  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:46.043184  6657 raft_consensus.cc:2393] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 2: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:46.045830  8682 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:45.757540  8605 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:45.989430  8648 raft_consensus.cc:2764] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
W20260709 12:17:45.989899  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 124960us
I20260709 12:17:45.990531  8640 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:17:45.990641  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 347314us
I20260709 12:17:46.049054  8640 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:46.049204  6550 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:46.049356  6576 raft_consensus.cc:1275] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:17:46.049492  8640 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:46.050076  6550 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:46.051030  8656 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:46.051120  7240 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:46.051441  8656 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:17:45.996809  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 84873.9us
I20260709 12:17:46.051867  8656 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.052366  6548 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:46.053167  8659 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:46.053661  8659 raft_consensus.cc:697] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:46.054586  8659 consensus_queue.cc:237] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:46.057704  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:46.058499  6576 raft_consensus.cc:2410] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:17:46.058439  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:46.059298  6653 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
I20260709 12:17:46.061585  7274 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:17:46.061208  8640 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:46.061972  7274 pending_rounds.cc:85] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9: Aborting all ops after (but not including) 5
I20260709 12:17:46.061926  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:46.062439  7274 pending_rounds.cc:107] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:17:46.062525  8563 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:46.062718  7274 raft_consensus.cc:2889] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:46.062885  8443 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.005250  6622 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:46.063360  8443 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:46.064219  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 59108.5us
I20260709 12:17:46.065250  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:46.065922  7271 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:46.065979  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:46.066346  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:46.066864  8637 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:46.067018  8483 consensus_queue.cc:1059] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:17:46.006103  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:46.068013  8483 raft_consensus.cc:3055] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:46.054107  6548 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:46.068449  6474 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:46.068974  8563 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:46.069661  8648 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:46.070477  8557 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:46.071772  6548 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:46.055332  8536 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:46.004103  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:46.076393  6580 raft_consensus.cc:2410] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:45.976559  6658 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:17:46.071888  8468 raft_consensus.cc:1064] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:17:46.073563  6622 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:46.068462  8483 raft_consensus.cc:740] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:46.031634  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:46.062937  6655 raft_consensus.cc:2393] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 6: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:46.070202  8367 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:46.080443  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 48894.1us
I20260709 12:17:46.081423  6550 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:46.078519  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:46.090477  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:46.078876  8483 consensus_queue.cc:260] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:46.091020  8593 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:17:46.091594  8483 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:46.092175  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:46.093130  6580 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:17:46.101217  8653 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:46.101992  8653 raft_consensus.cc:697] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:46.103271  8653 consensus_queue.cc:237] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:46.110057  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:46.111001  6501 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 6.
I20260709 12:17:46.112967  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:46.113999  6501 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
W20260709 12:17:46.120455  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:46.096997  6622 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:46.123404  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:46.124109  6658 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:17:46.089807  6654 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:17:46.136111  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:46.136953  6655 raft_consensus.cc:2393] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 5: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:17:46.097003  6473 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.506s (SENT) [suppressed 12 similar messages]
I20260709 12:17:46.096451  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:46.150318  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 63910.4us
I20260709 12:17:46.151384  6622 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:17:46.093312  8468 raft_consensus.cc:1068] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65: Unable to promote non-voter 20a202c76244485fbc8f1610473db8b9: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:17:46.158059  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:46.092881  8552 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:17:46.172047  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:46.079990  8543 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:46.095131  8563 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:46.095371  8637 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:46.260824  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:46.170771  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:46.095444  8451 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.095252  8176 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.122181  8570 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.095829  8536 raft_consensus.cc:697] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
W20260709 12:17:46.258958  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:46.095150  8648 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.095s
I20260709 12:17:46.186966  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:46.166049  8642 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:46.093619  8683 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:46.283025  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:46.259613  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:46.165473  8605 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:46.170181  8497 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:46.262279  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:46.262749  6505 raft_consensus.cc:1275] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:46.104068  8665 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:46.267664  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:46.268141  6652 raft_consensus.cc:1275] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:46.268747  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:46.297165  6475 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:46.307497  6500 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:46.277424  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:46.186472  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:46.278074  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:46.280501  8641 raft_consensus.cc:2764] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 12:17:46.313469  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:17:46.279717  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 182729us
I20260709 12:17:46.139763  8598 consensus_queue.cc:1059] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.130s
I20260709 12:17:46.283990  6576 raft_consensus.cc:2393] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 5: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:46.294323  6508 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:17:46.294587  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:46.270478  8637 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:46.262907  8468 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:46.176520  8666 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.113s
I20260709 12:17:46.155001  8682 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.124s
I20260709 12:17:46.284271  6649 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:46.309971  6503 raft_consensus.cc:2393] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 7: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:46.141077  8537 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:46.186373  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:46.314296  6654 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:46.313206  6475 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:46.319058  8653 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.319514  8659 consensus_queue.cc:1048] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:46.320204  8664 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:46.320484  8646 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.321516  8511 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:46.323925  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:46.324949  6509 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:46.330276  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:46.140697  8643 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:46.261711  8543 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:46.332088  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:46.333348  6653 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:17:46.281018  8570 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:46.301542  8563 raft_consensus.cc:697] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:46.438107  8668 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:46.439072  8668 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:46.439270  8563 consensus_queue.cc:237] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:46.346295  6655 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:17:46.278352  8176 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:46.357487  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 177486us
I20260709 12:17:46.326880  8679 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:46.281932  8451 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:46.315596  8642 raft_consensus.cc:697] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 7 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:46.334748  8637 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:46.369802  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:46.332517  6502 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 4.
I20260709 12:17:46.486670  6650 raft_consensus.cc:2393] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 6: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:17:46.376148  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:46.488885  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:46.373142  6394 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 6f94b1cd286648a9b82d5abf69bb86d5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica f2b06aebe97249a4877d798b8118ece4 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:17:46.489799  6394 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 116728us
I20260709 12:17:46.444808  6500 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:17:46.447324  6401 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57888 (request call id 214) took 10252 ms. Trace:
I20260709 12:17:46.473225  6506 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:46.326537  8667 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:46.357165  8468 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:46.490857  6550 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:46.471870  6654 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:17:46.485865  6576 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:46.369728  8656 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:46.574503  6649 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:17:46.555157  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:46.654547  6505 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
W20260709 12:17:46.553861  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 106938us
I20260709 12:17:46.529116  6504 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
W20260709 12:17:46.442984  8697 log.cc:927] Time spent T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:17:46.679826  6649 raft_consensus.cc:1275] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:46.370124  8653 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:46.448236  8367 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:17:46.704288  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 46478.9us
I20260709 12:17:46.472927  8679 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:46.701681  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:46.708874  6576 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:46.465253  8557 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:46.706477  6653 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:46.701864  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:46.723886  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:46.724543  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:46.417462  8511 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:46.718350  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:46.729353  6505 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:46.566787  8537 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:46.441052  8680 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.600s
I20260709 12:17:46.654744  8432 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:46.734999  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 421819us
W20260709 12:17:46.753580  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 573511us
I20260709 12:17:46.408937  8664 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:46.693049  8684 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.008s	sys 0.006s
I20260709 12:17:46.440986  8646 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:46.691984  8443 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:46.757243  6475 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:46.700654  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:46.732201  7238 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:17:46.700582  7240 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.048s	sys 0.085s
I20260709 12:17:46.724046  8679 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:46.761907  6576 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:46.465760  8536 consensus_queue.cc:237] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:46.697402  8640 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:46.766189  7240 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.048s	sys 0.085s
W20260709 12:17:46.771822  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 583942us
I20260709 12:17:46.758759  8557 raft_consensus.cc:697] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:46.759503  8552 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:17:46.764786  8642 consensus_queue.cc:237] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:46.791857  8653 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:46.793565  8468 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:46.719877  8367 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:46.797638  6577 raft_consensus.cc:1275] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:17:46.762704  8684 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.008s	sys 0.006s
I20260709 12:17:46.763012  8665 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:46.818706  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:46.671113  6401 rpcz_store.cc:276] 0709 12:17:36.195130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:36.195503 (+   373us) service_pool.cc:224] Handling call
0709 12:17:46.447251 (+10251748us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:46.001563 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:46.002269 (+   706us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:46.002312 (+    43us) write_op.cc:435] Acquired schema lock
0709 12:17:46.002329 (+    17us) tablet.cc:662] Decoding operations
0709 12:17:46.003005 (+   676us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:46.003117 (+   112us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:46.003141 (+    24us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:17:46.003589 (+   448us) tablet.cc:701] Row locks acquired
0709 12:17:46.003636 (+    47us) write_op.cc:260] PREPARE: finished
0709 12:17:46.003811 (+   175us) write_op.cc:270] Start()
0709 12:17:46.003956 (+   145us) write_op.cc:276] Timestamp: P: 1783599466003749 usec, L: 0
0709 12:17:46.003991 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 12:17:46.005792 (+  1801us) log.cc:844] Serialized 1290 byte log entry
0709 12:17:46.110081 (+104289us) op_driver.cc:464] REPLICATION: finished
0709 12:17:46.131188 (+ 21107us) write_op.cc:301] APPLY: starting
0709 12:17:46.131295 (+   107us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:46.131507 (+   212us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:46.131631 (+   124us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:46.132264 (+   633us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:46.132536 (+   272us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 12:17:46.132596 (+    60us) write_op.cc:312] APPLY: finished
0709 12:17:46.132864 (+   268us) log.cc:844] Serialized 35 byte log entry
0709 12:17:46.133165 (+   301us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:46.133255 (+    90us) write_op.cc:454] Released schema lock
0709 12:17:46.133309 (+    54us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:46.262295 (+128986us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:46.267321 (+  5026us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":25301,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":20804,"apply.run_cpu_time_us":2641,"apply.run_wall_time_us":25646,"num_ops":3,"prepare.queue_time_us":194440,"prepare.run_cpu_time_us":10374,"prepare.run_wall_time_us":13948,"raft.queue_time_us":94021,"raft.run_cpu_time_us":3739,"raft.run_wall_time_us":21166,"replication_time_us":105868,"spinlock_wait_cycles":544000,"thread_start_us":315897,"threads_started":4,"wal-append.queue_time_us":7006}]]}
I20260709 12:17:46.778070  8668 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:46.775282  8635 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:46.761067  8432 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:46.864472  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:46.775657  8598 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:46.798077  8570 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:46.865562  6509 raft_consensus.cc:2393] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 6: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:46.861805  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:46.855916  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:46.848400  6502 raft_consensus.cc:2393] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 6: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:46.819952  8665 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:46.849767  8483 consensus_meta.cc:287] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.758s	user 0.006s	sys 0.005s
I20260709 12:17:46.869298  8598 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:46.872251  8635 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:46.906761  8598 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:47.089131  6652 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.002s	sys 0.000s
I20260709 12:17:46.972934  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:17:47.089646  6652 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.002s	sys 0.000s
I20260709 12:17:47.090139  6583 raft_consensus.cc:2410] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:17:47.091039  6502 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 12:17:47.093600  8483 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:46.937282  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 124645us
I20260709 12:17:47.059157  8667 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:47.029808  8451 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:46.888221  8665 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:47.095675  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 325510us
I20260709 12:17:46.938755  8176 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:46.938511  8689 log.cc:927] Time spent T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.472s	user 0.001s	sys 0.000s
I20260709 12:17:46.902282  8543 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:47.100266  8641 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:47.110594  8641 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:47.086251  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:47.105129  8483 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:47.106585  6622 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:46.866158  8712 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:17:46.911329  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 420519us
I20260709 12:17:47.105398  6584 raft_consensus.cc:1275] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
W20260709 12:17:47.113309  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 343638us
W20260709 12:17:47.071512  6474 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.006s	sys 0.006s
I20260709 12:17:47.123853  6505 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:17:47.112282  6506 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:17:47.110044  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 1734) took 1222 ms. Trace:
W20260709 12:17:47.143158  6474 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.007s	sys 0.007s
W20260709 12:17:47.024466  8713 log.cc:927] Time spent T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 12:17:46.907954  8432 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:47.146366  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:47.194416  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:47.191802  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:47.142906  6656 rpcz_store.cc:276] 0709 12:17:45.887354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:45.887587 (+   233us) service_pool.cc:224] Handling call
0709 12:17:45.887909 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:45.888285 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:45.888312 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:45.888855 (+   543us) log.cc:844] Serialized 26 byte log entry
0709 12:17:46.288645 (+399790us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:46.288708 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:46.288808 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:47.109827 (+821019us) raft_consensus.cc:1685] finished
0709 12:17:47.109863 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:47.109989 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2013,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":145,"thread_start_us":399284,"threads_started":1,"wal-append.queue_time_us":456459}
I20260709 12:17:47.198133  6506 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:47.198343  6579 raft_consensus.cc:2393] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 2: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:47.199242  6584 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:47.188258  6651 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:17:47.200615  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 94173us
W20260709 12:17:47.120169  8705 log.cc:927] Time spent T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.351s	user 0.003s	sys 0.007s
W20260709 12:17:47.148620  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:47.179592  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:47.174428  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:47.211014  6653 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:47.149576  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:47.023612  8701 log.cc:927] Time spent T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.371s	user 0.001s	sys 0.000s
I20260709 12:17:47.205158  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:47.125012  8664 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:47.221293  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:47.222702  6505 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:46.933668  8643 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:47.125895  8646 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:47.146297  8714 log.cc:927] Time spent T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.270s	user 0.004s	sys 0.006s
W20260709 12:17:47.149293  8550 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.009s	sys 0.013s
I20260709 12:17:47.112286  8635 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:17:47.229089  6548 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.017s	sys 0.007s
I20260709 12:17:47.145653  8719 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:47.149442  8451 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:47.234059  6548 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.018s	sys 0.007s
I20260709 12:17:47.236974  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:47.237954  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:47.145744  8709 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:47.233505  8550 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.009s	sys 0.018s
I20260709 12:17:47.291106  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:47.227581  8640 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:47.149515  8667 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:47.296700  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 91640.2us
I20260709 12:17:47.297917  6658 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:47.316939  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:47.317880  6658 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:47.318797  6506 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:17:47.319082  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:47.319967  6658 raft_consensus.cc:2393] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 6: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:47.305122  6579 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:47.291740  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 53846.9us
I20260709 12:17:47.308485  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:47.322844  6651 raft_consensus.cc:2393] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 6: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:47.231137  8712 raft_consensus.cc:2764] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
W20260709 12:17:47.324810  6655 consensus_meta.cc:287] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.873s	user 0.006s	sys 0.014s
I20260709 12:17:47.331903  6655 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:17:47.326764  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:47.234787  8664 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:47.331307  6550 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:47.334095  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:47.336763  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:47.243845  8367 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:47.339648  6658 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 3.
I20260709 12:17:47.340562  6505 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:17:47.345330  6500 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 1565) took 1179 ms. Trace:
I20260709 12:17:47.345656  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 1564) took 1184 ms. Trace:
I20260709 12:17:47.346966  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:47.334231  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:47.340822  7270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38558 (request call id 371) took 1267 ms. Trace:
I20260709 12:17:47.343817  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58980 (request call id 1742) took 1184 ms. Trace:
I20260709 12:17:46.933244  8557 consensus_queue.cc:237] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:47.307797  8635 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:47.344602  6584 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:17:47.346580  6500 rpcz_store.cc:276] 0709 12:17:46.161499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:46.707377 (+545878us) service_pool.cc:224] Handling call
0709 12:17:46.731433 (+ 24056us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:46.755537 (+ 24104us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:17:47.089376 (+333839us) log.cc:844] Serialized 14 byte log entry
0709 12:17:47.340419 (+251043us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:47.340496 (+    77us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:47.340545 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:47.340620 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:47.340736 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12972,"prepare.queue_time_us":325038,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":22169,"thread_start_us":526076,"threads_started":2}
I20260709 12:17:47.350715  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 1569) took 1187 ms. Trace:
I20260709 12:17:47.359674  6508 rpcz_store.cc:276] 0709 12:17:46.162879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:46.727700 (+564821us) service_pool.cc:224] Handling call
0709 12:17:46.758036 (+ 30336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:46.768597 (+ 10561us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:46.768632 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:46.769098 (+   466us) log.cc:844] Serialized 26 byte log entry
0709 12:17:46.983521 (+214423us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:47.202977 (+219456us) log.cc:844] Serialized 14 byte log entry
0709 12:17:47.222662 (+ 19685us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:47.223226 (+   564us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:47.350503 (+127277us) raft_consensus.cc:1685] finished
0709 12:17:47.350528 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:47.350655 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46747,"prepare.queue_time_us":103599,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":102112,"raft.run_cpu_time_us":11309,"raft.run_wall_time_us":91165,"thread_start_us":384982,"threads_started":2,"wal-append.queue_time_us":326451}
I20260709 12:17:47.380927  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:47.382690  6548 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:47.349210  6655 rpcz_store.cc:276] 0709 12:17:46.154080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:46.181133 (+ 27053us) service_pool.cc:224] Handling call
0709 12:17:47.337972 (+1156839us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:47.346145  6507 rpcz_store.cc:276] 0709 12:17:46.161080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:46.705766 (+544686us) service_pool.cc:224] Handling call
0709 12:17:46.756206 (+ 50440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:46.761696 (+  5490us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:17:47.092644 (+330948us) log.cc:844] Serialized 14 byte log entry
0709 12:17:47.345319 (+252675us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:47.345372 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:47.345424 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:47.345503 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:47.345602 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28299,"prepare.queue_time_us":317959,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":25,"thread_start_us":492522,"threads_started":2}
I20260709 12:17:47.337415  8643 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:47.241163  8719 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:47.380916  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 49749.1us
I20260709 12:17:47.425335  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:47.439954  6576 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:17:47.348551  7270 rpcz_store.cc:276] 0709 12:17:46.073531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:46.073695 (+   164us) service_pool.cc:224] Handling call
0709 12:17:46.078339 (+  4644us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:46.079390 (+  1051us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:46.079423 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:46.085782 (+  6359us) log.cc:844] Serialized 26 byte log entry
0709 12:17:46.318505 (+232723us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:46.318547 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:46.318611 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:47.340609 (+1021998us) raft_consensus.cc:1685] finished
0709 12:17:47.340631 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:47.340764 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4182,"raft.queue_time_us":1250,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":9379,"thread_start_us":225676,"threads_started":1,"wal-append.queue_time_us":585120}
I20260709 12:17:47.342543  8664 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:47.347401  8715 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:47.429040  6506 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:47.427163  8643 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:47.326329  8451 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:47.431551  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 184) took 1314 ms. Trace:
I20260709 12:17:47.333570  8640 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:47.458914  6475 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:47.433915  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:47.346149  8710 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:47.334412  8667 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:47.463055  8667 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:47.384382  8717 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:47.463693  8717 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:47.387773  6656 raft_consensus.cc:2393] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 6: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:47.448436  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:47.432901  8726 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:47.466420  6576 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:47.450981  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:47.467296  6507 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:17:47.387274  8635 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:47.455425  6501 rpcz_store.cc:276] 0709 12:17:46.117551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:46.117689 (+   138us) service_pool.cc:224] Handling call
0709 12:17:46.118037 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:46.148247 (+ 30210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:46.148291 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:46.189312 (+ 41021us) log.cc:844] Serialized 26 byte log entry
0709 12:17:46.693360 (+504048us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:46.693424 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:46.693495 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:47.431348 (+737853us) raft_consensus.cc:1685] finished
0709 12:17:47.431369 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:47.431495 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4671,"raft.queue_time_us":4823,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":16160,"thread_start_us":392022,"threads_started":1,"wal-append.queue_time_us":489639}
I20260709 12:17:47.481380  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:47.481887  8719 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:47.483170  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:47.485091  6501 raft_consensus.cc:1275] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:47.457041  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:47.387302  8646 consensus_queue.cc:1048] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 12:17:47.458693  8653 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:47.460319  6655 raft_consensus.cc:2393] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 7: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:47.498221  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:47.429571  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 46981.1us
I20260709 12:17:47.502735  6653 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:17:47.503194  6548 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:47.506724  6548 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:47.507649  6548 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:47.455339  8715 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:47.459427  8483 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:47.508795  8483 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:47.509179  8483 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:47.509557  8483 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:47.510111  8715 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:47.510702  8715 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:47.511114  8715 raft_consensus.cc:697] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:47.474751  8640 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:47.512673  8640 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:47.454192  8641 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:47.476367  8451 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:47.477365  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:47.525497  6656 raft_consensus.cc:2393] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 6: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:47.477900  8643 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:47.478861  8665 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:47.494331  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 202295us
I20260709 12:17:47.526583  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:47.527287  6507 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 1.
I20260709 12:17:47.514209  6653 raft_consensus.cc:1275] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:17:47.528939  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 1570) took 1365 ms. Trace:
I20260709 12:17:47.529204  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:47.529865  6653 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:47.513760  8653 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:47.520531  8640 raft_consensus.cc:697] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:47.521253  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:47.513424  8570 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:47.525312  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:47.542510  6475 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:47.493470  8717 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:47.543376  8570 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:47.543949  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:47.500475  8484 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:47.575546  8484 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:47.577781  8484 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:47.579077  8730 log.cc:927] Time spent T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.471s	user 0.000s	sys 0.002s
I20260709 12:17:47.495477  8738 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:47.545948  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:47.584292  6656 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
W20260709 12:17:47.546828  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 89870.5us
I20260709 12:17:47.548655  8684 raft_consensus.cc:3055] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:47.549610  6651 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:47.552644  8725 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:47.554153  8719 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:47.556044  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:47.556288  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:47.556738  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:47.594091  6583 raft_consensus.cc:2368] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 4. Current term is 5.
I20260709 12:17:47.595055  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:47.557062  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:47.557476  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:47.558440  8637 consensus_meta.cc:287] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 1.014s	user 0.010s	sys 0.001s
I20260709 12:17:47.597282  8637 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:47.599968  8637 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:47.559068  8665 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:47.600936  6657 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:47.558888  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:47.560204  8728 log.cc:927] Time spent T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 12:17:47.560611  8653 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:17:47.560926  8679 consensus_meta.cc:287] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.779s	user 0.010s	sys 0.001s
I20260709 12:17:47.604668  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 1566) took 1442 ms. Trace:
I20260709 12:17:47.604552  8679 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:47.605382  6503 rpcz_store.cc:276] 0709 12:17:46.161830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:46.726855 (+565025us) service_pool.cc:224] Handling call
0709 12:17:46.727260 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:46.756807 (+ 29547us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:46.756834 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:46.769492 (+ 12658us) log.cc:844] Serialized 26 byte log entry
0709 12:17:47.224378 (+454886us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:47.224441 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:47.224539 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:47.604445 (+379906us) raft_consensus.cc:1685] finished
0709 12:17:47.604464 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:47.604608 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":58319,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":17602,"thread_start_us":362527,"threads_started":1,"wal-append.queue_time_us":524735}
I20260709 12:17:47.562537  8635 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:47.564968  8643 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:47.607796  8643 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:47.610466  8643 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:47.611155  8653 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:47.611814  8653 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:47.615970  8653 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:47.568622  8598 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:47.570643  8570 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:47.572670  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:47.620726  6507 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
I20260709 12:17:47.573062  8640 consensus_queue.cc:237] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:47.574609  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:47.590965  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:47.624779  6577 raft_consensus.cc:2368] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 6. Current term is 7.
I20260709 12:17:47.624706  6651 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:47.625623  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:47.626573  6651 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:17:47.627898  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:47.567211  6579 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:17:47.629004  6651 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:47.591876  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:47.629791  6656 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:17:47.630693  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:47.631758  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:47.632794  6656 raft_consensus.cc:2410] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20260709 12:17:47.636714  6651 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:17:47.511946  8715 consensus_queue.cc:237] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:47.648833  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:47.645454  8563 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:47.653293  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:47.654215  6584 raft_consensus.cc:2410] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:47.544448  8726 raft_consensus.cc:697] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:47.655637  6579 raft_consensus.cc:2393] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 3: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:47.662696  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:47.664106  6655 raft_consensus.cc:2410] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:47.639608  8570 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:47.593303  8725 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:47.589845  8684 raft_consensus.cc:740] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:47.671203  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:47.672910  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:47.674609  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:47.676775  8726 consensus_queue.cc:237] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:47.678972  8511 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:47.683053  8679 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:47.670014  8725 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:47.685621  8664 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:47.688745  8684 consensus_queue.cc:260] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:47.690222  8684 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:47.690399  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:47.691267  6584 raft_consensus.cc:2393] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 5: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:47.663170  8709 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:47.710685  8557 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:47.711432  8557 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:47.698553  8738 raft_consensus.cc:2764] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 12:17:47.529337  6509 rpcz_store.cc:276] 0709 12:17:46.163119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:46.866366 (+703247us) service_pool.cc:224] Handling call
0709 12:17:47.087026 (+220660us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:47.088231 (+  1205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:47.088276 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:47.106619 (+ 18343us) log.cc:844] Serialized 52 byte log entry
0709 12:17:47.436584 (+329965us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:47.468661 (+ 32077us) log.cc:844] Serialized 14 byte log entry
0709 12:17:47.500750 (+ 32089us) log.cc:844] Serialized 14 byte log entry
0709 12:17:47.528647 (+ 27897us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:47.528721 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:47.528765 (+    44us) raft_consensus.cc:1685] finished
0709 12:17:47.528777 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:47.528890 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80303,"prepare.queue_time_us":32403,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":79,"raft.queue_time_us":73769,"raft.run_cpu_time_us":308,"raft.run_wall_time_us":9429,"thread_start_us":298742,"threads_started":1,"wal-append.queue_time_us":362707}
I20260709 12:17:47.717695  8483 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:47.719213  8709 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:47.714534  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:47.767366  6654 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:47.768428  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:17:47.778411  8727 log.cc:927] Time spent T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.454s	user 0.000s	sys 0.002s
I20260709 12:17:47.780844  8570 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:47.781243  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:47.781863  6654 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:47.803315  8598 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:47.803555  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:47.804481  6584 raft_consensus.cc:2410] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:17:47.804490  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1253) took 1685 ms. Trace:
I20260709 12:17:47.805570  6652 rpcz_store.cc:276] 0709 12:17:46.119041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:46.305357 (+186316us) service_pool.cc:224] Handling call
0709 12:17:46.305725 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:46.340472 (+ 34747us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 12:17:46.340519 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:46.341139 (+   620us) log.cc:844] Serialized 26 byte log entry
0709 12:17:47.090059 (+748920us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:47.090101 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:47.090175 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:47.719958 (+629783us) spinlock_profiling.cc:243] Waited 7.27 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:17:47.781928 (+ 61970us) raft_consensus.cc:1685] finished
0709 12:17:47.781948 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:47.804417 (+ 22469us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":528,"raft.queue_time_us":19290,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":17459,"spinlock_wait_cycles":16711040,"thread_start_us":649449,"threads_started":1,"wal-append.queue_time_us":822008}
I20260709 12:17:47.806730  6657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 1740) took 1725 ms. Trace:
I20260709 12:17:47.807102  6657 rpcz_store.cc:276] 0709 12:17:46.081002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:46.296865 (+215863us) service_pool.cc:224] Handling call
0709 12:17:46.316581 (+ 19716us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:47.601416 (+1284835us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:47.806521 (+205105us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b540044acd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:47.806680 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2952990464}
I20260709 12:17:47.805190  6649 raft_consensus.cc:1275] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:47.817478  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:47.818892  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 146106us
I20260709 12:17:47.821575  8640 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:47.821439  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:47.820020  8664 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:47.822466  8640 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:47.823056  6622 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:47.823531  6654 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:47.825321  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:47.825963  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:47.829905  6657 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:47.834364  8483 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:47.826691  8570 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:17:47.827688  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 320007us
I20260709 12:17:47.837364  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:47.845687  6548 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:47.839542  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:47.842334  6509 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:17:47.846529  6577 raft_consensus.cc:2393] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 2: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:47.838111  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:47.843000  8725 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:47.844765  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:47.848841  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:47.845896  6581 raft_consensus.cc:2410] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:17:47.846657  6548 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:47.847054  6583 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:47.850905  8570 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:47.851737  8511 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:47.843991  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:47.852216  8511 raft_consensus.cc:697] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:47.852631  6575 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:47.852641  8738 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:47.853117  8511 consensus_queue.cc:237] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:47.822639  8664 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:47.855546  8664 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:47.856815  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:47.857496  6575 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:47.829036  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:47.849452  6579 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:47.851236  6548 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:47.860244  7240 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:47.867465  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:47.869801  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:47.870575  6655 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:17:47.872017  6548 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:47.890589  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:47.871510  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:47.884960  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:47.894778  6582 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:47.894126  6576 raft_consensus.cc:2376] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7. Re-sending same reply.
I20260709 12:17:47.895644  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:47.899403  8664 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:47.899941  8664 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:47.901083  8570 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:47.901901  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:47.903043  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:47.998000  8557 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:48.000972  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:48.024718  6582 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 4.
I20260709 12:17:48.025172  6508 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:48.004487  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:48.025941  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:48.026718  6581 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 3.
W20260709 12:17:47.966964  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 62082.5us
W20260709 12:17:47.922503  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 99536.9us
I20260709 12:17:47.866446  8483 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:47.907900  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:47.921736  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:48.030483  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:48.034472  6503 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:17:48.034700  6576 raft_consensus.cc:2393] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 3: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:48.031303  6650 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:48.031497  6583 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:17:48.035964  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:47.976408  6654 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:17:47.929946  6657 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:47.921258  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:48.036109  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:48.033810  6655 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:48.037545  6583 raft_consensus.cc:1240] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:17:47.922010  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:47.867786  8684 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:47.893136  8563 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:17:48.043900  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 49521.3us
I20260709 12:17:48.046512  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:47.869796  8704 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:17:47.921991  8754 log.cc:927] Time spent T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.097s	user 0.000s	sys 0.002s
I20260709 12:17:48.023874  8664 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:48.050629  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:48.057360  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:48.057507  6507 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:48.051934  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:47.870072  8707 consensus_queue.cc:1048] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.175s
I20260709 12:17:48.048249  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:47.921116  8726 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:48.054652  6548 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:48.055529  6577 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:48.074754  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:48.043660  8745 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:48.054214  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:48.009909  8640 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:48.070252  7238 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:48.054406  8704 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:48.004755  8757 consensus_queue.cc:1048] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.182s
I20260709 12:17:48.075484  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:48.050658  8563 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:48.043251  8684 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:48.058849  6506 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:17:48.073088  6582 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:48.073912  8738 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:48.086763  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:48.089493  6508 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 3.
I20260709 12:17:48.090593  6655 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 3.
I20260709 12:17:48.092145  6507 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
W20260709 12:17:48.077436  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 80717.9us
I20260709 12:17:48.044983  8605 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:17:48.065727  8483 raft_consensus.cc:697] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:48.076581  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:48.085027  8745 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:48.143260  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:48.143648  6576 raft_consensus.cc:2393] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 3: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:17:48.165354  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 90702us
I20260709 12:17:48.141568  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:48.209553  6580 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 1222) took 1898 ms (client timeout 2110 ms). Trace:
I20260709 12:17:48.085268  8726 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:48.361548  6577 raft_consensus.cc:2368] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 6. Current term is 7.
W20260709 12:17:48.169644  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:48.196996  6506 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
W20260709 12:17:48.143461  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 88892.6us
I20260709 12:17:48.114957  8684 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:48.115396  8563 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:48.141868  8738 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:48.380545  6622 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:48.380987  8543 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 12:17:48.382507  8726 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:48.382886  8738 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:48.383677  8738 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:48.384044  8738 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:48.148798  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:48.386899  6580 rpcz_store.cc:269] 0709 12:17:46.311444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:46.323692 (+ 12248us) service_pool.cc:224] Handling call
0709 12:17:48.209221 (+1885529us) spinlock_profiling.cc:243] Waited 734 ms on lock 0x7b540034b2d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:48.209462 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1687713024}
I20260709 12:17:48.377784  6548 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:48.091280  8704 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:48.396708  8483 consensus_queue.cc:237] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:48.403252  8656 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:48.405551  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:48.407310  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:48.408054  7273 raft_consensus.cc:2410] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:17:48.409677  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:48.409858  8725 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:48.410210  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:48.088362  8707 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.323s
I20260709 12:17:48.410995  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:48.091446  8761 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:48.093514  8643 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:48.415433  6548 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:48.417742  6548 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:48.424312  8640 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:48.424779  8640 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:48.425115  8640 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:48.425868  8570 raft_consensus.cc:1064] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:17:48.375082  8665 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:48.426297  8665 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:48.426653  8665 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:48.428562  8744 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:48.428610  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:48.429435  8744 raft_consensus.cc:697] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 7 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:48.430343  8744 consensus_queue.cc:237] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:48.428197  8570 consensus_queue.cc:237] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 4, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:48.440629  6506 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:48.443735  6508 raft_consensus.cc:1240] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:17:48.445535  6508 raft_consensus.cc:1275] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:48.412856  8738 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:48.454845  6549 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:48.454949  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:48.455770  6655 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:48.458527  8704 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:48.459653  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:48.464697  7273 raft_consensus.cc:2393] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 5: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:17:48.462182  8715 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:48.465255  6506 raft_consensus.cc:1275] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:48.465659  8704 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:48.465381  8715 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:48.465852  6655 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:48.466781  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:48.467190  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:48.467758  8715 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:48.468063  7270 raft_consensus.cc:2393] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 3: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:17:48.468695  8511 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:48.469570  6502 raft_consensus.cc:1275] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:48.468729  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:48.479877  6655 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:17:48.415282  8717 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:48.488701  8717 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:48.490047  6403 catalog_manager.cc:5697] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 3 to 4, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 4 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:48.490960  8717 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:48.423980  8653 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:48.494706  8709 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:48.500397  8653 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:48.491981  6403 catalog_manager.cc:5697] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 4 to 5, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1), pending config changed. New cstate: current_term: 5 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } } pending_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:17:48.508047  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 135114us
I20260709 12:17:48.507377  8653 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:48.545161  6403 catalog_manager.cc:5697] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 3 to 5, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 5 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:17:48.629900  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 249403us
I20260709 12:17:48.630084  6403 catalog_manager.cc:5697] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
W20260709 12:17:48.631317  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:48.636478  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:48.651481  7270 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:17:48.633908  6403 catalog_manager.cc:5697] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 1 to 5, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 5 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:48.504266  8640 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:48.654695  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:17:48.654814  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 198608us
I20260709 12:17:48.656945  6549 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:48.653676  6403 catalog_manager.cc:5697] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 4 to 5, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 5 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:48.661834  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:48.662532  6657 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:17:48.667181  6403 catalog_manager.cc:5697] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 1 to 3, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 3 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:48.669123  6403 catalog_manager.cc:5697] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 1 to 3, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 3 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:48.671212  6403 catalog_manager.cc:5697] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:17:48.673982  6549 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:48.680042  6403 catalog_manager.cc:5697] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 1 to 2, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 2 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:17:48.691944  7274 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38558 (request call id 367) took 2627 ms (client timeout 2070 ms). Trace:
W20260709 12:17:48.692595  7274 rpcz_store.cc:269] 0709 12:17:46.064127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:46.066129 (+  2002us) service_pool.cc:224] Handling call
0709 12:17:48.691612 (+2625483us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54004d7e54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:48.691797 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3133983616}
I20260709 12:17:48.694298  7274 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 7 LEARNER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:17:48.699299  6549 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:48.700773  8745 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:48.701799  6403 catalog_manager.cc:5697] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 3 to 4, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 4 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:48.703204  7270 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:17:48.711517  6657 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5.
I20260709 12:17:48.715909  6622 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:48.713861  7240 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:48.678552  8709 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:48.721742  6403 catalog_manager.cc:5697] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 1 to 3, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 3 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:48.674835  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:48.726606  7240 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:48.684974  8717 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:48.734082  6403 catalog_manager.cc:5697] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 2 to 5, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 5 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:17:48.736465  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:48.714694  8680 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.023s	sys 0.009s
I20260709 12:17:48.743389  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:48.748165  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:48.717619  8715 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:48.752969  6508 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:48.717422  8757 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:48.772858  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:48.773955  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 150746us
I20260709 12:17:48.675436  8563 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:48.779191  6509 raft_consensus.cc:1275] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:17:48.719564  8768 raft_consensus.cc:2764] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
W20260709 12:17:48.744445  8680 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.024s	sys 0.009s
W20260709 12:17:48.782124  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 66314.7us
I20260709 12:17:48.779861  6507 raft_consensus.cc:1275] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 2. (index mismatch)
I20260709 12:17:48.783335  6508 raft_consensus.cc:1275] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 12:17:48.744313  8605 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:17:48.784881  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:48.741560  8766 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:48.716574  8726 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:48.800631  6579 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:48.795535  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:48.804651  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:48.821422  8768 raft_consensus.cc:2764] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
W20260709 12:17:48.828058  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:48.834410  6622 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:48.780956  8718 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 12:17:48.841660  6507 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:48.802343  8605 raft_consensus.cc:697] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:48.825764  8776 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:48.830031  8766 raft_consensus.cc:697] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:48.811880  8744 consensus_queue.cc:1048] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 12:17:48.811735  8745 consensus_queue.cc:1059] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 12:17:48.811501  8717 consensus_queue.cc:1059] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 12:17:48.851859  8768 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:48.872138  8715 consensus_queue.cc:1048] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 12:17:48.885882  6622 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:48.872969  8640 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:48.888573  8709 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:48.888000  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:48.853942  8774 consensus_queue.cc:1048] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 12:17:48.854300  8776 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:48.892318  8763 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.180s	user 0.000s	sys 0.002s
I20260709 12:17:48.913128  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:48.875905  8684 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:48.900781  8717 raft_consensus.cc:3055] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:48.912971  8766 consensus_queue.cc:237] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:48.897490  8709 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:17:48.924019  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 70984.7us
I20260709 12:17:48.943293  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:48.948709  8726 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:48.949596  8563 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:48.957672  7272 raft_consensus.cc:2393] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 4: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:17:48.927357  8768 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:48.929168  8717 raft_consensus.cc:740] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:48.898295  8605 consensus_queue.cc:237] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:48.972643  6500 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 194) took 1080 ms. Trace:
I20260709 12:17:48.960995  8709 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:17:48.974484  8784 log.cc:927] Time spent T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 12:17:49.022374  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 153339us
I20260709 12:17:48.978299  8653 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:48.982654  8717 consensus_queue.cc:260] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:48.963897  8768 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:17:48.987604  8781 log.cc:927] Time spent T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:17:48.968679  8726 consensus_queue.cc:1048] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 12:17:48.993891  6579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 168) took 1126 ms. Trace:
I20260709 12:17:49.008651  6500 rpcz_store.cc:276] 0709 12:17:47.892832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:47.893700 (+   868us) service_pool.cc:224] Handling call
0709 12:17:47.967512 (+ 73812us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:47.974251 (+  6739us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:47.974286 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:48.025993 (+ 51707us) log.cc:844] Serialized 26 byte log entry
0709 12:17:48.439406 (+413413us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:48.439450 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:48.439526 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:48.972401 (+532875us) raft_consensus.cc:1685] finished
0709 12:17:48.972427 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:48.972580 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6265,"raft.queue_time_us":21572,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":74,"thread_start_us":374768,"threads_started":1,"wal-append.queue_time_us":471678}
W20260709 12:17:49.010449  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 67236.7us
I20260709 12:17:49.061210  6579 rpcz_store.cc:276] 0709 12:17:47.854786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:47.920681 (+ 65895us) service_pool.cc:224] Handling call
0709 12:17:47.992229 (+ 71548us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:48.801118 (+808889us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:48.980956 (+179838us) spinlock_profiling.cc:243] Waited 808 ms on lock 0x7b540032a854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:48.981168 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1858627840}
W20260709 12:17:49.075035  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 90686.9us
I20260709 12:17:49.061406  8709 raft_consensus.cc:697] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:49.063362  8768 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:49.073870  8665 consensus_meta.cc:287] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.646s	user 0.010s	sys 0.002s
I20260709 12:17:49.075902  8717 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:49.091140  6650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 205) took 1004 ms. Trace:
I20260709 12:17:49.092831  6650 rpcz_store.cc:276] 0709 12:17:48.086661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:48.086788 (+   127us) service_pool.cc:224] Handling call
0709 12:17:48.116351 (+ 29563us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:48.503284 (+386933us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:48.503322 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:48.519580 (+ 16258us) log.cc:844] Serialized 26 byte log entry
0709 12:17:48.815093 (+295513us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:48.815149 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:48.815236 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:49.090897 (+275661us) raft_consensus.cc:1685] finished
0709 12:17:49.090921 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:49.091067 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2191,"raft.queue_time_us":266720,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":84,"thread_start_us":532248,"threads_started":2,"wal-append.queue_time_us":220211}
I20260709 12:17:49.100929  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:49.106549  8653 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:17:49.107224  8770 log.cc:927] Time spent T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.254s	user 0.003s	sys 0.000s
I20260709 12:17:49.108520  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:17:49.132411  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:49.093488  8738 consensus_meta.cc:287] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.594s	user 0.012s	sys 0.000s
W20260709 12:17:49.210718  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 102311us
I20260709 12:17:49.112572  8764 raft_consensus.cc:2764] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260709 12:17:49.085099  8665 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:49.207679  8684 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:17:49.222422  6394 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 6f94b1cd286648a9b82d5abf69bb86d5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica f2b06aebe97249a4877d798b8118ece4 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:49.223984  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:49.214593  8738 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:49.225301  6622 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:49.226588  8656 raft_consensus.cc:3055] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:49.226832  6622 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:49.227543  8738 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:49.229374  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:49.229686  8795 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 12:17:49.231268  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:49.231803  8744 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.232486  8774 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:49.233012  8774 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:49.233669  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:49.228199  6622 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:49.235239  8774 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:49.232317  8744 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:49.236379  6622 leader_election.cc:400] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7.
I20260709 12:17:49.236828  8764 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:49.229387  8709 consensus_queue.cc:237] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:49.237576  8764 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:49.237634  6622 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:49.238160  8764 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:49.238590  8764 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:49.227010  8656 raft_consensus.cc:740] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:49.239351  6622 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:49.237862  8744 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:49.245179  6550 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:49.247988  6550 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:49.219048  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:49.249891  8709 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:49.253852  8709 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:49.254419  8709 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:49.254444  6550 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:49.256691  8794 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:49.256960  8563 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:49.257479  8563 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:49.258038  8563 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:49.223704  8640 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.258738  8640 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:49.261148  8640 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:49.248486  8643 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:49.261976  8643 raft_consensus.cc:697] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:49.262863  8643 consensus_queue.cc:237] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:49.228256  8738 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:49.269496  6550 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:49.271715  6550 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:49.250836  8744 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:49.276084  8640 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:49.277351  8774 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:49.278741  8665 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:49.240499  8656 consensus_queue.cc:260] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:49.281704  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:49.283365  8656 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:49.280506  8640 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:49.284054  8640 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:49.283448  8772 log.cc:927] Time spent T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.420s	user 0.000s	sys 0.001s
I20260709 12:17:49.287969  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 170) took 1432 ms. Trace:
I20260709 12:17:49.288396  6584 rpcz_store.cc:276] 0709 12:17:47.855317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:47.990301 (+134984us) service_pool.cc:224] Handling call
0709 12:17:48.033619 (+ 43318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:48.056211 (+ 22592us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:48.056242 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:48.056689 (+   447us) log.cc:844] Serialized 26 byte log entry
0709 12:17:48.411722 (+355033us) spinlock_profiling.cc:243] Waited 2.88 ms on lock 0x7b180005ed80. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:17:48.715068 (+303346us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:48.715130 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:48.715225 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:49.282873 (+567648us) spinlock_profiling.cc:243] Waited 4.22 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:17:49.287723 (+  4850us) raft_consensus.cc:1685] finished
0709 12:17:49.287746 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:49.287905 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66260,"raft.queue_time_us":11421,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":20602,"spinlock_wait_cycles":16312064,"thread_start_us":303131,"threads_started":1,"wal-append.queue_time_us":724971}
I20260709 12:17:49.268801  8709 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:49.293195  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:49.293035  8768 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:49.293709  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:49.280006  8744 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:49.294924  8744 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:49.281446  8774 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:49.296936  8774 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:49.294549  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:49.302011  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:49.302997  7266 raft_consensus.cc:2368] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4.
W20260709 12:17:49.306813  8739 log.cc:927] Time spent T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.529s	user 0.017s	sys 0.008s
I20260709 12:17:49.309851  6504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 1577) took 3019 ms. Trace:
I20260709 12:17:49.314101  6505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 186) took 1986 ms. Trace:
I20260709 12:17:49.322697  6474 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:49.324647  6550 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:49.316624  6505 rpcz_store.cc:276] 0709 12:17:47.327700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:47.345731 (+ 18031us) service_pool.cc:224] Handling call
0709 12:17:48.010405 (+664674us) spinlock_profiling.cc:243] Waited 3.1 ms on lock 0x7b5800002a98. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:17:48.010647 (+   242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:48.010905 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:17:48.034139 (+ 23234us) log.cc:844] Serialized 14 byte log entry
0709 12:17:49.266928 (+1232789us) spinlock_profiling.cc:243] Waited 8.52 ms on lock 0x7b18003490c0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a2923a 00007f2875e93a2a
0709 12:17:49.300959 (+ 34031us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:49.301031 (+    72us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:49.301095 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:49.313812 (+ 12717us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:49.314019 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":23037,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":4572,"raft.queue_time_us":547,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":100,"spinlock_wait_cycles":26730496,"thread_start_us":32835,"threads_started":1}
I20260709 12:17:49.310509  6504 rpcz_store.cc:276] 0709 12:17:46.290280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:47.022803 (+732523us) service_pool.cc:224] Handling call
0709 12:17:47.059643 (+ 36840us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:47.060129 (+   486us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:47.060175 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:47.078331 (+ 18156us) log.cc:844] Serialized 26 byte log entry
0709 12:17:47.432297 (+353966us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:47.452875 (+ 20578us) log.cc:844] Serialized 14 byte log entry
0709 12:17:47.493166 (+ 40291us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:47.493243 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:47.561501 (+ 68258us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b1800411700. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:17:47.718482 (+156981us) spinlock_profiling.cc:243] Waited 4.17 ms on lock 0x7b1800411700. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:17:48.716706 (+998224us) spinlock_profiling.cc:243] Waited 3.83 ms on lock 0x7b1800411700. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:17:49.306192 (+589486us) spinlock_profiling.cc:243] Waited 89.5 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:17:49.309444 (+  3252us) raft_consensus.cc:1685] finished
0709 12:17:49.309480 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:49.309720 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12938,"prepare.queue_time_us":36381,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"raft.queue_time_us":48865,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":1979,"spinlock_wait_cycles":248443264,"thread_start_us":256774,"threads_started":1,"wal-append.queue_time_us":381751}
I20260709 12:17:49.331887  6474 leader_election.cc:400] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4.
I20260709 12:17:49.327224  8570 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:49.336827  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:49.345237  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 105987us
I20260709 12:17:49.345479  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:49.346019  6550 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:49.346382  6654 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:49.337471  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:49.338812  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:49.347352  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:49.347568  6652 raft_consensus.cc:2376] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7. Re-sending same reply.
I20260709 12:17:49.347781  6655 raft_consensus.cc:2410] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:17:49.348460  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:49.349156  6652 raft_consensus.cc:2393] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 4: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:49.339141  6474 leader_election.cc:403] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:49.341125  6651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 1756) took 1827 ms. Trace:
I20260709 12:17:49.349615  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:49.349756  6651 rpcz_store.cc:276] 0709 12:17:47.514171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:47.645182 (+131011us) service_pool.cc:224] Handling call
0709 12:17:47.865964 (+220782us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:47.866082 (+   118us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:47.866097 (+    15us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:47.866119 (+    22us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:47.866140 (+    21us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:47.866168 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:49.340992 (+1474824us) spinlock_profiling.cc:243] Waited 220 ms on lock 0x7b54003dfed0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:49.341081 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":505470848}
I20260709 12:17:49.350816  8718 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:17:49.350891  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:49.351204  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:49.351495  6653 raft_consensus.cc:2376] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5. Re-sending same reply.
I20260709 12:17:49.351743  6651 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:49.352322  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:49.353053  8665 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:49.353420  8665 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:49.353713  6653 raft_consensus.cc:1275] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:17:49.353734  8665 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:49.354354  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:49.355235  6657 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:49.357509  6658 raft_consensus.cc:1275] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
W20260709 12:17:49.358623  6473 connection.cc:256] RPC call timeout handler was delayed by -1.2053853630000049s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1506s for a response
W20260709 12:17:49.359097  6473 connection.cc:256] RPC call timeout handler was delayed by -1.193685362999986s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1519s for a response
I20260709 12:17:49.361270  8764 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:49.362322  8664 consensus_queue.cc:1059] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:49.363474  8764 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:49.363528  8483 raft_consensus.cc:3055] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:49.335425  8768 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:49.364127  8483 raft_consensus.cc:740] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:49.366060  6655 raft_consensus.cc:1275] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:49.358661  6658 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:49.367424  6653 raft_consensus.cc:1275] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 2. (term mismatch)
I20260709 12:17:49.369006  6651 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:17:49.373106  6657 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:49.367766  6653 pending_rounds.cc:85] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 1
I20260709 12:17:49.378018  6653 pending_rounds.cc:107] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 4.2
I20260709 12:17:49.378804  6653 raft_consensus.cc:2889] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:49.378836  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:49.380403  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:49.383307  6584 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:17:49.385800  6578 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:17:49.386638  6653 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:49.364962  8715 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:49.377463  8570 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.374869  8768 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:49.392959  6579 raft_consensus.cc:2410] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:17:49.402588  6649 raft_consensus.cc:1275] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:17:49.381240  8794 consensus_queue.cc:1048] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 12:17:49.382644  8764 consensus_queue.cc:1059] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 12:17:49.391490  8483 consensus_queue.cc:260] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:49.413512  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:49.414877  6649 raft_consensus.cc:2376] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5. Re-sending same reply.
I20260709 12:17:49.404440  8807 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:17:49.425595  6649 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:17:49.396310  8768 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:49.427217  8715 consensus_queue.cc:1048] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:17:49.427855  8771 log.cc:927] Time spent T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.591s	user 0.002s	sys 0.000s
I20260709 12:17:49.428586  8768 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:49.429186  8715 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:49.428426  8757 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:17:49.418886  8751 log.cc:927] Time spent T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 1.421s	user 0.001s	sys 0.004s
I20260709 12:17:49.394523  8570 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:49.433598  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 83326.8us
I20260709 12:17:49.433746  6622 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:49.430744  8718 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4.
I20260709 12:17:49.432590  8483 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:49.443637  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:49.448670  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:49.450055  6582 raft_consensus.cc:2393] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 4: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:49.452801  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:49.460915  7238 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:49.454244  6582 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:49.454860  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:49.464080  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:49.462831  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:49.466135  6581 raft_consensus.cc:2368] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 7. Current term is 8.
I20260709 12:17:49.466173  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:49.469226  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:49.471994  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:49.471920  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:49.442691  8764 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:49.473261  6577 raft_consensus.cc:2393] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 7: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:49.465953  8598 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.474120  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:49.474634  8570 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:49.480295  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:49.438084  8665 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:49.505815  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:49.505955  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:49.506707  6653 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:49.507723  8776 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:49.508996  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:49.509064  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:49.510485  6584 raft_consensus.cc:1240] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 7. Current term is 8. Ops: []
W20260709 12:17:49.511759  8765 log.cc:927] Time spent T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.763s	user 0.006s	sys 0.008s
I20260709 12:17:49.513571  6658 raft_consensus.cc:3055] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:49.514021  6658 raft_consensus.cc:740] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:49.520074  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:49.444576  8563 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:49.507972  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:49.515064  6658 consensus_queue.cc:260] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:49.523461  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:49.524433  8809 log.cc:927] Time spent T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
W20260709 12:17:49.525012  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:49.528647  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:49.529237  8600 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:49.530431  8704 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 12:17:49.502724  8598 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:49.531158  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:49.541168  6578 raft_consensus.cc:2393] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 3: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:49.532135  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:49.542963  6579 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:49.479635  8764 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:49.532932  7238 leader_election.cc:400] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 6. Current term is 7.
I20260709 12:17:49.545678  7238 leader_election.cc:403] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:49.502018  8552 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:49.534408  6577 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:49.537496  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:49.547325  6584 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:49.546037  6658 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:49.554870  8563 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:49.556164  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:49.561121  7238 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:49.564008  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:49.582572  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 156387us
I20260709 12:17:49.585724  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:49.586585  6656 raft_consensus.cc:2410] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:17:49.590008  6579 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:49.590291  6622 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:49.517308  8570 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
W20260709 12:17:49.593043  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 155404us
W20260709 12:17:49.593917  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:49.596051  6622 leader_election.cc:400] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 7. Current term is 8.
I20260709 12:17:49.596534  6622 leader_election.cc:403] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:49.597297  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:49.548445  8764 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:49.602162  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:49.604452  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:49.604800  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:49.606515  6577 raft_consensus.cc:2393] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 3: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:49.592800  7267 raft_consensus.cc:2393] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 4: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:17:49.585016  8552 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 6. Current term is 7.
I20260709 12:17:49.610319  6622 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:49.607607  8536 consensus_queue.cc:1059] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 12:17:49.609858  8643 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:49.617331  8643 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:49.617936  8643 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:49.629979  8598 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:49.632808  8570 consensus_queue.cc:1048] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:49.599813  8600 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:49.638720  8640 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:49.640688  6575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 179) took 1736 ms. Trace:
I20260709 12:17:49.643316  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:49.648759  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:49.649711  6654 raft_consensus.cc:2468] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
I20260709 12:17:49.603588  8768 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:49.656651  7240 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:49.651964  6655 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:17:49.658985  6584 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:17:49.660699  6658 raft_consensus.cc:1275] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:17:49.665189  6658 pending_rounds.cc:85] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 2
I20260709 12:17:49.602124  8807 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 7. Current term is 8.
I20260709 12:17:49.641187  6575 rpcz_store.cc:276] 0709 12:17:47.903875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:48.047464 (+143589us) service_pool.cc:224] Handling call
0709 12:17:48.087491 (+ 40027us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:17:48.095251 (+  7760us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:48.095280 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:17:48.170599 (+ 75319us) log.cc:844] Serialized 52 byte log entry
0709 12:17:48.502957 (+332358us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:48.783205 (+280248us) log.cc:844] Serialized 14 byte log entry
0709 12:17:48.833076 (+ 49871us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:48.834174 (+  1098us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:49.309579 (+475405us) spinlock_profiling.cc:243] Waited 3.24 ms on lock 0x7b18002b4020. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:17:49.389475 (+ 79896us) spinlock_profiling.cc:243] Waited 2.48 ms on lock 0x7b18002b4020. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d6a2a6 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db
0709 12:17:49.511168 (+121693us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:17:49.640428 (+129260us) raft_consensus.cc:1685] finished
0709 12:17:49.640457 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:49.640619 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":257838,"prepare.queue_time_us":347333,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":2153,"raft.queue_time_us":74635,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":26243,"spinlock_wait_cycles":292410112,"thread_start_us":513091,"threads_started":3,"wal-append.queue_time_us":464106}
I20260709 12:17:49.671466  6583 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:17:49.695742  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:49.706732  6654 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:17:49.665838  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:49.667241  8600 raft_consensus.cc:3055] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:49.696684  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:49.698174  6550 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:49.609421  8812 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.701843  6651 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:17:49.653820  8709 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:49.607570  8738 consensus_queue.cc:1059] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.087s
I20260709 12:17:49.716075  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:49.663517  8768 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:49.665663  6658 pending_rounds.cc:107] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:17:49.734578  6658 raft_consensus.cc:2889] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:49.736960  6550 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:49.719512  6623 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:17:49.761384  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 153477us
I20260709 12:17:49.705646  8720 raft_consensus.cc:2804] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:49.779466  6550 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:49.784880  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:49.785465  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 200425us
I20260709 12:17:49.722342  8600 raft_consensus.cc:740] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:49.791796  6575 raft_consensus.cc:1240] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:17:49.740216  8768 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:49.741839  8656 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:49.739581  8653 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.795001  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:49.774709  8720 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:49.808681  6577 raft_consensus.cc:2468] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 3.
I20260709 12:17:49.787285  8511 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:49.800046  6584 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:17:49.801419  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:49.810148  6503 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:17:49.793453  8808 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 12:17:49.772377  8814 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:49.772775  8709 raft_consensus.cc:697] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:49.731830  8774 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:49.763422  8766 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 12:17:49.816444  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:49.831861  6509 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:49.817351  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:49.818303  6575 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 4. (term mismatch)
I20260709 12:17:49.793869  8665 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:49.798146  8807 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:49.813968  6550 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:49.833148  6506 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:49.742278  8812 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:49.799376  8794 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:49.799894  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:17:49.846786  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 127363us
I20260709 12:17:49.833137  6575 pending_rounds.cc:85] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 3
I20260709 12:17:49.816569  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:49.851492  6550 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:49.889731  6550 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:49.890164  6503 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
W20260709 12:17:49.835145  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 241114us
I20260709 12:17:49.812052  8720 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:49.785089  8664 consensus_queue.cc:1048] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 12:17:49.885380  6623 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:49.812731  8511 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:49.887877  6581 raft_consensus.cc:2393] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 6: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:49.897439  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:49.812290  8757 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.072s
I20260709 12:17:49.852612  8738 raft_consensus.cc:3055] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:49.813071  8653 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:49.880401  8656 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:49.885047  6575 pending_rounds.cc:107] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 1.4
I20260709 12:17:49.962841  6575 raft_consensus.cc:2917] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Aborting config change with OpId 1.4: Aborted: Op aborted by new leader
I20260709 12:17:49.963348  6575 raft_consensus.cc:2889] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:17:49.961877  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 53781.3us
I20260709 12:17:49.852954  8432 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 12:17:49.898952  8726 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:49.947055  6504 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:49.967823  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:49.972162  6509 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
W20260709 12:17:49.978525  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 312735us
I20260709 12:17:49.966694  6503 raft_consensus.cc:1275] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:17:49.850253  8823 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.123s
I20260709 12:17:49.830791  8814 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:49.953845  8794 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:17:49.966480  8744 consensus_meta.cc:287] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.671s	user 0.012s	sys 0.002s
W20260709 12:17:50.019862  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 134464us
I20260709 12:17:49.965796  8640 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:50.074111  6623 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:49.901703  8511 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:49.960788  8738 raft_consensus.cc:740] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:17:50.187656  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 223437us
I20260709 12:17:50.549387  6550 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:50.119490  8744 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:50.463385  6472 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.004s	sys 0.021s
W20260709 12:17:50.466444  6474 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.035s	sys 0.047s
W20260709 12:17:50.554308  6472 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.004s	sys 0.022s
W20260709 12:17:50.554409  6474 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.036s	sys 0.047s
I20260709 12:17:50.556308  6575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:56112 (request call id 73) took 13840 ms. Trace:
W20260709 12:17:50.527793  8536 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.003s	sys 0.004s
W20260709 12:17:50.533802  8807 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.001s	sys 0.005s
I20260709 12:17:50.534981  8830 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:50.536092  6622 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:50.536505  8831 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:17:50.537802  6506 consensus_meta.cc:287] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.687s	user 0.023s	sys 0.020s
W20260709 12:17:50.539148  6502 consensus_meta.cc:287] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.528s	user 0.000s	sys 0.021s
I20260709 12:17:50.680310  6506 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:50.680558  6502 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
W20260709 12:17:50.539435  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:49.900807  8821 consensus_queue.cc:1059] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.133s
I20260709 12:17:50.020226  8709 consensus_queue.cc:237] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:50.540918  8764 consensus_meta.cc:287] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.926s	user 0.003s	sys 0.012s
W20260709 12:17:50.464789  6473 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.002s	sys 0.000s
W20260709 12:17:50.689370  6473 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.854s	user 0.003s	sys 0.005s
W20260709 12:17:50.689698  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 854363us
I20260709 12:17:49.980979  8563 consensus_queue.cc:1048] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.140s
W20260709 12:17:50.466233  7240 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.001s	sys 0.001s
W20260709 12:17:50.699178  7240 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.901s	user 0.004s	sys 0.004s
W20260709 12:17:50.699604  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 918790us
W20260709 12:17:50.467267  6652 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.001s	sys 0.001s
W20260709 12:17:50.702759  6652 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.001s	sys 0.006s
I20260709 12:17:50.069245  8827 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:50.464087  6508 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.001s	sys 0.000s
W20260709 12:17:50.717011  6508 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.815s	user 0.006s	sys 0.002s
W20260709 12:17:50.465391  7238 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.001s	sys 0.001s
W20260709 12:17:50.718169  7238 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.920s	user 0.006s	sys 0.002s
W20260709 12:17:50.718529  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 943348us
W20260709 12:17:50.563271  6582 consensus_meta.cc:287] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.932s	user 0.014s	sys 0.000s
I20260709 12:17:50.720212  6582 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:17:50.645609  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:50.595219  8826 log.cc:927] Time spent T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.621s	user 0.002s	sys 0.005s
W20260709 12:17:50.618256  8817 log.cc:927] Time spent T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.821s	user 0.000s	sys 0.008s
W20260709 12:17:50.575919  8807 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.002s	sys 0.006s
W20260709 12:17:50.571201  8536 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.765s	user 0.004s	sys 0.004s
I20260709 12:17:50.557093  6575 rpcz_store.cc:276] 0709 12:17:36.701900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:37.132685 (+430785us) service_pool.cc:224] Handling call
0709 12:17:37.206294 (+ 73609us) log.cc:844] Serialized 480 byte log entry
0709 12:17:38.116931 (+910637us) spinlock_profiling.cc:243] Waited 9.63 ms on lock 0x7b180046bac0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:17:50.143228 (+12026297us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1554663,"raft.queue_time_us":78283,"raft.run_cpu_time_us":1690,"raft.run_wall_time_us":162642,"spinlock_wait_cycles":22144768,"thread_start_us":140838,"threads_started":1,"wal-append.queue_time_us":138433,"wal-append.run_cpu_time_us":1847,"wal-append.run_wall_time_us":1017929}
I20260709 12:17:50.561537  8794 raft_consensus.cc:697] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:50.619869  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:50.645165  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 95877.3us
I20260709 12:17:50.650439  6576 raft_consensus.cc:2393] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 5: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:17:50.650779  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 114098us
I20260709 12:17:50.667788  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:50.669354  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:50.671782  8820 log.cc:927] Time spent T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.853s	user 0.002s	sys 0.012s
I20260709 12:17:50.644467  8640 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:50.766376  8640 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:50.681928  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:50.770190  6502 raft_consensus.cc:2393] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 7: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:50.682415  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:50.772779  6506 raft_consensus.cc:2368] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7.
W20260709 12:17:50.686651  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 612580us
I20260709 12:17:50.605319  8774 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:50.646238  8848 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:50.735599  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:50.778946  6504 raft_consensus.cc:2368] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 7. Current term is 8.
I20260709 12:17:50.685907  8483 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:50.761255  6550 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:50.761834  6507 raft_consensus.cc:2393] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 5: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:50.998468  8849 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:50.788859  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 252525us
I20260709 12:17:50.829241  8653 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:50.736451  8738 consensus_queue.cc:260] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:50.914098  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 188) took 1139 ms. Trace:
I20260709 12:17:50.848866  8812 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:50.931202  8845 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:50.790208  8821 consensus_queue.cc:1048] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.079s
I20260709 12:17:50.995049  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 1795) took 1357 ms. Trace:
I20260709 12:17:50.944607  8744 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:50.914009  8600 consensus_queue.cc:260] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:50.788170  8726 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.005s	sys 0.038s
W20260709 12:17:51.050882  8726 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.976s	user 0.013s	sys 0.048s
I20260709 12:17:51.050938  8859 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:51.051326  8726 raft_consensus.cc:697] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:51.051749  8859 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:51.052495  8726 consensus_queue.cc:237] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:51.053222  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 1627) took 1034 ms. Trace:
I20260709 12:17:50.790597  8848 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:51.043135  8684 consensus_queue.cc:1059] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 12:17:51.054299  8859 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:51.055203  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:51.055614  8848 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:50.801139  8814 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:51.055981  6507 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:50.931416  8794 consensus_queue.cc:237] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:51.000516  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:17:50.939471  8643 consensus_meta.cc:287] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 1.321s	user 0.016s	sys 0.002s
I20260709 12:17:51.003666  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:50.931135  8842 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:50.931116  8851 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:50.742185  8807 raft_consensus.cc:697] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 5 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:50.721199  8764 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:50.675215  8831 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
W20260709 12:17:51.006529  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 315853us
I20260709 12:17:50.854038  8640 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:51.005334  8768 consensus_meta.cc:287] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 1.196s	user 0.015s	sys 0.000s
W20260709 12:17:51.037664  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 276478us
I20260709 12:17:50.919130  8827 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:50.863269  8483 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.034991  8838 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:50.943856  8843 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:50.944036  8829 raft_consensus.cc:2764] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:17:51.042474  6577 raft_consensus.cc:1240] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3. Ops: []
W20260709 12:17:51.133896  8828 log.cc:927] Time spent T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 1.054s	user 0.005s	sys 0.005s
I20260709 12:17:51.042999  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:51.187628  6506 raft_consensus.cc:2410] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:17:50.995322  8536 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:51.045189  6584 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:51.044895  6652 rpcz_store.cc:276] 0709 12:17:49.637252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.639156 (+  1904us) service_pool.cc:224] Handling call
0709 12:17:49.639437 (+   281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:49.878360 (+238923us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:49.878407 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:49.879168 (+   761us) log.cc:844] Serialized 26 byte log entry
0709 12:17:50.703307 (+824139us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:50.703370 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:50.703474 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:50.994765 (+291291us) raft_consensus.cc:1685] finished
0709 12:17:50.994788 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:50.994977 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":124363,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":2407,"thread_start_us":994455,"threads_started":2,"wal-append.queue_time_us":708804}
I20260709 12:17:50.873489  8774 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:51.055271  8859 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:51.050448  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:51.053623  6503 rpcz_store.cc:276] 0709 12:17:50.019101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:50.030558 (+ 11457us) service_pool.cc:224] Handling call
0709 12:17:50.030956 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:50.067717 (+ 36761us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:50.067750 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:50.552737 (+484987us) log.cc:844] Serialized 26 byte log entry
0709 12:17:50.772403 (+219666us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:17:50.772441 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:50.772503 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:51.053002 (+280499us) raft_consensus.cc:1685] finished
0709 12:17:51.053027 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:51.053163 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36141,"raft.queue_time_us":508994,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":92,"thread_start_us":78816,"threads_started":1,"wal-append.queue_time_us":424987}
I20260709 12:17:51.196503  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:51.197270  6503 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
W20260709 12:17:50.912992  8720 consensus_meta.cc:287] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 1.020s	user 0.011s	sys 0.001s
I20260709 12:17:51.120430  6507 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:17:51.201851  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:51.095486  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.110744  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:51.145714  6581 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1500) took 1608 ms (client timeout 1689 ms). Trace:
I20260709 12:17:51.168779  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:51.169224  8862 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:17:51.170101  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 128451us
I20260709 12:17:51.170298  8684 raft_consensus.cc:3055] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:51.036075  6583 rpcz_store.cc:276] 0709 12:17:49.775020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.775159 (+   139us) service_pool.cc:224] Handling call
0709 12:17:49.775564 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:49.776481 (+   917us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:49.776509 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:49.792860 (+ 16351us) log.cc:844] Serialized 26 byte log entry
0709 12:17:50.019167 (+226307us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:50.019227 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:50.019325 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:50.913851 (+894526us) raft_consensus.cc:1685] finished
0709 12:17:50.913880 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:50.914035 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4186,"raft.queue_time_us":1009,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":9136,"thread_start_us":209205,"threads_started":1,"wal-append.queue_time_us":1115780}
I20260709 12:17:51.213716  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:51.214538  6503 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:51.037529  8656 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:51.195175  8836 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.642s	user 0.001s	sys 0.001s
I20260709 12:17:51.184759  8640 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 12:17:51.184175  6550 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:51.184651  8865 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.042222  8825 consensus_queue.cc:1059] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:17:51.126163  8844 log.cc:927] Time spent T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:17:51.170478  8766 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:51.058270  8841 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:51.106961  8855 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:51.255939  8839 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:17:51.204108  6581 rpcz_store.cc:269] 0709 12:17:49.537759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.537961 (+   202us) service_pool.cc:224] Handling call
0709 12:17:51.145413 (+1607452us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b540034dad4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:51.145628 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":752328064}
I20260709 12:17:51.204320  6501 raft_consensus.cc:2393] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 5: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:51.210783  8859 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:51.256691  8855 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:51.257499  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:51.258116  6501 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:51.258668  8859 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:51.258867  8855 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:51.167311  6473 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:51.259107  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:51.259716  6501 raft_consensus.cc:2376] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6. Re-sending same reply.
W20260709 12:17:51.259829  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 92630.6us
I20260709 12:17:51.216102  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:51.260222  8821 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:51.260723  6473 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:51.261193  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:51.182396  8643 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:51.181200  8768 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:51.240509  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 56401.2us
I20260709 12:17:51.185925  8665 consensus_queue.cc:1048] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 12:17:51.182693  8483 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:51.184473  8807 consensus_queue.cc:237] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:51.215894  8684 raft_consensus.cc:740] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:51.253839  6579 raft_consensus.cc:1240] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:51.253551  8766 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:51.185222  8842 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:51.254341  8841 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:51.255326  6475 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:51.179936  8851 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:51.195148  8829 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.229745  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:51.262010  6473 leader_election.cc:400] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 4. Current term is 5.
I20260709 12:17:51.230659  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:51.216920  8543 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:51.310832  8543 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:51.313083  8543 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:51.256058  8764 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:51.314257  8764 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.314663  8764 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:51.315022  8811 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:51.037061  8849 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:51.316859  8768 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:51.268918  8830 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:51.317777  8871 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:51.318050  8830 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:17:51.318301  8871 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:51.188076  8843 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:51.252516  8865 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:51.320189  8829 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:51.269913  8855 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:51.271589  6582 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1492) took 1814 ms (client timeout 1569 ms). Trace:
I20260709 12:17:51.272434  8861 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.092s
I20260709 12:17:51.273686  8833 raft_consensus.cc:3055] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:51.273792  8858 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:51.275777  8846 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:17:51.277215  8866 log.cc:927] Time spent T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:17:51.277657  8808 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:51.273705  8847 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.092s
I20260709 12:17:51.278354  8859 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:51.283285  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:51.284219  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:51.289459  6550 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:51.181388  8827 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:51.215510  8774 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:51.210814  8536 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:51.217262  8720 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:51.259752  8845 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:51.185467  8838 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:51.306770  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 51521.2us
I20260709 12:17:51.269040  8870 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:51.256716  8839 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:51.549093  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 259703us
W20260709 12:17:51.549556  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 287689us
W20260709 12:17:51.352906  6623 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.4:44857 timed out after 1.630s (SENT) [suppressed 29 similar messages]
W20260709 12:17:51.498739  8511 consensus_meta.cc:287] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.924s	user 0.008s	sys 0.003s
I20260709 12:17:51.517277  6502 raft_consensus.cc:1240] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:17:51.321557  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:51.550616  8833 raft_consensus.cc:740] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:51.516532  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 1616) took 1712 ms. Trace:
I20260709 12:17:51.322238  8830 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:51.604681  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:51.549135  8849 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:51.554898  8846 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:51.610432  8846 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:51.611334  7272 raft_consensus.cc:2368] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 3. Current term is 4.
I20260709 12:17:51.312088  8841 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:51.499089  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:51.550184  8859 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:51.551368  8870 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:51.551887  8511 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:51.550819  8808 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:51.551967  6582 rpcz_store.cc:269] 0709 12:17:49.457249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.462466 (+  5217us) service_pool.cc:224] Handling call
0709 12:17:51.271305 (+1808839us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b54000366d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:51.271485 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":381723520}
I20260709 12:17:51.550540  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:51.461465  8842 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:51.417351  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:51.328800  8766 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:51.331459  8483 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:51.331770  8831 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:51.327006  8643 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:51.334043  8851 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:51.332181  8684 consensus_queue.cc:260] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:51.309135  8821 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:51.597576  8709 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.597882  8832 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:51.601591  8839 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:51.596468  6473 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:51.428853  8814 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:51.604918  6505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 1619) took 1799 ms. Trace:
I20260709 12:17:51.605662  6506 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:51.600518  6550 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:51.321262  8843 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:51.607839  6656 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5.
I20260709 12:17:51.609031  8858 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:51.612349  6504 raft_consensus.cc:2393] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 6: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:51.607767  6508 rpcz_store.cc:276] 0709 12:17:49.803821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.803951 (+   130us) service_pool.cc:224] Handling call
0709 12:17:49.804323 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:50.717484 (+913161us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:50.717519 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:50.740959 (+ 23440us) log.cc:844] Serialized 26 byte log entry
0709 12:17:51.170562 (+429603us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:51.170601 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:51.516006 (+345405us) spinlock_profiling.cc:243] Waited 97.4 ms on lock 0x7b54002c47d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:51.516060 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:51.516114 (+    54us) raft_consensus.cc:1685] finished
0709 12:17:51.516130 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:51.516437 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":637815,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":154,"spinlock_wait_cycles":224066304,"thread_start_us":1177095,"threads_started":2,"wal-append.queue_time_us":395394}
W20260709 12:17:51.625967  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 273090us
I20260709 12:17:51.613495  8795 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:51.636502  6503 raft_consensus.cc:1240] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:51.643292  8821 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:51.640965  8865 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:51.669425  8757 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:51.691323  8764 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:51.649433  8511 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:51.649616  8818 log.cc:927] Time spent T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 1.769s	user 0.007s	sys 0.003s
I20260709 12:17:51.703374  8858 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:51.704111  8827 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:51.704553  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:51.705175  7270 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:51.705391  8774 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:51.650725  8709 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:51.651449  8832 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:51.643980  8833 consensus_queue.cc:260] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:51.654564  8600 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:51.654634  8859 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:17:51.655055  8432 consensus_queue.cc:1175] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f2b06aebe97249a4877d798b8118ece4" responder_term: 2 status { last_received { term: 2 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 2 index: 5 } }
I20260709 12:17:51.719038  6577 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:17:51.666604  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 66201.7us
I20260709 12:17:51.663098  6505 rpcz_store.cc:276] 0709 12:17:49.805896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.806048 (+   152us) service_pool.cc:224] Handling call
0709 12:17:49.819475 (+ 13427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:49.827573 (+  8098us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:17:50.016149 (+188576us) log.cc:844] Serialized 14 byte log entry
0709 12:17:50.180559 (+164410us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:51.500230 (+1319671us) spinlock_profiling.cc:243] Waited 6 ms on lock 0x7b2c00000c88. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:17:51.500773 (+   543us) log.cc:844] Serialized 26 byte log entry
0709 12:17:51.500927 (+   154us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:51.500962 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:51.501026 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:51.525576 (+ 24550us) raft_consensus.cc:1685] finished
0709 12:17:51.525659 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:51.604845 (+ 79186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7602,"prepare.queue_time_us":1078441,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":69,"raft.queue_time_us":431130,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":2914,"spinlock_wait_cycles":13794688,"thread_start_us":295013,"threads_started":2,"wal-append.queue_time_us":949117}
I20260709 12:17:51.674872  6475 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:17:51.676179  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 81168.3us
I20260709 12:17:51.624193  8855 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:51.675797  8795 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:51.683502  8882 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:51.686303  8829 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:51.734606  8795 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:51.686196  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:51.687007  8870 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:51.701632  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:51.702751  8871 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:51.702886  8839 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.711454  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 1788) took 2279 ms. Trace:
I20260709 12:17:51.729386  8832 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:51.677793  8814 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:51.677920  8536 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:51.693451  8821 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:51.774664  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:51.736115  6576 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54204 (request call id 185) took 2275 ms (client timeout 1756 ms). Trace:
I20260709 12:17:51.788607  6550 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:51.739404  6623 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:51.866230  6649 rpcz_store.cc:276] 0709 12:17:49.432032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.432142 (+   110us) service_pool.cc:224] Handling call
0709 12:17:49.434414 (+  2272us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:49.434715 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:49.434746 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:49.435056 (+   310us) log.cc:844] Serialized 26 byte log entry
0709 12:17:49.712087 (+277031us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:49.712125 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:49.712592 (+   467us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:51.649215 (+1936623us) spinlock_profiling.cc:243] Waited 9.05 ms on lock 0x7b2400258ff8. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872c229c4 00007f2872c20acb 00007f2872ddbdc9 00007f2872ddba55 00007f2872ddb09d 00007f2875a0d10e 00007f2875a2488c 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221
0709 12:17:51.709019 (+ 59804us) raft_consensus.cc:1685] finished
0709 12:17:51.709044 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:51.711367 (+  2323us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11450,"raft.queue_time_us":1940,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":2098,"spinlock_wait_cycles":21966976,"thread_start_us":247718,"threads_started":1,"wal-append.queue_time_us":356784}
W20260709 12:17:51.864977  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 143261us
W20260709 12:17:51.755403  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 80625.1us
I20260709 12:17:51.692317  8846 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:51.883738  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 95252.6us
I20260709 12:17:51.890728  6502 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 1604) took 1148 ms. Trace:
W20260709 12:17:51.888063  7272 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38546 (request call id 278) took 2172 ms (client timeout 1630 ms). Trace:
W20260709 12:17:51.899446  7272 rpcz_store.cc:269] 0709 12:17:49.715533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.715743 (+   210us) service_pool.cc:224] Handling call
0709 12:17:51.887728 (+2171985us) spinlock_profiling.cc:243] Waited 1.89 s on lock 0x7b54005df054. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:51.887948 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4336869632}
I20260709 12:17:51.870689  6500 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 1621) took 2048 ms. Trace:
I20260709 12:17:51.868526  6475 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:51.899298  6502 rpcz_store.cc:276] 0709 12:17:50.742756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:50.772148 (+ 29392us) service_pool.cc:224] Handling call
0709 12:17:51.516987 (+744839us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:51.553350 (+ 36363us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:51.890395 (+337045us) spinlock_profiling.cc:243] Waited 744 ms on lock 0x7b54002c47d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:51.890631 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1711978880}
W20260709 12:17:51.864521  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:51.902877  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 163536us
I20260709 12:17:51.903867  6473 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:51.906085  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:51.900844  6500 rpcz_store.cc:276] 0709 12:17:49.822339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.822725 (+   386us) service_pool.cc:224] Handling call
0709 12:17:49.823021 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:51.261123 (+1438102us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b18002b7c80. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2875a314bc 00007f2872d7f48b 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d99455 00007f2875e81060 00007f2875e81bfb 00007f2875e8c64b
0709 12:17:51.261252 (+   129us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:51.261278 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:51.261763 (+   485us) log.cc:844] Serialized 26 byte log entry
0709 12:17:51.300739 (+ 38976us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:17:51.300775 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:51.300845 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:51.870319 (+569474us) raft_consensus.cc:1685] finished
0709 12:17:51.870349 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:51.870607 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37239,"raft.queue_time_us":122258,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":10607,"spinlock_wait_cycles":36026752,"thread_start_us":178510,"threads_started":2,"wal-append.queue_time_us":341492}
W20260709 12:17:51.865649  6576 rpcz_store.cc:269] 0709 12:17:49.443231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.443382 (+   151us) service_pool.cc:224] Handling call
0709 12:17:51.717894 (+2274512us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b540034dfd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:51.718120 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2709621632}
I20260709 12:17:51.909022  8816 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:51.909492  6473 leader_election.cc:400] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 6. Current term is 7.
I20260709 12:17:51.930859  6473 leader_election.cc:403] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:17:51.931098  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 290082us
I20260709 12:17:51.902491  8833 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:51.909825  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:51.709496  8764 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.735883  8855 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:51.939277  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:17:51.912137  6392 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 3a47d71c34b541b9b2589b4e80413fa2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) due to error UNKNOWN_ERROR; will retry: Aborted: Op aborted by new leader
I20260709 12:17:51.941604  7240 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:51.942610  8731 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:51.943033  8731 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:51.943418  8731 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:51.910328  8807 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:51.944713  8807 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:51.947929  8807 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:51.899622  8483 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:51.888996  8839 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:51.913756  8838 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:51.861323  8709 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:51.788151  8823 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:51.797992  8842 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:51.910015  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.935105  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:51.970172  6652 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7.
I20260709 12:17:51.937223  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:51.938072  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:51.787137  8774 consensus_queue.cc:1048] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.145s
I20260709 12:17:51.909777  8832 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:51.948948  8764 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:51.978418  8764 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:51.927299  8864 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:51.950871  6473 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:51.979234  8864 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:51.979940  6473 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:51.980800  8882 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:51.981591  8882 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:51.982425  6473 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:51.983186  8882 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:51.983915  6473 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:51.954885  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 48897.8us
I20260709 12:17:51.912425  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:51.960222  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:51.910725  8887 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:51.984176  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:51.984808  6475 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:51.984962  6658 raft_consensus.cc:2393] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 6: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:51.984863  6657 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 4.
I20260709 12:17:51.985304  6652 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:17:51.984995  8887 raft_consensus.cc:697] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:51.986728  8888 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:51.987489  8887 consensus_queue.cc:237] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:51.988816  8718 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:51.989559  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:51.990151  6652 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:51.990353  6473 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:51.991514  8882 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:51.991852  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:51.992529  6473 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:51.993261  8882 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:51.993597  6473 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:51.919241  8795 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:51.999722  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:52.001613  6649 raft_consensus.cc:2410] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:17:52.003269  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:52.003346  6473 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:52.004118  6653 raft_consensus.cc:2393] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 3: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:51.967633  8822 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:52.006177  6473 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:51.960980  8816 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 6. Current term is 7.
W20260709 12:17:52.006253  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:52.020529  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:51.998980  8823 raft_consensus.cc:697] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:51.999168  8709 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:52.036773  8709 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:52.039314  8709 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:52.029047  6473 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:52.034389  6575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 192) took 1168 ms. Trace:
I20260709 12:17:52.012733  8882 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:52.040356  6575 rpcz_store.cc:276] 0709 12:17:50.866464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:50.869003 (+  2539us) service_pool.cc:224] Handling call
0709 12:17:50.885421 (+ 16418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:51.617284 (+731863us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:17:51.628938 (+ 11654us) log.cc:844] Serialized 14 byte log entry
0709 12:17:51.656882 (+ 27944us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:51.656933 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:51.656963 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:52.033822 (+376859us) spinlock_profiling.cc:243] Waited 732 ms on lock 0x7b54000582d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:52.033887 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:52.034301 (+   414us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":27215,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":41,"raft.queue_time_us":89,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":11305,"spinlock_wait_cycles":1682451328,"thread_start_us":29634,"threads_started":2,"wal-append.queue_time_us":166468}
I20260709 12:17:51.995970  8842 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:52.052675  8842 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:52.014500  8718 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:17:52.064356  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 78119.8us
I20260709 12:17:52.035930  8664 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.067431  6475 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:52.041064  8822 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:52.056562  8861 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:52.056677  8882 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:52.108458  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:52.141177  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:52.148288  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:52.107609  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:52.109098  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:52.141474  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:52.108968  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:52.155222  6506 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:17:52.142473  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:52.155167  6509 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:52.212776  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:52.192065  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:17:52.084734  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 55821.2us
I20260709 12:17:52.216245  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:52.080484  8861 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:52.078047  8664 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:52.082762  8774 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.082091  8511 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.234579  6509 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
W20260709 12:17:52.260246  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:52.099983  8890 log.cc:927] Time spent T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:17:52.272313  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:52.155308  6501 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:52.277796  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:52.219719  7272 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:52.108031  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:17:52.182554  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 71989.3us
I20260709 12:17:52.148777  6500 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:52.271080  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:52.234462  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:52.083550  8842 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.270459  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:52.326632  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:52.103664  8879 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:52.327916  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 261548us
I20260709 12:17:52.328681  6550 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:52.354010  6550 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:52.235318  6473 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:52.234941  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:52.354835  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:52.355423  6504 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:52.105130  8832 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:17:52.346536  6622 connection.cc:256] RPC call timeout handler was delayed by -1.0321529909999754s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1811s for a response
W20260709 12:17:52.362068  6622 connection.cc:256] RPC call timeout handler was delayed by -1.0321529909999754s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1811s for a response
I20260709 12:17:52.098742  8862 raft_consensus.cc:3055] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:52.337641  6651 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7.
I20260709 12:17:52.086534  8882 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:52.099785  8814 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:17:52.358712  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 123457us
I20260709 12:17:52.097833  8846 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:52.212143  8847 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.285609  8901 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:52.098826  8731 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:52.083256  8822 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:52.099799  8720 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:52.098685  8795 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:52.235021  8852 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.108088  8839 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:52.335834  8861 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:52.375162  6473 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:52.376003  6654 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:52.422024  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 46922.9us
I20260709 12:17:52.359066  8842 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:52.363883  8879 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:52.387427  8814 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:52.358229  8856 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:17:52.389778  8898 log.cc:927] Time spent T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:17:52.392776  8846 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:52.393488  8847 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:52.348661  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:52.367604  8862 raft_consensus.cc:740] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:52.415171  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:17:52.399503  8821 consensus_meta.cc:287] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.517s	user 0.008s	sys 0.004s
I20260709 12:17:52.395884  8901 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:52.424108  6473 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:52.398399  8832 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:52.351506  8774 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:52.511189  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.100785  8823 consensus_queue.cc:237] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:52.522122  6649 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:52.522735  6651 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:52.426734  8903 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:52.533052  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 194) took 1294 ms. Trace:
I20260709 12:17:52.563381  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:52.534438  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 193) took 1318 ms. Trace:
I20260709 12:17:52.521163  8846 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:52.539615  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 185664us
W20260709 12:17:52.560153  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 136126us
I20260709 12:17:52.568964  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:52.565320  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.567863  6583 rpcz_store.cc:276] 0709 12:17:51.216079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:51.216320 (+   241us) service_pool.cc:224] Handling call
0709 12:17:51.216640 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:51.334834 (+118194us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:17:51.639190 (+304356us) log.cc:844] Serialized 14 byte log entry
0709 12:17:51.802731 (+163541us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:51.802808 (+    77us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:51.802875 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:52.534175 (+731300us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54000352d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:52.534234 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:52.534375 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20208,"prepare.queue_time_us":304372,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":1358,"raft.queue_time_us":37587,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":9813,"spinlock_wait_cycles":271051520,"thread_start_us":446397,"threads_started":2,"wal-append.queue_time_us":367146}
I20260709 12:17:52.539008  6654 raft_consensus.cc:2468] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 4.
I20260709 12:17:52.521471  8814 raft_consensus.cc:697] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:52.420845  8852 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:52.564170  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:52.567572  6576 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5.
I20260709 12:17:52.428036  8887 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.132s
I20260709 12:17:52.578385  6583 raft_consensus.cc:1240] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:17:52.572152  6550 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:52.567018  6577 rpcz_store.cc:276] 0709 12:17:51.238193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:51.238720 (+   527us) service_pool.cc:224] Handling call
0709 12:17:51.718571 (+479851us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:51.719641 (+  1070us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:52.532808 (+813167us) spinlock_profiling.cc:243] Waited 456 ms on lock 0x7b540034dfd0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:52.532983 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1048320768}
I20260709 12:17:52.536877  8901 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:52.591574  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:17:52.586783  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 224026us
I20260709 12:17:52.588812  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:52.571086  8850 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:52.601020  7272 raft_consensus.cc:2410] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:17:52.601135  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:17:52.586971  8894 log.cc:927] Time spent T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 12:17:52.343282  8511 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:52.614601  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.604661  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:52.616874  6582 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:52.576110  8846 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:52.606123  6583 raft_consensus.cc:2393] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 5: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:52.606559  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:52.618736  6577 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7.
I20260709 12:17:52.603482  6649 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:52.563863  8832 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:52.537062  8821 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:52.594062  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:52.594756  8849 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:52.612141  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:52.641296  6579 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7.
I20260709 12:17:52.571092  8899 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 12:17:52.643085  6651 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:17:52.657496  6507 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:52.596107  8862 consensus_queue.cc:260] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:52.633611  8795 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:17:52.666756  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 302595us
I20260709 12:17:52.636945  8664 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:52.668751  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:52.654682  8907 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 12:17:52.674544  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:52.676249  6582 raft_consensus.cc:2468] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 4.
I20260709 12:17:52.679520  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:52.690449  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:52.708235  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:17:52.712813  6500 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:52.714354  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:52.716266  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:52.717883  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:52.675024  8904 log.cc:927] Time spent T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.244s	user 0.001s	sys 0.001s
I20260709 12:17:52.672900  8862 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:52.719838  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:52.722692  6506 raft_consensus.cc:2393] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 4: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:52.669960  8795 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:52.718508  6505 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 4.
I20260709 12:17:52.692656  8731 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:52.739094  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:52.760615  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:52.706580  8850 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:52.792209  6651 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:52.792868  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:52.796777  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:52.801752  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:52.805126  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:52.736042  8795 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:52.666800  8814 consensus_queue.cc:237] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:52.715576  8432 consensus_queue.cc:1059] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 12:17:52.817752  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:52.759531  8822 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:52.823168  6651 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:52.794857  8842 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:52.824728  6506 raft_consensus.cc:2376] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8. Re-sending same reply.
I20260709 12:17:52.824617  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:52.834993  6651 raft_consensus.cc:1240] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:52.835332  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:52.836184  6502 raft_consensus.cc:2376] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3. Re-sending same reply.
I20260709 12:17:52.837531  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:52.841161  8910 log.cc:927] Time spent T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
W20260709 12:17:52.842941  8905 log.cc:927] Time spent T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
W20260709 12:17:52.844460  8906 log.cc:927] Time spent T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 12:17:52.819878  8731 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:17:52.847308  8852 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:52.820220  8543 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:52.851533  8543 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:52.852067  8543 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:52.828152  8847 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:52.862098  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:52.863312  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:52.864301  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:52.869900  8774 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:52.870780  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:52.871790  6505 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
new_leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:58944
I20260709 12:17:52.872761  6505 raft_consensus.cc:606] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Received request to transfer leadership to TS 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:52.869169  8808 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:52.873395  6505 raft_consensus.cc:614] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:52.874279  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:52.875224  6506 raft_consensus.cc:2393] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 4: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:52.875654  6505 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:17:52.879024  8879 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:52.879807  8511 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:52.882874  6651 raft_consensus.cc:1240] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:52.803627  8850 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:52.897462  8821 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:52.902454  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 321565us
I20260709 12:17:52.904021  6550 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:52.887670  8901 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:52.910382  6501 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 12:17:52.944192  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:52.902657  8833 consensus_meta.cc:287] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.964s	user 0.009s	sys 0.005s
I20260709 12:17:52.960976  6503 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 5. Current term is 6. Ops: []
W20260709 12:17:52.952677  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 378898us
W20260709 12:17:52.955157  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 224327us
I20260709 12:17:52.962477  7238 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:17:52.953405  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:52.969333  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:52.938651  8774 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:17:52.963428  8483 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.004s	sys 0.007s
I20260709 12:17:52.972524  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:52.979933  6579 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:17:52.980919  6583 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:52.979928  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:52.990422  6576 raft_consensus.cc:2376] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7. Re-sending same reply.
I20260709 12:17:52.994015  7238 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:17:52.980397  6581 raft_consensus.cc:1240] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:52.980888  6578 raft_consensus.cc:1240] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:52.960451  8832 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:52.981222  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:52.963805  8511 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:52.959939  6392 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 6f94b1cd286648a9b82d5abf69bb86d5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer 496c7c3792cd4587b4dbeba435f4be1e from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 4 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:17:52.990386  8483 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.004s	sys 0.008s
W20260709 12:17:53.028110  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:53.011356  6500 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:45870
I20260709 12:17:53.032634  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:53.011570  6473 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:52.998529  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:53.059407  6506 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:53.081019  6500 raft_consensus.cc:606] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Received request to transfer leadership
I20260709 12:17:53.046278  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:53.082583  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:53.084892  8511 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:53.079567  6392 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 122855us
I20260709 12:17:53.092010  8511 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:53.083400  6580 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:53.082401  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:53.098474  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:53.100761  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:53.100021  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:53.093258  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:53.093432  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:53.103086  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:53.104655  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:53.085714  6502 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 6.
I20260709 12:17:53.106221  6577 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:17:53.052538  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 72967.1us
I20260709 12:17:53.094925  8922 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:17:53.095607  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 84152.9us
I20260709 12:17:52.990486  8432 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:53.112772  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:53.060546  8842 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:53.113921  8821 consensus_queue.cc:1059] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:52.982033  8879 consensus_queue.cc:1059] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.133s
I20260709 12:17:53.114876  8821 consensus_queue.cc:1059] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:53.115275  8832 raft_consensus.cc:3055] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:53.115741  8832 raft_consensus.cc:740] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:53.115957  8821 raft_consensus.cc:3055] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:53.116496  8822 raft_consensus.cc:3055] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:53.116914  8822 raft_consensus.cc:740] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:53.116765  8832 consensus_queue.cc:260] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:53.118111  8832 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:53.117931  8822 consensus_queue.cc:260] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:53.119264  8822 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:17:53.120769  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:53.120944  8882 consensus_meta.cc:287] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.734s	user 0.012s	sys 0.000s
I20260709 12:17:53.125437  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:53.126111  6581 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:53.040917  8825 raft_consensus.cc:2804] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:53.095655  8795 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:53.113834  8842 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:53.121776  6622 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:53.107808  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:53.060598  8912 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:52.975157  8833 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.138s
I20260709 12:17:53.039134  8901 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:53.091588  8543 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:53.106012  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:53.107929  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:53.123826  6500 raft_consensus.cc:2376] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8. Re-sending same reply.
I20260709 12:17:53.039654  8840 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:53.125063  8882 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:53.158707  6508 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:17:52.981963  8808 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.131s
I20260709 12:17:53.129729  7240 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:17:53.130244  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 43958us
W20260709 12:17:53.169528  6548 connection.cc:256] RPC call timeout handler was delayed by -1.1567759569998088s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2577s for a response
I20260709 12:17:53.128379  8857 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.130883  6657 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:55116
I20260709 12:17:53.175089  6581 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:53.121377  8821 raft_consensus.cc:740] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:53.132071  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:53.135569  6622 leader_election.cc:400] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7.
I20260709 12:17:53.109376  7238 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:53.112671  8432 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:53.038244  8846 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:53.176452  6580 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 6.
I20260709 12:17:53.184757  6473 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:53.185072  6548 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:53.201398  6622 leader_election.cc:403] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:53.202622  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:53.156491  8861 consensus_meta.cc:287] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.668s	user 0.013s	sys 0.000s
I20260709 12:17:53.208977  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:53.138202  8927 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:17:53.217005  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 82991.2us
I20260709 12:17:53.156317  8543 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:17:53.222294  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 113024us
I20260709 12:17:53.202426  8923 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 12:17:53.228966  6622 leader_election.cc:400] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 7. Current term is 8.
I20260709 12:17:53.185075  8600 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:53.231108  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:53.234133  6657 raft_consensus.cc:606] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Received request to transfer leadership to TS 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:53.158048  8814 consensus_queue.cc:1059] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 12:17:53.205962  8432 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:53.158483  8922 consensus_queue.cc:1059] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.076s
I20260709 12:17:53.182850  8483 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 12:17:53.211292  8849 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:53.210589  8901 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
W20260709 12:17:53.235527  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 50539us
I20260709 12:17:53.239450  6657 raft_consensus.cc:614] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:53.242094  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:53.183413  8811 consensus_queue.cc:1059] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 12:17:53.237036  8851 raft_consensus.cc:2804] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:53.221472  8840 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7.
I20260709 12:17:53.238794  7238 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:53.229382  8825 raft_consensus.cc:697] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 4 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:53.254269  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:53.236891  8600 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:53.169533  8912 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:53.252622  6622 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:53.255102  6548 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:53.204636  8857 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:53.222226  8861 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:53.257555  6577 raft_consensus.cc:2368] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6.
I20260709 12:17:53.276221  6548 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:53.246496  8927 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:53.261291  8825 consensus_queue.cc:237] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:53.264811  8922 raft_consensus.cc:3055] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:53.254165  8851 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:53.266417  8849 raft_consensus.cc:697] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:53.267185  8903 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:53.265815  8912 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:53.269088  8882 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:53.272684  8930 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:53.274729  8945 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:53.274871  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 1617) took 1812 ms. Trace:
I20260709 12:17:53.291745  6507 rpcz_store.cc:276] 0709 12:17:51.462215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:51.639828 (+177613us) service_pool.cc:224] Handling call
0709 12:17:51.643907 (+  4079us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:52.658550 (+1014643us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:53.274632 (+616082us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540039f654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:53.274805 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2330412288}
I20260709 12:17:53.249620  8664 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:53.258677  6580 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:56112
I20260709 12:17:53.278754  8840 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:53.265813  8738 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:53.258066  8600 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:53.258119  8821 consensus_queue.cc:260] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:53.304222  8821 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:53.283553  6548 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:53.308993  8842 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:53.309648  8842 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:53.310168  8842 raft_consensus.cc:697] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:17:53.276316  8862 consensus_meta.cc:287] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.516s	user 0.011s	sys 0.000s
I20260709 12:17:53.284899  6473 leader_election.cc:400] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6.
I20260709 12:17:53.264091  8808 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:53.283051  8795 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:53.290443  8857 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:53.280522  8922 raft_consensus.cc:740] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:53.294611  6580 raft_consensus.cc:606] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Received request to transfer leadership
I20260709 12:17:53.279277  6622 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:53.285055  8851 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:53.298154  8664 raft_consensus.cc:697] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:53.286718  8944 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:53.292632  8903 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:53.300285  8849 consensus_queue.cc:237] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:53.326874  8940 log.cc:927] Time spent T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:17:53.327247  8842 consensus_queue.cc:237] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:53.313373  6473 leader_election.cc:403] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
W20260709 12:17:53.328612  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 43856.7us
I20260709 12:17:53.314792  8808 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:53.337707  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:53.334826  8907 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:53.322258  8857 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:53.337909  8846 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:53.330304  8664 consensus_queue.cc:237] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:53.339879  8922 consensus_queue.cc:260] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:53.341264  6509 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
W20260709 12:17:53.342396  8941 log.cc:927] Time spent T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:17:53.345816  8861 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:53.346653  6580 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:53.351660  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 103002us
I20260709 12:17:53.352386  6622 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:53.349795  8808 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:53.350579  8738 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:53.367036  8864 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:17:53.363380  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 124678us
I20260709 12:17:53.375262  7238 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:53.352751  8857 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:53.385608  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:53.361763  8840 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:53.367822  8432 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:53.393905  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:53.385761  8598 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:53.396103  8903 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:53.395956  8600 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:53.389673  8857 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:53.407074  6580 raft_consensus.cc:2368] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4.
I20260709 12:17:53.388882  8864 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.417281  6473 leader_election.cc:400] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4.
W20260709 12:17:53.399802  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 88815us
I20260709 12:17:53.417630  6473 leader_election.cc:403] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:17:53.392197  8922 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:53.392763  8731 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:53.418891  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:53.400055  8840 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:53.423494  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:17:53.431213  6654 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49792 (request call id 240) took 1494 ms (client timeout 1513 ms). Trace:
W20260709 12:17:53.431690  6654 rpcz_store.cc:269] 0709 12:17:51.936816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:51.936995 (+   179us) service_pool.cc:224] Handling call
0709 12:17:53.431034 (+1494039us) spinlock_profiling.cc:243] Waited 405 ms on lock 0x7b54001b0b54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:53.431149 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":930735360}
I20260709 12:17:53.433147  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:53.433768  6658 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:53.433967  8918 raft_consensus.cc:3055] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:53.434335  8918 raft_consensus.cc:740] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:53.435185  8918 consensus_queue.cc:260] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:53.436702  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 1632) took 1287 ms. Trace:
I20260709 12:17:53.437072  6508 rpcz_store.cc:276] 0709 12:17:52.158172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:52.270784 (+112612us) service_pool.cc:224] Handling call
0709 12:17:53.432161 (+1161377us) spinlock_profiling.cc:243] Waited 818 ms on lock 0x7b540001b654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:53.432261 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1880327552}
I20260709 12:17:53.406849  8861 consensus_queue.cc:1059] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 12:17:53.438546  8861 raft_consensus.cc:3055] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:53.438879  8861 raft_consensus.cc:740] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:53.439608  8861 consensus_queue.cc:260] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:53.442099  8600 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:53.442747  8850 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:17:53.445719  8925 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.318s	user 0.002s	sys 0.000s
I20260709 12:17:53.448664  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:53.449328  6651 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:53.422581  8864 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:53.455283  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:53.461673  6548 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:53.398926  8808 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:53.651903  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:53.532243  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:53.551380  6651 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:17:53.556914  6658 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
W20260709 12:17:53.452167  8932 log.cc:927] Time spent T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.345s	user 0.002s	sys 0.000s
W20260709 12:17:53.658726  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:53.426177  8840 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:53.424471  8849 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:53.662122  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:53.682722  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:17:53.689589  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 227953us
I20260709 12:17:53.691232  6548 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:53.697669  6548 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:53.699530  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1297) took 1015 ms. Trace:
I20260709 12:17:53.454927  8731 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:53.699891  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:53.700048  6649 rpcz_store.cc:276] 0709 12:17:52.684419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:52.794374 (+109955us) service_pool.cc:224] Handling call
0709 12:17:52.820081 (+ 25707us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:52.820361 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:17:52.831370 (+ 11009us) log.cc:844] Serialized 14 byte log entry
0709 12:17:53.030869 (+199499us) log.cc:844] Serialized 14 byte log entry
0709 12:17:53.435658 (+404789us) spinlock_profiling.cc:243] Waited 21 ms on lock 0x7b18003f7600. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d99455 00007f2875e81060 00007f2875e93716 00007f2875ea4287 00007f2875ea4208
0709 12:17:53.451657 (+ 15999us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:17:53.699178 (+247521us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:53.699227 (+    49us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:53.699273 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:53.699337 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:53.699479 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20071,"prepare.queue_time_us":37190,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":13484,"raft.queue_time_us":59152,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":158,"spinlock_wait_cycles":97253760,"thread_start_us":411195,"threads_started":2,"wal-append.queue_time_us":209403}
I20260709 12:17:53.466851  8855 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.502475  8432 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:53.651157  8918 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:53.701875  8483 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:53.657152  8861 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:53.757838  6549 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:17:53.761904  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 64334.7us
I20260709 12:17:53.762859  6548 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:53.763706  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:53.684978  8864 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:53.775369  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:53.703904  8731 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:53.765704  8757 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:17:53.761349  8932 log.cc:927] Time spent T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:17:53.788861  8731 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:53.824429  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 45743.7us
I20260709 12:17:53.766395  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:53.784250  8432 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:53.833510  8808 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:53.834743  6581 raft_consensus.cc:1240] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:53.854837  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:53.855404  6581 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:53.836622  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:53.788578  8923 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:53.838217  8907 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6.
W20260709 12:17:53.844041  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:53.844904  8955 raft_consensus.cc:993] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e: : Instructing follower 20a202c76244485fbc8f1610473db8b9 to start an election
I20260709 12:17:53.777608  8483 raft_consensus.cc:697] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:17:53.847193  8851 consensus_meta.cc:287] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.514s	user 0.008s	sys 0.000s
I20260709 12:17:53.762918  8862 consensus_queue.cc:1059] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.085s
I20260709 12:17:53.877044  6581 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:17:53.851902  8816 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.862157  8945 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:53.896935  8483 consensus_queue.cc:237] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:53.852306  8857 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:53.879536  8923 raft_consensus.cc:1081] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Signalling peer 20a202c76244485fbc8f1610473db8b9 to start an election
I20260709 12:17:53.864434  8903 consensus_queue.cc:1059] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 12:17:53.883635  8900 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.762341  8855 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:53.921684  8855 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:17:53.923851  8935 log.cc:927] Time spent T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.793s	user 0.010s	sys 0.010s
I20260709 12:17:53.929018  7238 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:53.930313  7238 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:53.931219  8862 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:17:53.931675  8918 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:53.932400  8918 raft_consensus.cc:697] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 8 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:53.933674  8918 consensus_queue.cc:237] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:53.935246  8857 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:17:53.940549  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:53.940801  6658 raft_consensus.cc:1240] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:53.761400  8774 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.850721  6548 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:53.849099  8757 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.004s
W20260709 12:17:53.858868  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 65178.5us
W20260709 12:17:53.948915  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 98260.2us
I20260709 12:17:53.887056  8816 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:53.887509  8664 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:53.946612  6578 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:53.951833  6548 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:53.847540  8849 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4.
I20260709 12:17:53.882861  8851 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:53.954612  6651 raft_consensus.cc:1275] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:17:53.958986  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 232295us
I20260709 12:17:53.940529  8900 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:53.967362  6548 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:53.953045  8952 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:53.956728  8842 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:17:53.952004  8774 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:53.959093  8664 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:53.968852  8847 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:53.986295  8664 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:53.982549  8840 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:53.975417  8945 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:54.016996  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 49970.5us
I20260709 12:17:54.024837  6656 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 5 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:54.026888  6656 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:54.027752  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:54.028767  6656 raft_consensus.cc:1240] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:54.029949  6575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1276) took 1052 ms. Trace:
I20260709 12:17:54.031494  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:54.026935  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:54.028911  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 72390.3us
I20260709 12:17:54.031391  6575 rpcz_store.cc:276] 0709 12:17:52.978048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:52.988990 (+ 10942us) service_pool.cc:224] Handling call
0709 12:17:52.994047 (+  5057us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:53.044281 (+ 50234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:53.044345 (+    64us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:53.044928 (+   583us) log.cc:844] Serialized 26 byte log entry
0709 12:17:53.206359 (+161431us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:53.269848 (+ 63489us) spinlock_profiling.cc:243] Waited 1.79 ms on lock 0x7b1800403540. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c522 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:17:54.029262 (+759414us) spinlock_profiling.cc:243] Waited 1.6 ms on lock 0x7b1800403540. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2875a314bc 00007f2872d7f48b 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d99455 00007f28798813c7 00007f2875e939a7 00007f2875ea30bf
0709 12:17:54.029414 (+   152us) log.cc:844] Serialized 14 byte log entry
0709 12:17:54.029642 (+   228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:54.029721 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:54.029760 (+    39us) raft_consensus.cc:1685] finished
0709 12:17:54.029771 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:54.029887 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50853,"prepare.queue_time_us":37748,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":2307,"raft.queue_time_us":52366,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":3305,"spinlock_wait_cycles":7788288,"thread_start_us":92772,"threads_started":2,"wal-append.queue_time_us":146200}
I20260709 12:17:54.035203  6655 raft_consensus.cc:2393] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 3: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:54.024462  8862 consensus_queue.cc:1048] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 12:17:54.037158  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:54.038015  6652 raft_consensus.cc:2393] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 5: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:54.035136  6653 raft_consensus.cc:1275] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:54.045521  6649 raft_consensus.cc:1240] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:17:54.049021  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:54.051255  6575 raft_consensus.cc:1275] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:17:54.053328  8840 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:54.038007  8684 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:54.065117  8718 raft_consensus.cc:3055] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:54.065773  8718 raft_consensus.cc:740] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:54.067169  8718 consensus_queue.cc:260] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:54.073235  8718 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:54.077566  8774 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:54.081545  8900 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:54.051652  6575 pending_rounds.cc:85] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 2
W20260709 12:17:54.084538  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 117236us
I20260709 12:17:54.085059  6575 pending_rounds.cc:107] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:17:54.085355  6548 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:54.085466  6575 raft_consensus.cc:2889] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:54.087934  6475 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:54.089272  6475 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:54.090456  6475 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:54.091749  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:54.092267  6650 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:54.094354  6475 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:54.033121  8511 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:54.101233  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:54.102244  6575 raft_consensus.cc:2393] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 3: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:54.102435  6475 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:54.103027  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:54.084074  8851 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:54.110270  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:54.114794  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:17:54.104730  8968 log.cc:927] Time spent T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:17:54.119220  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:54.119994  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:54.093068  8570 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.093731  8849 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.093021  8821 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:54.093565  8887 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:54.125059  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:54.097357  8814 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:54.126544  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:54.127563  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:54.100952  8903 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:54.113718  8942 log.cc:927] Time spent T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:17:54.106942  8900 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:17:54.136951  8900 raft_consensus.cc:697] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:54.137310  8888 raft_consensus.cc:3055] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:54.137728  8888 raft_consensus.cc:740] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:54.137909  8900 consensus_queue.cc:237] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:54.138535  8888 consensus_queue.cc:260] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:54.134934  6575 raft_consensus.cc:2393] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 8: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:17:54.142371  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:54.143097  6575 raft_consensus.cc:2376] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5. Re-sending same reply.
I20260709 12:17:54.143502  8720 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.143677  6403 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46184 (request call id 114) took 8007 ms. Trace:
I20260709 12:17:54.143975  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:54.145339  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:54.145931  6575 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:54.146733  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:54.150328  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:54.151456  6582 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:17:54.110882  8816 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:54.152554  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:54.114164  8907 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:54.153434  6622 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:54.154935  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:54.159578  7273 raft_consensus.cc:1240] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 8 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 7. Current term is 8. Ops: []
I20260709 12:17:54.159801  6622 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:54.160959  6622 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:54.161764  6622 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:54.157707  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:54.128278  8570 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:54.166355  7267 raft_consensus.cc:1240] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:54.166742  8570 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:54.168294  7270 raft_consensus.cc:1240] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:54.169695  8731 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:54.170040  8821 consensus_queue.cc:1059] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:54.170939  7266 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:54.171289  8811 raft_consensus.cc:3055] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:54.171793  8811 raft_consensus.cc:740] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:54.131908  8861 consensus_queue.cc:1059] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 12:17:54.173650  8861 raft_consensus.cc:3055] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:54.144129  6403 rpcz_store.cc:276] 0709 12:17:46.136087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:46.137031 (+   944us) service_pool.cc:224] Handling call
0709 12:17:46.451105 (+314074us) spinlock_profiling.cc:243] Waited 19.6 ms on lock 0x7b5400002210. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d650b2 00007f287bcde759 00007f287bc76a76 00007f287bc47d58 00007f287bd4604e 00007f287a2072a4 00007f287a207232 00007f287a207161 00007f287a2070dc
0709 12:17:54.143611 (+7692506us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:48.975102 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:48.975958 (+   856us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:48.976010 (+    52us) write_op.cc:435] Acquired schema lock
0709 12:17:48.976043 (+    33us) tablet.cc:662] Decoding operations
0709 12:17:48.978518 (+  2475us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:48.978611 (+    93us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:48.978657 (+    46us) tablet.cc:685] Acquiring locks for 13 operations
0709 12:17:49.021795 (+ 43138us) tablet.cc:701] Row locks acquired
0709 12:17:49.022031 (+   236us) write_op.cc:260] PREPARE: finished
0709 12:17:49.065316 (+ 43285us) write_op.cc:270] Start()
0709 12:17:49.069784 (+  4468us) write_op.cc:276] Timestamp: P: 1783599469065246 usec, L: 0
0709 12:17:49.071684 (+  1900us) op_driver.cc:348] REPLICATION: starting
0709 12:17:49.101044 (+ 29360us) log.cc:844] Serialized 5435 byte log entry
0709 12:17:49.465952 (+364908us) op_driver.cc:464] REPLICATION: finished
0709 12:17:49.793233 (+327281us) write_op.cc:301] APPLY: starting
0709 12:17:49.795211 (+  1978us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:49.892942 (+ 97731us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:49.945477 (+ 52535us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:50.300631 (+355154us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:50.678654 (+378023us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=13
0709 12:17:50.698600 (+ 19946us) write_op.cc:312] APPLY: finished
0709 12:17:50.831251 (+132651us) log.cc:844] Serialized 95 byte log entry
0709 12:17:51.211779 (+380528us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:51.239748 (+ 27969us) write_op.cc:454] Released schema lock
0709 12:17:51.239816 (+    68us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:51.293047 (+ 53231us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:51.296778 (+  3731us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":17421,"reported_tablets":17,"spinlock_wait_cycles":45048576,"child_traces":[["op",{"apply.queue_time_us":293033,"apply.run_cpu_time_us":33156,"apply.run_wall_time_us":1474677,"num_ops":13,"prepare.queue_time_us":122265,"prepare.run_cpu_time_us":63997,"prepare.run_wall_time_us":256526,"raft.queue_time_us":145034,"raft.run_cpu_time_us":50801,"raft.run_wall_time_us":279164,"replication_time_us":367009,"thread_start_us":775763,"threads_started":5,"wal-append.queue_time_us":540646,"wal-append.run_cpu_time_us":38490,"wal-append.run_wall_time_us":2614826}]]}
I20260709 12:17:54.176704  6650 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5.
I20260709 12:17:54.177727  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:54.180250  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:54.174021  8861 raft_consensus.cc:740] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:17:54.185556  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 91601.6us
W20260709 12:17:54.254232  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 98996.8us
I20260709 12:17:54.255291  6475 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:54.258167  8983 consensus_queue.cc:1059] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:17:54.258865  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 72304.2us
I20260709 12:17:54.259929  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:54.262321  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:54.175776  8850 consensus_queue.cc:1059] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.090s
I20260709 12:17:54.183562  8808 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:54.175419  8857 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:54.171829  8731 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:54.131069  8849 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:54.144011  8720 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:54.173055  8811 consensus_queue.cc:260] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:54.185258  8840 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:54.158890  8822 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.107s
I20260709 12:17:54.247660  8864 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:17:54.317968  8978 log.cc:927] Time spent T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:17:54.318266  8864 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:54.318883  8864 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:54.176748  8922 consensus_queue.cc:1048] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.095s
I20260709 12:17:54.321575  8922 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.322175  8922 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:54.262640  8570 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:54.324470  8850 consensus_queue.cc:1243] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Peer f2b06aebe97249a4877d798b8118ece4 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:17:54.248375  8960 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:54.325685  8960 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:54.326220  8960 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:54.262935  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:54.280079  8808 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:54.328774  8808 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:54.329164  8808 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:54.330947  8849 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:54.186190  8855 raft_consensus.cc:2764] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:17:54.299903  6622 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:54.312064  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:54.298177  8857 raft_consensus.cc:697] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 5 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:54.348814  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:54.325153  8850 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:54.300882  8887 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:54.302160  8814 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:54.325064  8570 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:54.313323  8840 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:54.356715  8922 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:54.358539  8888 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:54.379426  8840 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:54.360086  8855 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:54.381399  8855 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:54.381794  8855 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:54.364719  8857 consensus_queue.cc:237] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:54.384229  8570 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:54.368610  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 71894.6us
I20260709 12:17:54.386739  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:54.358235  8808 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:54.184554  8975 consensus_queue.cc:1059] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.165s
I20260709 12:17:54.355918  8960 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:54.390302  8973 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.394125  6653 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:54.312999  8861 consensus_queue.cc:260] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:54.398211  8856 raft_consensus.cc:3055] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:54.400544  8856 raft_consensus.cc:740] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:54.400451  8973 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:54.401705  8856 consensus_queue.cc:260] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:54.403385  8856 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:54.377195  8814 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:54.377311  8887 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:54.417573  8887 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:54.370412  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:54.526737  8973 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:54.533860  8922 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.534826  8922 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:54.485231  6509 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 1629) took 2341 ms (client timeout 2900 ms). Trace:
I20260709 12:17:54.536770  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:54.537861  8887 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:17:54.536976  6509 rpcz_store.cc:269] 0709 12:17:52.143495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:52.351918 (+208423us) service_pool.cc:224] Handling call
0709 12:17:54.484552 (+2132634us) spinlock_profiling.cc:243] Waited 985 ms on lock 0x7b540001b8d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:54.484912 (+   360us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2266262912}
I20260709 12:17:54.540956  8888 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:54.541184  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:54.542127  8888 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:54.542740  6576 raft_consensus.cc:2393] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 3: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:54.544699  8991 consensus_queue.cc:1059] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:54.545831  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:54.547423  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:54.550704  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 206) took 1838 ms. Trace:
I20260709 12:17:54.551352  6501 rpcz_store.cc:276] 0709 12:17:52.712106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:52.712217 (+   111us) service_pool.cc:224] Handling call
0709 12:17:52.758150 (+ 45933us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:52.912157 (+154007us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:54.549964 (+1637807us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54000ff354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:54.550396 (+   432us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":329126656}
I20260709 12:17:54.552006  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:54.592222  6579 raft_consensus.cc:2393] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 5: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:54.593616  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:54.594748  6579 raft_consensus.cc:2393] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 4: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:54.595834  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:54.478950  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:54.539584  8887 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:54.597482  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:54.598445  8887 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:54.510304  8808 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:54.553910  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 197) took 1991 ms. Trace:
I20260709 12:17:54.487262  6649 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:17:54.553778  8984 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:54.554745  8987 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:54.555804  8991 raft_consensus.cc:3055] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:54.555729  8990 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:54.600663  8991 raft_consensus.cc:740] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:54.600178  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:54.418777  8960 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:54.599666  6584 rpcz_store.cc:276] 0709 12:17:52.562272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:52.562924 (+   652us) service_pool.cc:224] Handling call
0709 12:17:52.583844 (+ 20920us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:52.891960 (+308116us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:52.892013 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:52.892649 (+   636us) log.cc:844] Serialized 26 byte log entry
0709 12:17:53.065770 (+173121us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:53.065829 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:54.552949 (+1487120us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b540034e4d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:54.553087 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:54.553192 (+   105us) raft_consensus.cc:1685] finished
0709 12:17:54.553211 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:54.553663 (+   452us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":100063,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":27501,"spinlock_wait_cycles":23864576,"thread_start_us":400318,"threads_started":2,"wal-append.queue_time_us":175698,"wal-append.run_cpu_time_us":1544,"wal-append.run_wall_time_us":1001568}
I20260709 12:17:54.601835  8990 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:54.602564  7268 raft_consensus.cc:2410] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:17:54.573239  8814 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:54.576460  7269 raft_consensus.cc:1275] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:54.574285  8840 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:54.578219  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:54.577396  7266 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5"
dest_uuid: "20a202c76244485fbc8f1610473db8b9"
 from {username='slave'} at 127.0.0.1:38562
I20260709 12:17:54.604952  7270 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:54.579425  7267 raft_consensus.cc:1240] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:54.605948  8990 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:54.606339  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:54.606488  8840 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:54.582019  8718 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:54.585400  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:54.587272  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 92017.6us
I20260709 12:17:54.544153  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:54.589859  8922 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:54.590345  8811 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:54.598943  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:54.558043  8816 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:54.603615  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:54.579197  8861 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:54.606838  7266 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:54.616611  7266 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:54.508762  8570 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:54.618391  8888 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:54.618988  8570 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:17:54.600914  8987 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:54.499655  6650 raft_consensus.cc:1240] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:54.540155  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:54.563931  6550 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:54.627027  6582 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:54.627655  8975 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:54.628391  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:54.628409  8975 raft_consensus.cc:697] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:54.622732  8816 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:54.622846  8825 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:54.600380  8984 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:54.603654  8991 consensus_queue.cc:260] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:54.855583  6575 raft_consensus.cc:2393] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 6: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:54.748826  7266 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:54.419093  8864 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:54.698652  8922 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:55.025360  8922 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:55.026325  6657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:55116 (request call id 3) took 1904 ms. Trace:
I20260709 12:17:55.026575  8598 raft_consensus.cc:3055] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:55.027055  8864 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:17:54.476016  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 220543us
I20260709 12:17:54.630010  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:54.893539  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 330138us
W20260709 12:17:54.820931  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:54.976794  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 145440us
I20260709 12:17:54.909345  6658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 260) took 1202 ms. Trace:
I20260709 12:17:55.033892  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.027063  8598 raft_consensus.cc:740] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:54.630518  6474 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:55.026862  6657 rpcz_store.cc:276] 0709 12:17:53.124943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:53.126730 (+  1787us) service_pool.cc:224] Handling call
0709 12:17:55.025699 (+1898969us) spinlock_profiling.cc:243] Waited 58.3 ms on lock 0x7b54001509d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e83853 00007f287ae5160a 00007f28743d77d4 00007f28743d7762 00007f28743d7691 00007f28743d760c 00007f28743d68c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:55.026047 (+   348us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":134080384}
I20260709 12:17:55.038290  6658 rpcz_store.cc:276] 0709 12:17:53.706923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:53.707123 (+   200us) service_pool.cc:224] Handling call
0709 12:17:53.707460 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:53.941269 (+233809us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:54.908610 (+967341us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b5400151154. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:54.909097 (+   487us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":536143232}
W20260709 12:17:54.907246  8977 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.741s	user 0.001s	sys 0.003s
I20260709 12:17:54.708359  8822 consensus_queue.cc:1048] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.336s
W20260709 12:17:54.822897  8933 log.cc:927] Time spent T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 1.668s	user 0.001s	sys 0.017s
I20260709 12:17:55.041208  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.045214  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.030917  8987 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:54.748586  8899 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:55.050932  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 116315us
I20260709 12:17:54.737625  8720 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:55.044633  8598 consensus_queue.cc:260] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:55.044479  8570 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:55.044580  8922 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:54.420568  8855 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:55.062183  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:55.064206  8987 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:55.064507  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:54.630564  8975 consensus_queue.cc:237] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:55.066982  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:55.068994  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:55.069842  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:55.071841  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:55.073370  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:55.073945  8856 consensus_meta.cc:287] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.670s	user 0.016s	sys 0.001s
I20260709 12:17:55.073877  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:55.075803  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:55.077044  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.085381  6578 raft_consensus.cc:2393] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 6: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:55.087603  7269 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:55.087888  8986 raft_consensus.cc:3055] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:55.088088  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.088439  8986 raft_consensus.cc:740] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:55.088852  6651 raft_consensus.cc:2393] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 4: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:55.089178  6582 raft_consensus.cc:2393] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 4: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:55.090281  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.089855  8986 consensus_queue.cc:260] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:55.091693  6582 raft_consensus.cc:2393] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 4: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:55.090907  8944 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 12:17:55.075922  6401 catalog_manager.cc:5697] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 2 to 5, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 5 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:17:55.101449  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.106279  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.092949  8899 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:55.105684  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.100782  7267 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:55.114418  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:55.120524  8924 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.934s	user 0.029s	sys 0.016s
I20260709 12:17:55.120960  6501 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:45870
I20260709 12:17:55.124825  6501 raft_consensus.cc:606] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Received request to transfer leadership
I20260709 12:17:55.125545  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:55.121013  6392 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 3a47d71c34b541b9b2589b4e80413fa2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 496c7c3792cd4587b4dbeba435f4be1e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:17:55.126637  8924 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.010s	user 0.029s	sys 0.016s
I20260709 12:17:55.122797  6655 raft_consensus.cc:1240] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 3. Current term is 4. Ops: []
W20260709 12:17:55.123394  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.121579  7266 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:55.117437  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:55.132797  8855 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:55.119016  6653 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:17:55.136163  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.138325  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:55.140928  6657 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 4: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:55.141537  8986 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:55.145277  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.148494  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.147137  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1523) took 1011 ms. Trace:
W20260709 12:17:55.151695  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 521386us
I20260709 12:17:55.150444  6577 rpcz_store.cc:276] 0709 12:17:54.138236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:55.093749 (+955513us) service_pool.cc:224] Handling call
0709 12:17:55.147080 (+ 53331us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:55.153782  6582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1525) took 1014 ms. Trace:
I20260709 12:17:55.154726  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:55.156004  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:55.135852  8864 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:55.154132  6582 rpcz_store.cc:276] 0709 12:17:54.138865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:55.109331 (+970466us) service_pool.cc:224] Handling call
0709 12:17:55.153703 (+ 44372us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:55.158211  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.160948  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.165730  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:55.166584  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1534) took 1025 ms. Trace:
I20260709 12:17:55.169046  6584 rpcz_store.cc:276] 0709 12:17:54.141529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:54.663043 (+521514us) service_pool.cc:224] Handling call
0709 12:17:55.156543 (+493500us) spinlock_profiling.cc:243] Waited 2.66 ms on lock 0x7b5800116198. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:17:55.166520 (+  9977us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6126336}
I20260709 12:17:55.170022  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.171823  6657 raft_consensus.cc:2393] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 6: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:55.170661  6549 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
W20260709 12:17:55.176360  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.175366  6577 raft_consensus.cc:1275] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:55.179519  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.180732  6655 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:17:55.181565  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.168610  8855 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:55.177587  8861 consensus_meta.cc:287] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.561s	user 0.011s	sys 0.003s
I20260709 12:17:55.178860  8840 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:55.177210  8987 consensus_queue.cc:1059] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 12:17:55.185483  8840 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:55.186420  8987 consensus_queue.cc:1243] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer 20a202c76244485fbc8f1610473db8b9 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:17:55.178231  8973 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:17:55.188112  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 106996us
I20260709 12:17:55.183239  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1516) took 1073 ms. Trace:
I20260709 12:17:55.176766  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:55.189630  8840 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:55.190692  8924 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:55.190732  6652 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:55.190915  8973 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:55.191233  8924 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:55.193528  8924 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:55.195077  8979 log.cc:927] Time spent T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 1.057s	user 0.002s	sys 0.000s
I20260709 12:17:55.197585  6401 catalog_manager.cc:5697] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 3 to 4, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 4 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:55.188820  6577 rpcz_store.cc:276] 0709 12:17:54.105109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:55.159492 (+1054383us) service_pool.cc:224] Handling call
0709 12:17:55.160015 (+   523us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:55.178240 (+ 18225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:55.178373 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:55.187232  8861 raft_consensus.cc:3055] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:55.200271  8987 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:55.202942  8861 raft_consensus.cc:740] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:55.203316  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:55.203583  8987 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:55.211269  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:55.181350  8864 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:55.210510  8973 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:55.213239  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:55.217458  6474 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:55.219300  6652 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:17:55.209007  8816 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:55.205132  8861 consensus_queue.cc:260] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:55.218211  8825 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 12:17:55.220727  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:55.223243  8944 consensus_queue.cc:1059] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:17:55.227128  6582 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:55.223562  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:55.225378  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:55.207369  8855 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:55.224797  8570 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:55.226449  6584 raft_consensus.cc:1275] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:55.228852  6650 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:55.228910  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:55.230121  6578 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:17:55.232965  6582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1535) took 1086 ms. Trace:
I20260709 12:17:55.233359  6582 rpcz_store.cc:276] 0709 12:17:54.141775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:55.172113 (+1030338us) service_pool.cc:224] Handling call
0709 12:17:55.209928 (+ 37815us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:55.227492 (+ 17564us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:55.227711 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:55.234360  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 217) took 1085 ms. Trace:
I20260709 12:17:55.234762  6584 rpcz_store.cc:276] 0709 12:17:54.147238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:55.189968 (+1042730us) service_pool.cc:224] Handling call
0709 12:17:55.223982 (+ 34014us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:55.231990 (+  8008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:55.232207 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:55.235694  6581 raft_consensus.cc:2393] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 6: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:55.237366  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1532) took 1091 ms. Trace:
I20260709 12:17:55.238529  8825 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:55.237712  6578 rpcz_store.cc:276] 0709 12:17:54.140999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:55.167917 (+1026918us) service_pool.cc:224] Handling call
0709 12:17:55.207908 (+ 39991us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:55.232495 (+ 24587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:55.232775 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:55.239228  8825 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:55.239203  8951 log.cc:927] Time spent T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.380s	user 0.002s	sys 0.001s
I20260709 12:17:55.244626  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.250245  6652 raft_consensus.cc:2393] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 6: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:55.250733  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.255239  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:55.255532  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.256853  6651 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:17:55.258021  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.259403  8892 consensus_queue.cc:1059] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:55.260623  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.262763  8720 raft_consensus.cc:3055] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:55.264155  6577 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:55.264230  6576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1537) took 1122 ms. Trace:
I20260709 12:17:55.264724  6576 rpcz_store.cc:276] 0709 12:17:54.142252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:55.254841 (+1112589us) service_pool.cc:224] Handling call
0709 12:17:55.264171 (+  9330us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:55.255475  6652 raft_consensus.cc:1240] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:55.246788  8825 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:55.248735  8987 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:55.272997  6575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1529) took 1129 ms. Trace:
W20260709 12:17:55.273599  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 83304.4us
I20260709 12:17:55.263218  8720 raft_consensus.cc:740] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:55.263293  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:55.280866  7271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38558 (request call id 395) took 1125 ms. Trace:
I20260709 12:17:55.260217  8543 consensus_queue.cc:1059] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:17:55.280961  8822 consensus_queue.cc:1059] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:17:55.281359  7271 rpcz_store.cc:276] 0709 12:17:54.155571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:54.159350 (+  3779us) service_pool.cc:224] Handling call
0709 12:17:55.262965 (+1103615us) spinlock_profiling.cc:243] Waited 57.3 ms on lock 0x7b5800125198. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:17:55.280790 (+ 17825us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":131822336}
I20260709 12:17:55.282047  8543 raft_consensus.cc:3055] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:55.282473  8543 raft_consensus.cc:740] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:55.282392  8822 consensus_queue.cc:1059] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:17:55.282842  8899 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:55.283489  8899 raft_consensus.cc:3055] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:55.282838  8922 raft_consensus.cc:3055] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:55.274994  8570 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:55.284422  8922 raft_consensus.cc:740] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:55.285218  8570 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:55.285713  8899 raft_consensus.cc:740] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:55.286552  8922 consensus_queue.cc:260] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:55.287034  8899 consensus_queue.cc:260] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:55.277599  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: true dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:55.288823  6654 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:55.284404  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1295) took 2004 ms. Trace:
I20260709 12:17:55.273329  6575 rpcz_store.cc:276] 0709 12:17:54.139937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:54.147574 (+  7637us) service_pool.cc:224] Handling call
0709 12:17:55.263843 (+1116269us) spinlock_profiling.cc:243] Waited 673 ms on lock 0x7b540034dad4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:55.269693 (+  5850us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1548420992}
I20260709 12:17:55.287729  8720 consensus_queue.cc:260] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:55.289896  6583 rpcz_store.cc:276] 0709 12:17:53.279957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:53.280110 (+   153us) service_pool.cc:224] Handling call
0709 12:17:53.280701 (+   591us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:17:53.285901 (+  5200us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:17:53.286285 (+   384us) log.cc:844] Serialized 14 byte log entry
0709 12:17:53.399403 (+113118us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:17:53.406953 (+  7550us) log.cc:844] Serialized 592 byte log entry
0709 12:17:53.407195 (+   242us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:17:53.412477 (+  5282us) log.cc:844] Serialized 14 byte log entry
0709 12:17:53.415280 (+  2803us) log.cc:844] Serialized 14 byte log entry
0709 12:17:53.415589 (+   309us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:53.415659 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:53.949815 (+534156us) spinlock_profiling.cc:243] Waited 2.11 ms on lock 0x7b180009b680. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:17:54.366828 (+417013us) spinlock_profiling.cc:243] Waited 3.1 ms on lock 0x7b180009b680. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:17:54.858261 (+491433us) spinlock_profiling.cc:243] Waited 9.78 ms on lock 0x7b180009b680. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:17:55.229715 (+371454us) spinlock_profiling.cc:243] Waited 19.3 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:17:55.283595 (+ 53880us) raft_consensus.cc:1685] finished
0709 12:17:55.283620 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:55.284024 (+   404us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24107,"prepare.queue_time_us":4372,"prepare.run_cpu_time_us":121,"prepare.run_wall_time_us":132,"raft.queue_time_us":18873,"raft.run_cpu_time_us":487,"raft.run_wall_time_us":11042,"spinlock_wait_cycles":78827776,"thread_start_us":96433,"threads_started":1,"wal-append.queue_time_us":129853}
I20260709 12:17:55.292789  8720 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:55.293972  8921 raft_consensus.cc:3055] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:55.294457  8921 raft_consensus.cc:740] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:55.294131  6474 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:55.295847  8989 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:55.288172  8922 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:55.283777  8543 consensus_queue.cc:260] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:55.298110  8543 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:55.287834  8570 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:55.302357  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.305676  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:55.309662  6501 raft_consensus.cc:2410] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 8 because replica has last-logged OpId of term: 7 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 2.
I20260709 12:17:55.305711  8984 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:55.316990  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:55.317826  6651 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 1.
I20260709 12:17:55.319124  6654 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 6.
I20260709 12:17:55.319530  8822 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:55.320452  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.320268  8822 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:55.321969  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:55.322649  8822 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:55.323336  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:55.320185  8864 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:55.324286  6500 raft_consensus.cc:2410] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:17:55.325316  8852 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:55.332850  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 218) took 1162 ms. Trace:
I20260709 12:17:55.325917  8852 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:55.335333  6579 raft_consensus.cc:2393] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 8: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:17:55.333526  6581 rpcz_store.cc:276] 0709 12:17:54.170692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:55.251669 (+1080977us) service_pool.cc:224] Handling call
0709 12:17:55.301618 (+ 49949us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:55.326476 (+ 24858us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:55.326509 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:55.332525 (+  6016us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:17:55.332575 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:55.332650 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:55.332790 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:17:55.340161  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:55.333343  6401 catalog_manager.cc:5697] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 4 to 6, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 6 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:17:55.336018  8598 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:55.346041  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:55.347218  6658 raft_consensus.cc:2410] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:17:55.348207  8864 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:55.348702  6651 raft_consensus.cc:1240] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:17:55.348961  8864 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:55.351045  6653 raft_consensus.cc:1240] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:55.351269  8864 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:55.295557  8921 consensus_queue.cc:260] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:55.353909  8921 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:55.354240  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:55.354941  6658 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:17:55.355808  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:55.356813  6658 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
W20260709 12:17:55.357697  8811 consensus_meta.cc:287] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.681s	user 0.008s	sys 0.005s
I20260709 12:17:55.357856  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:55.358050  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:55.358722  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:55.359190  6653 raft_consensus.cc:2410] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:17:55.360014  8944 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:55.360213  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:55.360987  6653 raft_consensus.cc:2393] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 4: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:55.360700  8944 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:55.361312  8899 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:55.362059  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:55.362774  6653 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:55.363359  6650 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:55.364370  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:55.364790  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:55.359947  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:55.365684  6653 raft_consensus.cc:2393] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 4: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:55.363204  8944 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:55.357714  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:55.373118  6577 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:55.374619  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.365746  6501 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:55.374652  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:55.377640  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.378381  6576 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:55.363219  8991 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:55.372977  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:55.310935  8924 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:55.384477  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.382516  6401 catalog_manager.cc:5697] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 4 to 5, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 5 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } }
W20260709 12:17:55.370353  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 162532us
I20260709 12:17:55.393982  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:55.394961  6653 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:55.396535  6401 catalog_manager.cc:5697] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 1 to 5, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 5 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:17:55.400755  6650 raft_consensus.cc:2410] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:17:55.401182  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 86320.2us
I20260709 12:17:55.402076  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:55.402918  6501 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
I20260709 12:17:55.372184  8922 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:55.411422  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:55.414784  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.415092  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:55.416198  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.415759  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:55.419713  6577 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:55.423542  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:55.425412  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.426034  6577 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:55.427804  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.428079  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:55.428978  6576 raft_consensus.cc:2393] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 6: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:55.428989  6508 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:17:55.436580  6401 catalog_manager.cc:5697] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:17:55.405627  8861 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:55.436493  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:55.444581  8855 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:55.444799  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:55.445499  7271 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:55.448755  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:55.449509  6650 raft_consensus.cc:2376] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4. Re-sending same reply.
I20260709 12:17:55.450683  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:55.451319  8924 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:55.451349  7269 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:55.426374  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.451375  6500 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:17:55.454839  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.452322  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:55.457177  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.458190  6509 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:55.458801  6579 raft_consensus.cc:2368] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5.
I20260709 12:17:55.452876  8852 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:55.464993  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:55.466037  6507 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:55.470012  6649 raft_consensus.cc:2410] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:17:55.475677  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.476498  6581 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 4.
I20260709 12:17:55.477134  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.478361  6583 raft_consensus.cc:2393] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 4: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:55.453936  8840 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:55.486330  6653 raft_consensus.cc:2410] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:17:55.489449  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.454523  8922 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:55.491375  8840 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:55.495460  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.490023  6578 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:17:55.484599  6401 catalog_manager.cc:5697] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 1 to 5, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 5 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
W20260709 12:17:55.501324  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 81890.2us
I20260709 12:17:55.505175  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:55.505967  6503 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:55.517508  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:55.519054  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:55.519773  6503 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:55.521481  8840 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:55.523346  7271 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:55.525849  6401 catalog_manager.cc:5697] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 2 to 4, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 4 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:17:55.528537  8922 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:55.529383  7240 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:17:55.534375  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 197425us
W20260709 12:17:55.537614  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:55.538384  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.539215  6509 raft_consensus.cc:1275] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:55.542768  6503 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:17:55.546440  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:55.541077  8598 raft_consensus.cc:2804] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:55.552775  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:55.554167  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:55.557375  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:55.555550  8598 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:17:55.580020  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.581452  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:55.582311  6509 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:55.583370  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:55.575938  8598 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:55.595736  6503 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:17:55.608055  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:55.589534  8852 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:55.611008  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:55.612665  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:55.617082  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:55.627692  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.641029  6501 raft_consensus.cc:2410] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:17:55.643160  6509 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:55.644930  6509 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:55.612241  8852 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:55.646560  7272 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:55.647485  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:55.648185  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:17:55.652415  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.653307  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:55.654163  6501 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
I20260709 12:17:55.657642  6623 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:55.658922  6623 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:55.659782  6623 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:55.659874  8598 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:55.660557  8855 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:55.660688  8840 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:55.661018  8855 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:55.661249  8840 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:55.661475  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: true dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:55.666989  6579 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:55.661762  8840 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:55.669085  6584 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:17:55.662938  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.672808  8856 consensus_queue.cc:1059] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:55.653810  8921 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:55.674190  6502 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:55.674309  8921 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:55.664988  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.666029  7271 raft_consensus.cc:1240] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:55.661559  8855 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:55.683228  6509 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:17:55.684811  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:55.693078  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:55.695207  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:55.686028  6509 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:55.707650  6579 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 6.
I20260709 12:17:55.708736  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:55.712018  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:55.712646  6501 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
W20260709 12:17:55.707676  6622 leader_election.cc:336] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.569s (SENT)
I20260709 12:17:55.719092  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:55.720669  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:55.724960  8864 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:55.726329  8864 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:55.728946  6508 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
I20260709 12:17:55.680184  8921 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:55.737578  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: true dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:55.686379  8986 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:55.756356  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:55.753795  6500 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:17:55.737838  6506 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
I20260709 12:17:55.767378  6509 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7.
I20260709 12:17:55.768814  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:55.769537  6506 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 4.
I20260709 12:17:55.770473  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:55.771209  6506 raft_consensus.cc:2393] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 6: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:55.738003  8852 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:55.773478  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:55.774809  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:55.775223  8598 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:55.775946  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:55.776424  6623 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:55.777555  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:55.778235  6503 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:55.778817  6501 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:55.783037  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:55.732872  8924 raft_consensus.cc:3055] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:55.802147  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:55.806846  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.807240  6506 raft_consensus.cc:2410] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:17:55.810155  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 155615us
I20260709 12:17:55.811193  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:55.811977  6652 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:55.803879  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:55.797437  6473 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:17:55.818953  9017 log.cc:927] Time spent T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.119s	user 0.000s	sys 0.002s
I20260709 12:17:55.829732  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:55.830636  6651 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
W20260709 12:17:55.831594  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 55301.6us
I20260709 12:17:55.788938  8924 raft_consensus.cc:740] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:55.846333  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:55.847889  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:55.848846  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.848871  6653 raft_consensus.cc:2410] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 9 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:17:55.850931  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:55.867357  6622 leader_election.cc:336] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.608s (SENT)
I20260709 12:17:55.870764  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:55.825834  8864 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:55.826217  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:55.868096  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:55.891745  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:55.891979  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 114332us
I20260709 12:17:55.865896  8822 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:55.915398  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:55.917182  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:55.916301  6506 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 6.
I20260709 12:17:55.919319  6505 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 6.
I20260709 12:17:55.886736  8986 raft_consensus.cc:697] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:17:55.922233  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 301853us
I20260709 12:17:55.906960  6500 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 6.
I20260709 12:17:55.907954  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:17:55.931048  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:55.908545  9023 consensus_queue.cc:1059] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 12:17:55.928396  6654 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:17:55.917868  8861 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:55.940266  8840 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:55.941185  8986 consensus_queue.cc:237] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:55.944425  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:55.983107  6656 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:17:55.983513  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:55.984907  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:55.986040  6504 raft_consensus.cc:1275] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:55.987581  6504 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:17:55.947510  9028 log.cc:927] Time spent T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.255s	user 0.004s	sys 0.000s
I20260709 12:17:55.921788  8864 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:55.927115  8855 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:55.948745  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 103960us
I20260709 12:17:55.929078  8924 consensus_queue.cc:260] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:55.917336  9033 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:55.953773  6473 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:55.957499  7266 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:55.969241  6501 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:55.959599  6503 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:55.988476  8861 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:55.997470  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 43771.1us
I20260709 12:17:55.998185  6502 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:55.945058  6548 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:56.005596  6501 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:17:55.999039  6652 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 2.
I20260709 12:17:56.007120  6503 raft_consensus.cc:1240] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:55.997624  6505 raft_consensus.cc:1240] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:55.991750  8864 raft_consensus.cc:697] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:56.005961  6501 pending_rounds.cc:85] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 4
I20260709 12:17:56.008026  7240 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:56.018858  6473 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:56.005229  8922 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:56.033931  6550 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:56.014140  6501 pending_rounds.cc:107] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:17:56.008587  8924 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:56.041571  6473 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:17:56.050680  6622 leader_election.cc:336] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.689s (SENT)
I20260709 12:17:56.047581  6501 raft_consensus.cc:2889] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:17:56.044116  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:56.046074  6500 raft_consensus.cc:1275] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:17:56.053284  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 157316us
W20260709 12:17:56.055701  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:56.056782  6508 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 5. Current term is 6. Ops: []
W20260709 12:17:56.060838  6622 leader_election.cc:336] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.689s (SENT)
I20260709 12:17:56.061584  6622 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:56.038520  9023 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:56.044821  8598 raft_consensus.cc:2804] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:17:56.066762  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 100200us
I20260709 12:17:56.047101  8922 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:56.067693  6548 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:56.075323  6475 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:56.074435  8598 raft_consensus.cc:697] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 2 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:56.078562  8864 consensus_queue.cc:237] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:56.079278  8598 consensus_queue.cc:237] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:56.081602  9021 log.cc:927] Time spent T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.415s	user 0.002s	sys 0.000s
I20260709 12:17:56.084190  8921 raft_consensus.cc:3055] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:56.072234  8922 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:56.072243  8975 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:56.071699  9033 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:56.074864  8840 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:17:56.120764  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 85466.2us
W20260709 12:17:56.121726  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 46455us
I20260709 12:17:56.127715  6475 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:56.084679  8921 raft_consensus.cc:740] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:56.129204  7238 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:56.129993  9020 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:56.147010  6622 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:56.133163  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:56.150712  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:56.152313  9046 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:56.152920  9046 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:56.153280  9046 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:56.159673  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:56.135114  9038 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:56.166517  6622 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:56.167199  8840 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:56.147332  9020 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:56.145258  8855 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:56.180727  7272 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:56.182405  6623 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:56.169734  9038 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:56.190408  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 63162.7us
I20260709 12:17:56.191437  6473 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:56.171130  9020 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:56.173605  8861 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:56.205915  6473 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:56.187302  9033 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:56.204483  6548 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:56.190719  9034 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:56.190721  8852 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:17:56.211606  9037 log.cc:927] Time spent T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
W20260709 12:17:56.213837  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 205565us
I20260709 12:17:56.199541  9038 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:56.218387  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 133239us
I20260709 12:17:56.219581  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:56.221035  9044 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:17:56.222854  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 56445.5us
I20260709 12:17:56.222884  9044 raft_consensus.cc:697] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:56.224776  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:56.224998  9044 consensus_queue.cc:237] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:56.226426  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:56.236488  8855 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:56.234810  8892 consensus_queue.cc:1048] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 12:17:56.237958  9042 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:17:56.238325  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 110686us
I20260709 12:17:56.238417  9042 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:56.239084  9042 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:56.240605  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:56.241235  6579 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:56.242090  7238 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:56.243314  9039 log.cc:927] Time spent T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 12:17:56.245414  9045 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:56.247514  6473 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:56.248709  9045 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:56.249148  9045 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:56.249179  7266 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:17:56.249423  6475 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:56.249735  9045 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:56.251123  8852 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:56.251572  8852 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:56.251503  6475 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:56.247337  9042 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:56.245311  9051 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:56.253794  9051 consensus_queue.cc:1059] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:56.254034  9042 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:56.254665  9042 consensus_queue.cc:1059] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:56.255064  9051 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:56.255501  9051 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:56.255671  9042 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:56.255884  9051 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:56.256117  9042 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:56.257712  9042 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:56.258203  9045 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:56.260267  9045 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:56.210119  8861 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:56.264443  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:56.265229  6579 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:56.252137  8852 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:56.232393  9034 raft_consensus.cc:697] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:56.272522  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:56.273193  6578 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:56.203280  8921 consensus_queue.cc:260] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:56.263571  8861 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:56.263099  9045 raft_consensus.cc:3055] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:56.284579  8921 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:56.285063  8840 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:56.289244  9034 consensus_queue.cc:237] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:56.295214  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 70538.6us
I20260709 12:17:56.282426  6550 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:56.326483  6579 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:17:56.336928  6473 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:56.295426  8861 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:56.347122  9047 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:56.347851  9047 raft_consensus.cc:697] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:56.353268  8840 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:17:56.353417  6549 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:17:56.355762  8840 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:56.355808  8856 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:56.356513  6622 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:56.356909  8856 raft_consensus.cc:697] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:56.358058  6622 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:56.358218  8856 consensus_queue.cc:237] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:56.359030  8855 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:56.359724  8855 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:56.359901  8892 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:56.360456  8892 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:56.362018  9040 log.cc:927] Time spent T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:17:56.364719  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:56.365471  6652 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:17:56.270912  8924 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.060s
I20260709 12:17:56.368809  8855 consensus_queue.cc:1048] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:56.371616  8855 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:56.369311  8892 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:56.376978  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 109877us
I20260709 12:17:56.377982  6550 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:56.379855  8944 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:56.380297  8944 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:56.380684  8944 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:56.381562  8975 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:56.382623  8975 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:56.384541  6577 raft_consensus.cc:1240] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:56.392848  9038 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:56.376138  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:56.296356  9045 raft_consensus.cc:740] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:56.413499  6578 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:56.415627  6473 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:56.345285  9022 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:56.420593  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 178574us
I20260709 12:17:56.352020  8922 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:56.424690  7238 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:56.409215  9052 consensus_queue.cc:1059] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
W20260709 12:17:56.435564  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 184128us
I20260709 12:17:56.436450  6475 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:56.426229  9054 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:56.437255  7269 raft_consensus.cc:1240] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 2 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:56.451090  8861 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:56.453504  8922 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:56.458279  8852 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:56.455780  9046 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:56.461503  6622 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:56.462373  8840 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:56.463773  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:56.464402  6579 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:56.467773  8944 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:56.471434  6577 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:56.472188  9046 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:56.474344  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:56.475073  6578 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:56.492017  9059 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:56.460054  9042 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:56.493353  6622 leader_election.cc:400] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5.
I20260709 12:17:56.525710  6578 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:17:56.527859  6473 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:56.528786  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:56.531128  6579 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:56.488827  9052 raft_consensus.cc:3055] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:56.487083  8922 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:56.535830  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:56.536939  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:56.544749  6501 raft_consensus.cc:2368] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5.
I20260709 12:17:56.544955  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:56.547094  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:56.547333  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:56.548844  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:56.550354  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:56.554426  6576 raft_consensus.cc:2393] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 4: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:56.558048  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:56.542975  6473 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:17:56.543098  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 49601.8us
I20260709 12:17:56.562904  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:56.565901  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:56.566200  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:56.532619  8839 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:56.561810  9054 raft_consensus.cc:697] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:56.489777  9047 consensus_queue.cc:237] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:56.572540  6578 raft_consensus.cc:1240] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:17:56.572758  9054 consensus_queue.cc:237] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:56.574288  6579 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 5. Current term is 6. Ops: [4.3-5.4]
I20260709 12:17:56.566048  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:56.579869  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:56.567301  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:56.581908  6503 raft_consensus.cc:2393] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 4: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:56.538781  9052 raft_consensus.cc:740] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:56.582787  6575 raft_consensus.cc:2368] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5.
I20260709 12:17:56.553022  8899 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:56.586359  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:56.524688  9046 consensus_queue.cc:1059] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 12:17:56.587613  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:56.571210  6622 connection.cc:256] RPC call timeout handler was delayed by -1.0772355660001267s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2189s for a response
W20260709 12:17:56.592443  6622 connection.cc:256] RPC call timeout handler was delayed by -1.0692063750000216s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16160000000000002s for a response
I20260709 12:17:56.592679  6501 raft_consensus.cc:2410] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:17:56.588210  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:56.588299  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:56.591444  6506 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:56.593405  6579 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:17:56.598006  6622 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.938s (SENT) [suppressed 51 similar messages]
I20260709 12:17:56.600772  9066 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:56.601522  9034 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:56.602092  9034 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:56.602933  6622 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:56.598214  8852 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:56.605188  6582 raft_consensus.cc:2410] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:17:56.605571  9034 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:56.606987  7271 raft_consensus.cc:1275] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:56.608861  8864 consensus_queue.cc:1048] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:56.610538  8864 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:56.612285  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:56.612854  9034 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:56.613817  8840 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:56.614405  8840 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:56.614625  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:56.614880  8840 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:56.615916  8864 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:56.616962  8864 raft_consensus.cc:697] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 5 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:56.618652  8864 consensus_queue.cc:237] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:56.619436  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:56.620081  9042 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:56.620842  9034 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:56.630744  6550 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:56.544219  9051 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:56.637919  9051 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:56.646046  6550 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:56.647197  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 55087.6us
I20260709 12:17:56.651814  8975 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:56.652918  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 228895us
I20260709 12:17:56.645627  8852 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:56.655665  6550 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:56.656957  6579 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:56.657642  6622 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:56.658178  6473 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:56.649197  8944 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:56.658751  9034 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:56.659303  9036 raft_consensus.cc:2764] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260709 12:17:56.650684  9066 raft_consensus.cc:697] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:17:56.652149  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 215791us
I20260709 12:17:56.590776  9020 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:56.660990  9066 consensus_queue.cc:237] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:56.661653  6475 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:56.662513  9020 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:56.662829  9069 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:56.649689  8899 raft_consensus.cc:697] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 6 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:56.664067  9063 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:56.665257  8899 consensus_queue.cc:237] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:56.666798  6581 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:56.659024  8852 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:56.674257  7238 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:56.686390  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:56.687422  8598 raft_consensus.cc:2804] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:56.694350  8852 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:56.695238  8598 raft_consensus.cc:697] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 6 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:56.696408  8598 consensus_queue.cc:237] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:56.698088  9074 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:56.699962  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 47657.2us
I20260709 12:17:56.703850  8861 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:56.707034  8839 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:56.706874  9045 consensus_queue.cc:260] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:56.707538  8839 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:56.708287  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 50207.2us
I20260709 12:17:56.710117  9047 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:56.708397  9045 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:56.711836  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:56.714071  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:56.694439  6475 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:56.715379  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:56.716754  6652 raft_consensus.cc:1275] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:17:56.718802  6654 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:56.719511  6652 pending_rounds.cc:85] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 1
I20260709 12:17:56.719959  6652 pending_rounds.cc:107] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:17:56.720248  6652 raft_consensus.cc:2889] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:56.723914  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:56.711032  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:56.725291  6649 raft_consensus.cc:2393] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 6: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:56.727221  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:56.727787  6577 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:56.730158  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:56.735672  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:56.736625  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:56.737196  6582 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7.
I20260709 12:17:56.740923  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:56.746259  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:56.749931  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:56.754390  6575 raft_consensus.cc:1275] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:17:56.756572  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 62241.1us
I20260709 12:17:56.757542  6475 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:17:56.755920  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 69592.3us
I20260709 12:17:56.766839  6475 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:56.737864  9034 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:56.788111  6622 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:56.788409  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:56.789680  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:56.790154  9033 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:56.790371  6622 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:56.792721  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:56.795051  6649 raft_consensus.cc:1275] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:17:56.796226  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:56.746337  8855 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:56.806339  6475 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:56.818140  6654 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:17:56.761417  9047 raft_consensus.cc:697] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 4 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:56.824276  6577 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5.
I20260709 12:17:56.833585  6475 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:56.842365  6475 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:56.820309  9081 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:56.821810  9074 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:56.820544  9046 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:56.820672  9059 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:56.807363  8922 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:56.856212  9073 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:56.841109  9066 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:56.857075  8511 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:56.803719  8864 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 12:17:56.849267  9048 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:56.840823  8861 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:56.845757  8822 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:56.825296  8855 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:56.855659  8899 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:17:56.870486  9033 raft_consensus.cc:697] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 3 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:56.860235  6550 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:56.878485  6582 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:56112
W20260709 12:17:56.879164  9056 log.cc:927] Time spent T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.292s	user 0.002s	sys 0.000s
I20260709 12:17:56.893195  6582 raft_consensus.cc:606] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Received request to transfer leadership
W20260709 12:17:56.880128  9079 log.cc:927] Time spent T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 12:17:56.849836  9081 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:56.868888  6577 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6. Ops: []
I20260709 12:17:56.880568  8861 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:56.868607  9022 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:17:56.890776  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:56.869850  9063 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:56.893653  6582 raft_consensus.cc:614] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:56.871021  6475 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:56.898314  6550 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:56.896238  8922 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:56.899338  9019 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:56.899631  9034 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:17:56.913369  6475 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:56.857105  9047 consensus_queue.cc:237] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:56.943605  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 153331us
W20260709 12:17:56.972311  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 59069.5us
I20260709 12:17:56.973039  6475 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:56.900277  9081 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:56.881729  9066 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:56.977313  9046 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:56.918359  8899 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:56.979182  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:56.881489  8511 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:56.981001  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:56.985252  6508 raft_consensus.cc:1240] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:56.986310  6500 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:56.874612  9059 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:56.998862  9051 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:17:56.999306  9086 log.cc:927] Time spent T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 12:17:56.918951  9034 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:57.003155  6506 raft_consensus.cc:2393] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 5: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:56.966804  9023 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:17:57.028481  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 130276us
I20260709 12:17:57.029906  6550 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:56.980239  9073 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:17:57.006165  6501 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
new_leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:58944
I20260709 12:17:57.007119  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:56.871191  9074 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:57.007145  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:57.014554  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:57.033634  6508 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:57.032388  6501 raft_consensus.cc:606] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Received request to transfer leadership to TS 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:57.043298  6508 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:17:56.990593  9046 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:57.033527  6505 raft_consensus.cc:2376] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6. Re-sending same reply.
I20260709 12:17:57.000286  8856 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:56.915151  9022 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:56.917845  9063 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:57.027118  9034 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:57.057386  7238 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:57.069368  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:57.070361  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:57.072511  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:57.070636  6506 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:17:57.089085  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:57.089903  6502 raft_consensus.cc:2376] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7. Re-sending same reply.
I20260709 12:17:57.092620  6582 raft_consensus.cc:1275] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:17:57.093945  8864 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:57.094600  8864 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:57.049685  9020 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:17:57.102182  9076 log.cc:927] Time spent T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.276s	user 0.001s	sys 0.000s
I20260709 12:17:57.103207  9078 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:57.104715  9092 log.cc:927] Time spent T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:17:57.104811  9087 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:57.081740  8598 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:56.966771  9033 consensus_queue.cc:237] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:57.046931  9051 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:57.105073  8855 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:57.067056  9023 consensus_queue.cc:1048] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
W20260709 12:17:57.130914  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 81985.3us
I20260709 12:17:57.108110  8598 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:57.065449  9046 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:57.081539  8856 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:57.137931  8598 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:57.188122  6501 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
new_leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:58944
I20260709 12:17:57.188956  6501 raft_consensus.cc:606] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Received request to transfer leadership to TS 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:57.189538  6501 raft_consensus.cc:614] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:17:57.199671  9038 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:57.103799  9078 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:57.181389  8864 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:57.126529  9087 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:57.203099  9101 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:57.195013  9061 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:57.156186  9090 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:17:57.226137  9080 log.cc:927] Time spent T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.357s	user 0.002s	sys 0.000s
I20260709 12:17:57.226763  8922 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:57.227720  8822 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:57.229851  8840 consensus_meta.cc:287] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.614s	user 0.013s	sys 0.000s
I20260709 12:17:57.227418  9052 consensus_queue.cc:260] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:57.251538  8840 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:57.252404  9052 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:57.253932  8840 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:57.222812  8856 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:57.258363  9022 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:57.260607  6501 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
new_leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:58944
I20260709 12:17:57.261076  9046 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:57.261448  6508 raft_consensus.cc:1240] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:57.261672  8511 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:57.258404  9087 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:57.260563  9081 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:57.263293  9078 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:57.263418  6505 raft_consensus.cc:1240] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:57.263948  9059 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:57.061735  8975 consensus_queue.cc:1048] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.111s
W20260709 12:17:57.220808  9089 log.cc:927] Time spent T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
W20260709 12:17:57.220877  9082 log.cc:927] Time spent T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
I20260709 12:17:57.249807  6500 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 6.
I20260709 12:17:57.221570  7271 raft_consensus.cc:1240] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5. Ops: []
W20260709 12:17:57.249807  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 102618us
I20260709 12:17:57.264427  9081 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:57.222976  9063 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:57.238040  9101 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:57.298944  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 319820us
I20260709 12:17:57.304221  6622 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:57.332404  6657 raft_consensus.cc:1240] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 4. Current term is 5. Ops: []
W20260709 12:17:57.332481  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 341979us
I20260709 12:17:57.352792  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:57.353662  6655 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 4.
W20260709 12:17:57.354056  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:57.357152  8839 consensus_meta.cc:287] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.608s	user 0.014s	sys 0.008s
I20260709 12:17:57.374079  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:17:57.378759  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:57.379297  7272 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:57.374763  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:57.305303  9101 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:57.369942  9040 log.cc:927] Time spent T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 12:17:57.368083  6392 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 3a47d71c34b541b9b2589b4e80413fa2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 496c7c3792cd4587b4dbeba435f4be1e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:57.381423  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:57.297959  9061 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:57.388221  9045 consensus_meta.cc:287] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.666s	user 0.009s	sys 0.003s
I20260709 12:17:57.390241  9072 raft_consensus.cc:3055] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:17:57.390348  9067 log.cc:927] Time spent T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.786s	user 0.001s	sys 0.003s
W20260709 12:17:57.393396  9071 log.cc:927] Time spent T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.772s	user 0.000s	sys 0.003s
I20260709 12:17:57.393695  9078 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:57.400693  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 96593.8us
I20260709 12:17:57.403651  6655 raft_consensus.cc:1275] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:57.368512  9087 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:57.331745  9066 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:57.373884  9103 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:57.421554  6652 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 4.
I20260709 12:17:57.387655  8839 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:57.474411  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:17:57.482999  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 248940us
W20260709 12:17:57.413187  6392 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 48103.7us
I20260709 12:17:57.494324  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:57.468356  9072 raft_consensus.cc:740] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:17:57.493008  9081 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:57.493366  9046 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:57.519510  7238 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:57.521641  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:17:57.522035  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 166578us
I20260709 12:17:57.521834  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:57.513643  9033 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:57.499718  9078 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:57.544513  9110 consensus_queue.cc:1059] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.134s
I20260709 12:17:57.520483  9081 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:57.545886  6621 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.012s	sys 0.033s
W20260709 12:17:57.546221  6621 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.012s	sys 0.033s
I20260709 12:17:57.547642  9115 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:57.548645  9116 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.031s
W20260709 12:17:57.553761  9119 log.cc:927] Time spent T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:17:57.558830  9122 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:57.559257  9072 consensus_queue.cc:260] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:57.559484  9122 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:57.560482  9072 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:57.561293  6622 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:57.545570  9110 raft_consensus.cc:3055] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:57.563475  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:57.566056  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:57.577979  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:57.506382  9103 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:57.560040  9121 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:57.580477  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:57.582165  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:57.583361  6650 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:17:57.583555  6655 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:57.561146  9054 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:57.583899  6650 pending_rounds.cc:85] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 2
I20260709 12:17:57.536818  9046 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:57.565613  9110 raft_consensus.cc:740] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:57.589830  6650 pending_rounds.cc:107] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:17:57.592087  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:17:57.589831  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 66153.5us
I20260709 12:17:57.612210  6650 raft_consensus.cc:2889] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:57.614053  7240 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:57.615170  7240 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:57.618301  6474 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:57.586784  9063 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:57.620055  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:17:57.621296  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 99562.2us
I20260709 12:17:57.616166  8839 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:57.587919  9109 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:57.623883  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:57.604049  9120 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:17:57.627790  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:57.628057  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:57.629290  6500 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 4.
I20260709 12:17:57.630553  9124 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:57.630970  9124 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:57.631323  9124 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:57.631726  9118 log.cc:927] Time spent T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
W20260709 12:17:57.632527  6455 auto_rebalancer.cc:893] Removing replace marker failed: Replica 496c7c3792cd4587b4dbeba435f4be1e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:17:57.633152  6455 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 12:17:57.633364  6455 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
W20260709 12:17:57.633559  9106 log.cc:927] Time spent T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:17:57.636119  9122 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:57.626121  9066 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:57.640754  9066 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:57.629319  7272 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:17:57.641306  9051 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:57.642251  9110 consensus_queue.cc:260] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:57.643143  9046 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:57.641155  6654 raft_consensus.cc:1240] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:17:57.643577  9110 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:57.644083  6657 raft_consensus.cc:1240] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:57.644680  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:57.641124  9066 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:57.596618  9054 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:57.645418  6474 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:57.645514  9120 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:57.645994  9094 log.cc:927] Time spent T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.610s	user 0.002s	sys 0.000s
W20260709 12:17:57.646337  9034 consensus_meta.cc:287] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.553s	user 0.010s	sys 0.002s
I20260709 12:17:57.646708  8839 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:57.648499  9063 raft_consensus.cc:697] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:57.648979  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:57.649633  9124 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:57.652472  8944 consensus_queue.cc:1059] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:57.653738  8511 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:17:57.654531  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:57.686435  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:57.654493  8861 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:57.655611  9074 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:57.688271  8944 raft_consensus.cc:3055] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:57.671012  8598 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:57.688825  8944 raft_consensus.cc:740] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:57.689005  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:57.689638  6507 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:57.689916  8944 consensus_queue.cc:260] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:57.658773  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:57.691725  7267 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:17:57.659194  7272 raft_consensus.cc:1240] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:57.692677  7266 raft_consensus.cc:2368] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7.
I20260709 12:17:57.660297  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:17:57.693747  9127 log.cc:927] Time spent T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:17:57.694116  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:57.680106  8839 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:57.695873  8839 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:57.703061  9019 consensus_meta.cc:287] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.737s	user 0.011s	sys 0.000s
I20260709 12:17:57.703625  9019 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:57.705662  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:57.706125  9124 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:57.710986  6475 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:57.711916  9078 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:17:57.712520  9070 log.cc:927] Time spent T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 1.067s	user 0.004s	sys 0.000s
I20260709 12:17:57.713073  9074 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:57.672214  9120 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:57.713207  9090 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:57.714192  9090 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:57.711036  9063 consensus_queue.cc:237] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:57.715987  9054 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:57.676760  9046 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:57.717581  9046 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:57.718541  9046 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:57.718657  9103 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:57.691313  8944 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:57.724282  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:57.682991  9034 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:57.725569  6653 raft_consensus.cc:1240] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 8. Current term is 9. Ops: []
I20260709 12:17:57.726065  9033 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:57.727231  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:57.727087  9033 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:57.729418  9052 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:57.730839  8840 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:57.616622  9081 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:57.678664  9023 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:57.735231  6549 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
W20260709 12:17:57.735421  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 85967.9us
I20260709 12:17:57.735268  9023 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:57.738106  9115 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:57.618765  9061 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:57.589365  9121 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:57.667974  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:57.678324  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:57.745005  9121 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:57.739351  9023 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:57.685633  6652 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:17:57.746588  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:57.748183  9101 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:57.750379  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 170006us
I20260709 12:17:57.751183  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:57.752813  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:57.753953  9103 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:17:57.754918  9099 log.cc:927] Time spent T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.611s	user 0.004s	sys 0.000s
I20260709 12:17:57.660928  6509 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:17:57.657351  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:57.756832  7271 raft_consensus.cc:2368] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7.
W20260709 12:17:57.757761  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:57.751348  9101 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:57.754810  9087 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:57.687803  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:57.757532  9103 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:57.714244  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:57.681629  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:57.726955  9034 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:57.755654  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:57.791939  6657 raft_consensus.cc:2393] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 4: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:57.748137  9033 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:57.691254  8598 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:57.781801  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 128348us
I20260709 12:17:57.784021  6507 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5.
I20260709 12:17:57.785034  6655 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:57.794014  9023 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:57.661173  6650 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:57.800477  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 44928.2us
W20260709 12:17:57.773315  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 45012.5us
I20260709 12:17:57.788292  9036 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:57.817795  6622 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:57.792034  9022 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:57.816148  9023 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:57.824779  6656 raft_consensus.cc:1240] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:57.808151  8840 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:57.799721  9033 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:57.838076  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:57.826357  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:57.807914  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:57.832897  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:57.839176  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:57.841528  6509 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:17:57.843271  6504 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:17:57.848321  6622 leader_election.cc:400] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5.
I20260709 12:17:57.848814  6622 leader_election.cc:403] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:17:57.871001  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:57.827315  9023 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:57.921456  6509 raft_consensus.cc:1275] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:57.845880  9034 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:57.842372  8840 raft_consensus.cc:697] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:57.826552  9036 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:57.934285  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:57.829833  9066 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:57.938802  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 90592.8us
I20260709 12:17:57.830739  9022 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:57.942785  6650 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:17:57.944217  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:57.945689  6650 raft_consensus.cc:2393] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 5: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:57.936689  9087 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:57.950506  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:57.950529  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:57.953289  6502 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 4.
I20260709 12:17:57.954916  6650 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:17:57.955770  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:57.956827  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:57.957751  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:57.920737  9090 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:57.942694  9033 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5.
I20260709 12:17:57.948477  9019 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:57.986060  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:57.987468  6650 raft_consensus.cc:3055] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:57.989703  6650 raft_consensus.cc:740] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:57.991986  6622 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:57.940721  8924 consensus_queue.cc:1059] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.029s
W20260709 12:17:58.003105  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 267949us
I20260709 12:17:58.007741  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:58.012816  6504 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:17:58.024304  6501 raft_consensus.cc:606] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 4 LEADER]: Received request to transfer leadership to TS 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:58.027827  6548 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:58.030294  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:58.011850  6650 consensus_queue.cc:260] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:58.021014  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:58.041318  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:58.045707  9033 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.046775  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:58.000667  9115 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:57.967042  9087 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:58.051285  6650 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:58.101920  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:58.103009  6504 raft_consensus.cc:2393] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 7: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:58.104758  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:58.105358  6504 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:57.996869  9019 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:58.115475  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:58.037031  9034 raft_consensus.cc:697] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:17:58.120707  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 116497us
I20260709 12:17:58.122577  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:58.090159  9115 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:58.127840  9115 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:58.131148  9022 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:58.132175  9022 consensus_queue.cc:1059] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:17:58.101384  8598 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:17:58.027552  8840 consensus_queue.cc:237] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:58.046770  9033 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:58.137506  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 58505.2us
W20260709 12:17:58.139873  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 112140us
W20260709 12:17:58.125121  9072 consensus_meta.cc:287] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.564s	user 0.011s	sys 0.000s
I20260709 12:17:58.140324  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:58.156692  6649 raft_consensus.cc:3055] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:58.154201  6654 raft_consensus.cc:1275] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:17:58.197511  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 57277us
I20260709 12:17:58.140498  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:58.200387  6549 leader_election.cc:400] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7.
I20260709 12:17:58.200773  6549 leader_election.cc:403] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:17:58.116703  9138 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.197647  6649 raft_consensus.cc:740] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:58.212162  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:58.307581  6507 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
W20260709 12:17:58.308904  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 110601us
I20260709 12:17:58.246418  6653 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:17:58.250862  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 124634us
I20260709 12:17:58.133144  8598 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:58.317955  7238 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:58.330410  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 283745us
I20260709 12:17:58.331799  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:58.331995  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:58.141173  9132 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:58.333400  6654 raft_consensus.cc:2393] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 4: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:58.334293  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:58.140793  8852 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.306092  9066 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:17:58.245747  8944 consensus_meta.cc:287] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.521s	user 0.012s	sys 0.000s
I20260709 12:17:58.210029  9103 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:58.341442  6507 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:58.142565  9087 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:58.214016  9112 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:17:58.346097  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 98813.3us
I20260709 12:17:58.146505  9033 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:58.347565  8840 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:58.336239  6649 consensus_queue.cc:260] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:58.327047  8598 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:58.143734  8839 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:58.157809  9124 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.183s
I20260709 12:17:58.363330  6649 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:58.330667  9101 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:58.320195  9130 consensus_queue.cc:1048] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 12:17:58.350291  9090 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:58.356419  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:58.245163  9120 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:58.357288  8924 consensus_queue.cc:1243] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer f2b06aebe97249a4877d798b8118ece4 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 12:17:58.407550  9091 log.cc:927] Time spent T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 12:17:58.358449  8852 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:58.363790  8840 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:58.328670  9138 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:58.420001  6654 raft_consensus.cc:3055] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:58.489804  6654 raft_consensus.cc:740] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:17:58.489940  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:58.423491  6649 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:17:58.365191  9033 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:58.336265  9144 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:58.433743  6504 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
I20260709 12:17:58.423688  9115 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:58.498368  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:58.499141  6504 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:58.441762  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 1702) took 1395 ms. Trace:
I20260709 12:17:58.436173  8511 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:17:58.461354  6473 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:58.464918  6505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 1703) took 1413 ms. Trace:
I20260709 12:17:58.479916  6507 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:17:58.460467  9072 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.445456  9042 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.485401  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:58.447211  9101 raft_consensus.cc:697] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:58.493305  7270 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 9: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:17:58.500383  6508 rpcz_store.cc:276] 0709 12:17:57.046818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.059883 (+ 13065us) service_pool.cc:224] Handling call
0709 12:17:57.060257 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:57.261841 (+201584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.441350 (+1179509us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b540039fb54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:58.441659 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":462290688}
I20260709 12:17:58.354138  9034 consensus_queue.cc:237] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:58.490751  6654 consensus_queue.cc:260] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:58.506119  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:58.565701  6507 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:17:58.491593  9110 consensus_meta.cc:287] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.847s	user 0.015s	sys 0.000s
I20260709 12:17:58.509145  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:58.491513  6649 pending_rounds.cc:85] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 3
I20260709 12:17:58.584976  6649 pending_rounds.cc:107] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:17:58.585335  6649 raft_consensus.cc:2889] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:58.504701  6505 rpcz_store.cc:276] 0709 12:17:57.051152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.059263 (+  8111us) service_pool.cc:224] Handling call
0709 12:17:57.084703 (+ 25440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:57.263969 (+179266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.464588 (+1200619us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b54000f3f54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:58.464838 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":410454784}
I20260709 12:17:58.501219  9019 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:58.555591  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:58.593508  6509 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
I20260709 12:17:58.457960  8924 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:58.460625  9047 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.557994  9063 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.564905  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:58.603132  9156 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:58.603757  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 271885us
I20260709 12:17:58.605083  6622 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:58.607957  6622 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:17:58.609120  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 257305us
I20260709 12:17:58.610423  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:58.611073  6509 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:17:58.598713  9023 consensus_meta.cc:287] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.669s	user 0.009s	sys 0.003s
I20260709 12:17:58.593860  6507 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5.
I20260709 12:17:58.589895  8856 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.612522  6622 leader_election.cc:400] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5.
I20260709 12:17:58.594913  6502 raft_consensus.cc:2393] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 5: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:58.591177  6654 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:58.625420  6504 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7.
I20260709 12:17:58.642772  6622 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:58.564232  9072 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:58.560315  9042 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:58.653483  6509 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7.
W20260709 12:17:58.654645  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 44202.4us
I20260709 12:17:58.655877  6509 raft_consensus.cc:1275] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:58.646013  7238 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:17:58.609084  9110 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:17:58.613339  9033 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:58.670099  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 240) took 1956 ms. Trace:
I20260709 12:17:58.670801  6505 raft_consensus.cc:1240] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:17:58.640137  8856 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:58.670619  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 209308us
I20260709 12:17:58.672500  6508 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
I20260709 12:17:58.671712  6509 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 6 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:17:58.675762  6473 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:58.612300  9063 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:58.677706  6505 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:17:58.673694  6578 rpcz_store.cc:276] 0709 12:17:56.713347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.731432 (+ 18085us) service_pool.cc:224] Handling call
0709 12:17:56.731963 (+   531us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:17:56.755231 (+ 23268us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:56.755260 (+    29us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:17:56.821712 (+ 66452us) log.cc:844] Serialized 592 byte log entry
0709 12:17:56.876122 (+ 54410us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:56.889868 (+ 13746us) log.cc:844] Serialized 14 byte log entry
0709 12:17:56.890327 (+   459us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:56.890424 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:57.198488 (+308064us) spinlock_profiling.cc:243] Waited 85.2 ms on lock 0x7b48000030d8. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287adb2aa1 00007f287ada9b37 00007f287ada99ca 00007f287aea126f 00007f287aea0f37 00007f287aeb555c 00007f287aeb5508 00007f287aeb5488 00007f287aeb5443 00007f287aeb46ec 00007f2879ddea96
0709 12:17:58.157064 (+958576us) spinlock_profiling.cc:243] Waited 1.99 ms on lock 0x7b18002e2a60. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:17:58.669796 (+512732us) raft_consensus.cc:1685] finished
0709 12:17:58.669819 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:58.669993 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39986,"prepare.queue_time_us":13549,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":38,"raft.queue_time_us":12063,"raft.run_cpu_time_us":3625,"raft.run_wall_time_us":444159,"spinlock_wait_cycles":200550400,"thread_start_us":15443,"threads_started":1,"wal-append.queue_time_us":50925}
I20260709 12:17:58.684942  6505 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:17:58.692045  9138 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:58.698894  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:58944 (request call id 3) took 1438 ms. Trace:
I20260709 12:17:58.699355  6501 rpcz_store.cc:276] 0709 12:17:57.260256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.260448 (+   192us) service_pool.cc:224] Handling call
0709 12:17:58.698700 (+1438252us) spinlock_profiling.cc:243] Waited 763 ms on lock 0x7b540001bdd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e83853 00007f287ae5160a 00007f28743d77d4 00007f28743d7762 00007f28743d7691 00007f28743d760c 00007f28743d68c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:58.698803 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1754702080}
I20260709 12:17:58.700132  8840 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:17:58.700366  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:58.670161  9129 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:58.620548  9148 consensus_queue.cc:1059] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
W20260709 12:17:58.708832  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 43586.9us
I20260709 12:17:58.712244  6654 raft_consensus.cc:1275] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:17:58.615141  8924 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:58.627099  9023 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:58.612702  9047 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:58.612905  9156 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:58.667479  9150 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:17:58.726101  6654 pending_rounds.cc:85] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 2
I20260709 12:17:58.731339  6654 pending_rounds.cc:107] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:17:58.737649  6654 raft_consensus.cc:2889] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:17:58.717346  9046 consensus_meta.cc:287] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.998s	user 0.013s	sys 0.002s
I20260709 12:17:58.670114  9144 consensus_queue.cc:1059] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 12:17:58.744565  9139 consensus_queue.cc:1048] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:17:58.745422  9136 log.cc:927] Time spent T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.635s	user 0.001s	sys 0.002s
I20260709 12:17:58.746759  6500 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 250) took 1116 ms. Trace:
I20260709 12:17:58.746951  6506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 248) took 1117 ms. Trace:
I20260709 12:17:58.747726  6506 rpcz_store.cc:276] 0709 12:17:57.629378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.629926 (+   548us) service_pool.cc:224] Handling call
0709 12:17:57.630198 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:57.716537 (+ 86339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:57.716568 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:17:57.716610 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:17:57.716658 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.746725 (+1030067us) spinlock_profiling.cc:243] Waited 86.1 ms on lock 0x7b5400396cd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:58.746774 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:58.746894 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":198098304}
I20260709 12:17:58.747169  6500 rpcz_store.cc:276] 0709 12:17:57.630590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.674241 (+ 43651us) service_pool.cc:224] Handling call
0709 12:17:57.674699 (+   458us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:57.675463 (+   764us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:17:57.675509 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:57.676155 (+   646us) log.cc:844] Serialized 26 byte log entry
0709 12:17:57.931380 (+255225us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:57.931437 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:57.931530 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:58.744821 (+813291us) spinlock_profiling.cc:243] Waited 2.53 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:17:58.746424 (+  1603us) raft_consensus.cc:1685] finished
0709 12:17:58.746446 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:58.746694 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":327,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":7264,"spinlock_wait_cycles":5828480,"thread_start_us":202451,"threads_started":1,"wal-append.queue_time_us":274005}
W20260709 12:17:58.752590  9153 log.cc:927] Time spent T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 12:17:58.752908  9154 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:58.753597  8856 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:58.753850  9154 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:58.754196  9046 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:17:58.754887  9141 log.cc:927] Time spent T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.192s	user 0.000s	sys 0.000s
I20260709 12:17:58.652556  9101 consensus_queue.cc:237] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:58.752252  9042 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:58.757123  9042 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:58.757715  9042 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:58.756469  9046 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:58.758925  6503 raft_consensus.cc:2368] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7.
W20260709 12:17:58.759184  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 83571.6us
I20260709 12:17:58.766477  6509 raft_consensus.cc:1275] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:58.682914  9155 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.083s
I20260709 12:17:58.739986  9129 raft_consensus.cc:697] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:58.741938  9161 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:58.779191  9148 raft_consensus.cc:3055] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:58.779976  9148 raft_consensus.cc:740] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:58.781096  9148 consensus_queue.cc:260] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:58.758984  9115 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:58.743803  9163 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:17:58.746946  9139 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:17:58.794554  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 69364.8us
I20260709 12:17:58.762388  9160 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:17:58.799357  6392 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 3a47d71c34b541b9b2589b4e80413fa2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 496c7c3792cd4587b4dbeba435f4be1e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:58.782616  9148 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:58.795707  9161 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:58.805977  9036 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:58.807633  9145 log.cc:927] Time spent T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.418s	user 0.000s	sys 0.005s
I20260709 12:17:58.788972  9046 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:58.808681  9046 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:58.786013  9168 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:58.818388  8511 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7.
I20260709 12:17:58.822597  9160 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:58.783670  9066 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:58.811470  9168 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:58.856467  8852 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:58.819589  9139 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:58.860575  9063 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:58.862347  9072 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:58.865566  8944 consensus_queue.cc:1059] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:58.865397  9129 consensus_queue.cc:237] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:58.866360  8975 consensus_queue.cc:1059] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 6.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:58.866962  9140 raft_consensus.cc:3055] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:58.867401  9140 raft_consensus.cc:740] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:58.867491  8975 raft_consensus.cc:3055] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:58.867945  8975 raft_consensus.cc:740] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:58.868371  9140 consensus_queue.cc:260] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:58.868783  7266 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38546 (request call id 299) took 1570 ms (client timeout 1717 ms). Trace:
I20260709 12:17:58.868832  8975 consensus_queue.cc:260] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.2, Last appended by leader: 2, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:58.869417  7266 rpcz_store.cc:269] 0709 12:17:57.298068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.302579 (+  4511us) service_pool.cc:224] Handling call
0709 12:17:58.868578 (+1565999us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b54004e7ad4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:58.868722 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":641075072}
I20260709 12:17:58.771852  9137 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:17:58.870944  9137 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:58.876420  9023 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:58.783433  9120 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:58.877641  9150 raft_consensus.cc:3055] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:17:58.878109  9150 raft_consensus.cc:740] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:17:58.878160  8598 consensus_meta.cc:287] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.503s	user 0.008s	sys 0.002s
I20260709 12:17:58.879294  9150 consensus_queue.cc:260] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:58.993393  9120 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:17:58.995826  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:58.996096  6475 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:58.993779  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:58.998011  7266 raft_consensus.cc:2368] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 4. Current term is 5.
I20260709 12:17:58.910283  7270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38558 (request call id 402) took 1245 ms. Trace:
I20260709 12:17:58.998481  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:58.989081  9140 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:58.994630  9047 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:58.998849  7270 rpcz_store.cc:276] 0709 12:17:57.665336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.665579 (+   243us) service_pool.cc:224] Handling call
0709 12:17:58.909067 (+1243488us) spinlock_profiling.cc:243] Waited 748 ms on lock 0x7b54006c2354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:58.910191 (+  1124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1721449856}
W20260709 12:17:58.993351  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:59.002810  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.003224  9115 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:59.003868  9115 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:59.004509  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:59.005494  6579 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
W20260709 12:17:59.001402  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.010463  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:59.011250  6655 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:17:58.995035  9154 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:58.879953  8598 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:58.988847  8975 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:59.001564  9168 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:58.994457  9046 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:17:58.990989  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 120021us
I20260709 12:17:59.024260  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:59.027469  6656 raft_consensus.cc:1275] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:17:59.033459  6658 raft_consensus.cc:1275] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:59.035537  6579 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:17:59.035737  6548 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:59.036063  6579 pending_rounds.cc:85] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 6
I20260709 12:17:59.036429  6579 pending_rounds.cc:107] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:17:59.036936  6548 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:59.037559  8944 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:59.037822  9162 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:17:59.041090  6654 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7. Ops: []
I20260709 12:17:59.040134  6655 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7.
I20260709 12:17:59.038184  6548 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:59.036959  6579 raft_consensus.cc:2889] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:59.045189  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:59.043641  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:59.046548  6658 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 2.
I20260709 12:17:58.989514  9150 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:59.053036  7271 raft_consensus.cc:2393] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 7: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:17:59.052452  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 1707) took 1224 ms. Trace:
I20260709 12:17:59.049820  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:59.045877  6655 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:59.053854  6503 rpcz_store.cc:276] 0709 12:17:57.819026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:57.819215 (+   189us) service_pool.cc:224] Handling call
0709 12:17:59.041083 (+1221868us) spinlock_profiling.cc:243] Waited 915 ms on lock 0x7b54002c4cd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:59.043562 (+  2479us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2104931072}
W20260709 12:17:59.056058  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 60025.9us
I20260709 12:17:59.059331  6474 leader_election.cc:400] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 4. Current term is 5.
I20260709 12:17:59.059932  6474 leader_election.cc:403] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:17:59.055532  6548 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:59.059968  6475 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:59.000402  9042 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:59.010800  9161 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:59.060842  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:59.046809  9112 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.000s	sys 0.034s
I20260709 12:17:59.041428  9169 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.072890  7270 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:17:59.076231  6548 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:59.078440  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:59.047348  8839 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:59.084390  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:17:59.085145  6651 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:59.085959  7240 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:17:59.086148  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.089005  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:59.089250  8892 consensus_queue.cc:1048] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:59.082459  9170 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:59.091120  9170 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.091408  8892 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:59.091549  9164 raft_consensus.cc:993] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65: : Instructing follower 496c7c3792cd4587b4dbeba435f4be1e to start an election
I20260709 12:17:59.092635  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:59.093165  9164 raft_consensus.cc:1081] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Signalling peer 496c7c3792cd4587b4dbeba435f4be1e to start an election
I20260709 12:17:59.065882  9047 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:59.093796  9047 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:17:59.096026  9047 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:59.063309  9162 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:17:59.074043  9112 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.000s	sys 0.035s
I20260709 12:17:59.073119  9066 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:59.108038  6584 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7.
I20260709 12:17:59.111411  6655 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 9.
I20260709 12:17:59.065507  9163 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.064366  9168 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:17:59.116088  9154 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.116278  9160 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:59.116907  9160 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:59.116770  9154 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:59.117316  9160 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:59.117702  9160 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:59.119089  9154 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:59.119921  9162 raft_consensus.cc:697] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:59.120570  9137 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:59.120939  9162 consensus_queue.cc:237] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:59.092054  8892 raft_consensus.cc:697] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:59.125128  8892 consensus_queue.cc:237] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:59.076408  9023 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:17:59.128595  9023 raft_consensus.cc:697] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:59.129653  9023 consensus_queue.cc:237] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:59.065321  9072 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:17:59.133735  9166 log.cc:927] Time spent T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.437s	user 0.000s	sys 0.003s
I20260709 12:17:59.133560  9072 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:59.135740  9072 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:59.135994  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1366) took 1165 ms. Trace:
I20260709 12:17:59.089027  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:59.071219  9046 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.103963  7274 raft_consensus.cc:1275] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:59.105356  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:59.079298  6657 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:59.091660  9170 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:59.115065  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:59.075373  9169 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:59.145013  9046 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:59.136379  6652 rpcz_store.cc:276] 0709 12:17:57.970557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.329203 (+358646us) service_pool.cc:224] Handling call
0709 12:17:58.329647 (+   444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:58.594571 (+264924us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:58.594618 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:58.595228 (+   610us) log.cc:844] Serialized 26 byte log entry
0709 12:17:58.658386 (+ 63158us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:17:58.674633 (+ 16247us) log.cc:844] Serialized 14 byte log entry
0709 12:17:58.680236 (+  5603us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:58.680381 (+   145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:17:59.133177 (+452796us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:17:59.135714 (+  2537us) raft_consensus.cc:1685] finished
0709 12:17:59.135811 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:17:59.135937 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5083,"prepare.queue_time_us":104644,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":64,"raft.queue_time_us":32675,"raft.run_cpu_time_us":327,"raft.run_wall_time_us":29766,"spinlock_wait_cycles":45411072,"thread_start_us":314576,"threads_started":3,"wal-append.queue_time_us":87395}
I20260709 12:17:59.152745  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:59.153280  6581 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:17:59.149423  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:59.172729  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.173132  6578 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:59.174853  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:59.075995  8852 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 12:17:59.108716  9112 raft_consensus.cc:697] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:59.099546  9042 consensus_queue.cc:1048] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 12:17:59.179986  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:59.183059  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:59.180167  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:59.178947  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:59.096665  9156 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:59.187497  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:59.147037  9115 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:59.184027  6583 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
W20260709 12:17:59.196506  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.205633  6580 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:17:59.206948  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:59.209767  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:59.211112  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:59.212119  6581 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:59.212953  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:17:59.182852  9072 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.217624  9072 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:59.217729  9101 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:59.219808  9072 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:59.229351  8598 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:59.230428  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:59.232080  7268 raft_consensus.cc:2410] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 5 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:17:59.189082  9044 consensus_queue.cc:1048] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 12:17:59.233645  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:59.234861  9112 consensus_queue.cc:237] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:59.239234  9169 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:59.243019  8924 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:59.105084  9120 raft_consensus.cc:2955] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 8 LEADER]: Committing config change with OpId 5.4: config changed from index 2 to 4, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:59.247332  9093 raft_consensus.cc:3055] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:17:59.247962  9093 raft_consensus.cc:740] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:59.249061  9093 consensus_queue.cc:260] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:59.250485  9093 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:59.254565  6582 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:17:59.256469  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.260612  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 1686) took 1237 ms. Trace:
I20260709 12:17:59.261148  6509 rpcz_store.cc:276] 0709 12:17:58.022757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.676876 (+654119us) service_pool.cc:224] Handling call
0709 12:17:58.681310 (+  4434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:17:58.767113 (+ 85803us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:59.260253 (+493140us) spinlock_profiling.cc:243] Waited 81.2 ms on lock 0x7b54003976d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:17:59.260516 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":186801536}
I20260709 12:17:59.220496  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:59.263523  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 1689) took 1240 ms. Trace:
I20260709 12:17:59.220938  9161 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:59.264001  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:59.264194  9161 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:59.266325  9161 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:59.266037  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:59.210417  6654 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
I20260709 12:17:59.226305  9170 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:59.217629  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:59.172317  9163 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:59.269496  9163 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:59.269548  9046 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:59.270212  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:59.271205  6583 raft_consensus.cc:2410] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:17:59.271457  9146 raft_consensus.cc:3055] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:59.271884  9146 raft_consensus.cc:740] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:59.204005  9156 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.272790  9146 consensus_queue.cc:260] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:59.273211  9156 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:59.273837  9146 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:59.275833  8944 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:17:59.263872  6508 rpcz_store.cc:276] 0709 12:17:58.023604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:58.564620 (+541016us) service_pool.cc:224] Handling call
0709 12:17:59.263366 (+698746us) spinlock_profiling.cc:243] Waited 70.5 ms on lock 0x7b5400434c54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:17:59.263471 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":162111872}
I20260709 12:17:59.285542  9139 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:59.290630  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:59.290958  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:59.291471  6502 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:59.295836  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:59.290684  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:59.296612  6508 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:59.303215  9066 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 4. Current term is 5.
W20260709 12:17:59.306025  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:59.315150  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.323716  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:59.324683  6509 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:59.331013  6578 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:59.326604  9160 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:59.333935  6584 raft_consensus.cc:2393] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 8: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:17:59.333523  6406 catalog_manager.cc:5697] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 1 to 2, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 2 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:59.335377  9160 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:59.335675  6406 catalog_manager.cc:5697] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 1 to 4, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 4 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:59.340361  9129 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:59.341368  9129 raft_consensus.cc:697] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:59.342527  9129 consensus_queue.cc:237] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:59.354372  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:59.355288  6578 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
I20260709 12:17:59.357419  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:59.357470  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:17:59.358408  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 295497us
I20260709 12:17:59.357962  6406 catalog_manager.cc:5697] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 3 to 4, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 4 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:59.361050  6475 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:59.359565  6651 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5.
I20260709 12:17:59.364821  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:59.366650  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:59.367709  6579 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:59.371649  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:59.399073  6658 raft_consensus.cc:1217] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Deduplicated request from leader. Original: 4.2->[6.3-8.4]   Dedup: 6.3->[8.4-8.4]
I20260709 12:17:59.399557  6578 raft_consensus.cc:1240] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:59.399890  6658 raft_consensus.cc:1240] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 8. Current term is 9. Ops: [6.3-8.4]
I20260709 12:17:59.401177  6583 raft_consensus.cc:2393] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 5: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:59.407501  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:59.408383  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:59.408994  6576 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:17:59.412315  9150 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:59.409614  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:17:59.411999  6579 raft_consensus.cc:2955] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Committing config change with OpId 5.4: config changed from index 2 to 4, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:59.413421  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:59.418586  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:59.419165  6581 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:17:59.419188  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:59.422465  6577 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 4.
I20260709 12:17:59.424073  6577 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:17:59.416594  6582 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 2.
I20260709 12:17:59.425786  6584 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
W20260709 12:17:59.428715  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 73576.9us
I20260709 12:17:59.429222  6509 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
W20260709 12:17:59.429373  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.433449  6575 raft_consensus.cc:2410] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:17:59.370585  6502 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:17:59.420413  6622 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:59.399065  9066 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:17:59.452381  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:59.455127  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:59.450714  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:59.414964  9034 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:59.497059  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 81772.2us
I20260709 12:17:59.434571  6406 catalog_manager.cc:5697] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 1 to 6. New cstate: current_term: 6 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:59.502580  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:59.421489  9115 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:59.503418  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 87136us
W20260709 12:17:59.505973  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 47974.8us
I20260709 12:17:59.507468  6651 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:17:59.504513  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:59.513226  6622 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:17:59.515224  6578 raft_consensus.cc:1275] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:17:59.430856  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:59.431166  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 345254us
I20260709 12:17:59.431741  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:17:59.522063  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 91536.7us
I20260709 12:17:59.525168  9139 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:59.525655  6575 raft_consensus.cc:1275] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:17:59.526964  9139 consensus_queue.cc:1059] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:59.532377  6550 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:59.533310  6401 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57914 (request call id 189) took 10059 ms. Trace:
I20260709 12:17:59.510763  9059 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:17:59.540048  6548 leader_election.cc:400] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7.
I20260709 12:17:59.443733  9168 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.105s
I20260709 12:17:59.551000  9168 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.556727  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:59.559727  7240 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:59.560783  9170 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.561162  8892 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:59.561342  9170 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:59.564132  9170 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:59.564695  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:59.565353  9148 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:59.567297  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 43504.2us
I20260709 12:17:59.569226  9072 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:59.570281  7240 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:59.416476  9140 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:59.431686  9156 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:59.581840  6576 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 9.
I20260709 12:17:59.498658  9109 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:59.551677  9168 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:59.417829  9046 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.087s
I20260709 12:17:59.575601  6581 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7.
I20260709 12:17:59.510417  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:59.571594  7238 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:59.601851  6473 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.839s	user 0.055s	sys 0.142s
I20260709 12:17:59.444073  8944 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.150s
I20260709 12:17:59.427551  9150 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:59.604188  6473 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.841s	user 0.056s	sys 0.142s
I20260709 12:17:59.456473  9047 consensus_queue.cc:1048] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.140s
I20260709 12:17:59.533864  6401 rpcz_store.cc:276] 0709 12:17:49.474057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:49.477241 (+  3184us) service_pool.cc:224] Handling call
0709 12:17:49.712133 (+234892us) spinlock_profiling.cc:243] Waited 91.4 ms on lock 0x7b540008acd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df3090 00007f2875e91f3a 00007f287bc47e9c 00007f287bd4604e 00007f287a2072a4 00007f287a207232 00007f287a207161 00007f287a2070dc 00007f287a206392 00007f28741c448c 00007f28741c38c6
0709 12:17:59.533193 (+9821060us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:17:55.942348 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:17:55.943145 (+   797us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:17:55.943197 (+    52us) write_op.cc:435] Acquired schema lock
0709 12:17:55.943229 (+    32us) tablet.cc:662] Decoding operations
0709 12:17:55.955311 (+ 12082us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:17:55.955378 (+    67us) write_op.cc:641] Partition lock acquired for write op
0709 12:17:55.955404 (+    26us) tablet.cc:685] Acquiring locks for 8 operations
0709 12:17:55.956197 (+   793us) tablet.cc:701] Row locks acquired
0709 12:17:55.956267 (+    70us) write_op.cc:260] PREPARE: finished
0709 12:17:55.956490 (+   223us) write_op.cc:270] Start()
0709 12:17:55.956644 (+   154us) write_op.cc:276] Timestamp: P: 1783599475956412 usec, L: 0
0709 12:17:55.956666 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:17:55.957356 (+   690us) log.cc:844] Serialized 3182 byte log entry
0709 12:17:56.628889 (+671533us) op_driver.cc:464] REPLICATION: finished
0709 12:17:56.970222 (+341333us) spinlock_profiling.cc:243] Waited 5.48 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2872d7f0ac 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f28798bfef0
0709 12:17:57.200504 (+230282us) write_op.cc:301] APPLY: starting
0709 12:17:57.200643 (+   139us) tablet.cc:1367] starting BulkCheckPresence
0709 12:17:57.200974 (+   331us) tablet.cc:1370] finished BulkCheckPresence
0709 12:17:57.201024 (+    50us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:17:57.204551 (+  3527us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:17:57.233177 (+ 28626us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 12:17:57.233235 (+    58us) write_op.cc:312] APPLY: finished
0709 12:17:57.251045 (+ 17810us) log.cc:844] Serialized 65 byte log entry
0709 12:17:57.302693 (+ 51648us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:57.349500 (+ 46807us) write_op.cc:454] Released schema lock
0709 12:17:57.371308 (+ 21808us) write_op.cc:341] FINISH: Updating metrics
0709 12:17:57.476995 (+105687us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:17:57.477043 (+    48us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":191220,"reported_tablets":32,"spinlock_wait_cycles":210154880,"child_traces":[["op",{"apply.queue_time_us":520740,"apply.run_cpu_time_us":7832,"apply.run_wall_time_us":215906,"num_ops":8,"prepare.queue_time_us":331807,"prepare.run_cpu_time_us":24646,"prepare.run_wall_time_us":147717,"raft.queue_time_us":233014,"raft.run_cpu_time_us":30561,"raft.run_wall_time_us":481102,"replication_time_us":671958,"spinlock_wait_cycles":12600192,"thread_start_us":872719,"threads_started":4,"wal-append.queue_time_us":250587,"wal-append.run_cpu_time_us":22411,"wal-append.run_wall_time_us":2282927}]]}
I20260709 12:17:59.594031  9072 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:59.612577  6406 catalog_manager.cc:5697] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 1 to 2, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 2 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
W20260709 12:17:59.617240  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 45724.3us
I20260709 12:17:59.615613  6406 catalog_manager.cc:5697] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:59.597934  9109 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:59.607354  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:59.595881  9059 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:59.606189  9140 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:59.588270  9148 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:59.511179  9034 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:59.606577  6550 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:17:59.606472  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 843534us
I20260709 12:17:59.583385  9115 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:17:59.638307  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.658816  6406 catalog_manager.cc:5697] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 1 to 3, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 3 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:59.662865  7238 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:17:59.663954  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 57438.2us
W20260709 12:17:59.630401  9069 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.031s	sys 0.033s
I20260709 12:17:59.628466  9109 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:59.629040  9101 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:17:59.672235  6550 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:59.667595  6473 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:59.678326  9059 raft_consensus.cc:697] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:59.679149  9150 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:59.696717  9150 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.696825  9124 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:17:59.697183  9168 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:59.697556  9124 raft_consensus.cc:697] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 9 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:59.697414  9150 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:59.679991  9072 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:59.699853  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:17:59.681090  9115 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:59.667779  8511 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:17:59.658272  9023 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.001s	sys 0.021s
I20260709 12:17:59.691910  6473 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:59.709882  6473 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:59.711153  6473 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:17:59.681979  9069 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.032s	sys 0.033s
I20260709 12:17:59.690265  9101 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:59.695468  9059 consensus_queue.cc:237] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:59.683990  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:17:59.708849  9023 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.001s	sys 0.021s
I20260709 12:17:59.700549  9047 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:59.688222  9180 consensus_queue.cc:1059] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:17:59.716004  9168 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:59.695870  9140 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:59.707131  9130 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:17:59.709259  8511 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:59.698678  9124 consensus_queue.cc:237] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:59.688834  9164 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
W20260709 12:17:59.727622  6656 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.004s	sys 0.014s
I20260709 12:17:59.732637  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:59.733490  6500 raft_consensus.cc:2393] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 7: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:17:59.708621  9115 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:59.734615  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:59.734783  9115 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:17:59.735553  6649 raft_consensus.cc:2393] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 4: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:17:59.737421  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:59.716486  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:17:59.729995  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:59.730199  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:59.743654  6508 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:17:59.716004  7267 raft_consensus.cc:1240] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 4. Current term is 5. Ops: []
I20260709 12:17:59.690375  9066 consensus_queue.cc:1048] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 12:17:59.749210  6509 raft_consensus.cc:1275] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:17:59.749293  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:59.749923  6504 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 4.
I20260709 12:17:59.726382  9171 raft_consensus.cc:2804] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:59.752177  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:59.726946  6473 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:59.718662  9101 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:59.733283  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:59.759238  6655 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
W20260709 12:17:59.743803  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.743047  6652 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:59.744944  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:59.761649  6473 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:17:59.762028  6650 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:59.745452  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:59.732668  6656 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.005s	sys 0.014s
I20260709 12:17:59.746001  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:59.764361  6654 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:59.737566  9115 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:59.765070  9069 raft_consensus.cc:3055] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:17:59.765374  9115 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:17:59.765462  9148 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:59.747181  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:59.719988  9072 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:59.725330  9150 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:17:59.738121  6649 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:17:59.718084  9178 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:59.752157  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:59.773425  6657 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
W20260709 12:17:59.753890  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 391189us
I20260709 12:17:59.730331  8511 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:59.781055  6508 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 9.
I20260709 12:17:59.729015  8598 raft_consensus.cc:3055] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:17:59.746172  9130 raft_consensus.cc:697] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:17:59.748785  6506 raft_consensus.cc:2955] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Committing config change with OpId 5.4: config changed from index 2 to 4, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:17:59.765513  9069 raft_consensus.cc:740] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:17:59.746803  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:17:59.767853  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:59.816664  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:59.817138  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.768508  6505 raft_consensus.cc:1275] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:59.784722  6473 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:59.786293  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:59.770128  9042 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:17:59.799886  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:59.803707  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:59.765714  6501 raft_consensus.cc:1240] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 8. Current term is 9. Ops: []
I20260709 12:17:59.829936  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:59.830803  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:59.808961  9195 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:17:59.769389  9169 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 12:17:59.818194  6508 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:59.818959  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:59.820361  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:59.838758  6654 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:17:59.838563  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:59.838505  6504 raft_consensus.cc:1240] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:17:59.822942  6473 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:59.824568  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:59.834508  9130 consensus_queue.cc:237] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:59.798657  9168 raft_consensus.cc:2764] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:17:59.899268  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:59.899709  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:59.839816  6654 raft_consensus.cc:3055] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:59.901372  6654 raft_consensus.cc:740] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:17:59.902595  6654 consensus_queue.cc:260] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:17:59.903735  6654 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:59.795737  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:59.904805  6649 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
W20260709 12:17:59.905918  9198 log.cc:927] Time spent T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 12:17:59.917217  9185 log.cc:927] Time spent T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 12:17:59.921131  9187 log.cc:927] Time spent T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:17:59.922675  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:59.923625  6658 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:17:59.799547  8598 raft_consensus.cc:740] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
W20260709 12:17:59.841687  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 328518us
I20260709 12:17:59.925180  6622 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:59.926038  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:59.841884  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:59.814749  9093 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:59.896696  6657 raft_consensus.cc:2955] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Committing config change with OpId 5.4: config changed from index 2 to 4, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:17:59.895125  9120 consensus_meta.cc:287] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.629s	user 0.009s	sys 0.005s
I20260709 12:17:59.841238  6473 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:17:59.826527  9072 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:17:59.843164  6503 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5.
I20260709 12:17:59.826839  9042 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:17:59.896451  6650 raft_consensus.cc:1275] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 12:17:59.896988  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:59.927011  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:59.938932  6505 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:17:59.927994  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:17:59.894192  9169 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:17:59.873668  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:59.896584  9156 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.936184  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:59.933728  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:17:59.941385  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:17:59.933859  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 92704.8us
I20260709 12:17:59.939589  6622 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:59.941385  6650 pending_rounds.cc:85] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 6
I20260709 12:17:59.942744  6650 pending_rounds.cc:107] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 12:17:59.942345  6473 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:17:59.943153  6650 raft_consensus.cc:2889] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:59.945093  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:17:59.949924  6508 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:45870
I20260709 12:17:59.953583  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:59.955271  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:17:59.956650  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:59.957454  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:59.958418  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:59.956199  6576 raft_consensus.cc:2393] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 7: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:17:59.959744  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:59.963109  6576 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:56116
I20260709 12:17:59.963656  6576 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:59.964001  6576 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:59.967247  6503 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:59.968083  9193 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.968626  9193 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:59.968827  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:59.929913  9150 raft_consensus.cc:2764] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:17:59.970264  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:59.940688  9042 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:17:59.971120  9150 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:17:59.971241  9193 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:17:59.971418  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:59.971698  9150 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:17:59.971853  6650 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 8. Current term is 9. Ops: []
I20260709 12:17:59.972174  9150 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:17:59.972817  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:59.973469  6579 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:17:59.974309  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:59.975167  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:17:59.975859  9197 log.cc:927] Time spent T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:17:59.978513  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:59.980125  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:17:59.981236  9195 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:17:59.981848  9195 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:17:59.938570  9170 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:59.935099  9139 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.975891  6584 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:17:59.989193  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:17:59.972134  6578 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:17:59.990752  6500 raft_consensus.cc:2368] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7.
I20260709 12:17:59.932866  9191 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:00.017522  6504 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:00.018769  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:00.019588  6658 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
W20260709 12:17:59.996397  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 51558.7us
W20260709 12:17:59.987699  9194 log.cc:927] Time spent T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:17:59.938512  9069 consensus_queue.cc:260] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:00.022800  6576 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:00.027915  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:00.342739  6584 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:18:00.254263  6654 raft_consensus.cc:1275] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:18:00.344090  6654 pending_rounds.cc:85] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 5
I20260709 12:18:00.344620  6654 pending_rounds.cc:107] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:18:00.345012  6654 raft_consensus.cc:2889] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:17:59.987134  9192 log.cc:927] Time spent T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
W20260709 12:18:00.283648  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.264778  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:00.351616  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:00.134312  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.025516  6473 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:18:00.353519  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 328076us
I20260709 12:17:59.986483  9190 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:18:00.253568  9189 log.cc:927] Time spent T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.356s	user 0.001s	sys 0.000s
I20260709 12:17:59.989265  9066 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:00.265827  6504 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:18:00.342948  6581 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:00.260283  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:00.064945  6658 raft_consensus.cc:1240] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:00.209865  9169 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:59.948410  9034 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:17:59.985423  9170 raft_consensus.cc:697] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:00.027577  9129 consensus_queue.cc:1048] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.344s
I20260709 12:18:00.086177  6500 raft_consensus.cc:1240] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:17:59.984290  9195 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:00.081789  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:00.396804  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:00.397542  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:00.065224  9182 log.cc:927] Time spent T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.435s	user 0.004s	sys 0.000s
I20260709 12:18:00.357638  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:00.401942  6584 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:17:59.990598  9115 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:00.364421  6576 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:00.062207  9046 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:00.335726  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 390692us
I20260709 12:17:59.988561  9156 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:00.013806  9042 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:00.357254  9193 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:17:59.986403  9139 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:00.359716  9069 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:00.028901  9191 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:00.384589  9109 consensus_meta.cc:287] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.684s	user 0.012s	sys 0.000s
I20260709 12:18:00.403869  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:18:00.427031  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.404407  9059 consensus_queue.cc:1048] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 12:18:00.408725  9046 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:00.384433  9169 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:00.429759  9046 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:00.411280  6622 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:00.431892  9169 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:00.433009  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:00.416198  9156 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:00.431612  6622 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:00.418139  9042 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:00.417496  9195 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 12:18:00.420480  9139 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:00.423760  6653 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:00.424525  9170 consensus_queue.cc:237] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:00.490456  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:00.491130  6503 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:18:00.435096  9150 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:00.424767  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:00.424911  6579 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:18:00.436051  9034 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:00.426100  8511 consensus_meta.cc:287] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.604s	user 0.011s	sys 0.000s
I20260709 12:18:00.425380  9109 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:00.416534  6580 raft_consensus.cc:1275] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:18:00.423527  9191 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:00.441454  9066 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:00.452281  6505 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:18:00.466943  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:00.066222  8598 consensus_queue.cc:260] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:00.434211  9101 consensus_meta.cc:287] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.655s	user 0.009s	sys 0.002s
I20260709 12:18:00.506803  9191 consensus_queue.cc:1048] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:18:00.507469  9101 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:00.508755  9191 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:00.509850  9191 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:00.504204  6508 raft_consensus.cc:606] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Received request to transfer leadership
I20260709 12:18:00.506659  8598 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:00.510705  6508 raft_consensus.cc:614] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:00.354884  6473 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:00.466718  9115 raft_consensus.cc:697] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 7 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:18:00.523106  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 168352us
I20260709 12:18:00.518298  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:00.524072  6650 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:18:00.524540  6473 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:00.525149  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:00.521872  6657 consensus_meta.cc:287] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.589s	user 0.010s	sys 0.000s
I20260709 12:18:00.526417  6473 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:18:00.527127  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 54402.6us
I20260709 12:18:00.528473  6473 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:00.525725  6650 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:18:00.545122  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:00.567644  6473 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:00.562462  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:00.571425  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 43653.6us
I20260709 12:18:00.572520  6655 raft_consensus.cc:1240] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:00.582623  6581 raft_consensus.cc:1275] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:18:00.518975  9191 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:00.586211  6581 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:18:00.527470  9205 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:00.570525  9190 raft_consensus.cc:697] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:00.566607  9046 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:00.571491  9042 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:00.578801  9169 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:00.578598  9193 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:00.578004  9156 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:00.517460  9034 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:00.517130  8511 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:00.624950  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:00.628099  9224 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:00.641959  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:00.642602  6581 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:00.615226  9042 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:00.629312  9225 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:00.631655  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:00.594203  9191 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:00.641491  9224 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:00.654681  9224 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:00.647468  7266 raft_consensus.cc:1240] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:00.656417  6577 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 5: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:00.617928  9169 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:00.649914  9225 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:00.651080  9203 log.cc:927] Time spent T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
W20260709 12:18:00.651845  9209 log.cc:927] Time spent T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
W20260709 12:18:00.628978  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.630306  9193 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:00.653919  9034 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:00.632990  9046 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:00.660562  6581 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5.
I20260709 12:18:00.644134  9195 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:00.656770  9042 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:00.736899  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 105524us
W20260709 12:18:00.748515  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.606952  9205 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:00.735473  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 161499us
I20260709 12:18:00.756937  6657 pending_rounds.cc:85] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 2
I20260709 12:18:00.766124  6657 pending_rounds.cc:107] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:18:00.766498  6657 raft_consensus.cc:2889] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:00.726924  9169 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:00.753826  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 182510us
I20260709 12:18:00.774055  6473 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:18:00.774462  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.760536  6475 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:00.641196  9156 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:00.763082  6581 raft_consensus.cc:1275] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:18:00.768219  6392 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 3a47d71c34b541b9b2589b4e80413fa2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 496c7c3792cd4587b4dbeba435f4be1e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:18:00.728694  9214 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:00.776300  6548 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:00.776028  6473 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:18:00.786624  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 355051us
I20260709 12:18:00.785492  6550 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:00.781278  7238 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:00.799806  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:00.752449  9193 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:00.764881  7267 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:18:00.804908  6550 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:00.800663  6622 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:00.779621  9066 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:00.830080  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 331) took 1005 ms. Trace:
I20260709 12:18:00.833503  6652 rpcz_store.cc:276] 0709 12:17:59.824185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.828279 (+  4094us) service_pool.cc:224] Handling call
0709 12:17:59.828631 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:59.829084 (+   453us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:59.829124 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:59.839143 (+ 10019us) log.cc:844] Serialized 26 byte log entry
0709 12:18:00.024163 (+185020us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:00.024200 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:00.024271 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:00.829830 (+805559us) raft_consensus.cc:1685] finished
0709 12:18:00.829859 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:00.830016 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3688,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":116,"thread_start_us":105288,"threads_started":1,"wal-append.queue_time_us":568552}
I20260709 12:18:00.837172  6550 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:00.838191  6550 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:00.841498  6548 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:00.653923  9190 consensus_queue.cc:237] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:00.822141  6622 connection.cc:256] RPC call timeout handler was delayed by -1.338347510000176s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1643s for a response
I20260709 12:18:00.806053  9139 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:00.846498  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:00.847354  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 71422.1us
I20260709 12:18:00.806620  9120 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:00.761442  9046 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:00.845404  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 249) took 1439 ms. Trace:
W20260709 12:18:00.849888  6622 connection.cc:256] RPC call timeout handler was delayed by -1.3032475100001193s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16820000000000002s for a response
I20260709 12:18:00.853047  6548 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:00.725098  9115 consensus_queue.cc:237] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:00.853684  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:00.853662  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:00.854676  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:00.854619  6550 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:00.811317  9034 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:00.851737  6652 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:18:00.813916  9150 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:00.879089  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:00.880475  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:00.882560  6500 raft_consensus.cc:1240] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5. Ops: []
W20260709 12:18:00.860862  6622 connection.cc:256] RPC call timeout handler was delayed by -1.2717475100000684s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17170000000000002s for a response
I20260709 12:18:00.884797  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:00.885632  6501 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:18:00.887643  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:00.855327  9069 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:00.765147  9195 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:00.862476  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:00.903951  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:00.851998  9222 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:00.860009  6583 rpcz_store.cc:276] 0709 12:17:59.406374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.408419 (+  2045us) service_pool.cc:224] Handling call
0709 12:17:59.426002 (+ 17583us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:17:59.518804 (+ 92802us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:17:59.518840 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:17:59.519350 (+   510us) log.cc:844] Serialized 26 byte log entry
0709 12:17:59.725173 (+205823us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:17:59.725233 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:17:59.725421 (+   188us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:00.402079 (+676658us) raft_consensus.cc:1685] finished
0709 12:18:00.402112 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:00.834730 (+432618us) spinlock_profiling.cc:243] Waited 1.03 ms on lock 0x7b5400420250. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:00.845336 (+ 10606us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":92424,"raft.queue_time_us":98396,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":160,"spinlock_wait_cycles":2358528,"thread_start_us":205397,"threads_started":1,"wal-append.queue_time_us":291323}
I20260709 12:18:00.796396  9124 consensus_queue.cc:1048] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 12:18:00.909463  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:00.866110  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:00.867364  6651 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:18:00.806768  9172 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 12:18:00.851409  9152 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:00.876282  9215 log.cc:927] Time spent T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 12:18:00.884021  6622 connection.cc:256] RPC call timeout handler was delayed by -1.2627475100000538s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17270000000000002s for a response
I20260709 12:18:00.847813  9093 consensus_queue.cc:1059] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 12:18:00.855955  9066 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:00.856101  9120 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:00.812886  9140 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:00.892494  9156 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:00.893242  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:00.892220  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:00.848358  8944 raft_consensus.cc:2764] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:18:00.858167  9044 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:00.864452  6656 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:00.901887  9139 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:18:00.903085  9223 log.cc:927] Time spent T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 12:18:00.871822  9233 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:00.798491  9214 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:00.920915  9066 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:00.917416  6622 connection.cc:256] RPC call timeout handler was delayed by -1.0692475099999683s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1942s for a response
I20260709 12:18:00.917007  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:00.901533  9034 raft_consensus.cc:697] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:00.905347  9148 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:18:00.936592  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 83617.4us
I20260709 12:18:00.937532  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:00.932024  6502 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
I20260709 12:18:00.935048  9120 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:18:00.945118  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 90557.5us
I20260709 12:18:00.976953  6550 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:00.915697  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 155243us
I20260709 12:18:00.975270  6581 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:00.975438  6548 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:00.920586  9152 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:00.978960  9044 consensus_queue.cc:1059] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:00.979147  9139 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:00.979660  9139 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:00.979803  9044 raft_consensus.cc:3055] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:00.978305  6475 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:00.932003  9150 consensus_queue.cc:1059] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 12:18:00.980247  9044 raft_consensus.cc:740] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:00.998077  8944 consensus_queue.cc:1059] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 12:18:00.943840  9233 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:00.961918  8511 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:01.003120  9233 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:00.981400  9195 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:01.003300  9044 consensus_queue.cc:260] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:00.977267  9109 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:01.005275  9238 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:01.005460  6475 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:00.980105  9139 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:01.006924  9034 consensus_queue.cc:237] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:01.015218  6656 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:18:01.016315  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:01.017719  6656 raft_consensus.cc:1275] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:18:01.018682  6656 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:18:01.018944  6656 pending_rounds.cc:85] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 3
I20260709 12:18:01.019152  6656 pending_rounds.cc:107] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:18:01.020305  6656 raft_consensus.cc:2889] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:00.940678  9046 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:00.932220  9222 raft_consensus.cc:697] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:01.378130  6475 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:01.302310  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:01.249384  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:01.027268  6548 leader_election.cc:400] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7.
I20260709 12:18:01.429409  6548 leader_election.cc:403] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:18:01.430331  6503 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:01.061024  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 84149.8us
W20260709 12:18:01.376085  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 220159us
W20260709 12:18:01.023829  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:01.435564  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:01.444283  6656 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 12:18:01.449066  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 421971us
I20260709 12:18:01.449301  6651 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:18:00.951678  9237 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:01.453972  6622 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
W20260709 12:18:01.244263  9235 log.cc:927] Time spent T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.284s	user 0.002s	sys 0.002s
W20260709 12:18:00.986596  9221 log.cc:927] Time spent T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:18:01.085356  9227 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:01.490187  9227 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:01.195451  9109 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.289s
I20260709 12:18:01.493500  6658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 343) took 1037 ms. Trace:
I20260709 12:18:01.494328  9101 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:01.494180  6658 rpcz_store.cc:276] 0709 12:18:00.456104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.582999 (+126895us) service_pool.cc:224] Handling call
0709 12:18:00.606229 (+ 23230us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:00.735741 (+129512us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:00.735776 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:00.736263 (+   487us) log.cc:844] Serialized 26 byte log entry
0709 12:18:00.838769 (+102506us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:00.838827 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:00.838918 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:01.243439 (+404521us) spinlock_profiling.cc:243] Waited 8.11 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:18:01.492873 (+249434us) raft_consensus.cc:1685] finished
0709 12:18:01.492911 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:01.493406 (+   495us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":149307,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":5446,"spinlock_wait_cycles":18642688,"thread_start_us":226019,"threads_started":2,"wal-append.queue_time_us":184590}
W20260709 12:18:01.448606  9224 consensus_meta.cc:287] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.793s	user 0.010s	sys 0.003s
I20260709 12:18:01.048586  9238 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:01.082111  9168 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:00.985807  9140 consensus_queue.cc:1059] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.499s
I20260709 12:18:01.454978  9059 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:00.976202  9066 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:00.992563  9205 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:00.980649  9152 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:01.244829  9171 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:01.001542  9214 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:01.434558  9148 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.502122  6503 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:18:01.085083  9172 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:01.461421  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:01.505308  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 53906.1us
W20260709 12:18:01.445629  9169 consensus_meta.cc:287] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.650s	user 0.005s	sys 0.017s
I20260709 12:18:01.457170  8511 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7.
I20260709 12:18:01.117817  9219 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:01.465276  9190 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:18:01.499892  9191 consensus_meta.cc:287] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.845s	user 0.012s	sys 0.000s
I20260709 12:18:01.453059  9046 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:18:01.545092  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 83792.3us
I20260709 12:18:01.565011  6622 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:01.564932  9191 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:01.566362  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:01.566686  9034 raft_consensus.cc:2804] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:01.506810  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:01.536422  9146 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:01.482091  9237 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:01.541831  9171 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:01.539734  9238 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:01.569635  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1387) took 1111 ms. Trace:
I20260709 12:18:01.544486  9172 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:01.545651  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 167634us
I20260709 12:18:01.536181  9059 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:01.518757  9168 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:01.560220  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 126703us
I20260709 12:18:01.586230  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:01.602761  6475 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:01.507761  9224 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:01.601799  6652 rpcz_store.cc:276] 0709 12:18:00.458690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.901238 (+442548us) service_pool.cc:224] Handling call
0709 12:18:00.901576 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:00.901936 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:00.901969 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:00.902580 (+   611us) log.cc:844] Serialized 26 byte log entry
0709 12:18:01.053977 (+151397us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:01.054048 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:01.054148 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:01.569371 (+515223us) raft_consensus.cc:1685] finished
0709 12:18:01.569398 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:01.569569 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14840,"raft.queue_time_us":4045,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":7406,"thread_start_us":126775,"threads_started":1,"wal-append.queue_time_us":627149}
I20260709 12:18:01.573872  9205 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:01.554209  9148 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:01.560124  9169 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:01.610642  6550 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:01.570976  9034 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:18:01.602996  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 103238us
I20260709 12:18:01.614420  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:01.633630  6507 raft_consensus.cc:1240] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:01.604151  9172 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:01.576133  9214 consensus_queue.cc:1048] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 12:18:01.641662  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:45870 (request call id 76) took 1692 ms. Trace:
I20260709 12:18:01.602938  9227 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:01.576923  9252 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:01.619138  6475 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:01.639238  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:01.617534  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:01.641415  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 84477.1us
I20260709 12:18:01.642254  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:01.648976  6503 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
I20260709 12:18:01.642716  6508 rpcz_store.cc:276] 0709 12:17:59.949670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.949805 (+   135us) service_pool.cc:224] Handling call
0709 12:18:01.641470 (+1691665us) spinlock_profiling.cc:243] Waited 554 ms on lock 0x7b54001e87d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e83853 00007f287ae5160a 00007f28743d77d4 00007f28743d7762 00007f28743d7691 00007f28743d760c 00007f28743d68c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:01.641596 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1273243904}
I20260709 12:18:01.649981  6501 raft_consensus.cc:1240] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:18:01.650492  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:01.612133  9195 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:01.602144  9231 raft_consensus.cc:2804] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:01.602200  9248 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:01.610810  9191 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:01.663275  9246 log.cc:927] Time spent T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260709 12:18:01.664906  9242 log.cc:927] Time spent T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:18:01.665503  9059 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:01.666671  9251 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:01.667234  9251 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:01.667675  6548 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:01.651173  6508 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
I20260709 12:18:01.570927  9046 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:01.623914  9042 consensus_meta.cc:287] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.844s	user 0.012s	sys 0.000s
I20260709 12:18:01.657789  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:01.625470  9034 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:01.619617  9205 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:01.720559  9205 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:01.625386  9254 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.652475  9253 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.722297  9254 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:01.722497  9253 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:01.662642  9231 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:01.724817  9231 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:01.667588  9251 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:01.727999  9224 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:01.602082  9240 consensus_queue.cc:1059] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.156s
I20260709 12:18:01.561731  9222 consensus_queue.cc:237] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:01.659278  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:01.603003  9109 consensus_queue.cc:1048] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 12:18:01.563977  9219 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:01.653180  9252 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:01.699955  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1399) took 1640 ms. Trace:
I20260709 12:18:01.710199  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:01.710669  6503 raft_consensus.cc:2393] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 3: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:01.660495  9227 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:01.614840  9238 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:01.716883  9148 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:01.731498  6500 raft_consensus.cc:1275] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:18:01.732828  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 113755us
W20260709 12:18:01.733867  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 88564.3us
W20260709 12:18:01.735183  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 124603us
W20260709 12:18:01.739840  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 80626.4us
I20260709 12:18:01.742293  8892 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.749976  9237 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:01.752538  9140 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:01.754038  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 333) took 1929 ms. Trace:
I20260709 12:18:01.755908  6504 raft_consensus.cc:1275] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 12:18:01.714593  9120 consensus_meta.cc:287] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.722s	user 0.005s	sys 0.006s
I20260709 12:18:01.757638  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1393) took 1256 ms. Trace:
W20260709 12:18:01.760458  6473 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.810s (SENT) [suppressed 6 similar messages]
I20260709 12:18:01.745218  6584 rpcz_store.cc:276] 0709 12:18:00.059079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.469128 (+410049us) service_pool.cc:224] Handling call
0709 12:18:00.493244 (+ 24116us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:00.565431 (+ 72187us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:18:00.754256 (+188825us) log.cc:844] Serialized 14 byte log entry
0709 12:18:01.140802 (+386546us) spinlock_profiling.cc:243] Waited 3.78 ms on lock 0x7b18000d9b40. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a2923a 00007f2875e93a2a
0709 12:18:01.525984 (+385182us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:01.526650 (+   666us) log.cc:844] Serialized 26 byte log entry
0709 12:18:01.526836 (+   186us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:01.526882 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:01.526981 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:01.694281 (+167300us) raft_consensus.cc:1685] finished
0709 12:18:01.699667 (+  5386us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:01.699860 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49358,"prepare.queue_time_us":242086,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":36,"raft.queue_time_us":108385,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":10052,"spinlock_wait_cycles":8685312,"thread_start_us":508932,"threads_started":2,"wal-append.queue_time_us":770068}
W20260709 12:18:01.737411  9193 consensus_meta.cc:287] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.912s	user 0.012s	sys 0.000s
I20260709 12:18:01.763096  6475 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:18:01.773527  6579 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 1388) took 1809 ms (client timeout 2349 ms). Trace:
I20260709 12:18:01.756453  9252 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:01.757012  9195 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:01.774262  6475 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:01.764935  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:01.768648  6655 rpcz_store.cc:276] 0709 12:17:59.824956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.826872 (+  1916us) service_pool.cc:224] Handling call
0709 12:17:59.827242 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:00.573178 (+745936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:01.753799 (+1180621us) spinlock_profiling.cc:243] Waited 745 ms on lock 0x7b54001b1554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:01.753976 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1712812800}
I20260709 12:18:01.770496  6649 rpcz_store.cc:276] 0709 12:18:00.501545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.999466 (+497921us) service_pool.cc:224] Handling call
0709 12:18:00.999867 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:01.021953 (+ 22086us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:01.021984 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:01.060723 (+ 38739us) log.cc:844] Serialized 26 byte log entry
0709 12:18:01.596239 (+535516us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:01.596302 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:01.596427 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:01.757430 (+161003us) raft_consensus.cc:1685] finished
0709 12:18:01.757448 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:01.757582 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21367,"raft.queue_time_us":84304,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":23474,"thread_start_us":447131,"threads_started":1,"wal-append.queue_time_us":592465}
I20260709 12:18:01.765643  6550 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:01.774290  6579 rpcz_store.cc:269] 0709 12:17:59.964176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.974084 (+  9908us) service_pool.cc:224] Handling call
0709 12:18:01.773378 (+1799294us) spinlock_profiling.cc:243] Waited 450 ms on lock 0x7b5400298e54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:01.773473 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1034973568}
I20260709 12:18:01.737221  9263 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:01.780507  9224 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:01.780645  9190 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:01.781653  9169 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:01.783051  9156 raft_consensus.cc:3055] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:01.713060  9042 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:01.786226  6475 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:01.818598  6475 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:01.780678  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:01.821707  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:01.761845  9262 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:01.785770  9195 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:01.760680  9227 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:01.776183  8892 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:01.790612  9190 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:01.837450  9214 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:01.837909  9265 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:18:01.839933  9139 consensus_meta.cc:287] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.833s	user 0.010s	sys 0.001s
I20260709 12:18:01.777971  9193 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:01.778124  9120 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:01.840921  9271 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.875773  9271 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:01.841671  9240 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:01.842429  9148 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:01.843871  9238 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:01.784296  9263 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:01.878307  9240 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:01.879136  9263 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:01.878950  9148 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:01.788196  9224 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:01.845806  9254 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:01.846158  9268 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:01.849000  9044 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:01.790910  9237 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:01.834596  9191 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:18:01.853271  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 185672us
I20260709 12:18:01.836946  9168 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:01.867594  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 107175us
I20260709 12:18:01.850970  9156 raft_consensus.cc:740] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:18:01.938529  9233 consensus_meta.cc:287] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.935s	user 0.011s	sys 0.002s
I20260709 12:18:01.861428  9214 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:01.841670  9219 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:01.854215  9195 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:01.855434  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:01.867676  9265 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:01.935647  9139 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:01.858033  9205 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:01.982422  9273 log.cc:927] Time spent T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 12:18:01.982307  9042 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:01.983745  9193 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:01.988638  9238 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:01.990185  9263 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:01.993595  6509 raft_consensus.cc:2393] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 7: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:01.860085  9190 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:01.994733  8839 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:01.996943  9271 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:01.998631  8892 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:01.987782  9034 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:01.999679  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 181167us
I20260709 12:18:02.040067  6475 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:01.869050  9169 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:18:01.979332  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 157713us
I20260709 12:18:01.980849  9224 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:01.995369  9254 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:02.048442  9254 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:02.049446  9251 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:02.051002  9254 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:02.000209  9237 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:02.051663  9237 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:02.051854  9251 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:02.012408  9265 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:02.059669  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:02.056226  9263 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:02.035585  9238 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:02.063199  9240 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:02.063898  9046 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:02.064419  9237 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:02.064463  9156 consensus_queue.cc:260] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:02.065434  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 1703) took 1186 ms. Trace:
I20260709 12:18:02.065800  6507 rpcz_store.cc:276] 0709 12:18:00.879506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.879810 (+   304us) service_pool.cc:224] Handling call
0709 12:18:00.880065 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:01.634122 (+754057us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:02.065207 (+431085us) spinlock_profiling.cc:243] Waited 753 ms on lock 0x7b540001c7d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:02.065378 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1732565888}
I20260709 12:18:02.067008  9205 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:02.068547  9231 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:02.068619  6550 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:02.030515  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:02.031545  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:02.031453  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:02.032289  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:02.004650  9191 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:02.009428  9214 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:02.010003  7266 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:01.998442  9066 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:02.009052  9262 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:02.008688  9269 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:02.042574  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:02.044066  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:02.044570  9193 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:18:02.047492  6609 heartbeater.cc:646] Failed to heartbeat to 127.5.58.62:36381 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.5.58.62:36381 timed out after 15.000s (SENT)
W20260709 12:18:02.054920  6653 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58980 (request call id 1906) took 2229 ms (client timeout 2241 ms). Trace:
I20260709 12:18:02.056152  9172 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:02.069782  9237 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:02.122556  9193 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:02.123067  9193 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:02.008023  9233 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:02.124660  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:02.124785  9265 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:02.125921  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:02.126085  9233 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:02.126631  6655 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 5: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:02.126861  6508 raft_consensus.cc:2368] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 7. Current term is 8.
I20260709 12:18:02.127465  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:02.114667  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:02.128702  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:02.129750  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:02.120126  9278 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:02.130491  6652 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:02.130894  9278 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:02.132691  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:02.133272  9278 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:02.119117  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 76647.4us
W20260709 12:18:02.135805  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 67351.7us
I20260709 12:18:02.136070  6584 raft_consensus.cc:2393] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 7: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:02.007628  9252 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:02.119836  9222 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:02.034399  9227 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:02.372864  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:02.136418  6653 rpcz_store.cc:269] 0709 12:17:59.825091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.825625 (+   534us) service_pool.cc:224] Handling call
0709 12:18:02.054616 (+2228991us) spinlock_profiling.cc:243] Waited 496 ms on lock 0x7b54003787d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:02.054826 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1140903808}
I20260709 12:18:02.201972  9240 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:02.311983  9195 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:02.297065  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:02.933815  6576 raft_consensus.cc:2393] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 5: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:02.885788  9168 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:02.936434  9152 raft_consensus.cc:3055] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:02.240864  8511 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:02.936986  9152 raft_consensus.cc:740] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:02.036813  8839 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:02.115921  9171 consensus_meta.cc:287] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.506s	user 0.005s	sys 0.005s
I20260709 12:18:02.938553  9152 consensus_queue.cc:260] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:02.358423  9238 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:02.940178  8839 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:02.940809  6502 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 5: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:02.159929  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:02.941799  9066 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:02.124168  6649 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:02.942664  6578 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
W20260709 12:18:02.305169  6577 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 1379) took 2351 ms (client timeout 1810 ms). Trace:
I20260709 12:18:02.943660  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:02.943902  6577 rpcz_store.cc:269] 0709 12:17:59.953733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.956424 (+  2691us) service_pool.cc:224] Handling call
0709 12:18:02.304800 (+2348376us) spinlock_profiling.cc:243] Waited 699 ms on lock 0x7b540034cbd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:02.305020 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1607510400}
I20260709 12:18:02.943909  8511 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:02.177745  9191 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:02.945508  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:02.946331  6577 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
I20260709 12:18:02.942055  6652 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:02.947711  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:02.302440  8892 consensus_queue.cc:1048] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.646s
W20260709 12:18:02.948652  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 904539us
I20260709 12:18:02.160405  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:02.234282  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:02.935688  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:02.376433  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:02.294797  9224 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:02.236238  9275 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:02.959986  9275 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:02.301715  9169 consensus_queue.cc:1059] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.636s
I20260709 12:18:02.935932  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:02.962864  6656 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:02.963438  9168 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:02.938788  9171 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:02.867512  9245 log.cc:927] Time spent T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 1.293s	user 0.003s	sys 0.002s
I20260709 12:18:02.966214  9171 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:02.934803  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:02.967689  6576 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 5: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:02.968626  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:02.934646  9195 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:02.969369  6576 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
I20260709 12:18:02.970561  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:02.945134  9148 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:02.119210  9205 consensus_queue.cc:1048] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.817s
I20260709 12:18:02.202479  9269 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:02.977444  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:02.978483  9269 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:02.955123  9238 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:02.979322  9238 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:02.979858  9238 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:02.842329  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:02.967720  7266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38546 (request call id 309) took 2117 ms. Trace:
I20260709 12:18:02.982255  7266 rpcz_store.cc:276] 0709 12:18:00.850588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.851570 (+   982us) service_pool.cc:224] Handling call
0709 12:18:00.875168 (+ 23598us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:02.937065 (+2061897us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:02.967448 (+ 30383us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54005816d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:02.967635 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2595640576}
I20260709 12:18:02.937063  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:02.985378  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:02.986254  6575 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:02.952163  6655 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:18:02.953198  6651 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 3.
W20260709 12:18:02.990634  9170 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.054s	user 0.001s	sys 0.005s
W20260709 12:18:02.991163  9170 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.062s	user 0.001s	sys 0.005s
I20260709 12:18:02.971621  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:02.939595  9252 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:02.882733  9156 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:02.991405  9275 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:02.928208  9283 log.cc:927] Time spent T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:18:02.993147  6578 raft_consensus.cc:2468] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:18:03.000924  6582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1398) took 2942 ms. Trace:
W20260709 12:18:03.001138  6658 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.012s	sys 0.016s
W20260709 12:18:03.001554  6658 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.012s	sys 0.016s
W20260709 12:18:02.855002  9193 consensus_meta.cc:287] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.714s	user 0.010s	sys 0.000s
I20260709 12:18:03.001446  6582 rpcz_store.cc:276] 0709 12:18:00.058800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.391763 (+332963us) service_pool.cc:224] Handling call
0709 12:18:00.392101 (+   338us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:00.401179 (+  9078us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:00.401228 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:02.060521 (+1659293us) spinlock_profiling.cc:243] Waited 5.13 ms on lock 0x7b2c00010c48. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:18:02.061078 (+   557us) log.cc:844] Serialized 52 byte log entry
0709 12:18:02.178722 (+117644us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:02.233750 (+ 55028us) log.cc:844] Serialized 14 byte log entry
0709 12:18:02.233959 (+   209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:02.234041 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:02.394693 (+160652us) spinlock_profiling.cc:243] Waited 5.14 ms on lock 0x7b18003ee540. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:18:03.000429 (+605736us) raft_consensus.cc:1685] finished
0709 12:18:03.000470 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:03.000838 (+   368us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":93913,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":277,"raft.queue_time_us":40696,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":48885,"spinlock_wait_cycles":23639168,"thread_start_us":117144,"threads_started":1,"wal-append.queue_time_us":483541}
I20260709 12:18:03.002936  9193 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:03.004182  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:03.008909  9172 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:03.000777  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:03.011178  9139 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:03.011682  6578 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:03.012521  6657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1386) took 2554 ms. Trace:
I20260709 12:18:02.981608  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:02.941319  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:02.935106  9222 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:03.013011  6657 rpcz_store.cc:276] 0709 12:18:00.457733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.728858 (+271125us) service_pool.cc:224] Handling call
0709 12:18:00.756548 (+ 27690us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:00.784254 (+ 27706us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:00.784291 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:02.069481 (+1285190us) spinlock_profiling.cc:243] Waited 5.68 ms on lock 0x7b2c0010c388. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:18:02.069887 (+   406us) log.cc:844] Serialized 26 byte log entry
0709 12:18:02.274830 (+204943us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:02.274889 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:02.274987 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:03.012112 (+737125us) raft_consensus.cc:1685] finished
0709 12:18:03.012133 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:03.012419 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13069696,"thread_start_us":158481,"threads_started":1,"wal-append.queue_time_us":872738}
I20260709 12:18:03.014410  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:02.202626  9262 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:03.004516  9115 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.010s	user 0.024s	sys 0.023s
W20260709 12:18:02.969242  6502 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 1718) took 1331 ms (client timeout 1626 ms). Trace:
W20260709 12:18:03.015604  9115 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.022s	user 0.024s	sys 0.023s
I20260709 12:18:03.016723  6654 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 7 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:03.016964  6583 raft_consensus.cc:2410] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:18:03.017176  9262 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:03.018040  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:03.018363  6654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 1916) took 3111 ms. Trace:
I20260709 12:18:02.399024  9120 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:03.018827  6654 rpcz_store.cc:276] 0709 12:17:59.906556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.352738 (+446182us) service_pool.cc:224] Handling call
0709 12:18:00.353068 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:03.017116 (+2664048us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.018152 (+  1036us) spinlock_profiling.cc:243] Waited 2.66 s on lock 0x7b54003e8fd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:03.018303 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6125037952}
I20260709 12:18:03.015748  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:02.995011  6652 raft_consensus.cc:2410] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:18:03.021296  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:03.022255  9034 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:03.022396  9231 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:02.612241  9251 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:03.023257  6579 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
I20260709 12:18:03.026891  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:02.417454  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:03.029217  9222 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:02.625809  9254 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.030017  9254 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:03.030277  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:02.392064  9230 raft_consensus.cc:3055] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:03.033491  9230 raft_consensus.cc:740] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:03.032217  6579 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 5: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:02.523885  9225 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:03.035322  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:02.441741  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:03.037029  6579 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:03.037818  6500 raft_consensus.cc:2393] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 7: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:03.038146  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:02.545697  9236 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:03.038935  9236 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:03.039289  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:03.040722  6579 raft_consensus.cc:2393] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 7: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:02.563331  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 1700) took 1684 ms. Trace:
I20260709 12:18:03.041724  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:02.523238  9233 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.039954  9238 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:03.042178  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:03.043385  6576 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
I20260709 12:18:03.044163  6576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1599) took 1012 ms. Trace:
I20260709 12:18:03.044628  6576 rpcz_store.cc:276] 0709 12:18:02.031795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:02.970297 (+938502us) service_pool.cc:224] Handling call
0709 12:18:03.044094 (+ 73797us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:03.044184  6579 raft_consensus.cc:2393] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 5: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:03.036868  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:03.041018  9236 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:02.352123  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1376) took 2393 ms. Trace:
I20260709 12:18:02.201726  9046 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:02.440615  9239 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:02.114313  7267 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:18:02.371320  9237 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:02.397408  6650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1391) took 1932 ms. Trace:
I20260709 12:18:02.974458  9265 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:03.023584  9251 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:02.934423  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:03.054893  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:03.065552  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:03.095248  6579 raft_consensus.cc:2393] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 5: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:03.096033  6581 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:03.041666  6509 rpcz_store.cc:276] 0709 12:18:00.878607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.878815 (+   208us) service_pool.cc:224] Handling call
0709 12:18:02.563091 (+1684276us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400204254. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:02.563246 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2561060096}
I20260709 12:18:03.013028  6577 raft_consensus.cc:2410] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:18:03.067072  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:03.101756  6576 raft_consensus.cc:2410] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:18:03.064623  6580 rpcz_store.cc:276] 0709 12:17:59.952653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:59.955777 (+  3124us) service_pool.cc:224] Handling call
0709 12:17:59.972130 (+ 16353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:00.502434 (+530304us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:02.345343 (+1842909us) spinlock_profiling.cc:243] Waited 444 ms on lock 0x7b54001cc354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:02.345893 (+   550us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1021225856}
I20260709 12:18:03.018824  6583 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:18:02.555876  9146 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:18:03.016075  6502 rpcz_store.cc:269] 0709 12:18:01.637982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:01.641991 (+  4009us) service_pool.cc:224] Handling call
0709 12:18:02.969052 (+1327061us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54001e7dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:02.969187 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2969922816}
I20260709 12:18:03.073730  6650 rpcz_store.cc:276] 0709 12:18:00.465201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:00.959258 (+494057us) service_pool.cc:224] Handling call
0709 12:18:02.055376 (+1096118us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:02.055696 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:02.055740 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:02.055794 (+    54us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:02.055842 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:02.055911 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:02.397153 (+341242us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54003787d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:02.397320 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2519819520}
I20260709 12:18:03.049343  9152 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:03.081938  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:03.113556  6584 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:03.115443  9225 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:03.115962  9225 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:03.111066  9146 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:03.118021  9225 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:03.118533  9225 consensus_queue.cc:1059] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:03.119227  6580 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:18:03.042389  9233 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:03.072252  9239 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:03.046013  9248 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:03.083110  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:03.083460  6578 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:02.383283  9253 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:03.080523  9240 raft_consensus.cc:697] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:03.087380  9171 consensus_queue.cc:1048] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:18:03.081286  9224 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:03.088461  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 155902us
I20260709 12:18:03.072139  9046 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:03.089020  9256 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:03.089980  7267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38546 (request call id 311) took 1013 ms. Trace:
I20260709 12:18:03.083482  9231 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.034544  9230 consensus_queue.cc:260] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:03.060163  9227 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:03.137355  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:03.100610  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:03.103184  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:03.104552  6655 raft_consensus.cc:1240] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:03.106299  6649 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 12:18:03.107890  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 79932.1us
I20260709 12:18:03.110028  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1598) took 1078 ms. Trace:
I20260709 12:18:03.112536  6576 raft_consensus.cc:1240] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:18:03.173398  6577 rpcz_store.cc:276] 0709 12:18:02.031514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:02.947429 (+915915us) service_pool.cc:224] Handling call
0709 12:18:03.109952 (+162523us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:03.079777  6575 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:18:03.124882  6581 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:18:03.174657  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1602) took 1141 ms. Trace:
I20260709 12:18:03.093192  9034 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:18:03.128168  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 177328us
I20260709 12:18:03.145780  9260 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:18:03.149456  7267 rpcz_store.cc:276] 0709 12:18:02.076362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:02.088462 (+ 12100us) service_pool.cc:224] Handling call
0709 12:18:02.114003 (+ 25541us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:03.074953 (+960950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.089870 (+ 14917us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:03.177695  9034 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:03.180641  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:03.181692  9046 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:03.182592  6509 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
new_leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:58944
I20260709 12:18:03.182449  9240 consensus_queue.cc:237] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:03.185498  6501 raft_consensus.cc:1240] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:03.078291  9191 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:03.187537  6500 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:03.188153  9191 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:03.188576  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:03.189152  6505 raft_consensus.cc:1275] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:18:03.156096  9213 consensus_queue.cc:1059] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 12:18:03.155540  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:03.189465  6504 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:18:03.190135  9213 raft_consensus.cc:3055] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:03.190539  9213 raft_consensus.cc:740] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:03.190738  6508 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:18:03.191776  6504 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:18:03.182461  6507 raft_consensus.cc:1275] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:18:03.194054  6508 raft_consensus.cc:1240] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:18:03.195214  7238 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:03.097009  9238 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:03.111303  9254 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:03.110697  6403 catalog_manager.cc:5697] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 5 to 8, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4), pending config changed. New cstate: current_term: 8 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } health_report { overall_health: HEALTHY } } } pending_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:18:03.147046  6650 raft_consensus.cc:1240] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:03.097874  9139 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:03.120450  9115 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:03.078627  9237 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:03.128278  9168 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:03.137246  9161 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.158267  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:03.097000  6579 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5. Ops: [2.5-4.6]
I20260709 12:18:03.191632  9213 consensus_queue.cc:260] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:03.168591  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:03.144249  9222 raft_consensus.cc:3055] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:03.174835  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:03.175148  6581 rpcz_store.cc:276] 0709 12:18:02.032935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:02.934179 (+901244us) service_pool.cc:224] Handling call
0709 12:18:03.174598 (+240419us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:03.176177  6583 raft_consensus.cc:1240] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:18:03.196019  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:03.197338  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:03.198477  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:03.180085  9034 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:03.202860  6650 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:03.205387  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:03.207132  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:03.207733  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:03.140118  9248 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:03.210175  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:03.212499  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:03.159969  9231 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:03.167660  6584 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
W20260709 12:18:03.346971  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:03.171700  9227 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:03.233484  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:03.200736  9195 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:03.366585  6503 raft_consensus.cc:2393] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 5: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:03.368680  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:03.369519  6654 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:18:03.370704  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:03.374749  6500 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:18:03.376233  6507 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:03.378508  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:03.379459  6649 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:18:03.380163  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:03.235787  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:03.383877  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:03.263340  6473 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.019s	sys 0.051s
I20260709 12:18:03.353001  6579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1606) took 1236 ms. Trace:
W20260709 12:18:03.385329  6473 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.019s	sys 0.058s
I20260709 12:18:03.365679  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:03.385398  6579 rpcz_store.cc:276] 0709 12:18:02.116669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.096081 (+979412us) service_pool.cc:224] Handling call
0709 12:18:03.096616 (+   535us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:03.352719 (+256103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:03.352942 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:03.268059  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:03.386732  6653 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:03.386865  6504 raft_consensus.cc:2393] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 6: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:03.386828  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:03.380163  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:03.192615  9230 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:03.321923  6509 raft_consensus.cc:606] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Received request to transfer leadership to TS 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:18:03.347054  9214 consensus_meta.cc:287] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 1.214s	user 0.005s	sys 0.005s
I20260709 12:18:03.405730  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:03.406417  6500 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
I20260709 12:18:03.407470  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:03.408151  6650 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:03.233206  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:03.344655  9239 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:03.399031  6502 raft_consensus.cc:1275] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:18:03.399940  6509 raft_consensus.cc:614] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:03.411206  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:03.401160  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:03.420192  6503 raft_consensus.cc:2368] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7.
I20260709 12:18:03.207016  8852 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:03.204380  9191 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:03.356532  9222 raft_consensus.cc:740] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:03.201445  8511 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:03.203778  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:03.396274  9236 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.411538  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:03.412249  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:03.413952  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:03.206532  9193 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:03.396600  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 454677us
I20260709 12:18:03.342931  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:03.430425  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:03.218556  9139 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:03.455506  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:03.291396  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:03.420840  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:03.421008  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:03.457660  6579 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 5: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:03.422485  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:03.403539  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 291512us
I20260709 12:18:03.459874  6656 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:18:03.417793  9238 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:03.204383  9146 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:03.233378  9260 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.201s
I20260709 12:18:03.411278  9066 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.433562  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:03.494475  6504 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 5: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:03.453191  6584 raft_consensus.cc:2393] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 5: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:03.495976  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:03.498474  6584 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:03.453455  6508 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:03.418875  6505 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 3.
I20260709 12:18:03.462556  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:03.372166  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:03.233927  9115 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:03.359211  9161 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:03.469933  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:03.511999  9115 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:03.470485  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:03.470633  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:03.514500  6576 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:03.472069  6655 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 5: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:03.471968  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:03.473313  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:03.516690  6581 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:18:03.516300  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:18:03.479264  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:03.410358  9120 consensus_queue.cc:1059] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 12:18:03.402904  9195 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:03.517776  6655 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:03.515482  6507 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5.
I20260709 12:18:03.515980  6578 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:03.522699  6652 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:18:03.519616  9195 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:03.524734  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:03.525733  6650 raft_consensus.cc:2393] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 5: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:03.528697  6654 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:03.528195  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:03.535818  6650 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:03.433869  8852 raft_consensus.cc:697] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:03.419104  9214 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:03.431730  7268 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:03.431489  9063 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 12:18:03.541380  6656 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:03.431236  9191 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:03.428081  9130 consensus_queue.cc:1059] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.107s
I20260709 12:18:03.431574  9256 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.108s
I20260709 12:18:03.452579  9216 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.102s
I20260709 12:18:03.551411  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:03.543870  6655 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:18:03.564775  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:03.555045  6650 raft_consensus.cc:2410] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:18:03.557226  6654 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:03.559155  6649 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:03.560107  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:03.574934  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:03.575295  6651 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:18:03.546453  6508 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
I20260709 12:18:03.577682  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:03.578405  6508 raft_consensus.cc:2393] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 6: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:03.539093  9299 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:03.566171  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 146949us
I20260709 12:18:03.565438  6656 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:18:03.543210  9233 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:03.454941  9236 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:03.575935  6649 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:03.543756  9240 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 12:18:03.583801  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:03.485569  9238 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:03.568766  9275 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:18:03.595124  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 125596us
I20260709 12:18:03.543831  9066 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:03.601991  6649 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:03.603391  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:03.604276  6508 raft_consensus.cc:2393] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 5: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:03.596217  6650 raft_consensus.cc:2393] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 7: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:03.606487  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:03.586930  9251 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:03.608536  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:03.608893  6550 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:03.609583  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:03.610252  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:03.610672  6579 raft_consensus.cc:2393] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 5: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:03.611303  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:03.609597  6581 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:03.605927  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:03.578293  9195 consensus_queue.cc:1059] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 12:18:03.613125  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:03.582556  9034 consensus_queue.cc:1059] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 12:18:03.622308  6550 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:03.622876  6584 raft_consensus.cc:2368] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7.
I20260709 12:18:03.614310  6576 raft_consensus.cc:2393] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 3: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:03.642788  6649 raft_consensus.cc:1240] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 5. Current term is 6. Ops: []
W20260709 12:18:03.624495  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 136793us
I20260709 12:18:03.546456  9222 consensus_queue.cc:260] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:03.647480  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:03.660003  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:03.584905  9299 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:03.624138  9293 log.cc:927] Time spent T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 12:18:03.641562  9146 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:03.614137  8511 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 12:18:03.600248  9237 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:03.703397  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:03.709242  6649 raft_consensus.cc:1240] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:03.663945  9227 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:03.606537  9275 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:03.675313  9295 raft_consensus.cc:3055] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:03.682301  9195 raft_consensus.cc:3055] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:03.717016  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:03.677626  9191 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:03.717767  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:03.719866  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:03.719625  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 139283us
I20260709 12:18:03.720618  6508 raft_consensus.cc:2393] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 8: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:03.720746  6507 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:03.726487  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:03.727174  6508 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:03.680997  9248 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:03.728046  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:03.710326  9231 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:03.728890  6508 raft_consensus.cc:2393] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 7: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:03.731034  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:03.731963  6507 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:03.732877  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:03.734302  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:03.735023  6507 raft_consensus.cc:2393] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 5: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:03.705760  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:03.729692  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:03.707728  9315 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:03.740835  6508 raft_consensus.cc:2393] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 5: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:03.741248  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:03.743040  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:03.701210  9130 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:18:03.745703  9309 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
W20260709 12:18:03.748751  9301 log.cc:927] Time spent T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
W20260709 12:18:03.749153  9152 consensus_meta.cc:287] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.628s	user 0.007s	sys 0.004s
I20260709 12:18:03.717090  9295 raft_consensus.cc:740] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:03.749825  6582 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:18:03.750624  9295 consensus_queue.cc:260] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:03.751086  9299 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:03.751104  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:03.752095  9295 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:03.752506  9214 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:03.756194  9238 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:03.756448  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:03.759362  9275 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:03.760982  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:03.738049  9139 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:03.742327  6507 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:18:03.765576  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:03.766515  6508 raft_consensus.cc:3055] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:03.767100  6508 raft_consensus.cc:740] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:03.770041  6507 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:03.771848  6507 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5. Ops: []
W20260709 12:18:03.752141  9303 log.cc:927] Time spent T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 12:18:03.768100  6508 consensus_queue.cc:260] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:03.648662  9239 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:03.657367  6550 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:03.718811  9195 raft_consensus.cc:740] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:03.641928  8852 consensus_queue.cc:237] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:03.780519  9190 consensus_meta.cc:287] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 1.740s	user 0.012s	sys 0.003s
I20260709 12:18:03.921794  6503 raft_consensus.cc:1240] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 5. Current term is 6. Ops: []
W20260709 12:18:03.911130  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 263195us
W20260709 12:18:03.874120  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 68990.1us
W20260709 12:18:03.864053  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 218923us
I20260709 12:18:03.894085  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:03.873858  9152 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:03.937928  6550 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:03.837817  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 98553.6us
I20260709 12:18:03.938489  9152 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:03.939067  9152 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:03.939324  9120 consensus_queue.cc:1059] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:18:03.927175  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:03.941083  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:03.941799  9322 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:03.942201  9322 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:03.942540  9322 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:03.948467  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 373873us
I20260709 12:18:03.949129  7238 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:03.950209  9120 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:03.950753  9120 raft_consensus.cc:697] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:03.953701  6500 raft_consensus.cc:1240] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 12:18:03.930154  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 376717us
I20260709 12:18:03.918831  6507 raft_consensus.cc:1240] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 12:18:03.939478  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:03.955348  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:03.956379  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:03.956700  9066 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:03.804301  9156 consensus_meta.cc:287] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.808s	user 0.005s	sys 0.005s
I20260709 12:18:03.845974  9216 raft_consensus.cc:3055] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:03.960839  9313 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:03.963184  9216 raft_consensus.cc:740] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:03.964148  9216 consensus_queue.cc:260] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:03.965456  9216 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:03.815510  9161 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:03.801398  9236 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:03.814345  9195 consensus_queue.cc:260] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:03.968439  9148 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:03.969947  9195 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:03.969131  9148 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:03.951673  9120 consensus_queue.cc:237] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:03.977864  6550 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:03.979441  6622 leader_election.cc:400] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 7. Current term is 8.
I20260709 12:18:03.979684  6622 leader_election.cc:403] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:03.980930  6550 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:03.986332  6622 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:03.981326  6403 catalog_manager.cc:5697] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:03.988425  6622 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:03.989737  6403 catalog_manager.cc:5697] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:03.971504  9171 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:04.002467  6403 catalog_manager.cc:5697] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 5 to 7, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 7 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:03.952806  9190 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:04.005538  6622 leader_election.cc:400] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7.
I20260709 12:18:04.008616  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:04.008976  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:04.009758  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:04.009778  6654 raft_consensus.cc:2393] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 5: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:04.008476  6622 leader_election.cc:403] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:04.010242  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:04.011010  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:04.011427  6650 raft_consensus.cc:2393] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 3: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:04.011605  6655 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:04.012744  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:04.013886  6656 raft_consensus.cc:2393] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 7: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:18:04.006842  6406 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57900 (request call id 187) took 17657 ms (client timeout 15000 ms). Trace:
I20260709 12:18:04.017858  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:04.005290  9115 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:04.020421  6657 raft_consensus.cc:1240] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:04.020912  9322 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:04.023238  9322 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:04.023382  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 439031us
I20260709 12:18:04.024443  9130 consensus_queue.cc:1059] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:04.025399  9239 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:04.025471  9168 consensus_queue.cc:1059] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:04.027691  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:04.028317  6656 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:04.028483  9253 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:04.028931  9253 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:04.029316  6455 auto_rebalancer.cc:893] Removing replace marker failed: Replica e7554c39679a41bbbca4659b72ab5c65 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:18:04.029719  6455 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 12:18:04.029994  6455 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 12:18:04.030864  9253 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:04.032006  6548 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:04.033179  9152 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:04.034629  9313 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:04.035354  6548 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:04.035660  9152 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:04.037428  6474 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:04.038156  9046 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:04.038520  9046 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:04.038834  9046 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:04.040691  9240 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:04.041303  9240 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:04.035688  6508 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:04.043692  9240 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:04.044775  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1610) took 1771 ms. Trace:
I20260709 12:18:04.045156  6583 rpcz_store.cc:276] 0709 12:18:02.273165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.103551 (+830386us) service_pool.cc:224] Handling call
0709 12:18:03.174504 (+ 70953us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:03.353798 (+179294us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:04.044608 (+690810us) spinlock_profiling.cc:243] Waited 70.6 ms on lock 0x7b540034d5d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:04.044723 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":162419584}
I20260709 12:18:04.046291  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:04.046473  9222 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:04.004542  9325 consensus_queue.cc:1059] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 12:18:04.049853  9046 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:03.990582  8511 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:04.051829  8511 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:04.052211  8511 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:04.052929  9046 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:04.036953  9152 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:04.054311  9152 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:04.054617  9152 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:04.014479  9214 consensus_queue.cc:1048] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 12:18:03.987128  9230 consensus_queue.cc:1048] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.072s
W20260709 12:18:04.058717  6392 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 3a47d71c34b541b9b2589b4e80413fa2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 496c7c3792cd4587b4dbeba435f4be1e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:18:04.060899  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:04.062162  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:04.062371  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 56964.7us
I20260709 12:18:04.063009  6622 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:03.990644  9275 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:04.006629  6403 catalog_manager.cc:5697] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 1 to 3, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 3 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:04.036231  6548 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:03.993634  9295 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:04.068235  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:04.085100  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 48948.4us
I20260709 12:18:04.086134  6548 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:04.078761  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:03.990199  9328 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 7. Current term is 8.
I20260709 12:18:04.088867  6500 raft_consensus.cc:1240] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:04.089833  6503 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 12:18:04.090508  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:04.091264  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:04.088116  9299 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:04.091907  9299 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:04.092106  9222 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:04.092312  9299 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:04.093257  6503 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:04.095417  6654 raft_consensus.cc:2393] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 8: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:04.097196  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:04.086907  6548 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:04.098829  6473 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:04.099742  6500 raft_consensus.cc:1240] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:04.101300  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:04.101313  6508 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:18:04.102236  6500 raft_consensus.cc:2393] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 5: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:04.026144  9239 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:04.103365  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:04.106197  6651 raft_consensus.cc:2393] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 7: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:18:04.104444  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.106218  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:04.107560  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:04.106940  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:04.110313  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:04.111554  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:04.103324  6475 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:04.112094  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:04.106498  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:04.115937  6473 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:04.115029  6656 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:18:04.120136  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:04.120851  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:04.121217  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:04.123524  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:18:04.101600  9310 log.cc:927] Time spent T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:18:04.139264  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:04.139374  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:04.140672  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 79071.3us
I20260709 12:18:04.142632  6656 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:18:04.103330  9240 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:04.097398  9265 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:04.145153  6656 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 12:18:04.145332  6473 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:04.093034  9222 raft_consensus.cc:697] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 7 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:03.993184  9324 consensus_queue.cc:1059] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.119s
I20260709 12:18:04.016832  9171 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:04.160386  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 57453.7us
W20260709 12:18:04.113957  9170 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 12:18:04.093120  9275 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:04.163623  6500 raft_consensus.cc:2376] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6. Re-sending same reply.
I20260709 12:18:04.113735  9156 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:18:04.168007  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.168479  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:04.015254  6406 rpcz_store.cc:269] 0709 12:17:46.344879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:46.437292 (+ 92413us) service_pool.cc:224] Handling call
0709 12:17:47.125951 (+688659us) spinlock_profiling.cc:243] Waited 7.35 ms on lock 0x7b5400002210. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d650b2 00007f287bcde759 00007f287bc76a76 00007f287bc47d58 00007f287bd4604e 00007f287a2072a4 00007f287a207232 00007f287a207161 00007f287a2070dc
0709 12:18:04.001810 (+16875859us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:00.203570 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:00.264376 (+ 60806us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:00.264437 (+    61us) write_op.cc:435] Acquired schema lock
0709 12:18:00.264466 (+    29us) tablet.cc:662] Decoding operations
0709 12:18:00.333095 (+ 68629us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:00.333201 (+   106us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:00.333235 (+    34us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:18:00.334031 (+   796us) tablet.cc:701] Row locks acquired
0709 12:18:00.334054 (+    23us) write_op.cc:260] PREPARE: finished
0709 12:18:00.334248 (+   194us) write_op.cc:270] Start()
0709 12:18:00.334387 (+   139us) write_op.cc:276] Timestamp: P: 1783599480334193 usec, L: 0
0709 12:18:00.334412 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:18:00.392047 (+ 57635us) log.cc:844] Serialized 2807 byte log entry
0709 12:18:00.895524 (+503477us) op_driver.cc:464] REPLICATION: finished
0709 12:18:01.153434 (+257910us) write_op.cc:301] APPLY: starting
0709 12:18:01.156553 (+  3119us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:01.194715 (+ 38162us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:01.194859 (+   144us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:01.214926 (+ 20067us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:01.215198 (+   272us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 12:18:01.215245 (+    47us) write_op.cc:312] APPLY: finished
0709 12:18:01.215540 (+   295us) log.cc:844] Serialized 59 byte log entry
0709 12:18:01.215727 (+   187us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:01.215837 (+   110us) write_op.cc:454] Released schema lock
0709 12:18:01.215881 (+    44us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:01.216310 (+   429us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:01.216354 (+    44us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":44022,"reported_tablets":36,"spinlock_wait_cycles":16895744,"child_traces":[["op",{"apply.queue_time_us":222408,"apply.run_cpu_time_us":3359,"apply.run_wall_time_us":71217,"num_ops":7,"prepare.queue_time_us":241534,"prepare.run_cpu_time_us":21301,"prepare.run_wall_time_us":535291,"raft.queue_time_us":199721,"raft.run_cpu_time_us":13657,"raft.run_wall_time_us":208993,"replication_time_us":522179,"thread_start_us":587172,"threads_started":4,"wal-append.queue_time_us":230253,"wal-append.run_cpu_time_us":74875,"wal-append.run_wall_time_us":1592880}]]}
I20260709 12:18:04.161960  6475 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:04.155090  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:04.066378  9115 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7.
I20260709 12:18:04.064194  9334 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.152s
I20260709 12:18:04.164390  6473 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:04.146068  6657 raft_consensus.cc:2393] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 8: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:04.143630  9225 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:04.165306  9291 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:04.117251  9328 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 12:18:04.172760  6650 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:04.116778  9262 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:04.105225  9046 consensus_queue.cc:1048] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 12:18:04.153285  9240 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:04.153136  9168 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:04.187462  6475 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:04.188493  6473 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:04.184851  6658 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:04.166976  9275 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:04.166044  9156 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:04.219439  6473 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:04.222045  6473 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:04.217653  6475 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:04.176473  9190 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:04.172904  9253 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:04.220552  9240 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:04.191144  9291 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:04.228575  9291 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:04.229303  9148 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:04.149039  9337 consensus_queue.cc:1059] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 12:18:04.229979  9336 raft_consensus.cc:3055] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:04.230361  9336 raft_consensus.cc:740] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:04.231652  9152 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:04.192294  9225 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:04.268427  9225 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:04.194186  9233 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:04.234148  9313 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:04.270473  9233 raft_consensus.cc:697] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:04.270766  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:04.271214  9222 consensus_queue.cc:237] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:04.238353  9171 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:04.238785  9168 raft_consensus.cc:697] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:04.238626  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:04.276798  6650 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:04.277458  9213 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:04.277231  9168 consensus_queue.cc:237] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:04.189878  9251 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:04.243602  9290 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:04.241504  6550 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:04.190733  9269 consensus_queue.cc:1059] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 12:18:04.280838  9290 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:04.243377  9063 raft_consensus.cc:3055] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:04.281510  9297 raft_consensus.cc:3055] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:04.281668  9063 raft_consensus.cc:740] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:04.281853  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:04.282027  9297 raft_consensus.cc:740] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:18:04.243585  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 57876.7us
I20260709 12:18:04.283025  9171 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:04.283877  9171 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:04.284230  9171 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:04.284265  9146 raft_consensus.cc:3055] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:04.244676  6475 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:04.284981  9146 raft_consensus.cc:740] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:04.285929  9146 consensus_queue.cc:260] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:04.246618  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:04.246980  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:04.248823  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:04.287900  7269 raft_consensus.cc:2393] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 9: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:04.249637  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:04.288442  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:04.289521  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:04.290349  6656 raft_consensus.cc:2410] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:18:04.245067  9124 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:04.291522  6658 raft_consensus.cc:1240] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:04.257727  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:04.258647  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:04.293041  6652 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5.
I20260709 12:18:04.262118  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:04.294212  6580 raft_consensus.cc:2393] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 8: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:04.262830  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:04.197595  9172 consensus_queue.cc:1059] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 12:18:04.295197  9146 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:04.296531  9269 raft_consensus.cc:3055] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:04.296928  9269 raft_consensus.cc:740] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:04.282912  9297 consensus_queue.cc:260] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:04.297758  9269 consensus_queue.cc:260] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:04.298482  9297 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:04.263947  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:04.221266  9224 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:04.226233  9156 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:04.254591  9236 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:04.238831  9239 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:04.265376  9046 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:04.265848  9253 raft_consensus.cc:697] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 6 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:04.281281  9290 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:04.282642  6550 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:04.305248  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:04.286059  6475 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:04.259864  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:04.308002  6475 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:04.309583  6475 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:04.311311  6550 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:04.313417  6550 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:04.314921  6550 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:04.320580  6656 raft_consensus.cc:1240] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:04.322247  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:04.322973  6577 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:04.323063  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:04.319990  6475 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:04.324870  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:04.327648  6578 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:04.329388  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:04.331681  6578 raft_consensus.cc:1240] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:18:04.333371  6550 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:04.336082  6473 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:04.246661  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:04.346583  6650 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
W20260709 12:18:04.349944  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.352049  6473 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:04.352174  6475 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:04.367201  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 225069us
I20260709 12:18:04.384060  6550 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:04.384440  6473 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:04.320672  9239 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:04.317215  9313 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:18:04.317168  9254 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:04.317341  9224 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:04.412798  6581 raft_consensus.cc:1240] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:04.304910  9046 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:04.271818  9233 consensus_queue.cc:237] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:04.320346  9315 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:18:04.432531  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:04.405774  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.341637  9148 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:04.336860  9236 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:04.414577  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:04.445267  9236 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:04.402184  9066 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:04.445276  9195 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:04.446691  9195 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:04.447492  9298 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:04.350996  8511 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:04.450800  8511 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:04.392369  9170 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:04.402381  9193 consensus_queue.cc:1059] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 12:18:04.452515  9193 raft_consensus.cc:3055] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:04.453080  9193 raft_consensus.cc:740] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:04.456211  6580 raft_consensus.cc:1240] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:04.350508  9296 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 12:18:04.454690  9193 consensus_queue.cc:260] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:04.446009  9236 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:04.374401  9161 consensus_queue.cc:1059] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.058s
I20260709 12:18:04.460103  9161 raft_consensus.cc:3055] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:04.460731  9161 raft_consensus.cc:740] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:04.461792  9161 consensus_queue.cc:260] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:04.422435  9046 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:04.350091  9069 consensus_queue.cc:1059] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.071s
I20260709 12:18:04.469440  9069 raft_consensus.cc:3055] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:04.469938  9069 raft_consensus.cc:740] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:04.470918  9069 consensus_queue.cc:260] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:04.439725  9239 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:04.473093  9253 consensus_queue.cc:237] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:04.428915  9291 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:04.282688  9063 consensus_queue.cc:260] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:04.295255  9336 consensus_queue.cc:260] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:04.476802  9256 raft_consensus.cc:3055] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:04.477228  9256 raft_consensus.cc:740] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:04.477264  9336 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:04.478030  9256 consensus_queue.cc:260] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:04.386861  9262 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:04.412591  9237 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:04.479269  9256 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:04.434080  6622 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:04.399951  6577 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 9.
I20260709 12:18:04.387195  9252 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:04.438556  6653 raft_consensus.cc:2410] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:18:04.477257  9063 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:04.481467  9237 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:04.482122  9237 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:04.483491  9225 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:18:04.490041  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 56049.8us
I20260709 12:18:04.459911  9193 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:04.415540  9320 log.cc:927] Time spent T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:18:04.493935  6622 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:04.504949  7240 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:04.507290  6622 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
W20260709 12:18:04.520905  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 345644us
I20260709 12:18:04.437669  9254 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:04.523471  6474 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:04.350817  9240 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:04.493940  9161 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:04.503961  9069 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:04.500505  9238 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:18:04.539676  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.432653  9313 consensus_queue.cc:1059] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.094s
I20260709 12:18:04.520968  9309 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:04.499564  9251 consensus_queue.cc:1059] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 12:18:04.521126  9225 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:04.526773  9152 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:04.537186  6622 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:04.549309  6575 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 10: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:18:04.489991  9260 consensus_queue.cc:1175] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e7554c39679a41bbbca4659b72ab5c65" responder_term: 6 status { last_received { term: 6 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 6 index: 6 } }
I20260709 12:18:04.535967  9262 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:18:04.588528  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 185317us
W20260709 12:18:04.591832  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.540560  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:04.545424  9328 raft_consensus.cc:2804] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:04.554721  9309 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:04.578872  9225 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:18:04.610265  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 60001.2us
W20260709 12:18:04.593487  8892 consensus_queue.cc:1175] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "496c7c3792cd4587b4dbeba435f4be1e" responder_term: 6 status { last_received { term: 6 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 6 index: 6 } }
W20260709 12:18:04.625869  9299 consensus_meta.cc:287] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.533s	user 0.007s	sys 0.007s
I20260709 12:18:04.603879  9328 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:18:04.663731  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:04.666810  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 282818us
W20260709 12:18:04.668063  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 315968us
I20260709 12:18:04.668679  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:04.669143  6475 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:04.670228  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:04.611135  9190 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:04.665606  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:18:04.677155  6475 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:18:04.661204  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.610641  9291 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:04.627767  9309 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:04.602969  9275 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:04.632779  9225 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:04.725006  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 57390.3us
W20260709 12:18:04.726401  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 60936.6us
W20260709 12:18:04.726931  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.672448  9328 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:04.672713  9152 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:04.714462  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:04.736490  6578 raft_consensus.cc:1240] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:18:04.734193  6550 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:04.743518  9139 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:04.743904  6550 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:04.744805  9148 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:04.745187  9148 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:04.715937  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:04.745616  6550 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:04.745605  9148 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:04.749073  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:04.749698  6584 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:18:04.742090  6475 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:04.746434  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:04.752458  6579 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:18:04.741130  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 204031us
I20260709 12:18:04.726079  9224 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:18:04.490907  9124 consensus_queue.cc:1175] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f2b06aebe97249a4877d798b8118ece4" responder_term: 3 status { last_received { term: 2 index: 5 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 2. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:18:04.754753  6475 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:04.724889  9296 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:04.721942  9170 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:04.673318  9299 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:04.759454  6550 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:04.792523  9346 log.cc:927] Time spent T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:18:04.792938  9239 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:04.751484  9152 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:04.758658  9139 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:04.800894  9139 raft_consensus.cc:697] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:04.801910  9139 consensus_queue.cc:237] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.2, Last appended by leader: 2, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:04.766315  9224 raft_consensus.cc:697] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 6 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:04.805979  9224 consensus_queue.cc:237] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:04.809428  9350 log.cc:927] Time spent T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 12:18:04.811728  9348 log.cc:927] Time spent T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
W20260709 12:18:04.815212  9354 log.cc:927] Time spent T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
W20260709 12:18:04.817131  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 57736.1us
W20260709 12:18:04.821331  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:04.822062  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:04.823218  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.824744  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:04.825347  6652 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:04.737051  8839 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:04.753533  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:04.828784  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:18:04.833364  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:04.750680  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:04.829696  6651 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:04.830813  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:04.758826  9291 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:18:04.880275  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 126812us
I20260709 12:18:04.887398  6649 raft_consensus.cc:1240] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:18:04.891775  6652 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:18:04.762477  9195 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:04.764237  9252 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:04.753989  9325 consensus_queue.cc:1059] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.137s
I20260709 12:18:04.891176  9295 raft_consensus.cc:2764] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 12:18:04.915184  8598 raft_consensus.cc:3055] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:04.915589  8598 raft_consensus.cc:740] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:04.916066  9299 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:04.916419  8598 consensus_queue.cc:260] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:04.894133  8839 raft_consensus.cc:697] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 9 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:04.893123  9291 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:04.791043  9296 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:04.905345  9152 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:04.905809  9252 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:18:04.931915  9156 consensus_meta.cc:287] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.632s	user 0.011s	sys 0.000s
W20260709 12:18:04.946967  9171 consensus_meta.cc:287] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.658s	user 0.009s	sys 0.001s
I20260709 12:18:04.940598  9291 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:04.953022  9252 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:04.963809  9275 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:04.975229  9240 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:04.986258  9256 consensus_meta.cc:287] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.507s	user 0.015s	sys 0.001s
W20260709 12:18:04.987010  9297 consensus_meta.cc:287] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.688s	user 0.012s	sys 0.000s
W20260709 12:18:04.984038  9290 consensus_meta.cc:287] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.648s	user 0.008s	sys 0.003s
I20260709 12:18:04.760905  6475 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:18:05.034580  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:05.046561  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 268338us
W20260709 12:18:05.066329  9069 consensus_meta.cc:287] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.521s	user 0.006s	sys 0.002s
I20260709 12:18:05.054316  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:05.069392  9069 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:05.040282  6580 raft_consensus.cc:1275] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:18:05.070554  9069 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:05.073275  6580 raft_consensus.cc:1275] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:04.990615  9290 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:05.079500  9146 consensus_meta.cc:287] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.784s	user 0.012s	sys 0.001s
I20260709 12:18:04.958528  9156 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:05.081516  9290 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:05.082528  9152 consensus_queue.cc:1059] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:05.083847  9156 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:04.964833  9171 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:05.088528  9069 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:05.091365  9295 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:05.092219  9362 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:05.137032  9362 raft_consensus.cc:697] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:05.137751  9171 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:05.138811  9362 consensus_queue.cc:237] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:05.093176  9296 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:05.098248  9372 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.141438  9372 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:05.142199  9254 raft_consensus.cc:3055] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:05.142786  9254 raft_consensus.cc:740] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:18:05.099071  9063 consensus_meta.cc:287] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.617s	user 0.018s	sys 0.005s
I20260709 12:18:05.145701  7274 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:05.106878  6576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 283) took 1131 ms. Trace:
I20260709 12:18:05.146224  9254 consensus_queue.cc:260] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:05.146768  6576 rpcz_store.cc:276] 0709 12:18:03.975056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:03.975183 (+   127us) service_pool.cc:224] Handling call
0709 12:18:03.975521 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:04.185444 (+209923us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:04.185481 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:04.191402 (+  5921us) log.cc:844] Serialized 26 byte log entry
0709 12:18:04.540714 (+349312us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:04.540777 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.105953 (+565176us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b540034d0d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:05.106045 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:05.106156 (+   111us) raft_consensus.cc:1685] finished
0709 12:18:05.106174 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:05.106600 (+   426us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4903,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":29288,"spinlock_wait_cycles":481961728,"thread_start_us":317672,"threads_started":1,"wal-append.queue_time_us":462507}
I20260709 12:18:05.148132  9254 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:05.151378  9269 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:05.154819  9372 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:05.096064  6577 raft_consensus.cc:1240] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 5. Current term is 6. Ops: []
W20260709 12:18:05.054616  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 293635us
I20260709 12:18:05.057484  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:05.058257  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:05.058197  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:18:05.054478  9046 consensus_meta.cc:287] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.586s	user 0.012s	sys 0.004s
W20260709 12:18:05.063378  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.062711  6550 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:05.047071  8839 consensus_queue.cc:237] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:05.052194  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.131891  6652 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
W20260709 12:18:05.134124  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 406251us
I20260709 12:18:05.079339  9148 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:05.109895  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:05.155871  6583 raft_consensus.cc:1240] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:18:05.163291  6649 raft_consensus.cc:1275] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:18:05.166335  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:05.168500  9046 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:05.170442  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:05.171324  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:05.172051  9315 raft_consensus.cc:3055] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:05.172260  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:05.172549  9315 raft_consensus.cc:740] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:05.173646  9290 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:05.173928  9315 consensus_queue.cc:260] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:05.174260  9252 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:05.177349  9299 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:05.178231  6582 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 10: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:05.159682  6652 pending_rounds.cc:85] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 5
I20260709 12:18:05.179260  6652 pending_rounds.cc:107] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:18:05.179658  6652 raft_consensus.cc:2889] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:05.181348  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:05.181556  9252 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:05.184149  6578 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:05.168246  6475 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:05.186215  9288 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:05.186865  6475 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:05.187034  9288 raft_consensus.cc:697] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:05.188288  9252 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:05.171345  9046 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:05.190356  9295 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:05.177570  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 119762us
I20260709 12:18:05.207556  6549 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:05.198081  9315 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:05.208698  9225 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:05.210882  9172 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:18:05.210981  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 148742us
I20260709 12:18:05.211592  9172 raft_consensus.cc:697] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:05.213485  6550 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:05.214609  9069 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:05.229100  9172 consensus_queue.cc:237] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:05.231156  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:05.243468  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 60505.6us
I20260709 12:18:05.248194  9195 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:18:05.248390  9237 consensus_meta.cc:287] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.610s	user 0.010s	sys 0.002s
I20260709 12:18:05.249002  9195 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:05.249168  9237 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:05.252092  9237 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:05.244227  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:05.268540  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.271205  9252 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:05.272742  7274 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38558 (request call id 427) took 1028 ms. Trace:
I20260709 12:18:05.278035  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 1740) took 1136 ms. Trace:
W20260709 12:18:05.280575  9161 consensus_meta.cc:287] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.738s	user 0.017s	sys 0.001s
I20260709 12:18:05.281375  9275 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:05.282254  6584 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:05.281985  9291 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:05.282891  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:05.283739  6651 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:05.282078  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:05.287146  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 79847.2us
I20260709 12:18:05.273218  7274 rpcz_store.cc:276] 0709 12:18:04.244397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:04.247558 (+  3161us) service_pool.cc:224] Handling call
0709 12:18:05.267995 (+1020437us) spinlock_profiling.cc:243] Waited 856 ms on lock 0x7b5400670cd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:05.272648 (+  4653us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1968777088}
I20260709 12:18:05.278542  6508 rpcz_store.cc:276] 0709 12:18:04.141264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:04.141390 (+   126us) service_pool.cc:224] Handling call
0709 12:18:04.141715 (+   325us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:04.142093 (+   378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:04.142122 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:04.142642 (+   520us) log.cc:844] Serialized 52 byte log entry
0709 12:18:04.526867 (+384225us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:04.661292 (+134425us) log.cc:844] Serialized 14 byte log entry
0709 12:18:04.663559 (+  2267us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:04.663641 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:05.148132 (+484491us) spinlock_profiling.cc:243] Waited 7.15 ms on lock 0x7b180010a2c0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:18:05.277414 (+129282us) raft_consensus.cc:1685] finished
0709 12:18:05.277517 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:05.277785 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2975,"prepare.queue_time_us":410780,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":3905,"raft.queue_time_us":10568,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":68060,"spinlock_wait_cycles":16452992,"thread_start_us":496174,"threads_started":2,"wal-append.queue_time_us":567418}
I20260709 12:18:05.299481  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:05.300561  6579 consensus_meta.cc:287] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.533s	user 0.000s	sys 0.024s
I20260709 12:18:05.300945  6656 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 4.
I20260709 12:18:05.301149  6579 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
W20260709 12:18:05.305068  9342 log.cc:927] Time spent T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.633s	user 0.004s	sys 0.000s
W20260709 12:18:05.306041  9190 consensus_meta.cc:287] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.578s	user 0.009s	sys 0.009s
I20260709 12:18:05.306572  9190 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:05.305295  6578 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:18:05.253435  9288 consensus_queue.cc:237] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:05.329363  9190 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.345052  9148 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:05.329445  9381 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:18:05.348517  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 49277.1us
W20260709 12:18:05.347740  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 56649.4us
I20260709 12:18:05.359225  6473 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:05.359899  8598 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:05.354794  6550 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:05.362252  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 291) took 1030 ms. Trace:
I20260709 12:18:05.360448  9362 consensus_queue.cc:1059] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:18:05.362669  6583 rpcz_store.cc:276] 0709 12:18:04.331466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:04.389774 (+ 58308us) service_pool.cc:224] Handling call
0709 12:18:05.155256 (+765482us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:05.184558 (+ 29302us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.362013 (+177455us) spinlock_profiling.cc:243] Waited 765 ms on lock 0x7b540034add0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:05.362199 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1759309568}
I20260709 12:18:05.365545  9195 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:05.366111  9195 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.366668  9195 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:05.367169  9328 consensus_meta.cc:287] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.616s	user 0.011s	sys 0.004s
I20260709 12:18:05.367635  9328 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:05.368147  9193 consensus_meta.cc:287] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.848s	user 0.012s	sys 0.011s
I20260709 12:18:05.296116  9291 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:05.346313  9225 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:05.373042  9224 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:05.373754  9361 log.cc:927] Time spent T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.443s	user 0.006s	sys 0.000s
I20260709 12:18:05.296226  9295 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:05.379174  9299 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:18:05.381721  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.383736  9362 raft_consensus.cc:3055] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:18:05.385291  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 86121us
I20260709 12:18:05.387172  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:18:05.379288  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 92873.5us
W20260709 12:18:05.363644  9309 consensus_meta.cc:287] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.632s	user 0.007s	sys 0.015s
I20260709 12:18:05.386441  6506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 288) took 1291 ms. Trace:
W20260709 12:18:05.404273  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:05.388234  9353 log.cc:927] Time spent T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.455s	user 0.006s	sys 0.002s
W20260709 12:18:05.391988  9236 consensus_meta.cc:287] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.933s	user 0.013s	sys 0.002s
I20260709 12:18:05.409597  6650 raft_consensus.cc:1240] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:05.384279  9224 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:05.384601  9362 raft_consensus.cc:740] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:05.416397  6652 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:05.422045  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:05.426020  9139 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:18:05.431510  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.391578  9322 consensus_queue.cc:1059] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:18:05.401422  6506 rpcz_store.cc:276] 0709 12:18:04.068857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:04.068986 (+   129us) service_pool.cc:224] Handling call
0709 12:18:04.069374 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:04.069894 (+   520us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:04.069938 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:04.070549 (+   611us) log.cc:844] Serialized 26 byte log entry
0709 12:18:04.403752 (+333203us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:04.403794 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:04.403874 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:05.294619 (+890745us) spinlock_profiling.cc:243] Waited 3.26 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:18:05.316236 (+ 21617us) raft_consensus.cc:1685] finished
0709 12:18:05.316277 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:05.360523 (+ 44246us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":936,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":10419,"spinlock_wait_cycles":7501056,"thread_start_us":240788,"threads_started":1,"wal-append.queue_time_us":423253}
I20260709 12:18:05.432770  9195 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:05.394160  9190 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:05.386234  9337 consensus_queue.cc:1059] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:18:05.452293  6656 raft_consensus.cc:1240] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:05.449509  6622 leader_election.cc:400] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7.
I20260709 12:18:05.450599  6475 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:05.454702  6583 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 4.
I20260709 12:18:05.454835  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:05.458119  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:05.458465  9362 consensus_queue.cc:260] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:05.459933  9362 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:05.409595  9309 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:05.463753  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:05.422438  9236 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:05.487353  6583 raft_consensus.cc:1275] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:05.488915  7271 raft_consensus.cc:1240] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 5. Current term is 6. Ops: []
W20260709 12:18:05.492923  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 58775.5us
I20260709 12:18:05.488607  9256 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.081s
I20260709 12:18:05.497989  6651 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
W20260709 12:18:05.499393  9384 log.cc:927] Time spent T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:18:05.503911  8892 raft_consensus.cc:3055] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:05.504501  8892 raft_consensus.cc:740] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:05.505723  8892 consensus_queue.cc:260] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:05.506395  9130 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.506953  9130 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:05.513353  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:05.506762  7238 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:05.524791  9216 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:05.525324  9216 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:05.525882  9216 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:05.528577  9343 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 12:18:05.530418  9063 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:05.531236  6654 raft_consensus.cc:1240] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 7 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:05.534685  9236 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:05.537652  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 108089us
I20260709 12:18:05.538861  9213 raft_consensus.cc:3055] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:05.539302  9213 raft_consensus.cc:740] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:05.541514  9190 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:05.544696  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 132133us
I20260709 12:18:05.545753  6622 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:05.547561  6575 raft_consensus.cc:1275] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:18:05.549597  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 287) took 1488 ms. Trace:
I20260709 12:18:05.551301  6654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 1963) took 1318 ms. Trace:
I20260709 12:18:05.551690  6654 rpcz_store.cc:276] 0709 12:18:04.232704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:04.248170 (+ 15466us) service_pool.cc:224] Handling call
0709 12:18:04.248469 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:05.531850 (+1283381us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.551061 (+ 19211us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b54000b7054. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:05.551207 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2911216000}
I20260709 12:18:05.502398  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:05.562124  9372 consensus_queue.cc:1059] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:05.563603  9120 raft_consensus.cc:3055] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:05.563959  9120 raft_consensus.cc:740] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:05.564985  9315 consensus_queue.cc:1059] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:05.566244  9315 consensus_queue.cc:1243] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer f2b06aebe97249a4877d798b8118ece4 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:18:05.567049  9148 raft_consensus.cc:3055] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:05.567391  9148 raft_consensus.cc:740] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:05.568219  9148 consensus_queue.cc:260] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:05.569732  9148 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:05.570326  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:05.571628  9328 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:05.511370  9252 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:05.512245  9224 consensus_queue.cc:1059] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 12:18:05.547881  9260 consensus_queue.cc:1059] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.026s
W20260709 12:18:05.573364  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.518096  6584 raft_consensus.cc:1275] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:18:05.540192  9213 consensus_queue.cc:260] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:05.576575  6473 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:05.577088  9130 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:05.506955  8892 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:05.591499  6550 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:05.596894  6652 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:05.550082  6501 rpcz_store.cc:276] 0709 12:18:04.061228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:04.104057 (+ 42829us) service_pool.cc:224] Handling call
0709 12:18:04.113860 (+  9803us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:04.247507 (+133647us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:04.247539 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:04.248154 (+   615us) log.cc:844] Serialized 52 byte log entry
0709 12:18:04.537048 (+288894us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:05.076490 (+539442us) log.cc:844] Serialized 14 byte log entry
0709 12:18:05.077141 (+   651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:05.168929 (+ 91788us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b54002c3dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:05.168972 (+    43us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:05.385352 (+216380us) spinlock_profiling.cc:243] Waited 3.66 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:18:05.549154 (+163802us) raft_consensus.cc:1685] finished
0709 12:18:05.549183 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:05.549394 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30802,"prepare.queue_time_us":428127,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":46084,"raft.run_cpu_time_us":416,"raft.run_wall_time_us":416,"spinlock_wait_cycles":259679744,"thread_start_us":746919,"threads_started":2,"wal-append.queue_time_us":649713}
I20260709 12:18:05.605717  6475 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:05.610123  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:05.597031  9252 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:05.618217  6550 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:05.620309  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 74680.7us
I20260709 12:18:05.621529  9172 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:05.622112  9172 raft_consensus.cc:697] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:05.623114  9172 consensus_queue.cc:237] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:05.623435  6622 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:05.620225  9269 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:05.624989  9269 raft_consensus.cc:697] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:18:05.625624  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.625957  9269 consensus_queue.cc:237] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:05.633553  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:05.643105  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.644644  6657 raft_consensus.cc:1275] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 2. Preceding OpId from leader: term: 8 index: 3. (index mismatch)
I20260709 12:18:05.637941  9381 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:18:05.646344  9152 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:05.648847  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 129023us
I20260709 12:18:05.640648  9252 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:05.655536  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:05.661203  9213 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:05.668944  9362 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:05.669515  9362 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:05.669950  9362 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:05.671176  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:05.672291  9224 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:05.672433  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:05.672966  9224 raft_consensus.cc:697] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 4 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:05.673177  6658 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 3.
I20260709 12:18:05.674315  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:05.674170  9224 consensus_queue.cc:237] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:05.675068  6584 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 3.
I20260709 12:18:05.565001  9120 consensus_queue.cc:260] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:05.677107  6473 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:05.650794  9315 consensus_queue.cc:1059] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:18:05.678658  9315 raft_consensus.cc:3055] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:05.679056  9315 raft_consensus.cc:740] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:05.680071  9290 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:05.680830  9290 consensus_queue.cc:1048] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:18:05.681516  6548 leader_election.cc:336] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.685s (SENT)
I20260709 12:18:05.681993  6548 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:05.682363  9288 consensus_queue.cc:1048] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:05.678851  9291 consensus_queue.cc:1059] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:05.684242  9291 raft_consensus.cc:3055] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:05.684645  9291 raft_consensus.cc:740] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:05.677150  9120 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:05.685611  9291 consensus_queue.cc:260] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:05.686875  9291 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:05.653234  9309 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:05.690117  9236 raft_consensus.cc:3055] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:05.690016  9254 consensus_queue.cc:1048] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:18:05.690639  9236 raft_consensus.cc:740] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:05.692484  9152 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:05.694376  9254 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:05.695091  9152 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:05.695041  9254 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:05.695953  9343 raft_consensus.cc:3055] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:05.691674  9236 consensus_queue.cc:260] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:05.675820  9381 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:05.705246  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:05.705694  7238 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
W20260709 12:18:05.695245  9387 log.cc:927] Time spent T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.150s	user 0.003s	sys 0.000s
I20260709 12:18:05.707643  9325 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.708351  9325 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:05.713192  8598 raft_consensus.cc:2804] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:05.714354  8598 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:05.720618  8598 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:05.696365  9343 raft_consensus.cc:740] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:05.703325  9381 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:05.741897  6548 leader_election.cc:336] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.716s (SENT)
I20260709 12:18:05.710978  9275 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:05.747048  7240 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:05.745317  6649 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:18:05.736409  9139 consensus_queue.cc:1048] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 12:18:05.752871  9139 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.754065  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:05.757115  6548 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:05.680029  9315 consensus_queue.cc:260] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:05.769423  9152 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:05.770371  9152 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:05.771011  9152 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:05.772480  9236 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:05.775856  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 190382us
I20260709 12:18:05.748509  9381 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:05.771399  9343 consensus_queue.cc:260] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:05.754755  9325 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:05.765947  9237 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:18:05.805651  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 45008.5us
I20260709 12:18:05.779978  8839 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:05.824811  8598 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:05.759254  9139 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:05.767972  9190 consensus_queue.cc:1048] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
W20260709 12:18:05.832028  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 96162.8us
W20260709 12:18:05.838357  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:05.840524  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 52670.8us
I20260709 12:18:05.846517  6550 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:05.823589  9254 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:05.852638  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 135208us
W20260709 12:18:05.860765  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.848408  9315 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:05.865345  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:05.849136  9343 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:05.866291  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:05.866391  6475 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:05.822706  9328 consensus_queue.cc:1048] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 12:18:05.853461  9148 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:05.872980  6583 raft_consensus.cc:1240] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:18:05.871236  6582 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 6.
W20260709 12:18:05.868637  9393 log.cc:927] Time spent T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 12:18:05.846899  9237 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:05.872568  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:05.879287  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:05.885991  6649 raft_consensus.cc:2393] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 7: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:05.872812  9148 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:05.875727  9152 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:05.877111  9328 consensus_queue.cc:1243] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer 496c7c3792cd4587b4dbeba435f4be1e log is divergent from this leader: its last log entry 8.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:18:05.907792  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:05.908644  6506 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:05.894366  9148 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:05.916242  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 64525.8us
I20260709 12:18:05.869654  9216 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:05.904314  9193 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:05.919565  6649 raft_consensus.cc:1275] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:18:05.919656  6392 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 6f94b1cd286648a9b82d5abf69bb86d5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 496c7c3792cd4587b4dbeba435f4be1e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:18:05.949321  8598 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:05.951371  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 78833.2us
I20260709 12:18:05.969172  6508 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:05.957302  9400 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.974769  6584 raft_consensus.cc:1275] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 3. (term mismatch)
I20260709 12:18:05.988615  9400 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:05.993700  9237 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:05.997589  9139 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:05.998515  9216 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:06.004098  9396 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:18:06.006021  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:06.007331  9402 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.007957  9402 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:05.931201  9069 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:18:05.985409  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:05.985863  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:05.986109  9397 log.cc:927] Time spent T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:18:05.986653  6653 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:06.007367  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:05.987022  9362 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:06.013513  6580 raft_consensus.cc:2393] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 6: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:05.988642  6584 pending_rounds.cc:85] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 2
I20260709 12:18:06.015455  6584 pending_rounds.cc:107] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 6.3
I20260709 12:18:06.015833  6584 raft_consensus.cc:2889] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:06.021490  6649 raft_consensus.cc:2393] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 7: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:06.024605  9362 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:06.025714  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:06.028013  7271 raft_consensus.cc:2468] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:18:06.033787  6658 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:06.012775  9152 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:06.014375  9254 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.040289  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:06.044237  6550 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:06.031886  9275 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.019798  9139 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.035267  9291 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:06.039876  9381 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:06.053521  6508 raft_consensus.cc:2393] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 5: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:06.020936  9252 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:06.059949  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:06.061553  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:06.062655  6583 raft_consensus.cc:2410] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:18:06.057495  9139 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:06.049397  9362 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.049656  9328 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.055207  9190 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.042660  9152 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:06.050160  9254 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:06.087992  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:06.088743  6658 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:06.092658  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:06.092489  9190 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:06.093391  7268 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:18:06.095227  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:18:06.101389  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:06.102083  6650 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:06.051890  9275 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:06.078162  9328 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:06.123260  6576 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
W20260709 12:18:06.118441  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:06.113363  9401 log.cc:927] Time spent T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
W20260709 12:18:06.176203  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 132579us
I20260709 12:18:06.197188  6506 raft_consensus.cc:2410] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:18:06.190675  9148 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:06.197969  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.126529  6474 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:06.130586  6623 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:06.081189  9291 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:06.084195  9362 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:06.140013  9403 log.cc:927] Time spent T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
W20260709 12:18:06.114226  9395 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 12:18:06.113811  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:06.221944  9290 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:06.222492  9290 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:06.222947  9290 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:06.235251  9290 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:06.102566  9411 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:06.103114  9190 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:06.241436  6505 raft_consensus.cc:2393] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 8: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:06.136296  9402 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:06.115806  9152 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:06.248463  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.115370  9381 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:06.113607  9400 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:06.246126  6658 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 4.
I20260709 12:18:06.131978  9383 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:06.127084  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:06.246810  6548 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:06.249632  6622 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:06.264917  7240 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:06.126142  9236 consensus_queue.cc:1048] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.121s
I20260709 12:18:06.240063  9288 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.247212  9193 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:06.200268  9410 consensus_queue.cc:1059] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 5.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 12:18:06.244730  9411 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.278957  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:06.290028  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 45781.2us
W20260709 12:18:06.290006  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 163541us
I20260709 12:18:06.293497  9222 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.293962  6622 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:06.294101  9222 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:06.295683  6622 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:06.297019  9233 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:06.297657  9233 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:06.299746  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:18:06.293349  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.253083  9190 raft_consensus.cc:2804] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:06.303813  9288 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:06.304287  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:06.304699  9148 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:06.305182  6505 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
W20260709 12:18:06.305431  9409 log.cc:927] Time spent T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:18:06.285297  9366 raft_consensus.cc:2804] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:06.285420  8892 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.315402  9291 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:06.315876  9404 log.cc:927] Time spent T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.255s	user 0.001s	sys 0.000s
I20260709 12:18:06.317905  9362 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:06.317746  8892 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:06.319900  8892 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:06.320571  9139 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:06.321489  9254 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:06.322436  9222 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:06.323608  9222 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:06.325455  9222 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:06.313037  9190 raft_consensus.cc:697] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 5 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:06.292781  9411 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:06.285283  7268 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
W20260709 12:18:06.328588  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.352662  6650 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5.
W20260709 12:18:06.353971  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 223597us
I20260709 12:18:06.355861  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:18:06.358045  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.351768  9366 raft_consensus.cc:697] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 4 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:06.280562  9383 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:06.361085  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:06.361533  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.351361  9381 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:18:06.366236  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 154500us
I20260709 12:18:06.367281  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:06.367626  9421 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:06.368012  6658 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:18:06.372651  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:06.373266  6658 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:06.377161  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:06.377044  9190 consensus_queue.cc:237] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:06.378438  6506 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:18:06.379760  9328 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:06.380384  9410 raft_consensus.cc:3055] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:06.380911  9410 raft_consensus.cc:740] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:06.381916  9410 consensus_queue.cc:260] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:06.372828  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:06.384999  6575 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:06.385475  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:18:06.386559  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:06.386804  6649 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:06.387295  6653 raft_consensus.cc:2393] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 7: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:06.387253  9290 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:06.386152  6658 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:18:06.388103  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:06.388666  6581 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:06.390812  9233 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:06.388262  9366 consensus_queue.cc:237] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:06.392391  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:06.396160  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:06.396761  6505 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
I20260709 12:18:06.401494  9275 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:06.405665  9383 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:06.413919  6583 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:18:06.415133  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:06.417796  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:18:06.418069  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 122508us
I20260709 12:18:06.417830  6506 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:18:06.419737  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:06.420840  9251 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.421409  9251 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:06.424947  6658 raft_consensus.cc:1275] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:06.426020  9288 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:06.427158  6652 raft_consensus.cc:1275] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:18:06.427659  6652 pending_rounds.cc:85] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 2
I20260709 12:18:06.428088  6651 raft_consensus.cc:1275] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:18:06.399312  8839 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:06.428572  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:06.428768  8839 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.429209  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:06.429309  6649 raft_consensus.cc:2393] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 8: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:06.429354  8839 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:06.430711  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:06.431735  6657 raft_consensus.cc:1275] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:18:06.428025  6652 pending_rounds.cc:107] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:18:06.433362  6652 raft_consensus.cc:2889] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:06.437065  6622 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:06.442009  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:06.442698  6583 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:18:06.442720  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:06.443151  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:06.448262  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:06.449431  6658 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:06.454941  9288 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:06.455842  9288 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:06.458815  9288 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:06.464547  6505 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:18:06.466511  9383 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:06.474412  9309 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:06.481752  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:06.482751  6579 raft_consensus.cc:2368] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 5. Current term is 6.
I20260709 12:18:06.304179  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:06.424280  9251 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:06.497553  7271 raft_consensus.cc:1275] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:18:06.496477  6403 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46184 (request call id 115) took 9797 ms. Trace:
I20260709 12:18:06.501979  6649 raft_consensus.cc:1275] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 12:18:06.500712  6403 rpcz_store.cc:276] 0709 12:17:56.699011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:56.699333 (+   322us) service_pool.cc:224] Handling call
0709 12:17:57.244571 (+545238us) spinlock_profiling.cc:243] Waited 378 ms on lock 0x7b540008acd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df3090 00007f2875e91f3a 00007f287bc47e9c 00007f287bd4604e 00007f287a2072a4 00007f287a207232 00007f287a207161 00007f287a2070dc 00007f287a206392 00007f28741c448c 00007f28741c38c6
0709 12:18:06.496202 (+9251631us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:04.630452 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:04.796252 (+165800us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:04.796302 (+    50us) write_op.cc:435] Acquired schema lock
0709 12:18:04.796359 (+    57us) tablet.cc:662] Decoding operations
0709 12:18:04.797276 (+   917us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:04.797374 (+    98us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:04.797416 (+    42us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:18:04.798099 (+   683us) tablet.cc:701] Row locks acquired
0709 12:18:04.798126 (+    27us) write_op.cc:260] PREPARE: finished
0709 12:18:04.798319 (+   193us) write_op.cc:270] Start()
0709 12:18:04.798469 (+   150us) write_op.cc:276] Timestamp: P: 1783599484798255 usec, L: 0
0709 12:18:04.798499 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 12:18:04.902275 (+103776us) log.cc:844] Serialized 2383 byte log entry
0709 12:18:05.201662 (+299387us) spinlock_profiling.cc:243] Waited 4.82 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2872d7f0ac 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b
0709 12:18:05.710088 (+508426us) op_driver.cc:464] REPLICATION: finished
0709 12:18:05.923545 (+213457us) write_op.cc:301] APPLY: starting
0709 12:18:05.940704 (+ 17159us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:05.991963 (+ 51259us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:05.992019 (+    56us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:05.993197 (+  1178us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:05.993637 (+   440us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 12:18:05.993707 (+    70us) write_op.cc:312] APPLY: finished
0709 12:18:05.994169 (+   462us) log.cc:844] Serialized 47 byte log entry
0709 12:18:05.994414 (+   245us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:05.994677 (+   263us) write_op.cc:454] Released schema lock
0709 12:18:05.994769 (+    92us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:05.995395 (+   626us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:05.995450 (+    55us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":802253,"reported_tablets":15,"spinlock_wait_cycles":869372800,"child_traces":[["op",{"apply.queue_time_us":169596,"apply.run_cpu_time_us":4321,"apply.run_wall_time_us":85494,"num_ops":5,"prepare.queue_time_us":423281,"prepare.run_cpu_time_us":50198,"prepare.run_wall_time_us":748940,"raft.queue_time_us":252595,"raft.run_cpu_time_us":48650,"raft.run_wall_time_us":117566,"replication_time_us":911377,"spinlock_wait_cycles":11085696,"thread_start_us":949363,"threads_started":4,"wal-append.queue_time_us":527549}]]}
I20260709 12:18:06.509483  6508 raft_consensus.cc:1240] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:18:06.512967  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:06.513729  6653 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:06.518482  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:06.523398  6581 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5.
I20260709 12:18:06.536623  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:06.537045  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:06.537331  6658 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 6.
I20260709 12:18:06.537736  6581 raft_consensus.cc:2368] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5.
I20260709 12:18:06.538580  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:06.543352  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:06.548877  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:06.550908  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 113980us
I20260709 12:18:06.542819  6575 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 4.
I20260709 12:18:06.554350  6581 raft_consensus.cc:2393] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 6: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:06.553050  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:06.559418  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:06.560101  6575 raft_consensus.cc:2393] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 5: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:06.561398  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:06.513012  9252 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:06.562837  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:06.556427  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:06.566370  6578 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:18:06.567628  6583 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 6: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:06.531661  9288 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:06.535938  9063 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.570457  6505 raft_consensus.cc:2393] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 9: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:06.575732  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:06.573618  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:06.576686  6658 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:18:06.543118  9399 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.535365  9146 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:18:06.562208  9381 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.571587  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:06.565188  9251 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:06.588538  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:06.595438  6505 raft_consensus.cc:1240] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:06.598948  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:06.580030  9236 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:06.599750  6508 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 4.
I20260709 12:18:06.606734  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:06.606673  9236 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:06.607437  9410 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:06.608946  9236 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:06.592672  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:06.610718  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:06.611325  7270 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
I20260709 12:18:06.607606  7274 raft_consensus.cc:2393] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 5: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:06.587293  9383 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:06.626072  6653 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:18:06.632458  9251 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:06.634812  9251 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:06.615216  6623 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:06.648643  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:06.649344  6581 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:06.651342  8839 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:06.593989  9381 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:06.617072  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:06.618126  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.580078  9063 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:06.618631  6503 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 5. Current term is 6. Ops: []
W20260709 12:18:06.620514  9405 log.cc:927] Time spent T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.355s	user 0.004s	sys 0.000s
I20260709 12:18:06.586640  9362 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:06.629248  7265 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
W20260709 12:18:06.614795  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 46578.7us
I20260709 12:18:06.605183  7238 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:06.616320  6474 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:06.608302  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:06.640266  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:06.655735  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:06.656550  9381 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:06.657367  7240 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:06.659343  9063 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:06.668040  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:06.582335  9288 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:06.672936  6622 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:06.673426  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 57178us
I20260709 12:18:06.674157  6474 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:06.675223  6581 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
I20260709 12:18:06.584856  9399 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:06.669157  7274 raft_consensus.cc:2393] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 7: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:06.669656  6576 raft_consensus.cc:2393] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 5: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:06.670271  6508 raft_consensus.cc:1275] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:18:06.664460  6504 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 3.
I20260709 12:18:06.665072  6501 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 6.
W20260709 12:18:06.683908  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 78920.4us
I20260709 12:18:06.687830  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:06.691051  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:06.694350  6502 raft_consensus.cc:1240] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:06.695686  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:06.696853  6502 raft_consensus.cc:2410] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:18:06.696698  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:06.697964  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:06.698068  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:06.698911  6509 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 10: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:06.699000  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:06.699775  6500 raft_consensus.cc:2393] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 8: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:06.699849  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:06.700503  9381 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:18:06.700685  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.700524  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:06.702414  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:06.663501  9383 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:06.713120  6509 raft_consensus.cc:2410] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:18:06.721017  6503 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:18:06.708407  9252 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:06.701192  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:18:06.723960  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.719861  6550 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:06.727414  6473 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:06.729382  6550 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:06.729451  6581 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:18:06.674353  9337 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:18:06.733021  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.734207  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:06.731292  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:06.735874  6653 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:06.736931  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:06.737537  6584 raft_consensus.cc:2393] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 8: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:06.739174  6650 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:06.740631  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:06.740880  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:06.741459  6578 raft_consensus.cc:2410] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:18:06.741438  6508 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 6.
I20260709 12:18:06.744701  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:06.745587  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 72820.4us
I20260709 12:18:06.746811  6502 raft_consensus.cc:2410] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:18:06.746734  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:06.707224  9063 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:06.749029  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:06.743637  9254 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:06.754467  6656 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:18:06.731017  9236 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:06.743471  6550 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:06.730589  9288 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:06.663381  9362 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:06.729115  9383 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.756295  9254 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.753549  6579 raft_consensus.cc:2393] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 6: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:06.775334  9254 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:06.733904  9426 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.734941  9315 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:06.735458  9337 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.768028  6508 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 12:18:06.774221  9428 log.cc:927] Time spent T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:18:06.774820  9383 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:06.734807  9224 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:06.775977  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:06.809054  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:06.869817  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:06.898927  9069 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:07.364027  9069 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:07.364566  9069 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:07.368685  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:07.370709  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:07.371316  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:07.371946  6584 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:06.750530  9411 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:07.374029  6582 raft_consensus.cc:2368] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 8. Current term is 9.
I20260709 12:18:07.375661  9411 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:06.827716  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:06.927470  9419 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.379617  9419 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:06.915199  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:06.904004  6504 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 7. Current term is 8. Ops: []
W20260709 12:18:07.381732  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 627180us
I20260709 12:18:07.359812  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:07.402976  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:06.753039  9172 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:07.219074  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:07.359019  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:07.428982  6509 raft_consensus.cc:2410] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:18:07.413513  9426 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:07.435285  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:07.017694  9146 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.371563  6507 raft_consensus.cc:2393] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 6: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:18:06.862219  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 118795us
I20260709 12:18:07.414090  7270 raft_consensus.cc:1275] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:18:07.433248  6500 raft_consensus.cc:2410] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 12:18:06.969646  9429 log.cc:927] Time spent T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
W20260709 12:18:07.430917  7238 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.600s (SENT) [suppressed 26 similar messages]
I20260709 12:18:06.763675  9063 raft_consensus.cc:697] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
W20260709 12:18:06.944898  9433 log.cc:927] Time spent T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 12:18:06.925526  9431 log.cc:927] Time spent T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
W20260709 12:18:07.438285  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:07.349839  8839 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:07.442091  6550 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:07.471195  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:07.471922  6500 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:07.430472  9216 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.472982  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:07.473351  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:07.405560  9315 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:07.454100  6652 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:18:07.473968  6509 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:07.455310  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:07.456064  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:07.446317  9146 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:07.457878  9426 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:07.438509  9288 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.439095  9420 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.487349  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:06.948459  9399 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:07.426796  9337 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:07.470362  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:07.508823  6579 raft_consensus.cc:2393] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 7: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:07.378557  9224 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:07.475947  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:07.480167  6658 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 6: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:07.483141  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:07.487962  6548 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.012s	sys 0.056s
W20260709 12:18:07.488314  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:07.489056  6581 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:18:07.512046  6654 raft_consensus.cc:2393] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 5: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:07.491293  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:07.512238  6548 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.744s	user 0.012s	sys 0.056s
I20260709 12:18:07.482563  9315 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:07.492789  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:07.514748  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 772193us
I20260709 12:18:07.515223  6500 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:07.451516  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:07.510126  6583 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:07.513049  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:07.519747  6582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1633) took 1099 ms. Trace:
I20260709 12:18:07.522919  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:07.523469  6657 raft_consensus.cc:2393] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 8: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:07.520233  6582 rpcz_store.cc:276] 0709 12:18:06.414607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.414759 (+   152us) service_pool.cc:224] Handling call
0709 12:18:07.513317 (+1098558us) spinlock_profiling.cc:243] Waited 958 ms on lock 0x7b540032b754. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:07.513445 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2202783488}
I20260709 12:18:07.527442  9411 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:07.527936  9411 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:07.528301  9411 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:07.472973  9381 consensus_queue.cc:1048] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:18:07.536034  6509 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:18:07.537943  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:07.538810  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:07.539372  6500 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
W20260709 12:18:07.529731  9236 consensus_meta.cc:287] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.754s	user 0.009s	sys 0.001s
I20260709 12:18:07.535954  6504 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:18:07.542598  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:07.508843  9420 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:07.546725  6509 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:07.544263  9146 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:07.487174  9288 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:07.547441  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1475) took 1029 ms. Trace:
I20260709 12:18:07.488085  9216 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:07.550568  6503 raft_consensus.cc:1275] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 2. Preceding OpId from leader: term: 8 index: 3. (index mismatch)
I20260709 12:18:07.551071  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:07.544239  9254 consensus_meta.cc:287] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.701s	user 0.012s	sys 0.004s
I20260709 12:18:07.555920  6506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 290) took 1093 ms. Trace:
I20260709 12:18:07.556301  6506 rpcz_store.cc:276] 0709 12:18:06.462493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.462684 (+   191us) service_pool.cc:224] Handling call
0709 12:18:06.464114 (+  1430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:06.466241 (+  2127us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:06.466284 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:06.466983 (+   699us) log.cc:844] Serialized 26 byte log entry
0709 12:18:06.562629 (+ 95646us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:06.562697 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:06.562804 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:07.555729 (+992925us) raft_consensus.cc:1685] finished
0709 12:18:07.555749 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:07.555867 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":95021,"threads_started":1,"wal-append.queue_time_us":261308}
I20260709 12:18:07.554733  6500 raft_consensus.cc:2376] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6. Re-sending same reply.
I20260709 12:18:07.560530  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:07.561367  6504 raft_consensus.cc:2410] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:18:07.564438  6508 raft_consensus.cc:1240] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:07.550351  6577 rpcz_store.cc:276] 0709 12:18:06.517714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.518990 (+  1276us) service_pool.cc:224] Handling call
0709 12:18:06.519325 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:06.519663 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:06.519691 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:06.520198 (+   507us) log.cc:844] Serialized 26 byte log entry
0709 12:18:06.638483 (+118285us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:06.638527 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:06.638602 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:07.547200 (+908598us) raft_consensus.cc:1685] finished
0709 12:18:07.547284 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:07.547389 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12230,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":7613,"thread_start_us":117857,"threads_started":1,"wal-append.queue_time_us":214049}
I20260709 12:18:07.549234  9399 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.553086  9236 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:07.558418  9254 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:07.558485  9224 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:07.572446  7238 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:07.557890  9419 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:07.556705  9381 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:07.574357  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 318) took 1097 ms. Trace:
I20260709 12:18:07.566672  9146 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.579489  9146 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:07.581907  9146 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:07.584262  6507 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:07.575104  9399 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:07.573135  9325 consensus_queue.cc:1059] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:18:07.566437  9260 consensus_queue.cc:1059] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 12:18:07.551520  9430 consensus_queue.cc:1059] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 12:18:07.578907  9309 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:07.577014  6580 rpcz_store.cc:276] 0709 12:18:06.476956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.477111 (+   155us) service_pool.cc:224] Handling call
0709 12:18:06.477461 (+   350us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:06.477937 (+   476us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:06.478247 (+   310us) log.cc:844] Serialized 14 byte log entry
0709 12:18:06.580423 (+102176us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:06.597490 (+ 17067us) log.cc:844] Serialized 52 byte log entry
0709 12:18:06.597725 (+   235us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:06.597790 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:06.597883 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:07.574098 (+976215us) raft_consensus.cc:1685] finished
0709 12:18:07.574120 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:07.574296 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":642,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":3514,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":5055,"thread_start_us":101344,"threads_started":1,"wal-append.queue_time_us":199046}
I20260709 12:18:07.583865  9383 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:07.545755  9063 consensus_queue.cc:237] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:07.603672  6500 raft_consensus.cc:1240] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:07.600610  6504 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 12:18:07.614373  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:07.595903  9437 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.574793  8839 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:18:07.621342  6504 pending_rounds.cc:85] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 3
I20260709 12:18:07.631420  6504 pending_rounds.cc:107] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:18:07.631699  6504 raft_consensus.cc:2889] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:07.601539  9146 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:07.610235  9309 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:07.623063  9325 raft_consensus.cc:3055] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:07.656090  9420 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:07.634153  9437 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:07.662844  9437 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:07.665968  9254 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:07.666091  9216 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:07.670081  9383 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:07.659134  9325 raft_consensus.cc:740] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:07.672238  9315 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:07.652156  9309 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:07.679982  6579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1643) took 1037 ms. Trace:
I20260709 12:18:07.655738  9146 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:07.701722  6579 rpcz_store.cc:276] 0709 12:18:06.642504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.808697 (+166193us) service_pool.cc:224] Handling call
0709 12:18:07.679799 (+871102us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54003abe54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:07.679927 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":339649920}
W20260709 12:18:07.679396  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:07.687567  9399 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:07.676131  9362 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:07.751533  9288 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:07.752573  9315 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:07.752449  6474 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:07.754803  6505 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5.
I20260709 12:18:07.755669  9146 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:07.756099  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 50706.9us
I20260709 12:18:07.759193  6502 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:07.759279  6575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1482) took 1113 ms. Trace:
I20260709 12:18:07.765707  6474 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:07.760588  6575 rpcz_store.cc:276] 0709 12:18:06.646121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:07.377779 (+731658us) service_pool.cc:224] Handling call
0709 12:18:07.688960 (+311181us) spinlock_profiling.cc:243] Waited 31 ms on lock 0x7b5800116198. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:18:07.758686 (+ 69726us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:07.758863 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:07.758888 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:07.758965 (+    77us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:07.758998 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:07.759058 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:07.759209 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":71296768}
W20260709 12:18:07.787935  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 345892us
I20260709 12:18:07.795862  6550 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:07.781744  9362 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:07.782958  9288 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:07.803876  9315 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:07.801946  9325 consensus_queue.cc:260] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:07.804526  6550 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:07.806910  9288 raft_consensus.cc:697] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 6 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:07.807957  9288 consensus_queue.cc:237] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:07.808758  9309 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:07.812083  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 364070us
W20260709 12:18:07.817893  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:07.811048  9309 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:07.782696  9069 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:07.788532  9411 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:07.829532  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 204489us
W20260709 12:18:07.831393  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:07.831852  9362 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.832348  9233 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:07.832487  9362 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:07.833173  6475 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:07.833552  9420 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:07.834204  9426 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:07.834142  9420 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:07.835247  9362 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:07.835711  9426 consensus_queue.cc:1048] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:07.835994  9362 consensus_queue.cc:1048] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:07.836347  9420 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:07.832123  6548 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:07.836135  6475 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:07.871547  6475 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:07.829142  9254 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:07.878401  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 306033us
I20260709 12:18:07.883167  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:07.838052  9443 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:07.883910  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 79448.2us
I20260709 12:18:07.891948  6475 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:07.898604  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:07.899293  6577 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
I20260709 12:18:07.900602  6579 raft_consensus.cc:1275] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:18:07.889891  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:07.906579  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:07.906683  6475 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:07.907261  6576 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:07.873086  9362 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:07.928372  6475 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:07.833221  9233 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:07.929119  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 88343.8us
I20260709 12:18:07.909025  9315 raft_consensus.cc:697] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:07.931800  6622 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:07.938124  6475 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:07.914541  9426 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:07.887101  9254 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:07.895561  9443 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:07.934298  9252 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:07.934638  9420 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:18:07.969090  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 79364.8us
I20260709 12:18:07.941757  9362 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:07.976029  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:07.980355  7271 raft_consensus.cc:2393] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 7: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:07.935942  9236 consensus_queue.cc:1048] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 12:18:07.990031  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:07.990702  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:07.992820  7270 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
W20260709 12:18:07.994549  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 162498us
I20260709 12:18:07.978940  9448 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:08.001094  9464 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 12:18:07.958210  9315 consensus_queue.cc:237] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:07.994585  9452 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:18:08.018154  9438 log.cc:927] Time spent T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 12:18:08.017872  6548 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:07.959611  9450 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 12:18:07.987618  9337 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:08.039671  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 107997us
I20260709 12:18:08.019677  9458 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:18:08.047724  9459 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:08.048627  9383 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:08.007763  9063 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.000761  9448 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:08.052227  6622 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:08.073730  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:08.066672  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 144024us
I20260709 12:18:08.049341  9465 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:08.076164  7238 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:08.079715  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.050172  9411 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:08.108561  7268 raft_consensus.cc:2393] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 7: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:08.068925  9069 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:08.112794  6576 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7.
W20260709 12:18:08.121143  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 45081us
I20260709 12:18:08.121541  9443 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:08.121832  9447 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:08.123615  9465 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:08.072491  9383 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:08.075982  9063 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:08.069674  9452 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:08.068979  9459 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:08.129202  9381 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:08.137271  9437 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:08.156661  9443 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:08.134696  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1480) took 1580 ms. Trace:
I20260709 12:18:08.025414  9456 consensus_queue.cc:1059] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.075s
I20260709 12:18:08.077059  9448 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:08.160036  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 86357.4us
I20260709 12:18:08.159471  9465 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:08.163213  9233 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:08.137212  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.174944  9456 raft_consensus.cc:3055] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:08.178785  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:08.168596  9381 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:08.172308  6578 rpcz_store.cc:276] 0709 12:18:06.553936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:07.362421 (+808485us) service_pool.cc:224] Handling call
0709 12:18:07.612768 (+250347us) spinlock_profiling.cc:243] Waited 49.7 ms on lock 0x7b5800116198. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:18:07.613011 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:07.613288 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:18:07.786242 (+172954us) log.cc:844] Serialized 14 byte log entry
0709 12:18:08.134303 (+348061us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:08.134366 (+    63us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:08.134419 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:08.134491 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:08.134636 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8262,"prepare.queue_time_us":288283,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":87,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":6050,"spinlock_wait_cycles":114260864,"thread_start_us":502228,"threads_started":2}
I20260709 12:18:08.197016  6622 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:08.174278  9443 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:08.173118  7269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38562 (request call id 528) took 1449 ms. Trace:
I20260709 12:18:08.196274  9260 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.184470  9470 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:08.196130  9254 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:08.170852  9437 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:08.195675  9456 raft_consensus.cc:740] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:08.219182  9337 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:08.200892  9381 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:08.217219  9465 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:08.218607  9172 consensus_meta.cc:287] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.781s	user 0.011s	sys 0.000s
I20260709 12:18:08.228192  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:08.230901  9465 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:08.231392  9465 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:08.231844  9465 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:08.231948  9063 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:08.230932  9459 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:08.233392  9383 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:08.234161  9383 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:08.234568  9383 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:08.234993  9383 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:08.235180  9172 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:08.207893  9452 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:08.207253  9443 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:08.251880  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:08.231410  6505 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41304 (request call id 302) took 1557 ms (client timeout 1830 ms). Trace:
I20260709 12:18:08.252710  9452 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:08.253414  6505 rpcz_store.cc:269] 0709 12:18:06.673948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.674103 (+   155us) service_pool.cc:224] Handling call
0709 12:18:08.231210 (+1557107us) spinlock_profiling.cc:243] Waited 855 ms on lock 0x7b54000ff354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:08.231315 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1967297792}
I20260709 12:18:08.254925  9452 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:08.259464  9383 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:08.221853  9260 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:08.227095  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:08.221668  9470 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:08.245584  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:08.273798  6575 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:18:08.219532  9456 consensus_queue.cc:260] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:08.243458  9437 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:08.267728  6582 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:18:08.270062  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:08.283365  6581 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
W20260709 12:18:08.212911  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 241691us
I20260709 12:18:08.273260  6583 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:18:08.248495  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:18:08.281960  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:08.287992  6577 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 6.
I20260709 12:18:08.302901  9479 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:08.303721  9479 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:08.305711  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:08.211297  7269 rpcz_store.cc:276] 0709 12:18:06.719105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.721170 (+  2065us) service_pool.cc:224] Handling call
0709 12:18:06.843408 (+122238us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:06.899989 (+ 56581us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:06.946039 (+ 46050us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:07.088093 (+142054us) log.cc:844] Serialized 26 byte log entry
0709 12:18:07.735975 (+647882us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:07.736022 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:07.736101 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:08.157730 (+421629us) raft_consensus.cc:1685] finished
0709 12:18:08.157752 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:08.168445 (+ 10693us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6035,"raft.queue_time_us":36244,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":46177,"thread_start_us":315983,"threads_started":1,"wal-append.queue_time_us":551439}
I20260709 12:18:08.293979  9480 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:08.379992  9466 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:08.380187  9480 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:08.363127  8839 raft_consensus.cc:3055] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:08.380722  9466 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:08.381528  8839 raft_consensus.cc:740] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:08.362792  9456 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:08.293722  9440 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:08.369939  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:08.370167  6577 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:08.371201  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:08.371493  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:08.373379  9260 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:08.373804  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:08.376433  9216 raft_consensus.cc:3055] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:08.378108  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 304) took 1685 ms. Trace:
I20260709 12:18:08.273247  9063 consensus_queue.cc:1048] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.032s
W20260709 12:18:08.380812  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.379647  6509 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:18:08.369057  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.385535  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:08.407040  6581 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:18:08.379382  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 1793) took 1018 ms. Trace:
I20260709 12:18:08.408931  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:08.414861  6575 raft_consensus.cc:2393] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 7: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:08.407982  6507 rpcz_store.cc:276] 0709 12:18:07.361109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:07.470356 (+109247us) service_pool.cc:224] Handling call
0709 12:18:08.325781 (+855425us) spinlock_profiling.cc:243] Waited 80.5 ms on lock 0x7b5400388454. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:08.379321 (+ 53540us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":185177472}
I20260709 12:18:08.409797  6649 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 9.
I20260709 12:18:08.405138  6501 rpcz_store.cc:276] 0709 12:18:06.692944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:06.723840 (+ 30896us) service_pool.cc:224] Handling call
0709 12:18:07.514189 (+790349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:07.519230 (+  5041us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:07.781834 (+262604us) log.cc:844] Serialized 14 byte log entry
0709 12:18:07.903268 (+121434us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:07.903322 (+    54us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:07.903372 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:07.903433 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:08.318666 (+415233us) spinlock_profiling.cc:243] Waited 774 ms on lock 0x7b54001e82d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:08.378030 (+ 59364us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22473,"prepare.queue_time_us":374335,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":23105,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":11948,"spinlock_wait_cycles":1793077760,"thread_start_us":326690,"threads_started":2,"wal-append.queue_time_us":259575}
I20260709 12:18:08.436266  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:08.436419  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:08.421778  9381 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:08.442225  9381 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:08.453102  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:08.455787  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 517700us
I20260709 12:18:08.458081  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:08.458606  6649 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:08.453867  6655 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:08.461912  6580 raft_consensus.cc:1275] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:18:08.463188  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 266251us
I20260709 12:18:08.465804  6473 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:18:08.473348  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 104778us
I20260709 12:18:08.473796  6657 raft_consensus.cc:1275] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:18:08.432245  9474 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:08.476398  6583 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:18:08.414978  9216 raft_consensus.cc:740] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:08.482144  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:08.483291  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:08.483906  6583 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7.
I20260709 12:18:08.487881  7238 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:18:08.491001  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:08.491696  6654 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:18:08.492565  9381 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:18:08.494745  9477 log.cc:927] Time spent T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.130s	user 0.003s	sys 0.000s
W20260709 12:18:08.496718  6548 leader_election.cc:336] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 2.087s (SENT)
I20260709 12:18:08.497215  6548 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:08.498654  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:08.499274  6655 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:18:08.478246  9458 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:08.501055  9458 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:08.502494  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:08.503206  9458 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:08.503850  9458 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:08.504976  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:08.505653  6657 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:08.507604  9466 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:08.463397  9288 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:08.517129  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:08.518110  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:08.509653  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 43931.5us
I20260709 12:18:08.520385  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:08.520962  6507 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:08.522949  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:08.524212  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:08.524888  6502 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
I20260709 12:18:08.527556  9325 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:08.527673  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:08.528448  6652 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:08.470960  9426 consensus_queue.cc:1048] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.066s
I20260709 12:18:08.532984  9216 consensus_queue.cc:260] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:08.534056  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:08.534819  6654 raft_consensus.cc:2393] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 6: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:08.508852  9474 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:08.536803  9474 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:08.543485  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 55679.1us
I20260709 12:18:08.549903  6473 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:08.551685  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:08.539031  6579 pending_rounds.cc:85] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 3
I20260709 12:18:08.552621  6579 pending_rounds.cc:107] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 12:18:08.552918  6579 raft_consensus.cc:2889] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:08.552274  6583 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:08.382511  8839 consensus_queue.cc:260] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:08.552289  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:08.557080  6581 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
I20260709 12:18:08.557958  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:08.559010  6581 raft_consensus.cc:2393] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 8: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:08.561131  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:08.561494  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:08.561648  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:08.563517  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:08.562963  6655 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:18:08.564770  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:08.564795  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:08.538595  9465 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:08.473608  9446 consensus_queue.cc:1048] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.086s
I20260709 12:18:08.567058  6501 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
I20260709 12:18:08.539700  9254 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:08.534090  9216 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:08.485049  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:08.565490  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.553633  6652 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:18:08.560281  9260 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:08.574656  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:08.574252  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 117400us
I20260709 12:18:08.588966  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:08.574563  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:08.572163  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:08.579017  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:08.579130  6653 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:08.598237  6655 raft_consensus.cc:2393] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 7: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:08.580678  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:08.599370  6652 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
W20260709 12:18:08.565402  6622 connection.cc:256] RPC call timeout handler was delayed by -1.3042981000000964s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1512s for a response
W20260709 12:18:08.601850  6622 connection.cc:256] RPC call timeout handler was delayed by -1.2808981000000585s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15380000000000002s for a response
W20260709 12:18:08.602248  6622 connection.cc:256] RPC call timeout handler was delayed by -1.2610981000000265s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15600000000000003s for a response
I20260709 12:18:08.602425  6578 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:18:08.602634  6622 connection.cc:256] RPC call timeout handler was delayed by -1.2538981000000149s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15680000000000002s for a response
W20260709 12:18:08.602965  6622 connection.cc:256] RPC call timeout handler was delayed by -1.2475981000000047s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
I20260709 12:18:08.563243  9288 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:18:08.603252  6622 connection.cc:256] RPC call timeout handler was delayed by -1.2142980999999509s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1612s for a response
I20260709 12:18:08.590086  6473 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:18:08.603901  6622 connection.cc:256] RPC call timeout handler was delayed by -1.2016980999999305s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1626s for a response
I20260709 12:18:08.584308  6507 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
W20260709 12:18:08.604240  6622 connection.cc:256] RPC call timeout handler was delayed by -1.1935980999999174s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1635s for a response
W20260709 12:18:08.604584  6622 connection.cc:256] RPC call timeout handler was delayed by -1.108998100000008s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17290000000000003s for a response
W20260709 12:18:08.604880  6622 connection.cc:256] RPC call timeout handler was delayed by -1.077498099999957s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1764s for a response
W20260709 12:18:08.605159  6622 connection.cc:256] RPC call timeout handler was delayed by -1.0639980999999352s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1779s for a response
I20260709 12:18:08.564463  9474 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.605727  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:08.610515  9480 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:08.611578  9465 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:08.612531  9482 log.cc:927] Time spent T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.250s	user 0.000s	sys 0.002s
I20260709 12:18:08.612830  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:08.640933  7240 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:08.587754  9260 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:08.590539  9440 raft_consensus.cc:697] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
W20260709 12:18:08.622581  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.623127  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:08.646592  6653 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7.
I20260709 12:18:08.628886  6583 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
I20260709 12:18:08.629443  6649 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:08.586517  8839 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:08.636816  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:08.650900  6654 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5.
I20260709 12:18:08.573079  9254 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:08.638058  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:08.643265  6473 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:08.572574  9443 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:08.646966  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.639380  9441 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.672818  9441 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:08.638080  9474 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:08.674064  9254 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:08.675386  9474 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:08.597730  9448 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:08.657920  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:08.649827  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:08.647746  9419 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 12:18:08.663573  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:08.664388  6656 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:18:08.664511  9309 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:08.627975  9447 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 12:18:08.597857  6658 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
W20260709 12:18:08.684166  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.643994  9224 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:08.716369  9288 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:08.716495  9224 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:08.716912  9288 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:08.717063  9224 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:08.717392  9288 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:08.679766  6473 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:08.720455  6473 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:08.720956  9252 raft_consensus.cc:2764] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:18:08.721297  6473 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:08.721696  6650 raft_consensus.cc:2393] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 6: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:08.722535  9426 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:08.722959  9426 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:08.723237  9448 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:08.675386  9441 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:08.724594  6658 raft_consensus.cc:1275] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:18:08.721195  9419 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:08.645850  9479 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:08.709640  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:08.713467  9440 consensus_queue.cc:237] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:08.709349  6657 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:08.715587  9443 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:08.723311  9426 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:08.727797  9470 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:08.728901  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 81446.8us
I20260709 12:18:08.729736  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:08.729974  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 50117.9us
I20260709 12:18:08.730994  6622 leader_election.cc:400] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 5. Current term is 6.
I20260709 12:18:08.731287  6622 leader_election.cc:403] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:18:08.722188  6473 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:08.646538  9260 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:08.732290  6622 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:08.734217  6550 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:08.735767  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:08.736681  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:08.755612  6578 raft_consensus.cc:1275] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:18:08.756090  6583 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 7. Current term is 8. Ops: []
W20260709 12:18:08.766417  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 178314us
I20260709 12:18:08.761309  6550 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:08.760789  6576 raft_consensus.cc:1275] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:18:08.752170  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:08.771968  6656 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:08.732290  7240 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:08.777005  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 44785.2us
I20260709 12:18:08.779639  6475 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:08.782133  6550 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:08.732522  9381 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:08.732735  9309 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:08.787642  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:08.739840  9485 log.cc:927] Time spent T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.375s	user 0.006s	sys 0.005s
I20260709 12:18:08.795068  6657 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:18:08.788581  6475 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:08.783645  6550 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:08.734879  9480 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 5. Current term is 6.
I20260709 12:18:08.798080  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 1805) took 1432 ms. Trace:
I20260709 12:18:08.763224  9481 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:08.734727  9443 raft_consensus.cc:2764] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:18:08.822402  9489 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:18:08.822891  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 90675.5us
I20260709 12:18:08.733265  9337 consensus_queue.cc:1059] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 12:18:08.736589  9252 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:08.733052  9063 consensus_queue.cc:1048] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 12:18:08.734340  9486 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:08.831904  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:08.865991  6500 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:08.833454  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:08.867334  6504 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
I20260709 12:18:08.816622  6509 rpcz_store.cc:276] 0709 12:18:07.365197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:07.552070 (+186873us) service_pool.cc:224] Handling call
0709 12:18:08.379321 (+827251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:08.404949 (+ 25628us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:08.797838 (+392889us) spinlock_profiling.cc:243] Waited 812 ms on lock 0x7b5400388450. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:08.798017 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1866640640}
W20260709 12:18:08.846823  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 63268.7us
I20260709 12:18:08.869881  6550 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:08.870944  6550 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:08.768795  6393 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 3a47d71c34b541b9b2589b4e80413fa2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
W20260709 12:18:08.872620  6393 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 109653us
I20260709 12:18:08.778070  9470 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:08.743283  7265 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:18:08.850070  6622 leader_election.cc:400] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5.
I20260709 12:18:08.880713  6622 leader_election.cc:403] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:18:08.740453  9419 consensus_queue.cc:1048] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.090s
I20260709 12:18:08.795148  9146 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:08.737517  9236 consensus_queue.cc:1059] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.102s
I20260709 12:18:08.819080  9309 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:08.857286  7271 raft_consensus.cc:1240] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:08.825196  9216 raft_consensus.cc:2804] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:08.745553  9254 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:08.824132  9481 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:08.862599  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:08.906371  6502 raft_consensus.cc:2393] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 8: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:08.853827  9399 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:08.826844  9490 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:08.840124  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:08.881464  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:08.824162  9491 consensus_queue.cc:1059] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.090s
I20260709 12:18:08.907850  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:08.918009  6509 raft_consensus.cc:2393] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 8: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:08.901485  9172 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:08.909741  9481 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:08.878643  9470 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:08.914686  6505 raft_consensus.cc:2393] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 5: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:08.879173  9252 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:08.904145  9216 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:08.901310  9464 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:18:08.925125  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 143356us
I20260709 12:18:08.909976  9443 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5.
I20260709 12:18:08.928015  9251 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:08.921262  9172 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:08.945142  6500 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
I20260709 12:18:08.928472  9470 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:08.930413  9216 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:08.996728  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:08.917155  9490 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:08.980996  9172 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:09.023408  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 139951us
W20260709 12:18:09.019785  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 231232us
I20260709 12:18:08.976617  9458 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:09.025918  6475 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:09.025038  6550 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:09.039203  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 201664us
I20260709 12:18:09.040036  6475 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:09.041234  7271 raft_consensus.cc:1275] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:18:09.045897  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:08.998950  9419 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:18:08.993093  9146 raft_consensus.cc:697] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:09.022446  9464 raft_consensus.cc:697] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:09.076946  9236 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
W20260709 12:18:09.090298  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 50390.3us
I20260709 12:18:09.091513  6622 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:09.102066  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:09.113572  6656 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:09.079658  9443 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:09.071815  9504 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:18:09.123522  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 98561.8us
I20260709 12:18:09.139597  6550 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:09.106899  9252 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:18:09.147898  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:09.135107  9381 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:09.156083  6622 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:09.162293  9464 consensus_queue.cc:237] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:09.169217  9490 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:09.170192  9498 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 12:18:09.169756  9481 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:09.145676  9254 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:09.172535  9146 consensus_queue.cc:237] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:09.172791  9443 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:09.179251  9309 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:09.180011  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:09.182363  6584 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.800s	user 0.008s	sys 0.026s
W20260709 12:18:09.209918  6584 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.827s	user 0.008s	sys 0.027s
I20260709 12:18:09.183602  9426 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:09.184070  9420 raft_consensus.cc:3055] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:09.116012  9172 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:09.214125  6622 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:09.177134  9381 raft_consensus.cc:697] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 7 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:09.213418  9480 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:09.215008  9420 raft_consensus.cc:740] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:09.285147  9443 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:18:09.322021  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:09.306878  9490 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:09.324635  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 110580us
I20260709 12:18:09.324541  9490 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:09.252768  9399 raft_consensus.cc:697] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:09.327277  9490 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:09.327431  9399 consensus_queue.cc:237] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:09.330240  9172 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:09.332607  9309 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:09.333366  9381 consensus_queue.cc:237] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:09.335605  9481 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:09.321789  7240 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:09.292855  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 252872us
I20260709 12:18:09.375051  6475 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:09.378904  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 57234.8us
W20260709 12:18:09.367956  9224 consensus_meta.cc:287] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.650s	user 0.010s	sys 0.003s
I20260709 12:18:09.376219  9529 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:09.385336  6474 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:09.393672  6475 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:09.399670  9254 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:09.371604  9480 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 12:18:09.389940  9309 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:09.426241  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:09.427932  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:09.428674  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:09.429924  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:09.430757  6655 raft_consensus.cc:2393] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 6: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:09.429188  6651 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:09.406337  9505 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:18:09.438045  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 111848us
I20260709 12:18:09.388921  9224 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:09.441215  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:09.442512  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:09.434175  9260 consensus_meta.cc:287] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.697s	user 0.011s	sys 0.000s
I20260709 12:18:09.449342  6507 raft_consensus.cc:1240] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:09.427333  9309 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:09.446344  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:09.424538  9443 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:09.447824  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:09.448395  9254 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:09.385305  9420 consensus_queue.cc:260] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:09.450789  6649 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:18:09.436297  9529 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:09.454627  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:09.457424  6622 leader_election.cc:400] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 8. Current term is 9.
I20260709 12:18:09.403386  9216 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:09.459517  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:09.466356  6508 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:09.462323  6622 leader_election.cc:403] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:18:09.465371  6652 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:09.455437  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:09.467367  6503 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:09.436012  9486 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:09.469740  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:18:09.470352  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:09.470433  6653 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:18:09.474514  6652 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 7. Current term is 8. Ops: [7.3-7.3]
I20260709 12:18:09.453722  9233 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:09.455111  9309 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:09.463464  9288 consensus_meta.cc:287] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.743s	user 0.002s	sys 0.011s
I20260709 12:18:09.480266  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:09.482142  6654 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5.
I20260709 12:18:09.460191  9426 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:09.454246  9260 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:09.487000  9526 log.cc:927] Time spent T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:18:09.461432  9254 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:09.494356  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:09.497112  9491 raft_consensus.cc:3055] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:09.498242  9216 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:09.498994  9461 raft_consensus.cc:3055] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:09.485219  9233 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:09.499711  6475 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:09.500056  9309 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:09.500630  9224 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:09.468847  9529 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:09.472308  9420 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:09.475972  9486 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:09.507167  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:09.513448  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:09.549878  6501 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:18:09.549986  6508 raft_consensus.cc:2393] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 7: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:09.528258  9491 raft_consensus.cc:740] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:09.508431  9288 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:09.553730  6500 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 11: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:09.556790  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:09.557915  9545 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 8. Current term is 9.
I20260709 12:18:09.526132  6504 raft_consensus.cc:2376] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5. Re-sending same reply.
I20260709 12:18:09.460521  9443 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:09.539717  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:09.547659  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:09.559592  6655 raft_consensus.cc:2393] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 8: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:09.561838  6653 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 11.
I20260709 12:18:09.517499  6503 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 4.
I20260709 12:18:09.529908  9461 raft_consensus.cc:740] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:09.536285  9426 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:09.536273  9505 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:09.573462  7240 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:09.579341  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 127077us
I20260709 12:18:09.579246  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:09.580325  6653 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:09.587002  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:09.587894  6501 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:18:09.578728  9450 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:09.591105  9450 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:09.591485  9450 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:09.592047  9260 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:09.596539  6650 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 7 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 6. Current term is 7. Ops: []
W20260709 12:18:09.599459  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 110876us
I20260709 12:18:09.604964  6653 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:18:09.611323  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:18:09.618757  9511 log.cc:927] Time spent T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.496s	user 0.007s	sys 0.007s
I20260709 12:18:09.619664  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:09.599617  9523 log.cc:927] Time spent T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
I20260709 12:18:09.581921  9475 consensus_queue.cc:1059] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:18:09.583693  9426 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:09.630416  9528 log.cc:927] Time spent T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 12:18:09.631045  9529 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:09.631754  9491 consensus_queue.cc:260] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:09.633188  9491 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:09.633549  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:09.634153  6655 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:18:09.637790  9426 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:09.638535  9470 consensus_meta.cc:287] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.629s	user 0.012s	sys 0.000s
I20260709 12:18:09.638929  9470 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:09.614431  9443 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:09.644503  6653 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:09.674238  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:09.675233  6500 raft_consensus.cc:2368] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11.
I20260709 12:18:09.677260  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:09.673539  6653 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7.
I20260709 12:18:09.679311  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:09.679919  6502 raft_consensus.cc:2393] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 8: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:09.681162  6655 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:09.691515  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:09.692471  6500 raft_consensus.cc:2410] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:18:09.692976  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:09.694687  6657 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:09.695457  7265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38558 (request call id 447) took 1055 ms. Trace:
I20260709 12:18:09.694115  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:09.695847  7265 rpcz_store.cc:276] 0709 12:18:08.639858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:08.640670 (+   812us) service_pool.cc:224] Handling call
0709 12:18:09.695250 (+1054580us) spinlock_profiling.cc:243] Waited 26.4 ms on lock 0x7b54000a50d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:09.695376 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":60685184}
I20260709 12:18:09.706234  9470 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:09.706459  9458 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:09.707182  9458 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:09.707546  9233 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:09.685990  9309 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:09.711194  6622 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:09.726465  6657 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 9.
I20260709 12:18:09.728150  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:09.731526  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:09.732985  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:09.733844  6500 raft_consensus.cc:2376] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6. Re-sending same reply.
I20260709 12:18:09.737102  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:09.737798  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:09.738202  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:09.738692  7269 raft_consensus.cc:2368] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7.
I20260709 12:18:09.738970  6500 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7.
I20260709 12:18:09.739944  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:09.740675  6623 leader_election.cc:400] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7.
I20260709 12:18:09.741009  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:09.741882  6500 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 10: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:09.742714  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:09.743320  6500 raft_consensus.cc:2376] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8. Re-sending same reply.
I20260709 12:18:09.744215  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:09.744843  6500 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:09.745051  9486 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:09.745805  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:09.746968  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:09.747542  6500 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:09.748395  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:09.749565  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:09.750684  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:09.751396  9383 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:09.751431  6500 raft_consensus.cc:2410] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:18:09.752701  6501 raft_consensus.cc:2410] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 8 because replica has last-logged OpId of term: 7 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:18:09.752941  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:09.754241  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:09.754853  6500 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:09.758975  9450 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:09.726303  9426 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:09.741009  6623 leader_election.cc:403] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:18:09.760638  9426 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:09.762099  6500 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:09.763111  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:09.763689  6500 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:18:09.763934  9426 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:09.764492  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:09.751829  9420 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:09.768086  6475 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:09.765076  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:09.769253  6475 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:09.769428  6506 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:09.765076  6500 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:09.737799  6503 raft_consensus.cc:2376] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5. Re-sending same reply.
I20260709 12:18:09.772675  9450 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:09.773624  9500 log.cc:927] Time spent T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.842s	user 0.000s	sys 0.013s
I20260709 12:18:09.771849  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:09.774854  9233 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:18:09.777789  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:09.780519  6583 raft_consensus.cc:1240] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:18:09.780546  6506 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:18:09.781922  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:09.782236  6503 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:09.782486  6506 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:09.783322  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:09.783519  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:09.783665  9254 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:09.783828  6506 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:18:09.784617  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1644) took 3141 ms. Trace:
I20260709 12:18:09.784749  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:09.785327  6506 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:09.784986  6584 rpcz_store.cc:276] 0709 12:18:06.642931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:07.372662 (+729731us) service_pool.cc:224] Handling call
0709 12:18:08.328791 (+956129us) spinlock_profiling.cc:243] Waited 34.4 ms on lock 0x7b5800116198. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:18:08.381511 (+ 52720us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:08.381801 (+   290us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:08.381829 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:08.382305 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 12:18:09.210454 (+828149us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:09.210513 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:09.210593 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:09.773038 (+562445us) spinlock_profiling.cc:243] Waited 7.51 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:18:09.784399 (+ 11361us) raft_consensus.cc:1685] finished
0709 12:18:09.784419 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:09.784537 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":96336896,"thread_start_us":827686,"threads_started":1,"wal-append.queue_time_us":549056}
I20260709 12:18:09.787194  9508 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:18:09.787298  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 647721us
I20260709 12:18:09.788131  9508 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:09.788600  9508 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:09.790556  9508 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:09.790984  9508 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:09.791409  9508 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:09.793437  9524 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:09.793844  9524 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:09.794221  9524 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:09.797077  9525 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:09.797610  9525 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:09.798774  9517 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:09.799268  9517 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:09.799697  6584 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:18:09.800451  6584 raft_consensus.cc:606] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:09.800647  6550 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:09.800510  9481 consensus_queue.cc:1048] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:18:09.796105  6500 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:09.813334  6579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1653) took 1276 ms. Trace:
I20260709 12:18:09.813426  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:09.814313  6657 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 11: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:09.801976  6550 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:09.801618  6503 raft_consensus.cc:2376] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6. Re-sending same reply.
I20260709 12:18:09.803877  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:09.820436  9399 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:09.771065  9529 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:09.825210  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:09.826120  6657 raft_consensus.cc:2393] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 8: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:09.801319  9513 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:09.827380  9513 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:09.828724  9426 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:09.829288  9426 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:09.830549  9513 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:09.831423  9426 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:09.833645  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:09.820377  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 48848.5us
I20260709 12:18:09.834373  6503 raft_consensus.cc:2393] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 6: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:09.835119  6475 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:09.841977  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:09.833451  6550 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:09.841768  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:09.813784  6579 rpcz_store.cc:276] 0709 12:18:08.537114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:08.538513 (+  1399us) service_pool.cc:224] Handling call
0709 12:18:08.538838 (+   325us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:08.569388 (+ 30550us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:08.569429 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:08.570121 (+   692us) log.cc:844] Serialized 52 byte log entry
0709 12:18:08.766187 (+196066us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:08.974292 (+208105us) log.cc:844] Serialized 14 byte log entry
0709 12:18:09.087820 (+113528us) log.cc:844] Serialized 14 byte log entry
0709 12:18:09.765733 (+677913us) spinlock_profiling.cc:243] Waited 4.71 ms on lock 0x7b1800439480. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d16c0a 00007f287c456219 00007f2872d9ea40 00007f2872d994e4 00007f2872d99455 00007f2875e81060 00007f2875e93716 00007f2875ea4287 00007f2875ea4208
0709 12:18:09.765938 (+   205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:09.765989 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:09.813106 (+ 47117us) raft_consensus.cc:1685] finished
0709 12:18:09.813130 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:09.813277 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":690676,"prepare.queue_time_us":325192,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":83,"raft.queue_time_us":823566,"raft.run_cpu_time_us":369,"raft.run_wall_time_us":14536,"spinlock_wait_cycles":10840448,"thread_start_us":441168,"threads_started":3,"wal-append.queue_time_us":243934}
W20260709 12:18:09.842514  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:09.844636  6503 raft_consensus.cc:1240] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:09.848014  6503 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:18:09.849143  9513 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:09.850256  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:09.852447  6506 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:18:09.851943  6500 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:09.857740  9520 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:09.858248  9520 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:09.840302  9518 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:09.862232  9524 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:09.862854  9518 raft_consensus.cc:697] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:09.864313  9461 consensus_queue.cc:260] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:09.840343  9448 consensus_queue.cc:1059] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 12:18:09.865688  9508 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:09.865885  9448 raft_consensus.cc:3055] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:09.866241  9448 raft_consensus.cc:740] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:09.867017  9448 consensus_queue.cc:260] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:09.868059  9448 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:09.868697  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:09.869386  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:09.897284  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:09.870111  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:09.889904  6500 raft_consensus.cc:1240] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 3. Current term is 4. Ops: []
I20260709 12:18:09.880554  6474 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:18:09.870296  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:09.846326  9491 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:09.863780  9518 consensus_queue.cc:237] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:09.872383  9172 consensus_queue.cc:1048] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 12:18:09.900853  9146 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 12:18:09.873405  6577 raft_consensus.cc:1240] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:09.874025  6402 catalog_manager.cc:5697] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 1 to 3, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 3 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
W20260709 12:18:09.878014  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 166893us
I20260709 12:18:09.923192  6622 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:09.878717  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 138192us
I20260709 12:18:09.883626  6584 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:09.882728  6548 leader_election.cc:400] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11.
I20260709 12:18:09.879050  6502 raft_consensus.cc:1275] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:18:09.896675  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:09.926378  6650 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:18:09.802314  9420 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:09.878142  9517 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:09.896675  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:09.878925  9443 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7.
I20260709 12:18:09.937372  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:09.911739  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:09.913717  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:09.896502  9362 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:18:09.947973  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:09.911027  9547 log.cc:927] Time spent T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:18:09.947973  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:09.917244  9491 raft_consensus.cc:697] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:09.846004  9399 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 12:18:09.860261  9520 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:09.865523  9461 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:09.930058  6402 catalog_manager.cc:5697] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 1 to 3, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 3 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:09.901983  9513 raft_consensus.cc:697] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:09.923622  9505 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:09.945709  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:10.005759  6650 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7.
I20260709 12:18:09.945910  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:09.946298  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:09.946959  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:10.009157  6577 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:09.948307  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:09.949539  6656 raft_consensus.cc:2393] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 6: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:09.883471  9486 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:09.901554  9481 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 12:18:09.955212  6622 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:10.022404  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 67269.5us
I20260709 12:18:09.956802  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:10.023494  6505 raft_consensus.cc:2393] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 7: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:18:09.958546  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:10.024682  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:09.899448  9524 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:09.964577  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:09.965545  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:10.031258  6575 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:09.966355  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:10.037492  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:09.991595  9420 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:09.980800  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:09.981072  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:09.986753  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 57064.8us
I20260709 12:18:10.045214  6548 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:09.990332  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:09.991564  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:09.995509  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:09.996769  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:09.874060  9522 consensus_queue.cc:1059] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.108s
I20260709 12:18:09.931891  9426 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.016891  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:10.011770  7238 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:09.994777  9446 consensus_queue.cc:1059] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 12:18:09.963110  9480 raft_consensus.cc:2764] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:18:10.067518  9527 raft_consensus.cc:3055] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:10.067999  9527 raft_consensus.cc:740] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:10.068912  9527 consensus_queue.cc:260] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:10.070691  9527 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:10.020548  6402 catalog_manager.cc:5697] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 2 to 5, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 5 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:10.077349  7240 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:09.975328  6651 raft_consensus.cc:1275] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:10.039983  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:10.005656  9505 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:10.005568  9520 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:10.119225  6576 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:09.981809  9517 consensus_queue.cc:1059] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 12:18:10.054775  7238 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:10.060901  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:10.124626  6584 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:10.060755  9491 consensus_queue.cc:237] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:10.064497  9426 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:10.155886  6576 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:18:10.114362  6578 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:10.051050  6548 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:10.121583  6575 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
W20260709 12:18:10.122244  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 44970.4us
W20260709 12:18:10.098379  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 95472.6us
I20260709 12:18:10.035180  9443 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:10.158927  7240 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:10.117156  9480 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.052266  9524 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:18:10.135820  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 150470us
I20260709 12:18:10.115876  9505 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:10.172775  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:10.171249  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:10.232462  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 1760) took 1409 ms. Trace:
I20260709 12:18:10.235491  6507 rpcz_store.cc:276] 0709 12:18:08.823087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:08.832523 (+  9436us) service_pool.cc:224] Handling call
0709 12:18:08.832930 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:09.449747 (+616817us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.232139 (+782392us) spinlock_profiling.cc:243] Waited 616 ms on lock 0x7b54001e5fd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:10.232383 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1417179904}
I20260709 12:18:10.233206  6504 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 6.
I20260709 12:18:10.177111  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:10.240293  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 63314.8us
I20260709 12:18:10.241971  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:10.246022  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:10.153955  9420 raft_consensus.cc:3055] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:10.126691  9216 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:10.251106  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:10.124785  9399 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:10.251611  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:10.261899  6402 catalog_manager.cc:5697] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
W20260709 12:18:10.229056  9498 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
W20260709 12:18:10.276809  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:10.260263  6504 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 11: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:10.290970  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:10.258868  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:10.296418  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:10.297046  6507 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:10.285275  6578 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:18:10.051569  9554 consensus_queue.cc:1048] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.205s
I20260709 12:18:10.228633  9172 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.231904  9443 raft_consensus.cc:697] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 6 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:10.307507  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:10.299507  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:10.309135  6508 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:10.300184  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 249174us
I20260709 12:18:10.311110  6548 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:10.228554  9521 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:10.254146  9420 raft_consensus.cc:740] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:10.305682  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:10.306257  9451 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:10.239261  9480 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:10.306643  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:10.306334  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:10.322839  6502 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:10.306893  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:10.302259  9437 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.307401  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:10.153589  9520 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:10.254355  9545 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:10.277917  9541 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:10.229485  9146 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:10.255107  9490 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.305024  6504 raft_consensus.cc:1275] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:10.314109  6548 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:10.079348  9309 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:10.316079  6584 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7.
W20260709 12:18:10.317435  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 58713.2us
I20260709 12:18:10.254854  9540 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:10.295573  9063 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:18:10.319162  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 264432us
I20260709 12:18:10.321287  6402 catalog_manager.cc:5697] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 3 to 6, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 6 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:10.327513  6507 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:18:10.296000  9216 raft_consensus.cc:697] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 9 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:10.298679  9481 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:10.307468  9519 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:10.346421  9437 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:10.351606  9224 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.228600  9513 consensus_queue.cc:237] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:10.357614  9561 consensus_queue.cc:1059] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 12:18:10.371978  6502 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7.
I20260709 12:18:10.365545  9443 consensus_queue.cc:237] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:10.382448  9451 raft_consensus.cc:697] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:10.289327  9399 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:10.388325  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 216754us
W20260709 12:18:10.384131  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 78091.7us
I20260709 12:18:10.330283  9517 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:18:10.436111  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 48671.6us
W20260709 12:18:10.585615  9461 consensus_meta.cc:287] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.580s	user 0.008s	sys 0.000s
I20260709 12:18:10.310345  9172 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:10.590132  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:10.590543  7271 raft_consensus.cc:2410] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 12:18:10.462488  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 146885us
I20260709 12:18:10.591367  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:10.591892  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:10.511781  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 197766us
I20260709 12:18:10.383637  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:10.539829  6473 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:10.558615  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 399698us
I20260709 12:18:10.457432  9518 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:18:10.562435  6577 consensus_meta.cc:287] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.552s	user 0.002s	sys 0.005s
I20260709 12:18:10.321190  9521 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:10.527441  9381 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:10.570557  9480 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:10.384022  9146 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:10.558845  9063 raft_consensus.cc:697] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 4 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:10.573016  9437 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:10.573387  9216 consensus_queue.cc:237] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:10.575680  6402 catalog_manager.cc:5697] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 2 to 4, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 4 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:10.577036  9399 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:10.532298  9224 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:10.528659  9481 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:10.579341  9458 consensus_meta.cc:287] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.872s	user 0.006s	sys 0.003s
I20260709 12:18:10.582628  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:10.594422  6548 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:10.601531  6508 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 9.
I20260709 12:18:10.602726  6577 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:18:10.608059  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:10.631312  6502 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
I20260709 12:18:10.615962  6550 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:10.619743  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 79993.9us
I20260709 12:18:10.512526  9420 consensus_queue.cc:260] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:10.389082  9490 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:10.633566  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:10.634228  6507 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:10.687390  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:10.695495  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:10.689548  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:10.689631  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 59960.1us
W20260709 12:18:10.689801  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 95478.1us
W20260709 12:18:10.690773  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 74862.7us
I20260709 12:18:10.633215  6473 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:10.688467  9309 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 12:18:10.697170  9519 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:10.697700  9576 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:10.699904  6582 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:10.619344  9446 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:10.642073  9426 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:10.743057  9426 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.743172  9490 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:10.743772  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 309) took 1289 ms. Trace:
I20260709 12:18:10.743530  9426 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:10.743935  9574 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.744952  9574 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:10.749605  9426 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:10.750918  9527 consensus_meta.cc:287] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.679s	user 0.007s	sys 0.006s
I20260709 12:18:10.751637  9426 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.756578  9288 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:10.757218  9522 raft_consensus.cc:3055] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:18:10.757545  9521 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:10.757676  9522 raft_consensus.cc:740] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:10.758599  9522 consensus_queue.cc:260] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:10.759678  9522 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:10.760743  9224 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:10.691386  9561 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:10.761508  9224 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:10.761976  9561 raft_consensus.cc:697] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:10.762976  9561 consensus_queue.cc:237] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:10.764818  9063 consensus_queue.cc:237] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:10.768122  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 384536us
I20260709 12:18:10.770004  9545 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:10.691563  9520 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:10.773512  6622 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:10.774663  6622 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:10.697724  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:10.776551  9533 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:10.777274  9533 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:10.776808  7268 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:18:10.778033  9233 raft_consensus.cc:3055] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:10.778434  9233 raft_consensus.cc:740] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:10.768718  9540 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.779223  9233 consensus_queue.cc:260] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:10.779269  9540 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:10.780584  6622 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:10.781735  9546 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:10.782322  9546 raft_consensus.cc:697] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 7 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:10.782989  9381 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:10.783540  9546 consensus_queue.cc:237] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:10.784776  9550 log.cc:927] Time spent T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 1.084s	user 0.004s	sys 0.003s
I20260709 12:18:10.787201  9508 raft_consensus.cc:993] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e: : Instructing follower f2b06aebe97249a4877d798b8118ece4 to start an election
I20260709 12:18:10.787575  6658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1475) took 1272 ms. Trace:
I20260709 12:18:10.788147  6658 rpcz_store.cc:276] 0709 12:18:09.515602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:09.516141 (+   539us) service_pool.cc:224] Handling call
0709 12:18:09.531912 (+ 15771us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:09.616516 (+ 84604us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:09.616548 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:09.617040 (+   492us) log.cc:844] Serialized 26 byte log entry
0709 12:18:09.678292 (+ 61252us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:18:09.678347 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:09.678451 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:10.784021 (+1105570us) spinlock_profiling.cc:243] Waited 9.65 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:18:10.787152 (+  3131us) raft_consensus.cc:1685] finished
0709 12:18:10.787174 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:10.787512 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10788,"raft.queue_time_us":339784,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"spinlock_wait_cycles":22194048,"thread_start_us":126787,"threads_started":2,"wal-append.queue_time_us":83464}
I20260709 12:18:10.792541  9508 raft_consensus.cc:1081] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Signalling peer f2b06aebe97249a4877d798b8118ece4 to start an election
W20260709 12:18:10.733650  9566 log.cc:927] Time spent T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
W20260709 12:18:10.734696  9564 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
W20260709 12:18:10.735718  9567 log.cc:927] Time spent T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 12:18:10.735461  9446 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:10.714121  9451 consensus_queue.cc:237] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:10.713799  9518 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
W20260709 12:18:10.742152  9569 log.cc:927] Time spent T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.276s	user 0.000s	sys 0.001s
W20260709 12:18:10.741400  9568 log.cc:927] Time spent T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.333s	user 0.000s	sys 0.001s
I20260709 12:18:10.796756  6549 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
W20260709 12:18:10.851516  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 54837.7us
I20260709 12:18:10.853045  9519 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:10.853495  9519 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:10.853904  9519 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:10.854490  9545 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:10.799470  6500 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
I20260709 12:18:10.856778  9146 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:10.796716  9574 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:10.860314  9533 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:10.715260  6548 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:10.783548  9381 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:10.768136  9172 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:10.803570  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:10.804271  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:10.804100  6402 catalog_manager.cc:5697] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 2 to 4, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 4 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:10.808157  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:10.809969  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:10.653417  9458 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:10.812685  9505 consensus_meta.cc:287] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.558s	user 0.009s	sys 0.002s
I20260709 12:18:10.782289  9540 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:10.877785  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:10.813746  7238 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:10.880023  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 66333.7us
I20260709 12:18:10.881033  7238 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:10.744170  6509 rpcz_store.cc:276] 0709 12:18:09.454487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:09.454588 (+   101us) service_pool.cc:224] Handling call
0709 12:18:09.459602 (+  5014us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:09.579418 (+119816us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:09.579464 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:09.587217 (+  7753us) log.cc:844] Serialized 26 byte log entry
0709 12:18:09.679133 (+ 91916us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:09.703142 (+ 24009us) log.cc:844] Serialized 14 byte log entry
0709 12:18:09.703538 (+   396us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:10.062443 (+358905us) spinlock_profiling.cc:243] Waited 4.26 ms on lock 0x7b180049be20. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c522 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:18:10.742877 (+680434us) spinlock_profiling.cc:243] Waited 5.86 ms on lock 0x7b180049be20. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d6a2a6 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419
0709 12:18:10.743488 (+   611us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b54003d3bd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:10.743532 (+    44us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:10.743575 (+    43us) raft_consensus.cc:1685] finished
0709 12:18:10.743590 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:10.743717 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":12510,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":50,"raft.queue_time_us":4827,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":18701,"spinlock_wait_cycles":277586304,"thread_start_us":85914,"threads_started":1,"wal-append.queue_time_us":155720,"wal-append.run_cpu_time_us":2210,"wal-append.run_wall_time_us":1003147}
I20260709 12:18:10.821750  6507 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
I20260709 12:18:10.827754  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:10.830065  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:10.885066  6653 raft_consensus.cc:2410] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:18:10.833650  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:10.834898  6508 raft_consensus.cc:1275] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:18:10.780257  9233 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:10.835690  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:10.838511  6654 raft_consensus.cc:1275] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:18:10.839300  6658 raft_consensus.cc:1275] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:18:10.752183  9426 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:10.853955  9446 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:10.783389  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:10.895143  6550 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:10.841468  9420 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:10.873617  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 158418us
I20260709 12:18:10.904743  6548 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:10.906028  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 122711us
I20260709 12:18:10.896869  6651 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:10.911259  6622 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:10.872021  9527 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:10.916484  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:10.916858  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:10.925117  6548 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:10.923635  7238 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:10.929466  9450 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:10.932260  9533 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:10.933043  9533 raft_consensus.cc:697] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 6 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:10.934412  9533 consensus_queue.cc:237] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:10.934716  9450 consensus_queue.cc:1059] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:10.935752  9572 raft_consensus.cc:3055] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:10.936146  9572 raft_consensus.cc:740] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:10.937052  9572 consensus_queue.cc:260] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:10.938406  9572 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:10.938607  6622 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:10.939980  9450 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:18:10.945422  9486 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:10.946950  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:10.950542  6548 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:10.952824  6548 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:10.963603  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:10.964282  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:10.964313  6508 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:10.964900  9576 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:10.965140  6507 raft_consensus.cc:2410] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:18:10.964411  6548 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:10.915896  6654 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:58980
I20260709 12:18:10.977200  6654 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:10.977617  6654 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:10.981488  9527 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:10.983520  9426 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:10.985224  9481 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:10.914999  9475 raft_consensus.cc:2804] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:10.986050  9475 raft_consensus.cc:697] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 8 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:10.986968  9475 consensus_queue.cc:237] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:10.988394  9146 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:10.988538  9541 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:10.912519  9443 consensus_queue.cc:1059] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.077s
I20260709 12:18:10.990332  9443 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:10.990478  9480 raft_consensus.cc:3055] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:18:10.991173  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:10.991140  9480 raft_consensus.cc:740] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:10.908592  9570 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:10.991954  9570 raft_consensus.cc:697] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:10.992223  9480 consensus_queue.cc:260] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:10.992786  9570 consensus_queue.cc:237] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:10.994004  9480 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:10.920534  9383 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:10.995916  9383 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:10.996294  9383 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:10.914535  9508 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:10.913802  9524 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:11.003508  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:11.004503  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:10.912842  9540 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:11.023077  6654 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:11.033805  9458 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:11.035981  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:10.950906  9580 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:10.990816  9443 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:11.014237  9486 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:18:11.049754  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 363619us
I20260709 12:18:11.050709  6548 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:10.889097  9505 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:11.057368  6474 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:10.991137  9448 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:10.961009  9520 raft_consensus.cc:2764] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
W20260709 12:18:11.062299  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 361561us
I20260709 12:18:10.917555  9521 consensus_queue.cc:1048] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.126s
I20260709 12:18:11.074765  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:11.031828  9383 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:11.075505  6508 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:11.081209  6654 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:11.089066  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:11.089944  6507 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
I20260709 12:18:11.093566  6548 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:11.098825  7273 raft_consensus.cc:1240] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 4 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 3. Current term is 4. Ops: []
W20260709 12:18:11.105872  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 472712us
I20260709 12:18:11.107394  6473 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:11.117591  6508 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
I20260709 12:18:11.062970  9399 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:11.062520  9448 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:11.127240  6548 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:11.128813  6548 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:11.130321  6548 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:11.062336  9426 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:11.133912  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:11.134073  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 70035.1us
W20260709 12:18:11.134464  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:11.137384  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:11.142495  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:11.145097  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:11.146256  6656 raft_consensus.cc:2410] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:18:11.149379  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:11.150214  7274 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 11: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:11.150341  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:11.151058  6658 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:11.152026  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:11.152540  9574 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:11.153221  9309 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.153731  9309 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:11.157459  9505 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:11.092700  9540 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:11.158565  9508 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:11.158655  9540 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:11.159318  9576 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:11.160048  9437 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:11.161046  9540 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:11.161192  6474 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:11.161640  9534 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:11.162175  9534 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:11.164816  9491 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:11.165697  9448 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:11.169464  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 62187.6us
I20260709 12:18:11.170569  6473 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:11.158202  9585 log.cc:927] Time spent T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.347s	user 0.000s	sys 0.003s
I20260709 12:18:11.088649  9486 raft_consensus.cc:697] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 6 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:18:11.185148  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 54886.1us
I20260709 12:18:11.185549  6473 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:11.185678  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:11.186283  6548 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:11.103040  9443 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:11.063365  9381 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:11.186460  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:11.176373  9518 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.209391  6654 raft_consensus.cc:2410] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 12:18:11.191401  6473 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:11.122792  9524 consensus_queue.cc:1048] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 12:18:11.187841  9529 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:11.189723  9576 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:11.188241  9491 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:11.215713  6473 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:11.205695  9521 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:11.232476  6653 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:11.248840  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:11.250479  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:11.250733  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:11.251365  6575 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:11.251590  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:11.255460  6584 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:18:11.229365  9480 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:11.260389  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:11.211686  9399 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:18:11.229367  9233 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:11.272050  6651 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5.
I20260709 12:18:11.278590  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:11.281635  6508 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:18:11.281741  7240 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:11.278817  6577 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:11.284606  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:11.218051  9518 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:11.255651  9576 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.319728  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:11.250118  9529 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:18:11.337580  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:11.280741  9571 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:11.351850  6473 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:11.286077  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:11.357663  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:11.357955  6581 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:11.358338  6575 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:18:11.320947  9451 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:18:11.368480  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:11.379392  9591 log.cc:927] Time spent T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 12:18:11.364358  9571 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:11.279480  9480 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:11.387709  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:11.388173  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:11.388921  6577 raft_consensus.cc:2368] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7.
I20260709 12:18:11.389951  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:11.390563  6577 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:11.391492  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:11.357313  9574 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.370885  9451 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:11.269762  9486 consensus_queue.cc:237] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:11.426800  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:11.438488  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 86721.3us
I20260709 12:18:11.441303  6473 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:11.326129  9576 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:11.372103  7269 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
I20260709 12:18:11.446686  6577 raft_consensus.cc:2393] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 6: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:11.399574  9525 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:11.464016  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:11.464885  6577 raft_consensus.cc:2393] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 6: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:11.393206  6550 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:11.442692  9593 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:11.436264  9233 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.465596  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:11.473714  6577 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:11.439260  9451 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:11.479755  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:11.466851  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 73720us
I20260709 12:18:11.448436  9575 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.489043  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:11.427649  9508 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 12:18:11.499647  6577 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:18:11.445428  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:11.473227  9589 log.cc:927] Time spent T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.262s	user 0.001s	sys 0.001s
I20260709 12:18:11.469616  9593 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:11.507402  6623 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:11.514896  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:11.436090  9574 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:11.509733  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:11.510454  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 59042.7us
I20260709 12:18:11.522612  6577 raft_consensus.cc:2368] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7.
I20260709 12:18:11.487519  9534 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:11.529461  6583 raft_consensus.cc:2393] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 7: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:11.512665  9575 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:11.530485  6473 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:11.535310  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:11.536201  6577 raft_consensus.cc:3055] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:11.536996  6577 raft_consensus.cc:740] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:11.538565  6651 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:18:11.543520  6473 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:11.546631  6473 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:11.533944  9590 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:11.554991  9590 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:11.555527  9590 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:11.556011  9590 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:11.487847  9233 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:11.521319  9505 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.538523  6577 consensus_queue.cc:260] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:11.560693  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:11.527655  9545 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:11.526079  9593 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:11.572461  6577 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:11.588410  6651 raft_consensus.cc:1240] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:11.572468  6583 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:11.535352  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:11.594499  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:11.527477  9448 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:11.596920  6506 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5.
I20260709 12:18:11.614471  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:11.628506  6651 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:11.581154  9545 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:11.636782  6577 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
I20260709 12:18:11.650115  6575 raft_consensus.cc:2393] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 6: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:11.580232  9529 consensus_queue.cc:1059] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.055s
I20260709 12:18:11.600713  9518 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:11.652544  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:11.580737  9505 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:11.666216  6577 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
W20260709 12:18:11.673483  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 126695us
I20260709 12:18:11.675887  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:11.651052  9545 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:11.646204  9561 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:11.646664  9576 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:11.685273  6577 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:18:11.686074  6577 raft_consensus.cc:606] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:11.686496  6577 raft_consensus.cc:614] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:18:11.663204  9426 consensus_meta.cc:287] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.502s	user 0.011s	sys 0.000s
W20260709 12:18:11.695865  9608 log.cc:927] Time spent T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 12:18:11.695925  9618 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:11.696720  9620 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:11.697577  9598 log.cc:927] Time spent T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:18:11.698112  9529 raft_consensus.cc:3055] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:11.698505  9529 raft_consensus.cc:740] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:11.699352  9529 consensus_queue.cc:260] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:11.700367  9571 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:11.703742  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:11.705410  9601 log.cc:927] Time spent T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
W20260709 12:18:11.707125  9612 log.cc:927] Time spent T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
W20260709 12:18:11.708317  9602 log.cc:927] Time spent T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:18:11.708381  9554 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:11.711062  9383 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:11.712697  6652 raft_consensus.cc:1275] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 12:18:11.716547  6581 raft_consensus.cc:1240] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:11.717883  6581 raft_consensus.cc:1275] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
W20260709 12:18:11.680938  9522 consensus_meta.cc:287] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.917s	user 0.010s	sys 0.004s
I20260709 12:18:11.666605  9592 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:18:11.739513  9519 consensus_meta.cc:287] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.885s	user 0.009s	sys 0.000s
I20260709 12:18:11.740298  9491 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:11.742188  6576 raft_consensus.cc:1275] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:18:11.688023  9621 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.689597  9561 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:11.760578  6576 raft_consensus.cc:1275] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:18:11.761834  6503 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:11.761765  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:11.722407  9426 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:11.762938  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:11.730615  9529 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:11.764322  6506 raft_consensus.cc:1275] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:18:11.766976  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:11.776185  6508 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:18:11.779428  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:11.743961  9592 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:11.780278  6502 raft_consensus.cc:2368] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7.
I20260709 12:18:11.696982  9618 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:11.697235  9620 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:11.709156  9554 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:11.755007  9063 consensus_queue.cc:1059] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:18:11.755445  9475 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 12:18:11.800858  6651 raft_consensus.cc:1275] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:18:11.899083  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:11.764394  9561 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:11.830353  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:11.901544  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:11.756966  9519 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:11.906529  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:11.835397  9613 log.cc:927] Time spent T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
W20260709 12:18:11.886574  9420 consensus_meta.cc:287] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.972s	user 0.007s	sys 0.006s
W20260709 12:18:11.834299  9609 log.cc:927] Time spent T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 12:18:11.815932  9574 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:11.903403  9632 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:18:11.769307  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:11.900882  9540 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.834920  9505 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:11.790879  9592 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:11.947190  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 45760.2us
I20260709 12:18:11.947445  6583 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:11.949031  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:11.952165  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:11.764007  9621 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:11.764047  9491 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:11.834836  9575 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:11.817389  9448 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:11.882669  9383 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:11.815682  9590 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:11.964325  6506 raft_consensus.cc:1275] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:18:11.961218  6622 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:11.974575  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:11.946344  9605 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:11.815692  9593 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:11.904125  9534 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.957753  9486 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:12.004711  9486 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:11.981609  9604 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:12.005221  9486 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:12.005632  9309 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:12.006019  9603 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:11.960614  7269 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:11.965512  9610 log.cc:927] Time spent T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.300s	user 0.000s	sys 0.001s
I20260709 12:18:11.985100  6508 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
W20260709 12:18:11.981890  9546 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.175s	user 0.014s	sys 0.053s
I20260709 12:18:11.983807  6583 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:11.989545  6582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 353) took 1090 ms. Trace:
W20260709 12:18:12.002285  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:11.988859  9464 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:12.011384  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 355) took 1053 ms. Trace:
I20260709 12:18:11.962002  9540 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:11.981755  9605 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:12.041222  6580 rpcz_store.cc:276] 0709 12:18:10.957686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.957841 (+   155us) service_pool.cc:224] Handling call
0709 12:18:10.980817 (+ 22976us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:11.046535 (+ 65718us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:11.046565 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:11.060982 (+ 14417us) log.cc:844] Serialized 26 byte log entry
0709 12:18:11.697466 (+636484us) spinlock_profiling.cc:243] Waited 1.28 ms on lock 0x7b1800133720. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:18:11.832029 (+134563us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:11.832111 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:11.832226 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:12.011057 (+178831us) raft_consensus.cc:1685] finished
0709 12:18:12.011081 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:12.011317 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63111,"raft.queue_time_us":69001,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":4654,"spinlock_wait_cycles":2950400,"thread_start_us":133627,"threads_started":1,"wal-append.queue_time_us":875121}
I20260709 12:18:11.976897  9383 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:12.047945  6576 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 12:18:12.048558  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 562746us
I20260709 12:18:12.049503  6579 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:18:12.052054  6576 raft_consensus.cc:1275] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 12:18:12.017561  9546 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.211s	user 0.014s	sys 0.054s
I20260709 12:18:12.017812  6582 rpcz_store.cc:276] 0709 12:18:10.899563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:10.899745 (+   182us) service_pool.cc:224] Handling call
0709 12:18:10.900012 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:10.900283 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:11.211243 (+310960us) log.cc:844] Serialized 14 byte log entry
0709 12:18:11.987620 (+776377us) spinlock_profiling.cc:243] Waited 5.09 ms on lock 0x7b18002930e0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d16c0a 00007f287c456219 00007f2872d9ea40 00007f2872d994e4 00007f2872d99455 00007f2875e81060 00007f2875e93716 00007f2875ea4287 00007f2875ea4208
0709 12:18:11.987946 (+   326us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:11.988021 (+    75us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:11.988091 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:11.988192 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:11.989455 (+  1263us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":131081,"prepare.queue_time_us":581972,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":59,"raft.queue_time_us":600203,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":3641,"spinlock_wait_cycles":11712256,"thread_start_us":464171,"threads_started":2,"wal-append.queue_time_us":244981}
I20260709 12:18:12.062592  9618 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:12.063568  9628 log.cc:927] Time spent T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
W20260709 12:18:12.065275  9627 log.cc:927] Time spent T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
W20260709 12:18:12.068608  9599 log.cc:927] Time spent T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260709 12:18:12.069311  9590 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:12.069480  9519 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:12.070288  9634 log.cc:927] Time spent T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:18:12.006669  9603 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:12.012032  9534 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:12.083460  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:12.049290  9605 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:12.095045  9624 log.cc:927] Time spent T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
I20260709 12:18:12.089313  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:12.091106  9534 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:12.100692  6656 raft_consensus.cc:1275] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:18:12.100684  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:12.115620  6651 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:12.101325  9172 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:12.104095  7269 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:12.106379  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:12.088267  6549 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:12.110436  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 132912us
I20260709 12:18:12.112948  6657 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:12.118049  6622 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:12.119980  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:12.070783  9521 consensus_queue.cc:1059] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:18:12.120326  9383 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:12.124279  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:12.121290  9172 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:12.134886  9592 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:12.157987  6505 raft_consensus.cc:2393] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 7: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:12.122421  9399 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:18:12.146535  9637 log.cc:927] Time spent T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:18:12.157586  9546 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:12.157944  9645 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:12.146952  9620 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:12.191541  9491 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:12.135032  9648 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:18:12.192826  9645 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:12.207858  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:12.208796  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:12.149830  9592 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:12.215009  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:12.227370  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 263762us
W20260709 12:18:12.194868  9646 log.cc:927] Time spent T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:18:12.214223  6576 raft_consensus.cc:1275] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:18:12.234524  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 114612us
I20260709 12:18:12.216907  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:12.274817  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:12.275766  7271 raft_consensus.cc:2368] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7.
I20260709 12:18:12.276408  6622 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:12.222913  9451 consensus_meta.cc:287] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.714s	user 0.013s	sys 0.001s
I20260709 12:18:12.223824  9629 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:12.294261  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 60687.7us
I20260709 12:18:12.166038  9604 raft_consensus.cc:697] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 7 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:12.223352  9426 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:12.296994  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:12.297334  9451 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:12.302462  9561 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:12.303449  9653 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:12.303092  9448 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:12.304666  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:12.304687  6575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1673) took 1036 ms. Trace:
I20260709 12:18:12.329952  6575 rpcz_store.cc:276] 0709 12:18:11.268247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:11.712658 (+444411us) service_pool.cc:224] Handling call
0709 12:18:11.712969 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:12.016819 (+303850us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:12.016850 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:12.017487 (+   637us) log.cc:844] Serialized 26 byte log entry
0709 12:18:12.111374 (+ 93887us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:12.111415 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.111485 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:12.304464 (+192979us) raft_consensus.cc:1685] finished
0709 12:18:12.304484 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:12.304623 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":319063,"raft.queue_time_us":304408,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":25113,"thread_start_us":42539,"threads_started":1,"wal-append.queue_time_us":117488}
I20260709 12:18:12.285663  9521 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.307740  6657 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:12.310441  9233 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:12.310640  9486 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:12.311873  9063 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:12.327674  6580 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:18:12.342471  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:12.353013  6473 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:12.353260  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:12.353117  6653 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:18:12.353912  6652 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:18:12.345358  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:12.348709  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:12.326326  9545 consensus_meta.cc:287] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.596s	user 0.011s	sys 0.000s
I20260709 12:18:12.354158  6473 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:12.365430  6657 raft_consensus.cc:1275] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:12.342878  6548 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:12.368268  6473 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:12.359537  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:12.369700  6653 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:12.372197  9518 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:12.233247  9605 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:12.376518  7269 raft_consensus.cc:2393] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 6: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:12.378340  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:12.379681  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:12.329548  9524 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.335143  9514 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.328791  9653 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:12.389024  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:12.389591  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:12.396399  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 109009us
W20260709 12:18:12.396165  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 186368us
I20260709 12:18:12.397243  6474 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:12.365124  9446 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.378196  9592 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:12.416589  6509 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:18:12.351439  9172 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:12.394913  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 118427us
I20260709 12:18:12.329190  9426 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:12.368767  6548 leader_election.cc:400] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7.
W20260709 12:18:12.425393  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 56849.3us
I20260709 12:18:12.364692  9288 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 12:18:12.426582  6548 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:12.345168  9448 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:12.397876  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:12.382053  9420 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:12.413049  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 1539) took 1139 ms. Trace:
I20260709 12:18:12.418430  6507 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:18:12.421538  6622 leader_election.cc:400] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7.
I20260709 12:18:12.435894  6583 rpcz_store.cc:276] 0709 12:18:11.273933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:11.613862 (+339929us) service_pool.cc:224] Handling call
0709 12:18:12.412820 (+798958us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b54000352d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:12.412975 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":762809088}
I20260709 12:18:12.437641  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:12.394867  6653 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:12.409505  6475 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:12.364939  9458 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 12:18:12.357197  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:12.395262  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:12.421389  9529 consensus_meta.cc:287] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.641s	user 0.012s	sys 0.000s
I20260709 12:18:12.429893  6548 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:12.431314  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:12.352471  9521 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:12.411585  9618 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:12.373920  9651 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 12:18:12.420006  9571 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:12.395375  9505 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:12.439440  6505 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 6.
I20260709 12:18:12.439411  6650 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:12.384228  9520 consensus_queue.cc:1048] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 12:18:12.312903  9629 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:12.413753  9592 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:12.480721  6622 leader_election.cc:400] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7.
I20260709 12:18:12.500658  6622 leader_election.cc:403] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:12.414016  9590 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:12.480806  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 50347.2us
I20260709 12:18:12.396441  9653 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:12.492200  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 62382.2us
I20260709 12:18:12.492733  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:12.509431  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:12.510416  6507 raft_consensus.cc:2393] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 6: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:12.418318  9446 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:12.396770  9524 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:12.377508  9545 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:12.498944  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:12.499549  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:12.442641  9420 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:12.434082  6652 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:12.505087  9592 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:12.480475  9437 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:12.496513  9508 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:12.433465  9620 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.074s
I20260709 12:18:12.491969  9517 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 12:18:12.497156  9618 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:12.478677  9574 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:12.570533  9420 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:12.572144  6505 raft_consensus.cc:1275] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:18:12.577095  6508 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:18:12.515152  9443 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:12.558369  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:12.482239  9575 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:12.615041  6507 raft_consensus.cc:2393] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 9: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:12.424177  9514 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:12.589382  6502 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:18:12.511507  9571 raft_consensus.cc:697] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:18:12.612803  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 119703us
I20260709 12:18:12.500882  9491 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
I20260709 12:18:12.575899  9172 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.519274  9590 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:12.576735  9619 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:12.622967  6507 raft_consensus.cc:1240] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7. Ops: []
W20260709 12:18:12.617594  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 136987us
I20260709 12:18:12.625385  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:12.566592  9660 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 12:18:12.617018  9575 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:12.631325  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:12.627071  6622 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:12.493777  9604 consensus_queue.cc:237] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:12.636144  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:12.611349  9574 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:12.620448  9519 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:12.620253  9451 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:12.639578  9659 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7.
I20260709 12:18:12.641078  9603 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:12.644963  9521 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:12.645104  9480 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:12.646015  9645 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:12.646308  6507 raft_consensus.cc:1275] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:18:12.690852  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:12.690428  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:12.696755  6652 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:12.629725  9605 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:12.647909  9514 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:12.631697  9443 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:18:12.650234  9662 log.cc:927] Time spent T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 12:18:12.653143  9524 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:12.653707  9508 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:18:12.655768  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 99772.2us
I20260709 12:18:12.707396  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:12.707808  6550 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:12.709170  7238 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:12.709904  6550 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:12.711810  9651 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:12.712047  9451 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:12.704406  9443 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:12.713276  9651 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:12.713726  9651 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:12.626540  9561 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:12.714085  9651 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:12.663645  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:12.626368  9630 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.659561  6475 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:12.667369  9448 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:12.668177  9381 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:12.668535  9629 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:12.669446  9650 log.cc:927] Time spent T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.243s	user 0.000s	sys 0.000s
I20260709 12:18:12.671438  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:12.626459  9172 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:12.627221  9619 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:12.678104  9590 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:12.662308  9519 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:12.637143  9575 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:12.631186  9486 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:12.681100  9426 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:12.682061  9545 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:12.684691  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:12.646983  9571 consensus_queue.cc:237] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:12.697206  6650 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:12.691099  9546 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 12:18:12.713351  9443 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:12.646054  9446 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:12.715588  6550 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:12.667543  9654 log.cc:927] Time spent T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 12:18:12.730454  9629 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.689028  9669 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:12.719205  9524 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:12.760617  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:12.765791  9172 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:12.727504  9668 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:12.767998  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1676) took 1443 ms. Trace:
I20260709 12:18:12.784487  9671 consensus_queue.cc:1048] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.124s
I20260709 12:18:12.720831  9630 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:12.799255  9619 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:12.734184  9448 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.830217  9448 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:12.762342  6654 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:12.832224  9448 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:12.812690  9420 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:12.834982  9651 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:12.734465  9576 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:12.812374  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:12.803478  9575 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:12.816661  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:12.817696  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:12.814461  9545 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:18:12.821322  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 105851us
I20260709 12:18:12.855760  6550 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:12.820912  6577 rpcz_store.cc:276] 0709 12:18:11.324207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:11.736948 (+412741us) service_pool.cc:224] Handling call
0709 12:18:11.737307 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:12.017789 (+280482us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:12.017824 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:12.041372 (+ 23548us) log.cc:844] Serialized 26 byte log entry
0709 12:18:12.188824 (+147452us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:12.329544 (+140720us) log.cc:844] Serialized 14 byte log entry
0709 12:18:12.330010 (+   466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:12.330089 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:12.767688 (+437599us) raft_consensus.cc:1685] finished
0709 12:18:12.767719 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:12.767918 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":280098,"prepare.queue_time_us":2694,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":304293,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":7796,"thread_start_us":264925,"threads_started":2,"wal-append.queue_time_us":289957}
I20260709 12:18:12.795888  9592 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:12.824942  9668 raft_consensus.cc:697] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:12.825198  9437 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:12.829628  9630 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:12.834012  9653 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:12.836030  7265 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
I20260709 12:18:12.838971  9522 raft_consensus.cc:3055] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:12.842286  9519 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:12.806608  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 168735us
W20260709 12:18:12.721367  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 61860.6us
I20260709 12:18:12.813858  9540 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:12.821249  9629 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:12.858295  9545 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:12.859773  9575 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:12.865906  6550 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:12.912181  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 46309.4us
I20260709 12:18:12.913097  6550 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:12.872925  9668 consensus_queue.cc:237] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:12.881395  6623 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:12.885463  9621 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:12.887238  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:12.888594  9653 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:12.850622  9576 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:12.890687  9420 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:12.866895  9508 consensus_queue.cc:1059] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 12:18:12.894887  6504 raft_consensus.cc:2393] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 9: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:12.895951  9592 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:12.896288  9522 raft_consensus.cc:740] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:12.930289  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:12.886498  9443 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:12.931417  9545 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:12.932438  9443 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:12.919450  9437 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:12.899113  9651 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:12.865140  9671 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:12.933365  9437 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:12.933759  9671 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:12.924799  9540 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:12.934280  9671 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:12.934686  9540 raft_consensus.cc:697] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 6 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:12.904778  6475 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:12.920284  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:12.903494  6622 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:12.929639  9629 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:12.897362  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:12.933825  9437 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:12.952593  9621 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.953181  9621 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:12.953519  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 48826.7us
I20260709 12:18:12.954614  6475 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:12.955317  9621 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:12.955866  9621 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:12.910898  9546 consensus_queue.cc:1048] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 12:18:12.956532  9619 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:12.957003  9619 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:12.957363  9619 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:12.919261  9448 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:12.958336  9448 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:12.958699  9448 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:12.958580  9446 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:12.918537  9519 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:12.943204  6622 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:12.943992  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:12.949255  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:12.964543  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:12.966114  6502 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 1865) took 1042 ms. Trace:
I20260709 12:18:12.969453  9546 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:12.936070  9540 consensus_queue.cc:237] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:12.971782  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:12.966835  6502 rpcz_store.cc:276] 0709 12:18:11.924229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:11.924645 (+   416us) service_pool.cc:224] Handling call
0709 12:18:12.965916 (+1041271us) spinlock_profiling.cc:243] Waited 536 ms on lock 0x7b54000f3554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:12.966048 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1233789184}
I20260709 12:18:12.979280  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:12.980000  9651 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:12.981515  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:12.986223  9172 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.986304  9518 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:12.986868  9172 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:12.987011  9518 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:12.989060  9172 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:12.989442  9518 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:12.987152  9514 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:12.992722  9574 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:12.993530  9554 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:12.992733  9514 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:12.999660  9514 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:13.000705  6501 raft_consensus.cc:2410] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 9 because replica has last-logged OpId of term: 7 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:18:13.000707  9574 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:13.002478  9574 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:13.005689  9649 log.cc:927] Time spent T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.720s	user 0.002s	sys 0.002s
I20260709 12:18:13.006397  6583 raft_consensus.cc:1275] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:18:13.006563  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:13.008165  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:13.008519  9593 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:13.008886  6577 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
I20260709 12:18:13.009124  9437 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:13.010902  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:13.013634  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:13.014540  6577 raft_consensus.cc:2393] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 7: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:13.015987  9618 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:13.016489  9619 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:13.017009  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:13.028610  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:13.029307  6501 raft_consensus.cc:2393] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 6: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:13.029960  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:13.030208  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:13.030578  6579 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:13.031478  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:13.017275  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:13.032169  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:13.032888  6576 raft_consensus.cc:2393] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 6: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:13.020005  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:13.033900  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:13.035121  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:13.035897  6501 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 11: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:13.036049  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:13.036825  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:13.037182  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:13.038123  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:13.039340  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:13.020105  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:13.022086  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:13.060789  6653 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:13.024047  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:13.026503  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:13.061653  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:13.062245  6504 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:13.026906  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:13.027459  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:13.040628  6508 raft_consensus.cc:1240] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:18:13.018647  9437 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:13.043582  6579 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:18:13.018532  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:13.052066  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:13.052564  6503 raft_consensus.cc:1275] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:18:12.937347  9576 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:12.961462  9519 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:13.056247  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:13.057164  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:13.071955  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:13.023846  6509 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:18:13.064061  6577 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:13.065237  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:12.962024  9545 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:12.956276  9621 raft_consensus.cc:697] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:13.074126  6503 raft_consensus.cc:2410] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:18:13.060847  6580 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
I20260709 12:18:13.077126  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:13.080596  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:13.045255  9464 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.083590  6653 raft_consensus.cc:2393] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 9: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:13.089548  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:13.091009  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:13.079650  9524 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.110481  9524 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:13.110765  9621 consensus_queue.cc:237] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:13.112838  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:13.113655  6650 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:13.070461  6458 thread.cc:641] client-negotiator [worker] (thread pool) Time spent creating pthread: real 0.762s	user 0.018s	sys 0.025s
I20260709 12:18:13.138321  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:13.140673  6501 raft_consensus.cc:2410] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:18:13.169034  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:13.174618  6622 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:13.175585  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:13.080715  9486 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.173684  6505 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
I20260709 12:18:13.076771  9546 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:18:13.191998  6548 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.881s (SENT) [suppressed 18 similar messages]
I20260709 12:18:13.193486  6505 raft_consensus.cc:1275] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:18:13.191655  6577 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
W20260709 12:18:13.139940  6458 thread.cc:608] client-negotiator [worker] (thread pool) Time spent starting thread: real 0.833s	user 0.019s	sys 0.026s
I20260709 12:18:13.135339  9674 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.094556  9574 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:13.140990  9309 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.211643  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:13.218086  9522 consensus_queue.cc:260] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:13.220572  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:13.051182  9669 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.090s
I20260709 12:18:13.220534  6504 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7.
I20260709 12:18:13.170049  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:13.199234  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:13.242976  6650 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:18:13.171051  9619 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:13.169827  9524 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:13.206610  9682 log.cc:927] Time spent T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:18:13.206254  9545 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:13.244380  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:13.293623  6501 raft_consensus.cc:1240] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:13.353922  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:13.328169  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:13.300506  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:13.184644  9554 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:13.347313  6652 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:13.250623  9645 consensus_queue.cc:1048] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.085s
I20260709 12:18:13.208480  9546 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:13.311502  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:13.135701  9464 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:13.396481  6658 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:18:13.391634  9593 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:18:13.391069  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 67515.4us
W20260709 12:18:13.407395  9679 log.cc:927] Time spent T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.266s	user 0.000s	sys 0.001s
I20260709 12:18:13.372076  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:13.430752  6650 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 11: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:13.289892  9508 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:18:13.404501  9687 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 12:18:13.423871  6652 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:13.444036  6652 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:13.388535  7265 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:18:13.446002  6653 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:13.391027  6475 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:13.369937  9629 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:18:13.266250  9640 log.cc:927] Time spent T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:18:13.396602  9522 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:13.223395  9486 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:13.415014  9655 log.cc:927] Time spent T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:18:13.265321  9480 consensus_queue.cc:1048] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.150s
I20260709 12:18:13.390964  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:13.302598  9309 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:13.448137  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 69896.2us
I20260709 12:18:13.248669  9674 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:13.405478  9659 consensus_queue.cc:1059] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 12:18:13.402192  9461 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:18:13.453953  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:13.454844  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:13.426203  9519 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:13.476256  7238 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:13.487262  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:13.455773  9619 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:13.501896  6501 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:13.420364  9554 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:13.507879  9593 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.511252  6655 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:13.446471  9508 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:13.527339  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 335327us
I20260709 12:18:13.452855  9629 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:13.509021  9643 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:13.505743  9619 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:13.486503  9622 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
W20260709 12:18:13.535996  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 291700us
W20260709 12:18:13.531733  9671 consensus_meta.cc:287] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.597s	user 0.003s	sys 0.009s
I20260709 12:18:13.527726  7268 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:18:13.543035  6577 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:13.551410  6548 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:13.554463  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:13.557308  6577 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:13.541844  9619 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:13.522672  9593 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:13.575613  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:13.546003  9643 raft_consensus.cc:697] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:13.550544  9534 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:13.551326  9695 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:13.550884  9437 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:13.550776  9518 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:18:13.586297  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.581063  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:13.580030  9480 consensus_queue.cc:1048] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 12:18:13.618851  6657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2090) took 1209 ms. Trace:
I20260709 12:18:13.619277  6657 rpcz_store.cc:276] 0709 12:18:12.409309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.409451 (+   142us) service_pool.cc:224] Handling call
0709 12:18:12.409730 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:12.650732 (+241002us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:12.650765 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:12.650819 (+    54us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:12.650855 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:13.618587 (+967732us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b54000758d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:13.618634 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:13.618782 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":553780992}
I20260709 12:18:13.585090  9605 consensus_queue.cc:1059] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
W20260709 12:18:13.588362  9692 log.cc:927] Time spent T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:18:13.597510  9546 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:13.598331  9643 consensus_queue.cc:237] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:13.627040  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:13.627530  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:13.600759  9437 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:13.601225  9554 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:13.601498  9518 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:13.602247  9691 log.cc:927] Time spent T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 12:18:13.556785  9671 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:13.603143  9604 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:13.603861  9674 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:13.604001  9690 log.cc:927] Time spent T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 12:18:13.604707  9694 log.cc:927] Time spent T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:18:13.609083  9695 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:13.611047  9508 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:13.612561  9629 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:13.646811  9570 raft_consensus.cc:3055] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:13.615501  9593 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:13.663980  9309 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:13.621645  6581 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:18:13.667939  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:13.668883  6500 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
I20260709 12:18:13.628043  6549 leader_election.cc:400] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7.
I20260709 12:18:13.670699  6549 leader_election.cc:403] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
W20260709 12:18:13.671551  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 50772.5us
I20260709 12:18:13.672433  6549 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:13.673806  6549 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:13.632488  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:13.632707  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:13.639619  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:13.641115  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.643810  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:13.676652  6584 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:13.677297  6651 raft_consensus.cc:2410] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:18:13.644104  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:13.646100  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:13.677870  6650 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:18:13.678000  6652 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:13.678937  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:13.625828  9520 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:13.653622  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:13.680485  6653 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 7: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:13.680353  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:13.613866  9486 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:13.680644  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:13.625335  9545 consensus_queue.cc:1048] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 12:18:13.681044  6578 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:13.649875  9554 raft_consensus.cc:3055] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:13.661566  9464 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:13.662515  9671 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:13.662672  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:13.656723  6657 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:13.645975  9605 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.699189  6550 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:13.701068  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:13.702001  6577 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:18:13.683017  6549 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:13.685932  6652 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:13.688323  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:13.706904  6549 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:13.692898  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 141552us
I20260709 12:18:13.693127  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:13.655165  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:13.667145  9448 consensus_meta.cc:287] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.708s	user 0.006s	sys 0.003s
I20260709 12:18:13.713112  9448 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:13.687829  9508 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7.
I20260709 12:18:13.687431  9520 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:13.715418  6575 raft_consensus.cc:2393] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 8: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:13.660137  9570 raft_consensus.cc:740] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:13.691305  9554 raft_consensus.cc:740] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:13.712509  7268 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:13.694332  9519 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:13.710358  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:13.726523  6578 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:13.728564  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:13.728556  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:13.730324  6578 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:13.732692  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:13.734433  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:13.718508  9520 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:13.706264  9464 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:13.730317  6584 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:13.736779  7269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38558 (request call id 458) took 1529 ms. Trace:
I20260709 12:18:13.737133  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:13.737406  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:13.737272  7269 rpcz_store.cc:276] 0709 12:18:12.207421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:12.216585 (+  9164us) service_pool.cc:224] Handling call
0709 12:18:13.736553 (+1519968us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b54000a1ed4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:13.736708 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":247192320}
I20260709 12:18:13.738073  6577 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:18:13.739269  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:13.740227  6577 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 11: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:13.714879  9695 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:13.705714  9703 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:13.748572  9519 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:13.749368  6577 raft_consensus.cc:1240] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 7. Current term is 8. Ops: [7.3-7.3]
I20260709 12:18:13.753974  9518 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:13.731619  9508 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:18:13.754763  9688 log.cc:927] Time spent T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.236s	user 0.000s	sys 0.002s
I20260709 12:18:13.707044  9605 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:13.717921  9671 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:13.756598  9399 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:13.756844  9448 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:13.757516  9554 consensus_queue.cc:260] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:13.749234  9443 raft_consensus.cc:3055] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:13.760910  6475 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:13.708781  9619 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:13.793627  6583 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:18:13.797147  6475 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:13.773125  6503 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.021s	sys 0.044s
W20260709 12:18:13.800901  6503 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.022s	sys 0.045s
I20260709 12:18:13.737968  6584 raft_consensus.cc:2393] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 9: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:18:13.745462  9576 consensus_meta.cc:287] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.659s	user 0.012s	sys 0.000s
I20260709 12:18:13.779393  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:13.781060  6581 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:18:13.781883  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:13.805210  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 106118us
I20260709 12:18:13.782532  9518 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:13.764191  9464 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:13.807464  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:13.811574  6578 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:13.803606  6579 raft_consensus.cc:1240] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:13.790752  9443 raft_consensus.cc:740] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:13.800882  9529 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:13.803787  9574 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:13.800472  9383 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:13.821158  9437 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:13.822778  9443 consensus_queue.cc:260] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:13.817361  9518 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:13.849999  9464 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:13.850271  9518 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:13.820240  9695 consensus_queue.cc:1048] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 12:18:13.783727  9671 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:13.853866  9671 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:13.835307  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:13.804520  9629 consensus_queue.cc:1059] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:18:13.816421  9554 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:13.815160  9604 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:13.774039  9570 consensus_queue.cc:260] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:13.848417  9718 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:13.871938  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 296262us
I20260709 12:18:13.837083  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:13.856833  6550 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:13.803994  9575 consensus_queue.cc:1059] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 12:18:13.875305  9703 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:13.875860  9703 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:13.876313  9703 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:13.806356  9576 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:13.881719  6582 raft_consensus.cc:2393] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 9: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:13.881716  9605 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:13.882855  9491 raft_consensus.cc:3055] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:13.883263  9491 raft_consensus.cc:740] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:13.883998  9491 consensus_queue.cc:260] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:13.884871  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:13.885427  6509 raft_consensus.cc:2376] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5. Re-sending same reply.
I20260709 12:18:13.886194  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:13.855942  6654 raft_consensus.cc:2393] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 6: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:13.844473  9529 raft_consensus.cc:697] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 6 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:13.895646  6550 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:13.846413  9574 raft_consensus.cc:697] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:13.934329  6550 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:13.936105  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 229257us
I20260709 12:18:13.880067  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:13.940541  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 60567.5us
I20260709 12:18:13.942847  6622 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:13.949522  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:13.950217  6654 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:13.950941  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:13.954372  9605 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:13.955924  6622 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:13.958832  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:13.846566  9383 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:13.960240  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:13.965224  6549 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:13.965570  6550 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:13.966401  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 167241us
I20260709 12:18:13.968066  6475 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:13.968066  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:13.970039  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:13.969084  6500 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:13.977494  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:13.981815  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 166889us
I20260709 12:18:13.952734  9546 raft_consensus.cc:2764] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:18:13.958657  9491 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:13.985136  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:13.958851  9653 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:18:13.991010  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:13.874636  9604 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:13.988363  6500 raft_consensus.cc:2410] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 12 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:18:13.989224  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:13.986392  6658 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:18:13.966701  9605 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:13.995198  6473 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:13.996932  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:13.987939  6654 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7.
I20260709 12:18:14.000284  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:14.041292  7269 raft_consensus.cc:2410] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 12 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:18:14.041500  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:14.042656  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:14.000656  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:13.969448  9448 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:14.004952  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:13.983114  9464 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:14.006496  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:14.012504  9726 log.cc:927] Time spent T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:18:14.013991  9529 consensus_queue.cc:237] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:14.018199  9383 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:14.018689  9486 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:13.968894  9534 raft_consensus.cc:697] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 7 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:13.976017  9715 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:14.020440  9574 consensus_queue.cc:237] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:14.024673  9717 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:13.991255  9575 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:13.879135  9718 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:14.026477  9619 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:14.037894  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 48744.7us
I20260709 12:18:14.039680  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:14.042912  6548 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:13.983233  9671 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:14.046337  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:14.047580  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:14.090955  6475 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:14.092629  6654 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:14.092923  6503 raft_consensus.cc:2393] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 5: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:14.093852  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:13.968163  9714 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:13.983681  9570 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:13.987185  9576 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:14.045630  9695 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:18:14.055934  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 60818.9us
I20260709 12:18:14.058713  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:14.060302  9723 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:14.063575  9521 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:18:14.071038  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.074582  9629 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:14.049643  9546 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:14.078352  9534 consensus_queue.cc:237] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:14.120668  9521 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:14.123234  9521 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:14.079317  9604 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:14.075452  9621 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:14.124740  9621 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:14.127694  9621 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:14.082054  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:14.128737  6649 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:18:14.109063  9695 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:14.129814  9695 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:14.106274  9605 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:14.081810  9718 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:14.088886  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:14.064373  9486 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:18:14.092043  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 49223.9us
I20260709 12:18:14.101478  6622 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:14.102276  6654 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:14.138702  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:14.139582  6654 raft_consensus.cc:2393] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 9: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:14.104981  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:14.107954  9713 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:14.110018  6656 raft_consensus.cc:2376] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7. Re-sending same reply.
I20260709 12:18:14.111092  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:14.145292  6578 raft_consensus.cc:2376] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7. Re-sending same reply.
I20260709 12:18:14.078063  9575 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:14.095388  9458 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:14.117441  6473 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:14.056092  9464 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:14.121068  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:14.128877  9443 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:14.130131  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:14.147194  6473 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:14.084138  6504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 1883) took 1021 ms. Trace:
I20260709 12:18:14.086133  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:14.087625  9520 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:14.092526  9717 raft_consensus.cc:697] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:14.150979  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:14.105880  9619 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:14.140264  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:14.137104  9605 raft_consensus.cc:697] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:14.137758  9486 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:14.148603  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.148396  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:14.165050  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:14.166234  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:14.167258  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:14.167884  6578 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
W20260709 12:18:14.151965  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.169261  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:14.170509  6578 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:14.173141  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:14.147766  6504 rpcz_store.cc:276] 0709 12:18:13.062571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.324417 (+261846us) service_pool.cc:224] Handling call
0709 12:18:13.324735 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:13.330400 (+  5665us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:13.330451 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:13.331032 (+   581us) log.cc:844] Serialized 26 byte log entry
0709 12:18:13.932504 (+601472us) spinlock_profiling.cc:243] Waited 24.3 ms on lock 0x7b1800303220. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:18:13.984460 (+ 51956us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:13.984498 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:13.984562 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:14.083831 (+ 99269us) raft_consensus.cc:1685] finished
0709 12:18:14.083851 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:14.084079 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1918,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":36228,"spinlock_wait_cycles":55899008,"thread_start_us":51732,"threads_started":1,"wal-append.queue_time_us":751300}
I20260709 12:18:14.152805  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:14.174890  6654 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:18:14.175040  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:14.175725  6504 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:18:14.175832  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:14.176661  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:14.177160  6654 raft_consensus.cc:2410] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 12 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:18:14.177299  6504 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:14.178306  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:14.178429  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:14.179008  6504 raft_consensus.cc:2393] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 7: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:14.179554  6654 raft_consensus.cc:2393] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 9: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:14.153436  6473 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:14.147380  6649 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:18:14.148095  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:14.155953  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 54523.3us
I20260709 12:18:14.150561  9713 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:14.157087  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:14.157217  9605 consensus_queue.cc:237] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:14.158951  9458 raft_consensus.cc:697] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:14.150717  9575 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:14.161772  6548 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:14.194293  6548 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:14.195034  6583 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:18:14.195494  6548 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:14.162115  9669 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:14.163079  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:14.182365  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:14.185042  6581 raft_consensus.cc:2393] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 8: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:14.184726  6622 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:14.181324  6473 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:14.218014  6622 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:14.217373  6656 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:14.138705  9546 raft_consensus.cc:697] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 11 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:14.132881  9629 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:14.251147  6583 raft_consensus.cc:1240] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7. Ops: [5.4-6.5]
W20260709 12:18:14.256732  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 165842us
I20260709 12:18:14.247480  6649 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:14.262887  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:14.192147  9592 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:14.151110  9723 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:14.248075  9381 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:14.237612  9714 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:14.238011  9518 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:14.248390  9645 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:14.238348  9671 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:14.280148  9713 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:14.281342  9631 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:14.296406  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:14.305130  6658 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:18:14.284844  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:14.306233  6504 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:14.197690  9643 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:14.217057  9604 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:14.217272  9669 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:14.302153  7240 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:14.287635  6656 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:14.317899  6654 raft_consensus.cc:1240] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 9. Current term is 10. Ops: [7.7-9.8]
W20260709 12:18:14.326722  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 145485us
I20260709 12:18:14.325210  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:14.333024  6649 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:14.342324  6473 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:14.295889  9381 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:14.291735  9529 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:14.350648  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:14.298130  9714 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:14.311115  9645 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:14.310842  9631 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:14.345155  9703 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:14.322774  9669 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:14.346320  9309 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:18:14.496088  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 278135us
I20260709 12:18:14.497553  6622 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:14.460273  9381 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:14.509449  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 167199us
I20260709 12:18:14.511109  6473 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:14.493253  9486 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:14.515762  9486 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:14.476616  9714 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:14.477768  9645 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:14.523002  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:14.272042  9458 consensus_queue.cc:237] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:14.533335  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 65366.7us
W20260709 12:18:14.501467  9721 log.cc:927] Time spent T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.391s	user 0.001s	sys 0.001s
I20260709 12:18:14.483886  9631 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:14.323813  9643 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:14.325781  9604 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:14.500713  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:14.577678  9643 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:14.578398  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.560530  9736 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:18:14.561889  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.555773  9671 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:14.589268  9671 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:14.591307  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 349) took 1565 ms. Trace:
I20260709 12:18:14.591650  9671 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:14.591913  6508 rpcz_store.cc:276] 0709 12:18:13.025996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.027379 (+  1383us) service_pool.cc:224] Handling call
0709 12:18:13.027683 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:13.063578 (+ 35895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:14.591007 (+1527429us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b54000f3054. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:14.591235 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29129472}
I20260709 12:18:14.597751  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:14.605973  9575 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:14.611727  9575 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:14.612637  9575 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:14.613289  9575 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:14.616559  9693 log.cc:927] Time spent T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 1.065s	user 0.004s	sys 0.007s
I20260709 12:18:14.617581  9381 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:14.618899  6649 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 12: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:14.619488  9288 raft_consensus.cc:3055] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:14.619985  9288 raft_consensus.cc:740] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:14.574069  9464 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:14.574824  9604 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:14.575291  9520 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:14.490936  9717 consensus_queue.cc:237] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:14.571880  9733 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 12:18:14.508636  9534 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:14.550025  9486 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:18:14.628561  9594 log.cc:927] Time spent T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 3.000s	user 0.014s	sys 0.017s
W20260709 12:18:14.632853  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.620812  6506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 1878) took 1594 ms. Trace:
I20260709 12:18:14.710475  6506 rpcz_store.cc:276] 0709 12:18:13.026369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.029870 (+  3501us) service_pool.cc:224] Handling call
0709 12:18:13.030177 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:13.030555 (+   378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:13.030586 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:13.031129 (+   543us) log.cc:844] Serialized 26 byte log entry
0709 12:18:13.323467 (+292338us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:13.323529 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:13.323629 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:14.615849 (+1292220us) spinlock_profiling.cc:243] Waited 2.18 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:18:14.620471 (+  4622us) raft_consensus.cc:1685] finished
0709 12:18:14.620500 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:14.620743 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13682,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":127,"spinlock_wait_cycles":5014272,"thread_start_us":218573,"threads_started":1,"wal-append.queue_time_us":477589}
I20260709 12:18:14.472932  9695 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:14.624884  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 1533) took 1568 ms. Trace:
I20260709 12:18:14.477375  9529 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:14.657958  6650 raft_consensus.cc:1240] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:14.650969  9521 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:14.663658  9288 consensus_queue.cc:260] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:14.664973  9381 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:14.479362  9546 consensus_queue.cc:237] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:14.718930  9521 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:14.712538  6655 rpcz_store.cc:276] 0709 12:18:13.056177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.220227 (+164050us) service_pool.cc:224] Handling call
0709 12:18:14.624649 (+1404422us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b540008f2d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:14.624785 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":396782848}
I20260709 12:18:14.715000  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:14.720410  9737 consensus_queue.cc:1059] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:18:14.723387  6578 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:14.729308  6658 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:18:14.732426  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:14.728832  9288 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:14.717872  9575 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:14.716578  9520 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:14.739310  9520 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:14.741096  6575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1698) took 1715 ms. Trace:
I20260709 12:18:14.741436  9521 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:14.741509  6575 rpcz_store.cc:276] 0709 12:18:13.025464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.026624 (+  1160us) service_pool.cc:224] Handling call
0709 12:18:14.740908 (+1714284us) spinlock_profiling.cc:243] Waited 652 ms on lock 0x7b54001d10d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:14.741037 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1499776256}
I20260709 12:18:14.742219  9520 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:14.744992  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:14.746152  6576 raft_consensus.cc:1240] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:18:14.713371  9534 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:14.726292  9464 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:18:14.758410  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 259446us
I20260709 12:18:14.743705  9629 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:14.748687  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:14.773495  6655 raft_consensus.cc:1240] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:14.777035  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:14.789700  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:14.790477  9713 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:18:14.791102  9554 consensus_meta.cc:287] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.918s	user 0.007s	sys 0.002s
I20260709 12:18:14.791074  9713 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:14.792258  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:14.793682  6650 raft_consensus.cc:1275] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:14.794119  9713 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:14.794881  6650 raft_consensus.cc:1240] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 7. Current term is 8. Ops: [7.3-7.3]
I20260709 12:18:14.795480  9731 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:14.796078  9695 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:14.796252  9731 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:14.797138  9529 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:14.800861  9731 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:14.801378  9534 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:14.802277  9534 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:14.803027  9534 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:14.805667  9534 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:14.806852  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:14.811540  9534 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:14.813198  6622 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:14.813463  6651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58980 (request call id 2107) took 1164 ms. Trace:
I20260709 12:18:14.818084  9733 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:14.818140  6651 rpcz_store.cc:276] 0709 12:18:13.648748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.680421 (+ 31673us) service_pool.cc:224] Handling call
0709 12:18:14.813276 (+1132855us) spinlock_profiling.cc:243] Waited 827 ms on lock 0x7b54000753d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:14.813397 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1901259136}
I20260709 12:18:14.818661  9733 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:14.815054  6658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 454) took 1011 ms. Trace:
I20260709 12:18:14.762802  9464 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:14.820626  6658 rpcz_store.cc:276] 0709 12:18:13.803299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:14.384803 (+581504us) service_pool.cc:224] Handling call
0709 12:18:14.625538 (+240735us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:14.729841 (+104303us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:14.814235 (+ 84394us) spinlock_profiling.cc:243] Waited 92.9 ms on lock 0x7b540008f2d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:14.814890 (+   655us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b540008f2d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:14.815003 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":533232768}
I20260709 12:18:14.821959  9464 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:14.786250  9545 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:14.822956  6658 raft_consensus.cc:1275] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:18:14.822880  9545 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:14.817662  9629 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:14.824946  9545 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:14.824610  9629 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:14.826272  9623 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:14.826927  9629 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:14.827109  9529 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:14.827629  9529 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:14.788815  9491 consensus_meta.cc:287] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.801s	user 0.011s	sys 0.000s
I20260709 12:18:14.829561  9529 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:14.830468  9534 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:18:14.834435  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:14.834903  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:14.820206  6651 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:14.836402  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:18:14.836364  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 325279us
I20260709 12:18:14.835551  6475 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:14.837893  6475 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:14.835551  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:14.839030  6473 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:14.801317  9619 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:14.840492  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:14.841408  6650 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:18:14.842660  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:14.843657  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:14.833933  9605 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.014s	sys 0.016s
I20260709 12:18:14.839895  9618 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:14.861685  6475 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:14.862869  6473 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:14.845608  9619 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:14.842046  9545 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 12:18:14.864346  9735 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:14.864791  9733 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:14.864842  9529 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:14.836206  9669 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:14.843640  9714 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:14.852056  9605 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.014s	sys 0.016s
I20260709 12:18:14.872900  9618 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:14.867853  9645 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:14.884315  6650 raft_consensus.cc:1240] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:18:14.880446  6475 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:14.915005  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:14.908236  6506 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:18:14.917145  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:14.917661  6506 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:14.918191  6576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 368) took 1737 ms. Trace:
I20260709 12:18:14.918602  6576 rpcz_store.cc:276] 0709 12:18:13.181313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.181536 (+   223us) service_pool.cc:224] Handling call
0709 12:18:14.743070 (+1561534us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:14.746598 (+  3528us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:14.918012 (+171414us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b54001d10d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:14.918134 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3590233856}
I20260709 12:18:14.908948  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:14.883505  6473 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:14.911103  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:14.879407  9491 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:14.912992  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:14.920038  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:14.922183  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:14.922722  6508 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
I20260709 12:18:14.921149  6651 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:14.925697  6500 raft_consensus.cc:2393] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 5: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:14.925807  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:14.884792  9723 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:14.921128  7273 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:14.930249  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:14.886531  9735 raft_consensus.cc:697] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:14.932482  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:14.936089  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:14.937537  6508 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:18:14.934535  6474 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:14.882334  9629 consensus_queue.cc:1048] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 12:18:14.913303  9618 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:18:14.939656  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 56234.5us
I20260709 12:18:14.942256  9676 consensus_queue.cc:1059] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:14.880728  9288 consensus_queue.cc:1048] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 12:18:14.883522  9575 consensus_queue.cc:1059] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 5.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 12:18:14.946935  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:14.965132  6650 raft_consensus.cc:2393] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 8: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:14.965426  6473 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:14.965883  6658 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 12:18:14.966706  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 116667us
I20260709 12:18:14.931653  9733 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:14.977244  6651 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:14.967733  6506 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:18:14.984851  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:14.985140  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:14.985806  7268 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:14.910082  9732 consensus_queue.cc:1059] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.080s
I20260709 12:18:14.988083  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:14.991616  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 57156.3us
I20260709 12:18:14.993194  6474 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:14.993306  9491 raft_consensus.cc:697] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:14.994793  9491 consensus_queue.cc:237] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:14.946329  9741 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:14.987202  9703 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:15.018925  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:15.022008  6650 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:18:15.017858  9529 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:15.020566  7265 raft_consensus.cc:1275] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:18:15.024853  9605 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.025818  9605 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:15.027226  9742 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:18:15.028796  9740 log.cc:927] Time spent T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:18:15.002839  9717 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.055897  6656 raft_consensus.cc:1275] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:18:15.002179  9545 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:15.013628  9741 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:15.016474  9546 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.088204  9669 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:15.092612  6654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 452) took 1290 ms. Trace:
I20260709 12:18:15.097419  6505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 1799) took 2063 ms. Trace:
I20260709 12:18:15.097546  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:15.105017  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 139662us
I20260709 12:18:15.107012  6654 rpcz_store.cc:276] 0709 12:18:13.802730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:14.218941 (+416211us) service_pool.cc:224] Handling call
0709 12:18:14.317546 (+ 98605us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:14.318388 (+   842us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.092431 (+774043us) spinlock_profiling.cc:243] Waited 98.3 ms on lock 0x7b54000b6b50. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:15.092556 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":226079488}
I20260709 12:18:15.108160  6473 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:15.109661  6654 raft_consensus.cc:1240] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:18:15.066540  9592 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:15.111083  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:15.067204  9743 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:15.079679  9728 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:15.126781  6504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 1817) took 1021 ms. Trace:
I20260709 12:18:15.107071  6505 rpcz_store.cc:276] 0709 12:18:13.033968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.209249 (+175281us) service_pool.cc:224] Handling call
0709 12:18:13.226248 (+ 16999us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:13.226518 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:18:13.311374 (+ 84856us) log.cc:844] Serialized 14 byte log entry
0709 12:18:14.805929 (+1494555us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b18000fbfe0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a2923a 00007f2875e93a2a
0709 12:18:15.097083 (+291154us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:15.097134 (+    51us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:15.097189 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.097259 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:15.097363 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54639,"prepare.queue_time_us":84213,"prepare.run_cpu_time_us":108,"prepare.run_wall_time_us":22403,"raft.queue_time_us":90993,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":4559,"spinlock_wait_cycles":35177728,"thread_start_us":248980,"threads_started":2}
I20260709 12:18:15.138280  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:15.129758  6473 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:15.124617  7271 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:15.133951  6623 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:15.079202  9676 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:15.137449  6550 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:15.142017  6550 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:15.143713  6550 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:15.144891  6550 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:15.093402  9741 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:15.104626  9736 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:15.080621  9545 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:15.160862  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:15.161136  7271 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:18:15.161557  7273 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
I20260709 12:18:15.136622  9728 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:15.148816  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:15.147749  6550 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:15.137380  9288 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:15.138442  6504 rpcz_store.cc:276] 0709 12:18:14.105696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:14.351671 (+245975us) service_pool.cc:224] Handling call
0709 12:18:14.492068 (+140397us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:14.492656 (+   588us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:14.492701 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:14.507780 (+ 15079us) log.cc:844] Serialized 52 byte log entry
0709 12:18:14.780603 (+272823us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:14.926931 (+146328us) log.cc:844] Serialized 14 byte log entry
0709 12:18:14.946165 (+ 19234us) log.cc:844] Serialized 14 byte log entry
0709 12:18:15.126420 (+180255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.126504 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:15.126556 (+    52us) raft_consensus.cc:1685] finished
0709 12:18:15.126570 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:15.126714 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27886,"prepare.queue_time_us":365461,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":3619,"raft.queue_time_us":183657,"raft.run_cpu_time_us":359,"raft.run_wall_time_us":22496,"thread_start_us":523777,"threads_started":3,"wal-append.queue_time_us":402486}
I20260709 12:18:15.149104  9695 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:15.165040  6623 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:15.150326  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 157190us
I20260709 12:18:15.148653  9172 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:15.148798  9464 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:15.149186  9697 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:15.148792  9669 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:15.164892  6473 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:15.112950  9546 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:15.091045  9717 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:15.168866  9728 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:15.177541  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:15.187647  6550 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:15.178285  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:15.177241  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:15.189100  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:15.189942  6505 raft_consensus.cc:2368] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 8. Current term is 9.
I20260709 12:18:15.111469  9735 consensus_queue.cc:237] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:15.150820  9676 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:15.176095  9753 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:15.179507  9621 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:15.206210  9676 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:15.191880  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:15.207769  9744 log.cc:927] Time spent T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
W20260709 12:18:15.209605  9745 log.cc:927] Time spent T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.184s	user 0.000s	sys 0.003s
I20260709 12:18:15.211439  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 1812) took 1112 ms. Trace:
I20260709 12:18:15.211826  6501 rpcz_store.cc:276] 0709 12:18:14.098762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:14.245009 (+146247us) service_pool.cc:224] Handling call
0709 12:18:14.250930 (+  5921us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:14.747047 (+496117us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:14.747094 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:14.748132 (+  1038us) log.cc:844] Serialized 52 byte log entry
0709 12:18:14.950924 (+202792us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:14.950960 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:14.951046 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:15.209024 (+257978us) spinlock_profiling.cc:243] Waited 43.5 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:18:15.211110 (+  2086us) raft_consensus.cc:1685] finished
0709 12:18:15.211131 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:15.211369 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":58387,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":5485,"spinlock_wait_cycles":100138368,"thread_start_us":659944,"threads_started":2,"wal-append.queue_time_us":267711}
I20260709 12:18:15.141357  9743 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:15.198848  9748 log.cc:927] Time spent T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 12:18:15.201176  9458 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:15.179597  9645 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:15.191032  9288 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 12:18:15.191104  9464 raft_consensus.cc:697] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:15.190922  9714 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:15.180604  9172 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:18:15.225490  9631 consensus_meta.cc:287] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.556s	user 0.011s	sys 0.000s
I20260709 12:18:15.217548  6622 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:15.184706  9604 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 12:18:15.224718  6654 raft_consensus.cc:1275] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:18:15.255657  6622 leader_election.cc:400] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 8. Current term is 9.
W20260709 12:18:15.225368  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 60543.5us
I20260709 12:18:15.219025  9739 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:15.225982  6550 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:15.189213  9514 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:15.180109  9659 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:15.214527  9648 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:15.224270  9458 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:15.196045  9592 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:15.256035  6622 leader_election.cc:403] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:18:15.256665  6473 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:18:15.246181  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.230545  9621 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:15.253419  9172 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:15.272383  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 46600.3us
I20260709 12:18:15.266613  7268 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:15.284185  6473 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:15.285264  6549 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:18:15.285784  6550 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:15.286885  7274 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:15.288304  6473 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:15.260609  9697 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:15.289530  9697 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:15.290108  9763 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:15.265987  9631 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:15.291687  9575 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:15.291813  9697 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:15.292378  9703 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:15.292634  6503 raft_consensus.cc:2393] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 9: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:15.265944  9514 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:15.293294  9514 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:15.293653  9621 raft_consensus.cc:697] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:15.294600  9621 consensus_queue.cc:237] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:15.265388  9739 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:15.290583  9763 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:15.296622  9739 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:15.297029  9763 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:15.292877  9703 raft_consensus.cc:697] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 7 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:15.301437  9703 consensus_queue.cc:237] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:15.304171  9288 consensus_queue.cc:1059] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 5.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:15.289479  9464 consensus_queue.cc:237] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:15.308629  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 1806) took 1731 ms. Trace:
I20260709 12:18:15.265786  9669 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:15.273191  9458 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:15.337764  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2119) took 1333 ms. Trace:
I20260709 12:18:15.311105  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:15.269747  9659 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:15.313968  9760 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:15.321592  9697 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:15.282989  9534 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 8. Current term is 9.
I20260709 12:18:15.270114  9648 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:15.281162  9695 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:15.309842  9743 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:15.377363  9668 raft_consensus.cc:3055] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:15.377902  9668 raft_consensus.cc:740] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:15.378610  9546 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:15.378713  9668 consensus_queue.cc:260] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:15.380240  9668 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:15.385385  9631 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:15.283237  9592 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:15.388383  9592 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:15.345227  9669 raft_consensus.cc:697] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:15.304306  9741 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:15.390453  9669 consensus_queue.cc:237] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:15.391448  9741 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:15.338495  6655 rpcz_store.cc:276] 0709 12:18:14.004755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:14.788984 (+784229us) service_pool.cc:224] Handling call
0709 12:18:14.826235 (+ 37251us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:14.826618 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:18:14.932709 (+106091us) log.cc:844] Serialized 14 byte log entry
0709 12:18:15.173293 (+240584us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:15.189919 (+ 16626us) log.cc:844] Serialized 52 byte log entry
0709 12:18:15.190195 (+   276us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:15.190258 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.190350 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:15.337297 (+146947us) raft_consensus.cc:1685] finished
0709 12:18:15.337417 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:15.337670 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":265752,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":5145,"raft.run_cpu_time_us":313,"raft.run_wall_time_us":9666,"thread_start_us":264907,"threads_started":2,"wal-append.queue_time_us":346493}
I20260709 12:18:15.375753  9576 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.343940  9659 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:15.375574  9619 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:15.376935  9545 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:15.310160  9288 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 12:18:15.399065  9576 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:15.403626  9576 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:15.371112  9648 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:15.406643  9514 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:15.435751  9717 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:15.436764  9733 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:15.437574  9623 raft_consensus.cc:3055] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:18:15.438807  9729 log.cc:927] Time spent T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.692s	user 0.000s	sys 0.005s
W20260709 12:18:15.446542  9746 log.cc:927] Time spent T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.341s	user 0.000s	sys 0.001s
I20260709 12:18:15.453555  9636 raft_consensus.cc:3055] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:15.472576  9738 raft_consensus.cc:3055] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:15.464041  9733 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:15.473186  9738 raft_consensus.cc:740] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:15.500833  9738 consensus_queue.cc:260] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:15.502748  9738 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:15.477275  9763 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:15.515254  9763 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:15.516872  9739 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:15.524514  9739 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:15.558715  6506 raft_consensus.cc:2393] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 6: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:15.316627  6507 rpcz_store.cc:276] 0709 12:18:13.577524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.695640 (+118116us) service_pool.cc:224] Handling call
0709 12:18:13.695965 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:13.696939 (+   974us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:13.696989 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:15.069292 (+1372303us) spinlock_profiling.cc:243] Waited 2.4 ms on lock 0x7b2c00000c88. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:18:15.081187 (+ 11895us) log.cc:844] Serialized 26 byte log entry
0709 12:18:15.177612 (+ 96425us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:15.177646 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.177881 (+   235us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:15.308172 (+130291us) raft_consensus.cc:1685] finished
0709 12:18:15.308210 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:15.308548 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":556,"raft.queue_time_us":1080,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":8754,"spinlock_wait_cycles":5513984,"thread_start_us":89814,"threads_started":1,"wal-append.queue_time_us":219119}
I20260709 12:18:15.314625  6473 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:15.566798  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:15.344722  6582 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 1553) took 1938 ms (client timeout 1576 ms). Trace:
W20260709 12:18:15.628772  6582 rpcz_store.cc:269] 0709 12:18:13.405950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.454534 (+ 48584us) service_pool.cc:224] Handling call
0709 12:18:15.344530 (+1889996us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b540034dad4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:15.344657 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":890919168}
I20260709 12:18:15.632453  9718 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.633109  9703 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.634006  9703 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:15.449566  9605 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:15.569000  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.590302  6658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1547) took 1523 ms. Trace:
I20260709 12:18:15.463635  9623 raft_consensus.cc:740] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:15.469497  9636 raft_consensus.cc:740] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:15.591616  9732 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:15.616232  6502 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 1794) took 2640 ms (client timeout 2246 ms). Trace:
W20260709 12:18:15.623951  9574 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.864s	user 0.018s	sys 0.004s
W20260709 12:18:15.644932  9574 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.891s	user 0.019s	sys 0.004s
W20260709 12:18:15.625602  9575 consensus_queue.cc:1175] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "496c7c3792cd4587b4dbeba435f4be1e" responder_term: 8 status { last_received { term: 8 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 8 index: 5 } }
I20260709 12:18:15.633637  9718 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:15.594807  9514 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:15.484308  9723 consensus_meta.cc:287] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.550s	user 0.010s	sys 0.000s
I20260709 12:18:15.641225  6658 rpcz_store.cc:276] 0709 12:18:14.066325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:14.827683 (+761358us) service_pool.cc:224] Handling call
0709 12:18:14.828005 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:14.984979 (+156974us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:15.589493 (+604514us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b54002a3ad4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:15.590029 (+   536us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":309688192}
W20260709 12:18:15.643563  6502 rpcz_store.cc:269] 0709 12:18:12.975870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.027165 (+ 51295us) service_pool.cc:224] Handling call
0709 12:18:15.615661 (+2588496us) spinlock_profiling.cc:243] Waited 804 ms on lock 0x7b5400387f54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:15.615965 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1849019136}
I20260709 12:18:15.666735  9623 consensus_queue.cc:260] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:15.669689  9636 consensus_queue.cc:260] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:15.671526  9636 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:15.666576  9732 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:15.678037  9723 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:15.678478  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:15.680529  9718 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:15.682346  9575 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:18:15.682297  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 367596us
I20260709 12:18:15.683272  9575 raft_consensus.cc:697] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:15.685063  9575 consensus_queue.cc:237] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:15.686949  9619 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:15.655265  9508 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:15.692966  9508 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:15.695807  9508 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:15.700165  9604 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:15.701282  9604 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:15.703920  9604 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:15.711732  9490 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:15.717108  6651 raft_consensus.cc:1240] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:18:15.719767  9732 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:15.713012  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 395220us
I20260709 12:18:15.724570  9622 consensus_queue.cc:1059] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:18:15.726949  6509 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 1898) took 2015 ms (client timeout 1970 ms). Trace:
I20260709 12:18:15.731963  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:15.731282  9623 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:15.733443  9703 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:15.751312  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:15.752651  6584 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 11.
I20260709 12:18:15.755219  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:15.756371  9703 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:15.757381  9703 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:15.759024  9546 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.760106  9381 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:15.759922  9546 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:15.760903  9381 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:15.763185  9381 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:15.764382  6655 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:18:15.764286  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:15.766199  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:15.766963  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 481218us
W20260709 12:18:15.738687  6509 rpcz_store.cc:269] 0709 12:18:13.711014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:13.885970 (+174956us) service_pool.cc:224] Handling call
0709 12:18:15.726430 (+1840460us) spinlock_profiling.cc:243] Waited 1.76 s on lock 0x7b54002c5bd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:15.726692 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4052466560}
I20260709 12:18:15.767623  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:15.756429  9480 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.781394  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:15.782393  6550 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:15.782757  6649 raft_consensus.cc:2393] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 6: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:15.762988  9546 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:15.814270  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:15.814495  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:15.816148  9518 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:18:15.817174  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 133325us
I20260709 12:18:15.800102  9703 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:15.834282  9458 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:15.836575  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 63961.7us
I20260709 12:18:15.843523  7268 raft_consensus.cc:1275] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 12:18:15.843971  7268 pending_rounds.cc:85] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9: Aborting all ops after (but not including) 6
I20260709 12:18:15.844218  7268 pending_rounds.cc:107] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 12:18:15.844540  7268 raft_consensus.cc:2889] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:15.786362  9604 consensus_queue.cc:1059] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:18:15.846072  9604 raft_consensus.cc:3055] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:15.846552  9604 raft_consensus.cc:740] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:15.797297  9480 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:15.852222  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:15.858052  9618 consensus_meta.cc:287] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.888s	user 0.008s	sys 0.011s
I20260709 12:18:15.852728  6509 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:15.865052  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:15.866907  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:15.867842  6576 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 6.
W20260709 12:18:15.868809  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 86621.5us
I20260709 12:18:15.869827  9703 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:15.873586  9480 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:15.875231  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:15.876165  6584 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:15.859086  9618 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:15.847918  9604 consensus_queue.cc:260] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:15.883147  9604 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:15.879547  6475 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:15.899519  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:15.902112  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 57520.8us
I20260709 12:18:15.902719  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:15.900528  6576 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
W20260709 12:18:15.905910  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 64481us
W20260709 12:18:15.908576  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:15.915166  6622 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:15.921150  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:15.922652  6582 raft_consensus.cc:2410] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:18:15.921797  6406 catalog_manager.cc:5697] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 4 to 5, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 5 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:15.925426  6509 raft_consensus.cc:1240] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:15.911936  9618 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:15.935415  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 145100us
I20260709 12:18:15.933396  9458 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:15.949010  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:15.959055  6575 raft_consensus.cc:1275] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:15.967640  6584 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:15.961208  9648 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:15.959071  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:15.969110  6406 catalog_manager.cc:5697] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 4 to 7, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 7 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:15.970501  6578 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
W20260709 12:18:15.970711  6402 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57888 (request call id 215) took 22872 ms (client timeout 15000 ms). Trace:
I20260709 12:18:15.971175  9648 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:15.952381  9723 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:15.966877  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:15.973290  9723 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:15.975943  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:15.971799  6406 catalog_manager.cc:5697] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 3 to 5, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 5 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:18:15.981065  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 93317.1us
I20260709 12:18:15.997416  6406 catalog_manager.cc:5697] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
W20260709 12:18:15.999815  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.000383  9580 raft_consensus.cc:3055] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:16.000890  9580 raft_consensus.cc:740] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:16.002163  9580 consensus_queue.cc:260] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:16.003494  9580 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:16.004374  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:16.004630  9773 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:15.991501  9518 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:16.005352  9773 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:15.999212  6406 catalog_manager.cc:5697] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 3 to 5, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 5 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:18:16.013226  9738 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:15.997535  9775 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:16.014518  9773 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:16.014470  9775 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:16.017194  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:16.016961  6406 catalog_manager.cc:5697] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 6 to 9, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 9 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:16.018651  6651 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:16.018883  6406 catalog_manager.cc:5697] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 2 to 5, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 5 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:18:16.019956  9172 consensus_meta.cc:287] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.715s	user 0.000s	sys 0.013s
I20260709 12:18:16.020991  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:15.991137  9648 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:16.025095  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:16.004657  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:16.027700  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:16.028260  6654 raft_consensus.cc:2393] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 9: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:16.028604  6656 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:16.028790  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:18:16.027851  9782 log.cc:927] Time spent T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 12:18:16.036293  7268 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:18:16.006196  9508 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:16.038925  6655 raft_consensus.cc:1240] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:16.040225  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:16.006014  9518 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:16.042821  6582 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:16.046380  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:16.043606  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:16.049422  6507 raft_consensus.cc:1275] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:18:16.049923  6507 pending_rounds.cc:85] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 4
I20260709 12:18:16.050249  6507 pending_rounds.cc:107] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:18:16.050531  6507 raft_consensus.cc:2889] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:16.032430  9514 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:18:16.052559  9733 consensus_meta.cc:287] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.574s	user 0.014s	sys 0.002s
I20260709 12:18:16.053170  9733 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:16.054904  6509 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:16.055588  9733 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:16.056468  6649 raft_consensus.cc:2376] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8. Re-sending same reply.
I20260709 12:18:16.052809  9514 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:16.058637  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:16.060652  9514 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:16.025267  9718 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:18:15.971112  6402 rpcz_store.cc:269] 0709 12:17:53.098661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:17:53.098898 (+   237us) service_pool.cc:224] Handling call
0709 12:18:15.970409 (+22871511us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:11.371786 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:11.372679 (+   893us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:11.372729 (+    50us) write_op.cc:435] Acquired schema lock
0709 12:18:11.372746 (+    17us) tablet.cc:662] Decoding operations
0709 12:18:11.446594 (+ 73848us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:11.462450 (+ 15856us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:11.472511 (+ 10061us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:18:11.691383 (+218872us) tablet.cc:701] Row locks acquired
0709 12:18:11.739922 (+ 48539us) write_op.cc:260] PREPARE: finished
0709 12:18:11.740109 (+   187us) write_op.cc:270] Start()
0709 12:18:11.740277 (+   168us) write_op.cc:276] Timestamp: P: 1783599491740038 usec, L: 0
0709 12:18:11.740305 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 12:18:11.741038 (+   733us) log.cc:844] Serialized 2798 byte log entry
0709 12:18:12.511624 (+770586us) op_driver.cc:464] REPLICATION: finished
0709 12:18:12.842125 (+330501us) write_op.cc:301] APPLY: starting
0709 12:18:12.842210 (+    85us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:12.842429 (+   219us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:12.842550 (+   121us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:12.851286 (+  8736us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:12.851543 (+   257us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 12:18:12.851598 (+    55us) write_op.cc:312] APPLY: finished
0709 12:18:12.851896 (+   298us) log.cc:844] Serialized 59 byte log entry
0709 12:18:12.852086 (+   190us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:12.852214 (+   128us) write_op.cc:454] Released schema lock
0709 12:18:12.852263 (+    49us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:12.852710 (+   447us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:12.852755 (+    45us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":998153,"reported_tablets":38,"child_traces":[["op",{"apply.queue_time_us":274675,"apply.run_cpu_time_us":3050,"apply.run_wall_time_us":10560,"num_ops":7,"prepare.queue_time_us":342958,"prepare.run_cpu_time_us":19157,"prepare.run_wall_time_us":571515,"raft.queue_time_us":247402,"raft.run_cpu_time_us":48170,"raft.run_wall_time_us":282476,"replication_time_us":756301,"thread_start_us":661215,"threads_started":4,"wal-append.queue_time_us":253102,"wal-append.run_cpu_time_us":36700,"wal-append.run_wall_time_us":1854075}]]}
I20260709 12:18:16.064766  9718 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:16.067380  9718 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:16.068800  9508 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:16.071133  9508 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:16.071931  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:16.072644  6579 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:16.075253  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.076767  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:16.077422  6578 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:16.078994  6548 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:16.080708  6548 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:16.081151  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:16.081838  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:16.082196  9773 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:16.082630  6582 raft_consensus.cc:2393] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 8: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:16.082655  9773 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:16.083302  9773 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:16.049713  9720 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:16.043298  6406 catalog_manager.cc:5697] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 4 to 5, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 5 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:18:16.087042  6576 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:18:16.093580  6579 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:16.094779  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:16.092816  9775 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:16.097761  6652 raft_consensus.cc:1240] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 9. Current term is 10. Ops: [7.7-9.8]
I20260709 12:18:16.098315  6406 catalog_manager.cc:5697] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:18:16.099354  9703 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:18:16.101435  9728 consensus_meta.cc:287] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.836s	user 0.008s	sys 0.016s
I20260709 12:18:16.101963  9728 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:16.103369  6504 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
W20260709 12:18:16.098817  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 58339.9us
W20260709 12:18:16.041540  9659 consensus_meta.cc:287] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.539s	user 0.004s	sys 0.012s
I20260709 12:18:16.100178  6406 catalog_manager.cc:5697] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:18:16.121253  9763 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:16.067934  9718 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
W20260709 12:18:16.122103  9787 log.cc:927] Time spent T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 12:18:16.122175  9763 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:16.023180  9172 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:16.123992  6475 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:16.129141  6651 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:18:16.132917  9172 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:16.133927  6622 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:16.135313  6406 catalog_manager.cc:5697] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 5 to 7, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 7 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:18:16.136776  9739 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:16.136518  9720 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:16.137180  6406 catalog_manager.cc:5697] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 5 to 7, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 7 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:16.139185  9720 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:16.141117  9763 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:16.141646  9728 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:16.142050  6584 raft_consensus.cc:1240] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:16.142652  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:18:16.146417  9723 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:16.147428  9546 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.148160  9546 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:16.150614  9790 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:16.151012  9790 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:16.151357  9790 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:16.151188  9546 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:16.152485  9546 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.153216  9546 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:16.153545  9728 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:16.154300  9728 raft_consensus.cc:697] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:16.155710  9618 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:16.158705  6503 raft_consensus.cc:1240] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:18:16.174989  6578 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
W20260709 12:18:16.180639  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 97896.3us
I20260709 12:18:16.155544  9728 consensus_queue.cc:237] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:16.196257  6509 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:16.095738  9738 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:16.200865  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:16.201243  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 58668.7us
I20260709 12:18:16.210568  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:16.185644  6406 catalog_manager.cc:5697] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 7 to 9, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4), now has a pending config: VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1), VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2), VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3), VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 9 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } } pending_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:18:16.214540  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:16.215994  6656 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:18:16.214640  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:16.198913  9763 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:16.217931  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 62262.8us
I20260709 12:18:16.200353  9648 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:16.221366  6582 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:16.221549  6502 raft_consensus.cc:1275] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:18:16.222054  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:16.141680  9659 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:16.226257  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:16.225208  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:16.228518  7273 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 12:18:16.229212  6575 raft_consensus.cc:2393] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 8: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:16.228978  7273 pending_rounds.cc:85] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9: Aborting all ops after (but not including) 6
I20260709 12:18:16.228912  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:16.229733  7273 pending_rounds.cc:107] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 12:18:16.230093  7273 raft_consensus.cc:2889] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:16.231302  6504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 1828) took 1054 ms. Trace:
I20260709 12:18:16.238703  9659 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:16.242645  6503 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 12: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:16.231798  6504 rpcz_store.cc:276] 0709 12:18:15.176620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:15.178609 (+  1989us) service_pool.cc:224] Handling call
0709 12:18:16.231135 (+1052526us) spinlock_profiling.cc:243] Waited 896 ms on lock 0x7b5400387554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:16.231245 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2060984064}
I20260709 12:18:16.245966  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:16.248411  6501 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:16.249883  6505 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:16.250980  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:16.246840  6473 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:16.254624  6406 catalog_manager.cc:5697] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 6 to 9, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 9 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:16.262946  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:16.260185  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:16.263522  6507 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:16.263773  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:16.266608  9618 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.269163  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:16.257381  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:16.272135  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:16.226800  9738 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:16.271914  6500 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
I20260709 12:18:16.275529  6576 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:16.271281  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:16.273012  6656 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:16.276983  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:16.274681  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:16.278088  6508 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:18:16.278473  7268 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 12:18:16.278820  7268 pending_rounds.cc:85] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9: Aborting all ops after (but not including) 7
I20260709 12:18:16.279078  7268 pending_rounds.cc:107] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 12:18:16.279341  7268 raft_consensus.cc:2889] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:16.281371  9546 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:16.284544  9518 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:16.287453  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:16.288575  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:16.288895  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:16.289553  6654 raft_consensus.cc:2368] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 5. Current term is 6.
I20260709 12:18:16.289297  6406 catalog_manager.cc:5697] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:18:16.244774  9763 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:16.299152  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 71988.4us
I20260709 12:18:16.306753  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:16.307049  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:16.308215  6575 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:16.312564  9621 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:16.297312  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:16.315264  6655 raft_consensus.cc:2393] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 6: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:16.320603  6473 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:16.262794  9574 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:16.326674  6474 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:16.322598  6622 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:16.335893  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:16.335397  6406 catalog_manager.cc:5697] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 1 to 5, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 5 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:18:16.337795  9621 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:16.338265  9621 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:16.337522  6406 catalog_manager.cc:5697] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:18:16.349668  6584 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:16.349321  6406 catalog_manager.cc:5697] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:18:16.350641  6406 catalog_manager.cc:5697] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 3 to 8. New cstate: current_term: 8 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:18:16.355566  9728 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:18:16.355824  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 51699.2us
I20260709 12:18:16.356070  9728 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:16.356535  9728 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:16.359608  6656 raft_consensus.cc:1275] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:18:16.351976  6406 catalog_manager.cc:5697] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: config changed from index 2 to 4, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1), VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2), VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3), VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 8 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:18:16.364497  9575 consensus_queue.cc:1048] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:16.367512  6550 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:16.368644  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:16.364930  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:16.370056  9576 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:16.370321  6550 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:16.370656  9576 raft_consensus.cc:697] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:16.370901  9464 consensus_queue.cc:1059] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:16.371290  9172 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:16.371790  9576 consensus_queue.cc:237] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:16.372411  9514 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:16.372658  9798 raft_consensus.cc:3055] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:16.373009  9514 raft_consensus.cc:697] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:18:16.373117  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 58556.1us
I20260709 12:18:16.373687  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:16.374329  7274 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
I20260709 12:18:16.374006  9514 consensus_queue.cc:237] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:16.365494  9728 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:16.366732  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:16.376941  7273 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:18:16.325843  9546 consensus_queue.cc:1059] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 12:18:16.378068  9546 raft_consensus.cc:3055] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:16.378119  9733 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:16.378546  9546 raft_consensus.cc:740] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:16.377390  9575 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:16.381959  6473 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
W20260709 12:18:16.393206  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 101875us
I20260709 12:18:16.400355  6576 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:18:16.391563  6507 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:18:16.401160  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:16.376942  9728 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:16.370721  6406 catalog_manager.cc:5697] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:18:16.406584  6575 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:18:16.408197  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:16.274981  9773 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:16.410091  6550 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:16.427887  7271 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:16.298537  9618 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:16.274159  9648 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.130s
I20260709 12:18:16.433254  6575 raft_consensus.cc:1240] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7. Ops: [4.4-6.5]
I20260709 12:18:16.426292  6550 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:16.422689  6406 catalog_manager.cc:5697] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:18:16.373154  9798 raft_consensus.cc:740] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:16.434494  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:16.465090  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:16.467962  6504 raft_consensus.cc:2410] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 9 because replica has last-logged OpId of term: 7 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 2.
I20260709 12:18:16.459410  6406 catalog_manager.cc:5697] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:18:16.432130  9720 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:16.469465  6550 leader_election.cc:400] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 5. Current term is 6.
I20260709 12:18:16.474157  6550 leader_election.cc:403] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:18:16.458276  9718 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:18:16.479943  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 157453us
I20260709 12:18:16.485052  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:16.465289  9574 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:16.453608  9659 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:16.497711  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:16.498996  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:16.473593  9720 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:16.469779  9546 consensus_queue.cc:260] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:16.453557  9728 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:18:16.522535  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:16.530136  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 148283us
I20260709 12:18:16.515522  6406 catalog_manager.cc:5697] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 1 to 4, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 4 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:16.496855  9574 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:16.506780  9775 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.512178  9720 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:16.540253  6473 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:16.497565  9604 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:18:16.524639  9172 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 5. Current term is 6.
I20260709 12:18:16.558570  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:16.533085  9574 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:16.549165  9480 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:16.530133  9381 consensus_queue.cc:1048] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
W20260709 12:18:16.609180  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 69021.5us
I20260709 12:18:16.610816  6473 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:16.612432  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:16.614408  6406 catalog_manager.cc:5697] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 2 to 5, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 5 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:18:16.514912  7273 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f"
dest_uuid: "20a202c76244485fbc8f1610473db8b9"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:48510
W20260709 12:18:16.634647  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 78878.7us
W20260709 12:18:16.636632  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.619040  9763 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:16.593724  9480 raft_consensus.cc:697] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 10 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:16.595244  9738 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:16.641023  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 84294.2us
I20260709 12:18:16.661716  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:16.594367  9723 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:16.585879  9775 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:16.674787  6505 raft_consensus.cc:1275] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:18:16.676306  9669 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:16.678197  9717 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:18:16.679262  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 68551.3us
I20260709 12:18:16.759709  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:16.760115  6473 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:16.680996  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:16.682828  9777 log.cc:927] Time spent T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 12:18:16.680137  6577 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:18:16.641646  9575 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:18:16.741087  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 348063us
I20260709 12:18:16.741910  6622 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:16.768952  6581 raft_consensus.cc:2368] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 8. Current term is 9.
I20260709 12:18:16.769301  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:16.689481  9773 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:16.732306  9738 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:16.760915  7238 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:16.781864  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 333684us
I20260709 12:18:16.778957  6473 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:16.606953  9798 consensus_queue.cc:260] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:16.740377  9604 consensus_queue.cc:1059] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:18:16.769865  9618 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:16.804123  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:16.737390  9480 consensus_queue.cc:237] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:16.785409  9723 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:16.804673  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:16.815085  9818 log.cc:927] Time spent T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 12:18:16.816224  9814 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:16.816865  9814 raft_consensus.cc:697] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:18:16.818200  9802 log.cc:927] Time spent T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.415s	user 0.004s	sys 0.000s
I20260709 12:18:16.817814  9814 consensus_queue.cc:237] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:16.820572  9810 log.cc:927] Time spent T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
W20260709 12:18:16.821599  9808 log.cc:927] Time spent T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
W20260709 12:18:16.823333  9812 log.cc:927] Time spent T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:18:16.824755  6583 raft_consensus.cc:1240] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:16.828112  9775 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:16.828992  9821 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:18:16.830106  9806 log.cc:927] Time spent T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
I20260709 12:18:16.830483  9720 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:16.830924  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 70087.8us
I20260709 12:18:16.832670  6582 raft_consensus.cc:2393] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 9: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:16.835965  6577 raft_consensus.cc:2393] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 6: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:18:16.841105  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 62235.5us
I20260709 12:18:16.845971  6509 raft_consensus.cc:1240] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 8. Current term is 9. Ops: []
W20260709 12:18:16.806588  9809 log.cc:927] Time spent T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
W20260709 12:18:16.855365  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 86141us
I20260709 12:18:16.856904  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:16.794361  9738 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:16.835563  9723 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:16.857854  7238 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:16.876485  6473 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:16.855104  9798 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:16.807850  9669 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:16.885484  9580 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:16.891861  9618 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:18:16.892302  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:16.892529  9618 raft_consensus.cc:697] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 6 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:16.893461  9618 consensus_queue.cc:237] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:16.894099  6653 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:18:16.895129  6577 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:16.895821  6579 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:16.878979  9723 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:16.884364  9720 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:16.917728  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:16.890652  9575 consensus_queue.cc:1059] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.084s
I20260709 12:18:17.111624  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:17.113507  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:17.114679  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 267637us
I20260709 12:18:17.120787  6576 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:18:16.920521  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:17.000367  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:17.122254  6507 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:16.947381  6649 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:18:17.003620  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:16.969635  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:17.131114  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:16.946659  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:16.945950  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:17.134126  6653 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:17.110641  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:17.120386  6505 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:17.011828  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:16.997280  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:16.920131  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:17.080154  6502 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:17.069730  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:16.919260  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:17.138016  6623 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:16.906373  9717 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:18:17.115775  9790 consensus_meta.cc:287] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.964s	user 0.013s	sys 0.001s
W20260709 12:18:16.969661  9621 consensus_meta.cc:287] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.631s	user 0.011s	sys 0.001s
W20260709 12:18:17.112700  9824 log.cc:927] Time spent T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:18:17.142560  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:17.134975  6503 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:17.143152  6655 raft_consensus.cc:2368] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 8. Current term is 9.
W20260709 12:18:17.150872  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:17.145658  6500 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:17.167953  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:17.172492  6579 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
I20260709 12:18:17.111826  9827 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:17.152474  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:17.175766  6508 raft_consensus.cc:2393] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 8: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:17.173815  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:17.177421  6502 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:17.180312  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:17.184846  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:17.146509  9717 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:17.215919  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:17.012410  9574 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:17.168193  9381 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:17.156086  9648 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:17.253034  6500 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:17.248813  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:17.221482  9381 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:17.260602  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:17.264074  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 263848us
I20260709 12:18:17.259347  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:17.268189  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:17.207048  9717 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:17.169070  9790 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:17.171952  9621 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:17.279968  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:16.947707  9825 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:17.267758  9381 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:17.294005  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 343697us
I20260709 12:18:17.300042  6579 raft_consensus.cc:1275] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 12:18:17.304167  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 448066us
I20260709 12:18:17.307750  7270 raft_consensus.cc:1240] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:17.308405  7273 raft_consensus.cc:606] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:17.308810  7273 raft_consensus.cc:614] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:17.309473  6579 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:18:17.257229  9827 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:17.295620  9825 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:17.276156  9648 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:17.315887  6474 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:17.320400  6507 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:17.322142  9829 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:17.329282  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:17.295357  9619 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:18:17.332015  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1717) took 1105 ms. Trace:
I20260709 12:18:17.282311  9841 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 12:18:17.332801  6578 rpcz_store.cc:276] 0709 12:18:16.221138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.225221 (+  4083us) service_pool.cc:224] Handling call
0709 12:18:16.245195 (+ 19974us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:16.305703 (+ 60508us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:18:16.376641 (+ 70938us) log.cc:844] Serialized 14 byte log entry
0709 12:18:16.562456 (+185815us) log.cc:844] Serialized 14 byte log entry
0709 12:18:17.325983 (+763527us) spinlock_profiling.cc:243] Waited 8.79 ms on lock 0x7b18003f4720. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d16c0a 00007f287c456219 00007f2872d9ea40 00007f2872d994e4 00007f2872d99455 00007f2875e81060 00007f2875e93716 00007f2875ea4287 00007f2875ea4208
0709 12:18:17.326127 (+   144us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:17.326180 (+    53us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:17.326230 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:17.326290 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:17.326400 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":799262,"prepare.queue_time_us":209169,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":89,"raft.queue_time_us":800652,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":15194,"spinlock_wait_cycles":20217216,"thread_start_us":126896,"threads_started":1,"wal-append.queue_time_us":211286}
I20260709 12:18:17.323531  9825 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:17.335194  6548 connection.cc:256] RPC call timeout handler was delayed by -1.390256360000194s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18280000000000002s for a response
I20260709 12:18:17.288225  9844 consensus_queue.cc:1059] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
W20260709 12:18:17.359239  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 53609.1us
W20260709 12:18:17.370205  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 72938.8us
W20260709 12:18:17.370846  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 68347us
I20260709 12:18:17.330590  9829 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:17.378371  9829 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:17.381031  6653 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:18:17.381811  6649 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:18:17.379779  7269 consensus_queue.cc:237] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:17.382505  9574 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:17.407047  9846 log.cc:927] Time spent T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:18:17.405093  9842 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:17.404239  9820 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:18:17.336596  9733 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:17.385509  6622 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:17.445907  9852 raft_consensus.cc:2764] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
W20260709 12:18:17.446954  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 62017.7us
I20260709 12:18:17.447780  6658 raft_consensus.cc:1240] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:17.448796  9669 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:17.450445  9546 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:17.450839  9798 consensus_meta.cc:287] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.559s	user 0.009s	sys 0.000s
I20260709 12:18:17.461597  6657 raft_consensus.cc:1240] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 5. Current term is 6. Ops: []
W20260709 12:18:17.482158  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 98066us
W20260709 12:18:17.501468  6548 connection.cc:256] RPC call timeout handler was delayed by -1.2624563600002148s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.197s for a response
W20260709 12:18:17.501855  6548 connection.cc:256] RPC call timeout handler was delayed by -1.2561563600002046s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19770000000000001s for a response
W20260709 12:18:17.492523  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 107127us
I20260709 12:18:17.493990  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:17.496049  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:17.498296  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 122546us
I20260709 12:18:17.502759  6579 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:17.498669  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:17.491863  6550 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:17.482191  9480 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.006s	sys 0.020s
W20260709 12:18:17.496987  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 112576us
I20260709 12:18:17.503465  6578 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:17.502826  6577 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:17.503896  6475 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:17.509321  6575 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
W20260709 12:18:17.510090  6548 connection.cc:256] RPC call timeout handler was delayed by -1.1733563600000707s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2069s for a response
W20260709 12:18:17.510681  6548 connection.cc:256] RPC call timeout handler was delayed by -1.0185563600000478s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22410000000000002s for a response
I20260709 12:18:17.510942  6550 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:17.504323  9480 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.006s	sys 0.020s
W20260709 12:18:17.525434  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:17.531744  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:17.532753  6579 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:17.520394  6475 leader_election.cc:400] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 8. Current term is 9.
I20260709 12:18:17.520913  6577 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:18:17.534565  6475 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:17.493422  9733 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:17.521310  9623 consensus_queue.cc:1059] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 12:18:17.539764  9648 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:17.539518  9825 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:17.479069  9381 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:17.488768  9844 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:18:17.502296  9829 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:17.557231  6502 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:18:17.546757  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:17.574306  6578 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
W20260709 12:18:17.551959  6622 connection.cc:256] RPC call timeout handler was delayed by -1.204975468999919s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16290000000000002s for a response
W20260709 12:18:17.574914  6622 connection.cc:256] RPC call timeout handler was delayed by -1.1640968300000623s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1758s for a response
W20260709 12:18:17.515049  9723 consensus_meta.cc:287] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.541s	user 0.000s	sys 0.012s
I20260709 12:18:17.575809  6475 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:17.596756  6475 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:18:17.553551  9843 log.cc:927] Time spent T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 12:18:17.541056  9845 log.cc:927] Time spent T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:18:17.541078  9821 raft_consensus.cc:3055] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:17.515254  9718 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:17.600510  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:17.601055  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:17.602769  6578 raft_consensus.cc:1240] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:17.603698  6652 raft_consensus.cc:1240] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:18:17.556592  9790 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:17.606257  6622 connection.cc:256] RPC call timeout handler was delayed by -1.0636754689999179s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1786s for a response
I20260709 12:18:17.549615  9659 raft_consensus.cc:3055] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:17.598232  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:17.524907  9852 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:18:17.627146  9669 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:17.604468  9575 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:17.627676  9669 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:17.629809  6578 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:17.549145  9763 raft_consensus.cc:3055] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:17.631407  9763 raft_consensus.cc:740] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:17.548219  9850 raft_consensus.cc:3055] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:17.632515  9850 raft_consensus.cc:740] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:17.633467  9850 consensus_queue.cc:260] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:17.634825  9850 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:17.636623  9733 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:17.642274  9717 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:17.643236  9381 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:17.643827  9619 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:17.644038  6651 raft_consensus.cc:2393] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 7: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:17.644416  9619 raft_consensus.cc:697] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 6 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:17.551733  9823 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:17.490314  9842 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:17.618585  9841 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.623123  9853 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:17.623427  9829 raft_consensus.cc:697] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:17.623787  9847 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:17.616196  9821 raft_consensus.cc:740] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:17.598376  9723 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:17.674851  6473 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:17.629884  9669 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:17.615170  6504 raft_consensus.cc:1240] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260709 12:18:17.693225  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 79061.5us
I20260709 12:18:17.647636  6579 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:17.654623  9825 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:17.656309  9842 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:17.620445  9659 raft_consensus.cc:740] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:17.664111  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:17.670399  6655 raft_consensus.cc:1240] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:17.688730  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:17.700969  9853 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.704895  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:17.705520  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:17.706215  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 199398us
I20260709 12:18:17.706629  6657 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:18:17.709964  6577 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:17.645345  9619 consensus_queue.cc:237] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:17.721112  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 46378us
I20260709 12:18:17.717242  6649 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:17.722038  6473 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:17.724223  6473 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:17.718508  6652 raft_consensus.cc:2410] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:18:17.718767  6657 pending_rounds.cc:85] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 6
I20260709 12:18:17.725636  6657 pending_rounds.cc:107] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:18:17.725852  6657 raft_consensus.cc:2889] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:17.727489  9842 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:17.727337  9860 consensus_queue.cc:1059] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:17.727948  9842 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:17.724510  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:17.735352  6655 raft_consensus.cc:1275] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:18:17.688599  9827 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:17.738013  6473 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:17.741946  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:18:17.742123  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 96587.8us
I20260709 12:18:17.680495  9820 consensus_queue.cc:1059] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 12:18:17.728397  9842 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:17.710510  9822 consensus_queue.cc:1059] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:18:17.688184  9841 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:17.786840  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:17.746734  9717 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:17.715780  9823 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:17.785284  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:17.799635  9604 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.753901  9574 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:17.740077  9852 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 12:18:17.725363  9763 consensus_queue.cc:260] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:17.863489  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:17.835505  6581 raft_consensus.cc:2393] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 6: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:17.779248  9733 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:17.763231  9773 consensus_queue.cc:1059] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.071s
I20260709 12:18:17.872701  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:17.740149  9853 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:17.912851  9648 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:17.913187  6581 raft_consensus.cc:1275] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 12:18:17.913566  6575 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1723) took 1445 ms (client timeout 1629 ms). Trace:
I20260709 12:18:17.873536  9820 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.862367  9840 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:17.863457  9717 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:18:17.926553  6575 rpcz_store.cc:269] 0709 12:18:16.471681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.475505 (+  3824us) service_pool.cc:224] Handling call
0709 12:18:17.913325 (+1437820us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54001cbe54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:17.913473 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2321819392}
I20260709 12:18:17.888136  9834 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:17.831748  9827 consensus_queue.cc:1059] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 12:18:17.897187  9480 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:17.936101  6622 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:17.884824  9847 consensus_queue.cc:1059] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 12:18:17.903389  9172 consensus_queue.cc:1059] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 12:18:17.831257  9829 consensus_queue.cc:237] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:17.924466  9841 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:17.903533  9604 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:17.964013  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 226003us
I20260709 12:18:17.939205  9723 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:17.923978  9826 consensus_queue.cc:1059] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 12:18:17.930318  9648 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:17.961414  9856 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:18:17.998081  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:17.939229  9717 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:18.006946  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.005363  9604 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:17.965055  9739 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:17.938977  9840 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:17.803149  9821 consensus_queue.cc:260] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:18.007421  9865 log.cc:927] Time spent T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:18:17.952129  9834 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:18.020082  9872 log.cc:927] Time spent T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:18:18.012210  9856 raft_consensus.cc:697] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:18.025434  9852 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:18.026356  9874 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:18:18.027734  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 120394us
W20260709 12:18:18.007723  9854 log.cc:927] Time spent T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.286s	user 0.000s	sys 0.001s
I20260709 12:18:18.063409  9874 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:18.034137  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:18.035503  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:18.064867  6658 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:18.037349  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:18.065732  9874 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:18.066395  6650 raft_consensus.cc:2393] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 7: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:18.041110  6649 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:18.070602  6649 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:18:18.070971  6649 pending_rounds.cc:85] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 4
I20260709 12:18:18.070348  9856 consensus_queue.cc:237] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:18.071178  6649 pending_rounds.cc:107] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:18:18.071421  6649 raft_consensus.cc:2889] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:17.963766  9480 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:18.074604  6657 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:18:18.074404  9659 consensus_queue.cc:260] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:18.075500  9840 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:18.076449  9464 raft_consensus.cc:3055] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:18.077009  6657 raft_consensus.cc:1240] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
W20260709 12:18:18.057462  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.065779  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:17.958832  9820 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:18.076563  9821 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:18.063445  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:18.007498  9648 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:18.068046  6650 raft_consensus.cc:1275] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 8 index: 3. (index mismatch)
I20260709 12:18:18.061014  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:18.010002  9733 raft_consensus.cc:697] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:18:18.061045  9859 log.cc:927] Time spent T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.299s	user 0.001s	sys 0.000s
W20260709 12:18:18.119782  9618 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.818s	user 0.009s	sys 0.010s
W20260709 12:18:18.129221  9618 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.845s	user 0.010s	sys 0.010s
I20260709 12:18:18.139210  9618 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:18:18.139638  9546 consensus_meta.cc:287] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.634s	user 0.008s	sys 0.000s
W20260709 12:18:18.140034  9863 log.cc:927] Time spent T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 12:18:18.140682  6650 raft_consensus.cc:1275] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:18:18.142088  9823 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:18.139873  9618 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:18.144579  6654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 1577) took 1091 ms. Trace:
I20260709 12:18:18.144953  6654 rpcz_store.cc:276] 0709 12:18:17.037443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.037627 (+   184us) service_pool.cc:224] Handling call
0709 12:18:18.060769 (+1023142us) spinlock_profiling.cc:243] Waited 7.26 ms on lock 0x7b5800015998. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:18:18.128250 (+ 67481us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16702336}
I20260709 12:18:18.145434  9618 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:18.153170  9853 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:18.074425  9480 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:18.157527  9834 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:18.164613  9820 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:18.166064  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 102698us
I20260709 12:18:18.170022  9739 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:18.174521  6655 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 10 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:18.168147  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:18.122267  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 216562us
I20260709 12:18:18.077203  9464 raft_consensus.cc:740] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:18.197686  6548 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:18.233080  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:18.266957  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:18.267375  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:18.268530  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:18.269021  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:18.201263  9850 consensus_meta.cc:287] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.566s	user 0.006s	sys 0.006s
I20260709 12:18:18.200546  9480 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.067862  9852 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:18.266366  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:18.282694  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:18.285669  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:18.276572  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:18.275906  9464 consensus_queue.cc:260] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:18.277985  9798 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:18.207880  9820 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:18.301702  6503 raft_consensus.cc:2368] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7.
I20260709 12:18:18.302393  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:18.295560  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:18.282823  6500 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:18:18.292003  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:18.284634  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:18.240181  9853 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.309940  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:18.313515  6508 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:18.273898  9546 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:18:18.322364  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 154388us
I20260709 12:18:18.317464  6502 raft_consensus.cc:1240] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:18.318918  6501 raft_consensus.cc:1275] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 3. (term mismatch)
I20260709 12:18:18.326200  6501 pending_rounds.cc:85] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 2
I20260709 12:18:18.326706  6501 pending_rounds.cc:107] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 7.3
I20260709 12:18:18.326990  6501 raft_consensus.cc:2889] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:18.308578  9480 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:18.300653  9738 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:18.321244  9820 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:18.315627  9798 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:18.322820  9773 consensus_queue.cc:1048] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:18.306423  9739 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.340790  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:18.340857  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:18.343686  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:18.346663  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:18.347549  6500 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:18.343281  9546 raft_consensus.cc:697] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:18.320119  9853 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:18.356578  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:18.361213  6502 raft_consensus.cc:2393] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 8: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:18.358454  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:18.367172  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:18.367846  6508 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:18:18.374146  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:18.376187  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:18.376838  6502 raft_consensus.cc:2376] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8. Re-sending same reply.
I20260709 12:18:18.378672  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:18.378875  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:18.379637  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:18.380281  6507 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:18.381498  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:18.382059  6548 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:18.385625  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:18.387645  6548 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:18.397156  9621 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:18.396082  9546 consensus_queue.cc:237] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:18.329157  9733 consensus_queue.cc:237] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:18.466850  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:18.392052  6475 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:18.468634  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 76672us
W20260709 12:18:18.470314  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 107865us
I20260709 12:18:18.469425  6475 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:18.475724  9856 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:18:18.476824  9895 log.cc:927] Time spent T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 12:18:18.476758  9876 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:18.477901  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:18.475096  6475 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:18.475541  9820 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:18.479125  6655 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:18.493253  6475 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:18.355948  9739 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:18.498503  6475 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:18.499053  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:18.402791  9888 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:18.501461  9888 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:18.469863  9819 raft_consensus.cc:3055] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:18.479889  9844 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:18.478504  9574 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:18.495913  9856 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:18.493213  9829 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:18.570770  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:18.572429  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1573) took 1601 ms. Trace:
I20260709 12:18:18.561964  9834 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:18.468149  9621 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:18:18.575248  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 296056us
I20260709 12:18:18.560654  9819 raft_consensus.cc:740] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
W20260709 12:18:18.583267  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 195693us
W20260709 12:18:18.583349  9900 log.cc:927] Time spent T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:18:18.469904  9826 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:18.496137  9876 consensus_queue.cc:1059] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 12:18:18.588797  9574 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:18.618806  6550 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:18.579138  6475 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.3:36107 timed out after 1.834s (SENT) [suppressed 18 similar messages]
I20260709 12:18:18.563337  9850 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:18.571559  9853 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:18.624701  6649 raft_consensus.cc:1275] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:18.503345  9880 consensus_queue.cc:1048] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 12:18:18.576269  9883 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:18.635371  6649 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:18:18.657191  9884 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.621485  9834 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:18.662467  9881 log.cc:927] Time spent T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.127s	user 0.000s	sys 0.013s
I20260709 12:18:18.621160  9829 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:18.622634  9621 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:18.632457  9850 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:18.789157  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 214536us
I20260709 12:18:18.707258  9480 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:18.830864  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 268619us
W20260709 12:18:18.799327  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 155123us
I20260709 12:18:18.833187  9464 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:18.833441  9819 consensus_queue.cc:260] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:18.834348  9480 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:18.835083  9819 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:18.801363  9898 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.230s	user 0.003s	sys 0.015s
W20260709 12:18:18.833483  9891 log.cc:927] Time spent T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.244s	user 0.005s	sys 0.003s
I20260709 12:18:18.837304  6653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2172) took 1168 ms. Trace:
I20260709 12:18:18.802232  9882 raft_consensus.cc:3055] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:18.837884  9882 raft_consensus.cc:740] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:18.838682  6622 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:18.839012  9882 consensus_queue.cc:260] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:18.806954  9886 log.cc:927] Time spent T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.334s	user 0.001s	sys 0.001s
I20260709 12:18:18.840219  6622 leader_election.cc:400] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 8. Current term is 9.
I20260709 12:18:18.747718  9826 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.074s
I20260709 12:18:18.841239  9856 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:18.808566  9508 raft_consensus.cc:3055] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:18.725042  9798 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:18.841890  9508 raft_consensus.cc:740] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:18.842475  9914 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:18.815621  9853 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:18.841782  9856 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:18.849073  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 269834us
I20260709 12:18:18.837755  6653 rpcz_store.cc:276] 0709 12:18:17.669018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.883821 (+214803us) service_pool.cc:224] Handling call
0709 12:18:17.906984 (+ 23163us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:17.927652 (+ 20668us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:17.927686 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:17.937637 (+  9951us) log.cc:844] Serialized 26 byte log entry
0709 12:18:18.246688 (+309051us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:18.363793 (+117105us) log.cc:844] Serialized 14 byte log entry
0709 12:18:18.364029 (+   236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:18.364101 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:18.837078 (+472977us) raft_consensus.cc:1685] finished
0709 12:18:18.837098 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:18.837242 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10944,"prepare.queue_time_us":155973,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":21166,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":6497,"thread_start_us":344189,"threads_started":2,"wal-append.queue_time_us":869698}
W20260709 12:18:18.824726  9842 consensus_meta.cc:287] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 1.038s	user 0.011s	sys 0.009s
I20260709 12:18:18.850035  7273 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38546 (request call id 370) took 1168 ms. Trace:
W20260709 12:18:18.845868  9821 consensus_meta.cc:287] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.727s	user 0.017s	sys 0.003s
I20260709 12:18:18.870647  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:18.832675  9850 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:18.876454  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:18.880038  6649 raft_consensus.cc:2410] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:18:18.853828  7273 rpcz_store.cc:276] 0709 12:18:17.676754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.679381 (+  2627us) service_pool.cc:224] Handling call
0709 12:18:17.714873 (+ 35492us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:17.888727 (+173854us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:17.888859 (+   132us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:17.948265 (+ 59406us) log.cc:844] Serialized 26 byte log entry
0709 12:18:18.262046 (+313781us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:18.262105 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:18.262218 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:18.844436 (+582218us) raft_consensus.cc:1685] finished
0709 12:18:18.844460 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:18.844786 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7985,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":17330,"thread_start_us":429799,"threads_started":2,"wal-append.queue_time_us":407045}
I20260709 12:18:18.851621  9882 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:18.845113  9888 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:18.813390  9884 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:18.903393  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 63317.5us
I20260709 12:18:18.906844  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:18.862076  9853 raft_consensus.cc:697] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 9 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:18.862135  9842 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:18.870606  9883 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:18.911474  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:18.843016  9508 consensus_queue.cc:260] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:18.919725  9834 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:18.925961  9899 log.cc:927] Time spent T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.350s	user 0.000s	sys 0.009s
I20260709 12:18:18.926257  9913 consensus_queue.cc:1059] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:18.928148  9914 raft_consensus.cc:3055] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:18.928689  9914 raft_consensus.cc:740] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:18.929896  9914 consensus_queue.cc:260] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:18.930984  9914 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:18.932267  9480 consensus_queue.cc:1059] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:18.933372  9834 raft_consensus.cc:3055] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:18.933763  9834 raft_consensus.cc:740] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:18.934664  9834 consensus_queue.cc:260] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:18.937862  9621 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:18.946344  9884 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:18.936787  6622 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:18.947904  9853 consensus_queue.cc:237] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:18.949465  9914 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:18.951074  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:18.952450  9917 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:18.953231  9914 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:18.953401  9917 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:18.896993  9907 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:18.906136  9893 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:18.901257  9880 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:18.978092  6506 consensus_meta.cc:287] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.681s	user 0.018s	sys 0.001s
I20260709 12:18:19.036801  6506 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:18:19.036954  6473 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:18.955955  9914 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:18:19.089979  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.113847  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1611) took 1641 ms. Trace:
I20260709 12:18:18.957579  9907 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:18.923632  9798 consensus_queue.cc:1059] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.192s
I20260709 12:18:19.064920  9884 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:19.114497  6580 rpcz_store.cc:276] 0709 12:18:17.472004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.472179 (+   175us) service_pool.cc:224] Handling call
0709 12:18:17.489702 (+ 17523us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:17.517865 (+ 28163us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:18:17.717858 (+199993us) log.cc:844] Serialized 14 byte log entry
0709 12:18:18.115498 (+397640us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:18.115971 (+   473us) log.cc:844] Serialized 26 byte log entry
0709 12:18:18.116102 (+   131us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:18.116136 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:18.116201 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:19.113552 (+997351us) raft_consensus.cc:1685] finished
0709 12:18:19.113579 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:19.113758 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27831,"prepare.queue_time_us":201319,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":8616,"raft.queue_time_us":12947,"raft.run_cpu_time_us":547,"raft.run_wall_time_us":12744,"thread_start_us":518847,"threads_started":2,"wal-append.queue_time_us":525384}
I20260709 12:18:19.061666  9852 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:19.044684  9836 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.105s
I20260709 12:18:19.129354  9917 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:18.923802  9883 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:19.196277  9915 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:19.196806  9458 raft_consensus.cc:3055] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:19.197239  9915 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:19.206110  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.118484  9619 consensus_queue.cc:1048] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.075s
W20260709 12:18:19.208694  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 219229us
I20260709 12:18:19.221570  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:19.221729  9919 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:19.222507  9919 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:19.230443  6622 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:19.234040  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:19.179318  9798 consensus_queue.cc:1243] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Peer f2b06aebe97249a4877d798b8118ece4 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:18:19.045490  9893 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:19.201383  9884 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:19.203514  9458 raft_consensus.cc:740] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:19.206759  9917 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.118695  9880 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:19.260900  6651 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 1568) took 2362 ms (client timeout 1834 ms). Trace:
W20260709 12:18:19.376235  6651 rpcz_store.cc:269] 0709 12:18:16.893080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.894739 (+  1659us) service_pool.cc:224] Handling call
0709 12:18:19.254776 (+2360037us) spinlock_profiling.cc:243] Waited 520 ms on lock 0x7b54003e99d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:19.254922 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1196002688}
I20260709 12:18:19.247553  9853 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:19.381314  9933 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:19.397888  9924 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:19.469766  9924 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:19.470383  9924 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:19.471082  9842 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:19.398716  9918 log.cc:927] Time spent T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
I20260709 12:18:19.394591  9829 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:19.478052  9880 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:19.396018  9912 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:19.443599  9836 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:19.467909  9458 consensus_queue.cc:260] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:19.378559  9917 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:19.469301  9915 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:19.381786  9884 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:19.462461  9914 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:18:19.517735  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 285602us
I20260709 12:18:19.466400  9920 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:18:19.519447  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 285457us
I20260709 12:18:19.405894  9763 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:19.436610  9853 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:19.626492  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:19.482263  9852 consensus_queue.cc:1048] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.129s
I20260709 12:18:19.638739  6577 raft_consensus.cc:1275] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:18:19.627518  6580 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:18:19.633744  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:19.634148  9853 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:19.634667  6581 raft_consensus.cc:1240] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 9 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 8. Current term is 9. Ops: []
W20260709 12:18:19.617810  9922 log.cc:927] Time spent T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 12:18:19.644835  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:19.645560  6575 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
W20260709 12:18:19.647425  9574 consensus_meta.cc:287] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 1.006s	user 0.012s	sys 0.000s
W20260709 12:18:19.648221  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:19.647884  9574 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:19.652588  9928 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:19.653488  9928 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:19.491612  9933 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:19.675230  9933 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:19.619966  9621 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:19.676645  9621 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:19.685364  9893 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:19.693761  9917 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:19.694566  9917 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.695122  9917 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:19.697757  9883 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:19.623117  9907 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:19.702530  9884 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:19.703224  9884 raft_consensus.cc:3055] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:19.703619  9884 raft_consensus.cc:740] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:19.498143  9912 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:19.707736  9912 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:19.710695  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:19.637277  9856 consensus_meta.cc:287] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.771s	user 0.009s	sys 0.005s
I20260709 12:18:19.712423  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:19.712174  9856 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:19.713820  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:19.714550  6581 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:19.714852  9856 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:19.680222  9919 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:19.716748  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:19.717535  6575 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:18:19.716107  6581 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:19.719086  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:19.719861  6584 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
W20260709 12:18:19.724565  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 112891us
I20260709 12:18:19.699987  9921 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:19.732321  6622 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:19.734139  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:19.734890  6575 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:19.760471  9884 consensus_queue.cc:260] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:19.763610  9926 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:18:19.746462  9915 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:19.767590  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:19.768224  6577 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
W20260709 12:18:19.755349  9733 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.033s	sys 0.069s
I20260709 12:18:19.771548  6575 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:19.775660  9880 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.776165  9880 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:19.777323  6504 raft_consensus.cc:1275] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:18:19.779726  9880 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:19.780675  9844 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:19.781308  9844 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:19.792188  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:19.794405  6575 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:18:19.794961  6575 pending_rounds.cc:85] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 3
I20260709 12:18:19.795249  6575 pending_rounds.cc:107] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:18:19.795559  6575 raft_consensus.cc:2889] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:19.797380  9926 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:19.798081  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:19.798678  6652 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
W20260709 12:18:19.799252  6652 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 1603) took 1415 ms (client timeout 1783 ms). Trace:
W20260709 12:18:19.772634  9882 consensus_meta.cc:287] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.871s	user 0.013s	sys 0.000s
W20260709 12:18:19.800065  6652 rpcz_store.cc:269] 0709 12:18:18.384091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.387008 (+  2917us) service_pool.cc:224] Handling call
0709 12:18:19.798021 (+1411013us) spinlock_profiling.cc:243] Waited 44 ms on lock 0x7b5c0023428c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f28797f3161 00007f2879994a79 00007f287ae4e9b6 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:19.799180 (+  1159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":101158272}
I20260709 12:18:19.745359  9912 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.028s
W20260709 12:18:19.825405  9850 consensus_meta.cc:287] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.933s	user 0.008s	sys 0.005s
I20260709 12:18:19.825937  9850 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:19.745142  9921 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:19.833421  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:19.834053  6579 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
W20260709 12:18:19.838240  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 51995.9us
I20260709 12:18:19.821028  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:19.839548  6575 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:18:19.819912  9928 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:19.898340  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:19.799217  6622 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:19.900843  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:19.931469  6575 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:19.939395  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 140224us
I20260709 12:18:19.963092  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 384) took 2273 ms. Trace:
I20260709 12:18:19.963649  6578 rpcz_store.cc:276] 0709 12:18:17.651678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.651815 (+   137us) service_pool.cc:224] Handling call
0709 12:18:17.652941 (+  1126us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:17.889354 (+236413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:17.889406 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:17.889472 (+    66us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:17.889544 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:19.924576 (+2035032us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b5400420254. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:19.924641 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:19.924848 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":495551232}
I20260709 12:18:19.910032  9882 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:19.999310  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:19.999480  9882 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:20.000283  6578 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:20.002626  9882 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:19.908811  9920 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:19.980810  9915 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:19.918594  9934 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:20.026326  9850 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:20.072649  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:20.073778  6578 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:20.130475  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:20.032943  9920 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:20.038177  9934 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:20.126123  9915 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:20.159080  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:20.162820  9574 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:20.164942  9480 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:20.165094  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 2.39216e+06us
I20260709 12:18:20.165710  9480 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:20.180619  9860 raft_consensus.cc:3055] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:20.180449  9815 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:20.181206  9860 raft_consensus.cc:740] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:20.181670  9508 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:20.181602  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:20.182180  9860 consensus_queue.cc:260] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:20.157459  9915 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:20.195727  9821 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.196322  9821 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:20.197304  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:20.197870  9819 consensus_meta.cc:287] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 1.362s	user 0.014s	sys 0.001s
I20260709 12:18:20.198287  7273 raft_consensus.cc:2410] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:18:20.196748  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:20.205595  7270 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:20.213665  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:20.214620  7268 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
W20260709 12:18:20.248066  6580 consensus_meta.cc:287] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.602s	user 0.008s	sys 0.000s
I20260709 12:18:20.250610  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:20.253665  6580 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
W20260709 12:18:20.252533  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.231840  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:20.255092  9860 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:20.264418  6575 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:20.261965  7270 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:18:20.278429  6578 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:20.280098  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:20.282331  6575 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:18:20.284227  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:20.196928  9822 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:20.207604  9926 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.215893  9739 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:20.262552  9821 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:20.277808  9773 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.277383  9623 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:18:20.304705  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 407663us
W20260709 12:18:20.299660  9924 consensus_meta.cc:287] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.829s	user 0.009s	sys 0.005s
W20260709 12:18:20.319140  6622 connection.cc:256] RPC call timeout handler was delayed by -1.1306701159999193s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1533s for a response
W20260709 12:18:20.319814  6622 connection.cc:256] RPC call timeout handler was delayed by -1.0847701159998451s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1584s for a response
I20260709 12:18:20.288177  9800 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:20.324900  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:20.299283  9480 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:20.327414  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.329061  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:20.330070  7273 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:18:20.298877  9822 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:20.333716  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:20.287606  9878 consensus_queue.cc:1059] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:18:20.299916  9815 consensus_queue.cc:1059] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:18:20.335708  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:20.340096  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:20.311000  9821 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:18:20.340760  6583 raft_consensus.cc:2376] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7. Re-sending same reply.
I20260709 12:18:20.341382  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:20.323745  9924 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:20.343454  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:20.345965  7269 raft_consensus.cc:2410] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:18:20.323792  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:20.308908  9773 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:20.317418  9623 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:20.298785  9926 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:20.328438  9739 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 12:18:20.330068  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:20.339617  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:20.354315  9819 raft_consensus.cc:3055] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:20.352842  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:20.358202  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:20.359403  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:20.356038  9574 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.369700  7273 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:18:20.345141  9920 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:20.376030  9942 log.cc:927] Time spent T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.220s	user 0.003s	sys 0.000s
I20260709 12:18:20.377619  9623 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:20.377698  9773 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:20.352401  9815 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:20.346552  9934 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:20.332760  9800 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:20.374042  9950 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:18:20.402187  9844 consensus_meta.cc:287] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.620s	user 0.009s	sys 0.004s
I20260709 12:18:20.402252  6658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1596) took 2020 ms. Trace:
I20260709 12:18:20.402838  9928 consensus_queue.cc:1048] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:18:20.410043  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 78897.3us
I20260709 12:18:20.410313  9954 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:20.353219  9915 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:20.413324  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 75808.2us
I20260709 12:18:20.369324  9819 raft_consensus.cc:740] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:20.368185  9574 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:20.406759  9912 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:20.421244  6658 rpcz_store.cc:276] 0709 12:18:18.382073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.382284 (+   211us) service_pool.cc:224] Handling call
0709 12:18:18.382527 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:18.382702 (+   175us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:18:20.361447 (+1978745us) spinlock_profiling.cc:243] Waited 1.5 ms on lock 0x7b18002544a0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2875a314bc 00007f2872d7f48b 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d99455 00007f28798813c7 00007f2875e939a7 00007f2875ea30bf
0709 12:18:20.361580 (+   133us) log.cc:844] Serialized 14 byte log entry
0709 12:18:20.401900 (+ 40320us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:20.401951 (+    51us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:20.401998 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:20.402074 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:20.402194 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":116711,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":2853,"spinlock_wait_cycles":3447552,"thread_start_us":238902,"threads_started":2,"wal-append.queue_time_us":40256}
W20260709 12:18:20.424455  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 99637.5us
I20260709 12:18:20.431272  6473 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:20.417179  9815 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:20.424641  9844 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:20.441534  9822 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:20.438823  9920 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 12:18:20.442394  6549 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:20.449523  9844 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:20.449070  9819 consensus_queue.cc:260] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:20.449990  9518 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:18:20.450173  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 75706.1us
I20260709 12:18:20.450496  9819 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:20.451910  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:20.453217  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:20.453893  6583 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 12.
I20260709 12:18:20.445716  9800 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:20.444217  9815 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:20.436275  9882 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:20.454877  9882 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:20.455289  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:20.455314  9882 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:20.456743  6581 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:20.460139  9926 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:20.460369  6577 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:20.461361  9924 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:20.464251  9882 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:20.422806  9950 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:20.438107  9912 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:20.472563  9458 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:20.466713  9882 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:20.476593  9574 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:20.481740  6504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 1946) took 2172 ms. Trace:
I20260709 12:18:20.482193  6504 rpcz_store.cc:276] 0709 12:18:18.308914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.315778 (+  6864us) service_pool.cc:224] Handling call
0709 12:18:18.337973 (+ 22195us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:19.777663 (+1439690us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:20.481459 (+703796us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b54002c42d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:20.481650 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3309269888}
I20260709 12:18:20.487155  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:20.488247  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:20.488287  7266 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:20.488960  7265 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:20.489859  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:20.489950  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:20.490674  7266 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:20.491003  7265 raft_consensus.cc:2410] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:18:20.491521  9882 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:20.487260  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:20.492033  9882 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:20.493986  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:20.494117  9882 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:20.495260  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:20.502151  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.504511  9928 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:20.505113  9928 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:20.507711  9928 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:20.509573  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:20.510017  6505 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:20.510989  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:20.511399  6507 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:18:20.514917  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 1.89303e+06us
I20260709 12:18:20.517985  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:20.518632  6503 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 12.
I20260709 12:18:20.519640  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:20.520126  9954 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:20.520872  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:20.521713  6502 raft_consensus.cc:2393] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 9: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:20.522734  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:20.522879  6508 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:20.524472  6504 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:20.524607  9790 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:20.515671  6548 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:20.526046  6474 connection.cc:256] RPC call timeout handler was delayed by -1.1767304699999386s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20120000000000002s for a response
I20260709 12:18:20.529184  6548 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:20.530493  6548 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:20.530977  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:20.531570  6658 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
I20260709 12:18:20.531615  6548 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:20.533126  6548 leader_election.cc:400] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7.
I20260709 12:18:20.533408  6548 leader_election.cc:403] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:20.533308  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:20.537281  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 115498us
I20260709 12:18:20.537343  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:20.538430  6623 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:20.538691  6658 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 4.
I20260709 12:18:20.540560  7240 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:20.543161  6548 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:20.548233  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:20.548532  7240 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:20.536908  9514 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:20.535228  9888 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:20.535058  9775 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:20.552603  6474 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:20.536485  9508 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:20.538228  9518 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7.
I20260709 12:18:20.563306  7240 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:20.556147  9939 log.cc:927] Time spent T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.559s	user 0.004s	sys 0.000s
I20260709 12:18:20.549983  9718 consensus_queue.cc:1059] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 12:18:20.555636  9823 raft_consensus.cc:2764] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:18:20.551764  9821 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:20.565038  6548 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:20.555828  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:20.567945  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.554420  9957 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:20.553928  9920 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:20.572350  6506 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 1937) took 2349 ms (client timeout 1646 ms). Trace:
I20260709 12:18:20.550133  9773 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
W20260709 12:18:20.573843  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.549975  9928 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:20.581714  6474 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:20.598774  6506 rpcz_store.cc:269] 0709 12:18:18.222507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.226695 (+  4188us) service_pool.cc:224] Handling call
0709 12:18:20.572128 (+2345433us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b540039f654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:20.572240 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39222656}
I20260709 12:18:20.579941  9822 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:20.559303  9815 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
W20260709 12:18:20.602536  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 169749us
I20260709 12:18:20.593225  9876 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:20.608268  6548 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:20.595814  9821 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:18:20.603008  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 172993us
I20260709 12:18:20.576283  9508 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:20.604422  9964 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:20.576527  9619 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:18:20.597581  9957 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:20.611346  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:20.612000  6622 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:20.612160  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 74038.4us
I20260709 12:18:20.616204  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:20.559273  9850 consensus_queue.cc:1059] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 12:18:20.578145  9800 consensus_queue.cc:1059] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:18:20.622726  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:20.626269  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 44620.7us
I20260709 12:18:20.605805  6473 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:20.614543  9518 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:20.629542  9954 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:20.629453  9964 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:20.630168  9954 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:20.630695  6650 raft_consensus.cc:2393] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 7: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:20.630949  9954 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:20.632277  9964 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:20.632988  9964 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:20.614871  9821 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:20.633744  9964 consensus_queue.cc:1059] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:20.608691  9876 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:20.539672  9648 consensus_queue.cc:1175] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "496c7c3792cd4587b4dbeba435f4be1e" responder_term: 8 status { last_received { term: 7 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 7. Current term is 8. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:18:20.651510  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:20.651793  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:20.652135  6502 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:20.611855  9960 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:20.612169  9775 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:20.602000  9920 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:20.656471  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:20.648685  6473 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:20.660301  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:20.658210  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:20.661092  6504 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
W20260709 12:18:20.672667  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 64479.5us
I20260709 12:18:20.670068  6650 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:20.673087  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:20.673501  6548 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:20.673684  6653 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:20.636300  9957 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:20.647917  9844 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:20.615265  9508 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:20.654312  9876 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:20.689500  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:20.690083  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 77740.9us
I20260709 12:18:20.694305  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:20.655813  9955 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.694950  6501 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
I20260709 12:18:20.699088  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:20.615036  9928 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:20.699405  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:20.700205  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 51638.7us
I20260709 12:18:20.700232  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:20.702713  6501 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
I20260709 12:18:20.703588  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:20.674196  9574 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:20.708482  6473 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:20.708539  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:20.709760  6507 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 7: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:20.677055  9800 consensus_queue.cc:1243] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 12:18:20.717286  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 76202.6us
I20260709 12:18:20.718993  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:20.725924  6653 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:18:20.720310  6504 raft_consensus.cc:2393] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 3: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:20.719470  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:20.729971  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:20.731112  6653 raft_consensus.cc:1240] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 5. Current term is 6. Ops: []
W20260709 12:18:20.736542  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:20.707197  9574 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:20.738495  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:20.740797  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:20.702332  9775 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:20.707823  9882 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:20.752671  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:20.710577  9844 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:20.742633  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:20.743659  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:20.746214  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:20.757841  9882 raft_consensus.cc:697] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:20.758287  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:20.759752  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:20.747845  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:20.704135  9955 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:20.716760  9518 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.754343  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:20.752400  6550 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:20.779515  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 106047us
I20260709 12:18:20.780593  6548 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:20.759279  9882 consensus_queue.cc:237] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:20.739316  6503 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:18:20.710817  6473 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:20.782792  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:20.742130  9574 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:20.812644  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 101883us
I20260709 12:18:20.813623  6473 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:20.818137  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:20.716732  9965 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.081s
I20260709 12:18:20.822671  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:20.698019  9960 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:20.751309  9773 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:18:20.827725  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 76227.6us
I20260709 12:18:20.828820  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:20.835079  6651 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:20.829149  6507 raft_consensus.cc:2393] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 9: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:20.823530  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:20.755643  9844 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:20.815177  9718 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:20.820909  9958 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:20.818149  9850 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.863798  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:20.787638  9518 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:20.843694  9844 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:20.878809  7265 raft_consensus.cc:3055] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:20.881409  7265 raft_consensus.cc:740] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:20.882994  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:20.873872  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:20.854162  9823 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:20.864185  9958 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:20.870270  9834 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:20.843278  9773 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:20.939981  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 104679us
I20260709 12:18:20.874603  9850 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:20.900030  9958 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:20.941993  6623 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:20.941164  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:20.948369  6502 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
W20260709 12:18:20.956007  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 175496us
I20260709 12:18:20.934235  6651 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:20.957861  6548 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:20.965255  9930 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:20.965862  9930 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:20.966281  9930 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:20.967583  9988 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:20.968258  9988 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:20.971026  9988 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:20.983999  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:20.985085  6651 raft_consensus.cc:2393] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 6: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:18:20.994833  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 181277us
I20260709 12:18:20.988775  9823 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:20.964076  9717 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:21.000823  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 58925.6us
I20260709 12:18:21.003657  6473 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:21.005110  6473 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:20.991741  9928 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:21.028251  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:21.029170  6651 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:21.033968  6473 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:18:21.017256  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 59506.5us
I20260709 12:18:21.052965  9717 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:21.054224  9717 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:21.054608  9717 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:21.054677  9984 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:21.055321  9984 raft_consensus.cc:697] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:20.982502  7265 consensus_queue.cc:260] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:21.056290  9984 consensus_queue.cc:237] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:21.057132  7265 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:21.051276  6548 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:21.051525  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 110425us
I20260709 12:18:21.060762  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:18.589047  6656 rpcz_store.cc:276] 0709 12:18:16.894577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:16.894866 (+   289us) service_pool.cc:224] Handling call
0709 12:18:16.960391 (+ 65525us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:17.039048 (+ 78657us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:17.039079 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:17.043268 (+  4189us) log.cc:844] Serialized 26 byte log entry
0709 12:18:18.275481 (+1232213us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b18002c6bc0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:18:18.380530 (+105049us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:18.488238 (+107708us) log.cc:844] Serialized 14 byte log entry
0709 12:18:18.489791 (+  1553us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:18.489895 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:18.489954 (+    59us) raft_consensus.cc:1685] finished
0709 12:18:18.495966 (+  6012us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:18.496096 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":826,"prepare.queue_time_us":105766,"prepare.run_cpu_time_us":1992,"prepare.run_wall_time_us":31979,"raft.queue_time_us":1636,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":43250,"spinlock_wait_cycles":5557120,"thread_start_us":212156,"threads_started":2,"wal-append.queue_time_us":1387582}
W20260709 12:18:21.086751  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 89532.6us
I20260709 12:18:21.088887  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:21.102494  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 61748.5us
I20260709 12:18:21.103856  6548 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:21.096208  6622 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:21.107582  6622 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:21.048451  9955 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:21.109397  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:21.046360  9982 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:21.013957  9979 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.097s
I20260709 12:18:21.071997  9798 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:21.071506  9988 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:21.137059  6651 raft_consensus.cc:2410] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 9 because replica has last-logged OpId of term: 8 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 2.
W20260709 12:18:21.147152  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:21.129973  6548 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:21.146274  9798 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:21.155704  9798 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:21.157648  6548 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:21.111193  9973 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:21.110553  9823 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:18:21.164290  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 130439us
I20260709 12:18:21.166572  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:21.167368  6584 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:21.161787  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:21.169832  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:21.195951  6548 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:21.127002  9987 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:21.138435  9993 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:21.203663  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:21.203600  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:21.206133  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:21.150146  9982 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:18:21.206967  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 389375us
I20260709 12:18:21.209842  6583 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:21.203962  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:21.160115  9518 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:21.216800  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:21.218501  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:21.219305  6651 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:18:21.238201  9988 raft_consensus.cc:697] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:21.239657  9988 consensus_queue.cc:237] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:21.193562  9973 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:21.249533  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:21.248268  6550 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:21.191370  9960 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:21.193071  9981 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:21.210928  9514 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:21.211150  9773 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:21.276077  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:21.212612  9993 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:18:21.259712  9975 log.cc:927] Time spent T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.322s	user 0.001s	sys 0.000s
I20260709 12:18:21.259940  9973 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:21.274933  9972 log.cc:927] Time spent T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.328s	user 0.001s	sys 0.001s
I20260709 12:18:21.287241  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:21.313931  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:21.314055  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:21.314550  6658 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:21.258652  9775 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:21.315804  6577 raft_consensus.cc:2393] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 8: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:21.263861  9955 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:21.275710  9850 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:21.247933  9930 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:21.337659  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:21.312469  9773 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:21.288699  9993 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:21.357012  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:21.358748  6577 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:21.331773  9823 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:21.279556  7267 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
W20260709 12:18:21.396258  9954 consensus_meta.cc:287] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.765s	user 0.007s	sys 0.005s
I20260709 12:18:21.396986  9621 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:21.396893  9954 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:21.370194  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:21.355228  9773 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:21.381388  9991 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:18:21.382179  9823 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:21.362071  9985 log.cc:927] Time spent T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
I20260709 12:18:21.385924  7265 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 12:18:21.386889  9821 consensus_meta.cc:287] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.753s	user 0.010s	sys 0.000s
W20260709 12:18:21.387445  9458 consensus_meta.cc:287] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.914s	user 0.013s	sys 0.000s
I20260709 12:18:21.390455  9950 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:21.390606  9930 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:21.390869  9957 consensus_meta.cc:287] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.693s	user 0.008s	sys 0.000s
I20260709 12:18:21.334844  9955 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:21.366811  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:21.410123  7238 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:21.413091  6583 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:21.415019  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:18:21.420907  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 224989us
W20260709 12:18:21.437861  6475 leader_election.cc:336] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Timed out: RequestConsensusVote RPC to 127.5.58.3:36107 timed out after 1.834s (SENT)
W20260709 12:18:21.396315  9508 consensus_meta.cc:287] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.687s	user 0.003s	sys 0.005s
I20260709 12:18:21.497009  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:21.497530  6579 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:18:21.416364  9773 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:21.509366  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 99433.3us
W20260709 12:18:21.503158  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 65363.1us
I20260709 12:18:21.502211  6577 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:21.510747  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:21.511483  6577 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:21.420565  9884 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:21.516503  6652 raft_consensus.cc:1240] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:18:21.514046  7238 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:21.414387  9981 raft_consensus.cc:697] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:21.519677  6652 raft_consensus.cc:1240] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:21.527659  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:21.511752  9514 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.516947  9580 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:21.417052  9798 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:21.530117  9876 consensus_meta.cc:287] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.828s	user 0.007s	sys 0.004s
I20260709 12:18:21.551898  9955 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:21.555408  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:21.558457  7273 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38562 (request call id 583) took 1229 ms (client timeout 1604 ms). Trace:
I20260709 12:18:21.564225  9917 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:21.566560  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:21.569546  7265 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
I20260709 12:18:21.569988  9981 consensus_queue.cc:237] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:21.569574  7273 rpcz_store.cc:269] 0709 12:18:20.328560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.332667 (+  4107us) service_pool.cc:224] Handling call
0709 12:18:21.558207 (+1225540us) spinlock_profiling.cc:243] Waited 31.2 ms on lock 0x7b54005ddc54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:21.558347 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":71791360}
I20260709 12:18:21.571357  6623 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:21.572723  9955 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:21.573263  9955 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:21.573747  9955 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:21.581893  9823 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:21.583869  9954 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:21.585599  6584 raft_consensus.cc:2393] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 7: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:21.585633  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:21.586786  9798 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:21.586931  9955 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:21.593842  6622 leader_election.cc:336] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.570s (SENT)
W20260709 12:18:21.608541  9834 consensus_meta.cc:287] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.675s	user 0.009s	sys 0.002s
W20260709 12:18:21.610122  9958 consensus_meta.cc:287] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.649s	user 0.004s	sys 0.005s
I20260709 12:18:21.532073  9950 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:21.508927  9957 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:21.516587  9508 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:21.496925  9821 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:21.567785  9514 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:21.614248  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:21.614522  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:21.615545  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:21.616750  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:18:21.656533  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 62764.3us
I20260709 12:18:21.656664  9921 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:21.657975  6579 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 1649) took 1761 ms (client timeout 2208 ms). Trace:
W20260709 12:18:21.658387  6579 rpcz_store.cc:269] 0709 12:18:19.896243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:19.896507 (+   264us) service_pool.cc:224] Handling call
0709 12:18:21.657804 (+1761297us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b5400031e54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:21.657915 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3603200256}
W20260709 12:18:21.660234  9490 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 12:18:21.661223  9957 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:21.661448  9920 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:21.617605  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:21.639511  9739 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.531854  9717 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:21.662757  9739 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:21.664026  6506 raft_consensus.cc:2410] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:18:21.619035  9876 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:21.539319  9574 consensus_meta.cc:287] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.702s	user 0.009s	sys 0.002s
I20260709 12:18:21.674254  6650 raft_consensus.cc:2410] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 12:18:21.548135  9844 consensus_meta.cc:287] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.652s	user 0.010s	sys 0.001s
W20260709 12:18:21.678356  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 164420us
W20260709 12:18:21.664978  9990 log.cc:927] Time spent T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.449s	user 0.003s	sys 0.000s
I20260709 12:18:21.666294  6505 raft_consensus.cc:1240] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:21.654943  6652 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:21.654980  6658 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:21.664242  9717 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:21.624415  6475 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:21.735020  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 225026us
I20260709 12:18:21.725013  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1598) took 3334 ms. Trace:
I20260709 12:18:21.650226  9958 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:21.735747  6655 rpcz_store.cc:276] 0709 12:18:18.382617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:18.501869 (+119252us) service_pool.cc:224] Handling call
0709 12:18:18.502222 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:18.859846 (+357624us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:18.859894 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:18.859958 (+    64us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:18.860010 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:21.716772 (+2856762us) spinlock_profiling.cc:243] Waited 357 ms on lock 0x7b54000744d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:21.716834 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:21.717009 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":821769600}
I20260709 12:18:21.737255  9958 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:21.737802  9958 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:21.738070  6473 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:21.738245  9958 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:21.741286  9882 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:21.741704  9882 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:21.741806  9958 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:21.742120  9882 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:21.742707  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:21.714582  6580 raft_consensus.cc:2393] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 5: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:18:21.735121  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 110837us
W20260709 12:18:21.746170  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 434252us
I20260709 12:18:21.747635  6475 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:21.639019  9950 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:21.748816  6655 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:21.749083  9950 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:21.749285  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:21.749933  6658 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:21.752733  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:21.738916  6473 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:21.739099  9580 raft_consensus.cc:2955] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Committing config change with OpId 5.6: config changed from index 3 to 6, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:18:21.703118  9844 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:21.745827  9988 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:21.761530  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:21.762068  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:18:21.727208  9983 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.000s	sys 0.029s
I20260709 12:18:21.704412  9574 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:21.762305  6475 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:21.715097  6622 leader_election.cc:336] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.661s (SENT)
I20260709 12:18:21.764001  6655 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:21.714912  9973 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:21.800616  6473 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:21.800654  6550 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:21.812077  6508 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:21.812183  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:21.812932  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:21.814620 10012 log.cc:927] Time spent T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:18:21.817055  9508 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:21.818468  7268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38562 (request call id 580) took 1532 ms. Trace:
I20260709 12:18:21.818689 10026 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 12:18:21.819288  9574 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:21.820031  6504 raft_consensus.cc:2393] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 10: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:21.822528  6579 raft_consensus.cc:2955] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 3 to 6, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:18:21.854156  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:18:21.855159  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 140061us
I20260709 12:18:21.850072  7268 rpcz_store.cc:276] 0709 12:18:20.285811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.291002 (+  5191us) service_pool.cc:224] Handling call
0709 12:18:20.291534 (+   532us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:20.427425 (+135891us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:20.427465 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:20.427510 (+    45us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:20.427545 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:21.818227 (+1390682us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b54005bd6d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:21.818301 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:21.818415 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":311977344}
I20260709 12:18:21.824015  9955 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:21.825968  9821 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:21.826509  9514 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:21.759598 10015 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:21.876562 10015 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:21.877010 10015 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:21.879776  7267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38558 (request call id 503) took 1393 ms. Trace:
I20260709 12:18:21.760787  9876 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:21.882349  6505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 1878) took 1186 ms. Trace:
I20260709 12:18:21.882983  9844 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:21.884721  9973 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:21.828178  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 65975.4us
I20260709 12:18:21.835515  9917 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:21.840688  6502 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
I20260709 12:18:21.882745  6505 rpcz_store.cc:276] 0709 12:18:20.696399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.701662 (+  5263us) service_pool.cc:224] Handling call
0709 12:18:20.714758 (+ 13096us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:21.745819 (+1031061us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:21.882089 (+136270us) spinlock_profiling.cc:243] Waited 951 ms on lock 0x7b5400387554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:21.882257 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2187850496}
I20260709 12:18:21.829447  6653 raft_consensus.cc:1240] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 8. Current term is 9. Ops: []
W20260709 12:18:21.859119 10020 log.cc:927] Time spent T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:18:21.861284  6508 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:21.862527  9993 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:21.869138 10021 log.cc:927] Time spent T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:18:21.887846  6475 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:21.805248  9842 raft_consensus.cc:2804] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:18:21.890944  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 90270.7us
I20260709 12:18:21.899699  6473 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:21.901124  6473 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:18:21.903666  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 104184us
I20260709 12:18:21.904562  6548 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:21.904559  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 103961us
I20260709 12:18:21.907231  6505 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:21.909127  6475 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:21.911753  6473 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:21.913679  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:21.913841 10014 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:21.914539 10014 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:21.915022 10014 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:21.915737  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:21.916389  6501 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:21.917594  9917 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:21.918120  9917 raft_consensus.cc:697] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:21.918174  9996 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:21.911376  9574 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:21.919107  9917 consensus_queue.cc:237] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:21.919425  9574 raft_consensus.cc:697] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:21.921147  9574 consensus_queue.cc:237] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:21.922111 10016 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:21.922678 10016 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:21.922909  9775 consensus_queue.cc:1059] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:21.893316  9844 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:21.924734  9981 raft_consensus.cc:3055] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:21.924831  9844 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:21.924021  9775 consensus_queue.cc:1059] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 8.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:18:21.925163  9981 raft_consensus.cc:740] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:21.925302  9844 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:21.925639  9775 raft_consensus.cc:3055] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:21.926010  9775 raft_consensus.cc:740] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:21.926158  9981 consensus_queue.cc:260] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:21.926693 10016 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:21.929934  9981 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:21.930048 10007 consensus_queue.cc:1048] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 12:18:21.930840 10026 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:21.931421 10026 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:21.931782 10026 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:21.932717  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:21.933350  6653 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:18:21.906208  9621 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:21.937451  6475 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:21.938241  9876 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:18:21.880187  7267 rpcz_store.cc:276] 0709 12:18:20.486714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.486849 (+   135us) service_pool.cc:224] Handling call
0709 12:18:21.879583 (+1392734us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b5400507dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:21.879711 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":782595968}
I20260709 12:18:21.926981  9775 consensus_queue.cc:260] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.3, Last appended by leader: 3, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:21.939352  6475 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:21.942806  6652 raft_consensus.cc:1240] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:21.944360 10014 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:21.947870 10019 log.cc:927] Time spent T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 12:18:21.949281 10014 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:21.949889 10013 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 12:18:21.950450 10016 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:21.959370  9957 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:21.959965  9957 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:21.960291  9619 raft_consensus.cc:3055] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:21.964865  9619 raft_consensus.cc:740] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:21.966701  9619 consensus_queue.cc:260] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:21.967749  9619 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:21.969511  6506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 1866) took 1331 ms. Trace:
I20260709 12:18:21.969918  6506 rpcz_store.cc:276] 0709 12:18:20.638590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.641138 (+  2548us) service_pool.cc:224] Handling call
0709 12:18:21.969311 (+1328173us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540046c8d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:21.969447 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2314351488}
W20260709 12:18:21.962025  7269 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38546 (request call id 372) took 1766 ms (client timeout 1775 ms). Trace:
W20260709 12:18:21.971094  7269 rpcz_store.cc:269] 0709 12:18:20.196079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.196476 (+   397us) service_pool.cc:224] Handling call
0709 12:18:21.961846 (+1765370us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b54004e7ad4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:21.961974 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":332702464}
I20260709 12:18:21.971884  9888 raft_consensus.cc:3055] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:21.972319  9888 raft_consensus.cc:740] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:21.973183  9888 consensus_queue.cc:260] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:21.974609  9912 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:21.976472  9993 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:21.977512  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:21.978142  7267 raft_consensus.cc:2376] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8. Re-sending same reply.
I20260709 12:18:21.982530  6473 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:21.984077  6504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 1884) took 1235 ms. Trace:
W20260709 12:18:21.984691  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 45436.7us
I20260709 12:18:21.985371  9965 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:21.985531  6475 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:21.986644  6475 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:21.986402  9773 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:21.987069  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:21.987679  6475 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:21.987854  6577 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:18:21.988492  9958 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:21.989090  9958 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:21.989396  9958 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:21.989723  9958 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:21.989630  9773 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:21.993434  9920 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:21.999766  9983 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.000592  6583 raft_consensus.cc:1240] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:22.000273  9983 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:22.002462  9983 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:22.004455  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.008348  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:21.963150  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:22.008996  6575 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:21.963676  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:21.963896  7274 raft_consensus.cc:1240] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:21.984709 10014 raft_consensus.cc:2764] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:18:22.001668  6475 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:22.012696  6473 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:22.014271  7273 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
I20260709 12:18:21.984503  6504 rpcz_store.cc:276] 0709 12:18:20.748859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.796692 (+ 47833us) service_pool.cc:224] Handling call
0709 12:18:21.983867 (+1187175us) spinlock_profiling.cc:243] Waited 983 ms on lock 0x7b54001e87d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:21.984004 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2260932352}
I20260709 12:18:21.974380  9888 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:18:22.024647  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 82307.5us
I20260709 12:18:22.015590  6623 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:22.028039  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:22.035805  6584 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:22.031345  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:18:22.037595  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 133217us
I20260709 12:18:21.962129 10016 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:22.045326  6473 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:22.034487  6550 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:22.058238  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:22.061566  6575 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
W20260709 12:18:22.063743  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 48266.6us
I20260709 12:18:22.065711  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:22.066928  7269 raft_consensus.cc:1240] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:21.904066  9842 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:22.081555  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:22.082499  7268 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:22.083305  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:22.083864  6580 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:22.089097  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:18:22.097714  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 52486.9us
W20260709 12:18:22.099102  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 57906.2us
I20260709 12:18:22.099220  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:22.099946  7270 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:22.102301  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:22.084860  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:22.112528  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:22.113270  6576 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:18:21.967660  9739 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:22.052623  9621 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:22.122737  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:22.063001  9997 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:22.073891  9623 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.082866  9508 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:22.131197  6584 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:22.118383  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:22.094048  9775 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:22.082710  9842 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:22.142596  6575 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:22.144685  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:22.157150  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 57291.5us
I20260709 12:18:22.158059  6549 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:22.140539  9623 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:22.160669  7265 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 12:18:22.160970 10025 log.cc:927] Time spent T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.252s	user 0.001s	sys 0.002s
I20260709 12:18:22.161607  9959 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:22.162230  9959 raft_consensus.cc:697] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 6 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:22.163169  9959 consensus_queue.cc:237] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:22.164458  9929 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:22.126456  9621 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:22.169273  6580 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:22.170224  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:22.170830  6580 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:22.141718  9508 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:22.111227  9876 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:18:22.172544  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 61915us
I20260709 12:18:22.179407  6576 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:22.183760  7240 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:22.147003  9983 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:22.184983  7240 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:22.122430  9965 consensus_queue.cc:1059] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.068s
I20260709 12:18:22.127683  9875 consensus_queue.cc:1059] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 12:18:22.212445  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:22.212544  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:22.217540  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:22.172549  9621 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:22.222114  6580 raft_consensus.cc:2376] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12. Re-sending same reply.
I20260709 12:18:22.165997 10011 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:22.228065  9875 raft_consensus.cc:3055] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:22.228662  9875 raft_consensus.cc:740] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:22.229632  9875 consensus_queue.cc:260] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:22.231853  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:22.232695  9893 raft_consensus.cc:3055] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:22.233114  9893 raft_consensus.cc:740] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:22.233469  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:22.233829  9893 consensus_queue.cc:260] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:22.234228  6583 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
I20260709 12:18:22.235024  9893 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:22.222136  9882 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:22.236734  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:22.237128 10029 log.cc:927] Time spent T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:18:22.237663  6508 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:18:22.238723  9882 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:22.232362  9953 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:22.242033  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:22.242147  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:22.243497  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:22.190008  9508 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:22.140926  9950 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:22.174469  9718 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:22.258500  6473 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:22.189321  9490 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:18:22.192165  9929 raft_consensus.cc:697] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:22.259634  6576 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:18:22.248881 10026 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:22.267287  6575 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:18:22.270378  6501 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:18:22.252120  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:22.275975  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 84239.9us
I20260709 12:18:22.283571  6579 raft_consensus.cc:2393] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 7: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:22.276624  7268 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:22.289237  7267 raft_consensus.cc:1240] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:22.289877  6473 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:22.295785  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:22.296355  6473 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:22.296729  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:22.297638  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:22.299415  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:22.304952  6584 raft_consensus.cc:2393] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 8: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:18:22.308408  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.309170  6473 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:22.288878  9921 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:22.317533  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:22.332194  6575 raft_consensus.cc:2393] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 10: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:22.318496  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:22.332827  6579 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:22.292109  9876 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:22.210992 10015 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:22.300732  9873 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.300666 10016 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:22.302708  9965 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:22.320612  9917 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:22.391669  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:22.257642 10011 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:22.405303 10011 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:22.408524  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:22.412317  6579 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:18:22.418614  6507 raft_consensus.cc:2393] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 9: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:22.394300  9958 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:22.422004  9950 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:22.422995 10017 log.cc:927] Time spent T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 12:18:22.422923  9958 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:22.424456 10033 log.cc:927] Time spent T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 12:18:22.337198  9490 consensus_queue.cc:1059] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 12:18:22.429702  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1620) took 1674 ms. Trace:
I20260709 12:18:22.430047 10015 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:22.430168  6649 rpcz_store.cc:276] 0709 12:18:20.755176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.755326 (+   150us) service_pool.cc:224] Handling call
0709 12:18:20.756211 (+   885us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:21.211247 (+455036us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:21.211291 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:21.274047 (+ 62756us) log.cc:844] Serialized 26 byte log entry
0709 12:18:21.637811 (+363764us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:21.637933 (+   122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:21.638005 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:22.429426 (+791421us) raft_consensus.cc:1685] finished
0709 12:18:22.429449 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:22.429634 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64651,"raft.queue_time_us":137890,"raft.run_cpu_time_us":71,"raft.run_wall_time_us":71,"thread_start_us":758813,"threads_started":2,"wal-append.queue_time_us":826509}
I20260709 12:18:22.432281  9917 raft_consensus.cc:697] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:22.433351  9917 consensus_queue.cc:237] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:22.390407  9912 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.390084  9773 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.394013  9873 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:22.438486  9965 raft_consensus.cc:697] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:22.450313  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:22.493867  6575 raft_consensus.cc:2393] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 7: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:22.441509  9790 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:18:22.453406  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 330679us
W20260709 12:18:22.482847  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.515974  6475 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:18:22.467274  6622 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.009s	sys 0.016s
W20260709 12:18:22.487468  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.441124  9926 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:18:22.516937  6622 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.010s	sys 0.016s
W20260709 12:18:22.517365  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 624795us
W20260709 12:18:22.517705  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.522781  6475 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:22.449203  6579 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:18:22.528841  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:22.529640  6653 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:22.530033  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:22.450542  9950 raft_consensus.cc:3055] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:22.531790  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:22.540477  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:22.541462  6575 raft_consensus.cc:2393] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 6: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:22.533464  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:22.542407  6653 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:22.549911  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.551687  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:22.553393  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:22.554978  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:22.555732  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:22.556600  6503 raft_consensus.cc:1240] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 8. Current term is 9. Ops: [8.3-8.3]
I20260709 12:18:22.520682 10011 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:22.567013  6503 raft_consensus.cc:1240] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 2. Current term is 3. Ops: []
I20260709 12:18:22.558739  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:22.559046  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:22.560372  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:22.561517  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:22.564675  6622 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:22.527773  9876 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:22.566148  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:22.528709  9882 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:22.587738  6500 raft_consensus.cc:2393] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 8: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:22.451489  9912 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:22.579206  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:22.588647  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:22.579391  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:22.538149  9997 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:22.588762  6503 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:18:22.450830  9773 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:22.593163  6505 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:22.595502  6653 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:22.539081  9950 raft_consensus.cc:740] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:22.598651  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:22.545997 10027 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:22.451277  9929 consensus_queue.cc:237] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:22.524441  9790 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:22.604259  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:22.605669  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:22.571694  9876 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:18:22.592661 10032 log.cc:927] Time spent T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 12:18:22.611310  6505 raft_consensus.cc:1240] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 8. Current term is 9. Ops: []
W20260709 12:18:22.580644  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 369395us
I20260709 12:18:22.528457  9926 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:22.617784  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:22.619280  6503 raft_consensus.cc:1275] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:22.605185  9965 consensus_queue.cc:237] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:22.611385  9960 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:18:22.623401 10038 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 12:18:22.624468 10035 log.cc:927] Time spent T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 12:18:22.629047  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:22.630184  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:22.630729  6655 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:18:22.587978 10011 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:22.631455  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:22.613037  9876 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:22.632953 10011 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:22.635390  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:22.636610  6656 raft_consensus.cc:3055] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:22.636900  6656 raft_consensus.cc:740] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:22.637573  6656 consensus_queue.cc:260] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:22.638460  6656 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:22.640697  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:22.641901  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:22.642987  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:22.643682  6575 raft_consensus.cc:2393] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 6: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:22.644449  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:22.644953  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:22.645048  6579 raft_consensus.cc:606] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:22.646143  6579 raft_consensus.cc:614] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:22.646646  6501 raft_consensus.cc:2410] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:18:22.648677  9888 consensus_meta.cc:287] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.583s	user 0.012s	sys 0.000s
I20260709 12:18:22.656908  6505 raft_consensus.cc:1240] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:22.656852  6504 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:18:22.671653  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:22.674275  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:22.675391  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:22.681465  6580 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:18:22.684525  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:22.685783  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:22.690213  6473 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:22.699620  6656 raft_consensus.cc:1275] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:22.702698  6474 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:22.703356  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:22.703806  6474 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:22.703961  6575 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
I20260709 12:18:22.608794 10027 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:22.657658  9921 raft_consensus.cc:697] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:22.708105  6504 raft_consensus.cc:2955] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 3 to 6, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:18:22.710166  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:22.713838  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:22.714569  6500 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
I20260709 12:18:22.721542  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:22.722837  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.732856  6548 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:22.734769  6548 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:22.737267  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:22.736316  6548 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:22.739543 10055 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:22.739130 10016 raft_consensus.cc:697] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:22.743633  6475 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:22.743929  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 179306us
I20260709 12:18:22.746372  6622 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:22.700974  9621 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:22.754416  6548 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:22.755450  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:22.756989  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:22.760655  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:22.683064  9950 consensus_queue.cc:260] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:22.763703  6500 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 7. Current term is 9. Ops: []
I20260709 12:18:22.727023 10053 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:22.746767  9775 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:22.747222  9888 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:22.726750  9790 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:22.761144  9893 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:22.780054  6503 raft_consensus.cc:1240] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:22.758219 10011 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:22.786402  9621 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:22.786868  6548 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:22.787492  9514 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:22.792379 10005 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:22.722772 10051 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
W20260709 12:18:22.758749  9844 consensus_meta.cc:287] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.830s	user 0.010s	sys 0.002s
I20260709 12:18:22.783298  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:22.804776  6582 raft_consensus.cc:1275] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:22.800948 10061 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:22.778019 10053 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:22.819026 10053 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:22.823658  9508 consensus_meta.cc:287] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.564s	user 0.009s	sys 0.000s
I20260709 12:18:22.804646  9950 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:18:22.798127  9580 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.013s	sys 0.020s
W20260709 12:18:22.824798  9580 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.014s	sys 0.020s
I20260709 12:18:22.825306  9580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:48498 (request call id 1) took 5650 ms. Trace:
I20260709 12:18:22.825783  9580 rpcz_store.cc:276] 0709 12:18:17.175355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.175485 (+   130us) service_pool.cc:224] Handling call
0709 12:18:17.378991 (+203506us) spinlock_profiling.cc:243] Waited 8.93 ms on lock 0x7b5800830fd0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875de50aa 00007f2875e67447 00007f2875e8470a 00007f2875e8576a 00007f2875e8491b 00007f2875e9003c 00007f2875e8f6b3 00007f2875e879fa 00007f287ae4f4c9 00007f28743cbc54
0709 12:18:17.491382 (+112391us) log.cc:844] Serialized 480 byte log entry
0709 12:18:20.164166 (+2672784us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b18004b41e0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d16c0a 00007f287c456219 00007f2872d9ea40 00007f2872d994e4 00007f2872d99455 00007f2875de2b0b 00007f2875e67b63 00007f2875e8f752 00007f2875e879fa
0709 12:18:22.825201 (+2661035us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2245926,"raft.queue_time_us":2423093,"raft.run_cpu_time_us":2082,"raft.run_wall_time_us":196233,"spinlock_wait_cycles":49834368,"thread_start_us":234810,"threads_started":1,"wal-append.queue_time_us":516864,"wal-append.run_cpu_time_us":2153,"wal-append.run_wall_time_us":1028400}
I20260709 12:18:22.803973  9921 consensus_queue.cc:237] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:22.830562  9926 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:22.831300  9773 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:22.831883  9912 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:22.834339  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 51129.7us
I20260709 12:18:22.835719  6622 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:22.831497 10053 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:22.802196  9893 raft_consensus.cc:697] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:22.838474  6622 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:22.847970  9912 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:22.849126 10027 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:22.851045  9842 consensus_meta.cc:287] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.699s	user 0.011s	sys 0.000s
W20260709 12:18:22.851846  9619 consensus_meta.cc:287] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.884s	user 0.008s	sys 0.008s
I20260709 12:18:22.852465  9926 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:22.824256  9508 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:22.864264 10061 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:22.865286  9876 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:22.868273  6622 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:22.871326 10053 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:22.874181  6582 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
W20260709 12:18:22.881824  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 170311us
I20260709 12:18:22.922773  6582 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 11. Current term is 12. Ops: [9.5-11.6]
I20260709 12:18:22.814756 10005 raft_consensus.cc:697] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:22.924741  6582 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:22.833492  9844 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:22.935359  6583 raft_consensus.cc:2368] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7.
I20260709 12:18:22.929054  6582 raft_consensus.cc:1240] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:22.942016  9959 consensus_queue.cc:1059] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:22.942440  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:22.936569  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 193048us
I20260709 12:18:22.812953 10016 consensus_queue.cc:237] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:22.957472  7238 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:22.926739  9621 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:22.958503  6582 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
W20260709 12:18:22.959347  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:22.935654 10061 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:22.897207  9842 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:22.948582  9960 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:22.964558  9950 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:22.963742 10061 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:22.996661  6504 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:18:22.944271  9619 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 12:18:22.999325  6505 raft_consensus.cc:1240] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 8. Current term is 9. Ops: []
W20260709 12:18:23.007082  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 104591us
I20260709 12:18:22.973846  9882 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:22.965634  9621 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:23.002099 10061 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:22.941066  9912 raft_consensus.cc:2955] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Committing config change with OpId 7.6: config changed from index 2 to 6, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:18:23.015719  9659 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:22.970139  9893 consensus_queue.cc:237] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:23.013875  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 53247.1us
I20260709 12:18:23.011437 10055 raft_consensus.cc:697] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:18:23.032721 10062 log.cc:927] Time spent T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
W20260709 12:18:23.034725 10059 log.cc:927] Time spent T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:18:23.041075 10055 consensus_queue.cc:237] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:23.042018  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1691) took 1028 ms. Trace:
I20260709 12:18:23.042416  6578 rpcz_store.cc:276] 0709 12:18:22.013413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.758311 (+744898us) service_pool.cc:224] Handling call
0709 12:18:22.758685 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:22.758910 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:22.758944 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:22.759456 (+   512us) log.cc:844] Serialized 26 byte log entry
0709 12:18:22.928053 (+168597us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:22.928089 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:22.928148 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:23.041791 (+113643us) raft_consensus.cc:1685] finished
0709 12:18:23.041810 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:23.041961 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":121228,"threads_started":1,"wal-append.queue_time_us":224436}
I20260709 12:18:23.051556  6475 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:23.056411  9935 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.057077  9935 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:23.058996  9844 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:23.061411  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:23.064157  6649 raft_consensus.cc:1240] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:23.064651  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:23.065243  6655 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:23.060699  6654 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:23.070580  6576 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:23.071698  6475 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:23.074216  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 1899) took 1238 ms. Trace:
I20260709 12:18:23.074630  6508 rpcz_store.cc:276] 0709 12:18:21.835488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:21.897460 (+ 61972us) service_pool.cc:224] Handling call
0709 12:18:21.897856 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:22.238043 (+340187us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.073967 (+835924us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b540046cdd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:23.074156 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":780892416}
I20260709 12:18:23.076077  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2206) took 1151 ms. Trace:
I20260709 12:18:23.076665  9508 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:23.076560  6652 rpcz_store.cc:276] 0709 12:18:21.924399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:21.928634 (+  4235us) service_pool.cc:224] Handling call
0709 12:18:21.928905 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:21.943220 (+ 14315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.075848 (+1132628us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b54002a3fd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:23.076017 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31532416}
I20260709 12:18:23.079134  9842 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:23.079456  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:23.080854  7269 raft_consensus.cc:1240] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:23.082332  9876 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:23.002414 10005 consensus_queue.cc:237] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:23.083039  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:23.084348  6655 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:23.088053  6650 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:23.090138  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:23.090566  6393 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3a47d71c34b541b9b2589b4e80413fa2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 11)
I20260709 12:18:23.090740  6651 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:23.091624  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:23.092814  7274 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38558 (request call id 497) took 2795 ms. Trace:
I20260709 12:18:23.093161  7274 rpcz_store.cc:276] 0709 12:18:20.297317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.297454 (+   137us) service_pool.cc:224] Handling call
0709 12:18:21.963722 (+1666268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:22.074512 (+110790us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.092629 (+1018117us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b54004e7ad0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:23.092739 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3831198464}
I20260709 12:18:23.093603  9621 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:23.094337 10014 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:23.094635  6475 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:23.095105 10014 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:23.095484 10014 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:23.095888  6475 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:23.098845  9884 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:23.099597  9884 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:23.100503  9921 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:23.101616  6576 raft_consensus.cc:1240] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:23.103976  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:23.105933  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:23.105762  6392 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 6f94b1cd286648a9b82d5abf69bb86d5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 496c7c3792cd4587b4dbeba435f4be1e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:18:23.019452  9882 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:23.106957  6500 raft_consensus.cc:1240] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:23.105072  6475 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:23.108177  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:23.108515  6649 raft_consensus.cc:1275] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:18:23.108946  6649 pending_rounds.cc:85] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 4
I20260709 12:18:23.109078  6652 raft_consensus.cc:2410] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:23.109177  6649 pending_rounds.cc:107] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:18:23.108325  9876 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:23.109589  6649 raft_consensus.cc:2889] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:23.110211  7267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38546 (request call id 393) took 1108 ms. Trace:
I20260709 12:18:23.095890 10014 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:23.110565  7267 rpcz_store.cc:276] 0709 12:18:22.002131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.002274 (+   143us) service_pool.cc:224] Handling call
0709 12:18:22.002564 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:22.289658 (+287094us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.109986 (+820328us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b54006c2354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:23.110160 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":658749184}
I20260709 12:18:23.111413  6580 raft_consensus.cc:1275] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:23.113013  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 1891) took 1314 ms. Trace:
I20260709 12:18:23.113421  6507 rpcz_store.cc:276] 0709 12:18:21.802681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:21.802873 (+   192us) service_pool.cc:224] Handling call
0709 12:18:23.112814 (+1309941us) spinlock_profiling.cc:243] Waited 575 ms on lock 0x7b54000fee54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:23.112952 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1323258112}
I20260709 12:18:23.113955  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:23.115553  6654 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 1612) took 2493 ms (client timeout 3214 ms). Trace:
I20260709 12:18:23.115953  6657 raft_consensus.cc:1240] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7. Ops: []
W20260709 12:18:23.115986  6654 rpcz_store.cc:269] 0709 12:18:20.622339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:20.622482 (+   143us) service_pool.cc:224] Handling call
0709 12:18:23.115376 (+2492894us) spinlock_profiling.cc:243] Waited 2.44 s on lock 0x7b54003e8fd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:23.115493 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5605134080}
I20260709 12:18:23.118952  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 1698) took 1079 ms. Trace:
I20260709 12:18:23.119410  6581 rpcz_store.cc:276] 0709 12:18:22.040114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.084576 (+ 44462us) service_pool.cc:224] Handling call
0709 12:18:23.118767 (+1034191us) spinlock_profiling.cc:243] Waited 896 ms on lock 0x7b540006de54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:23.118893 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2060918016}
I20260709 12:18:23.121080  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:23.120884  9876 consensus_queue.cc:1059] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:23.116245  6502 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:23.123865  6658 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:23.129817  6651 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:23.124830  6475 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:23.145531  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:23.146391  6649 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:23.154332  6654 raft_consensus.cc:3055] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:23.154575  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:23.155009  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:23.158434  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:23.154758  6654 raft_consensus.cc:740] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:23.159858  6650 raft_consensus.cc:1275] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:18:23.164639  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:23.098879  9926 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:23.165380  6657 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:23.160048  6654 consensus_queue.cc:260] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:23.169431  6654 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:23.111685 10061 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:23.174111  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:23.122449  9875 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:23.182169  9619 consensus_queue.cc:1048] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:18:23.192256  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 183174us
I20260709 12:18:23.162289  9621 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:23.199846  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:23.203186  6504 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:23.200999  6507 raft_consensus.cc:1240] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:23.201689  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:23.198822  6649 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:23.208261  6579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:53972 (request call id 17) took 1050 ms. Trace:
I20260709 12:18:23.208700  6579 rpcz_store.cc:276] 0709 12:18:22.168355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.443516 (+275161us) service_pool.cc:224] Handling call
0709 12:18:23.208019 (+764503us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b540034a8d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e83853 00007f287ae5160a 00007f28743d77d4 00007f28743d7762 00007f28743d7691 00007f28743d760c 00007f28743d68c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:23.208142 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":271108480}
I20260709 12:18:23.213056  9959 consensus_queue.cc:1059] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:23.213733  6622 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:23.214067  6653 raft_consensus.cc:2393] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 9: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:23.166042 10076 consensus_queue.cc:1059] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:18:23.207969 10061 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:23.221712 10076 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:23.223793  9623 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:23.227458  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:23.228173  6503 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7.
I20260709 12:18:23.167779  9842 consensus_queue.cc:1059] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:18:23.235697  9957 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:23.236711  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:23.237398  6657 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:23.237620  6650 raft_consensus.cc:2393] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 7: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:23.238514  9873 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:23.239257  6504 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 13.
I20260709 12:18:23.242040  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:23.242688  6650 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
I20260709 12:18:23.244943  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:23.219625  9621 raft_consensus.cc:697] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:23.246603  6649 raft_consensus.cc:1240] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:23.255021  6654 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:18:23.261359  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 47731.3us
I20260709 12:18:23.262622  6622 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:23.264310  6622 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:23.267690 10076 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:23.279553  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:23.287569  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:23.288414  7274 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:18:23.294713  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:23.295054  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1682) took 1288 ms. Trace:
W20260709 12:18:23.296074  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 509235us
I20260709 12:18:23.296906  6548 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:23.300565  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:23.274648 10061 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:23.259969  9884 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:18:23.306022  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:23.306803  6504 raft_consensus.cc:2376] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6. Re-sending same reply.
I20260709 12:18:23.295440  6577 rpcz_store.cc:276] 0709 12:18:22.006570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.056122 (+ 49552us) service_pool.cc:224] Handling call
0709 12:18:22.056552 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:22.056823 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:22.559647 (+502824us) log.cc:844] Serialized 14 byte log entry
0709 12:18:22.802668 (+243021us) log.cc:844] Serialized 14 byte log entry
0709 12:18:23.082078 (+279410us) spinlock_profiling.cc:243] Waited 1.78 ms on lock 0x7b18004627c0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:18:23.294512 (+212434us) spinlock_profiling.cc:243] Waited 1.99 ms on lock 0x7b18004627c0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d16c0a 00007f287c456219 00007f2872d9ea40 00007f2872d994e4 00007f2872d99455 00007f2875e81060 00007f2875e93716 00007f2875ea4287 00007f2875ea4208
0709 12:18:23.294670 (+   158us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:23.294735 (+    65us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:23.294797 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.294880 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:23.294999 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":554194,"prepare.queue_time_us":105192,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":72,"raft.queue_time_us":85,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":2225,"spinlock_wait_cycles":8665344,"thread_start_us":651463,"threads_started":2,"wal-append.queue_time_us":360937}
I20260709 12:18:23.301499  6508 raft_consensus.cc:2393] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 9: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:23.258206  9953 consensus_queue.cc:1059] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 12:18:23.317744  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:23.304015  6501 raft_consensus.cc:1240] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:23.312041  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:23.303994  6582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1771) took 1041 ms. Trace:
I20260709 12:18:23.314282  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:23.317236  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:23.318393  6507 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:18:23.320230  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:23.324312  6583 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
I20260709 12:18:23.318497  6509 raft_consensus.cc:1240] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:18:23.334645 10076 raft_consensus.cc:3055] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:23.335208 10076 raft_consensus.cc:740] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:23.338284  9873 raft_consensus.cc:3055] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:23.338784  9873 raft_consensus.cc:740] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:23.345598  9574 consensus_queue.cc:1048] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:23.357028 10014 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:23.359395 10014 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:23.360882  6550 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:23.301612  9888 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:23.364516  9798 consensus_queue.cc:1059] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:23.364912  9888 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:23.366369  9718 consensus_queue.cc:1048] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:23.367352  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 1694) took 1353 ms. Trace:
I20260709 12:18:23.367771  6584 rpcz_store.cc:276] 0709 12:18:22.014071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.144437 (+130366us) service_pool.cc:224] Handling call
0709 12:18:23.367137 (+1222700us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b54001cbe54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:23.367291 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":366315904}
I20260709 12:18:23.368425  9718 consensus_queue.cc:1059] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:23.336246 10076 consensus_queue.cc:260] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:23.369894  9718 raft_consensus.cc:3055] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:23.370239  9718 raft_consensus.cc:740] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:23.371022  9718 consensus_queue.cc:260] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:23.304783  9621 consensus_queue.cc:237] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:23.304692  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:23.318387 10046 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:23.372325  6550 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:23.417034  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:18:23.417286  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 52975.8us
I20260709 12:18:23.339815  9873 consensus_queue.cc:260] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:23.363822 10002 consensus_queue.cc:1059] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:18:23.414582  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:23.417635  6506 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:23.409205  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 112363us
I20260709 12:18:23.417439  6658 raft_consensus.cc:1275] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:18:23.419116 10082 log.cc:927] Time spent T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.096s	user 0.003s	sys 0.000s
I20260709 12:18:23.451850  6548 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:23.458691  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 139959us
W20260709 12:18:23.463207  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:23.464958  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 144488us
I20260709 12:18:23.418715 10011 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:23.424502  9518 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.472258  7274 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:23.450812  9718 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:23.473327  6549 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:23.424185  9514 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:23.475216  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:23.504958  6583 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:23.505518  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:23.505842  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:23.506901  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:23.507169  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:23.452445  9981 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:18:23.495846  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:23.496081  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 44341.1us
I20260709 12:18:23.512658  6577 raft_consensus.cc:2393] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 8: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:23.513075 10018 consensus_queue.cc:1048] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:23.513338  9775 consensus_queue.cc:1059] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 8.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:23.455780  9576 raft_consensus.cc:3055] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:23.451490 10046 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:23.503232  9823 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:18:23.497165  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:23.515328  9873 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:23.530598  7267 raft_consensus.cc:1240] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:23.508895  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:23.418769  9929 raft_consensus.cc:3055] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:23.534260  9929 raft_consensus.cc:740] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:18:23.535303 10080 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.342s	user 0.003s	sys 0.000s
I20260709 12:18:23.535214  9929 consensus_queue.cc:260] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:23.536495  9929 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:23.537631  6575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1695) took 1523 ms. Trace:
I20260709 12:18:23.538010  6575 rpcz_store.cc:276] 0709 12:18:22.014318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.775751 (+761433us) service_pool.cc:224] Handling call
0709 12:18:22.776031 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:22.806140 (+ 30109us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:22.944103 (+137963us) log.cc:844] Serialized 14 byte log entry
0709 12:18:23.416941 (+472838us) log.cc:844] Serialized 14 byte log entry
0709 12:18:23.429317 (+ 12376us) log.cc:844] Serialized 14 byte log entry
0709 12:18:23.472747 (+ 43430us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:23.472795 (+    48us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:23.472844 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.537342 (+ 64498us) spinlock_profiling.cc:243] Waited 29.9 ms on lock 0x7b54000587d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:23.537387 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:23.537578 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":438460,"prepare.queue_time_us":25444,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":73,"raft.queue_time_us":491835,"raft.run_cpu_time_us":315,"raft.run_wall_time_us":25234,"spinlock_wait_cycles":93164544,"thread_start_us":194093,"threads_started":2,"wal-append.queue_time_us":126698}
I20260709 12:18:23.541324 10061 consensus_queue.cc:1059] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:23.542318 10061 consensus_queue.cc:1059] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:23.542902  6654 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:18:23.543107 10061 raft_consensus.cc:3055] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:23.543591 10061 raft_consensus.cc:740] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:23.543751 10079 raft_consensus.cc:3055] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:23.514871 10018 consensus_queue.cc:1059] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:18:23.545615  6401 catalog_manager.cc:5697] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 5 to 7, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 7 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:23.548671  6584 raft_consensus.cc:1275] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:18:23.516597 10085 log.cc:927] Time spent T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.264s	user 0.000s	sys 0.001s
I20260709 12:18:23.535985  6622 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:23.554533 10075 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:23.555508  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:23.490897 10027 consensus_queue.cc:1059] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:18:23.556316 10075 raft_consensus.cc:2764] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260709 12:18:23.557181 10027 raft_consensus.cc:3055] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:23.557430  6622 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:23.557767 10027 raft_consensus.cc:740] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:23.522385  9576 raft_consensus.cc:740] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:23.558346 10075 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:23.558902  9576 consensus_queue.cc:260] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:23.558990 10027 consensus_queue.cc:260] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:23.496848 10007 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:23.546468  9929 raft_consensus.cc:3055] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:23.560202  6622 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:23.560606  9929 raft_consensus.cc:740] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:23.526077  9981 raft_consensus.cc:697] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:23.561477  9929 consensus_queue.cc:260] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:23.559407 10075 consensus_queue.cc:1059] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:23.562770 10075 consensus_queue.cc:1243] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:18:23.563340 10075 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:23.520839  9926 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:23.564193 10087 consensus_queue.cc:1059] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:23.564646  9926 raft_consensus.cc:697] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:23.565992  9926 consensus_queue.cc:237] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:23.503374  9518 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:23.564451  9993 raft_consensus.cc:3055] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:23.567979  9993 raft_consensus.cc:740] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:23.568933  9993 consensus_queue.cc:260] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:23.521492  9882 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:23.515015  9514 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:23.571983  6500 raft_consensus.cc:1240] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:23.563676 10075 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:23.544167 10079 raft_consensus.cc:740] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:18:23.648607  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:23.646306 10075 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:23.562740  9981 consensus_queue.cc:237] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:23.569828  9576 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:23.647921  9514 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:23.584833  6506 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
W20260709 12:18:23.562985 10037 log.cc:927] Time spent T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:18:23.551955  9508 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:18:23.646938  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:23.824536  6505 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:23.841444  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:23.841970  6583 raft_consensus.cc:1275] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:23.842506  6507 raft_consensus.cc:2393] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 9: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:18:23.843945  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:23.767467  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 254805us
I20260709 12:18:23.845309  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:23.845971  6548 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
W20260709 12:18:23.770973  6622 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 2.063s (SENT) [suppressed 25 similar messages]
W20260709 12:18:23.850175  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 79247.9us
W20260709 12:18:23.819235 10081 log.cc:927] Time spent T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.626s	user 0.000s	sys 0.007s
I20260709 12:18:23.854249  6579 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:18:23.856236  6548 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:23.673082  9882 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.750324  9949 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:18:23.861249  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 166758us
I20260709 12:18:23.842939  9518 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:23.864396  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:23.847329  9993 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:23.850520  9798 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:23.871265  6579 raft_consensus.cc:1275] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:23.871798  6577 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:23.871995  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:23.873348  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:23.877583  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:23.873219  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:23.873730  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:23.878216  6503 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:23.878597  6506 raft_consensus.cc:2393] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 9: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:23.860957 10027 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:23.881320  6577 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:23.863826  9514 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:23.886863  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:23.887745  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:23.887984  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:23.886138  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:23.885607  9739 raft_consensus.cc:3055] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:23.890020  9739 raft_consensus.cc:740] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:23.884815 10046 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:23.866637 10054 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:23.891170  6507 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:23.855268 10055 consensus_queue.cc:1048] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:18:23.890842  9739 consensus_queue.cc:260] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:23.900091  6506 raft_consensus.cc:1240] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:23.758349 10061 consensus_queue.cc:260] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:23.864698  9718 consensus_queue.cc:1059] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:18:23.868217  9882 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:23.884447  9798 raft_consensus.cc:697] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:23.937223 10098 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:23.937830 10098 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:23.931255  9926 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:23.938983  9798 consensus_queue.cc:237] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:23.939880 10098 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:23.942946  9882 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:23.933465  9739 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:23.946904  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:23.937121  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:18:23.949599  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:23.950204  6506 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:23.952358  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:23.953279  7267 raft_consensus.cc:2393] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 7: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:23.954136  9953 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:23.954789  9953 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:23.957950  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:23.959121  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:23.959702  6583 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:18:23.960578  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:23.961160  6583 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:23.963925  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:23.965126  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:23.965649  6579 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:23.968678  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:23.969911  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:23.970908  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:23.971591  6577 raft_consensus.cc:2393] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 6: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:23.972764  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:23.973938  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:23.974752  6576 raft_consensus.cc:2410] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:18:23.975509  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:23.936390  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:23.677330 10079 consensus_queue.cc:260] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:23.976063  6576 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 3.
I20260709 12:18:23.994484  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:23.984581  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:24.004577  6579 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:18:23.995093  6503 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:23.999298  6581 raft_consensus.cc:1275] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:24.000124  9718 consensus_queue.cc:1243] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:18:23.901252 10046 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.001250  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:24.007627  6576 raft_consensus.cc:2376] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9. Re-sending same reply.
I20260709 12:18:24.008517  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:23.901206 10054 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:24.002527  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:24.023815  6506 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:24.022056  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:24.031271  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:23.984287  6583 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
W20260709 12:18:24.054107  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:24.081745  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 104028us
I20260709 12:18:24.083087  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:24.063849  9576 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.065425  9775 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.065841 10046 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:24.099510  6506 raft_consensus.cc:1240] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 8. Current term is 9. Ops: []
W20260709 12:18:24.095980 10084 log.cc:927] Time spent T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.902s	user 0.001s	sys 0.007s
I20260709 12:18:24.084720 10075 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:24.090867  9576 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:24.112668  6581 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:18:24.102075  9659 raft_consensus.cc:3055] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:18:24.116771  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:24.117347  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.093895  9775 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:24.118680  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:24.118911  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 246981us
I20260709 12:18:24.119304  6654 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:24.119825  6622 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:24.120949  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:24.121315  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:24.121636  6654 raft_consensus.cc:2393] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 8: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:24.116472  9659 raft_consensus.cc:740] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:24.125763  9659 consensus_queue.cc:260] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:24.126338 10075 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:24.126766  9659 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:24.128010 10046 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:23.319866  6582 rpcz_store.cc:276] 0709 12:18:22.262557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:22.963532 (+700975us) service_pool.cc:224] Handling call
0709 12:18:22.964428 (+   896us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:23.303679 (+339251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:23.303708 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:23.303747 (+    39us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:23.303781 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.303839 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:23.303933 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":338843,"raft.queue_time_us":340382,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":15212}
I20260709 12:18:24.139657  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:24.112799 10110 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:24.134123  9949 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:18:24.134380 10101 log.cc:927] Time spent T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
W20260709 12:18:24.135046 10097 log.cc:927] Time spent T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:18:24.134899  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:24.149653  6506 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
W20260709 12:18:24.135639 10103 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 12:18:24.124675  6550 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:24.133486  9775 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:24.138015 10096 log.cc:927] Time spent T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 12:18:24.159521  6473 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:24.160925  6473 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:24.159580  6582 raft_consensus.cc:1275] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:18:24.161618  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.148020 10075 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:24.166250  6473 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:24.173525  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:24.174432  6583 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 3.
I20260709 12:18:24.148270  9949 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:24.176275  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 93252.4us
I20260709 12:18:24.163199 10046 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:24.164492  9739 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:24.177809  9917 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:24.178050  6473 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:24.187362  9935 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:24.192528 10054 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:24.193538 10106 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:24.158444  9619 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:24.195392  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1772) took 1932 ms. Trace:
I20260709 12:18:24.167209 10051 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:24.167395 10055 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:18:24.213821  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.175835  9621 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:18:24.203042 10102 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
W20260709 12:18:24.230932  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 84148.4us
W20260709 12:18:24.213207  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 91939.9us
I20260709 12:18:24.201377  7238 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:24.238327  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:24.237754  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:24.243893  6649 raft_consensus.cc:2393] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 10: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:24.248587  6548 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:24.253998  6651 raft_consensus.cc:1275] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:24.255535  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:24.255808  6651 raft_consensus.cc:1275] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:24.206835  9996 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:24.203634 10046 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:24.214502  6578 rpcz_store.cc:276] 0709 12:18:22.262887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:23.044135 (+781248us) service_pool.cc:224] Handling call
0709 12:18:23.044568 (+   433us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:23.408738 (+364170us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:23.408774 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:23.409454 (+   680us) log.cc:844] Serialized 52 byte log entry
0709 12:18:23.798233 (+388779us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:23.798296 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:23.798413 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:24.195154 (+396741us) raft_consensus.cc:1685] finished
0709 12:18:24.195180 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:24.195329 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":273576,"raft.queue_time_us":286934,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":108,"thread_start_us":306248,"threads_started":1,"wal-append.queue_time_us":496571}
W20260709 12:18:24.264097  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 62856.2us
I20260709 12:18:24.267486  7238 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:24.273281  6548 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:24.273610  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:24.274549  6548 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:24.269529  9957 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:24.280920  6548 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:24.288031  6656 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:24.281066  7238 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:24.205600  9821 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.274576  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:24.268122  9739 consensus_queue.cc:1059] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:18:24.293022  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:24.343549  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 63282.1us
W20260709 12:18:24.346596  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 65616.2us
I20260709 12:18:24.347702  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:24.349195  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 60694.8us
I20260709 12:18:24.350976  6548 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:24.309355  6577 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:18:24.268421 10054 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:24.267445  9798 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.224376  9844 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.080s
I20260709 12:18:24.224108  9574 consensus_queue.cc:1048] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.080s
I20260709 12:18:24.223084  9619 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:24.302066  9893 raft_consensus.cc:3055] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:24.369690  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:24.283042 10098 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:24.302714  9921 raft_consensus.cc:3055] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:24.301337 10099 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:24.361613 10047 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:24.282820  9957 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:24.283206  9508 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:24.264205  9920 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.105s
I20260709 12:18:24.378993  6583 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5.
I20260709 12:18:24.343931 10110 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:24.413357 10110 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:24.413858 10110 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:24.301129 10007 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:24.384619  6622 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:24.417950  6622 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:24.381608  9798 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:24.422559  9798 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:24.423331 10075 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:24.381981  6650 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 2.
I20260709 12:18:24.425732  6650 raft_consensus.cc:1275] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:24.265822  9775 consensus_queue.cc:1059] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.109s
I20260709 12:18:24.222740 10106 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:24.431905 10106 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:24.269253 10046 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:24.405165  6654 raft_consensus.cc:1240] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:24.434698  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:24.435297  6654 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:24.361356  9823 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:24.436514  9823 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:24.438710  9823 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:24.272482  9917 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.110s
W20260709 12:18:24.391652  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.432565 10075 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:24.420065  6622 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:24.447476  6622 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:24.448516 10075 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:18:24.454495  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.460657 10122 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:24.461961  9619 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:24.462323  9882 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:24.365587  9659 consensus_queue.cc:1059] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 12:18:24.463757  9659 raft_consensus.cc:3055] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:24.464164  9659 raft_consensus.cc:740] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:24.465113  9659 consensus_queue.cc:260] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:24.466395  9659 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:24.351967  9821 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:24.470635  9821 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:24.409390  9957 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:24.380867  9893 raft_consensus.cc:740] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:24.477051  9893 consensus_queue.cc:260] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:24.478158  9893 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:24.478696 10027 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.479255 10027 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:24.389714  9921 raft_consensus.cc:740] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:24.482596  9921 consensus_queue.cc:260] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:24.483671  9921 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:24.485505  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 105990us
I20260709 12:18:24.486672  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:24.487300  6658 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 4.
I20260709 12:18:24.499341  6550 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:24.405524  9935 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:18:24.507759  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.509503  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:24.515961  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 68580.5us
I20260709 12:18:24.517077  6622 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:24.522337  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.531759  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:24.524556 10115 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:24.534015  7270 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:18:24.535423  6651 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:18:24.540899  9576 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:24.543370  9929 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:24.545039  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:24.547808 10079 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:24.553668  6474 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:18:24.556888  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.502511  9917 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:18:24.575320  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 66534.1us
I20260709 12:18:24.524210  9739 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:18:24.588200  7265 raft_consensus.cc:1240] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:24.589560 10054 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:24.589197  9957 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:24.503046  9920 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 12:18:24.502878  9983 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
W20260709 12:18:24.595394  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 63694.1us
I20260709 12:18:24.569717  9798 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:18:24.602958  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 240791us
I20260709 12:18:24.625311  9739 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:18:24.604528  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 253620us
I20260709 12:18:24.607852  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:24.623338  9621 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:24.596832 10115 raft_consensus.cc:697] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:24.636875 10054 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:24.610822  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 123448us
I20260709 12:18:24.625095  9917 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:24.623955  9912 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:24.640583  9798 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:24.685721  9798 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:24.637642  6502 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:24.693383 10027 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:24.641073 10123 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 12:18:24.705446  9912 raft_consensus.cc:697] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 8 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:18:24.706244  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.708161  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:24.715231  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:18:24.716555  9949 consensus_meta.cc:287] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.502s	user 0.006s	sys 0.003s
I20260709 12:18:24.720106  9798 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:24.722142 10026 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:24.722848  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:24.727804  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:24.733734  6655 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:24.708057  7240 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:24.729281  6509 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:24.725515  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:18:24.657895  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 49460.9us
I20260709 12:18:24.738308  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:24.660741  6548 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
W20260709 12:18:24.740192  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 79510.7us
I20260709 12:18:24.740453  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:24.741093  6578 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:24.741312  6548 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:24.665017  7267 raft_consensus.cc:1240] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: []
W20260709 12:18:24.672111  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.683514  6653 raft_consensus.cc:3055] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:24.746474  6653 raft_consensus.cc:740] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:24.684932  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:24.712059 10061 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:24.755080  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:24.756276  7274 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:18:24.756714  6548 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:24.654838  6473 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:24.762624  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:24.767517  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 112757us
I20260709 12:18:24.751493  7265 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:24.769873  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:24.771427  9926 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:24.769712  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:24.771978  9926 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:24.772291  9929 consensus_queue.cc:1059] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 8.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:24.774323  9926 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:24.775146 10002 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:24.775573 10002 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:24.775967 10002 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:24.776216  9798 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:24.777264 10122 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:24.661609 10098 consensus_queue.cc:1059] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.117s
I20260709 12:18:24.664794  9957 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:24.657531  9508 raft_consensus.cc:3055] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:24.780587  9508 raft_consensus.cc:740] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:24.781032  9893 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.781569  9893 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:24.782232  6578 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 11.
I20260709 12:18:24.784493  9983 consensus_queue.cc:1059] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:24.785431  9983 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:18:24.785804  9983 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:24.786216  9983 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:24.787106 10055 raft_consensus.cc:3055] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:24.787524 10055 raft_consensus.cc:740] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:24.788491 10055 consensus_queue.cc:260] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:24.789978 10055 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:24.802498  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:24.805281  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:24.744813  9949 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:24.805281  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:24.807332  6504 raft_consensus.cc:2393] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 7: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:24.807484  9949 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:24.809338  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:24.790707  7240 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:24.793190 10106 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:24.820256  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:24.820955  6576 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
I20260709 12:18:24.793062 10007 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:24.661875 10099 raft_consensus.cc:3055] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:24.794232  9935 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:24.756735  9873 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:24.794363  9912 consensus_queue.cc:237] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:24.795542  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:24.835754  6654 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:24.679322  9917 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:24.678544 10011 raft_consensus.cc:3055] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:24.796685  9960 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:24.797578  9621 consensus_queue.cc:1048] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:24.751871  9988 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:18:24.799031  6649 raft_consensus.cc:1275] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:18:24.799482  6655 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 11.
I20260709 12:18:24.766098 10027 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:24.781507  9508 consensus_queue.cc:260] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:24.811183  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:24.818589  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 439274us
I20260709 12:18:24.641373  9739 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:24.819340  6652 raft_consensus.cc:1240] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:24.784076  9893 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:24.852552 10027 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:24.821885  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:24.824517  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:24.700196  9953 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:24.828466  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:24.794694  6474 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:24.819458  6653 consensus_queue.cc:260] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:24.646554 10054 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:24.651407 10047 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:24.837982  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:24.823606  9957 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:24.830062 10099 raft_consensus.cc:740] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:24.850313  6649 pending_rounds.cc:85] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 5
I20260709 12:18:24.835258  9873 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:24.762208 10110 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:24.856097  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 61484.2us
I20260709 12:18:24.867697  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:24.854346  6654 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:24.844085  6509 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:24.858269  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:24.858255  6475 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:24.645529 10115 consensus_queue.cc:237] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:24.845392 10011 raft_consensus.cc:740] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:24.863018  6649 pending_rounds.cc:107] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:18:24.865239  7269 raft_consensus.cc:1240] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:24.859613  9935 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:24.859530  9508 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:24.857266 10027 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:24.871819  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:24.846757 10046 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:24.881853  6649 raft_consensus.cc:2889] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:24.859752  9893 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:24.904106  6550 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:24.874212  9844 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:24.866531  9949 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.880786  9934 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 12:18:24.874312 10079 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:24.894423 10099 consensus_queue.cc:260] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:24.897856 10046 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:24.904237  9893 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:24.957103  9893 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:24.958725  7238 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:24.898200 10011 consensus_queue.cc:260] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.3, Last appended by leader: 3, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:24.908849  6622 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:24.909870  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:24.865576  9873 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:24.916234  9960 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:18:24.917011  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.926239  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:24.972765  6576 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:24.934689  6577 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:18:24.862313  9926 consensus_queue.cc:1059] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 12:18:24.953977  6575 raft_consensus.cc:1240] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:18:24.946854 10079 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:24.955452 10099 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:24.960634  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:24.960933  6655 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:18:24.961336  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:24.963121  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 54330.6us
I20260709 12:18:24.862357 10124 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:25.011642 10124 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:25.014819 10124 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:24.967969  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:25.015947  6652 raft_consensus.cc:2410] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:18:24.976444  6579 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:24.869988 10125 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.096s
I20260709 12:18:24.934139 10123 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:24.984615  6581 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:18:24.930035  9823 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:24.999218  6649 raft_consensus.cc:2393] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 7: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:25.002596 10016 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:25.003026  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:24.918850 10132 consensus_queue.cc:1048] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 12:18:24.933221  9929 consensus_queue.cc:1059] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.057s
W20260709 12:18:24.988364  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.028776  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:24.941103  9949 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:25.030424  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:25.031484  6508 raft_consensus.cc:1240] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:18:25.019707  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:25.034359  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:25.034562  6509 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:18:24.969790 10075 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:25.037770  6652 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:18:24.969504 10054 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:25.050024  6500 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:18:25.050290  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:25.022454  9926 consensus_queue.cc:1243] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:18:25.039485  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:25.029810  6503 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:18:24.985694  9965 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.998319 10079 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:25.042852  6475 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:25.036756  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:25.045912  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:25.046244  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:25.046329  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:25.040907  6473 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:25.014658  9983 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:25.045297  6622 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:25.021198  9508 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:24.980942 10046 raft_consensus.cc:3055] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:25.045718  9929 raft_consensus.cc:3055] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:25.055087  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:25.098848  6508 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 13: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:25.060065  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:25.049603 10098 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:25.037840  9917 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.048096 10011 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:25.041729 10123 raft_consensus.cc:697] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:25.054695 10087 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:25.099689  6509 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5.
I20260709 12:18:25.077795  9934 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:25.079831  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:25.068385 10054 raft_consensus.cc:697] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 11 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:25.067461 10121 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:25.083501  9875 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:24.969659  9619 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 12:18:25.038823 10016 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:25.051294 10066 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 12:18:25.087957 10046 raft_consensus.cc:740] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:25.102419 10075 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:25.038544  9823 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:25.098335  9929 raft_consensus.cc:740] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:25.081868  9965 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:25.098868  9508 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:25.131271  7265 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 13: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:25.121966  9875 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:25.116173  9917 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:25.112030 10098 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:25.136351 10110 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:25.126353 10121 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:25.144644 10075 raft_consensus.cc:697] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 10 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:25.166086  6501 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:25.148801 10122 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.140861  9798 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.148811  9875 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:25.144629  9739 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:25.163792  9821 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.172632 10076 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:25.179404 10128 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:25.261824  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.271214 10121 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:25.189144  9798 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:25.256055 10087 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.275768  9983 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:25.277048  9823 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:25.347324 10087 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:25.279394  9935 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:25.167443  9929 consensus_queue.cc:260] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:25.268677  9949 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:25.282572  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:25.274446 10140 log.cc:927] Time spent T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
W20260709 12:18:25.255781 10138 log.cc:927] Time spent T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 12:18:25.274140  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:25.360636  6658 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
I20260709 12:18:25.334820 10124 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:25.189801 10122 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:25.163730 10123 consensus_queue.cc:237] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:25.355509 10106 consensus_meta.cc:287] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.529s	user 0.010s	sys 0.000s
I20260709 12:18:25.340698  9981 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:25.346796  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:25.349558 10087 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:25.180267 10054 consensus_queue.cc:237] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:25.353245  6657 raft_consensus.cc:2393] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 9: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:25.257962  9993 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.194375  9873 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:25.188513 10046 consensus_queue.cc:260] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:25.368000  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.366878  7240 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:25.417963  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 51182.1us
I20260709 12:18:25.337051  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:25.364779  9823 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:25.382210  6575 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:25.217852  9821 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:25.256181 10128 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:25.381824  9929 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:25.283197 10075 consensus_queue.cc:237] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.3, Last appended by leader: 3, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:25.321816  9960 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:25.422186  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:25.485754  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:25.489288  6509 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
I20260709 12:18:25.364727 10124 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:25.384048 10106 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:25.498947  7274 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:25.461072  9997 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.418658  9508 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:25.427531 10016 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:25.424744 10002 consensus_meta.cc:287] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.648s	user 0.009s	sys 0.001s
I20260709 12:18:25.461315 10110 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:18:25.493485 10055 consensus_meta.cc:287] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.687s	user 0.008s	sys 0.003s
I20260709 12:18:25.465240 10087 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.530118  6504 raft_consensus.cc:2410] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:18:25.530469  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:25.531201  6657 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:25.534839  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:25.536626 10098 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:25.537145  9965 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:25.507280 10026 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:25.539258 10026 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:25.541647 10026 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:25.541764  9798 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:25.503413  9934 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:25.528195 10002 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:25.542614  9934 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:25.544481 10002 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:25.545336 10147 log.cc:927] Time spent T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 12:18:25.503890  9993 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:25.551576  6584 raft_consensus.cc:2393] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 10: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:25.521212  6575 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
W20260709 12:18:25.521826  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.467208  9621 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.386152  9981 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:25.557327  6550 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:25.488423 10046 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:25.464570  9823 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:25.463191  9996 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.485707 10051 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.524104  6656 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:25.564143  9621 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:25.573880 10124 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:25.574059  9621 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:25.525285 10110 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:25.561334 10151 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.574811 10110 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:25.561954 10026 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:25.575697 10026 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:25.576814 10098 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:25.577239 10098 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:25.577561 10098 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:25.577865 10026 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:25.575148 10151 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:25.522373  9997 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:25.580596 10151 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:25.581460  9997 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:25.581980  9821 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:25.558013  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:25.561524  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:25.586043  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:25.564981  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:25.557337  9926 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
W20260709 12:18:25.570875 10143 log.cc:927] Time spent T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:18:25.592526  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:25.593024  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:25.594944  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:25.597440  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:25.498404  9960 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:25.588562  9893 consensus_meta.cc:287] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.626s	user 0.011s	sys 0.001s
I20260709 12:18:25.602387  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:25.605619  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.603333  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:25.608668  6577 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:18:25.611227  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:25.611642  9875 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:25.614166  9875 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:25.595983 10151 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:25.615100 10151 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:25.590459  9993 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:25.553360 10087 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:25.616837 10027 consensus_meta.cc:287] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.724s	user 0.010s	sys 0.000s
I20260709 12:18:25.617301 10027 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:25.618384  6581 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:25.602898  9823 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:25.619642 10027 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:25.622567 10151 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:25.606637  7267 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
I20260709 12:18:25.625825  6550 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:25.627526  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:25.628248  6577 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
I20260709 12:18:25.629841  6622 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:25.632561  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:25.634742  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:18:25.635607  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 563601us
I20260709 12:18:25.635896  7273 raft_consensus.cc:2368] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 7. Current term is 8.
I20260709 12:18:25.637221  6549 leader_election.cc:400] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 7. Current term is 8.
W20260709 12:18:25.605355 10061 consensus_meta.cc:287] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.791s	user 0.008s	sys 0.004s
I20260709 12:18:25.637573  6549 leader_election.cc:403] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:18:25.633132  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:25.639564  6577 raft_consensus.cc:2368] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 3. Current term is 4.
I20260709 12:18:25.651623  6623 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:25.652150  6652 raft_consensus.cc:2410] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:18:25.653820  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:25.632309  9739 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:25.631124 10137 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.378s	user 0.004s	sys 0.000s
I20260709 12:18:25.662590  6548 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:25.666757  7238 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:25.671903  7238 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:25.629372  9953 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:25.571408 10051 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:25.677354  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:25.680919  6582 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:25.642671  9934 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:25.682478  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:25.638718  9993 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:25.684670  6548 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:25.655673  9959 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.691382  6622 leader_election.cc:400] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 3. Current term is 4.
I20260709 12:18:25.691846  6622 leader_election.cc:403] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:18:25.697538  6548 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:25.659618 10151 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:25.588400  9996 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:25.628266  9893 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:25.715142  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:25.704720  6548 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:25.659989 10026 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:25.638314 10098 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:25.687258 10152 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.717744  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:25.718245  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:25.719694  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:25.721133  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:25.722251  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:25.725419  6581 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:18:25.680024 10112 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:25.684506 10047 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:25.685000  9920 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.661825 10153 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:25.662796  9997 consensus_queue.cc:1059] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:18:25.689565  9823 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:25.747354  6505 raft_consensus.cc:1240] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:25.692608  9993 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:25.747351  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:25.750739  6500 raft_consensus.cc:2393] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 10: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:25.752017  6503 raft_consensus.cc:2393] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 10: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:25.753140  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:25.754467  6509 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:25.704454  9912 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:25.711591 10099 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:25.712308 10115 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:18:25.754928  6504 raft_consensus.cc:2393] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 9: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:25.703349 10110 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:25.732766  9988 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:25.767637  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:25.768049  6548 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:25.715888 10002 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:25.728473  9508 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:25.700867 10027 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 12:18:25.689878  9953 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:25.764571 10148 log.cc:927] Time spent T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.271s	user 0.000s	sys 0.001s
I20260709 12:18:25.754752  9935 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:25.729288 10055 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.751737  9929 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 7. Current term is 8.
I20260709 12:18:25.752008 10153 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:25.708106  9959 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:25.812494  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:25.758088  9993 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:25.732326  9873 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:25.714772 10154 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 12:18:25.748129 10061 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:25.725101  9574 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:25.749321 10047 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:25.769029 10128 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:25.725185 10151 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:25.798187  9912 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:25.791065  9988 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:25.810447 10158 log.cc:927] Time spent T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:18:25.811029  9960 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:25.792017 10113 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.813926  9739 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:25.765226  9920 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:25.762739 10152 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:25.815599 10153 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:25.792124 10026 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:25.805176  9935 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:25.826148 10047 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:25.835804 10151 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:25.836248 10159 log.cc:927] Time spent T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:18:25.836576 10150 log.cc:927] Time spent T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
W20260709 12:18:25.836853 10011 consensus_meta.cc:287] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.720s	user 0.003s	sys 0.007s
I20260709 12:18:25.824784  9981 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:25.845000 10098 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:25.846644  9935 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:25.836248 10113 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:25.848062 10152 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:25.848891 10113 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:25.825235 10061 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:25.837728 10110 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:25.849004 10047 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:25.850893 10061 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:25.851081 10047 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:25.847148 10076 consensus_meta.cc:287] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.621s	user 0.001s	sys 0.007s
I20260709 12:18:25.851794  9949 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:25.852222  9949 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:25.854844 10153 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:25.814899 10055 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:25.856803 10079 consensus_meta.cc:287] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.756s	user 0.008s	sys 0.004s
I20260709 12:18:25.856967 10153 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:25.857239 10079 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:25.858856 10079 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:25.859339 10115 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 3. Current term is 4.
I20260709 12:18:25.899083  6502 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 1927) took 1031 ms. Trace:
I20260709 12:18:25.900240  6502 rpcz_store.cc:276] 0709 12:18:24.867230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.867368 (+   138us) service_pool.cc:224] Handling call
0709 12:18:24.867668 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:24.867983 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:24.868011 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:24.868477 (+   466us) log.cc:844] Serialized 26 byte log entry
0709 12:18:25.115349 (+246872us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:25.115396 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.115467 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:25.898134 (+782667us) raft_consensus.cc:1685] finished
0709 12:18:25.898235 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:25.898723 (+   488us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":691,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":88,"thread_start_us":226546,"threads_started":1,"wal-append.queue_time_us":266283}
I20260709 12:18:25.903851  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:25.905234  6502 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:25.916206  6649 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:18:25.918713  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:25.911990  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 143866us
I20260709 12:18:25.919902  6650 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:25.921698  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:25.922772  6650 raft_consensus.cc:2393] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 7: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:25.923873  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:25.926070  6650 raft_consensus.cc:2393] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 9: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:25.927165  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:25.928856  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:25.931690  6475 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:25.938277  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:25.945482  6651 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58980 (request call id 2241) took 1339 ms (client timeout 1578 ms). Trace:
W20260709 12:18:25.946185  6651 rpcz_store.cc:269] 0709 12:18:24.606036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.607646 (+  1610us) service_pool.cc:224] Handling call
0709 12:18:25.936206 (+1328560us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54003dfed4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:25.945231 (+  9025us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2328152832}
I20260709 12:18:25.948254  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:25.953876  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:25.955513  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2247) took 1165 ms. Trace:
I20260709 12:18:25.960515  6656 rpcz_store.cc:276] 0709 12:18:24.790220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.794820 (+  4600us) service_pool.cc:224] Handling call
0709 12:18:24.795127 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:25.571930 (+776803us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.954916 (+382986us) spinlock_profiling.cc:243] Waited 729 ms on lock 0x7b540044acd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:25.955298 (+   382us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1676039424}
I20260709 12:18:25.961727  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:25.962759  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:25.962967  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:25.963899  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:25.967639  6501 raft_consensus.cc:2368] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 3. Current term is 4.
I20260709 12:18:25.968227  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:25.968994  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:25.969894  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:25.965858  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:25.964078  6506 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:25.970861  6503 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
I20260709 12:18:25.971081  6508 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 9.
I20260709 12:18:25.971356  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:25.972172  6506 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:25.975925  6548 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:25.976840  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:18:25.978549  6548 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:25.982437  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:25.983507  6501 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 11.
I20260709 12:18:25.984997  6548 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:25.994958  6653 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:25.825759  9574 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:25.999575  6548 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:25.964238  6509 raft_consensus.cc:2393] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 10: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:25.971987  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:25.984179  6649 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:18:25.948191  6502 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:18:26.035821  6506 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:26.041726  9821 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:25.833773  9912 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:26.045183  9739 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:26.046748 10026 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:26.047590 10026 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:26.047978  9912 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:26.048382 10026 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:26.048228  9739 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:26.051560  9981 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:26.072933  9935 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:26.073894 10098 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:26.073853  9935 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:26.074486  9917 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:26.074724 10098 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:26.076431  9619 raft_consensus.cc:2764] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:18:26.051895  9821 raft_consensus.cc:697] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 8 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:26.050129 10079 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:26.079818 10079 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:26.080688  6622 leader_election.cc:400] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 3 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 3. Current term is 4.
I20260709 12:18:26.082715 10079 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:26.083694  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1752) took 1154 ms. Trace:
I20260709 12:18:26.084729  9574 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:26.048048 10051 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:26.084143  6584 rpcz_store.cc:276] 0709 12:18:24.929428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.780639 (+851211us) service_pool.cc:224] Handling call
0709 12:18:25.799042 (+ 18403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:25.910651 (+111609us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:25.910754 (+   103us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:25.910847 (+    93us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:25.910926 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.083268 (+172342us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b540034d5d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:26.083316 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:26.083492 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":254853888}
I20260709 12:18:26.092880  6507 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:18:26.094363  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:26.072973  9926 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.095176  9926 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:26.095782  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:26.096889  7238 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:26.098220  6579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1755) took 1009 ms. Trace:
I20260709 12:18:26.097532  9821 consensus_queue.cc:237] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:26.096904  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:26.099882  6577 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:26.100735 10128 raft_consensus.cc:2804] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:26.101212 10128 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:26.098548  7238 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:26.101636 10128 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:26.102442  9993 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:26.104138  6655 raft_consensus.cc:2393] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 8: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:26.097853  9926 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:26.104429  9957 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:26.104929  9957 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:26.105386  9957 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:26.105684  9893 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:26.106276 10122 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:26.110435 10132 raft_consensus.cc:3055] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:26.110909 10132 raft_consensus.cc:740] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:26.111065  6507 raft_consensus.cc:2393] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 10: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:26.098608  6579 rpcz_store.cc:276] 0709 12:18:25.027714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.951052 (+923338us) service_pool.cc:224] Handling call
0709 12:18:25.951968 (+   916us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:25.956070 (+  4102us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:25.956127 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:25.956849 (+   722us) log.cc:844] Serialized 26 byte log entry
0709 12:18:26.031055 (+ 74206us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:26.031105 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.031178 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:26.035962 (+  4784us) raft_consensus.cc:1685] finished
0709 12:18:26.035986 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:26.036991 (+  1005us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":84663,"thread_start_us":73632,"threads_started":1,"wal-append.queue_time_us":74255}
I20260709 12:18:26.112142  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:26.117327  9920 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:26.117995  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 120426us
I20260709 12:18:26.130290  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:26.129901  9935 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:26.130529  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:26.097723 10087 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:26.135670  6653 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:26.137677  6581 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:26.139125  9993 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:26.163573 10051 consensus_queue.cc:1059] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:26.166288  6473 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:26.170006  7270 raft_consensus.cc:1240] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:26.170001  9921 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:18:26.172164  9921 raft_consensus.cc:697] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 4 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:26.173708  9921 consensus_queue.cc:237] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:26.180958  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:26.190025  6653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1687) took 1938 ms. Trace:
I20260709 12:18:26.149178  6650 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 4.
I20260709 12:18:26.186491  9993 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:26.139765  9508 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:26.198251  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:26.198182  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:26.200666  6584 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:26.201792  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:26.202694  6584 raft_consensus.cc:2393] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 8: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:26.203531  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:26.204474 10114 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 12:18:26.204633  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:26.205250  9935 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.199401  6507 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
W20260709 12:18:26.138636 10141 log.cc:927] Time spent T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.918s	user 0.007s	sys 0.000s
I20260709 12:18:26.195039  6653 rpcz_store.cc:276] 0709 12:18:24.251094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.251227 (+   133us) service_pool.cc:224] Handling call
0709 12:18:24.251555 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:24.683416 (+431861us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54001999a0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e00dc0 00007f2875e8b1c0 00007f2875e8c231 00007f2875e88d8f 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c
0709 12:18:25.993767 (+1310351us) spinlock_profiling.cc:243] Waited 708 ms on lock 0x7b580049a0d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875de50aa 00007f2875e67447 00007f2875e7ffe7 00007f2875e817de 00007f2875e8bb49 00007f2875e8c416 00007f2875e88d8f 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4
0709 12:18:26.157781 (+164014us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.189772 (+ 31991us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1932538496}
I20260709 12:18:26.118814 10132 consensus_queue.cc:260] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:26.251420  9949 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:26.254298  9953 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:26.242012  9959 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:26.266194  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:26.209822 10087 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:26.299839  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:26.252885  6575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1817) took 1232 ms. Trace:
I20260709 12:18:26.234603  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:18:26.260910 10172 log.cc:927] Time spent T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:18:26.258257  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:26.232173 10146 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.856s	user 0.001s	sys 0.003s
I20260709 12:18:26.245712 10132 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:26.311695  7268 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:18:26.308815  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:26.302002  6575 rpcz_store.cc:276] 0709 12:18:25.020117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.914938 (+894821us) service_pool.cc:224] Handling call
0709 12:18:25.915570 (+   632us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:25.923480 (+  7910us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:25.923521 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:25.926620 (+  3099us) log.cc:844] Serialized 26 byte log entry
0709 12:18:26.055178 (+128558us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:26.124967 (+ 69789us) log.cc:844] Serialized 14 byte log entry
0709 12:18:26.125351 (+   384us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.125481 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:26.252408 (+126927us) raft_consensus.cc:1685] finished
0709 12:18:26.252430 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:26.252573 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":399752,"prepare.queue_time_us":75834,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":28831,"raft.queue_time_us":80507,"raft.run_cpu_time_us":536,"raft.run_wall_time_us":431970,"thread_start_us":128028,"threads_started":1,"wal-append.queue_time_us":252793}
W20260709 12:18:26.312227  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 101403us
I20260709 12:18:26.210420  9935 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:26.236006 10098 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:26.237190 10055 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:26.325791  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:26.237221  9988 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:26.244172  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:26.299026  9957 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:26.038522  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:26.331077  7270 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:26.340941  6548 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:26.356649  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 50387.5us
I20260709 12:18:26.329362  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:26.336509  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:26.334940  6550 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:26.418910  7238 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:26.361418  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1737) took 1438 ms. Trace:
W20260709 12:18:26.346328  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:26.332819  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:26.346836  6579 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:26.369351  6584 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:26.349565  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1738) took 1425 ms. Trace:
W20260709 12:18:26.420224  6582 consensus_meta.cc:287] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.502s	user 0.011s	sys 0.011s
I20260709 12:18:26.305773  9993 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:26.349375  7265 raft_consensus.cc:2368] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 11. Current term is 12.
W20260709 12:18:26.360833 10157 log.cc:927] Time spent T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.702s	user 0.003s	sys 0.000s
I20260709 12:18:26.332150  9823 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.426326  6581 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:26.342947  9934 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.435381  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:26.336170  9935 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:26.433959  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 235178us
I20260709 12:18:26.350203  9775 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.439839  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:26.358429  9508 consensus_queue.cc:1059] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 12:18:26.423890  6583 rpcz_store.cc:276] 0709 12:18:24.923750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.926146 (+  2396us) service_pool.cc:224] Handling call
0709 12:18:24.953513 (+ 27367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:24.965640 (+ 12127us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:24.965671 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:24.980766 (+ 15095us) log.cc:844] Serialized 26 byte log entry
0709 12:18:25.127624 (+146858us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:25.127668 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.127737 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:26.115753 (+988016us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:18:26.349091 (+233338us) raft_consensus.cc:1685] finished
0709 12:18:26.349114 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:26.349376 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25272,"raft.queue_time_us":12350,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":15229,"spinlock_wait_cycles":30028032,"thread_start_us":74119,"threads_started":1,"wal-append.queue_time_us":138120}
I20260709 12:18:26.443148  6575 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:26.356122  9988 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.082s
I20260709 12:18:26.346077 10087 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:26.440796  6582 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 6.
I20260709 12:18:26.437165  9823 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:26.451100  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:26.457547  6654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2250) took 1574 ms. Trace:
I20260709 12:18:26.458210  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:26.461007 10054 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:18:26.468196  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 143824us
W20260709 12:18:26.472414  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 65407us
I20260709 12:18:26.474285  9957 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:26.474570  9981 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:26.475212  9934 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:26.494570  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 159774us
I20260709 12:18:26.495590  9934 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:26.476589 10002 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:26.422345  6578 rpcz_store.cc:276] 0709 12:18:24.923383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.925304 (+  1921us) service_pool.cc:224] Handling call
0709 12:18:24.925733 (+   429us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:18:24.953793 (+ 28060us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:24.953824 (+    31us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:18:24.954511 (+   687us) log.cc:844] Serialized 78 byte log entry
0709 12:18:25.017081 (+ 62570us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:25.017123 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.017193 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:26.357263 (+1340070us) spinlock_profiling.cc:243] Waited 3.33 ms on lock 0x7b54002adb20. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875dffaaa 00007f2875dfedd9 00007f2875e09930 00007f2875e098d8 00007f2875e09858 00007f2875e09813 00007f2875e08a9c 00007f287bcf42f6 00007f287bceeaa8 00007f2875e457cb
0709 12:18:26.360890 (+  3627us) raft_consensus.cc:1685] finished
0709 12:18:26.360912 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:26.361232 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16865,"raft.queue_time_us":20844,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"spinlock_wait_cycles":7657728,"thread_start_us":62190,"threads_started":1,"wal-append.queue_time_us":128459}
I20260709 12:18:26.497309 10128 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:26.500885  7274 raft_consensus.cc:2393] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 10: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:26.505224  6579 raft_consensus.cc:2410] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 12:18:26.481276  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 62447us
I20260709 12:18:26.488865  6584 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:18:26.505491  6581 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:26.489784  7270 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
I20260709 12:18:26.492668  9508 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:26.508833  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1741) took 1582 ms. Trace:
I20260709 12:18:26.514468  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:26.516178  7270 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:18:26.517364  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:26.524842  9508 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:26.526384  9823 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:26.527293  9508 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:26.528010  6584 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:18:26.537901  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:26.538651  6656 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
I20260709 12:18:26.539789  6653 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:26.540726  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:26.541265  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:26.541595  6649 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
I20260709 12:18:26.542599  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:26.543170  6651 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:18:26.540647  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:26.545118  6657 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:18:26.546120 10114 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:26.480736  6654 rpcz_store.cc:276] 0709 12:18:24.882753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.884029 (+  1276us) service_pool.cc:224] Handling call
0709 12:18:24.884311 (+   282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:24.884694 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:24.884723 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:24.885098 (+   375us) log.cc:844] Serialized 26 byte log entry
0709 12:18:25.521925 (+636827us) spinlock_profiling.cc:243] Waited 8.95 ms on lock 0x7b1800376540. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:18:25.624811 (+102886us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:25.624883 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.625073 (+   190us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:26.359850 (+734777us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:18:26.457176 (+ 97326us) raft_consensus.cc:1685] finished
0709 12:18:26.457194 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:26.457386 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":96,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":11297,"spinlock_wait_cycles":51227264,"thread_start_us":102597,"threads_started":1,"wal-append.queue_time_us":721005}
W20260709 12:18:26.549742  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 104514us
I20260709 12:18:26.491559  9775 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:26.553231  6548 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:26.547859  9921 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.561549  6650 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 9.
I20260709 12:18:26.512233  6580 rpcz_store.cc:276] 0709 12:18:24.926204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.926314 (+   110us) service_pool.cc:224] Handling call
0709 12:18:24.940303 (+ 13989us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:24.957335 (+ 17032us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:24.957366 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:24.957910 (+   544us) log.cc:844] Serialized 52 byte log entry
0709 12:18:25.194129 (+236219us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:25.756781 (+562652us) log.cc:844] Serialized 14 byte log entry
0709 12:18:25.762764 (+  5983us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:25.762870 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:26.230885 (+468015us) spinlock_profiling.cc:243] Waited 2.41 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:18:26.508200 (+277315us) raft_consensus.cc:1685] finished
0709 12:18:26.508376 (+   176us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:26.508620 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":478059,"prepare.queue_time_us":449207,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":5026,"raft.queue_time_us":27138,"raft.run_cpu_time_us":311,"raft.run_wall_time_us":40672,"spinlock_wait_cycles":5532928,"thread_start_us":369155,"threads_started":2,"wal-append.queue_time_us":406447}
I20260709 12:18:26.584568  7238 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:18:26.584980  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:26.585739  6500 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
I20260709 12:18:26.586424  9821 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:26.591724  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:26.591926  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:26.592573  6649 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:26.593292  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:26.595074  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:26.565294  9921 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:26.595866  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:26.596020  6656 raft_consensus.cc:2393] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 9: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:26.596531  6654 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:26.597450  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:26.597882  9921 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:26.598163  6653 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
W20260709 12:18:26.604871  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 90532.3us
I20260709 12:18:26.605935  6548 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:26.607515 10122 raft_consensus.cc:3055] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:26.607973 10122 raft_consensus.cc:740] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:18:26.611936  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 46214.7us
I20260709 12:18:26.614475  6583 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:18:26.626843  6657 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 6.
I20260709 12:18:26.552690 10114 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:26.625602  6655 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
I20260709 12:18:26.635041  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:26.632745  6473 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:26.625027 10128 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:26.645567 10046 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:26.647145  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:26.624583  9739 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:26.648519  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:26.658324  6508 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:26.649215  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:26.650094  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:26.650640 10154 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:26.654199  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:26.654701  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 99412us
I20260709 12:18:26.648049  6576 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:18:26.641068  9934 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:26.662066  6473 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:26.662374  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:26.655825  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:26.663230  6473 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:26.655529 10046 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:26.674906  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:26.675884  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:26.687271  6502 raft_consensus.cc:2393] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 7: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:26.666080  9993 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:26.662081  9934 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:26.676318 10046 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:26.697890  6473 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:26.707130  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:26.707712  6655 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:26.712779  6657 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:26.672683  9821 raft_consensus.cc:697] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 6 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:26.669566  9739 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:26.716051  6508 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
I20260709 12:18:26.718683  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:26.696298  9981 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:26.721252  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:26.697930  9983 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:26.710361 10061 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:26.718428 10002 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:18:26.730018  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 83917.6us
I20260709 12:18:26.708973 10087 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:26.725997  6650 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:26.726768  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:26.730190 10191 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:26.712492  9934 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:26.733533  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 88906.1us
I20260709 12:18:26.731005  7240 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:26.736570  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:26.724023  9983 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:26.737279  6473 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:26.780558  9996 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:26.867909  6651 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 12.
I20260709 12:18:26.748272  9775 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:26.734817 10191 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:26.805011  6653 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:26.777154 10196 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:26.698096 10122 consensus_queue.cc:260] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:26.781574 10189 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 12:18:26.731099  9823 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:26.784317  9821 consensus_queue.cc:237] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:26.749408  9983 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:26.740216 10132 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:18:26.780928 10194 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 12:18:26.870805  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 121431us
I20260709 12:18:26.729938 10192 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:26.870198  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:26.876706  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:26.884735  6657 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:26.891935  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:26.908847  6509 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:26.909072  6508 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:26.878412 10128 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 12:18:26.897115  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:26.899206  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 165150us
I20260709 12:18:26.890892  6550 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:26.903671  6503 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:18:26.899524 10191 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:26.905203  9996 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:26.913532  6503 pending_rounds.cc:85] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 5
I20260709 12:18:26.917128  6473 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:26.900045 10061 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:26.916436  6503 pending_rounds.cc:107] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:18:26.919937  6503 raft_consensus.cc:2889] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:26.921101 10061 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:26.927845  6508 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
I20260709 12:18:26.905186  9823 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:26.918021  9996 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:26.933190  9996 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:26.932674  9823 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:26.930154  9775 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:26.943131  6581 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:26.914518 10132 raft_consensus.cc:697] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:18:26.945649  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 76489.3us
I20260709 12:18:26.911816 10114 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:26.950503  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:26.950834  6509 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 12.
I20260709 12:18:26.951081  6579 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:26.946787  6401 catalog_manager.cc:5697] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:26.955749  9739 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:26.953240  6550 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:26.903270 10196 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:26.963092 10196 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:26.964174  7240 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:26.946013  9775 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:26.971545  6473 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:26.972869 10066 raft_consensus.cc:3055] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:26.972760  9934 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:26.973021 10132 consensus_queue.cc:237] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:26.975195  9934 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:26.976383 10195 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:26.976974 10195 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:26.977421 10195 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:26.973517 10066 raft_consensus.cc:740] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:26.979137 10066 consensus_queue.cc:260] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:26.972163  9775 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:26.980168 10066 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:26.985643 10051 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:26.986109 10051 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:26.986469 10051 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:26.986474  6473 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:26.993572  7240 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:26.965902 10128 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:18:26.986680 10055 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:26.966756 10182 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:27.006909  6576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1745) took 2079 ms. Trace:
I20260709 12:18:27.001969  6474 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:27.007490  6576 rpcz_store.cc:276] 0709 12:18:24.927437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:24.973707 (+ 46270us) service_pool.cc:224] Handling call
0709 12:18:24.974155 (+   448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:26.687199 (+1713044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.998622 (+311423us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b54003ab954. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:27.006821 (+  8199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3829112064}
I20260709 12:18:26.971264 10054 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:27.020545  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:27.022461  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:27.024673  6658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1724) took 1295 ms. Trace:
I20260709 12:18:27.025085  6658 rpcz_store.cc:276] 0709 12:18:25.699365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:25.998842 (+299477us) service_pool.cc:224] Handling call
0709 12:18:25.999178 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:26.112249 (+113071us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:26.112277 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:26.118973 (+  6696us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:26.119019 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:26.993276 (+874257us) spinlock_profiling.cc:243] Waited 51.2 ms on lock 0x7b540008fcd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:26.993351 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:26.994820 (+  1469us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":117668096}
I20260709 12:18:27.026582  9996 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:27.029980  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:26.945313 10192 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:27.042060  6548 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:27.043047  6652 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:27.004756 10182 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:27.047083  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 60668.4us
W20260709 12:18:27.045418  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 43575.1us
I20260709 12:18:27.051173  6576 raft_consensus.cc:1275] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:18:27.028501  9873 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:27.032733 10054 raft_consensus.cc:697] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 10 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:27.059995  9997 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:27.060452  9997 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.057932 10191 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:27.060953  9997 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:27.061442 10055 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:27.062645 10191 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:27.060474  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 43662.3us
I20260709 12:18:27.065956  6475 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:27.066788  9873 raft_consensus.cc:697] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 12 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:27.068819  9917 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:27.069545  9917 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.070109  9917 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:27.071264 10188 log.cc:927] Time spent T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.334s	user 0.005s	sys 0.000s
I20260709 12:18:27.021862 10046 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:27.076825 10124 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:27.077296 10124 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.077682 10124 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:27.078742 10046 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:27.053043 10002 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:27.083098 10054 consensus_queue.cc:237] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:27.084728 10190 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:27.089092  9997 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:27.095873  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.071525  6475 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:27.049616 10182 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:27.095274  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 357743us
W20260709 12:18:27.099647  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 51896.2us
I20260709 12:18:27.076009  6579 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
I20260709 12:18:27.067749  9873 consensus_queue.cc:237] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:27.057876  9996 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:27.115564  9957 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:27.116725  9775 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:27.120385  9983 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:27.125802 10192 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:27.133292  6548 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:18:27.137928  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 66512.7us
I20260709 12:18:27.143887  7238 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:27.146363  6652 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
W20260709 12:18:27.148216  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 472579us
I20260709 12:18:27.148425 10191 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:27.148479  9983 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:27.149464  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:27.149632  6622 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:27.150942 10110 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:27.151494  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:27.152626 10087 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:27.153584 10122 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:27.153258 10087 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:27.159545  9949 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:27.160988  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:27.161896  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:27.163496  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:27.165500  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:27.168718  6401 catalog_manager.cc:5697] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:27.170676  6401 catalog_manager.cc:5697] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:27.172605 10002 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:27.173136 10002 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:27.173878  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:27.174610  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:27.175379 10002 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:27.173026  6502 raft_consensus.cc:1275] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:18:27.176040  6582 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:18:27.176687  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:27.177260  6575 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:27.178824  6550 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:27.184588  6623 leader_election.cc:400] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 11. Current term is 12.
I20260709 12:18:27.185482  6623 leader_election.cc:403] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:27.188433 10061 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:27.188834 10061 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.188601 10110 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:27.189240 10061 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:27.189592  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:27.190301  6582 raft_consensus.cc:2393] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 9: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:27.192696  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:27.193483  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:27.193868 10110 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:27.156087 10087 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:27.194999  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 61781.2us
I20260709 12:18:27.152647 10075 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.202459  6473 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:27.203956  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:27.198050  6401 catalog_manager.cc:5697] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 1 to 4, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 4 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:27.209156  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:27.212541  6575 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:27.188050 10104 consensus_queue.cc:1059] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 12:18:27.149654  9993 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 12:18:27.214612  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:27.215458  6581 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:18:27.222123  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:27.216962  6473 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:27.218286  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:27.219378  6548 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:27.214076  6654 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:18:27.223260  6656 raft_consensus.cc:2393] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 8: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:27.200440 10046 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:27.221688  6401 catalog_manager.cc:5697] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 3 to 4, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 4 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:18:27.228154  6652 raft_consensus.cc:2410] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:18:27.225780  6473 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:27.220583 10087 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 11. Current term is 12.
W20260709 12:18:27.241037  6406 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57914 (request call id 190) took 18627 ms (client timeout 15000 ms). Trace:
I20260709 12:18:27.237866  6623 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:27.242614  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:27.243172  6583 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:27.218351 10190 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:27.242832  9739 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.199293 10051 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:27.277491  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:27.238217 10046 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.252092  9983 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:27.213686 10110 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:27.298791  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:27.251971 10002 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:18:27.300523  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 87998.5us
I20260709 12:18:27.303355  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:27.304687  6505 raft_consensus.cc:2368] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 11. Current term is 12.
I20260709 12:18:27.308254  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:27.311059  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 91370.5us
I20260709 12:18:27.281230  9823 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.313314  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:27.316625  6508 raft_consensus.cc:2393] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 8: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:27.218717 10075 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:27.298686  9983 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:27.320432  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:27.292774 10046 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:27.323844  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:27.311573 10114 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:27.340693  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:27.341939 10106 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.342386 10015 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:27.342808  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:27.342945  9997 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:27.344121 10123 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:27.327324  9983 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:27.348855 10204 log.cc:927] Time spent T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 12:18:27.349454  6508 raft_consensus.cc:1275] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 8 index: 3. (index mismatch)
I20260709 12:18:27.351608  6475 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:27.326267 10016 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:18:27.241443  6406 rpcz_store.cc:269] 0709 12:18:08.616641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:08.617044 (+   403us) service_pool.cc:224] Handling call
0709 12:18:27.220688 (+18603644us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:17.699661 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:17.700467 (+   806us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:17.700541 (+    74us) write_op.cc:435] Acquired schema lock
0709 12:18:17.700573 (+    32us) tablet.cc:662] Decoding operations
0709 12:18:17.703406 (+  2833us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:17.703634 (+   228us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:17.703662 (+    28us) tablet.cc:685] Acquiring locks for 25 operations
0709 12:18:17.799703 (+ 96041us) tablet.cc:701] Row locks acquired
0709 12:18:17.799759 (+    56us) write_op.cc:260] PREPARE: finished
0709 12:18:17.799937 (+   178us) write_op.cc:270] Start()
0709 12:18:17.800077 (+   140us) write_op.cc:276] Timestamp: P: 1783599497799878 usec, L: 0
0709 12:18:17.800103 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:18:17.826833 (+ 26730us) log.cc:844] Serialized 9948 byte log entry
0709 12:18:19.476746 (+1649913us) op_driver.cc:464] REPLICATION: finished
0709 12:18:19.821225 (+344479us) write_op.cc:301] APPLY: starting
0709 12:18:19.831359 (+ 10134us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:19.832045 (+   686us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:19.832216 (+   171us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:20.161369 (+329153us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:20.161878 (+   509us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=25
0709 12:18:20.161943 (+    65us) write_op.cc:312] APPLY: finished
0709 12:18:20.162564 (+   621us) log.cc:844] Serialized 170 byte log entry
0709 12:18:20.311197 (+148633us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:20.331502 (+ 20305us) write_op.cc:454] Released schema lock
0709 12:18:20.340090 (+  8588us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:20.361832 (+ 21742us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:20.361877 (+    45us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":219414,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":289377,"apply.run_cpu_time_us":36974,"apply.run_wall_time_us":559552,"num_ops":25,"prepare.queue_time_us":291157,"prepare.run_cpu_time_us":29625,"prepare.run_wall_time_us":609841,"raft.queue_time_us":973500,"raft.run_cpu_time_us":7560,"raft.run_wall_time_us":149991,"replication_time_us":1676417,"thread_start_us":1027893,"threads_started":5,"wal-append.queue_time_us":728115,"wal-append.run_cpu_time_us":36113,"wal-append.run_wall_time_us":2260902}]]}
I20260709 12:18:27.378978  6583 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:27.344035  9917 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:27.355985  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:27.303179 10195 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:27.321678 10047 raft_consensus.cc:2804] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:27.328022  6473 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:27.302202  9739 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:27.382682  6584 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:27.383448  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:27.383733 10110 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:27.384147  9988 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:18:27.428864  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:27.359349  6505 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:27.388867  9917 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:27.433369  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:27.391882  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:27.434507  7269 raft_consensus.cc:1240] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:27.371248  9823 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:27.436076  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:27.436848  6507 raft_consensus.cc:2393] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 10: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:27.437382  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:27.438591  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 118299us
I20260709 12:18:27.444288  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:27.444936  6654 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:27.447537  6652 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:27.393720  6583 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:27.397145  6501 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 14: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:27.427400  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:27.428696  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 100759us
I20260709 12:18:27.450147  6473 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:27.385814  9893 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:27.309273 10190 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:27.398347 10114 raft_consensus.cc:697] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 8 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:27.454691  6473 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:27.456060  6503 raft_consensus.cc:2393] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 10: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:27.458652  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:27.436403 10002 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:27.463686  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:27.443981  9775 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:27.473554  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:27.456840  9920 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:27.473939 10114 consensus_queue.cc:237] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:27.397089 10106 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:27.476442 10203 log.cc:927] Time spent T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 12:18:27.476614  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:27.491221  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:27.492245  6658 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:27.426551 10015 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:27.457783 10191 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 12:18:27.476634 10106 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:27.495098 10002 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:27.498437 10125 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:27.590534 10047 raft_consensus.cc:697] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 10 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:27.556110 10123 raft_consensus.cc:697] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:27.598150 10125 raft_consensus.cc:697] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:27.608234  9875 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:27.637732  9823 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:27.639643 10015 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:27.641484 10202 log.cc:927] Time spent T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.283s	user 0.000s	sys 0.001s
I20260709 12:18:27.657924  9739 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:27.657859 10123 consensus_queue.cc:237] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:27.659032 10214 log.cc:927] Time spent T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.182s	user 0.002s	sys 0.000s
W20260709 12:18:27.658679 10206 log.cc:927] Time spent T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
W20260709 12:18:27.635181 10124 consensus_meta.cc:287] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.557s	user 0.008s	sys 0.004s
W20260709 12:18:27.672775 10182 consensus_meta.cc:287] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.557s	user 0.000s	sys 0.011s
I20260709 12:18:27.673290 10182 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:27.665467 10124 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:27.675437 10182 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:27.676474 10125 consensus_queue.cc:237] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:27.666555 10061 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:27.692770 10190 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:27.667985 10195 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:27.656035  9983 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:27.697923 10047 consensus_queue.cc:237] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:27.704227  9983 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:27.707137 10213 log.cc:927] Time spent T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 12:18:27.708629 10002 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:27.701879 10124 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:27.709992 10061 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:27.718879 10046 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:27.731106 10046 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:27.761963 10208 log.cc:927] Time spent T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.283s	user 0.002s	sys 0.000s
I20260709 12:18:27.763329  6654 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
W20260709 12:18:28.418907  6581 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.983s	user 0.017s	sys 0.022s
W20260709 12:18:28.419652  6581 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.986s	user 0.017s	sys 0.022s
I20260709 12:18:28.561094 10128 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:28.562119 10128 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:28.565074 10128 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:28.593171 10242 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:28.737047 10242 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:28.739213 10242 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:28.746543  6649 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.289s	user 0.010s	sys 0.038s
I20260709 12:18:28.746652 10245 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:18:28.747632  6649 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.290s	user 0.010s	sys 0.038s
I20260709 12:18:28.747546 10245 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:28.748903  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1751) took 1552 ms. Trace:
I20260709 12:18:28.749260  6649 rpcz_store.cc:276] 0709 12:18:27.196360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.196500 (+   140us) service_pool.cc:224] Handling call
0709 12:18:27.196753 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:27.196925 (+   172us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:27.451513 (+254588us) log.cc:844] Serialized 14 byte log entry
0709 12:18:28.748259 (+1296746us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:28.748310 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:28.748656 (+   346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:28.748724 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:28.748849 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":200769,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"thread_start_us":1539002,"threads_started":2,"wal-append.queue_time_us":256477,"wal-append.run_cpu_time_us":1114,"wal-append.run_wall_time_us":1001124}
I20260709 12:18:28.751498 10245 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:28.764868  6500 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 413) took 1543 ms. Trace:
I20260709 12:18:28.765376  6500 rpcz_store.cc:276] 0709 12:18:27.221249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.221478 (+   229us) service_pool.cc:224] Handling call
0709 12:18:27.240991 (+ 19513us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:27.241367 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:27.241393 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:27.241880 (+   487us) log.cc:844] Serialized 26 byte log entry
0709 12:18:27.376867 (+134987us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:27.376908 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.376985 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:28.764581 (+1387596us) raft_consensus.cc:1685] finished
0709 12:18:28.764605 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:28.764768 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10809,"raft.queue_time_us":3527,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":28785,"spinlock_wait_cycles":751488,"thread_start_us":134620,"threads_started":1,"wal-append.queue_time_us":234453,"wal-append.run_cpu_time_us":2820,"wal-append.run_wall_time_us":1183749}
I20260709 12:18:28.767693  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:28.771410  6500 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 2034) took 1289 ms. Trace:
I20260709 12:18:28.772732  6500 rpcz_store.cc:276] 0709 12:18:27.481882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:28.767402 (+1285520us) service_pool.cc:224] Handling call
0709 12:18:28.771305 (+  3903us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:28.759343 10128 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:28.774199 10128 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:28.782681  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:28.783488  6503 raft_consensus.cc:2393] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 9: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:28.783885  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 2033) took 1317 ms. Trace:
I20260709 12:18:28.784153  6503 rpcz_store.cc:276] 0709 12:18:27.466093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.478043 (+ 11950us) service_pool.cc:224] Handling call
0709 12:18:28.783802 (+1305759us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:28.790697  6508 raft_consensus.cc:2393] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 7: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:28.794726 10128 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:28.795892  6503 raft_consensus.cc:1240] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:18:28.796558  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 1958) took 1328 ms. Trace:
I20260709 12:18:28.800431  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 1955) took 1373 ms. Trace:
I20260709 12:18:28.800877  6508 rpcz_store.cc:276] 0709 12:18:27.426903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.427108 (+   205us) service_pool.cc:224] Handling call
0709 12:18:28.800226 (+1373118us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b540039f654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:28.800376 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3085070208}
W20260709 12:18:28.809949  6579 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.353s	user 0.027s	sys 0.066s
I20260709 12:18:28.796883  6503 rpcz_store.cc:276] 0709 12:18:27.472629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:28.795312 (+1322683us) service_pool.cc:224] Handling call
0709 12:18:28.795679 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:28.796271 (+   592us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:28.796498 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:28.810612 10247 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:28.811933 10247 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:28.814538 10247 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:28.810592  6579 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.355s	user 0.027s	sys 0.066s
I20260709 12:18:28.815625 10245 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:28.816378  6579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1775) took 1642 ms. Trace:
I20260709 12:18:28.816900  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1758) took 1563 ms. Trace:
I20260709 12:18:28.799263  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1792) took 1594 ms. Trace:
I20260709 12:18:28.817467  6581 rpcz_store.cc:276] 0709 12:18:27.204430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.238360 (+ 33930us) service_pool.cc:224] Handling call
0709 12:18:27.277157 (+ 38797us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:27.318305 (+ 41148us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:27.368130 (+ 49825us) log.cc:844] Serialized 14 byte log entry
0709 12:18:28.743578 (+1375448us) log.cc:844] Serialized 14 byte log entry
0709 12:18:28.798492 (+ 54914us) log.cc:844] Serialized 14 byte log entry
0709 12:18:28.798760 (+   268us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:28.798820 (+    60us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:28.798920 (+   100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:28.799011 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:28.799187 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44004,"prepare.queue_time_us":395873,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":102,"raft.queue_time_us":97813,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":6259,"thread_start_us":1363549,"threads_started":3,"wal-append.queue_time_us":61395,"wal-append.run_cpu_time_us":887,"wal-append.run_wall_time_us":1001202}
I20260709 12:18:28.816818  6579 rpcz_store.cc:276] 0709 12:18:27.173697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.186081 (+ 12384us) service_pool.cc:224] Handling call
0709 12:18:27.203286 (+ 17205us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:27.222702 (+ 19416us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:27.285527 (+ 62825us) log.cc:844] Serialized 14 byte log entry
0709 12:18:28.816003 (+1530476us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:28.816057 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:28.816105 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:28.816186 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:28.816303 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28485,"prepare.queue_time_us":149353,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":221315,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":8261,"thread_start_us":1514601,"threads_started":2,"wal-append.queue_time_us":236928,"wal-append.run_cpu_time_us":1747,"wal-append.run_wall_time_us":1109683}
I20260709 12:18:28.797346 10243 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.821853 10243 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:28.817301  6656 rpcz_store.cc:276] 0709 12:18:27.253995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.254126 (+   131us) service_pool.cc:224] Handling call
0709 12:18:27.254459 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:27.254655 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:28.787815 (+1533160us) spinlock_profiling.cc:243] Waited 2.17 ms on lock 0x7b18003ee120. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2875a314bc 00007f2872d7f48b 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d99455 00007f28798813c7 00007f2875e939a7 00007f2875ea30bf
0709 12:18:28.787956 (+   141us) log.cc:844] Serialized 14 byte log entry
0709 12:18:28.815376 (+ 27420us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:28.815425 (+    49us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:28.815475 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:28.816741 (+  1266us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:28.816848 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":198813,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":2265,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":86,"spinlock_wait_cycles":4992896,"thread_start_us":222892,"threads_started":2,"wal-append.queue_time_us":26077}
I20260709 12:18:28.816110 10245 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:28.825158 10243 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:28.841265 10087 raft_consensus.cc:3055] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:28.841805 10087 raft_consensus.cc:740] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:28.842350 10128 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:28.842882 10128 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:28.842780 10087 consensus_queue.cc:260] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:28.845125 10128 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:28.846169 10087 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:28.859509  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:28.860888 10054 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:28.861701 10054 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:28.864053 10054 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:28.860845 10128 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:28.864778 10128 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:28.867311 10128 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:28.869196  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:28.869822  6656 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:18:28.871135  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:28.877877  6549 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.444s	user 0.018s	sys 0.050s
W20260709 12:18:28.878638  6549 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.444s	user 0.019s	sys 0.051s
I20260709 12:18:28.882292 10252 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:28.882939 10252 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:28.883543 10054 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:28.884141 10054 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:28.886077 10253 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.894202 10253 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:28.887660  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:28.895364  6579 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:18:28.891898 10122 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:28.896476 10122 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:28.898746 10122 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:28.899259 10122 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:28.899860  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:28.899727 10122 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:28.900542  7265 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:28.902096 10122 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:28.893613 10252 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:28.886332 10054 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:28.904563 10087 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.907841 10087 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:28.910674  6577 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.514s	user 0.020s	sys 0.020s
W20260709 12:18:28.911134  6577 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.514s	user 0.020s	sys 0.020s
I20260709 12:18:28.911878  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1779) took 1737 ms. Trace:
I20260709 12:18:28.914477 10087 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:28.905674 10253 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:28.918620  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:28.919972  6656 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:18:28.919359  6649 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 12:18:28.926729  6584 consensus_meta.cc:287] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 1.483s	user 0.004s	sys 0.014s
I20260709 12:18:28.927459  6584 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
W20260709 12:18:28.926976  6652 consensus_meta.cc:287] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 1.470s	user 0.012s	sys 0.006s
I20260709 12:18:28.928601  6652 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:28.933195 10087 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:28.933629  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:28.933744 10087 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:28.934289  6654 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7.
I20260709 12:18:28.935961 10087 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:28.938333  7240 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:28.912259  6577 rpcz_store.cc:276] 0709 12:18:27.174790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.177897 (+  3107us) service_pool.cc:224] Handling call
0709 12:18:27.183162 (+  5265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:27.200980 (+ 17818us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:27.369692 (+168712us) log.cc:844] Serialized 14 byte log entry
0709 12:18:28.911541 (+1541849us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:28.911594 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:28.911652 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:28.911720 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:28.911824 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10673,"prepare.queue_time_us":96155,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"thread_start_us":1644677,"threads_started":2,"wal-append.queue_time_us":265942,"wal-append.run_cpu_time_us":1525,"wal-append.run_wall_time_us":1021222}
I20260709 12:18:28.942312 10253 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:28.931651 10054 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 12:18:28.943008  7240 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:29.060650 10128 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:29.061339 10128 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:29.061992 10128 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:29.068486  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 125758us
I20260709 12:18:28.978551 10254 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.073575 10254 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:28.942631 10250 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:28.974066 10122 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:29.082942 10087 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:29.086248 10087 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:29.089277 10087 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:29.094940 10122 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:29.098050 10122 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:29.099377 10250 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:29.105187 10250 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:29.115792 10128 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:28.987488  6547 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.553s	user 0.010s	sys 0.010s
W20260709 12:18:29.157510  6547 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.723s	user 0.011s	sys 0.010s
I20260709 12:18:29.157094 10243 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:29.161655 10243 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:29.165402 10243 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:29.171161 10054 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.171952 10054 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:29.176939 10054 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:29.179244 10122 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.181512 10122 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:29.335876 10122 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:29.212359  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:29.218758  7238 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.833s (SENT) [suppressed 2 similar messages]
I20260709 12:18:28.957535 10196 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.339982 10196 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:29.344318 10196 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:29.361562 10128 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:29.363220  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:29.366923  6506 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 1959) took 1893 ms (client timeout 2223 ms). Trace:
W20260709 12:18:29.367588  6506 rpcz_store.cc:269] 0709 12:18:27.473034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.362883 (+1889849us) service_pool.cc:224] Handling call
0709 12:18:29.366767 (+  3884us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:29.373685 10259 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:29.374660 10259 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:29.378482 10259 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:28.951812  6575 raft_consensus.cc:2368] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10.
I20260709 12:18:28.943387  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 1797) took 1651 ms. Trace:
I20260709 12:18:29.416016  6584 rpcz_store.cc:276] 0709 12:18:27.277113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.355638 (+ 78525us) service_pool.cc:224] Handling call
0709 12:18:28.928241 (+1572603us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:29.315173 10087 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:29.316377 10243 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:29.318382 10247 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:28.952468 10184 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:28.946741  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:29.326943 10245 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:29.077176 10254 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:29.271781  6658 consensus_meta.cc:287] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 1.762s	user 0.012s	sys 0.004s
I20260709 12:18:29.437925  6658 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
W20260709 12:18:29.438681  6658 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 1762) took 2007 ms (client timeout 2239 ms). Trace:
W20260709 12:18:29.439208  6658 rpcz_store.cc:269] 0709 12:18:27.430818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.468516 (+ 37698us) service_pool.cc:224] Handling call
0709 12:18:29.438589 (+1970073us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:29.389104  9920 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:29.391587  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 172853us
I20260709 12:18:28.942970  7265 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:18:29.421609 10087 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:29.422403  6657 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.920s	user 0.013s	sys 0.016s
I20260709 12:18:29.404317 10260 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:29.454430 10260 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:29.432554  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:29.456830  6584 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:29.458143  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:29.459261  6584 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:29.435045  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:29.436698  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:29.436923  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:29.464706  6579 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:29.444077  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:29.469061  6581 raft_consensus.cc:2393] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 10: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:29.470571  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:29.471621  6581 raft_consensus.cc:2410] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:18:29.409216 10122 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.447949  7268 raft_consensus.cc:1240] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:29.448920  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:29.472681 10122 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:29.475503 10122 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:29.479095  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:29.479966  6581 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:29.428141 10259 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:29.485131  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:29.435109 10245 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:29.490314  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:29.443132 10264 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.489077  6579 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:18:29.477348  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:29.477490 10254 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:29.485635 10267 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:29.435304 10247 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:29.438702 10243 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:29.504942  6584 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:29.503611  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:29.502555 10268 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.505159  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:29.573411  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:29.452606  9920 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:29.512439 10273 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:29.636831  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 158009us
I20260709 12:18:29.502863 10122 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:29.442145 10184 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:29.503650 10272 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:29.503280 10264 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:29.583334 10269 consensus_queue.cc:1059] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 12:18:29.501170 10245 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:29.664243  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:29.590690 10257 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:29.500998 10196 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:29.653386  6654 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:18:29.507478 10259 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:29.511484 10254 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:29.589874  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:29.664913  7271 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:29.637755 10277 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.679144  6576 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
W20260709 12:18:29.667301 10250 consensus_meta.cc:287] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.561s	user 0.004s	sys 0.007s
I20260709 12:18:29.574795 10267 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:29.670864 10270 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:18:29.712670 10184 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:29.665280 10271 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:29.665594 10278 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.673496  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:29.670674 10254 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:29.668874 10247 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:29.667564 10122 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:29.728101 10051 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:29.729360  6657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1755) took 2516 ms. Trace:
I20260709 12:18:29.768909  7240 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:29.646292 10268 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:29.707561 10277 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:29.733515  6651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1744) took 2581 ms. Trace:
I20260709 12:18:29.705582 10260 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:29.672362 10259 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:29.698637 10196 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:29.768939  6657 rpcz_store.cc:276] 0709 12:18:27.213154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.213300 (+   146us) service_pool.cc:224] Handling call
0709 12:18:27.218857 (+  5557us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:27.434613 (+215756us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:27.434644 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:27.466735 (+ 32091us) log.cc:844] Serialized 26 byte log entry
0709 12:18:29.453789 (+1987054us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:29.453847 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:29.453931 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:29.729096 (+275165us) raft_consensus.cc:1685] finished
0709 12:18:29.729117 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:29.729263 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":94562,"raft.queue_time_us":78831,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":7672,"thread_start_us":2091065,"threads_started":2,"wal-append.queue_time_us":1991172}
I20260709 12:18:29.651309 10273 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:29.666726 10272 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:29.714818 10274 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:29.772675  6651 rpcz_store.cc:276] 0709 12:18:27.152228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.178262 (+ 26034us) service_pool.cc:224] Handling call
0709 12:18:27.192872 (+ 14610us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:27.193270 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:27.193297 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:27.193844 (+   547us) log.cc:844] Serialized 52 byte log entry
0709 12:18:27.404553 (+210709us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:27.443630 (+ 39077us) log.cc:844] Serialized 14 byte log entry
0709 12:18:27.454447 (+ 10817us) log.cc:844] Serialized 14 byte log entry
0709 12:18:27.454631 (+   184us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.454705 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:29.672606 (+2217901us) spinlock_profiling.cc:243] Waited 4.77 ms on lock 0x7b18002ddea0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c522 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:18:29.733198 (+ 60592us) raft_consensus.cc:1685] finished
0709 12:18:29.733230 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:29.733436 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34074,"prepare.queue_time_us":247560,"prepare.run_cpu_time_us":1494,"prepare.run_wall_time_us":10068,"raft.queue_time_us":18862,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":11235,"spinlock_wait_cycles":21830144,"thread_start_us":210260,"threads_started":1,"wal-append.queue_time_us":346431,"wal-append.run_cpu_time_us":3369,"wal-append.run_wall_time_us":1122240}
I20260709 12:18:29.710325 10250 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:29.766522  7266 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:29.797475 10122 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:29.793429 10051 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.798447 10051 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:29.799386 10267 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:29.801126 10196 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:29.801543 10268 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:29.802244  9920 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:29.802372 10272 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:29.802912 10277 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:29.803184 10243 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:29.734539 10278 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:29.805105 10278 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:29.704942 10257 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:29.807845 10257 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:29.769623 10271 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:29.810593 10271 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:29.810927 10273 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:29.796981 10250 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:29.797788 10184 raft_consensus.cc:3055] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:29.812011 10184 raft_consensus.cc:740] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:29.812916 10184 consensus_queue.cc:260] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:29.815805  6581 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:29.795962 10245 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:29.819872  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:29.785413 10276 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:29.788862 10087 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:29.822214 10051 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:29.824420 10276 raft_consensus.cc:697] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 8 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:29.826028 10276 consensus_queue.cc:237] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:29.830895  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:29.830693 10184 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:29.833179  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:29.835624  6653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 535) took 2831 ms. Trace:
I20260709 12:18:29.836038  6653 rpcz_store.cc:276] 0709 12:18:27.003915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.009594 (+  5679us) service_pool.cc:224] Handling call
0709 12:18:27.021678 (+ 12084us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:27.205201 (+183523us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:27.205245 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:27.211767 (+  6522us) log.cc:844] Serialized 26 byte log entry
0709 12:18:27.370452 (+158685us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:27.370536 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.370772 (+   236us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:27.386767 (+ 15995us) spinlock_profiling.cc:243] Waited 1.5 ms on lock 0x7b48000cda58. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287adb2aa1 00007f287ada9b37 00007f287ada99ca 00007f287aea126f 00007f287aea0f37 00007f287aeb555c 00007f287aeb5508 00007f287aeb5488 00007f287aeb5443 00007f287aeb46ec 00007f2879ddea96
0709 12:18:29.830218 (+2443451us) raft_consensus.cc:1685] finished
0709 12:18:29.830241 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:29.835557 (+  5316us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":161920,"raft.queue_time_us":160826,"raft.run_cpu_time_us":3162,"raft.run_wall_time_us":177421,"spinlock_wait_cycles":3451264,"thread_start_us":118002,"threads_started":1,"wal-append.queue_time_us":241722,"wal-append.run_cpu_time_us":3724,"wal-append.run_wall_time_us":1235464}
I20260709 12:18:29.829563 10274 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:29.840975  6550 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.407s	user 0.018s	sys 0.029s
I20260709 12:18:29.841017 10274 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:29.844859  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:29.846285  6651 raft_consensus.cc:2410] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:18:29.847069  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:29.848488  6657 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:29.849696  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:29.846046 10245 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:29.850890  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:29.850970 10245 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:29.853148 10245 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:29.861248  6657 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
W20260709 12:18:29.862325  6548 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.427s	user 0.009s	sys 0.020s
W20260709 12:18:29.862910  6548 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.428s	user 0.009s	sys 0.020s
I20260709 12:18:29.867096  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 1947) took 2584 ms. Trace:
W20260709 12:18:29.869225  6550 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.435s	user 0.018s	sys 0.029s
I20260709 12:18:29.876382  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:29.877070  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:29.879071  6657 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:29.884512 10273 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:29.885324 10273 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:29.886189  6549 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:18:29.887712 10268 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:29.888713  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:29.889672  7274 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:18:29.891809  6581 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:18:29.892601  6581 raft_consensus.cc:606] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:29.892944  6581 raft_consensus.cc:614] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:29.894093  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:29.894793  6581 raft_consensus.cc:2368] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 7. Current term is 8.
I20260709 12:18:29.888170 10268 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:29.895697  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:29.895910 10268 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:29.896683  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:29.897449  6576 raft_consensus.cc:2393] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 10: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:29.897312  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:29.898288  6656 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:29.898458  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:29.900183  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:29.902554  6549 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:18:29.890166  6509 rpcz_store.cc:276] 0709 12:18:27.283037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.283347 (+   310us) service_pool.cc:224] Handling call
0709 12:18:27.283626 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:27.484778 (+201152us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:28.874598 (+1389820us) log.cc:844] Serialized 14 byte log entry
0709 12:18:28.931915 (+ 57317us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:28.931971 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:28.932026 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:29.866823 (+934797us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b5400388454. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:29.866876 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:29.866994 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":26455,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":253,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"spinlock_wait_cycles":445867264,"thread_start_us":83270,"threads_started":2,"wal-append.queue_time_us":64586}
I20260709 12:18:29.902710 10260 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:29.907253 10260 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:29.910000 10260 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:29.910753 10260 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:29.911471 10260 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:29.903429  6657 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:29.906584 10285 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.913654 10257 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.913724 10285 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:29.913861  6549 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:18:29.912628 10259 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:29.915366  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:29.918717 10285 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:29.919531  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:29.919947  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:29.981595  6473 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.044s	sys 0.082s
W20260709 12:18:29.982067  6473 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.044s	sys 0.082s
I20260709 12:18:29.982033  7271 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:29.919363  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:29.920521 10254 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.922144  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:29.983525  7268 raft_consensus.cc:2393] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 6: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:29.983677  6657 raft_consensus.cc:2393] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 10: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:29.922227 10257 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:29.912724  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:29.923287  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:29.923918  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:29.924928  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:29.992743  6656 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:29.927161  7269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38558 (request call id 552) took 2902 ms. Trace:
I20260709 12:18:29.934741  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:29.938076 10272 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:29.922049 10191 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:29.942385  7270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38562 (request call id 616) took 1052 ms. Trace:
I20260709 12:18:29.943969 10294 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:29.994445  7269 rpcz_store.cc:276] 0709 12:18:27.024620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.025737 (+  1117us) service_pool.cc:224] Handling call
0709 12:18:27.026038 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:27.434884 (+408846us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:29.926890 (+2492006us) spinlock_profiling.cc:243] Waited 408 ms on lock 0x7b54005de654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:29.927054 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":938946304}
I20260709 12:18:29.887722 10273 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:29.945086  6575 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 1791) took 2745 ms (client timeout 2159 ms). Trace:
W20260709 12:18:29.998157  6575 rpcz_store.cc:269] 0709 12:18:27.200122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.237097 (+ 36975us) service_pool.cc:224] Handling call
0709 12:18:29.944860 (+2707763us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b54004303d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:29.944974 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3841371392}
I20260709 12:18:29.922016  6501 raft_consensus.cc:2393] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 10: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:29.968750  6581 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
W20260709 12:18:29.982740  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 547121us
I20260709 12:18:30.000063 10269 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:30.001502 10296 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:30.001216 10051 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:30.002501 10296 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:30.002635 10051 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:30.002506  6473 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:30.005291 10184 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:30.005653  6473 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:30.005854 10184 raft_consensus.cc:697] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:30.006287  6550 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:30.006726 10051 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:30.006927 10184 consensus_queue.cc:237] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.3, Last appended by leader: 3, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:30.007752  6473 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:30.008265 10051 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:30.008848 10051 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:30.009711 10051 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:30.011149 10281 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:30.000725 10269 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:30.012172 10281 raft_consensus.cc:697] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:30.013616 10281 consensus_queue.cc:237] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:30.016391  6550 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:30.009302  6473 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:30.021757  6473 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:30.024067  6473 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:30.022612 10283 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:30.025619 10051 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:30.026103 10274 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:18:30.028985  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 115970us
I20260709 12:18:30.029999 10297 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:30.025520 10301 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:29.996106  7270 rpcz_store.cc:276] 0709 12:18:28.890332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:28.986191 (+ 95859us) service_pool.cc:224] Handling call
0709 12:18:28.986645 (+   454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:29.078450 (+ 91805us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:29.078501 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:29.078559 (+    58us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:29.078609 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:29.942086 (+863477us) spinlock_profiling.cc:243] Waited 91.5 ms on lock 0x7b5400565754. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:29.942148 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:29.942313 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":210501504}
I20260709 12:18:30.096401 10301 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:30.096889 10301 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:30.014535 10269 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:30.015470 10298 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:18:30.048456  6504 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 2032) took 2582 ms (client timeout 2120 ms). Trace:
I20260709 12:18:29.980835 10259 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:30.054241  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.059602  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:30.060673  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.037765 10285 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:30.062216  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:18:30.063268  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 137240us
I20260709 12:18:30.128159  6657 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:30.091995 10260 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:30.046092 10294 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:30.058051  6548 connection.cc:256] RPC call timeout handler was delayed by -1.2075447610002357s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15200000000000002s for a response
I20260709 12:18:30.073211  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:30.074033  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 419173us
I20260709 12:18:29.996721 10254 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:30.138365 10254 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:30.045042 10191 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:30.075974 10273 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:30.099555 10295 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:30.124289 10259 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:30.093703  6582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1789) took 2901 ms. Trace:
I20260709 12:18:30.076951 10051 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:30.055819 10272 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:30.092130 10267 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:30.099803  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1798) took 2815 ms. Trace:
I20260709 12:18:30.127228  6474 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:18:30.100844  6504 rpcz_store.cc:269] 0709 12:18:27.457985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.469561 (+ 11576us) service_pool.cc:224] Handling call
0709 12:18:30.040703 (+2571142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:30.071339  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:18:30.133873  6548 connection.cc:256] RPC call timeout handler was delayed by -1.0230447610001647s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17250000000000001s for a response
I20260709 12:18:30.147603  6548 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:30.135433  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:30.095352 10297 raft_consensus.cc:697] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:30.095780 10283 raft_consensus.cc:697] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:18:30.140646  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 102587us
I20260709 12:18:30.140836  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.076737 10257 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:30.126209 10298 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:30.125692  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:30.130293  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:30.149261  6653 raft_consensus.cc:2393] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 9: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:30.151181  6657 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:18:30.124055 10286 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:30.143144  6582 rpcz_store.cc:276] 0709 12:18:27.186532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.191658 (+  5126us) service_pool.cc:224] Handling call
0709 12:18:27.191930 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:27.192173 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:29.823352 (+2631179us) spinlock_profiling.cc:243] Waited 3.36 ms on lock 0x7b180042d7e0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2875a314bc 00007f2872d7f48b 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d99455 00007f28798813c7 00007f2875e939a7 00007f2875ea30bf
0709 12:18:29.823560 (+   208us) log.cc:844] Serialized 14 byte log entry
0709 12:18:30.086883 (+263323us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:30.086957 (+    74us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:30.087036 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.087580 (+   544us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:30.087744 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":729,"prepare.queue_time_us":50319,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":14020,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":167,"spinlock_wait_cycles":7734144,"thread_start_us":344490,"threads_started":3,"wal-append.queue_time_us":212287}
I20260709 12:18:30.144822  6580 rpcz_store.cc:276] 0709 12:18:27.284777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.484565 (+199788us) service_pool.cc:224] Handling call
0709 12:18:29.675315 (+2190750us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:29.675539 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:29.771968 (+ 96429us) log.cc:844] Serialized 14 byte log entry
0709 12:18:30.099352 (+327384us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:30.099412 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:30.099457 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.099552 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:30.099727 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":114854,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":51,"raft.queue_time_us":45455,"thread_start_us":348903,"threads_started":2,"wal-append.queue_time_us":324444}
I20260709 12:18:30.092413 10293 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:18:30.155985  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 58159us
I20260709 12:18:30.157536  6548 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:30.159464  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.159759  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:30.165408  7240 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:30.167137  6507 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:30.167256  6505 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:30.160245  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:30.168486  6503 raft_consensus.cc:2393] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 10: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:18:30.150888 10299 log.cc:927] Time spent T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:18:30.156416 10184 consensus_queue.cc:1059] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 12:18:30.164021  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.160676  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.221927  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.222555  6581 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:30.223475  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.224325  6581 raft_consensus.cc:2410] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:18:30.225139  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.168797  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.225926  6507 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:18:30.169059  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.170908  6548 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:30.227291  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 56443.2us
I20260709 12:18:30.228083  6548 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:30.229700 10259 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:30.230710  9875 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:30.231205  9875 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:30.232402 10260 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:30.233135 10260 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:30.141342 10274 raft_consensus.cc:697] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:30.162413 10298 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:30.153373 10304 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:30.234361 10304 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:30.234777 10304 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:30.234658 10274 consensus_queue.cc:237] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:30.172179 10253 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:30.239446 10253 raft_consensus.cc:697] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 8 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:30.240578 10253 consensus_queue.cc:237] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:30.242270 10264 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:30.174053  6622 leader_election.cc:400] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 7. Current term is 8.
I20260709 12:18:30.174604 10242 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:30.243316 10298 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:30.177055  9926 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:30.178342 10283 consensus_queue.cc:237] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:30.179705 10297 consensus_queue.cc:237] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:30.181242 10191 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:30.134500 10285 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:30.181537 10294 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:30.144351 10273 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:30.169566 10284 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:30.182236 10254 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:30.158205 10267 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:30.161018 10300 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:30.280871  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:30.183182 10268 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:30.184121 10286 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:30.283605  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.178287  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.151852 10295 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:30.187040 10272 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:30.193688  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.193168 10293 raft_consensus.cc:697] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:30.209592  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:30.210261  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:30.290612  6657 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:18:30.211205  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.286094  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.211278  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:30.212134  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:30.208933  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:30.214977  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.215530  6505 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:30.216153  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.217413  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.218883 10301 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:30.220736 10196 raft_consensus.cc:3055] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:30.244176  6622 leader_election.cc:403] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:18:30.176203  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:30.278283  6581 raft_consensus.cc:2368] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 6. Current term is 7.
I20260709 12:18:30.282861  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.264528 10259 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:30.284493  6504 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:30.296669  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.297821 10273 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:30.300487  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 69282.5us
I20260709 12:18:30.300877  6506 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:30.308596  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.311960  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:30.312707  6657 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:30.308465  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.316362  6500 raft_consensus.cc:1275] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:30.318436  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:30.319746  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 145839us
I20260709 12:18:30.321527  6575 raft_consensus.cc:2393] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 8: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:30.321609  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.321704  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.322650  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.323616  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.334086 10293 consensus_queue.cc:237] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:30.336540  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:30.337913  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:30.338975  6653 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:18:30.303918 10196 raft_consensus.cc:740] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:30.354703  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.324771  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.341028  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.352180  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.358012  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.358701  6580 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:30.359606  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.313694 10075 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:30.343990  6505 raft_consensus.cc:1275] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:18:30.381093 10286 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:30.360200  6580 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
I20260709 12:18:30.504846  6505 pending_rounds.cc:85] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 4
I20260709 12:18:30.381548  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.382803  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.368745  6504 raft_consensus.cc:2410] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:18:30.369161  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.340235  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:30.360848  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.510914  6509 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:18:30.369107  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:30.369666  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.506187  6505 pending_rounds.cc:107] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:18:30.514735  6505 raft_consensus.cc:2889] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:30.380261  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:30.518256  6654 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:30.371459  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.373627  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:30.521127  6656 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:18:30.514124  6581 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:18:30.549206  6582 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 14: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:30.520054  6575 raft_consensus.cc:2468] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 5.
I20260709 12:18:30.509011  6500 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:30.366082  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1799) took 3079 ms. Trace:
I20260709 12:18:30.324992  6622 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:30.551483  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:30.552451  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:30.553344  6583 rpcz_store.cc:276] 0709 12:18:27.286602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.501791 (+215189us) service_pool.cc:224] Handling call
0709 12:18:30.279375 (+2777584us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:30.294731 (+ 15356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:30.294764 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:30.294815 (+    51us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:30.294847 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.365757 (+ 70910us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b5400266bd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:30.365826 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:30.366007 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34845952}
I20260709 12:18:30.558391  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:30.558442  6650 raft_consensus.cc:2368] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10.
I20260709 12:18:30.560262  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:30.337746 10301 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:30.560870  6656 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:30.317808  9926 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 7. Current term is 8.
I20260709 12:18:30.561380  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:30.565384  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:30.562999  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.361053  7265 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2"
dest_uuid: "20a202c76244485fbc8f1610473db8b9"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:48510
I20260709 12:18:30.563141  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.567093  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:30.564505  7238 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:30.565966  6653 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:30.564983  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:30.573174  6657 raft_consensus.cc:2393] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 8: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:30.573909  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.574154 10301 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:30.575769  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.576419  6575 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 6.
I20260709 12:18:30.342730 10295 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:30.340509  9875 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:30.565459 10075 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:30.574646 10301 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:30.579896  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 259357us
I20260709 12:18:30.580163  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.580770  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:30.582249  6503 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:18:30.572132  9821 raft_consensus.cc:2804] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:30.575421  7265 raft_consensus.cc:606] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:30.584789  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:30.599010  6656 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:30.570556 10196 consensus_queue.cc:260] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:30.564009 10304 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:30.600102  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:30.601562  6656 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:30.601655 10304 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:30.602928  9821 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:30.603462  9821 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:30.585253  9875 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:30.592200  6622 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:30.605481  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:30.606386  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:30.607125 10242 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:30.607793 10242 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:30.608177 10242 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:30.608626 10242 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:30.612207 10205 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 3.143s	user 0.016s	sys 0.014s
I20260709 12:18:30.612475  7265 raft_consensus.cc:630] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Rejecting request to transfer leadership because tablet server f2b06aebe97249a4877d798b8118ece4 is not a voter in the active config
I20260709 12:18:30.613062  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.613471  6622 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:30.613744  6577 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:30.612534 10075 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:30.614492 10284 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:30.614692  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.615338  6577 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 5.
I20260709 12:18:30.616214  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:30.617449 10284 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:30.619045 10196 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:30.616240  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.620874  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.622148  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.622804  6577 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:30.616394 10301 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:30.623730  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:30.623950  6475 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:30.624531  6577 raft_consensus.cc:2368] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10.
I20260709 12:18:30.625578  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:30.626127  6577 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:30.626672 10304 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:18:30.633589  6652 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 1759) took 3345 ms (client timeout 2693 ms). Trace:
W20260709 12:18:30.634068  6652 rpcz_store.cc:269] 0709 12:18:27.288421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.291774 (+  3353us) service_pool.cc:224] Handling call
0709 12:18:30.633353 (+3341579us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54000758d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:30.633477 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":330104192}
I20260709 12:18:30.619145  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:30.636973 10087 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:30.645979 10242 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:30.648651 10242 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:30.650070  6473 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:30.649353  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:30.650799  6550 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:30.651405  6622 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:30.651813  6550 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:30.652272 10087 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:30.652092  9821 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:30.652698 10087 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:30.658721 10274 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:30.666869  6475 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:18:30.673471 10209 log.cc:927] Time spent T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 3.086s	user 0.018s	sys 0.011s
I20260709 12:18:30.673725  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.809890  6504 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
W20260709 12:18:30.810032  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 143305us
I20260709 12:18:30.810611  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 439) took 3637 ms. Trace:
I20260709 12:18:30.810917  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:30.809096 10087 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:30.675590  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:30.813714  9920 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:30.813787  6649 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:18:30.811002  6578 rpcz_store.cc:276] 0709 12:18:27.173110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.176491 (+  3381us) service_pool.cc:224] Handling call
0709 12:18:27.176807 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:27.177153 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:27.177179 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:27.177664 (+   485us) log.cc:844] Serialized 26 byte log entry
0709 12:18:27.290046 (+112382us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:27.290106 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:27.290212 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:30.672895 (+3382683us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:18:30.810323 (+137428us) raft_consensus.cc:1685] finished
0709 12:18:30.810346 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:30.810550 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1008,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":11240,"spinlock_wait_cycles":24776704,"thread_start_us":111967,"threads_started":1,"wal-append.queue_time_us":288131}
I20260709 12:18:30.814867  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:30.815327  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.676635  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.816288 10255 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:30.816797 10255 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:30.817193 10255 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:30.817426  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.678185 10245 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:30.678125  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:30.818603  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:30.819710  6649 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:18:30.821368  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:30.822232  6649 raft_consensus.cc:2410] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:18:30.823671  6649 raft_consensus.cc:1275] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:18:30.814275  9920 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:30.685240  9821 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:30.824394  9920 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:30.825372  9821 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:30.666760  6473 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:18:30.688258  6650 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 1754) took 3484 ms (client timeout 2159 ms). Trace:
W20260709 12:18:30.826843  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 160164us
W20260709 12:18:30.826735  6650 rpcz_store.cc:269] 0709 12:18:27.203812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.203915 (+   103us) service_pool.cc:224] Handling call
0709 12:18:30.687996 (+3484081us) spinlock_profiling.cc:243] Waited 3.34 s on lock 0x7b54003e8ad4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:30.688178 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7674770816}
I20260709 12:18:30.827739  6473 leader_election.cc:400] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 6. Current term is 7.
I20260709 12:18:30.828048  6473 leader_election.cc:403] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:30.827899  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.828608  6578 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
W20260709 12:18:30.690050  6502 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.444s	user 0.015s	sys 0.027s
I20260709 12:18:30.829339  6473 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:30.829461  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.830411  6578 raft_consensus.cc:2410] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 11 because replica has last-logged OpId of term: 8 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 2.
I20260709 12:18:30.830813 10295 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:30.831239 10295 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:30.831272  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.831754 10295 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:30.694329  7265 raft_consensus.cc:1240] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:30.653688 10273 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:30.695196  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:30.697283  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:30.698686  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:30.702028  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:30.832135  6578 raft_consensus.cc:2393] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 8: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:30.836829  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.837404  6578 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
I20260709 12:18:30.837658 10286 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:30.838070 10286 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:30.838328  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.838451 10286 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:30.835721 10255 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:30.839831  7295 debug-util.cc:398] Leaking SignalData structure 0x7b08002f8020 after lost signal to thread 5355
I20260709 12:18:30.841109 10255 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:30.841266  7295 debug-util.cc:398] Leaking SignalData structure 0x7b08002f8540 after lost signal to thread 6441
I20260709 12:18:30.832710  6473 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:30.843581 10306 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:30.843989 10306 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:30.702207  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:30.844463 10306 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:30.844904  6575 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:30.704627 10298 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:30.705775  6501 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 2031) took 3257 ms (client timeout 3525 ms). Trace:
I20260709 12:18:30.705927  6509 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:18:30.706935  6577 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:30.849105  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.849807  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.850554  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:30.851315  6655 raft_consensus.cc:1275] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 12:18:30.713289 10054 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.899s	user 0.012s	sys 0.012s
W20260709 12:18:30.851845 10054 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.038s	user 0.012s	sys 0.012s
I20260709 12:18:30.718828  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:30.853612  6655 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:18:30.855125  6655 raft_consensus.cc:1275] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:30.714504  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.735061 10285 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:30.738497  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.856683  6580 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:30.856938  6508 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
I20260709 12:18:30.742587 10267 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:30.752712  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:30.857988  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.859309  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.750247 10296 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:30.760011  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:30.761941  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:30.764971  6622 connection.cc:256] RPC call timeout handler was delayed by -2.0387077190002856s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1505s for a response
I20260709 12:18:30.768273  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:30.770143  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.772696  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:30.775374  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:30.776935  7270 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:18:30.780741  7273 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 8 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:30.782099  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:30.782893  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:18:30.848214  6501 rpcz_store.cc:269] 0709 12:18:27.448744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.458400 (+  9656us) service_pool.cc:224] Handling call
0709 12:18:30.705533 (+3247133us) spinlock_profiling.cc:243] Waited 2.46 s on lock 0x7b540017c0d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:30.705701 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5662877056}
I20260709 12:18:30.654606 10301 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:30.788017  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.759032  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:30.861464  6622 connection.cc:256] RPC call timeout handler was delayed by -1.9811077190001924s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1569s for a response
W20260709 12:18:30.888365  6622 connection.cc:256] RPC call timeout handler was delayed by -1.9757077190001837s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
W20260709 12:18:30.888446  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 55823.2us
W20260709 12:18:30.888695  6622 connection.cc:256] RPC call timeout handler was delayed by -1.8461077190002015s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17190000000000003s for a response
W20260709 12:18:30.888998  6622 connection.cc:256] RPC call timeout handler was delayed by -1.7381077190002543s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1839s for a response
W20260709 12:18:30.889261  6622 connection.cc:256] RPC call timeout handler was delayed by -1.654407719000119s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1932s for a response
I20260709 12:18:30.861541  6582 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
I20260709 12:18:30.890787  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.891394  6582 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
W20260709 12:18:30.889546  6622 connection.cc:256] RPC call timeout handler was delayed by -1.407807719000175s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22060000000000002s for a response
W20260709 12:18:30.892721  6622 connection.cc:256] RPC call timeout handler was delayed by -1.3988077190001604s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22160000000000002s for a response
I20260709 12:18:30.892591 10296 consensus_queue.cc:1059] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:18:30.893069  6622 connection.cc:256] RPC call timeout handler was delayed by -1.3988077190001604s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22160000000000002s for a response
W20260709 12:18:30.893368  6622 connection.cc:256] RPC call timeout handler was delayed by -1.3925077190001502s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2223s for a response
I20260709 12:18:30.893448 10296 consensus_queue.cc:1243] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer 20a202c76244485fbc8f1610473db8b9 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 12:18:30.893692  6622 connection.cc:256] RPC call timeout handler was delayed by -1.3925077190001502s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2223s for a response
I20260709 12:18:30.893898 10267 raft_consensus.cc:3055] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:30.892235  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.810966  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:30.862725  6583 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:18:30.895318  6504 raft_consensus.cc:2393] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 10: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:30.896196  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.812996  6475 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:30.674475  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.865393  7266 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:30.897369  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.654865  6503 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
W20260709 12:18:30.898093  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 85191.6us
I20260709 12:18:30.898835  6475 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:30.876947  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:18:30.829643  6502 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.584s	user 0.015s	sys 0.028s
I20260709 12:18:30.899674 10283 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 6. Current term is 7.
I20260709 12:18:30.819185  6507 raft_consensus.cc:2393] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 10: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:30.683755  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1814) took 1233 ms. Trace:
I20260709 12:18:30.878064  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.900312  6502 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 412) took 3713 ms. Trace:
I20260709 12:18:30.901571  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.882182  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.898057  6504 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:30.902416  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.851437  6577 raft_consensus.cc:2393] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 8: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:30.903121  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.803259  7269 consensus_queue.cc:237] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:30.863814  7271 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:30.903672  6504 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:30.900661  9574 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:18:30.894013  6622 connection.cc:256] RPC call timeout handler was delayed by -1.3925077190001502s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2223s for a response
W20260709 12:18:30.905846  6622 connection.cc:256] RPC call timeout handler was delayed by -1.3925077190001502s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2223s for a response
I20260709 12:18:30.906625  6622 leader_election.cc:400] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 11. Current term is 12.
I20260709 12:18:30.733346  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:30.878275  7295 debug-util.cc:398] Leaking SignalData structure 0x7b08002f8e40 after lost signal to thread 6533
I20260709 12:18:30.882301 10301 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:30.880376 10272 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:18:30.900548  9875 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:18:30.909675  7295 debug-util.cc:398] Leaking SignalData structure 0x7b08002f8e20 after lost signal to thread 6534
I20260709 12:18:30.909817  9875 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:30.910228  9875 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:30.901481 10314 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:30.897429  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.881117  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.911064  6505 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:18:30.899092  6582 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:30.894470 10267 raft_consensus.cc:740] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:30.917152  6582 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:18:30.917811  6582 raft_consensus.cc:606] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:30.917486 10267 consensus_queue.cc:260] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:30.918330  6582 raft_consensus.cc:614] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:30.918987 10267 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:30.919643  6582 raft_consensus.cc:1240] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:30.878438  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.862112 10273 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:30.903736  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.925561  7271 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:30.925745  6577 raft_consensus.cc:1813] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Handling vote request from an unknown peer f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:30.926548  6577 raft_consensus.cc:2410] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:18:30.896219  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:30.927109  6474 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:30.898846  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.865587 10054 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:30.900596  6584 rpcz_store.cc:276] 0709 12:18:29.450571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:29.515724 (+ 65153us) service_pool.cc:224] Handling call
0709 12:18:29.516148 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:29.680190 (+164042us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:29.680242 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:29.680303 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:29.680383 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.683464 (+1003081us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b540034dad4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:30.683522 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:30.683687 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":376605568}
I20260709 12:18:30.901289  6502 rpcz_store.cc:276] 0709 12:18:27.187270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:27.187438 (+   168us) service_pool.cc:224] Handling call
0709 12:18:27.201706 (+ 14268us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:27.202482 (+   776us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:27.202524 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:27.203069 (+   545us) log.cc:844] Serialized 26 byte log entry
0709 12:18:30.899979 (+3696910us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:30.900016 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:30.900088 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:30.900127 (+    39us) raft_consensus.cc:1685] finished
0709 12:18:30.900142 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:30.900259 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2013,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":17667,"thread_start_us":3653965,"threads_started":1,"wal-append.queue_time_us":421635,"wal-append.run_cpu_time_us":2268,"wal-append.run_wall_time_us":1002281}
I20260709 12:18:30.908771 10301 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:30.876916  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 94883.4us
I20260709 12:18:30.930965  7238 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:30.910645  6576 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:30.921429  6509 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
W20260709 12:18:30.922152  7295 debug-util.cc:398] Leaking SignalData structure 0x7b0800394020 after lost signal to thread 6608
W20260709 12:18:30.933276  7295 debug-util.cc:398] Leaking SignalData structure 0x7b0800372860 after lost signal to thread 6682
W20260709 12:18:30.934409  7295 debug-util.cc:398] Leaking SignalData structure 0x7b080033a800 after lost signal to thread 7298
I20260709 12:18:30.934667  6581 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:30.787106 10268 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:30.887873  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.948601  6579 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
W20260709 12:18:30.948989  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 59914.2us
I20260709 12:18:30.832069  6650 raft_consensus.cc:1275] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:18:30.860296  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:30.950299  6580 raft_consensus.cc:2393] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 10: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:30.925283 10273 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:30.903600  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:30.927310  6583 raft_consensus.cc:2393] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 10: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:18:30.934813  7295 debug-util.cc:398] Leaking SignalData structure 0x7b08000f3d60 after lost signal to thread 7299
I20260709 12:18:30.922852  6622 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:30.937971  6575 raft_consensus.cc:2468] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 6.
I20260709 12:18:30.940610 10253 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:30.940651 10285 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:30.940665  6653 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:18:30.922910  7266 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:30.935505 10272 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:30.946043  6584 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:18:30.982694  6584 raft_consensus.cc:1275] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:18:30.801275  6401 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46184 (request call id 116) took 19800 ms (client timeout 15000 ms). Trace:
I20260709 12:18:30.949824  6473 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:30.924113  6655 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:30.984624  6473 leader_election.cc:400] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10.
I20260709 12:18:30.984944  6473 leader_election.cc:403] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:18:30.985203  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:30.985587  6584 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:30.957118  7238 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:30.957944  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:30.975981  6548 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:30.652726  6550 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:30.987226  6548 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:30.987973 10242 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:30.988265  6622 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:30.990139  6622 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:30.997169  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:30.977336  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:31.001412  6654 raft_consensus.cc:2368] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10.
I20260709 12:18:31.005010  6576 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
W20260709 12:18:31.005251  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 48234.9us
I20260709 12:18:31.005674  6576 raft_consensus.cc:606] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 LEADER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:31.006384  6584 raft_consensus.cc:1275] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 8 index: 3. Preceding OpId from leader: term: 10 index: 4. (index mismatch)
W20260709 12:18:31.006903  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 354217us
I20260709 12:18:31.009765  6548 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:31.027248  6579 raft_consensus.cc:1240] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:18:31.029093  6579 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:18:30.971692 10245 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:31.011188  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:31.058243  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:30.964313 10253 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:31.063521  6580 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 9. Current term is 10. Ops: [7.6-9.7]
I20260709 12:18:31.012557  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:31.012933  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:31.049227  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:31.065744  6651 raft_consensus.cc:2376] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10. Re-sending same reply.
I20260709 12:18:30.883365 10306 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:31.066952  6583 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:31.053735  6548 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:31.006904 10242 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:31.059049  6657 raft_consensus.cc:2376] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10. Re-sending same reply.
I20260709 12:18:31.060842  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:31.012276  6550 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:31.003289 10304 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:31.065508  6649 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
W20260709 12:18:31.055321  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 48844.9us
I20260709 12:18:31.004638  9926 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:18:30.983863  6401 rpcz_store.cc:269] 0709 12:18:11.000644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:11.000872 (+   228us) service_pool.cc:224] Handling call
0709 12:18:30.801188 (+19800316us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:27.596451 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:27.641344 (+ 44893us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:27.641409 (+    65us) write_op.cc:435] Acquired schema lock
0709 12:18:27.641437 (+    28us) tablet.cc:662] Decoding operations
0709 12:18:27.642423 (+   986us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:27.642510 (+    87us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:27.642552 (+    42us) tablet.cc:685] Acquiring locks for 6 operations
0709 12:18:27.643320 (+   768us) tablet.cc:701] Row locks acquired
0709 12:18:27.643386 (+    66us) write_op.cc:260] PREPARE: finished
0709 12:18:27.643593 (+   207us) write_op.cc:270] Start()
0709 12:18:27.643756 (+   163us) write_op.cc:276] Timestamp: P: 1783599507643525 usec, L: 0
0709 12:18:27.643777 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:18:27.684762 (+ 40985us) log.cc:844] Serialized 2458 byte log entry
0709 12:18:27.749353 (+ 64591us) op_driver.cc:464] REPLICATION: finished
0709 12:18:27.757258 (+  7905us) write_op.cc:301] APPLY: starting
0709 12:18:27.757369 (+   111us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:27.757637 (+   268us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:27.757786 (+   149us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:27.758887 (+  1101us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:27.759250 (+   363us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 12:18:27.759283 (+    33us) write_op.cc:312] APPLY: finished
0709 12:18:27.759554 (+   271us) log.cc:844] Serialized 53 byte log entry
0709 12:18:27.759780 (+   226us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:27.759897 (+   117us) write_op.cc:454] Released schema lock
0709 12:18:27.759961 (+    64us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:27.760399 (+   438us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:27.760432 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4385768,"reported_tablets":18,"spinlock_wait_cycles":2432,"child_traces":[["op",{"apply.queue_time_us":7544,"apply.run_cpu_time_us":3107,"apply.run_wall_time_us":3113,"num_ops":6,"prepare.queue_time_us":162730,"prepare.run_cpu_time_us":18477,"prepare.run_wall_time_us":210807,"raft.queue_time_us":1506,"raft.run_cpu_time_us":6429,"raft.run_wall_time_us":7119,"replication_time_us":105332,"spinlock_wait_cycles":54144,"thread_start_us":149656,"threads_started":4,"wal-append.queue_time_us":43619,"wal-append.run_cpu_time_us":4163,"wal-append.run_wall_time_us":1016055}]]}
I20260709 12:18:31.016870 10298 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:30.971848  9993 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.091s
I20260709 12:18:31.030062 10268 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:31.080631  6548 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:31.024768 10296 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:18:31.071261  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 66939.1us
W20260709 12:18:31.086185  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 73974.5us
I20260709 12:18:31.048118 10279 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:31.095463  6622 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:31.095949  6550 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:31.090744  6655 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:18:31.092952  6651 raft_consensus.cc:1275] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:31.093276  6654 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:31.098132  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:31.098867  6651 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:31.100728  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:31.067323 10253 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:31.068163 10262 raft_consensus.cc:2804] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:30.996990 10285 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:31.104269  6507 raft_consensus.cc:1813] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Handling vote request from an unknown peer f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:31.113394  6507 raft_consensus.cc:2393] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 6: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:31.072263 10308 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10.
I20260709 12:18:31.088794 10297 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:31.078538 10304 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:31.046805 10075 consensus_queue.cc:1059] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 12:18:31.083560 10303 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:18:31.117681  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:31.118602  6548 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:31.107822  6649 raft_consensus.cc:1275] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 8 index: 3. Preceding OpId from leader: term: 10 index: 4. (index mismatch)
I20260709 12:18:31.121420  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:31.087888  9821 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:18:31.102115 10268 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:31.125970  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:31.103052 10296 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:31.102128 10279 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:31.125905  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 554) took 1155 ms. Trace:
I20260709 12:18:31.115218 10122 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:31.142134  6656 rpcz_store.cc:276] 0709 12:18:29.970137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.602445 (+632308us) service_pool.cc:224] Handling call
0709 12:18:30.602741 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:30.602955 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:18:30.687656 (+ 84701us) log.cc:844] Serialized 14 byte log entry
0709 12:18:31.112116 (+424460us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:31.120276 (+  8160us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:31.125600 (+  5324us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:31.125723 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:31.125848 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53392,"prepare.queue_time_us":85268,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":101881,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":7612,"thread_start_us":455713,"threads_started":3,"wal-append.queue_time_us":323001}
W20260709 12:18:31.154625  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 153073us
I20260709 12:18:31.158870  6651 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:31.128706 10304 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:31.197301  9920 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:31.133793 10298 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:31.139585 10268 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:31.130580 10297 raft_consensus.cc:697] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:31.220389  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 556) took 1249 ms. Trace:
I20260709 12:18:31.222380  6652 rpcz_store.cc:276] 0709 12:18:29.970650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.635021 (+664371us) service_pool.cc:224] Handling call
0709 12:18:30.635335 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:30.635541 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:18:31.007045 (+371504us) log.cc:844] Serialized 14 byte log entry
0709 12:18:31.220022 (+212977us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:31.220077 (+    55us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:31.220126 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:31.220187 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:31.220309 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":52811,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":26,"thread_start_us":568330,"threads_started":2}
W20260709 12:18:31.200642  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 104761us
W20260709 12:18:31.150876 10196 consensus_meta.cc:287] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.531s	user 0.008s	sys 0.004s
I20260709 12:18:31.230381  6500 raft_consensus.cc:2393] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 9: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:31.231518  6550 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:31.208060  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 201272us
I20260709 12:18:31.122706  9775 consensus_queue.cc:1048] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.078s
I20260709 12:18:31.150985 10296 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:31.130755 10330 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:31.151171 10279 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:31.153565 10122 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:31.215397  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 120169us
I20260709 12:18:31.204566 10303 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:31.131853  9875 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:31.234234  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 76520.5us
I20260709 12:18:31.238695  6622 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:31.215485 10298 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:31.239044  6508 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:31.255048  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:31.242035  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 257505us
I20260709 12:18:31.214735 10272 raft_consensus.cc:697] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:31.262670  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:31.264258  6473 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:31.265669  6622 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:31.272873  6550 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:31.273762  6622 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:31.240552 10122 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:31.237717 10330 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:31.297760  6473 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:31.228253 10295 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:31.264128 10298 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:31.321488  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 69249.7us
I20260709 12:18:31.271368 10262 raft_consensus.cc:697] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 6 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:31.275585  9993 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:31.271998 10075 raft_consensus.cc:2764] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:18:31.261667 10242 consensus_queue.cc:1059] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 12:18:31.260653 10284 consensus_queue.cc:1059] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 12:18:31.237819 10340 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.091s
I20260709 12:18:31.296737 10334 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:31.342844  6506 raft_consensus.cc:2368] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 8. Current term is 9.
I20260709 12:18:31.318243 10330 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:31.335394 10324 log.cc:927] Time spent T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.256s	user 0.000s	sys 0.001s
I20260709 12:18:31.325891 10303 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:31.341499 10334 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:31.348510 10334 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:31.348585 10262 consensus_queue.cc:237] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:31.331197  9993 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:31.352581  9993 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:31.333977 10283 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:31.338580  9574 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:31.361619  6549 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
W20260709 12:18:31.352748 10326 log.cc:927] Time spent T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 12:18:31.347050  9821 consensus_queue.cc:1048] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:18:31.346894 10306 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:31.353117 10297 consensus_queue.cc:237] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:31.346262 10272 consensus_queue.cc:237] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:31.337971 10349 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 12:18:31.344187 10342 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.125s
W20260709 12:18:31.388280 10323 log.cc:927] Time spent T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.237s	user 0.001s	sys 0.000s
W20260709 12:18:31.431155  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 158445us
I20260709 12:18:31.535056  6550 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:31.437429  6582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1822) took 1380 ms. Trace:
W20260709 12:18:31.485496  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 160361us
I20260709 12:18:31.486089  6508 raft_consensus.cc:1240] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:31.536928  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:31.537856  6508 raft_consensus.cc:2393] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 7: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:31.487949  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:18:31.433324  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 159637us
I20260709 12:18:31.489355  6503 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 12:18:31.531601  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 199711us
I20260709 12:18:31.376281 10285 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:31.541081  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:31.541854  6503 raft_consensus.cc:2368] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 8. Current term is 9.
I20260709 12:18:31.372934 10075 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:31.375061  9920 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:31.390460 10303 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:31.535956  6582 rpcz_store.cc:276] 0709 12:18:30.057359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.920594 (+863235us) service_pool.cc:224] Handling call
0709 12:18:30.920894 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:30.933378 (+ 12484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:30.933410 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:30.938055 (+  4645us) log.cc:844] Serialized 26 byte log entry
0709 12:18:31.230276 (+292221us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:18:31.230318 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:31.230391 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:31.437216 (+206825us) raft_consensus.cc:1685] finished
0709 12:18:31.437236 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:31.437370 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1171,"raft.queue_time_us":12160,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":11412,"thread_start_us":225729,"threads_started":1,"wal-append.queue_time_us":470721}
I20260709 12:18:31.491061  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:31.543816  6550 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:31.489324  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:31.490437  6505 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:18:31.532691  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:31.433564 10242 consensus_queue.cc:1243] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:18:31.374653  9875 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:31.385820  9775 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:31.556510  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:31.372794 10339 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:31.563481  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:31.567178 10267 consensus_meta.cc:287] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.648s	user 0.005s	sys 0.007s
W20260709 12:18:31.567847 10301 consensus_meta.cc:287] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.616s	user 0.010s	sys 0.001s
I20260709 12:18:31.570289  6509 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 11.
I20260709 12:18:31.449226 10304 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:31.571425  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:31.571554 10122 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:31.573189  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:31.572609  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:31.577020  6550 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:31.480518  9574 raft_consensus.cc:697] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:31.597687  6508 raft_consensus.cc:2368] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 8. Current term is 9.
W20260709 12:18:31.598417  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 105328us
I20260709 12:18:31.601151  6509 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:18:31.633452  6503 raft_consensus.cc:1240] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:31.533800 10283 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:31.636003  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 59120.2us
I20260709 12:18:31.582422 10285 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:31.452883 10253 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:31.622376  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:18:31.622687  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 71736.6us
W20260709 12:18:31.623627  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 62866.1us
I20260709 12:18:31.633977  6584 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:31.640236  6550 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:31.674347  6550 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:31.576077 10340 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.086s
W20260709 12:18:31.612064  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 493487us
I20260709 12:18:31.562219 10303 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:31.597807 10339 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:31.464893 10333 consensus_queue.cc:1059] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.157s
I20260709 12:18:31.675541  6550 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:31.622469 10301 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:31.644593 10286 consensus_meta.cc:287] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.806s	user 0.008s	sys 0.004s
I20260709 12:18:31.630614  9775 raft_consensus.cc:697] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:31.682407 10330 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:31.682449 10304 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:31.556877 10350 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.089s
I20260709 12:18:31.685329  6548 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:31.642266 10267 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:31.635591 10355 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:31.641979 10285 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:31.713627  6578 raft_consensus.cc:2393] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 7: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:18:31.676390 10087 consensus_meta.cc:287] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.664s	user 0.011s	sys 0.000s
I20260709 12:18:31.706614  7266 raft_consensus.cc:1240] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:31.715142  9574 consensus_queue.cc:237] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:31.717509  9993 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:31.697073 10336 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:31.719699 10122 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:31.719779  9993 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:31.705472 10286 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:31.721216 10242 consensus_queue.cc:1059] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:31.721642  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:31.722152 10242 raft_consensus.cc:3055] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:31.722499 10242 raft_consensus.cc:740] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:31.722586 10286 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:31.723270 10242 consensus_queue.cc:260] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:31.724102 10253 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:31.725302 10348 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:31.725492  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:31.726024  6507 raft_consensus.cc:1240] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:31.695775 10303 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:31.756489  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:31.757311  6582 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:31.758777  6650 raft_consensus.cc:2393] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 10: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:31.695986 10339 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:31.645309 10283 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:31.730062 10285 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:31.764814  6500 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 1977) took 1021 ms. Trace:
I20260709 12:18:31.701395 10268 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:31.802006 10268 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:31.804371 10330 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:31.704612 10296 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:31.809839 10296 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:31.736449 10087 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:31.811208  6657 raft_consensus.cc:2393] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 11: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:31.811972 10087 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:31.811671  9775 consensus_queue.cc:237] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:31.813433 10245 consensus_meta.cc:287] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.750s	user 0.010s	sys 0.000s
I20260709 12:18:31.813829 10245 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:31.815284  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:18:31.783198 10279 consensus_meta.cc:287] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.510s	user 0.007s	sys 0.003s
I20260709 12:18:31.794651  6658 raft_consensus.cc:1240] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:18:31.797420 10334 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:31.790716  6500 rpcz_store.cc:276] 0709 12:18:30.743363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.817150 (+ 73787us) service_pool.cc:224] Handling call
0709 12:18:31.764618 (+947468us) spinlock_profiling.cc:243] Waited 412 ms on lock 0x7b540001bdd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:31.764755 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":947690880}
I20260709 12:18:31.761351 10122 raft_consensus.cc:2804] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:31.736778 10358 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:18:31.823436  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 92358.2us
W20260709 12:18:31.833566  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 141032us
I20260709 12:18:31.789328 10314 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.055s
W20260709 12:18:31.841667  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 156335us
I20260709 12:18:31.843806 10334 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:31.868767  6582 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7.
I20260709 12:18:31.827521 10242 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:31.878729 10295 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:31.810580 10298 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:31.820139 10301 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:31.885524  6550 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:31.858036 10356 log.cc:927] Time spent T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:18:31.828645 10122 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:31.754171 10336 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.109s
I20260709 12:18:31.899480  6622 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:31.900714  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:31.907401  6649 raft_consensus.cc:2410] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:18:31.901695  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:31.902599  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:31.911561  6654 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:18:31.911909  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:31.913686  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:31.916072  6622 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:31.827266 10279 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:31.918347  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:31.921847  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:31.922864  6622 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:18:31.906061  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 272936us
I20260709 12:18:31.924649  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:31.857538 10350 consensus_queue.cc:1059] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 12:18:31.907930 10122 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:31.940896  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 255682us
I20260709 12:18:31.897047 10358 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:31.940802  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:31.934571  6475 leader_election.cc:400] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10.
I20260709 12:18:31.923552  9993 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:31.948551  9993 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:31.949018  9993 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:31.919961 10357 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:31.952697 10357 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:31.953343 10357 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:31.956115 10279 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:31.956893 10279 raft_consensus.cc:993] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e: : Instructing follower f2b06aebe97249a4877d798b8118ece4 to start an election
I20260709 12:18:31.923477 10330 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:31.957325 10279 raft_consensus.cc:1081] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 LEADER]: Signalling peer f2b06aebe97249a4877d798b8118ece4 to start an election
I20260709 12:18:31.958922  6548 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:31.961306  6622 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:31.960045  6548 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:31.965713  6654 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:31.968303  6548 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:31.970275  6548 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:31.971756  6548 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:31.943390  6576 raft_consensus.cc:1240] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:31.944829  6475 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:31.974169  6475 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:31.906482 10268 consensus_queue.cc:1048] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
W20260709 12:18:31.976363  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 153940us
I20260709 12:18:31.943305 10284 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:31.957588 10330 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:31.980515  6548 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:31.976015  6622 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:32.001662  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:31.980372  6475 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:31.980891 10353 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:31.981093  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:18:31.981769 10368 log.cc:927] Time spent T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:18:31.972805  6649 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:58980
I20260709 12:18:31.973074 10301 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:31.984817 10295 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:31.986052  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1863) took 1234 ms. Trace:
I20260709 12:18:32.014246  7238 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:18:31.962726 10297 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:31.957290 10369 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 12:18:32.021181  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:32.054015  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:31.968533 10279 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:31.969952 10334 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:32.019879  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:18:32.023001 10087 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:18:32.034819  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 54529.7us
I20260709 12:18:31.973990 10336 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:18:32.042618  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 345261us
I20260709 12:18:31.972573 10267 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:32.021075  6577 raft_consensus.cc:1275] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:18:32.012993  9926 raft_consensus.cc:3055] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:32.012854 10355 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:32.013590 10330 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:32.059810  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:32.015895  9574 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:32.017174 10348 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:32.029266  6578 rpcz_store.cc:276] 0709 12:18:30.751834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.838097 (+ 86263us) service_pool.cc:224] Handling call
0709 12:18:31.985851 (+1147754us) spinlock_profiling.cc:243] Waited 874 ms on lock 0x7b54000573d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:31.985992 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2010872064}
I20260709 12:18:32.017330 10353 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:32.024683 10362 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:32.063205  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:32.022819 10268 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:32.070245  6650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 1787) took 1097 ms. Trace:
W20260709 12:18:32.036708 10273 consensus_meta.cc:287] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 1.073s	user 0.012s	sys 0.000s
I20260709 12:18:32.070971  6656 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:31.957640  9993 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.100715  6577 raft_consensus.cc:1240] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:32.038409 10314 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:32.103683  6649 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:32.109643  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:32.069710 10284 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:32.145470  6649 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:32.129580  6473 connection.cc:256] RPC call timeout handler was delayed by -1.3254412520000187s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 12:18:32.143580  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 108199us
I20260709 12:18:32.110680  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:32.150801  6582 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:32.113966  6650 rpcz_store.cc:276] 0709 12:18:30.972671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.975449 (+  2778us) service_pool.cc:224] Handling call
0709 12:18:31.125824 (+150375us) spinlock_profiling.cc:243] Waited 2.02 ms on lock 0x7b5800015998. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:18:32.069920 (+944096us) spinlock_profiling.cc:243] Waited 597 ms on lock 0x7b540019d7d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:32.070162 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1376611456}
I20260709 12:18:32.155257  6581 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:32.157389 10381 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:32.158013 10381 raft_consensus.cc:697] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 10 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:32.158982 10381 consensus_queue.cc:237] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.077189  9926 raft_consensus.cc:740] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:32.075665 10087 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:18:32.209549  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 100067us
W20260709 12:18:32.210788  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 58734.1us
I20260709 12:18:32.075129 10371 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:32.101682 10348 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:32.105769 10362 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:32.213894  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:32.217151  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:32.217382 10352 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:32.216964  9926 consensus_queue.cc:260] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 3, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.148680 10284 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:32.102958 10283 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:32.221625  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 185787us
I20260709 12:18:32.241904  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:32.112314 10353 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:32.226220  6582 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:18:32.227685  6658 raft_consensus.cc:2393] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 9: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:32.146210 10334 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:32.228463  6575 raft_consensus.cc:1275] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 8 index: 3. Preceding OpId from leader: term: 10 index: 4. (index mismatch)
I20260709 12:18:32.221105  6475 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:32.097728 10295 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.112s
W20260709 12:18:32.233378  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 173668us
I20260709 12:18:32.251415  6650 raft_consensus.cc:1275] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:18:32.251497  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:32.106210  9775 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:18:32.218266 10370 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 12:18:32.255193  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:32.226785  6649 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.148674 10273 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.104750 10267 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.220445 10087 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:32.219009 10328 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:32.210490 10314 raft_consensus.cc:697] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:32.148881 10268 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.250130 10284 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:32.257824  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:32.233273 10362 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:32.230563 10348 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:32.222970  6583 raft_consensus.cc:1275] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:32.300835  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:32.301211  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:32.265586  6548 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:32.271915 10386 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:32.272414  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:32.272923 10354 raft_consensus.cc:3055] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:32.313154 10354 raft_consensus.cc:740] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:32.313910 10273 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:32.314254 10354 consensus_queue.cc:260] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:32.315490 10354 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:32.275012 10245 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:32.316035 10369 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:32.316689 10245 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.317469 10388 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:32.317859 10267 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:32.276172  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:32.282305  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:32.285758 10285 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.232908 10371 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:32.287067 10375 log.cc:927] Time spent T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.236s	user 0.003s	sys 0.000s
I20260709 12:18:32.257131  6650 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:32.105886 10366 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.156s
I20260709 12:18:32.289434  6473 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:32.231026 10306 consensus_queue.cc:1048] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 12:18:32.248744 10352 raft_consensus.cc:697] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:32.253057 10334 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:32.237774 10054 consensus_queue.cc:1048] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:18:32.303701  6548 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:32.299971  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:32.305900  6576 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:32.306696  6508 raft_consensus.cc:1275] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:18:32.307865  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:32.346215  6548 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:32.346834  6500 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:32.310036  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:32.348325  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:32.348172  6548 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:32.322129  6507 raft_consensus.cc:1240] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:18:32.328038  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:18:32.330120  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 78747.6us
I20260709 12:18:32.339686  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:32.278846  9926 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:32.349777  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 60421us
I20260709 12:18:32.350356  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:32.355298  6548 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:32.358170  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:32.359179  6583 raft_consensus.cc:2410] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:18:32.358520  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:32.359706  6654 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:18:32.270273  9775 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.328681 10268 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:32.361842  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:32.328065 10328 raft_consensus.cc:697] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:32.356446  6473 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:32.399887  6473 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:32.399793 10328 consensus_queue.cc:237] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:32.401490 10258 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:32.401732 10306 consensus_queue.cc:1048] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:32.368884 10392 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:32.363279 10245 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:32.373031 10122 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:32.373951 10285 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:32.374423 10273 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:32.376701 10087 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.377887 10391 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:32.386180 10371 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:32.390859 10339 consensus_meta.cc:287] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.568s	user 0.007s	sys 0.002s
I20260709 12:18:32.306953 10295 consensus_queue.cc:1059] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 12:18:32.459614 10295 raft_consensus.cc:3055] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:32.459986 10295 raft_consensus.cc:740] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:32.460697 10122 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:32.460834 10295 consensus_queue.cc:260] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.462041 10295 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:32.392829 10377 log.cc:927] Time spent T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 12:18:32.394755  9993 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:32.364091 10369 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:32.395517 10335 raft_consensus.cc:3055] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:32.339213 10386 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:32.466598 10335 raft_consensus.cc:740] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:32.466780 10386 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:32.467577 10335 consensus_queue.cc:260] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.468928 10335 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:32.469193 10386 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:32.475169 10339 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:32.476387  6653 raft_consensus.cc:1240] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:32.437215 10245 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:18:32.478338 10339 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:32.478673  9775 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:32.480566 10330 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:32.481878 10242 consensus_meta.cc:287] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.590s	user 0.010s	sys 0.000s
I20260709 12:18:32.486023 10358 consensus_queue.cc:1048] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:32.437021 10268 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.487998 10268 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:32.490074  6655 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7.
W20260709 12:18:32.491915  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:32.494920 10253 raft_consensus.cc:3055] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:32.495329 10253 raft_consensus.cc:740] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:32.495955 10348 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:32.496127 10253 consensus_queue.cc:260] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:32.496682 10268 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:32.431828 10075 consensus_meta.cc:287] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.874s	user 0.005s	sys 0.006s
I20260709 12:18:32.497222 10253 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:32.497696 10075 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:32.498405 10348 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:32.400225 10255 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:32.401486  6473 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:32.499178 10255 raft_consensus.cc:697] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:18:32.499547  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 98126.1us
I20260709 12:18:32.500725  6473 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:32.501544  9875 consensus_queue.cc:1048] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:32.502988  9875 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:18:32.503306  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:32.499540 10306 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:18:32.370110 10379 log.cc:927] Time spent T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:18:32.506899  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 426) took 1015 ms. Trace:
I20260709 12:18:32.409305  6654 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:32.507232  6501 rpcz_store.cc:276] 0709 12:18:31.491104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:31.574234 (+ 83130us) service_pool.cc:224] Handling call
0709 12:18:31.574542 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:31.574863 (+   321us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:18:31.743362 (+168499us) log.cc:844] Serialized 14 byte log entry
0709 12:18:32.506404 (+763042us) spinlock_profiling.cc:243] Waited 9.98 ms on lock 0x7b18000e35c0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2875a314bc 00007f2872d7f48b 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a2923a
0709 12:18:32.506576 (+   172us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:32.506621 (+    45us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:32.506675 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:32.506738 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:32.506846 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":283451,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":1176,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":87,"spinlock_wait_cycles":22956288,"thread_start_us":241137,"threads_started":2,"wal-append.queue_time_us":72387}
W20260709 12:18:32.507390 10357 consensus_meta.cc:287] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.553s	user 0.009s	sys 0.001s
I20260709 12:18:32.510195  6575 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:32.510098 10357 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:32.511121  6500 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:32.511353  6575 raft_consensus.cc:1240] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:32.364836  6650 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:32.416531  6576 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:32.343729 10334 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:32.434266  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 71753.1us
I20260709 12:18:32.530867  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:32.531764  6548 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:32.531888  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:32.360572 10336 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:32.533129 10296 consensus_queue.cc:1059] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:32.533749 10336 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:32.533932 10296 raft_consensus.cc:3055] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:32.362445 10267 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:32.534480 10298 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:32.418179 10314 consensus_queue.cc:237] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.446004 10087 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:32.535211  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:32.534324 10336 raft_consensus.cc:697] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:32.534410 10296 raft_consensus.cc:740] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:32.536325 10336 consensus_queue.cc:237] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.534209  6575 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:32.448477 10332 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:32.397039 10352 consensus_queue.cc:237] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:32.545584  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 177145us
I20260709 12:18:32.398348 10384 consensus_queue.cc:1059] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:32.521263  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 458) took 1780 ms. Trace:
I20260709 12:18:32.547705  6584 rpcz_store.cc:276] 0709 12:18:30.740773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:31.013157 (+272384us) service_pool.cc:224] Handling call
0709 12:18:31.013402 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:31.685485 (+672083us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:32.521040 (+835555us) spinlock_profiling.cc:243] Waited 590 ms on lock 0x7b54001cd254. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:32.521204 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1356959616}
I20260709 12:18:32.521672 10330 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:32.513937 10357 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:32.522845 10388 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:32.523731 10348 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:32.523919 10075 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:32.508183  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:32.527033  6475 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:32.500099 10255 consensus_queue.cc:237] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.518409  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:18:32.441077 10303 consensus_meta.cc:287] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.677s	user 0.010s	sys 0.000s
I20260709 12:18:32.446527 10391 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.565650 10075 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.567420 10391 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:32.567812  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:32.567905 10075 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:32.568797 10397 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:32.568626  6501 raft_consensus.cc:2393] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 7: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:18:32.551222  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 197600us
I20260709 12:18:32.562983  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:32.568229  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:32.572566  6654 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 13: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:32.564007  6506 raft_consensus.cc:2393] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 6: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:32.564505  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:32.568348 10306 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:32.550150  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:32.573606  6658 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:32.568910  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:32.568967 10303 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.568499  6650 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:32.568691  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:32.570488 10397 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:32.537079 10296 consensus_queue.cc:260] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.579551 10306 raft_consensus.cc:697] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:32.580005  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:32.616581  6654 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:18:32.618369  6658 raft_consensus.cc:2410] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:18:32.587297 10334 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.589149  6622 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:32.620054  6622 leader_election.cc:400] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 8. Current term is 9.
I20260709 12:18:32.620437  6622 leader_election.cc:403] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:32.621556  6622 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:32.590797 10396 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:32.592284 10332 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.553293 10284 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.594272 10268 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:18:32.596261  6657 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 1799) took 1523 ms (client timeout 1769 ms). Trace:
W20260709 12:18:32.652971  6657 rpcz_store.cc:269] 0709 12:18:31.073257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:31.092255 (+ 18998us) service_pool.cc:224] Handling call
0709 12:18:32.596069 (+1503814us) spinlock_profiling.cc:243] Waited 709 ms on lock 0x7b540010ed54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:32.596203 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1631350528}
I20260709 12:18:32.598166  6578 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:32.605665  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:32.627166  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:32.628198  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:32.656242  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:32.657152  6584 raft_consensus.cc:2393] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 10: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:32.657912  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:32.658535  6584 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 13: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:32.659214  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:32.628515  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:32.631290  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:32.632265  6504 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:32.632779  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:32.640825  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 51288.2us
I20260709 12:18:32.642518  6501 raft_consensus.cc:1240] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 8. Current term is 10. Ops: [8.6-8.6]
I20260709 12:18:32.571807 10298 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:32.661129  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:32.668243  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:32.668968  6581 raft_consensus.cc:2393] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 11: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:32.669122  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:32.669833  6505 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:32.662467  6575 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:32.661971  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:32.667323  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:32.671648  6509 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:32.663192  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:32.674093  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:32.674582  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:32.553793 10087 consensus_queue.cc:1059] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.086s
I20260709 12:18:32.676478  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:32.667912  6502 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:18:32.661900  6503 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:32.660308  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:32.679873  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:32.681469  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:32.683102  6584 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 14: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:32.684459  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:32.685228  6584 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:18:32.640048 10075 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 8. Current term is 9.
I20260709 12:18:32.651928 10391 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:32.652505 10335 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:32.699122  6622 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:32.651846 10297 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.704847  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:32.709373  6500 raft_consensus.cc:1240] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:32.710791  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:32.573835 10348 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:32.561993 10353 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.689355 10242 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:32.666966 10381 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:32.713176  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:32.668140 10392 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:32.716776  6622 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:32.718286  6575 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:32.691890 10296 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:32.686156 10298 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.736301 10075 consensus_queue.cc:1059] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:32.736620 10343 consensus_queue.cc:1059] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 12:18:32.737232 10075 consensus_queue.cc:1243] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:18:32.738044 10343 raft_consensus.cc:3055] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:32.738474 10343 raft_consensus.cc:740] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:32.738211 10075 raft_consensus.cc:3055] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:32.739185 10075 raft_consensus.cc:740] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:32.739490 10343 consensus_queue.cc:260] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:32.669742 10362 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.741648  6501 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:18:32.742151 10362 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:32.742729 10330 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:32.715253 10391 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:32.693650 10339 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.663759 10355 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.731009  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:32.759567 10339 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.759864  6581 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:32.759732 10355 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.760905  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:32.761646  6581 raft_consensus.cc:2393] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 12: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:32.761744 10339 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:32.762395  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:32.762509 10355 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:32.730509 10242 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.763073  6581 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:32.764681  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:32.765240 10397 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:32.765928  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:32.769426 10353 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:32.729946 10306 consensus_queue.cc:237] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.723776 10386 consensus_queue.cc:1059] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:18:32.668114 10332 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:32.773001  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:32.773739 10332 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:32.774132  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:32.775319 10242 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:32.762445 10330 raft_consensus.cc:697] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 10 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:32.776109  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:32.776732  6657 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:32.777020 10330 consensus_queue.cc:237] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.733673  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:32.749629 10391 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:32.745939  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:32.781837  6584 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:32.778644  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:32.746433 10303 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:32.719725  9926 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:32.749107  6658 raft_consensus.cc:1240] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:32.675635 10268 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:32.733346 10383 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.751920 10284 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:32.726823 10297 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.753927 10334 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:32.855916 10335 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:18:32.858831  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:32.861407  6473 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:32.863232 10315 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:32.754573 10348 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:32.743280 10381 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.743598 10392 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.772009 10386 consensus_queue.cc:1243] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:18:32.783553  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:32.783911  6575 raft_consensus.cc:2393] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 7: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:32.784849  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:32.789283  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:32.722844  6509 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:18:32.797533  6578 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:32.806681  6505 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:18:32.807657  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49792 (request call id 571) took 1022 ms. Trace:
I20260709 12:18:32.888234  6473 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:32.811569  6651 raft_consensus.cc:1275] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:18:32.814574  7271 raft_consensus.cc:1240] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 6 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:32.888621  6584 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:18:32.816799  6550 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:32.889689  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:32.889719  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:32.890549  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 73875.6us
I20260709 12:18:32.819525  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:32.834950 10383 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.836539  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:32.837124  7270 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:18:32.837800  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:32.837901  7266 raft_consensus.cc:1240] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:32.838573  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:32.839315  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:32.894905  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:32.842365  6581 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:32.895581  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:32.842860 10298 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:32.838447  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:32.854058 10297 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:32.754375 10362 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 12:18:32.869088 10268 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:32.869234  6658 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 12:18:32.909720  6658 pending_rounds.cc:85] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 6
I20260709 12:18:32.910092  6658 pending_rounds.cc:107] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 12:18:32.910405  6658 raft_consensus.cc:2889] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:32.869767  6582 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:32.870906 10301 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:32.871917  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 298975us
I20260709 12:18:32.828291 10343 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:32.879186 10396 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:32.880119  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:32.920888  6652 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 14.
I20260709 12:18:32.880854  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:32.922029  6656 raft_consensus.cc:2393] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 10: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:32.808568  6657 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:32.923887  6657 raft_consensus.cc:1240] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:32.924933  6657 raft_consensus.cc:1240] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:32.888476  6655 rpcz_store.cc:276] 0709 12:18:31.785321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:31.785725 (+   404us) service_pool.cc:224] Handling call
0709 12:18:32.807390 (+1021665us) spinlock_profiling.cc:243] Waited 667 ms on lock 0x7b54000b7554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:32.807543 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1533082240}
I20260709 12:18:32.893553  6550 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:32.895618  7274 raft_consensus.cc:2410] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:18:32.897661  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:32.899082  6581 raft_consensus.cc:1240] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:32.928033  6582 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:18:32.915337  7267 raft_consensus.cc:1275] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 12:18:32.928942  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:32.904979  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:32.929522 10315 consensus_queue.cc:1059] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:32.931056  6656 raft_consensus.cc:1240] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:32.926268 10301 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:32.932416  6656 raft_consensus.cc:1240] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:32.908792 10296 consensus_queue.cc:1048] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:18:32.933862 10301 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:32.934993  6657 raft_consensus.cc:1275] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:18:32.936564  6656 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:32.938580  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:32.931259 10258 raft_consensus.cc:3055] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:32.940004 10335 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:32.940057 10258 raft_consensus.cc:740] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:18:32.940398  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 46947.6us
I20260709 12:18:32.940745 10335 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:32.941529 10303 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:32.941229  6550 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:32.943142  6550 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:32.944574 10297 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:32.945093 10297 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:32.945616 10297 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:32.948594 10298 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:32.949218 10395 consensus_queue.cc:1059] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:32.949790  6576 raft_consensus.cc:1275] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:18:32.906790 10355 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:32.928280  6581 raft_consensus.cc:1240] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:32.951069  6550 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:32.891456 10359 consensus_queue.cc:1059] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:18:32.908542 10332 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:18:32.903321 10333 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.138s	user 0.017s	sys 0.049s
I20260709 12:18:32.917012  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:32.965016  6654 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:32.966742  6656 raft_consensus.cc:3055] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:32.967134  6656 raft_consensus.cc:740] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:32.966691  6584 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
I20260709 12:18:32.908732 10362 consensus_queue.cc:1243] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:18:32.970075  6576 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:32.972504  6581 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:18:32.967998  6656 consensus_queue.cc:260] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.975147  6656 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:32.941234 10258 consensus_queue.cc:260] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:32.983811 10332 raft_consensus.cc:697] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:32.985092 10332 consensus_queue.cc:237] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:32.990316 10346 consensus_queue.cc:1059] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.099s
I20260709 12:18:32.972935 10369 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:32.996084 10350 consensus_queue.cc:1059] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:32.963800 10301 consensus_queue.cc:1059] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:18:32.997452 10397 raft_consensus.cc:3055] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:32.997896 10397 raft_consensus.cc:740] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:33.005781  6508 raft_consensus.cc:1275] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:18:33.005887  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:33.006974  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:32.965934 10333 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.230s	user 0.017s	sys 0.050s
I20260709 12:18:33.019526  6622 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:33.024670  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:33.027971  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:33.029094  6502 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:33.029139  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 121038us
I20260709 12:18:33.030021  6473 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:32.936826 10268 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 12:18:32.963946 10352 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 12:18:33.032307  6501 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:33.042049  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:33.043140  6508 raft_consensus.cc:2393] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 8: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:33.017364 10369 raft_consensus.cc:697] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:33.033633  6622 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:18:33.045357  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 71357.6us
I20260709 12:18:33.032601 10334 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:33.084450  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 50922us
I20260709 12:18:33.033008 10284 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:33.052250  6584 raft_consensus.cc:1240] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:18:33.094285  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:33.095603  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:33.051893  6502 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:18:33.039928 10355 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.031054  9920 consensus_queue.cc:1048] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.059s
I20260709 12:18:33.100937  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:33.115144  7238 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:33.122581  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:33.039122 10333 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:33.040544 10298 raft_consensus.cc:3055] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:33.133059  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:33.133762  7238 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:33.138047  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:33.085865  9926 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:33.137261  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:33.140477  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:33.143006  6650 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:33.150282  6650 raft_consensus.cc:1275] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:18:33.039543 10391 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:33.144933  6548 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:33.154383  6548 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:33.155032 10391 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:33.156610 10348 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:33.156710  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:33.157738 10348 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:33.158650 10348 consensus_queue.cc:1059] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:33.086496 10283 consensus_queue.cc:1059] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 12:18:33.164691 10411 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:33.165467  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:33.166229 10303 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:33.166682  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:33.167802  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:33.157884  6508 raft_consensus.cc:2393] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 12: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:33.169282 10343 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:33.169739  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:33.170302  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.170863  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:33.171605 10333 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:33.105283 10350 consensus_queue.cc:1059] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:18:33.156325 10369 consensus_queue.cc:237] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:33.172322  6508 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:18:33.173794  6650 raft_consensus.cc:1240] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:18:33.124913 10301 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.177271  6548 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:33.186103  6502 raft_consensus.cc:1240] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:33.186395  6548 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:33.187971  6502 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:18:33.188910  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 51780us
I20260709 12:18:33.189997  6622 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:33.192397  7238 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:33.096522 10406 consensus_queue.cc:1059] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:18:33.051019 10267 consensus_queue.cc:1048] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.091s
I20260709 12:18:33.133292 10298 raft_consensus.cc:740] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:33.051142 10346 consensus_queue.cc:1048] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.095s
I20260709 12:18:33.195914  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:33.135286 10395 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:33.194051 10296 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:33.143486 10392 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:33.209054  6649 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:33.208858 10298 consensus_queue.cc:260] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:33.209669  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:58980 (request call id 2318) took 1249 ms. Trace:
I20260709 12:18:33.210057  6649 rpcz_store.cc:276] 0709 12:18:31.964420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:31.964540 (+   120us) service_pool.cc:224] Handling call
0709 12:18:33.207915 (+1243375us) spinlock_profiling.cc:243] Waited 84.9 ms on lock 0x7b54003ea3d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e80a3d 00007f287ae50cd0 00007f28743d5254 00007f28743d51e2 00007f28743d5111 00007f28743d508c 00007f28743d4342 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:33.209552 (+  1637us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":195216640}
I20260709 12:18:33.129936 10352 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 12:18:33.143967 10396 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:33.206974  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:33.191514 10303 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:18:33.208135 10409 log.cc:927] Time spent T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:18:33.208046 10350 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:33.196406  6584 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:33.172142 10333 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:33.133661 10355 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:33.194761 10391 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:18:33.250816  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 60911.5us
I20260709 12:18:33.278436  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:33.200168 10384 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:33.263329  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1842) took 1155 ms. Trace:
I20260709 12:18:33.264179  6652 raft_consensus.cc:1240] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 6. Current term is 7. Ops: []
W20260709 12:18:33.338833  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 60530.6us
I20260709 12:18:33.094782 10397 consensus_queue.cc:260] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:33.264652  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 1855) took 1130 ms. Trace:
I20260709 12:18:33.344156  6578 rpcz_store.cc:276] 0709 12:18:32.133811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:32.257605 (+123794us) service_pool.cc:224] Handling call
0709 12:18:33.264446 (+1006841us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b540034c6d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:33.264563 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":696222976}
I20260709 12:18:33.346088  6578 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 12:18:33.266424  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 113780us
I20260709 12:18:33.338356  6580 rpcz_store.cc:276] 0709 12:18:32.108164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:32.109838 (+  1674us) service_pool.cc:224] Handling call
0709 12:18:32.110302 (+   464us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:32.305822 (+195520us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:32.305860 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:32.305928 (+    68us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:32.305977 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:33.263080 (+957103us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b540032b254. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:33.263132 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:33.263271 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":449152768}
I20260709 12:18:33.340022  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:33.233841  9926 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:33.350108  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:33.351578  6578 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:18:33.349666  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:33.355655  6584 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 14.
I20260709 12:18:33.357251  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:33.197751 10301 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:33.359752  6622 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:33.341734 10395 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:33.368897 10330 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:33.360441 10284 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:33.371500  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:33.388316  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:33.388737  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.372967 10342 consensus_queue.cc:1048] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:18:33.379302  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.360809 10392 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:33.345185 10396 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:33.372241 10391 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:18:33.348248 10303 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:33.416854  6578 raft_consensus.cc:1240] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:33.390064  9926 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:18:33.390326  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.359548 10332 raft_consensus.cc:3055] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:33.380324  6656 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:18:33.346498 10343 consensus_queue.cc:1059] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:18:33.393401 10330 raft_consensus.cc:697] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 11 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:33.409701 10284 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:33.410125 10392 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:33.431515  6548 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:33.433930  6548 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:33.439388  6548 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:33.439795 10301 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:33.440460 10301 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:33.440667  6548 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:33.441200 10301 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:33.441419 10296 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:33.443192 10268 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:33.420764  9926 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:33.444615  9926 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:33.444974  6548 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:33.446382 10358 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:33.447238 10355 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:33.448153 10330 consensus_queue.cc:237] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:33.448758 10355 consensus_queue.cc:1059] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:33.449929 10358 raft_consensus.cc:3055] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:33.450295 10358 raft_consensus.cc:740] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:33.451114 10358 consensus_queue.cc:260] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:33.451921  7265 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:18:33.420732 10332 raft_consensus.cc:740] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:18:33.453462 10297 consensus_meta.cc:287] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.507s	user 0.008s	sys 0.005s
I20260709 12:18:33.421380 10303 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:33.453666  9926 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:33.453918 10297 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:33.455173 10358 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:33.455435 10303 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:33.456022 10297 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:33.456768 10333 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:33.458950 10303 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:33.461140 10355 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:33.461598  9926 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:33.464742 10329 raft_consensus.cc:3055] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:18:33.464989 10412 log.cc:927] Time spent T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.300s	user 0.001s	sys 0.002s
I20260709 12:18:33.468812  6656 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
W20260709 12:18:33.473007  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.474803  6504 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:33.476074  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:33.438139 10392 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:33.471764 10283 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.482853  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:33.483099  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.482934  7267 raft_consensus.cc:1275] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 12:18:33.464243 10255 raft_consensus.cc:3055] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:33.487884  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:33.493273  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:33.480273  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:33.494813  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:33.487604  6502 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:33.492025  7267 pending_rounds.cc:85] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9: Aborting all ops after (but not including) 7
I20260709 12:18:33.496325  6508 raft_consensus.cc:1240] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:18:33.500355  7267 pending_rounds.cc:107] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 12:18:33.502959  7267 raft_consensus.cc:2889] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:33.504125  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 115933us
I20260709 12:18:33.505098  6622 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:33.481060 10329 raft_consensus.cc:740] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:33.506672  6622 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:33.495303 10423 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:33.509073  6548 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:18:33.503350 10380 log.cc:927] Time spent T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.604s	user 0.004s	sys 0.000s
I20260709 12:18:33.517716  6622 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:33.527065  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:33.500427 10255 raft_consensus.cc:740] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:33.513342 10355 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:33.509459 10396 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:33.535727  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:33.545818  6575 raft_consensus.cc:1240] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:18:33.501322 10283 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:33.550881  6579 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:33.512501 10395 consensus_queue.cc:1059] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:18:33.553241 10395 raft_consensus.cc:3055] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:33.529497  9926 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:33.553704 10395 raft_consensus.cc:740] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:33.554077 10283 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:33.554747 10395 consensus_queue.cc:260] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:33.490921 10332 consensus_queue.cc:260] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:33.573134 10329 consensus_queue.cc:260] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:33.525372 10284 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:33.574882 10329 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:33.561434  9926 raft_consensus.cc:697] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 5 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:33.561887 10396 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.575775 10396 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:33.576232  9926 consensus_queue.cc:237] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:33.578276 10396 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:33.580199 10392 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:33.582027 10304 raft_consensus.cc:3055] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:33.582497 10304 raft_consensus.cc:740] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:33.564842 10255 consensus_queue.cc:260] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:33.511435 10342 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:18:33.556471  6504 raft_consensus.cc:2376] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 11. Re-sending same reply.
I20260709 12:18:33.567214  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:33.509526 10424 consensus_queue.cc:1059] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:18:33.569195 10334 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:33.517940 10343 consensus_queue.cc:1059] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:18:33.570739  6579 raft_consensus.cc:1240] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:33.568027  6576 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:33.560806 10245 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:33.543953 10355 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:33.545264 10391 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:33.590384  6578 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:33.584504 10395 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:33.582818 10392 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:33.596257  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 53719.5us
I20260709 12:18:33.522105 10423 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:33.600642  6576 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:33.594235 10304 consensus_queue.cc:260] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:33.601486  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 44223.2us
I20260709 12:18:33.592995 10255 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:33.593871 10245 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:33.602784  6622 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:33.604792 10441 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:33.610234 10423 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:33.606923  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:33.607913 10245 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:33.597414 10355 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:33.634413 10383 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:33.621136  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:33.621927  6506 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 2075) took 1320 ms (client timeout 1612 ms). Trace:
I20260709 12:18:33.602018 10391 raft_consensus.cc:697] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 10 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:33.626566 10334 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:33.638532  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:33.628304 10346 raft_consensus.cc:3055] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:33.630985 10381 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:33.637455  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:33.637295  6506 rpcz_store.cc:269] 0709 12:18:32.301387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:32.305602 (+  4215us) service_pool.cc:224] Handling call
0709 12:18:33.621686 (+1316084us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b54000ff354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:33.621863 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":491634432}
I20260709 12:18:33.640405  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: true dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:33.621649 10304 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:33.641677 10391 consensus_queue.cc:237] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:33.622475 10392 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:18:33.649010 10354 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 12:18:33.646934  6507 raft_consensus.cc:2393] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 8: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:33.640375 10346 raft_consensus.cc:740] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:33.658313  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:33.668282  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:33.669190  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:33.678014 10397 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:33.667951 10392 raft_consensus.cc:697] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:33.681463 10392 consensus_queue.cc:237] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:33.682215  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.658313 10383 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.685695 10383 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:33.688644 10346 consensus_queue.cc:260] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:33.690035 10346 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:33.691715  6502 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
I20260709 12:18:33.698704  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:33.702436  6649 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:33.702966  7268 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:18:33.704787  6580 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:18:33.699527  6657 raft_consensus.cc:1240] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:18:33.706849  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:33.698215  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:33.707434  6654 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:33.707561  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:33.699309 10385 raft_consensus.cc:3055] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:33.711737  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:33.712576 10385 raft_consensus.cc:740] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:33.712842  7268 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:33.713526 10385 consensus_queue.cc:260] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:33.716032  6580 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:33.716964 10286 consensus_queue.cc:1048] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:33.717758  6578 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:33.719329  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:33.720705  6506 raft_consensus.cc:1240] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:18:33.722362  6503 pending_rounds.cc:85] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 5
I20260709 12:18:33.727456  6503 pending_rounds.cc:107] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:18:33.727779  6503 raft_consensus.cc:2889] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:33.699707 10308 raft_consensus.cc:3055] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:33.732443  6652 raft_consensus.cc:1275] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:18:33.696866 10383 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:33.730551  6654 raft_consensus.cc:1240] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 5 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:33.733886  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:33.743487  6506 raft_consensus.cc:1275] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:18:33.745563  6654 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 11 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:33.764369  6652 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:33.766359  6474 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:33.767578  6474 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:18:33.730602 10410 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.007s	sys 0.011s
I20260709 12:18:33.739746 10385 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:33.775480  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:33.777055  6652 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:18:33.780885  6582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1883) took 1128 ms. Trace:
I20260709 12:18:33.781524  6582 rpcz_store.cc:276] 0709 12:18:32.652585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:32.676173 (+ 23588us) service_pool.cc:224] Handling call
0709 12:18:33.780611 (+1104438us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b540024d454. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:33.780792 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":439284096}
I20260709 12:18:33.784677  6584 raft_consensus.cc:1240] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:18:33.793730 10354 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:33.793987  6584 raft_consensus.cc:1240] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:33.794371 10354 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:33.796804 10354 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:33.797645  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:33.805583  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.806051 10424 raft_consensus.cc:3055] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:33.806744 10424 raft_consensus.cc:740] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
W20260709 12:18:33.807670  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:33.811825  6500 raft_consensus.cc:1240] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
W20260709 12:18:33.812673 10359 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.003s	sys 0.000s
I20260709 12:18:33.815841  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:33.817075  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 47254.6us
I20260709 12:18:33.732316 10308 raft_consensus.cc:740] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:33.826455  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:33.829623  6500 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
W20260709 12:18:33.814181 10359 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.003s	sys 0.000s
W20260709 12:18:33.813850 10410 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.007s	sys 0.011s
I20260709 12:18:33.827347 10442 consensus_queue.cc:1059] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:33.816478 10421 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:33.813531 10051 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:33.814046 10406 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:33.767791 10329 consensus_queue.cc:1059] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
W20260709 12:18:33.831605 10443 log.cc:927] Time spent T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 12:18:33.851137  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 168286us
I20260709 12:18:33.765795 10245 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:33.808130 10424 consensus_queue.cc:260] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:33.814293 10355 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:33.929930  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:34.215013  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:33.996382  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.261896  6582 raft_consensus.cc:1240] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 10. Current term is 11. Ops: [10.4-10.4]
I20260709 12:18:34.266494  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2088) took 1254 ms. Trace:
W20260709 12:18:33.876391  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.278115  6501 rpcz_store.cc:276] 0709 12:18:33.011888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.184734 (+172846us) service_pool.cc:224] Handling call
0709 12:18:33.206213 (+ 21479us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:33.354937 (+148724us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:33.354982 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:33.802420 (+447438us) spinlock_profiling.cc:243] Waited 4.32 ms on lock 0x7b2c00000c88. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:18:33.802853 (+   433us) log.cc:844] Serialized 26 byte log entry
0709 12:18:33.813266 (+ 10413us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:33.813547 (+   281us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:33.813628 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:34.265991 (+452363us) raft_consensus.cc:1685] finished
0709 12:18:34.266022 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:34.266387 (+   365us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":737,"raft.queue_time_us":182595,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":11181,"spinlock_wait_cycles":9945984,"thread_start_us":153125,"threads_started":2,"wal-append.queue_time_us":119135}
I20260709 12:18:34.280083  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: true dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:33.818162  6654 raft_consensus.cc:2393] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 8: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:18:33.894562  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.282239  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:34.293454  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:34.294341  6579 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 8: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:33.837841 10354 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.446s
I20260709 12:18:33.919792 10442 consensus_queue.cc:1243] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:18:34.283744  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:33.852710  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:34.286253  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:34.031503  6584 raft_consensus.cc:1240] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:34.300254  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:34.272589  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:34.317337  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 464703us
I20260709 12:18:34.324148  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:34.074139 10445 log.cc:927] Time spent T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.150s	user 0.000s	sys 0.007s
I20260709 12:18:33.848234 10051 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:33.839514 10421 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:34.336972  6578 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
W20260709 12:18:34.334183  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:34.338027  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:34.344870  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.350412  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:33.863961 10406 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:34.330459 10391 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:34.357621 10391 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:34.358048 10391 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:34.360172 10434 log.cc:927] Time spent T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.749s	user 0.002s	sys 0.009s
W20260709 12:18:34.361179 10304 consensus_meta.cc:287] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.709s	user 0.012s	sys 0.000s
I20260709 12:18:34.362010 10328 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.362279 10382 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.362576 10328 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:34.362804 10382 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:34.365520 10382 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:34.329949 10442 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.330739 10334 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:34.374472  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:34.375272  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:34.369269  6507 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.019s	sys 0.029s
I20260709 12:18:34.340047 10051 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:34.376278  6507 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.020s	sys 0.029s
W20260709 12:18:34.378010  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.344501 10421 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:34.343166 10274 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.366333 10245 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:34.366231 10328 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:34.373287 10382 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:34.400744 10452 log.cc:927] Time spent T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.325s	user 0.000s	sys 0.001s
I20260709 12:18:34.423214 10075 consensus_queue.cc:260] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:34.424674 10075 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:34.425863 10298 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:34.495393  7265 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38546 (request call id 442) took 1614 ms (client timeout 1916 ms). Trace:
W20260709 12:18:34.496660  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 56776.7us
I20260709 12:18:34.497923  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:34.499632  6653 raft_consensus.cc:1275] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:18:34.500052  6653 pending_rounds.cc:85] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 5
I20260709 12:18:34.500375  6653 pending_rounds.cc:107] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:18:34.500674  6653 raft_consensus.cc:2889] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:34.496023  7265 rpcz_store.cc:269] 0709 12:18:32.818934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:32.823114 (+  4180us) service_pool.cc:224] Handling call
0709 12:18:34.432703 (+1609589us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b54004e7ad4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:34.432841 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":552072448}
I20260709 12:18:34.505208  6657 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:18:34.505652  6657 pending_rounds.cc:85] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 5
I20260709 12:18:34.505911  6657 pending_rounds.cc:107] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:18:34.506171  6657 raft_consensus.cc:2889] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:34.383502 10334 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:34.514360  6473 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 2.213s (SENT) [suppressed 17 similar messages]
I20260709 12:18:34.514766  6578 raft_consensus.cc:1240] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:34.382227 10442 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:34.422049 10382 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:34.525490 10382 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:34.398492 10274 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:34.530457 10459 log.cc:927] Time spent T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 12:18:34.533370 10334 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:34.530602 10397 consensus_meta.cc:287] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.852s	user 0.011s	sys 0.003s
I20260709 12:18:34.507427 10343 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:34.503152 10417 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:18:34.422794 10328 consensus_queue.cc:1048] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:34.514684 10262 consensus_queue.cc:1059] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 12:18:34.432730 10425 consensus_queue.cc:1059] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.153s
W20260709 12:18:34.602624  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 180707us
I20260709 12:18:34.619055  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:34.620235  6505 raft_consensus.cc:1240] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
W20260709 12:18:34.640700  6652 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.026s	sys 0.028s
W20260709 12:18:34.650658  6652 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.027s	sys 0.029s
I20260709 12:18:34.650942  6500 raft_consensus.cc:3055] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:34.665549  6500 raft_consensus.cc:740] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:34.621541 10334 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.667385  6505 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:18:34.599426 10343 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:34.666297 10410 consensus_queue.cc:1059] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:18:34.674710 10343 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:34.675076 10392 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:34.675746 10392 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:34.677047 10411 raft_consensus.cc:3055] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:34.677497 10411 raft_consensus.cc:740] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:34.678485 10411 consensus_queue.cc:260] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:34.679585 10411 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:34.682292 10392 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:34.684423 10451 log.cc:927] Time spent T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 12:18:34.685212 10258 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:34.693090 10458 log.cc:927] Time spent T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 12:18:34.696555 10354 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:34.697212 10354 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:34.697775 10274 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:34.698997  6622 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:34.705791  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 191414us
I20260709 12:18:34.699481  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:34.723637  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.621711 10255 consensus_queue.cc:1059] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 12:18:34.667557 10355 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:34.642251 10304 consensus_queue.cc:1059] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.077s
W20260709 12:18:34.724434 10460 log.cc:927] Time spent T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:18:34.735738  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:34.737180  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 96973.9us
I20260709 12:18:34.723417  6500 consensus_queue.cc:260] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:34.744319  6500 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:34.748942  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:34.765671  6501 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:34.735028 10468 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:34.746068 10462 log.cc:927] Time spent T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 12:18:34.771461  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:34.751165 10346 consensus_meta.cc:287] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 1.061s	user 0.012s	sys 0.002s
I20260709 12:18:34.702087 10334 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:34.777710 10339 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:34.779047 10334 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:34.780949  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:34.751893 10245 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:34.738373 10465 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:34.783900  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:34.751451 10466 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.769219 10381 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:34.861658  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:34.863407  6584 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
I20260709 12:18:34.785945  6578 raft_consensus.cc:1240] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:34.862459  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:34.786273  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:34.787346  6579 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:18:34.865048  6506 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
I20260709 12:18:34.787796  6582 raft_consensus.cc:1275] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:34.765041 10355 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:18:34.840077  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 62615.9us
I20260709 12:18:34.866184  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:34.770965 10328 raft_consensus.cc:3055] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:34.740010 10392 consensus_queue.cc:1059] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 8.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 12:18:34.868626  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:34.868901  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:34.854341  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:34.867143  6500 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
W20260709 12:18:34.867604  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.775684 10255 raft_consensus.cc:3055] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:34.871176  6500 pending_rounds.cc:85] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 5
I20260709 12:18:34.871750  6506 raft_consensus.cc:3055] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:34.873826  6506 raft_consensus.cc:740] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:34.873400  6500 pending_rounds.cc:107] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:18:34.876207  6500 raft_consensus.cc:2889] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:34.876929  6580 raft_consensus.cc:2393] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 11: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:34.876833  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:34.783978 10463 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.792301 10472 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.767787 10051 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:34.863778 10450 log.cc:927] Time spent T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.340s	user 0.001s	sys 0.001s
I20260709 12:18:34.874729  6506 consensus_queue.cc:260] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:34.941692  6506 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:18:34.939505  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:34.943572  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:34.888715  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:34.939774  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 63164.6us
W20260709 12:18:34.947319  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 156546us
I20260709 12:18:34.798244 10334 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.949275  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:34.954502  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 87796.2us
I20260709 12:18:34.868692 10328 raft_consensus.cc:740] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:34.877697 10465 raft_consensus.cc:697] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:18:34.962778  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.937573 10353 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:34.967319  6506 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:18:34.873121 10255 raft_consensus.cc:740] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:34.938694 10340 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:34.947263 10298 consensus_meta.cc:287] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.521s	user 0.004s	sys 0.008s
I20260709 12:18:34.973738  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:34.781524 10468 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:34.946480 10471 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:34.942726 10354 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:34.981851  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:34.966331 10391 consensus_meta.cc:287] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.608s	user 0.009s	sys 0.004s
I20260709 12:18:34.870751 10466 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:34.987834  6576 raft_consensus.cc:1275] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 12:18:34.988492  6576 pending_rounds.cc:85] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 4
I20260709 12:18:34.988801  6576 pending_rounds.cc:107] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 12:18:34.989078  6576 raft_consensus.cc:2889] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:34.968204 10339 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:34.982338  6581 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:34.869904 10386 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.108s
I20260709 12:18:34.872526 10355 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:34.986691 10466 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:34.968518 10442 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:35.001634  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:34.966192 10410 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:18:34.966922 10334 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:34.956382 10463 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:34.942008 10472 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:35.065598  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 65513.8us
W20260709 12:18:35.065965 10450 log.cc:927] Time spent T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:18:34.995595 10473 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.002710 10339 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:35.002472 10298 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.002096 10397 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.183516  6580 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
W20260709 12:18:35.192318  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 47998.8us
I20260709 12:18:34.990401 10340 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:35.005119 10354 consensus_queue.cc:1059] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.136s
I20260709 12:18:35.201719  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:35.203197  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:35.205652  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:35.191545  7238 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:34.995291 10353 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.189s
I20260709 12:18:35.119726 10381 consensus_queue.cc:1048] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 12:18:35.214494  6549 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:34.995291 10391 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:35.191233 10473 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:35.144559 10466 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:35.190707 10339 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:35.120729 10479 consensus_queue.cc:1059] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.147s
I20260709 12:18:35.067991 10481 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.218s
I20260709 12:18:35.222775 10442 raft_consensus.cc:3055] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:35.294632 10442 raft_consensus.cc:740] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:35.295599 10442 consensus_queue.cc:260] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:35.297009 10442 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:35.298147 10463 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:35.298774 10463 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:35.299326 10463 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:35.293383 10477 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:35.300861 10410 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:35.301156 10477 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:35.301528 10410 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:35.302676 10340 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:35.303373 10347 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:35.303884 10477 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:35.122184 10328 consensus_queue.cc:260] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:35.126922 10255 consensus_queue.cc:260] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:35.303959 10347 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:35.294173  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 101021us
W20260709 12:18:35.309015  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 117594us
I20260709 12:18:35.311056  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:35.311928  6652 raft_consensus.cc:2410] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:18:35.315295  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:35.001713 10471 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:35.301119 10485 log.cc:927] Time spent T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
W20260709 12:18:35.323426  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 138235us
I20260709 12:18:35.326905  6652 raft_consensus.cc:1240] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 12:18:35.332024  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:35.193996 10298 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:35.334519  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:35.337021  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:35.292102 10392 raft_consensus.cc:2764] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
I20260709 12:18:35.341437  7265 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:35.338658  6655 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:35.339298  6652 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:35.210620 10397 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:35.337368 10496 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:35.308794 10350 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:35.303799 10410 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:35.353166 10483 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:35.353873 10464 log.cc:927] Time spent T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.418s	user 0.001s	sys 0.001s
I20260709 12:18:35.215761 10486 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.134s
I20260709 12:18:35.354084 10483 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:35.354489 10488 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 12:18:35.249296 10476 consensus_queue.cc:1048] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.112s
I20260709 12:18:35.356788 10483 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:35.353142 10490 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:35.357687 10490 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.358162 10490 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:35.360321 10490 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:35.364423 10487 consensus_queue.cc:1048] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.241s
I20260709 12:18:35.215945 10465 consensus_queue.cc:237] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:35.367918 10397 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:35.368866 10456 log.cc:927] Time spent T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.436s	user 0.001s	sys 0.001s
W20260709 12:18:35.378239 10480 log.cc:927] Time spent T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
I20260709 12:18:35.378898  9993 raft_consensus.cc:3055] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:35.379582  9993 raft_consensus.cc:740] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:35.380911  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2014) took 1722 ms. Trace:
I20260709 12:18:35.381335  6503 rpcz_store.cc:276] 0709 12:18:33.658044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.721885 (+ 63841us) service_pool.cc:224] Handling call
0709 12:18:33.722175 (+   290us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:33.739824 (+ 17649us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:33.739855 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:33.740402 (+   547us) log.cc:844] Serialized 52 byte log entry
0709 12:18:34.720662 (+980260us) spinlock_profiling.cc:243] Waited 3.66 ms on lock 0x7b180037e1c0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:18:34.877897 (+157235us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:34.952743 (+ 74846us) log.cc:844] Serialized 14 byte log entry
0709 12:18:35.218729 (+265986us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.218821 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:35.380584 (+161763us) raft_consensus.cc:1685] finished
0709 12:18:35.380690 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:35.380848 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38509,"prepare.queue_time_us":78807,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":66880,"raft.run_cpu_time_us":327,"raft.run_wall_time_us":76984,"spinlock_wait_cycles":8409472,"thread_start_us":395310,"threads_started":2,"wal-append.queue_time_us":1226519}
I20260709 12:18:35.384301 10334 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:35.385453 10468 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:35.391381 10308 consensus_queue.cc:260] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:35.393615 10308 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:35.395712 10471 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:35.396637  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:35.397214  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:35.397492  7273 raft_consensus.cc:2393] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 11: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:35.398093  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:35.398304  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 73255.2us
W20260709 12:18:35.401217  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 404925us
I20260709 12:18:35.250773 10466 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:35.316730 10343 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:35.403007  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:35.340932 10328 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:35.344039 10255 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:35.397872 10421 consensus_meta.cc:287] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 1.004s	user 0.012s	sys 0.000s
I20260709 12:18:35.392740  9993 consensus_queue.cc:260] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.4, Last appended by leader: 4, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:35.423806 10474 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.008s
W20260709 12:18:35.425786 10075 consensus_meta.cc:287] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 1.001s	user 0.011s	sys 0.001s
W20260709 12:18:35.426203 10438 log.cc:927] Time spent T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 1.821s	user 0.007s	sys 0.008s
I20260709 12:18:35.426419 10075 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.426998 10075 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:35.428699  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.402801 10334 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:35.398018 10350 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:35.423871  9993 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:35.404419 10353 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:18:35.448127  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:35.451570  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.429623  6650 raft_consensus.cc:1275] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 12:18:35.446375  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2104) took 1936 ms. Trace:
W20260709 12:18:35.449366  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 126269us
I20260709 12:18:35.442299  6652 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:18:35.453214  6508 rpcz_store.cc:276] 0709 12:18:33.509934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.510122 (+   188us) service_pool.cc:224] Handling call
0709 12:18:33.520839 (+ 10717us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:33.521346 (+   507us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:33.521377 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:33.521874 (+   497us) log.cc:844] Serialized 26 byte log entry
0709 12:18:33.591062 (+ 69188us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:33.591101 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:33.591160 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:35.425455 (+1834295us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:18:35.428638 (+  3183us) raft_consensus.cc:1685] finished
0709 12:18:35.428661 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:35.446275 (+ 17614us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":144,"raft.queue_time_us":240,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":7845,"spinlock_wait_cycles":4452096,"thread_start_us":68901,"threads_started":1,"wal-append.queue_time_us":82564}
I20260709 12:18:35.455483  6650 pending_rounds.cc:85] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 8
I20260709 12:18:35.429140 10481 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:35.457442  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 54519us
I20260709 12:18:35.457844  6650 pending_rounds.cc:107] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:18:35.458552  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:35.458853  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 59883.4us
I20260709 12:18:35.459213  6650 raft_consensus.cc:2889] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:35.459458  6577 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:18:35.458823  6622 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:35.461740 10500 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:35.462539 10500 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:35.464578 10491 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:35.465163 10491 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:35.466051 10463 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:35.466883 10499 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:35.467749 10491 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:35.467525 10499 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:35.468590  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 2107) took 1142 ms. Trace:
I20260709 12:18:35.469040  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:35.468998  6501 rpcz_store.cc:276] 0709 12:18:34.326061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.326248 (+   187us) service_pool.cc:224] Handling call
0709 12:18:35.468405 (+1142157us) spinlock_profiling.cc:243] Waited 414 ms on lock 0x7b54002c4cd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:35.468524 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":952736512}
I20260709 12:18:35.469890  7273 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:35.471359 10315 raft_consensus.cc:3055] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:35.471426  6549 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:18:35.471978 10315 raft_consensus.cc:740] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:35.472510 10483 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.472625 10491 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:35.473495 10491 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:35.473165 10483 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:35.460778 10481 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:35.475641 10483 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:35.476394 10491 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:35.476747 10481 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:35.478420  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.479195 10466 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:35.479276 10391 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:35.451581 10350 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:35.480405  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:35.480963 10298 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:35.480602  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.482959 10051 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:35.485456  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:35.485757 10385 consensus_meta.cc:287] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 1.669s	user 0.008s	sys 0.006s
I20260709 12:18:35.486138  6654 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:18:35.486621  7268 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:18:35.487105  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:35.488202  7268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38558 (request call id 593) took 1115 ms. Trace:
I20260709 12:18:35.488507 10420 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.488696  7268 rpcz_store.cc:276] 0709 12:18:34.373154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.373869 (+   715us) service_pool.cc:224] Handling call
0709 12:18:35.488022 (+1114153us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54004bbc54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:35.488147 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2555436416}
W20260709 12:18:35.489449  6504 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41304 (request call id 437) took 2011 ms (client timeout 2140 ms). Trace:
I20260709 12:18:35.489071 10420 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:35.490494 10491 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:35.490928 10347 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:35.491855 10075 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:35.493789 10473 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:35.497282  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:35.498144  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:35.490167  6504 rpcz_store.cc:269] 0709 12:18:33.478059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.492980 (+ 14921us) service_pool.cc:224] Handling call
0709 12:18:35.489208 (+1996228us) spinlock_profiling.cc:243] Waited 62.5 ms on lock 0x7b540046c8d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:35.489303 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":143777536}
I20260709 12:18:35.501905  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:35.503469  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:35.503926  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:35.504438  6509 raft_consensus.cc:2410] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 12 because replica has last-logged OpId of term: 10 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 3.
I20260709 12:18:35.504383  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:35.505124  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:35.505410  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:35.506026  6509 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:18:35.422480 10355 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:35.505136  6508 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:35.415978 10343 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:35.420085 10383 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:18:35.507752  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:35.527415  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.473013 10315 consensus_queue.cc:260] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:35.466883 10455 log.cc:927] Time spent T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.815s	user 0.000s	sys 0.003s
W20260709 12:18:35.526085  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 66163.8us
I20260709 12:18:35.526785  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:35.528409  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:35.422552 10421 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:35.528035  7238 leader_election.cc:336] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 2.140s (SENT)
I20260709 12:18:35.547732  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:35.548005  7238 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:35.549561  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:35.575177  6623 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:35.548647  7273 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:35.510692 10253 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:35.602752  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 74731.6us
I20260709 12:18:35.654052  6474 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:35.656693  7240 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:35.457720 10353 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:35.665431  6501 raft_consensus.cc:2393] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 6: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:35.703044  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:18:35.525522  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 66799.4us
W20260709 12:18:35.545688  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.707412  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:35.605590  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
W20260709 12:18:35.711828  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 106351us
W20260709 12:18:35.711979  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 55400us
W20260709 12:18:35.715849  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 140761us
I20260709 12:18:35.717008  6623 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:35.525966 10075 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:35.718787  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 176482us
W20260709 12:18:35.555188 10411 consensus_meta.cc:287] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.869s	user 0.010s	sys 0.005s
I20260709 12:18:35.724263  6622 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:35.782574  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 58453.4us
W20260709 12:18:35.783015  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 129089us
I20260709 12:18:35.528425 10493 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.529114 10335 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:35.457945 10334 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:35.847100  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:35.789565  6508 raft_consensus.cc:2410] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:18:35.792735  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:35.531602 10479 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.784896  6474 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:18:35.813068  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.854156  6583 raft_consensus.cc:2393] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 9: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:18:35.854048  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 134146us
I20260709 12:18:35.707439 10496 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:35.667610 10442 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:35.856570  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:35.707124 10315 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:35.708926 10298 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:35.861022  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 77777.1us
I20260709 12:18:35.544312 10383 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:35.781625 10397 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:35.781376 10345 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:35.861779  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:35.876154  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:35.868768  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 589) took 2171 ms. Trace:
I20260709 12:18:35.876888  6583 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:18:35.877319  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:35.873963  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 157068us
I20260709 12:18:35.877178  6656 rpcz_store.cc:276] 0709 12:18:33.697333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.698019 (+   686us) service_pool.cc:224] Handling call
0709 12:18:33.698333 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:33.725811 (+ 27478us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:33.725864 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:33.726481 (+   617us) log.cc:844] Serialized 26 byte log entry
0709 12:18:34.016042 (+289561us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:34.016101 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:34.016213 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:35.466143 (+1449930us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:18:35.863312 (+397169us) raft_consensus.cc:1685] finished
0709 12:18:35.863335 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:35.868650 (+  5315us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27023,"raft.queue_time_us":35098,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":27830,"spinlock_wait_cycles":32116992,"thread_start_us":200904,"threads_started":1,"wal-append.queue_time_us":826866}
I20260709 12:18:35.879288  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:35.879880  6582 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:35.879966  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:35.881261  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:35.795771 10466 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:35.881589  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:35.886932  6583 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:18:35.881049  6656 raft_consensus.cc:2410] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:18:35.878010  6579 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:35.885416  7269 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:18:35.889122  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:35.889856  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:35.890218  6656 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:35.890753  6501 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:35.892108  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:35.893491  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:35.709918 10339 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:35.894527  6656 raft_consensus.cc:2410] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 12:18:35.896962  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 112155us
I20260709 12:18:35.897593  6582 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:18:35.858450 10388 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.863833 10411 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:35.909518  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 52549.5us
I20260709 12:18:35.909614  6623 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:35.865233 10417 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:35.905417  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:35.720171 10253 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:35.801579 10075 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:35.793488  9875 consensus_queue.cc:1059] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.081s
I20260709 12:18:35.887953 10397 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:35.931977  6649 raft_consensus.cc:2393] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 8: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:35.950318  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:35.884568 10345 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:35.962419  6656 raft_consensus.cc:2410] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:18:35.963766  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:35.964100  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:35.964725  6508 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:35.964963  6656 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:35.966604  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:35.967410  6656 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
W20260709 12:18:35.968923  6502 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 2006) took 2332 ms (client timeout 1714 ms). Trace:
W20260709 12:18:35.969409  6502 rpcz_store.cc:269] 0709 12:18:33.637022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.637213 (+   191us) service_pool.cc:224] Handling call
0709 12:18:35.968717 (+2331504us) spinlock_profiling.cc:243] Waited 45.6 ms on lock 0x7b54001e7dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:35.968846 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":104980864}
I20260709 12:18:35.969769  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:35.970552  6656 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 4.
I20260709 12:18:35.861632 10479 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:35.971984  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:35.974323  6656 raft_consensus.cc:1240] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:35.917707 10417 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:35.920456 10245 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.898970 10308 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 12:18:35.917603 10468 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:35.874359 10298 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:35.983430  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 45034.7us
I20260709 12:18:35.984905  6508 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:18:35.985682  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:35.913472 10354 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:35.980717  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 69819.3us
I20260709 12:18:35.932569 10481 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.994366 10245 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:36.004314  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:35.865739 10493 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:35.985055 10492 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:35.902163 10466 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:35.994714 10508 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:35.970173 10397 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:35.980321 10449 log.cc:927] Time spent T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 2.206s	user 0.005s	sys 0.008s
I20260709 12:18:35.908274 10410 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 12:18:35.969269 10345 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:36.018604  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:36.037314  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:36.038707  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:35.938820 10391 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 12:18:35.985831 10417 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:36.039462  6501 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:18:36.042850 10513 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:36.043411 10513 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:36.020474  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:36.044718  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.918490 10411 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:36.045950 10513 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:36.046995 10411 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:36.047539  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:35.882302 10350 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:36.049908  7268 raft_consensus.cc:1240] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:36.050711 10350 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:36.026662 10051 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:18:36.026957 10253 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:36.027083 10245 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:36.027850 10298 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:36.040839 10391 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:18:36.054279  6504 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.001s	sys 0.004s
W20260709 12:18:36.060708  6504 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.001s	sys 0.005s
I20260709 12:18:35.913872 10388 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:36.070961  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:35.997506 10468 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:36.077100  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:36.092705  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 69866.4us
I20260709 12:18:36.010577 10488 raft_consensus.cc:3055] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:36.096282  6506 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:18:36.100914  6581 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:36.015580 10389 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:36.104660  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:36.105959  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:36.112426  6582 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:18:36.112466  6579 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 4.
W20260709 12:18:36.094419 10502 log.cc:927] Time spent T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.571s	user 0.000s	sys 0.001s
I20260709 12:18:36.114764 10417 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:36.122264  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 81424.2us
I20260709 12:18:36.123029  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:36.130332  6578 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:18:36.132085  6548 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:36.096237 10468 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:36.091941 10245 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:36.137018  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:36.149475  6475 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:36.096556 10343 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:36.138401  6548 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:36.150869  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:36.151558  6578 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:36.098219 10488 raft_consensus.cc:740] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:36.021513 10481 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:36.101979 10499 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:36.111434 10421 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:36.092077 10283 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:18:36.127521 10482 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:36.157595  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:36.178424  6578 raft_consensus.cc:2393] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 11: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:18:36.160470  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 70713.9us
I20260709 12:18:36.161013  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:36.022782 10354 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:36.180310  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:36.176062  6654 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.002s	sys 0.022s
I20260709 12:18:36.139904 10473 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:36.100816  9875 consensus_queue.cc:1059] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 12:18:36.185827  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:36.184901  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:36.187994  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:36.189605  6578 raft_consensus.cc:2376] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9. Re-sending same reply.
I20260709 12:18:36.140038 10505 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:36.188935  6575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1876) took 2640 ms. Trace:
I20260709 12:18:36.192207  6622 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:36.192723  6575 rpcz_store.cc:276] 0709 12:18:33.548232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.548572 (+   340us) service_pool.cc:224] Handling call
0709 12:18:33.548911 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:33.549634 (+   723us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:33.549669 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:33.550883 (+  1214us) log.cc:844] Serialized 26 byte log entry
0709 12:18:33.814565 (+263682us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:33.814608 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:33.814692 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:35.526189 (+1711497us) spinlock_profiling.cc:243] Waited 2.81 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:18:36.188553 (+662364us) raft_consensus.cc:1685] finished
0709 12:18:36.188584 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:36.188865 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":88,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"spinlock_wait_cycles":6452096,"thread_start_us":231487,"threads_started":1,"wal-append.queue_time_us":187614}
I20260709 12:18:36.195466  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:36.196914  6575 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:36.198961  6575 raft_consensus.cc:1275] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:36.155870 10353 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:36.203910  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:36.183267  6654 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.002s	sys 0.022s
I20260709 12:18:36.184362  6584 raft_consensus.cc:2410] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:18:36.204713  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:36.159250 10497 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.033605 10492 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:36.184643 10473 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:36.213306 10473 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:36.157160 10507 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.344s
I20260709 12:18:36.129376 10389 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:36.182487 10499 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:36.177975 10432 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.165124 10308 consensus_queue.cc:1059] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 12:18:36.224215 10492 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:36.156844 10245 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:36.216454 10353 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:36.207844 10383 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:36.206022 10516 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:36.211103 10508 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:36.226338  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:36.207108 10518 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:18:36.216248 10328 consensus_meta.cc:287] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.798s	user 0.009s	sys 0.003s
I20260709 12:18:36.218668 10499 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:36.245780 10424 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:36.237421 10510 log.cc:927] Time spent T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
I20260709 12:18:36.191522 10482 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:36.260309 10354 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:36.239665 10479 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:36.235833 10516 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:36.217171 10497 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:36.230523 10353 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:36.237360 10421 raft_consensus.cc:3055] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:36.235471 10508 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:36.239548 10466 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:36.239059 10388 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:36.254690 10499 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:36.273108 10493 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:36.256641 10518 raft_consensus.cc:697] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:36.236254 10432 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:36.218168 10488 consensus_queue.cc:260] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:36.272195 10389 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:36.425786 10488 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:36.427212 10432 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:36.343441  6500 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
W20260709 12:18:36.350315  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 211951us
I20260709 12:18:36.431339  6548 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:36.351672  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:36.246858  9993 consensus_meta.cc:287] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.794s	user 0.008s	sys 0.000s
I20260709 12:18:36.233398 10075 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:36.464730  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1908) took 1512 ms. Trace:
I20260709 12:18:36.465037  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:36.465418  6580 rpcz_store.cc:276] 0709 12:18:34.955438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.955614 (+   176us) service_pool.cc:224] Handling call
0709 12:18:36.464249 (+1508635us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b54002657d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:36.464507 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":387873536}
I20260709 12:18:36.467165  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:36.374114 10245 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:36.376583 10051 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:18:36.381227 10255 consensus_meta.cc:287] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.961s	user 0.007s	sys 0.015s
W20260709 12:18:36.381633  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 232103us
I20260709 12:18:36.386276  9920 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:18:36.273236 10414 log.cc:927] Time spent T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 3.102s	user 0.004s	sys 0.023s
I20260709 12:18:36.266597 10421 raft_consensus.cc:740] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:36.501094  6658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1821) took 4225 ms. Trace:
I20260709 12:18:36.307612 10497 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:36.501613  6658 rpcz_store.cc:276] 0709 12:18:32.275279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:32.925032 (+649753us) service_pool.cc:224] Handling call
0709 12:18:32.925411 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:32.950962 (+ 25551us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:32.951012 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:32.970023 (+ 19011us) log.cc:844] Serialized 26 byte log entry
0709 12:18:33.132773 (+162750us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:33.132837 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:33.132940 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:36.272695 (+3139755us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:18:36.500503 (+227808us) raft_consensus.cc:1685] finished
0709 12:18:36.500543 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:36.500897 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13962,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":101,"spinlock_wait_cycles":30572288,"thread_start_us":106716,"threads_started":1,"wal-append.queue_time_us":172492}
I20260709 12:18:36.503374  6651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1815) took 4233 ms. Trace:
I20260709 12:18:36.503695  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:36.262647 10500 consensus_meta.cc:287] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.800s	user 0.011s	sys 0.000s
I20260709 12:18:36.291021 10505 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:36.269627 10508 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:36.264736 10516 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:36.525673  9993 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.278478 10518 consensus_queue.cc:237] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:36.416534  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1848) took 1089 ms. Trace:
W20260709 12:18:36.418290  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 213520us
I20260709 12:18:36.443176 10499 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:36.447085  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:36.266485 10353 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:36.450534  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:36.467206 10383 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:36.374960  6473 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:36.473955 10532 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.459618 10298 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:36.565055 10298 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:36.540725 10500 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:36.492162 10051 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:36.457265 10466 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.504123  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:36.501214  6649 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 1846) took 1186 ms (client timeout 1563 ms). Trace:
I20260709 12:18:36.511845 10493 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.512696  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:36.512887  7274 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:36.503810  6651 rpcz_store.cc:276] 0709 12:18:32.270056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:32.889416 (+619360us) service_pool.cc:224] Handling call
0709 12:18:33.143426 (+254010us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:33.244796 (+101370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:33.244849 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:33.244915 (+    66us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:33.244966 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.504690 (+2259724us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b540008c0d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:35.504739 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:36.503033 (+998294us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b540008c0d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:36.503209 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":815947392}
I20260709 12:18:36.513931 10245 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:36.512728 10421 consensus_queue.cc:260] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:36.524485 10468 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:36.499523 10345 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:36.542871 10075 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:36.586462 10075 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:36.552662 10272 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.588498  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:36.558015  6655 rpcz_store.cc:276] 0709 12:18:35.326478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.399578 (+ 73100us) service_pool.cc:224] Handling call
0709 12:18:35.399897 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:35.524896 (+124999us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:35.524952 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:35.528592 (+  3640us) log.cc:844] Serialized 26 byte log entry
0709 12:18:35.855290 (+326698us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:35.855328 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.855629 (+   301us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:36.410187 (+554558us) raft_consensus.cc:1685] finished
0709 12:18:36.410231 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:36.416142 (+  5911us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4299,"raft.queue_time_us":60059,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":1215,"thread_start_us":446175,"threads_started":2,"wal-append.queue_time_us":389588}
I20260709 12:18:36.588850 10272 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:36.589830  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:36.595512 10499 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:36.555665  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 124660us
I20260709 12:18:36.559769 10365 raft_consensus.cc:3055] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:36.565450 10383 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:36.504784  7270 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
W20260709 12:18:36.570197  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:36.579113  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 1.63314e+06us
I20260709 12:18:36.582834 10493 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:36.583382  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:36.584204 10051 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:36.584465  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.585114  6658 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7.
W20260709 12:18:36.566869  6649 rpcz_store.cc:269] 0709 12:18:35.314724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.314925 (+   201us) service_pool.cc:224] Handling call
0709 12:18:36.500714 (+1185789us) spinlock_profiling.cc:243] Waited 616 ms on lock 0x7b54000744d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:36.501017 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1415775488}
I20260709 12:18:36.590111  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:36.596781  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:36.591315 10272 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:36.591415 10421 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:36.526669  9993 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:36.593099 10298 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:36.593557 10345 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:36.593351 10245 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:36.594468  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 220039us
I20260709 12:18:36.594973 10255 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:36.595036  9574 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:36.622285  6501 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 13.
I20260709 12:18:36.623482  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:36.623572  6473 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:36.620515 10488 consensus_queue.cc:1059] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 12:18:36.624799  6473 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:36.625123  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.625604 10488 raft_consensus.cc:3055] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:36.625721  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 602) took 1144 ms. Trace:
I20260709 12:18:36.626165 10488 raft_consensus.cc:740] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:36.565089 10532 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:36.627516 10488 consensus_queue.cc:260] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:36.600827  6623 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:36.601647  6475 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:36.618677  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.601948  6548 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:36.599291 10365 raft_consensus.cc:740] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:36.621966  6652 raft_consensus.cc:1240] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:36.599000 10468 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:36.626238  6649 rpcz_store.cc:276] 0709 12:18:35.481113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.624698 (+1143585us) service_pool.cc:224] Handling call
0709 12:18:36.625122 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:36.625317 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:36.625348 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:36.625413 (+    65us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:36.625442 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.625502 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:36.625633 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:36.628597 10497 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:36.628969  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:36.629962 10532 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:36.598088 10505 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:36.597831  7269 raft_consensus.cc:1240] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:36.634083  6658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 601) took 1153 ms. Trace:
I20260709 12:18:36.641387  6658 rpcz_store.cc:276] 0709 12:18:35.480840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.618611 (+1137771us) service_pool.cc:224] Handling call
0709 12:18:36.633408 (+ 14797us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:36.633642 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:36.633679 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:36.633761 (+    82us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:36.633809 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.633874 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:36.633995 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:36.626530  6473 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:18:36.632654 10533 log.cc:927] Time spent T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:18:36.601686 10383 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:36.624272  6501 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:36.640153  6508 raft_consensus.cc:2393] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 9: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:36.596300 10499 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:36.639911  6509 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.866s	user 0.001s	sys 0.021s
W20260709 12:18:36.664207  6656 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.013s	sys 0.022s
W20260709 12:18:36.664795  6656 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.013s	sys 0.022s
I20260709 12:18:36.597282 10541 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:36.632658  9993 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:36.596771 10466 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:36.673977  6509 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.967s	user 0.001s	sys 0.021s
W20260709 12:18:36.674746  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 72970.1us
I20260709 12:18:36.682085 10245 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:36.683037 10051 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:36.689662  6655 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 9.
I20260709 12:18:36.682423  6651 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:18:36.693723 10442 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.694757 10515 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.694545 10442 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:36.695561 10515 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:36.686481 10488 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:36.705152  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1853) took 1377 ms. Trace:
I20260709 12:18:36.705569  6656 rpcz_store.cc:276] 0709 12:18:35.328061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.975596 (+647535us) service_pool.cc:224] Handling call
0709 12:18:35.975925 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:36.156262 (+180337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:36.156296 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:36.156788 (+   492us) log.cc:844] Serialized 26 byte log entry
0709 12:18:36.665314 (+508526us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:36.676798 (+ 11484us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.688142 (+ 11344us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:36.692033 (+  3891us) raft_consensus.cc:1685] finished
0709 12:18:36.692059 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:36.705087 (+ 13028us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76,"raft.queue_time_us":76960,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":180,"thread_start_us":687732,"threads_started":2,"wal-append.queue_time_us":507634}
I20260709 12:18:36.707770  6656 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:36.708310  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2339) took 1142 ms. Trace:
I20260709 12:18:36.708673  6656 rpcz_store.cc:276] 0709 12:18:35.566339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.707069 (+1140730us) service_pool.cc:224] Handling call
0709 12:18:36.707425 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:36.708081 (+   656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.708234 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:36.710124  6656 raft_consensus.cc:1275] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:18:36.710759  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.711145 10354 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:36.715794 10473 consensus_meta.cc:287] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.502s	user 0.004s	sys 0.013s
I20260709 12:18:36.716526 10513 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:36.721482 10542 log.cc:927] Time spent T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.059s	user 0.000s	sys 0.002s
I20260709 12:18:36.723448 10518 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:36.760516 10518 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:36.760996 10518 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:36.724566  6654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1850) took 1397 ms. Trace:
I20260709 12:18:36.728663  7271 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:18:36.730376 10245 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:36.692417 10432 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.682461 10255 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:36.753717 10534 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.693918 10386 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:36.768409  6504 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.005s	sys 0.020s
I20260709 12:18:36.703568 10500 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.776199 10245 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:36.778237 10489 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:18:36.780833  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 71237.2us
I20260709 12:18:36.781451 10501 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.783677 10406 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:36.785650  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 87913.4us
I20260709 12:18:36.785871  6501 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
W20260709 12:18:36.785683  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 100227us
I20260709 12:18:36.711862 10354 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:36.799860 10428 raft_consensus.cc:2764] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:18:36.716548 10329 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:18:36.794730  6504 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.006s	sys 0.021s
I20260709 12:18:36.792092  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2344) took 1217 ms. Trace:
I20260709 12:18:36.753867 10547 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:36.800882  6656 rpcz_store.cc:276] 0709 12:18:35.569180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.709396 (+1140216us) service_pool.cc:224] Handling call
0709 12:18:36.709722 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:36.710430 (+   708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.786439 (+ 76009us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:36.703323 10499 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:36.765223 10051 raft_consensus.cc:697] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:18:36.795343  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 194780us
I20260709 12:18:36.764544  6654 rpcz_store.cc:276] 0709 12:18:35.327118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.487960 (+160842us) service_pool.cc:224] Handling call
0709 12:18:35.488227 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:35.524570 (+ 36343us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:35.631635 (+107065us) log.cc:844] Serialized 14 byte log entry
0709 12:18:36.640584 (+1008949us) log.cc:844] Serialized 14 byte log entry
0709 12:18:36.723966 (+ 83382us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:36.724043 (+    77us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:36.724141 (+    98us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.724226 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:36.724394 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51121,"prepare.queue_time_us":455186,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":3037,"raft.queue_time_us":52324,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":9587,"thread_start_us":1106892,"threads_started":3,"wal-append.queue_time_us":638329}
W20260709 12:18:36.811293  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 184872us
I20260709 12:18:36.807252 10329 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:36.818310  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:36.819398 10354 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:36.820721 10410 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:36.823334  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:36.824275  6502 raft_consensus.cc:2393] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 11: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:36.825555  6650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1832) took 3126 ms. Trace:
I20260709 12:18:36.825930  6650 rpcz_store.cc:276] 0709 12:18:33.699335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.702178 (+  2843us) service_pool.cc:224] Handling call
0709 12:18:33.702479 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:35.459738 (+1757259us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.825131 (+1365393us) spinlock_profiling.cc:243] Waited 1.73 s on lock 0x7b54002279d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:36.825385 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3971042176}
I20260709 12:18:36.827145  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:18:36.828819  6550 connection.cc:256] RPC call timeout handler was delayed by -1.4313325219998205s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1946s for a response
I20260709 12:18:36.831513 10051 consensus_queue.cc:237] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:36.834210 10365 consensus_queue.cc:260] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:36.835548 10315 consensus_meta.cc:287] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.966s	user 0.017s	sys 0.013s
I20260709 12:18:36.835801 10365 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:36.836246  6506 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:36.837193 10245 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:36.839835  6504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2110) took 1340 ms. Trace:
I20260709 12:18:36.840543  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:36.841269 10329 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:36.841282  7266 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:36.845464  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:36.846206  6581 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:36.850173  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:36.853595  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 47243.1us
I20260709 12:18:36.839948  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:36.840971  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2333) took 1279 ms. Trace:
I20260709 12:18:36.855938  6649 rpcz_store.cc:276] 0709 12:18:35.561182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.670230 (+1109048us) service_pool.cc:224] Handling call
0709 12:18:36.769444 (+ 99214us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:36.815274 (+ 45830us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:36.815307 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:36.840657 (+ 25350us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:36.840703 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.840763 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:36.840900 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:36.858516  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2105) took 3116 ms. Trace:
I20260709 12:18:36.859007  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:36.859407 10255 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:36.860844  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 52623.7us
I20260709 12:18:36.803215 10547 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:36.792868 10432 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:36.813861 10466 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:36.866569  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:36.876489  6550 connection.cc:256] RPC call timeout handler was delayed by -1.2249071319999985s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21530000000000002s for a response
W20260709 12:18:36.876924  6550 connection.cc:256] RPC call timeout handler was delayed by -1.2069071319999694s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21730000000000002s for a response
I20260709 12:18:36.856382 10541 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:36.881042 10541 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:36.881898  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:36.883060 10339 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:36.884460  6653 raft_consensus.cc:2410] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:18:36.791092 10473 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:36.850319 10515 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:36.840372  6504 rpcz_store.cc:276] 0709 12:18:35.499447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.500100 (+   653us) service_pool.cc:224] Handling call
0709 12:18:35.500506 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:35.500733 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:35.542908 (+ 42175us) log.cc:844] Serialized 14 byte log entry
0709 12:18:36.839264 (+1296356us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:36.839402 (+   138us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:36.839474 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.839579 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:36.839744 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37939,"prepare.queue_time_us":44983,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":91592,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":7118,"thread_start_us":1222862,"threads_started":2,"wal-append.queue_time_us":508517}
I20260709 12:18:36.897302  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:36.900817  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:36.901731  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2118) took 1377 ms. Trace:
I20260709 12:18:36.897509  7240 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:36.812777 10513 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:36.909235  6657 raft_consensus.cc:1240] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:18:36.911850 10513 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:36.912477 10466 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.913256 10466 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:36.913578  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:36.914321  6583 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:18:36.915304  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:36.916648  6580 raft_consensus.cc:2410] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:18:36.918763  6650 raft_consensus.cc:2393] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 8: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:36.920017 10432 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:36.921901 10075 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:36.922891  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:36.924283  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:36.909188 10521 log.cc:927] Time spent T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 12:18:36.865811 10547 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:36.914300  6549 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
W20260709 12:18:36.886592 10397 consensus_meta.cc:287] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.790s	user 0.014s	sys 0.004s
I20260709 12:18:36.858956  6507 rpcz_store.cc:276] 0709 12:18:33.684377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.723106 (+ 38729us) service_pool.cc:224] Handling call
0709 12:18:33.723436 (+   330us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:33.742317 (+ 18881us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:33.742372 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:33.766010 (+ 23638us) log.cc:844] Serialized 52 byte log entry
0709 12:18:34.382109 (+616099us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:36.454491 (+2072382us) spinlock_profiling.cc:243] Waited 1.77 ms on lock 0x7b18002a1300. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2875a314bc 00007f2872d7f48b 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d99455 00007f28798813c7 00007f2875e939a7 00007f2875ea30bf
0709 12:18:36.454800 (+   309us) log.cc:844] Serialized 14 byte log entry
0709 12:18:36.701645 (+246845us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.754367 (+ 52722us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:36.764928 (+ 10561us) raft_consensus.cc:1685] finished
0709 12:18:36.790223 (+ 25295us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:36.800597 (+ 10374us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18273,"prepare.queue_time_us":317841,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":3183,"raft.queue_time_us":42272,"raft.run_cpu_time_us":353,"raft.run_wall_time_us":46017,"spinlock_wait_cycles":4068608,"thread_start_us":961519,"threads_started":3,"wal-append.queue_time_us":979430,"wal-append.run_cpu_time_us":4113,"wal-append.run_wall_time_us":1829246}
I20260709 12:18:36.931876  6584 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
I20260709 12:18:36.932466  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:36.932961  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:36.933135  6507 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
I20260709 12:18:36.933701  6584 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:36.934294  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:36.934717  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:36.815819 10500 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:36.943107  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 75493.3us
I20260709 12:18:36.943935  6654 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:18:36.864467 10386 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:36.956384  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:36.927510 10541 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:36.887926 10332 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:36.963425  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:36.929518 10493 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:36.813984 10501 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:36.957134  6580 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:36.903059  6509 rpcz_store.cc:276] 0709 12:18:35.524847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.525018 (+   171us) service_pool.cc:224] Handling call
0709 12:18:35.525351 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:35.525548 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:36.691861 (+1166313us) log.cc:844] Serialized 14 byte log entry
0709 12:18:36.806306 (+114445us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:36.815805 (+  9499us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:36.856468 (+ 40663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:36.865110 (+  8642us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:36.901663 (+ 36553us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20339,"prepare.queue_time_us":317296,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":10711,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":42906,"thread_start_us":1064798,"threads_started":2,"wal-append.queue_time_us":108137}
I20260709 12:18:36.815205 10489 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:36.987023  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:36.988411  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:36.970609  6584 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:18:36.989261  6654 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:36.860412 10534 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:36.993258  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:36.956998 10532 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:37.000391  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:37.005038 10497 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:37.006568  7240 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:37.007287  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:37.007696  7240 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:37.008114  6509 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 9.
I20260709 12:18:37.009123  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:37.009969  6509 raft_consensus.cc:2410] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:18:37.011531  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 442) took 1105 ms. Trace:
I20260709 12:18:36.995093 10513 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:36.947772 10397 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:37.012816 10513 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:37.014295 10397 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:37.014039 10353 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:37.014991 10513 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:37.015453  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:37.016794  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:37.017490  6580 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:37.021389 10499 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:37.024119 10500 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:37.024992  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:37.025671  6550 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:37.026018 10442 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:37.026629  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:37.001155  6504 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:18:37.011861  6509 rpcz_store.cc:276] 0709 12:18:35.906606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.010561 (+1103955us) service_pool.cc:224] Handling call
0709 12:18:37.010929 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:37.011193 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:37.011220 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:37.011279 (+    59us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:37.011313 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:37.011367 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:37.011461 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:36.994771 10128 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:37.029524  6656 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:37.026065  6581 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
I20260709 12:18:37.040669  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:37.042325  6583 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:36.999101 10532 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:37.037644 10397 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:37.026439 10420 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:37.033202 10128 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.056957  6654 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
W20260709 12:18:37.057226  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:37.064317 10555 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.038692 10442 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.065034 10555 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:37.044528 10541 consensus_queue.cc:1048] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 12:18:37.111192  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 443) took 1142 ms. Trace:
I20260709 12:18:37.115849  7273 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:37.121754  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:37.067677  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 103147us
I20260709 12:18:37.111121  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:37.128064  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 102507us
I20260709 12:18:37.129103  6507 rpcz_store.cc:276] 0709 12:18:35.962484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.031146 (+1068662us) service_pool.cc:224] Handling call
0709 12:18:37.054544 (+ 23398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:37.100970 (+ 46426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:37.101002 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:37.104310 (+  3308us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:37.104394 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:37.104461 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:37.104589 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:37.131108  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:37.116156  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:37.101819  6584 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:18:37.053428 10532 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:37.136319  7269 raft_consensus.cc:2393] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 9: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:37.121405  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:37.054831 10397 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.099960  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:37.141135  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:37.141721  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 96986.8us
W20260709 12:18:37.116534 10516 consensus_meta.cc:287] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.530s	user 0.013s	sys 0.000s
I20260709 12:18:37.065111 10128 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:37.146368  6584 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:37.146468  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1911) took 1270 ms. Trace:
I20260709 12:18:37.146992  6473 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:37.148307 10365 consensus_queue.cc:1048] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:37.149770 10274 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:37.151352  7268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38558 (request call id 594) took 1651 ms. Trace:
I20260709 12:18:37.151710  7268 rpcz_store.cc:276] 0709 12:18:35.524472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.526904 (+  2432us) service_pool.cc:224] Handling call
0709 12:18:35.527184 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:36.050285 (+523101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:37.150863 (+1100578us) spinlock_profiling.cc:243] Waited 522 ms on lock 0x7b54006c2354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:37.151166 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1201640192}
I20260709 12:18:37.152714  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:37.153546  6654 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:37.154142 10128 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:37.154516  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:37.155789  6654 raft_consensus.cc:2393] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 11: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:37.104700 10345 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:37.164624 10345 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:37.165122 10345 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:37.147049  6577 rpcz_store.cc:276] 0709 12:18:35.876144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.878564 (+  2420us) service_pool.cc:224] Handling call
0709 12:18:35.878873 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:36.028659 (+149786us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:18:36.028942 (+   283us) log.cc:844] Serialized 14 byte log entry
0709 12:18:36.065013 (+ 36071us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:36.065491 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 12:18:36.065652 (+   161us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:36.065690 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:37.121107 (+1055417us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b5400329e54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:37.121201 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:37.126939 (+  5738us) raft_consensus.cc:1685] finished
0709 12:18:37.134459 (+  7520us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:37.146264 (+ 11805us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":793,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"raft.queue_time_us":5357,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":24098,"spinlock_wait_cycles":343397760,"thread_start_us":35673,"threads_started":1,"wal-append.queue_time_us":36052,"wal-append.run_cpu_time_us":2558,"wal-append.run_wall_time_us":1009950}
I20260709 12:18:37.076566 10353 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:37.168900  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:37.177148 10508 consensus_meta.cc:287] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.651s	user 0.000s	sys 0.013s
I20260709 12:18:37.177719 10508 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:37.178982  6582 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
I20260709 12:18:37.120573 10505 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:37.145069 10397 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:37.150841 10365 raft_consensus.cc:2955] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Committing config change with OpId 7.6: config changed from index 2 to 6, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:18:37.067416 10546 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:37.185360  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:37.185364  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:37.192713  6653 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:37.196478  6509 raft_consensus.cc:1275] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:18:37.196123  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:37.188956  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 46237.9us
I20260709 12:18:37.191744  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:37.200009  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:37.197889  6579 raft_consensus.cc:1813] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Handling vote request from an unknown peer f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:37.164145  6474 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:37.192736  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41304 (request call id 444) took 1199 ms. Trace:
I20260709 12:18:37.199354  6650 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:37.183396  6657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 599) took 1703 ms. Trace:
I20260709 12:18:37.180634  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:37.192998 10421 consensus_meta.cc:287] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.555s	user 0.011s	sys 0.001s
I20260709 12:18:37.189769 10417 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:37.203469  6657 rpcz_store.cc:276] 0709 12:18:35.480227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.134473 (+654246us) service_pool.cc:224] Handling call
0709 12:18:36.156529 (+ 22056us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:36.910211 (+753682us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:37.183064 (+272853us) spinlock_profiling.cc:243] Waited 735 ms on lock 0x7b54001509d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:37.183342 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1690614656}
I20260709 12:18:37.200613  6509 pending_rounds.cc:85] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 5
I20260709 12:18:37.190050  6473 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:37.213390  6579 raft_consensus.cc:2410] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:18:37.211724  7270 raft_consensus.cc:1813] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Handling vote request from an unknown peer f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:37.214741  7270 raft_consensus.cc:2410] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:18:37.215294  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:37.202589  6508 rpcz_store.cc:276] 0709 12:18:35.993437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.004089 (+ 10652us) service_pool.cc:224] Handling call
0709 12:18:37.192512 (+1188423us) spinlock_profiling.cc:243] Waited 597 ms on lock 0x7b54001e82d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:37.192641 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1373574912}
I20260709 12:18:37.142014 10473 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:37.212208  6509 pending_rounds.cc:107] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:18:37.221355  6509 raft_consensus.cc:2889] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:37.221612  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:37.224309  7265 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38562 (request call id 649) took 1988 ms (client timeout 2252 ms). Trace:
I20260709 12:18:37.226006  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:37.226279  6550 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:37.226814  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:37.227109 10382 raft_consensus.cc:3055] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:37.227574 10382 raft_consensus.cc:740] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:37.227846  6550 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:37.228024  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:37.228415  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2035) took 1145 ms. Trace:
I20260709 12:18:37.228916  6584 raft_consensus.cc:2393] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 11: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:37.228865  6509 rpcz_store.cc:276] 0709 12:18:36.079701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.048565 (+968864us) service_pool.cc:224] Handling call
0709 12:18:37.145353 (+ 96788us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:37.225231 (+ 79878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:37.225397 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:37.229369  6550 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:37.229750 10499 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:37.230201 10499 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:18:37.230320  6500 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 2122) took 1672 ms (client timeout 1618 ms). Trace:
I20260709 12:18:37.230641 10499 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:37.230816  6500 rpcz_store.cc:269] 0709 12:18:35.558191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.708720 (+150529us) service_pool.cc:224] Handling call
0709 12:18:37.230025 (+1521305us) spinlock_profiling.cc:243] Waited 495 ms on lock 0x7b54001effd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:37.230245 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1137711488}
I20260709 12:18:37.173251 10516 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:37.224660  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2338) took 1656 ms. Trace:
I20260709 12:18:37.233479 10417 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:18:37.242656 10556 log.cc:927] Time spent T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 12:18:37.263129 10472 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:37.248108 10382 consensus_queue.cc:260] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:37.232110 10532 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:37.263787 10472 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.215204 10442 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:37.243809 10383 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:37.265403  6657 raft_consensus.cc:2393] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 10: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:37.265451 10383 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:37.243012  6473 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:37.266832  6581 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
I20260709 12:18:37.262251  6655 rpcz_store.cc:276] 0709 12:18:35.565976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.754440 (+1188464us) service_pool.cc:224] Handling call
0709 12:18:36.790126 (+ 35686us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:36.878905 (+ 88779us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:36.878941 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:36.879605 (+   664us) log.cc:844] Serialized 26 byte log entry
0709 12:18:37.028129 (+148524us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:37.028180 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:37.028280 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:37.222231 (+193951us) raft_consensus.cc:1685] finished
0709 12:18:37.222247 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:37.222363 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7375,"raft.queue_time_us":26760,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":108,"thread_start_us":148044,"threads_started":1,"wal-append.queue_time_us":259245}
I20260709 12:18:37.220504  6577 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
W20260709 12:18:37.232566  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 68562.9us
I20260709 12:18:37.264549 10382 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:37.251497  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1849) took 1924 ms. Trace:
I20260709 12:18:37.230821  6550 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:37.180277 10508 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:37.234701  6475 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:37.253343  7271 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38562 (request call id 651) took 1856 ms (client timeout 1832 ms). Trace:
W20260709 12:18:37.284989  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 63386.4us
W20260709 12:18:37.296806  7271 rpcz_store.cc:269] 0709 12:18:35.396850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.396972 (+   122us) service_pool.cc:224] Handling call
0709 12:18:37.253048 (+1856076us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b5400670cd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:37.253226 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3060097792}
I20260709 12:18:37.297451  6473 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:37.285641  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:37.238291 10334 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:37.210743 10507 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:37.300923  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:37.226145  7265 rpcz_store.cc:269] 0709 12:18:35.205031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.205338 (+   307us) service_pool.cc:224] Handling call
0709 12:18:37.134174 (+1928836us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b5400507dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:37.193181 (+ 59007us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":310501632}
I20260709 12:18:37.304091  7273 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:18:37.186372 10505 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:37.234493 10428 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:37.262821 10417 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.306552 10255 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:37.306710  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:37.307370  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:37.321026  6506 raft_consensus.cc:2410] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:18:37.236305 10541 raft_consensus.cc:2804] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:18:37.301780  6579 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7.
I20260709 12:18:37.310930  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:37.311087  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:37.274549  6652 rpcz_store.cc:276] 0709 12:18:35.326804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.455341 (+128537us) service_pool.cc:224] Handling call
0709 12:18:35.571457 (+116116us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:36.662348 (+1090891us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:37.251173 (+588825us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b540044a7d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:37.251428 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2399427456}
I20260709 12:18:37.314929  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:37.345183  6657 raft_consensus.cc:2393] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 11: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:37.317272  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:37.233088 10489 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:37.322482  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:37.319573  6502 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:37.323051  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:37.318096  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:37.349071  7271 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:37.308771  6583 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
I20260709 12:18:37.283742 10345 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:37.344206  6581 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 12.
I20260709 12:18:37.351954  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:37.260632 10355 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:37.352620  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:37.352182  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:37.307693 10532 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:37.355458  6576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1899) took 3070 ms. Trace:
I20260709 12:18:37.252797 10482 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:37.236931 10475 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.086s
W20260709 12:18:37.360468  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 63130us
W20260709 12:18:37.362637  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 131930us
I20260709 12:18:37.364833  6658 raft_consensus.cc:2393] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 7: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:37.325753 10417 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:37.325059 10255 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.359948  6576 rpcz_store.cc:276] 0709 12:18:34.284859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:34.315386 (+ 30527us) service_pool.cc:224] Handling call
0709 12:18:34.340183 (+ 24797us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:34.989717 (+649534us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:37.312357 (+2322640us) spinlock_profiling.cc:243] Waited 647 ms on lock 0x7b54003ab954. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:37.355232 (+ 42875us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1488619264}
I20260709 12:18:37.372795  7272 raft_consensus.cc:1240] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10. Ops: []
W20260709 12:18:37.373106  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 138494us
I20260709 12:18:37.343166 10541 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:37.382848 10541 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:37.385850  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:37.374224  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:37.375442  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:37.387758  6650 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 14: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:37.375640 10546 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:37.389406  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:37.376240 10453 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:37.348544 10473 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:37.377269 10491 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:37.377065 10355 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:37.378012 10482 consensus_queue.cc:1048] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:37.380208  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:37.380831  7273 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38558 (request call id 597) took 1727 ms (client timeout 2037 ms). Trace:
I20260709 12:18:37.386473  6581 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
I20260709 12:18:37.399857  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:37.343787 10472 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:37.355261 10507 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:37.386896  6502 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
W20260709 12:18:37.395151  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 253755us
I20260709 12:18:37.397868  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:37.404126  7271 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:37.389297  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:37.401363  6658 raft_consensus.cc:2410] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 12 because replica has last-logged OpId of term: 10 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 3.
I20260709 12:18:37.370436 10255 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:37.363714 10547 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:37.390581  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:37.405766  6652 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:18:37.352026 10383 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:37.408788  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:37.399151  7273 rpcz_store.cc:269] 0709 12:18:35.653298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.702669 (+ 49371us) service_pool.cc:224] Handling call
0709 12:18:37.380611 (+1677942us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54005de654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:37.380774 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3042021248}
I20260709 12:18:37.384769 10534 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:37.429471  6651 raft_consensus.cc:1240] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:18:37.481037  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:37.484019  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:37.476755  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:37.487174  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:37.488919  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:37.492074  6658 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10. Ops: []
W20260709 12:18:37.429369  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 58844.9us
I20260709 12:18:37.495033  7268 raft_consensus.cc:2368] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 7. Current term is 8.
W20260709 12:18:37.495918  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 113288us
I20260709 12:18:37.490185  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:37.500021  6504 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:18:37.456727  6501 raft_consensus.cc:1813] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Handling vote request from an unknown peer f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:37.502163  6501 raft_consensus.cc:2393] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 7: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:37.503242  6582 raft_consensus.cc:2393] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 10: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:18:37.477420 10562 log.cc:927] Time spent T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 12:18:37.504315  6657 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 14: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:37.492691  6649 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
W20260709 12:18:37.480649 10506 log.cc:927] Time spent T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 1.411s	user 0.000s	sys 0.006s
I20260709 12:18:37.502866  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:37.514276  7273 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:18:37.514700 10472 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:37.510409  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:37.447961 10518 consensus_meta.cc:287] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.686s	user 0.009s	sys 0.005s
I20260709 12:18:37.480459 10428 raft_consensus.cc:697] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:37.493233 10516 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:37.530649  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 166753us
W20260709 12:18:37.532670  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 138758us
I20260709 12:18:37.533280  7265 raft_consensus.cc:2410] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:18:37.472204 10397 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:37.545210  6651 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:37.550181 10547 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:37.551384  7267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38562 (request call id 646) took 2360 ms. Trace:
I20260709 12:18:37.551749 10507 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:37.551790  7267 rpcz_store.cc:276] 0709 12:18:35.190745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.190935 (+   190us) service_pool.cc:224] Handling call
0709 12:18:35.220806 (+ 29871us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:35.342812 (+122006us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:35.411297 (+ 68485us) log.cc:844] Serialized 14 byte log entry
0709 12:18:35.607459 (+196162us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:35.708377 (+100918us) log.cc:844] Serialized 26 byte log entry
0709 12:18:35.708568 (+   191us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:35.708618 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:35.708711 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:37.255951 (+1547240us) spinlock_profiling.cc:243] Waited 5.21 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:18:37.550927 (+294976us) raft_consensus.cc:1685] finished
0709 12:18:37.551031 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:37.551194 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":117931,"prepare.queue_time_us":109521,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":148003,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":64095,"spinlock_wait_cycles":11975040,"thread_start_us":294469,"threads_started":3,"wal-append.queue_time_us":472682}
I20260709 12:18:37.565079 10428 consensus_queue.cc:237] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:37.564266 10255 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:37.572299  6500 raft_consensus.cc:2393] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 12: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:18:37.572211  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 64736.5us
I20260709 12:18:37.572783 10355 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:37.570397  6550 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:37.573293  7238 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:37.576869  7270 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 15: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:37.578874  7269 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:18:37.582347  6658 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:37.583316  7274 raft_consensus.cc:2393] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 9: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
W20260709 12:18:37.583772  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 45534.7us
I20260709 12:18:37.579313  7269 pending_rounds.cc:85] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9: Aborting all ops after (but not including) 6
I20260709 12:18:37.585822  7269 pending_rounds.cc:107] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 11.7
I20260709 12:18:37.588534  7269 raft_consensus.cc:2917] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Aborting config change with OpId 11.7: Aborted: Op aborted by new leader
I20260709 12:18:37.544065 10345 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:37.590653 10304 consensus_queue.cc:1059] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 12:18:37.595506  7273 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:37.605729  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1902) took 1461 ms. Trace:
I20260709 12:18:37.606241  6578 rpcz_store.cc:276] 0709 12:18:36.144256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.200739 (+ 56483us) service_pool.cc:224] Handling call
0709 12:18:36.201190 (+   451us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:36.786126 (+584936us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:36.790827 (+  4701us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:36.867495 (+ 76668us) log.cc:844] Serialized 52 byte log entry
0709 12:18:37.046921 (+179426us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:37.054113 (+  7192us) log.cc:844] Serialized 14 byte log entry
0709 12:18:37.111271 (+ 57158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:37.132440 (+ 21169us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:37.593542 (+461102us) raft_consensus.cc:1685] finished
0709 12:18:37.593566 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:37.605493 (+ 11927us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":6965,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":2189,"raft.queue_time_us":74224,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":8746,"thread_start_us":558095,"threads_started":2,"wal-append.queue_time_us":363990}
I20260709 12:18:37.580658 10355 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:37.543634 10453 consensus_queue.cc:1059] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 5, Time since last communication: 0.051s
W20260709 12:18:37.622918  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 53311.1us
I20260709 12:18:37.531883 10518 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:37.632828  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:37.645282  6550 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:37.617583 10355 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.670810  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:37.677546  7269 raft_consensus.cc:2889] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:37.678819  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 94357.7us
I20260709 12:18:37.690172 10397 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:37.691085  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:37.691984  6655 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:37.590664 10314 consensus_queue.cc:1059] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 12:18:37.703361  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:37.714555  6654 raft_consensus.cc:2368] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 7. Current term is 8.
I20260709 12:18:37.685065 10355 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:37.717471  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:37.684571 10453 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:37.684432 10432 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:37.725627  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:37.730288  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:37.714352 10332 consensus_meta.cc:287] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.722s	user 0.006s	sys 0.008s
W20260709 12:18:37.733670  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 88502.6us
I20260709 12:18:37.734880  6550 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:37.742112  7269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:48498 (request call id 2) took 6932 ms. Trace:
I20260709 12:18:37.742589  7269 rpcz_store.cc:276] 0709 12:18:30.801468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.801650 (+   182us) service_pool.cc:224] Handling call
0709 12:18:30.905967 (+104317us) log.cc:844] Serialized 480 byte log entry
0709 12:18:37.724733 (+6818766us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1274,"raft.run_cpu_time_us":1140,"raft.run_wall_time_us":1139}
I20260709 12:18:37.754828  6502 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:37.754648  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:37.753254  6550 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:37.726540 10255 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:37.707245 10417 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:37.763797  6508 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:37.764118  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:37.749855 10547 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:37.773027  6509 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:37.778283 10258 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:37.778867 10564 log.cc:927] Time spent T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 12:18:37.756848 10546 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:37.756559 10488 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:37.789600  6504 raft_consensus.cc:2393] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 9: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:37.778877 10258 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:37.789629 10500 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.739107 10453 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:37.754635 10432 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:37.779656 10547 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:37.782239 10546 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.809542 10546 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:37.811952 10499 consensus_meta.cc:287] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.581s	user 0.005s	sys 0.004s
I20260709 12:18:37.782282 10473 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:37.804950  6503 raft_consensus.cc:1275] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:18:37.805346 10547 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:37.813194  6548 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:37.814484  6548 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:37.815928  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:37.807195  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 58172.6us
I20260709 12:18:37.816526  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:37.808692 10541 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:37.817167  6501 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:37.817859  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:37.824501  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:37.833277  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:37.836930  6576 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
I20260709 12:18:37.838379  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:37.839638  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:37.788105 10576 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:37.847026  6508 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:18:37.787884  7271 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:37.826875 10532 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:37.796976 10500 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:37.829906 10473 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:37.858937  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:37.836309 10274 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.856271  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:37.868182  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 50969.6us
I20260709 12:18:37.868675  6579 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:37.857045 10576 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:37.869705  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:37.876154  6550 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:37.877405  6509 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 14: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:37.875298 10518 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:37.883069 10255 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:37.883859 10255 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:37.884086  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:37.885468  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:37.885990 10255 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:37.881690  6508 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:18:37.899595 10385 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:37.900653  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:37.902482 10453 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:37.906016  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:37.906663  6509 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:37.908950  6653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1858) took 1182 ms. Trace:
I20260709 12:18:37.909822 10576 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:37.910458 10576 raft_consensus.cc:2804] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:37.910849 10576 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:37.911213 10576 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:37.896275 10274 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:37.915045 10274 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:37.916327 10541 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:37.917743  6509 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:37.920655  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:37.923213 10500 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:37.871001 10473 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:37.933207 10473 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:18:37.924080 10382 consensus_meta.cc:287] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.635s	user 0.010s	sys 0.003s
I20260709 12:18:37.909468  6653 rpcz_store.cc:276] 0709 12:18:36.726172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.908254 (+182082us) service_pool.cc:224] Handling call
0709 12:18:36.908668 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:37.200326 (+291658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:37.908615 (+708289us) spinlock_profiling.cc:243] Waited 284 ms on lock 0x7b54003e99d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:37.908872 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":652247808}
I20260709 12:18:37.915944  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:37.930330  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:37.925856  6650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1860) took 1199 ms. Trace:
I20260709 12:18:37.937557  6502 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:37.860298 10532 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:37.840533 10499 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:37.948058  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:37.949451  6509 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:37.975428  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:37.976570  6508 raft_consensus.cc:2393] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 7: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:37.937788  6650 rpcz_store.cc:276] 0709 12:18:36.726806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.396849 (+670043us) service_pool.cc:224] Handling call
0709 12:18:37.446887 (+ 50038us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:37.488523 (+ 41636us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:37.488569 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:37.540098 (+ 51529us) log.cc:844] Serialized 26 byte log entry
0709 12:18:37.757609 (+217511us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:37.757755 (+   146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:37.757974 (+   219us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:37.925619 (+167645us) raft_consensus.cc:1685] finished
0709 12:18:37.925638 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:37.925795 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11310,"raft.queue_time_us":40890,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":14048,"thread_start_us":158816,"threads_started":1,"wal-append.queue_time_us":309427}
I20260709 12:18:37.982122  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:37.982971  6508 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
W20260709 12:18:37.983881  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:37.985427  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 367296us
I20260709 12:18:37.986519  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:37.987340  6502 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:37.988991  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:37.948293 10532 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:37.992265  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:38.000815  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:38.001885  6508 raft_consensus.cc:2393] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 9: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:38.007529  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:38.009739  6508 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
W20260709 12:18:38.035192 10575 log.cc:927] Time spent T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:18:38.021939 10417 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:38.040726 10345 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:38.076007 10547 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:38.117555 10576 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:38.239264 10576 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:38.408318 10355 consensus_meta.cc:287] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.664s	user 0.017s	sys 0.000s
I20260709 12:18:38.410339 10355 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:38.416404 10355 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:38.473752  6393 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 3e0ab356fc074f589a6ff4f9b4996b09 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) due to error UNKNOWN_ERROR; will retry: Aborted: Op aborted by new leader
I20260709 12:18:38.510972 10128 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:38.517338 10573 log.cc:927] Time spent T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.791s	user 0.006s	sys 0.000s
W20260709 12:18:38.525687  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:38.089226 10345 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:38.575017 10345 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:38.592164 10386 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:38.592882 10386 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:38.593354 10386 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:38.595279 10386 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:38.595769 10386 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:38.596143 10386 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:38.598522 10386 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:38.601688  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:38.086696 10432 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:38.609218  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:38.610816  9875 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:38.611148  7272 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38558 (request call id 607) took 1438 ms. Trace:
I20260709 12:18:38.611356  9875 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:38.611575  7272 rpcz_store.cc:276] 0709 12:18:37.172562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.172714 (+   152us) service_pool.cc:224] Handling call
0709 12:18:37.175327 (+  2613us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:37.385180 (+209853us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:38.610928 (+1225748us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b54002e9fd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:38.611068 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":451385088}
I20260709 12:18:38.613710  9875 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:38.619975 10365 consensus_meta.cc:287] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 1.276s	user 0.017s	sys 0.001s
W20260709 12:18:38.622956  6656 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 1869) took 1891 ms (client timeout 1712 ms). Trace:
W20260709 12:18:38.623437  6656 rpcz_store.cc:269] 0709 12:18:36.731161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.804013 (+ 72852us) service_pool.cc:224] Handling call
0709 12:18:38.622683 (+1818670us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b54002a3fd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:38.622866 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":405115648}
I20260709 12:18:38.654254 10258 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:38.658915 10383 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:38.682753 10578 log.cc:927] Time spent T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.796s	user 0.004s	sys 0.002s
W20260709 12:18:38.686133  6506 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 2055) took 1306 ms (client timeout 1579 ms). Trace:
W20260709 12:18:38.686671  6506 rpcz_store.cc:269] 0709 12:18:37.379374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.392807 (+ 13433us) service_pool.cc:224] Handling call
0709 12:18:38.685874 (+1293067us) spinlock_profiling.cc:243] Waited 411 ms on lock 0x7b54003d3bd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:38.686056 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":944764800}
W20260709 12:18:38.690455  7274 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38546 (request call id 455) took 1581 ms (client timeout 1579 ms). Trace:
W20260709 12:18:38.690994  7274 rpcz_store.cc:269] 0709 12:18:37.109411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.114361 (+  4950us) service_pool.cc:224] Handling call
0709 12:18:38.690209 (+1575848us) spinlock_profiling.cc:243] Waited 453 ms on lock 0x7b54005816d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:38.690376 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1041988352}
I20260709 12:18:38.693096  6505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2046) took 1424 ms. Trace:
I20260709 12:18:38.693518  6505 rpcz_store.cc:276] 0709 12:18:37.268401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.284720 (+ 16319us) service_pool.cc:224] Handling call
0709 12:18:37.306874 (+ 22154us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:37.402254 (+ 95380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:37.402299 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:37.484542 (+ 82243us) log.cc:844] Serialized 52 byte log entry
0709 12:18:37.710978 (+226436us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:37.744964 (+ 33986us) log.cc:844] Serialized 14 byte log entry
0709 12:18:37.753916 (+  8952us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:37.754004 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:38.682002 (+927998us) spinlock_profiling.cc:243] Waited 7.18 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:18:38.692619 (+ 10617us) raft_consensus.cc:1685] finished
0709 12:18:38.692730 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:38.692916 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":169549,"prepare.queue_time_us":49474,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":102674,"raft.run_cpu_time_us":384,"raft.run_wall_time_us":35803,"spinlock_wait_cycles":16510720,"thread_start_us":165497,"threads_started":1,"wal-append.queue_time_us":309689}
I20260709 12:18:38.704505  6582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1934) took 1509 ms. Trace:
I20260709 12:18:38.705250  6582 rpcz_store.cc:276] 0709 12:18:37.195464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.195860 (+   396us) service_pool.cc:224] Handling call
0709 12:18:38.704213 (+1508353us) spinlock_profiling.cc:243] Waited 292 ms on lock 0x7b54001cc354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:38.704418 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":670703360}
I20260709 12:18:38.095279 10466 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:38.713758 10389 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:38.714530 10389 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:38.717952  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 703204us
I20260709 12:18:38.718045 10389 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:38.719504  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:38.721783 10389 raft_consensus.cc:2764] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 12:18:38.730427 10473 consensus_meta.cc:287] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.797s	user 0.016s	sys 0.000s
W20260709 12:18:38.731251  6550 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.847s	user 0.041s	sys 0.068s
I20260709 12:18:38.730934 10473 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:38.731786  6550 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.848s	user 0.042s	sys 0.068s
W20260709 12:18:38.732169  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 848459us
I20260709 12:18:38.734599  6550 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:38.735523 10466 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:38.736232 10473 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:38.736366  6550 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:38.737705  6550 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:38.738423 10314 raft_consensus.cc:3055] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:38.738550 10466 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:38.738963 10314 raft_consensus.cc:740] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:38.739456 10473 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:38.741037 10314 consensus_queue.cc:260] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:38.743558  6550 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:38.743621 10314 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:38.744519 10466 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:38.744925  6550 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:38.744987 10466 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:38.745723 10466 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:38.746899 10383 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:38.750918  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:38.751569  6650 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:38.751961  6655 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:38.752481 10500 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:38.753003 10386 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:38.753594 10386 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:38.753019 10500 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:38.760938  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:38.761915  6655 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:38.760874  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:38.767696  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:38.768301 10500 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:38.768705  6655 raft_consensus.cc:2410] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:18:38.766657  6657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 1872) took 1148 ms. Trace:
I20260709 12:18:38.769737  6657 rpcz_store.cc:276] 0709 12:18:37.618606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:38.760506 (+1141900us) service_pool.cc:224] Handling call
0709 12:18:38.766590 (+  6084us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44800}
I20260709 12:18:38.782291  6550 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:38.782241 10383 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:38.783743  6550 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:38.784188 10383 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:38.785293 10555 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:38.786638 10383 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:38.787343 10355 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:38.787388 10466 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:38.788066 10355 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:38.788564 10355 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:38.790966 10563 log.cc:927] Time spent T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 1.579s	user 0.008s	sys 0.000s
I20260709 12:18:38.789703 10500 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:38.792613 10500 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:38.793413  6575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 1905) took 1951 ms. Trace:
I20260709 12:18:38.793817  6575 rpcz_store.cc:276] 0709 12:18:36.841601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:36.843027 (+  1426us) service_pool.cc:224] Handling call
0709 12:18:36.843456 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:36.904005 (+ 60549us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:36.904029 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:36.953470 (+ 49441us) log.cc:844] Serialized 26 byte log entry
0709 12:18:37.132522 (+179052us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:37.132574 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:38.639254 (+1506680us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b540034add4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:38.639296 (+    42us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:38.782107 (+142811us) spinlock_profiling.cc:243] Waited 9.93 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:18:38.793037 (+ 10930us) raft_consensus.cc:1685] finished
0709 12:18:38.793061 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:38.793219 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16507,"raft.queue_time_us":24733,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":6893,"spinlock_wait_cycles":52022912,"thread_start_us":98686,"threads_started":1,"wal-append.queue_time_us":211576}
I20260709 12:18:38.801100 10386 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:38.802459  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:38.802070 10500 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:38.804267  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:38.795928 10383 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:38.812407  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:38.813021  6655 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:38.820461 10473 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:38.815357 10466 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:38.843843  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:18:38.849630 10491 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.967s	user 0.008s	sys 0.006s
W20260709 12:18:38.850802 10491 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.968s	user 0.009s	sys 0.006s
I20260709 12:18:38.849247 10355 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:38.851078  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:38.851826  7274 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:38.853205 10355 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:38.845387  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:38.848881  6655 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:38.854925  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:38.855460  6655 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:38.853994  6650 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:38.859659  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:38.859890  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:38.860394  7272 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:38.849979  7238 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:38.868099 10345 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:18:38.869325  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:18:38.870311  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:38.870764 10389 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:38.871212 10389 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:38.871603 10389 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:38.871649  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:38.872728  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:38.873936  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:38.875036  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:38.876143  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:38.876375  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:38.876941  6509 raft_consensus.cc:2393] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 9: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:38.877171  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:38.878207  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:38.879266  7269 raft_consensus.cc:2393] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 9: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:38.883817 10499 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:38.886576  6507 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.933s	user 0.000s	sys 0.057s
W20260709 12:18:38.886926  6507 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.936s	user 0.001s	sys 0.058s
I20260709 12:18:38.887792  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2059) took 1186 ms. Trace:
I20260709 12:18:38.104574 10546 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:38.889360  6501 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.970s	user 0.025s	sys 0.021s
W20260709 12:18:38.889673  6501 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.970s	user 0.025s	sys 0.021s
I20260709 12:18:38.890398  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2130) took 1084 ms. Trace:
I20260709 12:18:38.890594 10546 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:38.891429  7238 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:38.892299 10345 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:38.892436  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:38.892660 10547 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:38.892813 10345 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:38.893172  7266 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:38.893308 10345 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:38.888141  6507 rpcz_store.cc:276] 0709 12:18:37.701902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.702104 (+   202us) service_pool.cc:224] Handling call
0709 12:18:37.717021 (+ 14917us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:37.717207 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:37.948792 (+231585us) log.cc:844] Serialized 14 byte log entry
0709 12:18:38.887444 (+938652us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:38.887501 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:38.887559 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:38.887629 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:38.887740 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27428,"prepare.queue_time_us":85958,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"raft.queue_time_us":118,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":114,"spinlock_wait_cycles":1402379648,"thread_start_us":1126361,"threads_started":2,"wal-append.queue_time_us":153634}
I20260709 12:18:38.897966 10474 raft_consensus.cc:3055] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:38.898152 10128 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:38.898546 10474 raft_consensus.cc:740] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:38.898955 10128 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:38.901552  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:38.902192  6656 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:38.902208 10128 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:38.904659 10345 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:38.902330 10442 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:38.905646  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2111) took 3405 ms. Trace:
I20260709 12:18:38.906049 10389 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:38.935760 10389 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:38.935182  6403 catalog_manager.cc:5697] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 5 to 8, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 8 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
W20260709 12:18:38.911720  6654 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58980 (request call id 2363) took 1891 ms (client timeout 1725 ms). Trace:
W20260709 12:18:38.937215  6654 rpcz_store.cc:269] 0709 12:18:37.020081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.170637 (+150556us) service_pool.cc:224] Handling call
0709 12:18:38.911468 (+1740831us) spinlock_profiling.cc:243] Waited 509 ms on lock 0x7b54000b6654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:38.911604 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1171406080}
I20260709 12:18:38.912189  6656 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:38.907960  7274 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:38.914907 10258 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:38.916136  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:38.932525  6503 rpcz_store.cc:276] 0709 12:18:35.499767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:35.500529 (+   762us) service_pool.cc:224] Handling call
0709 12:18:35.500897 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:37.812854 (+2311957us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:38.896089 (+1083235us) spinlock_profiling.cc:243] Waited 2.3 s on lock 0x7b54003976d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:38.905564 (+  9475us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5280231296}
I20260709 12:18:38.890756  6501 rpcz_store.cc:276] 0709 12:18:37.806397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.825932 (+ 19535us) service_pool.cc:224] Handling call
0709 12:18:37.851511 (+ 25579us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:37.857885 (+  6374us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:18:37.918942 (+ 61057us) log.cc:844] Serialized 14 byte log entry
0709 12:18:38.890042 (+971100us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:38.890096 (+    54us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:38.890163 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:38.890236 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:38.890341 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":49314,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":1683,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":100,"spinlock_wait_cycles":1393652480,"thread_start_us":970634,"threads_started":1,"wal-append.queue_time_us":187461}
I20260709 12:18:38.918900 10501 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:38.899858 10474 consensus_queue.cc:260] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:38.920991  6650 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 9.
I20260709 12:18:38.921070  6550 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:38.921746  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 54903.1us
I20260709 12:18:38.927467 10488 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:38.928098 10274 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:38.951116 10274 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:38.951211 10488 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:38.950486  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:38.953179  6656 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
I20260709 12:18:38.933120  6655 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 5.
I20260709 12:18:38.929410  6475 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:38.955472  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:38.939316  7266 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:18:38.948163  7240 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:38.958410  7238 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:38.952265 10482 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:38.960011 10482 raft_consensus.cc:697] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:38.960305  6582 raft_consensus.cc:1240] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:38.960959 10482 consensus_queue.cc:237] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:38.965147  7238 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:38.964540 10501 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:38.952442 10258 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:38.966193 10128 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:38.957459 10382 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:38.967283 10382 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:38.967837 10389 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:38.968225 10258 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:38.964145 10488 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:38.968874  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:38.969578  6650 raft_consensus.cc:2393] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 9: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:38.969808 10382 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:38.970340  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:38.970865  6650 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:38.957723 10314 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:38.971659 10314 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:38.972043 10314 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:38.976706 10474 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:38.978999 10481 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:38.981366  6584 raft_consensus.cc:1240] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: []
W20260709 12:18:38.982020  6500 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 2056) took 1602 ms (client timeout 1817 ms). Trace:
I20260709 12:18:38.982486  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:38.982390  6500 rpcz_store.cc:269] 0709 12:18:37.379742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.379879 (+   137us) service_pool.cc:224] Handling call
0709 12:18:38.981864 (+1601985us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b54000f3a54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:38.981963 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":397907200}
I20260709 12:18:38.983286  6575 raft_consensus.cc:2393] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 6: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:38.983485  6576 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:38.984633  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:38.985718 10274 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:38.984265  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:38.986699  6582 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:38.981395  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:38.985301  6581 raft_consensus.cc:1240] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:38.987246  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:38.984709  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:38.987609  6658 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:38.987859  6657 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:39.004716 10580 log.cc:927] Time spent T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.867s	user 0.002s	sys 0.002s
I20260709 12:18:39.005010  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:39.006366  6577 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:18:39.008108  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1861) took 2281 ms. Trace:
I20260709 12:18:39.008661  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:39.008556  6652 rpcz_store.cc:276] 0709 12:18:36.727125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.406511 (+679386us) service_pool.cc:224] Handling call
0709 12:18:37.483495 (+ 76984us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:37.524903 (+ 41408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:37.524932 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:37.546381 (+ 21449us) log.cc:844] Serialized 26 byte log entry
0709 12:18:37.715271 (+168890us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:37.715323 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:37.715404 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:38.986392 (+1270988us) spinlock_profiling.cc:243] Waited 592 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:18:39.007776 (+ 21384us) raft_consensus.cc:1685] finished
0709 12:18:39.007802 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:39.008040 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13994,"raft.queue_time_us":12626,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":104,"spinlock_wait_cycles":1362542464,"thread_start_us":117356,"threads_started":1,"wal-append.queue_time_us":517242}
I20260709 12:18:39.009940  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:39.010110 10532 consensus_meta.cc:287] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.996s	user 0.012s	sys 0.000s
I20260709 12:18:39.010529  6575 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:39.010897  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:39.010586 10532 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:39.011301  6650 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:18:39.012040  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:38.986310  6578 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:39.004789  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:39.012218  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:39.013355  6654 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:39.013970  6650 raft_consensus.cc:2410] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:18:39.004794  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:39.017737  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:39.018890  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:39.021454  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:39.018217  6652 raft_consensus.cc:2376] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10. Re-sending same reply.
I20260709 12:18:39.022862  6658 raft_consensus.cc:2393] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 9: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:18:39.020961 10587 log.cc:927] Time spent T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.922s	user 0.005s	sys 0.001s
I20260709 12:18:39.023034  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:39.027110  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:39.025086  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:18:39.026221  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:39.030191 10345 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:39.032419  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:39.033099  6577 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:39.034387  6584 raft_consensus.cc:1240] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:39.035128 10314 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:39.035471  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:39.036640  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:39.037079  6576 raft_consensus.cc:1240] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:39.037424  6580 raft_consensus.cc:2410] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:18:39.038308  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:39.035382  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:39.041587  6584 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
W20260709 12:18:39.043103  6649 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58980 (request call id 2354) took 2025 ms (client timeout 1762 ms). Trace:
I20260709 12:18:39.043694  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:39.043591  6649 rpcz_store.cc:269] 0709 12:18:37.017189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.174504 (+157315us) service_pool.cc:224] Handling call
0709 12:18:39.042931 (+1868427us) spinlock_profiling.cc:243] Waited 277 ms on lock 0x7b540008bbd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:39.043030 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":636397952}
I20260709 12:18:39.044539  6500 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 11.
I20260709 12:18:39.044559  6652 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:39.045993 10579 log.cc:927] Time spent T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.939s	user 0.003s	sys 0.002s
I20260709 12:18:39.044685 10314 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:39.047037 10590 log.cc:927] Time spent T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.944s	user 0.002s	sys 0.004s
I20260709 12:18:39.024734 10481 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:39.053537 10481 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:39.053855  6403 catalog_manager.cc:5697] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:18:39.055842 10481 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:39.048961  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:39.056429  6658 raft_consensus.cc:2368] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 7. Current term is 8.
I20260709 12:18:39.056857  6583 raft_consensus.cc:2393] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 9: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:39.057747 10532 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:39.057756  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:39.059165  6583 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
W20260709 12:18:39.060189 10582 log.cc:927] Time spent T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.953s	user 0.001s	sys 0.004s
I20260709 12:18:39.060386  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:39.061259  6583 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:18:39.062400  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:39.051414 10491 consensus_queue.cc:1059] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:39.063619 10491 raft_consensus.cc:3055] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:39.064047 10491 raft_consensus.cc:740] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:39.065181 10491 consensus_queue.cc:260] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:39.066177  6577 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:18:39.066782 10491 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:39.067183  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:39.067452  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:39.067802  6577 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:39.068759  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:39.069358  7274 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:18:39.070122  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:39.071074  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:39.068765  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:39.072250  7272 raft_consensus.cc:2393] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 9: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:39.051956  6549 leader_election.cc:400] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 7. Current term is 8.
I20260709 12:18:39.052697 10442 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:39.049815  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:39.069303  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:39.071746  6501 raft_consensus.cc:2393] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 9: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:39.068272  7269 raft_consensus.cc:2410] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:18:39.052210  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:39.075026  6583 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:39.068135  6548 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:39.074434  6549 leader_election.cc:403] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
W20260709 12:18:39.081125  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:39.091395  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:39.055722  6403 catalog_manager.cc:5697] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 7 to 8, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 8 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:39.076411  6550 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:39.052481  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:39.092070  6502 consensus_meta.cc:287] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 1.099s	user 0.013s	sys 0.000s
I20260709 12:18:39.092572  6584 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:39.092628  6502 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:39.093679  6502 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 2142) took 1187 ms. Trace:
I20260709 12:18:39.097077  6657 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:18:39.094112  6502 rpcz_store.cc:276] 0709 12:18:37.906513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.950806 (+ 44293us) service_pool.cc:224] Handling call
0709 12:18:39.093597 (+1142791us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:39.099826  6504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 2064) took 1394 ms. Trace:
I20260709 12:18:39.100407  6504 rpcz_store.cc:276] 0709 12:18:37.705649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:37.711459 (+  5810us) service_pool.cc:224] Handling call
0709 12:18:39.099587 (+1388128us) spinlock_profiling.cc:243] Waited 55.3 ms on lock 0x7b54002c42d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:39.099752 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":127175936}
I20260709 12:18:39.089779  6623 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:39.104904  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:39.106010  7266 raft_consensus.cc:2393] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 8: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:39.106848  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:39.108810  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 56675.7us
W20260709 12:18:39.108916  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 57144.7us
I20260709 12:18:39.109586  6549 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:39.110069  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:39.110265  6575 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:39.110668  6657 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:18:39.112015  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:39.112782  7266 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:39.108182  6550 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:39.114674  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:39.115353  6655 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
W20260709 12:18:39.118096  6438 debug-util.cc:398] Leaking SignalData structure 0x7b080003f120 after lost signal to thread 5355
W20260709 12:18:39.119355  6438 debug-util.cc:398] Leaking SignalData structure 0x7b08003e7ca0 after lost signal to thread 6441
I20260709 12:18:39.120756  6550 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:39.120805  6438 debug-util.cc:398] Leaking SignalData structure 0x7b08003b42e0 after lost signal to thread 6533
W20260709 12:18:39.123144  6438 debug-util.cc:398] Leaking SignalData structure 0x7b080032fa60 after lost signal to thread 6608
W20260709 12:18:39.123489  6438 debug-util.cc:398] Leaking SignalData structure 0x7b0800296a80 after lost signal to thread 6609
I20260709 12:18:39.117405  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:39.129060  6438 debug-util.cc:398] Leaking SignalData structure 0x7b08003d2380 after lost signal to thread 6682
W20260709 12:18:39.130924  6438 debug-util.cc:398] Leaking SignalData structure 0x7b080037efe0 after lost signal to thread 7298
I20260709 12:18:39.132548  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:18:39.131337  6438 debug-util.cc:398] Leaking SignalData structure 0x7b0800364d40 after lost signal to thread 7299
W20260709 12:18:39.133388  6438 debug-util.cc:398] Leaking SignalData structure 0x7b0800151920 after lost signal to thread 10286
I20260709 12:18:39.135177  7272 raft_consensus.cc:2393] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 10: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:39.134874  6550 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:39.133497  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:39.137071  7266 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 8. Current term is 9. Ops: []
W20260709 12:18:39.133894  6438 debug-util.cc:398] Leaking SignalData structure 0x7b08000e28e0 after lost signal to thread 10295
I20260709 12:18:39.124521 10386 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:39.138774  7269 raft_consensus.cc:1240] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:39.131903  6622 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:18:39.139539  6438 debug-util.cc:398] Leaking SignalData structure 0x7b0800285da0 after lost signal to thread 10353
I20260709 12:18:39.140527  6582 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:39.143777  6577 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:39.139123  7238 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:39.147509  6438 debug-util.cc:398] Leaking SignalData structure 0x7b080038bf60 after lost signal to thread 10489
I20260709 12:18:39.147071  6579 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
W20260709 12:18:39.147913  6438 debug-util.cc:398] Leaking SignalData structure 0x7b0800378ea0 after lost signal to thread 10493
W20260709 12:18:39.145232  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 77172.1us
I20260709 12:18:39.149013  6548 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:39.150650  6438 debug-util.cc:398] Leaking SignalData structure 0x7b08002b77a0 after lost signal to thread 10498
I20260709 12:18:39.147405  6657 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:18:39.118805 10546 raft_consensus.cc:2804] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:18:39.152459  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:39.156095  7273 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:18:39.156636  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:39.173127  6650 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
W20260709 12:18:39.168638  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:39.174996  6549 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:18:39.124421 10555 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:39.107887 10532 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:18:39.181052  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 46327.2us
I20260709 12:18:39.182114  6548 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:39.125944 10355 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:39.134516 10482 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:39.179819  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:39.107901 10500 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 12:18:39.187752  6474 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:39.139925 10472 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:39.177799 10595 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:39.146438 10466 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:39.150178 10474 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.186968  6549 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:18:39.118399 10442 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:39.178470 10128 raft_consensus.cc:2764] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:18:39.173993 10499 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:39.209326  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:39.196594  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:39.209316  7272 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:39.184513 10532 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:39.179189 10555 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:39.147853  6584 raft_consensus.cc:2955] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Committing config change with OpId 7.6: config changed from index 2 to 6, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:18:39.189116 10488 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.208316 10466 raft_consensus.cc:697] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:39.185309 10481 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:39.191670 10497 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.185108 10482 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:39.182710 10386 raft_consensus.cc:697] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 8 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:39.191745 10383 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:39.214619  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:39.146191  6583 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:18:39.215018  6651 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:39.191864 10258 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.217727  6550 leader_election.cc:400] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 7. Current term is 8.
I20260709 12:18:39.217893  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:39.230875  6656 raft_consensus.cc:2393] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 7: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:39.232111  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:39.232784  6657 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:39.229068  6583 raft_consensus.cc:606] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:39.233680  6583 raft_consensus.cc:614] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:18:39.225186  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 115472us
I20260709 12:18:39.198809 10547 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:39.206593 10453 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:39.214711 10555 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:39.240569  6550 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:39.240716  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:39.241793  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:39.245610  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:39.238301  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:39.212433 10499 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:39.263759  6658 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:39.213979 10488 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:39.216526 10481 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:39.268483  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:39.292281  6651 raft_consensus.cc:2410] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:18:39.263252  6649 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:39.228379 10482 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:39.228165 10501 raft_consensus.cc:697] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 9 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:39.290815  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:39.297137  7273 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:18:39.215126 10474 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:39.281981 10365 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.003s	sys 0.014s
I20260709 12:18:39.305052  6650 raft_consensus.cc:1240] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:39.302624  6622 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:39.247346 10383 raft_consensus.cc:697] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:39.347009 10383 consensus_queue.cc:237] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:39.265550 10453 raft_consensus.cc:697] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:39.348941 10466 consensus_queue.cc:237] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:39.350211 10385 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:39.349987 10453 consensus_queue.cc:237] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:39.350843 10385 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:39.247457 10258 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:39.352720 10386 consensus_queue.cc:237] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:39.354071 10258 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:39.323455 10474 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:39.322908  6403 catalog_manager.cc:5697] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:39.325562 10442 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:39.294432 10481 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:39.328406 10365 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.003s	sys 0.014s
W20260709 12:18:39.329119  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 88618.2us
I20260709 12:18:39.369560  6550 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:39.229128 10497 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:39.268033 10499 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:39.330231 10515 raft_consensus.cc:2764] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:18:39.265704 10314 consensus_queue.cc:1059] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 12:18:39.335400 10488 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:39.301159 10595 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:39.336959 10501 consensus_queue.cc:237] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:39.343782 10382 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.361210  6657 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:39.369939  6652 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:39.377925 10609 raft_consensus.cc:2764] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 12:18:39.378679  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 170811us
I20260709 12:18:39.450259  6474 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:18:39.388545  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 203147us
I20260709 12:18:39.391201  6653 raft_consensus.cc:1240] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
W20260709 12:18:39.411386  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 229303us
I20260709 12:18:39.451201  6549 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:39.374917 10546 raft_consensus.cc:697] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:39.451471  6548 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:39.453678  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:39.456061  6549 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:39.463521  6657 raft_consensus.cc:1240] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:39.374979 10547 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:39.476430  6548 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:39.313719 10518 consensus_queue.cc:1048] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.135s
W20260709 12:18:39.479800 10589 log.cc:927] Time spent T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 1.373s	user 0.000s	sys 0.007s
I20260709 12:18:39.480140 10608 raft_consensus.cc:3055] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:18:39.480538 10604 log.cc:927] Time spent T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:18:39.400255 10541 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:39.465787 10466 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:18:39.501639  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 51439.1us
I20260709 12:18:39.403802 10595 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:39.519384  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:39.520678  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 44358.3us
I20260709 12:18:39.522945  6548 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:39.500105 10563 log.cc:927] Time spent T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 12:18:39.512100  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 209575us
I20260709 12:18:39.526547  6548 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:39.499192 10609 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:18:39.454010 10382 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:39.492797 10608 raft_consensus.cc:740] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:39.522089 10488 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:39.543787  6622 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:39.524537 10547 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:39.521646 10595 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:39.547490  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 177919us
I20260709 12:18:39.527892 10610 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:39.521341 10607 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:39.523583 10314 consensus_queue.cc:1059] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:18:39.554790  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:39.556514 10389 consensus_queue.cc:1059] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:39.548411  6403 catalog_manager.cc:5697] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 4 to 6, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 6 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:18:39.556188  7240 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:39.546386 10488 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:39.565240  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 458993us
I20260709 12:18:39.565465  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:39.579044  6577 raft_consensus.cc:2393] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 7: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:39.521051 10258 consensus_queue.cc:1059] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:18:39.559958  6622 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:39.553556  6550 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:39.594709  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:39.527521 10474 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:18:39.543180 10274 consensus_queue.cc:1059] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
W20260709 12:18:39.602566  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 49133.5us
I20260709 12:18:39.602968  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:39.603591  6501 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:39.560961 10610 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:39.609655  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:39.600215  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:39.610586  6508 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:39.610821  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:39.611482  6506 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:39.598037  6622 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:39.603873  6550 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:39.602969  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:39.609265  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 53164.8us
I20260709 12:18:39.609189  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:39.610581  6509 raft_consensus.cc:2393] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 9: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:39.613330  6584 raft_consensus.cc:2393] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 9: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:39.613500  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:39.616819  6500 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:39.601130 10518 raft_consensus.cc:2764] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:18:39.618348  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:39.580060 10488 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:39.619439  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:39.621932  6504 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:39.597445 10314 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:39.619927  6503 raft_consensus.cc:2393] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 9: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:39.580097 10607 raft_consensus.cc:697] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:39.649067  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:39.626528  6622 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:39.611976 10466 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:39.614257 10606 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:39.669665  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:39.680187 10624 raft_consensus.cc:2804] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:39.603536 10546 consensus_queue.cc:237] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:39.613677 10610 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:39.685803  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:18:39.682907  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 56456.1us
I20260709 12:18:39.683493  6584 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:18:39.687346  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:39.611949 10547 consensus_queue.cc:1048] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 12:18:39.706840  6622 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:39.670655 10613 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:39.686810 10386 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:39.572222 10608 consensus_queue.cc:260] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:39.710295  6501 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:18:39.711067  6508 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
W20260709 12:18:39.729600  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 103054us
I20260709 12:18:39.663393 10314 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:39.689575 10497 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:39.733603 10617 log.cc:927] Time spent T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
W20260709 12:18:39.734351 10620 log.cc:927] Time spent T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
W20260709 12:18:39.742677  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 123840us
I20260709 12:18:39.745425  6506 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:39.708249 10274 raft_consensus.cc:3055] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:39.720575 10515 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:39.724447 10474 raft_consensus.cc:3055] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:39.756721  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:39.724615 10615 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:18:39.739640 10601 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.198s	user 0.000s	sys 0.000s
W20260709 12:18:39.763021  6473 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.586s (SENT) [suppressed 31 similar messages]
I20260709 12:18:39.769291  6475 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:39.753188 10274 raft_consensus.cc:740] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:39.746716 10614 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:18:39.775889  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:39.774006  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:39.759033 10474 raft_consensus.cc:740] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:39.785144 10474 consensus_queue.cc:260] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:39.786623 10474 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:39.788162  6658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2375) took 1007 ms. Trace:
I20260709 12:18:39.788580  6658 rpcz_store.cc:276] 0709 12:18:38.780280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.305579 (+525299us) service_pool.cc:224] Handling call
0709 12:18:39.330661 (+ 25082us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:39.478091 (+147430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:39.478130 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:39.523146 (+ 45016us) log.cc:844] Serialized 26 byte log entry
0709 12:18:39.651146 (+128000us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:39.692718 (+ 41572us) log.cc:844] Serialized 14 byte log entry
0709 12:18:39.761372 (+ 68654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:39.761475 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:39.787946 (+ 26471us) raft_consensus.cc:1685] finished
0709 12:18:39.787966 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:39.788100 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15639,"prepare.queue_time_us":41369,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":2558,"raft.queue_time_us":46725,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":13938,"thread_start_us":301981,"threads_started":3,"wal-append.queue_time_us":189873}
W20260709 12:18:39.795053 10619 log.cc:927] Time spent T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
I20260709 12:18:39.802772  6500 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
W20260709 12:18:39.785771 10622 log.cc:927] Time spent T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 12:18:39.752748 10497 consensus_queue.cc:1059] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 12:18:39.839715  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:39.843030  6578 raft_consensus.cc:2393] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 9: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:39.781656 10382 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:39.852988 10627 consensus_queue.cc:1059] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:18:39.854020  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 147249us
W20260709 12:18:39.844900  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:39.856249 10612 log.cc:927] Time spent T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.263s	user 0.000s	sys 0.001s
I20260709 12:18:39.857475 10532 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:39.798766 10614 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:39.878057  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:39.867337  6504 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
W20260709 12:18:39.884028  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 113233us
I20260709 12:18:39.870796  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:39.896625  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:39.903358  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:39.902107  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:39.798261 10274 consensus_queue.cc:260] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:39.773257 10615 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:39.896672 10614 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:39.914423  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:39.916879 10615 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:39.896064 10500 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:18:39.921669  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:39.922501  6575 raft_consensus.cc:2393] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 7: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:39.923333  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:39.924722  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:39.925266  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:39.764067 10607 consensus_queue.cc:237] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:39.942754  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:39.942950  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:39.943483  7274 raft_consensus.cc:2376] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12. Re-sending same reply.
I20260709 12:18:39.945339  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:39.913447 10518 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:39.950539  7238 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:39.933650 10382 raft_consensus.cc:3055] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:39.957391  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:39.953277 10501 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:39.961275 10636 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:39.961685 10345 raft_consensus.cc:3055] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:18:39.963045 10128 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:39.963442 10488 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:39.955376 10274 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:39.959355  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:39.980232  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:39.981544  6581 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 9.
I20260709 12:18:39.940201 10634 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:39.981423  6622 connection.cc:256] RPC call timeout handler was delayed by -2.1628961119999985s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1882s for a response
I20260709 12:18:39.987545  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:39.996507 10634 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:39.953450 10453 raft_consensus.cc:3055] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:39.988701  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:39.993410  7270 raft_consensus.cc:2376] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10. Re-sending same reply.
I20260709 12:18:39.966251 10382 raft_consensus.cc:740] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:18:40.004972  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 120117us
W20260709 12:18:39.994491  6622 connection.cc:256] RPC call timeout handler was delayed by -1.9999343849999605s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1527s for a response
I20260709 12:18:39.983829 10636 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:40.003925  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 44647.2us
I20260709 12:18:39.986145 10345 raft_consensus.cc:740] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
W20260709 12:18:40.006361  6622 connection.cc:256] RPC call timeout handler was delayed by -1.9054343849998077s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1632s for a response
I20260709 12:18:40.010255  7238 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:40.005719  6475 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:39.992100 10518 raft_consensus.cc:697] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:18:40.011725  6622 connection.cc:256] RPC call timeout handler was delayed by -1.8973343850000219s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16410000000000002s for a response
W20260709 12:18:40.015264  6622 connection.cc:256] RPC call timeout handler was delayed by -1.5760961119999592s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2534s for a response
W20260709 12:18:40.015730  6622 connection.cc:256] RPC call timeout handler was delayed by -1.4905343849998189s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2093s for a response
I20260709 12:18:39.965379 10610 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.002856 10453 raft_consensus.cc:740] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:18:40.016126  6622 connection.cc:256] RPC call timeout handler was delayed by -1.433615319999717s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17930000000000001s for a response
I20260709 12:18:40.022847  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:40.008752 10501 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:40.023845  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:40.023541  6622 connection.cc:256] RPC call timeout handler was delayed by -1.4302343849999488s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21600000000000003s for a response
I20260709 12:18:40.012600 10314 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:40.027263 10595 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:40.032815  6583 raft_consensus.cc:1240] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:18:40.032740  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:40.037019  6506 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:40.011880 10636 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:40.018229 10555 consensus_meta.cc:287] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.714s	user 0.005s	sys 0.006s
I20260709 12:18:40.043239  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:40.031836  6622 connection.cc:256] RPC call timeout handler was delayed by -1.4268242229998123s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1564s for a response
I20260709 12:18:40.034565  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:40.045825  6622 connection.cc:256] RPC call timeout handler was delayed by -1.3341242229998898s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16670000000000001s for a response
I20260709 12:18:40.046135  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:40.046589  6622 connection.cc:256] RPC call timeout handler was delayed by -1.2703090069999234s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1817s for a response
I20260709 12:18:40.046830  6505 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
W20260709 12:18:40.047151  6622 connection.cc:256] RPC call timeout handler was delayed by -1.2374153199998545s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20110000000000003s for a response
I20260709 12:18:40.045874  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:40.047861  6622 connection.cc:256] RPC call timeout handler was delayed by -1.1346028010000282s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1552s for a response
I20260709 12:18:40.037158  6651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2379) took 1255 ms. Trace:
I20260709 12:18:40.020462 10128 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:40.051153 10631 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:40.052387 10634 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:40.052772  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:40.054669 10258 raft_consensus.cc:3055] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:40.055171 10258 raft_consensus.cc:740] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:18:40.059508  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 53867.2us
I20260709 12:18:40.060608  6475 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:40.061092  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 457246us
I20260709 12:18:40.061728  6475 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:40.062688  6550 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:40.063660  6550 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:40.063915 10635 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:40.064452 10635 raft_consensus.cc:697] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:40.065027 10621 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:40.065486 10621 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:40.065789 10641 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:40.066690 10641 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:40.066895 10472 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:40.067296 10641 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:40.065411 10635 consensus_queue.cc:237] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.067548 10472 raft_consensus.cc:697] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:40.068545 10547 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:40.069208 10547 raft_consensus.cc:697] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:40.068642 10518 consensus_queue.cc:237] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.069742 10634 consensus_queue.cc:1059] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:40.070238 10547 consensus_queue.cc:237] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.072958 10638 consensus_queue.cc:1059] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:18:40.073684 10499 consensus_meta.cc:287] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.620s	user 0.006s	sys 0.004s
I20260709 12:18:40.066190 10624 raft_consensus.cc:697] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:40.075456 10652 raft_consensus.cc:3055] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:40.075217 10624 consensus_queue.cc:237] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:40.075881 10652 raft_consensus.cc:740] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:40.076766 10652 consensus_queue.cc:260] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.078120 10652 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:40.080883  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:40.081502 10488 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:40.065923 10474 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:40.082293 10595 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:40.082485 10474 raft_consensus.cc:697] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:40.074002  6473 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:40.083233  6579 raft_consensus.cc:2368] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 8. Current term is 9.
I20260709 12:18:40.083593 10651 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:40.035492  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:40.084270 10651 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:40.081432  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:40.082386  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:40.085688  6576 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 8: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:40.083489 10474 consensus_queue.cc:237] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.086509  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:40.087051 10385 consensus_meta.cc:287] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.735s	user 0.009s	sys 0.002s
I20260709 12:18:40.085492  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:40.084719 10651 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:40.074117 10638 consensus_queue.cc:1059] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 12:18:40.092163 10638 raft_consensus.cc:3055] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:40.092631 10638 raft_consensus.cc:740] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:40.049487 10555 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:40.093730  6649 raft_consensus.cc:1240] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:40.094372 10638 consensus_queue.cc:260] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.094971 10555 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:40.034565 10382 consensus_queue.cc:260] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.066043 10621 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:40.088117  6473 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:40.048318  6651 rpcz_store.cc:276] 0709 12:18:38.781535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.331370 (+549835us) service_pool.cc:224] Handling call
0709 12:18:39.331787 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:39.379817 (+ 48030us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:39.379871 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:39.380520 (+   649us) log.cc:844] Serialized 26 byte log entry
0709 12:18:39.500126 (+119606us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:39.500166 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:39.500233 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:40.029972 (+529739us) spinlock_profiling.cc:243] Waited 3.61 ms on lock 0x7b2400259088. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872c229c4 00007f2872c20acb 00007f2872ddbdc9 00007f2872ddba55 00007f2872ddafb0 00007f2875a51488 00007f2875a51be8 00007f2875a29398 00007f2875a2476a 00007f2875a23bb6
0709 12:18:40.031003 (+  1031us) raft_consensus.cc:1685] finished
0709 12:18:40.031024 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:40.037094 (+  6070us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23015,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":82,"spinlock_wait_cycles":8302848,"thread_start_us":166660,"threads_started":2,"wal-append.queue_time_us":168963}
I20260709 12:18:40.087144  6576 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:40.097206  6501 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:40.084758  6508 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:40.089664  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:40.062772  6475 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:40.117259  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 54536.3us
W20260709 12:18:40.083288 10482 consensus_meta.cc:287] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.742s	user 0.010s	sys 0.000s
I20260709 12:18:40.117605  6473 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:40.131707  6473 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:40.119488  6475 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:40.095634 10638 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:40.138804  6622 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:40.139583  6475 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:40.136698  6473 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:40.138652  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:40.141031  6576 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:18:40.085608 10466 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:40.143646  7238 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:40.088083 10385 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:40.141975 10627 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:40.144712  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:40.144922 10650 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:40.147706 10650 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:40.148087 10650 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:40.056245 10258 consensus_queue.cc:260] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:40.136236 10314 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:40.150836 10314 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:40.151290 10314 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:40.134415 10482 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:40.154644 10482 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:40.155411  6580 raft_consensus.cc:1240] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:40.156805 10649 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:40.157269 10649 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:40.157627 10649 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:40.158915 10385 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:40.159552 10648 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:40.159795  7273 raft_consensus.cc:1240] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:18:40.161415 10608 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:40.162533 10481 consensus_meta.cc:287] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.713s	user 0.012s	sys 0.000s
I20260709 12:18:40.162997 10481 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.164055  6655 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:40.164763 10472 consensus_queue.cc:237] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.167838  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:40.169612  6581 raft_consensus.cc:2393] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 9: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:40.169590  7238 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:40.174762  6655 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:40.146989  6576 raft_consensus.cc:2376] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12. Re-sending same reply.
I20260709 12:18:40.133581 10643 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:40.136850 10488 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:40.178067  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:40.181696  6576 raft_consensus.cc:2393] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 6: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:40.074080 10499 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:40.178128 10355 consensus_meta.cc:287] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.989s	user 0.012s	sys 0.000s
I20260709 12:18:40.193365  6501 raft_consensus.cc:1240] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:40.197317 10481 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:40.197531  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:40.182818 10643 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:40.199393 10610 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:40.104766 10614 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.160095 10648 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:40.178581 10606 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:40.206058  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:40.194340  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:40.222483  6581 raft_consensus.cc:2376] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9. Re-sending same reply.
I20260709 12:18:40.186264 10128 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:40.218305  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:40.184377 10555 raft_consensus.cc:2764] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:18:40.181370 10488 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:40.200567 10355 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 7. Current term is 8.
W20260709 12:18:40.225116  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 86396.5us
I20260709 12:18:40.228086  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:40.215075 10643 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:40.233789  6584 raft_consensus.cc:606] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:40.176090 10651 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.145489 10345 consensus_queue.cc:260] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:40.243983  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 180367us
I20260709 12:18:40.244871  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:40.246162  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:40.247504  6506 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:40.247254  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:40.248893  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:40.249401  6581 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:40.249323  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 72290.9us
I20260709 12:18:40.244263 10627 raft_consensus.cc:697] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:40.253177 10627 consensus_queue.cc:237] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:40.226915 10482 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.228468 10658 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.267041  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:40.278123  6577 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:18:40.234947 10488 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:40.267203 10607 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.280483 10642 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:40.288054  6473 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:40.289113  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:40.285861 10636 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.292476 10607 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.293457  6582 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 14: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:40.295228 10607 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:40.283514 10481 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:40.297648 10481 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:40.298187 10481 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:40.279142 10482 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.303344 10636 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:40.267196 10621 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.305650 10621 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:40.305863 10499 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:40.303396 10614 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:40.308552  6473 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:40.310681  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:40.317598  6576 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:18:40.317967  6583 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:40.315462  6581 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 13.
I20260709 12:18:40.313583  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:40.276667 10658 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.321466  7272 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 9 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 8. Current term is 9. Ops: []
W20260709 12:18:40.315766  7271 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38546 (request call id 465) took 1249 ms (client timeout 1579 ms). Trace:
W20260709 12:18:40.328899  7271 rpcz_store.cc:269] 0709 12:18:39.066822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.069057 (+  2235us) service_pool.cc:224] Handling call
0709 12:18:40.315557 (+1246500us) spinlock_profiling.cc:243] Waited 714 ms on lock 0x7b54005816d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:40.315708 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1642547968}
I20260709 12:18:40.320411  6575 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
W20260709 12:18:40.332511  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 152904us
I20260709 12:18:40.333279 10642 raft_consensus.cc:697] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:18:40.336633 10656 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:18:40.336717 10642 consensus_queue.cc:237] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:40.314762 10613 consensus_meta.cc:287] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.578s	user 0.008s	sys 0.004s
I20260709 12:18:40.338071 10613 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:40.313608 10648 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:40.334304 10658 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:40.334753  6473 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:40.313891 10650 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.266959 10555 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 12:18:40.339859  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:40.340703  6656 raft_consensus.cc:2393] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 11: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:40.341058 10650 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:40.341163 10474 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:40.341950  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:40.342209  6651 raft_consensus.cc:1275] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:40.335484 10652 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:40.347846  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:40.332253  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:40.355412  6622 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:40.357106  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:40.358215  6473 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:40.360040  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:40.329623 10614 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:40.308585 10345 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:18:40.361706  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:40.362352  6653 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:18:40.362421 10631 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:40.362970 10631 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:40.363314  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:40.364351  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:40.314458 10453 consensus_queue.cc:260] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.365267  7273 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 15: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:40.365427 10453 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:40.366533  6651 raft_consensus.cc:1275] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:40.366878  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:40.368043  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:40.368803  6657 raft_consensus.cc:2393] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 11: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:40.369079 10636 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:40.369490 10636 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:40.369881 10636 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:40.369486  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:40.371802  6652 raft_consensus.cc:2368] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11.
I20260709 12:18:40.372720  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:40.373443  6505 raft_consensus.cc:2368] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 8. Current term is 9.
I20260709 12:18:40.374917  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:40.375082 10607 consensus_queue.cc:1048] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:40.376287  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:40.379034  6548 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:40.381750  6650 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:40.384189  6473 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:40.385805  6550 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:40.385893  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:40.323755 10274 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:40.304194 10482 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:40.388188  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:40.396003  6650 raft_consensus.cc:2376] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9. Re-sending same reply.
I20260709 12:18:40.386650  6651 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:18:40.393283 10474 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:40.393687 10627 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.397002  6548 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:40.394104 10663 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.397763  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:40.398566  6550 leader_election.cc:400] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11.
I20260709 12:18:40.398986  6550 leader_election.cc:403] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:18:40.399151  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:40.400547  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:40.400816  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:40.401063  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:40.400987  6502 raft_consensus.cc:3055] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:40.401504  6583 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:40.401899  6502 raft_consensus.cc:740] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:40.334336 10651 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:40.402637  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:40.403378  6650 raft_consensus.cc:2393] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 11: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:40.401242  6651 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:18:40.405224  6653 raft_consensus.cc:2393] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 11: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:40.405552  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:40.407096  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:40.407274  7238 leader_election.cc:400] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 8. Current term is 9.
I20260709 12:18:40.407634  7238 leader_election.cc:403] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:18:40.405052 10621 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:40.411224  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:40.397675 10466 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:18:40.413000 10666 log.cc:927] Time spent T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:18:40.413174 10466 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:40.413587 10466 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:40.310263 10314 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:40.415715 10314 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:40.382787 10555 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:40.416484 10555 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:40.416852 10555 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:40.418464  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:40.419081  6579 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:40.419317  7272 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38558 (request call id 619) took 1125 ms. Trace:
I20260709 12:18:40.419898  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:40.420889  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1883) took 1542 ms. Trace:
I20260709 12:18:40.421267  6649 rpcz_store.cc:276] 0709 12:18:38.878507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.348443 (+469936us) service_pool.cc:224] Handling call
0709 12:18:39.348881 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:40.094128 (+745247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:40.420285 (+326157us) spinlock_profiling.cc:243] Waited 745 ms on lock 0x7b5400151154. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:40.420829 (+   544us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1712502016}
I20260709 12:18:40.419749  7272 rpcz_store.cc:276] 0709 12:18:39.294102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.299946 (+  5844us) service_pool.cc:224] Handling call
0709 12:18:40.315863 (+1015917us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:40.332996 (+ 17133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:40.419142 (+ 86146us) spinlock_profiling.cc:243] Waited 957 ms on lock 0x7b54005816d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:40.419263 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2201389440}
I20260709 12:18:40.412971  6653 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:40.423077  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:40.423756  6582 raft_consensus.cc:2393] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 9: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:40.424539  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:40.425465  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:40.426385  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:40.427085  6582 raft_consensus.cc:2410] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:18:40.427779  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:40.398496 10274 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:40.428655 10274 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:40.428704  6500 raft_consensus.cc:1240] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:40.430811  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:40.431504  7269 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:40.386704 10629 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:40.432531 10615 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:40.432871  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 1897) took 1327 ms. Trace:
I20260709 12:18:40.432957 10615 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:40.431381  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:40.411113 10660 consensus_queue.cc:1048] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:40.452109  6507 raft_consensus.cc:2393] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 9: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:40.415350 10488 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.435681  6548 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:40.423199 10641 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.439368  6655 rpcz_store.cc:276] 0709 12:18:39.105721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.199307 (+ 93586us) service_pool.cc:224] Handling call
0709 12:18:40.432720 (+1233413us) spinlock_profiling.cc:243] Waited 935 ms on lock 0x7b540044a2d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:40.432818 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2150459648}
I20260709 12:18:40.441213  6579 raft_consensus.cc:2393] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 9: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:40.449105  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:40.395145 10644 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:18:40.462919  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 105893us
I20260709 12:18:40.463279  6548 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:40.463717  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:40.464480  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:40.465004  6548 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:40.465131  6622 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
W20260709 12:18:40.453646  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 55193.5us
W20260709 12:18:40.436851  6393 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 3e0ab356fc074f589a6ff4f9b4996b09 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 20a202c76244485fbc8f1610473db8b9 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:18:40.429272 10466 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.468168  6548 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:40.433642 10128 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:40.377138 10608 consensus_queue.cc:1175] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "496c7c3792cd4587b4dbeba435f4be1e" responder_term: 10 status { last_received { term: 10 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 10 index: 7 } }
I20260709 12:18:40.476099 10472 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:40.481518  6650 raft_consensus.cc:1275] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:18:40.451685 10615 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:40.398244 10474 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:40.402782  6502 consensus_queue.cc:260] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:40.468681  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.487851  6502 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:40.475004  6500 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:18:40.477596  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:40.406632 10663 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:40.457589 10629 consensus_queue.cc:1059] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:40.458058  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:40.522293  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:40.483508  6549 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:40.481595  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:40.492626  6623 leader_election.cc:336] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Timed out: RequestConsensusVote RPC to 127.5.58.4:44857 timed out after 1.579s (SENT)
I20260709 12:18:40.523317  6583 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:18:40.455420 10482 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:40.523190  6507 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:18:40.401580 10627 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.459059 10607 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:40.514348  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:40.513756 10624 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:18:40.484787  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:40.512985  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 44889.8us
I20260709 12:18:40.523743  6623 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:40.515497  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.521894  6503 raft_consensus.cc:1240] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:18:40.467674 10644 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:40.533634  6579 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
W20260709 12:18:40.527922  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 44473.2us
I20260709 12:18:40.537277  6548 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:40.530156  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:40.538050  6583 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:40.530864  6582 raft_consensus.cc:2393] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 12: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:18:40.531149  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.534209  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:40.483060 10621 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:40.544395  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 51819.3us
I20260709 12:18:40.514034  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:40.468252 10658 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:40.515306 10610 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.576146 10610 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.484845 10472 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:40.577572 10389 raft_consensus.cc:3055] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:40.577864 10472 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:40.578044 10389 raft_consensus.cc:740] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:40.484804 10386 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:40.579211 10386 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:40.578977 10389 consensus_queue.cc:260] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:40.521907 10474 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:40.521556 10488 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:40.617173  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.616883  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:40.624663  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:40.555207 10663 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:40.556218  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 71517.2us
I20260709 12:18:40.556795 10500 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:40.557370 10623 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:40.558362  6505 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:40.530277 10546 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:40.559114  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:40.530018 10647 raft_consensus.cc:3055] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:40.561681 10518 raft_consensus.cc:2764] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:18:40.563915  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:40.651237  6657 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:40.651355  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:40.565075  6583 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:18:40.529982 10473 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:40.567682 10481 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.658190 10647 raft_consensus.cc:740] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:40.659138 10647 consensus_queue.cc:260] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.659763 10481 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:40.660362 10647 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:40.660857  6654 raft_consensus.cc:2393] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 13: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:40.555207 10466 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:40.664561 10623 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.666299  6653 raft_consensus.cc:1275] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:40.666917 10623 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:40.668696  6649 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:18:40.669605  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:40.670588  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:40.671135  6475 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:40.514436 10671 consensus_queue.cc:1175] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f2b06aebe97249a4877d798b8118ece4" responder_term: 9 status { last_received { term: 7 index: 8 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 8. Current term is 9. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:18:40.543630 10644 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:40.672376  6475 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:40.675735 10664 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:40.677960 10664 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:40.678162  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:40.677450 10663 consensus_queue.cc:1059] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 12:18:40.679605 10641 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:40.680606 10638 raft_consensus.cc:3055] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:40.681133 10638 raft_consensus.cc:740] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:40.682297 10638 consensus_queue.cc:260] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:40.683776 10638 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:40.530397 10652 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.685019 10652 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.687150 10652 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:40.550797 10595 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:40.517062 10606 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:40.688320 10595 raft_consensus.cc:697] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:40.688697 10606 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:40.689231 10595 consensus_queue.cc:237] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.691082 10606 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:40.621882  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:40.609370 10627 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:40.675781 10671 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:40.624001  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:40.559221 10670 log.cc:927] Time spent T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8: Append to log took a long time: real 0.101s	user 0.002s	sys 0.000s
I20260709 12:18:40.646564  6650 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:18:40.580111 10386 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:40.656435  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:40.656400  6576 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:40.678386 10664 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:40.652266 10614 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:18:40.741551  6657 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:18:40.749850  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:40.623243 10610 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:40.699826  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54204 (request call id 504) took 1013 ms. Trace:
I20260709 12:18:40.602594 10649 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:40.768679  6577 rpcz_store.cc:276] 0709 12:18:39.686356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.687065 (+   709us) service_pool.cc:224] Handling call
0709 12:18:40.699671 (+1012606us) spinlock_profiling.cc:243] Waited 580 ms on lock 0x7b540034b7d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:40.699768 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1333775616}
I20260709 12:18:40.769408  6509 raft_consensus.cc:1240] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:40.702553  6649 raft_consensus.cc:1275] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:18:40.671132  6658 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:40.771584 10385 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:40.772984  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:40.771536 10649 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:40.714927  7273 raft_consensus.cc:1240] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:40.715423  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:40.714048  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:40.706408  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:40.726125  6503 raft_consensus.cc:1275] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:40.716130  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:40.716461  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:40.724300  6473 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:18:40.778995  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 54785.3us
I20260709 12:18:40.780810  6473 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:40.721589 10382 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:40.696324  6583 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:18:40.719511  6575 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:18:40.739987  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:40.792735  6507 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:18:40.739987  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:40.602589 10128 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 8. Current term is 9.
I20260709 12:18:40.630304 10258 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:40.750566  6506 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
I20260709 12:18:40.593086  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:40.755954  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:53972 (request call id 22) took 1067 ms. Trace:
I20260709 12:18:40.600189 10643 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:40.759778  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:40.812991  6576 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:18:40.531785 10609 consensus_queue.cc:1048] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.210s
I20260709 12:18:40.765820 10474 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:40.781980  6473 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:40.539944  6548 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:40.810161  6584 rpcz_store.cc:276] 0709 12:18:39.682662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.683383 (+   721us) service_pool.cc:224] Handling call
0709 12:18:40.619438 (+936055us) spinlock_profiling.cc:243] Waited 2.81 ms on lock 0x7b4c00520100. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874155a20 00007f2874166a29 00007f287412c644 00007f2874153f2d 00007f2874153b93 00007f2875e83c92 00007f2875e8384a 00007f287ae5160a 00007f28743d77d4 00007f28743d7762 00007f28743d7691
0709 12:18:40.750360 (+130922us) spinlock_profiling.cc:243] Waited 543 ms on lock 0x7b540032a354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e83853 00007f287ae5160a 00007f28743d77d4 00007f28743d7762 00007f28743d7691 00007f28743d760c 00007f28743d68c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:40.750443 (+    83us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1256114688}
I20260709 12:18:40.793915  6650 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 9.
W20260709 12:18:40.815204  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 275321us
I20260709 12:18:40.635301 10274 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.803015  7266 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:18:40.702641 10389 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:40.835000  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:40.719723 10555 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.776757 10613 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:40.834028  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:40.848671  6652 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:18:40.803382 10671 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:40.852473  6508 raft_consensus.cc:1240] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:18:40.832909  6548 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:40.837461  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 55542.2us
I20260709 12:18:40.749255 10472 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.855911  6473 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:40.876626  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 154080us
I20260709 12:18:40.901449  6550 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:40.835548 10610 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:40.884158  6656 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:40.910614  6579 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:40.889294 10621 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:18:40.929023 10631 consensus_meta.cc:287] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.566s	user 0.013s	sys 0.000s
I20260709 12:18:40.894709 10499 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:40.839825 10638 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:40.853706 10547 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 12:18:40.860885 10671 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:40.807319 10649 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:40.941283 10488 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:40.984565  7240 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:40.898890  6658 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
W20260709 12:18:40.934162  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 78328.5us
I20260709 12:18:40.910713  6654 raft_consensus.cc:1275] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:40.942715 10491 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:40.937844  6550 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:40.944480 10128 consensus_queue.cc:1059] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:40.948069  7272 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:40.973743 10385 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:40.999433  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 61718us
W20260709 12:18:40.974691 10636 consensus_meta.cc:287] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.604s	user 0.009s	sys 0.002s
I20260709 12:18:40.970947 10610 raft_consensus.cc:697] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 7 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:40.979856 10671 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:40.993963  7273 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:18:41.009552  7240 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:40.885725  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 52943us
I20260709 12:18:41.007627  6473 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:41.002584 10621 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:41.012125  6548 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:40.997426 10491 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:40.994330 10499 raft_consensus.cc:697] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:18:41.016916  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 282744us
I20260709 12:18:41.045264  6475 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:41.047272  6473 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:40.953434 10644 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:40.972295 10631 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:41.018844 10488 raft_consensus.cc:697] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:18:41.084971 10674 log.cc:927] Time spent T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
I20260709 12:18:41.013656 10474 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:41.193361 10615 consensus_meta.cc:287] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.660s	user 0.005s	sys 0.005s
I20260709 12:18:41.193912 10615 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:41.072027 10491 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:41.072706 10128 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:41.200743 10128 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:41.201210 10128 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:41.202873 10644 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:41.204005 10675 log.cc:927] Time spent T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.233s	user 0.000s	sys 0.001s
I20260709 12:18:41.008882 10501 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:41.067900 10642 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:41.006494 10614 consensus_queue.cc:1059] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.148s
I20260709 12:18:41.006367 10385 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:41.007179 10636 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:41.185642  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 138443us
I20260709 12:18:41.186906 10653 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:41.191483 10631 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:41.207916 10643 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:41.214337  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 560973us
W20260709 12:18:41.216845  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 204780us
I20260709 12:18:41.167768 10274 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:41.220489  6502 raft_consensus.cc:1275] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 12:18:41.255596  6502 pending_rounds.cc:85] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 8
I20260709 12:18:41.255863  6502 pending_rounds.cc:107] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:18:41.256228  6502 raft_consensus.cc:2889] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:41.060247 10466 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.149s
W20260709 12:18:41.216691  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 169989us
I20260709 12:18:41.257848  6475 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:41.060251 10621 raft_consensus.cc:697] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:18:41.184999  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:41.208045 10614 raft_consensus.cc:3055] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:41.251282  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:41.270476  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:41.251511  6548 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:41.172616 10676 log.cc:927] Time spent T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.372s	user 0.002s	sys 0.000s
I20260709 12:18:41.273097  6548 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:41.275300  6650 raft_consensus.cc:1240] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:41.215191 10608 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:41.290258  6622 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:41.250010 10609 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:41.216641 10642 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:41.292483  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:41.246507 10653 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:41.245945 10615 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:41.306690  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 297026us
I20260709 12:18:41.277169 10606 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.285173 10482 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:18:41.288046 10607 consensus_meta.cc:287] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.743s	user 0.011s	sys 0.000s
I20260709 12:18:41.304760 10274 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:41.307948 10442 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:18:41.348825  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 90703.6us
I20260709 12:18:41.266491 10614 raft_consensus.cc:740] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:41.277436 10555 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:41.330260 10609 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:41.335230 10664 consensus_meta.cc:287] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.569s	user 0.012s	sys 0.000s
I20260709 12:18:41.367265 10609 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:41.367376 10664 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:41.367238 10385 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:41.368892 10385 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:41.369527 10664 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:41.369372 10385 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:41.370200 10664 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:41.335071 10642 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:41.371542 10385 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:41.357156 10606 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:41.375294 10606 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:41.353577 10442 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:41.376699 10442 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:41.379274 10642 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:41.381021 10614 consensus_queue.cc:260] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:41.382609 10614 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:41.383853 10473 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:18:41.384797 10681 log.cc:927] Time spent T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.374s	user 0.000s	sys 0.001s
I20260709 12:18:41.255445 10547 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.130s
I20260709 12:18:41.383788 10609 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:41.310614 10472 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:41.386718 10555 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:18:41.387096  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 97200.8us
I20260709 12:18:41.387310 10609 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:41.249985 10610 consensus_queue.cc:237] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:41.388882  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:41.389623 10555 raft_consensus.cc:2764] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:18:41.306790 10680 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:41.284150 10651 consensus_queue.cc:1059] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
W20260709 12:18:41.365784 10258 consensus_meta.cc:287] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.553s	user 0.010s	sys 0.002s
W20260709 12:18:41.365964 10647 consensus_meta.cc:287] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.705s	user 0.006s	sys 0.003s
I20260709 12:18:41.396636  6504 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:18:41.345842 10499 consensus_queue.cc:237] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:41.391804  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:41.397712  6584 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
I20260709 12:18:41.266803 10474 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 12:18:41.419358 10671 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:41.392688  6473 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:41.393880  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:41.421017  6500 raft_consensus.cc:2368] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11.
I20260709 12:18:41.338593 10653 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:41.396044 10651 raft_consensus.cc:3055] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:41.396271  6579 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:41.346206 10488 consensus_queue.cc:237] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:41.348315 10607 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11.
I20260709 12:18:41.424106  6473 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:41.425719 10651 raft_consensus.cc:740] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:41.433260 10671 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:41.433624  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:41.427938 10474 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:41.429448 10680 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:18:41.433523  6580 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:41.435415 10128 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:41.436797 10274 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:41.430148  6622 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:41.448189  6500 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 2167) took 1041 ms. Trace:
I20260709 12:18:41.453053  6500 rpcz_store.cc:276] 0709 12:18:40.406460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.592676 (+186216us) service_pool.cc:224] Handling call
0709 12:18:41.447984 (+855308us) spinlock_profiling.cc:243] Waited 611 ms on lock 0x7b54002c5bd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:41.448120 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1404645760}
W20260709 12:18:41.470225  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 212380us
I20260709 12:18:41.476593 10649 consensus_queue.cc:1048] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:41.477921 10644 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:41.478469  6475 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:41.479673 10635 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:18:41.483228  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 54452.2us
I20260709 12:18:41.444579 10647 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.484254  6622 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:41.449359 10474 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:41.492309  6651 raft_consensus.cc:1275] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:41.494575  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:41.499082  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:41.500133  6650 raft_consensus.cc:2368] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 9. Current term is 10.
I20260709 12:18:41.500617  6475 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:41.460494 10274 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:41.482802 10389 consensus_meta.cc:287] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.642s	user 0.004s	sys 0.006s
I20260709 12:18:41.398497 10621 consensus_queue.cc:237] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:41.508800  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:41.513993  6655 raft_consensus.cc:2393] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 13: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:41.481962 10671 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.482980 10691 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.490831 10615 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:41.501770 10474 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:41.490479 10607 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:41.495994 10642 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:41.508387 10649 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:41.503895 10689 raft_consensus.cc:2764] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:18:41.510545 10274 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:41.520422 10636 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:41.501024 10647 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:41.534601 10671 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:41.538182 10389 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:18:41.534027 10691 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:41.601804  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:41.538005 10688 consensus_queue.cc:1059] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 12:18:41.603506  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:41.603751  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:41.608438  6502 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 12.
I20260709 12:18:41.578898 10482 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:41.608877  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:41.605352  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:41.572670 10636 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:41.607717  6505 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 13.
I20260709 12:18:41.541457 10607 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:41.611371  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 255736us
I20260709 12:18:41.575136 10649 raft_consensus.cc:697] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:41.570590 10128 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:41.609586  6501 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:41.614650 10643 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.541405 10651 consensus_queue.cc:260] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:41.617368  7238 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:41.641490  7270 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
W20260709 12:18:41.651706  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 379356us
I20260709 12:18:41.615463 10258 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.662842  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:41.663786  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:41.615978 10482 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:41.672516  6501 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 11.
I20260709 12:18:41.726074  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:41.730453  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 117292us
W20260709 12:18:41.746562  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 79462.9us
I20260709 12:18:41.746959  6649 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 12.
W20260709 12:18:41.744138  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 249642us
I20260709 12:18:41.688391  6651 raft_consensus.cc:2393] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 13: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:41.730732  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:41.748001  6622 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:41.750129  6652 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 11
W20260709 12:18:41.656708 10382 consensus_meta.cc:287] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.855s	user 0.008s	sys 0.004s
W20260709 12:18:41.751889  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 234625us
I20260709 12:18:41.753176  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:41.761412  6550 leader_election.cc:400] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 9. Current term is 10.
I20260709 12:18:41.761883  6550 leader_election.cc:403] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:18:41.764029  6548 leader_election.cc:400] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11.
I20260709 12:18:41.767923  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:41.774977  6508 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:18:41.664889 10644 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.829972  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:18:41.833134  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:41.828544  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:41.834576  6501 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:41.842756  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:41.637919 10491 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:41.651105 10643 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:41.666324 10688 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:41.893590  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:41.894117  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:41.768725 10631 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.887414  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:41.691972 10664 consensus_queue.cc:1175] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f2b06aebe97249a4877d798b8118ece4" responder_term: 8 status { last_received { term: 8 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 8 index: 5 } }
I20260709 12:18:41.897966  6650 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:41.773631 10636 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:41.898298  6502 raft_consensus.cc:1275] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:41.896453 10688 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:41.896486 10702 consensus_queue.cc:1048] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
W20260709 12:18:41.905100 10697 log.cc:927] Time spent T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.217s	user 0.000s	sys 0.000s
I20260709 12:18:41.882509 10614 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:18:41.908816  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:18:41.917838 10682 log.cc:927] Time spent T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.911s	user 0.001s	sys 0.001s
I20260709 12:18:41.918823 10491 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:41.919903 10698 raft_consensus.cc:3055] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:41.921644 10482 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:41.883052 10382 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:41.922783 10691 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:41.787576 10695 consensus_queue.cc:1048] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.136s
W20260709 12:18:41.924515 10386 consensus_meta.cc:287] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 1.156s	user 0.003s	sys 0.010s
I20260709 12:18:41.926574 10345 raft_consensus.cc:3055] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:41.747794 10696 consensus_queue.cc:1059] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.162s
W20260709 12:18:41.928530  6504 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 2079) took 1726 ms (client timeout 1936 ms). Trace:
I20260709 12:18:41.931404 10643 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:41.901796  6506 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 12:18:42.016119  6504 rpcz_store.cc:269] 0709 12:18:40.202070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.202248 (+   178us) service_pool.cc:224] Handling call
0709 12:18:41.928232 (+1725984us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54001e64d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:41.928433 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2688723712}
I20260709 12:18:41.935691  6549 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:42.017773  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 82214.4us
I20260709 12:18:42.017603 10386 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:42.018734 10653 consensus_meta.cc:287] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.590s	user 0.010s	sys 0.000s
I20260709 12:18:42.019183 10653 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:41.935997  6576 raft_consensus.cc:1240] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:42.009560 10382 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:41.992677 10710 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:42.021612  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:42.008855 10671 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:42.022403  6581 raft_consensus.cc:2393] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 13: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:42.022537 10382 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:42.023330 10702 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:42.023881 10702 raft_consensus.cc:697] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:42.024356  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:42.024783  6503 raft_consensus.cc:2393] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 13: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:41.995589 10491 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.024912 10702 consensus_queue.cc:237] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:42.025581 10491 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:42.025915  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:41.940753  6501 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:18:42.028034 10491 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:42.025056  6583 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:41.916121 10690 log.cc:927] Time spent T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.477s	user 0.001s	sys 0.000s
I20260709 12:18:41.757107 10442 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:41.694504 10258 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:41.979839  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:18:42.040736  6579 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 1971) took 1503 ms (client timeout 1750 ms). Trace:
I20260709 12:18:41.985338  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:18:42.041272  6579 rpcz_store.cc:269] 0709 12:18:40.537543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.538080 (+   537us) service_pool.cc:224] Handling call
0709 12:18:42.040507 (+1502427us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b5400298e54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:42.040659 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":597448448}
I20260709 12:18:42.042236  6649 raft_consensus.cc:2393] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 11: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:42.042627  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:42.043591  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:42.043973  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:42.046629  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:42.044539 10258 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:42.047988  6649 raft_consensus.cc:2410] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
W20260709 12:18:41.987727  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 226427us
I20260709 12:18:41.771415 10473 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.173s
I20260709 12:18:41.985296  7240 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:41.991757  6502 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:41.991161 10698 raft_consensus.cc:740] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
W20260709 12:18:42.051669 10685 log.cc:927] Time spent T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.658s	user 0.002s	sys 0.001s
I20260709 12:18:42.051451 10698 consensus_queue.cc:260] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:42.052539 10389 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:18:42.053069 10698 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:41.998507  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1908) took 1606 ms. Trace:
I20260709 12:18:41.752604 10649 consensus_queue.cc:237] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:42.056766  7267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38562 (request call id 674) took 1334 ms. Trace:
I20260709 12:18:42.057212  7267 rpcz_store.cc:276] 0709 12:18:40.722540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.724498 (+  1958us) service_pool.cc:224] Handling call
0709 12:18:40.724845 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:40.725530 (+   685us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:40.725562 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:40.751934 (+ 26372us) log.cc:844] Serialized 26 byte log entry
0709 12:18:41.010282 (+258348us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:41.010320 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:41.010383 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:42.051006 (+1040623us) spinlock_profiling.cc:243] Waited 3.58 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:18:42.056416 (+  5410us) raft_consensus.cc:1685] finished
0709 12:18:42.056441 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:42.056703 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":578,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":62264,"spinlock_wait_cycles":8244992,"thread_start_us":176286,"threads_started":1,"wal-append.queue_time_us":472684}
I20260709 12:18:41.946815 10688 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:42.056520  6656 rpcz_store.cc:276] 0709 12:18:40.396881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.409700 (+ 12819us) service_pool.cc:224] Handling call
0709 12:18:40.461086 (+ 51386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:40.941208 (+480122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:41.998223 (+1057015us) spinlock_profiling.cc:243] Waited 423 ms on lock 0x7b540010ed54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:41.998435 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":972398336}
I20260709 12:18:42.057085  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:42.061292  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:42.062260  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:42.063102  6649 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:42.063966  6656 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:41.998409 10647 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:41.999724 10482 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:42.005121 10715 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:42.005942 10607 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:41.989429 10609 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 9. Current term is 10.
I20260709 12:18:41.944320 10631 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:42.013836  6650 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:18:41.940218  6652 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 11.
I20260709 12:18:42.025539  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:42.029186  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:42.088610  6577 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:42.030705  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:41.902096 10644 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:41.883313  6475 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:42.033115  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:42.029950 10696 raft_consensus.cc:3055] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:18:42.051766  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 66612.4us
I20260709 12:18:42.107384  7240 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:42.065913  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 50231us
I20260709 12:18:42.109633  7240 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:42.044653  6579 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:41.995142  6548 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:42.111642  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:42.020174 10386 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:42.036991 10706 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:42.125010  6581 raft_consensus.cc:1275] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:42.015837 10345 raft_consensus.cc:740] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:42.075695 10482 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:42.014292 10274 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:42.088714  6654 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:42.103600  6577 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
W20260709 12:18:42.028218  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 144751us
I20260709 12:18:42.092734  6582 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:42.132124  6622 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:41.882220 10701 consensus_queue.cc:1059] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.218s
I20260709 12:18:42.133531 10635 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:42.134095 10701 raft_consensus.cc:3055] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:42.134655 10701 raft_consensus.cc:740] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:42.135617 10644 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:42.128698 10389 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:42.137184 10389 raft_consensus.cc:697] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 11 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:42.136969 10345 consensus_queue.cc:260] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:42.138024 10692 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.138368 10345 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:42.138170 10389 consensus_queue.cc:237] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:42.138597 10692 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:42.085110 10715 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:42.141177 10653 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:42.128611 10629 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.141945 10629 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:42.113067  6658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2418) took 1099 ms. Trace:
W20260709 12:18:42.114925  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 119862us
I20260709 12:18:42.144848  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:42.144912  6658 rpcz_store.cc:276] 0709 12:18:41.013816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:41.211696 (+197880us) service_pool.cc:224] Handling call
0709 12:18:41.212128 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:41.212534 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:41.212578 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:41.252061 (+ 39483us) log.cc:844] Serialized 26 byte log entry
0709 12:18:41.488790 (+236729us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:41.488830 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:41.488894 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:42.112792 (+623898us) raft_consensus.cc:1685] finished
0709 12:18:42.112822 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:42.112992 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4857,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":4307,"thread_start_us":159985,"threads_started":1,"wal-append.queue_time_us":308730}
I20260709 12:18:42.119887  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:42.146632  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:42.147571  6575 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
W20260709 12:18:42.129843  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 298813us
I20260709 12:18:42.148917  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:42.149082  6622 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:42.137456 10607 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:42.157776  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:42.158514  6501 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:18:42.122931  6583 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:18:42.153762  7266 raft_consensus.cc:2393] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 13: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:42.158877  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:42.152925  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:42.163336  6475 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:42.155073  6582 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:42.157641  6473 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:18:42.147816  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:42.123842 10474 consensus_meta.cc:287] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.582s	user 0.011s	sys 0.000s
I20260709 12:18:42.106029 10442 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:42.166594  6577 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:42.166467  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:42.170032  6622 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:42.172541 10710 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:42.172680  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:42.174707  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:42.194171  6622 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:42.194564  6502 raft_consensus.cc:2393] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 9: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:42.175556  6504 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:42.176976  6506 raft_consensus.cc:1275] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:42.164602  6575 raft_consensus.cc:2393] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 13: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:18:42.180787  7270 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38558 (request call id 627) took 1465 ms (client timeout 1811 ms). Trace:
I20260709 12:18:42.181154  6576 raft_consensus.cc:1275] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:18:42.195497  6509 raft_consensus.cc:2393] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 12: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:42.181766  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:42.199765  6583 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:18:42.181730  6579 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:42.190303  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:42.201349  6658 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:18:42.202325  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:42.114972 10696 raft_consensus.cc:740] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:42.200595  6474 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:18:42.196395  7270 rpcz_store.cc:269] 0709 12:18:40.715789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.715914 (+   125us) service_pool.cc:224] Handling call
0709 12:18:42.180222 (+1464308us) spinlock_profiling.cc:243] Waited 740 ms on lock 0x7b54004e70d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:42.180724 (+   502us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1702326016}
I20260709 12:18:42.203967  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:42.128957 10714 raft_consensus.cc:2804] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:42.208066  6475 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:42.199750  6622 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:42.206866  6575 raft_consensus.cc:1275] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:42.153324 10627 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.180177 10474 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:42.154287 10653 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.191066 10386 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:42.207270  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 54648.5us
I20260709 12:18:42.219297  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:42.220299  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1953) took 2143 ms. Trace:
I20260709 12:18:42.220732  6580 rpcz_store.cc:276] 0709 12:18:40.077076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:40.481356 (+404280us) service_pool.cc:224] Handling call
0709 12:18:42.220110 (+1738754us) spinlock_profiling.cc:243] Waited 910 ms on lock 0x7b5400058cd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:42.220220 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2092523264}
I20260709 12:18:42.196877 10614 raft_consensus.cc:2764] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:18:42.135679 10701 consensus_queue.cc:260] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:42.224385  6658 raft_consensus.cc:2393] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 13: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:42.196588 10258 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:42.233644  6548 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:42.230760  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:42.226945  6474 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:42.227145  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:42.207634 10488 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:42.206463 10680 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:42.212543 10714 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:42.192508 10635 consensus_queue.cc:1048] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:18:42.213497 10629 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:42.246555  6475 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:42.258111  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:42.194826 10688 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:42.219203 10482 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:42.236347 10710 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:42.236191 10258 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:42.231817 10648 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:42.231990 10647 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:42.298552  6502 raft_consensus.cc:1275] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:18:42.250136 10621 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 12:18:42.302790 10482 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:42.290244 10649 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:42.307380  6649 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:18:42.307516 10716 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:42.308173  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:42.323657  6581 raft_consensus.cc:2368] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 8. Current term is 9.
I20260709 12:18:42.291337 10644 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.308912  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:42.309660  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:42.223495 10627 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:42.259720 10714 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:42.236972 10607 consensus_queue.cc:1048] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 12:18:42.225005 10653 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:42.311854  6622 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:42.335966  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:42.314368  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:42.336177  6548 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:42.336764  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:42.313594  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:42.304840 10355 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.260596 10609 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:42.308710  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:42.324821  6583 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:18:42.324968  6653 raft_consensus.cc:1275] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:42.297787 10631 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:42.327910  6657 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 7.
I20260709 12:18:42.298265 10258 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:42.359856  6657 raft_consensus.cc:3055] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:42.360509  6657 raft_consensus.cc:740] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:42.329910  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:42.253319 10664 consensus_queue.cc:1059] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 12:18:42.144654 10706 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:42.299561 10692 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:42.314064 10518 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.312017  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:42.314672 10680 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:42.269361 10719 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 12:18:42.346071  6622 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:42.348872 10472 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:42.362298  6575 raft_consensus.cc:2393] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 12: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:42.326725 10481 raft_consensus.cc:2764] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
W20260709 12:18:42.364012  6393 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 3e0ab356fc074f589a6ff4f9b4996b09 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 20a202c76244485fbc8f1610473db8b9 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:18:42.311009 10702 consensus_queue.cc:1059] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:18:42.370303  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:42.370977  6576 raft_consensus.cc:2393] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 13: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:42.371205  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:42.373075  6500 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:42.334017 10649 raft_consensus.cc:697] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:42.344537 10698 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.377796  6651 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:42.358582 10614 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:42.345427 10689 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:42.472183  6396 maintenance_manager.cc:419] P 39c4be726d3d495181bfe7425c8c0bd8: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033339
I20260709 12:18:42.459213  6580 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:42.453128 10715 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:42.297603 10696 consensus_queue.cc:260] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:42.326192 10716 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:42.364212 10499 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.077s
W20260709 12:18:42.441119  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:42.358577 10695 consensus_queue.cc:1059] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 12:18:42.380117  6657 consensus_queue.cc:260] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:42.392107 10518 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:42.440442  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 94449.6us
I20260709 12:18:42.456595 10636 raft_consensus.cc:2764] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:18:42.326303 10482 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:42.499315 10722 log.cc:927] Time spent T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 12:18:42.381541 10274 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:42.357705 10355 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:42.465659  6658 raft_consensus.cc:2393] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 10: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:42.494481 10474 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:42.396278  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:42.419230  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 71573.9us
I20260709 12:18:42.404675 10472 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:18:42.488405 10727 log.cc:927] Time spent T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 12:18:42.470016 10708 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:42.496088  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:42.500072  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 135315us
I20260709 12:18:42.504245  6657 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:42.459064 10653 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:42.509819  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:42.332907 10644 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:42.490710 10615 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:42.394634 10680 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:42.522246  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:42.543515  7266 raft_consensus.cc:2393] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 9: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:42.523416  6581 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:42.525502  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:42.531132  6580 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:18:42.531314  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:42.491206 10698 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:42.534025  6575 raft_consensus.cc:1240] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:18:42.518177 10715 raft_consensus.cc:3055] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:42.541064  6654 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:42.546329  6654 raft_consensus.cc:1240] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:42.542228  6658 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:42.548503  7273 raft_consensus.cc:2393] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 11: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:42.548151  6653 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:42.527108 10472 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:42.553730  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:42.554672  6507 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:42.557135  6580 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 12:18:42.559158  6580 raft_consensus.cc:1240] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:42.561767  6580 raft_consensus.cc:1240] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:42.563551  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:42.532673 10614 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:42.545446 10688 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:42.566900 10466 consensus_queue.cc:1048] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:42.568019 10742 raft_consensus.cc:3055] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:42.568537 10742 raft_consensus.cc:740] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:18:42.568750 10729 log.cc:927] Time spent T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 12:18:42.569710 10742 consensus_queue.cc:260] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:42.572162 10482 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:42.572798 10482 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.573289 10482 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:42.575531 10482 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:42.580634 10706 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:42.581651 10716 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:42.586248  6654 raft_consensus.cc:1240] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:42.590905  7272 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:42.596547  6657 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:18:42.599669 10701 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:42.548575 10715 raft_consensus.cc:740] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:42.616477  6658 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:18:42.515843 10649 consensus_queue.cc:237] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.4, Last appended by leader: 4, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:42.533425 10708 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:42.609026 10555 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:42.544245 10615 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:42.631268  6549 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:42.622808 10742 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:42.626883 10627 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:42.699127  6581 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
W20260709 12:18:42.703066  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 71895.7us
I20260709 12:18:42.630409 10555 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:42.724808 10644 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:42.740139  6656 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:42.718446 10555 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:42.845664  6548 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:42.747505 10355 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:42.840453 10695 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.864144  6651 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:18:42.842597 10738 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.725286 10714 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:42.855782 10728 log.cc:927] Time spent T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.340s	user 0.001s	sys 0.001s
I20260709 12:18:42.842644 10747 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:42.857575 10691 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.858355 10518 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:42.884289  7270 raft_consensus.cc:1240] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:18:42.872337 10716 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.873682 10698 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:42.843212 10715 consensus_queue.cc:260] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:42.883016 10680 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:42.872483 10738 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:42.871860 10695 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:42.903461 10610 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:42.920564  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 355003us
I20260709 12:18:42.922636  6622 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
W20260709 12:18:42.923051 10741 log.cc:927] Time spent T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.200s	user 0.000s	sys 0.000s
I20260709 12:18:42.923657 10680 raft_consensus.cc:2764] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
W20260709 12:18:42.927278 10753 log.cc:927] Time spent T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 12:18:42.929656  6583 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 8. Current term is 9. Ops: []
W20260709 12:18:42.930001 10345 consensus_meta.cc:287] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.791s	user 0.011s	sys 0.001s
I20260709 12:18:42.918272 10615 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:42.916865 10482 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:42.933971  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:42.887635 10691 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:42.941151  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:42.942970 10691 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:42.950786  6622 leader_election.cc:400] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 8. Current term is 9.
I20260709 12:18:42.951261  6622 leader_election.cc:403] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:18:42.952904  6622 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:42.953217 10615 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 8. Current term is 9.
W20260709 12:18:42.949086 10744 log.cc:927] Time spent T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.239s	user 0.006s	sys 0.000s
I20260709 12:18:42.932816 10482 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:42.955356  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:42.904974 10716 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:42.945004 10708 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:42.962119 10695 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:42.955042 10518 consensus_queue.cc:1059] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 12:18:42.960624 10706 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:42.955101 10680 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:18:43.022253 10710 consensus_meta.cc:287] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.707s	user 0.008s	sys 0.003s
I20260709 12:18:43.029021 10345 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:43.030964 10345 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:43.032575 10128 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.033380 10128 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:43.034065 10345 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:43.034770 10345 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:18:43.035570 10386 consensus_meta.cc:287] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.811s	user 0.006s	sys 0.005s
I20260709 12:18:42.985423  6622 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:18:43.037743  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 52473.3us
I20260709 12:18:42.940008 10610 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:43.042173 10708 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:18:43.042454 10610 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:42.989419 10758 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:43.042930 10708 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:43.043396 10758 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:43.044503 10680 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:43.044734  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:43.045033 10680 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:43.045699  6656 raft_consensus.cc:2393] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 13: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:43.046271 10758 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:43.046736  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:43.047360 10128 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:43.049113 10615 consensus_queue.cc:1059] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:43.049482 10689 consensus_queue.cc:1059] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:43.050807  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:43.051473  6658 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:18:43.051687 10689 raft_consensus.cc:3055] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:43.052088 10689 raft_consensus.cc:740] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:43.052579  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:43.052985 10689 consensus_queue.cc:260] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:43.054795 10689 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:43.055136 10615 raft_consensus.cc:3055] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:43.055583 10615 raft_consensus.cc:740] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:43.056521 10615 consensus_queue.cc:260] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:43.057592 10615 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:43.062937  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:43.064811  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:42.994429 10482 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:42.997398  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 993973us
I20260709 12:18:43.005947 10738 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:43.006217 10631 raft_consensus.cc:3055] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:43.067411  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:43.074549  6550 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:43.077553  7273 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:18:43.080442  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:43.081116  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:42.955559 10752 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:18:43.047439 10680 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:43.084616  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:43.042483  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:43.093324  6651 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:43.048043 10706 raft_consensus.cc:697] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 11 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:43.071369 10710 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:42.991683 10714 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:43.066325  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:43.084637  7238 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:43.099763  7267 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f"
dest_uuid: "20a202c76244485fbc8f1610473db8b9"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:48510
I20260709 12:18:43.100397  7267 raft_consensus.cc:606] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:43.100737  7267 raft_consensus.cc:614] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:43.086081  6507 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:18:43.047266 10747 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:43.102051  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:43.088377  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:43.088321  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:43.084422  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:43.047688  6656 raft_consensus.cc:2410] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:18:43.035337 10345 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:43.105552  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:43.107213  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:43.104868  7266 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:43.108067  6575 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:18:43.067967 10386 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:18:43.085367 10744 log.cc:927] Time spent T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
W20260709 12:18:43.124799 10751 log.cc:927] Time spent T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 12:18:43.125563  7274 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:18:43.111102  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:43.126742  6658 raft_consensus.cc:2393] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 8: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:43.126999  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:43.127562  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:43.128129  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:43.129173  6656 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:43.129031  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:43.130125  6658 raft_consensus.cc:2410] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:18:43.113730  6651 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 13.
I20260709 12:18:43.131574  6651 raft_consensus.cc:1275] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:43.113637  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:43.114367  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:43.114661  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:43.139228  6583 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 13.
I20260709 12:18:43.115833  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:43.141737  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:43.143033  6582 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 9.
I20260709 12:18:43.084918 10355 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:43.137773  7238 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:18:43.076830 10518 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:43.157667  6509 raft_consensus.cc:1275] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:43.156904  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:43.144620  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:43.138149  6576 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 9.
I20260709 12:18:43.147141  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:43.149402  6505 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:43.144089  6575 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:18:43.165539  7265 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:18:43.165807  6575 raft_consensus.cc:606] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:43.166445  6575 raft_consensus.cc:614] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:43.044318 10708 consensus_queue.cc:1059] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 12:18:43.084604 10631 raft_consensus.cc:740] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:43.106747 10482 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:43.109263 10671 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:43.160226  7266 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:43.169938  6650 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:43.184257  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:43.185463  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:43.109880 10688 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:43.084232 10491 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 12:18:43.187197  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:43.167142 10128 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:43.087211 10609 consensus_queue.cc:1059] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
W20260709 12:18:43.194518  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 164817us
I20260709 12:18:43.192680  6550 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:43.207072  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:43.101364 10746 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
I20260709 12:18:43.196425  6623 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:43.194741  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:43.213618  6623 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:43.203572  6506 raft_consensus.cc:2410] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:18:43.187677  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:43.215899  6651 raft_consensus.cc:2410] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 12:18:43.191089 10258 consensus_meta.cc:287] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.814s	user 0.012s	sys 0.000s
I20260709 12:18:43.208134  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:43.204618 10719 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.022s	sys 0.072s
W20260709 12:18:43.201267 10690 log.cc:927] Time spent T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:18:43.224627  6549 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:43.220934  7266 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
W20260709 12:18:43.215674 10472 consensus_meta.cc:287] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.624s	user 0.013s	sys 0.000s
I20260709 12:18:43.226070  7271 raft_consensus.cc:1275] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:18:43.224665  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:43.199899 10482 raft_consensus.cc:697] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:43.230720  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:43.233970  6651 raft_consensus.cc:2393] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 6: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:43.206136 10128 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:43.174629 10518 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:43.232463  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:43.234047  6549 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:18:43.239780  6509 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 13.
I20260709 12:18:43.222565 10747 consensus_queue.cc:1048] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 12:18:43.239296  7238 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:43.239137 10472 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:43.245436 10768 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:43.245679  6580 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 11.
I20260709 12:18:43.246032 10768 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:43.246950 10472 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:43.224938 10719 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.023s	sys 0.073s
I20260709 12:18:43.223647 10709 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:43.250703  6549 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:18:43.222949 10708 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:43.251752  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:43.252763  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:43.253237  7238 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:43.219554 10761 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:43.259737  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:43.288264  6509 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 11.
I20260709 12:18:43.241045 10128 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:43.309799  6651 raft_consensus.cc:1275] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:18:43.237628 10609 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:43.307987  7240 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:43.295723  6656 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:43.220067 10714 consensus_queue.cc:1059] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 12:18:43.189152 10706 consensus_queue.cc:237] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:43.228274 10258 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:43.257942 10695 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:43.258262 10708 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:43.280151 10472 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:43.255295 10709 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:18:43.359378  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 51539.5us
I20260709 12:18:43.361327 10775 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:43.241149 10738 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 12:18:43.291657 10761 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:43.363560 10742 consensus_meta.cc:287] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.711s	user 0.007s	sys 0.005s
W20260709 12:18:43.358368 10614 consensus_meta.cc:287] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.766s	user 0.012s	sys 0.000s
W20260709 12:18:43.374043  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 120918us
I20260709 12:18:43.357357 10695 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:43.362555 10708 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:43.379333  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:43.367816 10688 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:43.369932 10709 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:43.390795  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 44495.4us
I20260709 12:18:43.366999 10472 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:43.371647 10768 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:43.417817  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:43.402832  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 70926.6us
I20260709 12:18:43.377609 10695 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:43.375212 10761 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:43.378530 10614 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:18:43.415366 10701 consensus_meta.cc:287] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.792s	user 0.013s	sys 0.000s
W20260709 12:18:43.440459 10767 log.cc:927] Time spent T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 12:18:43.417166 10472 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:43.444236  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:43.442140 10766 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:18:43.451226  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:43.442903  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:43.436878 10768 consensus_queue.cc:1048] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:43.461198  6548 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:43.416561 10345 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:43.461275  7265 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:18:43.470817  6548 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:43.462124  6575 raft_consensus.cc:1240] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:43.472419  7266 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f2592253d3214104865730480e057d09"
dest_uuid: "20a202c76244485fbc8f1610473db8b9"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:48510
I20260709 12:18:43.473392  7266 raft_consensus.cc:606] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:43.465425  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:43.473955  7266 raft_consensus.cc:614] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:43.476780  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:43.372468 10482 consensus_queue.cc:237] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:43.478099 10609 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:43.479527 10595 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:43.480460  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2184) took 1319 ms. Trace:
W20260709 12:18:43.502894  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 131871us
I20260709 12:18:43.442728 10688 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:43.507033  6503 rpcz_store.cc:276] 0709 12:18:42.161455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.161604 (+   149us) service_pool.cc:224] Handling call
0709 12:18:42.175659 (+ 14055us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:42.206342 (+ 30683us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:42.206372 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:43.059015 (+852643us) spinlock_profiling.cc:243] Waited 8.38 ms on lock 0x7b1800410c80. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d6a2a6 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a
0709 12:18:43.060477 (+  1462us) spinlock_profiling.cc:243] Waited 2.03 ms on lock 0x7b2c00000c88. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:18:43.060972 (+   495us) log.cc:844] Serialized 26 byte log entry
0709 12:18:43.108498 (+ 47526us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:43.108533 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:43.108598 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:43.480168 (+371570us) raft_consensus.cc:1685] finished
0709 12:18:43.480199 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:43.480382 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30878,"raft.queue_time_us":27912,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":5468,"spinlock_wait_cycles":23946496,"thread_start_us":47154,"threads_started":1,"wal-append.queue_time_us":163647}
I20260709 12:18:43.509079  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:43.509291  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:43.483048 10701 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:43.510603 10701 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:43.511225 10701 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:43.512220  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:43.472695 10648 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.513216 10648 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:43.513800 10614 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:43.514528 10614 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:43.516013 10648 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:43.518185 10614 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:43.518205 10555 consensus_meta.cc:287] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.675s	user 0.013s	sys 0.008s
I20260709 12:18:43.519279  6502 raft_consensus.cc:1275] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 12:18:43.527276  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 52023.1us
I20260709 12:18:43.532469  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:43.544348  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 75386.3us
I20260709 12:18:43.547005  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:43.556965  7273 raft_consensus.cc:1275] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 12:18:43.564007  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:43.525709 10555 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:43.574234 10615 consensus_meta.cc:287] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.516s	user 0.012s	sys 0.002s
I20260709 12:18:43.532706 10775 raft_consensus.cc:697] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 13 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:43.535557 10738 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:18:43.589094  6654 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:43.592252  6474 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:43.593834  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:43.599578  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:43.595018  7268 raft_consensus.cc:2376] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9. Re-sending same reply.
I20260709 12:18:43.603645 10793 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:43.604408 10793 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:43.607362 10793 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:43.607475 10791 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:43.592504 10771 log.cc:927] Time spent T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 12:18:43.601226 10482 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:43.620587 10345 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:43.621646 10128 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:43.623982 10345 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:43.614933 10258 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:43.635854  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:43.639039  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 109933us
I20260709 12:18:43.630605 10482 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:43.647152  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:43.647607 10345 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:43.593309 10775 consensus_queue.cc:237] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:43.651157  6548 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:43.649164 10128 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:43.652366  6651 raft_consensus.cc:1240] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:18:43.655496 10482 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:43.645844 10781 consensus_queue.cc:1048] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
W20260709 12:18:43.657140 10772 log.cc:927] Time spent T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:18:43.641238 10738 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.658205 10738 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:43.660012 10547 raft_consensus.cc:3055] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:43.660460 10547 raft_consensus.cc:740] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:43.661327 10547 consensus_queue.cc:260] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:43.664685  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1934) took 1909 ms. Trace:
I20260709 12:18:43.664825 10547 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:43.665131  6652 rpcz_store.cc:276] 0709 12:18:41.755762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.088157 (+332395us) service_pool.cc:224] Handling call
0709 12:18:42.129441 (+ 41284us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:42.129874 (+   433us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:18:42.992467 (+862593us) spinlock_profiling.cc:243] Waited 7.25 ms on lock 0x7b18001832e0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d99455 00007f28798813c7 00007f2875e939a7 00007f2875ea30bf 00007f2875ea3038
0709 12:18:43.129518 (+137051us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b48000cda58. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287adb2aa1 00007f287ada9b37 00007f287ada99ca 00007f287aea126f 00007f287aea0f37 00007f287aeb555c 00007f287aeb5508 00007f287aeb5488 00007f287aeb5443 00007f287aeb46ec 00007f2879ddea96
0709 12:18:43.160270 (+ 30752us) log.cc:844] Serialized 14 byte log entry
0709 12:18:43.369512 (+209242us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:43.378757 (+  9245us) log.cc:844] Serialized 52 byte log entry
0709 12:18:43.378970 (+   213us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:43.379021 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:43.379117 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:43.664411 (+285294us) raft_consensus.cc:1685] finished
0709 12:18:43.664442 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:43.664624 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":311363,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":18760,"raft.queue_time_us":15107,"raft.run_cpu_time_us":3760,"raft.run_wall_time_us":978901,"spinlock_wait_cycles":43849856,"thread_start_us":300567,"threads_started":2,"wal-append.queue_time_us":368126}
I20260709 12:18:43.630873 10795 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:43.662814 10776 raft_consensus.cc:3055] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:43.667706  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:43.676668  6651 raft_consensus.cc:1240] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:18:43.677156  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:43.668694 10738 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:43.678432  6506 raft_consensus.cc:2393] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 11: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:43.627841 10701 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:43.672045  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:43.650072 10258 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.625738 10695 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:43.696859  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:43.697082  6503 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:18:43.679507  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:43.681512 10738 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:18:43.683614 10689 consensus_meta.cc:287] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.628s	user 0.007s	sys 0.006s
I20260709 12:18:43.684127  6504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2194) took 1130 ms. Trace:
I20260709 12:18:43.689102 10651 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:43.689392 10706 consensus_queue.cc:1059] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:43.623076 10791 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:43.699499  6658 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:43.702637  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2183) took 1542 ms. Trace:
I20260709 12:18:43.705400  7266 raft_consensus.cc:2393] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 12: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:43.706387 10642 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:43.707414 10701 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:43.678452 10795 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:43.736698  7272 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38562 (request call id 683) took 1198 ms. Trace:
I20260709 12:18:43.737277  7272 rpcz_store.cc:276] 0709 12:18:42.538125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.538401 (+   276us) service_pool.cc:224] Handling call
0709 12:18:43.736476 (+1198075us) spinlock_profiling.cc:243] Waited 26.7 ms on lock 0x7b54004abad4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:43.736618 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":61449600}
I20260709 12:18:43.738660 10795 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:43.740983  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2430) took 1625 ms. Trace:
I20260709 12:18:43.741952  6549 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:18:43.742974 10795 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:18:43.743291  6650 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58980 (request call id 2429) took 1632 ms (client timeout 1850 ms). Trace:
I20260709 12:18:43.700948 10258 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:43.746863 10258 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:43.747607  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:18:43.676303 10776 raft_consensus.cc:740] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:43.750646 10776 consensus_queue.cc:260] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:43.740178  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:43.753400  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:43.759577  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:43.743793  6650 rpcz_store.cc:269] 0709 12:18:42.111181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.226880 (+115699us) service_pool.cc:224] Handling call
0709 12:18:43.743117 (+1516237us) spinlock_profiling.cc:243] Waited 933 ms on lock 0x7b540010e354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:43.743238 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2145600768}
I20260709 12:18:43.711468  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:43.715993  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:43.717840  6548 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:43.764860  6503 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:18:43.766028  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:43.720324  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:43.721508 10472 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:43.719964 10466 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:43.722911 10791 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:43.715988  6501 rpcz_store.cc:276] 0709 12:18:42.160227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.160399 (+   172us) service_pool.cc:224] Handling call
0709 12:18:42.160808 (+   409us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:42.181251 (+ 20443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:42.181280 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:42.193497 (+ 12217us) log.cc:844] Serialized 52 byte log entry
0709 12:18:42.322644 (+129147us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:42.494074 (+171430us) log.cc:844] Serialized 14 byte log entry
0709 12:18:42.496026 (+  1952us) log.cc:844] Serialized 14 byte log entry
0709 12:18:42.496393 (+   367us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:42.496528 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:43.604833 (+1108305us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b18003ba700. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:18:43.702403 (+ 97570us) raft_consensus.cc:1685] finished
0709 12:18:43.702440 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:43.702578 (+   138us) inbound_call.cc:177] Queueing success response
0709 12:18:43.706035 (+  3457us) spinlock_profiling.cc:243] Waited 3.17 ms on lock 0x7b18003ba700. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
Metrics: {"mutex_wait_us":20004,"prepare.queue_time_us":306462,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":2928,"raft.queue_time_us":30738,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":12074,"spinlock_wait_cycles":44852736,"thread_start_us":255996,"threads_started":2,"wal-append.queue_time_us":269809}
I20260709 12:18:43.688079 10708 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:43.724574 10709 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:43.725760 10482 consensus_queue.cc:1059] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:43.731110 10695 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:43.680306 10793 consensus_queue.cc:1059] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:18:43.741437  6649 rpcz_store.cc:276] 0709 12:18:42.115142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.515418 (+400276us) service_pool.cc:224] Handling call
0709 12:18:42.529010 (+ 13592us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:42.643840 (+114830us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:42.668267 (+ 24427us) log.cc:844] Serialized 14 byte log entry
0709 12:18:42.868753 (+200486us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:42.868833 (+    80us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:42.868901 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:43.740731 (+871830us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b5400373554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:43.740784 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:43.740920 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1110,"prepare.queue_time_us":24625,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":64,"raft.queue_time_us":106,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":9409,"spinlock_wait_cycles":263475072,"thread_start_us":199963,"threads_started":1,"wal-append.queue_time_us":277364}
I20260709 12:18:43.767552  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:43.767104 10796 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:43.768962  6550 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:43.712411  6504 rpcz_store.cc:276] 0709 12:18:42.553738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.554256 (+   518us) service_pool.cc:224] Handling call
0709 12:18:42.554542 (+   286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:42.561257 (+  6715us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:42.561298 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:42.561723 (+   425us) log.cc:844] Serialized 26 byte log entry
0709 12:18:42.745952 (+184229us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:42.845122 (+ 99170us) log.cc:844] Serialized 14 byte log entry
0709 12:18:42.845369 (+   247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:42.993842 (+148473us) spinlock_profiling.cc:243] Waited 2.67 ms on lock 0x7b180006e860. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d6a2a6 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1
0709 12:18:42.995836 (+  1994us) spinlock_profiling.cc:243] Waited 3.82 ms on lock 0x7b180006e860. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:18:43.683804 (+687968us) spinlock_profiling.cc:243] Waited 1.18 ms on lock 0x7b5400203d54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:43.683867 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:43.683920 (+    53us) raft_consensus.cc:1685] finished
0709 12:18:43.683935 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:43.684067 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5156,"prepare.queue_time_us":98972,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":7612,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":6641,"spinlock_wait_cycles":66443904,"thread_start_us":122078,"threads_started":1,"wal-append.queue_time_us":243152}
I20260709 12:18:43.768319  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:43.770629  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:43.772387  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 54662.2us
I20260709 12:18:43.801054  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:43.778000 10789 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.779285  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:43.804477  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:43.724444 10555 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:43.780704  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:43.809139  6506 raft_consensus.cc:2393] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 11: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:43.779671 10547 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:18:43.792197  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 200000us
I20260709 12:18:43.768225  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:43.784519 10345 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.805159  6501 raft_consensus.cc:2393] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 13: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:43.805189  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:43.812086  6575 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:43.803273  6503 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:18:43.814596 10801 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:18:43.818030 10723 log.cc:927] Time spent T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 1.434s	user 0.005s	sys 0.001s
I20260709 12:18:43.818593 10708 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:43.810184 10695 raft_consensus.cc:3055] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:43.819482 10695 raft_consensus.cc:740] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:43.819880  6653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2425) took 1710 ms. Trace:
I20260709 12:18:43.820441  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:43.820366 10695 consensus_queue.cc:260] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:43.821940 10695 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:43.823786  6654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2433) took 1498 ms. Trace:
I20260709 12:18:43.809307  6579 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7.
I20260709 12:18:43.781673 10701 consensus_queue.cc:1048] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:18:43.818610 10472 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:43.824785  6507 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:18:43.810804 10789 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:43.824195  6654 rpcz_store.cc:276] 0709 12:18:42.325573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.622536 (+296963us) service_pool.cc:224] Handling call
0709 12:18:42.641574 (+ 19038us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:43.589695 (+948121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:43.823536 (+233841us) spinlock_profiling.cc:243] Waited 947 ms on lock 0x7b54003e8fd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:43.823722 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2178369792}
I20260709 12:18:43.743638 10795 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:43.825558 10345 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:43.826082 10795 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:43.826344  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:43.826673 10789 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:43.827565  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:43.782361 10466 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:43.827936  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:43.829161  6501 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:43.829599 10466 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:43.829831  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2428) took 1719 ms. Trace:
I20260709 12:18:43.830291  6655 rpcz_store.cc:276] 0709 12:18:42.110828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.483349 (+372521us) service_pool.cc:224] Handling call
0709 12:18:42.483797 (+   448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:42.525939 (+ 42142us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:42.525987 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:42.533036 (+  7049us) log.cc:844] Serialized 26 byte log entry
0709 12:18:42.714484 (+181448us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:42.714545 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:43.829464 (+1114919us) spinlock_profiling.cc:243] Waited 41.6 ms on lock 0x7b540044b1d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:43.829510 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:43.829557 (+    47us) raft_consensus.cc:1685] finished
0709 12:18:43.829570 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:43.829773 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1705,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":5655,"spinlock_wait_cycles":95782784,"thread_start_us":126815,"threads_started":1,"wal-append.queue_time_us":156672,"wal-append.run_cpu_time_us":2219,"wal-append.run_wall_time_us":1003094}
I20260709 12:18:43.831568  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:43.831744  6508 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 8. Current term is 9. Ops: []
W20260709 12:18:43.832907  6581 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1971) took 1497 ms (client timeout 1509 ms). Trace:
I20260709 12:18:43.833263  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:43.833357  6581 rpcz_store.cc:269] 0709 12:18:42.335733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.345194 (+  9461us) service_pool.cc:224] Handling call
0709 12:18:43.832730 (+1487536us) spinlock_profiling.cc:243] Waited 47.1 ms on lock 0x7b54000587d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:43.832834 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":108248448}
I20260709 12:18:43.834028  6579 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 9.
I20260709 12:18:43.834784 10708 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:43.835381 10708 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:43.831362 10631 consensus_queue.cc:260] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:43.836896 10631 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:43.838392 10743 log.cc:927] Time spent T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 1.266s	user 0.002s	sys 0.000s
I20260709 12:18:43.821993 10749 raft_consensus.cc:3055] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:43.840462 10749 raft_consensus.cc:740] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:43.841557  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:43.841326  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:43.832934  6508 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:43.842679  6506 raft_consensus.cc:2393] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 13: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:43.843468  6508 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:43.820300  6653 rpcz_store.cc:276] 0709 12:18:42.109779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.360276 (+250497us) service_pool.cc:224] Handling call
0709 12:18:42.545892 (+185616us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:42.550220 (+  4328us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:43.819639 (+1269419us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b54003e9ed0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:43.819812 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":384753024}
I20260709 12:18:43.843899  6506 raft_consensus.cc:1240] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:43.844472  6504 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:43.828318  6507 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:43.852699  6473 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:43.853924 10709 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:43.850241 10689 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.852633 10708 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:43.855753 10638 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:43.855509 10689 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:43.856709  6622 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:43.852658 10749 consensus_queue.cc:260] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:43.858081 10355 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:43.858305 10749 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:43.858631 10355 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:43.859776  6575 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 9.
W20260709 12:18:43.860598  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 91695.6us
I20260709 12:18:43.861565 10747 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:43.861835  7272 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:43.862022 10747 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:43.862429 10747 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:43.847752  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 45774.3us
I20260709 12:18:43.848040  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:43.863808  6503 raft_consensus.cc:2393] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 11: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:43.863843  6548 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:43.864753  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:43.853942  6473 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:43.865643  6655 raft_consensus.cc:2393] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 12: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:43.866128  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:43.866305  6473 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:43.866496  6623 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:43.867472  6650 raft_consensus.cc:1275] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:18:43.847172  6475 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:43.869393  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:43.869947  6650 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:43.870590  6507 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 12.
I20260709 12:18:43.872581  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:43.872987  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:43.873744  6653 raft_consensus.cc:2393] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 8: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:43.873749  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:43.876116  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:43.876914  6653 raft_consensus.cc:2376] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12. Re-sending same reply.
I20260709 12:18:43.879515  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:43.879612  6473 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:43.880553  6657 raft_consensus.cc:2376] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9. Re-sending same reply.
I20260709 12:18:43.882171  6652 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:43.874528  6656 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:18:43.885111  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:43.886469  6655 raft_consensus.cc:2410] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:18:43.835989 10761 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:43.827598 10345 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:43.892874  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:18:43.893918  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 46838.8us
I20260709 12:18:43.897535  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:43.897768  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:43.898147  6654 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:43.899811  6622 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:43.895694 10258 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:43.901034 10644 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:43.901397 10258 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:43.901825 10258 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:43.901626 10644 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:43.903951 10644 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:43.904125 10386 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:18:43.904639  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:43.905153  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:43.909814  6622 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:43.878293 10547 raft_consensus.cc:697] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:43.878401 10648 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:43.913131  6650 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:18:43.913906  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:43.917531  6649 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:18:43.890362 10472 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:18:43.919080  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 52667.3us
I20260709 12:18:43.889725 10708 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:43.896629 10720 raft_consensus.cc:2764] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
W20260709 12:18:43.949415  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 57980.6us
I20260709 12:18:43.952596  6548 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:43.953539  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:43.896778 10706 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:43.952194  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:43.959412  6652 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:18:43.954614  6657 raft_consensus.cc:2393] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 11: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:43.908466 10709 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:43.887406 10482 consensus_queue.cc:1059] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
W20260709 12:18:43.957633 10805 log.cc:927] Time spent T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 12:18:43.966138 10760 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:43.893487 10651 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:18:43.896152 10647 consensus_queue.cc:1048] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 12:18:43.966703 10547 consensus_queue.cc:237] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:43.900957 10382 consensus_queue.cc:1059] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:18:43.971149  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:43.895722 10653 consensus_queue.cc:1048] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 12:18:43.951448 10648 raft_consensus.cc:697] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:43.915606 10638 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:43.892745 10793 consensus_queue.cc:1059] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
W20260709 12:18:43.981086  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 101539us
I20260709 12:18:43.809881 10796 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:43.997761  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:44.001406  6654 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
W20260709 12:18:44.002707  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 93004.5us
W20260709 12:18:43.985162 10754 log.cc:927] Time spent T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:18:44.010152  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:44.007632  7266 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:18:43.993546 10546 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.014511  6652 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:44.017810  6473 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:43.955889 10708 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:44.003392 10794 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:44.002089 10689 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:44.034191 10382 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:44.026032 10638 raft_consensus.cc:697] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:43.977921 10706 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:44.125659 10638 consensus_queue.cc:237] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:44.126693 10706 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:44.030088  6473 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:44.097242 10642 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:18:44.127810  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 97794.4us
I20260709 12:18:44.127557 10642 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:44.128667 10382 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:44.129251 10382 raft_consensus.cc:697] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:44.130349 10642 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:44.130702 10382 consensus_queue.cc:237] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:44.034551 10546 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:44.041452 10802 log.cc:927] Time spent T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 12:18:44.042380 10761 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:44.042503 10555 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:18:44.046964  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.020637 10811 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:44.048823 10648 consensus_queue.cc:237] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:44.012825 10615 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:18:44.059394  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.029059  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:18:44.185621  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 156623us
I20260709 12:18:44.186694  6622 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:44.012919 10482 raft_consensus.cc:3055] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:44.035279 10794 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:43.990635 10760 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:44.059438 10799 log.cc:927] Time spent T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.283s	user 0.000s	sys 0.001s
I20260709 12:18:44.102360 10258 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:44.106789 10796 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:44.110002  6500 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 2187) took 1947 ms (client timeout 1845 ms). Trace:
I20260709 12:18:44.111855  6549 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:44.117576  6655 raft_consensus.cc:1275] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:18:44.247475  6500 rpcz_store.cc:269] 0709 12:18:42.162635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.162752 (+   117us) service_pool.cc:224] Handling call
0709 12:18:44.109805 (+1947053us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b54002c51d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:44.109926 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":410793344}
I20260709 12:18:44.118336  7271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38562 (request call id 686) took 1041 ms. Trace:
I20260709 12:18:44.119154  7270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38562 (request call id 680) took 1612 ms. Trace:
I20260709 12:18:44.120805 10606 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:44.134773  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.132426 10715 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:44.119869 10355 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:44.188252  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:44.173416 10810 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:44.275435  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.239836 10706 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:44.244427 10709 raft_consensus.cc:697] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 11 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:44.251237  7270 rpcz_store.cc:276] 0709 12:18:42.506327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:42.506557 (+   230us) service_pool.cc:224] Handling call
0709 12:18:42.520065 (+ 13508us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:42.884831 (+364766us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.118911 (+1234080us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b5400670cd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:44.119094 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":836997504}
I20260709 12:18:44.240274 10794 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:18:44.278396  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 166642us
I20260709 12:18:44.344568  6577 raft_consensus.cc:1240] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:44.346184  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:44.347456  6580 raft_consensus.cc:2410] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:18:44.350164  6575 raft_consensus.cc:1240] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:44.251125  7271 rpcz_store.cc:276] 0709 12:18:43.077220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:43.078268 (+  1048us) service_pool.cc:224] Handling call
0709 12:18:43.110647 (+ 32379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:43.230635 (+119988us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:44.118071 (+887436us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b54002e40d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:44.118271 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":263547776}
I20260709 12:18:44.240119 10813 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:18:44.236141 10482 raft_consensus.cc:740] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:18:44.314260  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 417426us
I20260709 12:18:44.357417  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:44.358070  6581 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:18:44.293357 10796 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:44.360572  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:44.314064 10801 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.357982  6475 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:18:44.325663  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 137529us
W20260709 12:18:44.312434 10609 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.006s	sys 0.030s
W20260709 12:18:44.389178 10609 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.798s	user 0.007s	sys 0.031s
I20260709 12:18:44.386912  6582 raft_consensus.cc:1275] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:18:44.391111  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:44.391889  6582 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:18:44.392947  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:44.392923 10709 consensus_queue.cc:237] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:44.394093  6582 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:44.396637 10258 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:44.360526 10796 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:44.398543 10796 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:44.400377  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:44.401139  6579 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:44.358987  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:44.402907  6582 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:18:44.362315  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:44.403877  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:44.406930  6582 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
W20260709 12:18:44.377353  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.365947  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:44.392169 10816 log.cc:927] Time spent T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:18:44.332706 10606 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:44.328516 10555 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:44.364583  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:44.410641  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.382480  6575 raft_consensus.cc:1275] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:44.415761  6582 raft_consensus.cc:1240] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:18:44.435732  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:44.436527  6579 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:44.332298 10706 raft_consensus.cc:697] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:44.386929 10499 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:44.431903 10779 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.839s	user 0.025s	sys 0.053s
I20260709 12:18:44.332321 10811 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.441468 10606 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:44.443362  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:44.445142  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 87231.9us
W20260709 12:18:44.459153  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 295883us
I20260709 12:18:44.459342  6582 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
I20260709 12:18:44.459596  6475 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:44.438774  6622 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:44.457270 10779 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.869s	user 0.026s	sys 0.054s
I20260709 12:18:44.365720 10649 consensus_queue.cc:1048] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.084s
W20260709 12:18:44.465142  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 56553.7us
I20260709 12:18:44.332656 10810 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:44.446007 10760 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:44.473121  6475 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:44.484756  6475 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:44.504710  6579 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:44.505358  6475 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:44.471462 10813 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:44.471448 10638 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:44.466152 10820 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:44.539680  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:44.433040 10801 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:44.514510 10546 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:44.549413  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:44.516989 10810 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:44.467073 10811 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:44.552646  6651 raft_consensus.cc:1240] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:44.531366  6582 raft_consensus.cc:1275] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 8 index: 3. Preceding OpId from leader: term: 12 index: 5. (index mismatch)
I20260709 12:18:44.537809  6579 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:44.546867  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:44.542876  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:44.554821  6658 raft_consensus.cc:2393] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 13: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:44.550201  6509 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
I20260709 12:18:44.464679 10499 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:44.509418 10671 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:44.488600 10819 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:44.474084 10555 consensus_queue.cc:1059] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 12:18:44.549539  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:44.562227  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:44.562722  6656 raft_consensus.cc:2393] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 13: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:44.559162  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:44.563577  6500 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 9.
I20260709 12:18:44.533026 10355 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:44.469929 10482 consensus_queue.cc:260] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:44.541477 10737 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.384s	user 0.014s	sys 0.008s
I20260709 12:18:44.570217  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:44.558665 10606 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:44.572564  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:44.601538 10606 raft_consensus.cc:697] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 12 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:44.602871 10606 consensus_queue.cc:237] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:44.599013  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:44.606189  6500 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:18:44.607298  6656 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:44.550300 10258 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:44.507329 10818 consensus_queue.cc:1059] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 12:18:44.615681  6651 raft_consensus.cc:1275] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 10 index: 4. Preceding OpId from leader: term: 12 index: 5. (index mismatch)
I20260709 12:18:44.552649 10638 raft_consensus.cc:697] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:44.555837 10820 raft_consensus.cc:697] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:44.625401  6507 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:44.560534 10819 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:44.560220 10671 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:44.617419  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:44.528360 10794 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:44.570076 10614 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.566299 10546 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:44.600854 10749 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.653743  6548 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:44.654783  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:44.600073 10499 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:44.639168 10638 consensus_queue.cc:237] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:44.528548 10706 consensus_queue.cc:237] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:44.645463 10482 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:44.571990 10649 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:44.650279 10716 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:44.651935  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:44.654003  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:44.612095 10737 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.462s	user 0.016s	sys 0.010s
I20260709 12:18:44.636469  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:44.571738 10813 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:44.652861  6656 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
W20260709 12:18:44.662595  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.571790 10651 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.656230  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:44.613216 10760 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.699376  6503 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:18:44.637189 10671 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:44.692044 10747 consensus_meta.cc:287] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.804s	user 0.012s	sys 0.001s
I20260709 12:18:44.637858 10819 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:44.707957  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 51791us
I20260709 12:18:44.691115 10818 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:44.713945 10813 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:44.706557 10791 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:44.716549 10651 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:44.717726 10811 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:44.718622 10820 consensus_queue.cc:237] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:44.670305 10614 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:44.670919 10822 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:44.708488 10501 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:44.702884 10664 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:44.670488 10546 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:44.670766 10749 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:44.762327  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.701937 10649 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:44.739445 10442 raft_consensus.cc:3055] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:44.692521 10757 consensus_queue.cc:1059] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.081s
I20260709 12:18:44.760504 10818 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:44.774241  6548 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:44.751032 10747 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:44.739002 10760 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:44.765453 10501 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:44.827481  6507 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:44.811542  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.765997 10822 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:18:44.804409 10795 consensus_meta.cc:287] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.975s	user 0.007s	sys 0.007s
I20260709 12:18:44.761809 10791 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:44.798727 10442 raft_consensus.cc:740] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:44.862282  6581 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 8. Current term is 10. Ops: []
I20260709 12:18:44.795058 10664 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:44.880510  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:44.881451  6505 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:44.839356 10795 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:44.833236 10822 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:44.889858  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:44.830015 10501 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:44.882217  6507 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:18:44.834096 10779 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:44.905905 10442 consensus_queue.cc:260] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:44.907454 10442 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:44.915585  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:44.922493  6622 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.857s (SENT) [suppressed 13 similar messages]
I20260709 12:18:44.902674 10801 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:44.955276 10796 consensus_meta.cc:287] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.556s	user 0.000s	sys 0.009s
I20260709 12:18:44.956825 10749 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:44.957701 10696 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:44.958309 10664 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:44.958498 10836 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:18:44.959503  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:44.963855 10821 log.cc:927] Time spent T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.454s	user 0.001s	sys 0.002s
I20260709 12:18:44.964985 10760 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:44.965852 10790 raft_consensus.cc:3055] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:44.966480 10790 raft_consensus.cc:740] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:44.965896 10649 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:44.967306 10830 log.cc:927] Time spent T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 12:18:44.955834 10796 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:44.970144 10796 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:44.973905 10651 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:44.978593 10546 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:44.979740 10813 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:44.980185 10795 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:44.981611 10671 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:44.982648 10791 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:44.982959  6502 raft_consensus.cc:2368] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11.
I20260709 12:18:44.903470 10779 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:44.990285  6653 raft_consensus.cc:1240] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10. Ops: []
W20260709 12:18:44.970465 10828 log.cc:927] Time spent T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.205s	user 0.000s	sys 0.001s
I20260709 12:18:44.967585 10790 consensus_queue.cc:260] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:44.997565  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:45.003242  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:45.009254  6500 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 15: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:45.005009  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:44.996042 10779 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:44.985740 10614 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:45.008185  6653 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 6.
I20260709 12:18:45.078869  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:45.001500 10651 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:18:45.090520  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 168061us
I20260709 12:18:44.997144 10768 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:45.091128  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 61078.5us
I20260709 12:18:45.093365  6658 raft_consensus.cc:2393] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 11: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:45.017364 10794 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:45.099349  6653 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
W20260709 12:18:45.088263 10828 log.cc:927] Time spent T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:18:45.104137  6653 pending_rounds.cc:85] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 4
I20260709 12:18:45.109262  6653 pending_rounds.cc:107] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:18:45.109606  6653 raft_consensus.cc:2889] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:45.122665 10501 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:45.096885 10651 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:45.101305 10768 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:45.117151 10671 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:45.104496 10747 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:45.146245  6505 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:18:45.148751  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2215) took 1448 ms. Trace:
I20260709 12:18:45.149608  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:45.149924 10518 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:45.150498 10481 raft_consensus.cc:3055] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:45.152904  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:45.153023  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:45.092965 10258 raft_consensus.cc:697] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:45.153707  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:45.101168 10790 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:45.238735  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:45.241529  6583 raft_consensus.cc:2410] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:18:45.165647  6582 raft_consensus.cc:1240] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:45.211508  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:45.281894  6658 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:18:45.219635  6656 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:45.177781  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:45.287266  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:18:45.299865  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.294268 10621 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.298883 10258 consensus_queue.cc:237] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:45.313189 10621 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:45.315500 10501 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:45.298399 10768 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:45.280705 10760 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.317368  7266 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
I20260709 12:18:45.211084  6507 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:45.286682 10491 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.305429  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1959) took 1413 ms. Trace:
I20260709 12:18:45.345082  6655 rpcz_store.cc:276] 0709 12:18:43.891953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:43.892501 (+   548us) service_pool.cc:224] Handling call
0709 12:18:43.893414 (+   913us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:44.247892 (+354478us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:45.305131 (+1057239us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b540019d7d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:45.305358 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":514533120}
I20260709 12:18:45.346866  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:45.347765  6655 raft_consensus.cc:2393] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 9: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:45.316246  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:45.348863  6651 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:45.311722 10716 raft_consensus.cc:3055] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:45.304678  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:45.354466  6649 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:45.352006  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 348365us
I20260709 12:18:45.238219 10758 raft_consensus.cc:3055] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:45.317453 10760 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:45.282824 10481 raft_consensus.cc:740] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:45.166118 10781 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.297526 10708 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:45.318898  6658 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:45.387569 10781 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:45.388927  6579 raft_consensus.cc:1240] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:45.389761 10781 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:45.238618 10651 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:45.161376 10614 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:45.391191 10614 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:45.372712 10491 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:45.393440 10614 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:45.394766 10491 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:45.323032 10385 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:45.322402 10822 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:45.306036 10482 consensus_meta.cc:287] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.606s	user 0.006s	sys 0.006s
I20260709 12:18:45.323910  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:45.324055 10389 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:45.322630 10606 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.323689  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:45.328195  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:45.403512  7265 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
W20260709 12:18:45.177623  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.282806  6508 rpcz_store.cc:276] 0709 12:18:43.700225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:43.851162 (+150937us) service_pool.cc:224] Handling call
0709 12:18:43.851593 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:43.874655 (+ 23062us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:43.874692 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:43.888514 (+ 13822us) log.cc:844] Serialized 26 byte log entry
0709 12:18:43.953845 (+ 65331us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:43.982119 (+ 28274us) log.cc:844] Serialized 14 byte log entry
0709 12:18:43.982769 (+   650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:43.982853 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:45.141529 (+1158676us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b1800404e00. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:18:45.148416 (+  6887us) raft_consensus.cc:1685] finished
0709 12:18:45.148535 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:45.148686 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3368,"prepare.queue_time_us":28268,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":15320,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":1913,"spinlock_wait_cycles":36419584,"thread_start_us":32111,"threads_started":1,"wal-append.queue_time_us":101454}
I20260709 12:18:45.330950  6580 raft_consensus.cc:2393] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 14: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:45.294268  6550 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:45.404589  6548 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:18:45.404878  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 110734us
I20260709 12:18:45.243326  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:45.292706  6578 raft_consensus.cc:2368] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 8. Current term is 9.
I20260709 12:18:45.282096  6503 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:18:45.351286  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:18:45.363858  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.159930 10813 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:45.357039  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:45.286067 10595 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.405716  6550 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:45.364614 10768 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:45.165658 10715 consensus_meta.cc:287] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.836s	user 0.002s	sys 0.011s
I20260709 12:18:45.384658  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:45.411928  6500 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:45.353518 10716 raft_consensus.cc:740] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:45.415481 10472 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.416198 10472 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:45.414427 10708 consensus_queue.cc:1059] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:18:45.417232 10708 raft_consensus.cc:3055] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:45.418418  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:45.419029  6578 raft_consensus.cc:2376] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8. Re-sending same reply.
I20260709 12:18:45.386456 10481 consensus_queue.cc:260] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:45.399307  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.420389  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:45.421041  6578 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:45.239126 10818 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:45.398429  6658 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 9.
I20260709 12:18:45.422564  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:45.243604 10819 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:45.402241  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:45.403707  7274 raft_consensus.cc:2393] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 13: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:45.418589 10491 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:45.407496  6655 raft_consensus.cc:2393] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 6: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:45.455528  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:45.456643  6655 raft_consensus.cc:2393] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 9: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:18:45.407511  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.408197  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:45.409440 10822 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:45.412460  6622 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:45.362522 10758 raft_consensus.cc:740] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:45.276739 10779 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:45.404773 10385 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:45.443041 10481 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:45.380177  6649 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:45.476313  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:45.477119  6649 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:45.424181  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:45.479015  6658 raft_consensus.cc:2410] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:45.425324  7240 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:45.426002  6584 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:45.421926  6550 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:45.430526  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:45.440057  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:45.481163  6577 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:45.467109  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:45.487921  6655 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:45.489632  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:45.410327 10606 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:45.418702 10813 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:45.415539 10716 consensus_queue.cc:260] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:45.329137  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:45.503527  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:45.510099  6655 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:18:45.450603  6576 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:45.466425 10491 raft_consensus.cc:697] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:45.451926  6500 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
I20260709 12:18:45.457721  7272 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:18:45.421257 10644 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:45.459331  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 46942.4us
I20260709 12:18:45.460930  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:45.414399 10607 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:45.406672 10389 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:45.417564 10708 raft_consensus.cc:740] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:45.461256  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:45.499713  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:45.506080 10472 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:45.412371 10482 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:45.552421  6652 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:45.515132 10795 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:45.555696  6652 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11. Ops: []
W20260709 12:18:45.558876  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:45.564435  6584 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1977) took 2493 ms (client timeout 1857 ms). Trace:
W20260709 12:18:45.504405  6504 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 2216) took 1788 ms (client timeout 2244 ms). Trace:
W20260709 12:18:45.505810  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 91681.6us
I20260709 12:18:45.441282  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:45.460424 10442 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:45.422129 10768 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:45.499401 10761 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.525708  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:18:45.565158  6504 rpcz_store.cc:269] 0709 12:18:43.716703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:43.800703 (+ 84000us) service_pool.cc:224] Handling call
0709 12:18:45.504194 (+1703491us) spinlock_profiling.cc:243] Waited 42.5 ms on lock 0x7b54000f3554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:45.504346 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":97821952}
I20260709 12:18:45.569582  6651 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:45.528487  6622 leader_election.cc:400] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 8. Current term is 9.
I20260709 12:18:45.519902  6577 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
I20260709 12:18:45.571388  6622 leader_election.cc:403] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:45.532223  6658 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
I20260709 12:18:45.533057  6474 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:45.572072  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:45.535490  6649 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:18:45.525413  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:45.572772  6577 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
W20260709 12:18:45.516659 10825 log.cc:927] Time spent T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.808s	user 0.005s	sys 0.000s
W20260709 12:18:45.582034  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 49053.6us
I20260709 12:18:45.583499  6474 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:45.527173 10716 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:45.564975  6584 rpcz_store.cc:269] 0709 12:18:43.062064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:43.064506 (+  2442us) service_pool.cc:224] Handling call
0709 12:18:45.555444 (+2490938us) spinlock_profiling.cc:243] Waited 2.32 s on lock 0x7b54004303d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:45.555554 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5338539904}
I20260709 12:18:45.506130  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 2013) took 1144 ms. Trace:
I20260709 12:18:45.555133 10795 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:45.418639 10595 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:45.552457 10760 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:45.551964 10607 raft_consensus.cc:697] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 9 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:45.589494  6474 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:45.596927  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:45.597530  6504 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:45.598532  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:45.599231  6500 raft_consensus.cc:2393] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 11: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:45.602523  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:45.603170  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:45.603478  6508 raft_consensus.cc:2393] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 14: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:45.556254 10482 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:45.592808  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:45.608088  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:45.537703 10644 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:45.609372  6582 raft_consensus.cc:2410] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 11 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:18:45.609818  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:45.610675  6508 raft_consensus.cc:2410] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
W20260709 12:18:45.567934 10845 log.cc:927] Time spent T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.558s	user 0.000s	sys 0.009s
I20260709 12:18:45.593261  6654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1970) took 1026 ms. Trace:
I20260709 12:18:45.587847  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:45.568629  6578 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:18:45.604326  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:45.604516  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:45.618958  6654 rpcz_store.cc:276] 0709 12:18:44.566788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.606934 (+ 40146us) service_pool.cc:224] Handling call
0709 12:18:44.628573 (+ 21639us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:44.835991 (+207418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:44.836023 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:44.836946 (+   923us) log.cc:844] Serialized 52 byte log entry
0709 12:18:44.948016 (+111070us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:18:44.948067 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:45.572008 (+623941us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b54002a53d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:45.572065 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:45.572136 (+    71us) raft_consensus.cc:1685] finished
0709 12:18:45.572158 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:45.593192 (+ 21034us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6200,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":29677,"spinlock_wait_cycles":460508544,"thread_start_us":110613,"threads_started":1,"wal-append.queue_time_us":185068}
I20260709 12:18:45.593647 10795 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:45.599924  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:45.622617  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:45.594691 10631 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:45.606782  6506 raft_consensus.cc:1240] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:45.607154  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:45.609072  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:45.585703  6583 rpcz_store.cc:276] 0709 12:18:44.361963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.362082 (+   119us) service_pool.cc:224] Handling call
0709 12:18:45.505879 (+1143797us) spinlock_profiling.cc:243] Waited 836 ms on lock 0x7b540034d5d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:45.506055 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1923410176}
I20260709 12:18:45.614259 10607 consensus_queue.cc:237] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:45.616266 10644 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:45.616398 10595 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:45.593354 10820 raft_consensus.cc:2764] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election decision vote started in defunct term 9: won
I20260709 12:18:45.589529 10818 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 8. Current term is 9.
I20260709 12:18:45.589675 10761 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:45.565382 10758 consensus_queue.cc:260] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:45.621115  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:45.572827  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 44605us
I20260709 12:18:45.621649  6658 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:45.621950  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:45.620965  6504 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:45.601167 10258 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:45.623658  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:45.623250  6584 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:18:45.657109  6502 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 2228) took 1005 ms. Trace:
I20260709 12:18:45.719640  6502 rpcz_store.cc:276] 0709 12:18:44.651589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.654534 (+  2945us) service_pool.cc:224] Handling call
0709 12:18:45.656890 (+1002356us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b54002c51d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:45.657038 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":752650496}
I20260709 12:18:45.679355  6650 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:18:45.624181  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:45.713415  6578 raft_consensus.cc:2410] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:18:45.625151  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:45.625540  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:45.731463  6508 raft_consensus.cc:2393] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 8: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:45.610340 10482 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:45.634363 10606 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:45.739449  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:45.715579 10258 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:45.691191 10779 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:45.748831 10258 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:45.639911 10819 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:45.655192 10768 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:45.646430 10813 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:45.755851  6507 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
W20260709 12:18:45.756070  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.701871 10691 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:45.645181 10385 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:45.760710  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:45.762950  6501 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:18:45.763410  6578 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:45.763826  6651 raft_consensus.cc:2410] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
W20260709 12:18:45.762303  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.762243  6577 raft_consensus.cc:1275] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:45.766997  6622 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:45.768044  6575 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:45.766211 10813 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.770545  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:45.699659 10389 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:45.769126 10813 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:45.606673 10708 consensus_queue.cc:260] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:45.757241 10779 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:45.781071  6622 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:45.634855 10491 consensus_queue.cc:237] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:45.767752 10442 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.767637 10818 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:45.790777  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:45.819805  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:45.817503 10779 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:45.828147  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:45.828598 10615 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:45.784974 10386 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:18:45.811082  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.832086 10779 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:45.787837  6655 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:45.770263 10691 raft_consensus.cc:697] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 9 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:45.840680 10610 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.840553 10691 consensus_queue.cc:237] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:45.841461 10610 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:45.834583  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:45.816447 10595 consensus_queue.cc:1048] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 12:18:45.837010 10709 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:45.846956  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:45.776980 10760 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:18:45.857115  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.845916  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:45.858476  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:45.861292  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:45.859536  6622 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:45.785292 10696 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:45.863688 10696 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:45.866695 10696 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:45.870994 10482 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:45.876614 10868 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:45.877405 10868 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:45.877967 10868 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:45.849100 10610 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:45.855878 10709 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:45.884563 10709 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:45.885697  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:45.888721  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:45.889389  7274 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:45.849205 10442 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:45.893908  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:45.894255  7268 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 12 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:45.895087  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:45.898020  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 472759us
W20260709 12:18:45.898593  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.900089  6657 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:18:45.900280  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:45.901484  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.898566  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:45.909188  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:45.910198  6651 raft_consensus.cc:2393] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 10: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:18:45.912652  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:45.913087  7272 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:18:45.914031  6549 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:45.914121  7238 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.033s	sys 0.065s
W20260709 12:18:45.916678  7238 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.033s	sys 0.065s
I20260709 12:18:45.914009  7240 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:45.925046  7274 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:18:45.927284  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:45.928233  6655 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 6.
I20260709 12:18:45.926632  6622 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:45.931434  6549 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:45.935451  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2009) took 1576 ms. Trace:
I20260709 12:18:45.935978  6581 rpcz_store.cc:276] 0709 12:18:44.359113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:44.359344 (+   231us) service_pool.cc:224] Handling call
0709 12:18:44.386153 (+ 26809us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:44.863091 (+476938us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:45.935227 (+1072136us) spinlock_profiling.cc:243] Waited 476 ms on lock 0x7b54002661d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:45.935397 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1094321024}
I20260709 12:18:45.934742  6622 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:45.943245  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:45.849249 10818 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:45.922606 10760 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:45.921273 10865 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:45.938565 10386 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:45.934302 10606 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:45.942862 10481 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:45.934142 10738 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:45.959986  6549 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:18:45.966699  6549 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:18:45.969170  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 44158.2us
I20260709 12:18:45.970530 10739 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:45.971287 10739 raft_consensus.cc:697] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:45.972705 10739 consensus_queue.cc:237] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:45.948460 10610 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:45.989614  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:45.953909 10760 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:45.997808  7238 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:45.955567 10615 raft_consensus.cc:697] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 8 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:46.001088  7238 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:45.958976 10606 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:45.957432 10386 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:46.008891  7238 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:46.012221  7238 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:46.002537 10481 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:46.004309 10760 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:46.024930  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 84648.8us
I20260709 12:18:45.956853 10795 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:46.004842 10385 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:45.986829 10607 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:46.010231 10606 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:46.013198 10386 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:46.011385 10128 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:45.959913 10865 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:46.013772 10768 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:46.062834  6584 raft_consensus.cc:2393] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 13: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:46.025982 10481 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:46.036999 10385 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:46.055794 10691 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:46.010728 10867 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 12:18:46.062495 10128 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:46.060088 10876 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:46.059868 10607 raft_consensus.cc:697] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:46.059923 10813 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:46.026096 10874 consensus_queue.cc:1048] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 12:18:46.072710 10481 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:46.081008 10385 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:46.125113  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:46.097947 10795 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:46.071486 10442 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:46.099181 10882 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.101992 10614 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.136371 10614 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:46.139138 10614 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:46.076799 10615 consensus_queue.cc:237] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:46.141171  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:46.142108  6651 raft_consensus.cc:2410] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:46.142933  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:46.143277 10882 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:46.144042  7274 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:46.144260  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:46.144577  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:46.144902  7272 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
I20260709 12:18:46.145249  6581 raft_consensus.cc:2393] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 12: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:46.145737 10882 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:46.146159  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:46.146744  6581 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:18:46.146893 10607 consensus_queue.cc:237] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:46.074363 10482 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:46.123085  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:46.094503 10776 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:46.131731  6577 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:46.161994  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:46.162875  6577 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:46.132704  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:46.088613 10128 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:46.134500  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:46.114166  6550 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:46.089072 10876 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:18:46.127177 10871 log.cc:927] Time spent T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
W20260709 12:18:46.127301 10869 log.cc:927] Time spent T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
W20260709 12:18:46.129525 10870 log.cc:927] Time spent T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 12:18:46.127120 10812 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:18:46.166272  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 52236.8us
I20260709 12:18:46.129243 10644 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:46.145272 10258 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:46.167426  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:46.191442  6658 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:18:46.173359  6473 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:46.155148 10790 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.194382 10606 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:46.194785 10790 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:46.197185 10790 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:46.174355  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:46.172791 10482 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:46.157536 10651 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:46.167584 10876 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:46.204344  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:46.207432  6651 raft_consensus.cc:2393] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 11: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:46.193560  6550 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:46.199498 10872 log.cc:927] Time spent T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:18:46.186589 10818 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:46.217890  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 189034us
I20260709 12:18:46.219694  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:18:46.206403 10790 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:46.186901 10812 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:46.227175  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:46.228010  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:46.218461 10738 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:46.192795 10644 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:46.213693 10874 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.233667  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:46.224663 10691 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:46.211499 10651 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:46.234066  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:46.241933  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:46.237372  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:46.244819 10651 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:46.245131  6581 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:18:46.245469 10651 consensus_queue.cc:1059] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:46.246476  6550 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:46.249146 10795 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:46.249657 10795 raft_consensus.cc:697] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:46.250561 10795 consensus_queue.cc:237] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:46.254073 10621 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:46.254657  6550 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:46.239900  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:46.255940 10691 raft_consensus.cc:697] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 10 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:46.257023  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:46.256925 10691 consensus_queue.cc:237] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:46.257776  6651 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:46.255439  7269 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:46.260901 10389 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.261236  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:46.261392 10389 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:46.262465  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:46.263466  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:46.264096  6583 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:46.269145  6579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 1983) took 1115 ms. Trace:
I20260709 12:18:46.269574  6579 rpcz_store.cc:276] 0709 12:18:45.153580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:45.153899 (+   319us) service_pool.cc:224] Handling call
0709 12:18:45.260219 (+106320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:45.389301 (+129082us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.268869 (+879568us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b5400031e54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:46.269063 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":295261056}
I20260709 12:18:46.270640 10621 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:46.271097  6579 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:18:46.271555  6579 pending_rounds.cc:85] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 9
I20260709 12:18:46.271166 10621 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:46.271782  6579 pending_rounds.cc:107] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:18:46.272035  6579 raft_consensus.cc:2889] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:46.264967  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:46.272583  6654 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:46.273361 10621 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:46.275171  6576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1982) took 1121 ms. Trace:
I20260709 12:18:46.275660 10834 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.275558  6576 rpcz_store.cc:276] 0709 12:18:45.153268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:45.153488 (+   220us) service_pool.cc:224] Handling call
0709 12:18:46.275002 (+1121514us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b540034bcd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:46.275112 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":279874432}
I20260709 12:18:46.276453 10834 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:46.277208  7266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38546 (request call id 486) took 1059 ms. Trace:
I20260709 12:18:46.277719  7266 rpcz_store.cc:276] 0709 12:18:45.218340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:45.218604 (+   264us) service_pool.cc:224] Handling call
0709 12:18:46.277048 (+1058444us) spinlock_profiling.cc:243] Waited 34.2 ms on lock 0x7b54005ddc54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:46.277151 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":78637440}
I20260709 12:18:46.278306  6653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2461) took 1140 ms. Trace:
I20260709 12:18:46.278879 10834 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:46.278676  6653 rpcz_store.cc:276] 0709 12:18:45.138077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:45.157416 (+ 19339us) service_pool.cc:224] Handling call
0709 12:18:45.157784 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:45.551517 (+393733us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:45.567707 (+ 16190us) log.cc:844] Serialized 14 byte log entry
0709 12:18:46.025499 (+457792us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:46.025553 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:46.025603 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.278068 (+252465us) spinlock_profiling.cc:243] Waited 393 ms on lock 0x7b540044b1d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:46.278117 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:46.278248 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":37529,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":6463,"raft.queue_time_us":134505,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":25613,"spinlock_wait_cycles":904661888,"thread_start_us":457294,"threads_started":2,"wal-append.queue_time_us":436508}
I20260709 12:18:46.281369  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:46.281986  6653 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:46.282321 10709 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:46.283815 10631 raft_consensus.cc:3055] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:18:46.286309  6393 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 3e0ab356fc074f589a6ff4f9b4996b09 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 20a202c76244485fbc8f1610473db8b9 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:18:46.287554  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:46.288208  6658 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
W20260709 12:18:46.289155 10862 log.cc:927] Time spent T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.671s	user 0.003s	sys 0.000s
I20260709 12:18:46.285372 10386 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:46.289064 10760 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:46.291520 10760 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:45.614478  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:46.295372  6577 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:18:46.295276 10868 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:46.241452 10865 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:46.297567 10859 log.cc:927] Time spent T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.672s	user 0.000s	sys 0.002s
W20260709 12:18:46.299067 10873 log.cc:927] Time spent T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 12:18:46.302691 10644 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:46.241639 10874 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:46.305620 10812 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:46.239300  7238 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:46.329630  6506 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:46.312297  6651 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:18:46.330680  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:46.331559  6501 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:46.315044  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:46.315392  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:46.333024  6505 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:46.334196  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:46.335384  6505 raft_consensus.cc:2410] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 13 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:18:46.232568 10738 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:46.319314  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:46.319813  6504 raft_consensus.cc:1275] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 12:18:46.321205  6549 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:46.321645  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:46.324177  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 69602.1us
I20260709 12:18:46.307870 10886 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:46.330528  6654 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
W20260709 12:18:46.342459  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 103241us
W20260709 12:18:46.312979  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 57354.5us
I20260709 12:18:46.343492  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:46.344079  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:46.344528  6623 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:46.344791  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:46.337188  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.306891 10631 raft_consensus.cc:740] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:46.348371  6623 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:46.349557  7238 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:46.350100  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:46.350379  6622 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:46.352818  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:46.336709  6583 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
I20260709 12:18:46.358278  6579 raft_consensus.cc:1275] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:18:46.361361  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:46.364399  6622 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:46.366844  6550 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:46.329197 10716 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:46.368028  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:46.345930 10738 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:46.342159 10779 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:46.394029  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:46.407176 10128 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:46.358466 10866 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:46.399470  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:46.363077 10760 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.085s
I20260709 12:18:46.360002 10886 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:46.455927  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:46.368707 10442 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:46.447103  6657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58980 (request call id 2477) took 1234 ms. Trace:
W20260709 12:18:46.449028  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.541986  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1991) took 1039 ms. Trace:
I20260709 12:18:46.542603  6582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1990) took 1089 ms. Trace:
I20260709 12:18:46.373270  6550 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:46.403882 10781 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:46.434980 10874 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:46.443195 10389 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:46.450661 10867 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.135s
I20260709 12:18:46.559151 10738 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:46.443251 10876 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:46.471952  7240 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:46.368733 10878 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:18:46.605406  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 133524us
I20260709 12:18:46.444068 10258 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:46.363320 10818 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:46.359277 10813 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:46.368594 10882 consensus_queue.cc:1059] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.198s
I20260709 12:18:46.363242 10880 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:46.369226 10614 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:46.403726 10709 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:46.408628 10595 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:46.431166 10128 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:46.409291 10779 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:46.624738 10779 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:46.630319 10595 raft_consensus.cc:697] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:46.618930 10644 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:46.630733 10385 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:46.631381 10595 consensus_queue.cc:237] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:46.633069 10385 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:46.613603 10389 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:46.634769 10389 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:46.635169 10389 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:46.635521 10876 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:46.614780 10878 raft_consensus.cc:697] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 11 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:46.631506 10644 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:46.641821 10878 consensus_queue.cc:237] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:46.621719 10880 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:46.643451 10880 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:46.568897  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.445242 10631 consensus_queue.cc:260] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:46.365283 10491 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.207s
I20260709 12:18:46.363793 10501 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:18:46.441159 10858 log.cc:927] Time spent T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.835s	user 0.002s	sys 0.002s
W20260709 12:18:46.577529  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 204332us
I20260709 12:18:46.574975  7267 raft_consensus.cc:2393] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 10: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:46.363389 10768 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:46.576793  6584 rpcz_store.cc:276] 0709 12:18:45.503078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:45.717742 (+214664us) service_pool.cc:224] Handling call
0709 12:18:46.532304 (+814562us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b54001d15d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:46.541905 (+  9601us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":741458688}
I20260709 12:18:46.576557  6657 rpcz_store.cc:276] 0709 12:18:45.212603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:45.362141 (+149538us) service_pool.cc:224] Handling call
0709 12:18:46.446870 (+1084729us) spinlock_profiling.cc:243] Waited 455 ms on lock 0x7b540008f7d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:46.447024 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1045391744}
I20260709 12:18:46.577040  6582 rpcz_store.cc:276] 0709 12:18:45.453279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:45.460633 (+  7354us) service_pool.cc:224] Handling call
0709 12:18:46.533126 (+1072493us) spinlock_profiling.cc:243] Waited 81.3 ms on lock 0x7b5400298e54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:46.542546 (+  9420us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":187032448}
I20260709 12:18:46.472420  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:46.442344 10386 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:46.435395 10761 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:46.446364 10790 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:46.592819  6651 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:46.587587 10760 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.584645 10472 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:18:46.650115 10481 consensus_meta.cc:287] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.516s	user 0.008s	sys 0.002s
I20260709 12:18:46.447535 10868 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:46.463826  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:46.610948  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:46.614504  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:46.707901  6507 raft_consensus.cc:2410] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:18:46.708146  6653 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:46.644275 10644 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:46.619787 10887 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:46.645781 10886 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:46.713404 10389 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:46.690923 10760 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:46.717641  6653 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
I20260709 12:18:46.718782  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:46.710824 10887 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:46.719591  6653 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 16: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:46.720973  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:46.721632 10887 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:46.675664 10867 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 12:18:46.722796  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:46.620754  6474 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:46.622351  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:46.725219  6655 raft_consensus.cc:2393] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 10: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:46.707223 10481 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:46.725152  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:46.651353  6658 raft_consensus.cc:1275] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:18:46.726795 10868 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.727298 10868 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:46.665236  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:46.729351  6582 raft_consensus.cc:2393] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 10: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:46.665760  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:46.672967  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:46.681985  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:46.732209  6656 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:46.675630  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:46.733644  6649 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:46.734001  6653 raft_consensus.cc:2393] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 10: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:46.736765  6583 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:46.685597  6550 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:46.691560  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:46.598650  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 396262us
I20260709 12:18:46.740846  6473 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:46.690821 10472 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:46.652980 10631 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:46.646804 10738 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:46.697228  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:46.586535 10781 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:46.707870  6654 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 9.
W20260709 12:18:46.709921  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.749632  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:46.743917  6473 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:46.751834  6653 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 12. Current term is 13. Ops: []
W20260709 12:18:46.752542  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 67014.4us
I20260709 12:18:46.753232  6650 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:46.752164 10472 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:46.769709  6550 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:46.775803  6656 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 12. Current term is 13. Ops: []
W20260709 12:18:46.775733  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.785964  6502 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:18:46.764819 10481 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:46.764063  6583 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:18:46.788276  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:46.789021  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:46.806274  6508 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
I20260709 12:18:46.791846  6550 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:46.808012  6550 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:46.792929  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 425759us
I20260709 12:18:46.808904 10623 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:46.752708 10887 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:46.775063 10482 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:46.789844 10790 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.800585  6649 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:46.717639 10760 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:46.813449  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:46.796438 10779 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:46.826663  6622 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:46.812103 10389 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:46.805416 10386 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:46.804718 10820 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:18:46.765571 10258 consensus_queue.cc:1048] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 12:18:46.832223 10868 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:46.826969 10887 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 12:18:46.834789 10758 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:46.813547 10795 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:46.851125 10781 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:46.824970 10790 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:46.854243 10790 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:46.843844 10795 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:46.855257 10795 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:46.827795 10878 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:46.825018 10880 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:46.866233 10880 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:46.860781 10878 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:46.867865 10878 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:46.871413 10795 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:46.903954  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 283034us
W20260709 12:18:46.906716  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 80028.5us
I20260709 12:18:46.810550  6550 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:46.913254  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:18:46.914531  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 103915us
I20260709 12:18:46.914355  6504 raft_consensus.cc:1275] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:46.914722  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:46.916857  6508 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 12:18:46.913273  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.798190  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:46.913267  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:46.925448  6655 raft_consensus.cc:1240] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:46.932945  6502 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 16: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:46.841518 10386 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:46.938062 10386 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:46.933059 10491 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:46.928169  6475 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:46.940290 10834 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:46.941032 10834 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:46.941412  6581 raft_consensus.cc:1275] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:46.941740 10834 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:46.945261 10880 consensus_queue.cc:1048] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:46.940143  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:46.948987 10631 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:46.847236 10623 raft_consensus.cc:697] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:46.951203  6550 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:46.953907 10779 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:47.000679 10790 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:46.957631  6402 catalog_manager.cc:5697] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:18:47.001750 10790 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:46.962857  6575 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 1995) took 1437 ms (client timeout 1857 ms). Trace:
W20260709 12:18:47.004570  6575 rpcz_store.cc:269] 0709 12:18:45.525272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:45.575248 (+ 49976us) service_pool.cc:224] Handling call
0709 12:18:46.962262 (+1387014us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b54004303d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:46.962555 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":342117120}
I20260709 12:18:46.949697 10631 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:47.006474 10631 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:47.009835  7238 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:18:47.011873 10385 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:47.012590 10385 raft_consensus.cc:697] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 8 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:47.014657  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:46.964685  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 663) took 1056 ms. Trace:
I20260709 12:18:46.967610 10644 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:46.949047 10880 consensus_queue.cc:1059] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 12:18:46.968473  6475 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:46.969098  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 190355us
I20260709 12:18:46.974298 10708 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:46.977140  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:46.981827  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:47.028177  6508 raft_consensus.cc:2393] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 8: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:47.020481  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:46.982955 10761 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:46.984290  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:47.029109  7269 raft_consensus.cc:2393] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 13: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:47.030390  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:47.031479  7272 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:18:47.021344 10790 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:47.019658  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:46.989549  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:47.004385 10623 consensus_queue.cc:237] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:46.967301 10501 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:47.027896  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:46.997140  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:47.014251 10385 consensus_queue.cc:237] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:47.021502  7267 raft_consensus.cc:1240] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:47.016288  7240 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:47.048676  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 67498.4us
I20260709 12:18:47.037612 10491 raft_consensus.cc:697] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:47.051285  7265 raft_consensus.cc:2393] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 7: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:47.051501 10491 consensus_queue.cc:237] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:47.052897  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:47.054239  6509 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:47.055207 10795 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:47.055787 10790 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:18:47.056303  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 88060.1us
I20260709 12:18:47.057806  6475 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:47.059525 10651 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:47.060190 10651 raft_consensus.cc:697] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:18:47.061523  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 110596us
I20260709 12:18:47.061669 10651 consensus_queue.cc:237] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:47.063203 10606 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:47.064909 10747 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:47.065410 10747 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:47.065801 10747 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:47.069324  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:47.070235  6653 raft_consensus.cc:2393] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 7: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:47.019999  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:47.072853  6402 catalog_manager.cc:5697] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 3 to 6, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 6 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:18:47.086699  6502 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
W20260709 12:18:47.086776  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 71000.6us
I20260709 12:18:47.098405  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:47.099303  6650 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:47.101604  7270 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 16: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:47.103008 10865 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:18:47.110400  6473 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:47.103611 10865 raft_consensus.cc:697] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 16 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:47.119643  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:47.119772 10790 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:47.120736  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:47.121824 10610 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.122543 10610 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:47.124758  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 64599.7us
I20260709 12:18:47.128664 10812 consensus_queue.cc:1059] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:47.129454 10491 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.129892 10644 raft_consensus.cc:3055] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:47.130530 10644 raft_consensus.cc:740] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:47.130162 10491 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:47.131882 10644 consensus_queue.cc:260] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:47.133158 10491 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:47.133785 10644 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:47.133922 10761 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:47.134619 10761 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:47.127933  6475 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:47.128224  7274 raft_consensus.cc:1240] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:47.104082 10386 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:18:47.145262  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 75963.5us
I20260709 12:18:47.145984  7240 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:47.145720  6402 catalog_manager.cc:5697] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 4 to 6, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 6 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:47.146670 10877 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:47.148864 10878 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:47.150202 10790 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:47.151906 10812 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:47.165154 10874 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.153147 10887 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:47.165870 10874 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:47.169946  7268 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:18:47.116268 10795 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 12:18:47.158223 10644 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:18:47.163058 10790 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:47.174115 10790 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:47.172047  6550 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:47.176739  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 61872.2us
I20260709 12:18:47.177186  6550 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:47.178376  6550 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:47.178960  6548 leader_election.cc:400] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11.
I20260709 12:18:47.173044 10386 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:47.084487 10472 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:47.180631  7238 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:47.181257 10386 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:47.181607 10472 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:47.181963  7240 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:47.182193 10795 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:47.182554 10472 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:47.021733  6652 rpcz_store.cc:276] 0709 12:18:45.908409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:45.908514 (+   105us) service_pool.cc:224] Handling call
0709 12:18:45.908811 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:45.988317 (+ 79506us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:18:46.008746 (+ 20429us) log.cc:844] Serialized 14 byte log entry
0709 12:18:46.077569 (+ 68823us) log.cc:844] Serialized 14 byte log entry
0709 12:18:46.217288 (+139719us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:46.217339 (+    51us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:46.217381 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:46.256102 (+ 38721us) spinlock_profiling.cc:243] Waited 2.11 ms on lock 0x7b180018b8c0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d16c0a 00007f287c456219 00007f287adb2328 00007f287ada9901 00007f287ada98a7 00007f287aea1288 00007f287aea0f37 00007f287aeb555c 00007f287aeb5508
0709 12:18:46.963918 (+707816us) spinlock_profiling.cc:243] Waited 79.3 ms on lock 0x7b54000b6154. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:46.964055 (+   137us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:46.964321 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8210,"prepare.queue_time_us":20872,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":80,"raft.queue_time_us":95747,"raft.run_cpu_time_us":2373,"raft.run_wall_time_us":134819,"spinlock_wait_cycles":187132544,"thread_start_us":197129,"threads_started":2,"wal-append.queue_time_us":153553}
I20260709 12:18:47.182996 10768 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:47.183454 10867 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:47.182691 10795 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:47.184376 10867 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:47.184538 10795 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:47.185199 10768 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:47.188040  7240 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:47.189697  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 79498.2us
I20260709 12:18:47.196409 10887 consensus_queue.cc:1059] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:47.196954  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:47.136075 10813 raft_consensus.cc:3055] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:47.197264 10716 consensus_queue.cc:1059] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:47.197752 10813 raft_consensus.cc:740] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:47.197954  6650 raft_consensus.cc:2393] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 9: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:47.199266 10606 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:47.199038 10813 consensus_queue.cc:260] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:47.188882  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:18:47.200625  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.200770  6653 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:47.201181 10472 raft_consensus.cc:697] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:47.202306 10472 consensus_queue.cc:237] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:47.154495  6475 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:47.090572 10834 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:47.204810  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 50429.9us
I20260709 12:18:47.205281 10790 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:47.205688  6475 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:47.206687 10819 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:47.207176 10820 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:47.206177 10834 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:47.207559 10790 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:47.207870 10820 raft_consensus.cc:697] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:47.208621 10812 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:47.192785  6655 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:18:47.189685  6651 raft_consensus.cc:1275] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:18:47.196987 10696 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:47.213299 10696 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:47.213742 10696 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:47.215478  6578 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:47.216207 10893 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.216743 10893 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:47.192031 10878 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:47.207136 10819 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:47.191198 10865 consensus_queue.cc:237] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:47.189746 10877 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:47.199651 10716 raft_consensus.cc:3055] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:47.209260 10820 consensus_queue.cc:237] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:47.179347  6548 leader_election.cc:403] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:18:47.221608  6475 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:47.222671  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:47.223169  7269 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
W20260709 12:18:47.223796  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 120157us
I20260709 12:18:47.191919 10761 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:47.200618 10813 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:47.225854 10819 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:47.197335 10887 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.073s
I20260709 12:18:47.278525  6550 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:47.233486 10716 raft_consensus.cc:740] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:47.272667 10867 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:47.295184  6653 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:47.232390 10795 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:47.297876  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:47.304606  6650 raft_consensus.cc:1240] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:47.303151  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:47.326213 10258 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.362810  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:47.296207 10812 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:18:47.370574  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 145189us
I20260709 12:18:47.376538  6655 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:18:47.376683  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:47.300283 10651 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.072s
W20260709 12:18:47.380407 10899 log.cc:927] Time spent T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
W20260709 12:18:47.400604  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 192233us
I20260709 12:18:47.380935 10758 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11.
I20260709 12:18:47.412241  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2034) took 1249 ms. Trace:
I20260709 12:18:47.419947  6578 rpcz_store.cc:276] 0709 12:18:46.156321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:46.168805 (+ 12484us) service_pool.cc:224] Handling call
0709 12:18:46.169357 (+   552us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:47.215897 (+1046540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:47.401796 (+185899us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540006de54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:47.405948 (+  4152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2404397952}
I20260709 12:18:47.379103 10631 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:47.435075 10621 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:47.435693 10621 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:47.436214 10621 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:47.439980  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 261257us
I20260709 12:18:47.431950  6653 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:47.466689  6649 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:47.418213 10610 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:47.470405  6503 raft_consensus.cc:1240] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:18:47.380658 10716 consensus_queue.cc:260] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:47.476155  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 119420us
I20260709 12:18:47.478672 10758 consensus_queue.cc:1048] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:47.481148 10491 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:18:47.486037  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 44916.8us
I20260709 12:18:47.486493  6656 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 12:18:47.486939  6656 pending_rounds.cc:85] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 6
I20260709 12:18:47.487183  6656 pending_rounds.cc:107] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:18:47.487453  6656 raft_consensus.cc:2889] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:47.488746 10877 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:47.490635  6550 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:47.511293 10708 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:47.518163  6578 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:47.433192 10258 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:47.500665 10761 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:47.565850  6474 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:47.564733  6550 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:47.558033  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 262671us
I20260709 12:18:47.531886  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:18:47.586323  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 54391.1us
I20260709 12:18:47.589349  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:47.590116  6657 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:18:47.591564 10621 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:47.592553 10258 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:47.592718 10696 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:47.594262 10621 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:47.529996 10491 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:47.595328 10696 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:47.532121 10644 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 12:18:47.597348 10781 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:18:47.599792 10761 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:47.564389 10738 consensus_queue.cc:1059] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:18:47.601637 10607 raft_consensus.cc:3055] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:47.602074 10607 raft_consensus.cc:740] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:47.603222 10607 consensus_queue.cc:260] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:47.604516  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.606199 10779 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:47.606649 10779 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:47.607051 10779 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:47.607487  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:47.531574 10716 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:47.609290  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:47.610507  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:47.623052  6581 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:18:47.618268 10651 raft_consensus.cc:3055] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:47.532415  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:47.611707  6579 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 12. Current term is 13. Ops: [11.6-12.7]
I20260709 12:18:47.625188 10651 raft_consensus.cc:740] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:47.564030 10631 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:47.595235 10491 raft_consensus.cc:697] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 9 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:18:47.614121 10913 log.cc:927] Time spent T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.138s	user 0.002s	sys 0.000s
I20260709 12:18:47.582168 10758 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.645767  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:47.523463 10795 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:47.654968  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 54612.7us
W20260709 12:18:47.608322 10912 log.cc:927] Time spent T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.142s	user 0.000s	sys 0.004s
W20260709 12:18:47.647563  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 109156us
I20260709 12:18:47.652542  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:47.564062 10747 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:47.648746  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:18:47.642146  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:47.653527  6582 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:18:47.657038  6474 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:47.660694  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:47.662278  6653 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:18:47.667760  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:47.658614 10607 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:47.673180  6656 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:18:47.673609  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:47.696858  6580 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:47.699759 10903 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:47.674181  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:47.700865  6577 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:47.701802  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:47.701876  6550 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:47.702491  6577 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:18:47.675096 10491 consensus_queue.cc:237] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:47.703845  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:47.705451 10621 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:47.706014 10621 raft_consensus.cc:697] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:47.676296  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:47.703888  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:47.708513  6658 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:47.657992 10623 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.677120  6579 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:47.677207 10747 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:47.677726  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:47.681744 10819 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:47.683408  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:47.684226  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:47.664757 10644 consensus_queue.cc:1059] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 12:18:47.689142  6578 raft_consensus.cc:2410] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 12:18:47.689402 10917 log.cc:927] Time spent T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 12:18:47.709620  6651 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:47.722982  6508 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:18:47.721521  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:47.707018 10621 consensus_queue.cc:237] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:47.666838 10758 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:47.723532  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:47.741461  6576 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:47.748096  6577 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:47.749490  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:47.752421 10918 consensus_queue.cc:1059] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.096s
I20260709 12:18:47.753676 10918 raft_consensus.cc:3055] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:47.754140 10918 raft_consensus.cc:740] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:47.755285 10918 consensus_queue.cc:260] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:47.758618  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:47.717459 10878 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:47.726827 10631 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:47.765654  6649 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:18:47.747865  6506 raft_consensus.cc:1240] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:47.773833  6580 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:47.771536  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:47.780567  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 130275us
I20260709 12:18:47.788472 10615 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.789227 10615 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:47.791509 10615 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:47.793289 10758 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:47.796461  6500 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:18:47.799901  6549 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:47.803849 10472 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:18:47.804425 10472 raft_consensus.cc:697] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 9 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:47.804243 10795 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:47.805567  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:47.805430 10472 consensus_queue.cc:237] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:47.806342  6507 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:47.809063  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:47.809954  6507 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:47.810120  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:47.811405  6507 raft_consensus.cc:1240] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:47.812875  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:47.813203  6580 raft_consensus.cc:1240] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:47.813635  6501 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:47.829288  6658 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:18:47.778213  6550 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:47.832698  6651 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:47.834843 10918 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:47.837883  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:47.838140  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:47.839239  6500 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:47.823220  6504 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:18:47.713660 10651 consensus_queue.cc:260] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:47.847100  6575 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 10: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:18:47.843995  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 68611.2us
I20260709 12:18:47.849395  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:47.857501  7270 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:18:47.850081  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:47.853495 10819 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:47.863678  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:47.741578 10623 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:47.871748  7269 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
W20260709 12:18:47.872725  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 182005us
I20260709 12:18:47.868938  6550 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:47.837730 10874 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:47.861820 10758 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:47.878543 10651 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:47.875502 10880 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.906046  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:47.891110  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 234165us
I20260709 12:18:47.891292  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:47.913936  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:47.894654  6581 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:47.912925  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 280319us
I20260709 12:18:47.904017  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:47.904417  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:47.905315  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:47.903992  6578 raft_consensus.cc:1275] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:47.923425  6504 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:47.923274  6623 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:47.897352  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:47.890900  6577 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:18:47.934237  6655 raft_consensus.cc:1240] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:47.937157  6501 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
W20260709 12:18:47.906889 10813 consensus_meta.cc:287] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.630s	user 0.010s	sys 0.003s
I20260709 12:18:47.897974 10761 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:47.944700  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:47.944958  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:47.956274  6504 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:18:47.950954  6549 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:47.906172 10623 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:47.964995  6474 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:47.935333 10878 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:18:47.972122  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 214550us
I20260709 12:18:47.965909  6622 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:47.974064  6656 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:18:47.975098  6501 raft_consensus.cc:1275] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:18:47.967000  6550 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:47.973593  7270 raft_consensus.cc:1275] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:18:47.937879 10758 raft_consensus.cc:697] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:47.959442 10716 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:47.983336  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:47.986706  6579 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:18:47.983343  6473 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:47.995625  6550 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:48.002048  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 49719.3us
I20260709 12:18:48.002527  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:47.978351 10878 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:47.973133 10761 raft_consensus.cc:697] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:47.976238 10747 raft_consensus.cc:2804] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:47.999550  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:48.006417  6582 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
I20260709 12:18:48.012951  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:48.016153  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:48.016997  6501 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:47.988276 10696 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:48.021118  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:48.021898 10918 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:48.022380  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:48.022958  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:48.023561  6507 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 11. Current term is 12. Ops: [11.5-11.5]
I20260709 12:18:48.024046  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:48.024542  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:48.025089  6507 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:48.028685  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:18:48.029462  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 53525.8us
I20260709 12:18:48.030090  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:48.031718  6473 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:48.032925  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:48.034922  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:48.036042  6655 raft_consensus.cc:2393] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 13: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:48.036996  6507 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:18:48.034235  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:48.038681  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:48.039525 10758 consensus_queue.cc:237] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:48.040176  6507 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:48.042161 10738 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:48.045220  6502 raft_consensus.cc:3055] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:48.004017 10716 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:48.011104 10878 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:48.048501  6502 raft_consensus.cc:740] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:48.053016  6505 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:48.045506 10867 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:48.053984 10867 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:48.054132  6502 consensus_queue.cc:260] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:48.054888  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:48.047835  6649 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:18:48.055471  6502 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:48.055614  6651 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:48.049954  6650 raft_consensus.cc:1275] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:18:48.050864 10716 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:48.046499 10761 consensus_queue.cc:237] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:47.953459 10880 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:48.051165 10893 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:48.028077 10776 consensus_queue.cc:1059] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 12:18:48.071635  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:48.051841  6473 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:48.051422 10918 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 12:18:48.075594  6584 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:48.071988  6581 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:18:48.058012  6508 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:18:48.004135 10813 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:18:48.078154  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 98311.3us
I20260709 12:18:48.116559  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:48.118557  6504 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 15: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:48.014680 10747 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:48.131474  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:48.132802  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 74483.8us
I20260709 12:18:48.133937  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2138) took 1013 ms. Trace:
W20260709 12:18:48.137316  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 154169us
I20260709 12:18:48.134423  6503 rpcz_store.cc:276] 0709 12:18:47.115106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:47.115255 (+   149us) service_pool.cc:224] Handling call
0709 12:18:47.169504 (+ 54249us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:47.475326 (+305822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:48.127952 (+652626us) spinlock_profiling.cc:243] Waited 298 ms on lock 0x7b5400387554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:48.128191 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":685236480}
W20260709 12:18:48.147508 10910 log.cc:927] Time spent T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.775s	user 0.003s	sys 0.000s
I20260709 12:18:48.077714 10813 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:48.077689 10631 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:48.079118 10738 raft_consensus.cc:697] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:48.150851  6549 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:48.151134  6501 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:48.077891 10696 raft_consensus.cc:697] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:48.148463 10929 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:18:48.158550  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:48.158756  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:48.142649  6402 catalog_manager.cc:5697] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 5 to 10. New cstate: current_term: 10 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:48.129339 10747 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:48.160710  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:18:48.162889  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 111138us
I20260709 12:18:48.140924 10708 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:48.150717 10887 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:48.170907 10933 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:18:48.171085 10738 consensus_queue.cc:237] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:48.148658 10813 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:48.172207  6500 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:18:48.172793 10716 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:48.172674  6402 catalog_manager.cc:5697] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 2 to 6, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 6 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:48.170590  6509 raft_consensus.cc:3055] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:48.181186  6509 raft_consensus.cc:740] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:48.174067  6578 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:48.182693  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:48.175007  6403 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57900 (request call id 188) took 40106 ms (client timeout 15000 ms). Trace:
I20260709 12:18:48.075766 10779 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:48.170524 10926 log.cc:927] Time spent T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:18:48.177738 10708 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:48.178740  6579 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:18:48.184635  6502 raft_consensus.cc:1275] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:18:48.191706  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:48.177523  6507 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:18:48.193612  6508 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:18:48.202373  6503 raft_consensus.cc:3055] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:48.201952  6622 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:48.193430  6473 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:48.193209  6402 catalog_manager.cc:5697] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:18:48.206275  6402 catalog_manager.cc:5697] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 6 to 7, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 7 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:18:48.195935 10708 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:48.202767  6503 raft_consensus.cc:740] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:48.207687  6402 catalog_manager.cc:5697] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 3 to 8, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 8 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:48.217415  6575 raft_consensus.cc:2393] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 13: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:48.215969  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:48.212472  6503 consensus_queue.cc:260] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:48.222074  6503 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:48.176105 10878 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:48.222229 10696 consensus_queue.cc:237] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:48.224596 10877 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:48.225646 10776 raft_consensus.cc:3055] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:48.226415 10876 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:48.238389  6654 raft_consensus.cc:1275] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:18:48.226577  7238 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:48.239408  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:48.240625  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:48.240774  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:48.202196 10941 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:18:48.186972  6509 consensus_queue.cc:260] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:48.232345  6507 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:18:48.243040  6509 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:48.243971  6507 raft_consensus.cc:1275] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:18:48.244289 10501 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:48.230873  6505 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
W20260709 12:18:48.234678 10937 log.cc:927] Time spent T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:18:48.249176  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:48.234745  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:18:48.236162  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 48371.9us
I20260709 12:18:48.237171 10776 raft_consensus.cc:740] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:48.192194 10834 consensus_queue.cc:1059] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 12:18:48.229776 10933 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:48.252483 10865 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:48.253777  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:48.254020 10865 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:48.254508 10865 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:48.256719 10921 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:48.257737  6657 raft_consensus.cc:1240] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 12. Current term is 13. Ops: [12.5-12.5]
I20260709 12:18:48.257434 10776 consensus_queue.cc:260] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:48.258174 10921 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:48.259060  6657 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:18:48.260838 10878 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:48.261466 10876 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:48.264279 10779 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:48.258956 10834 raft_consensus.cc:3055] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:48.267210  6501 raft_consensus.cc:1275] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:48.267977  6509 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:18:48.276289  6654 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:48.267967  6507 raft_consensus.cc:3055] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:48.293664  6507 raft_consensus.cc:740] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:48.293742  6653 raft_consensus.cc:1240] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:48.297077  6505 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 12:18:48.301944  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 100079us
I20260709 12:18:48.291523 10834 raft_consensus.cc:740] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:48.298265  6507 consensus_queue.cc:260] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:48.303815  6653 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: []
W20260709 12:18:48.427703  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 122390us
I20260709 12:18:48.458117  6653 raft_consensus.cc:1275] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:48.462797 10834 consensus_queue.cc:260] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:48.464457 10834 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:48.441009 10877 consensus_queue.cc:1048] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:18:48.629952 10813 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:48.631907  6656 raft_consensus.cc:2410] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:48.431926 10949 consensus_queue.cc:1059] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.204s
I20260709 12:18:48.677796  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:48.679237  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 51621.6us
W20260709 12:18:48.840222  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:48.637569 10950 consensus_queue.cc:1048] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.283s
I20260709 12:18:48.698738 10951 raft_consensus.cc:2764] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:18:48.957430 10813 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:49.323164  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:49.341740 10501 consensus_meta.cc:287] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 1.097s	user 0.006s	sys 0.005s
I20260709 12:18:49.395164 10921 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:49.555222 10921 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:49.499802 10952 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.867s	user 0.000s	sys 0.002s
W20260709 12:18:49.800688  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:49.620105 10501 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:18:50.071889 10865 consensus_meta.cc:287] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 1.817s	user 0.011s	sys 0.003s
I20260709 12:18:50.073830 10921 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:50.079231 10867 consensus_meta.cc:287] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 2.025s	user 0.015s	sys 0.000s
W20260709 12:18:50.083638 10834 consensus_meta.cc:287] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 1.617s	user 0.008s	sys 0.006s
W20260709 12:18:50.091522 10747 consensus_meta.cc:287] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 1.910s	user 0.011s	sys 0.004s
W20260709 12:18:50.410264  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:50.200030 10865 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:50.432369 10708 consensus_meta.cc:287] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 2.209s	user 0.013s	sys 0.002s
I20260709 12:18:50.435469 10708 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:50.437287 10865 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:50.357445 10747 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:50.440164  7239 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.438s	user 0.007s	sys 0.022s
W20260709 12:18:50.440779  7239 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.449s	user 0.007s	sys 0.022s
I20260709 12:18:50.437675 10708 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:50.441782  7237 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.439s	user 0.005s	sys 0.022s
I20260709 12:18:50.366705 10867 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:50.442700  7237 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.451s	user 0.005s	sys 0.022s
I20260709 12:18:50.443958 10958 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:50.444080 10959 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:50.445246 10959 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:50.445163 10958 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:50.446491 10747 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:50.447676 10959 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:50.447783 10865 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:50.449875 10960 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:50.449649 10865 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:50.450414 10960 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:50.451984 10865 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:50.452597 10865 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.452832 10960 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:50.453068 10865 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:50.454011 10960 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:50.447986 10958 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:50.448661 10501 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:50.447677 10921 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.454526 10960 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:50.455627 10501 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:50.455689 10921 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:50.457270 10960 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:50.457952 10501 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:50.458005 10921 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:50.460055 10865 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:50.461593 10747 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:50.462231 10747 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:50.463966 10867 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:50.464582 10747 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:50.473230 10962 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.473342 10747 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:50.473970 10962 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:50.473965 10747 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:50.476478 10747 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:50.476505 10962 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:50.477301 10747 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.477764 10747 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:50.479543 10747 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:50.504762 10708 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:50.505204 10834 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.505508 10708 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:50.506038 10834 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:50.508272 10708 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:50.513602 10834 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:50.522115  6508 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.219s	user 0.035s	sys 0.049s
W20260709 12:18:50.526126  6508 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.223s	user 0.036s	sys 0.050s
I20260709 12:18:50.546784 10834 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.547495 10834 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:50.550096 10834 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:50.550668 10834 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.551151 10834 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:50.553571 10834 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:50.554553 10834 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:50.555141 10834 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:50.559010 10834 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:50.559450  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 508) took 2294 ms. Trace:
I20260709 12:18:50.560213  6508 rpcz_store.cc:276] 0709 12:18:48.261140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.289916 (+ 28776us) service_pool.cc:224] Handling call
0709 12:18:48.290321 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:48.290791 (+   470us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:18:50.526711 (+2235920us) log.cc:844] Serialized 14 byte log entry
0709 12:18:50.547633 (+ 20922us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:50.548304 (+   671us) log.cc:844] Serialized 26 byte log entry
0709 12:18:50.548493 (+   189us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:50.555175 (+  6682us) log.cc:844] Serialized 14 byte log entry
0709 12:18:50.555423 (+   248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.555501 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:50.555548 (+    47us) raft_consensus.cc:1685] finished
0709 12:18:50.555555 (+     7us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:50.555651 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1610789,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":2053,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":2894,"thread_start_us":2249991,"threads_started":3,"wal-append.queue_time_us":20198}
W20260709 12:18:50.562101 10908 log.cc:927] Time spent T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 3.177s	user 0.013s	sys 0.022s
W20260709 12:18:48.183795  6403 rpcz_store.cc:269] 0709 12:18:08.068862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:08.075765 (+  6903us) service_pool.cc:224] Handling call
0709 12:18:48.174690 (+40098925us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:40.052395 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:40.053390 (+   995us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:40.053460 (+    70us) write_op.cc:435] Acquired schema lock
0709 12:18:40.053516 (+    56us) tablet.cc:662] Decoding operations
0709 12:18:40.054661 (+  1145us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:40.054762 (+   101us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:40.054809 (+    47us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:18:40.055848 (+  1039us) tablet.cc:701] Row locks acquired
0709 12:18:40.055923 (+    75us) write_op.cc:260] PREPARE: finished
0709 12:18:40.056183 (+   260us) write_op.cc:270] Start()
0709 12:18:40.117836 (+ 61653us) write_op.cc:276] Timestamp: P: 1783599520056091 usec, L: 0
0709 12:18:40.130972 (+ 13136us) op_driver.cc:348] REPLICATION: starting
0709 12:18:40.242483 (+111511us) log.cc:844] Serialized 2100 byte log entry
0709 12:18:41.364480 (+1121997us) op_driver.cc:464] REPLICATION: finished
0709 12:18:41.896321 (+531841us) write_op.cc:301] APPLY: starting
0709 12:18:41.919523 (+ 23202us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:41.919749 (+   226us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:41.919906 (+   157us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:41.920853 (+   947us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:41.921194 (+   341us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 12:18:41.921245 (+    51us) write_op.cc:312] APPLY: finished
0709 12:18:41.921588 (+   343us) log.cc:844] Serialized 47 byte log entry
0709 12:18:41.922026 (+   438us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:41.922148 (+   122us) write_op.cc:454] Released schema lock
0709 12:18:41.922219 (+    71us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:41.997374 (+ 75155us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:41.999240 (+  1866us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":998563,"reported_tablets":51,"spinlock_wait_cycles":29056,"child_traces":[["op",{"apply.queue_time_us":382987,"apply.run_cpu_time_us":3598,"apply.run_wall_time_us":186084,"num_ops":5,"prepare.queue_time_us":235736,"prepare.run_cpu_time_us":22209,"prepare.run_wall_time_us":359992,"raft.queue_time_us":388533,"raft.run_cpu_time_us":83141,"raft.run_wall_time_us":248970,"replication_time_us":1212510,"thread_start_us":632726,"threads_started":4,"wal-append.queue_time_us":178165,"wal-append.run_cpu_time_us":33003,"wal-append.run_wall_time_us":2468269}]]}
W20260709 12:18:50.569109  6657 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.128s	user 0.091s	sys 0.102s
W20260709 12:18:50.574729  6657 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.134s	user 0.092s	sys 0.103s
I20260709 12:18:50.575517  6657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1997) took 2382 ms. Trace:
I20260709 12:18:50.575920  6657 rpcz_store.cc:276] 0709 12:18:48.195762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.259892 (+ 64130us) service_pool.cc:224] Handling call
0709 12:18:48.260147 (+   255us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:48.292970 (+ 32823us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:48.293032 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:48.302241 (+  9209us) log.cc:844] Serialized 26 byte log entry
0709 12:18:50.575112 (+2272871us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:50.575160 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.575234 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:50.575278 (+    44us) raft_consensus.cc:1685] finished
0709 12:18:50.575290 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:50.575426 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":28738,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":2806,"thread_start_us":2134041,"threads_started":1,"wal-append.queue_time_us":1618240}
W20260709 12:18:50.577270  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 2.27198e+06us
I20260709 12:18:50.565619 10966 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.578496  6475 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:50.578769 10966 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:50.579913 10968 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:50.580750 10968 consensus_queue.cc:1059] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 10.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:50.581267 10966 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:50.581712 10968 consensus_queue.cc:1059] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:18:50.589567  6500 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 2270) took 2551 ms (client timeout 1545 ms). Trace:
W20260709 12:18:50.590075  6500 rpcz_store.cc:269] 0709 12:18:48.021182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.022150 (+   968us) service_pool.cc:224] Handling call
0709 12:18:50.572493 (+2550343us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b5400434c54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:50.572646 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":326829952}
W20260709 12:18:50.592293 10258 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 12:18:50.593066 10258 raft_consensus.cc:3055] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:50.593595 10258 raft_consensus.cc:740] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:50.594849 10258 consensus_queue.cc:260] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:50.596607 10820 raft_consensus.cc:3055] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:50.597132 10820 raft_consensus.cc:740] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:18:50.597704  6654 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.157s	user 0.009s	sys 0.042s
W20260709 12:18:50.598070  6654 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.157s	user 0.009s	sys 0.042s
I20260709 12:18:50.598168 10820 consensus_queue.cc:260] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.5, Last appended by leader: 5, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:50.601812 10820 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:50.606106  6506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2245) took 2906 ms. Trace:
I20260709 12:18:50.606745  6506 rpcz_store.cc:276] 0709 12:18:47.699395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:47.711742 (+ 12347us) service_pool.cc:224] Handling call
0709 12:18:47.722740 (+ 10998us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:47.771197 (+ 48457us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.605706 (+2834509us) spinlock_profiling.cc:243] Waited 4.3 ms on lock 0x7b540001bdd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:50.606002 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9892736}
I20260709 12:18:50.618391 10966 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.619081 10966 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:50.625051  6475 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:50.627848  6654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 1998) took 2422 ms. Trace:
I20260709 12:18:50.632256 10966 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:50.632721  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 509) took 2370 ms. Trace:
I20260709 12:18:50.633167  6509 rpcz_store.cc:276] 0709 12:18:48.262526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.294786 (+ 32260us) service_pool.cc:224] Handling call
0709 12:18:48.295224 (+   438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:48.306201 (+ 10977us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:48.306250 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:48.322436 (+ 16186us) log.cc:844] Serialized 26 byte log entry
0709 12:18:50.608766 (+2286330us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:50.610798 (+  2032us) log.cc:844] Serialized 14 byte log entry
0709 12:18:50.610992 (+   194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.611068 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:50.632506 (+ 21438us) raft_consensus.cc:1685] finished
0709 12:18:50.632526 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:50.632661 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":356,"prepare.queue_time_us":1658,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":5254,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":111519,"spinlock_wait_cycles":28264192,"thread_start_us":42114,"threads_started":1,"wal-append.queue_time_us":63062}
I20260709 12:18:50.629544  6654 rpcz_store.cc:276] 0709 12:18:48.205098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.277473 (+ 72375us) service_pool.cc:224] Handling call
0709 12:18:48.297607 (+ 20134us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:48.300181 (+  2574us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:48.426881 (+126700us) log.cc:844] Serialized 14 byte log entry
0709 12:18:50.627139 (+2200258us) log.cc:844] Serialized 14 byte log entry
0709 12:18:50.627396 (+   257us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:50.627452 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:50.627568 (+   116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.627652 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:50.627780 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":157178,"prepare.run_cpu_time_us":157,"prepare.run_wall_time_us":5694,"raft.queue_time_us":247,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":1009,"spinlock_wait_cycles":19840,"thread_start_us":2184257,"threads_started":2,"wal-append.queue_time_us":1561289}
I20260709 12:18:50.635792 10820 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:50.658048  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:50.653328 10975 consensus_queue.cc:1059] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
W20260709 12:18:50.661227  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 1.76599e+06us
I20260709 12:18:50.662653  6622 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
W20260709 12:18:50.667270  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:50.667168 10976 consensus_queue.cc:1048] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:18:50.667970  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:50.668409  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:50.668960  6393 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 3e0ab356fc074f589a6ff4f9b4996b09 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 20a202c76244485fbc8f1610473db8b9 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:18:50.670878 10976 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:50.673139 10834 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:18:50.673264  6584 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 2021) took 2768 ms (client timeout 1822 ms). Trace:
I20260709 12:18:50.673681 10834 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:50.674242 10834 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:50.674753  6550 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.3:36107 timed out after 2.142s (SENT) [suppressed 8 similar messages]
W20260709 12:18:50.673709  6584 rpcz_store.cc:269] 0709 12:18:47.904922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:47.905050 (+   128us) service_pool.cc:224] Handling call
0709 12:18:50.673039 (+2767989us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54001cbe54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:50.673164 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":304197504}
I20260709 12:18:50.693922  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:50.695132 10874 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:50.695588 10874 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:50.696044 10874 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:50.695751  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:50.698752  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:50.702387  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:50.702951  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:50.699018 10776 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:50.671747 10976 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:50.724911 10708 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:50.725181  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:50.730454  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:50.730773  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:50.731060  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:50.731881  7274 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:50.706555 10976 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:50.733835 10747 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.733721 10874 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:50.728783 10610 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.728734 10758 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.738459  6650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2530) took 2877 ms. Trace:
W20260709 12:18:50.749519  6503 consensus_meta.cc:287] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 2.527s	user 0.014s	sys 0.030s
I20260709 12:18:50.752924  6650 rpcz_store.cc:276] 0709 12:18:47.861079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:47.926482 (+ 65403us) service_pool.cc:224] Handling call
0709 12:18:47.926869 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:48.056176 (+129307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.738176 (+2682000us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b54002a3ad4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:50.738395 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":282142976}
I20260709 12:18:50.755393  6503 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 12:18:50.755976  6503 pending_rounds.cc:85] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 4
I20260709 12:18:50.752085  7274 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:18:50.758131  6622 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:50.758796  6503 pending_rounds.cc:107] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 12:18:50.759122  6503 raft_consensus.cc:2889] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:50.743039 10708 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:50.760277 10708 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:50.756742 10975 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.760695 10708 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:50.761142 10975 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:50.763265 10975 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:50.760711 10974 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.753067 10976 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.768406  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2278) took 2704 ms. Trace:
I20260709 12:18:50.769995 10976 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:50.771193 10874 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:50.771917 10874 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:50.757408 10610 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:50.773237 10834 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:50.775027 10610 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:50.752558 10747 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:50.776885 10976 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:50.777657 10834 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:50.778434 10747 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:50.757691 10758 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:50.781524 10758 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:50.783210  6655 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.314s	user 0.027s	sys 0.067s
W20260709 12:18:50.783636  6655 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.315s	user 0.027s	sys 0.067s
I20260709 12:18:50.787082 10708 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:50.789932 10708 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:50.769191  6503 rpcz_store.cc:276] 0709 12:18:48.064211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.175491 (+111280us) service_pool.cc:224] Handling call
0709 12:18:48.182335 (+  6844us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:50.767273 (+2584938us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.768127 (+   854us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:50.795163  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2003) took 2588 ms. Trace:
I20260709 12:18:50.803236  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:50.802953 10758 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:50.769074 10975 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.798190  6655 rpcz_store.cc:276] 0709 12:18:48.206645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.296701 (+ 90056us) service_pool.cc:224] Handling call
0709 12:18:48.299917 (+  3216us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:48.304674 (+  4757us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:48.304707 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:48.441757 (+137050us) log.cc:844] Serialized 26 byte log entry
0709 12:18:50.784060 (+2342303us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:50.784109 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:50.784200 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:50.784254 (+    54us) raft_consensus.cc:1685] finished
0709 12:18:50.784268 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:50.795073 (+ 10805us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4419,"raft.queue_time_us":3107,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":11033,"thread_start_us":2315139,"threads_started":1,"wal-append.queue_time_us":1572567}
W20260709 12:18:50.810863  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 55051.9us
I20260709 12:18:50.811256 10975 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:50.815454 10258 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:50.816509  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:50.817189  7266 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:50.821976  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:50.822717  7272 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:18:50.822947  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:50.823706  7271 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:50.823799  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:50.824677  7272 raft_consensus.cc:2393] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 10: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:50.824851  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:50.825519  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:50.825860  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:50.826768  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:50.820092  6575 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 2018) took 2918 ms (client timeout 3310 ms). Trace:
I20260709 12:18:50.828505  7268 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11. Ops: []
W20260709 12:18:50.828372  6575 rpcz_store.cc:269] 0709 12:18:47.901627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:47.903743 (+  2116us) service_pool.cc:224] Handling call
0709 12:18:50.819831 (+2916088us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b54000578d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:50.820026 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":664982272}
I20260709 12:18:50.800307 10966 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:50.825538  7265 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 5.
I20260709 12:18:50.835700  7274 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:50.837114  6549 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:18:50.764528 10974 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:50.830071 10708 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.852660 10708 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:50.854025  7266 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:50.855610 10708 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:50.848151 10874 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.837841 10834 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.858036 10874 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:50.858513 10834 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:50.861335 10834 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:50.841732 10975 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:50.872666 10974 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:50.883586 10874 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:50.885215 10963 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:50.888546 10963 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:50.889525 10963 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:50.895699 10948 log.cc:927] Time spent T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 2.227s	user 0.000s	sys 0.015s
I20260709 12:18:50.898440  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:50.899142  6506 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 14.
I20260709 12:18:50.900799  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:50.901556  6503 raft_consensus.cc:2393] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 12: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:50.912014  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:50.912935  6509 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7.
I20260709 12:18:50.914345  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:50.915264  6503 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 9.
I20260709 12:18:50.901135 10708 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:50.917373 10708 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:50.919440  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:50.919915 10708 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:50.902438 10834 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:50.931070 10834 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:50.925426  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:50.935081  6503 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 9.
I20260709 12:18:50.928586  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:50.936545  6509 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
W20260709 12:18:50.938555  7238 connection.cc:256] RPC call timeout handler was delayed by -1.5052286939999249s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15280000000000002s for a response
W20260709 12:18:50.947189  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 108551us
I20260709 12:18:50.949138 10990 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:50.950104 10865 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:50.934180 10834 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:50.954300  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 117260us
I20260709 12:18:50.952812 10606 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 12:18:50.953693 10876 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:50.955910 10876 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:50.953155  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 195059us
I20260709 12:18:50.958266  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:50.962558  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:50.963241  6654 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:50.965200  6657 raft_consensus.cc:1275] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:18:51.016875  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:51.017891  6651 raft_consensus.cc:2468] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:18:50.978878 10984 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:18:51.030062  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 71904.2us
I20260709 12:18:51.029119  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:51.031313  6655 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:50.978310 10991 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:18:51.035805 10963 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:51.038249 10963 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:51.043509  7238 leader_election.cc:336] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.528s (SENT)
I20260709 12:18:51.044072  7238 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:51.047313  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58980 (request call id 2536) took 2994 ms. Trace:
I20260709 12:18:51.047730  6656 rpcz_store.cc:276] 0709 12:18:48.052673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.058520 (+  5847us) service_pool.cc:224] Handling call
0709 12:18:51.047077 (+2988557us) spinlock_profiling.cc:243] Waited 514 ms on lock 0x7b540044a2d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:51.047223 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1182702464}
I20260709 12:18:51.057519  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:51.052527 10776 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:51.071403  6656 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:51.072193  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:51.072890  6500 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:51.072556  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:51.081395  6500 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 11.
I20260709 12:18:51.054919  6655 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:18:51.048569 10984 raft_consensus.cc:697] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 11 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:51.083196  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:51.083880  6655 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:18:51.084103 10877 raft_consensus.cc:3055] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:51.084539 10877 raft_consensus.cc:740] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:51.084914  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:51.085788  6655 raft_consensus.cc:2410] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:18:51.085541 10877 consensus_queue.cc:260] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:51.087158 10877 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:51.087285  6623 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:51.088294 10834 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:51.088742 10834 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:51.089184 10834 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:51.091248  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 2.78486e+06us
I20260709 12:18:51.092084  6473 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:51.093922  6623 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:51.094566 10972 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:51.095176  6623 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:51.095587 10874 raft_consensus.cc:2804] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:18:51.096002 10874 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:51.096664  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:51.097083 10708 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:51.097314  6649 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:18:51.097633 10708 raft_consensus.cc:697] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 11 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:51.094111 10258 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:51.098161 10874 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:51.069586  6653 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:18:51.101765  6654 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:18:51.066468 10880 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:51.114462  6654 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:51.116796 10258 raft_consensus.cc:697] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 13 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:51.117957 10258 consensus_queue.cc:237] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:51.099360 10708 consensus_queue.cc:237] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:51.083596 10984 consensus_queue.cc:237] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:51.133693 10876 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:48.305939  6402 catalog_manager.cc:5697] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 5 to 7, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 7 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:51.155741  6402 catalog_manager.cc:5697] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 3 to 8, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 8 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
W20260709 12:18:51.177807  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:51.178436  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:51.185133  6402 catalog_manager.cc:5697] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:18:51.186432  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 231329us
W20260709 12:18:51.198479  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 84959.2us
I20260709 12:18:51.201144  6402 catalog_manager.cc:5697] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:51.211453  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:51.212357  6649 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:51.222317  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:51.222760  6649 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:51.223179  7269 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:51.219923  6402 catalog_manager.cc:5697] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 5 to 7, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 7 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:18:51.226863  6402 catalog_manager.cc:5697] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 4 to 6, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 6 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:51.220687 10738 consensus_queue.cc:1059] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 12:18:51.254397  6622 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:51.264024  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:51.273192 10877 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:51.274175 10877 raft_consensus.cc:697] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:51.275650 10877 consensus_queue.cc:237] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:51.233534 10747 consensus_queue.cc:1059] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 12:18:51.282418 10963 raft_consensus.cc:3055] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:51.286091  7269 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:51.293591  6622 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:51.250483 10991 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
W20260709 12:18:51.309720  6622 connection.cc:256] RPC call timeout handler was delayed by -3.4666624080000474s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16910000000000003s for a response
I20260709 12:18:51.304019 10708 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:51.289124 10963 raft_consensus.cc:740] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:18:51.318262  6622 connection.cc:256] RPC call timeout handler was delayed by -2.7350727319999351s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1825s for a response
W20260709 12:18:51.318853  6622 connection.cc:256] RPC call timeout handler was delayed by -2.5307727319998321s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20520000000000002s for a response
I20260709 12:18:51.290849 10834 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:51.322005  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 118154us
W20260709 12:18:51.321818  6622 connection.cc:256] RPC call timeout handler was delayed by -2.0612404259998129s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16140000000000002s for a response
I20260709 12:18:51.326766 10999 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:51.327001  6622 connection.cc:256] RPC call timeout handler was delayed by -1.874040425999965s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18220000000000003s for a response
W20260709 12:18:51.327363  6622 connection.cc:256] RPC call timeout handler was delayed by -1.5713330159996985s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.39820000000000005s for a response
I20260709 12:18:51.327441 10999 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:51.328238  6622 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:51.329336  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:51.329741  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:51.330152  6500 raft_consensus.cc:2410] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:18:51.331039  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:51.331136  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:51.331619  6503 raft_consensus.cc:2376] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11. Re-sending same reply.
I20260709 12:18:51.331836  6506 raft_consensus.cc:2376] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9. Re-sending same reply.
I20260709 12:18:51.332588  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:51.333256  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:51.332666  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:51.334244  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:51.336421  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:51.340157  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:51.336035  6508 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:18:51.344727 10999 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:51.345031  6508 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:18:51.345697  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:51.346398  6508 raft_consensus.cc:1275] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:18:51.347545  6575 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 12. Current term is 13. Ops: []
W20260709 12:18:51.352612  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:51.354575  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:51.355237  6502 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:51.358215  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:51.377243  6649 raft_consensus.cc:1240] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:51.320216 10993 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.389295  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:51.390635  6506 raft_consensus.cc:1275] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:51.396245  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:51.342273 10877 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:51.397138  6500 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:18:51.400238  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 75678.5us
I20260709 12:18:51.398000  6503 raft_consensus.cc:3055] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:51.401170  6503 raft_consensus.cc:740] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:51.321151 10998 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.323503 10880 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:51.404141  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:51.401279  6550 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:51.405333  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:51.406388  6584 raft_consensus.cc:2393] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 11: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:51.408999  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:51.411275  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 71211.7us
I20260709 12:18:51.411194  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:51.413395  6581 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 16: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:51.409492  6503 consensus_queue.cc:260] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:51.430647  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:51.431072  6503 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:51.430583  6500 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:18:51.435621  6584 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:51.424305  6550 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:51.395717 10984 consensus_queue.cc:1048] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:18:51.444741  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:51.445147 11015 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:51.445848 11015 raft_consensus.cc:697] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:51.447027  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:51.448618  6584 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:51.448860  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:51.447068 11015 consensus_queue.cc:237] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:51.451984  6577 raft_consensus.cc:1240] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 12. Current term is 13. Ops: [10.4-12.5]
I20260709 12:18:51.455006  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:51.455889  6577 raft_consensus.cc:1240] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:18:51.456413 10708 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:51.401264 10993 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:51.457201  6550 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:51.466501  6548 connection.cc:256] RPC call timeout handler was delayed by -3.094287131999863s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3134s for a response
W20260709 12:18:51.467180  6548 connection.cc:256] RPC call timeout handler was delayed by -3.0864010829998278s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15910000000000002s for a response
I20260709 12:18:51.465080  6550 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:51.469436  6583 raft_consensus.cc:1240] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:51.439684 10812 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.437485 11016 raft_consensus.cc:2804] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:51.416097 10880 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:51.474490  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 43946.1us
I20260709 12:18:51.407225 10998 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:51.483677  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:51.483901  6503 raft_consensus.cc:1275] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:18:51.460551 10877 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:51.410373 10963 consensus_queue.cc:260] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:51.492853  6622 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:51.484936  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:51.486167  6550 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:51.471632 10472 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:51.471419 10974 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:51.463117 10874 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:51.486037 10708 raft_consensus.cc:697] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 8 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:51.491204 11016 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:51.509590  6583 raft_consensus.cc:1275] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:18:51.515645  6577 raft_consensus.cc:1240] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:51.501842 10472 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:51.507169 10985 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:51.523736  6622 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:51.518077 10999 raft_consensus.cc:2804] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:51.522081 10472 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:51.490476 10812 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:51.509145 10974 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:51.514782 11016 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:51.530387 11010 log.cc:927] Time spent T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:18:51.520869 10984 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:51.517206 10877 raft_consensus.cc:697] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:51.522689 10985 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:51.543678  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:51.551110  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:51.618002  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:51.539707  6550 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:51.533300 10834 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:51.539422 10999 raft_consensus.cc:697] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:18:51.596184  6548 connection.cc:256] RPC call timeout handler was delayed by -2.9861711879998438s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2886s for a response
W20260709 12:18:51.635908  6548 connection.cc:256] RPC call timeout handler was delayed by -2.960744343999977s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3118s for a response
W20260709 12:18:51.636031  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 96404us
W20260709 12:18:51.636442  6548 connection.cc:256] RPC call timeout handler was delayed by -2.8943711879999228s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2988s for a response
W20260709 12:18:51.637130  6548 connection.cc:256] RPC call timeout handler was delayed by -2.8092010829998344s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1899s for a response
W20260709 12:18:51.637661  6548 connection.cc:256] RPC call timeout handler was delayed by -2.8038010829998257s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1905s for a response
I20260709 12:18:51.637961  6580 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: []
W20260709 12:18:51.638320  6548 connection.cc:256] RPC call timeout handler was delayed by -2.7264010829999279s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19910000000000003s for a response
I20260709 12:18:51.594559  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:51.616892  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:51.638999  6548 connection.cc:256] RPC call timeout handler was delayed by -2.5239010829998278s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22160000000000002s for a response
I20260709 12:18:51.639943  6651 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 10: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:18:51.567430 11011 log.cc:927] Time spent T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 12:18:51.539572 11012 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:51.654150  6581 raft_consensus.cc:1240] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:51.655157  6622 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:51.646817  6583 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:51.557758 10812 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:51.640727 10758 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:51.560240 10985 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:51.644598  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:51.672420  7269 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:51.586612 10880 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:51.662356  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:51.680025  6584 raft_consensus.cc:2410] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:18:51.584499 10998 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:51.557710 10974 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:51.584789 10874 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:51.541579 11017 consensus_queue.cc:1059] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.126s
I20260709 12:18:51.641086 10984 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.682163  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:51.687047  6581 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:18:51.685766  6505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 510) took 3411 ms. Trace:
I20260709 12:18:51.661424 11018 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:51.690884  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:51.682281 10834 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:51.692701 11024 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.559388 10877 consensus_queue.cc:237] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:51.696580 10993 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:51.583516 10708 consensus_queue.cc:237] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:51.698978 10966 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:51.691260 11018 raft_consensus.cc:697] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 11 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:51.692713  6584 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
W20260709 12:18:51.714653  6548 connection.cc:256] RPC call timeout handler was delayed by -2.4987010829997871s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22440000000000004s for a response
I20260709 12:18:51.715854  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:51.712074  7269 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
I20260709 12:18:51.720435  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:51.692061 11016 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:51.722769  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:51.715134 10966 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:18:51.746543  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 64222.7us
W20260709 12:18:51.724025  6548 connection.cc:256] RPC call timeout handler was delayed by -2.2552573479999865s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1545s for a response
W20260709 12:18:51.749720  6548 connection.cc:256] RPC call timeout handler was delayed by -2.2525573479999821s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15480000000000002s for a response
W20260709 12:18:51.750274  6548 connection.cc:256] RPC call timeout handler was delayed by -2.2084573479999108s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1597s for a response
I20260709 12:18:51.689409 10984 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:51.796793  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:51.797739 11016 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:51.689306 10999 consensus_queue.cc:237] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:51.798087  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:51.713820 10758 raft_consensus.cc:697] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:51.722784  6584 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
W20260709 12:18:51.750897  6548 connection.cc:256] RPC call timeout handler was delayed by -2.1661573480000698s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1644s for a response
W20260709 12:18:51.827203  6548 connection.cc:256] RPC call timeout handler was delayed by -2.0032573480000337s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1825s for a response
W20260709 12:18:51.781304  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:51.717921 11017 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:51.792387 11018 consensus_queue.cc:237] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:51.799243  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:51.828534  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:51.800659  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 51895.2us
W20260709 12:18:51.827741  6548 connection.cc:256] RPC call timeout handler was delayed by -1.9654573479999726s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1867s for a response
W20260709 12:18:51.834301  6548 connection.cc:256] RPC call timeout handler was delayed by -1.8988573479998649s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19410000000000002s for a response
W20260709 12:18:51.834925  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:51.837149  6548 connection.cc:256] RPC call timeout handler was delayed by -1.7416534429999047s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18710000000000002s for a response
I20260709 12:18:51.847882 10984 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:51.822460 11012 raft_consensus.cc:697] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:51.846174 10978 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.869313 10758 consensus_queue.cc:237] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:51.862982 10974 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.862777 10812 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:51.711777 11024 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:51.728497 11023 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:51.960548  6548 connection.cc:256] RPC call timeout handler was delayed by -1.5258010829998057s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3325s for a response
I20260709 12:18:51.961495  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:52.196446  6548 connection.cc:256] RPC call timeout handler was delayed by -1.1482573480000156s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2775s for a response
W20260709 12:18:51.970012  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 306107us
W20260709 12:18:52.029884  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.135622  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:52.200624 10812 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:52.202916  6650 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:52.203097 10812 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:52.203801 10812 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.204761  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:52.204521 11023 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:52.205595  7274 raft_consensus.cc:2410] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:18:52.205729  7269 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:18:51.864403 11017 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:52.206883 11023 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:52.208180 11017 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:52.204317 10812 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:52.215505  6473 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:52.227308  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.187069  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:52.194461  7268 raft_consensus.cc:1240] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:52.228703  6581 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
W20260709 12:18:52.211361 10472 consensus_meta.cc:287] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.673s	user 0.008s	sys 0.000s
I20260709 12:18:52.231895  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:52.232056  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 432640us
W20260709 12:18:52.232930  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.188271 10997 raft_consensus.cc:3055] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:52.246920  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:52.242825  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:52.245918  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:52.249092  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.256208  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:52.256558  6650 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 12:18:52.257208  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.257686  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:52.260084  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:52.269454  6650 raft_consensus.cc:1275] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:18:52.226722 10919 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:52.275967  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:52.273566  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:52.287492  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:52.246274 10997 raft_consensus.cc:740] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:52.217268 10974 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:52.295070  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:52.296466  6580 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:52.303310  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:52.305233  6580 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:52.310860  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:52.246516 10472 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:52.314410  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:52.317790  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 102365us
I20260709 12:18:52.194715 10978 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:52.318606  6473 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:52.331673  6623 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:52.339221  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.320211  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:52.320581  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:52.321038  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:52.342675  6650 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:52.320827  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:52.321801 10997 consensus_queue.cc:260] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:52.286993 10919 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:52.324067 11046 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:52.357728 10974 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:52.310829 11038 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:52.358395 10974 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:52.358871 11038 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:52.353294 10820 raft_consensus.cc:3055] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:52.361671 10820 raft_consensus.cc:740] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:52.362694  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:52.362569 10820 consensus_queue.cc:260] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:52.354580 10985 consensus_meta.cc:287] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.668s	user 0.005s	sys 0.006s
I20260709 12:18:52.355666 11023 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:52.349982 10877 raft_consensus.cc:2804] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:18:52.366968  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:52.340226 10812 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:52.373778  6580 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 10: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:52.377072 11050 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:18:52.379549  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 162062us
I20260709 12:18:52.378412  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:52.379071  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:52.377862  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 59333.4us
I20260709 12:18:52.381735  6581 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
I20260709 12:18:52.369166 10997 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:52.402134  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:52.404439  6473 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:52.406600 10472 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:52.378167 10820 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:52.407992  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:52.409112  6579 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:18:52.412616  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:52.417282  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:52.418361  6650 raft_consensus.cc:2468] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:52.416049  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:52.415966  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:52.420066  6579 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:52.410802 11050 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.425645  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:52.408615 10966 consensus_meta.cc:287] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.616s	user 0.005s	sys 0.006s
I20260709 12:18:52.378347 10877 raft_consensus.cc:697] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 6 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:18:52.427893  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 92484.7us
I20260709 12:18:52.438088  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:52.438771  6580 raft_consensus.cc:1813] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Handling vote request from an unknown peer f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:52.419804 10812 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.372768 10985 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:52.447981  6473 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:52.431057 11036 log.cc:927] Time spent T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
W20260709 12:18:52.454304  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.455255  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:52.449348  6580 raft_consensus.cc:2393] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 8: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
W20260709 12:18:52.464811  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 82507.8us
I20260709 12:18:52.467227  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:52.430652 11043 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:52.468735  6577 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:52.468926  6473 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:52.474282  6579 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:18:52.438256 10966 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:52.463070 11046 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:52.500277 10758 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:52.429878 11053 consensus_queue.cc:1059] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
W20260709 12:18:52.500854 11045 log.cc:927] Time spent T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:18:52.498164  6473 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:52.499460  6580 raft_consensus.cc:1240] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:52.456969 10812 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:52.514739 11038 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:52.516402 11024 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:52.517747 10919 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:52.437587 11050 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:52.522956 11050 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:52.490649 10472 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:18:52.526203 11051 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:52.526844 11051 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:52.306707 11012 consensus_queue.cc:237] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.4, Last appended by leader: 4, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:52.529947 10966 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:52.530406  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:18:52.551800  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 54699.3us
I20260709 12:18:52.553006  7238 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:52.553861 11068 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:52.554279  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:52.555254  7238 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:52.554440 11068 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:52.555640 11043 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.557593 11050 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:52.556704  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:18:52.558979 11014 log.cc:927] Time spent T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 12:18:52.560282  6580 raft_consensus.cc:1275] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:18:52.563009  6654 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 9. Current term is 10. Ops: [9.9-9.9]
W20260709 12:18:52.584868  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:52.559619  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 97250.5us
I20260709 12:18:52.615360  6654 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:18:52.617578  6580 raft_consensus.cc:1275] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 10 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
W20260709 12:18:52.654183  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 156100us
W20260709 12:18:52.616578  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 59948.9us
I20260709 12:18:52.655709  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:52.656773  6473 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:18:52.650862  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 186983us
I20260709 12:18:52.607211 11038 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.498453 11055 consensus_queue.cc:1059] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.120s
I20260709 12:18:52.662863  6548 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:52.606909 11068 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:52.606801 10919 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.561064 11057 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:52.612461 11050 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:18:52.676733  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:52.693985  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.664018  6550 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:52.562505 11066 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:52.511930 10877 consensus_queue.cc:237] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:52.707394  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:52.720218  6500 raft_consensus.cc:2410] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for term 15 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
W20260709 12:18:52.716229  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:52.693220 11036 log.cc:927] Time spent T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:18:52.722436 11063 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:52.669240 11051 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:52.669798 10985 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:52.752982  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:52.675549 11055 raft_consensus.cc:3055] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:18:52.669916 11065 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:18:52.697485 11046 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:52.728848  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 68209.7us
W20260709 12:18:52.762928  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:52.766373  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:52.769474  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 75021.9us
I20260709 12:18:52.718489 11050 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:52.674592 11067 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:18:52.786999  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 124360us
I20260709 12:18:52.669046 11038 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:52.776655  6473 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:52.701093 11057 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:52.792850  7238 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:18:52.777138 11053 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.763224 11055 raft_consensus.cc:740] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:52.717298 10919 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:52.762828 11074 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:52.825820 11074 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:52.826095  7266 raft_consensus.cc:1240] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 8 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:52.826321 11074 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:52.772202 10978 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:52.826943 11038 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:52.797111 11063 raft_consensus.cc:697] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 11 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:52.827803  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:52.806530 11081 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:52.828924 11076 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:52.829620 11031 raft_consensus.cc:2804] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:52.806764  7265 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41"
dest_uuid: "20a202c76244485fbc8f1610473db8b9"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:48510
I20260709 12:18:52.830169 11031 raft_consensus.cc:697] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:52.832247  7265 raft_consensus.cc:606] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 9 LEADER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:52.778966 11051 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:52.834770  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:52.794893 11067 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:52.836061  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:52.841444  6501 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:52.779210 11066 consensus_queue.cc:1048] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
W20260709 12:18:52.851666  6656 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58980 (request call id 2551) took 1804 ms (client timeout 1776 ms). Trace:
I20260709 12:18:52.853682  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:52.854938  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:52.855490  6500 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:52.852216  6656 rpcz_store.cc:269] 0709 12:18:51.047220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:51.072257 (+ 25037us) service_pool.cc:224] Handling call
0709 12:18:52.851382 (+1779125us) spinlock_profiling.cc:243] Waited 302 ms on lock 0x7b54000758d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:52.851527 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":693998464}
I20260709 12:18:52.856832  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:52.857453  6509 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:51.689158  6505 rpcz_store.cc:276] 0709 12:18:48.274028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.305382 (+ 31354us) service_pool.cc:224] Handling call
0709 12:18:48.427312 (+121930us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:50.536644 (+2109332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:50.536681 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:50.537202 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 12:18:51.588959 (+1051757us) spinlock_profiling.cc:243] Waited 21.8 ms on lock 0x7b54003186a0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df3090 00007f2875e006b8 00007f2875e8992b 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c
0709 12:18:51.589044 (+    85us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:51.643603 (+ 54559us) log.cc:844] Serialized 14 byte log entry
0709 12:18:51.682125 (+ 38522us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:51.682210 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:51.682261 (+    51us) raft_consensus.cc:1685] finished
0709 12:18:51.685543 (+  3282us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:51.685710 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":677,"prepare.queue_time_us":82766,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":2650,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":1427,"spinlock_wait_cycles":50198784,"thread_start_us":100307,"threads_started":4,"wal-append.queue_time_us":3711,"wal-append.run_cpu_time_us":2095,"wal-append.run_wall_time_us":1026295}
I20260709 12:18:52.858253  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:52.611647 11043 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:52.861902 11043 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:52.810650 11053 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:52.866055  6653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2006) took 4658 ms. Trace:
I20260709 12:18:52.866518  6653 rpcz_store.cc:276] 0709 12:18:48.207451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.459208 (+251757us) service_pool.cc:224] Handling call
0709 12:18:51.069189 (+2609981us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:51.102842 (+ 33653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:52.865820 (+1762978us) spinlock_profiling.cc:243] Waited 2.61 s on lock 0x7b54002a3ad0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:52.865971 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6001263360}
I20260709 12:18:52.801811 11057 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:52.675796 11033 consensus_queue.cc:1048] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.194s
I20260709 12:18:52.801894 11065 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:52.873075  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:52.873966  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2056) took 1348 ms. Trace:
I20260709 12:18:52.874385  6577 rpcz_store.cc:276] 0709 12:18:51.525390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:51.525602 (+   212us) service_pool.cc:224] Handling call
0709 12:18:51.537528 (+ 11926us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:52.477245 (+939717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:52.873580 (+396335us) spinlock_profiling.cc:243] Waited 931 ms on lock 0x7b540034c6d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:52.873865 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2141038336}
W20260709 12:18:52.877283 11075 log.cc:927] Time spent T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 12:18:52.878279  6501 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:52.881915  6500 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:18:52.891232  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:52.892154  6506 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:18:52.778883 11046 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:52.900722 11046 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:52.831362 11031 consensus_queue.cc:237] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:52.845121 11067 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:52.891577 10974 consensus_meta.cc:287] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.530s	user 0.013s	sys 0.000s
I20260709 12:18:52.906844  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:52.907030  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:52.828935 11081 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:52.908957  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:52.909348  6652 raft_consensus.cc:1240] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:52.880807 11053 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:52.914675  6656 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:52.915443  6653 raft_consensus.cc:1240] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:52.919993  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:52.913208  6506 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:18:52.910548  6655 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:52.918731  6508 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:52.922194 11085 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:52.922435  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:52.922652 11085 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:52.922951  6651 raft_consensus.cc:1240] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:52.923259  6656 raft_consensus.cc:2410] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 13 because replica has last-logged OpId of term: 11 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 4.
I20260709 12:18:52.878582 11051 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:52.927496  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:52.933292  6654 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:52.828303 11063 consensus_queue.cc:237] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:52.935899  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:52.911695 10974 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:18:52.941977  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 58241us
I20260709 12:18:52.942183  6651 raft_consensus.cc:1240] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 11 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:52.941715  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:52.943436  6508 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:52.940457  6653 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:52.939958  6652 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:18:52.948064  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:52.949098  6656 raft_consensus.cc:2393] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 12: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:52.952355  6654 raft_consensus.cc:1275] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 12:18:52.952699  6654 pending_rounds.cc:85] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 6
I20260709 12:18:52.952891  6654 pending_rounds.cc:107] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:18:52.953094  6654 raft_consensus.cc:2889] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:52.928195 11085 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:52.958984 11081 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:52.962097  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:52.962756  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:52.965785  6653 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:52.966861  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:52.969094 11068 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:52.972656 11068 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:52.974050  6654 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:52.906154 11055 consensus_queue.cc:260] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:52.977164 11024 consensus_queue.cc:1059] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:52.978869 11051 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:52.984416 10795 raft_consensus.cc:3055] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:52.984961 10795 raft_consensus.cc:740] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:52.986074 10795 consensus_queue.cc:260] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:52.987859 10795 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:52.989235  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 161769us
I20260709 12:18:52.990617  6548 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:52.990736 11076 consensus_queue.cc:1059] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 10.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:52.991120  6473 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:52.992090  6473 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:52.993259  6473 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:52.994032 11043 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:52.994418 11043 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:52.994764 11043 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:52.996711  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:52.997361 10997 consensus_meta.cc:287] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.594s	user 0.011s	sys 0.000s
I20260709 12:18:52.997546 11007 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:52.997575 11065 raft_consensus.cc:2764] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:18:52.997402  6652 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 14.
I20260709 12:18:52.998755  6584 raft_consensus.cc:2410] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:18:52.984419 11057 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:53.004557 10984 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:53.004648 11012 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:53.005056 11057 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:53.005295 10984 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:53.005407 11012 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:53.008430 11012 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:53.009052 11012 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:53.009545 11012 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:53.007704 10984 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:52.957356  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:53.013176  6653 raft_consensus.cc:1240] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 6. Current term is 7. Ops: []
W20260709 12:18:53.024938  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 145201us
I20260709 12:18:53.012115 11012 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:53.012236 11027 log.cc:927] Time spent T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8: Append to log took a long time: real 1.362s	user 0.003s	sys 0.008s
I20260709 12:18:53.029656  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:53.036479  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:53.026711  6656 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:18:52.992221 11076 raft_consensus.cc:3055] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:53.056099  6655 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:18:53.059774  6548 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:53.022815 11018 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.698s	user 0.011s	sys 0.008s
I20260709 12:18:53.063967  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:53.062595  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:53.078604  6655 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:53.078860  6501 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:18:53.061239  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:53.083478  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:53.081071 11018 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.022s	sys 0.012s
I20260709 12:18:53.058833 10812 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:53.091517  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:53.059147 11038 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:53.022694 11050 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:53.092975  6652 raft_consensus.cc:2393] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 8: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:53.083402 10978 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:53.061515 11076 raft_consensus.cc:740] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:53.080356 10820 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:53.096606  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 68466.9us
I20260709 12:18:53.097613 11031 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:53.098644 11077 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:53.103229  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:53.120374 11029 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.891s	user 0.007s	sys 0.006s
I20260709 12:18:53.092047 11074 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:53.279981 11067 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:53.306900 11067 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:53.091363 11066 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:53.309072 11066 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:53.282130  6548 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:53.282687 11085 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:53.319540 11047 log.cc:927] Time spent T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.997s	user 0.001s	sys 0.000s
I20260709 12:18:53.321177 11085 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:53.285140 11050 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:53.316131  6507 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:53.111336 11031 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:53.103309 11038 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:53.113443 10820 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:53.288681  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:53.288862 11076 consensus_queue.cc:260] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.4, Last appended by leader: 4, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:53.120508 11077 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:53.294292  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.333966  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:53.284096  6655 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:18:53.335830 10812 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:53.336874 11076 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:53.338053 11038 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:53.339956 10820 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:53.295078  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 188367us
I20260709 12:18:53.341384  6550 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:53.297237  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:53.342447  6504 raft_consensus.cc:2393] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 11: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:53.329751 11031 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:53.297482  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:53.343245  6503 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
W20260709 12:18:53.300367 11056 log.cc:927] Time spent T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.690s	user 0.002s	sys 0.000s
I20260709 12:18:53.346231  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2557) took 2171 ms. Trace:
I20260709 12:18:53.346683  6649 rpcz_store.cc:276] 0709 12:18:51.174405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:51.224259 (+ 49854us) service_pool.cc:224] Handling call
0709 12:18:51.255924 (+ 31665us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:51.377833 (+121909us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:53.345966 (+1968133us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b54000b7554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:53.346158 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":278392064}
I20260709 12:18:53.296809  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:53.352949  6501 raft_consensus.cc:2393] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 7: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:53.314005  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:53.316504 11074 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:53.303690 11029 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.116s	user 0.015s	sys 0.009s
I20260709 12:18:53.312060 11066 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:53.318197 11039 consensus_queue.cc:1048] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:53.342473 11077 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:53.319161  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.324795  6548 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:53.336592 10812 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:53.313598 10996 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:18:53.354784  7240 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.003s	sys 0.024s
W20260709 12:18:53.356462  7240 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.006s	sys 0.024s
W20260709 12:18:53.355767  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.357523 10812 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:53.295379 11094 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:53.357940  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:53.359280  6656 raft_consensus.cc:2393] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 10: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:53.359858  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:53.360200  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:53.360872  6656 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 12.
I20260709 12:18:53.362051  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:53.362426  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:53.362733  6656 raft_consensus.cc:2393] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 10: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:53.363756  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:53.366139  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:53.367165  7271 raft_consensus.cc:2393] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 11: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:53.368556  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:53.369247  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:53.369379  6649 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 14.
I20260709 12:18:53.369971  6656 raft_consensus.cc:2393] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 10: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:53.370857  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:53.371477  6656 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 9.
I20260709 12:18:53.379072  6507 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:18:53.379012  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:53.381259  7265 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:18:53.335918  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:53.285573 11043 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:53.384658 11083 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:53.385166 10962 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:53.388228 10996 raft_consensus.cc:697] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:18:53.387539  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 62779.6us
I20260709 12:18:53.388396 11094 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:53.384325  6655 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 14.
I20260709 12:18:53.391808  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 2035) took 1181 ms. Trace:
I20260709 12:18:53.398779  6584 rpcz_store.cc:276] 0709 12:18:52.210481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.314071 (+103590us) service_pool.cc:224] Handling call
0709 12:18:53.391509 (+1077438us) spinlock_profiling.cc:243] Waited 683 ms on lock 0x7b54001cd254. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:53.391722 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1571258624}
I20260709 12:18:53.399371  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:53.395828 11083 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:53.405336  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:53.400929 10996 consensus_queue.cc:237] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:53.409759  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:53.410456  6653 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 12.
I20260709 12:18:53.412747 10962 raft_consensus.cc:697] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:53.414268 11064 raft_consensus.cc:3055] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:53.414767 11064 raft_consensus.cc:740] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:53.417048  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:53.417764  6649 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:18:53.419207 11077 consensus_queue.cc:1059] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:18:53.420166 11077 raft_consensus.cc:3055] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:53.420534 11077 raft_consensus.cc:740] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:53.421810 10919 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:53.421526 11077 consensus_queue.cc:260] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:53.423383 11077 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:53.423517 10747 consensus_queue.cc:1059] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:53.425145  6650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2575) took 1098 ms. Trace:
W20260709 12:18:53.476658  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.415786 11064 consensus_queue.cc:260] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:53.492491  6649 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 11.
W20260709 12:18:53.492026  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 95135.7us
I20260709 12:18:53.414007 10962 consensus_queue.cc:237] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:53.409821 10795 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:53.428368  6650 rpcz_store.cc:276] 0709 12:18:52.326871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:52.419349 (+ 92478us) service_pool.cc:224] Handling call
0709 12:18:52.433093 (+ 13744us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:52.433373 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:52.493262 (+ 59889us) log.cc:844] Serialized 14 byte log entry
0709 12:18:52.494624 (+  1362us) log.cc:844] Serialized 14 byte log entry
0709 12:18:53.424440 (+929816us) spinlock_profiling.cc:243] Waited 3.75 ms on lock 0x7b18003569e0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d16c0a 00007f287c456219 00007f2875a3fd70 00007f2875a33293 00007f2875a28cea 00007f2875a2923a 00007f2875e93a2a 00007f2875ea30bf 00007f2875ea3038
0709 12:18:53.424726 (+   286us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:53.424783 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:53.424852 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:53.424928 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:53.425076 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4108,"prepare.queue_time_us":128613,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":800,"raft.queue_time_us":1128,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":4417,"spinlock_wait_cycles":8623872,"thread_start_us":43451,"threads_started":1}
W20260709 12:18:53.494585  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 69620.1us
I20260709 12:18:53.500985  6649 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:18:53.501741 10989 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:18:53.502457 10989 consensus_queue.cc:1243] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:18:53.507438  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:53.507905  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:53.510517  6577 raft_consensus.cc:1240] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:53.510645  6548 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:53.511711  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:53.512430  6577 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:18:53.513963  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:53.514523  6580 raft_consensus.cc:2393] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 9: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:53.515175  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:53.511761  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:53.513329  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:53.515867  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:53.504168 11064 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:53.496438 10985 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 12:18:53.519104  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:53.522146  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:53.523969  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:53.522292  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:53.525527  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:53.528414  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:53.529345  6580 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:53.499893 10795 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:53.534207 10795 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:53.540982  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:53.541778  6580 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:18:53.544445  6577 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:18:53.547415  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:53.503929 11031 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:53.549363 11043 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:53.550987  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:53.553727  6473 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:18:53.558063  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 47487.8us
I20260709 12:18:53.556988  6473 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:53.556032  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:53.517223 10758 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:18:53.561424  6473 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:53.563373  6581 raft_consensus.cc:1275] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:53.563540  6473 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:53.557507 10919 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:53.574018  6578 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 11. Current term is 12. Ops: [10.5-11.6]
I20260709 12:18:53.583500  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2046) took 2138 ms. Trace:
I20260709 12:18:53.583992  6578 rpcz_store.cc:276] 0709 12:18:51.445416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.558620 (+2113204us) service_pool.cc:224] Handling call
0709 12:18:53.565171 (+  6551us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:53.583024 (+ 17853us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:53.583423 (+   399us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:53.581379  6473 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:53.561582 11066 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:53.595319  6473 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:53.557430 11043 consensus_queue.cc:1059] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 12:18:53.563233 11039 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:53.592453  6548 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:53.586647 10919 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:53.582594 10820 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:53.594694 11038 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:53.565107 10812 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:53.593000 10996 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:53.564615 11017 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:18:53.600762  6548 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:53.591684 11031 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:53.599463 11039 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:53.604115 10919 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:53.603500 11094 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:53.605240 11038 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:53.629118  6508 raft_consensus.cc:1275] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 10 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 12:18:53.600978 10747 consensus_queue.cc:1059] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 12:18:53.638907 10978 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:53.639698 10978 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:53.641665  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:53.647814  7238 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.844s	user 0.005s	sys 0.011s
I20260709 12:18:53.618256 11039 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:53.617486 11023 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:18:53.656518  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.633000 11038 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:53.656111  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:53.658794  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:53.659662  6658 raft_consensus.cc:2410] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 14 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:18:53.661010  6658 raft_consensus.cc:1240] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 6 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 5. Current term is 6. Ops: []
I20260709 12:18:53.664165  6658 raft_consensus.cc:1275] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 12:18:53.677196  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 87612.1us
I20260709 12:18:53.617450 11066 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:18:53.679490  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.617130  6548 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:53.680567 11066 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:53.691411  7238 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.888s	user 0.006s	sys 0.011s
W20260709 12:18:53.691983  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 899137us
I20260709 12:18:53.689258 11066 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:53.698205 11085 consensus_queue.cc:1059] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:53.737257 10978 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:53.739573 10978 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:53.740074 11109 log.cc:927] Time spent T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 12:18:53.750551 11106 log.cc:927] Time spent T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:18:53.753278  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:53.754186  7274 raft_consensus.cc:2410] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:18:53.760517 10919 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:53.770432  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.776862 10472 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:18:53.778255  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.780040 10919 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:53.782694  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:53.783340  6501 raft_consensus.cc:2393] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 13: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:53.790014  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:53.790334  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:53.790342  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:53.790707  6500 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:18:53.791070  6502 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:53.791654  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:53.771709  6395 tablet_replica.cc:1406] Time spent applying in-flights took a long time: real 0.275s	user 0.000s	sys 0.000s
I20260709 12:18:53.793759 11050 raft_consensus.cc:3055] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:53.794160 11050 raft_consensus.cc:740] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:53.795073 11050 consensus_queue.cc:260] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:53.805759  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:53.811964  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:53.806181  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:53.808871  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 111147us
W20260709 12:18:53.811328  6622 connection.cc:256] RPC call timeout handler was delayed by -1.0550786920000519s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15880000000000002s for a response
W20260709 12:18:53.813650  6622 connection.cc:256] RPC call timeout handler was delayed by -1.0550786920000519s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15880000000000002s for a response
W20260709 12:18:53.814162  6622 connection.cc:256] RPC call timeout handler was delayed by -1.0109786919999806s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1637s for a response
I20260709 12:18:53.816776  6622 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:53.767740 11039 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:53.821866  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:53.823559  6506 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:53.824389  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:53.825083  6501 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:53.826263 11038 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:53.824465  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:53.828356  6503 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:53.815716 11085 consensus_queue.cc:1059] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 12:18:53.829205 11085 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:53.829450 11121 raft_consensus.cc:3055] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:53.829728 11050 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:53.829860 11121 raft_consensus.cc:740] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:53.834461 10963 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:53.837872  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:53.838665  6500 raft_consensus.cc:2393] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 11: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:53.823012  7238 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:53.842761  6503 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:18:53.842221 10812 raft_consensus.cc:3055] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:53.843945 10819 consensus_queue.cc:1048] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:18:53.846532 10812 raft_consensus.cc:740] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:18:53.846850 11111 log.cc:927] Time spent T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:18:53.844491 11012 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:18:53.850567  7268 raft_consensus.cc:1275] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:18:53.852459 11012 consensus_queue.cc:1048] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:53.854003  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:53.854229  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:53.854640  6504 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:53.856240  6501 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
I20260709 12:18:53.857614 11039 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:53.871006  6679 debug-util.cc:398] Leaking SignalData structure 0x7b080013cf20 after lost signal to thread 5355
I20260709 12:18:53.871513 11066 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:53.872860  6502 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 12. Current term is 13. Ops: []
W20260709 12:18:53.872200  6679 debug-util.cc:398] Leaking SignalData structure 0x7b0800373d00 after lost signal to thread 6441
W20260709 12:18:53.875715  6679 debug-util.cc:398] Leaking SignalData structure 0x7b08001c44c0 after lost signal to thread 6533
W20260709 12:18:53.876787  6679 debug-util.cc:398] Leaking SignalData structure 0x7b080030a2e0 after lost signal to thread 6608
W20260709 12:18:53.877028  6679 debug-util.cc:398] Leaking SignalData structure 0x7b080031c040 after lost signal to thread 6609
W20260709 12:18:53.878103  6679 debug-util.cc:398] Leaking SignalData structure 0x7b08001ca980 after lost signal to thread 6682
I20260709 12:18:53.840659  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:53.878648  6679 debug-util.cc:398] Leaking SignalData structure 0x7b08000b1440 after lost signal to thread 6683
I20260709 12:18:53.879398  6622 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:53.880686  6504 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
I20260709 12:18:53.882054  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:53.896136  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 80161.4us
I20260709 12:18:53.899114  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:53.906298  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:53.900277  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:53.906826  7240 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:53.904372  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 67201.6us
W20260709 12:18:53.908174  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:53.905315  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 63508.5us
I20260709 12:18:53.906986  6657 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
W20260709 12:18:53.911774  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.897997 11012 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:53.913776  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:53.913951  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 165260us
I20260709 12:18:53.897657 11038 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:53.914206  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:53.915762  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:53.957569  6651 raft_consensus.cc:2393] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 12: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:53.904680 10877 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:53.907985 10615 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:53.958853  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:53.945284  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:53.908851 11120 raft_consensus.cc:993] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9: : Instructing follower f2b06aebe97249a4877d798b8118ece4 to start an election
I20260709 12:18:53.959664  6580 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:53.916451 11123 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:18:53.963366  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 52486us
I20260709 12:18:53.964010  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:53.830780 11121 consensus_queue.cc:260] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:53.972420  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:53.973646  6654 raft_consensus.cc:2410] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:18:53.916224  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:53.971297  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:53.975414  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:53.957587 11074 raft_consensus.cc:2764] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 12:18:53.976327  6577 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:53.957348 11012 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:53.954916 11126 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:53.982081  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:53.971940 11110 log.cc:927] Time spent T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:18:53.995258  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:53.985611  6395 maintenance_manager.cc:643] P 39c4be726d3d495181bfe7425c8c0bd8: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 11.459s	user 0.214s	sys 0.067s Metrics: {"bytes_written":29221,"cfile_init":1,"dirs.queue_time_us":422979,"dirs.run_cpu_time_us":364,"dirs.run_wall_time_us":7980,"drs_written":1,"lbm_read_time_us":408,"lbm_reads_lt_1ms":4,"lbm_write_time_us":196293,"lbm_writes_1-10_ms":1,"lbm_writes_10-100_ms":8,"lbm_writes_lt_1ms":20,"peak_mem_usage":0,"rows_written":63,"spinlock_wait_cycles":9791179136,"thread_start_us":54011,"threads_started":1}
I20260709 12:18:53.978878 10615 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:53.992828 11121 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:53.993660 11126 raft_consensus.cc:1081] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 9 LEADER]: Signalling peer f2b06aebe97249a4877d798b8118ece4 to start an election
W20260709 12:18:53.994367  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 380736us
I20260709 12:18:54.007355  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:53.982753  6579 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:54.007396  6580 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 11.
I20260709 12:18:53.996101  6577 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:18:53.992014 11123 raft_consensus.cc:2955] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 16 LEADER]: Committing config change with OpId 7.6: config changed from index 2 to 6, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:18:53.966442 10472 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:18:53.977393 11066 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:53.982172 11012 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:54.028769  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 142602us
I20260709 12:18:54.028806  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:54.000852 10615 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:54.041950  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:54.042577  6583 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:54.031183 11007 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.056945 11007 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:54.059300 11007 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:54.062954  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 56252us
I20260709 12:18:54.076512  6579 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:18:54.064029  7238 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:54.041389 10615 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:54.085372  7238 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:54.086309  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:54.087984  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:54.088673  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:54.088861  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:54.089161  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:54.089509  6656 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:18:54.089766  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:54.090742  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:54.091030  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:54.091398  6657 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:54.092221  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:54.092905  6657 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:54.093899  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:54.094470  6655 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:18:54.096903 10877 raft_consensus.cc:697] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:54.097992  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:54.100425  6475 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:54.102470  6475 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:54.105427  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 97559.8us
I20260709 12:18:54.106223  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:54.118548  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:54.115532  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 66667.1us
I20260709 12:18:54.117393  6548 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:54.114431  6396 maintenance_manager.cc:419] P 39c4be726d3d495181bfe7425c8c0bd8: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7846 bytes on disk
I20260709 12:18:54.127844  7274 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09"
dest_uuid: "20a202c76244485fbc8f1610473db8b9"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:48510
W20260709 12:18:54.127859  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 228794us
I20260709 12:18:54.128799  7274 raft_consensus.cc:606] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:54.129325  7274 raft_consensus.cc:614] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:54.129808  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:54.131659  6576 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:18:54.134426  6475 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:54.082479 10738 consensus_queue.cc:1048] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 12:18:54.107836 11132 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:54.115366 11127 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:54.108429 11038 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:54.147269  6475 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:54.115257 11131 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:54.151424  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:54.126365 10819 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:54.130865 11118 log.cc:927] Time spent T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.341s	user 0.005s	sys 0.000s
I20260709 12:18:54.108702 11066 consensus_queue.cc:1059] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 12:18:54.131608 11043 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:54.135953 11049 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.163612  6395 maintenance_manager.cc:643] P 39c4be726d3d495181bfe7425c8c0bd8: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.018s	user 0.001s	sys 0.001s Metrics: {"cfile_init":1,"lbm_read_time_us":403,"lbm_reads_lt_1ms":4}
I20260709 12:18:54.152913 11115 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:18:54.170828  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 64680.6us
I20260709 12:18:54.175227  6548 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:54.146286 10919 consensus_queue.cc:1059] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:18:54.157729 11038 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:54.192790  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:54.161623 10472 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:54.196064  6649 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:54.199416  6548 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:54.197201  6658 raft_consensus.cc:1275] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:18:54.201444  6550 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:54.172619 11049 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:54.169188 11132 raft_consensus.cc:697] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 8 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:54.202970  6548 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:54.164369 11112 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:54.155061 10978 consensus_queue.cc:1059] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:18:54.169808 11127 raft_consensus.cc:697] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 10 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:54.173786 11131 raft_consensus.cc:697] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 10 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:54.197563 10990 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:54.197311 11038 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:54.201689 11115 consensus_queue.cc:1048] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 12:18:54.160995 10877 consensus_queue.cc:237] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:54.207536 11030 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:54.208035 11112 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:54.170498 10819 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:54.205560 10738 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 12:18:54.207369 11083 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:54.216086 10990 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:54.215767 10610 consensus_queue.cc:1048] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 12:18:54.249943 10819 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:54.251554 10991 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:18:54.216675  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 87488.8us
I20260709 12:18:54.250521 11132 consensus_queue.cc:237] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:54.240995 11127 consensus_queue.cc:237] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:54.267777  6622 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:54.278091  6622 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:54.258205 11092 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:54.266378 10990 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:54.228673 11112 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:54.285331 10472 consensus_queue.cc:1048] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:54.286794 10472 consensus_queue.cc:1048] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:54.245375 11131 consensus_queue.cc:237] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:54.223953 11137 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:54.290982 11137 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:54.291496 11137 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:54.293403  6657 raft_consensus.cc:2410] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 13 because replica has last-logged OpId of term: 11 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 4.
I20260709 12:18:54.251935 10978 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.305936  6654 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:54.282677 11092 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:54.274391  6583 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:18:54.295619  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:54.301162 11134 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:18:54.320295  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 148326us
I20260709 12:18:54.301648 11141 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:54.324571  6475 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:54.314993  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:54.312903 11092 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:54.323721 11142 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:54.333101 11066 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:54.334168 11142 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:54.335822 11142 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:54.334275 11066 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:54.336277 11066 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:54.332271  6475 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:54.324455 11141 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:54.307655 10978 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:54.342447  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:54.343443  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:54.345397  6579 raft_consensus.cc:1240] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:54.348469  6475 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:54.351208  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:54.356387  6578 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:54.356724  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:54.346464 11141 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:18:54.368551  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 461754us
I20260709 12:18:54.372467  7240 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:54.373867  7240 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:54.380846  7240 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:54.353546 11083 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:54.382122  7240 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:54.384207  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:54.356180 11144 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:54.393508  7240 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:54.378794 11131 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:18:54.387389 11121 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:54.401882  6579 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14. Ops: []
W20260709 12:18:54.399435  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 84532.7us
I20260709 12:18:54.370872 10615 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:54.389071 11130 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:54.402441  7240 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:54.409214  6622 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
W20260709 12:18:54.405876  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 51273.4us
I20260709 12:18:54.417487 11055 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:18:54.403993 11134 raft_consensus.cc:697] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 10 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:54.418550 11148 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:54.419104 11148 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:54.419597 11148 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:54.414942  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:54.420120 11134 consensus_queue.cc:237] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:54.403827 11139 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:54.401340 11121 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:54.431628  6657 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:49792
I20260709 12:18:54.396214 11012 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:54.432140 10819 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.431083 11120 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:54.433413  6475 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:54.450680  6657 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:54.473352  6657 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:54.403919 11147 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:54.479673 11112 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:54.484913 11152 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260709 12:18:54.445802 11150 log.cc:927] Time spent T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
W20260709 12:18:54.446412 11151 log.cc:927] Time spent T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:18:54.496400  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 63052.1us
I20260709 12:18:54.495226  6657 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:54.508323  6475 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:54.433681 11144 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:54.451619 11121 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:54.488834 11147 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:54.509462 11083 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.524631 11147 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:54.525017 11083 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:54.527447 11083 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:54.476986 10819 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:54.530217 10819 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:54.511193 10978 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:54.511752 10795 raft_consensus.cc:3055] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:54.532035 10795 raft_consensus.cc:740] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:54.506584 11148 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:54.443053 10877 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:54.441819 10615 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:54.507499 11139 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:54.499290 11120 raft_consensus.cc:697] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 11 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:54.515817  6473 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:54.517861  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:54.535476  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 126323us
I20260709 12:18:54.535770  6580 raft_consensus.cc:1240] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:54.537097  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:54.546243  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:54.547786  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:54.548017  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:54.558321  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:54.559020  6622 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:54.559731  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.511950 11115 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:18:54.515169 11112 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:54.565949  6473 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:54.533005 10795 consensus_queue.cc:260] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:54.564873 11058 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:54.514302 11157 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:54.539999 10877 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:18:54.592001  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 83756.6us
I20260709 12:18:54.593070  6475 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:54.533970 11066 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:54.517782 11142 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:54.573009 10615 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:54.533844 11038 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:54.600600 10819 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:54.601410 11154 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:54.601982 11120 consensus_queue.cc:237] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:54.634822  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 505) took 6444 ms. Trace:
I20260709 12:18:54.611943 11040 raft_consensus.cc:3055] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:54.612654 11012 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:54.613560 11123 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.615172  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:54.650157  7273 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 11 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:54.616601  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2170) took 1301 ms. Trace:
I20260709 12:18:54.617197  7269 raft_consensus.cc:1240] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:54.651546  6652 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:18:54.618119  7271 raft_consensus.cc:1275] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:18:54.571560 10978 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:54.585752 11083 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:18:54.634239  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.589238 10877 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:54.661501  6658 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:54.604832 10919 raft_consensus.cc:3055] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:54.628446 11154 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:54.635629  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:54.635243  6507 rpcz_store.cc:276] 0709 12:18:48.179922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.245184 (+ 65262us) service_pool.cc:224] Handling call
0709 12:18:48.245474 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:53.315664 (+5070190us) spinlock_profiling.cc:243] Waited 3.05 s on lock 0x7b58003f41d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875de50aa 00007f2875e67447 00007f2875e7ffe7 00007f2875e817de 00007f2875e8bb49 00007f2875e8c416 00007f2875e88d8f 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4
0709 12:18:53.380519 (+ 64855us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:54.623532 (+1243013us) spinlock_profiling.cc:243] Waited 22.4 ms on lock 0x7b54000ff354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:54.623717 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7062135424}
I20260709 12:18:54.615370  7266 raft_consensus.cc:1240] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 8 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:54.651252  6508 rpcz_store.cc:276] 0709 12:18:53.315592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:53.321067 (+  5475us) service_pool.cc:224] Handling call
0709 12:18:53.321388 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:53.631187 (+309799us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:54.616398 (+985211us) spinlock_profiling.cc:243] Waited 307 ms on lock 0x7b54001e73d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:54.616528 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":706763648}
W20260709 12:18:54.653569 11150 log.cc:927] Time spent T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:18:54.620252 11148 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:54.642477 11144 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:54.624150 10795 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:54.629890 10615 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:54.664062  7271 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 10 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:54.600180  6475 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:54.673601 11141 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:54.643061 11040 raft_consensus.cc:740] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:54.700067  7271 raft_consensus.cc:1275] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:18:54.643041 10996 consensus_queue.cc:1048] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 12:18:54.697055 10615 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:54.664685 10919 raft_consensus.cc:740] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:54.686175 11083 raft_consensus.cc:697] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:54.705608  7270 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 10 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:54.686539 10978 raft_consensus.cc:697] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:54.733381 10978 consensus_queue.cc:237] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:54.708467  6679 debug-util.cc:398] Leaking SignalData structure 0x7b08003d0da0 after lost signal to thread 7298
W20260709 12:18:54.736526  6679 debug-util.cc:398] Leaking SignalData structure 0x7b0800376a80 after lost signal to thread 7299
W20260709 12:18:54.736861  6679 debug-util.cc:398] Leaking SignalData structure 0x7b08001a8ca0 after lost signal to thread 10258
I20260709 12:18:54.737349 11066 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:54.686821 11154 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:54.664043 10990 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:54.692641  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 117008us
W20260709 12:18:54.724613 11159 log.cc:927] Time spent T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:18:54.742585  6473 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:54.725206 10812 consensus_queue.cc:260] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:54.727962  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.731426 11043 raft_consensus.cc:3055] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:54.661666 11123 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:54.743891  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:54.754383 10990 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:54.755240 10472 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:18:54.755357 10990 consensus_queue.cc:1059] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:54.758745  6650 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:54.758841 11066 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:54.759590 11066 raft_consensus.cc:697] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 13 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:54.760651 11066 consensus_queue.cc:237] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:54.764005  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:54.764680  7266 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:54.766300 11123 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:54.768187  6581 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:54.768967 11141 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:54.769994 11040 consensus_queue.cc:260] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:54.771565 11040 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:54.781487 11038 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:54.704061 11167 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:54.785421  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 63219.2us
W20260709 12:18:54.795646  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 195526us
I20260709 12:18:54.714491 11144 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:54.799093  7268 raft_consensus.cc:1275] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:54.796841  6475 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:54.802466 10812 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:54.804476  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:54.784399 11043 raft_consensus.cc:740] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:54.817705  6475 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:54.794241 10986 raft_consensus.cc:3055] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:54.824740  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:54.845598  7270 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:54.827069  7271 raft_consensus.cc:2468] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:54.797382 11169 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.797780 11167 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:54.833124 11144 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:54.789680 11092 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:54.843235  6475 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:54.812752 11142 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:54.784072 11137 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:54.879593  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:54.889907  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 46787.8us
I20260709 12:18:54.927546  6475 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:54.878742  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 144402us
I20260709 12:18:54.844817 10986 raft_consensus.cc:740] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:54.854350 11143 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:54.932976  6583 raft_consensus.cc:1275] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 8 index: 3. Preceding OpId from leader: term: 13 index: 5. (index mismatch)
W20260709 12:18:54.802211 11038 consensus_queue.cc:1175] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "496c7c3792cd4587b4dbeba435f4be1e" responder_term: 10 status { last_received { term: 7 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10. Ops: [9.5-9.5]" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:18:54.939985 11164 log.cc:927] Time spent T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:18:54.865214 11067 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:54.940834 11067 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:54.942631 11006 consensus_queue.cc:1048] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:54.942828 10996 consensus_queue.cc:1059] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:54.943572 11067 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:54.944103 10963 consensus_queue.cc:1059] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:54.944383 11006 consensus_queue.cc:1048] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:18:54.945062 10963 consensus_queue.cc:1243] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer 20a202c76244485fbc8f1610473db8b9 log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:18:54.945652 10963 consensus_queue.cc:1059] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:54.946516 10963 consensus_queue.cc:1243] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer 20a202c76244485fbc8f1610473db8b9 log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:18:54.947072 11074 raft_consensus.cc:3055] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:54.947481 11074 raft_consensus.cc:740] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:54.947584  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:18:54.948750 11170 log.cc:927] Time spent T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:18:54.948477 11074 consensus_queue.cc:260] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:54.947036 11006 raft_consensus.cc:3055] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:54.950110 11006 raft_consensus.cc:740] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:54.950440 10747 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:54.951193 10819 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:54.950984 11006 consensus_queue.cc:260] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:54.951022 10747 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:54.952358 11006 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:54.953898 10747 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:54.956493  6582 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:18:54.943711 10996 consensus_queue.cc:1243] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer 20a202c76244485fbc8f1610473db8b9 log is divergent from this leader: its last log entry 8.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:18:54.958348  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:54.959060  6583 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:18:54.958732 10986 consensus_queue.cc:260] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:54.876549 10962 consensus_queue.cc:1059] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 12:18:54.960500 10962 raft_consensus.cc:3055] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:54.961005 10962 raft_consensus.cc:740] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:54.963498  6650 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 15. Current term is 16. Ops: []
I20260709 12:18:54.964689 11137 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:18:54.926295 11055 consensus_meta.cc:287] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.508s	user 0.012s	sys 0.000s
I20260709 12:18:54.976842  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:54.964610  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:54.889467 11144 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:54.970736  6473 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:54.889907 11169 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:54.977504  6653 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 6.
I20260709 12:18:54.989078  6653 raft_consensus.cc:1240] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:54.978924  6584 raft_consensus.cc:1275] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:18:54.979415  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:54.992873  6577 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:54.996711  6550 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:54.999244  7270 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:18:55.000682  6623 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:54.981105  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:54.977511 11118 log.cc:927] Time spent T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:18:54.969228 11143 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:54.962399 10962 consensus_queue.cc:260] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:54.935599 11175 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:54.982649 11167 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:55.011250  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 437475us
I20260709 12:18:54.996871 11144 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:55.015556  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:55.008900 10963 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:18:54.874923 10919 consensus_queue.cc:260] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:54.890777 10877 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:55.029814  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:55.037835  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:54.921101 11083 consensus_queue.cc:237] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:55.010111 11143 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:55.055459  6582 raft_consensus.cc:606] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:55.038628 11167 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:55.058207  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:55.047967 10819 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:55.060781  6651 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:55.061369 11169 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:55.061522 11154 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:55.062321 11092 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:55.013509 11175 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:55.060798 10819 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:55.063107 11175 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:55.063416 10819 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:55.004313 11121 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:55.028980 11147 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:55.066326 11121 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:55.067662 11147 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:55.062413 11085 consensus_queue.cc:1059] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:18:55.068403 10877 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:55.069060 11085 raft_consensus.cc:3055] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:18:55.069267 10919 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:55.069484 11085 raft_consensus.cc:740] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:55.070521 11085 consensus_queue.cc:260] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:55.071581 11085 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:55.012914 11006 consensus_queue.cc:1059] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:18:54.977905 11043 consensus_queue.cc:260] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:55.075527  7265 raft_consensus.cc:1240] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:55.028743 10963 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:55.077968 10963 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:55.078956  6658 raft_consensus.cc:1240] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:55.082517 10819 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:55.084810 10819 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:55.086578  6657 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:55.049396  7238 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:55.064230 11154 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:55.091140 11127 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:55.092098  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:55.093981  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:55.094416  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:55.095121  7269 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:55.096670  6576 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 2038) took 3700 ms (client timeout 2073 ms). Trace:
W20260709 12:18:55.097162  6576 rpcz_store.cc:269] 0709 12:18:51.396160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:51.396393 (+   233us) service_pool.cc:224] Handling call
0709 12:18:55.096482 (+3700089us) spinlock_profiling.cc:243] Waited 2.72 s on lock 0x7b54000582d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:55.096599 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6259070720}
I20260709 12:18:55.098320  6507 raft_consensus.cc:1275] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:18:55.098834  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:55.099447  6501 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:55.099617  6508 raft_consensus.cc:1275] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:55.100000  6504 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:55.100643 11167 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.094376  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:55.101222 11167 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:55.103235 11167 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:55.104249 10812 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:55.103979 11144 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:55.104624 11049 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:55.104758 10812 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:55.106073 11144 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:55.106642 11144 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.106953 10812 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:55.107045 11144 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:55.107514 10812 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:55.107841  7269 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:18:55.110693  6474 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:55.113786 11006 raft_consensus.cc:3055] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:55.114269 11006 raft_consensus.cc:740] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:55.114951  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:55.115144  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:55.116068  7270 raft_consensus.cc:2393] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 11: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:55.117842  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:55.118094 11144 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:55.118525  6549 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:55.118695  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:55.119464  7270 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
W20260709 12:18:55.107591  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.112550 11112 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.120894  6580 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:55.120771 11112 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:55.121155  7274 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:18:55.112855  6579 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:18:55.103668 11167 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.122498  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:55.124516  6584 raft_consensus.cc:3055] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:18:55.124970  6584 raft_consensus.cc:740] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:55.125842 11112 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:55.123251 10610 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:18:55.125845  6584 consensus_queue.cc:260] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:55.127780  6584 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:18:55.130187  6653 raft_consensus.cc:1240] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:55.115180 11006 consensus_queue.cc:260] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:55.100591  6503 raft_consensus.cc:1240] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:55.132995  6474 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:55.140955  6549 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:18:55.142560  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:55.133484  7273 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:55.111832  6501 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:18:55.146983  6474 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:55.148414  7238 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:55.151643  6474 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:55.108014 10812 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:55.145267 11038 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:55.170173  6474 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:55.164247 11112 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:55.151162 11130 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:18:55.123185 11167 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:55.168295 11006 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:55.153074 11040 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:55.149502 11058 consensus_queue.cc:1059] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:18:55.170807 11144 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:55.164688 11183 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:18:55.150985 11147 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 12:18:55.150730 11049 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:18:55.171834 11038 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:55.194255  6502 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:55.194358  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:55.194712  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.176882 10819 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:55.149941 10615 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:55.208079  6658 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 12.
I20260709 12:18:55.195187 11038 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:55.182533 11040 raft_consensus.cc:697] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:55.209045  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:55.212839  6580 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:18:55.213078  6584 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 12:18:55.210078  6656 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:55.217971 11058 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:55.218609 11167 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:55.207592 10819 raft_consensus.cc:697] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:55.210438  7240 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:55.221808  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:55.222448  6584 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:55.222496  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:55.225301  6580 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:18:55.226202  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:55.227340  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:55.227507 11190 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.224211  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:55.228065 11190 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:55.231112  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:55.232484 11132 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:55.232734  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:55.233168 11132 raft_consensus.cc:697] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 12 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:55.233541  6652 raft_consensus.cc:1275] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:18:55.234252 11132 consensus_queue.cc:237] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:55.236060 11058 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:18:55.239171 11190 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:55.239821  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.240517  6473 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:55.241282 11190 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:55.241683 11190 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:55.242046 11190 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:55.246443  6584 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
W20260709 12:18:55.249934  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.252032  6473 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:55.256757  6473 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:55.259060  6473 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:55.260139  6654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2610) took 1069 ms. Trace:
I20260709 12:18:55.263329  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:55.263482  6654 rpcz_store.cc:276] 0709 12:18:54.190922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:54.194079 (+  3157us) service_pool.cc:224] Handling call
0709 12:18:54.305740 (+111661us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:54.309164 (+  3424us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:55.259846 (+950682us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b54002283d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:55.260058 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":246357248}
I20260709 12:18:55.267696  6473 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:55.254694 11167 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:55.277285 11167 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:55.277812 11167 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:55.273473 11112 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:55.279793 10963 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:55.280768 11112 raft_consensus.cc:697] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:55.281622  6580 raft_consensus.cc:2393] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 8: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:55.281801 11112 consensus_queue.cc:237] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.4, Last appended by leader: 4, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:55.284042 10963 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:55.276544 11147 consensus_queue.cc:1059] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:18:55.266196 11184 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:55.284858 11184 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:55.284956 11147 raft_consensus.cc:3055] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:55.285318 11184 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:55.285377 11147 raft_consensus.cc:740] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:55.286465 11147 consensus_queue.cc:260] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:55.297840 11143 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:55.289599  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:55.302552  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 375029us
I20260709 12:18:55.304236  6475 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:55.305620  6475 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:55.266446 11183 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:18:55.311781  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.309731 10919 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:55.311422 11144 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:18:55.309728 11023 consensus_queue.cc:1059] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 12:18:55.327999 10963 raft_consensus.cc:3055] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:18:55.328402 10963 raft_consensus.cc:740] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:55.329079 11130 raft_consensus.cc:3055] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:55.329556 11130 raft_consensus.cc:740] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:55.329346 10963 consensus_queue.cc:260] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:55.330909 10963 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:55.330518 11130 consensus_queue.cc:260] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:55.331820 11130 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:55.309556 11040 consensus_queue.cc:237] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:55.340866 11184 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
W20260709 12:18:55.341768  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.343132 11184 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:55.345409 11167 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:55.347559 11167 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:55.312760 11147 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:55.301111 11143 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:55.322381 11188 log.cc:927] Time spent T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:18:55.325315  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:55.326864 10615 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:55.308305 11083 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.323313 11175 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:55.308858  6473 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:55.364920  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:55.365769  6503 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 12:18:55.366518  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:55.383067  6503 pending_rounds.cc:85] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 7
I20260709 12:18:55.383358  6503 pending_rounds.cc:107] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:18:55.255512 10819 consensus_queue.cc:237] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:55.377216  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:55.386404  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:55.382555  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:18:55.382643  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 73871.8us
I20260709 12:18:55.389793  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:55.383387  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:55.391197  6504 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:55.383570  6507 raft_consensus.cc:2410] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:18:55.383698  6503 raft_consensus.cc:2889] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:55.377657  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:55.392913  6505 raft_consensus.cc:2410] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:18:55.401341  6509 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:18:55.351903 11190 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:55.403023  6507 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:55.411631  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:55.412519  6501 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:55.416929  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:55.417171  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:55.417826  6584 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:55.418777 11175 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:55.424237  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:55.424734  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:55.425154  6504 raft_consensus.cc:2410] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:18:55.425338  6500 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:18:55.426062  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:55.426581  6507 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:55.420454  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:55.420497  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:55.428194  6508 raft_consensus.cc:2393] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 12: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:18:55.387157 10738 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 12:18:55.436589  6622 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:55.426631  6505 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:55.436983  6584 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:18:55.432811  6500 raft_consensus.cc:1240] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 12. Current term is 13. Ops: []
W20260709 12:18:55.433934  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 433338us
I20260709 12:18:55.386201 11083 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:55.419201 11006 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:55.444276 10963 consensus_queue.cc:1059] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:55.444801 11083 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:55.443323 11049 consensus_queue.cc:1059] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:55.444118 11006 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:55.401144 11167 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:55.447432  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:55.447990  6581 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:55.445879  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: true dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:55.449498  7273 raft_consensus.cc:2393] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 10: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:55.420753  6503 raft_consensus.cc:1240] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 7. Current term is 8. Ops: []
I20260709 12:18:55.434508 11137 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:55.445680  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:55.450471  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:55.454629  6503 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:18:55.436647 10812 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:55.458334  6504 pending_rounds.cc:85] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 6
I20260709 12:18:55.384842 11127 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 12:18:55.460932  6504 pending_rounds.cc:107] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:18:55.445081 10963 consensus_queue.cc:1243] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer f2b06aebe97249a4877d798b8118ece4 log is divergent from this leader: its last log entry 11.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:18:55.461248  6501 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:18:55.462795  6504 raft_consensus.cc:2889] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:55.463291  6508 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 12:18:55.463629  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:55.464273  6501 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:18:55.470816  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.480628  6508 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 14. Current term is 15. Ops: []
I20260709 12:18:55.482406  6505 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:18:55.488385  6508 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:55.447211 11092 consensus_queue.cc:1059] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 12:18:55.445149 11007 consensus_queue.cc:1059] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:18:55.495972  6508 raft_consensus.cc:1240] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:18:55.504773  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:55.505436  6508 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 11.
I20260709 12:18:55.505808 11074 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:55.506556  6508 raft_consensus.cc:1240] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 12. Ops: []
W20260709 12:18:55.523749  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.536657  6582 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:18:55.537599  6582 raft_consensus.cc:606] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:55.538069  6582 raft_consensus.cc:614] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:55.549845 11006 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:55.569739  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:55.575103 11135 log.cc:927] Time spent T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 1.558s	user 0.000s	sys 0.006s
I20260709 12:18:55.576622  6579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 598) took 1061 ms. Trace:
I20260709 12:18:55.578114  7273 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42"
dest_uuid: "20a202c76244485fbc8f1610473db8b9"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:48510
I20260709 12:18:55.578908  7273 raft_consensus.cc:606] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:55.579407  7273 raft_consensus.cc:614] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:18:55.582767  7266 raft_consensus.cc:1275] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:55.583573 10472 raft_consensus.cc:3055] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:55.584022 10472 raft_consensus.cc:740] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:55.584954 10472 consensus_queue.cc:260] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:55.586216 10472 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:55.577095  6579 rpcz_store.cc:276] 0709 12:18:54.515711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:54.534643 (+ 18932us) service_pool.cc:224] Handling call
0709 12:18:55.112435 (+577792us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:55.121819 (+  9384us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:55.576394 (+454575us) spinlock_profiling.cc:243] Waited 567 ms on lock 0x7b54000582d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:55.576554 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1305144576}
I20260709 12:18:55.598997 11030 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:55.599689 11030 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:55.624357  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:55.626565  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:55.661602  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.614300 10990 raft_consensus.cc:3055] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:55.664062 10990 raft_consensus.cc:740] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:55.664983 10990 consensus_queue.cc:260] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.5, Last appended by leader: 5, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:55.666180 10990 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:55.667158  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:18:55.694818 11190 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:55.735054  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.759809  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:55.780388  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:55.785184  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.787312  6580 raft_consensus.cc:1275] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:18:55.555478 11038 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:55.598489 11083 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.181s
I20260709 12:18:55.797501 11040 raft_consensus.cc:3055] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:18:55.797946 11040 raft_consensus.cc:740] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:55.798357 11132 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:18:55.798811 11040 consensus_queue.cc:260] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:55.804479  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:55.805429  6580 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
W20260709 12:18:55.807569  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.809499  6501 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:55.811146  6580 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:18:55.813437 10962 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:18:55.835742  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.807389 11040 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:55.836973  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:18:55.841689  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.842219  6581 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:18:55.845381  6548 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:55.846575  6548 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:55.850847  6548 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:55.861979 11049 raft_consensus.cc:993] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e: : Instructing follower f2b06aebe97249a4877d798b8118ece4 to start an election
I20260709 12:18:55.884915  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:55.892922 11038 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:55.893730 11029 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:55.836304 11132 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 12:18:55.849342 11030 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:55.925268  6473 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:18:55.926359  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 75584.7us
I20260709 12:18:55.927472  6548 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:55.929548  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:55.932731 10986 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:55.934911  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:55.935750  6501 raft_consensus.cc:2393] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 12: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:18:55.935556  6393 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 6f94b1cd286648a9b82d5abf69bb86d5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 20a202c76244485fbc8f1610473db8b9 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:18:55.938197  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:55.940928 11055 raft_consensus.cc:3055] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:18:55.941504 11055 raft_consensus.cc:740] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:55.940174  6501 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:55.942508 11055 consensus_queue.cc:260] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:55.943864 11055 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:55.925529  7238 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:55.955855  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 519316us
W20260709 12:18:55.956284  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 60942.6us
I20260709 12:18:55.957736  6473 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:55.958962  6576 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:55.954669 11049 raft_consensus.cc:1081] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Signalling peer f2b06aebe97249a4877d798b8118ece4 to start an election
I20260709 12:18:55.954882 11023 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:55.948987 10738 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:55.967958  6473 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:55.957368 11147 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:56.014284  6501 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 12:18:56.014735  6501 pending_rounds.cc:85] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 5
I20260709 12:18:56.014978  6501 pending_rounds.cc:107] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 12:18:56.015213  6501 raft_consensus.cc:2889] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:55.959685 11038 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:18:55.961553 11190 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:18:55.982509 11130 consensus_meta.cc:287] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.650s	user 0.014s	sys 0.001s
W20260709 12:18:56.021250  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 93885.2us
I20260709 12:18:56.030107  6548 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:56.022187  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:56.031203  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.031416  6501 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 9. Current term is 10. Ops: []
W20260709 12:18:56.031566  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:56.036461  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:55.974534 10880 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:55.958911 11066 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 12:18:55.999091 10978 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:56.014009 11147 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:56.056908  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 188106us
I20260709 12:18:56.012698 11137 raft_consensus.cc:697] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:56.074827  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:56.079613  6580 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:56.075948  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.037665 11190 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:56.050938 11188 log.cc:927] Time spent T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:18:56.053112 11187 log.cc:927] Time spent T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:18:56.083215  6548 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:56.037974 11083 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.040447 11030 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:56.044322 11016 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:56.044940 11038 raft_consensus.cc:697] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:56.053630 11043 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:56.055030 11145 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:56.095528  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 137339us
W20260709 12:18:56.099805  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 658296us
I20260709 12:18:56.027153 11154 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 12:18:56.077454 11147 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:56.086373 11190 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:56.126456  6655 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:18:56.085654 11123 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:18:56.124583 11137 consensus_queue.cc:237] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:56.129803 11145 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:56.130822 11142 consensus_queue.cc:1048] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:18:56.121682  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:56.115275  6580 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
W20260709 12:18:56.110625  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.100533 11030 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:56.103456 11015 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:56.136198  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 53119.6us
I20260709 12:18:56.155758  6655 raft_consensus.cc:1240] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 11. Current term is 12. Ops: []
W20260709 12:18:56.145627  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 52125.5us
I20260709 12:18:56.097203 11092 consensus_queue.cc:1059] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:18:56.167438  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:56.169801  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:56.183235  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:56.104077 11083 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:56.195611  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:56.196237  6548 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:56.196750  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:56.197343  6653 raft_consensus.cc:1240] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:18:56.212229  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:56.213022  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:56.213637  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:56.184255  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:56.106045 11156 consensus_queue.cc:1048] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 12:18:56.208595 11038 consensus_queue.cc:237] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.5, Last appended by leader: 5, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:56.209890  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2628) took 1400 ms. Trace:
I20260709 12:18:56.218137  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:56.218422  6649 rpcz_store.cc:276] 0709 12:18:54.809428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:54.810267 (+   839us) service_pool.cc:224] Handling call
0709 12:18:54.844853 (+ 34586us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:54.886951 (+ 42098us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:18:54.932074 (+ 45123us) log.cc:844] Serialized 14 byte log entry
0709 12:18:55.571253 (+639179us) spinlock_profiling.cc:243] Waited 5.85 ms on lock 0x7b48000cda58. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287adb2aa1 00007f287ada9b37 00007f287ada99ca 00007f287aea126f 00007f287aea0f37 00007f287aeb555c 00007f287aeb5508 00007f287aeb5488 00007f287aeb5443 00007f287aeb46ec 00007f2879ddea96
0709 12:18:55.946928 (+375675us) spinlock_profiling.cc:243] Waited 2.3 ms on lock 0x7b180021e560. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d16c0a 00007f287c456219 00007f2872d9ea40 00007f2872d994e4 00007f2872d99455 00007f2875e81060 00007f2875e93716 00007f2875ea4287 00007f2875ea4208
0709 12:18:55.959955 (+ 13027us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:55.960688 (+   733us) log.cc:844] Serialized 52 byte log entry
0709 12:18:55.960868 (+   180us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:56.145001 (+184133us) log.cc:844] Serialized 14 byte log entry
0709 12:18:56.158227 (+ 13226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.158332 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:56.209663 (+ 51331us) raft_consensus.cc:1685] finished
0709 12:18:56.209685 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:56.209832 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":901042,"prepare.queue_time_us":48355,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":1380605,"raft.run_cpu_time_us":2764,"raft.run_wall_time_us":165331,"spinlock_wait_cycles":18760320,"thread_start_us":381917,"threads_started":2,"wal-append.queue_time_us":241835}
I20260709 12:18:56.220834  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:56.221525  6649 raft_consensus.cc:2393] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 11: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:56.157438 10738 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:56.154691 11030 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:56.215308  6650 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:56.215994  6654 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:18:56.181622 10880 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:56.178181 11015 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:56.181280 11141 consensus_queue.cc:1059] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
W20260709 12:18:56.194854 11206 log.cc:927] Time spent T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260709 12:18:56.197961 11154 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:56.222906  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:56.203625 11134 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:56.238461  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:56.246318  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:56.242441  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:56.243911  7273 raft_consensus.cc:1275] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:56.230759  6658 raft_consensus.cc:2393] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 14: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:56.237015  6500 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:56.256124  6583 raft_consensus.cc:2410] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:18:56.257349  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:56.242830  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.200228 10877 consensus_queue.cc:1059] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 12:18:56.238070 11049 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.179278 11146 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:56.230654  7268 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:18:56.239488 10996 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:56.256803 11040 consensus_queue.cc:1059] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
W20260709 12:18:56.221715  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.298197  6473 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:56.233530 11145 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:56.272076  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.249867 10880 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:56.280633 11015 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:56.237342 11050 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:18:56.257686 11142 raft_consensus.cc:3055] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:56.284842 11146 raft_consensus.cc:3055] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:56.289491 10738 raft_consensus.cc:3055] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:56.290030  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:56.303966  6654 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:18:56.296191  6650 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:18:56.227450 10615 consensus_queue.cc:1048] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:18:56.310209  6500 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:56.217485 11016 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 12:18:56.256247 11167 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.240056 11160 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 12:18:56.259933 10999 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.324736  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:56.327104  6651 raft_consensus.cc:2410] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:18:56.239025 11221 consensus_queue.cc:1059] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 12:18:56.335008  6654 raft_consensus.cc:2393] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 11: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:56.358786  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:56.301239 11141 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:56.360965  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:18:56.360965  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.363817  6501 raft_consensus.cc:1240] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:56.373198  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:56.375619  6501 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 12:18:56.377488  6501 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:18:56.313966 11142 raft_consensus.cc:740] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:56.318535 11074 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.318965 10738 raft_consensus.cc:740] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:56.382846  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:56.386535  6654 raft_consensus.cc:2410] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:18:56.326287 11123 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:56.319506 11146 raft_consensus.cc:740] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:56.388386  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:56.389577  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:56.390390  6650 raft_consensus.cc:2393] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 10: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:56.390825  6501 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:56.391111  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:56.300555 11049 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:56.379076 11216 log.cc:927] Time spent T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:18:56.410854  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:56.412386  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:56.413221  6650 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:56.414327  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:56.415323  6650 raft_consensus.cc:2393] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 8: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:18:56.406605  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.406880  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:56.401425  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:56.399502  6506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2304) took 1033 ms. Trace:
W20260709 12:18:56.384305 11206 log.cc:927] Time spent T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:18:56.378175 10978 raft_consensus.cc:697] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:56.422091  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:56.429086  6650 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:18:56.385090 11220 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:56.433997  6502 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2312) took 1064 ms. Trace:
I20260709 12:18:56.342375 11167 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:56.405198 11156 raft_consensus.cc:3055] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:56.358847 10999 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:56.444896  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2328) took 1068 ms. Trace:
I20260709 12:18:56.380757 11141 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:56.419384 11145 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:56.399873 11074 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:56.434722  6502 rpcz_store.cc:276] 0709 12:18:55.369246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.418041 (+ 48795us) service_pool.cc:224] Handling call
0709 12:18:55.418395 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:55.418764 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:55.592432 (+173668us) log.cc:844] Serialized 14 byte log entry
0709 12:18:56.292301 (+699869us) log.cc:844] Serialized 14 byte log entry
0709 12:18:56.292505 (+   204us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:56.309982 (+ 17477us) log.cc:844] Serialized 26 byte log entry
0709 12:18:56.310175 (+   193us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:56.318443 (+  8268us) log.cc:844] Serialized 14 byte log entry
0709 12:18:56.318707 (+   264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.318792 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:56.433774 (+114982us) raft_consensus.cc:1685] finished
0709 12:18:56.433799 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:56.433933 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":762,"prepare.queue_time_us":404784,"prepare.run_cpu_time_us":114,"prepare.run_wall_time_us":120,"raft.queue_time_us":138129,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":37595,"thread_start_us":864409,"threads_started":3,"wal-append.queue_time_us":546248}
I20260709 12:18:56.454787  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2321) took 1080 ms. Trace:
I20260709 12:18:56.445392  6508 rpcz_store.cc:276] 0709 12:18:55.376258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.507386 (+131128us) service_pool.cc:224] Handling call
0709 12:18:55.507699 (+   313us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:55.741465 (+233766us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:55.741528 (+    63us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:55.763791 (+ 22263us) log.cc:844] Serialized 52 byte log entry
0709 12:18:55.999435 (+235644us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:56.071230 (+ 71795us) log.cc:844] Serialized 14 byte log entry
0709 12:18:56.235161 (+163931us) spinlock_profiling.cc:243] Waited 8.53 ms on lock 0x7b1800177d60. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:18:56.436563 (+201402us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.436641 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:56.436685 (+    44us) raft_consensus.cc:1685] finished
0709 12:18:56.444660 (+  7975us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:56.444840 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":260702,"prepare.queue_time_us":121601,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":364852,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":23052,"spinlock_wait_cycles":71920768,"thread_start_us":561325,"threads_started":2,"wal-append.queue_time_us":338209}
I20260709 12:18:56.459932  6503 rpcz_store.cc:276] 0709 12:18:55.374131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.455711 (+ 81580us) service_pool.cc:224] Handling call
0709 12:18:55.456150 (+   439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:55.456954 (+   804us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:55.457007 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:55.466511 (+  9504us) log.cc:844] Serialized 26 byte log entry
0709 12:18:55.821624 (+355113us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:55.840616 (+ 18992us) log.cc:844] Serialized 14 byte log entry
0709 12:18:55.840806 (+   190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:55.840872 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:56.454577 (+613705us) raft_consensus.cc:1685] finished
0709 12:18:56.454599 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:56.454727 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2046,"prepare.queue_time_us":15808,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":942,"raft.run_cpu_time_us":366,"raft.run_wall_time_us":72944,"thread_start_us":320155,"threads_started":1,"wal-append.queue_time_us":484657}
I20260709 12:18:56.462265  6500 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:18:56.423070 11223 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:56.471830  6654 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:56.440469 11156 raft_consensus.cc:740] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:56.405893 11123 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:56.488646  6502 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:18:56.437106  6506 rpcz_store.cc:276] 0709 12:18:55.366450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.366567 (+   117us) service_pool.cc:224] Handling call
0709 12:18:55.387393 (+ 20826us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:55.387696 (+   303us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:55.387728 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:55.401066 (+ 13338us) log.cc:844] Serialized 26 byte log entry
0709 12:18:55.778046 (+376980us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:56.097936 (+319890us) log.cc:844] Serialized 14 byte log entry
0709 12:18:56.399122 (+301186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.399228 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:56.399279 (+    51us) raft_consensus.cc:1685] finished
0709 12:18:56.399294 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:56.399439 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1379,"prepare.queue_time_us":20524,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":159915,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":11356,"thread_start_us":929936,"threads_started":3,"wal-append.queue_time_us":355918}
W20260709 12:18:56.492831  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 296705us
I20260709 12:18:56.493721  6548 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:56.473816 11137 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:56.494906 11016 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:56.495412 11016 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:56.495975 11016 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:56.495061 11137 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:56.497816 11224 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.498469 11224 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:56.479995  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 240319us
I20260709 12:18:56.431869 10738 consensus_queue.cc:260] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:56.500257 11225 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.500303 11038 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.464381  6650 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:18:56.501022 11038 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:56.495533 11202 consensus_queue.cc:1059] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:18:56.493727  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:56.503183 11038 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:56.492627 11148 consensus_queue.cc:1048] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 12:18:56.500998 10738 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:18:56.495146  6473 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.725s (SENT) [suppressed 28 similar messages]
I20260709 12:18:56.504992 11016 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:56.533411  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.513733  6550 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:56.553551  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:56.554093  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:56.557411  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.447546 11190 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:56.470882 11221 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:56.559890  6475 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:56.475893 10880 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:56.561950  6550 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:56.439105 11220 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:56.561576 11015 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:18:56.569463 11015 raft_consensus.cc:697] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:56.569797 11156 consensus_queue.cc:260] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:56.570575 11015 consensus_queue.cc:237] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:56.571303 11156 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:56.484174 11012 consensus_queue.cc:1048] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.088s
I20260709 12:18:56.573503 11012 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:56.574007 11012 raft_consensus.cc:697] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:18:56.574576 11055 consensus_meta.cc:287] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.616s	user 0.010s	sys 0.000s
I20260709 12:18:56.574829 11012 consensus_queue.cc:237] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:56.575337 11218 log.cc:927] Time spent T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.324s	user 0.000s	sys 0.001s
I20260709 12:18:56.447232 11142 consensus_queue.cc:260] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:56.578002 11142 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:56.578398  6505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2325) took 1203 ms. Trace:
I20260709 12:18:56.579115 11220 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:56.492571 11160 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.087s
I20260709 12:18:56.486258 11083 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:56.552014 11162 consensus_queue.cc:1059] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 12:18:56.581734 11162 raft_consensus.cc:3055] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:56.582232 11190 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:56.471257 11146 consensus_queue.cc:260] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:56.585453 11146 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:56.569371 11221 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:56.587647  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.588385 11221 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:56.589700 11123 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:56.575407 11215 log.cc:927] Time spent T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 12:18:56.592260  6504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2314) took 1220 ms. Trace:
W20260709 12:18:56.592741 11212 log.cc:927] Time spent T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.288s	user 0.000s	sys 0.001s
I20260709 12:18:56.594475  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2330) took 1215 ms. Trace:
I20260709 12:18:56.592871  6504 rpcz_store.cc:276] 0709 12:18:55.371368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.437750 (+ 66382us) service_pool.cc:224] Handling call
0709 12:18:55.458134 (+ 20384us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:55.492407 (+ 34273us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:55.492437 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:55.817434 (+324997us) spinlock_profiling.cc:243] Waited 1.29 ms on lock 0x7b2c00000c88. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:18:55.817978 (+   544us) log.cc:844] Serialized 52 byte log entry
0709 12:18:56.137334 (+319356us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:56.183741 (+ 46407us) log.cc:844] Serialized 14 byte log entry
0709 12:18:56.195053 (+ 11312us) log.cc:844] Serialized 14 byte log entry
0709 12:18:56.195228 (+   175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.195298 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:56.592008 (+396710us) raft_consensus.cc:1685] finished
0709 12:18:56.592034 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:56.592191 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47781,"prepare.queue_time_us":118191,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":21463,"raft.queue_time_us":512766,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":18648,"spinlock_wait_cycles":13433856,"thread_start_us":318937,"threads_started":1,"wal-append.queue_time_us":515124}
W20260709 12:18:56.595602 10877 consensus_queue.cc:1175] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e7554c39679a41bbbca4659b72ab5c65" responder_term: 13 status { last_received { term: 11 index: 6 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 12. Current term is 13. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:18:56.595917 10978 consensus_queue.cc:237] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:56.597572 11141 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:56.578761  6505 rpcz_store.cc:276] 0709 12:18:55.375331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.483739 (+108408us) service_pool.cc:224] Handling call
0709 12:18:55.484086 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:55.484898 (+   812us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:18:55.591404 (+106506us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b18004b2200. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d99455 00007f28798813c7 00007f2875e939a7 00007f2875ea30bf 00007f2875ea3038
0709 12:18:55.822106 (+230702us) log.cc:844] Serialized 14 byte log entry
0709 12:18:56.059419 (+237313us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:56.086888 (+ 27469us) log.cc:844] Serialized 26 byte log entry
0709 12:18:56.087219 (+   331us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:56.123916 (+ 36697us) log.cc:844] Serialized 14 byte log entry
0709 12:18:56.440203 (+316287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.440298 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:56.578205 (+137907us) raft_consensus.cc:1685] finished
0709 12:18:56.578224 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:56.578345 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52850,"prepare.queue_time_us":356731,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":54,"raft.queue_time_us":402960,"raft.run_cpu_time_us":393,"raft.run_wall_time_us":31281,"spinlock_wait_cycles":2553088,"thread_start_us":754005,"threads_started":3,"wal-append.queue_time_us":342402}
I20260709 12:18:56.462247 11145 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:56.601032 11137 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:56.601629 11145 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:56.568043 11092 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:18:56.602231 11127 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:56.602792 11127 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:56.603121 11147 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:56.603956 11167 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:56.604241  7265 raft_consensus.cc:1240] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:18:56.478065 11223 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:56.605455 11127 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:56.606326 11223 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:56.579775 11049 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:56.582154 11162 raft_consensus.cc:740] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:56.562208 11224 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:56.594842  6509 rpcz_store.cc:276] 0709 12:18:55.379280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.647144 (+267864us) service_pool.cc:224] Handling call
0709 12:18:55.647567 (+   423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:55.715713 (+ 68146us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:55.715765 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:55.775967 (+ 60202us) log.cc:844] Serialized 26 byte log entry
0709 12:18:56.050467 (+274500us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:56.119036 (+ 68569us) log.cc:844] Serialized 14 byte log entry
0709 12:18:56.444881 (+325845us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.444960 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:56.594271 (+149311us) raft_consensus.cc:1685] finished
0709 12:18:56.594292 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:56.594415 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":111736,"prepare.queue_time_us":72959,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":325394,"raft.run_cpu_time_us":340,"raft.run_wall_time_us":22596,"thread_start_us":583602,"threads_started":2,"wal-append.queue_time_us":429715}
I20260709 12:18:56.606992  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2318) took 1233 ms. Trace:
W20260709 12:18:56.608793  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 49015.2us
I20260709 12:18:56.608804  6507 rpcz_store.cc:276] 0709 12:18:55.373181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.437405 (+ 64224us) service_pool.cc:224] Handling call
0709 12:18:55.437753 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:55.453655 (+ 15902us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:55.453706 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:55.459513 (+  5807us) log.cc:844] Serialized 26 byte log entry
0709 12:18:55.778422 (+318909us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:56.133897 (+355475us) log.cc:844] Serialized 14 byte log entry
0709 12:18:56.434228 (+300331us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.606637 (+172409us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b54001e87d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:56.606686 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:56.606731 (+    45us) raft_consensus.cc:1685] finished
0709 12:18:56.606810 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:56.606937 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12996,"prepare.queue_time_us":20316,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":23,"raft.queue_time_us":288879,"raft.run_cpu_time_us":383,"raft.run_wall_time_us":15596,"spinlock_wait_cycles":35374848,"thread_start_us":920636,"threads_started":3,"wal-append.queue_time_us":319468}
W20260709 12:18:56.607838  6583 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 2104) took 1647 ms (client timeout 1725 ms). Trace:
I20260709 12:18:56.611231  6475 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:56.612295 11234 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:18:56.611313  6583 rpcz_store.cc:269] 0709 12:18:54.960283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:54.964298 (+  4015us) service_pool.cc:224] Handling call
0709 12:18:56.607637 (+1643339us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54000573d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:56.607781 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2938224384}
I20260709 12:18:56.615283  6583 raft_consensus.cc:1275] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:56.615861  6576 raft_consensus.cc:1275] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 10 index: 4. Preceding OpId from leader: term: 12 index: 5. (index mismatch)
I20260709 12:18:56.615304 11016 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:56.616914 11016 consensus_queue.cc:1059] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:18:56.616801 11162 consensus_queue.cc:260] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:56.617882 11016 raft_consensus.cc:3055] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:56.618028 11162 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:56.618243 11016 raft_consensus.cc:740] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:56.619066 11016 consensus_queue.cc:260] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:56.621011 11023 raft_consensus.cc:3055] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:56.621465 11023 raft_consensus.cc:740] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:56.622364 11023 consensus_queue.cc:260] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:56.620131 11016 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:56.500975 11225 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:56.628765  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:56.629244  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:56.629532  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:56.629889  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:56.632463  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 52673us
I20260709 12:18:56.639339  6548 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:56.642055  6548 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:56.639328  6475 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:56.646629  6475 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:56.648653  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.556640 11226 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:18:56.699780  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 57817.6us
I20260709 12:18:56.702553  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:56.703351  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:56.703902  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:56.707312  7266 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:56.707588  6548 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:56.708806  7266 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:56.710263 11242 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:56.713835  6548 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:56.719094  6549 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:18:56.721320  6583 raft_consensus.cc:1275] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:56.728974  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:56.712251 10990 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:56.732528 10990 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:56.735237 10990 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:56.698588 11229 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.737166 11229 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:56.739562 11229 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:56.742221  6657 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10. Ops: []
I20260709 12:18:56.745649  6475 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:56.746757  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:56.705312 11224 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:18:56.748970  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:56.705554 11092 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:56.706115 11167 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:56.752110  6509 raft_consensus.cc:2393] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 11: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:56.712124 11049 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:18:56.836858  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 327807us
I20260709 12:18:56.822042  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:18:56.825359  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 79826.9us
I20260709 12:18:56.723716 11226 raft_consensus.cc:697] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:56.829496  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:56.850131  6505 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:56.701967 11237 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:18:56.821496 11232 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.837132  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:56.858834  6508 raft_consensus.cc:2393] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 14: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:18:56.755828  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:56.838325  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 124574us
I20260709 12:18:56.840298  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:18:56.840939  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 121932us
I20260709 12:18:56.842625  6506 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:56.843891  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:56.861601  6509 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:56.720800 11242 raft_consensus.cc:697] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:56.862144  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:56.844985  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 349853us
I20260709 12:18:56.862507  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:56.863013  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:56.841400  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:56.715297 11023 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:56.846508  6475 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:56.849373  6503 raft_consensus.cc:2393] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 11: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:18:56.832157  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.856114  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: true dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:18:56.847958 11043 consensus_meta.cc:287] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.725s	user 0.016s	sys 0.001s
I20260709 12:18:56.836302 11147 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:56.845427 11183 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:18:56.867436  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.874732  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:56.865118  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:56.875519  6656 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:56.839867 11190 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:56.880565  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:56.880854  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:56.881394  6582 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:56.882261  6508 raft_consensus.cc:1240] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:56.886127  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:56.847321 10996 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:56.824156 11184 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
W20260709 12:18:56.890095  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 396440us
W20260709 12:18:56.898438  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 52020.2us
I20260709 12:18:56.850334 11238 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:56.855486 11049 raft_consensus.cc:697] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:56.902581  6475 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:56.899292  6505 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:18:56.904888  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:56.912142  6575 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:56.917995  7266 raft_consensus.cc:1275] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:18:56.907678  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:56.919902  6509 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:18:56.905000  6582 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:18:56.909180  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:56.921200  7271 raft_consensus.cc:2393] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 15: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:56.903836 11237 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:18:56.909981  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:56.925998  6583 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:56.927932  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:56.908100 11167 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 12:18:56.929839  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:56.901791 11225 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:56.906534 11238 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:56.930680  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 540) took 1091 ms. Trace:
I20260709 12:18:56.936671  6475 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:56.897467 10996 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:18:56.909353 11030 consensus_meta.cc:287] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.659s	user 0.015s	sys 0.000s
I20260709 12:18:56.939200  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:56.963904  6583 raft_consensus.cc:2410] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:18:56.900481 10990 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.894613 11228 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:56.954329  6501 rpcz_store.cc:276] 0709 12:18:55.839347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.398403 (+559056us) service_pool.cc:224] Handling call
0709 12:18:56.404178 (+  5775us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:56.405294 (+  1116us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:56.405339 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:56.406895 (+  1556us) log.cc:844] Serialized 26 byte log entry
0709 12:18:56.635419 (+228524us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:56.722789 (+ 87370us) log.cc:844] Serialized 14 byte log entry
0709 12:18:56.727153 (+  4364us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.727222 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:56.930486 (+203264us) raft_consensus.cc:1685] finished
0709 12:18:56.930506 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:56.930624 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":79573,"prepare.queue_time_us":87401,"raft.queue_time_us":103075,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":11429,"spinlock_wait_cycles":37690240,"thread_start_us":186449,"threads_started":1,"wal-append.queue_time_us":470301}
W20260709 12:18:56.924834 11239 log.cc:927] Time spent T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:18:56.942767  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:56.873888 11092 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 12:18:56.952411 11242 consensus_queue.cc:237] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:56.941579  6656 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:56.958129  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:57.006474  6582 raft_consensus.cc:2410] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:18:56.898876 11040 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:18:56.970361  6475 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:57.011267  6475 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:56.975178  6584 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 6.
I20260709 12:18:56.976411 11220 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:56.972693 11030 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:56.913535 10819 consensus_queue.cc:1048] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 12:18:56.962055 11238 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:56.977347 10990 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:56.966068 10996 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:56.978431 11226 consensus_queue.cc:237] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:56.942059 11237 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:18:56.979621  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:56.980290  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 708) took 1742 ms. Trace:
I20260709 12:18:56.980881 11127 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:18:56.940462 11043 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:56.988099  6650 raft_consensus.cc:1275] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:56.992843  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:57.001138  6473 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:56.867203 11232 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:56.902909 11190 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:57.004808  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 62125.4us
I20260709 12:18:57.014945  6475 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:56.932474 11229 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 12:18:57.003484 11167 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:57.017656 11220 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:57.064522  6653 raft_consensus.cc:2393] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 11: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:18:57.048209  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 47137.2us
I20260709 12:18:57.050333  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:57.053596  6583 raft_consensus.cc:1240] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:56.971935 11085 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 12:18:57.017391 11183 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:18:57.070070  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:57.056619  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 158754us
I20260709 12:18:57.059616  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:57.054416  6652 rpcz_store.cc:276] 0709 12:18:55.238376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.238533 (+   157us) service_pool.cc:224] Handling call
0709 12:18:55.238847 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:55.947507 (+708660us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:56.093600 (+146093us) log.cc:844] Serialized 14 byte log entry
0709 12:18:56.852698 (+759098us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:56.853161 (+   463us) log.cc:844] Serialized 26 byte log entry
0709 12:18:56.853332 (+   171us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:56.853380 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.853465 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:56.980089 (+126624us) raft_consensus.cc:1685] finished
0709 12:18:56.980109 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:56.980232 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":832654,"prepare.queue_time_us":252914,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":825097,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":24902,"thread_start_us":864203,"threads_started":3,"wal-append.queue_time_us":489794}
I20260709 12:18:57.065959  6473 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:57.082893  6655 raft_consensus.cc:2393] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 12: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:57.084059  6473 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:57.086024  6656 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:57.019130 11143 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:57.056401 11184 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:57.055938 11121 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.004012 11228 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:57.062882 11246 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:57.014940 11233 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
I20260709 12:18:57.100430  6475 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:57.315062  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 214727us
I20260709 12:18:57.315527 11183 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:57.082273 11015 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.086922 11237 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:18:57.056613 11043 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:57.312134  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 405400us
I20260709 12:18:57.122394  6550 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:57.238901  6501 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:57.062439 11146 consensus_queue.cc:1048] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.190s
I20260709 12:18:57.250782 11184 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:57.096084  6655 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:58980
I20260709 12:18:57.018786 11254 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.293s
I20260709 12:18:57.379243  6655 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:57.379753  6655 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:57.245530 11234 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:57.086490 11230 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:57.247750 11167 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:57.313745 11246 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:57.401904 11184 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:57.314291 11121 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:57.050211 11049 consensus_queue.cc:237] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:57.332221 11016 consensus_meta.cc:287] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.701s	user 0.007s	sys 0.006s
W20260709 12:18:57.440052 11142 consensus_meta.cc:287] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.862s	user 0.010s	sys 0.003s
I20260709 12:18:57.472739 11134 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:18:57.453135  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 74378.3us
W20260709 12:18:57.492489 11156 consensus_meta.cc:287] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.921s	user 0.015s	sys 0.000s
I20260709 12:18:57.439420 11183 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:57.461274 11127 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.480175 11246 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:57.495417 11230 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:57.344466 11015 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:57.442274 11092 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:57.453325 11162 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:57.496960 11264 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:57.544587 11263 log.cc:927] Time spent T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
W20260709 12:18:57.536092 11257 log.cc:927] Time spent T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.211s	user 0.000s	sys 0.001s
I20260709 12:18:57.486440 11167 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:57.544587 11256 log.cc:927] Time spent T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
I20260709 12:18:57.536413 11030 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:57.538841 11162 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:57.505668 11134 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:57.526584 11183 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:57.531250 11127 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:57.550314 11190 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:57.557843 11258 log.cc:927] Time spent T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.103s	user 0.001s	sys 0.001s
I20260709 12:18:57.556226 11220 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:57.540795 11264 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:57.587504 11134 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:57.588284 11167 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:57.559062 11232 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:57.588996 11267 log.cc:927] Time spent T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.093s	user 0.003s	sys 0.000s
I20260709 12:18:57.589783 11162 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:57.593302 11230 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:57.593595 11074 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:57.583550 11043 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:57.599920 11183 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:57.621230 11227 log.cc:927] Time spent T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 1.188s	user 0.011s	sys 0.012s
W20260709 12:18:57.621778 11275 log.cc:927] Time spent T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 12:18:57.629235 11264 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:57.629968 11246 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:57.632814 11246 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:57.474435  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:57.421970  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:57.468879  6655 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:57.688249  6651 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:18:57.688786 11237 raft_consensus.cc:697] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 13 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:57.488466  6653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2648) took 1256 ms. Trace:
I20260709 12:18:57.692066 11237 consensus_queue.cc:237] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:57.510787  7238 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
W20260709 12:18:57.695003  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 184200us
I20260709 12:18:57.698655 10978 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.689458  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 620) took 1595 ms. Trace:
W20260709 12:18:57.439772  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 317453us
W20260709 12:18:57.440327  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 100621us
I20260709 12:18:57.732894 11233 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.734054 11233 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:57.455030  6501 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 12.
I20260709 12:18:57.709559 10978 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:57.709622  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:57.737766 11127 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:57.739127 10978 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:57.597894 11121 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:57.622310 11238 consensus_meta.cc:287] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.567s	user 0.008s	sys 0.002s
I20260709 12:18:57.725239  6506 raft_consensus.cc:1240] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:57.729012 11228 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:57.731385 11123 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:57.752988 11123 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:57.687412  6407 catalog_manager.cc:5697] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 8 to 10, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 10 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:18:57.760655 11162 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:57.699883  6578 rpcz_store.cc:276] 0709 12:18:56.093248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.093352 (+   104us) service_pool.cc:224] Handling call
0709 12:18:56.108463 (+ 15111us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:56.262135 (+153672us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:18:56.262436 (+   301us) log.cc:844] Serialized 14 byte log entry
0709 12:18:56.591767 (+329331us) spinlock_profiling.cc:243] Waited 39.2 ms on lock 0x7b1800463360. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a2923a 00007f2875e93a2a
0709 12:18:56.906803 (+315036us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:56.907251 (+   448us) log.cc:844] Serialized 26 byte log entry
0709 12:18:56.907369 (+   118us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:56.907401 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:56.989625 (+ 82224us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b540034b7d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:56.989674 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:57.596398 (+606724us) spinlock_profiling.cc:243] Waited 19.6 ms on lock 0x7b1800463360. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:18:57.688465 (+ 92067us) raft_consensus.cc:1685] finished
0709 12:18:57.688585 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:57.689053 (+   468us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18543,"prepare.queue_time_us":16851,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"raft.queue_time_us":18986,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":20699,"spinlock_wait_cycles":486538240,"thread_start_us":296681,"threads_started":1,"wal-append.queue_time_us":802996}
I20260709 12:18:57.762199 11162 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:57.763476  6655 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:57.764164  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:58980 (request call id 2655) took 1530 ms. Trace:
I20260709 12:18:57.764549  6655 rpcz_store.cc:276] 0709 12:18:56.233555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:57.095925 (+862370us) service_pool.cc:224] Handling call
0709 12:18:57.764010 (+668085us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:57.767472  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 711) took 1526 ms. Trace:
I20260709 12:18:57.772768 11277 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:57.774181 11277 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:56.041209  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:57.776798 11246 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:57.777853 11246 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:57.780762 11130 raft_consensus.cc:2804] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:18:57.781272 11130 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:57.781947 11130 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:57.782687 10996 consensus_meta.cc:287] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.731s	user 0.004s	sys 0.023s
I20260709 12:18:57.690301  6653 rpcz_store.cc:276] 0709 12:18:56.231764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:57.075045 (+843281us) service_pool.cc:224] Handling call
0709 12:18:57.075346 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:57.075574 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:18:57.245113 (+169539us) log.cc:844] Serialized 14 byte log entry
0709 12:18:57.487936 (+242823us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:57.488027 (+    91us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:57.488100 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:57.488200 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:57.488384 (+   184us) inbound_call.cc:177] Queueing success response
0709 12:18:57.504646 (+ 16262us) spinlock_profiling.cc:243] Waited 3.09 ms on lock 0x7b1800422aa0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:18:57.621247 (+116601us) spinlock_profiling.cc:243] Waited 2.78 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
Metrics: {"mutex_wait_us":18320,"prepare.queue_time_us":169327,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":32075,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":6920,"spinlock_wait_cycles":13484672,"thread_start_us":211457,"threads_started":1,"wal-append.queue_time_us":246946}
I20260709 12:18:57.785821  6650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2072) took 1577 ms. Trace:
I20260709 12:18:57.785977  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:57.786725  6653 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:18:57.786393  6650 rpcz_store.cc:276] 0709 12:18:56.207969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.552437 (+344468us) service_pool.cc:224] Handling call
0709 12:18:56.566920 (+ 14483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:57.047053 (+480133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:57.785252 (+738199us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b540019d7d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:57.785609 (+   357us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":967585536}
W20260709 12:18:57.745982 11023 consensus_meta.cc:287] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.876s	user 0.009s	sys 0.003s
I20260709 12:18:57.748070 11015 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:57.750388  7240 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.007s	sys 0.000s
I20260709 12:18:57.788156  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:57.767863  6655 rpcz_store.cc:276] 0709 12:18:56.241060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:57.765978 (+1524918us) service_pool.cc:224] Handling call
0709 12:18:57.766558 (+   580us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:57.767064 (+   506us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:57.767100 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:57.767160 (+    60us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:57.767199 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:57.767256 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:57.767367 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:57.783457 10996 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:57.788115  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:57.799101  7240 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.007s	sys 0.000s
I20260709 12:18:57.740082  6622 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:57.807346  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 2192) took 1059 ms. Trace:
I20260709 12:18:57.810793  6507 raft_consensus.cc:2393] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 10: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:57.811470  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 2194) took 1062 ms. Trace:
I20260709 12:18:57.812636  6653 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:18:57.812902 11246 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:57.815587 11220 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.816515 11220 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:57.817673  6651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2073) took 1609 ms. Trace:
I20260709 12:18:57.818219  6651 rpcz_store.cc:276] 0709 12:18:56.208183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.581921 (+373738us) service_pool.cc:224] Handling call
0709 12:18:56.582220 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:57.689258 (+1107038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:57.817229 (+127971us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540044acd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:57.817592 (+   363us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2540795776}
I20260709 12:18:57.819260 11220 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:57.822049 11121 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:57.822965 11121 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:57.826702 11085 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.829131 11162 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:57.830926 11121 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:57.833328  6654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 2069) took 1625 ms. Trace:
I20260709 12:18:57.833827  6654 rpcz_store.cc:276] 0709 12:18:56.207269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.401093 (+193824us) service_pool.cc:224] Handling call
0709 12:18:57.832911 (+1431818us) spinlock_profiling.cc:243] Waited 47.5 ms on lock 0x7b540044b1d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:57.833112 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":109208448}
I20260709 12:18:57.834467 11277 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:57.835824  6500 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 533) took 2047 ms. Trace:
I20260709 12:18:57.836462  6657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2047) took 1640 ms. Trace:
I20260709 12:18:57.836987  6657 rpcz_store.cc:276] 0709 12:18:56.195524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.196056 (+   532us) service_pool.cc:224] Handling call
0709 12:18:56.196373 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:56.742833 (+546460us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:57.835796 (+1092963us) spinlock_profiling.cc:243] Waited 546 ms on lock 0x7b54003e9ed4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:57.836215 (+   419us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1254712704}
W20260709 12:18:57.839138  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 99442.6us
I20260709 12:18:57.840832 11123 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:57.841740 11162 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:18:57.846144  6577 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.000s	sys 0.017s
W20260709 12:18:57.846632  6577 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.000s	sys 0.017s
I20260709 12:18:57.847656 11233 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:57.807775 11228 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.849077 11228 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:57.849782  7265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38546 (request call id 524) took 1613 ms. Trace:
I20260709 12:18:57.836272  6500 rpcz_store.cc:276] 0709 12:18:55.788327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.322521 (+534194us) service_pool.cc:224] Handling call
0709 12:18:56.371027 (+ 48506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:56.462777 (+ 91750us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:57.789125 (+1326348us) spinlock_profiling.cc:243] Waited 85.9 ms on lock 0x7b5400396cd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:57.835513 (+ 46388us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":197483904}
I20260709 12:18:57.857609 11238 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:57.868103  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:57.852062 11228 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:57.811853  6507 rpcz_store.cc:276] 0709 12:18:56.749219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.855759 (+106540us) service_pool.cc:224] Handling call
0709 12:18:57.809976 (+954217us) spinlock_profiling.cc:243] Waited 39.3 ms on lock 0x7b540021bc20. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e00dc0 00007f2875e8ce18 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6
0709 12:18:57.811382 (+  1406us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":90321664}
I20260709 12:18:57.827675 11085 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:57.872802  7269 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:57.759452  6407 catalog_manager.cc:5697] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:18:57.880223 11132 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.881232 11132 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:57.870561  7265 rpcz_store.cc:276] 0709 12:18:56.235817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.235975 (+   158us) service_pool.cc:224] Handling call
0709 12:18:56.236288 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:56.604661 (+368373us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:57.849101 (+1244440us) spinlock_profiling.cc:243] Waited 368 ms on lock 0x7b54000a1ed4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:57.849527 (+   426us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":845674880}
I20260709 12:18:57.807850  6503 rpcz_store.cc:276] 0709 12:18:56.748532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.748690 (+   158us) service_pool.cc:224] Handling call
0709 12:18:57.806755 (+1058065us) spinlock_profiling.cc:243] Waited 90.8 ms on lock 0x7b54002c42d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:57.807099 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208892160}
I20260709 12:18:57.884749  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:57.869160 11130 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:57.905705  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:57.906342 11085 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:57.907433  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:57.906909  6653 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:18:57.915740 11130 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:57.911859  6650 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:18:57.926705  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:57.927011 11130 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:57.928031  6653 raft_consensus.cc:2393] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 12: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:57.932771  6655 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:18:57.934255  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:57.935217  7269 raft_consensus.cc:2393] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 15: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:18:57.937769 10880 raft_consensus.cc:3055] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:18:57.938380 10880 raft_consensus.cc:740] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:57.940918 10963 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:57.941875 10963 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:57.943491 11132 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:57.945056 10990 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:57.917382  6622 leader_election.cc:336] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.507s (SENT)
I20260709 12:18:57.945670 10996 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:57.929562  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2652) took 1696 ms. Trace:
I20260709 12:18:57.953876 11184 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:57.954586 11238 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:57.956465  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:57.950718  6656 rpcz_store.cc:276] 0709 12:18:56.232848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:57.086840 (+853992us) service_pool.cc:224] Handling call
0709 12:18:57.087083 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:57.244703 (+157620us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:57.244756 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:57.244820 (+    64us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:57.244870 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:57.928957 (+684087us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b540044a7d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:57.929047 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:57.929332 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":361838464}
I20260709 12:18:57.957872  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:57.960965  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:57.961542  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:57.962531 11184 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:57.963878  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:57.964908  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:57.979172  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:57.979632  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:57.979835  6657 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:57.980453  6584 raft_consensus.cc:2368] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 10. Current term is 11.
I20260709 12:18:57.981485  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:57.965138  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:57.965582 11006 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.983631 11006 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:57.967543  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:57.986285  6582 raft_consensus.cc:2368] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 10. Current term is 11.
I20260709 12:18:57.968005  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:57.990553  6503 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:57.968878  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:57.969489  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:57.971496 10986 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:57.971762  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:57.972993  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2650) took 1740 ms. Trace:
I20260709 12:18:57.956313 11242 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:18:57.974092  7268 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38558 (request call id 674) took 2553 ms (client timeout 1562 ms). Trace:
I20260709 12:18:57.960964  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:57.974915 10877 consensus_queue.cc:1059] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:18:57.977680  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:58.002528 10986 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:58.005995 10986 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:58.006901  6575 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:58.009323 10999 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:58.012662  6651 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:18:58.014016  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:58.014827  6651 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 14.
I20260709 12:18:58.015844  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:58.017455 10615 consensus_queue.cc:1059] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:18:57.939901 10880 consensus_queue.cc:260] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:58.020453 11242 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:57.999058  7268 rpcz_store.cc:269] 0709 12:18:55.420106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:55.420266 (+   160us) service_pool.cc:224] Handling call
0709 12:18:57.973651 (+2553385us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b54005de654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:57.973827 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1425752832}
I20260709 12:18:58.024322  7240 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:57.996929  6652 rpcz_store.cc:276] 0709 12:18:56.232326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:57.077288 (+844962us) service_pool.cc:224] Handling call
0709 12:18:57.833239 (+755951us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:57.833771 (+   532us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:57.833807 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:57.833883 (+    76us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:57.833924 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:57.834017 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:57.972651 (+138634us) spinlock_profiling.cc:243] Waited 755 ms on lock 0x7b540044b1d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:57.972802 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1737308416}
I20260709 12:18:58.026206  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 623) took 1126 ms. Trace:
I20260709 12:18:58.037151  6577 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:18:58.038280  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.041433  6579 raft_consensus.cc:2410] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:18:58.034200  6583 rpcz_store.cc:276] 0709 12:18:56.900004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.975845 (+ 75841us) service_pool.cc:224] Handling call
0709 12:18:56.980233 (+  4388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:57.067038 (+ 86805us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.025767 (+958729us) spinlock_profiling.cc:243] Waited 72.3 ms on lock 0x7b5400057dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:58.026043 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":166379392}
I20260709 12:18:58.040205 10615 consensus_queue.cc:1243] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 14.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:18:58.044360 11146 raft_consensus.cc:3055] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:58.046842  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:58.043480  6575 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:58.053632  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.057772  6581 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6.
I20260709 12:18:58.059442 10880 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:58.062016  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:58.063284  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:58.063853  6658 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
I20260709 12:18:58.064818  6507 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:18:58.064973  6506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2197) took 1291 ms. Trace:
I20260709 12:18:58.056103  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.065948  6582 raft_consensus.cc:2368] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 11. Current term is 12.
I20260709 12:18:58.066272 11156 raft_consensus.cc:3055] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:18:58.066695 11156 raft_consensus.cc:740] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:58.067333  7271 raft_consensus.cc:1240] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7. Ops: []
W20260709 12:18:58.068075  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 43894.5us
I20260709 12:18:58.068924  7240 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:58.069200 11162 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:58.065446  6506 rpcz_store.cc:276] 0709 12:18:56.774116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:56.861348 (+ 87232us) service_pool.cc:224] Handling call
0709 12:18:56.896022 (+ 34674us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:57.748732 (+852710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.064652 (+315920us) spinlock_profiling.cc:243] Waited 828 ms on lock 0x7b54001e7dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:58.064913 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1904919424}
I20260709 12:18:58.069866 11254 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:18:58.069875 11162 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:58.070504 11254 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:58.070935 11254 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:58.070827  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:58.072474 11006 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:58.072759  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:58.073499  6623 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:58.056881 11228 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:18:58.074230  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 50045.2us
I20260709 12:18:58.074568 11228 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:58.075439 10877 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:58.075810 10877 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:58.076192 10877 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:58.076567  7240 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:58.077469 11233 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:58.077924 11233 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:58.078302 11233 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:58.078121  6407 catalog_manager.cc:5697] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 5 to 6, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 6 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:18:58.079164 11251 consensus_queue.cc:1059] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:18:58.080735 11012 raft_consensus.cc:3055] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:58.044819 11146 raft_consensus.cc:740] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:58.081192 11012 raft_consensus.cc:740] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:58.082168 11146 consensus_queue.cc:260] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 13.5, Last appended by leader: 5, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:18:58.082377 11012 consensus_queue.cc:260] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:58.083835 11146 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:58.080003  6407 catalog_manager.cc:5697] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:18:58.085669  6407 catalog_manager.cc:5697] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:18:58.072220 11162 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:58.081213 11228 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:58.088927  6402 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57888 (request call id 216) took 40770 ms (client timeout 15000 ms). Trace:
I20260709 12:18:58.073800  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:58.091706  6654 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 14.
I20260709 12:18:58.067780 11156 consensus_queue.cc:260] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:58.087431  6407 catalog_manager.cc:5697] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:18:58.092937  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:58.093369 11156 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:58.094772  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:58.096127  6407 catalog_manager.cc:5697] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:58.098151  6407 catalog_manager.cc:5697] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 8 to 10, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 10 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:18:58.100296 11012 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:58.100039  6407 catalog_manager.cc:5697] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 5 to 9, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 9 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
W20260709 12:18:58.101898  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 69436.5us
I20260709 12:18:58.056032  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:58.104398  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 72789.2us
I20260709 12:18:58.105211  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.105834  6582 raft_consensus.cc:2376] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 6. Re-sending same reply.
I20260709 12:18:58.106704  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:58.107510 11083 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:58.107815 11277 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:58.108376 11083 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:58.109287 11234 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:58.109179 11277 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:58.109922 11234 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:58.111030 11083 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:58.111527 11277 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:58.111897 11234 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:58.113374  6582 raft_consensus.cc:2368] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 10. Current term is 11.
W20260709 12:18:58.104007  6622 leader_election.cc:336] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.588s (SENT)
I20260709 12:18:58.103946  6652 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:18:58.057314  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:58.058665 11130 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:58.128116  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:58.117587  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:58.047626 10996 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.133116  6657 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:18:58.134061  6584 raft_consensus.cc:2376] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10. Re-sending same reply.
I20260709 12:18:58.132468  6580 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:18:58.138757  6578 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:18:58.145162  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:58.143464  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:58.147490  6509 raft_consensus.cc:1813] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Handling vote request from an unknown peer f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:58.149292  6509 raft_consensus.cc:2410] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 8 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:18:58.153844  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:58.158646  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:58.159610  6507 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:18:58.148227 11242 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:58.176827  6503 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:18:58.173810  6650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2659) took 1294 ms. Trace:
I20260709 12:18:58.180097  6650 rpcz_store.cc:276] 0709 12:18:56.874692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.151095 (+1276403us) service_pool.cc:224] Handling call
0709 12:18:58.153858 (+  2763us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:58.158542 (+  4684us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:58.158575 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:58.168940 (+ 10365us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:58.168986 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.169053 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:58.169189 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":318208}
I20260709 12:18:58.142362 10996 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:58.183113  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 712) took 1761 ms. Trace:
I20260709 12:18:58.183523  6649 rpcz_store.cc:276] 0709 12:18:56.389258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:57.778744 (+1389486us) service_pool.cc:224] Handling call
0709 12:18:57.779488 (+   744us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:57.802017 (+ 22529us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:57.802052 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:57.887628 (+ 85576us) log.cc:844] Serialized 26 byte log entry
0709 12:18:57.914300 (+ 26672us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:57.914340 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:57.914411 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:58.150083 (+235672us) raft_consensus.cc:1685] finished
0709 12:18:58.150102 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:58.150225 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20462,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":284,"thread_start_us":26230,"threads_started":1,"wal-append.queue_time_us":26692}
I20260709 12:18:58.185295  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:58.186056  6649 raft_consensus.cc:2393] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 10: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:58.186802  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:58.187335  6649 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:58.200208  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:58.169893 11251 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:58.201336  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:58.187213 10996 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:58.194541  6507 raft_consensus.cc:1240] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:58.204665  6657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2657) took 1330 ms. Trace:
I20260709 12:18:58.205155  6657 rpcz_store.cc:276] 0709 12:18:56.874093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.149173 (+1275080us) service_pool.cc:224] Handling call
0709 12:18:58.162020 (+ 12847us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:58.201612 (+ 39592us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:58.201640 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:58.204322 (+  2682us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:58.204381 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.204450 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:58.204602 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:58.202867  6509 raft_consensus.cc:1240] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 11. Current term is 12. Ops: [11.5-11.5]
I20260709 12:18:58.203090  6503 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:18:58.200042  6653 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 11.
I20260709 12:18:58.193009  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:58.207741  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:58.217149  7240 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:58.221452  6502 raft_consensus.cc:1240] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:18:58.221580 11251 raft_consensus.cc:697] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:58.223027 11251 consensus_queue.cc:237] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:58.224715 10999 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:58.225055  7271 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:58.225437 10999 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:58.228482  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.228690 10962 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.229482 10962 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:58.229872  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 125891us
I20260709 12:18:58.235383  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.208000 10986 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.236716 10986 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:58.237352 11238 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:58.238106 11238 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:58.239676  6656 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:18:58.241621  6508 raft_consensus.cc:1240] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 8. Current term is 9. Ops: [7.4-8.5]
I20260709 12:18:58.257360  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:58.243351  6622 leader_election.cc:336] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.637s (SENT)
I20260709 12:18:58.240926  6658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2656) took 1367 ms. Trace:
W20260709 12:18:58.090564  6402 rpcz_store.cc:269] 0709 12:18:17.316813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:17.316978 (+   165us) service_pool.cc:224] Handling call
0709 12:18:58.077773 (+40760795us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:51.526503 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:51.527316 (+   813us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:51.527371 (+    55us) write_op.cc:435] Acquired schema lock
0709 12:18:51.527391 (+    20us) tablet.cc:662] Decoding operations
0709 12:18:51.529162 (+  1771us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:51.529285 (+   123us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:51.529318 (+    33us) tablet.cc:685] Acquiring locks for 14 operations
0709 12:18:51.613599 (+ 84281us) tablet.cc:701] Row locks acquired
0709 12:18:51.613658 (+    59us) write_op.cc:260] PREPARE: finished
0709 12:18:51.613941 (+   283us) write_op.cc:270] Start()
0709 12:18:51.614116 (+   175us) write_op.cc:276] Timestamp: P: 1783599531613852 usec, L: 0
0709 12:18:51.614152 (+    36us) op_driver.cc:348] REPLICATION: starting
0709 12:18:51.615073 (+   921us) log.cc:844] Serialized 5431 byte log entry
0709 12:18:53.010562 (+1395489us) spinlock_profiling.cc:243] Waited 3.38 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:18:53.293231 (+282669us) op_driver.cc:464] REPLICATION: finished
0709 12:18:53.309427 (+ 16196us) write_op.cc:301] APPLY: starting
0709 12:18:53.309549 (+   122us) tablet.cc:1367] starting BulkCheckPresence
0709 12:18:53.310972 (+  1423us) tablet.cc:1370] finished BulkCheckPresence
0709 12:18:53.311018 (+    46us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:18:53.759607 (+448589us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:18:53.759934 (+   327us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=0,mrs_lookups=28
0709 12:18:53.759977 (+    43us) write_op.cc:312] APPLY: finished
0709 12:18:53.760465 (+   488us) log.cc:844] Serialized 188 byte log entry
0709 12:18:53.785530 (+ 25065us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:53.809406 (+ 23876us) write_op.cc:454] Released schema lock
0709 12:18:53.819616 (+ 10210us) write_op.cc:341] FINISH: Updating metrics
0709 12:18:53.853245 (+ 33629us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:18:53.853280 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1709056,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":15847,"apply.run_cpu_time_us":24754,"apply.run_wall_time_us":543797,"cfile_cache_hit":26,"cfile_cache_hit_bytes":27118,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6213,"cfile_init":1,"lbm_read_time_us":56476,"lbm_reads_10-100_ms":2,"lbm_reads_lt_1ms":6,"num_ops":14,"prepare.queue_time_us":124022,"prepare.run_cpu_time_us":29077,"prepare.run_wall_time_us":123767,"raft.queue_time_us":196741,"raft.run_cpu_time_us":1805,"raft.run_wall_time_us":7076,"replication_time_us":1678818,"spinlock_wait_cycles":7801472,"thread_start_us":254885,"threads_started":4,"wal-append.queue_time_us":34830,"wal-append.run_cpu_time_us":29231,"wal-append.run_wall_time_us":3142767}]]}
I20260709 12:18:58.258544  6658 rpcz_store.cc:276] 0709 12:18:56.873756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.140086 (+1266330us) service_pool.cc:224] Handling call
0709 12:18:58.199671 (+ 59585us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:58.240560 (+ 40889us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:58.240597 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:58.240657 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:58.240687 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.240742 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:58.240847 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:58.265381  6576 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:18:58.273234  7271 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:18:58.243585 11162 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:58.276163  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:58.276386  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:58.276942  6658 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:18:58.276782 11162 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:58.277411  6502 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:18:58.277944  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:58.278602  6658 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:18:58.279131 11162 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:58.240031 11023 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.279884 11023 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:58.281282  6622 leader_election.cc:336] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.669s (SENT)
I20260709 12:18:58.281864  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:58.279855  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:58.283407  6653 raft_consensus.cc:3055] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:58.283846  6653 raft_consensus.cc:740] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:58.284221  6658 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:18:58.284648  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.284740  6653 consensus_queue.cc:260] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:58.285846  6653 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:58.286348  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:58.287446  6581 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:18:58.239262 10986 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:58.295631  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.297910  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.295578  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:58.300959  6506 raft_consensus.cc:2393] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 12: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:58.296655 10738 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.304087  6507 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:18:58.304587  6576 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:58.296878  6407 catalog_manager.cc:5697] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 6 to 7, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 7 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:18:58.306444  6655 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:18:58.305718  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.307312 10962 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:58.313100 10880 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:18:58.304107 10738 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:58.307739 11162 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.310984  6583 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
W20260709 12:18:58.321641  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:58.328039  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:58.328449  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.334647  6581 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 7.
W20260709 12:18:58.321167  6622 leader_election.cc:336] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.756s (SENT)
W20260709 12:18:58.338364  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:58.338801  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.316299 10738 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:58.334196  6407 catalog_manager.cc:5697] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: config changed from index 2 to 6, term changed from 9 to 14, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1), VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2), VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3), VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 14 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:18:58.346952  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.340531  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:58.347033  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.350989  6506 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:18:58.353958  6507 raft_consensus.cc:1275] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:18:58.355623  6507 raft_consensus.cc:1240] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 8. Current term is 10. Ops: []
I20260709 12:18:58.336395 10999 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:58.335603 11233 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:58.356618  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:58.358355  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:58.358500 11233 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:58.359599  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:58.360556  6507 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 14.
I20260709 12:18:58.360878  6407 catalog_manager.cc:5697] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 7 to 10, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 10 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:18:58.361472  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:58.362159  6507 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 13.
I20260709 12:18:58.362366  6407 catalog_manager.cc:5697] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 3 to 7, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 7 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:18:58.363761  6658 raft_consensus.cc:1275] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 13 index: 5. Preceding OpId from leader: term: 14 index: 6. (index mismatch)
I20260709 12:18:58.322607 11130 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:58.364908  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:58.321349 11162 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:58.366176  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:58.367304 11162 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:58.367388  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:58.368273  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:58.368690  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.369064  6502 raft_consensus.cc:2393] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 13: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:18:58.369776  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:58.370052  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:58.370421  6502 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 13.
I20260709 12:18:58.370672  6655 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:18:58.371299  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:58.371636  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:58.371871  6502 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 14.
I20260709 12:18:58.372682  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:58.373347  6657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2663) took 1475 ms. Trace:
I20260709 12:18:58.363108  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:58.374047  6657 rpcz_store.cc:276] 0709 12:18:56.898306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.206297 (+1307991us) service_pool.cc:224] Handling call
0709 12:18:58.274807 (+ 68510us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:58.372754 (+ 97947us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:58.372797 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:58.372865 (+    68us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:18:58.372931 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.373086 (+   155us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:58.373247 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:58.374719  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.375335  6583 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:18:58.373409  6509 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:18:58.344732  6622 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:58.369776  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.375432  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:58.363924  6407 catalog_manager.cc:5697] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 3 to 9, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 9 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:18:58.322424 11295 log.cc:927] Time spent T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 12:18:58.349455  6579 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:18:58.372238  6655 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
W20260709 12:18:58.384033  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 62914.2us
I20260709 12:18:58.384738  6582 raft_consensus.cc:2393] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 12: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:58.380363  6581 raft_consensus.cc:1275] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:18:58.385020  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.388294  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.390957  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:58.391547  6583 raft_consensus.cc:2393] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 12: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:58.391775  6581 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:58.392997  6508 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:58.396620  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:58.397387  6503 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:18:58.398782  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:58.400435  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:58.402663  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:58.404366  6622 leader_election.cc:336] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.769s (SENT)
I20260709 12:18:58.405964  6657 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:18:58.407019  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:58.407747  6579 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:58.409019  6575 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 16: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:58.402616  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.413939  6622 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:58.416633  6653 raft_consensus.cc:1275] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 5. (term mismatch)
I20260709 12:18:58.417235  6653 pending_rounds.cc:85] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 4
I20260709 12:18:58.432408  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:58.433187  6655 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:58.435546  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.437865  6504 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:18:58.418736  6653 pending_rounds.cc:107] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 11.5
I20260709 12:18:58.438427  6653 raft_consensus.cc:2889] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:58.439708  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.440814  6575 raft_consensus.cc:2393] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 12: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:18:58.437448  6583 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:18:58.442747  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:58.443710  6657 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 16: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:58.444664  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:58.446511  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:58.447824  6654 raft_consensus.cc:2410] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:18:58.447629  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:58.450634  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.451102  6474 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:58.451359  6584 raft_consensus.cc:2376] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16. Re-sending same reply.
I20260709 12:18:58.456544  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.446738 11012 consensus_queue.cc:1059] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:18:58.457943  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.461311  6582 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:18:58.461769  6575 raft_consensus.cc:1275] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 8 index: 3. Preceding OpId from leader: term: 14 index: 6. (index mismatch)
I20260709 12:18:58.464955 10963 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:58.447309 11085 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:58.470181  6501 raft_consensus.cc:1275] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:18:58.474516  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:58.475419  6654 raft_consensus.cc:2393] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 14: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:58.479166  6653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2080) took 1418 ms. Trace:
I20260709 12:18:58.479614  6653 rpcz_store.cc:276] 0709 12:18:57.061293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.267329 (+1206036us) service_pool.cc:224] Handling call
0709 12:18:58.283228 (+ 15899us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:58.478916 (+195688us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.479106 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:18:58.481649  6579 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:58.483625  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.484486  6649 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:18:58.486382  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.432677 10877 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:58.509470  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:58.488245  6655 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:18:58.520313  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:58.517117  6407 catalog_manager.cc:5697] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 10 to 13, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 13 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:58.523259  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:58.520757  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2662) took 1628 ms. Trace:
I20260709 12:18:58.524053  6656 rpcz_store.cc:276] 0709 12:18:56.892596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.150710 (+1258114us) service_pool.cc:224] Handling call
0709 12:18:58.207279 (+ 56569us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:58.257731 (+ 50452us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.510676 (+252945us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b540019dcd0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:58.520683 (+ 10007us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42541312}
I20260709 12:18:58.547498  6582 raft_consensus.cc:1275] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:18:58.505357 11162 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:58.548394  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:58.548949  6651 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:58.551515  6584 raft_consensus.cc:2368] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 12. Current term is 13.
I20260709 12:18:58.553709  6658 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:58.558027  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:58.554930  7269 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.013s	sys 0.025s
W20260709 12:18:58.559692  7269 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.014s	sys 0.025s
I20260709 12:18:58.522854 11012 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.506798 11130 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 12:18:58.556027  7238 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:58.569278  6582 raft_consensus.cc:1240] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 8. Current term is 9. Ops: []
I20260709 12:18:58.560936  6407 catalog_manager.cc:5697] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:18:58.572245  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 173769us
I20260709 12:18:58.575994 11038 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.576817 11038 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:58.579643 11038 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:18:58.580320  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 44218.8us
I20260709 12:18:58.581310  6475 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:58.584542  6582 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 12. Current term is 13. Ops: []
I20260709 12:18:58.588709  6579 raft_consensus.cc:1240] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:58.590198  6582 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 11. Current term is 12. Ops: []
W20260709 12:18:58.587205  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 79317us
I20260709 12:18:58.563937 11132 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:58.597119  6475 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:58.606024  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.599416  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.620800 11147 consensus_queue.cc:1059] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:18:58.621809 11123 raft_consensus.cc:3055] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:58.622179 11123 raft_consensus.cc:740] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:58.623122 11123 consensus_queue.cc:260] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:58.623906 11023 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:58.621356  6579 raft_consensus.cc:2393] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 14: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:58.632885  6474 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:58.624316 11123 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:58.646749  6581 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:18:58.638808  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.603339 11038 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:58.572422 11277 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 12:18:58.572031 11012 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:58.630930 10963 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:58.658059  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:58.665313  6657 raft_consensus.cc:1275] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:18:58.670397  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 77650.8us
I20260709 12:18:58.666347  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:58.672257  7268 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:18:58.677248  7271 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:18:58.655634  6407 catalog_manager.cc:5697] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 6 to 10, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 10 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
W20260709 12:18:58.675043 11112 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.026s	sys 0.047s
I20260709 12:18:58.684226  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.654217 11184 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:58.652935 11233 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:18:58.668170 11038 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:18:58.701160  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 107976us
W20260709 12:18:58.684437 11112 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.026s	sys 0.048s
I20260709 12:18:58.704828  6549 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
W20260709 12:18:58.716881  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 62193.2us
I20260709 12:18:58.716951  6475 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:58.715871 11038 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:58.719506 10877 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:58.720022 11233 raft_consensus.cc:697] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 12 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:58.721215 11233 consensus_queue.cc:237] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:58.724485 11184 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:58.726889 11184 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:58.727479 11184 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:58.675483 11147 consensus_queue.cc:1059] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
W20260709 12:18:58.733685 11156 consensus_meta.cc:287] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.640s	user 0.011s	sys 0.003s
I20260709 12:18:58.733923 11147 raft_consensus.cc:3055] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:18:58.734077 11254 consensus_meta.cc:287] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.662s	user 0.012s	sys 0.001s
I20260709 12:18:58.734408 11147 raft_consensus.cc:740] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:58.734673  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.734738 11254 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:58.735404 11147 consensus_queue.cc:260] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:58.692389 11277 consensus_queue.cc:1243] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer f2b06aebe97249a4877d798b8118ece4 log is divergent from this leader: its last log entry 12.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:18:58.738197  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:58.700493 11228 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:58.785421 11228 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:18:58.785828 11228 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:58.741230 11012 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:58.789288  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:18:58.741925  6474 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:18:58.790621  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 48870.3us
W20260709 12:18:58.791702  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.742476  6550 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:18:58.792723  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 123457us
I20260709 12:18:58.793310  6550 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:58.793725  7273 raft_consensus.cc:1240] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7. Ops: []
I20260709 12:18:58.794665  6550 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:58.795225 11162 consensus_queue.cc:1059] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:18:58.741487  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 215994us
I20260709 12:18:58.796504 11023 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:58.796931 11023 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:58.797335 11023 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:58.744104 11277 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:18:58.800602 11277 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:58.796124 10999 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:58.801491 10999 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:58.801927 10999 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:58.802513  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:58.748618 11238 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:58.754367  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.803695 11238 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:58.804068 11238 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:58.747938 11132 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:58.804356  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.804772 11132 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:58.805053  6584 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:18:58.754765  7266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38546 (request call id 533) took 1059 ms. Trace:
I20260709 12:18:58.806459  7266 rpcz_store.cc:276] 0709 12:18:57.695067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:57.698113 (+  3046us) service_pool.cc:224] Handling call
0709 12:18:58.057941 (+359828us) spinlock_profiling.cc:243] Waited 2.63 ms on lock 0x7b5800125198. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:18:58.068235 (+ 10294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:58.074246 (+  6011us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:58.074278 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:58.074813 (+   535us) log.cc:844] Serialized 26 byte log entry
0709 12:18:58.487762 (+412949us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:58.487809 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.487877 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:58.754379 (+266502us) raft_consensus.cc:1685] finished
0709 12:18:58.754401 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:58.754613 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5404,"raft.queue_time_us":6733,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"spinlock_wait_cycles":6046592,"thread_start_us":412553,"threads_started":1,"wal-append.queue_time_us":634925}
I20260709 12:18:58.805985  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:58.743556  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 48711.9us
I20260709 12:18:58.808807  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:58.809108  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.810238  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.810340  6475 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:58.684690 10963 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:58.811301  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.811846  6584 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:18:58.758798 11220 raft_consensus.cc:3055] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:18:58.762635  6651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2076) took 2281 ms. Trace:
I20260709 12:18:58.812554 11220 raft_consensus.cc:740] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:58.812754  6651 rpcz_store.cc:276] 0709 12:18:56.480957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.133753 (+1652796us) service_pool.cc:224] Handling call
0709 12:18:58.158594 (+ 24841us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:58.556890 (+398296us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.762364 (+205474us) spinlock_profiling.cc:243] Waited 377 ms on lock 0x7b5400075dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:18:58.762556 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":866248576}
I20260709 12:18:58.766894 10990 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:58.813539 11220 consensus_queue.cc:260] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:58.814070 10990 raft_consensus.cc:697] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:58.814826 11220 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:18:58.815297  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.814967 10990 consensus_queue.cc:237] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:58.767338  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 62723.4us
I20260709 12:18:58.760839 11147 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:58.770576 11085 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:58.800907 11006 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 12:18:58.819855 11085 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:18:58.822125  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.759830  6658 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
I20260709 12:18:58.776643  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.824292  6581 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:18:58.825457  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.826535  6581 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:18:58.748750  6579 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:18:58.782465  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.822154  6550 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:58.777503  6407 catalog_manager.cc:5697] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 5 to 7, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 7 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:18:58.801023 11277 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:58.745771  6473 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:58.804436 11238 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:58.819228  6549 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:58.768687  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.828073  6583 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:18:58.829849  6550 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:58.832769  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.833073  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:58.836203  6581 raft_consensus.cc:3055] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:18:58.836755  6581 raft_consensus.cc:740] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:58.828735  7240 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:58.781100 11038 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:58.839138  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.844238  6584 raft_consensus.cc:1240] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 8. Current term is 10. Ops: [8.7-8.7]
I20260709 12:18:58.845403  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:18:58.847828  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:58.832662 11251 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.010s	sys 0.051s
W20260709 12:18:58.864089  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.832005 10738 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:18:58.832381 10963 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:58.843149 10986 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:18:58.870569  6407 catalog_manager.cc:5697] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 6 to 8, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 8 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:58.872753  6582 raft_consensus.cc:1240] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:18:58.873957  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.834328 11015 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:18:58.882141  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:18:58.837800  6581 consensus_queue.cc:260] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:58.881278  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.876094  6584 raft_consensus.cc:1240] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14. Ops: [10.4-13.5]
I20260709 12:18:58.854341 11130 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:18:58.886297  6581 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:58.886371  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:18:58.876493 11251 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.010s	sys 0.051s
I20260709 12:18:58.877734 10962 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:18:58.888090  6578 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.031s	sys 0.036s
W20260709 12:18:58.894152  6578 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.031s	sys 0.036s
W20260709 12:18:58.894627  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:58.889981  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 446265us
I20260709 12:18:58.889361  6407 catalog_manager.cc:5697] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 8 to 11, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 11 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
W20260709 12:18:58.899210  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:58.903699  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.878072 10738 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:58.882609 11015 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:58.915071  6407 catalog_manager.cc:5697] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:18:58.935778  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:58.894021 10962 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:18:58.938380  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:58.938601 10962 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:18:58.939288 11074 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 12:18:58.944653  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:18:58.884032 11184 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:18:58.914049 10738 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:58.908385 11130 raft_consensus.cc:697] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 15 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:18:58.937355  6579 raft_consensus.cc:1275] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:18:58.926913 11015 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:18:58.996131  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 303793us
I20260709 12:18:58.927367 10877 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:58.994133  6622 leader_election.cc:336] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.889s (SENT)
I20260709 12:18:59.013679  6407 catalog_manager.cc:5697] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 2 to 9, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 9 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
W20260709 12:18:59.021243 11291 log.cc:927] Time spent T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:18:58.993295 11156 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.000361 11184 consensus_queue.cc:1243] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer f2b06aebe97249a4877d798b8118ece4 log is divergent from this leader: its last log entry 13.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:18:59.022846  6407 catalog_manager.cc:5697] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 2 to 5, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 5 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:59.062355  6407 catalog_manager.cc:5697] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 8 to 10, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 10 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:18:59.036465  7238 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:59.022140 11083 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:18:59.081691  6650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2665) took 2167 ms. Trace:
W20260709 12:18:59.099377  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 353659us
I20260709 12:18:59.022055 10877 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:59.102734  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 292493us
I20260709 12:18:59.108543  6581 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
W20260709 12:18:59.112568  6654 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.019s	sys 0.055s
W20260709 12:18:59.113785  6654 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.019s	sys 0.055s
I20260709 12:18:59.088619  7238 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:59.082405  6650 rpcz_store.cc:276] 0709 12:18:56.914798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.209182 (+1294384us) service_pool.cc:224] Handling call
0709 12:18:58.220267 (+ 11085us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:58.220725 (+   458us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:18:58.647769 (+427044us) log.cc:844] Serialized 14 byte log entry
0709 12:18:59.081220 (+433451us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:59.081276 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:18:59.081324 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.081479 (+   155us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:59.081620 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":693603,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":3937,"raft.queue_time_us":89,"thread_start_us":677905,"threads_started":2}
I20260709 12:18:59.032624 11228 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:59.077555 11083 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:18:59.124821  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 186676us
I20260709 12:18:59.081912 10615 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:59.058224 11156 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:59.127885  6475 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:59.102552 10986 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.118384 11112 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:18:59.147962  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 47888.9us
I20260709 12:18:59.159268 11315 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:59.159842 11315 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:59.160225 11315 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:59.160372 11228 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:59.160530 11301 log.cc:927] Time spent T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 12:18:59.161391  6475 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:59.162555 11320 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.163074 11320 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:59.163446 11228 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.163950 11228 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:59.148252  6505 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.028s	sys 0.050s
W20260709 12:18:59.164779  6505 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.028s	sys 0.051s
I20260709 12:18:59.165305 11320 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:59.166033 11320 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.166301 11228 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:59.166921 11228 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:18:59.167321 11228 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:59.167891 11228 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:59.136096 11233 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:18:59.168519 11184 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:59.168931 11184 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:59.169299 11184 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:59.166496 11320 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:59.171701 11320 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:59.127930 11083 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:18:59.173079 11320 consensus_queue.cc:1048] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:18:59.177901  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2660) took 2303 ms. Trace:
W20260709 12:18:59.182124  6622 leader_election.cc:336] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.912s (SENT)
I20260709 12:18:59.182626  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:59.132308 11038 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:59.138329 10986 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:59.180605  6652 rpcz_store.cc:276] 0709 12:18:56.874913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.151076 (+1276163us) service_pool.cc:224] Handling call
0709 12:18:58.196810 (+ 45734us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:58.290150 (+ 93340us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:18:58.828730 (+538580us) log.cc:844] Serialized 14 byte log entry
0709 12:18:59.168331 (+339601us) log.cc:844] Serialized 14 byte log entry
0709 12:18:59.177576 (+  9245us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:59.177627 (+    51us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:59.177668 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.177728 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:59.177851 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":650901,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":73,"raft.queue_time_us":17953,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":122,"thread_start_us":728640,"threads_started":2}
I20260709 12:18:59.253854  6653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2090) took 1340 ms. Trace:
W20260709 12:18:59.254308  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 133427us
I20260709 12:18:59.262951  7238 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:59.128688 11130 consensus_queue.cc:237] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:59.255401 11123 consensus_meta.cc:287] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.603s	user 0.005s	sys 0.007s
I20260709 12:18:59.264814  7238 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:59.225569 11251 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:59.262585  6653 rpcz_store.cc:276] 0709 12:18:57.916916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.484288 (+567372us) service_pool.cc:224] Handling call
0709 12:18:58.511789 (+ 27501us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:58.522666 (+ 10877us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:18:59.049210 (+526544us) log.cc:844] Serialized 14 byte log entry
0709 12:18:59.248654 (+199444us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:59.248708 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:18:59.252930 (+  4222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.253012 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:59.253780 (+   768us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21298,"prepare.queue_time_us":408891,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":93,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":3655,"thread_start_us":618889,"threads_started":2}
I20260709 12:18:59.273173  7266 raft_consensus.cc:2410] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:18:59.251484 11234 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:59.267084  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:18:59.271837  6474 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:59.278545  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 129041us
W20260709 12:18:59.316442  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.277806  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:18:59.304948  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:18:59.305941  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 153343us
I20260709 12:18:59.268965 11233 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:18:59.333540  6506 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
W20260709 12:18:59.328545  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 452030us
W20260709 12:18:59.315604 11132 consensus_meta.cc:287] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.510s	user 0.008s	sys 0.003s
W20260709 12:18:59.347414  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 75675.3us
I20260709 12:18:59.332090  6654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 718) took 1436 ms. Trace:
W20260709 12:18:59.327384  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 61616.8us
W20260709 12:18:59.348978  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.394575 11335 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:18:59.395457  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 145295us
W20260709 12:18:59.399377  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 66201.1us
I20260709 12:18:59.395646  6500 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
W20260709 12:18:59.403832  6622 leader_election.cc:336] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.913s (SENT)
I20260709 12:18:59.353426  6654 rpcz_store.cc:276] 0709 12:18:57.895293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.476022 (+580729us) service_pool.cc:224] Handling call
0709 12:18:58.478730 (+  2708us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:18:58.511405 (+ 32675us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:18:59.129040 (+617635us) log.cc:844] Serialized 14 byte log entry
0709 12:18:59.331093 (+202053us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:18:59.331148 (+    55us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:59.331193 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.331260 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:59.332031 (+   771us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":422147,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":26,"raft.queue_time_us":771,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":11293,"thread_start_us":737046,"threads_started":2}
I20260709 12:18:59.400090  6473 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:59.406210 11147 consensus_meta.cc:287] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.579s	user 0.009s	sys 0.001s
I20260709 12:18:59.406399 11316 raft_consensus.cc:2804] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:18:59.407028 11316 raft_consensus.cc:697] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
W20260709 12:18:59.408596  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:59.409164  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:59.410486 11326 log.cc:927] Time spent T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260709 12:18:59.411989 11318 log.cc:927] Time spent T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 12:18:59.413183  6622 leader_election.cc:336] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.925s (SENT)
I20260709 12:18:59.413818  6622 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:59.399081 11132 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:59.419713 11132 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:59.421384 11319 log.cc:927] Time spent T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.260s	user 0.000s	sys 0.003s
W20260709 12:18:59.424060 11304 log.cc:927] Time spent T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.256s	user 0.000s	sys 0.001s
I20260709 12:18:59.424885 11156 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:59.425362  6473 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:59.425741  6502 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 546) took 1070 ms. Trace:
I20260709 12:18:59.426167  6502 rpcz_store.cc:276] 0709 12:18:58.355461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.388985 (+ 33524us) service_pool.cc:224] Handling call
0709 12:18:58.429602 (+ 40617us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:58.452671 (+ 23069us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:58.452736 (+    65us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:58.469717 (+ 16981us) log.cc:844] Serialized 26 byte log entry
0709 12:18:58.757019 (+287302us) spinlock_profiling.cc:243] Waited 8.57 ms on lock 0x7b1800457c60. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:18:59.114601 (+357582us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:18:59.114827 (+   226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.115111 (+   284us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:59.425406 (+310295us) raft_consensus.cc:1685] finished
0709 12:18:59.425435 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:59.425657 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2982,"raft.queue_time_us":8702,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":3072,"spinlock_wait_cycles":19715840,"thread_start_us":327643,"threads_started":1,"wal-append.queue_time_us":793805}
I20260709 12:18:59.428303  6505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2343) took 1279 ms. Trace:
I20260709 12:18:59.428750  6505 rpcz_store.cc:276] 0709 12:18:58.149398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.192884 (+ 43486us) service_pool.cc:224] Handling call
0709 12:18:58.246015 (+ 53131us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:58.312061 (+ 66046us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:18:58.359767 (+ 47706us) log.cc:844] Serialized 14 byte log entry
0709 12:18:59.165422 (+805655us) log.cc:844] Serialized 14 byte log entry
0709 12:18:59.269334 (+103912us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:59.349045 (+ 79711us) log.cc:844] Serialized 26 byte log entry
0709 12:18:59.349195 (+   150us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:59.349547 (+   352us) log.cc:844] Serialized 14 byte log entry
0709 12:18:59.396731 (+ 47184us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.396807 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:59.423392 (+ 26585us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:18:59.428087 (+  4695us) raft_consensus.cc:1685] finished
0709 12:18:59.428110 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:59.428244 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48745,"prepare.queue_time_us":73200,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":110,"raft.queue_time_us":166286,"raft.run_cpu_time_us":342,"raft.run_wall_time_us":15578,"spinlock_wait_cycles":35791744,"thread_start_us":908586,"threads_started":2,"wal-append.queue_time_us":807383}
W20260709 12:18:59.430557 11307 log.cc:927] Time spent T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.267s	user 0.000s	sys 0.002s
I20260709 12:18:59.430936 11322 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:18:59.432622 11023 consensus_meta.cc:287] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.635s	user 0.002s	sys 0.011s
I20260709 12:18:59.433094 11023 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:59.433611  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 545) took 1078 ms. Trace:
I20260709 12:18:59.434094 11322 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:59.435384 11023 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:59.433993  6507 rpcz_store.cc:276] 0709 12:18:58.355092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.374586 (+ 19494us) service_pool.cc:224] Handling call
0709 12:18:58.374951 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:58.375569 (+   618us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:58.375600 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:58.383631 (+  8031us) log.cc:844] Serialized 26 byte log entry
0709 12:18:58.883853 (+500222us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:58.917075 (+ 33222us) log.cc:844] Serialized 14 byte log entry
0709 12:18:59.251236 (+334161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.251327 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:59.429917 (+178590us) spinlock_profiling.cc:243] Waited 18.8 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:18:59.433294 (+  3377us) raft_consensus.cc:1685] finished
0709 12:18:59.433403 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:59.433551 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8369,"prepare.queue_time_us":32992,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":252974,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":216,"spinlock_wait_cycles":43298048,"thread_start_us":825261,"threads_started":2,"wal-append.queue_time_us":779137}
W20260709 12:18:59.436434 10999 consensus_meta.cc:287] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.634s	user 0.012s	sys 0.001s
I20260709 12:18:59.436960 10999 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:59.439759  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:18:59.335040 11050 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:18:59.448055  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 286734us
I20260709 12:18:59.473037  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:18:59.466049 11310 log.cc:927] Time spent T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 12:18:59.501503  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:59.490056 11085 consensus_meta.cc:287] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.669s	user 0.009s	sys 0.001s
W20260709 12:18:59.490520 11238 consensus_meta.cc:287] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.644s	user 0.010s	sys 0.000s
I20260709 12:18:59.492772  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:59.493196  6649 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 13. Current term is 14. Ops: []
I20260709 12:18:59.494033  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:59.512324  6657 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:18:59.495374  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:59.514477  6655 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:59.496304  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:59.497095 10986 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:59.516952  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 103811us
W20260709 12:18:59.497012  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 263923us
I20260709 12:18:59.400770 10962 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:59.485534  6473 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:18:59.521996  6653 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:18:59.457051 11012 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.477164 11156 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:18:59.407969 11316 consensus_queue.cc:237] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:59.476238 11339 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:18:59.524564  6473 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:18:59.487825 10999 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:59.395316 11335 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:59.516686 11336 log.cc:927] Time spent T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:18:59.527637  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:59.541265  6658 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:18:59.528923  6654 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:18:59.529445  6475 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:59.531468  6548 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:59.542902  6622 leader_election.cc:336] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.942s (SENT)
I20260709 12:18:59.520620 10877 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:59.539342  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:59.546236  6657 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:18:59.543432  6622 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:59.536821  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:18:59.549608  6653 raft_consensus.cc:2410] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:18:59.533424  6550 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:59.486537 11184 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:59.558466  6655 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:18:59.516928 11085 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:59.562316  6651 raft_consensus.cc:3055] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:18:59.571558  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 62428.2us
I20260709 12:18:59.576889  6475 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:59.519906 11238 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:59.580854  6583 raft_consensus.cc:1240] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 11. Current term is 12. Ops: [11.5-11.5]
I20260709 12:18:59.576310  6651 raft_consensus.cc:740] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:18:59.580245  6576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2139) took 1258 ms. Trace:
I20260709 12:18:59.543680 11340 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:18:59.556828 11339 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:18:59.586516  7265 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: []
I20260709 12:18:59.588074  6475 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:59.530530 11012 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:59.583451  6576 rpcz_store.cc:276] 0709 12:18:58.321949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.338363 (+ 16414us) service_pool.cc:224] Handling call
0709 12:18:58.376835 (+ 38472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:58.384101 (+  7266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:58.384157 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:18:58.405065 (+ 20908us) log.cc:844] Serialized 26 byte log entry
0709 12:18:58.901132 (+496067us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:18:58.901188 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:58.901284 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:59.579851 (+678567us) raft_consensus.cc:1685] finished
0709 12:18:59.579884 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:59.580131 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":684,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":10866,"thread_start_us":455651,"threads_started":1,"wal-append.queue_time_us":687560}
W20260709 12:18:59.590052  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 56708.2us
I20260709 12:18:59.591219  6550 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:18:59.597411  6651 consensus_queue.cc:260] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:59.598521  6651 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:18:59.598933 11085 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:59.591967 11308 log.cc:927] Time spent T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.325s	user 0.004s	sys 0.005s
I20260709 12:18:59.602324  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2140) took 1279 ms. Trace:
W20260709 12:18:59.602530 11325 log.cc:927] Time spent T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.272s	user 0.000s	sys 0.001s
I20260709 12:18:59.602907  6580 rpcz_store.cc:276] 0709 12:18:58.322497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.332549 (+ 10052us) service_pool.cc:224] Handling call
0709 12:18:58.377303 (+ 44754us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:18:58.405534 (+ 28231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:58.405581 (+    47us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:18:58.437931 (+ 32350us) log.cc:844] Serialized 78 byte log entry
0709 12:18:58.890669 (+452738us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:18:58.917387 (+ 26718us) log.cc:844] Serialized 14 byte log entry
0709 12:18:59.272851 (+355464us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.272943 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:59.601978 (+329035us) raft_consensus.cc:1685] finished
0709 12:18:59.602010 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:59.602240 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":26471,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":10589,"raft.queue_time_us":160760,"raft.run_cpu_time_us":443,"raft.run_wall_time_us":33012,"thread_start_us":803328,"threads_started":2,"wal-append.queue_time_us":579949}
I20260709 12:18:59.609699 11238 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:59.610632  6393 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 3e0ab356fc074f589a6ff4f9b4996b09 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 20a202c76244485fbc8f1610473db8b9 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:18:59.611933 11085 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:18:59.612540 11085 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:59.614889 11085 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:59.616423 10738 consensus_meta.cc:287] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.601s	user 0.006s	sys 0.005s
I20260709 12:18:59.616933  7269 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 11. Current term is 12. Ops: []
I20260709 12:18:59.619045 11012 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:59.619782  6651 raft_consensus.cc:1275] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:18:59.620177  6651 pending_rounds.cc:85] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 4
I20260709 12:18:59.620437  6651 pending_rounds.cc:107] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:18:59.620687  6651 raft_consensus.cc:2889] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:18:59.622756 11322 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:18:59.650233  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 59119.8us
I20260709 12:18:59.575198 11331 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.140s
I20260709 12:18:59.616840 10738 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:59.591740 11343 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:18:59.670096  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 127218us
I20260709 12:18:59.670163 10738 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:59.585923 11345 raft_consensus.cc:2764] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 12:18:59.672942  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 153790us
W20260709 12:18:59.675845  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 87820.6us
I20260709 12:18:59.592836 10996 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:18:59.713371  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:59.717175  6651 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:59.719674  6548 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:18:59.704205 11015 consensus_meta.cc:287] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.660s	user 0.007s	sys 0.003s
I20260709 12:18:59.635823 11238 consensus_queue.cc:1059] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 12:18:59.767998  6657 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
W20260709 12:18:59.712661 11220 consensus_meta.cc:287] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.897s	user 0.012s	sys 0.003s
W20260709 12:18:59.712965 11277 consensus_meta.cc:287] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.859s	user 0.011s	sys 0.000s
I20260709 12:18:59.708808 11339 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:59.760957 10615 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.725356 11331 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:59.797277 10615 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:59.797550 11341 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:18:59.786283 10962 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:18:59.799917 11311 log.cc:927] Time spent T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.635s	user 0.003s	sys 0.006s
W20260709 12:18:59.800987  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:59.796633 11083 consensus_meta.cc:287] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.571s	user 0.009s	sys 0.004s
I20260709 12:18:59.804522 11328 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:18:59.805076 11327 consensus_queue.cc:1048] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.209s
I20260709 12:18:59.807829 11006 raft_consensus.cc:3055] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:18:59.713904 10996 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:59.808619 11050 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:59.809891  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 2087) took 1126 ms. Trace:
I20260709 12:18:59.860387  6583 rpcz_store.cc:276] 0709 12:18:58.683260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.842510 (+159250us) service_pool.cc:224] Handling call
0709 12:18:59.580477 (+737967us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:18:59.582322 (+  1845us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.809725 (+227403us) spinlock_profiling.cc:243] Waited 737 ms on lock 0x7b54002657d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:59.809838 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1695500032}
I20260709 12:18:59.815433  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:18:59.820075  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 48417.5us
I20260709 12:18:59.825598  6474 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:18:59.828408  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 122692us
W20260709 12:18:59.831686  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.876368  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:18:59.876947  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:59.877447  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:59.877847  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:59.878247  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:18:59.878741  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.803493 11254 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:59.838429 11328 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:59.834704  6651 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:18:59.836500  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:59.890861  6658 raft_consensus.cc:2393] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 11: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:18:59.837767  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:59.787354 10990 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:18:59.845997  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 255909us
I20260709 12:18:59.835098  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:59.894173  6657 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:18:59.764972 10738 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:59.788041 11277 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:59.926592  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.876055  7240 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:18:59.927947  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 52040.2us
I20260709 12:18:59.834138  6655 raft_consensus.cc:1275] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:18:59.929327  6655 pending_rounds.cc:85] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 9
I20260709 12:18:59.929666  6655 pending_rounds.cc:107] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:18:59.929961  6655 raft_consensus.cc:2889] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:18:59.895915  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 70393us
W20260709 12:18:59.930239  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.896836  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:18:59.877882 11322 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 12:18:59.899176 10986 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:18:59.901587 11132 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:59.857928 11006 raft_consensus.cc:740] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:18:59.904534 10738 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:59.882064 11234 consensus_queue.cc:1048] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:18:59.829275 11112 consensus_queue.cc:1048] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 12:18:59.862107 10996 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:18:59.862900 11050 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:18:59.854750 11083 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:59.908773 11328 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:18:59.933759 11322 consensus_queue.cc:1243] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer f2b06aebe97249a4877d798b8118ece4 log is divergent from this leader: its last log entry 13.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 12:18:59.910513 11342 log.cc:927] Time spent T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.347s	user 0.000s	sys 0.000s
I20260709 12:18:59.878556 10999 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:18:59.856009 11341 raft_consensus.cc:697] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:18:59.911624 11315 consensus_meta.cc:287] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.751s	user 0.005s	sys 0.005s
I20260709 12:18:59.914901  6654 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 12:18:59.918324 11228 consensus_meta.cc:287] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.744s	user 0.008s	sys 0.002s
I20260709 12:18:59.919472  6657 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
W20260709 12:18:59.920651  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.925631  6548 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:18:59.774351 11015 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:59.957509 11341 consensus_queue.cc:237] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:59.934113  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 263125us
I20260709 12:18:59.905879 10615 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:18:59.958078 11277 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:59.936628  7240 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:59.958990 11085 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:18:59.945677 11302 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.784s	user 0.000s	sys 0.009s
I20260709 12:18:59.945547  7271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38558 (request call id 688) took 1069 ms. Trace:
I20260709 12:18:59.959666  7271 rpcz_store.cc:276] 0709 12:18:58.876239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.876413 (+   174us) service_pool.cc:224] Handling call
0709 12:18:59.945329 (+1068916us) spinlock_profiling.cc:243] Waited 642 ms on lock 0x7b54005de654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:18:59.945463 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1476461952}
I20260709 12:18:59.932963 11316 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:18:59.961599  6474 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:18:59.962606 11015 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:18:59.909520 11335 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:59.935092 11320 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:18:59.936378 11340 raft_consensus.cc:697] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:18:59.953145 10999 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:18:59.956782 11349 log.cc:927] Time spent T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 12:18:59.926774  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 547) took 1571 ms. Trace:
I20260709 12:18:59.983355  6509 rpcz_store.cc:276] 0709 12:18:58.355825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:58.389748 (+ 33923us) service_pool.cc:224] Handling call
0709 12:18:58.424747 (+ 34999us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:58.448867 (+ 24120us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:58.448899 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:18:58.476676 (+ 27777us) log.cc:844] Serialized 52 byte log entry
0709 12:18:58.903154 (+426478us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:18:59.429099 (+525945us) spinlock_profiling.cc:243] Waited 5.54 ms on lock 0x7b1800361300. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2875a314bc 00007f2872d7f48b 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d99455 00007f28798813c7 00007f2875e939a7 00007f2875ea30bf
0709 12:18:59.429237 (+   138us) log.cc:844] Serialized 14 byte log entry
0709 12:18:59.430530 (+  1293us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:18:59.430641 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:18:59.799084 (+368443us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:18:59.926556 (+127472us) raft_consensus.cc:1685] finished
0709 12:18:59.926576 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:18:59.926724 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19524,"prepare.queue_time_us":110959,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":39,"raft.queue_time_us":63815,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":35129,"spinlock_wait_cycles":41175680,"thread_start_us":627374,"threads_started":2,"wal-append.queue_time_us":607589}
I20260709 12:18:59.935863 10986 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:18:59.988111  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:18:59.942529 11038 consensus_queue.cc:1059] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:18:59.937458 11132 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:59.973706  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:18:59.977931  6622 leader_election.cc:336] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 2.038s (SENT)
I20260709 12:18:59.963920 11315 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:00.001755  6622 leader_election.cc:336] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 2.071s (SENT)
I20260709 12:19:00.002359 11315 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:18:59.966344 11006 consensus_queue.cc:260] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:18:59.964114 11228 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:18:59.979759 11083 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:00.003643  6622 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:18:59.991086 11015 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:00.004853 11015 raft_consensus.cc:697] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:00.005246 11132 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:00.006336 11340 consensus_queue.cc:237] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:00.007678  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 122849us
W20260709 12:19:00.009758 11346 log.cc:927] Time spent T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.300s	user 0.004s	sys 0.002s
I20260709 12:19:00.009620  6548 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:00.010123 10999 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:00.011258 10999 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:00.011736 10999 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:00.012060 10999 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:18:59.964470  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 82510.4us
I20260709 12:18:59.981997 11335 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:00.025043  7266 raft_consensus.cc:1240] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:00.025096  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:00.025669  7271 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 12:19:00.029961  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 487957us
I20260709 12:19:00.030972  6473 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:00.032105  6550 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:00.038151  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:00.039108  6655 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:19:00.041115  6550 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:00.029585 11321 log.cc:927] Time spent T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.801s	user 0.005s	sys 0.005s
I20260709 12:19:00.043506  6550 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:00.022898 11132 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.049381  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:00.045830  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:00.026803 11156 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:00.049028  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:00.050010  6650 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 11.
I20260709 12:19:00.052996  7240 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:00.036808 11162 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.057101  6473 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:00.064668 11334 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 12:19:00.064790 11130 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:00.065831 11130 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:00.066226 11130 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:00.055179 11348 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:00.043537 11345 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:00.082926 11345 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:00.083573 11345 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:00.069108 10986 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:00.069659  6650 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:19:00.051965 10615 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.071376 11156 consensus_queue.cc:1059] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:19:00.014307 11328 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:00.037750 11277 consensus_queue.cc:1059] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 12:19:00.110561  6651 pending_rounds.cc:85] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 6
I20260709 12:19:00.111049  6651 pending_rounds.cc:107] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 12:19:00.111302  6651 raft_consensus.cc:2889] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:00.111773 11345 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:00.113096  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:00.114157 11345 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:00.114372  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:00.114825 10996 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:00.052438 11228 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:00.092471 11162 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:00.116932 11228 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:00.078184 11074 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:00.050879 11023 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:00.118295 11023 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:00.118867 11023 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:00.118124 10996 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:00.090757  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:00.116797  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 59771.9us
I20260709 12:19:00.106638 11123 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.122958 11123 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:00.125213 11123 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:00.123760  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:00.125892 11238 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.054996 11132 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:00.100096  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:00.126368  6657 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:19:00.126436 11238 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:00.127409  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:00.126909  6649 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:19:00.125296 11156 consensus_queue.cc:1243] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:19:00.127964 11132 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:00.118371 11162 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:00.128942  6658 pending_rounds.cc:85] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 5
I20260709 12:19:00.123533 11358 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:19:00.129266  6658 pending_rounds.cc:107] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 12.6
I20260709 12:19:00.129525  6658 raft_consensus.cc:2889] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:00.131686  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:00.132517 11074 raft_consensus.cc:697] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:00.132704  6622 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:00.005793 11015 consensus_queue.cc:237] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:00.141430  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 72831.6us
I20260709 12:19:00.105635  6655 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:00.128182  6657 raft_consensus.cc:2393] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 12: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:00.135388  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:00.137287  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:00.147198  6649 raft_consensus.cc:2393] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 12: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:00.138417  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:00.132825 11156 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:00.098531  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:19:00.149194  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 59761.4us
I20260709 12:19:00.146394  6579 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:19:00.150274  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:00.150838  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:00.151716  6581 raft_consensus.cc:2393] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 12: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:00.152302  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:00.156872  6579 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:00.160678  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:00.160821  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:00.092160 10877 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.164044  6550 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:00.162397  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:00.162644  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:00.164661  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:00.164945  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:00.165494  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:00.165614  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:00.168642  6580 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:19:00.169150  6583 raft_consensus.cc:2393] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 14: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:00.168396  6581 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:00.165740  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:00.168498  6578 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:19:00.169657  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:00.170374  6580 raft_consensus.cc:2393] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 14: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:00.171137  6582 raft_consensus.cc:2393] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 12: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:00.110622 11328 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:00.171921  6550 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:00.181103  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:00.162750 11162 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:00.190812  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:00.163300 11358 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:00.192677  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:00.151089 11006 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:00.153322 11156 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:00.197553  6653 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:19:00.234221  6655 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 12:19:00.233589  6575 raft_consensus.cc:1240] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:19:00.244038  6655 pending_rounds.cc:85] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 6
I20260709 12:19:00.244517  6655 pending_rounds.cc:107] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:19:00.165141 11132 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:00.242758  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:00.247185  6655 raft_consensus.cc:2889] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:00.243036  6579 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:19:00.245720  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:00.249547  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:00.250967  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:00.241366  6506 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:19:00.256743  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:00.257369  6503 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:19:00.258271  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:00.251811  6500 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:19:00.259374  6505 raft_consensus.cc:1240] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:00.260488  6657 raft_consensus.cc:3055] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:00.260944  6657 raft_consensus.cc:740] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:00.182106 11238 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:00.196763 11348 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.153183 10990 consensus_queue.cc:1048] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.089s
I20260709 12:19:00.246796  6501 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:19:00.278191  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:00.278504  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:00.181489 10963 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:00.138975 10615 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:00.163090 11316 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.084s
I20260709 12:19:00.178181 11341 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:00.264520  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:00.286720  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:00.267369  6504 raft_consensus.cc:1240] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:00.286974  6653 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:19:00.272354  6581 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:19:00.245692 11156 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:00.178540 10877 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:00.285868  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:00.287164  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:00.292718  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 110901us
I20260709 12:19:00.287562  6500 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:19:00.262809  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:00.257826 11132 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:00.279424  6501 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:19:00.294217  6506 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:19:00.293552  6505 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:19:00.297116  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:00.300864  6503 raft_consensus.cc:1240] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:00.292219  6657 consensus_queue.cc:260] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:00.242764 10999 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:00.288590 11242 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:00.314291  6509 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:19:00.295147 10963 raft_consensus.cc:697] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:00.298429 11132 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:00.315165  6657 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:00.233865 11074 consensus_queue.cc:237] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:00.267591 11358 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:00.321694  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:00.323622 10615 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:00.323459 11332 consensus_queue.cc:1059] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.079s
I20260709 12:19:00.298162 11348 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:00.337023  6650 pending_rounds.cc:85] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 4
I20260709 12:19:00.337328  6650 pending_rounds.cc:107] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 13.5
I20260709 12:19:00.337544  6650 raft_consensus.cc:2889] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:00.324945 10963 consensus_queue.cc:237] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:00.299695 11341 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:00.334194  6653 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:00.345466  6508 raft_consensus.cc:1275] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:19:00.314785 11242 raft_consensus.cc:697] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:00.318061 11112 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.345769 11184 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:00.351881  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 180032us
W20260709 12:19:00.355309  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 174670us
W20260709 12:19:00.360774  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 183188us
I20260709 12:19:00.364691  6550 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:00.361874  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:00.332248 11343 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.371341  6502 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:19:00.373246  6657 raft_consensus.cc:1275] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:19:00.385473  6657 pending_rounds.cc:85] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 5
I20260709 12:19:00.385823  6657 pending_rounds.cc:107] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:19:00.386132  6657 raft_consensus.cc:2889] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:00.383985  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:19:00.382483  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 44039.9us
I20260709 12:19:00.372526 10996 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 12:19:00.328840 11220 consensus_queue.cc:1059] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
W20260709 12:19:00.362185 11339 consensus_meta.cc:287] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.535s	user 0.008s	sys 0.004s
I20260709 12:19:00.411536  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:00.395949  6649 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:00.411985  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:00.424492  6508 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:00.424527  6653 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
W20260709 12:19:00.440426  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 70625.3us
I20260709 12:19:00.441738  7238 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:00.333251 11050 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:00.428048 11358 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:00.453413  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 131912us
I20260709 12:19:00.423302 11340 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:00.481678  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:00.431689 11339 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:19:00.437698 11012 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 12:19:00.411657 11369 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:00.493841  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:00.484814  6579 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 11.
I20260709 12:19:00.382956 11112 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:00.506541  7269 raft_consensus.cc:2393] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 16: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:00.497809  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:00.507283  6657 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:00.443853 10962 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 12:19:00.518289  6509 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:19:00.410154 11343 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:00.510671 11316 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:00.503729 11340 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:00.572399  6579 raft_consensus.cc:1275] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:19:00.573882  6473 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:00.564839  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:00.610010  6509 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:00.569336  6622 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:00.589895  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:00.556280  6649 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:19:00.626325  6507 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
W20260709 12:19:00.641158  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 97355.1us
I20260709 12:19:00.621752  6657 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:19:00.633524  6473 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:00.650975  6473 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:00.663901  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:00.509644 11242 consensus_queue.cc:237] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:00.562192 10986 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.668272  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:00.622394 11376 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:00.585846 10999 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:00.675290  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:00.692037  6503 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:00.568984 11365 log.cc:927] Time spent T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 12:19:00.571229 11369 raft_consensus.cc:697] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:19:00.657809 11331 consensus_meta.cc:287] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.807s	user 0.008s	sys 0.005s
I20260709 12:19:00.578153 10877 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:00.672480 11184 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:00.657474 11327 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:19:00.728578  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 143003us
I20260709 12:19:00.544744 11012 consensus_queue.cc:1048] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.141s
I20260709 12:19:00.665863 11277 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:19:00.683810 11364 log.cc:927] Time spent T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.272s	user 0.001s	sys 0.000s
W20260709 12:19:00.739032  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 256378us
W20260709 12:19:00.738528  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 322832us
I20260709 12:19:00.647509 11358 consensus_queue.cc:1059] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 12:19:00.646459 11220 raft_consensus.cc:2764] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:19:00.747037  6622 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:00.752113  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:00.752434  6657 raft_consensus.cc:1275] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:19:00.752811  6506 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:00.754381  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:00.754940  6507 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:00.743541  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
W20260709 12:19:00.740536 11380 log.cc:927] Time spent T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 12:19:00.671303 11015 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:00.753602  6550 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:00.739871 11331 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:00.776588  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:00.780642  6505 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:00.742349 11348 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:00.791589  6503 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:19:00.793982  6580 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:00.775774  6508 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
W20260709 12:19:00.797152  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 43644.6us
I20260709 12:19:00.797354  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:00.797925  6550 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:00.793944  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:00.800653  6507 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:00.801424  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:00.755089 11184 raft_consensus.cc:697] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 13 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:19:00.803576  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 60101.9us
I20260709 12:19:00.728042 11376 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:00.813172  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 140936us
I20260709 12:19:00.813859  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:00.728638 10986 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:00.794646 11015 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:00.886550 11369 consensus_queue.cc:237] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.5, Last appended by leader: 5, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:00.891515  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:00.814738  6576 raft_consensus.cc:2368] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 15. Current term is 16.
I20260709 12:19:00.893324  6584 raft_consensus.cc:1275] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:19:00.887354  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:19:00.893936  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 87674.8us
I20260709 12:19:00.896133 11388 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:00.903940  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:00.795838 11277 raft_consensus.cc:697] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:00.905627 11277 consensus_queue.cc:237] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:00.907618  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:00.906385 11184 consensus_queue.cc:237] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:00.912268  7271 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:00.913834 11331 consensus_queue.cc:1059] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:00.915938  6654 raft_consensus.cc:1240] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:00.792958 11083 raft_consensus.cc:3055] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:00.898028  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:00.918777  6655 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:19:00.910663  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:00.921334  6502 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:00.940322  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:19:00.895094  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:00.781497 11341 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:00.814025  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:19:01.031155 11162 consensus_meta.cc:287] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.787s	user 0.010s	sys 0.000s
I20260709 12:19:00.907829 11382 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:01.032814  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:01.033677  6503 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:00.802971 11391 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:19:01.044194  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 103230us
I20260709 12:19:01.045163  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:01.018851  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 276035us
I20260709 12:19:00.903632 11388 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:19:01.027457  6578 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.003s	sys 0.008s
I20260709 12:19:01.031069  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:01.099556  6508 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:01.021152  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:01.023437  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:01.101186  6649 raft_consensus.cc:2410] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:19:01.031809 11162 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:01.102262  6509 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:19:01.102696  6502 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:19:01.103709  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:01.036783  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:01.104316  6502 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
I20260709 12:19:01.103681  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:01.038892  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:01.040550  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:01.102344  6578 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.004s	sys 0.016s
I20260709 12:19:00.939206 11083 raft_consensus.cc:740] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:01.057221  6506 raft_consensus.cc:1275] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:19:01.073002  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:01.105572  6576 raft_consensus.cc:2410] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:19:00.916610 11316 consensus_queue.cc:1048] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.132s
I20260709 12:19:01.086827 11340 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:01.088599 10986 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:01.064415 11388 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:01.090683 11360 log.cc:927] Time spent T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.907s	user 0.000s	sys 0.003s
I20260709 12:19:01.091027 11220 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:01.043803 11112 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:01.097007  6475 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:01.035552 11382 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:01.099445 11228 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:01.099519  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:01.138680  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:01.139436  6504 raft_consensus.cc:2393] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 16: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:00.893329 11373 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:01.104715  6507 raft_consensus.cc:2410] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 12:19:01.037513  6621 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.004s	sys 0.008s
W20260709 12:19:01.143149  6621 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.005s	sys 0.009s
W20260709 12:19:01.143515  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:01.143828  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:01.144125  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:01.144544  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:01.144838  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:01.145130  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:01.145406  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:01.101930  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:01.057163 11343 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:01.109879  6503 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
I20260709 12:19:01.111629  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:01.149312  6582 raft_consensus.cc:2393] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 16: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:01.107213 11341 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.117527  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:01.121721  6655 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:19:01.121923  6548 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:01.097308 10877 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:19:01.110522 11332 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.018s	sys 0.012s
I20260709 12:19:01.097820 11348 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:01.058950  6622 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:01.137389  6508 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
W20260709 12:19:01.121690 11132 consensus_meta.cc:287] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.796s	user 0.011s	sys 0.000s
W20260709 12:19:01.147769  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 50839.7us
I20260709 12:19:01.173691  6475 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:01.174114  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 52311.4us
I20260709 12:19:01.149374  6502 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 12.
I20260709 12:19:01.169566  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:01.175158  6655 raft_consensus.cc:2368] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 15. Current term is 16.
W20260709 12:19:01.175246  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 116417us
I20260709 12:19:01.175977  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:01.176200  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:01.176755 11394 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:01.177377  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:01.178231  6655 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:19:01.170655  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:01.191005  6656 raft_consensus.cc:1275] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:19:01.181525  6475 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:01.193298  6475 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:01.165835 11332 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.018s	sys 0.012s
I20260709 12:19:01.213466  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:01.141748 10738 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:01.139802 11382 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:01.150847 11112 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:01.226577  6475 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:01.191625  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:01.168533 10877 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:01.165115 11050 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:01.255089  6475 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:19:01.231918 11023 consensus_meta.cc:287] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 1.113s	user 0.006s	sys 0.006s
I20260709 12:19:01.218103 11184 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:01.226904 11340 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:01.148859 11228 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:01.151165 11389 consensus_queue.cc:1048] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.097s
I20260709 12:19:01.217366 11012 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:19:01.263501  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 71951.6us
I20260709 12:19:01.231139 11331 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:01.258623 11339 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:01.259650 10877 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:01.253926 11379 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:01.181237 11348 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:01.261538 11358 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:01.265072 11394 raft_consensus.cc:2764] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 12:19:01.295212 11006 consensus_meta.cc:287] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 1.030s	user 0.011s	sys 0.001s
I20260709 12:19:01.252403 11373 consensus_queue.cc:1059] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 12:19:01.289387 10986 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 12:19:01.296478 11373 raft_consensus.cc:3055] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:01.296783  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:01.297111 11348 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:01.298640  6622 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:01.277328 11012 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:01.300447 11012 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:01.300632 11384 log.cc:927] Time spent T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:19:01.302150 11162 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:01.302747 11162 raft_consensus.cc:697] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:01.285192 11369 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.303450 11369 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:01.265434 11023 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:01.304880 11358 raft_consensus.cc:697] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:01.305871 11369 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:01.306702 11023 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:01.307240  6577 raft_consensus.cc:2393] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 13: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:19:01.308228  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 50093.2us
I20260709 12:19:01.309003  6475 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:01.189725 11132 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:01.263736 11184 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:01.236754 11341 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:01.263190 10738 consensus_queue.cc:1059] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:19:01.314658  6475 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:01.296931 11373 raft_consensus.cc:740] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:01.345978  7266 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:01.218360 11083 consensus_queue.cc:260] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:01.303762 11162 consensus_queue.cc:237] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:01.272521 11112 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 12:19:01.285449 11015 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:01.315805 11184 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:01.355144  6475 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:19:01.359234 11405 log.cc:927] Time spent T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:19:01.338449 11394 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:19:01.362835 11130 consensus_meta.cc:287] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 1.296s	user 0.015s	sys 0.000s
I20260709 12:19:01.266712 11391 consensus_queue.cc:1059] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 12:19:01.334023 11348 raft_consensus.cc:2764] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 12:19:01.355496 11385 log.cc:927] Time spent T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.321s	user 0.000s	sys 0.002s
W20260709 12:19:01.375622  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 112566us
I20260709 12:19:01.348313 10963 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:01.364811 11331 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:01.381642  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2171) took 1239 ms. Trace:
I20260709 12:19:01.314666 11379 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:01.379541 11348 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:01.384790  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:01.385149 11348 raft_consensus.cc:697] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:01.386261 11348 consensus_queue.cc:237] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:01.387234  6622 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:01.388110 11343 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:01.388706 11343 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:01.389158 11343 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:01.305938 11358 consensus_queue.cc:237] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:01.391551  6548 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:01.390539  6578 rpcz_store.cc:276] 0709 12:19:00.142665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.170006 (+ 27341us) service_pool.cc:224] Handling call
0709 12:19:00.192136 (+ 22130us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:19:00.264800 (+ 72664us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:00.264849 (+    49us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:19:00.294298 (+ 29449us) log.cc:844] Serialized 78 byte log entry
0709 12:19:00.417159 (+122861us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:19:01.105462 (+688303us) log.cc:844] Serialized 14 byte log entry
0709 12:19:01.301762 (+196300us) spinlock_profiling.cc:243] Waited 6.18 ms on lock 0x7b48000030d8. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287adb2aa1 00007f287ada9b37 00007f287ada99ca 00007f287aea126f 00007f287aea0f37 00007f287aeb555c 00007f287aeb5508 00007f287aeb5488 00007f287aeb5443 00007f287aeb46ec 00007f2879ddea96
0709 12:19:01.375872 (+ 74110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.375999 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:01.376065 (+    66us) raft_consensus.cc:1685] finished
0709 12:19:01.381140 (+  5075us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:01.381580 (+   440us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15262,"prepare.queue_time_us":686044,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":65785,"raft.run_cpu_time_us":5318,"raft.run_wall_time_us":1015029,"spinlock_wait_cycles":14208896,"thread_start_us":1011320,"threads_started":3,"wal-append.queue_time_us":190784}
I20260709 12:19:01.395045  6548 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:01.397260  6505 consensus_meta.cc:287] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.616s	user 0.009s	sys 0.001s
I20260709 12:19:01.399430 11425 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 12:19:01.400475 11420 consensus_queue.cc:1048] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:19:01.405011 11228 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:01.408370  6505 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:19:01.403796 10877 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:01.408108 11391 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:01.415534  6548 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:01.391369 11394 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:19:01.377595 11130 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:01.429360 11427 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:01.409720 11220 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.430320  6548 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:01.434041 11132 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:01.435667 11341 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:01.435755 11382 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:01.391505 10999 consensus_queue.cc:1048] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 12:19:01.444348 11421 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:01.449298  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:01.450430 11012 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:01.450954 11379 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:01.451360  6622 leader_election.cc:400] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 12. Current term is 13.
W20260709 12:19:01.452833  7269 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38562 (request call id 795) took 1428 ms (client timeout 1649 ms). Trace:
I20260709 12:19:01.453217 11427 raft_consensus.cc:697] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:01.461617 11418 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.462780 11418 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:01.462595 11427 consensus_queue.cc:237] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:01.460875  7269 rpcz_store.cc:269] 0709 12:19:00.024724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.024856 (+   132us) service_pool.cc:224] Handling call
0709 12:19:01.452654 (+1427798us) spinlock_profiling.cc:243] Waited 480 ms on lock 0x7b540050c3d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:01.452775 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1105086336}
I20260709 12:19:01.458292  6581 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:19:01.466030  6581 raft_consensus.cc:606] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:01.391835 11373 consensus_queue.cc:260] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:01.469939  7271 raft_consensus.cc:2368] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 11. Current term is 12.
I20260709 12:19:01.456988 11391 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:01.474452  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:01.422260 11388 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:01.475849 11015 consensus_queue.cc:1059] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 12:19:01.443709 11220 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:01.474951 11391 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:01.479853 11391 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:01.481674  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:01.485198  6656 raft_consensus.cc:2393] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 16: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:01.500924  6575 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:19:01.501911  6575 raft_consensus.cc:606] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:01.506595  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 63705.8us
I20260709 12:19:01.483809 11220 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:01.494072 11156 consensus_meta.cc:287] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 1.194s	user 0.010s	sys 0.004s
I20260709 12:19:01.486104 11356 raft_consensus.cc:3055] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:01.484390 11343 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:01.518165 11156 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:01.527022 10615 raft_consensus.cc:3055] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:01.535679 10990 raft_consensus.cc:3055] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:01.540810 11015 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:01.543011 11361 log.cc:927] Time spent T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8: Append to log took a long time: real 1.359s	user 0.006s	sys 0.000s
I20260709 12:19:01.543421 11382 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:01.544216 10877 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:01.549441 11012 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:01.549381  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:01.553957  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:01.586509  6579 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:01.494511 11242 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:19:01.580475  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 71283.9us
I20260709 12:19:01.545861 11184 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:01.525556 11130 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:01.579573 10990 raft_consensus.cc:740] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:19:01.591554  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:01.565546 11356 raft_consensus.cc:740] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:01.598241  6581 raft_consensus.cc:1240] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:01.581195 10615 raft_consensus.cc:740] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:01.599781  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:01.599702  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:01.601933  6651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2105) took 1494 ms. Trace:
I20260709 12:19:01.605751  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:01.602535  6651 rpcz_store.cc:276] 0709 12:19:00.107248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.109900 (+  2652us) service_pool.cc:224] Handling call
0709 12:19:00.110296 (+   396us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:00.112043 (+  1747us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:00.112079 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:00.131300 (+ 19221us) log.cc:844] Serialized 52 byte log entry
0709 12:19:00.262474 (+131174us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:00.354267 (+ 91793us) log.cc:844] Serialized 14 byte log entry
0709 12:19:00.354572 (+   305us) log.cc:844] Serialized 14 byte log entry
0709 12:19:00.448602 (+ 94030us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:00.448711 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:01.540264 (+1091553us) spinlock_profiling.cc:243] Waited 94.9 ms on lock 0x7b240020f978. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872c229c4 00007f2872c20acb 00007f2872ddbdc9 00007f2872ddba55 00007f2872ddafb0 00007f2875a51488 00007f2875a51be8 00007f2875a29398 00007f2875a2476a 00007f2875a23bb6
0709 12:19:01.601629 (+ 61365us) raft_consensus.cc:1685] finished
0709 12:19:01.601668 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:01.601852 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":773141,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":7138,"raft.queue_time_us":14431,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":267,"spinlock_wait_cycles":218184064,"thread_start_us":291509,"threads_started":3,"wal-append.queue_time_us":187981}
I20260709 12:19:01.611107  6575 raft_consensus.cc:2393] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 16: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:01.610314  6473 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:01.613914  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:01.615051  6583 raft_consensus.cc:2393] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 14: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:01.620535  6473 leader_election.cc:400] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 15. Current term is 16.
I20260709 12:19:01.621124  6473 leader_election.cc:403] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:19:01.621680  6579 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 12.
I20260709 12:19:01.584194 11015 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:01.627432  6622 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:01.635516 11418 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:01.616930 11335 consensus_queue.cc:1059] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 12:19:01.609686 11277 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.629014 11432 consensus_queue.cc:1048] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 12:19:01.623502 11388 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:01.592876 11242 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:01.617205 11339 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:19:01.642767  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:01.626549 11340 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 15. Current term is 16.
I20260709 12:19:01.648506 11012 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:01.657810 11012 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:19:01.661557  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 273653us
I20260709 12:19:01.662936  6475 leader_election.cc:400] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 15. Current term is 16.
I20260709 12:19:01.665246  6475 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:01.668895  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:01.670733 11156 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:01.656504 10990 consensus_queue.cc:260] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:01.650650 11277 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:01.661715 11012 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:01.679903  6622 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:01.683351 11345 raft_consensus.cc:3055] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:01.683872 11345 raft_consensus.cc:740] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:01.684831 11345 consensus_queue.cc:260] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:01.686602 11184 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:01.690600 11366 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:01.690721 11242 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:01.653745 10615 consensus_queue.cc:260] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:01.695240 11156 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:01.695999 11433 log.cc:927] Time spent T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:19:01.691423 11366 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:01.696895 11015 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:01.692895 11339 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:01.695999 10615 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:01.692265  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:01.701581  6651 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:01.703799  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:01.698220 11366 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:01.706548 11332 raft_consensus.cc:3055] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:01.706974 11332 raft_consensus.cc:740] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:19:01.699007 11387 log.cc:927] Time spent T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.960s	user 0.008s	sys 0.003s
W20260709 12:19:01.710042 11434 log.cc:927] Time spent T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:19:01.630141 11356 consensus_queue.cc:260] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:01.710709  6650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2116) took 1378 ms. Trace:
I20260709 12:19:01.711489 11356 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:01.712096  7238 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.816s (SENT) [suppressed 19 similar messages]
I20260709 12:19:01.708035 11332 consensus_queue.cc:260] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:01.714902 11332 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:01.712021  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:01.711202  6650 rpcz_store.cc:276] 0709 12:19:00.332638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.336623 (+  3985us) service_pool.cc:224] Handling call
0709 12:19:00.336884 (+   261us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:00.362823 (+ 25939us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:00.362853 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:00.363507 (+   654us) log.cc:844] Serialized 52 byte log entry
0709 12:19:00.512000 (+148493us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:19:00.512055 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:00.512121 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:01.698423 (+1186302us) spinlock_profiling.cc:243] Waited 53.9 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:01.710372 (+ 11949us) raft_consensus.cc:1685] finished
0709 12:19:01.710394 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:01.710647 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7032,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":8258,"spinlock_wait_cycles":123884416,"thread_start_us":70169,"threads_started":1,"wal-append.queue_time_us":207436}
I20260709 12:19:01.720495  6651 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 12.
I20260709 12:19:01.677151 11389 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:01.722334  7240 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:01.724291 11432 consensus_queue.cc:1048] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:19:01.732775  6550 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:01.725525  6475 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:01.697659 11339 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:01.701215 10990 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:19:01.755123  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:01.729980 11327 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:01.705235 11343 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:01.745904 11156 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:19:01.732082  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:01.739077  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:01.756997 11156 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:01.757642  6656 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:01.758736 11428 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:01.739147  6650 raft_consensus.cc:1240] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:01.758860 11038 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:01.759508 11156 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:01.742893  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:01.760308  6650 raft_consensus.cc:1275] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:19:01.758903  6656 raft_consensus.cc:1240] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:19:01.741510  6651 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:01.747006  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:01.769294  6579 raft_consensus.cc:1275] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:19:01.748220  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:01.770958  7273 raft_consensus.cc:1240] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:01.772274  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:01.748883  6653 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:19:01.750535  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:01.775518  6658 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
W20260709 12:19:01.775786  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 95936.3us
I20260709 12:19:01.751830 11376 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:01.756605 11327 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:01.759274 11428 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:01.773443  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:01.776651  6622 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:01.777527  7271 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:19:01.746381  6550 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:01.699204 11391 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:01.800669  6651 raft_consensus.cc:1275] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 14 index: 6. (index mismatch)
I20260709 12:19:01.801137  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:01.800455 11083 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:01.803619  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:01.774982 11368 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:01.804155  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:01.839033  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:01.774396  7270 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:19:01.806236  7266 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:01.775132 11339 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:01.847041  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 66964.5us
I20260709 12:19:01.783488  6655 raft_consensus.cc:1275] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:19:01.782687  6548 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:19:01.828728  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 51962.3us
I20260709 12:19:01.860760  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:01.861316  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:01.774152 11421 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:01.836845  7272 raft_consensus.cc:2393] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 12: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:01.781940  6474 leader_election.cc:400] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 11. Current term is 12.
I20260709 12:19:02.068341  6474 leader_election.cc:403] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:19:02.069103  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 287279us
I20260709 12:19:01.788013  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:01.859208  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:02.071955  7269 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:19:01.807889 11316 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:02.007483 11437 log.cc:927] Time spent T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.281s	user 0.000s	sys 0.001s
I20260709 12:19:02.096851  6651 raft_consensus.cc:1275] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:19:01.978133  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:01.801020 11277 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:02.057335  6583 raft_consensus.cc:1240] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:19:01.860039 11368 raft_consensus.cc:697] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 12 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
W20260709 12:19:02.058534  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:01.859254 11156 raft_consensus.cc:2764] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:19:01.747527 11389 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:02.041939  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 329825us
I20260709 12:19:02.072039  6656 raft_consensus.cc:2393] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 14: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:02.064091 11220 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:02.081302 11112 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:19:02.089061 11038 consensus_queue.cc:1059] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:19:02.090024  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:02.125231  6500 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 13.
I20260709 12:19:02.125523  6505 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:58944
I20260709 12:19:02.125789  6500 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41304 (request call id 562) took 1061 ms. Trace:
I20260709 12:19:02.092681  6657 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 14.
W20260709 12:19:02.093572  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 316972us
W20260709 12:19:02.095096 11384 log.cc:927] Time spent T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.247s	user 0.004s	sys 0.006s
I20260709 12:19:02.115828  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:02.114444  7238 leader_election.cc:336] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.811s (SENT)
I20260709 12:19:02.126127  6500 rpcz_store.cc:276] 0709 12:19:01.064601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.064884 (+   283us) service_pool.cc:224] Handling call
0709 12:19:02.089806 (+1024922us) spinlock_profiling.cc:243] Waited 3.18 ms on lock 0x7b5800002a98. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:19:02.125734 (+ 35928us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7305088}
I20260709 12:19:02.125998  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:02.132566  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:02.133369  6501 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:02.126179  6505 raft_consensus.cc:606] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:02.134101  6505 raft_consensus.cc:614] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:02.127538  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:02.127728  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:02.135119  6508 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:02.136147  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:02.130357  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:02.136989  6508 raft_consensus.cc:2410] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:19:02.137550  6504 raft_consensus.cc:2393] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 14: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:19:02.139398  7238 leader_election.cc:336] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.855s (SENT)
I20260709 12:19:02.140159  7238 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:02.145409  6622 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:02.063314 11184 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.129205  6506 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:19:02.149346  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:02.149483  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:02.149976  6506 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:19:02.150202  6504 raft_consensus.cc:2376] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12. Re-sending same reply.
I20260709 12:19:02.151038  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:02.151305  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:02.129864  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:02.153083  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:02.153403  6475 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:02.154289  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:02.155270 10962 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.155422  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:02.155907 10962 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:02.143942 11015 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:02.157647 11015 raft_consensus.cc:697] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:02.158397 10962 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:02.158617 11015 consensus_queue.cc:237] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:02.159126 11340 consensus_queue.cc:1048] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:02.159715 11439 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:19:02.130453  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 289766us
I20260709 12:19:02.141772 11370 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
W20260709 12:19:02.162943  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.130601  6502 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:19:02.152175  6504 raft_consensus.cc:2393] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 13: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:02.142021 11132 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:19:02.131111  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 348468us
I20260709 12:19:02.165499  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:02.167713  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 421372us
I20260709 12:19:02.124126 10738 raft_consensus.cc:3055] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:19:02.174746  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.156234  6503 raft_consensus.cc:2376] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12. Re-sending same reply.
I20260709 12:19:02.178853  6550 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:02.180348 11254 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:02.185281  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:02.193543  6508 raft_consensus.cc:2376] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14. Re-sending same reply.
I20260709 12:19:02.186671  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:02.195663  6504 raft_consensus.cc:2376] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16. Re-sending same reply.
I20260709 12:19:02.101787 11316 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:02.207141  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:02.208925  6502 raft_consensus.cc:2376] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14. Re-sending same reply.
I20260709 12:19:02.209157  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:02.210007  6500 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:19:02.213050  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:02.219275  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:02.220813  6504 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:19:02.131790 11220 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:02.230613  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:02.181494 11409 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:19:02.240137  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 102149us
I20260709 12:19:02.197150 11156 raft_consensus.cc:2764] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:19:02.236435  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:02.248284  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:02.182727 10738 raft_consensus.cc:740] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:02.186347 11428 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:02.188180 11389 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:02.211392 11439 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:02.215250 11074 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:19:02.273862  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.279748  6622 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:02.294205  6549 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:02.298682  6505 raft_consensus.cc:1275] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:19:02.298601  6549 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:02.186384 11327 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:02.195844 11184 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:02.278954 11388 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:19:02.300441 11332 consensus_meta.cc:287] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.577s	user 0.013s	sys 0.000s
I20260709 12:19:02.195585 11012 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:02.296496 11439 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:02.297061 11331 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:02.325654 11328 raft_consensus.cc:993] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e: : Instructing follower f2b06aebe97249a4877d798b8118ece4 to start an election
I20260709 12:19:02.305101 11242 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:02.335896  6500 raft_consensus.cc:1275] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 14 index: 6. (index mismatch)
I20260709 12:19:02.343652 11345 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:02.357177 11446 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:19:02.375319 11316 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:02.376909 11391 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:02.314616 11156 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:19:02.387953 11442 log.cc:927] Time spent T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:19:02.320467 11388 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:02.272986 11368 consensus_queue.cc:237] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:02.395398  6503 raft_consensus.cc:2368] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11.
I20260709 12:19:02.396780 11220 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:02.305573 11409 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:02.446069  6500 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:02.424783 11242 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:02.430954 11328 raft_consensus.cc:1081] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Signalling peer f2b06aebe97249a4877d798b8118ece4 to start an election
I20260709 12:19:02.437036 11388 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:02.436429 11112 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 11. Current term is 12.
W20260709 12:19:02.405244  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 225838us
I20260709 12:19:02.291210 11370 consensus_queue.cc:1059] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.100s
W20260709 12:19:02.418462  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 101152us
I20260709 12:19:02.507867  6550 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:02.421792  6577 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 2164) took 2279 ms (client timeout 2769 ms). Trace:
I20260709 12:19:02.425782  6622 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:02.431737  6649 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:02.509791  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2691) took 2926 ms. Trace:
W20260709 12:19:02.508826  6577 rpcz_store.cc:269] 0709 12:19:00.136221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:00.136357 (+   136us) service_pool.cc:224] Handling call
0709 12:19:02.415310 (+2278953us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b540034dad4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:02.415475 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2634194688}
I20260709 12:19:02.403882  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:02.424232 11331 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:02.485435  6548 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:02.446338 11409 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:02.465260 11242 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:02.501827 11339 consensus_meta.cc:287] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.642s	user 0.010s	sys 0.001s
I20260709 12:19:02.503859  6501 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 14.
I20260709 12:19:02.525398  6507 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 12.
I20260709 12:19:02.528672  6550 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:02.531509  6501 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:19:02.532965  6501 raft_consensus.cc:1240] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:02.510385  6652 rpcz_store.cc:276] 0709 12:18:59.491705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.498674 (+  6969us) service_pool.cc:224] Handling call
0709 12:18:59.499058 (+   384us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:18:59.999273 (+500215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:18:59.999321 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:00.014314 (+ 14993us) log.cc:844] Serialized 52 byte log entry
0709 12:19:00.148760 (+134446us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:00.440057 (+291297us) log.cc:844] Serialized 14 byte log entry
0709 12:19:00.445116 (+  5059us) log.cc:844] Serialized 14 byte log entry
0709 12:19:00.643156 (+198040us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:01.090106 (+446950us) spinlock_profiling.cc:243] Waited 2.3 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:02.417262 (+1327156us) spinlock_profiling.cc:243] Waited 500 ms on lock 0x7b540019dcd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:02.417346 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:02.417427 (+    81us) raft_consensus.cc:1685] finished
0709 12:19:02.417448 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:02.417631 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4394,"prepare.queue_time_us":669033,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":70,"raft.queue_time_us":110110,"raft.run_cpu_time_us":378,"raft.run_wall_time_us":24936,"spinlock_wait_cycles":1154263296,"thread_start_us":601035,"threads_started":3,"wal-append.queue_time_us":153722,"wal-append.run_cpu_time_us":8003,"wal-append.run_wall_time_us":2242818}
W20260709 12:19:02.536361  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 51050.5us
W20260709 12:19:02.513263 11445 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.211s	user 0.002s	sys 0.002s
I20260709 12:19:02.537192  6548 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:02.517238 11409 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:02.543766 11397 consensus_queue.cc:1059] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 12:19:02.538919  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:02.520684 11328 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:19:02.575114  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.578027  7240 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:02.516597 11327 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:02.579226  6649 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:19:02.582649  6656 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:02.532552 11156 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:02.589059 11316 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:02.595662  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:02.600651  6657 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
W20260709 12:19:02.597877  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 60777.2us
W20260709 12:19:02.597949  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 174946us
I20260709 12:19:02.599328  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:02.548946 11112 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:02.538347 11220 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:02.587811 11328 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:02.604387  6548 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:02.531822 10999 consensus_queue.cc:1059] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 12:19:02.549104 11023 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:02.628988 11023 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:02.629468 11023 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:02.627337 11112 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:02.621169 11370 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:19:02.620488  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.641855 11397 raft_consensus.cc:3055] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:02.627699 11220 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:02.643563 11370 raft_consensus.cc:697] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:02.644708 11370 consensus_queue.cc:237] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:02.599928 11391 raft_consensus.cc:3055] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:02.646003 11391 raft_consensus.cc:740] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:02.646836 11391 consensus_queue.cc:260] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:02.648515 11395 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:02.649005 11395 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:02.649428 11395 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:02.535321 10990 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 12:19:02.536036 10986 consensus_queue.cc:1048] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 12:19:02.579897 11427 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:02.532799 11339 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:02.642424 11397 raft_consensus.cc:740] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:02.573851 11418 consensus_queue.cc:1048] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 12:19:02.604321 11316 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:02.626507  6548 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:02.664296  6657 raft_consensus.cc:1240] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:02.627946 11328 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:02.669787  6622 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:19:02.671686  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 45238.9us
I20260709 12:19:02.668701  7238 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:02.675309  6548 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:02.676023  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:19:02.685171  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.659751 11391 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:02.688256  6550 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:02.691377  6509 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:02.695449  6550 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:02.669728 11418 consensus_queue.cc:1243] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer f2b06aebe97249a4877d798b8118ece4 log is divergent from this leader: its last log entry 11.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:19:02.668560 11316 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:02.669942 11439 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:02.682487 11254 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:02.687557 11369 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:02.681205 11331 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:02.662818 11367 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:19:02.696238 11132 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:02.671036 11332 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:19:02.713611  6548 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:02.696938  6657 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:58980
I20260709 12:19:02.719044  6657 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:02.719480  6657 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:02.699992 10999 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:02.697983 11427 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:02.712203 11083 raft_consensus.cc:3055] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:19:02.724872  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.715116 11132 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:02.716470 11458 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:02.718446 11112 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:02.723889 11369 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:02.719290 11184 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:02.723591 10999 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:02.725667 11083 raft_consensus.cc:740] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:02.753088 11083 consensus_queue.cc:260] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:02.743417  6657 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:02.726223 11012 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:02.758774  6657 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:02.733363 11132 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:02.772544  7273 raft_consensus.cc:1240] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 12 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:19:02.746802 10999 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:02.794121 11454 log.cc:927] Time spent T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:19:02.794373 11339 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:02.795588 11457 log.cc:927] Time spent T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:19:02.707329 11397 consensus_queue.cc:260] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:02.799425 11397 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:02.803692 11409 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:02.811085 11450 log.cc:927] Time spent T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.487s	user 0.000s	sys 0.003s
W20260709 12:19:02.817221  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 147502us
I20260709 12:19:02.817788 11343 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:02.758949 11023 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:02.825060  6650 raft_consensus.cc:1240] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 8. Current term is 9. Ops: []
W20260709 12:19:02.827039 11449 log.cc:927] Time spent T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.454s	user 0.004s	sys 0.000s
I20260709 12:19:02.827558 11345 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:02.829039  6656 pending_rounds.cc:85] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 4
I20260709 12:19:02.910948  6656 pending_rounds.cc:107] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 11.5
I20260709 12:19:02.911374  6656 raft_consensus.cc:2889] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:02.829527 11341 raft_consensus.cc:3055] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:02.834667 11220 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:02.841277 10738 consensus_queue.cc:260] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:02.829063 11444 log.cc:927] Time spent T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.506s	user 0.002s	sys 0.000s
I20260709 12:19:02.853607  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:02.859592  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 797958us
I20260709 12:19:02.861553  7273 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38562 (request call id 806) took 1081 ms. Trace:
W20260709 12:19:02.862905 11459 log.cc:927] Time spent T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 12:19:02.865036  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.907083 11331 consensus_queue.cc:1048] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 12:19:02.867036  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58980 (request call id 2732) took 1124 ms. Trace:
I20260709 12:19:02.969245  6649 rpcz_store.cc:276] 0709 12:19:01.742807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.743487 (+   680us) service_pool.cc:224] Handling call
0709 12:19:02.866852 (+1123365us) spinlock_profiling.cc:243] Waited 603 ms on lock 0x7b5400373554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:02.866973 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1387685120}
I20260709 12:19:02.880030  6658 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
W20260709 12:19:02.885783  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:02.851724 11132 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:02.887094 11388 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:02.909258  6622 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:02.972460  6550 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:19:02.979063  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 69875.5us
I20260709 12:19:02.966856  6473 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:02.964960  7273 rpcz_store.cc:276] 0709 12:19:01.779899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.780054 (+   155us) service_pool.cc:224] Handling call
0709 12:19:01.780436 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:02.773141 (+992705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.861289 (+ 88148us) spinlock_profiling.cc:243] Waited 992 ms on lock 0x7b54005ddc54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:02.861479 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2280869632}
I20260709 12:19:02.919986 11341 raft_consensus.cc:740] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:02.961253 11327 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:02.952646 11184 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:02.920188 11332 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:02.994966  6473 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:03.049083  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 355781us
I20260709 12:19:03.048995  6650 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:19:03.049331  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 326064us
I20260709 12:19:03.054236  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:02.960814 11398 consensus_queue.cc:1059] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 12:19:02.966985 11397 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.084509  6653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2132) took 1288 ms. Trace:
W20260709 12:19:03.084635  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 89786.8us
I20260709 12:19:03.059187  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:03.086637  6653 rpcz_store.cc:276] 0709 12:19:01.796551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.861683 (+ 65132us) service_pool.cc:224] Handling call
0709 12:19:01.861964 (+   281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:02.062178 (+200214us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:19:02.062230 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:02.062934 (+   704us) log.cc:844] Serialized 26 byte log entry
0709 12:19:02.196252 (+133318us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:19:02.196314 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.196425 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:02.810395 (+613970us) spinlock_profiling.cc:243] Waited 76.8 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:03.054427 (+244032us) raft_consensus.cc:1685] finished
0709 12:19:03.054449 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:03.084450 (+ 30001us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":199625,"raft.queue_time_us":225968,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":122,"spinlock_wait_cycles":176612224,"thread_start_us":83498,"threads_started":1,"wal-append.queue_time_us":207804}
I20260709 12:19:02.952113 11156 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.097s
I20260709 12:19:02.908478 11316 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:03.089103  6653 raft_consensus.cc:1275] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:19:02.952405 11242 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:03.092059  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:03.092638  6650 raft_consensus.cc:1275] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:19:03.099290  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:03.100023  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.100203  6650 raft_consensus.cc:2393] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 13: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:03.048413 11012 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:03.106914  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:02.983512 11343 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.101910  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:03.105943  6654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 747) took 1619 ms. Trace:
W20260709 12:19:03.089805 11444 log.cc:927] Time spent T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 12:19:02.983795 11112 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:19:03.103046  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 51817.5us
I20260709 12:19:03.108112  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:03.119145  6650 raft_consensus.cc:2393] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 9: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:03.111374  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:03.112252 11389 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:03.084750 11332 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:03.109476  6654 rpcz_store.cc:276] 0709 12:19:01.486395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:01.486549 (+   154us) service_pool.cc:224] Handling call
0709 12:19:01.486834 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:01.487121 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:01.487140 (+    19us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:01.489411 (+  2271us) log.cc:844] Serialized 26 byte log entry
0709 12:19:02.092223 (+602812us) spinlock_profiling.cc:243] Waited 2 ms on lock 0x7b1800361a20. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:19:02.292846 (+200623us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:02.292895 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.292977 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:02.826432 (+533455us) spinlock_profiling.cc:243] Waited 27.9 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:03.105730 (+279298us) raft_consensus.cc:1685] finished
0709 12:19:03.105754 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:03.105884 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2017,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":3099,"spinlock_wait_cycles":68673152,"thread_start_us":194153,"threads_started":1,"wal-append.queue_time_us":883463}
I20260709 12:19:03.054533 11439 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:03.091037 11277 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:03.142967  6649 raft_consensus.cc:2393] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 11: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:03.142271  6622 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:03.094391 10738 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:03.093389 10963 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:03.101123 11376 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.153978  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:19:03.161087  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 69008.8us
I20260709 12:19:03.174273  7266 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:03.173076  6548 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:03.056315 11327 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:03.168043  7273 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:19:03.171123  6650 raft_consensus.cc:1240] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:03.093216 11367 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 12:19:03.119289 11462 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:03.143245 11012 raft_consensus.cc:697] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:03.116701 11398 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:03.120491 11345 raft_consensus.cc:3055] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:03.093272 11397 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:03.193174  6548 leader_election.cc:400] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11.
I20260709 12:19:03.056537 11184 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:03.216889  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 101609us
I20260709 12:19:03.217128  6548 leader_election.cc:403] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:19:03.144732 11343 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:03.223029 11343 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:03.121394 11395 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:03.223995 11332 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:03.225510 11395 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:03.147544 11328 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:03.187623 11370 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:19:03.231943  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 259580us
I20260709 12:19:03.200554 11220 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:03.149163 11341 consensus_queue.cc:260] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:03.194243 10986 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:19:03.209903 11421 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.882s	user 0.011s	sys 0.038s
I20260709 12:19:03.200963 11345 raft_consensus.cc:740] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:03.159265 11112 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:03.190835 11376 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:03.205168 11462 raft_consensus.cc:697] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 12 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:03.191687 10963 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:03.261723 11421 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.934s	user 0.011s	sys 0.039s
W20260709 12:19:03.233743  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 89773.9us
I20260709 12:19:03.251645  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:03.251683 11220 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:03.263047  6502 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2404) took 1121 ms. Trace:
W20260709 12:19:03.261973  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 58386.5us
I20260709 12:19:03.295179  6502 rpcz_store.cc:276] 0709 12:19:02.141150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.328387 (+187237us) service_pool.cc:224] Handling call
0709 12:19:02.328755 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:02.431014 (+102259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:02.431048 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:02.431086 (+    38us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:02.431126 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:03.262736 (+831610us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b540001b654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:03.262799 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:03.262969 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":234662272}
I20260709 12:19:03.197902 11427 consensus_queue.cc:1048] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:19:03.297436  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:19:03.269663  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 47216.8us
I20260709 12:19:03.271413  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:03.250726 11409 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:03.293296  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.114825 11458 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.150s
W20260709 12:19:03.267228  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 180296us
I20260709 12:19:03.302457  6473 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:03.237010 11370 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:03.238881 11464 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 12:19:03.199712 11015 consensus_queue.cc:1048] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.081s
I20260709 12:19:03.262722 11023 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:03.309047  6473 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:03.317286  6505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2403) took 1177 ms. Trace:
I20260709 12:19:03.317675  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:03.319447  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:03.317919  6505 rpcz_store.cc:276] 0709 12:19:02.140833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.308040 (+167207us) service_pool.cc:224] Handling call
0709 12:19:02.367828 (+ 59788us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:02.428814 (+ 60986us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:02.428863 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:02.429484 (+   621us) log.cc:844] Serialized 26 byte log entry
0709 12:19:02.623672 (+194188us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:02.715628 (+ 91956us) log.cc:844] Serialized 14 byte log entry
0709 12:19:02.715881 (+   253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:02.821548 (+105667us) spinlock_profiling.cc:243] Waited 1.84 ms on lock 0x7b1800395e60. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c522 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:19:03.316931 (+495383us) spinlock_profiling.cc:243] Waited 52.1 ms on lock 0x7b540039ec54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:03.316994 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:03.317038 (+    44us) raft_consensus.cc:1685] finished
0709 12:19:03.317064 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:03.317215 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":135974,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":717,"raft.queue_time_us":1350,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":10475,"spinlock_wait_cycles":124054656,"thread_start_us":186445,"threads_started":2,"wal-append.queue_time_us":248094}
I20260709 12:19:03.320283  6577 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 9.
I20260709 12:19:03.295516 11341 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:03.300992 11388 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:03.303246 11220 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:03.315369 11339 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:03.315768 10986 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:03.348762 11398 raft_consensus.cc:697] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:03.326684 11327 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:03.322160 11409 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:03.328840 11277 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:03.270581 11475 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.074s
W20260709 12:19:03.428535  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 79470.4us
I20260709 12:19:03.327703 11345 consensus_queue.cc:260] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:03.335531 11439 consensus_queue.cc:1048] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.128s
I20260709 12:19:03.480779 11328 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:03.481016 11397 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:03.313822 11012 consensus_queue.cc:237] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:03.328994 11332 consensus_queue.cc:1059] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.109s
I20260709 12:19:03.490751  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:19:03.328512  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 186323us
I20260709 12:19:03.491906  7238 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:03.493538  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:03.344374 11389 raft_consensus.cc:697] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 14 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:03.481058 11474 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:03.475703 10999 consensus_meta.cc:287] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.689s	user 0.006s	sys 0.013s
I20260709 12:19:03.512913  7273 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:19:03.525595  7238 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:03.489512 11112 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:03.497153 11409 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:03.516264 11464 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:03.498821 11015 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.356593 11462 consensus_queue.cc:237] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:03.539821  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:03.540537  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 347515us
W20260709 12:19:03.545936 11465 log.cc:927] Time spent T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.283s	user 0.000s	sys 0.001s
I20260709 12:19:03.534921 11184 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:03.534513 11490 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.537123 11367 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:19:03.552608 11469 log.cc:927] Time spent T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.241s	user 0.000s	sys 0.001s
I20260709 12:19:03.556604 11328 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:03.568455 11331 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:03.616415 11184 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.616886 10877 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:03.618157  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:03.535357 10999 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:03.573786 11477 log.cc:927] Time spent T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.093s	user 0.000s	sys 0.002s
W20260709 12:19:03.576321 11466 log.cc:927] Time spent T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.314s	user 0.001s	sys 0.000s
I20260709 12:19:03.556797 11487 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:03.625556  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:03.625742  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:03.577066 11388 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:03.628749  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:03.630162  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:03.554821 11015 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:03.538097 11474 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:03.633859  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:03.665686  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:03.666091  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:03.666308  6579 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:19:03.666458  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:03.634143  6502 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 15. Current term is 16. Ops: [14.7-15.8]
I20260709 12:19:03.667845  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:03.668000  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:03.668866  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:03.668947  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:03.585630 11376 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:03.635959 11487 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:03.667006  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:03.673251  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:03.611086 11439 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:03.662683  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:03.663129  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:03.676065  6582 raft_consensus.cc:2368] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14.
I20260709 12:19:03.676735  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:03.670157  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:03.671129  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:03.678146  6579 raft_consensus.cc:2368] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 12. Current term is 13.
I20260709 12:19:03.679140  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:03.679914  6579 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:19:03.680781  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:03.683406  6580 raft_consensus.cc:2410] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:19:03.685329  6506 raft_consensus.cc:2393] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 13: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:03.685842  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:03.686985  6576 raft_consensus.cc:2410] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:19:03.687706  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:03.688010  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:03.688730  6507 raft_consensus.cc:2393] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 11: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:03.688876  6578 raft_consensus.cc:2410] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 12:19:03.553248 11464 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:19:03.662175 11397 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.691191  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:03.696834  6583 raft_consensus.cc:2393] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 15: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:03.698437  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:03.663815 11112 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.624065 11328 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:03.585110 11490 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:03.677104 11370 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:03.628269 11184 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:03.711206  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:03.707845  6579 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:19:03.713423  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:03.709442  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:03.713636  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:03.627333 11331 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:03.713876  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:03.675968 11487 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:03.631958 10877 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:03.730106  6578 raft_consensus.cc:2410] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:19:03.585208 11398 consensus_queue.cc:237] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:03.729122 11499 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.731624  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:03.731472 11499 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:03.732793  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:03.733867  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: true dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:03.734333  6500 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:03.734879 11491 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.735492 11491 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:03.700908 11156 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:19:03.736956 11486 log.cc:927] Time spent T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
W20260709 12:19:03.738880 11493 log.cc:927] Time spent T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 12:19:03.734244 11499 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:03.745324  6575 raft_consensus.cc:2393] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 11: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:03.745525  6503 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:19:03.626160 11389 consensus_queue.cc:237] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:03.756699  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2747) took 1028 ms. Trace:
I20260709 12:19:03.757134  6652 rpcz_store.cc:276] 0709 12:19:02.728056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.728216 (+   160us) service_pool.cc:224] Handling call
0709 12:19:02.735817 (+  7601us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:02.818755 (+ 82938us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:02.818783 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:02.819346 (+   563us) log.cc:844] Serialized 52 byte log entry
0709 12:19:03.116892 (+297546us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:19:03.116932 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:03.117000 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:03.756449 (+639449us) raft_consensus.cc:1685] finished
0709 12:19:03.756472 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:03.756632 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82539,"raft.queue_time_us":83087,"raft.run_cpu_time_us":46,"raft.run_wall_time_us":46,"thread_start_us":297200,"threads_started":1,"wal-append.queue_time_us":398310}
I20260709 12:19:03.690289 11388 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:03.725499 11496 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.760895 11373 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:03.761332 11497 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.127s
W20260709 12:19:03.762462 11470 log.cc:927] Time spent T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.309s	user 0.000s	sys 0.001s
W20260709 12:19:03.762756 11471 log.cc:927] Time spent T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.302s	user 0.000s	sys 0.001s
W20260709 12:19:03.763031 11467 log.cc:927] Time spent T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.321s	user 0.001s	sys 0.000s
I20260709 12:19:03.710502 11503 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:03.760565 11489 log.cc:927] Time spent T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:19:03.701824 11439 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:03.794567 11500 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.715806 11112 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:03.788208 11495 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:03.790372  7299 heartbeater.cc:646] Failed to heartbeat to 127.5.58.62:36381 (2 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.5.58.62:36381 timed out after 15.000s (SENT)
I20260709 12:19:03.711817 11397 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:03.736951 11156 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
W20260709 12:19:03.809958 11501 log.cc:927] Time spent T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.079s	user 0.000s	sys 0.002s
I20260709 12:19:03.815891  6502 raft_consensus.cc:2393] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 11: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:19:03.804226 10738 consensus_meta.cc:287] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.630s	user 0.008s	sys 0.003s
I20260709 12:19:03.818158  6580 raft_consensus.cc:2393] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 12: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:19:03.820868  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.834627 11132 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:03.803622 11495 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:03.841383  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:03.842123  6578 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:19:03.843024  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:03.846063 11343 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:03.846661 11343 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:03.796066 11503 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:03.858366 11503 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:03.859778  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2749) took 1041 ms. Trace:
I20260709 12:19:03.860353  6656 rpcz_store.cc:276] 0709 12:19:02.818630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.828176 (+  9546us) service_pool.cc:224] Handling call
0709 12:19:02.828752 (+   576us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:02.912243 (+ 83491us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:02.912310 (+    67us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:02.913047 (+   737us) log.cc:844] Serialized 52 byte log entry
0709 12:19:03.199122 (+286075us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:19:03.199173 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:03.199262 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:03.859501 (+660239us) raft_consensus.cc:1685] finished
0709 12:19:03.859529 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:03.859704 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2464,"raft.queue_time_us":1101,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":41469,"thread_start_us":223782,"threads_started":1,"wal-append.queue_time_us":302669}
I20260709 12:19:03.868927  6651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2751) took 1035 ms. Trace:
I20260709 12:19:03.869549  6651 rpcz_store.cc:276] 0709 12:19:02.833232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.833382 (+   150us) service_pool.cc:224] Handling call
0709 12:19:02.833686 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:02.834141 (+   455us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:19:02.834460 (+   319us) log.cc:844] Serialized 14 byte log entry
0709 12:19:03.264764 (+430304us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:03.265319 (+   555us) log.cc:844] Serialized 26 byte log entry
0709 12:19:03.290595 (+ 25276us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:03.290634 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:03.290717 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:03.868730 (+578013us) raft_consensus.cc:1685] finished
0709 12:19:03.868749 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:03.868868 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3939,"prepare.queue_time_us":151,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":26699,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":15035,"thread_start_us":429759,"threads_started":2,"wal-append.queue_time_us":432309}
I20260709 12:19:03.868945  6473 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:03.781436 11496 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:03.876012  6507 raft_consensus.cc:1275] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:19:03.877480  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:03.795236 11500 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:03.900514  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.885823  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:03.903980  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:03.904114  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:03.905164  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:03.905160  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:03.905970  6651 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:19:03.906162  6577 raft_consensus.cc:2368] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 16. Current term is 17.
I20260709 12:19:03.907084  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:03.907212  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:03.907601  6577 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:03.907639  6549 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:03.908582  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:03.909571  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:03.910333  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 192974us
I20260709 12:19:03.917742  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:03.918421  6651 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:19:03.922585  6582 raft_consensus.cc:2393] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 13: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:03.923270  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:03.924017  6578 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:19:03.831812 11220 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:03.928970  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:03.932960  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:03.930768  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:03.933344  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:03.934221  6507 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:03.939719 11083 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:03.933947  6656 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 8.
I20260709 12:19:03.946671  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:03.947376  6656 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:03.947948  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:03.948837  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:19:03.952929  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.906140 11503 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:03.953148  6506 raft_consensus.cc:2368] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 12. Current term is 13.
W20260709 12:19:03.954932  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:03.956842  6473 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:03.919523 11475 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:03.966379 11500 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:03.970403 11341 consensus_meta.cc:287] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.636s	user 0.010s	sys 0.001s
I20260709 12:19:03.972391 11508 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:04.015763  6583 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:19:04.016604  6505 raft_consensus.cc:2410] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:19:03.977730 11397 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:03.914392 11395 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:03.982009  6651 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:19:03.986526 11331 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:03.929008 10738 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:03.992714  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:03.996737  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:04.013453  6500 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:19:04.014838  6507 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:19:04.016877  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:03.957686 11478 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:19:04.028764  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 72156.8us
I20260709 12:19:04.029210  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:03.961597 11503 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:04.029358  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2754) took 1148 ms. Trace:
I20260709 12:19:04.050120  6506 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:04.048236  6583 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:03.981153 11475 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:04.053503  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:03.950835 11481 consensus_queue.cc:1059] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 12:19:04.054046  6655 rpcz_store.cc:276] 0709 12:19:02.881252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.884636 (+  3384us) service_pool.cc:224] Handling call
0709 12:19:02.884918 (+   282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:02.885156 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:02.885183 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:02.885641 (+   458us) log.cc:844] Serialized 26 byte log entry
0709 12:19:03.106097 (+220456us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:03.106137 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:03.106204 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:04.029143 (+922939us) raft_consensus.cc:1685] finished
0709 12:19:04.029166 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:04.029303 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":220088,"threads_started":1,"wal-append.queue_time_us":353991}
I20260709 12:19:04.058158  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:04.058820  6507 raft_consensus.cc:2376] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15. Re-sending same reply.
I20260709 12:19:04.059743  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:04.062173 11220 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:04.021432 11500 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:04.065104 11500 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:04.065531 11500 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:04.068183  6583 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:04.068913  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:04.072283  6504 raft_consensus.cc:2393] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 14: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:19:04.081986 11339 consensus_meta.cc:287] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.732s	user 0.000s	sys 0.012s
I20260709 12:19:04.082547 11339 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:04.056183 10738 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:04.094406  6500 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:04.056746 11343 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:04.067636 11478 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:04.099532 11478 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:04.102499  6654 raft_consensus.cc:3055] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:04.102909  6654 raft_consensus.cc:740] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:04.105907 11490 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:04.103735  6654 consensus_queue.cc:260] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:04.108345 11409 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:04.060734 11331 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 12:19:04.108819  6654 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:04.110008 11388 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:04.113401  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:04.092084  6656 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:04.118692  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:04.057518 11503 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:04.118767  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:04.067642 11475 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:04.121613  6504 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:04.120296  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:04.132252  6578 raft_consensus.cc:2368] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 12. Current term is 13.
I20260709 12:19:04.113076 11369 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.139709 11369 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:04.140609  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:04.141408  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:04.145006  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:04.145803  6578 raft_consensus.cc:2393] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 13: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:04.146562  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:04.147076  6578 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:19:04.147842  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: true dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:04.148610  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:04.148850  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:04.150069  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2408) took 2006 ms. Trace:
I20260709 12:19:04.150430  6509 rpcz_store.cc:276] 0709 12:19:02.143276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:02.405032 (+261756us) service_pool.cc:224] Handling call
0709 12:19:02.408492 (+  3460us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:02.694512 (+286020us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:04.149785 (+1455273us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b54000f3554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:04.149994 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":650080000}
I20260709 12:19:04.096241 10738 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:04.151544  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:04.166096  6509 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:19:04.167145  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:04.168069  6509 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:04.169060  6504 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 13.
I20260709 12:19:04.116894 11398 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.152081  6506 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:04.049614 11395 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:04.031062 11508 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:04.186354 11395 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:04.187088 11508 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:04.191697 11242 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:04.194561 10963 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:04.195549 10999 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:04.196873 11487 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:04.199065 11487 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:04.200666 10986 consensus_meta.cc:287] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.849s	user 0.008s	sys 0.003s
I20260709 12:19:04.201182 10986 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11.
I20260709 12:19:04.135843  6500 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:19:04.170146  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:04.170152  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:04.127916 11428 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.176733  6578 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c27974d342ea4901aaa3946871539818"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:19:04.209798  6578 raft_consensus.cc:606] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:04.210167  6578 raft_consensus.cc:614] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:04.178232  6506 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:19:04.183058 11503 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:04.211412  6501 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:19:04.163498  6507 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 11.
I20260709 12:19:04.222328  6576 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
W20260709 12:19:04.214233  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 222780us
I20260709 12:19:04.230954  6507 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:04.225481  6579 raft_consensus.cc:1275] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:19:04.226627  6475 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:04.228976  6581 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:19:04.234067  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:04.244670  6504 raft_consensus.cc:1275] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:19:04.252838  6654 raft_consensus.cc:1275] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 8. (term mismatch)
I20260709 12:19:04.248301  6578 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:04.254026  6654 pending_rounds.cc:85] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 7
I20260709 12:19:04.253571  6579 raft_consensus.cc:1275] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:19:04.254803  6654 pending_rounds.cc:107] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 12:19:04.255260  6654 raft_consensus.cc:2889] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:04.256964  6579 raft_consensus.cc:1240] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:04.257181  6578 raft_consensus.cc:1275] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:19:04.258914  6579 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
W20260709 12:19:04.259831  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 193632us
I20260709 12:19:04.276598  6509 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:19:04.246533 11340 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:04.224826 11511 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:04.213654 11398 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:04.279531 11481 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 12:19:04.279907 11510 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:04.213436 11428 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:04.276700  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:04.287827 11083 consensus_queue.cc:1048] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:04.288630  6475 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:04.288940 11495 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:04.283327 11388 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:04.290967 11341 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:19:04.292459 11518 log.cc:927] Time spent T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:19:04.294981 11369 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:04.296674  6584 raft_consensus.cc:1240] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:04.303198 11398 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:04.305395 11503 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:04.306990 11439 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:04.311596  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:04.312836  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:04.315124  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:04.320279  6475 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:04.318353  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:04.327518  6584 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 12:19:04.327692  6654 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:19:04.339171  6657 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:19:04.341691  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:04.342017  6656 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 14.
I20260709 12:19:04.343034  6475 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:04.351886  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:04.353746  6550 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:04.352607  7268 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:19:04.357164  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:04.359674  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:04.358986  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:04.361132  6653 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:19:04.355952  6550 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:04.360316  6650 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:19:04.325451 11522 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:04.363982  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:04.364641  6650 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:19:04.328532 11495 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.365514  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:04.317741 11500 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:04.382758  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:04.383729  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:04.384591  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:04.384668  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:04.394210  7265 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:04.385723  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:04.333985 11481 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:04.389021  6474 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:04.391630  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:04.406098  6652 raft_consensus.cc:2410] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:19:04.386229  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:04.352583 11439 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:04.414700  6475 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:04.358270 11012 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.362788 11418 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:19:04.416123  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 60231.4us
I20260709 12:19:04.362602 11503 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:04.409533  6657 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
I20260709 12:19:04.328496 11388 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:04.425042 11526 log.cc:927] Time spent T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:19:04.429620  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:04.430471  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:04.436375  6652 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:19:04.437260  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:04.437871  6652 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:19:04.438927  6657 raft_consensus.cc:1240] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:04.443908  6509 raft_consensus.cc:1240] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:04.421428 11522 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:04.444576  6475 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:04.434204 11529 log.cc:927] Time spent T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:19:04.282476 11511 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:04.431792 11074 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:04.450865 11242 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:04.418288 11500 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:04.455253  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:04.456471  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:04.303771 11510 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:04.462194 11418 raft_consensus.cc:697] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:19:04.464177 11531 log.cc:927] Time spent T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:19:04.465260  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:04.465842  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:04.408057 11495 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:04.473791  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:04.474507  6657 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:19:04.446714 11522 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:04.478957 11522 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:04.479511 11495 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:04.479730  6584 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 12:19:04.428905 11012 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:04.481689  6584 pending_rounds.cc:85] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 10
I20260709 12:19:04.481966  6584 pending_rounds.cc:107] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 12:19:04.482230  6584 raft_consensus.cc:2889] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:04.415812 11481 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:04.477624 11511 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:04.430205 11475 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:04.481037  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:04.476943 11418 consensus_queue.cc:237] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:04.445811 11341 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:04.516157  6579 raft_consensus.cc:2393] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 10: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:19:04.531066  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 67588us
I20260709 12:19:04.532435 11012 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:04.496958  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:04.508468  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:04.535108  7265 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:04.482264 11074 raft_consensus.cc:697] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:04.536528  6657 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:19:04.537113  6548 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:04.538733  6548 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:04.541217  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:04.542021  6657 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:04.545125  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:04.545528  6474 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:04.548678  7238 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:04.550607  6548 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:04.545166  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:04.556267  6548 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:04.561371  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:04.562116  6657 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:19:04.563027  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:04.542169 11242 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:04.569765  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:04.571794  6475 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:04.543464 11425 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:04.577571  6502 raft_consensus.cc:2393] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 11: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:04.577836  6548 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:04.579572  6507 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
new_leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:58944
I20260709 12:19:04.580111  6475 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:04.580354  6507 raft_consensus.cc:606] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Received request to transfer leadership to TS 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:04.580875  6507 raft_consensus.cc:614] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:04.553306 11341 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:04.583750 11074 consensus_queue.cc:237] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:04.583155  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:04.585564  6475 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:04.586855  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:04.587579  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:04.588708  6657 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:04.556402 11254 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:04.558668 10963 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:04.577175 11503 raft_consensus.cc:2804] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:04.579071 11369 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:04.585287 11341 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:04.574545 11366 consensus_queue.cc:1059] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:19:04.553148 11485 consensus_queue.cc:1059] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:19:04.592836 11340 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:04.586400 11425 raft_consensus.cc:697] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:04.588872 11327 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:04.646185  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:04.656946  6507 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:19:04.592446 11397 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:04.648864  6550 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:04.589128 11428 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:04.654927  6657 raft_consensus.cc:1275] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:19:04.647080  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:19:04.586156 10738 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 12:19:04.596530 10963 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:04.662005  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:04.601857 11503 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:04.644613 11475 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:04.668818  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:04.645081 11508 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.670608  6550 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:04.592257 11490 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 12:19:04.682353  6657 raft_consensus.cc:2393] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 10: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:04.645705 11341 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:04.607193 11254 raft_consensus.cc:697] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 13 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:04.658169 11340 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:04.686475  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:04.660001 11478 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:19:04.690219  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 111106us
I20260709 12:19:04.665036 10963 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:04.669111 11503 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:04.694782  6550 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:04.697228  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:04.668874 11388 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:04.699457  6657 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:04.674821 11242 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:04.655611 11522 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:04.705724  6550 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:04.700604 11527 log.cc:927] Time spent T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 12:19:04.674646 11397 raft_consensus.cc:697] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:04.732241  6550 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:04.698315 11340 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:04.733780  6548 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:04.702626 11023 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:04.686630 11508 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:04.727015 11482 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:04.810202 11482 raft_consensus.cc:697] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 11 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:04.811256 11482 consensus_queue.cc:237] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:04.725951 11481 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:04.813228  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:04.814049 11508 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:04.814327  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:04.815318  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:04.815889  6507 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:19:04.707917 11242 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:04.726399 10990 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:04.782711 10999 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:04.800767 11521 consensus_queue.cc:1059] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 12:19:04.739279 11023 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:19:04.807982 11527 log.cc:927] Time spent T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:19:04.809496 11522 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:04.825210 10999 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:04.820075  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:04.842700 10999 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:04.836028 11023 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:04.820633 11481 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.783337 10738 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:04.862444  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:04.863118  6657 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:04.826437  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 92773.9us
I20260709 12:19:04.785624 11425 consensus_queue.cc:237] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:04.824002 11242 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:04.828087 11397 consensus_queue.cc:237] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:04.839238 11332 raft_consensus.cc:3055] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:04.836028 11490 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:04.861951 10738 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:04.824545 10990 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:04.819744  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:19:04.926519  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 106860us
I20260709 12:19:04.927374  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:04.871595  6548 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:04.834405 11254 consensus_queue.cc:237] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:04.944053  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 73242.4us
W20260709 12:19:04.943703  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 209012us
I20260709 12:19:04.872467 11510 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:04.948361 11541 log.cc:927] Time spent T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.210s	user 0.001s	sys 0.000s
I20260709 12:19:04.948872  6548 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:04.950038 11475 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:04.950994 11487 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.951445 11487 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:04.950002  6548 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:04.955902 11475 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:04.956346 11475 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:04.956761 11475 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:04.957456 11546 log.cc:927] Time spent T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 12:19:04.946125 11511 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:04.958630 11511 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:04.958964 11511 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:04.959911  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:04.960669  6509 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:04.961397 11487 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:04.961737  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:04.964808  6550 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:04.965966  6550 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:04.966998  6550 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:04.925751 10990 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:04.970815 11473 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:04.971343 11473 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:04.973016  6550 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:04.975157 11475 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:04.975337 10963 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:04.979149 10963 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:04.979846 10963 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:04.980558 10963 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:04.980753 11510 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:04.981328 10963 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:04.981745 10963 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:04.982143 10963 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:04.871687 11332 raft_consensus.cc:740] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:04.921913 10738 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:05.013650 11340 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:05.006266  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:04.947156 11471 log.cc:927] Time spent T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:19:04.864557 11481 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:05.008625  6657 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:19:04.984563 11485 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:04.979960 11341 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:05.019388  6509 raft_consensus.cc:2393] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 14: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:19:05.067648  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 94696.5us
I20260709 12:19:05.068861  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:05.068985  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:05.069458  6657 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:04.995455 11458 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:05.070963  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:05.074229  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:05.064850  6475 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:05.080891  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:05.088266  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:05.020375 11473 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:05.091140  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:04.976195 10999 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:05.096160  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:05.096751  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 169484us
I20260709 12:19:05.097441  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:05.078044 11458 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:05.082043  6475 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:05.092381  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.092271 11277 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:19:05.112916  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.100534 11458 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:05.116317  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:05.101593 11551 consensus_queue.cc:1048] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:19:05.094903 11332 consensus_queue.cc:260] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:05.127957  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:05.167966  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:19:05.206823  6581 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 2123) took 1537 ms (client timeout 1654 ms). Trace:
I20260709 12:19:05.202134  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:05.161924  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:05.162329  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:05.272607  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 44154.4us
I20260709 12:19:05.165314  6500 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 2423) took 1252 ms. Trace:
W20260709 12:19:05.169814  6505 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 2222) took 1555 ms (client timeout 1920 ms). Trace:
I20260709 12:19:05.273306  6500 rpcz_store.cc:276] 0709 12:19:03.913139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.059524 (+146385us) service_pool.cc:224] Handling call
0709 12:19:05.165117 (+1105593us) spinlock_profiling.cc:243] Waited 33.9 ms on lock 0x7b540017c0d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:05.165245 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":78046080}
I20260709 12:19:05.168632  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:05.276553  7272 raft_consensus.cc:2393] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 10: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:05.178259  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:05.276937  6548 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:05.275770  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:05.278800  6507 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:19:05.168292  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:05.271777  6652 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:19:05.289947  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:05.272220  6581 rpcz_store.cc:269] 0709 12:19:03.669416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.670934 (+  1518us) service_pool.cc:224] Handling call
0709 12:19:05.206575 (+1535641us) spinlock_profiling.cc:243] Waited 524 ms on lock 0x7b540006de54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:05.206713 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1205021952}
I20260709 12:19:05.292124 11562 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:05.294703  6550 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:05.291085  6652 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:19:05.169077  7267 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:05.128722  6501 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 2236) took 1464 ms (client timeout 1784 ms). Trace:
I20260709 12:19:05.298408  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:05.195851  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:05.299572  6475 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:05.298228  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:05.300555  6652 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:05.273815  6505 rpcz_store.cc:269] 0709 12:19:03.614100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.614273 (+   173us) service_pool.cc:224] Handling call
0709 12:19:05.169648 (+1555375us) spinlock_profiling.cc:243] Waited 384 ms on lock 0x7b54000fee54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:05.169761 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":882903296}
W20260709 12:19:05.299018  6501 rpcz_store.cc:269] 0709 12:19:03.664593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.665878 (+  1285us) service_pool.cc:224] Handling call
0709 12:19:05.128516 (+1462638us) spinlock_profiling.cc:243] Waited 486 ms on lock 0x7b540039fb54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:05.128651 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1116685184}
I20260709 12:19:05.305553  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:05.306376  6501 raft_consensus.cc:2393] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 11: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:05.116182 11521 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:05.312649  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:19:05.120566 11552 log.cc:927] Time spent T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:19:05.309729  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:05.313933  6576 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:05.316197  6501 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:05.314982  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:05.150857 11418 raft_consensus.cc:3055] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:05.211513 11388 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.317566  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:05.331038  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:05.331714  6507 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:05.118698 11277 raft_consensus.cc:697] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:05.319924  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:05.150067 11490 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:05.131278 11316 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:05.134274 11341 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:05.152452 11481 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:05.191330 10999 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:05.320626  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:05.542485  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:05.543931  7267 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:19:05.546051  6500 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 12. Current term is 13. Ops: [11.11-12.12]
I20260709 12:19:05.548882  6500 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:05.551699  6500 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:19:05.321116 11332 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:05.152905 11511 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:05.555235 11511 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:05.555521  6500 raft_consensus.cc:1275] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:19:05.148886 10738 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:05.558076  6651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 767) took 1249 ms. Trace:
I20260709 12:19:05.558621 10738 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:05.166977 11503 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:05.317099 11521 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:05.561559 11521 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:05.561802  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2765) took 1247 ms. Trace:
I20260709 12:19:05.530931 11388 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:05.309329 11478 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:05.134919 11475 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:05.568488  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.170192 11015 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:05.197438 11491 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:05.568523 11478 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:05.573158 11478 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:05.333195  6548 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:05.334028  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:05.579308  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 2444) took 1010 ms. Trace:
I20260709 12:19:05.579821  6507 rpcz_store.cc:276] 0709 12:19:04.568421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.333766 (+765345us) service_pool.cc:224] Handling call
0709 12:19:05.579222 (+245456us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:05.133679 11474 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:05.497432  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.519356  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:05.305838 11497 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:05.524501  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:05.597139  6505 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:05.527240  6501 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:19:05.597995  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 2440) took 1030 ms. Trace:
I20260709 12:19:05.598320  6501 rpcz_store.cc:276] 0709 12:19:04.567382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.319643 (+752261us) service_pool.cc:224] Handling call
0709 12:19:05.597932 (+278289us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:05.128619 11554 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:05.308355 11391 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:05.531965  6473 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:05.120031 11023 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:19:05.145812 11242 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:05.536715 11565 log.cc:927] Time spent T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.223s	user 0.000s	sys 0.000s
I20260709 12:19:05.536986  6653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2772) took 1177 ms. Trace:
I20260709 12:19:05.607514  6653 rpcz_store.cc:276] 0709 12:19:04.359699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.361744 (+  2045us) service_pool.cc:224] Handling call
0709 12:19:04.362029 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:04.503489 (+141460us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:04.503542 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:04.510891 (+  7349us) log.cc:844] Serialized 26 byte log entry
0709 12:19:04.659743 (+148852us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:04.659778 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.536564 (+876786us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b54001509d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:05.536618 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:05.536674 (+    56us) raft_consensus.cc:1685] finished
0709 12:19:05.536689 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:05.536921 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15785,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":5059,"spinlock_wait_cycles":277919616,"thread_start_us":101511,"threads_started":1,"wal-append.queue_time_us":191518}
I20260709 12:19:05.609426  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:05.610011  6653 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:05.558537  6651 rpcz_store.cc:276] 0709 12:19:04.309023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.313440 (+  4417us) service_pool.cc:224] Handling call
0709 12:19:04.313700 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:04.500914 (+187214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:04.500966 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:04.501033 (+    67us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:04.501085 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.557853 (+1056768us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b5400074ed4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:05.557903 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:05.558016 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":429870336}
I20260709 12:19:05.562225  6655 rpcz_store.cc:276] 0709 12:19:04.314420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.314566 (+   146us) service_pool.cc:224] Handling call
0709 12:19:04.314838 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:04.478436 (+163598us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:04.478466 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:04.478506 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:04.478537 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.561551 (+1083014us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b540008bbd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:05.561615 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:05.561743 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":375806208}
I20260709 12:19:05.171231 11184 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:05.565739 11388 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:05.566030  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.125010 11340 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:05.320845 11418 raft_consensus.cc:740] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:05.184849 10990 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:05.191161 10963 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:05.323799  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 207558us
I20260709 12:19:05.563134  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:19:05.590458  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 257374us
I20260709 12:19:05.591840  6652 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:05.601737  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:05.622203  6657 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:05.623219  6475 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:05.610831 11497 raft_consensus.cc:697] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:19:05.612427  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 80552.3us
I20260709 12:19:05.615875  6656 raft_consensus.cc:1240] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:19:05.632315  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2167) took 1074 ms. Trace:
I20260709 12:19:05.632694  6656 rpcz_store.cc:276] 0709 12:19:04.557948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.615341 (+1057393us) service_pool.cc:224] Handling call
0709 12:19:05.615670 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:05.632023 (+ 16353us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.632254 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:05.567579 11277 consensus_queue.cc:237] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:05.596833 11427 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:05.617444  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:05.654891  6655 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:05.605165 11391 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:05.629563 11340 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:05.626606  6548 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:05.602346 11397 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.012s	sys 0.058s
I20260709 12:19:05.628311  6653 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
W20260709 12:19:05.668583  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:05.635617 11499 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.629245  6622 leader_election.cc:400] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 12. Current term is 13.
I20260709 12:19:05.638635  6657 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:19:05.617203 11341 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:05.649581 10877 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:05.650990 11156 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:05.652408  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:05.661741 11391 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:05.691648 11478 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:05.694469  6580 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 2118) took 2026 ms (client timeout 2406 ms). Trace:
W20260709 12:19:05.676986 11397 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.013s	sys 0.059s
I20260709 12:19:05.657250 11464 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:05.695281  6575 raft_consensus.cc:1240] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 12. Current term is 13. Ops: []
W20260709 12:19:05.695883  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 66773.8us
I20260709 12:19:05.704679  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:05.697218  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:05.698369  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:05.677325 11554 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.695000  7268 raft_consensus.cc:2368] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 9. Current term is 10.
I20260709 12:19:05.658762 11331 consensus_queue.cc:1048] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:19:05.706322  7271 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:05.699342  6580 rpcz_store.cc:269] 0709 12:19:03.668215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.698213 (+ 29998us) service_pool.cc:224] Handling call
0709 12:19:05.694254 (+1996041us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b5400031e54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:05.694408 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":273372928}
I20260709 12:19:05.707669  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:05.658531 11428 consensus_queue.cc:1048] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 12:19:05.709041  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:05.681849 11512 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:05.713325  6622 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:05.714815  6623 leader_election.cc:400] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 9. Current term is 10.
I20260709 12:19:05.717728  6623 leader_election.cc:403] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:05.730752  6655 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:19:05.733136  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:05.750847  6655 raft_consensus.cc:2393] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 13: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:05.679092  6505 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:19:05.685899 11499 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:05.771338 11023 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:05.795657 11327 raft_consensus.cc:3055] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:05.769994 10963 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:05.813603 11327 raft_consensus.cc:740] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:05.801278 11242 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:05.796367 11328 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:05.793776  7271 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:19:05.711956 11156 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:05.815480 11156 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:05.815958 11156 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:05.798008  6503 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 2239) took 2127 ms (client timeout 2106 ms). Trace:
I20260709 12:19:05.817406  7240 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:19:05.817552  6503 rpcz_store.cc:269] 0709 12:19:03.670967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.691522 (+ 20555us) service_pool.cc:224] Handling call
0709 12:19:05.797761 (+2106239us) spinlock_profiling.cc:243] Waited 27.9 ms on lock 0x7b540001bdd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:05.797892 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64070912}
I20260709 12:19:05.789438  6579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54204 (request call id 677) took 1517 ms. Trace:
I20260709 12:19:05.804744  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 769) took 1495 ms. Trace:
I20260709 12:19:05.793864  6549 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:05.819268  6649 rpcz_store.cc:276] 0709 12:19:04.309568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.321570 (+ 12002us) service_pool.cc:224] Handling call
0709 12:19:04.321896 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:04.322124 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:19:04.355005 (+ 32881us) log.cc:844] Serialized 14 byte log entry
0709 12:19:05.539685 (+1184680us) spinlock_profiling.cc:243] Waited 7.34 ms on lock 0x7b18002e5280. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:19:05.803985 (+264300us) spinlock_profiling.cc:243] Waited 2.23 ms on lock 0x7b18002e5280. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2875a314bc 00007f2872d7f48b 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a2923a
0709 12:19:05.804318 (+   333us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:05.804403 (+    85us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:05.804462 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.804544 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:05.804677 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":32706,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"spinlock_wait_cycles":21998848,"thread_start_us":59592,"threads_started":1,"wal-append.queue_time_us":55721}
I20260709 12:19:05.818478  6579 rpcz_store.cc:276] 0709 12:19:04.271841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.276302 (+  4461us) service_pool.cc:224] Handling call
0709 12:19:05.789187 (+1512885us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b54004204d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:05.789367 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":463361792}
I20260709 12:19:05.835541  6549 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:05.840507 11491 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:19:05.843025  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 211473us
W20260709 12:19:05.789108 11560 log.cc:927] Time spent T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
W20260709 12:19:05.850852  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 191158us
I20260709 12:19:05.854080  6548 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:05.806675 11462 raft_consensus.cc:3055] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:19:05.708278 11184 consensus_queue.cc:1059] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.127s
I20260709 12:19:05.766800 11388 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:05.795094 11112 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:05.802820 11496 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:05.712716 11474 consensus_queue.cc:1059] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.131s
I20260709 12:19:05.772701 11478 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:05.847725 11328 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:05.765317 11481 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 9. Current term is 10.
I20260709 12:19:05.793056 11503 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:05.877274  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 772) took 1089 ms. Trace:
I20260709 12:19:05.965065  6649 rpcz_store.cc:276] 0709 12:19:04.788015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.847971 (+1059956us) service_pool.cc:224] Handling call
0709 12:19:05.876570 (+ 28599us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.876828 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:05.876870 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:05.876933 (+    63us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:05.876981 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.877053 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:05.877199 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:05.709741 11497 consensus_queue.cc:237] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:05.930598 11513 raft_consensus.cc:2764] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader election decision vote started in defunct term 14: won
I20260709 12:19:05.834148 11367 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.098s
I20260709 12:19:05.933799 11499 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:05.874423 11369 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:05.937717  6504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2427) took 1950 ms. Trace:
I20260709 12:19:05.951217 11512 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:05.789175 11391 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:05.977579  6504 rpcz_store.cc:276] 0709 12:19:03.987200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.246276 (+259076us) service_pool.cc:224] Handling call
0709 12:19:04.268432 (+ 22156us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:04.286071 (+ 17639us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:19:04.286326 (+   255us) log.cc:844] Serialized 14 byte log entry
0709 12:19:04.390753 (+104427us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:04.390822 (+    69us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:04.390890 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.937418 (+1546528us) spinlock_profiling.cc:243] Waited 17.4 ms on lock 0x7b54003976d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:05.937478 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:05.937636 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":776,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":35,"raft.queue_time_us":128,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":105,"spinlock_wait_cycles":40032896,"thread_start_us":103995,"threads_started":1,"wal-append.queue_time_us":118029,"wal-append.run_cpu_time_us":1906,"wal-append.run_wall_time_us":1063435}
I20260709 12:19:05.942793  6506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 578) took 1953 ms. Trace:
I20260709 12:19:05.767752 11521 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:05.981148  6506 rpcz_store.cc:276] 0709 12:19:03.989341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.412197 (+422856us) service_pool.cc:224] Handling call
0709 12:19:04.413315 (+  1118us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:04.432105 (+ 18790us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:04.432141 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:04.432675 (+   534us) log.cc:844] Serialized 26 byte log entry
0709 12:19:04.605739 (+173064us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:05.653115 (+1047376us) spinlock_profiling.cc:243] Waited 1.54 ms on lock 0x7b1800329f80. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d99455 00007f28798813c7 00007f2875e939a7 00007f2875ea30bf 00007f2875ea3038
0709 12:19:05.845424 (+192309us) log.cc:844] Serialized 14 byte log entry
0709 12:19:05.846063 (+   639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.851395 (+  5332us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:05.851437 (+    42us) raft_consensus.cc:1685] finished
0709 12:19:05.941427 (+ 89990us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:05.942718 (+  1291us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68844,"prepare.queue_time_us":116741,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":153049,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":6723,"spinlock_wait_cycles":3543808,"thread_start_us":364304,"threads_started":2,"wal-append.queue_time_us":235333}
I20260709 12:19:05.955185  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 2448) took 1385 ms. Trace:
I20260709 12:19:05.983888  6509 rpcz_store.cc:276] 0709 12:19:04.569424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.961476 (+392052us) service_pool.cc:224] Handling call
0709 12:19:05.954933 (+993457us) spinlock_profiling.cc:243] Waited 52.8 ms on lock 0x7b54000f3f54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:05.955106 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":121416064}
I20260709 12:19:05.985303  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:05.986980  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:19:05.932228 11458 consensus_meta.cc:287] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.763s	user 0.007s	sys 0.002s
I20260709 12:19:05.874182 11332 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:19:05.967469 11541 log.cc:927] Time spent T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:19:05.863284 11462 raft_consensus.cc:740] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:05.712845 11554 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:05.936955 11341 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:05.974753 11478 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:05.850293 11421 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.117s
I20260709 12:19:05.988481 11499 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:05.988610  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:06.023813  6504 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:05.989363  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:06.024991  6506 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:05.990713 11391 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:05.981350 11369 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:05.998136  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:19:05.998801  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.003954  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:06.029917  6649 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:06.014575  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:06.031313  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:06.032312 11369 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:06.028954 11499 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:05.982980 11474 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.035308  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:05.981173 11503 consensus_queue.cc:1059] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 12:19:06.024199 11332 raft_consensus.cc:697] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:06.057695  6649 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:19:05.982002 11512 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:06.017138 11458 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:06.071221  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:06.074599  6506 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:19:06.089341  6651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2169) took 1517 ms. Trace:
I20260709 12:19:06.089509  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:06.090390  6506 raft_consensus.cc:2393] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 9: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:06.089723  6651 rpcz_store.cc:276] 0709 12:19:04.572139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.619036 (+1046897us) service_pool.cc:224] Handling call
0709 12:19:05.619282 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.619425 (+   143us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:05.636131 (+ 16706us) log.cc:844] Serialized 14 byte log entry
0709 12:19:06.089029 (+452898us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:06.089080 (+    51us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:06.089126 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.089188 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.089294 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25572,"prepare.queue_time_us":40840,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"thread_start_us":452498,"threads_started":1}
I20260709 12:19:06.090545  6650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2774) took 1697 ms. Trace:
I20260709 12:19:06.091099  6650 rpcz_store.cc:276] 0709 12:19:04.393179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:04.393320 (+   141us) service_pool.cc:224] Handling call
0709 12:19:04.393617 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:04.445742 (+ 52125us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:04.445771 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:04.445811 (+    40us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:04.445851 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.090257 (+1644406us) spinlock_profiling.cc:243] Waited 51.9 ms on lock 0x7b54001b0b54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:06.090323 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.090475 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":119448448}
I20260709 12:19:06.092810  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:06.093673  6650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2174) took 1231 ms. Trace:
I20260709 12:19:06.094188  6650 rpcz_store.cc:276] 0709 12:19:04.862385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.092791 (+1230406us) service_pool.cc:224] Handling call
0709 12:19:06.093067 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:06.093270 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:06.093309 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:06.093366 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:06.093418 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.093487 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.093612 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:06.095352  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2170) took 1522 ms. Trace:
I20260709 12:19:06.079236  6475 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:06.095813  6652 rpcz_store.cc:276] 0709 12:19:04.572492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.622268 (+1049776us) service_pool.cc:224] Handling call
0709 12:19:05.622542 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.622736 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:19:05.663131 (+ 40395us) log.cc:844] Serialized 14 byte log entry
0709 12:19:05.765631 (+102500us) log.cc:844] Serialized 14 byte log entry
0709 12:19:06.094928 (+329297us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:06.094995 (+    67us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:06.095055 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.095142 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.095286 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":100677,"prepare.queue_time_us":131292,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":71,"raft.queue_time_us":4375,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":180,"thread_start_us":379096,"threads_started":2,"wal-append.queue_time_us":303031}
W20260709 12:19:06.079167 11575 log.cc:927] Time spent T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:19:06.041188 11474 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:06.093847  6649 raft_consensus.cc:2393] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 12: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:06.076227  6504 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:19:06.101241  6548 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:06.123308  6504 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:06.124752  6504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2248) took 1010 ms. Trace:
I20260709 12:19:06.103828  6650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2794) took 1011 ms. Trace:
I20260709 12:19:06.076115 11499 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:06.136188  6650 rpcz_store.cc:276] 0709 12:19:05.092920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.102855 (+1009935us) service_pool.cc:224] Handling call
0709 12:19:06.103224 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:06.103445 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:06.103482 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:06.103539 (+    57us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:06.103577 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.103641 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.103763 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:06.092948  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:06.138602  6506 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:19:06.140131  6650 raft_consensus.cc:1240] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:19:06.141592 11341 raft_consensus.cc:3055] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:06.142114 11341 raft_consensus.cc:740] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:06.143409  6650 raft_consensus.cc:1240] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:06.143122 11341 consensus_queue.cc:260] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:06.145418  6504 rpcz_store.cc:276] 0709 12:19:05.114243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.122760 (+1008517us) service_pool.cc:224] Handling call
0709 12:19:06.123064 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:06.124462 (+  1398us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.124689 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:06.146770  6504 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:06.147284 11345 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:06.128731  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2795) took 1035 ms. Trace:
I20260709 12:19:06.151291  6649 rpcz_store.cc:276] 0709 12:19:05.093207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.127884 (+1034677us) service_pool.cc:224] Handling call
0709 12:19:06.128171 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:06.128443 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:06.128465 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:06.128513 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:06.128544 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.128597 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.128680 (+    83us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:06.087785  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 374502us
I20260709 12:19:06.153115  6622 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:06.125770  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2453) took 1139 ms. Trace:
W20260709 12:19:06.127596  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 276310us
I20260709 12:19:06.156510  6501 rpcz_store.cc:276] 0709 12:19:04.986067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.598954 (+612887us) service_pool.cc:224] Handling call
0709 12:19:05.599266 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.599457 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:05.613050 (+ 13593us) log.cc:844] Serialized 14 byte log entry
0709 12:19:05.857682 (+244632us) log.cc:844] Serialized 14 byte log entry
0709 12:19:06.125365 (+267683us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:06.125432 (+    67us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:06.125502 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.125583 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.125710 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30922,"prepare.queue_time_us":476248,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":6258,"raft.queue_time_us":232215,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":25097,"thread_start_us":486325,"threads_started":2}
W20260709 12:19:06.135768 11567 log.cc:927] Time spent T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.204s	user 0.003s	sys 0.000s
I20260709 12:19:06.161542  6658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2166) took 1605 ms. Trace:
I20260709 12:19:06.161996  6658 rpcz_store.cc:276] 0709 12:19:04.555839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.129286 (+573447us) service_pool.cc:224] Handling call
0709 12:19:05.129614 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:05.292248 (+162634us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:05.292291 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:05.292922 (+   631us) log.cc:844] Serialized 26 byte log entry
0709 12:19:05.653599 (+360677us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:05.653641 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.653723 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:06.134866 (+481143us) spinlock_profiling.cc:243] Waited 8.92 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:06.161203 (+ 26337us) raft_consensus.cc:1685] finished
0709 12:19:06.161227 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.161471 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":81356,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":104,"spinlock_wait_cycles":20515712,"thread_start_us":522500,"threads_started":2,"wal-append.queue_time_us":565684}
I20260709 12:19:06.163341 11418 consensus_queue.cc:260] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:06.091835  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:06.171516  6651 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:19:06.186980  6575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2202) took 2350 ms. Trace:
I20260709 12:19:06.187620  6575 rpcz_store.cc:276] 0709 12:19:03.836264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.919762 (+ 83498us) service_pool.cc:224] Handling call
0709 12:19:05.695081 (+1775319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.709548 (+ 14467us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.186628 (+477080us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b5400031e50. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:06.186859 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4082100096}
I20260709 12:19:06.204952 11428 consensus_queue.cc:1059] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:19:06.164716 11418 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:06.150646  6576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 2139) took 1081 ms. Trace:
I20260709 12:19:06.224714  6576 rpcz_store.cc:276] 0709 12:19:05.069498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.071366 (+  1868us) service_pool.cc:224] Handling call
0709 12:19:05.091913 (+ 20547us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.316157 (+224244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.150316 (+834159us) spinlock_profiling.cc:243] Waited 222 ms on lock 0x7b540006e5d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:06.150545 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":510164736}
I20260709 12:19:06.227169 11341 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:06.240159 11123 consensus_queue.cc:1059] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:06.241667 11480 raft_consensus.cc:3055] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:06.242205 11480 raft_consensus.cc:740] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:06.243444 11480 consensus_queue.cc:260] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:06.249388 11480 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:06.250929  6651 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
W20260709 12:19:06.260775  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 159706us
I20260709 12:19:06.261896  6548 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:19:06.264840  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 111819us
I20260709 12:19:06.265828  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:19:06.275766  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 458373us
I20260709 12:19:06.276929  7240 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:06.105433 11478 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:06.108062 11388 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:19:06.086439  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 387978us
I20260709 12:19:06.160079  6501 raft_consensus.cc:1240] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:06.156082  6649 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 12:19:06.301939  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.302680  6622 leader_election.cc:400] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14.
I20260709 12:19:06.303079  6622 leader_election.cc:403] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:06.306289  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:06.307381  6578 raft_consensus.cc:2393] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 12: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:06.308730  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:06.311685  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:06.313616  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:06.314460  6579 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:19:06.315738  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:06.318084  6581 raft_consensus.cc:1240] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:06.321306  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:06.322844  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:06.323695  6579 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:19:06.324962  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:06.327773  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:06.328567  6581 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:06.329779  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:06.113596 11156 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:06.333250  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 55214.9us
I20260709 12:19:06.334022 11156 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:06.270982 10877 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:19:06.336534  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.336768 11513 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:06.337359 11513 raft_consensus.cc:697] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 10 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:06.338284 11513 consensus_queue.cc:237] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:06.340063 11389 consensus_queue.cc:1059] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:06.344310  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:06.320520  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:06.348732  6578 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 8.
I20260709 12:19:06.061417 11462 consensus_queue.cc:260] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:06.127327 11340 raft_consensus.cc:697] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:06.354976  6582 raft_consensus.cc:1240] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:06.359174  6575 raft_consensus.cc:2393] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 14: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:19:06.363430  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 60857us
W20260709 12:19:06.373389  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 111557us
I20260709 12:19:06.270495 10962 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:06.374365  6548 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:19:06.342190 11575 log.cc:927] Time spent T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:19:06.380340 11491 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:06.386968 11491 raft_consensus.cc:697] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:06.388046 11491 consensus_queue.cc:237] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:06.393747  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:06.411962  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.385041  6622 leader_election.cc:400] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 16. Current term is 17.
I20260709 12:19:06.414390  6622 leader_election.cc:403] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
W20260709 12:19:06.414849  6505 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.016s	sys 0.020s
W20260709 12:19:06.415735  6505 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.016s	sys 0.020s
I20260709 12:19:06.355463 11510 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:19:06.418747  6507 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.028s	sys 0.037s
W20260709 12:19:06.421774  6507 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.028s	sys 0.037s
W20260709 12:19:06.429183  6655 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.012s	sys 0.023s
W20260709 12:19:06.430372  6655 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.012s	sys 0.023s
I20260709 12:19:06.429711  6550 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:06.357916 11550 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:19:06.446241  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:06.469134  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 84267.4us
I20260709 12:19:06.472525  6622 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:06.410869 11462 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:06.482589  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:06.484928  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:06.485746  6582 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:06.487254  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:06.264008 11332 consensus_queue.cc:237] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:06.355755 11388 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:06.512893  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 83252.1us
W20260709 12:19:06.517526  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 45212.5us
I20260709 12:19:06.519013  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:06.428191 11328 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:06.436826 11277 raft_consensus.cc:3055] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:06.531100  6580 raft_consensus.cc:1240] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:19:06.468060 11472 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:06.509598  6575 raft_consensus.cc:2393] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 14: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:06.530156  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:06.468575 10962 raft_consensus.cc:697] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:19:06.513123 11568 log.cc:927] Time spent T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 12:19:06.539885  6584 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
W20260709 12:19:06.548866  6656 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.014s	sys 0.068s
W20260709 12:19:06.589303  6656 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.014s	sys 0.068s
I20260709 12:19:06.590227  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2789) took 2011 ms. Trace:
I20260709 12:19:06.590595  6656 rpcz_store.cc:276] 0709 12:19:04.579367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.633523 (+1054156us) service_pool.cc:224] Handling call
0709 12:19:05.633867 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.634135 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:06.042366 (+408231us) log.cc:844] Serialized 14 byte log entry
0709 12:19:06.589912 (+547546us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:06.589963 (+    51us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:06.590006 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.590070 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.590176 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31489,"prepare.queue_time_us":82509,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":57,"thread_start_us":902026,"threads_started":2}
W20260709 12:19:06.561750  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:06.586532  7266 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.012s	sys 0.066s
I20260709 12:19:06.479020 11521 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:06.593310  7266 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.012s	sys 0.066s
I20260709 12:19:06.527264 11023 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:06.511343 11510 raft_consensus.cc:697] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:19:06.595800  6654 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.028s	sys 0.068s
I20260709 12:19:06.511857 11579 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 16. Current term is 17.
W20260709 12:19:06.605463  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 86532.5us
W20260709 12:19:06.600421  6503 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.027s	sys 0.043s
W20260709 12:19:06.606609  6503 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.028s	sys 0.043s
W20260709 12:19:06.602556  6472 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.033s	sys 0.069s
I20260709 12:19:06.527669 11511 raft_consensus.cc:2764] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
W20260709 12:19:06.607465  6472 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.033s	sys 0.069s
W20260709 12:19:06.601297  6654 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.029s	sys 0.068s
I20260709 12:19:06.609236  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2792) took 1965 ms. Trace:
I20260709 12:19:06.530998 11277 raft_consensus.cc:740] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:06.552701 11472 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:06.535616 11554 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:06.531224 11524 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:19:06.611789  6653 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.026s	sys 0.022s
I20260709 12:19:06.520262 11123 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.083s
I20260709 12:19:06.475126 11550 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:06.629727  6653 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.026s	sys 0.022s
I20260709 12:19:06.646744  6653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2171) took 2059 ms. Trace:
I20260709 12:19:06.647351  6653 rpcz_store.cc:276] 0709 12:19:04.587588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.668278 (+1080690us) service_pool.cc:224] Handling call
0709 12:19:05.668682 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.668947 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:19:06.088289 (+419342us) log.cc:844] Serialized 14 byte log entry
0709 12:19:06.646232 (+557943us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:06.646311 (+    79us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:06.646391 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.646511 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.646670 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17700,"prepare.queue_time_us":95868,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"thread_start_us":941522,"threads_started":2}
I20260709 12:19:06.537667 11328 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:06.636761  6622 leader_election.cc:400] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 12. Current term is 13.
I20260709 12:19:06.654222  6622 leader_election.cc:403] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:06.618341 11156 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:06.621417 11472 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:06.628875 11558 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:06.650650 11568 log.cc:927] Time spent T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
W20260709 12:19:06.662781  6657 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.013s	sys 0.026s
I20260709 12:19:06.663424  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:06.666268  6475 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.011s	sys 0.016s
I20260709 12:19:06.640906 11023 raft_consensus.cc:697] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:19:06.678458  6509 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.018s	sys 0.035s
W20260709 12:19:06.679641  6509 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.019s	sys 0.035s
I20260709 12:19:06.655623 11458 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:06.663430 11572 log.cc:927] Time spent T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.551s	user 0.001s	sys 0.001s
I20260709 12:19:06.657351 11512 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:06.675550  6657 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.014s	sys 0.027s
I20260709 12:19:06.691067  6657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2790) took 2101 ms. Trace:
I20260709 12:19:06.691565  6657 rpcz_store.cc:276] 0709 12:19:04.589893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.673907 (+1084014us) service_pool.cc:224] Handling call
0709 12:19:05.674330 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.674616 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:19:06.094404 (+419788us) log.cc:844] Serialized 14 byte log entry
0709 12:19:06.690608 (+596204us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:06.690681 (+    73us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:06.690749 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.690835 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.690993 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29651,"prepare.queue_time_us":90449,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"thread_start_us":974712,"threads_started":2}
W20260709 12:19:06.675966  6475 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.011s	sys 0.016s
W20260709 12:19:06.676043  7273 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.011s	sys 0.031s
I20260709 12:19:06.652371 10877 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:06.695958  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:06.697434  6622 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:19:06.673658  6506 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.014s	sys 0.025s
W20260709 12:19:06.689858  6473 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.000s	sys 0.034s
W20260709 12:19:06.693650  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 528817us
W20260709 12:19:06.693943  7273 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.012s	sys 0.031s
W20260709 12:19:06.698437  6506 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.014s	sys 0.025s
I20260709 12:19:06.666736 11579 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 12. Current term is 13.
W20260709 12:19:06.704960  6473 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.000s	sys 0.035s
I20260709 12:19:06.677266 11156 raft_consensus.cc:697] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:06.702975  6653 raft_consensus.cc:1240] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:06.607677 11340 consensus_queue.cc:237] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:06.714392  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:06.715263  6653 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:06.682519 11583 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.709540  6656 raft_consensus.cc:1240] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:19:06.687506 11474 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:06.701045  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:06.726033  6657 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:19:06.720042  6579 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:19:06.730455 11583 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:06.734275 11345 consensus_meta.cc:287] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.586s	user 0.009s	sys 0.011s
I20260709 12:19:06.673823 11277 consensus_queue.cc:260] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:06.735299  6577 raft_consensus.cc:2410] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:19:06.677857 11558 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:06.737887 11558 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:06.738487 10877 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14.
I20260709 12:19:06.746182 11328 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:06.747768 11156 consensus_queue.cc:237] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:06.752736 11398 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.753420 11398 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:06.719476 11327 consensus_queue.cc:260] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:06.703823 11564 raft_consensus.cc:2764] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 12:19:06.703665 11554 consensus_queue.cc:1059] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 12:19:06.758245 11554 raft_consensus.cc:3055] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:06.758692 11554 raft_consensus.cc:740] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:06.695668 11580 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:06.765991  6582 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
W20260709 12:19:06.771829  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 472916us
I20260709 12:19:06.609850  6655 rpcz_store.cc:276] 0709 12:19:04.644167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.816960 (+1172793us) service_pool.cc:224] Handling call
0709 12:19:05.817229 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.817420 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:19:05.817699 (+   279us) log.cc:844] Serialized 14 byte log entry
0709 12:19:06.608789 (+791090us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:06.608859 (+    70us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:06.608921 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.609008 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.609164 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":176219,"prepare.queue_time_us":14226,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"thread_start_us":612864,"threads_started":1}
I20260709 12:19:06.751791 11590 consensus_queue.cc:1059] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 12:19:06.781010  6581 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 9.
I20260709 12:19:06.784441  6657 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
W20260709 12:19:06.769589 11418 consensus_meta.cc:287] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.546s	user 0.007s	sys 0.004s
I20260709 12:19:06.760497 11481 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:19:06.776127 11581 log.cc:927] Time spent T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.298s	user 0.001s	sys 0.001s
I20260709 12:19:06.759647 11554 consensus_queue.cc:260] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:06.707788 11023 consensus_queue.cc:237] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:06.766127 11580 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:19:06.789723  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:06.797537  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2452) took 1811 ms. Trace:
I20260709 12:19:06.694964 10962 consensus_queue.cc:237] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:06.797945  6507 rpcz_store.cc:276] 0709 12:19:04.985854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.592685 (+606831us) service_pool.cc:224] Handling call
0709 12:19:05.593048 (+   363us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:05.593470 (+   422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:05.593496 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:05.605940 (+ 12444us) log.cc:844] Serialized 52 byte log entry
0709 12:19:05.818095 (+212155us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:06.425255 (+607160us) log.cc:844] Serialized 14 byte log entry
0709 12:19:06.425531 (+   276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.425646 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:06.797340 (+371694us) raft_consensus.cc:1685] finished
0709 12:19:06.797358 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.797484 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4592,"prepare.queue_time_us":481082,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":6682,"raft.queue_time_us":167528,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":16237,"thread_start_us":750955,"threads_started":2,"wal-append.queue_time_us":456091}
I20260709 12:19:06.704036 11510 consensus_queue.cc:237] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:06.803982  6474 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:06.790333 11480 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 12:19:06.800345 11388 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:06.800125 11580 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:06.808626 11554 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:06.830178 11478 raft_consensus.cc:697] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:06.894114  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2455) took 1907 ms. Trace:
I20260709 12:19:06.894743  6503 rpcz_store.cc:276] 0709 12:19:04.986488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.834282 (+847794us) service_pool.cc:224] Handling call
0709 12:19:05.858243 (+ 23961us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.858486 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:19:06.607027 (+748541us) log.cc:844] Serialized 14 byte log entry
0709 12:19:06.893635 (+286608us) log.cc:844] Serialized 14 byte log entry
0709 12:19:06.893764 (+   129us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:06.893802 (+    38us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:06.893857 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.893924 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.894051 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1580,"prepare.queue_time_us":557939,"prepare.run_cpu_time_us":432,"prepare.run_wall_time_us":1469,"thread_start_us":954454,"threads_started":2}
I20260709 12:19:06.831615 11389 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 12:19:06.843884 11550 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:06.911265  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 213935us
I20260709 12:19:06.899092 11592 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:06.844889 11604 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:06.917981  6506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2458) took 1058 ms. Trace:
I20260709 12:19:06.918329  6506 rpcz_store.cc:276] 0709 12:19:05.859697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.139467 (+279770us) service_pool.cc:224] Handling call
0709 12:19:06.139818 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:06.140044 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:19:06.699075 (+559031us) log.cc:844] Serialized 14 byte log entry
0709 12:19:06.917640 (+218565us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:06.917694 (+    54us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:06.917741 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.917803 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.917916 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5707,"prepare.queue_time_us":454703,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":63,"thread_start_us":768876,"threads_started":2}
W20260709 12:19:06.915498  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 400574us
I20260709 12:19:06.899736 11603 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:06.926457  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:06.927541  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 193781us
W20260709 12:19:06.928153  6505 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.010s	sys 0.035s
W20260709 12:19:06.931609  6505 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.010s	sys 0.035s
I20260709 12:19:06.932633  6505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2454) took 1946 ms. Trace:
I20260709 12:19:06.932871  6473 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:06.926853  6622 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:06.936125  6622 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:06.933122  6505 rpcz_store.cc:276] 0709 12:19:04.986285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.751764 (+765479us) service_pool.cc:224] Handling call
0709 12:19:05.815783 (+ 64019us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:05.815993 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:19:06.417604 (+601611us) log.cc:844] Serialized 14 byte log entry
0709 12:19:06.932166 (+514562us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:06.932234 (+    68us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:06.932300 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.932409 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:06.932559 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":299633,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"thread_start_us":1115507,"threads_started":2}
I20260709 12:19:06.939616  6622 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:06.940631  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:06.941788  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:06.946444  6473 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:06.947086 11570 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:06.947791 11570 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:19:06.948868  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 134052us
I20260709 12:19:06.951188  6622 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:06.927220 11373 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:06.949577  7238 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:06.952732  6473 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:06.925938 11418 consensus_queue.cc:1059] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 12:19:06.937767 10738 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:06.927276 11458 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
W20260709 12:19:06.952806 11596 log.cc:927] Time spent T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 12:19:06.944530 11564 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:06.950906 11481 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:06.944269 11388 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:06.945040 11156 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:06.921871 11592 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:06.950603 11579 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:19:06.978185  6474 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.4:44857 timed out after 1.723s (SENT) [suppressed 1 similar messages]
I20260709 12:19:06.964799 11606 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:06.928047 11603 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:06.978165 10738 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:06.965680 11611 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:06.951642 11570 consensus_queue.cc:1059] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 12:19:06.999073  6473 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:06.963627 11373 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:06.994925 11598 log.cc:927] Time spent T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
W20260709 12:19:06.992041 11599 log.cc:927] Time spent T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 12:19:06.991160 11564 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:06.990134 11481 raft_consensus.cc:697] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 11 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:06.966517 11478 consensus_queue.cc:237] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:07.009599  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:07.012177 10990 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:07.004595 11600 log.cc:927] Time spent T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 12:19:07.014459  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 2212) took 3175 ms. Trace:
I20260709 12:19:07.020545  6577 rpcz_store.cc:276] 0709 12:19:03.838896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.909378 (+ 70482us) service_pool.cc:224] Handling call
0709 12:19:07.014232 (+3104854us) spinlock_profiling.cc:243] Waited 2.82 s on lock 0x7b5400329e54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:07.014400 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6496509312}
I20260709 12:19:06.976235 11616 consensus_queue.cc:1059] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:19:07.025219  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:07.028703  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:07.034528 11569 log.cc:927] Time spent T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 1.105s	user 0.000s	sys 0.003s
I20260709 12:19:06.997954 11619 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:07.036214  6473 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:07.001318 10738 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:07.050338  6500 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2451) took 2064 ms. Trace:
W20260709 12:19:07.039227  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:07.039987  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 61830us
W20260709 12:19:07.051730  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 100627us
I20260709 12:19:07.052000  6473 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:06.991904 11369 consensus_queue.cc:1059] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:19:07.053704  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:07.051030  6500 rpcz_store.cc:276] 0709 12:19:04.985593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.556846 (+571253us) service_pool.cc:224] Handling call
0709 12:19:05.557166 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:05.588102 (+ 30936us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:05.588134 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:05.590403 (+  2269us) log.cc:844] Serialized 26 byte log entry
0709 12:19:05.704681 (+114278us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:05.704724 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:05.704787 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:07.033887 (+1329100us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:07.036136 (+  2249us) raft_consensus.cc:1685] finished
0709 12:19:07.036157 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:07.050229 (+ 14072us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":30615,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":17483,"spinlock_wait_cycles":28859776,"thread_start_us":83148,"threads_started":1,"wal-append.queue_time_us":315753}
I20260709 12:19:07.013923 11331 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:07.058506  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:07.024984 11588 consensus_queue.cc:1059] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 14.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:19:07.038261 11418 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:07.061870  6622 leader_election.cc:400] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 12. Current term is 13.
I20260709 12:19:07.049363 11619 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:07.063278  6622 leader_election.cc:403] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:19:07.064962  6577 raft_consensus.cc:1240] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:19:07.066309  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:07.068602  6622 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:07.068771  6579 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:07.079881  6579 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:07.062036 11579 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:07.053367 11616 consensus_queue.cc:1243] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer 496c7c3792cd4587b4dbeba435f4be1e log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:19:06.999867 11611 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:07.055817 11604 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:07.065627  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:07.086483  6582 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:19:07.038087 11570 consensus_queue.cc:1059] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:19:07.056701 11331 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:07.058871 11156 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:19:07.083877 11462 consensus_meta.cc:287] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.570s	user 0.011s	sys 0.000s
I20260709 12:19:07.066349 11512 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.141607  7271 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:19:07.092056 11480 consensus_queue.cc:1059] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
W20260709 12:19:07.097226 11561 log.cc:927] Time spent T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 1.456s	user 0.000s	sys 0.010s
I20260709 12:19:07.099215 11340 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:07.140005  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2456) took 2153 ms. Trace:
I20260709 12:19:07.137399 11331 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:07.085666 11418 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:07.081202 11619 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:07.143453 11624 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.005s
W20260709 12:19:07.144526 11614 log.cc:927] Time spent T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
W20260709 12:19:07.174010 11615 log.cc:927] Time spent T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
W20260709 12:19:07.174597 11617 log.cc:927] Time spent T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
W20260709 12:19:07.147698 11341 consensus_meta.cc:287] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.919s	user 0.013s	sys 0.000s
I20260709 12:19:07.119576 11481 consensus_queue.cc:237] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:07.165362  6509 rpcz_store.cc:276] 0709 12:19:04.986710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.032113 (+1045403us) service_pool.cc:224] Handling call
0709 12:19:06.032423 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:06.032623 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:06.032647 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:06.033243 (+   596us) log.cc:844] Serialized 26 byte log entry
0709 12:19:06.681024 (+647781us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:06.710110 (+ 29086us) log.cc:844] Serialized 14 byte log entry
0709 12:19:06.718793 (+  8683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.718894 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:07.139792 (+420898us) raft_consensus.cc:1685] finished
0709 12:19:07.139813 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:07.139942 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34820,"prepare.queue_time_us":43818,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":34248,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"thread_start_us":602458,"threads_started":1,"wal-append.queue_time_us":773920}
W20260709 12:19:07.174664 11626 log.cc:927] Time spent T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 12:19:07.177745 11472 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:07.180373 11618 log.cc:927] Time spent T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 12:19:07.180598 11418 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:07.181075  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.181313 11462 raft_consensus.cc:3055] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:07.126281 11629 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:07.219050 11629 raft_consensus.cc:697] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:07.170840 11340 raft_consensus.cc:697] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:07.220048 11629 consensus_queue.cc:237] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:07.220904 11340 consensus_queue.cc:237] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:07.222409 11558 raft_consensus.cc:3055] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:07.222800 11558 raft_consensus.cc:740] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:07.185282 11580 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:07.189483 11611 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:07.161618 11579 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:07.190034  7273 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38562 (request call id 842) took 1108 ms. Trace:
I20260709 12:19:07.164379 11512 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:07.191624 11609 log.cc:927] Time spent T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 12:19:07.193352 11608 log.cc:927] Time spent T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.168s	user 0.001s	sys 0.001s
I20260709 12:19:07.138281 11616 consensus_queue.cc:1059] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:19:07.197422 11610 raft_consensus.cc:3055] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:07.194237 11331 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:07.199530 11621 log.cc:927] Time spent T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 12:19:07.169327 11592 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:07.190084 11603 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:07.212769 11462 raft_consensus.cc:740] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:07.275242 11156 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:07.283432 11554 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:07.311061  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:07.274504  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 188866us
I20260709 12:19:07.311607 11554 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:07.312831  6622 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:19:07.313977 11584 log.cc:927] Time spent T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.635s	user 0.000s	sys 0.001s
I20260709 12:19:07.314189 11637 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:07.314669 11637 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:07.315054 11637 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:07.314016  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:19:07.273725  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 221792us
W20260709 12:19:07.280203 11497 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.009s	sys 0.022s
I20260709 12:19:07.276835  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:07.277161  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:07.272125  7273 rpcz_store.cc:276] 0709 12:19:06.081574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.081806 (+   232us) service_pool.cc:224] Handling call
0709 12:19:06.082120 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:06.082503 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:19:06.698968 (+616465us) log.cc:844] Serialized 14 byte log entry
0709 12:19:06.914220 (+215252us) log.cc:844] Serialized 14 byte log entry
0709 12:19:06.915774 (+  1554us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:06.916358 (+   584us) log.cc:844] Serialized 26 byte log entry
0709 12:19:06.916523 (+   165us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:06.916572 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.916662 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:07.189816 (+273154us) raft_consensus.cc:1685] finished
0709 12:19:07.189845 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:07.189972 (+   127us) inbound_call.cc:177] Queueing success response
0709 12:19:07.204254 (+ 14282us) spinlock_profiling.cc:243] Waited 2.95 ms on lock 0x7b18004c25e0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
Metrics: {"mutex_wait_us":23153,"prepare.queue_time_us":700004,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":71,"raft.queue_time_us":27932,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":12393,"spinlock_wait_cycles":11173504,"thread_start_us":778884,"threads_started":2,"wal-append.queue_time_us":300297}
I20260709 12:19:07.380450  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:07.389689 11472 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:07.314272 11554 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:07.311173 11156 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:07.314220 11619 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:07.392896 11156 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:07.394378 11619 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:07.335861 11497 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.009s	sys 0.023s
I20260709 12:19:07.385174 11012 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:07.294305  6657 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:07.283286 11610 raft_consensus.cc:740] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:07.300366 11462 consensus_queue.cc:260] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:07.301178 11512 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:07.301600 11331 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:07.304926 11373 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:07.307945 11398 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:07.317112  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:07.317153  6473 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:07.323001  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:07.326109  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:07.419704  6649 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 7.
I20260709 12:19:07.328047  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:07.420276  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:07.421250  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:07.421598  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:07.422178  6652 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:07.422370  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:07.422967  6658 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:07.276407  7270 raft_consensus.cc:3055] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:07.424571  7270 raft_consensus.cc:740] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:07.425452  7270 consensus_queue.cc:260] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:07.426592  7270 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:07.275892  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:07.330796  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:19:07.429162  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 112114us
I20260709 12:19:07.429442  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:07.429906  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:07.399317  6656 raft_consensus.cc:2393] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 12: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:07.270803 11604 raft_consensus.cc:697] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:07.429982  6622 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:07.278882 11579 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:07.430820  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 113746us
I20260709 12:19:07.430971  6649 raft_consensus.cc:2393] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 14: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:07.251312 11558 consensus_queue.cc:260] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:07.445061  6473 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:07.431433  6652 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:19:07.379351 11510 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:07.430835  6653 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:07.434612  6622 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:07.448634  6473 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:07.450569 11112 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:07.451124 11112 raft_consensus.cc:697] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:07.452546  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:07.453328  6656 raft_consensus.cc:2393] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 13: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:07.416389 11316 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:07.464978  6622 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:07.440172 11277 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:07.441738 11619 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:07.442010 11462 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:07.445899 11156 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:07.474802  6658 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:19:07.474052  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:07.473935 11616 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:07.478870  6656 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:19:07.480110  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:07.452975 11391 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.410563 11012 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:07.453626 11331 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:07.487746  7270 raft_consensus.cc:1275] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 12:19:07.488452  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:07.488415  7270 pending_rounds.cc:85] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9: Aborting all ops after (but not including) 10
I20260709 12:19:07.489190  6579 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:19:07.489502  7270 pending_rounds.cc:107] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 12:19:07.485071  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:07.489813  7270 raft_consensus.cc:2889] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:07.486703  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:07.492856  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:07.493665  6575 raft_consensus.cc:2393] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 15: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:07.494526  6579 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:07.494611  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:07.495544  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:07.496853  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:07.482293 11510 raft_consensus.cc:2764] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:19:07.452011 11112 consensus_queue.cc:237] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:07.480223 11580 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:07.499374  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:07.501449  6584 raft_consensus.cc:2393] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 13: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:07.510738  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:07.511945  6579 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:07.512292  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:07.509850  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:07.516186  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:07.511099 11619 raft_consensus.cc:697] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:07.494156 11481 raft_consensus.cc:2764] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 12:19:07.520182  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:07.523634  6579 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
W20260709 12:19:07.504206  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 92302.9us
I20260709 12:19:07.500859  6509 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:19:07.474308 11316 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:07.505668  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:07.507766  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:07.508711  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:07.503419  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:07.505635  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:07.532712  6506 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:19:07.528064  6652 raft_consensus.cc:3055] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:07.504514 11340 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:07.530992  6584 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:07.536105  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:07.505298 11639 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:07.505123 11478 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:07.534696  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:07.540786  6658 raft_consensus.cc:1275] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:19:07.537925  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:07.490476 11391 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:07.518882 11389 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.547467  7269 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:07.533689  6652 raft_consensus.cc:740] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:07.500252 11629 consensus_queue.cc:1059] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:19:07.485682 11610 consensus_queue.cc:260] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:07.547349 11570 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:19:07.558077  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.525569 11604 consensus_queue.cc:237] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:07.565375 11570 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:07.567654 11570 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:07.539976 11340 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:07.547013 11478 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:07.570150 11340 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:07.571089  6505 raft_consensus.cc:2410] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 17 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 12:19:07.562156 11629 raft_consensus.cc:3055] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:07.562815 11619 consensus_queue.cc:237] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:07.579185  6652 consensus_queue.cc:260] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:07.582293 11658 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:07.583670  6652 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:07.548759 11331 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:19:07.571149 11478 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:07.551651 11389 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:07.577113 11629 raft_consensus.cc:740] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:19:07.591362 11650 log.cc:927] Time spent T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:19:07.591813 11184 raft_consensus.cc:3055] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:07.592211 11184 raft_consensus.cc:740] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:07.593257 11184 consensus_queue.cc:260] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:07.594274 11184 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:07.597183  7269 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:19:07.601133 11586 consensus_queue.cc:1048] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 12:19:07.546756 11639 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:07.605672 11639 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:07.609732  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.611624  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:07.616545 11316 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:07.595890 11629 consensus_queue.cc:260] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:07.618903 11629 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:07.625373 11327 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:07.634989 11585 log.cc:927] Time spent T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.906s	user 0.001s	sys 0.001s
I20260709 12:19:07.640497 11012 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:07.634795 11616 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:07.641371 11521 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.641498  6656 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:19:07.641932 11521 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:07.632637  6652 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 12:19:07.643952  6652 pending_rounds.cc:85] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 8
I20260709 12:19:07.644383  6652 pending_rounds.cc:107] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 12:19:07.644723  6652 raft_consensus.cc:2889] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:07.647101 11389 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:07.589500 11579 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:07.648514 11564 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 12:19:07.652740  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:07.653497  6509 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:19:07.650449  6550 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:07.655179  6506 raft_consensus.cc:1275] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:19:07.657483  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:07.633179  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:07.658145  6550 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:07.658427  6657 raft_consensus.cc:2393] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 15: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:07.658532  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:07.659842  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:19:07.660923  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.662320  6622 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:07.665848  6622 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:07.666895 11579 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:07.667073  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:07.667367 11579 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:07.667905 11579 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:07.668256 11616 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:07.668710 11616 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:07.669106 11616 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:07.671383  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:07.672664  6622 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:07.673487  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:07.674281  6584 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:19:07.675812  6550 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:07.663228 11184 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:07.679304  6622 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:07.683051  6550 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:07.684579  6622 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:19:07.685505  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.639190 11391 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:07.673053 11425 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:07.685855 11562 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:07.696982 11523 raft_consensus.cc:3055] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:07.694078  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:07.678918 11627 raft_consensus.cc:2804] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:07.699096  6550 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:07.681281 11473 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:07.680991 11511 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:07.686695 11184 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:07.585505 11658 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:07.683739 11629 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:07.701568  6622 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:07.687400 11496 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:07.679430 11012 consensus_queue.cc:1048] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:19:07.707850  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:07.688104 11521 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:07.698575 11510 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:07.711939  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:07.707046 11156 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:07.704464 11580 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:07.700840 11112 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:07.714608  6550 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:07.701988 11015 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:07.703043 11627 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:07.701997 11523 raft_consensus.cc:740] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:07.717263  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:07.705922 11421 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 12:19:07.713856 11511 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:07.722188  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:07.732600  6474 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:07.722775  6505 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:19:07.719867  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:07.718962  6550 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:07.717792 11629 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:07.722090 11156 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:07.747864  6550 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:19:07.743044 11638 log.cc:927] Time spent T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.134s	user 0.002s	sys 0.000s
I20260709 12:19:07.722213 11496 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:07.718815 11627 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:19:07.750093  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:07.747108 11580 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:07.745764 11428 raft_consensus.cc:2804] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:07.714025 11640 consensus_queue.cc:1059] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
W20260709 12:19:07.753073  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:07.783988 11647 log.cc:927] Time spent T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.307s	user 0.000s	sys 0.002s
I20260709 12:19:07.750738 11629 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:07.747098 11511 raft_consensus.cc:697] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:07.773739 11579 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:07.776991 11583 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:07.778765  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 2234) took 1451 ms. Trace:
I20260709 12:19:07.780715  6576 raft_consensus.cc:1240] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:07.800959  6583 rpcz_store.cc:276] 0709 12:19:06.326933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.327061 (+   128us) service_pool.cc:224] Handling call
0709 12:19:07.778236 (+1451175us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b5400057dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:07.778496 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3163737344}
I20260709 12:19:07.802052 11579 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:07.802174 11580 raft_consensus.cc:3055] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:07.802577 11580 raft_consensus.cc:740] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:07.803328  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:07.803557 11580 consensus_queue.cc:260] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:07.782362 11662 log.cc:927] Time spent T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:19:07.784250  6658 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:19:07.809629 11637 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:07.815308 11594 log.cc:927] Time spent T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 1.048s	user 0.000s	sys 0.005s
I20260709 12:19:07.817461  6657 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:07.817793 11511 consensus_queue.cc:237] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:07.784912  6655 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 12:19:07.820753  6655 pending_rounds.cc:85] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 8
I20260709 12:19:07.821048  6655 pending_rounds.cc:107] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:19:07.821285  6655 raft_consensus.cc:2889] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:07.755667 11496 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:07.822619 11562 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:07.822738 11637 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:07.822640 11496 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:07.786649  6578 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:19:07.788419  6654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2791) took 3190 ms. Trace:
I20260709 12:19:07.783892  6656 raft_consensus.cc:1275] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:19:07.825340  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:07.791081 11184 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:07.792688  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:07.829144  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:07.827548 11428 raft_consensus.cc:697] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:07.840976  6581 raft_consensus.cc:1240] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:07.844913  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:07.851946  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:07.852643  6583 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:07.853295  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:07.765444 11409 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:07.795225 11341 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.795763 11388 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 12. Current term is 13.
I20260709 12:19:07.752085 11398 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:07.863835 11428 consensus_queue.cc:237] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:07.868626 11398 raft_consensus.cc:697] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:07.869654 11398 consensus_queue.cc:237] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:07.762603  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:07.795894  6650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2799) took 2201 ms. Trace:
I20260709 12:19:07.719019 11112 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:07.798521  6550 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:07.781675 11658 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:07.806707 11616 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:07.799115 11629 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:07.824744  6654 rpcz_store.cc:276] 0709 12:19:04.598477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:05.807509 (+1209032us) service_pool.cc:224] Handling call
0709 12:19:05.807913 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:06.028262 (+220349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:06.028306 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:06.028861 (+   555us) log.cc:844] Serialized 26 byte log entry
0709 12:19:06.608309 (+579448us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:06.608402 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.788004 (+1179602us) spinlock_profiling.cc:243] Waited 220 ms on lock 0x7b54002a58d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:07.788057 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:07.788103 (+    46us) raft_consensus.cc:1685] finished
0709 12:19:07.788118 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:07.788359 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":870,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"spinlock_wait_cycles":505705344,"thread_start_us":562995,"threads_started":1,"wal-append.queue_time_us":710168,"wal-append.run_cpu_time_us":1884,"wal-append.run_wall_time_us":1002080}
I20260709 12:19:07.853291  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:07.792922 11627 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:07.857105  7265 raft_consensus.cc:3055] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:07.858173  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:07.862830  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:07.863512  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:07.750079 11523 consensus_queue.cc:260] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:07.843791 11015 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:07.830022 11327 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.873359  6650 rpcz_store.cc:276] 0709 12:19:05.594983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.144403 (+549420us) service_pool.cc:224] Handling call
0709 12:19:06.144697 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:06.268236 (+123539us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:06.268288 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:06.269524 (+  1236us) log.cc:844] Serialized 26 byte log entry
0709 12:19:07.525041 (+1255517us) spinlock_profiling.cc:243] Waited 25.1 ms on lock 0x7b1800374200. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:19:07.795476 (+270435us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b54001e2920. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df3090 00007f2875e006b8 00007f2875e8992b 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c
0709 12:19:07.795532 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:07.795576 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:07.795663 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:07.795701 (+    38us) raft_consensus.cc:1685] finished
0709 12:19:07.795713 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:07.795839 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":119301,"raft.queue_time_us":121652,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":28378,"spinlock_wait_cycles":89220352,"thread_start_us":32604,"threads_started":1,"wal-append.queue_time_us":1223105}
I20260709 12:19:07.859880 11654 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.833591 11637 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:19:07.886440  7265 raft_consensus.cc:740] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:07.866954 11590 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:07.878376 11409 raft_consensus.cc:697] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:07.884205 11421 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:07.895921  7265 consensus_queue.cc:260] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:07.905580  7265 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:07.903439 11590 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:07.918546  7271 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38562 (request call id 841) took 1847 ms (client timeout 1761 ms). Trace:
W20260709 12:19:07.919113  7271 rpcz_store.cc:269] 0709 12:19:06.070720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.070918 (+   198us) service_pool.cc:224] Handling call
0709 12:19:07.918197 (+1847279us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540037ad54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:07.918397 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2421998336}
I20260709 12:19:07.920636 11590 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:07.910306 11610 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:07.914770 11580 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:07.897119 11327 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:07.917745 11510 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:07.876827 11341 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:07.904906 11421 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:07.900286 11654 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:07.949378 11636 raft_consensus.cc:3055] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:07.949738 11654 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:07.950030 11636 raft_consensus.cc:740] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:07.951200 11636 consensus_queue.cc:260] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:07.958221 11524 raft_consensus.cc:3055] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:07.958803 11524 raft_consensus.cc:740] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:07.959815 11524 consensus_queue.cc:260] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:19:07.961208 11524 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:07.962944 11629 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:07.969022 11636 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:08.027717  6473 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:08.033358  6473 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:07.854663  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:07.863090  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2229) took 4230 ms. Trace:
I20260709 12:19:07.855582  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:08.038282  6508 rpcz_store.cc:276] 0709 12:19:03.632602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:03.632946 (+   344us) service_pool.cc:224] Handling call
0709 12:19:03.671020 (+ 38074us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:03.696795 (+ 25775us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:03.696843 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:03.708525 (+ 11682us) log.cc:844] Serialized 26 byte log entry
0709 12:19:05.980193 (+2271668us) spinlock_profiling.cc:243] Waited 8.37 ms on lock 0x7b18003477a0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:19:06.429469 (+449276us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:06.492191 (+ 62722us) log.cc:844] Serialized 14 byte log entry
0709 12:19:06.526167 (+ 33976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:06.526285 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:07.814633 (+1288348us) spinlock_profiling.cc:243] Waited 1.9 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:07.862849 (+ 48216us) raft_consensus.cc:1685] finished
0709 12:19:07.862876 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:07.863006 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":87566,"prepare.queue_time_us":224111,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":1727,"raft.queue_time_us":161528,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":15110,"spinlock_wait_cycles":23624832,"thread_start_us":449093,"threads_started":1,"wal-append.queue_time_us":2954072}
I20260709 12:19:08.041186  6500 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:08.042191  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:08.043470  6509 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:19:08.041880  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:08.044852  6502 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:19:08.045120  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:08.046645  6502 raft_consensus.cc:1275] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:19:08.046738  6503 raft_consensus.cc:2393] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 11: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:07.854763  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:08.048714  6504 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:19:07.889310  6582 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:07.891615  6583 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
W20260709 12:19:07.879859  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 81436.4us
I20260709 12:19:08.065328  6576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 680) took 1762 ms. Trace:
I20260709 12:19:08.066025  6576 rpcz_store.cc:276] 0709 12:19:06.303074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.324599 (+ 21525us) service_pool.cc:224] Handling call
0709 12:19:07.780510 (+1455911us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:07.801580 (+ 21070us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:08.064775 (+263195us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b5400057dd0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:08.065086 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3227634560}
I20260709 12:19:07.825344  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:08.069659  6505 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:08.070088  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:08.072518  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:08.075140  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:08.080994  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:08.081988  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.083595  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:08.084934  6501 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:08.088421 10963 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:08.090657 11112 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:08.093220 11331 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:08.094619 11340 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:19:08.094354 11331 consensus_queue.cc:1059] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 12.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:19:08.098896 11331 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:08.099891 11331 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:07.945768 11421 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:07.944949 11409 consensus_queue.cc:237] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:08.124830 11478 raft_consensus.cc:3055] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:08.107894 11327 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:08.076185  6581 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
W20260709 12:19:08.156840  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 72558.6us
I20260709 12:19:08.157377 11331 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:08.163900  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:08.170770 11462 consensus_meta.cc:287] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.687s	user 0.008s	sys 0.008s
I20260709 12:19:08.203739  6505 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:19:08.130280 11478 raft_consensus.cc:740] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:08.212473  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:08.212977  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:08.276369 11627 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:08.276582  6502 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:19:08.277123  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:08.277690  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:08.278438  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:08.278855  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:08.279155  6507 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:08.279628  6502 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
I20260709 12:19:08.214772 11112 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.280746 11112 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:08.283619 11112 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:08.273650 11550 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 12:19:08.218014 11523 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:08.219542  6504 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:08.286193  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:08.286715  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:08.287002  6649 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
I20260709 12:19:08.287509  6504 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:19:08.220278  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:08.288828  6503 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:19:08.289614 11558 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:08.290575  6503 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:19:08.237591  6506 raft_consensus.cc:1275] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 12:19:08.293121  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.295059  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:08.239995 11331 consensus_queue.cc:1059] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 12.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:19:08.241863  6582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 687) took 1022 ms. Trace:
I20260709 12:19:08.226061 11616 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:08.258570 11629 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:08.302894  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:08.265864  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.267081  6576 raft_consensus.cc:1275] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:08.271718  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:08.272871 11015 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.273067  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:08.273612 10999 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:19:08.288940  6657 raft_consensus.cc:1240] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:08.292207  6503 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:19:08.296139  6506 raft_consensus.cc:1240] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:08.255298  6475 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:08.307355  6500 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:19:08.308647  6502 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:08.309247 11627 consensus_queue.cc:1048] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:08.310436  6502 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 12:19:08.310801  6502 pending_rounds.cc:85] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 6
I20260709 12:19:08.311059  6502 pending_rounds.cc:107] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:19:08.311383  6502 raft_consensus.cc:2889] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:08.296680  6582 rpcz_store.cc:276] 0709 12:19:07.219039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:07.219212 (+   173us) service_pool.cc:224] Handling call
0709 12:19:07.219515 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:08.050552 (+831037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:08.241444 (+190892us) spinlock_profiling.cc:243] Waited 670 ms on lock 0x7b540032a854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:08.241691 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1539920256}
I20260709 12:19:08.316154  6501 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:19:08.279361 11421 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:08.321612 11015 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:08.322118 11421 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:08.329815  6502 raft_consensus.cc:1240] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:08.329698 11015 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:08.293143 11341 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:08.333494 11331 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:08.361891 11331 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.364205 11331 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:08.364218  6584 raft_consensus.cc:2393] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 12: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:08.369964  6502 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:19:08.371520  6506 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:08.374511  6503 raft_consensus.cc:1240] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:08.373122  7240 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:08.389284  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 313889us
I20260709 12:19:08.394964  6505 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 14. Current term is 15. Ops: [13.11-14.12]
I20260709 12:19:08.399230  7265 raft_consensus.cc:1275] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 12:19:08.399791  7265 pending_rounds.cc:85] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9: Aborting all ops after (but not including) 10
I20260709 12:19:08.400060  7265 pending_rounds.cc:107] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 12:19:08.400182  6503 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:08.400388  7265 raft_consensus.cc:2889] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:08.401690  6504 raft_consensus.cc:1240] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
W20260709 12:19:08.413223  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 173112us
I20260709 12:19:08.419067 11637 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:19:08.419800  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:08.421926  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.425208 11341 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.425834 11341 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:08.427999 11341 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:08.433099  7269 raft_consensus.cc:1240] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:08.437119 11478 consensus_queue.cc:260] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.5, Last appended by leader: 5, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:08.439141 11478 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:08.441180  6509 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:08.444651 11510 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:08.445358 11510 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:08.447626 11510 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:08.450325  6649 pending_rounds.cc:85] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 7
I20260709 12:19:08.450778  6649 pending_rounds.cc:107] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:19:08.451056  6649 raft_consensus.cc:2889] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:08.397365 11616 consensus_queue.cc:1048] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
W20260709 12:19:08.469702  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 52481.2us
I20260709 12:19:08.472410 11398 consensus_queue.cc:1059] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:08.473662 11398 raft_consensus.cc:3055] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:08.474026 11398 raft_consensus.cc:740] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:08.474944 11398 consensus_queue.cc:260] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:08.476184 11398 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:08.485186  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 144656us
I20260709 12:19:08.487792  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:08.496138  6503 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:19:08.503837  6622 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:08.504065 11341 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.504988 11341 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:08.505760  6622 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:08.507838 11341 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:08.508814  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:08.509289  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.512737 11367 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:08.513193 11367 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:08.513540 11367 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:08.514146  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:08.516611  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:08.516891  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:08.518730  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:08.519124 11480 consensus_queue.cc:1059] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:08.508831 11579 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:08.520354 11579 raft_consensus.cc:697] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:08.520174 11480 consensus_queue.cc:1243] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer 496c7c3792cd4587b4dbeba435f4be1e log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:19:08.521287  6579 raft_consensus.cc:1240] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:08.521427 11513 raft_consensus.cc:3055] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:08.522004 11513 raft_consensus.cc:740] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:08.523288 11513 consensus_queue.cc:260] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:08.520655  6582 raft_consensus.cc:1275] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:08.525985 11513 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:08.522053 11564 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:08.529794 11564 raft_consensus.cc:697] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:19:08.530587 11671 log.cc:927] Time spent T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.164s	user 0.001s	sys 0.001s
I20260709 12:19:08.531019 11564 consensus_queue.cc:237] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:08.513562  6583 raft_consensus.cc:1240] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15. Ops: []
W20260709 12:19:08.536375  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 50300us
I20260709 12:19:08.537549  6575 raft_consensus.cc:1275] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
W20260709 12:19:08.539355 11672 log.cc:927] Time spent T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 12:19:08.541878  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:08.542681  6505 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:08.544358  7268 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:19:08.546205  7273 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 11. (index mismatch)
I20260709 12:19:08.516194  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:08.554577  6473 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:08.555598 11478 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:08.556286 11478 raft_consensus.cc:697] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:08.557601 11478 consensus_queue.cc:237] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:08.565182  6656 raft_consensus.cc:1240] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:08.566488  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:08.567008 10999 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.568001  6509 raft_consensus.cc:1275] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:08.569945  6507 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:19:08.571424  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:19:08.568042  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.574457  6653 raft_consensus.cc:1240] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:08.575217  6505 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:19:08.567799 10999 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:08.577298  6657 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:08.583590  6655 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:19:08.582542  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:08.597548  6652 raft_consensus.cc:1275] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:19:08.597952  6584 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:19:08.601413  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:08.601668  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:08.521543 11579 consensus_queue.cc:237] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:08.605370  6575 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:19:08.599062  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:08.608843 11341 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:08.609746  6654 raft_consensus.cc:1240] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:08.611086 11012 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:08.611187 11112 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:08.613924 10999 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:08.589382 11496 consensus_meta.cc:287] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.608s	user 0.007s	sys 0.009s
I20260709 12:19:08.614924 11496 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:08.616281  6577 raft_consensus.cc:1240] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:08.617383 11496 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:08.581943 11480 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.618335 11480 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:08.620311  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:08.622433  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:08.623900  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:08.624680  6653 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:08.625710 11481 consensus_queue.cc:1048] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 12:19:08.629552  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:08.631443  6652 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:19:08.632855  6654 raft_consensus.cc:1240] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:08.633940  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:08.635610  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:08.636444  6651 raft_consensus.cc:2393] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 12: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:08.637705  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:08.639451  6651 raft_consensus.cc:1275] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:08.586828 11616 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:08.641759 11616 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.642391 11616 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:08.643764  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 68699.1us
I20260709 12:19:08.645056 11327 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.645331  6473 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:08.645666 11327 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:08.647163 11480 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:08.648890 11673 log.cc:927] Time spent T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.215s	user 0.004s	sys 0.001s
I20260709 12:19:08.650013  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:08.651865  6651 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 12.
I20260709 12:19:08.654696  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:08.656656  6655 raft_consensus.cc:1240] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:08.657543 11616 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:08.648031 11327 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:08.658388  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 55432.5us
I20260709 12:19:08.659106  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:08.659957  6501 raft_consensus.cc:1275] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 12:19:08.660048 11481 consensus_queue.cc:1048] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:08.660419  6622 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:08.661182 11629 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:08.661587 11481 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:08.661722 11629 raft_consensus.cc:697] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:08.662791 11629 consensus_queue.cc:237] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:08.657754  7240 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:08.666457  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:08.666929  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.667586  7238 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:08.670594  6582 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:08.680357  6508 raft_consensus.cc:1275] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:19:08.690274  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:08.692801  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:08.695302  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.675457  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:08.700707  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:08.697813  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:08.701730  6504 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:08.693796  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:08.703822  6654 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:08.705698  6501 raft_consensus.cc:2410] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260709 12:19:08.709591  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.714193  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:08.715838  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:08.725818 11474 consensus_queue.cc:1059] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
W20260709 12:19:08.727183  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.729565  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:08.731154  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:08.706259 11331 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:08.735466 11331 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:08.736487 11654 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:08.737779  7271 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 12:19:08.738307  7271 pending_rounds.cc:85] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9: Aborting all ops after (but not including) 11
I20260709 12:19:08.738693  7271 pending_rounds.cc:107] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 12:19:08.739035  7271 raft_consensus.cc:2889] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:08.741006  6652 raft_consensus.cc:2410] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:19:08.741773  6475 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:08.731050  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:08.745450  6580 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 11. (index mismatch)
W20260709 12:19:08.752297  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 43680.3us
I20260709 12:19:08.755447  6475 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:08.703944 11367 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:08.773854  6576 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:08.771441 11636 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:08.773058 11481 consensus_queue.cc:1048] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:19:08.765921 11616 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.776503 11474 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:08.789463  7270 raft_consensus.cc:1240] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:08.794636 11550 consensus_queue.cc:1059] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:08.798537  6582 raft_consensus.cc:1240] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:08.800156  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:08.801113  6579 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:19:08.769950 11012 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:08.804126  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:08.808223  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:08.800179  6575 raft_consensus.cc:1240] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:19:08.800787  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:08.817292  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:08.817996  6577 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 11.
I20260709 12:19:08.819514  6654 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
W20260709 12:19:08.824739  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 44923.5us
I20260709 12:19:08.829494  6504 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:19:08.832077  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:08.846002  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:08.846876  6576 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:08.821910 11481 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:08.858280  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 102953us
I20260709 12:19:08.860750  6652 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:19:08.863888  6577 raft_consensus.cc:1275] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:19:08.837479 11331 raft_consensus.cc:2764] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:19:08.828857 11012 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:08.874203  6475 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:08.878235  6475 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:08.879645  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:08.880481  6504 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:08.837158 11523 consensus_queue.cc:1059] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:19:08.823762 11616 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:08.886571  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:08.883991  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:08.958552  6652 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:08.888726  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:08.961759  6651 raft_consensus.cc:2410] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:19:08.885651  6549 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:08.878512 11012 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:08.985603  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:08.987598  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 109473us
I20260709 12:19:08.989322  6475 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:08.990707 11341 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:08.992724 11341 consensus_queue.cc:1059] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:19:08.993707  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 111685us
I20260709 12:19:08.993918 11341 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:08.994586 11341 raft_consensus.cc:3055] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:08.994897 11341 raft_consensus.cc:740] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:08.998082 11112 consensus_queue.cc:1048] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:09.000564 11409 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:19:08.968418 11550 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:19:08.957746 11474 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:08.880416 11481 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:09.006405 11481 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:09.006701 11551 raft_consensus.cc:3055] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:09.007289 11551 raft_consensus.cc:740] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:09.008936 11327 raft_consensus.cc:3055] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:09.009443 11327 raft_consensus.cc:740] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:09.010571 11327 consensus_queue.cc:260] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:09.013536  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:09.015084  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:09.024026  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:09.020421  6649 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.030s	sys 0.104s
W20260709 12:19:09.026450  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:09.026718  6649 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.030s	sys 0.104s
I20260709 12:19:09.011094 11616 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:09.028487  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:09.032203  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:09.030313  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 50944.6us
I20260709 12:19:09.056567  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:08.977243 11636 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:09.061368  6652 raft_consensus.cc:2393] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 11: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:09.063225  6582 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:09.071985  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:09.041620 11367 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:09.056115 11510 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:09.043802 11654 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:09.096961 11474 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:09.097265  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:09.062321 11636 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:09.093940  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:09.104120  6584 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
I20260709 12:19:09.091264 11510 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:09.093080 11327 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:09.027446 11551 consensus_queue.cc:260] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:09.101802 11636 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:09.111636  6548 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:09.137928  6652 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:09.092233  6549 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:09.093648 11409 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:19:09.116940 11510 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:09.113116 11421 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.013s	sys 0.033s
I20260709 12:19:09.164208 11551 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:09.142181 11341 consensus_queue.cc:260] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:09.174448  6401 catalog_manager.cc:5697] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: config changed from index 3 to 6, term changed from 5 to 6, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to <none>, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New cstate: current_term: 6 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:19:09.186319  6407 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57914 (request call id 191) took 29190 ms (client timeout 15000 ms). Trace:
I20260709 12:19:09.186249  6401 catalog_manager.cc:5697] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
W20260709 12:19:09.189218 11473 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.058s	sys 0.050s
I20260709 12:19:09.188164  6401 catalog_manager.cc:5697] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:19:09.189656 11473 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.058s	sys 0.050s
I20260709 12:19:09.191111  6401 catalog_manager.cc:5697] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 9 to 10, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 10 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:19:09.192867 11112 consensus_queue.cc:1059] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:19:09.176813  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 92230.5us
I20260709 12:19:09.104207 11654 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:09.208124  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 47083us
I20260709 12:19:09.210253  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:09.211107  6652 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:09.207134 11421 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.014s	sys 0.034s
I20260709 12:19:09.214359 11474 raft_consensus.cc:3055] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:09.216009  6473 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:09.205176 11550 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:09.216059 11409 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:09.226182 11341 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:09.216954 11474 raft_consensus.cc:740] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:09.232928  6652 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
I20260709 12:19:09.236840  7240 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:09.238030 11623 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:09.238629 11623 raft_consensus.cc:697] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 16 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:09.239590 11623 consensus_queue.cc:237] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:09.241358  6652 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:19:09.213425 11012 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:09.160207 11331 consensus_queue.cc:1059] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
W20260709 12:19:09.255410  6583 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.007s	sys 0.024s
W20260709 12:19:09.260133  6583 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.007s	sys 0.024s
W20260709 12:19:09.254748 11478 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.012s	sys 0.018s
W20260709 12:19:09.259048  7269 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.019s	sys 0.012s
W20260709 12:19:09.273212  7269 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.019s	sys 0.013s
W20260709 12:19:09.256839 11579 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.015s	sys 0.016s
W20260709 12:19:09.274387  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:09.281797  6506 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.014s	sys 0.019s
W20260709 12:19:09.282188  6506 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.014s	sys 0.019s
W20260709 12:19:09.276315  7273 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.011s	sys 0.021s
W20260709 12:19:09.285431  7273 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.011s	sys 0.021s
W20260709 12:19:09.286643  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 191605us
W20260709 12:19:09.272850 11478 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.013s	sys 0.019s
I20260709 12:19:09.282955  6401 catalog_manager.cc:5697] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:19:09.281652 11579 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.016s	sys 0.016s
W20260709 12:19:09.307044  6505 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.019s	sys 0.031s
I20260709 12:19:09.309856 11388 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:09.316953  6505 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.020s	sys 0.031s
I20260709 12:19:09.312527 11023 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:09.312983  6650 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.005s	sys 0.026s
W20260709 12:19:09.319039  6650 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.005s	sys 0.026s
I20260709 12:19:09.259505 11012 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:09.300565 11564 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.008s	sys 0.026s
I20260709 12:19:09.291884 11480 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:09.322500  6655 raft_consensus.cc:1240] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:09.317867 11367 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:19:09.321835 11564 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.008s	sys 0.026s
I20260709 12:19:09.342396 11012 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:09.344508  6548 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:09.344724  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:09.345130  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.345335 11421 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:09.187011  6407 rpcz_store.cc:269] 0709 12:18:39.995585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:39.995846 (+   261us) service_pool.cc:224] Handling call
0709 12:18:40.713604 (+717758us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b540008acd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df3090 00007f2875e91f3a 00007f287bc47e9c 00007f287bd4604e 00007f287a2072a4 00007f287a207232 00007f287a207161 00007f287a2070dc 00007f287a206392 00007f28741c448c 00007f28741c38c6
0709 12:19:09.186015 (+28472411us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:18:59.826855 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:18:59.924954 (+ 98099us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:18:59.925010 (+    56us) write_op.cc:435] Acquired schema lock
0709 12:18:59.925027 (+    17us) tablet.cc:662] Decoding operations
0709 12:18:59.961873 (+ 36846us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:18:59.961975 (+   102us) write_op.cc:641] Partition lock acquired for write op
0709 12:18:59.962006 (+    31us) tablet.cc:685] Acquiring locks for 24 operations
0709 12:19:00.107060 (+145054us) tablet.cc:701] Row locks acquired
0709 12:19:00.107112 (+    52us) write_op.cc:260] PREPARE: finished
0709 12:19:00.107317 (+   205us) write_op.cc:270] Start()
0709 12:19:00.107490 (+   173us) write_op.cc:276] Timestamp: P: 1783599540107244 usec, L: 0
0709 12:19:00.107526 (+    36us) op_driver.cc:348] REPLICATION: starting
0709 12:19:00.108382 (+   856us) log.cc:844] Serialized 9316 byte log entry
0709 12:19:01.540750 (+1432368us) spinlock_profiling.cc:243] Waited 6.88 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:01.837729 (+296979us) op_driver.cc:464] REPLICATION: finished
0709 12:19:02.318471 (+480742us) write_op.cc:301] APPLY: starting
0709 12:19:02.348454 (+ 29983us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:03.798273 (+1449819us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:03.805437 (+  7164us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:04.192262 (+386825us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:04.192725 (+   463us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=48,key_file_lookups=48,delta_file_lookups=0,mrs_lookups=0
0709 12:19:04.192790 (+    65us) write_op.cc:312] APPLY: finished
0709 12:19:04.193345 (+   555us) log.cc:844] Serialized 212 byte log entry
0709 12:19:04.193566 (+   221us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:04.193757 (+   191us) write_op.cc:454] Released schema lock
0709 12:19:04.193836 (+    79us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:04.254071 (+ 60235us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:04.255819 (+  1748us) write_op.cc:454] Released schema lock
0709 12:19:04.583759 (+327940us) spinlock_profiling.cc:243] Waited 49.5 ms on lock 0x7b180024d480. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c522 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
Metrics: {"mutex_wait_us":1243371,"reported_tablets":52,"spinlock_wait_cycles":446904192,"child_traces":[["op",{"apply.queue_time_us":396966,"apply.run_cpu_time_us":42161,"apply.run_wall_time_us":1931222,"cfile_cache_hit":98,"cfile_cache_hit_bytes":104255,"num_ops":24,"prepare.queue_time_us":292266,"prepare.run_cpu_time_us":23698,"prepare.run_wall_time_us":503576,"raft.queue_time_us":170188,"raft.run_cpu_time_us":39265,"raft.run_wall_time_us":246725,"replication_time_us":1730002,"spinlock_wait_cycles":129592576,"thread_start_us":508463,"threads_started":4,"wal-append.queue_time_us":465879,"wal-append.run_cpu_time_us":24936,"wal-append.run_wall_time_us":3673395}]]}
I20260709 12:19:09.346251 11654 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:09.335844 11023 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:19:09.353960  6508 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:09.324747  6401 catalog_manager.cc:5697] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 4 to 6, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 6 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:19:09.300316 11474 consensus_queue.cc:260] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:09.353485 11112 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:19:09.353416  6550 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.008s	sys 0.017s
W20260709 12:19:09.362941  6550 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.008s	sys 0.017s
W20260709 12:19:09.371742  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:09.363319  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 570310us
I20260709 12:19:09.324440 11388 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:09.358728 11409 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:19:09.365489  6656 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.007s	sys 0.018s
W20260709 12:19:09.374527  6656 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.008s	sys 0.018s
I20260709 12:19:09.353021 11367 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:09.382687  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:09.398849  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:09.400244  6500 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.862s	user 0.034s	sys 0.033s
I20260709 12:19:09.360502 11421 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:09.385118  7265 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.019s	sys 0.009s
W20260709 12:19:09.405135  7265 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.019s	sys 0.010s
W20260709 12:19:09.405964  6500 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.868s	user 0.035s	sys 0.033s
W20260709 12:19:09.400410  6501 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.014s	sys 0.019s
W20260709 12:19:09.406921  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:09.407493  6501 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.014s	sys 0.019s
W20260709 12:19:09.407648  6657 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.009s	sys 0.019s
W20260709 12:19:09.409113  6657 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.010s	sys 0.019s
I20260709 12:19:09.402904  6550 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:09.425107  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 101091us
I20260709 12:19:09.379798 11474 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:09.435519  6401 catalog_manager.cc:5697] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 7 to 9, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 9 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
W20260709 12:19:09.402788 11398 consensus_meta.cc:287] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.903s	user 0.009s	sys 0.004s
I20260709 12:19:09.433807  6550 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:09.483155  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 49459.8us
I20260709 12:19:09.427107 11629 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:09.480347  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:09.495359  6401 catalog_manager.cc:5697] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 4 to 8, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 8 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:19:09.493017  6581 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:19:09.496281  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:09.497201  6655 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 12.
W20260709 12:19:09.498742  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:09.503259 11699 log.cc:927] Time spent T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:19:09.443075 11409 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:09.495733  7270 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:19:09.510468  7240 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:09.492288 11579 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:09.497607 11629 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:09.481179 11480 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:09.496502 11496 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:09.522028  6401 catalog_manager.cc:5697] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 7 to 8, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 8 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:19:09.520232  7238 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.015s	sys 0.039s
W20260709 12:19:09.581964  7238 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.016s	sys 0.040s
W20260709 12:19:09.582731  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 705597us
I20260709 12:19:09.583585  6548 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:09.581359  6653 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.889s	user 0.009s	sys 0.028s
W20260709 12:19:09.589874  6653 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.898s	user 0.009s	sys 0.028s
I20260709 12:19:09.519526 11654 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:09.515502 11409 raft_consensus.cc:697] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:09.574307 11629 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:09.581779 11480 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:09.590466 11523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.033s	sys 0.024s
W20260709 12:19:09.608633  6475 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.017s	sys 0.016s
I20260709 12:19:09.616485  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:09.618263  7270 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
W20260709 12:19:09.619603  6475 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.018s	sys 0.016s
I20260709 12:19:09.599992 11579 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:09.694092  6504 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.027s	sys 0.017s
I20260709 12:19:09.647060  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 2262) took 1115 ms. Trace:
W20260709 12:19:09.720523  6504 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.030s	sys 0.019s
W20260709 12:19:09.708482  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 719146us
W20260709 12:19:09.690503  7271 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.014s	sys 0.030s
W20260709 12:19:09.723337  7271 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.014s	sys 0.035s
W20260709 12:19:09.633836  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 123890us
I20260709 12:19:09.723734  6475 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:09.693526  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 74291.3us
I20260709 12:19:09.633893  6623 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:09.607640 11654 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:09.614228 11015 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:19:09.621503 11523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.033s	sys 0.025s
I20260709 12:19:09.626410 11473 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:09.744844  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 425707us
I20260709 12:19:09.747596  6655 raft_consensus.cc:1275] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:19:09.634467 11564 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:09.628986 11112 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:09.709584 11367 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:09.633814 11683 log.cc:927] Time spent T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:19:09.623811 11277 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:09.634845  6401 catalog_manager.cc:5697] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 6 to 8, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 8 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:19:09.780596  6475 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:09.648955 11023 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:09.728763  6575 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.000s	sys 0.059s
W20260709 12:19:09.792912  6575 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.004s	sys 0.059s
W20260709 12:19:09.729979  6654 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.710s	user 0.010s	sys 0.014s
I20260709 12:19:09.782999  6473 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:09.733940 11421 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:09.725003  6581 rpcz_store.cc:276] 0709 12:19:08.511449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.516468 (+  5019us) service_pool.cc:224] Handling call
0709 12:19:09.626975 (+1110507us) spinlock_profiling.cc:243] Waited 975 ms on lock 0x7b540034c6d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:09.627119 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2242606464}
I20260709 12:19:09.795367  6548 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:09.743710 11654 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:09.798736  6584 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.000s	sys 0.028s
W20260709 12:19:09.793483  6654 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.011s	sys 0.014s
W20260709 12:19:09.799145  6584 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.000s	sys 0.028s
W20260709 12:19:09.755841  7268 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.025s	sys 0.025s
W20260709 12:19:09.800159  7268 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.027s	sys 0.027s
I20260709 12:19:09.802830 11421 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:09.781431 11112 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:09.752321 11388 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:09.747824 11623 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 12:19:09.803288 11421 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:09.787554 11331 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:19:09.802503 11564 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:09.843516 11616 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:19:09.844153  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 88847.3us
I20260709 12:19:09.845337  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:09.845625 11685 log.cc:927] Time spent T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.274s	user 0.001s	sys 0.001s
W20260709 12:19:09.846783 11327 consensus_meta.cc:287] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.729s	user 0.011s	sys 0.000s
I20260709 12:19:09.850801  6650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2836) took 1393 ms. Trace:
W20260709 12:19:09.862802  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.853235  6401 catalog_manager.cc:5697] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 2 to 8, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 8 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:09.743592 11409 consensus_queue.cc:237] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:09.852464  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 367514us
I20260709 12:19:09.897404 11513 raft_consensus.cc:3055] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:19:09.897879 11690 log.cc:927] Time spent T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.286s	user 0.000s	sys 0.001s
W20260709 12:19:09.899487  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:09.900009  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.900837 11015 consensus_queue.cc:1059] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:09.901079 11478 raft_consensus.cc:3055] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:09.882720 11327 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:19:09.884651 11692 log.cc:927] Time spent T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260709 12:19:09.902829 11496 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:19:09.884922  6658 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.010s	sys 0.036s
I20260709 12:19:09.903492 11496 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:09.904052  6658 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.887s	user 0.011s	sys 0.036s
I20260709 12:19:09.904053  6506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2275) took 1222 ms. Trace:
W20260709 12:19:09.885613 11701 log.cc:927] Time spent T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 12:19:09.905061 11720 consensus_queue.cc:1048] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:09.905148 11711 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
W20260709 12:19:09.886273  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:09.906849  6657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2851) took 1215 ms. Trace:
I20260709 12:19:09.907249  6657 rpcz_store.cc:276] 0709 12:19:08.691220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.700629 (+  9409us) service_pool.cc:224] Handling call
0709 12:19:08.700971 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:08.751484 (+ 50513us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:08.751530 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:08.766852 (+ 15322us) log.cc:844] Serialized 26 byte log entry
0709 12:19:09.409567 (+642715us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:09.409622 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:09.409710 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:09.906643 (+496933us) raft_consensus.cc:1685] finished
0709 12:19:09.906666 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:09.906794 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36476,"raft.queue_time_us":46239,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":30803,"thread_start_us":607600,"threads_started":1,"wal-append.queue_time_us":851868}
I20260709 12:19:09.852562 11388 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:09.909134 11388 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:09.886575 11696 log.cc:927] Time spent T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 12:19:09.886101 11687 log.cc:927] Time spent T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 12:19:09.794911 11473 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:09.893177 11277 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:19:09.893430  6472 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.007s	sys 0.048s
W20260709 12:19:09.936854  6472 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.768s	user 0.008s	sys 0.048s
W20260709 12:19:09.937978  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:09.893913  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 45850.9us
I20260709 12:19:09.894248 11704 consensus_queue.cc:1059] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:09.895512 11717 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:09.897838 11513 raft_consensus.cc:740] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:09.901607 11478 raft_consensus.cc:740] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:09.905606  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2861) took 1133 ms. Trace:
I20260709 12:19:09.909509  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:09.883930 11691 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
I20260709 12:19:09.889599 11708 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:19:09.891203 11700 log.cc:927] Time spent T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
W20260709 12:19:09.892688  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 112682us
I20260709 12:19:09.850364 11012 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:09.855621 11421 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:09.904976  6506 rpcz_store.cc:276] 0709 12:19:08.681657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.688326 (+  6669us) service_pool.cc:224] Handling call
0709 12:19:08.688741 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:08.689153 (+   412us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:08.689184 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:08.689664 (+   480us) log.cc:844] Serialized 26 byte log entry
0709 12:19:09.282591 (+592927us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:09.282641 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:09.282701 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:09.903775 (+621074us) raft_consensus.cc:1685] finished
0709 12:19:09.903803 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:09.903957 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4804,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":9746,"thread_start_us":592582,"threads_started":1,"wal-append.queue_time_us":915296}
I20260709 12:19:09.942785  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2858) took 1171 ms. Trace:
I20260709 12:19:09.980902  6401 catalog_manager.cc:5697] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 9 to 10, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 10 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:19:09.877357  6650 rpcz_store.cc:276] 0709 12:19:08.442640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.477892 (+ 35252us) service_pool.cc:224] Handling call
0709 12:19:08.577113 (+ 99221us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:08.601523 (+ 24410us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:19:09.319420 (+717897us) log.cc:844] Serialized 14 byte log entry
0709 12:19:09.834601 (+515181us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:09.835001 (+   400us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:09.835369 (+   368us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:09.835753 (+   384us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:09.836198 (+   445us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":705508,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"thread_start_us":1171439,"threads_started":2}
I20260709 12:19:09.878851 11616 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:09.901711 11015 consensus_queue.cc:1243] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer f2b06aebe97249a4877d798b8118ece4 log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:19:09.982002  6655 rpcz_store.cc:276] 0709 12:19:08.772263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.515025 (+742762us) service_pool.cc:224] Handling call
0709 12:19:09.582494 (+ 67469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:09.754875 (+172381us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:09.905348 (+150473us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b54002a53d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:09.905539 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":378795264}
I20260709 12:19:09.902861 11327 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:19:09.980614 11694 log.cc:927] Time spent T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.112s	user 0.003s	sys 0.000s
W20260709 12:19:09.936425 11636 consensus_meta.cc:287] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.724s	user 0.007s	sys 0.005s
I20260709 12:19:09.900837 11564 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:10.007128  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 131313us
I20260709 12:19:10.012244  6475 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:09.997733  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 214764us
I20260709 12:19:10.011000  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2850) took 1320 ms. Trace:
I20260709 12:19:09.942399 11277 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:10.022409  6650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 798) took 1184 ms. Trace:
I20260709 12:19:10.023093  6650 rpcz_store.cc:276] 0709 12:19:08.837878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.994329 (+1156451us) service_pool.cc:224] Handling call
0709 12:19:10.021613 (+ 27284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:10.021889 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:10.021939 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:10.022001 (+    62us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:10.022057 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:10.022138 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:10.022321 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:09.980108 11550 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:09.983978 11712 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.032080  6658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2194) took 1469 ms. Trace:
I20260709 12:19:09.994803 11616 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:10.035017  6473 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:10.052855  6548 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:10.013288  6656 rpcz_store.cc:276] 0709 12:19:08.690843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.701831 (+ 10988us) service_pool.cc:224] Handling call
0709 12:19:08.707873 (+  6042us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:08.715559 (+  7686us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:08.715600 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:08.732706 (+ 17106us) log.cc:844] Serialized 26 byte log entry
0709 12:19:09.379881 (+647175us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:09.379926 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:09.380004 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:10.010742 (+630738us) raft_consensus.cc:1685] finished
0709 12:19:10.010764 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:10.010920 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26897,"raft.queue_time_us":3342,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":2375,"thread_start_us":641851,"threads_started":1,"wal-append.queue_time_us":1024357}
I20260709 12:19:09.983999  6649 rpcz_store.cc:276] 0709 12:19:08.771332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.347680 (+576348us) service_pool.cc:224] Handling call
0709 12:19:09.348068 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:09.348417 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:09.348765 (+   348us) log.cc:844] Serialized 14 byte log entry
0709 12:19:09.942278 (+593513us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:09.942350 (+    72us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:09.942428 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:09.942530 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:09.942705 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":130147,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":4611,"raft.queue_time_us":188541,"thread_start_us":585851,"threads_started":2}
I20260709 12:19:10.029909  6657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 800) took 1017 ms. Trace:
I20260709 12:19:10.055346  6657 rpcz_store.cc:276] 0709 12:19:09.012987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.029024 (+1016037us) service_pool.cc:224] Handling call
0709 12:19:10.029356 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:10.029559 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:10.029585 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:10.029627 (+    42us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:10.029658 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:10.029720 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:10.029846 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:10.006044 11015 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:10.006374 11327 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:10.009127 11481 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.066361 11327 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:10.066571 11481 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:10.060235  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 470553us
I20260709 12:19:10.063007  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 799) took 1050 ms. Trace:
I20260709 12:19:10.068979  6655 rpcz_store.cc:276] 0709 12:19:09.012674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.035946 (+1023272us) service_pool.cc:224] Handling call
0709 12:19:10.062619 (+ 26673us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:10.062745 (+   126us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:10.062759 (+    14us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:10.062786 (+    27us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:10.062805 (+    19us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:10.062835 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:10.062952 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:09.990093 11012 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:10.042138 11579 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:10.042632 11331 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:19:10.080597  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 70625.2us
I20260709 12:19:10.029932 11723 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:10.065377 11708 raft_consensus.cc:2764] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
W20260709 12:19:10.077477 11341 consensus_meta.cc:287] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.851s	user 0.013s	sys 0.008s
I20260709 12:19:10.041862  6658 rpcz_store.cc:276] 0709 12:19:08.562710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.567179 (+  4469us) service_pool.cc:224] Handling call
0709 12:19:08.567502 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:08.567892 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:19:08.881366 (+313474us) log.cc:844] Serialized 14 byte log entry
0709 12:19:10.031622 (+1150256us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:10.031696 (+    74us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:10.031762 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:10.031853 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:10.032008 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":106003,"prepare.queue_time_us":735922,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":1892,"spinlock_wait_cycles":100326784,"thread_start_us":1200540,"threads_started":2,"wal-append.queue_time_us":927399}
I20260709 12:19:10.092633  6651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2855) took 1377 ms. Trace:
W20260709 12:19:10.093421  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 81257.4us
I20260709 12:19:10.030318 11636 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:10.093842  6401 catalog_manager.cc:5697] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 4 to 6, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 6 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:19:10.101709  7273 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38558 (request call id 732) took 1444 ms. Trace:
I20260709 12:19:10.084816 11409 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:10.101190  6651 rpcz_store.cc:276] 0709 12:19:08.715457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.088516 (+373059us) service_pool.cc:224] Handling call
0709 12:19:09.109525 (+ 21009us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:09.156159 (+ 46634us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:19:09.698202 (+542043us) log.cc:844] Serialized 14 byte log entry
0709 12:19:10.092260 (+394058us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:10.092311 (+    51us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:10.092379 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:10.092459 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:10.092574 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":117644,"prepare.queue_time_us":277742,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"thread_start_us":719572,"threads_started":3,"wal-append.queue_time_us":324474}
I20260709 12:19:10.121630  6505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2277) took 1430 ms. Trace:
I20260709 12:19:10.124221  6503 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:19:10.106513  7273 rpcz_store.cc:276] 0709 12:19:08.649402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.650626 (+  1224us) service_pool.cc:224] Handling call
0709 12:19:08.651053 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:08.651620 (+   567us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:08.651665 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:08.652283 (+   618us) log.cc:844] Serialized 26 byte log entry
0709 12:19:09.285994 (+633711us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:09.286056 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:09.286167 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:10.093805 (+807638us) raft_consensus.cc:1685] finished
0709 12:19:10.093824 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:10.093952 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1408,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":100,"thread_start_us":633209,"threads_started":1,"wal-append.queue_time_us":867319}
I20260709 12:19:10.122044  6505 rpcz_store.cc:276] 0709 12:19:08.691701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.700783 (+  9082us) service_pool.cc:224] Handling call
0709 12:19:08.711243 (+ 10460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:08.718497 (+  7254us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:08.718527 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:08.745331 (+ 26804us) log.cc:844] Serialized 26 byte log entry
0709 12:19:09.317515 (+572184us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:09.346988 (+ 29473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:09.347067 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:10.121404 (+774337us) raft_consensus.cc:1685] finished
0709 12:19:10.121424 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:10.121569 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4379,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":8851,"thread_start_us":546690,"threads_started":1,"wal-append.queue_time_us":831650}
I20260709 12:19:10.132707  6401 catalog_manager.cc:5697] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 5 to 11, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 11 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:19:10.056916 11712 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:10.073052 11015 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:10.142784  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2857) took 1383 ms. Trace:
W20260709 12:19:10.133261  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 136826us
I20260709 12:19:10.142253  6475 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:10.120736 11331 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.143373  6652 rpcz_store.cc:276] 0709 12:19:08.759717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.249656 (+489939us) service_pool.cc:224] Handling call
0709 12:19:09.250102 (+   446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:09.250394 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:19:09.707439 (+457045us) log.cc:844] Serialized 14 byte log entry
0709 12:19:10.142332 (+434893us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:10.142406 (+    74us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:10.142468 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:10.142558 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:10.142710 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":122960,"prepare.queue_time_us":190190,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"thread_start_us":735017,"threads_started":2}
I20260709 12:19:10.116509 11409 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:10.163118  7269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38558 (request call id 727) took 1621 ms. Trace:
I20260709 12:19:10.123432 11717 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:10.171455  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:10.125667 11341 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.169083  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:10.170195  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 83393.4us
I20260709 12:19:10.093135 11550 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:10.172116  6576 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 12.
I20260709 12:19:10.089047 11478 consensus_queue.cc:260] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:10.125505 11496 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:10.130193 11513 consensus_queue.cc:260] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:10.161680 11579 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:10.170680 11409 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:10.189906 11704 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:19:10.191009 11722 log.cc:927] Time spent T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 12:19:10.191628 11741 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.193096  6652 raft_consensus.cc:1275] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:19:10.194015  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:10.194659  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:10.215174  6655 raft_consensus.cc:2410] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:19:10.184296 11523 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:19:10.196623 11693 log.cc:927] Time spent T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.439s	user 0.003s	sys 0.000s
I20260709 12:19:10.197456  7238 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:10.167635  7269 rpcz_store.cc:276] 0709 12:19:08.541779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.573095 (+ 31316us) service_pool.cc:224] Handling call
0709 12:19:08.589823 (+ 16728us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:08.665283 (+ 75460us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:19:08.665596 (+   313us) log.cc:844] Serialized 14 byte log entry
0709 12:19:09.273845 (+608249us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:09.286433 (+ 12588us) log.cc:844] Serialized 26 byte log entry
0709 12:19:09.286586 (+   153us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:09.286637 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:09.383530 (+ 96893us) spinlock_profiling.cc:243] Waited 75 ms on lock 0x7b54005816d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:09.383591 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:09.907624 (+524033us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:10.162896 (+255272us) raft_consensus.cc:1685] finished
0709 12:19:10.162918 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:10.163058 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2294,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":275,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":7367,"spinlock_wait_cycles":199852672,"thread_start_us":607709,"threads_started":1,"wal-append.queue_time_us":1042947}
I20260709 12:19:10.116638 11723 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:10.209779 11725 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.210116 11744 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:10.210590 11012 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:10.184247 11367 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.175206 11331 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
W20260709 12:19:10.211114 11728 log.cc:927] Time spent T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 12:19:10.174487 11717 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:10.201959 11481 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:10.260315  6548 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:10.286594 11481 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:10.287046 11739 consensus_queue.cc:1048] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.083s
I20260709 12:19:10.287227 11481 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:10.272507 11623 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:10.289146 11742 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:10.185324 11341 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:10.247582 11704 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:10.289763 11742 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:10.291488 11341 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:10.271826  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 129631us
W20260709 12:19:10.292920  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 95567.3us
I20260709 12:19:10.293015  6475 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:10.293710  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:10.294783  6475 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:10.251977 11731 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.247072 11564 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:10.288471 11331 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:10.293423 11726 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 12:19:10.204696 11724 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:19:10.310456 11480 consensus_meta.cc:287] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.686s	user 0.007s	sys 0.003s
W20260709 12:19:10.269027 11510 consensus_meta.cc:287] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 1.095s	user 0.014s	sys 0.000s
W20260709 12:19:10.268726 11710 log.cc:927] Time spent T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
W20260709 12:19:10.309363  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 55818.4us
I20260709 12:19:10.310160 11724 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:10.292130 11704 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:10.312458 11730 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:19:10.314515 11734 log.cc:927] Time spent T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 12:19:10.315055 11327 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:10.315642 11473 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:10.228242 11496 consensus_queue.cc:1048] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 12:19:10.306417 11388 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:10.317463  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 2169) took 1732 ms. Trace:
I20260709 12:19:10.318081  6653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2847) took 1628 ms. Trace:
I20260709 12:19:10.318661 11558 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:10.270042 11367 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:10.322155  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:10.323135 10999 raft_consensus.cc:3055] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:19:10.323858 11629 consensus_meta.cc:287] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.696s	user 0.009s	sys 0.000s
I20260709 12:19:10.293701 11725 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:10.326306 11723 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:10.436471  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:10.327662 11706 log.cc:927] Time spent T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.430s	user 0.000s	sys 0.003s
I20260709 12:19:10.365685 11510 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:10.208305 11708 consensus_queue.cc:1059] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.091s
W20260709 12:19:10.328940 11716 log.cc:927] Time spent T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.444s	user 0.000s	sys 0.002s
I20260709 12:19:10.292171 11341 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.457934 11341 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:10.458833 11510 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:10.460412 11341 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:10.364777  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 70095.1us
W20260709 12:19:10.365381  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 69862.6us
I20260709 12:19:10.329257 11712 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:10.252478 11579 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:10.461886  6475 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:10.463451 11579 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:10.247695 11523 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:10.464619  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 92407.2us
I20260709 12:19:10.465492  6548 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:10.466140 11523 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:10.252110 11741 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:10.467176  6548 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:10.469966 11741 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:10.402199 10999 raft_consensus.cc:740] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:10.475620 10999 consensus_queue.cc:260] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:10.477062 10999 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:10.479955 11367 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:10.481671 11704 consensus_queue.cc:1059] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:10.307175 11331 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:10.483872 11462 raft_consensus.cc:3055] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:19:10.367530 11474 consensus_meta.cc:287] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.925s	user 0.005s	sys 0.007s
I20260709 12:19:10.380321  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:10.306507 11703 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:10.271286 11012 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:10.484752 11703 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:10.485236 11703 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:10.485046 11012 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:10.487571 11697 log.cc:927] Time spent T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.685s	user 0.002s	sys 0.001s
I20260709 12:19:10.487689 11012 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:10.385883 11558 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:10.372498 11730 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:10.488458 11558 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:10.488570 11730 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:10.489811  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:10.380972  6583 rpcz_store.cc:276] 0709 12:19:08.585082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.591836 (+  6754us) service_pool.cc:224] Handling call
0709 12:19:08.672544 (+ 80708us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:08.692554 (+ 20010us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:08.692592 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:08.693297 (+   705us) log.cc:844] Serialized 26 byte log entry
0709 12:19:09.269411 (+576114us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:09.269458 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:09.269529 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:10.119818 (+850289us) spinlock_profiling.cc:243] Waited 32.7 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:10.316200 (+196382us) raft_consensus.cc:1685] finished
0709 12:19:10.316227 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:10.317251 (+  1024us) spinlock_profiling.cc:243] Waited 67.3 ms on lock 0x7b540034b2d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:10.317397 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1627,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":11235,"spinlock_wait_cycles":230028800,"thread_start_us":566833,"threads_started":1,"wal-append.queue_time_us":1000440}
I20260709 12:19:10.491154  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:10.491696  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:10.361020 11480 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:10.492414  7265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38558 (request call id 731) took 1843 ms. Trace:
I20260709 12:19:10.492815  7265 rpcz_store.cc:276] 0709 12:19:08.649082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.672782 (+ 23700us) service_pool.cc:224] Handling call
0709 12:19:08.697175 (+ 24393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:08.707696 (+ 10521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:08.707727 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:08.715976 (+  8249us) log.cc:844] Serialized 26 byte log entry
0709 12:19:09.405610 (+689634us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:09.405666 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:09.405754 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:10.486807 (+1081053us) spinlock_profiling.cc:243] Waited 5.59 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:10.492096 (+  5289us) raft_consensus.cc:1685] finished
0709 12:19:10.492119 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:10.492352 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5893,"raft.queue_time_us":13249,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":2759,"spinlock_wait_cycles":12862720,"thread_start_us":639428,"threads_started":1,"wal-append.queue_time_us":976025}
W20260709 12:19:10.495497 11718 log.cc:927] Time spent T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.601s	user 0.000s	sys 0.002s
I20260709 12:19:10.410228 11717 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:10.306877 11623 consensus_queue.cc:1048] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 12:19:10.306712 11740 consensus_queue.cc:1059] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.098s
I20260709 12:19:10.272477 11744 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:10.381669  6653 rpcz_store.cc:276] 0709 12:19:08.689676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.690807 (+  1131us) service_pool.cc:224] Handling call
0709 12:19:08.691233 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:08.691788 (+   555us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:09.599729 (+907941us) log.cc:844] Serialized 14 byte log entry
0709 12:19:10.054507 (+454778us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:10.078280 (+ 23773us) log.cc:844] Serialized 26 byte log entry
0709 12:19:10.078460 (+   180us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:10.078505 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:10.078587 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:10.317853 (+239266us) raft_consensus.cc:1685] finished
0709 12:19:10.317878 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:10.318019 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73411,"prepare.queue_time_us":680609,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":1834,"raft.queue_time_us":212893,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":31416,"thread_start_us":1272782,"threads_started":3,"wal-append.queue_time_us":468780}
I20260709 12:19:10.306272 11731 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:19:10.438171  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:10.359987 11564 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:10.444900  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:10.445300  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:10.515846  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:10.327828 11550 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:10.453487  6655 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:19:10.518069  6575 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:19:10.457232 11738 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:10.481601 11421 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:10.500415 11754 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:10.485620 11721 log.cc:927] Time spent T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.578s	user 0.000s	sys 0.002s
W20260709 12:19:10.386999  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:10.494031 11480 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:10.413002 11682 log.cc:927] Time spent T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:19:10.496107  6622 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:10.546725  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 50681.2us
I20260709 12:19:10.412487 11745 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:10.401187 11388 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:10.424247  7271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38562 (request call id 863) took 1343 ms. Trace:
I20260709 12:19:10.518097  6576 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
W20260709 12:19:10.519556  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:10.525584 11747 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:10.515506  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:10.686412  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 190139us
I20260709 12:19:10.686866  6650 raft_consensus.cc:2410] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 12:19:10.445609 11742 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:10.455096 11636 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:10.484242 11462 raft_consensus.cc:740] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
W20260709 12:19:10.537510  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 75691.8us
I20260709 12:19:10.502835 11723 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:10.544504  6649 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
W20260709 12:19:10.526504 11112 consensus_meta.cc:287] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.691s	user 0.012s	sys 0.000s
I20260709 12:19:10.440481 11629 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:10.501516 11510 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.515285 11733 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:10.651787  7240 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:10.688099  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:10.695470  6475 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:10.546547 11579 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:10.689536  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:10.650579 11696 log.cc:927] Time spent T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 12:19:10.651533 11331 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:10.733134  6475 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:10.536777 11550 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.151s
W20260709 12:19:10.678565 11702 log.cc:927] Time spent T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.852s	user 0.000s	sys 0.003s
W20260709 12:19:10.699995  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 201511us
I20260709 12:19:10.739079  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:10.740447  7270 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
W20260709 12:19:10.693822  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:10.704396  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 52755.2us
I20260709 12:19:10.704865  6575 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 13.
I20260709 12:19:10.706491  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:10.678546 11729 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:10.682196  7271 rpcz_store.cc:276] 0709 12:19:09.080948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.090160 (+  9212us) service_pool.cc:224] Handling call
0709 12:19:09.105233 (+ 15073us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:09.109402 (+  4169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:09.109446 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:09.117253 (+  7807us) log.cc:844] Serialized 26 byte log entry
0709 12:19:09.723836 (+606583us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:09.723876 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:09.723946 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:10.423900 (+699954us) raft_consensus.cc:1685] finished
0709 12:19:10.423934 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:10.424156 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3874,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":14241,"thread_start_us":595075,"threads_started":1,"wal-append.queue_time_us":825063}
I20260709 12:19:10.708549  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:10.680003 11421 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:10.689167 11711 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.734536  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:10.693987 11758 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:19:10.750330  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 283228us
I20260709 12:19:10.750664  7273 raft_consensus.cc:2410] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 12:19:10.703189 11723 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:10.757195  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:10.545471 11738 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:10.827486  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:10.828481  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 74722us
I20260709 12:19:10.816056  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:10.690774 11388 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:10.739744 11341 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:10.732949 11367 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:10.816099  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:10.821496 11616 consensus_meta.cc:287] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.745s	user 0.011s	sys 0.000s
I20260709 12:19:10.731571 11510 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:10.815460 11727 log.cc:927] Time spent T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.451s	user 0.000s	sys 0.001s
W20260709 12:19:10.881637  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 49357us
I20260709 12:19:10.555361 11754 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:10.823467 11738 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:10.765507 11723 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:10.822028 11725 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:10.835752 11724 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:10.815922 11744 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:10.992942  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:11.020588  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 335078us
I20260709 12:19:10.831117 11729 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:10.749850 11474 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.133s
I20260709 12:19:10.942791  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:10.943584 11277 consensus_meta.cc:287] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.888s	user 0.011s	sys 0.000s
I20260709 12:19:10.815717 11012 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:10.746554 11717 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.128s
I20260709 12:19:10.706712 11112 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:10.936666  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 199723us
W20260709 12:19:11.058887 11750 log.cc:927] Time spent T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.293s	user 0.001s	sys 0.000s
I20260709 12:19:11.060259 11112 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:11.062455  7268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38562 (request call id 864) took 1865 ms. Trace:
I20260709 12:19:10.761188 11758 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:11.062855  7268 rpcz_store.cc:276] 0709 12:19:09.196680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.196846 (+   166us) service_pool.cc:224] Handling call
0709 12:19:09.215624 (+ 18778us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:09.218712 (+  3088us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:19:09.822945 (+604233us) log.cc:844] Serialized 14 byte log entry
0709 12:19:10.269657 (+446712us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:10.277830 (+  8173us) log.cc:844] Serialized 26 byte log entry
0709 12:19:10.278026 (+   196us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:10.278067 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:10.278147 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:11.062239 (+784092us) raft_consensus.cc:1685] finished
0709 12:19:11.062260 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:11.062395 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22653,"prepare.queue_time_us":613576,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":19866,"raft.queue_time_us":23269,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":18482,"thread_start_us":932727,"threads_started":2,"wal-append.queue_time_us":683645}
I20260709 12:19:11.065536 11768 consensus_queue.cc:1048] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.232s
I20260709 12:19:10.993083 11480 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 12:19:11.066916 11758 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:10.872952 11654 consensus_meta.cc:287] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 1.062s	user 0.008s	sys 0.004s
I20260709 12:19:10.841572 11327 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:11.046420 11737 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:19:11.068305 11709 log.cc:927] Time spent T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 1.115s	user 0.002s	sys 0.001s
I20260709 12:19:11.068744 11654 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:11.072079 11684 log.cc:927] Time spent T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 1.238s	user 0.003s	sys 0.000s
I20260709 12:19:11.073691  6500 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2270) took 2651 ms. Trace:
I20260709 12:19:11.074465  7269 raft_consensus.cc:2393] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 13: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:10.751129 11636 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:10.751446 11579 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:10.992412  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2278) took 2177 ms. Trace:
I20260709 12:19:10.695055 11747 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:11.074728 11012 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:11.099975 11771 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.934248  6623 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:11.011965 10999 consensus_meta.cc:287] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.517s	user 0.013s	sys 0.000s
W20260709 12:19:11.069922 11753 log.cc:927] Time spent T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.385s	user 0.000s	sys 0.001s
I20260709 12:19:10.991978 11746 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:10.991767 11761 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:11.054944 11729 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:10.758126 11711 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:10.749017 11733 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:10.938711 11616 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:11.065183 11755 log.cc:927] Time spent T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.313s	user 0.000s	sys 0.001s
I20260709 12:19:10.948661  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:11.013341 11725 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:10.890878 11703 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:11.074076  6500 rpcz_store.cc:276] 0709 12:19:08.422735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.434267 (+ 11532us) service_pool.cc:224] Handling call
0709 12:19:08.434634 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:08.449399 (+ 14765us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:08.449432 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:08.537325 (+ 87893us) spinlock_profiling.cc:243] Waited 30.4 ms on lock 0x7b2c00000c88. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:19:08.537762 (+   437us) log.cc:844] Serialized 26 byte log entry
0709 12:19:09.406578 (+868816us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:09.406632 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:09.406786 (+   154us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:11.050093 (+1643307us) spinlock_profiling.cc:243] Waited 4.29 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:11.073488 (+ 23395us) raft_consensus.cc:1685] finished
0709 12:19:11.073514 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:11.073636 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14286,"raft.queue_time_us":14724,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":124,"spinlock_wait_cycles":79897600,"thread_start_us":868399,"threads_started":1,"wal-append.queue_time_us":1207678}
I20260709 12:19:11.103117 11717 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:19:11.096009  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 150831us
I20260709 12:19:11.035900 11462 consensus_queue.cc:260] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:11.096860  6501 rpcz_store.cc:276] 0709 12:19:08.801887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.802047 (+   160us) service_pool.cc:224] Handling call
0709 12:19:08.833023 (+ 30976us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:08.833456 (+   433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:08.833502 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:08.834072 (+   570us) log.cc:844] Serialized 26 byte log entry
0709 12:19:09.407894 (+573822us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:09.407934 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:09.407996 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:10.329473 (+921477us) spinlock_profiling.cc:243] Waited 3.79 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:10.945705 (+616232us) raft_consensus.cc:1685] finished
0709 12:19:10.945738 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:10.979366 (+ 33628us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1321,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":120,"spinlock_wait_cycles":8726784,"thread_start_us":573409,"threads_started":1,"wal-append.queue_time_us":908843}
W20260709 12:19:11.049289  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 307064us
W20260709 12:19:11.131453  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:11.150282 11730 consensus_meta.cc:287] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.658s	user 0.007s	sys 0.004s
W20260709 12:19:11.141832  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 207670us
I20260709 12:19:11.069320 11737 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:11.055253  6651 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:19:11.068913 11510 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:11.070477 11731 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:10.950205 11558 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:11.138200 11746 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:11.164140  7273 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:19:11.073845 11654 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:11.177131  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 236196us
I20260709 12:19:11.215720 11717 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:11.218143 11717 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:11.219669 10999 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:11.220250 10999 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:11.220530  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:11.222412 10999 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:11.180974 11725 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:11.187722 11751 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 12:19:11.070530 11277 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:11.100517 11771 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:11.204164 11753 log.cc:927] Time spent T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:19:11.215287 11746 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:11.196548  6579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2281) took 1136 ms. Trace:
I20260709 12:19:11.212147  6658 raft_consensus.cc:2393] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 13: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:11.237147  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:11.167354 11773 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:11.233946  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:11.242563  6500 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:19:11.198432 11564 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:11.205329 11737 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:11.241171  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:11.240495  6578 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.000s	sys 0.016s
I20260709 12:19:11.244791  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:19:11.256371  6578 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.000s	sys 0.016s
I20260709 12:19:11.246857  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:11.258100  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:11.256135  6650 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
I20260709 12:19:11.252930  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:11.261881  6653 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:11.246346  6583 raft_consensus.cc:2368] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 12. Current term is 13.
I20260709 12:19:11.214517 11729 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:11.257082  6508 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:11.149060 11761 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:11.237231  6579 rpcz_store.cc:276] 0709 12:19:10.060237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.062135 (+  1898us) service_pool.cc:224] Handling call
0709 12:19:10.062610 (+   475us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:10.245434 (+182824us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:10.245465 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:10.246017 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 12:19:10.353448 (+107431us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:10.353484 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:10.353551 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:11.196236 (+842685us) raft_consensus.cc:1685] finished
0709 12:19:11.196264 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:11.196463 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2764,"raft.queue_time_us":38554,"spinlock_wait_cycles":8416640,"thread_start_us":244810,"threads_started":2,"wal-append.queue_time_us":296704}
I20260709 12:19:11.269639  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:11.271926  6500 raft_consensus.cc:2393] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 15: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:19:11.273185 11765 log.cc:927] Time spent T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.340s	user 0.000s	sys 0.001s
I20260709 12:19:11.273762 11775 consensus_queue.cc:1048] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:11.275974  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:11.276412 11757 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:11.276732  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:11.277009 11757 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:11.276691  6505 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:19:11.278126  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:11.278793  6506 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:19:11.277400  6503 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:19:11.278795  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:11.200426 11730 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:11.289629  7240 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:11.285844  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 225696us
I20260709 12:19:11.247131 11773 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:11.296567  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:11.300623  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:11.301631  6500 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:11.304576  6654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2854) took 2602 ms. Trace:
I20260709 12:19:11.304697  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:11.305213  6654 rpcz_store.cc:276] 0709 12:19:08.696020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.018513 (+322493us) service_pool.cc:224] Handling call
0709 12:19:09.019120 (+   607us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:09.019765 (+   645us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:19:09.799582 (+779817us) log.cc:844] Serialized 14 byte log entry
0709 12:19:10.256415 (+456833us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:10.289268 (+ 32853us) log.cc:844] Serialized 26 byte log entry
0709 12:19:10.289416 (+   148us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:10.289454 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:10.289525 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:11.294150 (+1004625us) raft_consensus.cc:1685] finished
0709 12:19:11.294170 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:11.294356 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13891,"prepare.queue_time_us":380545,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":35250,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":37107,"thread_start_us":1190722,"threads_started":2,"wal-append.queue_time_us":682848}
I20260709 12:19:11.309320  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:11.304445 11551 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.002s	sys 0.040s
I20260709 12:19:11.310240  6508 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:11.307798  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:11.279534 11757 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:11.318697  6504 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:19:11.319492  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:11.320247  6505 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:19:11.295821 11705 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:11.321544 11733 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:11.321719 11705 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:11.298448 11773 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:11.322345 11705 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:11.320261 11711 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:11.324747 11012 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:11.321180 11558 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:11.325730 11629 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:11.327867 11623 raft_consensus.cc:3055] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:19:11.321681 11775 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 12:19:11.288021 11388 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:11.263823 10999 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 12:19:11.334393  6502 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:11.316864  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2278) took 1257 ms. Trace:
I20260709 12:19:11.289721 11719 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:11.266446 11729 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:11.331416 11762 log.cc:927] Time spent T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.516s	user 0.000s	sys 0.013s
I20260709 12:19:11.317278 11747 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:11.315398 11749 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:11.355767 11409 consensus_meta.cc:287] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 1.143s	user 0.014s	sys 0.002s
I20260709 12:19:11.351955  6581 rpcz_store.cc:276] 0709 12:19:10.059001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.059189 (+   188us) service_pool.cc:224] Handling call
0709 12:19:10.076675 (+ 17486us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:10.091235 (+ 14560us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:10.091268 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:10.107825 (+ 16557us) log.cc:844] Serialized 26 byte log entry
0709 12:19:10.305561 (+197736us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:10.305693 (+   132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:10.368868 (+ 63175us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:11.316655 (+947787us) raft_consensus.cc:1685] finished
0709 12:19:11.316676 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:11.316805 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9904,"raft.queue_time_us":33089,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":8259,"thread_start_us":163025,"threads_started":1,"wal-append.queue_time_us":570834}
W20260709 12:19:11.317770 11551 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.003s	sys 0.040s
I20260709 12:19:11.367686  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:11.328317 11623 raft_consensus.cc:740] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:11.372522  6502 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:19:11.383112 11749 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:11.401386 11730 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:11.402864 11749 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:11.404850  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 50815.3us
I20260709 12:19:11.406564  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:11.407327  6502 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:11.409719 11729 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:11.412904  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2282) took 1352 ms. Trace:
I20260709 12:19:11.413511  6577 rpcz_store.cc:276] 0709 12:19:10.060587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.060735 (+   148us) service_pool.cc:224] Handling call
0709 12:19:10.084578 (+ 23843us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:10.180880 (+ 96302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:10.180907 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:10.203371 (+ 22464us) log.cc:844] Serialized 26 byte log entry
0709 12:19:10.378824 (+175453us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:10.378876 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:10.378942 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:11.330706 (+951764us) spinlock_profiling.cc:243] Waited 7.02 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:11.412473 (+ 81767us) raft_consensus.cc:1685] finished
0709 12:19:11.412505 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:11.412801 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34139,"raft.queue_time_us":35885,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":7441,"spinlock_wait_cycles":16149760,"thread_start_us":220000,"threads_started":2,"wal-append.queue_time_us":484348}
I20260709 12:19:11.417754 11761 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:11.419024 11277 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:11.419559  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2285) took 1023 ms. Trace:
I20260709 12:19:11.420454 11771 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:11.421664 11725 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:11.422029 11481 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:11.423641  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.425518 11730 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.425988 11730 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:11.426458 11747 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.427032 11747 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:11.428134 11730 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:11.429176 11747 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:11.430972 11754 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:11.431492  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:11.423228 11623 consensus_queue.cc:260] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:11.434247  6501 raft_consensus.cc:2410] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
W20260709 12:19:11.440855  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.450150  6653 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 13.
I20260709 12:19:11.420162  6578 rpcz_store.cc:276] 0709 12:19:10.396938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.398319 (+  1381us) service_pool.cc:224] Handling call
0709 12:19:10.438912 (+ 40593us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:10.439159 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:10.439189 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:10.446818 (+  7629us) log.cc:844] Serialized 26 byte log entry
0709 12:19:11.258956 (+812138us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:11.259012 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:11.259100 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:11.419250 (+160150us) raft_consensus.cc:1685] finished
0709 12:19:11.419281 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:11.419471 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":731201,"threads_started":1,"wal-append.queue_time_us":852847}
I20260709 12:19:11.457856  6500 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:11.463235  6657 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 12. Current term is 13. Ops: [10.10-12.11]
I20260709 12:19:11.463165  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:11.495754  6650 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:11.465961  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:11.491528  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:11.494339  6623 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:11.538085  7268 raft_consensus.cc:2393] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 13: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:11.360489 11719 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:11.544375  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.547116  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:11.383443 11409 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:11.547969  6501 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:19:11.558073  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:11.566092  6653 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:19:11.442864 11558 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.097s
W20260709 12:19:11.570534  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.575155  6651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58980 (request call id 2872) took 1192 ms. Trace:
I20260709 12:19:11.575600  6651 rpcz_store.cc:276] 0709 12:19:10.379858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.380014 (+   156us) service_pool.cc:224] Handling call
0709 12:19:11.571627 (+1191613us) spinlock_profiling.cc:243] Waited 570 ms on lock 0x7b54002a35d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:11.571771 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1311877888}
I20260709 12:19:11.544938 11481 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:11.577333  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:11.469903 11551 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.105s
I20260709 12:19:11.568353 11711 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.568288 11616 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:11.583719  6500 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 12:19:11.582609  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:11.591786  6500 pending_rounds.cc:85] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 10
I20260709 12:19:11.592103  6500 pending_rounds.cc:107] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 15.11
W20260709 12:19:11.592291  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.592422  6500 raft_consensus.cc:2889] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:11.588146  6550 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:11.594137  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:11.596540  6504 raft_consensus.cc:1240] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:11.597594  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:11.598155  6505 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 12.
I20260709 12:19:11.599500  6508 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:11.577086 11327 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.632573  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:11.633486  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:11.600044  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:11.636852 11719 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:11.637511  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:11.638315  6575 raft_consensus.cc:2393] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 14: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:11.605242  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:11.639520  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:11.640138  6508 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:19:11.606968  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:11.640985  6506 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:19:11.641014  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:11.641970  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:11.642558 11784 log.cc:927] Time spent T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:19:11.643227  6506 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:11.642879 11773 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:11.644544  6655 raft_consensus.cc:2393] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 14: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:11.579373 11481 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:11.599663 11776 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:11.645753 11737 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:11.646195 11776 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:11.647910 11737 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:11.607504 11579 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:11.620127 11759 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:11.649571 11759 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:11.650265 11773 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:11.651973 11759 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:11.627135  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:11.659472  6578 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:11.627592 11711 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:11.627951  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:11.586926 11703 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:11.629663  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:11.659716  6473 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:11.630522  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:11.630785 11705 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:11.665827  6576 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:19:11.631215  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:11.586792 11760 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:11.677772  6582 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:11.621551 11409 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:11.622285  6658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58980 (request call id 2874) took 1186 ms. Trace:
I20260709 12:19:11.684765  6658 rpcz_store.cc:276] 0709 12:19:10.435774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.436172 (+   398us) service_pool.cc:224] Handling call
0709 12:19:11.622104 (+1185932us) spinlock_profiling.cc:243] Waited 767 ms on lock 0x7b54000b6654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:11.622229 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1764965120}
I20260709 12:19:11.601835  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:11.608080  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:11.600088 10999 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:11.655424  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:11.632510 11327 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:11.637519  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:11.686163  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:11.618752 11513 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:11.690579  6503 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 18: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:11.656229  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:11.687961 11760 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:11.693601  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:11.694283  7267 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:11.694602 11760 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:11.657565  7270 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:11.658556  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:11.658710  7271 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:11.660961  6650 raft_consensus.cc:2410] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:19:11.688724 10999 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:11.700990  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:11.701195 10999 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:11.705142 11711 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:11.705653 11327 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:11.696302 11705 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:11.710448  6578 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:19:11.711360  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:11.713078 11746 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:11.714269  6581 raft_consensus.cc:2393] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 12: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:11.715287 11746 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:11.716645  6474 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:11.716921 11776 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:11.728116  6473 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:11.701241  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:11.737591  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:11.738055  7266 raft_consensus.cc:2393] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 11: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:11.737918  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:11.737643  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:11.739034  6500 raft_consensus.cc:2393] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 17: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:11.739419  6505 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:11.732801  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:11.663467  6507 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
I20260709 12:19:11.610387  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 802) took 2462 ms. Trace:
I20260709 12:19:11.740777  6656 rpcz_store.cc:276] 0709 12:19:09.148208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.063335 (+915127us) service_pool.cc:224] Handling call
0709 12:19:10.079671 (+ 16336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:10.381342 (+301671us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:10.381381 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:10.381423 (+    42us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:10.381454 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:11.610089 (+1228635us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x7b54002279d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:11.610154 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:11.610314 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":693306112}
W20260709 12:19:11.627266 11723 consensus_meta.cc:287] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.614s	user 0.007s	sys 0.003s
I20260709 12:19:11.667126 11730 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:11.755198  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:11.668620  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:11.669617  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:11.756469  7269 raft_consensus.cc:2410] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:19:11.645476 11481 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:11.620756  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58980 (request call id 2865) took 1428 ms. Trace:
I20260709 12:19:11.761984  6649 rpcz_store.cc:276] 0709 12:19:10.191857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.193733 (+  1876us) service_pool.cc:224] Handling call
0709 12:19:11.620595 (+1426862us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b54003e85d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:11.620699 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":758305152}
I20260709 12:19:11.687068  6651 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:19:11.689260  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:11.687654  7272 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:19:11.744838  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:11.745822  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.768831  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:11.745803  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:11.770695  6653 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:11.738270  7268 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:11.733151  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:11.758127  6654 raft_consensus.cc:1240] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
W20260709 12:19:11.760818  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 92304.5us
I20260709 12:19:11.774477  6658 raft_consensus.cc:2393] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 15: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:11.761379  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:11.777151  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 44084.7us
I20260709 12:19:11.778131  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:11.724851 11708 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:11.786018  6576 raft_consensus.cc:1240] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:19:11.793615  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:11.794922  6549 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:11.784526  7238 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:11.795179  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:11.801518  6582 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:11.794559  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:11.802732  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 370813us
I20260709 12:19:11.805248  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:11.806488  6622 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:11.805840  6549 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:11.747723 11711 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:11.829051  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:11.830017  6500 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
I20260709 12:19:11.839922  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:11.779654 11773 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:11.774910 11740 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:11.853688  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:11.853133  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:11.783293 11789 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:11.852962  7271 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:19:11.755430 11730 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:11.784029 11627 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:11.872371  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:11.877417 11730 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:11.859963  6505 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:19:11.878875  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:11.760758 11723 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:11.842886 11711 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:11.804451 11737 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:19:11.883876  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 77486us
I20260709 12:19:11.783432 11703 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.086s
I20260709 12:19:11.804371 11781 consensus_queue.cc:1059] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.077s
W20260709 12:19:12.059278  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 212104us
I20260709 12:19:12.086061  6650 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:11.969233  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:12.087395  6474 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:11.863504 11773 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:11.971567  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54204 (request call id 702) took 1487 ms. Trace:
I20260709 12:19:11.885855  6653 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:12.088716  6622 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
W20260709 12:19:11.936916  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 176453us
I20260709 12:19:12.091997  6583 rpcz_store.cc:276] 0709 12:19:10.484162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.491436 (+  7274us) service_pool.cc:224] Handling call
0709 12:19:11.971305 (+1479869us) spinlock_profiling.cc:243] Waited 746 ms on lock 0x7b540034d0d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:11.971496 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1715426560}
W20260709 12:19:12.089893  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 282168us
I20260709 12:19:11.895429  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2277) took 2689 ms. Trace:
I20260709 12:19:11.853763 11771 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:11.844231 11782 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:12.011003  7267 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:11.869184 11789 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:12.098683 11789 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:12.098562 11782 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:11.880120 11474 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.693s	user 0.008s	sys 0.035s
W20260709 12:19:11.898837 11791 log.cc:927] Time spent T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.289s	user 0.003s	sys 0.002s
I20260709 12:19:11.885630 11627 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:12.046908 11804 log.cc:927] Time spent T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 12:19:12.102751  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:12.106348 11799 log.cc:927] Time spent T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:19:12.106565 11725 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:12.107041 11725 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:12.108390  6622 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
W20260709 12:19:11.963605 11702 log.cc:927] Time spent T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:19:11.974977 11761 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.053810 11717 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:11.896801 11616 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:12.022971 11015 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:12.095758 11773 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:12.107389 11725 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:12.118749  7240 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:19:12.126783  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:11.976250 11623 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:12.100570 11766 raft_consensus.cc:2764] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:19:12.107821 11719 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:12.100270 11781 consensus_queue.cc:1243] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 11.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 12:19:12.108987 11474 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.917s	user 0.022s	sys 0.054s
I20260709 12:19:12.115813  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:12.151384  6622 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:12.152776  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:12.096065  6584 rpcz_store.cc:276] 0709 12:19:09.194113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:09.194284 (+   171us) service_pool.cc:224] Handling call
0709 12:19:09.194788 (+   504us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:09.215435 (+ 20647us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:09.215480 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:09.253448 (+ 37968us) log.cc:844] Serialized 52 byte log entry
0709 12:19:09.799563 (+546115us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:09.799604 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:09.799684 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:10.494775 (+695091us) spinlock_profiling.cc:243] Waited 46.2 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:11.762690 (+1267915us) spinlock_profiling.cc:243] Waited 3.09 ms on lock 0x7b24003d7498. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872c229c4 00007f2872c20acb 00007f2872ddbdc9 00007f2872ddba55 00007f2872ddafb0 00007f2875a51488 00007f2875a51be8 00007f2875a29398 00007f2875a2476a 00007f2875a23bb6
0709 12:19:11.883134 (+120444us) raft_consensus.cc:1685] finished
0709 12:19:11.883188 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:11.883383 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15733,"raft.queue_time_us":21897,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":3813,"spinlock_wait_cycles":113380352,"thread_start_us":511726,"threads_started":1,"wal-append.queue_time_us":624021}
I20260709 12:19:12.089859 11711 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:11.948729 11720 consensus_queue.cc:1059] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.186s
I20260709 12:19:12.177300  6622 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:12.107331 11627 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:11.873633 11740 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:12.093922 11737 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:12.187779 11737 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:12.188958 11737 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:12.189563 11737 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:12.190093 11737 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:12.137728 11616 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:12.210513  6582 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 13.
I20260709 12:19:12.182669 11814 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:12.133131 11756 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:12.149230 11719 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:12.136579 11708 consensus_queue.cc:1059] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 12:19:12.179524 11754 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:12.179329 11778 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:12.217711  6548 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:12.145573 11717 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:12.197626 11759 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:19:12.223984  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 129228us
I20260709 12:19:12.205322 11720 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:12.208519 11616 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:12.236495  6650 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 11.
W20260709 12:19:12.232515  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 377306us
I20260709 12:19:12.212260 11703 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:12.213052 11421 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:12.145640 11761 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:12.224150 11719 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:12.246214  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:12.219959 11756 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:12.221776 11754 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:12.233798 11759 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:12.259737 11759 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:12.263257 11766 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:12.263727 11766 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:12.227089 11778 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:12.264722 11778 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:19:12.268234 11803 log.cc:927] Time spent T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
W20260709 12:19:12.270259  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 151590us
W20260709 12:19:12.246464  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 440640us
I20260709 12:19:12.250414 11496 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:12.254451 11761 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:12.223006 11814 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:12.256774  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 56979us
W20260709 12:19:12.243192 11513 consensus_meta.cc:287] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.542s	user 0.012s	sys 0.001s
I20260709 12:19:12.258507 11737 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:12.282652  7240 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:12.264150 11766 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:12.313364 11759 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:12.315779 11759 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:12.270723 11787 log.cc:927] Time spent T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.818s	user 0.004s	sys 0.010s
I20260709 12:19:12.249845 11703 raft_consensus.cc:697] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 13 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:12.317545 11703 consensus_queue.cc:237] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:12.329036  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:12.291414 10999 consensus_meta.cc:287] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.589s	user 0.010s	sys 0.001s
W20260709 12:19:12.287744  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 110548us
I20260709 12:19:12.251684 11756 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:12.333302  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 245966us
I20260709 12:19:12.334314  6474 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:12.270501 11754 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:12.293092 11723 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:12.295100 11814 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:12.293113 11826 log.cc:927] Time spent T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:19:12.300943  6550 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:12.303385  7265 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
W20260709 12:19:12.310092 11806 log.cc:927] Time spent T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8: Append to log took a long time: real 0.250s	user 0.001s	sys 0.001s
I20260709 12:19:12.316541 11761 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:12.337317  6548 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:12.344058  6622 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:12.344815  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 98589.8us
I20260709 12:19:12.292032 11724 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:19:12.336645 11802 log.cc:927] Time spent T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:19:12.320660 11496 raft_consensus.cc:697] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:12.349524  6473 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:12.369601  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.373117  6582 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
W20260709 12:19:12.369531  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 68737.5us
W20260709 12:19:12.353004  6475 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.3:36107 timed out after 1.519s (SENT) [suppressed 8 similar messages]
I20260709 12:19:12.368813  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:12.380919  6473 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:12.385067  6473 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:12.385432 11813 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:12.374365 11767 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:12.351224 11480 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.347610 11831 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:12.344352 11627 consensus_queue.cc:1048] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:19:12.387573  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.344826 10999 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:12.400274  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.378047 11771 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:19:12.387863 11730 consensus_meta.cc:287] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.508s	user 0.010s	sys 0.002s
I20260709 12:19:12.403302  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.407181  6583 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:19:12.348052 11759 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:19:12.413748  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 45006.7us
I20260709 12:19:12.409443  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.416569  6579 raft_consensus.cc:2393] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 15: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:12.387598 11811 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.417419  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.385975 11823 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:12.416651  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:12.390883 11831 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:12.391347 11814 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:12.424381  6575 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:12.424559  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.425252  6584 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:12.426137  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.428150  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.428172  6579 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:12.403400 11277 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:12.430076 11496 consensus_queue.cc:237] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:12.431743 11824 log.cc:927] Time spent T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 12:19:12.430819 11816 log.cc:927] Time spent T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:19:12.404844 11771 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:12.387516 11720 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:19:12.434757  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:12.374765 11724 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:12.400588 11480 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:12.394023 11813 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:12.497032  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49792 (request call id 809) took 1273 ms. Trace:
W20260709 12:19:12.529101  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 94426.8us
W20260709 12:19:12.499850  7265 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38562 (request call id 868) took 1816 ms (client timeout 2015 ms). Trace:
W20260709 12:19:12.530761  7265 rpcz_store.cc:269] 0709 12:19:10.686868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:10.688224 (+  1356us) service_pool.cc:224] Handling call
0709 12:19:12.499662 (+1811438us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b54004bbc54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:12.499770 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3582371200}
I20260709 12:19:12.529191  6655 rpcz_store.cc:276] 0709 12:19:11.223706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.223814 (+   108us) service_pool.cc:224] Handling call
0709 12:19:12.496830 (+1273016us) spinlock_profiling.cc:243] Waited 406 ms on lock 0x7b54000b7554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:12.496970 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":934668672}
I20260709 12:19:12.453291  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.467664 11843 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:12.543243 11843 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:12.543655 11843 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:12.543560 11711 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
W20260709 12:19:12.490764 11828 log.cc:927] Time spent T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.233s	user 0.001s	sys 0.000s
I20260709 12:19:12.408527 11782 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:12.424415 11814 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:12.415050 11730 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:12.422501 11740 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:12.496891 11737 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:12.480516 11825 log.cc:927] Time spent T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 12:19:12.573458  6584 raft_consensus.cc:2393] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 13: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
W20260709 12:19:12.485006 11830 log.cc:927] Time spent T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 12:19:12.491845 11834 log.cc:927] Time spent T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.234s	user 0.000s	sys 0.000s
I20260709 12:19:12.604861  7238 leader_election.cc:400] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 12. Current term is 13.
W20260709 12:19:12.629997  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 82047us
I20260709 12:19:12.630756  7238 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:12.559264  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 206265us
W20260709 12:19:12.632514  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:12.481361 11833 log.cc:927] Time spent T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 12:19:12.607204  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 201037us
W20260709 12:19:12.607075  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 222057us
I20260709 12:19:12.608448  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.500223 11616 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:12.641090  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.426982 11831 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:12.489085 11789 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:12.467197 11837 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:12.502090 11579 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:12.647125  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.420635 11759 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:12.626747 11823 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:12.446738 11277 raft_consensus.cc:697] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:19:12.615289 11481 consensus_meta.cc:287] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.792s	user 0.012s	sys 0.000s
I20260709 12:19:12.628827 11711 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:12.635601  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:12.497393 11725 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:12.545955 11771 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:12.675488  6503 raft_consensus.cc:1240] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:12.638500  6582 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:19:12.545318 11840 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:12.611622 11756 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:12.638986  6473 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:12.425876 11811 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:12.497193 10999 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:12.496248 11719 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:12.645291  6500 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:12.639147 11773 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:12.644668  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:12.713101  6501 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 13.
I20260709 12:19:12.644829  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:12.616081 11842 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:12.667176 11481 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:12.714082 11842 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:12.714517 11842 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:12.500170 11778 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:12.650876  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.644081 11740 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:12.675921  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.692425  7269 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:12.612522 11814 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:12.701506  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:12.747648  6506 raft_consensus.cc:2393] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 16: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:12.701787  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:12.702149  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:12.748518  6503 raft_consensus.cc:2393] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 12: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:12.716423  6650 raft_consensus.cc:1240] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 12. Current term is 13. Ops: []
W20260709 12:19:12.716516  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 77603.5us
I20260709 12:19:12.725315  6578 raft_consensus.cc:2393] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 11: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:12.717108  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.721235  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.661222 11837 raft_consensus.cc:697] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:12.650807 11761 raft_consensus.cc:697] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 11 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:12.715978 11719 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:12.754565  6473 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:12.755106  6655 raft_consensus.cc:1240] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:12.759613  6582 raft_consensus.cc:2393] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 8: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:12.764725  6652 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:12.765152  6653 raft_consensus.cc:1240] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 12:19:12.769054  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:12.769168  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.770591  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:12.767256  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:12.716034 11481 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:12.772233  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:12.776335  6506 raft_consensus.cc:2393] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 9: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:12.776666  6657 raft_consensus.cc:1240] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 12:19:12.775323  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:12.780027  6503 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:19:12.771333  6656 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:19:12.783012  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.783694  6658 raft_consensus.cc:2393] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 13: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:12.784550  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.666865 11579 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:12.786197  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:12.786661  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:12.784803  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:12.768683 11816 log.cc:927] Time spent T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:19:12.789628  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.791740  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:12.724130 11848 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:12.792820  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.796521  6584 raft_consensus.cc:2410] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:19:12.791982  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:12.762514 11698 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:12.763655 11740 raft_consensus.cc:697] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 13 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:12.767975 11737 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:12.800393  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.801443  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.772974 11819 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:12.876668  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:12.864909  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.883674  6656 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:12.878811  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:12.864485  6653 raft_consensus.cc:1240] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:12.863610  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.888610  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:12.888928  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:12.889637  6657 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:19:12.884284  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.890731  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:12.891300  6581 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:12.891469  6657 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:12.894942  6506 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:19:12.896211  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.898774  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:12.899426  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:12.770279 11766 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:12.902612  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 486100us
I20260709 12:19:12.794699 11840 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:12.796085 11848 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:19:12.868860 11822 log.cc:927] Time spent T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.226s	user 0.000s	sys 0.000s
I20260709 12:19:12.927848 11848 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:12.878479 11808 log.cc:927] Time spent T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.848s	user 0.004s	sys 0.000s
W20260709 12:19:12.893970 11844 log.cc:927] Time spent T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.287s	user 0.000s	sys 0.001s
I20260709 12:19:12.929553  6654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2231) took 1483 ms. Trace:
I20260709 12:19:12.875771 11277 consensus_queue.cc:237] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:12.930135  6654 rpcz_store.cc:276] 0709 12:19:11.446341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.773081 (+326740us) service_pool.cc:224] Handling call
0709 12:19:11.794031 (+ 20950us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:11.879032 (+ 85001us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:11.879063 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:11.889029 (+  9966us) log.cc:844] Serialized 26 byte log entry
0709 12:19:11.983530 (+ 94501us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b48000cda58. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287adb2aa1 00007f287ada9b37 00007f287ada99ca 00007f287aea126f 00007f287aea0f37 00007f287aeb555c 00007f287aeb5508 00007f287aeb5488 00007f287aeb5443 00007f287aeb46ec 00007f2879ddea96
0709 12:19:12.215449 (+231919us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:12.215518 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:12.215621 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:12.929250 (+713629us) raft_consensus.cc:1685] finished
0709 12:19:12.929271 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:12.929490 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56442,"raft.queue_time_us":56622,"raft.run_cpu_time_us":2139,"raft.run_wall_time_us":100312,"spinlock_wait_cycles":34447616,"thread_start_us":246151,"threads_started":1,"wal-append.queue_time_us":655831}
W20260709 12:19:12.905931  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 151464us
W20260709 12:19:12.874606 11855 log.cc:927] Time spent T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:19:12.907210  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:12.934216  7271 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:12.934609 11766 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:12.865022 11837 consensus_queue.cc:237] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:12.909127  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.937122  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.938360 11480 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:12.938695  6582 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:19:12.909687  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:12.863716 11851 log.cc:927] Time spent T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 12:19:12.869272 11747 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:12.912755 11840 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:12.941085  6473 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:12.913427  6653 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:12.914842  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.789568 11754 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:12.914706 11725 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:12.915278  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:12.917255  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.794724 11704 consensus_queue.cc:1059] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.110s
I20260709 12:19:12.918491  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:12.875694 11811 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:12.873493 11724 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:12.921450 11773 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:12.922236 11759 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:12.960450  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.961400  6578 raft_consensus.cc:2393] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 15: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:12.924466 11761 consensus_queue.cc:237] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:12.925794 11789 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:12.925962 11817 raft_consensus.cc:3055] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:12.926458 11616 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:12.927384 11756 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:12.859066 11813 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:12.906705  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:12.863879 11698 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:12.943940  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:12.983814 11831 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:12.984658 11754 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:12.985726  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.986817  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.987380 11842 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:12.987923  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.988397  6655 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:12.989789  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:12.990203 11842 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:12.973029 11817 raft_consensus.cc:740] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:12.991935 11831 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:12.992568 11817 consensus_queue.cc:260] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:12.994174 11817 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:12.944872  7269 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:19:12.948673  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:12.951974  6583 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:19:12.954245  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:19:12.999500  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 50948.7us
I20260709 12:19:12.975220 11462 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:12.951155 11823 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:12.924106 11843 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:19:12.981343  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:12.981870  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:13.004130  6584 raft_consensus.cc:2393] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 15: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:12.981797 11740 consensus_queue.cc:237] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:13.000465  6622 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:12.980636  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:12.920188 11771 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:19:13.001874  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:13.002864  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:13.007202  6581 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:13.008314  6582 raft_consensus.cc:2393] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 13: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:13.008527  6506 raft_consensus.cc:2393] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 13: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:13.003710  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:13.004284  6577 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:19:12.900987 11737 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:13.013437  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:12.959076 11730 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:13.012143  6622 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:13.004524 11823 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:13.018066  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:13.023355  6657 raft_consensus.cc:2393] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 16: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:13.008505 11842 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:13.025740  6580 raft_consensus.cc:2393] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 16: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:13.014215 11616 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:12.900835 11819 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:13.030030  6622 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:13.035634  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:13.035995  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:13.037456  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:13.021145 11823 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:13.038275  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:13.036689  6503 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:19:13.021595 11756 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:13.041611  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:13.044006  6507 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:13.015421 11754 consensus_queue.cc:1059] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 12:19:13.042716  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:13.046908  6506 raft_consensus.cc:2393] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 15: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:13.043294  6582 raft_consensus.cc:2393] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 13: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:13.053403  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:13.026625 11842 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:13.033308 11708 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.057401  6503 raft_consensus.cc:1240] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:19:13.058512  6653 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:13.061162  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:13.056939  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:13.031744 11616 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:13.063359  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:13.069861 11819 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:13.072204  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:13.079574 11812 raft_consensus.cc:3055] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:13.039685 11773 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:13.080906 11773 raft_consensus.cc:697] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:13.081913 11773 consensus_queue.cc:237] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:13.082362 11478 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:13.085655  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:13.087378 11778 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:13.088913  6507 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:19:13.089025 11848 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:13.091135 11848 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:13.090878  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:13.037050 11814 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:13.094596 11814 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:13.095604  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:13.096549  6503 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 13: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:13.064436 11842 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:13.099766  7269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38562 (request call id 871) took 1373 ms. Trace:
I20260709 12:19:13.080231 11812 raft_consensus.cc:740] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:13.100308  7269 rpcz_store.cc:276] 0709 12:19:11.726624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.775221 (+ 48597us) service_pool.cc:224] Handling call
0709 12:19:13.099579 (+1324358us) spinlock_profiling.cc:243] Waited 885 ms on lock 0x7b54004b0d54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:13.099705 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2034397568}
I20260709 12:19:13.103132  6501 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:19:13.112825  6548 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:13.105376  6506 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:13.108215  6575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2300) took 1472 ms. Trace:
I20260709 12:19:13.115108  6575 rpcz_store.cc:276] 0709 12:19:11.635705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.638845 (+  3140us) service_pool.cc:224] Handling call
0709 12:19:11.639116 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:12.427421 (+788305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:13.107963 (+680542us) spinlock_profiling.cc:243] Waited 782 ms on lock 0x7b540013f954. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:13.108138 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1797953920}
I20260709 12:19:13.125946  6581 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:13.127213  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:13.128254  7269 raft_consensus.cc:2393] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 12: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:13.130229  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:13.130854  6649 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:13.139117  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:19:13.105995 11800 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 1.011s	user 0.003s	sys 0.000s
I20260709 12:19:13.142438  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:13.094596  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:13.146929  6653 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 13. Current term is 14. Ops: []
W20260709 12:19:13.116377  6622 connection.cc:256] RPC call timeout handler was delayed by -1.204923018000045s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1625s for a response
I20260709 12:19:13.147825  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:13.149930  6507 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:13.150413  6583 raft_consensus.cc:2393] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 11: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:19:13.150245  6622 connection.cc:256] RPC call timeout handler was delayed by -1.12302301800014s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1716s for a response
I20260709 12:19:13.147953  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:19:13.151003  6622 connection.cc:256] RPC call timeout handler was delayed by -1.029423018000216s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18200000000000002s for a response
I20260709 12:19:13.109719 11616 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:13.172588  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:13.172557  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:13.167748  6658 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 13: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:13.167928  6548 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:13.105116 11513 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:13.101833 11717 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:13.177640  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:13.179865  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:13.181843  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:13.130493 11827 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:13.187021  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:13.183702  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:13.189548  7271 raft_consensus.cc:1240] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 11 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:13.187881  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:13.190945  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:13.192449  6622 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:13.187094  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:13.176209 11739 consensus_queue.cc:1059] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 12:19:13.197117  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:13.197227  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:13.197504 11739 raft_consensus.cc:3055] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:13.197912 11739 raft_consensus.cc:740] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:13.198411  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:13.188427 11513 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:13.198747 11739 consensus_queue.cc:260] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:13.200042 11739 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:13.141322 11398 consensus_queue.cc:1059] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:19:13.201733  6548 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:13.200623 11513 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:13.190976 11782 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:13.196192  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:13.186949 11743 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:13.191107 11720 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.204303 11398 raft_consensus.cc:3055] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:13.207273 11398 raft_consensus.cc:740] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:13.207115  6622 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:13.208606 11759 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:13.197919  6583 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:19:13.210258 11843 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:13.208210 11398 consensus_queue.cc:260] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:13.213107 11737 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:13.213572 11398 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:13.214509  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:13.215593 11737 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:13.216960  6622 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:13.220356 11843 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:13.221845  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:13.173101 11327 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:13.224956 11327 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:13.227813 11327 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:13.109146 11708 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:13.228549  6583 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:19:13.231014  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:13.222185 11771 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:13.234599  6507 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:19:13.234589  6622 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:13.205356  6548 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:19:13.205426 11866 log.cc:927] Time spent T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:19:13.243595 11778 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:13.206496 11720 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:13.244038 11778 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:13.244465 11778 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:13.206662 11481 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:13.242178  6548 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:13.173807 11848 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:19:13.204929 11863 log.cc:927] Time spent T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:19:13.242954  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:13.174499 11756 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 12:19:13.261574  6654 raft_consensus.cc:1240] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:13.262086  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:13.192819 11840 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:13.253026  6507 raft_consensus.cc:2368] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 11. Current term is 12.
I20260709 12:19:13.263945  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:13.206323 11743 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:13.249303 11827 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:13.269316 11398 consensus_queue.cc:1059] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 12:19:13.251830 11754 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:13.260258 11771 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:13.267778  6508 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
W20260709 12:19:13.254626  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 84371.8us
I20260709 12:19:13.280433  6654 raft_consensus.cc:1240] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:13.280817  6622 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:13.265343  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:13.277760 11398 raft_consensus.cc:3055] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:13.304697  6656 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:13.239570 11759 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:13.263196 11848 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:13.269189 11820 log.cc:927] Time spent T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 1.016s	user 0.003s	sys 0.000s
I20260709 12:19:13.298834  6507 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:13.273571 11719 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:13.233419  6649 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
W20260709 12:19:13.315759  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 194471us
I20260709 12:19:13.315886  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:13.318711  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:13.296912  6475 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:13.333302  6507 raft_consensus.cc:2410] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:19:13.333674  6475 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:13.335253  6475 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:13.263218 11724 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 12:19:13.266639 11813 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
W20260709 12:19:13.339123  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:13.347550  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:13.294508 11769 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.348624  6507 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:13.293242 11478 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:13.320235  6623 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:13.281083 11827 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:13.335379  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:13.288055 11327 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:13.346011  6475 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:13.356217  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:13.357096  6506 raft_consensus.cc:2410] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 12:19:13.357281  6475 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:19:13.341305 11807 log.cc:927] Time spent T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 1.311s	user 0.000s	sys 0.003s
I20260709 12:19:13.316773 11398 raft_consensus.cc:740] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:13.361042  6507 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:19:13.363301  6623 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:13.362989  6475 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:13.337826 11754 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:13.348814 11636 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:13.319479 11842 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:13.395226  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:13.351192 11708 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:13.347972 11760 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:13.349000 11819 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:19:13.395349  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 60057.3us
I20260709 12:19:13.341411 11783 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:13.351749 11858 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:13.356693 11827 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:13.400409  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:13.470551  6507 raft_consensus.cc:2393] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 14: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:13.360306 11717 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:13.412874  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:13.414234 11868 log.cc:927] Time spent T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 12:19:13.415126 11837 raft_consensus.cc:3055] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:13.415622 11636 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:13.393923 11481 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:13.416496  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:13.476562  6575 raft_consensus.cc:2393] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 13: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:13.419019 11842 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:13.366520 11759 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:13.421572  6656 raft_consensus.cc:1240] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:13.424140  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:13.406770 11742 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:13.362282 11327 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:13.430279  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:13.435253 11579 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:13.443339  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:13.444849  6651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2226) took 1999 ms. Trace:
I20260709 12:19:13.401319 11756 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:13.448653 11823 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:13.457930 11862 log.cc:927] Time spent T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.477s	user 0.000s	sys 0.002s
I20260709 12:19:13.365890 11730 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:13.461863  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:13.462975  6658 raft_consensus.cc:1240] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:13.466162  7273 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:19:13.468693  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:19:13.473430  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:13.358896 11769 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:13.479979  6504 raft_consensus.cc:2393] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 12: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:13.428465  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:13.484495  6651 rpcz_store.cc:276] 0709 12:19:11.445189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.763483 (+318294us) service_pool.cc:224] Handling call
0709 12:19:11.763883 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:11.805367 (+ 41484us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:19:11.805405 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:11.853554 (+ 48149us) log.cc:844] Serialized 26 byte log entry
0709 12:19:12.127157 (+273603us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:12.127218 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:12.127320 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:13.239841 (+1112521us) spinlock_profiling.cc:243] Waited 15.5 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:13.444458 (+204617us) raft_consensus.cc:1685] finished
0709 12:19:13.444493 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:13.444776 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54311,"raft.queue_time_us":57433,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":15162,"spinlock_wait_cycles":35588352,"thread_start_us":182144,"threads_started":1,"wal-append.queue_time_us":336444}
I20260709 12:19:13.475150  6549 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:13.482946  6583 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 13: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:13.362553 11478 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:13.476946  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:13.499229  6576 raft_consensus.cc:2393] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 10: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:13.495944  7238 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:13.502547  6622 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:13.506939  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:13.507509  6575 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:13.511442  6504 raft_consensus.cc:1240] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 12:19:13.512591  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:13.475739 11837 raft_consensus.cc:740] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:13.514281  6576 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:13.479871 11636 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:13.510905  6622 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:13.543071 11478 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:13.551642 11698 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:13.509473 11015 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:13.552646 11015 raft_consensus.cc:697] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:13.553607 11513 consensus_queue.cc:1059] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:19:13.553776 11015 consensus_queue.cc:237] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:13.494804 11756 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:13.569199  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:13.585191  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 221940us
I20260709 12:19:13.494256 11832 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.589112 11832 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:13.412096 11812 consensus_queue.cc:260] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:13.590876 11812 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:13.591179  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:13.593014 11837 consensus_queue.cc:260] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:13.594311 11837 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:13.474025 11398 consensus_queue.cc:260] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:13.536585  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:13.475814 11778 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:13.599418  6575 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:13.604866  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:13.480944 11421 consensus_queue.cc:1059] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.074s
I20260709 12:19:13.505229 11481 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:13.570394 11759 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:19:13.602562  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 66072.5us
I20260709 12:19:13.599211 11756 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:13.508309 11730 raft_consensus.cc:697] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 17 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:13.527490 11740 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:13.617372  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:19:13.606103  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 243137us
I20260709 12:19:13.555975 11636 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:13.617452  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:13.587353 11742 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:13.471052 11783 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:13.604424 11875 log.cc:927] Time spent T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.052s	user 0.005s	sys 0.000s
I20260709 12:19:13.619616  6475 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:13.606156 11842 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:13.622116  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:13.625770  6475 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:13.627530  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:13.629168  6575 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:19:13.628201  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:13.630277  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:13.632575  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:13.633657  6623 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:13.633703  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:13.635185  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:13.635407  6475 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:13.609416 11398 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:13.630764  6576 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:19:13.637290  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:13.640805  6508 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 2317) took 2032 ms (client timeout 2377 ms). Trace:
W20260709 12:19:13.641284  6508 rpcz_store.cc:269] 0709 12:19:11.603844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.640771 (+ 36927us) service_pool.cc:224] Handling call
0709 12:19:13.636096 (+1995325us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b540001c7d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:13.636201 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3739416320}
I20260709 12:19:13.641288  6576 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:13.647629  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:13.648442  6575 raft_consensus.cc:2393] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 16: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:13.650072  6623 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:13.650530  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:13.652102  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:13.656915  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:13.658447  6656 raft_consensus.cc:2393] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 13: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:13.659454  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:13.622958 11717 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:13.660570  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:13.664775  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:13.664062  6623 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:13.636651 11421 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:13.634992 11831 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:13.638244 11277 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:13.612948 11616 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:13.672750  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:13.633924 11759 raft_consensus.cc:697] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:13.675961  6575 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:13.654901 11819 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:13.641181 11742 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:19:13.678630  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:13.679555  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:13.679797  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:13.685316  7238 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:13.695096  7238 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:13.635550 11478 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 12:19:13.696647  6473 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:13.704882  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:13.705767  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:13.707023  6575 raft_consensus.cc:1275] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:19:13.672264 11848 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:13.708949  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:13.698144  6654 raft_consensus.cc:1275] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:13.701975 11883 consensus_queue.cc:1059] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:13.677160 11729 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:19:13.744684 11876 log.cc:927] Time spent T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 12:19:13.668102 11842 raft_consensus.cc:697] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:13.745918 11842 consensus_queue.cc:237] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:13.746918 11845 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:13.747392 11845 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:13.747754 11845 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:13.748389 11740 raft_consensus.cc:2804] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:13.749080 11740 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:13.749706 11740 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:13.716511 11749 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:13.752206  6652 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:13.752490 11749 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:13.752683 11759 consensus_queue.cc:237] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:13.681528 11860 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:13.712792  6473 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:13.749585 11813 consensus_queue.cc:1059] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:13.757336 11813 consensus_queue.cc:1243] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer 20a202c76244485fbc8f1610473db8b9 log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:19:13.743124 11760 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:13.759552 11743 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:13.760654  6501 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
W20260709 12:19:13.769070  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 148117us
I20260709 12:19:13.770032  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:13.682258 11421 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:13.765544  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:13.776122 11761 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:13.769367  6652 raft_consensus.cc:1240] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:19:13.709959 11809 consensus_queue.cc:1059] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 12:19:13.779237  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:19:13.772912  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 60178.2us
I20260709 12:19:13.768743 11760 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:19:13.781018 11874 log.cc:927] Time spent T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 12:19:13.784650 11783 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:13.777380 11848 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:13.743508 11730 consensus_queue.cc:237] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:13.785576 11872 log.cc:927] Time spent T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.496s	user 0.001s	sys 0.001s
I20260709 12:19:13.788585 11654 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:19:13.789583 11881 log.cc:927] Time spent T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 12:19:13.790879 11616 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:13.791349 11823 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:13.799149  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:13.832505  6652 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:13.800040  6473 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:13.806120  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:13.811353  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:13.834765  6622 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:13.836486  6622 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:13.845476  6622 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:13.827701 11783 raft_consensus.cc:3055] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:13.812141 11885 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:13.853394 11860 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:13.853488 11729 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:19:13.864734  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 64770.1us
I20260709 12:19:13.916203  6473 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:13.876760  6622 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:13.849115 11761 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:13.849821 11889 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:13.827252 11760 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:13.920835  6652 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
W20260709 12:19:13.911376  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 136811us
W20260709 12:19:13.927934  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 51288.7us
I20260709 12:19:13.954619  6622 leader_election.cc:400] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 11. Current term is 12.
I20260709 12:19:13.955081  6622 leader_election.cc:403] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
W20260709 12:19:13.949916  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 258134us
I20260709 12:19:13.851719 11858 consensus_queue.cc:1059] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 12:19:13.958375  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:13.959560  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:13.907435 11778 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:13.910108 11616 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:13.911372 11823 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:13.909960 11783 raft_consensus.cc:740] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:13.965826  6622 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:13.912564 11860 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:13.954957 11711 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:19:13.978200  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:13.980892  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:13.952800 11481 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:13.970016 11480 raft_consensus.cc:3055] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:13.962989 11759 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 11. Current term is 12.
W20260709 12:19:13.990347  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:13.985038  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:13.956825 11729 raft_consensus.cc:697] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:13.991053 11480 raft_consensus.cc:740] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:13.994768 11616 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:13.998387 11729 consensus_queue.cc:237] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:13.999683 11481 raft_consensus.cc:697] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:13.965271 11897 consensus_queue.cc:1059] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:19:14.000816 11481 consensus_queue.cc:237] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:13.951237 11848 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:19:13.998153  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:13.998550 11480 consensus_queue.cc:260] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:13.975440 11885 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:19:13.984280 11892 log.cc:927] Time spent T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
W20260709 12:19:14.014253  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 52044.6us
I20260709 12:19:13.982777 11860 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:14.015784  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:13.964620 11703 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:19:13.991676 11889 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:13.954622 11761 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:13.995085 11711 raft_consensus.cc:697] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:19:14.049270  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 78113.8us
I20260709 12:19:14.034966 11480 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:14.033609 11885 raft_consensus.cc:697] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:14.053244  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:14.053771  6505 raft_consensus.cc:2368] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 12. Current term is 13.
I20260709 12:19:14.051923  7265 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 15. Current term is 16. Ops: []
W20260709 12:19:14.056262  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:14.060078 11711 consensus_queue.cc:237] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:14.062156 11636 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:14.064862 11761 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:14.066556 11885 consensus_queue.cc:237] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:14.068050 11832 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:14.071576 11888 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
W20260709 12:19:14.074198  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:14.076102 11827 consensus_meta.cc:287] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.598s	user 0.010s	sys 0.000s
I20260709 12:19:14.077378  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:14.088294  6652 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:19:14.078151 11760 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:14.078683  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:14.101428  6505 raft_consensus.cc:2393] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 16: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:14.081070 11845 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:14.081594  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 445018us
I20260709 12:19:14.034751 11783 consensus_queue.cc:260] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:14.086527  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:14.087649  6656 raft_consensus.cc:1275] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:19:14.111627  7267 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 13 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:14.102946 11827 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:14.113369 11783 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:14.115119 11827 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:14.115720  6579 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:14.116775 11636 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:14.122008 11860 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:14.117631  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:14.111629 11756 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:14.126394 11756 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:14.127473  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:14.128970  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:14.129968  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:14.130514 11761 consensus_queue.cc:1059] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:14.132611 11781 raft_consensus.cc:3055] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:14.133358 11781 raft_consensus.cc:740] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:14.133639  7272 raft_consensus.cc:2368] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 9. Current term is 10.
I20260709 12:19:14.132736 11845 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:14.130505  6505 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 9.
I20260709 12:19:14.136271  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:14.137451  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:14.138163  7273 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38558 (request call id 733) took 2919 ms. Trace:
I20260709 12:19:14.138509  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:14.139642  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:14.140733 11720 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:14.142010  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 2535) took 1037 ms. Trace:
I20260709 12:19:14.142607  6501 rpcz_store.cc:276] 0709 12:19:13.105065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.198130 (+ 93065us) service_pool.cc:224] Handling call
0709 12:19:14.141847 (+943717us) spinlock_profiling.cc:243] Waited 561 ms on lock 0x7b5400396cd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:14.141953 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1291282304}
I20260709 12:19:14.146972 11720 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:14.147912 11720 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:14.148911  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:14.149712  6655 raft_consensus.cc:2393] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 16: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:14.150628  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:14.151324  6579 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:14.135673 11781 consensus_queue.cc:260] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:14.152383 11781 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:14.152657 11720 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:14.157739  7238 leader_election.cc:400] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 12. Current term is 13.
I20260709 12:19:14.158159  7238 leader_election.cc:403] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:19:14.160903 11832 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 12. Current term is 13.
W20260709 12:19:14.162279  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 50111.6us
I20260709 12:19:14.169404  6475 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:14.171099  6651 raft_consensus.cc:1275] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:19:14.172201  7240 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:14.172540  6475 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:14.140298  6505 raft_consensus.cc:2393] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 16: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:14.138543  7273 rpcz_store.cc:276] 0709 12:19:11.218602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:11.222232 (+  3630us) service_pool.cc:224] Handling call
0709 12:19:11.222564 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:13.493803 (+2271239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.137936 (+644133us) spinlock_profiling.cc:243] Waited 2.24 s on lock 0x7b54000a50d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:14.138091 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5146838272}
I20260709 12:19:14.175073  6475 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:14.175724  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:14.176216  6475 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:14.178010  6475 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:14.179757  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:14.183421 11712 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:14.183854 11809 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:14.188889 11480 consensus_queue.cc:1059] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 12:19:14.195621 11789 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:14.182602 11848 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:14.201181 11740 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:14.205227  6657 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 2245) took 1315 ms (client timeout 1648 ms). Trace:
W20260709 12:19:14.211484  6657 rpcz_store.cc:269] 0709 12:19:12.889960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:12.893833 (+  3873us) service_pool.cc:224] Handling call
0709 12:19:14.204960 (+1311127us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b54003e85d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:14.205149 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":277729152}
I20260709 12:19:14.172065 11720 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:14.188529 11814 consensus_queue.cc:1048] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 12:19:14.207665  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:14.177270 11760 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:14.124172 11860 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:14.210146  6475 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:14.217708  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:14.216642  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:14.235785  6505 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:14.241977  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:14.242933  6580 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:19:14.246008  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:14.250873  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:14.253191  6580 raft_consensus.cc:2393] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 16: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:14.214263 11712 consensus_queue.cc:1059] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:19:14.238986 11809 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:14.254240  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:14.240175 11720 raft_consensus.cc:697] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:14.270639  6580 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:14.273151  6652 raft_consensus.cc:1240] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:14.273898  6501 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:14.276889  6505 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
I20260709 12:19:14.280505  7238 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:14.274803 11809 raft_consensus.cc:697] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:14.289948  6576 raft_consensus.cc:1240] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
W20260709 12:19:14.291435  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 127133us
I20260709 12:19:14.294059  6576 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:19:14.295751 11782 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:14.312469 11782 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:14.313364 11873 log.cc:927] Time spent T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.717s	user 0.002s	sys 0.000s
I20260709 12:19:14.305483 11703 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.314551 11703 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:14.315127 11782 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:14.328137  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:14.298885 11761 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:19:14.338493  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 50921.4us
I20260709 12:19:14.341837  7272 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:19:14.340621  6582 raft_consensus.cc:1275] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:14.333206 11913 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:14.306624 11720 consensus_queue.cc:237] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:14.335467 11740 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:14.351866 11920 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:19:14.355590 11880 log.cc:927] Time spent T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.760s	user 0.000s	sys 0.003s
I20260709 12:19:14.357079  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:14.338739 11809 consensus_queue.cc:237] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:14.406312 11782 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 12:19:14.411901 11769 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:14.407141 11910 log.cc:927] Time spent T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.078s	user 0.004s	sys 0.001s
I20260709 12:19:14.417663 11015 raft_consensus.cc:3055] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:14.418190 11015 raft_consensus.cc:740] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:14.406293  6582 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:14.407985 11478 raft_consensus.cc:3055] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:14.408653  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:14.422305  7240 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:14.420748  6474 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:14.422534  6656 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:14.409262  6580 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:14.439086  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:14.439617 11781 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:14.441097  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:14.442776  6474 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:14.445443 11760 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:14.419180 11769 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.406901 11913 raft_consensus.cc:697] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 10 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:14.422320 11478 raft_consensus.cc:740] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:14.419165 11015 consensus_queue.cc:260] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:14.426147 11711 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:14.449587 11015 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:14.453518 11882 log.cc:927] Time spent T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.815s	user 0.000s	sys 0.002s
I20260709 12:19:14.457726 11698 raft_consensus.cc:3055] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:14.458164 11698 raft_consensus.cc:740] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:14.458550 11327 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:14.458787  6656 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 9.
I20260709 12:19:14.459015 11698 consensus_queue.cc:260] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:14.446324 11636 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 12:19:14.446182 11782 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.460688  7240 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:14.461293 11782 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:14.461645 11832 raft_consensus.cc:2804] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:14.462371 11832 raft_consensus.cc:697] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 9 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:14.463263 11832 consensus_queue.cc:237] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:14.464008 11782 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:14.464526 11769 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:14.473356  7268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38558 (request call id 750) took 1362 ms. Trace:
I20260709 12:19:14.460170 11698 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:14.474061  7268 rpcz_store.cc:276] 0709 12:19:13.111178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.111318 (+   140us) service_pool.cc:224] Handling call
0709 12:19:13.141086 (+ 29768us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:13.141468 (+   382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:13.141495 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:13.176080 (+ 34585us) log.cc:844] Serialized 26 byte log entry
0709 12:19:13.465569 (+289489us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:13.466041 (+   472us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:13.466131 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:14.354788 (+888657us) spinlock_profiling.cc:243] Waited 2.63 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:14.472809 (+118021us) raft_consensus.cc:1685] finished
0709 12:19:14.472843 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:14.473263 (+   420us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1561,"raft.queue_time_us":5042,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":11642,"spinlock_wait_cycles":6041856,"thread_start_us":268198,"threads_started":1,"wal-append.queue_time_us":404704}
I20260709 12:19:14.478371 11848 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.478919 11848 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:14.469305 11711 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:14.447718  6474 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:14.471230 11789 raft_consensus.cc:3055] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:14.487357  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:14.487481 11789 raft_consensus.cc:740] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:14.471834  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:14.460760  6506 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 2536) took 1355 ms (client timeout 1669 ms). Trace:
W20260709 12:19:14.489147  6506 rpcz_store.cc:269] 0709 12:19:13.111643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.166760 (+ 55117us) service_pool.cc:224] Handling call
0709 12:19:14.460564 (+1293804us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b54002c51d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:14.460705 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":401000192}
W20260709 12:19:14.488888 11921 log.cc:927] Time spent T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:19:14.490727 11327 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:14.490995 11848 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:14.491128  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:14.491809  6506 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:14.491479 11913 consensus_queue.cc:237] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:14.495013 11789 consensus_queue.cc:260] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:14.496213 11789 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:14.499716  6506 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:14.500855  6622 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:14.502400 11782 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:14.502775  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2524) took 1401 ms. Trace:
I20260709 12:19:14.502897 11782 raft_consensus.cc:697] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:19:14.503052  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:14.489274 11703 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:14.503783 11782 consensus_queue.cc:237] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:14.505453  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:14.503226  6503 rpcz_store.cc:276] 0709 12:19:13.101305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.101464 (+   159us) service_pool.cc:224] Handling call
0709 12:19:13.103968 (+  2504us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:13.104897 (+   929us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:13.104929 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:13.105884 (+   955us) log.cc:844] Serialized 26 byte log entry
0709 12:19:13.340555 (+234671us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:14.502290 (+1161735us) log.cc:844] Serialized 14 byte log entry
0709 12:19:14.502501 (+   211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.502573 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:14.502609 (+    36us) raft_consensus.cc:1685] finished
0709 12:19:14.502621 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:14.502724 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1161386,"raft.queue_time_us":737,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":164,"thread_start_us":214392,"threads_started":1,"wal-append.queue_time_us":396309}
I20260709 12:19:14.508287  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:14.508864  6503 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:19:14.506666  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:14.501533  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2521) took 1479 ms. Trace:
W20260709 12:19:14.527307  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:14.529259  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:14.530169  6503 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:14.533649  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:14.535171  6506 raft_consensus.cc:2393] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 14: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:14.509738  6509 rpcz_store.cc:276] 0709 12:19:13.022425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.022584 (+   159us) service_pool.cc:224] Handling call
0709 12:19:13.023026 (+   442us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:13.023485 (+   459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:13.023530 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:13.024293 (+   763us) log.cc:844] Serialized 52 byte log entry
0709 12:19:13.309757 (+285464us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:14.312531 (+1002774us) spinlock_profiling.cc:243] Waited 3.9 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:14.500926 (+188395us) log.cc:844] Serialized 14 byte log entry
0709 12:19:14.501155 (+   229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.501221 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:14.501255 (+    34us) raft_consensus.cc:1685] finished
0709 12:19:14.501343 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:14.501476 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1191093,"raft.queue_time_us":2626,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":1533,"spinlock_wait_cycles":8971008,"thread_start_us":264520,"threads_started":1,"wal-append.queue_time_us":562736}
I20260709 12:19:14.559135  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:14.559733  6508 raft_consensus.cc:1813] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Handling vote request from an unknown peer f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:14.560137  6508 raft_consensus.cc:2368] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 9. Current term is 10.
I20260709 12:19:14.536873 11848 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:14.541914 11769 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:14.554203 11636 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:14.568663 11760 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:14.569205  6508 raft_consensus.cc:1275] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 12:19:14.569702  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:14.566653 11848 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:14.606212  6509 raft_consensus.cc:1240] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:14.611850  6508 raft_consensus.cc:1240] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:14.613481  6508 raft_consensus.cc:1240] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:14.614313 11769 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:14.616441 11769 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:14.618618 11769 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:14.619903  6500 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2525) took 1518 ms. Trace:
I20260709 12:19:14.620576 11654 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:14.621160 11654 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:14.620369  6500 rpcz_store.cc:276] 0709 12:19:13.101698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.101832 (+   134us) service_pool.cc:224] Handling call
0709 12:19:13.421353 (+319521us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b5800002a98. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:19:13.421546 (+   193us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:13.421923 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:13.421952 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:13.422529 (+   577us) log.cc:844] Serialized 52 byte log entry
0709 12:19:13.622760 (+200231us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:14.071024 (+448264us) spinlock_profiling.cc:243] Waited 2.37 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:14.608412 (+537388us) log.cc:844] Serialized 14 byte log entry
0709 12:19:14.616049 (+  7637us) log.cc:844] Serialized 14 byte log entry
0709 12:19:14.619584 (+  3535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.619660 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:14.619701 (+    41us) raft_consensus.cc:1685] finished
0709 12:19:14.619714 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:14.619844 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":893963,"prepare.queue_time_us":955379,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":60,"raft.queue_time_us":35818,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":250,"spinlock_wait_cycles":7991424,"thread_start_us":300365,"threads_started":2,"wal-append.queue_time_us":353248}
I20260709 12:19:14.623272  6500 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:14.633541 11848 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:14.592826 11636 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:14.639611 11421 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:14.644464  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:14.646522 11636 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:14.658903  6622 leader_election.cc:400] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 9. Current term is 10.
I20260709 12:19:14.659338  6622 leader_election.cc:403] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:14.661263 11749 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:14.663425 11773 raft_consensus.cc:3055] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:14.663857 11773 raft_consensus.cc:740] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:14.664860 11773 consensus_queue.cc:260] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:14.665949 11773 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:14.671847  6656 raft_consensus.cc:1240] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:19:14.673700  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:14.679080  6656 raft_consensus.cc:3055] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:14.679595  6656 raft_consensus.cc:740] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:14.640407 11654 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:14.690348  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:14.692890  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:14.693645  6658 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:19:14.695204  6503 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:19:14.699720  6503 raft_consensus.cc:1240] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:14.702478  6503 raft_consensus.cc:1275] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:19:14.703953  6503 raft_consensus.cc:1275] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:19:14.696964  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:14.680547  6656 consensus_queue.cc:260] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:14.706900  6656 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:14.707597  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:14.713394  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:14.739331  6503 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:19:14.712932 11885 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:19:14.712634 11858 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:14.712774 11636 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:14.689235 11854 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 12:19:14.756295  6649 raft_consensus.cc:2393] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 16: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:14.756425  6475 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:14.746992  6503 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:19:14.761737  6581 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:19:14.708065 11840 consensus_queue.cc:1059] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:19:14.712798 11616 consensus_queue.cc:1059] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 12:19:14.764056  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:14.715425 11860 consensus_queue.cc:1048] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:19:14.770529  6655 raft_consensus.cc:2393] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 10: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:14.771008  6503 raft_consensus.cc:1275] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
W20260709 12:19:14.808192  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 51855.9us
I20260709 12:19:14.812143 11474 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:14.812618 11474 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:14.813033 11474 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:14.772423 11481 consensus_queue.cc:1048] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:19:14.816846 11735 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:14.817745 11735 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:14.819967 11735 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:14.821069  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:14.826992  6502 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2530) took 1723 ms. Trace:
I20260709 12:19:14.827802  6475 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:14.826505 11474 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:14.827464  6502 rpcz_store.cc:276] 0709 12:19:13.103385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.483369 (+379984us) service_pool.cc:224] Handling call
0709 12:19:13.493078 (+  9709us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:13.493378 (+   300us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:13.493417 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:13.498439 (+  5022us) log.cc:844] Serialized 26 byte log entry
0709 12:19:13.663469 (+165030us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:14.800581 (+1137112us) log.cc:844] Serialized 14 byte log entry
0709 12:19:14.826688 (+ 26107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.826765 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:14.826819 (+    54us) raft_consensus.cc:1685] finished
0709 12:19:14.826830 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:14.826938 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":869111,"prepare.queue_time_us":913430,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":25030,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":87,"thread_start_us":368560,"threads_started":2,"wal-append.queue_time_us":244992}
I20260709 12:19:14.830618 11474 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:14.831342 11735 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:14.764544 11717 consensus_queue.cc:1059] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:19:14.834268  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:14.834909  6655 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:19:14.837327  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:14.837731 11654 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:14.837893  6655 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:14.759600 11636 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:14.841710  6650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 2254) took 1193 ms. Trace:
I20260709 12:19:14.842362 11636 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:14.842984  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 2312) took 1492 ms. Trace:
I20260709 12:19:14.843329  6581 rpcz_store.cc:276] 0709 12:19:13.350787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.352076 (+  1289us) service_pool.cc:224] Handling call
0709 12:19:14.842841 (+1490765us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54001d0bd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:14.842932 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3016205184}
I20260709 12:19:14.842096  6650 rpcz_store.cc:276] 0709 12:19:13.648607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.664969 (+ 16362us) service_pool.cc:224] Handling call
0709 12:19:14.841531 (+1176562us) spinlock_profiling.cc:243] Waited 849 ms on lock 0x7b540008c0d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:14.841647 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1952123264}
I20260709 12:19:14.848069  6503 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:14.849790  6655 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:19:14.846619 11842 raft_consensus.cc:3055] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:14.854772  6475 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:14.863482  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:14.865079  6655 raft_consensus.cc:2393] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 16: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:14.865012  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:14.862440  6475 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:14.859911 11842 raft_consensus.cc:740] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:14.864684 11474 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:14.863080 11858 raft_consensus.cc:697] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:19:14.874681  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 161484us
I20260709 12:19:14.874900  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2532) took 1770 ms. Trace:
I20260709 12:19:14.836781  6656 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 12:19:14.894424  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:14.894217  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:14.870349 11636 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:19:14.911186  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 838597us
W20260709 12:19:14.920403  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:14.921763  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:14.875458  6507 rpcz_store.cc:276] 0709 12:19:13.104144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.529977 (+425833us) service_pool.cc:224] Handling call
0709 12:19:13.530395 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:13.530641 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:14.709053 (+1178412us) log.cc:844] Serialized 14 byte log entry
0709 12:19:14.874557 (+165504us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:14.874609 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:14.874652 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.874715 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:14.874843 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":976394,"prepare.queue_time_us":1045797,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":152,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":3487,"thread_start_us":366649,"threads_started":2}
W20260709 12:19:14.927248  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 815316us
I20260709 12:19:14.928568  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:14.929225  6579 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:19:14.893452 11899 consensus_queue.cc:1059] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:19:14.927942  6550 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:14.933357  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:14.893508 11398 raft_consensus.cc:3055] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:14.937438  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:14.938246  6579 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:14.937547 11398 raft_consensus.cc:740] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:14.939574 11398 consensus_queue.cc:260] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:14.940475  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:14.911305 11840 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.941398  6584 raft_consensus.cc:2410] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:19:14.911710 11782 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:14.942320  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:14.873979 11474 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:14.942668  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:14.894284 11896 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.940661 11398 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:14.944087  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:14.945734  6581 raft_consensus.cc:1240] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:19:14.946338  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:14.954180  6582 raft_consensus.cc:1275] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:19:14.949967  6580 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 10
W20260709 12:19:14.956727  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:14.957409  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:14.957788  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:14.954293 11782 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:14.949157 11840 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:14.960775 11782 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:14.962212  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2339) took 1343 ms. Trace:
I20260709 12:19:14.962620 11840 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:14.964704  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:14.964869 11478 consensus_queue.cc:260] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:14.965909  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:14.966113 11478 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:14.967361  6505 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:19:14.971732  6576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 2236) took 1204 ms. Trace:
I20260709 12:19:14.972183  6576 rpcz_store.cc:276] 0709 12:19:13.767244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.297228 (+529984us) service_pool.cc:224] Handling call
0709 12:19:14.297629 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.297924 (+   295us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:19:14.304881 (+  6957us) log.cc:844] Serialized 14 byte log entry
0709 12:19:14.971388 (+666507us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:14.971442 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:14.971489 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.971556 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:14.971672 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":566220,"prepare.queue_time_us":7315,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":4364,"raft.queue_time_us":566958,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":77,"thread_start_us":99935,"threads_started":1,"wal-append.queue_time_us":128802}
I20260709 12:19:14.975766 11906 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.976559  6577 raft_consensus.cc:1275] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 12:19:14.976274 11906 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:14.976893  6577 pending_rounds.cc:85] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 6
I20260709 12:19:14.977597  6577 pending_rounds.cc:107] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:19:14.977708  6548 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:14.977888  6577 raft_consensus.cc:2889] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:14.962639  6501 rpcz_store.cc:276] 0709 12:19:13.618510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.143560 (+525050us) service_pool.cc:224] Handling call
0709 12:19:14.143918 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.275719 (+131801us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.961977 (+686258us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b54002c38d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:14.962145 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":298363136}
I20260709 12:19:14.979749  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:14.927951 11636 raft_consensus.cc:697] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 13 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:14.980298  6580 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
I20260709 12:19:14.981086  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:14.983110  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 55257.5us
I20260709 12:19:14.983522 11823 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:14.983794 11767 raft_consensus.cc:2764] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:19:14.984074 11823 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:14.974349  6505 raft_consensus.cc:1240] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:19:14.986178 11823 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:14.986377 11698 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:14.986716  6550 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:14.986881 11698 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:14.987426  6579 raft_consensus.cc:3055] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:14.987560 11698 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:14.987841  6579 raft_consensus.cc:740] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:14.988204  6550 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:14.988884  6579 consensus_queue.cc:260] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:14.989357  6550 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:14.981684  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:14.990013  6579 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:14.990314  6582 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:14.990482 11905 consensus_queue.cc:1059] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:19:14.991218 11904 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:14.991927 11904 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:14.983306  6581 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:19:14.992498 11813 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:14.957803 11835 raft_consensus.cc:3055] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:14.993476 11835 raft_consensus.cc:740] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:14.994159  6550 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:14.994553 11835 consensus_queue.cc:260] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:14.995325 11767 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:14.995582  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 622) took 1355 ms. Trace:
I20260709 12:19:14.995767 11835 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:14.996050  6509 rpcz_store.cc:276] 0709 12:19:13.640047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.553976 (+913929us) service_pool.cc:224] Handling call
0709 12:19:14.554347 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.606787 (+ 52440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.995342 (+388555us) spinlock_profiling.cc:243] Waited 51.6 ms on lock 0x7b540046c8d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:14.995517 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":118720896}
I20260709 12:19:14.997352  6577 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 12:19:14.997687  6577 pending_rounds.cc:85] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 6
I20260709 12:19:14.997900  6577 pending_rounds.cc:107] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:19:14.998102  6577 raft_consensus.cc:2889] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:14.884655 11842 consensus_queue.cc:260] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:14.983174  6500 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 631) took 1337 ms. Trace:
I20260709 12:19:14.981805  6502 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:15.001776  6500 rpcz_store.cc:276] 0709 12:19:13.645773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.623986 (+978213us) service_pool.cc:224] Handling call
0709 12:19:14.624227 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.662435 (+ 38208us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:19:14.662756 (+   321us) log.cc:844] Serialized 14 byte log entry
0709 12:19:14.846106 (+183350us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:14.846175 (+    69us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:14.846241 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.982971 (+136730us) spinlock_profiling.cc:243] Waited 38 ms on lock 0x7b54000f3054. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:14.983013 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:14.983123 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":625,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":75,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":1246,"spinlock_wait_cycles":88749568,"thread_start_us":182919,"threads_started":1,"wal-append.queue_time_us":218193}
I20260709 12:19:14.941306 11941 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:15.005332 11941 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:15.007529 11941 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:15.007951 11698 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:15.008723  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:15.009290  6584 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:15.011008  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:15.013386 11913 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:15.013734 11845 raft_consensus.cc:3055] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:15.013969 11913 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:15.015329 11845 raft_consensus.cc:740] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:15.015290 11636 consensus_queue.cc:237] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:15.013420  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:15.016398 11845 consensus_queue.cc:260] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:15.017858 11845 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:15.016213 11913 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:14.982962 11899 raft_consensus.cc:3055] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:15.031563  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:15.033392  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:15.034863  7267 raft_consensus.cc:2393] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 16: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:15.035430  6652 raft_consensus.cc:2393] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 14: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:15.036936  6651 raft_consensus.cc:1275] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 12:19:15.048537  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 50757us
I20260709 12:19:14.952441 11474 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:15.054665  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:15.054512 11913 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:15.056780 11698 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:14.979887 11896 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:15.063558  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:15.062227  7268 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:15.034394 11899 raft_consensus.cc:740] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:15.068090 11906 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:15.077555  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:15.071419 11904 consensus_queue.cc:1059] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:15.060166 11885 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 9. Current term is 10.
I20260709 12:19:15.076215  6576 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:15.086870  6550 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:15.084074 11906 raft_consensus.cc:3055] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:15.073290  7273 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:19:15.093598  6504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2330) took 1983 ms. Trace:
I20260709 12:19:15.081759 11899 consensus_queue.cc:260] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:15.082996  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:15.083299 11896 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:15.053486 11783 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.091106  6473 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:15.078204  6577 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 14.
I20260709 12:19:15.070779  6579 raft_consensus.cc:1275] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:19:15.109786  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:15.108794 11771 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:15.112502 11783 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:15.114986 11783 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:15.116163  6473 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:15.116583 11902 log.cc:927] Time spent T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:19:15.117084 11812 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:15.117750 11812 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:15.118147 11812 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:15.118289  6575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 2231) took 1407 ms. Trace:
I20260709 12:19:15.118543 11812 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:15.118645  6575 rpcz_store.cc:276] 0709 12:19:13.710838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.834640 (+123802us) service_pool.cc:224] Handling call
0709 12:19:13.856989 (+ 22349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:13.857296 (+   307us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:13.857340 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:13.864744 (+  7404us) log.cc:844] Serialized 26 byte log entry
0709 12:19:14.255207 (+390463us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:14.274646 (+ 19439us) log.cc:844] Serialized 14 byte log entry
0709 12:19:14.970759 (+696113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.970837 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:15.060251 (+ 89414us) spinlock_profiling.cc:243] Waited 2.74 ms on lock 0x7b18003ca240. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:19:15.117993 (+ 57742us) raft_consensus.cc:1685] finished
0709 12:19:15.118086 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:15.118234 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":695823,"prepare.queue_time_us":19198,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":696283,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":76,"spinlock_wait_cycles":6308608,"thread_start_us":291141,"threads_started":1,"wal-append.queue_time_us":335084}
I20260709 12:19:15.121114 11899 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:15.095687 11735 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:15.122545 11735 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:15.107537  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:14.888926 11858 consensus_queue.cc:237] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:15.117560  6473 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:15.109819  6550 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:15.134846  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:15.136621  6576 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 9.
I20260709 12:19:15.132020  6584 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:19:15.094054  6504 rpcz_store.cc:276] 0709 12:19:13.110743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.512229 (+401486us) service_pool.cc:224] Handling call
0709 12:19:14.464457 (+952228us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:14.464655 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:19:14.764167 (+299512us) log.cc:844] Serialized 14 byte log entry
0709 12:19:14.876398 (+112231us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:14.876459 (+    61us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:14.876501 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.877867 (+  1366us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:15.093439 (+215572us) spinlock_profiling.cc:243] Waited 947 ms on lock 0x7b54002c51d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:15.093550 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":93012,"prepare.queue_time_us":112561,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":96,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"spinlock_wait_cycles":2179117312,"thread_start_us":310530,"threads_started":2,"wal-append.queue_time_us":191535}
I20260709 12:19:15.130869  6581 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 12.
I20260709 12:19:15.151763  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2314) took 1800 ms. Trace:
W20260709 12:19:15.154686  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 199272us
I20260709 12:19:15.152252  6583 rpcz_store.cc:276] 0709 12:19:13.351424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:13.514810 (+163386us) service_pool.cc:224] Handling call
0709 12:19:13.586643 (+ 71833us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:13.644163 (+ 57520us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:13.859827 (+215664us) log.cc:844] Serialized 14 byte log entry
0709 12:19:14.103044 (+243217us) log.cc:844] Serialized 14 byte log entry
0709 12:19:14.893173 (+790129us) spinlock_profiling.cc:243] Waited 915 us on lock 0x7b180013edc0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d16c0a 00007f287c456219 00007f2879d5d58f 00007f2872d9c571 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd
0709 12:19:14.917845 (+ 24672us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:14.918041 (+   196us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:14.918096 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:14.972866 (+ 54770us) spinlock_profiling.cc:243] Waited 2.17 ms on lock 0x7b180013edc0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:19:15.151522 (+178656us) spinlock_profiling.cc:243] Waited 57.2 ms on lock 0x7b54001cc354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:15.151574 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:15.151698 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":876789,"prepare.queue_time_us":359129,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":80,"raft.queue_time_us":902905,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":11768,"spinlock_wait_cycles":138741760,"thread_start_us":369538,"threads_started":3,"wal-append.queue_time_us":174046}
I20260709 12:19:15.174646  7238 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:15.176717  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:15.177552  6583 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:15.092090 11906 raft_consensus.cc:740] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:15.181537  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:15.181849  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:15.184541  6549 leader_election.cc:400] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 9. Current term is 10.
I20260709 12:19:15.192375  6549 leader_election.cc:403] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:15.192914  6576 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:15.188858  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:15.196010  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:15.197379  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:15.184618  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:15.189911  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:15.199239  6579 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:19:15.203490  6579 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:15.215593  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:15.216917  6577 raft_consensus.cc:1275] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:19:15.219174  7238 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:15.169216 11783 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:15.221223  7273 raft_consensus.cc:1275] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:19:15.224249  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:15.276649  7272 raft_consensus.cc:2393] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 12: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:15.261709 11848 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:15.226724  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:15.226619  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:15.227248  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:15.171293 11823 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:15.229259  6550 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:15.279943  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 50758.2us
I20260709 12:19:15.280905  6550 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:15.281176  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:15.231331  6502 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:15.233645 11327 consensus_queue.cc:1048] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:15.233359 11906 consensus_queue.cc:260] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:15.235205  6580 raft_consensus.cc:1275] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 6. (term mismatch)
I20260709 12:19:15.181634 11789 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:15.236009  6508 raft_consensus.cc:1240] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:15.214396 11854 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 12:19:15.238137  6548 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:19:15.240690  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:15.242616  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 86985.2us
I20260709 12:19:15.243566 11616 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:15.245075 11735 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:15.217873 11880 log.cc:927] Time spent T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:19:15.176246 11812 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:15.256445 11827 consensus_queue.cc:1059] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:15.255954 11474 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:15.229825  6583 raft_consensus.cc:1275] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:19:15.226747 11698 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 9. Current term is 10.
I20260709 12:19:15.250303  6474 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:19:15.335956  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 85732.4us
I20260709 12:19:15.226578  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:15.224128 11864 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:15.286443  6580 pending_rounds.cc:85] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 5
I20260709 12:19:15.352047  6580 pending_rounds.cc:107] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 12.6
I20260709 12:19:15.289685  6550 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:15.307253  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:15.280288 11478 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 12:19:15.315873  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:15.306700 11899 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:15.319941  6658 raft_consensus.cc:1275] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 12:19:15.318403  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 80347.9us
I20260709 12:19:15.331065  6506 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:19:15.332166  6655 raft_consensus.cc:1240] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:19:15.282352 11836 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:15.358510  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:15.359329  6507 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:15.361290  6501 raft_consensus.cc:1275] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:19:15.361785  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:15.283265 11783 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:19:15.272444  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 53374.1us
I20260709 12:19:15.366469  7238 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:15.338174  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:15.370294  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:15.371045  6658 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:15.318984 11950 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:15.332468 11951 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 12:19:15.340135 11773 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.380481  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
W20260709 12:19:15.380908  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:15.341215  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:15.382408  6654 raft_consensus.cc:2393] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 14: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:15.341712 11474 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:15.343876 11948 log.cc:927] Time spent T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:19:15.344587  6650 raft_consensus.cc:1275] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:19:15.338232 11481 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 12:19:15.328742 11940 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:19:15.344141  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:15.397114  6622 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:15.310379 11823 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:15.352890  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 63322.3us
I20260709 12:19:15.398711  6550 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:15.315128  6652 raft_consensus.cc:1275] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:19:15.353302  6580 raft_consensus.cc:2889] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:15.322520 11904 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:15.310005 11771 consensus_queue.cc:1059] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:19:15.283839 11848 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:15.374295  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 204235us
I20260709 12:19:15.411051  6473 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:15.374603  6504 raft_consensus.cc:1275] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:15.350914 11735 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:15.328902 11885 consensus_queue.cc:1048] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:19:15.357527 11894 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:15.356662 11654 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.412940  6622 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:15.386168 11864 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:15.419432  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:15.416164 10999 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:15.393735 11913 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:15.414526 11899 consensus_queue.cc:1059] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:19:15.436847 11913 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:15.408529 11823 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:15.437386 11913 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:15.437844  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:15.440989 11955 log.cc:927] Time spent T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:19:15.433475  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:15.442047  6656 raft_consensus.cc:2393] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 9: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:15.415082 11940 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:15.417538 11858 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:15.444619  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:15.445454  6657 raft_consensus.cc:2368] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 13. Current term is 14.
I20260709 12:19:15.433878  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:15.449456  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:15.449883  6649 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:15.450314  6581 raft_consensus.cc:2393] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 14: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:15.424801  6575 raft_consensus.cc:1275] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:19:15.432751  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:15.454767  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:15.471107  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 58265.9us
I20260709 12:19:15.471783  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:15.472048  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:15.455479 11654 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:15.474756 11478 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:15.399546 11739 consensus_queue.cc:1059] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 12:19:15.476262 11940 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:15.421231 11735 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:15.476667 11743 consensus_queue.cc:1059] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:15.477110 11735 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:15.478021  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:15.481673 11848 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:15.381738 11950 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:15.483814 11950 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:15.483958  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 629) took 1839 ms. Trace:
I20260709 12:19:15.486295 11812 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:15.476678 11654 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:15.492789 11698 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:15.494161 11782 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.389776 11773 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:15.497175  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:15.482086 11767 raft_consensus.cc:3055] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:15.503647  6508 rpcz_store.cc:276] 0709 12:19:13.644987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.615409 (+970422us) service_pool.cc:224] Handling call
0709 12:19:14.615717 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:15.319428 (+703711us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:15.483690 (+164262us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b54001e64d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:15.483888 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1426079104}
I20260709 12:19:15.504607  6658 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:15.512712  6622 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:15.517342  7268 raft_consensus.cc:1240] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:15.518651  6581 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:19:15.521827  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:15.508128 11952 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:15.532820  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:15.509804 11861 raft_consensus.cc:2764] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:19:15.511812 11858 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:15.510006 11767 raft_consensus.cc:740] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:15.524824 11945 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:15.524646 11940 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:15.529912 11941 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:15.529333 11421 consensus_queue.cc:1059] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 12:19:15.595696  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:15.596433  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:15.604177 11773 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:15.612625 11940 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:19:15.612918 11953 log.cc:927] Time spent T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 12:19:15.613655 11767 consensus_queue.cc:260] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:15.615154 11743 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:19:15.616600 11954 log.cc:927] Time spent T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:19:15.617319  6550 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:15.595779 11952 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:15.513265 11782 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:15.532891 11951 consensus_queue.cc:1048] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.095s
I20260709 12:19:15.629136  7273 raft_consensus.cc:2468] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 7.
I20260709 12:19:15.629187  7268 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
W20260709 12:19:15.632839  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 248243us
I20260709 12:19:15.622176 11864 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:15.639312  6550 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:15.698688  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 59452.7us
I20260709 12:19:15.700294  6548 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:15.634270 11941 raft_consensus.cc:697] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:15.702463  6550 leader_election.cc:400] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 13. Current term is 14.
I20260709 12:19:15.702939  6550 leader_election.cc:403] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
W20260709 12:19:15.705067  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 292389us
I20260709 12:19:15.709548  6550 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:15.710994  6473 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:15.711544  6550 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:15.714462  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:15.716730  6550 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:15.640199 11952 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:15.635293 11940 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:15.705901 11767 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:15.641352 11896 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:15.706802 11958 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.638831 11743 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:15.733546  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 200709us
I20260709 12:19:15.719694 11904 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:15.715740 11813 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 13. Current term is 14.
I20260709 12:19:15.718488 11398 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:15.719161 11967 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.718684 11739 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:15.646673 11913 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:15.723892 11811 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:15.748008  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:15.724465 11952 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:15.750298  6473 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:15.750764 11778 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 12:19:15.748167 11885 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 12:19:15.722023 11964 consensus_queue.cc:1059] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:19:15.749917 11398 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:15.757362 11861 consensus_queue.cc:1048] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:15.757690  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:15.739625 11899 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:19:15.729327 11959 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:15.758088  6508 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:19:15.758940  6475 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:15.758673 11967 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:15.790266 11861 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:15.790874 11861 raft_consensus.cc:697] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:15.791530 11967 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:15.755607 11735 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:15.793249 11743 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:15.793465  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:15.794231  6508 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
I20260709 12:19:15.794622 11842 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:15.795377 11735 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:15.800997  7238 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:15.766966  6473 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:15.763432 11968 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:19:15.803725 11462 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:15.773433 11398 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:15.804267 11462 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:15.804744 11968 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:15.805209 11968 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:15.805614 11968 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:15.749176 11970 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.806469 11854 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:15.807017 11970 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:15.809533 11970 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:15.810207  6582 raft_consensus.cc:3055] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:15.810683  6582 raft_consensus.cc:740] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:15.782445  6501 raft_consensus.cc:1240] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:15.768805 11966 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:15.783035  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:15.784472  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:15.823558  6507 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:15.784947  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:15.824216  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 57477.8us
I20260709 12:19:15.770540 11965 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.765812 11739 raft_consensus.cc:697] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:15.770320 11889 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:19:15.820248 11947 log.cc:927] Time spent T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.555s	user 0.003s	sys 0.000s
I20260709 12:19:15.826131  6473 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:15.747550 11958 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:15.804783 11462 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:15.774729  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:15.811677  6582 consensus_queue.cc:260] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:15.821628 11906 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:15.795701 11773 consensus_queue.cc:1048] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 12:19:15.840752  6582 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:15.835412  6473 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:15.846582  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:15.847366  6579 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:15.791991 11861 consensus_queue.cc:237] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:15.850041  6509 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 13.
I20260709 12:19:15.763898 11941 consensus_queue.cc:237] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:15.856781 11782 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:15.857158 11913 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:15.859252 11735 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:15.862926 11823 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:15.866180  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:15.866909  6507 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:15.868253 11894 raft_consensus.cc:3055] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:15.868722 11894 raft_consensus.cc:740] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:15.870309 11958 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:15.870060 11894 consensus_queue.cc:260] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:15.832082 11965 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:15.884558  6507 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 13.
I20260709 12:19:15.844084 11773 consensus_queue.cc:1243] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer 496c7c3792cd4587b4dbeba435f4be1e log is divergent from this leader: its last log entry 14.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:19:15.888227  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:15.890014  6582 raft_consensus.cc:1275] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:15.856022 11957 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:15.889810 11809 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 12:19:15.836951 11739 consensus_queue.cc:237] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:15.905845 11959 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:15.896956 11958 consensus_queue.cc:1048] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 12:19:15.906587 11823 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.892661 11913 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:15.898720 11894 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:15.908545 11835 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:15.920951 11966 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:15.940991 11897 raft_consensus.cc:3055] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:15.953891 11897 raft_consensus.cc:740] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:15.922189 11913 raft_consensus.cc:697] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 13 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:15.954370 11836 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:15.955104 11836 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:15.954823 11897 consensus_queue.cc:260] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:15.955909 11897 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:15.914726 11959 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:15.953017 11940 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:15.918030 11823 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:15.958415 11959 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:15.958573 11823 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:15.954983 11913 consensus_queue.cc:237] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:15.935209 11835 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:15.961210 11835 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:15.949621  6653 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 2270) took 1278 ms (client timeout 1508 ms). Trace:
W20260709 12:19:15.965087  6653 rpcz_store.cc:269] 0709 12:19:14.670945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:14.673427 (+  2482us) service_pool.cc:224] Handling call
0709 12:19:15.949329 (+1275902us) spinlock_profiling.cc:243] Waited 655 ms on lock 0x7b540008fcd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:15.949533 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1507297152}
I20260709 12:19:15.967736  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:15.969239  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:15.970860  6651 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:15.971894  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:15.972451  6623 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:15.987149  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 45276.2us
I20260709 12:19:15.989723  6507 raft_consensus.cc:1240] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:15.991647 11845 consensus_queue.cc:1059] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:15.993090  6576 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:15.993412 11421 consensus_queue.cc:1059] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:15.982889 11977 raft_consensus.cc:2804] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:15.994663 11977 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:15.994745 11421 raft_consensus.cc:3055] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:15.995106 11977 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:15.995294 11421 raft_consensus.cc:740] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:16.001259 11421 consensus_queue.cc:260] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:16.003909 11421 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:15.967926 11940 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.007097 11767 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.007350 11940 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:16.007754 11767 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:16.008616  6652 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:16.009568 11940 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:16.017611 11968 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:16.021921  6653 raft_consensus.cc:1240] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:16.022663  6651 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:15.969449 11950 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:15.981158 11809 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.032292  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:16.021524 11462 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:16.043948  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:16.036377 11809 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:16.056699  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:16.057539  7272 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:16.047394 11327 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.059762  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:16.097595  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:16.101773  6504 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:16.102228  7269 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:19:16.103080  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:16.105321  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:16.038683 11906 consensus_queue.cc:1059] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 12:19:16.097626  7238 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:16.111276  6509 raft_consensus.cc:1240] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 13 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:16.106464  6501 raft_consensus.cc:1275] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:19:16.107539  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:16.023547 11767 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:16.113164  7269 raft_consensus.cc:2393] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 14: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:16.052924 11940 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.049125 11957 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:16.033970 11950 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:16.047627 11965 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:16.054378 11481 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:19:16.090873 11771 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 12:19:16.072446 11809 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:16.129771  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:16.063365 11966 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:16.133661  6474 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:16.104568 11327 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:16.143262  6548 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:16.115078 11885 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:16.064879 11782 consensus_queue.cc:1048] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 12:19:16.120431 11840 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:16.138767  7269 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:19:16.131807 11481 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:16.150279 11840 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:16.153735 11481 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:16.153944 11840 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:16.159343 11836 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:16.144168 11966 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:16.160753 11749 consensus_queue.cc:1059] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:16.161636 11749 consensus_queue.cc:1243] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:19:16.161976 11481 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:16.142376 11842 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.164685 11842 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:16.164280 11481 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:16.166391 11739 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:16.141994 11771 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.167052 11481 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.167312 11771 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:16.167644 11481 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:16.166662 11842 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:16.169571 11771 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:16.171002  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:16.171339 11327 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:16.172690 11481 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:16.172993 11913 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.173313 11967 raft_consensus.cc:3055] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:16.173624 11950 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:16.173533 11913 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:16.174693 11979 log.cc:927] Time spent T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:19:16.175766 11913 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:16.176482 11905 raft_consensus.cc:3055] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:16.176869 11905 raft_consensus.cc:740] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:16.177932  6473 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:16.178118  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.132112 11959 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.131767 11958 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.181880  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:16.181648 11958 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:16.183287 11398 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:16.124114 11883 consensus_queue.cc:1059] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 12:19:16.184809 11883 consensus_queue.cc:1243] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:19:16.185027 11968 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:16.185647 11968 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:16.185994 11968 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:16.186331 11968 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:16.186695  6576 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:16.187538 11906 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:16.189160 11789 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:16.189693 11983 log.cc:927] Time spent T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:19:16.190070  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:16.125193 11940 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:16.191327  6650 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:16.192390 11940 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:16.194697  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:16.195335  6580 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:19:16.195744 11827 raft_consensus.cc:3055] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:16.196149 11827 raft_consensus.cc:740] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:16.197131 11462 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:16.196983 11827 consensus_queue.cc:260] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:16.189706 11789 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:16.198283 11827 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:16.199617 11789 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:16.202104 11967 raft_consensus.cc:740] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:16.202270  6507 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:19:16.180687 11959 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:16.151964 11783 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:16.203042 11967 consensus_queue.cc:260] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:16.063391 11941 consensus_queue.cc:1059] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 12:19:16.204663 11959 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:16.204825 11941 raft_consensus.cc:3055] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:16.205240 11941 raft_consensus.cc:740] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:16.188025 11906 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:16.205968 11941 consensus_queue.cc:260] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:16.206360 11835 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:16.148761 11957 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:19:16.207078  6393 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 3e0ab356fc074f589a6ff4f9b4996b09 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 20a202c76244485fbc8f1610473db8b9 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:19:16.207659 11957 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:16.208020 11957 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:16.208086 11906 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:16.208706  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:16.209272  6580 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:16.209647 11958 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:16.210908  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:16.211239  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:16.211501  6504 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
I20260709 12:19:16.179921 11977 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:16.214425  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:16.214773 11952 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:16.215154  6653 raft_consensus.cc:1240] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 7. Current term is 8. Ops: []
I20260709 12:19:16.215196 11977 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:16.147146 11885 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:16.216866 11952 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:16.217029 11885 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:16.218307 11771 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.218807 11771 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:16.220104  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.221477  6649 raft_consensus.cc:1275] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:19:16.222473  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:16.223171  6658 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
I20260709 12:19:16.224687 11967 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:16.206943 11835 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:16.177624 11905 consensus_queue.cc:260] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:16.228947  7272 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:19:16.230166  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:16.222536  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:16.240882  6577 raft_consensus.cc:2410] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:19:16.207127 11941 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:16.246407  6576 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:16.248893  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:16.249536  7272 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:19:16.251330  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:16.251442  6550 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:16.262409  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:16.263135  6581 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:16.251955  6505 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:16.276988 11840 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:16.277843  7273 raft_consensus.cc:1240] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:16.278162  7272 raft_consensus.cc:1240] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:19:16.261864 11894 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.279024 11894 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:16.283654 11771 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:16.285567  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:16.286176  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:16.287468  6623 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:16.292943  6548 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:16.230041 11945 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:16.237854  6583 raft_consensus.cc:3055] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:16.300956  6583 raft_consensus.cc:740] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:16.064016 11951 consensus_queue.cc:1048] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.084s
I20260709 12:19:16.238122  6580 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:16.297940  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:16.281280 11813 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:16.310530  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:16.306597  6579 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
I20260709 12:19:16.312379  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:16.312604  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:16.313410  6576 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:16.313781  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:16.316046  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:16.230198 11462 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 12:19:16.307737  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:16.325440  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:16.327687  6581 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:19:16.331713  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:16.310909  6583 consensus_queue.cc:260] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:16.334381  6581 raft_consensus.cc:2410] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:19:16.300447 11977 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:16.214438 11966 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:16.337702  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:16.341266  6583 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:16.306100 11771 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:16.334283 11959 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.310614 11813 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:16.353518  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 158649us
I20260709 12:19:16.345227  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:16.347683  6584 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:16.344936  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:16.354985  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:16.356788  6548 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:16.354113  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:16.356276  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:16.359018  6654 raft_consensus.cc:2393] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 14: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:16.359323  6501 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 13.
I20260709 12:19:16.360258  6473 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:16.361488  6473 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:16.363394  6580 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:16.367499  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:16.368235  6580 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:16.346922  6579 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:19:16.372568  6473 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:16.300005 11616 consensus_queue.cc:1059] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 12:19:16.300868 11773 consensus_queue.cc:1059] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 12:19:16.374693  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:16.379908  6580 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:16.385308  6473 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:16.376971  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:16.317363 11906 consensus_queue.cc:1059] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 12:19:16.345575 11977 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:16.352000 11840 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:16.312294 11945 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:16.357151 11813 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:16.386726  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:19:16.388479  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 293505us
I20260709 12:19:16.388716  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:16.392390  6473 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:16.404474  6503 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:16.404822  6473 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:16.375944 11809 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:16.341286 11987 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.386669  6581 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:19:16.396574  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:16.415493  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:16.416014  6649 raft_consensus.cc:2410] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:19:16.374290 11771 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:16.413741  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:16.374770 11965 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:16.378906 11897 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:16.416869  6583 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:16.420826  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.422756 11398 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:16.353477 11421 consensus_queue.cc:1059] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 12:19:16.392549 11735 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:16.423842  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:16.427898  6649 raft_consensus.cc:2393] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 14: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:16.427037  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:16.400215 11861 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:16.400370 11977 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:16.431726  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:16.434700  6474 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:16.357628 11959 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:16.375798 11327 consensus_queue.cc:1059] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:19:16.443637  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:16.478255  7272 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:19:16.420109 11809 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:16.480149  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:16.480648  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:16.484401  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 57578.3us
I20260709 12:19:16.484889  6649 raft_consensus.cc:2410] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 12:19:16.461207  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.484790  7268 raft_consensus.cc:2393] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 16: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
W20260709 12:19:16.481599  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 46990us
I20260709 12:19:16.490868  6503 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 17.
I20260709 12:19:16.493643  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:16.493748  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:16.424449 11771 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:16.495159  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:16.427784 11730 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:16.423429 11952 raft_consensus.cc:3055] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:16.425951 11778 raft_consensus.cc:3055] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:16.500538  6509 raft_consensus.cc:1275] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:19:16.480309 11966 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:16.511654 11773 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.512825  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:16.425704 11987 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:16.514269  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:16.515563  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:16.515671 11987 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:16.435038 11894 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:16.517133  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:16.516978 11813 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:16.517807 11959 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:16.504771  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:16.518499  6508 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:19:16.519462 11885 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:16.522172  6652 raft_consensus.cc:2393] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 16: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:16.423629 11965 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:16.522918 11965 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:16.504038 11984 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.261s	user 0.001s	sys 0.000s
I20260709 12:19:16.520365 11813 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:16.501194  6583 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
I20260709 12:19:16.517962  6507 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 17: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:16.528086  7238 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:16.507290  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:16.529832  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:16.531591  6583 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:19:16.513605  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:16.533501  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:16.541245  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:16.541750  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:16.484108 11636 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:19:16.545413  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.496907 11809 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:16.546430  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:16.557786  6581 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:16.542158  6583 raft_consensus.cc:2393] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 14: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:16.562979  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:16.563696  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:16.564085  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:16.499426 11730 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:16.553821  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:16.508821 11952 raft_consensus.cc:740] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:16.498854 11861 raft_consensus.cc:697] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:16.545873  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:16.568472  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:16.586500  6583 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:16.534931  6656 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:16.529261 11836 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:16.507870 11823 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:16.529765 11889 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.563887  6657 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:19:16.508726 11778 raft_consensus.cc:740] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:16.577507  6508 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:19:16.557607 11616 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:16.587997  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:16.557719 11966 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:16.546937 11835 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:16.618357  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:16.619257  6507 raft_consensus.cc:2393] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 14: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:16.512157 11773 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:16.562129 11987 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:16.622717 11966 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:16.696012 11966 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:16.696916 10999 raft_consensus.cc:3055] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:16.697472 10999 raft_consensus.cc:740] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:16.698611 10999 consensus_queue.cc:260] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:16.592121  6500 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
W20260709 12:19:16.619861  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 64300.3us
I20260709 12:19:16.702771  6622 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:16.564600 11636 raft_consensus.cc:697] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:16.704135 11636 consensus_queue.cc:237] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:16.707947  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:16.710263 11743 raft_consensus.cc:3055] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:19:16.710685 11743 raft_consensus.cc:740] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:16.711346  6656 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:19:16.711567 11743 consensus_queue.cc:260] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:16.726310 11743 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:16.708806  6584 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:16.584702 11730 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:16.728704  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:16.729527  6584 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 19: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:16.730813  6584 raft_consensus.cc:1240] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 14. Current term is 15. Ops: [12.11-14.12]
I20260709 12:19:16.624154 11885 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:16.714668 11894 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:16.733129 11894 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:16.736020 11894 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:16.738981  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:16.740639  6548 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:16.741158  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:16.742302 11883 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:16.742758  6501 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:16.745800  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:16.744649  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:16.746712  6500 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:16.753844  6548 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:16.754292  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:16.755219 11906 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:16.756963  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:16.611060 11840 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:16.759191 11840 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:16.760151 11840 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:16.623307 11616 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:16.767371  6651 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:19:16.610507 11889 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:16.775107  6652 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:19:16.780143  6581 pending_rounds.cc:85] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 6
I20260709 12:19:16.780573  6581 pending_rounds.cc:107] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 14.7
I20260709 12:19:16.780884  6581 raft_consensus.cc:2889] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:16.582671 11951 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:16.787952  6584 raft_consensus.cc:1240] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:16.790020  6583 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:16.803570  6584 raft_consensus.cc:1275] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 12:19:16.807582  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 402794us
I20260709 12:19:16.621850 11823 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:16.787063 11835 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:16.811903  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 109210us
I20260709 12:19:16.831391  6656 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:16.689085 11950 consensus_queue.cc:1048] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.137s
I20260709 12:19:16.811412 11945 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:16.822110 11462 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:16.823477 11987 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:16.903308  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 72134.3us
I20260709 12:19:16.812657  6473 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:16.915738  6475 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:16.727504 11966 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:16.948679  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:16.955297  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:16.900530 11827 consensus_meta.cc:287] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.702s	user 0.005s	sys 0.006s
I20260709 12:19:16.771889 11773 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:16.952961  6500 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
W20260709 12:19:16.978284  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 165711us
I20260709 12:19:16.895993 11478 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:16.979064  6473 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:16.910328 11398 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:16.979813  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 64166.9us
I20260709 12:19:16.995682  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:16.996057  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:19:16.958570 11913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.013s	sys 0.052s
I20260709 12:19:16.849237 11782 consensus_queue.cc:1048] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.109s
I20260709 12:19:16.838750 11835 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
W20260709 12:19:17.078621  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.888572 11889 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:17.006933  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:16.807778 11951 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:17.087738  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:17.091507  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:17.092410  6650 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:17.093466  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 100440us
I20260709 12:19:16.957722 11462 raft_consensus.cc:697] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 15 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:17.094238  6473 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:19:17.094990  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 101511us
I20260709 12:19:17.095794  6475 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:16.902288 11885 consensus_queue.cc:1048] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.086s
I20260709 12:19:17.100865  7270 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 13 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:16.835284 11778 consensus_queue.cc:260] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:17.006410 11913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.013s	sys 0.052s
W20260709 12:19:17.144867  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 49160.8us
W20260709 12:19:17.149185  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 52561us
W20260709 12:19:17.148813  6578 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.018s	sys 0.037s
I20260709 12:19:16.992141 12011 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:16.985857 11945 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:19:17.010187 11957 consensus_meta.cc:287] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.802s	user 0.003s	sys 0.007s
W20260709 12:19:17.277354 11993 log.cc:927] Time spent T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.670s	user 0.001s	sys 0.000s
I20260709 12:19:17.277777 11957 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:17.279460  6505 raft_consensus.cc:2393] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 14: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:16.849213 11861 consensus_queue.cc:237] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:16.991839 11848 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:17.231173  6475 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:17.004613 11327 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:17.089640 11739 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:17.006143 11398 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:17.270591 11767 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 12:19:17.274084 12011 raft_consensus.cc:697] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:17.274832 11462 consensus_queue.cc:237] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:17.276921 11835 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:16.999373 11478 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:17.288552  7238 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.760s	user 0.014s	sys 0.002s
I20260709 12:19:17.290921 11945 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:17.279013 11996 log.cc:927] Time spent T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.656s	user 0.000s	sys 0.001s
I20260709 12:19:17.205279 11636 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:17.274013 11812 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:19:17.294226  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 449162us
I20260709 12:19:17.349737  6622 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:19:17.294639  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 63536.1us
I20260709 12:19:17.237347 11958 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:17.229593 11897 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:19:17.358754 12007 log.cc:927] Time spent T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
W20260709 12:19:17.312417  7238 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.784s	user 0.015s	sys 0.002s
W20260709 12:19:17.359417  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 831347us
I20260709 12:19:17.353360 11636 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:17.360098 11966 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:17.361068 12002 log.cc:927] Time spent T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.323s	user 0.000s	sys 0.001s
I20260709 12:19:17.361266  6622 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:17.361490 11636 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:17.349156 11812 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:17.362598 12012 log.cc:927] Time spent T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:19:17.362882  6622 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:17.315829  6650 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 17.
I20260709 12:19:17.364167 12031 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:17.364550  6658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2299) took 1011 ms. Trace:
I20260709 12:19:17.364794 12031 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:17.365007  6658 rpcz_store.cc:276] 0709 12:19:16.352839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.352958 (+   119us) service_pool.cc:224] Handling call
0709 12:19:16.353294 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:16.387105 (+ 33811us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:16.387150 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:16.387721 (+   571us) log.cc:844] Serialized 26 byte log entry
0709 12:19:16.597528 (+209807us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:16.597595 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:16.597697 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:17.364322 (+766625us) raft_consensus.cc:1685] finished
0709 12:19:17.364362 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:17.364491 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":103169,"raft.queue_time_us":56660,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":9155,"thread_start_us":209343,"threads_started":1,"wal-append.queue_time_us":554821}
I20260709 12:19:17.367026 11773 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:17.270887 11743 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:19:17.342523  7240 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.861s	user 0.013s	sys 0.004s
I20260709 12:19:17.368710 11743 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:17.369249  7240 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.888s	user 0.013s	sys 0.004s
W20260709 12:19:17.369611  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 888629us
I20260709 12:19:17.362468 11812 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:17.355813  6475 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:17.359093 11945 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:17.372568 11885 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:19:17.372826 12001 log.cc:927] Time spent T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.268s	user 0.001s	sys 0.000s
I20260709 12:19:17.373096 11885 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:17.373173 11945 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:19:17.373616 11885 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:17.374122  6653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2292) took 1038 ms. Trace:
I20260709 12:19:17.374517  6653 rpcz_store.cc:276] 0709 12:19:16.335777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.335945 (+   168us) service_pool.cc:224] Handling call
0709 12:19:16.357240 (+ 21295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:16.379651 (+ 22411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:16.379684 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:16.380222 (+   538us) log.cc:844] Serialized 26 byte log entry
0709 12:19:16.588642 (+208420us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:16.588687 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:16.588765 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:17.373905 (+785140us) raft_consensus.cc:1685] finished
0709 12:19:17.373928 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:17.374060 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52124,"raft.queue_time_us":23192,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":10901,"thread_start_us":208091,"threads_started":1,"wal-append.queue_time_us":583493}
I20260709 12:19:17.376929 11951 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:17.377933  6501 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:19:17.372360 11964 consensus_queue.cc:1048] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
W20260709 12:19:17.350414  6576 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.008s	sys 0.014s
W20260709 12:19:17.380774  6576 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.009s	sys 0.014s
I20260709 12:19:17.340816 11327 raft_consensus.cc:697] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:19:17.357952 11999 log.cc:927] Time spent T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.368s	user 0.000s	sys 0.001s
I20260709 12:19:17.358210  7269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38562 (request call id 891) took 1174 ms. Trace:
I20260709 12:19:17.362442  7238 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:17.388901  7269 rpcz_store.cc:276] 0709 12:19:16.184023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.228509 (+ 44486us) service_pool.cc:224] Handling call
0709 12:19:16.228801 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:16.229124 (+   323us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:16.229152 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:16.229634 (+   482us) log.cc:844] Serialized 26 byte log entry
0709 12:19:16.365773 (+136139us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:16.365815 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:16.365889 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:17.357975 (+992086us) raft_consensus.cc:1685] finished
0709 12:19:17.358004 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:17.358149 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":656,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":12649,"thread_start_us":95078,"threads_started":1,"wal-append.queue_time_us":310123}
I20260709 12:19:17.391688  7238 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:17.285458 11616 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:17.382337  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:17.372148 11967 consensus_meta.cc:287] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 1.147s	user 0.008s	sys 0.000s
I20260709 12:19:17.367817 11773 consensus_queue.cc:1059] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:17.398399  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:17.410032  7273 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
W20260709 12:19:17.407797  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 313613us
W20260709 12:19:17.390100 11730 consensus_meta.cc:287] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.661s	user 0.005s	sys 0.005s
I20260709 12:19:17.309149 11739 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:17.411480  6473 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:17.367342 12031 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:17.353878 11421 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:17.381138 11848 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:17.318867 11767 consensus_queue.cc:1059] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.076s
I20260709 12:19:17.405580 11860 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:17.395383 11636 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:17.396811 11958 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:17.433133  6506 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.029s	sys 0.014s
I20260709 12:19:17.406814 11913 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:17.359032 11897 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:17.406894 11991 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:19:17.565673  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 154332us
I20260709 12:19:17.657191  6473 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:17.382216  6508 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
new_leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:58944
I20260709 12:19:17.658871  6508 raft_consensus.cc:606] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Received request to transfer leadership to TS 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:17.659407  6508 raft_consensus.cc:614] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:17.422789  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:17.420194 11951 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:19:17.662047  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 239337us
I20260709 12:19:17.577848 11860 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:17.672883  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:17.421169 11730 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:17.474366 11782 consensus_queue.cc:1059] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.187s
I20260709 12:19:17.703193  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2363) took 1422 ms. Trace:
W20260709 12:19:17.628436  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 206464us
I20260709 12:19:17.704633  6577 rpcz_store.cc:276] 0709 12:19:16.280945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.285133 (+  4188us) service_pool.cc:224] Handling call
0709 12:19:16.295185 (+ 10052us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:16.305006 (+  9821us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:16.305053 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:16.305738 (+   685us) log.cc:844] Serialized 52 byte log entry
0709 12:19:16.620907 (+315169us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:17.219337 (+598430us) log.cc:844] Serialized 14 byte log entry
0709 12:19:17.268427 (+ 49090us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:17.268551 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:17.702874 (+434323us) raft_consensus.cc:1685] finished
0709 12:19:17.702906 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:17.703106 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66092,"prepare.queue_time_us":491631,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":50093,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":11304,"spinlock_wait_cycles":20881280,"thread_start_us":677825,"threads_started":2,"wal-append.queue_time_us":508336}
I20260709 12:19:17.717859  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:17.627113 11957 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:17.668841 11860 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:17.679265 12000 log.cc:927] Time spent T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.688s	user 0.001s	sys 0.001s
I20260709 12:19:17.748814  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:17.664968 11951 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:17.670163 11845 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:17.665073 12015 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.764671  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:19:17.577639  6506 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.029s	sys 0.014s
I20260709 12:19:17.677942 11889 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:17.710991 11913 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:17.713231 11991 raft_consensus.cc:697] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 17 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
W20260709 12:19:17.740090 11965 consensus_meta.cc:287] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 1.217s	user 0.010s	sys 0.009s
I20260709 12:19:17.564761 11835 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:17.566557 11945 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.556440 11421 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:17.758684  6550 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:17.779218 11977 consensus_meta.cc:287] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 1.285s	user 0.019s	sys 0.007s
I20260709 12:19:17.577422 11636 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:17.568265 12006 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.220s
I20260709 12:19:17.656213 11958 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:17.825927  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2386) took 1214 ms. Trace:
I20260709 12:19:17.826179 11958 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:17.826748 11958 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.827351 11958 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:17.808240 11945 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:17.830436 11945 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:17.810024 11636 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:17.777882 11951 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:17.780608 11966 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:17.819480 11991 consensus_queue.cc:237] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:17.816654 12008 log.cc:927] Time spent T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.243s	user 0.001s	sys 0.000s
I20260709 12:19:17.817174  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2362) took 1536 ms. Trace:
I20260709 12:19:17.797818 11965 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:17.817852 12026 log.cc:927] Time spent T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.402s	user 0.001s	sys 0.000s
I20260709 12:19:17.795130 11835 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:19:17.818599 12022 log.cc:927] Time spent T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.240s	user 0.001s	sys 0.000s
W20260709 12:19:17.819376 12036 log.cc:927] Time spent T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:19:17.788532 11845 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:17.745460 11782 consensus_queue.cc:1048] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
W20260709 12:19:17.820996 11968 consensus_meta.cc:287] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 1.634s	user 0.018s	sys 0.002s
W20260709 12:19:17.815820 12032 log.cc:927] Time spent T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:19:17.849050 11398 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:17.866909 12017 log.cc:927] Time spent T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.482s	user 0.000s	sys 0.002s
I20260709 12:19:17.852756  6578 rpcz_store.cc:276] 0709 12:19:16.280603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.286058 (+  5455us) service_pool.cc:224] Handling call
0709 12:19:16.315972 (+ 29914us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:16.316958 (+   986us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:16.317001 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:16.331439 (+ 14438us) log.cc:844] Serialized 52 byte log entry
0709 12:19:16.577069 (+245630us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:17.229402 (+652333us) log.cc:844] Serialized 14 byte log entry
0709 12:19:17.229621 (+   219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:17.229703 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:17.816937 (+587234us) raft_consensus.cc:1685] finished
0709 12:19:17.816957 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:17.817116 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":429932,"prepare.run_cpu_time_us":11035,"prepare.run_wall_time_us":92415,"raft.queue_time_us":21322,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":52626,"spinlock_wait_cycles":11467008,"thread_start_us":807772,"threads_started":2,"wal-append.queue_time_us":502885}
I20260709 12:19:17.836788 12018 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.759750 12035 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:17.811868 11478 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:17.793568 11913 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:17.807057 12028 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:17.831833 11885 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:17.826401  6509 rpcz_store.cc:276] 0709 12:19:16.611894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.613096 (+  1202us) service_pool.cc:224] Handling call
0709 12:19:16.613545 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:16.615292 (+  1747us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:16.615343 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:16.615974 (+   631us) log.cc:844] Serialized 26 byte log entry
0709 12:19:16.905288 (+289314us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:16.905350 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:16.905447 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:17.825714 (+920267us) raft_consensus.cc:1685] finished
0709 12:19:17.825735 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:17.825866 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54764,"raft.queue_time_us":2472,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":71265,"thread_start_us":288927,"threads_started":1,"wal-append.queue_time_us":1062271}
I20260709 12:19:17.910626  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:17.800195 11889 raft_consensus.cc:697] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:17.557382 11327 consensus_queue.cc:237] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:17.867277 11835 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:17.565925 12011 consensus_queue.cc:237] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:17.789283 12015 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:17.899516  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:17.843995 11977 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:17.901401  6474 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.4:44857 timed out after 1.587s (SENT) [suppressed 9 similar messages]
I20260709 12:19:17.882731 11959 raft_consensus.cc:3055] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:17.884361 12029 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:17.910552 11864 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.911692  6654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2300) took 1523 ms. Trace:
I20260709 12:19:17.893440 11964 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:19:17.913293  6576 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.000s	sys 0.007s
W20260709 12:19:17.931739  6576 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.000s	sys 0.008s
W20260709 12:19:17.917341  6549 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.003s	sys 0.005s
I20260709 12:19:17.902876 11739 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:17.935848 11739 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:17.936609 11739 raft_consensus.cc:697] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:17.937642 11739 consensus_queue.cc:237] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:17.944634 11964 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:17.947430 11964 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:17.900986 11945 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:17.903039 12023 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:19:17.949604 12034 log.cc:927] Time spent T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:19:17.949295 12023 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:17.950807 11995 log.cc:927] Time spent T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 1.144s	user 0.000s	sys 0.009s
I20260709 12:19:17.950922  6506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2378) took 1458 ms. Trace:
I20260709 12:19:17.951408  6506 rpcz_store.cc:276] 0709 12:19:16.492965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.505289 (+ 12324us) service_pool.cc:224] Handling call
0709 12:19:16.505667 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:16.822026 (+316359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:16.822065 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:16.822570 (+   505us) log.cc:844] Serialized 26 byte log entry
0709 12:19:17.766758 (+944188us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:17.767287 (+   529us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:17.767402 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:17.950631 (+183229us) raft_consensus.cc:1685] finished
0709 12:19:17.950654 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:17.950845 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":51292,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":14826,"thread_start_us":1173529,"threads_started":2,"wal-append.queue_time_us":941274}
I20260709 12:19:17.953298 11966 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:17.953413 11945 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:17.955719 11945 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:17.902335 12037 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:17.956357 12037 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:17.956990 11778 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:17.957472 11885 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:17.909425 12028 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:17.958451 12037 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:17.887681 11968 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:17.959456 11771 consensus_meta.cc:287] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 1.418s	user 0.003s	sys 0.016s
I20260709 12:19:17.959662 12028 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:17.960250  6504 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:17.961098 11968 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:17.961997  6504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 2380) took 1455 ms. Trace:
I20260709 12:19:17.962589  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2303) took 1560 ms. Trace:
I20260709 12:19:17.963792 11977 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:17.964572  6649 raft_consensus.cc:2393] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 14: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:17.966075  6652 raft_consensus.cc:2410] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 12:19:17.966372 12015 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:17.967938 11845 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:17.968488 11845 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:17.968954 11845 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:17.923686 11959 raft_consensus.cc:740] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:17.906493 12035 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:17.973583 12035 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:17.974170 12043 consensus_queue.cc:1048] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:19:17.976389 11421 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:17.977324 11889 consensus_queue.cc:237] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:17.977944 11965 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:17.931337 12029 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:17.980491 12029 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:17.984509 12046 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.962997  6655 rpcz_store.cc:276] 0709 12:19:16.402266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.416855 (+ 14589us) service_pool.cc:224] Handling call
0709 12:19:16.417330 (+   475us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:16.520756 (+103426us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:19:16.520785 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:16.521352 (+   567us) log.cc:844] Serialized 52 byte log entry
0709 12:19:17.001711 (+480359us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:17.001776 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:17.962180 (+960404us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b540010de54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:17.962230 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:17.962279 (+    49us) raft_consensus.cc:1685] finished
0709 12:19:17.962294 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:17.962529 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3837,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":91,"spinlock_wait_cycles":236879744,"thread_start_us":479861,"threads_started":1,"wal-append.queue_time_us":837093}
I20260709 12:19:17.962335  6504 rpcz_store.cc:276] 0709 12:19:16.506941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.507076 (+   135us) service_pool.cc:224] Handling call
0709 12:19:17.961826 (+1454750us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54000f3a54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:17.961944 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3287910784}
W20260709 12:19:17.933307  6549 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.003s	sys 0.005s
I20260709 12:19:17.930513  6654 rpcz_store.cc:276] 0709 12:19:16.388546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.388818 (+   272us) service_pool.cc:224] Handling call
0709 12:19:16.406422 (+ 17604us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:16.433422 (+ 27000us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:16.433452 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:16.479116 (+ 45664us) log.cc:844] Serialized 26 byte log entry
0709 12:19:16.810749 (+331633us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:16.810796 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:16.810871 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:17.911433 (+1100562us) raft_consensus.cc:1685] finished
0709 12:19:17.911457 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:17.911622 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35183,"raft.queue_time_us":73061,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":12364,"thread_start_us":331242,"threads_started":1,"wal-append.queue_time_us":926366}
I20260709 12:19:17.948635 11967 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:17.900748 12018 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:17.985021 12046 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:17.992039  6502 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2565) took 1782 ms. Trace:
I20260709 12:19:18.001696  6502 rpcz_store.cc:276] 0709 12:19:16.209357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.212168 (+  2811us) service_pool.cc:224] Handling call
0709 12:19:16.212519 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:16.212828 (+   309us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:16.212855 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:16.213275 (+   420us) log.cc:844] Serialized 26 byte log entry
0709 12:19:16.376418 (+163143us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:16.376462 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:16.376555 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:17.950195 (+1573640us) spinlock_profiling.cc:243] Waited 8.64 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:17.990313 (+ 40118us) raft_consensus.cc:1685] finished
0709 12:19:17.990350 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:17.991958 (+  1608us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":166,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":132,"spinlock_wait_cycles":19878656,"thread_start_us":162722,"threads_started":1,"wal-append.queue_time_us":348620}
W20260709 12:19:17.999035 12010 log.cc:927] Time spent T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.770s	user 0.000s	sys 0.001s
I20260709 12:19:17.953868 11966 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:17.935154 11864 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:17.965945 10999 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:17.928403 11767 consensus_queue.cc:1243] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer 20a202c76244485fbc8f1610473db8b9 log is divergent from this leader: its last log entry 13.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 12:19:18.000949 11812 consensus_meta.cc:287] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.630s	user 0.017s	sys 0.001s
I20260709 12:19:17.993314 11897 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:17.988797  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 596769us
I20260709 12:19:17.959867 11771 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:17.952131 12023 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:17.989143 11735 consensus_meta.cc:287] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 1.549s	user 0.014s	sys 0.005s
I20260709 12:19:18.004179  6502 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 12:19:18.006973  6582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2355) took 1810 ms. Trace:
I20260709 12:19:17.908154  6622 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:19:18.021992  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.013710  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 2262) took 1476 ms. Trace:
I20260709 12:19:18.028534  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:18.034657  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 289463us
W20260709 12:19:17.996171 11809 consensus_meta.cc:287] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 1.419s	user 0.012s	sys 0.008s
I20260709 12:19:17.933946  6576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2368) took 1651 ms. Trace:
W20260709 12:19:18.012684 11840 consensus_meta.cc:287] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 1.251s	user 0.002s	sys 0.010s
I20260709 12:19:18.045397  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:18.048497  6550 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:19:18.037501  6580 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.769s	user 0.002s	sys 0.014s
I20260709 12:19:18.049444  6502 pending_rounds.cc:85] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 7
I20260709 12:19:18.052316  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:18.009521 11864 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:18.046343  6582 rpcz_store.cc:276] 0709 12:19:16.196890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.238407 (+ 41517us) service_pool.cc:224] Handling call
0709 12:19:16.239324 (+   917us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:16.239813 (+   489us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:16.414314 (+174501us) log.cc:844] Serialized 14 byte log entry
0709 12:19:16.823008 (+408694us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:16.838705 (+ 15697us) log.cc:844] Serialized 52 byte log entry
0709 12:19:16.838965 (+   260us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:16.839166 (+   201us) log.cc:844] Serialized 14 byte log entry
0709 12:19:16.854983 (+ 15817us) log.cc:844] Serialized 14 byte log entry
0709 12:19:16.855171 (+   188us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:16.855259 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:17.998129 (+1142870us) spinlock_profiling.cc:243] Waited 8.33 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:18.003632 (+  5503us) spinlock_profiling.cc:243] Waited 19.2 ms on lock 0x7b48000030d8. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287adb2aa1 00007f287ada9b37 00007f287ada99ca 00007f287aea126f 00007f287aea0f37 00007f287aeb555c 00007f287aeb5508 00007f287aeb5488 00007f287aeb5443 00007f287aeb46ec 00007f2879ddea96
0709 12:19:18.006727 (+  3095us) raft_consensus.cc:1685] finished
0709 12:19:18.006755 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:18.006908 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1559,"prepare.queue_time_us":1316410,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":91,"raft.queue_time_us":60327,"raft.run_cpu_time_us":3510,"raft.run_wall_time_us":1137724,"spinlock_wait_cycles":150695040,"thread_start_us":460777,"threads_started":2,"wal-append.queue_time_us":503321}
I20260709 12:19:18.047912  6581 rpcz_store.cc:276] 0709 12:19:16.537525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.767786 (+230261us) service_pool.cc:224] Handling call
0709 12:19:16.768269 (+   483us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:16.781926 (+ 13657us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:16.781972 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:16.782638 (+   666us) log.cc:844] Serialized 52 byte log entry
0709 12:19:16.921661 (+139023us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:16.921879 (+   218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:18.010914 (+1089035us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b54004303d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:18.010972 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:18.011022 (+    50us) raft_consensus.cc:1685] finished
0709 12:19:18.011036 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:18.013610 (+  2574us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1557,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"spinlock_wait_cycles":26576640,"thread_start_us":138145,"threads_started":1,"wal-append.queue_time_us":585041}
I20260709 12:19:18.006513 12018 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:18.056006  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:18.020498 11845 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:18.059377  6580 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.791s	user 0.003s	sys 0.014s
I20260709 12:19:18.060853  6502 pending_rounds.cc:107] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 14.8
W20260709 12:19:18.061620  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 153571us
I20260709 12:19:18.067265  6502 raft_consensus.cc:2889] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:18.067615  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:18.067816  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 743) took 1704 ms. Trace:
W20260709 12:19:18.063807  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 162429us
I20260709 12:19:18.062386 11743 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:18.041249 11823 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:18.011278 11636 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.053424  6576 rpcz_store.cc:276] 0709 12:19:16.282498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.354978 (+ 72480us) service_pool.cc:224] Handling call
0709 12:19:16.378356 (+ 23378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:16.378614 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:19:16.787775 (+409161us) log.cc:844] Serialized 14 byte log entry
0709 12:19:17.932268 (+1144493us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:17.932402 (+   134us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:17.932498 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:17.932599 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:17.933861 (+  1262us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":428,"prepare.queue_time_us":562120,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":12949,"raft.queue_time_us":512732,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":8440,"thread_start_us":1530822,"threads_started":3,"wal-append.queue_time_us":877852}
I20260709 12:19:18.051355 11735 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:18.069350  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:18.030463 11860 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.068598  6580 rpcz_store.cc:276] 0709 12:19:16.363639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.383959 (+ 20320us) service_pool.cc:224] Handling call
0709 12:19:16.384211 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:16.384449 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:19:16.893704 (+509255us) log.cc:844] Serialized 14 byte log entry
0709 12:19:18.067268 (+1173564us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:18.067348 (+    80us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:18.067420 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:18.067550 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:18.067736 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":127223,"prepare.queue_time_us":605170,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":60,"raft.queue_time_us":149668,"spinlock_wait_cycles":11537792,"thread_start_us":1604495,"threads_started":3,"wal-append.queue_time_us":455350}
I20260709 12:19:18.060573 11840 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:18.078660 11860 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:18.078863 11840 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:18.009352 11812 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.062587 12023 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:18.080617 11950 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:18.012251 11967 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:18.081128 11950 raft_consensus.cc:697] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:18.081322  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:18.081992  6505 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41304 (request call id 648) took 1719 ms (client timeout 2153 ms). Trace:
I20260709 12:19:18.082008 11950 consensus_queue.cc:237] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.082993  6500 raft_consensus.cc:2393] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 11: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:18.083639  6509 raft_consensus.cc:2393] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 13: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:18.082230 11945 raft_consensus.cc:2764] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:19:18.082753 11967 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:18.084811  6503 raft_consensus.cc:1240] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
W20260709 12:19:18.086656 11997 log.cc:927] Time spent T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 1.275s	user 0.010s	sys 0.000s
I20260709 12:19:18.089257  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:18.090087  6582 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 9.
I20260709 12:19:18.093144  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:18.093631  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:18.093988  6580 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 11.
I20260709 12:19:18.094519  6582 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
W20260709 12:19:18.082412  6505 rpcz_store.cc:269] 0709 12:19:16.362646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.363905 (+  1259us) service_pool.cc:224] Handling call
0709 12:19:18.081803 (+1717898us) spinlock_profiling.cc:243] Waited 876 ms on lock 0x7b5400387554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:18.081938 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2014022400}
I20260709 12:19:18.097568  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:18.098364  6582 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:18.099835  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:18.101135  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:18.102559  6580 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
I20260709 12:19:18.103506  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:18.104053  6583 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:19:18.105543  6583 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:18.101945  6577 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 14.
I20260709 12:19:18.107120  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:18.108125  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:18.117679  6582 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 17.
I20260709 12:19:18.119066  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:18.041706 11951 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.120818  6576 raft_consensus.cc:3055] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:18.129762  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:18.130430  6576 raft_consensus.cc:740] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:18.119928  6579 raft_consensus.cc:2468] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:19:18.130653  6583 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:19:18.130666  6575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2360) took 1902 ms. Trace:
I20260709 12:19:18.132853  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49792 (request call id 854) took 1137 ms. Trace:
I20260709 12:19:18.133235  6652 rpcz_store.cc:276] 0709 12:19:16.995572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.995736 (+   164us) service_pool.cc:224] Handling call
0709 12:19:18.132666 (+1136930us) spinlock_profiling.cc:243] Waited 969 ms on lock 0x7b54001b1054. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:18.132798 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2228104064}
W20260709 12:19:18.134812  7273 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38558 (request call id 770) took 1690 ms (client timeout 1927 ms). Trace:
W20260709 12:19:18.135259  7273 rpcz_store.cc:269] 0709 12:19:16.444575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.445581 (+  1006us) service_pool.cc:224] Handling call
0709 12:19:18.134658 (+1689077us) spinlock_profiling.cc:243] Waited 914 ms on lock 0x7b54006c2354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:18.134757 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2102552448}
I20260709 12:19:18.135605  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:18.136216  6475 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:18.131835  6575 rpcz_store.cc:276] 0709 12:19:16.228413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.230813 (+  2400us) service_pool.cc:224] Handling call
0709 12:19:16.266362 (+ 35549us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:16.272363 (+  6001us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:16.272396 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:16.272864 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 12:19:16.405308 (+132444us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:16.405345 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:16.405482 (+   137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:18.084113 (+1678631us) spinlock_profiling.cc:243] Waited 3.24 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:18.119293 (+ 35180us) raft_consensus.cc:1685] finished
0709 12:19:18.119329 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:18.130578 (+ 11249us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5501,"raft.queue_time_us":5647,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"spinlock_wait_cycles":7449472,"thread_start_us":132156,"threads_started":1,"wal-append.queue_time_us":336432}
I20260709 12:19:17.993335 11952 consensus_queue.cc:260] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:18.106664 11743 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:18.060921 11809 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.152130  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:18.168170  6507 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:18.168599  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:18.131259  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:18.164230  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:18.165620  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:18.169205  6578 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:18.119597 12048 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.175706  6622 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:18.119179 12049 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:18.119658 12055 raft_consensus.cc:2764] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:19:18.126248 11861 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:18.169366  6576 consensus_queue.cc:260] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:18.148686 12018 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.147228 12011 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:18.181751  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:18.127142 11987 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 12:19:18.187032  7238 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:18.208648  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:18.211933  7238 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:18.213038  6508 raft_consensus.cc:2393] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 16: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:18.189116  6576 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:18.211627  7273 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:19:18.190308  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:18.189090  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:18.162264 12045 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:18.192219  7269 raft_consensus.cc:1217] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 13 LEADER]: Deduplicated request from leader. Original: 10.10->[11.11-12.12]   Dedup: 11.11->[12.12-12.12]
I20260709 12:19:18.181188 12051 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.215847  7269 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 13 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 12. Current term is 13. Ops: [11.11-12.12]
I20260709 12:19:18.181444 11990 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.186690 11860 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:18.221067  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:18.220027  6549 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:18.188026 11952 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:18.218369  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:18.218559  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:18.195587  6575 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:19:18.171820 11743 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.211064 12018 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.297427  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:18.305751  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:18.209916 11835 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.241979  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:18.307580  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 87678.9us
I20260709 12:19:18.286731  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:18.286226  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:18.309088 11809 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:18.311923 11478 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:18.312852  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:18.223222 11739 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.313079  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:18.352761  6651 raft_consensus.cc:2368] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17.
I20260709 12:19:18.313670  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:18.353220  6654 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:18.185096 12048 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:18.354202  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:18.215438 11966 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.216879 11941 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:18.355232 12048 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:18.318172  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2374) took 1826 ms. Trace:
I20260709 12:19:18.213658 12055 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.222764 12045 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.355808  6503 rpcz_store.cc:276] 0709 12:19:16.491677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.495005 (+  3328us) service_pool.cc:224] Handling call
0709 12:19:18.084587 (+1589582us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:18.086220 (+  1633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:18.317981 (+231761us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b5400387550. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:18.318114 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3654867328}
I20260709 12:19:18.356205 11739 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.357538  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:18.358237  6503 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:18.358757 11739 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:18.328900  6507 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:18.217036 11845 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:19:18.331660  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 114996us
I20260709 12:19:18.360448  7238 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:18.361290  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:18.362494 11836 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:18.362910 11836 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:18.363138  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:18.195318 12011 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:18.363339 11836 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:18.363827  7273 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:19:18.364394 12048 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:18.273459 12058 log.cc:927] Time spent T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:19:18.364850 12048 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.297526 11778 raft_consensus.cc:3055] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:18.366159 11778 raft_consensus.cc:740] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:18.365290  6549 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:18.363304 11462 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:18.367987 11462 raft_consensus.cc:697] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 17 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:18.368675 11783 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.369278 11783 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.368940 11462 consensus_queue.cc:237] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:18.372716 11783 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:18.374323 12048 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:18.374786 11783 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:18.241724  6652 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:18.287360 11951 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:18.348856  6650 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:18.349421  6658 raft_consensus.cc:2410] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 12:19:18.305220  6506 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:18.330586 11767 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:18.385248 11767 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:18.385771 11767 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:18.223019 11840 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:18.377082 11783 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:18.353159  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:18.355436 11941 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:18.356030 12055 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:18.330107  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.363827 12011 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:18.388536 11840 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:18.194900 11861 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.295044  6576 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 12. (term mismatch)
I20260709 12:19:18.403858  6576 pending_rounds.cc:85] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 11
I20260709 12:19:18.404194  6576 pending_rounds.cc:107] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 12.12
W20260709 12:19:18.341225  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 149303us
I20260709 12:19:18.530148  6473 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:18.534163  6473 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:18.537236  6473 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:18.408824  6506 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:18.408150  6503 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:18.546301  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:18.547076  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:18.548300  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:18.548952  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:18.549247  6506 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:19:18.550369  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:18.550630  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:18.551218  6506 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:19:18.412207  6507 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
W20260709 12:19:18.398667  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:18.557327  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.404526  6576 raft_consensus.cc:2889] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:18.441334  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.426524  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:18.566982  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.378820  7238 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:18.448592  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 851) took 1454 ms. Trace:
W20260709 12:19:18.568791  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 190416us
I20260709 12:19:18.568831  6656 rpcz_store.cc:276] 0709 12:19:16.994376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.997545 (+  3169us) service_pool.cc:224] Handling call
0709 12:19:16.997951 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:17.363127 (+365176us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:19:17.656556 (+293429us) log.cc:844] Serialized 14 byte log entry
0709 12:19:17.860639 (+204083us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:17.860712 (+    73us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:17.860773 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:18.448266 (+587493us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b540010e354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:18.448354 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:18.448516 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":537790,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":129,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":2339,"spinlock_wait_cycles":839225216,"thread_start_us":496558,"threads_started":2,"wal-append.queue_time_us":258496}
I20260709 12:19:18.573642  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:18.575424  6575 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:19:18.575219  6506 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
new_leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:58944
I20260709 12:19:18.576385  6506 raft_consensus.cc:606] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Received request to transfer leadership to TS 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:19:18.586048 11398 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 12:19:18.587142 11739 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.588397 11739 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.592381 11739 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:18.593153 11739 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:18.594295 11739 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:18.487375 11743 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:18.597026 11462 raft_consensus.cc:2804] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:19:18.597626 11743 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:18.597901 11462 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:18.598832 11462 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:19:18.598847 11743 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:18.335788 11749 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.604206 11749 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.393240 11783 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:18.409897 11840 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:18.380070 11845 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:18.694319 11845 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:18.285147 11860 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:18.699312 11860 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:18.488373 11861 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:18.351250 11906 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.700992 11906 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.703387 11906 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:18.254186 11990 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.705042  6653 raft_consensus.cc:2393] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 14: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:18.706275 11990 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:18.707157 11739 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:18.389189 11941 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:18.396982 11951 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.712803 11957 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:18.713590 11957 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:18.713132 11951 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.714093 11957 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:18.487254 11992 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:18.715718 11992 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:18.716181 11992 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:18.717739  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 180808us
I20260709 12:19:18.490576 12018 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:18.452766 12030 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:18.493847 12033 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:18.721649 12033 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.191819 12049 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:18.226123 12051 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:18.494042 12059 log.cc:927] Time spent T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.239s	user 0.001s	sys 0.000s
I20260709 12:19:18.645460 12055 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:18.661442  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 746) took 1297 ms. Trace:
I20260709 12:19:18.668838 11991 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:19:18.669585  6649 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 2311) took 2157 ms (client timeout 2831 ms). Trace:
I20260709 12:19:18.679611  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:18.684955  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:18.686779 11823 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:18.411597 12048 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:18.753037 11743 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:18.688199  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:18.728950 11478 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:18.759982 11972 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.760640 11950 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:18.723502  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 1.30441e+06us
I20260709 12:19:18.735296 11864 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 12:19:18.737165 11835 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:18.721529 12030 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.743388 11749 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:18.760917 11972 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.761427 11950 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.764950 11905 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:18.769724 11958 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:18.770424  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:18.771593  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:18.753103 12048 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.746325 11823 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:18.745112  6649 rpcz_store.cc:269] 0709 12:19:16.511576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:16.545598 (+ 34022us) service_pool.cc:224] Handling call
0709 12:19:18.669082 (+2123484us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54002279d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:18.669384 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3169599872}
I20260709 12:19:18.804302 11897 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.805788 11897 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.809648 12023 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:18.810537 12023 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.811782 11782 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:18.812647 11782 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.813319 12023 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:18.815483 11860 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.816282 11860 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:18.819144 11860 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:18.822098  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:18.823163  6651 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:19:18.827142 12030 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:18.830646 11730 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:18.842602  7270 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:18.792454 11743 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.845700  7273 raft_consensus.cc:1240] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 15 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:18.845826 11743 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.847426  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:18.847836  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:18.802538 11823 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:18.850638  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:18.837986  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:18.848281  6656 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:18.854867  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:18.858805 11913 consensus_meta.cc:287] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.873s	user 0.002s	sys 0.022s
I20260709 12:19:18.859249 11913 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:18.857648  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:18.744115  6584 rpcz_store.cc:276] 0709 12:19:17.363648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:17.364604 (+   956us) service_pool.cc:224] Handling call
0709 12:19:17.364948 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:17.851757 (+486809us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:19:17.867189 (+ 15432us) log.cc:844] Serialized 14 byte log entry
0709 12:19:18.122221 (+255032us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:18.122267 (+    46us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:18.122311 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:18.660990 (+538679us) spinlock_profiling.cc:243] Waited 487 ms on lock 0x7b54001d10d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:18.661049 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:18.661260 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23237,"prepare.queue_time_us":16208,"prepare.run_cpu_time_us":265,"prepare.run_wall_time_us":265,"raft.queue_time_us":23443,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":26764,"spinlock_wait_cycles":1183506688,"thread_start_us":233700,"threads_started":1,"wal-append.queue_time_us":312812}
I20260709 12:19:18.869678  6550 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:18.871837 11749 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:19:18.872211 12064 log.cc:927] Time spent T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.430s	user 0.003s	sys 0.000s
I20260709 12:19:18.872720  6653 raft_consensus.cc:2393] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 16: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:18.873726  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:18.873868  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:18.874615  6651 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:19:18.875658  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:18.875878  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:18.876411  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:18.864599 11782 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:18.877017  6652 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
W20260709 12:19:18.866175  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 68194us
I20260709 12:19:18.878887  6550 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:18.874635  6653 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:19:18.881007  6651 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:18.855655 11951 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:18.886777  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:18.892310 12005 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.508s	user 0.019s	sys 0.052s
W20260709 12:19:18.892824 12005 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.540s	user 0.019s	sys 0.052s
I20260709 12:19:18.893779 12080 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.894739  6656 raft_consensus.cc:2393] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 17: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:18.894534 12080 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.899323 11771 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:18.899986  6622 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:18.914866  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:18.916517  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:18.917289  6577 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:18.918242 11889 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:18.918789  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:18.919775 12044 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:18.901381 12069 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:18.920394 12044 raft_consensus.cc:697] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:18.920718 12069 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:18.902086  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:18.902729 12083 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:18.903070 11959 consensus_queue.cc:260] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.903818  6658 raft_consensus.cc:2410] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:19:18.904747 11636 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:18.905200  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:18.905781 11478 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:18.906601 11913 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:18.911927  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:18.917096  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:18.918872 11889 raft_consensus.cc:697] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:18.893668  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:18.900550 11812 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:18.916077  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:18.900794  6474 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:18.922806  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:18.878049 12033 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:18.928597 12083 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:18.954416 12083 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:18.928735 11894 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:18.929852 11950 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:18.921657 12044 consensus_queue.cc:237] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.931272  6577 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:18.872319 11749 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:18.937784  6507 raft_consensus.cc:1240] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:18.941686  6583 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:19:18.948765  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:18.949469  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:18.951438  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:18.951586  7272 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:18.951174 11889 consensus_queue.cc:237] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:18.940587 11478 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:18.953313  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 2390) took 1280 ms. Trace:
I20260709 12:19:18.953816  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:18.969611  6658 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:18.955827  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:18.958791 11894 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:18.959203 12049 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:18.956305  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:18.937091 11743 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:18.958084 11959 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:18.967092 11767 consensus_meta.cc:287] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.581s	user 0.016s	sys 0.010s
I20260709 12:19:18.968834  6508 rpcz_store.cc:276] 0709 12:19:17.672577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:17.764161 (+ 91584us) service_pool.cc:224] Handling call
0709 12:19:18.952854 (+1188693us) spinlock_profiling.cc:243] Waited 427 ms on lock 0x7b54001e69d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:18.953113 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":983098112}
I20260709 12:19:18.977584  6509 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:18.982692  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:19:18.983076  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 83999.6us
I20260709 12:19:18.997226  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:18.967967 11749 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:19.001932  7272 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
W20260709 12:19:19.005785  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 89898.1us
I20260709 12:19:18.982254 11767 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:19.014890  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:19.013074  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:19.012931  6579 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:19:19.023442  6550 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:19.012573 11812 raft_consensus.cc:2764] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:19:19.026197  6652 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:19:19.017757 11767 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:19.040578  6575 raft_consensus.cc:1240] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:19:19.017707 11992 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:19.025383 11950 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:19.047288  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:19:19.047669 11836 consensus_meta.cc:287] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.683s	user 0.007s	sys 0.009s
I20260709 12:19:19.048275  7270 raft_consensus.cc:2393] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 14: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:19.049021  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:19.049834 11992 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:19.054458 11840 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:19.030368 12080 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:19.056981 11950 raft_consensus.cc:697] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:19.057175  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:19.058248 11950 consensus_queue.cc:237] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:19.059695  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:19.060549  7270 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:19.064666  6507 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:19:19.044895  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:19.046208  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:19.019884 11897 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:19.049849  6583 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:19.072237  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:19.075981  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:19.078590  6579 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:19:19.079037  7272 raft_consensus.cc:2393] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 12: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:19.055898 11462 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:19.083384  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 45834.7us
W20260709 12:19:19.093008  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:19.098964  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:19.048203 11836 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:19.095086  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:19.070921 12069 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:19.080848 11743 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:19.080233 11966 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:19.115170  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:19.113257 11840 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:19.120555  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 136232us
W20260709 12:19:19.122190 12011 consensus_meta.cc:287] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.708s	user 0.012s	sys 0.003s
I20260709 12:19:19.123025 12045 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:19.122885 12011 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:19.122110  6550 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:19.098201 11972 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:19.080547 11783 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:19.130156  7273 raft_consensus.cc:2393] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 15: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
W20260709 12:19:19.132959  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:19.112653 11743 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:19:19.136752  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 46965us
I20260709 12:19:19.140820  6550 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:19.142241 11823 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:19.147382 12089 log.cc:927] Time spent T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 12:19:19.139056 11952 consensus_meta.cc:287] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.832s	user 0.006s	sys 0.009s
I20260709 12:19:19.185966 11972 consensus_queue.cc:1059] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:19.187552 11989 consensus_queue.cc:1059] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:19:19.188053 11941 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:19.188604 11989 consensus_queue.cc:1243] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:19:19.189437 11773 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.189980 11773 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:19.190390 11941 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:19.191433  6506 raft_consensus.cc:1240] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:19:19.150344 12011 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:19.153916  6577 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:19.193421  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:19.194727  6577 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:19.196866  6501 raft_consensus.cc:1240] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:19.198460  6654 raft_consensus.cc:2393] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 15: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:19:19.198761  6500 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 2393) took 1525 ms (client timeout 1625 ms). Trace:
W20260709 12:19:19.199280  6500 rpcz_store.cc:269] 0709 12:19:17.673945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:17.674794 (+   849us) service_pool.cc:224] Handling call
0709 12:19:19.198385 (+1523591us) spinlock_profiling.cc:243] Waited 362 ms on lock 0x7b540039ec54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:19.198604 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":832578816}
I20260709 12:19:19.159564  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:19.160681 12046 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:19.162122 12075 log.cc:927] Time spent T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.474s	user 0.000s	sys 0.002s
I20260709 12:19:19.167096  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:19.180259  6623 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:19.168135  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:19:19.156163 12052 log.cc:927] Time spent T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.933s	user 0.003s	sys 0.000s
I20260709 12:19:19.169663  6548 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:19.176047 11783 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:19.181875  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:19.211227  6576 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:19:19.183048 11823 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:19.184968 11478 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:19.157436 12048 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:19.154985 11462 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:19.203920  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 46400.2us
I20260709 12:19:19.138283 11836 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:19.205458 11778 consensus_queue.cc:260] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:19.208796  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:19.209731  6578 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:19:19.252347  7272 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:19.264400 11957 consensus_meta.cc:287] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.550s	user 0.004s	sys 0.010s
I20260709 12:19:19.264982 11957 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:19:19.267867 11957 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:19.240451  6584 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
W20260709 12:19:19.257735  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:19.181061 11743 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:19.251778  7274 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:19.240715 11941 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:19.269263  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:19.270223  6657 raft_consensus.cc:1240] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:19.270915 11897 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.274801  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:19.275470  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:19.313673  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:19.260951  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:19.315667  7265 raft_consensus.cc:2393] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 14: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:19.316672  7272 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:19:19.316016  6550 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:19.275405  6583 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:19.282294  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:19.284905 11845 consensus_meta.cc:287] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.590s	user 0.014s	sys 0.000s
I20260709 12:19:19.285357  6652 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:19.215107  6651 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:19:19.324764  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:19.325785  6651 raft_consensus.cc:2393] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 14: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:19.326965  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:19.327744  6651 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:19:19.253806  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:19.288743 11616 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.288067  6658 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
W20260709 12:19:19.256054  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 86683.3us
I20260709 12:19:19.329353 11616 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:19.330385  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:19.332238 11616 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:19.332585  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:19.246322 11966 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:19.334652 11966 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:19.260697 11778 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:19.338771  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:19.339516  6576 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:19:19.262202  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:19.340482  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:19.254248 11478 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:19.323194  6549 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:19.345574  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:19.309422  6577 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:19.253090  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:19.262178 12048 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:19.343951  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:19.351541  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:19.305644 11941 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:19.362282  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:19.319190 11783 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:19.337360 11966 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:19.342940 11894 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:19.367561  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 44452.5us
I20260709 12:19:19.259704 12083 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:19.343178 11897 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:19.371480  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:19.394454  6509 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
new_leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:58944
I20260709 12:19:19.395147  6509 raft_consensus.cc:606] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Received request to transfer leadership to TS 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:19.395539  6509 raft_consensus.cc:614] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:19.373237 12051 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:19.375429  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:19.376536 11767 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:19.376466 11827 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:19.346662 11845 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:19.381032  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:19.403149  6576 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:19.403445  6507 raft_consensus.cc:2393] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 14: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:19.383922  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:19.384492  6473 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:19:19.385061  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 287914us
I20260709 12:19:19.385516 11883 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:19.367756 11941 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:19.387049  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:19.389518  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:19.369537 11905 consensus_meta.cc:287] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.545s	user 0.008s	sys 0.004s
I20260709 12:19:19.382385 11783 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:19.406126  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:19.404052  6580 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
W20260709 12:19:19.410914  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 50243.9us
I20260709 12:19:19.415362  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:19.420480  6575 raft_consensus.cc:2393] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 17: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:19.421665  6658 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:19:19.421105  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:19.425329  6473 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:19.428390  7270 raft_consensus.cc:1240] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:19.430542 11327 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:19.431048 11327 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:19.431521 11327 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:19.431597  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:19.432747  6577 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:19:19.431056  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:19.433804  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:19.437976  7267 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:19.424083  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:19.410471 11883 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:19.444259  6583 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:19.444114  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:19.444832 11813 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:19.445025  6658 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
I20260709 12:19:19.445472  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:19.451573  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:19.456712 10999 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.457922 12087 raft_consensus.cc:2764] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 12:19:19.462373  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:19.465375  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:19.465655 11783 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:19.467281  6584 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:19.467483 11845 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:19.468369 11840 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:19.402473 11749 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:19.522802  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:19.524199  6577 raft_consensus.cc:2393] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 15: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:19.487885 11966 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:19.411691 11767 raft_consensus.cc:697] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:19.521086  7240 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:19.529731  6580 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:19.522931  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:19.531858  6509 raft_consensus.cc:3055] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:19:19.524628  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 192384us
I20260709 12:19:19.444151  6473 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:19.527302  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:19.532573  6509 raft_consensus.cc:740] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:19.504879 10999 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:19.500608 11883 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:19.550110  6651 raft_consensus.cc:2410] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:19:19.547392  6550 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:19.553185  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:19.553925  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 109887us
I20260709 12:19:19.560665  6548 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:19.563568  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:19.565264  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:19.567323  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:19.573218 11897 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:19.572856 11767 consensus_queue.cc:237] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:19.557803 12030 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:19.574335 12083 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:19.580546 11783 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:19.582700 11783 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:19.583163 11783 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:19.585584 11749 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:19.558055 11966 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:19.588673 11966 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:19.595484  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:19.606552  6653 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
I20260709 12:19:19.607669  6652 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:19:19.610913  6576 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:19:19.625260  6654 raft_consensus.cc:2393] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 15: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:19.580971  6473 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:19.614856  6550 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:19.633404  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:19.635881  6550 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:19.635901  6652 raft_consensus.cc:2393] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 15: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:19.627058  6658 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:19:19.633759  6583 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:19:19.638396  7268 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 17 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:19.639331  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 748) took 1550 ms. Trace:
I20260709 12:19:19.624585 11897 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:19.642611 11749 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:19.643083 11749 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:19.643514 11749 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:19.643846 11827 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:19.648464  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:19.658143  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2387) took 1485 ms. Trace:
I20260709 12:19:19.664597  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:19.681617  6579 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:19.639232  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:19.682454  6577 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 14: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:19.665620 10999 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:19.682611  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 46798.2us
I20260709 12:19:19.595029 11913 raft_consensus.cc:2804] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:19.694137 11913 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:19.694674 11913 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:19.694809 11749 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:19.680624  6578 rpcz_store.cc:276] 0709 12:19:18.172432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:18.172551 (+   119us) service_pool.cc:224] Handling call
0709 12:19:18.192706 (+ 20155us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:19.262522 (+1069816us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:19.657829 (+395307us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540032b754. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:19.658075 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2337696640}
I20260709 12:19:19.684011  6550 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:19.696177  6579 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:19:19.697052  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:19.697647  6578 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:19.698254 11897 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:19.698721 11897 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:19.699119 11897 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:19.699506 12051 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:19.699882 10999 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:19.700090 12051 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:19.685245  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:19.686887  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:19.639870  6581 rpcz_store.cc:276] 0709 12:19:18.088488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:18.088604 (+   116us) service_pool.cc:224] Handling call
0709 12:19:18.120396 (+ 31792us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:18.120665 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:19:18.148006 (+ 27341us) log.cc:844] Serialized 14 byte log entry
0709 12:19:19.203054 (+1055048us) spinlock_profiling.cc:243] Waited 2.21 ms on lock 0x7b1800360e80. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a2923a 00007f2875e93a2a
0709 12:19:19.638729 (+435675us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:19.638802 (+    73us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:19.638870 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:19.638970 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:19.639152 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12901,"prepare.queue_time_us":27129,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":2952,"raft.queue_time_us":22177,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":162,"spinlock_wait_cycles":5086080,"thread_start_us":386495,"threads_started":1}
I20260709 12:19:19.701753  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:19.702445  6579 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:19.702353  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 141765us
I20260709 12:19:19.705144  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:19.700517 10999 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:19.613049 12030 raft_consensus.cc:697] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:19.711150  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:19.711400  6577 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:19.711565  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:19.709584  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:19.682807  6509 consensus_queue.cc:260] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:19.707500  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 147004us
I20260709 12:19:19.683975 11827 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:19.604830 11743 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:19.728215  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:19.728621  6580 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
I20260709 12:19:19.729292  6581 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:19:19.728300  6473 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:19.729929  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:19.735664  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:19.734539  6509 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:19.733642  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:19.744736  6658 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:19:19.745774  6548 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:19.706461 11905 raft_consensus.cc:2804] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:19.752725  6652 raft_consensus.cc:2393] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 8: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:19.753225  6579 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:19.744103  6580 raft_consensus.cc:2393] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 16: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:19.755719  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:19.756932  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:19:19.757930  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 237032us
I20260709 12:19:19.758702  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:19.757897  6579 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:19:19.759995  6649 raft_consensus.cc:2393] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 8: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:19.721347 11989 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:19:19.768288  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 48491.2us
I20260709 12:19:19.733701 12005 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:19.780423  6577 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
I20260709 12:19:19.832188  6577 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
W20260709 12:19:19.788058 12084 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.777s	user 0.003s	sys 0.000s
I20260709 12:19:19.780541  6580 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:19.781266 10999 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:19.783203  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 53045.5us
I20260709 12:19:19.783246  6473 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:19.785305  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:19:19.842406  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 59305us
W20260709 12:19:19.842684  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 73810.5us
I20260709 12:19:19.781451 12087 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:19.794183  6550 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:19.795099  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2577) took 2410 ms. Trace:
I20260709 12:19:19.792161  6579 raft_consensus.cc:1240] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:19:19.798657  6509 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 12:19:19.768013 11894 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:19.801229  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:19.802680  6401 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57900 (request call id 189) took 46642 ms (client timeout 15000 ms). Trace:
I20260709 12:19:19.803093  6654 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:19.758351 11990 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.849730  6509 pending_rounds.cc:85] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 12
I20260709 12:19:19.758010 11905 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:19.780049 11957 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:19.855091  6473 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:19.791900  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:19.853425  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 59340.7us
I20260709 12:19:19.811866 11989 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:19.865271  6509 pending_rounds.cc:107] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 12:19:19.865649  6509 raft_consensus.cc:2889] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:19.866935 11989 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:19.867916  6508 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:19:19.872501 12009 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:19.873076 12009 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:19.873450 12009 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:19.875247 11990 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:19.851095 11894 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:19.876489 11957 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:19.878137 11990 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:19.879083  6650 raft_consensus.cc:2410] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 12:19:19.848526  6501 rpcz_store.cc:276] 0709 12:19:17.385098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:17.385215 (+   117us) service_pool.cc:224] Handling call
0709 12:19:17.385484 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:19.197351 (+1811867us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:19.794741 (+597390us) spinlock_profiling.cc:243] Waited 1.81 s on lock 0x7b54002c3dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:19.794991 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4164491520}
W20260709 12:19:19.879719  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 50888.3us
I20260709 12:19:19.880524  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:19.881261  7240 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:19.881623  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:19.882663  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:19.883353  6501 raft_consensus.cc:2393] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 14: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:19.883424 11840 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:19.860994 11905 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:19.810236  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 396625us
I20260709 12:19:19.884871  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:19.885910  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:19.886430  6507 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:19.846876  6581 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:19.887209  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:19.888314  6581 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:19:19.888587  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:19.889549  6581 raft_consensus.cc:1240] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:19.890410  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:19.891919  6548 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:19.890789 12092 log.cc:927] Time spent T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 12:19:19.857261  6474 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:19.866987  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:19:19.852737 11959 consensus_meta.cc:287] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.840s	user 0.012s	sys 0.000s
I20260709 12:19:19.827742  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:19.892117  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:19.910966  6649 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:19.911265  6655 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:19.893764  6581 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:19.894950 10999 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:19.888223  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:19.884140  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:19.901989  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 51260.2us
I20260709 12:19:19.878993 11957 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:19.919972  6509 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
new_leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:58944
I20260709 12:19:19.914898 11989 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:19.922343  6657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2939) took 1096 ms. Trace:
I20260709 12:19:19.922858  6657 rpcz_store.cc:276] 0709 12:19:18.820251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:18.820430 (+   179us) service_pool.cc:224] Handling call
0709 12:19:19.233564 (+413134us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:19.316021 (+ 82457us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:19.916587 (+600566us) spinlock_profiling.cc:243] Waited 390 ms on lock 0x7b540044a7d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:19.916766 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":896128384}
I20260709 12:19:19.924717  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:19.925187  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:19.925576  6657 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:19.925802  6656 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:19:19.927062  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:19.928047  6656 raft_consensus.cc:2393] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 16: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:19.922897 11989 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:19.930641 11989 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:19.930863  6581 raft_consensus.cc:1240] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 13. Current term is 14. Ops: []
W20260709 12:19:19.937036  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 88958.1us
I20260709 12:19:19.937577  6504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 653) took 1920 ms. Trace:
I20260709 12:19:19.938087  6504 rpcz_store.cc:276] 0709 12:19:18.017622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:18.605261 (+587639us) service_pool.cc:224] Handling call
0709 12:19:18.605990 (+   729us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:18.606789 (+   799us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:18.606831 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:18.607643 (+   812us) log.cc:844] Serialized 26 byte log entry
0709 12:19:18.745991 (+138348us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:18.746033 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:18.746563 (+   530us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:19.787191 (+1040628us) spinlock_profiling.cc:243] Waited 2.24 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:19.937043 (+149852us) raft_consensus.cc:1685] finished
0709 12:19:19.937075 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:19.937350 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":49105,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":31443,"spinlock_wait_cycles":5155712,"thread_start_us":137807,"threads_started":1,"wal-append.queue_time_us":281822}
W20260709 12:19:19.940989 11941 consensus_meta.cc:287] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.513s	user 0.009s	sys 0.000s
I20260709 12:19:19.941465 11941 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:19.851385  6401 rpcz_store.cc:269] 0709 12:18:33.160227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:33.160455 (+   228us) service_pool.cc:224] Handling call
0709 12:19:19.802361 (+46641906us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:11.070089 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:11.070923 (+   834us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:11.070982 (+    59us) write_op.cc:435] Acquired schema lock
0709 12:19:11.071001 (+    19us) tablet.cc:662] Decoding operations
0709 12:19:11.148878 (+ 77877us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:11.177875 (+ 28997us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:11.196429 (+ 18554us) tablet.cc:685] Acquiring locks for 14 operations
0709 12:19:11.632497 (+436068us) tablet.cc:701] Row locks acquired
0709 12:19:11.632565 (+    68us) write_op.cc:260] PREPARE: finished
0709 12:19:11.632813 (+   248us) write_op.cc:270] Start()
0709 12:19:11.633013 (+   200us) write_op.cc:276] Timestamp: P: 1783599551632726 usec, L: 0
0709 12:19:11.633061 (+    48us) op_driver.cc:348] REPLICATION: starting
0709 12:19:11.671355 (+ 38294us) log.cc:844] Serialized 5451 byte log entry
0709 12:19:12.106892 (+435537us) spinlock_profiling.cc:243] Waited 31.7 ms on lock 0x7b540008acd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e84e28 00007f28798bf581 00007f28798bed58 00007f28798c39f1 00007f28798c39a9 00007f28798c3939 00007f28798c3901 00007f28798c2b2d 0000000000388634 0000000000388479
0709 12:19:12.269463 (+162571us) spinlock_profiling.cc:243] Waited 9.54 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:12.789776 (+520313us) op_driver.cc:464] REPLICATION: finished
0709 12:19:13.244008 (+454232us) write_op.cc:301] APPLY: starting
0709 12:19:13.244110 (+   102us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:13.471229 (+227119us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:13.479126 (+  7897us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:14.342639 (+863513us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:14.363653 (+ 21014us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=28,key_file_lookups=28,delta_file_lookups=0,mrs_lookups=0
0709 12:19:14.363745 (+    92us) write_op.cc:312] APPLY: finished
0709 12:19:14.364370 (+   625us) log.cc:844] Serialized 129 byte log entry
0709 12:19:14.411600 (+ 47230us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:14.411753 (+   153us) write_op.cc:454] Released schema lock
0709 12:19:14.411823 (+    70us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:14.412313 (+   490us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:14.412342 (+    29us) write_op.cc:454] Released schema lock
0709 12:19:14.993817 (+581475us) spinlock_profiling.cc:243] Waited 4.46 ms on lock 0x7b1800317680. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c522 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
Metrics: {"mutex_wait_us":434434,"reported_tablets":53,"child_traces":[["op",{"apply.queue_time_us":443522,"apply.run_cpu_time_us":34976,"apply.run_wall_time_us":1168270,"cfile_cache_hit":58,"cfile_cache_hit_bytes":62535,"num_ops":14,"prepare.queue_time_us":302265,"prepare.run_cpu_time_us":43500,"prepare.run_wall_time_us":1069992,"raft.queue_time_us":226953,"raft.run_cpu_time_us":43953,"raft.run_wall_time_us":441500,"replication_time_us":1116033,"spinlock_wait_cycles":105075456,"thread_start_us":624465,"threads_started":4,"wal-append.queue_time_us":927716,"wal-append.run_cpu_time_us":19000,"wal-append.run_wall_time_us":2650120}]]}
I20260709 12:19:19.943893 11941 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:19.945407  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:19.852273 12087 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:19.953742  6654 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:19.954909  6649 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:19:19.822904 11478 consensus_queue.cc:1059] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.078s
I20260709 12:19:19.961026  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:19.761588 12030 consensus_queue.cc:237] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:19.860410 12005 raft_consensus.cc:697] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:19.972412  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:19.974355  6577 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:19:19.900461 11894 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:19.976239  6505 raft_consensus.cc:1240] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:19.977123  6578 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:19:19.965363  6576 raft_consensus.cc:2468] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 8.
I20260709 12:19:19.980122  6474 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:19.983145  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:19.990394  7274 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:19.974000  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 57341.2us
I20260709 12:19:19.997771  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:19.998740  7270 raft_consensus.cc:2468] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
W20260709 12:19:20.001814  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 109960us
I20260709 12:19:19.944691 11957 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:19.991549  6473 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:19.971547 12087 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:20.013224  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:20.024113  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:20.013502  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:20.018934  6550 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:20.040277  6623 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:20.042490  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 51053.4us
I20260709 12:19:20.054332  6576 raft_consensus.cc:1240] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 10. Current term is 11. Ops: []
W20260709 12:19:20.056842  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 76846.1us
I20260709 12:19:20.064368  7269 raft_consensus.cc:1275] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:19:20.036715  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:20.066987 10999 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:20.067489 10999 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:20.068122 10999 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:20.065809  6473 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:19:20.066382  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 47564.1us
I20260709 12:19:20.065351  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2408) took 1674 ms. Trace:
I20260709 12:19:20.027118 11813 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:20.080698  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 44114.1us
I20260709 12:19:20.087091  7238 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:20.083798 11717 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:19:20.088310  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 45022us
I20260709 12:19:20.056071 11989 raft_consensus.cc:2804] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:19:20.100594  6474 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:20.080090  6503 rpcz_store.cc:276] 0709 12:19:18.391277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:18.869747 (+478470us) service_pool.cc:224] Handling call
0709 12:19:18.870140 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:18.889507 (+ 19367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:18.889559 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:18.971723 (+ 82164us) log.cc:844] Serialized 26 byte log entry
0709 12:19:19.368743 (+397020us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:19.368814 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:19.368893 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:20.065133 (+696240us) raft_consensus.cc:1685] finished
0709 12:19:20.065156 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:20.065287 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25451,"raft.queue_time_us":18974,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":52070,"thread_start_us":337171,"threads_started":1,"wal-append.queue_time_us":604104}
W20260709 12:19:20.104432  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 62277.5us
I20260709 12:19:20.078397  6550 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:20.090952 11813 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:20.089426 11845 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:20.108619  6473 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:20.089524 12047 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:20.116822 11957 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:20.123428  6548 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:20.128103 11735 raft_consensus.cc:2764] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election decision vote started in defunct term 14: won
I20260709 12:19:20.037369 12048 consensus_queue.cc:1059] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
W20260709 12:19:20.137205  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 45032.7us
I20260709 12:19:20.106343 12051 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:20.107966 11959 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:20.148962 12031 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:20.149535 12031 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:20.151659 12031 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:20.152384 11959 raft_consensus.cc:697] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:20.153306 11959 consensus_queue.cc:237] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:20.120667 11840 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:19:20.160420 11478 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:20.161012 11478 raft_consensus.cc:697] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 13 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:20.161890 11478 consensus_queue.cc:237] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:20.165419 11773 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:20.166488 11827 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:20.172055 11813 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:20.175225 11813 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:20.176950 11813 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:20.173470  6473 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:20.178283 12031 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:20.178691 12031 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:20.179069 12031 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:20.094341 11941 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.086s
I20260709 12:19:20.186744 12009 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:20.187969  6504 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:20.102154 11989 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:20.188587 11989 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:20.189319 11735 consensus_queue.cc:1059] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:20.190237 11735 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:19:20.193537  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:20.192214 11735 consensus_queue.cc:1059] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 9.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:20.194803 11735 raft_consensus.cc:3055] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:20.195411 11735 raft_consensus.cc:740] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:20.196776 11735 consensus_queue.cc:260] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 9.11, Last appended by leader: 11, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:20.200562  6503 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:20.207600 10999 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:20.215845  6655 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:20.218467  6657 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:19:20.129426 11743 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:20.129528 12047 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:20.221588  7274 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:20.212268 11840 raft_consensus.cc:697] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 10 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:20.237676  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:20.262351  6658 raft_consensus.cc:2368] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15.
I20260709 12:19:20.244680  6653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49792 (request call id 855) took 1171 ms. Trace:
I20260709 12:19:20.244889 12005 consensus_queue.cc:237] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:20.246037 11952 raft_consensus.cc:3055] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:20.246816  6506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2402) took 2082 ms. Trace:
W20260709 12:19:20.246083  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 101778us
I20260709 12:19:20.192106 12031 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:20.248875  6500 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.046s	sys 0.074s
I20260709 12:19:20.199074 11845 raft_consensus.cc:697] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 16 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:20.217692 12051 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:20.212241 11735 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:20.260919  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2403) took 1054 ms. Trace:
I20260709 12:19:20.213156 11897 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:20.284580  6577 rpcz_store.cc:276] 0709 12:19:19.206128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.833766 (+627638us) service_pool.cc:224] Handling call
0709 12:19:19.974116 (+140350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:19.987182 (+ 13066us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:20.260716 (+273534us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b54000578d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:20.260857 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":273838336}
I20260709 12:19:20.262998  6653 rpcz_store.cc:276] 0709 12:19:19.079481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.079662 (+   181us) service_pool.cc:224] Handling call
0709 12:19:20.244469 (+1164807us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b54000b7554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:20.244594 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":253795584}
W20260709 12:19:20.274267  6500 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.046s	sys 0.074s
I20260709 12:19:20.271306  6506 rpcz_store.cc:276] 0709 12:19:18.164834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:18.682712 (+517878us) service_pool.cc:224] Handling call
0709 12:19:18.683282 (+   570us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:19.191915 (+508633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:20.246580 (+1054665us) spinlock_profiling.cc:243] Waited 508 ms on lock 0x7b540001c7d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:20.246758 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1167976832}
I20260709 12:19:20.255350 11827 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:20.264104 12047 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:20.297078  6549 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:20.298660 10999 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:20.296236  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:20.303850  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:20.304792  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:20.306001  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:20.271108 11952 raft_consensus.cc:740] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:20.309109  6658 raft_consensus.cc:1240] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:20.312469  6658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 858) took 1133 ms. Trace:
I20260709 12:19:20.313459  6651 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:20.312981  6658 rpcz_store.cc:276] 0709 12:19:19.178916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:20.290822 (+1111906us) service_pool.cc:224] Handling call
0709 12:19:20.308935 (+ 18113us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:20.312138 (+  3203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:20.312316 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:20.322052 11966 raft_consensus.cc:3055] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:20.322507 11966 raft_consensus.cc:740] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:20.322844 12031 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:20.327308  6655 raft_consensus.cc:1275] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:19:20.328941  6623 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:20.294163 12051 raft_consensus.cc:697] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:20.297756 12009 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:20.364804  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2340) took 1125 ms. Trace:
I20260709 12:19:20.366818  6653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 859) took 1122 ms. Trace:
I20260709 12:19:20.367762  6653 rpcz_store.cc:276] 0709 12:19:19.239149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:20.310076 (+1070927us) service_pool.cc:224] Handling call
0709 12:19:20.326089 (+ 16013us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:20.360848 (+ 34759us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:20.360875 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:20.360936 (+    61us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:20.360965 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:20.361030 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:20.361146 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:20.375811  6658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2346) took 1053 ms. Trace:
I20260709 12:19:20.376394  6658 rpcz_store.cc:276] 0709 12:19:19.322062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:20.327182 (+1005120us) service_pool.cc:224] Handling call
0709 12:19:20.369921 (+ 42739us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:20.375454 (+  5533us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:20.375486 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:20.375546 (+    60us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:20.375581 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:20.375645 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:20.375752 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:20.365394  6655 rpcz_store.cc:276] 0709 12:19:19.202643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:20.293151 (+1090508us) service_pool.cc:224] Handling call
0709 12:19:20.301904 (+  8753us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:20.327759 (+ 25855us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:20.328047 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:20.324519 11743 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:20.390983  6653 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 12:19:20.376705  6394 thread.cc:641] client-negotiator [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.023s	sys 0.140s
W20260709 12:19:20.407866  6394 thread.cc:608] client-negotiator [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.023s	sys 0.140s
W20260709 12:19:20.361088 11783 consensus_meta.cc:287] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.777s	user 0.011s	sys 0.002s
I20260709 12:19:20.333179 11941 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:20.374926 11913 consensus_meta.cc:287] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.680s	user 0.013s	sys 0.000s
I20260709 12:19:20.432687 11913 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:20.367410 11767 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.004s	sys 0.016s
W20260709 12:19:20.436599 11767 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.004s	sys 0.017s
I20260709 12:19:20.438926 11913 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:20.444592 12043 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:20.457933  6651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2354) took 1019 ms. Trace:
I20260709 12:19:20.367030 12031 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 12:19:20.372010 11773 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:20.465461  6651 rpcz_store.cc:276] 0709 12:19:19.435777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:20.368532 (+932755us) service_pool.cc:224] Handling call
0709 12:19:20.407029 (+ 38497us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:20.455154 (+ 48125us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:20.455184 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:20.455221 (+    37us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:20.455249 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:20.455308 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:20.455415 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:19:20.481283  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 110770us
I20260709 12:19:20.483814  6658 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:19:20.323513 11840 consensus_queue.cc:237] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 9.11, Last appended by leader: 11, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:20.494887  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:20.498184  6650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2349) took 1105 ms. Trace:
I20260709 12:19:20.323544 11966 consensus_queue.cc:260] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:20.498618  6650 rpcz_store.cc:276] 0709 12:19:19.384877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:20.366579 (+981702us) service_pool.cc:224] Handling call
0709 12:19:20.408261 (+ 41682us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:20.480854 (+ 72593us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:20.480897 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:20.490308 (+  9411us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:20.490370 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:20.490434 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:20.490540 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:20.499845 11966 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:20.501147  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 229043us
I20260709 12:19:20.502003  6474 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:20.504882  6475 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:20.500730 11883 consensus_meta.cc:287] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.926s	user 0.008s	sys 0.003s
I20260709 12:19:20.453830 11941 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:20.417693 11783 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:20.492786 11327 consensus_meta.cc:287] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 1.061s	user 0.015s	sys 0.000s
I20260709 12:19:20.470570 11905 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:20.534662  6582 raft_consensus.cc:1240] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:20.522192 12031 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:20.499068 11989 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:20.521600 12035 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.009s	sys 0.008s
I20260709 12:19:20.523793 12009 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:20.478176 11845 consensus_queue.cc:237] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:20.550693  6580 raft_consensus.cc:1240] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:20.500640 11773 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:20.552942  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 48154.2us
I20260709 12:19:20.484551 11952 consensus_queue.cc:260] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:20.566609  6651 raft_consensus.cc:1240] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 10. Current term is 11. Ops: []
W20260709 12:19:20.547310 12035 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.792s	user 0.010s	sys 0.008s
W20260709 12:19:20.569003  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 46381.5us
I20260709 12:19:20.519448 11883 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:20.551657 11897 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:20.526448 11327 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:20.582785 11889 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:20.584950  6582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2397) took 1440 ms. Trace:
I20260709 12:19:20.518854 12051 consensus_queue.cc:237] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:20.588400  6582 rpcz_store.cc:276] 0709 12:19:19.144667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.791299 (+646632us) service_pool.cc:224] Handling call
0709 12:19:19.791574 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:20.537394 (+745820us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:20.584640 (+ 47246us) spinlock_profiling.cc:243] Waited 743 ms on lock 0x7b54001d0bd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:20.584881 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1708298112}
I20260709 12:19:20.576583 11952 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:20.618073  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 43858.4us
W20260709 12:19:20.596524 12006 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.785s	user 0.004s	sys 0.030s
I20260709 12:19:20.604705 11773 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:20.619267  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:20.622541 11894 consensus_meta.cc:287] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.572s	user 0.007s	sys 0.002s
W20260709 12:19:20.624454  6624 thread.cc:641] server-negotiator [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.026s	sys 0.043s
W20260709 12:19:20.623808 12087 consensus_meta.cc:287] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.611s	user 0.006s	sys 0.004s
W20260709 12:19:20.627077  6654 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.013s	sys 0.002s
W20260709 12:19:20.631174  6624 thread.cc:608] server-negotiator [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.026s	sys 0.043s
I20260709 12:19:20.638069  7268 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 11. Current term is 12. Ops: []
W20260709 12:19:20.630738 12006 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.004s	sys 0.030s
I20260709 12:19:20.639946  7240 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:20.639091  6654 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.013s	sys 0.002s
I20260709 12:19:20.647930  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:20.648628  6649 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.005s	sys 0.012s
W20260709 12:19:20.657341  6649 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.006s	sys 0.012s
W20260709 12:19:20.652056  6578 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.004s	sys 0.013s
W20260709 12:19:20.657859  6578 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.004s	sys 0.013s
W20260709 12:19:20.645815  6548 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.020s	sys 0.051s
W20260709 12:19:20.658282  6548 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.021s	sys 0.051s
W20260709 12:19:20.631136  6579 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.002s	sys 0.013s
W20260709 12:19:20.658859  6579 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.002s	sys 0.014s
W20260709 12:19:20.658967  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:20.664989  6583 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.015s	sys 0.004s
W20260709 12:19:20.660864  6584 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.016s	sys 0.058s
W20260709 12:19:20.665846  6583 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.015s	sys 0.004s
W20260709 12:19:20.665956  6584 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.016s	sys 0.059s
W20260709 12:19:20.653111  7238 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.001s	sys 0.017s
I20260709 12:19:20.666705  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 769) took 1196 ms. Trace:
W20260709 12:19:20.667131  7238 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.001s	sys 0.017s
W20260709 12:19:20.667764  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 580698us
I20260709 12:19:20.671121  6579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2406) took 1463 ms. Trace:
I20260709 12:19:20.671407  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2400) took 1525 ms. Trace:
W20260709 12:19:20.716508  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:20.727478  6473 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.011s	sys 0.012s
W20260709 12:19:20.730358  6473 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.011s	sys 0.012s
I20260709 12:19:20.667228  6583 rpcz_store.cc:276] 0709 12:19:19.469916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.974821 (+504905us) service_pool.cc:224] Handling call
0709 12:19:20.008974 (+ 34153us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:20.009230 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:19:20.061368 (+ 52138us) log.cc:844] Serialized 14 byte log entry
0709 12:19:20.666342 (+604974us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:20.666396 (+    54us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:20.666464 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:20.666535 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:20.666647 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7403,"prepare.queue_time_us":80928,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":61,"raft.queue_time_us":114253,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"thread_start_us":597116,"threads_started":1}
I20260709 12:19:20.728686  6657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2963) took 1031 ms. Trace:
I20260709 12:19:20.731932  6657 rpcz_store.cc:276] 0709 12:19:19.697535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:20.478510 (+780975us) service_pool.cc:224] Handling call
0709 12:19:20.493626 (+ 15116us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:20.577957 (+ 84331us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:20.578002 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:20.578059 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:20.578103 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:20.728321 (+150218us) spinlock_profiling.cc:243] Waited 84.1 ms on lock 0x7b54001509d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:20.728419 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:20.728605 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":193340800}
I20260709 12:19:20.734143  6503 raft_consensus.cc:1275] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 12:19:20.705647  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:20.735752  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:20.737586  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:20.738960  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:20.739634  6505 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:20.730652  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 540665us
I20260709 12:19:20.741638  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2404) took 1535 ms. Trace:
I20260709 12:19:20.672086  6584 rpcz_store.cc:276] 0709 12:19:19.145491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.825608 (+680117us) service_pool.cc:224] Handling call
0709 12:19:19.826089 (+   481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:19.844943 (+ 18854us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:19.914525 (+ 69582us) log.cc:844] Serialized 14 byte log entry
0709 12:19:20.670835 (+756310us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:20.670909 (+    74us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:20.670973 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:20.671188 (+   215us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:20.671335 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4100,"prepare.queue_time_us":75300,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":43,"raft.queue_time_us":7399,"thread_start_us":687275,"threads_started":1}
I20260709 12:19:20.671705  6579 rpcz_store.cc:276] 0709 12:19:19.207251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.849524 (+642273us) service_pool.cc:224] Handling call
0709 12:19:19.867295 (+ 17771us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:19.867646 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:19:20.001154 (+133508us) log.cc:844] Serialized 14 byte log entry
0709 12:19:20.670626 (+669472us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:20.670701 (+    75us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:20.670793 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:20.670892 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:20.671049 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":77179,"prepare.queue_time_us":133945,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":11382,"thread_start_us":608634,"threads_started":1}
I20260709 12:19:20.745362  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:20.742374  6580 rpcz_store.cc:276] 0709 12:19:19.206501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.843643 (+637142us) service_pool.cc:224] Handling call
0709 12:19:19.864305 (+ 20662us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:20.552368 (+688063us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:20.730660 (+178292us) spinlock_profiling.cc:243] Waited 686 ms on lock 0x7b540032a354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:20.741562 (+ 10902us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1577749888}
W20260709 12:19:20.751549  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 87329.9us
I20260709 12:19:20.753484  6475 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:20.744203  7238 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:20.705168 11959 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.011s	sys 0.011s
W20260709 12:19:20.756556  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 633094us
I20260709 12:19:20.759502  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:20.760012  7273 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.001s	sys 0.021s
I20260709 12:19:20.645576 11894 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:20.760571  7273 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.002s	sys 0.021s
I20260709 12:19:20.762049  6503 raft_consensus.cc:1240] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:20.763366  6504 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
W20260709 12:19:20.765407 11959 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.011s	sys 0.013s
I20260709 12:19:20.648983 12087 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:20.761046  6652 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.011s	sys 0.014s
W20260709 12:19:20.769977  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:20.674088 12083 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:20.651499 11889 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:20.787279  6652 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.011s	sys 0.014s
I20260709 12:19:20.747664 11749 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:20.794183  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 123893us
I20260709 12:19:20.794641  6506 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14. Ops: []
W20260709 12:19:20.767745 12030 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.000s	sys 0.024s
I20260709 12:19:20.804949  6507 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:19:20.805799  6509 raft_consensus.cc:606] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Received request to transfer leadership to TS 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:20.806227  6509 raft_consensus.cc:614] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:20.807771  6501 raft_consensus.cc:1240] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14. Ops: []
W20260709 12:19:20.817678  6547 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.007s	sys 0.020s
W20260709 12:19:20.804746 12030 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.000s	sys 0.025s
W20260709 12:19:20.818310  6547 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.008s	sys 0.020s
W20260709 12:19:20.818622  6502 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.006s	sys 0.010s
W20260709 12:19:20.818967  6502 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.006s	sys 0.010s
I20260709 12:19:20.754546 11717 consensus_queue.cc:1059] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 12:19:20.819528 11767 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:20.820174 11717 raft_consensus.cc:3055] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:20.820631 11717 raft_consensus.cc:740] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:20.821522 11717 consensus_queue.cc:260] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:20.822952 11989 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:20.824891 11883 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:20.826870  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2410) took 1175 ms. Trace:
I20260709 12:19:20.827262  6501 rpcz_store.cc:276] 0709 12:19:19.650982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.926325 (+275343us) service_pool.cc:224] Handling call
0709 12:19:19.926654 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:20.825910 (+899256us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:20.826614 (+   704us) spinlock_profiling.cc:243] Waited 880 ms on lock 0x7b54001e7dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:20.826784 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2023679872}
I20260709 12:19:20.807299  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2338) took 1644 ms. Trace:
I20260709 12:19:20.828814  6504 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:20.828974 11327 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:20.786381 11913 raft_consensus.cc:2804] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:19:20.818133 11848 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:20.794742 11783 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:20.833443 11889 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:20.750725 11462 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 12:19:20.786444 12035 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:20.786260 11959 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:20.831856  6652 rpcz_store.cc:276] 0709 12:19:19.162679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.987582 (+824903us) service_pool.cc:224] Handling call
0709 12:19:20.082993 (+ 95411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:20.168906 (+ 85913us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:19:20.169246 (+   340us) log.cc:844] Serialized 14 byte log entry
0709 12:19:20.797006 (+627760us) log.cc:844] Serialized 14 byte log entry
0709 12:19:20.805580 (+  8574us) log.cc:844] Serialized 14 byte log entry
0709 12:19:20.805735 (+   155us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:20.807049 (+  1314us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:20.807091 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:20.807149 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:20.807248 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":58698,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":119,"raft.queue_time_us":25680,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":22872,"thread_start_us":627091,"threads_started":1}
I20260709 12:19:20.843307 11894 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:20.844147  6548 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:20.755508 12006 consensus_queue.cc:1059] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
W20260709 12:19:20.836079 11478 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.012s	sys 0.052s
W20260709 12:19:20.839676  6550 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.008s	sys 0.011s
W20260709 12:19:20.855815  6550 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.009s	sys 0.012s
I20260709 12:19:20.808821 11941 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:20.858156  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 779813us
I20260709 12:19:20.820106 11767 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
W20260709 12:19:20.855451 11478 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.013s	sys 0.052s
W20260709 12:19:20.868978 12048 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.009s	sys 0.013s
W20260709 12:19:20.896027  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 89928.6us
I20260709 12:19:20.838444 12031 consensus_queue.cc:1059] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:19:20.905874  6500 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2409) took 1635 ms. Trace:
I20260709 12:19:20.907562  6548 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:20.915071  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:20.920173  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:58944 (request call id 10) took 1441 ms. Trace:
W20260709 12:19:20.895468 12048 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.010s	sys 0.014s
I20260709 12:19:20.920686  6509 rpcz_store.cc:276] 0709 12:19:19.466598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.903460 (+436862us) service_pool.cc:224] Handling call
0709 12:19:20.907654 (+1004194us) spinlock_profiling.cc:243] Waited 885 ms on lock 0x7b54001e7dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e83853 00007f287ae5160a 00007f28743d77d4 00007f28743d7762 00007f28743d7691 00007f28743d760c 00007f28743d68c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:20.907799 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2035449216}
I20260709 12:19:20.862442 12043 raft_consensus.cc:697] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:20.931628  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:20.855355 11959 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:20.907076  6500 rpcz_store.cc:276] 0709 12:19:19.270848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.650685 (+379837us) service_pool.cc:224] Handling call
0709 12:19:19.651040 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:19.651289 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:20.293237 (+641948us) log.cc:844] Serialized 14 byte log entry
0709 12:19:20.508824 (+215587us) spinlock_profiling.cc:243] Waited 2.23 ms on lock 0x7b1800454c60. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a2923a 00007f2875e93a2a
0709 12:19:20.904428 (+395604us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:20.904496 (+    68us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:20.904565 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:20.905655 (+  1090us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:20.905811 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":619365,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":7333,"raft.queue_time_us":1906,"spinlock_wait_cycles":5129088,"thread_start_us":1036788,"threads_started":2}
I20260709 12:19:20.890990 12112 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:20.946192  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:20.954861 12087 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:20.955801  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 48334us
I20260709 12:19:20.958499  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:20.959229  6657 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:20.961617  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:20.962381  6651 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:20.963076  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:20.963687  6653 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:20.983883  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:20.985447  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 232017us
W20260709 12:19:21.002377  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 70748.9us
W20260709 12:19:21.004911  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 48261.3us
I20260709 12:19:20.933655 12120 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:21.018980  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:21.025784  6622 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:21.029829 11767 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:20.989902 11462 consensus_queue.cc:1059] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:19:21.029765 12043 consensus_queue.cc:237] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:21.036089 11767 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:21.037012 12131 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:21.036885 11767 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:21.041780 12083 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:21.043946 11959 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:21.045138 12115 log.cc:927] Time spent T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:19:21.049283  6502 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 670) took 1384 ms. Trace:
I20260709 12:19:21.050209  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2336) took 1888 ms. Trace:
I20260709 12:19:21.049760  6502 rpcz_store.cc:276] 0709 12:19:19.664741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:20.023253 (+358512us) service_pool.cc:224] Handling call
0709 12:19:20.070010 (+ 46757us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:20.089595 (+ 19585us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:20.089626 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:20.127450 (+ 37824us) log.cc:844] Serialized 26 byte log entry
0709 12:19:20.819351 (+691901us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:20.819413 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:20.819511 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:21.049057 (+229546us) raft_consensus.cc:1685] finished
0709 12:19:21.049082 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:21.049219 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":288,"raft.queue_time_us":425,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":161,"thread_start_us":691467,"threads_started":1,"wal-append.queue_time_us":919702}
I20260709 12:19:21.054589  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:21.056113  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:21.056751  6650 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:21.058043  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 2412) took 1406 ms. Trace:
I20260709 12:19:21.058451  6508 rpcz_store.cc:276] 0709 12:19:19.651575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.893218 (+241643us) service_pool.cc:224] Handling call
0709 12:19:21.057849 (+1164631us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54002c47d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:21.057986 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2472292608}
I20260709 12:19:21.061093 11813 raft_consensus.cc:3055] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:21.062301  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:21.050688  6656 rpcz_store.cc:276] 0709 12:19:19.162122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.977794 (+815672us) service_pool.cc:224] Handling call
0709 12:19:20.256898 (+279104us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:20.261404 (+  4506us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:19:20.307765 (+ 46361us) log.cc:844] Serialized 14 byte log entry
0709 12:19:20.848123 (+540358us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:20.848175 (+    52us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:20.848217 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:20.848292 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:21.050054 (+201762us) spinlock_profiling.cc:243] Waited 246 ms on lock 0x7b54002a58d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:21.050153 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":49936,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":35,"raft.queue_time_us":1460,"raft.run_cpu_time_us":373,"raft.run_wall_time_us":20368,"spinlock_wait_cycles":565700352,"thread_start_us":486136,"threads_started":1}
W20260709 12:19:21.069285  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 310295us
I20260709 12:19:21.061594 11813 raft_consensus.cc:740] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:21.040048 11767 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:21.067292 12121 log.cc:927] Time spent T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 12:19:21.042032  6550 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:21.085767  6475 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:21.100396  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:19:21.100878  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 58942.5us
I20260709 12:19:21.101449  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2335) took 1939 ms. Trace:
I20260709 12:19:21.101707  6550 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:21.101864  6649 rpcz_store.cc:276] 0709 12:19:19.161795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.973828 (+812033us) service_pool.cc:224] Handling call
0709 12:19:19.974136 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:19.996204 (+ 22068us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:20.659732 (+663528us) log.cc:844] Serialized 14 byte log entry
0709 12:19:21.101074 (+441342us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:21.101135 (+    61us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:21.101178 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:21.101244 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:21.101390 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9843,"prepare.queue_time_us":139164,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"thread_start_us":906576,"threads_started":2}
W20260709 12:19:21.104625  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 349471us
I20260709 12:19:21.069969 11897 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.108978  6653 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:19:21.063930 12083 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:21.090437  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 770) took 1619 ms. Trace:
I20260709 12:19:21.086941  6657 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:19:21.098610  6505 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:21.116104  6578 rpcz_store.cc:276] 0709 12:19:19.470533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.989230 (+518697us) service_pool.cc:224] Handling call
0709 12:19:20.012598 (+ 23368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:20.012828 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:19:20.658285 (+645457us) log.cc:844] Serialized 14 byte log entry
0709 12:19:21.089979 (+431694us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:21.090048 (+    69us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:21.090119 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:21.090211 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:21.090363 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41016,"prepare.queue_time_us":116010,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":1892,"raft.queue_time_us":1865,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":5997,"thread_start_us":1031174,"threads_started":2}
I20260709 12:19:21.079556  6473 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:21.110203  6550 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:21.098786 11735 consensus_meta.cc:287] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.793s	user 0.008s	sys 0.005s
W20260709 12:19:21.073499 11966 consensus_meta.cc:287] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.573s	user 0.010s	sys 0.002s
I20260709 12:19:21.119388 12116 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:21.111007 12129 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:21.121811  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 273463us
W20260709 12:19:21.122738  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 283166us
I20260709 12:19:21.107600 11767 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:21.126148  6654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2337) took 1963 ms. Trace:
I20260709 12:19:21.127584 11941 consensus_queue.cc:1059] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:21.128263 11773 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:19:21.128423  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 48947.2us
I20260709 12:19:21.128731 12005 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:21.129550 11989 consensus_queue.cc:1059] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:21.131043 11735 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:19:21.131445  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:21.131284  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:21.132000  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 46319us
I20260709 12:19:21.133078  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:21.135505  6550 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:21.108139 12048 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:21.115851 12131 raft_consensus.cc:2764] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:19:21.159384  6651 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:19:21.168061  6475 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:21.120452 11845 raft_consensus.cc:3055] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:21.148277  6654 rpcz_store.cc:276] 0709 12:19:19.162411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:19.983215 (+820804us) service_pool.cc:224] Handling call
0709 12:19:20.033637 (+ 50422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:20.064471 (+ 30834us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:20.657969 (+593498us) log.cc:844] Serialized 14 byte log entry
0709 12:19:21.125662 (+467693us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:21.125743 (+    81us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:21.125813 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:21.125913 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:21.126085 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9808,"prepare.queue_time_us":84713,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":25,"raft.queue_time_us":11807,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"thread_start_us":906991,"threads_started":2}
W20260709 12:19:21.155282  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:21.155782 11913 raft_consensus.cc:697] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 8 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:21.166692  6505 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:21.172596  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:21.193205  6579 raft_consensus.cc:2393] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 16: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:21.174046 12051 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:21.176992  6583 raft_consensus.cc:1240] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:21.177847 12112 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:21.178854  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:21.172688  6578 raft_consensus.cc:1240] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:21.115162 11897 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:21.195210 11966 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:19:21.178913 12125 log.cc:927] Time spent T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:19:21.204913  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:21.157385 11767 raft_consensus.cc:697] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:21.205878  6475 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:21.213320  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 85300.3us
I20260709 12:19:21.195924 11840 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:21.213027  6577 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:21.206588  6581 raft_consensus.cc:2393] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 15: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:21.170928 11845 raft_consensus.cc:740] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:21.216925  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:21.224077  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:21.217723  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:21.225092  6583 raft_consensus.cc:2468] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 8.
I20260709 12:19:21.179381 12005 raft_consensus.cc:697] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 13 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:21.193948 12048 raft_consensus.cc:697] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:21.228173  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:21.192741 12006 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:19:21.217265  6622 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:21.122995 11813 consensus_queue.cc:260] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:21.230806  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:21.231392  7273 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38546 (request call id 618) took 1049 ms. Trace:
I20260709 12:19:21.231634  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:21.217257 12051 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:21.212363 11941 raft_consensus.cc:3055] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:21.216699 12112 raft_consensus.cc:697] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:21.234704  6583 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:19:21.250715  6578 raft_consensus.cc:2393] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 11: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:21.251012  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:21.251693  6583 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:21.234884 11964 consensus_queue.cc:1059] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:19:21.235405 12146 consensus_queue.cc:1059] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:21.234647  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:21.235677 12005 consensus_queue.cc:237] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:21.237385 12123 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:21.237722  6475 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:21.261642  6475 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:21.238476 12147 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:21.266459 11905 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:21.267056 12139 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:21.267683 12139 raft_consensus.cc:697] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 17 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:21.268812 12087 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:21.238911  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:21.270623  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:21.271266  6577 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:21.150355 12129 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:21.272807 12043 raft_consensus.cc:3055] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:21.273203 12043 raft_consensus.cc:740] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:21.273981 12043 consensus_queue.cc:260] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:21.274920 12043 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:21.275758 12129 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:21.278461 12139 consensus_queue.cc:237] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:19:21.194504 12141 consensus_queue.cc:1059] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 12:19:21.245342  6580 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:21.257810  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:21.272245  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:21.265612  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:21.280400  6577 raft_consensus.cc:2376] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16. Re-sending same reply.
I20260709 12:19:21.280834  6579 raft_consensus.cc:2393] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 11: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:21.266366  6475 leader_election.cc:400] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15.
I20260709 12:19:21.247267  7273 rpcz_store.cc:276] 0709 12:19:20.182046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:20.234665 (+ 52619us) service_pool.cc:224] Handling call
0709 12:19:20.235054 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:20.235448 (+   394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:20.235474 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:20.235948 (+   474us) log.cc:844] Serialized 26 byte log entry
0709 12:19:20.772799 (+536851us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:20.772849 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:20.772924 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:21.231170 (+458246us) raft_consensus.cc:1685] finished
0709 12:19:21.231194 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:21.231330 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10950,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"thread_start_us":524620,"threads_started":1,"wal-append.queue_time_us":777192}
I20260709 12:19:21.281286  6475 leader_election.cc:403] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
W20260709 12:19:21.278946  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:21.261260  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:21.284534  6650 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:21.285621  6583 raft_consensus.cc:2393] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 18: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:19:21.289011  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 121904us
I20260709 12:19:21.292196  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:21.292832 12141 raft_consensus.cc:3055] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:21.292996  6579 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:19:21.293303 12141 raft_consensus.cc:740] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:21.294257  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:21.296902  6579 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:21.299139 11767 consensus_queue.cc:237] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:21.304262 12151 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:21.305215 12151 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:21.247790 11941 raft_consensus.cc:740] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:21.294935  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:21.306545  6578 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:21.295428  6475 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:21.296378 12111 log.cc:927] Time spent T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.289s	user 0.000s	sys 0.004s
I20260709 12:19:21.292261  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:21.292474 12048 consensus_queue.cc:237] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:21.308365  6577 raft_consensus.cc:2376] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15. Re-sending same reply.
I20260709 12:19:21.261391 11966 consensus_queue.cc:1059] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 12:19:21.294384 12141 consensus_queue.cc:260] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:21.321413  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:21.324628  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:21.325233  6577 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:21.325030 11959 consensus_queue.cc:1048] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:21.321908 11848 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:21.326568 11848 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:21.292516 11478 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15.
I20260709 12:19:21.327020 11848 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:21.241824 11735 consensus_queue.cc:1059] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 12:19:21.327759 11735 raft_consensus.cc:3055] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:21.328122 11735 raft_consensus.cc:740] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:21.328974 11735 consensus_queue.cc:260] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:21.330237 11735 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:21.331092 12141 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:21.331450 11717 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:21.333730 12112 consensus_queue.cc:237] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:21.317440  6579 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:21.333401 11913 consensus_queue.cc:237] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:21.337568  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:21.338445  6583 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 14: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:19:21.338486  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 77330.1us
I20260709 12:19:21.317704  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:21.341841  6582 raft_consensus.cc:2393] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 12: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:21.344717 12150 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:21.345288 12150 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:21.345650 12150 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:21.322114 11897 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:21.344813  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:21.352355  6580 raft_consensus.cc:2393] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 14: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:19:21.306784 12130 log.cc:927] Time spent T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:19:21.365449  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:21.365974  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:21.366022  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:21.367218  6657 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
W20260709 12:19:21.372818  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:21.369154  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:21.380793  6579 raft_consensus.cc:2393] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 16: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:21.381043  6651 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 12:19:21.391860  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:21.350205 12051 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:21.350239 11966 raft_consensus.cc:3055] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:21.382254  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:21.381705  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:21.405679  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:21.411646  6654 raft_consensus.cc:2393] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 17: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:21.407984  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:21.414908  6577 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:19:21.417223  6473 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:21.421816  6578 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:21.369127 11478 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.369904 11813 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:21.460631  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:21.462073  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:21.462383  6580 raft_consensus.cc:2393] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 11: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:19:21.463130  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 54947.2us
I20260709 12:19:21.381278 12145 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:21.467917  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:21.464543  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 292899us
I20260709 12:19:21.465703 10999 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:21.470371 10999 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:21.470273  7238 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:21.375831 12131 consensus_queue.cc:1059] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.087s
I20260709 12:19:21.476491  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:21.477244  6578 raft_consensus.cc:2393] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 13: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:21.474005  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:21.475226  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 58129.6us
I20260709 12:19:21.476884  6550 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:21.488566  6473 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:21.410194 11966 raft_consensus.cc:740] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:21.474232 11478 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:21.481845  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:21.443181 12030 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.491704  6550 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:21.492621  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:21.497352  6582 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:21.456965 12123 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:21.490195  6473 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:21.409919 11773 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 12:19:21.498713  6583 raft_consensus.cc:1240] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:19:21.500690  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:21.500631  6640 tablet_service.cc:1558] Processing DeleteTablet for tablet 3a47d71c34b541b9b2589b4e80413fa2 with delete_type TABLET_DATA_TOMBSTONED (TS f2b06aebe97249a4877d798b8118ece4 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:60084
I20260709 12:19:21.501197 11478 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:21.502357  6653 raft_consensus.cc:1275] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:19:21.503337 12153 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:21.508857 11952 consensus_meta.cc:287] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.883s	user 0.008s	sys 0.004s
I20260709 12:19:21.529299 12153 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:21.510655 11848 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:21.382843 11845 consensus_queue.cc:260] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:21.523952 12116 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:21.531694 11848 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:21.531917 12116 raft_consensus.cc:697] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:21.532881 12116 consensus_queue.cc:237] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:21.534113 12142 log.cc:927] Time spent T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.279s	user 0.003s	sys 0.001s
I20260709 12:19:21.383836 11941 consensus_queue.cc:260] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:21.521739  6550 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:21.486411 12139 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:21.522184 12051 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:21.478755 12145 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:21.546404  6583 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:21.549052  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:21.527036  6402 catalog_manager.cc:5697] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 5 to 8, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 8 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:21.418849 11889 consensus_queue.cc:1059] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.079s
I20260709 12:19:21.552817  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:21.552984  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:21.553174  6650 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:19:21.528640  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:21.553582  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:21.553853  6655 raft_consensus.cc:2410] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 20 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:19:21.554369  6501 raft_consensus.cc:2410] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:19:21.519726  6580 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:21.554494  6650 raft_consensus.cc:1240] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:21.555800  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:21.557307  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:21.558308  6657 raft_consensus.cc:2393] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 11: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:21.559432  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:21.492525 11957 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:21.492373 11767 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:19:21.528163 12009 consensus_meta.cc:287] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.964s	user 0.006s	sys 0.005s
I20260709 12:19:21.536860 12153 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:21.552821  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:21.553135  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:21.564519  6580 raft_consensus.cc:2468] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:19:21.565395  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:21.565693  6656 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:21.569531  6653 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:21.568022  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:21.583787  6502 raft_consensus.cc:2393] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 16: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:21.585381  6504 raft_consensus.cc:2393] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 15: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:21.546875 11897 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:21.543624 12139 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:21.546327 12118 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:19:21.590574  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 100529us
I20260709 12:19:21.590830  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:21.592111  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 214291us
I20260709 12:19:21.585139 12009 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:21.700279  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:21.560886 12141 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.717824  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:21.588826  6402 catalog_manager.cc:5697] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 5 to 12, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 12 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:21.591703  6473 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:21.717015  6501 raft_consensus.cc:2393] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 17: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:21.717830  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:19:21.720626  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 128984us
I20260709 12:19:21.541293 12030 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:21.721565  6473 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:21.723400  7270 raft_consensus.cc:1240] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:21.589948 11941 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:21.718160  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:21.566458 11957 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:21.523761 12123 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:21.731528  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:21.701159 11478 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:21.550956 12124 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.165s
I20260709 12:19:21.589565 11897 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:21.590232 12129 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.738216  7266 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 16 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 12:19:21.595115 12139 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:21.550662 12051 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:21.736932  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:21.717809  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 247584us
I20260709 12:19:21.772588  6402 catalog_manager.cc:5697] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 7 to 11, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 11 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:19:21.747390  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:21.739518  6500 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:19:21.722681 11952 raft_consensus.cc:3055] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:21.805333  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:21.761142  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:21.806587  6501 raft_consensus.cc:2393] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 16: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:21.806432  6507 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:19:21.808046  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:21.809021  6651 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 17.
I20260709 12:19:21.769124  6582 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:21.771646  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:21.594298 11894 consensus_queue.cc:1048] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.150s
W20260709 12:19:21.803864  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:21.850544  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:21.797947  7238 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:21.731736  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:21.851794  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:21.852624  7272 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:19:21.853211  6549 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:21.831301  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:21.761466 11735 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:21.758358 11957 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:21.760932 12009 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.736860 12131 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.101s
I20260709 12:19:21.849306 12048 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:21.798170 11897 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:21.873317  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 75463.1us
I20260709 12:19:21.739102 12031 consensus_queue.cc:1059] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.109s
I20260709 12:19:21.873471  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:21.886852  6506 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:19:21.805902 11952 raft_consensus.cc:740] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:21.737758 12141 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:21.859493 12087 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:21.918087  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:21.836859 12129 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:21.731478 12140 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:19:21.885056 11717 consensus_meta.cc:287] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.553s	user 0.010s	sys 0.000s
I20260709 12:19:21.879374  6402 catalog_manager.cc:5697] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 6 to 8, leader changed from <none> to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 8 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:19:21.921877  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 200404us
I20260709 12:19:21.886116 11735 raft_consensus.cc:697] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:21.885334 11905 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:21.878005 12163 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:21.930593  6502 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 17: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:21.880280 11889 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:21.934854  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 81631.9us
I20260709 12:19:21.878072 11767 raft_consensus.cc:697] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:21.887210 12009 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:21.949644  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:21.916553 11478 raft_consensus.cc:697] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:21.930263 12120 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:21.918227 12087 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:21.880713 12048 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:21.964114  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:21.981213  6402 catalog_manager.cc:5697] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:21.957338 12177 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:21.960119 12035 raft_consensus.cc:3055] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:21.965345 12172 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:19:21.968847 12160 log.cc:927] Time spent T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.251s	user 0.001s	sys 0.000s
W20260709 12:19:21.969667 12170 log.cc:927] Time spent T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:19:21.953428 12140 raft_consensus.cc:697] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:21.975896  7268 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:21.949123 11905 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:21.976943  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:21.989730  6579 raft_consensus.cc:2376] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17. Re-sending same reply.
I20260709 12:19:22.043323  6402 catalog_manager.cc:5697] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 10 to 13, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 13 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
W20260709 12:19:21.992604  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:21.994223  6580 raft_consensus.cc:1240] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15. Ops: []
W20260709 12:19:21.957638 12047 consensus_meta.cc:287] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 1.631s	user 0.015s	sys 0.003s
I20260709 12:19:22.000255  6509 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:19:22.003961 12129 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:21.957499 11952 consensus_queue.cc:260] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:21.960901 12163 consensus_queue.cc:1059] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 12:19:22.005589 12009 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:22.006738 12051 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:22.007287  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:21.948534 11889 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:22.008747  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:22.012514 12177 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:22.012050 11957 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:22.019282 12141 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:22.020874  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:22.035140 12083 consensus_meta.cc:287] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.916s	user 0.007s	sys 0.008s
W20260709 12:19:22.036728 12135 log.cc:927] Time spent T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.788s	user 0.007s	sys 0.009s
I20260709 12:19:22.038494 12178 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:19:21.996793 12150 consensus_meta.cc:287] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.651s	user 0.011s	sys 0.000s
I20260709 12:19:22.053450  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:22.110620  7272 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:22.109421  6402 catalog_manager.cc:5697] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 7 to 8, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 8 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:19:22.063733  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:22.059442  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 212865us
I20260709 12:19:21.996611 12035 raft_consensus.cc:740] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:22.091334  7266 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:19:22.091075  6507 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 14: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:22.100095  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:22.056015 12030 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:22.057781  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 125800us
I20260709 12:19:22.119467  6506 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:22.122218  6402 catalog_manager.cc:5697] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 6 to 9, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 9 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:22.101038 12123 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:22.166947  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:22.170104  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:22.170863  6579 raft_consensus.cc:2410] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:19:22.112358 11952 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:22.173177  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:22.174271  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:22.055032 11735 consensus_queue.cc:237] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:22.178064  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:19:22.181566 12168 log.cc:927] Time spent T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
W20260709 12:19:22.181929 12175 log.cc:927] Time spent T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:19:22.182771 12180 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:22.183643  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:22.184324  6584 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 17.
I20260709 12:19:22.185173 12169 tablet_replica.cc:333] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:19:22.185439  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:22.103404 12083 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:22.183373 12180 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:22.124900 12145 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:22.058293 12047 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:22.033602 12172 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:22.206683  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:22.113947 12150 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:22.224558 11813 consensus_meta.cc:287] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.753s	user 0.005s	sys 0.006s
I20260709 12:19:22.225368 12165 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:19:22.227674 12186 log.cc:927] Time spent T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:19:22.118327 12178 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:22.230135 12178 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:22.230904  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:22.231221 11897 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:22.232606 12165 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:22.235746 12180 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:22.237851 11897 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:22.241482  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:22.244439  6581 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:22.248908  6654 raft_consensus.cc:2410] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:19:22.250419 12172 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:22.093674 11767 consensus_queue.cc:237] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:22.233111  6622 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:19:22.254421 12043 consensus_meta.cc:287] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.979s	user 0.011s	sys 0.002s
I20260709 12:19:22.261477  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:22.262077 12154 log.cc:927] Time spent T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.984s	user 0.004s	sys 0.000s
I20260709 12:19:22.262462 12035 consensus_queue.cc:260] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:22.264456 12087 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:22.271142  6584 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:22.263742 11889 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:22.267293 12183 log.cc:927] Time spent T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.083s	user 0.000s	sys 0.002s
I20260709 12:19:22.266857  6582 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:22.213346 12123 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:22.256202  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:22.119030 11478 consensus_queue.cc:237] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:22.294972  6507 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:19:22.314999 12123 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:22.275637  6503 raft_consensus.cc:1240] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:22.191587 12131 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 12:19:22.319341 12189 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:19:22.319368  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 63250.4us
I20260709 12:19:22.321964  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:22.316206  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:22.323117  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:22.324084  6650 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:22.325435  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:22.331192  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 72856.4us
I20260709 12:19:22.259318 11957 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:22.332692  6582 raft_consensus.cc:1240] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:22.291294  6548 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:22.336175  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:19:22.326421 12134 log.cc:927] Time spent T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 1.018s	user 0.011s	sys 0.022s
I20260709 12:19:22.372887  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:22.287811 12177 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:22.333843  6475 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:22.172523 12140 consensus_queue.cc:237] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:22.335523  6655 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:19:22.365329  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:19:22.409463 12138 log.cc:927] Time spent T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 1.152s	user 0.005s	sys 0.004s
I20260709 12:19:22.358592 12131 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 12:19:22.429777  6402 catalog_manager.cc:5697] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:19:22.431764 12141 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:22.436520 12150 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:22.438531 12047 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:22.420114 12190 log.cc:927] Time spent T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.084s	user 0.001s	sys 0.001s
W20260709 12:19:22.421027  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 129837us
I20260709 12:19:22.448894  6548 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:22.449069  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:22.450619  6504 raft_consensus.cc:1240] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 15. Current term is 16. Ops: []
W20260709 12:19:22.418217 12136 log.cc:927] Time spent T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 1.156s	user 0.002s	sys 0.015s
I20260709 12:19:22.390657  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:22.357285  6653 raft_consensus.cc:1275] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 17 index: 7. (index mismatch)
W20260709 12:19:22.335634  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 77652.6us
I20260709 12:19:22.273584 11941 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:22.357898  6622 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:19:22.427971  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 94315.4us
I20260709 12:19:22.431504  6402 catalog_manager.cc:5697] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 7 to 10, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 10 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:19:22.451409  6548 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:22.456178  6579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2435) took 1039 ms. Trace:
I20260709 12:19:22.373618 12035 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:22.379643 12154 log.cc:927] Time spent T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 12:19:22.465148  6475 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:22.454448 12191 consensus_queue.cc:1059] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:22.345553 11889 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:22.456595  6579 rpcz_store.cc:276] 0709 12:19:21.417156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:22.222000 (+804844us) service_pool.cc:224] Handling call
0709 12:19:22.222424 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:22.296600 (+ 74176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:22.296647 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:22.296707 (+    60us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:22.296753 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:22.455951 (+159198us) spinlock_profiling.cc:243] Waited 73.9 ms on lock 0x7b54003ab954. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:22.456017 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:22.456124 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":169957120}
W20260709 12:19:22.452803 12139 consensus_meta.cc:287] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.605s	user 0.010s	sys 0.001s
I20260709 12:19:22.460084  6402 catalog_manager.cc:5697] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 5 to 12, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 12 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:22.446743 11767 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:22.492465 12139 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:22.462656 11897 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:22.463099  6658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2361) took 2706 ms. Trace:
I20260709 12:19:22.494691  7274 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38546 (request call id 621) took 1580 ms. Trace:
I20260709 12:19:22.493875  6658 rpcz_store.cc:276] 0709 12:19:19.757202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:20.519799 (+762597us) service_pool.cc:224] Handling call
0709 12:19:20.528226 (+  8427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:20.555936 (+ 27710us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:20.555980 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:20.565695 (+  9715us) log.cc:844] Serialized 26 byte log entry
0709 12:19:21.078588 (+512893us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:21.078648 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:21.078754 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:22.410026 (+1331272us) spinlock_profiling.cc:243] Waited 8.08 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:22.462825 (+ 52799us) raft_consensus.cc:1685] finished
0709 12:19:22.462847 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:22.463043 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18589056,"thread_start_us":460738,"threads_started":1,"wal-append.queue_time_us":649567}
I20260709 12:19:22.495062  7274 rpcz_store.cc:276] 0709 12:19:20.914163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:20.914766 (+   603us) service_pool.cc:224] Handling call
0709 12:19:22.494472 (+1579706us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54005df054. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:22.494609 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2887025536}
I20260709 12:19:22.497400 12139 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:22.452692  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:22.461735 12083 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:22.471304 12150 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:22.498814  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:22.499228 12150 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:22.499424  6507 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:22.499684 12150 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:22.500355  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:22.501112  6507 raft_consensus.cc:2410] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
W20260709 12:19:22.472425 12133 log.cc:927] Time spent T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 1.222s	user 0.008s	sys 0.015s
I20260709 12:19:22.473637 11848 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:19:22.501832  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 165658us
I20260709 12:19:22.502226 11848 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:22.502851  7269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38562 (request call id 920) took 2020 ms. Trace:
I20260709 12:19:22.502722 11848 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:22.478330 12123 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:22.455400 12189 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:22.482399  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:22.482699 12048 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:22.483867 11773 raft_consensus.cc:3055] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:22.484894 12191 raft_consensus.cc:3055] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:22.485033 10999 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:22.486547  6506 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:19:22.452467 12172 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:22.463145  6548 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:22.507663  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:22.508039  6622 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:22.510566  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:22.457978  7238 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:22.503294  7269 rpcz_store.cc:276] 0709 12:19:20.482335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:20.482557 (+   222us) service_pool.cc:224] Handling call
0709 12:19:20.483009 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:20.483296 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:20.483337 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:20.513637 (+ 30300us) log.cc:844] Serialized 26 byte log entry
0709 12:19:20.958717 (+445080us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:20.959240 (+   523us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:20.959323 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:22.471763 (+1512440us) spinlock_profiling.cc:243] Waited 4.9 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:22.502524 (+ 30761us) raft_consensus.cc:1685] finished
0709 12:19:22.502560 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:22.502771 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11275136,"thread_start_us":426214,"threads_started":1,"wal-append.queue_time_us":711571}
I20260709 12:19:22.513394  6402 catalog_manager.cc:5697] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 8 to 10, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 10 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:19:22.524377  6402 catalog_manager.cc:5697] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 11 to 14, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 14 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:19:22.525756  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:22.526007 12146 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:19:22.527000  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 69105us
I20260709 12:19:22.528797  7269 raft_consensus.cc:1240] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:22.534695  6579 raft_consensus.cc:1240] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:22.519321  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:22.537379  6506 raft_consensus.cc:2393] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 11: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:22.538668  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:22.538740  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:22.528316  6402 catalog_manager.cc:5697] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 6 to 13. New cstate: current_term: 13 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
W20260709 12:19:22.545459  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 82397.5us
I20260709 12:19:22.546542  6548 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:22.547938  6548 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:22.548635  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:22.549407  6507 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 14: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:22.552178  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:22.517661 11773 raft_consensus.cc:740] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:22.553444 10999 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:22.559630 10999 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:22.560035 10999 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:22.553987  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:22.554085  6548 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:22.561307  6507 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:58944
I20260709 12:19:22.561599  6548 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:22.561947  6507 raft_consensus.cc:606] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 17 LEADER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:22.554744  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:22.563117  6509 raft_consensus.cc:2410] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:19:22.518676 12191 raft_consensus.cc:740] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
W20260709 12:19:22.555255  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 88908.3us
I20260709 12:19:22.565479  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:22.565980  6475 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:22.468734 11735 consensus_queue.cc:1059] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 12:19:22.544358 11889 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:22.552038 12172 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:19:22.567075  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 68705us
I20260709 12:19:22.564093  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:22.568118  6579 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:19:22.515659 12189 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:19:22.572921  6548 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:22.574193  6501 raft_consensus.cc:2393] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 11: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:22.576159 11848 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:22.568774  6402 catalog_manager.cc:5697] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 10 to 13, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 13 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:22.544937 11952 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:19:22.556965 12048 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:22.581712  6509 raft_consensus.cc:1240] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:22.584539  6501 raft_consensus.cc:1275] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:19:22.585937 12087 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:22.586402 12087 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:22.586768 12087 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:22.587001 11773 consensus_queue.cc:260] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:22.588507 11773 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:22.595693  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:22.596522 12172 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:22.597121  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:22.600525 12191 consensus_queue.cc:260] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:22.601820 12191 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:22.578008 11897 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:22.605614  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:22.606135  6658 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:22.567503 12083 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:22.607355  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:22.608861  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2424) took 1343 ms. Trace:
I20260709 12:19:22.609264  6583 rpcz_store.cc:276] 0709 12:19:21.265626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.524824 (+259198us) service_pool.cc:224] Handling call
0709 12:19:21.544466 (+ 19642us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:21.551403 (+  6937us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:22.608664 (+1057261us) spinlock_profiling.cc:243] Waited 19.3 ms on lock 0x7b540032a850. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:22.608802 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44394368}
I20260709 12:19:22.613116 12169 raft_consensus.cc:2243] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Raft consensus shutting down.
I20260709 12:19:22.606166  6657 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:22.619355 11845 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:22.559156 11767 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 12:19:22.572299 12123 consensus_queue.cc:1059] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 12:19:22.622944 12123 raft_consensus.cc:3055] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:22.623617 12123 raft_consensus.cc:740] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:19:22.628715  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 53562us
I20260709 12:19:22.629047  6506 raft_consensus.cc:1240] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 13. Current term is 14. Ops: []
W20260709 12:19:22.629922 12199 log.cc:927] Time spent T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.169s	user 0.000s	sys 0.002s
I20260709 12:19:22.572855 11478 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 12:19:22.631435 11478 consensus_queue.cc:1059] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:22.636003 12006 raft_consensus.cc:3055] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:22.638428  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:22.643316 12204 log.cc:927] Time spent T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 12:19:22.643653  6509 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 14. (term mismatch)
I20260709 12:19:22.644017  6509 pending_rounds.cc:85] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 13
I20260709 12:19:22.643443  6402 catalog_manager.cc:5697] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:19:22.644224  6509 pending_rounds.cc:107] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 15.14
W20260709 12:19:22.644995  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:22.648413 12148 log.cc:927] Time spent T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 1.400s	user 0.001s	sys 0.024s
I20260709 12:19:22.648546 12123 consensus_queue.cc:260] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:22.649788 12123 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:22.644831  6402 catalog_manager.cc:5697] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 1 to 10, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 10 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:22.647151  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:22.653357  6653 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:22.658491  6509 raft_consensus.cc:2889] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:22.661579  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:22.663848 10999 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:22.661859  6473 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:22.667446  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:22.668059  6658 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:22.665336  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:22.669747  6583 raft_consensus.cc:2410] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:19:22.671634  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:22.671742  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:22.672411  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 121929us
I20260709 12:19:22.672811  6652 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:19:22.673269  6622 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:22.673524  6657 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:22.636475 12006 raft_consensus.cc:740] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:22.677330  6653 raft_consensus.cc:1275] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:22.677866  6501 raft_consensus.cc:1240] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:22.678768  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:22.679445  6583 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:22.683677 12189 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:22.683763  6402 catalog_manager.cc:5697] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 14 to 16, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 16 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:19:22.686167 11767 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:22.686928 11767 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:22.686484  6402 catalog_manager.cc:5697] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 9 to 10, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 10 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:22.688598  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:22.689360  6584 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:19:22.689757  6579 raft_consensus.cc:1240] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:22.690680  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:22.690853  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:22.691382  6584 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:22.692540 12009 consensus_queue.cc:1059] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:22.689401 11767 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:22.693672 11478 raft_consensus.cc:3055] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:22.696913 11478 raft_consensus.cc:740] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:22.697978 11478 consensus_queue.cc:260] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:22.699404 11478 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:22.689307  6402 catalog_manager.cc:5697] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 5 to 8, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 8 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:22.702292  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:22.702788  6402 catalog_manager.cc:5697] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 6 to 12. New cstate: current_term: 12 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:22.705150  6474 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:22.706367  6550 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:22.706296  6402 catalog_manager.cc:5697] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 6 to 9, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 9 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:22.707620 12172 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:22.708093 12172 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:22.708914 12048 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:22.709659 12120 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:22.710069 12120 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:22.710162 12172 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:22.710522 12120 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:22.711387 12141 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:22.713279 11889 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:22.715194 12083 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:22.715761 12083 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:22.691388  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:22.717432  6582 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:22.717610  6577 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:19:22.718036  6584 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:19:22.718266  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:22.691511  6579 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:22.707929  6402 catalog_manager.cc:5697] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 6 to 10, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 10 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:19:22.720201  6622 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
W20260709 12:19:22.721360  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:22.739990  6584 raft_consensus.cc:1275] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 17 index: 7. (index mismatch)
I20260709 12:19:22.740998  6622 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:22.658775 11735 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:22.695613 12087 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:22.745129 10999 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:22.746376  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:22.715907 12150 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:22.682190  6473 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:22.693009 12140 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:22.675904 11952 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:22.738195 11848 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:22.715234 12169 raft_consensus.cc:2272] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 12:19:22.762317  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:22.745208  6402 catalog_manager.cc:5697] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 10 to 14, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 14 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:22.777670  6577 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:19:22.786196  6402 catalog_manager.cc:5697] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 6 to 7, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 7 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:22.746362 11767 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:22.800973  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 60104.9us
I20260709 12:19:22.802001  6622 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:22.808589  6652 raft_consensus.cc:1240] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:19:22.810186  6583 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:22.788110  6582 raft_consensus.cc:1275] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:19:22.788297  6500 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 2590) took 1239 ms. Trace:
I20260709 12:19:22.811219  6500 rpcz_store.cc:276] 0709 12:19:21.548674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.552492 (+  3818us) service_pool.cc:224] Handling call
0709 12:19:22.788150 (+1235658us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b54001e5fd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:22.788245 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":364231040}
W20260709 12:19:22.790014  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 107918us
I20260709 12:19:22.766826 12189 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:22.795356  6658 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
W20260709 12:19:22.795521  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:22.761525 12157 raft_consensus.cc:2764] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
I20260709 12:19:22.763487 11735 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:22.821746 12132 log.cc:927] Time spent T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 1.574s	user 0.001s	sys 0.030s
I20260709 12:19:22.823347 11735 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:22.823917 12177 consensus_queue.cc:1059] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:22.788468 11952 raft_consensus.cc:697] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:22.825634 11952 consensus_queue.cc:237] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:22.831099 12169 ts_tablet_manager.cc:1916] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:19:22.805336 11767 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:22.833271 11767 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:22.830999  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:22.765939 12140 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:22.818249  6622 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:22.840823  6582 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 11. Preceding OpId from leader: term: 10 index: 12. (index mismatch)
I20260709 12:19:22.806238 11957 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:22.808115 12167 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:22.856529 11889 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:22.878396 11905 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:22.814164 12006 consensus_queue.cc:260] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:22.897446  7271 raft_consensus.cc:1275] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 12:19:22.900683  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 160142us
W20260709 12:19:22.902657  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 84518.6us
I20260709 12:19:22.910140  6622 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:19:22.911628  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 100692us
W20260709 12:19:22.912868  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 95076.5us
I20260709 12:19:22.919267  6473 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:22.920610  7238 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:22.901947  6548 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.628s (SENT) [suppressed 30 similar messages]
I20260709 12:19:22.927073  6622 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:22.928869  6622 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:22.933231  6473 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:22.924618  7238 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:22.940376  6473 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:22.906080 12083 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:22.918078 12182 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:22.945878  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:22.945178  6473 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:22.952782  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:23.000094  7274 raft_consensus.cc:1275] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:19:23.019548  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:23.021169  7272 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:22.937601 11941 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:22.935055 12191 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:22.934280 12128 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:23.033857  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:23.000819 12150 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:22.946978 12006 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:23.041342 12120 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:22.957885  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:23.030539 11905 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:23.045276  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:23.026710 12182 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:23.045311  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2434) took 1753 ms. Trace:
I20260709 12:19:23.053272  6580 rpcz_store.cc:276] 0709 12:19:21.291689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.584577 (+292888us) service_pool.cc:224] Handling call
0709 12:19:21.594028 (+  9451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:22.044536 (+450508us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:23.045102 (+1000566us) spinlock_profiling.cc:243] Waited 284 ms on lock 0x7b5400420754. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:23.045262 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":652767488}
W20260709 12:19:23.046388  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 101612us
I20260709 12:19:23.045830  6502 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 2595) took 1456 ms. Trace:
I20260709 12:19:23.054548  6502 rpcz_store.cc:276] 0709 12:19:21.589149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.589290 (+   141us) service_pool.cc:224] Handling call
0709 12:19:23.045660 (+1456370us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b54001e64d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:23.045779 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":438413696}
I20260709 12:19:23.025071 12216 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:23.031133 12178 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:22.955595  6549 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:23.003352 12155 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 12:19:23.034538 12191 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:23.055760  6473 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:23.076243  7272 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
W20260709 12:19:23.078214  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 122684us
I20260709 12:19:23.078852 12083 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:19:23.079078 12211 log.cc:927] Time spent T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 12:19:23.079478 12083 raft_consensus.cc:697] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:23.080634 12083 consensus_queue.cc:237] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:23.081607  7271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38562 (request call id 925) took 1340 ms. Trace:
I20260709 12:19:23.081523 12112 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:23.082051  7271 rpcz_store.cc:276] 0709 12:19:21.740830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.740977 (+   147us) service_pool.cc:224] Handling call
0709 12:19:21.774435 (+ 33458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:22.899295 (+1124860us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:23.080952 (+181657us) spinlock_profiling.cc:243] Waited 2.63 ms on lock 0x7b54005db4a0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e00dc0 00007f2875e8ca02 00007f2875e88dd6 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:19:23.081367 (+   415us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54005db1d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:23.081547 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2575462400}
I20260709 12:19:23.083263 12164 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.031301 11957 consensus_queue.cc:1059] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
W20260709 12:19:23.044780 12197 log.cc:927] Time spent T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.258s	user 0.000s	sys 0.001s
I20260709 12:19:23.019699 12209 consensus_queue.cc:1059] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:19:22.948679 12177 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:19:23.075722  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 129911us
I20260709 12:19:22.946470 12147 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:23.076807 12140 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:23.086732 12209 raft_consensus.cc:3055] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:19:23.087191 12208 log.cc:927] Time spent T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 12:19:22.944690 12123 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:23.087109 12203 consensus_queue.cc:1059] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:23.085932 12191 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:23.114257 10999 consensus_queue.cc:1048] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:23.114450  6549 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:23.116784  6622 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:23.048221 12217 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 12:19:23.064131 12215 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:23.084852 12164 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:23.064021 12216 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:23.132603  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:23.050585 11905 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:23.113400 12182 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.176601 12182 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:23.178942 12182 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:23.182582 11905 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:23.077314 12178 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:23.190539 12178 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:23.190663  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:23.192030 12215 raft_consensus.cc:697] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:23.193383 12215 consensus_queue.cc:237] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:23.115808 12209 raft_consensus.cc:740] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:23.113584 12087 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:23.164160  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 47455.9us
W20260709 12:19:23.163261 12210 log.cc:927] Time spent T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 12:19:23.174278 12223 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:23.174860 11897 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:19:23.163074  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 48740.5us
I20260709 12:19:23.119701 12140 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:23.234191 12140 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:23.234650 12140 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:23.211030  6622 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:23.242053  6622 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:23.243798  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:23.244529 11952 raft_consensus.cc:2804] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:19:23.245478 12150 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:19:23.224560  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 168883us
I20260709 12:19:23.222889  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:23.248927  6651 raft_consensus.cc:2410] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 12 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:19:23.249053  6473 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:23.167619 12216 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:23.253193  6578 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:23.255993  6575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2428) took 1965 ms. Trace:
I20260709 12:19:23.254631  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:23.259171  6575 rpcz_store.cc:276] 0709 12:19:21.290262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.570106 (+279844us) service_pool.cc:224] Handling call
0709 12:19:21.757081 (+186975us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:21.773386 (+ 16305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:21.773770 (+   384us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:21.858826 (+ 85056us) log.cc:844] Serialized 52 byte log entry
0709 12:19:22.196688 (+337862us) spinlock_profiling.cc:243] Waited 1.61 ms on lock 0x7b180031b760. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:19:22.454447 (+257759us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:22.454484 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:22.454549 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:23.225443 (+770894us) raft_consensus.cc:1685] finished
0709 12:19:23.225473 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:23.255904 (+ 30431us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1008,"raft.queue_time_us":123223,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":14664,"spinlock_wait_cycles":3697920,"thread_start_us":257555,"threads_started":1,"wal-append.queue_time_us":782601}
I20260709 12:19:23.268697 12209 consensus_queue.cc:260] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:23.270169 12209 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:19:23.276862 12223 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:23.280532 12128 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:23.288697  6501 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 12:19:23.291483 12195 log.cc:927] Time spent T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.847s	user 0.003s	sys 0.000s
I20260709 12:19:23.291803  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:23.292080  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:23.292968  7267 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38558 (request call id 789) took 1589 ms (client timeout 1803 ms). Trace:
W20260709 12:19:23.293388  7267 rpcz_store.cc:269] 0709 12:19:21.703358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.703525 (+   167us) service_pool.cc:224] Handling call
0709 12:19:23.291918 (+1588393us) spinlock_profiling.cc:243] Waited 953 us on lock 0x7b5800125198. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:19:23.292907 (+   989us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2192640}
I20260709 12:19:23.253293 11897 raft_consensus.cc:697] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 16 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:23.296407 11897 consensus_queue.cc:237] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:23.299834  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:23.300763  6501 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:23.253118 12229 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:23.253504 12087 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
W20260709 12:19:23.304796  6581 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 2306) took 2101 ms (client timeout 2156 ms). Trace:
W20260709 12:19:23.312800  6581 rpcz_store.cc:269] 0709 12:19:21.203443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.225042 (+ 21599us) service_pool.cc:224] Handling call
0709 12:19:23.304605 (+2079563us) spinlock_profiling.cc:243] Waited 972 ms on lock 0x7b540032b254. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:23.304711 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2236001152}
I20260709 12:19:23.315611  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:23.316071  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:23.308782 12219 log.cc:927] Time spent T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 12:19:23.309427 12120 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:23.276139 11478 consensus_meta.cc:287] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.576s	user 0.006s	sys 0.003s
I20260709 12:19:23.304137 12164 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:23.331727  6501 raft_consensus.cc:1240] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:23.335820  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:23.336735  6578 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:23.338472  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:23.338850  6548 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:23.339697  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:23.342067  6502 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:23.354089  6582 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:23.356038  6505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2441) took 1510 ms. Trace:
I20260709 12:19:23.357313  6578 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:23.358280  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:23.358350  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:23.359532  6578 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:19:23.359709  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:23.356527  6505 rpcz_store.cc:276] 0709 12:19:21.845566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.859978 (+ 14412us) service_pool.cc:224] Handling call
0709 12:19:21.860377 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:22.034130 (+173753us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:22.034159 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:22.040284 (+  6125us) log.cc:844] Serialized 26 byte log entry
0709 12:19:22.377807 (+337523us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:22.377863 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:22.603637 (+225774us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b54000f3a54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:22.603692 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:23.289858 (+686166us) spinlock_profiling.cc:243] Waited 1.28 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:23.355820 (+ 65962us) raft_consensus.cc:1685] finished
0709 12:19:23.355842 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:23.355978 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28573,"raft.queue_time_us":265,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":21298,"spinlock_wait_cycles":401179136,"thread_start_us":272707,"threads_started":1,"wal-append.queue_time_us":325747}
W20260709 12:19:23.364202  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:23.342047 11773 consensus_meta.cc:287] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.753s	user 0.010s	sys 0.001s
I20260709 12:19:23.364949  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:19:23.366247 12047 raft_consensus.cc:3055] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:23.367795 12047 raft_consensus.cc:740] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:23.368923 12047 consensus_queue.cc:260] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:23.370211 12047 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:23.370357  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:23.371840  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:19:23.372150  6576 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 2303) took 2195 ms (client timeout 2499 ms). Trace:
W20260709 12:19:23.372578  6576 rpcz_store.cc:269] 0709 12:19:21.176184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.223817 (+ 47633us) service_pool.cc:224] Handling call
0709 12:19:23.371883 (+2148066us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b54001d15d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:23.372040 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3642141568}
I20260709 12:19:23.376986  6581 raft_consensus.cc:1240] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 12:19:23.377435 12112 consensus_queue.cc:1059] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:23.378283  6584 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:23.378675 12120 raft_consensus.cc:3055] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:23.379143 12120 raft_consensus.cc:740] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:23.380124 12120 consensus_queue.cc:260] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:23.381142  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:23.381731 12120 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:23.383440  6504 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 14. Current term is 15. Ops: []
W20260709 12:19:23.388304  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:23.303805 12216 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:23.397082  6509 raft_consensus.cc:1240] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:23.399652 12164 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:23.388906  6579 raft_consensus.cc:1240] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14. Ops: []
W20260709 12:19:23.402748  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:23.390663  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2434) took 2659 ms. Trace:
I20260709 12:19:23.393528  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:23.404155  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:23.407300  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:23.405857  6503 rpcz_store.cc:276] 0709 12:19:20.731034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:20.807704 (+ 76670us) service_pool.cc:224] Handling call
0709 12:19:20.808138 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:22.319522 (+1511384us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:23.390375 (+1070853us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b54002c42d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:23.390592 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3373953280}
I20260709 12:19:23.403820  6550 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:23.409602  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:23.410833  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:23.429163  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:23.436501  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:23.436839  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:23.437849  6577 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:23.431566  6550 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:23.389896 12189 consensus_meta.cc:287] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.543s	user 0.009s	sys 0.001s
W20260709 12:19:23.390914 12208 log.cc:927] Time spent T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:19:23.448084  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:23.450958  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:23.452092  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:23.454262  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:23.456017  6501 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:19:23.361990 12124 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:23.486779  6579 raft_consensus.cc:2393] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 18: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:23.460788  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:23.489792  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:23.490520  6581 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:19:23.466461  6473 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:23.467326 10999 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:23.492096  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:23.492431 10999 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:23.495501  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:23.499315  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:23.499485 12030 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.499940  7265 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:19:23.500433  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:23.500103 12030 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:23.500618  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:23.501276  6501 raft_consensus.cc:2410] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:19:23.501784  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:23.502147  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:23.502485 12030 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:23.502804  6501 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:23.486954 12189 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:23.500173  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:23.504314  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:23.505672  6579 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:19:23.506302 11889 consensus_meta.cc:287] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.615s	user 0.006s	sys 0.003s
I20260709 12:19:23.505033  6503 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:19:23.506732 11889 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:23.507542  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:23.509022  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:23.509446 12169 ts_tablet_manager.cc:1929] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 5.5
I20260709 12:19:23.510004 12169 log.cc:1199] T 3a47d71c34b541b9b2589b4e80413fa2 P f2b06aebe97249a4877d798b8118ece4: Deleting WAL directory at /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-2-root/wals/3a47d71c34b541b9b2589b4e80413fa2
I20260709 12:19:23.502796  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:23.510433  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:23.504531  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:23.561794  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:23.562804  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:23.564551  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:23.537972  6577 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
W20260709 12:19:23.522524  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 56186.9us
I20260709 12:19:23.441035 12048 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:23.571481  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:23.570770  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:23.580926  6650 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:19:23.515370  6579 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 11.
I20260709 12:19:23.585191  6575 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:19:23.569235  6473 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:23.591589  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:23.487286 12229 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.459928 12087 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.388960 12140 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:23.613227  6501 raft_consensus.cc:1275] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 12:19:23.613616  6501 pending_rounds.cc:85] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 8
I20260709 12:19:23.613902  6501 pending_rounds.cc:107] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:19:23.614151  6501 raft_consensus.cc:2889] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:23.613415  6473 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:23.615483  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:23.617017  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:23.450554 11478 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.455451 11905 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:19:23.608870  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 158727us
I20260709 12:19:23.460403 12006 consensus_queue.cc:1059] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.141s
I20260709 12:19:23.621531 10999 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:23.639168 10999 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:23.443207 12166 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.139s
I20260709 12:19:23.400225 12164 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:23.455261 11767 consensus_queue.cc:1048] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.170s
I20260709 12:19:23.485311 11952 raft_consensus.cc:697] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:23.570751 12030 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:23.676775 12030 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:23.677336 12030 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:23.627226  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:23.678845  6656 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:23.605161 11889 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.621882 12087 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:23.680682 11889 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:23.683596 11889 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:23.643471  6503 raft_consensus.cc:1275] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:19:23.624509  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:23.588572 12048 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:23.620455 12163 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.605015 12222 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:23.709607  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:23.606873 12124 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.652805  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:23.513499 12118 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.135s
I20260709 12:19:23.667372 12169 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:60084 (request call id 0) took 2290 ms. Trace:
I20260709 12:19:23.640741 11478 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:23.390796 12191 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:23.655660 11905 raft_consensus.cc:697] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:23.623255 12229 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:23.581894 12139 consensus_queue.cc:1048] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 12:19:23.628000 12221 consensus_queue.cc:1059] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 12:19:23.701506  6651 raft_consensus.cc:2410] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:19:23.645007 11957 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:19:23.653116 12167 consensus_meta.cc:287] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.792s	user 0.011s	sys 0.000s
I20260709 12:19:23.706730  6549 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:23.710713  7273 raft_consensus.cc:2410] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 10 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 12:19:23.712026  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:23.622066 12217 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 12:19:23.726403  7266 raft_consensus.cc:2376] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16. Re-sending same reply.
I20260709 12:19:23.685005 11952 consensus_queue.cc:237] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:23.703963 12048 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:23.738703  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:23.740149  6503 raft_consensus.cc:2376] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15. Re-sending same reply.
I20260709 12:19:23.746588  7267 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 18: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
W20260709 12:19:23.746297  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 258520us
I20260709 12:19:23.756943 11905 consensus_queue.cc:237] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:23.760317  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 53710.6us
I20260709 12:19:23.763434  6549 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:23.764246  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:23.740186 11957 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:23.771772 11966 consensus_queue.cc:260] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:23.773053 12229 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:23.773206  6549 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:23.786381  6549 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:23.773315  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:23.774426  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:23.774619  6655 raft_consensus.cc:1275] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 12:19:23.786264  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 363551us
W20260709 12:19:23.774715 12202 log.cc:927] Time spent T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 1.252s	user 0.004s	sys 0.000s
I20260709 12:19:23.774998 11897 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:23.777599  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:23.779616 12238 log.cc:927] Time spent T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:19:23.781852 12191 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:23.741614 12167 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:23.790729 12112 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:23.712882 12163 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:23.761430 12157 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.797348  6655 raft_consensus.cc:1275] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 12:19:23.807272  6655 pending_rounds.cc:85] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 8
I20260709 12:19:23.807538  6655 pending_rounds.cc:107] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 12:19:23.807770  6655 raft_consensus.cc:2889] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:23.791922  6622 connection.cc:256] RPC call timeout handler was delayed by -1.1135228230000394s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:19:23.810304  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 2991) took 1875 ms. Trace:
I20260709 12:19:23.799259  6581 raft_consensus.cc:1240] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:23.801051  6501 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:19:23.802196  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:23.785071 11957 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:23.734808 12124 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:23.785645 12242 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:23.810865  6649 rpcz_store.cc:276] 0709 12:19:21.935144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.935316 (+   172us) service_pool.cc:224] Handling call
0709 12:19:21.954242 (+ 18926us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:21.954580 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:21.954609 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:22.021672 (+ 67063us) log.cc:844] Serialized 26 byte log entry
0709 12:19:23.809568 (+1787896us) spinlock_profiling.cc:243] Waited 56.4 ms on lock 0x7b18002bbac0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2875a314bc 00007f2872d7f48b 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1
0709 12:19:23.809759 (+   191us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:23.809810 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:23.809904 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:23.809960 (+    56us) raft_consensus.cc:1685] finished
0709 12:19:23.809977 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:23.810232 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23360,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":16248,"spinlock_wait_cycles":129791360,"thread_start_us":176889,"threads_started":1,"wal-append.queue_time_us":174697,"wal-append.run_cpu_time_us":2061,"wal-append.run_wall_time_us":1185193}
I20260709 12:19:23.820081  6658 raft_consensus.cc:1240] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:23.814440  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:23.821650  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:23.824455  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:23.817848  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:23.819546  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:23.836547  6655 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
W20260709 12:19:23.839957  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:23.813119  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:23.796355 12123 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.841111  6501 raft_consensus.cc:2393] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 14: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:23.842489  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:23.845188  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:23.846293  6503 raft_consensus.cc:2410] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:19:23.847820  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:23.848515  7266 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:23.848093  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:23.852289  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:23.815428 11966 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:23.815786 12112 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:23.851293 11773 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.841126 12242 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:23.864369  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:23.868855  6474 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:19:23.847498 12227 log.cc:927] Time spent T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.587s	user 0.000s	sys 0.004s
I20260709 12:19:23.878386  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:23.881381  6654 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 2366) took 2344 ms (client timeout 1870 ms). Trace:
W20260709 12:19:23.882001  6654 rpcz_store.cc:269] 0709 12:19:21.536709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.543707 (+  6998us) service_pool.cc:224] Handling call
0709 12:19:23.878490 (+2334783us) spinlock_profiling.cc:243] Waited 667 ms on lock 0x7b54002a35d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:23.881276 (+  2786us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1533119872}
I20260709 12:19:23.854948 12223 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.851044 12167 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:23.871981 12165 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:23.897739  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:23.847661 12123 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:23.878927 12139 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:19:23.917304  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:23.925112  6654 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15. Ops: []
W20260709 12:19:23.895748  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:23.872751 11773 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:23.726228 12169 rpcz_store.cc:276] 0709 12:19:21.468728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.500318 (+ 31590us) service_pool.cc:224] Handling call
0709 12:19:21.917662 (+417344us) ts_tablet_manager.cc:1151] Deleting tablet 3a47d71c34b541b9b2589b4e80413fa2
0709 12:19:21.937057 (+ 19395us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:19:23.276793 (+1339736us) tablet_metadata.cc:665] Metadata flushed
0709 12:19:23.509313 (+232520us) tablet_metadata.cc:665] Metadata flushed
0709 12:19:23.667274 (+157961us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":368697,"tablet-delete.queue_time_us":358232,"thread_start_us":253832,"threads_started":1}
I20260709 12:19:23.871280 12242 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:23.879375 12140 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:23.893335 12087 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:23.949126  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:23.894933 12163 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:23.941236  6655 raft_consensus.cc:1240] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:23.908236 12139 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:23.968552  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:23.969699  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:23.970327  6505 raft_consensus.cc:2368] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18.
I20260709 12:19:23.973583 11773 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:23.974138 11773 consensus_queue.cc:1059] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:23.974779 11773 consensus_queue.cc:1243] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:19:23.943053 11478 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:23.975658  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:23.976157  6654 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:19:23.976794 12163 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.977521 12163 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:23.978441  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:23.979632 12163 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:23.980731  7240 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:23.981529 12163 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:23.982441  6575 raft_consensus.cc:1240] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:23.984365 12123 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:23.984536  6394 catalog_manager.cc:5028] TS f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): tablet 3a47d71c34b541b9b2589b4e80413fa2 (table test-workload [id=f93362d0518e483fbe3a01f43a217e48]) successfully deleted
I20260709 12:19:23.986670  6503 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:19:23.987649  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:23.987746  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:23.988490  6504 raft_consensus.cc:2410] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:19:23.989267  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:23.992190  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:23.994349  6505 raft_consensus.cc:2368] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18.
I20260709 12:19:23.997125  6656 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:19:23.998543  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:23.999511  6656 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 18: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:23.999365  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:23.942611  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:23.897541 12217 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:23.898242 12165 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:23.899792 12164 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:24.014182  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:24.002876  6503 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 14: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:23.951378 12250 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:23.898975 12167 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:24.012635  6501 raft_consensus.cc:2393] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 11: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:24.001004  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:24.016047  6653 raft_consensus.cc:1240] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:24.017166  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:23.965116 12139 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:24.023665  6656 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 14.
I20260709 12:19:24.030087  7240 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:23.815692 12241 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.207s
I20260709 12:19:24.023169  6501 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 12:19:24.014618 12165 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:23.896250 12223 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:23.841888 12157 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:24.052372  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 266014us
I20260709 12:19:24.035380  6656 raft_consensus.cc:1275] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:19:24.025373  6503 raft_consensus.cc:2376] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13. Re-sending same reply.
I20260709 12:19:23.961777 12182 consensus_queue.cc:1059] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 12:19:24.038901  6505 raft_consensus.cc:1275] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 12:19:23.960201 12222 raft_consensus.cc:697] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:24.062110  6656 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 12:19:24.061123  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 128198us
I20260709 12:19:24.036877  6501 pending_rounds.cc:85] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 9
I20260709 12:19:24.060374  6505 pending_rounds.cc:85] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 8
I20260709 12:19:24.063423  6505 pending_rounds.cc:107] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:19:24.063818  6505 raft_consensus.cc:2889] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:24.015893 12217 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:24.065599  6501 pending_rounds.cc:107] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:19:24.065745  6655 raft_consensus.cc:1240] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:19:24.066069  6501 raft_consensus.cc:2889] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:24.055258  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 210978us
I20260709 12:19:24.019727 12167 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:24.059772 12246 log.cc:927] Time spent T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 12:19:24.065099  6654 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 12:19:24.078238  6654 pending_rounds.cc:85] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 7
I20260709 12:19:24.078807  6654 pending_rounds.cc:107] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 12:19:24.079257  6654 raft_consensus.cc:2889] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:24.080477  6582 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:24.017581 12250 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:24.086006  6656 raft_consensus.cc:3055] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:24.086542  6656 raft_consensus.cc:740] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:24.069118 12182 raft_consensus.cc:3055] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:24.070650 12124 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:24.089088 12176 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:24.079586 12167 raft_consensus.cc:3055] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:24.146039  6504 raft_consensus.cc:2376] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12. Re-sending same reply.
I20260709 12:19:24.140712  6656 consensus_queue.cc:260] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:24.166411 12223 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:24.167701 12248 log.cc:927] Time spent T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 12:19:24.173338 12157 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:24.179093 12123 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:24.181098  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 92467us
I20260709 12:19:24.096249 12182 raft_consensus.cc:740] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:24.183190  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:24.279480  7274 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:19:24.181088 12250 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:24.184242 12240 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.186470  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:24.187340 12217 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:24.187829 12153 raft_consensus.cc:3055] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:24.189142  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:24.194453  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:24.206689 12167 raft_consensus.cc:740] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:19:24.244081  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:24.275810  6656 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:24.276481  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 209373us
I20260709 12:19:24.172063 11957 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:24.282384  6649 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:19:24.122155 12222 consensus_queue.cc:237] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:24.310263  7266 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:19:24.291347  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:19:24.296995  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 219589us
I20260709 12:19:24.314158  6475 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:24.206861 12189 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.075s
I20260709 12:19:24.302870  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:24.317443  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:24.318279  6656 raft_consensus.cc:1275] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:24.320179  6656 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:19:24.310508  6579 raft_consensus.cc:1240] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 10. Current term is 11. Ops: [10.12-10.12]
I20260709 12:19:24.285810 12153 raft_consensus.cc:740] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:24.327247  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:24.327936  6576 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 14.
I20260709 12:19:24.208580 12048 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:24.308408 12030 consensus_meta.cc:287] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.631s	user 0.017s	sys 0.001s
I20260709 12:19:24.345482  6475 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:24.359573  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:24.360286  6579 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 12.
I20260709 12:19:24.361476  6475 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:24.208026 12176 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:24.364923  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:24.368100  6500 raft_consensus.cc:2393] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 19: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:24.282331 12123 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:24.328183 11966 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:24.307708 12250 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:24.376821  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:24.343746 11952 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:24.408159  6508 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
I20260709 12:19:24.345469  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:24.416720  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:24.366254 12252 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:19:24.416612 12197 log.cc:927] Time spent T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 12:19:24.382556 12182 consensus_queue.cc:260] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:24.413968 11952 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:24.406041 12009 raft_consensus.cc:3055] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:24.301244 12240 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:24.414608 12047 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.335928 12217 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 12:19:24.445004  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:24.363344 12140 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 12:19:24.383531 12228 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:24.416637 12164 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:24.364511 12165 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:24.452157  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 73935.9us
I20260709 12:19:24.364845 12263 consensus_queue.cc:1059] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 12:19:24.370646 12030 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:24.471652  6506 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:24.472625 12216 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:24.473819 12165 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:24.464633  6582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2462) took 1003 ms. Trace:
I20260709 12:19:24.476661  6582 rpcz_store.cc:276] 0709 12:19:23.461123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.465560 (+  4437us) service_pool.cc:224] Handling call
0709 12:19:23.465924 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:24.084116 (+618192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:24.464402 (+380286us) spinlock_profiling.cc:243] Waited 614 ms on lock 0x7b54000573d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:24.464569 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1412666112}
I20260709 12:19:24.477411  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 679) took 1164 ms. Trace:
W20260709 12:19:24.465251  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:24.477808  6509 rpcz_store.cc:276] 0709 12:19:23.313120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.315939 (+  2819us) service_pool.cc:224] Handling call
0709 12:19:23.316284 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:23.397886 (+ 81602us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:24.477150 (+1079264us) spinlock_profiling.cc:243] Waited 80.6 ms on lock 0x7b540017c5d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:24.477337 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":185354624}
I20260709 12:19:24.478655 12030 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:24.483750 11957 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:24.445617 12124 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.485015 12263 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:24.485194 12124 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:24.486454  7267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38558 (request call id 803) took 1089 ms. Trace:
I20260709 12:19:24.486846  7267 rpcz_store.cc:276] 0709 12:19:23.401122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.405187 (+  4065us) service_pool.cc:224] Handling call
0709 12:19:24.486241 (+1081054us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b54006c2354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:24.486380 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":647974656}
I20260709 12:19:24.487548 12124 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:24.489259 12176 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:24.449434 12009 raft_consensus.cc:740] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:24.490559 12009 consensus_queue.cc:260] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:24.489135  6576 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:24.491648 12009 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:24.449608 11952 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:24.497028 12223 raft_consensus.cc:3055] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:24.497539 12223 raft_consensus.cc:740] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:19:24.498957  6508 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41304 (request call id 672) took 2951 ms (client timeout 1514 ms). Trace:
W20260709 12:19:24.499526  6508 rpcz_store.cc:269] 0709 12:19:21.547781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.547935 (+   154us) service_pool.cc:224] Handling call
0709 12:19:24.498670 (+2950735us) spinlock_profiling.cc:243] Waited 2.64 s on lock 0x7b5400434c54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:24.498828 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6070066304}
I20260709 12:19:24.505589  6656 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:24.485603 12263 raft_consensus.cc:697] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:24.410878 12250 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:24.350780 12153 consensus_queue.cc:260] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:24.515865 12178 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.519834  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2463) took 1047 ms. Trace:
I20260709 12:19:24.523025  6577 rpcz_store.cc:276] 0709 12:19:23.493723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.567803 (+ 74080us) service_pool.cc:224] Handling call
0709 12:19:23.568143 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:24.063646 (+495503us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:19:24.063678 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:24.084700 (+ 21022us) log.cc:844] Serialized 26 byte log entry
0709 12:19:24.410351 (+325651us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:24.410455 (+   104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:24.410598 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:24.519632 (+109034us) raft_consensus.cc:1685] finished
0709 12:19:24.519651 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:24.519775 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":114129,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"thread_start_us":601836,"threads_started":2,"wal-append.queue_time_us":236347}
I20260709 12:19:24.453591 12255 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.523665  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:24.458470 12271 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:19:24.526631  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 181214us
I20260709 12:19:24.526825  6550 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:24.416410 12167 consensus_queue.cc:260] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:24.554492  7272 raft_consensus.cc:1275] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 10. (term mismatch)
I20260709 12:19:24.555006  7272 pending_rounds.cc:85] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9: Aborting all ops after (but not including) 9
I20260709 12:19:24.555286  7272 pending_rounds.cc:107] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9: Aborting uncommitted NO_OP operation due to leader change: 8.10
I20260709 12:19:24.555589  7272 raft_consensus.cc:2889] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:24.559149 11767 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:19:24.560289 12235 log.cc:927] Time spent T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.835s	user 0.001s	sys 0.010s
I20260709 12:19:24.568009  6575 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:19:24.567957 11773 consensus_queue.cc:1048] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:24.569259  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:24.569892  6575 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:19:24.570786  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:24.574365 12242 consensus_meta.cc:287] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.618s	user 0.008s	sys 0.003s
I20260709 12:19:24.574457 11952 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:24.574928 12242 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:24.576548 12258 log.cc:927] Time spent T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.178s	user 0.002s	sys 0.000s
I20260709 12:19:24.578473 12242 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:24.580183 12240 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:24.583410  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 52681.1us
I20260709 12:19:24.585461  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:24.585632  6577 raft_consensus.cc:1240] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:24.458272 11966 consensus_queue.cc:1048] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 12:19:24.588449  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:24.589170  6508 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:24.590230  6651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2390) took 1185 ms. Trace:
I20260709 12:19:24.591848  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:24.592509  6501 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:19:24.590853  6651 rpcz_store.cc:276] 0709 12:19:23.404937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.767545 (+362608us) service_pool.cc:224] Handling call
0709 12:19:23.767876 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:23.789598 (+ 21722us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:23.789627 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:23.796278 (+  6651us) log.cc:844] Serialized 26 byte log entry
0709 12:19:24.077876 (+281598us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:24.077935 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:24.078026 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:24.575845 (+497819us) spinlock_profiling.cc:243] Waited 61.5 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:24.587794 (+ 11949us) raft_consensus.cc:1685] finished
0709 12:19:24.587816 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:24.590164 (+  2348us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21287,"raft.queue_time_us":21769,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":6644,"spinlock_wait_cycles":141517056,"thread_start_us":251459,"threads_started":1,"wal-append.queue_time_us":535974}
I20260709 12:19:24.593833  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:24.594401  6651 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:24.596570  6650 raft_consensus.cc:1240] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:24.597577  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:24.598332  6503 raft_consensus.cc:2393] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 13: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:24.599227  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:24.581941 12123 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:24.604141  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:24.604805  6505 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:24.607771 11952 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:24.531383 11905 consensus_queue.cc:1048] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:19:24.527925 12178 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:24.539659 12255 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:24.599079 11767 consensus_queue.cc:1059] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 12:19:24.622440  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:24.604600 12009 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
W20260709 12:19:24.610554 12139 consensus_meta.cc:287] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.571s	user 0.011s	sys 0.000s
I20260709 12:19:24.547914 12223 consensus_queue.cc:260] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:24.641901  6582 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:19:24.610538 11840 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.503509 12047 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:24.650715  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:24.651705  6575 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:24.654619  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:24.608487 12123 consensus_queue.cc:1059] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 12:19:24.618418 12272 consensus_queue.cc:1048] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:19:24.617964 12176 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:19:24.649648 11957 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.650810 12048 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:24.648924 12217 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.648813 12276 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:19:24.730059  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:24.731349  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:24.737339  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:19:24.738134  6548 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.000s	sys 0.012s
W20260709 12:19:24.739377  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:24.740773  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:24.741966  6509 raft_consensus.cc:2393] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 14: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:24.743085  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:24.743952  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:24.738498  6548 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.000s	sys 0.012s
W20260709 12:19:24.749563  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 666132us
I20260709 12:19:24.750294  6548 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:24.609210 12263 consensus_queue.cc:237] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:24.754153  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:24.753436  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:24.760599  6508 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 18: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:24.711236 12223 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:24.757911  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:24.728447 12083 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.447s	user 0.015s	sys 0.008s
I20260709 12:19:24.772356  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:24.649170 12139 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:24.770710  6548 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:24.778924  6548 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:24.780609  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:24.777419 12083 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.496s	user 0.015s	sys 0.008s
I20260709 12:19:24.782922  6548 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:24.655521 11840 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:24.794490  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:24.752732 12277 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:24.800269  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:24.745975 11957 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:24.777099 12165 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.775128 12257 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:24.855866  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:24.775678 12120 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.782318 12048 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:24.751055 12217 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:24.793918 12256 raft_consensus.cc:3055] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:24.860809  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:24.863705  6657 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:24.864918  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:24.865607  6657 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:19:24.866696  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:24.868719  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 74273.7us
I20260709 12:19:24.869779  6622 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:24.815773 12083 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:24.862349 12165 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:24.877525  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 94565.8us
I20260709 12:19:24.879604  6548 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:24.881608 12083 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:24.882427 12083 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:24.883298 12083 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:24.894084  6548 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:24.904281  6548 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:24.834213 11905 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:24.851301 12277 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:24.862924 12120 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:24.753803 12276 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:24.922199 12269 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:24.922420  6658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 882) took 2125 ms. Trace:
I20260709 12:19:24.931984  6658 rpcz_store.cc:276] 0709 12:19:22.797352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:22.813922 (+ 16570us) service_pool.cc:224] Handling call
0709 12:19:22.814182 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:23.834165 (+1019983us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:24.922144 (+1087979us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540010e354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:24.922334 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2312875776}
W20260709 12:19:24.921559 12282 log.cc:927] Time spent T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:19:24.851368 11889 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:19:24.947803  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 78136.1us
I20260709 12:19:24.949833  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:24.865923 12232 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:24.948498 10999 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:24.863302 12256 raft_consensus.cc:740] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:24.955037  6622 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:24.967296 12269 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.967696  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2401) took 1507 ms. Trace:
I20260709 12:19:24.967905 12269 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:24.970508 12269 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:24.929177 11905 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:24.973644  6622 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:24.975682  6655 rpcz_store.cc:276] 0709 12:19:23.460407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.961413 (+501006us) service_pool.cc:224] Handling call
0709 12:19:23.966004 (+  4591us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:24.066225 (+100221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:24.967267 (+901042us) spinlock_profiling.cc:243] Waited 99.5 ms on lock 0x7b54000753d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:24.967596 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":228777728}
I20260709 12:19:24.953212 12048 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:24.965581 12189 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:24.991374  6504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 2660) took 1201 ms. Trace:
I20260709 12:19:24.965152 12164 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:24.974478 12257 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:24.993523  6473 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:24.991799  6504 rpcz_store.cc:276] 0709 12:19:23.789714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.989024 (+199310us) service_pool.cc:224] Handling call
0709 12:19:24.991184 (+1002160us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b540001ccd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:24.991310 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":359189888}
I20260709 12:19:24.979813 11905 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:24.976497 12295 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:24.977473 11840 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:25.003690  6473 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:24.986378 12287 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:19:24.985924 12280 log.cc:927] Time spent T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.130s	user 0.002s	sys 0.000s
I20260709 12:19:24.992211 12189 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:25.022753  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 2337) took 1684 ms. Trace:
I20260709 12:19:25.034647  6583 rpcz_store.cc:276] 0709 12:19:23.338414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.339350 (+   936us) service_pool.cc:224] Handling call
0709 12:19:23.339678 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:23.340034 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:23.340066 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:23.340544 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 12:19:24.493397 (+1152853us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b18002bf2a0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:19:24.729075 (+235678us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:24.729134 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:24.729237 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:25.008698 (+279461us) raft_consensus.cc:1685] finished
0709 12:19:25.008731 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:25.022668 (+ 13937us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":840,"raft.run_cpu_time_us":6501,"raft.run_wall_time_us":13561,"spinlock_wait_cycles":33877760,"thread_start_us":235423,"threads_started":1,"wal-append.queue_time_us":1409856}
W20260709 12:19:25.036022  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 62614.1us
I20260709 12:19:25.002739 12255 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:25.045877  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 142571us
I20260709 12:19:25.095029  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:25.005226 12164 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:25.099437 12124 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.000s	sys 0.019s
I20260709 12:19:25.098397  6548 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:25.008394 12222 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:25.009078 12257 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:25.037623 12189 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:25.034098 12272 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:25.035706 12221 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.271s	user 0.030s	sys 0.036s
I20260709 12:19:25.038231 11840 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.110639  6548 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:25.114748  6654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2402) took 1654 ms. Trace:
I20260709 12:19:25.305037  6654 rpcz_store.cc:276] 0709 12:19:23.460752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.977315 (+516563us) service_pool.cc:224] Handling call
0709 12:19:23.977605 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:24.079989 (+102384us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:25.114489 (+1034500us) spinlock_profiling.cc:243] Waited 84.5 ms on lock 0x7b54002a35d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:25.114685 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":194307456}
W20260709 12:19:25.130383  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:25.306770  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:25.277295  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 242807us
I20260709 12:19:25.097069 12287 raft_consensus.cc:697] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:19:25.106721 12124 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.000s	sys 0.019s
I20260709 12:19:25.115477  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:25.318325  6652 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:19:25.320571  6576 raft_consensus.cc:1275] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:25.287869 12141 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:19:25.321758 12288 log.cc:927] Time spent T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.369s	user 0.000s	sys 0.000s
W20260709 12:19:25.288259 12221 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.526s	user 0.030s	sys 0.036s
W20260709 12:19:25.322449 12231 log.cc:927] Time spent T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 1.713s	user 0.030s	sys 0.009s
W20260709 12:19:25.323515 12293 log.cc:927] Time spent T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.341s	user 0.001s	sys 0.000s
I20260709 12:19:25.324169  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:19:25.324920  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 214213us
I20260709 12:19:25.325403  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:25.325984 12276 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:25.326261  6583 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:19:25.326045  6650 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:25.327620 12297 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:25.329273 12222 raft_consensus.cc:697] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:25.308050  6473 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:25.108670 12112 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:25.325922  6548 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:25.282104  6475 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:25.323333  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:25.321496 12141 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:25.321842  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:25.324856  6577 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:25.037626 12176 consensus_queue.cc:1059] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.250s
W20260709 12:19:25.347306  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 65447.9us
W20260709 12:19:25.288550 12239 log.cc:927] Time spent T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 1.563s	user 0.008s	sys 0.007s
W20260709 12:19:25.326095  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:25.338413  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 786) took 2220 ms. Trace:
I20260709 12:19:25.280311 12139 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:25.339627  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:25.313483 12177 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:25.343371  6548 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:25.341095  6473 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:25.348083  6475 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:25.325611 12284 log.cc:927] Time spent T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.317s	user 0.000s	sys 0.000s
I20260709 12:19:25.351598  6580 rpcz_store.cc:276] 0709 12:19:23.113324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.113798 (+   474us) service_pool.cc:224] Handling call
0709 12:19:23.118199 (+  4401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:23.188816 (+ 70617us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:23.188847 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:23.189307 (+   460us) log.cc:844] Serialized 26 byte log entry
0709 12:19:23.334867 (+145560us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:23.334911 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:23.814731 (+479820us) spinlock_profiling.cc:243] Waited 70.2 ms on lock 0x7b54000582d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:23.814781 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:25.321784 (+1507003us) spinlock_profiling.cc:243] Waited 3.43 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:25.333489 (+ 11705us) raft_consensus.cc:1685] finished
0709 12:19:25.333516 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:25.333678 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2511,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"spinlock_wait_cycles":169346176,"thread_start_us":145019,"threads_started":1,"wal-append.queue_time_us":261114}
I20260709 12:19:25.339681  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:25.396077  6583 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:25.351163  6622 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:25.397140  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 46061.7us
I20260709 12:19:25.397331 12301 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:25.397635 12141 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:25.397843  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:25.111375 12255 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:25.023247 12295 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:25.398114 12301 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:25.388819 11889 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:25.400516 12165 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:25.330274 12222 consensus_queue.cc:237] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 9.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:25.401460 12165 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:25.401595 12287 consensus_queue.cc:237] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:25.334446 11840 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:25.343670 11957 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:25.352743 12124 consensus_queue.cc:1048] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 12:19:25.404778 12298 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:25.405239 11840 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:25.406780 11840 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:25.407279 11840 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:25.352591 12276 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:25.412703 12276 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:25.413089 12276 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:25.414125 12301 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:25.414230 12153 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:25.415854 11840 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:25.347419 11773 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:25.416782 11773 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:25.417876 12167 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:25.419175 11773 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:25.420629 12083 consensus_meta.cc:287] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.536s	user 0.011s	sys 0.003s
I20260709 12:19:25.420862 12277 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:25.011926 12256 consensus_queue.cc:260] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:25.421066 12083 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:25.344681 12272 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:19:25.423314  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:25.423939 12083 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:25.391069  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 50071.7us
I20260709 12:19:25.392764 11913 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:25.392897 12087 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:25.345608 12297 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:25.398253 12141 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:25.352964 12048 consensus_queue.cc:1048] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:19:25.402004 12165 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:25.355437  6579 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
W20260709 12:19:25.428135  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:25.386094  6577 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:19:25.431609  6652 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:19:25.349175 11478 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 12:19:25.460677  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:25.433961  6473 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:25.392995 12291 log.cc:927] Time spent T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.298s	user 0.000s	sys 0.001s
W20260709 12:19:25.439628  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 96325.8us
I20260709 12:19:25.461323  6652 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:19:25.461956  6548 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:25.462175  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:25.463140  6548 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:25.423357 12299 log.cc:927] Time spent T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:19:25.425771 12257 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:25.471489  6474 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:25.405296 12298 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:25.470908  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:25.426374 12120 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:25.473426 11478 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:25.473498  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:25.474117  6652 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
I20260709 12:19:25.476194 11478 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:25.436103 12217 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:25.443797 11913 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:25.439112 11767 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:25.465823 12165 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:25.485674  6548 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:25.485931 11897 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:25.442813 12087 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:25.435945 11957 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:25.438874 12281 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:25.443076 12124 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:19:25.518162  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:25.504351 11478 consensus_queue.cc:1059] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:25.479202 12176 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:25.518760  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 170744us
I20260709 12:19:25.487408 12297 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:25.511099  6548 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:25.520725  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:25.439105 12177 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:25.519351 11478 raft_consensus.cc:3055] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:25.565176 11478 raft_consensus.cc:740] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:25.459268 12141 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:25.477300 12298 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:25.521390  7272 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:25.559762  6548 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:25.557345  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:25.518327 12257 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:25.515259 12087 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:25.527124 12223 consensus_meta.cc:287] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.748s	user 0.005s	sys 0.005s
I20260709 12:19:25.513185 11767 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:25.563197 12297 raft_consensus.cc:697] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:25.658569 11767 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:25.659315  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:25.574185 12176 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:25.645026 11478 consensus_queue.cc:260] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:25.520468 11957 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:25.662961 11478 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:25.668002 11905 consensus_meta.cc:287] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.641s	user 0.005s	sys 0.000s
I20260709 12:19:25.662042 12176 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:25.564143 12048 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:25.520823 12281 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:25.638208 12124 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:25.672739 12281 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:25.653117 12177 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:25.672982 12124 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:25.674070  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 114517us
I20260709 12:19:25.675271 12124 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:25.675654  6548 leader_election.cc:400] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18.
I20260709 12:19:25.675849 12124 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:25.660000  7266 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:25.664880 11957 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:25.678966  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:25.679809 12295 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:25.573465 12083 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:19:25.681067 12314 log.cc:927] Time spent T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:19:25.681013 12083 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:25.682516  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2462) took 1063 ms. Trace:
I20260709 12:19:25.682471  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:25.683351  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:25.683634  7266 raft_consensus.cc:2410] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:19:25.683504 12083 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:25.657796 12297 consensus_queue.cc:237] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:25.639626 12141 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:25.689764  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2473) took 1244 ms. Trace:
W20260709 12:19:25.691373  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:25.646509  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 681) took 2332 ms. Trace:
I20260709 12:19:25.683079  6509 rpcz_store.cc:276] 0709 12:19:24.618987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.744279 (+125292us) service_pool.cc:224] Handling call
0709 12:19:24.744744 (+   465us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:24.774819 (+ 30075us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:24.776325 (+  1506us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:24.783389 (+  7064us) log.cc:844] Serialized 52 byte log entry
0709 12:19:25.339069 (+555680us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:25.522114 (+183045us) log.cc:844] Serialized 14 byte log entry
0709 12:19:25.560786 (+ 38672us) log.cc:844] Serialized 14 byte log entry
0709 12:19:25.560968 (+   182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:25.561057 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:25.682292 (+121235us) raft_consensus.cc:1685] finished
0709 12:19:25.682311 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:25.682459 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":111418,"raft.run_cpu_time_us":311,"raft.run_wall_time_us":67408,"thread_start_us":616231,"threads_started":2,"wal-append.queue_time_us":651116}
I20260709 12:19:25.690207  6548 leader_election.cc:400] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18.
I20260709 12:19:25.693583  6509 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1997e98759d0465ab0e3c607df127068"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:58944
I20260709 12:19:25.694228  6548 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:25.694391  6509 raft_consensus.cc:606] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:25.690235  6584 rpcz_store.cc:276] 0709 12:19:24.445226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.447799 (+  2573us) service_pool.cc:224] Handling call
0709 12:19:24.461578 (+ 13779us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:24.504829 (+ 43251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:24.504880 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:24.529033 (+ 24153us) log.cc:844] Serialized 26 byte log entry
0709 12:19:25.100708 (+571675us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:25.100773 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:25.100891 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:25.689555 (+588664us) raft_consensus.cc:1685] finished
0709 12:19:25.689577 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:25.689707 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":45903,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":1606,"thread_start_us":498784,"threads_started":1,"wal-append.queue_time_us":913602}
I20260709 12:19:25.697834  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3025) took 1114 ms. Trace:
I20260709 12:19:25.692878  6507 rpcz_store.cc:276] 0709 12:19:23.313798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.316719 (+  2921us) service_pool.cc:224] Handling call
0709 12:19:23.319034 (+  2315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:23.319460 (+   426us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:19:23.319519 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:23.325547 (+  6028us) log.cc:844] Serialized 26 byte log entry
0709 12:19:23.581158 (+255611us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:23.581201 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:23.581268 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:25.009834 (+1428566us) spinlock_profiling.cc:243] Waited 7.91 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:25.573872 (+564038us) raft_consensus.cc:1685] finished
0709 12:19:25.573908 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:25.646400 (+ 72492us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10925,"raft.queue_time_us":99,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":4887,"spinlock_wait_cycles":18186496,"thread_start_us":174631,"threads_started":1,"wal-append.queue_time_us":394025}
I20260709 12:19:25.698237  6649 rpcz_store.cc:276] 0709 12:19:24.583117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.588035 (+  4918us) service_pool.cc:224] Handling call
0709 12:19:24.596836 (+  8801us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:24.597136 (+   300us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:24.597163 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:24.605839 (+  8676us) log.cc:844] Serialized 26 byte log entry
0709 12:19:24.780147 (+174308us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:24.780190 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:24.780264 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:25.697633 (+917369us) raft_consensus.cc:1685] finished
0709 12:19:25.697651 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:25.697774 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6241,"raft.queue_time_us":239,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":2132,"thread_start_us":172025,"threads_started":1,"wal-append.queue_time_us":371870}
I20260709 12:19:25.519433 12164 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:25.717531  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:25.718792  6649 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
W20260709 12:19:25.724289  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 326512us
I20260709 12:19:25.707167 12083 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.656203 12276 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:25.706552 11905 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:25.758030  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 286607us
I20260709 12:19:25.813102  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:25.729909  6548 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:25.761224  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:25.736886  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:25.762704  6507 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
new_leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:58944
I20260709 12:19:25.773463  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:25.756680  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 281542us
I20260709 12:19:25.814911  6507 raft_consensus.cc:606] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Received request to transfer leadership to TS 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:25.815382  6507 raft_consensus.cc:614] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:25.815261  6509 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
I20260709 12:19:25.814328  6474 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:25.816740 12222 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:25.816448 12083 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:25.814404  6584 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
I20260709 12:19:25.815536  6473 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:25.820595 11478 consensus_queue.cc:1048] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:19:25.821619  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 91812us
I20260709 12:19:25.766994  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:25.712955 12295 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:25.730912  7272 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:19:25.727167  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:25.815091 12310 log.cc:927] Time spent T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.371s	user 0.000s	sys 0.001s
W20260709 12:19:25.833110  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 106012us
I20260709 12:19:25.835079  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:25.828953 12222 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:25.744514 12124 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.090s
I20260709 12:19:25.743603 12120 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.107s
I20260709 12:19:25.851300  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:25.831378 11767 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:25.853000  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:25.852756  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:25.854717  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:25.855346  6578 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:25.856221  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:19:25.843946 12189 consensus_meta.cc:287] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.511s	user 0.010s	sys 0.000s
I20260709 12:19:25.842206 11897 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.867028  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:25.839053 12319 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:25.828630 12139 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 12:19:25.865339  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:25.846861 12222 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:25.856410 12300 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:25.879779  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:25.879882  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:25.866542 12311 log.cc:927] Time spent T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:19:25.866997  6579 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:25.881078  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:25.857259  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:25.881824  6578 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:25.868216  6580 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:25.876230  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:25.874662  6654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2418) took 1138 ms. Trace:
I20260709 12:19:25.848861 12255 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.848199 12281 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:25.854422 11905 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:25.850275 12287 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:25.891722  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:25.882782  6654 rpcz_store.cc:276] 0709 12:19:24.736281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:25.345595 (+609314us) service_pool.cc:224] Handling call
0709 12:19:25.385592 (+ 39997us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:25.434274 (+ 48682us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:25.434305 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:25.439797 (+  5492us) log.cc:844] Serialized 26 byte log entry
0709 12:19:25.813978 (+374181us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:25.814031 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:25.814100 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:25.874441 (+ 60341us) raft_consensus.cc:1685] finished
0709 12:19:25.874465 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:25.874605 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3784,"raft.queue_time_us":3942,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":116,"thread_start_us":330684,"threads_started":1,"wal-append.queue_time_us":370901}
I20260709 12:19:25.889973  6649 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 15: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:25.893309 12167 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:25.854884 12295 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:25.892554 12277 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:25.890700 12255 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:25.894769 12277 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:25.895217 12277 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:25.891340 12300 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:25.877821 11897 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:25.898674 12300 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:25.879619 12222 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:25.901587 12297 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.902493  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 684) took 1253 ms. Trace:
I20260709 12:19:25.902383 12297 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:25.902865  6501 rpcz_store.cc:276] 0709 12:19:24.649395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:25.417345 (+767950us) service_pool.cc:224] Handling call
0709 12:19:25.417698 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:25.418047 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:25.418080 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:25.418562 (+   482us) log.cc:844] Serialized 26 byte log entry
0709 12:19:25.755093 (+336531us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:25.755153 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:25.755248 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:25.902292 (+147044us) raft_consensus.cc:1685] finished
0709 12:19:25.902310 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:25.902434 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2657,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":85,"thread_start_us":336110,"threads_started":1,"wal-append.queue_time_us":456823}
I20260709 12:19:25.899667 12295 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:25.904544  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:25.905067  6474 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:25.905681  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:25.905952 12176 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:25.894145  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2421) took 1138 ms. Trace:
I20260709 12:19:25.892977  6622 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:25.906744 12176 raft_consensus.cc:697] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:25.906508  6652 rpcz_store.cc:276] 0709 12:19:24.755824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:25.486313 (+730489us) service_pool.cc:224] Handling call
0709 12:19:25.486666 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:25.504262 (+ 17596us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:25.504292 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:25.504783 (+   491us) log.cc:844] Serialized 26 byte log entry
0709 12:19:25.836820 (+332037us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:25.836861 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:25.836946 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:25.893952 (+ 57006us) raft_consensus.cc:1685] finished
0709 12:19:25.893972 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:25.894092 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17111,"raft.queue_time_us":19834,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":12727,"thread_start_us":276448,"threads_started":1,"wal-append.queue_time_us":353412}
I20260709 12:19:25.907895 12176 consensus_queue.cc:237] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:25.894207  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:25.899858 12276 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:25.868705 11767 raft_consensus.cc:697] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:25.918332  6505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2666) took 1270 ms. Trace:
I20260709 12:19:25.918865  6505 rpcz_store.cc:276] 0709 12:19:24.647964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.744977 (+ 97013us) service_pool.cc:224] Handling call
0709 12:19:24.745271 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:24.965232 (+219961us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:24.965272 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:24.965334 (+    62us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:24.965367 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:25.918010 (+952643us) spinlock_profiling.cc:243] Waited 220 ms on lock 0x7b54001e87d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:25.918091 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:25.918273 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":505326848}
I20260709 12:19:25.921043  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:25.922000  6622 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:25.922101  6505 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:25.874547 12319 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:25.874393 12302 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:25.930510  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:25.934330 12223 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:25.934729 12322 log.cc:927] Time spent T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:19:25.931908  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:25.891155 12263 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:25.940596  6578 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:25.942202  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:25.874132 12189 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:25.945266  6578 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:25.951313  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:25.952170  6505 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:25.952041  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:25.952491  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:25.941670 12153 consensus_meta.cc:287] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.527s	user 0.014s	sys 0.000s
I20260709 12:19:25.953792  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:25.954530  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:25.929766 12300 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:25.952503  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:25.957216  6578 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:19:25.911728 12281 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:25.959815  6655 raft_consensus.cc:1240] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:19:25.958370  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:25.915808 12287 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:25.940768 11913 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:25.933069 12319 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:25.963104  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:25.968554  6579 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:25.969758  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:25.971331  6579 raft_consensus.cc:1275] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:19:25.961490 12307 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:25.961982 12318 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:25.972628 12307 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:25.972905 12318 raft_consensus.cc:697] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:25.974179  6579 raft_consensus.cc:1275] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:25.974155 12318 consensus_queue.cc:237] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:25.967747 12257 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:25.976811  6583 raft_consensus.cc:1240] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:25.960170  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:25.978820  6578 raft_consensus.cc:2410] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:19:25.966972  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:25.981442  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:25.981678  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:25.979637 12277 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:25.946501 12304 consensus_queue.cc:1059] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:19:25.942569 12302 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:25.966771 12295 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:26.011554 12165 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:26.044806 12165 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:26.045544 12165 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:26.017524 12281 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:26.018559 12332 log.cc:927] Time spent T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:19:26.046964 12112 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:26.020155 12277 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:26.020437  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:26.021947 12189 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:25.988343 11767 consensus_queue.cc:237] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:26.048285  6651 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
I20260709 12:19:26.034705 12182 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:26.050805  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:26.024052 12287 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:26.029412 12302 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:26.032192 11889 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:26.052050 11889 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:26.003469 12087 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:26.053988  6653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3017) took 2382 ms. Trace:
I20260709 12:19:26.055002 12087 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:26.034612  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:26.055768  7266 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:26.056299  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:26.035499  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:26.058774  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:26.059458  7272 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:19:26.059996  7265 raft_consensus.cc:1240] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:26.059429 12141 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:26.037367 12263 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.061091  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:26.061138 12263 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:26.061870  7273 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:26.020720  6548 leader_election.cc:336] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.964s (SENT)
I20260709 12:19:26.062495  6548 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:26.063197  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:26.063591 12263 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:26.063763  7265 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:26.066462 12141 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:26.037469 12083 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:26.067889  6623 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:26.068183 12083 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:26.052021  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:26.069226  6503 raft_consensus.cc:2393] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 18: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:26.070097  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:26.070633  6503 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:26.008450 12223 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:26.072512 12298 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:26.068728 12167 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:26.073031  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:26.073477 12167 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:26.074266 12167 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:26.073997  6652 raft_consensus.cc:1240] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:19:26.054345  6653 rpcz_store.cc:276] 0709 12:19:23.671497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.034408 (+362911us) service_pool.cc:224] Handling call
0709 12:19:24.967530 (+933122us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:24.968147 (+   617us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:24.968180 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:24.968866 (+   686us) log.cc:844] Serialized 26 byte log entry
0709 12:19:25.035491 (+ 66625us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:25.035545 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:25.035618 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:25.920533 (+884915us) raft_consensus.cc:1685] finished
0709 12:19:25.920547 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:26.053759 (+133212us) spinlock_profiling.cc:243] Waited 933 ms on lock 0x7b54000753d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:26.053933 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13933,"raft.queue_time_us":8613,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":5960,"spinlock_wait_cycles":2144905984,"thread_start_us":66141,"threads_started":1,"wal-append.queue_time_us":466568}
I20260709 12:19:26.076696  6651 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:26.077535 12319 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:26.081207  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:26.073800  6580 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:26.039044 12304 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:26.040206 12312 log.cc:927] Time spent T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.556s	user 0.001s	sys 0.000s
W20260709 12:19:26.041839 12294 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.715s	user 0.000s	sys 0.001s
I20260709 12:19:26.047534 12112 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:26.035840  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:26.096243  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:26.098608  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:26.100242  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:26.100585  7265 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:19:26.103264  7269 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:26.113626  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:26.114122  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:26.115345  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:26.116848 12240 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:26.123270 12047 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:26.123617  7273 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:19:26.091825  6503 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:26.126070  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:26.126678  6503 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:26.127480  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:26.128016  6503 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:26.131194  6580 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:26.124807 11913 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:19:26.132550  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:26.134967  6658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2414) took 1400 ms. Trace:
I20260709 12:19:26.136859  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:26.143813  7273 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:26.139163  6549 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:26.137902  6622 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:26.147625  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:26.141345  6658 rpcz_store.cc:276] 0709 12:19:24.735004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.950141 (+215137us) service_pool.cc:224] Handling call
0709 12:19:24.951812 (+  1671us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:24.954062 (+  2250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:24.954110 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:24.954742 (+   632us) log.cc:844] Serialized 26 byte log entry
0709 12:19:25.360703 (+405961us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:25.360789 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:25.361041 (+   252us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:26.039592 (+678551us) spinlock_profiling.cc:243] Waited 2.25 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:26.134609 (+ 95017us) raft_consensus.cc:1685] finished
0709 12:19:26.134641 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:26.134906 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1627,"raft.queue_time_us":3484,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":17489,"spinlock_wait_cycles":5163904,"thread_start_us":353740,"threads_started":1,"wal-append.queue_time_us":507340}
I20260709 12:19:26.145251  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:26.097991 12006 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:26.164585  6503 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:26.165841  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:26.157686  6580 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:26.166620  6503 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:26.167474  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:26.169049  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:26.169600  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:26.169855  6503 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:19:26.170795  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:26.175613  6622 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:26.081458 12165 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:26.157356 12241 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:26.157145 12263 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:26.158789 12302 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:26.105233 12304 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:26.178761 12287 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:26.181025 12189 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:26.157805 11913 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:26.200086 12277 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:26.221303  6657 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:26.174172 12006 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:26.203078 12302 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:26.221082  6580 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:26.255851  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:26.256592  6580 raft_consensus.cc:2393] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 15: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:26.227839 12167 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:26.228847 12256 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:26.229557 12334 log.cc:927] Time spent T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:19:26.216873 12277 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:19:26.230647  6500 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 2650) took 2626 ms (client timeout 1964 ms). Trace:
I20260709 12:19:26.231691  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3024) took 2080 ms. Trace:
I20260709 12:19:26.300797  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:26.300989  6656 rpcz_store.cc:276] 0709 12:19:24.150970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.413339 (+262369us) service_pool.cc:224] Handling call
0709 12:19:24.413650 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:24.506186 (+ 92536us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.231398 (+1725212us) spinlock_profiling.cc:243] Waited 91.7 ms on lock 0x7b540008f7d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:26.231621 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":210843392}
I20260709 12:19:26.302960  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:26.303619  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:26.232565  6575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2480) took 1666 ms. Trace:
I20260709 12:19:26.239248 12250 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:26.245836 11897 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:26.250768 12297 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:26.308367  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:26.253795 12255 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:26.205476 12241 raft_consensus.cc:697] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:26.248010  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:26.264424  6622 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:26.310535  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:26.311817  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 47518.8us
I20260709 12:19:26.306838 12277 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:26.212204 12263 raft_consensus.cc:697] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:26.297420  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:26.301743  7274 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:19:26.315845  6579 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:26.316054  6650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2425) took 1247 ms. Trace:
I20260709 12:19:26.216987 12319 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:26.300801  6500 rpcz_store.cc:269] 0709 12:19:23.603942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.805551 (+201609us) service_pool.cc:224] Handling call
0709 12:19:26.230380 (+2424829us) spinlock_profiling.cc:243] Waited 516 ms on lock 0x7b540046cdd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:26.230563 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1186923904}
I20260709 12:19:26.316579  6650 rpcz_store.cc:276] 0709 12:19:25.069013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.043715 (+974702us) service_pool.cc:224] Handling call
0709 12:19:26.044071 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:26.044281 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:26.160192 (+115911us) log.cc:844] Serialized 14 byte log entry
0709 12:19:26.315756 (+155564us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:26.315802 (+    46us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:26.315840 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.315896 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:26.315990 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":172036,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"thread_start_us":270690,"threads_started":2}
I20260709 12:19:26.318639  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:26.318339 12263 consensus_queue.cc:237] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:26.320436  6650 raft_consensus.cc:1275] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:19:26.316926  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:26.320703  6503 raft_consensus.cc:1240] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:26.221097 12287 raft_consensus.cc:697] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:26.251447 12271 raft_consensus.cc:3055] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:26.309155  6658 raft_consensus.cc:1240] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:26.276988 12302 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:26.276168 11952 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.212774 12189 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:26.313252  6622 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:26.304421  6580 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:26.304543  6575 rpcz_store.cc:276] 0709 12:19:24.565936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.586535 (+ 20599us) service_pool.cc:224] Handling call
0709 12:19:24.586773 (+   238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:24.652084 (+ 65311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.232264 (+1580180us) spinlock_profiling.cc:243] Waited 64.6 ms on lock 0x7b54002670d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:26.232485 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":148596480}
I20260709 12:19:26.326555  6549 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:26.328876  6503 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 18. Current term is 19. Ops: []
W20260709 12:19:26.348846  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:26.314808 12223 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:26.350090 12287 consensus_queue.cc:237] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:26.350888 12335 log.cc:927] Time spent T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 12:19:26.351931  6580 raft_consensus.cc:2410] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
W20260709 12:19:26.341951  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:26.353402 12112 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:26.356453 12083 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:26.367843 12083 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:26.368309 12083 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:26.368559 12006 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:26.323414 12222 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:26.371017 12222 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:26.325057 12271 raft_consensus.cc:740] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:26.372290 12271 consensus_queue.cc:260] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:26.373831 12167 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:26.375136 12307 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:26.372521  6501 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:26.340261 12189 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:26.361014  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 47840.7us
I20260709 12:19:26.381390  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:26.384366  6622 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:26.382047 12165 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:26.383499  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:26.330721 12298 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:26.377458 12336 log.cc:927] Time spent T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 12:19:26.365097 12340 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:26.393702  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:26.380097 12287 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.412895  6622 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:26.414202  6550 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:26.341307 11952 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:26.415522  7274 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
W20260709 12:19:26.422148 12290 log.cc:927] Time spent T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 1.572s	user 0.031s	sys 0.054s
W20260709 12:19:26.423614 12266 log.cc:927] Time spent T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 1.975s	user 0.011s	sys 0.000s
I20260709 12:19:26.425757  6622 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:26.428946  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2423) took 1377 ms. Trace:
I20260709 12:19:26.429366  6649 rpcz_store.cc:276] 0709 12:19:25.051939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:25.918587 (+866648us) service_pool.cc:224] Handling call
0709 12:19:25.939673 (+ 21086us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:25.940042 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:25.940089 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:25.940509 (+   420us) log.cc:844] Serialized 26 byte log entry
0709 12:19:26.137618 (+197109us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:26.137676 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.137769 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:26.428720 (+290951us) raft_consensus.cc:1685] finished
0709 12:19:26.428741 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:26.428888 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":78,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":5531,"thread_start_us":181003,"threads_started":1,"wal-append.queue_time_us":271258}
I20260709 12:19:26.430110  6550 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:26.394174 12340 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:26.439297  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2668) took 1790 ms. Trace:
I20260709 12:19:26.416301 12298 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:26.440585 11913 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:26.460074 12345 log.cc:927] Time spent T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:19:26.447635 12240 raft_consensus.cc:2764] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:19:26.445832 12178 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:26.425729 12287 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:26.451728  7238 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:26.451491 12277 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:26.416576 12167 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:26.455190  6508 rpcz_store.cc:276] 0709 12:19:24.648570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.798506 (+149936us) service_pool.cc:224] Handling call
0709 12:19:24.798843 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:24.919131 (+120288us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:24.919177 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:24.919736 (+   559us) log.cc:844] Serialized 26 byte log entry
0709 12:19:25.094221 (+174485us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:25.277052 (+182831us) log.cc:844] Serialized 14 byte log entry
0709 12:19:25.330890 (+ 53838us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.438846 (+1107956us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54002c3dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:26.438904 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:26.438958 (+    54us) raft_consensus.cc:1685] finished
0709 12:19:26.439063 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:26.439230 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31042,"prepare.queue_time_us":20255,"prepare.run_wall_time_us":3053,"raft.queue_time_us":38127,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":13890,"spinlock_wait_cycles":276916736,"thread_start_us":128491,"threads_started":1,"wal-append.queue_time_us":763223}
I20260709 12:19:26.427795 12297 raft_consensus.cc:2804] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:26.458251 12182 consensus_queue.cc:1048] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:26.372605 12241 consensus_queue.cc:237] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:26.453822 12223 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:26.454293 11889 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
W20260709 12:19:26.461140 12348 log.cc:927] Time spent T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:19:26.457352 12340 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:26.506778  6653 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:26.448015  6473 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:26.461721 12209 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:26.458312 12319 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:26.470664 12318 consensus_queue.cc:1048] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 12:19:26.463531  6654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2422) took 1412 ms. Trace:
I20260709 12:19:26.568418  6654 rpcz_store.cc:276] 0709 12:19:25.051616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:25.910862 (+859246us) service_pool.cc:224] Handling call
0709 12:19:25.931873 (+ 21011us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:25.932257 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:25.932290 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:25.932768 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 12:19:25.993768 (+ 61000us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:25.993824 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:25.993890 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:26.463332 (+469442us) raft_consensus.cc:1685] finished
0709 12:19:26.463352 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:26.463479 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":441,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":119,"thread_start_us":36345,"threads_started":1,"wal-append.queue_time_us":230326}
I20260709 12:19:26.486181  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:26.489867  6550 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:26.486836  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:26.571621  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:26.582159  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:26.465010 12112 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:26.495398 12164 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:26.483502 11462 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:26.486559 12297 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:26.553452  6506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 676) took 3970 ms. Trace:
I20260709 12:19:26.628774  6506 rpcz_store.cc:276] 0709 12:19:22.583279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.298628 (+1715349us) service_pool.cc:224] Handling call
0709 12:19:24.317202 (+ 18574us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:24.472097 (+154895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.553067 (+2080970us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b5400396cd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:26.553375 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":354696960}
W20260709 12:19:26.553776  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 732798us
I20260709 12:19:26.630505  6473 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:26.465356 12307 consensus_queue.cc:1059] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:19:26.457068 12304 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:26.560285 11889 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:26.618373  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 132281us
W20260709 12:19:26.619978  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 130173us
I20260709 12:19:26.643458  6550 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:26.489313 12223 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:26.624266  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:26.631966  6473 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:26.650822  6473 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:26.651088  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:26.568565 12316 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:19:26.639786  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:26.554924 12319 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:26.506772 12167 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:26.625680 11913 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:26.625927 12255 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:26.658787 12316 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:26.635735 12297 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:26.635180 12303 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:26.653296 12223 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:26.647053 11952 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:26.574636 12209 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:26.659410 12352 log.cc:927] Time spent T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.192s	user 0.000s	sys 0.000s
I20260709 12:19:26.653035 11767 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:26.655092 12215 raft_consensus.cc:2764] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:19:26.660379 12298 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:26.659586 12319 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:26.670128 11913 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:26.672403 12255 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:26.672667 11462 consensus_queue.cc:1059] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 12:19:26.704008 12316 raft_consensus.cc:697] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:26.659942 12254 consensus_queue.cc:1059] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 9.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
W20260709 12:19:26.729828  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:26.724721 11913 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:26.747430  6474 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:26.742180  6651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3034) took 1332 ms. Trace:
I20260709 12:19:26.748631  6651 rpcz_store.cc:276] 0709 12:19:25.410283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.052743 (+642460us) service_pool.cc:224] Handling call
0709 12:19:26.053103 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:26.077103 (+ 24000us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.741877 (+664774us) spinlock_profiling.cc:243] Waited 23.2 ms on lock 0x7b54000744d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:26.742115 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53414656}
I20260709 12:19:26.752066  7271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38562 (request call id 937) took 3156 ms. Trace:
I20260709 12:19:26.726560 12255 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:26.765419  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 122724us
I20260709 12:19:26.752830  7271 rpcz_store.cc:276] 0709 12:19:23.586548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.586741 (+   193us) service_pool.cc:224] Handling call
0709 12:19:23.621675 (+ 34934us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:23.850158 (+228483us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:23.850202 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:23.850844 (+   642us) log.cc:844] Serialized 26 byte log entry
0709 12:19:24.301044 (+450200us) spinlock_profiling.cc:243] Waited 7.45 ms on lock 0x7b18002a7420. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:19:24.410121 (+109077us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:24.410180 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.742878 (+2332698us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b54005bd6d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:26.742951 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:26.743011 (+    60us) raft_consensus.cc:1685] finished
0709 12:19:26.743029 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:26.743184 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37460,"raft.queue_time_us":34705,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":11048,"spinlock_wait_cycles":464864384,"thread_start_us":108878,"threads_started":1,"wal-append.queue_time_us":623771,"wal-append.run_cpu_time_us":2204,"wal-append.run_wall_time_us":1005397}
I20260709 12:19:26.680625 12167 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:26.757782 12277 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:26.762127 12215 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:19:26.780680 12349 log.cc:927] Time spent T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.293s	user 0.001s	sys 0.001s
W20260709 12:19:26.789631  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 144015us
W20260709 12:19:26.785051 12351 log.cc:927] Time spent T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.315s	user 0.001s	sys 0.000s
I20260709 12:19:26.791429  7238 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:26.746346 12083 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:26.783226 12260 log.cc:927] Time spent T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 2.440s	user 0.003s	sys 0.017s
W20260709 12:19:26.790467  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 303674us
I20260709 12:19:26.780279 12287 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:26.789060 12298 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:26.805759 11462 raft_consensus.cc:2764] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:19:26.812112 12229 raft_consensus.cc:3055] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:26.812711 12229 raft_consensus.cc:740] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:19:26.815071  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 164306us
I20260709 12:19:26.802289  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:26.802663 12215 raft_consensus.cc:697] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 13 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:26.830909  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:26.831705  6504 raft_consensus.cc:2393] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 18: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:26.831792 12362 consensus_queue.cc:1059] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:26.833542  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:26.836133 12256 raft_consensus.cc:3055] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:26.836632 12256 raft_consensus.cc:740] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:26.844110  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:19:26.844553  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:26.844834  6506 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:26.846964 12164 raft_consensus.cc:3055] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:26.847381 12164 raft_consensus.cc:740] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:26.848170 12164 consensus_queue.cc:260] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.12, Last appended by leader: 12, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:26.850466 12153 raft_consensus.cc:3055] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:26.850852 12153 raft_consensus.cc:740] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
W20260709 12:19:26.854561 12346 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.472s	user 0.000s	sys 0.008s
I20260709 12:19:26.855299  7238 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:26.856042 12254 raft_consensus.cc:2804] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:26.856431 12254 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:26.856775 12254 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:26.819947 12354 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:26.862792 12354 raft_consensus.cc:697] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:26.856285  6506 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:19:26.863680 12354 consensus_queue.cc:237] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:26.864668 12271 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:26.864737  7238 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:26.867910 12307 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:26.869990  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 53993.1us
I20260709 12:19:26.864033 12254 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:26.872085 12263 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.872668 12263 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:26.875384 12263 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:26.878283  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:26.878811  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:26.879438  6578 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 10.
I20260709 12:19:26.880288  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:26.883675  6583 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:26.848664 12209 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:26.879354  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:26.890784  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:26.891152  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:26.891673  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:26.896062  6656 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:26.899401  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:26.849756 12164 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:26.910158  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:26.903731  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:26.913498  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:26.914160  6579 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:26.909241 12357 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:26.917089  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:26.917650  6654 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:26.918165  7271 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:19:26.894912  6650 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:26.905753  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:26.851668 12153 consensus_queue.cc:260] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:26.919739  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:26.921840  6577 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 15: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:26.922015  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:26.813834 12229 consensus_queue.cc:260] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:26.858063 12189 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:26.936259  6582 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 2475) took 2413 ms (client timeout 2492 ms). Trace:
I20260709 12:19:26.921659  6504 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:26.930881  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:26.931715  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:26.936856  6582 rpcz_store.cc:269] 0709 12:19:24.523258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.523392 (+   134us) service_pool.cc:224] Handling call
0709 12:19:26.936094 (+2412702us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b540032b254. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:26.936203 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":264866560}
I20260709 12:19:26.937840  6580 raft_consensus.cc:2393] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 16: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:26.934234  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:26.938218  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:26.938483  6577 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:26.934249  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:26.941262  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:26.795064 12316 consensus_queue.cc:237] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:26.953779  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:26.955188  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:26.955870  6584 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
W20260709 12:19:26.957125  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:26.817770 12298 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:26.945343  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:26.946484  6502 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2634) took 3450 ms. Trace:
I20260709 12:19:26.948051  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2420) took 2197 ms. Trace:
I20260709 12:19:26.964077  6655 rpcz_store.cc:276] 0709 12:19:24.750856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:25.345938 (+595082us) service_pool.cc:224] Handling call
0709 12:19:25.346255 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:25.962269 (+616014us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:26.947763 (+985494us) spinlock_profiling.cc:243] Waited 613 ms on lock 0x7b540008fcd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:26.947984 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1410443520}
I20260709 12:19:26.948601 11767 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:26.949333 12240 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.966435 11767 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:26.966504 12240 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:26.952174 12300 raft_consensus.cc:3055] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:26.969013 12300 raft_consensus.cc:740] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:26.969758 12221 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:26.969913 12300 consensus_queue.cc:260] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:26.970341 12221 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:26.971002 12300 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:26.952672 12360 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:26.972179 12360 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:26.973067 12229 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:26.837549 12256 consensus_queue.cc:260] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:26.959147  6650 raft_consensus.cc:1240] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:26.963142  6502 rpcz_store.cc:276] 0709 12:19:23.496379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.583357 (+ 86978us) service_pool.cc:224] Handling call
0709 12:19:23.583824 (+   467us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:23.584401 (+   577us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:19:23.806705 (+222304us) log.cc:844] Serialized 14 byte log entry
0709 12:19:24.071455 (+264750us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:24.087498 (+ 16043us) log.cc:844] Serialized 52 byte log entry
0709 12:19:24.087713 (+   215us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:24.088086 (+   373us) log.cc:844] Serialized 14 byte log entry
0709 12:19:24.089467 (+  1381us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:24.089551 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:26.674786 (+2585235us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:26.946242 (+271456us) raft_consensus.cc:1685] finished
0709 12:19:26.946271 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:26.946410 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9245,"prepare.queue_time_us":325863,"prepare.run_cpu_time_us":159,"prepare.run_wall_time_us":42139,"raft.queue_time_us":93618,"raft.run_cpu_time_us":345,"raft.run_wall_time_us":20582,"spinlock_wait_cycles":253936512,"thread_start_us":388570,"threads_started":2,"wal-append.queue_time_us":428656}
I20260709 12:19:26.930238 12209 consensus_queue.cc:1059] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 12:19:26.974411 12360 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:26.905093 12250 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 12:19:26.979115  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:26.977317  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:26.988286  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:26.960039  6656 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:26.982961  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:26.989219  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:26.963809  6654 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:19:26.990036  7266 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:19:26.991370  6583 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:26.992571  6623 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:27.002957  7238 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:26.997386  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:26.977975 12153 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:27.007048  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 212536us
I20260709 12:19:27.005455  6582 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:26.996470  6579 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:19:26.996057  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:27.011795  6654 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:27.017565  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:27.020563  6579 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:27.034811  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:27.037221  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:27.039873  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:27.040963  6656 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
W20260709 12:19:27.045785  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 70101.5us
I20260709 12:19:27.065478  6583 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:27.078409  6579 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:26.995716 12287 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.101086  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:27.102701  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:27.110913  6550 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:27.111402  6650 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:27.014865 12302 consensus_meta.cc:287] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.673s	user 0.015s	sys 0.001s
I20260709 12:19:27.006839 12277 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:27.007367 12359 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.112293  6550 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:27.112895  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:27.114394 12112 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:27.115039 12363 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:19:27.122048 12342 log.cc:927] Time spent T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.756s	user 0.002s	sys 0.000s
I20260709 12:19:27.125196  7271 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:27.126262 12240 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:27.126453  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:27.133745 11462 raft_consensus.cc:3055] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:27.136554 12250 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:27.137126 12360 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:27.137903 12189 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:27.139123 12167 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:27.141021 12222 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:27.141880  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:27.007751 12297 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:27.144811  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:27.198949  6502 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:27.201409  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:27.145967  6650 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:19:27.204023  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:27.204818  6650 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:27.205997  6650 raft_consensus.cc:1240] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:26.998250 12254 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:27.146935  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 100131us
I20260709 12:19:27.006848 12221 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:27.162220  6649 raft_consensus.cc:2393] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 16: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:27.032249 12209 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:27.167486  6654 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 17.
I20260709 12:19:27.107048  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2465) took 3612 ms. Trace:
I20260709 12:19:27.217823  6581 rpcz_store.cc:276] 0709 12:19:23.494350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:23.813101 (+318751us) service_pool.cc:224] Handling call
0709 12:19:23.989860 (+176759us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:23.990268 (+   408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:23.990297 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:23.990780 (+   483us) log.cc:844] Serialized 26 byte log entry
0709 12:19:24.081581 (+ 90801us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:24.081635 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:24.081720 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:26.423090 (+2341370us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:26.432800 (+  9710us) raft_consensus.cc:1685] finished
0709 12:19:26.432823 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:27.106770 (+673947us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b540034a8d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:27.106958 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1149,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"spinlock_wait_cycles":440427264,"thread_start_us":90415,"threads_started":1,"wal-append.queue_time_us":407074}
W20260709 12:19:27.188207  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:27.189385  7269 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:27.110077 12318 consensus_queue.cc:1059] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.092s
I20260709 12:19:26.978104 12215 consensus_queue.cc:237] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:27.208738  6650 raft_consensus.cc:1275] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:19:27.225003  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:27.210476  6550 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:27.192451  6579 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 12:19:27.233981  6579 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15. Ops: []
W20260709 12:19:27.234624  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:27.177809 12277 raft_consensus.cc:697] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:27.236776  6582 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:27.240079  6579 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:19:27.191038 11462 raft_consensus.cc:740] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:27.242821  6580 raft_consensus.cc:2393] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 17: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:27.244401  6581 raft_consensus.cc:1240] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:27.161541 12302 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:27.154052 12287 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:27.226367 12223 consensus_meta.cc:287] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.512s	user 0.010s	sys 0.000s
I20260709 12:19:27.195518 12364 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:19:27.255836  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 45439.1us
I20260709 12:19:27.256845  7274 raft_consensus.cc:1240] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:27.170795 12359 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:27.266502  6581 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 16. Current term is 17. Ops: []
W20260709 12:19:27.273015  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 270111us
W20260709 12:19:27.274140  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:27.213960 12360 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:27.264086 12223 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:27.278015  7238 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:27.202158 12250 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:27.258541 12364 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:27.235013 12167 consensus_queue.cc:1059] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:19:27.282168  6657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2411) took 2548 ms. Trace:
I20260709 12:19:27.299010  6657 rpcz_store.cc:276] 0709 12:19:24.734003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:24.867830 (+133827us) service_pool.cc:224] Handling call
0709 12:19:24.868141 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:26.221835 (+1353694us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:27.281861 (+1060026us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b54003dfed4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:27.282094 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3111064832}
I20260709 12:19:27.246948 11952 consensus_queue.cc:1059] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:19:27.274586 12232 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:27.310904 12298 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:27.312814  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:27.312889 12277 consensus_queue.cc:237] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:27.296993 12364 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:27.312693 12318 raft_consensus.cc:3055] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:27.309535 12302 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:27.352571 12316 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.354450  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3033) took 2390 ms. Trace:
I20260709 12:19:27.371302  6652 rpcz_store.cc:276] 0709 12:19:24.964383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:25.908168 (+943785us) service_pool.cc:224] Handling call
0709 12:19:25.908533 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:26.075124 (+166591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:27.354195 (+1279071us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b5400227ed4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:27.354387 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":379341696}
W20260709 12:19:27.355144  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:27.355805  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:27.361761  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:27.373859  6503 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
I20260709 12:19:27.362134  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:27.289870 12357 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:27.365470  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:27.375860  6500 raft_consensus.cc:3060] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:27.392452  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:27.413276  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:27.366328 12318 raft_consensus.cc:740] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:27.425352  6658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 893) took 2002 ms. Trace:
I20260709 12:19:27.429924  6658 rpcz_store.cc:276] 0709 12:19:25.423149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.152079 (+728930us) service_pool.cc:224] Handling call
0709 12:19:26.165082 (+ 13003us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:26.325251 (+160169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:27.425123 (+1099872us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b5400151154. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:27.425290 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":330835328}
I20260709 12:19:27.369527 12232 raft_consensus.cc:697] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:27.447235  6502 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:27.450314  6504 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:27.318907 11462 consensus_queue.cc:260] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:27.456213  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:27.405886 12221 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 12:19:27.396466 12316 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:27.452365 12366 log.cc:927] Time spent T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.277s	user 0.001s	sys 0.000s
I20260709 12:19:27.409181 12375 consensus_queue.cc:1059] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.073s
W20260709 12:19:27.496145  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 271174us
I20260709 12:19:27.459434 11952 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:27.501391  6501 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 18.
I20260709 12:19:27.514181  6622 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:27.503229 12300 consensus_meta.cc:287] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.532s	user 0.009s	sys 0.004s
I20260709 12:19:27.529225  6622 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:27.545854 12298 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:19:27.521438 12319 consensus_meta.cc:287] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.792s	user 0.009s	sys 0.003s
W20260709 12:19:27.527112 12255 consensus_meta.cc:287] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.747s	user 0.011s	sys 0.001s
I20260709 12:19:27.550460 12360 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:27.548964 12379 log.cc:927] Time spent T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 12:19:27.564689 12368 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:19:27.517037 12232 consensus_queue.cc:237] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:27.565245 12382 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:27.564049 12223 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:27.589767  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 292482us
I20260709 12:19:27.567339 12298 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:27.597882  6549 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:27.502804 12370 consensus_queue.cc:1048] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 12:19:27.571445 12287 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:27.509704 11952 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:27.627527 12223 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.589872 12362 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:27.593039 12269 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:27.648777 12269 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:27.622222 12120 consensus_queue.cc:1059] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:19:27.593284 12363 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:27.586951 12319 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:27.582074 12381 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:27.653252 12381 raft_consensus.cc:697] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:27.601555 12302 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:27.654309 12319 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:27.654248 12381 consensus_queue.cc:237] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:27.654379 12302 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:27.656737 12223 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:27.589408 12255 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:27.659489  6576 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:27.660772 12255 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:27.663002 12240 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.663611 12240 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:27.498868 12318 consensus_queue.cc:260] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:27.636101 12112 consensus_queue.cc:1059] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 12:19:27.622711 12382 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:27.629518 12373 log.cc:927] Time spent T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.325s	user 0.002s	sys 0.000s
I20260709 12:19:27.647850 12362 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:27.665297 12365 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:27.601752 12300 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:27.666123 12363 raft_consensus.cc:697] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:27.669561 12167 raft_consensus.cc:3055] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:19:27.676509 12376 log.cc:927] Time spent T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:19:27.582491 12359 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:27.624938 12371 log.cc:927] Time spent T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.200s	user 0.001s	sys 0.001s
I20260709 12:19:27.678259 12120 consensus_queue.cc:1243] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer f2b06aebe97249a4877d798b8118ece4 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 12:19:27.710412 12383 log.cc:927] Time spent T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 12:19:27.623899 12298 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:27.763660  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:27.808890 12221 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.809176 12242 raft_consensus.cc:3055] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:27.809927 12365 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:27.819391 12167 raft_consensus.cc:740] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:27.850281 12272 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:27.866912 12272 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:27.868919 12380 log.cc:927] Time spent T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.380s	user 0.003s	sys 0.000s
I20260709 12:19:27.869822 12361 raft_consensus.cc:3055] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:27.870451 12361 raft_consensus.cc:740] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:27.870235 12363 consensus_queue.cc:237] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:27.875851  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 416803us
I20260709 12:19:27.855217 12221 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:27.819016 12300 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:27.860646 12365 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:27.903642  6475 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:27.856462 12242 raft_consensus.cc:740] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:27.881111 12300 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:27.908296  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:27.901072 12215 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.000s	sys 0.017s
W20260709 12:19:27.908399  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 83141.9us
W20260709 12:19:27.916453  6548 leader_election.cc:336] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.638s (SENT)
W20260709 12:19:27.911320  6473 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.010s	sys 0.004s
W20260709 12:19:27.917718  6473 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.011s	sys 0.005s
W20260709 12:19:27.918196  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 559723us
W20260709 12:19:27.923692  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:27.924722  6548 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.721s (SENT) [suppressed 26 similar messages]
I20260709 12:19:27.908741 12272 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:27.910807 12287 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:27.925886 12242 consensus_queue.cc:260] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:27.926517 12287 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:27.927542 12242 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:27.934165  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 334953us
I20260709 12:19:27.911984 11462 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:27.936494  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 674254us
I20260709 12:19:27.906605 11767 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:27.912181 12083 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:27.922962 12215 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.000s	sys 0.018s
W20260709 12:19:27.962188  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:27.971537  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:27.977170  6577 raft_consensus.cc:1275] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 12:19:27.980103  6656 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.018s	sys 0.029s
W20260709 12:19:27.980856  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:27.972295  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:19:27.973702  6500 consensus_meta.cc:287] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.597s	user 0.011s	sys 0.025s
I20260709 12:19:27.982396  6500 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 17.
W20260709 12:19:27.981050  6656 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.018s	sys 0.029s
I20260709 12:19:27.976152  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:27.914600 12269 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:27.992745  6581 raft_consensus.cc:2393] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 17: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:27.990001  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:27.994911  6577 raft_consensus.cc:1275] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:19:27.998912  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:27.998790  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:27.974076 12359 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.011585  6584 raft_consensus.cc:2393] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 15: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:28.018427  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:28.020174  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:28.021525  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:28.032960  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:28.001348  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2496) took 1118 ms. Trace:
I20260709 12:19:28.003434  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:28.004736  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:27.871815 12361 consensus_queue.cc:260] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:28.009583  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:28.035602  6578 rpcz_store.cc:276] 0709 12:19:26.882781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.884979 (+  2198us) service_pool.cc:224] Handling call
0709 12:19:26.885359 (+   380us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:26.894205 (+  8846us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:26.894251 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:26.909937 (+ 15686us) log.cc:844] Serialized 52 byte log entry
0709 12:19:27.194755 (+284818us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:27.252747 (+ 57992us) log.cc:844] Serialized 14 byte log entry
0709 12:19:27.586039 (+333292us) spinlock_profiling.cc:243] Waited 4.53 ms on lock 0x7b18004265e0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d16c0a 00007f287c456219 00007f2872d9ea40 00007f2872d994e4 00007f2872d99455 00007f2875e81060 00007f2875e93716 00007f2875ea4287 00007f2875ea4208
0709 12:19:28.000315 (+414276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:28.000419 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:28.000468 (+    49us) raft_consensus.cc:1685] finished
0709 12:19:28.000570 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:28.000736 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11215,"prepare.queue_time_us":58974,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":345710,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":22236,"spinlock_wait_cycles":10417408,"thread_start_us":661901,"threads_started":2,"wal-append.queue_time_us":368849}
I20260709 12:19:27.984460 12223 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:28.000811 12307 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:27.979319 12178 consensus_queue.cc:1059] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:19:27.934006 12167 consensus_queue.cc:260] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:27.999832 12215 raft_consensus.cc:2764] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
W20260709 12:19:28.041983  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:28.043385  7269 raft_consensus.cc:2393] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 17: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:27.976200 12112 consensus_queue.cc:1048] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:19:27.984274 12250 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:28.080830  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:28.019505 12362 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:28.032107 12271 consensus_meta.cc:287] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 1.167s	user 0.011s	sys 0.008s
I20260709 12:19:28.091449  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2501) took 1180 ms. Trace:
W20260709 12:19:28.092885 11913 consensus_meta.cc:287] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 1.326s	user 0.009s	sys 0.010s
I20260709 12:19:28.094002 12395 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.018293 12165 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:28.096305 12303 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.067418 12319 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.019259 12382 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:28.090173 11952 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:28.030462 12384 log.cc:927] Time spent T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.261s	user 0.000s	sys 0.002s
I20260709 12:19:28.127406  6473 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:28.021059 12316 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:28.126248  6583 rpcz_store.cc:276] 0709 12:19:26.910760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.072157 (+161397us) service_pool.cc:224] Handling call
0709 12:19:27.072632 (+   475us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:27.118074 (+ 45442us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:27.118883 (+   809us) log.cc:844] Serialized 14 byte log entry
0709 12:19:27.916828 (+797945us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:27.917390 (+   562us) log.cc:844] Serialized 26 byte log entry
0709 12:19:27.917590 (+   200us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:27.917635 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:27.917728 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:28.091195 (+173467us) raft_consensus.cc:1685] finished
0709 12:19:28.091222 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:28.091372 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":269570,"prepare.queue_time_us":360,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":308959,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":163,"thread_start_us":572873,"threads_started":2,"wal-append.queue_time_us":434991}
I20260709 12:19:28.131224  6575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2494) took 1252 ms. Trace:
I20260709 12:19:28.126403  6584 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
W20260709 12:19:28.127029  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 223535us
I20260709 12:19:28.077000 12287 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:28.144949  6508 raft_consensus.cc:1275] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:28.090854 12240 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:28.025427 12359 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:28.098006  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 63339.4us
I20260709 12:19:28.097040 12255 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:28.107915 12302 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:28.125293 12221 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:28.205593  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 78266.7us
W20260709 12:19:28.237365 12229 consensus_meta.cc:287] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 1.264s	user 0.015s	sys 0.005s
I20260709 12:19:28.206120  6575 rpcz_store.cc:276] 0709 12:19:26.878768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.880435 (+  1667us) service_pool.cc:224] Handling call
0709 12:19:26.880792 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:26.881036 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:19:26.881425 (+   389us) log.cc:844] Serialized 14 byte log entry
0709 12:19:28.097471 (+1216046us) log.cc:844] Serialized 14 byte log entry
0709 12:19:28.130838 (+ 33367us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:28.130920 (+    82us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:28.130962 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:28.131043 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:28.131166 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":361831,"prepare.queue_time_us":177,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":356894,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":5870,"spinlock_wait_cycles":142720,"thread_start_us":873000,"threads_started":3,"wal-append.queue_time_us":476086}
I20260709 12:19:28.208742 12221 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:28.130478 12250 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:28.208935 12386 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.202999 12319 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:28.233165 12404 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:28.197284 12303 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:28.180285 12167 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:28.208721  6475 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:28.207643  6580 raft_consensus.cc:1275] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 12:19:28.201678 12164 consensus_meta.cc:287] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 1.272s	user 0.012s	sys 0.006s
W20260709 12:19:28.201900 12340 consensus_meta.cc:287] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 1.634s	user 0.021s	sys 0.002s
W20260709 12:19:28.201526 12153 consensus_meta.cc:287] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 1.186s	user 0.013s	sys 0.005s
I20260709 12:19:28.090919 12307 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:28.173128 12361 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:28.240267 12371 log.cc:927] Time spent T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:19:28.260072 12250 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:28.284181  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2494) took 1360 ms. Trace:
I20260709 12:19:28.284614  6508 rpcz_store.cc:276] 0709 12:19:26.923462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.923620 (+   158us) service_pool.cc:224] Handling call
0709 12:19:26.923951 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:28.208786 (+1284835us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:28.283934 (+ 75148us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b540001bdd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:28.284108 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2806911872}
I20260709 12:19:28.285101 12250 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:28.201939 12402 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.286131 12402 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:28.067662 11767 consensus_queue.cc:1059] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.115s
I20260709 12:19:28.289412 11767 raft_consensus.cc:3055] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:19:28.289657 12364 consensus_meta.cc:287] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.909s	user 0.007s	sys 0.010s
I20260709 12:19:28.289778 11767 raft_consensus.cc:740] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:28.290140 12364 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:28.290673 11767 consensus_queue.cc:260] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:28.296020 12402 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:28.238124 12362 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:28.296743 12359 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:28.296785 12362 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:19:28.299259 12362 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:28.181272 12395 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:28.263105 12319 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:28.259050  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 286872us
I20260709 12:19:28.306738  6622 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:19:28.242231  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:28.265450  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:28.267864  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 58468.8us
I20260709 12:19:28.251986 12304 raft_consensus.cc:3055] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:28.208004 12255 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:28.241189 12316 consensus_queue.cc:1059] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 12:19:28.273324  6473 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
W20260709 12:19:28.276647  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:28.264115 12386 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:28.151612 11913 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:28.292047 11767 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:28.300964  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 92323us
I20260709 12:19:28.321437  6581 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 9 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 12:19:28.311160 12252 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:28.313555  6622 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:19:28.315593  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:28.322865  7266 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:28.323920 11913 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:28.266817 12404 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:28.323858  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 50636.8us
I20260709 12:19:28.335306 12255 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:28.330296 12272 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:28.324913  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:28.340426  7273 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:28.341809 12252 raft_consensus.cc:697] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:28.342736 12252 consensus_queue.cc:237] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:28.277072 12340 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:28.344115  6653 raft_consensus.cc:1240] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:28.335182  6473 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:28.345373 12340 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:28.345916  7266 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:19:28.325284  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:28.346529  6473 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:28.328722 12406 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:28.347826 12406 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:28.350168 12406 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:28.292228 12364 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:28.330283  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:28.333992  6501 raft_consensus.cc:1275] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:19:28.351127  6508 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:19:28.336203 12272 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:28.352165 12272 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:28.352370 12222 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:28.332950 12316 raft_consensus.cc:3055] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:28.336683 11897 raft_consensus.cc:3055] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:28.335623 12409 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.353300 12316 raft_consensus.cc:740] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:28.337510  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:28.327491 12303 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:28.354286  6502 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:28.353196 12222 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:28.338390 12412 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:28.312486 12304 raft_consensus.cc:740] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:28.326706 12215 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.353394 11897 raft_consensus.cc:740] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:28.356468 12304 consensus_queue.cc:260] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:28.357088 11897 consensus_queue.cc:260] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:28.357700 12304 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:28.358035 11897 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:28.353890 12409 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:28.360764 12409 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:28.361351  6505 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:19:28.362850  6509 raft_consensus.cc:3055] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:28.363205  6509 raft_consensus.cc:740] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:28.363876  6509 consensus_queue.cc:260] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:28.364482 12083 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.365031 12083 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:28.366066  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:28.366319 11897 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:28.366606 12413 log.cc:927] Time spent T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.100s	user 0.000s	sys 0.002s
I20260709 12:19:28.357957  6473 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:28.366787 11897 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:28.367295 12083 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:28.368597 12397 raft_consensus.cc:3055] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:28.368698  6509 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:28.369303  6582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 2381) took 1445 ms. Trace:
I20260709 12:19:28.408567  6582 rpcz_store.cc:276] 0709 12:19:26.924076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:27.237850 (+313774us) service_pool.cc:224] Handling call
0709 12:19:27.257720 (+ 19870us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:28.007419 (+749699us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:28.007452 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:28.038402 (+ 30950us) log.cc:844] Serialized 26 byte log entry
0709 12:19:28.202283 (+163881us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:28.271421 (+ 69138us) log.cc:844] Serialized 14 byte log entry
0709 12:19:28.271655 (+   234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:28.271730 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:28.366077 (+ 94347us) spinlock_profiling.cc:243] Waited 1.3 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:28.369012 (+  2935us) raft_consensus.cc:1685] finished
0709 12:19:28.369104 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:28.369246 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":336358,"prepare.queue_time_us":59378,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":332044,"raft.run_cpu_time_us":907,"raft.run_wall_time_us":34811,"spinlock_wait_cycles":2995584,"thread_start_us":553425,"threads_started":3,"wal-append.queue_time_us":175438}
I20260709 12:19:28.370328 12360 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:28.316658  7274 raft_consensus.cc:1275] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:19:28.321650  6475 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:28.354532 12316 consensus_queue.cc:260] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:28.372479  6473 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:28.372781 12404 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:28.325263  6504 raft_consensus.cc:1275] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:28.375138 12395 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:28.337517  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:19:28.445168  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 76935us
I20260709 12:19:28.446000  6473 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:28.377911  6502 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
W20260709 12:19:28.313553 12384 log.cc:927] Time spent T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 12:19:28.380473  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:28.350675  7273 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38546 (request call id 647) took 1368 ms. Trace:
I20260709 12:19:28.449648  7273 rpcz_store.cc:276] 0709 12:19:26.982375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.982648 (+   273us) service_pool.cc:224] Handling call
0709 12:19:28.350495 (+1367847us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54005816d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:28.350605 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3019207552}
I20260709 12:19:28.382157  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:28.356631 12215 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:28.355070 12256 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:28.394145 12374 log.cc:927] Time spent T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 1.013s	user 0.005s	sys 0.011s
I20260709 12:19:28.333817 12307 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:28.401772  6653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 898) took 1847 ms. Trace:
W20260709 12:19:28.452301  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 114855us
I20260709 12:19:28.453452  6653 rpcz_store.cc:276] 0709 12:19:26.554783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.554910 (+   127us) service_pool.cc:224] Handling call
0709 12:19:26.555228 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:28.344658 (+1789430us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:28.401495 (+ 56837us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b54000b7554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:28.401703 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4113518336}
I20260709 12:19:28.405746  6500 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:28.454428  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:19:28.414724  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:28.443856  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 122280us
I20260709 12:19:28.448271  6473 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:28.452351  7269 raft_consensus.cc:2410] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 12 because replica has last-logged OpId of term: 10 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 11.
I20260709 12:19:28.459596  6509 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 8. (term mismatch)
I20260709 12:19:28.460264  6509 pending_rounds.cc:85] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 7
I20260709 12:19:28.460608  6509 pending_rounds.cc:107] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 12:19:28.460932  6509 raft_consensus.cc:2889] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:28.383857 12272 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:28.476013  6622 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:28.317836 12381 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:28.407298 12359 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:28.496556  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:28.509990  6509 raft_consensus.cc:1240] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 17. Current term is 18. Ops: []
W20260709 12:19:28.505290  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 57070.1us
I20260709 12:19:28.502642  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:28.483107  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:28.526952  6652 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:19:28.527313  7269 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:19:28.528745  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:28.504812  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 48011us
I20260709 12:19:28.413664 12386 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:28.483551  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:28.441743 12167 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:28.503072  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 192538us
I20260709 12:19:28.504796  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:28.529299  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:28.451480 12409 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:28.481528  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:28.481281  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:28.545706  6655 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:28.547130  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:28.530941  7273 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 10 index: 12. Preceding OpId from leader: term: 11 index: 12. (term mismatch)
I20260709 12:19:28.547784  7273 pending_rounds.cc:85] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9: Aborting all ops after (but not including) 11
I20260709 12:19:28.548035  7273 pending_rounds.cc:107] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9: Aborting uncommitted NO_OP operation due to leader change: 10.12
I20260709 12:19:28.531591  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:28.532157  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:28.502918  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:28.480386  6653 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:19:28.483858  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:28.417317 12397 raft_consensus.cc:740] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:28.536921  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:28.537289  6501 raft_consensus.cc:1240] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:28.537585  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:28.538928  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:28.539111  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:28.441782 12406 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:28.540066  6651 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:19:28.537286  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:28.541378  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:28.530395  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:28.542026  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:28.453291 12370 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:28.473160 12277 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:28.543578  6575 raft_consensus.cc:2393] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 15: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:19:28.543663  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:28.547065  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:28.486483  6577 raft_consensus.cc:1275] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:28.501662 12189 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:19:28.577909  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:28.643127  6658 raft_consensus.cc:2393] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 15: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:28.575227  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:28.643851  6583 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:28.645377  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:28.649946  6500 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:28.650915  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:28.644613  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:28.644007  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:28.644603  6654 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:28.652632  6501 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:19:28.653349  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:28.645145  6657 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 12.
W20260709 12:19:28.558516  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:28.653975  6501 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:28.546240  6650 raft_consensus.cc:2393] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 17: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:28.548269  7273 raft_consensus.cc:2889] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:28.654609  7240 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:28.574565  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:28.548314  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:28.639899  6622 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:28.646435  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:28.655881  6509 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:19:28.473124 12222 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:28.401845 12365 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:28.657565 12390 consensus_queue.cc:1059] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:28.501413 12263 consensus_queue.cc:1048] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.157s
I20260709 12:19:28.659152  6576 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 19: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:28.548604 12167 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:28.660192 12397 consensus_queue.cc:260] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:28.661374 12397 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:28.661547 12221 consensus_queue.cc:1059] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:28.662223 12167 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:28.662768  6507 raft_consensus.cc:2393] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 15: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:28.663968 12390 raft_consensus.cc:3055] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:28.538751 12250 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.126s
I20260709 12:19:28.647970  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:28.669291  6655 raft_consensus.cc:2393] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 18: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:19:28.622135  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 69713.8us
I20260709 12:19:28.644683  6504 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:28.503616 12354 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:28.542425 12319 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:28.574126 11913 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:28.706786  6501 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:28.708046  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:28.708847  6501 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:28.709609  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:28.710564  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:28.711652  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:28.504211 12316 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:28.664372 12390 raft_consensus.cc:740] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:19:28.676735  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:28.652184  6502 raft_consensus.cc:2393] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 18: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:28.723292  6549 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:28.688263  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:28.501924 12358 consensus_queue.cc:1048] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.148s
I20260709 12:19:28.686627  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:28.683808 12410 log.cc:927] Time spent T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.270s	user 0.000s	sys 0.000s
I20260709 12:19:28.683339 12112 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.747148 12112 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:28.679910 12164 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.748934 12164 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:28.750308  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:28.752064 12215 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:28.647584 12409 raft_consensus.cc:697] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:28.753296 12390 consensus_queue.cc:260] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:28.753726 12409 consensus_queue.cc:237] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:28.754704 12390 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:28.762952 12297 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:28.650714 12277 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:28.540853 12359 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:28.774581  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:28.715126  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 75355.8us
I20260709 12:19:28.776142  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:28.717209  6500 raft_consensus.cc:1240] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:19:28.778119  6500 raft_consensus.cc:1275] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:19:28.718021  6501 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:28.720001  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:28.720975  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:28.725982  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:28.705300  6506 raft_consensus.cc:2393] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 17: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:28.574755 12360 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:28.732020  6502 raft_consensus.cc:1240] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:19:28.785487  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:28.786034  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:28.787292  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:28.733551  6657 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:28.722535 12319 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:28.727990  6473 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:28.744459  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:28.791150  6550 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:28.791604  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 63747.4us
I20260709 12:19:28.623119 12415 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.765765  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:28.624024 12083 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.765447 12263 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:28.767060 12404 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:28.529675 12307 consensus_queue.cc:1059] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.207s
I20260709 12:19:28.705641  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:28.813287  6582 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:28.813777  6509 raft_consensus.cc:2393] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 14: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:28.769848  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:28.550055 12381 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:28.774123  6651 raft_consensus.cc:1240] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:19:28.648270 12189 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:28.821707 12189 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:28.790020  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:28.790550  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:28.745226 12420 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:28.825414 12420 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:28.825837 12420 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:28.792071  6550 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:28.504438 12412 consensus_queue.cc:1048] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.271s
I20260709 12:19:28.791675  6578 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:19:28.795255  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:28.796607  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:28.714244 11913 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:28.793464 12358 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:28.812664  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:28.841734  6506 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:19:28.814301  6653 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:19:28.775533 12277 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:28.817121 12361 consensus_meta.cc:287] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.513s	user 0.010s	sys 0.000s
I20260709 12:19:28.694001 12406 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:28.820840 12381 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:28.793484 12340 consensus_queue.cc:1048] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 12:19:28.832119  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:28.832549  6658 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
W20260709 12:19:28.689013 12298 consensus_meta.cc:287] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.826s	user 0.005s	sys 0.006s
I20260709 12:19:28.834093  6550 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:28.832121  6576 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:28.795965 12395 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:28.797191 12215 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:28.801467 12263 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:28.798439 12360 raft_consensus.cc:697] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 17 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:28.800763 12281 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:28.835806 12229 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.852085 11897 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:28.843752 12399 consensus_queue.cc:1048] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 12:19:28.851771  7271 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:28.852272 12307 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:28.839815 12256 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:28.837045 12404 raft_consensus.cc:697] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:28.845590 12222 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:28.808307 12304 consensus_queue.cc:1048] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:19:28.815006 12083 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:28.800760 12415 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:28.857622 12395 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:28.861649 12272 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:28.864289 12263 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:28.860687 12215 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:28.863374 12398 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:28.850762 12358 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:28.858719 12298 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:28.897239 12340 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:28.898451 12083 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:28.812172  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.004422  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:28.849589  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:29.070528  6474 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:29.071177  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:29.013053  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.071832 12221 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:29.073119 12221 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:29.074892 12271 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.075304 12272 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.013576  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:29.077250 12221 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:29.078315  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:29.014400  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:28.857810  6578 raft_consensus.cc:2393] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 17: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:19:29.081035  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.082437  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:28.838538  6582 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:19:29.075886 12271 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.013072  6550 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:29.087379  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:29.020484  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:29.020381  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:29.088887 11897 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:29.089459 11897 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:29.089857 11897 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:29.090435  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:29.053694 11913 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:29.055956  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:28.908499 12358 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:28.874651 12256 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:28.875859 12281 raft_consensus.cc:697] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:29.135124 12256 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:29.061764  6650 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 12:19:29.137895 12297 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:29.064106  6502 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:19:28.909843 12215 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:28.829367  6657 raft_consensus.cc:1275] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:19:29.143244  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:29.087598 12222 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.144812 12222 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:29.146991  7238 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.066s	sys 0.041s
I20260709 12:19:29.147116  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:19:29.147552  7238 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.066s	sys 0.041s
I20260709 12:19:29.154170 12277 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.156317  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:29.157475  7268 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:29.159876  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:29.148138  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 605238us
I20260709 12:19:29.160413 12281 consensus_queue.cc:237] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.163518  7271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38558 (request call id 820) took 1154 ms. Trace:
I20260709 12:19:29.164745  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:29.165632  6502 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:19:29.166721  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:29.161239  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:29.088436  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 229972us
I20260709 12:19:28.871373 12229 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:28.877175 12307 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:29.176246 12307 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:29.176469 12277 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:28.946928 12319 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:28.919840 12340 raft_consensus.cc:697] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:29.178511 12153 raft_consensus.cc:3055] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:29.179057 12153 raft_consensus.cc:740] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:29.178582 12340 consensus_queue.cc:237] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:28.936877 12359 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:28.926622 12360 consensus_queue.cc:237] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:29.191376  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 384028us
I20260709 12:19:28.895052 12395 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:29.085330  6578 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:19:29.168268 12263 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:29.164000  7271 rpcz_store.cc:276] 0709 12:19:28.008903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.009364 (+   461us) service_pool.cc:224] Handling call
0709 12:19:29.163107 (+1153743us) spinlock_profiling.cc:243] Waited 773 ms on lock 0x7b5400507dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:29.163306 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1778610048}
I20260709 12:19:28.938232 12381 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:28.909147 12398 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:28.945322 12404 consensus_queue.cc:237] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:28.898770 12406 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:28.956489 12420 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:29.051522 11767 consensus_meta.cc:287] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.667s	user 0.009s	sys 0.009s
I20260709 12:19:29.072255  6509 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:29.090505  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:29.087291 12271 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:29.112221  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:29.087191 12164 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:29.121033  6658 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:29.229653  6473 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:29.128769  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:29.131637  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:29.092186 12272 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.115679 12303 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:29.147897 12222 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:29.167966  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:29.250583  6502 raft_consensus.cc:2393] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 18: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:29.172956  6657 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 20: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:29.187415  6506 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:29.188575  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:29.188927 12361 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.189646  6474 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:29.257737  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:29.258548  6500 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:19:29.259527  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:29.197635 11897 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.201924  6623 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:19:29.211879  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.189817 12316 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 12:19:29.176118 12229 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:29.190635  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:29.198764  6550 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:29.198907  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:29.222601  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:29.223268 12420 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:29.274379 12381 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:29.274714  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:29.202309 12252 raft_consensus.cc:3055] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:29.275568 12252 raft_consensus.cc:740] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:29.275812  6509 raft_consensus.cc:2393] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 15: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:29.276998  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:29.276937 12252 consensus_queue.cc:260] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.277935  6509 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:29.278455 12252 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:29.225253  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:29.281036  6582 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:29.282442  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:29.199956 12189 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.201865 12297 consensus_queue.cc:1059] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:19:29.210592 12398 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:29.284555 12297 raft_consensus.cc:3055] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:29.285023 12297 raft_consensus.cc:740] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:29.285889 12189 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:29.203215  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:29.286062 12297 consensus_queue.cc:260] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.287602  6578 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:19:29.289145  6654 raft_consensus.cc:2393] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 15: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:29.289497  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:29.290292  6582 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:19:29.227852  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:29.212772  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:29.292701  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.293184  6504 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 15: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:29.294214  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:29.294800 11897 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:29.294883  6504 raft_consensus.cc:2376] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17. Re-sending same reply.
I20260709 12:19:29.295833 12318 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:29.210136  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:29.248512 12303 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:29.260308  6500 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
W20260709 12:19:29.213359  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 422661us
I20260709 12:19:29.308207  7274 raft_consensus.cc:2393] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 15: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:29.256829  7272 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:29.296041  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:29.297926  6577 raft_consensus.cc:2393] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 18: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:29.288130 12297 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:29.302997 12256 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.305296 12272 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:29.279047  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:29.311340  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 109654us
W20260709 12:19:29.312985  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.311476  6504 raft_consensus.cc:2393] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 14: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:29.211089 12263 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:29.256829 12361 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:29.316032  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 126539us
I20260709 12:19:29.310745 12303 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:29.322820  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 124268us
W20260709 12:19:29.332957  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 103628us
I20260709 12:19:29.334939  6473 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:29.336700 12298 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:29.336866  7271 raft_consensus.cc:1240] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:19:29.319710 12222 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:29.338188  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:29.330808  6505 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:29.345851  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:29.350378  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:29.338665 12272 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:29.353505  6506 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:19:29.353259  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:29.351904  6474 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:29.355692 11767 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.212997 12153 consensus_queue.cc:260] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:29.357540  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:29.357745 12400 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:29.358367  6577 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:29.359583  6622 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:29.360440  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.364856 11767 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:29.366649 12400 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:29.367091 12400 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:29.367470 12400 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:29.367494 11767 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:29.339507 11897 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:29.339964 12222 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:29.369314 12222 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:29.359918 12215 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.374907  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:29.377827 12215 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:29.380193  6582 raft_consensus.cc:1240] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:29.380523  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:29.381263  7274 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:29.381529  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:29.382748  6582 raft_consensus.cc:2410] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:19:29.384725  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:29.385004 12153 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:29.385566  7271 raft_consensus.cc:2393] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 15: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:29.387665 12215 raft_consensus.cc:2764] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:19:29.387980 12365 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:29.388576 12395 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.391093 12395 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:29.392360  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:19:29.387780  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 45376.1us
I20260709 12:19:29.394802  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:29.395536  6582 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:29.397316  6474 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:19:29.399565 12437 log.cc:927] Time spent T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:19:29.400784  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:29.403924  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:29.404649  6577 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 12. (term mismatch)
W20260709 12:19:29.405128  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.406150  6508 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 15: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:29.406242  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:29.410202  7274 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:19:29.397183 11897 consensus_queue.cc:1059] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 12:19:29.405005  6577 pending_rounds.cc:85] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 11
W20260709 12:19:29.415854  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 51959us
I20260709 12:19:29.413681  6473 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:29.412962 12386 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:29.418629  6500 raft_consensus.cc:2393] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 17: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:29.421074 12120 raft_consensus.cc:3055] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:29.421489 12120 raft_consensus.cc:740] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:29.422662 12120 consensus_queue.cc:260] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:29.424057 12120 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:29.418902 12303 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:29.426034  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2514) took 1402 ms. Trace:
I20260709 12:19:29.426443  6581 rpcz_store.cc:276] 0709 12:19:28.023027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.079486 (+ 56459us) service_pool.cc:224] Handling call
0709 12:19:28.097956 (+ 18470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:28.321836 (+223880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:29.418302 (+1096466us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b5400265cd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:29.425843 (+  7541us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":513208064}
I20260709 12:19:29.426895  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:29.427753  6583 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 11.
I20260709 12:19:29.429196  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:29.430981  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:29.419831 12386 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:29.436892 12386 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:29.436964 12112 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:29.424005  6550 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:29.437907 12112 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:29.438333 12112 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:29.438745  7271 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:19:29.439150 12112 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:29.441020  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:29.438143  6550 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:29.415998  6577 pending_rounds.cc:107] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 12.12
I20260709 12:19:29.443418  6577 raft_consensus.cc:2889] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:29.427979  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:29.449448  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:29.449990 12415 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:29.450254  6502 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:29.450964  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:29.451351  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:29.451747  6658 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:29.452370  6576 raft_consensus.cc:2376] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16. Re-sending same reply.
I20260709 12:19:29.452953  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:29.453322  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:29.453714  6658 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:19:29.453894  6576 raft_consensus.cc:1240] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:29.454744  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:29.455427  6658 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:29.455644  6576 raft_consensus.cc:1240] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:29.439062  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:29.457273  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:29.458612  6581 raft_consensus.cc:2410] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:19:29.427248 12303 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:29.458268  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:29.438578 12256 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:29.469429  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.469286  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:29.476524  6650 raft_consensus.cc:1240] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:29.476686  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:29.477524  7267 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:29.479773  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:29.454763 12421 log.cc:927] Time spent T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.684s	user 0.002s	sys 0.000s
W20260709 12:19:29.481912  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.482245  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:29.482512  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.483186  6549 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:29.484404  6500 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 13 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:29.489903 12400 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.494973  6655 raft_consensus.cc:1240] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:29.497432  6650 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:29.498572 12215 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:29.464433 12360 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:29.500828 12360 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:29.501319 12360 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:29.504653 12395 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:29.505191 12395 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:29.505592 12395 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:29.509490  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:29.524621  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:29.525257  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.527215  7267 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:19:29.528751  7272 raft_consensus.cc:1240] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:29.529527  6474 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:29.530985 12215 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:29.531770 12215 raft_consensus.cc:697] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:29.533064 12215 consensus_queue.cc:237] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:29.535210  6651 raft_consensus.cc:1240] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:19:29.535457 11897 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:29.536278 11897 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:29.510324  6623 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:29.539636 12340 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:29.513751  6578 raft_consensus.cc:1275] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:19:29.515484 12223 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:29.535243  6577 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:29.542376 12223 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:29.543234 12223 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:29.515888  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 102407us
I20260709 12:19:29.516188  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3066) took 1032 ms. Trace:
I20260709 12:19:29.517691  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:29.522203 12400 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:29.510129  6655 raft_consensus.cc:1240] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 12:19:29.541873  6650 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 12:19:29.544804  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 147596us
I20260709 12:19:29.523929 12395 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:29.571597  6508 raft_consensus.cc:1275] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:19:29.577487  7238 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:29.580427  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:29.581349  7269 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 19: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:29.582710  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:29.583523 12395 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:29.584235 11462 raft_consensus.cc:2764] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:19:29.584511 12354 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:29.583325 12412 consensus_queue.cc:1059] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:29.584986 12354 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:29.585427 12354 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:29.585768 12412 raft_consensus.cc:3055] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:29.586462 12412 raft_consensus.cc:740] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:29.566815 12120 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:29.584302  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:29.593817  6473 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:29.600593 12395 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.601339 12395 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.603893 12395 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:29.608388  6658 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 14.
I20260709 12:19:29.609669  7240 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:29.610687 11462 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:29.611287 11462 raft_consensus.cc:697] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 14 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
W20260709 12:19:29.613252  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 44669.5us
I20260709 12:19:29.614106  6473 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:29.615075 11767 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:29.615335  6473 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:29.545459  6649 rpcz_store.cc:276] 0709 12:19:28.483834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.494177 (+ 10343us) service_pool.cc:224] Handling call
0709 12:19:28.536977 (+ 42800us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:28.729458 (+192481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:28.729505 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:29.126222 (+396717us) spinlock_profiling.cc:243] Waited 31.5 ms on lock 0x7b2c0010c388. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:19:29.127119 (+   897us) log.cc:844] Serialized 26 byte log entry
0709 12:19:29.272486 (+145367us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:29.310574 (+ 38088us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:29.515732 (+205158us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b54000b5c54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:29.515788 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:29.515833 (+    45us) raft_consensus.cc:1685] finished
0709 12:19:29.515847 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:29.516009 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83286,"raft.queue_time_us":67509,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":105,"spinlock_wait_cycles":359048832,"thread_start_us":142392,"threads_started":1,"wal-append.queue_time_us":218515}
I20260709 12:19:29.616106 12229 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:29.616607 12229 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:29.574172 12303 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.616741  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:29.615718 11767 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:29.617322 12388 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:29.617249 12303 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:29.617920 12388 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:29.617019 12229 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:29.618469 12388 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:29.619146 11767 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:29.619491 12303 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:29.620045 12303 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:29.593334 12256 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.620857 12256 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.621464  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:29.616983 11462 consensus_queue.cc:237] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:29.617417  7269 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:29.628485  6501 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:19:29.629518  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:29.631026  6582 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:19:29.632632  6652 raft_consensus.cc:2410] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 12:19:29.594174 12120 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:29.638237 12272 consensus_queue.cc:1059] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
W20260709 12:19:29.642988  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 58877.5us
W20260709 12:19:29.643567  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.623971 12303 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:29.588145 12412 consensus_queue.cc:260] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:29.652245  6579 raft_consensus.cc:2410] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:19:29.654878 12167 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:29.655409 12167 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:29.655850 12167 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:29.658607 12222 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:29.644598 12120 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:29.647224 12303 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:29.667752  7238 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:29.664202 12318 consensus_queue.cc:1048] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 12:19:29.672632  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:29.677405 12272 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 12:19:29.683387  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58980 (request call id 3068) took 1164 ms. Trace:
I20260709 12:19:29.696918  6652 rpcz_store.cc:276] 0709 12:19:28.518761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:28.527766 (+  9005us) service_pool.cc:224] Handling call
0709 12:19:29.683095 (+1155329us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54002a58d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:29.683302 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2526972800}
I20260709 12:19:29.672745 12412 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:29.674674 12256 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:29.705277  6548 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:29.701236  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:29.708205 12281 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:29.708652 12281 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:29.708978 12281 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:29.708948  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:29.709781 12407 consensus_queue.cc:1059] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:29.710604  6649 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 14.
I20260709 12:19:29.710955 12272 consensus_queue.cc:1059] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:29.711906  7240 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:29.712777 12415 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:29.713155 12415 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:29.713524 12415 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:29.714258 12388 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.702390 12386 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:29.702082 12361 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:29.700367 12222 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:29.700745 12318 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:29.731462  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 68367.1us
I20260709 12:19:29.721334 12153 raft_consensus.cc:2804] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:29.679294 12112 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.731467 12363 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:29.744817 12388 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:29.755312 12272 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:29.764817  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:29.764979 12167 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.767015 12223 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.768953 12167 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:29.688620 12229 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.766678 12112 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:29.771034 12229 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:29.771780  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 128812us
I20260709 12:19:29.772249 12398 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.773574  6576 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:29.773767  7238 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:29.775067 12361 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:29.775559 12361 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:29.775957 12361 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:29.776041 12398 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:29.776746  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:29.779003  6582 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
W20260709 12:19:29.775143  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.778272  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:29.777522  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:29.780655  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:29.781098  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:29.781435  6577 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 19: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:29.781816  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:29.782356  6582 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:29.782842 12222 raft_consensus.cc:3055] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:29.783270 12222 raft_consensus.cc:740] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:29.784318  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:29.784222 12222 consensus_queue.cc:260] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.785009  6508 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:19:29.785560 12222 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:29.782291  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:29.787582  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:29.788064  6577 raft_consensus.cc:2393] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 15: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:29.788725  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:29.790055  6578 raft_consensus.cc:1275] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 12:19:29.790477  6578 pending_rounds.cc:85] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 5
I20260709 12:19:29.769344 12406 raft_consensus.cc:3055] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:29.790685  6578 pending_rounds.cc:107] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 12:19:29.792270  6578 raft_consensus.cc:2889] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:29.769178 12223 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:29.795174  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:29.796891  6582 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
W20260709 12:19:29.799988  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:29.803617  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.811085  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:29.811785  6509 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:29.812737  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:29.815267  6578 raft_consensus.cc:1275] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:29.820500  6581 raft_consensus.cc:1240] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:29.727222 12354 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.821767  6583 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
W20260709 12:19:29.825395  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 111850us
I20260709 12:19:29.792187 12406 raft_consensus.cc:740] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:19:29.838815  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.794343 12153 raft_consensus.cc:697] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 17 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:29.841354 12223 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.861640  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:29.862440  6508 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:29.827287 12281 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.926556 12281 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:29.838236 12318 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.928895 12318 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:29.868721 12364 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.869433 12360 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.870822  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:29.872706  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:29.873517  6504 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:29.874356 12386 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:29.876673  6501 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
new_leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:58944
I20260709 12:19:29.878007  6473 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:29.879282 12252 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:29.860625 12167 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.881294  6654 raft_consensus.cc:1275] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 12:19:29.936506  6654 pending_rounds.cc:85] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 9
I20260709 12:19:29.936738  6654 pending_rounds.cc:107] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 12:19:29.934463  6501 raft_consensus.cc:606] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 13 LEADER]: Received request to transfer leadership to TS 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:29.936951  6654 raft_consensus.cc:2889] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:29.882836 11897 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:29.885248 12361 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:29.886605  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:29.893748 12354 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:29.893890 12316 raft_consensus.cc:3055] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:29.901759  6508 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:19:29.940034  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:19:29.902933  6548 leader_election.cc:336] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.567s (SENT)
I20260709 12:19:29.906002  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:29.947674  7268 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:29.908962  6502 raft_consensus.cc:1240] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:29.909725 12406 consensus_queue.cc:260] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.914408 12256 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.953065  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:29.953240  6509 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:29.915423 12303 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.920790 12153 consensus_queue.cc:237] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:29.958364  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:29.924538  6505 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:29.962754  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:29.940519  6580 raft_consensus.cc:2393] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 15: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:19:29.941462  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.864460 12223 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:29.945011  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 130679us
I20260709 12:19:29.901979  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:29.968928  7274 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:29.907106  7265 raft_consensus.cc:1275] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:19:29.970428  6658 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:29.971477  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:29.938056 12364 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.962877 12281 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:29.972635  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:29.973037 12281 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.974009 12364 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:29.975232 12281 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:29.977272  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:29.978996  6509 raft_consensus.cc:1275] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:19:29.950229  7269 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:19:29.952255  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:29.982223  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:29.983215  6649 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:29.983337  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:29.984054  6653 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 21: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:29.984155  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:29.984692  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:29.985005  6649 raft_consensus.cc:2393] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 16: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:29.984896  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:29.985438  6651 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 19: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:19:29.985726  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.955730  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:29.985949  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:29.944245 12167 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.986488  6651 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:29.988054 12167 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:29.965127 12457 log.cc:927] Time spent T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
W20260709 12:19:29.989277  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 86406.5us
I20260709 12:19:29.922703  6503 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 13. Current term is 15. Ops: []
I20260709 12:19:29.852022 12415 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:29.946710 12316 raft_consensus.cc:740] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:29.962754  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:29.998898  6650 raft_consensus.cc:2393] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 12: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:29.935410  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:29.944519  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 66640.9us
W20260709 12:19:29.994482  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:29.944700 12215 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:29.980237  6509 raft_consensus.cc:1240] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:19:29.988894  6656 raft_consensus.cc:2410] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 16 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:19:29.992215  6548 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:30.002627  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.012056  6501 raft_consensus.cc:1240] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 12:19:30.011174  6473 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:29.988724 12223 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:29.957926  6652 raft_consensus.cc:1275] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:29.985841  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:29.964360  7271 raft_consensus.cc:1240] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:30.025660  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:30.026350  6657 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:19:30.027442  6649 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:30.030222  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.029778  6622 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:30.031579  6549 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:30.035403  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:29.997882 12361 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:30.036118  6658 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:30.041658  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:30.042358  6583 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
I20260709 12:19:30.039436  6549 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:30.044889  6504 raft_consensus.cc:2368] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 14. Current term is 15.
I20260709 12:19:29.957687 12120 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.958276 12256 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:29.992272 12415 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:30.049958  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:30.032403 12386 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:30.060408  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:30.039163 11897 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:30.061012  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:30.063984  6576 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:30.064625  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:30.065178  6583 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:30.065894  6650 raft_consensus.cc:2393] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 15: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:30.068210  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:30.068938  6575 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 14.
W20260709 12:19:30.072400  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 104889us
I20260709 12:19:30.049348 12364 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:30.051278 12406 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:30.076014  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 83919.8us
I20260709 12:19:30.078081  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:30.080881 12256 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:30.082639 12120 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:30.085086 12303 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:30.086010  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:30.086870  6656 raft_consensus.cc:2410] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:19:30.087924 12404 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:30.088389 12404 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:30.088761 12404 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:30.089535  6653 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:30.088848  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:30.091300  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:30.092069  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:30.092835  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:30.093371  6654 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:30.077211  7268 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:19:30.075241  6583 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
I20260709 12:19:30.096073  6576 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 12.
I20260709 12:19:30.087935  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:30.060681 12167 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:30.095996  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:30.046319 12398 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 12:19:30.119913 12360 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:30.107280 12316 consensus_queue.cc:260] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:30.108847  6474 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:30.109462 12303 consensus_queue.cc:1059] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:30.072088 12386 raft_consensus.cc:697] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:30.071992 12354 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:30.088466  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:30.138734 12386 consensus_queue.cc:237] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:30.102573  6658 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:30.135950 12215 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:30.145874 12215 raft_consensus.cc:697] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:30.136781 12303 raft_consensus.cc:3055] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:30.146785 12303 raft_consensus.cc:740] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:30.146860 12215 consensus_queue.cc:237] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:30.147727 12303 consensus_queue.cc:260] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:30.137352 12354 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:30.153169  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 57222.6us
I20260709 12:19:30.133610  6649 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:30.150398  6654 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:19:30.156394  6657 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
I20260709 12:19:30.154368  7238 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:30.174361  7238 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:30.103502 12364 raft_consensus.cc:697] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:19:30.184128  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 96492.2us
I20260709 12:19:30.168622 12361 raft_consensus.cc:2764] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:19:30.169029 12303 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:30.190485  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.191856  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:30.195915  6548 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:30.170012 12223 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:30.196120  6576 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:19:30.194824  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:30.201869  7274 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:19:30.203512  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:30.204026  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:30.206313  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:30.206887  6582 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:30.209048  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:30.209336  6579 raft_consensus.cc:1240] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 11. Current term is 12. Ops: [11.12-11.12]
I20260709 12:19:30.209299  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:30.210649  6654 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:30.211884  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:30.198478 12307 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:30.212623  6657 raft_consensus.cc:2368] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 20. Current term is 21.
I20260709 12:19:30.212857 12307 raft_consensus.cc:697] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 12 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:30.213672 12307 consensus_queue.cc:237] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.12, Last appended by leader: 12, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:30.214368  6583 raft_consensus.cc:1275] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 12:19:30.216419  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.216735 12006 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:30.217334 12006 raft_consensus.cc:697] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 16 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:30.218253 12006 consensus_queue.cc:237] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:30.206450 12318 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.220911 12318 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:30.221635 12354 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:30.222879 12318 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:30.223927  6654 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:19:30.198088 12167 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.228750 12167 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:30.231519 12167 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:30.198129 12223 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:30.232237 12223 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:30.235857  6507 raft_consensus.cc:2410] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 6.
I20260709 12:19:30.200903 12272 consensus_queue.cc:1059] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:19:30.237578 12272 raft_consensus.cc:3055] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:30.237941 12272 raft_consensus.cc:740] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:30.238893 12272 consensus_queue.cc:260] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:30.227715 12404 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:30.247385  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.260193  6656 raft_consensus.cc:1275] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 12:19:30.203603 12360 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.260645  6656 pending_rounds.cc:85] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 8
I20260709 12:19:30.266139  6656 pending_rounds.cc:107] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 12:19:30.266458  6656 raft_consensus.cc:2889] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:30.240216 12272 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:30.279346  6576 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:19:30.279578  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:30.242528 12316 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:30.284822  6473 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:19:30.288686  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.287884  6580 raft_consensus.cc:1240] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 15. Current term is 16. Ops: []
W20260709 12:19:30.343362  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.320163  6582 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:19:30.363415  6550 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:30.282907 12256 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:19:30.365972  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 108918us
I20260709 12:19:30.368505  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:30.368635  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 180055us
I20260709 12:19:30.340034  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:30.261402 12281 consensus_queue.cc:1048] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.094s
I20260709 12:19:30.273319 12363 consensus_queue.cc:1059] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.103s
I20260709 12:19:30.274031 12222 consensus_queue.cc:1048] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.095s
W20260709 12:19:30.382740  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.379134  6622 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:30.337102 12229 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:30.355074 12252 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:30.395195  6548 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:30.313568 12398 consensus_queue.cc:1059] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 12:19:30.311899 12360 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:30.405092  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.262439 12406 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.119s
I20260709 12:19:30.426973  6577 raft_consensus.cc:1275] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 12:19:30.430814 12466 log.cc:927] Time spent T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 12:19:30.397573 12229 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:30.441604  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.443884 12229 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:30.445102 12404 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:30.445763 12404 raft_consensus.cc:3055] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:30.446174 12404 raft_consensus.cc:740] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:19:30.448549 12467 log.cc:927] Time spent T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:19:30.449059  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:30.447037 12404 consensus_queue.cc:260] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:30.457201  7272 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:30.458588  6549 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:30.457285 12404 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:30.460413 12318 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:30.461186 12318 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:19:30.473392 12120 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:30.474295 12297 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:19:30.477540  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 98496.7us
W20260709 12:19:30.508657  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 69549.2us
I20260709 12:19:30.413335 12252 raft_consensus.cc:697] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:30.423458 12398 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:30.524430  6583 raft_consensus.cc:1275] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 12:19:30.524847  6583 pending_rounds.cc:85] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 9
I20260709 12:19:30.525195  6583 pending_rounds.cc:107] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 12:19:30.524701 12252 consensus_queue.cc:237] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:30.525449  6583 raft_consensus.cc:2889] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:30.384716 12364 consensus_queue.cc:237] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:30.475935 12167 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:30.538197  7272 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 13. Current term is 14. Ops: []
W20260709 12:19:30.539758  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:30.540395  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.491884 12120 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:30.555181 12222 consensus_queue.cc:1059] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 12:19:30.545688  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:30.575826  6575 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
W20260709 12:19:30.566560 12471 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:19:30.553188 12297 raft_consensus.cc:697] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 16 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:30.558709 12120 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:30.568135  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 204796us
I20260709 12:19:30.536434 12406 consensus_queue.cc:1048] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:19:30.589471  6550 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:30.616711  6580 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:30.464159 12318 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:30.668650  6650 raft_consensus.cc:1275] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 12:19:30.643774 12468 log.cc:927] Time spent T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
W20260709 12:19:30.674508  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 154543us
I20260709 12:19:30.641594 12474 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:30.692608  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:30.694938  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 109047us
I20260709 12:19:30.696321  6473 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:30.698227  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:19:30.701571  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 314081us
I20260709 12:19:30.701778  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:30.702296  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:30.703029  6500 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 16: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:30.676214 12398 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:30.699029  6507 raft_consensus.cc:2376] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15. Re-sending same reply.
I20260709 12:19:30.706377  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:30.707136  6502 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 19: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:30.708809  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:30.709574  6583 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:19:30.705889  6550 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:30.713120  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:30.716277  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:30.716598  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:30.717106  6501 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:30.718878  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:30.720559  6622 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:30.721263  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:30.722216 12297 consensus_queue.cc:237] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:30.724440  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.726423 12360 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:30.710665 12474 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:30.729709 12474 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:30.694669 11462 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:19:30.732424 12475 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:30.733784  6548 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:30.737074 12223 consensus_meta.cc:287] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.504s	user 0.009s	sys 0.000s
I20260709 12:19:30.737493 12223 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:30.695670 12473 consensus_queue.cc:1059] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 14.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:19:30.741439 12473 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:30.742332 12473 raft_consensus.cc:3055] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:30.742728 12473 raft_consensus.cc:740] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:30.736086  6548 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:30.744922  6550 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:30.745940  6550 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:30.746464  6548 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:30.748678 12398 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:19:30.752468  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.708043  6504 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:19:30.706238 12303 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:30.740069 12223 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:30.761893  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.763772  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:30.769292  6501 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:30.772876  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:30.772733  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:30.773646  6505 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:30.776216  6509 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:19:30.779313  6501 raft_consensus.cc:2410] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 6.
I20260709 12:19:30.782044  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:30.783831  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:19:30.785914  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 77911.5us
I20260709 12:19:30.787408  6622 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:30.788287 12223 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:30.788785 12223 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:30.789016 12485 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:30.789243 12223 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:30.762048 12478 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:30.774573 12400 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:30.770107 12303 raft_consensus.cc:697] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:30.801669  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:30.803351  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.813597 12297 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:19:30.816238  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 271864us
I20260709 12:19:30.816725  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:30.816283  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 115300us
I20260709 12:19:30.791785 12398 raft_consensus.cc:697] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:19:30.845566  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.847307  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:30.848641  6473 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:30.851449  6658 raft_consensus.cc:2393] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 16: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:19:30.804020 12272 consensus_meta.cc:287] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.518s	user 0.011s	sys 0.000s
W20260709 12:19:30.856190  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.865108  6504 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:19:30.868465 12487 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:19:30.874361  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 127910us
I20260709 12:19:30.876088  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:30.876732  6506 raft_consensus.cc:2376] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15. Re-sending same reply.
I20260709 12:19:30.878067 12303 consensus_queue.cc:237] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:30.871151 12478 raft_consensus.cc:697] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:30.880743  6622 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:30.881645 12478 consensus_queue.cc:237] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:30.882310 12400 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:30.882805  6622 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:30.882944 12400 raft_consensus.cc:697] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:30.885272  6622 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:30.868973 12229 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:19:30.932412  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.904682  6505 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:30.904521  6548 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:30.945214  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 59999.7us
I20260709 12:19:30.956502  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:30.957299  6501 raft_consensus.cc:2393] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 15: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:19:30.959404  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.904029 12167 raft_consensus.cc:2764] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:19:30.961265 12318 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:30.961565  6582 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 15. Current term is 16. Ops: []
W20260709 12:19:31.002729  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:30.962235 12489 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:30.962298  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:31.005729  6506 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 19: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:19:30.961315  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 56929.5us
I20260709 12:19:30.906944 12229 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:30.974653  6505 raft_consensus.cc:1275] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:19:30.890183 12487 raft_consensus.cc:697] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:30.973606 12153 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 12:19:30.884027 12400 consensus_queue.cc:237] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:30.992684  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 100245us
I20260709 12:19:31.009760  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:30.983253 12215 raft_consensus.cc:3055] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:31.027499  6584 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:19:31.031971  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:31.002508 12222 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.022352  6473 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:30.978564 12485 raft_consensus.cc:697] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:31.011862 12229 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:30.976261 12281 consensus_queue.cc:1048] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:19:31.046243  6582 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:19:31.045691 12487 consensus_queue.cc:237] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:31.053421  6475 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:31.053448  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:31.053546  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:31.084762  6501 raft_consensus.cc:2376] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15. Re-sending same reply.
I20260709 12:19:31.024363 12215 raft_consensus.cc:740] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:30.981941 12479 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 12:19:30.983402 12120 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:31.093813  6579 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 11. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 12:19:31.082163  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:31.024540 12475 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.016746 12318 raft_consensus.cc:697] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:19:31.074826  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 56975.2us
I20260709 12:19:31.031200 12406 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:31.082052  6506 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 12:19:31.096264  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 54427.5us
I20260709 12:19:30.983268 12398 consensus_queue.cc:237] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:31.043556 12360 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.075973 12252 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:19:31.100742  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:31.101061  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 128083us
I20260709 12:19:31.123986  6580 raft_consensus.cc:1275] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 12:19:31.130429  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:31.142166  6548 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:31.082916 12222 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:31.103384 12252 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:31.151892 12400 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:31.152691 12481 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:31.131810 12167 consensus_queue.cc:1059] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 12:19:31.163344  6505 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:19:31.126494 12223 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:31.148542 12252 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:31.106652 12406 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:31.147743 12316 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:31.101390 12475 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:31.166182 11462 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.129382 12506 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 12:19:31.107635 12360 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:31.182461  7238 leader_election.cc:336] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.583s (SENT)
I20260709 12:19:31.147495 12303 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.186239  7238 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:31.161480 12485 consensus_queue.cc:237] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:31.193544 12303 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:31.195951 12303 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:31.197763 12406 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:31.181830 12388 consensus_queue.cc:1059] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 12:19:31.147068 12215 consensus_queue.cc:260] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:31.198997 12215 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:31.199496 12229 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:31.187559 12318 consensus_queue.cc:237] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:31.202584 12475 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:31.204905 12360 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:31.206895  6548 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:31.208364  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:31.208918  6548 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:31.209393 12492 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:31.209834 12499 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:31.210543 12499 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:31.212707  6548 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:31.169725 12481 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:31.213507 11462 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:31.241228 12153 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:31.242588 11462 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:31.243471 12496 log.cc:927] Time spent T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 12:19:31.243585 12473 consensus_queue.cc:260] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:31.245026 12505 log.cc:927] Time spent T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
W20260709 12:19:31.246749 12504 log.cc:927] Time spent T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 12:19:31.247797 12501 log.cc:927] Time spent T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:19:31.249630 12229 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:31.250106 12281 raft_consensus.cc:3055] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:31.250510 12281 raft_consensus.cc:740] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:31.251355 12281 consensus_queue.cc:260] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:31.253505 12223 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:31.254642 12512 log.cc:927] Time spent T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:19:31.256726 12222 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:31.232896 12475 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:19:31.261089 12509 log.cc:927] Time spent T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.068s	user 0.003s	sys 0.000s
I20260709 12:19:31.236382 12271 raft_consensus.cc:993] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65: : Instructing follower 496c7c3792cd4587b4dbeba435f4be1e to start an election
W20260709 12:19:31.266598 12484 log.cc:927] Time spent T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.451s	user 0.000s	sys 0.003s
I20260709 12:19:31.266978 12271 raft_consensus.cc:1081] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 13 LEADER]: Signalling peer 496c7c3792cd4587b4dbeba435f4be1e to start an election
I20260709 12:19:31.267320 12481 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:31.265060 12120 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:31.244947 12473 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:31.271903  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2548) took 1084 ms. Trace:
W20260709 12:19:31.273936  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 132142us
W20260709 12:19:31.281355  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 181359us
I20260709 12:19:31.272716  6584 rpcz_store.cc:276] 0709 12:19:30.187467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:30.187713 (+   246us) service_pool.cc:224] Handling call
0709 12:19:30.207935 (+ 20222us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:31.027982 (+820047us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:31.271659 (+243677us) spinlock_profiling.cc:243] Waited 819 ms on lock 0x7b54000352d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:31.271848 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1884128128}
W20260709 12:19:31.291049  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 108612us
I20260709 12:19:31.273268 12481 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:31.302656  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 249330us
W20260709 12:19:31.305078  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:31.310725  6509 raft_consensus.cc:1275] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 12:19:31.321585  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 108950us
I20260709 12:19:31.323788  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:19:31.324455  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:31.330746  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:31.331681  6505 raft_consensus.cc:2368] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 17. Current term is 18.
I20260709 12:19:31.332674  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:31.319496 12281 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:31.336416  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:31.330441  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:31.337937  6501 raft_consensus.cc:1240] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:31.336529  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:31.313532 12388 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:19:31.343986  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:31.343017  6502 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:31.343612  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:31.345724  6548 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:31.348362  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 44831us
I20260709 12:19:31.349253  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:31.351529  6475 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:31.309095 12481 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:31.360998  6622 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:31.364825  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:31.365654  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:31.365803  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:31.366930  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:31.374420  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:31.375917 11897 raft_consensus.cc:3055] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:31.378098 11897 raft_consensus.cc:740] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:31.379871  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:31.386400  6505 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:31.362026 12406 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:31.372572 12519 raft_consensus.cc:2764] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:19:31.391202  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:31.396886 12485 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:19:31.406664  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 61034.4us
I20260709 12:19:31.374274 12412 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:31.367314 12386 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:31.411422  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:31.415061  6502 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:31.417141  6650 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:31.414878  6657 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:31.389353 12167 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.361778 12388 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:31.450898 12388 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:31.396622 12354 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.422297  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:31.453137 12492 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:31.454267  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:31.455526  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:31.455812 12513 log.cc:927] Time spent T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.249s	user 0.000s	sys 0.001s
I20260709 12:19:31.456127  6502 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:19:31.458273  6502 raft_consensus.cc:1240] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:31.459395  6502 raft_consensus.cc:1240] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:31.453367 12354 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:31.423272  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:31.425864 12406 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:31.462558 12354 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:31.462620 12406 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:31.378890 11897 consensus_queue.cc:260] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:31.464241  6656 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:19:31.465695 11897 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:31.469704  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:31.426589 12511 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:31.471935 12511 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:31.474419 12511 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:31.477892 12474 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:31.478950  7272 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:19:31.479568 12474 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:31.480078  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:31.480124 12360 raft_consensus.cc:3055] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:31.465107 12406 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:31.483979 12360 raft_consensus.cc:740] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:19:31.484630  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:31.432543  6548 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:31.485121 12360 consensus_queue.cc:260] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:31.423004 12386 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:31.486218  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:31.486485 12363 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:31.487923 12386 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:31.488155  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:31.361608 12252 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:31.489354  6658 raft_consensus.cc:1275] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:19:31.490110 12252 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:31.490430  6506 raft_consensus.cc:2393] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 16: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:19:31.491989  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 79310.1us
I20260709 12:19:31.442885 12167 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:31.437330 12481 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:31.485205  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:31.485674 12492 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:31.493165  7265 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:19:31.493762  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:31.440218 12318 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:31.494976 12360 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:31.496414 12272 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:19:31.486232  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 53811.3us
I20260709 12:19:31.500257 12307 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:31.413923 12412 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:31.510811  7274 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 12:19:31.544896  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 43557.7us
I20260709 12:19:31.509395  6652 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 19.
I20260709 12:19:31.500296 12153 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 12:19:31.520654  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:31.527108  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:31.527850  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:31.529739  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:31.535879  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:31.551291  6578 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:31.551446  6580 raft_consensus.cc:1275] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:31.545024  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:31.512315  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:31.548036  6548 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:31.548926  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:31.553984  6583 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
W20260709 12:19:31.549342  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:31.549803  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:31.555763  6584 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:19:31.558279  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:31.569512 12167 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:31.569707  6403 catalog_manager.cc:5697] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
W20260709 12:19:31.571444  6402 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57900 (request call id 190) took 32406 ms (client timeout 15000 ms). Trace:
I20260709 12:19:31.519775 12492 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:31.577701  6475 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:31.548604 12412 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:31.534166 12272 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:31.593194  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:31.593924  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:31.594571  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:31.594655  6582 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:56116
I20260709 12:19:31.605479  6582 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:31.605842  6582 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:31.595906  6575 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 12:19:31.597851  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:31.599809  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 46999.3us
I20260709 12:19:31.599967  6578 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:31.610306  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:31.611188  6583 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
W20260709 12:19:31.614560  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 218915us
I20260709 12:19:31.614905  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:31.615685  6584 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:19:31.618176  6475 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:31.635157  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:31.639292  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:31.615384 12485 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:31.637835  6403 catalog_manager.cc:5697] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 7 to 12, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 12 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:31.620813 12456 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:31.642469 12215 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:19:31.685554  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 137603us
I20260709 12:19:31.619480 12252 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
W20260709 12:19:31.691237  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 73191.5us
I20260709 12:19:31.687059 12388 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:31.729415  7238 leader_election.cc:336] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.730s (SENT)
I20260709 12:19:31.732820  6403 catalog_manager.cc:5697] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 10 to 11, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 11 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:31.668586  6582 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:31.754704  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:31.762265 12524 log.cc:927] Time spent T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:19:31.785708  7238 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:31.789976  6473 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:31.788908 12215 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.793864 12272 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:31.793740 12215 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:31.794418  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 65064.7us
I20260709 12:19:31.795078 12307 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:19:31.795692  7238 leader_election.cc:336] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.793s (SENT)
I20260709 12:19:31.796188  7238 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:31.796396  6403 catalog_manager.cc:5697] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 7 to 10, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 10 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:19:31.798069  6403 catalog_manager.cc:5697] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:31.799865  6403 catalog_manager.cc:5697] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 9 to 10, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 10 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:31.735242 12388 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:31.802418 12485 consensus_queue.cc:1048] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:31.803539 12388 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:31.804548 12307 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:31.804773  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:31.806754  6473 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:31.807350 12485 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.807956  6473 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:31.808321 12485 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:31.809016 12456 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:31.809432 12456 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:31.809820 12456 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:31.810076  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:31.807044  6403 catalog_manager.cc:5697] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 7 to 12. New cstate: current_term: 12 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:31.812500 12229 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:31.812960 12485 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:31.812745 12515 consensus_queue.cc:1048] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:31.813217 12521 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:31.812960 12229 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:31.815970 12229 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:31.813660  6403 catalog_manager.cc:5697] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 9 to 11, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 11 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:19:31.600463  6402 rpcz_store.cc:269] 0709 12:18:59.164875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:59.173077 (+  8202us) service_pool.cc:224] Handling call
0709 12:19:31.571206 (+32398129us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:23.650155 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:23.757535 (+107380us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:23.757586 (+    51us) write_op.cc:435] Acquired schema lock
0709 12:19:23.757605 (+    19us) tablet.cc:662] Decoding operations
0709 12:19:24.052288 (+294683us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:24.068798 (+ 16510us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:24.071911 (+  3113us) tablet.cc:685] Acquiring locks for 25 operations
0709 12:19:24.161378 (+ 89467us) tablet.cc:701] Row locks acquired
0709 12:19:24.161443 (+    65us) write_op.cc:260] PREPARE: finished
0709 12:19:24.161713 (+   270us) write_op.cc:270] Start()
0709 12:19:24.161916 (+   203us) write_op.cc:276] Timestamp: P: 1783599564161627 usec, L: 0
0709 12:19:24.161963 (+    47us) op_driver.cc:348] REPLICATION: starting
0709 12:19:24.162978 (+  1015us) log.cc:844] Serialized 9689 byte log entry
0709 12:19:24.733439 (+570461us) op_driver.cc:464] REPLICATION: finished
0709 12:19:25.033482 (+300043us) write_op.cc:301] APPLY: starting
0709 12:19:25.096366 (+ 62884us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:26.353587 (+1257221us) spinlock_profiling.cc:243] Waited 38.3 ms on lock 0x7b18002b86a0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2875bc80ec 00007f2875c14c0a 00007f2875c144f7 00007f2875c14490 00007f2875bcbf48 00007f28798e01bb 00007f28798e0532 00007f2879927b93 00007f28797d4de3
0709 12:19:27.305200 (+951613us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:27.305243 (+    43us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:29.205013 (+1899770us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:29.205680 (+   667us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=50,key_file_lookups=50,delta_file_lookups=0,mrs_lookups=0
0709 12:19:29.205776 (+    96us) write_op.cc:312] APPLY: finished
0709 12:19:29.206795 (+  1019us) log.cc:844] Serialized 220 byte log entry
0709 12:19:29.339327 (+132532us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:29.434736 (+ 95409us) write_op.cc:454] Released schema lock
0709 12:19:29.434821 (+    85us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:29.435338 (+   517us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:29.435365 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":66,"reported_tablets":53,"child_traces":[["op",{"apply.queue_time_us":272365,"apply.run_cpu_time_us":70904,"apply.run_wall_time_us":4401858,"cfile_cache_hit":102,"cfile_cache_hit_bytes":108427,"num_ops":25,"prepare.queue_time_us":407327,"prepare.run_cpu_time_us":51987,"prepare.run_wall_time_us":832796,"raft.queue_time_us":216872,"raft.run_cpu_time_us":52172,"raft.run_wall_time_us":129186,"replication_time_us":571279,"spinlock_wait_cycles":88111488,"thread_start_us":836826,"threads_started":5,"wal-append.queue_time_us":438227,"wal-append.run_cpu_time_us":31838,"wal-append.run_wall_time_us":2346240}]]}
I20260709 12:19:31.818795  6403 catalog_manager.cc:5697] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 8 to 10, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 10 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:19:31.820941  6473 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:31.825729  7265 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:19:31.830507  6581 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:31.835916  6582 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:31.771736 12517 log.cc:927] Time spent T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 12:19:31.841604  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:31.841863  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:31.843061  6575 raft_consensus.cc:2410] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for term 20 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:19:31.842912  6473 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:31.769541 12318 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:31.847797  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:31.848174  6549 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:31.848502  6581 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:31.850200  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:31.851104  6649 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:31.863754 12406 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:31.856268 12318 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:31.866276  6473 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:31.867308  6473 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:31.868506 12515 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:31.869582  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:31.870033 12515 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:31.870402  6583 raft_consensus.cc:2393] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 16: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:31.870556 12515 raft_consensus.cc:697] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:31.870755  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:31.871719  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 81442.3us
I20260709 12:19:31.872560 12272 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:31.872999 12272 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:31.873425 12272 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:31.873478  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:31.874949 12386 consensus_queue.cc:1059] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 17.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:31.876163 12386 raft_consensus.cc:3055] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:31.876565 12386 raft_consensus.cc:740] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:31.877485 12386 consensus_queue.cc:260] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.7, Last appended by leader: 7, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:19:31.879794  6403 catalog_manager.cc:5697] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:19:31.868149  6473 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:31.883440 12406 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:31.852671 12521 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:31.884361 12406 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:31.897801  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:31.871326  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:31.901104  6583 raft_consensus.cc:2393] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 16: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:31.855293 12478 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 12:19:31.903201 12388 consensus_queue.cc:1059] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:31.905175  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:31.884899 12406 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:31.871580 12515 consensus_queue.cc:237] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:31.882409  6403 catalog_manager.cc:5697] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 9 to 11, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 11 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:19:31.911155  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:31.948437  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:31.955466  6583 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:31.945752  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:31.910897 12252 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 12:19:31.961215  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:31.963750  6575 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:31.950047  6475 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:31.910789 12388 raft_consensus.cc:3055] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:31.953480  6657 raft_consensus.cc:1275] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:19:31.944595 12386 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:31.945231 12215 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:31.965550  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:31.911887  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 64106.6us
I20260709 12:19:31.973448  6649 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:19:31.968957  6576 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
W20260709 12:19:31.956626  6622 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.010s	sys 0.022s
I20260709 12:19:31.953429 12485 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:31.975481  6622 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.010s	sys 0.022s
I20260709 12:19:31.976744  6576 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:19:31.977190  6576 pending_rounds.cc:85] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 8
I20260709 12:19:31.979485  6576 pending_rounds.cc:107] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:19:31.979321  6473 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:31.979930  6576 raft_consensus.cc:2889] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:31.965411 12388 raft_consensus.cc:740] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:31.987465  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:31.989010  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:31.992169  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:31.970757 12303 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:19:32.004832 12532 log.cc:927] Time spent T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:19:32.014314 12521 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:31.983786 12318 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:32.017007 12388 consensus_queue.cc:260] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:32.018359 12388 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:32.023316 12229 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:31.985143 12360 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:32.026163 12360 raft_consensus.cc:697] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:19:32.032117  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:32.011785  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 631920us
I20260709 12:19:32.033439  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:31.979950 12494 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 12:19:32.016026 12492 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:32.051270  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:32.027212 12360 consensus_queue.cc:237] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:31.983695 12485 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:32.057709 12456 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:32.060824  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 102678us
I20260709 12:19:32.058917 12229 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:32.064440  6622 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:32.071647  6575 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:32.072252  6583 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:19:32.047971 12363 consensus_queue.cc:1048] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
W20260709 12:19:32.062750  6548 leader_election.cc:336] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.561s (SENT)
I20260709 12:19:32.046056 12539 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 12:19:32.062929 12537 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:32.092830  6622 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:32.082161 12518 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:32.080484 12487 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:19:32.109508  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:32.119890  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:32.108875 12354 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:19:32.132889  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 132701us
I20260709 12:19:32.120065 12518 raft_consensus.cc:697] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 19 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:19:32.138703  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 75977.8us
W20260709 12:19:32.141476  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:32.141055  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 161083us
I20260709 12:19:32.115917 12318 consensus_queue.cc:1048] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 12:19:32.119803 12167 consensus_queue.cc:1059] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:19:32.143597  6550 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:32.147650  6548 leader_election.cc:336] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.728s (SENT)
I20260709 12:19:32.149893 12539 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:32.153414 12456 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:32.153836  6548 leader_election.cc:336] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.763s (SENT)
I20260709 12:19:32.140864 12400 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:32.160454 12406 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:32.173151 12406 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:32.163313 12538 log.cc:927] Time spent T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.061s	user 0.004s	sys 0.000s
I20260709 12:19:32.145959 12272 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:32.176075 12272 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:32.169938  6548 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:32.178360  6548 leader_election.cc:336] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.777s (SENT)
W20260709 12:19:32.179289  6548 leader_election.cc:336] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.785s (SENT)
I20260709 12:19:32.180742 12487 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:32.140547 12537 consensus_queue.cc:1059] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
W20260709 12:19:32.169116 12412 consensus_meta.cc:287] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.550s	user 0.016s	sys 0.001s
I20260709 12:19:32.191197 12412 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:32.193650 12412 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:32.194425  6650 raft_consensus.cc:1240] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:32.194715 12537 raft_consensus.cc:3055] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:32.195137 12537 raft_consensus.cc:740] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:32.197896 12400 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:32.198423 12400 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:32.196086 12537 consensus_queue.cc:260] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:32.200109 12537 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:32.212829 12539 raft_consensus.cc:697] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:32.213621 12485 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:32.214066  6649 raft_consensus.cc:1275] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:32.213971 12539 consensus_queue.cc:237] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:32.215835 12518 consensus_queue.cc:237] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:32.216799  6548 leader_election.cc:336] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 2.065s (SENT)
I20260709 12:19:32.228499  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:32.228549 12485 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:32.229322  6656 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
W20260709 12:19:32.240665  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 94466.3us
I20260709 12:19:32.244606  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:32.249656  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:32.252110  6550 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:32.255548  6657 raft_consensus.cc:2393] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 16: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:32.259935  6622 leader_election.cc:400] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 17. Current term is 18.
I20260709 12:19:32.260717  6622 leader_election.cc:403] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:19:32.243763 12363 consensus_queue.cc:1059] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 12:19:32.265018  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:32.264587  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 263070us
I20260709 12:19:32.245358 12473 consensus_queue.cc:1048] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 12:19:32.244458 12485 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:32.268726  6475 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:32.255867 12167 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:32.274261  6475 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:32.259066 12398 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:32.265604 12478 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:32.282446  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:32.265151 12556 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 17. Current term is 18.
W20260709 12:19:32.291165  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:32.342698  6582 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:19:32.267776 12472 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:19:32.325627  6575 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 13. Current term is 14. Ops: []
W20260709 12:19:32.334136  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 43878.8us
I20260709 12:19:32.337374  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:32.276301 12515 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:32.338780  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:32.276425 12500 raft_consensus.cc:3055] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:32.341536  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:32.343216  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:32.320719  6475 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:32.289165 12412 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:32.266542 12527 consensus_queue.cc:1059] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 12:19:32.324069 12478 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:32.328954 12406 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:32.347790  6579 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
W20260709 12:19:32.346764  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 68619.5us
I20260709 12:19:32.356398  6584 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
W20260709 12:19:32.372123  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 51551.8us
I20260709 12:19:32.379508  6475 leader_election.cc:400] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 20. Current term is 21.
I20260709 12:19:32.381722  6475 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:32.372470  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:32.373221  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:32.349275 12508 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.349371 12500 raft_consensus.cc:740] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:32.388721  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:19:32.377211 12558 log.cc:927] Time spent T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 12:19:32.374799 12478 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:32.372129 12354 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.377103 12388 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:32.389700  6578 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:32.374760 12400 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:32.283041 12398 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:32.400158  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:32.400288  6652 raft_consensus.cc:1275] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:32.400897  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:32.401567  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:32.402532  6658 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 12:19:32.404057  6548 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:32.419759  6658 pending_rounds.cc:85] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 12
I20260709 12:19:32.420073  6658 pending_rounds.cc:107] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 12:19:32.420318  6658 raft_consensus.cc:2889] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:32.390600  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:32.420982  6581 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:32.415714  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:32.405752 12318 raft_consensus.cc:3055] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:32.377202 12252 consensus_queue.cc:1059] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 12:19:32.393198 12412 raft_consensus.cc:697] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:32.424520  6650 raft_consensus.cc:2393] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 16: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:32.281968 12548 consensus_queue.cc:1048] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.137s
I20260709 12:19:32.434249  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:32.434507  6473 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:32.436721  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:32.437986  6658 raft_consensus.cc:2393] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 15: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:32.438064  6652 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:19:32.417953 12360 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.441685 12360 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:32.439994  6473 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:32.444145 12548 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:32.444607 12548 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:32.444868  6473 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:32.446800 12215 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:32.447268 12215 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:32.447499 12272 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:32.447687 12215 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:32.448496 12456 consensus_queue.cc:1059] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:32.449296 12456 consensus_queue.cc:1243] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer 496c7c3792cd4587b4dbeba435f4be1e log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:19:32.449838 12456 raft_consensus.cc:3055] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:32.450246 12456 raft_consensus.cc:740] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:19:32.451617 12543 log.cc:927] Time spent T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:19:32.451239 12456 consensus_queue.cc:260] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:32.453363 12456 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:19:32.453879 12551 log.cc:927] Time spent T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:19:32.456673  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:32.455101 12360 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:32.436468 12487 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:32.463249 12215 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:32.444988 12548 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:32.466637  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:32.467239  6582 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:32.394186 12508 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:32.470449 12508 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:32.412240 12354 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:32.474336 12354 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:32.391877 12515 consensus_queue.cc:1048] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
W20260709 12:19:32.435231  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 45500.6us
I20260709 12:19:32.492775 12412 consensus_queue.cc:237] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:32.494657  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:32.424569 12318 raft_consensus.cc:740] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:32.496610 12400 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:32.496994  6649 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:19:32.497812 12492 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.500100 12485 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:32.413295 12550 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:19:32.513561  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:32.607524  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:32.523110  6578 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
W20260709 12:19:32.529325  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 125389us
W20260709 12:19:32.611444  6548 leader_election.cc:336] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 2.676s (SENT)
I20260709 12:19:32.546368  6581 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:32.616181  6473 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:32.547283  6582 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:32.548575  6622 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:32.618134  6658 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:19:32.556901  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:19:32.565397  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:32.570544  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:32.622439  6652 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:32.623418  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:32.623848  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:32.624118  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:32.549307 12547 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.626334  6654 raft_consensus.cc:1240] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 13. Current term is 14. Ops: []
W20260709 12:19:32.629345  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 80842.2us
I20260709 12:19:32.559540 12487 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:32.631706  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:32.638253  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:32.633817  6548 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:32.635401  6649 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:19:32.499547 12478 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:32.475433 12557 consensus_queue.cc:1059] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.147s
I20260709 12:19:32.646661  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:32.647619  6504 raft_consensus.cc:2368] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 14. Current term is 15.
I20260709 12:19:32.648623  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:32.649418  6509 raft_consensus.cc:2393] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 15: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:32.612993 12252 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.649742  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:32.650925  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:32.639163  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:32.642647  6473 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:32.653352  6502 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 19.
I20260709 12:19:32.653769  6473 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:32.644250  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:32.646875  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:32.654399  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:32.655254  6501 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
I20260709 12:19:32.656786  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:32.641057  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:32.666199  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:32.666949  6652 raft_consensus.cc:2393] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 16: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:32.667892  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:32.673326  6505 raft_consensus.cc:1240] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 10. Current term is 11. Ops: []
I20260709 12:19:32.674870  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:32.639451 12303 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.659372 12550 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:32.687131  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:32.688030  6504 raft_consensus.cc:2393] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 16: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:32.690994  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:32.692214  6649 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:19:32.693217  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:32.679209  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:32.694303  6509 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 13.
I20260709 12:19:32.648044 12215 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:32.681262  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:32.645998 12473 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:32.684645  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:32.680064  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:32.669263 12229 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:32.703068  6501 raft_consensus.cc:2410] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:19:32.667248 12472 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:32.697839  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:32.613777 12492 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:32.699258  6507 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:32.704403  6656 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:32.704277  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:32.757772  6652 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:32.708635  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:32.758049  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:32.758648  6502 raft_consensus.cc:2393] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 16: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:32.709394  6650 raft_consensus.cc:1240] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:19:32.709653  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:32.760004  6509 raft_consensus.cc:2393] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 15: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:32.710204  6506 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 12.
I20260709 12:19:32.761117  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:32.761276  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:32.761754  6506 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:19:32.762645  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:32.763530  6506 raft_consensus.cc:2410] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
W20260709 12:19:32.710536 12552 log.cc:927] Time spent T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.273s	user 0.001s	sys 0.000s
I20260709 12:19:32.711575 12386 raft_consensus.cc:3055] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:32.710902 12500 consensus_queue.cc:260] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:32.717428  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 83682.4us
I20260709 12:19:32.726101 12398 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:32.636487 12547 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:32.740878 12562 raft_consensus.cc:3055] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:32.741988 12494 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.744211  6474 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:32.744966 12229 raft_consensus.cc:697] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:32.746445 12485 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:32.808706 12521 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:32.750252  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:32.810257  6505 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:19:32.811407  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:32.750150  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:32.812644 12537 consensus_meta.cc:287] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.612s	user 0.007s	sys 0.004s
I20260709 12:19:32.812040 12229 consensus_queue.cc:237] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:32.814477  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:32.801427 12562 raft_consensus.cc:740] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:32.815680  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:32.816152 12494 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:32.816732  6657 raft_consensus.cc:2410] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
W20260709 12:19:32.752310  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 60728.4us
I20260709 12:19:32.770202  6503 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 12:19:32.695852 12456 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:32.778182  6548 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:32.773890 12541 log.cc:927] Time spent T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:19:32.821209  6548 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:32.756022 12472 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:32.827298  6650 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:32.789278  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:32.772542 11897 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.674363 12252 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:32.702205 12550 raft_consensus.cc:697] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:32.803608  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:32.804593  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:32.798730  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:19:32.807211  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 63145.7us
I20260709 12:19:32.749622  6550 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:32.678232 12318 consensus_queue.cc:260] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:32.767894 12388 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:32.866122  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:19:32.867074  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 45633.9us
I20260709 12:19:32.775022  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:32.709425  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:32.816903  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:32.796270  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 99120.9us
I20260709 12:19:32.830890  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 2434) took 1015 ms. Trace:
I20260709 12:19:32.762501 12544 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 12:19:32.772886 12386 raft_consensus.cc:740] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:32.840270  6509 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:32.703007 12303 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:32.871080  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:32.872224  6505 raft_consensus.cc:2376] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18. Re-sending same reply.
I20260709 12:19:32.884372  6504 raft_consensus.cc:2410] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 12:19:32.825920 12485 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:32.870834  6577 rpcz_store.cc:276] 0709 12:19:31.815436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:31.976424 (+160988us) service_pool.cc:224] Handling call
0709 12:19:31.976850 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:32.264438 (+287588us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:32.264471 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:32.407509 (+143038us) spinlock_profiling.cc:243] Waited 3.85 ms on lock 0x7b2c00010c48. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:19:32.407973 (+   464us) log.cc:844] Serialized 26 byte log entry
0709 12:19:32.669228 (+261255us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:32.669265 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:32.669337 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:32.818323 (+148986us) raft_consensus.cc:1685] finished
0709 12:19:32.818343 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:32.830822 (+ 12479us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1597,"raft.queue_time_us":61268,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":3105,"spinlock_wait_cycles":8856320,"thread_start_us":532340,"threads_started":2,"wal-append.queue_time_us":405067}
I20260709 12:19:32.826236 12545 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:32.832293 12354 raft_consensus.cc:3055] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:32.833570 12472 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:32.890997  6500 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:32.892859  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:32.797525 12539 consensus_queue.cc:1059] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 12:19:32.895972  7272 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:19:32.891619  6502 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:32.901501  6622 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:32.840728 12456 raft_consensus.cc:697] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:19:32.891757 12561 log.cc:927] Time spent T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:19:32.913389  6508 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 17.
I20260709 12:19:32.913585  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:32.916894  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:32.849275 12398 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
W20260709 12:19:32.921041  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 267335us
I20260709 12:19:32.926240  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:32.930644  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:32.840478 12487 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 12:19:32.928351  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:32.849242 11897 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:32.983311  6506 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:32.905140 12354 raft_consensus.cc:740] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:19:33.007817  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:33.003906  6504 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 19: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:33.005188  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:32.864876 12548 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:32.921679 12318 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:33.009295  6506 raft_consensus.cc:2410] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 12:19:33.010377  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:33.011188  6506 raft_consensus.cc:2376] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17. Re-sending same reply.
I20260709 12:19:33.002594  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:33.012712  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:33.013511  6505 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 12:19:33.010380  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:33.016086  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:33.019315  6582 raft_consensus.cc:2393] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 14: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:33.005970  6509 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:19:32.883100 12388 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:33.007197  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:32.923826 12511 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:33.020670  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:32.815786 12562 consensus_queue.cc:260] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:32.985001 12167 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:33.027482  6582 raft_consensus.cc:1275] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 12:19:33.003717 12553 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:19:33.004405 12229 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:33.036082  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:33.029582  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:33.032017  6581 raft_consensus.cc:1240] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:33.034572  6505 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:33.034704  6504 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:19:32.924322 12508 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.105s
I20260709 12:19:33.035341  6584 raft_consensus.cc:1240] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
W20260709 12:19:33.046272  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:33.020011 12494 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:33.027447  6509 raft_consensus.cc:1240] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 12:19:33.007078 12527 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:33.038757  6579 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:19:33.040863  6506 raft_consensus.cc:1275] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 12:19:33.040638  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 194867us
I20260709 12:19:33.010866 12545 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:33.009415 12518 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:33.038839  6582 raft_consensus.cc:2393] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 15: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:33.028746 12511 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:33.060998  6652 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:33.043941 12478 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:33.071499  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:33.072122  6581 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:19:33.068650  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:33.073653  6501 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:19:33.044253 12229 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:33.076843 12229 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:33.082567  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 151996us
I20260709 12:19:33.083654  6622 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:33.084913  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:33.089102  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:33.093046  6505 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:33.094206  6622 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:33.095819 12578 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:33.096585  6622 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:33.096246 12400 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:33.098290  6622 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:33.098065 12578 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:33.099261 12400 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:33.100039  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:33.100924 12400 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:33.101307  6622 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:33.101425 12400 raft_consensus.cc:697] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:33.102252 12578 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:33.102814 12400 consensus_queue.cc:237] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:33.103447 12578 raft_consensus.cc:697] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:33.104781 12578 consensus_queue.cc:237] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:33.066629 12511 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:33.111054  6509 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:33.121507  6548 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:33.109175 12556 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:19:33.115757  7238 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.551s (SENT) [suppressed 41 similar messages]
I20260709 12:19:33.125054 12545 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:33.131386 12388 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:33.131670 12545 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:33.132761 12582 log.cc:927] Time spent T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
W20260709 12:19:33.134246 12573 log.cc:927] Time spent T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 12:19:33.135022 12303 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:33.135313 12585 log.cc:927] Time spent T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:19:33.054263 12456 consensus_queue.cc:237] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:33.136559 12167 consensus_queue.cc:1048] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:33.134155 12545 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:33.137907  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:33.091993 12527 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:33.143682  6507 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:33.039148 12550 consensus_queue.cc:237] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:33.131539 12587 log.cc:927] Time spent T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 12:19:33.116910 12575 log.cc:927] Time spent T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.227s	user 0.000s	sys 0.001s
I20260709 12:19:33.146723  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:33.148646  7238 leader_election.cc:336] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.807s (SENT)
I20260709 12:19:33.149083  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:33.143723  6503 raft_consensus.cc:2376] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15. Re-sending same reply.
I20260709 12:19:33.148164  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:33.157294  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:33.158346  6507 raft_consensus.cc:2393] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 13: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:19:33.161597  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:33.165518  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:33.167228  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:33.167634  6509 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 12:19:33.168027  6509 pending_rounds.cc:85] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 11
I20260709 12:19:33.168257  6509 pending_rounds.cc:107] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 12:19:33.168517  6509 raft_consensus.cc:2889] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:33.169322  6475 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:33.172183  6475 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:33.069451 12354 consensus_queue.cc:260] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:33.147804 12252 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:33.185438  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 64054us
I20260709 12:19:33.186576  6548 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:33.177573  7238 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:33.189275  6507 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
new_leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:58944
I20260709 12:19:33.193899  6507 raft_consensus.cc:606] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Received request to transfer leadership to TS 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:33.194375  6507 raft_consensus.cc:614] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:19:33.206619  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 457030us
I20260709 12:19:33.209815  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:33.210613  6652 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:33.217865  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:33.163897 12563 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:33.231474  6509 raft_consensus.cc:1275] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:19:33.233811  6509 raft_consensus.cc:1240] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 13. Current term is 14. Ops: [13.9-13.9]
W20260709 12:19:33.235538  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 49082.2us
I20260709 12:19:33.235123 12386 consensus_queue.cc:260] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:33.236285  6548 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:33.236424 12386 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:33.237524 12487 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:33.238400 12487 raft_consensus.cc:2764] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:19:33.239513 12500 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:33.249610  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:33.250316  6654 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:33.250488  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 147249us
W20260709 12:19:33.252960  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 75495us
I20260709 12:19:33.222276 12252 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:33.196734 12515 consensus_queue.cc:1048] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.082s
W20260709 12:19:33.254303  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 86984.4us
I20260709 12:19:33.265456  6652 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:33.280196  6650 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:19:33.196246 12481 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:33.282696  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:33.295122  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:33.300714  6652 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:19:33.301966  6654 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:33.224362 12354 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:33.322986 12588 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:33.323329 12598 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:33.323829 12598 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:19:33.331370 12547 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:33.197958 12518 consensus_queue.cc:1175] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "20a202c76244485fbc8f1610473db8b9" responder_term: 14 status { last_received { term: 14 index: 11 } last_committed_idx: 11 last_received_current_leader { term: 14 index: 11 } }
I20260709 12:19:33.297293  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2584) took 1013 ms. Trace:
W20260709 12:19:33.314612  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 78396.7us
I20260709 12:19:33.315536  6581 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:33.280310 12548 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:33.195956 12272 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.100s
I20260709 12:19:33.318300  6509 raft_consensus.cc:1275] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:19:33.255676 11897 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:33.321913  6654 raft_consensus.cc:1275] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:33.341928  6548 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:33.339843  6583 rpcz_store.cc:276] 0709 12:19:32.283617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:32.283832 (+   215us) service_pool.cc:224] Handling call
0709 12:19:32.284265 (+   433us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:32.337946 (+ 53681us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:32.337973 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:32.345771 (+  7798us) log.cc:844] Serialized 52 byte log entry
0709 12:19:32.607449 (+261678us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:32.607513 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:32.607618 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:33.297053 (+689435us) raft_consensus.cc:1685] finished
0709 12:19:33.297078 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:33.297229 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3642,"raft.queue_time_us":18180,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":24511,"thread_start_us":216222,"threads_started":1,"wal-append.queue_time_us":398269}
W20260709 12:19:33.367040  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 194910us
W20260709 12:19:33.369771  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 80511.3us
I20260709 12:19:33.371848  6475 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:33.373279  6501 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:19:33.363508  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:33.382287  6475 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:33.378823  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:33.295806 12578 consensus_queue.cc:1059] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 12:19:33.372059  7238 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:33.385848  6550 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:33.366549 12580 log.cc:927] Time spent T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.256s	user 0.001s	sys 0.000s
I20260709 12:19:33.346005 12487 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:19:33.394294  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 52454.6us
I20260709 12:19:33.394835  6622 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:33.395128  6548 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:33.399360  6550 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:33.400669  6501 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:19:33.401933  6501 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:19:33.402989  6501 raft_consensus.cc:1240] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 14. Current term is 15. Ops: []
W20260709 12:19:33.404795 12589 log.cc:927] Time spent T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.258s	user 0.001s	sys 0.000s
I20260709 12:19:33.406031 12252 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:33.406473 12567 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:33.407068 12616 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:33.324506 12566 consensus_queue.cc:1059] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.194s
I20260709 12:19:33.400889 12576 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:33.408692 12616 raft_consensus.cc:3055] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:19:33.408994 12565 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:33.409458 12565 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:33.409865 12565 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:33.411049 12539 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:33.411578 12539 raft_consensus.cc:697] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:19:33.412679 12568 log.cc:927] Time spent T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:19:33.412617 12539 consensus_queue.cc:237] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:33.416244 12596 log.cc:927] Time spent T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:19:33.419018 12615 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:33.420668 12614 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 12:19:33.421718  6622 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:19:33.422633  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:33.423650 12578 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:33.409027 12616 raft_consensus.cc:740] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:33.425452 12485 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:33.425710 12615 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:19:33.426498 12546 log.cc:927] Time spent T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.055s	user 0.002s	sys 0.000s
I20260709 12:19:33.426271 12616 consensus_queue.cc:260] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:33.398474 12545 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:33.429853 12318 consensus_queue.cc:1048] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:33.430989 12615 consensus_queue.cc:1059] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:19:33.431821 12318 raft_consensus.cc:3055] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:33.432478 12598 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:33.432173 12318 raft_consensus.cc:740] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:33.434578  6501 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:33.436578  6501 raft_consensus.cc:1275] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
W20260709 12:19:33.472919  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 90728.4us
I20260709 12:19:33.427758 12616 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:33.489563  6475 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:33.491235  6475 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:33.496394  6475 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:33.399772 12487 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:33.482847 12614 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:19:33.484563 12606 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 12:19:33.513669  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 118613us
I20260709 12:19:33.526911  6548 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:33.519559  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:33.520371  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:33.520999 12548 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:33.522094 12611 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:33.523355  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:33.528154  6658 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:19:33.498762 12515 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:33.548158  6548 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:33.557624  6650 raft_consensus.cc:1275] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:19:33.563218 12576 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:33.563707 12599 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:33.563792 12576 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:33.564610 12599 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:33.565173 12599 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:33.566613 12576 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:33.524705 12272 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:33.567929 12272 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:33.568413 12272 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:33.572875 12600 log.cc:927] Time spent T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.183s	user 0.000s	sys 0.000s
I20260709 12:19:33.573732 12584 consensus_queue.cc:1059] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.087s
I20260709 12:19:33.575145 12584 raft_consensus.cc:3055] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:33.575572 12584 raft_consensus.cc:740] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:33.576803 12609 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:19:33.578559 12604 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:19:33.585675  6548 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:33.602360  6506 raft_consensus.cc:1240] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:33.605809  6548 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:33.560714 12545 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:33.589394 12485 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:33.619654  6548 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:33.598456 12611 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:33.637403 12363 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:33.638026 12363 raft_consensus.cc:697] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:33.638978 12363 consensus_queue.cc:237] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:33.540023 12318 consensus_queue.cc:260] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:33.619959 12485 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:33.617599 12598 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:33.664528  6580 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:33.641973  6548 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:33.648164 12545 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:33.646025 12611 raft_consensus.cc:697] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:33.654135 12485 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:33.689354  6548 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:33.696012 12612 consensus_queue.cc:1048] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:33.697271 12545 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:33.687647 12500 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 12:19:33.699615 12545 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:33.701066 12398 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:33.702857 12545 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:33.707073  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:33.682524 12595 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:33.708782 12613 log.cc:927] Time spent T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 12:19:33.709406 12615 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:33.709918 12167 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:19:33.711822 12601 log.cc:927] Time spent T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:19:33.712196 12611 consensus_queue.cc:237] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:33.717281 12472 consensus_meta.cc:287] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.787s	user 0.011s	sys 0.000s
I20260709 12:19:33.755424  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:33.693590 12318 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:33.608529 12584 consensus_queue.cc:260] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:33.766068  6504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2560) took 1109 ms. Trace:
I20260709 12:19:33.770772  6622 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:33.682730 12565 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:33.750228 12537 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:33.769196  6504 rpcz_store.cc:276] 0709 12:19:32.644945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:33.043347 (+398402us) service_pool.cc:224] Handling call
0709 12:19:33.060912 (+ 17565us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:33.061210 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:19:33.070019 (+  8809us) log.cc:844] Serialized 14 byte log entry
0709 12:19:33.701833 (+631814us) log.cc:844] Serialized 14 byte log entry
0709 12:19:33.753988 (+ 52155us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:33.754180 (+   192us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:33.754246 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:33.754338 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:33.754569 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16464,"prepare.queue_time_us":22606,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":71,"raft.queue_time_us":182742,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":19251,"thread_start_us":631175,"threads_started":3,"wal-append.queue_time_us":532186}
I20260709 12:19:33.783206  6583 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 12:19:33.783581  6575 raft_consensus.cc:1275] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 12:19:33.783813  6583 pending_rounds.cc:85] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 10
I20260709 12:19:33.784284  6583 pending_rounds.cc:107] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 12:19:33.784672  6583 raft_consensus.cc:2889] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:33.768628 12527 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:33.757088 12272 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:33.784605 12518 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:33.778283 12487 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:33.814188 12565 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:33.767803 12472 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:33.793692 12527 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:33.799129 12584 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:33.787124 12537 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:33.839104 12583 log.cc:927] Time spent T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.805s	user 0.002s	sys 0.003s
W20260709 12:19:33.841512 12603 log.cc:927] Time spent T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.332s	user 0.001s	sys 0.001s
I20260709 12:19:33.843829 12562 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:33.875265 12511 consensus_meta.cc:287] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.728s	user 0.009s	sys 0.001s
I20260709 12:19:33.845649 12400 raft_consensus.cc:3055] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:33.884626 12400 raft_consensus.cc:740] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:33.885632 12400 consensus_queue.cc:260] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:33.847831 12595 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:33.848592 12153 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:33.886919  6506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2562) took 1226 ms. Trace:
W20260709 12:19:33.853471 12570 log.cc:927] Time spent T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.846s	user 0.004s	sys 0.015s
I20260709 12:19:33.887384  6506 rpcz_store.cc:276] 0709 12:19:32.660743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:33.062755 (+402012us) service_pool.cc:224] Handling call
0709 12:19:33.063039 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:33.602957 (+539918us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:33.886695 (+283738us) spinlock_profiling.cc:243] Waited 539 ms on lock 0x7b5400387f54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:33.886858 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1239754496}
I20260709 12:19:33.862279 12272 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:33.863025 12120 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:33.864424  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:33.864584  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:33.887401 12153 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:33.890870 12588 consensus_meta.cc:287] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.567s	user 0.009s	sys 0.000s
I20260709 12:19:33.891348 12588 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:33.891688 12153 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:33.892119  6582 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:33.844081 12472 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:33.880275  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41304 (request call id 712) took 1249 ms. Trace:
I20260709 12:19:33.895344 12599 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:33.895682  6508 rpcz_store.cc:276] 0709 12:19:32.631254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:32.631376 (+   122us) service_pool.cc:224] Handling call
0709 12:19:33.879999 (+1248623us) spinlock_profiling.cc:243] Waited 277 ms on lock 0x7b54002c4cd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:33.880176 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":636855040}
I20260709 12:19:33.880808  6507 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:19:33.835508 12527 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:33.898140  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2760) took 1217 ms. Trace:
I20260709 12:19:33.898218  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:33.898591  6507 rpcz_store.cc:276] 0709 12:19:32.682981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:33.222985 (+540004us) service_pool.cc:224] Handling call
0709 12:19:33.880434 (+657449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:33.897025 (+ 16591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:33.897912 (+   887us) spinlock_profiling.cc:243] Waited 605 ms on lock 0x7b54002c4cd0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:33.898061 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1392486272}
I20260709 12:19:33.888438  6576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 2437) took 2072 ms. Trace:
I20260709 12:19:33.900421  7269 raft_consensus.cc:1240] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:33.886935 12400 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:33.905995 12537 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:33.909313 12599 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:33.910223 12485 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:33.911223 12602 log.cc:927] Time spent T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.520s	user 0.006s	sys 0.000s
I20260709 12:19:33.911509  6581 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:19:33.911547  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:33.912742  6500 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 17.
W20260709 12:19:33.914121  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:33.914523  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:33.915011  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:33.888945 12120 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:33.893117 12386 consensus_meta.cc:287] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.655s	user 0.009s	sys 0.003s
I20260709 12:19:33.916296  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:33.899801  6576 rpcz_store.cc:276] 0709 12:19:31.816277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:31.983241 (+166964us) service_pool.cc:224] Handling call
0709 12:19:31.983597 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:32.253424 (+269827us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:32.253473 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:32.399708 (+146235us) spinlock_profiling.cc:243] Waited 5.48 ms on lock 0x7b2c00010c48. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:19:32.400113 (+   405us) log.cc:844] Serialized 26 byte log entry
0709 12:19:32.677704 (+277591us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:19:32.677743 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:32.677810 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:33.852796 (+1174986us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:33.888204 (+ 35408us) raft_consensus.cc:1685] finished
0709 12:19:33.888228 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:33.888383 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":618,"raft.queue_time_us":60319,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":5474,"spinlock_wait_cycles":37493888,"thread_start_us":539514,"threads_started":2,"wal-append.queue_time_us":462963}
I20260709 12:19:33.928550  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:33.929416  6657 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:33.931411  6502 raft_consensus.cc:2410] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:19:33.932116  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:33.932792  6657 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:33.946113 12588 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:33.949972  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:33.952322  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:33.960120  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:33.960458  6475 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:33.951295  6657 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:33.955816  6550 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:33.918936 12272 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:33.968380  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 179845us
I20260709 12:19:33.968457  6653 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:33.972303  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:33.972992  7269 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:19:33.965601  6550 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:33.913962 12511 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:33.930768 12616 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:33.980973  6651 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:33.964303 12578 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:33.984144  6623 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:33.987087  6550 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:33.981343 12547 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:33.972705 12588 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:33.994048  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:33.994297  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:33.994740  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:33.976643 12229 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:33.998019  6576 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:19:33.996210  6622 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:33.996261  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:33.997035  6656 raft_consensus.cc:1240] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:19:33.980296 12412 consensus_queue.cc:1048] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 12:19:33.997860 12229 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:33.972210 12272 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:33.984915 12616 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:34.006381  6649 raft_consensus.cc:1275] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 12:19:34.009291  6649 pending_rounds.cc:85] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 9
I20260709 12:19:34.009640  6649 pending_rounds.cc:107] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 12:19:34.009948  6649 raft_consensus.cc:2889] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:33.993244 12588 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:33.992107 12547 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:33.995941 12615 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:33.986344 12593 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 12:19:33.991511 12472 consensus_queue.cc:1059] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 12:19:34.010902 12229 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:33.991252 12578 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:34.026888  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:34.032124  6653 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:34.012905 12616 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:34.015832 12547 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:34.016448 12588 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:34.015563 12615 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:34.019405 12539 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:19:34.049038  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 49771.7us
I20260709 12:19:34.043766 12386 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.047322 12615 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:34.046802 12539 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:34.019629 12527 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:34.065140 12472 raft_consensus.cc:3055] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:34.053843 12572 consensus_queue.cc:1059] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.107s
I20260709 12:19:34.063376 12485 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:34.068032 12539 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:34.051860 12641 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:34.046746 12562 consensus_queue.cc:1059] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 12:19:34.041882 12636 consensus_queue.cc:1059] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 12:19:34.043987 12554 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.144s
I20260709 12:19:34.059819 12354 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.110656  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:34.111328  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 53997.8us
I20260709 12:19:34.049796 12611 consensus_queue.cc:1059] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
W20260709 12:19:34.086862  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 48525.8us
I20260709 12:19:34.125738  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:19:34.111020  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:34.076390 12472 raft_consensus.cc:740] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:34.119124  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:34.129110  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:34.122825 12545 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.123855 12636 raft_consensus.cc:3055] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:19:34.132597 12634 log.cc:927] Time spent T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 12:19:34.132870 12636 raft_consensus.cc:740] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:34.133836 12636 consensus_queue.cc:260] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:34.135085 12636 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:34.061270 12386 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:19:34.137468 12354 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:34.138103  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:34.138377 12386 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:34.138759  6507 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 19.
I20260709 12:19:34.139592  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:34.139838  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:34.140184  6500 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:34.140528 12562 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:34.144948  7238 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:34.131891 12472 consensus_queue.cc:260] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:34.126304 12595 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
W20260709 12:19:34.159179  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:34.167595 12472 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:34.168111  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:34.152804 12545 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:34.168821  6508 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:19:34.169409 12527 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:34.169734  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:34.170368 12545 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:34.170791  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:34.171694  6622 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:34.171864  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:34.139592 12354 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:34.172808 12354 consensus_queue.cc:1048] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:34.173204 12120 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:34.175814  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:34.175886  6548 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:34.125023 12511 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:34.185969  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2767) took 1495 ms. Trace:
I20260709 12:19:34.187906  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:34.188862  6508 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:19:34.190081  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:34.191687  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:34.192271 11462 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:34.192950  7240 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:34.186401  6509 rpcz_store.cc:276] 0709 12:19:32.690328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:33.236185 (+545857us) service_pool.cc:224] Handling call
0709 12:19:33.236557 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:33.365279 (+128722us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:34.185736 (+820457us) spinlock_profiling.cc:243] Waited 76.9 ms on lock 0x7b540001c7d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:34.185915 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":176854784}
I20260709 12:19:34.152467 12307 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:34.195940  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:34.151823 12641 raft_consensus.cc:697] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:34.196550  6509 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:34.194602 11462 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:34.198215  6508 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:34.199396  6548 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:34.204761  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 51493.2us
I20260709 12:19:34.208683  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:34.211467  6548 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:34.207530  6550 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:34.222081  6500 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:19:34.173686 12562 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:19:34.198989 12307 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:34.204947 12565 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:34.226428  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:34.237365 12639 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:34.215759 12595 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:34.238065 12639 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:34.240510 12644 log.cc:927] Time spent T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:19:34.240613 12641 consensus_queue.cc:237] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:34.241758 12642 log.cc:927] Time spent T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 12:19:34.242898 12654 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:34.243425 12654 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:34.211469 12578 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:34.245433  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:34.212491 12272 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:34.254134  6582 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:19:34.254022  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:34.254587  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:34.257277  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:34.224485 12645 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.266083  6576 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:34.266448  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:34.243822 12654 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:34.224103 12120 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:34.287346 12597 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:34.289839  6503 raft_consensus.cc:1240] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:19:34.291497  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2567) took 1613 ms. Trace:
I20260709 12:19:34.292649  6578 raft_consensus.cc:1240] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:34.236820 12307 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:34.275318  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:34.275995  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:34.277241 12492 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:34.280459 12624 log.cc:927] Time spent T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:19:34.240319 12639 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:34.283748  6501 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
I20260709 12:19:34.285467  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:34.244001 12651 raft_consensus.cc:3055] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:34.291857  6503 rpcz_store.cc:276] 0709 12:19:32.678284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:33.195864 (+517580us) service_pool.cc:224] Handling call
0709 12:19:33.274947 (+ 79083us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:34.290445 (+1015498us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:34.291200 (+   755us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540001bdd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:34.291409 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2333357824}
I20260709 12:19:34.282866  6502 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 2775) took 1371 ms. Trace:
I20260709 12:19:34.249853 12652 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:34.306030  6501 pending_rounds.cc:85] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 11
I20260709 12:19:34.265055 12584 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.265308 12611 consensus_queue.cc:1048] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 12:19:34.313330  6584 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:19:34.315799  6583 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:19:34.314586  6502 rpcz_store.cc:276] 0709 12:19:32.911868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:32.925625 (+ 13757us) service_pool.cc:224] Handling call
0709 12:19:34.282645 (+1357020us) spinlock_profiling.cc:243] Waited 915 ms on lock 0x7b540001c2d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:34.282803 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2105173760}
I20260709 12:19:34.318068  6501 pending_rounds.cc:107] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 12:19:34.325268  6578 raft_consensus.cc:1275] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:34.325335  6500 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
new_leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:58944
I20260709 12:19:34.329767  6501 raft_consensus.cc:2889] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:34.330106  6500 raft_consensus.cc:606] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Received request to transfer leadership to TS 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:34.330580  6500 raft_consensus.cc:614] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:34.312129  6508 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:34.332916  6508 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:19:34.293792 12645 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:34.335394  6508 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:19:34.340126  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:34.314433 12651 raft_consensus.cc:740] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:34.338786  6503 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 12:19:34.339251  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 165801us
I20260709 12:19:34.341302 12651 consensus_queue.cc:260] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:34.342350  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:34.342474  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:34.343137  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:34.343712  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:34.345010  6622 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:34.339572  6578 raft_consensus.cc:1275] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:19:34.344826  6473 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:34.338184 12646 log.cc:927] Time spent T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
W20260709 12:19:34.360605  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 154980us
I20260709 12:19:34.343250 12386 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.329161 12584 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:34.327078 12652 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:34.355100 12472 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:34.380443  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:34.363191 12641 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:19:34.389689 12658 log.cc:927] Time spent T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 12:19:34.373375 12578 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 12:19:34.381541  6503 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:34.376756 12653 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:34.448726 12584 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:34.449141 12653 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:34.449568 12653 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:34.372056 12651 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:34.406262 12655 log.cc:927] Time spent T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
W20260709 12:19:34.407080 12662 log.cc:927] Time spent T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
W20260709 12:19:34.407821 12660 log.cc:927] Time spent T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:19:34.429204 12641 raft_consensus.cc:697] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:34.432695  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:34.465628  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:34.466449  6506 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:34.435169  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 2448) took 1408 ms. Trace:
I20260709 12:19:34.441677  6655 raft_consensus.cc:1240] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:34.442112 12539 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:34.467434  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:34.467171  6580 rpcz_store.cc:276] 0709 12:19:33.026743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:33.029293 (+  2550us) service_pool.cc:224] Handling call
0709 12:19:34.434955 (+1405662us) spinlock_profiling.cc:243] Waited 578 ms on lock 0x7b54001d0bd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:34.435078 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1328965504}
I20260709 12:19:34.468466  6582 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:34.463609 12641 consensus_queue.cc:237] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:34.469110 12539 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:34.469740  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:34.470346  6654 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:19:34.474649  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:34.381484 12272 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:34.444088  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:34.444558 12594 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.378494 12386 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:34.472157 12572 raft_consensus.cc:3055] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:34.451086  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:34.452302  6650 raft_consensus.cc:1275] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:34.498055  6657 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:34.476308  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:34.403869 12472 raft_consensus.cc:697] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:34.465382 12615 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:34.368024 12229 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:34.460642 12646 log.cc:927] Time spent T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:19:34.443239 12616 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:34.477469  6509 raft_consensus.cc:3055] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:34.549819  6509 raft_consensus.cc:740] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:34.481181 12318 raft_consensus.cc:3055] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:34.483119  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:34.481920 12652 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:34.484645 11462 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:34.489538 12645 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:34.494204 12594 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:34.495904 12572 raft_consensus.cc:740] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:34.478520  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2508) took 1229 ms. Trace:
I20260709 12:19:34.593786  6649 rpcz_store.cc:276] 0709 12:19:33.249562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:33.252051 (+  2489us) service_pool.cc:224] Handling call
0709 12:19:33.252405 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:34.010518 (+758113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:34.478173 (+467655us) spinlock_profiling.cc:243] Waited 753 ms on lock 0x7b540010de54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:34.478407 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1732826368}
W20260709 12:19:34.531803 12640 log.cc:927] Time spent T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.443s	user 0.000s	sys 0.002s
I20260709 12:19:34.531303 12472 consensus_queue.cc:237] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:34.542400  7240 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:34.543191  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:34.555101  6657 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
W20260709 12:19:34.690331  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 345532us
I20260709 12:19:34.740811  6473 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:19:34.690912  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 473849us
I20260709 12:19:34.568514 12318 raft_consensus.cc:740] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:34.620113  7274 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:34.550912  6509 consensus_queue.cc:260] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:34.737025  7238 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:34.743547  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 201228us
I20260709 12:19:34.745771  6509 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:34.754678  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:34.755482  6649 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:34.737028  7269 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:19:34.746743  6473 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:34.649770 12645 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.788596  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:34.789443  6579 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:34.791050  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:34.788659  6509 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:19:34.793365  6509 pending_rounds.cc:85] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 8
I20260709 12:19:34.793190  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:34.793857  6509 pending_rounds.cc:107] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:19:34.794183  6584 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:34.794231  6509 raft_consensus.cc:2889] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:34.791887  6575 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:34.734810 12667 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:34.737709 12652 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:34.800969  6575 raft_consensus.cc:1275] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:34.750161 12610 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:34.771247 12588 consensus_meta.cc:287] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.712s	user 0.011s	sys 0.000s
I20260709 12:19:34.743616 12671 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:19:34.775379 12547 consensus_meta.cc:287] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.722s	user 0.017s	sys 0.000s
I20260709 12:19:34.810730  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:34.815841  6575 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:34.762640 12398 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:34.749571 12581 consensus_queue.cc:1048] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 12:19:34.786586 12564 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.830163  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:19:34.814131 12667 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:34.845773  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:34.849445  6584 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:19:34.852358  6549 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:34.818877 12412 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:34.822971 12537 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:34.856691  6548 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:34.805240 12645 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:34.841670 12565 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:34.833315 12653 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:34.818522 12652 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:34.872844  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 135891us
I20260709 12:19:34.820178 12588 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:34.824970 12547 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:19:34.887152  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:34.911186 12667 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:34.911885 12588 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:34.863832 12639 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:34.834055 12398 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:34.824769 12671 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:34.933661 12581 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.934239  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:34.934295 12581 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:34.934767 12398 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:34.841993 12564 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:34.938400  6582 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 12:19:34.939435 12653 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:34.913928 12229 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:34.873853 12537 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:34.944372 12671 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:34.944720 12537 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:34.818801 12572 consensus_queue.cc:260] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:34.869452 12412 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:34.946789 12547 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:34.947688 12412 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:34.948561 12545 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:34.949227  6578 raft_consensus.cc:2410] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 12:19:34.950348  6583 raft_consensus.cc:2393] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 19: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:34.880564 12616 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:34.951013 12645 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:34.887138 12615 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:34.951622 12652 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:34.873548 12481 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:34.962066  6548 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:34.965710 12598 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:34.928717 12639 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:34.967907 12639 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:34.969826  6550 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:34.973608 12636 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:34.974171 12636 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:34.974680 12494 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:19:34.977272 12654 consensus_meta.cc:287] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.675s	user 0.010s	sys 0.014s
I20260709 12:19:34.978278 12653 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:34.977891 12654 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:34.990932  7240 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:34.975448 12494 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:34.858232 12318 consensus_queue.cc:260] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:35.018663  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 245235us
I20260709 12:19:35.028997  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:35.030043  6650 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
I20260709 12:19:35.046885  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:35.055114  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:35.015142 12565 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:19:35.058691  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 321944us
I20260709 12:19:35.049283 12318 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:35.045137 12581 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:35.073997  6579 raft_consensus.cc:2410] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
W20260709 12:19:35.069399  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 322723us
I20260709 12:19:35.079020  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:35.052948 12636 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:35.053776 12654 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:35.087853  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:35.093709  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:35.094434  6577 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
W20260709 12:19:35.097802  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:35.112708  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:35.111340  7240 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:35.113781  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:35.148073  6579 raft_consensus.cc:2393] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 14: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:35.110589  6622 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:35.113447  6577 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
W20260709 12:19:35.153985  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 43504.5us
W20260709 12:19:35.155115  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 70638.1us
W20260709 12:19:35.156065  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 67400.3us
I20260709 12:19:35.158197  6506 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:35.159307  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:35.158682  6622 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:35.109867 12645 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:35.160178  6509 raft_consensus.cc:2410] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 12:19:35.113170 12537 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.253408 12645 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:35.254115 12537 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:35.255744 12645 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:35.256290 12537 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:35.150642 12616 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:35.221235  6501 raft_consensus.cc:1275] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:35.084851 12354 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:35.112577 12672 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.111s
W20260709 12:19:35.226336 12651 consensus_meta.cc:287] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.699s	user 0.010s	sys 0.002s
I20260709 12:19:35.261324 12354 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:35.260254 12616 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:35.221886 12500 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.234577 12594 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:35.238020 12215 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:35.243469 12386 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:35.245955 12229 consensus_queue.cc:1048] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:35.249202 12494 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:35.250628 12307 consensus_meta.cc:287] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.937s	user 0.010s	sys 0.002s
I20260709 12:19:35.251418 12481 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:19:35.257783  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 99208us
I20260709 12:19:35.256898 12564 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:35.257972 12670 consensus_queue.cc:1048] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 12:19:35.259442 12654 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:35.223717 12685 consensus_queue.cc:1059] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 12:19:35.226195  6475 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:35.259586 12556 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 12:19:35.276242 12616 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:35.260773 12537 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:35.289587 12594 consensus_queue.cc:1059] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:35.289394 12481 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:35.290815  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:35.282939 12539 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:35.311517  6622 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:35.277626 12500 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:35.299743  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 73614.7us
I20260709 12:19:35.301340  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:35.298437  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:35.290556 12557 consensus_queue.cc:1048] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 12:19:35.292691 12654 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:35.295626 12584 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:35.295017  6505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2803) took 1120 ms. Trace:
I20260709 12:19:35.298864 12537 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:35.287984 12354 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:35.331835  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:35.337885  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:35.338831  6501 raft_consensus.cc:2368] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 14. Current term is 15.
W20260709 12:19:35.340843  6507 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.037s	sys 0.067s
I20260709 12:19:35.302642 12492 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:35.341212  6475 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:35.341584  6507 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.038s	sys 0.067s
I20260709 12:19:35.311529 12685 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:35.331036 12167 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:35.291085 12307 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:35.345772  6501 raft_consensus.cc:1275] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:19:35.345677  6475 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:35.354403  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:35.336549 12654 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:35.356002 12581 consensus_queue.cc:1048] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:19:35.335747  6505 rpcz_store.cc:276] 0709 12:19:34.174833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:34.502946 (+328113us) service_pool.cc:224] Handling call
0709 12:19:34.503284 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:34.503658 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:19:34.503943 (+   285us) log.cc:844] Serialized 14 byte log entry
0709 12:19:34.877453 (+373510us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:34.884883 (+  7430us) log.cc:844] Serialized 26 byte log entry
0709 12:19:34.885022 (+   139us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:19:34.885321 (+   299us) log.cc:844] Serialized 14 byte log entry
0709 12:19:35.293817 (+408496us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:35.294706 (+   889us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:35.294768 (+    62us) raft_consensus.cc:1685] finished
0709 12:19:35.294789 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:35.294949 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18319,"prepare.queue_time_us":723,"prepare.run_cpu_time_us":133,"prepare.run_wall_time_us":7480,"raft.queue_time_us":110056,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":28068,"spinlock_wait_cycles":267008,"thread_start_us":780999,"threads_started":2,"wal-append.queue_time_us":581671}
I20260709 12:19:35.381214 12687 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:35.340108 12537 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:35.339251 12641 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:35.342018 12215 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:35.345602 12685 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:35.330892 12539 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:35.421657  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 76098.6us
I20260709 12:19:35.357954 12386 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:35.425844  7265 raft_consensus.cc:1240] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 19. Current term is 20. Ops: []
W20260709 12:19:35.418406 12678 log.cc:927] Time spent T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.264s	user 0.001s	sys 0.001s
I20260709 12:19:35.429450 12685 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:35.437788 12581 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:35.438369 12581 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:35.438740 12581 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:35.448669 12616 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:35.451470 12616 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:35.452128 12683 log.cc:927] Time spent T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 12:19:35.452651  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:35.457110 12557 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:35.433643 12215 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:35.458930  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2797) took 1534 ms. Trace:
I20260709 12:19:35.459410  6508 rpcz_store.cc:276] 0709 12:19:33.924369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:34.340314 (+415945us) service_pool.cc:224] Handling call
0709 12:19:34.342775 (+  2461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:34.512417 (+169642us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:34.512455 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:34.512874 (+   419us) log.cc:844] Serialized 26 byte log entry
0709 12:19:34.864148 (+351274us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:34.864200 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:34.864294 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:35.458663 (+594369us) raft_consensus.cc:1685] finished
0709 12:19:35.458691 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:35.458858 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":387,"raft.queue_time_us":21677,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":5363,"thread_start_us":267497,"threads_started":2,"wal-append.queue_time_us":523906}
I20260709 12:19:35.462893  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2585) took 1111 ms. Trace:
I20260709 12:19:35.463343  6503 rpcz_store.cc:276] 0709 12:19:34.351787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:34.592316 (+240529us) service_pool.cc:224] Handling call
0709 12:19:34.593508 (+  1192us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:34.747874 (+154366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:34.747920 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:34.788680 (+ 40760us) log.cc:844] Serialized 26 byte log entry
0709 12:19:35.154963 (+366283us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:35.155007 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:35.155084 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:35.462666 (+307582us) raft_consensus.cc:1685] finished
0709 12:19:35.462688 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:35.462828 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8627,"raft.queue_time_us":11320,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":29474,"thread_start_us":322786,"threads_started":1,"wal-append.queue_time_us":602563}
I20260709 12:19:35.406740 12564 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:35.416776 12636 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.357712 12388 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 12:19:35.427095 12670 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:35.470945  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:35.472694  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:35.411196 12167 raft_consensus.cc:697] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:35.509199  6650 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
I20260709 12:19:35.514982  6658 raft_consensus.cc:1240] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:35.519451  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:35.520779  7267 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:35.514897  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:35.522782  7240 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:35.523517  6652 raft_consensus.cc:1275] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 12:19:35.533396  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 47746.3us
I20260709 12:19:35.533727  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:35.537374  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:35.538130  6656 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:35.468479 12386 raft_consensus.cc:697] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:35.542349  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:35.548799  6658 raft_consensus.cc:2393] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 17: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:35.546169  7274 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38562 (request call id 1034) took 1045 ms. Trace:
I20260709 12:19:35.554841  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:35.555265  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:35.556013  6654 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:35.549948  7274 rpcz_store.cc:276] 0709 12:19:34.501178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:34.501336 (+   158us) service_pool.cc:224] Handling call
0709 12:19:34.501669 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:34.744044 (+242375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:35.545853 (+801809us) spinlock_profiling.cc:243] Waited 93.3 ms on lock 0x7b5400507dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:35.546092 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":214676736}
W20260709 12:19:35.546116  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 389069us
I20260709 12:19:35.558125  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:35.558876  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:35.560092 12167 consensus_queue.cc:237] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:35.533334 12492 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.562070 12492 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:35.563474 12500 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:35.564008 12492 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:35.565569 12307 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:35.559553  6473 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:35.525240 12695 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:35.525166 12689 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:35.508517 12651 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:19:35.512024 12636 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:35.532912 12494 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:35.580974  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:35.533643 12681 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:35.586668  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:35.510689 12564 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:35.578410  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:35.696272  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:35.591071  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:35.698268  6652 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:35.677312 12515 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.033s	sys 0.031s
I20260709 12:19:35.594072 12689 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:35.457847 12557 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:35.529469 12670 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:35.638335  6500 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2795) took 1714 ms. Trace:
I20260709 12:19:35.469519 12687 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:35.677062  6576 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.024s	sys 0.040s
W20260709 12:19:35.722117  6576 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.025s	sys 0.040s
W20260709 12:19:35.641846 12688 log.cc:927] Time spent T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
I20260709 12:19:35.678247 12492 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.728804  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:35.664656 12386 consensus_queue.cc:237] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:35.566922 12654 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:35.678929 12215 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:35.590255 12695 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:35.679101 12481 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:35.684507 12598 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.711243 12518 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:19:35.705432 12515 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.033s	sys 0.032s
I20260709 12:19:35.718622  6500 rpcz_store.cc:276] 0709 12:19:33.923802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:34.338964 (+415162us) service_pool.cc:224] Handling call
0709 12:19:34.364658 (+ 25694us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:34.496458 (+131800us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:34.496488 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:34.496928 (+   440us) log.cc:844] Serialized 26 byte log entry
0709 12:19:34.862832 (+365904us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:34.931918 (+ 69086us) log.cc:844] Serialized 14 byte log entry
0709 12:19:35.075655 (+143737us) spinlock_profiling.cc:243] Waited 24.7 ms on lock 0x7b18002d3400. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d16c0a 00007f287c456219 00007f2872d9ea40 00007f2872d994e4 00007f2872d99455 00007f2875e81060 00007f2875e93716 00007f2875ea4287 00007f2875ea4208
0709 12:19:35.075832 (+   177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:35.238840 (+163008us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b540001ccd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:35.238892 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:35.638040 (+399148us) raft_consensus.cc:1685] finished
0709 12:19:35.638068 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:35.638256 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":94182,"prepare.queue_time_us":16969,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":1063,"raft.queue_time_us":147263,"raft.run_cpu_time_us":300,"raft.run_wall_time_us":36830,"spinlock_wait_cycles":360757632,"thread_start_us":349121,"threads_started":1,"wal-append.queue_time_us":571947}
I20260709 12:19:35.729596  6651 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
W20260709 12:19:35.754993  6504 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.003s	sys 0.090s
W20260709 12:19:35.755357  6504 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.003s	sys 0.090s
I20260709 12:19:35.755894  6651 raft_consensus.cc:1240] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 15. Current term is 16. Ops: []
W20260709 12:19:35.743623 12692 log.cc:927] Time spent T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 12:19:35.743929 12636 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:35.744300 12697 log.cc:927] Time spent T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 12:19:35.706815 12681 raft_consensus.cc:697] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 15 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:35.655229 12547 consensus_queue.cc:1059] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.068s
W20260709 12:19:35.734797 12639 consensus_meta.cc:287] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.762s	user 0.013s	sys 0.000s
I20260709 12:19:35.448081 12685 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:19:35.801363  6652 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:19:35.699472 12494 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:35.792232  6509 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.015s	sys 0.026s
W20260709 12:19:35.782610 12653 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.024s	sys 0.049s
I20260709 12:19:35.738411 12492 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:35.824930 12494 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:35.821115 12653 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.809s	user 0.025s	sys 0.049s
I20260709 12:19:35.825583 12492 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:35.787891 12694 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:35.816252  6509 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.015s	sys 0.026s
I20260709 12:19:35.755990 12518 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:35.818353  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 48459.1us
I20260709 12:19:35.840890  6623 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:35.753394 12598 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:35.820063  6656 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:19:35.849017  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 2465) took 1071 ms. Trace:
I20260709 12:19:35.849443  6580 rpcz_store.cc:276] 0709 12:19:34.777672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:34.778254 (+   582us) service_pool.cc:224] Handling call
0709 12:19:34.778542 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:34.787349 (+  8807us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:34.787402 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:34.788052 (+   650us) log.cc:844] Serialized 26 byte log entry
0709 12:19:35.291885 (+503833us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:35.291946 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:35.292043 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:35.848815 (+556772us) raft_consensus.cc:1685] finished
0709 12:19:35.848835 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:35.848961 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8295,"raft.queue_time_us":8955,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":12295,"thread_start_us":449286,"threads_started":1,"wal-append.queue_time_us":708057}
I20260709 12:19:35.821875 12547 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:35.810420 12703 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:35.834640 12492 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:35.774396  6654 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
W20260709 12:19:35.847532 12318 consensus_meta.cc:287] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.775s	user 0.009s	sys 0.004s
I20260709 12:19:35.806828 12639 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:35.842495 12709 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:35.853740 12494 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:35.918766  6622 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.017s	sys 0.073s
W20260709 12:19:35.919271  6622 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.017s	sys 0.073s
W20260709 12:19:35.919975  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 608496us
I20260709 12:19:35.921025  6622 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:19:35.893846 12472 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.941s	user 0.027s	sys 0.024s
I20260709 12:19:35.927954  6583 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 11.
I20260709 12:19:35.928457  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:35.908064 12711 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:35.844005 12694 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:35.947592  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 388051us
W20260709 12:19:35.926982 12472 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.974s	user 0.028s	sys 0.024s
I20260709 12:19:35.926687 12670 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:35.919036 12653 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.953414  6650 raft_consensus.cc:1240] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:35.952229  6575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 2472) took 1111 ms. Trace:
I20260709 12:19:35.957029  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:35.965020  6580 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:35.966126  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:35.967386  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:35.958173  6576 raft_consensus.cc:1275] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:35.958809  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:35.959735  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:35.961248  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:35.906049 12547 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:35.967473  6584 raft_consensus.cc:1240] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:35.926879 12564 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:35.938300 12693 log.cc:927] Time spent T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.245s	user 0.001s	sys 0.001s
I20260709 12:19:35.907424 12492 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:35.968179  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:35.964815  6575 rpcz_store.cc:276] 0709 12:19:34.840643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:34.840776 (+   133us) service_pool.cc:224] Handling call
0709 12:19:34.841095 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:34.913946 (+ 72851us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:34.913978 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:34.914019 (+    41us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:34.914052 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:35.951999 (+1037947us) spinlock_profiling.cc:243] Waited 72.6 ms on lock 0x7b54002666d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:35.952044 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:35.952163 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":167068928}
I20260709 12:19:35.973810  6584 raft_consensus.cc:1240] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 12:19:35.975230  6582 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:35.979955  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:35.949745 12354 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.000s	sys 0.020s
I20260709 12:19:35.983105  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:35.951565 12711 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:35.987592  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:35.956676 12689 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:35.957254 12645 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:19:35.927165 12494 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:35.971921 12167 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:35.966456 12481 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:35.919697 12709 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:35.977682 12653 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:36.020916 12537 consensus_meta.cc:287] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.586s	user 0.007s	sys 0.004s
I20260709 12:19:36.021621 12651 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.021697 12537 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:36.007293  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:36.009882 12700 log.cc:927] Time spent T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:19:36.010023 12492 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:36.004875 12167 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:35.922434 12681 consensus_queue.cc:237] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:36.012591 12518 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:36.013964 12694 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:36.014312 12547 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:36.014781 12481 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:36.006705 12318 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:35.994558 12711 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:35.977396 12515 consensus_queue.cc:1059] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:19:35.983696 12654 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:36.016446 12709 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:36.018010 12494 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:36.018561  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:36.019145  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2582) took 1669 ms. Trace:
I20260709 12:19:36.002230 12687 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:36.025583  6582 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:19:36.025970  6580 raft_consensus.cc:2368] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19.
I20260709 12:19:35.992988 12710 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:36.075384  6473 connection.cc:256] RPC call timeout handler was delayed by -1.2019682180000473s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 12:19:35.984730 12354 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.000s	sys 0.021s
I20260709 12:19:36.011469 12598 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:36.044968  7238 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.012s	sys 0.024s
I20260709 12:19:36.093739  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:19:36.094214  7238 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.012s	sys 0.024s
I20260709 12:19:36.094547  6582 raft_consensus.cc:2393] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 19: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:19:36.094655  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 568347us
I20260709 12:19:36.007773 12695 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:36.072431  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:36.096712  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:36.035998 12167 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:36.031431 12670 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.030447 12720 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:36.085158  6507 rpcz_store.cc:276] 0709 12:19:34.350151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:34.591212 (+241061us) service_pool.cc:224] Handling call
0709 12:19:34.591681 (+   469us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:34.762883 (+171202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:34.762915 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:34.770249 (+  7334us) log.cc:844] Serialized 26 byte log entry
0709 12:19:35.342831 (+572582us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:35.342875 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:35.342975 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:36.018893 (+675918us) raft_consensus.cc:1685] finished
0709 12:19:36.018916 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:36.019064 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34353,"raft.queue_time_us":65867,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":9162,"spinlock_wait_cycles":175488,"thread_start_us":514344,"threads_started":1,"wal-append.queue_time_us":773769}
W20260709 12:19:36.089421  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:36.093719  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:36.074474 12694 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:36.073884 12557 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:36.022593 12651 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:36.105648 12689 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.077916 12318 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:36.137456  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:36.137584 12689 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:36.138181  6654 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:19:36.139145 12318 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:36.107965 12481 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:36.110641 12581 consensus_meta.cc:287] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.671s	user 0.011s	sys 0.000s
I20260709 12:19:36.139724 12689 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:36.112600 12639 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:36.089123 12685 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:36.114010  6473 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:36.140684  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:36.141021  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:36.142220  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:36.143206 12388 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:36.114500 12717 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:36.115710  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:36.145269  7265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38546 (request call id 701) took 1742 ms. Trace:
I20260709 12:19:36.145706 12717 consensus_queue.cc:1059] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:36.145615  7265 rpcz_store.cc:276] 0709 12:19:34.402893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:34.403186 (+   293us) service_pool.cc:224] Handling call
0709 12:19:34.403471 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:35.434749 (+1031278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:36.144919 (+710170us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54006c2354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:36.145188 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2350557952}
I20260709 12:19:36.147092 12717 raft_consensus.cc:3055] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:36.147256  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:36.147511 12717 raft_consensus.cc:740] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:36.147944  6651 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:36.143607  6473 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:36.149087  6473 leader_election.cc:400] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19.
I20260709 12:19:36.119670 12701 raft_consensus.cc:3055] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:36.150617  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:36.150828 12701 raft_consensus.cc:740] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:36.151077 12717 consensus_queue.cc:260] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:36.151928 12701 consensus_queue.cc:260] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:36.152683 12717 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:36.104712  7238 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:36.153546  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 48945.8us
I20260709 12:19:36.153657  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:36.150743  6473 leader_election.cc:403] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:19:36.156554 12705 log.cc:927] Time spent T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 12:19:36.156761 12547 consensus_queue.cc:1048] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:36.098459 12710 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:36.156323 12639 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.158252 12639 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:36.158712  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:36.159327 12710 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:36.159687 12689 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:19:36.098292 12687 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.127303 12720 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:36.130623 12670 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:36.135074  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:36.139501  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:36.163681  6550 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:36.143868 12388 consensus_queue.cc:1059] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:36.149187  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:36.149605 12481 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:36.140043 12581 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:36.174293 12611 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:36.105693 12521 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.175565 12581 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:36.175479 12521 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:36.176126  7269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38558 (request call id 857) took 1143 ms. Trace:
I20260709 12:19:36.092061 12515 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.160974  6652 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:36.166771 12557 consensus_queue.cc:1059] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 12:19:36.166500  6655 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:36.178912 12515 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:36.179414 12557 raft_consensus.cc:3055] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:36.179826 12557 raft_consensus.cc:740] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:36.180692 12557 consensus_queue.cc:260] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:36.181479 12515 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:36.182116 12557 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:36.165462 12694 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:36.184046  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:36.184764  6579 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:36.186071  6473 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:36.187058 12694 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:36.161320  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:36.162236  6582 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:19:36.187923  7268 raft_consensus.cc:2393] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 14: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:36.163344  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:36.168185 12494 consensus_queue.cc:1059] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 12:19:36.189304 12652 raft_consensus.cc:3055] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:36.126439 12539 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:36.159420  6576 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:36.165730  6654 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:36.140303  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:36.189800 12652 raft_consensus.cc:740] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:36.189847 12481 raft_consensus.cc:697] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:36.190055  7238 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:36.171180  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:36.190851 12652 consensus_queue.cc:260] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:36.171389  6550 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:36.176746 12664 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:36.192322 12388 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:36.192834 12388 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:36.192834 12664 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:36.194954  6473 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:36.193265 12388 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:36.196282  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:36.195659 12664 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:36.196944  6583 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:19:36.197352 12588 raft_consensus.cc:3055] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:36.197783 12588 raft_consensus.cc:740] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:36.198161 12515 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:36.198585 12515 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:36.198822 12539 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:36.198549 12588 consensus_queue.cc:260] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:36.199229 12515 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:36.199633 12588 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:36.191026 12481 consensus_queue.cc:237] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:36.202839  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:36.198668  6550 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:36.205430 12539 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:36.205826 12539 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:36.205912  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:36.206303 12539 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:36.160225 12639 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:36.208616 12507 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:36.210220  7265 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:36.212262  6550 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:36.153035 12701 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:36.214448  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:36.176627  7269 rpcz_store.cc:276] 0709 12:19:35.032300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:35.032501 (+   201us) service_pool.cc:224] Handling call
0709 12:19:35.032890 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:35.073191 (+ 40301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:35.073235 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:35.073297 (+    62us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:35.073344 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:36.175877 (+1102533us) spinlock_profiling.cc:243] Waited 40 ms on lock 0x7b54005de654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:36.175923 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:36.176047 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":92081536}
I20260709 12:19:36.213572  6550 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:36.219146  6474 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:36.221791  6474 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:36.172945  7272 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:19:36.228091 12611 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:36.227931  6549 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:36.160913 12687 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:36.230198 12591 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:36.230706 12591 raft_consensus.cc:697] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:36.219975  6550 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:36.232743  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:36.099215 12537 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:36.242089  6583 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:19:36.247589  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:36.177861 12521 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:36.233281 12494 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:36.233414 12567 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:36.231946 12272 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:36.238034 12581 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:19:36.254683  6474 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:36.255569  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:36.255738 12591 consensus_queue.cc:237] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:36.260975 12673 log.cc:927] Time spent T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 1.348s	user 0.000s	sys 0.010s
I20260709 12:19:36.268260 12729 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.317610 12729 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:36.319931 12729 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:36.273953 12651 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:36.218525 12500 consensus_queue.cc:1059] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
W20260709 12:19:36.302762  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 374334us
I20260709 12:19:36.358379  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:36.314283 12636 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:36.317034  6502 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2788) took 2395 ms. Trace:
I20260709 12:19:36.361975  6502 rpcz_store.cc:276] 0709 12:19:33.921776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:34.329587 (+407811us) service_pool.cc:224] Handling call
0709 12:19:34.336844 (+  7257us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:34.343468 (+  6624us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:19:34.367424 (+ 23956us) log.cc:844] Serialized 14 byte log entry
0709 12:19:35.058816 (+691392us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:35.064176 (+  5360us) log.cc:844] Serialized 26 byte log entry
0709 12:19:35.064377 (+   201us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:35.064900 (+   523us) log.cc:844] Serialized 14 byte log entry
0709 12:19:35.076485 (+ 11585us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:35.076585 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:35.250397 (+173812us) spinlock_profiling.cc:243] Waited 2.22 ms on lock 0x7b48000d0a58. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287adb2aa1 00007f287ada9b37 00007f287ada99ca 00007f287aea126f 00007f287aea0f37 00007f287aeb555c 00007f287aeb5508 00007f287aeb5488 00007f287aeb5443 00007f287aeb46ec 00007f2879ddea96
0709 12:19:36.244567 (+994170us) spinlock_profiling.cc:243] Waited 2.33 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:36.316801 (+ 72234us) raft_consensus.cc:1685] finished
0709 12:19:36.316824 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:36.316964 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43877,"prepare.queue_time_us":42457,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":101,"raft.queue_time_us":175590,"raft.run_cpu_time_us":2685,"raft.run_wall_time_us":175056,"spinlock_wait_cycles":10464384,"thread_start_us":631134,"threads_started":2,"wal-append.queue_time_us":527272}
I20260709 12:19:36.269062 12687 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:36.324934  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:36.326962 12539 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:36.334755  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 80181.3us
I20260709 12:19:36.335534 12694 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:36.337337 12720 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:36.341480 12670 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:36.344735 12492 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:36.345815 12653 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:36.250593 12652 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:36.275617 12598 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:19:36.305302 12704 log.cc:927] Time spent T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.240s	user 0.000s	sys 0.003s
I20260709 12:19:36.300855 12494 raft_consensus.cc:697] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:36.384968  6580 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:36.390544  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:36.374341 12639 raft_consensus.cc:3055] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:36.365937 12611 raft_consensus.cc:697] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:36.403609  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:36.404058  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:36.405391  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:36.408545  6502 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:36.409291  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:36.409551  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:36.411757  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:36.386250  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 166339us
I20260709 12:19:36.366137 12636 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:36.404667 12639 raft_consensus.cc:740] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:36.417232  6501 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:36.384754 12724 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:36.414764  6508 raft_consensus.cc:1275] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:19:36.425601  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:36.425838  6550 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:36.384968 12511 consensus_queue.cc:1059] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 14.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:19:36.426236  6506 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:36.414960  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:36.427412  6575 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:36.429528  6581 raft_consensus.cc:2376] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14. Re-sending same reply.
I20260709 12:19:36.405750 12598 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:36.443295 12598 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:36.444700 12639 consensus_queue.cc:260] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:36.445194 12711 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:36.445976 12639 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:36.447276 12711 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:36.448685 12611 consensus_queue.cc:237] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:36.448073  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:36.456563 12724 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:36.457042 12724 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:36.460824 12515 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:36.461449 12562 raft_consensus.cc:3055] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:36.461838 12562 raft_consensus.cc:740] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:36.462574 12562 consensus_queue.cc:260] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:36.463136 12515 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:36.463536 12562 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:36.186077  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:36.465322 12539 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:36.467734  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 233046us
I20260709 12:19:36.468384  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:36.468746  7271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38562 (request call id 1042) took 1356 ms. Trace:
I20260709 12:19:36.469015  6503 raft_consensus.cc:1275] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:19:36.469499 12494 consensus_queue.cc:237] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.7, Last appended by leader: 7, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:19:36.470044  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:36.470742  6503 raft_consensus.cc:2368] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18.
I20260709 12:19:36.471490  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:36.472666  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:36.473373 12229 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:19:36.474092 12706 log.cc:927] Time spent T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.449s	user 0.001s	sys 0.000s
I20260709 12:19:36.473948 12229 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:36.476076 12229 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:36.437031 12636 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:36.438217 12672 raft_consensus.cc:3055] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:36.439217  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:36.439517  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:36.440459  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:36.509483  6583 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:36.440609  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:36.510340  6579 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:36.441885  6550 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:36.511701  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 69869us
I20260709 12:19:36.442847  6504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2586) took 2090 ms. Trace:
I20260709 12:19:36.433262 12743 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:36.488049  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:36.489585  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:36.519429  6503 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:36.400746 12492 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:36.480569  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:36.477257 12388 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:36.491318  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:36.477300 12572 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:36.566191  6548 leader_election.cc:400] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18.
I20260709 12:19:36.566660  6548 leader_election.cc:403] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:36.469144  7271 rpcz_store.cc:276] 0709 12:19:35.113940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:35.114069 (+   129us) service_pool.cc:224] Handling call
0709 12:19:35.114354 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:35.554731 (+440377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:35.554776 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:35.554833 (+    57us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:35.554877 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:36.468531 (+913654us) spinlock_profiling.cc:243] Waited 440 ms on lock 0x7b5400565754. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:36.468577 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:36.468694 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1012202880}
I20260709 12:19:36.568068  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:36.481161 12687 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:36.568240  6548 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:36.576602  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:36.577473  6583 raft_consensus.cc:2393] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 17: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:36.476622 12694 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:36.484941 12567 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:36.487772  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:36.495728  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:36.583526  6575 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:36.503026 12354 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.504525 12511 raft_consensus.cc:3055] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:36.507330  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:36.513375  6504 rpcz_store.cc:276] 0709 12:19:34.352102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:34.757022 (+404920us) service_pool.cc:224] Handling call
0709 12:19:34.787589 (+ 30567us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:35.240787 (+453198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:35.240822 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:35.241283 (+   461us) log.cc:844] Serialized 26 byte log entry
0709 12:19:35.755844 (+514561us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:35.755902 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:35.839338 (+ 83436us) spinlock_profiling.cc:243] Waited 453 ms on lock 0x7b540039ec54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:35.839402 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:36.271763 (+432361us) spinlock_profiling.cc:243] Waited 3.09 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:36.442635 (+170872us) raft_consensus.cc:1685] finished
0709 12:19:36.442658 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:36.442789 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13300,"raft.run_cpu_time_us":70,"raft.run_wall_time_us":69,"spinlock_wait_cycles":1048536576,"thread_start_us":514114,"threads_started":1,"wal-append.queue_time_us":752231}
I20260709 12:19:36.516687  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:36.512894  6550 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:36.517426 12743 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:36.522971  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:36.529788  6475 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:36.539592  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:36.560122  6579 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
W20260709 12:19:36.596306  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 66654.2us
I20260709 12:19:36.569936  6657 raft_consensus.cc:1240] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:19:36.552839  6501 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:36.553951  6473 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:19:36.597893  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 44032.9us
I20260709 12:19:36.574841  6506 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:19:36.578006 11462 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:36.578219  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:36.578886  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:36.599779  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:36.579658  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:36.600212  6503 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:36.580892  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:36.601176  6576 raft_consensus.cc:2410] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:19:36.603873  6576 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:19:36.585091  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:36.588917  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:36.589939  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:36.521289 12672 raft_consensus.cc:740] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:19:36.591328  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 84076.3us
I20260709 12:19:36.521837  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:36.591980  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 79170.2us
I20260709 12:19:36.611434  6655 raft_consensus.cc:2368] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19.
I20260709 12:19:36.593984  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:36.538477 12636 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:36.617928  6622 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:36.619971 12664 consensus_queue.cc:1048] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:36.622275 12664 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:36.597106  6475 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:36.598275  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:36.581862  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:36.619540  6622 leader_election.cc:400] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 14. Current term is 15.
I20260709 12:19:36.595259  6509 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
I20260709 12:19:36.625082  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:36.624212  6622 leader_election.cc:403] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:19:36.627020  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:36.614338  6649 raft_consensus.cc:2393] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 19: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:36.566790 12717 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.615082  6502 raft_consensus.cc:2393] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 11: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:36.610754 11462 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:36.612699  6550 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:36.616246  6651 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:36.627622  6653 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:36.584348 12588 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18.
I20260709 12:19:36.583973 12738 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:36.646868  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 49842.2us
I20260709 12:19:36.592221 12653 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:36.587320 12567 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:36.661396  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:36.664722  6506 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:36.662276  6473 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:36.662875  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:36.669245  6501 raft_consensus.cc:2393] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 15: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:36.670114  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:36.669279  6579 raft_consensus.cc:1275] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:19:36.671841  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:36.595785 12711 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:36.590456 12511 raft_consensus.cc:740] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:36.586313 12500 consensus_queue.cc:1048] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.077s
I20260709 12:19:36.616956 12636 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:36.657562 12710 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 14. Current term is 15.
I20260709 12:19:36.586015 12167 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:36.611474 12651 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:36.661512  6575 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:19:36.702925  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:36.664078 12653 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:36.615569 12703 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:36.610651 12743 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:36.685083  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:36.687523  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:36.690621  6622 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:19:36.676548  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 81448.8us
I20260709 12:19:36.611279 12671 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:36.588215 12687 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:36.679677  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:36.595476 12694 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:36.801025  6502 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:36.789942  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 76320.5us
I20260709 12:19:36.705964  6506 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 18.
I20260709 12:19:36.594758 12354 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:36.659560 12717 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:36.789440  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:36.803687  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:36.664613 12318 consensus_queue.cc:1048] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:19:36.813385 12721 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:36.700717 12636 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:36.707587 12672 consensus_queue.cc:260] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:36.814208  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:36.814944 12672 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:36.815173 12726 log.cc:927] Time spent T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
W20260709 12:19:36.817001 12746 log.cc:927] Time spent T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 12:19:36.818463 12694 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:36.804792  6503 raft_consensus.cc:2393] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 17: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:36.821537  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:36.822180  6503 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:36.823016  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:36.823906  6503 raft_consensus.cc:2393] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 11: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:36.825233 12492 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:36.665395 12386 consensus_queue.cc:1059] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:19:36.826303  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:36.687203 12588 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:36.827214  6503 raft_consensus.cc:2393] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 15: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:36.705681 12724 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:36.827378 12588 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:36.827847 12588 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:36.829879 12636 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:36.832657  6652 raft_consensus.cc:1240] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:19:36.833144  6502 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:19:36.833571 12167 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:36.834168  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:36.834851  6649 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:36.711609 12653 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:36.828090  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:36.836742 12689 consensus_meta.cc:287] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.677s	user 0.010s	sys 0.000s
I20260709 12:19:36.837195 12689 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19.
I20260709 12:19:36.837598  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:36.838824  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:36.839915  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:36.841291 12636 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:36.843199  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:36.844295  6500 raft_consensus.cc:2393] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 19: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:36.845322  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:36.806725  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:36.810182  6654 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:36.812153  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:36.850080  6506 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
W20260709 12:19:36.808553  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 117799us
I20260709 12:19:36.846730  6507 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:19:36.810110 12703 raft_consensus.cc:697] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:36.851074  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:36.851732  6651 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:36.851909  6506 raft_consensus.cc:2393] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 17: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:36.846257 12717 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:36.706857 12651 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:36.854679  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:36.855912  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:36.857090  6506 raft_consensus.cc:2393] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 17: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:36.852005  6505 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:19:36.815062  6504 raft_consensus.cc:2410] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:19:36.834857  6502 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 11.
I20260709 12:19:36.852941  6500 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:19:36.859267  6500 raft_consensus.cc:1240] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:36.865757  6503 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:36.673951 12738 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:36.872416  6501 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:36.874076  6509 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18. Ops: []
W20260709 12:19:36.876319  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 63892.1us
I20260709 12:19:36.826210 12354 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:36.881537  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:36.854714 12564 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:36.811102 12710 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:36.881465 11462 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:36.947989  6579 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:19:36.855420 12386 raft_consensus.cc:3055] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:36.861341 12651 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:36.888222  6622 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:36.959172  6501 raft_consensus.cc:2410] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 16 because replica has last-logged OpId of term: 15 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 6.
I20260709 12:19:36.812778 12695 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:36.963189  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:36.992110  6654 raft_consensus.cc:2393] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 16: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:19:36.964788  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 302592us
W20260709 12:19:36.960817  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 72732.3us
I20260709 12:19:36.994707  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:36.965745 12703 consensus_queue.cc:237] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:36.966732 12748 log.cc:927] Time spent T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.265s	user 0.003s	sys 0.000s
I20260709 12:19:36.962955  6651 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:19:36.861974 12711 consensus_queue.cc:1048] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.086s
I20260709 12:19:36.947959 12681 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:36.941602 12564 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:37.000588  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:36.996503 12588 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:37.003000 12564 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:37.003801 12588 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:36.894780 12167 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.102s
I20260709 12:19:36.962594 12386 raft_consensus.cc:740] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:19:37.016872  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 403784us
I20260709 12:19:36.991726 12641 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:37.029987  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:36.961684 12743 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:37.044622  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:37.049510 12567 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:37.051331  6403 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57888 (request call id 217) took 45859 ms (client timeout 15000 ms). Trace:
I20260709 12:19:37.052189 12567 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:37.017987 12621 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:37.052757  6656 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:37.021697 12388 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:36.960804 12511 consensus_queue.cc:260] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:37.035490 12701 consensus_meta.cc:287] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.804s	user 0.004s	sys 0.007s
I20260709 12:19:37.031620 12615 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:37.076694  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:37.079537  6508 raft_consensus.cc:2410] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:19:37.058315 12594 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.017572 12681 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:37.094805  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:37.095475  6652 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:37.058157 12621 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:37.061677 12724 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:37.053658  6403 rpcz_store.cc:269] 0709 12:18:51.191636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:51.191867 (+   231us) service_pool.cc:224] Handling call
0709 12:19:37.051210 (+45859343us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:32.806684 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:32.807440 (+   756us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:32.807505 (+    65us) write_op.cc:435] Acquired schema lock
0709 12:19:32.807532 (+    27us) tablet.cc:662] Decoding operations
0709 12:19:32.808841 (+  1309us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:32.808955 (+   114us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:32.808987 (+    32us) tablet.cc:685] Acquiring locks for 11 operations
0709 12:19:32.810131 (+  1144us) tablet.cc:701] Row locks acquired
0709 12:19:32.810191 (+    60us) write_op.cc:260] PREPARE: finished
0709 12:19:32.810384 (+   193us) write_op.cc:270] Start()
0709 12:19:32.810578 (+   194us) write_op.cc:276] Timestamp: P: 1783599572810310 usec, L: 0
0709 12:19:32.810610 (+    32us) op_driver.cc:348] REPLICATION: starting
0709 12:19:32.811386 (+   776us) log.cc:844] Serialized 4304 byte log entry
0709 12:19:33.804255 (+992869us) op_driver.cc:464] REPLICATION: finished
0709 12:19:34.018173 (+213918us) write_op.cc:301] APPLY: starting
0709 12:19:34.034100 (+ 15927us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:34.286402 (+252302us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:34.286545 (+   143us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:34.833375 (+546830us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:34.937349 (+103974us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=0,mrs_lookups=0
0709 12:19:34.937400 (+    51us) write_op.cc:312] APPLY: finished
0709 12:19:34.937776 (+   376us) log.cc:844] Serialized 105 byte log entry
0709 12:19:35.356759 (+418983us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:35.410228 (+ 53469us) write_op.cc:454] Released schema lock
0709 12:19:35.412553 (+  2325us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:35.413095 (+   542us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:35.413127 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":410586,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":196178,"apply.run_cpu_time_us":55349,"apply.run_wall_time_us":1399318,"cfile_cache_hit":46,"cfile_cache_hit_bytes":50019,"num_ops":11,"prepare.queue_time_us":401449,"prepare.run_cpu_time_us":38773,"prepare.run_wall_time_us":254426,"raft.queue_time_us":419580,"raft.run_cpu_time_us":23844,"raft.run_wall_time_us":144822,"replication_time_us":993466,"thread_start_us":1156652,"threads_started":5,"wal-append.queue_time_us":849606,"wal-append.run_cpu_time_us":42463,"wal-append.run_wall_time_us":2927371}]]}
W20260709 12:19:37.089107  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:37.109392  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 307118us
I20260709 12:19:37.110293  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:37.116787  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:37.117404 12751 log.cc:927] Time spent T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
I20260709 12:19:37.119920 12386 consensus_queue.cc:260] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:37.121433 12386 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:37.124078  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:37.124794  6505 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:37.125563 12743 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:37.126021 12681 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:37.126783 12687 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:37.127466 12711 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:37.109514 12354 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:37.128060 12711 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:37.128803 12354 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:37.130630 12711 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:37.130932 12354 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:37.131258 12711 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:37.132282  6505 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:37.115157 12621 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:37.134411  6649 raft_consensus.cc:1240] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:37.141222  6509 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:19:37.142730  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:37.147047  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:37.155917  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:37.164969  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:37.170261  6652 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:37.082042 12615 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:37.089794 12594 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:37.131860 12711 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:37.271610  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:37.278724  6500 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:37.207100 12682 consensus_queue.cc:1059] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.085s
I20260709 12:19:37.286944 12711 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:37.307439  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 369699us
W20260709 12:19:37.309103  6650 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.020s	sys 0.038s
W20260709 12:19:37.310324  6650 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.020s	sys 0.038s
W20260709 12:19:37.311866  6475 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.010s	sys 0.047s
W20260709 12:19:37.313076  6475 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.010s	sys 0.047s
W20260709 12:19:37.313445  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 639262us
I20260709 12:19:37.314659  6475 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:37.296741 12738 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:37.306941 12562 consensus_meta.cc:287] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.843s	user 0.014s	sys 0.000s
W20260709 12:19:37.308705 12593 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.804s	user 0.023s	sys 0.034s
I20260709 12:19:37.343317 12567 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:37.344267 12694 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:37.336287 12672 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:37.337704  6475 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:37.371673  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:37.363808 12593 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.858s	user 0.024s	sys 0.034s
I20260709 12:19:37.375991  6622 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:37.376355  6475 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:37.392186  6475 leader_election.cc:400] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19.
I20260709 12:19:37.393013  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:19:37.391839 12598 consensus_meta.cc:287] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.947s	user 0.012s	sys 0.000s
I20260709 12:19:37.402707  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:37.396410 12671 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:37.367126 12567 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:37.411734  6475 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:37.400507 12641 raft_consensus.cc:697] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:19:37.408298  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:37.396620 12711 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:37.370157 12694 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:37.403628 12562 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:37.409134  6622 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:37.434440  6475 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:37.420120 12671 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 12:19:37.404747 12598 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:37.427526 12758 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:19:37.467825  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 448888us
W20260709 12:19:37.452749  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 43545.4us
I20260709 12:19:37.457120 12687 consensus_queue.cc:1059] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:37.457784 12672 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.458446  6475 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:37.459146 12681 raft_consensus.cc:3055] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:37.483744 12681 raft_consensus.cc:740] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:37.488445  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:37.489342  6652 raft_consensus.cc:2410] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 17 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 6.
I20260709 12:19:37.490132  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 2487) took 1052 ms. Trace:
I20260709 12:19:37.490547 12547 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:37.491168 12547 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:37.490512  6581 rpcz_store.cc:276] 0709 12:19:36.437958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:36.479550 (+ 41592us) service_pool.cc:224] Handling call
0709 12:19:36.479842 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:36.480039 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:19:36.542077 (+ 62038us) log.cc:844] Serialized 14 byte log entry
0709 12:19:37.378326 (+836249us) log.cc:844] Serialized 14 byte log entry
0709 12:19:37.489796 (+111470us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:37.489846 (+    50us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:37.489895 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.489958 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:37.490077 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":653170,"prepare.queue_time_us":703383,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":79,"raft.queue_time_us":188905,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":12080,"thread_start_us":354824,"threads_started":3,"wal-append.queue_time_us":143027}
W20260709 12:19:37.461038 12766 log.cc:927] Time spent T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:19:37.493762 12547 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:37.493835  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2644) took 1011 ms. Trace:
I20260709 12:19:37.461556 12500 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:37.409061 12664 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 12:19:37.494504 12500 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:37.465302  6509 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:37.496496 12500 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:37.471531  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:37.467290  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:37.456990 12729 consensus_queue.cc:1059] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:37.499146 12729 raft_consensus.cc:3055] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:37.499588 12729 raft_consensus.cc:740] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:37.500492 12729 consensus_queue.cc:260] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:37.501748 12729 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:37.502799 12694 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:37.503782 12651 consensus_meta.cc:287] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.509s	user 0.006s	sys 0.000s
W20260709 12:19:37.497563  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:37.505564 12598 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:37.473328 12615 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:37.509601 12594 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:37.467617 12763 log.cc:927] Time spent T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
W20260709 12:19:37.511219  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:37.511266  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2824) took 1039 ms. Trace:
I20260709 12:19:37.511934 12672 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:37.511826  6507 rpcz_store.cc:276] 0709 12:19:36.472220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:36.871218 (+398998us) service_pool.cc:224] Handling call
0709 12:19:36.878633 (+  7415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:36.878994 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:36.879037 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:36.879561 (+   524us) log.cc:844] Serialized 26 byte log entry
0709 12:19:37.031746 (+152185us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:37.056227 (+ 24481us) log.cc:844] Serialized 14 byte log entry
0709 12:19:37.057189 (+   962us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.057301 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:37.511032 (+453731us) raft_consensus.cc:1685] finished
0709 12:19:37.511057 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:37.511196 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41518,"prepare.queue_time_us":25110,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":769,"raft.run_cpu_time_us":449,"raft.run_wall_time_us":23471,"thread_start_us":151852,"threads_started":1,"wal-append.queue_time_us":433165}
W20260709 12:19:37.472759 12652 consensus_meta.cc:287] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 1.062s	user 0.002s	sys 0.010s
W20260709 12:19:37.497496 12653 consensus_meta.cc:287] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.662s	user 0.011s	sys 0.000s
I20260709 12:19:37.514681  6655 raft_consensus.cc:1240] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:19:37.514469 12653 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:37.514814  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 56450.6us
I20260709 12:19:37.479262  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:37.515671  6580 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:37.480672 12564 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:37.514110 12672 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:37.516485 12564 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:37.516914 12672 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:37.518499 12564 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:37.479538  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:37.469565  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:37.455053 12578 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:37.494266  6584 rpcz_store.cc:276] 0709 12:19:36.482607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:36.614926 (+132319us) service_pool.cc:224] Handling call
0709 12:19:36.615344 (+   418us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:36.965443 (+350099us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:36.965481 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:36.966028 (+   547us) log.cc:844] Serialized 52 byte log entry
0709 12:19:37.235685 (+269657us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:37.235747 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.235844 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:37.493588 (+257744us) raft_consensus.cc:1685] finished
0709 12:19:37.493609 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:37.493778 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":58019,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":5295,"thread_start_us":618821,"threads_started":2,"wal-append.queue_time_us":422411}
I20260709 12:19:37.415889 12641 consensus_queue.cc:237] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:37.530746  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:37.515607  6475 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:37.532260  6475 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:37.464933 12760 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:37.480216  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:37.535948  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:37.477226  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:37.536713  7274 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 20: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:37.537325  6656 raft_consensus.cc:2410] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:19:37.525058  6580 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:19:37.526510  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:37.526594  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:37.538853  6655 raft_consensus.cc:2393] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 12: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:37.514963 12584 raft_consensus.cc:3055] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:37.533438  6475 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:37.520772  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 51284.3us
I20260709 12:19:37.543066  6622 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:37.544957  6475 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:37.545998  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:37.547225  7238 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:37.548189  6548 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:37.549266  6548 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:37.553741  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:37.554287  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:37.557704  6652 raft_consensus.cc:1275] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:19:37.561169  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:37.497471 12567 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:37.564826  6622 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:37.504153 12651 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:37.527881 12653 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:37.563279 12758 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:37.531490 12664 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:37.529181 12652 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:37.565403  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:37.534890 12671 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:37.570233  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:37.561949 12703 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:37.563585 12680 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:37.575788  6655 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:37.542454 12584 raft_consensus.cc:740] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:37.508131 12621 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:37.616297  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:37.573895 12664 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:37.577158 12758 raft_consensus.cc:697] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 17 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:37.577765 12652 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:37.569592 12681 consensus_queue.cc:260] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:37.568611 12738 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:37.571123 12722 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:37.564193 12594 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:37.706554  6548 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:37.732113  6548 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:37.708745  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:37.567083 12481 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:37.572142 12167 consensus_queue.cc:1048] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 12:19:37.569072 12578 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:37.563474 12547 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:37.709640  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:37.709693  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:37.575237 12724 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 12:19:37.713403  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:37.719442 12680 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:37.715758  6578 raft_consensus.cc:2393] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 17: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:37.720633  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:37.846681  6651 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:37.560292 12581 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:19:37.770321  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:37.828159  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 96107.5us
I20260709 12:19:37.727807 12664 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:37.850828  6548 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:37.828715  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:37.726761 12386 consensus_meta.cc:287] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.605s	user 0.012s	sys 0.006s
I20260709 12:19:37.719295 12701 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.705703 12682 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:37.714376  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:37.854597  6651 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:37.835309 12653 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:37.733342 12760 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:37.836010 12481 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:37.838301 12593 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.854326 12581 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:37.877540  6649 raft_consensus.cc:1275] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 17 index: 7. Preceding OpId from leader: term: 19 index: 8. (index mismatch)
I20260709 12:19:37.840247 12754 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:37.849923 12680 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:37.850070 12711 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:19:37.874840 12768 log.cc:927] Time spent T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.330s	user 0.000s	sys 0.001s
I20260709 12:19:37.893163  6548 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:19:37.943475  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 50445.9us
W20260709 12:19:37.945153  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 235530us
I20260709 12:19:37.883719 12229 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:37.890926 12636 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:37.998587  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:37.985709  6649 raft_consensus.cc:1240] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:37.989078  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:37.938129 12615 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.938298 12777 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:37.903045 12754 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:38.041620  6622 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:37.896143 12481 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:37.996523 12752 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:38.113797 12752 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:38.114205 12752 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:38.114099  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 72574.2us
I20260709 12:19:38.115432 12562 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:38.080686  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:37.888783 12682 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:37.834820 12564 consensus_queue.cc:1175] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f2b06aebe97249a4877d798b8118ece4" responder_term: 16 status { last_received { term: 14 index: 11 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 15. Current term is 16. Ops: [15.12-15.12]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:19:38.072193  6508 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:19:38.119594  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:38.119932  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:37.903441 12581 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:38.122666  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:38.126978  6651 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:37.996713 12711 raft_consensus.cc:697] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:37.897753 12760 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:37.997366 12651 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:38.132212  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:38.132972  6622 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:38.140031  7271 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:38.042728 12167 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:38.124565  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:38.118062 12773 log.cc:927] Time spent T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.392s	user 0.000s	sys 0.001s
W20260709 12:19:38.089658 12775 log.cc:927] Time spent T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.232s	user 0.000s	sys 0.000s
I20260709 12:19:38.121344 12754 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:38.088995 12774 log.cc:927] Time spent T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.361s	user 0.000s	sys 0.000s
I20260709 12:19:38.037528 12594 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.133396  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:38.143033 12785 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.145349 12760 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:38.038694 12229 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:38.123687 12768 log.cc:927] Time spent T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 12:19:38.158782  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2620) took 1019 ms. Trace:
I20260709 12:19:37.943302 12593 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:38.194473  6623 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:38.200963  6651 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
W20260709 12:19:38.205073  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 51339.6us
I20260709 12:19:38.248562  6509 rpcz_store.cc:276] 0709 12:19:37.138900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.273946 (+135046us) service_pool.cc:224] Handling call
0709 12:19:37.465073 (+191127us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:37.496589 (+ 31516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:38.158511 (+661922us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b54002c5bd0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:38.158700 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":438607232}
I20260709 12:19:37.885485 12701 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:38.139950 12729 consensus_meta.cc:287] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.637s	user 0.009s	sys 0.008s
I20260709 12:19:38.113248 12615 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:38.092679 12694 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:38.242858  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:19:38.250698  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 56344.2us
I20260709 12:19:38.268040  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:38.206970 12772 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:19:38.282042  6655 consensus_meta.cc:287] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.566s	user 0.010s	sys 0.000s
I20260709 12:19:38.282537  6655 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:38.283461  6658 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:19:38.033816 12758 consensus_queue.cc:237] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:38.290236  6649 raft_consensus.cc:2393] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 17: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:38.211516 12652 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:38.291414  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 295742us
I20260709 12:19:38.291611  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:38.293715  6508 raft_consensus.cc:1240] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:38.298856  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:38.299561  6508 raft_consensus.cc:2376] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 18. Re-sending same reply.
I20260709 12:19:38.294808  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:38.300741  6500 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:38.297096  6657 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:19:38.301741  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:38.302650  6508 raft_consensus.cc:2368] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 15. Current term is 16.
I20260709 12:19:38.151796 12664 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:38.307578  6649 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:19:38.309999  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:38.314903  6649 raft_consensus.cc:1275] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:19:38.211771 12167 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:38.315004  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:38.318795 12651 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.319341 12651 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:38.125535 12777 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:38.320644  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:38.272388 12567 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:38.277158 12621 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:38.151798 12752 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:38.328462  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:38.329032  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:38.333415  6508 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:38.241091 12594 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:38.336794  6652 raft_consensus.cc:2368] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18.
I20260709 12:19:38.337832  6475 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:38.270970 12318 consensus_queue.cc:1059] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 12:19:38.304535 12722 raft_consensus.cc:697] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:38.304968 12729 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:19:38.343602  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:38.343786  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:38.312145 12721 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:38.251786 12785 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:38.313129 12572 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:38.359458  6509 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
I20260709 12:19:38.360113  6500 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 12.
I20260709 12:19:38.360780  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:38.361445  6509 raft_consensus.cc:2376] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19. Re-sending same reply.
I20260709 12:19:38.366623  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:38.327603 12695 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.367390 12695 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:38.361297  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:38.297715 12694 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:38.278053 12711 consensus_queue.cc:237] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:38.381281  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 113340us
I20260709 12:19:38.382042  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:38.383123  6505 raft_consensus.cc:2410] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 16 because replica has last-logged OpId of term: 15 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 6.
I20260709 12:19:38.357779 12621 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:38.386649  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:38.363806 12670 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:38.387266  6509 raft_consensus.cc:2393] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 17: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:38.388453  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:38.396754  6584 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:38.389091  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:38.393321  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:38.393905  6576 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:19:38.362569 12572 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:38.410998  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 133251us
I20260709 12:19:38.411427  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:38.415411  6502 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2617) took 1277 ms. Trace:
I20260709 12:19:38.415411  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:38.339641 12567 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:38.419909 12567 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:38.415935  6502 rpcz_store.cc:276] 0709 12:19:37.137887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.138086 (+   199us) service_pool.cc:224] Handling call
0709 12:19:37.180056 (+ 41970us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:37.181796 (+  1740us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:37.181845 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:37.182559 (+   714us) log.cc:844] Serialized 52 byte log entry
0709 12:19:37.423880 (+241321us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:37.435704 (+ 11824us) log.cc:844] Serialized 14 byte log entry
0709 12:19:37.451845 (+ 16141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.451920 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:38.415197 (+963277us) raft_consensus.cc:1685] finished
0709 12:19:38.415218 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:38.415347 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76459,"prepare.queue_time_us":12893,"prepare.run_cpu_time_us":594,"prepare.run_wall_time_us":17873,"raft.queue_time_us":21355,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":22345,"thread_start_us":145785,"threads_started":1,"wal-append.queue_time_us":352591}
I20260709 12:19:38.422695  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:38.423192  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:38.362138 12729 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:38.368932 12687 consensus_queue.cc:1048] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 12:19:38.483829  6577 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:38.487920  6502 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:38.490835  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:38.484571  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:19:38.492360  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:38.495527  6548 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.537s (SENT) [suppressed 8 similar messages]
I20260709 12:19:38.447578  6500 raft_consensus.cc:2410] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:19:38.513934  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:38.514782  6500 raft_consensus.cc:2410] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:19:38.490880  6508 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:38.502574  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:38.395495 12621 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:38.378844 12760 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.103s
I20260709 12:19:38.504392  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:38.404938 12572 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:19:38.509617  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 135647us
I20260709 12:19:38.528905  6500 raft_consensus.cc:1240] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:38.484809  6583 raft_consensus.cc:1240] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:19:38.517027  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:38.524504  6584 raft_consensus.cc:2410] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
W20260709 12:19:38.532850  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:38.533011  6505 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:38.481046 12772 raft_consensus.cc:697] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:38.500976 12703 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:19:38.522007 12578 consensus_meta.cc:287] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.674s	user 0.008s	sys 0.002s
I20260709 12:19:38.511775 12682 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:38.489058 12759 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
W20260709 12:19:38.561408  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 65853.7us
W20260709 12:19:38.568425  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 156417us
I20260709 12:19:38.579624  6548 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:38.527985 12621 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:38.580699  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:38.521768 12615 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:38.585943  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:38.587548  7269 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:19:38.593629  7238 leader_election.cc:400] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 15. Current term is 16.
I20260709 12:19:38.596678  7238 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:38.600781  6623 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:38.608875  6505 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:19:38.610319  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:38.612174  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:38.613145  6505 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
I20260709 12:19:38.614370  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:38.562234 12784 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.573220 12701 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:38.577527 12578 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:38.500645 12722 consensus_queue.cc:237] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:38.548318 12772 consensus_queue.cc:237] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:38.590262 12799 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:38.589648 12682 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:38.599601 12641 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:38.604700 12798 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.624997 12664 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:38.645416 12791 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:38.637703 12701 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:38.720999 12807 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:38.717034 12798 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:38.645088 12799 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:38.721789 12807 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:38.723448 12798 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:38.637687 12787 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:38.644770 12695 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:38.724637 12787 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:38.725718 12584 consensus_queue.cc:260] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:38.726907 12584 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:38.727160 12787 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:38.682523 12808 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:38.651847 12802 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:38.645282 12682 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:38.652661 12651 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:38.654268 12777 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:38.657518 12792 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:19:38.660451 12481 consensus_meta.cc:287] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.537s	user 0.011s	sys 0.000s
W20260709 12:19:38.669219 12790 log.cc:927] Time spent T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
W20260709 12:19:38.670012 12786 log.cc:927] Time spent T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 12:19:38.670713 12694 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:38.734678 12759 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:38.678603 12167 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:38.680876 12729 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:38.684994 12785 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:38.685603  6650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3188) took 1208 ms. Trace:
W20260709 12:19:38.787621  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 208068us
I20260709 12:19:38.690234  6504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2813) took 2220 ms. Trace:
I20260709 12:19:38.788621  6550 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:38.701400 12494 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.578903 12703 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:38.641935 12318 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.713495 12752 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:38.715407 12641 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:38.634673 12567 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:19:38.731225  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:38.744514  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 147951us
I20260709 12:19:38.732300 12808 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:38.672983 12594 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:38.629820 12784 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:38.787349  6650 rpcz_store.cc:276] 0709 12:19:37.477660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.477796 (+   136us) service_pool.cc:224] Handling call
0709 12:19:37.506176 (+ 28380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:37.562571 (+ 56395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:37.562601 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:37.562652 (+    51us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:37.562703 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:38.685340 (+1122637us) spinlock_profiling.cc:243] Waited 56.2 ms on lock 0x7b540019dcd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:38.685397 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:38.685534 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":129187584}
I20260709 12:19:38.729238 12701 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:38.789885  6548 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:38.801488  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:38.802170  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:38.819100  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:38.819705  6580 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 12.
I20260709 12:19:38.802358  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:38.803618  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:38.788545  6504 rpcz_store.cc:276] 0709 12:19:36.469555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:36.857924 (+388369us) service_pool.cc:224] Handling call
0709 12:19:36.881281 (+ 23357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:36.950480 (+ 69199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:36.950511 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:37.446827 (+496316us) spinlock_profiling.cc:243] Waited 5.63 ms on lock 0x7b2c00000c88. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:19:37.467930 (+ 21103us) log.cc:844] Serialized 26 byte log entry
0709 12:19:37.643752 (+175822us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:37.693976 (+ 50224us) log.cc:844] Serialized 14 byte log entry
0709 12:19:37.694459 (+   483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:37.694569 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:38.687194 (+992625us) spinlock_profiling.cc:243] Waited 3.42 ms on lock 0x7b18003ee720. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:19:38.690032 (+  2838us) raft_consensus.cc:1685] finished
0709 12:19:38.690053 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:38.690175 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":580,"prepare.queue_time_us":49765,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":80771,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":45214,"spinlock_wait_cycles":20814336,"thread_start_us":136193,"threads_started":1,"wal-append.queue_time_us":327987}
I20260709 12:19:38.805635  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:38.810267  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:38.780869 12799 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:38.812811  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:38.812170  7238 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:38.810794  6550 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:38.777123 12481 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:38.778182 12792 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:38.837833 12793 log.cc:927] Time spent T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.269s	user 0.000s	sys 0.001s
W20260709 12:19:38.836753 12794 log.cc:927] Time spent T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.259s	user 0.002s	sys 0.000s
I20260709 12:19:38.846944 12703 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:38.849728 12701 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:38.850359 12795 log.cc:927] Time spent T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.190s	user 0.005s	sys 0.000s
I20260709 12:19:38.852661 12813 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:19:38.853498  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 63658.4us
I20260709 12:19:38.824749 12567 raft_consensus.cc:697] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 18 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:38.813803 12808 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:38.854188 12799 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:38.855301 12682 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:38.856781 12641 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:38.857208 12820 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:38.857519  6577 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 17. Current term is 18. Ops: []
W20260709 12:19:38.814038 12680 consensus_meta.cc:287] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.815s	user 0.002s	sys 0.010s
I20260709 12:19:38.858840  6579 raft_consensus.cc:1275] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 8. (term mismatch)
I20260709 12:19:38.898849  6579 pending_rounds.cc:85] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 7
I20260709 12:19:38.899142  6579 pending_rounds.cc:107] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 12:19:38.859046  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:38.859745  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:38.899322  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:38.899391  6579 raft_consensus.cc:2889] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:38.900075  6583 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:38.901347  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:38.861502  6576 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: []
W20260709 12:19:38.862226 12636 consensus_meta.cc:287] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.823s	user 0.012s	sys 0.000s
I20260709 12:19:38.862914  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:38.908417  6580 raft_consensus.cc:2393] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 15: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:19:38.863909 12754 consensus_meta.cc:287] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.616s	user 0.009s	sys 0.001s
W20260709 12:19:38.864936 12581 consensus_meta.cc:287] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.709s	user 0.011s	sys 0.000s
I20260709 12:19:38.798913 12802 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:38.828368  7238 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:19:38.915398  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 87117.3us
I20260709 12:19:38.916465  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:38.842020  6622 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:19:38.872079  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 67997.9us
I20260709 12:19:38.787338 12694 consensus_queue.cc:1059] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:19:38.824543 12814 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:38.877408  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:38.919077  6550 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:38.919822  6583 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:19:38.884496  6548 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:38.854163 12784 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:38.889719 12806 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:38.890282 12594 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:38.941013 12806 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:38.941129 12594 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:38.943769 12806 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:38.945441 12681 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:38.946135 12814 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:38.947428  7271 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:19:38.950125 12792 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:38.918459 12581 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:38.811313 12318 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:38.953307 12581 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:38.953548  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:38.954062  6509 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:19:38.954555 12318 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:38.785562 12759 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:38.955180  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:38.956369  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:38.956758 12759 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:38.867316 12791 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:38.957639  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:38.956655  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:38.959161  6583 raft_consensus.cc:1275] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 12:19:38.959306  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 117437us
I20260709 12:19:38.959473 12722 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:38.959856  6550 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:38.960361  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:38.960934  6550 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:38.961211 12817 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:19:38.881599 12786 log.cc:927] Time spent T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:19:38.903159  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:38.906587  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:38.806250 12494 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:38.909751  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:38.910972  7269 raft_consensus.cc:1240] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:38.980978  6582 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:38.875538 12819 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:38.911619  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:38.912570  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:38.918637  6580 raft_consensus.cc:1240] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:38.812451 12807 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:39.005704  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:38.985169  6577 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:38.879859 12711 consensus_queue.cc:1059] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:19:38.911535 12636 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:38.907407 12680 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:38.914700 12754 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:38.933905 12804 log.cc:927] Time spent T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.275s	user 0.001s	sys 0.000s
W20260709 12:19:39.019654  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 209341us
I20260709 12:19:38.941526 12229 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:38.961957  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:38.892155  6502 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 14.
I20260709 12:19:38.962669  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:39.025616  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:38.961959  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:38.966665  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:39.024662  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:39.027665  6583 raft_consensus.cc:2393] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 17: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:38.986845  6579 raft_consensus.cc:2393] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 16: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:38.986689 12787 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:39.006685 12802 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:39.029831  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:39.030395  7274 raft_consensus.cc:1240] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:39.010042  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:39.030164  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 106013us
I20260709 12:19:39.031319  6649 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 20.
W20260709 12:19:39.031462  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 69583.1us
I20260709 12:19:39.017541  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:39.018115  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:38.943764 12594 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:39.033546  6506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2815) took 2558 ms. Trace:
I20260709 12:19:39.034237  6581 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 12:19:39.035118  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:39.035696  6500 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
I20260709 12:19:39.013952 12759 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.021185  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:39.042357  6501 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 14.
W20260709 12:19:39.042586  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:38.983345 12598 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:38.977337 12817 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:38.982704 12701 raft_consensus.cc:3055] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:39.022609  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:39.048785  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:38.987149 12682 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:38.986923 12799 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:38.994910  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:38.985849  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:38.891623 12813 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:39.033975  6506 rpcz_store.cc:276] 0709 12:19:36.470063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:36.860756 (+390693us) service_pool.cc:224] Handling call
0709 12:19:36.861049 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:36.864422 (+  3373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:36.864456 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:36.869061 (+  4605us) log.cc:844] Serialized 26 byte log entry
0709 12:19:37.847881 (+978820us) spinlock_profiling.cc:243] Waited 5.81 ms on lock 0x7b18002df0a0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:19:38.242565 (+394684us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:38.281102 (+ 38537us) log.cc:844] Serialized 14 byte log entry
0709 12:19:38.308663 (+ 27561us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:38.308756 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:38.704267 (+395511us) spinlock_profiling.cc:243] Waited 8.09 ms on lock 0x7b48000d0a58. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287adb2aa1 00007f287ada9b37 00007f287ada99ca 00007f287aea126f 00007f287aea0f37 00007f287aeb555c 00007f287aeb5508 00007f287aeb5488 00007f287aeb5443 00007f287aeb46ec 00007f2879ddea96
0709 12:19:39.028604 (+324337us) raft_consensus.cc:1685] finished
0709 12:19:39.028627 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:39.028745 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29902,"prepare.queue_time_us":46393,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":34618,"raft.run_cpu_time_us":2719,"raft.run_wall_time_us":403851,"spinlock_wait_cycles":31975552,"thread_start_us":394472,"threads_started":1,"wal-append.queue_time_us":1497794}
I20260709 12:19:39.052506  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:39.053421  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:39.053508  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:39.054178  6506 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:19:38.983690 12710 consensus_queue.cc:1059] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:19:39.059134  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:38.986678 12651 consensus_queue.cc:1059] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:19:39.039170 12798 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:39.047502  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:38.984272 12753 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 12:19:39.048861  6505 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 19.
W20260709 12:19:39.067754  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 183391us
I20260709 12:19:39.048107  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:39.068454  6505 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:19:39.070057  6505 raft_consensus.cc:1275] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:19:39.068715  6548 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:39.022878  6575 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:39.072207  6548 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:39.049464  6501 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:39.007019 12621 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:38.909401 12820 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:38.985947 12572 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:39.028174 12818 raft_consensus.cc:3055] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:39.020429  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2622) took 1059 ms. Trace:
I20260709 12:19:39.057070  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:39.057786  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:39.079010  6651 raft_consensus.cc:1240] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:19:39.078788  6507 rpcz_store.cc:276] 0709 12:19:37.946413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.946780 (+   367us) service_pool.cc:224] Handling call
0709 12:19:38.036721 (+ 89941us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:38.037070 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:38.037097 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:38.073612 (+ 36515us) log.cc:844] Serialized 52 byte log entry
0709 12:19:38.365638 (+292026us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:38.700285 (+334647us) spinlock_profiling.cc:243] Waited 36.5 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:38.812578 (+112293us) log.cc:844] Serialized 14 byte log entry
0709 12:19:38.917513 (+104935us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:38.917595 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:38.977147 (+ 59552us) raft_consensus.cc:1685] finished
0709 12:19:39.004989 (+ 27842us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:39.005232 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28547,"prepare.queue_time_us":117450,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":31672,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":52030,"spinlock_wait_cycles":83921792,"thread_start_us":649069,"threads_started":3,"wal-append.queue_time_us":501864}
I20260709 12:19:39.082832  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:39.058504 12813 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:39.059820  6655 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:39.060801  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:39.045768  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:39.062825  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:39.038794 12814 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:39.075819  6548 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:39.061960 12598 raft_consensus.cc:697] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:39.048214 12802 raft_consensus.cc:697] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:39.050202 12817 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:39.105813  6508 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:39.055791 12701 raft_consensus.cc:740] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:19:39.100569  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 53241.9us
I20260709 12:19:39.109987  6576 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:19:39.081470 12819 raft_consensus.cc:2764] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:19:39.128489  6549 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:39.129380  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:39.130088  6582 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
I20260709 12:19:39.130414  6501 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:19:39.132807  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:39.081848 12787 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:39.050372 12759 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
W20260709 12:19:39.113709  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:39.139593  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:39.113538  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:39.102375 12799 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:39.114317  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:39.144089  7272 raft_consensus.cc:2393] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 14: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:39.114451  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:39.114351  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:39.115545 12494 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:39.116144  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:39.109597 12710 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:39.117120 12802 consensus_queue.cc:237] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:39.078609 12818 raft_consensus.cc:740] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:39.160996 12818 consensus_queue.cc:260] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:39.162254 12818 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:39.109993 12481 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:39.108719 12641 raft_consensus.cc:3055] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:39.164731 12641 raft_consensus.cc:740] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:39.164496 12808 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:39.108603 12798 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:39.119248 12680 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:39.120991 12701 consensus_queue.cc:260] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:39.166415 12701 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:39.114670  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:39.168741 12808 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:39.078996 12784 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:39.127705  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:39.169282 12784 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:39.169745 12784 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:39.168100  6549 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:39.102558  6548 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:39.140364  6658 raft_consensus.cc:2368] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15.
I20260709 12:19:39.101600  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:39.142338  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 88987.6us
I20260709 12:19:39.146929  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:39.036967 12567 consensus_queue.cc:237] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:39.132093 12594 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 12:19:39.148316 12318 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:39.157629 12710 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:39.141530 12653 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:39.159001 12820 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:39.160068 12754 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:39.172802  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 211913us
I20260709 12:19:39.213873  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:39.214538  6550 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:39.215247  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:39.215633 12754 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:39.216162 12754 raft_consensus.cc:697] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:39.217067 12754 consensus_queue.cc:237] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:39.173022 12811 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:39.218951 12811 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:39.219419 12811 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:39.220564 12759 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:39.220721  6651 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:19:39.221688  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:39.173797 12167 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:39.174034  6622 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:39.173995 12652 consensus_queue.cc:1059] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:39.176117  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:39.227231  7272 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 20: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:39.181566  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:39.171175 12572 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:39.187569  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:39.229297  6658 raft_consensus.cc:2393] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 17: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:19:39.191869 12822 log.cc:927] Time spent T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 12:19:39.200553 12755 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:39.188460 12598 consensus_queue.cc:237] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:39.202208 12621 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:39.202371 12710 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:39.149778 12785 consensus_queue.cc:1059] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.040s
W20260709 12:19:39.206075  6578 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 2645) took 2723 ms (client timeout 3159 ms). Trace:
I20260709 12:19:39.209357 12789 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.248435 12789 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:39.209021 12817 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:39.223244  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:19:39.247397  6578 rpcz_store.cc:269] 0709 12:19:36.482833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:36.588691 (+105858us) service_pool.cc:224] Handling call
0709 12:19:39.205780 (+2617089us) spinlock_profiling.cc:243] Waited 975 ms on lock 0x7b540006e5d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:39.205961 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2243071744}
I20260709 12:19:39.236145  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:39.229878  7238 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:39.252369  7238 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:39.245229  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 71248us
I20260709 12:19:39.244442 12564 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:39.254921 12682 raft_consensus.cc:697] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 12 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:39.255272 12564 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:39.255721 12564 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:39.256983 12829 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:39.250958 12724 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:39.257443 12829 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:39.257745 12724 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:39.257943 12829 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:39.258324 12724 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:39.260519 12680 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:39.260453 12653 raft_consensus.cc:3055] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:39.261215 12680 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:39.261348 12653 raft_consensus.cc:740] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:39.261678 12680 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:39.262580 12653 consensus_queue.cc:260] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:39.264308 12653 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:39.267421 12641 consensus_queue.cc:260] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:39.268430 12641 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:39.246881 12711 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:39.272004 12711 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:39.274118 12711 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:39.275611 12812 log.cc:927] Time spent T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.627s	user 0.002s	sys 0.000s
I20260709 12:19:39.275967  6651 raft_consensus.cc:2410] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:19:39.238956 12753 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.277803  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:39.277926 12753 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:39.278869  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:39.279369 12710 consensus_queue.cc:1059] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:39.281234  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:39.280258  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:39.282127 12492 raft_consensus.cc:3055] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:39.282361 12572 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:39.282516 12492 raft_consensus.cc:740] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:39.255990 12682 consensus_queue.cc:237] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:39.283016  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:39.283692 12492 consensus_queue.cc:260] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:39.283049 12572 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:39.284951 12492 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:39.285116  6652 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58980 (request call id 3196) took 1710 ms (client timeout 1928 ms). Trace:
I20260709 12:19:39.284955 12572 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:39.286299  6653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58980 (request call id 3202) took 1238 ms. Trace:
I20260709 12:19:39.286654  6653 rpcz_store.cc:276] 0709 12:19:38.048163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:38.119275 (+ 71112us) service_pool.cc:224] Handling call
0709 12:19:39.286113 (+1166838us) spinlock_profiling.cc:243] Waited 713 ms on lock 0x7b540010ed54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:39.286244 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1640572800}
W20260709 12:19:39.285542  6652 rpcz_store.cc:269] 0709 12:19:37.574821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.612067 (+ 37246us) service_pool.cc:224] Handling call
0709 12:19:39.284935 (+1672868us) spinlock_profiling.cc:243] Waited 610 ms on lock 0x7b54002a35d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:39.285058 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1403000704}
I20260709 12:19:39.283665  6650 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
I20260709 12:19:39.290143  6652 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:39.282163  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:39.291360  6622 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:39.280273 12710 consensus_queue.cc:1048] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 12:19:39.292434 12806 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:39.293095 12806 raft_consensus.cc:697] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:39.293557 12710 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:39.294087 12806 consensus_queue.cc:237] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:39.294871  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:39.295461  7272 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:19:39.296481  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:39.297049  7265 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:39.279417  6651 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:39.287942  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:39.280130 12753 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:39.318472  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:39.319420  7266 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:39.311381 12829 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:39.343742 12829 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:39.346937 12564 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:39.349030 12564 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:39.349288 12789 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:39.319283 12784 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:39.361415 12817 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:39.365953 12784 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:39.172199  6500 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:39.123859  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2624) took 1138 ms. Trace:
I20260709 12:19:39.418733  6503 rpcz_store.cc:276] 0709 12:19:37.985694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.985859 (+   165us) service_pool.cc:224] Handling call
0709 12:19:37.986120 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:38.298288 (+312168us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:19:38.524069 (+225781us) log.cc:844] Serialized 14 byte log entry
0709 12:19:38.786837 (+262768us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:38.786890 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:38.786929 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:39.123014 (+336085us) spinlock_profiling.cc:243] Waited 9.27 ms on lock 0x7b1800066ee0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d6a2a6 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419
0709 12:19:39.123596 (+   582us) spinlock_profiling.cc:243] Waited 312 ms on lock 0x7b540039ec54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:39.123644 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:39.123793 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36588,"prepare.queue_time_us":36509,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":2719,"raft.queue_time_us":1230,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":139,"spinlock_wait_cycles":738742528,"thread_start_us":464491,"threads_started":2,"wal-append.queue_time_us":318790}
I20260709 12:19:39.423410  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:39.430789  6502 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41304 (request call id 742) took 1134 ms. Trace:
I20260709 12:19:39.431371  6502 rpcz_store.cc:276] 0709 12:19:38.295981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:38.504060 (+208079us) service_pool.cc:224] Handling call
0709 12:19:39.430345 (+926285us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b540039e754. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:39.430541 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":836729600}
I20260709 12:19:39.434826  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:39.437951  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:39.181787  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 79360.7us
W20260709 12:19:39.447207  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 191276us
I20260709 12:19:39.452011  7238 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:39.456064  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:39.457557  6507 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:19:39.458025  7238 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:39.458750  6623 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:39.460668  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 44392us
I20260709 12:19:39.465358 12511 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:39.465757  7267 raft_consensus.cc:1240] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:39.467033  7274 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38546 (request call id 722) took 1336 ms. Trace:
I20260709 12:19:39.467494  7274 rpcz_store.cc:276] 0709 12:19:38.130754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:38.131382 (+   628us) service_pool.cc:224] Handling call
0709 12:19:38.131717 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:39.034198 (+902481us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:39.466377 (+432179us) spinlock_profiling.cc:243] Waited 898 ms on lock 0x7b54004d7e54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:39.466824 (+   447us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2066151168}
I20260709 12:19:39.467960  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:39.477555 12500 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.490231 12567 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:39.491122 12500 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:39.493036  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:39.494938  6584 raft_consensus.cc:2393] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 15: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:39.481675  7272 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:19:39.478693 12547 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:39.479599  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:39.496526 12547 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:39.499333 12547 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:39.500013 12547 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:39.496122 12500 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:39.493045  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:39.501381 12167 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:39.501634 12547 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:39.502599  6623 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:39.502936  7274 raft_consensus.cc:1275] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 12:19:39.503382  7274 pending_rounds.cc:85] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9: Aborting all ops after (but not including) 11
I20260709 12:19:39.503739  7274 pending_rounds.cc:107] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 12:19:39.504343  7274 raft_consensus.cc:2889] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:39.504606  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:39.502557 12167 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:39.506076  6584 raft_consensus.cc:2410] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:19:39.508368 12547 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:39.508632 12641 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:39.516667 12641 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:39.517189 12641 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:39.521338  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 63668.7us
I20260709 12:19:39.521975  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:39.523151  6509 raft_consensus.cc:2393] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 17: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:39.523306  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:39.521371  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:39.529464 12639 consensus_queue.cc:1059] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:39.531152 12639 raft_consensus.cc:3055] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:39.531296 12681 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:39.532135 12681 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:39.532783 12681 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:39.534993  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:39.531641 12639 raft_consensus.cc:740] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:39.536484  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:39.537667  6583 raft_consensus.cc:2393] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 16: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:39.537833 12639 consensus_queue.cc:260] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:39.538674  6475 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:39.539908 12639 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:39.503785 12564 raft_consensus.cc:2764] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:19:39.540145  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:39.542747 12670 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:39.586329  6475 leader_election.cc:400] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15.
I20260709 12:19:39.586760  6475 leader_election.cc:403] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:19:39.596586  6500 raft_consensus.cc:1275] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:19:39.600740  6548 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:39.601039  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 61421us
I20260709 12:19:39.604782 12711 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:19:39.596966  6501 raft_consensus.cc:1275] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:19:39.604914  7274 raft_consensus.cc:1240] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:39.608912  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:39.596978  6502 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:19:39.508649 12167 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:39.611630  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:39.582926 12670 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:39.517040 12593 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:39.623821  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 75197.6us
I20260709 12:19:39.625238  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:39.627243  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 61713.6us
I20260709 12:19:39.629247 12711 raft_consensus.cc:697] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:39.630813 12806 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:39.630632 12711 consensus_queue.cc:237] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:39.632893 12813 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:19:39.634398 12851 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.254s	user 0.000s	sys 0.001s
W20260709 12:19:39.636045 12758 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.960s	user 0.004s	sys 0.004s
W20260709 12:19:39.636592 12758 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.985s	user 0.004s	sys 0.004s
I20260709 12:19:39.637709 12710 consensus_queue.cc:1059] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:19:39.639042 12710 raft_consensus.cc:3055] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:39.639483 12710 raft_consensus.cc:740] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:39.641577 12710 consensus_queue.cc:260] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:39.644085 12572 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:39.647018 12572 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:39.647581 12680 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:39.653672 12680 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:39.655260  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 54701.2us
I20260709 12:19:39.655369  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:39.656443  6506 raft_consensus.cc:2393] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 20: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:39.658507 12724 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:39.660029  6650 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:39.660701 12811 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:39.662489  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:39.663199 12811 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:39.663395  6576 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:39.663978  7271 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 21: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:39.664461  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:39.662535  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:39.665787  6658 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:39.665865 12819 consensus_queue.cc:1059] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:39.667224 12819 raft_consensus.cc:3055] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:39.667627 12819 raft_consensus.cc:740] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:19:39.668529  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 43712.6us
I20260709 12:19:39.670023  6473 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:39.671564 12757 consensus_queue.cc:1048] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:39.663223  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:39.672739 12584 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:39.675776  6658 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 21.
I20260709 12:19:39.621371 12670 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:39.676299  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:39.677073  6623 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:19:39.677119  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 53958.2us
I20260709 12:19:39.677346 12806 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:39.678345 12806 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:39.678751 12806 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:39.664942  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 176707us
I20260709 12:19:39.671458 12681 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:39.688258  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:19:39.696939  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 156915us
I20260709 12:19:39.686317 12757 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:39.698477  7272 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:39.710848  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:39.717033 12760 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:39.717258 12808 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:39.717922 12760 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:39.725303  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:39.727227  6475 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:39.728191 12758 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:39.728628 12758 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:39.728996 12758 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:39.733657  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:39.718412  7265 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:39.751564  6473 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:39.760950  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 197253us
I20260709 12:19:39.705853 12641 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:39.752851 12724 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:39.774729  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:39.780284  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:39.780975 12717 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:39.783902  6473 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:39.786346  6473 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:39.787503 12581 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:39.788246 12581 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:39.788411  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:39.789130  6507 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:39.789893  6473 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:39.762405 12797 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.021s	user 0.019s	sys 0.012s
W20260709 12:19:39.805022  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 88835.5us
I20260709 12:19:39.806043  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:39.809131 12717 raft_consensus.cc:2764] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:19:39.809662 12710 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:39.744118 12672 consensus_queue.cc:1059] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 12:19:39.747561 12584 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 12:19:39.824299  7272 raft_consensus.cc:1240] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 15. Current term is 16. Ops: []
W20260709 12:19:39.814316 12797 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.224s	user 0.019s	sys 0.014s
I20260709 12:19:39.835031  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:39.842069  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:39.759630 12819 consensus_queue.cc:260] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:39.857800  6584 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:19:39.834249 12760 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:39.789132 12581 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 12:19:39.865072 12796 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:39.865424  6576 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 12:19:39.866446  6407 catalog_manager.cc:5697] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 9 to 19, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 19 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:39.868512 12621 consensus_queue.cc:1048] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 12:19:39.872095  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:39.875000  6508 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:39.875809  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:39.870721  7238 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:39.888456  6550 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:39.892616  6654 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:39.835989 12670 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:39.895550  6623 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:39.861285 12681 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:39.860569 12797 raft_consensus.cc:3055] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:19:39.908800  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:39.903620  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:19:39.910878  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:39.839241 12808 consensus_queue.cc:1048] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 12:19:39.895129 12796 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:19:39.915410  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 70032.7us
I20260709 12:19:39.914549  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:39.921948 12641 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:39.922225 12771 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.922951 12771 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:39.898185 12672 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:39.924777 12672 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:39.927507 12672 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:39.928526 12796 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:39.930502 12670 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:39.931529 12717 raft_consensus.cc:3055] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:39.931675 12819 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:39.896169 12584 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:39.928454 12639 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:19:39.939199  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 68553.5us
I20260709 12:19:39.942962 12564 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:39.945266  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:39.948727 12564 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 12:19:39.950404 12653 consensus_meta.cc:287] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.684s	user 0.011s	sys 0.007s
I20260709 12:19:39.951131 12641 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:39.951752  6577 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:19:39.952287  6577 pending_rounds.cc:85] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 9
I20260709 12:19:39.952591  6577 pending_rounds.cc:107] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:19:39.952888  6577 raft_consensus.cc:2889] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:39.955304  6654 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 2563) took 2247 ms (client timeout 1836 ms). Trace:
I20260709 12:19:39.957069  6407 catalog_manager.cc:5697] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:19:39.958987  6407 catalog_manager.cc:5697] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:19:39.960680  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:39.961421  6503 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:39.932010 12717 raft_consensus.cc:740] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:39.963333 12717 consensus_queue.cc:260] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:39.965025 12717 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:39.898470 12511 consensus_queue.cc:1059] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:19:39.938323 12584 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:39.954658 12758 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:39.978926  6583 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
I20260709 12:19:39.979038  6507 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
W20260709 12:19:39.955957  6654 rpcz_store.cc:269] 0709 12:19:37.708116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:37.709334 (+  1218us) service_pool.cc:224] Handling call
0709 12:19:39.954969 (+2245635us) spinlock_profiling.cc:243] Waited 2.03 s on lock 0x7b54002283d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:39.955199 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4674299136}
I20260709 12:19:39.912776 12797 raft_consensus.cc:740] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:39.951759 12641 raft_consensus.cc:697] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:19:40.000358  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 210541us
I20260709 12:19:40.002547  6506 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:40.005357  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:19:40.011350  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:39.984592 12811 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:19:40.018240  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 154303us
I20260709 12:19:40.026279  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:40.017632  6407 catalog_manager.cc:5697] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:40.027390  6575 raft_consensus.cc:2393] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 16: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:40.028009 12753 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.028673  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:40.028755 12753 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:40.029178  6473 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:40.029584  6577 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:19:40.030751 12581 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:40.030803  6473 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:40.031251 12753 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:40.038113  6584 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 21.
I20260709 12:19:40.044381  6473 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:40.047576  6550 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:40.049571 12701 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:40.051371 12641 consensus_queue.cc:237] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:40.031335 12581 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:40.053907 12581 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:40.055891  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:40.056833  6582 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:19:40.056906 12797 consensus_queue.cc:260] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:40.057572 12584 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:40.058177  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:40.058393 12797 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:40.058816  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:40.061538  6501 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:40.028154  6407 catalog_manager.cc:5697] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:40.058837  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:40.036033 12756 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:40.072628  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:40.073037  6500 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:19:40.073009  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:40.073987  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:40.074448  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:40.070894  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:40.070012  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:40.080143  7272 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:40.052317 12806 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:40.078374  6473 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:40.084625  7238 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:40.063624 12771 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:40.093071  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:40.086571  6407 catalog_manager.cc:5697] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 10 to 15, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 15 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:19:40.095964  6577 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:40.095775 12777 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 12:19:40.073684 12511 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.065749 12811 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:40.105819  6473 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:40.106254 12811 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:40.106772  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:40.108373  6473 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:40.109194  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:40.109906  6578 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:40.111148 12855 log.cc:927] Time spent T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:19:40.112229  6473 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:40.112563  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:40.113368  6504 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:19:40.113976  6473 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:40.113917  6407 catalog_manager.cc:5697] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 9 to 13, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 13 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:19:40.116693 12701 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:40.116876  6584 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:40.117697 12672 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:40.096619 12753 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:40.128858  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:40.132148  6503 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:19:40.129547  6580 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:40.117982 12858 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:40.133236  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:40.134059  6500 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 12:19:40.144991  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 128931us
I20260709 12:19:40.115582  6407 catalog_manager.cc:5697] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 10 to 16, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 16 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:19:40.147048  6407 catalog_manager.cc:5697] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 7 to 15. New cstate: current_term: 15 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:19:40.148675 12811 consensus_queue.cc:1059] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:40.149631 12811 raft_consensus.cc:3055] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:40.148903  6407 catalog_manager.cc:5697] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 11 to 16, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 16 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:40.149957 12811 raft_consensus.cc:740] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:40.151345 12811 consensus_queue.cc:260] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:40.152721 12811 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:40.151268  6407 catalog_manager.cc:5697] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 10 to 15, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 15 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:40.167155  6501 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:19:40.168020  6584 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:19:40.171531  7272 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:19:40.172919  6578 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:19:40.116731 12564 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:40.181427  7274 raft_consensus.cc:1275] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:19:40.215299  6584 raft_consensus.cc:1240] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:19:40.215334  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:40.198661  6407 catalog_manager.cc:5697] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 11 to 14, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 14 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:40.225195  6578 raft_consensus.cc:1275] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:19:40.227000  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:40.226040  6500 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 12:19:40.173604 12511 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:40.191138 12564 raft_consensus.cc:697] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 21 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:40.231781  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:40.198297 12682 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:40.201295 12758 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:40.204440 12806 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:40.252732  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:40.240826 12682 raft_consensus.cc:697] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 16 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:40.244755  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:40.243960  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:40.245508  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:40.246560 12564 consensus_queue.cc:237] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:40.244789  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:40.221115 12753 consensus_queue.cc:1059] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 12:19:40.253234  6583 raft_consensus.cc:1275] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:19:40.254107  6578 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 21: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:40.255123  6504 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 21.
I20260709 12:19:40.240836  6407 catalog_manager.cc:5697] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:19:40.265473  6580 raft_consensus.cc:2393] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 20: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:40.273743 12672 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:40.259987 12796 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:40.276361 12796 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:40.257561  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:40.280233  6581 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
I20260709 12:19:40.280774  6654 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:40.285632 12511 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:40.296128 12806 consensus_queue.cc:1059] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:40.297920  6622 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:40.297971 12806 raft_consensus.cc:3055] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:40.298636 12806 raft_consensus.cc:740] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:40.300504 12771 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:40.303232  6654 raft_consensus.cc:1240] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:40.304807  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:40.305634  6654 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:40.306002  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:40.307158  6656 raft_consensus.cc:2368] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 17. Current term is 18.
I20260709 12:19:40.308351  7238 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:40.262202  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:40.313807  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:40.314107  6656 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:40.315318  6506 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
W20260709 12:19:40.319527  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 50677.6us
I20260709 12:19:40.324923  6584 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
W20260709 12:19:40.332298  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:40.332634 12859 log.cc:927] Time spent T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.279s	user 0.003s	sys 0.001s
I20260709 12:19:40.332906  7238 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:40.334664  7238 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:40.335657 12777 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:40.335822  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:40.336248 12758 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.336541  6654 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:40.301013 12806 consensus_queue.cc:260] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:40.336242 12777 raft_consensus.cc:697] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 15 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:40.336841 12758 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:40.337498  6579 raft_consensus.cc:1275] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:40.337786 12806 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:40.338003 12777 consensus_queue.cc:237] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:40.338599  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:40.340612  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:40.341884  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:40.342747 12758 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:40.349484 12784 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:40.350389  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:40.351302 12784 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:19:40.353873 12791 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:40.354478 12791 raft_consensus.cc:697] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 16 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:40.354574  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:40.355522  6658 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:40.355623 12791 consensus_queue.cc:237] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:40.362654 12682 consensus_queue.cc:237] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:40.368136  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:40.367918  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:40.375095  6655 raft_consensus.cc:1275] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:40.308552 12701 raft_consensus.cc:2764] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:19:40.385973  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:40.378086 12680 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:19:40.396483  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:40.319039 12672 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:40.400789 12672 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:40.401223 12639 raft_consensus.cc:3055] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:40.401788 12639 raft_consensus.cc:740] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:40.403198 12639 consensus_queue.cc:260] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:40.405804 12753 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.409178  6657 raft_consensus.cc:1240] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 16. Current term is 17. Ops: []
W20260709 12:19:40.412586  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:40.420598  6407 catalog_manager.cc:5697] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 9 to 14, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 14 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
W20260709 12:19:40.429647  6508 consensus_meta.cc:287] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.518s	user 0.010s	sys 0.001s
I20260709 12:19:40.429745  7274 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:40.430181  6508 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
I20260709 12:19:40.403229 12672 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:40.424216 12757 consensus_meta.cc:287] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.685s	user 0.010s	sys 0.003s
I20260709 12:19:40.430418  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:40.431100 12760 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:40.440218  6622 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:19:40.439116  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 322811us
I20260709 12:19:40.440824  6407 catalog_manager.cc:5697] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 10 to 14, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 14 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:19:40.459618  6407 catalog_manager.cc:5697] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 12 to 13, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 13 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:19:40.457794  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:40.439704 12757 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 14. Current term is 15.
I20260709 12:19:40.433413 12753 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:40.492157  6407 catalog_manager.cc:5697] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:19:40.452637 12639 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:40.499709  6473 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:40.457105 12681 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:40.494834  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 150295us
I20260709 12:19:40.489719 12806 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:40.451913 12819 consensus_queue.cc:1059] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 12:19:40.506021  6500 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
I20260709 12:19:40.515966  6652 raft_consensus.cc:1275] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:19:40.517729  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:40.519327  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:40.519783  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:40.518263  6623 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:40.523572  6508 raft_consensus.cc:1240] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 15. Current term is 16. Ops: []
W20260709 12:19:40.526129  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 88173.6us
I20260709 12:19:40.528236  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:40.527204  6407 catalog_manager.cc:5697] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 8 to 15, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 15 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:40.531487  7272 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 15 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:40.528160  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:40.538403  6509 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:40.539747  6509 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:40.529021  6503 raft_consensus.cc:2376] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20. Re-sending same reply.
I20260709 12:19:40.531775  6550 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:40.543015  6657 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:40.545287 12866 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.532433 12797 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:40.546792  6506 raft_consensus.cc:1240] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 16. Current term is 17. Ops: []
W20260709 12:19:40.547566  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 47962.1us
I20260709 12:19:40.524503 12757 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.548264 12757 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:40.550442 12757 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:40.550978 12757 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:40.552305  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:40.554741  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:40.520663 12811 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:40.540899 12717 consensus_meta.cc:287] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.575s	user 0.014s	sys 0.001s
I20260709 12:19:40.521579 12796 consensus_queue.cc:1048] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 12:19:40.528493 12581 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:40.574431  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:40.558527 12797 raft_consensus.cc:697] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:40.571403 12802 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:40.559973  6407 catalog_manager.cc:5697] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:19:40.581176  6407 catalog_manager.cc:5697] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 10 to 14, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 14 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:19:40.578926  6508 raft_consensus.cc:1275] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 17 index: 7. Preceding OpId from leader: term: 19 index: 8. (index mismatch)
I20260709 12:19:40.585932  6509 raft_consensus.cc:1240] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:40.591224  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:40.592546  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:40.595269  6583 raft_consensus.cc:2393] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 17: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:40.597177  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:40.600513  6407 catalog_manager.cc:5697] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 6 to 14. New cstate: current_term: 14 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:19:40.560734 12811 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:40.602751  6407 catalog_manager.cc:5697] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:40.611711  6578 raft_consensus.cc:2393] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 17: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:40.614612  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:40.618744  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:19:40.622340 12865 log.cc:927] Time spent T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:19:40.620693  6407 catalog_manager.cc:5697] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 12 to 15, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 15 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:19:40.627413  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:40.613524 12833 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.617554 12819 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:40.638074  6407 catalog_manager.cc:5697] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 10 to 13, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 13 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
W20260709 12:19:40.622363 12864 log.cc:927] Time spent T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:19:40.642663  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:40.643863  6501 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
W20260709 12:19:40.646620  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:40.588929 12802 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:40.647789  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:40.649235  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:40.649523  6507 raft_consensus.cc:2368] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 20. Current term is 21.
I20260709 12:19:40.650012  6581 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:40.651151  6622 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:19:40.660578  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 128908us
I20260709 12:19:40.663612  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:40.641987 12819 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:40.666491  6407 catalog_manager.cc:5697] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:19:40.675168  6550 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:40.685537  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:40.658877 12877 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:40.665601 12850 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:19:40.673487 12863 log.cc:927] Time spent T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 12:19:40.694000  6473 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:40.670276 12819 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:40.707156  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:40.708025  6507 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:40.714835 12880 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:19:40.718266  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 80550.4us
I20260709 12:19:40.721689  6507 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 18. Current term is 19. Ops: []
W20260709 12:19:40.724745  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 49720.9us
I20260709 12:19:40.728788 12492 raft_consensus.cc:3055] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:40.699046 12850 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:40.705623 12753 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:40.714879 12584 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:19:40.725374 12875 log.cc:927] Time spent T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:19:40.743752  6582 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
W20260709 12:19:40.741106  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 65592.6us
I20260709 12:19:40.744150 12785 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:40.744546  6582 raft_consensus.cc:606] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:40.744933  6582 raft_consensus.cc:614] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:40.700531 12653 consensus_queue.cc:1059] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 12:19:40.746447  6473 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:40.749089  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:40.668092 12797 consensus_queue.cc:237] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:40.707563 12877 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:40.751080  6473 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:40.729291 12492 raft_consensus.cc:740] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:40.658895 12833 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:40.545885 12866 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:40.752905 12670 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:40.757953 12581 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:40.747191 12785 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:40.760036 12811 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:40.761021 12870 log.cc:927] Time spent T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
W20260709 12:19:40.761503 12873 log.cc:927] Time spent T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 12:19:40.738408 12850 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:40.765306 12802 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:40.766248 12881 consensus_queue.cc:1059] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:40.767407 12878 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.782225  6505 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:40.776458 12672 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:40.737514 12880 consensus_queue.cc:1048] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 12:19:40.777904 12806 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:40.788609  6576 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:19:40.787892  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:40.791343  6576 raft_consensus.cc:606] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:40.791770  6576 raft_consensus.cc:614] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:40.780383 12785 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:40.783035 12819 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:40.793720 12878 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:40.794726  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:40.791628  6500 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:40.795105  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:40.808084  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:40.809350 12819 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:40.811944  6578 raft_consensus.cc:2393] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 18: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:40.812032  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:40.815852  6576 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "72269ad6925341548796bb34823b3e00"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
W20260709 12:19:40.816028  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 45265.7us
I20260709 12:19:40.815852  6502 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:40.816187  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:40.817421  6508 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 12.
I20260709 12:19:40.790355 12672 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:40.796190 12866 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:40.825116  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:40.826047  6501 raft_consensus.cc:2393] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 16: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
W20260709 12:19:40.822719  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 73207.4us
I20260709 12:19:40.823103  6576 raft_consensus.cc:606] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:40.822448  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:40.825569 12672 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:40.838404  6500 raft_consensus.cc:1275] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:40.831622 12802 consensus_queue.cc:1048] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 12:19:40.843423 12819 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:40.844117 12819 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:40.847270  6622 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:40.849771 12710 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:40.854959 12806 consensus_queue.cc:1048] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:40.856556  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:40.862946 12710 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:40.866523 12710 consensus_queue.cc:1048] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:40.869486  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:40.870592 12710 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:40.884449 12710 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:40.885151 12806 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:40.886752 12511 consensus_queue.cc:1048] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:40.920969  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:40.921689  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:40.922091  6475 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:40.922411  6507 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:40.923723  6475 leader_election.cc:400] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 17. Current term is 18.
I20260709 12:19:40.924083  6475 leader_election.cc:403] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
W20260709 12:19:40.890897  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:40.894012 12878 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:40.901615 12753 raft_consensus.cc:3055] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:40.925135 12670 consensus_queue.cc:1059] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:40.925644 12753 raft_consensus.cc:740] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:40.925185 12760 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:40.927099 12760 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:40.926738 12753 consensus_queue.cc:260] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:40.927548 12760 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:40.927968  6502 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:19:40.904455 12796 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.928925 12796 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:40.931559 12796 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:40.838086  6506 raft_consensus.cc:1275] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:19:40.906759  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:40.836364 12881 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 12:19:40.932839  6504 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:40.928059 12753 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:40.918869  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:40.937474 12760 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:40.938472  6500 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
W20260709 12:19:40.920020 12883 log.cc:927] Time spent T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:19:40.941448  6652 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:19:40.944283 12492 consensus_queue.cc:260] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:40.944628 12760 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:40.945976 12492 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:40.946920  6507 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:40.949528 12877 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:40.961184 12879 log.cc:927] Time spent T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.218s	user 0.001s	sys 0.004s
I20260709 12:19:40.963415  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:40.963905  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:40.964668  6502 raft_consensus.cc:2393] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 19: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:40.966892  7265 raft_consensus.cc:1275] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:19:40.968066  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:40.968667  6500 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:40.968982 12819 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:40.904011 12850 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:40.975534 12850 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:40.978173  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:40.978871  6578 raft_consensus.cc:1275] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:19:40.978961  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:41.013144  6581 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef43f1290994490e8ef291fa12012731"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:19:41.013849  6581 raft_consensus.cc:606] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:40.979554 12701 raft_consensus.cc:3055] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:41.016105  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:41.016637 12701 raft_consensus.cc:740] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:41.018431  6578 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:19:41.020161  6578 raft_consensus.cc:606] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:40.990137  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:40.991600 12511 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:40.992316 12833 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:40.996712  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:41.000667  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:41.020324 12796 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:41.019235 12701 consensus_queue.cc:260] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:41.022987  6649 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:41.013312  6658 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:41.023859 12701 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:40.984951 12564 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:40.979662  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:41.025825  6576 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:41.020776  6580 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:41.028649 12850 consensus_queue.cc:1048] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:41.022194 12511 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:41.033818 12511 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:41.038353 12672 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:41.040798 12806 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:41.041684 12806 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:40.985750  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 62191.9us
I20260709 12:19:41.052124  6475 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:41.023147 12796 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:41.022912  6578 raft_consensus.cc:614] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:41.068518  6500 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:19:41.076493 12760 raft_consensus.cc:2764] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:19:41.074156  6475 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:41.080947  6550 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:41.079159  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:41.083833  6580 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:19:41.089637  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:41.090221  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:41.090525  6650 raft_consensus.cc:1240] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:41.091084  6649 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:41.044232 12806 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:41.116071  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:41.129141  6653 raft_consensus.cc:2410] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 17 because replica has last-logged OpId of term: 15 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 6.
I20260709 12:19:41.074826 12717 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 17. Current term is 18.
W20260709 12:19:41.136191  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:41.100848  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:41.139246  6655 raft_consensus.cc:2410] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:19:41.086879 12584 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:41.093669 12819 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:41.066845 12850 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:41.098879 12886 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:41.194676 12833 consensus_queue.cc:1059] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:19:41.137960 12753 consensus_queue.cc:1059] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 12:19:41.195667 12833 consensus_queue.cc:1243] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer 20a202c76244485fbc8f1610473db8b9 log is divergent from this leader: its last log entry 15.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:19:41.196251 12833 raft_consensus.cc:3055] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:41.196795 12833 raft_consensus.cc:740] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:41.197577 12785 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:41.197791 12833 consensus_queue.cc:260] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:41.199111 12833 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:41.206800  6576 raft_consensus.cc:1275] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:19:41.200037 12785 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:41.196308 12753 raft_consensus.cc:3055] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:19:41.215368  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 141301us
I20260709 12:19:41.219295  6475 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:41.188558 12886 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:41.257392  6576 raft_consensus.cc:1275] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 12:19:41.248598  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:41.252102  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:41.261164  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:41.261570  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:41.215544 12796 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:41.268891  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:41.218505 12753 raft_consensus.cc:740] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:41.255395 12584 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:41.280941  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:41.201239 12850 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:41.291106  6576 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:41.292447  7274 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:41.201946 12819 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:41.259462 12886 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:41.263199 12672 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:41.398397  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 75451.2us
I20260709 12:19:41.404021  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:41.405450  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:41.406334  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:41.348459 12753 consensus_queue.cc:260] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:41.376143 12888 log.cc:927] Time spent T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:19:41.320114 12584 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:41.422655  6583 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:19:41.281371 12681 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.120s
I20260709 12:19:41.325212 12639 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:41.442170  6473 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:41.429742 12584 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:41.459690  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:41.460934  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:41.448182 12701 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:41.476907  6576 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:19:41.480739  6473 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:41.452679 12753 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:41.440641 12639 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:41.508859  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:41.460779 12785 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:41.516307  6622 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:41.503629 12701 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:41.525672 12904 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:41.526335 12904 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:41.526986 12850 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:41.528666 12819 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:41.528865 12850 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:41.529342 12850 raft_consensus.cc:697] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:41.521363 12701 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:41.530465 12639 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:41.529433 12904 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:41.553071  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 231621us
I20260709 12:19:41.519805 12785 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:41.523631 12710 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.028s
W20260709 12:19:41.579257  6548 leader_election.cc:336] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.537s (SENT)
I20260709 12:19:41.549144 12717 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:41.583532  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:41.588446  6650 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:41.584192  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:41.588932  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 298871us
I20260709 12:19:41.594748  6653 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:19:41.549106 12900 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:41.598378  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:41.599160  6653 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:41.601418  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:41.608592  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:41.624390  6654 raft_consensus.cc:2393] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 16: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:41.616729  6650 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:19:41.530395 12850 consensus_queue.cc:237] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:41.647097  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 116679us
W20260709 12:19:41.658401  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:41.673559  6649 consensus_meta.cc:287] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.546s	user 0.008s	sys 0.005s
I20260709 12:19:41.674057  6649 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
W20260709 12:19:41.680671  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 200040us
I20260709 12:19:41.597309 12680 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 12:19:41.682412  6656 raft_consensus.cc:1240] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: []
W20260709 12:19:41.684572  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 110410us
W20260709 12:19:41.693759  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:41.641286 12900 raft_consensus.cc:697] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:41.701153  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:41.704427  6505 raft_consensus.cc:1275] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:19:41.707211  6509 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:58944
I20260709 12:19:41.707894  6509 raft_consensus.cc:606] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:41.708266  6509 raft_consensus.cc:614] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:41.708770  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:41.709851  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:41.713032  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:41.709553 12717 raft_consensus.cc:697] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:41.722641  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:41.723555  6501 raft_consensus.cc:1275] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:19:41.723357 12717 consensus_queue.cc:237] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:41.724298  6502 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:19:41.730862  6653 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
W20260709 12:19:41.741698  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:41.742738  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 103907us
I20260709 12:19:41.744683  7238 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:41.761371 12906 log.cc:927] Time spent T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
I20260709 12:19:41.761618  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:41.762115  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:41.763870  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:41.764212 12909 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:41.764585  6502 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
W20260709 12:19:41.764845 12907 log.cc:927] Time spent T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:19:41.764993 12909 raft_consensus.cc:697] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 16 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:41.763077  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:41.765842 12899 log.cc:927] Time spent T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.234s	user 0.000s	sys 0.001s
I20260709 12:19:41.766126 12909 consensus_queue.cc:237] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:41.767354 12900 consensus_queue.cc:237] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:41.769588 12874 raft_consensus.cc:2764] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:19:41.719892 12895 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:41.776897  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:41.782024  6507 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
W20260709 12:19:41.777771  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:41.751205 12891 log.cc:927] Time spent T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
I20260709 12:19:41.788995  6579 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:41.812237  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:41.813169  6658 raft_consensus.cc:2393] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 16: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:41.812453  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:41.816519  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:41.817441  6501 raft_consensus.cc:2410] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 17 because replica has last-logged OpId of term: 15 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 6.
I20260709 12:19:41.817204  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:41.780838 12895 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:41.780409 12785 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:41.826984  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:41.845389 12922 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:41.846095 12922 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:41.830523 12796 consensus_queue.cc:1048] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 12:19:41.848860 12922 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:41.839643 12914 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:41.850394 12920 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:41.851084 12920 raft_consensus.cc:697] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:41.850771 12914 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:41.852212 12920 consensus_queue.cc:237] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:41.854344 12914 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:41.822366 12797 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:19:41.840040 12897 log.cc:927] Time spent T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.303s	user 0.000s	sys 0.001s
I20260709 12:19:41.830775 12895 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:41.894241 12584 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:41.901780 12760 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:41.905025  6475 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:19:41.905952  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 125259us
I20260709 12:19:41.914569 12903 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:41.916034 12584 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:41.920524 12896 log.cc:927] Time spent T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.289s	user 0.000s	sys 0.001s
I20260709 12:19:41.927533  6503 raft_consensus.cc:2410] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:19:41.927301  6473 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:41.897320 12796 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:19:41.947945  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:41.963527  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 58789.2us
I20260709 12:19:41.964315  6475 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:41.949702 12903 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:41.952783 12796 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:41.958688 12564 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 12:19:41.951695 12939 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:19:42.001926  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:42.003510  6575 raft_consensus.cc:2393] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 17: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:42.003893 12908 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 12:19:42.006352  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:19:41.970777 12937 raft_consensus.cc:2764] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:19:42.008307  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:41.974334 12934 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:41.969251 12903 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:42.053545 12903 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:42.068673  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:42.141331 12917 log.cc:927] Time spent T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.230s	user 0.001s	sys 0.000s
I20260709 12:19:42.142212 12940 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:19:42.144366 12928 log.cc:927] Time spent T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:19:42.151438 12796 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:42.156133 12753 consensus_meta.cc:287] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.648s	user 0.012s	sys 0.000s
I20260709 12:19:42.158553  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 2553) took 1167 ms. Trace:
I20260709 12:19:42.159596  6584 rpcz_store.cc:276] 0709 12:19:40.994325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:40.994479 (+   154us) service_pool.cc:224] Handling call
0709 12:19:40.994802 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:40.995025 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:19:40.995342 (+   317us) log.cc:844] Serialized 14 byte log entry
0709 12:19:41.448434 (+453092us) log.cc:844] Serialized 14 byte log entry
0709 12:19:42.157784 (+709350us) spinlock_profiling.cc:243] Waited 1.38 ms on lock 0x7b18003da2c0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d16c0a 00007f287c456219 00007f2872d9ea40 00007f2872d994e4 00007f2872d99455 00007f2875e81060 00007f2875e93716 00007f2875ea4287 00007f2875ea4208
0709 12:19:42.157997 (+   213us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:42.158097 (+   100us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:42.158166 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:42.158273 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:42.158424 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":689152,"prepare.queue_time_us":637,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":1983,"raft.queue_time_us":715516,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":8148,"spinlock_wait_cycles":9735808,"thread_start_us":450189,"threads_started":1,"wal-append.queue_time_us":535631}
I20260709 12:19:42.162106 12819 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:42.162680 12819 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:42.164494 12639 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.165058 12639 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:42.161335  6550 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:42.069284 12934 raft_consensus.cc:697] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:19:42.183758 12886 consensus_meta.cc:287] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.762s	user 0.008s	sys 0.005s
I20260709 12:19:42.184389 12886 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:42.188820  6657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3242) took 1100 ms. Trace:
I20260709 12:19:42.195988  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:42.196947  6650 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:42.136787 12903 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:42.189709  6657 rpcz_store.cc:276] 0709 12:19:41.073606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:41.074921 (+  1315us) service_pool.cc:224] Handling call
0709 12:19:41.075351 (+   430us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:41.450480 (+375129us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:19:41.450518 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:41.460451 (+  9933us) log.cc:844] Serialized 52 byte log entry
0709 12:19:41.587108 (+126657us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:41.587168 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:41.587296 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:42.173918 (+586622us) raft_consensus.cc:1685] finished
0709 12:19:42.173948 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:42.174142 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":141209,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":12396,"thread_start_us":410870,"threads_started":2,"wal-append.queue_time_us":320381}
W20260709 12:19:42.224485  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 63255.4us
I20260709 12:19:42.194687 12878 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:42.224267 12877 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.224076 12882 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.288269  6475 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:42.223866 12881 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:42.305378  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:42.307312  6475 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:42.286233 12701 consensus_meta.cc:287] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.713s	user 0.013s	sys 0.000s
I20260709 12:19:42.241396 12819 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:42.233635 12903 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:42.237619 12878 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:42.312444 12937 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:42.314457 12760 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:19:42.357730 12925 log.cc:927] Time spent T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 12:19:42.355978 12931 consensus_queue.cc:1059] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 12:19:42.358191 12912 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.312325 12877 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:42.389906 12877 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:42.390625 12877 raft_consensus.cc:993] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e: : Instructing follower f2b06aebe97249a4877d798b8118ece4 to start an election
I20260709 12:19:42.391119 12877 raft_consensus.cc:1081] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Signalling peer f2b06aebe97249a4877d798b8118ece4 to start an election
I20260709 12:19:42.319645 12701 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:42.393535  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 168380us
I20260709 12:19:42.331243 12760 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:42.338250 12886 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:42.311935 12882 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:42.316223 12881 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:42.394127 12833 consensus_meta.cc:287] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 1.187s	user 0.013s	sys 0.006s
I20260709 12:19:42.387253 12753 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.407418 12901 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:42.405350 12760 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:42.412475  6578 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 12:19:42.418576 12945 log.cc:927] Time spent T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 12:19:42.424374 12903 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:42.426488  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2696) took 1145 ms. Trace:
I20260709 12:19:42.426889  6580 rpcz_store.cc:276] 0709 12:19:41.280889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:41.422469 (+141580us) service_pool.cc:224] Handling call
0709 12:19:41.435541 (+ 13072us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:41.514159 (+ 78618us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:41.514203 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:41.514252 (+    49us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:41.514297 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:42.426268 (+911971us) spinlock_profiling.cc:243] Waited 78.4 ms on lock 0x7b5400266bd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:42.426314 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:42.426434 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":180216704}
I20260709 12:19:42.427769 12923 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:42.428632 12940 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:42.429116 12940 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:42.429613 12639 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:42.430610 12955 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:42.431243 12940 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:42.431805 12923 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:42.433985 12923 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:42.438465  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:42.439241  6658 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 17: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:42.448277  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:42.449108  6653 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:42.408557 12931 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:42.452750  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:42.330260 12934 consensus_queue.cc:237] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:42.456540  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:42.457163  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:42.458616  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:42.463660 12757 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:19:42.469954 12953 log.cc:927] Time spent T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
W20260709 12:19:42.471043 12942 log.cc:927] Time spent T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.158s	user 0.000s	sys 0.000s
I20260709 12:19:42.473345 12701 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:42.413857 12753 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:42.404592 12912 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:42.453368 12955 raft_consensus.cc:697] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:19:42.483948  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 69793us
W20260709 12:19:42.498818 12946 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.098s	user 0.000s	sys 0.004s
I20260709 12:19:42.503026  6653 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:42.490774 12908 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:42.505797  6548 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:42.491500  6658 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ef43f1290994490e8ef291fa12012731"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:58980
I20260709 12:19:42.507608  6658 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:42.507987  6658 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:42.416131 12901 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:42.483089 12931 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:42.504521 12895 consensus_meta.cc:287] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.593s	user 0.011s	sys 0.002s
I20260709 12:19:42.486367 12757 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:42.526034 12912 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:42.513581 12908 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:42.516855 12939 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:42.527184 12939 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:42.527661 12753 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:42.504210 12948 log.cc:927] Time spent T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 12:19:42.529418  7265 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:42.540792  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:42.523168 12913 log.cc:927] Time spent T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.825s	user 0.000s	sys 0.005s
I20260709 12:19:42.541492  6654 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
I20260709 12:19:42.545768  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:42.521356 12881 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:42.546075  6550 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:42.522042 12931 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:42.523847 12901 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:42.528311 12908 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:42.549269  7240 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:42.554181 12811 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:42.554911  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:42.559255  7274 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:19:42.567106  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:42.531847 12895 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:42.555392 12757 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:42.575773  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:42.574671 12753 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:42.557337 12887 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:42.554939 12882 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:42.582708  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:42.583179 12939 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:42.582767 12912 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:42.580271 12753 raft_consensus.cc:697] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:42.576066 12908 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:42.524500 12955 consensus_queue.cc:237] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:42.579479 12811 raft_consensus.cc:697] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:42.593575  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:42.594394 12811 consensus_queue.cc:237] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:42.594847  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:42.595498  6649 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:42.596161  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:42.588613 12887 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:42.585331  7274 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:19:42.612632 12760 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:42.590659  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:42.615455 12760 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:42.592433 12912 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:42.619295 12912 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:42.620555  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:42.624888 12753 consensus_queue.cc:237] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:42.594868 12895 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:42.591552 12908 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:42.646437 12908 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:42.663748  6623 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:42.664583  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 172439us
I20260709 12:19:42.665350  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:42.671913  6650 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:42.661034 12954 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.674257  6550 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:42.671844  6658 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:42.672578 12904 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.673514 12954 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:42.683198 12954 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:42.685197 12797 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:42.686112 12797 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:42.689896 12927 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:42.690388  6654 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:42.671036 12902 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:19:42.692476 12670 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:42.692624 12866 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:42.693028 12670 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:42.693231 12866 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:42.693763 12881 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:42.694334 12881 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:42.694725 12881 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:42.695145 12670 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:42.695418 12937 raft_consensus.cc:3055] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:42.695684 12670 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.696020 12937 raft_consensus.cc:740] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:42.693871 12878 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:42.696584 12760 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:42.697088 12760 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:42.697150 12937 consensus_queue.cc:260] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:42.698657 12908 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:42.699179 12760 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:42.699679 12760 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:42.699842  7267 raft_consensus.cc:2393] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 17: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:42.702929  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:42.690477 12927 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:42.705852  6623 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:42.712579  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:42.718024  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:42.718529  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:42.719089  6507 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:42.719234  6506 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:42.721673  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:42.722215  6503 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:42.722918  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:42.724496  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:42.725394  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:42.698670 12937 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:42.732132  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:42.715845 12931 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:42.690011 12887 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:42.709185 12927 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:42.732759  6548 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:42.741235  6658 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:42.719117 12903 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:42.700121 12760 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:42.677881 12904 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:42.709883 12670 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:42.752066  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:42.754958  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:42.757090  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:42.745693 12903 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:42.749668 12887 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:42.766861 12903 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:42.746337 12931 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:42.784261  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 66327.1us
I20260709 12:19:42.787225  6548 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:42.798194  6548 leader_election.cc:400] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 20. Current term is 21.
I20260709 12:19:42.798712  6548 leader_election.cc:403] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:42.800909  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:42.781536 12920 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:42.784957 12681 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:42.840158  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:42.840256  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:42.787426 12955 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
W20260709 12:19:42.850803  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 52814.6us
I20260709 12:19:42.838522 12887 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:42.856357  6548 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:42.835475 12878 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 20. Current term is 21.
I20260709 12:19:42.865099  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:42.870051  6500 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
W20260709 12:19:42.865913 12974 log.cc:927] Time spent T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
W20260709 12:19:42.874608 12949 log.cc:927] Time spent T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8: Append to log took a long time: real 0.495s	user 0.002s	sys 0.005s
I20260709 12:19:42.884625  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:42.876988  6506 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:19:42.868964 12866 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:42.881822  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:42.843993 12681 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:42.860575 12908 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:42.843425 12920 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:42.909302  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:42.909932  6500 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:42.910466  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:42.906670  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 54906.4us
I20260709 12:19:42.869496 12760 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:42.913053  6507 raft_consensus.cc:1275] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:19:42.873873 12954 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:42.931813  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:42.932709  6506 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:42.948035  6505 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:19:42.935988 12670 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:42.950453 12973 log.cc:927] Time spent T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.113s	user 0.001s	sys 0.001s
I20260709 12:19:42.953193  6548 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:42.941947 12902 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:42.958149  6503 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:19:42.960299 12653 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:42.960357  6548 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:42.965294  6500 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:19:42.966176  6548 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:42.967942  6548 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:42.968842 12887 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:42.970510  6548 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:42.978423  6506 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
W20260709 12:19:42.961159 12975 log.cc:927] Time spent T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:19:42.985895 12681 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:42.979943 12902 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:42.983999  6548 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:42.990427 12902 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:42.997856  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:43.000448 12653 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:42.967621 12878 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:42.969271 12887 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:43.003456 12887 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:42.967751 12955 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:43.004998 12955 consensus_queue.cc:1048] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:43.007635  6548 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:42.961987 12797 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:42.980598 12957 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:19:43.009958 12753 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:43.014070  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2683) took 2436 ms. Trace:
I20260709 12:19:43.015041 12920 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:43.015588 12920 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:43.016207 12920 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:43.014492  6577 rpcz_store.cc:276] 0709 12:19:40.584843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:40.585006 (+   163us) service_pool.cc:224] Handling call
0709 12:19:40.598850 (+ 13844us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:40.724454 (+125604us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:40.724495 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:40.880559 (+156064us) spinlock_profiling.cc:243] Waited 1.37 ms on lock 0x7b2c00010c48. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:19:40.881026 (+   467us) log.cc:844] Serialized 26 byte log entry
0709 12:19:41.189977 (+308951us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:41.201183 (+ 11206us) log.cc:844] Serialized 14 byte log entry
0709 12:19:41.201411 (+   228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:41.201481 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:43.012604 (+1811123us) spinlock_profiling.cc:243] Waited 1.31 ms on lock 0x7b24003d7528. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872c229c4 00007f2872c20acb 00007f2872ddbdc9 00007f2872ddba55 00007f2872ddafb0 00007f2875a51488 00007f2875a51be8 00007f2875a29398 00007f2875a2476a 00007f2875a23bb6
0709 12:19:43.013846 (+  1242us) raft_consensus.cc:1685] finished
0709 12:19:43.013865 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:43.013989 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8668,"prepare.queue_time_us":10413,"prepare.run_cpu_time_us":23,"prepare.run_wall_time_us":24,"raft.queue_time_us":49246,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":19761,"spinlock_wait_cycles":6184192,"thread_start_us":406070,"threads_started":2,"wal-append.queue_time_us":567815}
I20260709 12:19:43.018256 12920 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:43.018451 12957 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:43.020977  6548 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:43.022297 12753 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:43.022964  6548 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:43.025929  6548 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:43.029739 12989 log.cc:927] Time spent T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 12:19:43.031442 12980 log.cc:927] Time spent T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:19:43.032697 12653 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:43.033174 12653 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:43.033669 12653 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:19:43.036818 12962 log.cc:927] Time spent T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.486s	user 0.004s	sys 0.000s
I20260709 12:19:43.038565  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 937) took 1080 ms. Trace:
I20260709 12:19:43.028419  6548 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:43.040695  6584 rpcz_store.cc:276] 0709 12:19:41.957758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:42.210490 (+252732us) service_pool.cc:224] Handling call
0709 12:19:42.228408 (+ 17918us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:42.228698 (+   290us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:19:42.228740 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:42.268932 (+ 40192us) log.cc:844] Serialized 26 byte log entry
0709 12:19:42.482225 (+213293us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:42.482265 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:42.482336 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:43.036205 (+553869us) spinlock_profiling.cc:243] Waited 2.06 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:43.038282 (+  2077us) raft_consensus.cc:1685] finished
0709 12:19:43.038304 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:43.038507 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4746112,"thread_start_us":156407,"threads_started":1,"wal-append.queue_time_us":192560}
I20260709 12:19:43.046941  6548 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:43.049302  6548 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:43.052726 12963 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:43.053351 12963 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:43.054023 12931 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:43.023586 12954 consensus_queue.cc:1048] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:19:43.055620 12878 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:43.057265 12963 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:43.058801 12901 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:43.059403 12901 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:43.040786 12955 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:43.067039 12927 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:43.068490 12947 log.cc:927] Time spent T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.756s	user 0.004s	sys 0.000s
I20260709 12:19:43.070557 12927 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:43.072738  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:43.049480 12957 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:43.074024  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:43.075572  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:43.063601 12753 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:43.077451  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 90875us
I20260709 12:19:43.077558  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:43.062402 12955 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:43.079443  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:43.079897  6650 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:43.084826  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:43.087141  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:43.089154  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:43.092608  6576 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:43.093784  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:43.095065  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:43.092203  6650 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:19:43.089751  7267 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:43.099570  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:43.100131  6583 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:43.100958  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:43.109016  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:43.109800  6650 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:19:43.097345  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:43.111418  6576 raft_consensus.cc:2393] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 18: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:43.085873 12955 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:43.114820  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:43.115440  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:43.128172  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:43.086094 12878 consensus_queue.cc:1059] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:19:43.142586  6579 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:43.129659  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:43.131901  6576 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:19:43.121559  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:43.139511  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:43.114866  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:43.119524  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:43.170935  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:43.167898  6575 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 13.
I20260709 12:19:43.182650  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:43.183406  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:43.186859  6622 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:43.188720  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:43.190938  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:43.191911  6580 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:19:43.194815  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:43.196848  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:43.123519 12753 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:43.192288  6622 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:43.125289 12957 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:43.120934 12901 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:43.201246  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:43.203598  6583 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:43.204818  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:43.205963  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:43.210687  6581 raft_consensus.cc:1275] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:43.199553  6649 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:43.226814  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:43.212423  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:43.125715 12963 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:43.221465  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:43.224964  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 85716.4us
I20260709 12:19:43.227615  6575 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:43.134433 12887 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:43.119256 12903 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:43.125988 12881 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:43.236995  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:43.238386  6581 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:43.235898  6549 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:43.247588  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:43.242358  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:43.224689 12806 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:43.230501  6583 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:19:43.245361  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:43.251626  6649 raft_consensus.cc:2410] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:19:43.235778 12920 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:43.253929 12920 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:43.254336 12920 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:43.256096  6657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2613) took 1021 ms. Trace:
I20260709 12:19:43.257462  6576 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 12:19:43.257903  6576 pending_rounds.cc:85] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 10
I20260709 12:19:43.256528  6657 rpcz_store.cc:276] 0709 12:19:42.235840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:42.236092 (+   252us) service_pool.cc:224] Handling call
0709 12:19:42.236886 (+   794us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:42.237538 (+   652us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:42.237579 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:42.238034 (+   455us) log.cc:844] Serialized 26 byte log entry
0709 12:19:42.773629 (+535595us) spinlock_profiling.cc:243] Waited 3.14 ms on lock 0x7b18001b4ea0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:19:42.851767 (+ 78138us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:42.897841 (+ 46074us) log.cc:844] Serialized 14 byte log entry
0709 12:19:42.993150 (+ 95309us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:42.993282 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:43.252038 (+258756us) spinlock_profiling.cc:243] Waited 8.24 ms on lock 0x7b18001b4ea0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:19:43.255882 (+  3844us) raft_consensus.cc:1685] finished
0709 12:19:43.255905 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:43.256039 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":215,"prepare.queue_time_us":54544,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":26460,"raft.run_cpu_time_us":7214,"raft.run_wall_time_us":71705,"spinlock_wait_cycles":26175104,"thread_start_us":166107,"threads_started":2,"wal-append.queue_time_us":643747}
I20260709 12:19:43.258150  6576 pending_rounds.cc:107] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 12:19:43.258451  6576 raft_consensus.cc:2889] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:43.236163 12957 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:43.259534  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:43.250727  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:43.261723  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:43.262619  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:43.263255  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:43.250761  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:43.213568 12753 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:43.268700  6582 raft_consensus.cc:1240] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:43.270242  6575 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:19:43.271613  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:43.271711  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:43.273108  6575 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:43.272473  6654 raft_consensus.cc:1275] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:43.275257  6656 raft_consensus.cc:1240] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:43.275849  6580 raft_consensus.cc:2393] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 18: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:43.276134  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:43.277456  6583 raft_consensus.cc:3055] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:43.277980  6583 raft_consensus.cc:740] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:43.250748 12806 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:43.286953 12806 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:43.237205 12963 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:43.290524  6581 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:19:43.288650  6583 consensus_queue.cc:260] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:43.276822 12901 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:19:43.289108  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 53328.3us
I20260709 12:19:43.314901  6549 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:43.297585  6583 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:43.316519  6549 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:43.301422 12887 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:43.295482 12902 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 12:19:43.321645 12965 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:43.309576 12957 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:43.309471 12920 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:43.324731 12920 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:43.322077 12903 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:43.329576 13007 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:43.330683 12963 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:43.332266 12955 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:43.334625 12955 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:43.311033 12901 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:43.335515 12901 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:43.341224  6583 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:19:43.341955 12881 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:43.342608 12653 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:43.345079 12753 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:43.346763  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:43.346915  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:43.347245  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:43.356947  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 928) took 2147 ms. Trace:
I20260709 12:19:43.357366  6578 rpcz_store.cc:276] 0709 12:19:41.209929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:41.221413 (+ 11484us) service_pool.cc:224] Handling call
0709 12:19:41.221732 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:42.413827 (+1192095us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:43.356696 (+942869us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54000352d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:43.356883 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2737329920}
I20260709 12:19:43.372040  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:43.372854  6653 raft_consensus.cc:2393] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 17: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:43.373802  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:43.375602  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:43.375891 12806 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:43.377599 12639 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:43.378127 12639 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:43.378891 12935 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:43.379393 12935 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:43.379748  6473 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:43.381807 12584 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:43.381970 12935 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:43.382490 12935 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:43.382305 12584 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:43.382855 12935 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:43.383244 12935 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:43.384454 12931 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:43.384924 12931 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:43.385375 12670 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:43.385869 12670 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:43.386808 12931 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:43.385371  6474 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:43.387547 12941 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:43.387691 12670 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:43.388043 12941 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:43.388612 12584 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:43.389674  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:43.390105  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:43.390261  6507 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:43.312155  6622 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:43.398155  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:43.398793  6505 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
W20260709 12:19:43.398581  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 86536.7us
I20260709 12:19:43.400360  6474 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:43.400882  6473 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:43.397509  6575 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:19:43.380117 12639 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:43.408588  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:43.408124  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:43.410137  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:43.410871  6575 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:19:43.412205  6473 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:43.413697  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:43.414564  6505 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:19:43.420075  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:43.458987  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:43.461102  6578 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:43.428000  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:43.462268  6584 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:19:43.466837  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:43.468108  6576 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 9.
I20260709 12:19:43.397254 12653 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:43.395367 12753 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:43.471408  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:43.401935 12937 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:43.401794 12710 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:43.486608  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:43.426808 12581 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:43.490525  6622 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:43.428179 12758 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:43.426157 12584 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:43.462726 12806 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:43.419872 12931 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 12:19:43.498323  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:43.496208  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:43.499889  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:43.502125  6505 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:43.482887 12757 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:43.525303 12757 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:43.493777 12758 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:43.527140 12758 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:43.401861 13005 consensus_queue.cc:1048] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.092s
I20260709 12:19:43.491859 12581 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:43.493741 12937 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:43.530771 12581 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:43.536777  6550 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:43.396808 12901 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:43.490100 12753 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:43.494838 12992 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:43.494762 12850 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:43.494614 12903 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:43.521751 12584 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:43.522586 12710 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:43.524387  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:43.560746  6506 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:43.562021  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:43.426707 12866 consensus_queue.cc:1059] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 12:19:43.538726  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:43.509728 12995 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:43.568903  7272 raft_consensus.cc:2393] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 17: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:43.541718  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:43.545197  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:43.547737  6505 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:19:43.554714  6501 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:43.557688  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:43.558140  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:43.558410  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:43.500735 12902 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 12:19:43.554073 12850 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:43.578074  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:43.580808  6653 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 9.
I20260709 12:19:43.579187  6474 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:43.560036 12941 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:43.554422 13006 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:43.585999  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:43.568549 12903 raft_consensus.cc:697] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:43.588064  6506 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:43.591142 12639 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:43.593391  6658 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 17: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:43.594457  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:43.597182  6653 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:43.576838 12670 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:43.561264 12653 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:19:43.585144 12866 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:43.584756 12850 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:43.606431  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:43.606647  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:43.604827  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:43.614845  6501 raft_consensus.cc:2376] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18. Re-sending same reply.
W20260709 12:19:43.608183  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 112041us
I20260709 12:19:43.609164  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:43.617755  6505 raft_consensus.cc:2393] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 17: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:43.586668 12931 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:43.610446  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:43.621177  6507 raft_consensus.cc:2410] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 19 because replica has last-logged OpId of term: 18 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
I20260709 12:19:43.611987  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:43.621613  6500 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:43.612211  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:43.612725  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:43.613216  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:43.592379 12492 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:43.564790 12584 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:19:43.592970 13006 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:43.619702  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:43.582685  6474 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:43.630489  6502 raft_consensus.cc:2368] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 15. Current term is 16.
I20260709 12:19:43.630900  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:43.653754  6501 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:43.607275 12701 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:43.633388  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:43.632061  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:43.603214 12937 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:43.651950  6500 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:43.654726  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:43.656920  6622 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:43.662256  6622 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:43.664036  6658 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:43.659189  6651 raft_consensus.cc:1275] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:43.646168  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:43.666885  6508 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:43.626332  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:43.669047  6505 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:43.670480  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:43.669991  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:43.669050  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:43.677600  6500 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:43.671442  6550 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:43.682013  6502 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:19:43.682019  6500 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:43.655604 13006 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:43.658046 12796 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:43.684497  6500 raft_consensus.cc:1275] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:43.698017  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:43.698797  6500 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:43.630448 12639 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:43.688354  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:43.689034  6550 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:43.667874 12806 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:43.693171  6473 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:43.694250  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:43.701952  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 119372us
I20260709 12:19:43.659780  6653 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:43.648895 12931 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:43.669631 12963 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:43.708797  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:43.709841  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:43.712816  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:43.711740  6502 raft_consensus.cc:1275] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:43.716408 12963 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:43.718734 12963 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:43.712308 12760 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:43.719905 12760 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:43.720263 12760 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:43.720896 12757 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:43.691856 12887 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:43.712862 12927 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:43.710343 13013 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:43.725349 13010 log.cc:927] Time spent T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 12:19:43.727296 12931 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:43.713893  6576 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:19:43.730144  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:43.731011  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:43.734223 12914 raft_consensus.cc:3055] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:43.734759  6582 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:19:43.736989  6473 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:19:43.731746 13019 log.cc:927] Time spent T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 12:19:43.670334 13005 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:19:43.742175  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:43.739207  6505 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:43.668535 12701 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:43.748225  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:43.759524  6473 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:43.762389  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:43.762871  6502 raft_consensus.cc:2410] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 12:19:43.742184  6622 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:43.750079  6474 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:19:43.740579  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 51619.1us
I20260709 12:19:43.738642 12895 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:43.705896 13018 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:43.748457 12887 raft_consensus.cc:2804] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:43.783044  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:43.783625  6500 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:43.771759  6550 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:43.745317 12927 raft_consensus.cc:697] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:43.734766 12914 raft_consensus.cc:740] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:43.760203 12901 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:43.768668 12931 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:43.797959  6502 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:43.686674 12903 consensus_queue.cc:237] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:43.779934 12995 raft_consensus.cc:2804] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:19:43.772430 12895 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:43.814782  6583 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:43.794124 13013 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:43.795855 12511 consensus_queue.cc:1059] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:19:43.836829 12511 raft_consensus.cc:3055] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:43.837167 12511 raft_consensus.cc:740] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:43.837960 12511 consensus_queue.cc:260] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:43.839243 12511 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:43.796679 12757 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:43.841111  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:43.841653  6584 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:43.801122  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:43.809576  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:43.811880  6576 raft_consensus.cc:1275] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:19:43.790091 12887 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:43.839532 12935 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:43.805361  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:43.871373  6575 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:43.803813 12904 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:43.819767 13012 log.cc:927] Time spent T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.228s	user 0.000s	sys 0.001s
I20260709 12:19:43.823971 12753 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:43.825070 12639 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:43.827620 12927 consensus_queue.cc:237] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:43.776104 12957 consensus_queue.cc:1059] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:19:43.794018 13018 raft_consensus.cc:697] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:19:43.832737 12954 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 12:19:43.833288 12895 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:43.834736 12701 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:43.788592 12965 consensus_queue.cc:1048] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 12:19:43.807786  6582 raft_consensus.cc:1240] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:43.849943  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:43.851522  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:43.804042 12931 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:43.810963 12995 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:43.908681  6505 raft_consensus.cc:2376] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16. Re-sending same reply.
W20260709 12:19:43.906945  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 135247us
I20260709 12:19:43.910622  6575 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:43.893950  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3269) took 1182 ms. Trace:
I20260709 12:19:43.929281  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:43.930606  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:43.931501  6658 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:43.912976  6509 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 21. Current term is 22. Ops: []
I20260709 12:19:43.832756 12584 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:43.869342 12757 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:43.881844 12881 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:43.879029 12887 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:43.922605  6583 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:19:43.840596 12581 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:43.937834  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:43.946584  6505 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:19:43.947903  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:43.948621  6505 raft_consensus.cc:2393] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 17: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:43.949388  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:43.908381 12797 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:43.815567 12758 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:43.952515 12758 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:43.953579  6503 raft_consensus.cc:2393] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 20: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:43.954471  6505 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:19:43.967401  6584 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:19:43.962294  6550 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:43.974229  6550 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:43.928629  6655 rpcz_store.cc:276] 0709 12:19:42.711927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:42.712063 (+   136us) service_pool.cc:224] Handling call
0709 12:19:42.725832 (+ 13769us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:42.726022 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:19:43.388678 (+662656us) spinlock_profiling.cc:243] Waited 1.08 ms on lock 0x7b180036b140. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2875a314bc 00007f2872d7f48b 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d99455 00007f28798813c7 00007f2875e939a7 00007f2875e9c5e5
0709 12:19:43.388820 (+   142us) log.cc:844] Serialized 14 byte log entry
0709 12:19:43.893331 (+504511us) log.cc:844] Serialized 14 byte log entry
0709 12:19:43.893547 (+   216us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:43.893605 (+    58us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:19:43.893662 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:43.893731 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:43.893896 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15481,"prepare.queue_time_us":42603,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":11943,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":8487,"spinlock_wait_cycles":43592832,"thread_start_us":541087,"threads_started":3,"wal-append.queue_time_us":360221}
I20260709 12:19:43.935048 12931 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:43.937939 12954 raft_consensus.cc:3055] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:43.999804 12954 raft_consensus.cc:740] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:44.000764 12954 consensus_queue.cc:260] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:44.002061 12954 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:43.938189 12995 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:43.905442 12914 consensus_queue.cc:260] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:44.006847 12914 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:44.010310  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:44.011186  6506 raft_consensus.cc:2393] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 17: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:43.985229  6475 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:43.986577 13032 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:43.987105 12881 raft_consensus.cc:697] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:43.986800 13018 consensus_queue.cc:237] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:43.988521 12957 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:43.990849  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:43.991138 12904 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:19:43.991981 13023 log.cc:927] Time spent T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 12:19:43.995906  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:43.997941 12581 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:43.999313 12935 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:43.938889 12757 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:44.031726  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 46581.2us
I20260709 12:19:44.074220  6475 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:44.012148  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:44.045384  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:43.970608 12955 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:19:44.071365  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 80588.6us
I20260709 12:19:44.075909  6475 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:44.086920  6622 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:44.033910 13032 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:19:44.106990  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:44.094172  6475 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:44.116148  6622 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:44.122942 13006 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:44.124886 13028 log.cc:927] Time spent T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 12:19:44.085779 13013 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:44.048061 12701 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:44.133230 12701 raft_consensus.cc:697] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 17 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:44.134248 12701 consensus_queue.cc:237] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:44.114539 12904 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:43.984687 12760 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:44.166487 12760 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:44.120223 12941 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:44.114202 12584 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:44.176743  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:44.179033  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 63005.3us
I20260709 12:19:44.181263  6622 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:44.115123  6505 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:19:44.148746 12881 consensus_queue.cc:237] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:44.150069  6475 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:44.150728 13030 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:44.137854 13032 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:44.154078  7238 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.919s	user 0.038s	sys 0.014s
W20260709 12:19:44.196503  7238 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.961s	user 0.040s	sys 0.014s
W20260709 12:19:44.196862  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 961375us
I20260709 12:19:44.198261  7238 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:44.198781  6505 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:19:44.199482  7238 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:44.156205  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:44.157622 12797 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:44.158905 12653 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:44.159586 12757 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:44.209527  6622 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:44.209738  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:44.192813  6475 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:44.156318 12581 consensus_queue.cc:1048] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:44.203301  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:44.213718  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:44.213724  6578 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:44.214772  6475 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:44.191473 13005 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:44.114904 12955 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:44.147996 12850 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:44.203842 13023 log.cc:927] Time spent T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:19:44.214792  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:44.241956  6584 raft_consensus.cc:2393] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 20: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:44.208316  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:44.243152  6579 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:44.215983  6475 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:44.245890  6475 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:44.193643 13030 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:44.219286  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:44.249893  6582 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:44.210626  6622 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:44.201113 13013 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:44.205205 12797 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:44.209286 12904 raft_consensus.cc:697] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:19:44.259307  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 48723.4us
I20260709 12:19:44.213574 12914 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:44.213861 12710 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:44.265563  6622 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:44.235476 12908 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:44.211937 13033 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:44.218588 12912 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:44.233318 12937 raft_consensus.cc:2764] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 12:19:44.249624 12866 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:44.249280 12935 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:44.212265 12653 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:44.257699 13030 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:44.218439 13008 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.075s
I20260709 12:19:44.217218 12931 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:44.279162 12992 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:44.300294 12908 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:44.286720 12920 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:19:44.293208 12895 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:44.338362  6500 raft_consensus.cc:2393] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 13: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:44.318228 13033 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:44.339033 12992 raft_consensus.cc:697] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:44.356135 12992 consensus_queue.cc:237] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:44.336280 12908 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:44.321671 13006 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:44.316044 12874 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
W20260709 12:19:44.413564  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 166289us
I20260709 12:19:44.424743  6548 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:44.427026 13021 log.cc:927] Time spent T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.586s	user 0.002s	sys 0.000s
I20260709 12:19:44.433682 13009 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:44.444859 13013 raft_consensus.cc:697] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:19:44.446789  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:44.447216 12758 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:44.384500 13033 raft_consensus.cc:697] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:19:44.499294  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 74659.4us
I20260709 12:19:44.512061  6548 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:44.372387 13032 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:44.518198  6473 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
W20260709 12:19:44.521392  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 103619us
I20260709 12:19:44.353235 12904 consensus_queue.cc:237] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:44.527740  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:44.529215  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:44.530057  6503 raft_consensus.cc:2393] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 20: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:19:44.534577  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:44.531418  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:44.538129  6548 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:44.540639 12701 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:44.541148 12701 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:44.541644 12701 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:44.513041 12584 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:44.513271 12639 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:44.538975 12957 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:19:44.526376 12903 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.015s	sys 0.037s
I20260709 12:19:44.549856  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:44.554162  6579 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:44.551008  6548 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:44.522145 12757 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:44.557257  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 349473us
I20260709 12:19:44.560972  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:44.562548  6584 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:44.537770 12914 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:44.563786  6548 leader_election.cc:400] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 15. Current term is 16.
W20260709 12:19:44.566646  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:44.567060  6548 leader_election.cc:403] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
W20260709 12:19:44.555950 12903 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.015s	sys 0.037s
I20260709 12:19:44.568846 12757 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:44.585964 12757 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:44.586511 13031 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 15. Current term is 16.
W20260709 12:19:44.593977 13038 log.cc:927] Time spent T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.211s	user 0.000s	sys 0.000s
I20260709 12:19:44.594111 13007 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:44.594982 13007 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:44.598951  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:44.599617  6658 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:44.563187 12955 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:44.555042 13041 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:44.606600  6504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2715) took 1177 ms. Trace:
I20260709 12:19:44.607025  6504 rpcz_store.cc:276] 0709 12:19:43.429024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:43.431931 (+  2907us) service_pool.cc:224] Handling call
0709 12:19:43.482364 (+ 50433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:43.488133 (+  5769us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:43.488174 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:43.493268 (+  5094us) log.cc:844] Serialized 26 byte log entry
0709 12:19:43.687997 (+194729us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:43.688037 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:43.688470 (+   433us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:44.426038 (+737568us) spinlock_profiling.cc:243] Waited 6.34 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:44.566299 (+140261us) raft_consensus.cc:1685] finished
0709 12:19:44.566323 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:44.606522 (+ 40199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":544,"raft.queue_time_us":39468,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":122,"spinlock_wait_cycles":14584192,"thread_start_us":174813,"threads_started":1,"wal-append.queue_time_us":260978}
I20260709 12:19:44.608486 13007 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:44.610018 12895 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:44.615746 12887 consensus_meta.cc:287] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.646s	user 0.007s	sys 0.007s
I20260709 12:19:44.516847 12758 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:44.562704 12581 consensus_queue.cc:1059] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 12:19:44.556463 12850 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:44.576835 12653 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:44.578151  6548 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:44.578301 13013 consensus_queue.cc:237] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:44.579563 13045 log.cc:927] Time spent T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 12:19:44.580385 13044 log.cc:927] Time spent T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
W20260709 12:19:44.582705 13039 log.cc:927] Time spent T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 12:19:44.730501  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:44.583580 13042 log.cc:927] Time spent T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 12:19:44.584457 13036 log.cc:927] Time spent T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 12:19:44.584621 12931 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:44.618609  6584 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:19:44.741508  6579 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:19:44.613749 12701 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:44.620365  6658 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
I20260709 12:19:44.621657 13032 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:44.514201 13009 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:44.630568  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:44.631873  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 120346us
W20260709 12:19:44.735467 12511 consensus_meta.cc:287] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.896s	user 0.014s	sys 0.003s
I20260709 12:19:44.576292 13033 consensus_queue.cc:237] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:44.664450  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:44.719300  6508 raft_consensus.cc:1240] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 12. Current term is 13. Ops: []
W20260709 12:19:44.722203  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 144101us
I20260709 12:19:44.556789 12584 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:44.742537  6576 raft_consensus.cc:1240] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:44.660077 13034 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:44.752650  6650 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:44.753353  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:44.753780  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:44.755743  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 490229us
I20260709 12:19:44.622968 12914 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:44.767396  6582 consensus_meta.cc:287] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.509s	user 0.007s	sys 0.003s
I20260709 12:19:44.910341  6582 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:19:44.631423  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:44.767822  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:19:44.780754  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:44.782123  6548 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:44.660914 12887 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:44.781343  6653 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:44.921783  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 139784us
I20260709 12:19:44.922955  6548 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:44.919973  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:44.728734 12653 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:44.782124  6652 raft_consensus.cc:1240] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19. Ops: []
W20260709 12:19:44.893419  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 139988us
I20260709 12:19:44.941431  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:44.909409  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:44.940151  6473 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:19:44.897643  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 118499us
I20260709 12:19:44.908406  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:44.895388  6622 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:44.918363  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 127066us
I20260709 12:19:44.884692  6649 raft_consensus.cc:1275] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:44.667795 13030 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:44.954276  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 2590) took 1141 ms. Trace:
I20260709 12:19:44.959700  6577 rpcz_store.cc:276] 0709 12:19:43.813259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:43.814167 (+   908us) service_pool.cc:224] Handling call
0709 12:19:43.835361 (+ 21194us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:43.938748 (+103387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:43.938783 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:43.942547 (+  3764us) log.cc:844] Serialized 26 byte log entry
0709 12:19:44.200325 (+257778us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:44.200385 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:44.200455 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:44.456508 (+256053us) spinlock_profiling.cc:243] Waited 5.13 ms on lock 0x7b48000030d8. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287adb2aa1 00007f287ada9b37 00007f287ada99ca 00007f287aea126f 00007f287aea0f37 00007f287aeb555c 00007f287aeb5508 00007f287aeb5488 00007f287aeb5443 00007f287aeb46ec 00007f2879ddea96
0709 12:19:44.949246 (+492738us) raft_consensus.cc:1685] finished
0709 12:19:44.949267 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:44.954209 (+  4942us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37993,"raft.queue_time_us":57540,"raft.run_cpu_time_us":3392,"raft.run_wall_time_us":551063,"spinlock_wait_cycles":11808128,"thread_start_us":298853,"threads_started":2,"wal-append.queue_time_us":353417}
I20260709 12:19:44.960645  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:44.961666  6473 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:44.963124  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:44.966625  6657 raft_consensus.cc:2393] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 20: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:44.962491  6473 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:44.911340 12581 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:44.969784  6578 raft_consensus.cc:1240] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:44.966272  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:44.967458  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:19:44.972075  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 49242.5us
I20260709 12:19:44.972388  7238 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:44.973567  7238 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:44.974556  6653 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:44.974001  6548 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:44.976116  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:44.921164 12492 raft_consensus.cc:2764] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
W20260709 12:19:44.980863  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 85553.9us
I20260709 12:19:44.965229 12670 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:44.954931 13057 log.cc:927] Time spent T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:19:44.982949  7238 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:44.942340 13007 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:45.008625  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:45.009884  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 777) took 1196 ms. Trace:
I20260709 12:19:45.011725  6656 raft_consensus.cc:1240] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 16. Current term is 17. Ops: []
W20260709 12:19:45.024573  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 50923.9us
I20260709 12:19:44.948769 12957 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:45.041085  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:45.044389  6651 raft_consensus.cc:1240] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:44.969257 13049 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:44.937320 12653 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:45.011909  6509 rpcz_store.cc:276] 0709 12:19:43.812999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.017417 (+204418us) service_pool.cc:224] Handling call
0709 12:19:44.017748 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:44.018093 (+   345us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:44.018130 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:44.019025 (+   895us) log.cc:844] Serialized 26 byte log entry
0709 12:19:44.333318 (+314293us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:44.333359 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:44.333424 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:45.009621 (+676197us) raft_consensus.cc:1685] finished
0709 12:19:45.009645 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:45.009810 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2107,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":13452,"thread_start_us":215121,"threads_started":1,"wal-append.queue_time_us":317289}
I20260709 12:19:45.055706  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:44.909535 13034 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:45.009176 13036 log.cc:927] Time spent T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:19:44.981359 12878 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:45.145745  6501 raft_consensus.cc:1240] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:45.037407 13006 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:45.013386 13018 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:45.328179 12939 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.092s
I20260709 12:19:45.133214 12850 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:45.360584 13007 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:45.251348 13032 consensus_queue.cc:1059] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.118s
I20260709 12:19:45.275507 12511 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:45.054430 13015 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:19:45.281276 13020 log.cc:927] Time spent T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 1.487s	user 0.004s	sys 0.001s
I20260709 12:19:45.069515 12957 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:45.443883 12701 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:44.999785 12581 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:45.075938 12935 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:19:45.306514 13048 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.572s	user 0.000s	sys 0.001s
I20260709 12:19:45.131385 12653 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:45.500594 13050 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:45.511409 12584 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:45.304704 12950 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:45.305454 13058 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:45.033496 12670 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:45.356873  6502 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2720) took 1550 ms. Trace:
I20260709 12:19:45.249220 13052 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:45.521252 13015 raft_consensus.cc:697] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 13 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:45.146308 13049 raft_consensus.cc:697] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:45.564636 13015 consensus_queue.cc:237] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:45.565091 13049 consensus_queue.cc:237] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:45.248521 12963 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:45.453265  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:45.568701 12760 raft_consensus.cc:3055] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 12:19:45.484087  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:45.569427 12760 raft_consensus.cc:740] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:45.393216 12758 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:45.570675 12760 consensus_queue.cc:260] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:45.571648  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:45.354429 13059 consensus_queue.cc:1059] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.127s
I20260709 12:19:45.498476 13009 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:45.488024 13018 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:45.505154 13007 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:45.509189 12757 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:45.541837  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 486706us
W20260709 12:19:45.510282 12908 consensus_meta.cc:287] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 1.122s	user 0.007s	sys 0.002s
I20260709 12:19:45.510871  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2736) took 1076 ms. Trace:
W20260709 12:19:45.444350  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 291355us
W20260709 12:19:45.568696 13067 log.cc:927] Time spent T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
W20260709 12:19:45.498884 12995 consensus_meta.cc:287] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 1.495s	user 0.012s	sys 0.013s
W20260709 12:19:45.523501 13060 log.cc:927] Time spent T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:19:45.056244 12895 consensus_queue.cc:1059] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.241s
I20260709 12:19:45.597671  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:45.559262 13072 log.cc:927] Time spent T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:19:45.560246 13034 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:45.553248 12935 raft_consensus.cc:697] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:45.563339 12957 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:45.543555 12701 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:45.608677 12701 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:45.601544 12995 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:45.522002 12887 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:45.497140 13065 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 12:19:45.577690 12963 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:45.612562  6654 raft_consensus.cc:1240] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 9. Current term is 10. Ops: []
I20260709 12:19:45.606580 12954 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 12:19:45.613703 13007 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:45.614145 12963 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:45.615590 13057 log.cc:927] Time spent T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.557s	user 0.004s	sys 0.000s
I20260709 12:19:45.568833  6503 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:45.492645  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:45.079988 13041 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:45.603462  6548 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:45.539793  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 77424.5us
W20260709 12:19:45.539143 12797 consensus_meta.cc:287] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 1.250s	user 0.014s	sys 0.000s
W20260709 12:19:45.569337 13061 log.cc:927] Time spent T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:19:45.635566  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:45.555639  6502 rpcz_store.cc:276] 0709 12:19:43.805997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:43.874781 (+ 68784us) service_pool.cc:224] Handling call
0709 12:19:43.875112 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:43.875470 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:43.875496 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:43.887527 (+ 12031us) log.cc:844] Serialized 26 byte log entry
0709 12:19:44.206755 (+319228us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:44.234379 (+ 27624us) log.cc:844] Serialized 14 byte log entry
0709 12:19:44.234642 (+   263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:44.234712 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:44.443283 (+208571us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b1800358ea0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d16c0a 00007f287c456219 00007f2879d5d58f 00007f2872d9c571 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd
0709 12:19:45.347841 (+904558us) raft_consensus.cc:1685] finished
0709 12:19:45.356589 (+  8748us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:45.356794 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23347,"prepare.queue_time_us":27390,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":41,"raft.queue_time_us":3466,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":30630,"spinlock_wait_cycles":28415360,"thread_start_us":270688,"threads_started":1,"wal-append.queue_time_us":406701}
I20260709 12:19:45.487617 12866 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:45.584257  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:45.581344  6581 rpcz_store.cc:276] 0709 12:19:44.434255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.453551 (+ 19296us) service_pool.cc:224] Handling call
0709 12:19:44.454019 (+   468us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:44.521352 (+ 67333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:44.521385 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:44.526423 (+  5038us) log.cc:844] Serialized 26 byte log entry
0709 12:19:44.983712 (+457289us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:44.983809 (+    97us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:44.983880 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:45.510633 (+526753us) raft_consensus.cc:1685] finished
0709 12:19:45.510664 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:45.510816 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33722,"raft.queue_time_us":29064,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":10116,"thread_start_us":418373,"threads_started":1,"wal-append.queue_time_us":869332}
I20260709 12:19:45.580662 13050 raft_consensus.cc:3055] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:45.588469 12760 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:45.588775 13030 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:45.591868 12670 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:45.539368 13063 consensus_queue.cc:1059] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.065s
I20260709 12:19:45.515971 12511 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:45.578770 13018 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:45.585727 12908 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:45.619258 13069 log.cc:927] Time spent T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.266s	user 0.003s	sys 0.000s
I20260709 12:19:45.597026 13081 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:45.597193 12639 consensus_meta.cc:287] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 1.042s	user 0.007s	sys 0.005s
I20260709 12:19:45.611948 12995 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:45.639545  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:45.653352  6506 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:45.640002  6503 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:19:45.630303 13031 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:19:45.640609  6547 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.002s	sys 0.010s
W20260709 12:19:45.640390  6550 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.008s	sys 0.003s
W20260709 12:19:45.656378  6550 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.008s	sys 0.003s
W20260709 12:19:45.656239  6547 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.002s	sys 0.010s
I20260709 12:19:45.605834 13032 consensus_queue.cc:1048] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:19:45.641251 13050 raft_consensus.cc:740] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:45.662842 13006 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:45.663403 13006 raft_consensus.cc:697] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:45.663126 13050 consensus_queue.cc:260] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:45.664321 13006 consensus_queue.cc:237] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:45.668443 13031 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:45.669481 12935 consensus_queue.cc:237] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:45.654745 12941 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.012s	sys 0.000s
I20260709 12:19:45.690260  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:19:45.657382  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 54017.5us
I20260709 12:19:45.636750 12937 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:45.692565  6548 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:45.645208 12866 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:45.641396 12797 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:45.661579  7267 raft_consensus.cc:1240] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:45.662241 12908 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:45.691470  6506 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:19:45.692432 12937 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:45.652659  6473 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.004s	sys 0.009s
W20260709 12:19:45.708591  6473 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.004s	sys 0.009s
W20260709 12:19:45.708963  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 746463us
I20260709 12:19:45.713570  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:45.714970  6575 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 12:19:45.655242  7238 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.006s	sys 0.007s
W20260709 12:19:45.716442  7238 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.007s	sys 0.007s
W20260709 12:19:45.716773  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 639734us
I20260709 12:19:45.656183 13081 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:45.704969  6548 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:45.655037 12639 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:45.655251  6475 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.006s	sys 0.004s
W20260709 12:19:45.724016  6475 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.007s	sys 0.005s
W20260709 12:19:45.724761  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 669079us
I20260709 12:19:45.725834  6475 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:45.728041  6580 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:45.734400  6548 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:45.737375 12954 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:45.737974 12954 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:45.734838  6475 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:45.742280  6655 raft_consensus.cc:1240] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:45.739557  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:45.749399  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:45.749825  6582 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:19:45.642717 13041 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:45.728199 12887 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:19:45.753036  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:45.757969  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:45.760663  6577 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:45.760955 12954 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:45.761320 13031 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:45.744347 12701 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:45.739974 12581 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:45.772734  7239 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.004s	sys 0.008s
I20260709 12:19:45.772840 13065 raft_consensus.cc:3055] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:19:45.773458  7239 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.004s	sys 0.008s
I20260709 12:19:45.773494 13065 raft_consensus.cc:740] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:45.774205 12511 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:45.774590 13065 consensus_queue.cc:260] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:45.775570 13014 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.006s	sys 0.005s
I20260709 12:19:45.751471 12639 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:19:45.776052 13014 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.006s	sys 0.005s
I20260709 12:19:45.779124  6580 raft_consensus.cc:2393] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 21: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:45.786993  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:45.798457  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 63936.7us
I20260709 12:19:45.799408  6475 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:45.803718  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 67168.8us
I20260709 12:19:45.775977 13065 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:45.814778  6505 raft_consensus.cc:1240] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:19:45.757068 13075 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:45.817520  6509 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 12:19:45.824268  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:45.826164  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:45.829766  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:45.830471  6582 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:45.832878 12760 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:45.833418 12760 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:45.833972 13088 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:45.834970 13088 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:45.835638 12760 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:45.824378 12941 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:45.836812 13077 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 12:19:45.841459  6473 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:45.841709 12670 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:45.843715 13080 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:45.844569 13080 consensus_queue.cc:1059] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:45.836809 12941 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:45.846055 13080 raft_consensus.cc:3055] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:19:45.847277  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 47965.3us
I20260709 12:19:45.848284 12941 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:45.848634  6475 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:45.843401  6473 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:45.849493 12757 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:45.836189 12760 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:45.850121 12760 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:45.848806 13075 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:45.846462 13080 raft_consensus.cc:740] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:45.778478 13090 consensus_queue.cc:1059] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:19:45.852546  6473 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:45.810395 12701 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:19:45.855213  6475 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:45.856832  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 171713us
I20260709 12:19:45.853919  6577 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 14.
I20260709 12:19:45.782853 12850 raft_consensus.cc:697] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:45.865211  7238 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:45.858520 13014 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:45.860882  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:45.870092  6583 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:45.861815 12760 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:45.877082  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:45.877924  6653 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:45.871628  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:45.873250 13084 consensus_queue.cc:1059] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:45.874607 13054 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:45.878396  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:45.882364  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:45.879609  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:45.880466  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:45.885304  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:45.885915  6658 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:45.887288  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:45.887389  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:45.889284  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 150146us
I20260709 12:19:45.865485 13013 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:45.825728 12653 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:45.866914 13015 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:45.884928 13063 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:45.832088 13018 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:45.910519 13041 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:45.911216 13063 raft_consensus.cc:697] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:45.911878 13018 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:45.866672 13014 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:45.912184 13063 consensus_queue.cc:237] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:45.912956 13014 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:45.897019  6473 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:45.881410 13081 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:45.890017 13054 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:45.916435 13094 log.cc:927] Time spent T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:19:45.917100 13054 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:45.902006 13089 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:19:45.902786 12653 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:45.888114 12850 consensus_queue.cc:237] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:45.918581  6548 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:45.920763 12653 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:45.887312 13100 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:45.904548  6584 raft_consensus.cc:1240] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:45.897710 13080 consensus_queue.cc:260] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:45.888947 12760 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:45.908797 12797 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:45.909013 13088 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:45.909550 12937 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:45.929231 13083 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:45.930048 13080 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:45.926996 13015 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:45.942404 13015 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:45.942835 13015 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:45.928599  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:45.928970 13100 raft_consensus.cc:697] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 14 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:45.945559 13100 consensus_queue.cc:237] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:45.947731 12581 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:45.948302 12581 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:45.950649 12581 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:45.951231 12581 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:45.951716 12581 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:45.944087 13083 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:45.948309 13075 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:45.953256 13075 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:45.954348 12581 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:45.954923 12581 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:45.955646 13083 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:45.955418 12581 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:45.956501  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:45.921954  6548 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:45.961184  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:45.964265  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:45.966284  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:45.967928  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:45.968914  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:45.974335  6475 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:45.926286 12874 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:45.988528  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:45.990792  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:45.989387  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:46.001628  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:46.002461  6501 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:46.008154  6475 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:45.987812 12653 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:45.978451 13075 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:46.025774  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:46.031636  6501 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:19:46.026135 13077 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:45.959488 12760 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:46.011811 12937 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:46.035748  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:46.014596 13054 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:46.039726 13054 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:46.040136 13054 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:46.040267  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:46.042164  6577 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 18. Current term is 19. Ops: [16.9-18.10]
I20260709 12:19:46.038059  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:46.014750 13081 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:46.038327  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:46.041465  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:46.046032  6579 raft_consensus.cc:1240] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:46.029572 12963 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:46.052202  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:46.035252 13079 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:46.008975 12874 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:46.073997  6577 raft_consensus.cc:1275] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 12:19:46.077219  6504 raft_consensus.cc:1275] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:19:46.078130 13107 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:46.078933 13107 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:46.080150 13066 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:19:46.080919 13105 log.cc:927] Time spent T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
I20260709 12:19:46.080679 13066 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:46.082816 13066 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:46.083240 12581 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:46.066891 12904 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:46.089460 12904 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:46.091843 12904 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:46.050464 13081 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:46.094152 13081 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:46.094712 12760 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:46.033391 13077 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:46.098119 12914 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:46.098490 13077 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:46.101003  6583 raft_consensus.cc:1275] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:46.101361 13107 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:46.104166  6652 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:19:46.075392 13079 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:46.107376 13079 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:46.109041 13015 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:46.109081  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 944) took 1456 ms. Trace:
I20260709 12:19:46.111020 13015 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:46.111020 12874 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:46.111114  6578 rpcz_store.cc:276] 0709 12:19:44.652778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.654417 (+  1639us) service_pool.cc:224] Handling call
0709 12:19:44.722711 (+ 68294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:44.984196 (+261485us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:46.108745 (+1124549us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b5400420254. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:46.108998 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":567470976}
I20260709 12:19:46.113448  6502 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:46.115525 12881 raft_consensus.cc:3055] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:46.116083 12881 raft_consensus.cc:740] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:46.117045 12881 consensus_queue.cc:260] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:46.118120 12881 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:46.122471  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:46.123028  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:46.123888  6583 raft_consensus.cc:2393] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 10: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:46.125466  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:46.126152  6578 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:46.126258  6583 raft_consensus.cc:1240] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:19:46.127635  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:46.131886  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:46.136185  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:46.141439  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:46.142405  6575 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:46.143030  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:46.143834  6502 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:46.135390 12757 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:46.150563  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:46.151733 12757 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:46.154613 12757 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:46.156481 12492 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:46.157054 12492 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:46.159185 12492 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:46.149327 12904 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:46.163699 12904 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:46.165648 12904 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:46.165844  7267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38546 (request call id 777) took 1747 ms. Trace:
I20260709 12:19:46.166241  7267 rpcz_store.cc:276] 0709 12:19:44.418223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.420275 (+  2052us) service_pool.cc:224] Handling call
0709 12:19:44.421020 (+   745us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:45.706502 (+1285482us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:46.165615 (+459113us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54005db1d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:46.165788 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2852507008}
I20260709 12:19:46.150679 13104 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:46.172845 12902 raft_consensus.cc:3055] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:19:46.173555 12902 raft_consensus.cc:740] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:46.174386  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:46.174791 13065 consensus_queue.cc:1059] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:46.174573 12902 consensus_queue.cc:260] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:46.175591 13065 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:46.175905 12902 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:46.176294 13065 raft_consensus.cc:3055] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:19:46.172995  6550 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.3:36107 timed out after 1.577s (SENT) [suppressed 18 similar messages]
I20260709 12:19:46.179250  6579 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:19:46.179551  6548 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:46.179839  6579 raft_consensus.cc:606] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:46.180177  6579 raft_consensus.cc:614] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:46.183624  6548 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:46.172870 13104 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:19:46.185567  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:46.185855 13033 consensus_queue.cc:1059] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:46.186280  6506 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 18: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:46.186734 12797 raft_consensus.cc:3055] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:46.186889 13104 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:46.187146 12797 raft_consensus.cc:740] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:46.187456 12992 consensus_queue.cc:1048] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:19:46.187067  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:46.189005  6506 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 22: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:46.189335  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3294) took 1591 ms. Trace:
I20260709 12:19:46.189728  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:46.189680  6655 rpcz_store.cc:276] 0709 12:19:44.597570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.606526 (+  8956us) service_pool.cc:224] Handling call
0709 12:19:44.606795 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:45.746707 (+1139912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:46.189117 (+442410us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400227ed4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:46.189278 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2610843392}
I20260709 12:19:46.190388  6506 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:46.191286  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:46.191499  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2727) took 1051 ms. Trace:
I20260709 12:19:46.191614 12511 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:46.191975  6509 rpcz_store.cc:276] 0709 12:19:45.140479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:45.142492 (+  2013us) service_pool.cc:224] Handling call
0709 12:19:45.142813 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:45.819178 (+676365us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:46.191270 (+372092us) spinlock_profiling.cc:243] Waited 670 ms on lock 0x7b54002c38d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:46.191439 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1540387584}
I20260709 12:19:46.193459 12511 consensus_queue.cc:1059] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:46.194279 12511 raft_consensus.cc:3055] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:46.194603 12511 raft_consensus.cc:740] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:46.195381 12511 consensus_queue.cc:260] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:46.177320  6475 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:46.191812  6506 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:19:46.198761  6506 raft_consensus.cc:1275] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:46.203365  6501 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:46.135658 13015 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:46.183828  6575 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:19:46.206286 12904 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 12:19:46.184278 12492 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:46.208829  6504 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:46.214033  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:46.214383  6575 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "55df50d51529421789bb752f47dff41a"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:19:46.188081 12797 consensus_queue.cc:260] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:46.210888 12492 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:46.214955  6575 raft_consensus.cc:606] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:46.215958  6582 raft_consensus.cc:1275] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:19:46.196777 12878 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:46.184228 13065 raft_consensus.cc:740] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:46.205518 13077 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:46.228593 13077 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:46.227676  6575 raft_consensus.cc:614] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:46.229018 13077 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:46.138521 13014 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:46.216305  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:46.225118  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:46.233141  6504 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:19:46.225176  6579 raft_consensus.cc:1275] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:19:46.226290  6622 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:46.235476  6506 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 18. Current term is 19. Ops: []
W20260709 12:19:46.240325  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:46.205439 12511 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:46.262162  6622 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:46.215655 12797 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:46.217123 12492 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:46.270090  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:46.207785 13015 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:46.258817 13104 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:46.275388  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:46.276585  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:46.278879  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 105899us
I20260709 12:19:46.226001 12904 consensus_queue.cc:1059] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 12:19:46.269804 12881 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:46.284837 12927 consensus_queue.cc:1048] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:46.285748 13107 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:46.274586 13090 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:46.259148 13092 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 12:19:46.293973  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:46.286329  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:46.261044 13054 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:46.296721  6578 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:19:46.295428  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:46.295591  6576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 950) took 1628 ms. Trace:
I20260709 12:19:46.298329  6505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2731) took 1154 ms. Trace:
I20260709 12:19:46.279742 13104 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:46.297600  6576 rpcz_store.cc:276] 0709 12:19:44.666988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.667249 (+   261us) service_pool.cc:224] Handling call
0709 12:19:44.742320 (+ 75071us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:44.896131 (+153811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:46.295398 (+1399267us) spinlock_profiling.cc:243] Waited 74.7 ms on lock 0x7b540034e4d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:46.295536 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":171719040}
I20260709 12:19:46.298733  6505 rpcz_store.cc:276] 0709 12:19:45.143924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:45.144048 (+   124us) service_pool.cc:224] Handling call
0709 12:19:45.144364 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:45.815318 (+670954us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:46.298109 (+482791us) spinlock_profiling.cc:243] Waited 670 ms on lock 0x7b54001e91d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:46.298278 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1541161344}
I20260709 12:19:46.297327  6578 raft_consensus.cc:606] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:46.305068  6578 raft_consensus.cc:614] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:46.274134 13063 consensus_queue.cc:1059] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 12:19:46.301167  6502 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:46.300418  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 782) took 2386 ms. Trace:
I20260709 12:19:46.292583 12904 consensus_queue.cc:1243] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 19.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:19:46.306142  6504 raft_consensus.cc:1240] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:46.306563  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:46.306757  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:46.308022  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3299) took 1679 ms. Trace:
I20260709 12:19:46.278879 12895 consensus_queue.cc:1048] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:19:46.291687 13014 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:46.310647  6505 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 21. Current term is 22. Ops: []
I20260709 12:19:46.309892  6508 rpcz_store.cc:276] 0709 12:19:43.914254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.132945 (+218691us) service_pool.cc:224] Handling call
0709 12:19:44.667903 (+534958us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:44.883637 (+215734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:46.300139 (+1416502us) spinlock_profiling.cc:243] Waited 535 ms on lock 0x7b54000ff350. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:46.300287 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1229438208}
I20260709 12:19:46.313642  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:46.305487 13104 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:46.312934  6656 rpcz_store.cc:276] 0709 12:19:44.628653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.628925 (+   272us) service_pool.cc:224] Handling call
0709 12:19:44.629269 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:45.041965 (+412696us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:46.307756 (+1265791us) spinlock_profiling.cc:243] Waited 382 ms on lock 0x7b54001509d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:46.307959 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":878909312}
I20260709 12:19:46.323463  6502 raft_consensus.cc:1240] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:46.320024  7268 raft_consensus.cc:3055] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:46.330333  6505 raft_consensus.cc:1240] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:46.307041 13090 raft_consensus.cc:697] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:46.350344 13090 consensus_queue.cc:237] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:46.329209  7268 raft_consensus.cc:740] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:46.356474 12904 raft_consensus.cc:3055] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:46.356981 12904 raft_consensus.cc:740] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:46.358214 13015 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:46.358026 12904 consensus_queue.cc:260] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:46.358975 13110 log.cc:927] Time spent T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.357s	user 0.000s	sys 0.002s
I20260709 12:19:46.359630 12904 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:46.295485 13107 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:46.363595 13107 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:46.363366  7268 consensus_queue.cc:260] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:46.339289 13054 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:46.294299 12881 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:46.369557  7268 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:46.377544  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 103790us
I20260709 12:19:46.384539 12895 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:46.385102 12895 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:46.385663 12881 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:46.387280 12895 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:46.392297  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:46.392714  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:46.397377  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:46.390897  6550 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:46.406786  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:46.407232  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:46.407943  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:46.408670  6653 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 14.
I20260709 12:19:46.409828  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:46.410809  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:46.411157  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:46.413765  6651 raft_consensus.cc:1275] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:19:46.415968  7269 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:19:46.416463  6548 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:46.402134  6550 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:46.418433  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:46.418740  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:46.420297  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:46.422490  6655 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:46.422932  6653 raft_consensus.cc:2410] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 19 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:19:46.420491  6475 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:46.420640  6652 raft_consensus.cc:1240] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:19:46.423301  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:46.424456 13081 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:46.425024 13054 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:46.424870 13078 consensus_queue.cc:1059] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:46.425397 13054 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:46.425568 12935 raft_consensus.cc:3055] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:46.426244 12935 raft_consensus.cc:740] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:46.424480 12878 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:46.427714 12878 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:46.428124 12878 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:46.430301 13065 consensus_queue.cc:260] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:46.425769 13054 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:46.432763  7238 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:46.434161 12639 consensus_queue.cc:1059] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:46.437204  7268 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:19:46.411768 12881 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:46.441454 13091 raft_consensus.cc:3055] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:46.441913 13091 raft_consensus.cc:740] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:46.438853 13014 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 12:19:46.442833 13091 consensus_queue.cc:260] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:46.443751 13117 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:46.426065 13041 consensus_queue.cc:1059] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 12:19:46.457070 13078 consensus_queue.cc:1059] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:46.457156 12639 raft_consensus.cc:3055] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:46.445641 13092 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:46.458691 12639 raft_consensus.cc:740] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:46.459067 13092 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:46.459582 12639 consensus_queue.cc:260] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:46.460799 12639 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:46.461220 13092 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:46.461279 13015 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:46.461980 13015 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:46.464476 12866 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:46.464895 13015 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:46.415068 13079 consensus_queue.cc:1048] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:19:46.436259 13081 raft_consensus.cc:697] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:46.478610 13050 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:46.478747 13081 consensus_queue.cc:237] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:46.487316  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:46.427290 12935 consensus_queue.cc:260] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:46.489429  6655 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:19:46.478648  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:46.490365  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:46.491197  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:46.491597  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:46.487335  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:19:46.492923  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:46.495409  6622 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:46.496906  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:46.498152  6506 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:46.500602  6509 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 12:19:46.501585  6506 raft_consensus.cc:1275] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:19:46.503253  6505 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:19:46.489178 12935 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:46.516063  6652 raft_consensus.cc:1240] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:46.517112  6622 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:46.519690  6474 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:46.524832  6500 raft_consensus.cc:1275] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:19:46.521829  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:46.526031  6506 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:19:46.526912  6655 raft_consensus.cc:1275] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:19:46.529063  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:46.530300  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:46.488898 12492 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:46.534571  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:46.535437  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 103173us
I20260709 12:19:46.490842 13091 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:46.440153 13107 consensus_queue.cc:1059] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.090s
I20260709 12:19:46.538125  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:46.533319  6474 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:46.541879  6506 raft_consensus.cc:1240] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18. Ops: []
W20260709 12:19:46.544579  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 108735us
I20260709 12:19:46.544783  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:46.545853  6506 raft_consensus.cc:2393] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 17: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:46.529603 13087 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:46.530814 13065 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:46.549093  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:46.554554  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:46.555217 13088 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:46.524413 13058 consensus_queue.cc:1059] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 12:19:46.556982  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:46.562585 12492 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:46.563450  6581 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:19:46.567047 13054 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:46.546025 13033 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:46.533547 12992 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:46.533528 13014 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:46.573052  6577 raft_consensus.cc:3055] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:46.573618  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:46.535104 13125 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:19:46.576079  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:46.577091  6579 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:19:46.608286  6579 raft_consensus.cc:606] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 LEADER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:46.589035  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:46.609477  6500 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:46.591737  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:46.592375  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:46.610352  6583 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:19:46.594604 13129 consensus_queue.cc:1059] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 12:19:46.546761 13075 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:46.555878 13104 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:46.599900  6550 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:46.603875  6505 raft_consensus.cc:1240] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:46.605677  6577 raft_consensus.cc:740] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:46.606091  6651 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:19:46.551154 13087 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:46.537966 13030 consensus_queue.cc:1059] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 12:19:46.630841  6622 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:46.605645  6623 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:46.596607 13033 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:46.631192  6550 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:46.585160 12511 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:19:46.632048 13087 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:46.571209 12878 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:46.634181  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:46.635005 12887 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:46.637763  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:46.638319  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:46.638563  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:46.642755  6475 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:46.646111  6500 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:19:46.646992  6654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 1008) took 1878 ms. Trace:
I20260709 12:19:46.661528  6654 rpcz_store.cc:276] 0709 12:19:44.768482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.977450 (+208968us) service_pool.cc:224] Handling call
0709 12:19:44.977904 (+   454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:45.612985 (+635081us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:46.646770 (+1033785us) spinlock_profiling.cc:243] Waited 634 ms on lock 0x7b540008c5d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:46.646938 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1459023616}
I20260709 12:19:46.662410  6657 raft_consensus.cc:2410] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 18 because replica has last-logged OpId of term: 17 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 7.
I20260709 12:19:46.617031 13125 raft_consensus.cc:697] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:46.665443  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:46.639222  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:46.628988  6577 consensus_queue.cc:260] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:46.666775  6577 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:46.662706  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:46.598539 13127 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 12:19:46.655810 12927 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:46.603601 13092 consensus_queue.cc:1048] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:19:46.671288  6548 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:46.651451 12895 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:46.654119  6550 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:46.681468  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:46.684201  6579 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 17. Current term is 18. Ops: [16.10-17.11]
I20260709 12:19:46.684599  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:46.685003  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:46.694928  6575 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:46.695907  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:46.613323 13014 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:46.696969  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:46.657338 13130 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:46.686259  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:46.698007  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:46.706066  6575 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 12.
I20260709 12:19:46.685858  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:46.701548  6505 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:58944
I20260709 12:19:46.701968  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:46.707918  6505 raft_consensus.cc:606] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:46.708660  6505 raft_consensus.cc:614] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:46.709007  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:46.665459 12757 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:46.715703  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:46.736660  6578 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:19:46.647455 13063 consensus_queue.cc:1048] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 12:19:46.716665  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:46.722545  6577 raft_consensus.cc:1275] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
W20260709 12:19:46.737784  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 132219us
I20260709 12:19:46.740293  6577 pending_rounds.cc:85] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 9
I20260709 12:19:46.740873  6577 pending_rounds.cc:107] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:19:46.741272  6577 raft_consensus.cc:2889] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:46.698138 12895 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:46.659950 13100 consensus_queue.cc:1059] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 12:19:46.746785  6623 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:46.697356 12492 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:46.663836 12912 consensus_queue.cc:1059] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 12:19:46.755718  6575 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:19:46.698603 13079 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:46.738569 12914 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:46.670056 13007 consensus_queue.cc:1059] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.074s
W20260709 12:19:46.760459  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 106433us
I20260709 12:19:46.760982  6623 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:46.763695  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:46.757091 13129 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 12:19:46.744417 12887 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:46.756441 12895 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:46.773905  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:46.792183  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:46.757584  6577 raft_consensus.cc:1275] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 12:19:46.793352  6577 pending_rounds.cc:85] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 10
I20260709 12:19:46.793705  6577 pending_rounds.cc:107] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 12:19:46.793990  6577 raft_consensus.cc:2889] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:46.769454 12492 raft_consensus.cc:697] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:46.767341 12903 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:46.760605 12757 raft_consensus.cc:697] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:19:46.787382  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 155645us
I20260709 12:19:46.802781  6579 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:19:46.804921  6579 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:46.805145  6502 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 18. Current term is 19. Ops: [17.10-18.11]
I20260709 12:19:46.806704  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:46.807489  6502 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:19:46.811439  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 50538.6us
I20260709 12:19:46.813115  6507 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:19:46.813489  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:46.813357  6623 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:46.820159  6584 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:19:46.820289  6656 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:46.790213 13125 consensus_queue.cc:237] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:46.786564  6550 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:46.793169  6578 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
W20260709 12:19:46.814743 13077 consensus_meta.cc:287] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.581s	user 0.010s	sys 0.003s
I20260709 12:19:46.773818 13079 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:46.829192  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:46.840875 13140 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:46.842221 13127 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:46.842722 13127 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:46.843825 12887 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:19:46.845801  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 59298.6us
I20260709 12:19:46.822156  6577 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:19:46.847597  6577 raft_consensus.cc:606] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 LEADER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:46.850370 13127 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:46.852169  6580 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:19:46.856055  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:46.857127  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:46.858286  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:46.859316  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:46.860545  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:46.860672  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:46.861420  6575 raft_consensus.cc:2393] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 13: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:46.863010  6550 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:46.866242 12639 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:46.872591  6550 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:46.876281 12639 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:46.815794 13054 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:46.891551  6622 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.5.58.62:36381, user_credentials={real_user=slave}} blocked reactor thread for 63762.3us
I20260709 12:19:46.817234 13092 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:19:46.794858 13130 consensus_queue.cc:1048] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 12:19:46.823081 12902 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:46.928821  6550 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:46.939353  6502 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:19:46.916318  7238 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:46.920022  6549 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:46.956692  6549 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:46.829278 13104 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:46.985998 13007 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:19:46.987671 13007 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:46.988176 13007 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:46.988765 13007 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:46.985021  6549 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:46.824627 12511 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.084s
I20260709 12:19:47.005080  6578 raft_consensus.cc:1275] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 12:19:47.038041  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 53135us
W20260709 12:19:47.043955  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 127791us
W20260709 12:19:47.049204  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:47.045333  7238 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:46.828294 13077 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:47.007158 12887 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:46.958977 12902 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:46.956189 12492 consensus_queue.cc:237] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:47.043591 12904 consensus_queue.cc:1059] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.120s
I20260709 12:19:47.097290 12927 consensus_queue.cc:1048] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.080s
W20260709 12:19:47.196445  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 267718us
W20260709 12:19:47.201803  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 114407us
I20260709 12:19:47.174909 12639 raft_consensus.cc:993] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e: : Instructing follower f2b06aebe97249a4877d798b8118ece4 to start an election
W20260709 12:19:47.213155 13141 log.cc:927] Time spent T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.384s	user 0.000s	sys 0.000s
I20260709 12:19:47.212721  6550 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:47.214643 13144 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:19:47.217031 13137 log.cc:927] Time spent T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.212s	user 0.000s	sys 0.000s
I20260709 12:19:47.218688 12912 raft_consensus.cc:3055] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:47.219141 12912 raft_consensus.cc:740] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:47.220114 12912 consensus_queue.cc:260] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:47.221333 12912 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:47.223690 13054 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:47.224253 13054 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:47.226557 13054 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:47.227576 13135 log.cc:927] Time spent T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.427s	user 0.001s	sys 0.000s
I20260709 12:19:47.155726 13014 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:47.006585 12757 consensus_queue.cc:237] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:47.214569 13142 log.cc:927] Time spent T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 12:19:47.280380  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:47.294402  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:47.274442 13087 consensus_meta.cc:287] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.610s	user 0.006s	sys 0.009s
I20260709 12:19:47.290740 12639 raft_consensus.cc:1081] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 LEADER]: Signalling peer f2b06aebe97249a4877d798b8118ece4 to start an election
W20260709 12:19:47.368207  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 410169us
I20260709 12:19:47.371063  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:47.337213 13136 log.cc:927] Time spent T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.539s	user 0.000s	sys 0.002s
W20260709 12:19:47.351815 13152 log.cc:927] Time spent T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.150s	user 0.001s	sys 0.001s
I20260709 12:19:47.374075  6576 raft_consensus.cc:1275] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:19:47.332247 13100 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:47.342429 13014 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:47.384865  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:47.385581  6649 raft_consensus.cc:2376] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20. Re-sending same reply.
I20260709 12:19:47.363006 13157 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:47.335886 13087 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:19:47.391502  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:47.343012 13117 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.022s	sys 0.021s
W20260709 12:19:47.358480 12935 consensus_meta.cc:287] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.828s	user 0.010s	sys 0.002s
W20260709 12:19:47.373312 13091 consensus_meta.cc:287] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.832s	user 0.012s	sys 0.000s
W20260709 12:19:47.396780  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 105466us
I20260709 12:19:47.405285  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:47.385251 13079 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:47.405962  6649 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:19:47.402029  6654 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
W20260709 12:19:47.396373 13117 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.022s	sys 0.021s
I20260709 12:19:47.409722  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:47.407150 13147 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:47.389339 13014 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:47.424437  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:47.412945 13077 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:47.409595 13091 consensus_queue.cc:1059] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:19:47.452095  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:47.435597 12902 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:47.429304 13100 consensus_queue.cc:1059] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
W20260709 12:19:47.469630  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:47.469579  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 70023.5us
I20260709 12:19:47.470925  6622 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:47.474721  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:47.475719  6583 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:47.479059  6550 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:47.482975  6583 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:47.483314  6550 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:47.485148 13151 log.cc:927] Time spent T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:19:47.486795 13156 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:47.485199 13041 raft_consensus.cc:3055] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:47.487622 13041 raft_consensus.cc:740] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:47.488888 13041 consensus_queue.cc:260] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:47.487583 13156 raft_consensus.cc:697] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:47.436100 13147 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:47.491151 13156 consensus_queue.cc:237] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:47.493844 13147 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:47.509793 13014 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:47.511626 13079 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:47.467319 13117 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:19:47.521309 13154 log.cc:927] Time spent T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
W20260709 12:19:47.538906 13007 consensus_meta.cc:287] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.550s	user 0.006s	sys 0.004s
I20260709 12:19:47.539388 13007 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:47.540477  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:19:47.545315 12895 consensus_meta.cc:287] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.732s	user 0.012s	sys 0.000s
I20260709 12:19:47.545810 12895 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:47.547201  6649 raft_consensus.cc:2368] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 13. Current term is 14.
I20260709 12:19:47.550884  6649 raft_consensus.cc:1240] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:47.521052 13077 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:47.521632 12902 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:47.522799 12904 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:47.479142 13159 consensus_queue.cc:1048] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 12:19:47.537587 13117 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:47.524411 13065 consensus_meta.cc:287] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.949s	user 0.010s	sys 0.003s
I20260709 12:19:47.536903 13087 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:47.589465  6475 leader_election.cc:400] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 13. Current term is 14.
I20260709 12:19:47.617298  6475 leader_election.cc:403] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:47.608809  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 2752) took 1061 ms. Trace:
W20260709 12:19:47.610159  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 126923us
I20260709 12:19:47.617841  6507 rpcz_store.cc:276] 0709 12:19:46.496280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:46.496697 (+   417us) service_pool.cc:224] Handling call
0709 12:19:47.557512 (+1060815us) spinlock_profiling.cc:243] Waited 316 ms on lock 0x7b54000f3054. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:47.557611 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":725758208}
I20260709 12:19:47.614634 12797 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:47.618674  6550 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:47.619992  6658 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 12:19:47.628929  6658 pending_rounds.cc:85] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 9
I20260709 12:19:47.629312  6658 pending_rounds.cc:107] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 12:19:47.629660  6658 raft_consensus.cc:2889] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:47.621920 12757 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:19:47.633931 13115 raft_consensus.cc:3055] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:19:47.634405 13115 raft_consensus.cc:740] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:47.635362 13115 consensus_queue.cc:260] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:47.640537 12757 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 13. Current term is 14.
I20260709 12:19:47.647843 13117 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:47.609206 13077 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:47.615131 12904 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:47.671806 12902 raft_consensus.cc:697] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:47.667472 13115 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:19:47.668265 12963 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:47.674156 12963 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:47.674592 12963 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:19:47.678884  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:47.683192 13007 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:47.691545 12492 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:47.693118 13163 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:47.693706 13163 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:47.695197 13078 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:47.695755 13078 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:47.701936 12653 raft_consensus.cc:3055] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:47.701553  6405 catalog_manager.cc:5697] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 8 to 9, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 9 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:19:47.706189  6655 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 12:19:47.706579  6655 pending_rounds.cc:85] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 9
I20260709 12:19:47.706786  6655 pending_rounds.cc:107] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:19:47.707005  6655 raft_consensus.cc:2889] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:47.707858 13107 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:47.703748  6405 catalog_manager.cc:5697] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 5 to 13. New cstate: current_term: 13 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:47.708371 13107 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:47.709421 12902 consensus_queue.cc:237] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:47.710474 13107 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:47.710959 13107 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:47.711395 13107 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:47.715510  6501 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 2928) took 1745 ms (client timeout 2221 ms). Trace:
I20260709 12:19:47.716071  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:47.715951  6501 rpcz_store.cc:269] 0709 12:19:45.969869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:46.041225 (+ 71356us) service_pool.cc:224] Handling call
0709 12:19:47.715327 (+1674102us) spinlock_profiling.cc:243] Waited 58.1 ms on lock 0x7b5400388454. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:47.715449 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":133657856}
I20260709 12:19:47.719466  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 2754) took 1038 ms. Trace:
I20260709 12:19:47.732364 12963 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:47.737643 12914 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:47.738219 12914 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:47.743052 13161 log.cc:927] Time spent T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.288s	user 0.001s	sys 0.000s
I20260709 12:19:47.745558 12511 raft_consensus.cc:3055] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:47.746016 12511 raft_consensus.cc:740] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:47.747999 12874 raft_consensus.cc:3055] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:47.748483 12874 raft_consensus.cc:740] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:47.749343 12874 consensus_queue.cc:260] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:47.750435 13175 log.cc:927] Time spent T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:19:47.750545 12874 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:47.753114 12904 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:47.653278 12797 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:47.755230  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2661) took 1350 ms. Trace:
I20260709 12:19:47.755638  6656 rpcz_store.cc:276] 0709 12:19:46.404857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:46.405436 (+   579us) service_pool.cc:224] Handling call
0709 12:19:46.405795 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:46.820738 (+414943us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:47.754987 (+934249us) spinlock_profiling.cc:243] Waited 414 ms on lock 0x7b54003e99d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:47.755170 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":952805248}
I20260709 12:19:47.757203  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:47.757867 13171 log.cc:927] Time spent T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:19:47.758490  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:47.657021 13015 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:47.759578 13015 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:47.760924  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3302) took 1316 ms. Trace:
I20260709 12:19:47.761312  6652 rpcz_store.cc:276] 0709 12:19:46.444363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:46.445348 (+   985us) service_pool.cc:224] Handling call
0709 12:19:46.489934 (+ 44586us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:46.527916 (+ 37982us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:47.760722 (+1232806us) spinlock_profiling.cc:243] Waited 39.4 ms on lock 0x7b540044b1d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:47.760867 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":90727680}
I20260709 12:19:47.761866 13015 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:47.762529  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:47.763808  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:47.764400  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:47.765211  6576 raft_consensus.cc:2393] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 21: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:47.766080  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:47.766690  6576 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:47.767494 13077 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:47.768139  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:47.768617  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:47.769816  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:47.770798  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:47.771570 13033 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:47.772416 12895 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:47.702306 12653 raft_consensus.cc:740] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:47.773902 12653 consensus_queue.cc:260] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:47.775209  6658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2667) took 1368 ms. Trace:
I20260709 12:19:47.775626  6658 rpcz_store.cc:276] 0709 12:19:46.406608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:46.406714 (+   106us) service_pool.cc:224] Handling call
0709 12:19:46.406954 (+   240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:47.630295 (+1223341us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:47.774982 (+144687us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54000744d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:47.775148 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2788576512}
I20260709 12:19:47.777669  6658 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:58980
W20260709 12:19:47.778069  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:47.778244  6658 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:47.778689  6658 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:47.781154  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 2758) took 1097 ms. Trace:
I20260709 12:19:47.781550  6580 rpcz_store.cc:276] 0709 12:19:46.684149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:46.684314 (+   165us) service_pool.cc:224] Handling call
0709 12:19:47.780955 (+1096641us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54003abe54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:47.781065 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":361655680}
W20260709 12:19:47.784464  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:47.785094 13041 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:47.785326 13007 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:47.785979 13007 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:47.788713 13009 raft_consensus.cc:3055] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:47.789170 13009 raft_consensus.cc:740] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:47.790076 13009 consensus_queue.cc:260] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:47.791378 13009 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:47.692114 12492 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:47.793501  6654 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58980 (request call id 3319) took 1199 ms (client timeout 1539 ms). Trace:
W20260709 12:19:47.793942  6654 rpcz_store.cc:269] 0709 12:19:46.593721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:46.662455 (+ 68734us) service_pool.cc:224] Handling call
0709 12:19:47.793284 (+1130829us) spinlock_profiling.cc:243] Waited 733 ms on lock 0x7b540044a2d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:47.793408 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1685326080}
I20260709 12:19:47.794435 13078 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:47.795466  6654 raft_consensus.cc:1240] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:47.755561 12797 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:47.798236  6473 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:47.719887  6581 rpcz_store.cc:276] 0709 12:19:46.680882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:46.681189 (+   307us) service_pool.cc:224] Handling call
0709 12:19:47.719251 (+1038062us) spinlock_profiling.cc:243] Waited 917 ms on lock 0x7b54002661d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:47.719405 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2108357504}
I20260709 12:19:47.800700 13163 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:47.801308 12511 consensus_queue.cc:260] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:47.802522 12511 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:47.803154 12757 consensus_queue.cc:1059] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:47.803970 12757 consensus_queue.cc:1243] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer f2b06aebe97249a4877d798b8118ece4 log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:19:47.797207  6657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 2668) took 1390 ms. Trace:
I20260709 12:19:47.804486 12757 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:47.804983  6657 rpcz_store.cc:276] 0709 12:19:46.406895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:46.407000 (+   105us) service_pool.cc:224] Handling call
0709 12:19:47.797023 (+1390023us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b54002283d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:47.797151 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":584896768}
I20260709 12:19:47.764408  6583 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:47.806427 12757 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:47.808254 13014 raft_consensus.cc:3055] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:47.808682 13014 raft_consensus.cc:740] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:47.775204 12653 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:47.809456 13014 consensus_queue.cc:260] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:47.810673 13014 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:47.810986  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:47.812498 13065 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:47.813033 13065 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:47.813344  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 159661us
I20260709 12:19:47.815172 13065 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:47.817293 13015 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:47.817997 13015 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:47.817389 12914 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:47.820937 13107 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:47.822547 13007 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:47.822978  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:47.823230  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:47.824084  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:47.824687  6651 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:19:47.826686  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:53972 (request call id 38) took 1018 ms. Trace:
I20260709 12:19:47.827003  6473 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:47.827186  6577 rpcz_store.cc:276] 0709 12:19:46.808471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:46.822020 (+ 13549us) service_pool.cc:224] Handling call
0709 12:19:47.826278 (+1004258us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b54002a9f20. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e06520 00007f2875e83652 00007f287ae5160a 00007f28743d77d4 00007f28743d7762 00007f28743d7691 00007f28743d760c 00007f28743d68c2 00007f28741c448c 00007f28741c38c6
0709 12:19:47.826589 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":447840000}
I20260709 12:19:47.827951 13075 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:47.828477 13075 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:47.828927 13075 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:47.831596  6473 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:47.833105  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:47.834659  6652 raft_consensus.cc:1240] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:47.835484  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:47.836722  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:47.837864  6583 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:47.842299  6507 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 12:19:47.846490  6507 pending_rounds.cc:85] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 10
I20260709 12:19:47.846704  6507 pending_rounds.cc:107] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 12:19:47.846917  6507 raft_consensus.cc:2889] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:47.847107  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:47.843072  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:47.850063  6500 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 18. Current term is 19. Ops: []
W20260709 12:19:47.850467  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:47.845366  6506 raft_consensus.cc:1275] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:19:47.846002  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:47.853730  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:47.845628  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2680) took 1026 ms. Trace:
I20260709 12:19:47.833122  6473 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:47.857214  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:47.857800  6505 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:47.855638  6655 rpcz_store.cc:276] 0709 12:19:46.819174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:47.771617 (+952443us) service_pool.cc:224] Handling call
0709 12:19:47.771875 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:47.772056 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:47.772085 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:47.772135 (+    50us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:47.772173 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:47.772228 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:47.845573 (+ 73345us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:19:47.854386  6654 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:47.841112 12914 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:47.865638 12914 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:47.866026 12914 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:47.869473  6509 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 18. Current term is 19. Ops: [17.10-18.11]
I20260709 12:19:47.870411 13014 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:47.871078 13014 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:47.839829 13013 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:47.844697  6658 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:47.878219  6651 raft_consensus.cc:1240] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:47.888830  6475 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:47.889539  6473 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:47.890902  6474 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:47.909567  6658 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:47.918941  6505 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:19:47.920367  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:47.956249  6658 raft_consensus.cc:1275] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:19:47.957336  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:47.902948 13015 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:47.974017  6658 raft_consensus.cc:2393] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 19: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:19:47.974469  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 149251us
W20260709 12:19:47.979301  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 90546.6us
I20260709 12:19:47.927259 13104 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:47.927438 13077 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:48.001291  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:47.926801 12995 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:47.955753 13127 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:47.977509  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:48.010125  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 120703us
I20260709 12:19:48.017409  6473 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
W20260709 12:19:48.022576  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:48.024750  6473 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:48.026501  6473 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:47.979467 13015 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:48.029613  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 138821us
I20260709 12:19:48.001796 12757 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:47.998730 13007 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:48.045238  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:47.978024 13055 consensus_queue.cc:1048] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 12:19:48.053463  6658 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:48.027899 13013 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:19:48.063139  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:48.030913 12904 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:48.053853 12757 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:48.024982 12995 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:48.076925  7267 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
W20260709 12:19:48.078958 13134 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 1.273s	user 0.007s	sys 0.000s
I20260709 12:19:48.093266  6550 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
W20260709 12:19:48.142942  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 49761.2us
I20260709 12:19:48.100104  6575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 961) took 1241 ms. Trace:
I20260709 12:19:48.110780 12878 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:48.117005 13180 log.cc:927] Time spent T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:19:48.118716  6577 raft_consensus.cc:1240] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:48.143936  6575 rpcz_store.cc:276] 0709 12:19:46.858521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:46.861975 (+  3454us) service_pool.cc:224] Handling call
0709 12:19:46.862219 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:46.911893 (+ 49674us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:46.912030 (+   137us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:46.912227 (+   197us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:19:46.912381 (+   154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:48.099846 (+1187465us) spinlock_profiling.cc:243] Waited 48.8 ms on lock 0x7b54000582d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:48.099894 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:48.100036 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":112314240}
I20260709 12:19:48.142329  6504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 802) took 1567 ms. Trace:
I20260709 12:19:48.149582  6504 rpcz_store.cc:276] 0709 12:19:46.585579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:46.585752 (+   173us) service_pool.cc:224] Handling call
0709 12:19:46.591660 (+  5908us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:47.799867 (+1208207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:47.799901 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:47.799946 (+    45us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:19:47.799981 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:48.142082 (+342101us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b5400434c54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:48.142132 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:48.142271 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2778019072}
W20260709 12:19:48.151542  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:48.025480 13127 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:48.024642 13104 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:48.146167  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:19:48.155550 13013 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:48.163048  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:48.139474 12757 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:48.136543 13015 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:19:48.190325  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 60153.6us
W20260709 12:19:48.196192  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 151581us
I20260709 12:19:48.060122 13007 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:48.192595  6656 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:19:48.193420  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:48.205042  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:48.205155  6578 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:48.191990  6622 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:48.191033  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 148067us
I20260709 12:19:48.208118  6473 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:48.212136  6657 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:19:48.162889 13014 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:48.222267  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 968) took 1361 ms. Trace:
I20260709 12:19:48.232717 12492 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:48.207468 13015 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:19:48.236454  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:48.198587 13075 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:48.213068 12963 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:48.141808 12904 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:48.280493  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 60589.7us
I20260709 12:19:48.162711 12965 consensus_queue.cc:1059] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 19.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
I20260709 12:19:48.246605 12881 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:48.258059 13186 log.cc:927] Time spent T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:19:48.217149 13063 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:19:48.265816  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:48.242472 13162 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 12:19:48.188521 13160 consensus_queue.cc:1059] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 12:19:48.189282 13177 consensus_queue.cc:1048] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 12:19:48.161164 13193 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:19:48.267620  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 59385.1us
I20260709 12:19:48.270200  6578 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:48.275660  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2751) took 1781 ms. Trace:
I20260709 12:19:48.206915 13191 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.090s
I20260709 12:19:48.308772  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:48.309532  6655 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:48.310539  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:48.307629  6622 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:48.312839  6651 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 12:19:48.313242  6651 pending_rounds.cc:85] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 10
I20260709 12:19:48.313478  6651 pending_rounds.cc:107] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 12:19:48.313756  6651 raft_consensus.cc:2889] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:48.314126  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:48.314682  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:48.315671  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:48.316881  6655 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:19:48.318004  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:48.318405  6550 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:48.319008  6655 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:48.322134  7268 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:19:48.323138  6656 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:19:48.294720 13197 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:48.283079 13192 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 12:19:48.296452 12963 raft_consensus.cc:3055] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:48.297231  6508 raft_consensus.cc:1240] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:48.276605 13015 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:19:48.278986 13182 log.cc:927] Time spent T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:19:48.302131  6503 rpcz_store.cc:276] 0709 12:19:46.494291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:46.495459 (+  1168us) service_pool.cc:224] Handling call
0709 12:19:46.495769 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:46.523891 (+ 28122us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:46.523919 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:46.529669 (+  5750us) log.cc:844] Serialized 26 byte log entry
0709 12:19:47.519026 (+989357us) spinlock_profiling.cc:243] Waited 893 us on lock 0x7b18002c9e60. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:19:47.925986 (+406960us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:47.926025 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:47.926093 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:48.266555 (+340462us) raft_consensus.cc:1685] finished
0709 12:19:48.266578 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:48.275581 (+  9003us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":103,"raft.queue_time_us":4222,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":4130,"spinlock_wait_cycles":2054528,"thread_start_us":405579,"threads_started":1,"wal-append.queue_time_us":1470177}
I20260709 12:19:48.162897 13179 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:48.277463 13014 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:48.303743 12965 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:19:48.344897 13188 log.cc:927] Time spent T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:19:48.347121  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:48.348677 12995 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:48.348879  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:48.349375 12995 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:48.349823 12995 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:48.350178 12995 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:48.327818 13144 consensus_queue.cc:1059] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 12:19:48.351195 13144 raft_consensus.cc:3055] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:48.351550 13144 raft_consensus.cc:740] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:48.352356 13144 consensus_queue.cc:260] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:48.352993  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:48.353392 13075 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:48.354360 13181 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:48.356173 13181 consensus_queue.cc:1048] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:48.356536 12511 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:48.333623 13078 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:48.358477 13078 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:48.358574  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:48.358949 13078 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:48.359282  6505 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 19.
I20260709 12:19:48.330030 13197 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:48.355700 12965 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:48.292761 13063 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:48.331727  7268 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:19:48.360795 13007 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:48.156627 13081 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:48.326529  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:48.380821  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:48.366063  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:48.382625  6504 raft_consensus.cc:2368] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 20. Current term is 21.
I20260709 12:19:48.369623  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:48.371800  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:48.383524  6509 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:48.362751 13197 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:48.378827  6507 raft_consensus.cc:1240] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:48.384426  6502 raft_consensus.cc:1240] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:48.377132  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:48.385728  6622 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:48.327939 12963 raft_consensus.cc:740] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:48.396808  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:48.397152  6501 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 12:19:48.417649  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:48.426728  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:48.427625  6507 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:48.418660  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:48.423974  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:48.381165 13063 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:48.430184  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:48.428609  6579 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
W20260709 12:19:48.425067  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:48.431188  6622 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:48.434509  6505 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:19:48.435654  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:48.443562 12963 consensus_queue.cc:260] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.8, Last appended by leader: 8, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:19:48.491561 12963 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:48.498134  6474 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:48.499660 12935 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:48.449137  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:48.500073 12935 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:48.450358  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:48.500492 12935 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:48.452982 13191 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:48.454211 12511 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:48.504972  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:48.505613  6505 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:19:48.507081  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:48.454350 12965 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:48.457305 12904 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:48.404165 12850 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:48.403870 13197 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:48.459615 13041 consensus_meta.cc:287] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.674s	user 0.007s	sys 0.001s
I20260709 12:19:48.459733  6504 raft_consensus.cc:1275] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:19:48.225711  6584 rpcz_store.cc:276] 0709 12:19:46.883831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:46.884056 (+   225us) service_pool.cc:224] Handling call
0709 12:19:46.884434 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:46.884668 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:19:46.934476 (+ 49808us) log.cc:844] Serialized 14 byte log entry
0709 12:19:48.215068 (+1280592us) spinlock_profiling.cc:243] Waited 5.57 ms on lock 0x7b180047aa60. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2875a314bc 00007f2872d7f48b 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a2923a
0709 12:19:48.221906 (+  6838us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:48.221964 (+    58us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:19:48.222015 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:48.222091 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:48.222208 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6577,"prepare.queue_time_us":302105,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":9333,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":119,"spinlock_wait_cycles":12811520,"thread_start_us":432142,"threads_started":2,"wal-append.queue_time_us":520139}
W20260709 12:19:48.514212  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 300126us
I20260709 12:19:48.398783 13081 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:48.385099 13179 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:48.472445  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2676) took 1654 ms. Trace:
I20260709 12:19:48.641528  6649 rpcz_store.cc:276] 0709 12:19:46.817965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:47.551685 (+733720us) service_pool.cc:224] Handling call
0709 12:19:47.552016 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:47.585837 (+ 33821us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:47.585886 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:47.586519 (+   633us) log.cc:844] Serialized 26 byte log entry
0709 12:19:47.774261 (+187742us) spinlock_profiling.cc:243] Waited 3.22 ms on lock 0x7b18002e4b00. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:19:47.804166 (+ 29905us) spinlock_profiling.cc:243] Waited 32.8 ms on lock 0x7b18002e4b00. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d16c0a 00007f287c456219 00007f2879d5d58f 00007f2872d9c571 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd
0709 12:19:48.001902 (+197736us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:48.001952 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:48.002024 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:48.472211 (+470187us) raft_consensus.cc:1685] finished
0709 12:19:48.472226 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:48.472382 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5395,"raft.queue_time_us":32502,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":20229,"spinlock_wait_cycles":82829056,"thread_start_us":227429,"threads_started":1,"wal-append.queue_time_us":448454}
I20260709 12:19:48.437464 12992 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:48.487501 12881 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:48.558094 13104 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:48.505887  6509 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:48.659989  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:48.661134  6509 raft_consensus.cc:2393] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 18: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:48.510087  6505 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:19:48.433293  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:48.402459 13178 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:48.519709  6473 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:48.690649  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 171002us
I20260709 12:19:48.524938 12511 raft_consensus.cc:697] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:48.721145 12511 consensus_queue.cc:237] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:48.633821 13007 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:19:48.644560 12914 consensus_meta.cc:287] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.770s	user 0.015s	sys 0.000s
I20260709 12:19:48.726071 13007 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:48.726253 12914 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:48.524744 12850 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:48.512584 13191 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:48.582405  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:19:48.607205  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:48.513048 12492 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:48.512612 12797 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:48.561087  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:48.623586  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:48.754778  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:48.518949 13055 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:48.523317  6501 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
W20260709 12:19:48.632877  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:48.647475  6507 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:19:48.758580  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:19:48.678494  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:48.693001  6473 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:48.703920 13193 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:48.708425 13178 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:48.710055  6583 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:19:48.711434 13081 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:48.637550 13200 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:48.713835 13015 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:48.715230 13078 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:48.637861 12653 consensus_meta.cc:287] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.828s	user 0.013s	sys 0.005s
I20260709 12:19:48.716135 12757 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:48.716996 13063 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:48.802609  6581 raft_consensus.cc:1275] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:19:48.718515 13179 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:48.725170 13075 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:48.536235 13197 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:48.731539  6506 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:48.732271  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:48.739943  6548 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:19:48.806092  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 66273.8us
I20260709 12:19:48.807649  6548 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:48.748046  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:48.756806  6475 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:48.809942  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 53205.3us
I20260709 12:19:48.762688  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:48.811436  6649 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
W20260709 12:19:48.763171  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 70834.9us
I20260709 12:19:48.716176 12965 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:48.728242 13007 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:48.764632  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 336930us
I20260709 12:19:48.773247  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:48.776557  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:48.779026  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:48.754027 12992 raft_consensus.cc:697] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:48.760637 12878 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:48.762650 13077 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:48.796492  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 365373us
I20260709 12:19:48.765250 13013 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:48.762539 13055 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:48.857818  6507 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 22: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:48.844861  6575 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:19:48.839332  6503 raft_consensus.cc:2393] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 16: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:48.781669 13081 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 12:19:48.760525 12492 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:48.860508 12992 consensus_queue.cc:237] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.8, Last appended by leader: 8, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:19:48.843415 13199 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:48.848080 12965 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:48.851269  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:48.762526 12797 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:48.853683  6622 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:48.855373  6657 raft_consensus.cc:1240] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:48.838765 13129 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:48.855724  6548 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:48.839690  6500 raft_consensus.cc:1240] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:48.868803  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:48.805377 12914 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:48.844486 13107 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:48.843709 12963 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:48.845866 13041 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:19:48.873169  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:48.876204  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:48.872568  6548 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:48.882252  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:48.881453  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:48.783191 13200 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:48.898294  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:48.899154  6649 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:19:48.858628 13127 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:48.908139  6622 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:48.878595 13007 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:48.855942 12878 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:48.857666 13077 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:48.926309  6622 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:48.930308 12878 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:48.930889 13202 log.cc:927] Time spent T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
W20260709 12:19:48.931862 13201 log.cc:927] Time spent T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:19:48.896394 12887 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 12:19:48.912182 13149 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:48.894245 13013 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:19:48.922741 13107 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:19:48.955077  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 172233us
I20260709 12:19:48.919281 13181 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:48.919209 13041 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:48.923494 13199 raft_consensus.cc:697] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:48.941376 13205 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:48.902618 12963 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:48.956022  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 200149us
I20260709 12:19:48.981787  6550 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:48.989295  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:48.961190  7238 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:48.973542 12797 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:48.953739 13197 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:48.973848 13078 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:48.975257 13207 log.cc:927] Time spent T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:19:48.995927 13064 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:48.996549 13064 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:48.997017 13064 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:49.000929  7238 leader_election.cc:400] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 20. Current term is 21.
I20260709 12:19:49.001410  7238 leader_election.cc:403] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:48.978461  6502 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:19:48.978853  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:48.900619 13204 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.057s
I20260709 12:19:49.003381  6502 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:19:48.951910 13140 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:49.006562  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 48338.1us
W20260709 12:19:48.980130  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 126076us
I20260709 12:19:49.019976  6548 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:49.021903  6548 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:48.960927 13181 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:49.017704  6550 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:49.025120  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 212395us
W20260709 12:19:49.009210 12995 consensus_meta.cc:287] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.659s	user 0.010s	sys 0.001s
I20260709 12:19:49.028110  6473 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:48.981633 12965 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:48.983428 13107 raft_consensus.cc:697] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:49.014142 13078 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:49.032490  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:49.030917  6473 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:49.014952 13213 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 20. Current term is 21.
I20260709 12:19:49.013026 12878 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:49.050770  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:49.016541 13191 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:49.023744 13007 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:48.956192 13149 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:49.055179  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:49.065930  6502 raft_consensus.cc:2393] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 18: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:49.066944  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:49.068704 12903 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:49.069120  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:49.069562 13063 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:49.069665 13009 raft_consensus.cc:3055] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:49.070160 13009 raft_consensus.cc:740] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:49.070330 13063 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:49.070775 13063 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:49.071336 12904 consensus_queue.cc:1059] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:19:49.071122 13009 consensus_queue.cc:260] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:49.071857 12511 consensus_queue.cc:1048] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:19:49.072584 12887 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:49.072546 13206 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:19:49.073387 12511 raft_consensus.cc:3055] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:49.073290 12887 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:49.073817 12511 raft_consensus.cc:740] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:49.074185 13055 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:49.074327 12492 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:49.074963 12492 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:19:49.075464 12492 raft_consensus.cc:697] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:49.075506  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:49.075917 12757 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:49.076715 13140 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:49.077289 13140 raft_consensus.cc:697] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 19 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:49.078318 13140 consensus_queue.cc:237] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:49.024101 12850 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:49.062940 13007 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:49.029311 13065 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:49.031208 13181 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:49.007382 13041 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:49.063912 13211 log.cc:927] Time spent T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:19:49.068128  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:49.081350  7268 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:19:49.047336 13204 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:49.047580 12992 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:49.023608 12653 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 12:19:49.048220 13015 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:49.051177 12965 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:49.047137 13178 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:49.071151 13063 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:49.096077 13007 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:49.075484 12887 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:49.091610 13009 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:49.126524  6503 raft_consensus.cc:1240] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 18. Current term is 19. Ops: []
W20260709 12:19:49.126715  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 104884us
I20260709 12:19:49.061797 12878 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:49.099231 12903 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:49.098080 12797 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:19:49.120765 13219 log.cc:927] Time spent T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 12:19:49.034492 12995 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:49.054560 13078 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:49.110402 13198 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:49.108759 13200 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:49.074645 12511 consensus_queue.cc:260] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:49.113410 13215 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:49.110780 13212 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:49.115106 13222 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:49.124725 13220 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:49.184752 12887 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:49.185559 12653 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:49.186131 12653 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:49.186587 12995 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:49.096952 13199 consensus_queue.cc:237] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:49.174525 12963 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:49.172495  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:49.112632 12992 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:49.149456 12903 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:49.166006 13214 log.cc:927] Time spent T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
W20260709 12:19:49.196494  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:49.170750 13077 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:49.218557  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:49.110234 13204 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:49.221395  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:49.222306  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:49.224795  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:19:49.223105  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 50853.8us
I20260709 12:19:49.151216 12797 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:49.223235  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:49.196712 13149 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:49.232442  6509 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:19:49.208760 12511 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:49.230788  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 200005us
I20260709 12:19:49.126261 12492 consensus_queue.cc:237] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:49.230432 13138 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:49.127311 13107 consensus_queue.cc:237] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:49.267614  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:49.193917 13212 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:49.227888  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 2650) took 1073 ms. Trace:
W20260709 12:19:49.276996 13223 log.cc:927] Time spent T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:19:49.314456  6509 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:49.236584 12653 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:49.316845  6503 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:19:49.278472  6473 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:49.197340 13222 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:49.319463 12935 consensus_meta.cc:287] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.815s	user 0.010s	sys 0.002s
I20260709 12:19:49.310717 13138 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:49.181126 13220 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:19:49.324159  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 165255us
I20260709 12:19:49.323879  6581 rpcz_store.cc:276] 0709 12:19:48.154792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:48.478414 (+323622us) service_pool.cc:224] Handling call
0709 12:19:48.478704 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:48.803059 (+324355us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:49.227595 (+424536us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b540034d5d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:49.227817 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":744015232}
W20260709 12:19:49.344240  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 65873.4us
I20260709 12:19:49.347834  6473 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:49.360637 13221 log.cc:927] Time spent T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
W20260709 12:19:49.364430 13217 log.cc:927] Time spent T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 12:19:49.366381 13204 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:49.368042 13212 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:49.371659 13226 log.cc:927] Time spent T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:19:49.371865 13222 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:49.373523 12903 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:49.374701 13078 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:49.349349 13041 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:49.311756 13064 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:49.386731  6473 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:49.390928  6473 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:49.354635 13138 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:49.350287 13077 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:49.404757  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:49.405604  6577 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 17.
I20260709 12:19:49.406109  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:49.406977  6583 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
I20260709 12:19:49.407013  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:49.407342  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:49.408684  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:49.409495  7274 raft_consensus.cc:2410] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 12:19:49.412111  6473 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:49.412461  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:49.413141  6652 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
I20260709 12:19:49.415771  7267 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:49.421618  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:49.422290  6577 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:49.421691  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:49.389392 13222 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:49.399241 13013 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:49.351300 12935 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:49.399513 13216 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:49.427205  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:49.428139  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:49.428679  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:49.428860  6577 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:49.429442  6582 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:49.411159 13014 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:49.402526 13078 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:49.437577  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:49.435264  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 417608us
I20260709 12:19:49.423213 13200 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:49.433955  6581 raft_consensus.cc:2393] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 17: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:49.438652  6550 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:49.442744  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:49.443360  6582 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:49.443750 13064 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:49.444032 12935 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:49.430425 13013 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:49.455943  6582 raft_consensus.cc:2410] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 12:19:49.461513 13181 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:49.465049  7267 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:19:49.466702 13107 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:49.445675 13078 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:49.469485  6550 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:49.469679  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:49.480189  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:49.472294  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:49.472743  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:49.481101  6582 raft_consensus.cc:2410] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 12:19:49.481611  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:49.481877  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:49.482432  6582 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:19:49.482750  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:49.474080  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:49.484562  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:49.475836  6579 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 21. Current term is 22. Ops: [21.11-21.11]
I20260709 12:19:49.488989 13240 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:49.490366  6577 raft_consensus.cc:1275] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:49.493656 13144 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:49.497318 12797 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:49.497932 13181 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:49.498030 13078 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:49.468541 12992 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:49.500612 13241 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:49.501264 13241 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:49.506567 13238 log.cc:927] Time spent T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:19:49.509342  7240 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:49.511768  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 54746.4us
I20260709 12:19:49.512426  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:49.512573  6500 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 814) took 1145 ms. Trace:
I20260709 12:19:49.513419  6500 rpcz_store.cc:276] 0709 12:19:48.366675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:48.366805 (+   130us) service_pool.cc:224] Handling call
0709 12:19:48.378836 (+ 12031us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:48.871660 (+492824us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:49.512302 (+640642us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b540017c5d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:49.512517 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1059372288}
I20260709 12:19:49.471041 13008 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:49.517141 13242 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:49.516906 13008 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:49.517632 13242 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:49.518079 13242 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:49.519526 13008 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:49.521078 13241 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:49.478685 13200 raft_consensus.cc:697] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:19:49.522004  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:49.528134  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:49.524240  6576 raft_consensus.cc:1240] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:49.543939  6657 raft_consensus.cc:2368] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 19. Current term is 20.
I20260709 12:19:49.545509  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:49.546307  6576 raft_consensus.cc:2393] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 17: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:19:49.546922  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 77513.2us
I20260709 12:19:49.545581  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:49.547734  6550 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:49.553285  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:49.555145  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:49.555969  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:49.556774  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:49.557832  6655 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:19:49.525841  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:49.525583  6582 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:19:49.558702  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:49.560662  6473 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:49.554423  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:49.565181  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:49.565412  6651 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:49.565466  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:49.556656  6654 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:19:49.470772 13216 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:49.563763  7240 leader_election.cc:400] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 19. Current term is 20.
I20260709 12:19:49.566380  6473 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:49.566764  7240 leader_election.cc:403] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:19:49.477128 13013 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:49.435812 13222 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:49.568028  6650 raft_consensus.cc:2393] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 17: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:49.516208 13066 raft_consensus.cc:2804] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:49.557193  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:49.565742  6582 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:49.567909  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:49.571719 13197 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:49.573801  6473 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:49.574273  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:49.575842  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:49.571960 13191 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:49.495486 13007 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:49.562629 13078 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:49.567283 13107 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:49.584578 13063 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:49.576694 13197 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:49.577545  6584 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:19:49.594915  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:49.596169  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:49.575372 13066 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:49.591157  6473 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:49.567807 13237 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:49.572703 13241 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 19. Current term is 20.
I20260709 12:19:49.569293 13220 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:49.606961  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:49.608664  6651 raft_consensus.cc:2393] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 17: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:49.619139  6656 raft_consensus.cc:2368] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 18. Current term is 19.
I20260709 12:19:49.607748  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:49.612213  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:49.609619  6550 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:49.596856  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:49.586506 13191 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:49.609462  6473 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:49.609187 13078 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:49.608438 13229 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:49.615224  6582 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
W20260709 12:19:49.632903  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 120567us
I20260709 12:19:49.610158 13066 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:49.608021 13163 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:49.608533 13197 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:49.677227  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:49.635810  6550 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:49.645176  6473 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:49.611670 13077 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:49.573521 13216 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:49.610468 13237 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:49.612480 13129 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:49.607321 13107 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:49.709208 13222 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:49.712896  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 67808.2us
W20260709 12:19:49.672725 13009 consensus_meta.cc:287] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.524s	user 0.012s	sys 0.000s
I20260709 12:19:49.713836  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:49.718230  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 82519.5us
I20260709 12:19:49.705708 13138 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:49.641352 13014 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:19:49.641489 13191 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:49.653996 13212 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:49.732192  6506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 812) took 1366 ms. Trace:
I20260709 12:19:49.737176  6578 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:19:49.735138  6506 rpcz_store.cc:276] 0709 12:19:48.366030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:48.366546 (+   516us) service_pool.cc:224] Handling call
0709 12:19:48.393460 (+ 26914us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:48.839033 (+445573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:49.731959 (+892926us) spinlock_profiling.cc:243] Waited 338 ms on lock 0x7b54001e78d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:49.732137 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":776918400}
I20260709 12:19:49.622223 13220 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:49.652856 12935 consensus_queue.cc:1059] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
I20260709 12:19:49.748064  6657 raft_consensus.cc:2393] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 19: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:49.653427 13199 consensus_queue.cc:1048] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
I20260709 12:19:49.669641 13078 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:49.732719  6575 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 2652) took 1303 ms (client timeout 1646 ms). Trace:
I20260709 12:19:49.711316 13242 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:49.715536 13129 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:49.715811 13237 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:49.720547 13075 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:49.720520  6550 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:49.759366  6575 rpcz_store.cc:269] 0709 12:19:48.429140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:48.448886 (+ 19746us) service_pool.cc:224] Handling call
0709 12:19:49.732549 (+1283663us) spinlock_profiling.cc:243] Waited 344 ms on lock 0x7b5400058cd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:49.732662 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":791906560}
I20260709 12:19:49.780224  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:49.738574 13156 raft_consensus.cc:993] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e: : Instructing follower f2b06aebe97249a4877d798b8118ece4 to start an election
I20260709 12:19:49.762688  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:49.763926  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:49.782078  6649 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:49.654153 13200 consensus_queue.cc:237] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:49.769331  6623 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:49.792884  6649 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:19:49.781616  6652 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22.
I20260709 12:19:49.796979  6475 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:49.780911  6575 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 17.
I20260709 12:19:49.699821 13163 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:49.788968  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:49.797237  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:49.732807 13144 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:19:49.802677  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 82225.5us
I20260709 12:19:49.804400 13242 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:49.805943  6654 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:49.858139  6654 raft_consensus.cc:1275] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:49.802409 13144 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:49.845508  6550 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:49.808866 13078 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:49.811199 13216 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:49.811985 13007 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:49.799914 13156 raft_consensus.cc:1081] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 LEADER]: Signalling peer f2b06aebe97249a4877d798b8118ece4 to start an election
W20260709 12:19:49.816955 13250 log.cc:927] Time spent T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:19:49.817325 13063 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:49.771662 13206 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:49.820676  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:49.821921 13138 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:49.822350 13220 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:49.825835 13107 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:49.831553 13197 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:49.833395  6650 raft_consensus.cc:1240] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:49.800731 13077 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:49.835101 12878 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:49.836529 13192 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:19:49.836823  6578 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 2649) took 1682 ms (client timeout 1912 ms). Trace:
W20260709 12:19:49.916615  6578 rpcz_store.cc:269] 0709 12:19:48.154550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:48.319887 (+165337us) service_pool.cc:224] Handling call
0709 12:19:49.836655 (+1516768us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54001cbe54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:49.836768 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3063066880}
I20260709 12:19:49.838910  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:49.842572  7274 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:49.799399  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:49.855255 13245 raft_consensus.cc:2804] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:19:49.824076 13163 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:49.861001  6550 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:49.798261 12992 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:49.868185  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:49.864220 13253 consensus_queue.cc:1059] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 12.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 12:19:49.859983 13144 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:49.869438 13199 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:49.869400 13041 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:49.915373  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:49.892243 13242 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:49.920343 13192 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:49.926347 13245 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:49.941298  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 151388us
I20260709 12:19:49.913487 13206 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:49.942327  6622 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:49.932161 13041 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:49.937578 13192 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:49.938776 13245 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:49.926712 13077 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:49.936035 13242 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:49.939597 12992 raft_consensus.cc:697] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:49.929844 13013 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:49.935798 13237 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:49.950603 13041 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:49.951308 13197 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:49.948989 13157 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:49.935086 13199 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:49.952816 13204 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:49.954159 13241 consensus_queue.cc:1059] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.010s
I20260709 12:19:49.975310 13215 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:49.983615 13215 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:49.986382 13237 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:49.988022 13206 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:49.989473 13077 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:49.990530 12992 consensus_queue.cc:237] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:49.991757 13013 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:49.964404 13157 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:49.994938 13157 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:50.001446 13144 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:50.003803 13129 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:50.004174 13144 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:50.139681 13199 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:50.135406 13191 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:50.157604 13245 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:50.159847 13245 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:49.837782  6475 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:50.218015  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:49.951680  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:50.221767  6581 raft_consensus.cc:2393] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 11: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:19:50.225826  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:50.230696  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 392790us
I20260709 12:19:50.232539  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:50.235571  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:50.236013  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:50.236625  6506 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:50.238314  6507 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:19:50.240320  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:50.240970  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:50.241950  6505 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
W20260709 12:19:50.219623  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 358414us
W20260709 12:19:50.221788  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 270021us
I20260709 12:19:50.253203  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:50.255821  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:50.257108  6582 raft_consensus.cc:2368] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 15. Current term is 16.
I20260709 12:19:50.256875  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:50.257503  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:50.258661  6502 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:50.260215  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:50.265329  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:50.266556  6577 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:50.268646  6575 raft_consensus.cc:1275] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 19 index: 8. Preceding OpId from leader: term: 20 index: 9. (index mismatch)
I20260709 12:19:50.269254  6475 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:50.271775  6475 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:50.272755  6575 raft_consensus.cc:1275] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:19:50.272895  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:50.273527  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:50.274124  6582 raft_consensus.cc:2393] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 22: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:50.275545  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:50.278290 12492 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:50.280761  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:50.281672 12881 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:50.284260 13007 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:50.285027 12935 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:50.285110 13007 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:19:50.286049 12935 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:50.284216  6550 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:50.279481 12492 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:50.291927 12881 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:50.293249 12492 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:50.294167 13013 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:50.295190 13013 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:50.298589 13007 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:50.299445 12903 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:50.302023  6475 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:50.314904  6550 leader_election.cc:400] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 18. Current term is 19.
I20260709 12:19:50.315439  6550 leader_election.cc:403] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
W20260709 12:19:50.356623  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 79692.9us
I20260709 12:19:50.300086 12903 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:50.375928 13078 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:50.376559  6475 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:50.376856 12914 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:50.377625 12914 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:50.380120  6506 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:19:50.380551 13014 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:50.372581  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:50.159763 13215 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:50.390404  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:50.403106 13206 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:50.403678 13206 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:50.404157 13206 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:50.404795  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:50.391346  6502 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:19:50.381244 12903 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:50.391084  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 76556.1us
W20260709 12:19:50.390242  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 93503.9us
I20260709 12:19:50.380153 12914 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:50.392421 12935 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:50.427954 13197 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:19:50.429723 13199 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:50.431959 13198 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:50.393860 13013 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:50.141741 13066 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:50.440820 13197 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:50.454064 13197 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:50.456378  6506 raft_consensus.cc:1240] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 16. Current term is 17. Ops: []
W20260709 12:19:50.467664  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 143666us
I20260709 12:19:50.469575 13198 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:50.472390 13077 consensus_queue.cc:1059] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:19:50.472726 12914 consensus_queue.cc:1048] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:19:50.476718 13216 raft_consensus.cc:3055] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:50.461592 12881 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:50.478438 13199 raft_consensus.cc:697] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:19:50.497684  6622 connection.cc:256] RPC call timeout handler was delayed by -1.0669278709999617s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1912s for a response
I20260709 12:19:50.477274 13216 raft_consensus.cc:740] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:50.497781  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:50.505066 13216 consensus_queue.cc:260] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:50.505684 13199 consensus_queue.cc:237] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:50.507476  6577 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:19:50.508261 12878 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:50.508880 12878 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:50.475317 13149 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:50.510771 13058 raft_consensus.cc:3055] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:50.495347  6474 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:50.511289 13058 raft_consensus.cc:740] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:50.512243 13215 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:50.513247 13215 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:50.513863 13215 raft_consensus.cc:697] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 17 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:50.515231 13215 consensus_queue.cc:237] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:50.483804 13193 raft_consensus.cc:3055] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:19:50.472481 13075 consensus_queue.cc:1059] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 12:19:50.506979 13216 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:50.516254 13065 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:50.520694 13065 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:50.486446 13191 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:50.510531 13149 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:50.517274 13058 consensus_queue.cc:260] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:50.524475 13058 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:50.524564 13007 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:50.525466 13007 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:50.517436 13193 raft_consensus.cc:740] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:50.527684 13193 consensus_queue.cc:260] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:50.475872 12492 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:50.518734 13075 raft_consensus.cc:3055] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:50.545156 13193 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:50.547501 13065 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:50.545234 13075 raft_consensus.cc:740] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:19:50.548430  6583 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.007s	sys 0.012s
I20260709 12:19:50.549780 13075 consensus_queue.cc:260] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:50.552426 12992 consensus_queue.cc:1048] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:19:50.556536  6502 raft_consensus.cc:1275] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:19:50.528149 13007 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:50.557588  6501 raft_consensus.cc:1275] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 19 index: 8. Preceding OpId from leader: term: 20 index: 9. (index mismatch)
I20260709 12:19:50.557777 13007 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:50.558193 13129 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:50.556577  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:50.558945 13129 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:50.558547 13007 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:50.555413  6500 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:50.561056 13066 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:50.562116  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 2662) took 1040 ms. Trace:
W20260709 12:19:50.562703 13041 consensus_meta.cc:287] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.599s	user 0.011s	sys 0.007s
I20260709 12:19:50.563164 13041 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:50.564309  6579 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:19:50.564698 13007 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:50.524232 13149 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:50.562727  6583 rpcz_store.cc:276] 0709 12:19:49.521959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:49.645710 (+123751us) service_pool.cc:224] Handling call
0709 12:19:49.655849 (+ 10139us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:49.796036 (+140187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:49.796066 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:49.796645 (+   579us) log.cc:844] Serialized 52 byte log entry
0709 12:19:49.925103 (+128458us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:50.549146 (+624043us) log.cc:844] Serialized 14 byte log entry
0709 12:19:50.560709 (+ 11563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:50.561602 (+   893us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b54004303d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:50.561656 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:50.561755 (+    99us) raft_consensus.cc:1685] finished
0709 12:19:50.561898 (+   143us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:50.562057 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80646,"prepare.queue_time_us":24919,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":1323,"raft.queue_time_us":16169,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":2979,"spinlock_wait_cycles":314296064,"thread_start_us":734863,"threads_started":2,"wal-append.queue_time_us":160852}
I20260709 12:19:50.572173 13191 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:50.564044 12914 consensus_queue.cc:1059] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 12:19:50.578125  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:50.590961  6506 raft_consensus.cc:2368] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 19. Current term is 20.
I20260709 12:19:50.496271 13063 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:50.588255 13239 raft_consensus.cc:3055] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:19:50.592621 13239 raft_consensus.cc:740] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:50.592682  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:50.594147 13239 consensus_queue.cc:260] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:50.572281  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 805326us
I20260709 12:19:50.578554  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:50.551532 13075 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:50.581089 13077 raft_consensus.cc:3055] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:50.577608 13140 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:50.573024 13191 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:50.582430 13242 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:50.597824  6500 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 17.
I20260709 12:19:50.603807 13239 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:50.604619 13063 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:50.605342 13191 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:50.555930 12903 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:50.601704 13041 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:50.617981  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 111591us
I20260709 12:19:50.602293 13077 raft_consensus.cc:740] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:50.614970 13140 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:50.623570 13140 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:50.625754 13077 consensus_queue.cc:260] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:50.627408 13077 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:50.619621 12878 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:50.631191  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:50.632764 13281 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:50.633509 13281 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:50.636183  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:50.636641  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:50.635900 13281 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:50.636210  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:50.648655  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:48.744043  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:50.650537  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:50.650717  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:50.559587 13129 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:50.613210 13265 log.cc:927] Time spent T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.619s	user 0.000s	sys 0.014s
I20260709 12:19:50.611835 12492 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:50.657536  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:50.658533  6500 raft_consensus.cc:2368] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 17. Current term is 18.
I20260709 12:19:50.659297  6658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3356) took 1103 ms. Trace:
I20260709 12:19:50.659663  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:50.660665  7267 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:19:50.659698 12492 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:50.665303 13278 log.cc:927] Time spent T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.137s	user 0.000s	sys 0.002s
I20260709 12:19:50.666126  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:50.667182 13129 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:50.668577  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:50.670292  6656 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:50.669250  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:50.669582  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:50.672744  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:50.673095  6651 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:19:50.674134  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:50.675292  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:50.672745  6654 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:50.692013  6507 raft_consensus.cc:1240] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 12. Current term is 13. Ops: []
I20260709 12:19:50.693943  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:50.694577  6500 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:50.690685  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:50.696410  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:50.619848 13197 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:50.699798 13197 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:50.701128  6507 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:50.703558 13063 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:50.704362 13192 consensus_meta.cc:287] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.652s	user 0.010s	sys 0.007s
I20260709 12:19:50.704686  6653 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:58980
I20260709 12:19:50.705027 13192 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:50.659732  6658 rpcz_store.cc:276] 0709 12:19:49.555517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:49.563807 (+  8290us) service_pool.cc:224] Handling call
0709 12:19:49.564149 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:49.564783 (+   634us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:19:49.683861 (+119078us) log.cc:844] Serialized 14 byte log entry
0709 12:19:49.901361 (+217500us) log.cc:844] Serialized 14 byte log entry
0709 12:19:49.901528 (+   167us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:49.902690 (+  1162us) log.cc:844] Serialized 26 byte log entry
0709 12:19:49.902828 (+   138us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:49.902879 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:49.902959 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:50.579255 (+676296us) spinlock_profiling.cc:243] Waited 1.24 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:50.586800 (+  7545us) spinlock_profiling.cc:243] Waited 4.66 ms on lock 0x7b180025fe40. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:19:50.658922 (+ 72122us) raft_consensus.cc:1685] finished
0709 12:19:50.658944 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:50.659108 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37965,"prepare.queue_time_us":277924,"prepare.run_cpu_time_us":117,"prepare.run_wall_time_us":105,"raft.queue_time_us":2756,"raft.run_cpu_time_us":387,"raft.run_wall_time_us":14955,"spinlock_wait_cycles":13569664,"thread_start_us":285779,"threads_started":2,"wal-append.queue_time_us":265781}
I20260709 12:19:50.707849  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:50.705505  6653 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:50.724684  6653 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:50.729987  6658 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:50.733089 12903 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:50.724436  6655 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:19:50.736030  6655 raft_consensus.cc:1275] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:50.706871  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:50.711973  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:50.714335  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:50.718605  6622 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:50.738341  6657 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:50.738628  7274 raft_consensus.cc:2393] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 11: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:50.719875  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:50.724994  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:50.739586  6622 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:50.739720  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:50.709446  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:50.742147 13204 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:50.710631 13192 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:50.744071  6622 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:50.744946 13197 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:50.768427 13197 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:50.770846 13197 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:50.747184  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:50.773077  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:50.773928  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:50.774940  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:50.775928 13197 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:50.776813 13206 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:50.776684 13197 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:50.753212  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:50.757673  6405 catalog_manager.cc:5697] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 10 to 11, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 11 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:19:50.759828  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:50.779513 13197 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:50.655023 13078 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 18. Current term is 19.
I20260709 12:19:50.795859  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:50.796669  6583 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:50.798271  6507 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 12:19:50.800530  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:50.801719  6507 raft_consensus.cc:2393] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 17: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:50.803228  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:50.805085  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:50.806090  6649 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:19:50.806646  6583 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 18.
I20260709 12:19:50.817097  6658 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:50.819029  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:50.820238  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 76274.7us
I20260709 12:19:50.820688 13276 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:19:50.821076  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 61745.1us
I20260709 12:19:50.833326  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:50.837199  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:50.837925  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:50.840302  6578 raft_consensus.cc:1240] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:50.839413  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:50.842249  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:50.840374  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:50.851821  6654 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:19:50.854529  6651 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 11.
I20260709 12:19:50.877192  6577 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:50.879923  6584 raft_consensus.cc:1240] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:50.881626  6580 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 21. Current term is 22. Ops: [21.11-21.11]
W20260709 12:19:50.887914  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:50.779430 13206 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:50.890247  6650 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:50.896471  6575 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:19:50.898726  7267 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:19:50.888453 13193 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:50.915518  6582 raft_consensus.cc:2393] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 11: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:50.916127  6507 raft_consensus.cc:2393] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 19: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:50.925796 13276 raft_consensus.cc:697] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:50.932437  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:50.933380 13276 consensus_queue.cc:237] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:50.934152  6473 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:50.935369 13200 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:50.924573  6653 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:50.934229  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:50.916081  6500 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 18.
I20260709 12:19:50.918663  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:50.918783  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:50.918885  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:50.919497  6501 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:50.919626  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:50.919730  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:50.920363  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:50.920460  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:50.914947 13063 raft_consensus.cc:2764] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:19:50.947733  6655 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:50.938939  6579 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:19:50.926282  6656 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
W20260709 12:19:50.941205  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:50.920472  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:50.951506  6584 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:50.953058  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:50.961249  6653 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:50.960242  6576 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:19:50.962680  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:50.967222  6580 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:50.965271  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:50.966431  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:50.978178  6549 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:50.978559  6581 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:50.988287  6657 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:19:50.999073  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:50.957453 13193 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:51.000037  6505 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:51.004226  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:51.010620  7274 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 12:19:51.025585  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 66089.4us
I20260709 12:19:51.019181 13206 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:51.026733  6549 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:51.026841 13206 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:51.028378 13009 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.982s	user 0.041s	sys 0.050s
I20260709 12:19:51.009615 13129 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:19:51.045450 13009 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.110s	user 0.041s	sys 0.050s
W20260709 12:19:51.045675  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 309750us
W20260709 12:19:51.046109  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 83639.4us
I20260709 12:19:51.046324 13206 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:51.047767 12492 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:51.047854 13193 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:51.048480  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:51.051482  6473 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:51.052958  7238 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:51.054515  7238 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:51.055490  6622 leader_election.cc:400] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 15. Current term is 16.
I20260709 12:19:51.055831  6622 leader_election.cc:403] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:19:51.059095 13138 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:19:51.039542  6583 raft_consensus.cc:1240] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:19:51.042282  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:51.068439  6473 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:19:51.071086  6407 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57900 (request call id 191) took 25203 ms (client timeout 15000 ms). Trace:
I20260709 12:19:51.072646 13066 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:51.085918  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:51.080910 13242 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:51.093431  6473 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:51.096920 13181 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 12:19:51.088932 13129 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:51.097960 13129 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:51.101076  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:51.101632  6503 raft_consensus.cc:2393] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 21: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:51.111306  6581 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:19:51.100813 13013 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:51.119335  7238 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:51.071535  6407 rpcz_store.cc:269] 0709 12:19:25.867424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:25.896434 (+ 29010us) service_pool.cc:224] Handling call
0709 12:19:51.070862 (+25174428us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:41.525593 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:41.526395 (+   802us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:41.526452 (+    57us) write_op.cc:435] Acquired schema lock
0709 12:19:41.526491 (+    39us) tablet.cc:662] Decoding operations
0709 12:19:41.672570 (+146079us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:41.698344 (+ 25774us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:41.701351 (+  3007us) tablet.cc:685] Acquiring locks for 25 operations
0709 12:19:41.922668 (+221317us) tablet.cc:701] Row locks acquired
0709 12:19:41.922729 (+    61us) write_op.cc:260] PREPARE: finished
0709 12:19:41.922927 (+   198us) write_op.cc:270] Start()
0709 12:19:41.923135 (+   208us) write_op.cc:276] Timestamp: P: 1783599581922859 usec, L: 0
0709 12:19:41.923194 (+    59us) op_driver.cc:348] REPLICATION: starting
0709 12:19:41.924129 (+   935us) log.cc:844] Serialized 9644 byte log entry
0709 12:19:42.701265 (+777136us) spinlock_profiling.cc:243] Waited 7.57 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:43.066065 (+364800us) op_driver.cc:464] REPLICATION: finished
0709 12:19:43.250428 (+184363us) write_op.cc:301] APPLY: starting
0709 12:19:43.269761 (+ 19333us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:43.676396 (+406635us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:43.685354 (+  8958us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:45.850511 (+2165157us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:45.851008 (+   497us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=50,key_file_lookups=50,delta_file_lookups=0,mrs_lookups=0
0709 12:19:45.851077 (+    69us) write_op.cc:312] APPLY: finished
0709 12:19:45.851657 (+   580us) log.cc:844] Serialized 220 byte log entry
0709 12:19:46.065823 (+214166us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:46.075814 (+  9991us) write_op.cc:454] Released schema lock
0709 12:19:46.121529 (+ 45715us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:46.150929 (+ 29400us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:46.184052 (+ 33123us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":148259,"reported_tablets":53,"child_traces":[["op",{"apply.queue_time_us":131836,"apply.run_cpu_time_us":100138,"apply.run_wall_time_us":2891402,"cfile_cache_hit":102,"cfile_cache_hit_bytes":108427,"num_ops":25,"prepare.queue_time_us":515397,"prepare.run_cpu_time_us":27930,"prepare.run_wall_time_us":690140,"raft.queue_time_us":52191,"raft.run_cpu_time_us":22977,"raft.run_wall_time_us":150828,"replication_time_us":1105270,"spinlock_wait_cycles":17407872,"thread_start_us":799586,"threads_started":5,"wal-append.queue_time_us":510811,"wal-append.run_cpu_time_us":14461,"wal-append.run_wall_time_us":2730009}]]}
I20260709 12:19:51.122324  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:51.123191 13009 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:51.110540 12914 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:51.124838 12914 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:51.133543  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:51.137966  6505 raft_consensus.cc:2368] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 18. Current term is 19.
I20260709 12:19:51.131790  6473 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:51.141902  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:51.137173 12914 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:51.088521 13239 consensus_queue.cc:1059] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 12:19:51.159830  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:19:51.158203  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 46542.8us
I20260709 12:19:51.161072  6473 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:51.150345  6504 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:51.159160 13009 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:51.141146 13149 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:19:51.168170 13009 raft_consensus.cc:697] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 19 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:51.169346 13009 consensus_queue.cc:237] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:51.172906  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:51.165035 13200 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:51.173647  6500 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
I20260709 12:19:51.173832 13200 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:51.174244 13200 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:51.174322  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:51.175196  6508 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:19:51.175742  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:51.163728 13193 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:51.163189  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:51.176534 13075 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:51.173031  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:51.177937  6507 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:19:51.110677 12492 consensus_queue.cc:1059] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 12:19:51.174582  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:51.110983 13078 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 12:19:51.157675 13066 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:51.110839 13007 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 12:19:51.183782  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:51.176149  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:51.173524  6509 raft_consensus.cc:1275] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:51.187263  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:51.187353  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:51.181571 13215 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.023s	sys 0.088s
I20260709 12:19:51.192351  6507 raft_consensus.cc:2393] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 19: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:51.194011  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:51.194152  6500 raft_consensus.cc:2410] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 23 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 12:19:51.188897  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:51.197247  6655 raft_consensus.cc:1240] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 21. Current term is 22. Ops: []
W20260709 12:19:51.194131 13215 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.023s	sys 0.089s
I20260709 12:19:51.207634 12914 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:51.210546  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:51.212250  6509 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:19:51.212183  6504 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:51.215356  6473 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:51.219383  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:51.219790  6508 raft_consensus.cc:2393] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 19: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:51.201972 13066 raft_consensus.cc:697] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 18 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:51.213832 13129 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:51.224151 12914 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:51.230558  6622 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:51.239593  6584 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:19:51.240296 13282 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:51.228288 13285 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:51.208395 13205 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.102s
I20260709 12:19:51.248507 12914 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:51.215945 13199 consensus_queue.cc:1059] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 12:19:51.249428  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:51.324141  6657 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:51.324205  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 112275us
I20260709 12:19:51.259784  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:51.325134  6580 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:51.272267  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:51.273978  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:51.254441 13206 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:19:51.281870  6583 raft_consensus.cc:1275] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:19:51.260584 13007 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:51.291352 13066 consensus_queue.cc:237] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:51.298105  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:51.305751  7238 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.525s (SENT) [suppressed 2 similar messages]
I20260709 12:19:51.310204  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:51.337896  6656 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:51.338549  6652 raft_consensus.cc:1240] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:19:51.338721 13199 raft_consensus.cc:3055] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:51.339164 13199 raft_consensus.cc:740] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:51.317425  6579 raft_consensus.cc:1275] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:19:51.340071 13199 consensus_queue.cc:260] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:51.341122 13007 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:51.318032 13282 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:51.343395 13007 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:51.343565 13282 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:51.343776 12914 raft_consensus.cc:697] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:51.345108 12914 consensus_queue.cc:237] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:51.320820  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:51.308320 13193 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:51.348238 13193 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:51.348937 13292 log.cc:927] Time spent T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.057s	user 0.003s	sys 0.000s
I20260709 12:19:51.350517 13193 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:51.361121  6475 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:51.362478  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:51.363564 13149 raft_consensus.cc:3055] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:51.363685 13285 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:51.364990  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:51.365705  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:51.366132 12797 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:51.366784 12797 raft_consensus.cc:697] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 19 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:51.367295  6576 raft_consensus.cc:1240] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 21. Current term is 22. Ops: []
I20260709 12:19:51.367672 12797 consensus_queue.cc:237] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 17.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:51.372833 13041 raft_consensus.cc:3055] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:51.373307 13041 raft_consensus.cc:740] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:19:51.373492  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:51.376746 13129 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:51.377399 13129 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:51.378669 13285 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:51.379189 13285 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:51.379673 13129 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:51.380795  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:51.381309 13285 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:51.382737  6548 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:51.383229  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:51.381389  6655 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:51.384156  6548 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:51.362583 13215 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:51.364084 13149 raft_consensus.cc:740] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:51.396814  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:51.393766  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:51.384075  6578 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:51.398119  6580 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:19:51.397662  7269 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:19:51.409130  6656 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:19:51.410604  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:51.413090  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:51.414086  6650 raft_consensus.cc:2393] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 19: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:51.413129  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:51.423316  6505 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 19: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:51.431223  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:51.395251 13007 consensus_queue.cc:1059] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 12:19:51.420464 13199 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:51.413332 13129 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:51.434973  6474 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:51.432879  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:51.425550 12492 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:51.439543  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:51.443166 13215 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:51.437819  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: true dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:51.451474  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:51.450016 13215 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:51.454908  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:51.452456  6507 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:19:51.455914  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:51.433290  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 343477us
I20260709 12:19:51.463482  6657 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
W20260709 12:19:51.465603  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 101244us
I20260709 12:19:51.471278  6504 raft_consensus.cc:2393] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 19: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:51.467163  6475 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:51.473002  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 88966.1us
I20260709 12:19:51.438760 13129 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:51.432547 13200 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:51.490312 13200 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:51.468508 13215 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:51.492717  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:51.493304  6575 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:51.493481  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:51.495437 13205 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:51.496129  6503 raft_consensus.cc:2393] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 16: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:51.496789 13063 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:51.500468 13075 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:51.500823  6548 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:51.506551 12492 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:51.510195  6575 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:51.513952  6550 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:51.478852 13129 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:51.518224  6548 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:51.519220  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 52156.4us
I20260709 12:19:51.520237  6475 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:51.521529  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 58878.6us
I20260709 12:19:51.477414 13193 consensus_queue.cc:1059] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 12:19:51.534130  6650 raft_consensus.cc:1275] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:19:51.535465  6550 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:51.548873  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:51.550063  6656 raft_consensus.cc:2393] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 19: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:51.550140  6550 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:51.553537  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:51.473451 13149 consensus_queue.cc:260] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:51.526216 13075 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:51.556012  6505 raft_consensus.cc:2393] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 17: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:51.537705 12492 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:51.558888  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:51.539427 13007 raft_consensus.cc:2764] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:19:51.560859  6475 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:51.561980  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 43902.7us
I20260709 12:19:51.561512 12492 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:51.576960  6548 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:51.579154 13215 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:51.582337 13215 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:51.582882  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:51.583549  6575 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:19:51.588923 12992 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:51.592054  6475 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:51.592747  6575 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:19:51.593061 13298 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:51.594149 13298 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:51.594640 13298 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:51.595067 13298 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:51.605155  6655 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 23.
W20260709 12:19:51.609571  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:51.610131  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:51.610569  6475 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:51.575752 13075 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:51.575860 12492 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:51.606590 13138 consensus_meta.cc:287] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.547s	user 0.013s	sys 0.000s
I20260709 12:19:51.616317  6475 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:51.625648  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 66888.8us
I20260709 12:19:51.626597  6504 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 18. Current term is 19. Ops: []
W20260709 12:19:51.628613  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 322860us
W20260709 12:19:51.632786  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 55967.6us
I20260709 12:19:51.615350 13215 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:19:51.619464 13007 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:51.619745 13200 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:51.644676  6475 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:51.698731  7238 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:51.701740 13041 consensus_queue.cc:260] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:51.706724  7238 leader_election.cc:400] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 19. Current term is 20.
I20260709 12:19:51.706921 13041 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:51.708632 13063 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:19:51.713158  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 74428.6us
W20260709 12:19:51.714155  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 164095us
I20260709 12:19:51.631695 13138 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 15. Current term is 16.
I20260709 12:19:51.628216 13013 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:51.703264  6548 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:51.721544 13193 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:51.724123  6550 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
W20260709 12:19:51.724285  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 82731.7us
W20260709 12:19:51.727077 13295 log.cc:927] Time spent T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 12:19:51.720937 13066 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:51.742442  6658 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:19:51.724023 13193 raft_consensus.cc:697] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:51.751523  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:51.748096 13242 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:51.742601 12878 raft_consensus.cc:2804] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:19:51.757601 13310 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:19:51.759518  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 65642.6us
I20260709 12:19:51.755954  6550 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:51.759485  7238 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:51.744804 13215 consensus_queue.cc:1048] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:19:51.747980 13200 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:51.784631  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:51.757789 13216 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:51.821724  6500 raft_consensus.cc:1240] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:51.822969  7267 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:19:51.757020 13197 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:51.829831  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:51.830673  6656 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:51.834717  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:51.762141 12992 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:19:51.834954  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 77577us
I20260709 12:19:51.838876  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:51.751766 13138 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.073s
I20260709 12:19:51.751343 13181 consensus_queue.cc:1059] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 12:19:51.821538 13239 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:51.821861 13009 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:51.848022 13009 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:51.850203 13009 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:51.850807 13181 raft_consensus.cc:3055] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:51.851315 13181 raft_consensus.cc:740] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:51.852279 13181 consensus_queue.cc:260] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:51.853713 13181 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:51.834755 13310 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:51.856590 13310 raft_consensus.cc:697] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:51.856815  6549 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
W20260709 12:19:51.842777  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 95766us
I20260709 12:19:51.858708  6550 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:51.864763  6622 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:51.761987 12797 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.096s
I20260709 12:19:51.780159 13282 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:51.772382 13242 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:51.778347  6392 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 3e0ab356fc074f589a6ff4f9b4996b09 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 496c7c3792cd4587b4dbeba435f4be1e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 18 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:19:51.826874 13205 consensus_queue.cc:1059] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
W20260709 12:19:51.878100  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 115724us
I20260709 12:19:51.834414 13285 consensus_queue.cc:1048] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 12:19:51.832705 13216 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:51.834275 13200 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:51.885574  6392 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 107452us
I20260709 12:19:51.780624 13193 consensus_queue.cc:237] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:51.857509 13310 consensus_queue.cc:237] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:51.888041  6548 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:51.881151 12992 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:51.829288 13197 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:51.882953 12878 raft_consensus.cc:697] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:51.872910 13075 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.012s
W20260709 12:19:51.894167  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 73918.9us
I20260709 12:19:51.895153  6473 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:19:51.906199 13309 log.cc:927] Time spent T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 12:19:51.893850 12992 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:51.908726 13063 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:51.909597 13216 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:51.932833 13063 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:51.933549 13216 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:51.934152 13216 raft_consensus.cc:697] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:51.911942 13129 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:51.935096 13063 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:51.924753  6550 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:51.928961  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:51.928925  6473 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:51.926369 13317 consensus_queue.cc:1059] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.006s
I20260709 12:19:51.931932  6581 raft_consensus.cc:1240] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:51.905149 13078 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:19:51.898408 13313 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:19:51.932113 13308 log.cc:927] Time spent T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:19:51.939036  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:51.935097 13216 consensus_queue.cc:237] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:51.950109 13300 consensus_queue.cc:1059] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:19:51.954263  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 52576.4us
I20260709 12:19:51.955188  6550 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:51.938786  6473 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:51.941848  6548 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:51.957834  6548 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:51.958578 13318 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:51.959318 13318 raft_consensus.cc:697] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:51.942519 13129 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:51.890245 13298 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:51.936187 12992 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:51.925292 13315 raft_consensus.cc:2764] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 12:19:51.940240 13078 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:19:51.964088 13302 log.cc:927] Time spent T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
I20260709 12:19:51.967621 13197 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:51.969308 13149 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:51.971695 13285 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:19:51.973073 13291 log.cc:927] Time spent T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.682s	user 0.005s	sys 0.015s
W20260709 12:19:51.976589  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:51.983139 13193 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:51.983942 13314 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:19:51.984543 13320 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:51.987703 13200 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:51.989579 13282 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:51.980861 13313 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:51.926757 13007 consensus_queue.cc:1059] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:19:51.999078 12492 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:52.000976 13206 raft_consensus.cc:3055] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:52.002960 13298 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:52.012455 13193 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:51.993359 13078 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:19:52.021409 13041 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:51.929778 12878 consensus_queue.cc:237] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:52.026360 13315 consensus_queue.cc:1059] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:52.024976 13129 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 12:19:52.023442 13285 raft_consensus.cc:697] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:52.023703 13206 raft_consensus.cc:740] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:52.023439 13314 raft_consensus.cc:697] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:19:52.042068  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 85735.9us
I20260709 12:19:52.030766 13041 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:51.969152 13318 consensus_queue.cc:237] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:52.077808 13285 consensus_queue.cc:237] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:52.048723 13316 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:52.065725 13206 consensus_queue.cc:260] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:52.073585  6502 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 2985) took 1114 ms. Trace:
I20260709 12:19:52.075573 12492 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:52.077440 13199 raft_consensus.cc:3055] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:19:52.087961  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:52.100090  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:52.102909  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:52.105039  6653 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 12:19:52.106458  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:52.093645  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 134159us
I20260709 12:19:52.115214  6548 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:52.121129 13276 consensus_queue.cc:1059] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:19:52.122658 13276 consensus_queue.cc:1059] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:52.123937 13276 raft_consensus.cc:3055] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:52.097028  6575 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:19:52.174661 13276 raft_consensus.cc:740] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:52.176643 13276 consensus_queue.cc:260] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:52.128773 12797 raft_consensus.cc:3055] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:52.178402 12797 raft_consensus.cc:740] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:52.179736 12797 consensus_queue.cc:260] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:52.181476 12797 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:52.160256 12914 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:52.184947 12914 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:52.167874  6504 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:19:52.125078  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:52.190028  7274 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:52.197374  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:52.198377  6583 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:19:52.188166 12914 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:52.212616  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:52.212757  6473 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:52.215449  6575 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:19:52.218807 12914 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:52.219521 12914 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:52.220072 12914 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:52.224066 13276 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:52.232184  6503 raft_consensus.cc:1275] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:52.092541  6502 rpcz_store.cc:276] 0709 12:19:50.959289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:50.959463 (+   174us) service_pool.cc:224] Handling call
0709 12:19:50.968767 (+  9304us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:51.020722 (+ 51955us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:51.020774 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:51.085600 (+ 64826us) log.cc:844] Serialized 52 byte log entry
0709 12:19:51.569118 (+483518us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:51.747168 (+178050us) log.cc:844] Serialized 14 byte log entry
0709 12:19:51.756889 (+  9721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:51.756953 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:52.073196 (+316243us) raft_consensus.cc:1685] finished
0709 12:19:52.073227 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:52.073473 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27681,"prepare.queue_time_us":197945,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":52,"raft.queue_time_us":29387,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":12004,"thread_start_us":516439,"threads_started":2,"wal-append.queue_time_us":608445}
I20260709 12:19:52.103392  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:52.252873  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 146989us
I20260709 12:19:52.081516 13041 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:52.257890  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:52.261655  7274 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:19:52.264407  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:52.265255  6651 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:52.270352  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:52.270529  6577 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:19:52.273206  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:52.273869  6509 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:52.097990 13013 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:52.097846 13199 raft_consensus.cc:740] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:52.288879  6475 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:52.302095  6651 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:19:52.106107 13316 raft_consensus.cc:697] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:52.294903  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:52.270164 12492 raft_consensus.cc:3055] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:52.327303  6475 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:52.331422  6508 raft_consensus.cc:1275] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 12:19:52.311486 13325 log.cc:927] Time spent T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.279s	user 0.000s	sys 0.001s
I20260709 12:19:52.287250 13322 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:52.341248  6473 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:52.260890 13317 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 12:19:52.345386  6475 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:52.325067 13206 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:52.359297  6508 raft_consensus.cc:1275] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 12:19:52.348838 13322 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:52.336302 13200 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:52.361593 13239 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:19:52.363116 13242 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:52.333493 12492 raft_consensus.cc:740] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:19:52.343896 13311 log.cc:927] Time spent T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 12:19:52.346189 13013 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:52.390141  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 274991us
I20260709 12:19:52.351279 13197 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:19:52.418143  6548 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:52.389428 13215 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:52.389982 13322 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:52.380405 13336 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:19:52.480592  6548 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:52.523496  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 131403us
I20260709 12:19:52.478050 13197 raft_consensus.cc:697] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 23 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:52.405092 13138 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.092s
I20260709 12:19:52.405306 13063 consensus_queue.cc:1048] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.090s
I20260709 12:19:52.477667 13215 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:52.488790 13318 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:52.498749 13193 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:52.590535  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 117308us
W20260709 12:19:52.583693  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 238334us
I20260709 12:19:52.549297 13288 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:52.560442 13313 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:52.584214 13215 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:52.561167 13009 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:52.582075 13318 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:52.371697 13314 consensus_queue.cc:237] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:52.619020  6548 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:52.620441  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 60681.7us
I20260709 12:19:52.589218 13193 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:19:52.650919  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:52.589924 13007 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:52.620986  6550 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:52.653895 13326 log.cc:927] Time spent T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.318s	user 0.000s	sys 0.001s
I20260709 12:19:52.653487 13197 consensus_queue.cc:237] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:52.657061 13344 log.cc:927] Time spent T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
W20260709 12:19:52.659473 13078 consensus_meta.cc:287] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.627s	user 0.007s	sys 0.001s
I20260709 12:19:52.659600 13041 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:52.661123 13341 log.cc:927] Time spent T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:19:52.601044 13336 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:19:52.691113  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 70218.2us
W20260709 12:19:52.653153  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:52.715092  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:52.615427 13321 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:52.622254 13318 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:52.726207  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 60649.8us
I20260709 12:19:52.726684  6550 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:52.572621 12492 consensus_queue.cc:260] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:52.759730  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:52.659971 13078 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:52.910146  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:52.911201  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:52.908775 13335 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:52.620800 13315 consensus_queue.cc:1059] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.105s
I20260709 12:19:52.912775  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:52.912508  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:52.910876  6548 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:52.916435  6508 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:19:52.726315 13313 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:52.689785 13336 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:52.761818  6651 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:19:52.510003 13316 consensus_queue.cc:237] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:52.916093  6658 raft_consensus.cc:1240] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 12:19:52.622262 13288 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:52.964223 13288 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:52.662354 13282 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.250s
I20260709 12:19:52.663101 13009 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:52.966478  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:52.972016  6505 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 20.
W20260709 12:19:52.965297  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 238670us
I20260709 12:19:52.970772  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:52.974392  6580 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:19:52.977473  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:52.721885 13193 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:52.926193 13298 raft_consensus.cc:3055] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:52.984061  6657 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:19:52.954975 12878 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:52.951318 13129 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:52.688817 13007 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:52.950982 13336 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:53.016527  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 105740us
W20260709 12:19:53.019555  6509 consensus_meta.cc:287] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.745s	user 0.007s	sys 0.001s
I20260709 12:19:52.975764 13288 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:53.024578  6509 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
W20260709 12:19:53.028450  7240 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.000s	sys 0.027s
W20260709 12:19:53.029035 12992 consensus_meta.cc:287] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 1.026s	user 0.006s	sys 0.006s
I20260709 12:19:53.028798 13333 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.339s
I20260709 12:19:53.030025 13329 consensus_queue.cc:1059] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 12:19:53.030959 13315 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:19:53.032121 13348 log.cc:927] Time spent T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 12:19:53.032437 13289 consensus_queue.cc:1048] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.115s
W20260709 12:19:53.033056 13330 log.cc:927] Time spent T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.344s	user 0.000s	sys 0.000s
W20260709 12:19:53.037035 13346 log.cc:927] Time spent T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.374s	user 0.001s	sys 0.000s
I20260709 12:19:53.043392 13078 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:53.057266 13334 log.cc:927] Time spent T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
W20260709 12:19:53.095695 13206 consensus_meta.cc:287] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.714s	user 0.009s	sys 0.001s
W20260709 12:19:53.102281 12914 consensus_meta.cc:287] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.881s	user 0.011s	sys 0.000s
W20260709 12:19:53.080274  7240 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.823s	user 0.000s	sys 0.028s
I20260709 12:19:52.992548 13298 raft_consensus.cc:740] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:19:53.068284 13320 consensus_meta.cc:287] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 1.055s	user 0.006s	sys 0.002s
I20260709 12:19:53.011395 13216 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.101773 13129 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:19:53.109143  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 128528us
W20260709 12:19:53.094687 13324 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.024s	sys 0.045s
I20260709 12:19:53.153870  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:53.154675  7274 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 13.
I20260709 12:19:52.951293 13335 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:53.095049 12992 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:53.063971 13276 consensus_meta.cc:287] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.839s	user 0.009s	sys 0.000s
I20260709 12:19:53.060228 13007 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:53.126788 13013 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:52.929271 13321 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:53.114465 12878 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.116443 13205 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.157765  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:53.155426 13317 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.021s	sys 0.074s
I20260709 12:19:53.089538 13288 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:53.153782 13324 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.024s	sys 0.045s
I20260709 12:19:53.144272 13315 raft_consensus.cc:697] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:53.243535  6650 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:19:53.243522  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:53.244534  7267 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:53.244915  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:53.245256  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:53.153357 13320 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:53.248394 13013 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.255144 13013 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:53.256930 13368 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:53.257484 12992 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:53.257551 13368 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:53.260427 13338 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.261011 13338 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:53.262050 13200 consensus_meta.cc:287] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.866s	user 0.011s	sys 0.000s
I20260709 12:19:53.249344  7269 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:19:53.263962  6500 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2841) took 1094 ms. Trace:
I20260709 12:19:53.262755 13200 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:53.264369  6500 rpcz_store.cc:276] 0709 12:19:52.169621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:52.169786 (+   165us) service_pool.cc:224] Handling call
0709 12:19:52.170250 (+   464us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:52.170737 (+   487us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:52.170777 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:52.171471 (+   694us) log.cc:844] Serialized 26 byte log entry
0709 12:19:52.329711 (+158240us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:52.329788 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:52.329885 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:53.263729 (+933844us) raft_consensus.cc:1685] finished
0709 12:19:53.263758 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:53.263902 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2143,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":121,"thread_start_us":157741,"threads_started":1,"wal-append.queue_time_us":450022}
I20260709 12:19:53.166499 13255 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:53.151571 12914 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:53.165864 13007 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.297986 13338 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:53.299891  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:53.254169 13276 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:53.303516 13149 consensus_meta.cc:287] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 1.281s	user 0.010s	sys 0.000s
I20260709 12:19:53.307650 13362 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.313105 13041 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:53.311559 13298 consensus_queue.cc:260] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:53.321069 13357 log.cc:927] Time spent T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:19:53.263110 12878 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:53.324829 13315 consensus_queue.cc:237] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:53.157541 13216 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:53.293772 13242 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.367923  6503 raft_consensus.cc:1240] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 12:19:53.335619 13285 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.328346 13358 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.335975 13078 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:53.363517 13300 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.357853 13075 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:53.243328 13205 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:53.387656  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:53.337996 13007 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:53.357204 13276 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:53.395818  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 304820us
I20260709 12:19:53.397360  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:53.397976  6654 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:53.335006 13255 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:53.388233 13193 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:53.378422 13242 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:53.411039  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:53.366139 13362 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:53.380966 13285 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:53.384369 13078 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:53.384418 13300 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:53.418097  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:53.419550  6578 raft_consensus.cc:1240] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 12:19:53.412901 13371 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:53.421327  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:53.421967  6584 raft_consensus.cc:1240] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:53.394047 13075 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:53.413472  6474 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:53.433255  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:53.383523 13358 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:53.429747  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:53.429332  6582 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 19: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:53.432536 13365 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.408767 13289 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.458981 13353 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:53.459925  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:53.460891 13288 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:53.462256 13013 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:53.463689 13318 consensus_meta.cc:287] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.540s	user 0.005s	sys 0.002s
I20260709 12:19:53.465096 13216 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:53.466004 13367 log.cc:927] Time spent T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260709 12:19:53.414263 13129 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:53.484100  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:53.506750  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:53.486372  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2849) took 1280 ms. Trace:
I20260709 12:19:53.485407  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:53.487959  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 74564.7us
I20260709 12:19:53.485913  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:53.456881 13320 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:53.492492  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:53.506048  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:53.480480 12914 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:53.529841 13129 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:53.534293 13322 consensus_meta.cc:287] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.991s	user 0.014s	sys 0.000s
W20260709 12:19:53.534421  6580 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 2852) took 1322 ms (client timeout 1695 ms). Trace:
W20260709 12:19:53.535033  6580 rpcz_store.cc:269] 0709 12:19:52.212027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:52.212241 (+   214us) service_pool.cc:224] Handling call
0709 12:19:53.534226 (+1321985us) spinlock_profiling.cc:243] Waited 761 ms on lock 0x7b5400265cd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:53.534333 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1749673856}
I20260709 12:19:53.535055 13322 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:19:53.514806 13381 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:53.536298 13381 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:53.490034 13193 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.536947 13009 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:53.511164 13318 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:53.515156  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:53.537438 13193 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:53.538151  7265 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:53.539572 13318 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:53.494385 13365 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:53.541805 13362 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:53.542219 13365 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:53.518797 13364 log.cc:927] Time spent T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 12:19:53.542392 13362 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:53.502830 13353 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:53.544466 13342 raft_consensus.cc:3055] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:19:53.544902 13342 raft_consensus.cc:740] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:53.545073 13353 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:53.546026 13342 consensus_queue.cc:260] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:53.547127 13342 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:53.521833 13242 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:53.536924 13381 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:53.551352  7238 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:53.552856 13276 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:53.536875 13320 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:53.524180 13285 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:53.526443 13255 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:53.527052 13205 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:53.555735 13320 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:53.532136 13129 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:53.515187 13356 log.cc:927] Time spent T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 12:19:53.544981 13007 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:53.507058  6578 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
W20260709 12:19:53.535787  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:53.557761  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:53.558454  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:53.558813  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:53.559177  6578 raft_consensus.cc:1240] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 12:19:53.539656 13193 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:53.493408 13289 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:53.561735 13289 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:53.561870 13322 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:53.525112  6576 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:19:53.552927 13242 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:53.562423 13368 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:53.563023 13288 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.563548 13288 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:53.552599 13336 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:53.564903 13366 log.cc:927] Time spent T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.306s	user 0.003s	sys 0.000s
I20260709 12:19:53.558982 13381 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:53.565878 13288 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:53.566576  6577 raft_consensus.cc:1240] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:53.567425 13381 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:53.569576  6577 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:19:53.569198  6576 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 12:19:53.487498 13313 consensus_meta.cc:287] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.520s	user 0.011s	sys 0.000s
I20260709 12:19:53.570586 13199 consensus_queue.cc:260] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:53.571362 13313 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:53.571738 13199 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:53.572932 13359 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:19:53.575548 13300 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:53.576833  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2843) took 1342 ms. Trace:
I20260709 12:19:53.440258 13371 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:53.577258  6503 rpcz_store.cc:276] 0709 12:19:52.234562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:52.234928 (+   366us) service_pool.cc:224] Handling call
0709 12:19:52.235440 (+   512us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:53.374978 (+1139538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:53.576563 (+201585us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54001effd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:53.576767 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2598712704}
I20260709 12:19:53.578863  6502 raft_consensus.cc:2393] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 18: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:53.579367 13321 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:53.580724  7238 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:53.581358 13371 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:53.581681 13300 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:53.582130 13300 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:53.582474 13300 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:19:53.584381 13387 log.cc:927] Time spent T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:19:53.584692 13327 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.585178 13327 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:53.588058 13327 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:53.589093  6650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3393) took 1502 ms. Trace:
I20260709 12:19:53.525635 13078 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:53.528954 13358 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:53.566030 13336 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:53.591223  6652 raft_consensus.cc:1275] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:19:53.594282  6651 raft_consensus.cc:1275] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:19:53.597213  6580 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:53.508560  6583 rpcz_store.cc:276] 0709 12:19:52.200160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:52.200895 (+   735us) service_pool.cc:224] Handling call
0709 12:19:52.201388 (+   493us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:52.321074 (+119686us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:19:52.478198 (+157124us) log.cc:844] Serialized 14 byte log entry
0709 12:19:52.692640 (+214442us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:52.725972 (+ 33332us) log.cc:844] Serialized 26 byte log entry
0709 12:19:52.726140 (+   168us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:52.726174 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:52.726256 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:53.479920 (+753664us) raft_consensus.cc:1685] finished
0709 12:19:53.479941 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:53.480058 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":106448,"prepare.queue_time_us":185589,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":5007,"raft.queue_time_us":413659,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":52432,"thread_start_us":274294,"threads_started":3,"wal-append.queue_time_us":572567}
I20260709 12:19:53.603274  7267 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:19:53.607636  7267 raft_consensus.cc:1275] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:19:53.609650  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:53.610951  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:53.600868  6650 rpcz_store.cc:276] 0709 12:19:52.087022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:52.087141 (+   119us) service_pool.cc:224] Handling call
0709 12:19:52.087500 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:53.251755 (+1164255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:53.588845 (+337090us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b540019d7d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:53.589036 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2651247360}
I20260709 12:19:53.614609  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:53.615597  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:53.616276  6658 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:19:53.617332  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:53.614675  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:53.618476  6656 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22.
I20260709 12:19:53.614809  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:53.580753 13075 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:53.512033 13197 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:53.677106  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:53.694340  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: true dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:53.677181  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:53.695564  6651 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 14.
I20260709 12:19:53.589412 13335 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:53.691586  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:53.692147  6576 raft_consensus.cc:1240] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 20. Current term is 21. Ops: [20.9-20.9]
I20260709 12:19:53.677676  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:53.678016  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:53.679052  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:53.751652  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:53.751845  6650 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
W20260709 12:19:53.690086 13215 consensus_meta.cc:287] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 1.039s	user 0.007s	sys 0.004s
I20260709 12:19:53.754668 13215 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:53.755034  6576 raft_consensus.cc:1240] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:53.755234  6583 raft_consensus.cc:1240] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 16. Current term is 17. Ops: []
W20260709 12:19:53.756173  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 142037us
I20260709 12:19:53.756928 13215 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:53.757094  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:53.691079 13197 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:53.688138 12878 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:53.758522  6575 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:53.759126 13216 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.750788  6649 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 11.
I20260709 12:19:53.759634 13216 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:53.680454 13386 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.760850 13300 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:53.760929 13386 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:53.761701 13216 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:53.762068  7265 raft_consensus.cc:2393] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 21: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:53.757548 13215 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.005s
I20260709 12:19:53.677714 13255 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:53.764132 13255 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:53.764844  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:53.766145 13255 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:53.768220  7266 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:19:53.763402 13386 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:53.781427  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 374883us
I20260709 12:19:53.825474  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:53.825649  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:53.774335 13288 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.836952  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:53.837777  6651 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:53.836043  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:53.791823  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:53.845108  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:53.847703  6650 raft_consensus.cc:2393] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 18: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:53.845978  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:53.849149  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 57438.4us
I20260709 12:19:53.852737  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:53.853704  6651 raft_consensus.cc:2393] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 21: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:53.857913  6508 raft_consensus.cc:2368] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 13. Current term is 14.
I20260709 12:19:53.860433  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:53.863302  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:53.864957  6582 raft_consensus.cc:1275] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 12:19:53.867349  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:53.868546  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:53.829587 13386 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:53.872021  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:53.872946  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:53.876271  6655 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:19:53.878104  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:53.845192 13351 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:53.885736  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:53.887068  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:53.887140  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:53.887768  6508 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:19:53.888049  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:53.890309  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:53.868062 13360 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:53.898968  6503 raft_consensus.cc:2368] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 13. Current term is 14.
I20260709 12:19:53.892249  7240 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:53.905061  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:53.905400  7240 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:53.884431 13386 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:53.893573 13300 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:53.910496  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:53.893991  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:53.897202  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:53.904173  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:53.908308  6583 raft_consensus.cc:1275] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:19:53.914584  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:53.915302  6508 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:53.847193 13288 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:53.919819  6582 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:53.917356  6658 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:19:53.926388  7240 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:53.928920  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:53.929690  6576 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:19:53.915017 13359 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:19:53.975600  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:53.934437 13300 raft_consensus.cc:2804] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:19:53.934937 13359 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:53.926133 13360 raft_consensus.cc:697] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:53.931350 13058 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:53.929594 13362 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:53.891557 13351 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:53.979286 13300 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:53.989195 13359 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:53.997637 13058 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:54.016542 13300 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:54.028445 13389 log.cc:927] Time spent T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 12:19:54.029039 13058 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:54.042889  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:54.014914 13362 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:54.052043  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 184853us
W20260709 12:19:54.054019  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:54.064011  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:54.074246  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:54.106001  6505 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:19:54.106912  6508 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:19:54.054883 13360 consensus_queue.cc:237] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:54.105149 13288 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:54.125473 13300 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:54.125788 13351 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:54.128666  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:54.129128  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:54.129709  6649 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:19:54.129619  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:54.131342 13342 consensus_meta.cc:287] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.584s	user 0.010s	sys 0.000s
W20260709 12:19:54.132148  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:54.128053 13300 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:54.135991  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:54.144708  7271 raft_consensus.cc:2393] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 12: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:54.144860 13359 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:54.147192 13359 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:54.153653  6649 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 12.
I20260709 12:19:54.155103  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:54.156306  7240 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:54.156665  6649 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 16
W20260709 12:19:54.160553  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 107302us
I20260709 12:19:54.120222 13320 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:54.177764  6508 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:54.187448  6649 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
I20260709 12:19:54.163172 13400 consensus_queue.cc:1059] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 12:19:54.189311 13400 raft_consensus.cc:3055] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:54.197116  7240 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:54.200877  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:54.202859  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:54.202165  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:54.204770  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:54.201620  6580 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:19:54.207525  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:54.208519  6502 raft_consensus.cc:2410] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:19:54.209667  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:54.210566  6575 raft_consensus.cc:2410] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:19:54.205142  6583 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:19:54.190093 13400 raft_consensus.cc:740] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:54.216956  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:54.217960  6582 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:19:54.190145 13359 raft_consensus.cc:2804] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:19:54.223639  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:54.224427  6501 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:19:54.206960 13239 raft_consensus.cc:2804] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:19:54.231698  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:54.230540  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:54.175796 13320 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:54.219645 13255 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:54.236575  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:54.241179  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:54.241890  6507 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:19:54.226970 13359 raft_consensus.cc:697] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 12 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:54.242791  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:54.243366  6507 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:19:54.243543 13359 consensus_queue.cc:237] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:54.237804  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:54.245666  6582 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:19:54.246752  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:54.247372  6582 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:19:54.248768 12492 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:54.249697 13342 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:54.250270 13342 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:54.252529 13342 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:54.254310  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:54.260004 13392 log.cc:927] Time spent T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
W20260709 12:19:54.261466 13399 log.cc:927] Time spent T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 12:19:54.265081  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:54.264174  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:54.268306  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:54.267197  6576 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:54.273311  6549 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:54.281679  6549 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:54.273576  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:54.284058  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:54.284538 13329 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:54.285022 13329 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:54.285270  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:54.287034  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:54.285904  6576 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
I20260709 12:19:54.285455 13329 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:54.236102 13239 raft_consensus.cc:697] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 16 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:54.289851  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:54.285904 12878 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:54.290406 13310 consensus_queue.cc:1048] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 12:19:54.293269 13336 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:19:54.300848 13199 consensus_meta.cc:287] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.729s	user 0.005s	sys 0.009s
I20260709 12:19:54.317476  7269 raft_consensus.cc:1240] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:54.319669  6579 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:19:54.321282  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:54.321867  6579 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:19:54.322638  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:54.323195  6579 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22.
I20260709 12:19:54.326748 13215 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:54.327380 13215 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:54.327720 13063 consensus_queue.cc:1059] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:54.297338 13138 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:54.330405 13199 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:54.338959 13138 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:54.339354 13199 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:54.341444 13138 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:54.328914  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:54.342458  6582 raft_consensus.cc:3060] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:54.268700 13255 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:54.313408  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:54.346423 12492 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:54.335755 13215 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:54.291265 13400 consensus_queue.cc:260] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:54.331933 13058 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:54.336984 13197 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:54.336982 13063 raft_consensus.cc:3055] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:54.324025  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:54.314342 13336 raft_consensus.cc:697] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:54.341917 13199 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:54.329793 13391 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:54.299065 12878 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:54.358558  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:54.368245  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 115376us
I20260709 12:19:54.347055 12492 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:54.390085  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:19:54.392633  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 67361.3us
I20260709 12:19:54.393515  6475 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:54.406080  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:54.391682  7238 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:54.358072 13063 raft_consensus.cc:740] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:54.368777 13138 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:54.474766  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 81334.3us
I20260709 12:19:54.477802  6475 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:54.416585 13276 consensus_queue.cc:1059] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.077s
I20260709 12:19:54.416049 13300 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:54.388797 13320 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:54.350818 13239 consensus_queue.cc:237] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:54.495692  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:54.496699  6579 raft_consensus.cc:2393] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 20: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:54.497682  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:19:54.498984  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:54.499815  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:54.500861  6579 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:19:54.390473 13400 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:54.485081 13197 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:54.529843  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:54.532500  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 54809.3us
I20260709 12:19:54.537853  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:54.539225  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:54.540311  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:54.542665  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:54.544247  6579 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:54.545969 13329 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:54.501298 13199 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:19:54.546270  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:54.549053  6579 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:19:54.549670 13386 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:54.551177  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:54.554374  6550 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:54.378321 13391 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:54.560483  6473 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:54.566599  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:54.572525  6473 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:54.513747 13138 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:54.549042 13199 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:54.585709  6579 raft_consensus.cc:1275] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:19:54.571353 13298 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:54.585191 13063 consensus_queue.cc:260] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:54.416276 13371 consensus_queue.cc:1059] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.112s
I20260709 12:19:54.592821  6579 raft_consensus.cc:1275] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 12:19:54.594759  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:54.599947  6579 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 13. (term mismatch)
I20260709 12:19:54.600401  6579 pending_rounds.cc:85] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 12
I20260709 12:19:54.600685  6579 pending_rounds.cc:107] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 22.13
I20260709 12:19:54.607422  6579 raft_consensus.cc:2889] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:19:54.611174  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:54.614579 13329 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:54.596225 13063 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:54.534891 13336 consensus_queue.cc:237] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:54.584275 13215 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
W20260709 12:19:54.690851  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:54.598012 13396 log.cc:927] Time spent T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.451s	user 0.007s	sys 0.003s
I20260709 12:19:54.586199 13199 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:54.694283  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 156387us
I20260709 12:19:54.592455 13412 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:19:54.694621  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 161242us
I20260709 12:19:54.598886 13320 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:54.703480  6582 raft_consensus.cc:2468] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
I20260709 12:19:54.598027 13407 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:19:54.701565  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 286818us
I20260709 12:19:54.598042 13416 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:54.766592  7272 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38562 (request call id 1122) took 1795 ms. Trace:
I20260709 12:19:54.597684 13372 consensus_queue.cc:1059] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 19.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.173s
I20260709 12:19:54.769707  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:54.765403  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:54.768553  7272 rpcz_store.cc:276] 0709 12:19:52.959318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:52.959560 (+   242us) service_pool.cc:224] Handling call
0709 12:19:52.979840 (+ 20280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:53.415228 (+435388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:53.415261 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:53.415299 (+    38us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:19:53.415327 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:54.754008 (+1338681us) spinlock_profiling.cc:243] Waited 435 ms on lock 0x7b54005ddc54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:54.754059 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:54.754228 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1000797568}
I20260709 12:19:54.704957 13374 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:19:54.793016  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:54.702682 13255 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:54.794648  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 266331us
I20260709 12:19:54.766166 12492 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:54.807054  6622 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:54.789328 13409 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:54.764626 13407 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:54.832265  6622 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:54.781728 13371 consensus_queue.cc:1059] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 12:19:54.804352 13423 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:19:54.820034  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:54.799646 13403 raft_consensus.cc:3055] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:54.796106 13320 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:54.768690 13412 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:54.840269  6506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2868) took 1428 ms. Trace:
W20260709 12:19:54.820828  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 248366us
I20260709 12:19:54.805475 13417 consensus_queue.cc:1048] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.032s
W20260709 12:19:54.840924  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 286605us
I20260709 12:19:54.885684  6550 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:54.842576 13423 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:54.845000  6506 rpcz_store.cc:276] 0709 12:19:53.411348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:53.530754 (+119406us) service_pool.cc:224] Handling call
0709 12:19:53.531146 (+   392us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:53.588976 (+ 57830us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:19:53.589015 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:53.688421 (+ 99406us) log.cc:844] Serialized 52 byte log entry
0709 12:19:53.929312 (+240891us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:19:53.929355 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:53.929692 (+   337us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:54.542983 (+613291us) spinlock_profiling.cc:243] Waited 1.2 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:54.839942 (+296959us) raft_consensus.cc:1685] finished
0709 12:19:54.839967 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:54.840207 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57364,"raft.queue_time_us":68320,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":83459,"spinlock_wait_cycles":2766592,"thread_start_us":240496,"threads_started":1,"wal-append.queue_time_us":352404}
I20260709 12:19:54.882611 13409 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:19:54.925158  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:54.942982  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:54.947932  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:54.948887  6650 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:19:54.950150  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:54.953341  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:54.954510  6650 raft_consensus.cc:2393] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 16: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:54.969461  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:54.969661  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:54.970363  6657 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:19:54.970963 13009 raft_consensus.cc:3055] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:54.971513 13009 raft_consensus.cc:740] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:54.971961  6653 raft_consensus.cc:1240] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 11. Current term is 12. Ops: []
I20260709 12:19:54.972397  6650 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:19:54.972604  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:54.971086  6654 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:54.973383  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:54.908195 12492 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:19:54.974162  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:54.971525  6657 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:54.839869 13403 raft_consensus.cc:740] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:54.972401 13009 consensus_queue.cc:260] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:54.883715 13320 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:54.986215 13320 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:54.983170  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 280480us
I20260709 12:19:54.981292 12492 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:54.987197 12492 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:54.850709 13391 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:55.004608  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 133156us
I20260709 12:19:54.940718 13205 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:54.922120 13138 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:54.935109 13423 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:19:55.013912  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 181703us
I20260709 12:19:55.021422  6622 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:55.021804  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:55.023211  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:55.024202  6652 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:55.029595 13407 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:54.974112 13329 consensus_queue.cc:1059] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:19:55.035131  6622 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:55.048804  6475 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:55.037803 13199 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:55.052702  6475 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:55.055141  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:55.050240 13215 consensus_queue.cc:1048] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:55.073812  6622 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:55.038524 13407 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:55.025055 13138 consensus_queue.cc:1048] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 12:19:55.050787 13199 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:55.083425  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:55.083287  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:55.037487 13391 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:55.114450 13391 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:55.082583 13205 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:55.117409 13391 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:55.118139 13313 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:55.118873 13313 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:55.123934  6505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 3013) took 1291 ms. Trace:
I20260709 12:19:55.124363  6505 rpcz_store.cc:276] 0709 12:19:53.832405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:53.912227 (+ 79822us) service_pool.cc:224] Handling call
0709 12:19:54.104648 (+192421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:54.108091 (+  3443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:55.123740 (+1015649us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b540017cad0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:55.123874 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":411029376}
I20260709 12:19:55.125351  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:55.126644  6622 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:55.128580 13409 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:55.128598 13329 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:55.129120 13409 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:55.129593 12878 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:55.129832 13437 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:55.130324 13437 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:55.130750 13437 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:55.080665 13419 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:55.065433 13320 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:55.133147 13320 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:55.101315 13428 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:55.134002 13428 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:55.134434 13428 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:55.135383 13320 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:55.129607 13409 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:55.032662 13424 consensus_queue.cc:1048] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
W20260709 12:19:55.101229 13426 log.cc:927] Time spent T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:19:55.138072 13362 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:55.138883  6504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2864) took 1729 ms. Trace:
I20260709 12:19:55.139252  6504 rpcz_store.cc:276] 0709 12:19:53.409060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:53.409179 (+   119us) service_pool.cc:224] Handling call
0709 12:19:53.410437 (+  1258us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:53.512327 (+101890us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:53.512379 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:54.527408 (+1015029us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b2c00000c88. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:19:54.536271 (+  8863us) log.cc:844] Serialized 26 byte log entry
0709 12:19:54.798703 (+262432us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:54.798753 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:55.122846 (+324093us) spinlock_profiling.cc:243] Waited 93.7 ms on lock 0x7b540001b8d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:55.122928 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:55.138668 (+ 15740us) raft_consensus.cc:1685] finished
0709 12:19:55.138687 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:55.138819 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7671,"raft.queue_time_us":31923,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":92,"spinlock_wait_cycles":252487808,"thread_start_us":208411,"threads_started":1,"wal-append.queue_time_us":356999}
I20260709 12:19:55.105096 13386 raft_consensus.cc:3055] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:55.141656 13386 raft_consensus.cc:740] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:55.142525 13386 consensus_queue.cc:260] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:55.143033  6548 leader_election.cc:336] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.517s (SENT)
I20260709 12:19:55.143733 13386 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:55.145282  6649 raft_consensus.cc:1275] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:19:55.090483  6652 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 23.
I20260709 12:19:55.111358 13288 consensus_queue.cc:1059] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 12:19:55.146520 13288 raft_consensus.cc:3055] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:55.146561  6550 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:55.147013 13288 raft_consensus.cc:740] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:55.147933  6550 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:55.113754 13199 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:55.176018  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:55.129350 13329 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:55.098912 13434 log.cc:927] Time spent T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:19:55.132563 13435 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:19:55.141259 13397 log.cc:927] Time spent T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 1.074s	user 0.003s	sys 0.000s
I20260709 12:19:55.147809 13288 consensus_queue.cc:260] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:55.109196 13412 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:55.113340 13298 consensus_meta.cc:287] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.521s	user 0.009s	sys 0.001s
I20260709 12:19:55.164086 13009 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:55.152776  7269 raft_consensus.cc:1240] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 12 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:19:55.161965 13429 raft_consensus.cc:3055] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:55.156531  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:55.168735  6651 raft_consensus.cc:1275] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
W20260709 12:19:55.206645  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 154035us
I20260709 12:19:55.157184  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:55.176612  6649 raft_consensus.cc:1275] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:19:55.152977  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:55.152067 12492 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:55.178593  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:55.179210  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:55.180016  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:55.128173  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:55.184223  6550 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:55.190403  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:55.216883  6657 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
W20260709 12:19:55.169346 13395 log.cc:927] Time spent T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 1.090s	user 0.005s	sys 0.000s
I20260709 12:19:55.193847  6584 raft_consensus.cc:3055] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:19:55.207443  6475 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:55.221596  6584 raft_consensus.cc:740] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:19:55.103679 13420 log.cc:927] Time spent T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.513s	user 0.001s	sys 0.001s
I20260709 12:19:55.203086  6583 raft_consensus.cc:1275] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:19:55.181806 13199 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:55.165365 13058 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:55.188155 12878 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:19:55.225453  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:55.224606  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 96505.7us
I20260709 12:19:55.227404  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:19:55.228163  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 84208.5us
I20260709 12:19:55.218770  6550 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:55.187291 13320 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:55.222383  6584 consensus_queue.cc:260] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:55.229911  6584 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:19:55.196663 13400 consensus_meta.cc:287] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.672s	user 0.012s	sys 0.000s
I20260709 12:19:55.231940  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:55.222383  6475 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:55.234508  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 226467us
I20260709 12:19:55.236982 12492 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:55.238848  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:55.241989  7267 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:55.240137  7238 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:55.239804  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:55.246138  6622 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:55.223368 13355 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:55.248907  6654 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:55.220347 13429 raft_consensus.cc:740] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:55.256912  6475 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:55.259472  6475 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:55.188295 13409 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:55.253330 12492 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:55.262730 12492 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:55.229650 12878 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:55.235311 13288 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:55.264591  6475 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:55.237417 13298 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:55.255914  6550 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:55.247803  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:55.237715 13412 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:55.245885 13063 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:19:55.169670 13403 consensus_queue.cc:260] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.9, Last appended by leader: 9, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:19:55.267731  6581 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:19:55.237577 13427 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:19:55.241003 13407 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:55.226825 13359 consensus_queue.cc:1048] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 12:19:55.254210 13313 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:55.291006  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: true dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:55.252871 13335 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:55.272149  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 2696) took 1580 ms. Trace:
I20260709 12:19:55.331604 12492 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:55.272596 13412 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:55.350812  6577 rpcz_store.cc:276] 0709 12:19:53.691957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:53.692079 (+   122us) service_pool.cc:224] Handling call
0709 12:19:53.692440 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:53.692881 (+   441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:53.692915 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:53.693486 (+   571us) log.cc:844] Serialized 26 byte log entry
0709 12:19:53.978009 (+284523us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:53.978057 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:53.978131 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:55.140679 (+1162548us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:55.266977 (+126298us) raft_consensus.cc:1685] finished
0709 12:19:55.266999 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:55.272084 (+  5085us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":55137,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":9734,"spinlock_wait_cycles":45326592,"thread_start_us":284074,"threads_started":1,"wal-append.queue_time_us":332931}
I20260709 12:19:55.268905 13430 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:19:55.312945  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:55.298947  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:55.334280  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2854) took 1894 ms. Trace:
I20260709 12:19:55.334380  6622 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:55.256096 13355 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:55.335647  6584 raft_consensus.cc:1275] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 12:19:55.384928  6584 pending_rounds.cc:85] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 10
I20260709 12:19:55.385187  6584 pending_rounds.cc:107] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 12:19:55.385416  6584 raft_consensus.cc:2889] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:55.255398 13400 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:19:55.336943  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 68273.5us
I20260709 12:19:55.339040  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:55.387451  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 53143.7us
I20260709 12:19:55.342598  7267 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
W20260709 12:19:55.340245  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 75707us
I20260709 12:19:55.262620 13416 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:55.389179  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:55.265209 12878 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:55.250329 13215 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.084s
I20260709 12:19:55.330806 13314 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:55.262508 13424 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:55.374902 13412 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:55.278255  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:55.359579 13436 log.cc:927] Time spent T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 12:19:55.344174 13371 consensus_queue.cc:1059] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 12:19:55.350100 13403 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:55.365679 13409 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:55.334991 13427 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:55.429843 13427 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:55.334957 13320 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:55.374317  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:55.441118 13320 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:55.377149  6575 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
W20260709 12:19:55.443928 13441 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:19:55.336802 13351 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:55.445286 13351 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:55.445691 13351 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:55.334703 13342 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:55.448664 13342 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:55.380043  6582 raft_consensus.cc:2376] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19. Re-sending same reply.
I20260709 12:19:55.346101 13419 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:55.455142 13419 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:55.455674 13419 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:55.454221  6582 raft_consensus.cc:1240] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:19:55.388496  6475 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:55.384708 13239 consensus_queue.cc:1048] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 12:19:55.386341  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:55.463819  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 56951.7us
I20260709 12:19:55.380388  6578 rpcz_store.cc:276] 0709 12:19:53.439423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:53.559894 (+120471us) service_pool.cc:224] Handling call
0709 12:19:53.560180 (+   286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:53.589180 (+ 29000us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:53.589233 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:53.602965 (+ 13732us) log.cc:844] Serialized 26 byte log entry
0709 12:19:53.927611 (+324646us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:19:53.927652 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:53.927720 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:55.168668 (+1240948us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:55.274220 (+105552us) raft_consensus.cc:1685] finished
0709 12:19:55.274252 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:55.334220 (+ 59968us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10202,"raft.queue_time_us":137,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":10808,"spinlock_wait_cycles":24358784,"thread_start_us":238783,"threads_started":1,"wal-append.queue_time_us":377542}
I20260709 12:19:55.464821  6582 raft_consensus.cc:1240] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 20. Current term is 21. Ops: [20.9-20.9]
I20260709 12:19:55.392992  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:55.309128  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:55.395195  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 145919us
I20260709 12:19:55.389519 13355 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:55.469587  6578 raft_consensus.cc:1240] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:55.402088 13372 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:55.379042 13430 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:55.389859 13360 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:55.389664 13400 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:55.405155  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:55.408674  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:55.409888  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:55.403337 13391 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:55.410641 13421 raft_consensus.cc:3055] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:55.411294  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:55.412505  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:55.413528  6581 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 2691) took 1920 ms (client timeout 1562 ms). Trace:
I20260709 12:19:55.424518  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:55.427819  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:55.428896 13371 raft_consensus.cc:3055] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:55.429164 13417 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:55.436678  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:55.441336  6576 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:55.405936 13416 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 12:19:55.293759  6550 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:55.509846  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 216227us
W20260709 12:19:55.509931  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:55.262351 13362 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:55.350872  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:55.461692  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 73248.7us
W20260709 12:19:55.527120  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:55.467595  6582 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:55.466306  6580 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 23.
I20260709 12:19:55.469650  6502 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:19:55.474015  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:55.478597  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:55.479239 13342 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:55.265303 13425 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.138s
I20260709 12:19:55.482369  6622 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:55.488427 13320 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:55.483845  7271 raft_consensus.cc:1240] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:55.491919 13371 raft_consensus.cc:740] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:19:55.489661  6581 rpcz_store.cc:269] 0709 12:19:53.493337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:53.510763 (+ 17426us) service_pool.cc:224] Handling call
0709 12:19:55.413373 (+1902610us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b54000366d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:55.413469 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3704212736}
I20260709 12:19:55.469888 13440 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:55.390456 13429 consensus_queue.cc:260] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:55.516063  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:55.526943  6475 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:55.495646 13391 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:55.394663 13424 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:55.476856  6473 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:55.489002 13400 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:55.484931 13430 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:55.434712 13423 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:55.526306  7270 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
W20260709 12:19:55.533494  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 51196.7us
I20260709 12:19:55.537966 13063 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:55.487816  6501 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:19:55.595094  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:55.487030 13421 raft_consensus.cc:740] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:55.533669 13313 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:55.598785  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:55.599108  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:55.600240  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:55.606400  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 129605us
I20260709 12:19:55.607323  6473 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:55.611373  6548 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:55.634898  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 107714us
I20260709 12:19:55.639269  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:55.537220 13342 consensus_queue.cc:1048] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 12:19:55.564980 13424 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:55.642657  6473 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:55.644912  6503 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:55.645593  6550 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:55.650266  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 43532.8us
I20260709 12:19:55.651276  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:55.654947  6550 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:55.537739 13300 consensus_queue.cc:1048] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.075s
I20260709 12:19:55.606505 13407 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:55.529423 13417 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:55.660320  6473 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:55.667934  6473 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:55.565562 13442 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:55.604655 13313 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:19:55.649349 13396 log.cc:927] Time spent T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:19:55.651465 13362 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:19:55.553747 13437 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:55.696533 13437 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:55.595203 13215 consensus_queue.cc:1059] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 12:19:55.642491 13423 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:55.648391 13298 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:55.674819  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:55.675568 13459 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:55.672708 13442 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:55.680698 13313 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:55.699100  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 44222.1us
I20260709 12:19:55.656128 13197 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:55.671345 13276 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:55.702253  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:55.658855 13429 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:55.702987  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:55.659500 13425 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:19:55.700691  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:55.730557 12492 consensus_queue.cc:1048] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:55.732028  6657 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:55.643070 13428 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:55.732811  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:55.738396  6654 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:19:55.734845  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:55.729771  6550 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:55.745193  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 132866us
I20260709 12:19:55.750703 12492 consensus_queue.cc:1059] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:19:55.746464  6653 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22.
I20260709 12:19:55.704507 13409 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:55.703444 13336 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:55.755021  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:55.730914 13407 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:55.778307  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:55.725052 13442 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:19:55.779841  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 111974us
I20260709 12:19:55.781699  6652 raft_consensus.cc:1240] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 14. Current term is 15. Ops: []
I20260709 12:19:55.782482  6473 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:55.742872 13425 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:55.764571 13298 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:55.766254 13063 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:19:55.791807  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:55.778853 13374 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:55.776507 13407 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:55.675750 13421 consensus_queue.cc:260] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:55.808348  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 166011us
I20260709 12:19:55.780737 13423 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:55.791350 13425 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:55.764837 13418 consensus_queue.cc:1059] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.058s
I20260709 12:19:55.787039 13438 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:55.790894 13298 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:55.794927 13063 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:55.728770 13459 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:55.801014 13432 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:55.807578 13407 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:55.812987 13276 raft_consensus.cc:697] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 19 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:55.825201 13298 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:55.827674 13063 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:55.813221 13205 consensus_queue.cc:1059] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 12:19:55.827804 13317 consensus_queue.cc:1048] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 12:19:55.830422 13423 consensus_queue.cc:1048] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 12:19:55.848834 13418 raft_consensus.cc:3055] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:55.877521 13418 raft_consensus.cc:740] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:55.869256  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:55.869997  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:55.884852  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:55.890441  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:55.900513  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:55.904947  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:55.909066  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:55.908483 12492 consensus_queue.cc:1243] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer f2b06aebe97249a4877d798b8118ece4 log is divergent from this leader: its last log entry 20.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 12:19:55.876852 13457 log.cc:927] Time spent T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 12:19:55.919490  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:55.903021 13386 consensus_meta.cc:287] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.759s	user 0.011s	sys 0.001s
W20260709 12:19:55.928880 13413 log.cc:927] Time spent T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 1.347s	user 0.008s	sys 0.005s
W20260709 12:19:55.932813  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:55.935833  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:55.941098 13448 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.347s	user 0.002s	sys 0.000s
I20260709 12:19:55.942286 13459 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:55.858563 13428 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:55.892495 13391 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:55.944510 13418 consensus_queue.cc:260] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:55.949066 13288 consensus_meta.cc:287] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.677s	user 0.009s	sys 0.001s
I20260709 12:19:55.950138 13355 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.005251  6475 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:56.006253  6475 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:55.950822 13421 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:55.951864 13009 consensus_meta.cc:287] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.729s	user 0.009s	sys 0.000s
W20260709 12:19:55.951993 13403 consensus_meta.cc:287] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.538s	user 0.009s	sys 0.000s
I20260709 12:19:55.859467 13423 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:55.952857  6549 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:55.917191 13351 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:55.954674 13371 consensus_queue.cc:260] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:55.959028 13336 raft_consensus.cc:3055] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:55.857600 13317 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:55.963392 13417 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:55.849397 13432 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.115s
I20260709 12:19:55.952445 13276 consensus_queue.cc:237] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:55.965567 13362 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:19:55.967914 13427 consensus_meta.cc:287] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.537s	user 0.010s	sys 0.000s
I20260709 12:19:56.040248 13477 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:56.041260 13425 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.042802 13418 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:55.922634  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:56.001612  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:56.004508  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:55.972182 13199 consensus_meta.cc:287] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.729s	user 0.005s	sys 0.003s
I20260709 12:19:56.008209 13460 consensus_queue.cc:1059] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
W20260709 12:19:55.975857 13419 consensus_meta.cc:287] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.520s	user 0.012s	sys 0.000s
I20260709 12:19:56.109341  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:19:56.077589  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 124806us
I20260709 12:19:56.087900  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:56.131966  6504 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
W20260709 12:19:56.132210  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:56.087446  6652 raft_consensus.cc:1275] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:19:56.114095  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:56.096984  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:56.134034  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:56.048269  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:56.139027  6657 raft_consensus.cc:1240] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:19:56.038763  6583 raft_consensus.cc:2368] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 19. Current term is 20.
I20260709 12:19:56.039304 13462 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:19:56.008127 13355 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:56.048820  6655 raft_consensus.cc:2410] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 21 because replica has last-logged OpId of term: 20 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 8.
I20260709 12:19:56.062068  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:56.066649  6475 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:56.295671  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 229121us
I20260709 12:19:56.297510  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:56.160245  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:56.159060  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:56.158169  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:56.158681  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:56.170346  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:56.035229 13440 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:56.301389  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:56.191818  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:56.306380  6580 raft_consensus.cc:2410] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:19:56.193742  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:56.190536  6579 raft_consensus.cc:2410] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:19:56.049440 13329 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:56.198731  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 43809.3us
W20260709 12:19:56.474041  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 179043us
W20260709 12:19:56.477041  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 393619us
W20260709 12:19:56.477878  7238 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 2.220s (SENT) [suppressed 26 similar messages]
I20260709 12:19:56.199040  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:56.199915  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:56.196130  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:56.082782 13425 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:56.094616 13320 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:56.202910  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:56.161438  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:56.194156 13407 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.200663 13409 raft_consensus.cc:697] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 19 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:56.072234 13424 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.237311 13467 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.094s
I20260709 12:19:56.203064 13474 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:56.105643 13427 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:56.200541 13435 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.006s	sys 0.016s
I20260709 12:19:56.199664 13465 consensus_queue.cc:1059] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.080s
I20260709 12:19:56.189756 13458 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:56.077245 13371 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:56.077996 13314 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:19:56.074925 13403 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:56.040326 13310 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:56.239135  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:56.093523 13298 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.105459 13317 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.160396  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:56.518805  6503 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:56.519863 13298 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:56.104828 12878 consensus_meta.cc:287] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.707s	user 0.008s	sys 0.008s
I20260709 12:19:56.076665 13336 raft_consensus.cc:740] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:19:56.257404 13313 consensus_meta.cc:287] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.518s	user 0.007s	sys 0.003s
W20260709 12:19:56.254266 13197 consensus_meta.cc:287] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.512s	user 0.010s	sys 0.000s
W20260709 12:19:56.255486 13400 consensus_meta.cc:287] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.607s	user 0.007s	sys 0.003s
I20260709 12:19:56.510504 13314 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:19:56.255935 13430 consensus_meta.cc:287] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.613s	user 0.006s	sys 0.001s
I20260709 12:19:56.407249  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:56.497848 13320 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:56.259179 13374 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:56.455426  7271 raft_consensus.cc:1240] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:56.408048  7273 raft_consensus.cc:2393] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 19: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:56.456002  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:56.077798 13362 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.076570 13423 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.055608 13063 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.257566 13351 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:56.458367  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:56.142855 13199 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.459254  7265 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 13 index: 14. (index mismatch)
I20260709 12:19:56.533128 13009 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:56.458987 13288 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:56.248293 13442 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.299494  6656 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:19:56.158967 13462 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:56.482100  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:56.555447  6508 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:19:56.482985 13149 consensus_queue.cc:1048] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:56.484463  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:56.491987  6502 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 21.
I20260709 12:19:56.492090  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:56.492275  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:56.494939  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:56.481156 13215 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:56.502257  6649 raft_consensus.cc:1240] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:56.503377  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:56.496970  7238 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:56.495936 13429 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:19:56.501837 13435 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.897s	user 0.006s	sys 0.022s
I20260709 12:19:56.591198 13407 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:56.501428 13474 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.591777 13063 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:56.592640 13465 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:56.542855 13430 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.517668  6473 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:56.593101 13474 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:56.536177 13400 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:56.593865  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 76287.4us
I20260709 12:19:56.594527 13430 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:56.594619  6473 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:56.595315  6650 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 12:19:56.595889  6473 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:56.535144  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:56.598875 13374 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:56.599543 13374 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:56.600741  6473 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:56.601495 13374 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:56.601833  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:56.437990  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 140541us
I20260709 12:19:56.602483  6508 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:56.603258  6473 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:56.603472  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:56.538141  6500 raft_consensus.cc:2368] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 13. Current term is 14.
I20260709 12:19:56.604135 13374 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:56.530526 13355 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:56.605217 13355 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.605804 13289 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:56.606446 13289 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.607370 13355 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:56.519863 13317 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:56.511025 13458 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.159500 13419 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.609650 13458 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:56.548024  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:56.610353 13442 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:56.610549 13400 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:56.536099 13276 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:56.611173 13400 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:56.611387 13276 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:19:56.611912  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:56.613329  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:56.613447 13276 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:56.537817 13314 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:56.615026  6507 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 23 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 12:19:56.536016 13197 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:56.516081 13403 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:56.551779  6550 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:56.618189 13403 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:56.618611  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 66911us
I20260709 12:19:56.605232  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:56.620514  6500 raft_consensus.cc:2393] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 14: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:56.621477  6500 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:56.548555 13482 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:56.549604 13484 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:56.609651 13419 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:56.569051  6509 raft_consensus.cc:2376] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19. Re-sending same reply.
I20260709 12:19:56.625435  6509 raft_consensus.cc:1240] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:56.530979 12878 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.533768 13313 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.570711 13462 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:56.628650  6578 raft_consensus.cc:2393] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 20: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:56.629669  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 2745) took 1156 ms. Trace:
I20260709 12:19:56.629848 13313 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:56.629983  6578 rpcz_store.cc:276] 0709 12:19:55.473503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.473668 (+   165us) service_pool.cc:224] Handling call
0709 12:19:56.629494 (+1155826us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400031e54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:56.629616 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2525847424}
I20260709 12:19:56.630654 13313 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:56.631130 13313 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:56.631485 13313 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:56.589010  6658 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 24: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:56.633605 12878 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:56.635535  6501 raft_consensus.cc:1275] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:19:56.636304  6584 raft_consensus.cc:1240] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:56.618201 13320 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:56.556622 13298 consensus_queue.cc:1059] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.081s
I20260709 12:19:56.547786 13409 consensus_queue.cc:237] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.638361 13298 raft_consensus.cc:3055] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:56.638857 13298 raft_consensus.cc:740] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:56.641990 13423 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:56.644107 13427 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:56.568928  6582 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
W20260709 12:19:56.589177  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 92295.3us
I20260709 12:19:56.645710  6582 raft_consensus.cc:1240] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:56.647791 13459 raft_consensus.cc:3055] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:19:56.647670 13149 consensus_queue.cc:1059] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:19:56.648269 13459 raft_consensus.cc:740] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:56.649294 13359 raft_consensus.cc:3055] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:56.589751  6505 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:56.649791 13359 raft_consensus.cc:740] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:56.649864  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 2731) took 1316 ms. Trace:
W20260709 12:19:56.650616  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:56.650291  6583 rpcz_store.cc:276] 0709 12:19:55.333297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.408429 (+ 75132us) service_pool.cc:224] Handling call
0709 12:19:56.649664 (+1241235us) spinlock_profiling.cc:243] Waited 524 ms on lock 0x7b540034cbd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:56.649806 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1204319616}
I20260709 12:19:56.650767 13359 consensus_queue.cc:260] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:56.570075 13009 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:56.256953 13412 consensus_meta.cc:287] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.850s	user 0.003s	sys 0.007s
I20260709 12:19:56.652572 13412 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.653259 13009 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:56.597685 13463 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:56.583336 13329 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.015s
I20260709 12:19:56.656535 13412 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:56.656924  6579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 2730) took 1350 ms. Trace:
I20260709 12:19:56.657316  6579 rpcz_store.cc:276] 0709 12:19:55.307104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.308766 (+  1662us) service_pool.cc:224] Handling call
0709 12:19:56.656722 (+1347956us) spinlock_profiling.cc:243] Waited 723 ms on lock 0x7b54003ab954. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:56.656863 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1662735104}
I20260709 12:19:56.649295 13459 consensus_queue.cc:260] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:56.601950 13477 consensus_queue.cc:1059] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:56.658633 13477 raft_consensus.cc:3055] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:56.652133 13359 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:56.658993 13477 raft_consensus.cc:740] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:56.661612  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:56.663825  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 60632.4us
I20260709 12:19:56.652032 13438 raft_consensus.cc:3055] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:56.668573 13438 raft_consensus.cc:740] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:56.604210  6508 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:19:56.609943  6506 raft_consensus.cc:2393] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 20: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:56.539191  6503 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:19:56.619638  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:56.622640 13482 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.623517 13484 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.533643 13336 consensus_queue.cc:260] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.627445 13362 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:56.522977 13310 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:56.651372  6581 raft_consensus.cc:1240] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 12:19:56.590436 13429 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.671654  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:56.690914  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:56.671936  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:56.550118  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:56.604274  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:56.673095  7238 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:56.673990  6500 raft_consensus.cc:1240] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:56.674506  6658 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 12:19:56.643694 13314 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:56.694628  6583 raft_consensus.cc:2393] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 15: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:19:56.696111  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 91905.4us
I20260709 12:19:56.696477  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:56.659788 13477 consensus_queue.cc:260] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.649305 13424 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:56.698923  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:56.710335  6582 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:56.682252  6505 raft_consensus.cc:2410] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:19:56.658644 13459 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:56.697948  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:56.702571  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:56.707412  6473 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:56.676692 13458 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:56.709507  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:56.714342  6658 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:19:56.718470  6579 raft_consensus.cc:2393] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 20: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:56.684283 13329 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:56.720276  6473 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:56.718528  6392 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 6f94b1cd286648a9b82d5abf69bb86d5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 496c7c3792cd4587b4dbeba435f4be1e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 17 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:19:56.639974 13298 consensus_queue.cc:260] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:56.723063  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:56.700736 13289 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:56.714910 13463 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:56.706984 13063 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:56.773459  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:56.789722  6579 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:19:56.775892  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:56.719096 13362 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:56.720196 13336 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:56.794463  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:56.795285  6658 raft_consensus.cc:2393] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 20: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:56.802810  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:56.804798  6658 raft_consensus.cc:2393] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 20: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:56.731205 13477 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:56.814059  6582 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 23.
I20260709 12:19:56.774107 13419 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:56.816280  6473 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:56.722849 13458 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.722126 13215 consensus_queue.cc:1059] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.085s
I20260709 12:19:56.834800  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:56.794600 13362 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:56.787412 13483 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:56.843930  6658 raft_consensus.cc:2393] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 17: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:56.835628 13298 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:56.843995 13424 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:19:56.858114  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:56.858759  6656 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:56.859215  7273 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:19:56.861294 13310 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:56.864461 13149 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:56.866472  6656 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 23.
I20260709 12:19:56.867741  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:56.868129 13199 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:56.868830  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:56.730499 13329 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:56.869263  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:56.869728  6656 raft_consensus.cc:2393] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 18: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:56.870532  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:56.871078 13329 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:56.871261  6656 raft_consensus.cc:2393] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 22: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:56.831974 13371 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:56.872149  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:56.872900  6656 raft_consensus.cc:2393] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 20: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:19:56.875394 13418 consensus_meta.cc:287] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.792s	user 0.010s	sys 0.000s
I20260709 12:19:56.795717 13463 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.877302 13314 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:56.878618 13463 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:56.841311 13362 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:56.813848 13438 consensus_queue.cc:260] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.881018 13438 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:56.882062 13427 consensus_queue.cc:1175] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "496c7c3792cd4587b4dbeba435f4be1e" responder_term: 19 status { last_received { term: 19 index: 10 } last_committed_idx: 10 last_received_current_leader { term: 19 index: 10 } }
I20260709 12:19:56.883702 13429 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:56.885471 13482 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:56.893708  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:56.894730  7267 raft_consensus.cc:2393] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 17: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:56.895241  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:56.897095  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:56.897945 13215 raft_consensus.cc:3055] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:56.898346 13215 raft_consensus.cc:740] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:56.899294 13215 consensus_queue.cc:260] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.900586  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:56.900466 13483 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:56.901262  7274 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:19:56.901986 13482 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:56.902695 13483 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:56.902518 13482 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.813259 13138 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:56.925800  6623 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:56.926898  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:56.927616  7270 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:56.930262  6583 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:56.803081 13063 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.848057 13484 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:56.938345  6622 leader_election.cc:400] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 19. Current term is 20.
I20260709 12:19:56.938723  6622 leader_election.cc:403] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:19:56.939926  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:56.949795 13289 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:56.958340 13288 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:56.961715 13458 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:56.926702 13424 raft_consensus.cc:697] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 23 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:56.958438 13289 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:56.930764 13215 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:56.963162  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:56.967159  7270 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:19:56.968976  6650 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 18.
I20260709 12:19:56.970494  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:56.971208  6650 raft_consensus.cc:2393] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 18: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:56.970685  7240 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:56.979240  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:56.979955  6548 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:56.985232  6650 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:56.984211  6583 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:19:56.952409 13329 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:56.934661 13138 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.965143 13199 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:56.991628  6548 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:56.963558 13317 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:56.968775 13289 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:56.968456 13009 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:56.995062  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:56.995258 13355 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:56.962917 13424 consensus_queue.cc:237] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:56.985661 13449 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:56.989197 13459 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:57.008466  6500 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 3036) took 1123 ms. Trace:
I20260709 12:19:57.008858  6500 rpcz_store.cc:276] 0709 12:19:55.885216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:56.622099 (+736883us) service_pool.cc:224] Handling call
0709 12:19:56.622342 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:56.692957 (+ 70615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:57.008229 (+315272us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b5400204254. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:57.008410 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23605632}
I20260709 12:19:57.005002  6548 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:56.991212 13416 raft_consensus.cc:3055] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:57.006155  6658 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
I20260709 12:19:57.014074  6548 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:57.018332  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:56.997287 13329 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:57.021534 13329 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:57.021860  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:57.026005  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:57.002948 13355 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:57.002172 13317 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:19:57.012439 13320 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:57.033077  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:57.007159 13459 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:57.056805  6548 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:57.000609 13199 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:57.016820 13416 raft_consensus.cc:740] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:19:57.155630  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:57.018778 13474 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:57.027276 12878 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:57.003391 13009 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:57.162058  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:57.170624  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:57.170984  6575 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
W20260709 12:19:57.169912  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:57.173017  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:57.173748  6656 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:19:57.174780  6550 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:57.175909  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:57.176812 13149 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:19:57.177349 13492 log.cc:927] Time spent T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.172s	user 0.000s	sys 0.000s
W20260709 12:19:57.179157 13462 consensus_meta.cc:287] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.550s	user 0.007s	sys 0.003s
I20260709 12:19:57.179531 13462 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:57.181619 13462 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:57.182003 13313 consensus_meta.cc:287] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.550s	user 0.010s	sys 0.000s
I20260709 12:19:57.182454 13313 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:57.155550 13459 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:57.186225  6579 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22.
I20260709 12:19:57.187392  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:57.032451 13355 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:57.192701  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:57.193266 13317 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:57.194610 13138 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:57.196796 13474 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:57.197476 13474 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:57.200771 13199 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:57.196877  6507 raft_consensus.cc:1240] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:19:57.161732 13063 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:57.203771  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:57.011504 13449 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:57.204427  6575 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:19:57.205902 13298 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:57.206143 13449 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:57.206425 13298 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:57.206919 13298 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:57.208645 13459 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:57.218003 13009 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:57.154978 13314 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:19:57.171212 13482 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:57.222182  6576 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:19:57.171484 13481 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:57.223533 13481 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:57.225947 13481 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:57.229027 13416 consensus_queue.cc:260] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:57.230640 13416 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:57.155668 13374 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:57.237754 13374 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:57.238556  6653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3435) took 1118 ms. Trace:
I20260709 12:19:57.239014  6653 rpcz_store.cc:276] 0709 12:19:56.120292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:56.622740 (+502448us) service_pool.cc:224] Handling call
0709 12:19:56.623051 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:56.672822 (+ 49771us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:56.672866 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:56.672922 (+    56us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:19:56.672976 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:57.238323 (+565347us) spinlock_profiling.cc:243] Waited 49.6 ms on lock 0x7b54000b7554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:57.238376 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:57.238501 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":114007424}
I20260709 12:19:57.196270  6651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3433) took 1076 ms. Trace:
I20260709 12:19:57.241097  6651 rpcz_store.cc:276] 0709 12:19:56.119798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:56.599184 (+479386us) service_pool.cc:224] Handling call
0709 12:19:56.599459 (+   275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:56.599780 (+   321us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:56.599807 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:56.600302 (+   495us) log.cc:844] Serialized 26 byte log entry
0709 12:19:56.817892 (+217590us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:56.817931 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:56.817999 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:57.196068 (+378069us) raft_consensus.cc:1685] finished
0709 12:19:57.196089 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:57.196213 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5895,"raft.run_cpu_time_us":60,"raft.run_wall_time_us":60,"thread_start_us":217049,"threads_started":1,"wal-append.queue_time_us":367698}
I20260709 12:19:57.243497  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3434) took 1123 ms. Trace:
I20260709 12:19:57.243922  6652 rpcz_store.cc:276] 0709 12:19:56.120052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:56.610730 (+490678us) service_pool.cc:224] Handling call
0709 12:19:56.611028 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:56.708736 (+ 97708us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:56.708784 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:56.708849 (+    65us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:19:56.708896 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:57.243263 (+534367us) spinlock_profiling.cc:243] Waited 97.5 ms on lock 0x7b540008bbd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:57.243311 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:57.243440 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":224181120}
I20260709 12:19:57.245390  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:57.163303 12878 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:57.232524 13459 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:57.247903 12878 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:57.248446 13320 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:57.248881 13320 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:57.249274 13320 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:57.261245  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:57.269248  6580 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 24: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:19:57.162714 13495 log.cc:927] Time spent T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
I20260709 12:19:57.284034 13372 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:57.284372  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:57.284705 13372 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:57.285789  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:57.286510  6579 raft_consensus.cc:2393] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 19: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:57.287042 13372 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:57.287283  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:57.293040  6579 raft_consensus.cc:2393] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 23: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:57.157953 13329 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:57.296918  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 122215us
I20260709 12:19:57.254243 13288 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 19. Current term is 20.
I20260709 12:19:57.303746  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:57.331835 13500 log.cc:927] Time spent T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
I20260709 12:19:57.333580  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2773) took 1598 ms. Trace:
I20260709 12:19:57.333987  6579 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 10. (term mismatch)
I20260709 12:19:57.334515  6579 pending_rounds.cc:85] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 9
I20260709 12:19:57.334048  6649 rpcz_store.cc:276] 0709 12:19:55.735551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.737066 (+  1515us) service_pool.cc:224] Handling call
0709 12:19:55.737412 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:56.587732 (+850320us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:57.333327 (+745595us) spinlock_profiling.cc:243] Waited 765 ms on lock 0x7b54002a53d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:57.333519 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1758482176}
I20260709 12:19:57.334959  6579 pending_rounds.cc:107] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 19.10
I20260709 12:19:57.338029  6579 raft_consensus.cc:2889] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:57.338529  6654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 2780) took 1360 ms. Trace:
I20260709 12:19:57.338932  6654 rpcz_store.cc:276] 0709 12:19:55.978398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.979254 (+   856us) service_pool.cc:224] Handling call
0709 12:19:57.338301 (+1359047us) spinlock_profiling.cc:243] Waited 860 ms on lock 0x7b54003e8fd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:57.338438 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1976901376}
W20260709 12:19:57.341506  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 284774us
I20260709 12:19:57.342154  6506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 2895) took 1874 ms. Trace:
I20260709 12:19:57.342607  6506 rpcz_store.cc:276] 0709 12:19:55.466801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.467364 (+   563us) service_pool.cc:224] Handling call
0709 12:19:57.341410 (+1874046us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54001e91d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:57.341561 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2482195712}
I20260709 12:19:57.346724  6548 leader_election.cc:400] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 13. Current term is 14.
I20260709 12:19:57.348265  6548 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:57.305760 13329 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:57.354923  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:57.369912  6548 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:57.370751  6653 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 18.
I20260709 12:19:57.375598  7240 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:19:57.381465  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:57.395599  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2889) took 1192 ms. Trace:
I20260709 12:19:57.398561  6475 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:57.399080 13481 consensus_queue.cc:1059] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:57.400533 13442 raft_consensus.cc:3055] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:57.401029 13442 raft_consensus.cc:740] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:57.396203  6577 rpcz_store.cc:276] 0709 12:19:56.202865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:56.494142 (+291277us) service_pool.cc:224] Handling call
0709 12:19:56.500319 (+  6177us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:56.719852 (+219533us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:19:56.905878 (+186026us) log.cc:844] Serialized 14 byte log entry
0709 12:19:57.342595 (+436717us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:57.342653 (+    58us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:19:57.342704 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:57.395282 (+ 52578us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b540034d0d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:57.395345 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:57.395520 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":180363,"prepare.queue_time_us":281792,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":35997,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":72,"spinlock_wait_cycles":509787648,"thread_start_us":586616,"threads_started":3,"wal-append.queue_time_us":403240}
I20260709 12:19:57.408083  6475 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:57.349896 13374 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:57.412230  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:57.414321  7265 raft_consensus.cc:1240] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:57.418797  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:57.424295  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:57.400270 13481 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:57.422859  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:57.411880 13480 log.cc:927] Time spent T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.950s	user 0.011s	sys 0.032s
W20260709 12:19:57.433424  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:57.427573 13464 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:57.401966 13442 consensus_queue.cc:260] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:57.413193 13429 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:57.436681  6506 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:57.448155  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2882) took 1281 ms. Trace:
I20260709 12:19:57.436836 13336 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:19:57.437803  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 127208us
I20260709 12:19:57.419193 13391 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:57.419369 13459 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:57.415203 13215 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:57.442248  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:57.437279 13481 raft_consensus.cc:2764] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:19:57.420125 13483 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:57.470108  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:19:57.455730  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 85943.5us
I20260709 12:19:57.452253  6584 rpcz_store.cc:276] 0709 12:19:56.166544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:56.167018 (+   474us) service_pool.cc:224] Handling call
0709 12:19:56.167382 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:56.636845 (+469463us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:57.447847 (+811002us) spinlock_profiling.cc:243] Waited 469 ms on lock 0x7b54000587d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:57.448078 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1077981440}
I20260709 12:19:57.438737  6475 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:57.460000  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:57.467270  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:57.473181  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:57.475171  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:57.475960  6584 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
W20260709 12:19:57.476647  6502 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41304 (request call id 839) took 1878 ms (client timeout 2032 ms). Trace:
W20260709 12:19:57.477064  6502 rpcz_store.cc:269] 0709 12:19:55.598450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.598570 (+   120us) service_pool.cc:224] Handling call
0709 12:19:57.476515 (+1877945us) spinlock_profiling.cc:243] Waited 852 ms on lock 0x7b54000f3054. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:57.476597 (+    82us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1960250112}
I20260709 12:19:57.477409  6584 raft_consensus.cc:1240] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 13. Current term is 14. Ops: []
I20260709 12:19:57.477880  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:57.478027  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:57.455677  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:57.478652  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:57.478645  6502 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:19:57.481145  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:57.487159  6504 raft_consensus.cc:2393] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 22: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:57.489181  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:57.464207 13458 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:57.510844  6584 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:19:57.513233  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:57.519711  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:57.175856  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:57.454795 13464 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:57.529227  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:57.548573  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:57.537918  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:57.471597 13442 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:57.551119  6473 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:57.561100  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:57.568892  6577 raft_consensus.cc:2393] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 18: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:57.570322  6473 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:57.473904 13407 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:57.553956  6656 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:19:57.464169 13391 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:57.550727  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 77972.3us
I20260709 12:19:57.583067  6548 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:57.556747  6581 raft_consensus.cc:2393] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 21: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:57.557251  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:19:57.558187  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:57.584314  6548 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:57.586176  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:57.569787  6549 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:57.588125  6505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 3017) took 1749 ms. Trace:
I20260709 12:19:57.588780  6505 rpcz_store.cc:276] 0709 12:19:55.838928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:55.839208 (+   280us) service_pool.cc:224] Handling call
0709 12:19:57.587791 (+1748583us) spinlock_profiling.cc:243] Waited 705 ms on lock 0x7b540039ec54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:57.587973 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1622507392}
I20260709 12:19:57.590199  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:57.592147  6505 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:19:57.558421  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:57.559301  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:57.560184  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:57.566531  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:19:57.554688 13506 log.cc:927] Time spent T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.237s	user 0.001s	sys 0.000s
I20260709 12:19:57.469592 13215 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:57.529804  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:57.577365  6651 raft_consensus.cc:1240] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:57.474346 13483 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:57.497109 13424 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:19:57.519557 13480 log.cc:927] Time spent T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:19:57.529420  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:19:57.617596 13511 log.cc:927] Time spent T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:19:57.618112 13483 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:57.618697 13483 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:57.618875 13336 consensus_queue.cc:1059] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:19:57.619729 13510 log.cc:927] Time spent T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:19:57.561502 13374 consensus_queue.cc:1059] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 12:19:57.620765 13509 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:19:57.579735  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:57.591252 13199 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:57.599676 12878 consensus_queue.cc:1059] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:57.602718 13508 consensus_queue.cc:1048] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:57.603550 13516 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:57.529084 13481 consensus_queue.cc:1059] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.093s
I20260709 12:19:57.604287 13429 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:57.605225  6655 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58980 (request call id 3409) took 2660 ms (client timeout 1643 ms). Trace:
W20260709 12:19:57.644083 13289 consensus_meta.cc:287] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.637s	user 0.009s	sys 0.002s
W20260709 12:19:57.643937  6655 rpcz_store.cc:269] 0709 12:19:54.944754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:54.944901 (+   147us) service_pool.cc:224] Handling call
0709 12:19:55.424374 (+479473us) spinlock_profiling.cc:243] Waited 4.88 ms on lock 0x7b5800015998. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64eee 000000000036eb39 00007f287add81e0 00007f287aea10c4 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:19:57.590987 (+2166613us) spinlock_profiling.cc:243] Waited 558 ms on lock 0x7b54003e85d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:57.605131 (+ 14144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1295728128}
I20260709 12:19:57.579493 13478 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:19:57.644824 13289 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:57.646186 13374 raft_consensus.cc:3055] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:57.646369 12878 raft_consensus.cc:3055] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:57.646653 13374 raft_consensus.cc:740] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:19:57.646975 13477 consensus_meta.cc:287] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.824s	user 0.011s	sys 0.000s
I20260709 12:19:57.594326  6650 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
W20260709 12:19:57.586871 13362 consensus_meta.cc:287] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.707s	user 0.012s	sys 0.000s
I20260709 12:19:57.638064 13317 consensus_queue.cc:1059] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 12:19:57.638239 13516 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:57.648836 13362 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:57.649394 13516 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:57.650841 13467 raft_consensus.cc:3055] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:57.651291 13467 raft_consensus.cc:740] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:57.652447 13478 raft_consensus.cc:697] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 22 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:57.653337 13478 consensus_queue.cc:237] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:57.641521 13440 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:57.657030 13440 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:57.657595 13199 raft_consensus.cc:697] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:57.658605 13199 consensus_queue.cc:237] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:57.659171 13440 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:57.567440 13494 log.cc:927] Time spent T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 12:19:57.626124  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:57.663601  6622 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:57.664497  6622 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:57.623538  6549 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:57.532414  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:57.626641  6504 raft_consensus.cc:2376] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 21. Re-sending same reply.
I20260709 12:19:57.604080  6584 raft_consensus.cc:2393] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 18: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:57.666079  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:57.549873 13458 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:57.647931  6509 raft_consensus.cc:2410] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:19:57.667244  6504 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:19:57.672883  6622 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:57.639462  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:57.676107  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:57.646910 12878 raft_consensus.cc:740] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:57.651032 13362 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:57.647809 13374 consensus_queue.cc:260] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:57.683573  6505 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:19:57.638659 13459 raft_consensus.cc:697] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 18 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:57.674639 13508 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:19:57.674813 13407 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:57.690403 13417 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:19:57.690879 13407 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:57.691383 13407 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:57.674335 13276 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:19:57.686517  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:57.675779 13435 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:57.692754  6508 raft_consensus.cc:2410] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
W20260709 12:19:57.694428  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:57.678768 13009 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:57.703150  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:57.695430  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:57.703919  6655 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:57.704370  6505 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:57.619199 13483 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:57.696348  6504 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:19:57.652064 13467 consensus_queue.cc:260] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:57.689672 13329 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:57.714290  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:57.709272  6550 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:57.716311  7265 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:57.717144  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:57.705607  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 151007us
I20260709 12:19:57.713320 13313 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:57.696784 13276 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:57.718101  6505 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:19:57.718367  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:57.725497  6504 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:19:57.713402 13289 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:57.722074  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:57.706787 13374 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:57.739173  7238 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:57.742913  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:57.743734  6504 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
W20260709 12:19:57.747300  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 159341us
I20260709 12:19:57.748245  6473 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:57.752887 13419 consensus_queue.cc:1059] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:19:57.754526  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 126925us
I20260709 12:19:57.715593 13215 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:57.757915  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:57.758729  6582 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:19:57.758798  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:57.757861  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:57.759250  6550 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:57.759781  6584 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:19:57.732316 13276 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:57.763000  6582 raft_consensus.cc:1275] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:19:57.763352 13335 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:57.764253 13427 consensus_queue.cc:1059] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:57.765268 13419 raft_consensus.cc:3055] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:19:57.762292  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:57.765738 13419 raft_consensus.cc:740] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:57.766039  6655 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:57.766779 13419 consensus_queue.cc:260] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:57.768121 13419 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:57.765390 13362 raft_consensus.cc:3055] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:19:57.768874 13362 raft_consensus.cc:740] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:57.769775 13362 consensus_queue.cc:260] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:57.770795 13362 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:19:57.772544 13276 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:19:57.765707  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:57.776501  6582 raft_consensus.cc:2410] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:19:57.732534 13515 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:57.775624  6505 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:57.779874  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:57.779862  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:57.785122  6582 raft_consensus.cc:2393] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 19: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:19:57.777902 13416 consensus_meta.cc:287] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.547s	user 0.007s	sys 0.006s
I20260709 12:19:57.746922 13313 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:57.776576  6655 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:19:57.755668 13336 consensus_queue.cc:1243] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:19:57.760861 13464 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:57.789412 13515 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:57.800637  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:57.801329 13515 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:57.801857 13391 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:57.802901 13483 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:57.719455 12878 consensus_queue.cc:260] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 13.14, Last appended by leader: 14, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:57.791149  6500 raft_consensus.cc:2410] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 19 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 12:19:57.794008 13276 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:57.794510 13355 consensus_meta.cc:287] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.606s	user 0.005s	sys 0.005s
W20260709 12:19:57.795428 13507 log.cc:927] Time spent T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.448s	user 0.000s	sys 0.002s
I20260709 12:19:57.809525 13458 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:57.810369 13499 log.cc:927] Time spent T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.648s	user 0.001s	sys 0.001s
I20260709 12:19:57.811794  6657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 1082) took 1335 ms. Trace:
I20260709 12:19:57.809442 13355 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:57.812201  6657 rpcz_store.cc:276] 0709 12:19:56.476446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:56.663087 (+186641us) service_pool.cc:224] Handling call
0709 12:19:56.663452 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:56.663834 (+   382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:56.663869 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:56.672412 (+  8543us) log.cc:844] Serialized 26 byte log entry
0709 12:19:56.966429 (+294017us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:19:56.966480 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:56.966699 (+   219us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:57.809774 (+843075us) spinlock_profiling.cc:243] Waited 4.38 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:57.811458 (+  1684us) raft_consensus.cc:1685] finished
0709 12:19:57.811491 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:57.811728 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8684,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":8216,"spinlock_wait_cycles":10080000,"thread_start_us":263973,"threads_started":1,"wal-append.queue_time_us":333359}
I20260709 12:19:57.816677  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:57.817335  6502 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:19:57.790258 13459 consensus_queue.cc:237] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:57.818653  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:57.819252  6500 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:57.802765  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:57.820999  7274 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
W20260709 12:19:57.801374 13298 consensus_meta.cc:287] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.594s	user 0.012s	sys 0.000s
I20260709 12:19:57.822458 13298 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:57.825052 13355 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:57.825731  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 3028) took 1984 ms. Trace:
I20260709 12:19:57.792949  6550 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:57.827744  6507 rpcz_store.cc:276] 0709 12:19:55.841511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:56.615913 (+774402us) service_pool.cc:224] Handling call
0709 12:19:56.616232 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:57.201582 (+585350us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:57.825465 (+623883us) spinlock_profiling.cc:243] Waited 580 ms on lock 0x7b54000f3a54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:57.825666 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1334901120}
I20260709 12:19:57.826915  7273 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38562 (request call id 1139) took 1207 ms. Trace:
I20260709 12:19:57.828927  7273 rpcz_store.cc:276] 0709 12:19:56.619273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:56.619421 (+   148us) service_pool.cc:224] Handling call
0709 12:19:57.826674 (+1207253us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b54000a50d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:57.826823 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":431689472}
W20260709 12:19:57.830060 13320 consensus_meta.cc:287] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.580s	user 0.011s	sys 0.000s
I20260709 12:19:57.830674 13355 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:19:57.830559 13320 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:57.831959  6579 raft_consensus.cc:1240] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:57.832094  6578 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:57.832720 13313 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:19:57.833443 13490 log.cc:927] Time spent T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.844s	user 0.007s	sys 0.000s
I20260709 12:19:57.834319 13313 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:57.834484 13320 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:57.835337 13355 raft_consensus.cc:697] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:57.836274 13355 consensus_queue.cc:237] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:57.838155  6549 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:57.838963 13320 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:19:57.839403 13320 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:57.839831 13320 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:19:57.843734  7265 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22.
I20260709 12:19:57.789749 13529 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:57.844789  6549 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:57.844861 13529 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:57.848265 13529 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:57.848827  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:57.849654  6650 raft_consensus.cc:2410] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:19:57.850870  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:57.851353  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:57.852221 13430 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:19:57.853116  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:57.853281 13474 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:57.853937 13430 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:57.853938 13474 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:19:57.859867  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:57.865465 13467 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:57.866016 13474 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:57.866659 13474 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:19:57.854841 13009 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:57.847581  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:57.870994  6649 raft_consensus.cc:2393] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 15: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:57.875355  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:57.886453  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:57.891000  6500 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22.
I20260709 12:19:57.892251  6622 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:57.894729  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:57.895522  6623 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:57.896893  6407 catalog_manager.cc:5697] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 12 to 18, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 18 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:19:57.916651  6502 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22.
I20260709 12:19:57.887722  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:57.919721  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:57.920477  6622 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:57.922883  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:57.924132  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:57.930999  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:57.931756  6654 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:19:57.927819  6407 catalog_manager.cc:5697] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 19 to 22. New cstate: current_term: 22 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:57.934784  6474 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:57.934448  6407 catalog_manager.cc:5697] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 10 to 13, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 13 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:19:57.929322  6505 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:19:57.855223 13371 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:57.938244  6407 catalog_manager.cc:5697] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 13 to 17, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 17 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:57.943742  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:57.944454  6582 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:57.944630  6407 catalog_manager.cc:5697] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 10 to 14, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 14 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:19:57.939790  6474 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:57.941560  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:57.941792  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:57.950563  6502 raft_consensus.cc:2368] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 21. Current term is 22.
I20260709 12:19:57.950632  6500 raft_consensus.cc:2393] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 20: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:57.942744  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:57.913905 13509 raft_consensus.cc:697] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:57.957696  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:57.966290  6500 raft_consensus.cc:2393] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 20: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:19:57.959081  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 195264us
I20260709 12:19:57.957459  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:57.969740  6507 raft_consensus.cc:2410] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 19 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 12:19:57.946329  6407 catalog_manager.cc:5697] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 15 to 18, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 18 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:19:57.967671  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:57.914482 13400 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:19:57.979416  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:57.981444  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:57.973337  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:57.972862  6407 catalog_manager.cc:5697] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:19:57.983825  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:57.983460  6407 catalog_manager.cc:5697] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 15 to 18, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 18 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:57.929716 13276 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:19:57.924242 13482 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:19:57.989553  6405 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46184 (request call id 118) took 69763 ms (client timeout 15000 ms). Trace:
I20260709 12:19:57.976637  6582 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22.
I20260709 12:19:57.985491  6407 catalog_manager.cc:5697] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 10 to 16. New cstate: current_term: 16 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:19:57.990748  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:57.986217  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:57.992060  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:57.938021 13138 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:57.928635 13288 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:57.920321 13313 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:57.994290  6507 raft_consensus.cc:2393] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 16: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:57.997418  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:58.005970  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:57.958186 13371 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:57.945395 13362 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:57.989332  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:57.958526 13009 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:57.937912 13435 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 12:19:57.991366  6407 catalog_manager.cc:5697] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 10 to 18, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 18 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:58.024987  6407 catalog_manager.cc:5697] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:19:58.007156  6577 raft_consensus.cc:2393] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 17: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:57.957949 13521 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:58.027460  6581 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:19:57.996783 13138 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:57.965828 13438 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:58.010727  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:58.045325  6580 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:58.012223  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:58.014746  6509 raft_consensus.cc:2376] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19. Re-sending same reply.
I20260709 12:19:58.018570  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:58.046885  6502 raft_consensus.cc:2393] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 20: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:57.994238  7238 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:58.000397 13288 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:19:58.047930  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 53817.5us
W20260709 12:19:58.048174  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 124148us
I20260709 12:19:58.050458  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:58.009774  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:19:58.039218  6548 leader_election.cc:336] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.550s (SENT)
I20260709 12:19:58.007519 13415 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:58.052983  6548 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:58.042929  6407 catalog_manager.cc:5697] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:19:58.060724  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:19:58.088138  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 48968.8us
I20260709 12:19:58.060801  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:58.041852 13417 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:58.061240 13509 consensus_queue.cc:237] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:58.028427 13298 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:58.069557 13407 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:58.070367 13288 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:19:57.990048  6405 rpcz_store.cc:269] 0709 12:18:48.208901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:48.228450 (+ 19549us) service_pool.cc:224] Handling call
0709 12:19:57.972136 (+69743686us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:51.454383 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:51.498584 (+ 44201us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:51.498643 (+    59us) write_op.cc:435] Acquired schema lock
0709 12:19:51.498665 (+    22us) tablet.cc:662] Decoding operations
0709 12:19:51.499383 (+   718us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:51.499502 (+   119us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:51.499537 (+    35us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:19:51.500037 (+   500us) tablet.cc:701] Row locks acquired
0709 12:19:51.500091 (+    54us) write_op.cc:260] PREPARE: finished
0709 12:19:51.500312 (+   221us) write_op.cc:270] Start()
0709 12:19:51.522820 (+ 22508us) write_op.cc:276] Timestamp: P: 1783599591500238 usec, L: 0
0709 12:19:51.530190 (+  7370us) op_driver.cc:348] REPLICATION: starting
0709 12:19:51.703826 (+173636us) log.cc:844] Serialized 1275 byte log entry
0709 12:19:52.061778 (+357952us) spinlock_profiling.cc:243] Waited 24 ms on lock 0x7b540008acd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e84e28 00007f28798bf581 00007f28798bed58 00007f28798c39f1 00007f28798c39a9 00007f28798c3939 00007f28798c3901 00007f28798c2b2d 0000000000388634 0000000000388479
0709 12:19:52.266377 (+204599us) op_driver.cc:464] REPLICATION: finished
0709 12:19:52.809042 (+542665us) spinlock_profiling.cc:243] Waited 4.81 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2872d7f0ac 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f28798bfef0
0709 12:19:53.369125 (+560083us) write_op.cc:301] APPLY: starting
0709 12:19:53.369259 (+   134us) tablet.cc:1367] starting BulkCheckPresence
0709 12:19:53.421636 (+ 52377us) tablet.cc:1370] finished BulkCheckPresence
0709 12:19:53.434052 (+ 12416us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:19:53.686680 (+252628us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:19:53.686990 (+   310us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=0,mrs_lookups=0
0709 12:19:53.687062 (+    72us) write_op.cc:312] APPLY: finished
0709 12:19:53.687409 (+   347us) log.cc:844] Serialized 41 byte log entry
0709 12:19:53.987016 (+299607us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:53.997250 (+ 10234us) write_op.cc:454] Released schema lock
0709 12:19:54.012242 (+ 14992us) write_op.cc:341] FINISH: Updating metrics
0709 12:19:54.032003 (+ 19761us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:19:54.032056 (+    53us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3227976,"reported_tablets":24,"child_traces":[["op",{"apply.queue_time_us":997865,"apply.run_cpu_time_us":42241,"apply.run_wall_time_us":671940,"cfile_cache_hit":14,"cfile_cache_hit_bytes":16643,"num_ops":3,"prepare.queue_time_us":552581,"prepare.run_cpu_time_us":16689,"prepare.run_wall_time_us":632545,"raft.queue_time_us":90419,"raft.run_cpu_time_us":71739,"raft.run_wall_time_us":1226086,"replication_time_us":516658,"spinlock_wait_cycles":66213376,"thread_start_us":1649897,"threads_started":5,"wal-append.queue_time_us":508314,"wal-append.run_cpu_time_us":19714,"wal-append.run_wall_time_us":2161112}]]}
I20260709 12:19:58.047706 13138 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:58.072917 13320 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:58.075970  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:58.105304  6577 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 19.
I20260709 12:19:58.044620 13521 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:58.050206  6622 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:58.089318  6509 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
W20260709 12:19:58.090713  6548 leader_election.cc:336] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.640s (SENT)
W20260709 12:19:58.106515  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 56402.1us
W20260709 12:19:58.107089  6548 leader_election.cc:336] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.643s (SENT)
I20260709 12:19:58.107498  6548 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:58.017283 13009 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:58.110221  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:58.111377  6506 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:19:58.087260 13149 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:19:58.090389 13417 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:58.106388  6580 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:19:58.108445  7238 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:58.092581  6407 catalog_manager.cc:5697] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 15 to 16, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 16 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:19:58.092270 13336 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:58.119491  6407 catalog_manager.cc:5697] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 15 to 16, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 16 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:19:58.121644  6407 catalog_manager.cc:5697] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 14 to 16, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 16 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:19:58.123859  6407 catalog_manager.cc:5697] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 10 to 15. New cstate: current_term: 15 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:19:58.145004  6475 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:58.126096  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:58.149331  6502 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:58.130043  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:58.139897  6505 raft_consensus.cc:2410] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 20 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 12:19:58.128459  6473 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:58.114079 13009 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:58.181013  6407 catalog_manager.cc:5697] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 11 to 18. New cstate: current_term: 18 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
W20260709 12:19:58.182430  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 54137.9us
I20260709 12:19:58.184464  6475 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:58.184896  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 44007us
I20260709 12:19:58.115264 13409 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:58.187023  6407 catalog_manager.cc:5697] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 14 to 17, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 17 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:19:58.193646  6577 raft_consensus.cc:1275] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:19:58.197055  6407 catalog_manager.cc:5697] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 14 to 18, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 18 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:58.187664  6622 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:58.126678 13417 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:58.142378 13516 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:58.189141 13199 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:58.190797 13409 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:58.189145 13440 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:58.252974  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:58.181587 13374 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:58.275053 13482 raft_consensus.cc:697] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:58.275758  6506 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
W20260709 12:19:58.249017  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 64660.5us
I20260709 12:19:58.302420  6502 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
I20260709 12:19:58.309227  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:19:58.295177  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 107620us
I20260709 12:19:58.309355  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:19:58.299969 13535 log.cc:927] Time spent T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 12:19:58.261353 13427 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:19:58.262490 13483 raft_consensus.cc:2764] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader election decision vote started in defunct term 21: won
I20260709 12:19:58.192076 13537 consensus_queue.cc:1059] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.111s
I20260709 12:19:58.313602  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:58.334148  6506 raft_consensus.cc:2393] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 19: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:58.303333 13409 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:58.311017  6622 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:58.302863 13276 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:58.310585  6502 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 19.
I20260709 12:19:58.280674  6407 catalog_manager.cc:5697] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:19:58.336815  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:58.339463  6407 catalog_manager.cc:5697] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 7 to 16. New cstate: current_term: 16 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:19:58.344050  6407 catalog_manager.cc:5697] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 15 to 16, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 16 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:19:58.345973  6407 catalog_manager.cc:5697] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 16 to 20, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 20 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:58.348433  6407 catalog_manager.cc:5697] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 14 to 17, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 17 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:19:58.350435  6407 catalog_manager.cc:5697] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 13 to 20, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 20 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:19:58.352867  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:19:58.312798 13532 log.cc:927] Time spent T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:19:58.354285  6407 catalog_manager.cc:5697] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 13 to 17, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 17 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:19:58.360642  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 252284us
I20260709 12:19:58.336669 13438 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:19:58.360749  6407 catalog_manager.cc:5697] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 7 to 18, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 18 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:19:58.363415  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:58.363598  6407 catalog_manager.cc:5697] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 13 to 18, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 18 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:19:58.366667  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:58.367224  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:58.368234  6506 raft_consensus.cc:2410] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 20 because replica has last-logged OpId of term: 19 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 8.
I20260709 12:19:58.368808  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:58.361743 13537 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:58.356108 13435 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:19:58.371951  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:58.309324 13374 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:58.380786  6506 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:58.362923 13276 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:58.373347  6407 catalog_manager.cc:5697] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 16 to 21. New cstate: current_term: 21 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:19:58.379081  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 69914.2us
W20260709 12:19:58.361244 13539 log.cc:927] Time spent T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:19:58.383891  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:58.384815  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:58.384387  6475 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:19:58.384815  6407 catalog_manager.cc:5697] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 11 to 14, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 14 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:19:58.385563  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:58.387013  6651 raft_consensus.cc:1275] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:19:58.388073  6407 catalog_manager.cc:5697] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 13 to 16, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 16 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:19:58.385000  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:58.391393  6475 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:58.394778  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:58.393596  6407 catalog_manager.cc:5697] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 14 to 17, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 17 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:19:58.395740  6407 catalog_manager.cc:5697] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 7 to 9, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 9 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:19:58.397230  6407 catalog_manager.cc:5697] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 8 to 17, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 17 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:19:58.366839 13407 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:58.400450  6475 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:58.408213  6506 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:19:58.409443  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:58.410041  6506 raft_consensus.cc:2393] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 19: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:58.414354  6657 raft_consensus.cc:1275] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:58.417090  7238 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:58.419495  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:58.420511  6506 raft_consensus.cc:2393] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 20: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:19:58.399207 13541 log.cc:927] Time spent T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 12:19:58.392743 13440 raft_consensus.cc:697] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:58.400705 13438 raft_consensus.cc:2764] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:19:58.426072  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:58.427923  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:58.402020 13317 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:58.437438  6506 raft_consensus.cc:2410] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:19:58.399641 13276 raft_consensus.cc:697] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:58.382357 13537 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:58.440483  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:58.441541  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:58.443352  6657 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:19:58.371532 13482 consensus_queue.cc:237] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:58.416517 13419 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:58.442390  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:58.445786  6506 raft_consensus.cc:2393] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 20: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:58.426776 13521 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:58.422281 13548 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 12:19:58.458171 13527 consensus_queue.cc:1048] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:58.459057  6475 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:58.495919  6506 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:58944
I20260709 12:19:58.462826 13537 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:58.468815  6502 raft_consensus.cc:2410] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:19:58.472364 13313 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:58.477614 13429 raft_consensus.cc:3055] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:58.478679  6649 raft_consensus.cc:1275] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
W20260709 12:19:58.485396 13528 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.727s	user 0.004s	sys 0.002s
I20260709 12:19:58.489362 13415 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:58.492223 13424 raft_consensus.cc:3055] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:19:58.497105  6506 raft_consensus.cc:606] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:58.548422  6506 raft_consensus.cc:614] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:58.448452 13516 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:58.510572 13417 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:58.437620 13518 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:58.518273 13009 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:58.533437  6509 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
W20260709 12:19:58.534857  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 117913us
W20260709 12:19:58.448664 13467 consensus_meta.cc:287] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.583s	user 0.011s	sys 0.005s
I20260709 12:19:58.550472 13415 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:58.567737  6507 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
W20260709 12:19:58.467561 13474 consensus_meta.cc:287] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.600s	user 0.010s	sys 0.002s
I20260709 12:19:58.562990  6505 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 19. Current term is 20. Ops: [18.12-19.13]
I20260709 12:19:58.552385  6655 raft_consensus.cc:1240] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:58.569208  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:58.544083 13527 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:58.573073  6504 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:19:58.580935  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:58.545063 13429 raft_consensus.cc:740] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:58.547338 13424 raft_consensus.cc:740] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:58.553277 13419 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:58.555852 13483 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:58.558043 13518 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:58.572602 13423 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:58.628927  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 2907) took 1173 ms. Trace:
I20260709 12:19:58.628954  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 2918) took 1066 ms. Trace:
I20260709 12:19:58.633992  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:58.637401  6508 rpcz_store.cc:276] 0709 12:19:57.455296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:57.455412 (+   116us) service_pool.cc:224] Handling call
0709 12:19:58.628696 (+1173284us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b54003976d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:58.628841 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":489155072}
I20260709 12:19:58.638738  6501 rpcz_store.cc:276] 0709 12:19:57.563060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:57.566192 (+  3132us) service_pool.cc:224] Handling call
0709 12:19:58.628743 (+1062551us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b54002c47d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:58.628892 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":877222272}
I20260709 12:19:58.578639 13435 raft_consensus.cc:697] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:19:58.665048  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 73848.6us
I20260709 12:19:58.508158 13288 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:58.578934 13527 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:19:58.668380  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 87530.1us
I20260709 12:19:58.669597  6550 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:58.670367  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:58.595635 13518 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:58.671135  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:19:58.621842 13371 consensus_meta.cc:287] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.593s	user 0.010s	sys 0.000s
I20260709 12:19:58.679730  6655 raft_consensus.cc:2393] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 15: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:58.585897 13474 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:58.633548 13467 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:58.588555 13391 consensus_queue.cc:1059] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 12:19:58.620402 13464 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:58.596733 13554 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:58.661545 13458 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:19:58.686918  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:58.737563  6653 raft_consensus.cc:2368] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 20. Current term is 21.
W20260709 12:19:58.739029 13138 consensus_meta.cc:287] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.625s	user 0.006s	sys 0.005s
I20260709 12:19:58.691058 13417 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:58.691445 13288 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:58.693110 13474 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:58.595285 13419 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:58.686168 13371 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:58.595067 13483 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:58.588613 13409 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:58.703606 13009 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:58.705364 13336 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:19:58.750001 13336 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:58.750420 13336 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:58.750804 13542 raft_consensus.cc:3055] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:58.751222 13542 raft_consensus.cc:740] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:58.751889 13409 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:58.719076 13482 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:58.720506 13538 consensus_queue.cc:1048] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:19:58.723511  6622 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:19:58.725802 13435 consensus_queue.cc:237] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.9, Last appended by leader: 9, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:19:58.624522 13423 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:58.764266 13483 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:58.765944 13423 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:58.728055 13529 consensus_meta.cc:287] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.879s	user 0.012s	sys 0.000s
W20260709 12:19:58.728662  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 611699us
W20260709 12:19:58.766847  6548 leader_election.cc:336] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.697s (SENT)
I20260709 12:19:58.767343 13419 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:58.768138  6548 leader_election.cc:336] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.726s (SENT)
I20260709 12:19:58.768199  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:58.768788  6548 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:58.769019  6508 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
I20260709 12:19:58.769372 13371 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:58.593210 13276 consensus_queue.cc:237] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:58.770551 13009 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:58.734521 13467 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:58.595427 13440 consensus_queue.cc:237] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:58.770918 13458 consensus_queue.cc:1059] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:19:58.771693 13458 consensus_queue.cc:1243] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 21.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:19:58.681026 13527 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:58.772763 13467 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:58.629506 13429 consensus_queue.cc:260] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:58.743419  6550 leader_election.cc:400] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 20. Current term is 21.
I20260709 12:19:58.773770  6550 leader_election.cc:403] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:19:58.707158  6655 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:58.775544  7238 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:58.776324  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:58.777027  7274 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:19:58.777415  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:58.739481 13138 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:58.757712  6652 raft_consensus.cc:1240] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:58.755689 13482 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:58.795665 13482 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:58.766623 13529 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:58.769586 13474 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:58.779321  6548 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:58.780651  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:58.806661  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 2762) took 1832 ms. Trace:
I20260709 12:19:58.814771  6578 rpcz_store.cc:276] 0709 12:19:56.974671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:57.365496 (+390825us) service_pool.cc:224] Handling call
0709 12:19:57.379060 (+ 13564us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:57.832591 (+453531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:58.806368 (+973777us) spinlock_profiling.cc:243] Waited 453 ms on lock 0x7b540006e5d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:58.806585 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1041400064}
I20260709 12:19:58.786645  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:58.816726  6501 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:58.775193 13298 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:58.847242  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:19:58.847358  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 44083.4us
I20260709 12:19:58.848165  6505 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:58.772163 13458 raft_consensus.cc:3055] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:19:58.793072 13138 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:19:58.786710 13355 consensus_queue.cc:1175] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f2b06aebe97249a4877d798b8118ece4" responder_term: 19 status { last_received { term: 19 index: 11 } last_committed_idx: 11 last_received_current_leader { term: 19 index: 11 } }
I20260709 12:19:58.806519  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:58.859539  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 97567.6us
I20260709 12:19:58.786579 13523 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:58.741523  6650 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:19:58.805435  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:19:58.850486  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 71243.7us
I20260709 12:19:58.870158  6548 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:58.850351 13483 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:58.857964 13515 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:58.874986  6509 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:19:58.856974 13458 raft_consensus.cc:740] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:58.877063  6548 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:58.796356 13508 consensus_queue.cc:1059] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.078s
W20260709 12:19:58.799098 13533 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.734s	user 0.002s	sys 0.016s
I20260709 12:19:58.770989 13009 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:58.860728  6653 raft_consensus.cc:1240] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: []
I20260709 12:19:58.797811  6658 raft_consensus.cc:2368] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 17. Current term is 18.
I20260709 12:19:58.805678 13320 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:58.866351  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:58.872195 13523 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:58.883284 13515 raft_consensus.cc:697] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:58.908236 13423 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:58.908835 13423 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:58.911096 13423 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:58.912600 13423 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:58.913121 13423 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:58.882961 13467 consensus_queue.cc:1048] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 12:19:58.915392 13542 consensus_queue.cc:260] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:58.916280 13529 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:58.883245 13474 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:58.857689 13415 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:58.919287 13415 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:19:58.920490  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 54240.3us
I20260709 12:19:58.921383 13288 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:58.925794  6579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 2753) took 1982 ms. Trace:
I20260709 12:19:58.926340  6579 rpcz_store.cc:276] 0709 12:19:56.943268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:57.339005 (+395737us) service_pool.cc:224] Handling call
0709 12:19:57.339338 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:57.832428 (+493090us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:58.925474 (+1093046us) spinlock_profiling.cc:243] Waited 492 ms on lock 0x7b54001cc354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:58.925728 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1132383488}
I20260709 12:19:58.852160  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:58.854425  6474 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:58.796087 13527 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:58.873276 13435 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:58.873709 13483 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:58.906891 13362 consensus_queue.cc:1059] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:19:58.915956 13423 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:58.916786 13542 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:58.940897 13458 consensus_queue.cc:260] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:58.942435 13458 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:19:58.946398 13527 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:58.866809  6550 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:58.947355 13298 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 20. Current term is 21.
W20260709 12:19:58.947711  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 81016.2us
I20260709 12:19:58.948241  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 2915) took 1434 ms. Trace:
I20260709 12:19:58.948467  6501 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:19:58.948665  6503 rpcz_store.cc:276] 0709 12:19:57.513650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:57.514545 (+   895us) service_pool.cc:224] Handling call
0709 12:19:57.565114 (+ 50569us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:19:57.565596 (+   482us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:57.565641 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:19:57.575986 (+ 10345us) log.cc:844] Serialized 26 byte log entry
0709 12:19:57.823370 (+247384us) spinlock_profiling.cc:243] Waited 6.7 ms on lock 0x7b180031c600. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:19:57.997208 (+173838us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:19:57.997247 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:57.997372 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:58.798501 (+801129us) spinlock_profiling.cc:243] Waited 21.2 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:19:58.948020 (+149519us) raft_consensus.cc:1685] finished
0709 12:19:58.948040 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:58.948175 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30364,"raft.queue_time_us":1525,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":17322,"spinlock_wait_cycles":64077056,"thread_start_us":173649,"threads_started":1,"wal-append.queue_time_us":443355}
I20260709 12:19:58.950687  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:58.951617  6653 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:58.886723 13009 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:58.929339  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 75023.9us
I20260709 12:19:58.948112 13298 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:58.957319 13298 raft_consensus.cc:697] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:58.958232 13298 consensus_queue.cc:237] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:58.958709  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:58.959607  7274 raft_consensus.cc:2410] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 15 because replica has last-logged OpId of term: 13 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 13.
I20260709 12:19:58.767917 13336 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:58.960940  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:58.962275 13336 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:58.883582  6548 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:58.947045 13483 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:58.929236  6508 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:19:58.963564  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 81740us
I20260709 12:19:58.964715  6548 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:58.964989 13138 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:19:58.966329  6548 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:58.967325 13474 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:19:58.967396  6548 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:58.968202 13474 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:58.968489 13329 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:58.968713 13474 raft_consensus.cc:697] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:58.969111 13329 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:58.969563 13474 consensus_queue.cc:237] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:58.970263 13288 raft_consensus.cc:697] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:58.970523  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:58.971144 13288 consensus_queue.cc:237] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:58.971910 13435 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:58.974080 13435 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:58.974507  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:58.975136  6649 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:58.908269 13515 consensus_queue.cc:237] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:58.975894  6548 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:58.977042  6548 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:58.977742 13320 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:58.978350 13320 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:19:58.963891 13538 consensus_queue.cc:1059] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:19:58.980309 13538 raft_consensus.cc:3055] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:58.980721 13538 raft_consensus.cc:740] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:58.981096  6548 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:58.981825 13336 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:19:58.981597 13538 consensus_queue.cc:260] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:58.971869  6508 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:19:58.982929 13538 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:58.990408  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:58.990527  6653 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:19:58.991350  6650 raft_consensus.cc:2393] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 19: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:58.992017  6550 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:58.982934  6548 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:58.993052 13336 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:58.993986 13336 raft_consensus.cc:697] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:58.994853 13336 consensus_queue.cc:237] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:58.950747  6505 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:19:58.997133 13523 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:58.998564  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:58.999156  6649 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 24.
I20260709 12:19:59.000439  6550 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:58.886165 13516 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:19:58.892894 13558 log.cc:927] Time spent T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 12:19:59.004935  6548 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:59.006633  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:59.002768  6475 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:59.002887 13320 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:19:59.007583  6548 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:59.012431  6622 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:59.000450 13415 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:59.014776  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:59.021255  6650 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:59.016749  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:59.015839 13508 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:59.027722  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:59.029649  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:59.031008  6500 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:19:59.031853  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:59.047024  6506 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:19:59.001652 13523 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:19:59.049386  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:59.049283 13522 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:59.050324  6503 raft_consensus.cc:2393] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 19: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:59.050470 13522 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:59.052593  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:59.053222  6508 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:19:59.054456  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:59.057367 13522 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:59.043154 13435 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:19:59.059175 13435 raft_consensus.cc:697] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 19 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:59.059175  7273 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:19:59.059737 13009 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:59.060237 13435 consensus_queue.cc:237] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:59.061120 13508 raft_consensus.cc:697] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:59.044051 13415 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:59.061568  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:59.013826 13467 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:59.047755 13555 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:59.011386 13320 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:59.011313 13529 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:19:59.087080  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:59.014312 13362 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:59.068897 13483 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:59.088853 13362 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:59.089609 13362 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:59.077030  6650 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:19:59.082665 13555 raft_consensus.cc:697] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:59.094964 13555 consensus_queue.cc:237] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:59.096078  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:59.100862  6501 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 20. Current term is 21. Ops: []
I20260709 12:19:59.082491 13467 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:59.077752  6508 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:19:59.015916 13552 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:59.103034 13552 raft_consensus.cc:697] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:59.103816 13467 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:59.080116  6503 raft_consensus.cc:2368] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 20. Current term is 21.
I20260709 12:19:59.062304 13407 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:19:59.080760  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.096187  6475 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:59.114529  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:59.112790  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:59.147558  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:59.147598  6622 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:59.077350 13524 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:59.081534 13415 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:59.152277 13288 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:59.152550 13417 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:19:59.154713 13424 consensus_queue.cc:260] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:59.156679 13429 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:59.104213 13552 consensus_queue.cc:237] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:59.104516 13483 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:59.158025 13467 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:59.158309 13518 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:59.172712 13336 consensus_queue.cc:1059] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:59.173629 13336 raft_consensus.cc:3055] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:59.173995 13336 raft_consensus.cc:740] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:59.174763 13336 consensus_queue.cc:260] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:59.158715  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 62685.4us
I20260709 12:19:59.175906 13336 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:59.086797 13320 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:19:59.180935 13313 raft_consensus.cc:3055] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:59.171422 13467 raft_consensus.cc:697] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:59.167460 13288 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:59.183732  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:59.194218  6650 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:59.182531  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:59.151067 13553 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:19:59.197270  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:59.197896  6501 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:59.202359  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:19:59.206780  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:59.208568  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:59.209164  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.214859  6650 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 20.
I20260709 12:19:59.217999  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:59.192957 13313 raft_consensus.cc:740] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:59.201646 13288 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:59.221741  6503 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:19:59.189240 13518 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:59.224459  6501 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 20.
I20260709 12:19:59.161432 13362 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:59.230398  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:59.227365  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:59.232113  6501 raft_consensus.cc:2393] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 20: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:59.229826  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:59.167945 13524 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:59.227203  7238 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:59.235874  7273 raft_consensus.cc:2368] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 16. Current term is 17.
I20260709 12:19:59.237322  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:59.238240  6575 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:19:59.240286  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:59.233727  6475 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:59.242494  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:59.243326  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:59.246012  7265 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:19:59.217236 13458 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:59.219796 13549 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:59.111799 13508 consensus_queue.cc:237] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:59.219619 13474 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:59.193802 13400 consensus_queue.cc:1059] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:19:59.227474 13298 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:59.249982  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:59.250857  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:59.251942  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:59.252462  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:59.252712  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:59.256312  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:59.272567  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:19:59.256352  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:59.257555  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:59.277261  6578 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:59.265162  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:59.280640  6581 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:19:59.283970  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.265790 13515 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:59.266244  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:19:59.221545 13529 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:59.291633  6575 raft_consensus.cc:2393] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 19: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:59.292846  6500 raft_consensus.cc:2393] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 19: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:19:59.257946 13427 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 12:19:59.295058 13400 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:59.295642 13400 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:59.297875 13400 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:59.265545 13549 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:59.300438  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:59.304399  6501 raft_consensus.cc:2410] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 25 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 12:19:59.305245  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:59.257784 13483 raft_consensus.cc:2804] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:19:59.268937  6576 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:19:59.227275 13522 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:19:59.314064  6579 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:19:59.313006  6581 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:19:59.267570 13527 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:19:59.268666  6583 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:19:59.306963  6577 raft_consensus.cc:1240] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 15. Current term is 16. Ops: []
I20260709 12:19:59.232723 13482 consensus_queue.cc:1059] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 12:19:59.324931  6548 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:59.265208 13554 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:59.336143  6500 raft_consensus.cc:1240] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:19:59.284888 13515 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:59.330982  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:59.311314 13440 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:59.259236 13458 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:59.286803 13474 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:59.269970 13313 consensus_queue.cc:260] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:59.337662  6548 leader_election.cc:400] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 20. Current term is 21.
I20260709 12:19:59.358399  6579 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:19:59.371451  6578 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:19:59.372139  6579 raft_consensus.cc:606] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:59.347621  6502 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 3078) took 1327 ms. Trace:
I20260709 12:19:59.372705  6548 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:59.372586  6579 raft_consensus.cc:614] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:59.383061  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:19:59.372923  6502 rpcz_store.cc:276] 0709 12:19:58.020584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:58.504158 (+483574us) service_pool.cc:224] Handling call
0709 12:19:58.504569 (+   411us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:19:58.549351 (+ 44782us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:19:58.549396 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:19:58.555497 (+  6101us) log.cc:844] Serialized 52 byte log entry
0709 12:19:58.661331 (+105834us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:19:58.873472 (+212141us) log.cc:844] Serialized 14 byte log entry
0709 12:19:58.876024 (+  2552us) log.cc:844] Serialized 14 byte log entry
0709 12:19:58.876227 (+   203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:19:59.340106 (+463879us) spinlock_profiling.cc:243] Waited 44.3 ms on lock 0x7b54001e73d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:19:59.340169 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:19:59.347329 (+  7160us) raft_consensus.cc:1685] finished
0709 12:19:59.347357 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:19:59.347548 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2224,"prepare.queue_time_us":294031,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":2163,"raft.queue_time_us":6446,"raft.run_cpu_time_us":377,"raft.run_wall_time_us":9083,"spinlock_wait_cycles":101879552,"thread_start_us":277263,"threads_started":2,"wal-append.queue_time_us":153965}
I20260709 12:19:59.383852  7265 raft_consensus.cc:2393] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 17: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:59.335551 13542 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:59.388613  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:19:59.388670  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 57804.6us
I20260709 12:19:59.290375 13298 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:59.406288  6504 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 2927) took 1661 ms (client timeout 1536 ms). Trace:
I20260709 12:19:59.449265  6577 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:59.358139 13482 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:19:59.340317 13429 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:19:59.405318 13374 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:59.471725  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:19:59.344136 13483 raft_consensus.cc:697] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 20 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:59.324100 13529 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:19:59.448851  6504 rpcz_store.cc:269] 0709 12:19:57.745089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:57.745267 (+   178us) service_pool.cc:224] Handling call
0709 12:19:59.406083 (+1660816us) spinlock_profiling.cc:243] Waited 788 ms on lock 0x7b540001c7d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:59.406212 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1812741376}
I20260709 12:19:59.386319 13009 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:59.450618 13572 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
W20260709 12:19:59.515314  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 43694.2us
W20260709 12:19:59.510070  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 45195.7us
I20260709 12:19:59.386180 13580 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:59.335381 13522 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:59.340395 13554 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:59.452315 13549 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:59.454911 13482 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:19:59.407274 13310 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.111s
I20260709 12:19:59.378623 13440 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:59.334209 13467 consensus_queue.cc:237] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:59.512636 13474 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:59.551736  6577 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 17.
I20260709 12:19:59.525496 13009 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:19:59.496649 13542 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:59.533509 13464 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:59.533823 13482 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:19:59.573050  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:59.574005  6577 raft_consensus.cc:3060] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:59.546890 13362 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:59.559693 13549 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:19:59.587320  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:59.579746  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.587235  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:59.592000  6579 raft_consensus.cc:2376] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 17. Re-sending same reply.
W20260709 12:19:59.592114  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 75819.3us
I20260709 12:19:59.560449 13524 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:59.599341 13573 log.cc:927] Time spent T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.230s	user 0.000s	sys 0.001s
W20260709 12:19:59.593912 13516 consensus_meta.cc:287] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.582s	user 0.010s	sys 0.001s
I20260709 12:19:59.601433 13516 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:19:59.588230 13578 log.cc:927] Time spent T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.237s	user 0.001s	sys 0.001s
I20260709 12:19:59.607725 13415 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:59.609484  6505 raft_consensus.cc:2393] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 17: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:19:59.611107 13415 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:59.599238  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:59.584492 13400 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:59.583694 13374 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:59.636664 13374 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:59.579255 13464 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:59.639240 13374 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:59.641251 13458 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:59.644496 13529 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:59.597066 13483 consensus_queue.cc:237] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:59.653584 13542 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:59.661145 13298 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:59.666877 13400 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:59.668955 13400 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:19:59.669991  6507 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 3072) took 1715 ms (client timeout 1782 ms). Trace:
W20260709 12:19:59.670532  6507 rpcz_store.cc:269] 0709 12:19:57.954951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:58.011970 (+ 57019us) service_pool.cc:224] Handling call
0709 12:19:59.669828 (+1657858us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b54000ff354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:19:59.669935 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":478657920}
I20260709 12:19:59.633841 13515 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:59.597436 13549 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:59.670068  6580 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:19:59.628696  6577 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 20.
I20260709 12:19:59.675951  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:59.676966  6577 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:19:59.678014  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:59.679216  6577 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:19:59.680787  6577 raft_consensus.cc:606] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:59.682372  6577 raft_consensus.cc:614] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:19:59.683995  6579 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:19:59.684996  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:59.685614  6579 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
W20260709 12:19:59.686218  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 91474.6us
I20260709 12:19:59.689121  6577 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 18. Current term is 19. Ops: []
I20260709 12:19:59.689761  6622 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:59.694671 13440 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:59.695246  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:59.695542 13288 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:59.696087  6577 raft_consensus.cc:2393] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 17: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:59.697602 13596 raft_consensus.cc:3055] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:19:59.698052 13596 raft_consensus.cc:740] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:19:59.698318 13579 log.cc:927] Time spent T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.311s	user 0.002s	sys 0.009s
W20260709 12:19:59.698881  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:19:59.699421  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.698957 13596 consensus_queue.cc:260] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:19:59.701041 13320 consensus_meta.cc:287] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.522s	user 0.008s	sys 0.003s
I20260709 12:19:59.701473 13320 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:59.701788 13435 raft_consensus.cc:3055] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:19:59.703886 13320 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:59.685590 13464 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:59.688906 13592 consensus_queue.cc:1059] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 12:19:59.705477 13592 raft_consensus.cc:3055] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:19:59.705832 13592 raft_consensus.cc:740] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:59.704114 13435 raft_consensus.cc:740] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:59.706602 13592 consensus_queue.cc:260] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:59.706840 13435 consensus_queue.cc:260] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:59.707918 13435 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:59.708654  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:59.709311  6577 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:19:59.710299  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:59.711407  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:59.711771  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:59.712407  7265 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:59.712535  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:59.713421 13009 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:59.713462  6577 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:19:59.715297  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:59.715368  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:59.716807 13554 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:19:59.717991  6577 raft_consensus.cc:1275] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:19:59.720839  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:59.724188  6577 raft_consensus.cc:1275] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:19:59.724280  6579 raft_consensus.cc:1275] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:19:59.726215  6579 raft_consensus.cc:1240] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 17. Current term is 18. Ops: []
I20260709 12:19:59.725502  6577 raft_consensus.cc:3055] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:19:59.728684  7270 raft_consensus.cc:3055] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:19:59.728900  6577 raft_consensus.cc:740] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:59.729188  7270 raft_consensus.cc:740] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
W20260709 12:19:59.727748  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.722661  6473 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:59.730302  6549 leader_election.cc:400] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 16. Current term is 17.
I20260709 12:19:59.730715  6549 leader_election.cc:403] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:19:59.730505  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:59.731881  6579 raft_consensus.cc:1275] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 12:19:59.732252  6579 pending_rounds.cc:85] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 9
W20260709 12:19:59.732563  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.733251  6579 pending_rounds.cc:107] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:19:59.733539  6579 raft_consensus.cc:2889] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:59.733968  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:59.734524  6549 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:59.732051  6577 consensus_queue.cc:260] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:59.735536  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:59.735690  6577 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:59.738090  6473 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:59.740044  7265 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:19:59.741185  6549 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:59.700198 13596 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:19:59.743326  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 116763us
I20260709 12:19:59.738579 13440 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:59.742398  7270 consensus_queue.cc:260] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:59.744244 13320 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:19:59.749832  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.747466 13415 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:19:59.728217 13317 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:59.748394  7238 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:59.744822  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:59.764381  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:59.776681  6650 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:19:59.739044 13407 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:59.766052  6657 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:19:59.767527  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:19:59.781821  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:59.742498 13355 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 16. Current term is 17.
I20260709 12:19:59.744239 13509 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:59.746949 13440 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:59.792289 13440 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:19:59.792753  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:59.796720  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:59.797238  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:59.744937  6549 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:19:59.798017  6656 raft_consensus.cc:2393] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 20: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:19:59.798185  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 53290us
I20260709 12:19:59.776703  6579 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:59.801031  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:59.801827 13523 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:19:59.765880 13320 raft_consensus.cc:697] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:19:59.782764  6577 raft_consensus.cc:1275] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 12:19:59.803665  6577 pending_rounds.cc:85] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 10
I20260709 12:19:59.803936  6577 pending_rounds.cc:107] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 12:19:59.804168  6577 raft_consensus.cc:2889] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:59.804217 13320 consensus_queue.cc:237] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:59.805604 13549 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:59.808967  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:59.809758  6473 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:59.811128 13549 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:19:59.781212 13415 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:19:59.813669 13415 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:19:59.814987  6473 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:59.782760  6654 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 17.
I20260709 12:19:59.782263  7273 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:19:59.817451 12878 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:19:59.786283  6657 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:19:59.827653  7240 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:59.825016 13374 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:19:59.789678 13518 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:19:59.800215 13515 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:59.829710 13515 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:59.830180 13515 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:59.832072  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:59.838064  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:59.838670  6584 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:59.836954 13374 consensus_queue.cc:1048] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 12:19:59.849599 13335 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:19:59.850076 13335 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:59.850440 13335 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:19:59.851063  6654 raft_consensus.cc:1240] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:59.851930 13549 consensus_queue.cc:1048] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:19:59.852550 13518 raft_consensus.cc:697] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 17 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:19:59.837146 13419 consensus_queue.cc:1048] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 12:19:59.853463 13518 consensus_queue.cc:237] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:59.853988  6584 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:19:59.854909  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:59.855451  6584 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:19:59.856200  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:59.791100 13317 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:59.858860 13317 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:59.816025 13407 raft_consensus.cc:697] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:19:59.860136  6473 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:59.859939 13407 consensus_queue.cc:237] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:59.864923  6576 raft_consensus.cc:2410] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 15 because replica has last-logged OpId of term: 13 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 13.
I20260709 12:19:59.861871 13335 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:59.862053 13440 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:59.877178 13317 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:19:59.878010 13317 raft_consensus.cc:697] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:19:59.879154 13440 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:19:59.877071  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:19:59.896155  6654 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:19:59.866405  6473 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:59.866099  6584 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 25: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:19:59.899574  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:59.900388  6584 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:19:59.901453  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:59.902158  6584 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:19:59.903201  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:19:59.869488  6657 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
W20260709 12:19:59.871771  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 92163.7us
W20260709 12:19:59.892488  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:19:59.905689  6657 pending_rounds.cc:85] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 12
I20260709 12:19:59.907042  6657 pending_rounds.cc:107] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 12:19:59.907452  6657 raft_consensus.cc:2889] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:19:59.896354  6550 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:59.879040 13317 consensus_queue.cc:237] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:59.912030  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:59.910357  6623 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:19:59.910943  6473 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:59.913987  6584 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:19:59.865326 13509 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:59.913938  6550 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:59.922621  6656 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:19:59.925042  6550 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:19:59.915787  6473 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:19:59.923549  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 95979.6us
I20260709 12:19:59.914700 13549 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:59.913453 13440 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:59.928436 13554 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:19:59.934613 13415 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:19:59.922413 13374 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:59.943892  6584 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:19:59.920433 13289 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:19:59.921450 13508 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:19:59.898166 12878 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:19:59.932387 13581 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:19:59.928696 13580 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:19:59.952762  6654 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 25.
I20260709 12:19:59.933758 13298 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:19:59.923763 13464 consensus_queue.cc:1059] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 12:19:59.957202  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:19:59.955719  6473 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:19:59.977232  6584 raft_consensus.cc:2393] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 17: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:19:59.932232 13336 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 12:19:59.928229 13509 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:59.947736 13440 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:59.955078 13289 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:59.981810 13464 consensus_queue.cc:1243] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Peer 496c7c3792cd4587b4dbeba435f4be1e log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:19:59.954324 13554 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:19:59.957368 13581 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:59.982576 13522 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:19:59.990602 13467 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:19:59.961338  6550 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:00.005684  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 44427.9us
I20260709 12:19:59.955507 13374 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:59.941283 13529 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.037s
W20260709 12:20:00.001020  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 75974.6us
I20260709 12:19:59.946344 13320 consensus_queue.cc:1059] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:20:00.010404  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:19:59.950408 13549 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.011193 13523 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:00.021247 13335 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:00.023329  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:00.025331  6650 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:00.003624 13440 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:00.001048 13289 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:00.031279  6650 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 18.
I20260709 12:20:00.009923 13581 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:00.036559  7240 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:00.018915 13374 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:00.034284 13523 raft_consensus.cc:697] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:00.016403 13467 raft_consensus.cc:697] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:00.031805 13419 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:00.055814  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:00.043035 13508 raft_consensus.cc:3055] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Stepping down as leader of term 19
W20260709 12:20:00.059660  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.060482 13419 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.062644 13288 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:00.063289 13549 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:00.063575 13467 consensus_queue.cc:237] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.051327 13518 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:00.065940 13508 raft_consensus.cc:740] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:20:00.076515  6475 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.823s	user 0.035s	sys 0.033s
W20260709 12:20:00.077524  6475 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.824s	user 0.036s	sys 0.033s
W20260709 12:20:00.078183  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 844491us
I20260709 12:20:00.079463  6548 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:00.068042 13317 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:00.081166  6548 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:20:00.086355  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.066103 13482 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.088608  6656 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:00.082549 13529 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:00.086658 13320 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:00.076129 13523 consensus_queue.cc:237] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.089322 13518 raft_consensus.cc:697] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 18 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:00.089885 13549 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:00.102048 13549 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.107410  6548 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:00.111541 13515 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:00.113487 13567 log.cc:927] Time spent T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 1.205s	user 0.008s	sys 0.000s
I20260709 12:20:00.114750  7270 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:20:00.115324 13577 log.cc:927] Time spent T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.729s	user 0.000s	sys 0.001s
I20260709 12:20:00.100672 13508 consensus_queue.cc:260] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:00.105458 13606 log.cc:927] Time spent T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 12:20:00.122861  6505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41304 (request call id 864) took 1028 ms. Trace:
I20260709 12:20:00.123226  6505 rpcz_store.cc:276] 0709 12:19:59.094106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:59.095829 (+  1723us) service_pool.cc:224] Handling call
0709 12:20:00.122680 (+1026851us) spinlock_profiling.cc:243] Waited 512 ms on lock 0x7b540001ccd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:00.122806 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1178731264}
I20260709 12:20:00.124687  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:00.125480  6657 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:20:00.127844  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.087477 13317 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.121032  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:00.130800  6650 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 25: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:20:00.121858  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.131819  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:00.129554  7270 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 12:20:00.132550  7270 pending_rounds.cc:85] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9: Aborting all ops after (but not including) 13
I20260709 12:20:00.132784  7270 pending_rounds.cc:107] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 12:20:00.132989  7270 raft_consensus.cc:2889] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:20:00.133178  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.119472  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:00.135115  6656 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:20:00.136489  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:00.137323  6650 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
W20260709 12:20:00.119890 13607 log.cc:927] Time spent T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 12:20:00.143177  6577 raft_consensus.cc:2393] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 18: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:00.145879  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:00.134069 13508 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:00.147430 12878 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
W20260709 12:20:00.148476 13608 log.cc:927] Time spent T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:20:00.130584 13320 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:00.158183 13549 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:00.158818 13549 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:00.159276 13549 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.161247 13482 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:00.162668  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:00.164639  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.166148  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:00.167146  6656 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:20:00.161631 13554 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:00.177924  6657 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:20:00.185505  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:00.186456  6656 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:00.190272  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:00.196693  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:00.199404  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.188262 13440 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.202831 13440 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:00.204754 13482 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:00.205492 13482 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.174558 13320 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.209410 13320 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.208854 13419 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:00.212811  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:00.198312  6657 raft_consensus.cc:2410] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:20:00.214236  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:00.221997  6576 raft_consensus.cc:2393] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 18: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:00.222929  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:00.229159  6576 raft_consensus.cc:2410] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:20:00.230458  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:00.231513  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:00.231987  6576 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:00.232658  6658 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 2803) took 1848 ms (client timeout 2050 ms). Trace:
W20260709 12:20:00.233100  6658 rpcz_store.cc:269] 0709 12:19:58.383770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:58.384759 (+   989us) service_pool.cc:224] Handling call
0709 12:20:00.228648 (+1843889us) spinlock_profiling.cc:243] Waited 408 ms on lock 0x7b54000b5c54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:00.232590 (+  3942us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":937730304}
I20260709 12:20:00.233673  6656 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:20:00.212116 13320 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:00.208112 13482 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:00.250335 13529 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:20:00.250633 13598 log.cc:927] Time spent T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.512s	user 0.000s	sys 0.002s
I20260709 12:20:00.255172 13317 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:00.259143 13581 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.229218 13371 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:00.268761  6580 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:00.265625 13467 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.248155 13515 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:00.267495 13289 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:00.288056  6576 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:20:00.290175  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:00.291477  6576 raft_consensus.cc:2393] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 16: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:00.292840  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:00.293895  6576 raft_consensus.cc:2393] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 20: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:00.266250 13419 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.304282 13616 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:00.305199 13616 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:00.298998 13467 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:00.309806  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.310240 13467 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:00.299965 13440 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:00.312693 13424 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:00.313508 13440 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:00.314677 13427 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.315310 13427 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.316180 13440 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:00.317200  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.318157  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:00.320077  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:00.320207 13549 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:00.320827  6503 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:20:00.284996 13149 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:00.321810 13149 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:00.334406  6656 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:20:00.329291  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:00.333124  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:00.338028  6506 raft_consensus.cc:2393] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 21: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:00.339814  6506 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:00.329707  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:00.341022  6581 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:00.341569  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:00.328696  6505 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:00.345018  6508 raft_consensus.cc:1275] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 20 index: 9. Preceding OpId from leader: term: 22 index: 10. (index mismatch)
I20260709 12:20:00.234993 13518 consensus_queue.cc:237] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:00.346613  6580 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:20:00.346992  6507 raft_consensus.cc:1275] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:20:00.281458 13529 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.350853 13335 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:20:00.306106 13581 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:00.359566  6581 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:20:00.281150 13371 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.361202  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:00.361294  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:00.343261 13423 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:00.362588  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:00.369519  6503 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:20:00.369783  6506 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:00.372430  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:00.372869  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:00.373188  6500 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 14.
I20260709 12:20:00.325481 13419 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.366506 13149 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:00.367362 13616 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:00.366082  6474 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:00.376845  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:00.381758  6507 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:20:00.383818  6500 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 14.
I20260709 12:20:00.385527  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:00.386119  6575 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:20:00.386878  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:00.388240  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:00.374960 13615 log.cc:927] Time spent T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 12:20:00.396200  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:00.395123  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:00.398627  6506 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:20:00.398100  6475 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:00.360008 13529 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.410404  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 3098) took 1209 ms. Trace:
I20260709 12:20:00.410998  6475 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:00.411033  6501 rpcz_store.cc:276] 0709 12:19:59.201134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:59.272276 (+ 71142us) service_pool.cc:224] Handling call
0709 12:20:00.410200 (+1137924us) spinlock_profiling.cc:243] Waited 27.9 ms on lock 0x7b540001c2d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:00.410347 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64196352}
I20260709 12:20:00.412225  6575 raft_consensus.cc:1275] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 12:20:00.412724  6575 pending_rounds.cc:85] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 10
I20260709 12:20:00.412985  6575 pending_rounds.cc:107] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 12:20:00.413323  6575 raft_consensus.cc:2889] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:00.393129 13581 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.420531 13419 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:00.422685 13289 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:00.398996 13374 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.425503 13374 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:00.437134  6509 raft_consensus.cc:2393] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 16: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:00.437801  6575 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:20:00.374167 13423 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.411880 13618 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:00.443687  6475 leader_election.cc:400] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 17. Current term is 18.
I20260709 12:20:00.406850 13616 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:20:00.463193  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.447785  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:00.447423  6475 leader_election.cc:403] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:00.464775  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:00.447484  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:00.466243  7270 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:00.448550 13419 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:00.451364 13415 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:00.458377  6583 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:20:00.474586  6583 raft_consensus.cc:606] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:00.475003  6583 raft_consensus.cc:614] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:00.461413  6501 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:00.464118  6581 raft_consensus.cc:2410] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 24 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 12:20:00.400188 13427 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:00.478875  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:00.485911  6501 raft_consensus.cc:2393] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 23: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:20:00.484647  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:00.484753  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:00.490301  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.493441  6502 raft_consensus.cc:1275] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:20:00.489127  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:00.505127  6622 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:00.502934 13592 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:00.486476 13407 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.444375 13581 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.483651 13621 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 17. Current term is 18.
I20260709 12:20:00.509590 13317 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.514964 13317 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.516597 13482 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.517194 13482 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.506153 13289 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.520246 13289 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:00.521028 13582 log.cc:927] Time spent T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 1.132s	user 0.001s	sys 0.003s
I20260709 12:20:00.523310 13289 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:00.506377 13288 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:00.524323 13288 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:00.500873 13419 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.528728 13419 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:20:00.529524  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:00.529913  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:00.530692  6654 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:20:00.531519 13419 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:00.531726  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:00.509785 13435 raft_consensus.cc:2764] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
I20260709 12:20:00.509647 13555 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:00.533118 13555 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:00.533542 13555 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:00.534714  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:00.535019 13509 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:00.535652  6657 raft_consensus.cc:2393] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 20: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:00.535557 13407 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.537974 13407 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:00.540282  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:20:00.508597 13586 log.cc:927] Time spent T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8: Append to log took a long time: real 1.008s	user 0.007s	sys 0.003s
I20260709 12:20:00.542032  6650 raft_consensus.cc:1275] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 12:20:00.542279  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:00.548194  6650 pending_rounds.cc:85] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 12
I20260709 12:20:00.543018  6500 raft_consensus.cc:1275] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:20:00.548643  6650 pending_rounds.cc:107] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 12:20:00.545533  7270 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:20:00.545538  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:00.548660  6658 raft_consensus.cc:2410] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 24 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 12:20:00.549530  6649 raft_consensus.cc:2393] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 20: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:00.546725  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:00.545312  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:00.549088  6650 raft_consensus.cc:2889] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:00.551961  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:00.550034  6623 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:00.552567  6650 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:20:00.553212  7238 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:20:00.551342  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 48935.5us
I20260709 12:20:00.554327  6649 raft_consensus.cc:1275] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:20:00.553894  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:00.556113  6622 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:00.557420  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:00.561929  6509 raft_consensus.cc:1240] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:00.562911  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:00.563535  6655 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:20:00.566560  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:00.567236  6658 raft_consensus.cc:1275] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:20:00.568535  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:00.568696  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:00.564365  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:00.569725  6503 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:00.569366 13449 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:00.570756 13449 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:00.570912 13623 log.cc:927] Time spent T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:20:00.571201 13449 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:00.572144  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:00.575340 13423 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:00.575276  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:00.579128  6509 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
W20260709 12:20:00.590420  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.583766  6475 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:00.600234  6475 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:00.559262 13407 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:20:00.556485 13288 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:00.605017  6475 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:00.608505  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:20:00.572209 13622 log.cc:927] Time spent T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:20:00.609901  6507 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 12:20:00.625991  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.580413 13552 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:00.627555  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:00.633625  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.559271 13435 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 12:20:00.635553  6505 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:20:00.608709 13415 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:20:00.638940  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 82921.7us
I20260709 12:20:00.640621  6475 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:00.610689 13482 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:00.612949 13416 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:20:00.657883  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 97587.1us
I20260709 12:20:00.658300  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:00.632166 13424 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:00.636740 13552 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:00.662853  7238 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:00.618371 13419 consensus_queue.cc:1048] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 12:20:00.639426 13407 raft_consensus.cc:697] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:00.654353 13621 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:00.666706 13372 consensus_queue.cc:1048] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:20:00.693295 13581 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:00.695056 13407 consensus_queue.cc:237] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:00.665314 13529 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:00.660161 13416 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:00.660964 13415 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:00.703199 13313 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:00.666296 13595 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 12:20:00.709149 13520 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:00.733124 13520 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:00.709576 13449 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:00.687114 13552 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:00.634902 13427 consensus_queue.cc:1048] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:20:00.726078 13555 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.719936 13416 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:00.721371 13415 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:00.731552 13371 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:00.723608 13618 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:00.733659 13520 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:00.731593 13317 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:00.747337 13449 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:00.756587 13555 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:00.756091 13552 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.759049 13552 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:00.792435  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 151888us
I20260709 12:20:00.812495  6509 raft_consensus.cc:2410] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 25 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 12:20:00.812589  6507 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:20:00.857342  6649 raft_consensus.cc:2393] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 16: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:20:00.861166  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:00.862317  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:00.864312  6657 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
W20260709 12:20:00.867266  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 208968us
I20260709 12:20:00.870069 13288 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:00.871160 13288 raft_consensus.cc:697] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:00.873042 13138 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:00.873390 13288 consensus_queue.cc:237] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.874334 13138 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.881413 13289 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:00.882869 13289 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.887393 13419 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.888515 13419 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.868104  6654 raft_consensus.cc:2393] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 21: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:00.880962 13355 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:00.883136 13374 consensus_queue.cc:1048] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:20:00.880858 13317 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:00.896884 13317 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:00.909174 13419 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:00.908861 13355 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.912644 13424 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.913303  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:00.929056  6582 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
W20260709 12:20:00.932523  6582 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 2781) took 1690 ms (client timeout 1589 ms). Trace:
W20260709 12:20:00.933852  6582 rpcz_store.cc:269] 0709 12:19:59.242110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:59.242274 (+   164us) service_pool.cc:224] Handling call
0709 12:20:00.932030 (+1689756us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b54002661d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:00.932261 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3708845952}
I20260709 12:20:00.947928 13631 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.949335 13631 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.950024 13289 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:00.952314 13631 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:00.969830  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54204 (request call id 1088) took 1169 ms. Trace:
I20260709 12:20:00.970273 13449 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:00.920271 13317 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:00.973443 13424 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.974577  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 1076) took 1638 ms. Trace:
I20260709 12:20:00.982803 13633 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:00.987057  6577 rpcz_store.cc:276] 0709 12:19:59.800318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:59.808715 (+  8397us) service_pool.cc:224] Handling call
0709 12:20:00.969194 (+1160479us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b540034dfd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:00.969537 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":766399232}
I20260709 12:20:01.009035 13633 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:00.992723  6578 rpcz_store.cc:276] 0709 12:19:59.336520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:59.372656 (+ 36136us) service_pool.cc:224] Handling call
0709 12:19:59.372995 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:19:59.450944 (+ 77949us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:19:59.589495 (+138551us) log.cc:844] Serialized 14 byte log entry
0709 12:19:59.680544 (+ 91049us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:19:59.680593 (+    49us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:19:59.680635 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:00.112179 (+431544us) spinlock_profiling.cc:243] Waited 1.15 ms on lock 0x7b18003981a0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d6a2a6 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419
0709 12:20:00.250072 (+137893us) spinlock_profiling.cc:243] Waited 2.58 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:20:00.973820 (+723748us) spinlock_profiling.cc:243] Waited 77.7 ms on lock 0x7b54004204d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:00.973942 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:00.974298 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5225,"prepare.queue_time_us":139969,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":4246,"raft.queue_time_us":5374,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":2117,"spinlock_wait_cycles":187296768,"thread_start_us":167399,"threads_started":2,"wal-append.queue_time_us":97888}
I20260709 12:20:01.014053 13138 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:01.002954 13424 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:01.038794 13355 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:01.039880  6505 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:20:01.050786  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.051587  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:01.059475  6505 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
I20260709 12:20:01.061811  6475 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:01.065338  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:01.066615  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:01.077909 13633 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:01.078342 13374 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:01.078858 13374 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:01.079375 13374 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:20:01.080889  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 104519us
I20260709 12:20:01.093407  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:01.094415  6658 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:20:01.100726  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:01.111621  6473 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:01.108517  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:01.114401 13416 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:01.134963 13416 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:01.135406  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:01.136376  6658 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:20:01.126428 13424 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:01.163303  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:01.165234  6505 raft_consensus.cc:2393] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 15: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:20:01.169598  6652 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:01.172505  6653 raft_consensus.cc:1240] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:01.188278  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:01.189491  6650 raft_consensus.cc:2393] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 18: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:20:01.192655 13355 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:01.193938 13355 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:01.197474 13355 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:01.154767 13138 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:01.201047 13138 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:01.203292 13138 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:01.207304  6473 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:01.217758  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 128870us
I20260709 12:20:01.215847  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:01.224653  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 189041us
I20260709 12:20:01.206382 13374 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:01.239176  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:01.242580  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.230867 13416 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:01.247776 13355 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:01.248582  6507 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 23.
I20260709 12:20:01.249083  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:01.252241  6650 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:20:01.265048  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:01.265067 13508 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:01.267244 13508 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:20:01.269209 13427 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:01.270160 13508 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:01.270026 13427 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:01.271406  6505 raft_consensus.cc:2393] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 21: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:01.275807  6653 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:20:01.272593 13355 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:01.256248 13549 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:20:01.260082 13641 log.cc:927] Time spent T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 12:20:01.283144  6654 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:20:01.286468 13289 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:01.291410 13289 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:01.293881 13289 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:01.302287  6550 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:01.313990  7273 raft_consensus.cc:1275] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 12:20:01.318168  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 99169.6us
W20260709 12:20:01.332986  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 172431us
W20260709 12:20:01.334292  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.332860 13374 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:01.335045  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:01.346351 13355 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:01.347199  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:01.354346  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:01.355858 13520 consensus_meta.cc:287] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.608s	user 0.003s	sys 0.012s
I20260709 12:20:01.371701  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:01.383808  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:01.386844  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 82563.1us
I20260709 12:20:01.383193 13289 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
W20260709 12:20:01.391170 13648 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 12:20:01.392823  6653 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:20:01.324700 13415 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:01.394272 13427 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:01.394985 13647 log.cc:927] Time spent T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.171s	user 0.002s	sys 0.000s
I20260709 12:20:01.395323 13554 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:01.383694 13355 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
W20260709 12:20:01.400802  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 110440us
I20260709 12:20:01.404111 13288 consensus_queue.cc:1059] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:20:01.405241  6656 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:01.324290 13549 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:01.406435 13633 raft_consensus.cc:3055] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:01.406940 13633 raft_consensus.cc:740] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:01.408306 13549 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:01.409771  6473 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:01.412525  6475 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:01.411329 13355 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:01.415578 13650 log.cc:927] Time spent T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.122s	user 0.000s	sys 0.002s
I20260709 12:20:01.422088 13355 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:01.385159 13520 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:01.431248 13520 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:01.431356 13355 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:01.432305  7265 raft_consensus.cc:1240] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:01.433320 13355 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:01.435048 13317 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:01.389884 13508 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:20:01.436972  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 89968us
I20260709 12:20:01.452358  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:01.439992  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.395344 13415 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:01.431695 13645 log.cc:927] Time spent T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.240s	user 0.003s	sys 0.000s
I20260709 12:20:01.454993 13374 consensus_queue.cc:1059] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 12:20:01.471928 13618 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:01.478050 13467 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:01.478425 13424 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:01.472908  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:01.459828 13508 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:01.404939 13653 consensus_queue.cc:1059] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 12:20:01.483819  7273 raft_consensus.cc:2393] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 15: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:20:01.486964 13621 raft_consensus.cc:3055] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:01.487454 13621 raft_consensus.cc:740] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:01.488637 13621 consensus_queue.cc:260] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:01.481504 13424 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:01.493482 13508 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:20:01.519443  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 78221.6us
I20260709 12:20:01.503288 13424 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:01.485317 13355 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:01.464835 13633 consensus_queue.cc:260] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:01.511515 13621 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:01.523267 13640 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
W20260709 12:20:01.533380 13644 log.cc:927] Time spent T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.342s	user 0.000s	sys 0.005s
I20260709 12:20:01.544133  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:01.552992  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 79085.4us
W20260709 12:20:01.554128  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 144708us
I20260709 12:20:01.558197  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:01.557399  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:01.562835 13633 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:01.566378  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:01.567021  6473 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:01.569602 13427 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:01.570411 13427 raft_consensus.cc:697] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:01.571671  7238 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:01.574477 13449 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:20:01.575672  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:01.572184  6507 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:01.525888 13653 consensus_queue.cc:1243] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer f2b06aebe97249a4877d798b8118ece4 log is divergent from this leader: its last log entry 16.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:20:01.541170 13631 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:20:01.571689 13427 consensus_queue.cc:237] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:01.596622 13317 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:01.593937 13652 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:01.594553 13518 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:20:01.609294  6650 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
W20260709 12:20:01.610423  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:01.610385  6548 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:01.645649  6475 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:01.648994  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 140538us
I20260709 12:20:01.656837  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:01.657783  6657 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:01.654838  6550 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:01.656807  6475 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:01.654978  6473 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:01.666883  6650 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:20:01.624598 13138 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:01.692281 13467 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:01.693737 13649 log.cc:927] Time spent T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.407s	user 0.003s	sys 0.000s
I20260709 12:20:01.693379 13467 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:01.695976 13467 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:01.696432 13374 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:01.665992 13415 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:01.708624  6654 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:01.705964  6507 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
I20260709 12:20:01.677688 13549 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:01.724831 13660 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:01.725945 13660 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:01.730091 13654 raft_consensus.cc:3055] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:20:01.730860 13654 raft_consensus.cc:740] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:01.742447 13654 consensus_queue.cc:260] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:01.745019 13654 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:01.757539 13355 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:01.758283 13355 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:01.758905 13355 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:01.775115 13424 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:01.779520 13654 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:01.659360 13652 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:01.743214 13653 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:01.785674  7270 raft_consensus.cc:1240] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:01.782128  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:01.800442  6658 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
W20260709 12:20:01.745223 13651 log.cc:927] Time spent T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.159s	user 0.000s	sys 0.002s
I20260709 12:20:01.713785 13416 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:01.806944 13653 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:01.801857 13518 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.008s
W20260709 12:20:01.810783  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 259782us
I20260709 12:20:01.785342 13549 raft_consensus.cc:697] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:01.810299  6622 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:01.799371 13654 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:01.828531  6475 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:01.831368  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 50592.1us
I20260709 12:20:01.806267 13660 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:01.850185  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 199885us
I20260709 12:20:01.807993 13653 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:01.833431 13654 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:01.863545 13653 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:01.869752 13640 consensus_queue.cc:1059] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:01.870314 13592 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:01.871192 13631 raft_consensus.cc:3055] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:01.871675 13631 raft_consensus.cc:740] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:01.872535 13288 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:01.873521 13288 raft_consensus.cc:697] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:01.872987 13631 consensus_queue.cc:260] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:01.875059 13631 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:01.876529 13581 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:01.877883 13581 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:01.903476  6654 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:20:01.845155 13416 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:01.875186 13288 consensus_queue.cc:237] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:01.931347  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 69641.9us
I20260709 12:20:01.950301 13661 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:01.951349 13661 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:01.954548 13661 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:01.932118 13416 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:01.971542 13416 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:20:01.992540  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 309095us
I20260709 12:20:02.000267 13138 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:01.935482 13652 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:02.000648  7238 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:20:01.969133  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 123337us
I20260709 12:20:02.012385  6473 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:02.007784  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 354992us
I20260709 12:20:01.942061 13374 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:01.941071 13427 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:02.011366 13355 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:02.020830 13652 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 12:20:02.041668  6548 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:20:02.043143  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:02.016800 13449 raft_consensus.cc:697] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 15 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
W20260709 12:20:02.043965  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:02.027271 13518 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:20:02.077630  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 277056us
I20260709 12:20:02.035295 13508 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:02.087196 13374 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:02.088616  6657 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:20:02.084586  6550 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:02.151976  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 203194us
I20260709 12:20:02.074398 13317 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:02.050774 13652 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:20:02.171612  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 100478us
W20260709 12:20:02.151959 13633 consensus_meta.cc:287] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.584s	user 0.011s	sys 0.002s
W20260709 12:20:02.179791  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 109164us
I20260709 12:20:02.154562 13661 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:02.146766 13518 raft_consensus.cc:697] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 15 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:02.030618 13549 consensus_queue.cc:237] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:02.196730  6577 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:20:02.070691 13355 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.104s
I20260709 12:20:02.221335  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:02.070458 13427 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:02.183606 13653 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:02.265038  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:02.265117 13653 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:02.267882  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:02.271248  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:02.272506 13684 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:02.153586 13138 consensus_queue.cc:1048] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
W20260709 12:20:02.237995  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 232299us
W20260709 12:20:02.239178  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:02.222894 13317 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:02.242673  6584 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:20:02.244477  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:02.247789 13673 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:02.251905  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:02.252916  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:02.252650 13449 consensus_queue.cc:237] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.14, Last appended by leader: 14, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:02.297991  6583 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:20:02.253865  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:02.255143  6582 raft_consensus.cc:1275] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:20:02.257431 13508 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:02.257934 13427 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:02.261134 13662 log.cc:927] Time spent T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:20:02.262276 13518 consensus_queue.cc:237] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:02.235855 13660 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:20:02.263659  6656 consensus_meta.cc:287] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.679s	user 0.004s	sys 0.004s
I20260709 12:20:02.234901 13652 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:02.252030 13621 consensus_meta.cc:287] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.700s	user 0.010s	sys 0.000s
I20260709 12:20:02.285059  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:02.285760  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:02.221639 13661 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:02.289311  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:02.290246  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:02.221110 13374 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:02.225665 13581 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:02.293637 13416 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:02.296761  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 80990.2us
I20260709 12:20:02.278350  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:02.298595  6580 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22.
I20260709 12:20:02.300375  6575 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:20:02.310360  6576 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:02.268153 13653 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:02.328933  6656 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:20:02.332664  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:02.362428  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 74615.3us
I20260709 12:20:02.364789  6583 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:20:02.365336  6578 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:02.333230  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:02.363720  6577 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:20:02.375254  6579 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:20:02.355340 13660 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:02.381626  6507 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:20:02.356532 13684 raft_consensus.cc:697] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:02.394893  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:02.405225  6584 raft_consensus.cc:2410] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 22 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
W20260709 12:20:02.404166  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 133158us
W20260709 12:20:02.406975  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 187698us
I20260709 12:20:02.309201 13673 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:02.393633 13660 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:02.405452 13688 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:20:02.436291  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:02.396831 13652 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:02.442215  7265 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:20:02.420537 13677 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:02.428505 13618 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:02.467644  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:02.467919 13663 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 12:20:02.470113 13658 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:20:02.471953  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:02.429327  6475 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:02.476184  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:02.429126 13427 consensus_queue.cc:1048] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
W20260709 12:20:02.487363  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 178709us
I20260709 12:20:02.490391  6509 raft_consensus.cc:2393] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 18: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:02.497097  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:02.498797  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 69734.8us
I20260709 12:20:02.481566 13665 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:02.585426  6501 raft_consensus.cc:1275] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:20:02.505239 13675 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:02.581853 13653 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:02.598269 13653 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:02.601540 13653 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:02.602566 13653 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:02.504405 13663 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:02.609139  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 152138us
W20260709 12:20:02.615253  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 44729.2us
I20260709 12:20:02.581840 13518 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:02.574282 13658 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:02.620056 13658 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:02.490023 13618 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:02.594905 13581 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:02.609733 13317 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:02.610008 13676 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:02.611915 13416 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:02.611356 13675 raft_consensus.cc:697] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:02.635408  6578 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:20:02.617173 13374 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:02.632273 13694 log.cc:927] Time spent T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 12:20:02.570768 13677 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:02.699934  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 78209.2us
W20260709 12:20:02.700078  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 61436.8us
I20260709 12:20:02.611063 13665 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:02.636428 13661 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:02.587010 13684 consensus_queue.cc:237] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:02.684293 13700 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:02.627391 13518 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:02.620644 13663 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:02.699951 13658 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:02.705504 13621 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:02.735147  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:02.698439 13676 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:02.721791 13673 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:02.739459 13658 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:02.705749 13581 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:02.742187  6473 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:02.757856 13621 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:02.761670  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 47822.6us
I20260709 12:20:02.764367  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:02.745491 13661 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:02.727900 13700 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:02.758800 13704 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:02.776039  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:20:02.768491 13671 log.cc:927] Time spent T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8: Append to log took a long time: real 0.333s	user 0.002s	sys 0.000s
I20260709 12:20:02.788522  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:02.767632 13687 log.cc:927] Time spent T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.296s	user 0.001s	sys 0.000s
I20260709 12:20:02.748137  6505 raft_consensus.cc:2393] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 20: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:02.782976  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:02.737005 13652 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:02.802800 13707 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:02.769476 13665 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:02.767481 13703 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:02.803582 13707 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:02.803843 13703 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:02.791314  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:02.807185 13707 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:02.807963 13690 log.cc:927] Time spent T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
I20260709 12:20:02.808552  6549 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:02.806895 13703 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:02.809948  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 100798us
I20260709 12:20:02.810987  6549 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:02.812026 13665 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:20:02.744976 13675 consensus_queue.cc:237] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:02.793834  6509 raft_consensus.cc:2393] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 16: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:02.794452 13663 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:02.795401 13654 consensus_meta.cc:287] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.850s	user 0.012s	sys 0.000s
I20260709 12:20:02.795571  6504 raft_consensus.cc:2410] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
W20260709 12:20:02.817181 13691 log.cc:927] Time spent T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
I20260709 12:20:02.817822  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:02.818755  6500 raft_consensus.cc:2393] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 22: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:02.819870  6506 raft_consensus.cc:1275] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 12:20:02.821310 13706 log.cc:927] Time spent T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 12:20:02.825105 13518 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:02.828655 13621 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:02.831537  6578 raft_consensus.cc:1275] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 19 index: 8. Preceding OpId from leader: term: 22 index: 10. (index mismatch)
I20260709 12:20:02.831926  6584 raft_consensus.cc:1275] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:20:02.834074  6578 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:02.835063 13424 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:02.836236  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:02.836622  6577 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:02.837218  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:02.838093  6580 raft_consensus.cc:2410] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:20:02.838373  6579 raft_consensus.cc:1275] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:20:02.839656  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:02.802976 13677 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:02.765889 13712 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:02.841058 13712 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:02.841495  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:02.842163  6584 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:02.810027 13703 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:20:02.848410  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:02.850222 13549 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:02.850971 13549 raft_consensus.cc:697] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:02.794786 13708 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:02.852043 13549 consensus_queue.cc:237] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:02.852946 13708 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:02.858850  6576 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
I20260709 12:20:02.814399  6549 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:20:02.796017  6622 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.589s (SENT) [suppressed 12 similar messages]
I20260709 12:20:02.793532 13699 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:02.840312  6578 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:20:02.819716  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 77679us
I20260709 12:20:02.868518  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:02.786020 13661 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:02.877911  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:02.889366  6576 raft_consensus.cc:2410] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:20:02.802202 13704 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:02.895573 13704 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:02.899047  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:02.899242 13710 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:02.900126 13710 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:02.903857 13710 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:02.817230 13654 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:02.872206 13618 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:02.907289  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:02.909744  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 95434.6us
I20260709 12:20:02.887527 13508 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:02.915534  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:02.916822  6576 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:20:02.922499 13676 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:02.910538 13682 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:02.923599 13682 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:02.941330 13720 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:02.942289 13720 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:02.945303 13720 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:02.951865  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:02.952133  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:02.951915  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:02.908877 13699 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:02.956921 13654 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:02.957257 13699 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:02.957798  6473 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:02.959846  6473 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:02.960522 13677 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:02.961207 13677 consensus_queue.cc:1048] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:20:02.962492 13605 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.013s
W20260709 12:20:02.967146  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 171157us
I20260709 12:20:02.969125  6578 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:20:02.952260 13710 consensus_queue.cc:1059] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 12:20:02.980125  6584 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:20:02.976524  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:02.992715  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:02.994263  6580 raft_consensus.cc:2368] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 20. Current term is 21.
I20260709 12:20:02.981711 13677 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:03.004671 13700 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:03.005012 13677 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:03.005705 13677 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:03.011895 13676 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:20:03.013785 13661 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:03.014448  6473 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:03.014740 13708 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:03.016533  6473 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:03.016696 13467 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:03.017097 13467 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:03.017457 13467 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:20:03.019250 13660 consensus_meta.cc:287] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.577s	user 0.010s	sys 0.000s
I20260709 12:20:03.020918 13658 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:03.023449  6584 raft_consensus.cc:1275] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:20:03.023749 13689 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:03.024799 13712 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:03.024744  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:03.026044  6578 raft_consensus.cc:2393] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 16: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:20:03.025242 13680 log.cc:927] Time spent T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.776s	user 0.005s	sys 0.001s
I20260709 12:20:03.037401  6473 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:03.024219 13689 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:03.019861 13660 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:03.052439  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:03.052440  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:03.053189  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:03.053480  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:03.059613  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:03.064844 13692 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:03.065277  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:03.065507 13692 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:03.065991  6575 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 23.
I20260709 12:20:03.067327 13682 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:03.067714 13692 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:03.051942 13689 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:03.072600  6658 raft_consensus.cc:1275] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:20:03.072713  6580 raft_consensus.cc:2393] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 21: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:03.075191  6473 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:03.052706 13424 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:03.076231  6583 raft_consensus.cc:2368] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 23. Current term is 24.
I20260709 12:20:03.077342 13518 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:03.080950  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:03.080579  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:03.082032  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:03.085215  6657 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:03.085958 13518 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:03.091775 13692 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:03.077620 13581 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:03.092409 13692 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:03.093649 13518 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:03.094791 13692 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:03.100680  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 121397us
I20260709 12:20:03.104449  6473 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:03.109635  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:03.079025 13424 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:03.124643  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:03.123363  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:03.160280  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 55956.4us
I20260709 12:20:03.161191  6656 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:03.163762  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:03.173215 13660 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:03.124681 13697 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:20:03.159675 13508 raft_consensus.cc:3055] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:20:03.114030 13424 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:03.178905 13673 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:03.179682  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:03.181120  6658 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
W20260709 12:20:03.184041  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:03.188660 13675 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:03.174850  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:03.173776 13684 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:03.197850  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:03.203392  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:03.222695  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:03.202339  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:03.226783  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:03.227478  6578 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:03.229825  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:03.182931 13581 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:03.185017 13676 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:03.231643  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:03.239316  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:03.239343  6580 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 19.
I20260709 12:20:03.192576 13508 raft_consensus.cc:740] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:03.228667 13138 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:03.252364  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:03.247468  6475 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:03.259212  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:03.259433  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:03.260752  6475 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:03.265576  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:03.221405 13673 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:03.268298  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:03.270320  6658 raft_consensus.cc:2410] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:20:03.264415  6657 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:20:03.250294 13697 raft_consensus.cc:697] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:03.276372  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:03.317014  6657 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:20:03.281387  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:03.321668  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:03.323073  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:03.230738 13684 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:03.323792  6578 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:20:03.324774  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:03.325335  6578 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:20:03.325675 13684 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:03.228190 13675 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:03.282320  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:03.326110  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:03.327118  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:03.327222  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:03.327775  6658 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
I20260709 12:20:03.328315 13675 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:03.288398 13716 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:03.329290  6656 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22.
I20260709 12:20:03.329206 13716 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:03.330181  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:03.330744  6656 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:03.331434 13716 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:03.331578  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:03.289142 13701 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:20:03.290184 13725 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 12:20:03.294739  6550 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:03.300684  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:03.301950 13727 log.cc:927] Time spent T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 12:20:03.280560  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:03.264859 13518 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:03.250557 13581 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:03.315106  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 54436.8us
I20260709 12:20:03.328653  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:03.272468 13618 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:03.323659  6650 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:20:03.332157  6656 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:20:03.337843  6578 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:20:03.266049 13138 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:03.346168  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:03.354738 13729 log.cc:927] Time spent T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:20:03.365182 13744 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:20:03.369107  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:03.369902  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:03.371913 13581 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:03.372483  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:03.372782  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:03.387843  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:03.388718  6583 raft_consensus.cc:2376] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20. Re-sending same reply.
I20260709 12:20:03.344235 13518 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:03.390864  6657 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:20:03.393179  6651 raft_consensus.cc:2393] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 17: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:03.399006  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:03.399175 13737 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:03.399674 13744 raft_consensus.cc:697] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:03.399842 13737 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:03.400825  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:03.400852  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:03.400867 13744 consensus_queue.cc:237] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:03.402833  6656 raft_consensus.cc:2368] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 21. Current term is 22.
W20260709 12:20:03.404117 13733 log.cc:927] Time spent T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:20:03.404170  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:03.406680 13728 log.cc:927] Time spent T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 12:20:03.407581  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:03.408787 13138 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:03.409086  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:03.409518 13742 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:03.410315  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:03.410174 13742 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:03.411099  6656 raft_consensus.cc:2393] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 20: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:03.411945  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:03.332320 13701 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:03.402518  6583 raft_consensus.cc:2393] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 18: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:03.362890 13682 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:03.434291  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:03.439113  6581 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:20:03.396904 13518 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:03.454581 13424 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:03.451256  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:03.453003  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:03.453250  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:03.463143  6474 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:03.402441 13737 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:03.479806  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:03.481204  6583 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
W20260709 12:20:03.456173 13735 log.cc:927] Time spent T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:20:03.491019  6651 raft_consensus.cc:1240] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:20:03.494161  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:03.504218  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:03.507493  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:03.508132  6583 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:20:03.508983  6505 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 13. (term mismatch)
I20260709 12:20:03.509405  6505 pending_rounds.cc:85] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 12
I20260709 12:20:03.509238  6650 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 24.
I20260709 12:20:03.484440 13701 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:03.513669  6581 raft_consensus.cc:2393] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 16: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:03.375427 13508 consensus_queue.cc:260] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:03.517899  6504 raft_consensus.cc:1240] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:03.518422  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:03.520000  6578 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:20:03.522460  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 19 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:03.484658 13742 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:03.532940  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:03.445715 13682 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:03.493721 13618 raft_consensus.cc:697] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:03.528641  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:03.545269  7240 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:03.490993 13684 consensus_queue.cc:1048] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
W20260709 12:20:03.516592 13677 consensus_meta.cc:287] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.510s	user 0.007s	sys 0.007s
I20260709 12:20:03.534341  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:03.540891  6501 raft_consensus.cc:1240] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:20:03.541690  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:03.551685  6583 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:20:03.524204 13701 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:03.416553 13697 consensus_queue.cc:237] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:03.556562  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:03.509724  6505 pending_rounds.cc:107] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 23.13
I20260709 12:20:03.557415  6578 raft_consensus.cc:2368] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 19. Current term is 20.
I20260709 12:20:03.557672  6505 raft_consensus.cc:2889] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:03.559347  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:03.520552 13467 consensus_meta.cc:287] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.502s	user 0.010s	sys 0.002s
I20260709 12:20:03.561538 13467 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:20:03.513131 13737 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:03.534191  6622 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:03.568970  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:03.569924  6581 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:20:03.572019  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:03.573064  6581 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:20:03.579623  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:03.580046  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:03.580746  6509 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
I20260709 12:20:03.581394  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:03.582022  6583 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:20:03.582583  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 72897.1us
I20260709 12:20:03.582832  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:03.583721  6581 raft_consensus.cc:2410] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 16 because replica has last-logged OpId of term: 13 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 13.
W20260709 12:20:03.546979  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 55477.7us
I20260709 12:20:03.549095 13746 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:03.592355  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:03.592697 13741 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:03.567574 13737 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:03.593741 13737 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:03.581754 13740 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:03.593775  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:03.596920 13740 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:03.598412  6577 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
W20260709 12:20:03.599198 13743 log.cc:927] Time spent T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:20:03.599383  6577 raft_consensus.cc:606] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:03.599819 13747 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:03.600414 13751 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:03.600553 13747 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:03.600991 13747 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:03.601771 13618 consensus_queue.cc:237] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:03.605393 13739 log.cc:927] Time spent T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:20:03.582073 13508 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:20:03.610884 13745 log.cc:927] Time spent T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:20:03.611776  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:03.554504 13677 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:03.599885  6577 raft_consensus.cc:614] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:03.613375  6501 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 21: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:03.614342  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:03.614447 13682 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:03.580210 13701 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:03.615505  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:03.616505  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:03.617713  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:03.618332 13689 consensus_meta.cc:287] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.536s	user 0.009s	sys 0.000s
I20260709 12:20:03.618779 13689 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:03.619155  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:03.619767  6509 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:20:03.620662  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:03.621208  6509 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:03.619915  6583 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22.
I20260709 12:20:03.628027  6583 raft_consensus.cc:1275] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:20:03.649268  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:03.650187  6505 raft_consensus.cc:2368] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18.
I20260709 12:20:03.669513  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:03.670387  6583 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:20:03.671711  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:03.679637  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:03.639510 13138 consensus_queue.cc:1048] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:20:03.643386 13737 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:03.689939  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:03.690819  6583 raft_consensus.cc:2368] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 23. Current term is 24.
I20260709 12:20:03.600996 13751 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:03.708603  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:03.711589  6650 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 20. Current term is 21. Ops: []
W20260709 12:20:03.712908  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 47687.6us
I20260709 12:20:03.741608  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:03.748456 13734 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.108s
I20260709 12:20:03.744755 13748 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:03.743855  6501 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
I20260709 12:20:03.814133  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:03.777026 13518 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:03.816741  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:03.818136  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 126100us
I20260709 12:20:03.822018  6583 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:20:03.825595  6509 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:20:03.710819 13689 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:03.839690  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:03.840466  6509 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 25.
I20260709 12:20:03.846390  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:03.845669  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:03.847280  6583 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:20:03.841919 13748 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:03.888465  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:03.922708  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:03.925490  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:03.929767  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:03.949894  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:03.968106  6652 raft_consensus.cc:2410] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 21 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 12:20:03.950657  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:03.970101  6500 raft_consensus.cc:2410] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 21 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 12:20:03.972100  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:03.959502 13656 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:03.978932  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:03.981153 13701 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:03.983909 13760 log.cc:927] Time spent T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.214s	user 0.005s	sys 0.000s
W20260709 12:20:03.985737 13755 log.cc:927] Time spent T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.216s	user 0.000s	sys 0.003s
I20260709 12:20:03.985832  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:03.989493  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:03.990250  6578 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:20:03.992779  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 113462us
I20260709 12:20:03.888232 13748 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:03.884656 13673 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:03.882350 13740 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:03.885965 13518 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:03.875792 13765 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:04.029592  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:03.881124 13741 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.126s
I20260709 12:20:03.890712 13747 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:04.042600  6578 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 23.
I20260709 12:20:03.999574 13676 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:04.047518  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:04.049429  6578 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
W20260709 12:20:04.051146  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 103030us
I20260709 12:20:04.028499 13424 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:04.051265  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:04.102257  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:04.072790  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:04.050164  6578 raft_consensus.cc:606] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:04.110707  6654 raft_consensus.cc:1275] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:20:04.092842  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:04.044041 13518 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:04.028703 13765 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:04.126464 13770 log.cc:927] Time spent T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:20:04.112128  6500 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:20:04.127274 13765 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:04.128145  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:04.129031  6500 raft_consensus.cc:2368] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 21. Current term is 22.
I20260709 12:20:04.129851 13518 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:04.048915 13737 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:04.089022  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 153888us
I20260709 12:20:04.129694  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:04.143550  6578 raft_consensus.cc:614] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:04.053154 13676 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:04.133533 13764 log.cc:927] Time spent T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:20:04.131956  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:04.137765  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:04.132476  6500 raft_consensus.cc:1275] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
W20260709 12:20:04.113724 13508 consensus_meta.cc:287] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.505s	user 0.009s	sys 0.009s
I20260709 12:20:04.144083  6651 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
W20260709 12:20:04.112828 13721 log.cc:927] Time spent T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:20:04.152000  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:04.152375  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:04.152642  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:04.153481  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:04.154678  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:04.096324 13467 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:04.189222  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:04.190340  6500 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
I20260709 12:20:04.194856  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:04.196707  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:04.197386  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:04.198164  7265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38558 (request call id 1007) took 1018 ms. Trace:
I20260709 12:20:04.198479  7265 rpcz_store.cc:276] 0709 12:20:03.179424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:03.180136 (+   712us) service_pool.cc:224] Handling call
0709 12:20:04.197282 (+1017146us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b5800125198. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64eee 000000000036eb39 00007f287add81e0 00007f287aea10c4 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:20:04.198090 (+   808us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2316032}
I20260709 12:20:04.177543  6583 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:20:04.051604 13756 log.cc:927] Time spent T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
W20260709 12:20:04.209574 13753 log.cc:927] Time spent T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.400s	user 0.004s	sys 0.000s
W20260709 12:20:04.210716 13766 log.cc:927] Time spent T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
W20260709 12:20:04.212774 13757 log.cc:927] Time spent T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.255s	user 0.000s	sys 0.001s
I20260709 12:20:04.180166  6578 raft_consensus.cc:1275] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 12:20:04.151643  6650 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:20:04.183480  6654 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:20:04.217923  6502 raft_consensus.cc:1240] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:20:04.185395 13518 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:04.189182  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 3000) took 1015 ms. Trace:
I20260709 12:20:04.214387  6578 pending_rounds.cc:85] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 10
I20260709 12:20:04.201972  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:04.221519  6578 pending_rounds.cc:107] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 12:20:04.112610 13674 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:04.220862  6507 rpcz_store.cc:276] 0709 12:20:03.173406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:03.184294 (+ 10888us) service_pool.cc:224] Handling call
0709 12:20:03.202969 (+ 18675us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:03.253167 (+ 50198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:03.258742 (+  5575us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:03.890664 (+631922us) spinlock_profiling.cc:243] Waited 1.24 ms on lock 0x7b1800499120. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d6a2a6 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a
0709 12:20:03.900967 (+ 10303us) log.cc:844] Serialized 26 byte log entry
0709 12:20:04.110962 (+209995us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:04.111001 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:04.111070 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:04.188908 (+ 77838us) raft_consensus.cc:1685] finished
0709 12:20:04.188927 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:04.189124 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28496,"raft.queue_time_us":32319,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":3202,"spinlock_wait_cycles":3458816,"thread_start_us":112087,"threads_started":1,"wal-append.queue_time_us":262681}
I20260709 12:20:04.222023  6578 raft_consensus.cc:2889] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:04.223564  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:04.234151  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:04.234836  6506 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:04.214038 13751 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:04.223170 13737 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:04.248721 13737 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:04.250991 13737 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:04.261497  6583 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22.
I20260709 12:20:04.262550  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:04.263317  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:04.264837  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:04.228124 13771 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:20:04.293398  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 69911.8us
I20260709 12:20:04.294226  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:04.297459  6578 raft_consensus.cc:2410] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:20:04.298007  6580 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:20:04.271438 13758 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:04.234119 13656 consensus_queue.cc:1048] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 12:20:04.230881 13674 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:04.314515  6578 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:20:04.303222 13734 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:04.311823 13138 raft_consensus.cc:2764] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:20:04.319373  6583 raft_consensus.cc:1275] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:20:04.329993  6578 raft_consensus.cc:1275] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:20:04.329936  6580 raft_consensus.cc:1275] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:20:04.314087 13758 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:04.303526 13771 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:04.338294  6578 raft_consensus.cc:1275] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:20:04.347249  6622 leader_election.cc:400] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 20. Current term is 21.
I20260709 12:20:04.347637  6622 leader_election.cc:403] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:04.348472  6506 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 24.
I20260709 12:20:04.350940 13747 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:04.365558 13674 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:04.370442 13758 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:04.357954 13781 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 20. Current term is 21.
I20260709 12:20:04.372001 13771 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:04.372483  6509 raft_consensus.cc:1275] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:20:04.378969  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:04.359226 13676 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:04.382334  6622 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:04.385032  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:04.387312  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:04.387953 13781 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:20:04.393010 13781 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:04.389899 13771 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:04.394089 13771 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:04.394487 13771 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:04.389020 13742 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.007s
W20260709 12:20:04.397373  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 63028.7us
I20260709 12:20:04.400749  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:04.403465  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:04.410022  6576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 1110) took 1461 ms. Trace:
I20260709 12:20:04.410298  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:04.405705 13742 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:04.410501  6576 rpcz_store.cc:276] 0709 12:20:02.948844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:02.949605 (+   761us) service_pool.cc:224] Handling call
0709 12:20:02.949908 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:02.955191 (+  5283us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:02.955222 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:02.983958 (+ 28736us) log.cc:844] Serialized 26 byte log entry
0709 12:20:03.068790 (+ 84832us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:20:03.112750 (+ 43960us) log.cc:844] Serialized 14 byte log entry
0709 12:20:04.409464 (+1296714us) spinlock_profiling.cc:243] Waited 3.27 ms on lock 0x7b1800486100. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d16c0a 00007f287c456219 00007f2872d9ea40 00007f2872d994e4 00007f2872d99455 00007f2875e81060 00007f2875e93716 00007f2875ea4287 00007f2875ea4208
0709 12:20:04.409652 (+   188us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:04.409727 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:04.409770 (+    43us) raft_consensus.cc:1685] finished
0709 12:20:04.409846 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:04.409965 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":221768,"prepare.queue_time_us":44264,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":87,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":89,"spinlock_wait_cycles":7521664,"thread_start_us":44643,"threads_started":1,"wal-append.queue_time_us":132295,"wal-append.run_cpu_time_us":2322,"wal-append.run_wall_time_us":1016395}
I20260709 12:20:04.417706  6622 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:04.412881 13654 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:04.425712  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 1.13095e+06us
W20260709 12:20:04.426802  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:04.433053  6582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2972) took 1241 ms. Trace:
I20260709 12:20:04.433580  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 890) took 1035 ms. Trace:
I20260709 12:20:04.434396  6550 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:04.434670 13707 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:20:04.433470  6582 rpcz_store.cc:276] 0709 12:20:03.191248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:03.191987 (+   739us) service_pool.cc:224] Handling call
0709 12:20:03.224326 (+ 32339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:03.229647 (+  5321us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:20:03.233206 (+  3559us) log.cc:844] Serialized 14 byte log entry
0709 12:20:03.610161 (+376955us) spinlock_profiling.cc:243] Waited 6.56 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:20:04.432757 (+822596us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:04.432808 (+    51us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:04.432853 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:04.432903 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:04.432991 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1037647,"prepare.queue_time_us":4373,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"spinlock_wait_cycles":15082752,"thread_start_us":161339,"threads_started":1,"wal-append.queue_time_us":259419}
I20260709 12:20:04.435506 13712 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:04.435672  6550 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:04.433984  6503 rpcz_store.cc:276] 0709 12:20:03.392692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:03.392881 (+   189us) service_pool.cc:224] Handling call
0709 12:20:03.439742 (+ 46861us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:03.450326 (+ 10584us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:03.450359 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:03.450403 (+    44us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:20:03.450436 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:04.428234 (+977798us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b54002c3dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:04.428268 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:04.428431 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23871744}
I20260709 12:20:04.414032 13692 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:04.438910  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:04.438990  6550 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:04.436491 13758 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:04.444761  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:04.446928  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:04.447655  6622 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:04.450057  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:04.443599  6575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2971) took 1252 ms. Trace:
I20260709 12:20:04.449740 13654 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:04.454881 13661 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:04.455452 13661 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:04.456856 13654 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:04.457286 13656 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:04.457701 13661 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:04.457827 13734 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:04.458493 13734 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:04.458658  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:04.459013 13734 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:04.460018  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:04.460697  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:04.460580  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 274340us
I20260709 12:20:04.461927 13758 raft_consensus.cc:697] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:04.462954 13758 consensus_queue.cc:237] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:04.464763  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:04.465936  6580 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:20:04.466209  6583 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 13 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 12:20:04.466810  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:20:04.469036  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:20:04.472680 13759 log.cc:927] Time spent T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.625s	user 0.000s	sys 0.005s
I20260709 12:20:04.446668 13742 raft_consensus.cc:2764] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:20:04.457764 13656 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:04.464866  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:04.477957  6473 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:04.479180  6473 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:04.476235  6550 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:04.481339  6622 leader_election.cc:400] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 19. Current term is 20.
I20260709 12:20:04.481673  6622 leader_election.cc:403] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:20:04.482002  6583 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:20:04.483861  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:04.484577  6583 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:20:04.486559  6583 raft_consensus.cc:1275] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:20:04.474114  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:04.494925  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 46472.2us
I20260709 12:20:04.453909 13781 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:04.505714  6473 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:04.452968 13720 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:04.501531  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:04.510721  6473 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:04.495842  6622 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:04.497212  6550 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:04.453527  6575 rpcz_store.cc:276] 0709 12:20:03.190906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:03.191029 (+   123us) service_pool.cc:224] Handling call
0709 12:20:03.202005 (+ 10976us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:03.233883 (+ 31878us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:20:03.504542 (+270659us) log.cc:844] Serialized 14 byte log entry
0709 12:20:04.443318 (+938776us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:04.443366 (+    48us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:20:04.443415 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:04.443470 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:04.443555 (+    85us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":793482,"prepare.queue_time_us":23215,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":804884,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":82,"thread_start_us":356942,"threads_started":2,"wal-append.queue_time_us":156679}
I20260709 12:20:04.490834 13697 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:04.517781  6473 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:04.488978 13656 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:04.491133 13712 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:04.510561 13781 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:04.507629 13692 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 19. Current term is 20.
I20260709 12:20:04.526805 13716 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:04.507743 13661 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:04.503690 13653 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:04.529997 13605 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:04.507265 13748 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:04.520675 13697 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:04.514256 13676 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:04.539266  6473 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:04.565652  6473 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:04.528769 13707 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:04.537955  6550 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:04.510216 13699 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:04.559414  6622 leader_election.cc:400] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 23. Current term is 24.
I20260709 12:20:04.558559 13661 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:04.558223 13747 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:04.562103 13778 consensus_queue.cc:1048] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:04.520510 13742 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:04.538717  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3503) took 1295 ms. Trace:
I20260709 12:20:04.540409 13781 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:04.573300  6550 leader_election.cc:400] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 21. Current term is 22.
I20260709 12:20:04.638165  6550 leader_election.cc:403] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:20:04.574162  6622 leader_election.cc:403] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:04.538200 13677 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:04.537163 13712 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:04.662614 13712 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:04.666260 13767 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:04.667128 13767 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:04.670006  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 96836.4us
W20260709 12:20:04.660954  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 101675us
I20260709 12:20:04.635581  6656 rpcz_store.cc:276] 0709 12:20:03.225906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:03.438264 (+212358us) service_pool.cc:224] Handling call
0709 12:20:03.448567 (+ 10303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:03.555951 (+107384us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:20:03.570367 (+ 14416us) log.cc:844] Serialized 14 byte log entry
0709 12:20:04.214254 (+643887us) spinlock_profiling.cc:243] Waited 1.3 ms on lock 0x7b1800463c00. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2875a314bc 00007f2872d7f48b 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d99455 00007f2875e81060 00007f2875e93716 00007f2875ea4287
0709 12:20:04.214346 (+    92us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:04.214395 (+    49us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:20:04.214439 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:04.377724 (+163285us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b1800463c00. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c522 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:20:04.405964 (+ 28240us) spinlock_profiling.cc:243] Waited 45 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:20:04.521124 (+115160us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b540019dcd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:04.521172 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:04.521310 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":14687,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":40993,"raft.run_cpu_time_us":1771,"raft.run_wall_time_us":18690,"spinlock_wait_cycles":387581056,"thread_start_us":141808,"threads_started":2,"wal-append.queue_time_us":135623}
I20260709 12:20:04.673754  6548 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:04.677379  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:04.567298  6473 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:20:04.688711  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 121599us
I20260709 12:20:04.691246  6473 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:04.671128  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:04.578322  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:04.514987 13720 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:04.641142  7274 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:04.565413 13697 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:04.581914 13747 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:04.574707 13621 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:04.649675 13771 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:04.574054 13748 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:04.657840  6579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 2817) took 1694 ms. Trace:
I20260709 12:20:04.660873 13692 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 23. Current term is 24.
I20260709 12:20:04.638777 13708 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:04.699097  6473 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:04.701051  6622 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:04.738732  6473 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:04.739092  6622 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:04.702135  6548 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:04.706449 13467 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:04.700412 13751 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:04.708295 13763 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:04.712785 13355 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:04.698309  6550 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:04.749217  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 50979.2us
I20260709 12:20:04.720927  6652 raft_consensus.cc:1275] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:20:04.566354 13653 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:04.725312  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:04.578054 13661 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:04.577255 13658 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:04.577682 13676 raft_consensus.cc:697] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:04.774375 13710 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:04.669474 13767 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:04.733615  6579 rpcz_store.cc:276] 0709 12:20:02.963934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:02.966195 (+  2261us) service_pool.cc:224] Handling call
0709 12:20:02.973915 (+  7720us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:03.105204 (+131289us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:20:03.157435 (+ 52231us) log.cc:844] Serialized 14 byte log entry
0709 12:20:03.457506 (+300071us) spinlock_profiling.cc:243] Waited 3.58 ms on lock 0x7b18002ca5e0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:20:04.425320 (+967814us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:04.426401 (+  1081us) log.cc:844] Serialized 26 byte log entry
0709 12:20:04.492981 (+ 66580us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:20:04.493026 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:04.493103 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:04.657449 (+164346us) raft_consensus.cc:1685] finished
0709 12:20:04.657593 (+   144us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:04.657763 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2489434,"prepare.queue_time_us":50461,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":1222494,"raft.run_cpu_time_us":431,"raft.run_wall_time_us":47854,"spinlock_wait_cycles":8233856,"thread_start_us":162484,"threads_started":2,"wal-append.queue_time_us":421876,"wal-append.run_cpu_time_us":1506,"wal-append.run_wall_time_us":1006222}
I20260709 12:20:04.743286 13677 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:04.723009 13720 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:04.746474 13682 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:04.697460 13707 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 21. Current term is 22.
I20260709 12:20:04.700248 13654 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:04.574450 13518 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.160s
W20260709 12:20:04.747303  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 45303.7us
I20260709 12:20:04.741452  6473 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:04.726537 13747 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:04.753921  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:04.579831 13699 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:04.768752  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:04.753648 13737 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:20:04.749480 13740 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:04.772660  6651 raft_consensus.cc:1275] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:20:04.754508 13660 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:04.772878 13700 consensus_queue.cc:1059] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:04.773314  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:04.743093 13708 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:04.750602  6550 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:04.794879  6548 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:04.795246  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 53900.3us
I20260709 12:20:04.803194  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:04.780375 13677 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:04.811009  6473 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:04.808656  7238 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:04.819460  6548 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:04.820745  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 70260.8us
I20260709 12:20:04.822310  6548 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:04.774996 13710 raft_consensus.cc:697] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:20:04.871205  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 52057us
I20260709 12:20:04.878376  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:04.879292  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:04.876957  6550 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:04.881011  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 72449.5us
I20260709 12:20:04.877707  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:04.883813  6475 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:04.801898 13660 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.083s
I20260709 12:20:04.779443 13734 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:04.897846 13653 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:04.898499 13653 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:04.797436 13699 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:04.782173 13467 raft_consensus.cc:697] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 22 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:04.869589 13658 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:04.903126 13658 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:04.903582 13658 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:04.869571 13355 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:04.800940 13740 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:04.817056 13677 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:04.795832 13751 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:04.792492 13654 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:04.802107 13700 raft_consensus.cc:3055] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:04.777328 13656 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:04.885282 13554 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:20:04.902860 13621 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:04.940419 13751 raft_consensus.cc:697] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 23 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:04.940864 13621 raft_consensus.cc:697] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 22 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:04.775525 13748 consensus_queue.cc:1059] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.149s
I20260709 12:20:04.941416 13751 consensus_queue.cc:237] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 22.10, Last appended by leader: 10, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:20:04.791590 13712 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:04.930303 13700 raft_consensus.cc:740] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:04.945075 13712 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:04.880981 13508 consensus_queue.cc:1059] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 12:20:04.945325 13700 consensus_queue.cc:260] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:04.945999 13508 raft_consensus.cc:3055] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:04.946358 13508 raft_consensus.cc:740] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:04.946583 13700 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:04.902916 13467 consensus_queue.cc:237] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:04.947265 13508 consensus_queue.cc:260] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:04.947894 13771 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:04.948658 13716 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:20:04.943364  6658 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:04.775053 13676 consensus_queue.cc:237] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:04.792248  6398 catalog_manager.cc:5697] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 10 to 14, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 14 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:04.954236 13734 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:04.941426 13710 consensus_queue.cc:237] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:04.956768  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2986) took 1724 ms. Trace:
I20260709 12:20:04.928175 13355 raft_consensus.cc:697] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 24 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:04.957227  6584 rpcz_store.cc:276] 0709 12:20:03.232588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:03.625895 (+393307us) service_pool.cc:224] Handling call
0709 12:20:03.645039 (+ 19144us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:04.442905 (+797866us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:20:04.442942 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:04.492769 (+ 49827us) log.cc:844] Serialized 26 byte log entry
0709 12:20:04.886689 (+393920us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:20:04.886727 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:04.886786 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:04.956539 (+ 69753us) raft_consensus.cc:1685] finished
0709 12:20:04.956562 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:04.956704 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":797312,"raft.queue_time_us":808954,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":101,"thread_start_us":345885,"threads_started":1,"wal-append.queue_time_us":436850}
I20260709 12:20:04.924377 13740 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:04.958915 13355 consensus_queue.cc:237] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:04.925752 13781 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:04.931047 13707 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:04.963438  6656 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:04.899940 13661 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:04.990868 13656 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:05.001430  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 262384us
W20260709 12:20:05.004266  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 181982us
I20260709 12:20:05.005533  6548 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:04.998086 13508 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:05.013051  6548 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:05.034466 13707 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:20:05.037050  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 226102us
I20260709 12:20:05.048765  6473 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:05.055927  6473 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:05.057371 13697 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:05.059957 13697 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:05.061126 13621 consensus_queue.cc:237] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:05.064940 13708 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:05.001711 13781 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:05.066936  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 53975us
I20260709 12:20:05.071237  6548 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:05.068145 13707 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:05.034849 13656 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 12:20:05.080119 13734 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:05.087605  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 210715us
I20260709 12:20:05.035014 13682 consensus_queue.cc:1048] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 12:20:05.094110 13720 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:05.099898  6548 leader_election.cc:400] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18.
I20260709 12:20:05.109139  6548 leader_election.cc:403] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:20:05.100018  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 216171us
I20260709 12:20:05.091104 13767 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:05.091116 13697 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:05.091398 13781 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:05.103554 13708 raft_consensus.cc:2764] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:20:05.130250  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:05.131548  6656 raft_consensus.cc:2410] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 22 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 12:20:05.132725  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:05.133841  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:05.135053  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:05.135825  6658 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:20:05.137770  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:05.120958 13748 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:05.136997  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:05.142391  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:05.104034 13734 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:05.144531 13748 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:05.167081  6550 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:05.180083 13653 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:05.181373 13661 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:05.140388 13781 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:05.166071 13708 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18.
I20260709 12:20:05.182785 13654 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:05.117936 13658 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:05.118263 13747 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:05.164361  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 66148.9us
I20260709 12:20:05.191344  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2982) took 1978 ms. Trace:
I20260709 12:20:05.117774 13767 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:05.208092  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:05.210088  6508 raft_consensus.cc:1240] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:20:05.204100 13748 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:05.215305  6503 raft_consensus.cc:1240] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 23 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 22. Current term is 23. Ops: []
W20260709 12:20:05.279829 13807 log.cc:927] Time spent T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:20:05.198976 13763 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:05.198706 13668 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:05.213562  6548 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:05.214028  6581 rpcz_store.cc:276] 0709 12:20:03.213295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:03.612682 (+399387us) service_pool.cc:224] Handling call
0709 12:20:03.613033 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:04.436457 (+823424us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:04.436491 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:04.437092 (+   601us) log.cc:844] Serialized 26 byte log entry
0709 12:20:04.510727 (+ 73635us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:04.882185 (+371458us) log.cc:844] Serialized 14 byte log entry
0709 12:20:05.190615 (+308430us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:05.190887 (+   272us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:05.191007 (+   120us) raft_consensus.cc:1685] finished
0709 12:20:05.191032 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:05.191255 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":827417,"prepare.queue_time_us":257677,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":34,"raft.queue_time_us":851779,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":27552,"thread_start_us":721931,"threads_started":3,"wal-append.queue_time_us":221081}
I20260709 12:20:05.284286  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:05.152760 13798 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.063s
W20260709 12:20:05.298857  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 85631.4us
I20260709 12:20:05.212095 13653 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:05.300240  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:05.281424  6550 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:05.285544  6504 raft_consensus.cc:1240] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 16. Current term is 17. Ops: []
W20260709 12:20:05.286579  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 144332us
I20260709 12:20:05.300244  6548 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:05.305138  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:05.282202  6651 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:20:05.305235  6548 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:05.306806  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 249334us
I20260709 12:20:05.310904 13796 consensus_queue.cc:1059] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:05.312502  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:20:05.314971  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 96433us
I20260709 12:20:05.295212 13763 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:05.284935 13794 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:05.323945  6548 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:05.305235 13653 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:05.287523  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 175952us
I20260709 12:20:05.308202 13710 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:05.333535  6475 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:05.333837  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:05.336710  6548 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:05.312706 13661 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:05.338546  6548 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:05.344575  6505 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 22 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:20:05.344744  6548 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:05.322378 13712 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:05.332672 13794 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:05.349447  6548 leader_election.cc:400] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 21. Current term is 22.
I20260709 12:20:05.350509 13719 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:05.333624 13653 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:05.331820 13699 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:05.320641 13467 consensus_queue.cc:1048] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
W20260709 12:20:05.351014 13793 log.cc:927] Time spent T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:20:05.341569 13654 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:20:05.370633  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:05.372459  6548 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:05.373238  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:05.374034  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:05.375058  6548 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:05.380939  6548 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:05.387636  6548 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:05.341753 13810 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:05.391031  7271 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:05.351781 13801 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:05.354315 13720 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:05.334952 13813 consensus_queue.cc:1048] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:20:05.407327  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:05.398797  6475 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:05.357062 13814 consensus_queue.cc:1048] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:20:05.354568 13794 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:20:05.403559  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 69362.7us
I20260709 12:20:05.400991  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:05.369735 13798 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:05.419286  6473 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:05.390513 13710 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:05.389999 13719 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:05.417819  6475 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:05.418565  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:05.431015  6584 raft_consensus.cc:2376] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17. Re-sending same reply.
I20260709 12:20:05.419292  6650 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:20:05.433646 13812 log.cc:927] Time spent T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:20:05.359982 13751 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 12:20:05.421216  6580 raft_consensus.cc:1240] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 22. Current term is 23. Ops: [20.9-22.10]
I20260709 12:20:05.441592 13767 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:05.443679  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:05.445371  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:05.446080  6584 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:20:05.447010  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:05.448725  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:20:05.456434 13811 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 12:20:05.457012 13809 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:05.405218 13707 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:05.404184 13799 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:05.457464  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:05.426800 13676 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:20:05.449446 13808 log.cc:927] Time spent T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:20:05.421239  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:05.459091  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:05.451915 13781 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:05.454936 13677 consensus_meta.cc:287] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.521s	user 0.011s	sys 0.000s
I20260709 12:20:05.455363 13734 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:05.457511 13809 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:05.458581 13747 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:05.420013 13802 consensus_queue.cc:1059] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 12:20:05.440061 13658 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:05.404994 13692 consensus_queue.cc:1048] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 12:20:05.403388 13801 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:05.460704  6650 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:20:05.459010 13676 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:20:05.470428  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 52729.9us
W20260709 12:20:05.476122  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 127290us
I20260709 12:20:05.441504  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:05.475039  6550 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:05.421042  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:05.477270  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:05.477401  6582 raft_consensus.cc:2376] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21. Re-sending same reply.
I20260709 12:20:05.478682  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:05.462779 13707 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:05.419709 13741 consensus_queue.cc:1048] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:20:05.490307  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:05.473420 13467 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:05.469960 13809 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:05.478945 13676 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:05.462018 13798 consensus_queue.cc:1048] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 12:20:05.496470  6580 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 21. Current term is 22. Ops: [19.12-21.13]
I20260709 12:20:05.475867 13801 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:05.484210 13799 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:05.502489 13799 raft_consensus.cc:697] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:05.503562 13799 consensus_queue.cc:237] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:05.489715 13747 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:05.486447 13707 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:05.472693 13677 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:05.521739 13716 raft_consensus.cc:3055] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:05.522305 13716 raft_consensus.cc:740] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:05.523259 13716 consensus_queue.cc:260] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:05.515447 13621 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:05.526448 13621 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:05.532294 13748 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:05.534663 13748 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:05.484721 13781 consensus_queue.cc:1059] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:20:05.536971 13781 raft_consensus.cc:3055] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:05.537400 13781 raft_consensus.cc:740] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:05.538301 13781 consensus_queue.cc:260] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:05.539122 13621 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:05.539584 13781 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:05.487102 13820 consensus_queue.cc:1059] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 22.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 12:20:05.540788 13820 consensus_queue.cc:1243] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 23.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:20:05.541297 13820 raft_consensus.cc:3055] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:05.541628 13820 raft_consensus.cc:740] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:05.542560 13820 consensus_queue.cc:260] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.10, Last appended by leader: 10, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:20:05.543848 13820 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:05.538712 13716 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:05.550379 13817 log.cc:927] Time spent T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 12:20:05.551647 13697 consensus_queue.cc:1048] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:05.538635  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:05.497710 13767 consensus_queue.cc:1059] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.063s
I20260709 12:20:05.571198  6578 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:20:05.557072  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 2985) took 2333 ms. Trace:
I20260709 12:20:05.554930  6583 raft_consensus.cc:1240] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:05.557617  6500 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:20:05.562357  6550 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:05.592082  6581 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 20. Current term is 21. Ops: []
W20260709 12:20:05.552315 13700 consensus_meta.cc:287] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.605s	user 0.010s	sys 0.000s
I20260709 12:20:05.602205 13658 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:05.604893  6475 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:05.576579  6577 rpcz_store.cc:276] 0709 12:20:03.225486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:03.614265 (+388779us) service_pool.cc:224] Handling call
0709 12:20:03.614596 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:04.467366 (+852770us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:20:04.699324 (+231958us) log.cc:844] Serialized 14 byte log entry
0709 12:20:05.004104 (+304780us) log.cc:844] Serialized 14 byte log entry
0709 12:20:05.005566 (+  1462us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:05.012154 (+  6588us) log.cc:844] Serialized 26 byte log entry
0709 12:20:05.012310 (+   156us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:05.013335 (+  1025us) log.cc:844] Serialized 14 byte log entry
0709 12:20:05.013522 (+   187us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:05.013619 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:05.061242 (+ 47623us) spinlock_profiling.cc:243] Waited 22.3 ms on lock 0x7b18003388c0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c522 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:20:05.215283 (+154041us) spinlock_profiling.cc:243] Waited 4.89 ms on lock 0x7b18003388c0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c557 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:20:05.299731 (+ 84448us) spinlock_profiling.cc:243] Waited 5.67 ms on lock 0x7b18003388c0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:20:05.556846 (+257115us) raft_consensus.cc:1685] finished
0709 12:20:05.556868 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:05.557013 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":858158,"prepare.queue_time_us":329596,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":3905,"raft.queue_time_us":889038,"raft.run_cpu_time_us":364,"raft.run_wall_time_us":32864,"spinlock_wait_cycles":98777088,"thread_start_us":484637,"threads_started":2,"wal-append.queue_time_us":431111}
I20260709 12:20:05.610234  6475 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:05.611563  6582 raft_consensus.cc:2410] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:20:05.602810 13658 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:05.623628  6475 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:05.618233 13740 consensus_meta.cc:287] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.659s	user 0.014s	sys 0.000s
I20260709 12:20:05.620662 13814 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:05.615799 13653 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:05.611186 13697 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:05.631881  6475 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:05.637285  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:05.632582 13740 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:05.638712  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:05.639446 13740 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:05.639833  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:05.640799  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:05.597609 13822 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:20:05.643671  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:05.643744  6475 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:05.645169  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:05.600072 13794 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:05.652451  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:05.611466 13621 consensus_queue.cc:1048] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:20:05.654633  6583 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 20. Current term is 21. Ops: []
W20260709 12:20:05.694947  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 51315us
I20260709 12:20:05.626546 13658 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:05.696528 13824 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:05.633071 13804 consensus_queue.cc:1048] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.073s
I20260709 12:20:05.700311 13653 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:05.702857 13814 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:05.703783  6575 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:20:05.651892 13740 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:20:05.705394 13677 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:05.708082  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 63015.7us
I20260709 12:20:05.708595  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:05.725551  6651 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:05.634567 13748 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:20:05.726478  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 267469us
I20260709 12:20:05.731426  6575 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:20:05.732417 13824 raft_consensus.cc:697] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:05.737959 13824 consensus_queue.cc:237] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:05.739660  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:05.719653 13763 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:05.743556  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:05.731464  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:05.745748  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:05.746421  6509 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:20:05.748404  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:05.749001  6509 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 14.
I20260709 12:20:05.749958  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:05.751766  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:05.755151  6622 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:05.756573  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:05.757208  6504 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 22.
I20260709 12:20:05.757328  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:05.758411  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:05.758512 13823 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:05.759140 13823 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:05.759658 13823 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:05.718814 13467 consensus_queue.cc:1059] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 12:20:05.652778 13707 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:05.769299  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:05.770256 13828 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:05.782747  6658 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 25. Current term is 26. Ops: []
I20260709 12:20:05.783802 13677 raft_consensus.cc:3055] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:05.784422 13677 raft_consensus.cc:740] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:05.785006 13682 raft_consensus.cc:3055] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:05.785388 13682 raft_consensus.cc:740] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:05.785452 13677 consensus_queue.cc:260] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:05.788568 13677 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:05.783840 13794 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:05.755121 13769 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:05.786285 13682 consensus_queue.cc:260] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:05.796855 13682 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:05.774219 13692 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:05.803670 13707 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:05.804589  6580 raft_consensus.cc:1240] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:20:05.806671  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:05.804618 13804 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:05.807893  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:05.807844 13804 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:05.812934  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 43746.5us
I20260709 12:20:05.813328 13796 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:05.843748  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:05.844872  6583 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:05.817431  6658 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:20:05.847143  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:05.818071  6651 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:05.831140  6653 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:20:05.850760  6509 raft_consensus.cc:2393] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 17: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:05.857206  6651 raft_consensus.cc:1240] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:20:05.860095  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:05.860097  6653 raft_consensus.cc:1240] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 21. Current term is 22. Ops: [21.11-21.11]
I20260709 12:20:05.866961  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:05.868359  6656 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:05.870337  6505 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 13 index: 14. (index mismatch)
I20260709 12:20:05.815716 13801 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:05.878371  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 68730.8us
I20260709 12:20:05.882302  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:05.878734  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:05.747920 13814 consensus_queue.cc:1059] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.096s
I20260709 12:20:05.887454  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:05.891880  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 54493.1us
I20260709 12:20:05.893456  6622 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:05.907080  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:05.904733  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:05.919653  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:05.925633  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:05.924757  6658 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:05.929144  6658 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:20:05.921664  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:05.931416  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 118860us
I20260709 12:20:05.931586  6505 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:20:05.921738  6550 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:05.909124 13769 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:05.965801  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:05.966606  6583 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:20:05.970980  6656 raft_consensus.cc:1240] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:05.967780  7238 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:05.966693  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 45042.8us
I20260709 12:20:05.977399  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:05.979600  6580 raft_consensus.cc:2393] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 21: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:05.903012 13740 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 12:20:05.980260  6506 raft_consensus.cc:2393] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 21: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:05.953706 13138 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:05.940727 13809 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:05.949309 13799 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:05.920032 13660 consensus_queue.cc:1059] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 25.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 12:20:05.920116 13794 consensus_queue.cc:1059] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 12:20:05.939632 13720 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.042s
I20260709 12:20:05.937757 13798 consensus_queue.cc:1059] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:20:06.000674  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:05.939908 13676 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:05.977766 13719 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:05.953969 13796 consensus_queue.cc:1059] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 12:20:05.984309 13779 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:20:05.963057 13767 consensus_queue.cc:1059] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 12:20:06.022840 13741 raft_consensus.cc:3055] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:06.023576 13741 raft_consensus.cc:740] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:06.024523 13741 consensus_queue.cc:260] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:06.029333  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:06.030671  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:06.031253  6501 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:06.033445 13804 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:05.995573 13692 consensus_queue.cc:1059] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:20:06.034862 13799 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:06.037400 13799 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:06.039548  6398 catalog_manager.cc:5697] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 9 to 14, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 14 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:05.212709  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:06.042388  6398 catalog_manager.cc:5697] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 7 to 15, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 15 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:06.040702 13823 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:06.044889  6657 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:06.045013 13621 raft_consensus.cc:3055] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:06.045573 13621 raft_consensus.cc:740] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:06.046020  6398 catalog_manager.cc:5697] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 11 to 13, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 13 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:06.048472  6398 catalog_manager.cc:5697] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 11 to 12, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 12 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:06.003542  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:06.053759  6504 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:06.054430  6506 raft_consensus.cc:2393] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 18: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:20:06.005650 13800 log.cc:927] Time spent T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.853s	user 0.017s	sys 0.000s
I20260709 12:20:06.016661 13778 raft_consensus.cc:3055] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:20:06.017058  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:05.978123 13520 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:20:06.028043  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:06.066313  6505 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:06.018203 13779 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:06.064038  6503 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 20.
I20260709 12:20:05.999176 13138 raft_consensus.cc:697] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:06.078845  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:06.081866  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:06.082681 13692 raft_consensus.cc:3055] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:06.083469  6622 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:06.091507 13767 raft_consensus.cc:3055] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:06.091922 13656 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:06.073575  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:06.066022 13823 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:06.128922  6506 raft_consensus.cc:1275] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 12:20:06.130280  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 46906.6us
I20260709 12:20:06.131052  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:06.062436 13778 raft_consensus.cc:740] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:20:06.135429 13795 log.cc:927] Time spent T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:20:06.111006 13692 raft_consensus.cc:740] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:06.135437 13823 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:20:06.138965  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:06.139211  6583 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 12:20:06.175681  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:06.175834  6583 pending_rounds.cc:85] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 10
I20260709 12:20:06.109936 13767 raft_consensus.cc:740] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:06.140321  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:06.179409  6575 raft_consensus.cc:2393] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 21: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:06.181670  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:06.182762  6575 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:20:06.147399  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 204651us
I20260709 12:20:06.147992 13747 consensus_queue.cc:1059] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:06.149860 13467 raft_consensus.cc:3055] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:06.200958  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:20:06.151477  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 97582.1us
I20260709 12:20:06.153846 13751 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:06.132424 13771 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:06.157529  6578 raft_consensus.cc:1240] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:20:06.169796  6501 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:20:06.074120 13779 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:20:06.113415 13820 consensus_meta.cc:287] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.569s	user 0.008s	sys 0.004s
I20260709 12:20:06.176457  6583 pending_rounds.cc:107] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 12:20:06.223603  6583 raft_consensus.cc:2889] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:06.139174 13138 consensus_queue.cc:237] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:06.191792  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:06.195776  6548 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:06.110653 13719 raft_consensus.cc:697] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:06.199447  6502 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:20:06.160133 13741 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:06.160048 13676 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:06.235191  6654 raft_consensus.cc:1275] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 22 index: 10. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
W20260709 12:20:06.220929  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 235762us
I20260709 12:20:06.225315  6548 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:06.237838  6548 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:06.239382  6548 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:06.240437  6473 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:06.162787 13809 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:06.226397  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:06.241634  6505 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:20:06.244300  6473 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:06.227134  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:06.247450  6548 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:06.249748  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:06.245723  6500 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:06.252369 13660 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:06.200672 13467 raft_consensus.cc:740] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:06.264362 13708 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:20:06.264868  6575 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22.
I20260709 12:20:06.264727 13778 consensus_queue.cc:260] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:06.238454 13682 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:06.269739 13763 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:06.270354 13692 consensus_queue.cc:260] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:06.345304 13692 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:06.271838 13767 consensus_queue.cc:260] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:06.238628 13699 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:20:06.275681 13658 consensus_meta.cc:287] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.553s	user 0.005s	sys 0.004s
I20260709 12:20:06.278509 13676 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:20:06.215270 13677 consensus_queue.cc:1059] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 12:20:06.279673 13719 consensus_queue.cc:237] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:06.277539  6473 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:06.286657  7238 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:06.287793  6578 raft_consensus.cc:1275] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 19 index: 8. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
W20260709 12:20:06.291101  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 56961.5us
W20260709 12:20:06.268245  6609 heartbeater.cc:646] Failed to heartbeat to 127.5.58.62:36381 (5 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.5.58.62:36381 timed out after 15.000s (SENT)
I20260709 12:20:06.253750 13747 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:20:06.311484  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 64489.4us
I20260709 12:20:06.356532  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:06.361567  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 84137.4us
I20260709 12:20:06.362625  6473 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:06.340903 13467 consensus_queue.cc:260] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:06.291450 13605 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.061s
I20260709 12:20:06.338897 13720 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:20:06.369963  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 83433.3us
I20260709 12:20:06.338181 13809 consensus_queue.cc:1059] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 12:20:06.351426 13699 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:06.404882  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:06.349514 13653 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:06.384266 13763 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:06.383332 13767 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:06.357431 13658 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:06.363725 13708 consensus_queue.cc:1048] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 12:20:06.382440 13520 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:06.428438 13708 consensus_queue.cc:1243] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:20:06.429988  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:06.430856  6653 raft_consensus.cc:2393] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 22: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:20:06.437126 13843 log.cc:927] Time spent T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:20:06.421471 13699 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:20:06.409914  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 59140.2us
I20260709 12:20:06.414858 13467 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:06.412704 13697 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:06.449870  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:06.413520 13804 raft_consensus.cc:3055] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:06.422853 13827 raft_consensus.cc:3055] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:06.408469 13660 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:06.446785 13621 consensus_queue.cc:260] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:06.447620 13809 consensus_queue.cc:1059] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:06.448133 13797 consensus_queue.cc:1059] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:06.458963 13660 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:06.467701 13660 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:06.458595 13827 raft_consensus.cc:740] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:06.469579  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:06.469699 13827 consensus_queue.cc:260] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:06.471477 13827 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:06.453703 13677 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:06.474196  6577 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:06.423136 13751 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 12:20:06.478669 13797 consensus_queue.cc:1243] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer f2b06aebe97249a4877d798b8118ece4 log is divergent from this leader: its last log entry 20.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:20:06.479451 13740 raft_consensus.cc:3055] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:06.479924 13740 raft_consensus.cc:740] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:06.480796 13740 consensus_queue.cc:260] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:06.451344  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 88798.6us
I20260709 12:20:06.481817 13658 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:06.482664  6473 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:06.482897  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:06.483706 13697 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:06.484369 13697 raft_consensus.cc:697] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:06.485352 13697 consensus_queue.cc:237] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:06.451694  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:06.453184  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:06.488554  6505 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 22.
I20260709 12:20:06.444516 13763 raft_consensus.cc:697] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:06.489830 13763 consensus_queue.cc:237] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:06.458876  6502 raft_consensus.cc:1240] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:06.464570  6581 raft_consensus.cc:2393] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 17: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:06.453966 13804 raft_consensus.cc:740] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:06.448383  6654 raft_consensus.cc:1275] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:20:06.495510 13658 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:06.497555  6473 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:06.512167  7238 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:06.502771 13740 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:06.496403 13796 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:06.530961  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:06.502506 13737 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:06.536547  7238 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:06.514219 13809 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:06.545069  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:06.545809  6654 raft_consensus.cc:1275] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:20:06.524808 13783 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:20:06.553253  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:06.502228 13658 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:06.544916 13746 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:06.569800  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:06.539347 13796 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:06.543375 13737 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:06.525022 13804 consensus_queue.cc:260] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:06.569816 13605 consensus_queue.cc:1059] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 12:20:06.574158 13674 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:06.571244 13746 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:06.594998  6576 raft_consensus.cc:1240] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 21. Current term is 22. Ops: [21.11-21.11]
W20260709 12:20:06.591779 13824 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.017s	sys 0.023s
I20260709 12:20:06.598034 13605 raft_consensus.cc:3055] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:06.593010 13520 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:20:06.604655 13824 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.017s	sys 0.023s
I20260709 12:20:06.598165 13746 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:06.606609 13804 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:06.631407 13720 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:06.639451 13658 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:06.642170  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:06.643143  6654 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:06.652817  6506 raft_consensus.cc:1275] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 12:20:06.653316  6506 pending_rounds.cc:85] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 10
I20260709 12:20:06.653575  6506 pending_rounds.cc:107] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 12:20:06.653841  6506 raft_consensus.cc:2889] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:06.609479 13605 raft_consensus.cc:740] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:06.693907  6505 pending_rounds.cc:85] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 10
I20260709 12:20:06.694278  6505 pending_rounds.cc:107] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 12:20:06.694550  6505 raft_consensus.cc:2889] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:06.698199 13737 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:06.700368  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:06.701879  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:06.704470  6548 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:06.675846 13822 raft_consensus.cc:3055] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:06.710160  6548 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:06.695858 13699 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:06.716310  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:06.713243  6548 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:06.722150 13668 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:06.722119  6582 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 18. Current term is 19. Ops: []
W20260709 12:20:06.725121  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:06.683904 13605 consensus_queue.cc:260] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:06.737076  7273 raft_consensus.cc:1240] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:06.738603  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:06.749151  6582 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22.
I20260709 12:20:06.712442 13653 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:06.711884 13822 raft_consensus.cc:740] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:06.741142  6576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 3015) took 1006 ms. Trace:
I20260709 12:20:06.753831  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:06.757019  6576 rpcz_store.cc:276] 0709 12:20:05.734223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:05.970504 (+236281us) service_pool.cc:224] Handling call
0709 12:20:05.987028 (+ 16524us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:06.598189 (+611161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:06.740710 (+142521us) spinlock_profiling.cc:243] Waited 608 ms on lock 0x7b5400031e54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:06.740983 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1397565696}
I20260709 12:20:06.735311 13719 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:06.714973 13720 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.041s
W20260709 12:20:06.785051  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:06.761929 13741 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:06.845909  6577 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:06.764753 13767 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:06.821704 13720 consensus_queue.cc:1243] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 21.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:20:06.820518 13605 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:06.769753 13719 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:06.853799  6475 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:06.841547 13697 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:06.861980  6655 raft_consensus.cc:1275] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:20:06.862403  6475 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:06.878835  6577 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:20:06.860379 13719 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:06.849521 13741 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:20:06.859956 13740 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:06.855266 13767 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:06.867641 13467 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:06.867316 13827 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:20:06.878211 13779 consensus_meta.cc:287] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.637s	user 0.014s	sys 0.000s
I20260709 12:20:06.873320 13869 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:06.903957 13857 log.cc:927] Time spent T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 12:20:06.904825  6577 raft_consensus.cc:1275] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:20:06.907133  6577 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 21. Current term is 22. Ops: [21.11-21.11]
I20260709 12:20:06.909162 13778 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:06.913195 13699 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:06.870229 13763 consensus_queue.cc:1059] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:20:06.914548 13763 raft_consensus.cc:3055] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:06.914973 13763 raft_consensus.cc:740] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:20:06.904891 13866 log.cc:927] Time spent T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 12:20:06.897581 13767 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:06.916826  6398 catalog_manager.cc:5697] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 14 to 16, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 16 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:06.930365  6654 raft_consensus.cc:1275] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 12:20:06.930814  6654 pending_rounds.cc:85] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 12
I20260709 12:20:06.931107  6654 pending_rounds.cc:107] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 12:20:06.931384  6654 raft_consensus.cc:2889] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:06.864843 13697 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:06.872812 13751 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 12:20:06.898236 13863 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:20:06.953306  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:06.933722 13801 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:06.925550 13779 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:06.959702 13720 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:06.955366 13806 log.cc:927] Time spent T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 1.608s	user 0.017s	sys 0.004s
I20260709 12:20:06.915964 13763 consensus_queue.cc:260] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:06.934055 13869 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:06.933521 13740 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:06.946738 13870 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:06.984071  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41304 (request call id 898) took 1224 ms. Trace:
W20260709 12:20:07.001009  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.000571  6503 rpcz_store.cc:276] 0709 12:20:05.746743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:05.746870 (+   127us) service_pool.cc:224] Handling call
0709 12:20:06.970854 (+1223984us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b54001e78d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:06.970994 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":582639360}
I20260709 12:20:06.857113 13822 consensus_queue.cc:260] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:07.002674 13822 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:06.988022 13758 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:07.004309 13763 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:06.955204 13863 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:06.949581 13467 consensus_queue.cc:1048] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:20:06.957466 13824 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:20:06.998626  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.018141  6655 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 25. Current term is 26. Ops: []
I20260709 12:20:07.023381  6656 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:20:07.040733  6577 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 21. Current term is 22. Ops: [19.12-21.13]
W20260709 12:20:07.048914  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 161452us
W20260709 12:20:07.053938  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.054343  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:07.054931  6653 raft_consensus.cc:1240] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:07.055078  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:07.056125  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:07.063508 13861 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:07.071372 13861 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:07.073212 13863 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:07.075361  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:07.078195  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:07.042447 13824 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:07.004503 13758 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:07.090294  6579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 1134) took 2280 ms. Trace:
I20260709 12:20:07.033120 13870 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:07.075083 13719 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:07.090730  6579 rpcz_store.cc:276] 0709 12:20:04.807893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:04.809169 (+  1276us) service_pool.cc:224] Handling call
0709 12:20:04.817213 (+  8044us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:04.961564 (+144351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:04.961595 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:04.962060 (+   465us) log.cc:844] Serialized 26 byte log entry
0709 12:20:05.155353 (+193293us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:05.155400 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:05.653148 (+497748us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b540034e4d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:05.653204 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:06.849674 (+1196470us) spinlock_profiling.cc:243] Waited 2.2 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:20:07.087804 (+238130us) raft_consensus.cc:1685] finished
0709 12:20:07.087829 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:07.087959 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17139,"raft.queue_time_us":18522,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"spinlock_wait_cycles":296671232,"thread_start_us":192947,"threads_started":1,"wal-append.queue_time_us":341707}
I20260709 12:20:07.064145 13720 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:07.099956 13741 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:07.103089 13781 raft_consensus.cc:3055] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:07.105952 13870 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:07.109357  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.111968  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:07.087270 13880 consensus_queue.cc:1059] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:20:07.118213 13779 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:07.123982  6622 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:07.125532  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:07.127125  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:07.133113  6502 raft_consensus.cc:1240] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:07.141266  6508 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:07.133028 13746 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:07.134231 13850 log.cc:927] Time spent T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.553s	user 0.008s	sys 0.003s
I20260709 12:20:07.134822  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 2868) took 1497 ms. Trace:
I20260709 12:20:07.143502  6579 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:20:07.143656  6581 rpcz_store.cc:276] 0709 12:20:05.637412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:05.639591 (+  2179us) service_pool.cc:224] Handling call
0709 12:20:07.134631 (+1495040us) spinlock_profiling.cc:243] Waited 817 ms on lock 0x7b5400329954. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:07.134757 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1879275392}
I20260709 12:20:07.135293  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:07.135637 13720 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:07.136081  6580 raft_consensus.cc:1240] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:20:07.149298  6506 raft_consensus.cc:1240] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 21. Current term is 22. Ops: [21.11-21.11]
I20260709 12:20:07.137208  6582 raft_consensus.cc:1275] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:20:07.137691 13869 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:07.142169  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:07.150890  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:07.136229  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:07.086946 13668 raft_consensus.cc:697] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:07.151551  6651 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
W20260709 12:20:07.153621  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.153859  6506 raft_consensus.cc:1240] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
W20260709 12:20:07.154256  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.155454  6502 raft_consensus.cc:1240] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:07.159933 13719 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:07.129045 13781 raft_consensus.cc:740] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:07.164670  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:07.165489  6649 raft_consensus.cc:2393] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 22: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:07.165463 13781 consensus_queue.cc:260] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:07.168478  7267 raft_consensus.cc:1240] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:07.164737 13861 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:07.171552  6508 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
I20260709 12:20:07.172768  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 3013) took 1442 ms. Trace:
I20260709 12:20:07.173190  6583 rpcz_store.cc:276] 0709 12:20:05.730392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:05.968301 (+237909us) service_pool.cc:224] Handling call
0709 12:20:06.138837 (+170536us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:06.235403 (+ 96566us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:07.172558 (+937155us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b54002670d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:07.172703 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":391182720}
I20260709 12:20:07.185153  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:07.187043  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:07.188237  6502 raft_consensus.cc:2410] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 22 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:20:07.189522  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:07.190167 13656 consensus_queue.cc:1059] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:07.190686 13880 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:20:07.191337 13656 raft_consensus.cc:3055] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:07.191721 13656 raft_consensus.cc:740] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:07.192713 13740 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:07.193032  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:07.193395 13758 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:07.193804  7269 raft_consensus.cc:2393] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 20: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:20:07.194789 13656 consensus_queue.cc:260] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:07.196175 13656 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:07.200237 13797 raft_consensus.cc:3055] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:07.200765 13797 raft_consensus.cc:740] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:07.201689 13797 consensus_queue.cc:260] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:07.202483 13746 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:07.202783 13797 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:07.203402  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:07.238955  6622 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:07.239217  6548 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:07.153983 13819 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:20:07.227041  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:07.190696 13868 log.cc:927] Time spent T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.240s	user 0.002s	sys 0.000s
W20260709 12:20:07.213032  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.250623  6656 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 12:20:07.216919  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:07.246044  6502 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:07.247107  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:07.210671  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:07.255592  6583 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 23.
I20260709 12:20:07.254748  6656 pending_rounds.cc:85] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 10
I20260709 12:20:07.256740  6656 pending_rounds.cc:107] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 12:20:07.257179  6656 raft_consensus.cc:2889] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:07.261739  6502 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:07.249876 13712 raft_consensus.cc:3055] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:07.265884 13712 raft_consensus.cc:740] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:07.266980 13712 consensus_queue.cc:260] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:07.243624 13708 consensus_queue.cc:1048] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 12:20:07.238387 13861 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:07.245879 13758 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:07.244244 13872 consensus_queue.cc:1059] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.034s
I20260709 12:20:07.271241  6622 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:07.269456  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:07.282676  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:07.282915  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:07.243533 13719 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:20:07.285050  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 101549us
I20260709 12:20:07.269124  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:07.270893  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:07.278380  6504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 903) took 1530 ms. Trace:
I20260709 12:20:07.286366  6508 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:20:07.286712  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:07.286823  6474 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:20:07.279253  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 97334.6us
I20260709 12:20:07.289935  6504 rpcz_store.cc:276] 0709 12:20:05.748460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:05.757821 (+  9361us) service_pool.cc:224] Handling call
0709 12:20:05.758093 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:06.060271 (+302178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:07.278047 (+1217776us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b54002c4cd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:07.278280 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":679533312}
I20260709 12:20:07.268301 13712 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:07.273546 13861 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:07.310396  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 3022) took 1574 ms. Trace:
I20260709 12:20:07.310946  6580 rpcz_store.cc:276] 0709 12:20:05.736085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:06.431978 (+695893us) service_pool.cc:224] Handling call
0709 12:20:07.135678 (+703700us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:07.152373 (+ 16695us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:07.310115 (+157742us) spinlock_profiling.cc:243] Waited 703 ms on lock 0x7b5400329950. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:07.310317 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1617530752}
I20260709 12:20:07.311925 13861 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:07.293257 13819 consensus_queue.cc:1059] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 12:20:07.314257 13819 raft_consensus.cc:3055] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:07.314685 13819 raft_consensus.cc:740] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:07.295132 13747 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:07.315657 13819 consensus_queue.cc:260] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:07.318192 13819 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:07.308547  6575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 3021) took 1572 ms. Trace:
I20260709 12:20:07.291210 13771 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:20:07.307814 13740 consensus_queue.cc:1059] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 12:20:07.322883 13767 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:07.327579 13822 raft_consensus.cc:3055] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:07.273829 13827 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:07.328681  6657 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:07.329231 13860 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:07.323072  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:07.330116 13860 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:07.288575 13763 consensus_queue.cc:1059] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 12:20:07.323693  6575 rpcz_store.cc:276] 0709 12:20:05.735833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:06.342246 (+606413us) service_pool.cc:224] Handling call
0709 12:20:06.342610 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:06.342981 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:06.343013 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:06.343526 (+   513us) log.cc:844] Serialized 26 byte log entry
0709 12:20:06.503909 (+160383us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:20:06.503959 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:06.504044 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:07.133670 (+629626us) spinlock_profiling.cc:243] Waited 2 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:20:07.295883 (+162213us) raft_consensus.cc:1685] finished
0709 12:20:07.295917 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:07.308456 (+ 12539us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1712,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"spinlock_wait_cycles":4605696,"thread_start_us":159922,"threads_started":1,"wal-append.queue_time_us":213561}
I20260709 12:20:07.334002 13763 raft_consensus.cc:3055] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:07.334527 13763 raft_consensus.cc:740] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:07.335565 13763 consensus_queue.cc:260] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:07.287415 13518 raft_consensus.cc:3055] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:07.324841  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 3026) took 1696 ms. Trace:
I20260709 12:20:07.340925  6398 catalog_manager.cc:5697] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 5 to 13. New cstate: current_term: 13 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:07.343202 13767 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:07.320242 13824 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:07.273698 13668 consensus_queue.cc:237] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:07.328222 13822 raft_consensus.cc:740] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:07.348214  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:07.298971  6474 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:07.375458  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 76579.4us
I20260709 12:20:07.361569  6507 rpcz_store.cc:276] 0709 12:20:05.627944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:05.628072 (+   128us) service_pool.cc:224] Handling call
0709 12:20:05.648954 (+ 20882us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:05.649198 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:20:05.845869 (+196671us) log.cc:844] Serialized 14 byte log entry
0709 12:20:07.324225 (+1478356us) spinlock_profiling.cc:243] Waited 9.15 ms on lock 0x7b18003f1360. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2875a314bc 00007f2872d7f48b 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a2923a
0709 12:20:07.324463 (+   238us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:07.324533 (+    70us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:07.324584 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:07.324662 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:07.324781 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":794,"prepare.queue_time_us":314468,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":2786,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"spinlock_wait_cycles":21045120,"thread_start_us":270926,"threads_started":2,"wal-append.queue_time_us":92958,"wal-append.run_cpu_time_us":1347,"wal-append.run_wall_time_us":1053441}
W20260709 12:20:07.369076 13653 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.009s	sys 0.004s
W20260709 12:20:07.380190 13653 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.009s	sys 0.005s
I20260709 12:20:07.381101  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:07.381415 13653 raft_consensus.cc:697] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:07.381771  7269 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:20:07.382305 13653 consensus_queue.cc:237] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:07.386139  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:07.387048  6407 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57900 (request call id 192) took 16323 ms (client timeout 15000 ms). Trace:
I20260709 12:20:07.348743 13879 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:20:07.372217  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 77663.3us
I20260709 12:20:07.366060 13769 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:07.399399 13769 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:07.401961 13769 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:07.402940 13860 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:07.413988  6398 catalog_manager.cc:5697] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
W20260709 12:20:07.421293  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.425916 13769 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:07.426565 13769 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:07.428683 13769 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:07.430848 13822 consensus_queue.cc:260] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:07.393005 13880 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:20:07.387490  6407 rpcz_store.cc:269] 0709 12:19:51.048277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:51.178840 (+130563us) service_pool.cc:224] Handling call
0709 12:20:07.366880 (+16188040us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:19:58.776157 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:19:58.921984 (+145827us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:19:58.922022 (+    38us) write_op.cc:435] Acquired schema lock
0709 12:19:58.922041 (+    19us) tablet.cc:662] Decoding operations
0709 12:19:59.089207 (+167166us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:19:59.089320 (+   113us) write_op.cc:641] Partition lock acquired for write op
0709 12:19:59.089352 (+    32us) tablet.cc:685] Acquiring locks for 34 operations
0709 12:19:59.149704 (+ 60352us) tablet.cc:701] Row locks acquired
0709 12:19:59.151629 (+  1925us) write_op.cc:260] PREPARE: finished
0709 12:19:59.189895 (+ 38266us) write_op.cc:270] Start()
0709 12:19:59.196517 (+  6622us) write_op.cc:276] Timestamp: P: 1783599599182651 usec, L: 0
0709 12:19:59.209768 (+ 13251us) op_driver.cc:348] REPLICATION: starting
0709 12:19:59.306090 (+ 96322us) log.cc:844] Serialized 13081 byte log entry
0709 12:20:00.262163 (+956073us) spinlock_profiling.cc:243] Waited 4.23 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:20:00.710403 (+448240us) op_driver.cc:464] REPLICATION: finished
0709 12:20:00.756462 (+ 46059us) write_op.cc:301] APPLY: starting
0709 12:20:00.756570 (+   108us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:00.958529 (+201959us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:00.958620 (+    91us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:01.503517 (+544897us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:01.654870 (+151353us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=68,key_file_lookups=68,delta_file_lookups=0,mrs_lookups=0
0709 12:20:01.666675 (+ 11805us) write_op.cc:312] APPLY: finished
0709 12:20:01.751882 (+ 85207us) log.cc:844] Serialized 292 byte log entry
0709 12:20:02.044083 (+292201us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:02.243597 (+199514us) write_op.cc:454] Released schema lock
0709 12:20:02.243721 (+   124us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:02.244480 (+   759us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:02.244512 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":75345,"reported_tablets":53,"child_traces":[["op",{"apply.queue_time_us":40510,"apply.run_cpu_time_us":92030,"apply.run_wall_time_us":1493149,"cfile_cache_hit":138,"cfile_cache_hit_bytes":145975,"num_ops":34,"prepare.queue_time_us":132493,"prepare.run_cpu_time_us":28464,"prepare.run_wall_time_us":760985,"raft.queue_time_us":108938,"raft.run_cpu_time_us":11497,"raft.run_wall_time_us":32160,"replication_time_us":1491074,"spinlock_wait_cycles":9719552,"thread_start_us":641743,"threads_started":5,"wal-append.queue_time_us":742525,"wal-append.run_cpu_time_us":21001,"wal-append.run_wall_time_us":3688215}]]}
I20260709 12:20:07.352288  6502 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:20:07.334061 13697 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:07.366654 13518 raft_consensus.cc:740] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:07.360960 13861 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:07.461156 13861 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:07.463333  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:07.463716 13861 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:07.466801  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:07.396255  6474 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:07.469274  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 73077.1us
I20260709 12:20:07.470643 13697 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:07.471114 13697 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:07.471555 13697 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:07.473129 13518 consensus_queue.cc:260] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:07.397120 13763 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:07.474341 13518 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:07.428366 13879 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:07.379364  6654 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 23.
W20260709 12:20:07.440006  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 149411us
W20260709 12:20:07.444008  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 48123.1us
I20260709 12:20:07.492295  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:20:07.498363  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 203180us
I20260709 12:20:07.496881  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:07.503206  6575 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:07.507409  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:07.517654 13886 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:07.518275 13886 raft_consensus.cc:697] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:07.518321 13880 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:07.519188 13886 consensus_queue.cc:237] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:07.520108  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 913) took 1065 ms. Trace:
I20260709 12:20:07.520526  6501 rpcz_store.cc:276] 0709 12:20:06.455035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:06.455621 (+   586us) service_pool.cc:224] Handling call
0709 12:20:06.455819 (+   198us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:20:06.459064 (+  3245us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:06.459097 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:20:06.459692 (+   595us) log.cc:844] Serialized 52 byte log entry
0709 12:20:06.596663 (+136971us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:20:06.610174 (+ 13511us) log.cc:844] Serialized 14 byte log entry
0709 12:20:06.946555 (+336381us) spinlock_profiling.cc:243] Waited 43.5 ms on lock 0x7b18004c57c0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:20:07.519428 (+572873us) spinlock_profiling.cc:243] Waited 2.83 ms on lock 0x7b18004c57c0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d16c0a 00007f287c456219 00007f2872d9ea40 00007f2872d994e4 00007f2872d99455 00007f28798813c7 00007f2875e939a7 00007f2875ea30bf 00007f2875ea3038
0709 12:20:07.519614 (+   186us) log.cc:844] Serialized 14 byte log entry
0709 12:20:07.519825 (+   211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:07.519895 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:07.519935 (+    40us) raft_consensus.cc:1685] finished
0709 12:20:07.519946 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:07.520060 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14779,"prepare.queue_time_us":14327,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":40,"raft.queue_time_us":5006,"raft.run_cpu_time_us":349,"raft.run_wall_time_us":9886,"spinlock_wait_cycles":106502272,"thread_start_us":128291,"threads_started":1,"wal-append.queue_time_us":199468}
I20260709 12:20:07.524087 13822 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:07.524590  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:07.526904  7238 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:07.528749 13682 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:07.529237 13682 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:07.531242 13682 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:07.532645  6502 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:07.535218  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:07.497903 13674 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:07.536936  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:07.538053  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:07.538717  6575 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:07.543824  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:07.546939  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:07.547847  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:07.552345  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.499308 13767 consensus_queue.cc:1059] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 12:20:07.558188  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:07.503728 13138 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:07.560226  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:07.567565  6581 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:20:07.560935  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:07.561129  6502 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:20:07.508219 13883 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:07.569972  6575 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 23.
I20260709 12:20:07.537726 13863 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:07.557883 13754 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:20:07.507714 13656 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 12:20:07.557111 13660 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:07.609651 13660 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:07.601289  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.608938 13656 consensus_queue.cc:1243] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 21.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:20:07.616048 13697 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:07.608820  6622 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:07.618888  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:07.647754  6622 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:07.561465 13554 consensus_queue.cc:1059] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 18.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.060s
I20260709 12:20:07.646929  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:07.598851 13668 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:07.657755  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:20:07.657119  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 142909us
I20260709 12:20:07.661904  6650 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:07.618237 13754 raft_consensus.cc:697] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 15 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:07.628574 13767 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:07.636185 13656 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:07.626955 13660 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:07.677919  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:07.654197 13886 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:20:07.673666  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.659403  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:07.583516 13863 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:07.676905  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 2850) took 2268 ms. Trace:
I20260709 12:20:07.667100 13778 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:07.582538 13883 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:07.629220 13827 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 12:20:07.691519  6584 rpcz_store.cc:276] 0709 12:20:05.408161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:05.484153 (+ 75992us) service_pool.cc:224] Handling call
0709 12:20:05.486882 (+  2729us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:05.855749 (+368867us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:05.855780 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:06.085633 (+229853us) spinlock_profiling.cc:243] Waited 5.02 ms on lock 0x7b2c00010c48. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:20:06.086066 (+   433us) log.cc:844] Serialized 26 byte log entry
0709 12:20:06.201404 (+115338us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:06.201452 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:07.675809 (+1474357us) spinlock_profiling.cc:243] Waited 307 ms on lock 0x7b54001d0bd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:07.675882 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:07.675936 (+    54us) raft_consensus.cc:1685] finished
0709 12:20:07.675952 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:07.676843 (+   891us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72633,"raft.queue_time_us":108655,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":15668,"spinlock_wait_cycles":718504576,"thread_start_us":114966,"threads_started":1,"wal-append.queue_time_us":136273,"wal-append.run_cpu_time_us":2047,"wal-append.run_wall_time_us":1033472}
I20260709 12:20:07.674154 13767 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:07.632974 13677 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:20:07.703541  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:07.725396  6508 raft_consensus.cc:2393] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 20: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:20:07.715788  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:07.726600  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:07.717226  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:07.727689  6507 raft_consensus.cc:2393] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 19: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:07.667757 13794 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:07.690049 13740 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:07.734828  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:07.674350 13885 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 12:20:07.730460  6504 pending_rounds.cc:85] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 13
I20260709 12:20:07.756309  6504 pending_rounds.cc:107] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 21.14
I20260709 12:20:07.756683  6504 raft_consensus.cc:2889] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:07.666342 13668 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:07.737201  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.738236  6503 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:07.699932 13758 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:07.742399  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:07.744946  6502 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 11. (term mismatch)
I20260709 12:20:07.762216  6502 pending_rounds.cc:85] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 10
I20260709 12:20:07.762501  6502 pending_rounds.cc:107] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 21.11
I20260709 12:20:07.762768  6502 raft_consensus.cc:2889] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:07.735133  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:07.732926 13886 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:07.580004  6398 catalog_manager.cc:5697] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 9 to 14, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 14 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:07.761175  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:20:07.747164  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 116897us
I20260709 12:20:07.766939  6584 raft_consensus.cc:1240] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:07.731029 13767 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:07.704443 13887 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 12:20:07.710912 13881 consensus_queue.cc:1059] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
I20260709 12:20:07.809814  6651 raft_consensus.cc:1240] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:07.714686 13804 consensus_queue.cc:1048] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
W20260709 12:20:07.778015 13859 log.cc:927] Time spent T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:20:07.802421 13794 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:07.755920 13740 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:07.837307  6501 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:07.861292  6507 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:20:07.820408  6398 catalog_manager.cc:5697] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:20:07.855285 13819 consensus_meta.cc:287] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.536s	user 0.010s	sys 0.002s
W20260709 12:20:07.858731 13889 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
I20260709 12:20:07.869594  6580 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 22. Current term is 23. Ops: [22.11-22.11]
I20260709 12:20:07.854792 13697 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:07.859189 13656 raft_consensus.cc:697] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 23 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:07.877458  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 920) took 1150 ms. Trace:
W20260709 12:20:07.884272 13890 log.cc:927] Time spent T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.230s	user 0.000s	sys 0.001s
I20260709 12:20:07.886529  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:07.887478  6584 raft_consensus.cc:2468] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 9.
I20260709 12:20:07.821533 13754 consensus_queue.cc:237] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:07.882687  6509 rpcz_store.cc:276] 0709 12:20:06.726595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:06.726925 (+   330us) service_pool.cc:224] Handling call
0709 12:20:06.744939 (+ 18014us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:06.745191 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:20:07.688945 (+943754us) log.cc:844] Serialized 14 byte log entry
0709 12:20:07.875488 (+186543us) log.cc:844] Serialized 14 byte log entry
0709 12:20:07.876129 (+   641us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:07.876175 (+    46us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:20:07.876217 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:07.876283 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:07.877393 (+  1110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":812888,"prepare.queue_time_us":925171,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":85,"raft.queue_time_us":126589,"raft.run_cpu_time_us":359,"raft.run_wall_time_us":3532,"thread_start_us":316166,"threads_started":2}
I20260709 12:20:07.890122 13796 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:07.894136  6398 catalog_manager.cc:5697] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 10 to 16, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 16 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:07.831810 13794 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:07.896955 13668 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:07.875342 13860 consensus_queue.cc:1059] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 12:20:07.897886 13668 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:07.898154 13794 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:07.898365 13668 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:07.899204 13863 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:07.900746 13668 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:07.901871 13863 raft_consensus.cc:3055] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:20:07.902360 13863 raft_consensus.cc:740] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:07.906307 13740 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:07.906881  6658 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
W20260709 12:20:07.911156 13897 log.cc:927] Time spent T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 12:20:07.913591 13893 log.cc:927] Time spent T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
W20260709 12:20:07.916957 13712 consensus_meta.cc:287] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.570s	user 0.011s	sys 0.000s
I20260709 12:20:07.930174  6657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 2929) took 1071 ms. Trace:
I20260709 12:20:07.930696  6657 rpcz_store.cc:276] 0709 12:20:06.858533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:06.858740 (+   207us) service_pool.cc:224] Handling call
0709 12:20:06.869918 (+ 11178us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:07.329340 (+459422us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:07.929890 (+600550us) spinlock_profiling.cc:243] Waited 458 ms on lock 0x7b54002a35d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:07.930106 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1054210944}
I20260709 12:20:07.886580 13891 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:07.926568 13758 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:07.977623 13763 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:07.978518 13763 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:07.951674 13794 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:07.943231 13734 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:07.903462 13863 consensus_queue.cc:260] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:07.996138  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 336821us
W20260709 12:20:08.008859  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:08.010484  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:07.963410 13891 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:07.991055 13716 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:08.020279  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:08.000702 13883 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:08.022989  7267 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 15 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 14. Current term is 15. Ops: []
I20260709 12:20:07.983865 13758 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:08.025840  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:08.030076  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:08.001029 13794 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:08.032065  7267 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
W20260709 12:20:08.034610  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 306129us
I20260709 12:20:08.002159 13734 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:08.016390 13656 consensus_queue.cc:237] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:08.045150 13518 consensus_meta.cc:287] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.570s	user 0.011s	sys 0.000s
I20260709 12:20:08.026597 13716 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:08.045892 13763 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:08.047075  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:08.047327 13716 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:08.047804  6575 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22.
I20260709 12:20:08.050181  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:08.039108 13900 log.cc:927] Time spent T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 12:20:08.070576 13891 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:08.063097 13909 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:08.072551 13909 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:08.072943 13909 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:08.094790 13886 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.104740  7238 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:08.104954  6548 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:08.107582  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:08.109184  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:08.104164  6576 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:08.114326  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:08.096946 13922 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 12:20:08.141292  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:08.142127  6509 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 18. Current term is 19. Ops: []
W20260709 12:20:08.161590  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 56854.3us
W20260709 12:20:08.167729  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 63075.4us
I20260709 12:20:08.140909  6505 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:20:08.115206  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:08.176438  6655 raft_consensus.cc:2410] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 22 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:20:08.148815 13741 consensus_queue.cc:1059] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.017s
W20260709 12:20:08.207046  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:08.104233 13823 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:08.198367 13741 consensus_queue.cc:1243] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer 20a202c76244485fbc8f1610473db8b9 log is divergent from this leader: its last log entry 15.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
W20260709 12:20:08.214751 13918 log.cc:927] Time spent T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:20:08.139957 13915 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:08.223278  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 121543us
I20260709 12:20:08.232652  6623 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:08.227098 13919 log.cc:927] Time spent T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:20:08.229462 13909 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:08.217240 13916 log.cc:927] Time spent T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 12:20:08.230325 13734 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:08.198781 13861 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:08.223359  6654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49792 (request call id 1184) took 1007 ms. Trace:
I20260709 12:20:08.229846 13712 consensus_queue.cc:1059] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:08.242376  6654 rpcz_store.cc:276] 0709 12:20:07.216376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:07.216572 (+   196us) service_pool.cc:224] Handling call
0709 12:20:08.223197 (+1006625us) spinlock_profiling.cc:243] Waited 95.3 ms on lock 0x7b54000b6b54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:08.223303 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":219220224}
I20260709 12:20:08.217082 13865 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:08.257284  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:08.151060 13886 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:08.248696 13922 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:08.259879 13922 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:08.260963 13824 raft_consensus.cc:3055] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:20:08.261415 13824 raft_consensus.cc:740] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:08.262243 13824 consensus_queue.cc:260] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:08.263476 13824 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:20:08.265451 13794 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:08.249800 13929 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:08.266180 13929 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:08.266584 13929 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:08.256204 13861 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:08.240039 13915 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:08.269913 13827 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.270452 13827 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:08.271025 13915 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:08.272536 13621 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:08.250947 13712 raft_consensus.cc:3055] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:08.273131 13712 raft_consensus.cc:740] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:08.274245 13712 consensus_queue.cc:260] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:08.275743 13712 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:08.103591 13908 consensus_queue.cc:1059] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.139s
I20260709 12:20:08.234103 13518 consensus_queue.cc:1059] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 18.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.022s
I20260709 12:20:08.276886 13697 raft_consensus.cc:3055] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:08.277241 13697 raft_consensus.cc:740] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:08.256522 13767 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:08.278004 13697 consensus_queue.cc:260] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:08.278771 13892 log.cc:927] Time spent T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.591s	user 0.004s	sys 0.003s
I20260709 12:20:08.279209 13697 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:08.280052 13767 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:08.222997 13823 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:08.281173 13911 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.233816 13881 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.262594 13922 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:08.281886 13911 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:08.282296 13881 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:08.282684 13827 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:08.285538 13823 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:08.250172 13909 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:08.290355 13674 raft_consensus.cc:3055] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:08.290858 13674 raft_consensus.cc:740] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:08.291216 13881 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:08.295704  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:08.297926  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 53759.5us
I20260709 12:20:08.300653  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:08.302055  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:08.305589  6502 raft_consensus.cc:1240] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:08.306810  6507 pending_rounds.cc:85] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 10
I20260709 12:20:08.307158  6507 pending_rounds.cc:107] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 21.11
I20260709 12:20:08.307446  6507 raft_consensus.cc:2889] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:08.302930  7271 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:20:08.309607  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:08.310241  6509 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:20:08.310269  6474 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:20:08.311836  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:08.313985  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:08.298857  6548 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:08.317543  6548 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:08.323500  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:08.330979  6506 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:20:08.331439  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:08.329572  6622 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:08.259948 13886 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:08.340890  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:08.335246  7240 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:08.291888 13653 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:08.350544  6474 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:08.340680 13908 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.323328 13668 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:20:08.291848 13674 consensus_queue.cc:260] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:08.326771 13697 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:08.354439 13883 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:20:08.376220  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 46756.6us
I20260709 12:20:08.366283 13827 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:08.311041 13861 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 12:20:08.377841  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:08.345566 13692 consensus_queue.cc:1059] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.032s
I20260709 12:20:08.380555  7267 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:20:08.354156 13653 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:08.385140 13653 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:08.385732 13653 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:08.386263 13653 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:08.377669 13886 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:08.390283 13653 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:08.372735 13697 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:08.396085  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:08.398460 13781 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:08.377655 13888 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.400861 13888 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:08.403944 13888 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:08.375741 13668 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:08.405220 13674 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:20:08.406072 13927 log.cc:927] Time spent T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.194s	user 0.000s	sys 0.000s
I20260709 12:20:08.406613 13912 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.406438 13692 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:08.407135 13697 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:08.408355  6474 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:08.392503 13883 raft_consensus.cc:697] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 23 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:08.408087 13930 raft_consensus.cc:3055] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:08.420709 13863 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:20:08.424000  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 91377.4us
I20260709 12:20:08.377341 13911 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:08.428321  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:08.433220  6548 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:08.366430 13929 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:08.434015  6548 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:08.369043 13908 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:08.430179 13668 raft_consensus.cc:697] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:08.439883 13861 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:08.441344 13861 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:08.441946 13861 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:08.442416 13883 consensus_queue.cc:237] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:08.431524 13912 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:08.432636 13930 raft_consensus.cc:740] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:08.437142 13888 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:08.436455 13747 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:08.453401  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:08.453881 13929 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:08.456451 13660 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:08.445604 13653 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:08.464152  7267 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:08.463253 13888 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:08.475642  7240 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:08.479139 13754 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:08.473209 13653 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:08.485695 13653 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:08.486435 13928 log.cc:927] Time spent T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.345s	user 0.001s	sys 0.000s
I20260709 12:20:08.486675 13653 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:08.471909 13660 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:08.491576 13660 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:08.466542 13937 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:08.468990 13747 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:08.477922 13888 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:08.441773 13668 consensus_queue.cc:237] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:08.478379 13938 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:08.529063 13653 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:08.544716  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:08.550336  6581 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 20.
I20260709 12:20:08.546613  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:08.529812 13747 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:08.553540  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:08.554953 13912 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:08.556438  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:08.557400  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:08.558007  6578 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22.
I20260709 12:20:08.558967  6578 raft_consensus.cc:1240] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:20:08.559073  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:08.560014 13891 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.560817  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:08.560743 13891 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:08.534454 13778 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:08.563093 13778 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:08.565542 13697 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:08.552558 13930 consensus_queue.cc:260] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:08.568197 13930 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:08.570798 13911 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:08.560065 13621 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.571674 13621 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:08.571444 13911 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:08.574100 13621 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:08.574659 13911 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:08.577457  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:08.578146  7273 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:20:08.586745  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:08.584574  6473 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:08.587666  6657 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22.
I20260709 12:20:08.587908  6474 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:08.588871  6473 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:08.590735  7267 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:20:08.590893  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:08.592082 13740 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:08.595842 13716 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:08.596385  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:08.596642 13716 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:08.597198 13716 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:08.597666  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:08.598052 13798 consensus_queue.cc:1059] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:08.598480  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:08.599193  6578 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
W20260709 12:20:08.600401 13936 log.cc:927] Time spent T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 12:20:08.554280 13938 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:08.600332  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:08.605991  6575 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:20:08.609174  6509 raft_consensus.cc:1275] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:20:08.606325  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:08.608906  6623 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:08.607034  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:08.612571  6500 raft_consensus.cc:2393] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 21: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:08.607955  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:08.596141  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:08.614025  6583 raft_consensus.cc:2393] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 16: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:08.614292  7238 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:08.614710  6658 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 26: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:08.615067  6657 raft_consensus.cc:2393] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 18: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:08.606861  6578 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 26.
I20260709 12:20:08.549502 13660 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:08.625392  6657 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 12:20:08.626675  6500 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 12:20:08.602460 13923 log.cc:927] Time spent T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.470s	user 0.004s	sys 0.001s
I20260709 12:20:08.627432 13929 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:08.629571 13939 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:08.630934 13908 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:08.617393  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:08.633342  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:08.634413  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:08.634698  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:08.640858  6500 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:20:08.636389  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:08.616021 13751 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.641935  6578 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:08.616670 13824 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.646198 13824 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:08.647322  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:08.649147 13712 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:08.649756 13712 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:08.653426 13653 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:08.657433 13872 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:08.657989 13872 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:08.658355 13872 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:08.643446 13660 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:08.663314 13660 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:08.641350  6509 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:20:08.626921 13740 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.532522 13937 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:08.633021 13915 raft_consensus.cc:2804] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:20:08.669464  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 86262.6us
I20260709 12:20:08.675246  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:08.677083  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:08.678006  6473 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:08.678349  6500 raft_consensus.cc:2393] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 19: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:08.679422  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:08.683702  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:08.618666 13778 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:20:08.639914 13697 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:08.665731 13929 raft_consensus.cc:697] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:08.689985 13621 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:08.688318 13767 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:08.729367  6657 raft_consensus.cc:1240] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:08.675246 13915 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:08.730852  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:08.730706 13767 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:08.733819 13846 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 12:20:08.690237 13798 consensus_queue.cc:1059] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:20:08.744195  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:08.716634 13824 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:08.745606  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:08.745798 13767 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:08.746671  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:08.747507  6583 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:08.748466  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:08.749217 13767 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:08.727543 13621 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:08.668056 13751 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:08.749723 13621 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:08.751525 13751 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:08.643630 13939 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:08.754170 13939 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:08.755263  6583 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:20:08.745779 13929 consensus_queue.cc:237] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:08.756244  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:08.757519 13621 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:08.757804  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:08.758790  6622 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:08.686789 13740 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:08.775856  6578 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:20:08.692935 13861 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:08.741020 13915 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:20:08.781055  6622 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:08.789745  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:08.794752  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:08.795620  6506 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:20:08.789104  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:08.801260  6500 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22.
I20260709 12:20:08.800091  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:08.802439  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:08.752223 13751 raft_consensus.cc:3055] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:08.804116  6506 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:08.803272  6500 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:20:08.806967  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:08.808540  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:08.811020  6622 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:08.812510 13823 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:20:08.805838 13923 log.cc:927] Time spent T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:20:08.813581  6622 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:08.813660 13656 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:08.799695 13922 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:08.814229 13656 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:08.814675 13656 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:08.814759  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:08.814466 13933 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:08.815973 13933 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:08.817862  6502 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 22: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:08.817493  6504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 3173) took 1115 ms. Trace:
I20260709 12:20:08.816314 13933 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:08.823930  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:08.825066 13740 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:08.785549 13758 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:08.827227 13778 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:08.828999 13716 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:08.831082 13716 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:08.831837  6505 raft_consensus.cc:2393] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 18: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:08.792698 13938 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:08.832530  7274 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:20:08.832685 13937 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:08.784210 13767 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:08.794181 13697 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.834934 13861 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:08.835031 13697 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:08.824081 13819 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:08.821127  6504 rpcz_store.cc:276] 0709 12:20:07.701803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:07.701927 (+   124us) service_pool.cc:224] Handling call
0709 12:20:07.702265 (+   338us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:20:07.956523 (+254258us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:07.956592 (+    69us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:20:07.962757 (+  6165us) log.cc:844] Serialized 52 byte log entry
0709 12:20:08.024507 (+ 61750us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:20:08.055836 (+ 31329us) log.cc:844] Serialized 14 byte log entry
0709 12:20:08.057350 (+  1514us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:08.057449 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:08.253361 (+195912us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b48000d0a58. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287adb2aa1 00007f287ada9b37 00007f287ada99ca 00007f287aea126f 00007f287aea0f37 00007f287aeb555c 00007f287aeb5508 00007f287aeb5488 00007f287aeb5443 00007f287aeb46ec 00007f2879ddea96
0709 12:20:08.419154 (+165793us) spinlock_profiling.cc:243] Waited 1.97 ms on lock 0x7b180008abe0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c522 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:20:08.601767 (+182613us) spinlock_profiling.cc:243] Waited 18.1 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:20:08.817273 (+215506us) raft_consensus.cc:1685] finished
0709 12:20:08.817293 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:08.817422 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5596,"prepare.queue_time_us":29995,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"raft.queue_time_us":80297,"raft.run_cpu_time_us":7632,"raft.run_wall_time_us":422509,"spinlock_wait_cycles":84414080,"thread_start_us":231602,"threads_started":2,"wal-append.queue_time_us":92640}
I20260709 12:20:08.838339  6577 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 22: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:08.842576  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:08.842584  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:08.843451  7273 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:08.809882 13751 raft_consensus.cc:740] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:08.857113  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:08.857831  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:08.860067  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:08.860176  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:08.863065  6583 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 23.
I20260709 12:20:08.816531 13824 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:08.869127  6500 raft_consensus.cc:2393] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 19: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:08.842711 13819 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:08.876582  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:08.858147 13767 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.882521  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:08.863508 13953 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:08.882853  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 201967us
I20260709 12:20:08.865190 13930 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:08.873457 13824 raft_consensus.cc:697] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 19 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:08.891000  6506 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:20:08.814447 13922 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:08.894325  6475 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:08.894511  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:08.884042 13819 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:08.897199  6582 raft_consensus.cc:2410] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 23 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 12:20:08.899112  6504 raft_consensus.cc:1240] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:08.892143 13881 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:08.913066  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:08.909228 13846 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.955076 13922 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:08.959581  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:08.960372  6582 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:20:08.962117  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:08.962558  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:08.963157  6651 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 24.
I20260709 12:20:08.968508  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:08.969363  6654 raft_consensus.cc:2410] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 23 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 12:20:08.894048 13767 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:08.972569  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:08.997269  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:08.973640  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:08.974344  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:08.898248 13751 consensus_queue.cc:260] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:08.982154 13949 log.cc:927] Time spent T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:20:08.895466 13953 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:08.984616 13621 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:08.972892  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:08.998806  6582 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:20:08.998956  6658 raft_consensus.cc:2410] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 22 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 12:20:08.970290 13740 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:20:08.999451  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:08.967007 13922 consensus_queue.cc:1059] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 12:20:08.963094 13846 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:09.021641  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:09.022295  6579 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:20:09.025005  6583 raft_consensus.cc:1240] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:20:08.989948 13961 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:08.975862 13656 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.001952 13911 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.032140 13888 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.034307 13888 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:09.035158  6577 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:09.034968 13961 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:08.985839 13824 consensus_queue.cc:237] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.039295 13767 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:09.039690 13751 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:09.044166  7273 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:20:09.044967  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:09.045595  6504 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:20:08.977236 13872 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:09.045666  6474 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:09.049381  6582 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:09.064836  6577 raft_consensus.cc:1275] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:20:09.073624  6579 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:20:09.044612 13697 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:09.078612  7267 raft_consensus.cc:2393] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 21: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:20:09.045431 13922 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.017205 13740 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:09.049978 13911 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.082103 13964 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:09.104261  6475 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:09.104920 13952 log.cc:927] Time spent T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 12:20:09.105247 13967 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:09.105695 13967 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:09.106141 13967 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:09.106707  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:09.108566 13656 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:09.105628  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:09.116540 13697 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:09.118255 13964 raft_consensus.cc:697] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:09.118809 13967 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.119206 13964 consensus_queue.cc:237] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:09.120026  6507 raft_consensus.cc:2368] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 24. Current term is 25.
I20260709 12:20:09.122228 13967 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:09.089826 13922 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.125941 13846 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:09.141880  6474 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:09.150815 13822 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:09.151330 13822 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:09.151741 13822 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:09.138574 13712 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:09.154668 13933 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.138082 13872 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:09.157931  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:09.159453  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:09.160046  6500 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:20:09.162563 13747 consensus_meta.cc:287] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.569s	user 0.004s	sys 0.004s
W20260709 12:20:09.162690 13926 log.cc:927] Time spent T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 1.016s	user 0.000s	sys 0.008s
I20260709 12:20:09.133270 13922 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:09.167308 13956 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:09.181195  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 3180) took 1467 ms. Trace:
I20260709 12:20:09.181687  6508 rpcz_store.cc:276] 0709 12:20:07.714121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:07.740903 (+ 26782us) service_pool.cc:224] Handling call
0709 12:20:07.741483 (+   580us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:07.921978 (+180495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:07.922014 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:07.922690 (+   676us) log.cc:844] Serialized 26 byte log entry
0709 12:20:08.093752 (+171062us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:08.093807 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:08.093902 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:09.162085 (+1068183us) spinlock_profiling.cc:243] Waited 20 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:20:09.180827 (+ 18742us) raft_consensus.cc:1685] finished
0709 12:20:09.180850 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:09.181124 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":88072,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":23845,"spinlock_wait_cycles":46051584,"thread_start_us":349810,"threads_started":2,"wal-append.queue_time_us":223647}
I20260709 12:20:09.183892  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:09.206749 13911 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:09.190213  6576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 2877) took 2503 ms. Trace:
I20260709 12:20:09.207269  6576 rpcz_store.cc:276] 0709 12:20:06.704142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:07.040874 (+336732us) service_pool.cc:224] Handling call
0709 12:20:07.041181 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:08.160045 (+1118864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:09.189954 (+1029909us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54000578d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:09.190150 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2439272704}
W20260709 12:20:09.191066 13957 log.cc:927] Time spent T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 12:20:09.190719 13819 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.214010 13819 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:09.215818 13946 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:09.193333 13933 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:09.200066 13822 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:09.217880  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:20:09.218111  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:09.218690  6502 raft_consensus.cc:2393] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 21: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:09.218901 13822 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:09.219558  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:09.221674 13956 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:09.222188  6550 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:09.222936 13953 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:09.194546  6651 raft_consensus.cc:1275] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:20:09.225418  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:09.225788  6622 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:09.226157  7274 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:20:09.227185  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:09.227918  7273 raft_consensus.cc:2368] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18.
I20260709 12:20:09.195420 13828 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:09.228715 13888 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:09.228914 13933 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:20:09.229319 13710 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.229537 13888 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.230105  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:09.229904 13710 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.197180 13741 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:09.198227 13771 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.199371  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:09.239825  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:09.240438  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:09.228039  6622 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:09.241101  6506 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:09.241216  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:09.204633 13915 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:09.248075 13915 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:09.168951 13747 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:09.220180  6502 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:09.232666  6500 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22.
I20260709 12:20:09.233186  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:09.250141  6578 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:20:09.250473  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:09.237521  6508 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 18.
I20260709 12:20:09.249163 13741 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.256916  6548 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:09.202445  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:09.242100  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:09.269654 13712 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:09.259007  6506 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:20:09.276235 13712 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.223598  6550 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:09.251207 13660 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:09.279436 13747 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:09.252640  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:09.253084 13737 consensus_queue.cc:1059] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:09.280575  6509 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:09.281229 13737 raft_consensus.cc:3055] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:09.281620 13737 raft_consensus.cc:740] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:09.253892  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:09.282478 13737 consensus_queue.cc:260] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:09.278609  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 55099.5us
I20260709 12:20:09.283381 13740 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:09.253917 13710 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:09.255975  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:09.257577 13658 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:09.284343 13668 consensus_queue.cc:1048] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:09.286413  6578 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:20:09.286995 13961 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:09.287237 13668 consensus_queue.cc:1048] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:09.289535  6550 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:09.290810 13668 raft_consensus.cc:2764] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:20:09.291409 13656 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.293051 13796 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.293706 13796 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.294857 13692 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.295538  6655 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:09.296303  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:09.296582  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:09.292011 13656 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.297286  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:09.297716  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:09.297916  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:09.298794 13796 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:09.298997  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:09.299230 13656 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:09.297349  6656 raft_consensus.cc:2368] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20.
I20260709 12:20:09.255076 13771 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:09.300310  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:09.239283 13653 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.302076 13771 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:09.279834 13660 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:20:09.280113  6501 raft_consensus.cc:2393] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 18: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:09.302600 13653 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.303720 13769 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:09.304250 13769 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:09.305154 13769 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:09.305382  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:09.282989  6582 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:20:09.227939  6503 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:09.253986 13933 consensus_queue.cc:1048] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:20:09.309693  6577 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:20:09.259663  6508 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 23 index: 11. (term mismatch)
I20260709 12:20:09.277280  6506 raft_consensus.cc:1275] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:20:09.315357  6508 pending_rounds.cc:85] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 10
I20260709 12:20:09.315783  6508 pending_rounds.cc:107] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 22.11
I20260709 12:20:09.275740  6583 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:20:09.316094  6508 raft_consensus.cc:2889] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:09.243695  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:09.267076  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:09.316951  6507 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 23.
I20260709 12:20:09.240556  6581 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 19.
I20260709 12:20:09.275322 13891 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:09.242780  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:09.320712  6656 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:20:09.321417  6576 raft_consensus.cc:2393] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 21: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:09.262643 13781 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 12:20:09.307159  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:09.329567  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:09.319132  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:09.330206  6652 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:09.330643  6657 raft_consensus.cc:2393] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 24: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:09.295876 13692 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.333844  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:09.324851  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:09.308424  6549 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:09.283788 13737 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:20:09.325384  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 68568.8us
I20260709 12:20:09.355333  6548 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:09.311738  6502 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:09.222199 13956 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:09.357128  6579 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 26.
I20260709 12:20:09.332511  6653 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:20:09.336627  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:09.308955  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:09.339234  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:09.265043 13955 raft_consensus.cc:3055] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:09.339633  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:09.341221 13932 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:09.341353  7274 raft_consensus.cc:1240] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 9. Current term is 10. Ops: []
I20260709 12:20:09.341636 13922 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:09.341778 13740 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:09.342443  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:09.333065 13658 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 12:20:09.343722  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:09.296657 13930 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.345676 13653 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:09.347599 13712 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:09.348393  6550 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:09.394598  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:09.395298  6658 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:20:09.395556  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 47275.4us
I20260709 12:20:09.395805 13740 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.383436  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:09.396838  6550 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:09.397097 13888 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:09.397740 13888 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:09.398056 13740 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:09.353428  6655 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:09.329787  6580 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 23.
I20260709 12:20:09.357972  6503 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:20:09.398617  6550 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:09.401814 13712 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:09.403288  6550 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:09.404073 13712 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:09.383175 13932 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:09.244292 13946 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:09.405753  6550 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:09.406945 13712 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:09.407310 13712 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:09.407675 13712 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:09.408372 13946 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:09.408972 13879 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:09.407795 13932 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:09.409682 13881 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:09.410296 13881 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:09.415880 13712 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.388316  6509 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 23.
I20260709 12:20:09.417996 13712 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:09.418010  6658 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 25.
I20260709 12:20:09.418972 13946 consensus_queue.cc:1059] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:09.387418  6577 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 25.
I20260709 12:20:09.419898 13946 consensus_queue.cc:1243] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer f2b06aebe97249a4877d798b8118ece4 log is divergent from this leader: its last log entry 20.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:20:09.420324 13653 raft_consensus.cc:3055] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:09.420778  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:09.420693 13653 raft_consensus.cc:740] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:09.421406  6658 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:20:09.421442 13881 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.422212 13653 consensus_queue.cc:260] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.423779 13881 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:09.423946  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:09.424490  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:09.425288  6576 raft_consensus.cc:2393] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 16: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:09.425452  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:09.424628  6577 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:20:09.426008  6579 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:20:09.426975  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:09.427313  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:09.429859  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 1169) took 1520 ms. Trace:
I20260709 12:20:09.430214  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:09.432626  6475 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:09.433609  6475 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:09.434593  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:09.435392 13796 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:09.435774 13796 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:09.437337  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:09.436117 13796 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:09.438617  6622 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:09.439296 13974 log.cc:927] Time spent T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.090s	user 0.003s	sys 0.000s
I20260709 12:20:09.358806  6500 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:20:09.259044 13741 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:09.308538 13911 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:09.384339  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:09.388962  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 51865.7us
I20260709 12:20:09.430285  6584 rpcz_store.cc:276] 0709 12:20:07.909674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:08.542923 (+633249us) service_pool.cc:224] Handling call
0709 12:20:09.429208 (+886285us) spinlock_profiling.cc:243] Waited 5.86 ms on lock 0x7b5800116198. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64eee 000000000036eb39 00007f287add81e0 00007f287aea10c4 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:20:09.429361 (+   153us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:09.429537 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:09.429561 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:09.429618 (+    57us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:20:09.429650 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:09.429703 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:09.429805 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13530112}
I20260709 12:20:09.452847  6503 raft_consensus.cc:1275] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:20:09.454264  6502 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22.
I20260709 12:20:09.455942  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:09.391727  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:09.384189  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:09.458878  6653 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 26: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:09.384529  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:09.387775 13692 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:09.393399 13930 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.398553 13747 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.471129  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:09.427681  6658 raft_consensus.cc:2393] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 23: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:20:09.467207  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:09.438732 13932 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:09.228726 13828 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.441622  6550 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:09.442549  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 189215us
I20260709 12:20:09.397750  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:09.444222  6473 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:09.360291  6652 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:09.359316  6548 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:09.460914  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:09.458274  6656 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22.
I20260709 12:20:09.465077  7267 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:20:09.398419 13888 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:20:09.459707  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:09.481781  6550 leader_election.cc:400] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20.
I20260709 12:20:09.466568  6583 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:20:09.470534  7273 raft_consensus.cc:1275] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:20:09.472127  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:09.475394  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:09.381750 13955 raft_consensus.cc:740] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:09.482656  6550 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:09.444811 13964 consensus_queue.cc:1059] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 23.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.093s
I20260709 12:20:09.382833 13947 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:09.486505  7274 raft_consensus.cc:2393] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 18: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:09.486724  6649 raft_consensus.cc:2410] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 22 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 12:20:09.488201  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:09.464609 13716 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:09.457870 13881 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:09.492908  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:09.445102 13922 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:09.497316  6473 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:09.498780  6622 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:09.467456 13656 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:09.491925 13660 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:20:09.496913  6653 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22.
I20260709 12:20:09.492509  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
W20260709 12:20:09.499998  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 140745us
I20260709 12:20:09.501972  6623 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:09.468029 13933 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:09.508602 13692 consensus_queue.cc:1059] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 12:20:09.513419  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:09.514036  6658 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:20:09.478971 13741 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.487149 13668 raft_consensus.cc:2764] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:20:09.468853 13824 consensus_queue.cc:1048] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:20:09.502427 13879 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:09.525410  6473 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:09.519876  6474 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:09.519838  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:09.509801  6652 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 12:20:09.518066 13554 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:09.511483  6548 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:09.469992 13911 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:09.504509 13819 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:09.502305 13716 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:09.510332  6550 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:09.524068 13929 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:09.478919 13925 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:20:09.516762 13908 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:09.518570 13758 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:09.547506  6474 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:09.478117 13747 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:09.549306  7238 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:09.525182 13828 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:09.512712 13710 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:09.519032 13943 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:09.555924  6473 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:09.525938 13881 raft_consensus.cc:697] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 25 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:09.565340  6548 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:09.545452 13554 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:09.571251  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 60980.3us
I20260709 12:20:09.518863 13937 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 12:20:09.530422 13656 raft_consensus.cc:697] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:09.523913 13741 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:09.585047  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 67255.4us
I20260709 12:20:09.553550 13674 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:09.568591 13953 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:09.548662 13819 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:09.584206 13740 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:20:09.591676 13796 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.551587 13716 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:09.597993  6652 raft_consensus.cc:1240] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:09.544245 13879 raft_consensus.cc:697] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:09.549154 13467 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:09.548913 13941 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:09.642527  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:09.605343  6548 leader_election.cc:400] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 24. Current term is 25.
I20260709 12:20:09.590642 13605 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:20:09.607010  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 51184.7us
W20260709 12:20:09.607606  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 108916us
I20260709 12:20:09.523945 13975 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.056s
I20260709 12:20:09.559312 13958 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.080s
I20260709 12:20:09.581633 13929 raft_consensus.cc:697] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 23 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:09.610375 13953 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:09.585629 13712 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:09.642518  6550 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:09.604022 13554 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:09.604576 13947 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:09.650192  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:09.658423  6622 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:09.646759  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:09.653903  6475 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:09.529685 13955 consensus_queue.cc:260] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.646879  6548 leader_election.cc:403] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:09.655957 13771 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.608927 13964 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:09.659086 13740 raft_consensus.cc:697] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:09.668768 13771 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.669644 13740 consensus_queue.cc:237] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:09.670181  6475 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:09.671113 13771 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:09.666081 13860 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:09.673063 13822 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:20:09.668687 13941 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:09.675431  6475 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:09.672534 13797 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:09.647627  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:09.676383 13797 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:09.676798 13797 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:09.677228  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:09.677560 13747 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:09.678375  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:09.679458  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:09.584450 13710 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:09.660223  6650 raft_consensus.cc:2393] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 22: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:09.680603 13710 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:09.683133 13676 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:09.683655 13676 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:09.684062 13676 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:09.684432 13930 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:09.660988 13674 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.025s
I20260709 12:20:09.686319 13929 consensus_queue.cc:237] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.688530 13938 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 24. Current term is 25.
I20260709 12:20:09.688627 13941 raft_consensus.cc:697] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:09.689805 13941 consensus_queue.cc:237] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.690420 13741 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:09.667475  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 64095.9us
I20260709 12:20:09.692276  6654 raft_consensus.cc:2410] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 22 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:20:09.694440 13796 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:09.646463 13767 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 12:20:09.667477  6473 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:09.698282 13827 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:09.678123 13881 consensus_queue.cc:237] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.699447 13554 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:09.608351 13932 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:09.614145 13819 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:09.700366 13956 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.679672 13656 consensus_queue.cc:237] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.701997  6473 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:09.694478 13911 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:09.720193  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:09.703550  6475 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:09.703799 13710 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:09.707811  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:09.709070  6653 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:20:09.720216  6473 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:09.664398 13953 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:09.716007  6550 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:09.713821  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 55470.3us
I20260709 12:20:09.724213  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:09.725986  6577 raft_consensus.cc:2393] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 23: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:09.727336  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:09.728065  6576 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:20:09.730010  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:09.731097  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:09.732824  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:09.733791  7269 raft_consensus.cc:2368] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 19. Current term is 20.
I20260709 12:20:09.701990 13879 consensus_queue.cc:237] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:09.737054 13676 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.737398 13769 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.741670  6474 leader_election.cc:400] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 19. Current term is 20.
I20260709 12:20:09.742116  6474 leader_election.cc:403] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:09.726400  6622 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:09.748404  6509 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:20:09.750876  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:09.751643  6500 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:20:09.752622  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:09.753890  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:09.755890  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:09.740044  6550 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:09.754968 13888 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:09.769649  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:09.770023 13798 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:20:09.770342  7274 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:20:09.770661 13710 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:09.714883 13933 raft_consensus.cc:697] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:09.720744 13758 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:09.781468  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:09.773417  6502 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:09.722510 13955 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:09.785740  6651 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 18. Current term is 19. Ops: []
W20260709 12:20:09.777057  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 50708.3us
I20260709 12:20:09.778735  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:09.782516  6505 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:20:09.774860  6473 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:09.725049 13911 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:09.797821  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:09.790616  6505 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:09.794193  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:09.796376  6507 raft_consensus.cc:2393] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 19: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:09.787511  6474 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:09.801461  6576 raft_consensus.cc:2393] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 22: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:09.804476 13991 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:09.804867  6473 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:09.773905 13747 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:20:09.814661  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:09.815670  6505 raft_consensus.cc:1240] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:09.822618  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:09.816272  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:09.825613  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:09.828231  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:09.829154  6575 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 26.
I20260709 12:20:09.782351 13758 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:09.833737  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:09.834457  6584 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:20:09.786589 13767 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:09.836546 13767 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:09.836920 13767 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:09.837131 13933 consensus_queue.cc:237] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.773389 13621 consensus_queue.cc:1059] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 23.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:20:09.841327 13621 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:09.841727 13621 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:09.842065 13621 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:09.842573 13797 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.843986  6508 raft_consensus.cc:2393] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 24: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:09.844856 13676 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:09.845713 13769 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:09.846802 13554 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:09.819391  6577 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 12:20:09.777460 13947 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:09.849382 13947 raft_consensus.cc:697] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:09.850347 13947 consensus_queue.cc:237] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.783516 13882 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:20:09.852703 13882 raft_consensus.cc:697] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 26 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:09.853675 13882 consensus_queue.cc:237] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.833773 13758 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:09.820688 13747 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:09.857956 13747 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:09.797199 13911 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:09.860647 13911 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:09.805105 13991 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:20:09.723481 13908 consensus_queue.cc:1175] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "496c7c3792cd4587b4dbeba435f4be1e" responder_term: 22 status { last_received { term: 22 index: 13 } last_committed_idx: 11 last_received_current_leader { term: 22 index: 13 } }
I20260709 12:20:09.871837 13796 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.873888 13991 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:09.887496 13716 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.876725 13930 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:09.890468  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:09.890045 13953 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.892002  6507 raft_consensus.cc:2393] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 22: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:09.875272 13653 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:09.876065  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:09.907879  6577 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:09.878750  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:09.895115  6584 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22.
I20260709 12:20:09.898885  6550 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:09.821668 13879 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:09.913327  7273 raft_consensus.cc:1240] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 9. Current term is 10. Ops: []
W20260709 12:20:09.915469  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 51909.7us
I20260709 12:20:09.920414  6506 raft_consensus.cc:2368] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20.
I20260709 12:20:09.884158 13956 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:09.922263 13716 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:09.884155 13797 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:09.927172 13767 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.928323 13945 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:09.928791 13945 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:09.929170 13945 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:09.936010  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:09.943444  6577 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 20.
I20260709 12:20:09.924963 13621 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.949983  6658 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 23 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:09.924811  6584 raft_consensus.cc:1275] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 12:20:09.952832  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 148051us
I20260709 12:20:09.979110  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:09.908529 13796 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:09.946492 13668 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:09.960119  6549 leader_election.cc:400] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18.
I20260709 12:20:09.954474 13980 consensus_queue.cc:1059] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:20:09.962266  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:09.956075 13682 raft_consensus.cc:3055] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:20:09.987064  6549 leader_election.cc:403] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:20:09.956355  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 168943us
I20260709 12:20:10.001117 13767 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:10.001230 13621 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:10.002780 13953 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:10.004801 13758 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:10.010079 13796 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:09.988358 13798 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 19. Current term is 20.
I20260709 12:20:10.014801 13771 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:10.015429 13771 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:10.017719 13771 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:10.022162 13819 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:10.022634 13922 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:10.024492 13668 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:10.028291 13758 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:10.029917 13740 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:10.030514 13740 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:09.992064 13682 raft_consensus.cc:740] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:10.031716 13682 consensus_queue.cc:260] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:10.032709 13740 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:10.033236 13682 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:20:10.034549  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:10.035671  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:10.036953 13986 log.cc:927] Time spent T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:20:10.039804  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:10.048415  6507 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 22. Current term is 23. Ops: []
W20260709 12:20:10.049013  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 89025us
I20260709 12:20:10.050901  6503 raft_consensus.cc:2393] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 22: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:10.051656  6657 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:20:10.052464  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:10.052654  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:10.053932  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:10.054467  6506 raft_consensus.cc:2393] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 22: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:10.044098  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:10.042682  6652 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 23.
W20260709 12:20:10.068477  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:10.073416  7273 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:20:10.050006  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:10.097769  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:10.048098  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:10.113063  6580 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:20:10.114454  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:10.115166  6580 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:20:10.116041  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:10.117235  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:10.118479  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:10.119299  6580 raft_consensus.cc:2368] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 19. Current term is 20.
I20260709 12:20:10.122507  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:10.123351  6583 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:10.138085  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:10.139430  6506 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:20:10.141359  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:10.163230  6506 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:10.147202  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:10.149950  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:10.150518  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:10.152894  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 104868us
I20260709 12:20:10.154851  6549 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:10.158607  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:10.113741 13653 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:20:10.173166  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:10.168311  6584 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:20:10.114250 13554 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:10.145573  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:10.174182  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:10.178225  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:10.179049  6584 raft_consensus.cc:2393] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 22: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:10.168198  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:10.176463  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:10.182384  6580 raft_consensus.cc:2393] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 22: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:10.180215  6583 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 24.
I20260709 12:20:10.185418  6622 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:10.186539  6622 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:10.189121  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:10.190208  6583 raft_consensus.cc:2368] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 23. Current term is 24.
I20260709 12:20:10.162907 13933 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:10.183209  6549 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:10.166267 13819 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:10.167685 13668 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:10.195896  6622 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:10.194470  6506 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 23.
I20260709 12:20:10.198577  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:10.201579  7273 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:20:10.204566  6549 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:10.207940  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:10.156723 13991 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:10.211768  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:10.191813 13658 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:10.217533  6622 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:10.190774 13656 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:10.195758 13933 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:10.221272 13991 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:10.217831 13932 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:10.240511  6505 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:10.196941 13771 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:10.192931 13767 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:10.244457  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:10.202878 13660 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:10.200430 13710 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:10.181852 13692 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 12:20:10.193979 13758 consensus_queue.cc:1048] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 12:20:10.247273 13932 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:10.300657 13932 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:20:10.315960 13974 log.cc:927] Time spent T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:20:10.196209 13980 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 12:20:10.318300  6654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58980 (request call id 3566) took 1024 ms. Trace:
I20260709 12:20:10.318826  6654 rpcz_store.cc:276] 0709 12:20:09.293806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:09.300056 (+  6250us) service_pool.cc:224] Handling call
0709 12:20:10.318036 (+1017980us) spinlock_profiling.cc:243] Waited 391 ms on lock 0x7b54003787d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:10.318227 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":899044096}
I20260709 12:20:10.241122 13925 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:10.246934 13933 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:10.334354 13710 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:10.249764 13771 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:10.336621 13771 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:10.229665 13819 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:10.340974  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:10.210038 13929 consensus_queue.cc:1059] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.055s
W20260709 12:20:10.277195 14000 log.cc:927] Time spent T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 12:20:10.288609  6548 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:10.290549  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:10.344816 13796 raft_consensus.cc:3055] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:10.269703 13914 raft_consensus.cc:3055] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:10.321751 13767 raft_consensus.cc:697] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 19 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:10.345285  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:10.345757 13947 raft_consensus.cc:2764] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:20:10.346429  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:10.297960  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:10.347651  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:10.348481 13925 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:10.322552 13660 raft_consensus.cc:697] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:10.246809 13978 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:10.322381 13981 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:10.349694 13660 consensus_queue.cc:237] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:10.350736 13981 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:10.331054 13938 consensus_queue.cc:1059] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:10.352739 13938 raft_consensus.cc:3055] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:10.353065 13938 raft_consensus.cc:740] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:10.353762 13938 consensus_queue.cc:260] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:10.354811 13938 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:10.331087 13911 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:10.358212 13911 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:10.348207  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 130762us
I20260709 12:20:10.257771 13656 raft_consensus.cc:697] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:10.363340  6622 leader_election.cc:400] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 23. Current term is 24.
I20260709 12:20:10.363783  6622 leader_election.cc:403] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 12:20:10.274474  6509 raft_consensus.cc:1275] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:20:10.325757  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:10.249889 13747 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:10.365239  6650 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:10.367399 13747 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:10.368577  6504 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:10.346153 13767 consensus_queue.cc:237] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:10.368474  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:10.345398 13914 raft_consensus.cc:740] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:10.372833  6649 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:10.373736 13914 consensus_queue.cc:260] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:20:10.375450 13914 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:20:10.344974  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 56446.2us
I20260709 12:20:10.379743  6548 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:10.337389 13710 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:10.426062  6623 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:10.403256  6548 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:10.376400 13796 raft_consensus.cc:740] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:10.381381 13656 consensus_queue.cc:237] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:10.359205  6507 raft_consensus.cc:2368] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 19. Current term is 20.
I20260709 12:20:10.435571 13658 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:10.435061 13796 consensus_queue.cc:260] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:10.414512  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:10.436218 13658 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:10.437239 13947 consensus_queue.cc:1059] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:10.437500 13658 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:10.413129 13668 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:10.415299 13692 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:10.476991 13518 consensus_queue.cc:1059] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:10.478053 13518 raft_consensus.cc:3055] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:10.478367 13518 raft_consensus.cc:740] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:10.479229 13518 consensus_queue.cc:260] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:10.480695 13518 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:10.487078  6501 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 20.
I20260709 12:20:10.489820  6649 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 23.
I20260709 12:20:10.494374  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:10.495913  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:10.497507  6649 raft_consensus.cc:2393] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 22: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:10.504211  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:10.512593  6649 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22.
I20260709 12:20:10.503983 13904 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:10.519335  6650 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
W20260709 12:20:10.513492 13999 log.cc:927] Time spent T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.328s	user 0.006s	sys 0.000s
I20260709 12:20:10.526835 13981 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:10.529735  6550 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.3:36107 timed out after 1.530s (SENT) [suppressed 8 similar messages]
I20260709 12:20:10.509554  6475 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:10.553766  6475 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:10.556958  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 369493us
I20260709 12:20:10.564487  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:10.564901  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:10.561553  6475 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:10.572400  7273 raft_consensus.cc:1275] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 12:20:10.548681 13945 consensus_meta.cc:287] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.619s	user 0.007s	sys 0.008s
I20260709 12:20:10.572726  6474 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:10.561122 13798 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:10.537089 13771 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:10.575459  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 136180us
I20260709 12:20:10.577396  6548 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:10.575711  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:10.562700 13747 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:10.585973  6548 leader_election.cc:400] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20.
I20260709 12:20:10.558295 13554 raft_consensus.cc:697] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 20 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:10.540261 13904 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:10.571511 13880 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:10.574409 13908 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:10.583508 13798 raft_consensus.cc:697] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 23 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:10.588215 13941 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:10.587302 13747 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:10.628198  7267 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
W20260709 12:20:10.596966  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:10.575502 13945 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:10.609212 13980 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:10.610718 13653 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18.
I20260709 12:20:10.585801 13947 consensus_queue.cc:1059] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 12:20:10.588157 13922 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.046s
I20260709 12:20:10.644050  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:10.636569 13978 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:10.625860 13880 raft_consensus.cc:697] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:10.648555 13554 consensus_queue.cc:237] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:10.665093  6578 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:10.638005 13747 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:10.612108 13660 consensus_queue.cc:1059] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 12:20:10.653523 13879 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:10.695422  6578 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 23.
I20260709 12:20:10.672606 13978 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:10.688036 13879 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:10.712244  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 182536us
W20260709 12:20:10.706406 14013 log.cc:927] Time spent T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
W20260709 12:20:10.740157 14011 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 12:20:10.741343 13660 raft_consensus.cc:3055] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:10.758095 13880 consensus_queue.cc:237] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:10.761763 13658 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:10.723064 13798 consensus_queue.cc:237] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:10.817126  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 391214us
I20260709 12:20:10.826572  6652 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:10.830987  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:10.832131  6650 raft_consensus.cc:2393] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 23: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:10.850922  6652 raft_consensus.cc:1240] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:10.854187  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:10.855206  6550 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:10.884379  6577 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:10.773855 13660 raft_consensus.cc:740] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:10.768169 13946 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:10.893445 14016 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:10.896723  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:10.896832 14016 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:20:10.897459  6652 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 22.
I20260709 12:20:10.897636 13978 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:10.898851  7240 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:10.899340 14016 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:10.900502  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41304 (request call id 948) took 1023 ms. Trace:
I20260709 12:20:10.900906 14010 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:10.901486 14010 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:10.901935 14010 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:10.905483  6501 rpcz_store.cc:276] 0709 12:20:09.877585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:09.877742 (+   157us) service_pool.cc:224] Handling call
0709 12:20:10.900220 (+1022478us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b5400396cd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:10.900386 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1224007040}
I20260709 12:20:10.907265  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:10.907859  6501 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 23.
I20260709 12:20:10.908392 13660 consensus_queue.cc:260] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:10.909925 13658 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:10.910687 13658 raft_consensus.cc:2764] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:20:10.910562  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:10.911562 13658 consensus_queue.cc:1059] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:10.911541  6501 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:20:10.921060 14009 log.cc:927] Time spent T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:20:10.921371  6651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3565) took 1628 ms. Trace:
I20260709 12:20:10.922019  6651 rpcz_store.cc:276] 0709 12:20:09.293376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:09.293628 (+   252us) service_pool.cc:224] Handling call
0709 12:20:09.293921 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:09.786233 (+492312us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:10.921110 (+1134877us) spinlock_profiling.cc:243] Waited 492 ms on lock 0x7b5400373554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:10.921314 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1130499328}
I20260709 12:20:10.922827  6501 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:20:10.848938 13653 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:10.923620  6657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 1207) took 1204 ms. Trace:
W20260709 12:20:10.885792 14011 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:20:10.923899 13653 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:10.926487 13945 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:10.926556  6506 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:20:10.891727 13930 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:10.928969 13930 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:10.931684 13930 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:10.927098 13653 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:10.943475  6501 raft_consensus.cc:1275] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 12:20:10.899443 13933 consensus_meta.cc:287] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.565s	user 0.012s	sys 0.000s
I20260709 12:20:10.860613 13771 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:10.924100  6657 rpcz_store.cc:276] 0709 12:20:09.719174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.102191 (+383017us) service_pool.cc:224] Handling call
0709 12:20:10.102601 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:10.152889 (+ 50288us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:10.153035 (+   146us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:10.164149 (+ 11114us) log.cc:844] Serialized 26 byte log entry
0709 12:20:10.585024 (+420875us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:20:10.608473 (+ 23449us) log.cc:844] Serialized 14 byte log entry
0709 12:20:10.608807 (+   334us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:10.608919 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:10.899297 (+290378us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b180042c280. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:20:10.923261 (+ 23964us) raft_consensus.cc:1685] finished
0709 12:20:10.923361 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:10.923548 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48978,"prepare.queue_time_us":27635,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":49232,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":221,"spinlock_wait_cycles":29395840,"thread_start_us":374795,"threads_started":1,"wal-append.queue_time_us":531625}
I20260709 12:20:10.949420  6582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 1186) took 1264 ms. Trace:
I20260709 12:20:10.953035  6582 rpcz_store.cc:276] 0709 12:20:09.685207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:09.685372 (+   165us) service_pool.cc:224] Handling call
0709 12:20:09.685912 (+   540us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:09.701286 (+ 15374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:09.701317 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:09.701358 (+    41us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:20:09.701388 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:10.949175 (+1247787us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b540034dfd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:10.949220 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:10.949360 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34894720}
I20260709 12:20:10.888233 13692 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 23. Current term is 24.
I20260709 12:20:10.934999 13653 raft_consensus.cc:3055] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:10.909930 13660 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:10.939098 13904 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:10.960493 13754 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:10.963462 13653 raft_consensus.cc:740] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:10.948611 13946 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:11.070029  6582 raft_consensus.cc:3055] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:11.070580  6582 raft_consensus.cc:740] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:11.075038  6658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3581) took 1428 ms. Trace:
I20260709 12:20:11.079421  6583 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:20:11.082437  6501 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:20:10.951113 13933 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:11.075757  6658 rpcz_store.cc:276] 0709 12:20:09.646695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:09.649410 (+  2715us) service_pool.cc:224] Handling call
0709 12:20:09.649773 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:09.979694 (+329921us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:11.074748 (+1095054us) spinlock_profiling.cc:243] Waited 300 ms on lock 0x7b54003e99d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:11.074977 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":689949568}
I20260709 12:20:11.071738  6582 consensus_queue.cc:260] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:11.091509  6582 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:11.094646  6548 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:11.081379 13994 log.cc:927] Time spent T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 12:20:11.068261 13947 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.067891 13692 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:11.134367  6582 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 14. (term mismatch)
W20260709 12:20:11.141816  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 257137us
I20260709 12:20:10.979526 13754 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:11.083909 13771 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.154472  6582 pending_rounds.cc:85] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 13
I20260709 12:20:11.098840 13930 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:20:11.170473  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:11.120119 13692 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:11.120623 13947 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:11.133343 13653 consensus_queue.cc:260] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:11.162698 13771 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:11.198004 13754 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:11.201320 13771 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:11.203117 13796 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:11.204638 13653 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:11.205461 13692 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:11.206103 14014 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:20:11.220294 14001 log.cc:927] Time spent T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.740s	user 0.000s	sys 0.009s
W20260709 12:20:11.193007 14021 log.cc:927] Time spent T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:20:11.169536 13930 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:11.193766 13932 consensus_meta.cc:287] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.893s	user 0.010s	sys 0.003s
I20260709 12:20:11.233615 14014 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:20:11.231503 14023 log.cc:927] Time spent T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
W20260709 12:20:11.254871 13819 consensus_meta.cc:287] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.916s	user 0.013s	sys 0.000s
I20260709 12:20:11.249027 14014 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:11.272707 13933 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:11.277684 14028 log.cc:927] Time spent T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
W20260709 12:20:11.278081 14027 log.cc:927] Time spent T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 12:20:11.244314 13932 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:11.203192 13946 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:11.273857 13819 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:11.320989 14033 log.cc:927] Time spent T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
W20260709 12:20:11.323616 13993 log.cc:927] Time spent T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 1.096s	user 0.015s	sys 0.000s
I20260709 12:20:11.316666 13947 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:11.329473 14010 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:11.331283 14010 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:11.334632 14014 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:11.368202  6506 raft_consensus.cc:1240] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:11.372465  6500 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 3095) took 1341 ms. Trace:
I20260709 12:20:11.372954  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:11.373107  6500 rpcz_store.cc:276] 0709 12:20:10.031242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.226920 (+195678us) service_pool.cc:224] Handling call
0709 12:20:10.227419 (+   499us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:10.227995 (+   576us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:10.228042 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:10.229041 (+   999us) log.cc:844] Serialized 26 byte log entry
0709 12:20:10.670882 (+441841us) spinlock_profiling.cc:243] Waited 9.81 ms on lock 0x7b1800345e20. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:20:10.962628 (+291746us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:20:10.962685 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:10.962775 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:11.371740 (+408965us) raft_consensus.cc:1685] finished
0709 12:20:11.371790 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:11.372228 (+   438us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":116,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":10465,"spinlock_wait_cycles":22552320,"thread_start_us":291547,"threads_started":1,"wal-append.queue_time_us":942101}
I20260709 12:20:11.375433  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:11.377866  6501 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 12:20:11.161605  6582 pending_rounds.cc:107] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 22.14
W20260709 12:20:11.381515  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:11.381603  6582 raft_consensus.cc:2889] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:11.385967  6583 raft_consensus.cc:1275] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:20:11.387691  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:11.388844  6500 raft_consensus.cc:2368] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 22. Current term is 23.
I20260709 12:20:11.412406  6653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3584) took 1764 ms. Trace:
I20260709 12:20:11.413611  6656 raft_consensus.cc:2410] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 22 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 12:20:11.412953  6653 rpcz_store.cc:276] 0709 12:20:09.647525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:09.724949 (+ 77424us) service_pool.cc:224] Handling call
0709 12:20:09.725200 (+   251us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:09.726079 (+   879us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:09.726116 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:09.744863 (+ 18747us) log.cc:844] Serialized 26 byte log entry
0709 12:20:09.949112 (+204249us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:09.949151 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:09.949216 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:11.322955 (+1373739us) spinlock_profiling.cc:243] Waited 6.95 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:20:11.411747 (+ 88792us) raft_consensus.cc:1685] finished
0709 12:20:11.411789 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:11.412191 (+   402us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5534,"raft.queue_time_us":12874,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":28448,"spinlock_wait_cycles":15988992,"thread_start_us":126953,"threads_started":1,"wal-append.queue_time_us":371843}
I20260709 12:20:11.429642  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:11.446009 13554 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.446810 13656 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:11.446959 13554 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:11.447902 13656 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:11.448846 13605 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.450026 13605 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:11.451942 13656 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:11.455482 13554 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:11.456908 13668 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:11.462440 13668 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:11.457957 13692 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:11.464090 13779 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.465094 13692 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:11.465200 13779 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:11.458844 13710 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 23 current term: 23 is higher.
I20260709 12:20:11.459877 13682 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.469615 13779 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:11.469448 13682 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:11.460122  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:11.475533 13682 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:11.457095 13605 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:11.460966  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:11.480180 13692 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:11.479503 13668 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:11.482375 13668 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:20:11.483282 13747 consensus_meta.cc:287] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.794s	user 0.015s	sys 0.008s
I20260709 12:20:11.482878  6653 raft_consensus.cc:2468] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
W20260709 12:20:11.317654 13879 consensus_meta.cc:287] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.574s	user 0.006s	sys 0.006s
I20260709 12:20:11.483673 13668 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:11.323930 13932 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:11.491622 13930 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:11.495594 13938 consensus_meta.cc:287] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 1.140s	user 0.018s	sys 0.000s
I20260709 12:20:11.497062 13946 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.497784 13947 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:11.525101  6658 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 22.
I20260709 12:20:11.524224 13747 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:11.568089 14029 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:11.569514 14014 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:11.570062 14029 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:11.576742 13668 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:11.580780 13767 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.136s	user 0.021s	sys 0.016s
W20260709 12:20:11.581318 13767 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.162s	user 0.021s	sys 0.016s
I20260709 12:20:11.581439 13946 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:11.588877 13946 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:11.589828  7240 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:11.591259  7240 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:11.592104  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:11.592552 13554 raft_consensus.cc:2804] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:20:11.593042 13554 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:11.593191  6509 raft_consensus.cc:2393] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 23: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:11.593502 13554 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:20:11.595250 13605 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:11.595952 13605 raft_consensus.cc:697] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 22 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:11.597343 13605 consensus_queue.cc:237] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:11.600487 13879 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:11.608443  6501 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:11.609067  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 3111) took 1242 ms. Trace:
I20260709 12:20:11.613879  6509 rpcz_store.cc:276] 0709 12:20:10.364447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.365060 (+   613us) service_pool.cc:224] Handling call
0709 12:20:11.591638 (+1226578us) spinlock_profiling.cc:243] Waited 34.4 ms on lock 0x7b5800002a98. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:20:11.606886 (+ 15248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79116672}
W20260709 12:20:11.617959 13660 consensus_meta.cc:287] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.650s	user 0.014s	sys 0.009s
I20260709 12:20:11.618356  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 2991) took 1318 ms. Trace:
I20260709 12:20:11.618813  6656 rpcz_store.cc:276] 0709 12:20:10.299567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.346222 (+ 46655us) service_pool.cc:224] Handling call
0709 12:20:11.618081 (+1271859us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54000758d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:11.618280 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2450913664}
I20260709 12:20:11.619701  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:11.620620  7269 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:11.625888  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:11.628047  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:11.581039 13947 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:11.636983  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:11.638072  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:11.618825 13879 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:11.644580 13809 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.645573 13809 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:11.609127  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 392917us
I20260709 12:20:11.650761 13809 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:11.658146 13798 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:11.658886 13798 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:11.654814  7269 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
W20260709 12:20:11.672144 13925 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.153s	user 0.006s	sys 0.007s
W20260709 12:20:11.672796 13925 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.155s	user 0.006s	sys 0.008s
W20260709 12:20:11.680989  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:11.690910  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:11.682651  6506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 3217) took 1086 ms. Trace:
I20260709 12:20:11.694556  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:11.698325  6656 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:11.689105  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:11.685051  6548 leader_election.cc:400] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 22. Current term is 23.
I20260709 12:20:11.700126  6548 leader_election.cc:403] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:20:11.686784  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:11.695310  6506 rpcz_store.cc:276] 0709 12:20:10.596734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.928036 (+331302us) service_pool.cc:224] Handling call
0709 12:20:10.928373 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:11.370013 (+441640us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:11.682316 (+312303us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b54001e5fd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:11.682585 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1009207680}
I20260709 12:20:11.681494  6508 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
I20260709 12:20:11.715067 14029 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:11.682178 13947 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:11.685308 13741 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:11.702255 13747 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:11.717058 13741 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:11.718016 13747 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.718547 13747 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:11.719442 13741 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:11.681067 14015 raft_consensus.cc:3055] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:11.720160 14015 raft_consensus.cc:740] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:11.676064  6576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 3090) took 1624 ms. Trace:
I20260709 12:20:11.723263  6509 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:20:11.728755  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:11.729995  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:11.770067  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:11.770254  6657 raft_consensus.cc:2368] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 22. Current term is 23.
I20260709 12:20:11.733645  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:11.771517  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:11.772480  6652 raft_consensus.cc:2410] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 22 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:20:11.735174  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:11.745380  6507 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 3101) took 1683 ms (client timeout 2112 ms). Trace:
I20260709 12:20:11.731422  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:11.776769  6507 rpcz_store.cc:269] 0709 12:20:10.062264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.097482 (+ 35218us) service_pool.cc:224] Handling call
0709 12:20:11.744812 (+1647330us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b5400203d54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:11.745151 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":598508416}
I20260709 12:20:11.747277 13798 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:11.747567  6578 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 2927) took 1707 ms (client timeout 2181 ms). Trace:
I20260709 12:20:11.755088 14015 consensus_queue.cc:260] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:11.745657  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:11.779371  6503 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:20:11.760782 14029 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:11.780462 14015 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:11.780704 14029 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:11.759250  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:11.783964 14029 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:11.764611 13932 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.785985 13932 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:11.765264 13653 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:11.788772 13653 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:11.791491 13653 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:11.761418  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:11.792515  6654 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:20:11.768998  6576 rpcz_store.cc:276] 0709 12:20:10.051636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.051741 (+   105us) service_pool.cc:224] Handling call
0709 12:20:10.052004 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:10.052206 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:20:10.155628 (+103422us) log.cc:844] Serialized 14 byte log entry
0709 12:20:11.110507 (+954879us) spinlock_profiling.cc:243] Waited 1.51 ms on lock 0x7b1800333100. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a2923a 00007f2875e93a2a
0709 12:20:11.671465 (+560958us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:11.671626 (+   161us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:11.675336 (+  3710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:11.675509 (+   173us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:11.675778 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45031,"prepare.queue_time_us":103273,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":4986,"raft.queue_time_us":48597,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":20623,"spinlock_wait_cycles":3475840,"thread_start_us":461537,"threads_started":1,"wal-append.queue_time_us":1533492}
I20260709 12:20:11.763904 13668 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:11.794116 13668 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:11.764473 13938 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:11.796806 13938 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:11.778618  6578 rpcz_store.cc:269] 0709 12:20:10.039911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.043848 (+  3937us) service_pool.cc:224] Handling call
0709 12:20:11.747062 (+1703214us) spinlock_profiling.cc:243] Waited 609 ms on lock 0x7b540034cbd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:11.747268 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1401657600}
I20260709 12:20:11.802420 13932 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:11.771544  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:11.803822  6657 raft_consensus.cc:2393] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 23: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:11.772475  6649 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 25.
I20260709 12:20:11.809222 13938 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:11.766335  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:11.815591  6500 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 23.
I20260709 12:20:11.814999  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:11.818005  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:11.823653  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 138883us
I20260709 12:20:11.814715 13660 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:11.824790 13798 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:11.844513 13798 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:11.845290  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 2990) took 1546 ms. Trace:
I20260709 12:20:11.845743  6655 rpcz_store.cc:276] 0709 12:20:10.299214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.347445 (+ 48231us) service_pool.cc:224] Handling call
0709 12:20:11.844950 (+1497505us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b540010ed54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:11.845121 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2446070016}
I20260709 12:20:11.846810 13798 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:11.827262 13653 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 22. Current term is 23.
I20260709 12:20:11.827718 14032 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.848734 14032 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:11.850965 14032 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:11.854216  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:11.854962  7270 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:11.855777 13904 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:11.856942  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:11.857221 13653 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:11.857937 13653 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:11.828507 13880 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:11.828177 13554 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:11.830928 13809 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:11.831284  7273 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:20:11.832096  7271 raft_consensus.cc:1240] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:11.835616  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:11.863315  6656 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 23.
I20260709 12:20:11.864687 13653 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:11.872155  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:11.874855  6658 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 23.
I20260709 12:20:11.877439  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:11.860319 13904 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:11.880527  7267 raft_consensus.cc:2393] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 11: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:20:11.832466 14026 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:11.880612 13809 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:11.881433 14026 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:11.882423 13904 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:11.883167 13809 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:11.883926 13880 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:11.886214  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:11.888820 13880 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:11.888878  6652 raft_consensus.cc:2468] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 10.
I20260709 12:20:11.889680  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:11.890357  6657 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:11.891273  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:11.892025 13660 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:11.894488 13660 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:11.895159  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:11.896164  6653 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:20:11.901103  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:11.902120 13554 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:11.902621 13904 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:11.902799  6505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 3110) took 1538 ms. Trace:
I20260709 12:20:11.903355  6475 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:11.904181 13660 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:11.905211 13809 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:11.905957 13809 raft_consensus.cc:697] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 23 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:11.903258  6505 rpcz_store.cc:276] 0709 12:20:10.364103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.391255 (+ 27152us) service_pool.cc:224] Handling call
0709 12:20:11.684028 (+1292773us) spinlock_profiling.cc:243] Waited 7.95 ms on lock 0x7b5800002a98. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:20:11.684427 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:11.685095 (+   668us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:11.685123 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:11.685727 (+   604us) log.cc:844] Serialized 26 byte log entry
0709 12:20:11.817279 (+131552us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:11.817330 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:11.818303 (+   973us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:11.902318 (+ 84015us) raft_consensus.cc:1685] finished
0709 12:20:11.902338 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:11.902632 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":34621,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":147,"spinlock_wait_cycles":18274688,"thread_start_us":131131,"threads_started":1,"wal-append.queue_time_us":140667}
I20260709 12:20:11.903227  6550 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:11.906243  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:11.908645  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:11.903219 13904 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:11.909854  6550 leader_election.cc:400] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 22. Current term is 23.
I20260709 12:20:11.887156 13653 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:11.911682 13904 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:11.911309 13653 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:11.907189 13809 consensus_queue.cc:237] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:11.909855  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:11.913049  6507 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:20:11.913892  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:11.913942 13653 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:11.904999 13660 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:11.915041  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:11.915648  6507 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:11.917106  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:11.918586  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:11.919116 13747 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:11.920152 13660 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:11.909715  6501 raft_consensus.cc:2393] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 23: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:11.922514  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:11.925146  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:11.926226  6507 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:20:11.914708 13653 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:11.883839 14026 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:11.930176  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:11.930987  6658 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:20:11.930879  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:11.933449  7270 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:20:11.932773  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:11.939049  6504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 3201) took 2192 ms. Trace:
I20260709 12:20:11.939450  6504 rpcz_store.cc:276] 0709 12:20:09.746760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:09.773075 (+ 26315us) service_pool.cc:224] Handling call
0709 12:20:09.779693 (+  6618us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:10.369257 (+589564us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:11.938528 (+1569271us) spinlock_profiling.cc:243] Waited 589 ms on lock 0x7b540017c0d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:11.938867 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1353663744}
I20260709 12:20:11.940656 13668 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:11.946950  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 54986.8us
I20260709 12:20:11.948084 13880 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:11.949389 14047 consensus_queue.cc:1059] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:11.950430 13879 raft_consensus.cc:3055] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:11.950973 13879 raft_consensus.cc:740] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:11.952219 13879 consensus_queue.cc:260] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:11.953101 13945 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:11.953588 13879 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:11.953991 13945 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:11.963807  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:11.968924  6502 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 949) took 1818 ms. Trace:
I20260709 12:20:11.969452  6502 rpcz_store.cc:276] 0709 12:20:10.150071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:10.292995 (+142924us) service_pool.cc:224] Handling call
0709 12:20:11.967808 (+1674813us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b5800002a98. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:20:11.968183 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:11.968510 (+   327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:11.968536 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:11.968574 (+    38us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:11.968612 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:11.968663 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:11.968754 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":244647296}
I20260709 12:20:11.971292  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:11.993110 13747 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:11.994195 13747 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:11.991189  6404 catalog_manager.cc:5697] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:11.997681 13747 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:11.972110  6509 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:12.019313 13737 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:12.003206 14047 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:12.030375 13737 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:12.009980  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:12.033819  6504 raft_consensus.cc:2393] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 18: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:12.015201  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:12.009248  6652 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 26. Current term is 27. Ops: []
I20260709 12:20:12.021166  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:12.007085  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:12.026957  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:12.054960  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 121954us
I20260709 12:20:12.057905  6550 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:12.059573 13668 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:12.047611  6655 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22.
I20260709 12:20:12.061740  6550 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:12.062808 13653 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:12.063302 13653 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:12.063189  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:12.065079  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:12.065742  6576 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:20:12.065843  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:12.067231  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:12.067538  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:12.007309  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:12.068676 13653 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:12.069418  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:12.070309  6579 raft_consensus.cc:2410] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 24 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:20:12.071245  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:12.071900  6578 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:12.073220  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:12.073866  6576 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 23.
I20260709 12:20:12.039978 13737 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:12.076490  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:12.038805  7274 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:20:12.079380  6658 raft_consensus.cc:2468] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 11.
I20260709 12:20:12.078054  6550 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:12.038893 14047 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:12.086987 13945 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:12.087538  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:12.087877  6579 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22.
I20260709 12:20:12.091655 14037 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:12.092170 14037 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:12.092545 14037 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:12.095196  6651 raft_consensus.cc:1240] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:12.095206  7240 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:12.096531 13746 raft_consensus.cc:2804] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:20:12.097107 13746 raft_consensus.cc:697] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 11 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:12.097999 13746 consensus_queue.cc:237] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:12.098609  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:12.099367  6504 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 24.
I20260709 12:20:12.101635  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:12.102712  6504 raft_consensus.cc:2410] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 24 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:20:12.104516  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:12.023447  6473 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:12.115641  6550 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:12.127147  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:12.128192  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:12.135347  6579 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:20:12.131484  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:12.133496  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:12.137290  6550 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:12.138271  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:12.139406  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:12.136353  6504 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:20:12.142855  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:12.143862  6507 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 18.
I20260709 12:20:12.148815  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:12.149542  6579 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:12.156543 13958 consensus_queue.cc:1059] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 12:20:12.157996 13958 raft_consensus.cc:3055] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:12.158360 13958 raft_consensus.cc:740] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:12.170213  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:12.166712  6550 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:12.172006  6550 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:12.176988  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:12.178807  6579 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 27.
I20260709 12:20:12.180435 13809 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:20:12.180719  6652 raft_consensus.cc:1275] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:20:12.184202  6654 raft_consensus.cc:1275] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:20:12.190579 14053 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:12.191202 14053 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:12.191639 14053 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:12.192092  6583 raft_consensus.cc:1275] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:20:12.193169  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:12.194360 14030 raft_consensus.cc:2804] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:20:12.194805 14030 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:12.195155 14030 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:20:12.176385 13668 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:12.210086  6578 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:20:12.210309  6474 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:12.209451 13668 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:12.213744 13668 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:12.215503  6474 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:12.216358 13879 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:12.219740  6652 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:20:12.221885  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:12.172763 13653 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:12.216878 13879 raft_consensus.cc:697] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:12.227303  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:12.234247 13879 consensus_queue.cc:237] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:12.219571 13798 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:12.236681  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:12.239197 13710 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:12.239909 13710 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:12.244485  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:12.159176 13958 consensus_queue.cc:260] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:12.245198  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:12.249411  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:12.251320  6649 raft_consensus.cc:1275] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
W20260709 12:20:12.254798  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:12.256510  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:12.254217 13737 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:12.263243  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 240022us
I20260709 12:20:12.251462 13798 raft_consensus.cc:697] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:12.260069 13958 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:12.262143 13741 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:12.273459 13824 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:20:12.279407  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:12.274165 13824 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:12.280388 13946 consensus_queue.cc:1048] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:12.268201 14014 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:12.266314 14031 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:12.267630 13710 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:12.274565 13945 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:12.285707  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:12.336119  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:12.318342  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:12.344724  6578 raft_consensus.cc:2393] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 19: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:12.279170 13741 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:12.274886 13737 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:12.280104 14047 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:12.371161  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:12.307507  6583 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:20:12.352500 14053 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:12.383888 14025 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:12.389379 14025 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:12.390797 14053 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:12.379444 13941 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:12.391469 14025 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:12.391806 13941 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:12.393499  6578 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:12.394781 13941 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:12.398813 13798 consensus_queue.cc:237] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:12.400512 13945 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:12.401350 13945 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:12.318428  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:20:12.385166  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 45522.1us
I20260709 12:20:12.374432 14026 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:20:12.388247 14057 log.cc:927] Time spent T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:20:12.289608 14014 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:12.421103  6577 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:12.437866  6578 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:20:12.444906  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:12.384322 13653 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:12.419548 13824 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:12.319043 14031 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:12.452142  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 164119us
W20260709 12:20:12.451058  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 243439us
I20260709 12:20:12.460251  6548 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:12.461865  6582 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 26. Current term is 27. Ops: []
I20260709 12:20:12.424685 14026 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:12.461987  6475 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:12.467631  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:12.453805 13824 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:12.475208  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:12.476019 14047 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:20:12.476437  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 174789us
I20260709 12:20:12.477221  6622 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:12.474907  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:12.479725  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:20:12.491142  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:12.450055 14030 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:12.492372  6579 raft_consensus.cc:2468] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 10.
I20260709 12:20:12.494351 14030 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:12.497841  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:12.498394 14063 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:12.498605  6657 raft_consensus.cc:2368] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11.
I20260709 12:20:12.499050 14063 raft_consensus.cc:697] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 23 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:20:12.469581 14051 log.cc:927] Time spent T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.228s	user 0.003s	sys 0.000s
I20260709 12:20:12.501731  6473 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:12.502564  6582 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:20:12.468146 13653 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:20:12.507222  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 47064.1us
I20260709 12:20:12.501062 14047 consensus_queue.cc:1048] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 12:20:12.505154 13824 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:12.510681  6550 leader_election.cc:400] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11.
I20260709 12:20:12.510972  6550 leader_election.cc:403] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
W20260709 12:20:12.512836  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 49643.7us
I20260709 12:20:12.514389 13824 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:12.517376 13668 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:12.519017 14060 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:12.520479  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:12.521234  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:12.507097 13653 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:12.524791  7238 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:12.495108 14030 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:12.527443 14050 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:20:12.558346  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 91651.7us
I20260709 12:20:12.529018  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:12.466753 13809 consensus_queue.cc:1059] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 12:20:12.505873 13737 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:12.532675  7267 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 15. (term mismatch)
I20260709 12:20:12.561498  7267 pending_rounds.cc:85] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9: Aborting all ops after (but not including) 14
I20260709 12:20:12.543936  6548 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:12.545198  6584 raft_consensus.cc:2468] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 10.
W20260709 12:20:12.561864  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 83832.2us
I20260709 12:20:12.559502  6580 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 21.
I20260709 12:20:12.564045  6583 raft_consensus.cc:2368] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 22. Current term is 23.
I20260709 12:20:12.561782  7267 pending_rounds.cc:107] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9: Aborting uncommitted NO_OP operation due to leader change: 22.15
I20260709 12:20:12.569993  7267 raft_consensus.cc:2889] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:12.573096 14015 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:12.573854 14015 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:12.574373 14015 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:12.574791 14063 consensus_queue.cc:237] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:12.576824 14030 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:12.579138 13901 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:12.579422 14030 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:12.579638 13901 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:12.543442 13710 consensus_queue.cc:1059] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.037s
I20260709 12:20:12.580822 13710 raft_consensus.cc:3055] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:12.581202 13710 raft_consensus.cc:740] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:12.580111 13901 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:12.556694 13653 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:12.606469  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:12.563654 13946 raft_consensus.cc:3060] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:20:12.607173  6652 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:20:12.555269 14047 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:20:12.585726  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:12.561738 14050 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:12.608440 14050 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:12.609349  6550 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:12.610321  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:12.526446 13945 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:12.602180  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:12.605443 14031 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:12.613798  6580 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 18.
W20260709 12:20:12.623777  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 79929.5us
I20260709 12:20:12.611500 13809 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:12.626209 13809 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:12.627178 14066 consensus_queue.cc:1048] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:12.631215  6548 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:12.582149 13710 consensus_queue.cc:260] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.17, Last appended by leader: 17, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:12.631294  6473 leader_election.cc:400] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 19. Current term is 20.
I20260709 12:20:12.653560  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:12.631670 13809 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:12.660742  6548 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:12.661751  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:12.662408 14031 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:12.664951  6575 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:20:12.665112 13741 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:12.672883  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:12.680614  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:12.681347  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:12.683799  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:12.685622 13668 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:12.687812 13747 raft_consensus.cc:3055] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:20:12.690815 14015 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:12.695438 13945 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:12.697867 14037 consensus_meta.cc:287] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.605s	user 0.014s	sys 0.005s
I20260709 12:20:12.698359  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:12.698287 14037 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:12.629336 14030 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:12.701112 14030 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:12.701480 14030 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:12.702442  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:12.703604  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:12.688248 13747 raft_consensus.cc:740] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:12.706156 13747 consensus_queue.cc:260] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:12.707433 13747 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:20:12.710275  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 121155us
I20260709 12:20:12.710590  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:12.619691 14026 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:12.720664  6549 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:12.718474  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:12.728226  6657 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:12.719496  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:12.720304  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:12.729959  6654 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:12.722191 13941 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:20:12.723995  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 215318us
I20260709 12:20:12.717381  6577 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
W20260709 12:20:12.745153  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 84480.2us
I20260709 12:20:12.746558  6548 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:12.700217 14037 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:12.757880  6548 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:12.752398  6549 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:12.725648 14031 raft_consensus.cc:697] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:20:12.769095  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:12.800199 14014 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:12.802929 14025 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:12.803194  6503 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:20:12.803488 14025 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:12.804705  6507 raft_consensus.cc:1275] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:20:12.805685 14025 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:12.765444 14026 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:20:12.806869  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:12.807209  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 54916.4us
I20260709 12:20:12.808163 14014 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:12.808720 14014 raft_consensus.cc:697] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:12.809913  7265 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:12.809644 14014 consensus_queue.cc:237] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:12.815061 14071 log.cc:927] Time spent T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.100s	user 0.001s	sys 0.002s
I20260709 12:20:12.814968  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:12.818223  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:12.822187  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:12.823297  6656 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 23.
I20260709 12:20:12.824558  6550 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:12.825484 14026 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:12.825850 14026 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:12.826177 14026 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:12.753854 13668 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:12.765760 13947 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:20:12.850368  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 240445us
I20260709 12:20:12.829288 13710 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:12.851861  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 139523us
I20260709 12:20:12.766259 13941 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:12.861768 14026 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:12.777173  6548 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:12.761234 14047 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.089s
W20260709 12:20:12.865782  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 118363us
I20260709 12:20:12.879932 14031 consensus_queue.cc:237] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:12.881254  6473 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:12.882170 13941 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:12.882782  6473 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:12.894928  6473 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:12.897681 13901 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:12.872797  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:12.760293 13945 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:12.851939 13947 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:12.899537 13947 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:12.900084  7266 raft_consensus.cc:2368] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 18. Current term is 19.
I20260709 12:20:12.900822 14015 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:12.905222 13901 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:12.918181 14060 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:12.918979 14060 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:12.919605 14060 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:12.923056 13947 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:12.908736  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:12.916512 13668 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:12.877481  6657 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 23.
I20260709 12:20:12.926003 14026 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:12.930667  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:20:12.878461  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 101223us
I20260709 12:20:12.943606 13668 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:12.943939  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:12.944172 13668 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:12.944974  6653 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:12.946995 13653 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:12.952669  6649 raft_consensus.cc:1275] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:20:12.920114 14060 raft_consensus.cc:697] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 24 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:12.761075 13809 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.179s
I20260709 12:20:12.969162  6550 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:12.944041  6398 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57914 (request call id 193) took 41596 ms (client timeout 15000 ms). Trace:
I20260709 12:20:12.984277 14050 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:12.992492  6654 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 26.
I20260709 12:20:12.995224 14050 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:12.996973  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 124243us
I20260709 12:20:12.998047  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:12.999307 13901 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:13.000236 13901 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:13.001024 13901 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:12.986603 13883 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:13.015581 13746 consensus_queue.cc:1048] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:20:12.988891 14060 consensus_queue.cc:237] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:13.018110  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:20:13.021507  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 66763.2us
I20260709 12:20:13.027918 13945 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:13.028980 13945 raft_consensus.cc:697] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:13.030675 13945 consensus_queue.cc:237] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:13.037536  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 229643us
I20260709 12:20:12.989928  6622 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:13.038508  6549 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:13.039261 14065 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:13.039460 13941 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:13.040017 14065 raft_consensus.cc:697] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:13.041429 14065 consensus_queue.cc:237] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:13.040093  6549 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:13.045112  6550 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:13.045914  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 173601us
W20260709 12:20:13.047005  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 46736.1us
I20260709 12:20:13.048813  7238 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:13.051378 13941 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:13.051899 14053 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:20:13.052480 14053 raft_consensus.cc:697] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:13.052747  6508 raft_consensus.cc:1240] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 23 LEADER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:13.052851 13946 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 10. Current term is 11.
I20260709 12:20:13.053568 13947 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:13.053391 14053 consensus_queue.cc:237] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:12.990034 13653 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:13.058724  6652 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 23.
I20260709 12:20:13.058799  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:13.059916 14076 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:13.060428 14076 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:13.060829 14076 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:13.061025  6550 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:13.063088  6580 raft_consensus.cc:2468] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 11.
I20260709 12:20:13.064640 13941 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:13.065748  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:13.049127  6549 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:12.989673  6548 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:13.088395  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 98780.5us
W20260709 12:20:13.090884  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 193766us
I20260709 12:20:13.104609  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:13.018980 14030 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:13.089861 13904 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:13.113123  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:13.090999 13653 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:20:12.986928  6398 rpcz_store.cc:269] 0709 12:19:31.347717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:31.364704 (+ 16987us) service_pool.cc:224] Handling call
0709 12:20:12.943773 (+41579069us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:08.081231 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:08.217367 (+136136us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:08.233203 (+ 15836us) write_op.cc:435] Acquired schema lock
0709 12:20:08.239003 (+  5800us) tablet.cc:662] Decoding operations
0709 12:20:08.253017 (+ 14014us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:08.253120 (+   103us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:08.253148 (+    28us) tablet.cc:685] Acquiring locks for 11 operations
0709 12:20:08.254123 (+   975us) tablet.cc:701] Row locks acquired
0709 12:20:08.254176 (+    53us) write_op.cc:260] PREPARE: finished
0709 12:20:08.254364 (+   188us) write_op.cc:270] Start()
0709 12:20:08.254518 (+   154us) write_op.cc:276] Timestamp: P: 1783599608254292 usec, L: 0
0709 12:20:08.254540 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:20:08.255272 (+   732us) log.cc:844] Serialized 4303 byte log entry
0709 12:20:08.542971 (+287699us) op_driver.cc:464] REPLICATION: finished
0709 12:20:08.896153 (+353182us) write_op.cc:301] APPLY: starting
0709 12:20:08.922562 (+ 26409us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:09.112497 (+189935us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:09.112601 (+   104us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:09.139319 (+ 26718us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:09.140108 (+   789us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=0,mrs_lookups=0
0709 12:20:09.140158 (+    50us) write_op.cc:312] APPLY: finished
0709 12:20:09.140631 (+   473us) log.cc:844] Serialized 105 byte log entry
0709 12:20:09.140834 (+   203us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:09.140989 (+   155us) write_op.cc:454] Released schema lock
0709 12:20:09.141046 (+    57us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:09.169485 (+ 28439us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:09.170281 (+   796us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2258236,"reported_tablets":52,"spinlock_wait_cycles":1408,"child_traces":[["op",{"apply.queue_time_us":333082,"apply.run_cpu_time_us":17796,"apply.run_wall_time_us":253059,"cfile_cache_hit":46,"cfile_cache_hit_bytes":50019,"num_ops":11,"prepare.queue_time_us":82290,"prepare.run_cpu_time_us":12641,"prepare.run_wall_time_us":285279,"raft.queue_time_us":22450,"raft.run_cpu_time_us":10846,"raft.run_wall_time_us":299797,"replication_time_us":275197,"thread_start_us":272370,"threads_started":4,"wal-append.queue_time_us":174509,"wal-append.run_cpu_time_us":13327,"wal-append.run_wall_time_us":1720896}]]}
I20260709 12:20:13.120915  6473 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:13.090309 13941 raft_consensus.cc:697] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:13.131141  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:13.098152 13947 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:13.107931 14015 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.091941 13958 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.041s
I20260709 12:20:13.157224  7266 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:20:13.160506  6652 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 25. Current term is 26. Ops: [25.13-25.13]
I20260709 12:20:13.163612  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:13.164917  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:13.166317  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:13.167706  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:13.168943  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:13.095599 14037 consensus_queue.cc:1048] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:20:13.182528  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:20:13.189837  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 69006.5us
I20260709 12:20:13.190625  6549 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:13.193881  6654 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:20:13.206602  6549 leader_election.cc:400] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 18. Current term is 19.
I20260709 12:20:13.207185  6549 leader_election.cc:403] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:13.213212  6473 leader_election.cc:400] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 22. Current term is 23.
I20260709 12:20:13.213675  6473 leader_election.cc:403] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:20:13.217754  6653 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 24.
I20260709 12:20:13.218724  6658 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:20:13.131232 14050 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.057s
I20260709 12:20:13.233042  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:13.233707  6580 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:13.221495 13653 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:13.181627 13824 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 12:20:13.178875 14015 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:13.270269 14060 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:13.271528 14015 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:13.275223  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:13.209419 14076 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:13.281085 14076 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:13.241163  6577 raft_consensus.cc:1275] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:20:13.256772  6652 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22.
I20260709 12:20:13.257473  6575 raft_consensus.cc:1240] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:13.258567 14080 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:13.259016 14070 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:20:13.261816  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 71114.6us
I20260709 12:20:13.263180 13947 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.267503 14025 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.294426  6580 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 19.
W20260709 12:20:13.337247  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:13.220633 13668 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:13.339301  6549 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:13.226506 14063 consensus_queue.cc:1048] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 12:20:13.352461  6549 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:13.358058  6549 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:13.361555  6549 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:13.375705  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 84255.3us
I20260709 12:20:13.388299  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:13.389433  6578 raft_consensus.cc:2393] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 22: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:13.393337  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:13.399041  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:13.291342 14069 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:13.400274  6579 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 23.
I20260709 12:20:13.345901 14053 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 18. Current term is 19.
I20260709 12:20:13.412645  6582 raft_consensus.cc:1275] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 12:20:13.418844  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:13.357218 14031 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:20:13.409883  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:13.358111 14080 raft_consensus.cc:697] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:13.336697 14065 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 12:20:13.425370  6583 raft_consensus.cc:2410] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:20:13.374929 13653 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:13.416522  6582 pending_rounds.cc:85] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 12
I20260709 12:20:13.437431  6582 pending_rounds.cc:107] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 12:20:13.437734  6582 raft_consensus.cc:2889] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:13.394408 14015 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:13.398101 14037 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:13.432529 13879 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.432415  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:13.339783 13947 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:13.435935  6584 raft_consensus.cc:1275] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:20:13.347954 14025 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:13.392822 14060 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 22. Current term is 23.
I20260709 12:20:13.431491 14081 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:13.422981  6622 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:20:13.426908 13710 consensus_meta.cc:287] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.557s	user 0.006s	sys 0.005s
I20260709 12:20:13.291582 13941 consensus_queue.cc:237] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:13.413527 13880 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:13.453689  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:13.357877 14070 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:13.690454  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:13.553238 13879 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:13.601218  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 377766us
I20260709 12:20:13.540737 14080 consensus_queue.cc:237] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:13.414992 14069 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:13.399950 13901 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:13.598100  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 506403us
I20260709 12:20:13.510319 14030 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:13.548429 14092 consensus_queue.cc:1059] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:20:13.410074 13904 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.044s
I20260709 12:20:13.685469  6583 raft_consensus.cc:1240] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19. Ops: []
W20260709 12:20:13.689075  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 269351us
I20260709 12:20:13.696797  6575 raft_consensus.cc:2410] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 20 because replica has last-logged OpId of term: 19 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 16.
I20260709 12:20:13.700870  6584 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 23.
I20260709 12:20:13.707886  6550 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:13.692385 13747 consensus_meta.cc:287] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.984s	user 0.008s	sys 0.012s
I20260709 12:20:13.677912 13653 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.677147 14074 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:20:13.718533  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 124837us
I20260709 12:20:13.722162  6582 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 25. Current term is 26. Ops: []
W20260709 12:20:13.740069  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 328452us
I20260709 12:20:13.705551 14060 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.749966  7238 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:13.721308 14063 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:13.753307  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:13.760375  6583 raft_consensus.cc:2393] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 22: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:13.700287 13880 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:13.763859 14083 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.770160  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:13.694712 14081 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:13.760152 14015 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:13.750025 14074 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:13.788962 14078 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.792537 14091 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.814888 13880 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:13.815032 14091 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:13.772902 14108 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.793289 13668 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:13.793792 13958 consensus_queue.cc:1048] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:13.816799 14108 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:13.795652 13883 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:13.819429 14108 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:13.819720 14074 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:13.820189 14093 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.797405 14025 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:13.798774 13879 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:13.800565 13901 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:13.801167  6622 leader_election.cc:400] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 19. Current term is 20.
I20260709 12:20:13.738235 13653 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:13.811142  6577 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:13.812109 13947 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:13.812525 14015 raft_consensus.cc:697] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:13.749918 13747 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.026s
I20260709 12:20:13.802232 14083 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:13.814075 14078 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:13.823971  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:13.758536 14060 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:13.843699  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 129565us
I20260709 12:20:13.838770 13883 raft_consensus.cc:697] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 19 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:13.855103  6582 raft_consensus.cc:2393] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 22: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:13.820808 14093 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:13.858569  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:13.859930  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:13.851913 13824 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.872017  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:13.872495 13824 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:13.872187 13883 consensus_queue.cc:237] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:13.873818 13809 consensus_queue.cc:1059] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:13.874591 13824 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:13.858099 13880 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.850009 13879 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:13.875577 13809 raft_consensus.cc:3055] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:13.875710 13880 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:20:13.871848 14116 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:13.876046 13879 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:13.876864 14107 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:13.863430  6475 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:13.876031 13809 raft_consensus.cc:740] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:13.886842  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:20:13.889217  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 114477us
I20260709 12:20:13.889569  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:13.890322  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:13.891934  6576 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:20:13.893541 14116 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:13.894265 14116 raft_consensus.cc:697] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 22 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:13.895453 14116 consensus_queue.cc:237] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:13.904708  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:13.905485  6584 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:13.907437  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:13.908977  6577 raft_consensus.cc:1240] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 15. Current term is 16. Ops: []
W20260709 12:20:13.909339 14088 log.cc:927] Time spent T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 12:20:13.913865 13653 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:13.862115 14065 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:13.908277  6578 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:20:13.919085  6577 raft_consensus.cc:1275] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:20:13.932052  6473 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:13.889704 13947 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:13.877506 14107 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:13.953253  6622 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:13.954391  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:13.950865  6473 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:13.870724 14113 consensus_queue.cc:1059] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.082s
I20260709 12:20:13.956313  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:13.957834 14106 consensus_queue.cc:1059] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 15.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.023s
I20260709 12:20:13.947434 14070 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:13.979169  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:13.987622  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 44905.2us
I20260709 12:20:13.948829 14083 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:13.925258 14015 consensus_queue.cc:237] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:13.950101 14065 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:13.956173 13947 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:20:13.990837 14070 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:13.988626 14081 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:14.008260  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:14.012181  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:13.997000 14105 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.012655  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:14.016767  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:14.007367 14110 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:14.017855 14073 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:14.018105  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:14.018563  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:14.020221 14093 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:14.014050  6507 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:20:14.059540  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:14.053791  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:14.061066  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:14.061190  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:14.061698  6507 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:14.061903  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:14.062590  6500 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:14.063778  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:20:14.064626 14119 log.cc:927] Time spent T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:20:14.071313 14069 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:14.080224 14099 log.cc:927] Time spent T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.392s	user 0.000s	sys 0.001s
I20260709 12:20:14.083503  6505 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:14.083873  6507 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:20:14.030126 14107 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:14.031486 14065 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:14.032214  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:14.033295  6584 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 19.
I20260709 12:20:14.038527  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:14.006428 14091 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:14.088950  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:14.089743  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:14.052546  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:14.052852  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:14.013470 14070 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:14.086769  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 327617us
W20260709 12:20:14.096622  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 64468.9us
I20260709 12:20:14.052609  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:14.096529  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:14.098752  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:14.098826  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:14.099869  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:14.100471  6509 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:20:14.100792  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:14.100473  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:14.101446  6503 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:14.054593 14073 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:14.051390  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 100633us
I20260709 12:20:13.989399 13809 consensus_queue.cc:260] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:14.104818  6506 raft_consensus.cc:2393] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 23: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:14.105984  6658 raft_consensus.cc:1275] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 12:20:14.106098  6505 raft_consensus.cc:1275] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:20:14.109205  6581 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:14.110140  6473 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:14.111315  6500 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
W20260709 12:20:14.113265 14121 log.cc:927] Time spent T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:20:14.115602 14113 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.116182 14113 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:14.094278 14070 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:14.106588 14073 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:14.120669  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:14.121358  6655 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
W20260709 12:20:14.123538 14111 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.215s	user 0.003s	sys 0.001s
I20260709 12:20:14.124287  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:14.124958  6653 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 24.
I20260709 12:20:14.125097  6651 raft_consensus.cc:1275] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:20:14.092890 14107 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:14.134263 13946 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:14.136832 14130 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:14.137626 13879 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:14.109174 14065 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:14.141253  6504 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 23.
W20260709 12:20:14.142405 14129 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 12:20:14.137135  6550 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:14.144867  6473 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:14.085022 14105 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:14.148866  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:14.149394  6503 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 23.
I20260709 12:20:14.152814  6473 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:14.153007 13879 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:14.153658 14130 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:14.154062 14130 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:14.154461 14130 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:14.155980  6575 raft_consensus.cc:2410] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for term 21 because replica has last-logged OpId of term: 19 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 16.
I20260709 12:20:14.161276 13880 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:14.139747 14107 raft_consensus.cc:697] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 19 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:14.162804 14107 consensus_queue.cc:237] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:14.164796 13947 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:14.165423 14126 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 12:20:14.170948  6649 raft_consensus.cc:1275] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:20:14.171274  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:14.172209  6473 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:14.172420  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:14.146627 14065 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:14.174829 14075 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 12:20:14.175379  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:14.175546  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:14.148053 14131 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 12:20:14.176844 14105 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:14.176700 14112 consensus_queue.cc:1059] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:14.175220 14065 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:14.177508 14112 consensus_queue.cc:1243] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer 496c7c3792cd4587b4dbeba435f4be1e log is divergent from this leader: its last log entry 26.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:20:14.173255 13737 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 12:20:14.178949 13737 consensus_queue.cc:1243] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer 496c7c3792cd4587b4dbeba435f4be1e log is divergent from this leader: its last log entry 21.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:20:14.148423 14083 consensus_queue.cc:1048] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:20:14.181301 13879 consensus_queue.cc:1059] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:14.182111 14123 raft_consensus.cc:3055] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:14.182494 14123 raft_consensus.cc:740] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:14.183408 14123 consensus_queue.cc:260] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:14.173065  6650 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 23.
I20260709 12:20:14.176874  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:14.185774  6509 raft_consensus.cc:2410] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 12:20:14.186826  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:14.187263  6473 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:14.186441  6548 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:14.188997  6548 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:14.189924  6548 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:14.193562  6548 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:14.199153  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:14.200016  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:14.201555  6548 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:14.200592  7266 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
I20260709 12:20:14.152669 14015 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:14.204705  6549 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:14.209060  6548 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:14.182514 13879 raft_consensus.cc:3055] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:14.216610 13879 raft_consensus.cc:740] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:14.184719 13824 consensus_queue.cc:1048] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 12:20:14.217442 14120 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:14.198125 14091 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:20:14.220606  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 54653.2us
I20260709 12:20:14.192397 14105 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:14.166196 14062 consensus_queue.cc:1059] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 12:20:14.222386 14123 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:14.233258 14062 raft_consensus.cc:3055] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:14.233769 14062 raft_consensus.cc:740] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:14.215358 14025 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:14.234627 14025 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:14.235136 14025 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:14.234815 14062 consensus_queue.cc:260] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:14.217517 13879 consensus_queue.cc:260] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:14.242586 14118 consensus_queue.cc:1059] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 12:20:14.242717  6649 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:47858
I20260709 12:20:14.243645 14123 raft_consensus.cc:3055] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:14.207695 14083 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:14.203666 14037 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:14.244554 14037 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:14.245103 14037 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:14.252703 14083 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:14.253439 14123 raft_consensus.cc:740] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:14.254890  7266 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 19 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:14.210901 14015 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:14.209242 13947 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:14.243716  6649 raft_consensus.cc:606] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:14.260704  6649 raft_consensus.cc:614] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:14.225775  6576 raft_consensus.cc:1240] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:14.257185  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:14.223249  6548 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:14.262383  6548 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:14.265076 14093 consensus_queue.cc:1059] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 15.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:20:14.254366 14123 consensus_queue.cc:260] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:14.267401 14123 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:14.271039  6548 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:14.271350  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:14.227921 14120 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:14.276778  6548 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:14.278405  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:14.279453  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:14.283238  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 70778.2us
I20260709 12:20:14.283391 14108 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:14.260706 14015 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:14.288982  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:14.291522  6548 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:14.270268 14113 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:14.289147  6475 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:14.315719  7274 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:14.272799 13653 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:14.274863 14062 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:14.323966  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:14.329221  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:14.330025 14109 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:20:14.335925 13653 raft_consensus.cc:697] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:14.278192 14070 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:14.337271  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:14.337080 13653 consensus_queue.cc:237] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:14.337877  6507 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 23.
I20260709 12:20:14.327046  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:14.338752  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:14.276224 14120 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:14.341777  6503 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 12:20:14.341949  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 68764.2us
W20260709 12:20:14.344534 14090 log.cc:927] Time spent T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.906s	user 0.001s	sys 0.003s
I20260709 12:20:14.297837 14105 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:14.336283 14117 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:14.352578  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:14.352393  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:20:14.356483  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 142381us
I20260709 12:20:14.321578 14113 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:14.357813  6473 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:14.339243 13947 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:14.360176  6473 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:14.358453  6650 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:47858
I20260709 12:20:14.339656 14069 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:14.349684 14120 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:14.281265 14130 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:14.370138  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:14.317592 14047 consensus_queue.cc:1048] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:20:14.371747 14118 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:14.374132 14093 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:20:14.374930 14139 log.cc:927] Time spent T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 12:20:14.361619  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:14.376269  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:14.402462  6500 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:14.369798  6650 raft_consensus.cc:606] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 23 LEADER]: Received request to transfer leadership to TS 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:14.355192 14117 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:14.359441 14113 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:14.366457 14083 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:14.359719 14105 raft_consensus.cc:697] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:20:14.401583  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 110097us
W20260709 12:20:14.376499  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 49577.5us
I20260709 12:20:14.385902 14075 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:14.387698 13947 raft_consensus.cc:697] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:14.411580 14083 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:14.416271 14105 consensus_queue.cc:237] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:14.417776  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:14.409605 14117 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:14.396237 14118 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:20:14.448529  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 78498.3us
I20260709 12:20:14.397739 14114 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:14.385915 14069 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:14.410985 14113 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:20:14.449554  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 115672us
I20260709 12:20:14.447504 14083 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:14.443655  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:14.425189 14075 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:14.400825 14110 consensus_queue.cc:1048] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 12:20:14.453819  6500 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 23.
I20260709 12:20:14.463173  6650 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:47858
I20260709 12:20:14.451846 14118 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:14.461174 14075 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:14.501891 14130 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:14.503733  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 60223us
I20260709 12:20:14.504964  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:14.505412 14112 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:14.511178  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 46955.6us
I20260709 12:20:14.511896  7238 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:14.505514 13798 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:14.489820 14081 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:14.489272 14152 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.533679 14152 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:14.533720 13798 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:14.536003 14152 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:14.507277  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:14.440672 14073 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:20:14.538597  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:14.510149  6650 raft_consensus.cc:606] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:14.506515 14025 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:14.516770 14069 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:14.516907 14106 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:14.462301 14037 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:14.470564 13824 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:20:14.520573 14143 log.cc:927] Time spent T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:20:14.522987 14015 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:14.523696 14112 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:14.500872  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:14.555380  6500 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 24.
I20260709 12:20:14.544670  6658 raft_consensus.cc:2393] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 25: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:14.538760 14074 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:14.547811  6504 raft_consensus.cc:1275] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 13. (term mismatch)
I20260709 12:20:14.558418  6578 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:14.536314 13798 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:14.524122 13809 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:14.520610 13947 consensus_queue.cc:237] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:14.558408  6504 pending_rounds.cc:85] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 12
I20260709 12:20:14.559289  6504 pending_rounds.cc:107] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 25.13
I20260709 12:20:14.559530  6504 raft_consensus.cc:2889] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:14.557319  6622 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:14.551069 13737 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:20:14.557147 13824 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:14.563655 14074 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:14.572784  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:14.575762 13824 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:14.576633  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:14.578537  6474 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:14.568701 14074 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:14.581207 14015 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:14.582216 14060 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:14.583900 14025 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:14.586835 14073 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:14.601310  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:14.602054  6500 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:14.564160 14114 consensus_queue.cc:1048] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 12:20:14.604753  6474 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:14.607184 14098 raft_consensus.cc:3055] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:20:14.607754 14098 raft_consensus.cc:740] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:14.567507 14065 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:14.610669  6657 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:47858
I20260709 12:20:14.611791  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:14.612062  6657 raft_consensus.cc:606] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:14.612434  6657 raft_consensus.cc:614] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:14.615018 14112 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:14.617641  6474 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:14.620959 14137 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:14.621204 14083 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:14.625905 14037 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:14.626960 14117 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:14.627846 14078 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:14.628603 14098 consensus_queue.cc:260] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:14.629828 14098 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:20:14.631249  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:14.610152 13798 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:14.637177 13879 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:14.642454 14113 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:14.645756 14065 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:14.646409  6502 raft_consensus.cc:1240] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:14.603854 14060 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:14.648566 14075 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:14.649075  6649 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:20:14.650923 14075 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:14.610412 14080 consensus_queue.cc:1059] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 12:20:14.657351  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:14.657579  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:14.658303  6655 raft_consensus.cc:2368] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 24. Current term is 25.
I20260709 12:20:14.658859 14113 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:14.661809  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:14.662490  6656 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:20:14.676388  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:14.681154  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:14.685218 14083 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:14.686137 14025 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.686707 14025 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:14.687564 13946 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.688928 14025 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:14.692114  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:14.697824 14096 log.cc:927] Time spent T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 1.008s	user 0.002s	sys 0.003s
I20260709 12:20:14.704974 14076 raft_consensus.cc:3055] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:14.705549 14076 raft_consensus.cc:740] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:14.706707 14076 consensus_queue.cc:260] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:14.708905  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:14.709630 14076 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:14.710624  6550 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:14.711537 14031 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:14.711921 14031 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:14.712303 14031 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:14.718598  6548 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:14.724246  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 167011us
I20260709 12:20:14.738550  6548 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:14.728423  6579 raft_consensus.cc:1240] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 15. Current term is 16. Ops: []
I20260709 12:20:14.736358  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:14.738046  6581 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 12:20:14.734653  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 117137us
I20260709 12:20:14.758595  6575 raft_consensus.cc:1240] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 18. Current term is 19. Ops: []
W20260709 12:20:14.761739  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:14.688055 13946 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:14.636475 14118 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:14.725411  6500 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 24.
I20260709 12:20:14.724032  6656 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 24.
I20260709 12:20:14.766712  6507 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 12:20:14.771894  6507 pending_rounds.cc:85] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 14
I20260709 12:20:14.772197  6507 pending_rounds.cc:107] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 12:20:14.772482  6507 raft_consensus.cc:2889] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:14.769810  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:14.770677  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:14.774677  6580 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 23.
I20260709 12:20:14.774840  6509 raft_consensus.cc:2410] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 28 because replica has last-logged OpId of term: 26 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 13.
I20260709 12:20:14.738965 14025 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:14.776005  6584 raft_consensus.cc:1240] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 12:20:14.747876 14047 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:14.779894  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:14.781090  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 80557us
I20260709 12:20:14.779155  6548 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:14.786016  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:14.733781 14026 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:20:14.791129  6504 raft_consensus.cc:2393] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 22: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:14.791569  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:14.793759  6580 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:20:14.785832 13945 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.000s	sys 0.018s
I20260709 12:20:14.747931 14091 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 12:20:14.801707  6622 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:14.796353  7266 raft_consensus.cc:1240] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 19 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:14.748548 14120 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:14.778389 14025 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:14.790263 13737 raft_consensus.cc:697] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 27 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:14.810030  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:14.788834 14114 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:14.812258  6575 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:20:14.813948  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:14.814278 14147 consensus_queue.cc:1059] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.052s
I20260709 12:20:14.807142 13710 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:14.835325 13710 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:14.835772 13710 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:14.844291  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:14.844981  6584 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:20:14.845304  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:14.845927  6579 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 27.
I20260709 12:20:14.846022  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:14.847442  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:14.849565  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:14.817391  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:14.789772 14050 consensus_queue.cc:1059] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.034s
I20260709 12:20:14.851729 14050 consensus_queue.cc:1243] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer 496c7c3792cd4587b4dbeba435f4be1e log is divergent from this leader: its last log entry 19.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:20:14.852250 14050 raft_consensus.cc:3055] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:14.852644 14050 raft_consensus.cc:740] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:14.853477 14050 consensus_queue.cc:260] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:14.855306 14117 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:14.856307 14050 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:14.857820 14031 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:14.858747  6579 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 24. Current term is 25. Ops: [24.13-24.13]
I20260709 12:20:14.859020 14063 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:14.859696 14031 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:14.817651 14047 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:14.818101 14146 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:14.860694 14047 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:14.817926 14109 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 12:20:14.818591  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:14.862282  7271 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:20:14.862859 14047 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:14.847366  6584 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:14.819331 14074 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:14.797552 13945 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.000s	sys 0.019s
I20260709 12:20:14.864398  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:14.821892 14070 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:14.794014 13947 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:14.828701  6580 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 24.
I20260709 12:20:14.832202  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:14.865370  6575 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 24.
I20260709 12:20:14.844975  6576 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:14.868757  6473 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:14.864990  6474 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:14.875522 13710 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:14.879762 13737 consensus_queue.cc:237] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.13, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:14.886101  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:14.824465 14025 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:14.883442  6473 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:14.889660  6575 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22.
I20260709 12:20:14.897125  6473 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:14.890049  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 95899.5us
W20260709 12:20:14.899854 14128 log.cc:927] Time spent T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.727s	user 0.005s	sys 0.000s
I20260709 12:20:14.890753 14081 consensus_queue.cc:1059] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.009s
I20260709 12:20:14.900529  7272 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:20:14.929343  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:14.929613  6582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 3127) took 1065 ms. Trace:
W20260709 12:20:14.936816  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 104693us
I20260709 12:20:14.901845 14106 raft_consensus.cc:3055] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:14.925762 14114 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:20:14.933923  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 167503us
I20260709 12:20:14.935204  6656 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:14.889199 13879 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:14.937558  6622 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:14.891530 14123 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:14.940639  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:14.926116 14080 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:14.942461  6651 raft_consensus.cc:1240] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:14.966181  6473 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:14.967698  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:14.968503 14115 raft_consensus.cc:3055] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:14.968899 14115 raft_consensus.cc:740] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:14.969650 14115 consensus_queue.cc:260] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:14.970855 14115 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:14.971575  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:14.972178  6654 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 23.
I20260709 12:20:14.935856  6582 rpcz_store.cc:276] 0709 12:20:13.864545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:13.864676 (+   131us) service_pool.cc:224] Handling call
0709 12:20:13.881207 (+ 16531us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:13.916210 (+ 35003us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:13.916257 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:13.933995 (+ 17738us) log.cc:844] Serialized 26 byte log entry
0709 12:20:14.051268 (+117273us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:20:14.051657 (+   389us) log.cc:844] Serialized 14 byte log entry
0709 12:20:14.051881 (+   224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:14.051952 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:14.656013 (+604061us) spinlock_profiling.cc:243] Waited 29.5 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:20:14.902917 (+246904us) raft_consensus.cc:1685] finished
0709 12:20:14.929396 (+ 26479us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:14.929557 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8360,"prepare.queue_time_us":213,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":19859,"raft.run_cpu_time_us":275,"raft.run_wall_time_us":6962,"spinlock_wait_cycles":67950336,"thread_start_us":91583,"threads_started":1,"wal-append.queue_time_us":108127}
I20260709 12:20:14.986116 14080 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:14.986810 14080 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:14.990079  6501 raft_consensus.cc:2410] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 24 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 12:20:14.990247  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:14.992818  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:14.993561  6656 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 23.
I20260709 12:20:14.993899  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:14.931485  6548 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:14.995999  6550 leader_election.cc:400] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 24. Current term is 25.
W20260709 12:20:14.997033  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 65636.7us
I20260709 12:20:14.997392  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:14.998123  6500 raft_consensus.cc:2368] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 23. Current term is 24.
I20260709 12:20:14.999349  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:14.998394  6548 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:14.938050 14081 consensus_queue.cc:1243] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer 20a202c76244485fbc8f1610473db8b9 log is divergent from this leader: its last log entry 19.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:20:15.003400  6473 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:14.941859 14106 raft_consensus.cc:740] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:15.012660  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:14.988231 13904 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 12:20:14.950984 13947 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:14.958570 14114 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:15.022220  6504 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:14.965605 14123 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:14.958787 14112 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:14.964275 14075 raft_consensus.cc:2764] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 12:20:15.029786  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:15.030861  6622 connection.cc:256] RPC call timeout handler was delayed by -2.2664630480001051s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:20:15.019949  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:15.033936  6505 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:20:15.023612  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:15.008878 14109 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:15.025849  6548 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:15.009002 13809 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:20:15.013183 13946 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:15.014984 14081 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:15.041347 14109 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:15.024986 14114 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:15.048573  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:15.048031  6473 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:15.045256  6576 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:20:15.012637 14070 consensus_queue.cc:1059] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 12:20:15.032939 13668 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:15.012914 14015 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 12:20:15.033202 14123 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:15.009374 14060 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 12:20:15.053505 14109 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:15.082495 14063 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:15.025595 13947 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:15.084846 13710 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:15.085528  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 3254) took 1071 ms. Trace:
I20260709 12:20:15.085597 14074 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:15.086452  6503 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 13. (term mismatch)
I20260709 12:20:15.131098  6503 pending_rounds.cc:85] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 12
I20260709 12:20:15.131341  6503 pending_rounds.cc:107] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 23.13
I20260709 12:20:15.131597  6503 raft_consensus.cc:2889] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:15.129822  6501 rpcz_store.cc:276] 0709 12:20:14.014605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:14.014761 (+   156us) service_pool.cc:224] Handling call
0709 12:20:15.085302 (+1070541us) spinlock_profiling.cc:243] Waited 935 ms on lock 0x7b54000fee54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:15.085466 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2150201600}
I20260709 12:20:15.087989  6653 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:15.046846 13809 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:15.089386 14081 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:15.060595 14142 consensus_queue.cc:1059] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:20:15.096407 14120 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:15.040618 14098 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.025s
W20260709 12:20:15.092447  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:15.105834  6473 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:20:15.107867  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 81924.9us
W20260709 12:20:15.109653  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 157640us
I20260709 12:20:15.121227 14025 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:15.116165 13945 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:15.147984  6473 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:15.149153  6550 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:15.139643  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:15.151858  6653 raft_consensus.cc:2410] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 23 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 12:20:15.129966  6578 raft_consensus.cc:1275] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:20:15.133579 13809 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:15.138243 14137 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.178853  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:15.130585 13946 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 12:20:15.180697  6473 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:15.172545  6658 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:47858
I20260709 12:20:15.186100  6658 raft_consensus.cc:606] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 23 LEADER]: Received request to transfer leadership to TS 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:15.194098  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:15.196190  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:15.196898  6658 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:15.202497  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:15.153115 14031 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:20:15.207742  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 46279.7us
I20260709 12:20:15.210320  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:15.204715 13947 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:15.175027 14070 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:15.147624 14081 raft_consensus.cc:697] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 24 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:15.156798  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:15.219604  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:15.220008  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:15.220609  6576 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:20:15.220887  6580 raft_consensus.cc:2393] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 22: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:15.224319  6622 leader_election.cc:400] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 23. Current term is 24.
I20260709 12:20:15.224694  6622 leader_election.cc:403] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:15.228185  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:15.228585  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:15.229625  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:15.230779  6577 raft_consensus.cc:1240] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:15.233706  6579 raft_consensus.cc:2393] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 19: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:15.235837  6475 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:15.205229 14142 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:15.239995  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:15.236359  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:15.241780  6658 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 23.
I20260709 12:20:15.211293 14112 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:15.256830  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:15.258513  6505 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:15.261868  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:15.258474  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:15.272264  6657 raft_consensus.cc:2393] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 23: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:15.269200  6579 raft_consensus.cc:1275] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:20:15.269567  6504 raft_consensus.cc:1240] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:20:15.274837  6575 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 23. Current term is 24. Ops: [23.14-23.14]
I20260709 12:20:15.217808 14110 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:15.208789 14113 consensus_queue.cc:1059] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 12:20:15.272693  6583 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22.
I20260709 12:20:15.289620  6500 raft_consensus.cc:1275] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:20:15.279709  6507 raft_consensus.cc:1240] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:15.285854  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:15.180122 14137 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:15.279277  6475 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:15.245254 14074 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 23. Current term is 24.
I20260709 12:20:15.238438 14070 raft_consensus.cc:697] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 24 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:15.256165 14142 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:15.209313 14109 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:20:15.276753 13798 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:15.211467 13901 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.069s
I20260709 12:20:15.287887  6575 raft_consensus.cc:1240] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:20:15.301090  6584 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 22. Current term is 23. Ops: [20.12-22.13]
I20260709 12:20:15.265488 13945 raft_consensus.cc:697] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:15.325122  6656 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:20:15.278394 14065 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:15.307190 14142 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:15.156314 14106 consensus_queue.cc:260] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:15.329846  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 50613us
I20260709 12:20:15.309437  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:15.314162  6575 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:15.325687  6501 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 22. Current term is 23. Ops: [20.12-22.13]
I20260709 12:20:15.331542  6475 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:15.286540 14112 raft_consensus.cc:697] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:15.331712  6504 raft_consensus.cc:2393] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 24: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:15.261518 14069 consensus_queue.cc:1059] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 12:20:15.271163 14164 consensus_queue.cc:1059] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 12:20:15.333593  6500 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:15.286960 14060 consensus_queue.cc:1059] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 12:20:15.311015 13798 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:15.324954 13880 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:15.327782 14065 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:15.400836  6578 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:15.342146 14110 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:20:15.404727  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:15.405143  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:15.412428  6475 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:15.370810 13798 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:15.402685 14065 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:20:15.442469  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:15.376928 13880 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:15.381939 14164 consensus_queue.cc:1243] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 20.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:20:15.453346  6584 raft_consensus.cc:1240] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:15.338979 14081 consensus_queue.cc:237] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:15.395988 14091 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:15.418339  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:20:15.462212  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:15.496838  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 84553.7us
I20260709 12:20:15.467440  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:15.466979  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:15.503614  6578 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:15.514689  6584 raft_consensus.cc:1240] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:15.458946 14025 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:15.496657 13880 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:15.489089 14047 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:15.517218 14063 consensus_queue.cc:1048] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.012s
W20260709 12:20:15.529582 14179 log.cc:927] Time spent T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:20:15.530886  6657 raft_consensus.cc:3055] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:15.531319  6657 raft_consensus.cc:740] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:15.508387 14109 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:15.513610 13737 consensus_queue.cc:1059] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 12:20:15.535137 13737 raft_consensus.cc:3055] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:15.535497 13737 raft_consensus.cc:740] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:15.537277  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:15.538540  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:15.532143  6657 consensus_queue.cc:260] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:15.539606  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:15.540753  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:15.541771  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:15.454308 13945 consensus_queue.cc:237] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:15.544652  6550 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:15.540225  6657 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:15.545713 13809 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:15.550232 14113 consensus_queue.cc:1048] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:15.547870 13798 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:15.552940 14123 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:15.554401 13737 consensus_queue.cc:260] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:15.555346  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:15.558550 13809 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:15.555735 13737 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:15.557098  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:15.560156  6503 raft_consensus.cc:1240] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:15.560657 14131 raft_consensus.cc:3055] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:15.561290 14131 raft_consensus.cc:740] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:15.560392  6500 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:15.563098 13798 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:15.567818  6503 raft_consensus.cc:1275] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:20:15.577265 13747 raft_consensus.cc:3055] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:15.577790 13747 raft_consensus.cc:740] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:15.586678  6581 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:15.588799 14118 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:15.594274 14106 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:15.605775  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:15.608407 14166 consensus_queue.cc:1059] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:15.609714 13737 raft_consensus.cc:3055] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:15.610395 14075 raft_consensus.cc:993] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4: : Instructing follower e7554c39679a41bbbca4659b72ab5c65 to start an election
I20260709 12:20:15.610225 13737 raft_consensus.cc:740] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:15.612027 13737 consensus_queue.cc:260] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:15.610913 14075 raft_consensus.cc:1081] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Signalling peer e7554c39679a41bbbca4659b72ab5c65 to start an election
W20260709 12:20:15.613067  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 68485.4us
I20260709 12:20:15.613667 14083 raft_consensus.cc:3055] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:15.614203 14083 raft_consensus.cc:740] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:15.615229 14083 consensus_queue.cc:260] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:15.616897 14083 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:20:15.617100 14182 log.cc:927] Time spent T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 12:20:15.615129  6657 raft_consensus.cc:1275] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 12. (term mismatch)
I20260709 12:20:15.621483  6657 pending_rounds.cc:85] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 11
I20260709 12:20:15.623788  6657 pending_rounds.cc:107] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 12:20:15.626044  6657 raft_consensus.cc:2889] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:15.630918  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:20:15.634245  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 130198us
I20260709 12:20:15.463507 14185 consensus_queue.cc:1059] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 12:20:15.655750  6583 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 22. Current term is 23. Ops: [22.12-22.12]
W20260709 12:20:15.628597 14115 consensus_meta.cc:287] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.657s	user 0.008s	sys 0.008s
I20260709 12:20:15.667695  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:15.506335 14164 consensus_queue.cc:1059] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.147s
I20260709 12:20:15.415022 14070 consensus_queue.cc:237] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 23.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:15.652508 14074 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
I20260709 12:20:15.670861  6655 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 12:20:15.652781 14109 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:15.507818 14091 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:15.653286 14118 consensus_queue.cc:1048] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 12:20:15.617106 14137 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:15.683423 14162 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.482718 14112 consensus_queue.cc:237] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:15.663717  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:15.692835  6578 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 25.
W20260709 12:20:15.696633  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 364012us
I20260709 12:20:15.726771  6651 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:20:15.675401 14114 consensus_meta.cc:287] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.613s	user 0.010s	sys 0.000s
I20260709 12:20:15.627921  6500 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 23.
W20260709 12:20:15.707208  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 157492us
I20260709 12:20:15.717756  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:15.729977  6500 raft_consensus.cc:1275] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:20:15.732848  6580 raft_consensus.cc:2393] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 19: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:15.737951  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:15.737514  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:15.739496  6578 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:15.696278 13880 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:15.783531  6578 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 28.
W20260709 12:20:15.784209  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 270448us
I20260709 12:20:15.785589  6622 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:15.788738  6473 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:15.727933 14109 raft_consensus.cc:697] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:15.744468 14060 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.798331 14164 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:15.806559  6622 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:15.808362  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:15.813334  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:15.815930  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:15.793857 14137 consensus_queue.cc:1059] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 12:20:15.743460 14142 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:15.817354  6655 raft_consensus.cc:1275] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:20:15.820502  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:15.821224  6650 raft_consensus.cc:2368] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 24. Current term is 25.
I20260709 12:20:15.825376  6505 raft_consensus.cc:2393] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 23: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:15.818774 14142 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:15.811182 14080 consensus_meta.cc:287] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.824s	user 0.008s	sys 0.007s
I20260709 12:20:15.834304  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:15.836705  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:15.840968  6650 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:15.841025  6578 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:15.740438 14162 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:15.744946 14114 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:15.822739 14075 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:15.851768  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:15.853837  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:15.865636  6548 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:15.867131  6581 raft_consensus.cc:1240] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:15.828090 14110 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:20:15.851612  6550 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:15.871685  6502 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 24 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 23. Current term is 24. Ops: [23.14-23.14]
I20260709 12:20:15.877844 14118 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:15.878367 14197 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:15.892382 14197 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:15.820374 14060 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:15.896265 14025 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:15.896695 14063 raft_consensus.cc:3055] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:15.897209 14063 raft_consensus.cc:740] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:15.879935  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:15.898214 14063 consensus_queue.cc:260] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:15.899585  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:15.900457  6622 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:15.848703 14080 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:20:15.900840  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:15.901855  6504 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:15.902264  6622 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:15.904227  6509 raft_consensus.cc:2393] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 25: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:15.899677 14063 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:15.904914 14109 consensus_queue.cc:237] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:15.906232  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:15.907116  6509 raft_consensus.cc:2368] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 24. Current term is 25.
I20260709 12:20:15.871459 13901 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:15.909274 13901 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:15.908138  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:15.911075 14110 raft_consensus.cc:697] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 28 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:15.912253 14110 consensus_queue.cc:237] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:15.915737  6654 raft_consensus.cc:3055] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:15.916174  6654 raft_consensus.cc:740] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:20:15.882436  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:15.921470  6550 leader_election.cc:400] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 24. Current term is 25.
I20260709 12:20:15.917006  6509 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 24.
I20260709 12:20:15.887410  6649 raft_consensus.cc:1240] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:15.921878  6550 leader_election.cc:403] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
I20260709 12:20:15.932492  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:20:15.933104  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 67367.5us
I20260709 12:20:15.934180  6548 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:15.927389  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:15.941637  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:15.946095  6504 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 24.
I20260709 12:20:15.951171  6578 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 23.
I20260709 12:20:15.955070  6473 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:15.932868  6654 consensus_queue.cc:260] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:15.882661 13941 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 12:20:16.033078  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 78136.9us
I20260709 12:20:15.883539 14091 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:16.034724 14142 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:20:16.035429  6577 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 24. Current term is 25. Ops: [24.13-24.13]
I20260709 12:20:15.890740 13880 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:16.036585 13737 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:15.887354 14075 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:16.037201 14075 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:16.037659 14075 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:16.039577  6651 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 24.
I20260709 12:20:15.889250 13947 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.104s
I20260709 12:20:16.040362  6657 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:16.042469  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:15.993254 14053 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 24. Current term is 25.
I20260709 12:20:15.889336 14118 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:15.886749 14131 consensus_queue.cc:260] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:15.978060  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 43908.4us
I20260709 12:20:15.921573 14202 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:15.956313 14193 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:16.011394  6504 raft_consensus.cc:1240] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 23. Current term is 24. Ops: [21.11-23.12]
I20260709 12:20:16.034765  6654 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:15.921241 13945 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:20:15.890694 14114 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:16.050904  6473 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:20:16.068187 14065 consensus_meta.cc:287] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.599s	user 0.006s	sys 0.007s
I20260709 12:20:16.030561 13901 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:16.011716 14164 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:15.968802 13653 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:16.066491  6548 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:16.086380  6650 raft_consensus.cc:2410] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 23 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
W20260709 12:20:16.056893  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 154730us
I20260709 12:20:16.066283 13880 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:16.081017 14212 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:20:16.083345  6504 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:45880
I20260709 12:20:16.145129  6504 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 12:20:16.145367  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 78966.9us
I20260709 12:20:16.056870 13947 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:16.100752  6654 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 13. (term mismatch)
I20260709 12:20:16.146226  6654 pending_rounds.cc:85] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 12
I20260709 12:20:16.146586  6654 pending_rounds.cc:107] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 22.13
I20260709 12:20:16.146993  6654 raft_consensus.cc:2889] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:16.101141  7269 raft_consensus.cc:1240] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 19 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19. Ops: []
W20260709 12:20:16.105777  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:16.141351  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:16.111361 14207 log.cc:927] Time spent T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 12:20:16.112310 14203 log.cc:927] Time spent T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:20:16.064091 14197 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:16.122663 14118 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:16.124643 14069 raft_consensus.cc:3055] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:16.071127 14193 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:16.084218 13946 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:20:16.133131  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 82350.3us
I20260709 12:20:16.140694  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:16.145546  6504 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:15.981491 14196 consensus_queue.cc:1059] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.130s
I20260709 12:20:16.109179 13945 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:16.150456  6548 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:16.170099  6475 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:16.174038  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:16.167155 14025 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:20:16.143992 14202 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:16.136449 14110 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:16.137068 14065 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:16.183847 14137 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 23 current term: 23 is higher.
W20260709 12:20:16.217394  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 67033.8us
I20260709 12:20:16.217386  6583 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:20:16.233006  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:16.250202  6507 raft_consensus.cc:1240] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:16.177549 14069 raft_consensus.cc:740] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:16.147809 13880 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:16.170982 14053 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 12:20:16.222199 14050 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:16.172880 13947 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:16.260941  6548 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:16.275600  6548 leader_election.cc:400] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 24. Current term is 25.
I20260709 12:20:16.261756 14025 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:16.277716  6548 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:20:16.282267  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 141022us
I20260709 12:20:16.282727  6583 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 27. Current term is 28. Ops: []
I20260709 12:20:16.216867 13946 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:16.253964 14110 raft_consensus.cc:697] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 23 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:16.227843 13653 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:20:16.252463 14188 log.cc:927] Time spent T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:20:16.309842  6652 raft_consensus.cc:1275] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 12:20:16.310333  6652 pending_rounds.cc:85] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 10
I20260709 12:20:16.310578  6652 pending_rounds.cc:107] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 12:20:16.310809  6652 raft_consensus.cc:2889] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:20:16.257712  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:16.276268 14196 raft_consensus.cc:2764] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:20:16.222664 13945 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:16.284412 13941 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:16.319442  6584 raft_consensus.cc:1217] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Deduplicated request from leader. Original: 16.13->[18.14-18.14]   Dedup: 18.14->[]
I20260709 12:20:16.291224  6506 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 24 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:16.254395 13798 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 12:20:16.286536 14050 raft_consensus.cc:697] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 24 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:16.305792 13947 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:16.305936 13747 consensus_queue.cc:260] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:16.188632 14212 consensus_queue.cc:1059] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.084s
I20260709 12:20:16.279872 14091 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:16.259539 14015 consensus_queue.cc:1059] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 12:20:16.283524 14114 raft_consensus.cc:3055] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:16.267408 14109 consensus_queue.cc:1048] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:20:16.289197  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:16.320883  6584 raft_consensus.cc:1240] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 12:20:16.290153 14108 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:16.327739  6500 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 28 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 27. Current term is 28. Ops: []
I20260709 12:20:16.293275 14069 consensus_queue.cc:260] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:16.238137  6504 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:16.333384 14050 consensus_queue.cc:237] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:16.337077 13941 raft_consensus.cc:697] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 24 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:16.337092 13747 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:16.310632 13737 consensus_queue.cc:1059] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 23.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.027s
W20260709 12:20:16.338757 14217 log.cc:927] Time spent T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 12:20:16.344527 14225 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:20:16.347549 14191 log.cc:927] Time spent T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.652s	user 0.000s	sys 0.002s
I20260709 12:20:16.350414  6505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 3143) took 1115 ms. Trace:
I20260709 12:20:16.392612 14015 consensus_queue.cc:1243] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 16.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:20:16.389916  6501 raft_consensus.cc:1240] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:16.391304  6500 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:16.360164 14212 raft_consensus.cc:3055] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:16.465046  6504 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:16.359622 14114 raft_consensus.cc:740] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:16.471943  6505 rpcz_store.cc:276] 0709 12:20:15.235133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:15.285596 (+ 50463us) service_pool.cc:224] Handling call
0709 12:20:16.350157 (+1064561us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b54001e73d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:16.350317 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1219644288}
I20260709 12:20:16.367383  6651 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 12:20:16.490846  6500 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:20:16.467196 14118 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:16.394541 14108 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:16.484995 14053 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:16.521818 13901 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:16.523598 14220 log.cc:927] Time spent T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.196s	user 0.000s	sys 0.000s
W20260709 12:20:16.524544 14211 log.cc:927] Time spent T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 12:20:16.485898 14212 raft_consensus.cc:740] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:20:16.525202 14213 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 12:20:16.555745  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:16.474689 14225 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:16.466565 14110 consensus_queue.cc:237] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:16.525563 14193 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:16.543850 14065 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:16.543385 13880 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:16.588907  7272 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38562 (request call id 1225) took 1371 ms. Trace:
I20260709 12:20:16.509501 14118 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:16.557489 13901 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:16.590111  7272 rpcz_store.cc:276] 0709 12:20:15.207265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:15.207545 (+   280us) service_pool.cc:224] Handling call
0709 12:20:15.207903 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:15.216610 (+  8707us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:15.216651 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:15.256807 (+ 40156us) log.cc:844] Serialized 26 byte log entry
0709 12:20:15.483496 (+226689us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:15.483556 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:15.483651 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:16.346952 (+863301us) spinlock_profiling.cc:243] Waited 2.32 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:20:16.574687 (+227735us) raft_consensus.cc:1685] finished
0709 12:20:16.574718 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:16.578588 (+  3870us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8292,"raft.queue_time_us":9752,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":26088,"spinlock_wait_cycles":5336832,"thread_start_us":154521,"threads_started":1,"wal-append.queue_time_us":396593}
I20260709 12:20:16.543968 14053 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:16.558638 13941 consensus_queue.cc:237] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:16.619081 14108 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:16.631650 14075 consensus_meta.cc:287] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.580s	user 0.013s	sys 0.000s
I20260709 12:20:16.632164 14075 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:16.634337 14075 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:16.635020  6658 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:16.618052 14212 consensus_queue.cc:260] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:16.637104 14212 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:16.609035 13945 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:16.609887 14193 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:20:16.641748  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:16.645648 13901 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:16.609617 14065 consensus_queue.cc:1059] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:20:16.563706 14114 consensus_queue.cc:260] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:16.614224 13737 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:16.640604 14193 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:16.653986  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 3288) took 1142 ms. Trace:
I20260709 12:20:16.654887  6503 rpcz_store.cc:276] 0709 12:20:15.511490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:15.572715 (+ 61225us) service_pool.cc:224] Handling call
0709 12:20:15.574993 (+  2278us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:20:15.577326 (+  2333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:15.577516 (+   190us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:20:15.901726 (+324210us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b2c00000c88. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:20:15.902341 (+   615us) log.cc:844] Serialized 52 byte log entry
0709 12:20:16.135696 (+233355us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:16.135731 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:16.135881 (+   150us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:16.653275 (+517394us) raft_consensus.cc:1685] finished
0709 12:20:16.653306 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:16.653911 (+   605us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3767,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"spinlock_wait_cycles":33343872,"thread_start_us":232953,"threads_started":1,"wal-append.queue_time_us":350651}
W20260709 12:20:16.658291  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 190345us
I20260709 12:20:16.659188  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:16.663038  6622 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:16.652252 14075 raft_consensus.cc:3055] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:16.665925  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:16.654215 13737 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:16.666697 14231 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:20:16.667500 14230 log.cc:927] Time spent T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:20:16.682947 13737 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:16.676945 14063 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:16.684515 14219 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:16.685179 14219 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:16.687754 14219 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:16.685200 14063 raft_consensus.cc:697] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 25 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:16.709367 14063 consensus_queue.cc:237] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 23.11, Last appended by leader: 11, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:20:16.666323 14115 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:16.690227 14193 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:16.693377 14053 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:16.696239 14114 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:16.712707 14115 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:16.716145 13946 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:16.717343  6509 raft_consensus.cc:1275] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:20:16.718575  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 988) took 1080 ms. Trace:
I20260709 12:20:16.718951  6509 rpcz_store.cc:276] 0709 12:20:15.638409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:15.924175 (+285766us) service_pool.cc:224] Handling call
0709 12:20:15.924513 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:16.717755 (+793242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:16.718340 (+   585us) spinlock_profiling.cc:243] Waited 792 ms on lock 0x7b54002c3dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:16.718506 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1822578432}
I20260709 12:20:16.721388  6509 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 12:20:16.722805  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 3068) took 1049 ms. Trace:
I20260709 12:20:16.701036 13653 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:16.702592 14225 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:16.703842 14118 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:16.672904 14075 raft_consensus.cc:740] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:20:16.724182  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 58337.5us
I20260709 12:20:16.723202  6656 rpcz_store.cc:276] 0709 12:20:15.631266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:15.632291 (+  1025us) service_pool.cc:224] Handling call
0709 12:20:15.632715 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:15.633032 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:20:15.663933 (+ 30901us) log.cc:844] Serialized 14 byte log entry
0709 12:20:16.616722 (+952789us) spinlock_profiling.cc:243] Waited 1.47 ms on lock 0x7b18004381c0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d99455 00007f2875e81060 00007f2875e93716 00007f2875ea4287 00007f2875ea4208
0709 12:20:16.680609 (+ 63887us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:16.680680 (+    71us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:20:16.680744 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:16.680832 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:16.680982 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":30902,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":62,"raft.queue_time_us":9229,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"spinlock_wait_cycles":3388800,"thread_start_us":63538,"threads_started":1}
I20260709 12:20:16.730366  6503 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 24 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:16.732036  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:16.743098 14075 consensus_queue.cc:260] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:16.744441 14075 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:16.748940  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:16.749728  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:16.749768  6580 raft_consensus.cc:2393] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 19: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:16.750772  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:16.751350  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:16.754421  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:16.754758 14166 consensus_queue.cc:1059] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:16.753906 14232 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:16.756757 14232 raft_consensus.cc:3055] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:20:16.757143 14232 raft_consensus.cc:740] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:20:16.757615  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:16.755695 14166 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:16.730762 14114 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:16.750752 14115 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:16.755695 14212 consensus_queue.cc:1059] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 12:20:16.765707 14212 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:16.766259 14212 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:16.768226 14120 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:16.768836 14120 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:16.769546 14214 log.cc:927] Time spent T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.505s	user 0.009s	sys 0.000s
I20260709 12:20:16.760186 14166 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:16.771298 14120 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:16.771836 14166 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:16.772625 13879 raft_consensus.cc:3055] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:16.773056 13879 raft_consensus.cc:740] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:16.773773 14080 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:16.773854 13879 consensus_queue.cc:260] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:16.774950 13879 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:16.778574  6502 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 3135) took 1555 ms. Trace:
I20260709 12:20:16.778944  6502 rpcz_store.cc:276] 0709 12:20:15.223442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:15.223581 (+   139us) service_pool.cc:224] Handling call
0709 12:20:15.259015 (+ 35434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:15.877555 (+618540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:16.778329 (+900774us) spinlock_profiling.cc:243] Waited 612 ms on lock 0x7b54002c51d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:16.778515 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1408125312}
I20260709 12:20:16.757949 14232 consensus_queue.cc:260] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 27.14, Last appended by leader: 14, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:16.779855  6657 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 14. (term mismatch)
I20260709 12:20:16.780195  6657 pending_rounds.cc:85] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 13
I20260709 12:20:16.780321 14232 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:16.783037 14117 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:16.783576 14117 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:16.785463  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: true dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:16.785655 14117 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:16.786036  6658 raft_consensus.cc:3055] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:16.786360  6658 raft_consensus.cc:740] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:16.787253  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:16.787107  6658 consensus_queue.cc:260] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:16.787986  6658 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:16.788607  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:16.780458  6657 pending_rounds.cc:107] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 23.14
I20260709 12:20:16.789497  6657 raft_consensus.cc:2889] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:16.789641  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:16.790571  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:16.791725  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:16.792354  7274 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:20:16.791080  6657 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:20:16.796075  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:16.796815  7272 raft_consensus.cc:2393] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 22: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
W20260709 12:20:16.797796  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:16.798462  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:16.799034  7271 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:20:16.788488  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:16.799856  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:16.800480 14212 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:16.800776  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:16.801035  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:16.802371 14166 raft_consensus.cc:3055] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:16.802738 14166 raft_consensus.cc:740] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:16.802989 14081 consensus_queue.cc:1059] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:16.804052 14081 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:16.804915 14115 raft_consensus.cc:3055] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:16.805321 14115 raft_consensus.cc:740] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:16.806252 14115 consensus_queue.cc:260] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:16.807449 14115 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:16.809113  6473 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:16.812628 14193 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:16.816541 14117 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:16.818725  6658 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:20:16.788111  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 3292) took 1271 ms. Trace:
I20260709 12:20:16.819070  6475 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:16.821503  6549 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:16.823963  6652 raft_consensus.cc:1275] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:20:16.825724  6475 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:16.827960  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:16.820081  6508 rpcz_store.cc:276] 0709 12:20:15.516662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:15.728189 (+211527us) service_pool.cc:224] Handling call
0709 12:20:15.780879 (+ 52690us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:15.917564 (+136685us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:15.917608 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:15.918125 (+   517us) log.cc:844] Serialized 26 byte log entry
0709 12:20:16.170635 (+252510us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:20:16.261000 (+ 90365us) log.cc:844] Serialized 14 byte log entry
0709 12:20:16.262754 (+  1754us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:16.786707 (+523953us) spinlock_profiling.cc:243] Waited 4.31 ms on lock 0x7b1800035260. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d6a2a6 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419
0709 12:20:16.787820 (+  1113us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b540046cdd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:16.787868 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:16.787909 (+    41us) raft_consensus.cc:1685] finished
0709 12:20:16.787924 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:16.788060 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8282,"prepare.queue_time_us":104419,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":39,"raft.queue_time_us":5185,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":20486,"spinlock_wait_cycles":323054208,"thread_start_us":217607,"threads_started":2,"wal-append.queue_time_us":257013}
W20260709 12:20:16.844720  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 55388.5us
I20260709 12:20:16.847681  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:16.848552  6658 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:20:16.817087 14117 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:16.816967 14166 consensus_queue.cc:260] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:16.854501 14166 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:16.854895 14117 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:16.850394 14083 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:16.855634 14083 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:16.856014 14083 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:16.864676  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:16.818154 14120 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:16.869086 14120 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:16.870219 14222 raft_consensus.cc:3055] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:16.870430 14123 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:16.870708 14222 raft_consensus.cc:740] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:16.872016 14222 consensus_queue.cc:260] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:16.852502 14075 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:20:16.873334  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:16.873343  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:16.850268  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:16.876592  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 55195.8us
I20260709 12:20:16.876900  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:16.877338  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:16.877746  6650 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:20:16.877942  6549 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:16.878993 14222 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:16.872984 14075 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:16.879752 14075 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:16.880115 14131 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:16.824925 14091 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:16.884774 14091 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:16.894506  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 68854.2us
I20260709 12:20:16.887809 14091 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:16.872411 14120 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:16.905416  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:16.908586  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:16.893805 14183 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:16.950670 14183 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:16.953140 14183 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:16.954250 14222 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:16.954854 14222 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:16.957175 14222 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:16.913482 13941 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 12:20:16.959187  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:16.895278 14123 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:16.960546 14123 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:16.961709 14222 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:16.962330 14222 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:16.915040 14075 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:16.915838 14117 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:16.920403 13883 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:16.920843 14195 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:16.921377  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 43528.9us
I20260709 12:20:16.969857 13883 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:16.921720  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:16.967023 14117 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:16.972590 13883 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:16.972992 14117 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:16.921864 14053 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:16.973843 14053 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:16.974251 14053 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:16.976519  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:16.977278  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:16.977174 14195 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:16.978559  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:16.979553 14195 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:16.979765  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:16.922587 14171 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:16.981678 14060 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:16.972492 14075 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:16.981885 14171 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:16.984078 14171 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:16.926959  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:16.927716  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:16.897125 14189 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:16.897022 13653 consensus_queue.cc:1059] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 12:20:16.962658 14123 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:17.032251 13653 raft_consensus.cc:3055] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:17.033053 13653 raft_consensus.cc:740] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:17.033819 13653 consensus_queue.cc:260] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:17.034969 13653 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:17.035164  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 3144) took 1761 ms. Trace:
I20260709 12:20:17.035637  6507 rpcz_store.cc:276] 0709 12:20:15.274196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:15.291024 (+ 16828us) service_pool.cc:224] Handling call
0709 12:20:15.310287 (+ 19263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:16.250851 (+940564us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:17.034932 (+784081us) spinlock_profiling.cc:243] Waited 940 ms on lock 0x7b54002c42d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:17.035103 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2160898816}
I20260709 12:20:16.851680 14114 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:16.935662 14083 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:16.925261 14026 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:20:16.986467 14014 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:16.998935  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:16.988479 14233 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:17.004108  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:16.986742  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:17.050300  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:16.987844  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:17.051221  6658 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:20:17.052245  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:17.053138  6658 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 19.
I20260709 12:20:17.053313  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:17.054292  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:17.055032  6658 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 24.
I20260709 12:20:17.055984  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:17.057011  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:17.058393  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:16.987468 13947 raft_consensus.cc:3055] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:16.045735  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:20:17.013429 14201 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.915s	user 0.014s	sys 0.031s
W20260709 12:20:17.016870  6404 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46184 (request call id 119) took 70818 ms (client timeout 15000 ms). Trace:
I20260709 12:20:17.072504  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:17.017268  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:17.020474 14222 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:17.022418  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:17.022557  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:17.073853  6505 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:20:17.074522  6582 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:17.075153  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:17.075222  6548 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:17.061021 14014 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:17.076496  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:17.076781 14171 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:17.077071  6505 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 15.
I20260709 12:20:17.077275 14171 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:17.015599  7271 consensus_queue.cc:237] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:17.077817 14171 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:17.078008 14014 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:17.078276 14115 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:17.078810 14115 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:17.027017  6651 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 23 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 12:20:17.027220  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 3053) took 2059 ms. Trace:
I20260709 12:20:16.984934 14069 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:17.030465  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:17.031066  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:16.906106  6508 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:17.036746  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:17.037400  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:17.038128  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:17.084228  6575 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:20:17.081388  6649 rpcz_store.cc:276] 0709 12:20:14.967517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:14.970903 (+  3386us) service_pool.cc:224] Handling call
0709 12:20:14.971162 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:15.930253 (+959091us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:17.026986 (+1096733us) spinlock_profiling.cc:243] Waited 916 ms on lock 0x7b540008fcd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:17.027158 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2106644736}
I20260709 12:20:17.086807  6548 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:17.088040  6651 raft_consensus.cc:1240] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:17.037882 14053 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:17.089108  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:17.088280  6582 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 23 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 12:20:17.041061  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 3167) took 1302 ms. Trace:
I20260709 12:20:17.042412  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 3156) took 1196 ms. Trace:
I20260709 12:20:17.093446  6501 rpcz_store.cc:276] 0709 12:20:15.845676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:16.085132 (+239456us) service_pool.cc:224] Handling call
0709 12:20:16.182887 (+ 97755us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:16.477689 (+294802us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:17.042074 (+564385us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b5400387554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:17.042338 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":475643776}
W20260709 12:20:17.045938  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 45603us
I20260709 12:20:17.051230  6507 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:20:17.061296  6583 raft_consensus.cc:2393] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 25: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:17.064498  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:17.022975  6657 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 27 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:20:17.048501 14189 raft_consensus.cc:697] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:17.083189  6581 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22.
W20260709 12:20:17.065807 14201 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.033s	user 0.015s	sys 0.032s
I20260709 12:20:17.104135 13653 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:17.104744 13653 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:17.105233 13653 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:17.089682  6505 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:20:17.092531  6584 rpcz_store.cc:276] 0709 12:20:15.738804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:15.740142 (+  1338us) service_pool.cc:224] Handling call
0709 12:20:16.135249 (+395107us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:16.368377 (+233128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:17.039842 (+671465us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b5400057dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:17.040801 (+   959us) spinlock_profiling.cc:243] Waited 393 ms on lock 0x7b5400057dd0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:17.040984 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1327708160}
I20260709 12:20:17.118083  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:17.118289  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:17.064257 14233 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:17.118957  6507 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
I20260709 12:20:17.119920  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:17.115453  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:17.120687  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:17.135391  6508 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
I20260709 12:20:17.120921 14233 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:17.121333  6548 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:17.123143 14189 consensus_queue.cc:237] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:17.124523  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.124562 14196 consensus_queue.cc:1059] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:17.125793 14112 consensus_queue.cc:1059] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:17.125109  6399 catalog_manager.cc:5697] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 11 to 13, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 13 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:20:17.129621 14083 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:17.060889 13947 raft_consensus.cc:740] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:17.132665  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:17.152053  6579 raft_consensus.cc:2393] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 20: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:17.151870  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:17.132844  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:17.152666  6509 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
I20260709 12:20:17.134763  6507 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 24.
I20260709 12:20:17.136474 14222 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:17.116271 14195 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:17.117936  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:17.154596  6505 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:17.022305 14117 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:17.139561  6473 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:17.137933  7240 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:17.146116  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:17.144883 13653 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:17.153254  6548 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:17.153682  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:17.135445  6501 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:20:17.048262 14114 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:17.072376  6404 rpcz_store.cc:269] 0709 12:19:06.198702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:06.199004 (+   302us) service_pool.cc:224] Handling call
0709 12:20:17.016793 (+70817789us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:12.908506 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:12.909271 (+   765us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:12.909332 (+    61us) write_op.cc:435] Acquired schema lock
0709 12:20:12.909370 (+    38us) tablet.cc:662] Decoding operations
0709 12:20:12.909895 (+   525us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:12.910026 (+   131us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:12.910068 (+    42us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:20:12.910426 (+   358us) tablet.cc:701] Row locks acquired
0709 12:20:12.910581 (+   155us) write_op.cc:260] PREPARE: finished
0709 12:20:12.910799 (+   218us) write_op.cc:270] Start()
0709 12:20:12.910963 (+   164us) write_op.cc:276] Timestamp: P: 1783599612910717 usec, L: 0
0709 12:20:12.911020 (+    57us) op_driver.cc:348] REPLICATION: starting
0709 12:20:12.911964 (+   944us) log.cc:844] Serialized 527 byte log entry
0709 12:20:13.941757 (+1029793us) op_driver.cc:464] REPLICATION: finished
0709 12:20:14.139099 (+197342us) write_op.cc:301] APPLY: starting
0709 12:20:14.139207 (+   108us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:14.143806 (+  4599us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:14.143954 (+   148us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:14.214845 (+ 70891us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:14.249347 (+ 34502us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:20:14.249419 (+    72us) write_op.cc:312] APPLY: finished
0709 12:20:14.249719 (+   300us) log.cc:844] Serialized 25 byte log entry
0709 12:20:14.249933 (+   214us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:14.250027 (+    94us) write_op.cc:454] Released schema lock
0709 12:20:14.250087 (+    60us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:14.250594 (+   507us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:14.250634 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1930144,"reported_tablets":24,"spinlock_wait_cycles":1920,"child_traces":[["op",{"apply.queue_time_us":175331,"apply.run_cpu_time_us":5597,"apply.run_wall_time_us":111461,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8299,"num_ops":1,"prepare.queue_time_us":268544,"prepare.run_cpu_time_us":23501,"prepare.run_wall_time_us":293003,"raft.queue_time_us":593866,"raft.run_cpu_time_us":10074,"raft.run_wall_time_us":235889,"replication_time_us":978428,"spinlock_wait_cycles":8448,"thread_start_us":589487,"threads_started":4,"wal-append.queue_time_us":351575,"wal-append.run_cpu_time_us":17488,"wal-append.run_wall_time_us":1994480}]]}
I20260709 12:20:17.156425  6505 raft_consensus.cc:1275] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:20:17.157207  6650 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:17.157579  6473 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:17.167135  6473 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:17.153699  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:17.168818  6509 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 22.
I20260709 12:20:17.159654  7240 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:17.169207  6548 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:17.160413  7238 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:17.117497 14123 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 12:20:17.147068 14014 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:17.176734  6580 raft_consensus.cc:2368] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17.
I20260709 12:20:17.170140  6548 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:17.162671 13653 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:17.172364  6501 raft_consensus.cc:1240] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:20:17.180632  6506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 3134) took 1957 ms. Trace:
I20260709 12:20:17.181425  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 3173) took 1258 ms. Trace:
I20260709 12:20:17.140972 13883 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 12:20:17.165001 13879 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:17.186134  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:17.181928  6577 rpcz_store.cc:276] 0709 12:20:15.923251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:15.924829 (+  1578us) service_pool.cc:224] Handling call
0709 12:20:15.925287 (+   458us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:16.035839 (+110552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:17.181131 (+1145292us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b54004303d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:17.181358 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":252731136}
I20260709 12:20:17.189610 13945 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:17.191606 14108 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:17.192075 14108 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:17.192472 14108 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:17.181041  6506 rpcz_store.cc:276] 0709 12:20:15.223136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:15.223258 (+   122us) service_pool.cc:224] Handling call
0709 12:20:15.259506 (+ 36248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:16.320462 (+1060956us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:17.180397 (+859935us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b540017bbd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:17.180574 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2372170624}
I20260709 12:20:17.161679 14076 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:20:17.185364  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.154451 14115 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:17.204468  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.168581 13946 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:17.183883 14014 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:17.183748 14070 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:17.175385 14091 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:17.183480 14083 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:17.185817 13554 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:17.180536 13710 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:17.197228 14109 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:17.165295 14195 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:17.166842 14222 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:17.198833 14076 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:17.245062  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.224787 14014 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:17.251266  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.252075 13747 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:17.252594 13747 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:17.252980  6501 raft_consensus.cc:1275] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 12:20:17.253959 13901 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:17.254370 13901 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:17.254684 13747 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:17.227597 13947 consensus_queue.cc:260] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:17.255640 13947 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:17.202098 14015 consensus_queue.cc:1048] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 12:20:17.207278 14193 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:17.219825 14075 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:17.229264 13554 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:17.197209 13945 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:17.236707 14109 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:17.228428 14091 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:17.310844  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:17.305540 13554 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:17.254719 13901 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:17.262215 14076 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:17.269901 14115 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:17.225538 13946 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:17.322520  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:17.328222  6500 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22.
I20260709 12:20:17.328851  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:17.329308  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:17.329388  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:17.329977  6656 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:17.330443  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:17.324249  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:17.312691 14109 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:17.312098 14091 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:20:17.336282  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 175951us
I20260709 12:20:17.338642  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:17.293730 14026 consensus_queue.cc:1048] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:20:17.345477  6500 raft_consensus.cc:2393] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 24: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:17.347651  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:17.239807 13710 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:17.270839 14193 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:17.333600  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:17.351210  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:17.310406 14075 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:17.355379  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:17.332093 14201 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:17.353330  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:17.358839  6500 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 23.
I20260709 12:20:17.354194  7238 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:17.358737  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 104287us
W20260709 12:20:17.362473  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 192296us
I20260709 12:20:17.368986  6505 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:17.345538 14053 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:17.370050  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:17.370820  6505 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:20:17.378365  6505 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:20:17.332775 14115 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:17.382659  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:17.372705 14117 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:17.394798  6550 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:17.395579  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.389477  6656 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
I20260709 12:20:17.379861  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:17.392210 14108 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:17.393496  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:17.371227 14069 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:17.398082  6505 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:20:17.400728  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:17.401721  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:17.392006 13710 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:17.262416 13653 consensus_queue.cc:1175] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e7554c39679a41bbbca4659b72ab5c65" responder_term: 18 status { last_received { term: 16 index: 14 } last_committed_idx: 13 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18. Ops: [17.14-17.14]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:20:17.394479 14114 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:17.413744 14248 log.cc:927] Time spent T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 12:20:17.397882 13945 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:17.418824 14247 log.cc:927] Time spent T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
W20260709 12:20:17.420714 14244 log.cc:927] Time spent T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 12:20:17.437323 14222 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:17.437593 14108 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:17.415551 14025 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:17.442559 14256 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:20:17.501883  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.416038 14069 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:17.603261 14257 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:20:17.630821  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 266989us
W20260709 12:20:17.631572  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 233778us
W20260709 12:20:17.642318  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 259724us
W20260709 12:20:17.626114 14249 log.cc:927] Time spent T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.298s	user 0.002s	sys 0.000s
I20260709 12:20:17.645654  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: true dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:17.642334 14257 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:17.710238 14257 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:17.711110 14162 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:17.656126 14226 log.cc:927] Time spent T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 1.113s	user 0.008s	sys 0.006s
I20260709 12:20:17.713356  6653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3696) took 1670 ms. Trace:
I20260709 12:20:17.713771  6653 rpcz_store.cc:276] 0709 12:20:16.043420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:16.149255 (+105835us) service_pool.cc:224] Handling call
0709 12:20:16.149625 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:16.168146 (+ 18521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:16.168201 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:16.171574 (+  3373us) log.cc:844] Serialized 26 byte log entry
0709 12:20:16.325425 (+153851us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:16.325466 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:16.325549 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:17.655450 (+1329901us) spinlock_profiling.cc:243] Waited 3.38 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:20:17.713028 (+ 57578us) raft_consensus.cc:1685] finished
0709 12:20:17.713051 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:17.713289 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18005,"raft.queue_time_us":122429,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":11256,"spinlock_wait_cycles":7775616,"thread_start_us":73913,"threads_started":1,"wal-append.queue_time_us":261637}
I20260709 12:20:17.700142 14193 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:17.649694  7271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:17.716598 14193 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:17.661974  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:17.663412  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:17.665045  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:17.666520  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:17.666595 14014 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:17.667425 14025 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:17.668052  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:17.669385  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:17.672547 14185 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:17.674943  6504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 3302) took 1746 ms. Trace:
I20260709 12:20:17.745872  6504 rpcz_store.cc:276] 0709 12:20:15.928742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:16.543187 (+614445us) service_pool.cc:224] Handling call
0709 12:20:16.543533 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:16.615835 (+ 72302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:16.615869 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:16.615910 (+    41us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:16.615954 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:17.674635 (+1058681us) spinlock_profiling.cc:243] Waited 72.1 ms on lock 0x7b540001bdd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:17.674698 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:17.674868 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":165857152}
I20260709 12:20:17.678650 14075 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:17.680648 14115 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:17.691296  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:17.748063  6505 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 21.
W20260709 12:20:17.694805 14171 consensus_meta.cc:287] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.616s	user 0.007s	sys 0.002s
W20260709 12:20:17.695367 14247 log.cc:927] Time spent T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:20:17.695147 14106 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:20:17.696487  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:17.763260  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.698601 14069 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:17.701288  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 3303) took 1415 ms. Trace:
I20260709 12:20:17.764822  6503 rpcz_store.cc:276] 0709 12:20:16.285895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:16.676912 (+391017us) service_pool.cc:224] Handling call
0709 12:20:16.677257 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:16.732459 (+ 55202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:17.700985 (+968526us) spinlock_profiling.cc:243] Waited 52.9 ms on lock 0x7b54002c38d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:17.701210 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":121557888}
I20260709 12:20:17.710242  7269 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:17.717120 14193 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:17.666115  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.771818  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:17.772644  6654 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 23.
I20260709 12:20:17.745440 14025 consensus_queue.cc:1243] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer f2b06aebe97249a4877d798b8118ece4 log is divergent from this leader: its last log entry 20.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:20:17.735064  6501 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:20:17.737906  6508 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:20:17.756284  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:17.722078 14014 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:17.730320  6474 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:17.797886  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 67683.1us
W20260709 12:20:17.798707  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.799507  6474 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:17.751274 14185 consensus_queue.cc:1243] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer f2b06aebe97249a4877d798b8118ece4 log is divergent from this leader: its last log entry 27.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:20:17.722003 14257 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:17.800316  6474 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:17.801049  7268 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:20:17.804473  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:20:17.809834  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 53101.1us
I20260709 12:20:17.811240  6550 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:17.741851 13554 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:17.792385 13945 consensus_queue.cc:1048] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 12:20:17.827121 13747 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:20:17.795558 14265 consensus_queue.cc:1059] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 12:20:17.833278 14265 raft_consensus.cc:3055] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:17.833659 14265 raft_consensus.cc:740] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:17.834227  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:17.834542 14265 consensus_queue.cc:260] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:17.836019 14265 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:17.806692 14069 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:17.837495  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:17.863889  6503 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:17.864041  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:17.767041 14171 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:17.768249 14259 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.078s
I20260709 12:20:17.841606 14193 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:17.809091 14050 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:17.844718 13554 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:17.806787 14076 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:17.846046 14091 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:17.807458 14120 consensus_queue.cc:1048] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 12:20:17.851013  6550 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:17.767877 14106 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:17.861124  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.838985  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:17.870398  6504 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:17.865041  6508 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:17.858696 14117 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:17.882741  7269 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:20:17.878896 14257 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:17.891039  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:17.891790  6502 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:17.894001  6550 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:17.898715  6503 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
I20260709 12:20:17.899319  6508 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 25.
I20260709 12:20:17.901568  6550 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:17.903898  6504 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
W20260709 12:20:17.894491 14109 consensus_meta.cc:287] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.547s	user 0.006s	sys 0.005s
I20260709 12:20:17.894383 14117 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:17.909227  6550 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:17.911131 14109 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:17.906206 14108 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:17.915581 14109 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:17.916802 14091 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:17.906924 14233 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:17.924721  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:17.925333  6504 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:17.927651  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:20:17.929857  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 129598us
I20260709 12:20:17.931005  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:17.931023  6474 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:17.932466 14259 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:17.933055 14259 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:17.915618 14171 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:17.940541  6504 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 23.
I20260709 12:20:17.940481  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:17.942097 14106 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:17.944075 14076 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:17.928742 14025 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:17.945251 14025 raft_consensus.cc:697] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 20 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:17.946267 14025 consensus_queue.cc:237] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:17.957355  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:17.962550 14131 raft_consensus.cc:3055] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:17.963047 14131 raft_consensus.cc:740] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:17.963881 14131 consensus_queue.cc:260] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:17.964964 14131 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:17.952889 14227 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:17.979259 14259 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:17.981447 13901 consensus_meta.cc:287] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.654s	user 0.008s	sys 0.004s
I20260709 12:20:17.982030 13901 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:17.983716  7272 raft_consensus.cc:2368] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17.
W20260709 12:20:17.992877  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:17.992726  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:17.988094 14195 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:17.995069 14265 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:18.007102  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 673502us
I20260709 12:20:18.008194  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:18.005163  6502 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 23.
I20260709 12:20:18.010049  6622 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:18.008514  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:18.019537  6509 raft_consensus.cc:2393] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 17: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:18.018864  6503 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:20:18.020699 14114 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:20:18.022104  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:18.024231  6579 raft_consensus.cc:1275] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:20:18.025738  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:18.017973 14201 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:18.029711 14201 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:18.030067 14196 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:18.010749 14265 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:18.030647 14196 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:18.032384 14265 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:18.033255 14196 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:18.035526  6475 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:18.038959  6651 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:20:18.039619  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:18.044994  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:18.045397  6509 raft_consensus.cc:2410] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 24 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:20:18.041481  6581 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 13. (term mismatch)
I20260709 12:20:18.046545  6581 pending_rounds.cc:85] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 12
I20260709 12:20:18.046838  6581 pending_rounds.cc:107] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 23.13
I20260709 12:20:18.047134  6581 raft_consensus.cc:2889] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:18.048883  6575 raft_consensus.cc:2393] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 23: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:18.051965  6581 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
W20260709 12:20:18.053864  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 43921.4us
I20260709 12:20:18.054824  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:18.030122 14201 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:18.056006  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:18.056986  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:18.057796 14114 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:18.058426  6658 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:18.059689  6656 pending_rounds.cc:85] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 11
I20260709 12:20:18.059523  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:18.060036  6656 pending_rounds.cc:107] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 12:20:18.040467 14164 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:18.040553 14241 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:18.048684 14222 raft_consensus.cc:3055] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:18.052407 14142 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:18.060324  6657 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:18.063452  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:18.060472  6656 raft_consensus.cc:2889] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:18.063582  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:18.061381 14164 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:18.062065 14196 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:18.062100 14222 raft_consensus.cc:740] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:18.068629 14164 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:18.061786 14241 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:18.071070 13946 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:18.048851 13901 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:18.076725 14222 consensus_queue.cc:260] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:18.077592  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:18.090662  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:18.078824  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:18.063164  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:18.079295  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:18.107910  6583 raft_consensus.cc:2393] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 20: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:18.108457  6649 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:18.078655 14166 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:18.100512  6657 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
I20260709 12:20:18.111620  6623 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:18.112444 14164 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:18.112691 14216 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:18.114718  6623 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:18.116065 14196 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:18.116513 14196 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:18.116814  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:18.071089 14241 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:18.117583  6579 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 23.
I20260709 12:20:18.118580 14241 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:18.078109 14222 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:18.121860  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:18.122473  6580 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 22.
I20260709 12:20:18.116851 14196 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:18.123987  6579 raft_consensus.cc:1275] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 12:20:18.125254  6580 raft_consensus.cc:1240] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:18.127365  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:18.127426  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:18.128100  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:18.128767  6580 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:18.135331  6474 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:18.136912  6474 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:18.139314  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:18.141299  6658 raft_consensus.cc:2393] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 20: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:18.142043  6575 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:20:18.144837 14183 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:18.145318 14183 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:18.145731 14183 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:18.146253 13798 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:18.146593 14142 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:18.147356 14142 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:18.147889  6649 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
I20260709 12:20:18.149327  6474 leader_election.cc:400] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17.
I20260709 12:20:18.149741  6474 leader_election.cc:403] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:18.150900 14193 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17.
W20260709 12:20:18.155159  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 64630.7us
I20260709 12:20:18.157083  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:18.163036  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:18.164824  6473 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:18.080626  7267 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:20:18.119767 14201 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:18.226169  6651 raft_consensus.cc:1275] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 23 index: 11. (term mismatch)
I20260709 12:20:18.226567  6651 pending_rounds.cc:85] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 10
I20260709 12:20:18.173595  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:18.194669  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:18.214488  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:18.166944  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 52316.4us
I20260709 12:20:18.226897  6651 pending_rounds.cc:107] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 12:20:18.171123  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:18.213657  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:18.228379  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 57376.1us
I20260709 12:20:18.228796  6649 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 23.
I20260709 12:20:18.213078  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:18.218298  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:18.230029  7272 raft_consensus.cc:2393] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 20: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:20:18.055225 14092 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.105s
W20260709 12:20:18.225276  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 57900us
I20260709 12:20:18.170845  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:18.232753  6473 leader_election.cc:400] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17.
I20260709 12:20:18.237429  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:18.227329  6651 raft_consensus.cc:2889] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:18.239516  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:18.234193  6473 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:18.173100  6548 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:18.242297  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:18.242913  6650 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:20:18.244431  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:18.245924  6651 raft_consensus.cc:2393] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 25: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:18.246807  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:18.229162  6550 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:18.247798  6651 raft_consensus.cc:2410] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 24 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:20:18.248317  6583 raft_consensus.cc:2393] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 17: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:18.193746  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:18.180809  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:18.164239  6575 raft_consensus.cc:2393] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 21: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:18.249564  6657 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:18.250437  6473 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:18.179873  6580 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 23.
I20260709 12:20:18.249619  6509 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 25.
I20260709 12:20:18.258250  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:18.259052  7270 raft_consensus.cc:2368] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 22. Current term is 23.
I20260709 12:20:18.163888 14114 raft_consensus.cc:2764] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election decision vote started in defunct term 19: won
I20260709 12:20:18.262246  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:18.262768  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 3308) took 1363 ms. Trace:
I20260709 12:20:18.262594  6550 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:18.263985  6507 rpcz_store.cc:276] 0709 12:20:16.899248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:17.164852 (+265604us) service_pool.cc:224] Handling call
0709 12:20:17.165269 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:17.224309 (+ 59040us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:17.224460 (+   151us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:18.017497 (+793037us) spinlock_profiling.cc:243] Waited 1.28 ms on lock 0x7b2c00000c88. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:20:18.021436 (+  3939us) log.cc:844] Serialized 26 byte log entry
0709 12:20:18.046073 (+ 24637us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:18.046113 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:18.262446 (+216333us) spinlock_profiling.cc:243] Waited 57 ms on lock 0x7b54003976d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:18.262501 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:18.262547 (+    46us) raft_consensus.cc:1685] finished
0709 12:20:18.262562 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:18.262696 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3559,"raft.queue_time_us":1888,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":8645,"spinlock_wait_cycles":133944832,"thread_start_us":11675,"threads_started":1,"wal-append.queue_time_us":46912}
I20260709 12:20:18.212466 14193 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:20:18.271639  7268 raft_consensus.cc:2393] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 17: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:20:18.262880  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:18.250379  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:18.272610  7238 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:18.276014  6657 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 25.
I20260709 12:20:18.277642  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:18.285218  6650 raft_consensus.cc:2410] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 29 because replica has last-logged OpId of term: 27 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 13.
I20260709 12:20:18.285748  6473 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:18.287835  6652 raft_consensus.cc:2393] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 23: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:18.277746  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:18.212852 14233 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:18.246632 14256 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:18.291070  7240 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:18.166458 14080 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.105s
I20260709 12:20:18.242156 14075 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:20:18.296130  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 123150us
I20260709 12:20:18.307133  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:18.308269  6506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 3324) took 1165 ms. Trace:
I20260709 12:20:18.263988 14112 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:18.269903 14189 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:18.263216 14241 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:18.280102 14065 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:18.286072 14164 raft_consensus.cc:697] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:18.317376 14257 consensus_queue.cc:1059] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.006s
I20260709 12:20:18.294270  6651 raft_consensus.cc:1217] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Deduplicated request from leader. Original: 21.12->[25.13-28.15]   Dedup: 25.13->[26.14-28.15]
I20260709 12:20:18.277357 14025 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:18.311986  6548 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:18.312317  6506 rpcz_store.cc:276] 0709 12:20:17.142921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:17.194954 (+ 52033us) service_pool.cc:224] Handling call
0709 12:20:17.211740 (+ 16786us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:17.212272 (+   532us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:20:17.943521 (+731249us) spinlock_profiling.cc:243] Waited 2.11 ms on lock 0x7b18003795a0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d99455 00007f28798813c7 00007f2875e939a7 00007f2875ea30bf 00007f2875ea3038
0709 12:20:17.959133 (+ 15612us) log.cc:844] Serialized 14 byte log entry
0709 12:20:18.049607 (+ 90474us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:18.055744 (+  6137us) log.cc:844] Serialized 26 byte log entry
0709 12:20:18.055895 (+   151us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:18.055924 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:18.055986 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:18.294259 (+238273us) raft_consensus.cc:1685] finished
0709 12:20:18.308059 (+ 13800us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:18.308208 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3999,"prepare.queue_time_us":145922,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":940,"raft.queue_time_us":3936,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":15171,"spinlock_wait_cycles":4862336,"thread_start_us":68565,"threads_started":2,"wal-append.queue_time_us":231668}
I20260709 12:20:18.305022 13710 raft_consensus.cc:2764] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election decision vote started in defunct term 22: won
I20260709 12:20:18.286026 14114 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:18.326848  6622 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:18.294123 14252 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:18.304643 14216 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:18.330855  6652 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:20:18.278851 14274 consensus_queue.cc:1048] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.073s
I20260709 12:20:18.259510 14015 consensus_queue.cc:1059] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.063s
I20260709 12:20:18.309314 14070 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:18.358084 14070 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:18.358482 14070 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:18.333088  6651 pending_rounds.cc:85] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 13
I20260709 12:20:18.329468 14265 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:18.359735  6651 pending_rounds.cc:107] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 27.14
I20260709 12:20:18.360061  6651 raft_consensus.cc:2889] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:20:18.335427  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 72945.6us
I20260709 12:20:18.309486 14060 consensus_queue.cc:1048] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:20:18.336225  6548 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:18.308863 14256 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:18.318619 13945 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:18.334702  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:18.285279 14162 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 12:20:18.305527 14115 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 12:20:18.324957 13901 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:18.259922 14123 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 12:20:18.329270 13946 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:18.360177  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 70269.1us
I20260709 12:20:18.338335 14241 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:18.363422  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:18.310695 14233 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:20:18.325268 14075 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:18.343118 14105 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:20:18.377701  6548 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:18.343366 14263 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:18.366321 14069 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:18.349282 14025 raft_consensus.cc:697] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:18.331235 14196 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:18.336210  6502 raft_consensus.cc:1275] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 23 index: 11. Preceding OpId from leader: term: 25 index: 12. (index mismatch)
I20260709 12:20:18.342374 14065 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:18.404428  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 90162us
I20260709 12:20:18.424211  6475 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:18.426155  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:18.427266 14112 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:18.429052 14080 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:18.405267  6549 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:18.406210  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:18.366370 13798 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:18.433470 13798 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:18.435843 13798 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:18.374872 14256 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:18.331466 14284 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:18.440644 14284 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:18.442027 14112 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:18.443176 14284 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:18.370801 14171 raft_consensus.cc:3055] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:18.397884  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:18.351512 14114 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:18.443749 14171 raft_consensus.cc:740] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:20:18.444972  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 47163.2us
I20260709 12:20:18.445165 14171 consensus_queue.cc:260] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:18.445744  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:18.446017 14114 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:18.388825 14026 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:18.446621 14026 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:18.447021 14026 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:18.447338 14195 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:18.448108 14195 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:18.448290  7238 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:18.448505 14252 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:18.448665 14195 raft_consensus.cc:697] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 23 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:18.449784 14195 consensus_queue.cc:237] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:18.452049  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 74434.9us
I20260709 12:20:18.360846 14216 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:18.456720  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:18.455991  6656 raft_consensus.cc:1275] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 12. (term mismatch)
I20260709 12:20:18.457981  6656 pending_rounds.cc:85] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 11
I20260709 12:20:18.458501  6656 pending_rounds.cc:107] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 23.12
I20260709 12:20:18.458899  6656 raft_consensus.cc:2889] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:18.372428 14265 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:18.458047  6653 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:20:18.410456 13901 raft_consensus.cc:697] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 25 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:18.396699 13747 consensus_queue.cc:1048] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 12:20:18.467082  6475 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:18.389787 13945 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:18.472433  6649 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 23. Current term is 24. Ops: [23.12-23.12]
I20260709 12:20:18.488416  6657 raft_consensus.cc:1275] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 12. (term mismatch)
I20260709 12:20:18.488883  6657 pending_rounds.cc:85] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 11
I20260709 12:20:18.461067 13883 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:20:18.459594 14262 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:18.408861 14185 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 12:20:18.489132  6657 pending_rounds.cc:107] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 12:20:18.499584 14196 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:18.461159 14114 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:18.420208 14069 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:18.517580  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 53599.8us
I20260709 12:20:18.519958  6657 raft_consensus.cc:2889] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:18.457585 14285 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:18.411368 13946 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:18.505281  6655 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:20:18.491114 14227 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:18.463135 14189 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 12:20:18.494020 14105 raft_consensus.cc:697] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:18.524897 14114 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:18.525494 14117 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:18.455130 14164 consensus_queue.cc:237] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:18.532538 14201 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:18.536827 14227 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:18.533408 14284 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:18.557344  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 230619us
I20260709 12:20:18.542896 14296 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:18.541440 14091 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:18.567345 14284 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:18.567799 14091 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:18.569499 14284 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:18.569922 14091 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:18.571250  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 166041us
I20260709 12:20:18.570972 14296 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:18.572890  6549 leader_election.cc:400] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 22. Current term is 23.
I20260709 12:20:18.575498 14296 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:18.577255  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:18.580746 13901 consensus_queue.cc:237] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:18.585865 14114 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:18.585508 14105 consensus_queue.cc:237] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:18.588883 14233 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:18.590725 14075 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:18.593794 14216 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:18.599888 14069 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:18.542183 14292 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:18.601217  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:18.603969  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:18.606088  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:18.606302  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:18.613230  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:18.615182  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:18.619961  6549 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:18.491067 14025 consensus_queue.cc:237] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:18.541718 14262 consensus_queue.cc:1048] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 12:20:18.634776  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:18.622920  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:20:18.626075  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 171584us
I20260709 12:20:18.628006  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:18.629024 14117 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:20:18.629310  6549 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:18.638862  6549 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:18.629477 14053 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:18.630189 14292 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:18.560580 14290 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:18.566586 14227 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:18.649967 14290 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:18.637313  6505 raft_consensus.cc:2393] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 23: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:18.612987 14296 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:18.641743  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:18.644774  6548 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:18.644035  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:18.647413 14286 log.cc:927] Time spent T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
W20260709 12:20:18.648546 14289 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
W20260709 12:20:18.632169 14226 log.cc:927] Time spent T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:20:18.656327  6578 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 24.
I20260709 12:20:18.659966 14053 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:18.670110  6548 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:18.669894  6473 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:20:18.685535  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:18.687057  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:18.621723 14075 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 12:20:18.689116  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:18.691732 14241 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:18.661703 14296 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:18.683792 14290 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:18.729014 14065 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:18.674708 14256 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:18.653445 14298 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:18.694600 14123 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:20:18.760205  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 90169.2us
I20260709 12:20:19.123246  6548 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:19.026486 14142 consensus_meta.cc:287] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.879s	user 0.010s	sys 0.011s
W20260709 12:20:19.027050 14183 consensus_meta.cc:287] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.881s	user 0.013s	sys 0.005s
I20260709 12:20:19.027740  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:18.669245 14117 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:19.029338  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:19.097391 13946 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:19.031316  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:19.033097  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:19.033941  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:19.035476  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:19.040163 14053 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:19.040917  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 345366us
I20260709 12:20:19.041571  6575 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:20:19.044549 13947 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:19.150163  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:19.046651 14262 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:19.055130 14106 raft_consensus.cc:3055] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 12:20:19.156450  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 57349.1us
W20260709 12:20:19.058250 14026 consensus_meta.cc:287] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.611s	user 0.015s	sys 0.007s
W20260709 12:20:19.095595  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 54248.2us
W20260709 12:20:19.027235 14166 consensus_meta.cc:287] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.916s	user 0.012s	sys 0.006s
I20260709 12:20:19.109268 13747 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:19.100158 14296 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:19.134543 14189 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:19.121312 14293 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:19.134869 14292 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:19.098541 14060 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:19.135270 14298 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:19.135818 13945 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:19.133036  6622 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:19.144204  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:19.144649  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:19.045311  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:19.182090 14070 consensus_meta.cc:287] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.823s	user 0.011s	sys 0.008s
I20260709 12:20:19.129887 14290 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:19.137744 14183 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:19.128860 14265 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:19.151603 14171 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:19.128785 14142 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:19.121924 14123 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:19.200573  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 67652.8us
I20260709 12:20:19.199889  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:19.195412  6651 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
I20260709 12:20:19.196832  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:19.196039  6649 raft_consensus.cc:2393] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 19: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:19.156320 14106 raft_consensus.cc:740] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:19.186419 14166 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:20:19.155020 14263 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.664s	user 0.020s	sys 0.023s
W20260709 12:20:19.248996 14263 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.020s	sys 0.023s
W20260709 12:20:19.212459  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 63057.1us
I20260709 12:20:19.249398 14263 raft_consensus.cc:697] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:19.220709  6584 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 18.
I20260709 12:20:19.156697 13947 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:19.250483 14263 consensus_queue.cc:237] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:19.251843 13947 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:19.200083 13941 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:20:19.226438  6622 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:19.253453  6584 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:19.196259 14293 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:19.235312  6504 raft_consensus.cc:1275] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:20:19.185428 14026 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:19.255934 14293 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:19.256006 14112 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:19.256493 14293 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:19.256654 14112 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:19.257001 14293 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:19.258867 14112 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:19.259020 14293 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:19.259722 14241 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:19.196055 14189 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:19.190407 13747 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:19.218365 14318 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:19.224232 14123 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:19.201012 14060 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:19.263177  6582 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:19.238762  6650 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 24.
I20260709 12:20:19.268416 14298 consensus_queue.cc:1059] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:19.270591 14183 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:19.206269 14070 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:19.207453 14290 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:19.237735 14075 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:19.286957  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:19.342743  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 181358us
I20260709 12:20:19.343894  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:19.288275  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:19.289764  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:19.290302  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:19.344784  6500 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:20:19.292908  6475 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:19.339195  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:19.345768  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:19.346416  6504 raft_consensus.cc:2393] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 21: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:20:19.342056  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 179394us
I20260709 12:20:19.344964  6502 raft_consensus.cc:2410] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 26 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
W20260709 12:20:19.347376  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 54544.9us
I20260709 12:20:19.256264 14026 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:19.296531 14296 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:19.354218  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:19.395339  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 48032.1us
I20260709 12:20:19.389832  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:19.396699  6651 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 24.
I20260709 12:20:19.397742  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:19.334362 14288 log.cc:927] Time spent T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:20:19.270141 14292 raft_consensus.cc:2955] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 21 LEADER]: Committing config change with OpId 21.13: config changed from index 6 to 13, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:20:19.390074  6500 raft_consensus.cc:2393] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 18: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:19.398963  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:19.407039  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:19.408486  6651 raft_consensus.cc:2393] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 24: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:19.415513  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:19.419907 14320 consensus_queue.cc:1048] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:20:19.421657  6504 raft_consensus.cc:1240] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:19.423545  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:19.424371  6584 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:20:19.394946 14227 consensus_meta.cc:287] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.741s	user 0.008s	sys 0.009s
I20260709 12:20:19.285454 14241 consensus_queue.cc:1048] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.118s
I20260709 12:20:19.420424 14256 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:19.398811 14183 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:19.445456  6654 raft_consensus.cc:2410] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 26 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 12:20:19.457723  6584 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22.
I20260709 12:20:19.310788 14318 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:19.428280 14142 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:19.311756 14106 consensus_queue.cc:260] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:19.493856  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:19.495293  6584 raft_consensus.cc:2393] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 23: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:19.449855 14183 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:19.517258  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:19.668462  6622 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.503s (SENT)
I20260709 12:20:19.735908  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:20:19.737174  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:19.444010 14227 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:19.514693 14142 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:19.516757 14183 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:19.714643  6657 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.005s	sys 0.009s
W20260709 12:20:19.882573  6657 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.007s	sys 0.009s
W20260709 12:20:19.772362  6507 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.007s	sys 0.008s
W20260709 12:20:19.883394  6507 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.008s	sys 0.008s
I20260709 12:20:19.520038 14069 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:19.874257  6475 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:19.746848 14106 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:19.738524 14065 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:20:19.715256 14262 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.000s	sys 0.014s
W20260709 12:20:19.813452 14308 log.cc:927] Time spent T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.294s	user 0.000s	sys 0.000s
I20260709 12:20:19.926234  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:19.947338  6502 raft_consensus.cc:2368] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 21. Current term is 22.
I20260709 12:20:19.944770  6657 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:20:19.747112 14320 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:19.951853  6505 raft_consensus.cc:2368] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20.
I20260709 12:20:19.978462 14227 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:19.978965 14326 log.cc:927] Time spent T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260709 12:20:19.915550 14241 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:19.979588 14241 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:19.981477 14330 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:19.981696 14241 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:19.982002 14330 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:19.982545 14329 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:19.983052 14329 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:19.983736 14318 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:19.984210 14317 log.cc:927] Time spent T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.563s	user 0.000s	sys 0.001s
I20260709 12:20:19.984288 14318 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:20:19.984935 14339 log.cc:927] Time spent T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:20:19.987128 14227 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:19.946177 14262 raft_consensus.cc:697] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:19.987733 14227 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:19.989475 14262 consensus_queue.cc:237] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:19.993642 14343 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:19.994185 14343 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:19.996049 14227 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:19.999401 14290 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:19.998991 14227 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:20.001222 14227 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:20:19.951999 14289 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 12:20:19.997848 14296 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:19.945853 13747 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:20.002837 14227 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:19.953393 14327 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:19.908699 14142 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:20.084924  6548 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.813s	user 0.000s	sys 0.009s
W20260709 12:20:20.094116  6548 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.822s	user 0.000s	sys 0.010s
I20260709 12:20:20.095482  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:19.774199  6584 raft_consensus.cc:2368] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 21. Current term is 22.
W20260709 12:20:20.075773  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 407063us
I20260709 12:20:20.097025  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:20.097216  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 164424us
I20260709 12:20:20.098917  7238 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:20.101918  6577 raft_consensus.cc:1275] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 23 index: 11. Preceding OpId from leader: term: 25 index: 12. (index mismatch)
I20260709 12:20:20.104761  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:20.108390  7268 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:20.031159 14060 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:20.084931 14070 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:20.100035 13747 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.114347 13747 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:20.081637 14321 log.cc:927] Time spent T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.624s	user 0.001s	sys 0.000s
I20260709 12:20:20.130434  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:20:20.086961  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 212758us
I20260709 12:20:19.925384 14069 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:20.136283  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3734) took 1713 ms. Trace:
I20260709 12:20:20.107502  6584 raft_consensus.cc:2393] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 25: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:20.135419  7268 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22.
I20260709 12:20:20.148516  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:20.153927  6577 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:20:20.151746  6652 rpcz_store.cc:276] 0709 12:20:18.422858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:18.423765 (+   907us) service_pool.cc:224] Handling call
0709 12:20:18.430743 (+  6978us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:18.471434 (+ 40691us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:18.471484 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:18.471526 (+    42us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:20:18.471557 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:20.136043 (+1664486us) spinlock_profiling.cc:243] Waited 40.5 ms on lock 0x7b54000749d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:20.136094 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:20.136230 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":93115136}
I20260709 12:20:19.952507 14065 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:20.135389 14296 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:20.184929 14296 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:20.185985 14142 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:20.101598 14327 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:20.187287 14296 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:19.970343 14320 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:20.128484 14351 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:20.190301 14320 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:20.190402 14351 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:20.192617 14330 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:20.193882 14346 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.194383 14346 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:19.984962 14318 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:20.199437 14065 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:20.200027 14065 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.117074 14294 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.201428  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 3215) took 1560 ms. Trace:
I20260709 12:20:20.203920 14359 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:20:20.214645  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 75335.8us
I20260709 12:20:20.217593  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:20.221809  6502 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:20:20.169624 14343 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:20.233055  6658 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 23.
I20260709 12:20:20.233409 14343 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.213176  6581 rpcz_store.cc:276] 0709 12:20:18.641387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:18.642310 (+   923us) service_pool.cc:224] Handling call
0709 12:20:18.642597 (+   287us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:20:18.668684 (+ 26087us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:18.668717 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:20:18.757604 (+ 88887us) log.cc:844] Serialized 52 byte log entry
0709 12:20:19.196596 (+438992us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:20:19.196632 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:19.196697 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:20.201135 (+1004438us) raft_consensus.cc:1685] finished
0709 12:20:20.201166 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:20.201354 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":317163,"raft.queue_time_us":32499,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":12556,"thread_start_us":61481,"threads_started":1,"wal-append.queue_time_us":212176}
I20260709 12:20:20.231933  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:20.235090  6584 raft_consensus.cc:2468] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 11.
I20260709 12:20:20.236621  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:20.245368  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:20.229633  6622 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:20.254819  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:20.171715 13747 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:20.274032  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 52845.8us
I20260709 12:20:20.293349  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:20.219141 14070 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.302752  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:20.232491 14359 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:20.267654 14320 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.226083 14060 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:20:20.306226  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 167399us
I20260709 12:20:20.306311  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:20.314303  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:20.303701  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 167256us
I20260709 12:20:20.324580  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:20.330274  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 44453.7us
I20260709 12:20:20.225696 14065 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:20.332283  6475 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:20.348431  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:20.363189  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:20.369835  6577 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:20:20.372406  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:20.312557 14351 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:20.343385 14329 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:20.387668  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:20.374986  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:20.388317  6657 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:20.311189 14359 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:20.348665 14060 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:20.311039 14070 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:20.227159 14294 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:20.376546  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:20.266291 14343 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:20.369241 14365 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:20:20.382422  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:20.358006  6502 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 25.
I20260709 12:20:20.383497  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:20.420538  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:20.420852  6652 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 19.
I20260709 12:20:20.421335  6502 raft_consensus.cc:2393] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 20: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:20.422091  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:20.343534 14327 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:20.382541  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 57712.7us
I20260709 12:20:20.383175  6474 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:20:20.372305 14292 consensus_meta.cc:287] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.849s	user 0.007s	sys 0.004s
I20260709 12:20:20.408025  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:20.344911 14346 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:20.384063  6658 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:20:20.344751 14320 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:20.463392  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:20.348812 14358 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.101s
I20260709 12:20:20.426414  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:20.427742 14294 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:20.429499 14070 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:20.430406 14183 consensus_meta.cc:287] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.505s	user 0.008s	sys 0.003s
I20260709 12:20:20.432525 14362 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.467200 14362 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:20.468912 14358 consensus_queue.cc:1059] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:20:20.469830 14362 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:20.470224 14294 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:20.409250 14293 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:20.470675 14183 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:20.470822 14294 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:20.471823  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:20.473446 14183 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:20.472098  6581 raft_consensus.cc:2393] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 25: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:20.473362  6577 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 25.
I20260709 12:20:20.474982  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:20.473969 14294 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:20.476583  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:20.451337  6657 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:20:20.476224  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:20.480135  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:20.480747  6657 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:20:20.481429  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:20.481657  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:20.423478  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:20.482328  6657 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:20:20.479538  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:20.393323 14361 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:20.536222  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:20.541579  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:20.502734  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:20.541810  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:20.523723  6575 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 3213) took 1882 ms (client timeout 2236 ms). Trace:
I20260709 12:20:20.637962  6584 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:20:20.487211  6577 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:20:20.486447  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:20.569533  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:20.558223  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:20.640542  6504 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:20:20.641037  7238 leader_election.cc:400] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 21. Current term is 22.
W20260709 12:20:20.637617  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 260720us
I20260709 12:20:20.571753  6501 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 12:20:20.641439  7238 leader_election.cc:403] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:20:20.641909  6506 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:20:20.502621  6658 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:20.636950  6509 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:20.504117  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 126526us
I20260709 12:20:20.526130  6653 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3746) took 1008 ms. Trace:
W20260709 12:20:20.639810  6575 rpcz_store.cc:269] 0709 12:20:18.640835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:18.641493 (+   658us) service_pool.cc:224] Handling call
0709 12:20:20.523498 (+1882005us) spinlock_profiling.cc:243] Waited 72 ms on lock 0x7b5400298e54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:20.523644 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":165566720}
I20260709 12:20:20.650166  6506 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:20:20.636802  6581 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 23.
W20260709 12:20:20.515919 14323 log.cc:927] Time spent T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.505s	user 0.001s	sys 0.001s
I20260709 12:20:20.649184  6653 rpcz_store.cc:276] 0709 12:20:19.518222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:19.774477 (+256255us) service_pool.cc:224] Handling call
0709 12:20:19.774850 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:20.204731 (+429881us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:20.204765 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:20.204812 (+    47us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:20.204846 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:20.525900 (+321054us) spinlock_profiling.cc:243] Waited 430 ms on lock 0x7b54002a49d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:20.525942 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:20.526074 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":988138880}
I20260709 12:20:20.696357  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:20.706174  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:20.413249 14351 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:20.571586 14363 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.698520  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:20.629316 13746 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.477530 14294 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.211s
I20260709 12:20:20.652148 14256 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 21. Current term is 22.
I20260709 12:20:20.534413 14333 raft_consensus.cc:3055] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:20:20.699334 14336 log.cc:927] Time spent T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.529s	user 0.004s	sys 0.000s
I20260709 12:20:20.581835 14185 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:20.698668  6658 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 24.
I20260709 12:20:20.422154 14142 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:20.709816  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:20.709841  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:20.706868  6653 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:20:20.751271  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:20.710932  6509 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
W20260709 12:20:20.528762 14331 log.cc:927] Time spent T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.374s	user 0.000s	sys 0.001s
W20260709 12:20:20.596415 14259 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.109s	user 0.005s	sys 0.032s
W20260709 12:20:20.580590 14123 consensus_meta.cc:287] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 1.246s	user 0.010s	sys 0.003s
I20260709 12:20:20.760630 14065 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:20.753253 14185 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:20.760776 14123 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:20.746114 13746 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:20.763074 14123 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:20.711148  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 273695us
I20260709 12:20:20.483290  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:20.763841  6501 raft_consensus.cc:2955] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Committing config change with OpId 21.13: config changed from index 6 to 13, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:20:20.765147  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:20.765658  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 1298) took 1688 ms. Trace:
I20260709 12:20:20.766208  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:20.766975 14106 consensus_meta.cc:287] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.816s	user 0.006s	sys 0.005s
I20260709 12:20:20.766073  6655 rpcz_store.cc:276] 0709 12:20:19.077137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:19.084992 (+  7855us) service_pool.cc:224] Handling call
0709 12:20:19.086672 (+  1680us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:19.095592 (+  8920us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:19.095617 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:19.100958 (+  5341us) log.cc:844] Serialized 480 byte log entry
0709 12:20:19.234787 (+133829us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:19.234855 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:19.234939 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:20.765407 (+1530468us) raft_consensus.cc:1685] finished
0709 12:20:20.765428 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:20.765589 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":97857,"threads_started":1,"wal-append.queue_time_us":360054}
I20260709 12:20:20.767509  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:20.734707  6651 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.022s	sys 0.023s
I20260709 12:20:20.715844  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:20.768445  6651 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.023s	sys 0.023s
I20260709 12:20:20.768743  7267 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:20:20.768078  6657 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 23.
I20260709 12:20:20.769678  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 3220) took 1571 ms. Trace:
I20260709 12:20:20.770116  6578 rpcz_store.cc:276] 0709 12:20:19.197761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:19.201018 (+  3257us) service_pool.cc:224] Handling call
0709 12:20:19.201390 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:19.225292 (+ 23902us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:19.225327 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:19.225880 (+   553us) log.cc:844] Serialized 26 byte log entry
0709 12:20:19.429389 (+203509us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:19.429445 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:19.429552 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:20.769456 (+1339904us) raft_consensus.cc:1685] finished
0709 12:20:20.769485 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:20.769623 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23413,"raft.queue_time_us":44889,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":13984,"thread_start_us":203030,"threads_started":1,"wal-append.queue_time_us":726453}
I20260709 12:20:20.540639 14362 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:20.774679  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:20.743948  6577 raft_consensus.cc:1275] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:20:20.521020 14060 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:20.650369 14320 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:20.530685 14069 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:20.738010 14333 raft_consensus.cc:740] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:20.504400 14293 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:20.711679 14351 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:20.792797 14351 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:20.794206 14379 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.020s
I20260709 12:20:20.769910 14142 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:20.798885 14142 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:20.749265 14363 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:20.763132 14185 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:20.740289  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:20.767889  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:20.760105  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 64659.4us
I20260709 12:20:20.773626  6474 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:20:20.760381 14259 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.285s	user 0.008s	sys 0.044s
I20260709 12:20:20.538236 14227 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:20.749570  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 312075us
I20260709 12:20:20.804402  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:20.781256  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:20.694569 14361 raft_consensus.cc:697] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 23 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:20.787777  6653 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:20:20.790787 14333 consensus_queue.cc:260] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:20.791455 14293 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:20.791742 14343 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:20.779829 14162 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.572s	user 0.032s	sys 0.028s
I20260709 12:20:20.800549 14381 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:20.802636  6506 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 25.
I20260709 12:20:20.705003 14081 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:20.803900  6655 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 25.
I20260709 12:20:20.805655  6652 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:20:20.806552  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:20:20.862051  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 55576.4us
I20260709 12:20:20.807097 14360 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.784106 14216 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:20.858680  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 3117) took 2230 ms. Trace:
I20260709 12:20:20.863529  6622 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:20.781916 14362 raft_consensus.cc:697] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 22 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:20.866011  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:20.867377  7270 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
W20260709 12:20:20.862792 14162 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.655s	user 0.032s	sys 0.028s
I20260709 12:20:20.869920  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:20.870627  6509 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
W20260709 12:20:20.871122 14342 log.cc:927] Time spent T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 12:20:20.874701  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:20.875294  6509 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:20:20.877113  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:20.877866  6509 raft_consensus.cc:2393] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 24: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:20.880597  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 3189) took 1657 ms. Trace:
I20260709 12:20:20.800560 14377 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.085s
I20260709 12:20:20.890046 14337 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:20.890487 14337 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:20.891268 14337 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:20.881017  6507 rpcz_store.cc:276] 0709 12:20:19.222800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:19.223398 (+   598us) service_pool.cc:224] Handling call
0709 12:20:19.236595 (+ 13197us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:19.236982 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:19.237021 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:19.237548 (+   527us) log.cc:844] Serialized 26 byte log entry
0709 12:20:19.942724 (+705176us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:19.942793 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:19.942919 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:20.880397 (+937478us) raft_consensus.cc:1685] finished
0709 12:20:20.880419 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:20.880540 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37,"raft.queue_time_us":22513,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":86,"thread_start_us":620194,"threads_started":1,"wal-append.queue_time_us":1336535}
I20260709 12:20:20.865705  6656 rpcz_store.cc:276] 0709 12:20:18.626428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:18.626685 (+   257us) service_pool.cc:224] Handling call
0709 12:20:18.627006 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:18.651948 (+ 24942us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:18.651979 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:18.652481 (+   502us) log.cc:844] Serialized 26 byte log entry
0709 12:20:18.681823 (+ 29342us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:20:18.681875 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:20.807269 (+2125394us) spinlock_profiling.cc:243] Waited 24.6 ms on lock 0x7b54001b1054. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:20.807346 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:20.807418 (+    72us) raft_consensus.cc:1685] finished
0709 12:20:20.807438 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:20.856964 (+ 49526us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5714,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":17395,"spinlock_wait_cycles":56583424,"thread_start_us":26490,"threads_started":1,"wal-append.queue_time_us":24626,"wal-append.run_cpu_time_us":1946,"wal-append.run_wall_time_us":1034381}
I20260709 12:20:20.884378  6623 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:20.883615  6475 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:20.888311 14361 consensus_queue.cc:237] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:20.894958  6475 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:20.897475  6475 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:20.898370 14123 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:20.898365  6475 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:20.898757 14060 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:20.899050 14123 raft_consensus.cc:697] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:20.899255 14060 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:20.899586 14131 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:20.899720 14060 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:20.900079 14131 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:20.900594 14131 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:20.899986 14123 consensus_queue.cc:237] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:20.903949  6475 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:20.914273 14350 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:20.914901 14350 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:20.915325 14350 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:20:20.946839  6399 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46184 (request call id 120) took 54066 ms (client timeout 15000 ms). Trace:
I20260709 12:20:20.896225 14290 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:20.950394 14290 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:20.894102 14197 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:20.952450  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:20.952848 14197 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:20.955477 14197 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:20.955440 14131 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:20.892248 14360 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:20.958554 14131 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:20.862350 14381 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:20.958587 14360 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:20.961920  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 77657.2us
I20260709 12:20:20.963325  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:20.965368  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:20.965597  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:20.966769  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:20.967391  6653 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:20.967623  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:20.970189  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:20.970759  6655 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 23.
I20260709 12:20:20.970954 14383 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:20.971076 14381 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:20.968688  6475 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:20.976816  6653 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:20:20.980469 14131 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:20.894331 14343 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:20.960068 14293 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:20.986191  6475 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:20.869664 14216 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:20.866757 14081 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:20.980728 14065 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:20.992547 14382 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:20.984359 14330 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:20:20.991948 14359 consensus_meta.cc:287] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.530s	user 0.008s	sys 0.003s
I20260709 12:20:21.000782  6475 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:20.993407 14293 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:20.996979 14294 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:20.971901 14383 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:21.006573  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:21.005275 14227 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:21.013330  7270 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:20:20.997228 14290 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:21.012394  6475 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:20.999912 14065 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:21.004192 14320 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:21.002489 14330 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:21.010191 14293 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:21.018771 14183 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:20:20.947398  6399 rpcz_store.cc:269] 0709 12:19:26.880853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:26.939618 (+ 58765us) service_pool.cc:224] Handling call
0709 12:19:27.794604 (+854986us) spinlock_profiling.cc:243] Waited 36.8 ms on lock 0x7b5400002210. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d650b2 00007f287bcde759 00007f287bc76a76 00007f287bc47d58 00007f287bd4604e 00007f287a2072a4 00007f287a207232 00007f287a207161 00007f287a2070dc
0709 12:19:28.340064 (+545460us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b5400002210. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d65640 00007f287bc3c846 00007f287bd4690e 00007f287a2072a4 00007f287a207232 00007f287a207161 00007f287a2070dc 00007f287a206392 00007f28741c448c 00007f28741c38c6
0709 12:20:20.946746 (+52606682us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:17.900740 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:17.930446 (+ 29706us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:17.930479 (+    33us) write_op.cc:435] Acquired schema lock
0709 12:20:17.930499 (+    20us) tablet.cc:662] Decoding operations
0709 12:20:17.932796 (+  2297us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:17.932885 (+    89us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:17.932928 (+    43us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:20:17.933169 (+   241us) tablet.cc:701] Row locks acquired
0709 12:20:17.933193 (+    24us) write_op.cc:260] PREPARE: finished
0709 12:20:17.933369 (+   176us) write_op.cc:270] Start()
0709 12:20:17.933498 (+   129us) write_op.cc:276] Timestamp: P: 1783599617933317 usec, L: 0
0709 12:20:17.933546 (+    48us) op_driver.cc:348] REPLICATION: starting
0709 12:20:17.934316 (+   770us) log.cc:844] Serialized 527 byte log entry
0709 12:20:18.108529 (+174213us) op_driver.cc:464] REPLICATION: finished
0709 12:20:18.292641 (+184112us) write_op.cc:301] APPLY: starting
0709 12:20:18.292749 (+   108us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:18.293922 (+  1173us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:18.302699 (+  8777us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:18.357703 (+ 55004us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:18.357902 (+   199us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:20:18.357949 (+    47us) write_op.cc:312] APPLY: finished
0709 12:20:18.358142 (+   193us) log.cc:844] Serialized 25 byte log entry
0709 12:20:18.358298 (+   156us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:18.358384 (+    86us) write_op.cc:454] Released schema lock
0709 12:20:18.358422 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:18.358828 (+   406us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:18.358856 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3364,"reported_tablets":24,"spinlock_wait_cycles":108480128,"child_traces":[["op",{"apply.queue_time_us":169229,"apply.run_cpu_time_us":3795,"apply.run_wall_time_us":66213,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8299,"num_ops":1,"prepare.queue_time_us":432808,"prepare.run_cpu_time_us":48780,"prepare.run_wall_time_us":122000,"raft.queue_time_us":47154,"raft.run_cpu_time_us":14628,"raft.run_wall_time_us":150314,"replication_time_us":174822,"thread_start_us":565823,"threads_started":4,"wal-append.queue_time_us":110573,"wal-append.run_cpu_time_us":33651,"wal-append.run_wall_time_us":1337281}]]}
I20260709 12:20:21.044486  6623 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:21.026696 14346 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:21.051519 14350 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:21.073925  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:21.075904  6475 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:21.052234 14081 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:21.052402 14065 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:21.054811  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:21.057760  6505 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 3351) took 1766 ms (client timeout 1633 ms). Trace:
I20260709 12:20:21.004081 14363 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:21.077984  6475 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:21.027933 14330 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:20:21.080940  6505 rpcz_store.cc:269] 0709 12:20:19.291162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:19.343636 (+ 52474us) service_pool.cc:224] Handling call
0709 12:20:21.057475 (+1713839us) spinlock_profiling.cc:243] Waited 607 ms on lock 0x7b540039fb54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:21.057662 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1396430208}
I20260709 12:20:21.085085  6475 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:21.062163 14183 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:21.090272  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:21.075570 14346 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:21.015506 14294 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:20.996791 14362 consensus_queue.cc:237] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:21.062214 14320 raft_consensus.cc:697] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:21.006465 14359 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:21.075762 14388 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:21.075330 14382 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:21.095366  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:20:21.095973  6654 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58980 (request call id 3742) took 1946 ms (client timeout 1926 ms). Trace:
I20260709 12:20:21.092381 14363 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:21.079751 14069 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:21.083678 14290 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:21.104660  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:21.090374 14340 log.cc:927] Time spent T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.561s	user 0.000s	sys 0.005s
I20260709 12:20:21.090323 14081 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:20:21.111114  6654 rpcz_store.cc:269] 0709 12:20:19.149718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:19.149863 (+   145us) service_pool.cc:224] Handling call
0709 12:20:21.095767 (+1945904us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b54003e8ad4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:21.095909 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":676662528}
I20260709 12:20:21.116768  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:21.092132 14183 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:21.097507 14346 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:21.119985  6655 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 25.
I20260709 12:20:21.140770  7268 raft_consensus.cc:2410] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
W20260709 12:20:21.141646  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 63648.4us
I20260709 12:20:21.143373  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:21.109820 14388 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:21.124625 14363 raft_consensus.cc:697] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:21.125073 14081 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:21.156450 14350 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:21.161262 14388 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:21.186224 14383 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:21.187932  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:21.188308 14359 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:21.193388 14392 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:21.119094 14382 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:21.198626 14293 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:21.173724 14081 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:21.198880 14337 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:21.310935 14293 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:21.310923 14382 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:21.312705 14337 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:21.216802  6579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 3211) took 2822 ms. Trace:
I20260709 12:20:21.314162  6579 rpcz_store.cc:276] 0709 12:20:18.394255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:18.394437 (+   182us) service_pool.cc:224] Handling call
0709 12:20:18.394743 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:18.527573 (+132830us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:20:18.527623 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:18.528260 (+   637us) log.cc:844] Serialized 26 byte log entry
0709 12:20:18.570372 (+ 42112us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:18.607019 (+ 36647us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:21.216297 (+2609278us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b540013f954. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:21.216381 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:21.216431 (+    50us) raft_consensus.cc:1685] finished
0709 12:20:21.216445 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:21.216713 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14205,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":5995,"spinlock_wait_cycles":304409344,"thread_start_us":41733,"threads_started":1,"wal-append.queue_time_us":109948,"wal-append.run_cpu_time_us":1994,"wal-append.run_wall_time_us":1067073}
W20260709 12:20:21.218655 14374 log.cc:927] Time spent T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.692s	user 0.000s	sys 0.001s
I20260709 12:20:21.219659 14351 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:21.233573 14363 consensus_queue.cc:237] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:21.186735  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 3120) took 2026 ms. Trace:
I20260709 12:20:21.263015 14392 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:21.208983 14320 consensus_queue.cc:237] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:21.292500 14383 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:21.327638 14294 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:21.329550 14318 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:21.263283 14216 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:21.343731  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:21.344132  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:21.347078  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:21.359151  6653 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:21.339396  6649 rpcz_store.cc:276] 0709 12:20:19.160713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:19.229859 (+ 69146us) service_pool.cc:224] Handling call
0709 12:20:19.230227 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:19.239243 (+  9016us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:19.239290 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:19.239790 (+   500us) log.cc:844] Serialized 26 byte log entry
0709 12:20:19.453051 (+213261us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:19.453116 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:19.453184 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:21.047615 (+1594431us) spinlock_profiling.cc:243] Waited 40.4 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:20:21.186391 (+138776us) raft_consensus.cc:1685] finished
0709 12:20:21.186421 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:21.186667 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":92,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":8214,"spinlock_wait_cycles":92874496,"thread_start_us":188605,"threads_started":1,"wal-append.queue_time_us":1277773}
I20260709 12:20:21.364679  6473 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:21.339087 14290 raft_consensus.cc:697] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 25 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:21.363616 14293 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:21.378926 14293 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:21.355798 14383 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:21.363667 14069 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:21.391662  6473 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:21.396072 14131 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:21.372922 14352 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:21.387233 14383 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:21.421021  6653 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:20:21.443411 14351 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:21.433322 14142 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:20:21.482764 14060 consensus_meta.cc:287] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.509s	user 0.010s	sys 0.002s
I20260709 12:20:21.485241 14346 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:21.401796 14069 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:21.494577 14069 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:21.490795 14060 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:21.497247 14398 log.cc:927] Time spent T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:20:21.498848 14346 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:21.497241 14060 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:21.506709 14131 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:21.518126 14392 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:21.539383 14293 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:21.481395 14290 consensus_queue.cc:237] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:21.543845  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 1270) took 2345 ms. Trace:
I20260709 12:20:21.517450 14142 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:21.544260  6580 rpcz_store.cc:276] 0709 12:20:19.198786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:19.235845 (+ 37059us) service_pool.cc:224] Handling call
0709 12:20:19.236070 (+   225us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:19.274427 (+ 38357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:19.274473 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:19.274528 (+    55us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:20:19.274572 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:21.543593 (+2269021us) spinlock_profiling.cc:243] Waited 38.1 ms on lock 0x7b54004204d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:21.543638 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:21.543786 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":87687552}
I20260709 12:20:21.546463 14142 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:21.547220  7268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38558 (request call id 1102) took 1171 ms. Trace:
I20260709 12:20:21.547605  7268 rpcz_store.cc:276] 0709 12:20:20.376106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:20.376253 (+   147us) service_pool.cc:224] Handling call
0709 12:20:21.547006 (+1170753us) spinlock_profiling.cc:243] Waited 728 ms on lock 0x7b54004bbc54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:21.547134 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1673654656}
I20260709 12:20:21.549058 14183 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:21.551734 14183 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:21.552145 14065 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:21.584888  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 3214) took 2943 ms. Trace:
I20260709 12:20:21.586200 14065 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:21.585340  6583 rpcz_store.cc:276] 0709 12:20:18.641114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:18.641247 (+   133us) service_pool.cc:224] Handling call
0709 12:20:18.641650 (+   403us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:20:18.668466 (+ 26816us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:18.668502 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:20:18.699139 (+ 30637us) log.cc:844] Serialized 52 byte log entry
0709 12:20:19.341257 (+642118us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:19.341850 (+   593us) log.cc:844] Serialized 14 byte log entry
0709 12:20:21.584581 (+2242731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:21.584661 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:21.584699 (+    38us) raft_consensus.cc:1685] finished
0709 12:20:21.584709 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:21.584835 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2309430,"prepare.queue_time_us":169,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":2269611,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":14665,"thread_start_us":199745,"threads_started":1,"wal-append.queue_time_us":618314,"wal-append.run_cpu_time_us":2880,"wal-append.run_wall_time_us":1219122}
I20260709 12:20:21.589967  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:21.590696  7274 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:21.601819  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:21.557350 14388 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:21.594899 14397 log.cc:927] Time spent T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.460s	user 0.005s	sys 0.011s
W20260709 12:20:21.609639  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 1.84005e+06us
I20260709 12:20:21.577998 14330 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:21.610623  6550 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:21.613420  7274 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:20:21.618928  7268 raft_consensus.cc:1240] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:21.619482  7238 leader_election.cc:400] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for earlier term 21. Current term is 22.
I20260709 12:20:21.645350 14081 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:21.660729 14316 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:21.661643 14316 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:21.664008  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 53485.8us
I20260709 12:20:21.665189 13653 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:21.665373  6548 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:21.665704 13653 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:21.666134 13653 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:21.667985 14120 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:21.667286  6548 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:21.647532  7270 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:20:21.673151 14298 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:21.677403 13653 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:21.674986 14120 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:21.679401 13653 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:21.684777  6654 raft_consensus.cc:1240] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:21.649991 14065 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:21.685715  6653 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 20. Current term is 21. Ops: []
W20260709 12:20:21.686245  6651 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.018s	sys 0.041s
I20260709 12:20:21.689520  6654 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 12:20:21.690577  6651 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.018s	sys 0.041s
W20260709 12:20:21.689615  6656 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.010s	sys 0.053s
W20260709 12:20:21.691376  6656 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.010s	sys 0.053s
I20260709 12:20:21.691856  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:21.692646  6654 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:21.692623  6651 raft_consensus.cc:1240] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:21.680506 14120 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:21.697544 14316 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:21.696671 14065 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:21.736644 14388 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:21.757184 14383 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:21.760864 14383 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:21.795405  6622 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.020s	sys 0.034s
W20260709 12:20:21.796399  6622 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.020s	sys 0.035s
W20260709 12:20:21.797178  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 653775us
I20260709 12:20:21.798784  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:21.800644  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:21.802982  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 135706us
I20260709 12:20:21.807538  6652 raft_consensus.cc:2393] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 22: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:21.804065  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:21.816713  6548 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:21.826541 14290 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:20:21.853255  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 453658us
I20260709 12:20:21.827162 14290 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:21.862149 14065 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:21.864466 14081 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:21.882170 14015 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:21.865437 14330 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:21.866326 13710 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:21.867087 14383 raft_consensus.cc:2764] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:20:21.860491 13653 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:21.870836 14395 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:21.871677 13946 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:21.877187 13947 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:21.883291 14015 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:21.891898 13710 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:21.893157  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 3346) took 2607 ms. Trace:
I20260709 12:20:21.893687  6508 rpcz_store.cc:276] 0709 12:20:19.285114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:19.285248 (+   134us) service_pool.cc:224] Handling call
0709 12:20:19.285708 (+   460us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:19.516582 (+230874us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:20:19.744799 (+228217us) log.cc:844] Serialized 14 byte log entry
0709 12:20:20.303439 (+558640us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:20.303510 (+    71us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:20.303561 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:21.654355 (+1350794us) spinlock_profiling.cc:243] Waited 8.9 ms on lock 0x7b1800377e60. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:20:21.892488 (+238133us) spinlock_profiling.cc:243] Waited 230 ms on lock 0x7b540017cad4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:21.892574 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:21.892861 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72289,"prepare.queue_time_us":429553,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":80279,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":42960,"spinlock_wait_cycles":550421504,"thread_start_us":490310,"threads_started":1,"wal-append.queue_time_us":616948}
I20260709 12:20:21.897460  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 1017) took 1409 ms. Trace:
I20260709 12:20:21.897950  6501 rpcz_store.cc:276] 0709 12:20:20.488214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:20.642045 (+153831us) service_pool.cc:224] Handling call
0709 12:20:20.642452 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:20.763728 (+121276us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:20:20.777601 (+ 13873us) log.cc:844] Serialized 14 byte log entry
0709 12:20:20.867346 (+ 89745us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:20.867421 (+    75us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:20:20.867472 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:21.896845 (+1029373us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b54001e64d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:21.896914 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:21.897230 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":79,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":1210,"spinlock_wait_cycles":278389504,"thread_start_us":89268,"threads_started":1,"wal-append.queue_time_us":119522,"wal-append.run_cpu_time_us":1175,"wal-append.run_wall_time_us":1001179}
I20260709 12:20:21.899956 14015 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:21.900960 14171 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:21.902143  6502 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 3194) took 1688 ms (client timeout 2054 ms). Trace:
I20260709 12:20:21.901953 14171 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:21.902655  6502 rpcz_store.cc:269] 0709 12:20:20.213912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:20.423234 (+209322us) service_pool.cc:224] Handling call
0709 12:20:21.901664 (+1478430us) spinlock_profiling.cc:243] Waited 652 ms on lock 0x7b54001e69d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:21.901919 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1499907456}
I20260709 12:20:21.904879 14233 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:21.905725 14233 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:21.908324 14259 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:21.909680  6582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 1267) took 3265 ms. Trace:
I20260709 12:20:21.909502 14259 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:21.912097 14262 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:21.912019 13946 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:21.915165 13746 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:21.886098 14290 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:21.910092  6582 rpcz_store.cc:276] 0709 12:20:18.643975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:18.644913 (+   938us) service_pool.cc:224] Handling call
0709 12:20:18.650598 (+  5685us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:19.309938 (+659340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:21.909007 (+2599069us) spinlock_profiling.cc:243] Waited 612 ms on lock 0x7b54000352d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:21.909427 (+   420us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1408315648}
I20260709 12:20:21.889094 14395 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:21.919564 14233 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:21.925417 14171 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:21.926466 13946 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:21.927211  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 122941us
I20260709 12:20:21.928540  6548 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:21.929785 13947 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:21.938808 14065 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:21.938344 14395 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:21.943429 14026 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:21.945529 14316 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:21.945688 14026 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:21.946529 14316 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:21.948812  6650 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 3116) took 3348 ms (client timeout 2078 ms). Trace:
I20260709 12:20:21.951831 14294 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:21.952297 14298 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:20:21.949333  6650 rpcz_store.cc:269] 0709 12:20:18.600711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:18.600894 (+   183us) service_pool.cc:224] Handling call
0709 12:20:21.944517 (+3343623us) spinlock_profiling.cc:243] Waited 629 ms on lock 0x7b54002a3ad4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:21.948612 (+  4095us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1445965568}
I20260709 12:20:21.953476 14298 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:21.949746 14383 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:21.955236 14383 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:21.957593  6658 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 3137) took 1565 ms (client timeout 1527 ms). Trace:
I20260709 12:20:21.952839 14294 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:21.958240  6658 rpcz_store.cc:269] 0709 12:20:20.392034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:20.463118 (+ 71084us) service_pool.cc:224] Handling call
0709 12:20:21.957095 (+1493977us) spinlock_profiling.cc:243] Waited 28.6 ms on lock 0x7b54003787d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:21.957333 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65742592}
I20260709 12:20:21.943117 13947 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:21.942015 14265 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:21.962177 14026 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:21.969836 14262 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:21.970904 13946 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:21.972066 14233 consensus_queue.cc:1059] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:21.973050  6576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 3221) took 2774 ms. Trace:
I20260709 12:20:21.972628 14065 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:21.973495  6576 rpcz_store.cc:276] 0709 12:20:19.198024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:19.201717 (+  3693us) service_pool.cc:224] Handling call
0709 12:20:19.214605 (+ 12888us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:19.240581 (+ 25976us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:19.240613 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:21.560619 (+2320006us) spinlock_profiling.cc:243] Waited 1.05 ms on lock 0x7b2c00010c48. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:20:21.561059 (+   440us) log.cc:844] Serialized 26 byte log entry
0709 12:20:21.850424 (+289365us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:20:21.850564 (+   140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:21.850743 (+   179us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:21.972423 (+121680us) raft_consensus.cc:1685] finished
0709 12:20:21.972457 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:21.972841 (+   384us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17794,"raft.queue_time_us":53019,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"spinlock_wait_cycles":2412928,"thread_start_us":288450,"threads_started":1,"wal-append.queue_time_us":288930}
I20260709 12:20:21.949671 14263 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:21.977463 14263 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:21.962116  7240 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:21.985198  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 48932.2us
I20260709 12:20:21.986001 14259 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:21.991561 14241 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:21.991928 14298 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:21.977469  6654 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:20:21.988616 14065 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:21.980254 14263 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:22.008067 14105 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.003s
I20260709 12:20:21.989075 14294 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:22.015985  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:21.992256 14241 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:22.021018 14404 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:22.024436 14092 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:22.031155 14263 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:22.032204 14025 raft_consensus.cc:3055] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:21.999050 14395 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:22.009963 14298 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:22.045298 14263 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:22.045636 14404 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:22.048084 14263 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:22.042364 14025 raft_consensus.cc:740] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:22.011579 14383 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:22.061085 14395 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:22.061275  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:22.062438  6576 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:20:22.063432 14241 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:22.064591 14105 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:22.065552 14105 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:22.040496 14092 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:22.069526  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:22.048126 14404 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:22.054559 14316 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:22.075958  6654 raft_consensus.cc:1275] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:20:22.073192 14092 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:22.050179 14420 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:22.100664  6649 raft_consensus.cc:2393] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 21: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:22.112036  6650 raft_consensus.cc:2955] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Committing config change with OpId 21.13: config changed from index 6 to 13, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:20:22.116858 14025 consensus_queue.cc:260] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:22.118826 14025 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:22.127825  6654 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 23. Current term is 24. Ops: [23.12-23.12]
I20260709 12:20:22.132133  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:22.128973  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:22.134188  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:22.135314  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:22.130100  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:22.136672  6649 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
W20260709 12:20:22.127861 14367 log.cc:927] Time spent T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 1.597s	user 0.001s	sys 0.005s
I20260709 12:20:22.118295 14420 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:22.144999 14420 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:22.146310 14189 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:22.147457  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:22.149452 14290 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:22.139940  6404 catalog_manager.cc:5697] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 8 to 16, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 16 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:22.153065 14105 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:22.153203  6404 catalog_manager.cc:5697] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:22.160425  6404 catalog_manager.cc:5697] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 13 to 17, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 17 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:20:22.164834  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:22.165603 14316 consensus_queue.cc:1059] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:20:22.166893 13710 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:22.168915  6655 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:22.170502 14333 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:22.170432  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:22.158953  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 71782us
I20260709 12:20:22.166762  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:22.154042  6548 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:22.166724 14316 raft_consensus.cc:3055] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:22.172060 13946 raft_consensus.cc:3055] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:22.174103 14316 raft_consensus.cc:740] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:22.174654 13946 raft_consensus.cc:740] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:22.180891  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:20:22.183571  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:22.188011  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:22.188555  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:22.183717  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:22.184530  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:22.184890  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:22.179903  6656 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:20:22.186035  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:22.186913  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:22.187502  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:22.097816 14262 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:22.199106  6580 raft_consensus.cc:2410] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
W20260709 12:20:22.199867  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 112263us
I20260709 12:20:22.190164  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:22.208319  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:22.209156  6652 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 15.
I20260709 12:20:22.210711  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:22.211398  6576 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:22.214712  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:22.215683  6582 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:20:22.220786  7267 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:20:22.221168  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:22.222043  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:22.223477 14233 consensus_queue.cc:1059] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:22.210059  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:22.096719 14298 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:22.163208  6404 catalog_manager.cc:5697] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 17 to 20. New cstate: current_term: 20 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:22.233695  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:22.238771  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:22.239346  6653 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:22.240007  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:22.225546 14290 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:22.229991 14233 raft_consensus.cc:3055] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:22.245067  6658 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:20:22.250339  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:22.252244 14117 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:22.253175  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:22.247220  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:22.254096  6584 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:20:22.255841  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:22.256227  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:22.259816  6656 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:22.176482 14316 consensus_queue.cc:260] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:22.264570  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:22.272526  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:22.248073 14233 raft_consensus.cc:740] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:22.262686 14262 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:22.278918  6651 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:22.280290  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:22.280509 14316 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:22.278314  6548 leader_election.cc:400] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20.
I20260709 12:20:22.284409  6548 leader_election.cc:403] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:22.285344  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:22.285847  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 57897.4us
I20260709 12:20:22.288220  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:22.288301  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:22.289917  6575 raft_consensus.cc:2368] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 20. Current term is 21.
I20260709 12:20:22.291016  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:22.291304  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:22.292068  6581 raft_consensus.cc:2393] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 21: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:22.293001  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:22.293447  6582 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:20:22.362841  6578 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:22.283622  6577 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:20:22.334494  7268 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:22.366871 14117 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20.
I20260709 12:20:22.380527  6651 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 24.
I20260709 12:20:22.381008  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:22.331615 14424 log.cc:927] Time spent T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
W20260709 12:20:22.368388 14292 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.666s	user 0.038s	sys 0.043s
I20260709 12:20:22.387655  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:22.334242 14262 raft_consensus.cc:697] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:22.364326  6404 catalog_manager.cc:5697] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 14 to 16, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 16 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:20:22.390985 14292 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.855s	user 0.038s	sys 0.045s
I20260709 12:20:22.405349  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:22.406693  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:22.335391 13710 consensus_queue.cc:1059] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
W20260709 12:20:22.407327  7267 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38558 (request call id 1108) took 1591 ms (client timeout 2055 ms). Trace:
I20260709 12:20:22.424625  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:22.423998  6404 catalog_manager.cc:5697] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 18 to 23, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 23 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:22.427559  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:22.429517  6404 catalog_manager.cc:5697] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 12 to 23. New cstate: current_term: 23 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } }
I20260709 12:20:22.435133 13710 raft_consensus.cc:3055] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:22.435786 13710 raft_consensus.cc:740] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:22.437633  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:22.441823  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:22.442384 13710 consensus_queue.cc:260] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:22.420955  7267 rpcz_store.cc:269] 0709 12:20:20.803492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:20.803638 (+   146us) service_pool.cc:224] Handling call
0709 12:20:22.376309 (+1572671us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b54005bde54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:22.395125 (+ 18816us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3255013120}
I20260709 12:20:22.444653 14233 consensus_queue.cc:260] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:22.444041 13710 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:22.446099 14233 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:22.448722  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:22.449749  6502 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:20:22.452015  6404 catalog_manager.cc:5697] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 12 to 21, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 21 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:22.308597 13946 consensus_queue.cc:260] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:22.465104  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:22.465991  6508 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 26.
I20260709 12:20:22.464548  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:22.474766  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:20:22.482308  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 48810.2us
I20260709 12:20:22.491778  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:22.493243  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:22.494176  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:22.495952  6651 raft_consensus.cc:1275] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:20:22.472131  6404 catalog_manager.cc:5697] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 18 to 20, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 20 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:22.500190  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:22.500998  6582 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:20:22.500773  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:22.498993  6583 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 25.
I20260709 12:20:22.502130  6575 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 23.
I20260709 12:20:22.506000  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:22.488722 13946 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:22.495000  6550 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:22.536046  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:22.513592  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:22.554029  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:22.512965  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:22.551054  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 68542.3us
W20260709 12:20:22.561352  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 76058.2us
I20260709 12:20:22.555534  6577 raft_consensus.cc:2376] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20. Re-sending same reply.
I20260709 12:20:22.574147  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:20:22.579144  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 537433us
I20260709 12:20:22.508615 14292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:48498 (request call id 15) took 5503 ms. Trace:
I20260709 12:20:22.553658 14298 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:22.568953  6404 catalog_manager.cc:5697] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 16 to 19, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 19 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:22.600011  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:22.552264 14189 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:20:22.613515  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 151357us
I20260709 12:20:22.633818  6577 raft_consensus.cc:1275] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:20:22.645073  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:22.657037  6583 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:20:22.659440 14185 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:22.659921  6550 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:22.638247 14265 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:22.660214 14185 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:22.671414  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:22.671610  6501 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 23.
I20260709 12:20:22.672263  6502 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:22.673113 14185 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:22.673656  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 230354us
I20260709 12:20:22.675189 14195 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:22.675851 14195 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:22.678176  6550 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:22.679126 14195 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:22.679752 14117 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:22.680166 14117 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:22.680554 14117 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:22.680497 13710 consensus_queue.cc:1059] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:22.682310 14109 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:22.682688 14109 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:22.683044 14109 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:22.671407 14189 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:22.595499 14292 rpcz_store.cc:276] 0709 12:20:16.990705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:16.990852 (+   147us) service_pool.cc:224] Handling call
0709 12:20:17.079675 (+ 88823us) log.cc:844] Serialized 480 byte log entry
0709 12:20:17.998087 (+918412us) spinlock_profiling.cc:243] Waited 2.99 ms on lock 0x7b18003e69e0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:20:22.485612 (+4487525us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10993,"raft.queue_time_us":407152,"raft.run_cpu_time_us":907,"raft.run_wall_time_us":24347,"spinlock_wait_cycles":6883584,"thread_start_us":283877,"threads_started":2,"wal-append.queue_time_us":189095,"wal-append.run_cpu_time_us":17928,"wal-append.run_wall_time_us":1772408}
I20260709 12:20:22.686525 14265 raft_consensus.cc:697] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:22.687927 14265 consensus_queue.cc:237] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:22.670933 14337 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:22.717231 14189 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:22.718333  6393 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c22b6d97ae844b7195c5e9c024203d48 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:20:22.572593 14262 consensus_queue.cc:237] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:22.738608  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 71812.9us
I20260709 12:20:22.740792 14306 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.113s
I20260709 12:20:22.760748 14353 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:22.761494 14353 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:22.763968 14353 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:22.764467 14423 log.cc:927] Time spent T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.446s	user 0.006s	sys 0.000s
W20260709 12:20:22.764463  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 80999.2us
I20260709 12:20:22.766311 14109 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:22.767237  6475 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:22.768504 14081 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:22.769147 14109 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:22.771466 14092 consensus_meta.cc:287] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.663s	user 0.004s	sys 0.009s
I20260709 12:20:22.773283  6404 catalog_manager.cc:5697] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 12 to 24, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 24 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:22.774453  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:22.778388  6653 consensus_meta.cc:287] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.513s	user 0.003s	sys 0.007s
I20260709 12:20:22.778820  6653 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 24.
I20260709 12:20:22.789361  7268 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:20:22.722956 14337 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:20:22.810076 14337 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:22.821452  6578 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 25.
W20260709 12:20:22.786928 14025 consensus_meta.cc:287] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.667s	user 0.011s	sys 0.006s
W20260709 12:20:22.829870  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 75638.6us
I20260709 12:20:22.772166 14092 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:22.844316  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:22.844316  6502 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22.
I20260709 12:20:22.845582  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:22.860248  7270 raft_consensus.cc:2393] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 20: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:20:22.861073  6475 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:22.900285  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 53602.1us
I20260709 12:20:22.854759  6404 catalog_manager.cc:5697] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 16 to 20, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 20 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:20:22.912161  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:22.914974  6501 raft_consensus.cc:2393] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 22: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:22.894642 14081 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:22.905263  6549 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:22.937748  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:22.938572 14290 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:22.938726 14337 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:22.939469  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:22.939352 14290 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:22.954844  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:22.962327 14290 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:22.939364 14337 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:22.971782 14337 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:22.965343 14353 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:22.972685 14353 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:22.973178 14353 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:22.967255 14399 consensus_queue.cc:1048] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:22.938628  6501 raft_consensus.cc:2468] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:20:22.988128  6475 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:22.995632  6576 consensus_meta.cc:287] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.784s	user 0.016s	sys 0.000s
I20260709 12:20:22.996258  6576 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 24.
W20260709 12:20:23.003104  6656 consensus_meta.cc:287] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.704s	user 0.008s	sys 0.003s
W20260709 12:20:23.003423 14233 consensus_meta.cc:287] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.557s	user 0.012s	sys 0.001s
W20260709 12:20:22.970307  6622 leader_election.cc:336] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.503s (SENT)
I20260709 12:20:23.007575  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:20:22.971881  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:22.940320  6506 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:20:23.012567  6404 catalog_manager.cc:5697] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:20:23.027479  6582 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22.
I20260709 12:20:23.003650  6656 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:20:23.008925 13554 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:23.027881 14109 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:23.033943  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:22.995806 14117 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:23.039222  6404 catalog_manager.cc:5697] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 15 to 24, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 24 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:23.041260 14233 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:23.042026 14092 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:23.023733 14290 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:23.048528  6581 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:20:23.031315 14399 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:23.056447 13554 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:23.061625 14109 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:23.062989  6506 raft_consensus.cc:2393] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 25: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:20:23.065219  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 370076us
I20260709 12:20:23.068651  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:23.059463 14330 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:23.035029 14081 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:23.075044  6404 catalog_manager.cc:5697] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 18 to 21, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 21 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:23.083029  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:20:23.084524  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 114296us
I20260709 12:20:23.072798 14290 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:23.084824 14330 raft_consensus.cc:697] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:23.080052 14353 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:23.138738  6506 raft_consensus.cc:2393] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 22: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:20:23.114653 14316 consensus_meta.cc:287] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.830s	user 0.012s	sys 0.002s
I20260709 12:20:23.116179  6404 catalog_manager.cc:5697] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
W20260709 12:20:23.147179  6622 leader_election.cc:336] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.514s (SENT)
I20260709 12:20:23.147698  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:23.082067 14399 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:23.138787 14092 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:23.162091  6579 raft_consensus.cc:1240] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 20. Current term is 21. Ops: []
W20260709 12:20:23.172789  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 321390us
I20260709 12:20:23.115717 14290 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:23.142889 14233 consensus_queue.cc:1059] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.031s
I20260709 12:20:23.176266 13554 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:23.194991 14431 log.cc:927] Time spent T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.254s	user 0.003s	sys 0.004s
I20260709 12:20:23.195407 14233 raft_consensus.cc:3055] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:23.195806 14290 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:23.195906 14233 raft_consensus.cc:740] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:23.196588 14290 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:23.197333 14233 consensus_queue.cc:260] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.16, Last appended by leader: 16, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:23.199524 14353 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:23.200001 14233 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:23.200277 14353 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:20:23.200824 14430 log.cc:927] Time spent T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.266s	user 0.000s	sys 0.002s
I20260709 12:20:23.194996  6503 raft_consensus.cc:1275] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
W20260709 12:20:23.204319  6623 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.009s	sys 0.021s
W20260709 12:20:23.204770  6623 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.010s	sys 0.021s
I20260709 12:20:23.202294 14117 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:23.176558 14092 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:23.214761  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.206301  6404 catalog_manager.cc:5697] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 17 to 19, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 19 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
W20260709 12:20:23.215792  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 226054us
I20260709 12:20:23.216427  6581 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:20:23.197180 14290 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:23.226691  6623 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:23.233413  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:23.215065 14081 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:23.248499  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.255919  6576 raft_consensus.cc:1240] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:23.255251 14265 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:23.261112  6404 catalog_manager.cc:5697] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 18 to 22, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 22 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:23.229362 14117 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:23.262738  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.237707 14316 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:23.261739 14265 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:23.263182  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.261129  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:23.267719  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.269346  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:23.270984  6651 raft_consensus.cc:2393] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 20: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:23.272985  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.270478  6575 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:20:23.287345  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.237845 14337 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:23.289731  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:23.297068  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:23.301760  6577 raft_consensus.cc:2410] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 24 because replica has last-logged OpId of term: 23 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 12:20:23.305822  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.307083  6581 raft_consensus.cc:2410] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 12:20:23.311538  6578 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 20.
I20260709 12:20:23.311918  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:23.319408  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.332198  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:23.334543  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:23.396962  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 141411us
I20260709 12:20:23.225353 14330 consensus_queue.cc:237] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:23.288455 14298 raft_consensus.cc:3055] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Stepping down as leader of term 19
W20260709 12:20:23.403823  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 100146us
I20260709 12:20:23.407558  6501 raft_consensus.cc:2393] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 26: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:23.402634  6650 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 24.
I20260709 12:20:23.407819  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.404093  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:23.411631  6652 raft_consensus.cc:1240] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:23.419224  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.420115  6577 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
I20260709 12:20:23.410266  6475 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:23.411029  6404 catalog_manager.cc:5697] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 16 to 22. New cstate: current_term: 22 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:23.425081  6653 raft_consensus.cc:2393] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 24: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:23.428094  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:23.428608  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.430016  6649 raft_consensus.cc:1240] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:23.288511 14316 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:23.438582  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:23.444352  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:23.444298  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 138964us
I20260709 12:20:23.410997 14298 raft_consensus.cc:740] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:23.451671  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:23.460299  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.443339  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:23.464744  6404 catalog_manager.cc:5697] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 10 to 21. New cstate: current_term: 21 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
W20260709 12:20:23.468894  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 44208.1us
I20260709 12:20:23.437641 14399 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:23.470958  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.410502 14117 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:23.479355  6578 raft_consensus.cc:2393] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 25: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:23.467895  6404 catalog_manager.cc:5697] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:20:23.486303  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:23.493891  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.549870 14265 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:23.551595 14092 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:23.552909 14436 log.cc:927] Time spent T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.285s	user 0.000s	sys 0.001s
I20260709 12:20:23.479317 14446 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:23.554107  6550 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:23.555203  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 131285us
I20260709 12:20:23.554363 14446 raft_consensus.cc:697] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 24 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:23.463563 14448 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:23.554826  6404 catalog_manager.cc:5697] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:23.568397  6652 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:23.569187  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.562480 14262 raft_consensus.cc:2764] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election decision vote started in defunct term 24: won
I20260709 12:20:23.571503  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:23.571974 14262 consensus_queue.cc:1059] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:23.572827 14117 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:23.573033 14262 raft_consensus.cc:3055] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:23.573444 14262 raft_consensus.cc:740] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:23.573203 14446 consensus_queue.cc:237] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:23.574311 14262 consensus_queue.cc:260] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:23.575486 14262 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:23.579298  6657 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 20. Current term is 21. Ops: [20.12-20.12]
W20260709 12:20:23.580510  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 451356us
W20260709 12:20:23.581243  6622 leader_election.cc:336] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.534s (SENT)
I20260709 12:20:23.581620  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:20:23.582937  6622 leader_election.cc:336] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.565s (SENT)
I20260709 12:20:23.583352  6622 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:23.607465  6579 raft_consensus.cc:2410] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for term 18 because replica has last-logged OpId of term: 17 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 15.
I20260709 12:20:23.598748  6475 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:23.568348 14290 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:23.625501  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 228797us
I20260709 12:20:23.624400  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:23.624042  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 115226us
W20260709 12:20:23.619997 13946 consensus_meta.cc:287] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 1.066s	user 0.014s	sys 0.000s
I20260709 12:20:23.626962  6404 catalog_manager.cc:5697] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:20:23.640448 14290 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:23.616619 14233 consensus_queue.cc:1059] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
W20260709 12:20:23.645260 14447 log.cc:927] Time spent T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 12:20:23.645076 14298 consensus_queue.cc:260] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:23.651844  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 84554.8us
I20260709 12:20:23.652613 14445 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:20:23.653196 14452 log.cc:927] Time spent T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:20:23.653527  6474 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:23.666877  6584 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:20:23.680930  6404 catalog_manager.cc:5697] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 17 to 19, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 19 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:20:23.673250  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 44238.8us
W20260709 12:20:23.678470  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 117759us
W20260709 12:20:23.688644  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:23.692739  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:23.678455 14445 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:23.680100 14109 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:23.691970  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:23.695715  6657 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 24.
I20260709 12:20:23.693384 14290 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:23.696197 14298 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:23.722364  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:23.723160  6651 raft_consensus.cc:2393] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 24: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:23.724963 13946 raft_consensus.cc:3055] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:23.725427 13946 raft_consensus.cc:740] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:23.726331 13946 consensus_queue.cc:260] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:23.696868 14316 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:23.727624 13946 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:23.731683  6582 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 20.
I20260709 12:20:23.734710  6475 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:23.698607  6474 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:23.719172  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:23.690353 14233 consensus_queue.cc:1059] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 12:20:23.737658  6550 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:23.736198  7267 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
I20260709 12:20:23.696985 14445 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:23.740739  6658 raft_consensus.cc:1240] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 23. Current term is 24. Ops: [23.14-23.14]
I20260709 12:20:23.742087 14461 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:23.742583 14461 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:23.742812  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:23.744875 14461 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:23.745368 14461 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:23.745919  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.746892  6582 raft_consensus.cc:2410] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 12:20:23.745811 14461 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:23.748521  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:23.748931 14233 raft_consensus.cc:3055] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:23.749351  6581 raft_consensus.cc:2410] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 24 because replica has last-logged OpId of term: 23 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 11.
I20260709 12:20:23.749357 14233 raft_consensus.cc:740] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:20:23.749761  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:23.750270  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:23.753592  6583 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 20.
I20260709 12:20:23.754492  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.755002  6583 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 24.
I20260709 12:20:23.691648 14449 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:23.757282  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:23.721717 14451 log.cc:927] Time spent T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:20:23.762010  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:23.770413  7267 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:20:23.764580  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:23.764182  7270 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:23.688418 14448 raft_consensus.cc:697] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:23.764788  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:23.767709  6577 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 18.
I20260709 12:20:23.770154  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:23.771088  7268 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
W20260709 12:20:23.753013  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 56555.5us
I20260709 12:20:23.772864 14446 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:23.773027  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:23.773581 14185 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:23.774031 14185 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:23.774173  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:23.774447 14185 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:23.774813  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:23.775424 14446 raft_consensus.cc:697] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:23.775542  7266 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:20:23.776203 14329 consensus_queue.cc:1059] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.011s
I20260709 12:20:23.773170  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.777462  6576 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 22.
I20260709 12:20:23.776412  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:23.778477  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:23.776436 14446 consensus_queue.cc:237] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:23.783761  6475 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:23.784956 14330 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:23.785188  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:23.785501 14330 raft_consensus.cc:697] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 24 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:23.786016  6575 raft_consensus.cc:2410] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 12:20:23.786377 14330 consensus_queue.cc:237] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:23.789264  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.789769  6475 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:23.770130  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 76409.2us
I20260709 12:20:23.790095 14189 consensus_queue.cc:1048] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:23.790658 14440 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:23.790742  6475 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:23.791006  6474 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:23.791422 14440 raft_consensus.cc:697] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 24 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:23.791747 14189 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:23.792694  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:23.793527 14440 consensus_queue.cc:237] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:23.794198  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 3262) took 1301 ms. Trace:
I20260709 12:20:23.794559  6584 rpcz_store.cc:276] 0709 12:20:22.492506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:22.492948 (+   442us) service_pool.cc:224] Handling call
0709 12:20:23.794017 (+1301069us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54001cc354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:23.794119 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2697389952}
I20260709 12:20:23.792896  6506 raft_consensus.cc:1240] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 25. Current term is 26. Ops: [25.16-25.16]
I20260709 12:20:23.795742  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.750511 14233 consensus_queue.cc:260] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:23.797008 14450 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:20:23.797405 14233 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:23.797791  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.798655  6474 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:23.798920  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.799170 14448 consensus_queue.cc:237] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:23.775082  6579 raft_consensus.cc:2393] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 22: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:23.800753 14081 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:23.801003  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:23.801846  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:23.800051  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.801263 14081 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:23.805280 14081 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:23.806622  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:23.806113  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.808079  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:23.808241 14454 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:23.808921  7272 raft_consensus.cc:2410] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 23 because replica has last-logged OpId of term: 20 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 17.
I20260709 12:20:23.808832 14454 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:23.810248  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.810956  6577 raft_consensus.cc:2393] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 25: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:23.811213 14454 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:23.811703  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.800854  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 3245) took 1513 ms. Trace:
I20260709 12:20:23.812232  6580 rpcz_store.cc:276] 0709 12:20:22.287642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:22.287795 (+   153us) service_pool.cc:224] Handling call
0709 12:20:23.800655 (+1512860us) spinlock_profiling.cc:243] Waited 966 ms on lock 0x7b54000352d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:23.800800 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2221456512}
I20260709 12:20:23.812747  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:23.807418 14337 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:23.790012  6581 raft_consensus.cc:2393] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 24: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:23.792273 14189 raft_consensus.cc:697] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:23.786836  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.801882  6579 raft_consensus.cc:2393] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 20: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:20:23.822700  6577 raft_consensus.cc:2410] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
W20260709 12:20:23.822817  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 239905us
I20260709 12:20:23.817378  6576 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22.
I20260709 12:20:23.824430  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:23.824586  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.830420  6578 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:23.832147  6575 raft_consensus.cc:2393] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 17: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:23.768678 14298 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:23.837834  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.838428  6577 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 28.
I20260709 12:20:23.842459  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.843367  6576 raft_consensus.cc:2393] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 20: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:23.844188  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:23.848636  6580 raft_consensus.cc:1275] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
W20260709 12:20:23.857710  6622 leader_election.cc:336] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.623s (SENT)
I20260709 12:20:23.857836  6577 raft_consensus.cc:1275] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:20:23.864388  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.865406  6622 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:23.867081  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.838367 14298 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:23.874146  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:23.874821  6658 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 18.
W20260709 12:20:23.878518  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:23.830200 14262 consensus_queue.cc:1059] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 12:20:23.879989  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.883985  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:23.889429  6502 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 19.
I20260709 12:20:23.891064  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.889299  6575 raft_consensus.cc:2376] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22. Re-sending same reply.
I20260709 12:20:23.894006  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:23.895200  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.896252  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.896544 14439 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:20:23.896791  6581 raft_consensus.cc:2376] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21. Re-sending same reply.
I20260709 12:20:23.898144  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:23.898469  7269 raft_consensus.cc:3055] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:23.898757  6581 raft_consensus.cc:2376] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20. Re-sending same reply.
I20260709 12:20:23.899741  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:23.899873  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.898851  7269 raft_consensus.cc:740] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:23.901646  7269 consensus_queue.cc:260] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:23.906755  6578 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 26.
I20260709 12:20:23.902925  7269 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:23.875187 14298 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:23.874241 14448 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:23.914937  7240 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:23.914696  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:23.904613 14185 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:23.916553 14329 consensus_queue.cc:1059] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 12:20:23.917112 14109 raft_consensus.cc:3055] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:23.917358  7238 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:23.917631 14189 consensus_queue.cc:237] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:23.918292  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:23.917142  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:23.919974 14443 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:23.924868 14262 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:23.924666  6651 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:47858
I20260709 12:20:23.951664  6651 raft_consensus.cc:606] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 25 LEADER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:23.925359 14434 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:23.948076  7273 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:20:23.929939 14252 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:23.930518  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:23.931082  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:23.956660  6582 raft_consensus.cc:2376] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21. Re-sending same reply.
I20260709 12:20:23.935691  6584 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22.
I20260709 12:20:23.936724  6502 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:23.938985  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:23.935843 14298 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:23.948945  7272 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
W20260709 12:20:23.957361  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 99669.4us
I20260709 12:20:23.955744 14434 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:23.961844 13554 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:23.963322  6505 raft_consensus.cc:2393] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 18: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:23.963596 14434 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:23.952574  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:23.955987 14252 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:23.964577 14252 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:23.965008  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:23.966336  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:23.966722  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:23.967870 14333 raft_consensus.cc:3055] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:23.968017  6473 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:23.968305 14333 raft_consensus.cc:740] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:23.955745 14262 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:23.969055  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:23.969329  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:23.970359 14262 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:23.971599 14449 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:23.972474  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:23.972855  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:23.973285  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:23.944124  6575 raft_consensus.cc:2376] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 24. Re-sending same reply.
I20260709 12:20:23.914165 14461 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:23.958086  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.945407  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.962889 14443 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:23.976190  6474 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:23.953562  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.973483  6654 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 24.
I20260709 12:20:23.946370 14109 raft_consensus.cc:740] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:23.974522  6549 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:23.977773  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.985864  6581 raft_consensus.cc:2410] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:20:23.986052  6582 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:20:23.991174  6653 raft_consensus.cc:1275] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:20:23.991374 14185 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:23.991592 14434 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:23.990976  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:23.992017 14434 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:23.992424  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:23.992735 14434 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:23.991407  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.992200  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:23.990854  6549 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:23.996158  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:23.998668  6550 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:24.000063  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 82775.8us
I20260709 12:20:24.003252 14185 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:24.007215  6549 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:24.009402  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:24.010282  6575 raft_consensus.cc:2393] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 26: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:24.010474  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:24.011698  6580 raft_consensus.cc:2410] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:20:24.012737  6576 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:24.018004 13946 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:24.018488 13946 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:24.018930 13946 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:24.021993  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:24.023185  6584 raft_consensus.cc:2393] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 21: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:24.025682  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:24.032052  6622 leader_election.cc:336] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.641s (SENT)
I20260709 12:20:24.033080  6652 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:20:24.033938  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:23.994895 14330 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:24.003993 14185 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:24.038497  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:24.037115  6505 raft_consensus.cc:1240] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:24.035671  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:24.046273  7269 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:20:24.046489  6508 raft_consensus.cc:1240] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:24.040067  6549 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:24.037454 14330 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:24.024459 14233 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:24.032234 14442 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:24.052254  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:24.036485 14448 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:24.057168  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:24.069200  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:24.044701 14092 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:23.992003 14109 consensus_queue.cc:260] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:24.047935 14316 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:24.188287  6503 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 24 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:23.969349 14333 consensus_queue.cc:260] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:24.097558  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 48130.2us
I20260709 12:20:24.051381 14330 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:24.035482 14081 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:24.116988  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 76986.3us
I20260709 12:20:24.190580  6549 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:24.055974 14442 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:24.119410  6653 raft_consensus.cc:1240] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:24.063632 14265 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:24.120059  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:24.071981  6650 raft_consensus.cc:2393] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 26: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:20:24.121438  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:24.108412 14465 log.cc:927] Time spent T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 12:20:24.193929  6549 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
W20260709 12:20:24.187693  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:24.190045  7238 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:24.120708 14471 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:20:24.210141  6549 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:24.214164  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:24.214682  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 175744us
I20260709 12:20:24.221930  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:24.217897  6549 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:24.217828  6503 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:20:24.225347  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:24.216848  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 180951us
I20260709 12:20:24.169411 14092 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:24.229813  6508 raft_consensus.cc:2393] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 25: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:24.191960 14442 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:24.208904 14265 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:24.234658  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:24.233637 14185 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:24.236521  6653 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:47858
I20260709 12:20:24.237021  6650 raft_consensus.cc:1275] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:20:24.237136  6653 raft_consensus.cc:606] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:24.217371 14233 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:24.224030 14262 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:24.212734 14476 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:20:24.238512  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:24.165184 14448 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:24.255465  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:24.233021 14092 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:24.256808  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:24.257210  6500 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 20. Current term is 21. Ops: []
W20260709 12:20:24.272379 14438 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
I20260709 12:20:24.238763 14399 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:24.273339 14399 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:24.275576 14399 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:24.257251  6502 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:24.239254 14265 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:24.278664 14434 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:24.282104 14081 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:24.282171 14448 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:24.287061 14252 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:24.289094 14467 log.cc:927] Time spent T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 12:20:24.291313 14464 log.cc:927] Time spent T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.239s	user 0.001s	sys 0.000s
I20260709 12:20:24.294430 14117 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:24.261188  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 261592us
I20260709 12:20:24.261961  6473 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:24.197028 14316 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:24.248602 14233 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:24.249552 14109 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:24.217242 14445 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:24.308372  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 90507.1us
I20260709 12:20:24.309851  6549 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:24.311236  6549 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:24.300858 14233 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:24.312322  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:24.313746  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:24.316763  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:24.317193 14479 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:24.317770  6576 raft_consensus.cc:2410] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for term 18 because replica has last-logged OpId of term: 17 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 15.
I20260709 12:20:24.317732 14479 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:24.318879 14479 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:20:24.319759 14475 log.cc:927] Time spent T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.062s	user 0.000s	sys 0.003s
I20260709 12:20:24.299069 14298 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:24.321039 14298 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:20:24.304983 14469 log.cc:927] Time spent T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
W20260709 12:20:24.321511  6622 leader_election.cc:336] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.759s (SENT)
I20260709 12:20:24.300609 14337 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:24.321674 14439 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:24.323992 14439 raft_consensus.cc:697] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 20 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:24.327078  6501 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 15. Current term is 16. Ops: []
W20260709 12:20:24.327900  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 66039.3us
I20260709 12:20:24.329223  6473 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:24.331290  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:24.316782 14483 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:20:24.330945 14117 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.019s
I20260709 12:20:24.345002 14252 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:24.346614  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 44986us
I20260709 12:20:24.327586 14081 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:24.330156 14337 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:24.307446 14265 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:24.366310  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:24.369661  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:24.346661 14440 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:24.325042 14439 consensus_queue.cc:237] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:24.374363  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 45276.2us
I20260709 12:20:24.346231 14435 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:20:24.373445  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:24.363495 14337 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:24.382390 14316 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:24.396098 14265 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:24.360652 14189 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.018s
I20260709 12:20:24.398504 14449 consensus_queue.cc:1048] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:24.398399 14233 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:24.400213 14434 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:24.402115 14233 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:24.402809  7274 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:20:24.403820 14185 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:20:24.404557 14185 consensus_queue.cc:1243] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer 496c7c3792cd4587b4dbeba435f4be1e log is divergent from this leader: its last log entry 19.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
W20260709 12:20:24.377228  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 155366us
W20260709 12:20:24.433848  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 112371us
W20260709 12:20:24.448894  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:24.453408  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:24.454552  7268 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:24.457060  6508 raft_consensus.cc:1240] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:24.457482  6505 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:24.371060 14081 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:24.458598  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:24.459936 14081 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:24.460281  6473 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:24.463315  7238 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
W20260709 12:20:24.464769  6622 leader_election.cc:336] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.767s (SENT)
I20260709 12:20:24.466602  6622 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:20:24.466173  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:24.468880 14480 consensus_queue.cc:1059] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 12:20:24.379251 14092 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:24.474089 14480 consensus_queue.cc:1243] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer 496c7c3792cd4587b4dbeba435f4be1e log is divergent from this leader: its last log entry 21.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:20:24.474579 14480 raft_consensus.cc:3055] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:24.474874 14480 raft_consensus.cc:740] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:24.475905 14092 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:24.475998  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:24.475694 14480 consensus_queue.cc:260] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:24.469357 14484 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:24.459813 14330 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:24.485773  6506 raft_consensus.cc:1240] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18. Ops: [17.16-17.16]
I20260709 12:20:24.447438  6474 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:24.469096  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:24.487439  7272 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:20:24.469197  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:24.432123 14443 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:24.476894  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:24.480779  6651 raft_consensus.cc:1275] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:20:24.481048  7273 raft_consensus.cc:2393] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 21: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:20:24.465003 14445 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:24.481237  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:24.494776  7268 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:24.471240 14185 consensus_queue.cc:1048] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 12:20:24.489466  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:24.468963 14449 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:24.481488 14435 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:20:24.487373  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 81509.6us
I20260709 12:20:24.490902  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:24.502239  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:24.502646  6549 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:24.504279  6507 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
W20260709 12:20:24.510666  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 44447.8us
I20260709 12:20:24.493103 14189 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:20:24.512677  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:24.513713 14480 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:24.514657 14442 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:24.496296 14484 raft_consensus.cc:697] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 18 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:24.520490  6650 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:47858
I20260709 12:20:24.521163  6650 raft_consensus.cc:606] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:24.521515  6650 raft_consensus.cc:614] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:20:24.522454 13946 consensus_meta.cc:287] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.503s	user 0.010s	sys 0.001s
I20260709 12:20:24.522879 13946 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:24.523481 14330 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:24.525960  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 61252.9us
W20260709 12:20:24.526907  6622 leader_election.cc:336] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.776s (SENT)
I20260709 12:20:24.527400  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:24.509296 14449 raft_consensus.cc:697] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 25 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:24.528519 14456 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:20:24.529311 14449 consensus_queue.cc:237] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:24.464323 14479 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:24.544471  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:24.510878  6549 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:24.539610  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:24.536818  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:24.546700  6651 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 24.
I20260709 12:20:24.550100  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:24.551438  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:24.551753  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:24.552320  6650 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:20:24.554265  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:24.555214  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:24.557004  6649 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:20:24.531047 14483 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:20:24.558410  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 46545.6us
I20260709 12:20:24.559900  6653 raft_consensus.cc:1275] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:20:24.563200  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:24.549813 14262 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:24.564844  7269 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 22. Current term is 23. Ops: [22.18-22.18]
I20260709 12:20:24.565852  6504 raft_consensus.cc:1240] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 24 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 23. Current term is 24. Ops: []
W20260709 12:20:24.567929  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 99668.9us
I20260709 12:20:24.548578 14189 raft_consensus.cc:697] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:24.571951  6475 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:24.571861  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:24.571487  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:24.574545  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:24.564999 14483 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:24.576224 14480 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:24.581902  6656 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:47858
I20260709 12:20:24.582489  6656 raft_consensus.cc:606] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Received request to transfer leadership to TS 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:24.582854  6656 raft_consensus.cc:614] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:24.583700  6473 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:24.585191 14109 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:24.585657 14109 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:24.587714 14109 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:24.587977  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:24.588248 14442 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:24.588658  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:24.589288  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:24.573014 14292 consensus_queue.cc:1059] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.015s
I20260709 12:20:24.590178  6504 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:20:24.595413 14265 consensus_queue.cc:1048] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:24.596017  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:24.581063  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:24.597677  6506 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 24.
I20260709 12:20:24.599659  6656 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:24.599802 14440 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:24.600800  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:24.599655  6473 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:24.577422 13946 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:24.607396  7272 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 12:20:24.608098  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:24.610849  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:24.613204  6650 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:20:24.613562  6651 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:24.613543  6652 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:20:24.573015 14446 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:24.571619 14262 raft_consensus.cc:697] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:24.619005  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:24.580897 14483 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:20:24.621600  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 50795.7us
I20260709 12:20:24.521000 14484 consensus_queue.cc:237] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:24.622603  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 92915.6us
I20260709 12:20:24.623418  7238 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:24.620790 14445 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.021s
I20260709 12:20:24.641606 14485 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:24.642872  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:24.644819  7238 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:24.645582 14092 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:24.646392 14092 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:20:24.623231  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:24.653748 14337 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:24.654770  6475 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:24.656522 14330 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:24.663786  6550 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:24.656689 14189 consensus_queue.cc:237] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:24.662048  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:24.661414 14445 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:24.651201 13946 consensus_queue.cc:1059] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.019s
I20260709 12:20:24.664178 14479 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:24.671626  6548 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:24.671617 13946 raft_consensus.cc:3055] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:24.672495 13946 raft_consensus.cc:740] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:24.673345  6548 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:24.673261 13946 consensus_queue.cc:260] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:24.674861 13946 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:20:24.669513  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 49461us
I20260709 12:20:24.677410  6473 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:24.680574  6651 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 25.
I20260709 12:20:24.656042 14337 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:24.619184 14440 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:24.699934  6550 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:24.696506 14490 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:20:24.694717 14450 consensus_queue.cc:1059] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.007s
I20260709 12:20:24.703364  6473 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:24.707628 14454 raft_consensus.cc:3055] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:24.708101 14454 raft_consensus.cc:740] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:24.708616 14330 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:20:24.709085 14454 consensus_queue.cc:260] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:24.711302  6475 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:24.713119 14449 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:20:24.713379  6475 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:24.714395 14450 raft_consensus.cc:3055] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:24.714336 14443 consensus_queue.cc:1048] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:20:24.714963 14450 raft_consensus.cc:740] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:24.715884 14450 consensus_queue.cc:260] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:24.717223 14450 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:24.698945 14446 consensus_queue.cc:1059] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 12:20:24.719025 14480 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:24.719517 14480 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:24.719897 14480 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:24.720150 14233 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:20:24.665341 14489 log.cc:927] Time spent T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8: Append to log took a long time: real 0.104s	user 0.001s	sys 0.002s
W20260709 12:20:24.697043  6622 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.878s (SENT) [suppressed 59 similar messages]
I20260709 12:20:24.615382 14109 consensus_queue.cc:1059] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.073s
W20260709 12:20:24.732697  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 87941.7us
I20260709 12:20:24.738360  7238 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:24.721608 14445 raft_consensus.cc:3055] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 23 LEADER]: Stepping down as leader of term 23
W20260709 12:20:24.752599  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:24.739544 14476 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:24.745390 13554 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:24.782600  6655 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:47858
I20260709 12:20:24.757092 14454 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:24.783291  6655 raft_consensus.cc:606] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:24.786752  6655 raft_consensus.cc:614] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:20:24.784399  6622 leader_election.cc:336] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.920s (SENT)
I20260709 12:20:24.660648 14262 consensus_queue.cc:237] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:24.787946  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:24.757684 14445 raft_consensus.cc:740] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:24.789361  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:20:24.789633  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 67480us
I20260709 12:20:24.790724  6584 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:24.789364  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:24.790962  6475 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:24.791817  6583 raft_consensus.cc:2393] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 21: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:24.760777 14265 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:24.798924  6475 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:24.801483 14448 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:24.805915 14265 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:24.806867 14491 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:24.807363 14491 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:24.807750 14491 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:24.793033 14503 consensus_queue.cc:1048] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 12:20:24.811563  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:20:24.813364  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:24.786326 14476 raft_consensus.cc:697] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:24.814383  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:24.814793 14476 consensus_queue.cc:237] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:24.816829 14483 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:24.820274  6548 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:24.822644  6548 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:24.768621 14442 consensus_queue.cc:1059] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 12:20:24.828648  6548 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:24.829214 14483 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:24.833092 14498 log.cc:927] Time spent T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:20:24.840648  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:24.841431  7272 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:24.842947  7273 raft_consensus.cc:1240] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:24.844377  7238 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:24.851819 14442 consensus_queue.cc:1243] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 24.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:20:24.852118 14483 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:24.853026 14445 consensus_queue.cc:260] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:24.854321 14445 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:24.796908 14189 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:24.836496 14443 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:24.855461  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:24.826728 14434 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:24.855764  6548 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:24.856037 14333 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:24.857734  6652 raft_consensus.cc:1275] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:20:24.852051  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:24.808251 14265 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:24.822965 14185 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:20:24.869768  6584 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 23.
I20260709 12:20:24.838114 14399 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:24.872013  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:24.871934 14399 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:24.874171  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:24.874816 14399 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:24.876580  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:24.877185  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:24.877705 14292 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:24.878228 14292 raft_consensus.cc:697] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 23 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:24.879169  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:24.879124 14292 consensus_queue.cc:237] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:24.881561  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:24.883061  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:24.857419 14298 raft_consensus.cc:3055] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:24.842900 14486 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:20:24.888358  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:24.899186  6500 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:24.889233  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:20:24.900806  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 45630.5us
I20260709 12:20:24.826264 14330 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 12:20:24.864720 14439 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:24.918108  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:24.859553 14189 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:24.925338  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:24.916936  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:24.920374  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:24.926949  6502 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:24.928822  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:20:24.929258  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 160119us
I20260709 12:20:24.920780  6548 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:24.922164  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:24.923447  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:24.923558  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:24.944623  6548 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:24.898511 14298 raft_consensus.cc:740] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:24.955704  6508 raft_consensus.cc:1275] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
I20260709 12:20:24.956231  6508 pending_rounds.cc:85] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 11
I20260709 12:20:24.956318  6503 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 23. Current term is 24. Ops: []
W20260709 12:20:24.960214  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 256909us
I20260709 12:20:24.914145 14265 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:24.965255  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:24.965902  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:24.870808 14455 consensus_queue.cc:1059] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.070s
I20260709 12:20:24.970788  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:24.916759 14443 raft_consensus.cc:697] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:24.931165 14189 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:24.956604  6508 pending_rounds.cc:107] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 12:20:24.990648  6508 raft_consensus.cc:2889] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:24.953760 14117 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:20:24.954155 14490 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:24.930032 14434 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.048s
I20260709 12:20:25.002195  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:25.008381  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:25.010303  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 47642us
I20260709 12:20:24.978883 14316 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:25.009692  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:25.025072  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:25.004295  6502 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:20:24.978226 14506 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
W20260709 12:20:25.033255  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 73550.5us
I20260709 12:20:25.048475  6548 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:25.049008  6473 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:25.059365  6548 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:25.064970  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:25.053758 14495 log.cc:927] Time spent T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:20:25.068468  6548 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:25.065948  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 86241.8us
I20260709 12:20:25.073894  6548 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:25.078672  6657 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 23. Current term is 24. Ops: []
W20260709 12:20:25.092633  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 62714.3us
W20260709 12:20:25.096931 14501 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 12:20:25.129809 14265 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:25.085225 14504 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:25.075295 14490 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:20:25.107260 14500 log.cc:927] Time spent T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.059s	user 0.000s	sys 0.002s
I20260709 12:20:25.132624 14399 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:25.133164 14399 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:25.134636  6473 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:25.135485 14399 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:25.135493 14504 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:25.045697 14316 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:25.071328 14476 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:25.111773 14488 raft_consensus.cc:3055] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:20:25.143117 14488 raft_consensus.cc:740] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:25.143574 14445 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:25.144207 14445 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:25.144155 14488 consensus_queue.cc:260] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:25.118232 14262 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:20:25.110891 14480 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:25.120441 14443 consensus_queue.cc:237] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:25.127337  6504 raft_consensus.cc:2410] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 24 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 12:20:25.082928 14483 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:25.158656 14316 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:25.158923 14483 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:25.159812  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:25.161912 14483 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:25.162132 14480 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:25.139935  6473 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:25.164322  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:25.165091  6653 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:20:25.166412  6550 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:25.146466  6622 leader_election.cc:336] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 2.003s (SENT)
I20260709 12:20:25.055326 14439 consensus_queue.cc:1059] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 12:20:25.078639 14298 consensus_queue.cc:260] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:25.154336  7272 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:25.156677  6500 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:20:25.158236 14445 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:25.064783 14233 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:25.170019  6509 raft_consensus.cc:2393] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 25: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:25.167140 14504 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:20:25.074663  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 82759.3us
I20260709 12:20:25.178954 14488 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:25.180140 14490 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:25.215046  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:20:25.234711  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:25.246155  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:25.247872  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:25.248574  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:25.251415  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 177589us
I20260709 12:20:25.251855  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:25.181861 13946 consensus_queue.cc:1059] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 12:20:25.255050  6548 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:25.181758 14506 consensus_queue.cc:1059] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:20:25.222856 14490 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:25.260916  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:25.262470  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:25.263358  6506 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:20:25.273216  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:25.241780 14454 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:25.212498 14491 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:25.290548  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 144111us
I20260709 12:20:25.290930  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:25.298032  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:25.259928 14233 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:25.265328 14490 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:25.313450  6658 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:25.314117  6579 raft_consensus.cc:1240] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:25.324321  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:25.325374  6577 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
W20260709 12:20:25.305227 14450 consensus_meta.cc:287] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.588s	user 0.012s	sys 0.000s
I20260709 12:20:25.337193  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:25.340770  7270 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:25.339138  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:25.342447  6579 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:20:25.317719 14512 log.cc:927] Time spent T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:20:25.344811  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:25.307540 14265 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:25.346015  6622 leader_election.cc:336] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 2.182s (SENT)
I20260709 12:20:25.370390  6576 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:20:25.356385  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:25.371381  6584 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:20:25.370656  6622 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:25.371836  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:25.373095  6649 raft_consensus.cc:2393] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 25: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:25.374449  6577 raft_consensus.cc:2410] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:20:25.356956  6583 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:25.376690  6507 raft_consensus.cc:2410] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 23 because replica has last-logged OpId of term: 22 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 17.
I20260709 12:20:25.376572  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:25.311807 14442 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:25.385965  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 48443.5us
I20260709 12:20:25.422748  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:25.423619  6577 raft_consensus.cc:2376] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22. Re-sending same reply.
I20260709 12:20:25.424163  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:25.424727  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:25.426052  6577 raft_consensus.cc:2393] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 25: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:25.344805 14486 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:25.438129  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:25.455632 14486 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:20:25.437157  6506 raft_consensus.cc:1240] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:25.458561  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:25.462803  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:25.463462  6583 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 25.
I20260709 12:20:25.460459  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:25.464869  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:25.465584  6583 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 24.
I20260709 12:20:25.469270  6582 raft_consensus.cc:2393] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 24: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:25.426175 14521 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:25.448892 14500 log.cc:927] Time spent T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
W20260709 12:20:25.479269  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 95705us
I20260709 12:20:25.377174 14265 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:25.480219  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:25.490185  6583 raft_consensus.cc:2368] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 21. Current term is 22.
I20260709 12:20:25.491762  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:25.481112  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:25.500763  6548 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:25.418648 14442 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:25.504010  6548 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:25.505173  6583 raft_consensus.cc:2393] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 25: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:25.481837 14446 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:25.507792  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:25.512710  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:25.514243  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:25.490159 14521 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:25.488224 14476 consensus_queue.cc:1048] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 12:20:25.533474  6575 raft_consensus.cc:2376] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22. Re-sending same reply.
I20260709 12:20:25.500571 14333 raft_consensus.cc:3055] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:25.559489  6577 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 24.
I20260709 12:20:25.660915  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:25.657990  6658 raft_consensus.cc:1275] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:20:25.547468  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:25.674314  6579 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:20:25.587541  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:25.704135  6507 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 23 index: 18. (term mismatch)
I20260709 12:20:25.707836  6507 pending_rounds.cc:85] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 17
W20260709 12:20:25.558383 14507 log.cc:927] Time spent T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.304s	user 0.000s	sys 0.001s
W20260709 12:20:25.676668 14524 log.cc:927] Time spent T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.143s	user 0.004s	sys 0.000s
I20260709 12:20:25.716023  6507 pending_rounds.cc:107] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 22.18
I20260709 12:20:25.716826  6507 raft_consensus.cc:2889] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:20:25.636431 14540 log.cc:927] Time spent T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:20:25.709863 14486 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:25.711081  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:25.711798  6582 raft_consensus.cc:2410] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:20:25.711675  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:25.651767 14491 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:25.723814  6579 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:20:25.646634  6654 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:20:25.548095 14265 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:25.712859  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:25.549297 14529 log.cc:927] Time spent T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:20:25.507355 14262 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:25.716180  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:25.736650  6503 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 25.
W20260709 12:20:25.653720 14189 consensus_meta.cc:287] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.652s	user 0.000s	sys 0.011s
I20260709 12:20:25.717226  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:25.545599 14298 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:25.504791 14399 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:25.538270 14117 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:25.641533 14442 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:25.673969 14316 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:25.709154  6506 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
W20260709 12:20:25.657058 14526 log.cc:927] Time spent T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
W20260709 12:20:25.541167  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 206169us
I20260709 12:20:25.546689 14435 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:25.579640 14542 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:25.711059 14521 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:25.726431  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:25.727247  6576 raft_consensus.cc:2393] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 17: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:20:25.729518  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:25.705871 14333 raft_consensus.cc:740] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
W20260709 12:20:25.731253 14440 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.003s	sys 0.015s
I20260709 12:20:25.732352  6577 raft_consensus.cc:1240] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:25.541491 14446 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:25.733727 14455 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:25.792650  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:25.791863  6622 connection.cc:256] RPC call timeout handler was delayed by -1.7419205400001374s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1523s for a response
W20260709 12:20:25.806787  6622 connection.cc:256] RPC call timeout handler was delayed by -1.7419205400001374s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1523s for a response
W20260709 12:20:25.807163  6622 connection.cc:256] RPC call timeout handler was delayed by -1.7131205400000908s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1555s for a response
W20260709 12:20:25.807493  6622 connection.cc:256] RPC call timeout handler was delayed by -1.7131205400000908s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1555s for a response
W20260709 12:20:25.807833  6622 connection.cc:256] RPC call timeout handler was delayed by -1.5988205400001334s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16820000000000002s for a response
W20260709 12:20:25.808357  6622 connection.cc:256] RPC call timeout handler was delayed by -1.4109213300000647s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1706s for a response
W20260709 12:20:25.808750  6622 connection.cc:256] RPC call timeout handler was delayed by -1.403520540000045s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1899s for a response
W20260709 12:20:25.809113  6622 connection.cc:256] RPC call timeout handler was delayed by -1.403520540000045s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1899s for a response
W20260709 12:20:25.809468  6622 connection.cc:256] RPC call timeout handler was delayed by -1.3810205400000086s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19240000000000002s for a response
W20260709 12:20:25.809839  6622 connection.cc:256] RPC call timeout handler was delayed by -1.3054205400001138s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2008s for a response
W20260709 12:20:25.810286  6622 connection.cc:256] RPC call timeout handler was delayed by -1.3054205400001138s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2008s for a response
I20260709 12:20:25.802080  6474 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:25.802976 14476 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:25.802767 14538 consensus_queue.cc:1059] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:20:25.806036 14531 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.073s
W20260709 12:20:25.810842  6622 connection.cc:256] RPC call timeout handler was delayed by -1.2262205399999857s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2096s for a response
W20260709 12:20:25.813380  6622 connection.cc:256] RPC call timeout handler was delayed by -1.2084213299999647s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19310000000000002s for a response
W20260709 12:20:25.813823  6622 connection.cc:256] RPC call timeout handler was delayed by -1.2064205400001811s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2118s for a response
I20260709 12:20:25.795728 14117 raft_consensus.cc:697] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:25.754549 14189 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:25.798714 14440 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.003s	sys 0.015s
W20260709 12:20:25.814697  6622 connection.cc:256] RPC call timeout handler was delayed by -1.2064205400001811s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2118s for a response
W20260709 12:20:25.833695  6622 connection.cc:256] RPC call timeout handler was delayed by -1.1850213300001542s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1957s for a response
W20260709 12:20:25.834213  6622 connection.cc:256] RPC call timeout handler was delayed by -1.1850213300001542s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1957s for a response
W20260709 12:20:25.834535  6622 connection.cc:256] RPC call timeout handler was delayed by -1.1542205400000967s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21760000000000002s for a response
W20260709 12:20:25.839457  6622 connection.cc:256] RPC call timeout handler was delayed by -1.1274213300000611s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2021s for a response
W20260709 12:20:25.840103  6622 connection.cc:256] RPC call timeout handler was delayed by -1.1274213300000611s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2021s for a response
I20260709 12:20:25.850876 14265 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:25.851680 14265 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:25.853006 14545 log.cc:927] Time spent T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:20:25.874564  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 72558.3us
I20260709 12:20:25.793444 14316 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:25.814371 14262 raft_consensus.cc:697] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:25.837731 14330 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:25.856083  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:25.872654 14490 consensus_meta.cc:287] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.526s	user 0.010s	sys 0.000s
I20260709 12:20:25.794807 14435 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:25.854823 14443 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:20:25.887993  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 47191.3us
I20260709 12:20:25.889452  6622 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:25.808377 14542 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:25.813962 14476 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:25.872323 14479 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:25.816099 14117 consensus_queue.cc:237] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:25.806083 14455 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:25.897457  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:25.899632  6584 raft_consensus.cc:2376] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22. Re-sending same reply.
I20260709 12:20:25.879045 14333 consensus_queue.cc:260] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:25.910619 14480 raft_consensus.cc:3055] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:25.911087 14480 raft_consensus.cc:740] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:25.921063 14445 raft_consensus.cc:3055] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:25.921713 14445 raft_consensus.cc:740] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:25.922772 14445 consensus_queue.cc:260] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:25.889698 14330 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:25.934338 14330 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:25.940589  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 3405) took 1053 ms. Trace:
W20260709 12:20:25.940961 14548 log.cc:927] Time spent T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:20:25.906149  6622 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:25.942428 14544 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:25.942814  6622 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:25.943599  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:25.945322  6575 raft_consensus.cc:1275] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
I20260709 12:20:25.945664  6575 pending_rounds.cc:85] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 11
I20260709 12:20:25.945886  6575 pending_rounds.cc:107] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 12:20:25.946102  6575 raft_consensus.cc:2889] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:25.942955 14544 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:25.947451 14544 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:25.948105 14448 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:25.948434 14543 raft_consensus.cc:993] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4: : Instructing follower e7554c39679a41bbbca4659b72ab5c65 to start an election
I20260709 12:20:25.949062  6584 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 23. Current term is 24. Ops: [22.12-23.13]
I20260709 12:20:25.949543 14543 raft_consensus.cc:1081] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Signalling peer e7554c39679a41bbbca4659b72ab5c65 to start an election
I20260709 12:20:25.950605  6577 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 22. Current term is 23. Ops: [20.17-22.18]
I20260709 12:20:25.951139 14438 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:25.951687  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:25.944078  6622 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:25.941051  6509 rpcz_store.cc:276] 0709 12:20:24.887379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:24.887489 (+   110us) service_pool.cc:224] Handling call
0709 12:20:25.940362 (+1052873us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b54002c5bd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:25.940529 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":512094592}
I20260709 12:20:25.898803 14399 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:20:25.895968 14546 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:25.983207  6652 raft_consensus.cc:1275] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:20:25.984359  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:25.984985  6656 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:25.985131  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:25.985937  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:25.985316 14535 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:25.987136 14535 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:25.994418  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:25.996366  6584 raft_consensus.cc:1240] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 17. Current term is 18. Ops: []
I20260709 12:20:25.997241  6577 raft_consensus.cc:1275] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:20:25.997924  6584 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:25.998994  6577 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 18 index: 16. (term mismatch)
I20260709 12:20:25.999509  6577 pending_rounds.cc:85] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 15
I20260709 12:20:25.999800  6577 pending_rounds.cc:107] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 17.16
I20260709 12:20:26.002578  6575 raft_consensus.cc:1275] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 14. (term mismatch)
I20260709 12:20:26.003089  6575 pending_rounds.cc:85] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 13
I20260709 12:20:26.003393  6575 pending_rounds.cc:107] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 23.14
I20260709 12:20:26.003690  6575 raft_consensus.cc:2889] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:26.004968  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:26.007879  6582 raft_consensus.cc:1275] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:20:26.014065 14535 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:26.019197  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:26.022544  6550 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:26.024070 14486 consensus_queue.cc:1048] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:26.026494  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:26.029467  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:26.032511 13946 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:26.032961 13946 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:26.033308 13946 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:26.036842  6549 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:26.037658 14491 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:26.038180 14491 raft_consensus.cc:697] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:26.039052 14491 consensus_queue.cc:237] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:26.041957  6549 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:26.000099  6577 raft_consensus.cc:2889] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:26.046010 14536 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:26.044811 13946 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:26.051966 13946 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:26.059141  6649 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:20:25.979365 14546 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:25.924245 14445 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:25.890887 14490 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:26.070825  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 89223.8us
I20260709 12:20:25.911953 14480 consensus_queue.cc:260] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:25.899268 14479 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:25.979206 14399 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:20:26.090737  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 210204us
I20260709 12:20:26.085323  6657 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 12:20:26.208722  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 127198us
I20260709 12:20:26.175335 14535 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:26.179437  6577 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:20:26.182132 14455 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:26.072382 14438 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.032s
I20260709 12:20:26.217286  7274 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:20:26.093499 14399 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:20:26.077175 14546 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:26.165722 14533 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:26.164513 14518 consensus_queue.cc:1059] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.183s
I20260709 12:20:26.093679  6578 raft_consensus.cc:2368] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 21. Current term is 22.
W20260709 12:20:26.207970  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:26.174464 14109 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:26.193338 14542 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:26.221551  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:26.093663 14265 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:26.190179 14446 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:26.192294 14476 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:26.201349 14435 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:26.101292 14262 consensus_queue.cc:237] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:26.254593 14486 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.017s
I20260709 12:20:26.253026 14550 consensus_queue.cc:1059] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 12:20:26.276129 14550 raft_consensus.cc:3055] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:26.278595  6473 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:26.287545  6473 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:26.212311 14443 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 12:20:26.261699 14503 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:26.222453 14535 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:26.265740 14518 consensus_queue.cc:1243] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 16.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:20:26.231878 14440 consensus_queue.cc:1059] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.064s
I20260709 12:20:26.255399 14448 consensus_queue.cc:1048] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 12:20:26.256682 14109 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:26.282011 14533 raft_consensus.cc:697] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:26.317193  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:26.324443  6654 raft_consensus.cc:2410] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 27 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
I20260709 12:20:26.284052 14550 raft_consensus.cc:740] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:26.294576 14446 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:26.352082 14446 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:26.352519 14446 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:26.293836 14558 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:26.355783 14558 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:26.358026  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:26.358201 14558 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:26.343465 14298 consensus_meta.cc:287] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.589s	user 0.007s	sys 0.002s
W20260709 12:20:26.322420 14560 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:20:26.349040  6652 raft_consensus.cc:1275] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:20:26.345360 14329 raft_consensus.cc:3055] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:26.372509 14558 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:26.363430 14479 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:26.320822 14316 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:26.378063 14535 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:26.360046 14450 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:26.310411 14503 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:26.380534 14109 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:26.381831 14550 consensus_queue.cc:260] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:26.363565 14189 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:26.383802 14455 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:26.385555 14478 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:26.388094  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:26.393535  6583 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 24.
I20260709 12:20:26.404026  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:26.437096  7270 raft_consensus.cc:2410] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 27 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
I20260709 12:20:26.392575 14329 raft_consensus.cc:740] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:26.412065  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:20:26.412042  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:26.439709  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:26.432621  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:26.440631  6650 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 24.
I20260709 12:20:26.438376  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 3055) took 1087 ms. Trace:
W20260709 12:20:26.453377  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 388899us
I20260709 12:20:26.459482  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:26.461548  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:26.464903  6508 raft_consensus.cc:1275] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:20:26.467082  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:26.444320  6581 rpcz_store.cc:276] 0709 12:20:25.351346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:25.735886 (+384540us) service_pool.cc:224] Handling call
0709 12:20:25.736218 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:25.798517 (+ 62299us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:25.798585 (+    68us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:25.813222 (+ 14637us) log.cc:844] Serialized 26 byte log entry
0709 12:20:26.079125 (+265903us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:20:26.434645 (+355520us) log.cc:844] Serialized 14 byte log entry
0709 12:20:26.438088 (+  3443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:26.438162 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:26.438204 (+    42us) raft_consensus.cc:1685] finished
0709 12:20:26.438215 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:26.438324 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44170,"raft.queue_time_us":64205,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":18695,"thread_start_us":543173,"threads_started":2,"wal-append.queue_time_us":411705}
I20260709 12:20:26.470767  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:26.476941  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:26.477829  6650 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
I20260709 12:20:26.419979 14490 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:26.479928  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:26.486477  6500 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:26.484400  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:26.487457  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:26.487567  6501 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 26.
I20260709 12:20:26.485775  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:26.489009 14561 log.cc:927] Time spent T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 12:20:26.488826  6576 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 17.
I20260709 12:20:26.488739 14329 consensus_queue.cc:260] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:26.438894 14455 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:26.490346 14329 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:26.492218 14455 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:26.493937 14435 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:26.493574 14455 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:26.494460 14435 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:26.496199 14503 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:26.402215 14558 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:26.510381 14558 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:26.511751 14533 consensus_queue.cc:237] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:26.438526 14450 raft_consensus.cc:697] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 22 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:26.514065 14450 consensus_queue.cc:237] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:26.492622  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:26.524006  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 47251.8us
I20260709 12:20:26.524848  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:26.519776  6503 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 26.
I20260709 12:20:26.527825  6549 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:26.528275  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:26.515478 14560 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:20:26.525678  7269 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:20:26.481699 14550 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:26.468832 14535 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:26.468686 14109 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:26.487334 14316 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:26.552362  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 264831us
I20260709 12:20:26.553346  6473 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:26.533783 14503 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:26.549253  6549 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:26.541200 14435 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:26.443003 14478 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:26.547813  6548 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:26.576939  6473 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:26.557859 14490 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:26.560233 14555 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:26.611788  6473 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:26.611984  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 64254.9us
I20260709 12:20:26.615864  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:26.616705  6650 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 25.
I20260709 12:20:26.623144  6548 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:26.626372  6548 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:26.556535 14535 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:26.586328 14491 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:26.559953 14109 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:26.561817 14316 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:26.641085  6548 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:26.586764 14536 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:20:26.644841  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 169588us
I20260709 12:20:26.615327 14117 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:26.647732  6548 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:26.569427 14189 consensus_queue.cc:1048] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 12:20:26.619463 14562 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:26.619263 14443 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:26.652701  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:26.653861  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:26.622620 14543 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:20:26.655560  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 43846.6us
I20260709 12:20:26.634078 14486 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:26.645637 14491 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:26.655512  7240 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:26.691716 14574 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:26.655028 14262 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:26.670730 14443 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:26.706756  6622 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.022s	sys 0.021s
W20260709 12:20:26.707231  6622 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.022s	sys 0.021s
W20260709 12:20:26.707504  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 629455us
I20260709 12:20:26.710590 14109 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:26.711658  6578 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 3301) took 1397 ms (client timeout 1792 ms). Trace:
I20260709 12:20:26.650517 14490 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:20:26.699674 14564 log.cc:927] Time spent T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 12:20:26.712106  6578 rpcz_store.cc:269] 0709 12:20:25.314392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:25.314524 (+   132us) service_pool.cc:224] Handling call
0709 12:20:26.711447 (+1396923us) spinlock_profiling.cc:243] Waited 755 ms on lock 0x7b540006de54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:26.711603 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1737607808}
I20260709 12:20:26.686882 14543 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:26.713868 14316 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:26.714790 14543 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:26.704581 14262 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:26.711771  6584 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:26.716497 14262 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:26.717181 14399 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:26.719940 14399 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:26.690399 14542 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:26.720788 14542 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:26.721271 14535 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:26.670321 14555 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:20:26.723366 14542 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:26.727902 14480 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:20:26.728590 14547 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:20:26.729283 14478 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:26.632726 14558 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
W20260709 12:20:26.730401  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 74064.8us
I20260709 12:20:26.612807 14503 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:26.691274  7240 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:26.671676 14298 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:26.736860  6622 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:20:26.737825  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 63984.2us
I20260709 12:20:26.747601  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:26.749406  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:26.755261  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 3056) took 1403 ms. Trace:
W20260709 12:20:26.739459  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 267094us
I20260709 12:20:26.775604  6580 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
W20260709 12:20:26.775571  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 84417.9us
I20260709 12:20:26.755877  6583 rpcz_store.cc:276] 0709 12:20:25.351685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:25.513921 (+162236us) service_pool.cc:224] Handling call
0709 12:20:26.754551 (+1240630us) spinlock_profiling.cc:243] Waited 859 ms on lock 0x7b54000578d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:26.755173 (+   622us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1976689920}
I20260709 12:20:26.656531 14446 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:26.690829 14491 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:26.777980  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:26.733520 14518 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:20:26.783192 14546 consensus_meta.cc:287] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.515s	user 0.009s	sys 0.003s
I20260709 12:20:26.784722 14587 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:20:26.785426  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 46754.9us
I20260709 12:20:26.786186 14081 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:20:26.786995 14544 consensus_meta.cc:287] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.839s	user 0.004s	sys 0.005s
I20260709 12:20:26.789202 14555 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:26.792135  6475 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:26.706840 14443 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:26.793216 14570 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:26.858650 14570 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:26.852630 14546 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:26.859189 14570 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:26.859335 14559 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:26.673666 14536 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:26.758102 14541 consensus_queue.cc:1059] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.060s
I20260709 12:20:26.778729 14542 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:26.800047 14503 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:26.800614  6505 raft_consensus.cc:3055] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:26.801931  6622 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:26.807771 14333 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:26.814744  7272 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:20:26.815979  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:26.817080  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:26.826498  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:26.828799 14446 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:26.829062  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:26.829962  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:26.894142  6652 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:26.838125  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:26.874167 14316 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:26.838869  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:26.832487 14518 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:26.895820 14316 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:26.903113 14518 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:26.893288 14587 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:26.893010 14490 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:26.905004 14587 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:26.905530 14587 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:26.905413 14490 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:26.847905  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:26.858798  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 66751.6us
I20260709 12:20:26.860982  7273 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:20:26.796542 14476 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:26.861745  6473 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:26.862306  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:26.911499  6649 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 16.
I20260709 12:20:26.862558  6655 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 26. Current term is 27. Ops: []
I20260709 12:20:26.870507  7270 raft_consensus.cc:1275] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 12:20:26.870927  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 69074.8us
I20260709 12:20:26.870193  6505 raft_consensus.cc:740] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:26.913872  6583 raft_consensus.cc:2393] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 18: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:26.900270  6658 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 26.
I20260709 12:20:26.864846 14189 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:26.916299 14316 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:26.919543 14189 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:26.860683 14546 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:26.915115  6658 raft_consensus.cc:3055] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:26.914930  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:26.853029 14488 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.046s
I20260709 12:20:26.944172  6657 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:26.943591  6658 raft_consensus.cc:740] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:26.928917  6622 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:26.945942  6475 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:26.945739  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:26.951277  6653 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:26.911166 13946 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:20:26.958022  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 96315.2us
I20260709 12:20:26.866431 14550 consensus_queue.cc:1059] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.077s
I20260709 12:20:26.958253  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:26.861640 14544 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:26.945648  6658 consensus_queue.cc:260] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:26.966180  6658 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:26.909322 14479 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:26.947928 14488 consensus_queue.cc:1243] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer 496c7c3792cd4587b4dbeba435f4be1e log is divergent from this leader: its last log entry 19.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:20:26.978325  6623 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:26.979746  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:26.874648 14542 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:26.956691 14399 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:26.957890 14485 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:26.989638  6656 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:26.984805  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:26.980006  6550 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:26.988673  6622 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:26.987028 14541 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:26.993578  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:26.865737 14555 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:26.995021  6550 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:26.980691 14591 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 12:20:27.000048  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:26.989423  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:26.995613  6473 leader_election.cc:400] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 21. Current term is 22.
I20260709 12:20:26.962944 14185 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:20:26.969796 14496 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.058s
I20260709 12:20:26.971294 14490 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:26.982697 14479 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:26.958176 14081 consensus_queue.cc:1048] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 12:20:26.972486 14521 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.022s
I20260709 12:20:26.928268 14476 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:26.999509  6658 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:20:27.027910  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:27.001214  6654 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 24.
I20260709 12:20:27.002507  6550 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:27.049474  7273 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 16.
I20260709 12:20:26.987195 14550 consensus_queue.cc:1243] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer 496c7c3792cd4587b4dbeba435f4be1e log is divergent from this leader: its last log entry 22.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:20:27.036274  6622 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:26.992981 14543 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:20:26.950250 14262 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:20:26.996860 14442 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:26.993592 14541 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:26.962481 13946 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:26.962810 14316 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.040s
I20260709 12:20:27.037003 14265 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:26.986565 14480 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.065s
I20260709 12:20:27.052744  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:27.054059  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:26.997771 14448 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:26.987461 14399 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:20:27.038869  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 48729.7us
I20260709 12:20:26.990163 14189 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:27.039274 14329 consensus_meta.cc:287] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.548s	user 0.011s	sys 0.003s
I20260709 12:20:27.051386  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:27.065382  6474 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:27.072295  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:27.077117  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:27.077754  6654 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 24.
I20260709 12:20:27.080363 14443 raft_consensus.cc:3055] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:27.080793 14443 raft_consensus.cc:740] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:27.081691 14443 consensus_queue.cc:260] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:27.085062 14544 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:26.999275 14485 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:27.085778  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3822) took 1067 ms. Trace:
I20260709 12:20:27.086148  6656 rpcz_store.cc:276] 0709 12:20:26.018238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:26.018380 (+   142us) service_pool.cc:224] Handling call
0709 12:20:26.018815 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:26.991976 (+973161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:27.085528 (+ 93552us) spinlock_profiling.cc:243] Waited 968 ms on lock 0x7b540008bbd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:27.085694 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2225990016}
I20260709 12:20:27.087221 14485 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:27.090154 14588 log.cc:927] Time spent T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 12:20:27.091380 14443 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:27.092231 14518 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:27.094969  7268 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 12:20:27.095338  7268 pending_rounds.cc:85] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9: Aborting all ops after (but not including) 14
I20260709 12:20:27.095544  7268 pending_rounds.cc:107] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 12:20:27.095814  7268 raft_consensus.cc:2889] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:27.066182 14541 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:27.054535 14445 consensus_queue.cc:1048] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 12:20:27.102201  7269 raft_consensus.cc:1217] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Deduplicated request from leader. Original: 20.16->[21.17-21.17]   Dedup: 21.17->[]
I20260709 12:20:27.112485  7269 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 21. Current term is 22. Ops: [21.17-21.17]
I20260709 12:20:27.075485 14399 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:20:27.117141  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 45979.8us
I20260709 12:20:27.128756  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:27.130126  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:27.130946  6579 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 24.
I20260709 12:20:27.146564  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:27.127849  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:27.159442  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:27.162528  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:27.158959  6583 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22.
I20260709 12:20:27.166860  6577 raft_consensus.cc:1240] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:27.172678  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:27.173807  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:27.174474  6506 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 25.
I20260709 12:20:27.174892  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:27.175467  6576 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 23.
I20260709 12:20:27.176834  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:27.173959  6583 raft_consensus.cc:1240] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:27.178354  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:27.160266  6580 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 25.
I20260709 12:20:27.179010  6575 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:27.183033  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:27.183200  6500 raft_consensus.cc:1275] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:20:27.117604 14544 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:20:27.193805  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 128486us
I20260709 12:20:27.192881  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:27.192997  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:27.171912 14586 log.cc:927] Time spent T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.276s	user 0.001s	sys 0.000s
I20260709 12:20:27.195080  6502 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 20.
I20260709 12:20:27.195149  6503 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:27.197070  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:27.161871 14189 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 12:20:27.202437  6504 raft_consensus.cc:2393] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 24: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:27.207115 13946 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:27.212970 14536 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:27.214577  6505 consensus_queue.cc:260] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:27.215282 14449 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:27.215785 14449 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:27.215842  6505 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:27.216296 14449 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:27.221321 14117 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:27.221987 14117 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:27.192801 14185 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:27.224725 14185 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:27.225939  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 184334us
I20260709 12:20:27.227097 14479 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:27.228891 14490 consensus_queue.cc:1048] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:20:27.230589  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 228124us
I20260709 12:20:27.232270  6550 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:27.232604 14476 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:27.233160  6550 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:27.233243 14476 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:27.234997  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:27.239110  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:27.235776  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:27.221875 14518 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:27.236450  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 193714us
I20260709 12:20:27.236604  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:27.237924  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:27.235996  6657 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:27.251662  7238 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:27.252226  6653 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:20:27.242179  6475 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:27.248546  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 212316us
I20260709 12:20:27.182969 14550 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:27.258015  6622 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:27.157740 14559 consensus_queue.cc:1059] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.083s
I20260709 12:20:27.259966  6475 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:27.239365 14476 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:27.239085 14081 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:27.253867 14189 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:20:27.267067  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:27.248173 14543 raft_consensus.cc:697] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:27.255105 14265 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:27.278043  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:20:27.265830 14594 log.cc:927] Time spent T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 12:20:27.259151 14480 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:27.259078 14485 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:27.309798  6505 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 17. (term mismatch)
I20260709 12:20:27.240707 14109 consensus_queue.cc:1059] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 26.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 12:20:27.315459  6505 pending_rounds.cc:85] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 16
I20260709 12:20:27.315774  6505 pending_rounds.cc:107] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 21.17
I20260709 12:20:27.316045  6505 raft_consensus.cc:2889] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:27.319595  6578 pending_rounds.cc:85] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 16
I20260709 12:20:27.319936  6578 pending_rounds.cc:107] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 19.17
I20260709 12:20:27.320178  6578 raft_consensus.cc:2889] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:27.322409  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:27.312417 14233 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.015s
I20260709 12:20:27.327339 14233 consensus_queue.cc:1243] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer f2b06aebe97249a4877d798b8118ece4 log is divergent from this leader: its last log entry 21.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:20:27.327456  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:27.328405  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:27.329737  6548 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:27.330473  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 52529.4us
I20260709 12:20:27.269620 14476 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:27.265446 14506 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:27.331424  6622 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:27.265246 14117 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:27.339215  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 104684us
I20260709 12:20:27.277300 14559 consensus_queue.cc:1243] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer 496c7c3792cd4587b4dbeba435f4be1e log is divergent from this leader: its last log entry 20.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:20:27.342276 14559 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:27.326174  6652 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 23.
I20260709 12:20:27.343792  6550 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:27.262212 14550 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:27.289753 14542 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:27.312546 14568 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:27.287863 14265 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:27.289331 14521 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:27.372089  6622 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:27.375546  6505 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 12:20:27.384955 14596 log.cc:927] Time spent T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.185s	user 0.002s	sys 0.000s
I20260709 12:20:27.322629 14485 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:27.334285 14566 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:20:27.389606  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 45899.8us
I20260709 12:20:27.341249 14506 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:27.343885 14533 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:27.279062 14081 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:27.490410 14081 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:27.416927 14505 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 22 current term: 22 is higher.
I20260709 12:20:27.401902 14506 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:27.423702 14535 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
I20260709 12:20:27.388309 14265 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:27.500826 14600 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:20:27.433655 14570 consensus_meta.cc:287] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.574s	user 0.007s	sys 0.004s
I20260709 12:20:27.502025 14570 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:27.523130 14581 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:27.523867 14581 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:27.526490 14581 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:27.441716  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 69756.8us
I20260709 12:20:27.442330 14568 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:27.449687 14550 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:27.451469 14109 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:27.348711 14559 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:27.453681 14594 log.cc:927] Time spent T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:20:27.387014 14555 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:27.404875 14485 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:27.454739  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:27.368881 14262 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:27.455858 14483 consensus_queue.cc:1059] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:27.456394  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:27.669505  6654 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 26. Current term is 27. Ops: []
I20260709 12:20:27.457094  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:27.670475  6658 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 24.
I20260709 12:20:27.457782  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:27.671571  6655 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 23.
I20260709 12:20:27.394203 14536 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:27.672151 14536 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:27.672669 14536 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:20:27.672569  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 58773.5us
I20260709 12:20:27.674247  6622 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:27.425139 14543 consensus_queue.cc:237] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:27.675472  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:27.527854  6507 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 18.
I20260709 12:20:27.551309 14570 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 27 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:27.457036 14449 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:27.678606 14449 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:27.566778  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:20:27.438644 14491 consensus_meta.cc:287] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.609s	user 0.009s	sys 0.000s
W20260709 12:20:27.591113 14333 consensus_meta.cc:287] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.675s	user 0.011s	sys 0.000s
I20260709 12:20:27.680094 14491 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:27.680896 14597 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:27.681397 14597 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:27.682608 14491 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:27.683408  7274 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 24.
I20260709 12:20:27.684216  6580 raft_consensus.cc:2393] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 18: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:27.607062 14399 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:27.686363 14399 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:27.638916 14109 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:27.687234 14109 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:27.688850  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:27.689502  6649 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:20:27.684252 14483 raft_consensus.cc:3055] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:27.598574  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:27.599990 14587 consensus_meta.cc:287] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.694s	user 0.005s	sys 0.008s
I20260709 12:20:27.601783  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:20:27.607069  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:27.608173  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 341644us
I20260709 12:20:27.665424 14610 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:27.463119  6575 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 26.
I20260709 12:20:27.704926  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:27.711195  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:27.716567  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:27.720363  7266 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:20:27.715670  7273 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.017s	sys 0.070s
W20260709 12:20:27.722843  7273 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.017s	sys 0.070s
I20260709 12:20:27.692886 14483 raft_consensus.cc:740] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:27.725513  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:20:27.734392  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 368187us
I20260709 12:20:27.734519  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:27.735567  6501 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:27.709365 14445 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:27.712993 14399 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:20:27.749069  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:27.713328 14555 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:27.734019 14600 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:27.766341  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:27.734289 14554 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:27.709623 14587 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:27.774552  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 444859us
I20260709 12:20:27.719122 14581 consensus_queue.cc:1048] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 12:20:27.747813 14568 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:20:27.713137 14610 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:20:27.751241 14399 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:27.785483  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:27.792816  7272 raft_consensus.cc:2393] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 18: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
W20260709 12:20:27.771694 14541 consensus_meta.cc:287] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.637s	user 0.001s	sys 0.010s
I20260709 12:20:27.741034 14330 consensus_queue.cc:1059] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 12:20:27.783915  6548 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:27.806012  6649 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 18.
I20260709 12:20:27.772769 14262 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:27.777976 14566 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:20:27.814035  6473 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.021s	sys 0.013s
I20260709 12:20:27.761976 14445 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:27.816700  6473 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.021s	sys 0.013s
I20260709 12:20:27.800946 14399 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:20:27.761577 14555 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:27.829833  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:27.797508 14568 raft_consensus.cc:697] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 26 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:20:27.824962  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 53926.9us
I20260709 12:20:27.807858 14544 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:27.836154  6548 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:27.780676 14554 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:27.817349 14490 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:20:28.026007 14613 log.cc:927] Time spent T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 12:20:27.988906 14109 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:27.930202  6502 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 19.
I20260709 12:20:28.038779 14587 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:27.950742 14455 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:27.949322 14542 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:27.821389 14262 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:27.831809 14265 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:27.809284 14476 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:28.048871 14568 consensus_queue.cc:237] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:28.053223 14445 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:28.045282 14555 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:28.051579 14615 log.cc:927] Time spent T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:20:28.010671  6501 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 27.
W20260709 12:20:28.008992 14605 log.cc:927] Time spent T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:20:27.834985 14521 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:27.940806 14619 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.023s
I20260709 12:20:27.815630 14614 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:27.941274  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:27.804284 14483 consensus_queue.cc:260] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:28.028184 14593 log.cc:927] Time spent T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 1.086s	user 0.005s	sys 0.028s
I20260709 12:20:28.129508 14483 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:27.858137 14559 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:28.029525 14399 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:28.132719  6650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3830) took 1309 ms. Trace:
I20260709 12:20:28.134323 14399 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:28.133143  6650 rpcz_store.cc:276] 0709 12:20:26.823830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:26.823983 (+   153us) service_pool.cc:224] Handling call
0709 12:20:26.824445 (+   462us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:20:26.824984 (+   539us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:26.825028 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:20:26.834903 (+  9875us) log.cc:844] Serialized 52 byte log entry
0709 12:20:26.942673 (+107770us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:26.958944 (+ 16271us) log.cc:844] Serialized 14 byte log entry
0709 12:20:26.960444 (+  1500us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:26.960645 (+   201us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:27.220853 (+260208us) spinlock_profiling.cc:243] Waited 3.59 ms on lock 0x7b18003bf1a0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:20:27.231550 (+ 10697us) spinlock_profiling.cc:243] Waited 2.69 ms on lock 0x7b18003bf1a0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d6a2a6 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872c229c4 00007f2872c20acb 00007f2872ddbdc9
0709 12:20:28.027679 (+796129us) spinlock_profiling.cc:243] Waited 9.95 ms on lock 0x7b240020f978. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872c229c4 00007f2872c20acb 00007f2872ddbdc9 00007f2872ddba55 00007f2872ddb09d 00007f2875a0d10e 00007f2875a2488c 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221
0709 12:20:28.132437 (+104758us) raft_consensus.cc:1685] finished
0709 12:20:28.132469 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:28.132648 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1157,"prepare.queue_time_us":18381,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":1377,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":10527,"spinlock_wait_cycles":37320192,"thread_start_us":67285,"threads_started":1,"wal-append.queue_time_us":78119}
W20260709 12:20:28.059912  7274 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38562 (request call id 1283) took 1347 ms (client timeout 1696 ms). Trace:
I20260709 12:20:27.828794 14566 raft_consensus.cc:697] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:20:28.017274 14618 log.cc:927] Time spent T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:20:28.042752 14536 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:28.128100 14521 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:28.142678 14265 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:28.126814 14610 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:28.149844 14521 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:28.150272 14554 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:28.150750 14566 consensus_queue.cc:237] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:28.153860  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:28.154465  6579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 3343) took 1009 ms. Trace:
I20260709 12:20:28.137257 14614 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:28.154994  6579 rpcz_store.cc:276] 0709 12:20:27.145130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:27.161312 (+ 16182us) service_pool.cc:224] Handling call
0709 12:20:27.195712 (+ 34400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:27.311344 (+115632us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:27.311397 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:27.312078 (+   681us) log.cc:844] Serialized 26 byte log entry
0709 12:20:27.765126 (+453048us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:27.765169 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:27.765240 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:28.123523 (+358283us) spinlock_profiling.cc:243] Waited 5.31 ms on lock 0x7b48000030d8. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287adb2aa1 00007f287ada9b37 00007f287ada99ca 00007f287aea126f 00007f287aea0f37 00007f287aeb555c 00007f287aeb5508 00007f287aeb5488 00007f287aeb5443 00007f287aeb46ec 00007f2879ddea96
0709 12:20:28.154183 (+ 30660us) raft_consensus.cc:1685] finished
0709 12:20:28.154206 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:28.154410 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":125462,"raft.queue_time_us":119975,"raft.run_cpu_time_us":7972,"raft.run_wall_time_us":806993,"spinlock_wait_cycles":12223872,"thread_start_us":397475,"threads_started":1,"wal-append.queue_time_us":545223}
I20260709 12:20:28.156085 14262 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:28.156920 14614 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:28.158627  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:28.166380  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:28.169979  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:28.170750  6650 raft_consensus.cc:2393] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 18: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:28.174278  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:28.175272  6657 raft_consensus.cc:2410] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for term 23 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 12:20:28.149739 14476 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:28.187752 14581 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:28.188392 14581 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:28.189122  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:28.189702  6657 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:20:28.194047  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:28.097695 14455 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:28.207613  6657 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 27.
I20260709 12:20:27.928438  6473 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:28.073822 14478 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:28.106458  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:28.120594  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:28.112946 14506 consensus_meta.cc:287] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.621s	user 0.013s	sys 0.000s
I20260709 12:20:27.808822 14541 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:28.135320  7274 rpcz_store.cc:269] 0709 12:20:26.712258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:26.737758 (+ 25500us) service_pool.cc:224] Handling call
0709 12:20:28.059640 (+1321882us) spinlock_profiling.cc:243] Waited 935 ms on lock 0x7b5400502654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:28.059828 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2150077312}
I20260709 12:20:28.142809 14536 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:28.154513  6501 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 24.
I20260709 12:20:28.079020  7266 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:20:28.221753  6582 raft_consensus.cc:2393] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 23: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:28.227519  7273 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38562 (request call id 1291) took 1137 ms. Trace:
I20260709 12:20:28.225541  7240 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:28.190539 14581 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:27.846009  6622 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:28.244971 14399 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:20:28.245260  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 410592us
I20260709 12:20:28.246591  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:28.247256 14399 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:28.247720 14399 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:20:28.135221 14316 consensus_queue.cc:1175] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e7554c39679a41bbbca4659b72ab5c65" responder_term: 23 status { last_received { term: 23 index: 13 } last_committed_idx: 13 last_received_current_leader { term: 23 index: 13 } }
I20260709 12:20:28.205487 14505 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:28.248876  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 363060us
I20260709 12:20:28.263861  6473 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:28.228027  7273 rpcz_store.cc:276] 0709 12:20:27.083938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:27.084731 (+   793us) service_pool.cc:224] Handling call
0709 12:20:27.085070 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:27.101068 (+ 15998us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:27.101096 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:27.101539 (+   443us) log.cc:844] Serialized 26 byte log entry
0709 12:20:27.725168 (+623629us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:20:27.725228 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:27.725327 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:28.221339 (+496012us) raft_consensus.cc:1685] finished
0709 12:20:28.221358 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:28.221477 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46875,"raft.queue_time_us":23670,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":2842,"thread_start_us":545969,"threads_started":1,"wal-append.queue_time_us":658608}
W20260709 12:20:28.080156  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 244054us
I20260709 12:20:28.217206 14597 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:28.252396  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:28.229765 14577 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:28.247644  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:28.265880  6548 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:28.226971 14506 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:28.255862 14399 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:28.276149  6548 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:28.269784 14581 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:28.265666 14491 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:28.280042 14117 raft_consensus.cc:3055] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:20:28.266909 14329 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:28.269888 14536 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:28.269817 14505 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:28.286636  6473 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:28.286927 14109 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:28.287285 14552 consensus_queue.cc:1059] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
W20260709 12:20:28.340530 14625 log.cc:927] Time spent T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:20:28.340961 14552 raft_consensus.cc:3055] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:28.317437 14541 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:28.224470 14478 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:28.329877  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 53812.4us
I20260709 12:20:28.331314  6622 leader_election.cc:400] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 21. Current term is 22.
I20260709 12:20:28.271205 14496 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:20:28.334640  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 48102.1us
I20260709 12:20:28.350172  6622 leader_election.cc:403] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:28.262974 14185 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:28.347546  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 115593us
I20260709 12:20:28.348250 14552 raft_consensus.cc:740] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:20:28.356184 14622 log.cc:927] Time spent T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:20:28.356756 14552 consensus_queue.cc:260] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:28.357519  6578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 1319) took 1038 ms. Trace:
I20260709 12:20:28.358044 14478 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:28.358989 14581 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:28.358171 14552 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:28.354900  7238 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:28.360275  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 3439) took 1188 ms. Trace:
I20260709 12:20:28.357875  6578 rpcz_store.cc:276] 0709 12:20:27.319021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:27.319131 (+   110us) service_pool.cc:224] Handling call
0709 12:20:27.319416 (+   285us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:20:27.781624 (+462208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:27.781655 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:20:27.782192 (+   537us) log.cc:844] Serialized 52 byte log entry
0709 12:20:28.098785 (+316593us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:20:28.098823 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:28.098904 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:28.357302 (+258398us) raft_consensus.cc:1685] finished
0709 12:20:28.357322 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:28.357467 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5603,"raft.queue_time_us":38329,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":7369,"thread_start_us":767941,"threads_started":2,"wal-append.queue_time_us":442831}
I20260709 12:20:28.361934  7238 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:28.361549  6509 rpcz_store.cc:276] 0709 12:20:27.172254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:27.172462 (+   208us) service_pool.cc:224] Handling call
0709 12:20:27.199201 (+ 26739us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:27.199361 (+   160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:27.199384 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:27.199427 (+    43us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:20:27.199456 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:27.199504 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:28.360147 (+1160643us) spinlock_profiling.cc:243] Waited 2.52 ms on lock 0x7b540001ccd0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:28.360230 (+    83us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5798784}
I20260709 12:20:28.364138  6473 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:28.331804 13946 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:28.369730  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:28.331836 14117 raft_consensus.cc:740] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:20:28.378170  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:28.385535  7238 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:28.385911  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:28.386945  6501 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 19.
I20260709 12:20:28.394241  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:28.394963  6505 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:28.398213  6549 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:28.399878  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:28.400180  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:28.342002 14443 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:28.359505 14496 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:28.433395  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:28.421520  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:28.436610  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:28.439749  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 214313us
I20260709 12:20:28.438477  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:28.433022  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 45032.7us
I20260709 12:20:28.439517  6509 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:28.407202  6653 pending_rounds.cc:85] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 15
I20260709 12:20:28.440811  6502 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 25.
W20260709 12:20:28.440258  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 168431us
I20260709 12:20:28.411718  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:28.441731  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:28.442363  6655 raft_consensus.cc:2393] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 22: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:28.443007  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:28.441924  6578 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 23.
I20260709 12:20:28.441684  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:28.441160  6653 pending_rounds.cc:107] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 21.16
I20260709 12:20:28.444666  6653 raft_consensus.cc:2889] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:28.445412  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:28.332581 14491 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:28.408306  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:28.458591  6550 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:28.445995  6503 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:28.459651  6656 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22.
I20260709 12:20:28.400185  6651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 3252) took 1348 ms. Trace:
I20260709 12:20:28.407752  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:20:28.446846  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 61425.3us
I20260709 12:20:28.464879  6505 raft_consensus.cc:2468] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 23.
I20260709 12:20:28.466140  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:28.457065 14623 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:28.468693  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:28.469062 14623 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:28.438516 14438 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:20:28.471256 13946 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:28.471494 14623 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:28.471746 13946 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:28.472220 13946 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:28.460322  6651 rpcz_store.cc:276] 0709 12:20:27.051620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:27.053899 (+  2279us) service_pool.cc:224] Handling call
0709 12:20:28.400000 (+1346101us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400075dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:28.400126 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2416433408}
I20260709 12:20:28.476773  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:28.477908  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:28.478336  6473 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:28.387547 14478 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:20:28.480484 14531 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:28.387208 14109 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.096s
I20260709 12:20:28.482978 14109 consensus_queue.cc:1243] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 16.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:20:28.483664 14505 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:28.484673 14448 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:28.484916  6550 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:28.485250 14448 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:28.485814 14624 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:28.486357 14624 raft_consensus.cc:697] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:28.487500 14624 consensus_queue.cc:237] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:28.488795 14490 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:28.489101  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:28.489395 14490 raft_consensus.cc:697] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:28.489742  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:28.487519 14448 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:28.490428 14490 consensus_queue.cc:237] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:28.491611  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 49999.8us
I20260709 12:20:28.491801  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:28.492398  6576 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22.
I20260709 12:20:28.492460  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:28.493569  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:28.494300 14566 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:28.494786 14566 raft_consensus.cc:697] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 24 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:28.489733  6580 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 25.
I20260709 12:20:28.375159 14185 consensus_queue.cc:1059] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.121s
I20260709 12:20:28.495774 14566 consensus_queue.cc:237] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:28.496680 14185 raft_consensus.cc:3055] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:28.497088 14185 raft_consensus.cc:740] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:28.497931 14185 consensus_queue.cc:260] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:28.498731  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:28.457484  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:20:28.473066 14628 log.cc:927] Time spent T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:20:28.499429  6576 raft_consensus.cc:2393] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 26: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:28.500248  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:28.445734 14496 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:28.483074 14543 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:28.486287  6550 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:28.516379  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:28.522368  6622 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:28.502993  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:28.504562  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:28.506461  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:28.524137  6584 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:28.524294  6502 raft_consensus.cc:2393] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 26: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:28.509958  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:28.512077  6656 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
W20260709 12:20:28.459969 14485 consensus_meta.cc:287] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.793s	user 0.012s	sys 0.003s
I20260709 12:20:28.433171 14577 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:20:28.512632  6473 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:28.513849  6548 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:28.505513  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:28.400309 13554 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:28.400473 14329 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:28.532701  6474 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:28.532727  6655 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Advancing to term 26
I20260709 12:20:28.531850  6473 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:28.444713 14443 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:28.536135  7268 raft_consensus.cc:2393] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 19: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:20:28.537853  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:28.539065  6502 raft_consensus.cc:2393] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 22: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:28.376228 14536 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.145s
I20260709 12:20:28.511473 14109 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:28.541538  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:28.511881 14558 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:28.516767 14587 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:28.549481  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:28.530119 14233 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:28.515300 14478 raft_consensus.cc:697] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 27 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:28.526587 14533 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:28.543901 14440 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:28.531209 14185 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:28.565938  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:28.549958 14531 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:20:28.617051  6507 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41304 (request call id 1051) took 1446 ms (client timeout 1651 ms). Trace:
I20260709 12:20:28.618072  6578 raft_consensus.cc:2393] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 26: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:28.552839 14577 raft_consensus.cc:697] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 26 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:28.540076 14450 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:28.534601 13554 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:28.526897 14448 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:28.596318  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:28.541265 14443 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:28.550185 14329 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:28.562448 14587 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:20:28.617635  6507 rpcz_store.cc:269] 0709 12:20:27.170535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:27.172408 (+  1873us) service_pool.cc:224] Handling call
0709 12:20:28.616881 (+1444473us) spinlock_profiling.cc:243] Waited 354 ms on lock 0x7b54001e82d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:28.616993 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":815223552}
I20260709 12:20:28.629976  6503 raft_consensus.cc:2468] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 26.
I20260709 12:20:28.632030  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:28.550122  6502 pending_rounds.cc:85] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 13
I20260709 12:20:28.638450  6502 pending_rounds.cc:107] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 12:20:28.598136 14442 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:20:28.611379 14615 log.cc:927] Time spent T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:20:28.535563 14485 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:28.632002  6584 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 25.
I20260709 12:20:28.563956 14558 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:28.612089 14440 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:28.638032 14337 raft_consensus.cc:3055] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:28.635926 14450 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:28.532866 14117 consensus_queue.cc:260] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.13, Last appended by leader: 13, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:28.611836 14333 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:28.639017  6502 raft_consensus.cc:2889] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:28.561581 14109 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:28.636137 13554 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:28.636684 14329 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:28.651805  7265 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:28.638846 14587 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:28.632719 14486 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:28.635716 14455 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:28.639492 14531 raft_consensus.cc:697] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 17 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:28.644304 14442 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:28.643777 14533 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:28.666009  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 124564us
I20260709 12:20:28.651389 14440 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:28.651602 14337 raft_consensus.cc:740] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:28.674258 14109 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:28.698400  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:28.686254  7266 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 16. Current term is 17. Ops: []
I20260709 12:20:28.689275  6622 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:28.654770 14450 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:20:28.672125 14483 consensus_meta.cc:287] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.542s	user 0.006s	sys 0.001s
I20260709 12:20:28.692821  6657 raft_consensus.cc:1240] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 22. Current term is 23. Ops: []
W20260709 12:20:28.696247 14636 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 12:20:28.703821  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:28.710649  6651 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:20:28.704387  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:28.709472  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:28.684312 14446 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:28.690999 14442 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:28.730479 14558 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:28.732688  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 43512.1us
I20260709 12:20:28.732467 14446 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:28.734866 14446 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:28.723098  6475 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:28.736891  6475 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:28.738106  6475 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:28.738929 14562 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:20:28.682015 14606 consensus_queue.cc:1175] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f2b06aebe97249a4877d798b8118ece4" responder_term: 27 status { last_received { term: 26 index: 13 } last_committed_idx: 13 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 26. Current term is 27. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:20:28.740868 14562 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:28.741300 14562 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:28.741717 14562 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:28.741984 14485 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:28.742630 14638 log.cc:927] Time spent T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:20:28.742798 14541 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:28.697839 14577 consensus_queue.cc:237] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:28.702327 14544 raft_consensus.cc:2764] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:20:28.747848 14337 consensus_queue.cc:260] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:28.749078 14531 consensus_queue.cc:237] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:28.724506  6656 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:47858
I20260709 12:20:28.751837  6656 raft_consensus.cc:606] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:28.725598  6650 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:28.727824  6509 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 27.
I20260709 12:20:28.726595  6658 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:20:28.664515 14478 consensus_queue.cc:237] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:28.754168  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:28.755677  6584 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 24.
I20260709 12:20:28.727782 14581 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 21. Current term is 22.
I20260709 12:20:28.757704  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:28.759778 13554 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:28.661896 14333 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:28.706771 14330 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:28.767709 14399 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:28.770452 14496 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:28.770875 14440 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:28.687686 14486 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:28.776654  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:28.777223  6475 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:28.784305  6655 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 26.
I20260709 12:20:28.785827 14450 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:28.789584  7270 raft_consensus.cc:2410] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 22 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 12:20:28.791477 14333 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:28.793262 14330 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:28.796619  6582 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:20:28.810825  6654 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 24. Current term is 25. Ops: []
W20260709 12:20:28.797595  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 283813us
I20260709 12:20:28.802428  6652 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22.
I20260709 12:20:28.749099 14337 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:28.808413 13554 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:28.811261  6475 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:28.813685  6504 raft_consensus.cc:2393] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 23: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:28.815192  6475 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:28.809983 14446 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:28.817732 14446 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:28.818455 14446 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:28.818828 14446 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:28.819082 14450 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:28.819186 14446 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:28.821899 14330 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:28.711293 13946 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:28.822844  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:28.823642  6577 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:28.825202  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:28.826748  6548 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:28.837610  6548 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:28.804032 14541 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:28.853356 14440 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:28.836051 14483 raft_consensus.cc:2804] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:28.860011  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:28.900966  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:28.909207 14552 consensus_meta.cc:287] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.540s	user 0.010s	sys 0.008s
I20260709 12:20:28.899128 14606 raft_consensus.cc:2804] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:20:28.912089 14483 raft_consensus.cc:697] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:28.898856 14330 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:28.918893  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:28.919426  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:28.922753  6583 raft_consensus.cc:2368] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 25. Current term is 26.
I20260709 12:20:28.927724 13946 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:28.931473  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:28.932458 14606 raft_consensus.cc:697] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 26 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:28.933591 14606 consensus_queue.cc:237] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 25.12, Last appended by leader: 12, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:20:28.937062 14491 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:28.938355 14658 log.cc:927] Time spent T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:20:28.940219  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 3336) took 1813 ms. Trace:
I20260709 12:20:28.940634  6581 rpcz_store.cc:276] 0709 12:20:27.126640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:27.126790 (+   150us) service_pool.cc:224] Handling call
0709 12:20:27.172039 (+ 45249us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:27.275396 (+103357us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:20:27.334030 (+ 58634us) log.cc:844] Serialized 14 byte log entry
0709 12:20:28.298415 (+964385us) spinlock_profiling.cc:243] Waited 9.43 ms on lock 0x7b48000030d8. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287adb2aa1 00007f287ada9b37 00007f287ada99ca 00007f287aea126f 00007f287aea0f37 00007f287aeb555c 00007f287aeb5508 00007f287aeb5488 00007f287aeb5443 00007f287aeb46ec 00007f2879ddea96
0709 12:20:28.310084 (+ 11669us) spinlock_profiling.cc:243] Waited 911 us on lock 0x7b18003dd320. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a2923a 00007f2875e93a2a
0709 12:20:28.611403 (+301319us) log.cc:844] Serialized 14 byte log entry
0709 12:20:28.611619 (+   216us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:28.631370 (+ 19751us) log.cc:844] Serialized 26 byte log entry
0709 12:20:28.631504 (+   134us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:20:28.631738 (+   234us) log.cc:844] Serialized 14 byte log entry
0709 12:20:28.632564 (+   826us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:28.632662 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:28.697252 (+ 64590us) spinlock_profiling.cc:243] Waited 4.54 ms on lock 0x7b48000030d8. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287adb2aa1 00007f287ada9b37 00007f287ada99ca 00007f287aea126f 00007f287aea0f37 00007f287aeb555c 00007f287aeb5508 00007f287aeb5488 00007f287aeb5443 00007f287aeb46ec 00007f2879ddea96
0709 12:20:28.939932 (+242680us) raft_consensus.cc:1685] finished
0709 12:20:28.939963 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:28.940119 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":114934,"prepare.queue_time_us":138504,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":8159,"raft.queue_time_us":143148,"raft.run_cpu_time_us":11208,"raft.run_wall_time_us":1157512,"spinlock_wait_cycles":73021696,"thread_start_us":287176,"threads_started":2,"wal-append.queue_time_us":1174440}
I20260709 12:20:28.943050  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:28.943846  6581 raft_consensus.cc:2393] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 23: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:28.946072  7265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38562 (request call id 1295) took 1207 ms. Trace:
I20260709 12:20:28.946533  7265 rpcz_store.cc:276] 0709 12:20:27.738344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:27.738547 (+   203us) service_pool.cc:224] Handling call
0709 12:20:27.739038 (+   491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:28.659828 (+920790us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:28.945819 (+285991us) spinlock_profiling.cc:243] Waited 912 ms on lock 0x7b54000a50d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:28.946000 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2098669952}
W20260709 12:20:28.947605  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:28.951386 14543 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:28.955893 13946 consensus_queue.cc:1059] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:20:28.957211 13946 raft_consensus.cc:3055] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:28.957604 13946 raft_consensus.cc:740] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:28.961059  6577 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 23.
I20260709 12:20:28.962951  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:28.964916  6649 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 25.
I20260709 12:20:28.955005 14496 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:28.926138 14330 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:28.967694 14399 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:28.970866  6622 leader_election.cc:400] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 25. Current term is 26.
I20260709 12:20:28.971385  6622 leader_election.cc:403] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:28.975509  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:28.938349 14448 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:28.981153 14587 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:28.981882  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:28.982481  6579 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:28.983666 14587 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:28.984529 13946 consensus_queue.cc:260] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:28.986030 13946 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:28.992841 14442 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:28.993257 14329 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:28.980867 14330 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:28.999756  6577 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 24.
I20260709 12:20:29.009017  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:29.026683  6580 raft_consensus.cc:2393] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 26: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:29.011567  7238 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:29.015486  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:29.016857  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:29.030488  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:29.016602 14483 consensus_queue.cc:237] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:29.037227  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:29.038605  6584 raft_consensus.cc:1240] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:29.042827  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:29.044571 14438 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:28.975390 14543 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 25. Current term is 26.
I20260709 12:20:29.049413 14446 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:29.054759  6584 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 25. Current term is 26. Ops: [24.13-25.14]
I20260709 12:20:29.032435  6579 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 25.
I20260709 12:20:29.047223 14611 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:29.075145  6651 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "72269ad6925341548796bb34823b3e00"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:47858
I20260709 12:20:29.079638  6651 raft_consensus.cc:606] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:29.047290 14438 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:29.084540 14438 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:29.086104 14189 consensus_queue.cc:1059] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:29.066497 14455 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:29.091944  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:29.078045  6579 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 26. Current term is 27. Ops: []
I20260709 12:20:29.051330 14443 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:29.097230  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:29.097535  6506 raft_consensus.cc:1240] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:29.103336  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:29.104148  6503 raft_consensus.cc:2393] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 24: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:29.105798  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:29.107357  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:29.107479  6579 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:29.091795 14455 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:29.109949  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:29.121227  6506 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:20:29.141326  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:29.166330  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:29.194590  6651 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:47858
I20260709 12:20:29.195340  6651 raft_consensus.cc:606] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 24 LEADER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:29.091516 14611 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:29.197943  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:29.184072 14443 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:29.180642 14455 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:29.199868  6506 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 25.
I20260709 12:20:29.201064 14446 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:29.189003 14442 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:29.192369 14641 log.cc:927] Time spent T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:20:29.209713 14562 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:29.210963  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:29.212613  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:29.220120  6576 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:29.222131  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:29.224026  6507 raft_consensus.cc:1240] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:29.231016  6651 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:47858
I20260709 12:20:29.214640 14531 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:29.231737  6651 raft_consensus.cc:606] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:29.233050  6651 raft_consensus.cc:614] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:29.223292 14543 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:29.233765 14543 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:29.236361  6651 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:47858
I20260709 12:20:29.237090  6651 raft_consensus.cc:606] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Received request to transfer leadership to TS 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:29.237501  6651 raft_consensus.cc:614] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:29.237408  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:29.240514  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:20:29.241055  6579 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 21.
I20260709 12:20:29.217576 14438 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:29.243948  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:29.246834  6576 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 25.
I20260709 12:20:29.250349  6576 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:29.265525 13554 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:29.266374 13554 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:29.324491 14611 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:29.239759 14531 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:29.332171  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:29.336930  6506 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 22.
I20260709 12:20:29.339130 14438 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:29.339354  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:29.344827  6581 raft_consensus.cc:1275] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:20:29.351693  6622 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:29.353466  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:29.353734 14647 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:29.354540 14647 raft_consensus.cc:697] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 24 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:29.355365 14448 consensus_queue.cc:1059] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:29.355496  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:29.355609 14647 consensus_queue.cc:237] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:29.361503 14435 raft_consensus.cc:3055] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:29.356364 14448 consensus_queue.cc:1059] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 12:20:29.361997 14435 raft_consensus.cc:740] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:29.345713 14562 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:29.368505  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:29.368930  6622 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:29.357075 14663 log.cc:927] Time spent T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 12:20:29.374475 14435 consensus_queue.cc:260] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:29.375263 14650 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:29.375854 14435 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:29.376210 14448 consensus_queue.cc:1059] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:29.376610 13554 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:29.380429  6506 raft_consensus.cc:2393] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 26: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:29.385030  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:29.400696  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:29.401557  6577 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:20:29.404480  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:20:29.409041  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 882295us
I20260709 12:20:29.410020  6550 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:29.411330  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:29.415747  6550 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:29.407287 14664 log.cc:927] Time spent T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 12:20:29.404506 14650 raft_consensus.cc:697] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 25 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:29.425037 14455 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:29.432859  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:29.436890 14531 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:29.399369 14562 consensus_queue.cc:1059] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 24.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 12:20:29.438241 14562 raft_consensus.cc:3055] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:29.438666 14562 raft_consensus.cc:740] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:29.439529 14562 consensus_queue.cc:260] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:29.442691  6506 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 18.
I20260709 12:20:29.445034  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:29.446691  6580 raft_consensus.cc:2393] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 18: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:29.445576 14455 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:29.447103  6550 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:29.419595 14611 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:29.420105 14316 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:29.450035 14316 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:29.450606 14316 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:29.446139 14650 consensus_queue.cc:237] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:29.459290  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 85188.6us
I20260709 12:20:29.462271  6623 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:29.450074 14496 raft_consensus.cc:3055] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:29.465329  6550 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:29.468653 14669 raft_consensus.cc:3055] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:20:29.469152 14669 raft_consensus.cc:740] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:29.470137 14669 consensus_queue.cc:260] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:29.465034 14611 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:29.465888 14496 raft_consensus.cc:740] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:29.480055 14496 consensus_queue.cc:260] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:29.480654 14611 raft_consensus.cc:697] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:29.481160 14496 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:29.481650 14611 consensus_queue.cc:237] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:29.488229 14329 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:29.489804 14562 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:29.502094 14435 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:29.509961  6509 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:20:29.510663  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:29.523303  6505 raft_consensus.cc:2393] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 20: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:29.524719 14435 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:29.538751  6507 raft_consensus.cc:1275] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 25 index: 12. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 12:20:29.541837  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:29.542785  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:20:29.543542  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:29.553808  7268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38546 (request call id 1096) took 1064 ms. Trace:
I20260709 12:20:29.554342  7268 rpcz_store.cc:276] 0709 12:20:28.489025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:28.489515 (+   490us) service_pool.cc:224] Handling call
0709 12:20:29.553544 (+1064029us) spinlock_profiling.cc:243] Waited 45.6 ms on lock 0x7b54004d7e54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:29.553731 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":104791936}
I20260709 12:20:29.556883  6507 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 23. Current term is 24. Ops: []
W20260709 12:20:29.557951  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:29.513367 14117 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:20:29.562189  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 97602.5us
I20260709 12:20:29.563601  7268 raft_consensus.cc:1240] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:29.566300  6473 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:29.567911  6550 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:29.573850  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 223353us
I20260709 12:20:29.571276  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:29.579985  6550 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:29.579628  6473 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:29.578810 14435 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:29.583699  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:29.585376  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:29.589394  6473 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:29.583482 14442 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:29.593904  6550 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:29.591087  7238 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:29.601449  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:29.578962 14443 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:29.575856 14623 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:20:29.607254  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:29.608424  6654 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:20:29.584758 14438 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:29.587432 14619 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:29.587201 14446 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:29.618883  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:29.623870  6654 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 20.
W20260709 12:20:29.631635  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 257865us
I20260709 12:20:29.604254 14668 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:29.635385  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:29.606616 14478 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:29.640734  6548 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:29.652706  6548 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:20:29.626015 14483 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.011s	sys 0.008s
I20260709 12:20:29.655705  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:29.626559 14442 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:20:29.619338 14619 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:29.634140 14445 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:29.625016 14446 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:29.638203 14506 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:29.657073 14623 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:20:29.658595 14647 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:29.624845 14265 consensus_queue.cc:1059] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 12:20:29.708863  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:29.828855  6654 raft_consensus.cc:2393] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 26: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:29.711854  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:29.654345 14668 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:29.720266  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:29.723307  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:29.662676 14490 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:20:29.675845 14483 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.012s	sys 0.008s
W20260709 12:20:29.811800  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:29.663286 14478 raft_consensus.cc:697] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 21 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
W20260709 12:20:29.815534  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:29.819613  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:29.671504 14486 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:29.714648  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:29.883778  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 294443us
I20260709 12:20:29.662236 14606 consensus_queue.cc:1059] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.150s
I20260709 12:20:29.847231  6653 raft_consensus.cc:2410] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 23 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 12:20:29.799772 14435 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:29.809626 14619 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:20:29.850937  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 239774us
I20260709 12:20:29.880693 14189 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:29.881793  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:29.814714 14446 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:29.838600  6650 raft_consensus.cc:2393] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 27: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:29.899173  7240 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:29.903084  6584 raft_consensus.cc:1240] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:29.911458  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:20:29.912915  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 234286us
I20260709 12:20:29.913054  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:29.914463  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:29.946918  7238 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:29.953728  6651 raft_consensus.cc:2393] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 20: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
W20260709 12:20:29.960704  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 323270us
W20260709 12:20:29.965124  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 502991us
W20260709 12:20:29.966806  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 308658us
I20260709 12:20:29.967834  6548 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:29.986763  6657 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 28. Current term is 29. Ops: []
I20260709 12:20:29.905616 14443 raft_consensus.cc:697] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 25 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:29.838987 14647 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:29.839813 14667 consensus_queue.cc:1059] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.139s
W20260709 12:20:29.975286 14316 consensus_meta.cc:287] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.524s	user 0.012s	sys 0.000s
I20260709 12:20:29.998148  6657 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 25 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:29.995097  6653 raft_consensus.cc:3055] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:30.006590  6653 raft_consensus.cc:740] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:30.001684  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 3281) took 1322 ms. Trace:
I20260709 12:20:30.009037  6649 rpcz_store.cc:276] 0709 12:20:28.678966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:28.680965 (+  1999us) service_pool.cc:224] Handling call
0709 12:20:30.001521 (+1320556us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b54000744d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:30.001627 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":576902528}
I20260709 12:20:29.984467 14483 raft_consensus.cc:3055] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:29.996176 14490 raft_consensus.cc:3055] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:30.007989  6653 consensus_queue.cc:260] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:29.995749 14435 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:30.034053  6653 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:20:30.035873  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:29.946085 14189 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:30.057631  6504 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 3457) took 1664 ms (client timeout 2146 ms). Trace:
W20260709 12:20:30.060539  6504 rpcz_store.cc:269] 0709 12:20:28.393502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:28.393890 (+   388us) service_pool.cc:224] Handling call
0709 12:20:30.057383 (+1663493us) spinlock_profiling.cc:243] Waited 314 ms on lock 0x7b5400204254. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:30.057570 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":722033536}
I20260709 12:20:30.022961 14483 raft_consensus.cc:740] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:30.032123 14666 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:30.085996  6653 raft_consensus.cc:1275] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 15. (term mismatch)
I20260709 12:20:30.086957  6653 pending_rounds.cc:85] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 14
I20260709 12:20:30.087334  6653 pending_rounds.cc:107] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 16.15
I20260709 12:20:30.087693  6653 raft_consensus.cc:2889] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:30.014326 14316 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:30.034994 14490 raft_consensus.cc:740] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:30.093236  6474 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:30.053579 14611 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:30.053975 14442 raft_consensus.cc:697] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 26 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:30.063201 14606 consensus_queue.cc:1059] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 12:20:30.058981 14435 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:30.127908  6582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 3362) took 1634 ms. Trace:
I20260709 12:20:30.130381  6582 rpcz_store.cc:276] 0709 12:20:28.493768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:28.496111 (+  2343us) service_pool.cc:224] Handling call
0709 12:20:28.496954 (+   843us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:28.816213 (+319259us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:30.127560 (+1311347us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b540006de54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:30.127834 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":688532864}
I20260709 12:20:30.028630 14478 consensus_queue.cc:237] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:30.131294  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:30.125399 14505 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.102113 14684 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:30.123246 14445 raft_consensus.cc:697] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 23 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:30.128248 14679 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:20:30.185913  6475 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.3:36107 timed out after 1.993s (SENT)
W20260709 12:20:30.182854 14672 log.cc:927] Time spent T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.208s	user 0.001s	sys 0.001s
I20260709 12:20:30.199143  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:30.202236 14562 consensus_meta.cc:287] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.684s	user 0.010s	sys 0.001s
I20260709 12:20:30.179702  6505 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:45880
I20260709 12:20:30.223971  6505 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:30.224393  6505 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:20:30.205130  6652 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58972 (request call id 3282) took 1515 ms (client timeout 1993 ms). Trace:
I20260709 12:20:30.102181 14443 consensus_queue.cc:237] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:30.212757 14647 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:30.183215 14490 consensus_queue.cc:260] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:30.183182 14505 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:30.229552  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:30.230320  6509 raft_consensus.cc:2368] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 25. Current term is 26.
W20260709 12:20:30.183161  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 248194us
I20260709 12:20:30.213842 14399 raft_consensus.cc:3055] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:30.216432 14189 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:30.216773 14440 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:30.217118 14445 consensus_queue.cc:237] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:30.187318 14674 log.cc:927] Time spent T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 12:20:30.175189 14483 consensus_queue.cc:260] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:30.219612 14671 log.cc:927] Time spent T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
W20260709 12:20:30.220345 14676 log.cc:927] Time spent T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 12:20:30.226181  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 279468us
I20260709 12:20:30.179610 14684 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:30.258425  7240 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:30.225631  6652 rpcz_store.cc:269] 0709 12:20:28.689847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:28.690300 (+   453us) service_pool.cc:224] Handling call
0709 12:20:30.204920 (+1514620us) spinlock_profiling.cc:243] Waited 88.5 ms on lock 0x7b540019d7d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:30.205041 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":203537664}
I20260709 12:20:30.238612  6505 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:30.181318 14611 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
W20260709 12:20:30.231208 14117 consensus_meta.cc:287] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.654s	user 0.009s	sys 0.001s
I20260709 12:20:30.259182  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:30.268010  6509 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 23: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:30.268829  6505 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:30.232224 14399 raft_consensus.cc:740] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:30.276530  6509 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 26 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 25. Current term is 26. Ops: [25.14-25.14]
I20260709 12:20:30.232662 14442 consensus_queue.cc:237] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:30.280628  6575 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 27.
I20260709 12:20:30.266073 14690 raft_consensus.cc:3055] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Stepping down as leader of term 22
W20260709 12:20:30.291183  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 105282us
I20260709 12:20:30.266415 14684 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:30.299144  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:30.301306  6656 raft_consensus.cc:1275] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:20:30.270823 14634 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:20:30.293565  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 346697us
I20260709 12:20:30.298574  6653 raft_consensus.cc:1240] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:30.298732  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:30.302996  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:30.308179  6650 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 24.
I20260709 12:20:30.309167  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:30.295634  6473 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:20:30.291844 14438 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.005s	sys 0.010s
I20260709 12:20:30.256966 14440 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:30.321405  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:30.311625  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:30.321415  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:30.321775  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:30.325994  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:30.325449  6657 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:30.326632  6505 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:30.293509 14690 raft_consensus.cc:740] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:20:30.318773 14438 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.006s	sys 0.011s
I20260709 12:20:30.335427  6580 raft_consensus.cc:1240] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:30.308972 14647 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:20:30.334379  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:30.319427 14683 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:30.344878  6656 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:20:30.310789 14702 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:30.347420 14521 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:30.348052 14521 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:30.350250  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:30.331835 14708 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:30.351459 14708 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:30.351881 14708 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:30.309391 14677 consensus_queue.cc:1048] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 12:20:30.353134  6505 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 27.
W20260709 12:20:30.354666  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:30.340145 14438 raft_consensus.cc:697] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 25 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:30.333609 14316 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:30.328400 14691 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:30.334990 14705 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:30.362003  6505 raft_consensus.cc:1240] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:30.339222 14506 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.367297 14438 consensus_queue.cc:237] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:30.372946 14700 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:30.333962 14496 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.383234  6657 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 26.
I20260709 12:20:30.406013  6505 raft_consensus.cc:1240] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:30.357475 14713 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:30.327385 14399 consensus_queue.cc:260] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:30.377099 14440 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:30.408505 14700 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:30.436162 14505 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:30.355605 14702 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:30.438886 14702 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:30.439890 14399 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:30.355492 14683 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:30.352703 14647 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:30.428380 14713 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:30.392123 14705 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:30.463377 14647 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:30.463575 14696 log.cc:927] Time spent T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 12:20:30.464075 14705 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:30.390527 14435 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:30.465212 14446 consensus_meta.cc:287] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.507s	user 0.011s	sys 0.001s
I20260709 12:20:30.465406 14683 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:30.465641 14700 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:30.465765 14446 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:30.467010  6509 raft_consensus.cc:1240] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 25. Current term is 26. Ops: []
I20260709 12:20:30.468104  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:30.468616  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:30.469041  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:30.469967  6500 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 26. Current term is 27. Ops: []
I20260709 12:20:30.468622  6507 raft_consensus.cc:2393] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 24: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:30.463392 14713 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:30.474893 14619 consensus_meta.cc:287] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.523s	user 0.010s	sys 0.000s
I20260709 12:20:30.475286  6503 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 25.
I20260709 12:20:30.475327 14619 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:30.420562 14496 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:30.476635  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:30.477133  6505 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:30.478229 14496 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:30.406257 14506 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:30.480392 14506 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:30.480863  6583 raft_consensus.cc:1240] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 26 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 25. Current term is 26. Ops: []
I20260709 12:20:30.481532  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 3080) took 1557 ms. Trace:
I20260709 12:20:30.481879  6583 rpcz_store.cc:276] 0709 12:20:28.924290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:29.047991 (+123701us) service_pool.cc:224] Handling call
0709 12:20:30.208709 (+1160718us) spinlock_profiling.cc:243] Waited 6.97 ms on lock 0x7b5800116198. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:20:30.208922 (+   213us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:30.480688 (+271766us) spinlock_profiling.cc:243] Waited 60.1 ms on lock 0x7b5400246420. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e00dc0 00007f2875e8b1c0 00007f2875e8c231 00007f2875e88d8f 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c
0709 12:20:30.481299 (+   611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:30.481481 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":154342528}
I20260709 12:20:30.481714  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:30.489315  6509 raft_consensus.cc:2368] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 19. Current term is 20.
I20260709 12:20:30.491003 14446 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:30.495639  6502 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22.
I20260709 12:20:30.500880  6582 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:30.502554  6651 raft_consensus.cc:1275] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:20:30.491865 14669 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:30.479976 14708 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:30.505556 13946 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:20:30.498337 14623 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.000s	sys 0.014s
I20260709 12:20:30.506199 13946 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:30.511237 14623 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.000s	sys 0.014s
I20260709 12:20:30.511584 14623 raft_consensus.cc:697] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:30.512503 14623 consensus_queue.cc:237] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:30.513111 13946 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:30.507205  6550 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:30.516948 13946 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:30.517379 13946 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:30.517772 13946 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:30.524294  6550 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:30.536168  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:30.538712  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:30.540987 14543 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:30.542723 14668 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:30.545599 14670 log.cc:927] Time spent T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.596s	user 0.003s	sys 0.000s
I20260709 12:20:30.547392 13946 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:30.548755  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:30.550331  6653 raft_consensus.cc:3055] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:30.550729  6653 raft_consensus.cc:740] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:30.553753 13946 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:30.551759  6653 consensus_queue.cc:260] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:30.554797  6653 raft_consensus.cc:3060] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:30.536514 14714 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:30.558163 14708 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:30.559657 14619 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:30.507758  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 121392us
I20260709 12:20:30.508424  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:30.561062  6656 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:30.560886 14684 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:30.561702  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:30.562364  7273 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:20:30.562894 14687 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.563418 14687 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:30.508692  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:30.563892  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:30.564150 14666 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:30.564596 13554 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:30.564800  7270 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:30.565111 14666 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:30.566707 14634 raft_consensus.cc:697] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 20 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:30.568804 14706 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:30.569267 14490 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:30.569365 14706 raft_consensus.cc:697] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 18 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:30.567766 14634 consensus_queue.cc:237] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:30.570490 14706 consensus_queue.cc:237] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:30.563679 14684 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:30.572171  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 260624us
I20260709 12:20:30.573457 14698 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:30.575294 14698 consensus_queue.cc:1059] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:20:30.576299  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:30.576454 14710 raft_consensus.cc:3055] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:30.576912 14710 raft_consensus.cc:740] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:30.576934  6509 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:30.578045 14710 consensus_queue.cc:260] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:30.579804 14710 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:30.514696 14713 consensus_queue.cc:1048] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.067s
I20260709 12:20:30.582190  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:30.572173  6623 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:30.564617  6657 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
I20260709 12:20:30.585018 14543 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:30.585445 14543 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:30.585768 14543 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:20:30.586548  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:30.565126 13554 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:30.592373  7272 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38562 (request call id 1299) took 1027 ms. Trace:
I20260709 12:20:30.592787  7272 rpcz_store.cc:276] 0709 12:20:29.564939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:29.565095 (+   156us) service_pool.cc:224] Handling call
0709 12:20:30.592171 (+1027076us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400502654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:30.592298 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2315974528}
I20260709 12:20:30.594272  6549 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:30.595417 14713 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:20:30.601091  6507 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 17 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 16. Current term is 17. Ops: []
W20260709 12:20:30.602540  6575 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 3377) took 1849 ms (client timeout 1774 ms). Trace:
I20260709 12:20:30.602769  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:20:30.602998  6575 rpcz_store.cc:269] 0709 12:20:28.753281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:28.755004 (+  1723us) service_pool.cc:224] Handling call
0709 12:20:30.602361 (+1847357us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b540032b754. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:30.602471 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3395592448}
W20260709 12:20:30.609666  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:30.610334  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:30.609689  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:20:30.610745  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:30.613356 14692 raft_consensus.cc:3055] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:30.613636 14687 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:30.510000  6502 raft_consensus.cc:1240] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:30.614382  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:30.615067  6507 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 24.
I20260709 12:20:30.621873 14694 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:30.622412 14694 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:30.623379 14686 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:30.623988 14686 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:30.625896 14686 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:30.627457  6584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 3085) took 1672 ms. Trace:
I20260709 12:20:30.616318 14483 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:30.627848  6584 rpcz_store.cc:276] 0709 12:20:28.955208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:29.057177 (+101969us) service_pool.cc:224] Handling call
0709 12:20:29.057471 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:29.906077 (+848606us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:30.627248 (+721171us) spinlock_profiling.cc:243] Waited 845 ms on lock 0x7b540034d5d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:30.627403 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1944115456}
I20260709 12:20:30.631184  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:30.631881  6509 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 23.
I20260709 12:20:30.630532  6475 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:30.635921  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:30.636576  6576 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:20:30.638296  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:30.639117  6655 raft_consensus.cc:2410] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 28 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
I20260709 12:20:30.640645  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: true dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:30.641278  6655 raft_consensus.cc:3055] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:30.641583  6655 raft_consensus.cc:740] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:30.641521  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:30.643365  6576 raft_consensus.cc:2393] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 24: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:30.643512  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:30.631193  6656 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 24.
I20260709 12:20:30.645978  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:30.646695  6654 raft_consensus.cc:2393] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 26: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:30.649425  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:30.644891  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:30.644527  6506 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 25.
I20260709 12:20:30.652026  6653 raft_consensus.cc:1275] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 12. (term mismatch)
I20260709 12:20:30.653228  6653 pending_rounds.cc:85] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 11
I20260709 12:20:30.653211  6657 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22.
I20260709 12:20:30.653641  6653 pending_rounds.cc:107] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 22.12
I20260709 12:20:30.653671  6651 raft_consensus.cc:1240] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:30.654040  6653 raft_consensus.cc:2889] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:30.654119  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:30.654745  6657 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:20:30.655150  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:30.655910  6656 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 23.
I20260709 12:20:30.652249  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:20:30.656770  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 132549us
I20260709 12:20:30.657537  6550 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:30.658346  6503 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 25 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:30.658200  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:30.660482  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:30.662420  6550 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:30.668480  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:30.671653  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:30.672439  6579 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:30.673041  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:30.673442  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:30.675293  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:30.655282  6655 consensus_queue.cc:260] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:30.679148  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:30.613816 14692 raft_consensus.cc:740] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:30.682822  6505 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:20:30.685211  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:30.686262  6550 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:30.689568  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:30.699770  6501 raft_consensus.cc:2368] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 25. Current term is 26.
I20260709 12:20:30.684607  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:30.698395  6550 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:30.703248 13554 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:30.683699  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:30.691870  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 61415.3us
I20260709 12:20:30.692314  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:30.705389 13946 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:30.706060 13946 raft_consensus.cc:697] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:30.671424 14490 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:30.707882  6622 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:30.708246  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:30.708693  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:30.678220 14703 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:30.709445  6475 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:30.689967 14647 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:30.690337 14614 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:20:30.709702  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:30.678714 14680 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:30.714484  6584 raft_consensus.cc:2393] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 21: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:20:30.700914 14399 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:20:30.684795 14435 consensus_queue.cc:1175] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "20a202c76244485fbc8f1610473db8b9" responder_term: 18 status { last_received { term: 18 index: 13 } last_committed_idx: 13 last_received_current_leader { term: 18 index: 13 } }
I20260709 12:20:30.702275 14445 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:20:30.707594 14713 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:30.693572 14669 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.733052 13946 consensus_queue.cc:237] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:30.707413 14189 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:30.710356 14694 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:30.712711  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:30.773706  6506 raft_consensus.cc:2410] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 24 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:20:30.752998  6475 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:30.754045  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:30.755277  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:30.775161  6475 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:30.756688  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:20:30.757110  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 146267us
I20260709 12:20:30.757546 14443 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:20:30.757874  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:30.746148 14623 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:30.716640  6622 leader_election.cc:400] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 19. Current term is 20.
I20260709 12:20:30.802233  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:30.804764  6622 leader_election.cc:403] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:30.805110  6580 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:30.801268  7268 raft_consensus.cc:2393] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 23: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:20:30.803048  7269 raft_consensus.cc:1275] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:20:30.805750  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 89317.4us
I20260709 12:20:30.806653  6622 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:30.761416  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:30.807613  6507 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 25.
I20260709 12:20:30.693832 14262 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 12:20:30.712023 14587 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:20:30.769338  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:30.722011 14675 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:30.745281 14714 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.782617 14690 consensus_queue.cc:260] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:30.758119 14717 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.715564 14606 consensus_queue.cc:1048] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 12:20:30.760455 14721 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:30.801692  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:30.806874  7265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38562 (request call id 1302) took 1222 ms. Trace:
I20260709 12:20:30.829659  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: true dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:30.769456 14399 raft_consensus.cc:697] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 27 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:30.804121  6473 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:30.814189  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:30.814568  6577 raft_consensus.cc:1217] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Deduplicated request from leader. Original: 22.12->[24.13-25.14]   Dedup: 24.13->[25.14-25.14]
I20260709 12:20:30.814934  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:30.816545  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:30.817483  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:30.803696 14669 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:30.817680 14580 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:30.811414 14683 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 19. Current term is 20.
I20260709 12:20:30.817502  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:30.768074 14692 consensus_queue.cc:260] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:30.845175  6582 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 27.
I20260709 12:20:30.845386 14692 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:30.817677  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:30.813567  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:30.848109  6580 raft_consensus.cc:1240] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 24. Current term is 25. Ops: [22.12-24.13]
I20260709 12:20:30.848095  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:30.827096 14717 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:30.851573 14717 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:30.796737 14555 consensus_queue.cc:1059] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 23.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:20:30.767889 14702 consensus_queue.cc:1059] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 12:20:30.823019  7268 raft_consensus.cc:1275] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:20:30.819787 14623 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:30.765483 14316 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:30.712122 14704 consensus_queue.cc:1059] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 25.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.104s
W20260709 12:20:30.827320  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 52217.1us
I20260709 12:20:30.865397  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:30.865641  6475 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:30.792248 14189 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:30.830387  7274 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:30.728884  6579 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:20:30.770087 14713 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:30.823597 14703 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:30.832728  7266 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 24.
I20260709 12:20:30.837374  6577 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 25. Current term is 26. Ops: [24.13-25.14]
I20260709 12:20:30.837961  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:30.792526 14731 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:20:30.875023  6473 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:30.840392  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:30.834084 14536 consensus_queue.cc:1059] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.013s
I20260709 12:20:30.841971 14443 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:30.876994 14506 consensus_queue.cc:1059] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:30.877557 14443 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:30.878041 14443 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:30.878203 14496 consensus_queue.cc:1059] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:20:30.878702 14189 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:30.879673 14440 raft_consensus.cc:2764] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election decision vote started in defunct term 26: won
I20260709 12:20:30.829288 14330 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:20:30.852901  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:30.853838  6582 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:20:30.829113  7265 rpcz_store.cc:276] 0709 12:20:29.584785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:29.585033 (+   248us) service_pool.cc:224] Handling call
0709 12:20:29.603783 (+ 18750us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:29.604032 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:20:29.953064 (+349032us) log.cc:844] Serialized 14 byte log entry
0709 12:20:30.806406 (+853342us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:30.806459 (+    53us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:20:30.806502 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:30.806652 (+   150us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:30.806807 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":293356,"prepare.queue_time_us":612307,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":284329,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"thread_start_us":880931,"threads_started":3,"wal-append.queue_time_us":505640}
I20260709 12:20:30.762795 14614 raft_consensus.cc:697] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:30.828766 14486 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:30.763207 14680 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:30.858902  6657 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:20:30.862609  6474 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:30.867848 14316 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:30.864048  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:30.827262 14714 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:30.869408  6475 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:30.870862  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:30.876893 14703 raft_consensus.cc:697] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:30.855641 14683 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:20:30.935541  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 66177.1us
I20260709 12:20:30.938313  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:30.938071 14703 consensus_queue.cc:237] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:30.940539 14683 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:30.943053 14683 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:30.952576 14714 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:30.960956  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 98459.2us
I20260709 12:20:30.887319  7274 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:20:30.870502 14677 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:30.908856  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:30.871767 14713 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:30.913596 14399 consensus_queue.cc:237] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.13, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:30.910990  6655 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:30.986157 14521 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:30.990464 14490 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:30.991317  6578 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:31.001158  6655 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:20:30.876598  6473 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:30.834378 14721 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:30.909499 14668 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:30.925436 14614 consensus_queue.cc:237] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:30.927696  6652 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 17.
I20260709 12:20:30.816818  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:30.883139 14727 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.052s
I20260709 12:20:31.009709  7240 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:31.005869  6474 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:20:31.009871  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 193138us
I20260709 12:20:31.013913  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:31.014931  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:30.926501 14666 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:31.017776  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 141217us
I20260709 12:20:30.964779 14446 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:30.995769 14506 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:31.032794 14721 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:31.007601 14668 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:30.960364 14680 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:31.038638  6549 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:31.021410 14692 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:30.931834 14486 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:31.061096 14692 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:31.063603 14692 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:31.064589 14486 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:31.067464  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:31.047873  6474 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:31.027733  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:31.026794  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:31.070523  6475 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:31.035696  6578 raft_consensus.cc:1275] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:20:31.057492  6622 leader_election.cc:400] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 25. Current term is 26.
I20260709 12:20:31.018987 14675 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:31.022662 14679 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:31.018196 14677 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.056s
I20260709 12:20:31.028365 14683 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:31.028280 14496 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:31.085657  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:31.061607 14445 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:20:31.088284  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:31.062626 14669 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:31.076745  6474 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:31.091660 14316 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:31.058292 14587 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.028s
W20260709 12:20:31.095893 14543 consensus_meta.cc:287] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.510s	user 0.007s	sys 0.000s
I20260709 12:20:31.111418 14543 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:31.113691 14543 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:31.098407 14668 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:31.079214 14731 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:31.058799 14490 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 12:20:31.081426 14531 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:20:31.115700 14496 raft_consensus.cc:697] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:31.077870 14521 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:31.028159 14704 consensus_queue.cc:1059] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.072s
I20260709 12:20:31.078027 14721 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:31.086062 14679 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.104126  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:31.034878 14189 consensus_queue.cc:1059] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 28.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.069s
I20260709 12:20:31.139191  6652 raft_consensus.cc:1240] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:20:31.146799  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:31.141139  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 64492us
I20260709 12:20:31.107012 14446 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
W20260709 12:20:31.152120  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 48079.5us
I20260709 12:20:31.152917  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:31.154503  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:31.155637  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:31.108740 14683 raft_consensus.cc:697] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 25 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:20:31.157148  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 71556.4us
I20260709 12:20:31.159770  6475 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:31.137900 14673 log.cc:927] Time spent T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 1.075s	user 0.005s	sys 0.000s
I20260709 12:20:31.128593 14623 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:31.139047 14679 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:31.166721  6622 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:31.174084  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:31.175611  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:31.143142 14445 raft_consensus.cc:697] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 24 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:31.167575  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:31.190986  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:31.195561  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:31.196238  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:31.162011 14555 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:31.163637 14693 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:31.161336 14668 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:20:31.188235 14734 log.cc:927] Time spent T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.274s	user 0.000s	sys 0.001s
I20260709 12:20:31.202826  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:31.168099 14435 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:31.116735 14496 consensus_queue.cc:237] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:31.145812 14721 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:31.191542 14704 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:31.211678  6622 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:31.164129 14675 consensus_queue.cc:1059] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 25.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:20:31.213581  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:31.197259 14587 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:31.219539 14555 raft_consensus.cc:697] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:31.220597 14675 consensus_queue.cc:1243] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer 496c7c3792cd4587b4dbeba435f4be1e log is divergent from this leader: its last log entry 26.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:20:31.227445 14555 consensus_queue.cc:237] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:31.219380 14693 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:31.231165 14713 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:31.205868 14668 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:31.190229 14692 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:31.237957  6474 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:31.238452  6655 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 27 index: 14. (term mismatch)
I20260709 12:20:31.239015  6655 pending_rounds.cc:85] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 13
I20260709 12:20:31.239341  6655 pending_rounds.cc:107] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 26.14
I20260709 12:20:31.239655  6655 raft_consensus.cc:2889] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:31.215191 14399 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:31.246534  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:31.251662  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:31.252265  7274 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:31.254166  6655 raft_consensus.cc:1275] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 23 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 12:20:31.222046 14703 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:31.256886  6652 raft_consensus.cc:1240] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:31.244366 14668 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:31.267171  7274 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:20:31.246496 14693 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:31.273193  6623 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:31.251582 14704 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.260119 14680 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:31.290966 14690 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:31.280258 14587 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:31.292305 14587 raft_consensus.cc:697] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 19 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:31.293493 14587 consensus_queue.cc:237] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:31.290709 14713 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:31.329952 14703 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.330766 14703 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:31.304461 14531 raft_consensus.cc:3055] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:20:31.332850 14531 raft_consensus.cc:740] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:31.333859 14531 consensus_queue.cc:260] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.15, Last appended by leader: 15, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:31.305516 14669 raft_consensus.cc:3055] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:31.335779 14669 raft_consensus.cc:740] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:31.335357 14443 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:31.278420 14733 log.cc:927] Time spent T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
I20260709 12:20:31.336824 14669 consensus_queue.cc:260] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:31.273244 14542 consensus_queue.cc:1048] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 12:20:31.337949 14669 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:20:31.308172 14737 log.cc:927] Time spent T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 12:20:31.310453 14440 raft_consensus.cc:3055] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:31.339313 14440 raft_consensus.cc:740] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:31.339540 14731 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:20:31.306495  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:31.266101 14521 consensus_queue.cc:1059] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 12:20:31.340618 14521 raft_consensus.cc:3055] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:31.317333 14692 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:31.322245 14717 raft_consensus.cc:3055] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:31.344513 14717 raft_consensus.cc:740] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:31.345350 14703 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:31.345414 14717 consensus_queue.cc:260] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:31.346896 14717 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:31.245868 14445 consensus_queue.cc:237] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:31.281111 14735 log.cc:927] Time spent T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 12:20:31.280524 14693 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:31.340184 14440 consensus_queue.cc:260] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:31.312603 14505 raft_consensus.cc:3055] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:31.353053 14505 raft_consensus.cc:740] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:31.354302 14693 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:31.353974 14505 consensus_queue.cc:260] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:31.355297 14505 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:31.340987 14521 raft_consensus.cc:740] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:31.356801  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:31.284256 14704 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:31.357630  7272 raft_consensus.cc:2393] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 20: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:20:31.357720  6651 raft_consensus.cc:3055] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:31.357107 14521 consensus_queue.cc:260] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:31.358134  6651 raft_consensus.cc:740] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:31.358722 14521 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:31.246609 14683 consensus_queue.cc:237] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:31.322908 14721 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:31.358955 14704 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:31.271471 14623 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 12:20:31.370020  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:31.370880  6473 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:31.371888  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:31.373060  7274 raft_consensus.cc:2393] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 24: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:31.372387 14694 consensus_queue.cc:1048] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:31.371912 14562 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:31.375372  7272 raft_consensus.cc:1240] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:20:31.377521 14542 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:31.380143  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:31.380995  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:31.358958  6651 consensus_queue.cc:260] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.12, Last appended by leader: 12, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:20:31.335382 14531 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:20:31.386801  6549 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:31.387887 14623 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:31.384085  6651 raft_consensus.cc:3060] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:20:31.363464  6404 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57900 (request call id 193) took 13460 ms (client timeout 15000 ms). Trace:
I20260709 12:20:31.406618  6550 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:31.408615  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:31.409365  6653 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:20:31.411904  6581 raft_consensus.cc:1240] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 23. Current term is 24. Ops: []
W20260709 12:20:31.330261 14647 consensus_meta.cc:287] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.568s	user 0.009s	sys 0.001s
I20260709 12:20:31.409323  6578 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
W20260709 12:20:31.422905  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:31.429867  6578 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:20:31.418587 14708 raft_consensus.cc:3055] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:31.431897 14708 raft_consensus.cc:740] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:31.432288 14721 consensus_queue.cc:1059] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 24.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:31.432732 14708 consensus_queue.cc:260] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:31.436786  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 277091us
I20260709 12:20:31.434108  6473 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:31.438062  6475 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:31.439296  6581 raft_consensus.cc:1240] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:31.458146  6651 raft_consensus.cc:1275] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 25 index: 12. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 12:20:31.478350  6578 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:20:31.478960  6475 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:31.400625  6404 rpcz_store.cc:269] 0709 12:20:17.907564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:17.908429 (+   865us) service_pool.cc:224] Handling call
0709 12:20:18.035897 (+127468us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b540008acd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df3090 00007f2875e91f3a 00007f287bc47e9c 00007f287bd4604e 00007f287a2072a4 00007f287a207232 00007f287a207161 00007f287a2070dc 00007f287a206392 00007f28741c448c 00007f28741c38c6
0709 12:20:31.363363 (+13327466us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:24.003741 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:24.004885 (+  1144us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:24.004953 (+    68us) write_op.cc:435] Acquired schema lock
0709 12:20:24.004994 (+    41us) tablet.cc:662] Decoding operations
0709 12:20:24.018550 (+ 13556us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:24.018679 (+   129us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:24.018711 (+    32us) tablet.cc:685] Acquiring locks for 24 operations
0709 12:20:24.107004 (+ 88293us) tablet.cc:701] Row locks acquired
0709 12:20:24.108109 (+  1105us) write_op.cc:260] PREPARE: finished
0709 12:20:24.121062 (+ 12953us) write_op.cc:270] Start()
0709 12:20:24.216926 (+ 95864us) write_op.cc:276] Timestamp: P: 1783599624115620 usec, L: 0
0709 12:20:24.231151 (+ 14225us) op_driver.cc:348] REPLICATION: starting
0709 12:20:24.269141 (+ 37990us) log.cc:844] Serialized 9271 byte log entry
0709 12:20:24.383996 (+114855us) spinlock_profiling.cc:243] Waited 28.6 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2872d7f0ac 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b
0709 12:20:25.121934 (+737938us) op_driver.cc:464] REPLICATION: finished
0709 12:20:25.831906 (+709972us) write_op.cc:301] APPLY: starting
0709 12:20:25.843382 (+ 11476us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:27.579354 (+1735972us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:27.579412 (+    58us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:28.658081 (+1078669us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:28.816429 (+158348us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=48,key_file_lookups=48,delta_file_lookups=0,mrs_lookups=0
0709 12:20:28.816514 (+    85us) write_op.cc:312] APPLY: finished
0709 12:20:28.817068 (+   554us) log.cc:844] Serialized 212 byte log entry
0709 12:20:28.837874 (+ 20806us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:28.838095 (+   221us) write_op.cc:454] Released schema lock
0709 12:20:28.838364 (+   269us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:28.916005 (+ 77641us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:28.916835 (+   830us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":56162,"reported_tablets":53,"spinlock_wait_cycles":34622336,"child_traces":[["op",{"apply.queue_time_us":626109,"apply.run_cpu_time_us":57060,"apply.run_wall_time_us":3105380,"cfile_cache_hit":98,"cfile_cache_hit_bytes":104255,"num_ops":24,"prepare.queue_time_us":101799,"prepare.run_cpu_time_us":23915,"prepare.run_wall_time_us":560102,"raft.queue_time_us":315913,"raft.run_cpu_time_us":33507,"raft.run_wall_time_us":182012,"replication_time_us":890520,"spinlock_wait_cycles":65712512,"thread_start_us":818768,"threads_started":5,"wal-append.queue_time_us":312877,"wal-append.run_cpu_time_us":43947,"wal-append.run_wall_time_us":2631972}]]}
I20260709 12:20:31.417959 14623 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:20:31.483641  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 49637.4us
I20260709 12:20:31.442644 14708 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:31.488721  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:31.490520  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:31.492165  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:31.419202 14647 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:31.352118 14679 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:31.495721  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:31.514429  6584 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 20. Current term is 21. Ops: []
W20260709 12:20:31.483711 14650 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 26 current term: 26 is higher.
I20260709 12:20:31.482972 14739 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:31.483556 14666 consensus_queue.cc:1059] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 12:20:31.516968  6503 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:20:31.517952 14739 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:31.517570 14703 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:31.520773  6503 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:20:31.527595  6649 raft_consensus.cc:1240] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:31.437593 14619 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.081s
I20260709 12:20:31.523458  6651 raft_consensus.cc:1275] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 14. (term mismatch)
I20260709 12:20:31.496483 14702 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:31.558400 14702 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:31.531177  6653 raft_consensus.cc:1275] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:20:31.495883 14555 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:20:31.477751 14448 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:31.560103 14448 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:31.560374  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:31.497695 14562 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:31.518409 14739 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:31.534369  6651 pending_rounds.cc:85] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 13
I20260709 12:20:31.480546 14580 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.046s
I20260709 12:20:31.538743 14679 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:31.530488 14703 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:31.529304 14623 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.493818 14693 consensus_queue.cc:1059] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 12:20:31.549589 14443 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:31.515669 14704 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:31.564648  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:31.587641  6581 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:20:31.522163  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 303439us
I20260709 12:20:31.564698  6651 pending_rounds.cc:107] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 25.14
I20260709 12:20:31.567485  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:31.516079 14682 consensus_queue.cc:1048] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 12:20:31.535465 14647 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:31.562170 14448 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:31.575552 14562 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:31.608569 14562 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:31.610066  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:31.576676 14531 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:31.612884  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:31.612636 14531 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:31.614845 14531 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:31.593638  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:31.618440  6653 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:31.594264  6651 raft_consensus.cc:2889] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:31.599677  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:31.624049  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:31.573498 14745 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.585522 14668 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:31.606698  7273 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 12:20:31.624675  6500 raft_consensus.cc:2393] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 25: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:31.651503  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:31.652940  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:31.653348  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:31.654299  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:31.626062  6502 raft_consensus.cc:1275] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:20:31.655820  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 28 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:31.627179  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:31.627696  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:31.657382  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:31.658151  6509 raft_consensus.cc:2376] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 25. Re-sending same reply.
I20260709 12:20:31.659235  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:31.628012  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:31.628791  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:31.652997  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:31.662424  6578 raft_consensus.cc:2410] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:20:31.653646  6576 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 23.
I20260709 12:20:31.669039  6508 raft_consensus.cc:2393] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 23: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:31.672081  6576 raft_consensus.cc:1275] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 26 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 12:20:31.672004  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:31.676417  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:31.677991  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:31.688385 14717 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.689005 14717 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:31.691275 14717 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:31.709052  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:31.711017  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 28 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:31.712939 14562 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:31.716025 14562 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:31.722235 14440 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:31.724473 14443 consensus_queue.cc:1048] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:20:31.725394 14443 consensus_queue.cc:1243] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer f2b06aebe97249a4877d798b8118ece4 log is divergent from this leader: its last log entry 24.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:20:31.729660 14536 raft_consensus.cc:3055] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:31.730191  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:31.730249 14536 raft_consensus.cc:740] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:31.732635 14536 consensus_queue.cc:260] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:31.617036 14647 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:31.617275 14704 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:31.748898 14691 raft_consensus.cc:3055] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:31.749400 14691 raft_consensus.cc:740] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:31.750269 14691 consensus_queue.cc:260] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:31.687282  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:31.766923  6508 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 24.
I20260709 12:20:31.768203  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:31.769308  6655 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 25. Current term is 26. Ops: []
I20260709 12:20:31.629689 14714 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:31.771327  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:31.772766  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:31.587653 14623 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:31.630213 14448 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.152s
I20260709 12:20:31.784060 14714 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:31.807859  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:31.823323  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:31.823813  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 79755.5us
W20260709 12:20:31.827512  6472 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.011s	sys 0.044s
W20260709 12:20:31.828083  6472 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.012s	sys 0.044s
I20260709 12:20:31.835539  6581 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:20:31.861383  6548 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:31.873618  6575 raft_consensus.cc:1275] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:20:31.874434  6506 raft_consensus.cc:1240] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:31.875053  6577 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:20:31.875427  6548 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:31.881320  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 399232us
I20260709 12:20:31.885372  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:31.885658  6653 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 18.
I20260709 12:20:31.888487  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:31.890169  6548 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:31.890616  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:31.891371  6653 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:31.876185  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:31.666512 14745 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:31.806398 14675 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.857272 14536 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:31.857254 14702 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:31.868021 14691 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:31.904196  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:31.904688  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:31.920737  6509 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 23.
I20260709 12:20:31.896057  6505 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:20:31.872546 14521 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:31.868821 14619 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.912063  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:31.857933 14731 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:20:31.913568  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 49942.4us
I20260709 12:20:31.917740  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:31.917829  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 51711.9us
I20260709 12:20:31.918028  6502 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 27.
I20260709 12:20:31.897496  6548 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:31.924124  6508 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 26.
I20260709 12:20:31.919210  6653 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 18.
I20260709 12:20:31.868633 14555 consensus_queue.cc:1048] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 12:20:31.931325  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:31.935886  6509 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 25.
I20260709 12:20:31.925938  6473 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:31.894512 14647 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:31.938418  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:31.940660  6508 raft_consensus.cc:2410] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:20:31.886893 14721 consensus_queue.cc:1048] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:20:31.902654 14690 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:31.947184  6655 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:31.948403  6509 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:20:31.949694  6509 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 22. Current term is 23. Ops: [22.17-22.17]
I20260709 12:20:31.906116 14486 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:31.906725 14683 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:31.916402 14687 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:31.952875  6508 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:31.898162  6622 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:20:32.066494  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 168426us
I20260709 12:20:32.067595  6622 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:31.945034 14448 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.012086  7238 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:32.013296  6652 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:20:32.083438 14746 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:20:32.086853  6622 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:32.092121  6655 raft_consensus.cc:1275] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
W20260709 12:20:32.143520  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 205471us
I20260709 12:20:32.147866  6653 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
W20260709 12:20:32.152266  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 65482.7us
I20260709 12:20:31.912196 14675 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:31.954295 14728 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:20:32.017683 14733 log.cc:927] Time spent T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 12:20:32.230439  6507 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.018s	sys 0.059s
W20260709 12:20:32.249305  6507 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.018s	sys 0.059s
I20260709 12:20:32.234102  6508 raft_consensus.cc:1275] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 12:20:32.235417  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 108929us
I20260709 12:20:31.946617 14542 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:20:32.254669  6508 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:20:31.935917 14731 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:31.935842 14619 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:32.258246  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 246262us
I20260709 12:20:32.102877 14703 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:32.212864 14666 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.057s	sys 0.103s
I20260709 12:20:32.280572  7238 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:32.240028 14650 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:32.238256 14435 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:32.213655 14448 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:32.074191 14748 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:32.012719 14683 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:32.270951 14704 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:32.280150 14189 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.146790 14687 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:32.238512 14647 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.065s
I20260709 12:20:32.269448 14757 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:32.256076 14717 consensus_queue.cc:1059] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 25.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.065s
I20260709 12:20:32.013121  6473 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:20:32.270180  6503 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.020s	sys 0.047s
W20260709 12:20:32.407553  6503 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.020s	sys 0.047s
W20260709 12:20:32.410193  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 512746us
W20260709 12:20:32.404160  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 390873us
W20260709 12:20:32.275633  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:32.417524  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:32.418960  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:32.419312  6652 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:20:32.294026  7238 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:32.424773  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 130739us
I20260709 12:20:32.350363  7240 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:32.405985  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 126897us
I20260709 12:20:32.349323  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:32.451805 14531 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.442615 14330 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:32.467835 14330 raft_consensus.cc:697] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:32.303273 14435 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:32.471410 14435 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:20:32.292788 14666 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.775s	user 0.057s	sys 0.104s
I20260709 12:20:32.475536  6508 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 12:20:32.333366 14675 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:32.340152 14448 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:32.319870 14679 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:32.480989 14706 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:20:32.481791  7274 raft_consensus.cc:1275] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:20:32.476581 14666 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:20:32.348045 14708 consensus_meta.cc:287] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.862s	user 0.015s	sys 0.000s
I20260709 12:20:32.363410 14687 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:32.341481 14189 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:32.454598 14677 consensus_queue.cc:1059] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:32.338289 14683 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:32.457916 14580 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:32.365437 14619 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:32.356917 14623 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:32.452349  7273 raft_consensus.cc:2393] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 20: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:20:32.337381 14668 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:32.461198  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:32.465179 14496 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:32.502022  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 151625us
W20260709 12:20:32.502641  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 75034.7us
W20260709 12:20:32.349844 14751 log.cc:927] Time spent T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.398s	user 0.000s	sys 0.001s
W20260709 12:20:32.338722 14754 log.cc:927] Time spent T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.250s	user 0.000s	sys 0.000s
I20260709 12:20:32.520197  6654 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3900) took 1145 ms. Trace:
I20260709 12:20:32.520859  6654 rpcz_store.cc:276] 0709 12:20:31.374893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:31.375513 (+   620us) service_pool.cc:224] Handling call
0709 12:20:31.375805 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:31.376142 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:31.376172 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:31.376678 (+   506us) log.cc:844] Serialized 26 byte log entry
0709 12:20:31.535148 (+158470us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:31.535187 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:31.535259 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:32.519546 (+984287us) raft_consensus.cc:1685] finished
0709 12:20:32.519604 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:32.519926 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1035,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":82,"thread_start_us":157108,"threads_started":1,"wal-append.queue_time_us":537823}
I20260709 12:20:32.522734  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:32.523240  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49792 (request call id 1414) took 1124 ms. Trace:
W20260709 12:20:32.333719 14756 log.cc:927] Time spent T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:20:32.524860  6649 raft_consensus.cc:1240] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:32.523802  6656 rpcz_store.cc:276] 0709 12:20:31.398521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:31.398679 (+   158us) service_pool.cc:224] Handling call
0709 12:20:31.399035 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:31.399706 (+   671us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:31.399739 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:31.400390 (+   651us) log.cc:844] Serialized 26 byte log entry
0709 12:20:31.575713 (+175323us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:20:31.575759 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:31.575840 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:32.522560 (+946720us) raft_consensus.cc:1685] finished
0709 12:20:32.522620 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:32.523000 (+   380us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":118,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"thread_start_us":175015,"threads_started":1,"wal-append.queue_time_us":523824}
I20260709 12:20:32.470014 14330 consensus_queue.cc:237] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:32.364886 14745 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:32.335546 14755 log.cc:927] Time spent T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.418s	user 0.001s	sys 0.000s
W20260709 12:20:32.334914 14747 log.cc:927] Time spent T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.880s	user 0.009s	sys 0.005s
I20260709 12:20:32.307448 14731 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:32.530915  6658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3902) took 1125 ms. Trace:
I20260709 12:20:32.484678 14531 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:32.296664 14728 raft_consensus.cc:697] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 23 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:32.535588 14531 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:32.347473 14748 raft_consensus.cc:697] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:32.484050 14706 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:32.537592  6650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3885) took 1405 ms. Trace:
I20260709 12:20:32.538087 14706 raft_consensus.cc:697] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 18 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:32.507802  6500 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 24.
W20260709 12:20:32.333573 14750 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.325s	user 0.000s	sys 0.001s
I20260709 12:20:32.360144 14714 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:32.373427 14749 log.cc:927] Time spent T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 12:20:32.541719  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:32.372634 14739 consensus_meta.cc:287] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.787s	user 0.008s	sys 0.009s
I20260709 12:20:32.542552 14706 consensus_queue.cc:237] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:32.543977 14739 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:32.544523  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:32.514767  6576 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:20:32.545866  6504 raft_consensus.cc:1240] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:32.505930 13554 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:20:32.540778  6655 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 23.
I20260709 12:20:32.507700 14717 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:32.514874  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:32.550999  6550 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:32.551374  6579 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 25.
I20260709 12:20:32.551229 14717 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:32.535207 14496 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:32.546608  6656 raft_consensus.cc:2410] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 24 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:20:32.516765 14189 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:32.539045 14748 consensus_queue.cc:237] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:32.554628 14496 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:32.538232  6650 rpcz_store.cc:276] 0709 12:20:31.132582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:31.133461 (+   879us) service_pool.cc:224] Handling call
0709 12:20:31.134014 (+   553us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:31.148856 (+ 14842us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:31.148900 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:31.151750 (+  2850us) log.cc:844] Serialized 26 byte log entry
0709 12:20:31.415834 (+264084us) spinlock_profiling.cc:243] Waited 2.22 ms on lock 0x7b1800483040. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:20:31.513437 (+ 97603us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:31.513474 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:31.513533 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:32.334944 (+821411us) spinlock_profiling.cc:243] Waited 8.31 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:20:32.536952 (+202008us) raft_consensus.cc:1685] finished
0709 12:20:32.537002 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:32.537336 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14391,"raft.queue_time_us":16215,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":10243,"spinlock_wait_cycles":24214656,"thread_start_us":97429,"threads_started":1,"wal-append.queue_time_us":714897}
I20260709 12:20:32.535815 14728 consensus_queue.cc:237] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:32.531556  6658 rpcz_store.cc:276] 0709 12:20:31.405096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:31.405224 (+   128us) service_pool.cc:224] Handling call
0709 12:20:31.405598 (+   374us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:20:31.406000 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:31.406030 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:20:31.406650 (+   620us) log.cc:844] Serialized 52 byte log entry
0709 12:20:31.823287 (+416637us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:31.823336 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:31.823755 (+   419us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:32.530101 (+706346us) raft_consensus.cc:1685] finished
0709 12:20:32.530157 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:32.530553 (+   396us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2478,"raft.queue_time_us":1171,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":3658,"thread_start_us":416223,"threads_started":1,"wal-append.queue_time_us":540238}
I20260709 12:20:32.559090 14739 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:32.561691 14768 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:32.565760  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:32.554632 14717 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:32.594712  6650 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 29: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:32.575906 14768 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:32.601619 14768 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:32.601663 14679 consensus_queue.cc:1059] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.007s
I20260709 12:20:32.608909  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:32.611266  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 466632us
I20260709 12:20:32.614218 14774 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:32.616163  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:32.618748 14683 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:20:32.621622  6475 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:32.616176 14536 consensus_meta.cc:287] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.702s	user 0.006s	sys 0.017s
I20260709 12:20:32.619681  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:32.628952 14739 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 12:20:32.624158 14694 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:32.636844 14728 raft_consensus.cc:3055] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:32.637576 14728 raft_consensus.cc:740] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:32.615406 14774 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:32.647119 14774 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:32.651615 14694 consensus_queue.cc:1059] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:20:32.653967 14760 log.cc:927] Time spent T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.330s	user 0.004s	sys 0.000s
I20260709 12:20:32.656157  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:32.656586 14762 log.cc:927] Time spent T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.318s	user 0.000s	sys 0.003s
I20260709 12:20:32.658769  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 1076) took 1171 ms. Trace:
I20260709 12:20:32.635839 14775 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.008s
I20260709 12:20:32.659235  6507 rpcz_store.cc:276] 0709 12:20:31.487335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:31.487450 (+   115us) service_pool.cc:224] Handling call
0709 12:20:31.487796 (+   346us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:20:31.502477 (+ 14681us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:31.502529 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:20:31.519134 (+ 16605us) log.cc:844] Serialized 52 byte log entry
0709 12:20:32.249918 (+730784us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:20:32.585521 (+335603us) log.cc:844] Serialized 14 byte log entry
0709 12:20:32.586486 (+   965us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:32.586623 (+   137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:32.655561 (+ 68938us) spinlock_profiling.cc:243] Waited 25.6 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:20:32.658263 (+  2702us) raft_consensus.cc:1685] finished
0709 12:20:32.658390 (+   127us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:32.658555 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23007,"prepare.queue_time_us":398197,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":81,"raft.queue_time_us":14503,"raft.run_cpu_time_us":275,"raft.run_wall_time_us":7874,"spinlock_wait_cycles":58811264,"thread_start_us":861289,"threads_started":2,"wal-append.queue_time_us":805002}
W20260709 12:20:32.662549 14763 log.cc:927] Time spent T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.303s	user 0.001s	sys 0.000s
I20260709 12:20:32.662681  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:32.663519  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:32.665289  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:32.666369  6507 raft_consensus.cc:1275] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 26 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 12:20:32.628525 14435 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:32.672966  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 101059us
I20260709 12:20:32.689174  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:32.693110  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 3269) took 1023 ms. Trace:
I20260709 12:20:32.693647  6509 rpcz_store.cc:276] 0709 12:20:31.645066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:31.950322 (+305256us) service_pool.cc:224] Handling call
0709 12:20:31.950638 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:32.143658 (+193020us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:32.143692 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:32.213033 (+ 69341us) log.cc:844] Serialized 26 byte log entry
0709 12:20:32.587049 (+374016us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:20:32.587141 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:32.587335 (+   194us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:32.666413 (+ 79078us) raft_consensus.cc:1685] finished
0709 12:20:32.666431 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:32.668765 (+  2334us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":204475,"raft.queue_time_us":195548,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":33684,"thread_start_us":169059,"threads_started":1,"wal-append.queue_time_us":357612}
I20260709 12:20:32.696422 14706 consensus_queue.cc:1059] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
W20260709 12:20:32.698607 14486 consensus_meta.cc:287] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.689s	user 0.016s	sys 0.006s
I20260709 12:20:32.699276 14486 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:32.707741  6657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3901) took 1332 ms. Trace:
W20260709 12:20:32.709177  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 89732.9us
I20260709 12:20:32.710633  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:32.712670 14448 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:32.698518 14706 consensus_queue.cc:1243] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 19.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:20:32.725613  6649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 3326) took 1194 ms. Trace:
I20260709 12:20:32.726123  6649 rpcz_store.cc:276] 0709 12:20:31.531526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:31.597929 (+ 66403us) service_pool.cc:224] Handling call
0709 12:20:32.524180 (+926251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:32.525595 (+  1415us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:32.725419 (+199824us) spinlock_profiling.cc:243] Waited 920 ms on lock 0x7b54000758d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:32.725552 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2116435328}
I20260709 12:20:32.726837 14768 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:32.730079 14486 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.730973 14486 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:32.733592  6504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 3484) took 1133 ms. Trace:
I20260709 12:20:32.734112  6504 rpcz_store.cc:276] 0709 12:20:31.600495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:31.600625 (+   130us) service_pool.cc:224] Handling call
0709 12:20:31.600943 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:32.546510 (+945567us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:32.733213 (+186703us) spinlock_profiling.cc:243] Waited 944 ms on lock 0x7b540001c7d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:32.733520 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2171971456}
I20260709 12:20:32.736939 14728 consensus_queue.cc:260] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:32.738489 14440 consensus_meta.cc:287] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 1.015s	user 0.003s	sys 0.022s
I20260709 12:20:32.739220 14728 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:32.746119  6507 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:20:32.756584 14496 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.757347 14496 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:32.757867  7272 raft_consensus.cc:3055] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:32.758225  7272 raft_consensus.cc:740] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
W20260709 12:20:32.759462 14691 consensus_meta.cc:287] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.721s	user 0.011s	sys 0.015s
I20260709 12:20:32.759253  7272 consensus_queue.cc:260] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:32.759749 14496 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:32.756789  7273 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:20:32.764952  7272 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:32.770673  7266 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 18 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 12:20:32.771451  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:32.772899  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:32.773573 14694 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.773888  6548 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:32.774902  6655 raft_consensus.cc:1275] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:20:32.775391 14330 consensus_queue.cc:1059] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:32.774448 14694 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:32.777580 14330 consensus_queue.cc:1243] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer 20a202c76244485fbc8f1610473db8b9 log is divergent from this leader: its last log entry 18.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:20:32.775887  6548 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:32.742221 14748 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.708510  6657 rpcz_store.cc:276] 0709 12:20:31.375257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:31.376728 (+  1471us) service_pool.cc:224] Handling call
0709 12:20:31.377289 (+   561us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:31.377678 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:20:31.582799 (+205121us) log.cc:844] Serialized 14 byte log entry
0709 12:20:31.922063 (+339264us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:31.922574 (+   511us) log.cc:844] Serialized 26 byte log entry
0709 12:20:31.922705 (+   131us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:20:31.922747 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:31.922823 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:32.468282 (+545459us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b18004d8900. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:20:32.661582 (+193300us) spinlock_profiling.cc:243] Waited 2.63 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:20:32.707268 (+ 45686us) raft_consensus.cc:1685] finished
0709 12:20:32.707298 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:32.707561 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":499479,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":7894,"raft.run_cpu_time_us":410,"raft.run_wall_time_us":411,"spinlock_wait_cycles":32406528,"thread_start_us":543413,"threads_started":2,"wal-append.queue_time_us":759490}
I20260709 12:20:32.787753  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:32.787987 14536 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:32.792763 14542 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:32.792883  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:32.793857  6509 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:20:32.794623  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:32.794796  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:32.794981  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:32.795771  6509 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:32.795897  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:32.797533  6506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 3504) took 1090 ms. Trace:
I20260709 12:20:32.788908 14435 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:32.797952  6506 rpcz_store.cc:276] 0709 12:20:31.706944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:32.796407 (+1089463us) service_pool.cc:224] Handling call
0709 12:20:32.796799 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:32.797072 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:32.797094 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:32.797148 (+    54us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:32.797179 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:32.797226 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:32.797366 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:20:32.795926  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:32.805974  7272 raft_consensus.cc:1275] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 12:20:32.806341  7272 pending_rounds.cc:85] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9: Aborting all ops after (but not including) 14
I20260709 12:20:32.807125 14486 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:32.807405  6657 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 25. Current term is 26. Ops: []
I20260709 12:20:32.808261  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:32.808676  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:32.809191  6506 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 29.
I20260709 12:20:32.811908  6658 raft_consensus.cc:1217] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Deduplicated request from leader. Original: 20.11->[22.12-25.14]   Dedup: 22.12->[23.13-25.14]
I20260709 12:20:32.812532  6658 pending_rounds.cc:85] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 12
I20260709 12:20:32.812826  6658 pending_rounds.cc:107] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 24.13
I20260709 12:20:32.813138  6658 raft_consensus.cc:2889] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:20:32.813251 14778 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:20:32.815587  6506 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:20:32.816574  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:32.817000 14448 consensus_queue.cc:1048] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:32.817819 14668 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:20:32.818372 14668 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:32.818965 14668 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:32.822359 14748 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:32.824678  6507 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 23.
I20260709 12:20:32.802848 14189 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.025s
I20260709 12:20:32.828513  6504 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:20:32.806633  7272 pending_rounds.cc:107] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 12:20:32.832333  7272 raft_consensus.cc:2889] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:32.823786 14692 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.005s
I20260709 12:20:32.845736  6501 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 25.
I20260709 12:20:32.849471  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:32.828685 14694 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:32.857189  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:32.873412 14587 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:32.874864 14748 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:32.875547 14748 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.876276  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3908) took 1255 ms. Trace:
I20260709 12:20:32.877130  6652 rpcz_store.cc:276] 0709 12:20:31.620705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:32.213283 (+592578us) service_pool.cc:224] Handling call
0709 12:20:32.217456 (+  4173us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:32.420352 (+202896us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:32.875973 (+455621us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b54000744d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:32.876183 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":460572928}
I20260709 12:20:32.882256 14189 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.008s
W20260709 12:20:32.893559  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 117853us
I20260709 12:20:32.876080 14748 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:32.897473  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:32.903385  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:32.897490 14562 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 12:20:32.909739 14748 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:32.910775  6548 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:32.911736 14486 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:32.913523  6548 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:32.915958 14486 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:32.886600 14668 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:32.938113 14668 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:32.927392  6500 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45880 (request call id 3266) took 1281 ms. Trace:
I20260709 12:20:32.945672  6500 rpcz_store.cc:276] 0709 12:20:31.643873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:31.677700 (+ 33827us) service_pool.cc:224] Handling call
0709 12:20:32.918251 (+1240551us) spinlock_profiling.cc:243] Waited 829 ms on lock 0x7b54001e73d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:32.925349 (+  7098us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1905686912}
I20260709 12:20:32.952659 14506 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:32.956913  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:32.958053  6584 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 20.
W20260709 12:20:32.969585 14753 log.cc:927] Time spent T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.661s	user 0.003s	sys 0.000s
W20260709 12:20:32.971648 14769 log.cc:927] Time spent T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.610s	user 0.002s	sys 0.000s
W20260709 12:20:32.973347  6501 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 3490) took 1346 ms (client timeout 1709 ms). Trace:
W20260709 12:20:32.973986  6501 rpcz_store.cc:269] 0709 12:20:31.627063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:31.627721 (+   658us) service_pool.cc:224] Handling call
0709 12:20:32.973029 (+1345308us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400388454. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:32.973255 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2481317632}
I20260709 12:20:32.977934 14675 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.978607 14675 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:32.982393 14521 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:32.983227 14521 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:32.985888 14521 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:32.982460 14675 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:32.982393 14687 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.005s
W20260709 12:20:32.989189  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 75813.8us
I20260709 12:20:32.990141  6548 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:32.991657 14708 raft_consensus.cc:3055] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:20:32.992225 14708 raft_consensus.cc:740] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:32.992446 14521 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:32.993263 14521 consensus_queue.cc:1243] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 21.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:20:32.997534 14721 raft_consensus.cc:3055] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:32.998140 14721 raft_consensus.cc:740] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:33.001441  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:33.002218  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:33.003585 14668 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:33.004035  6622 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:33.005384 14668 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:33.010691  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:33.010995  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:33.012490  6578 raft_consensus.cc:1240] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 12:20:33.024511  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:33.026603 14713 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:20:33.027577  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:33.028258  6505 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:33.022272 14668 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:33.029160 14668 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:33.032212  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:33.033190  6509 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:20:33.033944  6502 raft_consensus.cc:1275] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 23 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 12:20:33.034621  6502 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 3505) took 1327 ms. Trace:
I20260709 12:20:33.035010  6502 rpcz_store.cc:276] 0709 12:20:31.707215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:33.033009 (+1325794us) service_pool.cc:224] Handling call
0709 12:20:33.033449 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:33.034357 (+   908us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:33.034554 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:20:32.992244 14682 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:33.036053 14682 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:33.036170  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:33.036497 14682 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:20:33.037400 14765 log.cc:927] Time spent T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.698s	user 0.003s	sys 0.000s
I20260709 12:20:33.040208  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 3482) took 1441 ms. Trace:
I20260709 12:20:33.040668  6503 rpcz_store.cc:276] 0709 12:20:31.598986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:31.599196 (+   210us) service_pool.cc:224] Handling call
0709 12:20:31.599709 (+   513us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:31.600138 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:31.600180 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:31.697461 (+ 97281us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b2c00000c88. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:20:31.697926 (+   465us) log.cc:844] Serialized 26 byte log entry
0709 12:20:32.412517 (+714591us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:32.412650 (+   133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:32.412808 (+   158us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:33.036462 (+623654us) spinlock_profiling.cc:243] Waited 23.2 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:20:33.039925 (+  3463us) raft_consensus.cc:1685] finished
0709 12:20:33.039947 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:33.040145 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8574,"raft.queue_time_us":17891,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":30821,"spinlock_wait_cycles":55741440,"thread_start_us":709768,"threads_started":1,"wal-append.queue_time_us":640575}
I20260709 12:20:33.044366 14768 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:33.046388  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:33.047120  6657 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 29.
I20260709 12:20:33.047914  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:33.023882 14675 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.032s
W20260709 12:20:33.029888 14759 log.cc:927] Time spent T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.669s	user 0.000s	sys 0.002s
I20260709 12:20:33.058718  6582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 3422) took 1648 ms. Trace:
I20260709 12:20:33.059160  6582 rpcz_store.cc:276] 0709 12:20:31.409972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:31.410099 (+   127us) service_pool.cc:224] Handling call
0709 12:20:31.414331 (+  4232us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:20:31.419730 (+  5399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:31.419779 (+    49us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:20:31.439053 (+ 19274us) log.cc:844] Serialized 78 byte log entry
0709 12:20:31.918819 (+479766us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:31.918880 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:31.918976 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:33.028970 (+1109994us) spinlock_profiling.cc:243] Waited 1.87 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:20:33.058111 (+ 29141us) raft_consensus.cc:1685] finished
0709 12:20:33.058135 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:33.058508 (+   373us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4946,"raft.queue_time_us":5133,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":102,"spinlock_wait_cycles":4301696,"thread_start_us":399987,"threads_started":1,"wal-append.queue_time_us":824405}
I20260709 12:20:32.999647 14721 consensus_queue.cc:260] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:33.090852 14721 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:33.099773  6501 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 26.
I20260709 12:20:33.106521  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:33.108990  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:33.109750  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:33.110100  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:33.110469  6579 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 25.
I20260709 12:20:33.110786  6576 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 25.
W20260709 12:20:33.141664  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 101300us
I20260709 12:20:33.100317  6505 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 24.
I20260709 12:20:33.154054  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:33.154953  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:33.156520  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 116404us
I20260709 12:20:33.155596  6652 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 23.
I20260709 12:20:33.158883  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:33.078584 14708 consensus_queue.cc:260] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:33.183195  6582 raft_consensus.cc:1275] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:20:33.184651  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:33.190032  7268 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
W20260709 12:20:33.192456 14634 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
I20260709 12:20:33.192700 14675 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:33.193091 14675 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:33.193459 14675 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:33.197453 14521 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:33.198084 14521 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:33.202303  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 56858.5us
I20260709 12:20:33.204783 14692 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:33.205428 14692 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:33.207931 14692 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:33.208736 14683 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:33.209059 14713 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:33.209995 14713 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:33.212590  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:32.400395  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:33.196004 14486 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:33.241044  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:33.241797  6583 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22.
I20260709 12:20:33.242316 14682 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:33.243158  6583 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:20:33.244741  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:33.245527  6577 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 28.
I20260709 12:20:33.246588  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:33.247292  6582 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:20:33.247480  6580 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 24: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:33.246634  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:33.249055  6581 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 27.
W20260709 12:20:33.247872 14785 log.cc:927] Time spent T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:20:33.252547  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:33.255429  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:33.256315 14448 raft_consensus.cc:2764] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:20:33.257225 14448 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:33.258814 14448 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:33.259204 14448 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:33.259553 14448 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:33.234253 14713 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:33.273916  6584 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:20:33.223875 14768 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:33.276011  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:33.246999  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:33.287456  6655 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:33.239401 14708 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:33.255697 14682 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:33.290392  7238 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:33.325791 14684 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:33.326278 14684 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:33.326617 14684 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:33.323892  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:33.313337 14521 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:33.332520 14682 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.005s
W20260709 12:20:33.351372  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 61089.9us
I20260709 12:20:33.350382  6653 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 12:20:33.359054  7238 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:33.372411  6650 raft_consensus.cc:1275] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:20:33.314317 14768 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:33.376348 14687 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:33.350378 14680 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.170s
I20260709 12:20:33.381529 14748 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:33.369647 14521 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:33.393808 14330 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 12:20:33.439993 14443 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:33.440776 14443 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:33.441756 14543 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:33.442337 14543 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:33.443465 14443 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:33.447826 14791 log.cc:927] Time spent T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:20:33.417954 14692 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:33.449843 14800 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:20:33.450109 14692 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:33.450968 14521 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:33.452786 14587 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:33.453060 14521 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:33.453383 14587 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:33.456789 14587 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:33.441558  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:33.438549  7274 raft_consensus.cc:1275] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:20:33.474823  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:33.485214  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:33.475723  7273 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:20:33.423573 14562 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:33.421900 14748 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:33.497740  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 140632us
W20260709 12:20:33.507321  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:33.510643  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:33.512506  6575 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:20:33.492041 14692 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:33.520562  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:33.526765  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 358247us
I20260709 12:20:33.515203 14435 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 12:20:33.535768  6651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3907) took 1914 ms. Trace:
I20260709 12:20:33.492846 14684 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:33.546198  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 173723us
I20260709 12:20:33.546595  6548 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
W20260709 12:20:33.557003  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 157529us
I20260709 12:20:33.540843  6649 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 25.
I20260709 12:20:33.496865 14562 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:33.537993  6651 rpcz_store.cc:276] 0709 12:20:31.547279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:31.655410 (+108131us) service_pool.cc:224] Handling call
0709 12:20:31.655679 (+   269us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:31.656055 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:31.656085 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:31.673735 (+ 17650us) log.cc:844] Serialized 26 byte log entry
0709 12:20:32.990476 (+1316741us) spinlock_profiling.cc:243] Waited 67.5 ms on lock 0x7b180010c480. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:20:33.204781 (+214305us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:20:33.204832 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:33.204929 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:33.446704 (+241775us) raft_consensus.cc:1685] finished
0709 12:20:33.447598 (+   894us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:33.461628 (+ 14030us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3947,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":16016,"spinlock_wait_cycles":155197696,"thread_start_us":167669,"threads_started":1,"wal-append.queue_time_us":1651423}
W20260709 12:20:33.588282  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 280706us
I20260709 12:20:33.589396  6622 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:33.589891  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 43616.7us
I20260709 12:20:33.585455  6475 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:33.596485  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:33.597312 14803 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:33.597813 14803 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:33.598272 14803 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:33.598577  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:33.594645  6475 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:33.600427 14814 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:33.600678 14820 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:33.601274 14820 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:33.601672 14820 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:33.564637 14521 consensus_queue.cc:1059] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 12:20:33.607514  6548 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:33.594763 14506 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:20:33.604739 14692 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:33.608508  6473 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:33.592197 14768 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:33.615841  6473 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:33.622366 14811 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:33.624935 14684 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:33.624109 14692 raft_consensus.cc:697] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 24 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:33.604748 14521 consensus_queue.cc:1243] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 22.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:20:33.625491 14821 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:33.609232 14506 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:33.647501  6655 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 24.
W20260709 12:20:33.648865  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 53712.8us
I20260709 12:20:33.649847 14692 consensus_queue.cc:237] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:33.629354 14713 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:33.643308 14811 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:33.645834  6550 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:33.628304 14675 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:33.645444 14506 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:20:33.663478 14683 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:33.664148 14683 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:33.664558 14803 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:33.655283 14820 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:33.670388 14820 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:33.671703 14821 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:20:33.672878  6548 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:33.673588 14768 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:33.673979  6548 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:33.674362 14768 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:20:33.674614 14721 consensus_meta.cc:287] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.582s	user 0.009s	sys 0.002s
I20260709 12:20:33.674942 14768 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:33.675448 14521 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:33.675736 14486 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:33.676188 14748 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:33.676278 14486 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:33.677836 14811 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:33.677714 14448 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:33.675882 14814 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:33.678593 14486 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:33.685742  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 90030.4us
I20260709 12:20:33.655813 14713 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:33.667905 14803 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:33.675486 14768 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:20:33.697522 14713 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:20:33.735097  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 108471us
I20260709 12:20:33.742697 14668 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:33.697359 14486 consensus_queue.cc:1048] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.073s
I20260709 12:20:33.786518 14634 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:33.791751 14562 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:33.792035 14634 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:33.809994  6649 raft_consensus.cc:1240] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 25. Current term is 26. Ops: []
I20260709 12:20:33.812824  6651 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:20:33.819927 14721 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:33.811895  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:33.822804  6649 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
W20260709 12:20:33.824839  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 127755us
I20260709 12:20:33.821538  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:33.827994  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:33.829814  6654 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 24: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:33.830389  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:33.832433  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:33.833693  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:33.834025  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:33.839395  6651 raft_consensus.cc:2393] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 26: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:20:33.845381  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 229586us
I20260709 12:20:33.820745 14721 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:33.846917  6473 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:33.852404  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:33.841480 14675 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:33.856366  6584 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:20:33.858587  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:33.872449  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:33.856889  6473 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:33.875291  6473 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:33.876595  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:33.856323 14827 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:33.877743 14829 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:33.878038 14827 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:33.878449 14829 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:33.878656 14827 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:33.879094 14829 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:33.880544 14811 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:33.881832 14506 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:33.867599  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:33.873571  6501 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22.
I20260709 12:20:33.872637  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:33.883788  6473 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:33.841295 14683 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:33.882738 14811 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:33.884176  6507 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:33.885684  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:33.889009  6504 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 30: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:33.897619  6584 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:33.890658 14506 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 30 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:33.899461  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:33.899972 14829 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:33.900978  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:33.903050  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:33.903771  7272 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:20:33.907923  6507 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 28.
I20260709 12:20:33.911554  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:33.912760  6581 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:33.932286  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:33.937387  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:33.932251  6549 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
W20260709 12:20:33.941347  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 57651.4us
I20260709 12:20:33.943459  6580 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:33.945470  6509 raft_consensus.cc:2393] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 21: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:20:33.945756  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 90025.3us
I20260709 12:20:33.950408  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:33.950922  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:33.916616 14562 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:33.952937  6577 raft_consensus.cc:2410] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 24 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 12:20:33.930523 14634 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:33.958226  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:33.959326  6507 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:20:33.959983  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:33.959983  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:33.945007 14506 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:20:33.962320  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:33.946753 14801 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:33.965797 14829 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:33.966140  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:33.966848  6577 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:20:33.968092  6582 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 25. Current term is 26. Ops: [25.15-25.15]
I20260709 12:20:33.968619  6577 pending_rounds.cc:85] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 12
I20260709 12:20:33.969046  6577 pending_rounds.cc:107] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 12:20:33.969408  6577 raft_consensus.cc:2889] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:33.969939  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:33.970817  6580 raft_consensus.cc:2393] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 25: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:33.964133  6579 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22.
I20260709 12:20:33.972736 14812 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:33.973662 14812 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:33.989589  6581 raft_consensus.cc:2468] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 21.
I20260709 12:20:33.997278  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:33.998952  7266 raft_consensus.cc:1240] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
W20260709 12:20:34.001072  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 56335.1us
I20260709 12:20:33.980240 14801 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:34.002709 14801 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:34.002884 14834 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:34.003440 14834 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:34.005535 14834 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:34.012853  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:34.014357  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:34.015175  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:34.015452  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:34.015939  6506 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 23.
I20260709 12:20:34.025511  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:34.016376  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:34.017124  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:34.002712 14812 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:34.017938  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:34.018720  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:34.030287  6505 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 24: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:34.019459  6473 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:34.025552  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:34.032024  6578 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 26.
I20260709 12:20:33.960906 14562 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:34.034132  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:34.035069  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:34.035980 14562 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:34.042922  6581 raft_consensus.cc:2393] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 21: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:34.043131 14713 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:34.043423 14634 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:34.044171  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:34.044924  6508 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:34.045318 14721 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:34.045890  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:34.046188  6576 raft_consensus.cc:1240] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:34.028772  6509 raft_consensus.cc:2376] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 25. Re-sending same reply.
I20260709 12:20:34.049490  6584 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 24.
I20260709 12:20:34.050422  6507 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 24.
I20260709 12:20:34.029565  6503 raft_consensus.cc:2376] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21. Re-sending same reply.
I20260709 12:20:34.057649  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:34.073436 14781 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:20:34.074237  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:34.074559 14713 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:34.077278 14827 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:34.079629 14827 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:34.072063  6506 raft_consensus.cc:1275] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 15. (term mismatch)
W20260709 12:20:34.082170  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 62961.7us
I20260709 12:20:34.082463  6505 pending_rounds.cc:85] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 11
I20260709 12:20:34.082927  6506 pending_rounds.cc:85] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 14
I20260709 12:20:34.093092  6506 pending_rounds.cc:107] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 17.15
I20260709 12:20:34.093375  6506 raft_consensus.cc:2889] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:34.085052  6505 pending_rounds.cc:107] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 12:20:34.092029  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:34.095824  6505 raft_consensus.cc:2889] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:34.088558  7238 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:34.112687 14813 consensus_queue.cc:1059] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 12:20:34.124588  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:20:34.091822 14809 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.009s	sys 0.026s
I20260709 12:20:34.089188 14828 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:20:34.131855  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 168205us
I20260709 12:20:34.139055  6473 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:34.140262  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:34.138540  6622 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:20:34.133587 14809 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.010s	sys 0.026s
I20260709 12:20:34.147110 13554 raft_consensus.cc:2804] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:34.148010 13554 raft_consensus.cc:697] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 21 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:34.154523  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:34.155166  6575 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:20:34.082146 14801 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:34.120704 14666 consensus_queue.cc:1059] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.023s
I20260709 12:20:34.135486 14828 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:34.160275 13554 consensus_queue.cc:237] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:34.162581 14189 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:34.143643 14562 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.022s
I20260709 12:20:34.174453  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:34.182969 14838 log.cc:927] Time spent T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:20:34.178069 14813 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:20:34.185828 14813 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:34.186357 14813 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:34.179661 14189 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:34.191380 14189 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:34.176671  6622 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:34.154371 14748 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 12:20:34.163863 14828 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:20:34.220543  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 132022us
W20260709 12:20:34.224862  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 63150.2us
I20260709 12:20:34.228289  6506 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:20:34.221939  7238 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:34.233642  6473 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:34.250484  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:34.200171 14768 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:34.251554  6578 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 24: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:34.253286  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:34.255010  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:34.255831  6584 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:20:34.231483 14851 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:34.258618  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:34.264995  6584 raft_consensus.cc:2393] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 26: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:34.242908 14634 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:34.262609  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:34.266326  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:20:34.266917  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 90359.3us
I20260709 12:20:34.267966  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:20:34.269202  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:34.272805  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:34.273522  6581 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 25.
I20260709 12:20:34.274494  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:34.251184 14801 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:34.278518  6548 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:34.276564  6583 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 29.
I20260709 12:20:34.279285  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:34.279606  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:34.280046  6652 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:20:34.261907 14843 log.cc:927] Time spent T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:20:34.280519  6584 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:34.276187  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:34.289731  6578 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:20:34.294796  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:34.252153 14694 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.026s
I20260709 12:20:34.306530  6654 raft_consensus.cc:2393] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 25: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:34.330498  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:34.331207  6649 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:20:34.312983 14809 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:34.340767  6506 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 25. Current term is 26. Ops: []
I20260709 12:20:34.303772 14802 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:34.305714 14854 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:34.349035  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 79463.4us
I20260709 12:20:34.332204 14855 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:20:34.355346  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 87532.2us
I20260709 12:20:34.356060  6548 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:34.357940  6548 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:34.362262  6548 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:34.316215 14840 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.028s
I20260709 12:20:34.337715 14847 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:34.367897 14812 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:34.368042 14713 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:34.368700 14713 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:34.369217 14713 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:34.297039 14851 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:34.384112 14845 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:34.385389 14845 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:34.386799 14853 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.098s
I20260709 12:20:34.373392 14801 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:34.367308  6548 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:34.400888 14683 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:34.415938 14757 raft_consensus.cc:3055] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:34.425484  6584 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 26.
I20260709 12:20:34.376463 14849 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:20:34.432717 14844 log.cc:927] Time spent T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.233s	user 0.001s	sys 0.000s
W20260709 12:20:34.450265 14859 log.cc:927] Time spent T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 12:20:34.416549 14757 raft_consensus.cc:740] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:34.451244 14854 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
W20260709 12:20:34.453624 14860 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 12:20:34.350600 14809 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:34.421023 14801 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:34.489564  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 122321us
I20260709 12:20:34.497709  6548 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:34.477789 14802 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:20:34.501122  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 135226us
I20260709 12:20:34.505239  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:34.506069 14668 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:34.506472 14668 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:34.506821 14668 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:34.513175  6548 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:34.514470  6548 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:34.514508 14809 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:34.515319 14682 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:34.516177 14803 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:34.516531 14812 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:20:34.516881 14803 raft_consensus.cc:697] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:34.517997 14803 consensus_queue.cc:237] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:34.519222 14813 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:34.497711 14864 log.cc:927] Time spent T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:20:34.524703  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:34.529780  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:34.500908 14801 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:34.531332  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:34.537497  6578 raft_consensus.cc:1240] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 12:20:34.481592 14757 consensus_queue.cc:260] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:34.531432 14869 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:34.541668 14869 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:34.542039 14869 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:34.529855 14692 consensus_queue.cc:1059] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.015s
I20260709 12:20:34.531759 14768 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:34.549242 14757 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:20:34.549475 14789 log.cc:927] Time spent T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.932s	user 0.005s	sys 0.000s
I20260709 12:20:34.549865 14713 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:34.527325 14802 raft_consensus.cc:697] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:34.557305  6652 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 30.
I20260709 12:20:34.559657 14543 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:34.560405 14802 consensus_queue.cc:237] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:34.583623 14692 raft_consensus.cc:3055] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:34.584108 14692 raft_consensus.cc:740] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:34.585644 14692 consensus_queue.cc:260] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:34.587185 14692 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:34.568379 14851 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:34.593750  6623 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:34.595652  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:34.596113 14854 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:34.596563  6500 raft_consensus.cc:2393] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 25: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:34.596856 14854 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:34.600495  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:34.599789 14854 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:34.617515 14878 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:34.627892 14878 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:34.628371 14878 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:34.629133 14713 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:34.619831  6658 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 26. Current term is 27. Ops: [26.14-26.14]
I20260709 12:20:34.625965  6582 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 23. Current term is 24. Ops: [23.18-23.18]
I20260709 12:20:34.626471 14668 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:34.596133 14828 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:34.633422  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:20:34.617523  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 87822.1us
I20260709 12:20:34.617841  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:34.637145  6649 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
I20260709 12:20:34.637645  6622 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:34.659897  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:34.581594 14880 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:20:34.677512  6622 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:34.564150 14801 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:34.679678  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:34.680716  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:34.682211  6658 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:20:34.691520  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:34.691789  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:34.693691  6582 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 25. Current term is 26. Ops: [25.13-25.13]
I20260709 12:20:34.693629  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:34.695993  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:34.564150 14869 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:34.669632 14543 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:34.704311  6649 raft_consensus.cc:2368] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 21. Current term is 22.
I20260709 12:20:34.670481 14713 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:34.711568  6650 raft_consensus.cc:2393] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 23: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:34.700598 14675 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:20:34.695463 14872 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:20:34.707031 14543 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:34.701130 14880 raft_consensus.cc:697] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 26 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:34.708031 14521 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:34.711190 14536 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:34.710963 14435 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:34.728896 14872 raft_consensus.cc:697] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 28 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:34.717948 14813 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:34.734042 14801 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:34.735208 14828 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:34.736076 14542 raft_consensus.cc:3055] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:34.736718 14880 consensus_queue.cc:237] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:34.740432 14882 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:34.710682 14713 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:34.745102 14872 consensus_queue.cc:237] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 27.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:34.731463 14536 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:34.795095 14536 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:34.751989  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:34.795861  7269 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:34.797281 14713 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:34.773638 14542 raft_consensus.cc:740] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:34.798624 14542 consensus_queue.cc:260] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:34.799700 14542 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:34.722144 14543 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:34.729740 14521 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:34.758981  6658 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:34.759210  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:34.762948  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:34.766230 14448 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:34.804637  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:34.781778  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:34.809072  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:34.810266  7269 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:34.810968  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:34.783051 14802 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:34.753520 14435 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:34.811733  6549 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:34.823191 14801 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:34.823782 14801 raft_consensus.cc:697] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:34.824762 14801 consensus_queue.cc:237] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:34.818087 14448 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:20:34.810060 14521 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:34.810253 14851 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:20:34.834198 14871 consensus_queue.cc:1059] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 24.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 12:20:34.836580  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:34.837193  6580 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:20:34.838806 14878 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:34.819061 14683 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:34.846730 14811 raft_consensus.cc:3055] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:34.810377 14666 consensus_queue.cc:1059] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.023s
I20260709 12:20:34.853971 14811 raft_consensus.cc:740] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:34.862725 14811 consensus_queue.cc:260] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:34.868527 14706 raft_consensus.cc:3055] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:34.869120 14706 raft_consensus.cc:740] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:34.870209 14869 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:34.870164 14706 consensus_queue.cc:260] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:34.872120 14706 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:34.879839 14542 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:34.880657 14542 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:34.884059 14768 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:34.884406 14542 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:34.891083 14435 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:34.835623 14668 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:34.896764  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:34.897542  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:34.897781  6652 raft_consensus.cc:2368] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 29. Current term is 30.
I20260709 12:20:34.898456  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:34.871155 14845 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:34.880244  6550 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:34.904431  6550 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:34.892375  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 48625.1us
I20260709 12:20:34.898941  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:34.819818 14802 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:34.899426 14871 raft_consensus.cc:3055] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:34.909633  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:34.864192 14811 raft_consensus.cc:3060] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:34.919099  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:34.920145  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 408247us
I20260709 12:20:34.920279  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:34.919413  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:34.919791  6578 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 24.
I20260709 12:20:34.925540  6579 raft_consensus.cc:2393] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 23: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:34.926075  6475 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:34.908998 14542 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:20:34.927791  6475 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:34.932619  6473 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:34.913125 14871 raft_consensus.cc:740] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:34.913669 14803 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:34.931134  6550 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:34.920053 14562 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:34.913327 14521 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:34.939857  6580 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22.
I20260709 12:20:34.937373  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:34.944154  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:34.947530  6582 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:20:34.912089 14768 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:34.943634 14536 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:34.948606  6509 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 27. Current term is 28. Ops: []
I20260709 12:20:34.936825 14858 consensus_queue.cc:1059] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.008s
I20260709 12:20:34.956993  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:34.957461  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:34.958947  6506 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 25. Current term is 26. Ops: []
I20260709 12:20:34.940518 14803 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:34.957659  6500 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 24.
I20260709 12:20:34.964977 14536 raft_consensus.cc:697] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:34.969156 14781 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:34.969532  6402 catalog_manager.cc:5697] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:20:34.970618  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:34.970961  6402 catalog_manager.cc:5697] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 13 to 18, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 18 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:20:34.972800 14892 log.cc:927] Time spent T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:20:34.951395 14521 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:34.937266 14821 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 12:20:34.947003 14812 consensus_queue.cc:1059] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:20:34.949735 14562 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:20:34.949189 14680 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:20:34.978353  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:34.966892 14803 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:34.990867  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:34.991503  6507 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:34.992631  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:34.987795 14680 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:34.977129 14829 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:35.002530 14680 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:35.004957 14536 consensus_queue.cc:237] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:35.006242  6576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 3127) took 1141 ms. Trace:
I20260709 12:20:35.006656  6576 rpcz_store.cc:276] 0709 12:20:33.864774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:33.864888 (+   114us) service_pool.cc:224] Handling call
0709 12:20:33.892110 (+ 27222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:34.046692 (+154582us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:35.006000 (+959308us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b540032b254. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:35.006182 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":353836928}
I20260709 12:20:35.002751 14708 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:35.008864 14708 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:35.010908  6650 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1997e98759d0465ab0e3c607df127068"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:47858
I20260709 12:20:35.012420 14861 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.012951 14861 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:35.015101 14861 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:35.015856 14878 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:35.016476 14878 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:35.016959 14878 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:35.017449 14827 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:35.018318 14827 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:20:35.019652  6550 leader_election.cc:400] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 21. Current term is 22.
I20260709 12:20:35.019979  6550 leader_election.cc:403] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:35.020802  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:35.000128 14802 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:35.027356  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:35.028401  6507 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 23.
I20260709 12:20:35.027987  6506 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:20:35.017518 14562 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:35.010931 14708 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:35.025046 14878 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:35.056995  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:35.024637 14542 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 21. Current term is 22.
W20260709 12:20:35.058233  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 130484us
W20260709 12:20:35.065095  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 132822us
I20260709 12:20:35.066982  6473 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:35.020855 14829 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:35.074774  6473 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:35.075241  6506 raft_consensus.cc:2393] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 24: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:35.002288 14871 consensus_queue.cc:260] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:35.112739  6473 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:35.114081  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:35.119915  7268 raft_consensus.cc:1240] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 21 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 12:20:35.121001  6622 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:35.060963 14543 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:35.126897  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:35.128093  6622 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:35.110473 14821 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:35.130400  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:35.131376  6622 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:35.124560 14851 consensus_queue.cc:1059] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 27.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.007s
W20260709 12:20:35.136511  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 62180.7us
I20260709 12:20:35.138422  6583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 3130) took 1201 ms. Trace:
I20260709 12:20:35.120536 14684 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:35.120399 14862 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:20:35.141090  6583 rpcz_store.cc:276] 0709 12:20:33.934003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:33.943287 (+  9284us) service_pool.cc:224] Handling call
0709 12:20:35.134976 (+1191689us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b54001cc854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:35.135117 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":747281280}
I20260709 12:20:35.150409  6473 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:35.151974 14684 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:35.153960 14521 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:35.155125 14684 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:35.160570 14899 log.cc:927] Time spent T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:20:35.161783  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:35.174316 14543 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:35.150452  6622 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:35.178188 14680 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:35.179464  6509 raft_consensus.cc:2393] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 22: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:35.182255 14683 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:35.146819 14834 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:35.183411 14834 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:35.183768 14834 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:35.187690 14821 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:20:35.188694  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 190949us
I20260709 12:20:35.132823 14895 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:35.190426 14543 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:35.137147 14849 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:35.132823 14668 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:35.134429 14845 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:35.194092  6473 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:35.195129 14828 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:35.145066 14871 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:35.196513 14506 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.197460 14906 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:35.136669 14768 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:35.198781  6550 leader_election.cc:400] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 29. Current term is 30.
I20260709 12:20:35.266126  6550 leader_election.cc:403] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 election: Cancelling election due to peer responding with higher term
W20260709 12:20:35.266762  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 68404.2us
I20260709 12:20:35.198634 14829 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:35.198060  6622 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.952s (SENT) [suppressed 9 similar messages]
I20260709 12:20:35.195685 14708 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.211702  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:35.257918  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 120539us
I20260709 12:20:35.274555  6653 raft_consensus.cc:1240] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 26. Current term is 27. Ops: []
I20260709 12:20:35.192245 14882 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:35.264531  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:20:35.270161  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 76898.6us
W20260709 12:20:35.201946 14791 log.cc:927] Time spent T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 1.069s	user 0.006s	sys 0.000s
I20260709 12:20:35.281479  6475 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:35.264107 14506 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:35.276979  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:35.284060 14506 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:35.277228  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:35.274607  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:35.286264  6655 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 26.
I20260709 12:20:35.209956 14845 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:35.272159 14682 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:20:35.281013  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:35.282331 14862 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:20:35.292807  6657 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:20:35.285795  6654 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 24.
I20260709 12:20:35.214299 14828 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:35.215651 14849 raft_consensus.cc:697] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 23 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:20:35.287925  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:35.289222  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:35.300752  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:35.302229  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:35.302990  6504 raft_consensus.cc:2393] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 23: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:35.277506 14708 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:35.290629  6503 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 27. Current term is 28. Ops: [27.14-27.14]
I20260709 12:20:35.290199  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:35.308768  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:35.309151  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:35.309437  6500 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:35.309634  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:35.310282  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:35.311370  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:35.312448  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:35.265417 14906 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:35.313218 14906 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:35.313344  6508 raft_consensus.cc:1275] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 16. (term mismatch)
I20260709 12:20:35.314311  6508 pending_rounds.cc:85] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 15
I20260709 12:20:35.314616  6508 pending_rounds.cc:107] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 12:20:35.314972  6508 raft_consensus.cc:2889] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:35.293566  6502 raft_consensus.cc:1240] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 23. Current term is 24. Ops: [23.14-23.14]
I20260709 12:20:35.296923  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:35.296954  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:35.307848 14862 raft_consensus.cc:697] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 30 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:35.275571 14811 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:20:35.323938  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:35.329181  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 50251.1us
I20260709 12:20:35.330562  6473 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:35.338032  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:35.342418  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:35.343003  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:35.293118 14845 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:35.347259  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:35.347986  6652 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 26.
I20260709 12:20:35.298388 14828 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:35.353539  6658 raft_consensus.cc:1240] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 12:20:35.364008  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:20:35.368076  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 170026us
I20260709 12:20:35.369784  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:35.370437  6658 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 27.
I20260709 12:20:35.387836  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:35.370962 14774 consensus_queue.cc:1048] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 12:20:35.371183 14858 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:35.356743 14894 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:35.395185 14894 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:35.351948 14668 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:35.335529 14758 consensus_queue.cc:1059] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.044s
I20260709 12:20:35.371807 14811 raft_consensus.cc:697] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 26 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:35.419852 14706 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:35.425491  6500 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 24.
I20260709 12:20:35.428200  6503 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 23. Current term is 24. Ops: [22.17-23.18]
I20260709 12:20:35.386281 14882 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:35.437247  6504 raft_consensus.cc:1275] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
W20260709 12:20:35.457705  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 86383.7us
I20260709 12:20:35.463549 14894 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:35.439085 14706 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:35.441906 14882 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:35.466957 14706 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:35.467442 14882 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:35.459641 14708 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:35.471851  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:35.422286 14668 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:35.487381 14683 consensus_queue.cc:1048] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.059s
I20260709 12:20:35.518206 14680 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:35.440964 14849 consensus_queue.cc:237] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:35.441243 14862 consensus_queue.cc:237] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.15, Last appended by leader: 15, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:35.581827  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 161604us
I20260709 12:20:35.583475  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:35.572979 14680 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:35.593066  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 162239us
I20260709 12:20:35.582832 14895 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.546083 14811 consensus_queue.cc:237] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:35.585348 14912 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.612279  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:35.625366  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:35.651958 14803 consensus_meta.cc:287] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.654s	user 0.007s	sys 0.002s
I20260709 12:20:35.596467 14895 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:35.601054 14680 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:35.679843 14917 log.cc:927] Time spent T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 12:20:35.681579 14668 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:35.684427 14916 log.cc:927] Time spent T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 12:20:35.685169 14880 raft_consensus.cc:3055] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:20:35.685676 14880 raft_consensus.cc:740] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:35.684280 14827 raft_consensus.cc:3055] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:35.687602 14827 raft_consensus.cc:740] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:35.688609 14827 consensus_queue.cc:260] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:35.686641  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:35.690122 14827 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:35.690920 14880 consensus_queue.cc:260] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.14, Last appended by leader: 14, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:35.695971 14880 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:35.635908 14758 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:35.697088 14758 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:35.699361 14758 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:35.635455 14894 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:35.618204 14919 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:35.635721 14882 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:20:35.707508  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 113796us
I20260709 12:20:35.711804  6504 raft_consensus.cc:1275] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:20:35.621795 14912 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:35.700799 14894 raft_consensus.cc:697] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:35.706317 14713 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:35.708782 14882 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:35.653386 14543 consensus_queue.cc:1059] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.063s
I20260709 12:20:35.658849 14803 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:35.729799 14873 consensus_queue.cc:1048] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 12:20:35.657472 14834 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:35.733289 14882 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:35.762512 14894 consensus_queue.cc:237] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:35.763339  6475 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:35.764986 14706 raft_consensus.cc:3055] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:35.765443 14706 raft_consensus.cc:740] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:35.765898 14562 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:35.766419 14562 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:35.768508 14918 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:20:35.766430 14706 consensus_queue.cc:260] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:35.770959 14706 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:35.772851  6500 raft_consensus.cc:1240] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:35.773278  6475 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:35.775157 14708 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:35.774967 14845 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:35.775591 14708 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:35.776072 14708 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:35.776628 14873 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:35.777123 14873 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:35.775185 14803 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:35.778796 14896 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:35.779202 14873 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:35.779313 14896 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:35.738166 14713 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:35.781913 14713 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:35.783205 14834 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:35.783949 14895 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:35.785369  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:35.785624 14562 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:35.786542 14912 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:35.787019 14562 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:35.787575 14562 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:35.791649 14683 raft_consensus.cc:3055] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:35.792363 14683 raft_consensus.cc:740] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:35.779202 14918 consensus_queue.cc:1059] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 12:20:35.793257 14683 consensus_queue.cc:260] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:35.794327 14683 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:35.794246 14562 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:35.796804  6475 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:35.797091  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:35.797749 14443 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:20:35.798677  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:35.799000  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:35.747880 14924 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:35.802075  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:35.804922  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:35.805959  6579 raft_consensus.cc:2393] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 26: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:35.801784 14924 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:35.814134 14902 raft_consensus.cc:3055] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:20:35.814607 14902 raft_consensus.cc:740] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:35.815632  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:35.815485 14902 consensus_queue.cc:260] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:35.817674 14902 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:35.816984 14675 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:35.819034 14845 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:35.819258 14675 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:35.819890  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:35.821761 14675 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:35.821867 14896 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:35.825058 14821 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:35.825672 14821 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:35.808107 14828 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:35.832296  6509 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 25.
I20260709 12:20:35.833729  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:35.834774 14828 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:35.841138 14924 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:35.841307 14906 consensus_meta.cc:287] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.527s	user 0.015s	sys 0.000s
I20260709 12:20:35.842298  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:35.843027  6579 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 24.
I20260709 12:20:35.843196  6402 catalog_manager.cc:5697] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:35.841969 14906 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:35.844559 14896 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:35.844908  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:35.844714  6402 catalog_manager.cc:5697] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:35.845126 14896 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:35.846519  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:35.846896 14680 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:35.847127 14896 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:35.847227  6577 raft_consensus.cc:2393] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 22: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:35.846725  6402 catalog_manager.cc:5697] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 15 to 19, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 19 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:35.847424 14680 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:35.848845 14906 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:35.848905  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:35.848016 14680 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:35.850644  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:35.852641  6475 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
W20260709 12:20:35.853323 14871 consensus_meta.cc:287] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.595s	user 0.012s	sys 0.006s
I20260709 12:20:35.853820  6650 raft_consensus.cc:606] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:35.854281  6650 raft_consensus.cc:614] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:35.854314 14918 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:20:35.854857 14918 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:35.855239 14918 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:35.856509  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:35.857210  6655 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 25.
I20260709 12:20:35.859908 14680 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:35.861020  6655 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:47858
I20260709 12:20:35.861714  6655 raft_consensus.cc:606] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:35.862131  6655 raft_consensus.cc:614] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:35.862375 14821 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:35.864758  6655 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:47858
I20260709 12:20:35.865491  6655 raft_consensus.cc:606] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Received request to transfer leadership to TS 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:35.791728 14682 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Leader election lost for term 29. Reason: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 29. Current term is 30.
I20260709 12:20:35.865903  6655 raft_consensus.cc:614] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:35.843946  6622 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:20:35.867502  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 270660us
I20260709 12:20:35.869967  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:35.873636  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:35.878027  6577 raft_consensus.cc:2410] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 25 because replica has last-logged OpId of term: 23 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 12:20:35.879071  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:35.878937  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:35.869992  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:35.888541  6649 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 26. Current term is 27. Ops: [26.14-26.14]
I20260709 12:20:35.884147  6622 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:35.892060  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:35.893651  6654 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:20:35.898157  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:35.898604  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:35.896095  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:35.904606  6653 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:35.906490  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:35.911548  6657 raft_consensus.cc:2393] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 27: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:35.911820  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:35.911953  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:35.912541  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:35.912640  6650 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:20:35.916002  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:35.920418  6651 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 28 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 27. Current term is 28. Ops: []
I20260709 12:20:35.920665  6658 raft_consensus.cc:1240] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:35.921758  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:35.927168  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:35.928066  6654 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:20:35.933156  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:35.933881  6651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3933) took 1305 ms. Trace:
I20260709 12:20:35.934435  6502 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:20:35.917186  6550 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:35.934248  6651 rpcz_store.cc:276] 0709 12:20:34.628580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:34.704162 (+ 75582us) service_pool.cc:224] Handling call
0709 12:20:34.711387 (+  7225us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:35.920835 (+1209448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:35.933676 (+ 12841us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b540044acd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:35.933826 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2779903360}
I20260709 12:20:35.877089 14871 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:35.940907  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:35.944880  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:35.945744  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:35.945999  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:35.943178  6475 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:35.911242 14675 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:35.948869  6505 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:35.958971  6651 raft_consensus.cc:1240] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:35.936880  6550 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:35.937841 14828 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:35.960903  6550 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:35.966107  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:35.932176 14912 consensus_queue.cc:1059] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.033s
I20260709 12:20:35.950757  6548 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:35.960467 14330 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:35.963742  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:35.976868  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:35.942605 14521 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:35.982142  6650 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22.
I20260709 12:20:35.974830 14682 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:35.970412 14901 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:35.981280 14521 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:35.980495 14330 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:35.968848 14828 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:35.984798  6655 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "83301542a88f46cea64bc905b3199975"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:47858
I20260709 12:20:36.000552  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:35.998843  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:36.018483  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:36.013213  6475 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:36.019351  6581 raft_consensus.cc:2368] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 25. Current term is 26.
W20260709 12:20:36.029345  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 78680us
I20260709 12:20:36.031316  6548 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:36.011647 14521 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:35.994918 14933 raft_consensus.cc:3055] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:20:35.999711 14901 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:35.997929 14682 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:36.007396 14330 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:36.052281  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:35.983563 14829 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:20:36.041774 14929 log.cc:927] Time spent T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:20:36.041539 14443 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:36.011914 14906 consensus_queue.cc:1059] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 12:20:36.039687 14877 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:36.062939  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:36.071045  6579 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 27.
I20260709 12:20:36.052906 14901 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:36.046087 14933 raft_consensus.cc:740] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:20:36.079435  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:36.076809  6655 raft_consensus.cc:606] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:36.080380  6655 raft_consensus.cc:614] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:36.054348 14682 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:36.081698 14910 raft_consensus.cc:3055] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:36.082106 14910 raft_consensus.cc:740] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:36.082938 14910 consensus_queue.cc:260] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:36.084139 14910 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:35.983382 14757 consensus_queue.cc:1059] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.073s
I20260709 12:20:36.071523 14443 raft_consensus.cc:697] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 22 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:20:36.087642  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:36.088667  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:36.046602 14708 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:36.086253 14443 consensus_queue.cc:237] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:36.096264 14882 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:36.069862 14932 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:36.061929 14829 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:36.119004  6473 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:36.120838  6473 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:36.061214  6548 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:36.123360  7273 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:20:36.126754  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:36.125228  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 156400us
I20260709 12:20:36.078627 14877 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:36.135949  6473 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:36.137603  6473 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:36.135249  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:36.124110 14934 raft_consensus.cc:3055] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:20:36.143085  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 81955.8us
I20260709 12:20:36.130398 14926 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:36.107193 14918 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:36.132424 14809 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:36.146054 14680 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:36.157083  6653 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 26.
I20260709 12:20:36.157644  6548 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:36.138517 14932 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:20:36.169515  6548 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:36.161535 14821 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:36.179937  6503 raft_consensus.cc:1240] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 27. Current term is 28. Ops: []
I20260709 12:20:36.143316 14934 raft_consensus.cc:740] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:36.183307  6652 raft_consensus.cc:1240] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 12:20:36.186090  6475 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:36.156754 14680 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:36.167634 14896 raft_consensus.cc:2764] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 25: won
I20260709 12:20:36.175022 14828 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:36.170929 14713 consensus_queue.cc:1059] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 12:20:36.150988 14902 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.042s
W20260709 12:20:36.206427  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:36.208220 14936 log.cc:927] Time spent T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 12:20:36.200207 14680 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:36.221216  6505 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:20:36.224681 14829 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:36.224833 14935 log.cc:927] Time spent T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 12:20:36.195904 14708 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:36.228704 14902 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.025s
I20260709 12:20:36.228749 14943 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:36.256494 14943 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:36.261592 14683 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:36.199049 14821 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:20:36.267733  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 109384us
I20260709 12:20:36.269315  6550 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:36.270342  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 121344us
I20260709 12:20:36.260196 14521 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:36.317802 14758 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:36.319021  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 149613us
I20260709 12:20:36.135393 14933 consensus_queue.cc:260] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:36.325582 14713 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:20:36.337580  6548 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:36.328178 14918 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:36.312098 14682 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:36.350994 14953 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:36.351924 14953 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:36.354629 14682 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:36.354907 14952 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.355844 14952 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:36.358522 14940 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:20:36.366614 14713 raft_consensus.cc:697] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:36.368067 14713 consensus_queue.cc:237] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:36.377290 14774 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.378185 14774 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:36.324081 14758 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:36.399977  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 62488.6us
I20260709 12:20:36.403043  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:36.419523  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:36.421558  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:36.375775 14680 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:36.400049 14877 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:36.430166  7268 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22.
I20260709 12:20:36.361266 14330 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:36.451591 14330 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:36.452416 14932 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:20:36.454583 14901 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:36.437422 14910 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:20:36.460304  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:36.476253  6548 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:36.354719 14951 consensus_queue.cc:1059] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 27.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.107s
I20260709 12:20:36.430456 14849 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:36.527323 14917 log.cc:927] Time spent T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:20:36.489317 14952 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:36.508500 14486 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:20:36.602555  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:36.483093 14910 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:36.566648 14957 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:36.438439 14934 consensus_queue.cc:260] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:36.613718 14774 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:36.634704  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 106406us
I20260709 12:20:36.636252  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:36.563090 14849 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:36.637956  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:36.637969  6548 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:36.638094  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:36.640280  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:36.637161  6577 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 16.
I20260709 12:20:36.645576  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:36.629730 14951 consensus_queue.cc:1243] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer f2b06aebe97249a4877d798b8118ece4 log is divergent from this leader: its last log entry 28.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:20:36.649905 14959 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:20:36.638362 14932 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:36.652552 14932 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:36.655567 14932 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:36.634568 14713 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:36.656569 14713 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:36.660576 14957 raft_consensus.cc:697] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:36.662214 14957 consensus_queue.cc:237] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:36.635607 14872 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.639191 14934 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:36.680598 14803 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:36.689189  6584 raft_consensus.cc:1275] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:20:36.681226 14951 consensus_queue.cc:1059] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:36.682667  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:36.638187 14963 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:36.720844  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:36.737883  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:36.683987 14713 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:36.741530  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:36.746459  6579 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 30.
I20260709 12:20:36.748826  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:36.749259  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:36.749663  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 3537) took 1457 ms. Trace:
I20260709 12:20:36.751061  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:36.750443  6501 rpcz_store.cc:276] 0709 12:20:35.292315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:35.292490 (+   175us) service_pool.cc:224] Handling call
0709 12:20:36.749429 (+1456939us) spinlock_profiling.cc:243] Waited 803 ms on lock 0x7b540001c2d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:36.749576 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1848036480}
I20260709 12:20:36.754541  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:36.758801  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:36.760001  6578 raft_consensus.cc:1240] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 12:20:36.740691 14956 raft_consensus.cc:3055] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:36.690881 14872 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:36.757054 14774 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.756174 14758 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:36.692914 14963 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:36.757789 14680 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:36.777731  6581 raft_consensus.cc:1275] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:20:36.778692  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:36.761663 14901 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:36.781134  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:36.771320 14713 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.763427 14956 raft_consensus.cc:740] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:36.783583  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:36.799556  6507 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:36.783329  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:36.784989  6501 raft_consensus.cc:1275] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:20:36.801860  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:36.788132  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:36.789012  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:36.777364 14951 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.800705  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:36.779158 14964 log.cc:927] Time spent T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 12:20:36.807670  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:36.804101  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:36.796880 14861 raft_consensus.cc:3055] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:36.803993  6504 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:20:36.817348  6500 raft_consensus.cc:2393] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 28: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:36.778040 14910 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:36.819499  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:36.822563  6502 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 12:20:36.778378 14774 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:36.817148 14861 raft_consensus.cc:740] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:36.820717 14951 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:36.841169 14963 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:36.843765 14951 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:36.844198 14521 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:36.853235 14896 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:36.853953 14896 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:36.856711 14896 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:36.856966 14956 consensus_queue.cc:260] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:36.858711 14956 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:36.861091 14849 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:36.865849 14872 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:36.866459 14774 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:36.814745 14713 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:36.869635 14713 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:36.875098  6507 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 26.
W20260709 12:20:36.883018 14965 log.cc:927] Time spent T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.318s	user 0.007s	sys 0.000s
W20260709 12:20:36.888171  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 167004us
W20260709 12:20:36.895406  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:36.898749 14971 log.cc:927] Time spent T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.109s	user 0.002s	sys 0.000s
I20260709 12:20:36.880096 14861 consensus_queue.cc:260] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:36.904785  6549 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:36.905330  6575 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 28 index: 15. Preceding OpId from leader: term: 30 index: 16. (index mismatch)
I20260709 12:20:36.906495 14882 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:36.907174 14882 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:36.908854 14896 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.909405 14896 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:36.909713 14970 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:36.910113 14953 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:36.910530 14970 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:36.911481 14896 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:36.912039 14896 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.913812 14966 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.979087  6657 raft_consensus.cc:1240] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:36.914053  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:36.914309  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:36.914687  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:36.914832 14970 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:36.915148 14872 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:36.916210  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:36.937032  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:37.054107  7272 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22.
I20260709 12:20:37.054144  6653 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:37.061971  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:37.062870  6657 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:20:37.010836  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 75585us
W20260709 12:20:37.030153  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:37.081563  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:36.974902  6622 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:37.053376  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:36.907703 14882 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:37.085280  6583 raft_consensus.cc:2393] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 26: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:37.086297  6576 raft_consensus.cc:2393] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 22: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:37.080310  6653 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 25.
I20260709 12:20:37.089536 14970 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:37.083232  6651 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 28.
I20260709 12:20:37.092136  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:37.092841  6505 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 21.
I20260709 12:20:37.097165  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:37.098136  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:37.098110  6503 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 24.
I20260709 12:20:37.093864  7238 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:37.090210 14970 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:37.104871 14970 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:37.109241  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:37.112160  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:37.113838 14768 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:37.114513 14768 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:37.052654 14966 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:36.980849 14952 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:37.117990 14768 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:37.071790 14774 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:37.122296 14774 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:37.124653 14880 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:20:36.981438 14849 consensus_queue.cc:1048] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.145s
I20260709 12:20:37.083187 14521 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.046s
I20260709 12:20:37.134547 14882 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:36.977880 14953 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:36.974393 14543 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:37.082706 14872 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:37.117518 14966 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:37.152261  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 58526.8us
I20260709 12:20:37.014873 14861 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:36.981570 14896 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:37.157459 14666 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:37.171139  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:37.173159  6652 raft_consensus.cc:1275] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:20:37.178047  6651 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 22.
I20260709 12:20:37.175028 14896 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:37.184854 14774 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:37.192767  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:37.193709  7272 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 26.
I20260709 12:20:37.201252  6549 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
W20260709 12:20:37.203029  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 228224us
I20260709 12:20:37.203065  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:37.204243  6583 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 22.
I20260709 12:20:37.204456  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:37.177222 14666 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:37.211340  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:37.206028  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:37.208600 14521 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:37.215579 14521 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:37.215503 14894 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:37.216396 14521 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:37.216867 14943 raft_consensus.cc:3055] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:20:37.217272 14943 raft_consensus.cc:740] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:37.218076 14943 consensus_queue.cc:260] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.14, Last appended by leader: 14, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:37.230690  6657 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:37.231156 14813 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:37.235543  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:37.236738  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:37.237399  6584 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:37.235154 14813 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:37.243546 14813 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:37.234309  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:37.202236 14956 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:20:37.253940 14955 log.cc:927] Time spent T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.849s	user 0.002s	sys 0.009s
I20260709 12:20:37.210099  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:37.254863  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:37.266433 14954 log.cc:927] Time spent T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.949s	user 0.012s	sys 0.000s
I20260709 12:20:37.268757 14952 consensus_queue.cc:1059] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 24.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:37.269068 14933 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:37.274451 14872 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:37.295727  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 206523us
I20260709 12:20:37.233922 14521 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:37.158238 14543 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:37.292835  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 81589.7us
I20260709 12:20:37.156801 14953 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:37.297173  6657 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:37.298440  6550 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:37.297308  6584 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 25.
I20260709 12:20:37.277953 14666 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:37.304760  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:37.306080  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 143230us
I20260709 12:20:37.306665  6622 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:37.317629  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:37.317979 14980 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:37.308175  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:37.318634  6622 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:37.309249  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:37.319140  7238 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:37.320264  6622 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:37.321086 14924 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:37.321841 14924 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:37.322220 14924 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:37.322589 14924 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:37.319136  6658 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 24.
I20260709 12:20:37.309456  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:37.326587 14872 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:37.327314 14872 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:37.328295  6658 raft_consensus.cc:1240] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:37.330101  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:37.332856  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:37.334230  6622 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:37.335114 14980 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:20:37.336004  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:37.343676  6652 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 26.
I20260709 12:20:37.345476  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:37.344852  6550 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:37.297430 14871 raft_consensus.cc:2764] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Leader pre-election decision vote started in defunct term 29: won
I20260709 12:20:37.351537  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:37.349496  6550 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:37.357549  6550 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:37.359429  6583 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:20:37.363358  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:37.368551  6653 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 12:20:37.290186 14956 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:37.371431  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:37.346365 14768 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:37.355171 14486 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:20:37.380609  6656 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 28 index: 15. Preceding OpId from leader: term: 30 index: 16. (index mismatch)
I20260709 12:20:37.381117  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:37.364941 14901 raft_consensus.cc:2804] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:37.385128  6651 raft_consensus.cc:2393] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 27: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:37.371240 14968 consensus_queue.cc:1048] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 12:20:37.396819 14953 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:37.397573 14953 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:37.398219 14953 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:37.401425 14521 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:37.404597 14708 consensus_queue.cc:1059] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:20:37.407608  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 227420us
I20260709 12:20:37.408522  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:37.410974  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:37.413094  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:37.416981  6475 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:37.384654 14486 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:37.417791 14486 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:37.419221 14910 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:20:37.406625 14708 raft_consensus.cc:3055] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:37.424714 14708 raft_consensus.cc:740] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:20:37.426026 14947 log.cc:927] Time spent T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 1.045s	user 0.008s	sys 0.005s
I20260709 12:20:37.426945 14543 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:37.431582 14680 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:37.431874  6658 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:20:37.432178 14680 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:37.409566 14901 raft_consensus.cc:697] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:37.434713 14680 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:37.425634 14708 consensus_queue.cc:260] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:37.435217 14901 consensus_queue.cc:237] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:37.435942 14708 raft_consensus.cc:3060] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:37.427089  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:37.361114 14666 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:37.442523  6505 raft_consensus.cc:2393] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 24: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:37.442878  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:37.371490 14963 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:37.444149  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:37.444576  6507 raft_consensus.cc:1275] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:20:37.446368  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:37.449344  6506 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:37.451009  6507 raft_consensus.cc:1240] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:37.431190 14921 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:37.454265  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 227159us
I20260709 12:20:37.458815  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:37.456600  6654 raft_consensus.cc:1275] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:20:37.466171  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:37.466948  7273 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 18. (term mismatch)
I20260709 12:20:37.467372  7273 pending_rounds.cc:85] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9: Aborting all ops after (but not including) 17
I20260709 12:20:37.467594  7273 pending_rounds.cc:107] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9: Aborting uncommitted NO_OP operation due to leader change: 23.18
I20260709 12:20:37.467820  7273 raft_consensus.cc:2889] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:37.468708 14968 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:20:37.469197 14934 consensus_meta.cc:287] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.777s	user 0.014s	sys 0.000s
I20260709 12:20:37.469354  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:37.469341 14968 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:37.470878 14956 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:37.471809  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:37.473135  6503 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:20:37.472527  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:37.473598 14666 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:37.473912 14968 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:37.416229 14980 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:37.383584 14924 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:37.474684  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 61623.9us
I20260709 12:20:37.417203 14896 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:37.475736  7269 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 26.
I20260709 12:20:37.479306  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:37.480712  6473 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:37.505124  6654 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 26.
I20260709 12:20:37.509552  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:37.510936  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:37.512785  6506 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 26.
I20260709 12:20:37.513309  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:37.416710 14926 consensus_queue.cc:1059] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.068s
I20260709 12:20:37.516584 14924 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:37.449471 14871 consensus_queue.cc:1048] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.066s
I20260709 12:20:37.517952 14926 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:37.518426  6578 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 27.
I20260709 12:20:37.518651 14926 raft_consensus.cc:3055] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:37.512382  6549 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:37.519640  7240 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:37.519943  6473 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:37.520541  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:37.477473 14953 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:37.519146 14926 raft_consensus.cc:740] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:37.510712 14982 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:37.511715 14872 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:37.543614  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:37.543738  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:37.482698 14968 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.046s
I20260709 12:20:37.524740 14961 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:20:37.571913 14961 raft_consensus.cc:697] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:37.573233 14961 consensus_queue.cc:237] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:37.573813 14970 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:37.574635 14970 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:37.549845  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:37.549199 14982 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:37.541283 14983 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:37.551672  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:37.482887 14921 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:37.553546  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:37.544250 14862 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.010s
I20260709 12:20:37.561457  6473 leader_election.cc:400] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 25. Current term is 26.
I20260709 12:20:37.590996  6473 leader_election.cc:403] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 election: Cancelling election due to peer responding with higher term
I20260709 12:20:37.540921 14858 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.022s
I20260709 12:20:37.544322 14680 consensus_queue.cc:1048] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:20:37.570118 14872 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:37.591146  6502 raft_consensus.cc:2393] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 26: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:37.584976  6506 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 24.
I20260709 12:20:37.598191  6473 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:37.612999  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:37.613987  6508 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:20:37.613862  6474 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:37.580289 14982 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:37.621207  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:37.636528  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:37.637632  6654 raft_consensus.cc:2410] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 27 because replica has last-logged OpId of term: 25 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 12.
I20260709 12:20:37.638386  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:37.618340  7238 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:37.580701 14983 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:37.641881  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:37.639945  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:37.655893  6656 raft_consensus.cc:2393] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 19: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:37.659461  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:20:37.665706  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 258076us
W20260709 12:20:37.667641  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:37.668886  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:37.623474 14768 raft_consensus.cc:697] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:37.620750 14813 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:20:37.693480  6550 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:37.656652 14983 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:37.699900  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:37.657402 14924 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:37.671684 14921 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:37.702363  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:37.672427 14861 consensus_meta.cc:287] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.504s	user 0.006s	sys 0.006s
I20260709 12:20:37.701512  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:37.700716  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:37.700157  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:37.700481  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:37.696866  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 134574us
I20260709 12:20:37.713160  6475 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:37.713150  6550 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:37.709473  6576 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 25.
I20260709 12:20:37.661532 14910 raft_consensus.cc:697] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 26 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:20:37.720393  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 122236us
I20260709 12:20:37.730926  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:37.724256  6579 raft_consensus.cc:1275] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:20:37.667481 14713 raft_consensus.cc:3055] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:37.736066  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:37.685286 14956 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:37.741041  6654 raft_consensus.cc:2393] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 20: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:20:37.740464  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 126776us
I20260709 12:20:37.743957  6474 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:37.621340 14926 consensus_queue.cc:260] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:37.704865 14683 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:37.758440  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 99057.8us
I20260709 12:20:37.768872  6503 raft_consensus.cc:2468] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22.
I20260709 12:20:37.704306 14682 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:37.728091 14901 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:37.777650  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:37.735996 14713 raft_consensus.cc:740] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:37.780737  6581 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 26.
I20260709 12:20:37.781191 14902 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:37.781318 14713 consensus_queue.cc:260] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:37.782691 14713 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:37.775107 14990 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader election lost for term 25. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 25. Current term is 26.
I20260709 12:20:37.785564 15000 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:37.785938 15000 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:37.786250 15000 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:20:37.787390  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 61398.7us
W20260709 12:20:37.788281  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 44512.8us
I20260709 12:20:37.789053  6474 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:37.791970  6582 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 24.
I20260709 12:20:37.790632  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:37.774667 14987 consensus_queue.cc:1059] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.063s
I20260709 12:20:37.810333  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 28 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:37.834139  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 105994us
I20260709 12:20:37.868791  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:37.865043  6580 raft_consensus.cc:1240] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 24. Current term is 25. Ops: [22.13-24.14]
I20260709 12:20:37.865957  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:37.776870 14708 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:20:37.848300  6508 raft_consensus.cc:1275] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:20:37.775924 14682 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:37.893175  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:37.894094  6506 raft_consensus.cc:2410] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 28 because replica has last-logged OpId of term: 26 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 13.
I20260709 12:20:37.868171  6575 raft_consensus.cc:1275] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:20:37.779217 14970 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:37.897075 14970 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:37.898645 14970 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:37.883245  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:37.773526 14861 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:37.908464  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 28 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:37.910066  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:37.910717 14943 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:37.884740 14910 consensus_queue.cc:237] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:37.914527 14995 log.cc:927] Time spent T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 12:20:37.822408 14979 consensus_queue.cc:1059] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.093s
I20260709 12:20:37.884562 14521 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:37.992565 14543 raft_consensus.cc:3055] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:37.993183 14543 raft_consensus.cc:740] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:37.822233 14894 consensus_queue.cc:1048] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.094s
I20260709 12:20:37.848814 14999 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:37.994234 14543 consensus_queue.cc:260] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:37.832036 14768 consensus_queue.cc:237] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:37.995705 14543 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:37.820428 14933 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:37.999365 14933 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:38.002175 14933 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:37.921088 14982 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:37.921766  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:38.003072 14982 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:37.923645  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:38.004824 14758 raft_consensus.cc:3055] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:20:38.005283 14758 raft_consensus.cc:740] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:38.006305 14758 consensus_queue.cc:260] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:38.007247  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:38.007712 14758 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:38.007954  6651 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 19.
I20260709 12:20:37.881994 14921 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:38.009138 14921 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:37.924813  7265 raft_consensus.cc:1240] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 16. Current term is 17. Ops: []
W20260709 12:20:37.930214 14991 log.cc:927] Time spent T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
W20260709 12:20:37.931157 14997 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:20:38.011816 14921 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:37.931600 14901 consensus_queue.cc:1048] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:38.012739 14872 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:37.935969  6623 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:38.014228  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 78333.9us
I20260709 12:20:38.015463  6575 raft_consensus.cc:2410] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 28 because replica has last-logged OpId of term: 26 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 13.
I20260709 12:20:37.949329  6548 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:37.961820  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:38.017731  7272 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 26.
I20260709 12:20:38.017912 15010 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:37.988020 14858 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:38.018374 15010 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:38.018822 15010 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:38.018728 14858 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:37.990643  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 46872.1us
I20260709 12:20:37.990932 14970 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:37.768683 14956 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:38.025286  7240 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:38.122058  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 96875.1us
I20260709 12:20:37.779333 14683 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:38.064107  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:38.066339  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:38.133420  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 223849us
W20260709 12:20:38.007295 14993 log.cc:927] Time spent T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
W20260709 12:20:38.080036  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 269343us
I20260709 12:20:38.163148  6473 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:38.163456  6655 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 25: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:38.050947 14982 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:38.106024  7238 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:38.112517  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 60753.7us
I20260709 12:20:38.026805  6474 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:37.810797 14486 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:38.021320 14858 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:38.157398  6548 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:38.088694  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:37.936427 15001 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:20:38.184716  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:38.134224  6650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 3960) took 1220 ms. Trace:
I20260709 12:20:38.090619 14934 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:38.183881  7265 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
W20260709 12:20:38.199836  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 93899.8us
I20260709 12:20:38.158497 15009 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:38.200284  6548 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:38.202988  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 58322.2us
I20260709 12:20:38.203384  6548 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:38.160989 14968 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:38.206700  6473 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:38.166760 14901 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:20:38.209069  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 182354us
I20260709 12:20:38.166901 14680 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:38.204608  6650 rpcz_store.cc:276] 0709 12:20:36.913610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:36.935613 (+ 22003us) service_pool.cc:224] Handling call
0709 12:20:36.936049 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:37.164902 (+228853us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:37.164957 (+    55us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:37.165026 (+    69us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:37.165080 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:38.035204 (+870124us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b5400227ed4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:38.133969 (+ 98765us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:38.134156 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":525579648}
I20260709 12:20:38.182425 14682 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:38.186514 15001 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:38.220394  6473 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:38.200964 14982 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:38.290501  6548 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:38.204923 15009 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:38.208120  6623 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:38.199699 14953 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:38.295611  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:38.297016  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:38.298899  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 78650.2us
I20260709 12:20:38.299026  6657 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 23.
W20260709 12:20:38.299362  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 91326.6us
I20260709 12:20:38.299789  6473 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:38.298923  6655 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 28.
I20260709 12:20:38.216921 14990 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:38.208868 14963 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:38.206476 14989 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:20:38.212805 14680 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:38.156152 14871 consensus_queue.cc:1059] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.140s
I20260709 12:20:38.291270 15001 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:38.291517 14985 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:20:38.305711 14872 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:38.346280  7268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38562 (request call id 1336) took 1237 ms. Trace:
I20260709 12:20:38.319602 14963 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:38.346844  7268 rpcz_store.cc:276] 0709 12:20:37.100705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:37.100933 (+   228us) service_pool.cc:224] Handling call
0709 12:20:37.108194 (+  7261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:37.123629 (+ 15435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:37.123676 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:37.123732 (+    56us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:20:37.123775 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:38.338131 (+1214356us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b54005ddc54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:38.338205 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:38.338402 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34915200}
I20260709 12:20:38.323791 14680 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:38.217417 14988 consensus_queue.cc:1059] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.121s
I20260709 12:20:38.307577 14990 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:38.303129 14968 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:20:38.344722  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 460340us
I20260709 12:20:38.311650  6548 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:38.301687 15009 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:38.322921 14983 consensus_meta.cc:287] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.622s	user 0.006s	sys 0.005s
I20260709 12:20:38.323526 14708 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:38.319612 14952 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:38.209422 15011 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.121s
I20260709 12:20:38.332275 14858 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:38.267356 14901 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:38.357214  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:38.329501 15014 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:20:38.377424  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 65870.1us
I20260709 12:20:38.313804 14953 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:38.346061 14872 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:38.391674  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:38.355122 14963 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:38.375056 15002 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:38.405678 15015 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
W20260709 12:20:38.408872 15012 log.cc:927] Time spent T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:20:38.409099 14871 consensus_queue.cc:1048] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:38.412098  6575 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:20:38.412840 14486 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:38.413429 14683 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:38.395839 15014 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:38.400396 14894 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:38.414278 14953 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:38.358567 14985 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:38.385440 14988 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:38.489500  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:38.384565 14968 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:38.418579  6622 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:38.405225 14872 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:20:38.457393 15017 log.cc:927] Time spent T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 12:20:38.457468 14858 raft_consensus.cc:697] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 25 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:38.456517  6575 raft_consensus.cc:606] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:38.459813 15014 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:38.378769 14983 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:38.418844 14956 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:38.455493 15002 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:38.461194 14894 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:38.420210 15008 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:38.404670 14970 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:38.481916 14768 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:20:38.467386 15013 log.cc:927] Time spent T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 12:20:38.489319  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:38.358282 14990 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:38.526310  7273 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:38.526281  6501 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:20:38.378679 14682 raft_consensus.cc:3055] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:38.491673  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:38.490900  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:38.529251  6505 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 18.
I20260709 12:20:38.529491  6502 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 20.
W20260709 12:20:38.482357 15000 consensus_meta.cc:287] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.696s	user 0.010s	sys 0.000s
I20260709 12:20:38.377899 14910 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.110s
I20260709 12:20:38.495524  6575 raft_consensus.cc:614] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:38.498544  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:38.506561  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:38.506603  6550 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.015s	sys 0.011s
I20260709 12:20:38.507089  6500 raft_consensus.cc:1217] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Deduplicated request from leader. Original: 24.13->[27.14-27.14]   Dedup: 27.14->[]
I20260709 12:20:38.509586  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:38.512673  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:38.495832 14882 raft_consensus.cc:2764] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:20:38.549170  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:38.550014  6500 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 27. Current term is 28. Ops: [27.14-27.14]
I20260709 12:20:38.489347 14988 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:38.526224  6504 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 12:20:38.522723 14970 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:38.480181 15009 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:38.495086 14933 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:38.579977 14933 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:38.527176 14768 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:38.582122 14933 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:38.549476  6550 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.793s	user 0.017s	sys 0.012s
W20260709 12:20:38.582780  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 826693us
I20260709 12:20:38.583448  6550 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:38.584187 14902 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:38.585515 14768 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:38.492693 14968 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:38.551934  6500 raft_consensus.cc:1275] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:20:38.552022  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:38.509578 14894 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:20:38.490463  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 71940.5us
I20260709 12:20:38.592640 14934 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:38.593140 14934 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:38.593982 15002 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:38.556731 14970 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:38.534267 14926 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:38.555408  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:38.600660  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:38.555298 14988 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:38.606699  6622 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:38.557010  6658 raft_consensus.cc:2393] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 19: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:38.608903  6473 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:38.611380 14926 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:38.611837 14926 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:20:38.612231  6400 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57914 (request call id 192) took 89919 ms (client timeout 15000 ms). Trace:
W20260709 12:20:38.612906  6400 rpcz_store.cc:269] 0709 12:19:08.692801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:08.693163 (+   362us) service_pool.cc:224] Handling call
0709 12:19:08.983321 (+290158us) spinlock_profiling.cc:243] Waited 22.2 ms on lock 0x7b5400002210. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d650b2 00007f287bcde759 00007f287bc76a76 00007f287bc47d58 00007f287bd4604e 00007f287a2072a4 00007f287a207232 00007f287a207161 00007f287a2070dc
0709 12:19:09.169362 (+186041us) spinlock_profiling.cc:243] Waited 84.4 ms on lock 0x7b5400002210. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64e9c 00007f2872d650f6 00007f287bcde759 00007f287bc76a76 00007f287bc47d58 00007f287bd4604e 00007f287a2072a4 00007f287a207232 00007f287a207161 00007f287a2070dc
0709 12:20:38.612100 (+89442738us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48319,"reported_tablets":52,"spinlock_wait_cycles":245173248}
I20260709 12:20:38.522313 15008 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:38.615234  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:38.632012  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:38.632082  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:38.633496  7269 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:20:38.634817  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:38.635548  6658 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:38.558907 15010 consensus_meta.cc:287] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.539s	user 0.013s	sys 0.000s
I20260709 12:20:38.559532 14521 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:38.493119 14985 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:38.561571 14543 raft_consensus.cc:3055] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:20:38.528415 14682 raft_consensus.cc:740] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:38.567016 14858 consensus_queue.cc:237] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:38.574082 14943 consensus_meta.cc:287] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.598s	user 0.009s	sys 0.000s
I20260709 12:20:38.574993  7265 raft_consensus.cc:2368] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 19. Current term is 20.
I20260709 12:20:38.574712 14680 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:38.575948 15009 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:38.576396 14983 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:38.534950 15000 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:38.579434 14901 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:38.590911 14990 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:38.615510  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:38.616098  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:38.617763  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:38.618455  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:38.622098  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:38.723347  6584 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 23.
I20260709 12:20:38.622363  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:38.640053  6474 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:38.657778  6506 raft_consensus.cc:2393] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 28: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:38.658666  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:38.725621  6649 raft_consensus.cc:1275] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:20:38.658910  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:38.726485  6504 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:20:38.727397  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:38.728315 14680 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:38.728480  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:38.729599  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:38.651268  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:38.612221 14926 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:38.706702 14486 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:38.736681 14486 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:38.738930 14486 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:38.740465 14990 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:38.741153  6502 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 26. Current term is 27. Ops: []
I20260709 12:20:38.745451 14985 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:38.746918  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 106994us
I20260709 12:20:38.690536  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:38.706755  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 201776us
I20260709 12:20:38.660841 14521 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:38.707026  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:20:38.714010  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:38.753579  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:38.754208  6582 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 25.
I20260709 12:20:38.644177 14882 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:38.714949  6581 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 27 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:20:38.659910 14543 raft_consensus.cc:740] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:38.689288 14980 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:38.756933  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:38.758507  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:38.759092  6584 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 25.
I20260709 12:20:38.716521  6475 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:38.722373  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:38.747893  6474 leader_election.cc:400] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 19. Current term is 20.
I20260709 12:20:38.761349  6474 leader_election.cc:403] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:38.724293  6500 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 23.
I20260709 12:20:38.725665  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:38.749750  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:38.714568  6501 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 27.
I20260709 12:20:38.630182 14963 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:38.764861  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:38.764961  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:38.765790  6577 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 23.
I20260709 12:20:38.766271  6582 raft_consensus.cc:1275] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 12:20:38.766642  6582 pending_rounds.cc:85] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 12
I20260709 12:20:38.766875  6582 pending_rounds.cc:107] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 12:20:38.767100  6582 raft_consensus.cc:2889] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:38.760965  6656 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 27.
I20260709 12:20:38.659291 15010 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:38.769881  6658 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 20.
W20260709 12:20:38.776949  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 159235us
I20260709 12:20:38.777630  7240 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:38.777884  6622 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:38.771425  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 54992.1us
I20260709 12:20:38.772809  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:38.779533  7273 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:38.773639  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:38.779551  6475 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:38.784514  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:38.785188  6576 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 26. Current term is 27. Ops: []
I20260709 12:20:38.800791  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:38.801640  6583 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 28.
I20260709 12:20:38.808350  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:38.761137 14921 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:38.695189 15001 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:38.817229  6578 raft_consensus.cc:2393] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 23: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:38.782418 14985 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:38.820544  6655 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 28.
I20260709 12:20:38.784564 14680 raft_consensus.cc:3060] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:20:38.802754 14979 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:20:38.834815  6548 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:38.837906  6655 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:38.842059  7238 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:38.848466 14543 consensus_queue.cc:260] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:38.849742 14543 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:20:38.775573 14713 consensus_queue.cc:1059] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.052s
I20260709 12:20:38.813202 14683 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:38.852125 14683 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:38.800311 14934 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:38.852536 14683 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:38.854688 14934 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:38.856586  6503 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:38.817042 14774 raft_consensus.cc:2804] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:38.813771 14921 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:38.821858 15011 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:20:38.872088  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 94256.2us
I20260709 12:20:38.805611 14896 consensus_queue.cc:1048] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 12:20:38.833838 14985 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:38.881319  6550 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:38.878491  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:38.876821  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:38.882866  7272 raft_consensus.cc:1240] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 16. Current term is 17. Ops: []
W20260709 12:20:38.881171  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:38.800420 14980 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:38.877422  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:38.875900 15011 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:38.885783  6655 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:20:38.887939  6548 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:38.888528  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:38.884927  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:38.867097 14998 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:38.803201 14682 consensus_queue.cc:260] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:38.889117  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:38.862952 14952 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:38.855683 14902 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.030s
I20260709 12:20:38.906718  6504 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 12:20:38.889113  6548 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:38.891719  6656 raft_consensus.cc:2393] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 27: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:38.875037 14921 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:38.874637 14943 raft_consensus.cc:3055] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:38.889681  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:38.907508 15024 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:38.911289 15000 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:38.890190 15011 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:20:38.912660  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:20:38.912984 15033 log.cc:927] Time spent T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:20:38.913853 15030 log.cc:927] Time spent T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:20:38.914847 15010 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:38.917601 15001 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:38.888135 14979 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:38.907326 14998 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:38.967186  7273 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 27.
I20260709 12:20:38.909008  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:38.925251  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:38.882851 14774 raft_consensus.cc:697] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 20 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
W20260709 12:20:38.938202 15028 log.cc:927] Time spent T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 12:20:38.928223  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:20:38.994169  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 69010.6us
I20260709 12:20:38.912072 14872 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:38.995884  7269 raft_consensus.cc:2393] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 23: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:20:38.890211 14901 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:20:38.939157  6502 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:38.935806 15008 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:38.944739  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:38.946009 14902 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:38.950547 14680 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 19. Current term is 20.
I20260709 12:20:39.003916  6622 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:38.953890 14980 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:38.919147  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:38.970553  6503 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 23.
I20260709 12:20:38.890450  6550 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:39.025604  7272 raft_consensus.cc:2368] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 22. Current term is 23.
I20260709 12:20:38.882454 14985 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:38.885519 14486 consensus_queue.cc:1048] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 12:20:38.935528 14871 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:38.937024 14682 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:38.928970 14943 raft_consensus.cc:740] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:38.998714  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:20:39.025840  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 135474us
W20260709 12:20:39.137142  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 248082us
I20260709 12:20:38.956041 14521 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:38.991532 14998 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:38.991021 14861 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:38.988555 14979 raft_consensus.cc:697] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 24 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:20:39.053069  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 43965.2us
I20260709 12:20:38.978389 14768 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:39.054134  6658 raft_consensus.cc:2393] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 21: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
W20260709 12:20:39.136202  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 132397us
I20260709 12:20:39.138490  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:39.000808 14901 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:39.007337  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:39.144634  6622 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:20:38.938545 14713 consensus_queue.cc:1175] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f2b06aebe97249a4877d798b8118ece4" responder_term: 25 status { last_received { term: 25 index: 17 } last_committed_idx: 17 last_received_current_leader { term: 25 index: 17 } }
I20260709 12:20:38.960238 14970 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:39.026719 14999 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:39.052942 15008 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:20:39.151635  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:39.167574 15010 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:20:39.144049 14963 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:39.170343 14968 consensus_meta.cc:287] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.583s	user 0.009s	sys 0.001s
I20260709 12:20:39.170524  6622 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:39.170838 14968 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:39.172122 15021 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:39.172562 15021 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:39.172941 15021 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:39.173588 14968 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:39.176693  6584 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:20:39.179040  7269 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 26. Current term is 27. Ops: []
I20260709 12:20:39.024724 14902 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:39.180019 14774 consensus_queue.cc:237] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:39.182101  7238 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:39.182433 14902 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:39.183377 15009 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:20:39.192675 14979 consensus_queue.cc:237] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:39.148016 14901 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:39.194105  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:20:39.139926 15037 log.cc:927] Time spent T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 12:20:39.154062  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:39.215816  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:20:39.143425 14988 consensus_meta.cc:287] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.538s	user 0.010s	sys 0.000s
I20260709 12:20:39.143409 14933 consensus_queue.cc:1059] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.022s
I20260709 12:20:39.166054  6656 raft_consensus.cc:2393] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 27: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:39.198916  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:39.220942  7272 raft_consensus.cc:2393] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 23: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:20:39.000142 15036 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:39.200054  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:20:39.221858 15044 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:20:39.204522  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:39.205026 14968 consensus_queue.cc:1048] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:39.204087 15043 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
I20260709 12:20:39.205328 14921 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:39.220790 14894 consensus_meta.cc:287] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.565s	user 0.008s	sys 0.001s
I20260709 12:20:39.208915 14970 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:39.203943  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:39.257023  6583 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:20:39.209511 14872 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:39.025990 14896 consensus_queue.cc:1048] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.127s
I20260709 12:20:39.202979 14543 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:39.191376 15021 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:39.196077 15011 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:39.263003 14521 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:39.215631  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:39.263659 15021 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:39.163698  6501 raft_consensus.cc:2468] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 20.
I20260709 12:20:39.217334  6473 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:39.269960  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 52709.9us
I20260709 12:20:39.203049 14952 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:39.141327 15001 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:39.292788 14921 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:39.139717 15040 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:39.203130 14748 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.212550 14999 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:39.254907  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 55972.7us
I20260709 12:20:39.209558 15008 raft_consensus.cc:697] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:39.273914  6502 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 25.
I20260709 12:20:39.275493 14683 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:39.281658 15014 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:39.284262  6474 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:39.318773  6474 leader_election.cc:400] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 22. Current term is 23.
I20260709 12:20:39.319196  6474 leader_election.cc:403] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:20:39.291771 14680 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:39.292526 15001 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:39.324071 15001 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:39.152146 14871 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:39.150200 14861 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:39.224095 14902 consensus_queue.cc:1059] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.043s
I20260709 12:20:39.212968 14998 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:39.306591  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 3180) took 1516 ms. Trace:
I20260709 12:20:39.308153  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:39.309981  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:20:39.316412  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 174208us
I20260709 12:20:39.321916 14708 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:20:39.326581  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:39.222604 14988 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:39.331840  6650 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 26. Current term is 27. Ops: []
I20260709 12:20:39.333079 15002 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 22. Current term is 23.
I20260709 12:20:39.266649 14894 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:39.327975 14680 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:39.367660  6583 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 27.
I20260709 12:20:39.375105  6657 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:20:39.321938 14926 consensus_meta.cc:287] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.590s	user 0.010s	sys 0.000s
I20260709 12:20:39.377714  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:39.388985  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:39.388659 14926 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:39.389739  6505 raft_consensus.cc:2393] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 27: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:39.379959  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:39.391090 14926 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:39.391506  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:39.392679  6505 raft_consensus.cc:2393] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 21: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:39.391558  6504 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22.
I20260709 12:20:39.334116  6581 rpcz_store.cc:276] 0709 12:20:37.790100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:37.790908 (+   808us) service_pool.cc:224] Handling call
0709 12:20:37.837935 (+ 47027us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:38.756232 (+918297us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:39.306337 (+550105us) spinlock_profiling.cc:243] Waited 875 ms on lock 0x7b54000366d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:39.306529 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2013113728}
W20260709 12:20:39.381685  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 238438us
I20260709 12:20:39.381829  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:39.395776  6548 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:39.397964  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:39.318122 15014 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:39.400144 15014 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:39.402890  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:39.300988 15006 consensus_queue.cc:1048] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 12:20:39.385296  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:39.409201  7269 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:39.410185  7265 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:39.409250 14708 raft_consensus.cc:697] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 27 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:39.303422 14748 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:39.415130 14708 consensus_queue.cc:237] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:39.417382 14683 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:39.416134 14680 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:39.420018  6657 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 29.
W20260709 12:20:39.424414  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 97168.4us
I20260709 12:20:39.425834  6649 raft_consensus.cc:1240] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 24. Current term is 25. Ops: []
W20260709 12:20:39.427106  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 102287us
I20260709 12:20:39.423079  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:39.411728 14989 raft_consensus.cc:3055] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:20:39.434710  6548 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:39.439242 14998 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:39.319356 15008 consensus_queue.cc:237] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:39.440438  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:39.441641 14748 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:39.433336 14989 raft_consensus.cc:740] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:39.444269 14989 consensus_queue.cc:260] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:39.445580 14989 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:39.445895 14985 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:39.447739  6504 raft_consensus.cc:2393] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 24: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:39.449824 14988 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:39.452682 14861 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:39.458151  7265 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 21.
I20260709 12:20:39.423887 14858 consensus_queue.cc:1059] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
I20260709 12:20:39.458860 15047 consensus_queue.cc:1059] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.002s
I20260709 12:20:39.455124  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
W20260709 12:20:39.475025  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:39.459661 14902 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:20:39.483013  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 48405.3us
I20260709 12:20:39.005829  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:39.484616 14682 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:39.485520 14953 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.485667  7272 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:39.460692 14985 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:39.484010  6548 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:39.492758  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:39.493755  6651 raft_consensus.cc:2393] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 19: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:39.486931 14999 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:39.501260  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41304 (request call id 1123) took 1012 ms. Trace:
I20260709 12:20:39.501706  6503 rpcz_store.cc:276] 0709 12:20:38.488923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:38.489038 (+   115us) service_pool.cc:224] Handling call
0709 12:20:39.501080 (+1012042us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b540001ccd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:39.501197 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":762495744}
I20260709 12:20:39.502343 14872 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:39.502915 14872 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:39.505084 14872 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:39.455089 14683 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:39.513201  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 49129.8us
I20260709 12:20:39.465713 14979 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:39.516287  6548 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:39.524315  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:39.575903 15056 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:39.524943 14961 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:39.580607 14680 raft_consensus.cc:3055] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:20:39.581169 14680 raft_consensus.cc:740] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:39.582099 14680 consensus_queue.cc:260] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:39.485996 14953 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:39.516669 14683 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:39.591156 14683 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:39.515305 14768 consensus_queue.cc:1059] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.078s
I20260709 12:20:39.598173 14543 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.502312 14901 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:39.601215  6580 raft_consensus.cc:2368] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 27. Current term is 28.
I20260709 12:20:39.602357 14901 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:39.603163  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 79518.9us
I20260709 12:20:39.603562 14901 consensus_queue.cc:1059] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:39.604122 14894 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:39.604871 14901 raft_consensus.cc:3055] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:39.605173  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:39.590656 14953 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:39.610368  6640 tablet_service.cc:1558] Processing DeleteTablet for tablet 3a47d71c34b541b9b2589b4e80413fa2 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 6)) from {username='slave'} at 127.0.0.1:56214
I20260709 12:20:39.615294  6649 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:39.619191  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:39.619943  6649 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:39.598738 14543 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:39.615171 14943 consensus_queue.cc:260] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:39.627054 14943 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:39.627872 14543 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:39.605268 14901 raft_consensus.cc:740] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:39.517114 14979 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:39.643388 14979 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:39.645639 14901 consensus_queue.cc:260] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:39.650158  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:39.659983  6548 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:39.671442  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 57754.3us
I20260709 12:20:39.687529  6504 raft_consensus.cc:2393] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 28: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:39.697007  6548 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:39.668958 15064 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.703510  6653 raft_consensus.cc:1275] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:20:39.699990  6649 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 28.
I20260709 12:20:39.701956 14988 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:39.712751 14956 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:39.736547  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:39.736771  6623 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:39.669888 15014 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:39.716768 14901 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:39.720245 14543 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.727119  6550 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:39.739691 14543 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:39.744968 14543 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:39.740154 15056 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:20:39.733201 15064 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:39.742791 15027 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.002s
I20260709 12:20:39.747525  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:39.748158  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 56769.8us
W20260709 12:20:39.749272  6548 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:39.763741  6582 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 23.
I20260709 12:20:39.770087  7269 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 28.
I20260709 12:20:39.770982  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:39.771854  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:39.772619  6577 raft_consensus.cc:2393] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 27: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:39.772661  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:39.773073 15064 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:39.773864  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:39.774436  6577 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:39.774642 14902 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.775233 14902 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:39.777530 14989 raft_consensus.cc:3055] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:39.777959 14989 raft_consensus.cc:740] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:39.779150  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:39.778692 14902 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:39.779754  6653 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 27.
I20260709 12:20:39.778800 14989 consensus_queue.cc:260] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:39.780897  6550 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:39.783406  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:39.784042  6576 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 25.
I20260709 12:20:39.753675 14543 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:20:39.786245  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:39.784935  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:39.769484 15056 raft_consensus.cc:697] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:39.738212 14872 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 12:20:39.786521  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:39.787032  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:39.789604  6583 raft_consensus.cc:2410] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:20:39.789665  6579 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:39.787302  6653 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 27.
I20260709 12:20:39.793388  6578 raft_consensus.cc:1240] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 22. Current term is 23. Ops: [22.11-22.11]
I20260709 12:20:39.794080  7240 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:39.797538  6622 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:39.780761 14989 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:39.826905  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:39.828377  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:39.830488  6581 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 25.
I20260709 12:20:39.837184 15053 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:20:39.886168  6549 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:39.911470  6578 raft_consensus.cc:1240] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:39.827055 15064 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:20:39.949182  6577 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 24.
I20260709 12:20:39.949399 15064 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:39.949960 15064 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:20:39.826776 14902 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:20:39.952593 14902 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:39.953050 14902 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:39.933876  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41304 (request call id 1130) took 1045 ms. Trace:
I20260709 12:20:39.967123  6501 rpcz_store.cc:276] 0709 12:20:38.888067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:38.888266 (+   199us) service_pool.cc:224] Handling call
0709 12:20:39.933673 (+1045407us) spinlock_profiling.cc:243] Waited 246 ms on lock 0x7b54003d3bd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:39.933803 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":564769152}
I20260709 12:20:39.912757 14683 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:20:39.942749 15014 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:39.977799  6579 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 27.
I20260709 12:20:39.996708 15009 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:39.997391 15009 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:40.000972 14757 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:40.001804 14757 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:40.004020  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:40.009801 14896 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:40.008873 14979 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.010985 14896 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:40.014503 14896 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:40.021644 14757 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:40.015539 14979 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:40.017809  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:40.025437 14901 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:40.018389 15010 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:40.026254 14901 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:40.026909 15011 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:40.026758 15010 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.029346 15010 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:40.032593  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:40.037377 14680 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:20:40.050288  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 195740us
I20260709 12:20:40.051438  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:40.053416  6622 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:40.054412  6549 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:40.052285  6649 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 28. Current term is 29. Ops: []
W20260709 12:20:40.057001  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 174866us
I20260709 12:20:40.064311  6622 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:40.067979  6622 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:40.070191  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:40.073207  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:40.054960 14708 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.034s	sys 0.002s
W20260709 12:20:40.056681 15008 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.009s	sys 0.015s
I20260709 12:20:40.022665 15009 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:40.081594 14896 consensus_queue.cc:1059] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.004s
I20260709 12:20:40.082345  6622 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:40.056816 14901 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:40.068603 15051 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:40.084636 14708 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.034s	sys 0.002s
I20260709 12:20:40.099808  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:40.068077 15053 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:40.101796  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:40.081944 15010 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:39.788066 15056 consensus_queue.cc:237] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 26.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.090529  6576 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 25.
W20260709 12:20:40.085034 15008 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.009s	sys 0.015s
I20260709 12:20:40.106134 14872 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:40.078593 14933 consensus_queue.cc:1059] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 12:20:40.101177 14980 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:20:40.110788  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:40.117331  6506 raft_consensus.cc:2410] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:20:40.112815  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:40.117897  6503 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 25.
I20260709 12:20:40.119838  6655 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:40.114014  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:40.094074 14882 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:40.121793  6508 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:40.130748  6508 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 25.
I20260709 12:20:40.081615 15077 consensus_queue.cc:1059] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 12:20:40.108369 15053 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:40.108605 15010 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:40.110003 15021 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:40.136809 15051 consensus_queue.cc:1059] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:40.147141 14882 raft_consensus.cc:697] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 24 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:40.148236 14882 consensus_queue.cc:237] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.143349 15053 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:40.143656 15010 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:40.116555 14872 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:40.146543 15021 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.157526 14980 raft_consensus.cc:697] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 27 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:40.170217 14902 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.171620 15024 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:20:40.181620 15049 log.cc:927] Time spent T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.903s	user 0.000s	sys 0.026s
I20260709 12:20:40.175814 14713 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:40.190904 15021 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:40.190766 14713 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.193300 14713 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:40.194147 14980 consensus_queue.cc:237] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.197319 15024 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:40.197788 14872 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:40.199046  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:40.203366  6623 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:40.203886  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56116 (request call id 3536) took 1197 ms. Trace:
I20260709 12:20:40.204435  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:40.217900  7269 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:20:40.209565  7238 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:40.230988 15064 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:40.233548 15064 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:40.236742 14902 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:40.238912 15024 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:40.240571  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 773792us
I20260709 12:20:40.241312  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:40.242122  7265 raft_consensus.cc:2393] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 17: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:20:40.243430  6473 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:40.245249  6623 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:40.246246  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:40.246862  6653 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:20:40.247612 15021 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:20:40.247684 14992 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:40.248111 15021 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:40.248528 14901 raft_consensus.cc:3055] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:20:40.248588 14985 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:20:40.249245 15083 log.cc:927] Time spent T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:20:40.217105  6580 rpcz_store.cc:276] 0709 12:20:39.006801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:39.006981 (+   180us) service_pool.cc:224] Handling call
0709 12:20:40.203701 (+1196720us) spinlock_profiling.cc:243] Waited 456 ms on lock 0x7b54000573d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:40.203833 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1048880896}
I20260709 12:20:40.249789  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:40.250532  6649 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:40.250648  6550 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:40.249171 14901 raft_consensus.cc:740] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:40.252266 14901 consensus_queue.cc:260] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 28.15, Last appended by leader: 15, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.253140 15009 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:40.253640 15009 raft_consensus.cc:697] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 23 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:40.254587 15009 consensus_queue.cc:237] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:40.253720 14901 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:20:40.258481  6473 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:20:40.260141  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 831199us
W20260709 12:20:40.259569  6622 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.5.58.62:36381, user_credentials={real_user=slave}} blocked reactor thread for 44409.1us
I20260709 12:20:40.263753  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:40.264843  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:40.265904  6580 raft_consensus.cc:2393] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 27: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:40.260540  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 3363) took 1642 ms. Trace:
I20260709 12:20:40.271010  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:40.248540 15021 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:40.274613  6649 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 28.
I20260709 12:20:40.247714 15057 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:20:40.275822 15057 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:40.276162 15057 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:40.276767  6550 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:40.277979 15077 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:40.278470 15077 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.280457 15024 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:40.281096 15024 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.282074 15021 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:40.283854 15024 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:40.261035 15008 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:20:40.285369  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:40.285874  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:40.286911  6653 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:20:40.267071  6509 rpcz_store.cc:276] 0709 12:20:38.618027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:38.748441 (+130414us) service_pool.cc:224] Handling call
0709 12:20:38.748724 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:38.748923 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:20:38.935401 (+186478us) log.cc:844] Serialized 14 byte log entry
0709 12:20:39.745422 (+810021us) spinlock_profiling.cc:243] Waited 1.18 ms on lock 0x7b180045bce0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d16c0a 00007f287c456219 00007f2872d9ea40 00007f2872d994e4 00007f2872d99455 00007f28798813c7 00007f2875e939a7 00007f2875ea30bf 00007f2875ea3038
0709 12:20:39.769007 (+ 23585us) log.cc:844] Serialized 14 byte log entry
0709 12:20:40.260198 (+491191us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:40.260249 (+    51us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:20:40.260294 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:40.260375 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:40.260488 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":641087,"prepare.queue_time_us":891985,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":79,"raft.queue_time_us":2318,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":4316,"spinlock_wait_cycles":2707200,"thread_start_us":182789,"threads_started":2}
I20260709 12:20:40.295051  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:40.301492 15008 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:40.302047 15008 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:20:40.304579  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:40.305303 15052 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:40.305398  6474 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:40.305917 15052 raft_consensus.cc:697] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 25 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:40.306766 15052 consensus_queue.cc:237] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.308319  7269 raft_consensus.cc:2368] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 28. Current term is 29.
I20260709 12:20:40.314029  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:40.314283  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:40.314813  6501 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:40.315522  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:40.316959  6508 raft_consensus.cc:1275] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
W20260709 12:20:40.320818  7238 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.568s (SENT) [suppressed 3 similar messages]
I20260709 12:20:40.325868  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:40.326145  6622 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:40.290431 15002 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:40.329895  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:40.331493  6622 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:40.326516  6509 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 27.
I20260709 12:20:40.326149  6653 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 29.
I20260709 12:20:40.330669  6501 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 23.
I20260709 12:20:40.288421 15021 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:40.348198  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 90628.9us
I20260709 12:20:40.324851 14933 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:40.368906  6473 leader_election.cc:400] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 27. Current term is 28.
I20260709 12:20:40.337090 15077 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:40.299413 14902 raft_consensus.cc:2804] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader election won for term 28
W20260709 12:20:40.370323  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 103497us
I20260709 12:20:40.342554 14979 raft_consensus.cc:3055] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:39.846053  6394 catalog_manager.cc:5028] TS f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): tablet 3a47d71c34b541b9b2589b4e80413fa2 (table test-workload [id=f93362d0518e483fbe3a01f43a217e48]) successfully deleted
I20260709 12:20:40.328882 15088 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:40.301518 14748 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:40.369261  6473 leader_election.cc:403] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 election: Cancelling election due to peer responding with higher term
I20260709 12:20:40.343622 15010 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.325350 15011 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:20:40.373529  6394 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 527599us
W20260709 12:20:40.378993  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 47570.5us
W20260709 12:20:40.380718  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 50084.1us
I20260709 12:20:40.369354 14486 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:40.388619  6622 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:40.392127  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:40.392907  6575 raft_consensus.cc:2393] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 24: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:40.371942 14979 raft_consensus.cc:740] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:40.375291 15058 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:20:40.388810  6475 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:40.374006 15056 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:40.376513 14902 raft_consensus.cc:697] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 28 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:40.334676 15002 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:40.399348  6576 raft_consensus.cc:1275] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:20:40.408097  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:40.412828  6579 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:20:40.409603  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:40.410075  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:40.372541 14933 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.385499 15088 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:40.398940 15014 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:40.409214 15011 raft_consensus.cc:697] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:40.461258  6623 leader_election.cc:400] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 election: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 28. Current term is 29.
I20260709 12:20:40.461632  6623 leader_election.cc:403] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 election: Cancelling election due to peer responding with higher term
I20260709 12:20:40.461809 15011 consensus_queue.cc:237] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:40.463493 15008 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:40.431609 14813 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:40.431696 15058 raft_consensus.cc:697] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 27 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:40.432185 14926 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:40.432271 14708 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:20:40.432780  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 44012.2us
I20260709 12:20:40.432883 14910 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:20:40.433264  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 45707.4us
I20260709 12:20:40.433518 14861 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:20:40.434823 14968 raft_consensus.cc:3055] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:40.436908 14894 consensus_queue.cc:1048] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:40.440433  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:40.441764  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:40.441694 15057 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:40.442814  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:40.443015 14902 consensus_queue.cc:237] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.443778  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:40.444859 15010 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:40.446606 15002 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:40.447211  6505 raft_consensus.cc:2393] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 28: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:40.406131 14871 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:40.447427 14933 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:40.377344 14748 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.549038 14748 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:40.398595 14486 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.551545 14486 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:40.449272 15053 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.553771 15053 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:40.451095  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:40.451476  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:40.554771  6649 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 26.
I20260709 12:20:40.431078 15005 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:40.555675 15005 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.557932 15005 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:40.504215  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:40.558393 15057 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:40.559063  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 54931.6us
I20260709 12:20:40.559331  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:40.520407  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:40.561046 14924 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:20:40.561707  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:40.561815 14924 raft_consensus.cc:697] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 25 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:40.563273 14924 consensus_queue.cc:237] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:40.567427  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:40.573102 15058 consensus_queue.cc:237] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.495985 14872 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Leader election lost for term 28. Reason: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 28. Current term is 29.
I20260709 12:20:40.578959 15008 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:40.502005 14980 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:40.582574 14980 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.584892 14980 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:40.585021  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:40.586005  6654 raft_consensus.cc:2368] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18.
I20260709 12:20:40.513459 14813 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:40.590958 14813 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:40.592355  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 141306us
I20260709 12:20:40.593398  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:40.513345 14882 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:40.595470 14882 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.596159 14543 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:40.596763 14543 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:40.597229 14543 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:40.598423 14882 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:40.599344 14882 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:40.601552  6550 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:40.602897  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:40.603605  7272 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:40.604590  6550 leader_election.cc:400] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18.
I20260709 12:20:40.604991  6550 leader_election.cc:403] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:20:40.605751 14486 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:20:40.606202 14486 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:40.606555 14486 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:40.530755 14968 raft_consensus.cc:740] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:20:40.602958  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:40.609150  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:40.608955 14968 consensus_queue.cc:260] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.609838  6657 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 30.
I20260709 12:20:40.610132 14861 raft_consensus.cc:697] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 29 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:40.611250 14861 consensus_queue.cc:237] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 27.14, Last appended by leader: 14, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.525540 14708 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:40.530961 14985 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:40.615921 14985 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.534066 14894 consensus_queue.cc:1243] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 22.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:20:40.525671 14680 consensus_queue.cc:1059] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.095s
I20260709 12:20:40.621343 14680 raft_consensus.cc:3055] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:20:40.621850 14680 raft_consensus.cc:740] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:40.622975 14680 consensus_queue.cc:260] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.623363 14979 consensus_queue.cc:260] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.624845 14979 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:40.627568  7240 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:40.627887 14968 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:40.512740 14926 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.630245 15064 raft_consensus.cc:2804] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:20:40.630739 15064 raft_consensus.cc:697] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 29 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:40.631558 15064 consensus_queue.cc:237] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:40.531752 14910 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:20:40.634913  6580 raft_consensus.cc:2410] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 29 because replica has last-logged OpId of term: 28 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 14.
I20260709 12:20:40.521680  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:40.637672  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:40.522372  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:40.638392  6584 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 26.
I20260709 12:20:40.412616 15056 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.524827  6582 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:20:40.527408  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:40.640451  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:40.520622  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:40.641196  6580 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:40.521180  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:40.640471 14963 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:20:40.643158  6650 raft_consensus.cc:2368] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 27. Current term is 28.
I20260709 12:20:40.528844  6581 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 26. Current term is 27. Ops: [25.15-26.16]
I20260709 12:20:40.537359 15086 raft_consensus.cc:3055] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:40.543085  6575 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 25.
I20260709 12:20:40.644623 15086 raft_consensus.cc:740] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:40.644183  6584 raft_consensus.cc:1240] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 22. Current term is 23. Ops: [22.11-22.11]
I20260709 12:20:40.645545 15086 consensus_queue.cc:260] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.646847 15086 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:40.568182  6652 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:20:40.662613  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:40.663391  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:40.664052  6509 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:40.668911  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:40.669782  6505 raft_consensus.cc:2393] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 26: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:40.692080  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:40.701572  6550 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:40.707226  6473 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:40.708318 14992 consensus_queue.cc:1048] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:20:40.710232 14992 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:40.710598 14992 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:40.711349 14992 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:40.728515  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:40.729741  6584 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:40.735359  6658 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:20:40.766935  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:40.767242  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:40.768728  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:40.771656  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:40.772362  6655 raft_consensus.cc:2368] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 25. Current term is 26.
I20260709 12:20:40.600051 14882 raft_consensus.cc:697] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 23 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:20:40.776654  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 75193.1us
I20260709 12:20:40.781710  6501 pending_rounds.cc:85] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 13
W20260709 12:20:40.781710  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:40.782411  6501 pending_rounds.cc:107] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 22.14
I20260709 12:20:40.782763  6501 raft_consensus.cc:2889] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:40.780272  7272 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
I20260709 12:20:40.778343  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:40.792848  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:40.793907  7272 raft_consensus.cc:2368] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18.
W20260709 12:20:40.798426  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:40.799901  6473 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:40.803220  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:40.816457  6580 raft_consensus.cc:2468] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 26.
I20260709 12:20:40.818073  6658 raft_consensus.cc:1240] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 25. Current term is 26. Ops: []
I20260709 12:20:40.819711 15023 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:20:40.797122 15096 log.cc:927] Time spent T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.128s	user 0.000s	sys 0.004s
W20260709 12:20:40.803290 15097 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.141s	user 0.005s	sys 0.001s
I20260709 12:20:40.814628  6623 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:40.803139 15095 log.cc:927] Time spent T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.143s	user 0.000s	sys 0.007s
I20260709 12:20:40.786808 14680 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:40.858156  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:20:40.800226 15094 log.cc:927] Time spent T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.143s	user 0.005s	sys 0.001s
W20260709 12:20:40.859287  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 59450.6us
I20260709 12:20:40.789265 15009 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:40.876819 14708 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:40.877192  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:40.880085  6658 raft_consensus.cc:2393] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 29: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:20:40.814138 14894 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:40.822656 15053 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:40.811864 15024 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.070s
W20260709 12:20:40.897409  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:40.878213 15056 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:40.820416 15023 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:40.892861 15077 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:20:40.900925 15052 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:20:40.945555  6402 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57888 (request call id 219) took 55183 ms (client timeout 15000 ms). Trace:
I20260709 12:20:40.965482 15023 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:40.967131  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:40.969502  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 111436us
W20260709 12:20:40.971263  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:40.892706 14894 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.890221 15009 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:40.904987 15053 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:40.925534 15011 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:40.926301 14910 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:40.979153  6622 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:40.984886  6584 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 26.
I20260709 12:20:40.944597 15052 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:40.962368 14926 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:40.967703 15077 raft_consensus.cc:697] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:41.119617  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:41.133363  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:41.134670  6576 raft_consensus.cc:1240] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 25. Current term is 26. Ops: [22.13-25.15]
I20260709 12:20:41.141108  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:41.136090  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:41.143116  6576 raft_consensus.cc:2410] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 27 because replica has last-logged OpId of term: 25 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 16.
I20260709 12:20:41.153614  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:41.152601  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 338021us
I20260709 12:20:41.160465  6578 raft_consensus.cc:2393] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 25: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:41.175678  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:41.178109  6549 leader_election.cc:400] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 election: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18.
I20260709 12:20:41.115604 14956 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:20:41.114264 15052 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:41.154389 14757 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:41.161386 15098 log.cc:927] Time spent T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.192s	user 0.000s	sys 0.000s
I20260709 12:20:40.927181 14882 consensus_queue.cc:237] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:41.191299  6503 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 24. Current term is 25. Ops: [22.17-24.18]
I20260709 12:20:41.236472  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:41.191530  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:41.189867  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:41.192586  6575 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:20:41.191412  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:41.194113  6505 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 22. Current term is 23. Ops: []
W20260709 12:20:41.189036  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:41.241176  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:41.242569  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 60537.5us
I20260709 12:20:41.163291 15000 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:41.195303  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 391173us
I20260709 12:20:41.196599  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:41.197116  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:41.232183  6500 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:20:41.162995 15053 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:41.239638  6504 raft_consensus.cc:2393] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 23: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:20:41.244640  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:41.237212  6503 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 26.
W20260709 12:20:40.946134  6402 rpcz_store.cc:269] 0709 12:19:45.762001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:45.811979 (+ 49978us) service_pool.cc:224] Handling call
0709 12:20:40.945463 (+55133484us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:36.257931 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:36.314259 (+ 56328us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:36.323734 (+  9475us) write_op.cc:435] Acquired schema lock
0709 12:20:36.327669 (+  3935us) tablet.cc:662] Decoding operations
0709 12:20:36.365916 (+ 38247us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:36.366025 (+   109us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:36.366060 (+    35us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:20:36.592843 (+226783us) tablet.cc:701] Row locks acquired
0709 12:20:36.612215 (+ 19372us) write_op.cc:260] PREPARE: finished
0709 12:20:36.636526 (+ 24311us) write_op.cc:270] Start()
0709 12:20:36.674126 (+ 37600us) write_op.cc:276] Timestamp: P: 1783599636630930 usec, L: 0
0709 12:20:36.690144 (+ 16018us) op_driver.cc:348] REPLICATION: starting
0709 12:20:36.786255 (+ 96111us) log.cc:844] Serialized 2034 byte log entry
0709 12:20:37.510767 (+724512us) op_driver.cc:464] REPLICATION: finished
0709 12:20:38.018623 (+507856us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b540008acd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8632a 00007f2875e0bf14 00007f2875e0bec3 00007f2875e0be38 00007f2875e0bdf3 00007f2875e0b0dc 00007f2875e34e16 00007f2875e1ccbb 00007f2875e0739d 00007f2875e0ad75
0709 12:20:38.203958 (+185335us) write_op.cc:301] APPLY: starting
0709 12:20:38.210792 (+  6834us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:38.414380 (+203588us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:38.414424 (+    44us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:38.597328 (+182904us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:38.597615 (+   287us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=0,mrs_lookups=0
0709 12:20:38.597671 (+    56us) write_op.cc:312] APPLY: finished
0709 12:20:38.597982 (+   311us) log.cc:844] Serialized 57 byte log entry
0709 12:20:38.709803 (+111821us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:38.714096 (+  4293us) write_op.cc:454] Released schema lock
0709 12:20:38.714164 (+    68us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:38.738663 (+ 24499us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:38.738699 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2000678,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":558876,"apply.run_cpu_time_us":17718,"apply.run_wall_time_us":540675,"cfile_cache_hit":22,"cfile_cache_hit_bytes":24987,"num_ops":5,"prepare.queue_time_us":170606,"prepare.run_cpu_time_us":93565,"prepare.run_wall_time_us":628528,"raft.queue_time_us":301722,"raft.run_cpu_time_us":25544,"raft.run_wall_time_us":597734,"replication_time_us":764268,"spinlock_wait_cycles":43859328,"thread_start_us":722401,"threads_started":5,"wal-append.queue_time_us":243103,"wal-append.run_cpu_time_us":69297,"wal-append.run_wall_time_us":2617022}]]}
W20260709 12:20:41.254972  6509 consensus_meta.cc:287] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.590s	user 0.009s	sys 0.042s
I20260709 12:20:41.273818  6506 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 27.
I20260709 12:20:41.191857 14708 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:41.112777 15011 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:41.272083  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:41.274643  6509 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 26.
I20260709 12:20:41.294992  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:41.261492  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 282421us
I20260709 12:20:41.276034  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:41.272415  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:41.278151  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:41.298843  6504 raft_consensus.cc:1813] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Handling vote request from an unknown peer 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:41.267220  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:41.179123 14926 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:41.279712  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 97861us
I20260709 12:20:41.280248  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:41.307530  6503 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 27.
I20260709 12:20:41.308735  6503 raft_consensus.cc:1240] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 29 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 28. Current term is 29. Ops: []
I20260709 12:20:41.283597  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:41.297011  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:41.300036  6500 raft_consensus.cc:3060] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:41.188136 14901 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:41.302588  6504 raft_consensus.cc:2410] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 12:20:41.303709  6658 raft_consensus.cc:1275] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:20:41.311002  6473 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:41.195531 14862 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:41.313812  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:41.313877  6622 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:41.330839  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:41.302109 14708 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:41.288005 15077 consensus_queue.cc:237] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:41.329423 14926 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:41.343245  6508 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:41.281497 15053 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:41.345891  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:41.238889 14757 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:41.359437  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:41.330838 14979 consensus_meta.cc:287] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.705s	user 0.014s	sys 0.004s
I20260709 12:20:41.363181  6509 raft_consensus.cc:1275] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:20:41.302737 15059 raft_consensus.cc:2764] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader election decision vote started in defunct term 25: won
W20260709 12:20:41.365278  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:41.367702  6508 raft_consensus.cc:2410] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 23 because replica has last-logged OpId of term: 22 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 10.
I20260709 12:20:41.373517  6500 raft_consensus.cc:2468] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 28.
I20260709 12:20:41.273620 15000 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:41.378551  6652 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 23 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:41.341634 15060 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:41.338251 15103 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:41.396364 15103 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:41.399405 15103 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:41.401784  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:41.402532  6509 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 28.
W20260709 12:20:41.404687  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 90305.1us
I20260709 12:20:41.405251  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:41.354024 15115 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:20:41.359920 14902 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:41.409822  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:41.364418 15056 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:41.349027 14862 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:41.394011 15053 raft_consensus.cc:2804] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:20:41.423928  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:41.348394 14901 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:41.425518  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:41.427217  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:41.429111  6655 raft_consensus.cc:2410] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 25 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 12:20:41.354180 15109 consensus_queue.cc:1059] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.079s
I20260709 12:20:41.433341  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:41.433954  6509 raft_consensus.cc:3060] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:20:41.435549  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:41.440472  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:41.445681  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 68602us
I20260709 12:20:41.445780  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:41.455655  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:41.456809  6575 raft_consensus.cc:2393] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 23: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:20:41.447664  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 91868.1us
I20260709 12:20:41.438002 15053 raft_consensus.cc:697] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 26 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:41.436684 14894 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:41.465268  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:41.444778 15115 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:41.424641 15056 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:41.463362  6622 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:41.472503  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:41.421371 14902 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:41.477684  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:41.445832 15009 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:41.469838  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:41.483266  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:41.512061  6653 raft_consensus.cc:1275] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:20:41.514874  6509 raft_consensus.cc:2468] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 29.
I20260709 12:20:41.516410  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:41.517194  6509 raft_consensus.cc:2376] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 20. Re-sending same reply.
I20260709 12:20:41.572396 15053 consensus_queue.cc:237] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:41.581974 14901 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:41.590188 14902 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:41.594324 14543 consensus_meta.cc:287] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.997s	user 0.014s	sys 0.005s
I20260709 12:20:41.594996 14543 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:41.596611  6656 raft_consensus.cc:1240] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 26. Current term is 27. Ops: []
I20260709 12:20:41.601760 14757 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:41.604986 15086 consensus_meta.cc:287] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.958s	user 0.009s	sys 0.007s
I20260709 12:20:41.613989 14862 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:41.523408 15125 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:20:41.626611 15119 log.cc:927] Time spent T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
I20260709 12:20:41.520092 15000 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:41.636289 15122 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
W20260709 12:20:41.639941 15014 consensus_meta.cc:287] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 1.175s	user 0.007s	sys 0.004s
I20260709 12:20:41.641873 15125 raft_consensus.cc:697] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 26 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:20:41.653185  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:41.665697  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:41.715838  6509 raft_consensus.cc:2376] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 23. Re-sending same reply.
I20260709 12:20:41.441280 15106 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.053s
W20260709 12:20:41.676754 15118 log.cc:927] Time spent T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 12:20:41.532898 15011 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:41.735436 15064 consensus_queue.cc:1048] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.032s
I20260709 12:20:41.482494 15115 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:41.772240  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:41.763861 15056 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:41.750277 15125 consensus_queue.cc:237] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:41.781822 15115 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:41.717269 15014 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader election lost for term 27. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 27. Current term is 28.
W20260709 12:20:41.803903  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:41.806758 14968 consensus_meta.cc:287] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 1.178s	user 0.008s	sys 0.010s
W20260709 12:20:41.813796 14486 consensus_meta.cc:287] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 1.207s	user 0.013s	sys 0.009s
I20260709 12:20:41.814508 14486 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:41.846114  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:41.846392 14992 consensus_meta.cc:287] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 1.134s	user 0.009s	sys 0.008s
I20260709 12:20:41.847437 14992 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:41.849076  6505 raft_consensus.cc:2368] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20.
I20260709 12:20:41.852232 14543 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:41.855120 14963 consensus_meta.cc:287] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 1.212s	user 0.013s	sys 0.007s
I20260709 12:20:41.855777 14963 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 17. Current term is 18.
I20260709 12:20:41.867604  6582 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18.
W20260709 12:20:41.871867 15052 consensus_meta.cc:287] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.635s	user 0.013s	sys 0.000s
I20260709 12:20:41.887557 15124 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:20:41.892585 14680 consensus_meta.cc:287] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 1.014s	user 0.013s	sys 0.000s
W20260709 12:20:41.895015  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 157001us
I20260709 12:20:41.923592  6475 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:41.933719  6501 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.041s	sys 0.206s
I20260709 12:20:41.936503  6502 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 28.
W20260709 12:20:41.937086  6507 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.001s	sys 0.013s
W20260709 12:20:41.947140  6507 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.001s	sys 0.013s
W20260709 12:20:41.938436  6506 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.023s	sys 0.052s
W20260709 12:20:41.947882  6506 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.023s	sys 0.052s
W20260709 12:20:41.938719  7238 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.005s	sys 0.010s
W20260709 12:20:41.948673  7238 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.005s	sys 0.011s
W20260709 12:20:41.949466  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 639811us
W20260709 12:20:41.946403  6501 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.042s	sys 0.206s
I20260709 12:20:41.948815  6475 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:41.950657  6501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 3396) took 1169 ms. Trace:
I20260709 12:20:41.943497 15128 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:41.953929  6579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56116 (request call id 3553) took 1543 ms. Trace:
I20260709 12:20:41.957710  6475 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:41.951047  6501 rpcz_store.cc:276] 0709 12:20:40.781023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:40.781168 (+   145us) service_pool.cc:224] Handling call
0709 12:20:40.781509 (+   341us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:20:40.817768 (+ 36259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:40.817824 (+    56us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:20:40.818391 (+   567us) log.cc:844] Serialized 52 byte log entry
0709 12:20:41.138428 (+320037us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:20:41.364678 (+226250us) log.cc:844] Serialized 14 byte log entry
0709 12:20:41.950332 (+585654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:41.950410 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:41.950454 (+    44us) raft_consensus.cc:1685] finished
0709 12:20:41.950466 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:41.950599 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23790,"prepare.queue_time_us":502046,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":55,"raft.queue_time_us":81350,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":20768,"thread_start_us":1084125,"threads_started":3,"wal-append.queue_time_us":529228}
W20260709 12:20:42.004151  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:41.996186  6579 rpcz_store.cc:276] 0709 12:20:40.411364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:40.413627 (+  2263us) service_pool.cc:224] Handling call
0709 12:20:40.428371 (+ 14744us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:40.550125 (+121754us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:20:40.550455 (+   330us) log.cc:844] Serialized 14 byte log entry
0709 12:20:41.109367 (+558912us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:41.109429 (+    62us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:20:41.110632 (+  1203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:41.953557 (+842925us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b54001cc854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:41.953631 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:41.953833 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27647,"prepare.queue_time_us":737,"prepare.run_cpu_time_us":140,"prepare.run_wall_time_us":138,"raft.queue_time_us":52747,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":11331,"spinlock_wait_cycles":279387392,"thread_start_us":530818,"threads_started":2,"wal-append.queue_time_us":360733}
I20260709 12:20:41.946767 15006 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:41.945327 14992 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:42.121163  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 163523us
I20260709 12:20:42.122160  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:42.123642  6475 leader_election.cc:400] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 27. Current term is 28.
I20260709 12:20:42.124886  6475 leader_election.cc:403] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 election: Cancelling election due to peer responding with higher term
I20260709 12:20:42.036077 14680 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:42.112957 14543 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:42.084148 14902 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:42.128651  6475 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:42.149233  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 3479) took 1383 ms. Trace:
I20260709 12:20:42.149893  6656 rpcz_store.cc:276] 0709 12:20:40.765797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:40.769165 (+  3368us) service_pool.cc:224] Handling call
0709 12:20:40.769576 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:41.597195 (+827619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:42.148818 (+551623us) spinlock_profiling.cc:243] Waited 827 ms on lock 0x7b54003e8ad4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:42.149130 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1901323136}
I20260709 12:20:42.056638 15052 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:41.952711 15124 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:42.177902  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 49325.6us
I20260709 12:20:42.129164 15011 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:42.129896 15060 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:20:42.205029  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:42.154619 15115 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.211086  6651 raft_consensus.cc:2410] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 27 because replica has last-logged OpId of term: 25 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 16.
I20260709 12:20:42.129606 15126 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader election lost for term 27. Reason: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 27. Current term is 28.
I20260709 12:20:42.180114 14680 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:20:42.087005 15128 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:42.216531 15115 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:42.221211 15115 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:42.213390  6475 leader_election.cc:400] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 25. Current term is 26.
W20260709 12:20:42.224012  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:42.131623 14543 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:42.222654 15124 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:42.176983 14902 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:42.226015  6475 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:42.218704 15128 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:42.231293  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 171484us
I20260709 12:20:42.232024 15127 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:20:42.234076  6475 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:42.261086  6657 raft_consensus.cc:1275] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:20:42.231038 14486 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:42.266093  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:42.268474  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:42.270860  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:42.229622 15060 consensus_queue.cc:1059] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 12:20:42.275197  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:42.275915  6653 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:42.271471  6657 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 23.
I20260709 12:20:42.273150  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:42.278869 15129 log.cc:927] Time spent T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:20:42.279230 14992 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.279771 14992 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:42.282027 14992 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:42.302913 14543 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:42.304543 14902 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:42.306025  6507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 3615) took 1069 ms. Trace:
I20260709 12:20:42.311339 14956 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:42.311959 14956 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:20:42.315327 15130 log.cc:927] Time spent T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:20:42.306600  6507 rpcz_store.cc:276] 0709 12:20:41.236611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:41.326187 (+ 89576us) service_pool.cc:224] Handling call
0709 12:20:41.326581 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:41.415483 (+ 88902us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:41.415513 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:41.421397 (+  5884us) log.cc:844] Serialized 26 byte log entry
0709 12:20:41.947579 (+526182us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:20:42.058665 (+111086us) log.cc:844] Serialized 14 byte log entry
0709 12:20:42.059081 (+   416us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:42.305411 (+246330us) spinlock_profiling.cc:243] Waited 88.5 ms on lock 0x7b54001e91d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:42.305499 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:42.305576 (+    77us) raft_consensus.cc:1685] finished
0709 12:20:42.305729 (+   153us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:42.305943 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8657,"prepare.queue_time_us":110753,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":26733,"raft.run_cpu_time_us":275,"raft.run_wall_time_us":31923,"spinlock_wait_cycles":203626752,"thread_start_us":524754,"threads_started":1,"wal-append.queue_time_us":781206}
I20260709 12:20:42.272554 15006 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:20:42.322285  6653 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:20:42.324512  7238 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:42.286067 15126 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:42.328656  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:42.329392  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:42.333472 15115 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.328835 14543 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.334036 15002 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:42.338685 14543 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:42.340034 15135 consensus_queue.cc:1048] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:42.327571 14871 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:42.341120 14543 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:42.330634 15136 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:42.341240 14871 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:42.343163 15052 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:42.343508 14871 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:42.325760 15014 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.335631  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:42.331874 15126 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:42.348608 15126 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:42.323468 15123 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:42.349462 15123 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:42.351835 15123 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:42.337880  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:42.344251 15014 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:42.361959  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:42.338848 15002 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:42.334129 15011 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.363822  6501 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:20:42.364023 15011 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:42.365758 15014 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:42.366212 15011 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:42.366541 14902 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:42.366956 15121 consensus_queue.cc:1048] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:20:42.367117 14902 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:42.367748 15002 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:42.367885 14963 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.333986 15115 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:42.368521 14963 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:42.369526 14902 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:42.363332 15103 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:42.370088 14902 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:42.370527 14963 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:42.370301 15103 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:42.370669 14902 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:42.372288  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:42.372630  6505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 3604) took 1169 ms. Trace:
I20260709 12:20:42.373090  6505 rpcz_store.cc:276] 0709 12:20:41.203499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:41.240675 (+ 37176us) service_pool.cc:224] Handling call
0709 12:20:42.372406 (+1131731us) spinlock_profiling.cc:243] Waited 551 ms on lock 0x7b540039ec54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:42.372537 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1267152128}
I20260709 12:20:42.374358  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:42.374863 15103 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:42.376487  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:42.377591  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:42.378176  6584 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 26.
I20260709 12:20:42.380295  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:42.380692  6584 raft_consensus.cc:1275] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:20:42.381129  6575 raft_consensus.cc:2410] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 30 because replica has last-logged OpId of term: 28 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 14.
I20260709 12:20:42.381311  7238 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:42.383134  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:42.383857  6584 raft_consensus.cc:2368] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 27. Current term is 28.
I20260709 12:20:42.384843  6653 raft_consensus.cc:1275] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:20:42.384671  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:42.386969 15138 log.cc:927] Time spent T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 12:20:42.384536 14872 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:42.389309 14872 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:42.391793 14872 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:42.392570  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:42.392686 15057 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:42.384738 15136 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:42.394611 15137 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:42.396905  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:42.355918 14956 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:42.395161  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:42.395558  7238 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:42.396477  6584 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:20:42.393370 15057 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:42.402241  6583 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 23.
I20260709 12:20:42.397763  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:42.399344  6580 raft_consensus.cc:1275] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 12:20:42.391678  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:42.400673  6507 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:20:42.401000  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:42.403954  6575 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 23.
I20260709 12:20:42.404681  7273 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:20:42.408675  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:42.409313  6505 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 25.
I20260709 12:20:42.403522  6578 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 25.
I20260709 12:20:42.406044 15057 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:42.401671  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:42.410396  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:42.410756  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:42.401643  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:42.411196  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:42.416654  6577 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 26.
I20260709 12:20:42.410472  6576 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:20:42.383625 15115 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:42.421339  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:42.424005  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:42.421648  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:42.422711  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:42.425117  6581 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 23.
W20260709 12:20:42.426257  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:42.427631  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:42.428231  6505 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:42.428967  6501 raft_consensus.cc:1275] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:20:42.430792  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:42.403861 14992 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:42.432118  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:42.426155  6577 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:20:42.433085  6576 raft_consensus.cc:1275] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:20:42.433570  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:42.436144  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:42.436815  6576 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 24.
I20260709 12:20:42.443753  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:42.446794 14956 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 31 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:42.449746 14926 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:42.451172 14708 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:42.451251 14992 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:42.454039 14992 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:42.426703 15136 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:42.455255  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:42.460788 14985 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:42.462774 14813 raft_consensus.cc:3055] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:42.463034  6505 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 28.
I20260709 12:20:42.463408 14813 raft_consensus.cc:740] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:42.451772  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:42.464587  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 44198.4us
I20260709 12:20:42.451377  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:42.434500  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:42.473255  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:42.474236 14813 consensus_queue.cc:260] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:42.469704 15136 raft_consensus.cc:697] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 23 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:42.477733  6502 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 3609) took 1272 ms. Trace:
I20260709 12:20:42.478376 15136 consensus_queue.cc:237] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:42.479871  6504 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:20:42.468221 14910 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.481189  6504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 3620) took 1187 ms. Trace:
I20260709 12:20:42.481271 14910 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:42.481581  6504 rpcz_store.cc:276] 0709 12:20:41.293932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:41.331061 (+ 37129us) service_pool.cc:224] Handling call
0709 12:20:42.479571 (+1148510us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:42.480223 (+   652us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:42.480990 (+   767us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54002c38d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:42.481135 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2640569600}
I20260709 12:20:42.482915  6652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 4031) took 1752 ms. Trace:
I20260709 12:20:42.483268  6652 rpcz_store.cc:276] 0709 12:20:40.730530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:40.732699 (+  2169us) service_pool.cc:224] Handling call
0709 12:20:40.733098 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:41.385091 (+651993us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:42.482705 (+1097614us) spinlock_profiling.cc:243] Waited 645 ms on lock 0x7b54003e9ed4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:42.482851 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1483794176}
I20260709 12:20:42.484541  6509 raft_consensus.cc:1275] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:20:42.478083  6502 rpcz_store.cc:276] 0709 12:20:41.204879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:41.283264 (+ 78385us) service_pool.cc:224] Handling call
0709 12:20:42.477547 (+1194283us) spinlock_profiling.cc:243] Waited 522 ms on lock 0x7b54002c38d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:42.477675 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1201745664}
I20260709 12:20:42.493599  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:42.495657  6506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 3607) took 1291 ms. Trace:
I20260709 12:20:42.492980  6475 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:42.498124  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:42.496121  6506 rpcz_store.cc:276] 0709 12:20:41.204394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:41.311589 (+107195us) service_pool.cc:224] Handling call
0709 12:20:41.312375 (+   786us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:41.394420 (+ 82045us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:41.394471 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:41.395121 (+   650us) log.cc:844] Serialized 26 byte log entry
0709 12:20:41.948280 (+553159us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:41.948428 (+   148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:42.495191 (+546763us) spinlock_profiling.cc:243] Waited 74.2 ms on lock 0x7b540001bdd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:42.495254 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:42.495308 (+    54us) raft_consensus.cc:1685] finished
0709 12:20:42.495326 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:42.495587 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7383,"raft.queue_time_us":26409,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":4122,"spinlock_wait_cycles":170627456,"thread_start_us":522943,"threads_started":1,"wal-append.queue_time_us":794538}
I20260709 12:20:42.500031 15060 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:42.502084 15123 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:20:42.506301  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:42.506340  6548 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:42.508159  6548 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:42.508648  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:42.508270  6475 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:42.510546  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:42.511330  6506 raft_consensus.cc:2368] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 27. Current term is 28.
I20260709 12:20:42.511873  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:42.512495  7265 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 27.
I20260709 12:20:42.492903 14902 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:42.536434  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:42.483518 14910 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:42.518143  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:42.518214  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:42.537761  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:42.538130  6654 raft_consensus.cc:3060] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:42.519040  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:42.519778 14988 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:42.520232 14963 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.541039 14963 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:42.542557 15060 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:42.543454 14963 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:42.519794  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:42.520756 14985 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:42.520973 15008 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:42.545658 14956 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:20:42.545325 15008 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:42.547365 14901 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:42.521598  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:42.547900 14901 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:42.548774  7265 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 24.
I20260709 12:20:42.522606  6475 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:42.550591 15008 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:42.532260  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:42.552248  6475 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:42.552325 15060 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:42.534379  6501 raft_consensus.cc:1275] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 12:20:42.553896  6475 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:42.550961 14486 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:42.554750 14486 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:42.520576 15056 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:42.555658 15056 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:42.556835 14486 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:42.556998  6505 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 16. (term mismatch)
I20260709 12:20:42.550680 14901 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:42.552807 15008 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:42.557714 15056 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:42.533638  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:42.557864 15008 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:42.553068 15022 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:42.559113  6500 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41304 (request call id 1141) took 1173 ms. Trace:
I20260709 12:20:42.559942  6548 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:42.551374 14985 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:42.569026 14988 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:42.569904 15008 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:42.571031 14813 raft_consensus.cc:3060] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:42.559520  6500 rpcz_store.cc:276] 0709 12:20:41.385367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:41.385635 (+   268us) service_pool.cc:224] Handling call
0709 12:20:41.385929 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:41.415858 (+ 29929us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:41.415909 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:41.415954 (+    45us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:41.415989 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:42.558878 (+1142889us) spinlock_profiling.cc:243] Waited 29.7 ms on lock 0x7b5400434c54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:42.558938 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:42.559057 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":68328832}
I20260709 12:20:42.557430  6505 pending_rounds.cc:85] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 15
I20260709 12:20:42.574182  6505 pending_rounds.cc:107] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 26.16
I20260709 12:20:42.574606  6505 raft_consensus.cc:2889] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:42.562901  6475 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:42.584625  6548 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:42.588670  6475 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:42.593919  6548 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:42.569181 14858 raft_consensus.cc:2764] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:20:42.596103  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:42.598009  6548 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:42.607966  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:42.609095  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 143547us
I20260709 12:20:42.610495  6548 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:42.588783 15008 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:20:42.614950  6548 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:42.592556 14910 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:42.621281  6652 raft_consensus.cc:1275] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 27 index: 14. Preceding OpId from leader: term: 29 index: 15. (index mismatch)
I20260709 12:20:42.594570 14988 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:20:42.594413 15022 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:42.629932 15022 raft_consensus.cc:697] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:42.631271 15022 consensus_queue.cc:237] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:42.629810 15059 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:42.603518 15024 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:42.627864  6654 raft_consensus.cc:2468] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 28.
I20260709 12:20:42.642155  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:42.615896 15056 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:42.616103 14901 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:42.596124 14985 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.042s
I20260709 12:20:42.649017  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:42.652776  6501 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 25.
I20260709 12:20:42.654770  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:42.655479  6576 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 24.
I20260709 12:20:42.656435  7238 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:42.657120  6500 raft_consensus.cc:2393] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 26: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:42.636600 14486 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:42.642913 15024 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:42.665966  6548 leader_election.cc:400] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20.
I20260709 12:20:42.666761  6548 leader_election.cc:403] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
W20260709 12:20:42.668701  6582 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 3219) took 1904 ms (client timeout 2141 ms). Trace:
W20260709 12:20:42.669119  6582 rpcz_store.cc:269] 0709 12:20:40.763924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:40.764073 (+   149us) service_pool.cc:224] Handling call
0709 12:20:42.668546 (+1904473us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54001d10d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:42.668647 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2421126528}
I20260709 12:20:42.670661  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:42.671077  6508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45874 (request call id 3600) took 1479 ms. Trace:
I20260709 12:20:42.671468  6508 rpcz_store.cc:276] 0709 12:20:41.191200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:41.191339 (+   139us) service_pool.cc:224] Handling call
0709 12:20:42.670904 (+1479565us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b540039fb54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:42.671025 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":242310912}
I20260709 12:20:42.671996  6576 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:20:42.672145  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:42.672231  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:42.674088  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:42.675179  6582 raft_consensus.cc:2410] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 for term 28 because replica has last-logged OpId of term: 26 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 17.
I20260709 12:20:42.677834  6548 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:42.686159 14963 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.686831 14963 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:42.691075  6548 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:42.692662 14486 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:42.693203 14486 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:42.693703 14486 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:42.696655 15144 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:20:42.698818 15144 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:42.655687 15008 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:42.662106 14902 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:42.669229 15024 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:42.709596  6548 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:42.673403 15137 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:42.700193 14988 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:42.711030 15008 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:42.727003 15008 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:42.718281 15142 raft_consensus.cc:2804] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:20:42.712234 14902 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:42.753150 14963 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:42.759193 14988 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20.
I20260709 12:20:41.262202  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:20:42.819057  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:42.821640  6406 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46184 (request call id 117) took 132568 ms (client timeout 15000 ms). Trace:
W20260709 12:20:42.822252  6406 rpcz_store.cc:269] 0709 12:18:30.253617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:18:30.253892 (+   275us) service_pool.cc:224] Handling call
0709 12:20:42.821320 (+132567428us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":116577,"reported_tablets":24,"spinlock_wait_cycles":2304}
I20260709 12:20:42.829563  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:42.832360  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:42.836773  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 123264us
W20260709 12:20:42.839223  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:42.839998  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:42.841086  6575 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 25.
I20260709 12:20:42.840320  6403 catalog_manager.cc:5697] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:20:42.842651  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:42.843106  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:42.846562  6474 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:42.849078  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:42.850005  6580 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 23.
I20260709 12:20:42.851042  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:42.851735  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:42.852208  6502 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:20:42.853109  6581 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 18. (term mismatch)
I20260709 12:20:42.853434  6581 pending_rounds.cc:85] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 17
I20260709 12:20:42.848069  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:42.853726  6581 pending_rounds.cc:107] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 26.18
I20260709 12:20:42.854070  6581 raft_consensus.cc:2889] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:42.846784  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:42.851759  6580 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 27.
I20260709 12:20:42.858503  6583 raft_consensus.cc:1240] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 25. Current term is 26. Ops: [22.13-25.15]
I20260709 12:20:42.859752  6581 raft_consensus.cc:1275] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 28 index: 15. Preceding OpId from leader: term: 29 index: 15. (term mismatch)
I20260709 12:20:42.860183  6581 pending_rounds.cc:85] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 14
I20260709 12:20:42.860671  6581 pending_rounds.cc:107] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 28.15
I20260709 12:20:42.861043  6581 raft_consensus.cc:2889] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:42.861394  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:42.905596  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 56144.6us
I20260709 12:20:42.925765  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:42.927503 14910 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.928351 14910 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:42.964399 15006 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:42.968426 15056 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:42.969440 15142 raft_consensus.cc:697] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 28 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:42.975113 14926 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:42.980787 14902 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:42.934211 14963 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:20:42.988062  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 139526us
I20260709 12:20:43.002637 15116 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:43.003716 15116 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:43.006129  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 159978us
I20260709 12:20:43.016515 14543 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:43.016812  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:43.017236 14543 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:43.027400 15056 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:43.015911 15142 consensus_queue.cc:237] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:43.034961 15006 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:20:43.008051  6474 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:43.040627 14926 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:43.056368  7273 raft_consensus.cc:1275] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:20:43.058696 15134 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:20:43.060820  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 77500us
I20260709 12:20:43.079031 14910 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:43.082962  6578 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20.
I20260709 12:20:43.049014 15116 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:43.089560  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 81704.8us
I20260709 12:20:43.104494 14910 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:43.105791 14543 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:43.107596  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:43.110850  6474 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:43.113930 14858 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:43.134600 15024 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:43.136794  6548 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:43.136773 14813 consensus_meta.cc:287] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.565s	user 0.007s	sys 0.016s
I20260709 12:20:43.156646 15056 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:43.162889  6504 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:20:43.162757  6474 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:43.173463  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 53385.6us
I20260709 12:20:43.178910  7265 raft_consensus.cc:1275] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:20:43.180567 14910 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:43.139711 14926 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:43.185034  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 101445us
I20260709 12:20:43.185380 14901 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:20:43.186360  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 174138us
I20260709 12:20:43.189155  6550 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:43.196493 14543 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:43.196707 15116 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.208555  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:43.210605  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:43.213254  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:43.216866  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:43.226306 15024 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:43.297210 15024 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.297564 15116 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:43.297828 15024 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:43.298520  6579 raft_consensus.cc:1275] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:20:43.299823 15024 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:43.199574  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:43.295258 14543 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:43.307482  6582 raft_consensus.cc:1275] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:20:43.308833  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:43.310552  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:43.311300  6505 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 23.
I20260709 12:20:43.312599  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:43.314926  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:43.315587  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:43.316188  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:43.316844  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:43.317034  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:43.320371 14543 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:43.322206  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:43.325821  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:43.326022  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:43.326560  6579 raft_consensus.cc:2376] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 18. Re-sending same reply.
I20260709 12:20:43.327508  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:43.328325  6501 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 25.
I20260709 12:20:43.330493  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:43.333021 14998 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:43.299898 15116 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:43.333253  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 116680us
I20260709 12:20:43.333467 14998 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:43.333945 14998 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:43.335440  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:43.336411  6577 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 31: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:43.336118 15064 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:20:43.338474  7238 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:43.340412 15136 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:43.340924 15024 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:20:43.341430  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 152474us
I20260709 12:20:43.343269  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:43.344200 14998 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:43.344873 15009 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:20:43.346858 14998 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:43.346963 15136 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 12:20:43.347673 14486 consensus_meta.cc:287] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.653s	user 0.004s	sys 0.017s
I20260709 12:20:43.353039  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:43.356652  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 157338us
W20260709 12:20:43.357681  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:43.384914  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:43.348119 14486 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:43.366215 15116 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:43.381880 14543 consensus_queue.cc:1048] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 12:20:43.389747  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:43.391691 15134 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:43.404297  6575 raft_consensus.cc:2393] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 26: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:43.395773 15116 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:43.408703  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:43.428895  7265 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:43.413477 15116 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:43.426862  6622 leader_election.cc:400] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 27. Current term is 28.
I20260709 12:20:43.433657  6622 leader_election.cc:403] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 election: Cancelling election due to peer responding with higher term
I20260709 12:20:43.437407  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:43.456297  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:43.409122 15134 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:43.461336  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:43.459254  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:43.464354  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:43.461858  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 65063.9us
I20260709 12:20:43.475872 14871 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader election lost for term 27. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 27. Current term is 28.
I20260709 12:20:43.478986  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:43.481043  6657 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:47858
I20260709 12:20:43.481806  6657 raft_consensus.cc:606] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:43.482236  6657 raft_consensus.cc:614] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:43.477339  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:43.493860 15134 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:43.497480  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:43.498281  6506 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:43.498555 15125 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:43.498978  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:43.505252 14926 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.506100  6656 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:20:43.506000 14926 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:43.507608 14486 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:43.508638  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:43.505328  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:43.515499 14708 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:43.522701  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:43.511938  6549 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:43.527149  6653 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "83301542a88f46cea64bc905b3199975"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:47858
I20260709 12:20:43.527801  6653 raft_consensus.cc:606] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 24 LEADER]: Received request to transfer leadership to TS 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:43.521883 14926 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:43.543111 15024 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:43.541738  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:43.540933  6474 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:43.531107 14486 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:43.595161  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 81600.5us
I20260709 12:20:43.599764  6653 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:47858
I20260709 12:20:43.602237  6653 raft_consensus.cc:606] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:43.602636  6653 raft_consensus.cc:614] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:43.590610 15006 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:43.606671  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:43.606760 14708 consensus_queue.cc:1059] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.004s
I20260709 12:20:43.638000  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:20:43.644680  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 267623us
I20260709 12:20:43.645542  6549 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:43.652550  6656 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:43.657250 14910 raft_consensus.cc:3055] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:43.657692 14910 raft_consensus.cc:740] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:43.659087  6575 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:20:43.659679  6575 raft_consensus.cc:606] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:43.660058  6575 raft_consensus.cc:614] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:20:43.661794 15011 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.009s	sys 0.012s
W20260709 12:20:43.662235 15011 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.009s	sys 0.012s
I20260709 12:20:43.658753 14910 consensus_queue.cc:260] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:43.665658  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:43.668129  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:20:43.669831  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:43.670836  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:43.599748 15024 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:43.680652  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 183644us
I20260709 12:20:43.634490 14486 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:43.702338  6549 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:43.715878 14486 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:43.716548 14486 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:43.717476 15142 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:43.702279 14968 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:20:43.724937 15158 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:20:43.733000 15170 log.cc:927] Time spent T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:20:43.727444  6622 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:43.772375  6657 raft_consensus.cc:1275] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:20:43.726296 14968 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:43.725431 15158 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:20:43.784785  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 80315.6us
W20260709 12:20:43.801414  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 202060us
I20260709 12:20:43.778496 14968 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:43.780609 15158 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:20:43.819077  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 119743us
I20260709 12:20:43.820079  6549 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:43.814877  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 222189us
W20260709 12:20:43.845889 15173 log.cc:927] Time spent T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 12:20:43.847373  6549 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:43.851868  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:43.843981  6475 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:43.853137  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:43.860003 15177 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:43.860535 15177 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:43.860908 15177 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:20:43.863471  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 45664.3us
W20260709 12:20:43.865195  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 45503.6us
I20260709 12:20:43.841493 14901 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:43.862298 14858 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:43.841881 15142 consensus_queue.cc:1048] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.025s
I20260709 12:20:43.877003  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:43.884507 15155 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:43.885906  6549 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:43.891697 15176 log.cc:927] Time spent T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:20:43.901458  6506 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 27.
I20260709 12:20:43.899529 14543 consensus_queue.cc:1048] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 12:20:43.907488 15144 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:20:43.886983 15024 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:43.911748  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 46399.2us
I20260709 12:20:43.900620 14858 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:43.921480  6657 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:20:43.926162  6656 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 27.
W20260709 12:20:43.929659  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 45486.5us
I20260709 12:20:43.929811 15174 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:43.930768  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:43.930666 15174 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:43.944985 15171 log.cc:927] Time spent T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 12:20:43.944706 14858 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:43.948205  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:43.950276  6579 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 25. Current term is 26. Ops: [25.19-25.19]
I20260709 12:20:43.950585 14910 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:43.953807 15116 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:43.956182 15116 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:43.957211  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:43.943908  6548 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:43.960709  6658 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 26. Current term is 27. Ops: []
I20260709 12:20:43.965170  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:43.978333 15157 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:43.978955 15157 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:43.979354 15157 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:43.981271  6548 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:44.010982 15174 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:44.012583  6575 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:20:43.952045 15158 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:44.015384 14708 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:43.952181 14968 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:44.032974  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 102308us
I20260709 12:20:44.016499 14486 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:20:44.039012  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:44.038390  6550 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:44.018515 14910 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 12:20:44.020257 14708 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:44.033587 14543 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:20:44.063606  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 81039.1us
W20260709 12:20:44.091624  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 53358.4us
I20260709 12:20:44.053941  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:44.053684  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:44.052556  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:44.109664  6548 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:20:44.094797  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 56124.4us
I20260709 12:20:44.051555  6473 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:44.043541 15196 log.cc:927] Time spent T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:20:44.153934  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:44.145656  6507 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 26.
I20260709 12:20:44.054355 15116 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:20:44.093969 15199 log.cc:927] Time spent T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
W20260709 12:20:44.043241 15194 log.cc:927] Time spent T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8: Append to log took a long time: real 0.081s	user 0.002s	sys 0.006s
I20260709 12:20:44.045439 15192 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:44.162231  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:44.057411 14708 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:44.094445 14813 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:44.059855 14543 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:44.174237 15192 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:44.164007  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:44.154096 15137 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:44.056170 14858 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:44.167759  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:44.168220 15189 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:44.172431 14871 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:44.148243 15024 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:20:44.167073 15116 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:20:44.108469 15177 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:44.266731  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 171700us
I20260709 12:20:44.266304 14813 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:44.283923  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 192491us
I20260709 12:20:44.225986 14543 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:44.185240 14968 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:44.041294 15195 consensus_queue.cc:1059] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.165s
I20260709 12:20:44.332075 15134 raft_consensus.cc:2764] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader election decision vote started in defunct term 26: won
I20260709 12:20:44.176863 14901 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:44.177067 15011 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:44.295315 15008 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:44.050384 15158 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:44.346323  6548 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:44.347188 15142 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:44.340895 15056 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:44.336349 15192 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:44.365694  6506 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 25.
W20260709 12:20:44.370229  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:44.362366 15024 raft_consensus.cc:697] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:44.344064 14871 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:44.378356  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 50674.7us
I20260709 12:20:44.378386  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:44.380488  6579 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 12:20:44.385087  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 333614us
I20260709 12:20:44.386363  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:44.386613  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:44.388469  6576 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 31 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 30. Current term is 31. Ops: []
I20260709 12:20:44.366447 15185 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:44.351811 15189 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:44.373389 15180 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:44.399463 15179 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:44.399974 15179 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:44.399688 15024 consensus_queue.cc:237] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:44.392055 15142 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:44.400717  6651 raft_consensus.cc:1240] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:44.400961  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:44.401537 15067 consensus_queue.cc:1059] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.058s
W20260709 12:20:44.410640  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:44.349164 15137 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:44.463045  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:44.467434  6549 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:44.466159  6578 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 26.
I20260709 12:20:44.393658 14901 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:44.464783  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:20:44.472993  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 126756us
I20260709 12:20:44.376700 15011 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:44.465065  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:44.471012  6581 raft_consensus.cc:2376] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 20. Re-sending same reply.
I20260709 12:20:44.398303 15158 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:44.398253 14998 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:44.460997 15179 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:44.480180  6548 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:44.482687  7273 raft_consensus.cc:2393] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 25: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:20:44.474058  6622 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:44.496726 15125 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:44.497319 15125 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:44.497510 14757 raft_consensus.cc:3055] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:44.497817 15125 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:44.498057 14757 raft_consensus.cc:740] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:44.497601 14858 raft_consensus.cc:3055] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:20:44.499864 14858 raft_consensus.cc:740] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:44.500830 14858 consensus_queue.cc:260] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:44.501255 15208 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:44.502538 14858 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:44.506466 15207 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:44.507097 15207 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:44.509244 15191 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:44.509862 15191 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:44.511389 14871 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:44.512864 15116 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:44.515027 15011 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:44.515692 15202 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:44.516131 14813 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:44.403614 15056 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:44.526012  6473 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:44.403440 15008 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:44.531294  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 64025.9us
I20260709 12:20:44.463872 15180 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:44.516162 15202 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:44.486418 15144 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:44.536422 14901 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:44.542234 15180 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:44.542996  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:44.487588 14998 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:44.544857  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:44.542156 15177 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:44.548228 15056 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:44.548465 15195 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:44.552635  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 154180us
W20260709 12:20:44.552922  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 79006.1us
I20260709 12:20:44.486624 15158 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:44.528807 14968 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:44.464847 15185 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:44.614130  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:44.616667  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:44.557416  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:44.557056  6578 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 18.
W20260709 12:20:44.546069 15204 log.cc:927] Time spent T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:20:44.623963 15207 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:44.625525 15207 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:44.626039 15056 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:44.626132 15207 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:44.626957 15056 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:44.629132 15189 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:44.629597 15056 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:44.630270 15056 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:44.630352  6473 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:44.531046 14708 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:44.630936 15056 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:44.631449 14708 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:44.535257 14858 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:44.632629 14858 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:44.633077 14858 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:44.633782 14708 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:44.550663 15202 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:44.613729 14968 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:44.636844 14968 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:44.638938  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:44.641304 15158 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:44.630537 14998 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:44.641309  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:44.647680 15185 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:44.650547 15056 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:44.557052 15144 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:44.562479 14901 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:44.540128 14543 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:44.660177 14901 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:44.655151 14871 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.171s
W20260709 12:20:44.660686  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 94305.2us
I20260709 12:20:44.828022  6548 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:44.830195 14486 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:44.832657  6548 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:44.833670 15155 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:44.834177  6548 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:44.835187 15155 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:44.661235 15189 consensus_queue.cc:1048] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.175s
I20260709 12:20:44.661451 15195 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:44.669694  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:44.682829 14858 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:44.685590 15179 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:44.844172 14858 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:44.844524 15179 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:44.840037 15195 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:44.701711 15191 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:44.717398  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:44.853904  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:44.855316  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:44.718303 14757 consensus_queue.cc:260] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:44.857549 14757 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:44.858893  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:44.733541 15142 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:44.736763  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:44.863086  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:44.868471  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:44.863188  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:44.738394 15206 log.cc:927] Time spent T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.268s	user 0.003s	sys 0.000s
I20260709 12:20:44.748172  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:44.878578  6650 raft_consensus.cc:2393] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 24: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:44.653775 15008 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:44.657753 15192 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:44.882880 15192 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:44.655795  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:44.884886  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 229180us
I20260709 12:20:44.887460 14871 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:44.888085 14871 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:44.888581 14871 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:44.685216 14968 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:44.897310 14968 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:44.657991 15144 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:44.898928  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:44.900552  6656 raft_consensus.cc:2393] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 26: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:44.904933  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:44.698184 15116 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:44.910020 15116 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:44.705348  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:44.690515 14543 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:44.915146 15116 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:44.717767  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:44.720225  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:44.710666 15207 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:44.927451 15177 consensus_queue.cc:1048] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:44.929769 15177 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:20:44.939743  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 45730.9us
I20260709 12:20:44.941303 15134 consensus_queue.cc:1048] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:20:44.718099 15157 consensus_meta.cc:287] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.738s	user 0.005s	sys 0.008s
W20260709 12:20:44.956904  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 122823us
I20260709 12:20:44.958103  6548 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:44.961050 15207 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:44.964655  6548 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:44.652302 15125 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:44.970857  6622 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:44.981926 15137 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:44.985294 15024 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.040s	sys 0.105s
I20260709 12:20:44.992857  6549 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:45.007433  7238 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.016s	sys 0.067s
W20260709 12:20:45.031162  7238 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.016s	sys 0.067s
I20260709 12:20:44.987943  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:45.034001  6549 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:45.035034  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 292618us
W20260709 12:20:45.036818  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 75436.2us
W20260709 12:20:45.011852  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 445680us
W20260709 12:20:45.044489  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 567403us
I20260709 12:20:44.951428 15157 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:45.059648 14901 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:20:45.030458 15024 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.040s	sys 0.105s
I20260709 12:20:45.062345  6654 raft_consensus.cc:1240] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 23. Current term is 24. Ops: []
W20260709 12:20:45.056142 15217 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:20:45.056933  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:45.069602  6575 raft_consensus.cc:2393] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 24: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:20:45.071396  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 64450.2us
I20260709 12:20:45.065014 14901 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:20:45.073330 15185 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:45.073954 15185 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:45.065434 15056 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:20:45.077548  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 447272us
I20260709 12:20:45.079644  6473 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:45.058110 15116 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:45.076059 15185 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:45.089677 15180 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:20:45.095052 15216 consensus_queue.cc:1059] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:45.045951 14871 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:45.096921  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:45.097262  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:45.099907  6549 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:45.100870  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 44911.5us
I20260709 12:20:45.065974 15209 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:45.106189  6582 raft_consensus.cc:1240] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 25. Current term is 26. Ops: [25.19-25.19]
I20260709 12:20:45.106938  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:45.069866 15195 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:45.111402  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:45.112006  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 141271us
I20260709 12:20:45.097491 14926 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:45.115337  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:45.116137  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:45.111591  6548 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:45.119272  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:45.121822  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:45.122000  6548 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:45.124965  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:45.110183 15179 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:45.125696  6508 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 21.
I20260709 12:20:45.109163 14708 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:45.125092 15207 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:20:45.129482 15155 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:20:45.166805  6403 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46184 (request call id 121) took 60592 ms (client timeout 15000 ms). Trace:
I20260709 12:20:45.118168 14998 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.037s
I20260709 12:20:45.112491 15202 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:45.165596  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 86083.6us
I20260709 12:20:45.141542 14901 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:20:45.121228 15209 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:45.164350 15155 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:45.149008 15179 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:45.166580 15208 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:45.183351  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:45.178931 14968 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:45.187283  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 79973.7us
I20260709 12:20:45.159442 14708 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:45.187211  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 78066.4us
W20260709 12:20:45.216583  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 91503.8us
I20260709 12:20:45.181645 15125 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:45.189394 15155 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:45.188783 15207 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:45.220552  7238 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:45.227310  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:45.229993  6505 raft_consensus.cc:1275] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:20:45.231904  6474 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:45.232275  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:45.234792  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:45.235803  6506 raft_consensus.cc:2393] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 25: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:20:45.280680  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:45.282887 15209 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:45.283979  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:45.284456 15137 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:20:45.284768  6505 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 23.
I20260709 12:20:45.284962 15137 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:45.285106 15006 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:20:45.232923 15202 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:45.285460 15137 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:45.286247  6473 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:45.287611  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:45.289825  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:45.290529  6507 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 26.
I20260709 12:20:45.287891  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:45.291487  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:45.292419  6501 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 26: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:45.292727  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:45.293622 14871 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:45.237704 14757 consensus_queue.cc:1059] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.057s
I20260709 12:20:45.295152  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:45.296728  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:45.297148  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 79080.7us
I20260709 12:20:45.297603  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:45.289856  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:45.298533  7238 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:45.303545  6472 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:45.167327  6403 rpcz_store.cc:269] 0709 12:19:44.574126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:44.587335 (+ 13209us) service_pool.cc:224] Handling call
0709 12:19:44.755246 (+167911us) spinlock_profiling.cc:243] Waited 20.1 ms on lock 0x7b5400002210. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d650b2 00007f287bcde759 00007f287bc76a76 00007f287bc47d58 00007f287bd4604e 00007f287a2072a4 00007f287a207232 00007f287a207161 00007f287a2070dc
0709 12:20:45.166715 (+60411469us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:43.690048 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:43.848506 (+158458us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:43.848559 (+    53us) write_op.cc:435] Acquired schema lock
0709 12:20:43.848591 (+    32us) tablet.cc:662] Decoding operations
0709 12:20:43.849116 (+   525us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:43.849198 (+    82us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:43.854866 (+  5668us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:20:43.897507 (+ 42641us) tablet.cc:701] Row locks acquired
0709 12:20:43.897572 (+    65us) write_op.cc:260] PREPARE: finished
0709 12:20:43.897800 (+   228us) write_op.cc:270] Start()
0709 12:20:43.898015 (+   215us) write_op.cc:276] Timestamp: P: 1783599643897722 usec, L: 0
0709 12:20:43.898067 (+    52us) op_driver.cc:348] REPLICATION: starting
0709 12:20:43.898972 (+   905us) log.cc:844] Serialized 527 byte log entry
0709 12:20:44.399725 (+500753us) op_driver.cc:464] REPLICATION: finished
0709 12:20:44.543533 (+143808us) write_op.cc:301] APPLY: starting
0709 12:20:44.543672 (+   139us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:44.545784 (+  2112us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:44.545916 (+   132us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:44.691411 (+145495us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:44.691831 (+   420us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:20:44.692042 (+   211us) write_op.cc:312] APPLY: finished
0709 12:20:44.692431 (+   389us) log.cc:844] Serialized 25 byte log entry
0709 12:20:44.692819 (+   388us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:44.692916 (+    97us) write_op.cc:454] Released schema lock
0709 12:20:44.692980 (+    64us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:44.693493 (+   513us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:44.693529 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1830,"reported_tablets":24,"spinlock_wait_cycles":46229504,"child_traces":[["op",{"apply.queue_time_us":143403,"apply.run_cpu_time_us":5495,"apply.run_wall_time_us":149980,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8299,"num_ops":1,"prepare.queue_time_us":580560,"prepare.run_cpu_time_us":21076,"prepare.run_wall_time_us":289303,"raft.queue_time_us":65543,"raft.run_cpu_time_us":39772,"raft.run_wall_time_us":131674,"replication_time_us":501473,"thread_start_us":648582,"threads_started":4,"wal-append.queue_time_us":32790}]]}
I20260709 12:20:45.305527  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:45.306525  6501 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:45.223392 15218 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:45.317458  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:45.218879 14968 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:45.320901 15006 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:20:45.233739 15157 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:45.323108  6548 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:45.330356  6548 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:45.330374  6506 raft_consensus.cc:2410] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 25 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 12:20:45.333679  6548 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:45.334764 15215 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:45.335314 15158 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:20:45.335976 15158 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:45.336548 15158 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:45.308422 14543 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:20:45.317785 15220 log.cc:927] Time spent T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:20:45.341813  6548 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:45.233486 15208 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:45.349784  6548 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:45.332700 15006 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:45.243885 15207 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:45.332857 15172 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:45.353448  6548 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:45.339517 15008 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:20:45.355875  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:45.338403 15209 consensus_queue.cc:1048] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.019s
W20260709 12:20:45.387707  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 150045us
W20260709 12:20:45.388156  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 89695us
I20260709 12:20:45.388942  6548 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:45.389318  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 100174us
I20260709 12:20:45.348223  6402 catalog_manager.cc:5697] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 20 to 23, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 23 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:45.392243  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:45.352164 15142 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:45.330924 15218 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:45.397248  6548 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:45.358247 14486 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:45.407238  6548 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:45.397763  6402 catalog_manager.cc:5697] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 16 to 19, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 19 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:20:45.355297 15157 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:45.401824 15219 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:45.403793 15142 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:45.414929 15142 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:20:45.415352 15230 log.cc:927] Time spent T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:20:45.417071 15121 raft_consensus.cc:3055] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:45.417621 15121 raft_consensus.cc:740] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:45.390976 15144 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:20:45.422772  6548 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:45.411024  6402 catalog_manager.cc:5697] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 18 to 21. New cstate: current_term: 21 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:45.394516 14871 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:20:45.418674 15121 consensus_queue.cc:260] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:45.410029 15223 log.cc:927] Time spent T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.231s	user 0.002s	sys 0.000s
I20260709 12:20:45.354077 15006 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:20:45.424388  7238 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.1:36643 timed out after 1.593s (SENT) [suppressed 2 similar messages]
I20260709 12:20:45.409407 15008 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
W20260709 12:20:45.441114  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 98239.5us
I20260709 12:20:45.427721  6548 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:45.446934  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 46783.3us
I20260709 12:20:45.424474 15219 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:45.514580  6622 leader_election.cc:400] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 27. Current term is 28.
I20260709 12:20:45.480823  6402 catalog_manager.cc:5697] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 14 to 24, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 24 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
W20260709 12:20:45.518312  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 90727.8us
I20260709 12:20:45.522661  6402 catalog_manager.cc:5697] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 23 to 25, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 25 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:45.531189 15218 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:45.532455 14871 consensus_queue.cc:1048] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:45.535415  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:45.531082  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:45.539597 14543 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:20:45.540519 14543 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:45.422410 14486 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:45.546402 14486 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:45.525933 15235 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:45.548020 15235 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:45.549180 14543 consensus_queue.cc:1048] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:45.550776 15235 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:45.552067  6622 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:45.553319 14543 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:45.559799  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:45.562896 15212 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:45.562893  6473 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:45.441346 15185 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:45.565872 15212 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:45.569408 15207 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:45.569873 14543 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:45.571177 14968 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:45.508446 15179 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:45.526089 14708 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:45.510270 15180 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:45.574386 15180 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:45.575850  6473 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:45.578596 15208 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:45.579473  6652 raft_consensus.cc:1240] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:45.588901  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:45.590505 15195 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:45.590629  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:45.591284 15195 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:45.582978 14871 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:45.599218  6402 catalog_manager.cc:5697] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:45.600214 15121 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:45.601349  6402 catalog_manager.cc:5697] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 23 to 26, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 26 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } }
W20260709 12:20:45.604264  7239 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:45.584273  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:45.602908 15212 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:45.607038 15180 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:45.608532  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:45.609997 15195 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:45.611922  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:45.614701 15137 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:45.675240  7265 raft_consensus.cc:1240] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 18. Current term is 19. Ops: []
I20260709 12:20:45.679132 15226 consensus_queue.cc:1059] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:20:45.680688 15224 raft_consensus.cc:3055] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:45.681097 15224 raft_consensus.cc:740] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:45.682379  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:45.621078 14543 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:20:45.683149  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:45.683890  6656 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:45.624274 15179 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:45.617974 15142 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:45.684800  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:45.685686  6657 raft_consensus.cc:2393] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 26: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:45.686785 15142 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:45.688187  6393 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet c22b6d97ae844b7195c5e9c024203d48 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 6 but the committed config has opid_index of 13
I20260709 12:20:45.690342  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:45.636938  6501 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 24.
I20260709 12:20:45.691149  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:45.694083 15137 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:45.694588  6549 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:45.604568  6402 catalog_manager.cc:5697] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 18 to 25. New cstate: current_term: 25 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:45.696393  6402 catalog_manager.cc:5697] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 15 to 19, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 19 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:45.701156 14543 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:20:45.701675 15157 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:45.706377  6550 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:45.707816  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:45.708191 15208 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:45.708648 15208 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:45.709050 15208 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:45.633562 15158 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:45.634454 15212 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:45.638023  6473 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:45.718039  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 80103us
I20260709 12:20:45.718824 15158 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:45.719548  6473 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:45.720978 15212 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:45.721568 15212 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:45.722028 15212 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:45.731694  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:45.586349  6651 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:47858
I20260709 12:20:45.641206 14708 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:45.729483  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:45.643283 15155 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:45.650276  6658 raft_consensus.cc:1240] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:45.659528 15024 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:45.667861 15195 consensus_queue.cc:1059] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:45.671945  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:45.673275  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:45.674229  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:45.682032 15224 consensus_queue.cc:260] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:45.631652 14858 consensus_queue.cc:1059] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 30.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:20:45.622565  6622 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:45.735025  6651 raft_consensus.cc:606] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:45.766424  6651 raft_consensus.cc:614] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:45.724768 15179 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:20:45.746574  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 322231us
I20260709 12:20:45.732767  6579 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 25.
I20260709 12:20:45.756803  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:45.736505  6582 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 24.
I20260709 12:20:45.524873 15008 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:45.760838  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:45.766633  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:45.780460  6578 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 28.
I20260709 12:20:45.659647 15006 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:45.760074  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:45.760844  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:45.763267  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:45.766650  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:45.768808  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:45.787201  6506 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 28.
W20260709 12:20:45.775017  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 152540us
I20260709 12:20:45.775975  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:45.778637  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:45.698086  6402 catalog_manager.cc:5697] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 18 to 24, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 24 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:20:45.783845  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:45.784596  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:45.786983  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:45.791925  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:45.792654  6575 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:20:45.766737 15024 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:45.797432 15224 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:45.780758 15116 raft_consensus.cc:2749] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:45.808534  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:45.756474 14708 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:45.814816  6502 raft_consensus.cc:2410] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 26 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 14.
I20260709 12:20:45.814838  6583 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 26: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:45.833815  6578 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 27. Current term is 28. Ops: [27.18-27.18]
I20260709 12:20:45.849781  6581 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 24.
I20260709 12:20:45.851980  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:45.833608  6402 catalog_manager.cc:5697] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 12 to 18, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 18 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:20:45.849856  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:45.859668  6506 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:20:45.860438  6579 raft_consensus.cc:2393] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 28: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:45.864152  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:20:45.849373 15216 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.710s	user 0.033s	sys 0.015s
I20260709 12:20:45.832623 15137 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:20:45.879390  6402 catalog_manager.cc:5697] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:45.884521  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:45.900324 15024 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:45.901036 15155 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:45.901298 14708 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:45.902164  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:45.880793 15137 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:45.903092 15137 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:45.907684 15006 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:45.911027 15208 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:45.873828 15216 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.034s	sys 0.015s
I20260709 12:20:45.914661 15227 raft_consensus.cc:3055] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:20:45.915181  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:45.916821  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:45.917703  6501 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:45.919471  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:45.920835  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:45.924746  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:45.927089  6584 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:45.929183  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:45.929930  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:45.930584  6584 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:45.873824 14968 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:45.932802 14968 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:45.935006 14968 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:45.915176 15227 raft_consensus.cc:740] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:45.938035  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:45.938534 15227 consensus_queue.cc:260] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 30.16, Last appended by leader: 16, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:45.939272  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:45.941864  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:45.943100  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:45.943240  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:45.904119  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:45.945792  6583 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:45.945665  6578 raft_consensus.cc:2468] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 24.
I20260709 12:20:45.947393  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:45.916095  6505 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:45.856655 15056 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:45.943786  6582 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 19.
I20260709 12:20:45.959524  7238 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:45.961362  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:45.962234  6580 raft_consensus.cc:3060] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:45.972158  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:45.943953  6655 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 27.
I20260709 12:20:45.982270  6475 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:45.981724  6402 catalog_manager.cc:5697] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 16 to 22. New cstate: current_term: 22 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:45.983784  6402 catalog_manager.cc:5697] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 16 to 21, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 21 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:45.986269  6658 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:47858
I20260709 12:20:45.987054  6658 raft_consensus.cc:606] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 26 LEADER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:45.983651  6475 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:45.992405  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:45.998286  6402 catalog_manager.cc:5697] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 13 to 22, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 22 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:45.954267 15008 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:46.015501  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:46.011296  6473 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:45.989391 15228 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:45.942761 15212 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:46.020928  6402 catalog_manager.cc:5697] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 11 to 27, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 27 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:46.027181  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:46.029543  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:46.032956  6506 raft_consensus.cc:2376] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 25. Re-sending same reply.
I20260709 12:20:46.034323  6509 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:46.006600 15226 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:46.037555  6402 catalog_manager.cc:5697] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 13 to 21, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 21 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:20:46.034446  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:46.072794  6501 raft_consensus.cc:2468] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 24.
I20260709 12:20:46.092537  6402 catalog_manager.cc:5697] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 21 to 24, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 24 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:46.007162 15179 consensus_queue.cc:1059] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:20:46.028445 15228 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:20:46.098691  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 64331.2us
I20260709 12:20:46.109788  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:46.110930  6576 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:46.076104 15191 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:46.151069  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:46.152057  6579 raft_consensus.cc:2393] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 23: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:46.100078 15224 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:46.108295 15228 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:46.032213 15008 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:46.105624 14858 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:46.166663 14858 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:46.169484 14858 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:46.170676  6402 catalog_manager.cc:5697] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 19 to 21, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 21 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:46.110591 15229 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:46.164294  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:46.165797  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:46.173061  6654 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:46.114233 15155 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:20:46.181337  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 170120us
I20260709 12:20:46.182271  6473 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:46.163062 15208 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:46.163012 15179 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:46.189604  6402 catalog_manager.cc:5697] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:46.187691 15229 raft_consensus.cc:697] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 25 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
W20260709 12:20:46.205080 15244 log.cc:927] Time spent T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 12:20:46.204895 15229 consensus_queue.cc:237] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:46.207563  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:46.209236  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:46.212700  6584 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 24.
I20260709 12:20:46.215680  6622 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:46.217983  6402 catalog_manager.cc:5697] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 18 to 25, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 25 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:46.220049  6622 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:46.223277  6622 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:46.155113 15191 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:46.196394 15179 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:46.245332 15179 raft_consensus.cc:3060] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:46.245750  6402 catalog_manager.cc:5697] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:46.246254 15191 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:46.253086  6575 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 26. Current term is 27. Ops: []
I20260709 12:20:46.253624  6500 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 19.
I20260709 12:20:46.254770  6576 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 28.
I20260709 12:20:46.258657  6584 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 23. Current term is 24. Ops: [23.14-23.14]
I20260709 12:20:46.259850  6584 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 27. Current term is 28. Ops: [25.15-27.16]
W20260709 12:20:46.235685 15245 log.cc:927] Time spent T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
W20260709 12:20:46.236050 15248 log.cc:927] Time spent T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 12:20:46.236938 15116 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:46.189073 15155 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:46.262266  6579 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 26.
I20260709 12:20:46.263839 15155 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:46.240567 15202 raft_consensus.cc:2804] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:46.282532 15202 raft_consensus.cc:697] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 24 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:20:46.283200  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 61703us
I20260709 12:20:46.283762 15202 consensus_queue.cc:237] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:46.285490  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:46.288183  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:46.288782  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:46.289502  6506 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 28.
I20260709 12:20:46.289736  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:46.290253  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:46.290750  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:46.292096  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:46.293397  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:46.293836 15106 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:46.294616 15106 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:46.295198  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:46.243680 15212 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:46.267948 14926 raft_consensus.cc:3055] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:46.298395 14926 raft_consensus.cc:740] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:46.298579  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:46.299474  6653 raft_consensus.cc:2393] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 24: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:46.299383 14926 consensus_queue.cc:260] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:46.301134 14926 raft_consensus.cc:3060] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:46.238917  6581 raft_consensus.cc:1240] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 25. Current term is 26. Ops: [25.14-25.14]
I20260709 12:20:46.272670  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:46.303689  6577 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:46.247119  6402 catalog_manager.cc:5697] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 17 to 25. New cstate: current_term: 25 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:20:46.304018  6581 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:46.316315 15227 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:20:46.254346  6575 raft_consensus.cc:1240] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 26. Current term is 27. Ops: [26.13-26.13]
I20260709 12:20:46.318202 15106 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:46.275321 15008 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:46.275818 15235 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:46.319125 15235 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:46.321276 15235 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:46.277261  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:46.277737 15121 consensus_queue.cc:1059] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:20:46.278875 15193 raft_consensus.cc:3055] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:46.322570  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:46.322883 15193 raft_consensus.cc:740] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:46.323071 15121 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:46.323503  6504 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 27: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:46.323823  6505 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 27.
I20260709 12:20:46.194573 15246 raft_consensus.cc:3055] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:46.324599 15246 raft_consensus.cc:740] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:46.324254 15193 consensus_queue.cc:260] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:46.324819  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:46.325474 15193 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:46.325600 15246 consensus_queue.cc:260] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:46.327245 15246 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:46.294008  6503 raft_consensus.cc:3060] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:46.329496  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:46.330354 15172 raft_consensus.cc:3055] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:46.330709 15172 raft_consensus.cc:740] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:46.331005  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:46.331640 15172 consensus_queue.cc:260] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:46.332223  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:46.332934 15172 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:20:46.322613  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 45442.1us
I20260709 12:20:46.339054  6500 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 31.
W20260709 12:20:46.344995  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 121759us
W20260709 12:20:46.303205  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:46.346275  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:46.304824  6402 catalog_manager.cc:5697] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 20 to 27. New cstate: current_term: 27 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:46.306311  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:46.349457  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:46.309186  6653 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 31 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 30. Current term is 31. Ops: []
I20260709 12:20:46.351457  6503 raft_consensus.cc:2468] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 24.
I20260709 12:20:46.352424  6580 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 19.
I20260709 12:20:46.311117  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:46.242319 15137 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:46.314077  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:46.352065  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 169879us
I20260709 12:20:46.355530  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:46.356846  6650 raft_consensus.cc:2393] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 27: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:46.356792  6473 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:46.274796  6583 raft_consensus.cc:2468] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 26.
I20260709 12:20:46.362309  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:46.366566 14543 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:46.368186  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:46.361701 15116 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:20:46.369624  6500 raft_consensus.cc:1240] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 26. Current term is 27. Ops: [26.13-26.13]
I20260709 12:20:46.387074 14543 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:20:46.220059  6548 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:46.370543  6503 raft_consensus.cc:1240] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 27. Current term is 28. Ops: [27.16-27.16]
I20260709 12:20:46.387701 14543 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:46.372721  6654 raft_consensus.cc:2468] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 23.
I20260709 12:20:46.388170 14543 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:20:46.388577  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 168639us
I20260709 12:20:46.388602 15155 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:46.389088  6652 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:46.393363 15155 raft_consensus.cc:697] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:46.394824 15155 consensus_queue.cc:237] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:46.370910  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:46.399623  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:46.380165  6402 catalog_manager.cc:5697] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:20:46.378975  6509 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 27.
I20260709 12:20:46.348141  6475 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:46.377225  6505 raft_consensus.cc:1240] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 25. Current term is 26. Ops: [25.14-25.14]
I20260709 12:20:46.401814  6548 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:46.413944  7268 raft_consensus.cc:1240] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 21. Current term is 22. Ops: []
W20260709 12:20:46.415179  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 58463.2us
I20260709 12:20:46.416286  6473 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:20:46.418391  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 70338.1us
I20260709 12:20:46.420091  6550 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:46.420207  6548 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:46.422204  6550 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:46.424381  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:46.371315 15227 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:46.426432  6581 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 26.
I20260709 12:20:46.436991  6402 catalog_manager.cc:5697] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 13 to 20, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 20 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:46.440672  6473 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:46.433053  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:46.433584  6548 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:46.432224  6622 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:46.449090  6622 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:46.453234  6402 catalog_manager.cc:5697] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 20 to 24, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 24 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:46.454545  6502 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 23. Current term is 24. Ops: [23.14-23.14]
I20260709 12:20:46.417618 15116 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:20:46.457516  6548 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:46.430595 14543 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:46.423540 15216 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:20:46.421777 15208 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:46.472626  6654 raft_consensus.cc:1240] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:46.473297  6577 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 25.
I20260709 12:20:46.473531  6657 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:47858
I20260709 12:20:46.463974  6402 catalog_manager.cc:5697] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 16 to 21, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 21 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:46.426090 15226 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:20:46.476158  6657 raft_consensus.cc:606] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Received request to transfer leadership to TS 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:46.466641  6622 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:46.455724 14871 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:46.432642 15256 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:46.438531 15158 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:46.435088 15227 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:20:46.484601  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 64622.9us
I20260709 12:20:46.498323  6473 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:46.469218 15024 raft_consensus.cc:2804] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:46.491161 14543 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:46.471933 15216 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:46.495370  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:46.531939  6509 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 26.
I20260709 12:20:46.527136  6473 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:46.489233  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:46.492194 15056 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:20:46.482720  6402 catalog_manager.cc:5697] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 9 to 11, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 11 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:20:46.478118 15226 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:46.549031 15226 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:46.473161 15202 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:46.457291 15240 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:20:46.486712 15008 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:20:46.500550  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 93861.8us
I20260709 12:20:46.540038  6473 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:46.549073  6502 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 26. Current term is 27. Ops: []
I20260709 12:20:46.553644  6508 pending_rounds.cc:85] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 14
I20260709 12:20:46.566025  6508 pending_rounds.cc:107] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 25.15
I20260709 12:20:46.566481  6508 raft_consensus.cc:2889] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:46.500685 15227 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:46.530715 15116 raft_consensus.cc:2804] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:20:46.579149  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 93895.9us
I20260709 12:20:46.552069 15224 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:20:46.558348  6402 catalog_manager.cc:5697] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 19 to 22. New cstate: current_term: 22 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:46.534917 15216 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:20:46.569458  6473 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
W20260709 12:20:46.570215  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 103695us
I20260709 12:20:46.494561 15228 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:46.585894  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 139900us
I20260709 12:20:46.542413 15024 raft_consensus.cc:697] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:46.558874 15240 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:46.587745  6473 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:46.586611  6622 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:46.567157 15243 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:46.572518 15195 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:46.572492 15191 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:46.593987  6548 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:46.585532 15137 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:46.597093 14543 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:46.603715  6473 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:46.604697  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:46.595871 15006 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:20:46.616945  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:46.590572 15116 raft_consensus.cc:697] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 24 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:46.618647  6548 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:46.597805  6402 catalog_manager.cc:5697] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 22 to 26, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 26 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:46.602861 15240 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:20:46.632745 15259 consensus_queue.cc:1059] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:46.633211 15271 consensus_queue.cc:1059] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:46.625654 14543 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:20:46.692827 14871 raft_consensus.cc:3055] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:46.693058 14543 raft_consensus.cc:697] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 27 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:46.693219 14871 raft_consensus.cc:740] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:46.694110 14543 consensus_queue.cc:237] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:46.695572 15264 log.cc:927] Time spent T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260709 12:20:46.697120 15260 log.cc:927] Time spent T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
I20260709 12:20:46.609938 15224 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:20:46.637727  6473 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:46.702025  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 64358.3us
I20260709 12:20:46.702832 15256 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:20:46.703440 15256 raft_consensus.cc:697] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 28 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:46.595695 15008 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.031s
I20260709 12:20:46.645294 15242 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:46.609381 15202 consensus_queue.cc:1059] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.029s
I20260709 12:20:46.590231 15255 consensus_queue.cc:1059] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.055s
I20260709 12:20:46.646728 15116 consensus_queue.cc:237] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:46.647976 14901 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:46.612280 15195 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:46.657213  6503 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:46.665505 15253 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:20:46.670071 15228 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:46.694185 14871 consensus_queue.cc:260] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:46.631256 15191 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:46.706236  6502 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:20:46.706506  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 88031.2us
I20260709 12:20:46.721782 15273 raft_consensus.cc:2804] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:46.707854  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:46.708675  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:46.712630  6577 raft_consensus.cc:1240] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:46.704281 15256 consensus_queue.cc:237] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:46.717598  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:46.759474  6582 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 27: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:46.644225  6402 catalog_manager.cc:5697] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 18 to 26, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 26 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
W20260709 12:20:46.731091  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 109383us
I20260709 12:20:46.731649 15121 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:20:46.736815  6473 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:46.746929  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:46.753535  6548 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:46.687651 15266 log.cc:927] Time spent T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 12:20:46.761848  6584 raft_consensus.cc:2393] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 24: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:46.763180  6548 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:46.768575  6548 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:46.755262 14871 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:46.773030 15242 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:20:46.773973 15242 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:46.774330 15242 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:20:46.776496 15056 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:20:46.777119 15056 raft_consensus.cc:697] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:46.778087 15056 consensus_queue.cc:237] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:46.767215 15273 raft_consensus.cc:697] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:46.789804 15224 raft_consensus.cc:3055] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:46.673547 15024 consensus_queue.cc:237] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:46.793763  6657 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 28. Current term is 29. Ops: []
I20260709 12:20:46.790315 15224 raft_consensus.cc:740] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:46.754045 15191 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:20:46.800423  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 419515us
I20260709 12:20:46.828487  6402 catalog_manager.cc:5697] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 10 to 24, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 24 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:46.765964 15207 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:20:46.750950 14901 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:46.764467 14926 raft_consensus.cc:3055] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:20:46.836000  7238 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:46.822389 15191 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:46.862212  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:46.871631  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:46.854310 14926 raft_consensus.cc:740] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
W20260709 12:20:46.890021  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 128890us
I20260709 12:20:46.894212 15137 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:46.874404 15234 raft_consensus.cc:2804] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:20:46.872907 14998 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:46.789830 15273 consensus_queue.cc:237] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:46.897725 15179 consensus_meta.cc:287] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.652s	user 0.007s	sys 0.004s
I20260709 12:20:46.916056 15234 raft_consensus.cc:697] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 19 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:46.875720 15202 consensus_queue.cc:1059] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 24.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 12:20:46.917977  6507 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 25.
I20260709 12:20:46.923677 15234 consensus_queue.cc:237] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:46.905370 14998 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:46.932952  6504 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22.
I20260709 12:20:46.931913 15116 consensus_queue.cc:1059] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.012s
I20260709 12:20:46.941064 15202 raft_consensus.cc:3055] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:20:46.941183 15278 consensus_queue.cc:1059] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:46.942586 15278 raft_consensus.cc:3055] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:20:46.943020 15278 raft_consensus.cc:740] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:46.941582 15202 raft_consensus.cc:740] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:46.944290 15202 consensus_queue.cc:260] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 26.13, Last appended by leader: 13, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:46.946005 15202 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:46.932374  6579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 3268) took 1054 ms. Trace:
I20260709 12:20:46.910939 15227 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:46.935565 14998 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:46.950865  6579 rpcz_store.cc:276] 0709 12:20:45.877839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:46.263416 (+385577us) service_pool.cc:224] Handling call
0709 12:20:46.263721 (+   305us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:20:46.264063 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:46.264086 (+    23us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:20:46.264666 (+   580us) log.cc:844] Serialized 52 byte log entry
0709 12:20:46.488388 (+223722us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:20:46.587172 (+ 98784us) log.cc:844] Serialized 14 byte log entry
0709 12:20:46.589107 (+  1935us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:46.589191 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:46.775153 (+185962us) spinlock_profiling.cc:243] Waited 3.23 ms on lock 0x7b18002ce240. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:20:46.909355 (+134202us) spinlock_profiling.cc:243] Waited 5.09 ms on lock 0x7b18002ce240. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c522 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:20:46.932036 (+ 22681us) raft_consensus.cc:1685] finished
0709 12:20:46.932062 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:46.932228 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1687,"prepare.queue_time_us":61959,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":818,"raft.queue_time_us":5200,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":9632,"spinlock_wait_cycles":19155456,"thread_start_us":285241,"threads_started":2,"wal-append.queue_time_us":340969}
I20260709 12:20:46.916653 15179 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:46.988977  6500 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:46.996598  6506 raft_consensus.cc:2393] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 26: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:46.944279 14926 consensus_queue.cc:260] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:47.014855 15275 raft_consensus.cc:3055] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:20:47.015326 15275 raft_consensus.cc:740] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:47.027915  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:47.030117  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:47.030321  6580 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:20:47.030584  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:47.030866  6580 raft_consensus.cc:606] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:47.031256  6580 raft_consensus.cc:614] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:47.037331  6583 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
I20260709 12:20:47.037490 15006 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:47.049567 15006 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:47.058230  6502 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 29.
I20260709 12:20:47.060745  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:20:47.062822  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:47.064109  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:47.067760  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:47.069023  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:47.076383 15006 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:47.076592 15227 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:47.051903 14926 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:47.077165 15278 consensus_queue.cc:260] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:47.078083 15242 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:47.081180 15179 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:47.082027 15275 consensus_queue.cc:260] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:47.083547 15275 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:47.085150  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:20:47.062181 15226 consensus_meta.cc:287] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.512s	user 0.014s	sys 0.000s
I20260709 12:20:47.019321 14998 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:47.037719 15155 consensus_queue.cc:1059] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.052s
I20260709 12:20:47.080207 15242 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 32 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:47.128545  6503 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
W20260709 12:20:47.130496  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 176923us
I20260709 12:20:47.135378 15215 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:47.139071 15215 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:47.140882  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:47.099175 15226 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:47.147583  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:47.155911  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:47.144032 15276 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:47.168630 15276 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:47.169270  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:47.147835 14926 consensus_queue.cc:1059] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.010s
I20260709 12:20:47.170948 15276 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:47.148825 15191 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:47.159060 15289 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:47.133839 15024 consensus_queue.cc:1059] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 28.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.019s
I20260709 12:20:47.151868 15288 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:47.174362 15270 raft_consensus.cc:3055] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:20:47.174798 15270 raft_consensus.cc:740] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:47.146411 14871 consensus_queue.cc:1059] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.031s
I20260709 12:20:47.156834  7265 raft_consensus.cc:1240] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 27. Current term is 28. Ops: []
W20260709 12:20:47.178336  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:47.181417  6623 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:47.182240 15157 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:47.182719 15157 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:47.183377 15226 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:47.183682  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:47.184118  6506 raft_consensus.cc:2368] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 27. Current term is 28.
I20260709 12:20:47.184401  6651 raft_consensus.cc:2393] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 24: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:47.189304 14998 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 27 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:47.189939  6501 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 28. Current term is 29. Ops: []
I20260709 12:20:47.169492 15215 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:47.206441  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:47.211845  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:47.212769  6654 raft_consensus.cc:2410] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 27 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 13.
I20260709 12:20:47.226970  6652 raft_consensus.cc:2393] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 26: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:47.209695  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:47.228315  6473 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:47.205726  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:47.230671  6650 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:20:47.227725  6657 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 24.
I20260709 12:20:47.237288  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:47.237957  6504 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:20:47.248699  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:47.251557  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:47.252395  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:47.252494  6500 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 25.
I20260709 12:20:47.256805  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:47.258749  6548 leader_election.cc:400] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 27. Current term is 28.
I20260709 12:20:47.259111  6548 leader_election.cc:403] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:47.249677  6506 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 26.
I20260709 12:20:47.202673 15191 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:47.287566  6502 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:20:47.294485  6473 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:47.205644 15288 raft_consensus.cc:697] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:47.202864 15289 raft_consensus.cc:697] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 25 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:47.203477 15226 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:20:47.322969  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 166941us
I20260709 12:20:47.175560 15270 consensus_queue.cc:260] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.15, Last appended by leader: 15, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:47.325870  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:47.240852 15008 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:47.239629 15253 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:47.286230 15243 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:47.226778 15275 consensus_queue.cc:1059] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.099s
W20260709 12:20:47.321992 15216 consensus_meta.cc:287] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.713s	user 0.005s	sys 0.005s
I20260709 12:20:47.354712  6504 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 32.
I20260709 12:20:47.321861 15157 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:47.364539  6504 raft_consensus.cc:1275] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:20:47.366412  6502 raft_consensus.cc:1240] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 24. Current term is 25. Ops: []
W20260709 12:20:47.367460  6622 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:47.362811 15270 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:20:47.428086  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 195995us
I20260709 12:20:47.364818 15193 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:20:47.348757 15226 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:47.368798 15295 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:20:47.354732 15008 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:47.361783 15216 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:47.448890  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 190275us
W20260709 12:20:47.440183 15240 consensus_meta.cc:287] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.731s	user 0.011s	sys 0.000s
I20260709 12:20:47.356698 15253 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:47.486222 15253 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:47.435737 15024 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:20:47.445923 15275 consensus_queue.cc:1059] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.053s
I20260709 12:20:47.453516 15289 consensus_queue.cc:237] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:47.456372 15294 consensus_queue.cc:1059] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:47.464969 15240 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:47.515174 14901 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:47.516477 15155 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:47.517194 15155 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:47.517968 15224 consensus_queue.cc:260] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:47.467094  6509 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 28. Current term is 29. Ops: []
I20260709 12:20:47.468279 15216 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:47.469650 15226 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:47.470006 14871 raft_consensus.cc:3055] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:47.449510 15288 consensus_queue.cc:237] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:47.478127 15278 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:20:47.519055  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 258252us
I20260709 12:20:47.530702 14871 raft_consensus.cc:740] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:47.519604 15008 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:47.536126 14871 consensus_queue.cc:260] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:47.536864 15207 raft_consensus.cc:3055] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:47.537358 15207 raft_consensus.cc:740] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:47.537679 14871 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:47.538256 15207 consensus_queue.cc:260] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:47.539654 15207 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:47.516352 15240 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:47.544668  7240 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:47.532230  6650 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 32.
I20260709 12:20:47.549556  6652 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:47.550614  7238 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:47.551440  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:20:47.551928  6475 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:47.552647  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:47.553081  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:47.554854  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:47.547644  6548 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:47.556906  6548 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:47.563191  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:47.564208  6500 raft_consensus.cc:2410] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 29 because replica has last-logged OpId of term: 27 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 17.
I20260709 12:20:47.563324 15155 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:47.565577  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 271155us
I20260709 12:20:47.566015  6548 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:47.568265  6473 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:47.530762 15224 raft_consensus.cc:3060] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:47.570542  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:47.567373 15121 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.008s
I20260709 12:20:47.575788 15276 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:20:47.565086 15137 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:47.576422 15155 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:20:47.576933 15155 raft_consensus.cc:697] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 32 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:47.577766 15155 consensus_queue.cc:237] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 30.16, Last appended by leader: 16, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:47.578747 15267 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:20:47.571041  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:47.581118 14757 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:47.582173  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:47.583132  6504 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:47.572742 15240 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:47.569577 15297 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:47.590518 15240 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:47.590593 15297 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:47.594806 15302 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:47.596035 15302 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:47.598629 15302 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:47.581626 14757 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:47.599669 14757 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:47.603453 15195 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:47.605011 15279 raft_consensus.cc:3055] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:47.610257 15297 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:47.623248  6504 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 27.
I20260709 12:20:47.585778 15299 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 12:20:47.595695 15240 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:47.595057 15303 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:47.649708 15279 raft_consensus.cc:740] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:47.650969 15279 consensus_queue.cc:260] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:47.653826 14757 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:47.657344  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:47.658052  6657 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:20:47.660696  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 212181us
W20260709 12:20:47.662181  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 72154.3us
I20260709 12:20:47.664525  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:47.667153  6622 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:47.681073  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:47.691285 15301 raft_consensus.cc:2804] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:20:47.692058 15301 raft_consensus.cc:697] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 26 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:47.693025 15301 consensus_queue.cc:237] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:47.696507 15286 raft_consensus.cc:993] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4: : Instructing follower 496c7c3792cd4587b4dbeba435f4be1e to start an election
I20260709 12:20:47.697074 15286 raft_consensus.cc:1081] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Signalling peer 496c7c3792cd4587b4dbeba435f4be1e to start an election
I20260709 12:20:47.699659  6579 raft_consensus.cc:1275] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:20:47.702278 15024 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 27. Current term is 28.
I20260709 12:20:47.703054  6577 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 27. Current term is 28. Ops: []
I20260709 12:20:47.705452  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:47.661669 15279 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:47.710901  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:47.707837  6579 raft_consensus.cc:1275] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 12:20:47.711562  6583 raft_consensus.cc:3060] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:47.716046  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:47.717229  7268 raft_consensus.cc:2368] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 24. Current term is 25.
I20260709 12:20:47.717473  6502 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 23. Current term is 24. Ops: [23.14-23.14]
W20260709 12:20:47.714831  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 161817us
I20260709 12:20:47.665354 14757 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:47.737546  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:47.738507  7268 raft_consensus.cc:2410] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 29 because replica has last-logged OpId of term: 27 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 17.
I20260709 12:20:47.739773  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:47.730509  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:47.741096  6580 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 23.
I20260709 12:20:47.731581  6504 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 27. Current term is 28. Ops: []
I20260709 12:20:47.715106 14901 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:47.732633  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:47.743845  6583 raft_consensus.cc:2468] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 27.
I20260709 12:20:47.744549  6576 raft_consensus.cc:1240] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:47.744519  6657 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 25.
I20260709 12:20:47.747488  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:47.750007  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:47.750528  7240 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:47.748107  7274 raft_consensus.cc:1240] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:47.749354  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:47.752417 15300 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.005s
I20260709 12:20:47.755457 15291 raft_consensus.cc:3055] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:47.755863 15291 raft_consensus.cc:740] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:47.756242  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:47.756716 15291 consensus_queue.cc:260] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:47.757454  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:47.758942 15235 raft_consensus.cc:2804] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:47.759425 15235 raft_consensus.cc:697] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 25 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:47.760252 15235 consensus_queue.cc:237] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:47.760926  6583 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 28.
I20260709 12:20:47.763230  6579 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:20:47.763095 15275 consensus_queue.cc:1059] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:47.765384  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:47.766799  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:47.739137  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:47.768465  6577 raft_consensus.cc:2393] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 26: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:47.780251  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 3442) took 1449 ms. Trace:
W20260709 12:20:47.784464  6547 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:47.746996  7238 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:47.780802  6509 rpcz_store.cc:276] 0709 12:20:46.299379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:46.532923 (+233544us) service_pool.cc:224] Handling call
0709 12:20:46.533337 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:47.518915 (+985578us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:47.748708 (+229793us) spinlock_profiling.cc:243] Waited 933 ms on lock 0x7b54000fee54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:47.748859 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2146941184}
I20260709 12:20:47.750977 15295 raft_consensus.cc:697] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 29 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:47.788166 15295 consensus_queue.cc:237] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:47.795634  6623 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.003s	sys 0.012s
W20260709 12:20:47.796083  6623 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.003s	sys 0.012s
W20260709 12:20:47.796574  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 568035us
I20260709 12:20:47.821477  6506 raft_consensus.cc:1275] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:20:47.830121  6549 leader_election.cc:400] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 24. Current term is 25.
I20260709 12:20:47.830976  6549 leader_election.cc:403] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:47.792431 15228 raft_consensus.cc:2804] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:20:47.833524  6504 raft_consensus.cc:1275] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:20:47.797462  6577 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:56124
I20260709 12:20:47.836038  6577 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:47.836501  6577 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:47.835233  6549 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:47.846628  7274 raft_consensus.cc:1240] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:47.857918  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:47.861704  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:47.767871 14901 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:47.863297  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 60748.4us
I20260709 12:20:47.865483  6473 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:47.868611  6473 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:47.869673  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 57434.9us
I20260709 12:20:47.885812  6653 raft_consensus.cc:1275] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:20:47.803107 15271 consensus_queue.cc:1059] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.057s
I20260709 12:20:47.887303  6473 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:47.821859 15278 consensus_queue.cc:1048] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 12:20:47.852293 15008 raft_consensus.cc:3060] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:20:47.895712  6653 raft_consensus.cc:1275] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
W20260709 12:20:47.889937  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 54818.2us
I20260709 12:20:47.867477 15291 raft_consensus.cc:3060] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:47.899482  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:47.888746  6473 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:47.910539  6473 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:47.912411  6577 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:47.914273 15056 consensus_queue.cc:1048] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:47.914347 15144 consensus_queue.cc:1048] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:47.915978  6577 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:47.869213 15286 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:47.886173 15216 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:47.866289 15155 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:47.866461 15228 raft_consensus.cc:697] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 27 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:47.896405 15299 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:20:47.926478  6506 raft_consensus.cc:1275] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 12:20:47.890563 15202 raft_consensus.cc:3055] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:47.935266  6473 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:47.892386 15179 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:20:47.898307 15227 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:20:47.944522  6550 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:47.948267  6550 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:47.949661  6473 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:47.922586 15256 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:47.952471  6473 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:47.929348 15273 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:47.960954  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: true dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:47.946400 15247 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 12:20:47.974540  6653 raft_consensus.cc:3055] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:20:47.974907  6653 raft_consensus.cc:740] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:47.975708  6653 consensus_queue.cc:260] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:47.934298 15275 consensus_queue.cc:1048] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:20:47.977469  6653 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:47.943608 14926 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:20:47.979355 15228 consensus_queue.cc:237] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:47.958516 15226 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:20:47.946444 15155 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:47.973434 15273 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:47.985739  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: true dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:47.995957  7274 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:20:47.945570 15202 raft_consensus.cc:740] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:47.956032 15207 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:47.952950 15006 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:20:48.027940  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 75591.5us
I20260709 12:20:48.028532  6576 raft_consensus.cc:1275] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:20:47.959250 15157 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:20:47.950002 15116 consensus_queue.cc:1059] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.056s
I20260709 12:20:47.989567 15289 consensus_queue.cc:1059] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 12:20:48.036190 14543 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:48.049988  6576 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:48.040912 15247 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:20:47.941293 15286 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:48.075795  6576 raft_consensus.cc:1275] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:20:48.082546  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:47.958781 15288 consensus_queue.cc:1059] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 28.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.089s
I20260709 12:20:47.988881 15271 consensus_queue.cc:1048] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 12:20:48.083262  6503 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 26.
I20260709 12:20:48.091140  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:48.077621 15273 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:48.077911 15286 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:48.077759 14543 raft_consensus.cc:3055] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:48.092648 14543 raft_consensus.cc:740] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:48.096731  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:48.098510  6505 raft_consensus.cc:1275] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:20:48.104307  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:48.041160 14926 raft_consensus.cc:697] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 28 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:48.093698 14543 consensus_queue.cc:260] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:48.117054  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:48.125650  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:48.125720  6505 raft_consensus.cc:2393] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 25: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
W20260709 12:20:48.126977  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 178812us
I20260709 12:20:48.132741  6549 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:48.135871  6653 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:20:48.148706  6550 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:48.155049  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:48.159164  6653 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 25.
I20260709 12:20:48.143662 15273 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:48.143357 15300 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:48.146736 15288 consensus_queue.cc:1048] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 12:20:48.159794 15215 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:20:48.199270 15309 log.cc:927] Time spent T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
W20260709 12:20:48.200953 15312 log.cc:927] Time spent T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.241s	user 0.000s	sys 0.001s
I20260709 12:20:48.163535 15172 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:48.208947 15172 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:48.213325 15301 raft_consensus.cc:3055] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:20:48.213846 15301 raft_consensus.cc:740] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:48.214898 15301 consensus_queue.cc:260] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.15, Last appended by leader: 15, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:48.218429 15313 log.cc:927] Time spent T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 12:20:48.220580  6508 raft_consensus.cc:1240] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 23. Current term is 24. Ops: []
W20260709 12:20:48.225162 15314 log.cc:927] Time spent T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:20:48.232448 15300 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:48.233296 15300 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:48.238103 15288 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:48.239388 15317 log.cc:927] Time spent T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:20:48.238880 15288 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:48.240666 14926 consensus_queue.cc:237] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:48.241753 15288 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:48.254587 15215 raft_consensus.cc:697] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:48.255966 15215 consensus_queue.cc:237] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:48.257356 15155 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:48.157017 15299 consensus_queue.cc:1048] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.103s
I20260709 12:20:48.262912  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:48.156591 15157 consensus_queue.cc:1048] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.108s
I20260709 12:20:48.291038  6650 raft_consensus.cc:2410] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 25 because replica has last-logged OpId of term: 23 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
I20260709 12:20:48.129392 15202 consensus_queue.cc:260] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:48.145989 15294 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.147s
I20260709 12:20:48.292575 14901 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:48.295755 14871 raft_consensus.cc:3055] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:20:48.296303 14871 raft_consensus.cc:740] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:48.297308 14871 consensus_queue.cc:260] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:48.301678  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:48.302387  6577 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 23.
I20260709 12:20:48.309638 15243 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:48.310377 15243 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:48.193140 15240 consensus_meta.cc:287] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.532s	user 0.012s	sys 0.000s
I20260709 12:20:48.326473 15240 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:48.328411 15202 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:48.328683  6576 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 27: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:48.336063 14543 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:20:48.336760 15297 consensus_meta.cc:287] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.684s	user 0.011s	sys 0.000s
I20260709 12:20:48.337198 15297 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:48.338297  6578 raft_consensus.cc:2393] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 26: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:48.340190  6580 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 25. Current term is 26. Ops: [25.20-25.20]
I20260709 12:20:48.341671  7238 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:48.365623 15229 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:48.366214 15229 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:48.366623 15229 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:48.367815 15243 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:48.369480  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:48.370031 15301 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:20:48.370147  7270 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 28.
I20260709 12:20:48.372828 14871 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:48.377496  6474 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:48.378579 15157 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:48.379112 15157 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:48.379696 15300 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:48.343819 15315 log.cc:927] Time spent T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.207s	user 0.002s	sys 0.000s
I20260709 12:20:48.344759 15240 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:48.345458  6473 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:48.346987  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:48.409276  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:48.349768  6657 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 14. (term mismatch)
W20260709 12:20:48.410259  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 64897.6us
I20260709 12:20:48.410609  6657 pending_rounds.cc:85] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 13
I20260709 12:20:48.410995  6657 pending_rounds.cc:107] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 23.14
I20260709 12:20:48.411346  6657 raft_consensus.cc:2889] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:48.411603 14708 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:48.413931 15300 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:48.351584  7274 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 19.
I20260709 12:20:48.212087 15172 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:48.381292 15157 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:48.434075  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:48.434801  6577 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 28.
I20260709 12:20:48.438211  6578 raft_consensus.cc:1275] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 12:20:48.381449 15229 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:48.454329 15246 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:48.454974 15246 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:48.457389 15246 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:48.392596 15243 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:20:48.458810 15243 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:48.342042 15297 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:48.459249  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:48.459275 15243 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:20:48.460312  7274 raft_consensus.cc:2393] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 25: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
W20260709 12:20:48.466333  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 331800us
I20260709 12:20:48.468513  6576 raft_consensus.cc:1275] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:20:48.471333  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:48.471992  6576 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 23.
I20260709 12:20:48.474588  6506 raft_consensus.cc:1275] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:20:48.474743  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:48.407214 15008 consensus_meta.cc:287] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.501s	user 0.005s	sys 0.002s
I20260709 12:20:48.495201  6500 raft_consensus.cc:1240] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:48.496479  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:48.500607  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:48.500954 15300 consensus_queue.cc:1048] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:48.501850  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:48.502431  6504 raft_consensus.cc:3060] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:48.486652  6473 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:48.507791  6584 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 27: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:20:48.510252  6509 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.011s	sys 0.042s
W20260709 12:20:48.511464  6509 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.011s	sys 0.042s
I20260709 12:20:48.514606  6578 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 28. Current term is 29. Ops: []
I20260709 12:20:48.520403  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:48.521200  6576 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:20:48.530633  6504 raft_consensus.cc:2468] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 27.
W20260709 12:20:48.513746 15295 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.022s	sys 0.033s
I20260709 12:20:48.453841 15229 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:48.565780  6622 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:48.533665 15195 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:48.572050  6576 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 25.
I20260709 12:20:48.511756 15008 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) with higher term. Message: Invalid argument: T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 24. Current term is 25.
I20260709 12:20:48.571754  6475 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:48.559839 15295 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.022s	sys 0.033s
I20260709 12:20:48.590433  7238 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:48.605816 14543 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:48.606383 14543 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:48.602195  6622 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:48.612596  6576 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:20:48.556106 15172 consensus_queue.cc:1048] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.047s
I20260709 12:20:48.535439 15235 consensus_queue.cc:1059] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 24.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.068s
I20260709 12:20:48.619313  6508 raft_consensus.cc:1240] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 28 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 27. Current term is 28. Ops: [27.15-27.15]
I20260709 12:20:48.619972 14543 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:48.597523 15195 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:48.604158 15255 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:48.603638 15226 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:48.597831 14871 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:48.640106  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:48.642083  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:48.640225  6550 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:48.573473 15291 consensus_queue.cc:1048] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 12:20:48.660149  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:48.622128 15304 raft_consensus.cc:2804] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:20:48.662866 15304 raft_consensus.cc:697] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 27 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:48.663965 15304 consensus_queue.cc:237] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:48.607885 15298 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:48.606813 15179 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:48.674270  6582 raft_consensus.cc:2468] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 25.
I20260709 12:20:48.651371  6649 raft_consensus.cc:1275] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:20:48.615445 15276 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:48.680195  6657 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 32 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 30 index: 16. Preceding OpId from leader: term: 32 index: 17. (index mismatch)
I20260709 12:20:48.687156  6622 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:48.681927  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:20:48.697117  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 56974.8us
I20260709 12:20:48.649428 15195 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:48.622612 15229 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.081s
I20260709 12:20:48.672993 15179 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:48.691500 15240 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:48.726310  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:48.708194 15158 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:48.739575  6507 raft_consensus.cc:1275] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:20:48.707238 15202 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:48.709008 15215 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 12:20:48.655750 15255 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:48.731979 14871 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:20:48.735456  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 133771us
W20260709 12:20:48.749814  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 43549.1us
I20260709 12:20:48.738425 15320 raft_consensus.cc:3055] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:48.675522 15226 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:48.714426 15276 raft_consensus.cc:697] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 25 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:48.758721  6550 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:48.738760 15158 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:48.740690 15240 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:48.810621 15240 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:48.758879 15271 consensus_queue.cc:1048] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.017s
I20260709 12:20:48.782272 15305 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:48.784011 15276 consensus_queue.cc:237] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:48.787348 15179 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:48.788518 15273 consensus_queue.cc:1048] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:48.788869 15255 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:48.789105 15207 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:20:48.790220  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:48.791749 15332 log.cc:927] Time spent T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
W20260709 12:20:48.792532 15330 log.cc:927] Time spent T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
W20260709 12:20:48.793587 15331 log.cc:927] Time spent T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:20:48.764914 15158 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:20:48.795943 15325 log.cc:927] Time spent T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.158s	user 0.000s	sys 0.000s
W20260709 12:20:48.796684 15329 log.cc:927] Time spent T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
W20260709 12:20:48.797194 15326 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.181s	user 0.001s	sys 0.001s
I20260709 12:20:48.799304 15224 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:48.802042 14708 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:48.802145  6583 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:20:48.804301  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:48.755301 15202 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:48.808503 15226 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:48.759203 15320 raft_consensus.cc:740] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:48.822407  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:48.822901  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:48.827278  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:48.803862 15243 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:48.820555 15301 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:48.878804 15224 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:48.881103 15255 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:48.885601  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:48.894325  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:48.914052  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:48.915446  6506 raft_consensus.cc:2393] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 27: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:48.917593  6506 raft_consensus.cc:1275] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:20:48.924007  6509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 3454) took 1127 ms. Trace:
I20260709 12:20:48.927002  6509 rpcz_store.cc:276] 0709 12:20:47.796434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:47.797356 (+   922us) service_pool.cc:224] Handling call
0709 12:20:47.797649 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:47.824291 (+ 26642us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:47.824321 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:47.877614 (+ 53293us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b2c00000c88. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:20:47.891721 (+ 14107us) log.cc:844] Serialized 26 byte log entry
0709 12:20:48.512793 (+621072us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:20:48.512852 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:48.512939 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:48.922150 (+409211us) raft_consensus.cc:1685] finished
0709 12:20:48.922169 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:48.923946 (+  1777us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22440,"raft.queue_time_us":30669,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":13503,"spinlock_wait_cycles":33468928,"thread_start_us":566278,"threads_started":1,"wal-append.queue_time_us":706443}
I20260709 12:20:48.929306  6509 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 25. Current term is 26. Ops: []
I20260709 12:20:48.948767  6509 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 32 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 30 index: 16. Preceding OpId from leader: term: 32 index: 17. (index mismatch)
I20260709 12:20:48.926065 14901 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:48.973553  6502 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 3723) took 1055 ms. Trace:
I20260709 12:20:48.936676 15227 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:48.974014  6502 rpcz_store.cc:276] 0709 12:20:47.918227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:47.918383 (+   156us) service_pool.cc:224] Handling call
0709 12:20:47.918773 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:47.919646 (+   873us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:47.919677 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:47.934488 (+ 14811us) spinlock_profiling.cc:243] Waited 2.87 ms on lock 0x7b2c00000c88. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:20:47.943625 (+  9137us) log.cc:844] Serialized 26 byte log entry
0709 12:20:48.167328 (+223703us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:20:48.167374 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:48.167435 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:48.958969 (+791534us) raft_consensus.cc:1685] finished
0709 12:20:48.959016 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:48.973485 (+ 14469us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":547,"raft.queue_time_us":933,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"spinlock_wait_cycles":6634624,"thread_start_us":175639,"threads_started":1,"wal-append.queue_time_us":603259}
I20260709 12:20:48.976714 14901 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:48.957818 15226 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:48.921911 15301 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:49.035732 15158 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:49.043179 15301 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:49.044955  6650 raft_consensus.cc:2410] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 28 because replica has last-logged OpId of term: 26 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 12.
I20260709 12:20:49.045183 15158 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:49.045754 15243 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:48.991021 15227 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:49.051450 15227 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:49.052881  6581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 3287) took 1072 ms. Trace:
I20260709 12:20:49.053102 15202 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:49.050729  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:49.054260  6580 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:20:49.060597  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:49.061213  7270 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:20:49.085183  6580 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 29.
I20260709 12:20:49.053318  6581 rpcz_store.cc:276] 0709 12:20:47.980139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:47.982163 (+  2024us) service_pool.cc:224] Handling call
0709 12:20:47.997293 (+ 15130us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:20:47.997967 (+   674us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:47.998014 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:20:48.021810 (+ 23796us) log.cc:844] Serialized 52 byte log entry
0709 12:20:48.505361 (+483551us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:20:48.510141 (+  4780us) log.cc:844] Serialized 14 byte log entry
0709 12:20:48.574472 (+ 64331us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:48.574571 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:48.717893 (+143322us) spinlock_profiling.cc:243] Waited 7.82 ms on lock 0x7b18002dbf80. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:20:49.052584 (+334691us) raft_consensus.cc:1685] finished
0709 12:20:49.052684 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:49.052823 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":111030,"prepare.queue_time_us":8197,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":49,"raft.queue_time_us":90746,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":3537,"spinlock_wait_cycles":28070016,"thread_start_us":428936,"threads_started":1,"wal-append.queue_time_us":567764}
I20260709 12:20:49.093804  7274 raft_consensus.cc:1275] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:20:49.099596  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:49.100565  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:49.087739  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:49.103031  6500 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 27.
I20260709 12:20:49.101755  6473 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:49.100221  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:49.077158 15202 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:48.992630 15226 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:49.157698 15353 log.cc:927] Time spent T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 12:20:49.160234  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:49.162036 15349 log.cc:927] Time spent T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 12:20:49.162088  6657 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 28 index: 16. (term mismatch)
I20260709 12:20:49.163125  6657 pending_rounds.cc:85] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 15
I20260709 12:20:49.163491  6657 pending_rounds.cc:107] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 27.16
I20260709 12:20:49.163748  6657 raft_consensus.cc:2889] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:48.973131 15320 consensus_queue.cc:260] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 25.20, Last appended by leader: 20, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:49.168675  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:49.169790  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:49.116782 15255 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:20:49.175207 15224 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:49.176095 14901 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:49.177043 15311 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:49.177947 15226 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:49.167800 15202 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:49.180297  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:49.102991 15339 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.072s
I20260709 12:20:49.180269  7270 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 29.
I20260709 12:20:49.208943  6657 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 15. (term mismatch)
I20260709 12:20:49.211117  6657 pending_rounds.cc:85] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 14
I20260709 12:20:49.211436  6657 pending_rounds.cc:107] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 27.15
I20260709 12:20:49.211683  6657 raft_consensus.cc:2889] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:49.219461  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:49.231155  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:49.232458  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 33 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:49.197115 15320 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:49.241237 15256 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:20:49.239331  6475 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:49.210291 15255 raft_consensus.cc:697] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 29 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:49.247648  6475 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:49.249167  6548 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:49.228915 15289 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:49.249714 15256 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:49.216984 15311 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:49.256448 15345 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:49.283591  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:49.286137  6507 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 27.
I20260709 12:20:49.287099  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:49.287792  6508 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:20:49.250895 15144 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:49.256523 15158 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:20:49.298266  7274 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 22.
I20260709 12:20:49.300156  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:49.256449 15338 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:49.311549  6506 raft_consensus.cc:2393] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 25: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:49.259840 15227 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 12:20:49.352732  6474 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:49.258175 15289 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:49.313340 15338 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:49.366398  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 101378us
I20260709 12:20:49.312655 15157 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.049s
I20260709 12:20:49.301864 15144 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:49.371945  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 75095.1us
I20260709 12:20:49.374945  6653 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 32.
W20260709 12:20:49.376008  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 76152us
W20260709 12:20:49.382547  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 129675us
I20260709 12:20:49.369063 15256 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:49.388126  6508 raft_consensus.cc:1275] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:20:49.394413  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:49.391356  6548 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:49.314015 15255 consensus_queue.cc:237] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:49.390954 15195 consensus_meta.cc:287] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.666s	user 0.006s	sys 0.006s
I20260709 12:20:49.392168 15158 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:49.403823 15289 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:49.409631 15158 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:49.400835  6622 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:49.414740 15289 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:20:49.415155 15289 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:49.415534 15289 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:49.417282 15144 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:49.411788 15155 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:20:49.415361 15300 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:49.429315 15311 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:49.400020 15273 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:49.432986 15273 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:49.435286 15273 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:49.436105 15289 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:49.425511 15195 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:49.433370 15158 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:49.440289  6548 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:49.443158  6503 raft_consensus.cc:2393] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 25: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:49.448067 15338 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:49.448565 15369 consensus_queue.cc:1048] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:20:49.452231  6582 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54204 (request call id 1493) took 1554 ms (client timeout 1711 ms). Trace:
W20260709 12:20:49.452726  6582 rpcz_store.cc:269] 0709 12:20:47.897845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:47.899208 (+  1363us) service_pool.cc:224] Handling call
0709 12:20:49.452059 (+1552851us) spinlock_profiling.cc:243] Waited 699 ms on lock 0x7b54001cd254. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:49.452170 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1607390592}
I20260709 12:20:49.453198  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 33 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:49.454515  6503 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 32.
I20260709 12:20:49.458102  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54204 (request call id 1501) took 1093 ms. Trace:
I20260709 12:20:49.458536  6577 rpcz_store.cc:276] 0709 12:20:48.365185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:48.435406 (+ 70221us) service_pool.cc:224] Handling call
0709 12:20:49.456556 (+1021150us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:49.456845 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:49.456872 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:49.456912 (+    40us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:20:49.456950 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:49.457010 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:49.457932 (+   922us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54001cd250. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:49.458047 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2347664256}
I20260709 12:20:49.460851 15334 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:49.461663 15334 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:49.420583 15327 log.cc:927] Time spent T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.245s	user 0.000s	sys 0.003s
I20260709 12:20:49.444320 15195 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:49.472630 14901 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:49.428292 15300 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:49.475687 15288 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:49.480774 15288 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:49.481459  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:49.482448  7268 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:20:49.483083 15116 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:49.483806 15116 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:49.484700  6623 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:49.485893 15295 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:49.486411 15295 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:49.486639 15116 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:49.486888 15295 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:49.488798 15288 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:49.489425  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:49.497534  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 57373.1us
I20260709 12:20:49.498741 15300 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:49.513801  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:49.518812  7265 raft_consensus.cc:2393] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 20: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:20:49.515370  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:49.514873  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:49.520905  6508 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 29.
I20260709 12:20:49.522840  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:49.523002  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:49.470170 15334 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:49.621582  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:49.632659  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:49.499188  6548 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
W20260709 12:20:49.636478  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 137316us
I20260709 12:20:49.639797  6622 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:49.640723 15307 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:49.640866  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:49.647542 15300 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:20:49.647981 15300 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:49.648201  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:49.648404 15300 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:20:49.649478  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:49.559917 15288 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:20:49.652688 15288 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:49.653151 15288 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:20:49.655377  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:49.546988 15298 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:49.660725  6656 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 4173) took 1018 ms. Trace:
I20260709 12:20:49.661190  6656 rpcz_store.cc:276] 0709 12:20:48.642384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:48.645331 (+  2947us) service_pool.cc:224] Handling call
0709 12:20:48.646201 (+   870us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:48.676568 (+ 30367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:48.676615 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:48.695308 (+ 18693us) log.cc:844] Serialized 26 byte log entry
0709 12:20:48.903676 (+208368us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:20:48.903737 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:49.658726 (+754989us) spinlock_profiling.cc:243] Waited 29.9 ms on lock 0x7b5400373554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:49.658788 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:49.658835 (+    47us) raft_consensus.cc:1685] finished
0709 12:20:49.658848 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:49.660654 (+  1806us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2608,"raft.queue_time_us":95,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":15522,"spinlock_wait_cycles":68763008,"thread_start_us":170957,"threads_started":1,"wal-append.queue_time_us":251794}
I20260709 12:20:49.663223  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:49.663854  6656 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 25.
I20260709 12:20:49.660789 15224 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:49.664974 15224 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:49.674079  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:49.609329 15375 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:49.684191  6652 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 25.
I20260709 12:20:49.603444 15270 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:49.638861  6548 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:49.685477  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:49.645561 15024 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:20:49.697727  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 58989us
I20260709 12:20:49.703665  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:49.704385  6504 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:49.678222 15334 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:49.716307  6504 raft_consensus.cc:2468] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 28.
I20260709 12:20:49.691076 15024 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:49.721530 15024 raft_consensus.cc:3060] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:49.708436  6475 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:49.687227 15375 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:49.726495 15375 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:49.727576 15289 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:49.717981  6622 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:49.730298  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:49.731913  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:49.732681  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:49.704407 15207 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:49.679446 15298 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:49.782611  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:49.764212  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 33 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:49.695428 15270 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:49.723879 15295 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:49.699026 15300 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:49.742631  6575 raft_consensus.cc:1275] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:20:49.803684  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:49.760250 15289 raft_consensus.cc:2804] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:20:49.758138 14926 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:20:49.804662  6582 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 29.
I20260709 12:20:49.806105  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:49.807307  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:49.729092 15334 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:49.809592  6578 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:20:49.810752  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:49.790336 15155 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:49.829800  6658 raft_consensus.cc:2368] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 20. Current term is 21.
W20260709 12:20:49.812160 15373 log.cc:927] Time spent T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
W20260709 12:20:49.832975 15384 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:20:49.799597 15207 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:49.809230 14926 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:49.888005 15380 log.cc:927] Time spent T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 12:20:49.888825 15382 log.cc:927] Time spent T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:20:49.893666 15298 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:49.895293 15295 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:49.897699 15300 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:49.814715 15289 raft_consensus.cc:697] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 28 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:49.881431  6651 raft_consensus.cc:1275] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:20:49.907992  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:49.965972 15255 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:49.985113  6505 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:20:50.024155 15305 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:50.041054 15305 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:49.929136 14926 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:20:50.056849 15381 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:50.044020 15305 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:49.840401 15155 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:50.024601 15333 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:20:50.133051 15386 log.cc:927] Time spent T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.284s	user 0.001s	sys 0.000s
W20260709 12:20:50.134670 15389 log.cc:927] Time spent T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.237s	user 0.001s	sys 0.000s
I20260709 12:20:50.138072  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:50.139214  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:50.145694  6508 raft_consensus.cc:2393] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 25: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:50.145730  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:49.881433 15207 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:50.150390  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:50.154642  6583 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:20:50.158634  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:50.025302 15255 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:50.145174 15270 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:50.093137 15381 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:50.182296 15024 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:50.182932 15381 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:50.183902  6577 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 28: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:50.179931  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:50.185879 15255 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:50.125273 15289 consensus_queue.cc:237] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 26.13, Last appended by leader: 13, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:50.144951 15333 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:50.190392 15333 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:50.194705  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:50.195576  6505 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 30.
I20260709 12:20:50.201676  6583 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:20:50.203658  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:50.186034 15024 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:50.209298  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:50.209403  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:50.210371  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:50.210644  6505 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 28: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:50.216133  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:50.216419  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:50.216835  6583 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:20:50.218536  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:50.206809 15298 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:50.211561 15155 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:50.212373 15116 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:20:50.224282  6622 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:50.227885  6583 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 30.
I20260709 12:20:50.218567 15333 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:20:50.219061 15024 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:50.229954 15371 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:50.220902 15304 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:20:50.226935 15376 log.cc:927] Time spent T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.234s	user 0.001s	sys 0.000s
I20260709 12:20:50.236574  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:50.226492 15295 consensus_queue.cc:1048] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.014s
I20260709 12:20:50.224236 15298 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:50.240428 15288 consensus_meta.cc:287] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4: Time spent flushing consensus metadata: real 0.586s	user 0.006s	sys 0.005s
I20260709 12:20:50.256111  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:50.239493 15371 raft_consensus.cc:697] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:50.237298 15024 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:50.257526  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:50.262989 15202 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:50.327915 15024 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:50.261185 15304 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:20:50.334946  6650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58980 (request call id 4184) took 1504 ms. Trace:
I20260709 12:20:50.335427  6650 rpcz_store.cc:276] 0709 12:20:48.826955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:48.827062 (+   107us) service_pool.cc:224] Handling call
0709 12:20:50.330832 (+1503770us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b540044a2d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:50.331631 (+   799us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":346837376}
I20260709 12:20:50.256090 15288 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:50.338997  6505 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "55df50d51529421789bb752f47dff41a"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
new_leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:58944
I20260709 12:20:50.340183  6505 raft_consensus.cc:606] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Received request to transfer leadership to TS 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:50.340476 15381 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:50.341207 15381 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:50.343364 15381 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:50.347361 15224 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:50.339563 15024 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:50.348383 15024 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:50.351032 15024 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:50.352865  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:50.362004  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:50.366191  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:50.374761  6657 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 33 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 32. Current term is 33. Ops: [32.17-32.17]
I20260709 12:20:50.351361 15207 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:50.379703  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:50.340049 15304 raft_consensus.cc:697] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 30 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:50.403692  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:50.404517  6654 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 25.
I20260709 12:20:50.404757 15224 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:50.405364 14708 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:20:50.409168  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 201218us
I20260709 12:20:50.410053  6473 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:50.429611  6651 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "83301542a88f46cea64bc905b3199975"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:47858
I20260709 12:20:50.430509  6651 raft_consensus.cc:606] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:50.431066  6651 raft_consensus.cc:614] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:20:50.435438  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 61617.1us
I20260709 12:20:50.406072 14708 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:50.420281 15270 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:50.424710 15307 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:50.405402 15224 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:50.424997 15116 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:50.378481 15371 consensus_queue.cc:237] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:50.456574  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:50.465127  7270 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:50.451181 15270 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:50.471385 15270 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:50.462199 15116 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:50.472802 15288 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 33 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:50.474278 15116 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:50.475790 15224 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:50.479249  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:50.483467  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:50.487165  7270 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
W20260709 12:20:50.487816  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 222125us
I20260709 12:20:50.489888 14543 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:50.490588 14543 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:50.444826 15392 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:50.442767 15155 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:50.506388  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 33 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:50.450171 15298 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:50.446969 15240 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:50.527134 15240 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:50.529474 15240 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:50.514777  7268 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 29 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:20:50.535079 15345 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:50.537113 15355 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:50.537676 15355 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:50.538722 15233 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:50.539296 15233 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:50.539969 15355 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:50.519774  6508 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:58944
I20260709 12:20:50.521356  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:50.493259 15393 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:50.545552 15155 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:50.519177  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:50.558324  6500 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:20:50.476233 15246 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:50.535823 15345 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:50.548674  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:50.561790  6650 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 25.
I20260709 12:20:50.457520 15300 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:50.552109 14543 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:50.544675 15298 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:50.555754  6501 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 17.
I20260709 12:20:50.438076 15304 consensus_queue.cc:237] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.15, Last appended by leader: 15, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:50.510018 15224 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:50.457813 15307 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:50.607999  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:50.583950 15207 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:50.620606 14543 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:50.568271 15300 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:50.622112 15233 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:50.622538 15289 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:50.629060 14543 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:50.621515 15333 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:50.631592  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:50.632066  6500 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 23.
I20260709 12:20:50.524035 15392 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:50.632375  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 33 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:50.572809 15298 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:50.564370 15393 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:50.579770 15224 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:50.673520 15224 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:50.674229 15411 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:50.674785 15411 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:50.634784 15307 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:50.636204 15345 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:50.646096  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 126341us
I20260709 12:20:50.615159 15305 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:50.675925  6508 raft_consensus.cc:606] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:50.677963 15411 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:50.695981  6508 raft_consensus.cc:614] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:50.699649  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:50.701179 14543 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:50.638643 15399 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:50.666126 15289 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:50.709539  6475 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:50.713315  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:50.714525  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:50.715785  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:50.717422 14543 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:50.714639  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:50.721675  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:50.713965  7238 connection.cc:256] RPC call timeout handler was delayed by -1.3024728010000217s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17110000000000003s for a response
I20260709 12:20:50.723264  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:50.724624  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:50.728195  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:50.729903 15419 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:50.730427 15419 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:50.731745  6657 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58980 (request call id 4188) took 1086 ms. Trace:
I20260709 12:20:50.732836 15419 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:50.733244  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:50.669714 15333 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:50.739698  6500 raft_consensus.cc:1275] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 28 index: 14. (index mismatch)
I20260709 12:20:50.740610  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:50.741613  6579 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 12.
I20260709 12:20:50.742575  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:50.746337 15289 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:50.704730 15305 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:50.750312 15393 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:50.752913 15333 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:50.754073 15392 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:50.754696 15398 log.cc:927] Time spent T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 12:20:50.720149 15255 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:50.757356 15255 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:50.757519 15298 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:50.757890 15255 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:50.756026 14708 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:50.749665 15300 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:50.760406  6500 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:20:50.773119 15155 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:50.764081 15298 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:50.711926 15399 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:50.774186 15298 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:50.735479 15401 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.053s
I20260709 12:20:50.732321  6657 rpcz_store.cc:276] 0709 12:20:49.645004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:49.652679 (+  7675us) service_pool.cc:224] Handling call
0709 12:20:49.707661 (+ 54982us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:50.375462 (+667801us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:50.730621 (+355159us) spinlock_profiling.cc:243] Waited 667 ms on lock 0x7b540008f7d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:50.731520 (+   899us) spinlock_profiling.cc:243] Waited 54.6 ms on lock 0x7b540008f7d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:50.731640 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1659095168}
I20260709 12:20:50.788903  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:50.790650  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:20:50.795192  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 244463us
I20260709 12:20:50.796413  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:50.776269 14926 consensus_meta.cc:287] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65: Time spent flushing consensus metadata: real 0.629s	user 0.008s	sys 0.001s
I20260709 12:20:50.800693  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:50.781641 15195 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:50.804114 15195 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:50.793262 15410 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:50.806583 15410 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:50.806910 15195 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:50.808954 15410 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:50.781250 15255 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:50.813261 15255 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:50.777082 15399 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:50.817255 15397 log.cc:927] Time spent T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.210s	user 0.002s	sys 0.000s
I20260709 12:20:50.818303  6580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56124 (request call id 3309) took 1087 ms. Trace:
I20260709 12:20:50.820937  6550 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:50.824183 15423 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:50.824741 15307 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:20:50.825157  6658 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58980 (request call id 4196) took 1170 ms. Trace:
I20260709 12:20:50.824775 15423 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:50.825707 15423 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:50.787038 15298 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:50.825610  6658 rpcz_store.cc:276] 0709 12:20:49.654818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:49.655155 (+   337us) service_pool.cc:224] Handling call
0709 12:20:50.824968 (+1169813us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b54000b6654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:50.825101 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":370311040}
I20260709 12:20:50.818727  6580 rpcz_store.cc:276] 0709 12:20:49.731248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:49.733965 (+  2717us) service_pool.cc:224] Handling call
0709 12:20:50.452634 (+718669us) spinlock_profiling.cc:243] Waited 2.45 ms on lock 0x7b5800116198. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:20:50.452828 (+   194us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:50.453145 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:20:50.618044 (+164899us) log.cc:844] Serialized 14 byte log entry
0709 12:20:50.809348 (+191304us) spinlock_profiling.cc:243] Waited 4.56 ms on lock 0x7b1800428ce0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2875a314bc 00007f2872d7f48b 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a2923a
0709 12:20:50.817915 (+  8567us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:50.817993 (+    78us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:20:50.818056 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:50.818122 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:50.818240 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8229,"prepare.queue_time_us":190899,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":8360,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":135,"spinlock_wait_cycles":16132736,"thread_start_us":246229,"threads_started":2,"wal-append.queue_time_us":129428}
I20260709 12:20:50.827062  6577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56124 (request call id 3303) took 1097 ms. Trace:
I20260709 12:20:50.827446  6577 rpcz_store.cc:276] 0709 12:20:49.729600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:49.732446 (+  2846us) service_pool.cc:224] Handling call
0709 12:20:50.826869 (+1094423us) spinlock_profiling.cc:243] Waited 450 ms on lock 0x7b540034cbd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:20:50.827003 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1035640704}
I20260709 12:20:50.830390 15333 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:50.802786 14926 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:50.832226  6503 raft_consensus.cc:1240] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 33 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 32. Current term is 33. Ops: [32.17-32.17]
I20260709 12:20:50.833496 15334 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:50.834623  6503 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45874 (request call id 3744) took 1026 ms. Trace:
I20260709 12:20:50.835016  6503 rpcz_store.cc:276] 0709 12:20:49.807991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:49.808255 (+   264us) service_pool.cc:224] Handling call
0709 12:20:50.136565 (+328310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:50.832657 (+696092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:50.834367 (+  1710us) spinlock_profiling.cc:243] Waited 695 ms on lock 0x7b540001c2d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:50.834561 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1599267584}
I20260709 12:20:50.834625  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:50.803150  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 82965.1us
I20260709 12:20:50.838759  7274 raft_consensus.cc:2410] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 12:20:50.837780  6550 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:50.841125  6623 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:50.843204 15392 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:50.840675  6622 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:50.841703  6550 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:20:50.856555  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 133578us
I20260709 12:20:50.858354  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:50.858718  6550 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:50.859561  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:50.858381  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:50.862291  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:50.851387 15240 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:50.859113  6500 raft_consensus.cc:2393] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 23: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:50.865721  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:50.870488  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:50.870510  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:50.871467  6509 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:20:50.874161  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:50.859928 15392 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:50.857164 15216 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:50.862388 15024 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:50.862666 15374 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:20:50.898458  6622 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.2:45101 timed out after 1.767s (SENT)
I20260709 12:20:50.851706 15307 consensus_queue.cc:1048] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 12:20:50.968571 15216 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:50.977957 15216 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:50.985569  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 127598us
I20260709 12:20:50.877023 15240 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.097s
I20260709 12:20:50.880400 15294 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:51.005997  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:51.006351  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:50.880967 15392 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:51.010423  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.013116  6657 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 19.
W20260709 12:20:51.016301  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 117890us
I20260709 12:20:51.052618  6622 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
W20260709 12:20:51.010774 15387 log.cc:927] Time spent T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
W20260709 12:20:51.148942  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 61572.2us
I20260709 12:20:50.990410 15374 raft_consensus.cc:493] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:20:51.154213  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 296664us
I20260709 12:20:51.012785 15255 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:51.018231 15392 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:50.971688 15399 consensus_queue.cc:1059] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 30.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.082s
W20260709 12:20:51.161854  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 288151us
I20260709 12:20:51.164944  6550 leader_election.cc:400] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 20. Current term is 21.
I20260709 12:20:51.165333  6550 leader_election.cc:403] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:51.053407 15410 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:51.152033 15304 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:20:51.199451  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 47257.6us
I20260709 12:20:51.159859 15294 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:51.161549 15334 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:51.160600 15374 raft_consensus.cc:515] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:51.196246 15304 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:51.199864 15399 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:51.204286 15024 raft_consensus.cc:697] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:51.214430 15024 consensus_queue.cc:237] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:51.205590 15240 raft_consensus.cc:3055] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:20:51.216965 15240 raft_consensus.cc:740] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:51.217765 15240 consensus_queue.cc:260] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 32.17, Last appended by leader: 17, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:51.219019 15240 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:20:51.162729 15255 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:51.203787 14926 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 33 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:51.208518 15334 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:51.198268 15410 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:51.220767 15304 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:20:51.238210  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 1.58213e+06us
I20260709 12:20:51.244529 15334 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:51.246155  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:51.206770 15294 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:51.254346  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:51.259624  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:51.241640 15424 log.cc:927] Time spent T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:20:51.268424  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:51.232668 15432 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 20. Current term is 21.
I20260709 12:20:51.226140 15399 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:51.271201  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 113950us
I20260709 12:20:51.273007  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:51.279011  7272 raft_consensus.cc:3060] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:20:51.289057  7240 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:51.264046 15155 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:51.298043  6475 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:51.311298  7272 raft_consensus.cc:2468] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 23.
I20260709 12:20:51.308643 15369 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:51.303825 15216 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:51.318667 15369 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:51.318976 15422 consensus_queue.cc:1048] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:20:51.294556 14708 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:51.295063 15399 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:51.319435 15216 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:20:51.326568  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 161816us
I20260709 12:20:51.330680  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.331352  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.333434  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.335851 15255 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:51.336054 15374 leader_election.cc:290] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:51.337110 15431 log.cc:927] Time spent T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:20:51.337867  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.340322  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:51.342970  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.345276  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.347529  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:51.348905 15396 log.cc:927] Time spent T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.736s	user 0.014s	sys 0.017s
I20260709 12:20:51.349891  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.350090  6655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58972 (request call id 3641) took 2090 ms. Trace:
I20260709 12:20:51.350512  6655 rpcz_store.cc:276] 0709 12:20:49.260031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:49.260473 (+   442us) service_pool.cc:224] Handling call
0709 12:20:50.259405 (+998932us) spinlock_profiling.cc:243] Waited 1.53 ms on lock 0x7b5800015998. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:20:50.259593 (+   188us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:50.264056 (+  4463us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:50.264100 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:50.264665 (+   565us) log.cc:844] Serialized 26 byte log entry
0709 12:20:50.428402 (+163737us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:20:50.622908 (+194506us) log.cc:844] Serialized 14 byte log entry
0709 12:20:50.675233 (+ 52325us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:50.675308 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:51.320920 (+645612us) spinlock_profiling.cc:243] Waited 3.12 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:20:51.349871 (+ 28951us) raft_consensus.cc:1685] finished
0709 12:20:51.349891 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:51.350032 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11199,"prepare.queue_time_us":205149,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":20226,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":28754,"spinlock_wait_cycles":10707328,"thread_start_us":329237,"threads_started":3,"wal-append.queue_time_us":239350}
I20260709 12:20:51.352547 15410 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:51.353695  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.354763 14871 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:51.352183  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.355369 14871 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:51.357127  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.359117 15369 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:51.300160 15155 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:51.367563  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.327481 15216 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:51.357501 14871 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:51.327858 14708 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:51.426863 15432 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:51.428752 15301 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:51.428838 15432 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:51.429339 15301 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:20:51.431200 15301 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:51.431543 15432 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:51.425661 14871 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:51.432209 14871 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:51.434327 14871 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:51.434504  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:51.428159  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.440179  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.442134  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.448577  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:51.449481  7273 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 26.
I20260709 12:20:51.451433  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.454383  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.455031  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:20:51.455168  6652 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 25.
I20260709 12:20:51.451509  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:51.456571  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.457459  6654 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 26.
I20260709 12:20:51.458741  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:51.459684 15423 consensus_meta.cc:287] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.633s	user 0.008s	sys 0.000s
I20260709 12:20:51.413606 15374 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:51.472262  6474 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:51.508036  6577 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 25. Current term is 26. Ops: [25.20-25.20]
I20260709 12:20:51.508960  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:51.528228 15403 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:51.528797 15403 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:51.528828 15304 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:51.529428  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:51.530709  6575 raft_consensus.cc:2393] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 23: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:51.530599  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:51.531383  6578 raft_consensus.cc:2368] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for earlier term 22. Current term is 23.
I20260709 12:20:51.462383 15423 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:51.539103 15423 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:51.473264 15255 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:51.548755 15255 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:51.529368 15403 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:51.473801 15429 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:51.566365 15429 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:51.568826 15294 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:51.500181 15374 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:51.506129 15024 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:51.575101 15024 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:51.577626 15024 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:51.579066 15305 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:51.558887 15436 log.cc:927] Time spent T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:20:51.482016 15434 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:51.580962  6581 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:20:51.572734 15374 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:51.607909  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:51.609052  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:51.609386  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:51.618700  6500 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 25.
I20260709 12:20:51.610600  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 33 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:51.612428  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 286847us
I20260709 12:20:51.613337  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:51.613754 15392 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:51.614174  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.621762  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.623354  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:51.617120  7273 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22.
I20260709 12:20:51.625684  6623 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:51.623934  6654 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 24.
I20260709 12:20:51.639038  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:51.640857  6577 raft_consensus.cc:2410] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 33 because replica has last-logged OpId of term: 32 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 16.
I20260709 12:20:51.642176  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:51.642438  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:51.642843  6654 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:51.642944  6500 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:51.643277  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:51.548120 15216 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:51.645153 15304 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:51.645668 15429 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:51.644845  6622 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:51.650032 15298 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:51.650552 15298 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:51.651897  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:51.652432 15374 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:51.652935 15255 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:51.652894 15374 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:51.653379  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:51.655144 15374 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:51.655472  6579 raft_consensus.cc:1275] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 28 index: 14. (index mismatch)
I20260709 12:20:51.656550  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:51.657276  6576 raft_consensus.cc:2393] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 28: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:51.658551  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:51.659188 15392 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:51.659926  6651 raft_consensus.cc:1240] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:51.660167 15255 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:51.660737 15255 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:51.661425  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:51.622668 15434 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:51.647853 15305 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:51.672180 15305 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:51.672577 15434 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:51.672791  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:51.676846  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.648068 15438 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:51.678614 15438 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:51.680095 15195 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:51.681571  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:51.647598 15216 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:51.680640 15195 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:51.685278 15304 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:51.685811 15304 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:51.686193 15304 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:51.686205 15345 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:51.687765 15439 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:20:51.688303 15439 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:51.687983 15345 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:20:51.688633  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:51.688728 15439 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:51.693960 15155 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:51.699818  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:51.709779  6548 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:51.724294  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:51.728608 15434 raft_consensus.cc:2804] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:51.729185 15434 raft_consensus.cc:697] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:51.729238 15298 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:51.730969  6506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45880 (request call id 3499) took 1241 ms. Trace:
I20260709 12:20:51.731420  6506 rpcz_store.cc:276] 0709 12:20:50.490028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:50.490327 (+   299us) service_pool.cc:224] Handling call
0709 12:20:50.508182 (+ 17855us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:20:50.570502 (+ 62320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:50.570549 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:20:50.570612 (+    63us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:20:50.570661 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:51.730732 (+1160071us) spinlock_profiling.cc:243] Waited 62 ms on lock 0x7b5400204254. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:20:51.730777 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:51.730891 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":142700416}
I20260709 12:20:51.736315  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:51.735719  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:51.726644  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:51.646248 14708 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:51.680683 15438 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:51.743670  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:51.756287  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.757221  6509 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:51.749559  6654 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 26.
I20260709 12:20:51.751842  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.753065  6500 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22.
W20260709 12:20:51.724639 15449 log.cc:927] Time spent T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 12:20:51.752729  6622 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:51.765280  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 33 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:51.748431 15256 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:51.776270 15256 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:51.779285 15256 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:51.782660 15425 log.cc:927] Time spent T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.602s	user 0.001s	sys 0.003s
I20260709 12:20:51.767378 15301 raft_consensus.cc:2804] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:20:51.785178 15301 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:51.785616 15301 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:51.786528 15403 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:51.774693  6649 raft_consensus.cc:2393] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 33: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:51.801105  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:51.822973  6550 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:51.824823  6550 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:51.828717  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:51.834540  6509 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:20:51.836272  6550 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:51.837327 15429 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:51.842084 15446 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:51.843655  6550 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:51.844115 15434 consensus_queue.cc:237] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:51.844671  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 30 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.845816  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.846923  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.849668  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.850847  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.846974 15345 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:51.851909 15446 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:20:51.853449  6550 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:51.854297  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.855345  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:51.856127  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.874820  6550 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:51.876152  6550 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:51.877254  6550 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:51.877916 15399 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:51.878070  6550 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:51.878623 15399 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:51.879071 15429 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:51.880518 15429 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:20:51.883325  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 82333.8us
I20260709 12:20:51.884418  6622 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:51.889580  6622 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:51.892092  6650 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 25.
I20260709 12:20:51.892189 15255 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:51.905417  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 147593us
I20260709 12:20:51.908440  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:51.881683 15446 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:51.856789 15305 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:51.886085 15447 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:51.887331 15369 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:51.897305 15294 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:51.882618 15304 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:51.909577 15295 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:20:51.959255  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 53460.3us
I20260709 12:20:51.945441 15195 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:51.970798  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:51.941579 15345 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:51.973390  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:51.978878  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:51.989156  6507 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:20:51.979220  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:51.952459 15392 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:20:51.903564 15439 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:51.985083  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.985764  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:52.001436  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:51.990319  6500 raft_consensus.cc:2393] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 27: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:51.982609  6508 raft_consensus.cc:1240] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:51.991647  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:52.003153  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.994519  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.995260  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:52.001471  6582 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 25.
I20260709 12:20:51.984652  6581 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:20:51.996932  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:51.941804 15307 consensus_queue.cc:1048] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 12:20:51.999995  6655 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 25. Current term is 26. Ops: []
I20260709 12:20:52.005820  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:52.006989  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:51.967799 15423 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:52.003456  6577 raft_consensus.cc:2468] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 19.
I20260709 12:20:52.003857  6656 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 27.
I20260709 12:20:52.015416  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:52.018277  6580 raft_consensus.cc:2468] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 17.
I20260709 12:20:52.019754  6475 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:52.021157  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:52.021559  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:52.021078  6550 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:52.023211  6475 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:52.023605  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:52.024179  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 115870us
I20260709 12:20:52.025967  6475 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:52.026826  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:51.997747 15345 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:52.024978 15403 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:52.035404 15345 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:52.036139 15439 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:52.037653  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:52.036520  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:52.038640  7268 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:52.032157  7240 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:52.027211  6475 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:52.017861 15195 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:52.044113 15195 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:52.046399 15195 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:52.031795 14708 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:52.021886 15423 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:52.051436  7268 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 23.
I20260709 12:20:52.051919 15228 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:20:52.028129 15255 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:52.052800 15228 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:52.053326 15228 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:52.031404 15246 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:20:52.026719 15438 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:52.054955 15438 raft_consensus.cc:697] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:52.053436 15304 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:52.056824  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 33 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:52.058002  6580 raft_consensus.cc:2410] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 33 because replica has last-logged OpId of term: 32 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 16.
I20260709 12:20:52.042351  6475 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:52.058353  6623 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:52.060557  6475 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:52.061668  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:52.062846  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:52.063993  6475 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:52.069593  6475 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:52.060839 15345 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:52.078434  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:52.081605  6576 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 26.
I20260709 12:20:52.102535  6475 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:52.069185 15256 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:52.111382 15256 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:52.111852 15256 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:52.076860 15403 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:20:52.105312 15195 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:52.107577  7268 raft_consensus.cc:1275] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:20:52.066213 15255 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:52.063735 15246 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:52.112247  6658 raft_consensus.cc:1240] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 25. Current term is 26. Ops: []
I20260709 12:20:52.111647  6475 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:52.066620 15439 raft_consensus.cc:2804] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:20:52.149637  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:52.154029  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:52.155210  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:52.152832 15195 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:52.151371 15246 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:52.158805 15246 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:52.159181 15195 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:52.161232  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:52.160808  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:20:52.161473  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 49895.2us
I20260709 12:20:52.055925 15438 consensus_queue.cc:237] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:52.165854  6475 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:52.172991  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:52.174526  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 30 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:52.145515 15298 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:52.122545 15345 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:52.186223  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:52.191779  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:52.201994  6508 raft_consensus.cc:1813] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Handling vote request from an unknown peer 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:52.203058  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:52.206414  6508 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 22: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:52.169518 14708 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:52.153535 15403 raft_consensus.cc:697] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 26 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:52.211690  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:52.153326 15427 consensus_queue.cc:1059] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.052s
I20260709 12:20:52.224941  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 33 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:52.225879  6577 raft_consensus.cc:2410] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 33 because replica has last-logged OpId of term: 32 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 16.
W20260709 12:20:52.229529  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 115981us
I20260709 12:20:52.189671 15216 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:20:52.205731 15345 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:52.234061  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:52.235222  6577 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:52.242116  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 31 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:52.242838  6576 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 30.
I20260709 12:20:52.243903  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:52.248991  7268 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:52.255102 15228 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:52.233994 15216 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:52.256801 15216 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:52.257346  7268 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 27.
I20260709 12:20:52.258406 15462 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:52.259825 15462 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:52.264881  6474 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:52.267922 15462 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:52.268479  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:52.270007  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:52.270859  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:52.272308  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:52.272733 15403 consensus_queue.cc:237] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:52.278317  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:52.280685 15369 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:52.280810  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:52.285944  7273 consensus_queue.cc:237] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:52.286895 15439 raft_consensus.cc:697] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 23 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:52.287642  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:52.287880 15439 consensus_queue.cc:237] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:52.289862 15207 consensus_queue.cc:1059] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:52.291159 15207 raft_consensus.cc:3055] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:52.291553 15207 raft_consensus.cc:740] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:52.233155 15400 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:52.293973 15400 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:52.298121 15442 raft_consensus.cc:3055] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:52.298629 15442 raft_consensus.cc:740] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:52.235152 15301 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:52.300557  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 231242us
I20260709 12:20:52.301290  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:52.301708  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:52.302563  6501 raft_consensus.cc:2393] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 27: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:52.288304  6509 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 25.
I20260709 12:20:52.303275  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:52.304349  6508 raft_consensus.cc:2368] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 25. Current term is 26.
I20260709 12:20:52.309717  7238 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:52.315523  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 31 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:52.324817  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:52.331218  6622 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:52.332649  6649 raft_consensus.cc:1240] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:52.334710  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:52.299621 15442 consensus_queue.cc:260] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:52.337321  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:52.330489 15333 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.010s
I20260709 12:20:52.342757  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:52.347673  6651 raft_consensus.cc:1240] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 12. Current term is 13. Ops: []
I20260709 12:20:52.347958  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:52.348819  6548 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:52.334954 15415 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:52.365139  6548 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:52.365657  6653 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:20:52.358994  6509 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 30 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 28 index: 15. Preceding OpId from leader: term: 30 index: 16. (index mismatch)
I20260709 12:20:52.339139 15298 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:20:52.368491  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:52.351749 15288 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:52.349948 15294 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:52.386941  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:52.387670  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:52.392388  6548 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:52.334857 15427 consensus_queue.cc:1059] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 12:20:52.481763  6548 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:52.484206  6577 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 27.
W20260709 12:20:52.489115  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:52.374589 15233 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.116s
I20260709 12:20:52.369051 15400 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:52.373035 15301 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:52.342381 15428 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.055s
I20260709 12:20:52.479512 15155 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:52.332306 15207 consensus_queue.cc:260] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:52.495649 15207 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:52.496640  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:52.497325  6576 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 25.
I20260709 12:20:52.499119 15305 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:52.499694 15305 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:52.389671 15298 raft_consensus.cc:697] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 27 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:52.503896 15298 consensus_queue.cc:237] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.20, Last appended by leader: 20, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:52.504814 15228 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:52.504901 15305 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:52.500981 15246 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:52.369220 15446 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:52.507143 15246 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:52.502027 15345 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:52.508580  6502 raft_consensus.cc:1240] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 25. Current term is 26. Ops: []
I20260709 12:20:52.510227 15345 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:52.510456 15423 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:52.475841 15256 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:52.514416 15256 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:52.516266  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 123875us
I20260709 12:20:52.517828  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:52.518224  6622 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:52.519183  6506 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:52.525395  6622 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
W20260709 12:20:52.530670  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 49728.4us
I20260709 12:20:52.544672  6508 raft_consensus.cc:1240] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:52.546320  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:52.549823  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:52.550932  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:52.551914  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:52.554765  6652 raft_consensus.cc:2393] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 26: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:20:52.479732 15438 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:20:52.556218  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 74550us
I20260709 12:20:52.559382 15294 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:52.562808  6507 raft_consensus.cc:1240] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 26. Current term is 27. Ops: []
I20260709 12:20:52.524226 15256 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:52.535104  6549 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:52.573860  6549 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:52.574590 15240 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:20:52.575161 15442 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:52.575407  6548 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:52.577265  6548 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:52.578245 15240 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:52.579001 15240 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:20:52.577998  6549 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:52.580142 15240 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:20:52.582402 15240 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:52.584661  7273 raft_consensus.cc:1240] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:52.584635  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b17fd01574f4601b33dd37e7c78bc8f" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:52.586563  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:52.592608  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:52.537160 14871 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:52.597756  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:52.601079  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:52.601873  7272 raft_consensus.cc:2393] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 27: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:52.547545 15278 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:52.613804  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:52.614804  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:52.615607  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 51929.6us
W20260709 12:20:52.616066  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:52.617455  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:52.626298  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 30 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:52.644995  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:52.564726 15374 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:20:52.682245  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 105039us
I20260709 12:20:52.683276  6548 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:52.657294 15423 consensus_queue.cc:1059] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 12:20:52.701239  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:52.705274  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:52.706436  6579 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 21.
I20260709 12:20:52.710408  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:52.716706  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:52.720914  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:52.624117 15438 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:52.724181  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:52.724912  6583 raft_consensus.cc:1275] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:20:52.739405  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:52.728149  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:52.730615  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:52.732297  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:52.568115 15422 consensus_queue.cc:1059] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.108s
I20260709 12:20:52.726459  6548 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:52.748096  6548 leader_election.cc:400] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 25. Current term is 26.
I20260709 12:20:52.743047  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:52.743175  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:52.648475 14871 raft_consensus.cc:697] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:52.752957  6576 raft_consensus.cc:2393] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 27: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:52.756975  6583 raft_consensus.cc:2393] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 25: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:20:52.758684  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:52.672133 15255 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:52.686545 15278 raft_consensus.cc:697] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:52.623054 15216 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:52.615159 15256 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:52.773840  6549 leader_election.cc:304] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:52.740552 15374 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:20:52.695591 15307 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:52.785665  6656 raft_consensus.cc:1240] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 23 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 22. Current term is 23. Ops: []
I20260709 12:20:52.752396 15462 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:52.821190  6500 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 27. Current term is 28. Ops: []
I20260709 12:20:52.789613 15307 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:52.829602 15434 raft_consensus.cc:3055] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:52.856621 15434 raft_consensus.cc:740] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:52.794744 14871 consensus_queue.cc:237] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:52.776705 15255 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:52.903904  7273 raft_consensus.cc:1275] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:20:52.878015  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:52.899644 15403 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.026s	sys 0.066s
W20260709 12:20:52.905774 15403 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.026s	sys 0.066s
I20260709 12:20:52.906039 15255 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:52.907027 15434 consensus_queue.cc:260] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:52.875429 15469 log.cc:927] Time spent T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.305s	user 0.002s	sys 0.000s
I20260709 12:20:52.773710 15419 consensus_queue.cc:1059] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.089s
I20260709 12:20:52.911190 15419 raft_consensus.cc:3055] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:20:52.911701 15419 raft_consensus.cc:740] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:52.914698 15419 consensus_queue.cc:260] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:52.754786 15438 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:52.861294 15233 raft_consensus.cc:3055] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:52.933462 15233 raft_consensus.cc:740] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:52.933673 15419 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:52.934938 15438 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:52.866881 15288 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:52.935580 15288 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:52.937862 15288 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:52.791813 15423 consensus_queue.cc:1059] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:20:52.939180 15423 raft_consensus.cc:3055] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:20:52.939584 15423 raft_consensus.cc:740] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:52.940513 15423 consensus_queue.cc:260] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:52.941856 15423 raft_consensus.cc:3060] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:20:52.836269 15278 consensus_queue.cc:237] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:52.946535  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:52.839241 15256 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:52.777257 15446 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:52.949038 15446 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:52.949321  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:52.841897 15216 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:52.870375 15369 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
W20260709 12:20:52.925593  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 151886us
I20260709 12:20:52.934417 15233 consensus_queue.cc:260] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:52.950919 15369 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:52.951514 15233 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:52.952469  6549 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:52.953305 15489 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:20:52.954540  6549 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:52.954737 15369 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:52.955348 15489 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:52.862396 15307 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:52.949518 15446 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:52.961117  6549 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:52.961721  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:52.868288 15463 log.cc:927] Time spent T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.281s	user 0.002s	sys 0.001s
I20260709 12:20:52.859547 15155 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:52.987061  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 34 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:52.989032  6549 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:52.987942  6653 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 33.
I20260709 12:20:52.995224  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 30 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:52.970692 15447 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.013s	sys 0.032s
I20260709 12:20:53.000829  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:53.003185  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:53.003486  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:53.013700  7273 raft_consensus.cc:1240] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 19. Current term is 20. Ops: []
W20260709 12:20:53.014868  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 369972us
I20260709 12:20:53.016125  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:53.016618  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6029ffeecb94e4da196c61c032b862b" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:53.018719  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:53.019569  6575 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:20:53.021113  6579 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 28 index: 15. Preceding OpId from leader: term: 30 index: 16. (index mismatch)
I20260709 12:20:53.022471  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:53.024570  6580 raft_consensus.cc:1275] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:20:53.026367  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:53.032527  6575 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 13.
I20260709 12:20:52.989917 15434 raft_consensus.cc:3060] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:52.990339 15489 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:53.042698  7238 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:53.047482  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 34 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:53.048242  6507 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 33.
I20260709 12:20:53.013576 15374 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:20:53.034955  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 338245us
W20260709 12:20:53.004410 15447 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.013s	sys 0.032s
I20260709 12:20:53.006891  6550 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:53.093684  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 57915.8us
I20260709 12:20:53.032990 15240 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.024s
W20260709 12:20:53.098346  6402 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57900 (request call id 194) took 16239 ms (client timeout 15000 ms). Trace:
I20260709 12:20:53.059063 15415 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:20:53.103399  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 43764.6us
I20260709 12:20:53.046468 15392 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.047s
I20260709 12:20:53.046558 15450 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.048s
I20260709 12:20:53.108855  7273 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 25 index: 20. Preceding OpId from leader: term: 27 index: 21. (index mismatch)
W20260709 12:20:53.106920  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 100068us
I20260709 12:20:53.112986  7238 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:52.969491 15429 consensus_queue.cc:1175] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f2b06aebe97249a4877d798b8118ece4" responder_term: 23 status { last_received { term: 23 index: 18 } last_committed_idx: 17 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 22. Current term is 23. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:20:53.136179  6473 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:53.111222 15415 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:53.141505  7238 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:53.114964 15473 raft_consensus.cc:2749] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:53.135126 15447 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:20:53.160068  7273 raft_consensus.cc:1275] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:20:53.162375  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:53.128561 15305 consensus_queue.cc:1059] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 12:20:53.149204 15415 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:53.148263 15410 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:20:53.162669 15428 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:53.151895 15465 consensus_queue.cc:1048] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:20:53.183499 15415 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:53.184438 15207 consensus_queue.cc:1059] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 26.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:53.186846 15415 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 22 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:53.187201 15429 raft_consensus.cc:3055] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:20:53.187682 15429 raft_consensus.cc:740] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:20:53.171746  7238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:20:53.174219 15492 log.cc:927] Time spent T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:20:53.174852 15480 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:53.152648 15255 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.027s
I20260709 12:20:53.158625 15447 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:53.194913  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:53.197103 15442 consensus_meta.cc:287] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9: Time spent flushing consensus metadata: real 0.621s	user 0.014s	sys 0.000s
I20260709 12:20:53.195494 15447 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 33 FOLLOWER]: Advancing to term 34
W20260709 12:20:53.109534  6402 rpcz_store.cc:269] 0709 12:20:36.853292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:41.348022 (+4494730us) service_pool.cc:224] Handling call
0709 12:20:53.062736 (+11714714us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:47.008242 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:47.009145 (+   903us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:47.009185 (+    40us) write_op.cc:435] Acquired schema lock
0709 12:20:47.009571 (+   386us) tablet.cc:662] Decoding operations
0709 12:20:47.177711 (+168140us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:47.177788 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:47.177820 (+    32us) tablet.cc:685] Acquiring locks for 33 operations
0709 12:20:47.181241 (+  3421us) tablet.cc:701] Row locks acquired
0709 12:20:47.181260 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:20:47.181400 (+   140us) write_op.cc:270] Start()
0709 12:20:47.181518 (+   118us) write_op.cc:276] Timestamp: P: 1783599647181358 usec, L: 0
0709 12:20:47.181540 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:20:47.182272 (+   732us) log.cc:844] Serialized 12603 byte log entry
0709 12:20:47.783213 (+600941us) op_driver.cc:464] REPLICATION: finished
0709 12:20:48.244476 (+461263us) write_op.cc:301] APPLY: starting
0709 12:20:48.244584 (+   108us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:48.504700 (+260116us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:48.504811 (+   111us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:48.918361 (+413550us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:48.918826 (+   465us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=66,key_file_lookups=66,delta_file_lookups=0,mrs_lookups=0
0709 12:20:48.918888 (+    62us) write_op.cc:312] APPLY: finished
0709 12:20:48.919544 (+   656us) log.cc:844] Serialized 284 byte log entry
0709 12:20:48.983423 (+ 63879us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:49.087776 (+104353us) write_op.cc:454] Released schema lock
0709 12:20:49.099551 (+ 11775us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:49.197360 (+ 97809us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:49.206027 (+  8667us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":921704,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":381026,"apply.run_cpu_time_us":57231,"apply.run_wall_time_us":923876,"cfile_cache_hit":134,"cfile_cache_hit_bytes":141803,"num_ops":33,"prepare.queue_time_us":85498,"prepare.run_cpu_time_us":42830,"prepare.run_wall_time_us":347744,"raft.queue_time_us":318708,"raft.run_cpu_time_us":19061,"raft.run_wall_time_us":275584,"replication_time_us":601491,"thread_start_us":867892,"threads_started":5,"wal-append.queue_time_us":440444,"wal-append.run_cpu_time_us":69893,"wal-append.run_wall_time_us":2422959}]]}
W20260709 12:20:53.239820  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 102387us
W20260709 12:20:53.246731  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 111344us
W20260709 12:20:53.248840  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 129637us
I20260709 12:20:53.264101  6548 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:53.267226  6548 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:53.269552  6504 raft_consensus.cc:1240] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:53.270560  6654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:53.279210  6582 raft_consensus.cc:2468] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 23.
I20260709 12:20:53.297230  6473 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:53.307607 15305 consensus_queue.cc:1059] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 27.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 12:20:53.309788  7237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:53.309881 15195 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:20:53.197598 15429 consensus_queue.cc:260] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.13, Last appended by leader: 13, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:53.317891 15429 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:53.322711 15419 raft_consensus.cc:2955] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 20 LEADER]: Committing config change with OpId 20.11: config changed from index 4 to 11, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:20:53.323745 15116 consensus_queue.cc:1048] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:20:53.330540 15255 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:20:53.332746 15298 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:53.350010  6658 raft_consensus.cc:2955] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Committing config change with OpId 20.11: config changed from index 4 to 11, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:20:53.358523  6623 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:53.360910 15440 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:53.364681 15438 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:20:53.393182 15446 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:53.412014  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 112707us
I20260709 12:20:53.413601  6473 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:53.414503  7273 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:20:53.416739 15246 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:20:53.418655  6473 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:53.419611 15369 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:53.423367 15246 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:20:53.427242  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 100976us
I20260709 12:20:53.437395 15423 raft_consensus.cc:3055] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 LEADER]: Stepping down as leader of term 24
W20260709 12:20:53.445570 15477 log.cc:927] Time spent T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.512s	user 0.005s	sys 0.000s
I20260709 12:20:53.381436 15438 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:20:53.454224 15419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:48498 (request call id 17) took 1425 ms. Trace:
I20260709 12:20:53.454484  7268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38546 (request call id 1196) took 1088 ms. Trace:
W20260709 12:20:53.457453  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 169861us
I20260709 12:20:53.455147  7268 rpcz_store.cc:276] 0709 12:20:52.366482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:52.366753 (+   271us) service_pool.cc:224] Handling call
0709 12:20:52.380709 (+ 13956us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:20:52.381086 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:20:52.381119 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:20:52.389122 (+  8003us) log.cc:844] Serialized 26 byte log entry
0709 12:20:52.586532 (+197410us) spinlock_profiling.cc:243] Waited 26.5 ms on lock 0x7b1800155020. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:20:52.774770 (+188238us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:20:52.774850 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:20:52.774956 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:20:53.444612 (+669656us) spinlock_profiling.cc:243] Waited 57.4 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:20:53.447444 (+  2832us) raft_consensus.cc:1685] finished
0709 12:20:53.447470 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:20:53.454263 (+  6793us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40328,"raft.queue_time_us":503,"raft.run_cpu_time_us":10583,"raft.run_wall_time_us":89685,"spinlock_wait_cycles":192983168,"thread_start_us":187973,"threads_started":1,"wal-append.queue_time_us":432220}
I20260709 12:20:53.464152 15446 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:53.440409 15423 raft_consensus.cc:740] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:53.473308 15423 consensus_queue.cc:260] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:53.475302 15423 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:53.478224 15369 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:53.479104 15369 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:53.493144 15498 raft_consensus.cc:3055] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:20:53.493817 15498 raft_consensus.cc:740] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:53.495270 15498 consensus_queue.cc:260] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:53.501673 15493 log.cc:927] Time spent T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.562s	user 0.003s	sys 0.000s
I20260709 12:20:53.516615  7270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:53.525738  7270 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:20:53.454785 15419 rpcz_store.cc:276] 0709 12:20:52.028407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:52.028550 (+   143us) service_pool.cc:224] Handling call
0709 12:20:52.284045 (+255495us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b540050c6a0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e06520 00007f2875e8fb42 00007f2875e8e800 00007f2875e879fa 00007f287ae4f4c9 00007f28743cbc54 00007f28743cbbe2 00007f28743cbb11 00007f28743cba8c 00007f28743cad42
0709 12:20:52.296323 (+ 12278us) log.cc:844] Serialized 480 byte log entry
0709 12:20:52.580453 (+284130us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b540050c6a0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875dfedbe 00007f2875e09930 00007f2875e098d8 00007f2875e09858 00007f2875e09813 00007f2875e08a9c 00007f287bcf42f6 00007f287bceeaa8 00007f2875e457cb 00007f2875e482c0
0709 12:20:53.449092 (+868639us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":108927,"raft.queue_time_us":125211,"raft.run_cpu_time_us":1343,"raft.run_wall_time_us":1338,"spinlock_wait_cycles":282395904,"thread_start_us":148728,"threads_started":2,"wal-append.queue_time_us":81359}
I20260709 12:20:53.532757 15457 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 12:20:53.548857  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 76921.2us
I20260709 12:20:53.561492  6393 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3e0ab356fc074f589a6ff4f9b4996b09 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 14)
I20260709 12:20:53.569352 15369 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:53.581390  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 142198us
I20260709 12:20:53.589979  7270 raft_consensus.cc:1240] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:53.575755 15429 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:20:53.607008 15434 consensus_meta.cc:287] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e: Time spent flushing consensus metadata: real 0.562s	user 0.007s	sys 0.013s
I20260709 12:20:53.608191 15505 consensus_queue.cc:1059] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:20:53.612628  6396 maintenance_manager.cc:419] P 39c4be726d3d495181bfe7425c8c0bd8: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033382
I20260709 12:20:53.613379  6505 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:20:53.613885 15447 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:53.601619 15498 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:53.649899  6473 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:53.658735 15207 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:53.659631 15207 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:53.695113  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:53.633246 15429 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:53.701821  6651 raft_consensus.cc:1275] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:20:53.703627 15403 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:53.704277 15403 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:53.708108 15392 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.012s
I20260709 12:20:53.709331 15434 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:53.709918 15434 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:53.712486 15505 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:53.713121 15505 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:53.714463 15468 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:20:53.715507 15505 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:53.717020  6584 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 25. Current term is 26. Ops: [25.19-25.19]
I20260709 12:20:53.720494 15497 raft_consensus.cc:3055] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:53.720986 15497 raft_consensus.cc:740] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:53.721931 15497 consensus_queue.cc:260] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:53.722625 15447 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 34 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:53.723954 15497 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:53.724560  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:53.726193 15429 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:53.734632  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 53050.1us
I20260709 12:20:53.735651 15369 consensus_queue.cc:1059] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:20:53.737303  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 77916.1us
I20260709 12:20:53.703533  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:53.682817 15207 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:53.776466  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:53.762357 15403 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:20:53.779160 15508 log.cc:927] Time spent T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 12:20:53.781034  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:53.787892  6576 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:53.793715  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:53.799819  6500 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:20:53.803373  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 34 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:53.805563  6504 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:53.808717  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:53.813416  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 210799us
I20260709 12:20:53.814565  6473 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:53.820565 15403 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:20:53.822307  6655 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:20:53.765775 15498 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:53.771323 15195 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:53.836949  6506 raft_consensus.cc:2393] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 26: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:53.782819 15434 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:53.842908  6622 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:53.852284  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:53.845112  6473 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:53.863174  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:53.864748 15440 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:53.865547 15440 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:53.868728 15369 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:20:53.869503 15423 consensus_queue.cc:1048] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:20:53.838799 15498 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:53.880582  6583 raft_consensus.cc:2468] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 26.
I20260709 12:20:53.881042  6507 raft_consensus.cc:1240] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 25. Current term is 26. Ops: []
I20260709 12:20:53.883358  6576 raft_consensus.cc:2468] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 22.
W20260709 12:20:53.885058  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 193147us
I20260709 12:20:53.887068  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:53.888223 15369 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:53.888752 15392 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:53.906502  6475 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:53.894615 15508 log.cc:927] Time spent T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:20:53.911458  6500 raft_consensus.cc:1275] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:20:53.931181 14926 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:53.934351  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:53.936836  7238 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:53.940783  6657 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
I20260709 12:20:53.925581 15498 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:53.944911  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 58079.2us
I20260709 12:20:53.945757  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:53.952301  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:53.957877  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 34 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:53.958673  6506 raft_consensus.cc:3060] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:20:53.959013  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:53.957823  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:53.932536 15419 consensus_queue.cc:1059] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.023s
W20260709 12:20:53.967854  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 144308us
I20260709 12:20:53.943259 15447 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:53.950431 15440 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:53.923127 15195 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:53.973701 15422 raft_consensus.cc:2804] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:20:53.976706 15422 raft_consensus.cc:697] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 22 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:53.977741  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:53.978120 15516 log.cc:927] Time spent T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 12:20:53.977959 15422 consensus_queue.cc:237] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:53.979305 15515 log.cc:927] Time spent T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:20:53.984828 14708 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:53.988458  6548 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:53.990178 15207 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 12:20:53.995611  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 62926.5us
I20260709 12:20:53.997381  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:53.998632  6550 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:54.006183  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:53.992868 15447 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:54.018091  6548 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:54.015691  6473 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:54.038265  6473 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:54.039736 15484 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:20:54.040612 14926 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:54.040791 15484 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:54.041692 15484 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:54.020954 15305 consensus_queue.cc:1048] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.030s
I20260709 12:20:54.003664 15473 consensus_queue.cc:1059] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 12:20:54.048353 15473 raft_consensus.cc:3055] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:54.048825 15473 raft_consensus.cc:740] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:20:54.105799  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 76095.2us
I20260709 12:20:54.060653  6548 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:54.108426  6584 raft_consensus.cc:1240] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Rejecting Update request from peer 20a202c76244485fbc8f1610473db8b9 for earlier term 25. Current term is 26. Ops: [25.19-25.19]
I20260709 12:20:54.048974  6473 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:54.067473  6655 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 34.
I20260709 12:20:54.093734  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:54.110298  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 49782.7us
I20260709 12:20:54.115477 14708 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:20:54.115912 14708 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 30 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:54.116285 14708 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:20:54.101784  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:54.117786 15414 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:54.084947  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:54.024799 15434 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:54.120117  6550 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:54.057301 15447 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:54.054013 15442 raft_consensus.cc:3055] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:54.084056 15484 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:54.109464 15305 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:20:54.132896  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:54.032811 15195 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:54.054481  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:54.143167  6548 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:54.140854  6622 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:20:54.134160 15423 raft_consensus.cc:2804] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:20:54.129629 15442 raft_consensus.cc:740] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:54.124261 15434 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:54.157855 15410 consensus_queue.cc:1059] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.017s
I20260709 12:20:54.156755 15414 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:20:54.172243 15510 log.cc:927] Time spent T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.334s	user 0.004s	sys 0.000s
I20260709 12:20:54.173149 15442 consensus_queue.cc:260] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:54.174458 15442 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:54.185144  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:54.191056  6475 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:54.239702  6475 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:54.186414 15434 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:20:54.246927  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 97270.2us
W20260709 12:20:54.248595  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 199871us
I20260709 12:20:54.242218  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:54.271428  6506 raft_consensus.cc:2468] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 34.
I20260709 12:20:54.236865 15423 raft_consensus.cc:697] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 34 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:20:54.283782  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 132285us
I20260709 12:20:54.291484 15491 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:54.292217 15307 raft_consensus.cc:3055] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:20:54.292287 15289 raft_consensus.cc:2804] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:20:54.292750 15307 raft_consensus.cc:740] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:54.293342 15289 raft_consensus.cc:697] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:54.293828 15307 consensus_queue.cc:260] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:54.294938 15307 raft_consensus.cc:3060] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:20:54.294777 15289 consensus_queue.cc:237] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:54.204531 15473 consensus_queue.cc:260] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:54.305328 15473 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:54.308583  6473 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:54.310185  6473 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:54.311657  6473 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:54.312623 15484 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:20:54.313398 15484 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:54.314280 15484 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:20:54.316823 15482 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:54.318466  6580 raft_consensus.cc:2955] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Committing config change with OpId 20.11: config changed from index 4 to 11, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:20:54.320945  6395 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.344s	user 0.008s	sys 0.004s
W20260709 12:20:54.325452  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 195769us
I20260709 12:20:54.325475  6548 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:54.334513  6548 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:54.341806 15533 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:54.347194 15423 consensus_queue.cc:237] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 32.17, Last appended by leader: 17, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:54.348435  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:54.353569 15529 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:54.354369 15529 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:54.354847 15529 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:54.389276  6622 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:54.393561 14708 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:54.414935  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 103373us
I20260709 12:20:54.520232  6473 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:54.520730 14708 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 31 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:54.417677 15536 raft_consensus.cc:2804] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:20:54.568233 15536 raft_consensus.cc:697] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 26 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:20:54.568559  6621 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:54.569473 15536 consensus_queue.cc:237] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:54.398515 15533 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:54.578299 15533 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:54.579408 15473 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:20:54.429370 15528 log.cc:927] Time spent T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.152s	user 0.002s	sys 0.000s
I20260709 12:20:54.430290  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:54.581189 15498 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:20:54.433517  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 75204.8us
I20260709 12:20:54.581697 15498 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:54.583818 15498 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:54.437265 15529 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:54.440889  7268 raft_consensus.cc:1240] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 22 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 21. Current term is 22. Ops: []
I20260709 12:20:54.469553 15434 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:54.521718  6473 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:54.416802 15278 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:54.522181 15484 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:54.523221  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:54.524096 15429 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:54.543160  6656 raft_consensus.cc:1275] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:20:54.579967 15473 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:54.588092  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:54.589653  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:54.586138  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:54.602156  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:54.603201 15278 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:54.603951 15537 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:54.604916 15429 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:54.605309 15531 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:54.605636 15278 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:54.606072 15531 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:54.607600 15480 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:20:54.607944  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 86331.9us
I20260709 12:20:54.586455 15447 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:54.609786  6473 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:54.610046 15447 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:54.611213  6473 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:54.595350 15484 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:54.613456 15446 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:54.614063 15484 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:54.614365 15446 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:54.616086 15538 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:20:54.616882  6584 raft_consensus.cc:2393] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 21: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:54.619755  6473 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:54.646394  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:54.620478 15446 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:54.620851 15533 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:54.622047 15537 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:20:54.584283 15498 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:54.631951  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:54.638309  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:54.638840 15307 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:54.654479 15498 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:54.656025 15410 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:54.656637 15410 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:54.657339 15498 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:54.658998 15410 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:54.639591  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 47242.7us
I20260709 12:20:54.639843  6622 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:54.644634  6582 raft_consensus.cc:1275] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:20:54.662472 15307 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:54.664441 15484 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:54.644879 15531 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:54.648159  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:54.668541  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 48922.3us
I20260709 12:20:54.650808  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:54.669413  7265 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 13.
I20260709 12:20:54.669648  6473 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:54.670814 15537 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:20:54.671425 15538 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:20:54.672768  6473 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:54.674485  6473 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:54.674476 15538 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:54.678395  6580 raft_consensus.cc:1240] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 12:20:54.551026  6395 maintenance_manager.cc:643] P 39c4be726d3d495181bfe7425c8c0bd8: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.842s	user 0.045s	sys 0.010s Metrics: {"bytes_written":5874,"delete_count":0,"lbm_write_time_us":1177,"lbm_writes_lt_1ms":6,"reinsert_count":0,"thread_start_us":130096,"threads_started":1,"update_count":226,"wal-append.queue_time_us":208427}
I20260709 12:20:54.682456  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:54.689422  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:54.691380  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:54.691992  6396 maintenance_manager.cc:419] P 39c4be726d3d495181bfe7425c8c0bd8: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.257270
I20260709 12:20:54.692853 15307 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:54.693195 15531 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:54.695194  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:54.695914  6575 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 27.
I20260709 12:20:54.697700  6503 raft_consensus.cc:2393] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 27: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:54.696044 15531 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:54.678089 15255 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:20:54.700207  6548 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:54.702482 15497 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:20:54.720201  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:54.721578 15369 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:54.727469 15369 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:54.728048 15369 raft_consensus.cc:3060] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:54.729504 15429 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:20:54.731709  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 57335.9us
I20260709 12:20:54.709462 15531 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:54.736604  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:54.736716  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:54.749719  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:54.750845  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:54.752806  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:54.755424  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:54.755738  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:54.757149  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:54.757730  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:54.758972  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:54.759115 15473 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:54.761925  6549 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:54.762511 15429 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:54.763217 15447 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:54.763058 15429 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:54.763887 15446 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:20:54.764564 15446 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:54.765457 15429 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:54.766827  6549 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:54.772850  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:54.773257  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:54.776254  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:54.781616  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:54.784754  6501 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 31 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 30. Current term is 31. Ops: []
I20260709 12:20:54.784520 15465 consensus_queue.cc:1048] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 12:20:54.789234  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:54.766036 15429 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:20:54.785152 15531 raft_consensus.cc:2804] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:20:54.792209 15531 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:54.792639 15531 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:54.797134 15446 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:54.801419 15536 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:54.802037 15536 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:54.802318  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:54.804430 15531 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:54.806875  6576 raft_consensus.cc:2393] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 14: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:20:54.807837  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:54.808849 15536 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:54.810819 15531 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:54.811573  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:54.812269  7272 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:54.813326  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:54.817488  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:54.826820 15369 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:54.834385  6548 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:54.835747  6548 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:54.837932 15369 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:54.841794  6622 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:54.843467 15429 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:54.844844 15535 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:54.849205  6548 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:54.849712 15434 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:54.850383 15434 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:54.850608 15531 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:54.851956  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:54.853772 15484 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:54.859491  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:54.860523  6506 raft_consensus.cc:2393] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 27: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:54.860039  6405 catalog_manager.cc:5697] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:54.861480 15289 consensus_queue.cc:1059] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 30.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:20:54.862564 15289 raft_consensus.cc:3055] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:20:54.862954 15289 raft_consensus.cc:740] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:54.863730  6405 catalog_manager.cc:5697] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 18 to 20, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 20 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:54.866670  6405 catalog_manager.cc:5697] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 19 to 21, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 21 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:54.864019 15289 consensus_queue.cc:260] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 30.16, Last appended by leader: 16, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:54.869756 15289 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:20:54.869923  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:54.869247  6405 catalog_manager.cc:5697] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 13 to 18. New cstate: current_term: 18 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:54.871032  6502 raft_consensus.cc:2393] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 25: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:54.885237  6405 catalog_manager.cc:5697] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:54.887216 15535 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:54.888500  7272 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 14.
I20260709 12:20:54.890136  6549 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:54.891148 15531 raft_consensus.cc:2804] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:20:54.891621 15531 raft_consensus.cc:697] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:20:54.892402  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 77240.9us
I20260709 12:20:54.892522 15531 consensus_queue.cc:237] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:54.893457  7238 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:54.894526 15481 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:20:54.850646  6548 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:54.900642  6656 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
W20260709 12:20:54.901463  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 50896us
I20260709 12:20:54.901352  6580 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 28.
I20260709 12:20:54.902350  6548 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:54.906697  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:54.906592  6548 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:54.912262  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:54.917343  6548 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:54.917804  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:54.904580 15473 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:20:54.915467  6401 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57888 (request call id 218) took 93590 ms (client timeout 15000 ms). Trace:
I20260709 12:20:54.905710 15446 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:20:54.922111  6401 rpcz_store.cc:269] 0709 12:19:21.297220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:21.300563 (+  3343us) service_pool.cc:224] Handling call
0709 12:20:54.887049 (+93586486us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65569,"reported_tablets":52}
I20260709 12:20:54.925175  6580 raft_consensus.cc:1240] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:54.921663  6550 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:54.923736  6501 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 28.
I20260709 12:20:54.914752 15414 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:20:54.927942 15533 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:54.939666 15497 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:20:54.975194  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 53612.8us
W20260709 12:20:54.984925  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 58491.3us
I20260709 12:20:54.990669  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:54.978639 15497 raft_consensus.cc:697] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:20:54.999191 15546 log.cc:927] Time spent T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:20:55.003425  6582 raft_consensus.cc:2468] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 27.
I20260709 12:20:54.986032 15278 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:55.021301  6473 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:55.024693 15537 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:55.023080  6473 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:55.027408  6473 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:55.028718 15403 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:20:55.006697 15278 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:55.032944  6577 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 15. (term mismatch)
I20260709 12:20:55.041137  6548 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:55.005699  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:55.050876  6579 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 27 index: 21. (index mismatch)
I20260709 12:20:55.063750  6651 raft_consensus.cc:1240] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 20. Current term is 21. Ops: []
I20260709 12:20:55.066105  6581 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 25.
I20260709 12:20:55.065712  6577 pending_rounds.cc:85] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 14
I20260709 12:20:55.072566  6577 pending_rounds.cc:107] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 25.15
I20260709 12:20:55.072913  6577 raft_consensus.cc:2889] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:55.074446  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:55.071398  6622 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:20:55.086825  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 45776.7us
I20260709 12:20:55.087595  6548 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:55.092866  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:55.093811 15392 raft_consensus.cc:2804] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:20:55.110054  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
W20260709 12:20:55.137410  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 44612.6us
I20260709 12:20:55.095163 15255 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:20:55.095750 15305 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:20:55.143430  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:55.098029 15498 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:20:55.095539 15403 raft_consensus.cc:697] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 28 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:55.095350 15554 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:20:55.144644 15550 log.cc:927] Time spent T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:20:55.156666  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 129392us
I20260709 12:20:55.146221 15392 raft_consensus.cc:697] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 27 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:55.152922 15498 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:55.204386  6584 raft_consensus.cc:1240] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14. Ops: []
W20260709 12:20:55.204124  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 113441us
I20260709 12:20:55.187913 15116 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:20:55.220687  7265 raft_consensus.cc:1240] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:20:55.221918  6651 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 34 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 30 index: 16. Preceding OpId from leader: term: 34 index: 18. (index mismatch)
I20260709 12:20:55.201160 15498 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:20:55.149717 15497 consensus_queue.cc:237] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:55.223384 15116 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:55.218932 15419 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:55.210170 15484 consensus_queue.cc:1059] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 12:20:55.164511 15554 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:55.284409  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 77800.9us
I20260709 12:20:55.234427 15446 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.030s
I20260709 12:20:55.253294 15116 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:55.292277 15392 consensus_queue.cc:237] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.13, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:55.298079 15419 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:55.292536 15536 consensus_queue.cc:1059] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 12:20:55.302352 15484 raft_consensus.cc:3055] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:55.313248 15484 raft_consensus.cc:740] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:55.313861 15419 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:55.314143 15484 consensus_queue.cc:260] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:55.315411 15484 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:55.320629  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:55.322995  6655 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 19.
I20260709 12:20:55.306512 15554 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:55.326290  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:20:55.345923  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 60287.4us
I20260709 12:20:55.324389 15403 consensus_queue.cc:237] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:55.322114 15536 raft_consensus.cc:3055] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:20:55.354346 14708 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
W20260709 12:20:55.354280  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 70832.5us
I20260709 12:20:55.296588 15255 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.056s
I20260709 12:20:55.365584  7240 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:55.373566  6473 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:55.375761  6502 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 34 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 30 index: 16. Preceding OpId from leader: term: 34 index: 18. (index mismatch)
I20260709 12:20:55.376725 15491 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:20:55.379105 15530 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:20:55.379617 15530 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:55.379994 15530 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:55.384284  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:55.356465 15498 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:55.398108 15554 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:20:55.356659 15536 raft_consensus.cc:740] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:55.400511 15536 consensus_queue.cc:260] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:55.401736 15536 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:20:55.403689 15116 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:55.420511  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:55.423266  6580 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 20.
W20260709 12:20:55.440578  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 67082.5us
I20260709 12:20:55.441783  6473 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:55.404879 15278 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:55.453001 15498 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:55.454685  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:55.455343  6651 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Advancing to term 29
I20260709 12:20:55.472890 15530 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:55.473861 15116 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:55.475778  6502 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
new_leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:58944
I20260709 12:20:55.476668  6502 raft_consensus.cc:606] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 27 LEADER]: Received request to transfer leadership to TS 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:55.480528  6656 raft_consensus.cc:1275] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 12:20:55.480958  6656 pending_rounds.cc:85] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4: Aborting all ops after (but not including) 11
I20260709 12:20:55.481189  6656 pending_rounds.cc:107] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 12:20:55.481428  6656 raft_consensus.cc:2889] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:20:55.482571  6624 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:55.496652 15571 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:20:55.512445  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
W20260709 12:20:55.506706  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 268768us
I20260709 12:20:55.486052 15530 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:55.525897  6548 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:55.526456  6501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:55.529671  6508 raft_consensus.cc:1240] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 25. Current term is 26. Ops: []
W20260709 12:20:55.531452  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 100983us
W20260709 12:20:55.524698  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 93721.1us
I20260709 12:20:55.515029 15447 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:55.534972  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:55.537678  6653 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:20:55.545038  6508 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:20:55.564410  7265 raft_consensus.cc:1275] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:20:55.549906  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:55.566071  6506 raft_consensus.cc:2393] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 25: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:55.566947  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:55.567606  6579 raft_consensus.cc:3060] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:55.574590  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:55.575407  6580 raft_consensus.cc:2368] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 28. Current term is 29.
I20260709 12:20:55.597034  6653 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 21.
I20260709 12:20:55.598359  7240 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:55.599428 15481 raft_consensus.cc:2804] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:55.599247 15535 consensus_queue.cc:1059] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 12:20:55.600045  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 53991.8us
I20260709 12:20:55.600977 15561 raft_consensus.cc:3055] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:55.601365 15561 raft_consensus.cc:740] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:55.601217 15505 consensus_queue.cc:1048] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:20:55.602195 15561 consensus_queue.cc:260] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:55.603431 15561 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:55.604619 15521 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:55.579468 15531 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:20:55.627422  6579 raft_consensus.cc:2468] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 21.
W20260709 12:20:55.628923  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 58012.9us
I20260709 12:20:55.640759  6500 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:58944
I20260709 12:20:55.548810 15447 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:55.649031  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:55.600041 15481 raft_consensus.cc:697] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 21 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
W20260709 12:20:55.654152  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 70415us
I20260709 12:20:55.641674  6500 raft_consensus.cc:606] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 23 LEADER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:55.676683  6622 leader_election.cc:400] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 28. Current term is 29.
I20260709 12:20:55.677107  6622 leader_election.cc:403] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:55.677884 15447 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:55.678817  6651 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 29.
I20260709 12:20:55.682085 15369 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Leader pre-election lost for term 28. Reason: Vote denied by peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101) with higher term. Message: Invalid argument: T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 for earlier term 28. Current term is 29.
I20260709 12:20:55.677182  6578 raft_consensus.cc:1275] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:20:55.681041 15481 consensus_queue.cc:237] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:55.687775  6550 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:55.695152  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:55.696168  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:55.705662  6548 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:55.662122 15531 consensus_queue.cc:1048] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 12:20:55.700057 15305 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:20:55.727280  6504 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:58944
I20260709 12:20:55.729542  6500 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 26 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 12:20:55.717854 15414 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:55.728052  6504 raft_consensus.cc:606] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:55.724565 15571 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:55.743774  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 49853.8us
I20260709 12:20:55.736615 15305 raft_consensus.cc:697] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:55.742023 15423 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:55.745257 15558 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 12:20:55.780054 15423 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:55.786365  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:55.787768  7265 raft_consensus.cc:2410] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 27 because replica has last-logged OpId of term: 25 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 18.
I20260709 12:20:55.747205 15571 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:55.806254  6653 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:20:55.800205  6577 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 25.
I20260709 12:20:55.783833 15423 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:55.863651  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:55.863202  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:55.870337  6503 raft_consensus.cc:2410] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for term 27 because replica has last-logged OpId of term: 25 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 18.
I20260709 12:20:55.873375 15586 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:55.874168 15586 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:55.882481  6548 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:55.892194  6577 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:20:55.878746 15307 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:20:55.894806 15307 raft_consensus.cc:697] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 25 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:55.895939 15307 consensus_queue.cc:237] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:55.897755  6577 raft_consensus.cc:606] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:55.898298  6577 raft_consensus.cc:614] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:55.880705 15429 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:55.876947 15414 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 12:20:55.935729 15429 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:55.922030 15586 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:55.801740 15305 consensus_queue.cc:237] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:55.939919 15429 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:55.940693 15553 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:55.930873 15571 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:55.944069  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:55.945034  6582 raft_consensus.cc:2410] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 27 because replica has last-logged OpId of term: 25 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 14.
I20260709 12:20:55.951975  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:55.952687  6653 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 26.
I20260709 12:20:55.953924  6550 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:20:55.962898 15579 log.cc:927] Time spent T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.261s	user 0.005s	sys 0.000s
I20260709 12:20:55.965936 15423 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:56.011170  6579 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 26 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 12:20:55.959913 15536 consensus_queue.cc:1048] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.042s
I20260709 12:20:56.029282 15537 raft_consensus.cc:993] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65: : Instructing follower 496c7c3792cd4587b4dbeba435f4be1e to start an election
I20260709 12:20:56.029757 15537 raft_consensus.cc:1081] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 27 LEADER]: Signalling peer 496c7c3792cd4587b4dbeba435f4be1e to start an election
I20260709 12:20:55.987210 15423 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:56.036423 15423 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:56.002669  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:56.039279  6578 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 26.
I20260709 12:20:56.044615  6582 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:56116
I20260709 12:20:56.045262  6582 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:56.045621  6582 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:56.053843  7238 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:56.065094  6582 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:56.066689  6582 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:56.067643  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: true dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:56.095504  7272 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:56.059669 15582 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:56.101697  6655 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 28. Current term is 29. Ops: []
I20260709 12:20:56.081110  6577 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 28. Current term is 29. Ops: []
I20260709 12:20:56.087109 15536 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:56.092816  6503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:56.092950  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:56.100684  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:56.116042  6501 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
new_leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:58944
I20260709 12:20:56.118052  6501 raft_consensus.cc:606] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Received request to transfer leadership to TS 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:56.127406  6656 raft_consensus.cc:2410] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 for term 27 because replica has last-logged OpId of term: 25 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 14.
I20260709 12:20:56.117442  6504 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 25.
I20260709 12:20:56.116853  6503 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 26.
I20260709 12:20:56.115437 15582 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:56.115507 15423 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:56.155362 15582 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:56.183151 15403 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:56.183811 15403 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:56.188302  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:56.195343 15255 raft_consensus.cc:993] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65: : Instructing follower f2b06aebe97249a4877d798b8118ece4 to start an election
I20260709 12:20:56.196007 15255 raft_consensus.cc:1081] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 23 LEADER]: Signalling peer f2b06aebe97249a4877d798b8118ece4 to start an election
I20260709 12:20:56.211786 15429 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:56.212515 15429 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:56.214987 15571 consensus_queue.cc:1059] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:20:56.215823 15571 consensus_queue.cc:1243] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer 496c7c3792cd4587b4dbeba435f4be1e log is divergent from this leader: its last log entry 29.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:20:56.216369 15571 raft_consensus.cc:3055] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:20:56.216696 15423 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:56.216709 15571 raft_consensus.cc:740] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:56.229998  7272 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 28.
I20260709 12:20:56.237341  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:56.238174  6653 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 31.
I20260709 12:20:56.242244  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:56.242942  6653 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:56.215003 15429 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:56.261333  6550 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:56.282269 15403 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:56.285127  6658 raft_consensus.cc:1275] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:20:56.287685 15497 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:20:56.287598 15481 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:56.288172 15497 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:56.288606 15497 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:20:56.292469 15571 consensus_queue.cc:260] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.19, Last appended by leader: 19, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:56.293931 15571 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:20:56.308658  6549 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:56.309667 15447 raft_consensus.cc:2804] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:20:56.310222 15447 raft_consensus.cc:697] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:56.311142 15447 consensus_queue.cc:237] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 27.21, Last appended by leader: 21, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:56.323237  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:56.328310  6653 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 27.
W20260709 12:20:56.335651  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 179785us
I20260709 12:20:56.336862  6475 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:56.337666  6550 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:56.350586  6474 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:56.375130 14708 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:56.382330  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:20:56.386055 15591 raft_consensus.cc:993] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65: : Instructing follower 496c7c3792cd4587b4dbeba435f4be1e to start an election
I20260709 12:20:56.386687  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:56.379436 15484 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:56.398523  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:56.399739  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:56.352897 15498 raft_consensus.cc:2804] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:20:56.412227  6405 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57914 (request call id 194) took 57139 ms (client timeout 15000 ms). Trace:
I20260709 12:20:56.415007  6655 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:58972
I20260709 12:20:56.415668  6655 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:56.416062  6655 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:20:56.412858  6405 rpcz_store.cc:269] 0709 12:19:59.272908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:19:59.273460 (+   552us) service_pool.cc:224] Handling call
0709 12:20:00.152448 (+878988us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b540008acd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df3090 00007f2875e91f3a 00007f287bc47e9c 00007f287bd4604e 00007f287a2072a4 00007f287a207232 00007f287a207161 00007f287a2070dc 00007f287a206392 00007f28741c448c 00007f28741c38c6
0709 12:20:56.412127 (+56259679us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:55.028085 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:55.038799 (+ 10714us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:55.038854 (+    55us) write_op.cc:435] Acquired schema lock
0709 12:20:55.038880 (+    26us) tablet.cc:662] Decoding operations
0709 12:20:55.039625 (+   745us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:55.039725 (+   100us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:55.039751 (+    26us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:20:55.066664 (+ 26913us) tablet.cc:701] Row locks acquired
0709 12:20:55.097481 (+ 30817us) write_op.cc:260] PREPARE: finished
0709 12:20:55.152256 (+ 54775us) write_op.cc:270] Start()
0709 12:20:55.156272 (+  4016us) write_op.cc:276] Timestamp: P: 1783599655142718 usec, L: 0
0709 12:20:55.156303 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 12:20:55.244288 (+ 87985us) log.cc:844] Serialized 2034 byte log entry
0709 12:20:55.369998 (+125710us) op_driver.cc:464] REPLICATION: finished
0709 12:20:55.554879 (+184881us) write_op.cc:301] APPLY: starting
0709 12:20:55.554990 (+   111us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:55.557910 (+  2920us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:55.558019 (+   109us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:55.679695 (+121676us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:55.680040 (+   345us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=10,mrs_lookups=0
0709 12:20:55.680093 (+    53us) write_op.cc:312] APPLY: finished
0709 12:20:55.681645 (+  1552us) log.cc:844] Serialized 57 byte log entry
0709 12:20:55.681875 (+   230us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:55.681992 (+   117us) write_op.cc:454] Released schema lock
0709 12:20:55.682056 (+    64us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:55.682508 (+   452us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:55.682541 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":148,"reported_tablets":52,"spinlock_wait_cycles":546147584,"child_traces":[["op",{"apply.queue_time_us":159731,"apply.run_cpu_time_us":9218,"apply.run_wall_time_us":127689,"cfile_cache_hit":22,"cfile_cache_hit_bytes":24987,"num_ops":5,"prepare.queue_time_us":103042,"prepare.run_cpu_time_us":4926,"prepare.run_wall_time_us":279292,"replication_time_us":204452,"thread_start_us":139854,"threads_started":2}]]}
I20260709 12:20:56.440454  6578 raft_consensus.cc:1275] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:20:56.454361 15410 consensus_queue.cc:1048] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:56.436147 15498 raft_consensus.cc:697] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:56.463780 15498 consensus_queue.cc:237] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:56.466094 15582 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:56.475847 15566 raft_consensus.cc:1081] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Signalling peer 496c7c3792cd4587b4dbeba435f4be1e to start an election
I20260709 12:20:56.476112  6505 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:20:56.485178  6655 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:56.495302  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:56.496002  6504 raft_consensus.cc:3060] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:56.510864  6500 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:20:56.513015  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:56.514973  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: true dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:56.519351  6402 catalog_manager.cc:5697] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:56.521317  6402 catalog_manager.cc:5697] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
W20260709 12:20:56.525136  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 53791.3us
I20260709 12:20:56.528535  6582 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:56116
I20260709 12:20:56.529096  6582 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:56.529466  6582 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:56.490641 15497 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:56.543087 15497 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 31 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:56.551740  6504 raft_consensus.cc:2468] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 27.
I20260709 12:20:56.555419 15582 leader_election.cc:290] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 27 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:56.557691  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:56.558279  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:56.558609  6506 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:20:56.558905  6584 raft_consensus.cc:3060] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:56.516970 15536 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 12:20:56.567569  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:56.569842  6584 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 27.
I20260709 12:20:56.554852  6582 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:56.577076 15565 raft_consensus.cc:993] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65: : Instructing follower f2b06aebe97249a4877d798b8118ece4 to start an election
I20260709 12:20:56.579788 14708 raft_consensus.cc:1081] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Signalling peer f2b06aebe97249a4877d798b8118ece4 to start an election
W20260709 12:20:56.581010 15599 log.cc:927] Time spent T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:20:56.581425  6582 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:56.582710  6502 raft_consensus.cc:2393] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 31: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:56.589847  6507 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 29 index: 19. (term mismatch)
I20260709 12:20:56.590363  6507 pending_rounds.cc:85] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65: Aborting all ops after (but not including) 18
I20260709 12:20:56.590610  6507 pending_rounds.cc:107] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65: Aborting uncommitted NO_OP operation due to leader change: 28.19
I20260709 12:20:56.590873  6507 raft_consensus.cc:2889] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:56.595650  6658 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
W20260709 12:20:56.598492  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 89446.7us
I20260709 12:20:56.604197  6658 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:58972
I20260709 12:20:56.604866  6658 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:56.605276  6658 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:56.607115 15552 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:56.523690  6402 catalog_manager.cc:5697] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 20 to 24, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 24 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:56.614166  6402 catalog_manager.cc:5697] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 19 to 25, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 25 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:56.617125  7238 leader_election.cc:304] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:56.617223  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: true dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:56.620440  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:56.647132  7265 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:56.647439  6504 raft_consensus.cc:2393] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 31: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:20:56.620692  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: true dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:56.623050  6580 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 LEADER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 27. Current term is 28. Ops: []
I20260709 12:20:56.626452 15554 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:20:56.650753  6502 raft_consensus.cc:3055] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:20:56.651079  6502 raft_consensus.cc:740] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:56.651858  6502 consensus_queue.cc:260] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:56.652935  6502 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:56.654604  6506 raft_consensus.cc:2468] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 27.
I20260709 12:20:56.625339  6658 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:56.657965  6658 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:56.658933  6578 raft_consensus.cc:1240] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 19. Current term is 20. Ops: []
I20260709 12:20:56.662792 15600 consensus_queue.cc:1059] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 27.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:20:56.663527 15600 consensus_queue.cc:1243] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer 496c7c3792cd4587b4dbeba435f4be1e log is divergent from this leader: its last log entry 28.22 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 21
I20260709 12:20:56.664628 15600 consensus_queue.cc:1059] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:20:56.665320  6578 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:20:56.665644 15591 raft_consensus.cc:3055] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:20:56.666033 15591 raft_consensus.cc:740] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:56.666776 15591 consensus_queue.cc:260] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:56.668570  6502 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 24.
I20260709 12:20:56.669656  7265 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
I20260709 12:20:56.677716  6651 raft_consensus.cc:1240] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 20. Current term is 21. Ops: []
W20260709 12:20:56.678208  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 61157.6us
I20260709 12:20:56.678373 15582 raft_consensus.cc:2804] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:20:56.682619  6658 raft_consensus.cc:1240] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 23. Current term is 24. Ops: []
I20260709 12:20:56.647746  6402 catalog_manager.cc:5697] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 18 to 19, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 19 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:20:56.695438  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: true dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:56.703418  6402 catalog_manager.cc:5697] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 26 to 28. New cstate: current_term: 28 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:56.705173  6402 catalog_manager.cc:5697] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 22 to 23, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 23 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:20:56.696156  6656 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 31.
I20260709 12:20:56.707310  6550 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:20:56.706769  6549 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:20:56.708549  6402 catalog_manager.cc:5697] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 21 to 22, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 22 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:56.709653 14708 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:56.710258 14708 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:56.710624  6402 catalog_manager.cc:5697] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 27 to 29, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 29 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:56.712401  6402 catalog_manager.cc:5697] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 24 to 28, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 28 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:56.714383 15591 raft_consensus.cc:3060] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:20:56.706043 15582 raft_consensus.cc:697] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 27 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:20:56.718109  6622 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:56.719036 15569 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:20:56.718540 15582 consensus_queue.cc:237] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:56.719581 15569 raft_consensus.cc:697] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 24 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:56.718381  6402 catalog_manager.cc:5697] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 19 to 27, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 27 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:56.721103  6402 catalog_manager.cc:5697] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 24 to 25, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 25 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:56.718128  6505 raft_consensus.cc:3055] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:20:56.725631  6505 raft_consensus.cc:740] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:56.739096  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: true dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:56.739869  6504 raft_consensus.cc:3055] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:20:56.740226  6504 raft_consensus.cc:740] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:56.723253 15498 raft_consensus.cc:2804] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:20:56.726730  6402 catalog_manager.cc:5697] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 16 to 27. New cstate: current_term: 27 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:20:56.755136  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: true dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:56.755215 15569 consensus_queue.cc:237] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:56.756028  6583 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:56.725200 15305 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:20:56.770689 15305 raft_consensus.cc:697] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 31 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:56.771688  6655 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:56.757829  6402 catalog_manager.cc:5697] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 20 to 26, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 26 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:56.760599 15457 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.012s
I20260709 12:20:56.786947  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: true dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:20:56.787876  7272 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:20:56.790517  6402 catalog_manager.cc:5697] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 21 to 22, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 22 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:56.772135  6505 consensus_queue.cc:260] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 27.21, Last appended by leader: 21, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:56.773146  6504 consensus_queue.cc:260] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:56.783823 15498 raft_consensus.cc:697] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:20:56.796801 15598 log.cc:927] Time spent T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.348s	user 0.005s	sys 0.007s
I20260709 12:20:56.805624  6504 raft_consensus.cc:3060] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:20:56.813794 15498 consensus_queue.cc:237] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:56.818360  7265 consensus_queue.cc:237] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:56.827629 15305 consensus_queue.cc:237] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 30.16, Last appended by leader: 16, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:56.831749  6583 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:20:56.833055 14708 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:56.797950  6402 catalog_manager.cc:5697] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 24 to 27, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 27 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:56.838570  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:56.839316  6579 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 25.
I20260709 12:20:56.838459  6402 catalog_manager.cc:5697] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 18 to 28, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 28 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:56.840704  6473 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:56.852139  7272 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 24.
I20260709 12:20:56.850700  6402 catalog_manager.cc:5697] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:56.857445  6402 catalog_manager.cc:5697] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 21 to 26, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 26 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:56.860056  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:56.859872  6402 catalog_manager.cc:5697] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 18 to 28, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 28 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:56.862483  6402 catalog_manager.cc:5697] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:56.864725  6402 catalog_manager.cc:5697] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 18 to 24, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 24 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:56.872705  7272 raft_consensus.cc:1240] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Rejecting Update request from peer e7554c39679a41bbbca4659b72ab5c65 for earlier term 27. Current term is 28. Ops: []
I20260709 12:20:56.858636 15565 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:20:56.874315 15565 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:56.874692 15565 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:20:56.883478  6402 catalog_manager.cc:5697] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 26 to 28, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 28 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
W20260709 12:20:56.886502  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 103735us
I20260709 12:20:56.888878  6402 catalog_manager.cc:5697] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 26 to 29, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 29 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:56.889508  6622 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:56.895117  6402 catalog_manager.cc:5697] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 22 to 25, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 25 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:20:56.896725  6402 catalog_manager.cc:5697] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 23 to 24, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 24 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:20:56.897099  6504 raft_consensus.cc:2468] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:20:56.900542  6402 catalog_manager.cc:5697] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 16 to 19, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 19 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:20:56.907052  6474 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:56.866482  6395 maintenance_manager.cc:643] P 39c4be726d3d495181bfe7425c8c0bd8: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 2.158s	user 0.110s	sys 0.062s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":26148,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":300429,"dirs.run_cpu_time_us":564,"dirs.run_wall_time_us":1770,"lbm_read_time_us":49797,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":22,"lbm_write_time_us":1927,"lbm_writes_lt_1ms":11,"peak_mem_usage":278083,"reinsert_count":0,"thread_start_us":350889,"threads_started":2,"update_count":226}
I20260709 12:20:56.919934  6500 raft_consensus.cc:2468] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
I20260709 12:20:56.922668 15521 raft_consensus.cc:2804] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:20:56.935261 15521 raft_consensus.cc:697] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:56.936267 15521 consensus_queue.cc:237] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:56.920267 15305 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.026s
I20260709 12:20:56.964931  6504 raft_consensus.cc:1275] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:20:56.966743 15497 consensus_queue.cc:1048] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:56.977327  6505 raft_consensus.cc:3060] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:20:57.017032 15591 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:57.029973 15591 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:57.034421  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:57.035873 15600 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:57.036505 15600 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:57.038766 15600 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:57.042215  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:57.042910  6655 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 26.
I20260709 12:20:57.043260  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:57.043874  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:57.044536  6649 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 31.
I20260709 12:20:57.048436  6475 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:57.051995  6475 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:57.048615 15591 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:57.054315 15600 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:57.071120 15600 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:57.071823 15600 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:57.056048 15484 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:20:57.086269  6655 raft_consensus.cc:1275] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
W20260709 12:20:57.087965  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 184175us
I20260709 12:20:57.091035  6655 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:20:57.093690 15446 consensus_queue.cc:1048] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.007s
I20260709 12:20:57.086527 15484 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:57.084806 15305 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:57.205186  7265 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 27 index: 21. Preceding OpId from leader: term: 28 index: 22. (index mismatch)
W20260709 12:20:57.209275  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 112092us
I20260709 12:20:57.150869 15484 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:20:57.180277 15565 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:57.230870 15446 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
W20260709 12:20:57.233110  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 139475us
W20260709 12:20:57.238193  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 46416.4us
I20260709 12:20:57.239784  6505 raft_consensus.cc:2468] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 28.
I20260709 12:20:57.241495  6507 raft_consensus.cc:1275] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 27 index: 21. Preceding OpId from leader: term: 28 index: 22. (index mismatch)
I20260709 12:20:57.244589  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:57.232074 15565 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:57.201272 15305 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:57.257324  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 369473us
W20260709 12:20:57.254185 15564 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 28 current term: 28 is higher.
W20260709 12:20:57.314584  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 98550.4us
W20260709 12:20:57.345752  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 84721.8us
W20260709 12:20:57.349717 15623 log.cc:927] Time spent T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 12:20:57.352097  6509 raft_consensus.cc:1275] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:20:57.352845 15600 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:57.354410 15484 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:57.354825 15600 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:57.310719 15542 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
W20260709 12:20:57.378340 15618 log.cc:927] Time spent T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.309s	user 0.000s	sys 0.001s
I20260709 12:20:57.379261  6506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:57.380133  6506 raft_consensus.cc:2368] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 26. Current term is 27.
I20260709 12:20:57.381054  6505 raft_consensus.cc:1275] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:20:57.381527  6507 raft_consensus.cc:1240] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 25. Current term is 26. Ops: []
I20260709 12:20:57.389134  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:57.389753  6649 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:57.391109 15305 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:57.397524  6649 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 27.
I20260709 12:20:57.416604  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:57.421192  6649 raft_consensus.cc:2368] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 26. Current term is 27.
I20260709 12:20:57.422358  6550 leader_election.cc:400] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 26. Current term is 27.
I20260709 12:20:57.422750  6550 leader_election.cc:403] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:20:57.423374 15305 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:20:57.419350  6475 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:57.432668 15628 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:20:57.433109 15621 log.cc:927] Time spent T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.235s	user 0.003s	sys 0.001s
I20260709 12:20:57.433326 15628 raft_consensus.cc:697] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 27 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:57.434430 15628 consensus_queue.cc:237] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:57.438795 15305 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 26. Current term is 27.
I20260709 12:20:57.456643 15484 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 32 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:57.456807 15569 consensus_queue.cc:1059] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:20:57.479986  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:57.481089  6656 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:20:57.485086 15614 raft_consensus.cc:3055] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:20:57.499986  6501 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
W20260709 12:20:57.504561  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 291386us
I20260709 12:20:57.491039 15614 raft_consensus.cc:740] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:20:57.508862 15614 consensus_queue.cc:260] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:57.519865  6656 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 32.
I20260709 12:20:57.494606 15410 consensus_queue.cc:1048] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 12:20:57.525840  6475 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:57.530767  7265 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:20:57.531149 15628 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:20:57.544732 15628 raft_consensus.cc:697] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 32 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:57.555740  6501 raft_consensus.cc:1275] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:20:57.560062 15423 consensus_queue.cc:1048] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:20:57.555984  6583 raft_consensus.cc:1275] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:20:57.562830 15521 consensus_queue.cc:1048] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:20:57.528734 15614 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:20:57.567698 15643 log.cc:927] Time spent T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8: Append to log took a long time: real 0.054s	user 0.002s	sys 0.000s
I20260709 12:20:57.545621 15628 consensus_queue.cc:237] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 30.16, Last appended by leader: 16, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:57.531594 15626 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.046s
I20260709 12:20:57.596381  6548 leader_election.cc:400] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Vote denied by peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643) with higher term. Message: Invalid argument: T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 26. Current term is 27.
I20260709 12:20:57.607926  6583 raft_consensus.cc:1275] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:20:57.591001 15536 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.018s
I20260709 12:20:57.635171  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:57.645283  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:57.655220  6579 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 27.
I20260709 12:20:57.615011 15633 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.022s
I20260709 12:20:57.663365  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:57.692551  6473 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:57.699971 15566 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:20:57.704282  7272 raft_consensus.cc:1275] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:20:57.707615  6509 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 32 LEADER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 31. Current term is 32. Ops: []
I20260709 12:20:57.739961 15531 consensus_queue.cc:1059] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.005s
I20260709 12:20:57.756876 15531 consensus_queue.cc:1243] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer e7554c39679a41bbbca4659b72ab5c65 log is divergent from this leader: its last log entry 32.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:20:57.757457 15531 raft_consensus.cc:3055] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:20:57.757820 15531 raft_consensus.cc:740] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:20:57.707741 15639 raft_consensus.cc:2955] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 17 LEADER]: Committing config change with OpId 17.13: config changed from index 3 to 13, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:20:57.758675 15531 consensus_queue.cc:260] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 31.17, Last appended by leader: 17, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:57.759991 15531 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:20:57.731382 15635 consensus_queue.cc:1048] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 17, Time since last communication: 0.031s
I20260709 12:20:57.723652 15429 consensus_queue.cc:1048] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:20:57.832162  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:57.840878 15639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:48498 (request call id 18) took 1059 ms. Trace:
W20260709 12:20:57.851132  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 179516us
W20260709 12:20:57.885401  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 158512us
W20260709 12:20:57.891536  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 185091us
I20260709 12:20:57.894529 15634 consensus_queue.cc:1048] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:20:57.899819  6649 raft_consensus.cc:1240] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 32 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 31. Current term is 32. Ops: []
I20260709 12:20:57.901382  6580 raft_consensus.cc:1275] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:20:57.841416 15639 rpcz_store.cc:276] 0709 12:20:56.781072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:56.781243 (+   171us) service_pool.cc:224] Handling call
0709 12:20:56.852009 (+ 70766us) log.cc:844] Serialized 480 byte log entry
0709 12:20:57.087726 (+235717us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b180034b6a0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d16c0a 00007f287c456219 00007f2872d9ea40 00007f2872d994e4 00007f2872d99455 00007f2875de2b0b 00007f2875e67b63 00007f2875e8f752 00007f2875e879fa
0709 12:20:57.436052 (+348326us) spinlock_profiling.cc:243] Waited 26.1 ms on lock 0x7b24002fcc18. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872c229c4 00007f2872c20acb 00007f2872ddbdc9 00007f2872ddba55 00007f2872ddafb0 00007f2875a51488 00007f2875a51be8 00007f2875a29398 00007f2875a2476a 00007f2875a23bb6
0709 12:20:57.840803 (+404751us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":188427,"raft.queue_time_us":187895,"raft.run_cpu_time_us":1256,"raft.run_wall_time_us":13888,"spinlock_wait_cycles":84407296,"thread_start_us":14299,"threads_started":1,"wal-append.queue_time_us":63739}
I20260709 12:20:57.934706  6649 raft_consensus.cc:2955] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index 3 to 13, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:20:57.962380 15661 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.017s
W20260709 12:20:57.976922  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 45582us
I20260709 12:20:58.005210  6393 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6f94b1cd286648a9b82d5abf69bb86d5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 15)
I20260709 12:20:57.989413  6507 raft_consensus.cc:2955] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index 3 to 13, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:20:58.022960  6649 raft_consensus.cc:1275] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:20:58.025931  6583 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:20:58.026667  6583 raft_consensus.cc:606] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 LEADER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:58.030586  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 125928us
W20260709 12:20:58.032996  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 119202us
I20260709 12:20:58.031154 15565 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.010s
W20260709 12:20:58.086802 15666 log.cc:927] Time spent T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 12:20:58.110044  6508 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
new_leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:58944
I20260709 12:20:58.110668  6508 raft_consensus.cc:606] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Received request to transfer leadership to TS 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:58.111012  6508 raft_consensus.cc:614] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:58.112723  6509 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:20:58.113867  6658 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 32 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 30 index: 16. Preceding OpId from leader: term: 32 index: 17. (index mismatch)
W20260709 12:20:58.151049  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 72425.9us
W20260709 12:20:58.168548  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 60868us
W20260709 12:20:58.170792 15668 log.cc:927] Time spent T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
W20260709 12:20:58.179700  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 62036.9us
W20260709 12:20:58.205439  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 90103.1us
I20260709 12:20:58.218135 15497 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:58.218930 15497 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:58.221838 15497 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:58.229205  7268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:20:58.231233  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 79331.2us
W20260709 12:20:58.259248  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 45510.5us
I20260709 12:20:58.274932  6577 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 32 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 31 index: 17. Preceding OpId from leader: term: 32 index: 17. (term mismatch)
I20260709 12:20:58.275390  6577 pending_rounds.cc:85] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting all ops after (but not including) 16
I20260709 12:20:58.275625  6577 pending_rounds.cc:107] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e: Aborting uncommitted NO_OP operation due to leader change: 31.17
I20260709 12:20:58.275869  6577 raft_consensus.cc:2889] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 32 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:58.285148 15656 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:20:58.297984  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:58.302598 14708 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:20:58.318554 15661 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.022s
W20260709 12:20:58.366430  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 54337.5us
I20260709 12:20:58.425909  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
W20260709 12:20:58.433122  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 113445us
W20260709 12:20:58.461233  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 111944us
I20260709 12:20:58.505741  6550 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:58.527806  6407 catalog_manager.cc:5697] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:20:58.529313 15630 raft_consensus.cc:993] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e: : Instructing follower f2b06aebe97249a4877d798b8118ece4 to start an election
I20260709 12:20:58.529922 15630 raft_consensus.cc:1081] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 LEADER]: Signalling peer f2b06aebe97249a4877d798b8118ece4 to start an election
W20260709 12:20:58.710026 15686 log.cc:927] Time spent T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.261s	user 0.001s	sys 0.000s
W20260709 12:20:58.711766  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 206100us
I20260709 12:20:58.712766 15429 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:20:58.745229 15685 log.cc:927] Time spent T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.234s	user 0.000s	sys 0.000s
W20260709 12:20:58.757035 15681 log.cc:927] Time spent T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
W20260709 12:20:58.767324  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 54752.5us
W20260709 12:20:58.771849  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 209614us
W20260709 12:20:58.783167  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 221632us
I20260709 12:20:58.788458  6508 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c27974d342ea4901aaa3946871539818"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
new_leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:58944
I20260709 12:20:58.789310  6508 raft_consensus.cc:606] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Received request to transfer leadership to TS 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:58.789785  6508 raft_consensus.cc:614] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:20:58.810431  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
W20260709 12:20:58.815657  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 246969us
W20260709 12:20:58.826851  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 319064us
I20260709 12:20:58.827910  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:58.828827  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:58.833607  6656 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:58980
I20260709 12:20:58.834362  6656 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:58.834856  6656 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:58.903481  6656 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:58.922509  6656 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:20:58.992704  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 102800us
W20260709 12:20:58.997309  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 73564.5us
I20260709 12:20:59.046595  6658 raft_consensus.cc:1240] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 24. Current term is 25. Ops: []
I20260709 12:20:59.054538  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: true dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:59.055333  6580 raft_consensus.cc:3055] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:20:59.055703  6580 raft_consensus.cc:740] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:20:59.061388  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 134854us
I20260709 12:20:59.069012  6580 consensus_queue.cc:260] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:59.199910  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 266619us
I20260709 12:20:59.201154  6580 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:59.206609  6402 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57900 (request call id 195) took 4911 ms. Trace:
I20260709 12:20:59.207140  6402 rpcz_store.cc:276] 0709 12:20:54.291221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:54.291454 (+   233us) service_pool.cc:224] Handling call
0709 12:20:59.200475 (+4909021us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:57.247171 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:57.380845 (+133674us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:57.380886 (+    41us) write_op.cc:435] Acquired schema lock
0709 12:20:57.380906 (+    20us) tablet.cc:662] Decoding operations
0709 12:20:57.382928 (+  2022us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:57.383026 (+    98us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:57.383054 (+    28us) tablet.cc:685] Acquiring locks for 27 operations
0709 12:20:57.419019 (+ 35965us) tablet.cc:701] Row locks acquired
0709 12:20:57.419048 (+    29us) write_op.cc:260] PREPARE: finished
0709 12:20:57.419205 (+   157us) write_op.cc:270] Start()
0709 12:20:57.419332 (+   127us) write_op.cc:276] Timestamp: P: 1783599657419154 usec, L: 0
0709 12:20:57.419356 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:20:57.420126 (+   770us) log.cc:844] Serialized 10357 byte log entry
0709 12:20:57.592702 (+172576us) op_driver.cc:464] REPLICATION: finished
0709 12:20:57.712495 (+119793us) write_op.cc:301] APPLY: starting
0709 12:20:57.712576 (+    81us) tablet.cc:1367] starting BulkCheckPresence
0709 12:20:57.863726 (+151150us) tablet.cc:1370] finished BulkCheckPresence
0709 12:20:57.898683 (+ 34957us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:20:58.338365 (+439682us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:20:58.338736 (+   371us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=54,key_file_lookups=54,delta_file_lookups=0,mrs_lookups=0
0709 12:20:58.338781 (+    45us) write_op.cc:312] APPLY: finished
0709 12:20:58.341387 (+  2606us) log.cc:844] Serialized 236 byte log entry
0709 12:20:58.341576 (+   189us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:58.360171 (+ 18595us) write_op.cc:454] Released schema lock
0709 12:20:58.377575 (+ 17404us) write_op.cc:341] FINISH: Updating metrics
0709 12:20:58.419446 (+ 41871us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:20:58.424425 (+  4979us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":158272,"reported_tablets":47,"spinlock_wait_cycles":497408,"child_traces":[["op",{"apply.queue_time_us":119514,"apply.run_cpu_time_us":41505,"apply.run_wall_time_us":685502,"cfile_cache_hit":110,"cfile_cache_hit_bytes":116771,"cfile_init":1,"lbm_read_time_us":577,"lbm_reads_lt_1ms":4,"num_ops":27,"prepare.queue_time_us":298170,"prepare.run_cpu_time_us":9549,"prepare.run_wall_time_us":262916,"raft.queue_time_us":23520,"raft.run_cpu_time_us":30439,"raft.run_wall_time_us":77263,"replication_time_us":173169,"thread_start_us":297099,"threads_started":4,"wal-append.queue_time_us":78712}]]}
I20260709 12:20:59.070170 15633 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:20:59.247308  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 101855us
I20260709 12:20:59.249918  6580 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 25.
W20260709 12:20:59.354449  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 288941us
I20260709 12:20:59.229743 15536 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:59.417044  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 61271.2us
W20260709 12:20:59.451547  6550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:59.246753 15633 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:59.470319  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 274735us
W20260709 12:20:59.457523 15674 log.cc:927] Time spent T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
I20260709 12:20:59.444882 15536 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:20:59.523977  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 275612us
W20260709 12:20:59.543975  6474 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.009s	sys 0.007s
W20260709 12:20:59.544485  6474 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.009s	sys 0.007s
W20260709 12:20:59.540139 15693 log.cc:927] Time spent T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.325s	user 0.001s	sys 0.000s
I20260709 12:20:59.669754 15701 raft_consensus.cc:493] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:59.678015 15633 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:59.702153 15714 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:59.702913 15714 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:59.704955 15696 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:59.705476 15696 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:59.707526 15696 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:59.675240 15701 raft_consensus.cc:515] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:59.733302  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:59.734251  6580 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 25.
I20260709 12:20:59.757696 15701 leader_election.cc:290] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:59.759910  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:59.761042  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:59.760756  6656 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 26.
I20260709 12:20:59.766938 15708 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:59.768811 15708 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:59.771389  6475 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:20:59.773708 15565 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:20:59.774190 15565 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:59.774592 15565 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:59.783586 15565 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:59.785986 15565 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:59.788069 15714 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:59.791608 15714 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:59.792304 15714 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:59.802227  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:20:59.803011  6576 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:20:59.808979  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:59.809351 15625 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:59.810221 15625 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:59.819650  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:59.819650  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:59.828449  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:20:59.828866  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:59.829071  6658 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:20:59.829844  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:59.832988  6576 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 27.
I20260709 12:20:59.837355  6473 leader_election.cc:304] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:20:59.843446  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 411558us
I20260709 12:20:59.772167 15708 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:59.839841  6473 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:20:59.846503  6475 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:59.853191  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:59.855880  6658 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 27.
I20260709 12:20:59.801321 15536 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:59.859259  6475 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:59.830489 15675 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:59.830299 15714 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:59.864457 15715 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:20:59.864297  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: true dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:20:59.865959  6505 raft_consensus.cc:3060] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:20:59.866050 15680 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:20:59.830878 15683 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:59.836014 15565 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:59.871662  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:20:59.860383 15484 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:20:59.846004 15566 raft_consensus.cc:2749] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:20:59.842736 15586 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:59.880743  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e0ab356fc074f589a6ff4f9b4996b09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:59.890053  6505 raft_consensus.cc:2468] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 25.
I20260709 12:20:59.858645 15712 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:59.882994  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:59.905293  6657 raft_consensus.cc:2368] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 26. Current term is 27.
I20260709 12:20:59.901414 15586 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:20:59.910754 15692 log.cc:927] Time spent T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.873s	user 0.003s	sys 0.019s
I20260709 12:20:59.911290 15586 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:59.913708  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:59.902879 15661 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:59.915465 15661 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:59.877902 15473 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:20:59.879418 15484 raft_consensus.cc:697] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 27 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:20:59.881167 15715 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:59.880288 15683 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:59.938268 15683 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:59.881665 15565 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:59.940927 15625 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:59.942286  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:59.945362 15484 consensus_queue.cc:237] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:59.874260 15675 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:59.963366 15675 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:20:59.963704 15661 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:20:59.933180 15473 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:20:59.964257  6650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:59.964672 15661 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:20:59.965504 15661 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:20:59.968163  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:59.971257  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:59.973973  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:20:59.973276 14708 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:20:59.970008  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:20:59.980091  6475 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:59.970008  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:20:59.948325 15565 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:00.016548 14708 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:00.020224  6474 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:00.024420  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:21:00.030576  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 124851us
I20260709 12:21:00.025375  6657 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 27.
I20260709 12:20:59.974125 15473 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:00.038794 15624 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:00.048777 15624 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:00.050984 15624 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:00.029634 15679 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:21:00.019174 15683 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:21:00.053105 14708 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:00.053409 15683 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:00.055686 15683 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:00.064270  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:00.065042  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:00.041906  6475 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:20:59.968314 15661 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:20:59.917948 15712 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:00.038584 15628 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:21:00.073884  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:00.074842 14708 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:21:00.075292 14708 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:00.075647 14708 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:21:00.075724  6475 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:20:59.953609 15715 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:00.093955  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:00.077565 15701 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:00.108112 15701 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:00.114554  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:00.115816 15715 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:00.117300 15701 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:00.139925  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:00.120716  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:21:00.122778 14708 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:00.125833  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:21:00.101409 15565 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:21:00.138654  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:00.142959  6550 leader_election.cc:400] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 26. Current term is 27.
I20260709 12:21:00.143049  6504 raft_consensus.cc:1240] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 27. Current term is 28. Ops: []
I20260709 12:21:00.147796  6550 leader_election.cc:403] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:21:00.148305  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:00.151547  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:21:00.152577 14708 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:00.152187  6507 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
I20260709 12:21:00.154078  6550 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:00.153162 15423 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: Vote denied by peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107) with higher term. Message: Invalid argument: T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e for earlier term 26. Current term is 27.
I20260709 12:21:00.172683  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:21:00.182029  6505 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 22.
I20260709 12:21:00.182461  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:21:00.189961 15710 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:00.190793 15710 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:00.193050  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:21:00.198681  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:21:00.199591 15552 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:00.209509  7238 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:00.211225  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:00.206060  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:00.212244  7238 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:00.172295 15712 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:00.213390  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:00.206362  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:00.214623  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:00.218472  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:21:00.218829  6579 raft_consensus.cc:2393] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 27: Already voted for candidate e7554c39679a41bbbca4659b72ab5c65 in this term.
I20260709 12:21:00.204880 15695 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:21:00.219712  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:21:00.220693  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:21:00.223903 15582 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:21:00.217168 15639 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:21:00.225317 15639 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:00.225739 15639 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:21:00.207860 15552 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:00.229372 15710 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:00.214272 15687 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:21:00.233566 15687 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:00.234019 15687 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:21:00.234611 15552 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:00.232182  6622 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:21:00.241809 15712 raft_consensus.cc:2804] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:21:00.242199 15687 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:00.242573 15712 raft_consensus.cc:697] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 25 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:21:00.243525 15712 consensus_queue.cc:237] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:00.243840  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:00.250305  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:00.258013  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:00.259816  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:21:00.261976 15639 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:00.263144  7265 raft_consensus.cc:1240] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 22. Current term is 23. Ops: []
I20260709 12:21:00.266247 15675 consensus_queue.cc:1059] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:21:00.267390 15656 raft_consensus.cc:3055] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:21:00.267772 15656 raft_consensus.cc:740] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:21:00.268648 15656 consensus_queue.cc:260] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:00.272475 15656 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:21:00.285451 15701 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:21:00.286087 15701 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:00.288753 15701 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:00.248893  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:00.291854  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:00.292721  6548 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:00.293550 15429 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:21:00.294628  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 66718us
I20260709 12:21:00.295380  6548 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:00.296142 15429 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:21:00.297621 15429 consensus_queue.cc:1059] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: INVALID_TERM, Last received: 27.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:21:00.298676 15721 raft_consensus.cc:3055] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:21:00.299058 15721 raft_consensus.cc:740] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:21:00.299887 15721 consensus_queue.cc:260] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:00.301385 15721 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:21:00.302102  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:00.304740  6473 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:21:00.306450 14708 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:21:00.307976  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 54923.2us
I20260709 12:21:00.308919  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:21:00.309648 15639 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:00.314328 15687 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:21:00.321098  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 77323.3us
I20260709 12:21:00.322417  6475 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:21:00.322578  6393 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet 6f94b1cd286648a9b82d5abf69bb86d5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 3 but the committed config has opid_index of 13
I20260709 12:21:00.322993 14708 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:21:00.323786 15713 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:21:00.325168  6475 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:00.325501  6657 raft_consensus.cc:2468] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 23.
I20260709 12:21:00.328274  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:00.332422  6402 catalog_manager.cc:5697] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 19 to 27. New cstate: current_term: 27 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:00.333577 15713 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:21:00.340165  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:21:00.342880  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:00.343847  7240 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:21:00.345511  6402 catalog_manager.cc:5697] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 24 to 28, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 28 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:00.347980  6402 catalog_manager.cc:5697] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:00.349241  6649 raft_consensus.cc:2393] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 23: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:21:00.349448  6402 catalog_manager.cc:5697] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 14 to 27. New cstate: current_term: 27 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:21:00.349923 15721 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:00.351037 15721 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:00.350987  6402 catalog_manager.cc:5697] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 19 to 22, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 22 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:00.352887  6402 catalog_manager.cc:5697] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 25 to 27, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 27 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:21:00.354266 15721 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:00.354171  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:00.354791 15687 raft_consensus.cc:2804] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:21:00.355825 15713 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:00.354166  6402 catalog_manager.cc:5697] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 25 to 29, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 29 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:00.356666 15713 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:00.357803  6402 catalog_manager.cc:5697] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 19 to 21, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 21 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:00.360234 15713 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:00.355448 15687 raft_consensus.cc:697] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 23 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:21:00.360778  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:00.361331 15687 consensus_queue.cc:237] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:00.361950  6622 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:21:00.362816 15670 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:21:00.365545  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:00.366299 15675 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:21:00.367586  6402 catalog_manager.cc:5697] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 24 to 26, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 26 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:21:00.370959  6402 catalog_manager.cc:5697] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 27 to 32, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 32 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:21:00.369614  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:21:00.372483  6407 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46184 (request call id 122) took 58375 ms (client timeout 15000 ms). Trace:
I20260709 12:21:00.375335  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:21:00.375810  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:21:00.376878  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:21:00.372990  6407 rpcz_store.cc:269] 0709 12:20:01.997244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:07.450224 (+5452980us) service_pool.cc:224] Handling call
0709 12:20:08.990630 (+1540406us) spinlock_profiling.cc:243] Waited 1.56 ms on lock 0x7b5400002210. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d650b2 00007f287bcde759 00007f287bc76a76 00007f287bc47d58 00007f287bd4604e 00007f287a2072a4 00007f287a207232 00007f287a207161 00007f287a2070dc
0709 12:21:00.372401 (+51381771us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:20:59.032412 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:20:59.434401 (+401989us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:20:59.445976 (+ 11575us) write_op.cc:435] Acquired schema lock
0709 12:20:59.450736 (+  4760us) tablet.cc:662] Decoding operations
0709 12:20:59.478662 (+ 27926us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:20:59.481564 (+  2902us) write_op.cc:641] Partition lock acquired for write op
0709 12:20:59.493830 (+ 12266us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:20:59.509327 (+ 15497us) tablet.cc:701] Row locks acquired
0709 12:20:59.510258 (+   931us) write_op.cc:260] PREPARE: finished
0709 12:20:59.522393 (+ 12135us) write_op.cc:270] Start()
0709 12:20:59.540482 (+ 18089us) write_op.cc:276] Timestamp: P: 1783599659521366 usec, L: 0
0709 12:20:59.541738 (+  1256us) op_driver.cc:348] REPLICATION: starting
0709 12:20:59.706536 (+164798us) log.cc:844] Serialized 527 byte log entry
0709 12:21:00.127495 (+420959us) op_driver.cc:464] REPLICATION: finished
0709 12:21:00.232264 (+104769us) write_op.cc:301] APPLY: starting
0709 12:21:00.232369 (+   105us) tablet.cc:1367] starting BulkCheckPresence
0709 12:21:00.235906 (+  3537us) tablet.cc:1370] finished BulkCheckPresence
0709 12:21:00.236006 (+   100us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:21:00.236763 (+   757us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:21:00.236948 (+   185us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:21:00.236993 (+    45us) write_op.cc:312] APPLY: finished
0709 12:21:00.237173 (+   180us) log.cc:844] Serialized 25 byte log entry
0709 12:21:00.237936 (+   763us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:00.238016 (+    80us) write_op.cc:454] Released schema lock
0709 12:21:00.238057 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 12:21:00.238455 (+   398us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:00.238483 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":237187,"reported_tablets":24,"spinlock_wait_cycles":3580288,"child_traces":[["op",{"apply.queue_time_us":104530,"apply.run_cpu_time_us":3202,"apply.run_wall_time_us":6174,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8299,"num_ops":1,"prepare.queue_time_us":187785,"prepare.run_cpu_time_us":58871,"prepare.run_wall_time_us":909091,"raft.queue_time_us":188935,"raft.run_cpu_time_us":67984,"raft.run_wall_time_us":88123,"replication_time_us":585583,"spinlock_wait_cycles":66944,"thread_start_us":519082,"threads_started":4,"wal-append.queue_time_us":225769}]]}
I20260709 12:21:00.383953  6548 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:00.389698 15429 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:21:00.390844  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:21:00.391180  6475 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:00.395979 15713 raft_consensus.cc:3060] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:21:00.372287  6402 catalog_manager.cc:5697] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 25 to 29. New cstate: current_term: 29 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:00.407684  6402 catalog_manager.cc:5697] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 20 to 21, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 21 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:21:00.408983  6402 catalog_manager.cc:5697] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 22 to 34. New cstate: current_term: 34 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:00.411495  6475 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:00.413079  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:00.414752  6548 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:00.415073  7238 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:00.416119 15429 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:21:00.410192 15675 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:21:00.431972 15639 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:21:00.410283  6402 catalog_manager.cc:5697] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 19 to 20, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 20 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:00.430984 15710 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:21:00.463613  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:00.459457  6402 catalog_manager.cc:5697] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 24 to 26, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 26 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:21:00.474732  6402 catalog_manager.cc:5697] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: config changed from index 4 to 11, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New cstate: current_term: 20 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:21:00.477313  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 60611.2us
I20260709 12:21:00.478071  6402 catalog_manager.cc:5697] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 20 to 21, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 21 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:00.482280  6402 catalog_manager.cc:5697] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 12 to 14, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 14 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:00.501678  6577 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:21:00.502395  6577 raft_consensus.cc:606] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:00.502792  6577 raft_consensus.cc:614] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:21:00.490340 15652 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:21:00.495013 15713 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:21:00.547972  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 53649.5us
W20260709 12:21:00.556582  6393 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet c22b6d97ae844b7195c5e9c024203d48 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
W20260709 12:21:00.600696  6393 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 44194us
I20260709 12:21:00.621760  7265 raft_consensus.cc:1240] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 15. Current term is 16. Ops: []
I20260709 12:21:00.632261 15652 consensus_queue.cc:1059] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 12:21:00.706585  6582 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:21:00.751622 15656 raft_consensus.cc:3055] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:21:00.752120 15656 raft_consensus.cc:740] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:21:00.753037 15656 consensus_queue.cc:260] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:00.755038 15656 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:21:00.770236  6658 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:21:00.772213  6649 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:47858
I20260709 12:21:00.772815  6649 raft_consensus.cc:606] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 24 LEADER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
W20260709 12:21:00.774137  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 52460.7us
I20260709 12:21:00.775585  6649 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:47858
I20260709 12:21:00.776232  6649 raft_consensus.cc:606] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 29 LEADER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:00.776216 15710 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:21:00.780464  6508 raft_consensus.cc:1275] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:21:00.781482  6656 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:47858
I20260709 12:21:00.782055  6656 raft_consensus.cc:606] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Received request to transfer leadership to TS e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:00.794317  6656 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
mode: GRACEFUL
new_leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:47858
I20260709 12:21:00.794950  6656 raft_consensus.cc:606] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Received request to transfer leadership to TS 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:21:00.797076 14708 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:21:00.799808  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:21:00.801553  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:21:00.802271  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 76117.1us
I20260709 12:21:00.812165  6652 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:21:00.813745 15542 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:21:00.814381 15675 consensus_queue.cc:1048] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:21:00.829947 15712 raft_consensus.cc:993] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4: : Instructing follower e7554c39679a41bbbca4659b72ab5c65 to start an election
I20260709 12:21:00.830462 15712 raft_consensus.cc:993] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4: : Instructing follower e7554c39679a41bbbca4659b72ab5c65 to start an election
I20260709 12:21:00.830510 15670 raft_consensus.cc:1081] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 24 LEADER]: Signalling peer e7554c39679a41bbbca4659b72ab5c65 to start an election
I20260709 12:21:00.830983 15712 raft_consensus.cc:1081] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 29 LEADER]: Signalling peer e7554c39679a41bbbca4659b72ab5c65 to start an election
I20260709 12:21:00.835714  6508 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:45880
I20260709 12:21:00.836418  6508 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:00.836830  6508 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:21:00.840548  6500 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:45880
I20260709 12:21:00.841630  6500 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:00.842343  6500 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:21:00.881196  6578 raft_consensus.cc:1275] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:21:00.885849  6500 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 30 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:00.889125 15652 consensus_queue.cc:1048] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:21:00.897003  6508 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:00.897921  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:00.900574  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: true dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:21:00.901259  7265 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:21:00.907589 15652 raft_consensus.cc:993] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4: : Instructing follower 496c7c3792cd4587b4dbeba435f4be1e to start an election
I20260709 12:21:00.908239 15652 raft_consensus.cc:1081] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Signalling peer 496c7c3792cd4587b4dbeba435f4be1e to start an election
I20260709 12:21:00.910650  6582 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
 from {username='slave'} at 127.0.0.1:56124
I20260709 12:21:00.911356  6582 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:00.911736  6582 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:21:00.912885  6508 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:00.909607  7265 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 25.
I20260709 12:21:00.913981  6500 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 30 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:00.916606  6474 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:21:00.917969 15701 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:21:00.918613 15701 raft_consensus.cc:697] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 25 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:21:00.933095  6582 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:00.937875  6582 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:00.939033  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: true dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:21:00.939738  6656 raft_consensus.cc:3055] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:21:00.940060  6656 raft_consensus.cc:740] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:21:00.942750  6579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: true dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:21:00.943431  6579 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:21:00.944384  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: true dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:21:00.945096  6657 raft_consensus.cc:3055] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:21:00.945387  6657 raft_consensus.cc:740] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:21:00.947384  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: true dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:21:00.948098  6649 raft_consensus.cc:3055] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:21:00.948585  6649 raft_consensus.cc:740] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:21:00.946455  6657 consensus_queue.cc:260] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:00.952381  6507 raft_consensus.cc:1275] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
W20260709 12:21:00.952929  6393 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet c22b6d97ae844b7195c5e9c024203d48 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 6 but the committed config has opid_index of 13
I20260709 12:21:00.954541  6657 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:21:00.957001  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: true dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:21:00.957573  6507 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:21:00.959115  6579 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 30.
I20260709 12:21:00.959358  6656 consensus_queue.cc:260] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:00.963917  6656 raft_consensus.cc:3060] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:21:00.963692 15697 consensus_queue.cc:1048] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:21:00.968586  6649 consensus_queue.cc:260] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.16, Last appended by leader: 16, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:00.970497  6583 raft_consensus.cc:1240] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Rejecting Update request from peer f2b06aebe97249a4877d798b8118ece4 for earlier term 29. Current term is 30. Ops: []
I20260709 12:21:00.969729  6649 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:21:00.919795 15701 consensus_queue.cc:237] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:00.975684  6657 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 25.
I20260709 12:21:00.998082  6649 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 30.
I20260709 12:21:00.999503  6507 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 23.
I20260709 12:21:01.001634  6656 raft_consensus.cc:2468] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 23.
I20260709 12:21:01.001006  6548 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:21:01.012650  6475 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:21:01.023794 14708 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:21:01.024525 14708 raft_consensus.cc:697] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 30 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:21:01.014845 15498 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:21:01.027606 15498 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:01.028751 14708 consensus_queue.cc:237] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.16, Last appended by leader: 16, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:01.032104 15498 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:01.034281  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:01.037297  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:01.039256  6549 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:01.040108  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:21:01.021143 15552 raft_consensus.cc:2804] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:21:01.053602 15552 raft_consensus.cc:697] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:21:01.059684 15552 consensus_queue.cc:237] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:01.048447 15498 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:21:01.075435  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 58232.7us
I20260709 12:21:01.118626 15708 raft_consensus.cc:993] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4: : Instructing follower e7554c39679a41bbbca4659b72ab5c65 to start an election
I20260709 12:21:01.126523 15654 raft_consensus.cc:1081] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Signalling peer e7554c39679a41bbbca4659b72ab5c65 to start an election
I20260709 12:21:01.129637  6504 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
 from {username='slave'} at 127.0.0.1:45880
I20260709 12:21:01.130260  6504 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:01.130663  6504 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:21:01.157804  6504 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:01.195283  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: true dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:21:01.195935  7265 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:21:01.196641  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: true dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:21:01.197247  6649 raft_consensus.cc:3055] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:21:01.197536  6649 raft_consensus.cc:740] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:21:01.198312  6649 consensus_queue.cc:260] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:01.199419  6649 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:21:01.210379  7265 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22.
I20260709 12:21:01.218341  6474 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:21:01.219183 15713 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:21:01.213078  6504 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 election: Requested vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:01.219868 15713 raft_consensus.cc:697] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 22 LEADER]: Becoming Leader. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:21:01.224133 15713 consensus_queue.cc:237] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:01.251546  6398 catalog_manager.cc:5697] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:01.240921  6402 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57900 (request call id 196) took 1041 ms. Trace:
I20260709 12:21:01.253755  6402 rpcz_store.cc:276] 0709 12:21:00.199936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:00.200092 (+   156us) service_pool.cc:224] Handling call
0709 12:21:01.240828 (+1040736us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:21:00.636516 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:21:00.663786 (+ 27270us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:21:00.663820 (+    34us) write_op.cc:435] Acquired schema lock
0709 12:21:00.663838 (+    18us) tablet.cc:662] Decoding operations
0709 12:21:00.668012 (+  4174us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:21:00.672739 (+  4727us) write_op.cc:641] Partition lock acquired for write op
0709 12:21:00.672798 (+    59us) tablet.cc:685] Acquiring locks for 18 operations
0709 12:21:00.680098 (+  7300us) tablet.cc:701] Row locks acquired
0709 12:21:00.680119 (+    21us) write_op.cc:260] PREPARE: finished
0709 12:21:00.680262 (+   143us) write_op.cc:270] Start()
0709 12:21:00.680404 (+   142us) write_op.cc:276] Timestamp: P: 1783599660680216 usec, L: 0
0709 12:21:00.680428 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:21:00.709847 (+ 29419us) log.cc:844] Serialized 6932 byte log entry
0709 12:21:00.712323 (+  2476us) op_driver.cc:464] REPLICATION: finished
0709 12:21:00.893604 (+181281us) write_op.cc:301] APPLY: starting
0709 12:21:00.893694 (+    90us) tablet.cc:1367] starting BulkCheckPresence
0709 12:21:01.050870 (+157176us) tablet.cc:1370] finished BulkCheckPresence
0709 12:21:01.051018 (+   148us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:21:01.123837 (+ 72819us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:21:01.125921 (+  2084us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=0,mrs_lookups=0
0709 12:21:01.125989 (+    68us) write_op.cc:312] APPLY: finished
0709 12:21:01.127227 (+  1238us) log.cc:844] Serialized 164 byte log entry
0709 12:21:01.128483 (+  1256us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:01.129123 (+   640us) write_op.cc:454] Released schema lock
0709 12:21:01.129348 (+   225us) write_op.cc:341] FINISH: Updating metrics
0709 12:21:01.130710 (+  1362us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:01.130803 (+    93us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":34081,"reported_tablets":23,"child_traces":[["op",{"apply.queue_time_us":142258,"apply.run_cpu_time_us":23723,"apply.run_wall_time_us":236845,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79223,"num_ops":18,"prepare.queue_time_us":73978,"prepare.run_cpu_time_us":6198,"prepare.run_wall_time_us":83457,"replication_time_us":31746,"thread_start_us":186105,"threads_started":2}]]}
I20260709 12:21:01.268718  6649 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7554c39679a41bbbca4659b72ab5c65 in term 22.
I20260709 12:21:01.253314  6398 catalog_manager.cc:5697] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 22 to 23, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 23 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:01.365157  6402 catalog_manager.cc:5697] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 22 to 27, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 27 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:21:01.366926  6402 catalog_manager.cc:5697] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 22 to 23, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 23 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:01.368503  6402 catalog_manager.cc:5697] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 24 to 25, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 25 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:21:01.393108  6656 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:21:01.396974 14708 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:21:01.402251  6399 catalog_manager.cc:5697] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 21 to 23, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 23 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:01.404495  6399 catalog_manager.cc:5697] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: config changed from index 6 to 13, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New cstate: current_term: 21 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:01.406723  6399 catalog_manager.cc:5697] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 14 to 15, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 15 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:01.416929  6657 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
I20260709 12:21:01.419822 15701 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:21:01.429405  6399 catalog_manager.cc:5697] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:01.469009  6507 raft_consensus.cc:1275] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:21:01.481709  6399 catalog_manager.cc:5697] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 17 to 18, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 18 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:01.479588 15497 consensus_queue.cc:1048] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.015s
I20260709 12:21:01.496905  6583 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
I20260709 12:21:01.498391 15710 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:21:01.506610  6652 raft_consensus.cc:1275] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:21:01.516621 15630 consensus_queue.cc:1048] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:21:01.519696  6398 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57888 (request call id 220) took 48066 ms (client timeout 15000 ms). Trace:
W20260709 12:21:01.520207  6398 rpcz_store.cc:269] 0709 12:20:13.509337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:13.700606 (+191269us) service_pool.cc:224] Handling call
0709 12:20:14.713766 (+1013160us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b540008acd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df3090 00007f2875e91f3a 00007f287bc47e9c 00007f287bd4604e 00007f287a2072a4 00007f287a207232 00007f287a207161 00007f287a2070dc 00007f287a206392 00007f28741c448c 00007f28741c38c6
0709 12:21:01.519624 (+46805858us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:21:01.347003 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:21:01.347598 (+   595us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:21:01.347638 (+    40us) write_op.cc:435] Acquired schema lock
0709 12:21:01.347661 (+    23us) tablet.cc:662] Decoding operations
0709 12:21:01.348074 (+   413us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:21:01.348158 (+    84us) write_op.cc:641] Partition lock acquired for write op
0709 12:21:01.348189 (+    31us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:21:01.348529 (+   340us) tablet.cc:701] Row locks acquired
0709 12:21:01.348559 (+    30us) write_op.cc:260] PREPARE: finished
0709 12:21:01.348707 (+   148us) write_op.cc:270] Start()
0709 12:21:01.348822 (+   115us) write_op.cc:276] Timestamp: P: 1783599661348656 usec, L: 0
0709 12:21:01.348846 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:21:01.349523 (+   677us) log.cc:844] Serialized 901 byte log entry
0709 12:21:01.351356 (+  1833us) op_driver.cc:464] REPLICATION: finished
0709 12:21:01.351807 (+   451us) write_op.cc:301] APPLY: starting
0709 12:21:01.351899 (+    92us) tablet.cc:1367] starting BulkCheckPresence
0709 12:21:01.353247 (+  1348us) tablet.cc:1370] finished BulkCheckPresence
0709 12:21:01.353337 (+    90us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:21:01.354632 (+  1295us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:21:01.354802 (+   170us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 12:21:01.354847 (+    45us) write_op.cc:312] APPLY: finished
0709 12:21:01.355087 (+   240us) log.cc:844] Serialized 33 byte log entry
0709 12:21:01.355248 (+   161us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:01.355344 (+    96us) write_op.cc:454] Released schema lock
0709 12:21:01.355383 (+    39us) write_op.cc:341] FINISH: Updating metrics
0709 12:21:01.355753 (+   370us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:01.364382 (+  8629us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":355113,"reported_tablets":52,"spinlock_wait_cycles":502362496,"child_traces":[["op",{"apply.queue_time_us":222,"apply.run_cpu_time_us":3897,"apply.run_wall_time_us":3897,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8344,"num_ops":2,"prepare.queue_time_us":18232,"prepare.run_cpu_time_us":2848,"prepare.run_wall_time_us":2848,"replication_time_us":2377,"spinlock_wait_cycles":44800,"thread_start_us":25427,"threads_started":1}]]}
I20260709 12:21:01.548990  7265 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:21:01.560017 15713 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.005s
I20260709 12:21:01.607928  6494 tablet_service.cc:1558] Processing DeleteTablet for tablet 3e0ab356fc074f589a6ff4f9b4996b09 with delete_type TABLET_DATA_TOMBSTONED (TS e7554c39679a41bbbca4659b72ab5c65 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:53614
W20260709 12:21:01.631129  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 45809.2us
I20260709 12:21:01.642014 15552 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:01.642819 15552 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:01.653349  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:01.654243  6657 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 27.
I20260709 12:21:01.656759  6550 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:21:01.657886 15498 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:21:01.659955 15757 tablet_replica.cc:333] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:01.661644 15757 raft_consensus.cc:2243] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:01.664278  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:21:01.665114  6505 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 28.
I20260709 12:21:01.670544 15757 raft_consensus.cc:2272] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:01.672571  6401 catalog_manager.cc:5697] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 18 to 21, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 21 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:01.680454 15498 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:01.680991 15498 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:21:01.681941  6393 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet 6f94b1cd286648a9b82d5abf69bb86d5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 3 but the committed config has opid_index of 13
I20260709 12:21:01.684233 15552 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:01.686237 15757 ts_tablet_manager.cc:1916] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:01.691205  6399 catalog_manager.cc:5244] ChangeConfig:REMOVE_PEER RPC for tablet 3e0ab356fc074f589a6ff4f9b4996b09 with cas_config_opid_index 4: aborting the task: latest config opid_index 11; task opid_index 4
I20260709 12:21:01.674516  6401 catalog_manager.cc:5697] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 25 to 29. New cstate: current_term: 29 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:01.704231  6401 catalog_manager.cc:5697] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 15 to 20. New cstate: current_term: 20 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:01.748193  6657 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:21:01.751490 15484 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 12:21:01.770709  6399 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46184 (request call id 123) took 37284 ms (client timeout 15000 ms). Trace:
W20260709 12:21:01.771179  6399 rpcz_store.cc:269] 0709 12:20:24.512222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:24.516518 (+  4296us) service_pool.cc:224] Handling call
0709 12:20:24.675717 (+159199us) spinlock_profiling.cc:243] Waited 35.9 ms on lock 0x7b540008acd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df3090 00007f2875e91f3a 00007f287bc47e9c 00007f287bd4604e 00007f287a2072a4 00007f287a207232 00007f287a207161 00007f287a2070dc 00007f287a206392 00007f28741c448c 00007f28741c38c6
0709 12:21:01.770632 (+37094915us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:21:01.518264 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:21:01.518786 (+   522us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:21:01.518819 (+    33us) write_op.cc:435] Acquired schema lock
0709 12:21:01.518836 (+    17us) tablet.cc:662] Decoding operations
0709 12:21:01.519515 (+   679us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:21:01.519566 (+    51us) write_op.cc:641] Partition lock acquired for write op
0709 12:21:01.519593 (+    27us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:21:01.520106 (+   513us) tablet.cc:701] Row locks acquired
0709 12:21:01.520125 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:21:01.520257 (+   132us) write_op.cc:270] Start()
0709 12:21:01.520463 (+   206us) write_op.cc:276] Timestamp: P: 1783599661520218 usec, L: 0
0709 12:21:01.520488 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:21:01.521100 (+   612us) log.cc:844] Serialized 2024 byte log entry
0709 12:21:01.523042 (+  1942us) op_driver.cc:464] REPLICATION: finished
0709 12:21:01.523478 (+   436us) write_op.cc:301] APPLY: starting
0709 12:21:01.523559 (+    81us) tablet.cc:1367] starting BulkCheckPresence
0709 12:21:01.526553 (+  2994us) tablet.cc:1370] finished BulkCheckPresence
0709 12:21:01.526640 (+    87us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:21:01.530168 (+  3528us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:21:01.530379 (+   211us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=0,mrs_lookups=0
0709 12:21:01.530450 (+    71us) write_op.cc:312] APPLY: finished
0709 12:21:01.530813 (+   363us) log.cc:844] Serialized 57 byte log entry
0709 12:21:01.535988 (+  5175us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:01.536099 (+   111us) write_op.cc:454] Released schema lock
0709 12:21:01.536140 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 12:21:01.536476 (+   336us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:01.536502 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":48628,"reported_tablets":24,"spinlock_wait_cycles":82476928,"child_traces":[["op",{"apply.queue_time_us":254,"apply.run_cpu_time_us":7585,"apply.run_wall_time_us":12962,"cfile_cache_hit":20,"cfile_cache_hit_bytes":20860,"num_ops":5,"prepare.queue_time_us":427,"prepare.run_cpu_time_us":3037,"prepare.run_wall_time_us":3096,"replication_time_us":2452}]]}
I20260709 12:21:01.783102 15498 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:01.821632  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:21:01.824473  6652 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Advancing to term 29
I20260709 12:21:01.824702  7272 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:21:01.826165 15713 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 12:21:01.841109  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 110703us
I20260709 12:21:01.851457  6504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:21:01.852147  6504 raft_consensus.cc:3060] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:21:01.855338  6401 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57888 (request call id 222) took 6706 ms. Trace:
I20260709 12:21:01.858314  6652 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 29.
I20260709 12:21:01.859714  6550 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:21:01.822395 15498 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:01.860785 15498 raft_consensus.cc:2804] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:21:01.861265 15498 raft_consensus.cc:697] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:21:01.862092 15498 consensus_queue.cc:237] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:01.863721  6504 raft_consensus.cc:2468] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 29.
I20260709 12:21:01.911777 15757 ts_tablet_manager.cc:1929] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 20.10
I20260709 12:21:01.888715 15712 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:01.855871  6401 rpcz_store.cc:276] 0709 12:20:55.149192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:55.154981 (+  5789us) service_pool.cc:224] Handling call
0709 12:21:01.855252 (+6700271us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:21:01.768034 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:21:01.768870 (+   836us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:21:01.768926 (+    56us) write_op.cc:435] Acquired schema lock
0709 12:21:01.768952 (+    26us) tablet.cc:662] Decoding operations
0709 12:21:01.769825 (+   873us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:21:01.769889 (+    64us) write_op.cc:641] Partition lock acquired for write op
0709 12:21:01.769915 (+    26us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:21:01.770247 (+   332us) tablet.cc:701] Row locks acquired
0709 12:21:01.770273 (+    26us) write_op.cc:260] PREPARE: finished
0709 12:21:01.770454 (+   181us) write_op.cc:270] Start()
0709 12:21:01.770571 (+   117us) write_op.cc:276] Timestamp: P: 1783599661770400 usec, L: 0
0709 12:21:01.770588 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 12:21:01.773104 (+  2516us) log.cc:844] Serialized 1275 byte log entry
0709 12:21:01.775266 (+  2162us) op_driver.cc:464] REPLICATION: finished
0709 12:21:01.775722 (+   456us) write_op.cc:301] APPLY: starting
0709 12:21:01.775805 (+    83us) tablet.cc:1367] starting BulkCheckPresence
0709 12:21:01.777950 (+  2145us) tablet.cc:1370] finished BulkCheckPresence
0709 12:21:01.778126 (+   176us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:21:01.781616 (+  3490us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:21:01.781860 (+   244us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=0,mrs_lookups=0
0709 12:21:01.781926 (+    66us) write_op.cc:312] APPLY: finished
0709 12:21:01.782201 (+   275us) log.cc:844] Serialized 41 byte log entry
0709 12:21:01.783468 (+  1267us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:01.783561 (+    93us) write_op.cc:454] Released schema lock
0709 12:21:01.783602 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 12:21:01.789625 (+  6023us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:01.789654 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3343,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":249,"apply.run_cpu_time_us":5919,"apply.run_wall_time_us":13865,"cfile_cache_hit":12,"cfile_cache_hit_bytes":12516,"num_ops":3,"prepare.queue_time_us":126,"prepare.run_cpu_time_us":3714,"prepare.run_wall_time_us":22753,"replication_time_us":4540}]]}
I20260709 12:21:01.953331 15757 log.cc:1199] T 3e0ab356fc074f589a6ff4f9b4996b09 P e7554c39679a41bbbca4659b72ab5c65: Deleting WAL directory at /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-0-root/wals/3e0ab356fc074f589a6ff4f9b4996b09
I20260709 12:21:01.937749 15712 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:21:02.010586  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 114149us
W20260709 12:21:02.087924  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 90046.3us
I20260709 12:21:02.131670  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:02.208658 15697 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:02.238116 15697 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:21:02.266798 15768 log.cc:927] Time spent T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.261s	user 0.001s	sys 0.000s
W20260709 12:21:02.268291 15767 log.cc:927] Time spent T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.261s	user 0.000s	sys 0.001s
I20260709 12:21:02.272495  6568 tablet_service.cc:1558] Processing DeleteTablet for tablet c22b6d97ae844b7195c5e9c024203d48 with delete_type TABLET_DATA_TOMBSTONED (TS 496c7c3792cd4587b4dbeba435f4be1e not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:57158
W20260709 12:21:02.407850  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 280916us
I20260709 12:21:02.380499 15712 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:02.384002 15778 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:02.420070 15778 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:21:02.425096  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 425358us
I20260709 12:21:02.442433  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:02.443151  7272 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 16.
I20260709 12:21:02.446319  6623 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:21:02.487772 15781 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:21:02.498728  6505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:21:02.499351  6505 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
I20260709 12:21:02.495013 15792 tablet_replica.cc:333] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
W20260709 12:21:02.511395 15780 log.cc:927] Time spent T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 12:21:02.511864  6579 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:21:02.512794 15697 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:02.517306  6579 raft_consensus.cc:606] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:02.497231 15781 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:02.530381  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:21:02.531118  6507 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 15.
I20260709 12:21:02.527566 15781 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:21:02.546471  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:02.581883  6507 raft_consensus.cc:1275] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:21:02.597191  6581 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:53972
I20260709 12:21:02.597795  6581 raft_consensus.cc:606] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 LEADER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:02.605036  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:02.621320 15778 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:02.625730  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:21:02.650092  7238 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65; no voters: 
I20260709 12:21:02.658334  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:02.659484  7272 raft_consensus.cc:2393] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7554c39679a41bbbca4659b72ab5c65 in current term 16: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:21:02.663158 15687 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:21:02.761446  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 178543us
I20260709 12:21:02.798528  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:02.802726 15781 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:02.804052  6656 raft_consensus.cc:2393] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 17: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:21:02.806843 15713 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:21:02.807744 15687 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:02.808189 15687 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:21:02.808353 15781 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:02.811296  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:21:02.864739  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:21:02.871296 15707 raft_consensus.cc:993] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e: : Instructing follower f2b06aebe97249a4877d798b8118ece4 to start an election
I20260709 12:21:02.871683 15766 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:21:02.871901 15552 raft_consensus.cc:1081] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 LEADER]: Signalling peer f2b06aebe97249a4877d798b8118ece4 to start an election
I20260709 12:21:02.872248 15766 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:02.874372 15766 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:02.878820 15764 raft_consensus.cc:993] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e: : Instructing follower f2b06aebe97249a4877d798b8118ece4 to start an election
I20260709 12:21:02.879344 15764 raft_consensus.cc:1081] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Signalling peer f2b06aebe97249a4877d798b8118ece4 to start an election
I20260709 12:21:02.882207  6649 raft_consensus.cc:1275] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:21:02.882382  6657 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:58980
I20260709 12:21:02.882907  6657 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:02.883260  6657 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:21:02.884377  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:02.893216 15792 raft_consensus.cc:2243] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:02.894410 15792 raft_consensus.cc:2272] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:02.899369 15789 consensus_queue.cc:1048] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:21:02.899822 15792 ts_tablet_manager.cc:1916] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:02.904920  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:02.906131  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:02.904239  6652 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:58980
I20260709 12:21:02.910409  6652 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:02.911175  6652 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:21:02.911779  6401 catalog_manager.cc:5697] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 29 to 30, leader changed from f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 30 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:21:02.914847 15687 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:21:02.914240  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 52738.8us
I20260709 12:21:02.916321  7272 raft_consensus.cc:2393] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 17: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:21:02.468552  6392 catalog_manager.cc:5028] TS e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): tablet 3e0ab356fc074f589a6ff4f9b4996b09 (table test-workload [id=f93362d0518e483fbe3a01f43a217e48]) successfully deleted
W20260709 12:21:02.917225  6392 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 467828us
I20260709 12:21:02.921700 15687 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:02.923133 15625 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:02.923707 15625 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:02.924301  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:21:02.930302  6649 raft_consensus.cc:2393] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 17: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:21:02.933918 15713 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:21:02.934897  6507 raft_consensus.cc:3060] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:21:02.938261 15625 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:02.946504  6657 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:02.946941 15766 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:21:02.947540 15766 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:02.946504  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:02.949642 15766 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:02.950965  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:02.953740  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:02.946507  6507 raft_consensus.cc:2468] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 17.
I20260709 12:21:02.955482  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:21:02.957168  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:21:02.961839  7238 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:02.962203  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:02.956352  6652 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:02.989651 15784 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:21:02.990396  6652 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:02.957199  6549 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
W20260709 12:21:02.992884  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 193390us
W20260709 12:21:03.017148  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 52448.5us
W20260709 12:21:03.029350  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 72216.3us
I20260709 12:21:03.031224 15630 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:21:03.036512 15804 consensus_queue.cc:1048] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:21:03.092217  6500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: true dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:21:03.093052  6500 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:21:03.099725  6657 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:03.103561  6658 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 29. Current term is 30. Ops: []
I20260709 12:21:03.093042  6549 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:03.104720  6508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: true dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:21:03.105275  6508 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:21:03.109804 15707 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:21:03.122869  6508 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 30.
I20260709 12:21:03.128322  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:03.143090 15784 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:21:03.155493  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: true dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:21:03.156165  6583 raft_consensus.cc:3055] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:21:03.156518  6583 raft_consensus.cc:740] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:21:03.157477  6500 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22.
I20260709 12:21:03.176056  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: true dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:21:03.157262  6583 consensus_queue.cc:260] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:03.178439  6583 raft_consensus.cc:3060] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:21:03.183918 15803 consensus_queue.cc:1059] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: INVALID_TERM, Last received: 29.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.032s
I20260709 12:21:03.194778  6622 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:21:03.198005  6583 raft_consensus.cc:2468] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22.
W20260709 12:21:03.199172  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 45889.1us
I20260709 12:21:03.201144  6622 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9
I20260709 12:21:03.204602 15781 raft_consensus.cc:2804] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:21:03.159658 15763 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:03.228427 15801 raft_consensus.cc:2804] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:21:03.208233 15782 raft_consensus.cc:493] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:21:03.232844 15763 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:03.256814 15763 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:03.275137  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:03.240998 15801 raft_consensus.cc:697] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
W20260709 12:21:03.281252  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 64204.7us
W20260709 12:21:03.282612  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 71635.7us
W20260709 12:21:03.285221  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 83610.5us
I20260709 12:21:03.286329  6622 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:21:03.289202 15814 raft_consensus.cc:2804] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:21:03.289780 15814 raft_consensus.cc:697] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:21:03.290851  6580 raft_consensus.cc:3055] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:21:03.291204  6580 raft_consensus.cc:740] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:21:03.291949  6580 consensus_queue.cc:260] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:03.293007  6580 raft_consensus.cc:3060] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:21:03.293622 15814 consensus_queue.cc:237] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:03.297729 15781 raft_consensus.cc:697] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 30 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:21:03.300261 15781 consensus_queue.cc:237] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:03.306290  6504 raft_consensus.cc:1240] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 30 FOLLOWER]: Rejecting Update request from peer 496c7c3792cd4587b4dbeba435f4be1e for earlier term 29. Current term is 30. Ops: []
I20260709 12:21:03.309806  6507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:21:03.335341  6580 raft_consensus.cc:2468] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 30.
W20260709 12:21:03.342300 15789 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 30 current term: 30 is higher.
I20260709 12:21:03.268849 15782 raft_consensus.cc:515] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:03.352263 15782 leader_election.cc:290] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:03.354266 15801 consensus_queue.cc:237] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:03.335271  6494 tablet_service.cc:1558] Processing DeleteTablet for tablet 3e0ab356fc074f589a6ff4f9b4996b09 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 4 (current committed config index is 11)) from {username='slave'} at 127.0.0.1:53614
I20260709 12:21:03.366061  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:03.374650  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7223833e1f47b7b68c0080facbbe95" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:03.416514  6550 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:03.445283 15717 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:21:03.462610  6550 leader_election.cc:304] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:03.452857 15792 ts_tablet_manager.cc:1929] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 21.12
I20260709 12:21:03.474509 15792 log.cc:1199] T c22b6d97ae844b7195c5e9c024203d48 P 496c7c3792cd4587b4dbeba435f4be1e: Deleting WAL directory at /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-1-root/wals/c22b6d97ae844b7195c5e9c024203d48
I20260709 12:21:03.475847 15792 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57158 (request call id 0) took 1477 ms. Trace:
I20260709 12:21:03.476418 15792 rpcz_store.cc:276] 0709 12:21:02.008573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:02.183167 (+174594us) service_pool.cc:224] Handling call
0709 12:21:02.455884 (+272717us) ts_tablet_manager.cc:1151] Deleting tablet c22b6d97ae844b7195c5e9c024203d48
0709 12:21:02.467802 (+ 11918us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:21:03.225022 (+757220us) tablet_metadata.cc:665] Metadata flushed
0709 12:21:03.447451 (+222429us) tablet_metadata.cc:665] Metadata flushed
0709 12:21:03.475740 (+ 28289us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":369867,"tablet-delete.queue_time_us":168651,"thread_start_us":77684,"threads_started":1}
I20260709 12:21:03.464810 15707 raft_consensus.cc:2749] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:21:03.532493  6400 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57888 (request call id 221) took 29831 ms (client timeout 15000 ms). Trace:
W20260709 12:21:03.533926  6400 rpcz_store.cc:269] 0709 12:20:33.539983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:38.905600 (+5365617us) service_pool.cc:224] Handling call
0709 12:21:03.342003 (+24436403us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17750,"reported_tablets":52}
I20260709 12:21:03.537560  6401 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57900 (request call id 198) took 1565 ms. Trace:
I20260709 12:21:03.537544  6392 catalog_manager.cc:5028] TS 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): tablet c22b6d97ae844b7195c5e9c024203d48 (table test-workload [id=f93362d0518e483fbe3a01f43a217e48]) successfully deleted
I20260709 12:21:03.539553  6392 catalog_manager.cc:5028] TS e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): tablet 3e0ab356fc074f589a6ff4f9b4996b09 (table test-workload [id=f93362d0518e483fbe3a01f43a217e48]) successfully deleted
I20260709 12:21:03.538309  6401 rpcz_store.cc:276] 0709 12:21:01.972032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:01.972253 (+   221us) service_pool.cc:224] Handling call
0709 12:21:03.537228 (+1564975us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:21:03.239884 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:21:03.343033 (+103149us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:21:03.343077 (+    44us) write_op.cc:435] Acquired schema lock
0709 12:21:03.343097 (+    20us) tablet.cc:662] Decoding operations
0709 12:21:03.343465 (+   368us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:21:03.343521 (+    56us) write_op.cc:641] Partition lock acquired for write op
0709 12:21:03.343556 (+    35us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:21:03.343753 (+   197us) tablet.cc:701] Row locks acquired
0709 12:21:03.343773 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:21:03.343940 (+   167us) write_op.cc:270] Start()
0709 12:21:03.344065 (+   125us) write_op.cc:276] Timestamp: P: 1783599663343889 usec, L: 0
0709 12:21:03.344088 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:21:03.344833 (+   745us) log.cc:844] Serialized 532 byte log entry
0709 12:21:03.497789 (+152956us) op_driver.cc:464] REPLICATION: finished
0709 12:21:03.502163 (+  4374us) write_op.cc:301] APPLY: starting
0709 12:21:03.502255 (+    92us) tablet.cc:1367] starting BulkCheckPresence
0709 12:21:03.503479 (+  1224us) tablet.cc:1370] finished BulkCheckPresence
0709 12:21:03.503584 (+   105us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:21:03.504261 (+   677us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:21:03.504488 (+   227us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:21:03.504516 (+    28us) write_op.cc:312] APPLY: finished
0709 12:21:03.504719 (+   203us) log.cc:844] Serialized 25 byte log entry
0709 12:21:03.504894 (+   175us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:03.504975 (+    81us) write_op.cc:454] Released schema lock
0709 12:21:03.505011 (+    36us) write_op.cc:341] FINISH: Updating metrics
0709 12:21:03.505388 (+   377us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:03.505413 (+    25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":322743,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":4099,"apply.run_cpu_time_us":3208,"apply.run_wall_time_us":3213,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8299,"num_ops":1,"prepare.queue_time_us":295281,"prepare.run_cpu_time_us":50687,"prepare.run_wall_time_us":224144,"raft.queue_time_us":13738,"raft.run_cpu_time_us":3860,"raft.run_wall_time_us":4680,"replication_time_us":153552,"thread_start_us":229948,"threads_started":4,"wal-append.queue_time_us":136912}]]}
I20260709 12:21:03.655227 15803 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:21:03.657367 15803 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:03.732406 15803 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:03.733798  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:03.753441  6580 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:21:03.758813 15781 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:21:03.761068  6509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e7554c39679a41bbbca4659b72ab5c65" is_pre_election: true
I20260709 12:21:03.769119  6403 catalog_manager.cc:5697] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 18 to 20, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 20 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:03.785233  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:03.782107  6548 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:03.789700 15763 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:21:03.827215  6500 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:21:03.831758 15781 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:21:03.836851  7265 raft_consensus.cc:1275] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:21:03.842721 15656 consensus_queue.cc:1048] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:21:03.862752  6578 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:21:03.897897  6401 catalog_manager.cc:5697] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 27 to 29. New cstate: current_term: 29 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:03.902967  6500 raft_consensus.cc:1275] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 30 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:21:03.928539 15656 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:21:03.933702 15656 consensus_queue.cc:1048] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:21:03.957149  6502 consensus_queue.cc:237] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.15, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:21:03.994650  6500 raft_consensus.cc:1275] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:21:03.994083  6399 catalog_manager.cc:5697] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:04.000593  6399 catalog_manager.cc:5697] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 20 to 22, leader changed from 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) to e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1). New cstate: current_term: 22 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:04.004891 15811 consensus_queue.cc:1048] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:21:04.055181 15830 log.cc:927] Time spent T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.073s	user 0.002s	sys 0.000s
I20260709 12:21:04.049319  6399 catalog_manager.cc:5697] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:04.073378  6399 catalog_manager.cc:5697] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: config changed from index 3 to 13, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New cstate: current_term: 17 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:21:04.091953  6403 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57914 (request call id 196) took 18174 ms (client timeout 15000 ms). Trace:
W20260709 12:21:04.095789  6403 rpcz_store.cc:269] 0709 12:20:45.917445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:45.923544 (+  6099us) service_pool.cc:224] Handling call
0709 12:21:04.091788 (+18168244us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:21:03.850172 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:21:03.851067 (+   895us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:21:03.851124 (+    57us) write_op.cc:435] Acquired schema lock
0709 12:21:03.851144 (+    20us) tablet.cc:662] Decoding operations
0709 12:21:03.851723 (+   579us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:21:03.851859 (+   136us) write_op.cc:641] Partition lock acquired for write op
0709 12:21:03.851894 (+    35us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:21:03.854886 (+  2992us) tablet.cc:701] Row locks acquired
0709 12:21:03.854944 (+    58us) write_op.cc:260] PREPARE: finished
0709 12:21:03.855159 (+   215us) write_op.cc:270] Start()
0709 12:21:03.855342 (+   183us) write_op.cc:276] Timestamp: P: 1783599663855091 usec, L: 0
0709 12:21:03.855366 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:21:03.856190 (+   824us) log.cc:844] Serialized 527 byte log entry
0709 12:21:03.859073 (+  2883us) op_driver.cc:464] REPLICATION: finished
0709 12:21:03.859697 (+   624us) write_op.cc:301] APPLY: starting
0709 12:21:03.859794 (+    97us) tablet.cc:1367] starting BulkCheckPresence
0709 12:21:03.861416 (+  1622us) tablet.cc:1370] finished BulkCheckPresence
0709 12:21:03.861536 (+   120us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:21:03.862304 (+   768us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:21:03.862523 (+   219us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:21:03.862571 (+    48us) write_op.cc:312] APPLY: finished
0709 12:21:03.862855 (+   284us) log.cc:844] Serialized 25 byte log entry
0709 12:21:03.863060 (+   205us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:03.863149 (+    89us) write_op.cc:454] Released schema lock
0709 12:21:03.866617 (+  3468us) write_op.cc:341] FINISH: Updating metrics
0709 12:21:03.867102 (+   485us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:03.867147 (+    45us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":14925,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":307,"apply.run_cpu_time_us":4001,"apply.run_wall_time_us":7370,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4172,"num_ops":1,"prepare.queue_time_us":304,"prepare.run_cpu_time_us":3962,"prepare.run_wall_time_us":6479,"replication_time_us":3530}]]}
I20260709 12:21:04.146610  6658 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 28 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:21:04.152661 15484 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.007s
W20260709 12:21:04.191929  6474 consensus_peers.cc:597] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1997e98759d0465ab0e3c607df127068. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:04.192497  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 44192.6us
I20260709 12:21:04.200205  6578 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 28 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:21:04.205613 15795 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:21:04.216003  6403 catalog_manager.cc:5697] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 21 to 22, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 22 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:21:04.223582  6403 catalog_manager.cc:5697] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 29 to 30, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 30 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:21:04.232648 14708 raft_consensus.cc:2955] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 28 LEADER]: Committing config change with OpId 28.16: config changed from index -1 to 16, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:21:04.287197  6568 tablet_service.cc:1558] Processing DeleteTablet for tablet 6f94b1cd286648a9b82d5abf69bb86d5 with delete_type TABLET_DATA_TOMBSTONED (TS 496c7c3792cd4587b4dbeba435f4be1e not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:57158
I20260709 12:21:04.303539  6399 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57888 (request call id 223) took 1835 ms. Trace:
I20260709 12:21:04.317292  6649 raft_consensus.cc:2955] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Committing config change with OpId 28.16: config changed from index -1 to 16, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:21:04.327008  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:04.381222 15839 tablet_replica.cc:333] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:04.304603  6399 rpcz_store.cc:276] 0709 12:21:02.451342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:02.452559 (+  1217us) service_pool.cc:224] Handling call
0709 12:21:04.286664 (+1834105us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:21:04.165079 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:21:04.167170 (+  2091us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:21:04.167216 (+    46us) write_op.cc:435] Acquired schema lock
0709 12:21:04.167232 (+    16us) tablet.cc:662] Decoding operations
0709 12:21:04.172986 (+  5754us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:21:04.174755 (+  1769us) write_op.cc:641] Partition lock acquired for write op
0709 12:21:04.176305 (+  1550us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:21:04.184436 (+  8131us) tablet.cc:701] Row locks acquired
0709 12:21:04.184455 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:21:04.184623 (+   168us) write_op.cc:270] Start()
0709 12:21:04.184744 (+   121us) write_op.cc:276] Timestamp: P: 1783599664184574 usec, L: 0
0709 12:21:04.184762 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 12:21:04.185381 (+   619us) log.cc:844] Serialized 1650 byte log entry
0709 12:21:04.187997 (+  2616us) op_driver.cc:464] REPLICATION: finished
0709 12:21:04.188445 (+   448us) write_op.cc:301] APPLY: starting
0709 12:21:04.188524 (+    79us) tablet.cc:1367] starting BulkCheckPresence
0709 12:21:04.190913 (+  2389us) tablet.cc:1370] finished BulkCheckPresence
0709 12:21:04.191029 (+   116us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:21:04.201001 (+  9972us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:21:04.201196 (+   195us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=0,mrs_lookups=0
0709 12:21:04.201244 (+    48us) write_op.cc:312] APPLY: finished
0709 12:21:04.201461 (+   217us) log.cc:844] Serialized 49 byte log entry
0709 12:21:04.201634 (+   173us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:04.201740 (+   106us) write_op.cc:454] Released schema lock
0709 12:21:04.201781 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 12:21:04.202155 (+   374us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:04.202181 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":365713,"reported_tablets":27,"child_traces":[["op",{"apply.queue_time_us":226,"apply.run_cpu_time_us":6264,"apply.run_wall_time_us":13659,"cfile_cache_hit":16,"cfile_cache_hit_bytes":16688,"num_ops":4,"prepare.queue_time_us":118,"prepare.run_cpu_time_us":4200,"prepare.run_wall_time_us":23941,"replication_time_us":3115}]]}
I20260709 12:21:04.398945 15839 raft_consensus.cc:2243] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:04.361286  6580 raft_consensus.cc:2955] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Committing config change with OpId 28.16: config changed from index -1 to 16, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:21:04.402900 15839 raft_consensus.cc:2272] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:04.403988  6404 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57914 (request call id 195) took 39319 ms (client timeout 15000 ms). Trace:
W20260709 12:21:04.407723  6404 rpcz_store.cc:269] 0709 12:20:25.104258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:31.509781 (+6405523us) service_pool.cc:224] Handling call
0709 12:21:04.403765 (+32893984us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":161686,"reported_tablets":52,"spinlock_wait_cycles":15104}
I20260709 12:21:04.420553 15839 ts_tablet_manager.cc:1916] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:04.441854  6406 catalog_manager.cc:5244] ChangeConfig:REMOVE_PEER RPC for tablet 6f94b1cd286648a9b82d5abf69bb86d5 with cas_config_opid_index 3: aborting the task: latest config opid_index 13; task opid_index 3
I20260709 12:21:04.442533  6406 catalog_manager.cc:5244] ChangeConfig:REMOVE_PEER RPC for tablet 3e0ab356fc074f589a6ff4f9b4996b09 with cas_config_opid_index 4: aborting the task: latest config opid_index 11; task opid_index 4
W20260709 12:21:04.443601  6406 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46184 (request call id 124) took 21651 ms (client timeout 15000 ms). Trace:
W20260709 12:21:04.444088  6406 rpcz_store.cc:269] 0709 12:20:42.792681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:42.831799 (+ 39118us) service_pool.cc:224] Handling call
0709 12:21:04.443525 (+21611726us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12967,"reported_tablets":24,"spinlock_wait_cycles":11008}
I20260709 12:21:04.466646  6658 consensus_queue.cc:237] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
W20260709 12:21:04.498448  6623 consensus_peers.cc:597] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3318181e3c0b433a9e1cee517969105c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:04.499023  6502 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:21:04.500280  6580 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:21:04.503144 15781 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:21:04.507225 15781 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:21:04.516073 15793 raft_consensus.cc:2955] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Committing config change with OpId 22.16: config changed from index -1 to 16, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:21:04.527516  6580 raft_consensus.cc:2955] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Committing config change with OpId 22.16: config changed from index -1 to 16, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:21:04.528712  6405 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46184 (request call id 125) took 6124 ms. Trace:
I20260709 12:21:04.527516  6500 raft_consensus.cc:2955] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Committing config change with OpId 22.16: config changed from index -1 to 16, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:21:04.554982 15839 ts_tablet_manager.cc:1929] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 17.12
I20260709 12:21:04.555608 15839 log.cc:1199] T 6f94b1cd286648a9b82d5abf69bb86d5 P 496c7c3792cd4587b4dbeba435f4be1e: Deleting WAL directory at /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-1-root/wals/6f94b1cd286648a9b82d5abf69bb86d5
I20260709 12:21:04.557785  6392 catalog_manager.cc:5028] TS 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): tablet 6f94b1cd286648a9b82d5abf69bb86d5 (table test-workload [id=f93362d0518e483fbe3a01f43a217e48]) successfully deleted
I20260709 12:21:04.530798  6405 rpcz_store.cc:276] 0709 12:20:58.404441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:20:58.404775 (+   334us) service_pool.cc:224] Handling call
0709 12:21:04.528494 (+6123719us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8394,"reported_tablets":24}
I20260709 12:21:04.641552 15841 ts_tablet_manager.cc:933] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643)
I20260709 12:21:04.681442  6404 catalog_manager.cc:5697] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: config changed from index -1 to 16, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 28 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:21:04.690155 15841 tablet_copy_client.cc:323] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.1:36643
I20260709 12:21:04.695775  6518 tablet_copy_service.cc:140] P e7554c39679a41bbbca4659b72ab5c65: Received BeginTabletCopySession request for tablet 1997e98759d0465ab0e3c607df127068 from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:41304)
I20260709 12:21:04.698926  6518 tablet_copy_service.cc:161] P e7554c39679a41bbbca4659b72ab5c65: Beginning new tablet copy session on tablet 1997e98759d0465ab0e3c607df127068 from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:41304: session id = 20a202c76244485fbc8f1610473db8b9-1997e98759d0465ab0e3c607df127068
I20260709 12:21:04.717450  6406 catalog_manager.cc:5697] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:04.723285  6580 consensus_queue.cc:237] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:21:04.715449  6509 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef43f1290994490e8ef291fa12012731"
dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
mode: GRACEFUL
new_leader_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:58944
I20260709 12:21:04.732859  6518 tablet_copy_source_session.cc:215] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:04.736603  6509 raft_consensus.cc:606] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 26 LEADER]: Received request to transfer leadership to TS f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:04.748613  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:21:04.739986 15841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1997e98759d0465ab0e3c607df127068. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:04.775079  6404 catalog_manager.cc:5697] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: config changed from index -1 to 16, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 22 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:21:04.789628  6407 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57914 (request call id 197) took 3625 ms. Trace:
I20260709 12:21:04.790036  6407 rpcz_store.cc:276] 0709 12:21:01.163680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:01.163803 (+   123us) service_pool.cc:224] Handling call
0709 12:21:04.789383 (+3625580us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":86577,"reported_tablets":52}
I20260709 12:21:04.876399  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:04.877736  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:04.951442  6658 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:21:04.953205 15717 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:21:04.961875  6502 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:21:04.965418 15717 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:21:05.019651 15707 raft_consensus.cc:2955] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 28 LEADER]: Committing config change with OpId 28.18: config changed from index -1 to 18, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:21:05.037739 15845 ts_tablet_manager.cc:933] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:21:05.045364  6549 consensus_peers.cc:597] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 27473d665a69492b9d08ab7409de0b5f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:05.046607 15797 raft_consensus.cc:993] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65: : Instructing follower f2b06aebe97249a4877d798b8118ece4 to start an election
I20260709 12:21:05.047277 15797 raft_consensus.cc:1081] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 26 LEADER]: Signalling peer f2b06aebe97249a4877d798b8118ece4 to start an election
I20260709 12:21:05.064507  6649 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ef43f1290994490e8ef291fa12012731"
dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
 from {username='slave'} at 127.0.0.1:58972
I20260709 12:21:05.065193  6649 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:05.065608  6649 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:21:05.071121  6509 consensus_queue.cc:237] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:21:05.075482  6649 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:05.084853  6658 raft_consensus.cc:2955] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Committing config change with OpId 28.18: config changed from index -1 to 18, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:21:05.088699  6502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: true dest_uuid: "e7554c39679a41bbbca4659b72ab5c65"
I20260709 12:21:05.089712  6502 raft_consensus.cc:3055] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:21:05.090121  6502 raft_consensus.cc:740] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: e7554c39679a41bbbca4659b72ab5c65, State: Running, Role: LEADER
I20260709 12:21:05.091002  6502 consensus_queue.cc:260] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:05.092428  6502 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:21:05.079003  6500 raft_consensus.cc:2955] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Committing config change with OpId 28.18: config changed from index -1 to 18, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:21:05.106237  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: true dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:21:05.121992  6580 raft_consensus.cc:3060] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:21:05.135916 15841 tablet_copy_client.cc:806] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:05.136749 15841 tablet_copy_client.cc:670] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:05.118440  6649 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:05.119895 15845 tablet_copy_client.cc:323] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.3:36107
I20260709 12:21:05.142593  6668 tablet_copy_service.cc:140] P f2b06aebe97249a4877d798b8118ece4: Received BeginTabletCopySession request for tablet 3318181e3c0b433a9e1cee517969105c from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:49792)
I20260709 12:21:05.143317  6668 tablet_copy_service.cc:161] P f2b06aebe97249a4877d798b8118ece4: Beginning new tablet copy session on tablet 3318181e3c0b433a9e1cee517969105c from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:49792: session id = 20a202c76244485fbc8f1610473db8b9-3318181e3c0b433a9e1cee517969105c
I20260709 12:21:05.171000  6668 tablet_copy_source_session.cc:215] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:05.176023 15845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3318181e3c0b433a9e1cee517969105c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:05.202735  6580 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 27.
I20260709 12:21:05.211732  6502 raft_consensus.cc:2468] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 27.
I20260709 12:21:05.200229  6404 catalog_manager.cc:5697] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: config changed from index -1 to 18, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 28 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:05.216673  6474 consensus_peers.cc:597] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 55df50d51529421789bb752f47dff41a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:05.218793  6658 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:21:05.223528  6583 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:21:05.231736 15797 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:21:05.234292 15797 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:21:05.236052  6622 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:21:05.239060 15840 raft_consensus.cc:2804] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:21:05.239745 15840 raft_consensus.cc:697] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 27 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:21:05.240836 15840 consensus_queue.cc:237] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:05.312463 15841 tablet_copy_client.cc:538] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:05.349552 15841 tablet_bootstrap.cc:492] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
I20260709 12:21:05.343510  6404 catalog_manager.cc:5697] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 26 to 27, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 27 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:05.372779 15845 tablet_copy_client.cc:806] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:05.373462 15845 tablet_copy_client.cc:670] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:05.390198 15845 tablet_copy_client.cc:538] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:05.410328 15797 raft_consensus.cc:2955] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Committing config change with OpId 20.12: config changed from index -1 to 12, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:21:05.415105  6583 raft_consensus.cc:2955] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Committing config change with OpId 20.12: config changed from index -1 to 12, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:21:05.440128 15845 tablet_bootstrap.cc:492] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
I20260709 12:21:05.508273  6649 raft_consensus.cc:2955] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Committing config change with OpId 20.12: config changed from index -1 to 12, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:21:05.458024  6404 catalog_manager.cc:5697] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: config changed from index -1 to 12, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 20 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:21:05.567885 15857 ts_tablet_manager.cc:933] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:05.590866 15857 tablet_copy_client.cc:323] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.2:45101
I20260709 12:21:05.593811  6594 tablet_copy_service.cc:140] P 496c7c3792cd4587b4dbeba435f4be1e: Received BeginTabletCopySession request for tablet 27473d665a69492b9d08ab7409de0b5f from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:54204)
I20260709 12:21:05.594388  6594 tablet_copy_service.cc:161] P 496c7c3792cd4587b4dbeba435f4be1e: Beginning new tablet copy session on tablet 27473d665a69492b9d08ab7409de0b5f from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:54204: session id = 20a202c76244485fbc8f1610473db8b9-27473d665a69492b9d08ab7409de0b5f
I20260709 12:21:05.603924  6594 tablet_copy_source_session.cc:215] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:05.610446 15857 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27473d665a69492b9d08ab7409de0b5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:05.658561  6580 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:21:05.668836 15857 tablet_copy_client.cc:806] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:05.676070 15857 tablet_copy_client.cc:670] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:05.681378 15857 tablet_copy_client.cc:538] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:05.684900 15854 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:21:05.697258 15845 tablet_bootstrap.cc:492] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=16 overwritten=0 applied=16 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:05.703610 15845 tablet_bootstrap.cc:492] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:21:05.704643 15845 ts_tablet_manager.cc:1403] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 0.265s	user 0.052s	sys 0.009s
I20260709 12:21:05.708232 15845 raft_consensus.cc:359] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9 [term 22 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:21:05.702486 15857 tablet_bootstrap.cc:492] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
I20260709 12:21:05.711912 15845 raft_consensus.cc:740] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9 [term 22 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
I20260709 12:21:05.724804 15845 consensus_queue.cc:260] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:21:05.758742  6509 raft_consensus.cc:1275] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:21:05.760953 15793 consensus_queue.cc:1048] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:21:05.771703 15845 ts_tablet_manager.cc:1434] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.067s	user 0.001s	sys 0.007s
I20260709 12:21:05.800686 15865 ts_tablet_manager.cc:933] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643)
I20260709 12:21:05.834198  6668 tablet_copy_service.cc:342] P f2b06aebe97249a4877d798b8118ece4: Request end of tablet copy session 20a202c76244485fbc8f1610473db8b9-3318181e3c0b433a9e1cee517969105c received from {username='slave'} at 127.0.0.1:49792
I20260709 12:21:05.834860  6668 tablet_copy_service.cc:434] P f2b06aebe97249a4877d798b8118ece4: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-3318181e3c0b433a9e1cee517969105c on tablet 3318181e3c0b433a9e1cee517969105c with peer 20a202c76244485fbc8f1610473db8b9
I20260709 12:21:05.902281 15865 tablet_copy_client.cc:323] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.1:36643
I20260709 12:21:05.901314 15841 tablet_bootstrap.cc:492] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=19 overwritten=3 applied=16 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:05.926950 15841 tablet_bootstrap.cc:492] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:21:05.927770 15841 ts_tablet_manager.cc:1403] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 0.578s	user 0.054s	sys 0.024s
I20260709 12:21:05.935331  7268 raft_consensus.cc:1217] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9 [term 22 LEARNER]: Deduplicated request from leader. Original: 22.15->[22.16-22.16]   Dedup: 22.16->[]
I20260709 12:21:05.951092 15841 raft_consensus.cc:359] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [term 28 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:21:05.951975 15841 raft_consensus.cc:740] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [term 28 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
I20260709 12:21:05.953327  6518 tablet_copy_service.cc:140] P e7554c39679a41bbbca4659b72ab5c65: Received BeginTabletCopySession request for tablet 55df50d51529421789bb752f47dff41a from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:41304)
I20260709 12:21:05.953899  6518 tablet_copy_service.cc:161] P e7554c39679a41bbbca4659b72ab5c65: Beginning new tablet copy session on tablet 55df50d51529421789bb752f47dff41a from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:41304: session id = 20a202c76244485fbc8f1610473db8b9-55df50d51529421789bb752f47dff41a
I20260709 12:21:05.952648 15841 consensus_queue.cc:260] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:21:05.966357  6518 tablet_copy_source_session.cc:215] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:05.971056 15841 ts_tablet_manager.cc:1434] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.043s	user 0.005s	sys 0.003s
I20260709 12:21:05.974397  6518 tablet_copy_service.cc:342] P e7554c39679a41bbbca4659b72ab5c65: Request end of tablet copy session 20a202c76244485fbc8f1610473db8b9-1997e98759d0465ab0e3c607df127068 received from {username='slave'} at 127.0.0.1:41304
I20260709 12:21:05.986264  6518 tablet_copy_service.cc:434] P e7554c39679a41bbbca4659b72ab5c65: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-1997e98759d0465ab0e3c607df127068 on tablet 1997e98759d0465ab0e3c607df127068 with peer 20a202c76244485fbc8f1610473db8b9
I20260709 12:21:06.005084 15865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55df50d51529421789bb752f47dff41a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:06.062795  7268 raft_consensus.cc:1217] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [term 28 LEARNER]: Deduplicated request from leader. Original: 28.15->[28.16-28.16]   Dedup: 28.16->[]
I20260709 12:21:06.162650 15865 tablet_copy_client.cc:806] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:06.163647 15865 tablet_copy_client.cc:670] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:06.170616 15857 tablet_bootstrap.cc:492] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=18 overwritten=0 applied=18 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:06.171775 15857 tablet_bootstrap.cc:492] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:21:06.172084 15865 tablet_copy_client.cc:538] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:06.199630 15865 tablet_bootstrap.cc:492] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
I20260709 12:21:06.172663 15857 ts_tablet_manager.cc:1403] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 0.470s	user 0.062s	sys 0.012s
I20260709 12:21:06.235734 15857 raft_consensus.cc:359] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [term 28 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:21:06.243207 15857 raft_consensus.cc:740] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [term 28 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
I20260709 12:21:06.247807 15857 consensus_queue.cc:260] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:21:06.250726 15857 ts_tablet_manager.cc:1434] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.048s	user 0.005s	sys 0.001s
I20260709 12:21:06.287520  6594 tablet_copy_service.cc:342] P 496c7c3792cd4587b4dbeba435f4be1e: Request end of tablet copy session 20a202c76244485fbc8f1610473db8b9-27473d665a69492b9d08ab7409de0b5f received from {username='slave'} at 127.0.0.1:54204
I20260709 12:21:06.288021  6594 tablet_copy_service.cc:434] P 496c7c3792cd4587b4dbeba435f4be1e: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-27473d665a69492b9d08ab7409de0b5f on tablet 27473d665a69492b9d08ab7409de0b5f with peer 20a202c76244485fbc8f1610473db8b9
I20260709 12:21:06.313848  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:06.350832 15840 raft_consensus.cc:1064] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:21:06.353372 15840 consensus_queue.cc:237] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 14, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:06.408668  6504 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:21:06.409694  7268 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9 [term 22 LEARNER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:21:06.411375  6583 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:21:06.413450 15793 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:21:06.448669 15865 tablet_bootstrap.cc:492] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=1 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:06.458308 15852 raft_consensus.cc:1064] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:21:06.458834 15865 tablet_bootstrap.cc:492] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:21:06.469125 15865 ts_tablet_manager.cc:1403] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 0.270s	user 0.045s	sys 0.007s
I20260709 12:21:06.464309 15854 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:21:06.472966 15840 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:21:06.460942 15852 consensus_queue.cc:237] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 14, Current term: 28, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:06.496754  6651 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:21:06.498479 15797 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:21:06.502673  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:21:06.505146  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:06.506136  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:06.506863  7268 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [term 28 LEARNER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:21:06.508389 15851 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.002s
I20260709 12:21:06.533094 15811 raft_consensus.cc:2955] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Committing config change with OpId 22.17: config changed from index 16 to 17, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:06.548828  7272 raft_consensus.cc:1217] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [term 28 LEARNER]: Deduplicated request from leader. Original: 28.17->[28.18-28.18]   Dedup: 28.18->[]
I20260709 12:21:06.552472  6583 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
W20260709 12:21:06.617700 15878 log.cc:927] Time spent T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 12:21:06.650911 15865 raft_consensus.cc:359] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [term 20 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
W20260709 12:21:06.658406  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 80392.8us
I20260709 12:21:06.670171  6509 raft_consensus.cc:2955] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Committing config change with OpId 22.17: config changed from index 16 to 17, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:06.672956 15886 raft_consensus.cc:2955] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 28 LEADER]: Committing config change with OpId 28.17: config changed from index 16 to 17, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:06.691162 15883 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:21:06.652086 15865 raft_consensus.cc:740] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [term 20 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
I20260709 12:21:06.701018 15865 consensus_queue.cc:260] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:21:06.722884 15865 ts_tablet_manager.cc:1434] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.235s	user 0.009s	sys 0.008s
I20260709 12:21:06.735145  6580 raft_consensus.cc:2955] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Committing config change with OpId 22.17: config changed from index 16 to 17, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:06.746855  6404 catalog_manager.cc:5697] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 reported cstate change: config changed from index 16 to 17, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New cstate: current_term: 22 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:06.760829  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 70336.4us
I20260709 12:21:06.763458  6518 tablet_copy_service.cc:342] P e7554c39679a41bbbca4659b72ab5c65: Request end of tablet copy session 20a202c76244485fbc8f1610473db8b9-55df50d51529421789bb752f47dff41a received from {username='slave'} at 127.0.0.1:41304
I20260709 12:21:06.763974  6518 tablet_copy_service.cc:434] P e7554c39679a41bbbca4659b72ab5c65: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-55df50d51529421789bb752f47dff41a on tablet 55df50d51529421789bb752f47dff41a with peer 20a202c76244485fbc8f1610473db8b9
I20260709 12:21:06.767088  6658 raft_consensus.cc:2955] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Committing config change with OpId 28.17: config changed from index 16 to 17, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:06.845098  7268 raft_consensus.cc:2955] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Committing config change with OpId 28.17: config changed from index 16 to 17, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:06.886662  6580 raft_consensus.cc:2955] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Committing config change with OpId 28.17: config changed from index 16 to 17, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:21:06.891608 15887 log.cc:927] Time spent T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 12:21:06.902750  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 63146.9us
W20260709 12:21:06.929168  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 66622.4us
I20260709 12:21:06.937115  7268 raft_consensus.cc:1217] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [term 20 LEARNER]: Deduplicated request from leader. Original: 20.11->[20.12-20.12]   Dedup: 20.12->[]
I20260709 12:21:06.960024 15763 raft_consensus.cc:1064] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:21:06.963254 15763 consensus_queue.cc:237] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 16, Current term: 28, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:07.031802  7268 raft_consensus.cc:2955] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Committing config change with OpId 22.17: config changed from index 16 to 17, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:21:07.048812  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 60598.8us
I20260709 12:21:07.066681  6407 catalog_manager.cc:5697] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: config changed from index 16 to 17, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New cstate: current_term: 28 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:07.080681  7265 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [term 28 LEARNER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:21:07.082500 15881 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:21:07.118539  6658 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:21:07.123342 15882 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 12:21:07.162271  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 63315.2us
W20260709 12:21:07.178002  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 80536.4us
I20260709 12:21:07.208914  6502 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:21:07.218799 15876 raft_consensus.cc:2955] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 28 LEADER]: Committing config change with OpId 28.19: config changed from index 18 to 19, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:07.229172  6651 raft_consensus.cc:2955] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Committing config change with OpId 28.19: config changed from index 18 to 19, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:07.291496  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:07.280723 15763 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.024s
W20260709 12:21:07.401703  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 44460.5us
W20260709 12:21:07.424151  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 58206.6us
I20260709 12:21:07.554566 15888 raft_consensus.cc:1064] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:21:07.557324 15888 consensus_queue.cc:237] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 10, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:21:07.652527  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 88537.5us
I20260709 12:21:07.656071  6509 raft_consensus.cc:2955] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Committing config change with OpId 28.19: config changed from index 18 to 19, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:07.743830  6404 catalog_manager.cc:5697] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 reported cstate change: config changed from index 18 to 19, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New cstate: current_term: 28 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:21:07.764690  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 100157us
W20260709 12:21:07.900522  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 55022.8us
W20260709 12:21:07.921547 15909 log.cc:927] Time spent T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 12:21:07.951696  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 105529us
W20260709 12:21:08.043267 15908 log.cc:927] Time spent T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: Append to log took a long time: real 0.133s	user 0.001s	sys 0.001s
W20260709 12:21:08.049144  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 79663.7us
I20260709 12:21:08.067047  7274 raft_consensus.cc:2955] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Committing config change with OpId 28.19: config changed from index 18 to 19, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:08.081720  6658 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:21:08.125314  7272 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [term 20 LEARNER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:21:08.139226  5352 tablet_server.cc:179] TabletServer@127.5.58.1:0 shutting down...
I20260709 12:21:08.145887  6651 consensus_queue.cc:237] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:21:08.148792  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 44206.3us
W20260709 12:21:08.156726  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 64855.3us
W20260709 12:21:08.160530  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 47643.6us
W20260709 12:21:08.169190  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 70686.1us
I20260709 12:21:08.180649 15877 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:21:08.184587  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 58395.5us
I20260709 12:21:08.190241  6583 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Refusing update from remote peer e7554c39679a41bbbca4659b72ab5c65: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:21:08.196534 15888 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:21:08.205164  6622 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 7 similar messages]
I20260709 12:21:08.205380 15926 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:21:08.211333  6578 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:21:08.213693  7274 raft_consensus.cc:1275] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:21:08.249884 15894 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.028s
W20260709 12:21:08.264881  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 45720.9us
W20260709 12:21:08.266073  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 60932.6us
W20260709 12:21:08.290055  6392 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 81322.3us
W20260709 12:21:08.296545  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 75322.1us
I20260709 12:21:08.295306 15840 consensus_queue.cc:1048] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.032s
W20260709 12:21:08.434650  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 60393.2us
W20260709 12:21:08.366394  6548 consensus_peers.cc:597] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:08.543258  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 176910us
W20260709 12:21:08.544029  6548 consensus_peers.cc:597] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:08.535493 15840 raft_consensus.cc:2955] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Committing config change with OpId 22.18: config changed from index 17 to 18, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:21:08.544698  6548 consensus_peers.cc:597] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:08.545295  6548 consensus_peers.cc:597] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:08.545814  6548 consensus_peers.cc:597] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:08.546413  6548 consensus_peers.cc:597] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:08.546953  6548 consensus_peers.cc:597] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:08.547446  6548 consensus_peers.cc:597] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:08.547920  6548 consensus_peers.cc:597] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:08.540026  7268 raft_consensus.cc:2955] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Committing config change with OpId 20.13: config changed from index 12 to 13, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:21:08.334607  7238 consensus_peers.cc:597] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:08.550427  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 215856us
W20260709 12:21:08.550967  7238 consensus_peers.cc:597] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:08.551489  7238 consensus_peers.cc:597] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:08.533764 15888 raft_consensus.cc:2955] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Committing config change with OpId 20.13: config changed from index 12 to 13, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:08.556226  6583 consensus_queue.cc:237] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:21:08.560958  6392 catalog_manager.cc:4755] TS e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): ChangeConfig:REMOVE_PEER RPC failed for tablet 1997e98759d0465ab0e3c607df127068: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:21:08.583703  6548 consensus_peers.cc:597] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:08.596833  6394 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3318181e3c0b433a9e1cee517969105c with cas_config_opid_index 17: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:08.636672  7272 raft_consensus.cc:2955] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Committing config change with OpId 22.18: config changed from index 17 to 18, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:08.641788  6405 catalog_manager.cc:5697] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 reported cstate change: config changed from index 12 to 13, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New cstate: current_term: 20 leader_uuid: "e7554c39679a41bbbca4659b72ab5c65" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:08.658998  6649 raft_consensus.cc:2955] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Committing config change with OpId 20.13: config changed from index 12 to 13, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:08.667855  6407 catalog_manager.cc:5697] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 reported cstate change: config changed from index 17 to 18, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New cstate: current_term: 22 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:08.700548  6548 consensus_peers.cc:597] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:08.704857  6622 consensus_peers.cc:597] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:08.709359  6548 consensus_peers.cc:597] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:08.739383  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 118238us
W20260709 12:21:08.739996  7238 consensus_peers.cc:597] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:08.753180  7238 consensus_peers.cc:597] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:08.758700  7238 consensus_peers.cc:597] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:08.804705  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 83328.1us
W20260709 12:21:08.806219  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 65621.8us
W20260709 12:21:08.808943  6623 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 64642.4us
W20260709 12:21:08.855258  6622 consensus_peers.cc:597] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:08.855809  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 141080us
W20260709 12:21:08.858922  6622 consensus_peers.cc:597] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:08.865810  6622 consensus_peers.cc:597] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:08.853847 15926 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:21:08.876154  6622 consensus_peers.cc:597] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:08.889837  6622 consensus_peers.cc:597] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:08.963909  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 69188.5us
W20260709 12:21:08.974836  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 78028.8us
I20260709 12:21:08.894276 15945 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:21:09.046842  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 71054.1us
I20260709 12:21:08.886299 15926 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:09.046519 15939 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:21:09.098747  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 59097.4us
I20260709 12:21:09.056303 15945 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:09.111999 15901 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:09.112732 15877 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:09.156749 15902 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:09.157645 15902 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:09.160506 15877 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:09.162722 15877 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:09.163329 15911 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:09.163880 15911 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:21:09.179956  6392 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 190555us
W20260709 12:21:09.180992  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 192855us
I20260709 12:21:09.221311 15902 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:09.095732 15939 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:21:09.272750 15941 log.cc:927] Time spent T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.393s	user 0.001s	sys 0.000s
W20260709 12:21:09.308718 15938 log.cc:927] Time spent T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e: Append to log took a long time: real 0.275s	user 0.001s	sys 0.000s
I20260709 12:21:09.174705 15901 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:21:09.339299 15944 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:09.376147  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:09.377833  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:09.379308  6580 raft_consensus.cc:2955] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Committing config change with OpId 20.13: config changed from index 12 to 13, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:09.383414  7272 raft_consensus.cc:1275] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:21:09.384485 15944 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:09.347604 15955 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:09.385751 15955 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:09.387091 15944 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:09.339758 15911 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:09.401831 15926 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:09.408175  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:09.412680  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd6fb503c26a49bab14a18c9b5e2d792" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:21:09.430366  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 61432.3us
I20260709 12:21:09.446372 15888 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:09.447598 15888 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:09.449267  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 35 candidate_status { last_received { term: 34 index: 18 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:09.456736  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:09.458527 15963 raft_consensus.cc:493] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:21:09.459173 15963 raft_consensus.cc:515] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:09.459972 15973 raft_consensus.cc:493] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:21:09.423825 15901 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:09.460513 15973 raft_consensus.cc:515] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:09.461606 15963 leader_election.cc:290] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:09.461987 15904 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 32 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:09.474254  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:09.476955 15945 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:09.456923 15926 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:09.455215 15797 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:21:09.548416  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 57652.5us
W20260709 12:21:09.548533  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 57184us
W20260709 12:21:09.550654  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 56536.2us
W20260709 12:21:09.586781  6622 consensus_peers.cc:597] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:09.587911  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:09.492937 15958 consensus_queue.cc:1048] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.098s
I20260709 12:21:09.591269 15973 leader_election.cc:290] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:09.591347  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91732822ea14700aa3230fbb6b1cfac" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:09.592310  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e423f656ad70421f959a9a3b81dac708" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 35 candidate_status { last_received { term: 34 index: 18 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:09.592478  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:09.593243  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:09.594831  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:09.595764 15911 raft_consensus.cc:493] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:21:09.595849 15974 raft_consensus.cc:493] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:21:09.596482 15911 raft_consensus.cc:515] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:09.596587 15974 raft_consensus.cc:515] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:09.598876 15911 leader_election.cc:290] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:09.599136 15974 leader_election.cc:290] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:09.599964 15943 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:09.600622 15902 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:21:09.600587 15943 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:09.611434 15943 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:09.613008  6582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:09.601246 15902 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:09.616039 15902 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:09.595904 15877 raft_consensus.cc:493] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:21:09.617478 15877 raft_consensus.cc:515] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:09.617867 15797 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:09.619580 15877 leader_election.cc:290] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:09.474933 15904 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:09.622611 15904 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:09.625854 15888 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:09.588454  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb7f6a796adb4b8c89579ba77274985f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:09.630422  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab430742d4f44bf79c4591f613055cd6" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:09.630823 15955 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:09.636287 15939 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:09.637269 15797 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:09.638542  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b5198a4b6741cdb13e49cf4e90173c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:09.641299  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:09.646055  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c226c3677f8446f837400fd0df3062e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:21:09.647816  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 95730us
I20260709 12:21:09.649353  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 33 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:21:09.644076  6622 consensus_peers.cc:597] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:09.653578  6622 consensus_peers.cc:597] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:09.670850  6622 consensus_peers.cc:597] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:09.677480  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 81068us
W20260709 12:21:09.672137  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 189499us
W20260709 12:21:09.678480  6548 consensus_peers.cc:597] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:09.679351  7238 consensus_peers.cc:597] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:09.671404  6622 consensus_peers.cc:597] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:09.683240  6548 consensus_peers.cc:597] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:09.684551 15920 raft_consensus.cc:493] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:21:09.685150 15920 raft_consensus.cc:515] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:09.656548 15926 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:21:09.687893  6622 consensus_peers.cc:597] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:09.689759 15953 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:09.687589 15926 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:09.690496 15953 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:09.722635  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ab2c6da5994ae2a955fc11d1761c41" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:09.763965 15888 raft_consensus.cc:493] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:21:09.764724 15888 raft_consensus.cc:515] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:09.767251 15888 leader_election.cc:290] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:09.687224 15920 leader_election.cc:290] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:09.768815 15920 raft_consensus.cc:493] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:09.769316 15920 raft_consensus.cc:515] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:09.769922 15902 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:09.770581 15902 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:09.717111 15939 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:09.771801 15970 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:09.772197 15939 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:21:09.706709  7238 consensus_peers.cc:597] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:09.794881  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 64542.8us
W20260709 12:21:09.797223  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 99930.2us
I20260709 12:21:09.784822  6473 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:09.798749  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:09.701504 15974 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:09.803458 15920 leader_election.cc:290] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:09.803828 15974 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:09.772847 15902 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:09.806277 15974 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:09.806588 15963 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:21:09.773623 15797 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:09.807646 15797 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:09.772703 15970 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:09.809978 15976 raft_consensus.cc:2955] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 28 LEADER]: Committing config change with OpId 28.20: config changed from index 19 to 20, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:09.811185 15797 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:09.813970  7268 raft_consensus.cc:2955] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Committing config change with OpId 28.20: config changed from index 19 to 20, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:09.815524 15939 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:09.788360 15953 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:09.816948  6475 leader_election.cc:304] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:09.818493 15940 raft_consensus.cc:2749] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:21:09.819384  6473 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:09.820194 15920 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:21:09.821732  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c22b6d97ae844b7195c5e9c024203d48" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:09.825708  6473 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:09.826999  6473 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:09.835224  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:09.846866  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:09.836311  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:09.837891  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:09.838896  7267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:09.844698  7272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:09.827373 15901 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:21:09.812124 15970 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:09.852430  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:09.853672  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:09.865515  7274 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:09.854511  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:21:09.866370  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 66880.9us
I20260709 12:21:09.865537  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f94b1cd286648a9b82d5abf69bb86d5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:09.855636  6473 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:09.871381  6577 raft_consensus.cc:2955] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Committing config change with OpId 22.18: config changed from index 17 to 18, VOTER e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:21:09.875416  6622 consensus_peers.cc:597] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:09.878811  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c50fba48b82e4ea29ce3f314b054d4ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:09.885568  6473 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:09.865780 15920 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:21:09.892899 15939 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:21:09.891172  6474 leader_election.cc:304] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:09.900141  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:09.917095  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:09.917264  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:09.891192 15940 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:21:09.924022  6473 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:21:09.926149  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:09.936403  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:09.936947  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:09.938134  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:09.940732  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:09.942317  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:09.949254 15982 raft_consensus.cc:2749] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:21:09.970683  6474 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:21:09.957669 15901 raft_consensus.cc:493] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:21:09.978694  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 54766.8us
I20260709 12:21:09.998196  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e07e97c48184f8e99a4fc5a580724a5" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:10.005424 15953 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:10.006182 15953 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:09.977288 15940 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:09.978142 15939 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:21:10.019586  6474 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 48985.4us
I20260709 12:21:09.991665 15974 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:21:09.990501 15970 raft_consensus.cc:493] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:10.026883  6575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:10.026810  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:10.025959  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a47d71c34b541b9b2589b4e80413fa2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:10.025492  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:10.025983  6578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:21:10.041016  6392 catalog_manager.cc:4755] TS e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): ChangeConfig:REMOVE_PEER RPC failed for tablet 1997e98759d0465ab0e3c607df127068: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:10.045994  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2592253d3214104865730480e057d09" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:09.976796 15901 raft_consensus.cc:515] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:10.050233 15939 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:10.051342  6576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14cf1c3a63624d94aa6c9c356e6108e2" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:10.052417 15953 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:10.052794 15939 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:10.050719  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83301542a88f46cea64bc905b3199975" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:10.055958  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492a9464009416aa4c3a11e145dec61" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:10.012261 15963 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:10.058706 15978 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:10.062307  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:10.032467 15982 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:21:10.064600  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e5417fa1d54082a5690b657f407f70" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:10.071019  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f03345855041f8aabfa3e94a1867d9" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:10.061467 15926 raft_consensus.cc:493] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:10.070968 15939 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:10.084812 15939 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:10.087213 15939 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:10.088218  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:10.065579 15963 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:10.054075 15970 raft_consensus.cc:515] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:10.090952  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:10.021461 15940 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:10.094766  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72269ad6925341548796bb34823b3e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:10.095736  6475 leader_election.cc:304] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:10.099239  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b42468eaf14fd49be100af079d3921" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:10.109447 15901 leader_election.cc:290] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:10.113240  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ea6f08346334787a6b558ca10a70249" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:10.118086  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:10.119573  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:10.124050 15976 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:57158 (request call id 2) took 1807 ms. Trace:
I20260709 12:21:10.124529 15976 rpcz_store.cc:276] 0709 12:21:08.298409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:08.298523 (+   114us) service_pool.cc:224] Handling call
0709 12:21:08.553548 (+255025us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b2c00010c48. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874791a0a 00007f2874791693 00007f2874791631 00007f2875ec4902 00007f2875ec46dd 00007f2875e8ff3f 00007f2875e8f6b3 00007f2875e879fa 00007f287ae4f4c9 00007f28743cbc54
0709 12:21:08.589898 (+ 36350us) log.cc:844] Serialized 480 byte log entry
0709 12:21:09.390246 (+800348us) spinlock_profiling.cc:243] Waited 5.23 ms on lock 0x7b2c00010c48. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874791a0a 00007f2874791693 00007f2874791631 00007f2875ec5b52 00007f2875ec5cf5 00007f2875e02691 00007f2875de2eaa 00007f2875de9d53 00007f2875de9cd9 00007f2875de9c69
0709 12:21:10.106129 (+715883us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45409,"raft.queue_time_us":323844,"raft.run_cpu_time_us":9248,"raft.run_wall_time_us":503075,"spinlock_wait_cycles":48687872,"thread_start_us":274946,"threads_started":3,"wal-append.queue_time_us":262964}
I20260709 12:21:10.128010 15970 leader_election.cc:290] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:10.129484 15940 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:10.133718 15963 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:10.102118 15920 raft_consensus.cc:2749] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:21:10.063563 15978 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:10.080098 15926 raft_consensus.cc:515] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:10.139070 15926 leader_election.cc:290] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:10.149567  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:10.150610  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:21:10.152034  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 50200.6us
I20260709 12:21:10.152586  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:10.155396  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:10.157618  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:10.159147  6475 leader_election.cc:304] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:10.160416  6475 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:10.161495  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9f8f5d393744ebabbed2f3e0131cf40" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:10.162693  6473 leader_election.cc:304] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:21:10.162997  6475 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:10.165871  6475 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:10.162177 15926 raft_consensus.cc:2749] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:21:10.174640  6473 leader_election.cc:304] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:21:10.164016 15901 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:21:10.184894  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:21:10.186978  6392 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 82643.5us
I20260709 12:21:10.187799  6392 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 27473d665a69492b9d08ab7409de0b5f with cas_config_opid_index 19: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:10.187814  6473 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:21:10.176715 15939 raft_consensus.cc:2749] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:21:10.176069 15963 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:21:10.179430 15953 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:21:10.191064  6473 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:10.194281 15797 raft_consensus.cc:2749] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:21:10.198515  6475 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:10.199677 15901 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:21:10.201462 15978 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:10.189384 15902 raft_consensus.cc:2749] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:21:10.209067  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:10.210402  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27473d665a69492b9d08ab7409de0b5f" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:10.212944  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43f1290994490e8ef291fa12012731" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:10.213384  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:10.214075  6652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf5342782d14193986ec66c46ce646e" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:10.214855  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e50fd265d348779de32dff1ac6db39" candidate_uuid: "e7554c39679a41bbbca4659b72ab5c65" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:21:10.220824  6548 consensus_peers.cc:597] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:10.222170  6548 leader_election.cc:336] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:10.222645  6548 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:10.223472 15976 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:21:10.223955  6622 leader_election.cc:336] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:10.224574  6622 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:10.227120 15962 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:21:10.228976 15797 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:21:10.230952  6473 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:10.234941 15974 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:21:10.244709  6405 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46184 (request call id 140) took 1605 ms. Trace:
I20260709 12:21:10.238547  6473 leader_election.cc:304] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:21:10.245033  6405 rpcz_store.cc:276] 0709 12:21:08.638938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:08.639109 (+   171us) service_pool.cc:224] Handling call
0709 12:21:10.244658 (+1605549us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:21:08.879131 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:21:09.080017 (+200886us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:21:09.081632 (+  1615us) write_op.cc:435] Acquired schema lock
0709 12:21:09.083374 (+  1742us) tablet.cc:662] Decoding operations
0709 12:21:09.100235 (+ 16861us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:21:09.112920 (+ 12685us) write_op.cc:641] Partition lock acquired for write op
0709 12:21:09.118346 (+  5426us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:21:09.144560 (+ 26214us) tablet.cc:701] Row locks acquired
0709 12:21:09.148231 (+  3671us) write_op.cc:260] PREPARE: finished
0709 12:21:09.181844 (+ 33613us) write_op.cc:270] Start()
0709 12:21:09.263447 (+ 81603us) write_op.cc:276] Timestamp: P: 1783599669177562 usec, L: 0
0709 12:21:09.274088 (+ 10641us) op_driver.cc:348] REPLICATION: starting
0709 12:21:09.358421 (+ 84333us) log.cc:844] Serialized 587 byte log entry
0709 12:21:09.619513 (+261092us) spinlock_profiling.cc:243] Waited 2.66 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2872d7f0ac 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b
0709 12:21:10.086426 (+466913us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b540008b220. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e07343 00007f2875e0ad75 00007f2875e0ad09 00007f2875e0ac99 00007f2875e0ac61 00007f2875e09f5d 0000000000388634 0000000000388479 00007f2872d9c475 00007f2872d9e7c1
0709 12:21:10.105724 (+ 19298us) op_driver.cc:464] REPLICATION: finished
0709 12:21:10.194481 (+ 88757us) spinlock_profiling.cc:243] Waited 7.05 ms on lock 0x7b540008acd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8632a 00007f2875e0bf14 00007f2875e0bec3 00007f2875e0be38 00007f2875e0bdf3 00007f2875e0b0dc 00007f2875e34e16 00007f2875e1ccbb 00007f2875e0739d 00007f2875e0ad75
0709 12:21:10.239197 (+ 44716us) write_op.cc:301] APPLY: starting
0709 12:21:10.239285 (+    88us) tablet.cc:1367] starting BulkCheckPresence
0709 12:21:10.240315 (+  1030us) tablet.cc:1370] finished BulkCheckPresence
0709 12:21:10.240368 (+    53us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:21:10.241007 (+   639us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:21:10.241197 (+   190us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:21:10.241237 (+    40us) write_op.cc:312] APPLY: finished
0709 12:21:10.242329 (+  1092us) log.cc:844] Serialized 25 byte log entry
0709 12:21:10.242494 (+   165us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:10.242567 (+    73us) write_op.cc:454] Released schema lock
0709 12:21:10.242602 (+    35us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":5558,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":124038,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8299,"num_ops":1,"prepare.queue_time_us":206284,"prepare.run_cpu_time_us":20193,"prepare.run_wall_time_us":819769,"raft.queue_time_us":374861,"raft.run_cpu_time_us":20009,"raft.run_wall_time_us":129808,"replication_time_us":821291,"spinlock_wait_cycles":45875200,"thread_start_us":663597,"threads_started":4,"wal-append.queue_time_us":446822}]]}
I20260709 12:21:10.253705  6406 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57888 (request call id 238) took 1445 ms. Trace:
I20260709 12:21:10.254150  6406 rpcz_store.cc:276] 0709 12:21:08.808311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:08.808447 (+   136us) service_pool.cc:224] Handling call
0709 12:21:10.253636 (+1445189us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20260709 12:21:10.257441  6407 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57914 (request call id 209) took 1651 ms. Trace:
I20260709 12:21:10.257812  6407 rpcz_store.cc:276] 0709 12:21:08.605843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:08.605977 (+   134us) service_pool.cc:224] Handling call
0709 12:21:10.257375 (+1651398us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:21:09.684863 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:21:09.685426 (+   563us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:21:09.685461 (+    35us) write_op.cc:435] Acquired schema lock
0709 12:21:09.685480 (+    19us) tablet.cc:662] Decoding operations
0709 12:21:09.685811 (+   331us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:21:09.685857 (+    46us) write_op.cc:641] Partition lock acquired for write op
0709 12:21:09.685885 (+    28us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:21:09.686068 (+   183us) tablet.cc:701] Row locks acquired
0709 12:21:09.686086 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:21:09.686223 (+   137us) write_op.cc:270] Start()
0709 12:21:09.686335 (+   112us) write_op.cc:276] Timestamp: P: 1783599669686176 usec, L: 0
0709 12:21:09.686366 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 12:21:09.687149 (+   783us) log.cc:844] Serialized 527 byte log entry
0709 12:21:10.194878 (+507729us) op_driver.cc:464] REPLICATION: finished
0709 12:21:10.252917 (+ 58039us) write_op.cc:301] APPLY: starting
0709 12:21:10.252988 (+    71us) tablet.cc:1367] starting BulkCheckPresence
0709 12:21:10.253752 (+   764us) tablet.cc:1370] finished BulkCheckPresence
0709 12:21:10.253863 (+   111us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:21:10.254569 (+   706us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:21:10.254732 (+   163us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:21:10.254782 (+    50us) write_op.cc:312] APPLY: finished
0709 12:21:10.254955 (+   173us) log.cc:844] Serialized 25 byte log entry
0709 12:21:10.255663 (+   708us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:10.255739 (+    76us) write_op.cc:454] Released schema lock
0709 12:21:10.255773 (+    34us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":57860,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4172,"num_ops":1,"prepare.queue_time_us":944966,"prepare.run_cpu_time_us":2528,"prepare.run_wall_time_us":2526,"replication_time_us":508392,"threads_started":1}]]}
I20260709 12:21:10.264667 15797 raft_consensus.cc:2749] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:21:10.265136  6473 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:10.265822 15797 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:21:10.266242  6473 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:10.266880 15797 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:21:10.267283  6473 leader_election.cc:304] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:10.280966  6473 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:21:10.281005 15955 raft_consensus.cc:2749] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:21:10.287142  6473 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:10.287196 15955 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:21:10.294782 15939 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:21:10.296396  6473 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:10.310729  6473 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:10.311735 15955 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:21:10.321527 15797 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:21:10.356786  6407 catalog_manager.cc:5697] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 reported cstate change: config changed from index 19 to 20, VOTER f2b06aebe97249a4877d798b8118ece4 (127.5.58.3) evicted. New cstate: current_term: 28 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:10.323899  6473 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:21:10.384454  6475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 64242.4us
I20260709 12:21:10.386682 15939 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:21:10.389423  6473 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 65610.6us
I20260709 12:21:10.397751  6473 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:10.401684 15797 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:21:10.404951  6473 leader_election.cc:304] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:10.415633 15797 raft_consensus.cc:2749] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:21:10.414752  6473 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e
I20260709 12:21:10.420276 15797 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:21:10.421599  6473 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:10.455775 15797 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:21:10.457126  6473 leader_election.cc:304] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:10.463974  6640 tablet_service.cc:1558] Processing DeleteTablet for tablet 27473d665a69492b9d08ab7409de0b5f with delete_type TABLET_DATA_TOMBSTONED (TS f2b06aebe97249a4877d798b8118ece4 not found in new config with opid_index 20) from {username='slave'} at 127.0.0.1:56214
I20260709 12:21:10.474427 15797 raft_consensus.cc:2749] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:21:10.475828  6473 leader_election.cc:304] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7554c39679a41bbbca4659b72ab5c65; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:10.482569 15797 raft_consensus.cc:2749] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:21:10.535909  5352 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:10.538668  5352 tablet_replica.cc:333] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.544191  5352 raft_consensus.cc:2243] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.548398  5352 raft_consensus.cc:2272] T 3a47d71c34b541b9b2589b4e80413fa2 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.554589  5352 tablet_replica.cc:333] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.555577  5352 raft_consensus.cc:2243] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.556365  5352 raft_consensus.cc:2272] T 48e50fd265d348779de32dff1ac6db39 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.589444  5352 tablet_replica.cc:333] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.590431  5352 raft_consensus.cc:2243] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:21:10.591647  5352 raft_consensus.cc:2272] T 555356e0b31d45c1bc3963dfe39b2df3 P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.596421 15990 tablet_replica.cc:333] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:10.597636 15990 raft_consensus.cc:2243] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.598600 15990 raft_consensus.cc:2272] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.601902 15990 ts_tablet_manager.cc:1916] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:10.604570  5352 tablet_replica.cc:333] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.638918  5352 raft_consensus.cc:2243] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.648761  5352 raft_consensus.cc:2272] T 99e5417fa1d54082a5690b657f407f70 P e7554c39679a41bbbca4659b72ab5c65 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.651696  5352 tablet_replica.cc:333] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.668789  5352 raft_consensus.cc:2243] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.669829  5352 raft_consensus.cc:2272] T d0bd537a4ec844a69ca8227a47d1a8f3 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.670145 15990 ts_tablet_manager.cc:1929] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 28.19
I20260709 12:21:10.670629 15990 log.cc:1199] T 27473d665a69492b9d08ab7409de0b5f P f2b06aebe97249a4877d798b8118ece4: Deleting WAL directory at /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-2-root/wals/27473d665a69492b9d08ab7409de0b5f
I20260709 12:21:10.672775  5352 tablet_replica.cc:333] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.689229  5352 raft_consensus.cc:2243] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:21:10.690579  5352 raft_consensus.cc:2272] T 55df50d51529421789bb752f47dff41a P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.693198  6394 catalog_manager.cc:5028] TS f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): tablet 27473d665a69492b9d08ab7409de0b5f (table test-workload [id=f93362d0518e483fbe3a01f43a217e48]) successfully deleted
I20260709 12:21:10.702386  5352 tablet_replica.cc:333] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.703612  5352 raft_consensus.cc:2243] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.704314  5352 raft_consensus.cc:2272] T 7492a9464009416aa4c3a11e145dec61 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.707508  5352 tablet_replica.cc:333] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.708600  5352 raft_consensus.cc:2243] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:21:10.709710  5352 raft_consensus.cc:2272] T 6d6c58127aec4100b761ec4b52b842a0 P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.713070  5352 tablet_replica.cc:333] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.713891  5352 raft_consensus.cc:2243] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.714725  5352 raft_consensus.cc:2272] T 27473d665a69492b9d08ab7409de0b5f P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.717655  5352 tablet_replica.cc:333] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.718457  5352 raft_consensus.cc:2243] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.720224  5352 raft_consensus.cc:2272] T 8ea6f08346334787a6b558ca10a70249 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.722949  5352 tablet_replica.cc:333] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.733232  5352 raft_consensus.cc:2243] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:21:10.734550  5352 raft_consensus.cc:2272] T 5b0b7ace18d848afa42e884bcd2475a4 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.737710  5352 tablet_replica.cc:333] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.738549  5352 raft_consensus.cc:2243] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.739279  5352 raft_consensus.cc:2272] T 83301542a88f46cea64bc905b3199975 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.742617  5352 tablet_replica.cc:333] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.743462  5352 raft_consensus.cc:2243] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:21:10.755689  5352 raft_consensus.cc:2272] T 72cc6298c451423d8fe8c456777eb890 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.758384  5352 tablet_replica.cc:333] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.759137  5352 raft_consensus.cc:2243] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.759859  5352 raft_consensus.cc:2272] T ab430742d4f44bf79c4591f613055cd6 P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.770465  5352 tablet_replica.cc:333] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.771348  5352 raft_consensus.cc:2243] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.772022  5352 raft_consensus.cc:2272] T dbf5342782d14193986ec66c46ce646e P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.774827  5352 tablet_replica.cc:333] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.775636  5352 raft_consensus.cc:2243] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.776212  5352 raft_consensus.cc:2272] T e423f656ad70421f959a9a3b81dac708 P e7554c39679a41bbbca4659b72ab5c65 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.795339  5352 tablet_replica.cc:333] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.796200  5352 raft_consensus.cc:2243] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:21:10.797324  5352 raft_consensus.cc:2272] T 8e01550f5e994006a4031ca81edcc9aa P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.816567  5352 tablet_replica.cc:333] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.817476  5352 raft_consensus.cc:2243] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.818084  5352 raft_consensus.cc:2272] T ef43f1290994490e8ef291fa12012731 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.839425  5352 tablet_replica.cc:333] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.846655  5352 raft_consensus.cc:2243] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.847338  5352 raft_consensus.cc:2272] T 13ab2c6da5994ae2a955fc11d1761c41 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.857787  5352 tablet_replica.cc:333] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.858693  5352 raft_consensus.cc:2243] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.859382  5352 raft_consensus.cc:2272] T 6f94b1cd286648a9b82d5abf69bb86d5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.862906  5352 tablet_replica.cc:333] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.865597  5352 raft_consensus.cc:2243] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.866221  5352 raft_consensus.cc:2272] T c6b5198a4b6741cdb13e49cf4e90173c P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.868655  5352 tablet_replica.cc:333] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.869410  5352 raft_consensus.cc:2243] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:21:10.870646  5352 raft_consensus.cc:2272] T 2ebfdf7a730a407185ead1aedcf4b8c4 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.873289  5352 tablet_replica.cc:333] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.874051  5352 raft_consensus.cc:2243] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.874595  5352 raft_consensus.cc:2272] T f9f8f5d393744ebabbed2f3e0131cf40 P e7554c39679a41bbbca4659b72ab5c65 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.877452  5352 tablet_replica.cc:333] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.878230  5352 raft_consensus.cc:2243] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.878890  5352 raft_consensus.cc:2272] T 14cf1c3a63624d94aa6c9c356e6108e2 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.881451  5352 tablet_replica.cc:333] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.883100  5352 raft_consensus.cc:2243] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:21:10.884199  5352 raft_consensus.cc:2272] T 591037e9504e4348a9cdf8842f34be22 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.887423  5352 tablet_replica.cc:333] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.888231  5352 raft_consensus.cc:2243] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:21:10.889286  5352 raft_consensus.cc:2272] T fa09afefaa884dc5943b210cb23a07b3 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.891963  5352 tablet_replica.cc:333] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.892746  5352 raft_consensus.cc:2243] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.893532  5352 raft_consensus.cc:2272] T 23f03345855041f8aabfa3e94a1867d9 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.896377  5352 tablet_replica.cc:333] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.897135  5352 raft_consensus.cc:2243] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.897735  5352 raft_consensus.cc:2272] T 72269ad6925341548796bb34823b3e00 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.900249  5352 tablet_replica.cc:333] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.901126  5352 raft_consensus.cc:2243] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.901672  5352 raft_consensus.cc:2272] T e91732822ea14700aa3230fbb6b1cfac P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.908125  5352 tablet_replica.cc:333] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.908914  5352 raft_consensus.cc:2243] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.909471  5352 raft_consensus.cc:2272] T e416f9423f964ba5a29e536e8160c627 P e7554c39679a41bbbca4659b72ab5c65 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.922451  5352 tablet_replica.cc:333] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.923357  5352 raft_consensus.cc:2243] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.924016  5352 raft_consensus.cc:2272] T f5b42468eaf14fd49be100af079d3921 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.926645  5352 tablet_replica.cc:333] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.927428  5352 raft_consensus.cc:2243] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:21:10.928546  5352 raft_consensus.cc:2272] T c7346d993d414b4197cec453cd7bf127 P e7554c39679a41bbbca4659b72ab5c65 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.944999  5352 tablet_replica.cc:333] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.947005  5352 raft_consensus.cc:2243] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.947877  5352 raft_consensus.cc:2272] T cb7f6a796adb4b8c89579ba77274985f P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.952184  5352 tablet_replica.cc:333] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.953181  5352 raft_consensus.cc:2243] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.953816  5352 raft_consensus.cc:2272] T c50fba48b82e4ea29ce3f314b054d4ec P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.965696  5352 tablet_replica.cc:333] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.966517  5352 raft_consensus.cc:2243] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.967285  5352 raft_consensus.cc:2272] T f2592253d3214104865730480e057d09 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.970209  5352 tablet_replica.cc:333] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.971078  5352 raft_consensus.cc:2243] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.984651  5352 raft_consensus.cc:2272] T bd6fb503c26a49bab14a18c9b5e2d792 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.987522  5352 tablet_replica.cc:333] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.988272  5352 raft_consensus.cc:2243] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.988835  5352 raft_consensus.cc:2272] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:10.991446  5352 tablet_replica.cc:333] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:10.992221  5352 raft_consensus.cc:2243] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:10.992844  5352 raft_consensus.cc:2272] T cbb34f14626f4aa895bae63ea2466e00 P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.006465  5352 tablet_replica.cc:333] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:11.007495  5352 raft_consensus.cc:2243] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.008103  5352 raft_consensus.cc:2272] T 6462df23af154f06b34f32a1ef4a45b2 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.011432  5352 tablet_replica.cc:333] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:11.012306  5352 raft_consensus.cc:2243] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.013477  5352 raft_consensus.cc:2272] T c9d981014b2c4cda80bca766fe58fe42 P e7554c39679a41bbbca4659b72ab5c65 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.022493  5352 tablet_replica.cc:333] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:11.023265  5352 raft_consensus.cc:2243] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.024572  5352 raft_consensus.cc:2272] T 1997e98759d0465ab0e3c607df127068 P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.027310  5352 tablet_replica.cc:333] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:11.038756  5352 raft_consensus.cc:2243] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.039661  5352 raft_consensus.cc:2272] T 3318181e3c0b433a9e1cee517969105c P e7554c39679a41bbbca4659b72ab5c65 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.054555  5352 tablet_replica.cc:333] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:11.055608  5352 raft_consensus.cc:2243] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.056200  5352 raft_consensus.cc:2272] T 53dc690d5068460a820e0ba9e24494ec P e7554c39679a41bbbca4659b72ab5c65 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.058897  5352 tablet_replica.cc:333] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:11.059608  5352 raft_consensus.cc:2243] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.060191  5352 raft_consensus.cc:2272] T 1c226c3677f8446f837400fd0df3062e P e7554c39679a41bbbca4659b72ab5c65 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.062917  5352 tablet_replica.cc:333] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:11.063768  5352 raft_consensus.cc:2243] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.064465  5352 raft_consensus.cc:2272] T b7d79febb5824e78a295b2c993c14108 P e7554c39679a41bbbca4659b72ab5c65 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.067235  5352 tablet_replica.cc:333] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:11.067932  5352 raft_consensus.cc:2243] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.068550  5352 raft_consensus.cc:2272] T c27974d342ea4901aaa3946871539818 P e7554c39679a41bbbca4659b72ab5c65 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.071154  5352 tablet_replica.cc:333] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:11.071887  5352 raft_consensus.cc:2243] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.084708  5352 raft_consensus.cc:2272] T 3e07e97c48184f8e99a4fc5a580724a5 P e7554c39679a41bbbca4659b72ab5c65 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.139712  5352 tablet_replica.cc:333] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:11.140447  5352 raft_consensus.cc:2243] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 30 LEADER]: Raft consensus shutting down.
W20260709 12:21:11.146400  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 67496.8us
W20260709 12:21:11.144575  6392 catalog_manager.cc:4755] TS e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): ChangeConfig:REMOVE_PEER RPC failed for tablet 1997e98759d0465ab0e3c607df127068: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:21:11.188680  5352 raft_consensus.cc:2272] T 3af4a543c38a49bfa8ece9e702577ea1 P e7554c39679a41bbbca4659b72ab5c65 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.191373  5352 tablet_replica.cc:333] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:11.192175  5352 raft_consensus.cc:2243] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.202034  5352 raft_consensus.cc:2272] T c22b6d97ae844b7195c5e9c024203d48 P e7554c39679a41bbbca4659b72ab5c65 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.206828  5352 tablet_replica.cc:333] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:11.208894  5352 raft_consensus.cc:2243] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.211768  5352 raft_consensus.cc:2272] T 7ba56fbb3caa417db1c8d78a9caa28da P e7554c39679a41bbbca4659b72ab5c65 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.214361  5352 tablet_replica.cc:333] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65: stopping tablet replica
I20260709 12:21:11.215219  5352 raft_consensus.cc:2243] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:21:11.216459  5352 raft_consensus.cc:2272] T de5f3f59d0794f4c898c5e7f2b721abe P e7554c39679a41bbbca4659b72ab5c65 [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:11.239755  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 44853.7us
W20260709 12:21:11.272820  7238 consensus_peers.cc:597] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.397567  6548 consensus_peers.cc:597] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.401257  6548 consensus_peers.cc:597] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.409399  7238 connection.cc:573] client connection to 127.5.58.1:36643 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 12:21:11.507087  6548 consensus_peers.cc:597] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.526397  6548 consensus_peers.cc:597] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.582651  6622 consensus_peers.cc:597] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.703549  6622 consensus_peers.cc:597] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.708153  6548 consensus_peers.cc:597] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.720063  6548 consensus_peers.cc:597] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.718924  6622 consensus_peers.cc:597] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.730942  7238 consensus_peers.cc:597] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:11.749156 15977 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:11.749810 15977 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:11.755465 15964 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:11.756070 15964 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:11.752089 15977 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:11.769013  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:11.773135 15964 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:21:11.783108  6622 consensus_peers.cc:597] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.789162  6622 leader_election.cc:336] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:11.789891  6622 consensus_peers.cc:597] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.794191  7238 consensus_peers.cc:597] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.797348  7238 leader_election.cc:336] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:11.797791  7238 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:11.798480 15964 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:21:11.798823  7238 consensus_peers.cc:597] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.815088  6548 consensus_peers.cc:597] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:11.820446  7238 consensus_peers.cc:597] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:11.854854  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 30 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:11.865270  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:11.870425 15993 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:21:11.871832  7238 consensus_peers.cc:597] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:11.980957 15993 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:11.981616 15993 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:11.994722 15993 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:12.001662 15993 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.002990 15993 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:12.004499 16002 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.005093 16003 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.005201 16002 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:12.005666 16003 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:12.009560 16004 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 32 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.009774 15993 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:12.010182 16004 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:12.012967 16002 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:21:12.030635  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 54219.9us
I20260709 12:21:12.013660 16004 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:12.037581 16004 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.038270 16004 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:12.039328 16003 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:12.050812 15977 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.052155 15763 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.052659 16009 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.052729 15763 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:12.053300 16009 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:12.055519 16009 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:12.057730 15763 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:12.057854 16007 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.058425 16007 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:12.055742 16004 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:12.051498 15977 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:12.069582 16007 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:12.061131 16009 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.070906 16009 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:12.072556 15977 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:12.075732 16009 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:12.081305  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:12.082043  6651 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 22.
I20260709 12:21:12.083124 15992 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.083637 15992 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:12.084656  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:12.085304  6658 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 20.
I20260709 12:21:12.085708 15992 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:12.084184  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:12.087978  6656 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 25.
I20260709 12:21:12.089617  7240 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:21:12.090701 15994 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:21:12.091125 15994 raft_consensus.cc:493] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.091434 15994 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:21:12.096021 16005 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.096437  7240 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:21:12.096611 16005 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:12.097410 15997 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:21:12.097816 15997 raft_consensus.cc:493] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.098228 15997 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:21:12.098644 16005 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:12.103833  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:12.104203  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:12.104560 16010 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.104859  7265 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
I20260709 12:21:12.105209 16010 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:12.106460  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:12.107034  7273 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:21:12.107553  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:12.108116  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:12.108219  6653 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 28.
I20260709 12:21:12.108618  6651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 33 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:12.109356  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:12.109740  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 30 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:12.109920  6658 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 20.
I20260709 12:21:12.110301  6649 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 29.
I20260709 12:21:12.109391  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:12.112063  6655 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 27.
I20260709 12:21:12.108729  7269 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 21.
I20260709 12:21:12.116134 15763 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.116719 15763 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:12.118717 15763 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:12.119308 15997 raft_consensus.cc:515] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:12.119648  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:21:12.122651  6548 consensus_peers.cc:597] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:12.128163  6548 consensus_peers.cc:597] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:12.132277  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 135294us
I20260709 12:21:12.133968 16010 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:12.136792 15997 leader_election.cc:290] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 26 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:21:12.136430  6622 consensus_peers.cc:597] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:12.124511  6550 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:21:12.141065  6550 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:21:12.141856 16005 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:21:12.141944  6550 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:21:12.142366 16005 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.142849 16005 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:21:12.143003 15952 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:21:12.143106  6550 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:21:12.143630 15952 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.144039 15952 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:21:12.145103 16004 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:21:12.145515 16004 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.145869 16004 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:21:12.147850  6622 consensus_peers.cc:597] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:12.148542  6622 leader_election.cc:336] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:12.148835 15998 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:21:12.149207  6622 leader_election.cc:336] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:12.149302 15998 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.149837 15998 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:21:12.149935  6622 consensus_peers.cc:597] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:12.151553 15952 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:12.154237 15952 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:21:12.154863  6622 consensus_peers.cc:597] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:12.157433 15997 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:21:12.158143  7238 leader_election.cc:336] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:12.158008 15997 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:12.160421 15997 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:12.161895  6584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:12.162715  6584 raft_consensus.cc:2393] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20a202c76244485fbc8f1610473db8b9 in current term 21: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
W20260709 12:21:12.163002  6622 consensus_peers.cc:597] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:12.163466  6622 leader_election.cc:336] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:12.163363  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
W20260709 12:21:12.164009  6622 leader_election.cc:336] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:12.164117  6577 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 28.
W20260709 12:21:12.166560  6622 consensus_peers.cc:597] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:12.166648  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:21:12.167388  7269 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:21:12.168813 15998 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:12.175204 15998 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:12.169852 16005 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:12.177168 16005 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:21:12.181944  6548 leader_election.cc:336] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:12.183015  6548 leader_election.cc:336] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:12.183141  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ebfdf7a730a407185ead1aedcf4b8c4" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:21:12.184053  6657 raft_consensus.cc:3060] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:21:12.185840 15994 raft_consensus.cc:515] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:12.187126  7266 raft_consensus.cc:2393] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 23: Already voted for candidate 20a202c76244485fbc8f1610473db8b9 in this term.
I20260709 12:21:12.189639  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:12.192848 15994 leader_election.cc:290] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:12.183426  6548 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:12.193894  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e01550f5e994006a4031ca81edcc9aa" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:21:12.194347  6548 leader_election.cc:336] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:12.194547  6655 raft_consensus.cc:3060] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:21:12.195183  6548 leader_election.cc:336] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:12.196076  6548 leader_election.cc:336] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:12.196834  6548 consensus_peers.cc:597] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:12.197255  6548 leader_election.cc:336] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:12.197839  6548 consensus_peers.cc:597] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:12.198572 16002 raft_consensus.cc:2749] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:21:12.202608  6657 raft_consensus.cc:2468] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 26.
I20260709 12:21:12.203822  7240 leader_election.cc:304] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:21:12.209558  6623 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:12.210784  6623 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:12.211499 15992 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:21:12.211915 15992 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.212294 15992 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:21:12.214321 16010 raft_consensus.cc:2804] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:21:12.214867 16010 raft_consensus.cc:697] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 26 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:21:12.215732 16010 consensus_queue.cc:237] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:12.221335  6550 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:12.222776  6655 raft_consensus.cc:2468] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 23.
W20260709 12:21:12.224031  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 65159.4us
I20260709 12:21:12.224370 15993 raft_consensus.cc:2749] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:21:12.224712  7238 consensus_peers.cc:597] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:12.225332  7238 leader_election.cc:336] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:12.227276  7240 leader_election.cc:304] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:21:12.228035 15997 raft_consensus.cc:2804] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:21:12.228562 15997 raft_consensus.cc:697] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 23 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:21:12.229449 15997 consensus_queue.cc:237] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:12.232446 15992 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:12.234961  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:12.240221 16005 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:21:12.221542  6622 consensus_peers.cc:597] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:12.247740 15992 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:12.248941  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:12.250000  6583 raft_consensus.cc:2393] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 21: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:21:12.251345  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:12.251979  6583 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 27.
I20260709 12:21:12.267432  7269 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
W20260709 12:21:12.267639  6548 leader_election.cc:336] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:12.270320  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:21:12.270876  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72cc6298c451423d8fe8c456777eb890" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:21:12.271554  6649 raft_consensus.cc:3060] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:21:12.272666  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591037e9504e4348a9cdf8842f34be22" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:21:12.273339  6658 raft_consensus.cc:3060] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:21:12.275086  7266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6c58127aec4100b761ec4b52b842a0" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:21:12.275698  7266 raft_consensus.cc:3060] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:21:12.270933  6656 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:21:12.282621  6622 leader_election.cc:336] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:12.283314  6649 raft_consensus.cc:2468] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 28.
I20260709 12:21:12.287079  7266 raft_consensus.cc:2468] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 22.
I20260709 12:21:12.207733 16004 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:12.269775  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:12.290550 16004 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 30 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:12.291028  6657 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 27.
I20260709 12:21:12.292307  6658 raft_consensus.cc:2468] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 29.
I20260709 12:21:12.283350  6656 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 21.
I20260709 12:21:12.298647 16005 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.299350 16005 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:12.301649 16005 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:21:12.304250  7238 leader_election.cc:336] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:12.304795  7238 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:12.314124 15993 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.314352 15991 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.314867 15993 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:12.315011 15991 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:12.317482 15993 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:12.317886  6548 leader_election.cc:336] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:12.317500 15991 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:12.318562  6548 leader_election.cc:336] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:12.319097  6548 consensus_peers.cc:597] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:12.319527  6548 leader_election.cc:336] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:12.319626  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:12.320396  7269 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 29.
I20260709 12:21:12.321746  6549 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:21:12.322782 16004 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:21:12.323194 16004 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.323551 16004 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:21:12.323706  6622 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:12.324469  6623 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:12.325013  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:12.325384 16011 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:21:12.325811 15977 raft_consensus.cc:2804] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:21:12.326215 16011 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:21:12.326368 15977 raft_consensus.cc:697] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:21:12.326745 16011 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.327235 16011 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:21:12.327297 15977 consensus_queue.cc:237] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:21:12.332233  6548 consensus_peers.cc:597] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:12.332746  6548 leader_election.cc:336] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:12.333186  6548 leader_election.cc:336] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:12.333231  6550 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:21:12.334611  7238 leader_election.cc:336] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:12.335012  7238 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:12.336513  6550 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:12.338037  6550 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:12.338754 15763 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:21:12.339108 15763 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.339458 15763 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:21:12.340646 16002 raft_consensus.cc:2804] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:21:12.341349 16002 raft_consensus.cc:697] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:21:12.339123  6550 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
W20260709 12:21:12.337921  7238 leader_election.cc:336] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:12.342376 16008 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:21:12.335628 15994 raft_consensus.cc:2749] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:21:12.348763 16005 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:21:12.344247 15952 raft_consensus.cc:2804] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:21:12.350805 15952 raft_consensus.cc:697] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:21:12.351800 15952 consensus_queue.cc:237] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:12.352758 16002 consensus_queue.cc:237] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:21:12.368162  7238 leader_election.cc:336] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:12.357228 16005 raft_consensus.cc:697] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:21:12.377462  7238 consensus_peers.cc:597] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:12.378207  7238 leader_election.cc:336] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:12.378600  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:12.381011 15991 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.377511  6404 catalog_manager.cc:5697] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 25 to 26, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 26 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:12.382081 15991 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:12.382467 15994 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:21:12.389039 15991 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:12.396605 15763 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:12.376072 16005 consensus_queue.cc:237] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:12.403007 15763 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:21:12.411262  6548 consensus_peers.cc:597] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:12.414137  6548 leader_election.cc:336] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:12.415810  5352 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 6530)
W20260709 12:21:12.422782  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 59231.2us
I20260709 12:21:12.427407 16011 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:12.435695 16011 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:12.455999  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:12.456233  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:12.457329  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:12.458290  6580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:21:12.458004  6581 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 28.
I20260709 12:21:12.460455  6580 raft_consensus.cc:2393] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate f2b06aebe97249a4877d798b8118ece4 in current term 28: Already voted for candidate 496c7c3792cd4587b4dbeba435f4be1e in this term.
I20260709 12:21:12.461755  6407 catalog_manager.cc:5697] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 21 to 22, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 22 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:12.464809 15991 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.465400 15991 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:12.467618 15991 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:12.468626 15763 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.469138 15763 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:12.482450 15763 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:12.483814  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "555356e0b31d45c1bc3963dfe39b2df3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 30 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:21:12.484550  6656 raft_consensus.cc:3060] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:21:12.495915  6622 leader_election.cc:336] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:12.496724  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:12.497560 16011 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:21:12.498855  6622 leader_election.cc:336] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:12.499615  6622 leader_election.cc:336] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:12.504029  6622 leader_election.cc:336] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:12.507333  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:12.510874 16021 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:21:12.516252  6656 raft_consensus.cc:2468] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 30.
I20260709 12:21:12.495810 16004 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:12.524966 16004 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 30 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:21:12.535365  6622 consensus_peers.cc:597] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:12.536871  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:21:12.537803  6656 raft_consensus.cc:2393] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 496c7c3792cd4587b4dbeba435f4be1e in current term 28: Already voted for candidate f2b06aebe97249a4877d798b8118ece4 in this term.
I20260709 12:21:12.538429  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9d981014b2c4cda80bca766fe58fe42" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:21:12.539036  7269 raft_consensus.cc:3060] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:21:12.536881  6622 leader_election.cc:336] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:12.545854  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:12.548393  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:12.549129  6658 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 25.
I20260709 12:21:12.550519 16021 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:21:12.554418  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:12.555714  6583 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 19.
I20260709 12:21:12.555781  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:12.556502  7273 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 28.
I20260709 12:21:12.557860  6550 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:12.559284 15763 raft_consensus.cc:2804] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:21:12.559439  6550 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:12.559767 15763 raft_consensus.cc:697] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:21:12.560715 15763 consensus_queue.cc:237] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 29.15, Last appended by leader: 15, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:12.577354  6623 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:12.584406  6550 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:21:12.589304  7269 raft_consensus.cc:2468] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 30.
I20260709 12:21:12.590988  6406 catalog_manager.cc:5697] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 27 to 28, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 28 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:12.593736 16005 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:21:12.590412  6549 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e; no voters: 
I20260709 12:21:12.594216 16005 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.582020 15952 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:21:12.582813 15991 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:21:12.594697 16005 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:21:12.604868 15991 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.605386 15991 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:21:12.606942 16004 raft_consensus.cc:2804] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:21:12.607564 16004 raft_consensus.cc:697] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:21:12.608465 16004 consensus_queue.cc:237] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:21:12.623008  6548 leader_election.cc:336] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:12.612624 16021 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
W20260709 12:21:12.624802  6548 leader_election.cc:336] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:12.617139 16015 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.636607 16015 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:12.638675 16015 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:12.638841  7238 consensus_peers.cc:597] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:12.652683  6622 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:12.660660 16024 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.661252 16024 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:12.663308 16024 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:12.639488 16021 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:12.665843  6657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:12.667275  6657 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 30.
I20260709 12:21:12.668509  6550 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:21:12.677130  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 33 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:12.677959  6583 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 32.
I20260709 12:21:12.682070 16021 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:12.686602  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:21:12.687815 15992 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:21:12.688201 15992 raft_consensus.cc:493] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 32 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.688578 15992 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:21:12.686582  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e" is_pre_election: true
I20260709 12:21:12.697184 16005 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:12.697369 15992 raft_consensus.cc:515] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:12.699672 15992 leader_election.cc:290] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 33 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:12.699714 16024 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:21:12.700426 16024 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.700881 16024 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:21:12.655771 15993 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:21:12.716322 16004 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.716933 16004 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:21:12.717768  6622 leader_election.cc:336] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:12.718912 16004 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:21:12.723325  6622 leader_election.cc:336] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:12.723896  6622 leader_election.cc:336] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:12.723332  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:12.725090  6658 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 27.
I20260709 12:21:12.727644 15993 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.728308 15993 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:21:12.728497  6550 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:21:12.731189 16025 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:21:12.723445 15991 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:12.746033 15991 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:12.753731  7273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:21:12.754463  7273 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:21:12.760684  6548 leader_election.cc:336] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:12.762293  7273 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 29.
I20260709 12:21:12.767279  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:12.768975 16005 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:12.772207 16017 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:21:12.732543 16025 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting leader election (detected failure of leader e7554c39679a41bbbca4659b72ab5c65)
I20260709 12:21:12.774623 16025 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:21:12.775115  6658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba56fbb3caa417db1c8d78a9caa28da" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:21:12.775724  6658 raft_consensus.cc:3060] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:21:12.783337  6622 leader_election.cc:336] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:12.782209  6658 raft_consensus.cc:2468] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 26.
I20260709 12:21:12.786128  6550 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:21:12.790216  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7346d993d414b4197cec453cd7bf127" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 33 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:21:12.790545  6658 raft_consensus.cc:1275] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:21:12.791198  6583 raft_consensus.cc:3060] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:21:12.793665  6658 raft_consensus.cc:1275] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 23 index: 19. (index mismatch)
I20260709 12:21:12.794894  6581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1997e98759d0465ab0e3c607df127068" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:21:12.795593  6581 raft_consensus.cc:3060] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:21:12.798939 16004 raft_consensus.cc:2804] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:21:12.799721 16004 raft_consensus.cc:697] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 26 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:21:12.800788  6622 consensus_peers.cc:597] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:12.806789  6657 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:21:12.808730  6658 raft_consensus.cc:1275] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:21:12.782245 16024 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:12.812599 16007 consensus_queue.cc:1048] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:21:12.815454 16008 consensus_queue.cc:1048] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:21:12.816605 15952 consensus_queue.cc:1048] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:21:12.818369  6583 raft_consensus.cc:2468] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 33.
I20260709 12:21:12.809302 15763 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.012s
W20260709 12:21:12.816882  6548 leader_election.cc:336] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:12.821807  6548 leader_election.cc:336] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:12.820928 16004 consensus_queue.cc:237] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:12.826012 16024 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 31 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:12.828956  6581 raft_consensus.cc:2468] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 29.
I20260709 12:21:12.831245  6622 leader_election.cc:304] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:12.843676  6622 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:12.845851 15991 raft_consensus.cc:2804] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:21:12.846421 15991 raft_consensus.cc:697] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 33 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:21:12.848459  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af4a543c38a49bfa8ece9e702577ea1" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:21:12.849061  6656 raft_consensus.cc:3060] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:21:12.850211 15992 raft_consensus.cc:2804] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:21:12.850739 15992 raft_consensus.cc:697] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 29 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:21:12.851599 15992 consensus_queue.cc:237] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 29, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:21:12.854655  6548 consensus_peers.cc:597] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:12.847345 15991 consensus_queue.cc:237] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 32.17, Last appended by leader: 17, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:12.866904  6656 raft_consensus.cc:2468] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 31.
I20260709 12:21:12.868280  6550 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:21:12.869552 15763 raft_consensus.cc:2804] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:21:12.870054 15763 raft_consensus.cc:697] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 31 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:21:12.870913 15763 consensus_queue.cc:237] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.17, Last appended by leader: 17, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:21:12.878221  7238 consensus_peers.cc:597] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:12.891759  6605 debug-util.cc:398] Leaking SignalData structure 0x7b08002bff20 after lost signal to thread 5355
I20260709 12:21:12.892530 16025 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:21:12.893151  6605 debug-util.cc:398] Leaking SignalData structure 0x7b080037f220 after lost signal to thread 6441
W20260709 12:21:12.894518  6605 debug-util.cc:398] Leaking SignalData structure 0x7b0800307340 after lost signal to thread 6608
W20260709 12:21:12.895745  6605 debug-util.cc:398] Leaking SignalData structure 0x7b08003507a0 after lost signal to thread 6682
I20260709 12:21:12.895799 16025 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:12.899008  6656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
W20260709 12:21:12.900030  6605 debug-util.cc:398] Leaking SignalData structure 0x7b0800229140 after lost signal to thread 7298
I20260709 12:21:12.900067  6656 raft_consensus.cc:3060] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:21:12.916778  6656 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 28.
I20260709 12:21:12.919267  6550 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:21:12.920672 15952 raft_consensus.cc:2804] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:21:12.921154 15952 raft_consensus.cc:697] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 28 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:21:12.953267  6392 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 115032us
I20260709 12:21:12.956118  7269 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:21:12.974542 15763 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.005s
I20260709 12:21:12.986419  7269 raft_consensus.cc:1275] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:21:13.001237 15991 consensus_queue.cc:1048] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:21:12.982553 15952 consensus_queue.cc:237] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:21:13.027747  6548 leader_election.cc:336] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:13.028895  6548 consensus_peers.cc:597] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:13.035914  6656 raft_consensus.cc:1275] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:21:13.054782 15993 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:21:13.079360  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 89349.8us
W20260709 12:21:13.107223  6392 catalog_manager.cc:4755] TS e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 3318181e3c0b433a9e1cee517969105c: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:13.107939  6548 consensus_peers.cc:597] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:13.114475 15993 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:13.059420 15998 consensus_queue.cc:1048] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.056s
W20260709 12:21:13.128557  6548 leader_election.cc:336] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:13.139438  6583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:21:13.140053  6583 raft_consensus.cc:3060] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:21:13.163522  7266 raft_consensus.cc:1275] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 30 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:21:13.165555 15998 consensus_queue.cc:1048] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:21:13.169520  6656 raft_consensus.cc:1275] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 29 index: 15. Preceding OpId from leader: term: 30 index: 16. (index mismatch)
I20260709 12:21:13.182922  6407 catalog_manager.cc:5697] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 28 to 29, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 29 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } }
I20260709 12:21:13.189531  6407 catalog_manager.cc:5697] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 22 to 23, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 23 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:13.172252 16002 consensus_queue.cc:1048] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:21:13.190802  6407 catalog_manager.cc:5697] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 28 to 29, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 29 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:13.193432  6407 catalog_manager.cc:5697] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 32 to 33, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 33 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:13.195746  6407 catalog_manager.cc:5697] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 20 to 21, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 21 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:13.206180  6583 raft_consensus.cc:2468] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 20.
I20260709 12:21:13.208549  6622 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
W20260709 12:21:13.226827  7238 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111) [suppressed 354 similar messages]
I20260709 12:21:13.227111 15993 raft_consensus.cc:2804] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:21:13.233202 15993 raft_consensus.cc:697] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:21:13.234136 15993 consensus_queue.cc:237] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:21:13.245775  6548 consensus_peers.cc:597] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:13.258544  6622 leader_election.cc:336] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:13.265632  6583 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "55df50d51529421789bb752f47dff41a"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:57158
I20260709 12:21:13.266320  6583 raft_consensus.cc:606] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Received request to transfer leadership
W20260709 12:21:13.289687  6548 consensus_peers.cc:597] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:13.292891  6577 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "55df50d51529421789bb752f47dff41a"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:57158
I20260709 12:21:13.293748  6577 raft_consensus.cc:606] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Received request to transfer leadership
I20260709 12:21:13.272537 16002 raft_consensus.cc:993] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e: : Instructing follower 20a202c76244485fbc8f1610473db8b9 to start an election
I20260709 12:21:13.308211 16004 raft_consensus.cc:1081] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Signalling peer 20a202c76244485fbc8f1610473db8b9 to start an election
I20260709 12:21:13.310822  7273 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "55df50d51529421789bb752f47dff41a"
dest_uuid: "20a202c76244485fbc8f1610473db8b9"
 from {username='slave'} at 127.0.0.1:38562
I20260709 12:21:13.311396  7273 raft_consensus.cc:493] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:13.311760  7273 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:21:13.315802  6577 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "55df50d51529421789bb752f47dff41a"
dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:57158
I20260709 12:21:13.316641  6577 raft_consensus.cc:606] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Received request to transfer leadership
I20260709 12:21:13.324304  7273 raft_consensus.cc:515] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:21:13.326467  6455 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 12:21:13.326921  6455 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 55df50d51529421789bb752f47dff41a already in progress
W20260709 12:21:13.327201  6455 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 55df50d51529421789bb752f47dff41a already in progress
W20260709 12:21:13.332202  7238 leader_election.cc:336] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:13.346760  6577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: true dest_uuid: "496c7c3792cd4587b4dbeba435f4be1e"
I20260709 12:21:13.347414  6577 raft_consensus.cc:3055] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:21:13.347712  6577 raft_consensus.cc:740] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
I20260709 12:21:13.348528  6577 consensus_queue.cc:260] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:13.350066  6577 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:21:13.357919  7265 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [term 29 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:21:13.358876  6577 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 22.
I20260709 12:21:13.360409 16015 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:21:13.367461  7273 leader_election.cc:290] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 22 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:13.381767  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55df50d51529421789bb752f47dff41a" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: true dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:21:13.382467  6653 raft_consensus.cc:3060] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:21:13.384946  6656 raft_consensus.cc:1275] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:21:13.383315  6404 catalog_manager.cc:5697] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 29 to 30, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 30 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:13.391068 16002 consensus_queue.cc:1048] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:21:13.399904  6622 consensus_peers.cc:597] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:13.416589  5352 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 6530)
I20260709 12:21:13.417869  6656 raft_consensus.cc:1275] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 31 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 30 index: 17. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 12:21:13.425590 16043 consensus_queue.cc:1048] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:21:13.437197  6653 raft_consensus.cc:2468] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 22.
I20260709 12:21:13.438702  7240 leader_election.cc:304] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:13.452199 16008 raft_consensus.cc:2804] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:21:13.453037 16008 raft_consensus.cc:697] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [term 22 LEADER]: Becoming Leader. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Running, Role: LEADER
I20260709 12:21:13.454054 16008 consensus_queue.cc:237] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:13.457576  6407 catalog_manager.cc:5697] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 25 to 26, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 26 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:13.459590  6407 catalog_manager.cc:5697] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 30 to 31, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 31 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:13.461311  6407 catalog_manager.cc:5697] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 29 to 30, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 30 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:13.465451  6407 catalog_manager.cc:5697] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 27 to 28, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 28 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:13.517855  6549 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44857, user_credentials={real_user=slave}} blocked reactor thread for 46723.2us
I20260709 12:21:13.539427  6577 raft_consensus.cc:1275] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 33 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 32 index: 17. Preceding OpId from leader: term: 33 index: 18. (index mismatch)
I20260709 12:21:13.564278  6577 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
W20260709 12:21:13.598335  6548 consensus_peers.cc:597] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:13.600469  6406 catalog_manager.cc:5697] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 19 to 20, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 20 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } }
W20260709 12:21:13.632655 16060 log.cc:927] Time spent T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
W20260709 12:21:13.650260  6548 consensus_peers.cc:597] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:13.651018  6548 consensus_peers.cc:597] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:13.672967  6406 catalog_manager.cc:5697] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 reported cstate change: term changed from 21 to 22, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to 20a202c76244485fbc8f1610473db8b9 (127.5.58.4). New cstate: current_term: 22 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:21:13.687810  6622 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:45101, user_credentials={real_user=slave}} blocked reactor thread for 64607.5us
I20260709 12:21:13.702028  6653 raft_consensus.cc:1275] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
W20260709 12:21:13.704213  6548 consensus_peers.cc:597] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:13.708683 16000 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:21:13.710206 16028 consensus_queue.cc:1048] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:21:13.713274 15991 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:21:13.805997  6583 raft_consensus.cc:1275] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 12:21:13.838877  6622 consensus_peers.cc:597] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:13.864123 16067 consensus_queue.cc:1048] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:21:14.021672  6548 consensus_peers.cc:597] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.090812  6622 consensus_peers.cc:597] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:14.130802  7238 consensus_peers.cc:597] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:14.131441  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 50496.7us
I20260709 12:21:14.176105  6578 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:21:14.194527 16008 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.003s
I20260709 12:21:14.197109  6655 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:21:14.198788 16008 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:21:14.210294  6622 consensus_peers.cc:597] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.236325  7238 consensus_peers.cc:597] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.252914  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 58842.1us
W20260709 12:21:14.282687  6622 consensus_peers.cc:597] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.304222  6622 consensus_peers.cc:597] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.378608  6548 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 53751.5us
W20260709 12:21:14.417313  5352 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 6530)
W20260709 12:21:14.438156  6622 consensus_peers.cc:597] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.481798  6548 consensus_peers.cc:597] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.490253  6548 consensus_peers.cc:597] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.493218  7238 consensus_peers.cc:597] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.498986  6548 consensus_peers.cc:597] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.500892  7238 consensus_peers.cc:597] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.501454  6548 consensus_peers.cc:597] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.504083  7238 consensus_peers.cc:597] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.503954  6548 consensus_peers.cc:597] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.507472  6548 consensus_peers.cc:597] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:14.567716 16000 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:14.568455 16000 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:14.573197  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:14.574165  6653 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 28.
I20260709 12:21:14.576226  6550 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:21:14.577353 16000 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:14.577420 16002 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:21:14.578029 16002 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:14.578423 16002 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:21:14.594698  7238 consensus_peers.cc:597] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.600374  7238 consensus_peers.cc:597] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:14.603240 16002 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:14.606779  6653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b0b7ace18d848afa42e884bcd2475a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4"
I20260709 12:21:14.607494  6653 raft_consensus.cc:3060] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:21:14.613971 16002 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 election: Requested vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:21:14.616576  6548 leader_election.cc:336] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:14.628368  6653 raft_consensus.cc:2468] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 496c7c3792cd4587b4dbeba435f4be1e in term 29.
W20260709 12:21:14.635044  6622 consensus_peers.cc:597] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:14.652230  6550 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:21:14.653501 16000 raft_consensus.cc:2804] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:21:14.654049 16000 raft_consensus.cc:697] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 LEADER]: Becoming Leader. State: Replica: 496c7c3792cd4587b4dbeba435f4be1e, State: Running, Role: LEADER
W20260709 12:21:14.654158  6548 leader_election.cc:336] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:14.655133  6622 consensus_peers.cc:597] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:14.655072 16000 consensus_queue.cc:237] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:21:14.660524  6622 consensus_peers.cc:597] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.683785  6622 consensus_peers.cc:597] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.701542  6548 consensus_peers.cc:597] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.705391  6548 consensus_peers.cc:597] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.705880  6548 consensus_peers.cc:597] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.715898  6622 consensus_peers.cc:597] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:14.724861  6407 catalog_manager.cc:5697] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: term changed from 27 to 29, leader changed from e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1) to 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2). New cstate: current_term: 29 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:14.759806  6622 consensus_peers.cc:597] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.760556  6622 consensus_peers.cc:597] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.764168  6548 consensus_peers.cc:597] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.890151  6622 consensus_peers.cc:597] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.919088  7238 consensus_peers.cc:597] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:14.986662  7238 consensus_peers.cc:597] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:14.990065  6548 consensus_peers.cc:597] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:15.033126  6548 consensus_peers.cc:597] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:15.139438  6653 raft_consensus.cc:1275] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:21:15.142283 16002 consensus_queue.cc:1048] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 12:21:15.167121  6548 consensus_peers.cc:597] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:15.198719  7273 consensus_queue.cc:237] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:15.199180  6655 consensus_queue.cc:237] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.18, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:15.209239  6658 raft_consensus.cc:1275] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Refusing update from remote peer 20a202c76244485fbc8f1610473db8b9: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:21:15.214869 16010 consensus_queue.cc:1048] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.002s
W20260709 12:21:15.221632  7238 consensus_peers.cc:597] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:15.226924 16008 raft_consensus.cc:2955] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [term 22 LEADER]: Committing config change with OpId 22.16: config changed from index 13 to 16, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:15.230259  6657 raft_consensus.cc:2955] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Committing config change with OpId 22.16: config changed from index 13 to 16, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:21:15.234437  7238 consensus_peers.cc:597] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:15.260797  6622 consensus_peers.cc:597] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:15.316432  7273 raft_consensus.cc:1275] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [term 29 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 29 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:21:15.320004 16027 consensus_queue.cc:1048] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:21:15.345364  6393 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 55df50d51529421789bb752f47dff41a with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 9)
W20260709 12:21:15.364892  6622 consensus_peers.cc:597] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:15.369926  6406 catalog_manager.cc:5697] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 reported cstate change: config changed from index 13 to 16, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New cstate: current_term: 22 leader_uuid: "20a202c76244485fbc8f1610473db8b9" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:15.370942 16027 raft_consensus.cc:2955] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 29 LEADER]: Committing config change with OpId 29.19: config changed from index 17 to 19, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:15.381579  7273 raft_consensus.cc:2955] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [term 29 FOLLOWER]: Committing config change with OpId 29.19: config changed from index 17 to 19, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
W20260709 12:21:15.395226  6622 consensus_peers.cc:597] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:15.396625  6392 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 81596.1us
W20260709 12:21:15.412287  6392 catalog_manager.cc:4755] TS e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 3318181e3c0b433a9e1cee517969105c: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:15.417423  6394 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1997e98759d0465ab0e3c607df127068 with cas_config_opid_index 17: ChangeConfig:REMOVE_PEER succeeded (attempt 9)
W20260709 12:21:15.417842  5352 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 6530)
W20260709 12:21:15.494787  6548 consensus_peers.cc:597] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:15.507249  6568 tablet_service.cc:1558] Processing DeleteTablet for tablet 55df50d51529421789bb752f47dff41a with delete_type TABLET_DATA_TOMBSTONED (TS 496c7c3792cd4587b4dbeba435f4be1e not found in new config with opid_index 16) from {username='slave'} at 127.0.0.1:57158
I20260709 12:21:15.517509 16096 tablet_replica.cc:333] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:15.518987  6407 catalog_manager.cc:5697] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 reported cstate change: config changed from index 17 to 19, VOTER 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) evicted. New cstate: current_term: 29 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:15.599735  6568 tablet_service.cc:1558] Processing DeleteTablet for tablet 1997e98759d0465ab0e3c607df127068 with delete_type TABLET_DATA_TOMBSTONED (TS 496c7c3792cd4587b4dbeba435f4be1e not found in new config with opid_index 19) from {username='slave'} at 127.0.0.1:57158
W20260709 12:21:15.604300  7238 consensus_peers.cc:597] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:15.614194 16096 raft_consensus.cc:2243] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:15.615621 16096 raft_consensus.cc:2272] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:15.619053 16096 ts_tablet_manager.cc:1916] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:15.671691 16096 ts_tablet_manager.cc:1929] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 22.15
W20260709 12:21:15.679215  6548 consensus_peers.cc:597] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:15.696964 16096 log.cc:1199] T 55df50d51529421789bb752f47dff41a P 496c7c3792cd4587b4dbeba435f4be1e: Deleting WAL directory at /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-1-root/wals/55df50d51529421789bb752f47dff41a
I20260709 12:21:15.699921 16096 tablet_replica.cc:333] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:15.702659  6392 catalog_manager.cc:5028] TS 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): tablet 55df50d51529421789bb752f47dff41a (table test-workload [id=f93362d0518e483fbe3a01f43a217e48]) successfully deleted
I20260709 12:21:15.711776 16096 raft_consensus.cc:2243] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:15.721068 16096 raft_consensus.cc:2272] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:15.725636  6548 consensus_peers.cc:597] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:15.744585  6548 consensus_peers.cc:597] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:15.748006 16096 ts_tablet_manager.cc:1916] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:15.814260 16096 ts_tablet_manager.cc:1929] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 29.18
I20260709 12:21:15.814759 16096 log.cc:1199] T 1997e98759d0465ab0e3c607df127068 P 496c7c3792cd4587b4dbeba435f4be1e: Deleting WAL directory at /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599341860723-5352-0/minicluster-data/ts-1-root/wals/1997e98759d0465ab0e3c607df127068
I20260709 12:21:15.816947  6392 catalog_manager.cc:5028] TS 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): tablet 1997e98759d0465ab0e3c607df127068 (table test-workload [id=f93362d0518e483fbe3a01f43a217e48]) successfully deleted
W20260709 12:21:15.872674  6548 consensus_peers.cc:597] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:15.977170  6548 consensus_peers.cc:597] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:16.053186  6548 consensus_peers.cc:597] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:16.282434  6622 consensus_peers.cc:597] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:16.299872  6622 consensus_peers.cc:597] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:16.312462  6548 consensus_peers.cc:597] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:16.379663  6548 consensus_peers.cc:597] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:16.418299  5352 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 6530)
W20260709 12:21:16.594610  7238 consensus_peers.cc:597] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:16.612236  7238 consensus_peers.cc:597] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:16.630174  6548 consensus_peers.cc:597] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:16.721112  6622 consensus_peers.cc:597] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:16.732364  6622 consensus_peers.cc:597] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:16.866541  6548 consensus_peers.cc:597] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:16.903088  7238 consensus_peers.cc:597] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:16.910034  6622 consensus_peers.cc:597] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:16.941850  6548 consensus_peers.cc:597] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:16.942067  6622 consensus_peers.cc:597] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:16.950692  6548 consensus_peers.cc:597] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:16.988128  6548 consensus_peers.cc:597] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:17.023401  6622 consensus_peers.cc:597] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:17.037336  6622 consensus_peers.cc:597] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:17.072314  6658 consensus_queue.cc:237] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.14, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
W20260709 12:21:17.100265  7238 consensus_peers.cc:597] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:17.113051  6623 consensus_peers.cc:597] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d0bd537a4ec844a69ca8227a47d1a8f3. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:17.117470  6581 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 27 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:21:17.121451 16027 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:21:17.122483  7238 consensus_peers.cc:597] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:17.129042  6622 consensus_peers.cc:597] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:17.154654  6622 consensus_peers.cc:597] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:17.161124  7238 consensus_peers.cc:597] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:17.195412  6548 consensus_peers.cc:597] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:17.199892 16100 raft_consensus.cc:2955] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 27 LEADER]: Committing config change with OpId 27.15: config changed from index -1 to 15, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:21:17.249177  6581 raft_consensus.cc:2955] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Committing config change with OpId 27.15: config changed from index -1 to 15, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:21:17.282076  6583 consensus_queue.cc:237] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
W20260709 12:21:17.287706  7238 consensus_peers.cc:597] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:17.287117  6622 consensus_peers.cc:597] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:17.311278  6549 consensus_peers.cc:597] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: de5f3f59d0794f4c898c5e7f2b721abe. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:17.315613  6407 catalog_manager.cc:5697] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: config changed from index -1 to 15, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 27 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:17.318775  6658 raft_consensus.cc:1275] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:21:17.321298 16084 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 12:21:17.322674  6622 consensus_peers.cc:597] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:17.323261  6622 consensus_peers.cc:597] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:17.328815  6548 consensus_peers.cc:597] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:17.332214  6622 consensus_peers.cc:597] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:17.334812  6548 consensus_peers.cc:597] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:17.335433  6548 consensus_peers.cc:597] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:17.381814  6655 raft_consensus.cc:2955] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Committing config change with OpId 28.18: config changed from index -1 to 18, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:21:17.375151 16084 raft_consensus.cc:2955] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 28 LEADER]: Committing config change with OpId 28.18: config changed from index -1 to 18, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
W20260709 12:21:17.428517  6548 consensus_peers.cc:597] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:17.432847  6622 consensus_peers.cc:597] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:17.463882  6622 consensus_peers.cc:597] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:17.484886  6548 consensus_peers.cc:597] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:17.487630  6581 consensus_queue.cc:237] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 34.18, Last appended by leader: 17, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
W20260709 12:21:17.490702  6548 consensus_peers.cc:597] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:17.500113  6406 catalog_manager.cc:5697] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 reported cstate change: config changed from index -1 to 18, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 28 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:21:17.519436  6655 raft_consensus.cc:1275] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 34 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 34 index: 18. Preceding OpId from leader: term: 34 index: 19. (index mismatch)
I20260709 12:21:17.521011 16112 consensus_queue.cc:1048] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 12:21:17.522614  6548 consensus_peers.cc:597] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:17.522765  6549 consensus_peers.cc:597] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e423f656ad70421f959a9a3b81dac708. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:17.553134 16115 ts_tablet_manager.cc:933] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:17.565325 16115 tablet_copy_client.cc:323] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.3:36107
I20260709 12:21:17.571280  6668 tablet_copy_service.cc:140] P f2b06aebe97249a4877d798b8118ece4: Received BeginTabletCopySession request for tablet d0bd537a4ec844a69ca8227a47d1a8f3 from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:49792)
I20260709 12:21:17.572158  6668 tablet_copy_service.cc:161] P f2b06aebe97249a4877d798b8118ece4: Beginning new tablet copy session on tablet d0bd537a4ec844a69ca8227a47d1a8f3 from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:49792: session id = 20a202c76244485fbc8f1610473db8b9-d0bd537a4ec844a69ca8227a47d1a8f3
I20260709 12:21:17.583330 16104 raft_consensus.cc:2955] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 34 LEADER]: Committing config change with OpId 34.19: config changed from index -1 to 19, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
W20260709 12:21:17.596540  7238 consensus_peers.cc:597] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:17.624109  6655 raft_consensus.cc:2955] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 34 FOLLOWER]: Committing config change with OpId 34.19: config changed from index -1 to 19, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:21:17.634382  6668 tablet_copy_source_session.cc:215] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:17.658090 16115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0bd537a4ec844a69ca8227a47d1a8f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:17.666764  6655 consensus_queue.cc:237] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
W20260709 12:21:17.709955  6623 consensus_peers.cc:597] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 -> Peer 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857): Couldn't send request to peer 20a202c76244485fbc8f1610473db8b9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7492a9464009416aa4c3a11e145dec61. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:17.715006  6407 catalog_manager.cc:5697] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 reported cstate change: config changed from index -1 to 19, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 34 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
W20260709 12:21:17.734118  6550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 83544.6us
I20260709 12:21:17.738024  6581 raft_consensus.cc:1275] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
W20260709 12:21:17.768208  6548 consensus_peers.cc:597] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:17.776575  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 75738.1us
I20260709 12:21:17.804785 16099 consensus_queue.cc:1048] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:21:17.813151  7238 consensus_peers.cc:597] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:17.830838 16122 ts_tablet_manager.cc:933] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:17.844511  6622 consensus_peers.cc:597] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:17.867848 16115 tablet_copy_client.cc:806] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:17.878507 16115 tablet_copy_client.cc:670] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:17.882429 16122 tablet_copy_client.cc:323] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.2:45101
W20260709 12:21:17.892359  7238 consensus_peers.cc:597] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:17.913133  6594 tablet_copy_service.cc:140] P 496c7c3792cd4587b4dbeba435f4be1e: Received BeginTabletCopySession request for tablet de5f3f59d0794f4c898c5e7f2b721abe from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:54204)
I20260709 12:21:17.913872  6594 tablet_copy_service.cc:161] P 496c7c3792cd4587b4dbeba435f4be1e: Beginning new tablet copy session on tablet de5f3f59d0794f4c898c5e7f2b721abe from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:54204: session id = 20a202c76244485fbc8f1610473db8b9-de5f3f59d0794f4c898c5e7f2b721abe
I20260709 12:21:17.923682  6594 tablet_copy_source_session.cc:215] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:17.928953 16115 tablet_copy_client.cc:538] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:17.933766 16100 raft_consensus.cc:2955] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 24 LEADER]: Committing config change with OpId 24.13: config changed from index -1 to 13, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:21:17.945137 16122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de5f3f59d0794f4c898c5e7f2b721abe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:18.060293  6583 raft_consensus.cc:2955] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Committing config change with OpId 24.13: config changed from index -1 to 13, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:21:18.097223 16115 tablet_bootstrap.cc:492] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
W20260709 12:21:18.146901  6548 consensus_peers.cc:597] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:18.150452  6407 catalog_manager.cc:5697] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e reported cstate change: config changed from index -1 to 13, NON_VOTER 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) added. New cstate: current_term: 24 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } }
I20260709 12:21:18.185257 16127 ts_tablet_manager.cc:933] T e423f656ad70421f959a9a3b81dac708 P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:18.194833  6622 consensus_peers.cc:597] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:18.203001 16127 tablet_copy_client.cc:323] T e423f656ad70421f959a9a3b81dac708 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.2:45101
I20260709 12:21:18.216464  6594 tablet_copy_service.cc:140] P 496c7c3792cd4587b4dbeba435f4be1e: Received BeginTabletCopySession request for tablet e423f656ad70421f959a9a3b81dac708 from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:54204)
I20260709 12:21:18.217133  6594 tablet_copy_service.cc:161] P 496c7c3792cd4587b4dbeba435f4be1e: Beginning new tablet copy session on tablet e423f656ad70421f959a9a3b81dac708 from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:54204: session id = 20a202c76244485fbc8f1610473db8b9-e423f656ad70421f959a9a3b81dac708
I20260709 12:21:18.229218  6594 tablet_copy_source_session.cc:215] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:18.233177 16127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e423f656ad70421f959a9a3b81dac708. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:18.246189 16122 tablet_copy_client.cc:806] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
W20260709 12:21:18.254369  6622 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111) [suppressed 490 similar messages]
I20260709 12:21:18.300813 16122 tablet_copy_client.cc:670] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:18.326699 16129 ts_tablet_manager.cc:933] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9: Initiating tablet copy from peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:18.328819 16122 tablet_copy_client.cc:538] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:18.352638 16129 tablet_copy_client.cc:323] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Beginning tablet copy session from remote peer at address 127.5.58.3:36107
I20260709 12:21:18.355145  6668 tablet_copy_service.cc:140] P f2b06aebe97249a4877d798b8118ece4: Received BeginTabletCopySession request for tablet 7492a9464009416aa4c3a11e145dec61 from peer 20a202c76244485fbc8f1610473db8b9 ({username='slave'} at 127.0.0.1:49792)
I20260709 12:21:18.355692  6668 tablet_copy_service.cc:161] P f2b06aebe97249a4877d798b8118ece4: Beginning new tablet copy session on tablet 7492a9464009416aa4c3a11e145dec61 from peer 20a202c76244485fbc8f1610473db8b9 at {username='slave'} at 127.0.0.1:49792: session id = 20a202c76244485fbc8f1610473db8b9-7492a9464009416aa4c3a11e145dec61
I20260709 12:21:18.389079 16122 tablet_bootstrap.cc:492] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
W20260709 12:21:18.390398  7238 consensus_peers.cc:597] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:18.394217  5352 tablet_server.cc:196] TabletServer@127.5.58.1:0 shutdown complete.
I20260709 12:21:18.402665  6668 tablet_copy_source_session.cc:215] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:21:18.404568  6548 consensus_peers.cc:597] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:18.405333  6548 consensus_peers.cc:597] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:18.406491 16129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7492a9464009416aa4c3a11e145dec61. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:21:18.427354  6548 consensus_peers.cc:597] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:18.434967  6548 consensus_peers.cc:597] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:18.443008 16127 tablet_copy_client.cc:806] T e423f656ad70421f959a9a3b81dac708 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:18.448262 16127 tablet_copy_client.cc:670] T e423f656ad70421f959a9a3b81dac708 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:18.453111 16127 tablet_copy_client.cc:538] T e423f656ad70421f959a9a3b81dac708 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:18.478703 16127 tablet_bootstrap.cc:492] T e423f656ad70421f959a9a3b81dac708 P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
I20260709 12:21:18.507143 16129 tablet_copy_client.cc:806] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:18.508620 16129 tablet_copy_client.cc:670] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:18.516217 16129 tablet_copy_client.cc:538] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:21:18.542897  6622 consensus_peers.cc:597] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:18.590435 16129 tablet_bootstrap.cc:492] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9: Bootstrap starting.
W20260709 12:21:18.641956  6548 consensus_peers.cc:597] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:18.741080 16115 tablet_bootstrap.cc:492] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=16 overwritten=1 applied=15 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:18.742555 16115 tablet_bootstrap.cc:492] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:21:18.743625 16115 ts_tablet_manager.cc:1403] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 0.647s	user 0.039s	sys 0.024s
I20260709 12:21:18.767557 16115 raft_consensus.cc:359] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9 [term 27 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:21:18.796792 16115 raft_consensus.cc:740] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9 [term 27 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
I20260709 12:21:18.797842 16115 consensus_queue.cc:260] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:21:18.833704  5352 tablet_server.cc:179] TabletServer@127.5.58.2:0 shutting down...
I20260709 12:21:18.834403 16115 ts_tablet_manager.cc:1434] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.090s	user 0.008s	sys 0.003s
I20260709 12:21:18.854115  6668 tablet_copy_service.cc:342] P f2b06aebe97249a4877d798b8118ece4: Request end of tablet copy session 20a202c76244485fbc8f1610473db8b9-d0bd537a4ec844a69ca8227a47d1a8f3 received from {username='slave'} at 127.0.0.1:49792
I20260709 12:21:18.854765  6668 tablet_copy_service.cc:434] P f2b06aebe97249a4877d798b8118ece4: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-d0bd537a4ec844a69ca8227a47d1a8f3 on tablet d0bd537a4ec844a69ca8227a47d1a8f3 with peer 20a202c76244485fbc8f1610473db8b9
W20260709 12:21:18.860361  6622 consensus_peers.cc:597] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:18.919597  6548 consensus_peers.cc:597] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:18.934370 16127 tablet_bootstrap.cc:492] T e423f656ad70421f959a9a3b81dac708 P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=20 overwritten=1 applied=19 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:18.935508 16127 tablet_bootstrap.cc:492] T e423f656ad70421f959a9a3b81dac708 P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:21:18.936365 16127 ts_tablet_manager.cc:1403] T e423f656ad70421f959a9a3b81dac708 P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 0.458s	user 0.066s	sys 0.022s
I20260709 12:21:18.940332 16127 raft_consensus.cc:359] T e423f656ad70421f959a9a3b81dac708 P 20a202c76244485fbc8f1610473db8b9 [term 34 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:21:18.941366 16127 raft_consensus.cc:740] T e423f656ad70421f959a9a3b81dac708 P 20a202c76244485fbc8f1610473db8b9 [term 34 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
I20260709 12:21:18.944484 16127 consensus_queue.cc:260] T e423f656ad70421f959a9a3b81dac708 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 19, Last appended: 34.19, Last appended by leader: 19, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:21:18.950493 16127 ts_tablet_manager.cc:1434] T e423f656ad70421f959a9a3b81dac708 P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
W20260709 12:21:18.962631  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 68079.8us
I20260709 12:21:18.987164 16129 tablet_bootstrap.cc:492] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=0 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:18.988423 16129 tablet_bootstrap.cc:492] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:21:18.989319 16129 ts_tablet_manager.cc:1403] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 0.399s	user 0.051s	sys 0.003s
I20260709 12:21:18.990428 16122 tablet_bootstrap.cc:492] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=18 overwritten=0 applied=18 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:18.992667 16122 tablet_bootstrap.cc:492] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9: Bootstrap complete.
I20260709 12:21:19.018098 16122 ts_tablet_manager.cc:1403] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9: Time spent bootstrapping tablet: real 0.629s	user 0.059s	sys 0.028s
I20260709 12:21:19.017278 16129 raft_consensus.cc:359] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [term 24 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:21:19.035784  5352 tablet_copy_service.cc:362] P 496c7c3792cd4587b4dbeba435f4be1e: Destroying tablet copy session 20a202c76244485fbc8f1610473db8b9-e423f656ad70421f959a9a3b81dac708 due to service shutdown
I20260709 12:21:19.036262  5352 tablet_copy_service.cc:434] P 496c7c3792cd4587b4dbeba435f4be1e: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-e423f656ad70421f959a9a3b81dac708 on tablet e423f656ad70421f959a9a3b81dac708 with peer 20a202c76244485fbc8f1610473db8b9
I20260709 12:21:19.036834  5352 tablet_copy_service.cc:362] P 496c7c3792cd4587b4dbeba435f4be1e: Destroying tablet copy session 20a202c76244485fbc8f1610473db8b9-de5f3f59d0794f4c898c5e7f2b721abe due to service shutdown
I20260709 12:21:19.037097  5352 tablet_copy_service.cc:434] P 496c7c3792cd4587b4dbeba435f4be1e: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-de5f3f59d0794f4c898c5e7f2b721abe on tablet de5f3f59d0794f4c898c5e7f2b721abe with peer 20a202c76244485fbc8f1610473db8b9
I20260709 12:21:19.044075 16129 raft_consensus.cc:740] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [term 24 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
I20260709 12:21:19.056496  7269 raft_consensus.cc:1217] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9 [term 27 LEARNER]: Deduplicated request from leader. Original: 27.14->[27.15-27.15]   Dedup: 27.15->[]
W20260709 12:21:19.059680  6622 consensus_peers.cc:597] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:19.060387  6622 consensus_peers.cc:597] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:19.059870  7238 consensus_peers.cc:597] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:19.067996 16129 consensus_queue.cc:260] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:21:19.069223 16122 raft_consensus.cc:359] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [term 28 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:21:19.070345 16122 raft_consensus.cc:740] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [term 28 LEARNER]: Becoming Follower/Learner. State: Replica: 20a202c76244485fbc8f1610473db8b9, State: Initialized, Role: LEARNER
W20260709 12:21:19.075456  7238 consensus_peers.cc:597] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:19.082647  7238 consensus_peers.cc:597] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:19.085757  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 53923.2us
W20260709 12:21:19.092475  7238 consensus_peers.cc:597] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:19.093995  7238 consensus_peers.cc:597] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:19.113022 16129 ts_tablet_manager.cc:1434] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.123s	user 0.011s	sys 0.000s
I20260709 12:21:19.080363 16122 consensus_queue.cc:260] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
W20260709 12:21:19.118384  6548 consensus_peers.cc:597] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:19.121284  6622 consensus_peers.cc:597] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:19.125758  6622 consensus_peers.cc:597] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:19.133530  6668 tablet_copy_service.cc:342] P f2b06aebe97249a4877d798b8118ece4: Request end of tablet copy session 20a202c76244485fbc8f1610473db8b9-7492a9464009416aa4c3a11e145dec61 received from {username='slave'} at 127.0.0.1:49792
I20260709 12:21:19.137463  6668 tablet_copy_service.cc:434] P f2b06aebe97249a4877d798b8118ece4: ending tablet copy session 20a202c76244485fbc8f1610473db8b9-7492a9464009416aa4c3a11e145dec61 on tablet 7492a9464009416aa4c3a11e145dec61 with peer 20a202c76244485fbc8f1610473db8b9
I20260709 12:21:19.137818 16122 ts_tablet_manager.cc:1434] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9: Time spent starting tablet: real 0.106s	user 0.009s	sys 0.000s
W20260709 12:21:19.141132  7238 consensus_peers.cc:597] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:19.179733  7269 raft_consensus.cc:1217] T e423f656ad70421f959a9a3b81dac708 P 20a202c76244485fbc8f1610473db8b9 [term 34 LEARNER]: Deduplicated request from leader. Original: 34.18->[34.19-34.19]   Dedup: 34.19->[]
W20260709 12:21:19.184653  7238 consensus_peers.cc:597] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:19.185215  7238 consensus_peers.cc:597] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:19.191529  7238 consensus_peers.cc:597] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:19.192531  6548 consensus_peers.cc:597] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:19.193998  6622 consensus_peers.cc:597] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:19.207960  6622 consensus_peers.cc:597] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:19.208776  6622 consensus_peers.cc:597] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:19.209437  6622 consensus_peers.cc:597] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:19.211755  6622 consensus_peers.cc:597] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:19.213418  6622 consensus_peers.cc:597] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:19.214088  6622 consensus_peers.cc:597] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:19.223064  6622 consensus_peers.cc:597] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:19.226531  6622 consensus_peers.cc:597] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:19.232093  6622 consensus_peers.cc:597] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:19.246387  6622 consensus_peers.cc:597] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:19.258313  7269 raft_consensus.cc:1217] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [term 28 LEARNER]: Deduplicated request from leader. Original: 28.17->[28.18-28.18]   Dedup: 28.18->[]
W20260709 12:21:19.306878  6622 consensus_peers.cc:597] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:19.308182  6622 consensus_peers.cc:597] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:19.298488  6548 consensus_peers.cc:597] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:19.315115  5352 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:19.317150  5352 tablet_replica.cc:333] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:19.318290  5352 raft_consensus.cc:2243] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.320745  5352 raft_consensus.cc:2272] T 48e50fd265d348779de32dff1ac6db39 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:19.323612  6548 consensus_peers.cc:597] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:19.331158  6622 consensus_peers.cc:597] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:19.325994  5352 tablet_replica.cc:333] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:19.337977  5352 raft_consensus.cc:2243] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.345273  5352 raft_consensus.cc:2272] T 3a47d71c34b541b9b2589b4e80413fa2 P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.348894  5352 tablet_replica.cc:333] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:19.369109  5352 raft_consensus.cc:2243] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.369804  5352 raft_consensus.cc:2272] T 99e5417fa1d54082a5690b657f407f70 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:19.371995  7238 consensus_peers.cc:597] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:19.372666  5352 tablet_replica.cc:333] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:19.395783  5352 raft_consensus.cc:2243] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.396651  5352 raft_consensus.cc:2272] T d0bd537a4ec844a69ca8227a47d1a8f3 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.399559  5352 tablet_replica.cc:333] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:19.400583  5352 raft_consensus.cc:2243] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.401615  5352 raft_consensus.cc:2272] T 7492a9464009416aa4c3a11e145dec61 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.407090  5352 tablet_replica.cc:333] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:19.413183  5352 raft_consensus.cc:2243] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 14 LEADER]: Raft consensus shutting down.
W20260709 12:21:19.411995  7238 consensus_peers.cc:597] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:19.417047  7238 consensus_peers.cc:597] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:19.418946  5352 raft_consensus.cc:2272] T e6167e4ddf454440bfc602ffbcd6da3e P 496c7c3792cd4587b4dbeba435f4be1e [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.424201 16125 raft_consensus.cc:1064] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
W20260709 12:21:19.424980  7238 consensus_peers.cc:597] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:19.428273 16125 consensus_queue.cc:237] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 13, Current term: 27, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:19.429543  5352 tablet_replica.cc:333] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:19.430838  5352 raft_consensus.cc:2243] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 LEADER]: Raft consensus shutting down.
W20260709 12:21:19.431631  7238 consensus_peers.cc:597] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:19.441833  5352 raft_consensus.cc:2272] T 555356e0b31d45c1bc3963dfe39b2df3 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.443720  7269 raft_consensus.cc:1217] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [term 24 LEARNER]: Deduplicated request from leader. Original: 24.12->[24.13-24.13]   Dedup: 24.13->[]
I20260709 12:21:19.445448  5352 tablet_replica.cc:333] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
W20260709 12:21:19.438691  6622 consensus_peers.cc:597] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:19.449326  5352 raft_consensus.cc:2243] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.453979  5352 raft_consensus.cc:2272] T 5b17fd01574f4601b33dd37e7c78bc8f P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.462141  5352 tablet_replica.cc:333] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:19.463369  5352 raft_consensus.cc:2243] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.464414  5352 raft_consensus.cc:2272] T 8ea6f08346334787a6b558ca10a70249 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.469656  5352 tablet_replica.cc:333] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:19.473444  5352 raft_consensus.cc:2243] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.472177  7269 raft_consensus.cc:1275] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9 [term 27 LEARNER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:21:19.476840  5352 raft_consensus.cc:2272] T 3e0ab356fc074f589a6ff4f9b4996b09 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.478405 16126 consensus_queue.cc:1048] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:21:19.484278  5352 tablet_replica.cc:333] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:19.499863  5352 raft_consensus.cc:2243] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.501119  5352 raft_consensus.cc:2272] T 5b0b7ace18d848afa42e884bcd2475a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.504307  5352 tablet_replica.cc:333] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:19.505337  5352 raft_consensus.cc:2243] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.506590  5352 raft_consensus.cc:2272] T 83301542a88f46cea64bc905b3199975 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.510269  5352 tablet_replica.cc:333] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:19.515424  5352 raft_consensus.cc:2243] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.516259  5352 raft_consensus.cc:2272] T d6029ffeecb94e4da196c61c032b862b P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:19.518519  6622 consensus_peers.cc:597] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:19.520220  6622 consensus_peers.cc:597] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:19.528077  5352 tablet_replica.cc:333] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
W20260709 12:21:19.531345  6548 consensus_peers.cc:597] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:19.532135  6548 consensus_peers.cc:597] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:19.532508  5352 raft_consensus.cc:2243] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.534582  5352 raft_consensus.cc:2272] T 27473d665a69492b9d08ab7409de0b5f P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.548559  5352 tablet_replica.cc:333] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:19.549889  5352 raft_consensus.cc:2243] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.551581  5352 raft_consensus.cc:2272] T cb7f6a796adb4b8c89579ba77274985f P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.556001  5352 tablet_replica.cc:333] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:19.563927  5352 raft_consensus.cc:2243] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.565320  5352 raft_consensus.cc:2272] T c50fba48b82e4ea29ce3f314b054d4ec P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:19.568953  6622 consensus_peers.cc:597] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:19.572530  5352 tablet_replica.cc:333] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
W20260709 12:21:19.577140  6548 consensus_peers.cc:597] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:19.581857  5352 raft_consensus.cc:2243] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.583551  5352 raft_consensus.cc:2272] T c7346d993d414b4197cec453cd7bf127 P 496c7c3792cd4587b4dbeba435f4be1e [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.588708  5352 tablet_replica.cc:333] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
W20260709 12:21:19.593415  6548 consensus_peers.cc:597] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:19.593436  5352 raft_consensus.cc:2243] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Raft consensus shutting down.
W20260709 12:21:19.597947  6392 catalog_manager.cc:4755] TS e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 3318181e3c0b433a9e1cee517969105c: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:19.590016  6622 consensus_peers.cc:597] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:19.599136  5352 raft_consensus.cc:2272] T 3e7223833e1f47b7b68c0080facbbe95 P 496c7c3792cd4587b4dbeba435f4be1e [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:19.599833  6622 consensus_peers.cc:597] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:19.602474  5352 tablet_replica.cc:333] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
W20260709 12:21:19.613096  6622 consensus_peers.cc:597] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:19.615587  5352 raft_consensus.cc:2243] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.628139  5352 raft_consensus.cc:2272] T dbf5342782d14193986ec66c46ce646e P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.641786  5352 tablet_replica.cc:333] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:19.642700  5352 raft_consensus.cc:2243] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 34 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.661813 16138 raft_consensus.cc:1064] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
W20260709 12:21:19.662256 16138 raft_consensus.cc:1068] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e: Unable to promote non-voter 20a202c76244485fbc8f1610473db8b9: Illegal state: RaftConsensus is not running: State = Stopping
W20260709 12:21:19.693398  6622 consensus_peers.cc:597] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:19.706218  5352 raft_consensus.cc:2272] T e423f656ad70421f959a9a3b81dac708 P 496c7c3792cd4587b4dbeba435f4be1e [term 34 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:19.712280  7238 consensus_peers.cc:597] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:19.727270 16112 raft_consensus.cc:1064] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:21:19.730064 16112 consensus_queue.cc:237] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 16, Current term: 28, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:19.745517  5352 tablet_replica.cc:333] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:19.746642  5352 raft_consensus.cc:2243] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.747522  5352 raft_consensus.cc:2272] T ef43f1290994490e8ef291fa12012731 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.751897  7269 raft_consensus.cc:1275] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [term 28 LEARNER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:21:19.759092 16136 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:21:19.766959  5352 tablet_replica.cc:333] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:19.774806  6655 raft_consensus.cc:1275] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Refusing update from remote peer 496c7c3792cd4587b4dbeba435f4be1e: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
W20260709 12:21:19.776424  6622 consensus_peers.cc:597] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:19.776773 16137 consensus_queue.cc:1048] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 12:21:19.790316  7238 consensus_peers.cc:597] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:19.796098 16112 raft_consensus.cc:2955] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 28 LEADER]: Committing config change with OpId 28.19: config changed from index 18 to 19, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:19.797340  7269 raft_consensus.cc:2955] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Committing config change with OpId 28.19: config changed from index 18 to 19, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:19.768076  5352 raft_consensus.cc:2243] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.801468  5352 raft_consensus.cc:2272] T f9f8f5d393744ebabbed2f3e0131cf40 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:19.810374  6548 consensus_peers.cc:597] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:19.825349 16137 raft_consensus.cc:493] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:19.825946 16137 raft_consensus.cc:515] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:19.834736  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa09afefaa884dc5943b210cb23a07b3" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:19.828080 16137 leader_election.cc:290] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:21:19.839586  6622 consensus_peers.cc:597] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:19.842239  6655 raft_consensus.cc:2955] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Committing config change with OpId 28.19: config changed from index 18 to 19, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:19.819662  5352 tablet_replica.cc:333] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:19.859017  5352 raft_consensus.cc:2243] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.859323 16125 raft_consensus.cc:1064] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4: attempting to promote NON_VOTER 20a202c76244485fbc8f1610473db8b9 to VOTER
I20260709 12:21:19.859511  6406 catalog_manager.cc:5697] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 reported cstate change: config changed from index 18 to 19, 20a202c76244485fbc8f1610473db8b9 (127.5.58.4) changed from NON_VOTER to VOTER. New cstate: current_term: 28 leader_uuid: "496c7c3792cd4587b4dbeba435f4be1e" committed_config { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } }
I20260709 12:21:19.861972 16125 consensus_queue.cc:237] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 11, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:21:19.881393  6622 consensus_peers.cc:597] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:19.884598  7269 raft_consensus.cc:1275] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [term 24 LEARNER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
W20260709 12:21:19.884301  6548 leader_election.cc:336] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:19.887048  6548 leader_election.cc:304] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:19.889330 16077 consensus_queue.cc:1048] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:21:19.891444 16112 raft_consensus.cc:493] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:21:19.892522  5352 raft_consensus.cc:2272] T 14cf1c3a63624d94aa6c9c356e6108e2 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:19.897818  6622 consensus_peers.cc:597] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:19.896997 16136 raft_consensus.cc:2749] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:21:19.892081 16112 raft_consensus.cc:515] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:19.906860 16112 leader_election.cc:290] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:19.907691  5352 tablet_replica.cc:333] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:19.913779  5352 raft_consensus.cc:2243] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.922050  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:19.925593 16112 raft_consensus.cc:493] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:19.937131 16112 raft_consensus.cc:515] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:19.939069  5352 raft_consensus.cc:2272] T 591037e9504e4348a9cdf8842f34be22 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.942847 16112 leader_election.cc:290] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:19.944408  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb34f14626f4aa895bae63ea2466e00" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:19.950636  5352 tablet_replica.cc:333] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:19.951689  5352 raft_consensus.cc:2243] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.952894  5352 raft_consensus.cc:2272] T 72269ad6925341548796bb34823b3e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:19.954748  6622 consensus_peers.cc:597] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:19.960795  6548 leader_election.cc:336] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:19.961400  6548 leader_election.cc:304] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:19.964145  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8066a9d7ccf34c89ad8d3df18c4e86c2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:19.964251 16136 raft_consensus.cc:493] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:21:19.968902 16136 raft_consensus.cc:515] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:19.969998  5352 tablet_replica.cc:333] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:19.970882  5352 raft_consensus.cc:2243] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:21:19.971484 16136 leader_election.cc:290] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:19.973459  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:19.985258 16136 raft_consensus.cc:493] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:21:19.986039 16136 raft_consensus.cc:515] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:19.988229 16136 leader_election.cc:290] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:19.989040  5352 raft_consensus.cc:2272] T e91732822ea14700aa3230fbb6b1cfac P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.989835  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebce9eca2a5496bb4a3ab473a243166" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:19.991618 16154 raft_consensus.cc:493] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:19.991777 16112 raft_consensus.cc:2749] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:21:19.992753 16154 raft_consensus.cc:515] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:20.001358  5352 tablet_replica.cc:333] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:20.002789  5352 raft_consensus.cc:2243] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:20.003720  5352 raft_consensus.cc:2272] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:20.005442 16154 leader_election.cc:290] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:20.006172  6549 leader_election.cc:304] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:20.006916  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d80f18ce04d4d97b0b3c5dc2ed0e4a4" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:20.007079 16136 raft_consensus.cc:493] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:20.007748  5352 tablet_replica.cc:333] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:20.007684 16136 raft_consensus.cc:515] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:20.008793  5352 raft_consensus.cc:2243] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:20.014608  5352 raft_consensus.cc:2272] T fa09afefaa884dc5943b210cb23a07b3 P 496c7c3792cd4587b4dbeba435f4be1e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:20.012012 16152 raft_consensus.cc:2749] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:21:20.015488  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c27974d342ea4901aaa3946871539818" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:20.017575  5352 tablet_replica.cc:333] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:20.013377  6549 leader_election.cc:304] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:20.010746 16136 leader_election.cc:290] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:21:20.022009  6548 leader_election.cc:336] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:20.026470 16136 raft_consensus.cc:2749] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:21:20.028815  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e416f9423f964ba5a29e536e8160c627" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:21:20.029862  7238 consensus_peers.cc:597] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:20.025794  5352 raft_consensus.cc:2243] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:21:20.036970  5352 raft_consensus.cc:2272] T 23f03345855041f8aabfa3e94a1867d9 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:20.040706  6549 leader_election.cc:304] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:21:20.041599  6549 raft_consensus.cc:2686] T e416f9423f964ba5a29e536e8160c627 P 496c7c3792cd4587b4dbeba435f4be1e: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 12:21:20.031490  6548 consensus_peers.cc:597] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:20.050380 16154 raft_consensus.cc:493] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:20.050979 16154 raft_consensus.cc:515] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:20.056886 16154 leader_election.cc:290] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:20.057534  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6462df23af154f06b34f32a1ef4a45b2" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:20.062896  5352 tablet_replica.cc:333] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:20.064033  5352 raft_consensus.cc:2243] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:21:20.066213  5352 raft_consensus.cc:2272] T f5b42468eaf14fd49be100af079d3921 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:20.069670  5352 tablet_replica.cc:333] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
W20260709 12:21:20.071250  6548 leader_election.cc:336] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:20.073433  5352 raft_consensus.cc:2243] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:20.073730  6548 leader_election.cc:304] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:20.074513 16154 raft_consensus.cc:2749] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:21:20.076082  6548 leader_election.cc:336] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:20.076556  6548 leader_election.cc:304] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:20.077684 16154 raft_consensus.cc:2749] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:21:20.080557  5352 raft_consensus.cc:2272] T f2592253d3214104865730480e057d09 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:20.081485 16154 raft_consensus.cc:493] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:20.082098 16154 raft_consensus.cc:515] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:20.084033  5352 tablet_replica.cc:333] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:20.085119  5352 raft_consensus.cc:2243] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:21:20.085878 16136 raft_consensus.cc:493] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 20a202c76244485fbc8f1610473db8b9)
I20260709 12:21:20.086238  5352 raft_consensus.cc:2272] T bd6fb503c26a49bab14a18c9b5e2d792 P 496c7c3792cd4587b4dbeba435f4be1e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:20.086468 16136 raft_consensus.cc:515] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:20.088714  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7d79febb5824e78a295b2c993c14108" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:20.089534  5352 tablet_replica.cc:333] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:20.091127  5352 raft_consensus.cc:2243] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:20.091300 16136 leader_election.cc:290] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:20.092990  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:20.094842 16154 leader_election.cc:290] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:20.095417  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1778f67a2ba4fc2825ec2be5097d77a" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:20.098129  6549 leader_election.cc:304] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:20.091828  5352 raft_consensus.cc:2272] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P 496c7c3792cd4587b4dbeba435f4be1e [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:20.107146  5352 tablet_replica.cc:333] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:20.102947 16154 raft_consensus.cc:2749] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:21:20.112298  5352 raft_consensus.cc:2243] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:20.113615  5352 raft_consensus.cc:2272] T eebce9eca2a5496bb4a3ab473a243166 P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:20.117730 16154 raft_consensus.cc:493] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:20.118414 16154 raft_consensus.cc:515] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:20.122185  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53dc690d5068460a820e0ba9e24494ec" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:20.125015 16154 leader_election.cc:290] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:21:20.131990  7238 consensus_peers.cc:597] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:20.132440  6548 leader_election.cc:336] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:20.133142  6548 leader_election.cc:304] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:20.133991 16154 raft_consensus.cc:2749] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:21:20.137799  6622 consensus_peers.cc:597] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:20.137799  6548 leader_election.cc:336] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:20.146067  6548 leader_election.cc:304] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:20.149643 16154 raft_consensus.cc:2749] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:21:20.150444 16154 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:20.151348 16154 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:21:20.154431  6548 leader_election.cc:336] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:20.154922  6548 leader_election.cc:304] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:20.155900 16136 raft_consensus.cc:2749] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:21:20.159176 16154 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:20.160097  5352 tablet_replica.cc:333] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:20.161108  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:20.161120  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3318181e3c0b433a9e1cee517969105c" candidate_uuid: "496c7c3792cd4587b4dbeba435f4be1e" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:20.164073  6550 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 496c7c3792cd4587b4dbeba435f4be1e; no voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:20.165181 16154 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:21:20.170540  5352 raft_consensus.cc:2243] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:20.174460  5352 raft_consensus.cc:2272] T 6462df23af154f06b34f32a1ef4a45b2 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:20.177893  5352 tablet_replica.cc:333] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:20.178818  5352 raft_consensus.cc:2243] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:21:20.184821  5352 raft_consensus.cc:2272] T c9d981014b2c4cda80bca766fe58fe42 P 496c7c3792cd4587b4dbeba435f4be1e [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:20.187978  5352 tablet_replica.cc:333] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:20.188773  5352 raft_consensus.cc:2243] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:20.189358  5352 raft_consensus.cc:2272] T cbb34f14626f4aa895bae63ea2466e00 P 496c7c3792cd4587b4dbeba435f4be1e [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:20.192003  5352 tablet_replica.cc:333] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:20.192775  5352 raft_consensus.cc:2243] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:20.193635  5352 raft_consensus.cc:2272] T 53dc690d5068460a820e0ba9e24494ec P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:20.196858  5352 tablet_replica.cc:333] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:20.197583  5352 raft_consensus.cc:2243] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:21:20.198659  5352 raft_consensus.cc:2272] T 1c226c3677f8446f837400fd0df3062e P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:20.201524  5352 tablet_replica.cc:333] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:20.202247  5352 raft_consensus.cc:2243] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:20.203114  5352 raft_consensus.cc:2272] T 3318181e3c0b433a9e1cee517969105c P 496c7c3792cd4587b4dbeba435f4be1e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:20.205600  5352 tablet_replica.cc:333] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:20.206281  5352 raft_consensus.cc:2243] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:20.206950  5352 raft_consensus.cc:2272] T b7d79febb5824e78a295b2c993c14108 P 496c7c3792cd4587b4dbeba435f4be1e [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:20.209894  5352 tablet_replica.cc:333] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:20.210633  5352 raft_consensus.cc:2243] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:20.211208  5352 raft_consensus.cc:2272] T d1778f67a2ba4fc2825ec2be5097d77a P 496c7c3792cd4587b4dbeba435f4be1e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:20.213955  5352 tablet_replica.cc:333] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:20.214730  5352 raft_consensus.cc:2243] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:21:20.216003  5352 raft_consensus.cc:2272] T 3af4a543c38a49bfa8ece9e702577ea1 P 496c7c3792cd4587b4dbeba435f4be1e [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:20.220165  5352 tablet_replica.cc:333] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:20.220872  5352 raft_consensus.cc:2243] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:20.221670  5352 raft_consensus.cc:2272] T c27974d342ea4901aaa3946871539818 P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:20.224277  5352 tablet_replica.cc:333] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:20.224995  5352 raft_consensus.cc:2243] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:20.225548  5352 raft_consensus.cc:2272] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 496c7c3792cd4587b4dbeba435f4be1e [term 11 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:20.236788  7238 consensus_peers.cc:597] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:20.238893  5352 tablet_replica.cc:333] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:20.239761  5352 raft_consensus.cc:2243] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:21:20.241215  5352 raft_consensus.cc:2272] T 7ba56fbb3caa417db1c8d78a9caa28da P 496c7c3792cd4587b4dbeba435f4be1e [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:20.248620  5352 tablet_replica.cc:333] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:20.249374  5352 raft_consensus.cc:2243] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:21:20.250964  5352 raft_consensus.cc:2272] T ab430742d4f44bf79c4591f613055cd6 P 496c7c3792cd4587b4dbeba435f4be1e [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:20.253989  5352 tablet_replica.cc:333] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:20.254691  5352 raft_consensus.cc:2243] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:21:20.255928  5352 raft_consensus.cc:2272] T 72cc6298c451423d8fe8c456777eb890 P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:20.258821  5352 tablet_replica.cc:333] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e: stopping tablet replica
I20260709 12:21:20.259495  5352 raft_consensus.cc:2243] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:21:20.260998  5352 raft_consensus.cc:2272] T de5f3f59d0794f4c898c5e7f2b721abe P 496c7c3792cd4587b4dbeba435f4be1e [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:20.456815  6622 consensus_peers.cc:597] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:20.482573 16077 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:20.483156 16077 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:20.485720 16077 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:20.535797  6622 leader_election.cc:336] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:20.536600 16122 tablet_copy_client.cc:1131] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9: tablet copy: Unable to close tablet copy session: Network error: failure ending tablet copy session: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:20.536860 16139 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:21:20.537667  6622 leader_election.cc:336] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:20.538328  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:20.539875 16077 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:21:20.538048 16139 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:20.543023 16077 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:20.543738 16139 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:20.543538 16077 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:20.545707 16077 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:20.554443  7238 leader_election.cc:336] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:20.555250  7238 leader_election.cc:336] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:20.555675  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:20.557842 16145 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:21:20.565347  7238 consensus_peers.cc:597] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:20.576997  6622 leader_election.cc:336] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:20.582547  6622 leader_election.cc:336] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:20.586663  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:20.589720 16099 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:21:20.674261 16127 tablet_copy_client.cc:1131] T e423f656ad70421f959a9a3b81dac708 P 20a202c76244485fbc8f1610473db8b9: tablet copy: Unable to close tablet copy session: Network error: failure ending tablet copy session: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:20.735826 16145 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:20.737455 16145 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:20.743636 16145 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:20.747195  6655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
W20260709 12:21:20.759409  7238 leader_election.cc:336] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:20.760300  7238 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:20.762324 16145 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:21:20.836910 16100 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:20.837641 16100 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:20.840009 16100 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:20.859104 16145 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:20.859730 16145 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:21:20.860945  6622 leader_election.cc:336] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:20.861707  6622 leader_election.cc:336] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:20.862037 16145 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:20.862128  6622 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:20.864575 16077 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:21:20.867832  7238 leader_election.cc:336] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:20.868618  7238 leader_election.cc:336] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:20.869119  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:20.869905 16145 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:21:20.886545 16100 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:20.887347 16100 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:20.888968 16077 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:20.889537 16077 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:20.891641 16077 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:20.891496  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:20.892079 16100 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:20.892570  7269 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 14.
I20260709 12:21:20.893635 16099 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:20.894210  6623 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:21:20.894105 16099 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:20.896613 16099 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:20.896781 16077 raft_consensus.cc:2804] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:21:20.897185 16077 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:20.897521 16077 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:21:20.905812 16077 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:20.908542  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6167e4ddf454440bfc602ffbcd6da3e" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9"
I20260709 12:21:20.909125  7269 raft_consensus.cc:3060] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:21:20.923754  7269 raft_consensus.cc:2468] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 15.
W20260709 12:21:20.931427  6622 leader_election.cc:336] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
W20260709 12:21:20.932407  6622 leader_election.cc:336] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:20.932745  6623 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 
I20260709 12:21:20.933593 16100 raft_consensus.cc:2804] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:21:20.934229 16100 raft_consensus.cc:697] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Becoming Leader. State: Replica: f2b06aebe97249a4877d798b8118ece4, State: Running, Role: LEADER
I20260709 12:21:20.935149 16100 consensus_queue.cc:237] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:21:20.933600  6622 leader_election.cc:336] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:20.936817  6622 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:20.937661 16099 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:21:20.937810  6622 leader_election.cc:336] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:20.941745 16077 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: Requested vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
W20260709 12:21:20.946702  6622 leader_election.cc:336] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:20.947203  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:21:20.948570  6622 leader_election.cc:336] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:20.949985 16077 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:21:20.992719  6406 catalog_manager.cc:5697] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 reported cstate change: term changed from 14 to 15, leader changed from 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2) to f2b06aebe97249a4877d798b8118ece4 (127.5.58.3). New cstate: current_term: 15 leader_uuid: "f2b06aebe97249a4877d798b8118ece4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:21.020421 16145 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:21.021019 16145 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:21.023058 16145 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:21.030267  7238 leader_election.cc:336] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
W20260709 12:21:21.031122  7238 leader_election.cc:336] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:21.031704  7238 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:21.032591 16145 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:21:21.051462 16077 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:21.052039 16077 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:21:21.052505 16099 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:21.053121 16099 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:21.053966 16077 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:21.055447 16099 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:21.071996 16099 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:21.072606 16099 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:21.074837 16099 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:21.096673  6622 leader_election.cc:336] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:21.097426  6622 leader_election.cc:336] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:21.098023  6622 leader_election.cc:336] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:21.098940  6622 leader_election.cc:336] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:21.099339  6622 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:21.100077 16099 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:21:21.100217  6622 leader_election.cc:336] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:21.100760  6622 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:21.101528 16099 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:21:21.102131  6622 leader_election.cc:336] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:21.102574  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:21.103266 16099 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:21:21.129688  6622 consensus_peers.cc:597] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:21.135097  7238 consensus_peers.cc:597] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:21.162391 16099 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:21.163033 16099 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:21:21.165586 16099 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:21.173717  6622 leader_election.cc:336] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:21.174562  6622 leader_election.cc:336] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:21.174970  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:21.175684 16099 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
W20260709 12:21:21.185978  6622 consensus_peers.cc:597] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:21.192008  7238 consensus_peers.cc:597] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:21.202343  6622 consensus_peers.cc:597] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:21.214726  7238 consensus_peers.cc:597] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:21.227861  6622 consensus_peers.cc:597] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:21.244256  6622 consensus_peers.cc:597] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:21.284641 16099 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:21.285320 16099 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:21.290448 16099 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:21.293761  6622 consensus_peers.cc:597] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:21.301846  6622 leader_election.cc:336] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:21.302805  6622 leader_election.cc:336] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:21.303282  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:21.308559 16099 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:21:21.327013 16160 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:21.327687 16160 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:21.330340 16160 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:21.344192 16100 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:21.344969 16100 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:21.345916  6649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "20a202c76244485fbc8f1610473db8b9" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "f2b06aebe97249a4877d798b8118ece4" is_pre_election: true
I20260709 12:21:21.346513  6649 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a202c76244485fbc8f1610473db8b9 in term 28.
I20260709 12:21:21.350405 16100 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:21.353323  7269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
W20260709 12:21:21.354051  6622 leader_election.cc:336] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:21.354182  7269 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 28.
W20260709 12:21:21.354939  6622 leader_election.cc:336] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:21.355346  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:21.356717 16100 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:21:21.361255  7238 leader_election.cc:336] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:21.363405  7238 leader_election.cc:336] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:21.363811  7238 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:21.364604 16166 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:21:21.425637 16100 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:21.426224 16100 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:21.429953 16099 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:21.430729 16099 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:21.428283 16100 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:21.434289 16099 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:21.436944 16099 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:21:21.437445  6622 leader_election.cc:336] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:21.437463 16099 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:21:21.439795  6622 leader_election.cc:336] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:21.440205  6622 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:21.440965 16100 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:21:21.443521 16099 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:21.444572  6622 leader_election.cc:336] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:21.460688 16166 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:21.461309 16166 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:21.463716 16166 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:21.464308  6622 consensus_peers.cc:597] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:21.464921  6622 leader_election.cc:336] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:21.465410  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:21.466270 16099 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:21:21.470515  6622 leader_election.cc:336] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
W20260709 12:21:21.471144  7238 leader_election.cc:336] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:21.472496  7238 leader_election.cc:336] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:21.472981  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:21.473789 16166 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:21:21.482671  6622 leader_election.cc:336] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:21.483235  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:21.484058 16166 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:21.484675 16166 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:21.487212 16166 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:21.488128  6622 consensus_peers.cc:597] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:21:21.489043 16099 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:21:21.488915  6622 consensus_peers.cc:597] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:21.494794  7238 leader_election.cc:336] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:21.495535  7238 leader_election.cc:336] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:21.495981  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:21.496738 16166 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:21:21.502463  7269 raft_consensus.cc:1275] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Refusing update from remote peer f2b06aebe97249a4877d798b8118ece4: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:21:21.509524 16077 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:21.510118 16077 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:21.512156 16077 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:21.519167 16077 consensus_queue.cc:1048] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:21:21.521154  6622 leader_election.cc:336] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:21.523684  6622 leader_election.cc:336] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
W20260709 12:21:21.527915  7238 consensus_peers.cc:597] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:21.529294  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:21.531738 16077 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:21:21.549538  6622 consensus_peers.cc:597] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:21.578645  7238 consensus_peers.cc:597] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:21.582410  6622 consensus_peers.cc:597] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:21.604207  7238 consensus_peers.cc:597] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:21.606578  6622 consensus_peers.cc:597] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:21.611268  6622 consensus_peers.cc:597] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:21.616228  7238 consensus_peers.cc:597] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:21.616925  7238 consensus_peers.cc:597] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:21.626188  6622 consensus_peers.cc:597] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:21.629944 16099 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:21.630654 16099 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:21.633121 16099 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:21.640364  7238 consensus_peers.cc:597] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:21.642993  7238 consensus_peers.cc:597] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:21.648525  6622 consensus_peers.cc:597] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:21.649091  6622 leader_election.cc:336] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
W20260709 12:21:21.649832  6622 leader_election.cc:336] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:21.650271  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:21.651695 16077 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:21:21.681443  6622 consensus_peers.cc:597] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:21:21.682830 16172 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:21.683490 16172 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:21.685940 16172 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:21.703241  6622 leader_election.cc:336] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:21.705169  6622 leader_election.cc:336] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:21.705765  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:21.708420 16171 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:21:21.727078  6622 consensus_peers.cc:597] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:21.800722  7238 consensus_peers.cc:597] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:21.854368  7238 consensus_peers.cc:597] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:21.861912  6622 consensus_peers.cc:597] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:21.868114  6622 consensus_peers.cc:597] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:21.869009  6622 consensus_peers.cc:597] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:21.914471  7238 consensus_peers.cc:597] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:21.972092  7238 consensus_peers.cc:597] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:22.017382  7238 consensus_peers.cc:597] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:22.073553  6622 consensus_peers.cc:597] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:22.110219  6622 consensus_peers.cc:597] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:22.128233 16100 raft_consensus.cc:493] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:22.128819 16100 raft_consensus.cc:515] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:22.130916 16100 leader_election.cc:290] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:22.149269  6622 leader_election.cc:336] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:22.150763  6622 leader_election.cc:336] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:22.151193  6622 leader_election.cc:304] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:22.153098 16100 raft_consensus.cc:2749] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:21:22.161654  6622 consensus_peers.cc:597] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:22.167403  6622 consensus_peers.cc:597] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:22.170272  6622 consensus_peers.cc:597] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:22.173239  7238 consensus_peers.cc:597] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:22.256072  7238 consensus_peers.cc:597] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:21:22.273845 16100 raft_consensus.cc:493] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:22.274514 16100 raft_consensus.cc:515] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:22.276782 16100 leader_election.cc:290] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:22.287669  6622 leader_election.cc:336] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:22.291366  6622 leader_election.cc:336] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:22.292459  6622 leader_election.cc:304] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:22.293673 16100 raft_consensus.cc:2749] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:21:22.323629  6622 consensus_peers.cc:597] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:22.326684  6622 consensus_peers.cc:597] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:22.347932  6622 consensus_peers.cc:597] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:22.348500  6622 consensus_peers.cc:597] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:22.412381  6622 consensus_peers.cc:597] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:22.451220  6622 consensus_peers.cc:597] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:22.471966  6622 consensus_peers.cc:597] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:22.562695  7238 consensus_peers.cc:597] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:22.597098  6622 consensus_peers.cc:597] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:22.628844  7238 consensus_peers.cc:597] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:21:22.689960 16100 raft_consensus.cc:493] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:22.690555 16100 raft_consensus.cc:515] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:22.701404 16100 leader_election.cc:290] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:22.727721  6622 leader_election.cc:336] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:22.731123 16166 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:22.731901 16166 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:22.734829 16166 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:22.745431  7238 leader_election.cc:336] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
W20260709 12:21:22.746275  7238 leader_election.cc:336] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:22.746747  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:22.747503 16166 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:21:22.791982  6622 leader_election.cc:336] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:22.792637  6622 leader_election.cc:304] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:22.793551 16176 raft_consensus.cc:2749] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:21:22.807243  6622 consensus_peers.cc:597] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:22.842244  7238 consensus_peers.cc:597] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:21:22.880560  5352 tablet_server.cc:196] TabletServer@127.5.58.2:0 shutdown complete.
I20260709 12:21:22.938015 16176 raft_consensus.cc:493] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:22.938728 16176 raft_consensus.cc:515] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:22.941330 16176 leader_election.cc:290] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:22.949662 16175 raft_consensus.cc:493] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:22.950337 16175 raft_consensus.cc:515] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } }
I20260709 12:21:22.952947 16175 leader_election.cc:290] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:22.958045  6622 leader_election.cc:336] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:22.958523 16167 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:21:22.959295  6622 leader_election.cc:336] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:22.959085 16167 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:22.959707  6622 leader_election.cc:304] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:22.961351 16167 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:22.967635  7238 leader_election.cc:336] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:22.961424 16176 raft_consensus.cc:2749] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:21:22.968813  7238 leader_election.cc:336] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:22.969316  7238 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:22.970214 16167 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:21:22.976519 16176 raft_consensus.cc:493] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:22.977278 16176 raft_consensus.cc:515] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:22.979063 16167 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:22.979759 16167 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:22.982259 16167 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:22.980249 16176 leader_election.cc:290] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:22.983556  6622 leader_election.cc:336] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
W20260709 12:21:22.993861  6622 leader_election.cc:336] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:22.994621  7238 leader_election.cc:336] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:23.008574  6622 leader_election.cc:304] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:21:23.009362  7238 leader_election.cc:336] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:23.009927  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:23.010926 16166 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:21:23.012948 16176 raft_consensus.cc:2749] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:21:23.018287 16176 raft_consensus.cc:493] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:23.018957 16176 raft_consensus.cc:515] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:23.021509 16176 leader_election.cc:290] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:23.022727  6622 leader_election.cc:336] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:23.034644  6622 leader_election.cc:336] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:23.035175  6622 leader_election.cc:304] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:23.035902 16176 raft_consensus.cc:2749] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:21:23.047091  6622 leader_election.cc:336] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:23.048281  6622 leader_election.cc:336] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:23.057845  6622 leader_election.cc:304] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:23.059072 16176 raft_consensus.cc:2749] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:21:23.112258  5352 tablet_server.cc:179] TabletServer@127.5.58.3:0 shutting down...
I20260709 12:21:23.132699 16176 raft_consensus.cc:493] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:23.133283 16176 raft_consensus.cc:515] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:23.135793 16176 leader_election.cc:290] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:23.138827  7240 consensus_peers.cc:597] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:23.150223  6622 leader_election.cc:336] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:23.151329  6622 leader_election.cc:336] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:23.160549 16177 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:23.162384  6622 leader_election.cc:304] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:23.163245 16100 raft_consensus.cc:2749] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:21:23.163408 16177 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:23.165604 16177 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:23.172685  7238 leader_election.cc:336] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:23.176432 16100 raft_consensus.cc:493] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:23.177053 16100 raft_consensus.cc:515] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:23.179488 16100 leader_election.cc:290] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:23.180945  7240 consensus_peers.cc:597] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:23.181913  7238 leader_election.cc:336] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:23.182350  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:23.184295 16177 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:21:23.188514 16175 raft_consensus.cc:493] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:21:23.188840  6622 leader_election.cc:336] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:23.200487  6622 leader_election.cc:336] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:23.201098  6622 leader_election.cc:304] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:23.190997 16175 raft_consensus.cc:515] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:23.202013 16100 raft_consensus.cc:2749] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:21:23.207072 16175 leader_election.cc:290] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:23.210299 16175 raft_consensus.cc:493] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:23.211037 16175 raft_consensus.cc:515] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:23.217104 16175 leader_election.cc:290] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:23.226536  6622 leader_election.cc:336] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:23.234393  6622 leader_election.cc:336] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:23.242204  6622 leader_election.cc:304] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:21:23.246506  7240 consensus_peers.cc:597] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:23.249166  6622 leader_election.cc:336] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:23.245034 16176 raft_consensus.cc:2749] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:21:23.257019  7240 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 687 similar messages]
W20260709 12:21:23.262598  7240 consensus_peers.cc:597] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:23.264782  6622 leader_election.cc:336] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:23.265246  6622 leader_election.cc:304] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:23.266204 16176 raft_consensus.cc:2749] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:21:23.269212  7240 consensus_peers.cc:597] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:23.276800 16175 raft_consensus.cc:493] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:23.277482 16175 raft_consensus.cc:515] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:23.277832 16176 raft_consensus.cc:493] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:21:23.284036  7240 consensus_peers.cc:597] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:23.284199 16176 raft_consensus.cc:515] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } }
I20260709 12:21:23.288445 16176 leader_election.cc:290] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:23.286298 16175 leader_election.cc:290] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:23.293510 16166 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:23.294432 16166 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:23.298889 16166 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:23.300707 16176 raft_consensus.cc:493] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:23.301276 16176 raft_consensus.cc:515] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:23.303097 16176 leader_election.cc:290] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:23.303515  7238 leader_election.cc:336] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:23.305568  7238 leader_election.cc:336] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:23.305969  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:23.306697 16166 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:21:23.314415  7240 consensus_peers.cc:597] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:23.331458 16166 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
W20260709 12:21:23.332660  6622 leader_election.cc:336] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:23.332439 16166 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:21:23.333334  6622 leader_election.cc:336] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:23.341629 16166 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:21:23.344585  6622 leader_election.cc:336] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:23.345371  6622 leader_election.cc:304] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:21:23.345746  7240 consensus_peers.cc:597] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:23.347851  7240 leader_election.cc:336] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:21:23.351912  7238 consensus_peers.cc:597] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:23.346139 16176 raft_consensus.cc:2749] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:21:23.346295  6622 leader_election.cc:336] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:23.353170  6622 leader_election.cc:304] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:23.354066 16176 raft_consensus.cc:2749] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
W20260709 12:21:23.355113  6622 leader_election.cc:336] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
W20260709 12:21:23.373771  6622 leader_election.cc:336] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:23.379019  6622 leader_election.cc:304] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:21:23.376053  7238 leader_election.cc:336] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:23.379977  7238 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:23.380908 16167 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:21:23.384735 16176 raft_consensus.cc:2749] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:21:23.386775  7240 consensus_peers.cc:597] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:23.393311  7238 leader_election.cc:336] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:23.402235  5352 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:23.405138  5352 tablet_replica.cc:333] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.406296  5352 raft_consensus.cc:2243] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:21:23.407461  5352 raft_consensus.cc:2272] T 48e50fd265d348779de32dff1ac6db39 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.410643  5352 tablet_replica.cc:333] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.411820  5352 raft_consensus.cc:2243] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:23.417974 16176 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:23.418731 16176 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:23.422266 16176 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), 20a202c76244485fbc8f1610473db8b9 (127.5.58.4:44857)
I20260709 12:21:23.418720  5352 raft_consensus.cc:2272] T 555356e0b31d45c1bc3963dfe39b2df3 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.423465  7265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de5f3f59d0794f4c898c5e7f2b721abe" candidate_uuid: "f2b06aebe97249a4877d798b8118ece4" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "20a202c76244485fbc8f1610473db8b9" is_pre_election: true
I20260709 12:21:23.424201  7265 raft_consensus.cc:2468] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b06aebe97249a4877d798b8118ece4 in term 28.
I20260709 12:21:23.425685  5352 tablet_replica.cc:333] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.426754  5352 raft_consensus.cc:2243] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:21:23.427552  5352 raft_consensus.cc:2272] T 99e5417fa1d54082a5690b657f407f70 P f2b06aebe97249a4877d798b8118ece4 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.431022  5352 tablet_replica.cc:333] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
W20260709 12:21:23.439927  7238 consensus_peers.cc:597] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:23.451653  7238 consensus_peers.cc:597] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:23.459014  6622 leader_election.cc:336] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:23.459964  6622 leader_election.cc:336] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:23.460393  6622 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9, f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:23.461349 16176 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:21:23.463160  5352 raft_consensus.cc:2243] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:21:23.464603  5352 pending_rounds.cc:70] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4: Trying to abort 1 pending ops.
I20260709 12:21:23.464870  5352 pending_rounds.cc:77] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4: Aborting op as it isn't in flight: id { term: 27 index: 16 } timestamp: 7305624286930915328 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" old_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } } new_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } } }
I20260709 12:21:23.466054  5352 raft_consensus.cc:2917] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 27 LEADER]: Aborting config change with OpId 27.16: Aborted: Op aborted
I20260709 12:21:23.466459  5352 raft_consensus.cc:2889] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 27 LEADER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 12:21:23.466816  5352 raft_consensus.cc:2272] T d0bd537a4ec844a69ca8227a47d1a8f3 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.476938  5352 tablet_replica.cc:333] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.477923  5352 raft_consensus.cc:2243] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:21:23.479251  5352 raft_consensus.cc:2272] T e6167e4ddf454440bfc602ffbcd6da3e P f2b06aebe97249a4877d798b8118ece4 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.484696  5352 tablet_replica.cc:333] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.485551  5352 raft_consensus.cc:2243] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:23.486451  5352 raft_consensus.cc:2272] T 55df50d51529421789bb752f47dff41a P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.489118  5352 tablet_replica.cc:333] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.490223  5352 raft_consensus.cc:2243] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:21:23.491525  5352 raft_consensus.cc:2272] T d6029ffeecb94e4da196c61c032b862b P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.494225  5352 tablet_replica.cc:333] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.495129  5352 raft_consensus.cc:2243] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:23.496006  5352 raft_consensus.cc:2272] T 83301542a88f46cea64bc905b3199975 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.499008  5352 tablet_replica.cc:333] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.499825  5352 raft_consensus.cc:2243] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:23.500594  5352 raft_consensus.cc:2272] T c50fba48b82e4ea29ce3f314b054d4ec P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.503398  5352 tablet_replica.cc:333] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.504176  5352 raft_consensus.cc:2243] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:21:23.505273  5352 raft_consensus.cc:2272] T c7346d993d414b4197cec453cd7bf127 P f2b06aebe97249a4877d798b8118ece4 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.507901  5352 tablet_replica.cc:333] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.508773  5352 raft_consensus.cc:2243] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:23.509446  5352 raft_consensus.cc:2272] T 3e7223833e1f47b7b68c0080facbbe95 P f2b06aebe97249a4877d798b8118ece4 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.513684  5352 tablet_replica.cc:333] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.521459 16176 raft_consensus.cc:493] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:23.522084 16176 raft_consensus.cc:515] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
W20260709 12:21:23.524801  7240 consensus_peers.cc:597] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:23.534790 16175 raft_consensus.cc:493] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:23.535575 16175 raft_consensus.cc:515] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } }
I20260709 12:21:23.538735 16175 leader_election.cc:290] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:23.524214 16176 leader_election.cc:290] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:23.542202  7240 consensus_peers.cc:597] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:23.548825  6622 leader_election.cc:336] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
W20260709 12:21:23.558284  6622 leader_election.cc:336] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:23.559190  6622 leader_election.cc:304] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:23.558753  5352 raft_consensus.cc:2243] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:23.560212 16176 raft_consensus.cc:2749] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:21:23.561292  5352 raft_consensus.cc:2272] T 8e01550f5e994006a4031ca81edcc9aa P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:23.567325  7240 consensus_peers.cc:597] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:23.572036  5352 tablet_replica.cc:333] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.578346  5352 raft_consensus.cc:2243] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:21:23.579813  5352 raft_consensus.cc:2272] T ef43f1290994490e8ef291fa12012731 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.582983  5352 tablet_replica.cc:333] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
W20260709 12:21:23.575533  6622 leader_election.cc:336] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
W20260709 12:21:23.583976  6622 leader_election.cc:336] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:23.584539  6622 leader_election.cc:304] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2b06aebe97249a4877d798b8118ece4; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:23.585397 16175 raft_consensus.cc:2749] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:21:23.589468  5352 raft_consensus.cc:2243] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:21:23.593154  5352 raft_consensus.cc:2272] T dbf5342782d14193986ec66c46ce646e P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:23.591723  6622 consensus_peers.cc:597] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:23.596405  5352 tablet_replica.cc:333] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.601362  5352 raft_consensus.cc:2243] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:21:23.602761  5352 raft_consensus.cc:2272] T 6d6c58127aec4100b761ec4b52b842a0 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.606122  5352 tablet_replica.cc:333] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.607558  5352 raft_consensus.cc:2243] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:23.608567  5352 raft_consensus.cc:2272] T 5b17fd01574f4601b33dd37e7c78bc8f P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:23.617693  6622 consensus_peers.cc:597] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:23.620483  5352 tablet_replica.cc:333] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.621817  5352 raft_consensus.cc:2243] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:23.622594  5352 raft_consensus.cc:2272] T 2ebfdf7a730a407185ead1aedcf4b8c4 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.625705  5352 tablet_replica.cc:333] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.626727  5352 raft_consensus.cc:2243] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:23.628660  5352 raft_consensus.cc:2272] T 591037e9504e4348a9cdf8842f34be22 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:23.657980  7240 consensus_peers.cc:597] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:23.642617  5352 tablet_replica.cc:333] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
W20260709 12:21:23.669606  6622 consensus_peers.cc:597] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:23.671571  5352 raft_consensus.cc:2243] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:23.672683  5352 raft_consensus.cc:2272] T 13ab2c6da5994ae2a955fc11d1761c41 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.677975  5352 tablet_replica.cc:333] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.679059  5352 raft_consensus.cc:2243] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:23.680244  5352 raft_consensus.cc:2272] T c6b5198a4b6741cdb13e49cf4e90173c P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.683055  5352 tablet_replica.cc:333] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.684056  5352 raft_consensus.cc:2243] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:23.684715  5352 raft_consensus.cc:2272] T e423f656ad70421f959a9a3b81dac708 P f2b06aebe97249a4877d798b8118ece4 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.687243  5352 tablet_replica.cc:333] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.688059  5352 raft_consensus.cc:2243] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:21:23.689251  5352 raft_consensus.cc:2272] T 6462df23af154f06b34f32a1ef4a45b2 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.691907  5352 tablet_replica.cc:333] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.693620  5352 raft_consensus.cc:2243] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:21:23.694904  5352 raft_consensus.cc:2272] T cbb34f14626f4aa895bae63ea2466e00 P f2b06aebe97249a4877d798b8118ece4 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.697930  5352 tablet_replica.cc:333] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.699942  5352 raft_consensus.cc:2243] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:23.700698  5352 raft_consensus.cc:2272] T ab430742d4f44bf79c4591f613055cd6 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.703426  5352 tablet_replica.cc:333] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.704298  5352 raft_consensus.cc:2243] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:23.705013  5352 raft_consensus.cc:2272] T 72cc6298c451423d8fe8c456777eb890 P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.708665  5352 tablet_replica.cc:333] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.709605  5352 raft_consensus.cc:2243] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:21:23.711229  5352 raft_consensus.cc:2272] T fa09afefaa884dc5943b210cb23a07b3 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:23.719758  6622 consensus_peers.cc:597] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:21:23.722476  5352 tablet_replica.cc:333] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.723569  5352 raft_consensus.cc:2243] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:23.724576  5352 raft_consensus.cc:2272] T 72269ad6925341548796bb34823b3e00 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.727312  5352 tablet_replica.cc:333] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.728155  5352 raft_consensus.cc:2243] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:23.728902  5352 raft_consensus.cc:2272] T e91732822ea14700aa3230fbb6b1cfac P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:23.738458  6622 consensus_peers.cc:597] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:23.747742  5352 tablet_replica.cc:333] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.749179  5352 raft_consensus.cc:2243] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:23.749859  5352 raft_consensus.cc:2272] T f5b42468eaf14fd49be100af079d3921 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.752751  5352 tablet_replica.cc:333] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.753829  5352 raft_consensus.cc:2243] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:21:23.756543  5352 raft_consensus.cc:2272] T 1997e98759d0465ab0e3c607df127068 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.760146  5352 tablet_replica.cc:333] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.761459  5352 raft_consensus.cc:2243] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:21:23.762687  5352 raft_consensus.cc:2272] T 3318181e3c0b433a9e1cee517969105c P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.778690  5352 tablet_replica.cc:333] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.781288  5352 raft_consensus.cc:2243] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:21:23.782759  5352 raft_consensus.cc:2272] T 53dc690d5068460a820e0ba9e24494ec P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.785872  5352 tablet_replica.cc:333] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.788580  5352 raft_consensus.cc:2243] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:23.789393  5352 raft_consensus.cc:2272] T 1c226c3677f8446f837400fd0df3062e P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.793350  5352 tablet_replica.cc:333] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.794168  5352 raft_consensus.cc:2243] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:23.794843  5352 raft_consensus.cc:2272] T 5b0b7ace18d848afa42e884bcd2475a4 P f2b06aebe97249a4877d798b8118ece4 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.798309  5352 tablet_replica.cc:333] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.799172  5352 raft_consensus.cc:2243] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:23.800503  5352 raft_consensus.cc:2272] T 3e0ab356fc074f589a6ff4f9b4996b09 P f2b06aebe97249a4877d798b8118ece4 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.803774  5352 tablet_replica.cc:333] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.804709  5352 raft_consensus.cc:2243] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:23.805382  5352 raft_consensus.cc:2272] T 8ea6f08346334787a6b558ca10a70249 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.808598  5352 tablet_replica.cc:333] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.809435  5352 raft_consensus.cc:2243] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:23.810441  5352 raft_consensus.cc:2272] T bd6fb503c26a49bab14a18c9b5e2d792 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.813560  5352 tablet_replica.cc:333] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.814505  5352 raft_consensus.cc:2243] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:21:23.815789  5352 raft_consensus.cc:2272] T 1d80f18ce04d4d97b0b3c5dc2ed0e4a4 P f2b06aebe97249a4877d798b8118ece4 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.818532  5352 tablet_replica.cc:333] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.819370  5352 raft_consensus.cc:2243] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:23.820035  5352 raft_consensus.cc:2272] T eebce9eca2a5496bb4a3ab473a243166 P f2b06aebe97249a4877d798b8118ece4 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.822983  5352 tablet_replica.cc:333] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.823858  5352 raft_consensus.cc:2243] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:21:23.825675  5352 pending_rounds.cc:70] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4: Trying to abort 1 pending ops.
I20260709 12:21:23.825975  5352 pending_rounds.cc:77] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4: Aborting op as it isn't in flight: id { term: 24 index: 14 } timestamp: 7305624288708423680 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "7492a9464009416aa4c3a11e145dec61" old_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } } new_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } } }
I20260709 12:21:23.827358  5352 raft_consensus.cc:2917] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 24 LEADER]: Aborting config change with OpId 24.14: Aborted: Op aborted
I20260709 12:21:23.827775  5352 raft_consensus.cc:2889] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 24 LEADER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 12:21:23.828150  5352 raft_consensus.cc:2272] T 7492a9464009416aa4c3a11e145dec61 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.834867  5352 tablet_replica.cc:333] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.835665  5352 raft_consensus.cc:2243] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:21:23.836781  5352 raft_consensus.cc:2272] T b7d79febb5824e78a295b2c993c14108 P f2b06aebe97249a4877d798b8118ece4 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:23.836838  7238 consensus_peers.cc:597] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:23.839545  5352 tablet_replica.cc:333] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.840382  5352 raft_consensus.cc:2243] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 12:21:23.840991  7238 consensus_peers.cc:597] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:23.841043  5352 raft_consensus.cc:2272] T d1778f67a2ba4fc2825ec2be5097d77a P f2b06aebe97249a4877d798b8118ece4 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.843979  5352 tablet_replica.cc:333] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.844857  5352 raft_consensus.cc:2243] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:21:23.846138  5352 raft_consensus.cc:2272] T c27974d342ea4901aaa3946871539818 P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.849411  5352 tablet_replica.cc:333] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.850339  5352 raft_consensus.cc:2243] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:21:23.851728  5352 raft_consensus.cc:2272] T 3e07e97c48184f8e99a4fc5a580724a5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.854981  5352 tablet_replica.cc:333] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.855914  5352 raft_consensus.cc:2243] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:23.856586  5352 raft_consensus.cc:2272] T 3af4a543c38a49bfa8ece9e702577ea1 P f2b06aebe97249a4877d798b8118ece4 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.859266  5352 tablet_replica.cc:333] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.860087  5352 raft_consensus.cc:2243] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:23.860769  5352 raft_consensus.cc:2272] T c22b6d97ae844b7195c5e9c024203d48 P f2b06aebe97249a4877d798b8118ece4 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.863682  5352 tablet_replica.cc:333] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.864557  5352 raft_consensus.cc:2243] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:23.865243  5352 raft_consensus.cc:2272] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P f2b06aebe97249a4877d798b8118ece4 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.868119  5352 tablet_replica.cc:333] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.868911  5352 raft_consensus.cc:2243] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:23.869864  5352 raft_consensus.cc:2272] T 6f94b1cd286648a9b82d5abf69bb86d5 P f2b06aebe97249a4877d798b8118ece4 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.872515  5352 tablet_replica.cc:333] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.873229  5352 raft_consensus.cc:2243] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:23.874631  5352 raft_consensus.cc:2272] T de5f3f59d0794f4c898c5e7f2b721abe P f2b06aebe97249a4877d798b8118ece4 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:23.877986  5352 tablet_replica.cc:333] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4: stopping tablet replica
I20260709 12:21:23.879071  5352 raft_consensus.cc:2243] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:23.879930  5352 raft_consensus.cc:2272] T 7ba56fbb3caa417db1c8d78a9caa28da P f2b06aebe97249a4877d798b8118ece4 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:23.911801  7238 consensus_peers.cc:597] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:23.944378  7238 consensus_peers.cc:597] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:24.064944  7238 consensus_peers.cc:597] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:24.122938  7238 consensus_peers.cc:597] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:24.185863  7238 consensus_peers.cc:597] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:24.187961  7238 consensus_peers.cc:597] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:24.224191  7238 consensus_peers.cc:597] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:24.224758  7238 consensus_peers.cc:597] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:24.337463  7238 consensus_peers.cc:597] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:24.380918  7238 consensus_peers.cc:597] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:24.381721  7238 consensus_peers.cc:597] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:24.434257 16166 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:24.435220 16166 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:24.438431 16166 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:24.446331  7238 consensus_peers.cc:597] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:24.457171  7238 leader_election.cc:336] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
W20260709 12:21:24.470352  7238 leader_election.cc:336] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:24.471155  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:24.472239 16166 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:21:24.644503 16166 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:24.645370 16166 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:24.646961 16177 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:24.647917 16166 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:24.647933 16177 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:24.655179 16166 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:24.657042 16177 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:24.657593 16166 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:24.661494 16166 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:21:24.661980  7238 leader_election.cc:336] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:24.680745  7240 leader_election.cc:336] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111)
I20260709 12:21:24.680799 16177 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:21:24.688130  7240 leader_election.cc:336] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111)
I20260709 12:21:24.688185 16177 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:21:24.689035  7238 leader_election.cc:336] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:24.689502  7238 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:21:24.694653  7238 leader_election.cc:336] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:24.695186  7238 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:24.696848 16179 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:21:24.697710 16179 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:21:24.698984  7240 leader_election.cc:336] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111)
I20260709 12:21:24.691633 16177 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:21:24.700006  7238 leader_election.cc:336] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:24.700609  7238 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:24.701495 16177 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:21:24.702389  7238 leader_election.cc:336] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:24.711447  7238 leader_election.cc:336] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:24.712033  7238 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:24.713183 16166 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:21:24.836292 16177 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:24.837160 16177 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:24.840907 16177 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:24.842916 16177 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
W20260709 12:21:24.846491  7238 leader_election.cc:336] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:24.847307  7240 leader_election.cc:336] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111)
I20260709 12:21:24.847776  7240 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:24.848541 16166 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:21:24.843902 16177 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:24.852171 16177 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:21:24.864050  7238 leader_election.cc:336] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:24.867048  7240 leader_election.cc:336] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111)
I20260709 12:21:24.869474  7240 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:24.871129 16177 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:21:24.901147 16177 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:24.901887 16177 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:24.904038 16177 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:24.906703 16166 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:24.915788 16166 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:24.917959 16166 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:24.918350  7238 leader_election.cc:336] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:24.920137  7238 leader_election.cc:336] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:24.920548  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:24.921404 16177 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:21:24.928964  7238 leader_election.cc:336] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
W20260709 12:21:24.929764  7238 leader_election.cc:336] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:24.930359  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:24.931313 16177 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:21:24.956187  7238 consensus_peers.cc:597] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:21:25.053913 16177 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:25.054642 16177 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:21:25.056798  7238 consensus_peers.cc:597] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:21:25.057178 16177 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:21:25.064471  7238 leader_election.cc:336] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:25.065290  7240 leader_election.cc:336] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111)
I20260709 12:21:25.065724  7240 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:25.066494 16177 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:21:25.070919  5352 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 6679)
W20260709 12:21:25.116323  7238 consensus_peers.cc:597] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:21:25.152796 16179 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:25.153712 16179 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:25.165169 16179 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:21:25.165721  7240 leader_election.cc:336] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111)
W20260709 12:21:25.170271  7238 leader_election.cc:336] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:25.170842  7238 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:25.171732 16179 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:21:25.172789  7238 leader_election.cc:336] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:25.174302 16179 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:25.175290 16179 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:25.177801 16179 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:21:25.184001  7240 leader_election.cc:336] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111)
I20260709 12:21:25.191113 16179 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:25.192027 16179 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:25.194502 16179 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:25.201750  7238 leader_election.cc:336] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:25.202394  7238 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:25.203240 16179 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:21:25.219254  7238 leader_election.cc:336] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
W20260709 12:21:25.221133  7238 leader_election.cc:336] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:25.221652  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:25.222479 16179 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:21:25.285688  7238 consensus_peers.cc:597] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:25.460157  7240 consensus_peers.cc:597] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:25.671482  7240 consensus_peers.cc:597] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:25.746572  7240 consensus_peers.cc:597] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:25.804946  7240 consensus_peers.cc:597] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:25.849788  7238 consensus_peers.cc:597] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:25.857899  7240 consensus_peers.cc:597] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:25.878414  7240 consensus_peers.cc:597] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:25.940909  7240 consensus_peers.cc:597] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:25.952555  7240 consensus_peers.cc:597] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:25.966277  7240 consensus_peers.cc:597] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:25.998011  7240 consensus_peers.cc:597] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:26.071421  5352 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 6679)
W20260709 12:21:26.082206  7238 consensus_peers.cc:597] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:26.143736  7238 consensus_peers.cc:597] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:26.152824  7238 consensus_peers.cc:597] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:26.290361 16179 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:26.290908 16179 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:26.293088 16179 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:21:26.297832  7238 leader_election.cc:336] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
W20260709 12:21:26.333762  7238 consensus_peers.cc:597] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:26.472167  7238 consensus_peers.cc:597] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:26.515841  7238 consensus_peers.cc:597] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:26.547663  7238 consensus_peers.cc:597] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:26.688078  7238 consensus_peers.cc:597] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:26.693040  7238 consensus_peers.cc:597] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:21:26.698946 16177 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:26.699658 16177 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:26.702817 16177 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:21:26.706933  7238 leader_election.cc:336] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
W20260709 12:21:26.742520  7238 consensus_peers.cc:597] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:26.744405  7238 consensus_peers.cc:597] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:26.756654  7238 consensus_peers.cc:597] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:21:26.774076 16179 raft_consensus.cc:493] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:26.774715 16179 raft_consensus.cc:515] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:26.777146 16179 leader_election.cc:290] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:27.072070  5352 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 6679)
I20260709 12:21:27.140161 16183 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:27.140946 16183 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:27.172053 16184 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:27.173807 16184 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:27.175840 16183 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:27.177204 16184 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:21:27.185127  7240 leader_election.cc:336] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111)
I20260709 12:21:27.185606  7240 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
W20260709 12:21:27.188365  7240 leader_election.cc:336] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111)
I20260709 12:21:27.189996 16185 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:27.191898 16185 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:27.207132 16185 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:27.207883 16185 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:27.209448 16185 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:27.211938 16177 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:21:27.216240 16188 raft_consensus.cc:493] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:27.228051 16188 raft_consensus.cc:515] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:27.230007 16185 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:27.221231 16183 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:27.231631 16188 leader_election.cc:290] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:27.231752 16183 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:27.241191 16183 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:21:27.237232  7238 leader_election.cc:336] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:27.243717  7238 leader_election.cc:336] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:27.244423  7238 leader_election.cc:304] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
W20260709 12:21:27.195709  7240 consensus_peers.cc:597] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:27.250689  7240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:36107, user_credentials={real_user=slave}} blocked reactor thread for 55037.9us
W20260709 12:21:27.251655  7240 consensus_peers.cc:597] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:21:27.252221  7240 leader_election.cc:336] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111)
I20260709 12:21:27.252722  7240 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:27.253597 16188 raft_consensus.cc:2749] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:21:27.253819  7240 consensus_peers.cc:597] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:21:27.254308 16188 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:21:27.273483  7240 leader_election.cc:336] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111)
W20260709 12:21:27.277207  7238 leader_election.cc:336] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:27.277704  7238 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
W20260709 12:21:27.283553  7240 leader_election.cc:336] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111)
W20260709 12:21:27.284185  7240 leader_election.cc:336] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111)
W20260709 12:21:27.284694  7240 leader_election.cc:336] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111)
I20260709 12:21:27.285604 16187 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:21:27.299302  7240 leader_election.cc:336] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111)
W20260709 12:21:27.304033  7238 leader_election.cc:336] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:27.304724  7238 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:27.305560 16187 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:21:27.307035  7238 leader_election.cc:336] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:27.307548  7238 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:27.308396 16187 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:21:27.309470  7238 leader_election.cc:336] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:27.309944  7238 leader_election.cc:304] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:27.310704 16187 raft_consensus.cc:2749] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:21:27.311661  7238 leader_election.cc:336] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:27.312160  7238 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:27.312875 16187 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:21:27.321640  7238 consensus_peers.cc:597] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:27.322790  7238 consensus_peers.cc:597] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:27.323424  7238 leader_election.cc:336] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
W20260709 12:21:27.326359  7238 consensus_peers.cc:597] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:27.326900  7238 leader_election.cc:336] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
W20260709 12:21:27.329841  7238 leader_election.cc:336] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:27.330317  7238 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:27.331110 16179 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:21:27.343163  7238 consensus_peers.cc:597] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:21:27.543376  7238 consensus_peers.cc:597] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:27.822600  6392 catalog_manager.cc:4755] TS e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 3318181e3c0b433a9e1cee517969105c: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:27.837704  7238 consensus_peers.cc:597] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:27.902900  7295 debug-util.cc:398] Leaking SignalData structure 0x7b08000e4480 after lost signal to thread 5355
W20260709 12:21:27.904196  7295 debug-util.cc:398] Leaking SignalData structure 0x7b0800382dc0 after lost signal to thread 6441
W20260709 12:21:27.905702  7295 debug-util.cc:398] Leaking SignalData structure 0x7b08003ca1c0 after lost signal to thread 7298
W20260709 12:21:28.072824  5352 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 6679)
W20260709 12:21:28.117725  7238 consensus_peers.cc:597] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:21:28.259531  7238 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111) [suppressed 411 similar messages]
W20260709 12:21:28.263367  7238 consensus_peers.cc:597] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:21:28.272967  7238 consensus_peers.cc:597] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:21:28.328608 16187 raft_consensus.cc:493] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:28.329550 16187 raft_consensus.cc:515] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:28.341388 16187 leader_election.cc:290] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:28.343479 16179 raft_consensus.cc:493] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:28.346325 16179 raft_consensus.cc:515] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:28.353590 16179 leader_election.cc:290] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:28.353680  7238 leader_election.cc:336] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:28.357754  7238 leader_election.cc:336] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:28.359475  7238 leader_election.cc:304] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:28.360533 16179 raft_consensus.cc:2749] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:21:28.368551  7238 leader_election.cc:336] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:28.369451  7238 leader_election.cc:336] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:28.372783  7238 leader_election.cc:304] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:28.373636 16193 raft_consensus.cc:2749] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:21:28.398653 16193 raft_consensus.cc:493] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:28.399586 16193 raft_consensus.cc:515] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } attrs { replace: true } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:28.403452 16193 leader_election.cc:290] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:21:28.411798  7240 leader_election.cc:336] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111)
W20260709 12:21:28.425347  7238 leader_election.cc:336] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:28.426288  7238 leader_election.cc:304] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:28.427546 16193 raft_consensus.cc:2749] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:21:28.428157  7238 leader_election.cc:336] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
W20260709 12:21:28.609771  7240 consensus_peers.cc:597] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:28.698766  7240 consensus_peers.cc:597] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:28.712560  7240 consensus_peers.cc:597] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:28.769219  7240 consensus_peers.cc:597] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:28.806726  7238 consensus_peers.cc:597] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:28.859018 16193 raft_consensus.cc:493] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:28.859625 16193 raft_consensus.cc:515] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:21:28.868037  7238 leader_election.cc:336] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:28.868891 16193 leader_election.cc:290] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:21:28.871676  7240 leader_election.cc:336] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111)
I20260709 12:21:28.872184  7240 leader_election.cc:304] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:28.872972 16193 raft_consensus.cc:2749] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:21:28.911628  7238 consensus_peers.cc:597] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:21:28.925482  7238 consensus_peers.cc:597] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:28.953714  7240 consensus_peers.cc:597] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:28.957131  7240 consensus_peers.cc:597] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:28.959709 16193 raft_consensus.cc:493] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:28.960312 16193 raft_consensus.cc:515] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:28.966634 16193 leader_election.cc:290] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:21:28.968246  7238 leader_election.cc:336] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
W20260709 12:21:28.974227  7240 leader_election.cc:336] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111)
I20260709 12:21:28.974817  7240 leader_election.cc:304] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:28.976163 16193 raft_consensus.cc:2749] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:21:29.044284  7240 consensus_peers.cc:597] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:29.073670  5352 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 6679)
I20260709 12:21:29.092028 16193 raft_consensus.cc:493] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:29.092770 16193 raft_consensus.cc:515] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:29.095113 16193 leader_election.cc:290] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
I20260709 12:21:29.106902 16187 raft_consensus.cc:493] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:29.107573 16187 raft_consensus.cc:515] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:21:29.119218  7240 leader_election.cc:336] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111)
W20260709 12:21:29.109375  7238 leader_election.cc:336] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:29.128053  7240 consensus_peers.cc:597] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:29.129918 16187 leader_election.cc:290] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:21:29.129160  7238 leader_election.cc:336] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:29.130735  7238 leader_election.cc:304] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:29.132122 16187 raft_consensus.cc:2749] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:21:29.137961  7238 consensus_peers.cc:597] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:21:29.141060  7238 consensus_peers.cc:597] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:29.141685  7238 consensus_peers.cc:597] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:29.147294  7238 leader_election.cc:336] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:29.147863  7238 leader_election.cc:304] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:29.148738 16187 raft_consensus.cc:2749] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:21:29.149838  7238 leader_election.cc:336] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
W20260709 12:21:29.180418  7240 consensus_peers.cc:597] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:29.250512  7240 consensus_peers.cc:597] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:21:29.270151 16187 raft_consensus.cc:493] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 496c7c3792cd4587b4dbeba435f4be1e)
I20260709 12:21:29.270805 16187 raft_consensus.cc:515] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:29.273530 16187 leader_election.cc:290] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101)
W20260709 12:21:29.275733  7238 consensus_peers.cc:597] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:29.303644  7238 leader_election.cc:336] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
W20260709 12:21:29.318204  7240 consensus_peers.cc:597] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:29.320200  7238 leader_election.cc:336] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:29.328615  7238 leader_election.cc:304] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, e7554c39679a41bbbca4659b72ab5c65
I20260709 12:21:29.329790 16187 raft_consensus.cc:2749] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:21:29.340898  7240 consensus_peers.cc:597] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:29.343951  7238 consensus_peers.cc:597] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:29.356886  7238 consensus_peers.cc:597] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:21:29.380995  7238 consensus_peers.cc:597] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:21:29.381609  7238 consensus_peers.cc:597] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:21:29.387084  7240 consensus_peers.cc:597] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 -> Peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Couldn't send request to peer f2b06aebe97249a4877d798b8118ece4. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:21:29.396425  7238 consensus_peers.cc:597] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:29.490833  7238 consensus_peers.cc:597] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:21:29.582731  7238 consensus_peers.cc:597] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:21:29.630525  7238 consensus_peers.cc:597] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:29.651489 16187 raft_consensus.cc:493] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:29.652050 16187 raft_consensus.cc:515] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:29.654387 16187 leader_election.cc:290] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:21:29.659720  7240 leader_election.cc:336] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111)
W20260709 12:21:29.666940  7238 leader_election.cc:336] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:29.668812  7238 leader_election.cc:304] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:29.669538 16187 raft_consensus.cc:2749] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:21:29.823082  7238 consensus_peers.cc:597] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 -> Peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Couldn't send request to peer 496c7c3792cd4587b4dbeba435f4be1e. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:21:29.859054 16187 raft_consensus.cc:493] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:29.859715 16187 raft_consensus.cc:515] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:29.865520 16187 leader_election.cc:290] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:29.866676 16187 raft_consensus.cc:493] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:29.868364 16187 raft_consensus.cc:515] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
W20260709 12:21:29.886636  7240 leader_election.cc:336] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111)
W20260709 12:21:29.887284  7240 leader_election.cc:336] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111)
I20260709 12:21:29.889845 16187 leader_election.cc:290] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
I20260709 12:21:29.893370 16187 raft_consensus.cc:493] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f2b06aebe97249a4877d798b8118ece4)
I20260709 12:21:29.894999 16187 raft_consensus.cc:515] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } }
I20260709 12:21:29.899899 16187 leader_election.cc:290] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643), f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107)
W20260709 12:21:29.919485  7238 leader_election.cc:336] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
W20260709 12:21:29.923198  7240 leader_election.cc:336] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer f2b06aebe97249a4877d798b8118ece4 (127.5.58.3:36107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:36107: connect: Connection refused (error 111)
I20260709 12:21:29.938725  7238 leader_election.cc:304] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:29.940039 16195 raft_consensus.cc:2749] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:21:29.943281  7238 leader_election.cc:336] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 496c7c3792cd4587b4dbeba435f4be1e (127.5.58.2:45101): Network error: Client connection negotiation failed: client connection to 127.5.58.2:45101: connect: Connection refused (error 111)
I20260709 12:21:29.943869  7238 leader_election.cc:304] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: 496c7c3792cd4587b4dbeba435f4be1e, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:29.944633 16195 raft_consensus.cc:2749] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:21:29.951098  7238 leader_election.cc:336] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:29.956497  7238 leader_election.cc:304] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20a202c76244485fbc8f1610473db8b9; no voters: e7554c39679a41bbbca4659b72ab5c65, f2b06aebe97249a4877d798b8118ece4
I20260709 12:21:29.957402 16196 raft_consensus.cc:2749] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:21:29.996068  5352 tablet_server.cc:196] TabletServer@127.5.58.3:0 shutdown complete.
W20260709 12:21:30.002063  7238 consensus_peers.cc:597] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:21:30.241421  7238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:36643, user_credentials={real_user=slave}} blocked reactor thread for 135650us
I20260709 12:21:30.278044  5352 tablet_server.cc:179] TabletServer@127.5.58.4:0 shutting down...
I20260709 12:21:30.333678  5352 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:21:30.336622  5352 tablet_replica.cc:333] T e423f656ad70421f959a9a3b81dac708 P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.337538  5352 raft_consensus.cc:2243] T e423f656ad70421f959a9a3b81dac708 P 20a202c76244485fbc8f1610473db8b9 [term 34 LEARNER]: Raft consensus shutting down.
I20260709 12:21:30.338183  5352 raft_consensus.cc:2272] T e423f656ad70421f959a9a3b81dac708 P 20a202c76244485fbc8f1610473db8b9 [term 34 LEARNER]: Raft consensus is shut down!
I20260709 12:21:30.341488  5352 tablet_replica.cc:333] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.342482  5352 raft_consensus.cc:2243] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.343465  5352 raft_consensus.cc:2272] T de5f3f59d0794f4c898c5e7f2b721abe P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:30.345888  7238 consensus_peers.cc:597] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 -> Peer e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): Couldn't send request to peer e7554c39679a41bbbca4659b72ab5c65. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:21:30.346848  5352 tablet_replica.cc:333] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.347681  5352 raft_consensus.cc:2243] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.349304  5352 raft_consensus.cc:2272] T 55df50d51529421789bb752f47dff41a P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.352082  5352 tablet_replica.cc:333] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.352844  5352 raft_consensus.cc:2243] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.354130  5352 raft_consensus.cc:2272] T 3a47d71c34b541b9b2589b4e80413fa2 P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.356559  5352 tablet_replica.cc:333] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.357300  5352 raft_consensus.cc:2243] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.358549  5352 raft_consensus.cc:2272] T 3318181e3c0b433a9e1cee517969105c P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.362072  5352 tablet_replica.cc:333] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.363013  5352 raft_consensus.cc:2243] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.364377  5352 raft_consensus.cc:2272] T 1997e98759d0465ab0e3c607df127068 P 20a202c76244485fbc8f1610473db8b9 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.367028  5352 tablet_replica.cc:333] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.367678  5352 raft_consensus.cc:2243] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.368294  5352 raft_consensus.cc:2272] T c9d981014b2c4cda80bca766fe58fe42 P 20a202c76244485fbc8f1610473db8b9 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.370620  5352 tablet_replica.cc:333] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.371250  5352 raft_consensus.cc:2243] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.372462  5352 raft_consensus.cc:2272] T c22b6d97ae844b7195c5e9c024203d48 P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.374753  5352 tablet_replica.cc:333] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.375448  5352 raft_consensus.cc:2243] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.376736  5352 raft_consensus.cc:2272] T 2ebfdf7a730a407185ead1aedcf4b8c4 P 20a202c76244485fbc8f1610473db8b9 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.378840  5352 tablet_replica.cc:333] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.379462  5352 raft_consensus.cc:2243] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.380118  5352 raft_consensus.cc:2272] T f9f8f5d393744ebabbed2f3e0131cf40 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.382532  5352 tablet_replica.cc:333] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.383423  5352 raft_consensus.cc:2243] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.384830  5352 raft_consensus.cc:2272] T f2592253d3214104865730480e057d09 P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.387522  5352 tablet_replica.cc:333] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.388123  5352 raft_consensus.cc:2243] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.389097  5352 raft_consensus.cc:2272] T e6167e4ddf454440bfc602ffbcd6da3e P 20a202c76244485fbc8f1610473db8b9 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.391275  5352 tablet_replica.cc:333] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.391808  5352 raft_consensus.cc:2243] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.392863  5352 raft_consensus.cc:2272] T 8e01550f5e994006a4031ca81edcc9aa P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.394923  5352 tablet_replica.cc:333] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.395498  5352 raft_consensus.cc:2243] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.396512  5352 raft_consensus.cc:2272] T 3e0ab356fc074f589a6ff4f9b4996b09 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.398622  5352 tablet_replica.cc:333] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.399205  5352 raft_consensus.cc:2243] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.400264  5352 raft_consensus.cc:2272] T eebce9eca2a5496bb4a3ab473a243166 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.402321  5352 tablet_replica.cc:333] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.402895  5352 raft_consensus.cc:2243] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.403519  5352 raft_consensus.cc:2272] T 6d6c58127aec4100b761ec4b52b842a0 P 20a202c76244485fbc8f1610473db8b9 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.405764  5352 tablet_replica.cc:333] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.406309  5352 raft_consensus.cc:2243] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.407428  5352 raft_consensus.cc:2272] T 5b17fd01574f4601b33dd37e7c78bc8f P 20a202c76244485fbc8f1610473db8b9 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.409672  5352 tablet_replica.cc:333] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.410255  5352 raft_consensus.cc:2243] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.411139  5352 raft_consensus.cc:2272] T c6b5198a4b6741cdb13e49cf4e90173c P 20a202c76244485fbc8f1610473db8b9 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.413228  5352 tablet_replica.cc:333] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.413856  5352 raft_consensus.cc:2243] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.414830  5352 raft_consensus.cc:2272] T d1778f67a2ba4fc2825ec2be5097d77a P 20a202c76244485fbc8f1610473db8b9 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.416813  5352 tablet_replica.cc:333] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.417428  5352 raft_consensus.cc:2243] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.417912  5352 pending_rounds.cc:70] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9: Trying to abort 1 pending ops.
I20260709 12:21:30.418150  5352 pending_rounds.cc:77] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9: Aborting op as it isn't in flight: id { term: 24 index: 14 } timestamp: 7305624288708423680 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "7492a9464009416aa4c3a11e145dec61" old_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } } new_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } attrs { replace: true } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } } }
I20260709 12:21:30.419162  5352 raft_consensus.cc:2917] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [term 24 FOLLOWER]: Aborting config change with OpId 24.14: Aborted: Op aborted
I20260709 12:21:30.419556  5352 raft_consensus.cc:2889] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [term 24 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 12:21:30.419842  5352 raft_consensus.cc:2272] T 7492a9464009416aa4c3a11e145dec61 P 20a202c76244485fbc8f1610473db8b9 [term 24 LEARNER]: Raft consensus is shut down!
I20260709 12:21:30.422227  5352 tablet_replica.cc:333] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.422812  5352 raft_consensus.cc:2243] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.423398  5352 raft_consensus.cc:2272] T cb7f6a796adb4b8c89579ba77274985f P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.425495  5352 tablet_replica.cc:333] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.426115  5352 raft_consensus.cc:2243] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.427104  5352 raft_consensus.cc:2272] T 6f94b1cd286648a9b82d5abf69bb86d5 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.430250  5352 tablet_replica.cc:333] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.430919  5352 raft_consensus.cc:2243] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 11 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.431895  5352 raft_consensus.cc:2272] T 8066a9d7ccf34c89ad8d3df18c4e86c2 P 20a202c76244485fbc8f1610473db8b9 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.433913  5352 tablet_replica.cc:333] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.434449  5352 raft_consensus.cc:2243] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.435371  5352 raft_consensus.cc:2272] T 14cf1c3a63624d94aa6c9c356e6108e2 P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.437523  5352 tablet_replica.cc:333] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.438032  5352 raft_consensus.cc:2243] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.438982  5352 raft_consensus.cc:2272] T 13ab2c6da5994ae2a955fc11d1761c41 P 20a202c76244485fbc8f1610473db8b9 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.440984  5352 tablet_replica.cc:333] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.441485  5352 raft_consensus.cc:2243] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.442065  5352 raft_consensus.cc:2272] T 3e07e97c48184f8e99a4fc5a580724a5 P 20a202c76244485fbc8f1610473db8b9 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.443877  5352 tablet_replica.cc:333] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.444442  5352 raft_consensus.cc:2243] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.445335  5352 raft_consensus.cc:2272] T 3e7223833e1f47b7b68c0080facbbe95 P 20a202c76244485fbc8f1610473db8b9 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.447304  5352 tablet_replica.cc:333] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.447916  5352 raft_consensus.cc:2243] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.448876  5352 raft_consensus.cc:2272] T 27473d665a69492b9d08ab7409de0b5f P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.451261  5352 tablet_replica.cc:333] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.451771  5352 raft_consensus.cc:2243] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.452409  5352 raft_consensus.cc:2272] T d6029ffeecb94e4da196c61c032b862b P 20a202c76244485fbc8f1610473db8b9 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.454819  5352 tablet_replica.cc:333] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.455477  5352 raft_consensus.cc:2243] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.455965  5352 pending_rounds.cc:70] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9: Trying to abort 1 pending ops.
I20260709 12:21:30.456235  5352 pending_rounds.cc:77] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9: Aborting op as it isn't in flight: id { term: 27 index: 16 } timestamp: 7305624286930915328 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "d0bd537a4ec844a69ca8227a47d1a8f3" old_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: NON_VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: true } } } new_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "e7554c39679a41bbbca4659b72ab5c65" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 36643 } attrs { replace: true } } peers { permanent_uuid: "496c7c3792cd4587b4dbeba435f4be1e" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45101 } } peers { permanent_uuid: "f2b06aebe97249a4877d798b8118ece4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 36107 } } peers { permanent_uuid: "20a202c76244485fbc8f1610473db8b9" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44857 } attrs { promote: false } } } }
I20260709 12:21:30.457208  5352 raft_consensus.cc:2917] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9 [term 27 FOLLOWER]: Aborting config change with OpId 27.16: Aborted: Op aborted
I20260709 12:21:30.457670  5352 raft_consensus.cc:2889] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9 [term 27 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 12:21:30.458005  5352 raft_consensus.cc:2272] T d0bd537a4ec844a69ca8227a47d1a8f3 P 20a202c76244485fbc8f1610473db8b9 [term 27 LEARNER]: Raft consensus is shut down!
I20260709 12:21:30.460119  5352 tablet_replica.cc:333] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.460635  5352 raft_consensus.cc:2243] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:21:30.461509  5352 raft_consensus.cc:2272] T e416f9423f964ba5a29e536e8160c627 P 20a202c76244485fbc8f1610473db8b9 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.463431  5352 tablet_replica.cc:333] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9: stopping tablet replica
I20260709 12:21:30.463935  5352 raft_consensus.cc:2243] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:30.464675  5352 raft_consensus.cc:2272] T 23f03345855041f8aabfa3e94a1867d9 P 20a202c76244485fbc8f1610473db8b9 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:30.910403  5352 tablet_server.cc:196] TabletServer@127.5.58.4:0 shutdown complete.
I20260709 12:21:30.983542  5352 master.cc:562] Master@127.5.58.62:36381 shutting down...
W20260709 12:21:31.035365  6455 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 12:21:31.035737  6455 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 12:21:31.036031  6455 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 12:21:31.110018  6438 debug-util.cc:398] Leaking SignalData structure 0x7b080000a440 after lost signal to thread 5355
W20260709 12:21:31.111025  6438 debug-util.cc:398] Leaking SignalData structure 0x7b08000c1220 after lost signal to thread 6441
W20260709 12:21:31.112183  6438 debug-util.cc:398] Leaking SignalData structure 0x7b08001b9ce0 after lost signal to thread 7298
W20260709 12:21:44.233152  6392 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111) [suppressed 171 similar messages]
W20260709 12:21:44.237130  6392 catalog_manager.cc:4755] TS e7554c39679a41bbbca4659b72ab5c65 (127.5.58.1:36643): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 3318181e3c0b433a9e1cee517969105c: Network error: Client connection negotiation failed: client connection to 127.5.58.1:36643: connect: Connection refused (error 111)
I20260709 12:21:44.654633  5352 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:44.655701  5352 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:44.656112  5352 tablet_replica.cc:333] T 00000000000000000000000000000000 P 39c4be726d3d495181bfe7425c8c0bd8: stopping tablet replica
I20260709 12:21:44.895572  5352 master.cc:584] Master@127.5.58.62:36381 shutdown complete.
[       OK ] LeaderRebalancerTest.AddTserver (302697 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 12:21:45.016322  5352 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.58.62:39363
I20260709 12:21:45.017596  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:45.023020 16201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:45.024827 16202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:45.025492  5352 server_base.cc:1061] running on GCE node
W20260709 12:21:45.025017 16204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:45.029676  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:45.029881  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:45.030082  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599705030048 us; error 0 us; skew 500 ppm
I20260709 12:21:45.030669  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:45.033699  5352 webserver.cc:533] Webserver started at http://127.5.58.62:43223/ using document root <none> and password file <none>
I20260709 12:21:45.034209  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:45.034406  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:45.034693  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:45.035938  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599341860723-5352-0/minicluster-data/master-0-root/instance:
uuid: "9e68af265dd04372955944ded6c1437a"
format_stamp: "Formatted at 2026-07-09 12:21:45 on dist-test-slave-xs2d"
I20260709 12:21:45.040527  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:21:45.043860 16209 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:45.044724  5352 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:21:45.045053  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599341860723-5352-0/minicluster-data/master-0-root
uuid: "9e68af265dd04372955944ded6c1437a"
format_stamp: "Formatted at 2026-07-09 12:21:45 on dist-test-slave-xs2d"
I20260709 12:21:45.045359  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599341860723-5352-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599341860723-5352-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599341860723-5352-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:45.071974  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:45.073334  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:45.113785  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.62:39363
I20260709 12:21:45.113945 16260 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.62:39363 every 8 connection(s)
I20260709 12:21:45.117872 16261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:45.129328 16261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a: Bootstrap starting.
I20260709 12:21:45.134485 16261 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:45.138855 16261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a: No bootstrap required, opened a new log
I20260709 12:21:45.141602 16261 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e68af265dd04372955944ded6c1437a" member_type: VOTER }
I20260709 12:21:45.142023 16261 raft_consensus.cc:385] T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:45.142323 16261 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e68af265dd04372955944ded6c1437a, State: Initialized, Role: FOLLOWER
I20260709 12:21:45.142928 16261 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e68af265dd04372955944ded6c1437a" member_type: VOTER }
I20260709 12:21:45.143389 16261 raft_consensus.cc:399] T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:21:45.143610 16261 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:21:45.143927 16261 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:45.149169 16261 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e68af265dd04372955944ded6c1437a" member_type: VOTER }
I20260709 12:21:45.149767 16261 leader_election.cc:304] T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e68af265dd04372955944ded6c1437a; no voters: 
I20260709 12:21:45.150916 16261 leader_election.cc:290] T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:21:45.151399 16264 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:45.152877 16264 raft_consensus.cc:697] T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a [term 1 LEADER]: Becoming Leader. State: Replica: 9e68af265dd04372955944ded6c1437a, State: Running, Role: LEADER
I20260709 12:21:45.153648 16264 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e68af265dd04372955944ded6c1437a" member_type: VOTER }
I20260709 12:21:45.153894 16261 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:21:45.163576 16266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e68af265dd04372955944ded6c1437a. Latest consensus state: current_term: 1 leader_uuid: "9e68af265dd04372955944ded6c1437a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e68af265dd04372955944ded6c1437a" member_type: VOTER } }
I20260709 12:21:45.163529 16265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e68af265dd04372955944ded6c1437a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e68af265dd04372955944ded6c1437a" member_type: VOTER } }
I20260709 12:21:45.164572 16266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a [sys.catalog]: This master's current role is: LEADER
I20260709 12:21:45.164650 16265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a [sys.catalog]: This master's current role is: LEADER
I20260709 12:21:45.166352 16274 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:21:45.174175 16274 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:21:45.175154  5352 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:21:45.183588 16274 catalog_manager.cc:1383] Generated new cluster ID: 2df6e77cc80a433caa9bf52202984474
I20260709 12:21:45.183867 16274 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:21:45.200466 16274 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:21:45.201864 16274 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:21:45.213137 16274 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a: Generated new TSK 0
I20260709 12:21:45.213789 16274 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:21:45.242027  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:45.248946 16282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:45.250064 16283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:45.251750 16285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:45.253188  5352 server_base.cc:1061] running on GCE node
I20260709 12:21:45.255904  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:45.256134  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:45.256319  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599705256295 us; error 0 us; skew 500 ppm
I20260709 12:21:45.256927  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:45.259394  5352 webserver.cc:533] Webserver started at http://127.5.58.1:43597/ using document root <none> and password file <none>
I20260709 12:21:45.259886  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:45.260093  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:45.260434  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:45.261646  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599341860723-5352-0/minicluster-data/ts-0-root/instance:
uuid: "ef57beab0e1d40d585983873c55107e8"
format_stamp: "Formatted at 2026-07-09 12:21:45 on dist-test-slave-xs2d"
I20260709 12:21:45.266562  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:21:45.270278 16290 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:45.271243  5352 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:21:45.271626  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599341860723-5352-0/minicluster-data/ts-0-root
uuid: "ef57beab0e1d40d585983873c55107e8"
format_stamp: "Formatted at 2026-07-09 12:21:45 on dist-test-slave-xs2d"
I20260709 12:21:45.272019  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599341860723-5352-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599341860723-5352-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599341860723-5352-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:45.297315  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:45.298521  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:45.299969  5352 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:45.302695  5352 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:45.302870  5352 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:45.303223  5352 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:45.303380  5352 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:45.349234  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.1:41805
I20260709 12:21:45.349315 16352 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.1:41805 every 8 connection(s)
I20260709 12:21:45.355242  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:45.364485 16358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:45.366151 16357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:45.367080 16360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:45.368206  5352 server_base.cc:1061] running on GCE node
I20260709 12:21:45.370714 16353 heartbeater.cc:344] Connected to a master server at 127.5.58.62:39363
I20260709 12:21:45.371112 16353 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:45.371510  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:45.371757  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:45.371940  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599705371909 us; error 0 us; skew 500 ppm
I20260709 12:21:45.371976 16353 heartbeater.cc:507] Master 127.5.58.62:39363 requested a full tablet report, sending...
I20260709 12:21:45.372725  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:45.374553 16226 ts_manager.cc:194] Registered new tserver with Master: ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:45.375803  5352 webserver.cc:533] Webserver started at http://127.5.58.2:38613/ using document root <none> and password file <none>
I20260709 12:21:45.376411  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:45.376601  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:45.376878  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:45.376796 16226 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53612
I20260709 12:21:45.378569  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599341860723-5352-0/minicluster-data/ts-1-root/instance:
uuid: "f99066d13e524238aec6aa2024a8bd50"
format_stamp: "Formatted at 2026-07-09 12:21:45 on dist-test-slave-xs2d"
I20260709 12:21:45.383476  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:21:45.387929 16365 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:45.388880  5352 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:21:45.389194  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599341860723-5352-0/minicluster-data/ts-1-root
uuid: "f99066d13e524238aec6aa2024a8bd50"
format_stamp: "Formatted at 2026-07-09 12:21:45 on dist-test-slave-xs2d"
I20260709 12:21:45.389467  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599341860723-5352-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599341860723-5352-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599341860723-5352-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:45.399693  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:45.401062  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:45.402573  5352 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:45.404937  5352 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:45.405176  5352 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:45.405547  5352 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:45.405754  5352 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:45.450855  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.2:41367
I20260709 12:21:45.451007 16427 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.2:41367 every 8 connection(s)
I20260709 12:21:45.456529  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:45.466012 16432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:45.466140 16431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:45.469305  5352 server_base.cc:1061] running on GCE node
W20260709 12:21:45.470288 16434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:45.471323  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:45.471591  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:45.471792  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599705471768 us; error 0 us; skew 500 ppm
I20260709 12:21:45.472465  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:45.472455 16428 heartbeater.cc:344] Connected to a master server at 127.5.58.62:39363
I20260709 12:21:45.473096 16428 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:45.474130 16428 heartbeater.cc:507] Master 127.5.58.62:39363 requested a full tablet report, sending...
I20260709 12:21:45.475621  5352 webserver.cc:533] Webserver started at http://127.5.58.3:46453/ using document root <none> and password file <none>
I20260709 12:21:45.476279  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:45.476549  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:45.476826  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:45.476647 16226 ts_manager.cc:194] Registered new tserver with Master: f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:45.478169  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599341860723-5352-0/minicluster-data/ts-2-root/instance:
uuid: "52ab50b0237346c1a13ede947f6a63c2"
format_stamp: "Formatted at 2026-07-09 12:21:45 on dist-test-slave-xs2d"
I20260709 12:21:45.478456 16226 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53622
I20260709 12:21:45.483261  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.004s
I20260709 12:21:45.486872 16439 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:45.487843  5352 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:21:45.488160  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599341860723-5352-0/minicluster-data/ts-2-root
uuid: "52ab50b0237346c1a13ede947f6a63c2"
format_stamp: "Formatted at 2026-07-09 12:21:45 on dist-test-slave-xs2d"
I20260709 12:21:45.488531  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599341860723-5352-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599341860723-5352-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599341860723-5352-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:45.524008  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:45.525393  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:45.526881  5352 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:45.529177  5352 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:45.529399  5352 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:45.529639  5352 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:45.529812  5352 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:45.577996  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.3:33107
I20260709 12:21:45.578115 16501 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.3:33107 every 8 connection(s)
I20260709 12:21:45.583535  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:45.590870 16505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:45.592612 16506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:45.594379 16508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:45.596433  5352 server_base.cc:1061] running on GCE node
I20260709 12:21:45.597769  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:45.598090  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:45.597970 16502 heartbeater.cc:344] Connected to a master server at 127.5.58.62:39363
I20260709 12:21:45.598441  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599705598401 us; error 0 us; skew 500 ppm
I20260709 12:21:45.598658 16502 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:45.599241  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:45.599660 16502 heartbeater.cc:507] Master 127.5.58.62:39363 requested a full tablet report, sending...
I20260709 12:21:45.602259 16226 ts_manager.cc:194] Registered new tserver with Master: 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:45.602398  5352 webserver.cc:533] Webserver started at http://127.5.58.4:37091/ using document root <none> and password file <none>
I20260709 12:21:45.603096  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:45.603315  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:21:45.603649  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:21:45.604023 16226 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53626
I20260709 12:21:45.605124  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599341860723-5352-0/minicluster-data/ts-3-root/instance:
uuid: "610f4feda11942739236f2b121a4b9c1"
format_stamp: "Formatted at 2026-07-09 12:21:45 on dist-test-slave-xs2d"
I20260709 12:21:45.610244  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:21:45.614034 16513 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:45.614830  5352 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:21:45.615144  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599341860723-5352-0/minicluster-data/ts-3-root
uuid: "610f4feda11942739236f2b121a4b9c1"
format_stamp: "Formatted at 2026-07-09 12:21:45 on dist-test-slave-xs2d"
I20260709 12:21:45.615417  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599341860723-5352-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599341860723-5352-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599341860723-5352-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:45.628654  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:45.629922  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:45.631433  5352 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:45.633823  5352 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:21:45.634058  5352 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:45.634308  5352 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:21:45.634490  5352 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:45.678323  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.4:44491
I20260709 12:21:45.678478 16575 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.4:44491 every 8 connection(s)
I20260709 12:21:45.693723 16576 heartbeater.cc:344] Connected to a master server at 127.5.58.62:39363
I20260709 12:21:45.694227 16576 heartbeater.cc:461] Registering TS with master...
I20260709 12:21:45.695098 16576 heartbeater.cc:507] Master 127.5.58.62:39363 requested a full tablet report, sending...
I20260709 12:21:45.697214 16226 ts_manager.cc:194] Registered new tserver with Master: 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:45.698175  5352 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.016490612s
I20260709 12:21:45.698567 16226 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53636
I20260709 12:21:45.728896 16226 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53644:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 12:21:45.989646 16467 tablet_service.cc:1511] Processing CreateTablet for tablet 95b81a29889843548dd6ed149807ecc8 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:21:45.991842 16467 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95b81a29889843548dd6ed149807ecc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:45.999265 16393 tablet_service.cc:1511] Processing CreateTablet for tablet 95b81a29889843548dd6ed149807ecc8 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:21:46.000902 16393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95b81a29889843548dd6ed149807ecc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.006052 16466 tablet_service.cc:1511] Processing CreateTablet for tablet a60dbe0067044806b384e90805c877bc (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:21:46.007861 16466 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a60dbe0067044806b384e90805c877bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.008348 16317 tablet_service.cc:1511] Processing CreateTablet for tablet edff15255cdf492dbc9c1189b889aa78 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:21:46.010017 16317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edff15255cdf492dbc9c1189b889aa78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.009639 16316 tablet_service.cc:1511] Processing CreateTablet for tablet a60dbe0067044806b384e90805c877bc (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:21:46.011224 16316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a60dbe0067044806b384e90805c877bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.006142 16318 tablet_service.cc:1511] Processing CreateTablet for tablet 95b81a29889843548dd6ed149807ecc8 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:21:46.012017 16465 tablet_service.cc:1511] Processing CreateTablet for tablet 4a0e81993a5641399dd0ca4ded8b4037 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:21:46.012552 16318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95b81a29889843548dd6ed149807ecc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.013567 16465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a0e81993a5641399dd0ca4ded8b4037. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.018028 16464 tablet_service.cc:1511] Processing CreateTablet for tablet 33037c31b4434e838d90b2a2065ef52d (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:21:46.019814 16464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33037c31b4434e838d90b2a2065ef52d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.021821 16392 tablet_service.cc:1511] Processing CreateTablet for tablet edff15255cdf492dbc9c1189b889aa78 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:21:46.023362 16392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edff15255cdf492dbc9c1189b889aa78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.025805 16390 tablet_service.cc:1511] Processing CreateTablet for tablet 33037c31b4434e838d90b2a2065ef52d (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:21:46.027487 16390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33037c31b4434e838d90b2a2065ef52d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.029711 16463 tablet_service.cc:1511] Processing CreateTablet for tablet 48bd85a0339b41fba1759ca092d6e95e (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:21:46.031632 16463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48bd85a0339b41fba1759ca092d6e95e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.033550 16462 tablet_service.cc:1511] Processing CreateTablet for tablet 1a79e67586f842a7ba4c25dfd7cdf4d5 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:21:46.035027 16462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a79e67586f842a7ba4c25dfd7cdf4d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.037225 16461 tablet_service.cc:1511] Processing CreateTablet for tablet 3c4fd014cdf249c996641fe7793f42a5 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:21:46.037508 16389 tablet_service.cc:1511] Processing CreateTablet for tablet 434028125a2d48998f4f5106c59a9d47 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:21:46.038617 16461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c4fd014cdf249c996641fe7793f42a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.038931 16389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 434028125a2d48998f4f5106c59a9d47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.041607 16387 tablet_service.cc:1511] Processing CreateTablet for tablet 3c4fd014cdf249c996641fe7793f42a5 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:21:46.043088 16387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c4fd014cdf249c996641fe7793f42a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.045619 16315 tablet_service.cc:1511] Processing CreateTablet for tablet 33037c31b4434e838d90b2a2065ef52d (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:21:46.046444 16460 tablet_service.cc:1511] Processing CreateTablet for tablet 025780567e194437bc14e26465b690e3 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:21:46.046634 16311 tablet_service.cc:1511] Processing CreateTablet for tablet 3ba84aa22298489eb2845bec782a34f2 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:21:46.047266 16315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33037c31b4434e838d90b2a2065ef52d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.047916 16460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 025780567e194437bc14e26465b690e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.048110 16311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ba84aa22298489eb2845bec782a34f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.049643 16313 tablet_service.cc:1511] Processing CreateTablet for tablet 48bd85a0339b41fba1759ca092d6e95e (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:21:46.051152 16313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48bd85a0339b41fba1759ca092d6e95e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.053663 16312 tablet_service.cc:1511] Processing CreateTablet for tablet 3c4fd014cdf249c996641fe7793f42a5 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:21:46.054435 16459 tablet_service.cc:1511] Processing CreateTablet for tablet 0b9c348d235e432d92d8f8f20f1f1317 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:21:46.055176 16312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c4fd014cdf249c996641fe7793f42a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.055855 16459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b9c348d235e432d92d8f8f20f1f1317. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.057610 16386 tablet_service.cc:1511] Processing CreateTablet for tablet 3ba84aa22298489eb2845bec782a34f2 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:21:46.058931 16391 tablet_service.cc:1511] Processing CreateTablet for tablet 4a0e81993a5641399dd0ca4ded8b4037 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:21:46.057626 16388 tablet_service.cc:1511] Processing CreateTablet for tablet 1a79e67586f842a7ba4c25dfd7cdf4d5 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:21:46.059044 16386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ba84aa22298489eb2845bec782a34f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.061511 16314 tablet_service.cc:1511] Processing CreateTablet for tablet 434028125a2d48998f4f5106c59a9d47 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:21:46.061532 16458 tablet_service.cc:1511] Processing CreateTablet for tablet 162cebc248914fb5aa57de3a383a4159 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:21:46.063063 16458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 162cebc248914fb5aa57de3a383a4159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.064771 16388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a79e67586f842a7ba4c25dfd7cdf4d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.063063 16314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 434028125a2d48998f4f5106c59a9d47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.068723 16391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a0e81993a5641399dd0ca4ded8b4037. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.090838 16310 tablet_service.cc:1511] Processing CreateTablet for tablet 025780567e194437bc14e26465b690e3 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:21:46.091790 16385 tablet_service.cc:1511] Processing CreateTablet for tablet 0b9c348d235e432d92d8f8f20f1f1317 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:21:46.093163 16310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 025780567e194437bc14e26465b690e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.093259 16385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b9c348d235e432d92d8f8f20f1f1317. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.100085 16384 tablet_service.cc:1511] Processing CreateTablet for tablet 162cebc248914fb5aa57de3a383a4159 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:21:46.101527 16384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 162cebc248914fb5aa57de3a383a4159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.111248 16309 tablet_service.cc:1511] Processing CreateTablet for tablet 162cebc248914fb5aa57de3a383a4159 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:21:46.112965 16309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 162cebc248914fb5aa57de3a383a4159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.158565 16460 tablet_service.cc:1511] Processing CreateTablet for tablet 4777f1999cdc462c8526206c2f39fe0b (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:21:46.160030 16460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4777f1999cdc462c8526206c2f39fe0b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:21:46.167834 16273 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:46.171247 16463 tablet_service.cc:1511] Processing CreateTablet for tablet 57fce16c9dc745ce8d5a631936c11e00 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:21:46.173285 16463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57fce16c9dc745ce8d5a631936c11e00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.177666 16460 tablet_service.cc:1511] Processing CreateTablet for tablet 9cd596d6e69a4b58804e76d2dd4f51f1 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:21:46.179059 16460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cd596d6e69a4b58804e76d2dd4f51f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.183843 16464 tablet_service.cc:1511] Processing CreateTablet for tablet 2da151f033444a308804c713dd5ab691 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:21:46.185334 16464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2da151f033444a308804c713dd5ab691. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.154655 16317 tablet_service.cc:1511] Processing CreateTablet for tablet aadc86310ab642c69b7dbbf94be01070 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:21:46.209857 16390 tablet_service.cc:1511] Processing CreateTablet for tablet 57fce16c9dc745ce8d5a631936c11e00 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:21:46.212141 16392 tablet_service.cc:1511] Processing CreateTablet for tablet 9cd596d6e69a4b58804e76d2dd4f51f1 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:21:46.219266 16467 tablet_service.cc:1511] Processing CreateTablet for tablet 766cb112f8024db5af150d6be611a597 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:21:46.214962 16463 tablet_service.cc:1511] Processing CreateTablet for tablet 44b9e8076a1d43319b0c86d4b356ac68 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:21:46.218565 16390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57fce16c9dc745ce8d5a631936c11e00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.224726 16392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cd596d6e69a4b58804e76d2dd4f51f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.226430 16318 tablet_service.cc:1511] Processing CreateTablet for tablet 9cd596d6e69a4b58804e76d2dd4f51f1 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:21:46.228240 16318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cd596d6e69a4b58804e76d2dd4f51f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.229894 16384 tablet_service.cc:1511] Processing CreateTablet for tablet 472c1e509cf2401e8153946529cfba2e (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:21:46.234300 16384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 472c1e509cf2401e8153946529cfba2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.239970 16309 tablet_service.cc:1511] Processing CreateTablet for tablet 2da151f033444a308804c713dd5ab691 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:21:46.241465 16309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2da151f033444a308804c713dd5ab691. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.241268 16462 tablet_service.cc:1511] Processing CreateTablet for tablet d46e3072dec049b18b323d65470603ec (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:21:46.242942 16388 tablet_service.cc:1511] Processing CreateTablet for tablet aadc86310ab642c69b7dbbf94be01070 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:21:46.244285 16388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aadc86310ab642c69b7dbbf94be01070. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.246541 16317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aadc86310ab642c69b7dbbf94be01070. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.249784 16318 tablet_service.cc:1511] Processing CreateTablet for tablet dbdcb11d24644ca0bef26ea87647f4ac (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:21:46.251039 16318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbdcb11d24644ca0bef26ea87647f4ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.253734 16314 tablet_service.cc:1511] Processing CreateTablet for tablet 766cb112f8024db5af150d6be611a597 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:21:46.255060 16314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 766cb112f8024db5af150d6be611a597. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.255554 16310 tablet_service.cc:1511] Processing CreateTablet for tablet 472c1e509cf2401e8153946529cfba2e (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:21:46.256826 16310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 472c1e509cf2401e8153946529cfba2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.257164 16315 tablet_service.cc:1511] Processing CreateTablet for tablet 44b9e8076a1d43319b0c86d4b356ac68 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:21:46.258426 16315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44b9e8076a1d43319b0c86d4b356ac68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.263278 16596 tablet_bootstrap.cc:492] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:46.264281 16311 tablet_service.cc:1511] Processing CreateTablet for tablet 49b3f77bde864e0ab25c63401793c709 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:21:46.270989 16311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49b3f77bde864e0ab25c63401793c709. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.265946 16316 tablet_service.cc:1511] Processing CreateTablet for tablet 4777f1999cdc462c8526206c2f39fe0b (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:21:46.274818 16316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4777f1999cdc462c8526206c2f39fe0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.255038 16467 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 766cb112f8024db5af150d6be611a597. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.280722 16463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44b9e8076a1d43319b0c86d4b356ac68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.281210 16595 tablet_bootstrap.cc:492] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:46.290830 16462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d46e3072dec049b18b323d65470603ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.294889 16312 tablet_service.cc:1511] Processing CreateTablet for tablet 191e052863dd48fe92ebe151197330a8 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:21:46.296417 16312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 191e052863dd48fe92ebe151197330a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.297981 16595 tablet_bootstrap.cc:654] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:46.301061 16596 tablet_bootstrap.cc:654] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:46.305431 16541 tablet_service.cc:1511] Processing CreateTablet for tablet edff15255cdf492dbc9c1189b889aa78 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:21:46.306999 16541 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edff15255cdf492dbc9c1189b889aa78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.309513 16540 tablet_service.cc:1511] Processing CreateTablet for tablet a60dbe0067044806b384e90805c877bc (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:21:46.310959 16540 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a60dbe0067044806b384e90805c877bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.311601 16536 tablet_service.cc:1511] Processing CreateTablet for tablet 1a79e67586f842a7ba4c25dfd7cdf4d5 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:21:46.313234 16536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a79e67586f842a7ba4c25dfd7cdf4d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.315584 16537 tablet_service.cc:1511] Processing CreateTablet for tablet 48bd85a0339b41fba1759ca092d6e95e (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:21:46.325299 16537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48bd85a0339b41fba1759ca092d6e95e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.317850 16538 tablet_service.cc:1511] Processing CreateTablet for tablet 434028125a2d48998f4f5106c59a9d47 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:21:46.329380 16538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 434028125a2d48998f4f5106c59a9d47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.319820 16539 tablet_service.cc:1511] Processing CreateTablet for tablet 4a0e81993a5641399dd0ca4ded8b4037 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:21:46.333381 16539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a0e81993a5641399dd0ca4ded8b4037. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.338135 16535 tablet_service.cc:1511] Processing CreateTablet for tablet 3ba84aa22298489eb2845bec782a34f2 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:21:46.338558 16534 tablet_service.cc:1511] Processing CreateTablet for tablet 025780567e194437bc14e26465b690e3 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:21:46.339987 16535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ba84aa22298489eb2845bec782a34f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.340365 16459 tablet_service.cc:1511] Processing CreateTablet for tablet 95954ea821224e4d962974a83d83ad44 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:21:46.341660 16459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95954ea821224e4d962974a83d83ad44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.341243 16533 tablet_service.cc:1511] Processing CreateTablet for tablet 0b9c348d235e432d92d8f8f20f1f1317 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:21:46.342670 16533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b9c348d235e432d92d8f8f20f1f1317. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.343650 16532 tablet_service.cc:1511] Processing CreateTablet for tablet aadc86310ab642c69b7dbbf94be01070 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:21:46.345086 16532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aadc86310ab642c69b7dbbf94be01070. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.347357 16534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 025780567e194437bc14e26465b690e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.359020 16464 tablet_service.cc:1511] Processing CreateTablet for tablet 1215b509464e4efe977922ffdeb0b064 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:21:46.365638 16393 tablet_service.cc:1511] Processing CreateTablet for tablet 49b3f77bde864e0ab25c63401793c709 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:21:46.322867 16458 tablet_service.cc:1511] Processing CreateTablet for tablet 472c1e509cf2401e8153946529cfba2e (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:21:46.370350 16389 tablet_service.cc:1511] Processing CreateTablet for tablet d46e3072dec049b18b323d65470603ec (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:21:46.388307 16353 heartbeater.cc:499] Master 127.5.58.62:39363 was elected leader, sending a full tablet report...
I20260709 12:21:46.389020 16313 tablet_service.cc:1511] Processing CreateTablet for tablet 1215b509464e4efe977922ffdeb0b064 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:21:46.389663 16389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d46e3072dec049b18b323d65470603ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.390364 16313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1215b509464e4efe977922ffdeb0b064. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.395905 16386 tablet_service.cc:1511] Processing CreateTablet for tablet dbdcb11d24644ca0bef26ea87647f4ac (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:21:46.397655 16386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbdcb11d24644ca0bef26ea87647f4ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.397317 16391 tablet_service.cc:1511] Processing CreateTablet for tablet 44b9e8076a1d43319b0c86d4b356ac68 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:21:46.399333 16391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44b9e8076a1d43319b0c86d4b356ac68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.416949 16597 tablet_bootstrap.cc:492] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:46.420754 16393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49b3f77bde864e0ab25c63401793c709. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.429602 16385 tablet_service.cc:1511] Processing CreateTablet for tablet 95954ea821224e4d962974a83d83ad44 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:21:46.432462 16385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95954ea821224e4d962974a83d83ad44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.432976 16390 tablet_service.cc:1511] Processing CreateTablet for tablet 191e052863dd48fe92ebe151197330a8 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:21:46.435225 16392 tablet_service.cc:1511] Processing CreateTablet for tablet e88d061c8b85425aa7478ab3874c4be4 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:21:46.438035 16464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1215b509464e4efe977922ffdeb0b064. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.445004 16540 tablet_service.cc:1511] Processing CreateTablet for tablet 191e052863dd48fe92ebe151197330a8 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:21:46.447317 16458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 472c1e509cf2401e8153946529cfba2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.444082 16392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e88d061c8b85425aa7478ab3874c4be4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.458431 16465 tablet_service.cc:1511] Processing CreateTablet for tablet e88d061c8b85425aa7478ab3874c4be4 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:21:46.459825 16465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e88d061c8b85425aa7478ab3874c4be4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.465114 16599 tablet_bootstrap.cc:492] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:46.466790 16535 tablet_service.cc:1511] Processing CreateTablet for tablet 4777f1999cdc462c8526206c2f39fe0b (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:21:46.477229 16317 tablet_service.cc:1511] Processing CreateTablet for tablet 6a90f33738a74f2c9fbf18bb0199dccf (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:21:46.478616 16461 tablet_service.cc:1511] Processing CreateTablet for tablet 6a90f33738a74f2c9fbf18bb0199dccf (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:21:46.489941 16461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a90f33738a74f2c9fbf18bb0199dccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.497241 16317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a90f33738a74f2c9fbf18bb0199dccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.479789 16597 tablet_bootstrap.cc:654] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:46.480634 16390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 191e052863dd48fe92ebe151197330a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.481295 16428 heartbeater.cc:499] Master 127.5.58.62:39363 was elected leader, sending a full tablet report...
I20260709 12:21:46.482681 16384 tablet_service.cc:1511] Processing CreateTablet for tablet 6a90f33738a74f2c9fbf18bb0199dccf (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:21:46.488927 16540 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 191e052863dd48fe92ebe151197330a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.523708 16534 tablet_service.cc:1511] Processing CreateTablet for tablet 57fce16c9dc745ce8d5a631936c11e00 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:21:46.526347 16384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a90f33738a74f2c9fbf18bb0199dccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.527455 16599 tablet_bootstrap.cc:654] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:46.528508 16535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4777f1999cdc462c8526206c2f39fe0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.534448 16537 tablet_service.cc:1511] Processing CreateTablet for tablet 49b3f77bde864e0ab25c63401793c709 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:21:46.535791 16537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49b3f77bde864e0ab25c63401793c709. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.558688 16595 tablet_bootstrap.cc:492] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:46.559420 16595 ts_tablet_manager.cc:1403] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.279s	user 0.011s	sys 0.011s
I20260709 12:21:46.562031 16536 tablet_service.cc:1511] Processing CreateTablet for tablet 2da151f033444a308804c713dd5ab691 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:21:46.568745 16596 tablet_bootstrap.cc:492] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:46.569339 16596 ts_tablet_manager.cc:1403] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.306s	user 0.015s	sys 0.007s
I20260709 12:21:46.569461 16538 tablet_service.cc:1511] Processing CreateTablet for tablet d46e3072dec049b18b323d65470603ec (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:21:46.577387 16541 tablet_service.cc:1511] Processing CreateTablet for tablet dbdcb11d24644ca0bef26ea87647f4ac (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:21:46.578439 16534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57fce16c9dc745ce8d5a631936c11e00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.579066 16538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d46e3072dec049b18b323d65470603ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.581647 16596 raft_consensus.cc:359] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:46.583381 16316 tablet_service.cc:1511] Processing CreateTablet for tablet 5f88a73c287745feb816089d0e08d10b (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:21:46.587102 16316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f88a73c287745feb816089d0e08d10b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.588682 16536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2da151f033444a308804c713dd5ab691. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.581699 16388 tablet_service.cc:1511] Processing CreateTablet for tablet 5f88a73c287745feb816089d0e08d10b (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:21:46.589784 16388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f88a73c287745feb816089d0e08d10b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.577524 16539 tablet_service.cc:1511] Processing CreateTablet for tablet 766cb112f8024db5af150d6be611a597 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:21:46.591503 16539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 766cb112f8024db5af150d6be611a597. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.585933 16596 raft_consensus.cc:385] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:46.600816 16596 raft_consensus.cc:740] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:46.601864 16596 consensus_queue.cc:260] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:46.607066 16502 heartbeater.cc:499] Master 127.5.58.62:39363 was elected leader, sending a full tablet report...
I20260709 12:21:46.609468 16463 tablet_service.cc:1511] Processing CreateTablet for tablet 7fe8595f696842c8821880524389c68b (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:21:46.610953 16463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fe8595f696842c8821880524389c68b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.613270 16311 tablet_service.cc:1511] Processing CreateTablet for tablet 7fe8595f696842c8821880524389c68b (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:21:46.614684 16311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fe8595f696842c8821880524389c68b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.585772 16533 tablet_service.cc:1511] Processing CreateTablet for tablet 95954ea821224e4d962974a83d83ad44 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:21:46.620361 16389 tablet_service.cc:1511] Processing CreateTablet for tablet b7b4960094cc44ceb337b594b1c269e6 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:21:46.629513 16389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7b4960094cc44ceb337b594b1c269e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.637959 16599 tablet_bootstrap.cc:492] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:46.638640 16599 ts_tablet_manager.cc:1403] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.174s	user 0.028s	sys 0.020s
I20260709 12:21:46.641983 16599 raft_consensus.cc:359] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:46.651286 16599 raft_consensus.cc:385] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:46.623833 16541 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbdcb11d24644ca0bef26ea87647f4ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.657583 16458 tablet_service.cc:1511] Processing CreateTablet for tablet b7b4960094cc44ceb337b594b1c269e6 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:21:46.659188 16458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7b4960094cc44ceb337b594b1c269e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.662952 16596 ts_tablet_manager.cc:1434] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.093s	user 0.011s	sys 0.001s
I20260709 12:21:46.670096 16532 tablet_service.cc:1511] Processing CreateTablet for tablet 1215b509464e4efe977922ffdeb0b064 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:21:46.664146 16596 tablet_bootstrap.cc:492] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:46.653034 16599 raft_consensus.cc:740] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:46.690093 16599 consensus_queue.cc:260] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:46.611209 16595 raft_consensus.cc:359] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:46.698403 16597 tablet_bootstrap.cc:492] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:46.698508 16533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95954ea821224e4d962974a83d83ad44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.699029 16597 ts_tablet_manager.cc:1403] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.283s	user 0.020s	sys 0.000s
I20260709 12:21:46.700309 16393 tablet_service.cc:1511] Processing CreateTablet for tablet e768efb7fcef4624b08d1e122c6dc825 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:21:46.702102 16576 heartbeater.cc:499] Master 127.5.58.62:39363 was elected leader, sending a full tablet report...
I20260709 12:21:46.707279 16461 tablet_service.cc:1511] Processing CreateTablet for tablet e768efb7fcef4624b08d1e122c6dc825 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:21:46.709187 16603 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:46.710024 16461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e768efb7fcef4624b08d1e122c6dc825. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.709694 16603 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:46.714241 16603 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:46.711122 16534 tablet_service.cc:1511] Processing CreateTablet for tablet 5f88a73c287745feb816089d0e08d10b (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:21:46.694387 16595 raft_consensus.cc:385] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:46.720315 16393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e768efb7fcef4624b08d1e122c6dc825. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.731984 16599 ts_tablet_manager.cc:1434] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.093s	user 0.010s	sys 0.002s
I20260709 12:21:46.733217 16536 tablet_service.cc:1511] Processing CreateTablet for tablet a4bb7e1a96a94612b2f6759156f5f7c0 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:21:46.732898 16392 tablet_service.cc:1511] Processing CreateTablet for tablet a4bb7e1a96a94612b2f6759156f5f7c0 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:21:46.718470 16595 raft_consensus.cc:740] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:46.753268 16595 consensus_queue.cc:260] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:46.741943 16596 tablet_bootstrap.cc:654] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:46.746436 16459 tablet_service.cc:1511] Processing CreateTablet for tablet c0b76873ad2645ebb97d6f5b03655773 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:21:46.746948 16536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4bb7e1a96a94612b2f6759156f5f7c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.747853 16537 tablet_service.cc:1511] Processing CreateTablet for tablet 7fe8595f696842c8821880524389c68b (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:21:46.749378 16597 raft_consensus.cc:359] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:46.700196 16535 tablet_service.cc:1511] Processing CreateTablet for tablet e88d061c8b85425aa7478ab3874c4be4 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:21:46.755190 16599 tablet_bootstrap.cc:492] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:46.765494 16392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4bb7e1a96a94612b2f6759156f5f7c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.765847 16459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0b76873ad2645ebb97d6f5b03655773. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.754218 16313 tablet_service.cc:1511] Processing CreateTablet for tablet e768efb7fcef4624b08d1e122c6dc825 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:21:46.767422 16313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e768efb7fcef4624b08d1e122c6dc825. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.762456 16597 raft_consensus.cc:385] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:46.779477 16597 raft_consensus.cc:740] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:46.780400 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:46.780620 16597 consensus_queue.cc:260] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:21:46.791348 16292 leader_election.cc:343] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:46.772289 16538 tablet_service.cc:1511] Processing CreateTablet for tablet c0b76873ad2645ebb97d6f5b03655773 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:21:46.792567 16604 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:46.793334 16604 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:46.796797 16604 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:46.797998 16534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f88a73c287745feb816089d0e08d10b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.820369 16463 tablet_service.cc:1511] Processing CreateTablet for tablet d1faa3c7bc3e4073bb65968a654e815d (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:21:46.821815 16463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1faa3c7bc3e4073bb65968a654e815d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.825788 16541 tablet_service.cc:1511] Processing CreateTablet for tablet d1faa3c7bc3e4073bb65968a654e815d (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:21:46.827840 16532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1215b509464e4efe977922ffdeb0b064. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.828573 16537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fe8595f696842c8821880524389c68b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.834687 16535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e88d061c8b85425aa7478ab3874c4be4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.837237 16541 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1faa3c7bc3e4073bb65968a654e815d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.827231 16388 tablet_service.cc:1511] Processing CreateTablet for tablet d1faa3c7bc3e4073bb65968a654e815d (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:21:46.841143 16388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1faa3c7bc3e4073bb65968a654e815d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.842299 16595 ts_tablet_manager.cc:1434] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.282s	user 0.017s	sys 0.004s
I20260709 12:21:46.843194 16538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0b76873ad2645ebb97d6f5b03655773. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.844480 16539 tablet_service.cc:1511] Processing CreateTablet for tablet b7b4960094cc44ceb337b594b1c269e6 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:21:46.843603 16595 tablet_bootstrap.cc:492] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:46.850458 16599 tablet_bootstrap.cc:654] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:46.854223 16539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7b4960094cc44ceb337b594b1c269e6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:21:46.858968 16354 tablet.cc:2406] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:21:46.860631 16389 tablet_service.cc:1511] Processing CreateTablet for tablet 14d8123994de43ec8e36a4e02961bc90 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:21:46.862056 16389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14d8123994de43ec8e36a4e02961bc90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.863818 16458 tablet_service.cc:1511] Processing CreateTablet for tablet 14d8123994de43ec8e36a4e02961bc90 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:21:46.866101 16458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14d8123994de43ec8e36a4e02961bc90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.868817 16540 tablet_service.cc:1511] Processing CreateTablet for tablet 4e0c5dbd539e4fa1a5f1e3fac0802b81 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:21:46.870146 16540 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e0c5dbd539e4fa1a5f1e3fac0802b81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.880395 16596 tablet_bootstrap.cc:492] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:46.881035 16596 ts_tablet_manager.cc:1403] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.217s	user 0.018s	sys 0.024s
I20260709 12:21:46.885550 16597 ts_tablet_manager.cc:1434] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.186s	user 0.011s	sys 0.018s
I20260709 12:21:46.891597 16384 tablet_service.cc:1511] Processing CreateTablet for tablet 4e0c5dbd539e4fa1a5f1e3fac0802b81 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:21:46.893348 16384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e0c5dbd539e4fa1a5f1e3fac0802b81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.899376 16597 tablet_bootstrap.cc:492] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:46.900475 16596 raft_consensus.cc:359] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:46.901408 16596 raft_consensus.cc:385] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:46.901762 16596 raft_consensus.cc:740] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:46.902606 16596 consensus_queue.cc:260] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:46.907848 16465 tablet_service.cc:1511] Processing CreateTablet for tablet 6d860a1605704f96b47bc65033809793 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:21:46.909241 16465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d860a1605704f96b47bc65033809793. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.919759 16596 ts_tablet_manager.cc:1434] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.038s	user 0.008s	sys 0.000s
I20260709 12:21:46.927043 16597 tablet_bootstrap.cc:654] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:46.934947 16533 tablet_service.cc:1511] Processing CreateTablet for tablet 6d860a1605704f96b47bc65033809793 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:21:46.937129 16533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d860a1605704f96b47bc65033809793. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.968261 16317 tablet_service.cc:1511] Processing CreateTablet for tablet a4bb7e1a96a94612b2f6759156f5f7c0 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:21:46.969820 16317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4bb7e1a96a94612b2f6759156f5f7c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.983954 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:21:46.985657 16514 leader_election.cc:343] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:46.986049 16313 tablet_service.cc:1511] Processing CreateTablet for tablet c0b76873ad2645ebb97d6f5b03655773 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:21:46.987401 16313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0b76873ad2645ebb97d6f5b03655773. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.994498 16603 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:46.991140 16389 tablet_service.cc:1511] Processing CreateTablet for tablet 07c17e01135a4bae850c5038c62d2ba2 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:21:47.001405 16603 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:47.001477 16310 tablet_service.cc:1511] Processing CreateTablet for tablet 6d860a1605704f96b47bc65033809793 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:21:47.002298 16389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07c17e01135a4bae850c5038c62d2ba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.003141 16310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d860a1605704f96b47bc65033809793. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.004241 16603 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:46.999874 16316 tablet_service.cc:1511] Processing CreateTablet for tablet 4e0c5dbd539e4fa1a5f1e3fac0802b81 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:21:47.000624 16596 tablet_bootstrap.cc:492] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:47.003153 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:46.992182 16463 tablet_service.cc:1511] Processing CreateTablet for tablet 07c17e01135a4bae850c5038c62d2ba2 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:21:47.008962 16463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07c17e01135a4bae850c5038c62d2ba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:46.998104 16311 tablet_service.cc:1511] Processing CreateTablet for tablet 14d8123994de43ec8e36a4e02961bc90 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:21:46.995832 16537 tablet_service.cc:1511] Processing CreateTablet for tablet 07c17e01135a4bae850c5038c62d2ba2 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:21:47.022593 16388 tablet_service.cc:1511] Processing CreateTablet for tablet 3851c76367ba4b3eaf3a1b5833ef4835 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:21:47.023295 16316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e0c5dbd539e4fa1a5f1e3fac0802b81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.024223 16388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3851c76367ba4b3eaf3a1b5833ef4835. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.031509 16595 tablet_bootstrap.cc:700] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2: Time spent opening tablet: real 0.138s	user 0.001s	sys 0.001s
I20260709 12:21:47.032684 16595 tablet_bootstrap.cc:654] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:47.037705 16311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14d8123994de43ec8e36a4e02961bc90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.039188 16465 tablet_service.cc:1511] Processing CreateTablet for tablet 3851c76367ba4b3eaf3a1b5833ef4835 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:21:47.045109 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:47.048724 16465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3851c76367ba4b3eaf3a1b5833ef4835. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.053515 16532 tablet_service.cc:1511] Processing CreateTablet for tablet 7117f6656fa14807ac801392259e843a (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:21:47.055886 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:21:47.057703 16291 leader_election.cc:343] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:47.070056 16599 tablet_bootstrap.cc:492] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:47.077303 16389 tablet_service.cc:1511] Processing CreateTablet for tablet 7117f6656fa14807ac801392259e843a (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
W20260709 12:21:47.079421 16293 leader_election.cc:343] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:47.080044 16293 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:47.080137 16309 tablet_service.cc:1511] Processing CreateTablet for tablet 7117f6656fa14807ac801392259e843a (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:21:47.081094 16603 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:47.081678 16309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7117f6656fa14807ac801392259e843a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:21:47.082588 16293 leader_election.cc:343] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:47.078833 16389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7117f6656fa14807ac801392259e843a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.086169 16532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7117f6656fa14807ac801392259e843a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.093007 16537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07c17e01135a4bae850c5038c62d2ba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.094494 16610 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:47.095140 16610 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:47.097039 16317 tablet_service.cc:1511] Processing CreateTablet for tablet 3851c76367ba4b3eaf3a1b5833ef4835 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:21:47.098553 16317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3851c76367ba4b3eaf3a1b5833ef4835. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.099888 16610 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:47.106374 16458 tablet_service.cc:1511] Processing CreateTablet for tablet 7889f3ab253b473eafb86002e9612a25 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:21:47.107836 16458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7889f3ab253b473eafb86002e9612a25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.110031 16596 tablet_bootstrap.cc:654] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:47.115866 16541 tablet_service.cc:1511] Processing CreateTablet for tablet 7889f3ab253b473eafb86002e9612a25 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:21:47.083298 16599 ts_tablet_manager.cc:1403] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.331s	user 0.028s	sys 0.016s
I20260709 12:21:47.129880 16541 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7889f3ab253b473eafb86002e9612a25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.132500 16599 raft_consensus.cc:359] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:47.133415 16599 raft_consensus.cc:385] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:47.133747 16599 raft_consensus.cc:740] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:47.134585 16599 consensus_queue.cc:260] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:47.135064 16388 tablet_service.cc:1511] Processing CreateTablet for tablet aea54881468a40aaabad162d385fc6d9 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:21:47.136576 16388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aea54881468a40aaabad162d385fc6d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.119329 16597 tablet_bootstrap.cc:492] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:47.143260 16597 ts_tablet_manager.cc:1403] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.257s	user 0.018s	sys 0.027s
I20260709 12:21:47.145265 16465 tablet_service.cc:1511] Processing CreateTablet for tablet aea54881468a40aaabad162d385fc6d9 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:21:47.146770 16465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aea54881468a40aaabad162d385fc6d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.147012 16597 raft_consensus.cc:359] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:47.147866 16597 raft_consensus.cc:385] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:47.124821 16293 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:47.148847 16603 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:21:47.149834 16293 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 67286us
I20260709 12:21:47.159840 16536 tablet_service.cc:1511] Processing CreateTablet for tablet aea54881468a40aaabad162d385fc6d9 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:21:47.161186 16536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aea54881468a40aaabad162d385fc6d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.160799 16317 tablet_service.cc:1511] Processing CreateTablet for tablet 7889f3ab253b473eafb86002e9612a25 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:21:47.162207 16317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7889f3ab253b473eafb86002e9612a25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.163338 16384 tablet_service.cc:1511] Processing CreateTablet for tablet c1a154d032654a179da41ad8f10fa339 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:21:47.164741 16384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1a154d032654a179da41ad8f10fa339. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.165820 16309 tablet_service.cc:1511] Processing CreateTablet for tablet c1a154d032654a179da41ad8f10fa339 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:21:47.166288 16458 tablet_service.cc:1511] Processing CreateTablet for tablet c1a154d032654a179da41ad8f10fa339 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:21:47.167223 16309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1a154d032654a179da41ad8f10fa339. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.167541 16458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1a154d032654a179da41ad8f10fa339. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.164520 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:47.169469 16534 tablet_service.cc:1511] Processing CreateTablet for tablet ef53202d665e46c39113572380be6c36 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:21:47.170727 16534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef53202d665e46c39113572380be6c36. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:21:47.171664 16273 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 12:21:47.177018 16515 leader_election.cc:343] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:47.177590 16515 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:21:47.178501 16599 ts_tablet_manager.cc:1434] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.049s	user 0.006s	sys 0.000s
I20260709 12:21:47.178248 16313 tablet_service.cc:1511] Processing CreateTablet for tablet ef53202d665e46c39113572380be6c36 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:21:47.179059 16604 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:47.179649 16599 tablet_bootstrap.cc:492] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:47.179872 16313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef53202d665e46c39113572380be6c36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.187870 16595 tablet_bootstrap.cc:492] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:47.188517 16595 ts_tablet_manager.cc:1403] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.345s	user 0.012s	sys 0.007s
I20260709 12:21:47.148237 16597 raft_consensus.cc:740] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:47.192318 16595 raft_consensus.cc:359] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:47.193162 16595 raft_consensus.cc:385] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:47.193501 16595 raft_consensus.cc:740] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:47.201081 16599 tablet_bootstrap.cc:654] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:47.201462 16597 consensus_queue.cc:260] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:47.204080 16597 ts_tablet_manager.cc:1434] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.060s	user 0.006s	sys 0.000s
I20260709 12:21:47.208154 16596 tablet_bootstrap.cc:492] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:47.208863 16596 ts_tablet_manager.cc:1403] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.243s	user 0.016s	sys 0.002s
I20260709 12:21:47.210968 16392 tablet_service.cc:1511] Processing CreateTablet for tablet ef53202d665e46c39113572380be6c36 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:21:47.214186 16392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef53202d665e46c39113572380be6c36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.214587 16596 raft_consensus.cc:359] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:47.215409 16596 raft_consensus.cc:385] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:47.215760 16596 raft_consensus.cc:740] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:47.221375 16458 tablet_service.cc:1511] Processing CreateTablet for tablet 8c02e60dcbc14b2fa3a2c11a214a93a6 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:21:47.222702 16458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c02e60dcbc14b2fa3a2c11a214a93a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.226382 16597 tablet_bootstrap.cc:492] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:47.312325 16532 tablet_service.cc:1511] Processing CreateTablet for tablet 8c02e60dcbc14b2fa3a2c11a214a93a6 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:21:47.275818 16595 consensus_queue.cc:260] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:47.310454 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:47.320550 16313 tablet_service.cc:1511] Processing CreateTablet for tablet 8c02e60dcbc14b2fa3a2c11a214a93a6 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:21:47.322086 16313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c02e60dcbc14b2fa3a2c11a214a93a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.322786 16388 tablet_service.cc:1511] Processing CreateTablet for tablet b6d1babdc37d40e4a71ffde9b310feaa (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
W20260709 12:21:47.324038 16366 leader_election.cc:343] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:47.322660 16596 consensus_queue.cc:260] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:47.324270 16388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6d1babdc37d40e4a71ffde9b310feaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.327179 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:47.328064 16599 tablet_bootstrap.cc:492] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:47.327754 16458 tablet_service.cc:1511] Processing CreateTablet for tablet b6d1babdc37d40e4a71ffde9b310feaa (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:21:47.328271 16532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c02e60dcbc14b2fa3a2c11a214a93a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.331583 16537 tablet_service.cc:1511] Processing CreateTablet for tablet b6d1babdc37d40e4a71ffde9b310feaa (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
W20260709 12:21:47.332659 16367 leader_election.cc:343] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:47.345577 16367 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:21:47.346912 16610 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:47.348272 16537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6d1babdc37d40e4a71ffde9b310feaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.348483 16604 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:47.349048 16604 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:47.334228 16599 ts_tablet_manager.cc:1403] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.155s	user 0.011s	sys 0.008s
I20260709 12:21:47.355484 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:47.337393 16311 tablet_service.cc:1511] Processing CreateTablet for tablet defc374629a649a8a47502f058be2c8c (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:21:47.240677 16608 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:47.357266 16311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet defc374629a649a8a47502f058be2c8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.356942 16608 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:21:47.361837 16515 leader_election.cc:343] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:47.362253 16608 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:47.355155 16604 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:47.370671 16597 tablet_bootstrap.cc:654] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:47.355818 16599 raft_consensus.cc:359] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:47.373050 16599 raft_consensus.cc:385] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:47.373536 16599 raft_consensus.cc:740] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:47.381642 16458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6d1babdc37d40e4a71ffde9b310feaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.382311 16599 consensus_queue.cc:260] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:47.394088 16603 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:47.394667 16603 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:47.396461 16465 tablet_service.cc:1511] Processing CreateTablet for tablet defc374629a649a8a47502f058be2c8c (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:21:47.397010 16603 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:47.397987 16465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet defc374629a649a8a47502f058be2c8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.398294 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:47.393525 16388 tablet_service.cc:1511] Processing CreateTablet for tablet defc374629a649a8a47502f058be2c8c (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:21:47.399021 16551 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 0.
I20260709 12:21:47.400068 16388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet defc374629a649a8a47502f058be2c8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.401729 16292 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:21:47.397029 16599 ts_tablet_manager.cc:1434] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.044s	user 0.009s	sys 0.001s
I20260709 12:21:47.402818 16603 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:47.403257 16603 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:47.403640 16603 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:47.415123 16596 ts_tablet_manager.cc:1434] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.206s	user 0.008s	sys 0.000s
I20260709 12:21:47.417963 16599 tablet_bootstrap.cc:492] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:47.423581 16539 tablet_service.cc:1511] Processing CreateTablet for tablet 701eedfc29544bbeafd695f04ccea545 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:21:47.424989 16539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 701eedfc29544bbeafd695f04ccea545. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.425190 16596 tablet_bootstrap.cc:492] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:47.426721 16595 ts_tablet_manager.cc:1434] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.238s	user 0.027s	sys 0.023s
I20260709 12:21:47.428719 16603 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:47.429832 16595 tablet_bootstrap.cc:492] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:47.431975 16313 tablet_service.cc:1511] Processing CreateTablet for tablet 701eedfc29544bbeafd695f04ccea545 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:21:47.433326 16313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 701eedfc29544bbeafd695f04ccea545. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.434657 16604 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:47.435153 16604 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:47.429456 16392 tablet_service.cc:1511] Processing CreateTablet for tablet 701eedfc29544bbeafd695f04ccea545 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:21:47.437754 16392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 701eedfc29544bbeafd695f04ccea545. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.454084 16604 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:47.455192 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:47.455360 16603 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:47.457055 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:47.458985 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:47.459599 16551 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:47.464672 16514 leader_election.cc:343] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:47.465546 16596 tablet_bootstrap.cc:654] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:47.465981 16515 leader_election.cc:343] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:47.466452 16515 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:21:47.466660 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:21:47.467227 16465 tablet_service.cc:1511] Processing CreateTablet for tablet de44585291c44bfe9a6f7168d04169b8 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:21:47.467907 16622 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:47.468797 16622 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:47.473289 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:21:47.474252 16291 leader_election.cc:343] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:47.474380 16604 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:21:47.480247 16291 leader_election.cc:343] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:47.477404 16595 tablet_bootstrap.cc:654] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:47.482656 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:21:47.484519 16516 leader_election.cc:343] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:47.485085 16516 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:47.486402 16604 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:47.487396 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:47.488065 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:47.488150 16317 tablet_service.cc:1511] Processing CreateTablet for tablet de44585291c44bfe9a6f7168d04169b8 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:21:47.489408 16597 tablet_bootstrap.cc:492] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:47.490365 16317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de44585291c44bfe9a6f7168d04169b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.490223 16597 ts_tablet_manager.cc:1403] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.264s	user 0.005s	sys 0.022s
I20260709 12:21:47.494256 16597 raft_consensus.cc:359] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:47.494953 16597 raft_consensus.cc:385] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:47.495258 16597 raft_consensus.cc:740] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
W20260709 12:21:47.490188 16442 leader_election.cc:343] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:47.496505 16442 leader_election.cc:343] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:47.497296 16597 consensus_queue.cc:260] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:47.495843 16551 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 1.
I20260709 12:21:47.501504 16465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de44585291c44bfe9a6f7168d04169b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.474589 16622 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:47.508697 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:47.510743 16458 tablet_service.cc:1511] Processing CreateTablet for tablet 4ba50aa1e4bd43928d6d7e4a7d6a60d1 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:21:47.511942 16458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ba50aa1e4bd43928d6d7e4a7d6a60d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.521569 16532 tablet_service.cc:1511] Processing CreateTablet for tablet de44585291c44bfe9a6f7168d04169b8 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:21:47.523559 16532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de44585291c44bfe9a6f7168d04169b8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:21:47.524112 16441 leader_election.cc:343] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:47.510465 16392 tablet_service.cc:1511] Processing CreateTablet for tablet 4ba50aa1e4bd43928d6d7e4a7d6a60d1 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:21:47.524691 16441 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:47.525703 16392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ba50aa1e4bd43928d6d7e4a7d6a60d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.526757 16292 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:21:47.527915 16603 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:47.525897 16622 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:47.545166 16541 tablet_service.cc:1511] Processing CreateTablet for tablet 4ba50aa1e4bd43928d6d7e4a7d6a60d1 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:21:47.550992 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:21:47.552430 16441 leader_election.cc:343] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:47.552973 16441 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:47.554136 16622 raft_consensus.cc:2749] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:47.557106 16599 tablet_bootstrap.cc:654] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:47.560813 16597 ts_tablet_manager.cc:1434] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.070s	user 0.005s	sys 0.006s
I20260709 12:21:47.564401 16541 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ba50aa1e4bd43928d6d7e4a7d6a60d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.561600 16603 raft_consensus.cc:697] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 1 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:21:47.566735 16603 consensus_queue.cc:237] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:47.567350 16388 tablet_service.cc:1511] Processing CreateTablet for tablet 05c8566780d1400ab2e42d9529d97ceb (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:21:47.568145 16596 tablet_bootstrap.cc:492] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:47.568711 16596 ts_tablet_manager.cc:1403] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.144s	user 0.007s	sys 0.012s
I20260709 12:21:47.568764 16388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05c8566780d1400ab2e42d9529d97ceb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.572909 16597 tablet_bootstrap.cc:492] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:47.576587 16610 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:47.577193 16610 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:47.579807 16610 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:47.581177 16597 tablet_bootstrap.cc:654] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:47.583628 16596 raft_consensus.cc:359] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:47.584242 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:47.584558 16596 raft_consensus.cc:385] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:47.584911 16596 raft_consensus.cc:740] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:47.585865 16596 consensus_queue.cc:260] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:47.588685 16596 ts_tablet_manager.cc:1434] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 12:21:47.589159 16313 tablet_service.cc:1511] Processing CreateTablet for tablet 05c8566780d1400ab2e42d9529d97ceb (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:21:47.589818 16596 tablet_bootstrap.cc:492] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:47.590616 16313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05c8566780d1400ab2e42d9529d97ceb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.597040 16596 tablet_bootstrap.cc:654] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:47.601001 16367 leader_election.cc:343] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:47.609469 16465 tablet_service.cc:1511] Processing CreateTablet for tablet 05c8566780d1400ab2e42d9529d97ceb (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:21:47.611244 16465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05c8566780d1400ab2e42d9529d97ceb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.621068 16532 tablet_service.cc:1511] Processing CreateTablet for tablet 6474b10917204b6994269be17387f38d (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:21:47.628373 16532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6474b10917204b6994269be17387f38d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.649549 16597 tablet_bootstrap.cc:492] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:47.650314 16597 ts_tablet_manager.cc:1403] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.078s	user 0.018s	sys 0.011s
I20260709 12:21:47.649705 16224 catalog_manager.cc:5697] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:47.653513 16597 raft_consensus.cc:359] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:47.655160 16311 tablet_service.cc:1511] Processing CreateTablet for tablet 6474b10917204b6994269be17387f38d (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:21:47.661223 16311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6474b10917204b6994269be17387f38d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.656358 16388 tablet_service.cc:1511] Processing CreateTablet for tablet 6474b10917204b6994269be17387f38d (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:21:47.662178 16388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6474b10917204b6994269be17387f38d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.649636 16595 tablet_bootstrap.cc:492] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:47.662845 16595 ts_tablet_manager.cc:1403] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.233s	user 0.020s	sys 0.063s
I20260709 12:21:47.663842 16541 tablet_service.cc:1511] Processing CreateTablet for tablet ad2476e68a0d4ae695099a41865c99c3 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:21:47.665100 16541 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad2476e68a0d4ae695099a41865c99c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.666566 16595 raft_consensus.cc:359] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:47.667377 16595 raft_consensus.cc:385] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:47.667685 16595 raft_consensus.cc:740] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:47.662215 16458 tablet_service.cc:1511] Processing CreateTablet for tablet ad2476e68a0d4ae695099a41865c99c3 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:21:47.669675 16458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad2476e68a0d4ae695099a41865c99c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.657410 16599 tablet_bootstrap.cc:492] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:47.658174 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:47.659997 16596 tablet_bootstrap.cc:492] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:47.673647 16328 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 0.
I20260709 12:21:47.673920 16596 ts_tablet_manager.cc:1403] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.084s	user 0.013s	sys 0.002s
I20260709 12:21:47.674181 16392 tablet_service.cc:1511] Processing CreateTablet for tablet 7ca74e9d2a1f4ef6bc63f8a1deeb05fa (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:21:47.675546 16392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ca74e9d2a1f4ef6bc63f8a1deeb05fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.675268 16367 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:21:47.684111 16599 ts_tablet_manager.cc:1403] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.266s	user 0.009s	sys 0.009s
I20260709 12:21:47.684944 16463 tablet_service.cc:1511] Processing CreateTablet for tablet 7ca74e9d2a1f4ef6bc63f8a1deeb05fa (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:21:47.685705 16597 raft_consensus.cc:385] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:47.686343 16610 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:47.692283 16597 raft_consensus.cc:740] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:47.686277 16463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ca74e9d2a1f4ef6bc63f8a1deeb05fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.678622 16595 consensus_queue.cc:260] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:47.693176 16610 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:47.695529 16610 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:47.695883 16595 ts_tablet_manager.cc:1434] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.033s	user 0.004s	sys 0.004s
I20260709 12:21:47.678850 16313 tablet_service.cc:1511] Processing CreateTablet for tablet ad2476e68a0d4ae695099a41865c99c3 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:21:47.698326 16313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad2476e68a0d4ae695099a41865c99c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.678140 16596 raft_consensus.cc:359] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:47.699824 16596 raft_consensus.cc:385] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:47.700166 16596 raft_consensus.cc:740] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:47.702000 16595 tablet_bootstrap.cc:492] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:47.704584 16596 consensus_queue.cc:260] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:47.705242 16603 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:47.705742 16603 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:47.705744 16597 consensus_queue.cc:260] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:47.712399 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:47.713056 16477 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 0.
I20260709 12:21:47.714776 16317 tablet_service.cc:1511] Processing CreateTablet for tablet 7ff57aebc0fb4c6dbb6077f83ac8a60e (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:21:47.716073 16317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ff57aebc0fb4c6dbb6077f83ac8a60e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.717072 16459 tablet_service.cc:1511] Processing CreateTablet for tablet 7ff57aebc0fb4c6dbb6077f83ac8a60e (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:21:47.718762 16291 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:21:47.707329 16596 ts_tablet_manager.cc:1434] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.033s	user 0.009s	sys 0.003s
I20260709 12:21:47.707896 16603 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:47.709735 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:47.727236 16596 tablet_bootstrap.cc:492] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:47.727015 16599 raft_consensus.cc:359] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:47.727826 16599 raft_consensus.cc:385] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:47.711472 16539 tablet_service.cc:1511] Processing CreateTablet for tablet 7ca74e9d2a1f4ef6bc63f8a1deeb05fa (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:21:47.729347 16539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ca74e9d2a1f4ef6bc63f8a1deeb05fa. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:21:47.727671 16293 leader_election.cc:343] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:47.731351 16637 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:47.731890 16637 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:47.733767 16595 tablet_bootstrap.cc:654] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:47.734139 16637 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:47.734898 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:47.735291 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:21:47.736651 16366 leader_election.cc:343] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:47.739482 16537 tablet_service.cc:1511] Processing CreateTablet for tablet 14f9bc0dbb7e4c6f94aaf8146768ed79 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:21:47.721881 16389 tablet_service.cc:1511] Processing CreateTablet for tablet 7ff57aebc0fb4c6dbb6077f83ac8a60e (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:21:47.741046 16537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14f9bc0dbb7e4c6f94aaf8146768ed79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.720784 16459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ff57aebc0fb4c6dbb6077f83ac8a60e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:21:47.743189 16367 leader_election.cc:343] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:47.723675 16622 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:47.743698 16367 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:21:47.743863 16622 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:47.746099 16622 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:47.748000 16636 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:47.748193 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:47.749912 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:47.750533 16551 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 0.
I20260709 12:21:47.728143 16599 raft_consensus.cc:740] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:47.753119 16599 consensus_queue.cc:260] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:47.754462 16597 ts_tablet_manager.cc:1434] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.086s	user 0.005s	sys 0.005s
I20260709 12:21:47.755826 16597 tablet_bootstrap.cc:492] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:47.755908 16441 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
W20260709 12:21:47.756955 16441 leader_election.cc:343] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:47.719756 16630 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:47.757627 16461 tablet_service.cc:1511] Processing CreateTablet for tablet 2546c2ffdb52493c972a8bf000c78ed5 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:21:47.757982 16630 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:47.758452 16630 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:47.758972 16461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2546c2ffdb52493c972a8bf000c78ed5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.760241 16384 tablet_service.cc:1511] Processing CreateTablet for tablet 14f9bc0dbb7e4c6f94aaf8146768ed79 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:21:47.760608 16622 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:47.761188 16622 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:47.761624 16622 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:47.748208 16610 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:47.741174 16389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ff57aebc0fb4c6dbb6077f83ac8a60e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.773214 16384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14f9bc0dbb7e4c6f94aaf8146768ed79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.785169 16541 tablet_service.cc:1511] Processing CreateTablet for tablet 2546c2ffdb52493c972a8bf000c78ed5 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:21:47.786525 16541 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2546c2ffdb52493c972a8bf000c78ed5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.786818 16597 tablet_bootstrap.cc:654] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:47.793087 16311 tablet_service.cc:1511] Processing CreateTablet for tablet 14f9bc0dbb7e4c6f94aaf8146768ed79 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:21:47.804973 16622 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:47.799868 16630 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:47.813303 16599 ts_tablet_manager.cc:1434] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.129s	user 0.007s	sys 0.000s
I20260709 12:21:47.813180 16610 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:47.815555 16313 tablet_service.cc:1511] Processing CreateTablet for tablet 2546c2ffdb52493c972a8bf000c78ed5 (DEFAULT_TABLE table=test-workload [id=d580354349594467bc7361af3f4401db]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:21:47.816241 16311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14f9bc0dbb7e4c6f94aaf8146768ed79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.816919 16313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2546c2ffdb52493c972a8bf000c78ed5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:47.817621 16622 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:47.819816 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:47.820487 16551 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:47.821204 16599 tablet_bootstrap.cc:492] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:47.823310 16630 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:47.828035 16327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:47.828938 16596 tablet_bootstrap.cc:654] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:47.829351 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:47.830509 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:21:47.831096 16477 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:47.824968 16595 tablet_bootstrap.cc:492] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:47.832870 16595 ts_tablet_manager.cc:1403] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.131s	user 0.012s	sys 0.005s
I20260709 12:21:47.836910 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:47.837652 16550 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:47.838858 16551 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:21:47.839442 16599 tablet_bootstrap.cc:654] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:47.845041 16293 leader_election.cc:343] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:47.824697 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:47.848413 16328 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:47.851866 16367 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:47.852299 16597 tablet_bootstrap.cc:492] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:47.852838 16610 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:47.852965 16597 ts_tablet_manager.cc:1403] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.097s	user 0.009s	sys 0.009s
I20260709 12:21:47.853416 16610 raft_consensus.cc:697] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:21:47.854822 16610 consensus_queue.cc:237] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:47.836030 16595 raft_consensus.cc:359] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:21:47.846330 16441 leader_election.cc:343] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:47.857520 16597 raft_consensus.cc:359] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:47.858286 16597 raft_consensus.cc:385] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:47.858634 16597 raft_consensus.cc:740] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:47.859520 16597 consensus_queue.cc:260] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:47.856276 16595 raft_consensus.cc:385] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:47.860834 16595 raft_consensus.cc:740] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:47.862104 16597 ts_tablet_manager.cc:1434] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:21:47.861796 16595 consensus_queue.cc:260] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:47.863112 16477 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 1.
I20260709 12:21:47.864554 16291 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:47.865384 16630 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:47.865770 16550 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:21:47.865849 16630 raft_consensus.cc:697] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 1 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:21:47.856040 16328 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:21:47.863314 16597 tablet_bootstrap.cc:492] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:47.866904 16630 consensus_queue.cc:237] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:47.870328 16441 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:21:47.870864 16595 ts_tablet_manager.cc:1434] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.000s
I20260709 12:21:47.871528 16622 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:47.880703 16595 tablet_bootstrap.cc:492] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:47.909761 16597 tablet_bootstrap.cc:654] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:47.913625 16622 raft_consensus.cc:697] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:21:47.915606 16622 consensus_queue.cc:237] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:47.920830 16599 tablet_bootstrap.cc:492] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:47.921604 16599 ts_tablet_manager.cc:1403] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.101s	user 0.022s	sys 0.022s
I20260709 12:21:47.923130 16595 tablet_bootstrap.cc:654] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:47.921664 16596 tablet_bootstrap.cc:492] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:47.924721 16596 ts_tablet_manager.cc:1403] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.198s	user 0.012s	sys 0.009s
I20260709 12:21:47.925513 16599 raft_consensus.cc:359] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:47.926373 16599 raft_consensus.cc:385] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:47.926735 16599 raft_consensus.cc:740] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:47.927629 16599 consensus_queue.cc:260] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:47.931450 16223 catalog_manager.cc:5697] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 0 to 1, leader changed from <none> to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 1 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:47.936908 16225 catalog_manager.cc:5697] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:47.940295 16599 ts_tablet_manager.cc:1434] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 12:21:47.942802 16599 tablet_bootstrap.cc:492] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:47.944240 16224 catalog_manager.cc:5697] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:47.928153 16596 raft_consensus.cc:359] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:47.948661 16596 raft_consensus.cc:385] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:47.949051 16596 raft_consensus.cc:740] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:47.950318 16599 tablet_bootstrap.cc:654] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:47.949951 16596 consensus_queue.cc:260] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:47.952986 16597 tablet_bootstrap.cc:492] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:47.956068 16595 tablet_bootstrap.cc:492] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:47.956672 16595 ts_tablet_manager.cc:1403] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.076s	user 0.010s	sys 0.004s
I20260709 12:21:47.961282 16630 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:47.961889 16595 raft_consensus.cc:359] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:47.961959 16630 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:47.963236 16597 ts_tablet_manager.cc:1403] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.100s	user 0.013s	sys 0.003s
I20260709 12:21:47.965003 16630 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:47.962656 16595 raft_consensus.cc:385] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:47.965838 16595 raft_consensus.cc:740] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:47.966213 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:47.966568 16597 raft_consensus.cc:359] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:47.967300 16597 raft_consensus.cc:385] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:47.966815 16595 consensus_queue.cc:260] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:47.967621 16597 raft_consensus.cc:740] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
W20260709 12:21:47.967912 16293 leader_election.cc:343] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:47.968597 16597 consensus_queue.cc:260] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:47.969159 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:47.969901 16550 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 0.
I20260709 12:21:47.971379 16292 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:47.972293 16630 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:47.972749 16630 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:47.973003 16610 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:47.973192 16630 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:47.973807 16636 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:47.974375 16636 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:47.973534 16610 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:47.976538 16595 ts_tablet_manager.cc:1434] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.002s
I20260709 12:21:47.976948 16636 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:47.977761 16595 tablet_bootstrap.cc:492] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:47.978041 16599 tablet_bootstrap.cc:492] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:47.978597 16599 ts_tablet_manager.cc:1403] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.036s	user 0.002s	sys 0.008s
I20260709 12:21:47.978727 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:47.979481 16477 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 0.
I20260709 12:21:47.980887 16610 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:47.980978 16366 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:47.982579 16630 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:47.984642 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:47.984488 16599 raft_consensus.cc:359] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:47.985381 16599 raft_consensus.cc:385] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:47.985644 16630 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:21:47.986107 16367 leader_election.cc:343] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:47.986433 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:47.985728 16595 tablet_bootstrap.cc:654] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:47.988001 16622 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:47.988571 16622 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:47.985764 16599 raft_consensus.cc:740] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:47.990222 16599 consensus_queue.cc:260] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:47.991143 16622 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:47.992888 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:47.993625 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:47.994331 16550 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 0.
I20260709 12:21:47.996227 16595 tablet_bootstrap.cc:492] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:47.996256 16599 ts_tablet_manager.cc:1434] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.002s
W20260709 12:21:47.996697 16366 leader_election.cc:343] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:47.996999 16595 ts_tablet_manager.cc:1403] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.003s
I20260709 12:21:47.998013 16599 tablet_bootstrap.cc:492] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:47.998071 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:21:47.999696 16293 leader_election.cc:343] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:48.000648 16595 raft_consensus.cc:359] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:21:48.001030 16441 leader_election.cc:343] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:48.001482 16595 raft_consensus.cc:385] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:48.001863 16595 raft_consensus.cc:740] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:48.002048 16441 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:21:48.003050 16622 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:48.002957 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:48.003585 16622 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:48.003937 16328 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 0.
I20260709 12:21:48.004037 16622 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:48.004552 16595 consensus_queue.cc:260] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:48.005757 16630 consensus_queue.cc:1048] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:48.006150 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:48.006886 16550 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:48.007220 16367 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:21:48.007597 16595 ts_tablet_manager.cc:1434] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:21:48.008679 16636 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:48.009995 16599 tablet_bootstrap.cc:654] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:48.011425 16636 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:48.011914 16636 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:48.012058 16595 tablet_bootstrap.cc:492] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:48.016574 16610 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:48.017114 16610 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:48.017742 16610 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:48.035786 16636 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:48.037762 16596 ts_tablet_manager.cc:1434] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.113s	user 0.008s	sys 0.000s
I20260709 12:21:48.040122 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:21:48.041141 16477 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:48.040925 16622 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:48.042349 16595 tablet_bootstrap.cc:654] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:48.043572 16622 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:48.040199 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:48.045353 16636 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:21:48.047631 16367 leader_election.cc:343] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:48.049266 16291 consensus_peers.cc:597] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 -> Peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Couldn't send request to peer 52ab50b0237346c1a13ede947f6a63c2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:48.050109 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:48.050969 16549 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:48.053516 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:48.056526 16595 tablet_bootstrap.cc:492] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:48.057346 16595 ts_tablet_manager.cc:1403] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.046s	user 0.008s	sys 0.007s
I20260709 12:21:48.058557 16477 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
W20260709 12:21:48.061276 16441 leader_election.cc:343] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:48.061002 16595 raft_consensus.cc:359] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:48.062182 16595 raft_consensus.cc:385] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:48.062325 16366 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:21:48.062702 16595 raft_consensus.cc:740] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:48.063823 16636 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:48.063802 16595 consensus_queue.cc:260] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:47.990824 16597 ts_tablet_manager.cc:1434] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.027s	user 0.004s	sys 0.003s
I20260709 12:21:48.065817 16636 raft_consensus.cc:697] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:21:48.064297 16610 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:48.067022 16597 tablet_bootstrap.cc:492] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:48.066854 16636 consensus_queue.cc:237] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:48.071130 16610 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:48.068886 16599 tablet_bootstrap.cc:492] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:48.071935 16599 ts_tablet_manager.cc:1403] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.074s	user 0.007s	sys 0.043s
I20260709 12:21:48.072105 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:48.073438 16328 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:48.075551 16599 raft_consensus.cc:359] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:48.076985 16599 raft_consensus.cc:385] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:48.077589 16599 raft_consensus.cc:740] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:48.078670 16599 consensus_queue.cc:260] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:48.069118 16595 ts_tablet_manager.cc:1434] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 12:21:48.080819 16595 tablet_bootstrap.cc:492] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:48.081581 16599 ts_tablet_manager.cc:1434] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.005s
I20260709 12:21:48.082818 16599 tablet_bootstrap.cc:492] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:48.089982 16595 tablet_bootstrap.cc:654] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:48.090107 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:21:48.091199 16599 tablet_bootstrap.cc:654] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:48.092061 16366 leader_election.cc:343] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:48.079384 16597 tablet_bootstrap.cc:654] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:48.094990 16596 tablet_bootstrap.cc:492] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:48.103157 16549 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:21:48.106547 16441 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:21:48.106956 16597 tablet_bootstrap.cc:492] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:48.107599 16597 ts_tablet_manager.cc:1403] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.041s	user 0.015s	sys 0.007s
I20260709 12:21:48.107815 16622 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:48.108595 16622 raft_consensus.cc:697] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:21:48.109366 16599 tablet_bootstrap.cc:492] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:48.109861 16599 ts_tablet_manager.cc:1403] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.009s
I20260709 12:21:48.110249 16622 consensus_queue.cc:237] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:48.111213 16597 raft_consensus.cc:359] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:48.112205 16597 raft_consensus.cc:385] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:48.112617 16597 raft_consensus.cc:740] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:48.113583 16597 consensus_queue.cc:260] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:48.123837 16597 ts_tablet_manager.cc:1434] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.003s
I20260709 12:21:48.114523 16550 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 1.
I20260709 12:21:48.125751 16292 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:48.127633 16630 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:48.128119 16630 raft_consensus.cc:697] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 1 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:21:48.128113 16599 raft_consensus.cc:359] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:48.139494 16599 raft_consensus.cc:385] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:48.139830 16599 raft_consensus.cc:740] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:48.139909 16630 consensus_queue.cc:237] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:48.132432 16597 tablet_bootstrap.cc:492] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:48.099068 16328 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:21:48.143810 16595 tablet_bootstrap.cc:492] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:48.144460 16595 ts_tablet_manager.cc:1403] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.064s	user 0.027s	sys 0.011s
I20260709 12:21:48.144845 16367 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:21:48.144110 16225 catalog_manager.cc:5697] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 0 to 1, leader changed from <none> to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 1 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:48.146112 16636 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:48.146688 16636 raft_consensus.cc:697] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:21:48.147660 16636 consensus_queue.cc:237] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:48.148066 16595 raft_consensus.cc:359] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:48.148869 16595 raft_consensus.cc:385] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:48.148614 16599 consensus_queue.cc:260] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:48.149176 16595 raft_consensus.cc:740] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:48.150398 16595 consensus_queue.cc:260] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:48.151036 16599 ts_tablet_manager.cc:1434] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.000s
I20260709 12:21:48.152100 16599 tablet_bootstrap.cc:492] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:48.152503 16622 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:48.153046 16622 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:48.151607 16224 catalog_manager.cc:5697] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:48.138540 16596 tablet_bootstrap.cc:654] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:48.155369 16622 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:48.160831 16599 tablet_bootstrap.cc:654] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:48.160220 16595 ts_tablet_manager.cc:1434] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 12:21:48.166494 16595 tablet_bootstrap.cc:492] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:48.159480 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:21:48.170322 16441 leader_election.cc:343] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:48.156473 16597 tablet_bootstrap.cc:654] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:48.182438 16595 tablet_bootstrap.cc:654] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:48.183107 16273 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:48.157724 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:48.183979 16402 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 0.
I20260709 12:21:48.184085 16223 catalog_manager.cc:5697] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 0 to 1, leader changed from <none> to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 1 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:48.185308 16442 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:21:48.186887 16622 raft_consensus.cc:2804] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:48.187477 16622 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:48.187958 16622 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:48.195087 16597 tablet_bootstrap.cc:492] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:48.195694 16597 ts_tablet_manager.cc:1403] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.064s	user 0.016s	sys 0.009s
I20260709 12:21:48.196026 16596 tablet_bootstrap.cc:492] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:48.196650 16596 ts_tablet_manager.cc:1403] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.102s	user 0.010s	sys 0.008s
I20260709 12:21:48.199293 16597 raft_consensus.cc:359] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:48.200145 16597 raft_consensus.cc:385] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:48.200482 16597 raft_consensus.cc:740] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:48.198280 16622 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:48.201618 16597 consensus_queue.cc:260] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:48.202984 16596 raft_consensus.cc:359] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:48.203912 16596 raft_consensus.cc:385] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:48.204957 16596 raft_consensus.cc:740] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:48.205228 16597 ts_tablet_manager.cc:1434] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.004s
I20260709 12:21:48.206254 16597 tablet_bootstrap.cc:492] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:48.205912 16596 consensus_queue.cc:260] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:48.207736 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:48.208367 16402 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:48.208892 16599 tablet_bootstrap.cc:492] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:48.209506 16596 ts_tablet_manager.cc:1434] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:21:48.210940 16596 tablet_bootstrap.cc:492] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:48.211972 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:21:48.213760 16441 leader_election.cc:343] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:48.209561 16599 ts_tablet_manager.cc:1403] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.058s	user 0.012s	sys 0.019s
I20260709 12:21:48.217726 16599 raft_consensus.cc:359] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:48.218261 16596 tablet_bootstrap.cc:654] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:48.218415 16599 raft_consensus.cc:385] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:48.218868 16599 raft_consensus.cc:740] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:48.219395 16597 tablet_bootstrap.cc:654] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:48.219543 16622 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:48.220283 16599 consensus_queue.cc:260] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:48.229310 16225 catalog_manager.cc:5697] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:48.236716 16597 tablet_bootstrap.cc:492] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:48.237303 16597 ts_tablet_manager.cc:1403] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.031s	user 0.010s	sys 0.004s
I20260709 12:21:48.237547 16622 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:48.238065 16622 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:48.240227 16597 raft_consensus.cc:359] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:48.240731 16622 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:48.241026 16597 raft_consensus.cc:385] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:48.241338 16597 raft_consensus.cc:740] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:48.241180 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:48.242578 16599 ts_tablet_manager.cc:1434] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.000s
I20260709 12:21:48.242218 16597 consensus_queue.cc:260] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:21:48.243671 16442 leader_election.cc:343] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:48.245781 16595 tablet_bootstrap.cc:492] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:48.246274 16595 ts_tablet_manager.cc:1403] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.080s	user 0.015s	sys 0.016s
I20260709 12:21:48.248428 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:48.249203 16595 raft_consensus.cc:359] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:48.249861 16595 raft_consensus.cc:385] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:48.249842 16604 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:48.250499 16604 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:21:48.251296 16441 leader_election.cc:343] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:48.251780 16441 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:48.252576 16604 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:48.252931 16622 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:48.253065 16595 raft_consensus.cc:740] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:48.253758 16595 consensus_queue.cc:260] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:48.254382 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:48.254838 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:48.254848 16599 tablet_bootstrap.cc:492] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:48.255513 16403 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 0.
I20260709 12:21:48.255766 16595 ts_tablet_manager.cc:1434] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
W20260709 12:21:48.255890 16514 leader_election.cc:343] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:48.256794 16595 tablet_bootstrap.cc:492] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:48.257174 16516 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:21:48.258309 16604 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:48.258710 16604 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:48.259054 16604 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:48.261545 16599 tablet_bootstrap.cc:654] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:48.262046 16597 ts_tablet_manager.cc:1434] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 12:21:48.263079 16597 tablet_bootstrap.cc:492] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:48.276787 16402 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:21:48.277576 16595 tablet_bootstrap.cc:654] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:48.282225 16599 tablet_bootstrap.cc:492] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:48.285297 16599 ts_tablet_manager.cc:1403] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.031s	user 0.013s	sys 0.004s
I20260709 12:21:48.285969 16442 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:21:48.287148 16622 raft_consensus.cc:2804] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:48.287647 16622 raft_consensus.cc:697] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:21:48.288516 16622 consensus_queue.cc:237] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:48.282277 16597 tablet_bootstrap.cc:654] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:48.282609 16604 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:48.291162 16604 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:48.288730 16599 raft_consensus.cc:359] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:48.291872 16599 raft_consensus.cc:385] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:48.292380 16599 raft_consensus.cc:740] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:48.292397 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:21:48.294569 16599 consensus_queue.cc:260] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:48.295046 16644 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:48.296186 16644 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:48.297276 16599 ts_tablet_manager.cc:1434] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
W20260709 12:21:48.295114 16514 leader_election.cc:343] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:48.298682 16599 tablet_bootstrap.cc:492] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:48.299437 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:48.300004 16644 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:48.300240 16328 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 0.
I20260709 12:21:48.302006 16441 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:21:48.294188 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:48.303332 16402 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:48.304765 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:48.305558 16403 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 0.
W20260709 12:21:48.305912 16441 consensus_peers.cc:597] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:48.306567 16644 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:48.307014 16644 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:48.307389 16599 tablet_bootstrap.cc:654] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:48.310122 16597 tablet_bootstrap.cc:492] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:48.307394 16644 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:48.310451 16596 tablet_bootstrap.cc:492] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:48.310657 16597 ts_tablet_manager.cc:1403] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.048s	user 0.013s	sys 0.001s
I20260709 12:21:48.311084 16596 ts_tablet_manager.cc:1403] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.100s	user 0.037s	sys 0.032s
I20260709 12:21:48.315758 16597 raft_consensus.cc:359] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:48.316007 16596 raft_consensus.cc:359] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:48.316524 16597 raft_consensus.cc:385] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:48.316795 16596 raft_consensus.cc:385] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:48.316838 16597 raft_consensus.cc:740] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:48.317303 16596 raft_consensus.cc:740] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:48.318524 16644 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:48.318667 16597 consensus_queue.cc:260] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:48.320801 16595 tablet_bootstrap.cc:492] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:48.321312 16595 ts_tablet_manager.cc:1403] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.065s	user 0.018s	sys 0.005s
I20260709 12:21:48.321345 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:48.322098 16328 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:48.324643 16595 raft_consensus.cc:359] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:48.325393 16595 raft_consensus.cc:385] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:48.325702 16595 raft_consensus.cc:740] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:48.326571 16595 consensus_queue.cc:260] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:48.328538 16597 ts_tablet_manager.cc:1434] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 12:21:48.328222 16596 consensus_queue.cc:260] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:48.331002 16596 ts_tablet_manager.cc:1434] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.000s
I20260709 12:21:48.332408 16596 tablet_bootstrap.cc:492] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:48.334858 16402 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:21:48.329341 16328 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:21:48.337153 16597 tablet_bootstrap.cc:492] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:48.337868 16636 consensus_queue.cc:1048] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:48.339334 16610 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:48.349073 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:48.349753 16403 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:48.348879 16610 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:48.355543 16597 tablet_bootstrap.cc:654] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:48.356364 16401 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:48.358326 16596 tablet_bootstrap.cc:654] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:48.359676 16516 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:21:48.359761 16441 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:21:48.361112 16677 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:48.351456 16644 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:48.367305 16636 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:48.367484 16677 raft_consensus.cc:697] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:21:48.368376 16636 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:48.369825 16677 consensus_queue.cc:237] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:48.370872 16636 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:48.373435 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:48.374164 16549 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 0.
I20260709 12:21:48.365398 16604 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:48.374185 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:48.375998 16595 ts_tablet_manager.cc:1434] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.054s	user 0.013s	sys 0.005s
I20260709 12:21:48.377627 16595 tablet_bootstrap.cc:492] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:48.375468 16477 raft_consensus.cc:2393] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 1: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:21:48.375281 16403 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:21:48.381667 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:48.382315 16328 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 0.
I20260709 12:21:48.386765 16367 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:48.388108 16636 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:48.388561 16636 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:48.389739 16636 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:48.396519 16637 consensus_queue.cc:1048] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:48.398706 16222 catalog_manager.cc:5697] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:48.402789 16599 tablet_bootstrap.cc:492] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:48.403527 16599 ts_tablet_manager.cc:1403] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.105s	user 0.016s	sys 0.000s
I20260709 12:21:48.404412 16636 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:48.405026 16644 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:48.408660 16644 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:48.410924 16644 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:48.407025 16327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:48.412132 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:48.405601 16610 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:48.416937 16326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:48.416651 16636 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:21:48.418805 16441 leader_election.cc:343] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:48.419896 16549 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:48.425544 16327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:48.425426 16367 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:21:48.426440 16327 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:48.427382 16636 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:48.428210 16610 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:48.428846 16610 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:48.436626 16596 tablet_bootstrap.cc:492] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:48.437577 16610 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:48.439167 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:48.439746 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:48.440405 16477 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 0.
W20260709 12:21:48.440769 16367 leader_election.cc:343] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:48.437386 16596 ts_tablet_manager.cc:1403] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.105s	user 0.011s	sys 0.014s
I20260709 12:21:48.445819 16366 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:21:48.446480 16644 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:48.447192 16610 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:48.446955 16644 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:48.447633 16610 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:48.448118 16610 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:48.449358 16644 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:48.451272 16326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:48.452620 16326 raft_consensus.cc:2393] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 1: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:21:48.454826 16595 tablet_bootstrap.cc:654] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:48.443470 16604 raft_consensus.cc:697] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:21:48.443197 16549 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:21:48.459318 16599 raft_consensus.cc:359] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:48.461531 16599 raft_consensus.cc:385] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:48.461787 16599 raft_consensus.cc:740] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:48.453033 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:48.464002 16630 consensus_queue.cc:1048] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:48.467069 16442 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:48.470067 16636 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:48.470623 16636 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:48.471832 16597 tablet_bootstrap.cc:492] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:48.472601 16597 ts_tablet_manager.cc:1403] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.136s	user 0.028s	sys 0.038s
I20260709 12:21:48.473037 16630 consensus_queue.cc:1048] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:48.474792 16622 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:48.476063 16604 consensus_queue.cc:237] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:48.462383 16599 consensus_queue.cc:260] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:48.494750 16597 raft_consensus.cc:359] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:48.496037 16597 raft_consensus.cc:385] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:48.496531 16597 raft_consensus.cc:740] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:48.491884 16630 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:48.500378 16630 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:48.502847 16327 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:21:48.502923 16630 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:48.498917 16367 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:48.504707 16637 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:48.505440 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:48.505896 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:48.507066 16597 consensus_queue.cc:260] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:48.506230 16551 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 0.
W20260709 12:21:48.509332 16291 leader_election.cc:343] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:48.505771 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:48.509734 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:48.510233 16403 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 0.
I20260709 12:21:48.510613 16549 raft_consensus.cc:2393] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 1: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:21:48.510178 16441 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:21:48.499367 16644 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:48.515007 16644 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:48.516213 16293 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:21:48.517607 16630 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:48.518033 16630 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:48.518378 16630 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:48.523037 16681 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:48.523618 16681 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:48.517575 16644 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:48.526579 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:48.527365 16403 raft_consensus.cc:2393] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 1: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:21:48.529507 16630 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:48.530697 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:48.531052 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:48.534165 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:21:48.535527 16291 leader_election.cc:343] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:48.534721 16222 catalog_manager.cc:5697] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:48.538735 16630 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:48.540808 16441 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:21:48.505810 16636 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:48.543872 16677 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:48.546859 16327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:48.545284 16610 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:48.574720 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:48.575695 16680 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:48.575723 16401 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:48.576828 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:48.578752 16514 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:48.579871 16604 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:48.587082 16225 catalog_manager.cc:5697] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 1 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:48.552228 16644 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:48.551964 16677 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:48.560956 16637 raft_consensus.cc:697] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:21:48.593364 16596 raft_consensus.cc:359] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:48.594379 16596 raft_consensus.cc:385] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:48.594759 16596 raft_consensus.cc:740] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:48.561828 16681 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:48.597036 16596 consensus_queue.cc:260] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:48.601725 16597 ts_tablet_manager.cc:1434] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.129s	user 0.018s	sys 0.016s
I20260709 12:21:48.603212 16597 tablet_bootstrap.cc:492] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:48.604458 16599 ts_tablet_manager.cc:1434] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.201s	user 0.018s	sys 0.007s
I20260709 12:21:48.605553 16599 tablet_bootstrap.cc:492] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
W20260709 12:21:48.588783 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 48157.4us
I20260709 12:21:48.593154 16677 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:48.611596 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 97215.4us
I20260709 12:21:48.618775 16367 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:21:48.619397 16599 tablet_bootstrap.cc:654] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:48.588589 16680 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:48.633658 16596 ts_tablet_manager.cc:1434] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.191s	user 0.018s	sys 0.012s
I20260709 12:21:48.638355 16596 tablet_bootstrap.cc:492] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:48.643884 16597 tablet_bootstrap.cc:654] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:48.646572 16690 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:48.655040 16549 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:48.656563 16401 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 1.
I20260709 12:21:48.658233 16610 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:48.663883 16596 tablet_bootstrap.cc:654] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:48.664523 16604 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:48.666119 16622 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:48.637758 16637 consensus_queue.cc:237] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:48.676681 16644 consensus_queue.cc:1048] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:48.684885 16680 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:48.678717 16687 consensus_queue.cc:1048] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:48.686596 16695 consensus_queue.cc:1048] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:48.682667 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:21:48.688405 16476 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:48.684151 16610 consensus_queue.cc:1048] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:48.702193 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:48.704097 16475 raft_consensus.cc:2393] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 1: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:21:48.716776 16599 tablet_bootstrap.cc:492] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:48.717319 16599 ts_tablet_manager.cc:1403] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.112s	user 0.015s	sys 0.015s
I20260709 12:21:48.720170 16599 raft_consensus.cc:359] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:48.720882 16599 raft_consensus.cc:385] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:48.721163 16599 raft_consensus.cc:740] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:48.721671 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:48.721927 16599 consensus_queue.cc:260] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:48.723356 16690 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:48.724517 16599 ts_tablet_manager.cc:1434] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:21:48.725160 16688 consensus_queue.cc:1048] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:48.725543 16599 tablet_bootstrap.cc:492] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:48.707154 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:48.729785 16516 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:21:48.730020 16367 leader_election.cc:343] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:48.730840 16604 raft_consensus.cc:2749] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:48.730906 16610 consensus_queue.cc:1048] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:48.702131 16677 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:48.753738 16595 tablet_bootstrap.cc:492] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:48.754267 16595 ts_tablet_manager.cc:1403] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.377s	user 0.006s	sys 0.021s
I20260709 12:21:48.754757 16476 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:21:48.757925 16677 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:48.750666 16293 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:21:48.760677 16596 tablet_bootstrap.cc:492] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:48.761256 16596 ts_tablet_manager.cc:1403] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.126s	user 0.010s	sys 0.006s
I20260709 12:21:48.760605 16595 raft_consensus.cc:359] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:48.761976 16595 raft_consensus.cc:385] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:48.762392 16595 raft_consensus.cc:740] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:48.763303 16595 consensus_queue.cc:260] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:48.765506 16695 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:48.766073 16695 raft_consensus.cc:697] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 1 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:21:48.766557 16630 consensus_queue.cc:1048] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:48.766991 16695 consensus_queue.cc:237] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:48.768071 16597 tablet_bootstrap.cc:492] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:48.768589 16597 ts_tablet_manager.cc:1403] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.166s	user 0.007s	sys 0.011s
W20260709 12:21:48.769946 16441 consensus_peers.cc:597] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:48.770437 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 86649.7us
I20260709 12:21:48.771407 16597 raft_consensus.cc:359] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:48.772119 16597 raft_consensus.cc:385] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:48.772218 16690 consensus_queue.cc:1048] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:48.774745 16226 catalog_manager.cc:5697] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 0 to 1, leader changed from <none> to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 1 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:48.780570 16597 raft_consensus.cc:740] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:48.777351 16595 ts_tablet_manager.cc:1434] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.001s
I20260709 12:21:48.784096 16597 consensus_queue.cc:260] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:48.791930 16604 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:48.792683 16604 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:48.796080 16604 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:48.796144 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:21:48.797711 16515 leader_election.cc:343] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:48.789584 16366 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:21:48.824129 16595 tablet_bootstrap.cc:492] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:48.824715 16610 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:48.827405 16610 raft_consensus.cc:697] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:21:48.827726 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:21:48.828581 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 48807.4us
I20260709 12:21:48.828413 16610 consensus_queue.cc:237] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:48.830858 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:48.832954 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:48.833810 16547 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:48.834892 16516 leader_election.cc:343] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:48.835388 16516 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:48.839107 16604 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:48.816476 16476 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:48.824975 16597 ts_tablet_manager.cc:1434] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.056s	user 0.000s	sys 0.007s
I20260709 12:21:48.847235 16597 tablet_bootstrap.cc:492] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:48.863415 16677 consensus_queue.cc:1048] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:21:48.870136 16441 leader_election.cc:343] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:48.871495 16326 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:48.871604 16687 consensus_queue.cc:1048] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:48.873503 16596 raft_consensus.cc:359] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:48.874555 16596 raft_consensus.cc:385] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:48.874895 16596 raft_consensus.cc:740] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:48.878515 16599 tablet_bootstrap.cc:654] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:48.880584 16596 consensus_queue.cc:260] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:48.885860 16687 consensus_queue.cc:1048] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:48.886546 16596 ts_tablet_manager.cc:1434] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.125s	user 0.007s	sys 0.000s
I20260709 12:21:48.887533 16596 tablet_bootstrap.cc:492] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
W20260709 12:21:48.898239 16514 consensus_peers.cc:597] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 -> Peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Couldn't send request to peer 52ab50b0237346c1a13ede947f6a63c2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:48.910270 16547 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:21:48.925644 16595 tablet_bootstrap.cc:654] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:48.951368 16597 tablet_bootstrap.cc:654] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:48.956390 16604 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:48.957024 16604 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:48.960285 16680 consensus_queue.cc:1048] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:48.963395 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:48.962828 16226 catalog_manager.cc:5697] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:48.965169 16514 leader_election.cc:343] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:48.966773 16604 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:21:48.980399 16718 log.cc:927] Time spent T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:21:48.982358 16637 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:48.983031 16637 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:48.985796 16596 tablet_bootstrap.cc:654] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:48.987553 16687 consensus_queue.cc:1048] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:48.947743 16441 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:21:48.989220 16687 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:21:48.989259 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 59673.5us
I20260709 12:21:48.989940 16687 raft_consensus.cc:697] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:21:48.994129 16622 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:48.994709 16622 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:48.998224 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:48.970314 16327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:21:49.011883 16515 leader_election.cc:343] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:49.012434 16515 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:21:49.013758 16604 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:48.991026 16687 consensus_queue.cc:237] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:49.029944 16599 tablet_bootstrap.cc:492] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:49.009266 16637 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:21:49.038856 16366 leader_election.cc:343] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:49.044430 16599 ts_tablet_manager.cc:1403] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.319s	user 0.015s	sys 0.004s
I20260709 12:21:49.049113 16595 tablet_bootstrap.cc:492] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:49.050266 16595 ts_tablet_manager.cc:1403] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.226s	user 0.010s	sys 0.013s
I20260709 12:21:49.056577 16597 tablet_bootstrap.cc:492] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:49.057119 16597 ts_tablet_manager.cc:1403] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.210s	user 0.021s	sys 0.008s
I20260709 12:21:49.058804 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:49.059752 16622 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:49.060269 16327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:49.060981 16327 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 0.
I20260709 12:21:49.064950 16596 tablet_bootstrap.cc:492] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:49.065658 16596 ts_tablet_manager.cc:1403] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.178s	user 0.014s	sys 0.003s
I20260709 12:21:49.075835 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:21:49.079921 16442 leader_election.cc:343] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:49.083500 16367 leader_election.cc:343] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:49.084234 16367 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:21:49.085290 16637 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:49.098073 16595 raft_consensus.cc:359] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:49.099217 16595 raft_consensus.cc:385] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.099592 16595 raft_consensus.cc:740] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.107437 16596 raft_consensus.cc:359] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:49.111644 16596 raft_consensus.cc:385] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.104105 16595 consensus_queue.cc:260] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:49.111361 16599 raft_consensus.cc:359] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:49.114604 16596 raft_consensus.cc:740] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.115307 16599 raft_consensus.cc:385] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.115629 16599 raft_consensus.cc:740] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.117365 16599 consensus_queue.cc:260] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:49.119699 16599 ts_tablet_manager.cc:1434] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.075s	user 0.005s	sys 0.002s
I20260709 12:21:49.120723 16599 tablet_bootstrap.cc:492] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:49.124464 16597 raft_consensus.cc:359] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:49.125264 16597 raft_consensus.cc:385] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.125566 16597 raft_consensus.cc:740] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.126296 16597 consensus_queue.cc:260] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:49.128022 16597 ts_tablet_manager.cc:1434] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.071s	user 0.004s	sys 0.000s
I20260709 12:21:49.128566 16599 tablet_bootstrap.cc:654] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.129024 16597 tablet_bootstrap.cc:492] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:49.131312 16223 catalog_manager.cc:5697] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 0 to 1, leader changed from <none> to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 1 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:49.133313 16595 ts_tablet_manager.cc:1434] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.083s	user 0.009s	sys 0.000s
I20260709 12:21:49.136222 16595 tablet_bootstrap.cc:492] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:49.133800 16596 consensus_queue.cc:260] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:49.149696 16597 tablet_bootstrap.cc:654] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.154209 16596 ts_tablet_manager.cc:1434] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.088s	user 0.006s	sys 0.003s
I20260709 12:21:49.155922 16596 tablet_bootstrap.cc:492] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:49.159348 16599 tablet_bootstrap.cc:492] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:49.159806 16595 tablet_bootstrap.cc:654] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.160400 16599 ts_tablet_manager.cc:1403] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.040s	user 0.011s	sys 0.021s
W20260709 12:21:49.138582 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 75396.9us
I20260709 12:21:49.171116 16596 tablet_bootstrap.cc:654] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.181932 16637 consensus_queue.cc:1048] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:49.185755 16441 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:49.187778 16687 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:49.188191 16687 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:49.188571 16687 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:49.195241 16597 tablet_bootstrap.cc:492] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:49.195884 16637 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.198488 16597 ts_tablet_manager.cc:1403] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.070s	user 0.017s	sys 0.005s
I20260709 12:21:49.188591 16599 raft_consensus.cc:359] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:49.199491 16599 raft_consensus.cc:385] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.199672 16599 raft_consensus.cc:740] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.204526 16597 raft_consensus.cc:359] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:49.205360 16597 raft_consensus.cc:385] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.205648 16597 raft_consensus.cc:740] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.206393 16597 consensus_queue.cc:260] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:49.209406 16637 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:49.209859 16599 consensus_queue.cc:260] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:49.212307 16599 ts_tablet_manager.cc:1434] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.052s	user 0.004s	sys 0.002s
I20260709 12:21:49.214828 16597 ts_tablet_manager.cc:1434] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 12:21:49.197295 16687 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:49.217479 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:49.218070 16597 tablet_bootstrap.cc:492] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:49.217701 16687 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:49.220429 16599 tablet_bootstrap.cc:492] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:49.222247 16595 tablet_bootstrap.cc:492] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:49.222672 16595 ts_tablet_manager.cc:1403] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.087s	user 0.011s	sys 0.000s
I20260709 12:21:49.224526 16476 raft_consensus.cc:2393] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 1: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:21:49.225298 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:49.225942 16403 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:49.226105 16326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:49.226758 16326 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:49.227114 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:49.230754 16597 tablet_bootstrap.cc:654] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.226495 16695 consensus_queue.cc:1048] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:49.234117 16367 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:21:49.244858 16595 raft_consensus.cc:359] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:49.249226 16595 raft_consensus.cc:385] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.249526 16595 raft_consensus.cc:740] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.220491 16637 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:21:49.248481 16273 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:49.253440 16326 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:21:49.250185 16595 consensus_queue.cc:260] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:49.253836 16403 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:21:49.258302 16599 tablet_bootstrap.cc:654] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.258455 16441 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:21:49.258533 16687 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.259227 16595 ts_tablet_manager.cc:1434] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.036s	user 0.009s	sys 0.000s
I20260709 12:21:49.259220 16687 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:49.260402 16595 tablet_bootstrap.cc:492] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:49.263918 16622 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:49.267130 16622 raft_consensus.cc:697] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:21:49.261982 16694 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:49.274545 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:49.268306 16622 consensus_queue.cc:237] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:49.286005 16596 tablet_bootstrap.cc:492] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:49.286468 16595 tablet_bootstrap.cc:654] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.286634 16596 ts_tablet_manager.cc:1403] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.131s	user 0.023s	sys 0.042s
I20260709 12:21:49.287951 16687 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:49.291875 16637 consensus_queue.cc:1048] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:49.292302 16476 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:49.293146 16596 raft_consensus.cc:359] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:49.293351 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:49.293905 16596 raft_consensus.cc:385] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.294230 16596 raft_consensus.cc:740] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.295531 16697 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.295634 16441 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:49.296134 16697 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:49.298451 16697 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:49.299549 16599 tablet_bootstrap.cc:492] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:49.300107 16599 ts_tablet_manager.cc:1403] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.080s	user 0.017s	sys 0.013s
I20260709 12:21:49.300071 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:49.300954 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:49.302812 16291 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:21:49.304980 16697 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:49.304528 16688 consensus_queue.cc:1048] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:49.308027 16597 tablet_bootstrap.cc:492] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:49.308665 16597 ts_tablet_manager.cc:1403] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.093s	user 0.025s	sys 0.004s
I20260709 12:21:49.310660 16222 catalog_manager.cc:5697] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:49.311869 16597 raft_consensus.cc:359] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:49.314807 16597 raft_consensus.cc:385] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.315140 16597 raft_consensus.cc:740] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.319741 16597 consensus_queue.cc:260] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:49.316787 16596 consensus_queue.cc:260] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:49.324580 16599 raft_consensus.cc:359] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:49.327829 16599 raft_consensus.cc:385] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.328266 16599 raft_consensus.cc:740] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.336473 16596 ts_tablet_manager.cc:1434] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.049s	user 0.008s	sys 0.000s
I20260709 12:21:49.338135 16596 tablet_bootstrap.cc:492] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
W20260709 12:21:49.340278 16291 consensus_peers.cc:597] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 -> Peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Couldn't send request to peer 52ab50b0237346c1a13ede947f6a63c2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:49.378975 16599 consensus_queue.cc:260] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:49.379354 16597 ts_tablet_manager.cc:1434] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.070s	user 0.009s	sys 0.000s
I20260709 12:21:49.380247 16699 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.380646 16596 tablet_bootstrap.cc:654] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.380910 16699 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:49.382072 16551 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:49.384968 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:49.385015 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:49.386628 16548 raft_consensus.cc:2468] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 0.
W20260709 12:21:49.388156 16291 leader_election.cc:343] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:49.388530 16699 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:49.388886 16292 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:21:49.390029 16697 raft_consensus.cc:2804] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:49.390545 16697 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:49.391002 16697 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:49.395794 16599 ts_tablet_manager.cc:1434] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.094s	user 0.014s	sys 0.000s
I20260709 12:21:49.396385 16595 tablet_bootstrap.cc:492] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:49.396853 16599 tablet_bootstrap.cc:492] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:49.396874 16595 ts_tablet_manager.cc:1403] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.137s	user 0.017s	sys 0.000s
I20260709 12:21:49.403110 16595 raft_consensus.cc:359] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:49.403828 16595 raft_consensus.cc:385] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.404145 16595 raft_consensus.cc:740] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.408263 16690 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.408171 16595 consensus_queue.cc:260] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:49.408799 16690 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:49.410331 16595 ts_tablet_manager.cc:1434] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 12:21:49.410933 16690 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:49.411263 16595 tablet_bootstrap.cc:492] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:49.417833 16326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:49.418844 16597 tablet_bootstrap.cc:492] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:49.403525 16599 tablet_bootstrap.cc:654] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.421509 16687 raft_consensus.cc:2749] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:49.423570 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:49.424319 16549 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 0.
I20260709 12:21:49.425549 16692 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.426091 16692 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:49.421572 16697 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:21:49.448594 16367 leader_election.cc:343] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:49.449921 16326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:49.450793 16367 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:21:49.452477 16693 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:21:49.451663 16515 leader_election.cc:343] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:49.453011 16693 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:49.453403 16693 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:49.453966 16595 tablet_bootstrap.cc:654] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.459545 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:21:49.461285 16291 leader_election.cc:343] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:49.462384 16692 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:49.464006 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:49.469671 16690 consensus_queue.cc:1048] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:49.473780 16596 tablet_bootstrap.cc:492] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:49.474267 16596 ts_tablet_manager.cc:1403] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.136s	user 0.010s	sys 0.002s
I20260709 12:21:49.476270 16596 raft_consensus.cc:359] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:49.468180 16693 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:49.477065 16596 raft_consensus.cc:385] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.477384 16596 raft_consensus.cc:740] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
W20260709 12:21:49.465724 16516 leader_election.cc:343] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:49.478142 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:49.478425 16516 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:49.478843 16551 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:49.479483 16692 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:49.482672 16599 tablet_bootstrap.cc:492] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:49.486603 16551 raft_consensus.cc:2468] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 1.
I20260709 12:21:49.487263 16693 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:49.488124 16292 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:21:49.489336 16699 raft_consensus.cc:2804] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:49.489845 16699 raft_consensus.cc:697] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 1 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:21:49.490727 16699 consensus_queue.cc:237] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:21:49.491518 16441 consensus_peers.cc:597] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:49.496137 16599 ts_tablet_manager.cc:1403] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.099s	user 0.008s	sys 0.009s
I20260709 12:21:49.478073 16596 consensus_queue.cc:260] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:49.500383 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:49.501302 16549 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:49.502936 16687 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.503561 16687 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:49.504304 16326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:49.505403 16596 ts_tablet_manager.cc:1434] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.001s
I20260709 12:21:49.505700 16687 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:49.506611 16596 tablet_bootstrap.cc:492] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:49.507256 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:49.507751 16599 raft_consensus.cc:359] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:49.507901 16547 raft_consensus.cc:2468] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 0.
I20260709 12:21:49.508002 16326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:49.508476 16599 raft_consensus.cc:385] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.508844 16599 raft_consensus.cc:740] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.500809 16223 catalog_manager.cc:5697] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
W20260709 12:21:49.509502 16441 leader_election.cc:343] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:49.510346 16441 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:21:49.511531 16687 raft_consensus.cc:2804] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:49.478097 16697 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:49.509578 16599 consensus_queue.cc:260] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:49.512501 16687 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:49.513835 16687 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:49.515486 16599 ts_tablet_manager.cc:1434] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:21:49.516566 16599 tablet_bootstrap.cc:492] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
W20260709 12:21:49.525507 16367 leader_election.cc:343] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:49.525923 16599 tablet_bootstrap.cc:654] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.527315 16597 tablet_bootstrap.cc:654] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.536305 16549 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:21:49.538904 16367 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:21:49.545830 16687 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:49.548247 16690 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:49.549161 16690 raft_consensus.cc:697] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:21:49.549368 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:49.550000 16549 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:49.550074 16690 consensus_queue.cc:237] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:49.555859 16326 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:49.556563 16599 tablet_bootstrap.cc:492] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:49.557154 16599 ts_tablet_manager.cc:1403] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.041s	user 0.011s	sys 0.000s
I20260709 12:21:49.559844 16599 raft_consensus.cc:359] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:49.560570 16599 raft_consensus.cc:385] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.560840 16599 raft_consensus.cc:740] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.546412 16225 catalog_manager.cc:5697] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:49.564296 16597 tablet_bootstrap.cc:492] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:49.565419 16597 ts_tablet_manager.cc:1403] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.147s	user 0.007s	sys 0.010s
I20260709 12:21:49.568425 16595 tablet_bootstrap.cc:492] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:49.569022 16595 ts_tablet_manager.cc:1403] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.158s	user 0.032s	sys 0.014s
I20260709 12:21:49.569967 16596 tablet_bootstrap.cc:654] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.572681 16599 consensus_queue.cc:260] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:49.575691 16549 raft_consensus.cc:2468] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:21:49.576659 16687 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:49.578502 16441 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:21:49.578814 16599 ts_tablet_manager.cc:1434] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 12:21:49.579984 16599 tablet_bootstrap.cc:492] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:49.579609 16597 raft_consensus.cc:359] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:49.579787 16325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:49.580358 16597 raft_consensus.cc:385] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.580678 16597 raft_consensus.cc:740] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.581607 16595 raft_consensus.cc:359] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:49.582320 16595 raft_consensus.cc:385] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.582588 16595 raft_consensus.cc:740] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
W20260709 12:21:49.583488 16441 leader_election.cc:343] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:49.583819 16595 consensus_queue.cc:260] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:49.586071 16595 ts_tablet_manager.cc:1434] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 12:21:49.586458 16599 tablet_bootstrap.cc:654] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.585772 16225 catalog_manager.cc:5697] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 0 to 1, leader changed from <none> to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 1 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:49.584741 16687 raft_consensus.cc:2804] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:49.586630 16597 consensus_queue.cc:260] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:49.588380 16595 tablet_bootstrap.cc:492] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:49.590045 16687 raft_consensus.cc:697] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:21:49.591281 16687 consensus_queue.cc:237] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:49.593962 16597 ts_tablet_manager.cc:1434] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 12:21:49.595377 16597 tablet_bootstrap.cc:492] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:49.601452 16595 tablet_bootstrap.cc:654] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.601195 16622 consensus_queue.cc:1048] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:49.601833 16597 tablet_bootstrap.cc:654] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.620291 16595 tablet_bootstrap.cc:492] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:49.620471 16597 tablet_bootstrap.cc:492] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:49.620874 16595 ts_tablet_manager.cc:1403] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.033s	user 0.009s	sys 0.006s
I20260709 12:21:49.621070 16597 ts_tablet_manager.cc:1403] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.026s	user 0.016s	sys 0.002s
I20260709 12:21:49.626164 16597 raft_consensus.cc:359] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:49.627038 16597 raft_consensus.cc:385] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.627379 16597 raft_consensus.cc:740] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.629612 16595 raft_consensus.cc:359] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:49.630463 16595 raft_consensus.cc:385] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.630090 16597 consensus_queue.cc:260] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:49.630787 16595 raft_consensus.cc:740] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.639062 16597 ts_tablet_manager.cc:1434] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.001s
I20260709 12:21:49.640308 16597 tablet_bootstrap.cc:492] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:49.641697 16595 consensus_queue.cc:260] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:49.644158 16223 catalog_manager.cc:5697] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:49.646792 16595 ts_tablet_manager.cc:1434] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.003s
I20260709 12:21:49.647893 16595 tablet_bootstrap.cc:492] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:49.652527 16690 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.653105 16690 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:49.654017 16597 tablet_bootstrap.cc:654] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.657579 16690 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:49.661975 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:49.662874 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:21:49.664429 16366 leader_election.cc:343] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:49.665011 16367 leader_election.cc:343] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:49.665581 16367 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:21:49.666725 16690 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:49.673700 16595 tablet_bootstrap.cc:654] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.675098 16690 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.676601 16690 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:49.679008 16690 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:49.681205 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:49.681327 16687 consensus_queue.cc:1048] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:49.684623 16596 tablet_bootstrap.cc:492] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:49.685115 16596 ts_tablet_manager.cc:1403] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.179s	user 0.019s	sys 0.006s
I20260709 12:21:49.685822 16599 tablet_bootstrap.cc:492] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:49.686252 16599 ts_tablet_manager.cc:1403] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.106s	user 0.040s	sys 0.011s
W20260709 12:21:49.687186 16367 leader_election.cc:343] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:49.687745 16596 raft_consensus.cc:359] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:49.688542 16596 raft_consensus.cc:385] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.688280 16326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:21:49.689720 16367 leader_election.cc:343] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:49.690153 16367 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:21:49.690438 16687 consensus_queue.cc:1048] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:49.692134 16596 raft_consensus.cc:740] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.697798 16596 consensus_queue.cc:260] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:49.696995 16690 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:49.703095 16596 ts_tablet_manager.cc:1434] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 12:21:49.702976 16599 raft_consensus.cc:359] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:49.703768 16599 raft_consensus.cc:385] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.704171 16599 raft_consensus.cc:740] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.704771 16596 tablet_bootstrap.cc:492] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:49.705013 16599 consensus_queue.cc:260] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:49.700765 16677 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.709103 16599 ts_tablet_manager.cc:1434] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.001s
I20260709 12:21:49.713425 16599 tablet_bootstrap.cc:492] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:49.709807 16677 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:49.718389 16690 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.728168 16690 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:49.731499 16690 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:49.733006 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:49.734221 16549 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 0.
I20260709 12:21:49.736483 16367 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:49.746009 16690 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:49.746639 16690 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:49.747104 16690 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:49.754942 16597 tablet_bootstrap.cc:492] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:49.755452 16597 ts_tablet_manager.cc:1403] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.115s	user 0.009s	sys 0.010s
I20260709 12:21:49.762554 16597 raft_consensus.cc:359] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:49.763332 16597 raft_consensus.cc:385] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.763621 16597 raft_consensus.cc:740] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.764299 16597 consensus_queue.cc:260] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:49.766732 16597 ts_tablet_manager.cc:1434] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:21:49.767691 16597 tablet_bootstrap.cc:492] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:49.774740 16597 tablet_bootstrap.cc:654] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.781243 16599 tablet_bootstrap.cc:654] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.771888 16690 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:49.785143 16622 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.796746 16622 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:49.797905 16677 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:49.800258 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:49.801054 16477 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 0.
I20260709 12:21:49.802541 16476 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:49.803861 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:49.804680 16549 raft_consensus.cc:2393] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 1: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:21:49.806013 16325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:49.810623 16622 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:49.814170 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:49.815395 16403 raft_consensus.cc:2393] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 1: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:21:49.816242 16441 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:21:49.817716 16622 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:49.825304 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:49.826057 16548 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 0.
I20260709 12:21:49.827612 16622 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:49.830098 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:21:49.830715 16477 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:49.831511 16690 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:49.831930 16596 tablet_bootstrap.cc:654] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.832618 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:49.835697 16548 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:49.833958 16595 tablet_bootstrap.cc:492] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:49.836987 16595 ts_tablet_manager.cc:1403] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.189s	user 0.011s	sys 0.007s
I20260709 12:21:49.839783 16595 raft_consensus.cc:359] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:49.840292 16599 tablet_bootstrap.cc:492] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:49.841126 16599 ts_tablet_manager.cc:1403] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.128s	user 0.012s	sys 0.001s
I20260709 12:21:49.841683 16441 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:49.842854 16548 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:21:49.840472 16595 raft_consensus.cc:385] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.844599 16595 raft_consensus.cc:740] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.845407 16477 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:21:49.845265 16595 consensus_queue.cc:260] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:49.845650 16367 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:49.846372 16687 raft_consensus.cc:2764] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:21:49.847566 16595 ts_tablet_manager.cc:1434] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 12:21:49.847195 16599 raft_consensus.cc:359] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:49.848541 16595 tablet_bootstrap.cc:492] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:49.848281 16694 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:49.849215 16694 raft_consensus.cc:697] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:21:49.849604 16599 raft_consensus.cc:385] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.850040 16694 consensus_queue.cc:237] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:49.835096 16597 tablet_bootstrap.cc:492] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:49.852901 16597 ts_tablet_manager.cc:1403] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.085s	user 0.006s	sys 0.006s
I20260709 12:21:49.854969 16599 raft_consensus.cc:740] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.857728 16597 raft_consensus.cc:359] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:49.858207 16595 tablet_bootstrap.cc:654] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.858487 16597 raft_consensus.cc:385] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.858764 16597 raft_consensus.cc:740] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.859537 16597 consensus_queue.cc:260] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:49.861745 16597 ts_tablet_manager.cc:1434] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:21:49.862779 16597 tablet_bootstrap.cc:492] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:49.866822 16596 tablet_bootstrap.cc:492] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:49.868682 16597 tablet_bootstrap.cc:654] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.868887 16595 tablet_bootstrap.cc:492] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:49.869304 16596 ts_tablet_manager.cc:1403] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.165s	user 0.018s	sys 0.001s
I20260709 12:21:49.869318 16595 ts_tablet_manager.cc:1403] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.021s	user 0.004s	sys 0.007s
I20260709 12:21:49.872646 16596 raft_consensus.cc:359] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:49.873056 16595 raft_consensus.cc:359] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:49.856316 16599 consensus_queue.cc:260] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:49.877226 16595 raft_consensus.cc:385] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.879014 16599 ts_tablet_manager.cc:1434] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.000s
I20260709 12:21:49.879469 16596 raft_consensus.cc:385] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.879841 16596 raft_consensus.cc:740] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.877589 16595 raft_consensus.cc:740] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.881157 16599 tablet_bootstrap.cc:492] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:49.881573 16595 consensus_queue.cc:260] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:49.884027 16595 ts_tablet_manager.cc:1434] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.000s
I20260709 12:21:49.885284 16597 tablet_bootstrap.cc:492] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:49.885779 16597 ts_tablet_manager.cc:1403] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.002s
I20260709 12:21:49.885691 16699 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.886314 16699 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:49.888396 16599 tablet_bootstrap.cc:654] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.889549 16595 tablet_bootstrap.cc:492] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:49.889971 16750 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.889911 16596 consensus_queue.cc:260] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:49.890618 16750 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:49.892433 16596 ts_tablet_manager.cc:1434] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 12:21:49.892831 16750 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:49.893659 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:49.894349 16403 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 0.
I20260709 12:21:49.896402 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:49.896860 16293 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:49.897060 16476 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 0.
I20260709 12:21:49.897114 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:49.898088 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:49.898124 16548 raft_consensus.cc:2393] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 1: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:21:49.900002 16699 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:49.901113 16293 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:49.902700 16750 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:49.903095 16750 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:49.903498 16750 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:49.906834 16225 catalog_manager.cc:5697] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 0 to 1, leader changed from <none> to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 1 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:49.908218 16699 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:49.893666 16596 tablet_bootstrap.cc:492] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:49.921636 16750 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:49.925686 16699 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:49.927213 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:21:49.927489 16597 raft_consensus.cc:359] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:49.927217 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:49.928287 16597 raft_consensus.cc:385] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.928756 16597 raft_consensus.cc:740] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.929524 16597 consensus_queue.cc:260] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:49.927907 16476 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:49.933422 16597 ts_tablet_manager.cc:1434] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.023s	user 0.002s	sys 0.004s
I20260709 12:21:49.934464 16597 tablet_bootstrap.cc:492] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:49.937181 16596 tablet_bootstrap.cc:654] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.937250 16697 consensus_queue.cc:1048] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:49.928773 16403 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:49.943908 16599 tablet_bootstrap.cc:492] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:49.944713 16599 ts_tablet_manager.cc:1403] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.064s	user 0.008s	sys 0.007s
I20260709 12:21:49.946331 16750 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:49.948429 16597 tablet_bootstrap.cc:654] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.950109 16476 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 1.
I20260709 12:21:49.950730 16403 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 1.
I20260709 12:21:49.951373 16291 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:21:49.953464 16750 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:49.954144 16595 tablet_bootstrap.cc:654] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:49.973337 16750 raft_consensus.cc:697] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 1 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:21:49.974726 16750 consensus_queue.cc:237] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:49.978216 16599 raft_consensus.cc:359] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:49.978955 16599 raft_consensus.cc:385] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.980024 16677 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.980571 16677 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:49.979210 16599 raft_consensus.cc:740] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:49.982862 16599 consensus_queue.cc:260] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:49.984751 16595 tablet_bootstrap.cc:492] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:49.986150 16595 ts_tablet_manager.cc:1403] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.097s	user 0.006s	sys 0.007s
I20260709 12:21:49.991251 16677 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:49.991850 16690 consensus_queue.cc:1048] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:49.992030 16325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:49.993160 16596 tablet_bootstrap.cc:492] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:49.993695 16596 ts_tablet_manager.cc:1403] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.100s	user 0.016s	sys 0.014s
I20260709 12:21:49.996555 16596 raft_consensus.cc:359] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:49.997328 16596 raft_consensus.cc:385] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:49.997643 16596 raft_consensus.cc:740] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.001029 16597 tablet_bootstrap.cc:492] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:50.001554 16597 ts_tablet_manager.cc:1403] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.067s	user 0.020s	sys 0.013s
I20260709 12:21:50.002022 16596 consensus_queue.cc:260] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:50.003144 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:50.004918 16403 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 0.
W20260709 12:21:50.005259 16441 leader_election.cc:343] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:50.003151 16595 raft_consensus.cc:359] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:50.004000 16599 ts_tablet_manager.cc:1434] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.059s	user 0.006s	sys 0.000s
I20260709 12:21:50.005983 16595 raft_consensus.cc:385] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.006289 16595 raft_consensus.cc:740] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.006672 16599 tablet_bootstrap.cc:492] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:50.006567 16442 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:21:50.006938 16595 consensus_queue.cc:260] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:50.008553 16595 ts_tablet_manager.cc:1434] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 12:21:50.009861 16697 consensus_queue.cc:1048] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:50.011830 16622 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:50.012228 16622 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:50.012627 16622 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.012957 16596 ts_tablet_manager.cc:1434] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.001s
I20260709 12:21:50.013918 16596 tablet_bootstrap.cc:492] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:50.028405 16595 tablet_bootstrap.cc:492] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:50.028050 16693 consensus_queue.cc:1048] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:50.040779 16766 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.041374 16766 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:50.048887 16677 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.049922 16677 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:50.056705 16326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:50.057046 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:50.057442 16326 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 0.
I20260709 12:21:50.058660 16765 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.059196 16765 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:50.061736 16765 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:50.063125 16599 tablet_bootstrap.cc:654] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.062999 16515 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:21:50.063309 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:50.075961 16403 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 0.
I20260709 12:21:50.076165 16765 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:50.064250 16326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:50.076671 16765 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:50.077098 16765 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:50.067493 16441 leader_election.cc:343] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:50.077235 16516 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:50.078277 16604 raft_consensus.cc:2804] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:50.078640 16604 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:50.078972 16604 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.082844 16223 catalog_manager.cc:5697] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:50.069566 16596 tablet_bootstrap.cc:654] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.086532 16604 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:50.087718 16677 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:50.089532 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:50.090289 16327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:50.056456 16597 raft_consensus.cc:359] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:50.077044 16326 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 0.
I20260709 12:21:50.092794 16597 raft_consensus.cc:385] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.093145 16597 raft_consensus.cc:740] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.069909 16622 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:50.096365 16595 tablet_bootstrap.cc:654] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.095633 16597 consensus_queue.cc:260] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:50.105216 16597 ts_tablet_manager.cc:1434] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.103s	user 0.010s	sys 0.000s
I20260709 12:21:50.106479 16597 tablet_bootstrap.cc:492] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:50.098067 16692 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:50.099363 16441 leader_election.cc:343] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:50.115502 16692 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:50.126457 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:50.130074 16403 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.126932 16604 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:50.116204 16441 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:21:50.137725 16599 tablet_bootstrap.cc:492] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:50.138240 16599 ts_tablet_manager.cc:1403] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.132s	user 0.010s	sys 0.016s
I20260709 12:21:50.132467 16622 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:50.135399 16604 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.134104 16732 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:50.141113 16599 raft_consensus.cc:359] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:50.143039 16599 raft_consensus.cc:385] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.143431 16599 raft_consensus.cc:740] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.144127 16597 tablet_bootstrap.cc:654] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.146267 16596 tablet_bootstrap.cc:492] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:50.146850 16596 ts_tablet_manager.cc:1403] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.133s	user 0.019s	sys 0.017s
I20260709 12:21:50.139828 16765 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:50.147778 16599 consensus_queue.cc:260] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:50.149003 16766 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:50.150846 16599 ts_tablet_manager.cc:1434] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:21:50.153697 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:50.153770 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:50.154331 16599 tablet_bootstrap.cc:492] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:50.154393 16400 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.154498 16402 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 0.
I20260709 12:21:50.157908 16596 raft_consensus.cc:359] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:50.158592 16596 raft_consensus.cc:385] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.158885 16596 raft_consensus.cc:740] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.159582 16596 consensus_queue.cc:260] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:21:50.147330 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 49698.1us
I20260709 12:21:50.141134 16604 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:50.165426 16596 ts_tablet_manager.cc:1434] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.018s	user 0.001s	sys 0.005s
I20260709 12:21:50.170521 16599 tablet_bootstrap.cc:654] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.171731 16328 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.172200 16596 tablet_bootstrap.cc:492] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:50.167042 16326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:50.164170 16325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:50.163159 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:21:50.177845 16515 leader_election.cc:343] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:50.162004 16692 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:21:50.187634 16516 leader_election.cc:343] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:50.190315 16516 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:50.187726 16732 consensus_queue.cc:1048] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:21:50.193660 16765 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:50.194244 16765 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:50.195137 16596 tablet_bootstrap.cc:654] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.195819 16325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:50.196762 16325 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:50.200700 16441 leader_election.cc:343] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:50.201804 16400 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:21:50.206458 16442 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:21:50.207242 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:50.207757 16732 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:50.208089 16402 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.208390 16732 raft_consensus.cc:697] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:21:50.209352 16732 consensus_queue.cc:237] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:50.213727 16604 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:50.215116 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:21:50.219311 16514 leader_election.cc:343] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:50.220927 16403 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:21:50.230264 16516 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:21:50.230283 16622 consensus_queue.cc:1048] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:50.231719 16402 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:21:50.231762 16692 raft_consensus.cc:2804] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:50.232621 16692 raft_consensus.cc:697] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:21:50.233753 16692 consensus_queue.cc:237] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:50.234565 16595 tablet_bootstrap.cc:492] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:50.235071 16595 ts_tablet_manager.cc:1403] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.207s	user 0.016s	sys 0.023s
I20260709 12:21:50.236874 16325 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:21:50.238562 16516 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:50.238925 16595 raft_consensus.cc:359] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:50.239677 16595 raft_consensus.cc:385] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.239776 16765 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:50.240151 16595 raft_consensus.cc:740] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.240515 16765 raft_consensus.cc:697] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:21:50.241519 16786 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.241245 16595 consensus_queue.cc:260] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:50.224421 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:50.242008 16786 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:50.241539 16765 consensus_queue.cc:237] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:50.247071 16786 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:50.247278 16599 tablet_bootstrap.cc:492] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:50.247822 16599 ts_tablet_manager.cc:1403] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.094s	user 0.023s	sys 0.005s
I20260709 12:21:50.248958 16595 ts_tablet_manager.cc:1434] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 12:21:50.250821 16599 raft_consensus.cc:359] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:50.249406 16400 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 0.
I20260709 12:21:50.251909 16693 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.252511 16693 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:50.254010 16595 tablet_bootstrap.cc:492] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:50.254180 16326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:50.251503 16599 raft_consensus.cc:385] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.254869 16599 raft_consensus.cc:740] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.255559 16599 consensus_queue.cc:260] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:21:50.256356 16515 leader_election.cc:343] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:50.259616 16597 tablet_bootstrap.cc:492] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:50.259841 16599 ts_tablet_manager.cc:1434] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:21:50.260118 16597 ts_tablet_manager.cc:1403] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.154s	user 0.017s	sys 0.018s
I20260709 12:21:50.249934 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:50.265443 16599 tablet_bootstrap.cc:492] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:50.271864 16596 tablet_bootstrap.cc:492] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:50.264846 16693 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:50.273998 16516 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:21:50.274447 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:50.274024 16694 consensus_queue.cc:1048] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:21:50.293768 16548 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 0.
I20260709 12:21:50.295485 16367 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:50.296442 16694 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:50.296840 16694 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:50.297169 16694 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:50.302950 16273 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:50.277029 16325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:50.277660 16599 tablet_bootstrap.cc:654] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:50.277801 16514 leader_election.cc:343] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:50.288879 16596 ts_tablet_manager.cc:1403] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.117s	user 0.017s	sys 0.001s
I20260709 12:21:50.292199 16595 tablet_bootstrap.cc:654] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.292420 16786 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:50.304154 16514 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:21:50.307157 16786 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:50.313814 16786 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.314630 16596 raft_consensus.cc:359] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:50.316926 16596 raft_consensus.cc:385] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:21:50.311527 16367 leader_election.cc:343] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:50.317238 16596 raft_consensus.cc:740] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.318179 16596 consensus_queue.cc:260] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:50.312144 16692 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:50.320595 16596 ts_tablet_manager.cc:1434] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:21:50.321699 16596 tablet_bootstrap.cc:492] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:50.323652 16694 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:50.326923 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:50.327600 16548 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.330063 16596 tablet_bootstrap.cc:654] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.339314 16693 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.340044 16693 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:50.343624 16325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:50.346062 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:50.347561 16693 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:21:50.347885 16367 leader_election.cc:343] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:50.349255 16597 raft_consensus.cc:359] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:50.349979 16597 raft_consensus.cc:385] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.350257 16597 raft_consensus.cc:740] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.352653 16694 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:50.355156 16599 tablet_bootstrap.cc:492] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:50.355670 16599 ts_tablet_manager.cc:1403] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.095s	user 0.022s	sys 0.002s
I20260709 12:21:50.354799 16226 catalog_manager.cc:5697] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 1 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:50.357365 16226 catalog_manager.cc:5697] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 1 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:50.361780 16597 consensus_queue.cc:260] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:50.343564 16786 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:50.363066 16225 catalog_manager.cc:5697] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
W20260709 12:21:50.365765 16792 log.cc:927] Time spent T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:21:50.373908 16599 raft_consensus.cc:359] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:50.366770 16325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:50.374764 16599 raft_consensus.cc:385] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:21:50.366055 16366 leader_election.cc:343] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:50.377185 16597 ts_tablet_manager.cc:1434] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.117s	user 0.008s	sys 0.000s
I20260709 12:21:50.381429 16597 tablet_bootstrap.cc:492] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:50.387614 16548 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:21:50.387917 16325 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.388927 16597 tablet_bootstrap.cc:654] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.391538 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:21:50.378065 16599 raft_consensus.cc:740] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.394516 16786 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:50.387465 16366 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:21:50.455068 16367 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:21:50.464210 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 60930.7us
W20260709 12:21:50.395851 16514 leader_election.cc:343] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:50.471522 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 75708.6us
I20260709 12:21:50.457016 16595 tablet_bootstrap.cc:492] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
W20260709 12:21:50.463240 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 98196us
I20260709 12:21:50.473470 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:50.474119 16400 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:50.456530 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 55714.6us
I20260709 12:21:50.476979 16595 ts_tablet_manager.cc:1403] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.226s	user 0.019s	sys 0.025s
I20260709 12:21:50.477135 16325 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:21:50.477380 16690 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:50.477833 16693 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:50.485533 16400 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:21:50.495407 16596 tablet_bootstrap.cc:492] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:50.496281 16596 ts_tablet_manager.cc:1403] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.175s	user 0.018s	sys 0.025s
I20260709 12:21:50.498112 16695 consensus_queue.cc:1048] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:50.500058 16596 raft_consensus.cc:359] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:50.500823 16596 raft_consensus.cc:385] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.501066 16596 raft_consensus.cc:740] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.501515 16516 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:21:50.501808 16596 consensus_queue.cc:260] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:50.502744 16597 tablet_bootstrap.cc:492] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:50.503249 16597 ts_tablet_manager.cc:1403] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.122s	user 0.030s	sys 0.040s
I20260709 12:21:50.503252 16786 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:50.504000 16596 ts_tablet_manager.cc:1434] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:21:50.504278 16786 raft_consensus.cc:697] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:21:50.505084 16596 tablet_bootstrap.cc:492] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:50.504971 16595 raft_consensus.cc:359] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:50.505846 16595 raft_consensus.cc:385] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.506299 16595 raft_consensus.cc:740] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.505317 16786 consensus_queue.cc:237] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:50.506992 16597 raft_consensus.cc:359] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:50.507059 16595 consensus_queue.cc:260] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:50.507913 16597 raft_consensus.cc:385] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.508309 16597 raft_consensus.cc:740] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.509132 16597 consensus_queue.cc:260] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:50.512158 16596 tablet_bootstrap.cc:654] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.519203 16597 ts_tablet_manager.cc:1434] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.008s
I20260709 12:21:50.520730 16597 tablet_bootstrap.cc:492] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:50.527825 16597 tablet_bootstrap.cc:654] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.539816 16597 tablet_bootstrap.cc:492] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:50.540447 16597 ts_tablet_manager.cc:1403] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.020s	user 0.008s	sys 0.009s
I20260709 12:21:50.533639 16226 catalog_manager.cc:5697] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 1 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:50.539148 16599 consensus_queue.cc:260] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:50.543840 16599 ts_tablet_manager.cc:1434] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.188s	user 0.012s	sys 0.002s
I20260709 12:21:50.544941 16599 tablet_bootstrap.cc:492] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:50.548380 16597 raft_consensus.cc:359] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:50.549149 16597 raft_consensus.cc:385] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.549472 16597 raft_consensus.cc:740] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.550145 16597 consensus_queue.cc:260] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:50.552291 16786 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.552855 16786 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:50.554602 16599 tablet_bootstrap.cc:654] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.555042 16786 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:50.559084 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:50.559832 16477 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 0.
I20260709 12:21:50.565793 16597 ts_tablet_manager.cc:1434] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.000s
I20260709 12:21:50.566856 16597 tablet_bootstrap.cc:492] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:50.569280 16514 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:21:50.570382 16786 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:50.572499 16786 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:50.573441 16597 tablet_bootstrap.cc:654] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.573586 16786 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.579725 16596 tablet_bootstrap.cc:492] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:50.580240 16596 ts_tablet_manager.cc:1403] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.075s	user 0.009s	sys 0.008s
I20260709 12:21:50.583138 16596 raft_consensus.cc:359] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:50.583782 16596 raft_consensus.cc:385] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.584045 16596 raft_consensus.cc:740] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.584757 16596 consensus_queue.cc:260] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:50.588248 16695 consensus_queue.cc:1048] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:50.588771 16596 ts_tablet_manager.cc:1434] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:21:50.588784 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:50.589646 16401 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 0.
I20260709 12:21:50.589906 16596 tablet_bootstrap.cc:492] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:50.596029 16786 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:50.606551 16596 tablet_bootstrap.cc:654] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.599464 16597 tablet_bootstrap.cc:492] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:50.598593 16595 ts_tablet_manager.cc:1434] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.120s	user 0.005s	sys 0.001s
I20260709 12:21:50.609743 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:50.610412 16597 ts_tablet_manager.cc:1403] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.041s	user 0.011s	sys 0.000s
I20260709 12:21:50.610704 16692 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.617754 16595 tablet_bootstrap.cc:492] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:50.602123 16693 raft_consensus.cc:697] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:21:50.613756 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:21:50.622905 16476 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.628902 16597 raft_consensus.cc:359] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:50.629633 16597 raft_consensus.cc:385] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.629941 16597 raft_consensus.cc:740] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.629997 16693 consensus_queue.cc:237] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:50.633502 16599 tablet_bootstrap.cc:492] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:50.634678 16599 ts_tablet_manager.cc:1403] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.089s	user 0.036s	sys 0.010s
I20260709 12:21:50.636364 16325 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.637270 16476 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:21:50.638514 16599 raft_consensus.cc:359] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:50.638653 16514 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:21:50.639235 16599 raft_consensus.cc:385] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.639555 16599 raft_consensus.cc:740] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.639722 16765 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:50.640281 16765 raft_consensus.cc:697] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:21:50.641281 16694 consensus_queue.cc:1048] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:50.641214 16765 consensus_queue.cc:237] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:50.614233 16786 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:50.617507 16692 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:50.645591 16604 consensus_queue.cc:1048] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:50.646852 16692 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:50.610424 16401 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.640259 16599 consensus_queue.cc:260] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:50.649547 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:50.650957 16599 ts_tablet_manager.cc:1434] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.000s
I20260709 12:21:50.651973 16599 tablet_bootstrap.cc:492] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:50.652912 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:21:50.653174 16515 leader_election.cc:343] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:50.653643 16476 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 0.
I20260709 12:21:50.654844 16514 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:21:50.656292 16692 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:50.656762 16692 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:50.657068 16692 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.657403 16732 consensus_queue.cc:1048] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:50.658293 16597 consensus_queue.cc:260] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:50.660225 16401 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:21:50.663271 16597 ts_tablet_manager.cc:1434] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.046s	user 0.006s	sys 0.001s
I20260709 12:21:50.664307 16597 tablet_bootstrap.cc:492] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:50.687391 16597 tablet_bootstrap.cc:654] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.701413 16596 tablet_bootstrap.cc:492] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:50.701939 16596 ts_tablet_manager.cc:1403] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.112s	user 0.019s	sys 0.000s
I20260709 12:21:50.704418 16596 raft_consensus.cc:359] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:50.705157 16596 raft_consensus.cc:385] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.705448 16596 raft_consensus.cc:740] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.710999 16599 tablet_bootstrap.cc:654] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.716796 16695 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.717456 16695 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:50.718775 16595 tablet_bootstrap.cc:654] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.721949 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:50.722811 16226 catalog_manager.cc:5697] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 1 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:50.715605 16596 consensus_queue.cc:260] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:50.737051 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:50.739391 16291 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:50.726660 16695 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:50.740371 16699 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:50.716504 16765 consensus_queue.cc:1048] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 12:21:50.743419 16224 catalog_manager.cc:5697] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 0 to 1, leader changed from <none> to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 1 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:50.740787 16692 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:50.748695 16596 ts_tablet_manager.cc:1434] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.046s	user 0.009s	sys 0.000s
I20260709 12:21:50.749958 16596 tablet_bootstrap.cc:492] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:50.757280 16692 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:50.759687 16597 tablet_bootstrap.cc:492] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:50.760198 16597 ts_tablet_manager.cc:1403] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.096s	user 0.013s	sys 0.027s
I20260709 12:21:50.747058 16699 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.763470 16699 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:50.764426 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:21:50.765257 16476 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.768921 16819 consensus_queue.cc:1048] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:50.779305 16326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:50.779585 16599 tablet_bootstrap.cc:492] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:50.780047 16326 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.780140 16599 ts_tablet_manager.cc:1403] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.128s	user 0.008s	sys 0.004s
I20260709 12:21:50.784094 16599 raft_consensus.cc:359] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:50.784792 16599 raft_consensus.cc:385] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.785061 16599 raft_consensus.cc:740] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.785704 16599 consensus_queue.cc:260] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:50.786798 16692 consensus_queue.cc:1048] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:50.788388 16597 raft_consensus.cc:359] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:50.792479 16597 raft_consensus.cc:385] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.794559 16597 raft_consensus.cc:740] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.789808 16599 ts_tablet_manager.cc:1434] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.004s
I20260709 12:21:50.793622 16699 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:50.797152 16595 tablet_bootstrap.cc:492] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:50.797879 16595 ts_tablet_manager.cc:1403] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.180s	user 0.015s	sys 0.012s
I20260709 12:21:50.799245 16599 tablet_bootstrap.cc:492] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:50.802477 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:50.803373 16595 raft_consensus.cc:359] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:50.804199 16595 raft_consensus.cc:385] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.804525 16595 raft_consensus.cc:740] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.805271 16595 consensus_queue.cc:260] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:50.799805 16597 consensus_queue.cc:260] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:50.818601 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:50.822535 16292 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:21:50.824484 16695 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:50.826284 16476 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:21:50.828749 16599 tablet_bootstrap.cc:654] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.830549 16326 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:21:50.828740 16514 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:21:50.834654 16692 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:50.835289 16692 raft_consensus.cc:697] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:21:50.838582 16595 ts_tablet_manager.cc:1434] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.003s
I20260709 12:21:50.839524 16595 tablet_bootstrap.cc:492] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:50.842682 16692 consensus_queue.cc:237] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:50.847105 16596 tablet_bootstrap.cc:654] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.848488 16604 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.849056 16604 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:50.851333 16604 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:50.854337 16597 ts_tablet_manager.cc:1434] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.094s	user 0.007s	sys 0.000s
I20260709 12:21:50.855763 16597 tablet_bootstrap.cc:492] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:50.857334 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:21:50.863999 16514 leader_election.cc:343] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:50.866405 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:21:50.867947 16515 leader_election.cc:343] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:50.868446 16327 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.872809 16694 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.873065 16326 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.873466 16694 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:50.868527 16515 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:21:50.877211 16786 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:50.877789 16694 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:50.879293 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:50.880123 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:21:50.880728 16366 leader_election.cc:343] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:50.882544 16367 leader_election.cc:343] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:50.883200 16367 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:21:50.884321 16694 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:50.886744 16694 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.887241 16694 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:50.889601 16694 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:50.891548 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:50.891739 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:21:50.893342 16367 leader_election.cc:343] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:50.894117 16367 leader_election.cc:343] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:50.894663 16597 tablet_bootstrap.cc:654] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.894587 16367 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:21:50.896262 16694 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:50.899904 16595 tablet_bootstrap.cc:654] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.901628 16687 consensus_queue.cc:1048] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:50.905617 16596 tablet_bootstrap.cc:492] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:50.906309 16596 ts_tablet_manager.cc:1403] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.157s	user 0.011s	sys 0.000s
I20260709 12:21:50.909902 16596 raft_consensus.cc:359] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:50.910706 16596 raft_consensus.cc:385] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.911000 16596 raft_consensus.cc:740] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.911813 16596 consensus_queue.cc:260] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:50.914233 16694 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.914762 16694 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:50.916039 16597 tablet_bootstrap.cc:492] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:50.916682 16597 ts_tablet_manager.cc:1403] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.061s	user 0.013s	sys 0.007s
I20260709 12:21:50.916730 16687 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.917652 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:50.917496 16687 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:50.918776 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:21:50.919201 16367 leader_election.cc:343] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:50.919876 16687 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:50.916882 16694 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:50.920639 16597 raft_consensus.cc:359] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:21:50.921316 16366 leader_election.cc:343] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:50.921983 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:50.922289 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:50.922487 16597 raft_consensus.cc:385] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.922950 16403 raft_consensus.cc:2393] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 1: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:21:50.922963 16597 raft_consensus.cc:740] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.922322 16366 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:21:50.924427 16694 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:50.924119 16597 consensus_queue.cc:260] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:50.925756 16441 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:50.927173 16732 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.928287 16687 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.928849 16687 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:50.932282 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:50.933420 16597 ts_tablet_manager.cc:1434] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
W20260709 12:21:50.933763 16441 leader_election.cc:343] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:50.934660 16597 tablet_bootstrap.cc:492] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:50.935181 16687 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:50.936764 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:50.927244 16596 ts_tablet_manager.cc:1434] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.000s
I20260709 12:21:50.937306 16732 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:50.937530 16403 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 0.
I20260709 12:21:50.938349 16596 tablet_bootstrap.cc:492] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:50.942662 16597 tablet_bootstrap.cc:654] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.938795 16595 tablet_bootstrap.cc:492] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:50.939129 16442 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:21:50.949065 16595 ts_tablet_manager.cc:1403] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.110s	user 0.012s	sys 0.000s
I20260709 12:21:50.949875 16732 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:50.950356 16732 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:50.950765 16732 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:50.957176 16599 tablet_bootstrap.cc:492] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:50.957767 16599 ts_tablet_manager.cc:1403] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.159s	user 0.010s	sys 0.004s
I20260709 12:21:50.958180 16596 tablet_bootstrap.cc:654] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:50.965713 16692 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.966305 16692 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:50.973826 16765 consensus_queue.cc:1048] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:50.976909 16327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:21:50.980207 16515 leader_election.cc:343] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:50.980296 16692 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:50.981346 16732 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:50.982273 16595 raft_consensus.cc:359] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:50.982970 16595 raft_consensus.cc:385] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.977408 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:50.978412 16599 raft_consensus.cc:359] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:50.983266 16595 raft_consensus.cc:740] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.983757 16599 raft_consensus.cc:385] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:50.980195 16224 catalog_manager.cc:5697] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 1 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:50.984186 16599 raft_consensus.cc:740] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:50.985582 16599 consensus_queue.cc:260] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:50.987632 16599 ts_tablet_manager.cc:1434] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.000s
I20260709 12:21:50.988749 16699 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.989259 16699 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:50.991330 16699 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:21:50.991322 16514 leader_election.cc:343] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:50.992067 16514 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:21:50.993680 16765 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:50.994756 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:50.995631 16403 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 0.
I20260709 12:21:50.998148 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:51.001546 16599 tablet_bootstrap.cc:492] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:51.005302 16597 tablet_bootstrap.cc:492] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:51.005816 16597 ts_tablet_manager.cc:1403] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.071s	user 0.013s	sys 0.035s
I20260709 12:21:51.001763 16595 consensus_queue.cc:260] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:51.001216 16293 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:51.009609 16327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:21:51.009654 16292 leader_election.cc:343] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.009819 16699 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:21:51.010978 16441 leader_election.cc:343] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.011001 16699 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.011572 16699 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.011847 16597 raft_consensus.cc:359] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:51.012559 16597 raft_consensus.cc:385] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.012832 16597 raft_consensus.cc:740] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.013530 16597 consensus_queue.cc:260] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:51.015693 16597 ts_tablet_manager.cc:1434] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:21:51.019593 16699 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:51.023442 16597 tablet_bootstrap.cc:492] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:51.023599 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:51.024868 16699 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:21:51.024935 16292 leader_election.cc:343] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.025213 16604 consensus_queue.cc:1048] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:51.029229 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:51.029898 16597 tablet_bootstrap.cc:654] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.030036 16403 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.032897 16599 tablet_bootstrap.cc:654] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.034653 16595 ts_tablet_manager.cc:1434] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.085s	user 0.007s	sys 0.001s
I20260709 12:21:51.035403 16597 tablet_bootstrap.cc:492] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:51.035619 16595 tablet_bootstrap.cc:492] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:51.035910 16597 ts_tablet_manager.cc:1403] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 12:21:51.039897 16596 tablet_bootstrap.cc:492] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:51.039565 16597 raft_consensus.cc:359] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:51.040416 16596 ts_tablet_manager.cc:1403] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.102s	user 0.041s	sys 0.013s
I20260709 12:21:51.040498 16597 raft_consensus.cc:385] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.041005 16597 raft_consensus.cc:740] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.044409 16732 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:51.044487 16807 consensus_queue.cc:1048] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:51.046543 16597 consensus_queue.cc:260] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:51.047587 16694 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.048107 16694 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:51.049463 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:51.050069 16401 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.055485 16694 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:51.056464 16732 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.056964 16732 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:51.058856 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:51.059137 16732 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:51.062532 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:21:51.063874 16441 leader_election.cc:343] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.064081 16403 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 1.
I20260709 12:21:51.065405 16293 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:21:51.066219 16699 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:51.066732 16699 raft_consensus.cc:697] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 1 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:21:51.067378 16367 leader_election.cc:343] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.068794 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:51.069425 16548 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 0.
I20260709 12:21:51.067432 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:51.067543 16699 consensus_queue.cc:237] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:51.070371 16402 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 0.
I20260709 12:21:51.074829 16401 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:21:51.078722 16442 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:21:51.081794 16732 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:51.082310 16732 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.082767 16732 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.083494 16597 ts_tablet_manager.cc:1434] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.000s
I20260709 12:21:51.084823 16597 tablet_bootstrap.cc:492] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:51.079774 16367 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:21:51.087510 16694 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:51.087440 16442 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:21:51.088042 16694 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.088472 16694 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.089169 16687 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:51.089854 16687 raft_consensus.cc:697] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:21:51.090764 16687 consensus_queue.cc:237] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:51.069991 16596 raft_consensus.cc:359] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:51.093324 16596 raft_consensus.cc:385] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.093719 16596 raft_consensus.cc:740] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.094503 16596 consensus_queue.cc:260] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:51.093379 16599 tablet_bootstrap.cc:492] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:51.096073 16599 ts_tablet_manager.cc:1403] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.108s	user 0.026s	sys 0.005s
I20260709 12:21:51.101141 16599 raft_consensus.cc:359] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:51.101779 16599 raft_consensus.cc:385] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.102115 16599 raft_consensus.cc:740] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.103165 16596 ts_tablet_manager.cc:1434] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.062s	user 0.006s	sys 0.000s
I20260709 12:21:51.102892 16599 consensus_queue.cc:260] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:51.104257 16596 tablet_bootstrap.cc:492] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:51.109095 16599 ts_tablet_manager.cc:1434] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:21:51.110136 16599 tablet_bootstrap.cc:492] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:51.110845 16596 tablet_bootstrap.cc:654] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.111394 16807 consensus_queue.cc:1048] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:51.101800 16694 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:51.117141 16694 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:51.118244 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:51.118892 16549 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.120874 16327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:51.121773 16699 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:51.122400 16367 leader_election.cc:343] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.122336 16699 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:51.124765 16699 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:51.126744 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:51.127671 16549 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:21:51.128892 16367 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
W20260709 12:21:51.129431 16291 leader_election.cc:343] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.129114 16225 catalog_manager.cc:5697] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:51.131325 16693 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.131398 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:51.132035 16693 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:51.132357 16549 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 0.
I20260709 12:21:51.133239 16599 tablet_bootstrap.cc:654] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.134306 16693 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:51.134855 16801 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:51.135383 16801 raft_consensus.cc:697] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:21:51.136308 16801 consensus_queue.cc:237] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:51.136713 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:21:51.138819 16366 leader_election.cc:343] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.142727 16327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:21:51.148877 16367 leader_election.cc:343] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.149381 16367 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:21:51.151237 16694 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:51.152182 16222 catalog_manager.cc:5697] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:51.155920 16596 tablet_bootstrap.cc:492] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:51.156590 16596 ts_tablet_manager.cc:1403] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.053s	user 0.014s	sys 0.004s
I20260709 12:21:51.159406 16596 raft_consensus.cc:359] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:51.163727 16596 raft_consensus.cc:385] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.165482 16595 tablet_bootstrap.cc:654] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.169420 16597 tablet_bootstrap.cc:654] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.164074 16596 raft_consensus.cc:740] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.170449 16596 consensus_queue.cc:260] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:51.170828 16292 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:21:51.176466 16695 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:51.176945 16695 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.177268 16695 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.172574 16596 ts_tablet_manager.cc:1434] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.001s
I20260709 12:21:51.178407 16596 tablet_bootstrap.cc:492] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:51.180897 16604 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.181977 16604 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:51.186012 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:51.186867 16400 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 0.
I20260709 12:21:51.188561 16516 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:51.190084 16765 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:51.190546 16765 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.190958 16765 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.197063 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:21:51.199030 16514 leader_election.cc:343] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.199810 16604 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:51.172896 16732 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:51.203426 16595 tablet_bootstrap.cc:492] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:51.206401 16225 catalog_manager.cc:5697] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 0 to 1, leader changed from <none> to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 1 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:51.208706 16596 tablet_bootstrap.cc:654] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.211951 16765 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:51.214208 16765 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:51.214877 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:51.215433 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:21:51.215596 16400 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.216267 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:21:51.217729 16514 leader_election.cc:343] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.218204 16732 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:21:51.217751 16441 leader_election.cc:343] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.217880 16595 ts_tablet_manager.cc:1403] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.182s	user 0.010s	sys 0.003s
I20260709 12:21:51.222893 16400 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:21:51.230679 16595 raft_consensus.cc:359] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:51.231483 16595 raft_consensus.cc:385] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.231755 16595 raft_consensus.cc:740] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.231755 16516 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:21:51.231985 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:51.232728 16403 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.232649 16595 consensus_queue.cc:260] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:51.227151 16695 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:51.234967 16595 ts_tablet_manager.cc:1434] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.016s	user 0.002s	sys 0.003s
I20260709 12:21:51.235975 16595 tablet_bootstrap.cc:492] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:51.237442 16695 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:51.238545 16604 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:51.240542 16604 raft_consensus.cc:697] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:21:51.241660 16604 consensus_queue.cc:237] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:51.242419 16596 tablet_bootstrap.cc:492] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:51.243103 16596 ts_tablet_manager.cc:1403] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.065s	user 0.018s	sys 0.004s
I20260709 12:21:51.237223 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:21:51.244026 16477 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.246060 16596 raft_consensus.cc:359] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:51.246809 16596 raft_consensus.cc:385] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.247073 16596 raft_consensus.cc:740] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.247924 16596 consensus_queue.cc:260] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:51.250723 16596 ts_tablet_manager.cc:1434] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:21:51.251760 16596 tablet_bootstrap.cc:492] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:51.252764 16595 tablet_bootstrap.cc:654] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.254393 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:51.255205 16548 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.258800 16596 tablet_bootstrap.cc:654] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.271776 16765 consensus_queue.cc:1048] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:51.277596 16765 consensus_queue.cc:1048] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:51.291059 16403 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:21:51.292624 16548 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 1.
I20260709 12:21:51.305152 16292 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:21:51.306614 16699 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:51.307320 16699 raft_consensus.cc:697] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 1 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:21:51.310006 16699 consensus_queue.cc:237] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:51.311909 16442 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:21:51.313850 16732 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:51.314622 16732 raft_consensus.cc:697] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:21:51.321460 16599 tablet_bootstrap.cc:492] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:51.322050 16599 ts_tablet_manager.cc:1403] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.212s	user 0.021s	sys 0.013s
I20260709 12:21:51.323314 16477 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 1.
I20260709 12:21:51.323668 16596 tablet_bootstrap.cc:492] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:51.323596 16765 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.324322 16765 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:51.315765 16732 consensus_queue.cc:237] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:51.326478 16765 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:51.329181 16327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:51.350859 16597 tablet_bootstrap.cc:492] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:51.351403 16597 ts_tablet_manager.cc:1403] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.267s	user 0.025s	sys 0.013s
I20260709 12:21:51.353545 16596 ts_tablet_manager.cc:1403] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.102s	user 0.020s	sys 0.035s
I20260709 12:21:51.358186 16596 raft_consensus.cc:359] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:51.362676 16604 consensus_queue.cc:1048] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:51.363525 16596 raft_consensus.cc:385] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.364029 16596 raft_consensus.cc:740] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.364748 16596 consensus_queue.cc:260] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:51.366958 16699 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.367455 16699 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:51.367499 16222 catalog_manager.cc:5697] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 1 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:51.369693 16699 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:51.371543 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:51.372298 16403 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 0.
I20260709 12:21:51.374133 16293 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:51.376375 16695 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:51.376935 16695 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.377368 16695 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.354247 16597 raft_consensus.cc:359] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:51.353523 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:51.360378 16599 raft_consensus.cc:359] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:51.389212 16599 raft_consensus.cc:385] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.389626 16599 raft_consensus.cc:740] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.390627 16599 consensus_queue.cc:260] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:51.384049 16597 raft_consensus.cc:385] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.393764 16597 raft_consensus.cc:740] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.395901 16596 ts_tablet_manager.cc:1434] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.000s
I20260709 12:21:51.397565 16225 catalog_manager.cc:5697] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:51.400300 16596 tablet_bootstrap.cc:492] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:51.402709 16400 raft_consensus.cc:2393] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 1: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:21:51.404814 16694 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.405370 16694 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:51.406111 16516 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:51.407464 16786 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.407697 16694 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:21:51.410089 16816 log.cc:927] Time spent T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.076s	user 0.002s	sys 0.000s
I20260709 12:21:51.412319 16599 ts_tablet_manager.cc:1434] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.090s	user 0.010s	sys 0.000s
I20260709 12:21:51.416033 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:51.416260 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:51.416800 16549 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 0.
I20260709 12:21:51.421437 16367 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:51.422402 16694 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:51.422827 16694 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.423331 16694 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:51.436514 16366 leader_election.cc:343] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.447499 16765 consensus_queue.cc:1048] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:51.454833 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:51.455729 16595 tablet_bootstrap.cc:492] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:51.449580 16786 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:51.456125 16699 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.456223 16595 ts_tablet_manager.cc:1403] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.221s	user 0.021s	sys 0.004s
I20260709 12:21:51.465930 16595 raft_consensus.cc:359] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:51.466688 16595 raft_consensus.cc:385] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.466974 16595 raft_consensus.cc:740] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
W20260709 12:21:51.456461 16291 leader_election.cc:343] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.468019 16595 consensus_queue.cc:260] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:51.463204 16699 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:51.471292 16595 ts_tablet_manager.cc:1434] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.003s
I20260709 12:21:51.471503 16699 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:51.472277 16595 tablet_bootstrap.cc:492] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:51.472298 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:51.473292 16403 raft_consensus.cc:2393] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 1: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:21:51.458303 16694 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:51.475646 16599 tablet_bootstrap.cc:492] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:51.479465 16595 tablet_bootstrap.cc:654] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.481724 16223 catalog_manager.cc:5697] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:51.484576 16694 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:51.485780 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:21:51.490751 16597 consensus_queue.cc:260] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:21:51.491899 16366 leader_election.cc:343] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.493702 16596 tablet_bootstrap.cc:654] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.487376 16599 tablet_bootstrap.cc:654] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.475471 16695 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:51.500200 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:51.500892 16548 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.515941 16597 ts_tablet_manager.cc:1434] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.164s	user 0.012s	sys 0.001s
I20260709 12:21:51.530151 16597 tablet_bootstrap.cc:492] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
W20260709 12:21:51.551735 16291 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 82844.6us
I20260709 12:21:51.562523 16599 tablet_bootstrap.cc:492] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:51.563448 16599 ts_tablet_manager.cc:1403] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.088s	user 0.017s	sys 0.003s
I20260709 12:21:51.569164 16548 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:21:51.572456 16327 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.577265 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:51.578027 16403 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.579571 16732 consensus_queue.cc:1048] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:51.582423 16695 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:51.585158 16367 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:21:51.585986 16807 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:51.588231 16807 raft_consensus.cc:697] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:21:51.591503 16597 tablet_bootstrap.cc:654] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.594600 16694 consensus_queue.cc:1048] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:51.598163 16596 tablet_bootstrap.cc:492] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:51.598726 16596 ts_tablet_manager.cc:1403] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.201s	user 0.030s	sys 0.022s
I20260709 12:21:51.599051 16858 consensus_queue.cc:1048] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:51.592434 16732 consensus_queue.cc:1048] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:21:51.602843 16599 raft_consensus.cc:359] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:51.603293 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:21:51.603591 16599 raft_consensus.cc:385] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.603938 16599 raft_consensus.cc:740] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.604672 16599 consensus_queue.cc:260] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:51.603938 16474 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.606841 16599 ts_tablet_manager.cc:1434] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.043s	user 0.004s	sys 0.000s
I20260709 12:21:51.601634 16596 raft_consensus.cc:359] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:51.607858 16403 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 1.
I20260709 12:21:51.607859 16599 tablet_bootstrap.cc:492] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:51.608680 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:51.609802 16293 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:51.610970 16865 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:51.611471 16865 raft_consensus.cc:697] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 1 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:21:51.608263 16596 raft_consensus.cc:385] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.614938 16801 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.592453 16807 consensus_queue.cc:237] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:51.615592 16801 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:51.615487 16865 consensus_queue.cc:237] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:51.617908 16801 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:51.611970 16596 raft_consensus.cc:740] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.635397 16597 tablet_bootstrap.cc:492] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:51.635896 16597 ts_tablet_manager.cc:1403] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.119s	user 0.017s	sys 0.007s
I20260709 12:21:51.636740 16595 tablet_bootstrap.cc:492] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:51.637279 16595 ts_tablet_manager.cc:1403] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.165s	user 0.019s	sys 0.010s
I20260709 12:21:51.640712 16597 raft_consensus.cc:359] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:51.641413 16597 raft_consensus.cc:385] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.642624 16595 raft_consensus.cc:359] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:51.643327 16595 raft_consensus.cc:385] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.643569 16595 raft_consensus.cc:740] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.644176 16595 consensus_queue.cc:260] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:51.644953 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:51.646540 16595 ts_tablet_manager.cc:1434] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:21:51.647514 16595 tablet_bootstrap.cc:492] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:51.650130 16291 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:51.651587 16869 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:51.641755 16597 raft_consensus.cc:740] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.657181 16595 tablet_bootstrap.cc:654] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.657367 16597 consensus_queue.cc:260] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:21:51.658658 16514 consensus_peers.cc:597] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 -> Peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Couldn't send request to peer 52ab50b0237346c1a13ede947f6a63c2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:51.659745 16597 ts_tablet_manager.cc:1434] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.024s	user 0.003s	sys 0.002s
W20260709 12:21:51.659734 16366 leader_election.cc:343] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.660601 16693 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.661273 16597 tablet_bootstrap.cc:492] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:51.661139 16693 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:51.663411 16693 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:51.677680 16597 tablet_bootstrap.cc:654] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.682911 16599 tablet_bootstrap.cc:654] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.686669 16859 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.687216 16859 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:51.689049 16327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:51.689492 16859 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:51.693734 16327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:21:51.696192 16367 leader_election.cc:343] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.691267 16596 consensus_queue.cc:260] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:51.696714 16367 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:21:51.693070 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:51.699725 16858 consensus_queue.cc:1048] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:51.700001 16804 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:51.700059 16550 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 0.
I20260709 12:21:51.702028 16441 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:21:51.703101 16687 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:21:51.705142 16441 leader_election.cc:343] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.703460 16687 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.708472 16223 catalog_manager.cc:5697] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 0 to 1, leader changed from <none> to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 1 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:51.719818 16474 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 1.
I20260709 12:21:51.717260 16687 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:51.722106 16273 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:51.740197 16869 consensus_queue.cc:1048] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:51.749238 16596 ts_tablet_manager.cc:1434] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.150s	user 0.011s	sys 0.000s
I20260709 12:21:51.750171 16596 tablet_bootstrap.cc:492] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:51.756948 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:51.772971 16765 consensus_queue.cc:1048] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:51.770800 16547 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 0.
I20260709 12:21:51.757365 16687 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:51.776103 16599 tablet_bootstrap.cc:492] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:51.776736 16599 ts_tablet_manager.cc:1403] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.169s	user 0.014s	sys 0.035s
I20260709 12:21:51.777592 16595 tablet_bootstrap.cc:492] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:51.778079 16595 ts_tablet_manager.cc:1403] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.131s	user 0.016s	sys 0.027s
I20260709 12:21:51.778650 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:51.779382 16550 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.781316 16595 raft_consensus.cc:359] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:51.782001 16595 raft_consensus.cc:385] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.782271 16595 raft_consensus.cc:740] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:51.783259 16327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:51.779662 16599 raft_consensus.cc:359] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:51.786505 16599 raft_consensus.cc:385] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:51.786893 16599 raft_consensus.cc:740] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
W20260709 12:21:51.787369 16441 leader_election.cc:343] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:51.787916 16687 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:51.787819 16599 consensus_queue.cc:260] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:51.790318 16599 ts_tablet_manager.cc:1434] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:21:51.790675 16595 consensus_queue.cc:260] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:51.793617 16550 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:21:51.793877 16595 ts_tablet_manager.cc:1434] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 12:21:51.794674 16441 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:21:51.797017 16732 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:51.798256 16599 tablet_bootstrap.cc:492] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:51.800411 16732 raft_consensus.cc:697] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:21:51.800988 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:51.801800 16328 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 0.
I20260709 12:21:51.801448 16732 consensus_queue.cc:237] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:51.787813 16596 tablet_bootstrap.cc:654] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.804171 16367 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:51.810464 16693 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:51.811012 16693 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.811343 16595 tablet_bootstrap.cc:492] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:51.811435 16693 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:51.837268 16367 consensus_peers.cc:597] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:51.840402 16693 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:51.866099 16693 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:51.871706 16858 consensus_queue.cc:1048] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:51.870446 16599 tablet_bootstrap.cc:654] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:51.876322 16595 tablet_bootstrap.cc:654] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:51.889837 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 51074.3us
I20260709 12:21:51.894786 16732 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.895354 16732 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:51.896294 16222 catalog_manager.cc:5697] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:51.898723 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:51.899485 16328 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 0.
W20260709 12:21:51.901157 16441 consensus_peers.cc:597] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491): Couldn't send request to peer 610f4feda11942739236f2b121a4b9c1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:51.903318 16441 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:21:51.897473 16732 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:51.923687 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:51.924433 16551 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.931579 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:51.932732 16549 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 0.
I20260709 12:21:51.936575 16551 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:21:51.919499 16859 raft_consensus.cc:2804] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:51.937768 16859 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.938256 16859 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.955736 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:51.956637 16328 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:51.963500 16597 tablet_bootstrap.cc:492] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:51.970419 16597 ts_tablet_manager.cc:1403] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.309s	user 0.023s	sys 0.014s
I20260709 12:21:51.970273 16687 consensus_queue.cc:1048] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:51.967670 16367 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:51.975574 16694 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:51.992611 16694 raft_consensus.cc:697] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:21:51.999935 16694 consensus_queue.cc:237] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:52.002193 16328 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:21:52.016279 16859 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:52.017370 16599 tablet_bootstrap.cc:492] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:52.017593 16596 tablet_bootstrap.cc:492] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:52.017922 16599 ts_tablet_manager.cc:1403] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.220s	user 0.044s	sys 0.072s
I20260709 12:21:52.018139 16596 ts_tablet_manager.cc:1403] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.268s	user 0.025s	sys 0.018s
I20260709 12:21:52.022982 16595 tablet_bootstrap.cc:492] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:52.023684 16595 ts_tablet_manager.cc:1403] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.217s	user 0.033s	sys 0.037s
I20260709 12:21:52.025262 16596 raft_consensus.cc:359] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:52.026113 16596 raft_consensus.cc:385] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.026428 16596 raft_consensus.cc:740] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.027206 16596 consensus_queue.cc:260] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:52.029582 16859 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:52.029752 16596 ts_tablet_manager.cc:1434] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.002s
I20260709 12:21:52.030256 16599 raft_consensus.cc:359] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:52.031307 16599 raft_consensus.cc:385] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.031714 16599 raft_consensus.cc:740] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.032766 16599 consensus_queue.cc:260] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:52.035157 16599 ts_tablet_manager.cc:1434] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:21:52.036227 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:52.036614 16599 tablet_bootstrap.cc:492] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:52.036906 16550 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:52.043583 16596 tablet_bootstrap.cc:492] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:52.047518 16222 catalog_manager.cc:5697] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:52.058981 16595 raft_consensus.cc:359] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:52.060227 16595 raft_consensus.cc:385] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.060689 16595 raft_consensus.cc:740] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.061599 16595 consensus_queue.cc:260] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:52.065169 16595 ts_tablet_manager.cc:1434] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.041s	user 0.009s	sys 0.000s
I20260709 12:21:52.068254 16595 tablet_bootstrap.cc:492] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:52.072477 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:52.073285 16328 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:52.072158 16596 tablet_bootstrap.cc:654] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:52.092216 16869 consensus_queue.cc:1048] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:52.116704 16595 tablet_bootstrap.cc:654] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:52.120234 16550 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:21:52.121989 16441 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:21:52.103125 16328 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:21:52.122890 16859 raft_consensus.cc:2804] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:52.123418 16859 raft_consensus.cc:697] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:21:52.129820 16694 consensus_queue.cc:1048] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:21:52.124301 16859 consensus_queue.cc:237] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:52.149864 16597 raft_consensus.cc:359] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:52.153677 16599 tablet_bootstrap.cc:654] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:52.155531 16869 consensus_queue.cc:1048] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:52.157851 16597 raft_consensus.cc:385] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.167388 16597 raft_consensus.cc:740] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.183240 16597 consensus_queue.cc:260] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:52.189976 16597 ts_tablet_manager.cc:1434] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.218s	user 0.004s	sys 0.004s
I20260709 12:21:52.191406 16597 tablet_bootstrap.cc:492] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:52.206509 16595 tablet_bootstrap.cc:492] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:52.207158 16595 ts_tablet_manager.cc:1403] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.139s	user 0.015s	sys 0.016s
I20260709 12:21:52.206511 16474 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:52.210215 16595 raft_consensus.cc:359] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:52.210301 16222 catalog_manager.cc:5697] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:52.211126 16595 raft_consensus.cc:385] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.211598 16595 raft_consensus.cc:740] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.212311 16595 consensus_queue.cc:260] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:52.218189 16595 ts_tablet_manager.cc:1434] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.004s
I20260709 12:21:52.219259 16595 tablet_bootstrap.cc:492] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:52.219815 16225 catalog_manager.cc:5697] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 0 to 1, leader changed from <none> to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 1 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
W20260709 12:21:52.232861 16366 consensus_peers.cc:597] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 -> Peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Couldn't send request to peer 52ab50b0237346c1a13ede947f6a63c2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:52.239452 16596 tablet_bootstrap.cc:492] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:52.241211 16596 ts_tablet_manager.cc:1403] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.201s	user 0.018s	sys 0.020s
I20260709 12:21:52.243001 16595 tablet_bootstrap.cc:654] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:52.246699 16597 tablet_bootstrap.cc:654] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:52.243904 16596 raft_consensus.cc:359] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:52.249822 16596 raft_consensus.cc:385] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.254624 16864 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:52.255230 16864 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:52.257642 16864 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:52.258452 16327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:52.250176 16596 raft_consensus.cc:740] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.260898 16596 consensus_queue.cc:260] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:52.266320 16596 ts_tablet_manager.cc:1434] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 12:21:52.269110 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:52.268795 16596 tablet_bootstrap.cc:492] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
W20260709 12:21:52.273511 16516 leader_election.cc:343] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:52.266906 16327 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 0.
I20260709 12:21:52.281662 16599 tablet_bootstrap.cc:492] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:52.282229 16599 ts_tablet_manager.cc:1403] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.246s	user 0.015s	sys 0.007s
I20260709 12:21:52.284554 16547 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:52.284610 16515 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:52.290446 16864 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:52.292588 16864 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:52.293967 16864 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:52.304558 16599 raft_consensus.cc:359] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:52.305338 16599 raft_consensus.cc:385] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.305586 16599 raft_consensus.cc:740] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.306195 16599 consensus_queue.cc:260] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:52.311753 16596 tablet_bootstrap.cc:654] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:52.323455 16595 tablet_bootstrap.cc:492] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:52.324043 16595 ts_tablet_manager.cc:1403] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.105s	user 0.008s	sys 0.004s
I20260709 12:21:52.324749 16732 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:21:52.325289 16732 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:52.326694 16595 raft_consensus.cc:359] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:52.327301 16864 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:52.348222 16858 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:52.327282 16595 raft_consensus.cc:385] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.362530 16597 tablet_bootstrap.cc:492] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:52.366153 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:52.390842 16597 ts_tablet_manager.cc:1403] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.200s	user 0.015s	sys 0.013s
I20260709 12:21:52.363631 16599 ts_tablet_manager.cc:1434] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.081s	user 0.006s	sys 0.005s
I20260709 12:21:52.377486 16864 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:52.394959 16599 tablet_bootstrap.cc:492] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:52.388804 16595 raft_consensus.cc:740] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.383174 16858 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:52.398285 16328 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:52.398020 16597 raft_consensus.cc:359] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:52.371505 16732 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:52.398903 16597 raft_consensus.cc:385] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.377220 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:52.399322 16597 raft_consensus.cc:740] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.400156 16597 consensus_queue.cc:260] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:52.379894 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:52.401170 16400 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:52.402520 16597 ts_tablet_manager.cc:1434] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:21:52.403725 16597 tablet_bootstrap.cc:492] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:52.378775 16327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:52.405087 16327 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:52.384927 16677 consensus_queue.cc:1048] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260709 12:21:52.406035 16596 tablet_bootstrap.cc:492] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:52.411624 16596 ts_tablet_manager.cc:1403] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.144s	user 0.012s	sys 0.002s
I20260709 12:21:52.411914 16595 consensus_queue.cc:260] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:52.414400 16441 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:52.417300 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:52.418146 16551 raft_consensus.cc:2393] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 1: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:21:52.420895 16677 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:52.421729 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:52.422971 16596 raft_consensus.cc:359] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:52.423694 16596 raft_consensus.cc:385] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.423962 16596 raft_consensus.cc:740] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.424639 16596 consensus_queue.cc:260] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:52.426610 16595 ts_tablet_manager.cc:1434] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.102s	user 0.001s	sys 0.011s
I20260709 12:21:52.427064 16291 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:21:52.427618 16595 tablet_bootstrap.cc:492] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:52.420172 16599 tablet_bootstrap.cc:654] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:52.436825 16596 ts_tablet_manager.cc:1434] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.000s
I20260709 12:21:52.438127 16596 tablet_bootstrap.cc:492] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:52.440109 16786 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:21:52.440725 16786 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:52.446291 16595 tablet_bootstrap.cc:654] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:52.456646 16597 tablet_bootstrap.cc:654] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:52.457633 16596 tablet_bootstrap.cc:654] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:52.459399 16786 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:52.460562 16599 tablet_bootstrap.cc:492] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:52.461163 16599 ts_tablet_manager.cc:1403] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.066s	user 0.013s	sys 0.008s
I20260709 12:21:52.450680 16858 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:52.463479 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:52.473704 16894 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:52.469324 16694 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:52.476897 16694 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:52.479348 16694 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:52.483289 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:52.484726 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:52.485476 16476 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 0.
I20260709 12:21:52.486799 16366 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:52.488955 16516 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:52.490492 16692 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:52.491734 16694 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:52.492345 16694 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:52.492779 16694 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:52.494194 16400 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:21:52.498056 16327 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:21:52.503135 16884 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:52.503682 16884 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:52.522159 16326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:52.525060 16599 raft_consensus.cc:359] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:52.526228 16599 raft_consensus.cc:385] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.526602 16599 raft_consensus.cc:740] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.527752 16677 consensus_queue.cc:1048] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:52.533288 16515 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:21:52.535338 16786 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:52.548940 16367 leader_election.cc:343] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:52.527833 16599 consensus_queue.cc:260] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:52.536015 16786 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:52.558226 16599 ts_tablet_manager.cc:1434] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.096s	user 0.008s	sys 0.000s
I20260709 12:21:52.553082 16692 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:52.566263 16884 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:52.566247 16786 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:52.568033 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:21:52.569618 16366 leader_election.cc:343] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:52.571373 16687 consensus_queue.cc:1048] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:52.573621 16595 tablet_bootstrap.cc:492] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:52.574139 16595 ts_tablet_manager.cc:1403] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.147s	user 0.017s	sys 0.014s
I20260709 12:21:52.577750 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:52.579293 16595 raft_consensus.cc:359] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:52.580027 16595 raft_consensus.cc:385] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.579288 16599 tablet_bootstrap.cc:492] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:52.586583 16807 consensus_queue.cc:1048] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:52.587256 16597 tablet_bootstrap.cc:492] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:52.587711 16597 ts_tablet_manager.cc:1403] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.184s	user 0.018s	sys 0.045s
I20260709 12:21:52.588945 16596 tablet_bootstrap.cc:492] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:52.589442 16596 ts_tablet_manager.cc:1403] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.152s	user 0.016s	sys 0.040s
I20260709 12:21:52.580931 16326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:52.591192 16694 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:52.594632 16476 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:52.580312 16595 raft_consensus.cc:740] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
W20260709 12:21:52.580368 16516 leader_election.cc:343] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:52.598784 16515 leader_election.cc:343] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:52.599834 16515 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:52.601100 16864 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:52.611368 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:52.612187 16550 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 0.
I20260709 12:21:52.614362 16596 raft_consensus.cc:359] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:52.616118 16596 raft_consensus.cc:385] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.616515 16599 tablet_bootstrap.cc:654] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:52.611958 16694 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:52.616576 16596 raft_consensus.cc:740] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.617605 16596 consensus_queue.cc:260] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:52.619247 16596 ts_tablet_manager.cc:1434] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.007s
I20260709 12:21:52.620194 16596 tablet_bootstrap.cc:492] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:52.622320 16692 raft_consensus.cc:697] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:21:52.620311 16884 consensus_queue.cc:1048] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:21:52.614566 16597 raft_consensus.cc:359] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:52.624802 16597 raft_consensus.cc:385] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.625063 16597 raft_consensus.cc:740] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.612314 16595 consensus_queue.cc:260] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:52.614984 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:21:52.628159 16367 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:21:52.628855 16597 consensus_queue.cc:260] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:52.637835 16807 raft_consensus.cc:2804] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:52.639339 16807 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:52.639830 16807 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:52.645709 16595 ts_tablet_manager.cc:1434] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.071s	user 0.008s	sys 0.002s
I20260709 12:21:52.650125 16595 tablet_bootstrap.cc:492] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:52.638085 16475 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:52.660109 16597 ts_tablet_manager.cc:1434] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.072s	user 0.005s	sys 0.004s
I20260709 12:21:52.661167 16597 tablet_bootstrap.cc:492] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:52.662895 16732 consensus_queue.cc:1048] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:52.672887 16595 tablet_bootstrap.cc:654] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:52.706630 16475 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:21:52.627012 16692 consensus_queue.cc:237] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:52.677677 16324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:52.721112 16324 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:52.713850 16366 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:52.733131 16694 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:52.733459 16596 tablet_bootstrap.cc:654] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:52.724371 16807 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:52.733711 16694 raft_consensus.cc:697] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:21:52.734680 16694 consensus_queue.cc:237] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:52.736274 16807 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:52.738700 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:21:52.743764 16366 leader_election.cc:343] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:52.751578 16324 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:21:52.790200 16597 tablet_bootstrap.cc:654] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:52.803876 16856 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:52.804502 16856 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:52.806669 16856 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:52.832720 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:52.833734 16551 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:52.851445 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 52962.9us
I20260709 12:21:52.856906 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:52.857846 16403 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 0.
I20260709 12:21:52.849392 16699 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:52.877420 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:52.879446 16596 tablet_bootstrap.cc:492] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:52.880089 16596 ts_tablet_manager.cc:1403] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.260s	user 0.018s	sys 0.025s
I20260709 12:21:52.881222 16442 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:52.875411 16699 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:52.883256 16687 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:52.883782 16687 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:52.884816 16328 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 0.
I20260709 12:21:52.884195 16687 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:52.914444 16599 tablet_bootstrap.cc:492] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:52.915019 16599 ts_tablet_manager.cc:1403] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.336s	user 0.026s	sys 0.023s
I20260709 12:21:52.928848 16599 raft_consensus.cc:359] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:52.929613 16599 raft_consensus.cc:385] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.929864 16599 raft_consensus.cc:740] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.909076 16699 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:52.932967 16596 raft_consensus.cc:359] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:52.937160 16596 raft_consensus.cc:385] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.937505 16596 raft_consensus.cc:740] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.941001 16595 tablet_bootstrap.cc:492] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:52.941537 16595 ts_tablet_manager.cc:1403] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.293s	user 0.014s	sys 0.009s
I20260709 12:21:52.947685 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:52.938277 16596 consensus_queue.cc:260] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:52.953287 16595 raft_consensus.cc:359] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:52.954083 16595 raft_consensus.cc:385] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.958642 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:52.934177 16599 consensus_queue.cc:260] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:52.959372 16403 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 0.
I20260709 12:21:52.954396 16595 raft_consensus.cc:740] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.962191 16596 ts_tablet_manager.cc:1434] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.074s	user 0.008s	sys 0.000s
I20260709 12:21:52.963011 16596 tablet_bootstrap.cc:492] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:52.961803 16595 consensus_queue.cc:260] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:52.964818 16599 ts_tablet_manager.cc:1434] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.050s	user 0.007s	sys 0.001s
I20260709 12:21:52.968000 16293 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:52.977412 16223 catalog_manager.cc:5697] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 0 to 1, leader changed from <none> to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 1 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:52.982921 16597 tablet_bootstrap.cc:492] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:52.983459 16597 ts_tablet_manager.cc:1403] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.322s	user 0.016s	sys 0.006s
I20260709 12:21:52.980548 16894 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:52.985119 16894 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:52.985492 16894 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:52.989529 16596 tablet_bootstrap.cc:654] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:52.992614 16595 ts_tablet_manager.cc:1434] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.051s	user 0.005s	sys 0.003s
I20260709 12:21:52.993733 16595 tablet_bootstrap.cc:492] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:52.996351 16597 raft_consensus.cc:359] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:52.997119 16597 raft_consensus.cc:385] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:52.997437 16597 raft_consensus.cc:740] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:52.985985 16551 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:21:52.999776 16599 tablet_bootstrap.cc:492] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:53.018674 16597 consensus_queue.cc:260] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:53.002720 16595 tablet_bootstrap.cc:654] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.020761 16367 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:21:53.021901 16596 tablet_bootstrap.cc:492] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:53.022378 16596 ts_tablet_manager.cc:1403] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.060s	user 0.014s	sys 0.012s
I20260709 12:21:53.007354 16925 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:53.025075 16596 raft_consensus.cc:359] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:53.025770 16596 raft_consensus.cc:385] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.025626 16925 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:53.026047 16596 raft_consensus.cc:740] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.027869 16925 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:53.029196 16693 raft_consensus.cc:2804] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:53.029738 16693 raft_consensus.cc:697] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:21:53.030556 16693 consensus_queue.cc:237] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:53.029212 16599 tablet_bootstrap.cc:654] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.033890 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:53.034556 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:53.034804 16551 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 0.
I20260709 12:21:53.038174 16597 ts_tablet_manager.cc:1434] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.054s	user 0.005s	sys 0.001s
W20260709 12:21:53.038287 16441 leader_election.cc:343] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:53.039316 16597 tablet_bootstrap.cc:492] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:53.027930 16894 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:53.045586 16923 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:53.046299 16923 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:53.051429 16441 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:21:53.014356 16687 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:53.052698 16859 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:53.053107 16859 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:53.053473 16859 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.054525 16687 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:21:53.016986 16273 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:53.053113 16475 raft_consensus.cc:2393] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 1: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:21:53.056393 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:53.057542 16328 raft_consensus.cc:2393] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 1: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:21:53.061900 16894 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:53.062417 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:21:53.063220 16475 raft_consensus.cc:2393] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 1: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:21:53.063462 16596 consensus_queue.cc:260] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:53.064671 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:53.065335 16400 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.066895 16596 ts_tablet_manager.cc:1434] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.044s	user 0.008s	sys 0.000s
I20260709 12:21:53.068081 16596 tablet_bootstrap.cc:492] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:53.071386 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:53.072009 16222 catalog_manager.cc:5697] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 1 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:53.073125 16599 tablet_bootstrap.cc:492] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:53.073674 16599 ts_tablet_manager.cc:1403] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.074s	user 0.011s	sys 0.004s
I20260709 12:21:53.073855 16442 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:53.075137 16687 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:21:53.076136 16694 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:53.076658 16694 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:53.076676 16599 raft_consensus.cc:359] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:53.077734 16599 raft_consensus.cc:385] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.078107 16599 raft_consensus.cc:740] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.078830 16694 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:53.079290 16597 tablet_bootstrap.cc:654] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.078948 16599 consensus_queue.cc:260] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:53.081596 16599 ts_tablet_manager.cc:1434] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:21:53.082551 16599 tablet_bootstrap.cc:492] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:53.085947 16595 tablet_bootstrap.cc:492] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:53.086467 16595 ts_tablet_manager.cc:1403] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.093s	user 0.012s	sys 0.000s
I20260709 12:21:53.086560 16596 tablet_bootstrap.cc:654] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.116511 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:53.117388 16328 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 0.
I20260709 12:21:53.123456 16923 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:53.099265 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:53.128504 16551 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 0.
I20260709 12:21:53.127815 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:53.132604 16599 tablet_bootstrap.cc:654] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.136577 16596 tablet_bootstrap.cc:492] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:53.137073 16596 ts_tablet_manager.cc:1403] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.069s	user 0.029s	sys 0.013s
I20260709 12:21:53.139008 16859 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:53.139639 16597 tablet_bootstrap.cc:492] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:53.140153 16597 ts_tablet_manager.cc:1403] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.101s	user 0.022s	sys 0.022s
I20260709 12:21:53.139882 16596 raft_consensus.cc:359] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:53.140552 16596 raft_consensus.cc:385] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.140890 16596 raft_consensus.cc:740] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.141559 16596 consensus_queue.cc:260] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:53.141918 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:53.142567 16328 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 0.
I20260709 12:21:53.142802 16367 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:53.143604 16596 ts_tablet_manager.cc:1434] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:21:53.138604 16225 catalog_manager.cc:5697] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 0 to 1, leader changed from <none> to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 1 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:53.144212 16400 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 1.
I20260709 12:21:53.142035 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:53.145689 16293 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:21:53.146371 16596 tablet_bootstrap.cc:492] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:53.125485 16595 raft_consensus.cc:359] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:53.147243 16595 raft_consensus.cc:385] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.147548 16595 raft_consensus.cc:740] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.148618 16894 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:53.149322 16894 raft_consensus.cc:697] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 1 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:21:53.149545 16693 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:53.150238 16693 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:53.150588 16693 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.150411 16894 consensus_queue.cc:237] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:53.145925 16551 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.160909 16596 tablet_bootstrap.cc:654] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.162777 16597 raft_consensus.cc:359] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:53.163538 16597 raft_consensus.cc:385] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.163965 16597 raft_consensus.cc:740] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.170253 16595 consensus_queue.cc:260] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:53.171145 16687 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:53.171572 16687 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:53.172596 16595 ts_tablet_manager.cc:1434] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.086s	user 0.007s	sys 0.002s
I20260709 12:21:53.173017 16596 tablet_bootstrap.cc:492] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:53.173498 16596 ts_tablet_manager.cc:1403] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.027s	user 0.010s	sys 0.000s
I20260709 12:21:53.173573 16595 tablet_bootstrap.cc:492] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:53.163081 16693 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:53.195089 16225 catalog_manager.cc:5697] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:53.196403 16693 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:53.178989 16596 raft_consensus.cc:359] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:53.178247 16597 consensus_queue.cc:260] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:53.197014 16596 raft_consensus.cc:385] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.197321 16596 raft_consensus.cc:740] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.197985 16596 consensus_queue.cc:260] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:53.198539 16597 ts_tablet_manager.cc:1434] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.058s	user 0.006s	sys 0.000s
I20260709 12:21:53.199997 16596 ts_tablet_manager.cc:1434] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.025s	user 0.003s	sys 0.001s
I20260709 12:21:53.200942 16596 tablet_bootstrap.cc:492] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:53.202934 16595 tablet_bootstrap.cc:654] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.203717 16694 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:53.204223 16694 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:53.206817 16694 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:53.209375 16596 tablet_bootstrap.cc:654] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.209265 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:21:53.215910 16366 leader_election.cc:343] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:53.187402 16475 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 0.
I20260709 12:21:53.188472 16551 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:21:53.188694 16322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:53.190373 16599 tablet_bootstrap.cc:492] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:53.218113 16322 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 0.
I20260709 12:21:53.218657 16599 ts_tablet_manager.cc:1403] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.136s	user 0.006s	sys 0.027s
I20260709 12:21:53.221204 16596 tablet_bootstrap.cc:492] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:53.221714 16596 ts_tablet_manager.cc:1403] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.003s
I20260709 12:21:53.221632 16441 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:21:53.223089 16732 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:53.223518 16732 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:53.223901 16732 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.224720 16596 raft_consensus.cc:359] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:53.225483 16596 raft_consensus.cc:385] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.190004 16687 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:53.225656 16597 tablet_bootstrap.cc:492] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:53.191063 16859 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:53.191987 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:53.227005 16328 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.221740 16599 raft_consensus.cc:359] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:53.227806 16599 raft_consensus.cc:385] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.228142 16599 raft_consensus.cc:740] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.229046 16599 consensus_queue.cc:260] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:53.231468 16599 ts_tablet_manager.cc:1434] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:21:53.231810 16597 tablet_bootstrap.cc:654] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.193912 16367 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:53.232931 16599 tablet_bootstrap.cc:492] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:53.225756 16596 raft_consensus.cc:740] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.236215 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:53.237056 16403 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 0.
I20260709 12:21:53.238468 16801 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:53.238879 16801 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:53.239254 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:53.239678 16325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:53.239959 16549 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.241644 16325 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.244742 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:53.245500 16551 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 0.
I20260709 12:21:53.239259 16801 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.235394 16596 consensus_queue.cc:260] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:53.252132 16596 ts_tablet_manager.cc:1434] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 12:21:53.253139 16596 tablet_bootstrap.cc:492] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:53.254649 16441 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:21:53.255581 16910 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:53.256173 16910 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:53.258625 16910 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:53.259994 16597 tablet_bootstrap.cc:492] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:53.260471 16597 ts_tablet_manager.cc:1403] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.035s	user 0.010s	sys 0.001s
I20260709 12:21:53.256582 16677 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:53.262108 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:53.262297 16677 raft_consensus.cc:697] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:21:53.263572 16514 leader_election.cc:343] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:53.263345 16677 consensus_queue.cc:237] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:53.264079 16596 tablet_bootstrap.cc:654] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.263373 16597 raft_consensus.cc:359] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:53.266552 16597 raft_consensus.cc:385] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.266820 16597 raft_consensus.cc:740] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.267458 16597 consensus_queue.cc:260] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:53.269270 16597 ts_tablet_manager.cc:1434] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:21:53.270249 16597 tablet_bootstrap.cc:492] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:53.272989 16322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:53.273746 16322 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 0.
I20260709 12:21:53.275311 16515 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:21:53.276165 16692 raft_consensus.cc:2804] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:53.276537 16692 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:53.276889 16692 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.277156 16597 tablet_bootstrap.cc:654] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.280139 16765 consensus_queue.cc:1048] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:53.283538 16858 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:53.284124 16858 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:53.286383 16858 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:53.293318 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:53.302116 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:53.217492 16595 tablet_bootstrap.cc:492] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:53.305891 16595 ts_tablet_manager.cc:1403] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.133s	user 0.018s	sys 0.003s
I20260709 12:21:53.311437 16292 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:53.312543 16223 catalog_manager.cc:5697] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:53.316573 16858 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:53.302580 16549 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:21:53.321684 16367 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:21:53.322592 16694 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:53.309415 16595 raft_consensus.cc:359] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:53.323815 16595 raft_consensus.cc:385] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.323583 16694 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:53.324131 16595 raft_consensus.cc:740] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.274816 16801 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:53.324944 16595 consensus_queue.cc:260] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:53.341389 16595 ts_tablet_manager.cc:1434] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.000s
I20260709 12:21:53.338460 16367 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:53.344168 16920 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:53.344913 16920 raft_consensus.cc:697] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:21:53.346055 16920 consensus_queue.cc:237] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:53.384485 16595 tablet_bootstrap.cc:492] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:53.387888 16599 tablet_bootstrap.cc:654] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.341547 16923 raft_consensus.cc:2804] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:53.394428 16694 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:53.332782 16693 consensus_queue.cc:1048] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.055s
I20260709 12:21:53.399290 16856 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:53.396708 16923 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:53.404359 16923 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.399906 16856 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:53.411989 16595 tablet_bootstrap.cc:654] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.343081 16692 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:53.396854 16325 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:21:53.395673 16732 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:53.444255 16801 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:53.446110 16596 tablet_bootstrap.cc:492] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:53.446700 16596 ts_tablet_manager.cc:1403] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.194s	user 0.033s	sys 0.036s
I20260709 12:21:53.447091 16323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:53.451571 16328 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:21:53.452551 16692 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:53.457126 16732 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:53.458128 16327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:53.458873 16327 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.458315 16856 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:53.467794 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:21:53.469429 16514 leader_election.cc:343] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:53.470244 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:53.471115 16549 raft_consensus.cc:2393] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 1: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:21:53.473265 16327 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:21:53.475163 16923 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:53.476742 16441 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:21:53.477851 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:53.477964 16923 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:53.478829 16403 raft_consensus.cc:2393] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 1: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:21:53.480871 16949 consensus_queue.cc:1048] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:53.483045 16475 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.484737 16925 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:53.485003 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:53.485757 16925 raft_consensus.cc:697] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:21:53.486876 16925 consensus_queue.cc:237] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:53.491652 16597 tablet_bootstrap.cc:492] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
W20260709 12:21:53.491951 16366 leader_election.cc:343] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:53.492288 16597 ts_tablet_manager.cc:1403] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.222s	user 0.054s	sys 0.078s
I20260709 12:21:53.495685 16597 raft_consensus.cc:359] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:53.496256 16597 raft_consensus.cc:385] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.461835 16596 raft_consensus.cc:359] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:53.496634 16597 raft_consensus.cc:740] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.497133 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:21:53.497134 16596 raft_consensus.cc:385] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.498045 16596 raft_consensus.cc:740] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.498757 16596 consensus_queue.cc:260] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:21:53.499478 16366 leader_election.cc:343] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:53.498963 16597 consensus_queue.cc:260] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:53.501673 16596 ts_tablet_manager.cc:1434] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.055s	user 0.008s	sys 0.000s
I20260709 12:21:53.503589 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:53.504765 16596 tablet_bootstrap.cc:492] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:53.518846 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:53.519634 16549 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.520547 16910 consensus_queue.cc:1048] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:53.522164 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:21:53.528402 16597 ts_tablet_manager.cc:1434] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.000s
I20260709 12:21:53.530723 16807 consensus_queue.cc:1048] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:53.532478 16442 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:53.534384 16597 tablet_bootstrap.cc:492] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:53.536749 16922 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:53.537760 16807 consensus_queue.cc:1048] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:53.539407 16599 tablet_bootstrap.cc:492] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:53.539963 16599 ts_tablet_manager.cc:1403] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.307s	user 0.017s	sys 0.010s
I20260709 12:21:53.542024 16549 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:21:53.547308 16596 tablet_bootstrap.cc:654] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.551421 16323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:53.552385 16323 raft_consensus.cc:2393] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 1: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:21:53.554275 16595 tablet_bootstrap.cc:492] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:53.554775 16595 ts_tablet_manager.cc:1403] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.171s	user 0.010s	sys 0.011s
I20260709 12:21:53.554996 16599 raft_consensus.cc:359] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:53.555703 16599 raft_consensus.cc:385] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.556264 16599 raft_consensus.cc:740] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.559162 16367 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:21:53.563458 16599 consensus_queue.cc:260] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:53.578516 16693 raft_consensus.cc:2804] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:53.579310 16693 raft_consensus.cc:697] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
W20260709 12:21:53.584429 16367 leader_election.cc:343] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:53.585517 16367 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:21:53.593670 16595 raft_consensus.cc:359] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:53.594794 16595 raft_consensus.cc:385] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.595155 16595 raft_consensus.cc:740] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.600975 16599 ts_tablet_manager.cc:1434] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.061s	user 0.008s	sys 0.000s
I20260709 12:21:53.602592 16599 tablet_bootstrap.cc:492] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:53.603848 16801 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:53.601315 16693 consensus_queue.cc:237] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:53.605690 16595 consensus_queue.cc:260] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:53.618098 16595 ts_tablet_manager.cc:1434] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.063s	user 0.007s	sys 0.002s
I20260709 12:21:53.619021 16595 tablet_bootstrap.cc:492] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:53.621245 16597 tablet_bootstrap.cc:654] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.625461 16595 tablet_bootstrap.cc:654] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.638103 16367 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:21:53.645936 16884 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:21:53.658416 16328 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.651415 16869 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:53.667768 16596 tablet_bootstrap.cc:492] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:53.668414 16596 ts_tablet_manager.cc:1403] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.164s	user 0.027s	sys 0.031s
I20260709 12:21:53.667640 16222 catalog_manager.cc:5697] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 0 to 1, leader changed from <none> to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 1 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
W20260709 12:21:53.693413 16293 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 45954.8us
I20260709 12:21:53.700585 16596 raft_consensus.cc:359] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:53.701898 16596 raft_consensus.cc:385] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.702229 16596 raft_consensus.cc:740] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.702924 16596 consensus_queue.cc:260] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:53.663084 16327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:53.707027 16596 ts_tablet_manager.cc:1434] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.000s
I20260709 12:21:53.707588 16327 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.691849 16869 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:53.708176 16596 tablet_bootstrap.cc:492] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:53.709452 16869 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:53.712224 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:53.712947 16551 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 0.
I20260709 12:21:53.714156 16292 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:21:53.721359 16966 consensus_queue.cc:1048] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:53.724089 16596 tablet_bootstrap.cc:654] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.723989 16894 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:53.726204 16894 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:53.726802 16894 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:53.728389 16925 consensus_queue.cc:1048] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:53.747227 16224 catalog_manager.cc:5697] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:53.779016 16599 tablet_bootstrap.cc:654] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.789009 16222 catalog_manager.cc:5697] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 0 to 1, leader changed from <none> to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 1 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:53.796746 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 44672.3us
I20260709 12:21:53.800967 16595 tablet_bootstrap.cc:492] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:53.806608 16595 ts_tablet_manager.cc:1403] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.188s	user 0.014s	sys 0.003s
I20260709 12:21:53.811918 16595 raft_consensus.cc:359] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:53.815477 16595 raft_consensus.cc:385] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.816154 16595 raft_consensus.cc:740] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.817569 16595 consensus_queue.cc:260] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:53.786896 16869 consensus_queue.cc:1048] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.038s
I20260709 12:21:53.819857 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:21:53.820062 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 64669.8us
I20260709 12:21:53.871222 16595 ts_tablet_manager.cc:1434] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.063s	user 0.015s	sys 0.007s
I20260709 12:21:53.873438 16595 tablet_bootstrap.cc:492] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
W20260709 12:21:53.898413 16964 log.cc:927] Time spent T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:21:53.848946 16400 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 0.
I20260709 12:21:53.923954 16597 tablet_bootstrap.cc:492] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:53.925523 16597 ts_tablet_manager.cc:1403] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.391s	user 0.026s	sys 0.012s
I20260709 12:21:53.927712 16327 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:21:53.933800 16597 raft_consensus.cc:359] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:53.937325 16597 raft_consensus.cc:385] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:53.937700 16597 raft_consensus.cc:740] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:53.940289 16599 tablet_bootstrap.cc:492] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:53.986233 16599 ts_tablet_manager.cc:1403] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.384s	user 0.017s	sys 0.024s
I20260709 12:21:53.946144 16515 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:21:53.948495 16596 tablet_bootstrap.cc:492] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:53.997040 16596 ts_tablet_manager.cc:1403] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.289s	user 0.014s	sys 0.022s
W20260709 12:21:53.998365 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 52301.9us
I20260709 12:21:54.005770 16595 tablet_bootstrap.cc:654] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:53.943163 16597 consensus_queue.cc:260] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:54.008888 16765 raft_consensus.cc:2804] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:54.021994 16765 raft_consensus.cc:697] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:21:54.022966 16765 consensus_queue.cc:237] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:53.968532 16859 consensus_queue.cc:1048] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.057s
I20260709 12:21:53.960599 16894 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:54.027647 16597 ts_tablet_manager.cc:1434] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.102s	user 0.006s	sys 0.003s
I20260709 12:21:54.028272 16894 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:54.030752 16596 raft_consensus.cc:359] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:54.031447 16596 raft_consensus.cc:385] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.031710 16596 raft_consensus.cc:740] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.034694 16599 raft_consensus.cc:359] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:54.035419 16599 raft_consensus.cc:385] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.035669 16599 raft_consensus.cc:740] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.039932 16597 tablet_bootstrap.cc:492] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:54.041288 16596 consensus_queue.cc:260] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:54.036294 16599 consensus_queue.cc:260] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:53.968663 16856 consensus_queue.cc:1048] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.071s
I20260709 12:21:54.082820 16596 ts_tablet_manager.cc:1434] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.085s	user 0.008s	sys 0.000s
I20260709 12:21:54.083909 16599 ts_tablet_manager.cc:1434] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.097s	user 0.006s	sys 0.001s
I20260709 12:21:54.083976 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:54.084730 16549 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.090399 16599 tablet_bootstrap.cc:492] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:54.129256 16596 tablet_bootstrap.cc:492] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:54.134593 16984 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.134939 16597 tablet_bootstrap.cc:654] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.136924 16549 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 1.
I20260709 12:21:54.135196 16984 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:54.141921 16595 tablet_bootstrap.cc:492] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:54.142409 16595 ts_tablet_manager.cc:1403] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.269s	user 0.007s	sys 0.013s
I20260709 12:21:54.142370 16984 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:54.145531 16595 raft_consensus.cc:359] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:54.146252 16595 raft_consensus.cc:385] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.146508 16595 raft_consensus.cc:740] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.147389 16595 consensus_queue.cc:260] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:54.151806 16595 ts_tablet_manager.cc:1434] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:21:54.152915 16595 tablet_bootstrap.cc:492] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:54.163885 16599 tablet_bootstrap.cc:654] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.149816 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:54.170163 16400 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.174499 16966 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.174816 16323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:54.175124 16966 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:54.175673 16323 raft_consensus.cc:2393] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 1: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:21:54.176020 16595 tablet_bootstrap.cc:654] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.176501 16693 consensus_queue.cc:1048] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:21:54.177825 16693 consensus_queue.cc:1048] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:54.172019 16292 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:21:54.139322 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 48812.6us
I20260709 12:21:54.184304 16982 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:54.196350 16982 raft_consensus.cc:697] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 1 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:21:54.197600 16982 consensus_queue.cc:237] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:54.205019 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:54.207430 16966 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:54.211381 16859 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.211941 16859 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:54.213982 16859 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:54.220451 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:54.221457 16473 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 0.
I20260709 12:21:54.226426 16441 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:21:54.227131 16291 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:21:54.228034 16400 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 1.
I20260709 12:21:54.228695 16966 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:54.229453 16966 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:54.229535 16984 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.230201 16966 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:54.230770 16366 consensus_peers.cc:597] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 -> Peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Couldn't send request to peer 52ab50b0237346c1a13ede947f6a63c2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:54.237479 16596 tablet_bootstrap.cc:654] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.239032 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:54.239768 16400 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 0.
W20260709 12:21:54.244185 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 67373us
I20260709 12:21:54.247159 16323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:54.249667 16595 tablet_bootstrap.cc:492] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:54.251344 16595 ts_tablet_manager.cc:1403] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.099s	user 0.012s	sys 0.001s
I20260709 12:21:54.252992 16966 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:54.254093 16595 raft_consensus.cc:359] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:54.254781 16595 raft_consensus.cc:385] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.255048 16595 raft_consensus.cc:740] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.255134 16966 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:54.255733 16595 consensus_queue.cc:260] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:54.282263 16595 ts_tablet_manager.cc:1434] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.031s	user 0.000s	sys 0.007s
I20260709 12:21:54.283224 16595 tablet_bootstrap.cc:492] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:54.286154 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:54.286911 16400 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 0.
I20260709 12:21:54.297510 16984 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:54.298396 16677 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.300093 16677 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:54.303187 16677 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:54.303236 16596 tablet_bootstrap.cc:492] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:54.303983 16596 ts_tablet_manager.cc:1403] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.216s	user 0.015s	sys 0.004s
I20260709 12:21:54.305922 16597 tablet_bootstrap.cc:492] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:54.306437 16597 ts_tablet_manager.cc:1403] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.267s	user 0.023s	sys 0.038s
I20260709 12:21:54.309306 16597 raft_consensus.cc:359] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:54.309055 16596 raft_consensus.cc:359] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:54.310381 16597 raft_consensus.cc:385] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.310534 16596 raft_consensus.cc:385] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.310760 16597 raft_consensus.cc:740] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.310961 16596 raft_consensus.cc:740] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.311640 16596 consensus_queue.cc:260] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:54.311594 16597 consensus_queue.cc:260] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:54.314021 16599 tablet_bootstrap.cc:492] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:54.314045 16597 ts_tablet_manager.cc:1434] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:21:54.314823 16599 ts_tablet_manager.cc:1403] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.225s	user 0.020s	sys 0.029s
I20260709 12:21:54.315590 16597 tablet_bootstrap.cc:492] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:54.317413 16596 ts_tablet_manager.cc:1434] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:21:54.318270 16596 tablet_bootstrap.cc:492] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:54.318125 16599 raft_consensus.cc:359] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:54.318977 16599 raft_consensus.cc:385] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.319350 16599 raft_consensus.cc:740] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.320097 16599 consensus_queue.cc:260] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:54.321762 16599 ts_tablet_manager.cc:1434] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
W20260709 12:21:54.373556 16293 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 128836us
I20260709 12:21:54.322764 16597 tablet_bootstrap.cc:654] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.391070 16442 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:54.393429 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:54.256201 16323 raft_consensus.cc:2393] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 1: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:21:54.411144 16694 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.395728 16547 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 0.
I20260709 12:21:54.406980 16596 tablet_bootstrap.cc:654] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.414868 16978 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.426003 16978 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:54.322790 16599 tablet_bootstrap.cc:492] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:54.412272 16694 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:21:54.440562 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 50081.5us
I20260709 12:21:54.444487 16856 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.446194 16856 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:54.446869 16988 consensus_queue.cc:1048] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:54.446585 16984 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:54.448568 16984 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:54.449059 16984 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.451228 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:21:54.459893 16949 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.460886 16949 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:54.466828 16949 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:54.472641 16694 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:54.466159 16441 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:21:54.411693 16807 consensus_queue.cc:1048] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.043s
I20260709 12:21:54.486766 16856 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:54.496487 16982 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.497485 16982 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:54.500707 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:54.501824 16402 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 0.
W20260709 12:21:54.503718 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 65929.7us
I20260709 12:21:54.505456 16856 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:54.506052 16856 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:54.506433 16856 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.527901 16982 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:54.537715 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:54.538094 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:54.539134 16546 raft_consensus.cc:2393] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 1: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:21:54.544693 16597 tablet_bootstrap.cc:492] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:54.545734 16597 ts_tablet_manager.cc:1403] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.230s	user 0.024s	sys 0.024s
I20260709 12:21:54.549991 16323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:54.550810 16323 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 0.
I20260709 12:21:54.551293 16597 raft_consensus.cc:359] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:54.557199 16597 raft_consensus.cc:385] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.561199 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:54.562453 16597 raft_consensus.cc:740] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.566272 16597 consensus_queue.cc:260] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:54.570115 16597 ts_tablet_manager.cc:1434] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.002s
I20260709 12:21:54.573045 16597 tablet_bootstrap.cc:492] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:54.578958 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:21:54.572460 16293 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 148516us
W20260709 12:21:54.582828 16273 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:54.584854 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:54.585680 16402 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.587162 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:54.565973 16546 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 0.
W20260709 12:21:54.589134 16291 leader_election.cc:343] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.532374 16978 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:54.584229 16223 catalog_manager.cc:5697] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 1 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:54.590459 16595 tablet_bootstrap.cc:654] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.593775 16599 tablet_bootstrap.cc:654] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.594339 16547 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 0.
I20260709 12:21:54.599939 16984 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:54.601811 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:54.602121 16473 raft_consensus.cc:2393] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 1: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:21:54.604156 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:54.612867 16984 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:54.623003 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:54.624125 16399 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 0.
I20260709 12:21:54.630900 16472 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:54.640780 17001 log.cc:927] Time spent T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
W20260709 12:21:54.648451 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 61208.9us
I20260709 12:21:54.648720 16292 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:21:54.650059 16597 tablet_bootstrap.cc:654] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.655239 16982 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:54.655731 16982 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:54.656122 16982 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.664093 16402 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 1.
W20260709 12:21:54.666687 16291 leader_election.cc:343] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.667502 16367 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:54.687248 16367 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:21:54.699410 16366 leader_election.cc:343] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:54.699781 16949 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:54.700407 16949 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:54.700870 16949 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.704398 16856 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:54.706239 16595 tablet_bootstrap.cc:492] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:54.709863 16595 ts_tablet_manager.cc:1403] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.427s	user 0.014s	sys 0.002s
I20260709 12:21:54.708695 16596 tablet_bootstrap.cc:492] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:54.713378 16596 ts_tablet_manager.cc:1403] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.395s	user 0.015s	sys 0.002s
I20260709 12:21:54.724860 16894 consensus_queue.cc:1048] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:21:54.725297 16476 raft_consensus.cc:2393] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 1: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:21:54.727129 16765 consensus_queue.cc:1048] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:21:54.728447 16596 raft_consensus.cc:359] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:54.729635 16596 raft_consensus.cc:385] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.731532 16596 raft_consensus.cc:740] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.735379 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:54.736007 17000 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:54.735239 16595 raft_consensus.cc:359] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:54.733266 16596 consensus_queue.cc:260] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:54.739328 16595 raft_consensus.cc:385] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.743798 16595 raft_consensus.cc:740] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.741293 17000 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:54.747804 17000 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.752345 16595 consensus_queue.cc:260] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:54.761390 16595 ts_tablet_manager.cc:1434] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.051s	user 0.007s	sys 0.000s
I20260709 12:21:54.742934 16293 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:21:54.743360 16982 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:54.764195 16596 ts_tablet_manager.cc:1434] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.051s	user 0.007s	sys 0.002s
I20260709 12:21:54.765050 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:54.776355 16398 raft_consensus.cc:2393] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 1: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:21:54.777199 16327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:54.778393 16327 raft_consensus.cc:2393] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 1: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:21:54.770845 16599 tablet_bootstrap.cc:492] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:54.782784 16765 consensus_queue.cc:1048] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:54.783186 16442 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:54.784687 16984 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:54.788525 16597 tablet_bootstrap.cc:492] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:54.791234 16597 ts_tablet_manager.cc:1403] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.220s	user 0.017s	sys 0.020s
I20260709 12:21:54.793236 16894 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:54.793763 16894 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:54.794150 16894 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.796849 16596 tablet_bootstrap.cc:492] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
W20260709 12:21:54.796599 16293 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 58231.7us
I20260709 12:21:54.799661 16856 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:54.801517 16599 ts_tablet_manager.cc:1403] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.479s	user 0.030s	sys 0.016s
I20260709 12:21:54.790848 16982 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:54.807689 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:21:54.808486 16474 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.814131 16949 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:54.825222 16599 raft_consensus.cc:359] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:54.825250 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:54.826334 16599 raft_consensus.cc:385] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:54.826733 16599 raft_consensus.cc:740] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.826821 16551 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:54.827607 16599 consensus_queue.cc:260] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:54.833035 16293 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:21:54.847169 16595 tablet_bootstrap.cc:492] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:54.836249 16474 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 1.
I20260709 12:21:54.850294 16949 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:54.877669 16291 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:21:54.855774 16596 tablet_bootstrap.cc:654] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.893039 16599 ts_tablet_manager.cc:1434] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.091s	user 0.002s	sys 0.006s
I20260709 12:21:54.896412 16599 tablet_bootstrap.cc:492] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:54.909029 16599 tablet_bootstrap.cc:654] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.836263 16597 raft_consensus.cc:359] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:54.932930 16597 raft_consensus.cc:385] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:21:54.934479 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 197195us
I20260709 12:21:54.933313 16597 raft_consensus.cc:740] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:54.887782 16982 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:54.941821 16981 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.942464 16981 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:54.945065 16982 raft_consensus.cc:697] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 1 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:21:54.945541 16595 tablet_bootstrap.cc:654] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:54.946318 16982 consensus_queue.cc:237] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:54.880781 16894 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:54.951258 16894 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:54.937418 16441 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:54.868925 17000 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:54.968357 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:21:54.970078 16475 raft_consensus.cc:2393] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 1: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:21:54.971400 17000 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:54.972290 16687 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:21:54.983201 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:21:54.968487 16597 consensus_queue.cc:260] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:54.936764 16978 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:54.960855 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:54.996539 16549 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:55.010751 16599 tablet_bootstrap.cc:492] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:55.011431 16599 ts_tablet_manager.cc:1403] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.116s	user 0.019s	sys 0.005s
I20260709 12:21:55.004154 16597 ts_tablet_manager.cc:1434] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.213s	user 0.005s	sys 0.003s
I20260709 12:21:55.016944 16981 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:55.018710 16597 tablet_bootstrap.cc:492] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:55.020859 16599 raft_consensus.cc:359] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:55.021624 16599 raft_consensus.cc:385] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:55.021929 16599 raft_consensus.cc:740] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:55.031667 16551 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 1.
I20260709 12:21:55.031448 16599 consensus_queue.cc:260] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:55.033748 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:21:55.042186 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:55.048187 16599 ts_tablet_manager.cc:1434] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.036s	user 0.005s	sys 0.005s
I20260709 12:21:55.051636 16599 tablet_bootstrap.cc:492] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:55.054039 16597 tablet_bootstrap.cc:654] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:55.067714 16599 tablet_bootstrap.cc:654] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:55.042378 16978 raft_consensus.cc:697] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 1 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:21:55.070856 16978 consensus_queue.cc:237] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:55.103871 16549 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:21:55.112926 16595 tablet_bootstrap.cc:492] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:55.113862 16595 ts_tablet_manager.cc:1403] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.267s	user 0.037s	sys 0.047s
I20260709 12:21:55.117461 16597 tablet_bootstrap.cc:492] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:55.117976 16599 tablet_bootstrap.cc:492] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:55.118984 16597 ts_tablet_manager.cc:1403] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.101s	user 0.010s	sys 0.015s
I20260709 12:21:55.119433 16599 ts_tablet_manager.cc:1403] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.068s	user 0.014s	sys 0.003s
I20260709 12:21:55.123071 16599 raft_consensus.cc:359] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:55.123878 16599 raft_consensus.cc:385] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:55.125034 16596 tablet_bootstrap.cc:492] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:55.125837 16596 ts_tablet_manager.cc:1403] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.330s	user 0.017s	sys 0.006s
I20260709 12:21:55.124145 16599 raft_consensus.cc:740] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:55.129357 16596 raft_consensus.cc:359] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:55.130214 16596 raft_consensus.cc:385] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:55.130568 16596 raft_consensus.cc:740] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:55.131315 16596 consensus_queue.cc:260] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:55.131932 16595 raft_consensus.cc:359] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:55.132805 16595 raft_consensus.cc:385] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:21:55.133157 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 150813us
I20260709 12:21:55.133164 16595 raft_consensus.cc:740] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:55.134146 16596 ts_tablet_manager.cc:1434] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:21:55.129101 16599 consensus_queue.cc:260] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:55.135435 16596 tablet_bootstrap.cc:492] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:55.136042 16595 consensus_queue.cc:260] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:55.136848 16599 ts_tablet_manager.cc:1434] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 12:21:55.137928 16599 tablet_bootstrap.cc:492] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:55.139163 16595 ts_tablet_manager.cc:1434] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 12:21:55.140755 16595 tablet_bootstrap.cc:492] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:55.142593 16596 tablet_bootstrap.cc:654] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:55.144291 16599 tablet_bootstrap.cc:654] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:55.150463 16595 tablet_bootstrap.cc:654] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:55.156093 16596 tablet_bootstrap.cc:492] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:55.156616 16596 ts_tablet_manager.cc:1403] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.007s
I20260709 12:21:55.157127 16599 tablet_bootstrap.cc:492] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:55.157620 16599 ts_tablet_manager.cc:1403] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.005s
I20260709 12:21:55.159341 16596 raft_consensus.cc:359] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:55.160073 16596 raft_consensus.cc:385] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:55.160286 16599 raft_consensus.cc:359] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:55.161278 16599 raft_consensus.cc:385] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:55.161767 16599 raft_consensus.cc:740] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
I20260709 12:21:55.162554 16599 consensus_queue.cc:260] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:55.161250 16596 raft_consensus.cc:740] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:55.163945 16596 consensus_queue.cc:260] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:55.165551 16595 tablet_bootstrap.cc:492] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:55.165727 16596 ts_tablet_manager.cc:1434] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:21:55.166100 16595 ts_tablet_manager.cc:1403] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.026s	user 0.007s	sys 0.006s
I20260709 12:21:55.166993 16596 tablet_bootstrap.cc:492] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:55.169592 16595 raft_consensus.cc:359] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:55.170814 16595 raft_consensus.cc:385] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:55.171088 16595 raft_consensus.cc:740] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:55.172070 16599 ts_tablet_manager.cc:1434] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.004s
I20260709 12:21:55.172003 16595 consensus_queue.cc:260] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:55.173724 16599 tablet_bootstrap.cc:492] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1: Bootstrap starting.
I20260709 12:21:55.176789 16595 ts_tablet_manager.cc:1434] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.003s
I20260709 12:21:55.178095 16595 tablet_bootstrap.cc:492] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:55.181769 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:55.182605 16599 tablet_bootstrap.cc:654] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:55.182796 16400 raft_consensus.cc:2393] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 1: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:21:55.184686 16596 tablet_bootstrap.cc:654] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:55.187306 16595 tablet_bootstrap.cc:654] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:55.207337 16293 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 374366us
W20260709 12:21:54.985841 16291 leader_election.cc:343] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:21:55.217821 16291 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 232032us
I20260709 12:21:55.213838 16978 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.219929 16978 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:55.233558 16978 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:55.247378 16963 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.247989 16963 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:55.251683 16963 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:55.259128 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:55.260164 16472 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 0.
I20260709 12:21:55.262187 16514 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:21:55.264451 16981 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:55.264868 16981 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:55.265240 16981 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:55.292025 16596 tablet_bootstrap.cc:492] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:55.293206 16596 ts_tablet_manager.cc:1403] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.126s	user 0.028s	sys 0.015s
I20260709 12:21:55.294028 16595 tablet_bootstrap.cc:492] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:55.296366 16595 ts_tablet_manager.cc:1403] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.119s	user 0.021s	sys 0.032s
I20260709 12:21:55.296875 16596 raft_consensus.cc:359] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:55.297751 16596 raft_consensus.cc:385] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:55.306362 16981 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:55.298110 16596 raft_consensus.cc:740] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:55.309336 16596 consensus_queue.cc:260] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:55.311933 16596 ts_tablet_manager.cc:1434] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.001s
I20260709 12:21:55.313336 16596 tablet_bootstrap.cc:492] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:55.314502 16595 raft_consensus.cc:359] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:55.317252 16595 raft_consensus.cc:385] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:55.318698 16595 raft_consensus.cc:740] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
W20260709 12:21:55.329545 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 195563us
I20260709 12:21:55.329882 16596 tablet_bootstrap.cc:654] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:55.322448 16595 consensus_queue.cc:260] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:21:55.331135 16293 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 105057us
I20260709 12:21:55.343672 16367 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:55.364961 16442 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:55.373615 16599 tablet_bootstrap.cc:492] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1: No bootstrap required, opened a new log
I20260709 12:21:55.374176 16599 ts_tablet_manager.cc:1403] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1: Time spent bootstrapping tablet: real 0.201s	user 0.024s	sys 0.027s
I20260709 12:21:55.377523 16981 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:55.378808 16599 raft_consensus.cc:359] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:55.381695 16595 ts_tablet_manager.cc:1434] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.084s	user 0.008s	sys 0.003s
I20260709 12:21:55.383253 16595 tablet_bootstrap.cc:492] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:55.385905 16694 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.386487 16694 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:55.379606 16599 raft_consensus.cc:385] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:55.388996 16599 raft_consensus.cc:740] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Initialized, Role: FOLLOWER
W20260709 12:21:55.345129 16291 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 94441.3us
I20260709 12:21:55.404439 16599 consensus_queue.cc:260] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:55.400063 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:21:55.402370 17015 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:21:55.404373 16597 raft_consensus.cc:359] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:55.419612 16597 raft_consensus.cc:385] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:55.420028 16597 raft_consensus.cc:740] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:55.366573 16807 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:55.424747 16807 raft_consensus.cc:697] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:21:55.425756 16807 consensus_queue.cc:237] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:55.426529 16599 ts_tablet_manager.cc:1434] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1: Time spent starting tablet: real 0.052s	user 0.007s	sys 0.000s
I20260709 12:21:55.416476 16324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:55.423926 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:55.443974 16328 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 0.
I20260709 12:21:55.454339 16597 consensus_queue.cc:260] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:55.428831 16472 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:55.472225 16324 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 1: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:21:55.474475 16325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:55.475272 16325 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 0.
I20260709 12:21:55.472225 16694 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:21:55.479359 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 128191us
I20260709 12:21:55.479298 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:55.482334 16328 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:55.497687 16597 ts_tablet_manager.cc:1434] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.378s	user 0.007s	sys 0.002s
I20260709 12:21:55.498974 16597 tablet_bootstrap.cc:492] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50: Bootstrap starting.
I20260709 12:21:55.510313 16515 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:21:55.514724 16765 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:55.515317 16765 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:55.515758 16765 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:55.517388 16981 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.518026 16981 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:55.526700 16328 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
W20260709 12:21:55.579752 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 46089.2us
I20260709 12:21:55.584285 16595 tablet_bootstrap.cc:654] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
W20260709 12:21:55.590349 16366 leader_election.cc:343] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:55.614672 17033 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.615558 17033 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:55.618387 17033 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:55.640241 17044 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.640933 17044 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:55.647519 17034 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.648283 17034 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:55.650384 17034 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:55.658278 16859 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.660377 16859 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:55.662942 16859 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:55.673893 16765 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:21:55.679692 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 88231us
I20260709 12:21:55.681468 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:55.682287 16399 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 0.
W20260709 12:21:55.682682 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 100134us
I20260709 12:21:55.684719 16367 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:21:55.686795 16923 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:21:55.690078 16327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:55.690929 16327 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 0.
I20260709 12:21:55.701368 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:55.702226 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:55.702937 16403 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 0.
I20260709 12:21:55.702529 16398 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 1: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:21:55.701606 16367 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:55.707952 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:55.709347 16981 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:55.709527 16293 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:55.710484 17041 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:55.710477 16293 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:55.711313 17041 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:55.711774 17041 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:55.715355 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:55.716362 16397 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 0.
I20260709 12:21:55.709419 17033 consensus_queue.cc:1048] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:21:55.719362 17025 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.719674 17033 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:21:55.720008 17025 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:55.721876 16442 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:55.722002 17041 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:55.722581 17025 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:55.723212 16859 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:55.723651 16859 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:55.724167 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:55.724622 17041 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:55.724123 16859 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:55.725651 16472 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:21:55.727346 16514 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:21:55.725227 16397 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 1: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:21:55.728307 16981 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:55.729059 16981 raft_consensus.cc:697] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:21:55.730108 16981 consensus_queue.cc:237] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:21:55.748317 16293 leader_election.cc:343] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:55.751075 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:55.751699 16403 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:55.756880 16859 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:55.760754 16859 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:55.762421 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:55.763069 16398 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:55.767454 16403 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 1.
I20260709 12:21:55.777669 16293 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:55.778008 16398 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:21:55.779207 16442 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:55.779939 16597 tablet_bootstrap.cc:654] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:55.782094 16978 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:55.782675 16978 raft_consensus.cc:697] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 1 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:21:55.783529 16978 consensus_queue.cc:237] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:55.780905 17025 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:55.786777 17025 raft_consensus.cc:697] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:21:55.787128 17044 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:55.787678 17025 consensus_queue.cc:237] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:55.790187 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:55.796931 16596 tablet_bootstrap.cc:492] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8: No bootstrap required, opened a new log
I20260709 12:21:55.797525 16596 ts_tablet_manager.cc:1403] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.485s	user 0.035s	sys 0.027s
I20260709 12:21:55.805469 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:55.806413 16547 raft_consensus.cc:2393] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 1: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:21:55.811228 17048 raft_consensus.cc:2764] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:21:55.822974 16596 raft_consensus.cc:359] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:55.824496 16596 raft_consensus.cc:385] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:55.824815 16596 raft_consensus.cc:740] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:55.825664 16596 consensus_queue.cc:260] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:55.844452 16321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:55.847764 16320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:55.849161 16320 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 0.
W20260709 12:21:55.879768 17039 log.cc:927] Time spent T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.268s	user 0.000s	sys 0.005s
I20260709 12:21:55.849862 16981 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.852941 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:55.881933 16981 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:55.882251 16398 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 0.
I20260709 12:21:55.858457 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:55.859540 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:55.883862 16403 raft_consensus.cc:2393] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 1: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:21:55.883981 16401 raft_consensus.cc:2393] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 1: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:21:55.887008 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:21:55.892568 16514 leader_election.cc:343] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:21:55.892664 16981 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:55.899309 16515 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:21:55.901095 17045 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:55.901562 17045 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:55.901962 17045 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:55.864262 16596 ts_tablet_manager.cc:1434] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.066s	user 0.018s	sys 0.007s
I20260709 12:21:55.866222 16597 tablet_bootstrap.cc:492] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50: No bootstrap required, opened a new log
I20260709 12:21:55.873898 16226 catalog_manager.cc:5697] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:55.904956 16597 ts_tablet_manager.cc:1403] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50: Time spent bootstrapping tablet: real 0.406s	user 0.022s	sys 0.014s
I20260709 12:21:55.907564 16320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:55.908063 16597 raft_consensus.cc:359] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:55.908574 16320 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 0.
I20260709 12:21:55.911026 16515 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:21:55.909041 16597 raft_consensus.cc:385] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:55.914148 16981 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:55.914438 16597 raft_consensus.cc:740] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Initialized, Role: FOLLOWER
I20260709 12:21:55.914620 16981 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:55.915117 16981 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:55.915993 16597 consensus_queue.cc:260] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:55.923035 17045 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:55.926985 16595 tablet_bootstrap.cc:492] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:55.930428 16595 ts_tablet_manager.cc:1403] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.547s	user 0.014s	sys 0.011s
I20260709 12:21:55.943105 17025 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:21:55.966924 17025 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:21:55.969617 17025 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:21:55.980072 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:55.983249 16549 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:55.981417 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:55.980870 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:55.988808 16597 ts_tablet_manager.cc:1434] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50: Time spent starting tablet: real 0.083s	user 0.007s	sys 0.016s
I20260709 12:21:55.991310 16982 consensus_queue.cc:1048] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:55.995034 16595 raft_consensus.cc:359] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:55.995759 16595 raft_consensus.cc:385] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:55.996155 16595 raft_consensus.cc:740] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:56.004139 16595 consensus_queue.cc:260] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:56.018007 16595 ts_tablet_manager.cc:1434] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.087s	user 0.007s	sys 0.000s
I20260709 12:21:56.021435 16549 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:21:55.999574 16981 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:56.023443 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:56.064837 16595 tablet_bootstrap.cc:492] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2: Bootstrap starting.
I20260709 12:21:56.068204 16516 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:21:56.069463 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 82401.2us
I20260709 12:21:56.070192 16516 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:56.060684 16326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:56.074479 16326 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 1: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:21:56.074345 16325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:56.075083 16325 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:56.075265 17066 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:21:56.075646 17045 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:56.077459 16981 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:56.086090 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:21:56.086879 16472 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:56.089857 17045 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:56.091037 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:56.091637 16403 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:56.094717 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:56.095597 16548 raft_consensus.cc:2393] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 1: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:21:56.099340 16595 tablet_bootstrap.cc:654] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:21:56.095722 16472 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
W20260709 12:21:56.106105 16273 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:56.099844 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:56.113656 16403 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:21:56.114892 16548 raft_consensus.cc:2393] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 1: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:21:56.111821 16949 consensus_queue.cc:1048] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:21:56.118366 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:56.100811 16765 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:56.124013 16328 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:56.133169 16516 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:56.131651 16441 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:56.145175 17063 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:56.155767 16328 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:21:56.156942 16515 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:21:56.147698 17066 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:56.152632 16325 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:21:56.162909 16765 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:56.168519 17033 consensus_queue.cc:1048] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:56.176668 16984 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:21:56.177407 16984 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:56.179842 16984 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:56.182482 17067 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:21:56.183502 16765 raft_consensus.cc:697] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:21:56.165099 17066 raft_consensus.cc:697] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:21:56.185408 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:56.185386 16765 consensus_queue.cc:237] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:56.188524 16320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:56.188035 17066 consensus_queue.cc:237] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:56.197968 17048 consensus_queue.cc:1048] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:56.201151 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:21:56.201661 16223 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53636 (request call id 76) took 1609 ms. Trace:
I20260709 12:21:56.183066 17067 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:55.486299 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:56.218103 17067 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:56.210213 17025 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:56.218220 16546 raft_consensus.cc:2393] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 1: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:21:56.218123 17063 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:56.241451 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:56.211653 16223 rpcz_store.cc:276] 0709 12:21:54.576019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:54.576166 (+   147us) service_pool.cc:224] Handling call
0709 12:21:56.184876 (+1608710us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:21:54.695489 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:21:54.696200 (+   711us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:21:54.696262 (+    62us) write_op.cc:435] Acquired schema lock
0709 12:21:54.696309 (+    47us) tablet.cc:662] Decoding operations
0709 12:21:54.715206 (+ 18897us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:21:54.715304 (+    98us) write_op.cc:641] Partition lock acquired for write op
0709 12:21:54.715343 (+    39us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:21:54.724381 (+  9038us) tablet.cc:701] Row locks acquired
0709 12:21:54.742183 (+ 17802us) write_op.cc:260] PREPARE: finished
0709 12:21:54.746859 (+  4676us) write_op.cc:270] Start()
0709 12:21:54.747023 (+   164us) write_op.cc:276] Timestamp: P: 1783599714744265 usec, L: 0
0709 12:21:54.747073 (+    50us) op_driver.cc:348] REPLICATION: starting
0709 12:21:54.748895 (+  1822us) log.cc:844] Serialized 531 byte log entry
0709 12:21:54.951024 (+202129us) op_driver.cc:464] REPLICATION: finished
0709 12:21:55.105875 (+154851us) write_op.cc:301] APPLY: starting
0709 12:21:55.105970 (+    95us) tablet.cc:1367] starting BulkCheckPresence
0709 12:21:55.106088 (+   118us) tablet.cc:1370] finished BulkCheckPresence
0709 12:21:55.106233 (+   145us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:21:55.106618 (+   385us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:21:55.106867 (+   249us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:21:55.106933 (+    66us) write_op.cc:312] APPLY: finished
0709 12:21:55.107284 (+   351us) log.cc:844] Serialized 22 byte log entry
0709 12:21:55.107514 (+   230us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:55.107616 (+   102us) write_op.cc:454] Released schema lock
0709 12:21:55.107901 (+   285us) write_op.cc:341] FINISH: Updating metrics
0709 12:21:55.109711 (+  1810us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:55.109752 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":24203,"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":154441,"apply.run_cpu_time_us":2612,"apply.run_wall_time_us":3350,"num_ops":1,"prepare.queue_time_us":83107,"prepare.run_cpu_time_us":3886,"prepare.run_wall_time_us":59931,"replication_time_us":203744,"thread_start_us":181407,"threads_started":2}]]}
I20260709 12:21:56.250332 16320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:56.284400 16595 tablet_bootstrap.cc:492] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2: No bootstrap required, opened a new log
I20260709 12:21:56.284997 16595 ts_tablet_manager.cc:1403] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2: Time spent bootstrapping tablet: real 0.235s	user 0.008s	sys 0.012s
I20260709 12:21:56.288391 17064 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:21:56.288326 16595 raft_consensus.cc:359] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:56.289090 16595 raft_consensus.cc:385] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:21:56.289453 16595 raft_consensus.cc:740] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Initialized, Role: FOLLOWER
I20260709 12:21:56.289026 17064 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:56.295006 16441 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:21:56.299105 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:56.301645 16442 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:56.295368 16595 consensus_queue.cc:260] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:56.299286 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:56.304685 16472 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 0.
I20260709 12:21:56.312999 16475 raft_consensus.cc:2393] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 1: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:21:56.313995 16985 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:56.314570 17055 consensus_queue.cc:1048] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:56.315201 16894 consensus_queue.cc:1048] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:56.320719 17062 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:56.321144 17064 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:56.326742 16325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:21:56.326965 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 167962us
I20260709 12:21:56.341333 17064 consensus_queue.cc:1048] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:21:56.389763 16595 ts_tablet_manager.cc:1434] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2: Time spent starting tablet: real 0.104s	user 0.007s	sys 0.002s
I20260709 12:21:56.397266 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:56.408886 17061 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.409504 17061 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:56.398646 16291 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:21:56.415760 17053 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:21:56.437125 16442 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:56.446722 17064 consensus_queue.cc:1048] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:56.451831 16328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:21:56.452589 16328 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 0.
I20260709 12:21:56.453504 16976 consensus_queue.cc:1048] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:56.452065 16859 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:56.472489 16367 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:56.484133 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:56.486935 16476 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 0.
I20260709 12:21:56.484205 16765 consensus_queue.cc:1048] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:21:56.490406 16291 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:56.485302 16976 consensus_queue.cc:1048] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:21:56.496368 17046 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:56.496773 17046 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:56.499995 17045 consensus_queue.cc:1048] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:56.498790 17046 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:56.484757 17061 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:56.512964 17050 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:21:56.513571 17050 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:56.513953 17050 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:21:56.598975 16424 debug-util.cc:398] Leaking SignalData structure 0x7b08002f3780 after lost signal to thread 5355
I20260709 12:21:56.602865 17050 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:21:56.603690 16424 debug-util.cc:398] Leaking SignalData structure 0x7b08002f3e60 after lost signal to thread 16260
W20260709 12:21:56.605135 16424 debug-util.cc:398] Leaking SignalData structure 0x7b08002b8880 after lost signal to thread 16352
W20260709 12:21:56.605465 16424 debug-util.cc:398] Leaking SignalData structure 0x7b08002cecc0 after lost signal to thread 16353
I20260709 12:21:56.605372 17046 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:21:56.606688 16424 debug-util.cc:398] Leaking SignalData structure 0x7b080020f140 after lost signal to thread 16427
W20260709 12:21:56.607105 16424 debug-util.cc:398] Leaking SignalData structure 0x7b0800264d20 after lost signal to thread 16428
I20260709 12:21:56.607623 17046 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:21:56.609838 16424 debug-util.cc:398] Leaking SignalData structure 0x7b08003e8520 after lost signal to thread 16501
W20260709 12:21:56.610170 16424 debug-util.cc:398] Leaking SignalData structure 0x7b08003dd8a0 after lost signal to thread 16502
I20260709 12:21:56.624449 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:21:56.625264 16474 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:21:56.627911 17050 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:56.628556 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:56.629727 16398 raft_consensus.cc:2393] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 1: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:21:56.647317 16320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:21:56.648510 16320 raft_consensus.cc:2393] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 1: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
W20260709 12:21:56.651764 16424 debug-util.cc:398] Leaking SignalData structure 0x7b08001f27c0 after lost signal to thread 16575
W20260709 12:21:56.652896 16424 debug-util.cc:398] Leaking SignalData structure 0x7b080034de60 after lost signal to thread 17012
I20260709 12:21:56.666107 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:21:56.701817 16366 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:21:56.705055 16694 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:21:56.676920 16474 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 1.
I20260709 12:21:56.716719 16765 consensus_queue.cc:1048] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:21:56.727532 16291 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:56.728529 17043 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:21:56.729077 17043 raft_consensus.cc:697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 1 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:21:56.730037 17043 consensus_queue.cc:237] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:56.722895 16981 consensus_queue.cc:1048] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:21:56.754930 17044 consensus_queue.cc:1048] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:21:56.755818 16223 catalog_manager.cc:5697] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 1 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:21:56.757380 16223 catalog_manager.cc:5697] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
I20260709 12:21:56.758678 16223 catalog_manager.cc:5697] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:21:56.763248 16226 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53612 (request call id 67) took 1959 ms. Trace:
I20260709 12:21:56.765736 16225 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53626 (request call id 68) took 1278 ms. Trace:
I20260709 12:21:56.766211 16225 rpcz_store.cc:276] 0709 12:21:55.487647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:55.495226 (+  7579us) service_pool.cc:224] Handling call
0709 12:21:56.765605 (+1270379us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":377733,"reported_tablets":3}
I20260709 12:21:56.770097 16224 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53622 (request call id 70) took 1409 ms. Trace:
I20260709 12:21:56.770628 16224 rpcz_store.cc:276] 0709 12:21:55.360662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:55.360902 (+   240us) service_pool.cc:224] Handling call
0709 12:21:56.769925 (+1409023us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":452565,"reported_tablets":6}
I20260709 12:21:56.763761 16226 rpcz_store.cc:276] 0709 12:21:54.806076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:54.806271 (+   195us) service_pool.cc:224] Handling call
0709 12:21:56.763197 (+1956926us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:21:56.144596 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:21:56.145149 (+   553us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:21:56.145185 (+    36us) write_op.cc:435] Acquired schema lock
0709 12:21:56.145204 (+    19us) tablet.cc:662] Decoding operations
0709 12:21:56.145552 (+   348us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:21:56.145629 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 12:21:56.145664 (+    35us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:21:56.145848 (+   184us) tablet.cc:701] Row locks acquired
0709 12:21:56.145866 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:21:56.146011 (+   145us) write_op.cc:270] Start()
0709 12:21:56.146114 (+   103us) write_op.cc:276] Timestamp: P: 1783599716145965 usec, L: 0
0709 12:21:56.146137 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:21:56.146840 (+   703us) log.cc:844] Serialized 531 byte log entry
0709 12:21:56.360947 (+214107us) spinlock_profiling.cc:243] Waited 2.08 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2872d7f0ac 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b
0709 12:21:56.632901 (+271954us) op_driver.cc:464] REPLICATION: finished
0709 12:21:56.722527 (+ 89626us) write_op.cc:301] APPLY: starting
0709 12:21:56.724232 (+  1705us) tablet.cc:1367] starting BulkCheckPresence
0709 12:21:56.725162 (+   930us) tablet.cc:1370] finished BulkCheckPresence
0709 12:21:56.725208 (+    46us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:21:56.726076 (+   868us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:21:56.726354 (+   278us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:21:56.726404 (+    50us) write_op.cc:312] APPLY: finished
0709 12:21:56.726599 (+   195us) log.cc:844] Serialized 22 byte log entry
0709 12:21:56.726785 (+   186us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:56.726884 (+    99us) write_op.cc:454] Released schema lock
0709 12:21:56.726925 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 12:21:56.727375 (+   450us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:56.727404 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":778317,"reported_tablets":10,"child_traces":[["op",{"apply.queue_time_us":87701,"apply.run_cpu_time_us":1926,"apply.run_wall_time_us":4783,"num_ops":1,"prepare.queue_time_us":26847,"prepare.run_cpu_time_us":26437,"prepare.run_wall_time_us":262815,"raft.queue_time_us":51365,"raft.run_cpu_time_us":61569,"raft.run_wall_time_us":81801,"replication_time_us":474945,"spinlock_wait_cycles":4794112,"thread_start_us":401931,"threads_started":4,"wal-append.queue_time_us":341885}]]}
I20260709 12:21:56.800781 17045 consensus_queue.cc:1048] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:56.759816 16223 catalog_manager.cc:5697] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 1 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:56.824225 16223 catalog_manager.cc:5697] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:21:56.829324 16223 catalog_manager.cc:5697] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 1 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:56.831123 16223 catalog_manager.cc:5697] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 1 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:56.885210 16226 catalog_manager.cc:5697] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:21:56.927443 16224 catalog_manager.cc:5697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:57.143106 16273 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:21:57.153654 17034 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:57.167805 17033 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:57.408351 17043 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:57.409063 17043 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:57.414023 17043 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:57.415267 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:57.415784 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:57.416074 16475 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 1.
I20260709 12:21:57.416934 16399 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 1.
I20260709 12:21:57.417953 16291 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:21:57.418900 17043 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:57.419312 17043 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:57.419695 17043 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:57.427054 17043 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:57.429077 17043 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:57.430202 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:21:57.430233 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:21:57.431113 16399 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:57.430823 16475 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:57.442679 16475 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:21:57.444164 16291 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:21:57.445528 17034 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:57.446102 17034 raft_consensus.cc:697] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:21:57.447320 16399 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:21:57.447113 17034 consensus_queue.cc:237] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:57.471956 16225 catalog_manager.cc:5697] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 0 to 2, leader changed from <none> to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 2 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:58.010371 16472 raft_consensus.cc:1275] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:21:58.011854 17034 consensus_queue.cc:1048] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:58.035116 16399 raft_consensus.cc:1275] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:21:58.042796 17043 consensus_queue.cc:1048] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:21:58.423789  5352 tablet_server.cc:179] TabletServer@127.5.58.1:0 shutting down...
W20260709 12:21:58.460431 16367 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:21:58.469278 16367 consensus_peers.cc:597] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:58.479239 16515 consensus_peers.cc:597] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:58.494935 16515 consensus_peers.cc:597] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:58.501632  5352 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:21:58.503744 16441 consensus_peers.cc:597] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:58.504714  5352 tablet_replica.cc:333] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.505504  5352 raft_consensus.cc:2243] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.506028  5352 raft_consensus.cc:2272] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.508548  5352 tablet_replica.cc:333] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.509788  5352 raft_consensus.cc:2243] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.510329  5352 raft_consensus.cc:2272] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:58.512346 16367 consensus_peers.cc:597] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:58.512470 16441 consensus_peers.cc:597] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:58.512795  5352 tablet_replica.cc:333] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.513553  5352 raft_consensus.cc:2243] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.513998  5352 raft_consensus.cc:2272] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.515774  5352 tablet_replica.cc:333] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.516526  5352 raft_consensus.cc:2243] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:58.517270  5352 raft_consensus.cc:2272] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.519574  5352 tablet_replica.cc:333] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.520392  5352 raft_consensus.cc:2243] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:21:58.524487  5352 raft_consensus.cc:2272] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.529969  5352 tablet_replica.cc:333] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.530927  5352 raft_consensus.cc:2243] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.531447  5352 raft_consensus.cc:2272] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.533916  5352 tablet_replica.cc:333] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.534585  5352 raft_consensus.cc:2243] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.535069  5352 raft_consensus.cc:2272] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.537891  5352 tablet_replica.cc:333] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.538694  5352 raft_consensus.cc:2243] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.539220  5352 raft_consensus.cc:2272] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.542322  5352 tablet_replica.cc:333] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.543252  5352 raft_consensus.cc:2243] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.543820  5352 raft_consensus.cc:2272] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.546701  5352 tablet_replica.cc:333] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.547595  5352 raft_consensus.cc:2243] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:21:58.548355 16441 consensus_peers.cc:597] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:58.548357  5352 raft_consensus.cc:2272] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:58.550081 16367 consensus_peers.cc:597] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:58.552419  5352 tablet_replica.cc:333] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.553418  5352 raft_consensus.cc:2243] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.554014  5352 raft_consensus.cc:2272] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:58.555526 16367 consensus_peers.cc:597] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:58.557168  5352 tablet_replica.cc:333] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.558033  5352 raft_consensus.cc:2243] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:58.559667  5352 raft_consensus.cc:2272] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.564079  5352 tablet_replica.cc:333] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.565147  5352 raft_consensus.cc:2243] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.565827  5352 raft_consensus.cc:2272] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:58.567620 16441 consensus_peers.cc:597] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:58.569037  5352 tablet_replica.cc:333] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.570145  5352 raft_consensus.cc:2243] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.571002  5352 raft_consensus.cc:2272] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.575466  5352 tablet_replica.cc:333] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.576277  5352 raft_consensus.cc:2243] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.576902  5352 raft_consensus.cc:2272] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.580134  5352 tablet_replica.cc:333] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
W20260709 12:21:58.580835 16441 consensus_peers.cc:597] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:58.580982  5352 raft_consensus.cc:2243] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:58.582281  5352 raft_consensus.cc:2272] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.585155  5352 tablet_replica.cc:333] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.586081  5352 raft_consensus.cc:2243] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:58.587164  5352 raft_consensus.cc:2272] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.590032  5352 tablet_replica.cc:333] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.590925  5352 raft_consensus.cc:2243] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.591553  5352 raft_consensus.cc:2272] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.594610  5352 tablet_replica.cc:333] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.595433  5352 raft_consensus.cc:2243] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:58.596556  5352 raft_consensus.cc:2272] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.601007  5352 tablet_replica.cc:333] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.602285  5352 raft_consensus.cc:2243] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:58.603807  5352 raft_consensus.cc:2272] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.607280  5352 tablet_replica.cc:333] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.609761  5352 raft_consensus.cc:2243] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:58.611030  5352 raft_consensus.cc:2272] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.615871  5352 tablet_replica.cc:333] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.617304  5352 raft_consensus.cc:2243] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.617918  5352 raft_consensus.cc:2272] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:58.619369 16515 consensus_peers.cc:597] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:58.620900  5352 tablet_replica.cc:333] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.621580  5352 raft_consensus.cc:2243] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.622159  5352 raft_consensus.cc:2272] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.625025  5352 tablet_replica.cc:333] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.625835  5352 raft_consensus.cc:2243] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.626425  5352 raft_consensus.cc:2272] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.629357  5352 tablet_replica.cc:333] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.630146  5352 raft_consensus.cc:2243] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:58.631603  5352 raft_consensus.cc:2272] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.633980  5352 tablet_replica.cc:333] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.634878  5352 raft_consensus.cc:2243] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.635540  5352 raft_consensus.cc:2272] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:58.636399 16441 consensus_peers.cc:597] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:58.638487  5352 tablet_replica.cc:333] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
W20260709 12:21:58.639230 16441 consensus_peers.cc:597] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:58.639550  5352 raft_consensus.cc:2243] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.640205  5352 raft_consensus.cc:2272] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.643054  5352 tablet_replica.cc:333] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.643693  5352 raft_consensus.cc:2243] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:58.644658  5352 raft_consensus.cc:2272] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.649205  5352 tablet_replica.cc:333] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.650509  5352 raft_consensus.cc:2243] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.651808  5352 raft_consensus.cc:2272] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:58.653728 16515 consensus_peers.cc:597] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:58.656026  5352 tablet_replica.cc:333] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.657325  5352 raft_consensus.cc:2243] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.658092  5352 raft_consensus.cc:2272] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.663741  5352 tablet_replica.cc:333] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.664562  5352 raft_consensus.cc:2243] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.665035  5352 raft_consensus.cc:2272] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.668105  5352 tablet_replica.cc:333] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
W20260709 12:21:58.669184 16367 consensus_peers.cc:597] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:58.669265  5352 raft_consensus.cc:2243] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:58.670751  5352 raft_consensus.cc:2272] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.675565  5352 tablet_replica.cc:333] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.676429  5352 raft_consensus.cc:2243] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:58.677474  5352 raft_consensus.cc:2272] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.680372  5352 tablet_replica.cc:333] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.681164  5352 raft_consensus.cc:2243] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:58.682310  5352 raft_consensus.cc:2272] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.685228  5352 tablet_replica.cc:333] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.685833  5352 raft_consensus.cc:2243] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.686427  5352 raft_consensus.cc:2272] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.689046  5352 tablet_replica.cc:333] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.689654  5352 raft_consensus.cc:2243] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.690163  5352 raft_consensus.cc:2272] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:58.692150 16367 consensus_peers.cc:597] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:58.693403  5352 tablet_replica.cc:333] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.694407  5352 raft_consensus.cc:2243] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:58.695271  5352 raft_consensus.cc:2272] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.699424  5352 tablet_replica.cc:333] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
W20260709 12:21:58.699733 16515 consensus_peers.cc:597] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:58.701897  5352 raft_consensus.cc:2243] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.702664  5352 raft_consensus.cc:2272] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.706008  5352 tablet_replica.cc:333] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
W20260709 12:21:58.706336 16367 consensus_peers.cc:597] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:58.706782 16441 consensus_peers.cc:597] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:58.706841  5352 raft_consensus.cc:2243] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.707685  5352 raft_consensus.cc:2272] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:58.710731 16515 consensus_peers.cc:597] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:58.710944  5352 tablet_replica.cc:333] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.711824  5352 raft_consensus.cc:2243] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.712491  5352 raft_consensus.cc:2272] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.715451  5352 tablet_replica.cc:333] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.716213  5352 raft_consensus.cc:2243] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.716820  5352 raft_consensus.cc:2272] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:58.718214 16515 consensus_peers.cc:597] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:58.719730  5352 tablet_replica.cc:333] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.720546  5352 raft_consensus.cc:2243] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.721134  5352 raft_consensus.cc:2272] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.724164  5352 tablet_replica.cc:333] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.724982  5352 raft_consensus.cc:2243] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:58.725947  5352 raft_consensus.cc:2272] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:21:58.726974 16367 consensus_peers.cc:597] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:58.728919  5352 tablet_replica.cc:333] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.729919  5352 raft_consensus.cc:2243] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:58.730508  5352 raft_consensus.cc:2272] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.733096  5352 tablet_replica.cc:333] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8: stopping tablet replica
I20260709 12:21:58.733820  5352 raft_consensus.cc:2243] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:21:58.734681  5352 raft_consensus.cc:2272] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:58.817699  5352 tablet_server.cc:196] TabletServer@127.5.58.1:0 shutdown complete.
W20260709 12:21:58.836937 16441 consensus_peers.cc:597] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:58.844585 16367 consensus_peers.cc:597] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:58.852561 16441 consensus_peers.cc:597] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:58.891819 16441 consensus_peers.cc:597] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:58.907424 16515 consensus_peers.cc:597] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:58.941721  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:21:59.081931 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 60417us
W20260709 12:21:59.102837 17126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:59.111286 16441 consensus_peers.cc:597] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:59.121433 17128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:21:59.152875 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 55123.7us
I20260709 12:21:59.178638  5352 server_base.cc:1061] running on GCE node
W20260709 12:21:59.194509 17136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:21:59.204308  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:21:59.204979  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:21:59.205286  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599719205264 us; error 0 us; skew 500 ppm
I20260709 12:21:59.206190  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:21:59.256790  5352 webserver.cc:533] Webserver started at http://127.5.58.1:43597/ using document root <none> and password file <none>
I20260709 12:21:59.257551  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:21:59.257829  5352 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:21:59.277522  5352 fs_manager.cc:714] Time spent opening directory manager: real 0.018s	user 0.006s	sys 0.008s
I20260709 12:21:59.290493 17141 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:21:59.292744  5352 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.003s	sys 0.001s
I20260709 12:21:59.293573  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599341860723-5352-0/minicluster-data/ts-0-root
uuid: "ef57beab0e1d40d585983873c55107e8"
format_stamp: "Formatted at 2026-07-09 12:21:45 on dist-test-slave-xs2d"
I20260709 12:21:59.294694  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599341860723-5352-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599341860723-5352-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599341860723-5352-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:21:59.338438  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:21:59.340102  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:21:59.359179  5352 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:21:59.365943 17148 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
I20260709 12:21:59.521961 17121 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:21:59.522572 17121 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:59.525027 17121 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:59.526374 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:21:59.532203 16441 leader_election.cc:336] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:21:59.532876 16441 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:59.533835 17111 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:59.546505 17129 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:21:59.547340 16981 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:21:59.547089 17129 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:59.547976 16981 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:59.550995 17131 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:21:59.551661 17129 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:59.551661 17131 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:59.559146 17131 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:21:59.563462 16515 leader_election.cc:336] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:21:59.563838 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:59.565976 16516 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:59.568961 17131 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:59.569514 16981 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:59.571857 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:21:59.572851 16515 leader_election.cc:336] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:21:59.573740 16516 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:59.577046 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:59.577857 16472 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
I20260709 12:21:59.579203 17131 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:21:59.580384 17131 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:59.579780 16981 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:59.584165 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:21:59.584848 16367 leader_election.cc:336] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:21:59.588809 16366 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:21:59.590055 17134 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:59.590413 17134 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:21:59.590754 17134 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:59.593295 17131 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:59.598282 17121 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:21:59.598948 17121 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:21:59.600028 16515 leader_election.cc:336] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:21:59.600648 16515 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:59.603586 17131 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:59.605922 17121 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:59.611649 17134 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:59.623008 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:59.626922 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:21:59.627671 16472 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:59.633914 17129 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:21:59.634855 17129 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:59.637641 17129 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:21:59.648289 16441 leader_election.cc:336] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:21:59.648810 16441 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:59.646219 17134 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:21:59.653555 16367 leader_election.cc:336] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:21:59.654700 17127 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:59.660964 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:59.661643 16475 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:21:59.669771 16472 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 3.
I20260709 12:21:59.670037 16366 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:59.682803 17129 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:59.683261 17129 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:21:59.683645 17129 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:21:59.687542 16367 leader_election.cc:336] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:21:59.696039 16366 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:21:59.697108 17134 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:59.697693 17134 raft_consensus.cc:697] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 3 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:21:59.698724 17134 consensus_queue.cc:237] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:59.708155 17129 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:59.715425 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:21:59.716027 16475 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:59.716012 16981 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:21:59.720670 16981 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:59.723084 16981 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:59.724594 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:59.735998 17127 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:21:59.736534 17127 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:59.738919 17127 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:59.740298  5352 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 12:21:59.740713  5352 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.377s	user 0.010s	sys 0.000s
I20260709 12:21:59.741072  5352 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
W20260709 12:21:59.743860 16515 leader_election.cc:336] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:21:59.744346 16515 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:21:59.749177 16981 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:59.747570 16475 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
I20260709 12:21:59.751037 17129 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:59.752710 17148 tablet_bootstrap.cc:492] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:59.753360 16366 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:59.755124 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:59.754092 16225 catalog_manager.cc:5697] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 2 to 3, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 3 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:59.757552 16441 leader_election.cc:336] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:21:59.759727 16442 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:59.760998 17121 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:21:59.762780 16367 leader_election.cc:336] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:21:59.762844 17135 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:21:59.763537 17135 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:59.766446 17134 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:59.767076 17134 raft_consensus.cc:697] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:21:59.765934 17135 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:59.768392 16981 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:21:59.768399 17134 consensus_queue.cc:237] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:59.769148 16981 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:59.771013 17127 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:21:59.771492 17127 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:59.772650 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:59.774137 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:59.774770 16548 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:21:59.775327 16981 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:59.777436 17135 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:21:59.778975 16367 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:59.777951 17135 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:59.780047 17130 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:59.780575 17130 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:21:59.783003 17130 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:59.788038 17135 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:59.788841 17127 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:59.789373 17127 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:21:59.789820 17127 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:59.800252 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:59.815419 17130 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:59.819911 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:59.821142 16548 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:21:59.826082 17154 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:21:59.827162 17154 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:59.829463 17129 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:21:59.830122 17129 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:59.831019 17154 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:59.837224 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:21:59.840605 17130 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:59.833169 17127 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:59.832309 17129 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:21:59.847076 16441 leader_election.cc:336] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:21:59.847599 16441 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:21:59.850435 16515 leader_election.cc:336] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:21:59.851183 16515 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:21:59.852275 16981 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:59.854151 17148 tablet_bootstrap.cc:492] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:59.855225 17148 tablet_bootstrap.cc:492] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:21:59.855834 17148 ts_tablet_manager.cc:1403] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.103s	user 0.017s	sys 0.012s
I20260709 12:21:59.858484 17148 raft_consensus.cc:359] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:59.859187 17148 raft_consensus.cc:740] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:59.860509 17127 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:21:59.863047 16367 leader_election.cc:336] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:21:59.868800 16441 leader_election.cc:336] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:21:59.859958 17148 consensus_queue.cc:260] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:59.871481 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:59.872246 16548 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:21:59.877714 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:59.878373 16548 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:59.879673 17148 ts_tablet_manager.cc:1434] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.024s	user 0.012s	sys 0.000s
I20260709 12:21:59.880580 17148 tablet_bootstrap.cc:492] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:59.890362 16548 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
I20260709 12:21:59.897935 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:21:59.900305 16367 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:59.901593 17155 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:59.902431 17155 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:21:59.901657 16367 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:59.903266 17155 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:59.904131 17129 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:59.904544 17129 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:21:59.904616 16367 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:21:59.904919 17129 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:59.905925 17130 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:59.906502 17130 raft_consensus.cc:697] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:21:59.907444 17130 consensus_queue.cc:237] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:59.909380 17127 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:21:59.909938 17127 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:21:59.911568 16367 leader_election.cc:336] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:21:59.912923 17148 tablet_bootstrap.cc:492] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:59.915732 17148 tablet_bootstrap.cc:492] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:21:59.913880 17127 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:59.926862 17129 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:59.929623 17155 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:21:59.931838 17155 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:59.935302 17129 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:59.936198 17148 ts_tablet_manager.cc:1403] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.056s	user 0.019s	sys 0.009s
I20260709 12:21:59.936527 16442 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:21:59.939633 17111 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:59.940299 17111 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:21:59.940773 17111 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:59.941409 17148 raft_consensus.cc:359] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:59.942142 17148 raft_consensus.cc:740] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:21:59.942901 17148 consensus_queue.cc:260] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:59.943368 17111 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:21:59.945782 17148 ts_tablet_manager.cc:1434] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:21:59.946339 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:21:59.947065 16544 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:21:59.949944 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:59.950554 16544 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:59.951020 17148 tablet_bootstrap.cc:492] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:21:59.962453 16544 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
I20260709 12:21:59.969321 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:21:59.969877 17129 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:21:59.970141 16548 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:59.970503 17129 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:21:59.973037 17129 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:21:59.982096 17127 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:21:59.982721 17127 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:21:59.984777 17127 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:21:59.985016 16548 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
W20260709 12:21:59.992216 16367 leader_election.cc:336] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:21:59.996441 16367 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:00.000674 17129 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:00.000878 16367 leader_election.cc:336] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:22:00.006179 16367 leader_election.cc:336] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:22:00.007077 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:00.009514 17156 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:00.008319 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:00.010324 17156 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:00.013087 17129 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:00.013649 17129 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:00.015910 17129 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:00.019696 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:00.025309 17127 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:00.026029 17127 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:00.027113 17111 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:00.027527 17111 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:00.027887 17111 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:00.028573 17127 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:00.029549 17130 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:00.030279 17130 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:00.030663 17155 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:00.031334 17155 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:00.032634 17130 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:00.033178 17121 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:00.033852 17121 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:00.038480 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:00.039258 16475 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:22:00.042579 16514 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:00.044159 16367 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:00.049228 16367 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:00.048322 17155 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:00.053213 17131 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:00.054585 17130 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:00.055060 17131 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:00.055564 17131 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:00.058923 17111 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:00.059396 16515 leader_election.cc:336] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:22:00.055367 17130 raft_consensus.cc:697] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:00.060942 17130 consensus_queue.cc:237] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:00.063901 16225 catalog_manager.cc:5697] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 1 to 2, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 2 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:00.068043 16225 catalog_manager.cc:5697] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 1 to 2, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 2 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:00.065819 17157 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:00.072546 17157 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:00.048493 17121 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:00.053900 17129 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:00.082226 17129 raft_consensus.cc:697] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:00.083104 17129 consensus_queue.cc:237] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:00.048313 17156 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:00.084822 16367 leader_election.cc:336] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:22:00.086016 16367 leader_election.cc:336] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:22:00.086068 17157 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:00.077926 17164 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:00.080008 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:00.092021 16401 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:22:00.091745 17164 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:00.098158 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:00.100445 17148 tablet_bootstrap.cc:492] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:00.100574 16442 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:00.101562 17148 tablet_bootstrap.cc:492] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:00.102317 17148 ts_tablet_manager.cc:1403] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.152s	user 0.046s	sys 0.048s
I20260709 12:22:00.106720 17111 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:00.106992 17148 raft_consensus.cc:359] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:00.107388 17111 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:00.107672 17148 raft_consensus.cc:740] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:00.107743 17111 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:00.108608 17148 consensus_queue.cc:260] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:00.111131 16367 leader_election.cc:336] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:22:00.113581 17164 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:22:00.115129 16367 leader_election.cc:336] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:22:00.116225 17148 ts_tablet_manager.cc:1434] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
W20260709 12:22:00.116812 16367 leader_election.cc:336] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:22:00.117475 17148 tablet_bootstrap.cc:492] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
W20260709 12:22:00.118153 16367 leader_election.cc:336] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:22:00.108186 17111 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:22:00.135040 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 56705.9us
I20260709 12:22:00.135066 17156 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:00.136579 16963 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:22:00.142683 16441 leader_election.cc:336] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:22:00.144682 16441 leader_election.cc:336] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:22:00.145027 16441 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:00.145849 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:00.146582 16544 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:22:00.147020 16963 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:00.149392 16963 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:00.151705 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:00.140008 17156 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:00.152484 16401 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:22:00.153895 16516 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:00.154083 17156 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:00.154242 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:00.154994 16398 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:22:00.155207 17156 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:00.155701 17156 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:00.156100 17156 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:22:00.157541 16367 leader_election.cc:336] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:22:00.157961 16367 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:00.159549 17157 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:00.159963 16442 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:00.135361 17131 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:00.170080 16225 catalog_manager.cc:5697] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 1 to 2, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 2 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:00.170948 17165 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:00.172005 17165 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:00.172704 17165 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:00.173241 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:00.173967 16471 raft_consensus.cc:2468] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:22:00.174768 17121 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:00.175078 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:00.175290 17121 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:00.175832 16472 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
W20260709 12:22:00.175848 16441 leader_election.cc:336] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:22:00.177958 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:00.178658 16473 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:22:00.179243 17131 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:00.180347 16469 raft_consensus.cc:1275] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:00.176960 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:00.181774 16366 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:22:00.183068 16441 leader_election.cc:336] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:22:00.183295 16366 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
W20260709 12:22:00.183616 16441 leader_election.cc:336] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:22:00.184072 17155 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:00.184105 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:22:00.184504 16441 leader_election.cc:336] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:22:00.184938 16475 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:22:00.185042 16441 leader_election.cc:336] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:22:00.186213 17130 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:00.186705 17130 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:00.187094 17130 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:00.187484 17168 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:00.183302 16514 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:00.189435 16981 raft_consensus.cc:2804] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:00.189853 16981 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:00.190254 16981 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:00.195433 16366 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:00.197077 17155 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:00.197471 17155 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:00.197863 17155 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:00.171977 16225 catalog_manager.cc:5697] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 1 to 2, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 2 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:00.211710 17121 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:00.212505 16441 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:00.213143 17111 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:00.216284 17111 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:00.217022 17134 consensus_queue.cc:1048] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:00.218637 16474 raft_consensus.cc:2393] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 2: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:00.219179 17154 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:00.219687 17154 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:00.221683 17154 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:00.223768 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:00.227451 16441 leader_election.cc:336] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:22:00.227999 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:00.228801 16474 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:22:00.230836 16981 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:00.232033 16548 raft_consensus.cc:2393] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 2: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:00.236863 16366 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:00.235860 16475 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:00.243867 16441 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:00.239738 16366 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:00.222332 17156 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:00.240620 16514 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
W20260709 12:22:00.248827 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 88712.6us
I20260709 12:22:00.232553 17111 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:00.251029 17111 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:00.253396 17111 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:00.253901 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:00.247305 17157 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:00.256382 16981 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:00.231773 17130 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:00.266736 17130 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:00.251449 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:00.268973 16544 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:00.250643 17164 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:00.274389 17157 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:00.267159  5352 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 12:22:00.275275  5352 ts_tablet_manager.cc:595] Time spent register tablets: real 0.534s	user 0.341s	sys 0.003s
I20260709 12:22:00.258494 17165 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:00.259961 17129 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:22:00.261204 16441 leader_election.cc:336] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:22:00.254930 16963 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:00.264348 17155 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:00.286737 17155 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:00.289947 17148 tablet_bootstrap.cc:492] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:00.282591 17129 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:00.292827 17129 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:00.293001 17148 tablet_bootstrap.cc:492] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:00.293728 17148 ts_tablet_manager.cc:1403] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.177s	user 0.021s	sys 0.000s
I20260709 12:22:00.296319 17148 raft_consensus.cc:359] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:00.297021 16963 raft_consensus.cc:697] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 2 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:00.297016 17148 raft_consensus.cc:740] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:00.298309 16963 consensus_queue.cc:237] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:00.299703 17148 consensus_queue.cc:260] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:00.301753 17165 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:00.304095 17148 ts_tablet_manager.cc:1434] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:22:00.305299 16544 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:00.305737 17148 tablet_bootstrap.cc:492] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:00.276973 17156 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:00.306073 17164 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:00.307847 17129 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:00.309564 16441 leader_election.cc:336] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:22:00.310063 17129 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:22:00.315507 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 67319us
W20260709 12:22:00.316668 16367 consensus_peers.cc:597] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:00.320631 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:00.321364 16398 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:22:00.321502 16475 raft_consensus.cc:2468] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
W20260709 12:22:00.323891 16515 leader_election.cc:336] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:22:00.325013 16515 leader_election.cc:336] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:22:00.325706 16515 leader_election.cc:336] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:22:00.328830 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:22:00.330134 16367 leader_election.cc:336] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:22:00.333387 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:00.334697 17165 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:00.335235 17165 raft_consensus.cc:697] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:22:00.338522 16515 leader_election.cc:336] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:22:00.338610 16398 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
W20260709 12:22:00.339425 16515 leader_election.cc:336] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:22:00.342845 16225 catalog_manager.cc:5697] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 1 to 2, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 2 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
W20260709 12:22:00.343732 16367 leader_election.cc:336] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:22:00.336233 17165 consensus_queue.cc:237] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:00.345033 17111 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:00.347332 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 107694us
W20260709 12:22:00.348241 16367 consensus_peers.cc:597] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:00.348712 16367 leader_election.cc:336] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:22:00.349118 16367 leader_election.cc:336] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:22:00.353241 16514 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:00.354768 17156 raft_consensus.cc:2804] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:00.355540 17156 raft_consensus.cc:697] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 2 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:00.356530 17156 consensus_queue.cc:237] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:00.358268 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:00.358907 16398 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:00.370788 16550 raft_consensus.cc:1275] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:00.372025 16398 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:00.373835 17178 consensus_queue.cc:1048] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:00.378963 16442 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
W20260709 12:22:00.377527 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 44233.5us
I20260709 12:22:00.384814 17176 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:00.385290 17176 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:00.385679 17176 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:00.399056 17176 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:00.401146 17111 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:00.401508 17176 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:00.409276 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:00.410142 16475 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:00.410789 16442 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:00.414289 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:00.415496 16469 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:00.416657 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:00.417392 16403 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:00.417649 16516 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:00.418506 17156 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:00.419013 17156 raft_consensus.cc:697] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 2 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:00.419880 17156 consensus_queue.cc:237] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:00.416051 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:00.420995 16397 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:00.429452 17165 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:00.430094 17165 raft_consensus.cc:697] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:00.431002 17165 consensus_queue.cc:237] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:00.433933 16469 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
I20260709 12:22:00.434515 16397 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:00.436298 16403 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:00.444664 16475 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
W20260709 12:22:00.454989 16367 consensus_peers.cc:597] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:00.447165 17181 log.cc:927] Time spent T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.085s	user 0.009s	sys 0.025s
I20260709 12:22:00.455758 17148 tablet_bootstrap.cc:492] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:22:00.456574 17179 log.cc:927] Time spent T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.139s	user 0.002s	sys 0.013s
W20260709 12:22:00.457983 17182 log.cc:927] Time spent T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.082s	user 0.000s	sys 0.008s
W20260709 12:22:00.460747 17183 log.cc:927] Time spent T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.057s	user 0.003s	sys 0.000s
I20260709 12:22:00.446168 16442 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:00.467381 16442 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:00.466157 17180 log.cc:927] Time spent T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.146s	user 0.000s	sys 0.015s
I20260709 12:22:00.466235 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:00.476917 16474 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:00.477052 17172 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:00.477842 17172 raft_consensus.cc:697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:00.478940 17172 consensus_queue.cc:237] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:00.485630 17148 tablet_bootstrap.cc:492] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:00.486415 17148 ts_tablet_manager.cc:1403] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.181s	user 0.017s	sys 0.002s
I20260709 12:22:00.487507 16475 raft_consensus.cc:1275] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:00.475131 17111 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:00.491560 17111 raft_consensus.cc:697] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:00.493237 17111 consensus_queue.cc:237] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:00.507905 16366 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:00.509232 16366 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:00.510293 17129 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:22:00.510686 17151 log.cc:927] Time spent T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a: Append to log took a long time: real 0.117s	user 0.000s	sys 0.013s
I20260709 12:22:00.510881 17129 raft_consensus.cc:697] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:00.511819 17129 consensus_queue.cc:237] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:00.519840 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:00.524192 16544 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:00.523434 17175 consensus_queue.cc:1048] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:22:00.531776 16474 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
I20260709 12:22:00.533325 17155 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:00.533870 17155 raft_consensus.cc:697] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:00.534782 17155 consensus_queue.cc:237] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:00.536620 17148 raft_consensus.cc:359] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:00.538070 17148 raft_consensus.cc:740] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
W20260709 12:22:00.540319 16441 leader_election.cc:336] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:22:00.541554 16441 leader_election.cc:336] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:22:00.539053 17148 consensus_queue.cc:260] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:00.555148 16366 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:00.556778 17148 ts_tablet_manager.cc:1434] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.070s	user 0.008s	sys 0.000s
I20260709 12:22:00.564776 17148 tablet_bootstrap.cc:492] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:00.560938 17134 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:00.587057 17134 raft_consensus.cc:697] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:00.635321 16226 catalog_manager.cc:5697] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 1 to 2, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 2 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:00.639192 16226 catalog_manager.cc:5697] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 1 to 2, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 2 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:00.642482 16224 catalog_manager.cc:5697] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 1 to 2, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 2 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:00.682356 17134 consensus_queue.cc:237] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:00.685961 17191 log.cc:927] Time spent T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:22:00.682420 16550 raft_consensus.cc:1275] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:00.704157 16548 raft_consensus.cc:1275] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:00.690726 16544 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:00.801090 17148 tablet_bootstrap.cc:492] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:00.809257 17148 tablet_bootstrap.cc:492] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:00.817309 17148 ts_tablet_manager.cc:1403] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.253s	user 0.020s	sys 0.029s
I20260709 12:22:00.826463 16225 catalog_manager.cc:5697] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 1 to 2, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 2 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:00.828917 16441 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:00.828603 16225 catalog_manager.cc:5697] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 1 to 2, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 2 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:00.829808 17127 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:00.830459 17127 raft_consensus.cc:697] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:00.831377 17127 consensus_queue.cc:237] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:00.841379 16403 raft_consensus.cc:1275] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:00.830916 16225 catalog_manager.cc:5697] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 1 to 2, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 2 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:00.843065 17156 consensus_queue.cc:1048] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:00.841948 17148 raft_consensus.cc:359] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:00.850558 17148 raft_consensus.cc:740] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:00.852731 17148 consensus_queue.cc:260] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:00.859107 17148 ts_tablet_manager.cc:1434] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.041s	user 0.005s	sys 0.002s
I20260709 12:22:00.853266 16225 catalog_manager.cc:5697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 1 to 2, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 2 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
W20260709 12:22:00.856905 16515 consensus_peers.cc:597] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:00.860369 17148 tablet_bootstrap.cc:492] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
W20260709 12:22:00.891217 16515 consensus_peers.cc:597] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:00.898752 16515 consensus_peers.cc:597] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:00.905581 16475 raft_consensus.cc:1275] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:00.907574 16475 raft_consensus.cc:1275] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:00.907322 17171 consensus_queue.cc:1048] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:00.909142 16981 consensus_queue.cc:1048] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:22:00.914456 16515 consensus_peers.cc:597] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:00.948266 16403 raft_consensus.cc:1275] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:00.949543 16544 raft_consensus.cc:1275] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:00.961817 16402 raft_consensus.cc:1275] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:00.963096 17167 consensus_queue.cc:1048] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:00.966048 17178 consensus_queue.cc:1048] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:00.966764 17154 consensus_queue.cc:1048] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:00.990114 17165 consensus_queue.cc:1048] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:01.025142 17148 tablet_bootstrap.cc:492] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:22:01.026480 16367 consensus_peers.cc:597] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:01.030485 16367 consensus_peers.cc:597] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:01.033457 16226 catalog_manager.cc:5697] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 1 to 2, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 2 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:01.022449 17165 consensus_queue.cc:1048] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 12:22:01.045966 16226 catalog_manager.cc:5697] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 1 to 2, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 2 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
W20260709 12:22:01.052281 16515 consensus_peers.cc:597] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:01.053735 17148 tablet_bootstrap.cc:492] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:01.055359 17148 ts_tablet_manager.cc:1403] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.195s	user 0.054s	sys 0.023s
W20260709 12:22:01.069003 16367 consensus_peers.cc:597] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:01.098613 17148 raft_consensus.cc:359] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:01.118760 17148 raft_consensus.cc:740] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
W20260709 12:22:01.120410 16367 consensus_peers.cc:597] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:01.128434 16367 consensus_peers.cc:597] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:01.125787 17148 consensus_queue.cc:260] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:01.120671 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 126906us
W20260709 12:22:01.130023 16441 consensus_peers.cc:597] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:01.133190 17148 ts_tablet_manager.cc:1434] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.077s	user 0.008s	sys 0.004s
I20260709 12:22:01.134085 17148 tablet_bootstrap.cc:492] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:01.147164 16402 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:01.149626 16475 raft_consensus.cc:1275] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:01.151070 16474 raft_consensus.cc:1275] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:01.162349 16225 catalog_manager.cc:5697] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 1 to 2, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 2 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:01.160923 17148 tablet_bootstrap.cc:492] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:01.188144 17218 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:01.199297 17148 tablet_bootstrap.cc:492] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:01.200470 17148 ts_tablet_manager.cc:1403] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.067s	user 0.020s	sys 0.006s
I20260709 12:22:01.169309 17154 consensus_queue.cc:1048] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 12:22:01.182801 17135 consensus_queue.cc:1048] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
W20260709 12:22:01.235123 16441 consensus_peers.cc:597] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:01.239408 17148 raft_consensus.cc:359] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:01.252707 17148 raft_consensus.cc:740] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
W20260709 12:22:01.241731 16367 consensus_peers.cc:597] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:01.254281 16367 consensus_peers.cc:597] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:01.254966 16367 consensus_peers.cc:597] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:01.277801 17148 consensus_queue.cc:260] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:01.290170 17148 ts_tablet_manager.cc:1434] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.089s	user 0.008s	sys 0.000s
W20260709 12:22:01.296505 16441 consensus_peers.cc:597] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:01.297432 16441 consensus_peers.cc:597] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:01.297938 16441 consensus_peers.cc:597] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:01.298454 16441 consensus_peers.cc:597] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:01.298949 16441 consensus_peers.cc:597] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:01.316567 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 148517us
I20260709 12:22:01.325207 17148 tablet_bootstrap.cc:492] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
W20260709 12:22:01.359915 16441 consensus_peers.cc:597] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:01.363437 16473 raft_consensus.cc:1275] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:22:01.383095 16515 consensus_peers.cc:597] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:01.388861 16441 consensus_peers.cc:597] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:01.392249 16515 consensus_peers.cc:597] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:01.393035 16515 consensus_peers.cc:597] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:01.398298 16515 consensus_peers.cc:597] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:01.395295 17157 consensus_queue.cc:1048] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
W20260709 12:22:01.439726 16367 consensus_peers.cc:597] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:01.440810 16367 consensus_peers.cc:597] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:01.458889 17148 tablet_bootstrap.cc:492] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:01.460032 17148 tablet_bootstrap.cc:492] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:01.460767 17148 ts_tablet_manager.cc:1403] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.136s	user 0.019s	sys 0.008s
I20260709 12:22:01.466339 17148 raft_consensus.cc:359] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:01.467149 17148 raft_consensus.cc:740] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:01.468140 17148 consensus_queue.cc:260] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:01.471750 17148 ts_tablet_manager.cc:1434] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 12:22:01.475502 16544 raft_consensus.cc:1275] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:01.480230 17148 tablet_bootstrap.cc:492] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
W20260709 12:22:01.490809 16441 consensus_peers.cc:597] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:01.491722 16441 consensus_peers.cc:597] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:01.494040 16367 consensus_peers.cc:597] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:01.497848 16515 consensus_peers.cc:597] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:01.494817 16367 consensus_peers.cc:597] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:01.503085 16367 consensus_peers.cc:597] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:01.530503 17218 consensus_queue.cc:1048] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:22:01.540583 16441 consensus_peers.cc:597] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:22:01.550155 16367 consensus_peers.cc:597] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:01.562098 16441 consensus_peers.cc:597] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:01.586683 16441 consensus_peers.cc:597] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:01.592294 17160 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:01.593103 17160 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:01.601634 17160 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:01.658553 17148 tablet_bootstrap.cc:492] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:01.687714 17148 tablet_bootstrap.cc:492] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:01.688894 17148 ts_tablet_manager.cc:1403] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.209s	user 0.019s	sys 0.008s
I20260709 12:22:01.720715 17148 raft_consensus.cc:359] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:01.721501 17148 raft_consensus.cc:740] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
W20260709 12:22:01.731849 16441 consensus_peers.cc:597] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:22:01.747236 16441 consensus_peers.cc:597] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:01.772166 17160 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:01.772892 17160 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:01.776154 17160 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:01.722275 17148 consensus_queue.cc:260] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:01.801564 17160 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:01.823949 17160 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:01.826382 17160 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:01.831130 17160 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:01.831864 17160 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:01.845419 16441 consensus_peers.cc:597] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:01.868790 17148 ts_tablet_manager.cc:1434] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.179s	user 0.042s	sys 0.023s
I20260709 12:22:01.869788 17148 tablet_bootstrap.cc:492] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:01.872052 17160 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:01.872668 17160 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:01.873103 17160 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:01.898257 17160 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:22:01.950050 16515 consensus_peers.cc:597] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:01.963382 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:01.986146 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:02.007061 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:02.008783 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:02.008843 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:02.012741 17143 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:02.013696 17160 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:02.008675 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:02.044133 17148 tablet_bootstrap.cc:492] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:02.045321 17148 tablet_bootstrap.cc:492] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:02.046063 17148 ts_tablet_manager.cc:1403] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.176s	user 0.015s	sys 0.007s
I20260709 12:22:02.049181 17148 raft_consensus.cc:359] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:02.049973 17148 raft_consensus.cc:740] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:02.051192 17148 consensus_queue.cc:260] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:02.054520 17148 ts_tablet_manager.cc:1434] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:22:02.055359 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:02.056748 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:02.056115 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:02.056615 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:02.060500 17148 tablet_bootstrap.cc:492] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
W20260709 12:22:02.188277 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 125399us
W20260709 12:22:02.197062 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 125950us
I20260709 12:22:02.201097 17144 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:02.202373 17160 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:02.203545 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 140144us
I20260709 12:22:02.204638 17144 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:02.226905 17160 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:02.262001 17144 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:02.265928 17160 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:02.274327 17148 tablet_bootstrap.cc:492] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:02.275477 17148 tablet_bootstrap.cc:492] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:02.276106 17148 ts_tablet_manager.cc:1403] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.216s	user 0.032s	sys 0.004s
I20260709 12:22:02.278155 17144 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:02.279100 17160 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:02.279156 17148 raft_consensus.cc:359] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:02.280357 17148 raft_consensus.cc:740] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:02.281688 17148 consensus_queue.cc:260] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:02.284552 17148 ts_tablet_manager.cc:1434] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:22:02.285585 17148 tablet_bootstrap.cc:492] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:02.301506 17160 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:02.302153 17160 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:02.305706 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:02.307691 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:02.310886 17142 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:02.312389 17260 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:02.323419 17160 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:02.380067 17148 tablet_bootstrap.cc:492] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:02.382030 17148 tablet_bootstrap.cc:492] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:02.382905 17148 ts_tablet_manager.cc:1403] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.098s	user 0.029s	sys 0.013s
I20260709 12:22:02.386384 17148 raft_consensus.cc:359] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:02.402783 17148 raft_consensus.cc:740] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:02.403920 17148 consensus_queue.cc:260] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:02.406526 17148 ts_tablet_manager.cc:1434] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 12:22:02.407343 17148 tablet_bootstrap.cc:492] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:02.455631 17148 tablet_bootstrap.cc:492] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:02.456755 17148 tablet_bootstrap.cc:492] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:02.457485 17148 ts_tablet_manager.cc:1403] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.050s	user 0.009s	sys 0.018s
I20260709 12:22:02.460635 17148 raft_consensus.cc:359] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:02.461362 17148 raft_consensus.cc:740] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:02.462092 17148 consensus_queue.cc:260] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:02.465085 17148 ts_tablet_manager.cc:1434] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:22:02.466306 17148 tablet_bootstrap.cc:492] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:02.499553 17160 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:02.500489 17160 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:02.506457 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:02.510666 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:02.514498 17143 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:02.515750 17260 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:02.517258 17160 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:02.598809 17299 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.1:41805 every 8 connection(s)
I20260709 12:22:02.600678  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.1:41805
I20260709 12:22:02.600795 17148 tablet_bootstrap.cc:492] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:02.606949 17148 tablet_bootstrap.cc:492] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:02.607640 17148 ts_tablet_manager.cc:1403] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.142s	user 0.041s	sys 0.030s
I20260709 12:22:02.611294 17148 raft_consensus.cc:359] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:02.612323 17148 raft_consensus.cc:740] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:02.613533 17148 consensus_queue.cc:260] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:02.616677 17148 ts_tablet_manager.cc:1434] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:22:02.617545 17148 tablet_bootstrap.cc:492] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:02.701880 17148 tablet_bootstrap.cc:492] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:02.702883 17148 tablet_bootstrap.cc:492] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:02.703534 17148 ts_tablet_manager.cc:1403] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.086s	user 0.040s	sys 0.039s
I20260709 12:22:02.706367 17148 raft_consensus.cc:359] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:02.707010 17148 raft_consensus.cc:740] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:02.707851 17148 consensus_queue.cc:260] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:02.710333 17148 ts_tablet_manager.cc:1434] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.007s	user 0.009s	sys 0.000s
I20260709 12:22:02.711187 17148 tablet_bootstrap.cc:492] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:02.728513 17300 heartbeater.cc:344] Connected to a master server at 127.5.58.62:39363
I20260709 12:22:02.729154 17300 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:02.730124 17300 heartbeater.cc:507] Master 127.5.58.62:39363 requested a full tablet report, sending...
I20260709 12:22:02.745203 17148 tablet_bootstrap.cc:492] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:02.746497 17148 tablet_bootstrap.cc:492] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:02.747228 17148 ts_tablet_manager.cc:1403] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.036s	user 0.025s	sys 0.001s
I20260709 12:22:02.750523 17148 raft_consensus.cc:359] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:02.751348 17148 raft_consensus.cc:740] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:02.752127 17148 consensus_queue.cc:260] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:02.754498 17148 ts_tablet_manager.cc:1434] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:22:02.755434 17148 tablet_bootstrap.cc:492] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:02.755699 16226 ts_manager.cc:194] Re-registered known tserver with Master: ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:02.788691 17270 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:02.812124 17269 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:02.831841 17148 tablet_bootstrap.cc:492] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:02.833127 17148 tablet_bootstrap.cc:492] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:02.833863 17148 ts_tablet_manager.cc:1403] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.079s	user 0.028s	sys 0.012s
I20260709 12:22:02.849628 16226 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59300
I20260709 12:22:02.864166 17300 heartbeater.cc:499] Master 127.5.58.62:39363 was elected leader, sending a full tablet report...
I20260709 12:22:02.872937 17148 raft_consensus.cc:359] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:02.873636 17148 raft_consensus.cc:740] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:02.874316 17148 consensus_queue.cc:260] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:02.888226 17148 ts_tablet_manager.cc:1434] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.054s	user 0.008s	sys 0.000s
I20260709 12:22:02.889146 17148 tablet_bootstrap.cc:492] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:02.898756 17269 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:22:02.915540 16367 consensus_peers.cc:597] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:02.932356 17270 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:02.946007 17266 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:02.961787 16367 consensus_peers.cc:597] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:02.967454 17148 tablet_bootstrap.cc:492] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:02.969106 17148 tablet_bootstrap.cc:492] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:02.969744 17148 ts_tablet_manager.cc:1403] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.081s	user 0.021s	sys 0.028s
W20260709 12:22:02.966851 16367 consensus_peers.cc:597] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:02.973409 17148 raft_consensus.cc:359] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:02.975977 17148 raft_consensus.cc:740] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:02.976678 17271 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:02.976775 17148 consensus_queue.cc:260] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:02.979341 17148 ts_tablet_manager.cc:1434] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:22:02.980172 17148 tablet_bootstrap.cc:492] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:03.052632 17266 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:03.070676 17148 tablet_bootstrap.cc:492] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.071830 17148 tablet_bootstrap.cc:492] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:03.072623 17148 ts_tablet_manager.cc:1403] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.093s	user 0.032s	sys 0.019s
I20260709 12:22:03.075517 17148 raft_consensus.cc:359] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:03.076197 17148 raft_consensus.cc:740] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.077186 17148 consensus_queue.cc:260] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:03.080147 17148 ts_tablet_manager.cc:1434] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:22:03.081599 17148 tablet_bootstrap.cc:492] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:03.103444 17148 tablet_bootstrap.cc:492] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.104255 17148 tablet_bootstrap.cc:492] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:03.104920 17148 ts_tablet_manager.cc:1403] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.024s	user 0.021s	sys 0.000s
I20260709 12:22:03.106824 17148 raft_consensus.cc:359] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:03.107476 17148 raft_consensus.cc:740] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.108350 17148 consensus_queue.cc:260] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:03.110118 17148 ts_tablet_manager.cc:1434] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.004s
I20260709 12:22:03.111595 17148 tablet_bootstrap.cc:492] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:03.176122 17148 tablet_bootstrap.cc:492] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.177546 17148 tablet_bootstrap.cc:492] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:03.178342 17148 ts_tablet_manager.cc:1403] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.067s	user 0.013s	sys 0.022s
I20260709 12:22:03.181667 17148 raft_consensus.cc:359] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:03.182587 17148 raft_consensus.cc:740] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.183535 17148 consensus_queue.cc:260] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:03.185829 17148 ts_tablet_manager.cc:1434] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:22:03.186749 17148 tablet_bootstrap.cc:492] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:03.213392 17148 tablet_bootstrap.cc:492] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.214519 17148 tablet_bootstrap.cc:492] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:03.215339 17148 ts_tablet_manager.cc:1403] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.029s	user 0.013s	sys 0.011s
I20260709 12:22:03.218214 17148 raft_consensus.cc:359] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:03.218953 17148 raft_consensus.cc:740] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.219683 17148 consensus_queue.cc:260] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:03.221998 17148 ts_tablet_manager.cc:1434] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:22:03.223266 17148 tablet_bootstrap.cc:492] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:03.247992 17148 tablet_bootstrap.cc:492] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.248996 17148 tablet_bootstrap.cc:492] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:03.249781 17148 ts_tablet_manager.cc:1403] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.027s	user 0.019s	sys 0.000s
I20260709 12:22:03.253310 17148 raft_consensus.cc:359] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:03.253958 17148 raft_consensus.cc:740] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.254563 17148 consensus_queue.cc:260] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:03.256525 17148 ts_tablet_manager.cc:1434] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:22:03.257272 17148 tablet_bootstrap.cc:492] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
W20260709 12:22:03.279402 16515 consensus_peers.cc:597] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:22:03.283887 17148 tablet_bootstrap.cc:492] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.284904 17148 tablet_bootstrap.cc:492] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:03.285506 17148 ts_tablet_manager.cc:1403] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.015s
I20260709 12:22:03.288101 17148 raft_consensus.cc:359] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:03.288798 17148 raft_consensus.cc:740] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.289461 17148 consensus_queue.cc:260] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:03.292725 17148 ts_tablet_manager.cc:1434] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:22:03.293839 17148 tablet_bootstrap.cc:492] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:03.310247 17148 tablet_bootstrap.cc:492] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.323462 17148 tablet_bootstrap.cc:492] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
W20260709 12:22:03.323563 16367 consensus_peers.cc:597] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:22:03.324230 17148 ts_tablet_manager.cc:1403] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.031s	user 0.016s	sys 0.000s
I20260709 12:22:03.332423 17148 raft_consensus.cc:359] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:03.333288 17148 raft_consensus.cc:740] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.334210 17148 consensus_queue.cc:260] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:03.340531 17148 ts_tablet_manager.cc:1434] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.003s
I20260709 12:22:03.341493 17148 tablet_bootstrap.cc:492] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:03.362957 17271 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:03.465487 17148 tablet_bootstrap.cc:492] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.466593 17148 tablet_bootstrap.cc:492] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:03.467303 17148 ts_tablet_manager.cc:1403] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.126s	user 0.017s	sys 0.032s
W20260709 12:22:03.492656 16441 consensus_peers.cc:597] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:03.479538 17148 raft_consensus.cc:359] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:03.498375 17148 raft_consensus.cc:740] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.499935 17148 consensus_queue.cc:260] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:03.502717 17148 ts_tablet_manager.cc:1434] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.001s
I20260709 12:22:03.508690 17148 tablet_bootstrap.cc:492] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
W20260709 12:22:03.528777 16515 consensus_peers.cc:597] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:22:03.540248 17148 tablet_bootstrap.cc:492] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.541522 17148 tablet_bootstrap.cc:492] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:03.542286 17148 ts_tablet_manager.cc:1403] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.034s	user 0.016s	sys 0.008s
I20260709 12:22:03.545365 17148 raft_consensus.cc:359] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:03.546257 17148 raft_consensus.cc:740] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.546965 17148 consensus_queue.cc:260] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:03.549391 17148 ts_tablet_manager.cc:1434] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:22:03.550275 17148 tablet_bootstrap.cc:492] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:03.554086 17270 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:22:03.571444 16367 consensus_peers.cc:597] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:22:03.584267 17148 tablet_bootstrap.cc:492] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.585357 17148 tablet_bootstrap.cc:492] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:03.586002 17148 ts_tablet_manager.cc:1403] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.003s
I20260709 12:22:03.587379 17269 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:03.589547 17148 raft_consensus.cc:359] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:03.590222 17148 raft_consensus.cc:740] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.591033 17148 consensus_queue.cc:260] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:03.593284 17148 ts_tablet_manager.cc:1434] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:22:03.594035 17148 tablet_bootstrap.cc:492] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:03.623847 17148 tablet_bootstrap.cc:492] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.624846 17148 tablet_bootstrap.cc:492] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:03.625524 17148 ts_tablet_manager.cc:1403] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.032s	user 0.017s	sys 0.010s
W20260709 12:22:03.628124 16441 consensus_peers.cc:597] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:22:03.628043 17148 raft_consensus.cc:359] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:03.628736 17148 raft_consensus.cc:740] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.629582 17148 consensus_queue.cc:260] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:03.631908 17148 ts_tablet_manager.cc:1434] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:22:03.632822 17148 tablet_bootstrap.cc:492] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:03.654779 17269 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:03.665927 17148 tablet_bootstrap.cc:492] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.667330 17148 tablet_bootstrap.cc:492] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:03.668041 17148 ts_tablet_manager.cc:1403] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.035s	user 0.020s	sys 0.011s
I20260709 12:22:03.670857 17148 raft_consensus.cc:359] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:03.671602 17148 raft_consensus.cc:740] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.672493 17148 consensus_queue.cc:260] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:03.677364 17148 ts_tablet_manager.cc:1434] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:22:03.679021 17148 tablet_bootstrap.cc:492] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
W20260709 12:22:03.696353 16515 consensus_peers.cc:597] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:22:03.705000 17148 tablet_bootstrap.cc:492] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.706126 17148 tablet_bootstrap.cc:492] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:03.706879 17148 ts_tablet_manager.cc:1403] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.028s	user 0.016s	sys 0.006s
I20260709 12:22:03.709654 17148 raft_consensus.cc:359] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:03.710462 17148 raft_consensus.cc:740] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.711447 17148 consensus_queue.cc:260] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:03.714680 17148 ts_tablet_manager.cc:1434] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:22:03.715643 17148 tablet_bootstrap.cc:492] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
W20260709 12:22:03.720005 16441 consensus_peers.cc:597] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:22:03.735289 16515 consensus_peers.cc:597] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:22:03.746294 17148 tablet_bootstrap.cc:492] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.747161 17148 tablet_bootstrap.cc:492] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:03.747938 17148 ts_tablet_manager.cc:1403] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.033s	user 0.016s	sys 0.004s
I20260709 12:22:03.750417 17148 raft_consensus.cc:359] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:03.751091 17148 raft_consensus.cc:740] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.751765 17148 consensus_queue.cc:260] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:03.753640 17148 ts_tablet_manager.cc:1434] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.003s
I20260709 12:22:03.754659 17148 tablet_bootstrap.cc:492] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:03.781848 17148 tablet_bootstrap.cc:492] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.782868 17148 tablet_bootstrap.cc:492] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
W20260709 12:22:03.783396 16367 consensus_peers.cc:597] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:22:03.783821 17148 ts_tablet_manager.cc:1403] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.029s	user 0.016s	sys 0.005s
I20260709 12:22:03.786990 17148 raft_consensus.cc:359] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:03.787552 17269 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:03.787703 17148 raft_consensus.cc:740] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.788921 17148 consensus_queue.cc:260] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:03.791110 17148 ts_tablet_manager.cc:1434] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:22:03.792301 17148 tablet_bootstrap.cc:492] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:03.812588 17270 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:03.870971 17148 tablet_bootstrap.cc:492] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.873450 17148 tablet_bootstrap.cc:492] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:03.874140 17148 ts_tablet_manager.cc:1403] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.082s	user 0.035s	sys 0.037s
I20260709 12:22:03.876899 17148 raft_consensus.cc:359] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:03.877844 17148 raft_consensus.cc:740] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.878497 17148 consensus_queue.cc:260] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:03.880137 17148 ts_tablet_manager.cc:1434] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:22:03.881574 17148 tablet_bootstrap.cc:492] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:03.898762 17148 tablet_bootstrap.cc:492] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.899773 17148 tablet_bootstrap.cc:492] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:03.900450 17148 ts_tablet_manager.cc:1403] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.019s	user 0.017s	sys 0.000s
I20260709 12:22:03.902989 17148 raft_consensus.cc:359] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:03.903633 17148 raft_consensus.cc:740] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.904601 17148 consensus_queue.cc:260] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:03.909150 17148 ts_tablet_manager.cc:1434] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:22:03.910084 17148 tablet_bootstrap.cc:492] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:03.933903 17148 tablet_bootstrap.cc:492] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:03.934952 17148 tablet_bootstrap.cc:492] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:03.935659 17148 ts_tablet_manager.cc:1403] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.026s	user 0.018s	sys 0.000s
I20260709 12:22:03.938134 17148 raft_consensus.cc:359] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:03.938849 17148 raft_consensus.cc:740] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.939692 17148 consensus_queue.cc:260] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:03.946362 17148 ts_tablet_manager.cc:1434] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
I20260709 12:22:03.947384 17148 tablet_bootstrap.cc:492] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
W20260709 12:22:04.017820 16441 consensus_peers.cc:597] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:22:04.088119 17148 tablet_bootstrap.cc:492] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:04.104599 17148 tablet_bootstrap.cc:492] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:04.105358 17148 ts_tablet_manager.cc:1403] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.158s	user 0.055s	sys 0.076s
I20260709 12:22:04.107908 17148 raft_consensus.cc:359] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:04.108609 17148 raft_consensus.cc:740] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:04.109305 17148 consensus_queue.cc:260] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:04.110841 17148 ts_tablet_manager.cc:1434] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:22:04.124372 17148 tablet_bootstrap.cc:492] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:04.149325 17148 tablet_bootstrap.cc:492] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:04.150344 17148 tablet_bootstrap.cc:492] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:04.151024 17148 ts_tablet_manager.cc:1403] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.027s	user 0.019s	sys 0.004s
I20260709 12:22:04.153772 17148 raft_consensus.cc:359] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:04.154443 17148 raft_consensus.cc:740] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:04.155233 17148 consensus_queue.cc:260] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:04.164113 17148 ts_tablet_manager.cc:1434] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 12:22:04.164984 17148 tablet_bootstrap.cc:492] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:04.173417 17270 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:04.215217 17148 tablet_bootstrap.cc:492] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:04.216504 17148 tablet_bootstrap.cc:492] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:04.217446 17148 ts_tablet_manager.cc:1403] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.053s	user 0.021s	sys 0.002s
I20260709 12:22:04.220981 17148 raft_consensus.cc:359] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:04.221724 17148 raft_consensus.cc:740] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:04.222582 17148 consensus_queue.cc:260] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:04.234006 17148 ts_tablet_manager.cc:1434] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 12:22:04.234944 17148 tablet_bootstrap.cc:492] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:04.252843 17148 tablet_bootstrap.cc:492] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:04.253857 17148 tablet_bootstrap.cc:492] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:04.254585 17148 ts_tablet_manager.cc:1403] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.000s
I20260709 12:22:04.277045 17148 raft_consensus.cc:359] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:04.277902 17148 raft_consensus.cc:740] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:04.278828 17148 consensus_queue.cc:260] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:04.281113 17148 ts_tablet_manager.cc:1434] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 12:22:04.282022 17148 tablet_bootstrap.cc:492] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:04.301477 17148 tablet_bootstrap.cc:492] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:04.302477 17148 tablet_bootstrap.cc:492] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:04.303166 17148 ts_tablet_manager.cc:1403] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.021s	user 0.006s	sys 0.010s
I20260709 12:22:04.306445 17148 raft_consensus.cc:359] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:04.307163 17148 raft_consensus.cc:740] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:04.307898 17148 consensus_queue.cc:260] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:04.310151 17148 ts_tablet_manager.cc:1434] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:22:04.310994 17148 tablet_bootstrap.cc:492] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:04.350033 17148 tablet_bootstrap.cc:492] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:04.351131 17148 tablet_bootstrap.cc:492] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:04.351951 17148 ts_tablet_manager.cc:1403] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.041s	user 0.020s	sys 0.000s
I20260709 12:22:04.361536 17148 raft_consensus.cc:359] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:04.362277 17148 raft_consensus.cc:740] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:04.363258 17148 consensus_queue.cc:260] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:04.383338 17148 ts_tablet_manager.cc:1434] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.001s
I20260709 12:22:04.384231 17148 tablet_bootstrap.cc:492] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:04.435981 17148 tablet_bootstrap.cc:492] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:04.437572 17148 tablet_bootstrap.cc:492] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:04.441396 17148 ts_tablet_manager.cc:1403] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.057s	user 0.026s	sys 0.001s
I20260709 12:22:04.444407 17148 raft_consensus.cc:359] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:04.445185 17148 raft_consensus.cc:740] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:04.446002 17148 consensus_queue.cc:260] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:04.448592 17148 ts_tablet_manager.cc:1434] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:22:04.449961 17148 tablet_bootstrap.cc:492] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8: Bootstrap starting.
I20260709 12:22:04.513787 17270 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:04.527324 17148 tablet_bootstrap.cc:492] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:04.532375 17148 tablet_bootstrap.cc:492] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8: Bootstrap complete.
I20260709 12:22:04.533084 17148 ts_tablet_manager.cc:1403] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8: Time spent bootstrapping tablet: real 0.083s	user 0.032s	sys 0.031s
I20260709 12:22:04.535642 17148 raft_consensus.cc:359] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:04.536525 17148 raft_consensus.cc:740] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Initialized, Role: FOLLOWER
I20260709 12:22:04.537271 17148 consensus_queue.cc:260] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:04.547945 17148 ts_tablet_manager.cc:1434] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 12:22:04.662711 17266 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:22:06.190308 16257 debug-util.cc:398] Leaking SignalData structure 0x7b08002c5a80 after lost signal to thread 5355
W20260709 12:22:06.191725 16257 debug-util.cc:398] Leaking SignalData structure 0x7b080039fcc0 after lost signal to thread 16260
W20260709 12:22:06.193542 16257 debug-util.cc:398] Leaking SignalData structure 0x7b08001d1280 after lost signal to thread 16427
W20260709 12:22:06.195250 16257 debug-util.cc:398] Leaking SignalData structure 0x7b08002d1f80 after lost signal to thread 16501
W20260709 12:22:06.196628 16257 debug-util.cc:398] Leaking SignalData structure 0x7b08002e5ae0 after lost signal to thread 16575
W20260709 12:22:06.198134 16257 debug-util.cc:398] Leaking SignalData structure 0x7b08002fab20 after lost signal to thread 17299
W20260709 12:22:08.670238 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 89746.9us
W20260709 12:22:08.770658 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 66510us
W20260709 12:22:08.817477 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 45306.3us
W20260709 12:22:09.637410 16572 debug-util.cc:398] Leaking SignalData structure 0x7b0800049f40 after lost signal to thread 5355
W20260709 12:22:09.677322 16572 debug-util.cc:398] Leaking SignalData structure 0x7b0800090320 after lost signal to thread 16260
W20260709 12:22:09.678792 16572 debug-util.cc:398] Leaking SignalData structure 0x7b0800142740 after lost signal to thread 16427
W20260709 12:22:09.680119 16572 debug-util.cc:398] Leaking SignalData structure 0x7b0800178b60 after lost signal to thread 16501
W20260709 12:22:09.681468 16572 debug-util.cc:398] Leaking SignalData structure 0x7b0800317ee0 after lost signal to thread 16575
W20260709 12:22:09.682852 16572 debug-util.cc:398] Leaking SignalData structure 0x7b08001185e0 after lost signal to thread 17299
W20260709 12:22:10.342641 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 49489.5us
W20260709 12:22:11.441778 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 46671.2us
W20260709 12:22:12.822784 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 244459us
I20260709 12:22:12.849570  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:22:12.937407 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 54111.5us
I20260709 12:22:13.171159 16402 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2"
dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:60492
I20260709 12:22:13.172297 16402 raft_consensus.cc:606] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:13.269644 16468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
mode: GRACEFUL
new_leader_uuid: "610f4feda11942739236f2b121a4b9c1"
 from {username='slave'} at 127.0.0.1:39000
I20260709 12:22:13.270296 16468 raft_consensus.cc:606] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Received request to transfer leadership to TS 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:13.273725 16468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:39000
I20260709 12:22:13.274210 16468 raft_consensus.cc:606] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:22:13.277134 16468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:39000
I20260709 12:22:13.277664 16468 raft_consensus.cc:606] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:22:13.280539 16468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:39000
I20260709 12:22:13.281047 16468 raft_consensus.cc:606] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:22:13.284581 16468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:39000
I20260709 12:22:13.285175 16468 raft_consensus.cc:606] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:22:13.290652  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:22:13.292809  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:13.293941  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:13.442181 17452 raft_consensus.cc:993] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2: : Instructing follower ef57beab0e1d40d585983873c55107e8 to start an election
I20260709 12:22:13.442808 17452 raft_consensus.cc:1081] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Signalling peer ef57beab0e1d40d585983873c55107e8 to start an election
I20260709 12:22:13.445945 17258 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d"
dest_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:45224
I20260709 12:22:13.446748 17258 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:13.447198 17258 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.456544 17438 raft_consensus.cc:993] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2: : Instructing follower 610f4feda11942739236f2b121a4b9c1 to start an election
I20260709 12:22:13.457163 17438 raft_consensus.cc:1081] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Signalling peer 610f4feda11942739236f2b121a4b9c1 to start an election
I20260709 12:22:13.459453 16545 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
 from {username='slave'} at 127.0.0.1:37242
I20260709 12:22:13.460108 16545 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:13.460726 16545 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.509012 17438 raft_consensus.cc:993] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2: : Instructing follower ef57beab0e1d40d585983873c55107e8 to start an election
I20260709 12:22:13.509797 17438 raft_consensus.cc:1081] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Signalling peer ef57beab0e1d40d585983873c55107e8 to start an election
I20260709 12:22:13.517385 17263 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773"
dest_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:45224
I20260709 12:22:13.518020 17263 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:13.518625 17263 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.517927 17258 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:13.515971 16545 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:13.527310 17438 raft_consensus.cc:993] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2: : Instructing follower ef57beab0e1d40d585983873c55107e8 to start an election
I20260709 12:22:13.527892 17438 raft_consensus.cc:1081] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Signalling peer ef57beab0e1d40d585983873c55107e8 to start an election
I20260709 12:22:13.529721 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:13.529929 17261 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e"
dest_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:45224
I20260709 12:22:13.530493 16402 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.530762 17261 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:13.531311 17261 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:13.534124 16545 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:13.536293 17258 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:13.536736 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:13.537354 16400 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.538504 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:13.539254 16468 raft_consensus.cc:3055] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:13.539873 16468 raft_consensus.cc:740] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:13.540833 16468 consensus_queue.cc:260] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:13.542205 16468 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.536152 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:13.543869 16476 raft_consensus.cc:3055] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:13.544173 16476 raft_consensus.cc:740] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:13.544984 16476 consensus_queue.cc:260] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:13.546324 16476 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.590596 16402 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:13.604806 17144 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:13.609313 17457 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:13.623785 17457 raft_consensus.cc:697] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:13.625196 17457 consensus_queue.cc:237] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:13.599526 17263 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:13.630575 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:13.631235 16475 raft_consensus.cc:3055] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:13.631517 16475 raft_consensus.cc:740] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:13.631690 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:13.612231 16468 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:13.632293 16545 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.632319 16475 consensus_queue.cc:260] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:13.633752 16475 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.629115 17263 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:13.618090 16400 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:13.611490 16476 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:13.645015 16516 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:13.645615 16475 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:13.646076 17441 raft_consensus.cc:2804] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:13.647461 17261 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:13.656704 17441 raft_consensus.cc:697] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 2 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:13.657825 17441 consensus_queue.cc:237] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:13.663748 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:13.664212 17142 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:13.677888 17458 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:13.664495 16468 raft_consensus.cc:3055] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:13.680706 16468 raft_consensus.cc:740] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:13.681735 16468 consensus_queue.cc:260] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:13.695081 17261 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:13.698712 16545 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:13.710445 16468 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:13.711203 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:13.714748 16400 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:13.760608 17458 raft_consensus.cc:697] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:13.763617 17458 consensus_queue.cc:237] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:13.769138 17451 raft_consensus.cc:993] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2: : Instructing follower ef57beab0e1d40d585983873c55107e8 to start an election
I20260709 12:22:13.770713 17447 raft_consensus.cc:1081] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Signalling peer ef57beab0e1d40d585983873c55107e8 to start an election
I20260709 12:22:13.775642 16468 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
W20260709 12:22:13.781085 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 63877.8us
I20260709 12:22:13.780540 17142 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:13.784706 16400 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:13.789251 17457 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:13.790112 17457 raft_consensus.cc:697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:13.790874 17261 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339"
dest_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:45224
I20260709 12:22:13.791535 17261 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:13.792778 17261 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.791450 17457 consensus_queue.cc:237] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:13.809870 16225 catalog_manager.cc:5697] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 1 to 2, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 2 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:13.827924 17261 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:13.842343 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:13.842967 16400 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.845655 17261 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:22:13.847038 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 59353.7us
I20260709 12:22:13.847325 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:13.848006 16475 raft_consensus.cc:3055] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:13.848348 16475 raft_consensus.cc:740] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:13.849290 16475 consensus_queue.cc:260] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:13.850667 16475 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.872553 16475 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:13.876732 17142 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:13.878861 17439 raft_consensus.cc:993] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50: : Instructing follower 52ab50b0237346c1a13ede947f6a63c2 to start an election
I20260709 12:22:13.879424 17458 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:13.879487 17439 raft_consensus.cc:1081] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Signalling peer 52ab50b0237346c1a13ede947f6a63c2 to start an election
I20260709 12:22:13.888537 17458 raft_consensus.cc:697] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:13.878952 16400 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:13.895764 17458 consensus_queue.cc:237] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:13.896826 16468 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:34840
I20260709 12:22:13.901095 16468 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:13.901612 16468 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.909219 16468 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:13.917716 16468 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:13.919088 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:13.919895 16545 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:13.921247 16226 catalog_manager.cc:5697] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 1 to 2, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 2 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:13.923638 16226 catalog_manager.cc:5697] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 1 to 2, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 2 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:13.925897 16226 catalog_manager.cc:5697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 2 to 3, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 3 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:13.930184 16545 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:13.931598 16441 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:13.932834 17451 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:13.933694 17451 raft_consensus.cc:697] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:13.934828 17451 consensus_queue.cc:237] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:14.013337 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:14.013955 16400 raft_consensus.cc:3055] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:14.014225 16400 raft_consensus.cc:740] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:14.014986 16400 consensus_queue.cc:260] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:14.016292 16400 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:14.027593 16400 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:14.033006 16223 catalog_manager.cc:5697] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 1 to 2, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 2 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:14.070313 16400 raft_consensus.cc:1275] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:14.076408 17441 consensus_queue.cc:1048] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:14.077432 16226 catalog_manager.cc:5697] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 1 to 2, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 2 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:14.140070 16472 raft_consensus.cc:1275] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:14.142421 17458 consensus_queue.cc:1048] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:14.150781 16468 raft_consensus.cc:1275] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:14.152508 17458 consensus_queue.cc:1048] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:14.165908 16402 raft_consensus.cc:1275] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:14.167208 17458 consensus_queue.cc:1048] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:14.195890 16475 raft_consensus.cc:1275] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:14.225504 17435 consensus_queue.cc:1048] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:14.277767 16548 raft_consensus.cc:1275] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:14.284977 17458 consensus_queue.cc:1048] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:14.309937 16475 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:14.312325 17458 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:14.375028 16402 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:14.377524 17458 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:14.463572 16475 raft_consensus.cc:1275] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:14.465345 17458 consensus_queue.cc:1048] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:14.475466 16545 raft_consensus.cc:1275] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:14.475926 16402 raft_consensus.cc:1275] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:14.479290 17447 consensus_queue.cc:1048] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:14.482699 17438 consensus_queue.cc:1048] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:14.488056 16402 raft_consensus.cc:1275] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:14.492425 17457 consensus_queue.cc:1048] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:14.521620  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:15.042001 16548 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:22:15.043437 16548 raft_consensus.cc:606] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:22:15.048080 16475 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:39000
I20260709 12:22:15.048900 16475 raft_consensus.cc:606] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:22:15.100538 16399 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81"
dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:60492
I20260709 12:22:15.101251 16399 raft_consensus.cc:606] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:22:15.104385 17499 raft_consensus.cc:993] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1: : Instructing follower ef57beab0e1d40d585983873c55107e8 to start an election
I20260709 12:22:15.105079 17499 raft_consensus.cc:1081] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Signalling peer ef57beab0e1d40d585983873c55107e8 to start an election
I20260709 12:22:15.109702 17258 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064"
dest_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:45228
I20260709 12:22:15.110435 17258 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:15.110930 17258 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:15.131281 16402 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1"
dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:60492
I20260709 12:22:15.132041 16402 raft_consensus.cc:606] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:22:15.136663 16402 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef53202d665e46c39113572380be6c36"
dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:60492
I20260709 12:22:15.137241 16402 raft_consensus.cc:606] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:22:15.137339 17258 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:15.138906  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:22:15.144271  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:15.145452  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:15.158679 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:15.159708 16472 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:15.163133 17258 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:15.164887 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:15.165674 16551 raft_consensus.cc:3055] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:15.166045 16551 raft_consensus.cc:740] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:15.167064 16551 consensus_queue.cc:260] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:15.172114 16551 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:15.195806 17469 raft_consensus.cc:993] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50: : Instructing follower ef57beab0e1d40d585983873c55107e8 to start an election
I20260709 12:22:15.196429 17469 raft_consensus.cc:1081] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Signalling peer ef57beab0e1d40d585983873c55107e8 to start an election
I20260709 12:22:15.198729 17270 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81"
dest_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:45230
I20260709 12:22:15.199268 17270 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:15.199616 17270 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:15.208752 17463 raft_consensus.cc:993] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50: : Instructing follower ef57beab0e1d40d585983873c55107e8 to start an election
I20260709 12:22:15.209502 17439 raft_consensus.cc:1081] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Signalling peer ef57beab0e1d40d585983873c55107e8 to start an election
I20260709 12:22:15.211441 17258 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1"
dest_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:45230
I20260709 12:22:15.212121 17258 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:15.212603 17258 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:15.213160 16551 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:15.213809 16472 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:15.214802 17270 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:15.218940 17270 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:15.218753 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:15.219583 16548 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:15.220088 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:15.220681 16402 raft_consensus.cc:3055] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:15.221033 16402 raft_consensus.cc:740] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:15.222014 16402 consensus_queue.cc:260] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:15.223192 16402 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:15.231175 16548 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:15.232699 16402 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:15.234964 17142 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:15.236020 17457 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:15.236613 17457 raft_consensus.cc:697] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:15.237794 17457 consensus_queue.cc:237] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:15.245436 17143 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:15.246752 17461 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:15.247314 17461 raft_consensus.cc:697] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:15.251633 17461 consensus_queue.cc:237] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:15.300827 17258 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:15.308969 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:15.309724 16402 raft_consensus.cc:3055] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:15.310166 16402 raft_consensus.cc:740] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:15.312979 16402 consensus_queue.cc:260] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:15.314319 16402 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:15.323699 16224 catalog_manager.cc:5697] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 1 to 2, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 2 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:15.327455 17258 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:15.328641 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:15.332562 16472 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:15.346537 16402 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:15.348441 17144 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:15.349452 17461 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:15.349992 17461 raft_consensus.cc:697] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:15.350984 17461 consensus_queue.cc:237] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:15.375017 16472 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:15.389253 17451 raft_consensus.cc:993] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2: : Instructing follower ef57beab0e1d40d585983873c55107e8 to start an election
I20260709 12:22:15.389964 17449 raft_consensus.cc:1081] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Signalling peer ef57beab0e1d40d585983873c55107e8 to start an election
I20260709 12:22:15.398219 17258 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e"
dest_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:45224
I20260709 12:22:15.399179 17258 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:15.400009 17258 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:15.435989 17258 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:15.440662 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:15.441272 16402 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:15.453639 16402 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:15.464457 17144 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:15.465528 17461 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:15.466457 17461 raft_consensus.cc:697] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:15.467427 17461 consensus_queue.cc:237] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:15.459224 16224 catalog_manager.cc:5697] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 1 to 2, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 2 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:15.462951 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:15.473320 16475 raft_consensus.cc:3055] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:15.473667 16475 raft_consensus.cc:740] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:15.474509 16475 consensus_queue.cc:260] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:15.475965 16475 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:15.487154 16475 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:15.489543 17258 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:15.489135 16224 catalog_manager.cc:5697] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 2 to 3, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 3 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:15.575538 16224 catalog_manager.cc:5697] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 1 to 2, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 2 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:15.726933 16548 raft_consensus.cc:1275] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:15.729101 17461 consensus_queue.cc:1048] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:15.746042 16551 raft_consensus.cc:1275] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:15.764168 17486 consensus_queue.cc:1048] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:16.708918 17513 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:16.709736 17513 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:16.716670 17513 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:16.773295 17513 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:16.781282 17512 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:16.782526 17512 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:16.785687 17512 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:16.785984 17511 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:16.786878 17511 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:16.790124 17511 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:16.774214 17513 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:16.803711 17513 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:16.804454 17511 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:16.875236 17513 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:16.875422 17511 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:16.875907 17513 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:16.878062 17511 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:16.878698 17511 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:22:16.826174 16257 debug-util.cc:398] Leaking SignalData structure 0x7b08002fa860 after lost signal to thread 5355
I20260709 12:22:16.809151 17512 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:16.880877 17512 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:16.982858 17511 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:16.883841 17512 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:17.012239 17512 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.013101 17512 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:16.878705 17513 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:17.040367 17511 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.040902 17511 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:17.041150 17513 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.041446 17511 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:17.042184 17513 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:17.043665 17511 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.044222 17511 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.047092 17513 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.044718 17511 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:17.060808 17511 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.047626 17512 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.066524 17512 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:22:17.067073 16366 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.030s	sys 0.036s
W20260709 12:22:17.067582 16366 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.031s	sys 0.036s
I20260709 12:22:17.068037 17513 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:17.070166 17513 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:16.371546  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:17.067207 17512 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:17.074934 17516 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.075682 17516 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:17.077735 17512 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.078274 17512 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:17.087982 17513 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.088248 17512 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:17.089696 17513 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.090330 17513 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:17.092821 17512 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.093699 17512 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:17.084625 16224 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59300 (request call id 80) took 1569 ms. Trace:
I20260709 12:22:17.097988 17516 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.098665 17516 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.099233 17516 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:17.102257 17516 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.107538 17512 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:17.105877 17513 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.103504 17511 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:17.105623 17429 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:17.115070 17429 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:17.114001 17435 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.118059 17429 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:17.097136 16224 rpcz_store.cc:276] 0709 12:22:15.543024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:15.543205 (+   181us) service_pool.cc:224] Handling call
0709 12:22:17.083786 (+1540581us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:22:15.605527 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:22:15.606175 (+   648us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:22:15.606229 (+    54us) write_op.cc:435] Acquired schema lock
0709 12:22:15.606252 (+    23us) tablet.cc:662] Decoding operations
0709 12:22:15.606675 (+   423us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:22:15.606730 (+    55us) write_op.cc:641] Partition lock acquired for write op
0709 12:22:15.606759 (+    29us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:22:15.606933 (+   174us) tablet.cc:701] Row locks acquired
0709 12:22:15.606950 (+    17us) write_op.cc:260] PREPARE: finished
0709 12:22:15.607093 (+   143us) write_op.cc:270] Start()
0709 12:22:15.607255 (+   162us) write_op.cc:276] Timestamp: P: 1783599735607055 usec, L: 0
0709 12:22:15.607282 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 12:22:15.608023 (+   741us) log.cc:844] Serialized 531 byte log entry
0709 12:22:15.610923 (+  2900us) op_driver.cc:464] REPLICATION: finished
0709 12:22:15.611520 (+   597us) write_op.cc:301] APPLY: starting
0709 12:22:15.611618 (+    98us) tablet.cc:1367] starting BulkCheckPresence
0709 12:22:15.611730 (+   112us) tablet.cc:1370] finished BulkCheckPresence
0709 12:22:15.611822 (+    92us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:22:15.612074 (+   252us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:22:15.612252 (+   178us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:22:15.612310 (+    58us) write_op.cc:312] APPLY: finished
0709 12:22:15.612957 (+   647us) log.cc:844] Serialized 22 byte log entry
0709 12:22:15.613125 (+   168us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:15.613225 (+   100us) write_op.cc:454] Released schema lock
0709 12:22:15.613262 (+    37us) write_op.cc:341] FINISH: Updating metrics
0709 12:22:15.613920 (+   658us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:15.613958 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":327,"apply.run_cpu_time_us":1936,"apply.run_wall_time_us":2322,"num_ops":1,"prepare.queue_time_us":738,"prepare.run_cpu_time_us":2826,"prepare.run_wall_time_us":2828,"replication_time_us":3477}]]}
I20260709 12:22:17.171452 17451 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.172399 17451 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:17.128132 17435 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:17.129372 17511 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:17.173462 17525 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:17.206173 17435 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:17.206360 17451 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.210356 17511 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:17.212133 17512 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.225554 17528 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:17.226257 17528 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:17.229324 17528 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.235836 17528 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:17.236531 17528 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:17.237275 17529 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:17.238768 17528 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.238170 17529 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:17.239370 17528 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:17.243815 17512 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.241652 17529 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.239945 17528 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:17.243017 17429 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:17.244910 17519 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:17.256029 17429 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:17.256668 17519 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:17.258656 17429 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:17.259366 17519 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:17.243953 17525 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:17.255716 17515 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:17.262707 17525 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:17.255903 17524 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:17.263128 17515 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:17.263973 17524 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:17.259441 17527 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.265553 17527 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:17.266701 17524 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:17.276613 17526 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:17.277406 17526 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:17.268537 17527 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.295790 17526 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.255705 17518 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:17.313977 17518 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:17.329975 17523 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:17.330757 17523 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:17.332793 17521 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.333562 17521 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:17.336898 17532 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.336964 17531 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.244956 17512 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:17.337728 17531 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:17.340286 17512 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:17.340646 17531 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.341355 17531 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:17.341904 17531 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:17.337713 17532 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:17.345981 17531 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:17.346608 17531 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.347205 17531 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:17.370105 17531 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.371563 17523 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:17.395044 17512 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.336211 17521 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:17.395901 17512 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:17.455224 17512 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.455683 17515 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.469282 17518 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:17.471472 17527 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:17.472432 17527 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:17.475234 17527 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:17.476359 16368 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.728s	user 0.003s	sys 0.023s
I20260709 12:22:17.479070 17531 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:22:17.479535 16257 debug-util.cc:398] Leaking SignalData structure 0x7b08002fafa0 after lost signal to thread 16260
I20260709 12:22:17.480612 17536 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.507871 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.508513 17536 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:17.509615 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.511065 17536 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.511654 17536 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.482095 17532 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.512619 17532 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.513105 17532 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:17.513870 17499 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.484489 17429 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:22:17.515319 16368 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.767s	user 0.006s	sys 0.023s
I20260709 12:22:17.515738 17530 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.515544 17429 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:17.516402 17525 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:17.475980 17518 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.485657 17521 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:17.488353 17439 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:17.488968 17463 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:17.489598 17524 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:17.255262 17528 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.490806 17519 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:17.493414 17522 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.498047 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.499547 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.499977 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.515338 17532 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.500711 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.500968 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.554461 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:17.501953 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.563957 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:17.564112 16394 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:22:17.564988 17519 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:17.565215 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.566627 16394 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:17.567140 17519 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.555706 17522 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:17.568547 16401 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:17.569926 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.570155 17522 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:17.504606 17529 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:17.504980 17523 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.573400 17529 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:17.505286 17515 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:17.574381 17515 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:17.574501 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:17.505971 17526 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:17.576443 17515 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:17.514501 17499 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:17.576355 17526 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:17.578547 17526 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:17.578992 17499 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.579071 16395 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:17.484973 17435 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.526538 17530 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:17.579761 17435 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:17.581214 17530 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:17.533792 17463 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:17.582319 17435 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:17.507634 17531 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:17.583565 17463 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:17.584362 17531 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:17.535490 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.531318 17524 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:17.529570 17521 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:17.528739 17439 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:17.527513 17525 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:17.537055 17429 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:17.539014 17528 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:17.539536 17537 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.529039 17518 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:17.512151 17536 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:17.526815 17527 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:17.548537 17512 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.550757 17511 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:17.551128 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:17.551501 17451 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.552415 17520 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.563041 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:17.563282 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.554972 16403 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:22:17.552464 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:17.573467 17523 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:17.547744 17449 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:17.535815 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.571791 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.611162 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:17.575565 17529 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:17.485577 17535 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:17.600793 17521 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:17.600987 17525 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:17.595630 16396 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:17.633054 16476 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:17.601456 17439 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:17.601749 17538 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:17.576241 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:17.617223 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.625344 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.620033 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.624641 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.575567 16470 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:22:17.567857 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.614637 16442 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:17.611809 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:17.630707 17461 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.646376 17538 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:17.652714 17461 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:17.632695 16468 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:22:17.653682 16401 raft_consensus.cc:1275] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:17.656100 17439 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:17.656778 17439 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:17.657349 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:17.658627 17439 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.659134 17439 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:17.659538 17439 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:17.607285 17528 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:17.661943 17528 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:17.607601 17511 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:17.633074 17537 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:17.664690 17511 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.630807 17512 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:17.666173 17537 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.666960 17537 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:17.668012 17512 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.617487 17520 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:17.670522 17520 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.646564 17535 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:17.671509 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.672421 16542 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
I20260709 12:22:17.673406 17535 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:17.674585 17542 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.641633 16403 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:22:17.616506 17536 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.623116 17518 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:17.648592 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.677904 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.684836 16402 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:22:17.655970 17461 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.685745 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:17.686317 16472 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:22:17.640635 17449 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:17.687613 17461 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:17.655957 17538 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.651013 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.640990 16398 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:17.689551 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.651979 16395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159"
dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:60492
I20260709 12:22:17.689978 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.690140 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.690680 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.624475 17524 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:17.689865 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.679992 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.681555 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:17.693957 17439 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:17.682971 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.686110 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.649602 16442 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:17.695518 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 46936.5us
I20260709 12:22:17.695892 17523 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:17.696677 16442 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:17.697333 16257 debug-util.cc:398] Leaking SignalData structure 0x7b0800328a80 after lost signal to thread 16427
I20260709 12:22:17.688633 17449 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:17.698567 16257 debug-util.cc:398] Leaking SignalData structure 0x7b08003e6840 after lost signal to thread 16501
I20260709 12:22:17.675174 17542 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:17.690603 16395 raft_consensus.cc:606] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:22:17.650833 17516 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:17.700255 17516 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:17.700668 17516 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.700752 17542 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.701889 17542 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:17.702466 17520 raft_consensus.cc:2804] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:17.702404 17542 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:17.703058 17520 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:17.703649 17520 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:17.667603 17537 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:17.688160 17461 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:17.705909 17537 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.706192 17461 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.706463 17537 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.706687 17461 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:17.706970 17537 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:17.707226 17461 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:17.708540 17538 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.709549 17538 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:17.713443 16257 debug-util.cc:398] Leaking SignalData structure 0x7b08001f1700 after lost signal to thread 16575
I20260709 12:22:17.698683 17449 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:17.617587 17451 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:17.677004 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:22:17.714751 16257 debug-util.cc:398] Leaking SignalData structure 0x7b08000ede80 after lost signal to thread 17299
W20260709 12:22:17.716658 16257 debug-util.cc:398] Leaking SignalData structure 0x7b0800141040 after lost signal to thread 17300
I20260709 12:22:17.629566 17527 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:17.726763 16442 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:17.734515 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.735545 16551 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:17.717309 17449 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:17.750404 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.750629 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.754803 17449 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:17.782196 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.743918 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.743336 17542 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.785565 17539 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:17.790365 17542 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:17.792676 17539 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:17.792999 17542 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:17.795140 17539 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:17.795400 17542 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.744087 17512 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:17.796492 16403 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5"
dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:60492
I20260709 12:22:17.796931 17512 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:17.797137 16403 raft_consensus.cc:606] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:17.797451 17512 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.797729 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.799062 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.743317 17538 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:17.792414 16514 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
W20260709 12:22:17.801054 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 83511.7us
I20260709 12:22:17.784130 16442 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:17.801977 16514 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:17.793054 17541 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:17.807072 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.807255 17541 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:17.807735 17527 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:17.807914 17538 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.808511 17538 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:17.808751 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.809540 17541 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.792378 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.810073 17541 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:17.810578 17538 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:17.810842 17447 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:17.811043 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.811326 17447 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:17.811712 17447 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:17.807003 17544 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.812188 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.808523 17537 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:17.800191 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.812865 17544 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:17.813532 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.814459 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.814759 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.815749 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.816542 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.816583 17544 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.821384 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.822235 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.823714 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.824781 17537 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:17.825356 17537 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:17.827778 17537 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.828846 16442 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:17.829280 17548 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:17.830231 17546 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:17.830439 17548 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:17.830802 17546 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:17.832775 17548 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.832888 17546 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.833211 17451 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.833612 17546 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.834035 17451 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:17.834455 17451 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:17.834174 17546 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:17.834828 17451 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.836575 17547 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:17.836953 17544 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:17.837117 17547 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:17.837492 17544 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:17.839376 17547 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.839476 17544 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.805546 17499 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:17.840183 17499 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:17.840611 17499 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:17.842813 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.843117 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.844305 16442 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:17.845396 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.846001 17511 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:17.846104 16546 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:22:17.819897 17449 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:17.846486 17511 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:17.848305 17511 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:17.848604 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.849475 17449 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.849923 17461 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:17.850723 17547 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:17.853506 17547 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:17.855127 17516 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:17.855983 17544 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:17.856142 17461 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.856578 17544 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:17.856900 16442 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:17.856807 17461 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:17.855607 17547 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.860514 17548 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.859664 17538 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:17.860925 17520 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:17.861682 17548 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:17.861908 17538 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:17.862564 17544 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:17.863080 17537 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.859427 16367 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:17.863303 17520 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.863559 17547 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.863739 17548 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.868880 16514 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:17.837203 17546 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.875876 17451 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:17.864233 17538 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.810002 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.881634 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.876039 17528 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:17.904403 17528 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:17.904636 17517 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:17.904855 17528 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:17.905205 17517 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:17.905652 17517 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:17.877076 17537 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:17.909798 17451 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.893654 16395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835"
dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:60492
I20260709 12:22:17.910702 17545 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:17.911190 16395 raft_consensus.cc:606] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:22:17.911320 17545 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:17.878261 17429 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:17.912909 17429 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:17.913779 17545 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:22:17.914746 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 45780us
I20260709 12:22:17.915735 16516 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:17.916548 17518 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:17.916862 16516 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:17.894677 16514 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:17.895293 17523 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:17.865746 17542 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.867030 17535 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:17.858963 17461 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:17.898121 17548 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:17.919711 17535 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.920871 17535 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.924459 17521 raft_consensus.cc:2804] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:17.924787 17552 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.925006 17521 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.925397 17552 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:17.925726 17521 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.927644 17552 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:17.928432 17538 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.919639 17523 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:17.929109 17538 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:17.919926 17548 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:17.931340 17538 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.931855 17538 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:17.879091 17544 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:17.810591 17541 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:17.903750 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:17.918298 17435 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:17.897620 17546 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:17.933516 16400 raft_consensus.cc:1275] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:17.936239 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:17.919093 16516 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:17.936936 16399 raft_consensus.cc:1275] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:17.938478 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.937232 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.954542 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.938966 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.955758 16548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 13) took 2222 ms. Trace:
I20260709 12:22:17.955857 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.939314 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:17.956626 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:17.939653 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.919932 17542 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:17.958701 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.959875 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.933918 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.941942 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.960852 17542 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.942108 17541 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.932227 17548 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.942798 17516 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.943040 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.938627 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.943112 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.962980 17512 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:17.963984 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.944581 17435 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:17.907485 17511 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:17.945232 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:17.964998 17435 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.936806 17544 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:17.967847 16470 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:17.969029 17511 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.947897 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:17.969447 17544 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.961895 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.929127 17523 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.944110 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.936650 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:17.945777 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.973937 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.972500 16514 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
W20260709 12:22:17.975679 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 56757.4us
I20260709 12:22:17.943472 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.980929 17540 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:17.945029 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.981901 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.982823 16395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0"
dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:60492
I20260709 12:22:17.983413 16395 raft_consensus.cc:606] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:22:17.984846 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.947618 17447 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:17.987686 17447 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.938030 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:17.949378 17546 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:17.964519 17499 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:17.991353 17546 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.991705 17499 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:17.956126 16548 rpcz_store.cc:276] 0709 12:22:15.733257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:17.613394 (+1880137us) service_pool.cc:224] Handling call
0709 12:22:17.613935 (+   541us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:17.636004 (+ 22069us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:17.636131 (+   127us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:17.654726 (+ 18595us) log.cc:844] Serialized 26 byte log entry
0709 12:22:17.750187 (+ 95461us) spinlock_profiling.cc:243] Waited 21.7 ms on lock 0x7b18002b5ca0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:22:17.877830 (+127643us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:22:17.877865 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:17.877943 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:17.955438 (+ 77495us) raft_consensus.cc:1685] finished
0709 12:22:17.955461 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:17.955671 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12467,"raft.queue_time_us":25290,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":4958,"spinlock_wait_cycles":49830784,"thread_start_us":127462,"threads_started":1,"wal-append.queue_time_us":298761}
I20260709 12:22:17.943735 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.994112 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:17.994737 16472 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:17.991981 16547 raft_consensus.cc:1240] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:17.996937 16471 raft_consensus.cc:1240] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:17.999172 16399 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070"
dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:60492
I20260709 12:22:17.999727 16399 raft_consensus.cc:606] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:22:18.000696 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:18.002621 17429 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:17.941964 17538 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:18.005249  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:22:18.006026 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.007119  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:18.007350 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.008033  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:18.008567 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.008728 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.009714 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.011313 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:17.949194 16468 raft_consensus.cc:1240] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:18.005312 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:17.977618 17544 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:18.012856 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.013525 17544 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:18.014233 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:18.015516 17544 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:17.895473 17547 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:18.016646 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:18.017871 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:17.910741 17537 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:18.020010 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:18.020635 16472 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:18.037129 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:17.947996 17517 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:18.021440 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:18.022059 17548 consensus_queue.cc:1048] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:18.038215 17523 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:18.039822 17517 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:18.022651 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:18.024641 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:18.025087 17435 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:18.025700 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:18.026091 17547 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:18.026218 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:18.027004 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:18.027575 17546 consensus_queue.cc:1048] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:18.028239 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:18.020737 17538 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:18.029064 17544 consensus_queue.cc:1048] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:17.976080 17512 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:18.030503 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:18.031257 17521 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:18.031872 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:18.032246 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.047263 17528 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:18.048229 16470 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 3: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:18.047951 17429 consensus_queue.cc:1059] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:18.049275 17518 raft_consensus.cc:3055] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:18.049474 17435 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:18.049772 17518 raft_consensus.cc:740] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:18.050480 17521 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:18.034272 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:18.035156 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:18.033213 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:18.051908 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:18.035789 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:18.050554 16549 raft_consensus.cc:1240] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:18.050781 17518 consensus_queue.cc:260] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:18.054030 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:18.054041 17518 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:18.055959 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:18.056118 17539 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:18.058097 17539 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:18.058524 17539 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:18.035336 17535 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:18.062754 16516 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.064597 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:18.064781 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:18.065061 17521 consensus_queue.cc:1059] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:18.066252 17521 raft_consensus.cc:3055] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:18.066627 17521 raft_consensus.cc:740] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:18.067597 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.067520 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.064106 16514 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:18.069051 16469 raft_consensus.cc:2393] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 3: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:18.067415 17521 consensus_queue.cc:260] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:18.072149 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:18.072717 16399 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:18.073366 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:18.071451 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:18.074717 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:18.075119 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:18.076215 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:18.080459 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:18.089881 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:18.090972 16468 raft_consensus.cc:1240] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:18.091500 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:18.090191 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:18.095240 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:18.091997 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:18.092243 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:18.093586 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.068317 17528 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:18.090728 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:18.094939 16470 raft_consensus.cc:2393] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 4: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:18.101089 16469 raft_consensus.cc:1240] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:18.083209 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.102211 16472 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
I20260709 12:22:18.102267 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.076647 17499 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:18.096349 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:18.097771 16367 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:18.106809 17521 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.107136 16477 raft_consensus.cc:1275] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:18.109488 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:18.083061 17429 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.112241 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:18.118647 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:18.114751 17529 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:18.121518 17529 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:18.121876 17529 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.098306 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:18.115166 17523 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:18.126717 16399 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:18.127727 16398 raft_consensus.cc:1240] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:22:18.133671 16477 raft_consensus.cc:1240] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:18.135761 17535 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:18.148844 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:18.156708 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.168180 17539 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:18.182403 17539 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:18.185560 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:18.186251 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:18.186581 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:18.187556 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:18.182502 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:18.192986 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.193213 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.194005 17258 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:22:18.194274 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.196507 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.196739 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.197376 17263 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:18.198431 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.198827 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.199498 17254 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:18.199574 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.200387 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.201017 17254 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:18.201843 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.202396 17254 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:22:18.203222 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.203738 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.204209 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.204777 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.205067 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.205400 17270 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:22:18.200208 17266 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:22:18.205564 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.206295 17261 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:22:18.206365 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.209175 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.212756 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.205646 17263 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:22:18.209980 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.219125 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.222891 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:18.224787 17258 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:18.224913 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.225145 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.225579 17263 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:18.225826 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.208387 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.226469 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.226912 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.203768 17254 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:22:18.206822 17544 consensus_queue.cc:1059] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 12:22:18.228540 16472 raft_consensus.cc:1240] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:18.228961 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.229137 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.229267 17538 raft_consensus.cc:3055] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:18.229563 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.229797 17538 raft_consensus.cc:740] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:18.230400 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.230171 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.230854 16472 raft_consensus.cc:1240] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:18.208827 16469 raft_consensus.cc:1240] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:18.198879 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.206382 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:18.207891 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.231985 17255 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:18.209558 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.207840 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.208395 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.233072 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.209398 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:18.220217 17529 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:18.233664 17255 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:22:18.232089 16469 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:18.234221 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.234624 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.235167 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.235483 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.206496 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.211103 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.240495 16549 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:18.231009 17261 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:22:18.242475 17521 consensus_queue.cc:1059] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:18.233073 17264 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:18.243656 17521 consensus_queue.cc:1059] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:18.244733 17531 raft_consensus.cc:3055] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:18.245146 17531 raft_consensus.cc:740] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:18.247308 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:18.247924 16475 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:18.227002 17269 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:22:18.235921 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.230962 17538 consensus_queue.cc:260] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:18.226217 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.227164 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.235807 17529 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:18.256445 16548 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:18.231285 17254 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:18.227578 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:18.259915 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:18.261979 16441 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:18.275558 16441 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.275419 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.263515 17499 raft_consensus.cc:697] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 3 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:18.265780 16475 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:18.271281 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:18.227916 17271 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:22:18.246057 17531 consensus_queue.cc:260] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:18.281064 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:18.280484 16514 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:18.282747 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.288530 16550 raft_consensus.cc:1240] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:18.285588 16441 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.283584 16366 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:18.292716 17539 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:22:18.293365 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:18.281883 17538 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:18.301932 17531 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.294740 17553 raft_consensus.cc:2764] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:22:18.295639 17499 consensus_queue.cc:237] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:18.310531 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.311074 16441 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.294356 16516 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.297868 17447 raft_consensus.cc:2749] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.299935 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:18.313090 16551 raft_consensus.cc:2393] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 2: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:18.283000 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:18.288393 17517 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.294108 17518 raft_consensus.cc:3055] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:18.314510 17518 raft_consensus.cc:740] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:18.315655 17524 raft_consensus.cc:993] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50: : Instructing follower 52ab50b0237346c1a13ede947f6a63c2 to start an election
I20260709 12:22:18.315392 17518 consensus_queue.cc:260] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:18.316146 17524 raft_consensus.cc:1081] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Signalling peer 52ab50b0237346c1a13ede947f6a63c2 to start an election
I20260709 12:22:18.316898 17518 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.319190 16516 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.320942 16516 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.322796 16441 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.324283 16441 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.322044 17521 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.328892 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.331646 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:18.332629 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.332937 17520 raft_consensus.cc:2749] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.334975 17553 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.337623 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.337952 17527 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.338819 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.339989 17530 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.340144 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.340618 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.341310 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.341722 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.342893 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:18.317519 17539 raft_consensus.cc:697] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 3 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:18.346225 17539 consensus_queue.cc:237] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:18.321894 17435 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.348574 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.349022 17463 consensus_queue.cc:1059] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:18.350499 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.351409 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.352913 17463 raft_consensus.cc:3055] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:18.353317 17463 raft_consensus.cc:740] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:18.354140 17463 consensus_queue.cc:260] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:18.354709 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.354943 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.355419 17463 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.355995 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.356843 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:18.357307 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.322777 16366 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:18.361943 17520 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.364275 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.364616 16441 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:18.366163 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 43516.6us
I20260709 12:22:18.369827 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.370453 16441 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.371613 16366 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:18.374362 16366 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:18.376505 16441 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.380203 16441 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.375680 16442 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.369197 17530 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:18.384733 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.373490 17553 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.385974 16442 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:18.386556 17451 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.385993 16515 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:18.388556 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.389317 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.390017 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.393561 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:18.394599 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:18.394985 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:18.395381 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:18.383427 17447 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.382380 17516 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.398212 16441 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:18.398639 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:18.375787 17529 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.405361 16515 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.408493 17435 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:18.411712 17435 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:18.408493 17493 raft_consensus.cc:2749] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.412113 17435 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.407437 17143 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.373701 17520 consensus_queue.cc:1059] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 12:22:18.408350 16366 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:18.410488 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:18.416153 17143 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.405776 17517 raft_consensus.cc:2804] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:18.418040 16515 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.407950 17511 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:18.426098 16366 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:18.429545 16515 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.432055 17529 raft_consensus.cc:993] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50: : Instructing follower ef57beab0e1d40d585983873c55107e8 to start an election
I20260709 12:22:18.432578 17550 consensus_queue.cc:1059] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:18.435703 17529 raft_consensus.cc:1081] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Signalling peer ef57beab0e1d40d585983873c55107e8 to start an election
I20260709 12:22:18.436017 17550 raft_consensus.cc:3055] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:18.436470 17550 raft_consensus.cc:740] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:18.420850 17541 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.434223 17556 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:18.438680 16515 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.441170 17142 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:18.437261 17550 consensus_queue.cc:260] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:18.448757 17548 consensus_queue.cc:1059] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:18.449267 17550 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:18.453511 17541 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:18.440112 17556 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:18.454496 17142 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.454725 17556 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.455015 17523 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:18.455428 17523 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:18.455780 17523 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.459901 17525 raft_consensus.cc:993] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50: : Instructing follower ef57beab0e1d40d585983873c55107e8 to start an election
I20260709 12:22:18.461055 17525 raft_consensus.cc:993] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50: : Instructing follower ef57beab0e1d40d585983873c55107e8 to start an election
I20260709 12:22:18.461473 17525 raft_consensus.cc:1081] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Signalling peer ef57beab0e1d40d585983873c55107e8 to start an election
I20260709 12:22:18.462347 17525 raft_consensus.cc:1081] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Signalling peer ef57beab0e1d40d585983873c55107e8 to start an election
I20260709 12:22:18.463269 17536 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:18.463670 17536 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:18.464015 17536 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.441414 16515 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
W20260709 12:22:18.469651 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 53555.3us
I20260709 12:22:18.470206 16515 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.470485 17143 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.470981 17521 raft_consensus.cc:2804] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:18.471354 16515 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:18.471666 17143 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:18.471333 17521 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:18.420761 17539 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:22:18.474468 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 51722.8us
I20260709 12:22:18.473145 16515 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.426577 17513 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:18.427112 17530 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.476202 16515 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:18.471295 17518 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:22:18.475773 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 49778us
I20260709 12:22:18.472379 17521 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.486073 16366 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:18.472179 17535 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.472523 17517 raft_consensus.cc:697] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:18.485149 16515 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:18.474040 17511 raft_consensus.cc:697] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 3 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:18.473888 17539 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:18.517184 17539 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:18.517021 17511 consensus_queue.cc:237] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:18.473129 17537 consensus_queue.cc:1048] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 12:22:18.491017 16366 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:18.482632 17548 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:18.482321 17493 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:18.490178 17439 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:18.551167 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 96727.9us
I20260709 12:22:18.482805 17527 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:22:18.553261 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 95198.4us
I20260709 12:22:18.554044 17142 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.485365 17531 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:18.486487 17518 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:18.479202 17542 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:18.528208 16515 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:18.526420 17547 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.569535 17499 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.554329 17527 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:18.570935 17527 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.555238 17530 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:18.578933 17258 raft_consensus.cc:2410] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:22:18.580134 16515 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:18.581552 17517 consensus_queue.cc:237] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:18.586606 17531 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:18.587092 17531 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:18.587466 17531 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.595541 17531 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:18.597077 16515 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:18.554884 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 83291.7us
I20260709 12:22:18.533180 17537 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.533447 17535 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:18.560148 16367 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:18.556208 17493 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:18.603569 17143 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.604097 17493 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:18.604223 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 44164.1us
I20260709 12:22:18.605057 16367 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:18.605991 17499 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:18.606155 16367 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:18.606493 17499 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:18.606959 17499 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:18.607460 17530 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:18.607738 17463 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:18.608223 17530 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.561543 17524 raft_consensus.cc:993] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50: : Instructing follower ef57beab0e1d40d585983873c55107e8 to start an election
I20260709 12:22:18.609490 16367 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:18.610134 17524 raft_consensus.cc:1081] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Signalling peer ef57beab0e1d40d585983873c55107e8 to start an election
I20260709 12:22:18.610607 16367 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:18.611008 17524 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:18.611389 17528 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:18.562402 17516 raft_consensus.cc:3055] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:18.559823 17518 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.612524 17516 raft_consensus.cc:740] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:18.532840 17435 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:18.613276 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:18.613389 17516 consensus_queue.cc:260] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:18.614418 17435 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:18.614698 17516 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.615906 16367 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:18.615967 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:18.616678 17530 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.617974 16367 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:18.618657 17530 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.610153 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:18.619051 16515 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.619500 17566 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:22:18.619616 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 65657.7us
I20260709 12:22:18.620949 17142 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:18.622326 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 50033.3us
I20260709 12:22:18.623428 17547 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:18.624183 17547 consensus_queue.cc:1059] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:18.624737 17142 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.625460 17547 raft_consensus.cc:3055] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:18.625988 17547 raft_consensus.cc:740] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:18.626227 17541 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:18.626757 17524 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:18.627017 17547 consensus_queue.cc:260] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:18.628269 17142 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.628719 17547 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:18.629664 16367 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:18.631508 17142 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:18.633594 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 145764us
I20260709 12:22:18.634316 17142 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.634728 16441 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:18.635651 17142 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.637679 17531 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:18.638916 16399 raft_consensus.cc:1240] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:18.639183 16515 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:18.630002 17143 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:18.640717 17142 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.643815 17555 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.639365 17513 raft_consensus.cc:697] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:18.645989 17513 consensus_queue.cc:237] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:18.648528 17546 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:18.643554 16515 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.651348 17539 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:18.652666 17548 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.653759 17539 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:18.654673 17548 raft_consensus.cc:3055] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:18.655062 17548 raft_consensus.cc:740] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:18.632284 17537 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.657007 17537 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.657531 17142 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.649375 17524 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:18.659515 17538 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:18.660033 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:18.662518 17521 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:18.632366 17550 consensus_queue.cc:1059] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 12:22:18.639858 17569 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.641868 16399 raft_consensus.cc:1240] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:18.666056 17143 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:18.642794 17535 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:18.665542 17538 raft_consensus.cc:3055] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:18.667464 17143 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:18.641512 16441 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:18.649294 17546 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:18.672094 17538 raft_consensus.cc:740] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:18.676383 16441 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:18.713871 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 44296.2us
W20260709 12:22:18.713901 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 70421.4us
I20260709 12:22:18.712626 17447 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:18.720438 16550 raft_consensus.cc:1240] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 1. Current term is 2. Ops: []
W20260709 12:22:18.722105 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 54728.2us
I20260709 12:22:18.722929 17143 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:18.724851 16550 raft_consensus.cc:1240] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:18.650417 17511 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:18.718230 16441 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.715026 17540 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:18.732149 17567 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.732484 17568 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:18.640249 17536 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:18.671495 17569 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.730175 17143 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:18.739368 17143 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.656212 17527 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:18.668597 17535 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:18.642536 17523 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:18.735354 16441 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.754316 16441 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.642632 17542 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:18.655869 17548 consensus_queue.cc:260] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:18.760210 16441 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:18.744462 17541 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:18.744625 17493 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:18.763610 17493 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:18.764186 17523 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:18.730432 17537 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:18.764554 17571 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:18.765044 17571 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:18.765488 17451 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:18.765650 17571 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.765944 17451 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:18.766374 17451 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:18.766678 17553 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:18.742309 17143 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:18.741108 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:18.768298 17537 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.749979 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:18.719983 17447 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:18.769991 17447 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:18.733055 17568 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:18.771792 17568 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:18.772744 16548 raft_consensus.cc:1240] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:18.752087 17567 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.775887 16476 raft_consensus.cc:1275] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:18.770380 17536 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:18.760860 17511 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:18.777305 17511 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:18.777743 17511 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:18.779633 17548 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:18.783051 17527 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:18.767453 17518 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:18.785604 16551 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:18.769899 17143 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.788506 17555 consensus_queue.cc:1048] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:18.789070 17143 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:18.789460 17537 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.790350 17143 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:18.790709 17537 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.791149 17550 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.766098 16441 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:18.642148 17556 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:18.748971 17521 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:18.764982 16544 raft_consensus.cc:1240] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:18.794164 16441 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.791854 17143 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.793015 16547 raft_consensus.cc:1240] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:18.752525 17538 consensus_queue.cc:260] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:18.798223 17547 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.796631 17513 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.800244 16475 raft_consensus.cc:1240] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:22:18.753499 17535 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:18.802533 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 83806.7us
I20260709 12:22:18.797355 16441 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.805681 17143 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.813145 17143 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:18.815958 16516 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.816870 17143 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.818845 17143 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:18.821717 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:18.822758 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:18.824249 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:18.826272 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:18.828446 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:18.828960 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:18.835173 17143 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.840605 17518 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:18.798378 17553 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:18.801316 17538 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:18.842564 17563 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:18.843057 17563 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:18.843468 17563 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:18.801318 17556 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:18.846784 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:18.847524 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:18.848062 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:18.832767 17567 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:18.849642 17555 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.823843 17569 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:18.854946 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:18.856369 17499 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:18.874382 17499 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:18.877960 17568 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:18.882627 17571 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:18.883101 16547 raft_consensus.cc:1240] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:18.884735 16476 raft_consensus.cc:1217] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 12:22:18.885044 16544 raft_consensus.cc:1240] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:18.885401 16476 raft_consensus.cc:1240] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:22:18.823580 17547 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.851950 17447 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:18.852594 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:18.892114 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:18.894117 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:18.895228 16551 raft_consensus.cc:1240] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:18.855549 17537 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:22:18.894568 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 59471.4us
I20260709 12:22:18.887302 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:18.901328 17576 consensus_queue.cc:1059] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
W20260709 12:22:18.944483 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 50809.3us
I20260709 12:22:18.901531 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:18.912863 16474 raft_consensus.cc:1240] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:18.945505 16475 raft_consensus.cc:1275] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:18.904249 16224 catalog_manager.cc:5697] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 2 to 3, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 3 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
W20260709 12:22:18.944105 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 52803.2us
I20260709 12:22:18.947629 16515 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:18.947978 16224 catalog_manager.cc:5697] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 2 to 3, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 3 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:18.910398 17447 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:18.950662 16515 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.952915 16515 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.902822 17255 raft_consensus.cc:1275] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:18.954276 16515 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:18.955267 17521 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.955643 17556 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.956848 16515 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.896488 17567 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.950881 17527 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.896817 17568 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:18.960875 16515 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.948717 17143 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.899677 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:18.969050 16515 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:18.950258 17451 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:18.957231 17571 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:18.956230 17523 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.965626 17143 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:18.972283 17499 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.963763 17556 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.952292 17536 consensus_queue.cc:1059] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 12:22:18.978155 17143 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:19.009039 17536 raft_consensus.cc:3055] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:19.009460 17143 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:19.010016 17451 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:19.010771 17143 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:19.011396 17535 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:18.953840 17531 consensus_queue.cc:1048] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 12:22:18.968307 17515 consensus_queue.cc:1059] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 12:22:18.973424 17529 consensus_queue.cc:1059] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 12:22:18.950095 17511 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:18.953763 17493 consensus_queue.cc:1059] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 12:22:18.978380 16515 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:18.963981 17528 consensus_queue.cc:1059] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 12:22:18.966106 17537 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.970381 17463 consensus_queue.cc:1059] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 12:22:18.970778 17521 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:19.007793 17524 raft_consensus.cc:3055] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:19.011857 17538 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:18.937009 17563 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:18.956691 17550 consensus_queue.cc:1059] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 12:22:18.992712 17547 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:19.019749 17261 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835"
dest_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:45230
I20260709 12:22:19.046021 16551 raft_consensus.cc:1240] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 3 LEADER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:19.046778 17261 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:19.022007 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:22:19.023229 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 44924.8us
I20260709 12:22:18.976935 17517 consensus_queue.cc:1059] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 12:22:19.025701 16549 raft_consensus.cc:1240] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:19.013247 17567 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:19.055794 17439 raft_consensus.cc:3055] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:19.056321 17439 raft_consensus.cc:740] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:19.057155 17576 raft_consensus.cc:3055] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:19.057976 17576 raft_consensus.cc:740] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:19.057284 17439 consensus_queue.cc:260] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:19.059638 17439 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:19.059118 17576 consensus_queue.cc:260] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:19.060074 17517 raft_consensus.cc:3055] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:19.060599 17517 raft_consensus.cc:740] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:19.060882 17463 raft_consensus.cc:3055] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:19.061324 17463 raft_consensus.cc:740] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:19.061661 17517 consensus_queue.cc:260] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:19.062291 17463 consensus_queue.cc:260] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:19.063058 17517 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:19.063581 17463 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:22:19.064560 17577 log.cc:927] Time spent T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
W20260709 12:22:19.064700 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 48837.4us
I20260709 12:22:19.065783 17535 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:19.065932 17529 consensus_queue.cc:1059] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:22:19.065814 17574 log.cc:927] Time spent T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:22:19.066987 17529 consensus_queue.cc:1059] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:19.067373 17530 raft_consensus.cc:3055] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:19.069819 17565 raft_consensus.cc:3055] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:19.070257 17565 raft_consensus.cc:740] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:19.071118 17565 consensus_queue.cc:260] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:19.073247 17567 consensus_queue.cc:1059] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:19.025703 17499 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:22:19.073699 17548 raft_consensus.cc:3055] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:19.039110 17258 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159"
dest_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:45230
I20260709 12:22:19.034360 17511 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:19.009466 17536 raft_consensus.cc:740] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:19.075079 17258 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:19.075594 17258 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:19.068015 17529 raft_consensus.cc:3055] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:19.076159 17529 raft_consensus.cc:740] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:19.075836 17536 consensus_queue.cc:260] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:19.077339 17536 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:19.077137 17529 consensus_queue.cc:260] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:19.079195 16549 raft_consensus.cc:1240] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:19.047354 17261 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:19.032711 17271 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0"
dest_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:45230
I20260709 12:22:19.047854 17255 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070"
dest_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:45230
I20260709 12:22:19.083619 17271 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:19.083753 17255 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:19.049050 16515 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:19.084118 17271 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:19.084321 17255 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:19.042658 17524 raft_consensus.cc:740] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:19.073873 17576 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:19.086045 17524 consensus_queue.cc:260] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:19.087352 17524 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:19.013556 17569 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:19.074288 17567 consensus_queue.cc:1243] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer 610f4feda11942739236f2b121a4b9c1 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:22:19.074303 17548 raft_consensus.cc:740] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:19.076398 17563 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:19.078496 17529 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:19.093842 17548 consensus_queue.cc:260] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:19.094238 17261 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:19.095335 17269 raft_consensus.cc:1240] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:19.099320 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:19.099996 16396 raft_consensus.cc:3055] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:19.100982 16396 raft_consensus.cc:740] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:19.102270 16396 consensus_queue.cc:260] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:19.102854 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:19.103376 16396 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:19.103809 16475 raft_consensus.cc:2393] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 3: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:19.068197 16367 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:19.107244 17515 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:19.041318 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 231466us
I20260709 12:22:19.108081 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:19.111264 17493 raft_consensus.cc:3055] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:19.111802 17493 raft_consensus.cc:740] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:19.111527 16441 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:19.079566 16516 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:19.113912 16441 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:19.114111 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 46020.5us
I20260709 12:22:19.115149 16367 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:19.113921 17261 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:19.067845 17530 raft_consensus.cc:740] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:19.116662 17563 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:19.117301 17563 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:19.117368 17530 consensus_queue.cc:260] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:19.117916 17563 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:19.118917 17530 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:19.120040 17255 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:19.095347 17548 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:19.123303 17255 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:19.124672 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:19.129685 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:19.131112 16551 raft_consensus.cc:2393] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 2: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:19.132732 17565 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:19.114969 17258 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:19.143577 16367 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:19.143761 16441 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:19.145589 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:19.147334 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 98390.3us
I20260709 12:22:19.147670 16367 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:19.148938 16367 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:19.154738 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:19.158771 17258 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:22:19.160852 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 81380.8us
I20260709 12:22:19.131718 17493 consensus_queue.cc:260] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:19.163740 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:19.169064 16441 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:19.145371 17524 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:19.177451 16515 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:19.155557 17531 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:19.156292 17528 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:19.150390 17529 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:19.204244  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:19.187959 17578 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:19.188601 17580 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:19.164754 17571 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:19.195226 17511 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:19.194290 16441 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:19.196182 17493 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:19.170156 17499 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:22:19.201982 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:19.211804 16399 raft_consensus.cc:3055] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:19.212157 16399 raft_consensus.cc:740] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:19.200429 17536 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:19.155465 17521 consensus_queue.cc:1048] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 12:22:19.206200 17515 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:19.212874 17512 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:19.207695 17271 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:19.218041 17271 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:19.221978 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:19.222823 16551 raft_consensus.cc:2393] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 2: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:19.213052 16399 consensus_queue.cc:260] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:19.225415 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:19.225745 16399 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:19.226174 16395 raft_consensus.cc:3055] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:19.226531 16395 raft_consensus.cc:740] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:19.227789 16395 consensus_queue.cc:260] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:19.228967 16395 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:19.237085 16399 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:19.238323 17144 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:19.239303 17569 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:19.210631 17511 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:19.239912 17569 raft_consensus.cc:697] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:19.239964 17511 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:19.241180 17569 consensus_queue.cc:237] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:19.231902 16398 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:19.254125 16548 raft_consensus.cc:1240] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:19.258309 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:19.259379 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:19.260021 17258 raft_consensus.cc:1275] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:22:19.260592 17258 pending_rounds.cc:85] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8: Aborting all ops after (but not including) 1
I20260709 12:22:19.264230 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:19.266559 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:19.267109 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:19.260809 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:19.260917 17258 pending_rounds.cc:107] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:22:19.271742 17258 raft_consensus.cc:2889] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:19.277248 17270 raft_consensus.cc:1275] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:19.285691 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:19.270108 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:19.263013 17563 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:19.291281 16476 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:34840
I20260709 12:22:19.291975 16476 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:19.292511 16468 raft_consensus.cc:1240] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:19.292840 16476 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:19.293804 16471 raft_consensus.cc:1240] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:19.275872 17447 consensus_queue.cc:1059] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:22:19.295161 16468 raft_consensus.cc:1240] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:19.295347 16475 raft_consensus.cc:2393] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 2: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:19.302241 16223 catalog_manager.cc:5697] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 2 to 3, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 3 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } }
W20260709 12:22:19.312356 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:19.302966 16471 raft_consensus.cc:1240] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:19.313968 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:19.288861 17520 consensus_queue.cc:1059] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 12:22:19.297937 17563 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:19.323060 17511 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:19.324301 16472 raft_consensus.cc:1240] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:19.329522 16395 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:19.329830 17536 consensus_queue.cc:1059] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:19.331022 17536 raft_consensus.cc:3055] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:19.331118 17144 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:19.331432 17536 raft_consensus.cc:740] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:19.309827 16396 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:19.332554 17567 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:19.333052 17567 raft_consensus.cc:697] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:19.333482 17144 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:19.334200 16473 raft_consensus.cc:1240] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:19.333930 17567 consensus_queue.cc:237] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:19.334391 16474 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
I20260709 12:22:19.334798 17550 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:19.335287 17550 raft_consensus.cc:697] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:19.335418 16550 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:19.336180 17550 consensus_queue.cc:237] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:19.315237 17493 consensus_queue.cc:1059] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 12:22:19.339371 17493 raft_consensus.cc:3055] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:19.339845 17493 raft_consensus.cc:740] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:19.347888 17521 consensus_queue.cc:1048] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:19.350247 17536 consensus_queue.cc:260] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:19.353307 16551 raft_consensus.cc:1240] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:19.353765 17144 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:19.359076 17547 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:19.359565 17547 raft_consensus.cc:697] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:19.360615 16476 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:19.365197 16477 raft_consensus.cc:1240] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:19.368492 16476 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:19.373785 16468 raft_consensus.cc:1240] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:19.385263 17565 consensus_queue.cc:1059] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:19.385651 17517 raft_consensus.cc:3055] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:19.386482 17517 raft_consensus.cc:740] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:19.387280 17528 raft_consensus.cc:3055] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:19.386453 17565 consensus_queue.cc:1059] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:19.387720 17528 raft_consensus.cc:740] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:19.388464 17565 raft_consensus.cc:3055] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:19.381510 17447 raft_consensus.cc:3055] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:19.365252 17547 consensus_queue.cc:237] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:19.399168 17565 raft_consensus.cc:740] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:19.383496 17536 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:22:19.410059 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 253047us
I20260709 12:22:19.405223 17447 raft_consensus.cc:740] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:22:19.429812 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:19.381381 17493 consensus_queue.cc:260] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:19.437654 17515 consensus_queue.cc:1175] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ef57beab0e1d40d585983873c55107e8" responder_term: 2 status { last_received { term: 2 index: 2 } last_committed_idx: 2 last_received_current_leader { term: 2 index: 2 } }
I20260709 12:22:19.430583 17520 consensus_queue.cc:1048] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:22:19.441802 17493 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:19.443629 17447 consensus_queue.cc:260] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:19.445019 17447 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:19.449638 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:19.436024 17565 consensus_queue.cc:260] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:19.463096 17517 consensus_queue.cc:260] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:19.492298 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RunLeaderElection -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 54450.3us
I20260709 12:22:19.501614 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:19.418216 17528 consensus_queue.cc:260] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:19.506059 16402 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:19.500140 17517 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:19.507186 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:19.508522 16403 raft_consensus.cc:1275] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:22:19.509765 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 73454.1us
I20260709 12:22:19.502390 17565 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:19.501315 17511 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:19.522604 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:19.509466 16396 raft_consensus.cc:1240] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:19.535851 16396 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:19.536249 17537 consensus_queue.cc:1059] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:19.537366 17537 raft_consensus.cc:3055] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:19.537698 17537 raft_consensus.cc:740] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:19.538511 17537 consensus_queue.cc:260] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:19.539804 17537 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:19.539978 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:19.547533 16397 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:19.523592 17528 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:19.549938 17547 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:19.550482 17547 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:19.552760 17547 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:19.554301 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:19.555215 16394 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:19.556560 17144 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:19.556941 16551 raft_consensus.cc:1240] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:19.557370 17547 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:19.557960 17547 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:19.558684 17547 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:19.526813 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:19.560778 16399 raft_consensus.cc:2393] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 2: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:19.532809 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:19.561745 16398 raft_consensus.cc:3055] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:19.562086 16398 raft_consensus.cc:740] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:19.563026 16398 consensus_queue.cc:260] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:19.564049 16398 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:22:19.534070 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:19.511430 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:19.558207 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:19.571004 16403 raft_consensus.cc:2393] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 3: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:19.571061 17520 consensus_queue.cc:1048] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:19.575601 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:22:19.578897 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 54204.9us
I20260709 12:22:19.580035 16441 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:19.586712 16398 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
W20260709 12:22:19.592319 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:19.596307 17547 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:19.601655 17447 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:19.602264 17447 raft_consensus.cc:697] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 3 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:19.603201 17447 consensus_queue.cc:237] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:19.598999 16402 raft_consensus.cc:1275] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:22:19.524534 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:19.605295 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:22:19.606035 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:19.606499 16366 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:19.607009 16402 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb"
dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:60492
I20260709 12:22:19.607717 17439 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:19.607349 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:19.610024 16398 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:19.610210 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:19.610773 16550 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:19.611466 16551 raft_consensus.cc:1240] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 1. Current term is 2. Ops: []
W20260709 12:22:19.612425 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:19.613022 17270 raft_consensus.cc:1240] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:22:19.620174 17439 consensus_queue.cc:1059] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:19.622205 16398 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:19.618330 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:19.623597 17595 consensus_queue.cc:1059] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:19.624720 16366 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:19.625946 17576 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:19.626425 17576 raft_consensus.cc:697] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:19.627185 17439 raft_consensus.cc:3055] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:19.618378 17451 consensus_queue.cc:1059] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 12:22:19.627498 17144 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:19.628765 17451 raft_consensus.cc:3055] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:19.629137 17451 raft_consensus.cc:740] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:19.629969 17451 consensus_queue.cc:260] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:19.633067 17595 raft_consensus.cc:3055] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:19.633569 17595 raft_consensus.cc:740] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:19.633827 16542 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:19.634131 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:19.634523 17595 consensus_queue.cc:260] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:19.627285 17576 consensus_queue.cc:237] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:19.636113 17595 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:19.636516 17439 raft_consensus.cc:740] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:19.637424 17439 consensus_queue.cc:260] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:19.638897 17439 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:19.640297 16550 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:19.644979 17567 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:19.641583 17528 consensus_queue.cc:1059] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 12:22:19.646637 17567 raft_consensus.cc:697] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:19.650457 17515 raft_consensus.cc:3055] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:19.651072 17515 raft_consensus.cc:740] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:19.651633 16396 raft_consensus.cc:1275] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:19.652361 17515 consensus_queue.cc:260] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:19.658581 16441 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:19.659518 16441 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:19.659616 17520 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:19.660759 17520 raft_consensus.cc:697] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:19.659782 17567 consensus_queue.cc:237] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:19.661216 16442 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:19.669809 17516 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:19.661752 17520 consensus_queue.cc:237] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:19.646605 16400 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:19.672783 16402 raft_consensus.cc:606] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:22:19.673146 16402 raft_consensus.cc:614] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:19.659617 17515 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:19.608556 17547 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:19.666736 16396 raft_consensus.cc:1275] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:19.668205 17451 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:19.684670 17516 raft_consensus.cc:697] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:19.684898 17517 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:22:19.685158 16442 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:19.685734 17516 consensus_queue.cc:237] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:19.687252 16442 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:19.688009 17571 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:19.688501 17511 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:22:19.689288 17571 raft_consensus.cc:697] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:19.690259 17571 consensus_queue.cc:237] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:19.691809 16442 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:19.690984 17547 consensus_queue.cc:1048] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 12:22:19.698549 17511 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:19.698010 16542 raft_consensus.cc:1275] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:19.714161 16442 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:19.697451 16398 raft_consensus.cc:1240] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:22:19.690876 16401 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:19.697110 16550 raft_consensus.cc:1275] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:19.700312 17595 consensus_queue.cc:1048] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:19.701094 16396 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90"
dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:60492
I20260709 12:22:19.717872 16396 raft_consensus.cc:606] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:22:19.718286 16396 raft_consensus.cc:614] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:19.720427 17580 consensus_queue.cc:1048] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:19.722445 17595 consensus_queue.cc:1048] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:19.725827 16401 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5"
dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
mode: GRACEFUL
new_leader_uuid: "610f4feda11942739236f2b121a4b9c1"
 from {username='slave'} at 127.0.0.1:60492
I20260709 12:22:19.726423 16401 raft_consensus.cc:606] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Received request to transfer leadership to TS 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:19.726764 16401 raft_consensus.cc:614] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:19.728965 16442 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:19.731185 16477 raft_consensus.cc:1275] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:19.732733 17451 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:19.755414 17451 raft_consensus.cc:697] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 3 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:19.756691 17451 consensus_queue.cc:237] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:19.731019 17568 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:19.732254 17517 consensus_queue.cc:1048] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 12:22:19.787084 16400 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1"
dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:60492
I20260709 12:22:19.789176 16400 raft_consensus.cc:606] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:19.789639 16400 raft_consensus.cc:614] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:19.786037 17568 raft_consensus.cc:697] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:19.752713 17547 consensus_queue.cc:1048] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 12:22:19.803098 16477 raft_consensus.cc:1275] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:19.814544 16400 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b"
dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:60492
I20260709 12:22:19.815348 16400 raft_consensus.cc:606] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:22:19.815690 16400 raft_consensus.cc:614] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:19.818305  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:22:19.807149 17567 consensus_queue.cc:1048] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 12:22:19.820232  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:19.799192 17568 consensus_queue.cc:237] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:19.836421 16400 raft_consensus.cc:1275] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:19.845347 17537 consensus_queue.cc:1048] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:19.847183 16224 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53636 (request call id 128) took 1005 ms. Trace:
I20260709 12:22:19.847607 16224 rpcz_store.cc:276] 0709 12:22:18.841868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:18.857529 (+ 15661us) service_pool.cc:224] Handling call
0709 12:22:19.847095 (+989566us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:22:19.247014 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:22:19.247642 (+   628us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:22:19.247682 (+    40us) write_op.cc:435] Acquired schema lock
0709 12:22:19.247701 (+    19us) tablet.cc:662] Decoding operations
0709 12:22:19.279067 (+ 31366us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:22:19.281075 (+  2008us) write_op.cc:641] Partition lock acquired for write op
0709 12:22:19.282799 (+  1724us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:22:19.309860 (+ 27061us) tablet.cc:701] Row locks acquired
0709 12:22:19.312876 (+  3016us) write_op.cc:260] PREPARE: finished
0709 12:22:19.320059 (+  7183us) write_op.cc:270] Start()
0709 12:22:19.342631 (+ 22572us) write_op.cc:276] Timestamp: P: 1783599739320004 usec, L: 0
0709 12:22:19.342664 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 12:22:19.343431 (+   767us) log.cc:844] Serialized 910 byte log entry
0709 12:22:19.667006 (+323575us) op_driver.cc:464] REPLICATION: finished
0709 12:22:19.728109 (+ 61103us) write_op.cc:301] APPLY: starting
0709 12:22:19.732869 (+  4760us) tablet.cc:1367] starting BulkCheckPresence
0709 12:22:19.753561 (+ 20692us) tablet.cc:1370] finished BulkCheckPresence
0709 12:22:19.788672 (+ 35111us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:22:19.811415 (+ 22743us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:22:19.811611 (+   196us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 12:22:19.811667 (+    56us) write_op.cc:312] APPLY: finished
0709 12:22:19.811877 (+   210us) log.cc:844] Serialized 28 byte log entry
0709 12:22:19.812084 (+   207us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:19.812174 (+    90us) write_op.cc:454] Released schema lock
0709 12:22:19.812230 (+    56us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":50222,"apply.run_wall_time_us":107142,"num_ops":2,"prepare.queue_time_us":288831,"prepare.run_cpu_time_us":56477,"prepare.run_wall_time_us":240239,"raft.queue_time_us":158424,"raft.run_cpu_time_us":28109,"raft.run_wall_time_us":141202,"replication_time_us":324178,"spinlock_wait_cycles":16252160,"thread_start_us":441502,"threads_started":4,"wal-append.queue_time_us":167512}]]}
I20260709 12:22:19.821113  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:19.864883 17515 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:19.877422 17515 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:19.879606 17515 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:19.887324 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:19.889394 16476 raft_consensus.cc:2393] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 4: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:19.892057 16551 raft_consensus.cc:1275] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:22:19.891558 17613 log.cc:927] Time spent T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 12:22:19.902927 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:19.915635 17270 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:19.931044 16367 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:19.906689 17537 consensus_queue.cc:1048] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 12:22:19.949908 17515 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:19.959338 17515 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:19.959779 17515 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:20.005120 17576 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:20.019017 16398 raft_consensus.cc:1275] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:20.005813 17576 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:20.061913 17595 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:20.062750 17595 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:20.029192 17579 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:20.073894 17579 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:20.077088 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:20.083283 16402 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:22:20.079698 16403 raft_consensus.cc:1275] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:20.085326 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:20.086032 16398 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:22:20.087708 16442 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:20.081485 16225 catalog_manager.cc:5697] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 1 to 2, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 2 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:20.065260 17595 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:20.089538 16225 catalog_manager.cc:5697] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 1 to 2, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 2 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:20.091001 16225 catalog_manager.cc:5697] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 2 to 3, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 3 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:20.094661 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:20.113291 16544 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
I20260709 12:22:20.095168 16516 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:20.114245 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:20.115664 16367 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:20.116452 17439 raft_consensus.cc:2804] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:20.116888 17439 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:20.117342 17439 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:20.119962 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:20.120814 16474 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:22:20.129153 17516 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:20.076303 17579 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:20.080435 17576 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:20.136674 16475 raft_consensus.cc:1275] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:20.091355 17517 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:20.137957 17439 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:20.114244 17521 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:20.128862 17581 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:20.129760 17527 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:20.081363 17537 consensus_queue.cc:1048] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.055s
I20260709 12:22:20.151904 17439 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:20.137240 17565 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:20.143304 17517 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:20.174670 16475 raft_consensus.cc:1275] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:22:20.175477 16475 pending_rounds.cc:85] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 1
I20260709 12:22:20.175803 16475 pending_rounds.cc:107] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:22:20.129719 17516 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:20.176165 16475 raft_consensus.cc:2889] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:20.150591 17527 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:20.153501 17550 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:20.181983 17439 consensus_queue.cc:1048] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:20.181603 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:20.183116 16544 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:20.181638 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:20.169243 17517 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:20.189128 16474 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:22:20.149765 17538 consensus_queue.cc:1048] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 12:22:20.152482 17521 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:20.194967 16551 raft_consensus.cc:1275] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:20.197817 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:20.154099 17581 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:20.203908 16548 raft_consensus.cc:1275] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:20.184583 17527 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:20.185001 17576 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:20.207659 16474 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 3.
I20260709 12:22:20.169436 17565 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:20.212965 17567 consensus_queue.cc:1048] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:22:20.216509 17581 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:20.216974 17576 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:20.219982 17254 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:20.226065 17521 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:20.188266 17550 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:20.229063 16551 raft_consensus.cc:1275] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:20.236871 16544 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 3.
I20260709 12:22:20.239039 16367 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:20.230846 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:20.180954 17515 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:20.246420 17439 raft_consensus.cc:2804] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:20.247398 17576 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:20.246474 16475 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:20.250082 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:20.250800 16401 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:22:20.251065 16514 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:20.211463 17579 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:20.206490 17547 consensus_queue.cc:1048] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 12:22:20.265913 17439 raft_consensus.cc:697] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 3 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:20.267469 17439 consensus_queue.cc:237] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:20.274405 17515 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:22:20.274621 17614 log.cc:927] Time spent T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.214s	user 0.006s	sys 0.009s
I20260709 12:22:20.276055 17565 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:20.258562 17579 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:20.279992 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 69911.7us
I20260709 12:22:20.256474 17576 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:20.267961 17521 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:20.281899 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:20.248430 17528 consensus_queue.cc:1048] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 12:22:20.258333 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:20.284801 16401 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:20.291549 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:20.292167 16400 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:22:20.292968 16402 raft_consensus.cc:1275] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:20.294312 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:20.295156 16475 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:20.290234 17271 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:20.297017 17631 consensus_queue.cc:1048] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:20.287207 17550 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:20.300491 17254 raft_consensus.cc:1275] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:20.303320 17527 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:20.304633 16550 raft_consensus.cc:2393] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 3: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:20.305688 17579 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:20.289403 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:20.286412 17521 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:20.307601 17521 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:20.310366 16401 raft_consensus.cc:1275] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:20.311146 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:22:20.311995 17618 log.cc:927] Time spent T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:22:20.312160 16477 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:20.314057 17517 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:20.315303 16468 raft_consensus.cc:2393] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 3: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:20.318639 17581 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:20.319414 17581 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:20.320808 17527 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:20.324062 17521 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:20.328720 16401 raft_consensus.cc:1275] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:20.330978 17142 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:20.335664 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:20.336695 16401 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:22:20.337877 16367 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:20.339584 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:20.343238 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:20.288615 17576 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:20.338553 17547 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:22:20.347492 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 81254.8us
I20260709 12:22:20.347332 16442 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:20.341439 17528 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:20.357547 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:20.358301 16400 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:20.351399 17547 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:20.356280 16542 raft_consensus.cc:2368] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 2. Current term is 3.
I20260709 12:22:20.354430 17528 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:20.357582 16402 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:20.361333 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:20.361691 16401 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:20.363559 16516 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:20.364478 17579 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:20.364838 17579 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:20.365175 17579 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:20.361677 17528 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:20.367837 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:20.371810 17269 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:20.374003 16367 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:20.374997 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:20.358916 17571 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:20.379097 17571 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:20.379498 17571 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:20.375604 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:20.379884 17631 consensus_queue.cc:1048] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:20.368877 17447 consensus_queue.cc:1048] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:22:20.376376 17439 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:20.384383 17439 raft_consensus.cc:697] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 4 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:20.360560 17547 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:20.377373 16442 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:20.385633 17439 consensus_queue.cc:237] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:20.387800 16514 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:20.399508 17571 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:20.399935 17516 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:20.408151 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:20.411432 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:20.412227 16399 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:22:20.406160 17447 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:20.401999 17571 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:20.416687 17624 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:20.375811 17632 consensus_queue.cc:1048] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.068s
I20260709 12:22:20.469887 16399 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:20.432219 17634 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:22:20.474005 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 66605.7us
I20260709 12:22:20.474903 16442 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:20.480136 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 66205.2us
I20260709 12:22:20.467780 17447 raft_consensus.cc:697] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:20.482179 17516 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:20.486495 17516 raft_consensus.cc:697] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 3 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:22:20.483037 17639 log.cc:927] Time spent T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:22:20.487725 17521 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:20.487483 17516 consensus_queue.cc:237] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:20.489969 17447 consensus_queue.cc:237] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:20.496745 17576 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:20.497517 17547 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:20.507166 16397 raft_consensus.cc:1275] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:22:20.510485 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:20.511121 16401 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:20.517473 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:20.480051 17634 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:20.523356 17634 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:20.539124 16401 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:20.526750 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:20.523977 17517 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:20.552058 17650 consensus_queue.cc:1048] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:20.529429 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:20.556550 16551 raft_consensus.cc:2393] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 2: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:20.531015 17547 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:20.553601 17565 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:20.570166 17581 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:20.573292 17531 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:20.573959 17531 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:20.555833 17550 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:20.618832 17144 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:20.619860 17652 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:20.621006 17656 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:20.625253 17654 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:20.653324 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:20.684993 17657 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:20.685680 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:20.685655 17657 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:20.636933 17550 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:20.689841 17269 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:20.633713 17654 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:20.691265 17550 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:20.654191 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:20.694635 16468 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:20.695351 17439 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:20.695833 17439 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:20.585947 17565 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:20.697368 17576 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:20.697989 16403 raft_consensus.cc:1275] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:20.707770 17439 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:20.711336 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 53291.1us
I20260709 12:22:20.711393 16551 raft_consensus.cc:1275] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:20.710539 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:20.713382 16550 raft_consensus.cc:2393] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 3: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:20.720495 17633 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:20.693531 17648 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:20.730366 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:20.731900 16551 raft_consensus.cc:2368] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3.
I20260709 12:22:20.696691 17550 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:20.754220 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:20.632525 17652 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:20.711175 17634 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:20.721208 17633 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:20.765503 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:20.729406 17528 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:20.772886 17565 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:20.729095 17567 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:20.776232 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:20.793694 16474 raft_consensus.cc:2393] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 4: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:20.776991 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:20.777909 17652 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:20.715034 17576 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:20.780496 17571 consensus_queue.cc:1048] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:20.755563 17579 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:20.631162 17656 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:20.781405 17451 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:20.781605 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 44753.7us
I20260709 12:22:20.749490 17648 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:20.782100 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:20.770802 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:20.787645 16551 raft_consensus.cc:1275] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:20.715969 17657 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:20.794193 16476 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 3.
I20260709 12:22:20.791378 17625 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:20.735841 17654 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:20.795325 16468 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:20.794974 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:20.802273 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:20.701313 17658 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:20.829948 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:20.792788 17515 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:20.791061 17642 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:20.833750 16441 leader_election.cc:400] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3.
I20260709 12:22:20.836495 16544 raft_consensus.cc:2393] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 2: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:20.837815 16394 raft_consensus.cc:2393] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 3: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:20.840866 16548 raft_consensus.cc:2393] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 3: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:20.756461 17550 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:20.857461 16441 leader_election.cc:403] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:20.803777 17581 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:22:20.865339 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 66617.4us
I20260709 12:22:20.863366 17439 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:20.792569 17634 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:20.799734 17630 consensus_queue.cc:1048] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.061s
I20260709 12:22:20.867290 17439 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:20.882907 17142 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:20.884904 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:20.798581 17567 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:20.860805 17631 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:20.885855 16402 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:20.892683 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:20.892956 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:20.893362 16402 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:20.897662 17439 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:20.893518 16545 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
I20260709 12:22:20.857618 17447 consensus_queue.cc:1048] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 12:22:20.803754 17451 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:20.880228 17548 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:20.860936 17515 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:20.887015 17517 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3.
I20260709 12:22:20.910518 17654 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:20.928139 17654 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:20.928547 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:20.929250 16403 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:20.923355 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:20.930675 17654 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:20.919471 17266 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:20.932991 17528 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:20.917785 17548 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:20.921851 17656 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:20.935714 17548 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:20.922973 17144 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:20.921855 17531 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:20.927495 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:20.937215 17144 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:20.937580 17271 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:22:20.938242 17144 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:20.938679 17567 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:20.939260 17567 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:20.939594 17567 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:20.939940 17567 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:20.840061 17625 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:20.941154 16367 leader_election.cc:400] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 2. Current term is 3.
I20260709 12:22:20.941519 16367 leader_election.cc:403] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:20.941768 16550 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:20.894366 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:20.890350 17658 raft_consensus.cc:697] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:20.942642 16367 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:20.920463 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:20.909827 17270 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:20.944137 16551 raft_consensus.cc:1275] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:20.938453 17657 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:20.954553 17439 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 2. Current term is 3.
W20260709 12:22:20.998859 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 161635us
I20260709 12:22:20.941779 16515 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:20.862421 17642 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:20.886411 17581 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:20.899739 17661 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:22:21.036850 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 94269.2us
I20260709 12:22:20.923882 17255 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:21.037978 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 96272.4us
I20260709 12:22:21.041005 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:20.922067 17659 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:21.127077 16551 raft_consensus.cc:2393] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 3: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:21.131386 17439 consensus_queue.cc:1048] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 12:22:21.008716 17537 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:22:21.078284 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 211691us
I20260709 12:22:21.028384 17624 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:21.143347 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:21.147272 16401 raft_consensus.cc:2393] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 3: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:21.031556 17657 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:21.139276 17258 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:22:21.139842 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:20.897907 17631 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:21.149206 16397 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
W20260709 12:22:21.155803 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 88753.4us
I20260709 12:22:21.157100 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:21.157370 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:21.157737 16401 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:21.159956 17270 raft_consensus.cc:1275] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:21.157898 16476 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:21.163520 16514 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:21.164500 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:21.165115 16551 raft_consensus.cc:2393] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 2: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:21.169247 17266 raft_consensus.cc:1275] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:21.171257 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:21.171880 17270 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:21.134709 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:21.041368 17535 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:21.135600 17625 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:22:21.029841 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 90637us
I20260709 12:22:21.179330 17568 consensus_queue.cc:1048] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:21.171396 17576 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:21.207206 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 44389.1us
I20260709 12:22:21.208918 16516 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:21.182955 16401 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:21.177281 17654 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:22:21.183737 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 115868us
I20260709 12:22:21.186678 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:21.187539 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:21.216144 16475 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:21.127532 17650 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:21.192691 16474 raft_consensus.cc:1275] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:21.151249 17537 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:21.162189 17657 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:21.194902 17270 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:21.184952 17579 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:21.158955 16473 raft_consensus.cc:2393] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 2: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:21.221585 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:21.200742 17633 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:21.200438 17447 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:21.223732 16401 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:21.196885 16546 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
W20260709 12:22:21.213361 17636 log.cc:927] Time spent T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:22:21.209609 17531 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:21.133805 17659 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:21.244249 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 57240.4us
I20260709 12:22:21.197084 17535 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:21.165781 17624 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:21.241469 16475 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:21.246578 17451 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:21.244302 17531 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:21.250615 17651 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:21.230187 17550 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:21.222483 17537 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:21.224073 17571 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:21.247318 17648 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:21.292564 17531 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:21.304422 17668 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:21.304980 17668 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:21.305305 17668 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:21.237819 17447 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:21.308521 17677 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:22:21.309033 17662 log.cc:927] Time spent T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.279s	user 0.001s	sys 0.000s
W20260709 12:22:21.310231 17675 log.cc:927] Time spent T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:22:21.197057 17658 consensus_queue.cc:237] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:21.308601 17664 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:21.313581 17664 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:21.315958 16468 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
W20260709 12:22:21.294216 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 145623us
I20260709 12:22:21.239552 17634 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:21.344637 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:21.221453 17654 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:21.345675 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 91470.8us
I20260709 12:22:21.365793 16401 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:21.292618 17641 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:21.375314 17684 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:21.376377 16442 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:21.229825 17650 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:21.393778 17515 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:21.397748 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 71634.5us
I20260709 12:22:21.402050 17679 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:21.402244 17579 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:21.402566 17679 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:21.402951 17659 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:21.402894 17579 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:21.403878 17642 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:21.400326 16516 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:21.405689 17677 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:21.406309 17677 raft_consensus.cc:697] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 3 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:21.406848 17679 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:21.408854 17535 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:21.408659 17684 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:21.391916 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 228450us
I20260709 12:22:21.412151 17664 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:21.411837 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:21.407371 17677 consensus_queue.cc:237] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:21.315649 17651 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:21.413497 17672 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:21.413916 17672 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:21.414235 17672 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:21.414655 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:21.414969 17651 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:21.415345 17264 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:22:21.407366 17144 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:21.417383 17550 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:21.417644 17624 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:21.417821 17550 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:21.418322 17550 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:21.420161 17579 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:21.422439 16515 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:21.423588 17672 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:21.412894 16476 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:22:21.424873 17625 raft_consensus.cc:2804] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:21.425343 17625 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:21.425683 17625 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:21.428457 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 52218.7us
I20260709 12:22:21.428690 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:21.429745 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:21.432622 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:21.433571 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:21.435643 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:21.436118 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:21.436590 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:21.440834 17142 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:21.335084 16366 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
W20260709 12:22:21.442270 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 107280us
I20260709 12:22:21.434161 17264 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:21.443972 16516 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:21.446303 16515 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:21.344118 17633 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:21.448693 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 202532us
I20260709 12:22:21.449836 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:21.450471 17270 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:22:21.450673 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:21.451749 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:21.452625 16397 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:21.453485 16515 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:21.456207 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:21.456844 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:21.457464 16477 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:21.458007 16397 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:21.460366 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:21.461566 16516 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:21.462432 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:21.465210 16476 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:21.465291 17672 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:21.467576 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:21.390231 17641 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:21.469121 17263 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:22:21.468257 16477 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:21.345278 17571 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:21.475934 16366 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:21.477567 16551 raft_consensus.cc:1275] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:21.448364 17659 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:21.459587 17579 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:21.460685 17679 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:21.485818 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:21.487279 16399 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:21.471530 17644 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:22:21.474248 17682 log.cc:927] Time spent T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:22:21.474942 17641 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:21.481302 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:21.480674 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:21.523803 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:21.528502 16397 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:21.479013 17681 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:21.488734 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:21.546198 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:21.493801 16472 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:21.486611 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:21.469118 17633 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:21.475768 17515 consensus_queue.cc:1048] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 12:22:21.542698 16477 raft_consensus.cc:1275] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:21.471854 17683 consensus_queue.cc:1048] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.073s
I20260709 12:22:21.534330 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:21.493139 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:21.490636 17679 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:21.487085 17576 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:21.486835 17579 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:21.544657 17681 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:21.543751 16394 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.007s	sys 0.012s
I20260709 12:22:21.492244 17644 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:21.647789 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:21.543243 17672 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:21.642302 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:21.661986 16400 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 4: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:21.655241 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:21.662827 16477 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:21.657590 17579 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:21.649678 16546 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:21.655715 17679 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:21.654857 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:21.660948 16394 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.007s	sys 0.012s
I20260709 12:22:21.664742 17644 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:21.660362 17681 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:21.649839 17595 consensus_queue.cc:1048] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 12:22:21.658303 16396 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:21.661396 17576 raft_consensus.cc:697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 5 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:21.651011 17657 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:21.687374 17567 consensus_meta.cc:287] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.738s	user 0.011s	sys 0.000s
I20260709 12:22:21.667639 17672 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:21.163009  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:21.675248 17670 consensus_queue.cc:1048] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.260s
I20260709 12:22:21.728775 16224 catalog_manager.cc:5697] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 1 to 2, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 2 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:21.702814 17565 consensus_queue.cc:1048] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 12:22:21.733883 17678 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:21.734372 17654 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:21.753459 17648 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:21.715344 17567 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:21.785530 17537 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:21.787447 17697 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:21.768759 17696 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:21.738471 17668 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:21.807348 17255 consensus_meta.cc:287] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.669s	user 0.009s	sys 0.000s
I20260709 12:22:21.776001 17571 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:21.775084 17678 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:21.807713 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 145914us
I20260709 12:22:21.873822 16514 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:21.778045 17650 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:22:21.807942 17531 consensus_meta.cc:287] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.503s	user 0.012s	sys 0.000s
I20260709 12:22:21.807801 17576 consensus_queue.cc:237] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:21.844482 17657 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:21.845676 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:21.885128 16403 raft_consensus.cc:2368] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3.
I20260709 12:22:21.848759 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:22:21.849395 17694 log.cc:927] Time spent T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:22:21.851866 17447 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:21.853248 17633 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:21.828557 17696 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:21.866060 17631 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:21.829694 17697 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:21.868959 17550 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:21.870543 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:21.872597 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:21.892905 17263 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:22:21.875901 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:21.859711 17579 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:21.782675 17648 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:21.887373 17142 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:21.893007 17266 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:21.877470 17678 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:21.880699 17677 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:21.900794 17142 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:21.909055 17657 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:21.909998 17648 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:21.911585 17651 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:21.912014 17651 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:21.892278 16470 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:21.912395 17651 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:21.913703 17625 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:21.917775 17661 consensus_meta.cc:287] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.789s	user 0.011s	sys 0.000s
I20260709 12:22:21.921403 17567 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:21.921190 17651 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:21.914868 17641 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:21.925184 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:21.926136 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:21.926337 16397 raft_consensus.cc:1275] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:21.927861 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:21.930860 17261 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:21.936254 16367 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:21.941170 16549 raft_consensus.cc:2368] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3.
I20260709 12:22:21.928460 17569 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:21.899231 17255 raft_consensus.cc:1275] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:21.950103 17569 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:21.950475 17255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 483) took 1043 ms. Trace:
I20260709 12:22:21.900250 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:21.951704 16548 raft_consensus.cc:2393] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 3: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:21.884809 17531 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:21.932147 17677 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:21.905536 17631 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:21.956194 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:21.952960 17565 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:21.958640 17625 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:21.959211 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:21.931903 17537 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:21.981788 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:21.983264 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:21.960580 16367 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:21.961861 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:21.962404 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:21.959916 17691 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:21.964669 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:21.964782 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:21.963249 17515 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:21.945842 17661 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:21.962679 17142 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:21.968441 16472 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:21.972621 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:21.992822 17663 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:21.967475 17698 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:21.993266 17663 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:21.985406 16475 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:21.993811 17663 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:21.973371 17633 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:21.996588 17633 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:21.996737 17269 raft_consensus.cc:1275] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:21.997376 17691 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:21.989046 16397 raft_consensus.cc:1275] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:21.988442 16545 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:21.976251 16395 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:21.999125 17531 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:21.993105 16470 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:21.969431 17565 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:22.000895 17565 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:22.001792 17668 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:21.996846 16547 raft_consensus.cc:2393] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 2: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:22.005402 17672 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:21.992431 17515 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:22.014643 16475 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:21.969543 17677 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:21.976688 17550 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:21.999300 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:22.018016 17142 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:22.018114 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:22.033269 16397 raft_consensus.cc:2368] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3.
I20260709 12:22:22.033717 16472 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:22:22.034430 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:22:22.007977 17692 log.cc:927] Time spent T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.462s	user 0.000s	sys 0.005s
I20260709 12:22:22.016942 16514 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:21.990829 17651 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:21.968163 16394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 605) took 1010 ms. Trace:
I20260709 12:22:22.016705 17698 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:22.007294 17625 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:22.015481 17537 raft_consensus.cc:2804] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:21.956648 17255 rpcz_store.cc:276] 0709 12:22:20.906694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:20.910624 (+  3930us) service_pool.cc:224] Handling call
0709 12:22:20.923748 (+ 13124us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:21.950195 (+1026447us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:21.950419 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:22.017284 17528 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:22.039861 17142 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:22.023773 17535 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:22.043573 17556 raft_consensus.cc:2804] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:22.043979 17691 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:21.978278 17631 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:22.044569 17550 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:22.042554 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:22.059013 16472 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:22.059978 16442 leader_election.cc:400] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3.
I20260709 12:22:22.060400 16442 leader_election.cc:403] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:22.045892 17537 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:22.064189 17142 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:22.065408 17658 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:22.066044 17658 raft_consensus.cc:697] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:22.066854 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:22.067516 16402 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:22.067239 17658 consensus_queue.cc:237] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:22.071465 17641 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:22.074476 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:22.075099 16468 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:22:22.050838 17528 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:22.079761 17698 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:22.055022 17535 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:22.080690 17528 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:22.082610 17515 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:22.082774 17535 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:22.048549 17625 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:22.084256 17633 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:22.085522 17625 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:22.085932 17697 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:22.086792 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:22.075578 17580 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3.
I20260709 12:22:22.087617 16544 raft_consensus.cc:2393] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 4: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:22.088853 17696 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:22.089257 16396 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:22.062613 17567 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:22.090663 17567 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:22.090955 17569 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:22.091231 17567 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:22.088213 17644 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:22.059530 17550 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:22.096661 16470 raft_consensus.cc:2368] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4.
I20260709 12:22:22.096804 17144 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:22.060568 17556 raft_consensus.cc:697] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 3 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:22.096692 17644 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:22.100587 17556 consensus_queue.cc:237] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:22.090677 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:22.098374 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:22.103475 16544 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:22.104895 16551 raft_consensus.cc:2393] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 3: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:22.106685 16402 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:22.107620 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:22.108229 16477 raft_consensus.cc:2393] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 2: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:22.118710 16468 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:22.119707 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:22.077226 17537 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:22.127682 17631 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:22.129001 16471 raft_consensus.cc:1275] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:22.130780 17681 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:22.096684 17550 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:22.156497 17681 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:22.135573 17569 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:22:22.135690 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 62656.1us
I20260709 12:22:22.162735 17144 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:22.165012 17269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 484) took 1258 ms. Trace:
I20260709 12:22:22.165299 17659 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:22.164703 16224 catalog_manager.cc:5697] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 1 to 2, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 2 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:22:22.165872 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:22.166605 16398 raft_consensus.cc:2368] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 3. Current term is 4.
I20260709 12:22:22.137604 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:22.170629 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:22.170979 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:22.171730 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:22.172243 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:22.138245 16514 leader_election.cc:400] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4.
I20260709 12:22:22.145332 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:22.078282 17579 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:22.136312 17565 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:22.164018 17144 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:22.165418 17269 rpcz_store.cc:276] 0709 12:22:20.907039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:20.907203 (+   164us) service_pool.cc:224] Handling call
0709 12:22:20.907627 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:21.997337 (+1089710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:22.164686 (+167349us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400210054. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:22.164872 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2503732480}
I20260709 12:22:22.202261 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:22.202978 16468 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:22.204516 17681 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:22.205102 17681 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:22.150703 16545 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:22.062053 17517 consensus_queue.cc:1048] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.077s
I20260709 12:22:22.172924 16403 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:22.173062 16514 leader_election.cc:403] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:22:22.167984 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:22.165889 17659 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:22.169428 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:22.175844 17537 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:22.219476 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 81449.2us
I20260709 12:22:22.198102 16399 raft_consensus.cc:2368] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3.
I20260709 12:22:22.172312 16398 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:22.226053 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:22.227388 17258 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:22.227478 16514 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:22.188006 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:22.225601 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:22.229070 16514 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:22.212809 16516 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:22.186270 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:22.190866 16442 leader_election.cc:400] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 3. Current term is 4.
I20260709 12:22:22.230888 16442 leader_election.cc:403] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:22.219666 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:22:22.196466 17706 log.cc:927] Time spent T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.164s	user 0.002s	sys 0.000s
I20260709 12:22:22.212936 17565 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:22.304701 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 72662us
I20260709 12:22:22.175770 17550 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:22.249080 17661 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:22.201059 17144 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:22.196799 17569 consensus_queue.cc:1048] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 12:22:22.247676 17579 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4.
W20260709 12:22:22.436659 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 151579us
I20260709 12:22:22.179848 17677 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:22.437381 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 151552us
I20260709 12:22:22.472450 16543 raft_consensus.cc:2368] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3.
I20260709 12:22:22.438052 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:22.172674 16396 raft_consensus.cc:2368] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4.
I20260709 12:22:22.197711 17633 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:22.144798 16544 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 3.
I20260709 12:22:22.446125 16398 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:22.437191 17659 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:22.446883 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:22.444427 17515 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:22.455849 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:22.447813 16468 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:22.448678 17642 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:22.444357 17641 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:22.449261 17624 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:22.449038 17685 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:22.451810 17258 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:22.175472 17663 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:22.480624 17658 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:22.483325 17651 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:22:22.470490 17679 consensus_meta.cc:287] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.781s	user 0.015s	sys 0.004s
I20260709 12:22:22.463778 17271 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:22.448673 17447 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 3. Current term is 4.
I20260709 12:22:22.464848 17576 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:22.492985 16514 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:22.468292 17661 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:22.478035 17565 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:22.476631 16551 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:22.541307 16397 raft_consensus.cc:2368] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3.
I20260709 12:22:22.448400 17531 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:22.488087 17536 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:22.499912 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 298925us
I20260709 12:22:22.521179 17703 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:22.521734 17669 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:22.486482 17633 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:22.485705 17521 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:22.491012 17527 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:22.490759 17535 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:22.495522 17642 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:22.542822 17661 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:22.518628 17624 raft_consensus.cc:697] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 3 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:22:22.523505 17678 consensus_meta.cc:287] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.584s	user 0.011s	sys 0.001s
I20260709 12:22:22.497954 17515 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:22.506372 17641 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:22.505654 17685 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:22.544988 17681 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:22.527654 17658 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:22.575874 16551 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 3.
I20260709 12:22:22.563463 17642 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:22.571429 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 82483.2us
I20260709 12:22:22.536082 17679 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:22.557461 17691 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:22.556779 17550 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:22.548660 17536 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:22.543305 17576 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:22.547065 17565 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:22.569661 17556 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:22.557235 17703 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:22.563727 17535 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:22.590934 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 98049.1us
I20260709 12:22:22.572383 17684 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:22.559631 17669 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:22.561576 17521 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:22.561515 17527 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:22.591114 16550 raft_consensus.cc:2368] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 3. Current term is 4.
I20260709 12:22:22.586205 17677 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:22.600898 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:22.586874 16367 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:22.608449 16472 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:22.603883 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 75561.6us
I20260709 12:22:22.570045 17678 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:22.589036 17681 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:22:22.595160 17567 consensus_meta.cc:287] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.501s	user 0.015s	sys 0.002s
I20260709 12:22:22.044636 16394 rpcz_store.cc:276] 0709 12:22:20.939583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:20.940580 (+   997us) service_pool.cc:224] Handling call
0709 12:22:20.940952 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:20.941171 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:22:21.699764 (+758593us) log.cc:844] Serialized 14 byte log entry
0709 12:22:21.894661 (+194897us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:21.899402 (+  4741us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:21.907771 (+  8369us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:21.938826 (+ 31055us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:21.949766 (+ 10940us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3949,"prepare.queue_time_us":758403,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":2779,"raft.queue_time_us":4081,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":2006,"spinlock_wait_cycles":46049024,"thread_start_us":909482,"threads_started":2,"wal-append.queue_time_us":264293}
I20260709 12:22:22.614217 17679 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:22.614635 17624 consensus_queue.cc:237] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:22.584750 17144 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:22.616274 17658 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:22.617592 17641 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:22.619292 17703 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:22.619588 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:22.621281 17576 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:22.621833 17669 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:22.624046 17527 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:22.594997 17556 raft_consensus.cc:697] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 3 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:22.624730 16472 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:22.626550 17678 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:22.586865 17691 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:22.648699 17685 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:22.691255 17659 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:22.694425 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:22:22.665871 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 56712.5us
W20260709 12:22:22.672394 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 51562.9us
I20260709 12:22:22.695676 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:22.697678 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:22.698299 16400 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:22.698761 16514 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:22.699131 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:22.699806 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:22.700718 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:22.701166 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:22.701710 17254 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:22.702561 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:22.703632 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:22.703747 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:22.704499 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:22.704711 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:22.705085 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:22.705171 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:22.705360 17255 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:22.706249 16476 raft_consensus.cc:1275] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:22.706532 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:22.674829 17658 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:22.708379 17633 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:22.708556 17625 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:22.708992 17725 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:22.709369 17625 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:22.627681 16542 raft_consensus.cc:2393] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 4: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:22.629036 17528 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:22.609270 17581 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:22.640152 17663 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:22.632043 17535 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:22.635655 17565 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:22.664150 17567 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:22.646955 17521 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:22.750627 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:22.751346 16398 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:22.689591 17670 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:22.690644 17556 consensus_queue.cc:237] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:22.691228 17691 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:22.667222 17681 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:22.701861 17264 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:22.704140 17270 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:22.641124 17515 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:22.628146 17536 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:22.708393 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:22.754246 16441 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:22.700155 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:22.719396 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:22.765178 16400 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:22.765326 17261 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:22.722038 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:22.723219 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:22.769289 17263 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:22:22.725482 16548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37242 (request call id 679) took 1083 ms. Trace:
I20260709 12:22:22.770236 16548 rpcz_store.cc:276] 0709 12:22:21.641848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:21.642013 (+   165us) service_pool.cc:224] Handling call
0709 12:22:22.725135 (+1083122us) spinlock_profiling.cc:243] Waited 290 ms on lock 0x7b54000884d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:22.725390 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":665980672}
I20260709 12:22:22.727125 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:22.754199 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:22.695463 17659 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:22.709589 17725 raft_consensus.cc:697] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 3 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:22.776140 16400 raft_consensus.cc:1275] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:22.724576 17633 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:22:22.714643 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 136634us
I20260709 12:22:22.726990 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:22.726347 17528 raft_consensus.cc:697] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 3 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:22.813973 16441 leader_election.cc:400] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3.
I20260709 12:22:22.814604 16441 leader_election.cc:403] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:22.806977 16398 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:22.810153 17270 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:22.717922 16396 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:22.774911 17515 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:22:22.840932 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 230434us
I20260709 12:22:22.847316 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:22.801543 17688 raft_consensus.cc:2764] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:22:22.848690 17270 raft_consensus.cc:2468] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:22:22.766541 17677 consensus_queue.cc:1048] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.067s
I20260709 12:22:22.848733 16367 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:22.827381 16394 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:22.775434 17670 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:22.822799 17684 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3.
I20260709 12:22:22.833842 17625 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:22.848433 17651 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:22.815117 17633 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:22.851689 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:22.902556 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:22.924213 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:22.911391 17537 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:22.912493 17528 consensus_queue.cc:237] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:22.914793 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:22.860870 17550 consensus_queue.cc:1048] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 12:22:22.877542 17565 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:22.917531 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:22.971751 17266 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:22.929654 16544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 878) took 1009 ms. Trace:
I20260709 12:22:22.941232 16545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37242 (request call id 680) took 1294 ms. Trace:
W20260709 12:22:22.943120 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 82961.8us
W20260709 12:22:22.908514 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 196595us
I20260709 12:22:22.857935 17515 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:22.954882 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:22:22.956406 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 98898.2us
I20260709 12:22:22.979176 16402 raft_consensus.cc:1275] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:22.980319 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:22.982081 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:22.927205 17271 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:22.973114 16544 rpcz_store.cc:276] 0709 12:22:21.920285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.488268 (+567983us) service_pool.cc:224] Handling call
0709 12:22:22.489909 (+  1641us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:22.570713 (+ 80804us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:22.570759 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:22.570819 (+    60us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:22:22.570864 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:22.929303 (+358439us) spinlock_profiling.cc:243] Waited 80.5 ms on lock 0x7b5400bb5654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:22.929374 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:22.929577 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":185206528}
I20260709 12:22:23.011492 17264 raft_consensus.cc:1275] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:22.918781 17651 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:22.999462 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:23.018671 16543 raft_consensus.cc:2368] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3.
I20260709 12:22:22.974682 16545 rpcz_store.cc:276] 0709 12:22:21.647305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:21.647463 (+   158us) service_pool.cc:224] Handling call
0709 12:22:22.941000 (+1293537us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b5400896a54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:22.941150 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":752852864}
I20260709 12:22:22.990597 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:22.921106 17642 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:22.957692 17537 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:22.926921 17661 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:22.971444 16546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52680 (request call id 66) took 1069 ms. Trace:
I20260709 12:22:23.033610 16548 raft_consensus.cc:1275] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:23.035669 17266 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
W20260709 12:22:23.033380 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 145772us
I20260709 12:22:22.988852 17658 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:22:23.204492 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 166214us
I20260709 12:22:23.210575 17269 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:23.041642 16547 raft_consensus.cc:2368] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 3. Current term is 4.
I20260709 12:22:23.021111 17665 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:23.218377 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:23.224303 17270 raft_consensus.cc:2393] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 3: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:23.225524 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:23.220108 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:23.036957 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:23.228086 17261 raft_consensus.cc:2410] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:22:23.227123 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:23.019743 17568 consensus_queue.cc:1048] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.197s
I20260709 12:22:23.238337 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:23.239207 17261 raft_consensus.cc:2393] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 3: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:23.230424 17270 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:23.200583 17567 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:23.214033 17581 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:23.202291 17531 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:23.215246 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:23.298285 17143 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:23.037499 17537 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:23.268671 17254 consensus_meta.cc:287] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.566s	user 0.013s	sys 0.000s
I20260709 12:22:23.307497 17531 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:22.960518 17725 consensus_queue.cc:237] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:23.318833 17633 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:23.320554 17567 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:23.322714 17651 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:23.226881 17144 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:23.222295 17670 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:23.223269 17700 consensus_queue.cc:1048] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.126s
I20260709 12:22:23.233496 17517 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:23.272464 17681 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:23.349876 17143 leader_election.cc:400] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3.
I20260709 12:22:23.392632 16399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58430 (request call id 109) took 1410 ms. Trace:
W20260709 12:22:23.392917 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 58904.4us
I20260709 12:22:23.394927 17143 leader_election.cc:400] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3.
I20260709 12:22:23.218915 17658 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:23.037921 16546 rpcz_store.cc:276] 0709 12:22:21.895604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:21.896815 (+  1211us) service_pool.cc:224] Handling call
0709 12:22:22.832934 (+936119us) spinlock_profiling.cc:243] Waited 17 ms on lock 0x7b58005e9298. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:22:22.964803 (+131869us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39167872}
I20260709 12:22:23.406387 17537 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:23.407238 17661 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:23.407599 16401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 614) took 1371 ms. Trace:
I20260709 12:22:23.235010 17665 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:23.417074 17741 log.cc:927] Time spent T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:22:23.417919 17665 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:23.418169 16542 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37226 (request call id 877) took 1498 ms (client timeout 1564 ms). Trace:
I20260709 12:22:23.419654 17663 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:23.420481 17635 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:23.421118 17635 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:23.423635 17635 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:23.393352 16399 rpcz_store.cc:276] 0709 12:22:21.962393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:21.964439 (+  2046us) service_pool.cc:224] Handling call
0709 12:22:23.372045 (+1407606us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b54009bc8d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:23.372255 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":481908096}
W20260709 12:22:23.418970 16542 rpcz_store.cc:269] 0709 12:22:21.919993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:21.925919 (+  5926us) service_pool.cc:224] Handling call
0709 12:22:23.417927 (+1492008us) spinlock_profiling.cc:243] Waited 679 ms on lock 0x7b5400304654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:23.418084 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1562452224}
I20260709 12:22:23.429519 17269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45228 (request call id 387) took 1199 ms. Trace:
I20260709 12:22:23.429935 17269 rpcz_store.cc:276] 0709 12:22:22.225182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.225329 (+   147us) service_pool.cc:224] Handling call
0709 12:22:23.416286 (+1190957us) spinlock_profiling.cc:243] Waited 981 ms on lock 0x7b540018a954. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:23.424691 (+  8405us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2257045888}
I20260709 12:22:23.337397 17581 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:23.360855 17254 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:23.399215 17517 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:23.400547 17531 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:23.403033 17642 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:23.403326 17658 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:23.408041 16401 rpcz_store.cc:276] 0709 12:22:22.036095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.464101 (+428006us) service_pool.cc:224] Handling call
0709 12:22:22.469362 (+  5261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:22.601020 (+131658us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:22.601069 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:22.601131 (+    62us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:22.601177 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:23.407229 (+806052us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b5400a104d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:23.407412 (+   183us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:23.407541 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":302094208}
I20260709 12:22:23.400576 16473 raft_consensus.cc:2393] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 4: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:23.395566 17143 leader_election.cc:403] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
W20260709 12:22:23.464418 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 69670.6us
I20260709 12:22:23.465508 17143 leader_election.cc:400] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 3. Current term is 4.
I20260709 12:22:23.468187 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:23.468468 17269 raft_consensus.cc:1275] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:23.469368 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:23.472203 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:23.473124 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:23.473874 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:23.473894 17270 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:22:23.474458 17537 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:23.474699 17261 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:23.475706 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:23.473282 17269 raft_consensus.cc:2393] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 4: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:23.428854 17569 raft_consensus.cc:2764] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 12:22:23.421321 17663 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:23.470837 17263 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
W20260709 12:22:23.477039 16403 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.003s	sys 0.005s
I20260709 12:22:23.429813 17633 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:23.477039 17261 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 3.
W20260709 12:22:23.482000 16403 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.004s	sys 0.005s
W20260709 12:22:23.478533 16367 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.011s	sys 0.028s
W20260709 12:22:23.486207 16367 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.011s	sys 0.028s
I20260709 12:22:23.439263 16441 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:23.483454 17270 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 3.
W20260709 12:22:23.427284 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 211259us
I20260709 12:22:23.434149 17650 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:22:23.487511 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 509551us
I20260709 12:22:23.489511 17258 raft_consensus.cc:2393] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 4: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:23.437088 17665 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:23.491837 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:23.477051 17745 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:23.531088 16400 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.000s	sys 0.023s
W20260709 12:22:23.493433 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 54275.9us
I20260709 12:22:23.521724 17642 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:22:23.533650 16400 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.000s	sys 0.024s
I20260709 12:22:23.562798 16400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 619) took 1514 ms. Trace:
I20260709 12:22:23.535748 17581 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:23.524085 17686 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:23.539422 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:23.482612 17576 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:23.584190 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:23.533701 17731 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:23.587173 16550 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52680 (request call id 63) took 1711 ms (client timeout 2039 ms). Trace:
I20260709 12:22:23.533438 17515 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:22:23.565311 16367 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.1:41805 timed out after 1.564s (SENT) [suppressed 343 similar messages]
I20260709 12:22:23.563655 16400 rpcz_store.cc:276] 0709 12:22:22.048786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.796420 (+747634us) service_pool.cc:224] Handling call
0709 12:22:22.812804 (+ 16384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:22.813086 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:22:22.904088 (+ 91002us) log.cc:844] Serialized 14 byte log entry
0709 12:22:23.562281 (+658193us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:23.562362 (+    81us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:23.562423 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:23.562536 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:23.562718 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":86382,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":52,"raft.queue_time_us":370841,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":14463,"thread_start_us":721398,"threads_started":3,"wal-append.queue_time_us":314229}
I20260709 12:22:23.591950 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:23.490199 17633 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:23.594470 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:23.533136 17650 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:23.602799 17650 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:23.603439 17672 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:23.605582 17581 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:23.606431 17581 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:23.536595 17144 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:23.541680 17745 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:23.609314 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 79100.3us
I20260709 12:22:23.610698 17144 leader_election.cc:400] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3.
I20260709 12:22:23.611021 17144 leader_election.cc:403] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:23.611660 17569 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:23.611688 17745 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:23.592058 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:23.597754 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:22:23.589577 16550 rpcz_store.cc:269] 0709 12:22:21.875473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:21.875636 (+   163us) service_pool.cc:224] Handling call
0709 12:22:23.586921 (+1711285us) spinlock_profiling.cc:243] Waited 697 ms on lock 0x7b5400ac8354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:23.587095 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1603596544}
I20260709 12:22:23.599330 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:23.582008 17642 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:23.612522 17658 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3.
I20260709 12:22:23.562288 17665 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:23.612797 17686 raft_consensus.cc:697] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:23.614394 17633 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:23.595402 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:23.615290 17517 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:23.616678 17537 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3.
I20260709 12:22:23.616818 17663 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:23.618731 17748 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:22:23.622916 16394 consensus_meta.cc:287] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.734s	user 0.009s	sys 0.002s
I20260709 12:22:23.597599 17731 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:23.636857 16477 raft_consensus.cc:2393] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 3: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:23.612171 17569 raft_consensus.cc:697] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:23.603983 17672 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:23.616735 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:23.667887 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:23.667197 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:23.671525 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:23.676918 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:23.653667 17270 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:23.672784 16441 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:23.641199 17447 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:23.678200 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:23.680541 16441 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:23.678421 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:23.685941 17739 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:23.715497 16441 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:23.701908 17144 leader_election.cc:400] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3.
I20260709 12:22:23.696131 17447 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:23.648873 16394 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:23.597105 17576 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:23.706707 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:23.705087 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:23.723302 16441 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:23.715570 17261 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 3.
I20260709 12:22:23.648510 17748 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:23.750736 17748 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:23.751271 17748 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:23.751396 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:23.751940 16366 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:23.696560 16395 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:23.752724 16395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 624) took 1605 ms. Trace:
I20260709 12:22:23.753082 16395 rpcz_store.cc:276] 0709 12:22:22.147640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.219811 (+ 72171us) service_pool.cc:224] Handling call
0709 12:22:23.616616 (+1396805us) spinlock_profiling.cc:243] Waited 8.16 ms on lock 0x7b5800125198. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64eee 000000000036eb39 00007f287add81e0 00007f287aea10c4 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:22:23.752661 (+136045us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18767616}
I20260709 12:22:23.720413 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:23.754449 16546 raft_consensus.cc:2368] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3.
I20260709 12:22:23.738731 17661 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:23.755288 16394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46250 (request call id 552) took 1090 ms. Trace:
W20260709 12:22:23.739018 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 62745us
I20260709 12:22:23.755940 16394 rpcz_store.cc:276] 0709 12:22:22.645617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.673854 (+ 28237us) service_pool.cc:224] Handling call
0709 12:22:23.735398 (+1061544us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:23.758483 17569 consensus_queue.cc:237] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:23.762241 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:23.764065 17642 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:23.764679 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:22:23.765856 17255 consensus_meta.cc:287] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 1.060s	user 0.012s	sys 0.000s
I20260709 12:22:23.713567 17686 consensus_queue.cc:237] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:23.766374 17255 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:23.767095 17255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45228 (request call id 397) took 1069 ms. Trace:
I20260709 12:22:23.767508 17255 rpcz_store.cc:276] 0709 12:22:22.697842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.704447 (+  6605us) service_pool.cc:224] Handling call
0709 12:22:23.767022 (+1062575us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:23.765879 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:23.739843 17684 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:23.740521 17520 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:23.741621 17665 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:23.725863 17581 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:23.743320 16473 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:23.721292 17143 leader_election.cc:400] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 3. Current term is 4.
I20260709 12:22:23.751739 17748 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:23.769323 16399 raft_consensus.cc:2368] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 3. Current term is 4.
I20260709 12:22:23.711836 17687 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:23.773777 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:23.790988 16401 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 1.
I20260709 12:22:23.777092 16366 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:23.729743 17595 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:23.780951 16225 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59300 (request call id 82) took 4184 ms. Trace:
W20260709 12:22:23.782687 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 61596.9us
I20260709 12:22:23.786979 16441 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:23.784080 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:23.804273 16225 rpcz_store.cc:276] 0709 12:22:19.590928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:19.591490 (+   562us) service_pool.cc:224] Handling call
0709 12:22:23.774872 (+4183382us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:22:20.341801 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:22:20.393520 (+ 51719us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:22:20.393580 (+    60us) write_op.cc:435] Acquired schema lock
0709 12:22:20.393604 (+    24us) tablet.cc:662] Decoding operations
0709 12:22:20.394332 (+   728us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:22:20.394408 (+    76us) write_op.cc:641] Partition lock acquired for write op
0709 12:22:20.394446 (+    38us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:22:20.394842 (+   396us) tablet.cc:701] Row locks acquired
0709 12:22:20.394864 (+    22us) write_op.cc:260] PREPARE: finished
0709 12:22:20.395063 (+   199us) write_op.cc:270] Start()
0709 12:22:20.395202 (+   139us) write_op.cc:276] Timestamp: P: 1783599740395007 usec, L: 0
0709 12:22:20.395226 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:22:20.396044 (+   818us) log.cc:844] Serialized 1289 byte log entry
0709 12:22:20.495837 (+ 99793us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b180014de20. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f28798c041d 00007f28798c271a 00007f28798c26a8 00007f28798c2628 00007f28798c25e3 00007f28798c17fc 00007f287bcf42f6 00007f287bceeaa8 00007f2875e95310
0709 12:22:20.495903 (+    66us) op_driver.cc:464] REPLICATION: finished
0709 12:22:20.719497 (+223594us) write_op.cc:301] APPLY: starting
0709 12:22:20.719602 (+   105us) tablet.cc:1367] starting BulkCheckPresence
0709 12:22:20.719749 (+   147us) tablet.cc:1370] finished BulkCheckPresence
0709 12:22:20.719771 (+    22us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:22:20.720231 (+   460us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:22:20.720446 (+   215us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 12:22:20.720492 (+    46us) write_op.cc:312] APPLY: finished
0709 12:22:20.720690 (+   198us) log.cc:844] Serialized 34 byte log entry
0709 12:22:20.720850 (+   160us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:20.720938 (+    88us) write_op.cc:454] Released schema lock
0709 12:22:20.720975 (+    37us) write_op.cc:341] FINISH: Updating metrics
0709 12:22:20.721291 (+   316us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:20.721323 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":43672,"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":214297,"apply.run_cpu_time_us":1808,"apply.run_wall_time_us":1809,"num_ops":3,"prepare.queue_time_us":133689,"prepare.run_cpu_time_us":4004,"prepare.run_wall_time_us":67503,"replication_time_us":4455,"spinlock_wait_cycles":30818176,"thread_start_us":100968,"threads_started":2}]]}
I20260709 12:22:23.808212 17751 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:23.808389 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:23.776408 17684 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:23.808681 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:23.809125 17684 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:23.809927 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:23.810541 16468 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:23.814242 17143 leader_election.cc:400] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3.
I20260709 12:22:23.814602 17143 leader_election.cc:403] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:22:23.809139 16476 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 3.
I20260709 12:22:23.816542 17567 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:23.782400 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 217121us
I20260709 12:22:23.818707 17684 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:23.824175 16397 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:23.825515 16366 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:23.828114 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:23.828549 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:23.829038 16474 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
I20260709 12:22:23.832199 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:23.840488 16397 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5"
dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:60492
I20260709 12:22:23.841424 16397 raft_consensus.cc:606] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:23.842378 16397 raft_consensus.cc:614] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:23.777665 17668 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:23.846635 16366 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:23.855346 17663 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:23.855734 17663 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:23.855822 16366 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:23.755756 17661 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:23.797675 17512 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:23.878155 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:23.781656 17520 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:23.882161 17520 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:23.881403 17748 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:23.844424 17687 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:23.884982 17687 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:23.887002 17672 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:23.883993 17731 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:23.888381 17658 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:23.878189 17512 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:23.876686 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:23.891608 17512 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:23.891872 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:23.876806 17661 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:23.876849 17447 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:23.854375 17595 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:23.894156 17576 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:23.893476 17144 leader_election.cc:400] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 3. Current term is 4.
I20260709 12:22:23.895769 17144 leader_election.cc:403] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:22:23.876662 17684 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:23.897509 17144 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:23.866531 17633 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:23.898643 17651 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 3. Current term is 4.
I20260709 12:22:23.898871 17633 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:23.899070 17528 consensus_queue.cc:1048] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:23.888124 17687 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:23.897372 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 57206.7us
I20260709 12:22:23.899379 17633 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:23.856160 17663 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:23.880079 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:23.915371 16367 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
W20260709 12:22:23.968287 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 53017.4us
I20260709 12:22:23.970067 16367 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:23.975060 17264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 401) took 1276 ms. Trace:
I20260709 12:22:23.931329 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:23.966100 17651 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:23.916047 17658 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:23.973752 17748 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:23.981717 16367 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:23.975523 17264 rpcz_store.cc:276] 0709 12:22:22.698957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.774081 (+ 75124us) service_pool.cc:224] Handling call
0709 12:22:23.011147 (+237066us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:23.011984 (+   837us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:23.974871 (+962887us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b540005f0d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:23.975009 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":434617600}
I20260709 12:22:23.983390 17528 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:23.981014 17556 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:23.982126 17651 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:23.986323 17748 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:23.992955 16367 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:23.992214 17576 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:23.996315 17748 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:23.997666 17651 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:23.988499 17661 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:24.018370 17751 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:24.019678 16367 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:23.996628 17556 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:24.028079 17661 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:24.032555 17668 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:24.046896 17750 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:24.051769 17661 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:22:24.057055 17766 log.cc:927] Time spent T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:22:24.038285 17751 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 12:22:24.066334 17658 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:24.067142 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:24.067364 17748 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:24.070175 17748 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:24.083808 17633 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:24.087009 17633 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:24.036314 17742 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:24.089990 17556 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:22:24.090972 17765 log.cc:927] Time spent T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 12:22:24.091768 17761 log.cc:927] Time spent T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 12:22:24.087873 17567 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3.
I20260709 12:22:24.095450 17663 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:24.097095 17665 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:24.109279 16395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47"
dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:60492
I20260709 12:22:24.104765 17742 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:24.106179 17651 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:24.157362 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:24.157666 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:24.159060 17263 raft_consensus.cc:2393] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 4: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:24.159430 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:24.161365 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:24.161620 16403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 625) took 1575 ms. Trace:
I20260709 12:22:24.163022 16367 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:22:24.134523 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 48328.8us
I20260709 12:22:24.154404 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:24.161966 16403 rpcz_store.cc:276] 0709 12:22:22.585997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.954139 (+368142us) service_pool.cc:224] Handling call
0709 12:22:22.959188 (+  5049us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:23.486196 (+527008us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:23.486229 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:23.492726 (+  6497us) log.cc:844] Serialized 26 byte log entry
0709 12:22:23.715157 (+222431us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:22:23.715191 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:23.715254 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:24.159324 (+444070us) raft_consensus.cc:1685] finished
0709 12:22:24.159345 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:24.159472 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8009,"raft.queue_time_us":367686,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":147,"thread_start_us":667856,"threads_started":2,"wal-append.queue_time_us":390773}
I20260709 12:22:24.155344 16395 raft_consensus.cc:606] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 3 LEADER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:22:24.162011 17263 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 3.
I20260709 12:22:24.167069 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:24.167212 16367 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:24.168584 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:24.134244 17665 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:24.156664 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:24.168074 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:24.169154 17745 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:24.179327 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:24.180073 17271 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:24.189679 17663 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:24.190500 16471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34840 (request call id 679) took 1211 ms. Trace:
I20260709 12:22:24.190291 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:24.189594 17742 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:24.193132 16471 rpcz_store.cc:276] 0709 12:22:22.979280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.988633 (+  9353us) service_pool.cc:224] Handling call
0709 12:22:24.164428 (+1175795us) spinlock_profiling.cc:243] Waited 3.44 ms on lock 0x7b5800602a98. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64eee 000000000036eb39 00007f287add81e0 00007f287aea10c4 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:22:24.187061 (+ 22633us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7922048}
I20260709 12:22:24.195289 17651 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:24.199030 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:24.201304 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:24.175781 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:24.202658 17642 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:24.182046 17745 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:24.203495 17745 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:24.203238 17642 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:24.203770 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:24.204753 16397 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:24.205475 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:24.206065 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:24.208130 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:24.208274 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:24.208866 16546 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
I20260709 12:22:24.209211 16547 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
I20260709 12:22:24.209915 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:24.210922 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:24.211448 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:24.213891 17642 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:24.165783 17515 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:24.220237 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:24.221325 17271 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:24.221474 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:24.191661 17665 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:24.222707 17665 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:24.222796 16515 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:24.223186 17665 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:24.223658 17696 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:24.223987 16395 raft_consensus.cc:1240] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:24.224162 17696 raft_consensus.cc:697] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:24.224596 17777 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:24.225101 17777 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:24.225695 16515 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:24.227242 17777 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:24.227921 17556 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:24.225102 17696 consensus_queue.cc:237] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:24.228240 16397 raft_consensus.cc:1240] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:24.229275 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:24.229440 16224 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53622 (request call id 131) took 4355 ms. Trace:
I20260709 12:22:24.229822 16397 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:24.230710 16515 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:24.231881 16515 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:24.232504 17725 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:24.232709 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:24.233089 17664 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:24.233433 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:24.234709 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:24.235316 16515 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:24.226631 16402 raft_consensus.cc:1275] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:24.236095 17642 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:24.236608 17642 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:24.236979 17642 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:24.237159 17665 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:24.237787 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:24.239037 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:24.222116 16543 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 3.
I20260709 12:22:24.242769 17651 consensus_queue.cc:1059] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:24.243345 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:24.244489 16515 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:24.245671 16515 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:24.247190 16367 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:24.187979 17263 raft_consensus.cc:1275] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:24.249899 16544 raft_consensus.cc:1275] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:24.242949 17144 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:24.246783 16515 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:24.253662 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:24.202795 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:24.253546 17697 raft_consensus.cc:2764] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:22:24.257193 17515 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:24.295653 16399 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:24.255061 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:24.257495 17665 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:24.298657 17665 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:24.296883 17745 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:24.267366 17760 log.cc:927] Time spent T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.340s	user 0.000s	sys 0.002s
I20260709 12:22:24.286969 17270 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:24.299081 17665 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:24.283665 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:24.284752 17659 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:24.285569 17632 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:24.287292 16398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 623) took 2174 ms. Trace:
I20260709 12:22:24.309173 17658 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:24.309026 17632 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:24.310473 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 67610.7us
I20260709 12:22:24.256317 16367 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:24.311551 17632 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:24.312700 16398 rpcz_store.cc:276] 0709 12:22:22.112785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.819201 (+706416us) service_pool.cc:224] Handling call
0709 12:22:22.827266 (+  8065us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:22.827489 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:22.827529 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:22.827589 (+    60us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:22.827632 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:22.827706 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:24.287120 (+1459414us) spinlock_profiling.cc:243] Waited 7.57 ms on lock 0x7b5400543150. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:24.287236 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17400192}
I20260709 12:22:24.262902 16403 raft_consensus.cc:1275] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:24.229835 16224 rpcz_store.cc:276] 0709 12:22:19.874071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:19.894306 (+ 20235us) service_pool.cc:224] Handling call
0709 12:22:24.229357 (+4335051us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:22:22.461377 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:22:22.462159 (+   782us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:22:22.462213 (+    54us) write_op.cc:435] Acquired schema lock
0709 12:22:22.462239 (+    26us) tablet.cc:662] Decoding operations
0709 12:22:22.462791 (+   552us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:22:22.462925 (+   134us) write_op.cc:641] Partition lock acquired for write op
0709 12:22:22.462968 (+    43us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:22:22.463395 (+   427us) tablet.cc:701] Row locks acquired
0709 12:22:22.463419 (+    24us) write_op.cc:260] PREPARE: finished
0709 12:22:22.463591 (+   172us) write_op.cc:270] Start()
0709 12:22:22.463704 (+   113us) write_op.cc:276] Timestamp: P: 1783599742463547 usec, L: 0
0709 12:22:22.463727 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:22:22.503271 (+ 39544us) log.cc:844] Serialized 910 byte log entry
0709 12:22:22.900830 (+397559us) op_driver.cc:464] REPLICATION: finished
0709 12:22:23.271904 (+371074us) write_op.cc:301] APPLY: starting
0709 12:22:23.300173 (+ 28269us) tablet.cc:1367] starting BulkCheckPresence
0709 12:22:23.329000 (+ 28827us) tablet.cc:1370] finished BulkCheckPresence
0709 12:22:23.329215 (+   215us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:22:23.329744 (+   529us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:22:23.330008 (+   264us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 12:22:23.330085 (+    77us) write_op.cc:312] APPLY: finished
0709 12:22:23.330387 (+   302us) log.cc:844] Serialized 28 byte log entry
0709 12:22:23.330649 (+   262us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:23.330784 (+   135us) write_op.cc:454] Released schema lock
0709 12:22:23.330845 (+    61us) write_op.cc:341] FINISH: Updating metrics
0709 12:22:23.331369 (+   524us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:23.331426 (+    57us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1512733,"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":320990,"apply.run_cpu_time_us":2815,"apply.run_wall_time_us":104827,"num_ops":2,"prepare.queue_time_us":271403,"prepare.run_cpu_time_us":17766,"prepare.run_wall_time_us":124308,"raft.queue_time_us":262490,"raft.run_cpu_time_us":6369,"raft.run_wall_time_us":97009,"replication_time_us":433551,"thread_start_us":651934,"threads_started":4,"wal-append.queue_time_us":126113}]]}
I20260709 12:22:24.315714 16516 leader_election.cc:400] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4.
W20260709 12:22:24.295363 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 48642.9us
W20260709 12:22:24.312083 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 55867us
I20260709 12:22:24.320209 16515 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:24.296106 17556 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:24.298198 16442 leader_election.cc:400] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3.
I20260709 12:22:24.263388 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:24.331120 16516 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:24.332568 17266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 500) took 1115 ms. Trace:
I20260709 12:22:24.258612 17781 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:24.301497 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:24.307087 16397 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:24.308115 17659 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:24.346565 17659 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:24.243872 17651 consensus_queue.cc:1059] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.084s
I20260709 12:22:24.260731 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:24.338554 16403 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709"
dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:60492
I20260709 12:22:24.366233 16403 raft_consensus.cc:606] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:22:24.313588 17515 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:24.331182 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:24.270710 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:24.333072 17266 rpcz_store.cc:276] 0709 12:22:23.217159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:23.217302 (+   143us) service_pool.cc:224] Handling call
0709 12:22:23.217622 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:23.370861 (+153239us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:23.370900 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:23.370950 (+    50us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:22:23.370980 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:24.332368 (+961388us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b540005e6d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:24.332412 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:24.332515 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":351845248}
I20260709 12:22:24.347896 16401 raft_consensus.cc:2393] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 3: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:24.253834 17777 raft_consensus.cc:3055] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:24.358230 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:24.301935 17568 consensus_queue.cc:1048] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 12:22:24.376914 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:24.377319 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:24.380656 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:24.380002 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:24.271920 17642 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:24.395673 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:24.398931 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:24.392145 16476 raft_consensus.cc:2368] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3.
I20260709 12:22:24.391232 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:24.401374 16474 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:24.402305 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:24.402412 17642 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:24.402926 16474 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:22:24.403149 17642 raft_consensus.cc:2804] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:24.403635 17642 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:24.404444 17745 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:24.405278 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:24.406680 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:24.407727 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:24.408883 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:24.410281 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:24.410671 17665 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:24.411437 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:24.349129 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:24.412686 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:24.414279 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:24.415870 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:24.404067 17642 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:24.415794 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:24.417899 16544 raft_consensus.cc:1240] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:24.375433 16394 raft_consensus.cc:2368] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 3. Current term is 4.
I20260709 12:22:24.419471 17269 raft_consensus.cc:2393] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 3: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:24.419847 17143 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:24.420928 16477 raft_consensus.cc:2368] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3.
I20260709 12:22:24.416666 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:24.421902 16397 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:24.423519 17142 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:24.425995 17556 raft_consensus.cc:697] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:24.433600 17556 consensus_queue.cc:237] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:24.435575 16397 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:24.403892 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:24.429191 16442 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:24.430014 17565 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.393s	user 0.015s	sys 0.019s
W20260709 12:22:24.432029 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 111927us
I20260709 12:22:24.439941 16515 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:24.439589 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:24.442811 16515 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:24.426014 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:22:24.428182 17691 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.550s	user 0.030s	sys 0.073s
I20260709 12:22:24.378506 17777 raft_consensus.cc:740] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:24.382400 17576 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:24.388154 17696 raft_consensus.cc:2804] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:24.447235 16472 raft_consensus.cc:1275] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:24.339699 17723 consensus_queue.cc:1048] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.080s
I20260709 12:22:24.389243 17515 raft_consensus.cc:697] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 4 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:24.449416 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:24.452194 17142 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:24.458937 17144 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:24.459810 17142 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:24.445102 17565 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.425s	user 0.016s	sys 0.019s
W20260709 12:22:24.445245 17691 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.579s	user 0.030s	sys 0.073s
I20260709 12:22:24.360730 17781 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:24.464681 16472 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "95954ea821224e4d962974a83d83ad44"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
mode: GRACEFUL
new_leader_uuid: "610f4feda11942739236f2b121a4b9c1"
 from {username='slave'} at 127.0.0.1:39000
I20260709 12:22:24.340360 17537 consensus_queue.cc:1048] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.111s
I20260709 12:22:24.467649 17777 consensus_queue.cc:260] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:24.451481 17725 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:24.468075 17515 consensus_queue.cc:237] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:24.442333 17569 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:24.470152 16472 raft_consensus.cc:606] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Received request to transfer leadership to TS 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:24.450364 17696 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:22:24.502187 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 184747us
I20260709 12:22:24.445545 16471 raft_consensus.cc:1275] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:24.458804 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:24.463699 17651 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:22:24.463860 17565 raft_consensus.cc:697] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 3 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:24.470860 17677 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:24.529125 16516 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:24.530347 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 87588.3us
I20260709 12:22:24.452675 17576 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:24.568586 17686 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:24.580057 17565 consensus_queue.cc:237] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:24.594391 17750 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:24.532994 17777 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:24.581558 17784 log.cc:927] Time spent T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:22:24.592602 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:24.658509 17686 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:24.503052 17661 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:24.660909 17661 raft_consensus.cc:697] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:24.661065 17686 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:24.661864 17661 consensus_queue.cc:237] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:24.570183 17684 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:24.544497 17569 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:24.503368 17659 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:22:24.548832 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 119719us
I20260709 12:22:24.642810 17723 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:24.537590 17730 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:24.529429 17696 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:22:24.587634 17778 log.cc:927] Time spent T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.433s	user 0.000s	sys 0.006s
I20260709 12:22:24.548367 17697 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:24.529642 17745 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:24.582191 17793 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:24.502382 17665 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:24.458688 17791 consensus_queue.cc:1048] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
W20260709 12:22:24.549952 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 90165.6us
I20260709 12:22:24.758312 17142 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:24.544981 16395 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:24.536072 17746 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:24.586393 17781 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:22:24.555897 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 97051.9us
W20260709 12:22:24.636755 16441 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.787s	user 0.008s	sys 0.026s
W20260709 12:22:24.640959 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 111932us
I20260709 12:22:24.648456 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:24.542069 17790 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:24.662122 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:24.663268 16545 raft_consensus.cc:1240] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 3. Current term is 4. Ops: []
W20260709 12:22:24.667541 17789 log.cc:927] Time spent T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.099s	user 0.002s	sys 0.000s
I20260709 12:22:24.669603 17658 raft_consensus.cc:3055] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:24.624429 16394 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d46e3072dec049b18b323d65470603ec"
dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:60492
I20260709 12:22:24.694984 17261 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45230 (request call id 492) took 1096 ms. Trace:
W20260709 12:22:24.652973 17800 log.cc:927] Time spent T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:22:24.506709 17782 consensus_queue.cc:1048] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.091s
I20260709 12:22:24.691699 17750 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:24.720008 16442 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:24.721194 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:24.760249 17142 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:24.706496 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:24.696689 17569 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:24.707619 17659 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:22:24.779989 16441 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.930s	user 0.008s	sys 0.026s
W20260709 12:22:24.803400 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 953802us
I20260709 12:22:24.780434 16549 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:22:24.780452 16516 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:24.792255 17261 rpcz_store.cc:276] 0709 12:22:23.598535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:23.599048 (+   513us) service_pool.cc:224] Handling call
0709 12:22:24.694783 (+1095735us) spinlock_profiling.cc:243] Waited 23.3 ms on lock 0x7b5400bc4dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:24.694919 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53650816}
I20260709 12:22:24.798516 17142 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:24.713707 17730 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:24.801331 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 81453.6us
I20260709 12:22:24.802546 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:24.708871 17642 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:24.779743 17144 leader_election.cc:400] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 3. Current term is 4.
I20260709 12:22:24.806156 16516 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:24.807557 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:24.793185 17658 raft_consensus.cc:740] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:24.829286 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:24.757963 17697 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:24.818610 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:24.759680 17745 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:24.819077 17576 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:24.786417 17790 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:24.781980 17746 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:24.808557 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:24.820025 17799 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:24.822547 17142 leader_election.cc:400] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3.
I20260709 12:22:24.825789 17144 leader_election.cc:403] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:22:24.841816 17142 leader_election.cc:403] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:22:24.779438 17742 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:24.821518 16397 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 1.
I20260709 12:22:24.800544 16394 raft_consensus.cc:606] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:24.715462 17723 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:24.800616 17781 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:24.832638 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:24.850608 17632 consensus_queue.cc:1048] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 12:22:24.801762 17677 raft_consensus.cc:697] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 3 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:24.815184 17659 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:24.835602 16442 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:24.897332 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 61812.3us
I20260709 12:22:24.836644 16516 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:24.813565 17569 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:24.837126 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 333598us
I20260709 12:22:24.837953 16551 raft_consensus.cc:1240] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:24.953053 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:24.827448 17725 raft_consensus.cc:697] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:24.953926 17263 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
W20260709 12:22:24.952953 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 136010us
I20260709 12:22:24.954772 17725 consensus_queue.cc:237] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:24.833696 17670 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:24.956256 16367 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:24.956218 17670 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:24.847306 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 67861.2us
I20260709 12:22:24.958011 17576 raft_consensus.cc:2804] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:24.958418 17576 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:24.848716 17737 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3.
I20260709 12:22:24.959021 17515 raft_consensus.cc:993] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50: : Instructing follower ef57beab0e1d40d585983873c55107e8 to start an election
I20260709 12:22:24.848142 17537 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 3. Current term is 4.
I20260709 12:22:24.959337 17670 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:24.815294 17568 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:22:24.958768 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:24.831138 17792 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:24.943480 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:24.960602 17792 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:24.960985 17792 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:24.961156 17144 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:24.945266 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:24.963711 17742 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:24.964258 17742 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:24.964766 17568 raft_consensus.cc:697] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:24.965292 17537 raft_consensus.cc:2804] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:24.965847 17537 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:24.966413 17537 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:24.966317 17568 consensus_queue.cc:237] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:24.967489 17742 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:24.948602 17266 raft_consensus.cc:1240] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:24.874657 17631 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:24.944456 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 107728us
I20260709 12:22:24.959635 17515 raft_consensus.cc:1081] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Signalling peer ef57beab0e1d40d585983873c55107e8 to start an election
I20260709 12:22:24.970836 17631 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:24.848623 17142 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:24.972971 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 124426us
I20260709 12:22:24.973280 17631 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:24.970158 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:24.974184 16400 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:24.974426 16397 raft_consensus.cc:1240] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:22:24.976361 17144 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:24.977063 17737 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:24.978192 17737 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:24.978526 17737 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:24.978858 17737 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:24.958779 17576 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:24.955258 16366 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:24.983878 17515 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:24.985414 17266 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709"
dest_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:45230
I20260709 12:22:24.986085 17266 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:24.986442 17266 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:24.982918 16401 raft_consensus.cc:1240] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 4 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:24.946954 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:25.001477 17142 leader_election.cc:400] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3.
I20260709 12:22:25.001935 17142 leader_election.cc:403] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:22:25.002951 17537 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:25.008496 17537 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:25.018488 17792 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:25.028035 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 72894us
I20260709 12:22:24.984699 17742 consensus_queue.cc:1059] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 12:22:25.080780 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:24.852860 17556 consensus_queue.cc:1048] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.091s
I20260709 12:22:24.861930 17658 consensus_queue.cc:260] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:25.494163 17742 raft_consensus.cc:3055] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:25.494869 17742 raft_consensus.cc:740] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:25.491411 17565 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
W20260709 12:22:25.437763 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 436304us
I20260709 12:22:25.443140 17266 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:25.432516 16550 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.023s	sys 0.026s
I20260709 12:22:25.407487  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:22:25.433372 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:25.439381 17537 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3.
I20260709 12:22:25.159318 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:24.926258 17651 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:24.946182 17650 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:25.471642 17642 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:25.178155 17677 consensus_queue.cc:237] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:25.610807 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:22:25.613031 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 776314us
I20260709 12:22:25.614992 16516 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:25.097851 17782 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:25.616709 17556 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:25.616425 17782 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:25.617112 17271 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:22:25.618021 17556 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:25.618505 17556 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:25.618723 17269 raft_consensus.cc:1240] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:25.618881 17556 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:25.619654 17782 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:22:25.187847 17801 log.cc:927] Time spent T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.246s	user 0.000s	sys 0.000s
W20260709 12:22:25.487414 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 456693us
I20260709 12:22:25.620461 17781 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:25.498374 17723 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:25.621474 17781 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:25.514212 17730 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:25.514549 17697 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:25.516848 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:25.631385 17723 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:22:25.632527 16470 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34826 (request call id 591) took 1309 ms (client timeout 1691 ms). Trace:
I20260709 12:22:25.632639 17723 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:25.633148 16470 rpcz_store.cc:269] 0709 12:22:24.322650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:24.330669 (+  8019us) service_pool.cc:224] Handling call
0709 12:22:25.516199 (+1185530us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b5800602a98. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64e9c 00007f2872d64a5b 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:22:25.632307 (+116108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27676544}
I20260709 12:22:25.636238 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:25.610886 17650 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:25.517491 17749 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:25.638988 17749 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:25.518919 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:25.641094 17642 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:25.642161 17642 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:25.514581 17746 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:22:25.541173 16550 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.031s	sys 0.034s
I20260709 12:22:25.644881 16473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 588) took 1323 ms. Trace:
I20260709 12:22:25.494186 17658 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:25.645506 16473 rpcz_store.cc:276] 0709 12:22:24.321603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:24.330797 (+  9194us) service_pool.cc:224] Handling call
0709 12:22:24.389915 (+ 59118us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:24.451244 (+ 61329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:24.458359 (+  7115us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:24.535994 (+ 77635us) log.cc:844] Serialized 26 byte log entry
0709 12:22:24.865546 (+329552us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:22:24.874117 (+  8571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:24.881208 (+  7091us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:25.576670 (+695462us) raft_consensus.cc:1685] finished
0709 12:22:25.585543 (+  8873us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:25.644552 (+ 59009us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":78971,"raft.queue_time_us":11450,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":14288,"thread_start_us":264353,"threads_started":1,"wal-append.queue_time_us":1002866}
I20260709 12:22:25.648859 17665 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:25.650283 17723 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:25.651335 17642 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:25.656091 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:25.656489 17567 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:25.564632 17792 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:22:25.539011 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 66047.2us
W20260709 12:22:25.593981 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 640401us
I20260709 12:22:25.596879 17142 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:25.597548 17742 consensus_queue.cc:260] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:25.599220 17266 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:22:25.572721 16543 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37226 (request call id 899) took 1322 ms (client timeout 1564 ms). Trace:
I20260709 12:22:25.608889 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:25.662109 16397 raft_consensus.cc:3055] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:25.609045 17651 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:24.930608 17771 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:25.668901 17771 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:25.669632 17771 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:25.662715 16397 raft_consensus.cc:740] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:25.669011 17650 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:22:25.667754 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 51540.6us
W20260709 12:22:25.661478 16543 rpcz_store.cc:269] 0709 12:22:24.250644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:24.254768 (+  4124us) service_pool.cc:224] Handling call
0709 12:22:25.567576 (+1312808us) spinlock_profiling.cc:243] Waited 516 ms on lock 0x7b5400304654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:25.572466 (+  4890us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1186888576}
W20260709 12:22:25.679380 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 82781.1us
I20260709 12:22:25.679862 17782 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:25.680323 17651 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:25.680976 17782 consensus_queue.cc:1048] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:25.683560 17799 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:25.684460 17799 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:25.686090 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:25.689194 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:25.690644 16400 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:25.692476 17790 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:25.694154 17567 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:25.695190 16514 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:25.696141 17745 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:25.663221 17781 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:25.698137 17642 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:25.723927 17790 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:25.725473 17790 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:25.713244 16397 consensus_queue.cc:260] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:25.728101 17688 raft_consensus.cc:993] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2: : Instructing follower 610f4feda11942739236f2b121a4b9c1 to start an election
I20260709 12:22:25.744864 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:25.734688 17790 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:25.746944 17144 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:25.755216 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:25.755290 17568 consensus_queue.cc:1059] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:25.756672 16395 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:25.757032 17568 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:25.758013 17568 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:25.760295 17520 raft_consensus.cc:1081] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Signalling peer 610f4feda11942739236f2b121a4b9c1 to start an election
I20260709 12:22:25.760751 16397 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:25.761382 16441 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:25.761741 17520 raft_consensus.cc:3055] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:25.762251 17520 raft_consensus.cc:740] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:22:25.760430 17822 log.cc:927] Time spent T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:22:25.763921 16441 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:25.766287 17819 raft_consensus.cc:2804] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:25.766997 17819 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:25.767553 17819 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:25.778017 17568 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:25.763729 17520 consensus_queue.cc:260] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:25.787654 17806 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:25.788179 17806 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:25.788708 17806 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:25.793761 16366 leader_election.cc:336] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.717s (SENT)
I20260709 12:22:25.794693 16366 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:25.796075 17790 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:25.753885 16516 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:25.800832 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 59573.5us
I20260709 12:22:25.806214 16223 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53636 (request call id 129) took 5740 ms. Trace:
I20260709 12:22:25.817034 16223 rpcz_store.cc:276] 0709 12:22:20.065276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:20.065450 (+   174us) service_pool.cc:224] Handling call
0709 12:22:25.805791 (+5740341us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":5}
I20260709 12:22:25.832297 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:25.785856 17665 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:25.801039 17520 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:25.836896 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 142256us
I20260709 12:22:25.801827 17796 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:22:25.845739 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 98845.8us
W20260709 12:22:25.848445 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:25.841380 17665 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:25.841142 17796 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:25.853109  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:25.856707 17796 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:25.898008 17797 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:25.898257 17825 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:25.898741 16551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 701) took 1240 ms. Trace:
I20260709 12:22:25.898949 17825 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:25.899227 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:25.899166 16551 rpcz_store.cc:276] 0709 12:22:24.658392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:24.661648 (+  3256us) service_pool.cc:224] Handling call
0709 12:22:24.661955 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:24.953615 (+291660us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:25.893213 (+939598us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b5400285354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:25.898559 (+  5346us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":404202240}
I20260709 12:22:25.900425 16543 raft_consensus.cc:2393] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 2: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:25.901834 17825 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:25.902103 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:25.898092 17670 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:25.898887 17797 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:25.906930 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:25.910813 17567 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:22:25.911939 17576 consensus_meta.cc:287] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.931s	user 0.018s	sys 0.015s
W20260709 12:22:25.919064 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 104734us
I20260709 12:22:25.888173 16224 catalog_manager.cc:5697] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
W20260709 12:22:25.920006 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 156430us
I20260709 12:22:25.920568  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:22:25.920677 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 65126.8us
I20260709 12:22:25.920006 17749 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:25.933252 16514 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:25.933797 17737 consensus_meta.cc:287] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.954s	user 0.009s	sys 0.035s
I20260709 12:22:25.932548 16224 catalog_manager.cc:5697] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 3 to 4, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 4 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:25.934331 17737 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:25.936585 17581 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:25.937383 17581 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:25.940881 17664 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:25.940951 16514 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:25.942767 17664 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:25.944052 17659 consensus_meta.cc:287] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.997s	user 0.013s	sys 0.021s
I20260709 12:22:25.944640 17659 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:25.945528 17794 consensus_queue.cc:1048] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:25.924384 17819 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:25.925212 17750 consensus_meta.cc:287] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 1.122s	user 0.016s	sys 0.012s
I20260709 12:22:25.946867 17771 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:25.947512 17750 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:25.948817 17819 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:25.947966 16514 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:25.927810 17696 consensus_meta.cc:287] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 1.170s	user 0.014s	sys 0.011s
I20260709 12:22:25.951259 17797 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:25.951488 17696 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:25.952952 16548 raft_consensus.cc:1240] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 1. Current term is 2. Ops: []
W20260709 12:22:25.953855 17777 consensus_meta.cc:287] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 1.277s	user 0.023s	sys 0.015s
I20260709 12:22:25.927659 17817 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:25.954648 17817 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:25.956465 17659 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:25.957258 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:25.940269 17581 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:25.958030 17782 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:22:25.915352 17569 consensus_meta.cc:287] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.747s	user 0.025s	sys 0.003s
I20260709 12:22:25.958746 17782 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:25.959177 17737 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:25.959470 17569 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:25.960119 17737 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:25.961066 17737 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:25.961742 17569 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:25.962445 17569 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:25.962862 17569 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:25.964732 17270 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:25.959179 16402 raft_consensus.cc:1275] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:25.950150 16441 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:25.950749 17742 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:25.957337 17781 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:25.927134 17255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 518) took 1457 ms. Trace:
I20260709 12:22:25.969988 17696 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:25.970604 16397 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:25.937983 16224 catalog_manager.cc:5697] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 1 to 2, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 2 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:26.001952 17781 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:26.003723 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 53759.1us
I20260709 12:22:26.003538 16224 catalog_manager.cc:5697] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 2 to 3, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 3 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:26.005043 16441 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:25.948971 16546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 693) took 1309 ms. Trace:
I20260709 12:22:25.963984 17737 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:26.005388 16224 catalog_manager.cc:5697] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 2 to 3, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 3 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:26.007882 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:26.005663 17781 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:22:26.012076 16367 leader_election.cc:336] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491): Timed out: RequestConsensusVote RPC to 127.5.58.4:44491 timed out after 1.564s (SENT)
I20260709 12:22:25.994617 17794 raft_consensus.cc:2749] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:26.015156 16400 raft_consensus.cc:2393] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 2: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:26.020507 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:26.021710 16224 catalog_manager.cc:5697] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 1 to 2, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 2 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:26.022599 16367 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:25.970130 17576 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:26.031109 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 83265.8us
I20260709 12:22:26.018414 17797 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:26.027422 17569 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:26.000305 17255 rpcz_store.cc:276] 0709 12:22:24.469342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:24.471582 (+  2240us) service_pool.cc:224] Handling call
0709 12:22:24.536040 (+ 64458us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:24.578205 (+ 42165us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:24.597611 (+ 19406us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:25.779862 (+1182251us) spinlock_profiling.cc:243] Waited 3.25 ms on lock 0x7b2c00000708. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:22:25.780748 (+   886us) log.cc:844] Serialized 26 byte log entry
0709 12:22:25.851867 (+ 71119us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:22:25.870723 (+ 18856us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:25.890002 (+ 19279us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:25.926530 (+ 36528us) raft_consensus.cc:1685] finished
0709 12:22:25.926555 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:25.926883 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":32353,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":134,"spinlock_wait_cycles":7479680,"thread_start_us":64010,"threads_started":1,"wal-append.queue_time_us":110880}
I20260709 12:22:26.041213 17782 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:26.043013 17806 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:26.040145 17797 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:26.009115 16546 rpcz_store.cc:276] 0709 12:22:24.639682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:24.653742 (+ 14060us) service_pool.cc:224] Handling call
0709 12:22:25.868372 (+1214630us) spinlock_profiling.cc:243] Waited 18 ms on lock 0x7b58005e9298. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:22:25.868705 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:25.869135 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:22:25.869532 (+   397us) log.cc:844] Serialized 14 byte log entry
0709 12:22:25.948424 (+ 78892us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:25.948500 (+    76us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:25.948571 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:25.948647 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:25.948780 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":189,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":67,"raft.queue_time_us":10395,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":113,"spinlock_wait_cycles":41395072,"thread_start_us":58482,"threads_started":1}
I20260709 12:22:26.048882 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:26.050130 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:26.050457 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:26.052495 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:26.034390 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:26.054119 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:26.055751 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:26.033738 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:26.036630 16400 raft_consensus.cc:1240] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:26.057192 17556 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:26.038722 17665 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:26.059588 17817 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:26.059947 17670 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:26.060708 17670 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:26.061441 17665 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:26.061391 17829 consensus_queue.cc:1059] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:26.035130 16394 raft_consensus.cc:1275] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:26.062613 17829 consensus_queue.cc:1243] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer f99066d13e524238aec6aa2024a8bd50 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:22:26.016378 17581 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:26.040426 17819 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:22:26.035046 17824 log.cc:927] Time spent T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.174s	user 0.004s	sys 0.000s
I20260709 12:22:26.085388 16468 raft_consensus.cc:1275] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:26.093585 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:22:26.098408 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 93566us
W20260709 12:22:26.068898 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:26.063809 17670 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:26.095707 16474 raft_consensus.cc:1240] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:22:26.080325 17556 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:26.087205 17270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 519) took 1136 ms. Trace:
I20260709 12:22:26.086094 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:26.104423 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:22:26.093053 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 84880.5us
I20260709 12:22:26.119441 16224 catalog_manager.cc:5697] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 1 to 3, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 3 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:26.066004 17771 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:26.120533 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:26.118285 17631 consensus_queue.cc:1059] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:26.121285 16396 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:26.121798 17631 raft_consensus.cc:3055] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:26.122254 17631 raft_consensus.cc:740] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:26.085095 16401 raft_consensus.cc:2393] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 2: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:26.123437 17631 consensus_queue.cc:260] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:26.083009 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:26.124958 16402 raft_consensus.cc:1240] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:26.073639 17796 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:26.085067 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:26.069799 17790 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:26.075096 16548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 705) took 1263 ms. Trace:
I20260709 12:22:26.129583 17796 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:26.129801 16548 rpcz_store.cc:276] 0709 12:22:24.812093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:24.812925 (+   832us) service_pool.cc:224] Handling call
0709 12:22:24.813279 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:25.953363 (+1140084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:26.074509 (+121146us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400bb4c54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:26.075016 (+   507us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2619918720}
I20260709 12:22:26.130750 16396 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
W20260709 12:22:26.113587 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 96975.1us
I20260709 12:22:26.109787 17797 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:22:26.079654 17831 log.cc:927] Time spent T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.230s	user 0.000s	sys 0.001s
I20260709 12:22:26.132092 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:26.085745 17651 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:26.067600 17750 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:26.119007 17270 rpcz_store.cc:276] 0709 12:22:24.950404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:24.950521 (+   117us) service_pool.cc:224] Handling call
0709 12:22:24.950833 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:26.009760 (+1058927us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:26.086904 (+ 77144us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54005f6c54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:26.087150 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2330685696}
I20260709 12:22:26.121529 16224 catalog_manager.cc:5697] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 1 to 3, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 3 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:26.064780 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:26.134441 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:26.063740 16549 raft_consensus.cc:1275] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:26.063931 17829 raft_consensus.cc:3055] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:26.068092 17799 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:26.141592 17144 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:26.068169 16442 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:26.086526 17576 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:26.145818 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:26.099431 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:22:26.148036 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 44177.8us
I20260709 12:22:26.065320 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:26.103317 17806 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:26.151993 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:26.159528 16398 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:26.160303 16550 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:22:26.160962 16401 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:26.141438 17829 raft_consensus.cc:740] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:26.150813 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:26.147146 17797 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:26.168464 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:26.156976 16224 catalog_manager.cc:5697] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 2 to 3, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 3 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:26.170878 16516 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:26.167963 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:26.140908 17651 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:26.164654 17796 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:26.167656 17631 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:26.179502 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:26.167666 17844 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:26.184253 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:26.189741 17661 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:26.190480 17661 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:26.191074 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:26.191706 17255 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:26.194443 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:26.170048 16470 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:26.195197 17266 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:22:26.196043 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:26.196537 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:26.196756 16400 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:26.183218 17780 consensus_queue.cc:1059] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 12:22:26.197237 17269 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
I20260709 12:22:26.183290 17797 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:26.197726 17780 raft_consensus.cc:3055] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:26.198115 17780 raft_consensus.cc:740] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:22:26.185731 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 117691us
I20260709 12:22:26.199379 16224 catalog_manager.cc:5697] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 3 to 4, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 4 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:22:26.203395 16472 raft_consensus.cc:2393] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 4: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:26.199260 17144 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:26.212041 17829 consensus_queue.cc:260] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:26.213971 17829 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:26.216879 17780 consensus_queue.cc:260] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:26.222687 17725 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:26.223408 17661 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:26.223912 17144 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:26.224045 17661 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:22:26.224733 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:26.207010 17763 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:26.226467 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:26.226653 17799 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:26.229514 17799 raft_consensus.cc:697] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:26.230809 17799 consensus_queue.cc:237] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:26.235468 17651 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:26.225109 17144 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:26.205022 17844 raft_consensus.cc:697] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:26.200779 17796 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:26.236423 17266 raft_consensus.cc:1275] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:26.243526 17144 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:26.218611 17780 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:26.236873 17763 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:26.248275 17763 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:26.251477 16398 raft_consensus.cc:1275] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:26.253302 17750 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:26.253553 17642 consensus_queue.cc:1059] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:26.254132 17750 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:26.254482 17664 raft_consensus.cc:3055] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:26.253556 17677 consensus_queue.cc:1048] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:22:26.255108 17664 raft_consensus.cc:740] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:26.260443 17650 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:26.260293 17664 consensus_queue.cc:260] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:26.260938 17650 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:26.262398 16400 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:26.224910 17665 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:26.267805 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:26.270370 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:26.270747 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:26.272590 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:26.273350 16394 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:26.274590 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:22:26.275458 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 111297us
I20260709 12:22:26.276654 16516 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:26.278976 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:26.245947 17651 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:26.280803 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:26.264875 17650 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:26.282264 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:26.285156 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:26.284045 17269 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:26.279287 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:26.289888 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:26.290534 16395 raft_consensus.cc:1275] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:26.291208 17664 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:26.291917 17266 raft_consensus.cc:1240] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:26.293900 17725 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:26.294651 17725 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:26.296742 17750 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:26.287887 17556 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:26.300037 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:26.300904 17258 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:22:26.300777 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:22:26.301467 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:26.301990 17263 raft_consensus.cc:2393] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 2: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:26.302105 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:26.298391 17796 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:26.302891 16542 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 1.
I20260709 12:22:26.300948 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:26.307539 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:26.307771 17797 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:26.308925 17254 raft_consensus.cc:2410] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:22:26.309113 17725 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:26.310336 17254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45224 (request call id 519) took 1368 ms. Trace:
I20260709 12:22:26.310720 17254 rpcz_store.cc:276] 0709 12:22:24.941803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:24.946030 (+  4227us) service_pool.cc:224] Handling call
0709 12:22:25.610395 (+664365us) spinlock_profiling.cc:243] Waited 43.6 ms on lock 0x7b58003b8298. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64eee 000000000036eb39 00007f287add81e0 00007f287aea10c4 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:22:26.310094 (+699699us) spinlock_profiling.cc:243] Waited 697 ms on lock 0x7b54005f7b54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:26.310252 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1702333440}
I20260709 12:22:26.288522 17270 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 1.
I20260709 12:22:26.311707 17661 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:26.312593 16398 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 1.
I20260709 12:22:26.313550 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:26.314062 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:26.314424 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:26.314978 16548 raft_consensus.cc:2393] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 4: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:26.316795 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:26.317118 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:26.317354 17271 raft_consensus.cc:1275] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:26.317871 17264 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:26.318516 17742 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:26.316062 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:26.319085 17742 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:26.321363 17742 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:26.312376 17661 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:26.322875 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:26.323640 16547 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:26.324497 17661 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:26.324817 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:26.326186 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:26.326725 17266 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:26.336645 17270 raft_consensus.cc:1275] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:26.338238 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:26.285355 17651 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:26.341183 17651 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:26.344434 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:26.329213 17665 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:26.345229 17254 raft_consensus.cc:1275] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:26.345523 17271 raft_consensus.cc:2410] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:22:26.345582 16546 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:26.349619 17665 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:26.349287 17844 consensus_queue.cc:237] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:26.352751 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:22:26.356676 17845 log.cc:927] Time spent T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:22:26.357051 17650 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:26.283188 16367 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:26.358619 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 99513.6us
I20260709 12:22:26.359565 17750 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:26.360018 17750 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:26.360081 17790 consensus_queue.cc:1048] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:26.360487 17750 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:26.362130 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:26.363664 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:26.364121 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:26.365018 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:26.366333 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:26.367592 16469 raft_consensus.cc:1240] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 LEADER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:26.368755 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:26.368891 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:26.367561 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:22:26.377383 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 100838us
I20260709 12:22:26.359128 17650 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:26.391894 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:26.395123 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:26.393245 16516 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:26.391523 17144 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:26.397302 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:26.398157 16400 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:26.399829 16469 raft_consensus.cc:1275] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:26.399991 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:26.400990 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:26.401772 16546 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:26.396467 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:26.419956 16475 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:26.423835 17269 raft_consensus.cc:1275] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:26.428512 17143 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:26.372371 17750 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:26.371703 17763 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:26.435704 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:26.435431 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:26.443572 17143 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:26.449052 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:26.459079 16516 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:26.392949 17665 consensus_queue.cc:1048] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 12:22:26.433037 17650 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:22:26.463987 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:26.466369 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:26.469305 16472 raft_consensus.cc:1240] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:26.449323 17725 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:26.467530 17143 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:26.475061 17143 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:26.454789 17797 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:26.476243 17797 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:26.476910 17797 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:26.488624 17569 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:26.489174 17569 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:26.489653 17569 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:26.462278 17255 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:26.491030 17799 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:26.491542 17799 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:26.491942 17799 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:26.494688 16403 raft_consensus.cc:1275] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:26.495532 17763 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:26.460662 17661 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:26.503266 16475 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 3.
I20260709 12:22:26.498179 16469 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:26.501529 16400 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:26.453998 17790 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:26.472457 17651 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:26.504755 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:26.462342 17650 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:22:26.512738 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 53756us
I20260709 12:22:26.513393 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:26.468710 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:26.507761 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 72559.5us
I20260709 12:22:26.435969 17796 consensus_queue.cc:1048] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 12:22:26.489272 17143 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:26.505338 17794 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:26.456779 17677 consensus_queue.cc:1048] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
W20260709 12:22:26.510918 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 84410.4us
I20260709 12:22:26.502776 17806 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:26.545207 17264 raft_consensus.cc:1240] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 4. Current term is 5. Ops: []
W20260709 12:22:26.548496 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 59336.3us
I20260709 12:22:26.513916 16516 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:26.516657 17834 consensus_queue.cc:1048] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:22:26.517765 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:26.554297 16469 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:26.529538 17797 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:26.531819 16471 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:26.525754 16394 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:26.532397 17790 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:26.463177 17696 consensus_queue.cc:1059] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 12:22:26.506387 17144 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:26.545158 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:26.515115 17650 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:22:26.561506 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 55202.7us
I20260709 12:22:26.553272 17261 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:26.553328 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:26.525750 16473 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:26.559129 17143 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:26.564159 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 50368us
I20260709 12:22:26.578464 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:26.583544 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:26.582991 16476 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:26.552795 17742 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:26.589219 16475 raft_consensus.cc:1240] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:26.589641 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:26.591041 17144 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:26.594360 17255 raft_consensus.cc:2393] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 2: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:26.596637 17696 raft_consensus.cc:3055] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:26.597072 17696 raft_consensus.cc:740] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:26.597949 17696 consensus_queue.cc:260] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:26.599162 17696 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:26.600498 17651 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:26.606046 17790 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:26.579304 17661 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:26.607152 17661 raft_consensus.cc:697] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:26.608197 17661 consensus_queue.cc:237] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:26.608964 17750 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:26.611867 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:26.613258 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:26.613744 16476 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:26.614248 17797 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:26.615350 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:26.619059 17142 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:26.556394 17806 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:26.620035 17844 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:26.620644 17844 raft_consensus.cc:697] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:26.621377 17269 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:26.621562 17806 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:26.622215 17782 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:26.623095 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:26.621667 17844 consensus_queue.cc:237] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:26.624233 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:26.624439 17665 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:26.591264 17742 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:26.625304 17742 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:26.625437 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:26.625779 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:26.626593 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:26.626895 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:26.586301 17780 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:26.631100 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:26.629623 17266 raft_consensus.cc:2468] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:26.637444 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:26.639120 16542 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:26.639036 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:26.644627 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:26.645253 16397 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:26.645169 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:22:26.645846 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:26.649256 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:26.649986 16475 raft_consensus.cc:2393] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 4: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:26.623554 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:26.661171 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:26.673308 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:26.674652 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:26.676281 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:26.682209 17142 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:26.683768 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:26.684540 16547 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:26.684592 17806 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:26.686600 16367 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:26.688066 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:26.693972 17750 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:26.694677 17750 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:26.694897 17790 consensus_queue.cc:1059] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:26.695365 17750 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:26.691887 17143 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:22:26.710947 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:26.710763 16476 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:26.696491 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:26.711864 17806 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:26.727949 16470 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:26.735634 16542 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:26.744520 17143 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:26.750438 17651 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:26.760106 16397 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:22:26.738727 17782 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:26.766536 17144 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:26.762324 17651 raft_consensus.cc:697] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:26.768512 16441 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:26.769204 17569 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:26.773840 16473 raft_consensus.cc:1275] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:26.774456 17650 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:26.743355 17806 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:26.777087 17782 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:26.786274 17780 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:26.791735 17688 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:22:26.800393 17859 log.cc:927] Time spent T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
W20260709 12:22:26.808384 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 121856us
I20260709 12:22:26.777369 17799 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:26.812885 17857 log.cc:927] Time spent T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 12:22:26.812282 17651 consensus_queue.cc:237] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:26.816093 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 47790.5us
I20260709 12:22:26.807379 17782 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:26.824551 17258 raft_consensus.cc:2393] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 4: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
W20260709 12:22:26.824631 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 260981us
I20260709 12:22:26.829702 16441 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:26.820781 16367 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
W20260709 12:22:26.834517 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 152367us
I20260709 12:22:26.837905 17142 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
W20260709 12:22:26.859555 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 276913us
I20260709 12:22:26.861109 16514 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:26.862789 16516 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:26.833566 17688 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:26.866624 17725 consensus_queue.cc:1059] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
W20260709 12:22:26.886056 17858 log.cc:927] Time spent T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
W20260709 12:22:26.885207 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 69183.3us
I20260709 12:22:26.886505 16515 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:26.830163 17742 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:26.891726 17862 raft_consensus.cc:2804] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:26.892079 16514 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:26.888675 17780 raft_consensus.cc:697] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:26.895936 17799 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:26.897382 17806 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:26.889109 17794 consensus_queue.cc:1048] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 12:22:26.880153 17856 consensus_queue.cc:1048] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 12:22:26.899276 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:26.899652 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:26.893584 17142 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:26.871846 17665 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:26.910693 17869 raft_consensus.cc:3055] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:26.924022 16468 raft_consensus.cc:1240] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:26.924461 17864 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:26.924908 17569 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:26.925132 17864 raft_consensus.cc:697] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:26.924086 17869 raft_consensus.cc:740] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:26.926430 17864 consensus_queue.cc:237] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:26.927522 17869 consensus_queue.cc:260] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:26.914541 17650 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:26.916519 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:22:26.912504 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 88517.7us
W20260709 12:22:26.963486 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 69985.3us
I20260709 12:22:26.929472 17780 consensus_queue.cc:237] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:26.964427 17142 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:26.886916 17661 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:26.930289 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 43936.6us
I20260709 12:22:26.930873 17796 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:26.931619 17862 raft_consensus.cc:697] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:26.913913 17868 consensus_queue.cc:1048] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:26.912106 17855 raft_consensus.cc:3055] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:22:26.937704 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 133020us
I20260709 12:22:26.928995 17665 raft_consensus.cc:697] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:26.978940 17142 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:26.987576 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:26.988582 17142 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:26.970834 17576 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:26.967303 17796 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:27.002686 17796 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:27.007453 17782 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:27.009598 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:26.984082 17650 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:27.010535 16546 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:26.977380 17569 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:27.014406 17665 consensus_queue.cc:237] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:26.970317 17855 raft_consensus.cc:740] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:27.016999 17855 consensus_queue.cc:260] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:26.994860 17642 consensus_queue.cc:1059] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 12:22:26.994606 17576 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:26.995044 17661 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:22:27.025583 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 119284us
I20260709 12:22:27.027043 16441 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:27.030959 17142 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:26.991679 17869 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:27.028036 16367 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:27.024843 17855 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:27.042047 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:27.027964 17576 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:27.044394 16401 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:27.027782 17868 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:27.046751 16367 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:27.050945 16400 raft_consensus.cc:1275] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:22:27.053973 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 60769.1us
I20260709 12:22:27.056668 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:27.057487 16549 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:27.059057 16398 raft_consensus.cc:1240] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:22:27.040856 17873 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:27.044176 17670 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:27.046998 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:27.064929 17750 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:27.065851 17142 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:27.066706 16441 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:27.067905 17742 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:27.070524 17569 consensus_queue.cc:1048] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:27.073331 17661 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:27.076349 17873 raft_consensus.cc:697] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:27.091626 16542 raft_consensus.cc:1275] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:27.092428 17651 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:27.087808 16515 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:27.085636 16401 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:27.088341 17142 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:27.091153 17688 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:27.091547 16441 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:27.094336 16549 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:27.094611 17144 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:27.060366 17875 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:27.043826 17862 consensus_queue.cc:237] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:27.053260 17868 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:27.134711 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 47023.8us
W20260709 12:22:27.138867 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 50601.4us
I20260709 12:22:27.143873 16468 raft_consensus.cc:3055] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:27.107131 17670 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:27.107761 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:27.122938 17873 consensus_queue.cc:237] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:27.151845 17670 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:27.152060 16547 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:27.152467 17670 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:27.125958 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:22:27.160416 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 65887.2us
I20260709 12:22:27.152613 17885 consensus_queue.cc:1048] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:27.107175 17882 consensus_queue.cc:1048] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 12:22:27.101931 17661 consensus_queue.cc:1048] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 12:22:27.122756 17750 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:27.154655 16515 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:27.160378 17266 raft_consensus.cc:1240] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:27.167833 16546 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:27.163208 17834 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:27.163498 16394 raft_consensus.cc:1275] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:27.163883 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:27.172634 17143 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:27.161741 16468 raft_consensus.cc:740] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:27.177047 16547 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
W20260709 12:22:27.195539 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 120922us
W20260709 12:22:27.082707 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 148960us
I20260709 12:22:27.172549 17881 consensus_queue.cc:1048] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 12:22:27.199692 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:22:27.202565 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:27.094135 17877 raft_consensus.cc:2764] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:22:27.211019 16367 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:27.152679 17696 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:27.155821 17651 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:27.154937 17875 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:27.234869 17875 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:27.234930 17884 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:27.235905 17884 consensus_queue.cc:1059] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:27.236683 17797 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:27.237203 17797 raft_consensus.cc:697] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:27.088769 17790 consensus_queue.cc:1048] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.121s
I20260709 12:22:27.200640 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:27.237807 17696 raft_consensus.cc:3055] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:27.239602 16367 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:27.126410 17796 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:27.232813 17271 raft_consensus.cc:1240] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:27.233677 17806 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:27.214391 17834 raft_consensus.cc:697] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:27.232271 17750 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:22:27.257506 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 86010us
I20260709 12:22:27.252562 16468 consensus_queue.cc:260] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:27.263888 16401 raft_consensus.cc:1275] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:27.263962 16515 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:27.280726 16515 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:27.247664 17861 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:27.249123 17782 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:27.250151 17696 raft_consensus.cc:740] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:27.256165 17651 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:27.293499 16515 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:27.293675 16401 raft_consensus.cc:1240] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:27.262653 17888 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:22:27.288870 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 116311us
I20260709 12:22:27.263029 17750 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:27.304545 17750 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:27.306171 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:27.307750 17796 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:27.296355 16394 raft_consensus.cc:1240] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:27.296543 17861 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:27.301831 16366 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:27.317934 16468 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:27.328322 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:27.296100 17867 consensus_queue.cc:1048] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 12:22:27.341090 17696 consensus_queue.cc:260] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:27.336052 17782 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:27.252539 17797 consensus_queue.cc:237] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:27.265507 17806 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:27.319433 17888 raft_consensus.cc:697] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 5 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
W20260709 12:22:27.347649 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 54229.7us
I20260709 12:22:27.329224 17556 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:27.335842 17885 raft_consensus.cc:2804] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:27.345252 16401 raft_consensus.cc:1240] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:27.360728 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:27.361105 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:22:27.361615 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:27.361711 17696 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:27.312001 17631 consensus_queue.cc:1059] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 12:22:27.348577 17790 raft_consensus.cc:2749] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:27.363050 17261 raft_consensus.cc:1240] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:27.296931 17651 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:27.372493 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:27.526885 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 225105us
I20260709 12:22:27.357144 17642 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:22:27.420018 16543 raft_consensus.cc:1240] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:27.416788 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:27.534974 16468 raft_consensus.cc:1275] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:27.375469 17885 raft_consensus.cc:697] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 2 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:27.539826 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:27.411971 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:27.411579 16476 raft_consensus.cc:1240] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:27.411281 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:22:27.413715 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:27.430855 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:27.415396 16542 raft_consensus.cc:1240] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:27.387977 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:27.543313 16551 raft_consensus.cc:2410] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:22:27.542639 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:27.543764 16402 raft_consensus.cc:2393] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 4: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:27.426188 17888 consensus_queue.cc:237] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:27.410104 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:27.424947 17868 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:27.355504 17834 consensus_queue.cc:237] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:27.432042 17861 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:27.429121 17631 raft_consensus.cc:3055] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:27.539427 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:27.554800 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:27.557292 17261 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:27.372750 17576 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:27.365348 17556 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:22:27.541683 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 185132us
I20260709 12:22:27.567104 16366 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:27.397608 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:27.540544 17269 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:27.355167 17782 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:27.530856 17790 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:27.548671 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:27.550248 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:27.552318 16401 raft_consensus.cc:1240] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:27.419075 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:27.560344 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:27.562558 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:27.562719 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:27.562986 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:27.563689 16398 raft_consensus.cc:1240] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:27.565425 16400 raft_consensus.cc:1240] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:27.547982 17725 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:27.573277 16544 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
I20260709 12:22:27.588215 16550 raft_consensus.cc:2368] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 3. Current term is 4.
I20260709 12:22:27.592284 16551 raft_consensus.cc:2393] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 3: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:27.596314 17258 raft_consensus.cc:1240] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:27.556802 17631 raft_consensus.cc:740] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:27.603196 16549 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
I20260709 12:22:27.604090 17266 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:27.583431 17556 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:27.609375 16516 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:27.612478 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:27.614675 16544 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:22:27.554154 17874 consensus_queue.cc:1059] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.063s
I20260709 12:22:27.622751 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:27.623714 16549 raft_consensus.cc:2393] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 4: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:27.575126 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:22:27.648391 17905 log.cc:927] Time spent T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:22:27.650233 17261 raft_consensus.cc:1240] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:27.651748 17269 raft_consensus.cc:1240] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:27.589943 17906 consensus_queue.cc:1059] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 12:22:27.603567 17642 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:27.619412 17661 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:27.653281 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:27.598387 17725 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:27.654342 17576 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:27.621086 16367 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:27.695039 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 74626.9us
I20260709 12:22:27.695794 16367 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:27.697443 16367 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:27.647218 17885 consensus_queue.cc:237] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:27.697904 17263 raft_consensus.cc:1275] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:27.698553 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:27.699719 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:27.615388 16546 raft_consensus.cc:2393] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 4: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:27.603458 17665 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:27.661788 16544 raft_consensus.cc:1240] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:22:27.590521 17790 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:27.672049 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 102779us
I20260709 12:22:27.672643 17631 consensus_queue.cc:260] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:27.674239 17806 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:27.622834 17877 raft_consensus.cc:3055] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:27.709704 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:27.711902 16544 raft_consensus.cc:2410] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:22:27.675918 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:27.625183 17874 consensus_queue.cc:1048] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 12:22:27.688354 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:27.689050 17264 raft_consensus.cc:1240] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:22:27.717314 17142 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:27.689644 17867 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:27.719165 17867 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:27.719529 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:27.690771 16395 raft_consensus.cc:1275] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:27.691679 17915 consensus_queue.cc:1048] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:27.720912 16367 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:27.721554 17867 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:27.721766 16544 raft_consensus.cc:2410] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:22:27.722342 16401 raft_consensus.cc:1275] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:27.723268 16545 raft_consensus.cc:1240] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:27.724565 17520 consensus_queue.cc:1059] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:22:27.726902 17903 log.cc:927] Time spent T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.316s	user 0.000s	sys 0.003s
I20260709 12:22:27.629010 17750 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:27.596786 16547 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "95954ea821224e4d962974a83d83ad44"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
 from {username='slave'} at 127.0.0.1:37242
I20260709 12:22:27.624666 17881 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:27.702039 17143 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:27.706183 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:27.718653 17873 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:27.722162 17915 consensus_queue.cc:1059] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:27.674943 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:22:27.729305 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:27.730144 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 175678us
I20260709 12:22:27.694242 17725 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:27.731441 16548 raft_consensus.cc:1240] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:27.658481 17670 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:27.711416 17877 raft_consensus.cc:740] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:27.703930 17868 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:27.740796 17269 raft_consensus.cc:2410] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:22:27.706537 17904 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:27.706771 17576 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:27.751082 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:22:27.753510 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 51553.8us
I20260709 12:22:27.779083 16403 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:27.735891 16547 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:27.779842 16547 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:27.775887 16366 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:27.779421 17143 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:27.701160 17642 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:27.790041 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 72739.5us
I20260709 12:22:27.703153 17661 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:27.740943 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:27.778079 17868 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:27.831493 17868 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:22:27.832067 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 111219us
I20260709 12:22:27.766358 17569 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:27.836046 16474 raft_consensus.cc:1240] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 1. Current term is 2. Ops: []
W20260709 12:22:27.787614 17875 consensus_meta.cc:287] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.552s	user 0.012s	sys 0.000s
I20260709 12:22:27.714207 17665 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:27.839644 16543 raft_consensus.cc:1275] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:27.767287 17651 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:27.841611 16367 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:27.838302 16547 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:27.739540 17899 consensus_queue.cc:1048] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.093s
I20260709 12:22:27.849382 16472 raft_consensus.cc:3055] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:27.849640 17261 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:27.767613 17867 consensus_queue.cc:1048] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.077s
I20260709 12:22:27.824366 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:27.858532 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:27.824090 17873 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:27.857157 16547 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:27.858177 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:27.832505 17915 raft_consensus.cc:3055] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:27.875020 17270 raft_consensus.cc:1275] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:27.871275 16472 raft_consensus.cc:740] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:27.781395 16394 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:27.883862 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:27.840102 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:22:27.886322 17910 log.cc:927] Time spent T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 12:22:27.887115 17900 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:27.838830 17888 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:27.858419 16546 raft_consensus.cc:1275] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:22:27.888633 17696 consensus_queue.cc:1059] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:27.857612 17790 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:27.847015 17576 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:27.843608 17869 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:27.898692 16474 raft_consensus.cc:3055] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:27.857373 17917 raft_consensus.cc:3055] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:27.848441 17631 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:27.871929 17569 raft_consensus.cc:697] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:27.906422 16546 pending_rounds.cc:85] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1: Aborting all ops after (but not including) 2
I20260709 12:22:27.895983 17915 raft_consensus.cc:740] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:27.857832 17913 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:27.914223 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:27.916745 16547 raft_consensus.cc:1240] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 4. Ops: []
I20260709 12:22:27.916018 16474 raft_consensus.cc:740] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:27.851241 17875 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:27.964074 17782 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:27.917299 16472 consensus_queue.cc:260] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:27.959544 17899 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:27.893764 17877 consensus_queue.cc:260] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:27.965950 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 133100us
I20260709 12:22:27.906456 17874 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:27.898167 16473 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:28.027397 17665 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:28.014667 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 238835us
I20260709 12:22:27.918933 17917 raft_consensus.cc:740] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:28.013835 16547 raft_consensus.cc:1240] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:28.009310 16472 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:28.025656 17911 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:28.036581 16367 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:27.906000 16395 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:27.951740 17873 raft_consensus.cc:697] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:27.953342 16546 pending_rounds.cc:107] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1: Aborting uncommitted NO_OP operation due to leader change: 4.3
W20260709 12:22:28.045631 17894 log.cc:927] Time spent T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:22:28.047000 17750 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:28.048864 17670 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:27.966461 17913 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:28.050861 17569 consensus_queue.cc:237] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:28.052294 17915 consensus_queue.cc:260] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:28.053278 17877 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:28.055948 17144 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:28.057183 17677 raft_consensus.cc:3055] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:28.057312 16394 raft_consensus.cc:1275] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:28.080693 17144 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:28.041064 16474 consensus_queue.cc:260] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:27.915015 17900 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:28.042287 17874 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:28.054173 17875 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:28.068817 16546 raft_consensus.cc:2889] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:28.012146 17899 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:28.084743 16366 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:28.087553 17144 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:28.088521 16474 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:22:28.081046 17677 raft_consensus.cc:740] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:28.078027 17888 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:28.085911 17806 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:28.086980 17874 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:28.106853 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 70412.6us
I20260709 12:22:28.107834 16367 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:28.067433 17911 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:28.086289 17556 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:28.106233 17915 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:28.104501 17875 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:28.106966 17806 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:28.126446 17806 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:28.108367 17782 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:28.116433 17834 consensus_queue.cc:1048] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:22:28.121810 16394 raft_consensus.cc:3055] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:22:28.116581 17862 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.003s	sys 0.005s
I20260709 12:22:28.125298 17913 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:28.132550 16474 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
W20260709 12:22:28.136062 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 51404.6us
I20260709 12:22:28.139572 16551 raft_consensus.cc:1240] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:28.118918 17888 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:28.169790 16366 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:22:28.135456 17862 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.004s	sys 0.005s
I20260709 12:22:28.132256 17782 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:28.135640 16394 raft_consensus.cc:740] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:28.136678 17797 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:28.205256 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:28.206672 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:28.213601 17750 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:28.214337 17750 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:28.214664 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:28.215449 16397 raft_consensus.cc:2393] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 3: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:28.103686 17917 consensus_queue.cc:260] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:28.218062 17917 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:28.218515 17750 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:28.208010 16472 raft_consensus.cc:1275] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:28.171372 17868 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:28.222218 17868 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:28.208282 17900 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:28.225417 16514 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:28.227142 17664 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:28.183480 16394 consensus_queue.cc:260] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:28.175417 17677 consensus_queue.cc:260] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:28.228354 16394 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:28.228525 17677 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:28.114502 17873 consensus_queue.cc:237] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:28.229945 17642 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:28.208546 17782 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:28.184562 17790 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:28.185585 17925 consensus_queue.cc:1048] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:22:28.187065 17927 log.cc:927] Time spent T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
W20260709 12:22:28.233475 17892 log.cc:927] Time spent T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.873s	user 0.005s	sys 0.000s
I20260709 12:22:28.235424 16396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 583) took 1331 ms. Trace:
I20260709 12:22:28.235808 16396 rpcz_store.cc:276] 0709 12:22:26.903635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:26.905761 (+  2126us) service_pool.cc:224] Handling call
0709 12:22:26.910959 (+  5198us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:27.060122 (+149163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:27.072122 (+ 12000us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:27.102712 (+ 30590us) log.cc:844] Serialized 26 byte log entry
0709 12:22:27.312437 (+209725us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:22:27.325070 (+ 12633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:27.332466 (+  7396us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:28.232705 (+900239us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:22:28.235108 (+  2403us) raft_consensus.cc:1685] finished
0709 12:22:28.235126 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:28.235342 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":108179,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":104,"spinlock_wait_cycles":27610368,"thread_start_us":324334,"threads_started":2,"wal-append.queue_time_us":257409}
I20260709 12:22:28.238368 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:28.240756 17797 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:28.241348 17797 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:28.216231 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:22:28.246361 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:28.189494 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:22:28.196388 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 477491us
I20260709 12:22:28.201694 17899 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:28.203301 17911 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:28.227670 17664 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:28.178352 17556 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:28.254089 16516 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:28.258138 17862 consensus_queue.cc:1048] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:28.259904 17556 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:28.261420 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:28.263756 16514 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:28.240561 17902 raft_consensus.cc:3055] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:22:28.265411 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 157674us
I20260709 12:22:28.268436 16367 leader_election.cc:400] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 3. Current term is 4.
I20260709 12:22:28.269057 16367 leader_election.cc:403] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:22:28.265556 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:28.269790 16470 raft_consensus.cc:1275] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:28.271818 16472 raft_consensus.cc:2393] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 2: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:28.282002 16398 raft_consensus.cc:1240] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:28.287315 16549 raft_consensus.cc:3055] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:28.287763 16549 raft_consensus.cc:740] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:28.291183 16398 raft_consensus.cc:1240] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:22:28.292090 17925 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:28.292776 16367 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:28.293795 17790 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:28.294214 16367 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:28.258173 17664 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:28.295395 17790 raft_consensus.cc:2804] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:28.270793 17862 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:28.299155 17271 raft_consensus.cc:1275] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:28.305248 17258 raft_consensus.cc:1240] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:28.318622 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:28.319063 16402 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:28.273079 17576 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:28.306424 16473 raft_consensus.cc:1240] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4. Ops: []
W20260709 12:22:28.309154 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:28.325281 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:28.325850 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:28.309074 17142 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:28.309929 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:28.326923 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:28.312922 16397 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:28.327509 16468 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:28.330435 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:28.331336 17911 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:28.331764 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:28.332815 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:28.333127 16471 raft_consensus.cc:1275] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:28.334676 17270 raft_consensus.cc:1275] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:28.335201 17264 raft_consensus.cc:2393] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 3: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:28.331974 17911 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:28.336470 16474 raft_consensus.cc:1275] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:28.337363 16549 consensus_queue.cc:260] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:28.338271 17911 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:28.339990 16468 raft_consensus.cc:1275] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:28.342233 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:28.314276 16367 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:28.270570 17799 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:28.305615 16400 raft_consensus.cc:1240] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:28.267079 17902 raft_consensus.cc:740] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:28.299182 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:28.262470 17834 consensus_queue.cc:1048] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 12:22:28.324404 17261 raft_consensus.cc:1240] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:28.311308 17661 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:22:28.343619 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 47585us
I20260709 12:22:28.347919 16469 raft_consensus.cc:3055] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:28.348477 16469 raft_consensus.cc:740] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:28.344218 16398 raft_consensus.cc:1240] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:28.353997 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:28.322101 17790 raft_consensus.cc:697] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:28.352519 16367 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:28.358103 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:28.345167 16394 raft_consensus.cc:1275] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:28.346422 17867 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:28.344760 17868 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:28.317890 17797 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:28.354257 16469 consensus_queue.cc:260] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:28.357285 17862 raft_consensus.cc:697] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:28.418980 17862 consensus_queue.cc:237] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:28.420881 17829 consensus_queue.cc:1059] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:28.421797 17829 raft_consensus.cc:3055] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:28.422176 17829 raft_consensus.cc:740] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:28.423112 17829 consensus_queue.cc:260] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:28.350700 17642 consensus_queue.cc:1059] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 12:22:28.378023 17742 consensus_queue.cc:1048] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 12:22:28.390180 17871 consensus_queue.cc:1048] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 12:22:28.427296 16441 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:28.429757 16441 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:28.357120 17677 consensus_queue.cc:1048] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 12:22:28.395179 17902 consensus_queue.cc:260] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:28.355698 17799 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:28.388047 17872 consensus_queue.cc:1048] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 12:22:28.403134 16476 raft_consensus.cc:1275] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:22:28.433897 16476 pending_rounds.cc:85] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 2
I20260709 12:22:28.355887 16395 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:28.434163 17799 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:28.381558 17869 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:28.392822 16400 raft_consensus.cc:1275] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:28.355655 17864 consensus_queue.cc:1059] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
W20260709 12:22:28.407478 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:27.660607  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:22:28.437358 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:28.407688 17790 consensus_queue.cc:237] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:28.434377 16476 pending_rounds.cc:107] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:22:28.438099 16476 raft_consensus.cc:2889] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:28.406653 17906 consensus_queue.cc:1059] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:28.417243 17797 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:28.417734 17917 consensus_queue.cc:1059] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:28.430867 17913 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:22:28.414835 17806 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:28.416512 17940 log.cc:927] Time spent T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 12:22:28.447907 16543 raft_consensus.cc:1240] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:22:28.448709 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:22:28.453114 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:28.454901 16469 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:22:28.455811 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:28.345554 17874 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:28.471624 16367 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:28.473827 16367 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:22:28.457533 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 83747.6us
I20260709 12:22:28.460052 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:28.377002 17569 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:28.471079 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:28.436801 17864 raft_consensus.cc:3055] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:28.485152 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:28.476820 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:28.442289 17829 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:28.478598 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 48942.7us
I20260709 12:22:28.543215 16402 raft_consensus.cc:1275] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:28.489040 17143 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:28.550715 16441 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:28.551750 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 72487.1us
I20260709 12:22:28.482719 16544 raft_consensus.cc:1275] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:28.482877 16515 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:28.544595 16548 raft_consensus.cc:1275] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:22:28.567727 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 78779us
I20260709 12:22:28.567710 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:28.569247 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:28.491999 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:28.487504 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:28.573380 16469 raft_consensus.cc:1275] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:28.577795 16224 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53626 (request call id 125) took 3671 ms. Trace:
W20260709 12:22:28.578351 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 95576.1us
I20260709 12:22:28.579991 17143 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:28.572696 17868 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:22:28.542820 17936 log.cc:927] Time spent T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.169s	user 0.000s	sys 0.000s
I20260709 12:22:28.481668 17750 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:22:28.586886 17143 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:28.584623 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:28.589805 17258 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:22:28.459640 17799 consensus_queue.cc:1048] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.116s
I20260709 12:22:28.545132 17874 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:22:28.596875 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 116310us
I20260709 12:22:28.602176 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:28.484618 17864 raft_consensus.cc:740] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:28.478736 17844 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:28.605022 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 54394.2us
I20260709 12:22:28.486987 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:28.555051 17576 consensus_queue.cc:1059] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.113s
I20260709 12:22:28.584837 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:28.545476 17782 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:28.589622 17650 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:28.588485 17900 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:28.588914 17855 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:28.595520 17915 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:28.617846 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:28.567054 17725 consensus_queue.cc:1048] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 12:22:28.567097 17780 consensus_queue.cc:1048] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 12:22:28.615845 17917 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:28.481101 17569 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:28.627506 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:28.574182 17664 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:28.613093 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:28.585973 17899 consensus_queue.cc:1048] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 12:22:28.609355 17888 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:28.637718 16367 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:28.661548 16367 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:28.613742 17867 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:28.647970 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:28.623947 16473 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:28.671358 16549 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:28.578287 16224 rpcz_store.cc:276] 0709 12:22:24.818386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:24.824175 (+  5789us) service_pool.cc:224] Handling call
0709 12:22:28.489627 (+3665452us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:22:26.517326 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:22:26.517998 (+   672us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:22:26.518050 (+    52us) write_op.cc:435] Acquired schema lock
0709 12:22:26.518083 (+    33us) tablet.cc:662] Decoding operations
0709 12:22:26.519207 (+  1124us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:22:26.519319 (+   112us) write_op.cc:641] Partition lock acquired for write op
0709 12:22:26.519346 (+    27us) tablet.cc:685] Acquiring locks for 10 operations
0709 12:22:26.520267 (+   921us) tablet.cc:701] Row locks acquired
0709 12:22:26.520314 (+    47us) write_op.cc:260] PREPARE: finished
0709 12:22:26.552133 (+ 31819us) write_op.cc:270] Start()
0709 12:22:26.586463 (+ 34330us) write_op.cc:276] Timestamp: P: 1783599746542006 usec, L: 0
0709 12:22:26.626201 (+ 39738us) op_driver.cc:348] REPLICATION: starting
0709 12:22:26.626950 (+   749us) log.cc:844] Serialized 3939 byte log entry
0709 12:22:26.961770 (+334820us) op_driver.cc:464] REPLICATION: finished
0709 12:22:27.065377 (+103607us) write_op.cc:301] APPLY: starting
0709 12:22:27.065499 (+   122us) tablet.cc:1367] starting BulkCheckPresence
0709 12:22:27.065765 (+   266us) tablet.cc:1370] finished BulkCheckPresence
0709 12:22:27.065941 (+   176us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:22:27.067390 (+  1449us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:22:27.067798 (+   408us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 12:22:27.067868 (+    70us) write_op.cc:312] APPLY: finished
0709 12:22:27.083164 (+ 15296us) log.cc:844] Serialized 76 byte log entry
0709 12:22:27.090540 (+  7376us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:27.090704 (+   164us) write_op.cc:454] Released schema lock
0709 12:22:27.090779 (+    75us) write_op.cc:341] FINISH: Updating metrics
0709 12:22:27.091317 (+   538us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:27.091359 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":17,"child_traces":[["op",{"apply.queue_time_us":103230,"apply.run_cpu_time_us":4192,"apply.run_wall_time_us":25969,"num_ops":10,"prepare.queue_time_us":252643,"prepare.run_cpu_time_us":25104,"prepare.run_wall_time_us":293783,"raft.queue_time_us":44604,"raft.run_cpu_time_us":25212,"raft.run_wall_time_us":92359,"replication_time_us":335317,"thread_start_us":351562,"threads_started":4,"wal-append.queue_time_us":220910,"wal-append.run_cpu_time_us":10111,"wal-append.run_wall_time_us":1236935}]]}
I20260709 12:22:28.628578 17651 consensus_queue.cc:1048] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 12:22:28.618834 17844 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:28.631703 17661 consensus_queue.cc:1048] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 12:22:28.696996 17142 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:28.670887 17875 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:28.655982 17631 consensus_queue.cc:1059] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:22:28.676797 17872 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:28.686424 16396 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:28.720629 17569 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:28.678295 17925 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 3. Current term is 4.
W20260709 12:22:28.726327 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 104778us
I20260709 12:22:28.726678 17271 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:28.711687 17875 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:28.686417 17877 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:22:28.736493 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:28.759824 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:28.740641 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:28.710242 17900 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:28.744226 16549 raft_consensus.cc:1275] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:28.716094 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:28.764264 16400 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:28.663656 17864 consensus_queue.cc:260] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:28.769259 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:28.771704 16469 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:22:28.758198 17875 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:28.778298 16551 raft_consensus.cc:1275] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:28.780591 16477 raft_consensus.cc:1275] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:28.762879 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:28.810107 16441 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:28.781637 16469 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:22:28.734872 17872 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:28.765688 17900 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:28.770143 17806 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:28.763626 17631 consensus_queue.cc:1059] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 12:22:28.821542 17142 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:28.834584 16400 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:22:28.800426 17864 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:28.841445 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:28.768118 17926 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:28.844060 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:28.868930 16400 raft_consensus.cc:1240] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:28.810107 17911 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:28.819298 17899 consensus_queue.cc:1048] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
W20260709 12:22:28.833352 17945 log.cc:927] Time spent T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 12:22:28.838840 17900 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:28.776089 17873 consensus_queue.cc:1048] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.070s
I20260709 12:22:28.825774 16542 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:28.844558 17661 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:28.844759 17915 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:28.886575 16402 raft_consensus.cc:1240] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 3. Current term is 4. Ops: []
W20260709 12:22:28.885615 17949 log.cc:927] Time spent T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 12:22:28.806545 17957 consensus_queue.cc:1059] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.097s
I20260709 12:22:28.907928 16551 raft_consensus.cc:1275] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:22:28.912796 17143 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:22:28.919757 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 109750us
I20260709 12:22:28.907131 17661 raft_consensus.cc:697] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:28.925488 16402 raft_consensus.cc:1240] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:28.877756 17911 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:28.928767 17871 raft_consensus.cc:2804] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:28.957923 16441 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:28.960558 17952 log.cc:927] Time spent T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 12:22:28.876935 17926 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:28.929149 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 349693us
I20260709 12:22:28.914701 17869 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:28.932247 17661 consensus_queue.cc:237] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:28.929720 17651 consensus_queue.cc:1048] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
W20260709 12:22:28.936424 17929 log.cc:927] Time spent T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:22:28.936970 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:28.937096 17664 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:22:28.939544 17956 log.cc:927] Time spent T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:22:28.940472 17950 log.cc:927] Time spent T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
W20260709 12:22:28.943823 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:28.957580 17871 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:28.981666 17782 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:28.944679 17861 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:28.973889 17834 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:29.014097 17270 raft_consensus.cc:1275] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:28.915498 17915 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:28.986168 17872 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:28.979946 17925 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:28.995852 17966 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:28.987852 17871 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:29.005218 17271 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:28.950762 16396 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:28.947853 17875 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:29.017707 16515 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:28.998242 16223 catalog_manager.cc:5697] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
W20260709 12:22:29.036659 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 78876.5us
I20260709 12:22:29.037742 16223 catalog_manager.cc:5697] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 1 to 2, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 2 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:28.992105 17642 consensus_queue.cc:1048] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 12:22:29.040606 16223 catalog_manager.cc:5697] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:29.045183 16441 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:29.043571 16515 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:29.022225 17864 consensus_queue.cc:1048] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:29.046448 16441 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:29.032852 17915 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:29.016598 17861 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:29.042660 17664 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:29.019308 17834 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:29.060549 17856 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:29.057243 17855 consensus_queue.cc:1048] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 12:22:29.073105 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:29.096601 16397 raft_consensus.cc:3055] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:29.097731 16397 raft_consensus.cc:740] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:29.077993 17957 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:29.033597 17925 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:29.101131 16397 consensus_queue.cc:260] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:29.105315 16397 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:29.110829 16472 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
mode: GRACEFUL
new_leader_uuid: "610f4feda11942739236f2b121a4b9c1"
 from {username='slave'} at 127.0.0.1:39000
I20260709 12:22:29.104255 17900 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:29.113035 16472 raft_consensus.cc:606] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Received request to transfer leadership to TS 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:29.114494 17862 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:29.115142 17926 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:29.115085 17862 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:29.115301 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:29.120967 17875 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:29.138018 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:29.138643 17270 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:29.144402 17915 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:29.145020 17976 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:29.145470 17976 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:29.145265 16395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47"
dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:60492
I20260709 12:22:29.145954 17976 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:29.146274 16395 raft_consensus.cc:606] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:22:29.146775 16395 raft_consensus.cc:614] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:29.147300 17867 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:29.104468 17957 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:29.163183 16397 raft_consensus.cc:1275] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:22:29.164496 16397 pending_rounds.cc:85] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 1
I20260709 12:22:29.164896 16397 pending_rounds.cc:107] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:22:29.165266 16397 raft_consensus.cc:2889] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:29.134888 17966 raft_consensus.cc:2764] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:22:29.169999 17900 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:22:29.171438 17970 log.cc:927] Time spent T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:22:29.175808 17270 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
I20260709 12:22:29.181674 17925 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:29.183784 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:29.184746 17976 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:29.193382 17976 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:29.154878 17725 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:29.193892 17861 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:29.167589 17862 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:29.197196 16474 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:39000
I20260709 12:22:29.308895 16474 raft_consensus.cc:606] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:22:29.309324 16474 raft_consensus.cc:614] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:29.185403 17834 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:29.311800 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:29.198854 16476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 635) took 1435 ms. Trace:
I20260709 12:22:29.312922 16394 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:29.313131 16476 rpcz_store.cc:276] 0709 12:22:27.763908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:27.764103 (+   195us) service_pool.cc:224] Handling call
0709 12:22:27.788460 (+ 24357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:28.457538 (+669078us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:29.198546 (+741008us) spinlock_profiling.cc:243] Waited 614 ms on lock 0x7b5400a69ad4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:29.198779 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1412653952}
W20260709 12:22:29.202360 17954 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 12:22:29.315661 17972 consensus_queue.cc:1059] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:29.316494 17983 consensus_queue.cc:1059] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:29.317957 17983 raft_consensus.cc:3055] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:29.318423 17983 raft_consensus.cc:740] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:29.318672 17972 raft_consensus.cc:3055] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:29.319156 17972 raft_consensus.cc:740] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:29.208863 17665 raft_consensus.cc:3055] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:29.320595 17665 raft_consensus.cc:740] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:29.321382 17972 consensus_queue.cc:260] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:29.321820 17665 consensus_queue.cc:260] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:29.322835 17972 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:29.323557 17665 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:29.211030 17844 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:22:29.218250 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 171880us
I20260709 12:22:29.218647 16396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46250 (request call id 595) took 1003 ms. Trace:
I20260709 12:22:29.324806 16396 rpcz_store.cc:276] 0709 12:22:28.214846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:28.238051 (+ 23205us) service_pool.cc:224] Handling call
0709 12:22:29.218431 (+980380us) spinlock_profiling.cc:243] Waited 438 ms on lock 0x7b5400543154. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:29.218577 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1006310272}
I20260709 12:22:29.326435 16441 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:29.327459 16470 raft_consensus.cc:1275] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:29.327805 16441 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:29.212085 17871 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:29.329056 17829 consensus_queue.cc:1059] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:29.329963 17871 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:29.330615 17915 consensus_queue.cc:1059] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:29.224684 17900 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:29.331439 17829 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:29.331544 17900 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:29.333225 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:22:29.229600 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:29.334055 16401 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:29.231499 17264 raft_consensus.cc:1240] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:22:29.337302 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:29.338171 16547 raft_consensus.cc:2410] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:22:29.233433 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:29.342427 16472 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:29.310508 17725 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:29.240379 16367 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:29.285036 17944 consensus_queue.cc:1048] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.072s
I20260709 12:22:29.241997 17143 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
W20260709 12:22:29.249555 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:29.282969 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 43582.8us
I20260709 12:22:29.288911 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:29.282819 17925 consensus_queue.cc:1048] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 12:22:29.298606 17953 raft_consensus.cc:3055] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:29.299873 17913 raft_consensus.cc:3055] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:29.287907 16473 raft_consensus.cc:1240] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:29.303791 16400 raft_consensus.cc:1240] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:22:29.188014 17957 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:22:29.348734 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 108478us
I20260709 12:22:29.353263 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:29.361371 16469 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:29.355015 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:29.362185 16472 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
W20260709 12:22:29.367007 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:29.367223 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:29.361397 16367 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:29.387940 16401 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:29.392230 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:29.394361 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:29.319490 17983 consensus_queue.cc:260] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:29.396742 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 154839us
I20260709 12:22:29.400125 16514 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:29.398296 16367 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:29.348282 17725 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:22:29.406968 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 50817.1us
I20260709 12:22:29.408390 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:29.334080 17900 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:29.352680 17884 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:29.403352 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:29.419220 16551 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:22:29.409734 16514 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:29.391167 17972 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:29.426542 16367 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:29.359143 17954 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:29.418056 17144 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:29.353065 17861 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:29.414309 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:29.415001 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:29.357657 17953 raft_consensus.cc:740] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:29.360515 17913 raft_consensus.cc:740] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:29.361784 17862 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:29.410293 16397 raft_consensus.cc:1240] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:29.423089 17271 raft_consensus.cc:1240] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:29.424484 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:29.413414 17875 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:22:29.406606 17983 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:29.357559 17906 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:29.428309 16469 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "95954ea821224e4d962974a83d83ad44"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
mode: GRACEFUL
new_leader_uuid: "610f4feda11942739236f2b121a4b9c1"
 from {username='slave'} at 127.0.0.1:39000
I20260709 12:22:29.405706 17966 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:29.429761 17725 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:29.432114 16544 raft_consensus.cc:2393] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 3: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:29.432159 16472 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:29.434394 17264 raft_consensus.cc:1240] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:29.442461 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:29.402722 17874 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:29.401991 17881 consensus_queue.cc:1048] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 12:22:29.421506 17946 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:29.477306 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:29.466111 16469 raft_consensus.cc:606] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Received request to transfer leadership to TS 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:29.443296 17270 raft_consensus.cc:2368] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 6. Current term is 7.
I20260709 12:22:29.490517 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:29.487330 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:29.430215 17972 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:29.425856 17782 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:29.495762 17258 raft_consensus.cc:2393] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 5: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:29.431078 17957 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:29.486727 16394 raft_consensus.cc:1240] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:29.493089 17742 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:29.498060 17953 consensus_queue.cc:260] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:29.498821 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:29.443001 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:29.443336 17861 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:29.501161 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 52855.8us
W20260709 12:22:29.505909 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 79410.7us
I20260709 12:22:29.526360 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:29.422428 17893 consensus_queue.cc:1048] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.078s
I20260709 12:22:29.529968 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:29.442224 17650 consensus_queue.cc:1048] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.078s
I20260709 12:22:29.510756 16469 raft_consensus.cc:614] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:29.463636 17877 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:29.442821 17954 raft_consensus.cc:697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 7 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:29.499466 17913 consensus_queue.cc:260] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:29.429926 17884 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:29.530706 16474 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:29.542842 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:29.510756 17271 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:29.525936 17957 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:29.536687 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 115430us
I20260709 12:22:29.537308 17994 consensus_queue.cc:1059] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:29.540282 17992 consensus_queue.cc:1048] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:29.510737 17725 raft_consensus.cc:697] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:29.494710 17862 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:29.519552 17867 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:29.560386 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:29.515309 17972 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:29.513594 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:29.548748 16476 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:22:29.513515 17874 raft_consensus.cc:697] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 5 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:29.556531 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:29.558681 16544 raft_consensus.cc:1240] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:29.544847 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:29.508930 17806 consensus_queue.cc:1048] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 12:22:29.567320 16548 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:29.569464 17271 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:29.492218 17906 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:29.494539 17966 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:29.555904 17951 log.cc:927] Time spent T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.740s	user 0.005s	sys 0.000s
W20260709 12:22:29.573985 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:29.524246 17782 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:29.583941 16547 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
W20260709 12:22:29.597921 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 61753.9us
W20260709 12:22:29.644865 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 94517.2us
W20260709 12:22:29.646118 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:29.655397 16469 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
I20260709 12:22:29.576249 17254 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:29.687556 17790 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:29.570643 17829 consensus_queue.cc:1059] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 12:22:29.695128 16366 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:29.697194 16516 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
W20260709 12:22:29.700435 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 149376us
I20260709 12:22:29.581338 17877 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:29.714434 16441 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:29.715436 16469 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:29.717761 16476 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
I20260709 12:22:29.719358 17925 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:29.723237 18001 raft_consensus.cc:2804] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:29.572563 17867 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:29.597431 17969 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:29.727375 16548 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
W20260709 12:22:29.809414 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:29.576128 17915 consensus_queue.cc:1059] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.123s
I20260709 12:22:29.680290 17670 raft_consensus.cc:3055] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:29.668788 17904 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:29.682545 17913 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:29.861999 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 146124us
I20260709 12:22:29.863644 16516 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:29.703779 17651 consensus_queue.cc:1048] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.149s
I20260709 12:22:29.833356 17969 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:29.839368 16469 raft_consensus.cc:1275] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:29.710729 16542 raft_consensus.cc:1240] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3. Ops: []
W20260709 12:22:29.895308 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 189979us
I20260709 12:22:29.849975 16468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 158) took 1399 ms. Trace:
I20260709 12:22:29.857759 16551 raft_consensus.cc:1240] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:29.896474 16514 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:29.824613 18001 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:29.833001 16476 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:39000
I20260709 12:22:29.893064 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:29.848749 17670 raft_consensus.cc:740] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:29.822166 17877 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:29.936697 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:29.897684 16468 rpcz_store.cc:276] 0709 12:22:28.450750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:28.450932 (+   182us) service_pool.cc:224] Handling call
0709 12:22:28.451371 (+   439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:28.548722 (+ 97351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:28.548755 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:28.549248 (+   493us) log.cc:844] Serialized 26 byte log entry
0709 12:22:28.617042 (+ 67794us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:28.780713 (+163671us) log.cc:844] Serialized 14 byte log entry
0709 12:22:28.805804 (+ 25091us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:28.805884 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:29.290160 (+484276us) spinlock_profiling.cc:243] Waited 5.47 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:22:29.826512 (+536352us) raft_consensus.cc:1685] finished
0709 12:22:29.849734 (+ 23222us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:29.849909 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49259,"prepare.queue_time_us":212203,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":20681,"raft.queue_time_us":158906,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":13974,"spinlock_wait_cycles":36255872,"thread_start_us":196886,"threads_started":2,"wal-append.queue_time_us":148806}
I20260709 12:22:29.900414 16476 raft_consensus.cc:606] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:22:29.906054 17688 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:29.912484 17902 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:29.917903 17893 raft_consensus.cc:3055] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:29.723994 17954 consensus_queue.cc:237] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:29.712317 17790 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:29.920055 17670 consensus_queue.cc:260] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:29.901940 17875 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:29.922855 17906 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:29.923341 17985 consensus_queue.cc:1059] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.377s
I20260709 12:22:29.924935 17884 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:29.925629 17664 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:29.926196 17995 log.cc:927] Time spent T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 12:22:29.927325 17996 log.cc:927] Time spent T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 12:22:29.929663 17990 log.cc:927] Time spent T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.228s	user 0.000s	sys 0.000s
I20260709 12:22:29.975947 17742 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:29.976601 17742 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:29.977353 16544 raft_consensus.cc:1240] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 2. Current term is 3. Ops: []
W20260709 12:22:29.930714 17998 log.cc:927] Time spent T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.220s	user 0.000s	sys 0.000s
I20260709 12:22:29.982892 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:29.983631 17972 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:29.877965 18004 consensus_queue.cc:1059] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.074s
I20260709 12:22:29.933717 17782 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:29.849983 17725 consensus_queue.cc:237] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:29.934654 17989 log.cc:927] Time spent T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.419s	user 0.000s	sys 0.001s
I20260709 12:22:29.834739 17925 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:29.940558 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 226282us
I20260709 12:22:29.986147 17670 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:29.900256 17969 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:29.834319 17980 consensus_queue.cc:1059] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.062s
W20260709 12:22:29.946067 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 49728us
I20260709 12:22:30.008809 16441 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:29.903015 18001 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:29.938771 17994 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:22:29.942579 17951 log.cc:927] Time spent T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:22:29.941972 17688 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:29.944314 17893 raft_consensus.cc:740] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:29.968560 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:29.945221 18002 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:29.988399 17871 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:29.973033 17875 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:29.951301 17556 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:29.984218 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:29.872568 17904 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:29.987903 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 292861us
I20260709 12:22:30.026724 17264 raft_consensus.cc:1240] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:29.896512 17874 consensus_queue.cc:237] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:30.001572 17862 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:29.985601 17742 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:29.953195 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:30.014439 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 423187us
I20260709 12:22:30.032991 17144 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:30.016171 17983 consensus_meta.cc:287] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.522s	user 0.012s	sys 0.000s
I20260709 12:22:29.972627 17957 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:30.042169 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:30.019822 16441 leader_election.cc:400] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 6. Current term is 7.
I20260709 12:22:30.042907 16441 leader_election.cc:403] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:30.021088 17258 raft_consensus.cc:1240] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:22:30.021145 17263 raft_consensus.cc:1240] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:30.044606 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:30.045496 16401 raft_consensus.cc:2410] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:22:29.967978 17664 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:30.028779 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:30.008754 17985 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:30.009394 17782 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:30.012897 17688 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:30.009769 18008 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:30.057094 16441 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:30.057305 18008 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:30.004998 17861 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:30.058837 17977 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:30.059518 17977 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 6. Current term is 7.
I20260709 12:22:30.059931 18008 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:30.061343 17867 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:30.026351 17871 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:30.065589 18006 consensus_queue.cc:1059] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:30.066303 17972 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:30.066888 17790 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:30.067863 17981 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:30.068419 17981 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:30.070570 17981 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:30.049897 18013 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:30.074805 17782 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:30.074934 18013 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:30.076078 16394 raft_consensus.cc:1275] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:30.076658 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:30.077327 16395 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:30.018529 17980 consensus_queue.cc:1243] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer ef57beab0e1d40d585983873c55107e8 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:22:30.078059 17790 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:30.078655 17790 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:30.080369 17925 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:30.080889 17790 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:30.082530 17665 consensus_meta.cc:287] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.758s	user 0.014s	sys 0.002s
I20260709 12:22:30.048377 17742 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:30.065392 17985 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:30.083416 17742 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:30.074160 17904 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:30.085688 17742 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:30.085778 17985 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:30.030069 17875 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:30.086834 17782 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:30.087976 16475 raft_consensus.cc:2410] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:22:30.088119 17875 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:30.089612 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:30.090469 16395 raft_consensus.cc:2393] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 3: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:30.092075 16366 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:30.092787 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:22:30.093389 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:30.024288 17556 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:30.093994 17904 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:30.094137 17556 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:30.094573 17904 raft_consensus.cc:697] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:30.038323 17915 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:30.100710 16472 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 3.
I20260709 12:22:30.095712 17904 consensus_queue.cc:237] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:30.060081 17977 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:30.069809 17664 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:30.120707 17913 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:30.115118 17871 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:30.131167 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:30.133513 18008 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:30.123934 17977 raft_consensus.cc:697] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:30.157018 17913 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:30.203372 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 170476us
W20260709 12:22:30.222699 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 88509.9us
I20260709 12:22:30.119882 17875 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:30.226902 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:22:30.227928 16366 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 2.014s (SENT) [suppressed 21 similar messages]
I20260709 12:22:30.228228 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:30.228996 17264 raft_consensus.cc:1240] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:30.114562 17915 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:30.056381 17893 consensus_queue.cc:260] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:30.113567 17868 consensus_queue.cc:1048] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 12:22:30.023298 18002 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:30.240064 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:22:30.242686 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:30.244601 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:30.246918 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:30.242373 17688 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:30.247692 16401 raft_consensus.cc:2393] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 2: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:30.247290 17977 consensus_queue.cc:237] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:30.248564 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:30.249286 17688 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:30.249569 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:30.239336 18011 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:30.250954 18011 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:30.244954 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:30.253085 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:30.249501 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:30.245394 16402 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 1.
I20260709 12:22:30.227483 17875 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:30.258813 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:30.221807 17877 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:30.266074 18002 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:30.266913 17258 raft_consensus.cc:1240] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:22:30.267361 17556 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:30.269062 17911 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:30.237628 18013 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:30.241763 17254 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:30.279722 17143 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:30.262893 17925 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:30.264081 18009 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:30.275688 17893 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:30.293634 16543 raft_consensus.cc:1240] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:30.263795 17688 consensus_queue.cc:1059] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 12:22:30.300843 17972 raft_consensus.cc:993] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2: : Instructing follower ef57beab0e1d40d585983873c55107e8 to start an election
I20260709 12:22:30.301420 17670 raft_consensus.cc:1081] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Signalling peer ef57beab0e1d40d585983873c55107e8 to start an election
W20260709 12:22:30.302130 18021 log.cc:927] Time spent T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 12:22:30.301756 18001 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:30.304690 18001 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:30.304869 17631 raft_consensus.cc:3055] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:22:30.309571 18015 log.cc:927] Time spent T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:22:30.309410 17631 raft_consensus.cc:740] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:30.295710 17925 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:30.333762 16549 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:30.342432 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:30.296968 18009 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:30.391894 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 59136.5us
I20260709 12:22:30.392740 17144 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:30.393151 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 113538us
I20260709 12:22:30.344012 16402 raft_consensus.cc:2393] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 2: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:30.405052 16441 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:30.409571 17969 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:30.410151 18012 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:30.410432 17670 consensus_queue.cc:1059] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:30.410719 18012 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:30.411520 17972 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:30.412046 17972 raft_consensus.cc:697] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:30.412798 16550 raft_consensus.cc:1240] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:22:30.412945 17972 consensus_queue.cc:237] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:30.414381 17925 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:22:30.415539 18026 log.cc:927] Time spent T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 12:22:30.411131 17670 consensus_queue.cc:1243] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer ef57beab0e1d40d585983873c55107e8 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:22:30.418089 17263 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8"
dest_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:45224
I20260709 12:22:30.422029 17263 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:30.422466 17263 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:30.404651 17913 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:30.440299 18016 raft_consensus.cc:2804] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:30.440824 16402 raft_consensus.cc:1275] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:30.440956 18016 raft_consensus.cc:697] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:30.412196 17969 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:30.445102 18027 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:22:30.446135 18029 log.cc:927] Time spent T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:22:30.446909 17556 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:30.468971 18012 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:30.469275 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:30.470398 18011 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:30.470886 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:30.471367 18002 consensus_queue.cc:1048] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:30.472429 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:30.434116 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:30.473732 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:30.466372 17954 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:30.486268 17954 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:30.447686 17875 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:30.485636 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:30.407106 17631 consensus_queue.cc:260] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:30.463485 18027 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:30.492586 17263 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:30.499311 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:30.442015 18016 consensus_queue.cc:237] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:30.493453 18011 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:30.506714 18011 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:30.509112 18011 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:30.496851 18027 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:30.498411 16470 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:30.514909 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:30.514570 16545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 734) took 1141 ms. Trace:
I20260709 12:22:30.517943 16545 rpcz_store.cc:276] 0709 12:22:29.373263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.373402 (+   139us) service_pool.cc:224] Handling call
0709 12:22:29.409497 (+ 36095us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:29.557626 (+148129us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:29.557677 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:29.557744 (+    67us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:29.557789 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:30.514255 (+956466us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b5400729554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:30.514325 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:30.514492 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":340117760}
I20260709 12:22:30.501571 17631 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:30.529422 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:30.532539 17875 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:30.533443 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:30.523533 17263 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:30.547585 16549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37242 (request call id 728) took 1180 ms. Trace:
I20260709 12:22:30.547973 16549 rpcz_store.cc:276] 0709 12:22:29.366747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.366939 (+   192us) service_pool.cc:224] Handling call
0709 12:22:30.547381 (+1180442us) spinlock_profiling.cc:243] Waited 956 ms on lock 0x7b54004d7454. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:30.547481 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2198262528}
I20260709 12:22:30.540284 16395 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:30.540313 17875 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:30.554289 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:30.554975 17954 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:30.555840 17875 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:30.555764 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:22:30.556502 18010 log.cc:927] Time spent T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.510s	user 0.003s	sys 0.001s
I20260709 12:22:30.540520 16470 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:22:30.564008 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:30.564728 16470 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
I20260709 12:22:30.564862 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:30.565186 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:30.565665 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:22:30.566001 16546 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37242 (request call id 729) took 1198 ms (client timeout 1524 ms). Trace:
I20260709 12:22:30.566277 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:22:30.566491 16546 rpcz_store.cc:269] 0709 12:22:29.367167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.377972 (+ 10805us) service_pool.cc:224] Handling call
0709 12:22:30.565029 (+1187057us) spinlock_profiling.cc:243] Waited 4.66 ms on lock 0x7b58005e9298. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:22:30.565940 (+   911us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10715392}
I20260709 12:22:30.567598 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:30.566431 16470 raft_consensus.cc:3055] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:30.572134 16473 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:30.573776 16442 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:30.575011 18012 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:30.575853 17979 raft_consensus.cc:3055] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:30.576252 17979 raft_consensus.cc:740] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:30.577268 17875 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:30.577189 17979 consensus_queue.cc:260] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:30.579285 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:30.581151 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:30.581552 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:30.582340 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:30.582823 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:30.584168 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:30.584712 17868 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:30.584812 16403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 678) took 1193 ms. Trace:
I20260709 12:22:30.585273 17868 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:30.585346 16403 rpcz_store.cc:276] 0709 12:22:29.391820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.391966 (+   146us) service_pool.cc:224] Handling call
0709 12:22:30.584592 (+1192626us) spinlock_profiling.cc:243] Waited 943 ms on lock 0x7b54001487d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:30.584728 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2169037184}
I20260709 12:22:30.577848 17875 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:30.587342 17868 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:30.589908 17904 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:30.592303 16474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 733) took 1070 ms. Trace:
I20260709 12:22:30.592528 17904 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:30.592744 16474 rpcz_store.cc:276] 0709 12:22:29.521783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.556772 (+ 34989us) service_pool.cc:224] Handling call
0709 12:22:29.557113 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:29.571305 (+ 14192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:29.571357 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:29.597261 (+ 25904us) log.cc:844] Serialized 26 byte log entry
0709 12:22:29.939252 (+341991us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:29.939299 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:29.939367 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:30.555905 (+616538us) spinlock_profiling.cc:243] Waited 50.6 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:22:30.591625 (+ 35720us) raft_consensus.cc:1685] finished
0709 12:22:30.591647 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:30.592226 (+   579us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7015,"raft.queue_time_us":11327,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":13102,"spinlock_wait_cycles":116344704,"thread_start_us":118437,"threads_started":1,"wal-append.queue_time_us":305413}
I20260709 12:22:30.594704 17904 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:30.594905 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:30.595547 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:30.596155 17266 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 3.
I20260709 12:22:30.595579 16474 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
I20260709 12:22:30.570684 16470 raft_consensus.cc:740] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:30.597729 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:30.589912 17874 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:30.600350 17915 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:22:30.601775 18018 log.cc:927] Time spent T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.355s	user 0.000s	sys 0.001s
I20260709 12:22:30.603022 18025 consensus_queue.cc:1059] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:30.603471 17904 consensus_queue.cc:1059] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:30.604008 18025 raft_consensus.cc:3055] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:30.604398 18025 raft_consensus.cc:740] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:30.604933 17904 raft_consensus.cc:3055] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:30.605677 17904 raft_consensus.cc:740] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:30.605376 18025 consensus_queue.cc:260] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:30.607020 18025 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:30.606948 17904 consensus_queue.cc:260] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:30.609946 16367 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:30.616891 17266 raft_consensus.cc:1240] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:30.617187 16366 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:30.599009 17874 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:30.624221 17976 raft_consensus.cc:3055] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:30.589521 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:30.624840 17976 raft_consensus.cc:740] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:30.624902 17790 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:30.625162 17874 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:30.625752 17790 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:30.626161 17790 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:30.596715 17868 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:30.627146 18009 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:30.627833 17904 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:30.629137 16474 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a60dbe0067044806b384e90805c877bc"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:39000
I20260709 12:22:30.625857 17976 consensus_queue.cc:260] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:30.626946 17868 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:30.630080 16474 raft_consensus.cc:606] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:22:30.626564 17790 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:30.633154 16474 raft_consensus.cc:614] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:30.634742  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:22:30.635844 16477 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
W20260709 12:22:30.638744 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:30.643005 16366 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:30.645138 16366 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:30.625232 16473 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:30.636953 17829 consensus_queue.cc:1059] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
W20260709 12:22:30.649446 18041 log.cc:927] Time spent T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:22:30.652792 16470 consensus_queue.cc:260] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:30.653009 18046 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:30.650583 17954 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:30.658206 17953 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:30.653898 17994 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:30.670495 17875 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:30.662135 17954 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:30.671227 17954 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:30.632360 18027 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:30.662035 18046 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:30.674605 18046 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:30.675257 17966 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:30.664273 16470 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:30.670730 17994 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:30.680066 17868 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:30.685956 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:30.690230 16395 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
W20260709 12:22:30.691243 18042 log.cc:927] Time spent T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:22:30.692286 17144 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:30.656270 16366 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:30.693398 17661 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:30.694026 17661 raft_consensus.cc:697] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:30.695147 17661 consensus_queue.cc:237] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:30.705912 16550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 157) took 1293 ms. Trace:
I20260709 12:22:30.701285 16470 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:30.706687 16550 rpcz_store.cc:276] 0709 12:22:29.412693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.711745 (+299052us) service_pool.cc:224] Handling call
0709 12:22:29.712113 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:30.413337 (+701224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:30.705607 (+292270us) spinlock_profiling.cc:243] Waited 700 ms on lock 0x7b54005eea54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:30.705837 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1610664832}
I20260709 12:22:30.708432 16470 raft_consensus.cc:1275] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:30.710634 18027 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:22:30.721138 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 65601.4us
W20260709 12:22:30.725981 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:30.745486 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:30.729825 16366 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:30.751001 16402 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:22:30.733407 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:30.754531 17263 raft_consensus.cc:1240] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:30.735870 17966 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:30.748894 17799 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:30.730572 18025 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:30.780031 17966 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:30.780614 16402 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
W20260709 12:22:30.754603 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 144717us
W20260709 12:22:30.784184 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 54315.4us
I20260709 12:22:30.734799 16470 raft_consensus.cc:1240] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:30.790094 17144 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:30.784102 17977 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.009s	sys 0.007s
I20260709 12:22:30.813230 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:30.790957 17269 raft_consensus.cc:1275] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:30.821527 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:30.792639 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:30.795074 16474 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:30.790359 17868 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:30.786302 18025 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:30.801828 17994 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:30.803205 18004 raft_consensus.cc:3055] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:30.855012 18004 raft_consensus.cc:740] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:30.855997 18004 consensus_queue.cc:260] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:30.857321 18004 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:30.807472 18046 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:30.803718 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 793632us
I20260709 12:22:30.859395 18046 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:30.785662 17799 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:30.814035 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:30.730765 17954 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:30.815543 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:30.810849 17861 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:30.826930 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:30.827172 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:30.828178 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:30.828835 17664 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:30.832641 17725 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:30.833726 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:22:30.834949 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 44931.1us
I20260709 12:22:30.831997 17790 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:30.852573 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:30.883500 16441 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:30.847016 18025 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:30.831946 17977 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.009s	sys 0.007s
I20260709 12:22:30.857211 16476 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:30.810595 17862 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:30.812070 16516 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:30.875918 16549 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
W20260709 12:22:30.877708 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:30.898825 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:30.829433 18001 consensus_queue.cc:1048] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:30.853811 17966 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:30.888612 16403 raft_consensus.cc:1240] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:30.861891 18020 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:30.832370 17868 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:30.890336 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:30.896636 16366 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:30.873140 17953 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:30.862147 17904 consensus_queue.cc:1048] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 12:22:30.912629 16474 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:30.893275 17829 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:30.881002 17631 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:30.888772 17254 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:30.920167 16477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34826 (request call id 657) took 1686 ms. Trace:
I20260709 12:22:30.873754 17861 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:30.920174 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 93940.2us
I20260709 12:22:30.911058 17665 consensus_queue.cc:1059] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 12:22:30.893278 18046 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:22:30.928753 17873 consensus_queue.cc:1059] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:30.893931 17725 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:30.929939 17873 raft_consensus.cc:3055] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:30.930336 17873 raft_consensus.cc:740] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:30.930390 18005 raft_consensus.cc:3055] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:30.930925 18005 raft_consensus.cc:740] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:30.931187 17725 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:30.905774 17862 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:30.931298 17873 consensus_queue.cc:260] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:30.921500 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:30.931885 18005 consensus_queue.cc:260] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:30.933527 18005 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:30.895321 18047 consensus_queue.cc:1059] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 12:22:30.934063 17862 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:30.934641 18047 raft_consensus.cc:3055] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:30.935133 16543 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 1.
I20260709 12:22:30.890550 17664 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:30.935961 18020 raft_consensus.cc:697] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:30.937024 18020 consensus_queue.cc:237] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:30.937907 17664 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:30.920362 17992 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:30.940713 17992 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:30.941125 17992 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:30.905035 18009 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:30.920902 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:30.943683 16474 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
W20260709 12:22:30.943961 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:30.944960 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:30.923799 16477 rpcz_store.cc:276] 0709 12:22:29.234104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.234392 (+   288us) service_pool.cc:224] Handling call
0709 12:22:30.499202 (+1264810us) spinlock_profiling.cc:243] Waited 2.93 ms on lock 0x7b5800602a98. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64eee 000000000036eb39 00007f287add81e0 00007f287aea10c4 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:22:30.920002 (+420800us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b540080acd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:30.920110 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":319361408}
I20260709 12:22:30.946684 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:30.948467 16366 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:30.915138 17875 raft_consensus.cc:2749] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:30.914412 17868 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:30.950584 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:30.952579 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:30.935108 18047 raft_consensus.cc:740] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:30.956161 17829 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:30.957463 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:30.959331 16475 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:30.919859 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:30.965072 16474 raft_consensus.cc:2393] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 2: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:30.952122 18046 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:30.982940 18046 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:30.970314 17994 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:30.970512 17790 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:30.971822 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:30.926491 17631 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:30.972914 17861 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:30.973309 17799 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:30.921407 17953 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:31.004573 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:30.958113 18009 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:30.985884 17873 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:30.981379 18022 log.cc:927] Time spent T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.730s	user 0.006s	sys 0.000s
I20260709 12:22:30.981700 18047 consensus_queue.cc:260] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:30.985375 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 58256.4us
I20260709 12:22:31.017041 17954 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:30.983505 18046 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:30.975929 17992 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:30.979919 17829 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:31.008824 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:30.985028 17966 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:22:31.025511 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:31.027215 17966 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:31.014191 17631 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:31.020574 16475 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:31.003731 17877 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:31.037647 16366 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:31.044500 16547 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37242 (request call id 730) took 1673 ms (client timeout 1824 ms). Trace:
W20260709 12:22:31.046401 16472 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34840 (request call id 735) took 1524 ms (client timeout 1700 ms). Trace:
I20260709 12:22:31.049567 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:31.050493 16468 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:31.051393 17263 raft_consensus.cc:1275] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:31.053310 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:31.054747 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:31.057821 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:31.058794 17142 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:31.062675 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:22:31.045176 16547 rpcz_store.cc:269] 0709 12:22:29.371467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.371661 (+   194us) service_pool.cc:224] Handling call
0709 12:22:31.044144 (+1672483us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b5400aa69d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:31.044317 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":402083200}
I20260709 12:22:31.070679 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:31.072549 16400 raft_consensus.cc:1240] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:31.063721 17142 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:31.074656 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 150843us
I20260709 12:22:31.075933 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:31.077334 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:31.077431 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:31.078588 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:31.029973 17966 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:31.047001 16472 rpcz_store.cc:269] 0709 12:22:29.522382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.541967 (+ 19585us) service_pool.cc:224] Handling call
0709 12:22:31.046157 (+1504190us) spinlock_profiling.cc:243] Waited 481 ms on lock 0x7b540058c854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:31.046307 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1106521856}
I20260709 12:22:31.081405 17142 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:31.083278 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:31.085544 16398 raft_consensus.cc:1240] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:31.085809 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:31.086112 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:31.087281 16477 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:31.098708 16468 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 3.
I20260709 12:22:31.101078 16366 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:31.103956 16475 raft_consensus.cc:2393] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 5: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:31.105948 16515 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:31.056295 17954 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:31.260515 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:22:31.261008 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 155152us
I20260709 12:22:31.264408 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:31.265118 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 453100us
I20260709 12:22:31.265558 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:31.069301 18027 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:31.278043 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 196794us
I20260709 12:22:31.279080 17142 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:31.284080 17142 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:31.270208 16468 raft_consensus.cc:1275] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:31.078892 17844 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:31.266243 16516 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:31.348750 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:22:31.349294 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 83166.5us
I20260709 12:22:31.350509 16516 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:31.353502 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:31.355023 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:31.355594 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:31.361584 16547 raft_consensus.cc:1240] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:31.365854 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:31.358183 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:31.374795 16398 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:31.064738 18025 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:31.055611 17877 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:31.264470 17942 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:31.488202 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:31.496975 17258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45224 (request call id 563) took 1078 ms. Trace:
W20260709 12:22:31.490180 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 206218us
I20260709 12:22:31.498893 17142 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:31.274739 17861 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:31.068670 17977 consensus_queue.cc:1048] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.289s
W20260709 12:22:31.347431 18058 log.cc:927] Time spent T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.325s	user 0.000s	sys 0.001s
I20260709 12:22:31.501807 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:31.270610 18019 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:31.520108 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:31.521898 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:31.530813 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 52745.4us
I20260709 12:22:31.504159 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:31.532805 17266 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:31.424623 17696 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:31.518639 18057 log.cc:927] Time spent T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.442s	user 0.000s	sys 0.000s
I20260709 12:22:31.519155 17977 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:31.519673 17953 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:31.480192 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:31.497526 17258 rpcz_store.cc:276] 0709 12:22:30.418172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:30.418435 (+   263us) service_pool.cc:224] Handling call
0709 12:22:31.487973 (+1069538us) spinlock_profiling.cc:243] Waited 1 ms on lock 0x7b58003b8298. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:22:31.496884 (+  8911us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2303360}
I20260709 12:22:31.509967 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:31.424505 18004 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:31.480926 17872 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:31.549516 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 50737.2us
I20260709 12:22:31.507947 17861 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:31.483268 18037 log.cc:927] Time spent T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
W20260709 12:22:31.591218 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 239149us
I20260709 12:22:31.566466 17142 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:31.503585 17806 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:31.563041 16400 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:31.578963 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:31.580310 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:31.595011 17142 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:31.349843 18027 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:31.564208 18002 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:31.593791 17861 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:31.614989 17142 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:31.653126 17266 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:31.594657 17953 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:31.611727 17844 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:31.610707 17900 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:31.595527 17872 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:31.561826 17977 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:31.674801 16366 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.020s	sys 0.056s
W20260709 12:22:31.685434 16366 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.020s	sys 0.060s
I20260709 12:22:31.658555 18002 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:31.667734 17829 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:31.685926 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 584924us
I20260709 12:22:31.686058 18002 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:31.666268 18014 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:31.687292 17829 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:31.685779 17992 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:22:31.693739 16550 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.029s	sys 0.040s
W20260709 12:22:31.695103 16550 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.030s	sys 0.040s
I20260709 12:22:31.667215 18019 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:31.710299 18014 consensus_queue.cc:1048] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:31.687831 17829 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:31.710592 18019 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:31.693723 18005 consensus_meta.cc:287] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.760s	user 0.010s	sys 0.000s
I20260709 12:22:31.715251 18071 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:22:31.716276 17868 consensus_meta.cc:287] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.743s	user 0.011s	sys 0.000s
I20260709 12:22:31.698137 16473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 742) took 1391 ms. Trace:
I20260709 12:22:31.720935 18070 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:31.721586 17877 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:31.694187 18009 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:31.726132 17829 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:31.681144 17900 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:22:31.728557 16477 consensus_meta.cc:287] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.636s	user 0.005s	sys 0.004s
I20260709 12:22:31.759694 16477 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:31.681221 17953 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:31.761168 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:31.726145 17875 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:22:31.766520 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:31.770061 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:31.762320 17900 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:31.762588 18071 raft_consensus.cc:697] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 3 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:31.775254 18072 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:31.783666 17988 log.cc:927] Time spent T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 2.224s	user 0.002s	sys 0.012s
I20260709 12:22:31.780236 18025 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:22:31.792932 18046 consensus_meta.cc:287] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.763s	user 0.011s	sys 0.000s
I20260709 12:22:31.774099 17875 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:31.803385 18069 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:31.828105 16442 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:31.732689 17868 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:31.765725 18070 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:31.830128 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:31.811542 18046 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:31.826083 18027 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:31.862777 16442 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:31.804240 18072 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:31.829202 18005 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:31.844381 17954 raft_consensus.cc:697] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 5 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:31.875396 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:31.843868 18071 consensus_queue.cc:237] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:31.730189 16473 rpcz_store.cc:276] 0709 12:22:30.307309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:30.658098 (+350789us) service_pool.cc:224] Handling call
0709 12:22:30.658532 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:30.695842 (+ 37310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:30.695880 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:30.696412 (+   532us) log.cc:844] Serialized 26 byte log entry
0709 12:22:30.921686 (+225274us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:30.924945 (+  3259us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:30.941920 (+ 16975us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:31.673973 (+732053us) raft_consensus.cc:1685] finished
0709 12:22:31.675091 (+  1118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:31.694322 (+ 19231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7271,"raft.queue_time_us":1303,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"thread_start_us":214291,"threads_started":1,"wal-append.queue_time_us":322725}
I20260709 12:22:31.861436 17829 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:31.898335 16442 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:31.891649 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:31.897349 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:31.893648 16476 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53186 (request call id 168) took 1959 ms (client timeout 1730 ms). Trace:
I20260709 12:22:31.896647 17264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 565) took 1108 ms. Trace:
I20260709 12:22:31.863595 18061 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:31.917768 16544 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
W20260709 12:22:31.920107 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:31.918164 16476 rpcz_store.cc:269] 0709 12:22:29.934263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.968144 (+ 33881us) service_pool.cc:224] Handling call
0709 12:22:31.893487 (+1925343us) spinlock_profiling.cc:243] Waited 843 ms on lock 0x7b5400b2fcd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:31.893588 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1939601152}
I20260709 12:22:31.869359 17872 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:31.920047 16442 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:31.925629 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:31.899319 16401 raft_consensus.cc:1240] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 4. Current term is 5. Ops: []
W20260709 12:22:31.926714 16475 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.004s	sys 0.032s
I20260709 12:22:31.918366 17264 rpcz_store.cc:276] 0709 12:22:30.788542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:30.788707 (+   165us) service_pool.cc:224] Handling call
0709 12:22:30.789174 (+   467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:30.862470 (+ 73296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:30.862514 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:30.862573 (+    59us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:30.862617 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:31.896357 (+1033740us) spinlock_profiling.cc:243] Waited 73.1 ms on lock 0x7b54005f3a54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:31.896428 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:31.896579 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":168016768}
I20260709 12:22:31.886457 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:31.887228 16396 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46250 (request call id 605) took 1860 ms (client timeout 1667 ms). Trace:
I20260709 12:22:31.878073 18060 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:31.870077 17992 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:31.890865 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:31.877818 18066 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:31.892498 18019 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:22:31.934401 16475 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.004s	sys 0.032s
I20260709 12:22:31.887414 18000 raft_consensus.cc:2804] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:31.892149 17868 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:31.931452 16476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 676) took 1007 ms. Trace:
I20260709 12:22:31.828590 18069 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:31.888187 18072 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:31.924822 17829 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:31.899766 17977 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:31.941833 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:32.011224 16544 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
I20260709 12:22:32.012385 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:32.012508 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:32.013269 16546 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:32.014098 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:32.014317 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:32.015578 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:31.923919 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 339183us
I20260709 12:22:31.938397 16442 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:31.950230 18002 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:31.965517 17979 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:32.016798 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 78526.8us
I20260709 12:22:32.018596 18002 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:31.877835 18005 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:32.019250 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:32.019893 16546 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:32.020637 18005 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:31.929396 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:32.022388 16399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 221) took 1797 ms. Trace:
I20260709 12:22:32.022526 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:32.023151 16542 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:32.022768 16399 rpcz_store.cc:276] 0709 12:22:30.224725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:30.229469 (+  4744us) service_pool.cc:224] Handling call
0709 12:22:30.254062 (+ 24593us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:30.254495 (+   433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:30.254535 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:30.265267 (+ 10732us) log.cc:844] Serialized 26 byte log entry
0709 12:22:31.573484 (+1308217us) spinlock_profiling.cc:243] Waited 8.3 ms on lock 0x7b1800110da0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:22:31.830526 (+257042us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:22:31.830567 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:31.830638 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:32.022099 (+191461us) raft_consensus.cc:1685] finished
0709 12:22:32.022117 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:32.022336 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6101,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":1951,"spinlock_wait_cycles":19097984,"thread_start_us":256862,"threads_started":1,"wal-append.queue_time_us":1735164}
W20260709 12:22:31.940407 16396 rpcz_store.cc:269] 0709 12:22:30.026515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:30.029932 (+  3417us) service_pool.cc:224] Handling call
0709 12:22:31.824661 (+1794729us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b5800125198. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:22:31.887155 (+ 62494us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30357248}
I20260709 12:22:32.021947 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:31.990936 17992 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:32.026110 18069 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:32.026206 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:22:32.027307 18074 log.cc:927] Time spent T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:22:32.027537 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:31.992075 18066 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:32.028599 17696 raft_consensus.cc:2749] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:32.016763 17875 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:32.000012 16473 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:32.030285 18066 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:31.929689 18061 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:31.967746 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:32.001314 18000 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:32.016705 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:31.944950 18073 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:32.000458 16476 rpcz_store.cc:276] 0709 12:22:30.924376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:31.930502 (+1006126us) service_pool.cc:224] Handling call
0709 12:22:31.930833 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:31.931024 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:31.931056 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:31.931102 (+    46us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:31.931142 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:31.931203 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:31.931327 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:32.009217 17873 consensus_meta.cc:287] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.999s	user 0.012s	sys 0.000s
I20260709 12:22:32.032618 16472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 754) took 1229 ms. Trace:
I20260709 12:22:32.033308 16550 raft_consensus.cc:2410] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:22:31.937742 17953 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:32.032477 16514 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:22:31.984359 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 378723us
I20260709 12:22:32.045130 16398 raft_consensus.cc:1240] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:32.027522 17992 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:32.040323 18084 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:32.050482 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:32.058688 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:31.962458 17954 consensus_queue.cc:237] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:31.977118 18060 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:32.040542 18000 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:32.068277 16514 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:32.057964 16547 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
W20260709 12:22:32.072377 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 177334us
I20260709 12:22:32.074623 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:32.059283 16472 rpcz_store.cc:276] 0709 12:22:30.803188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:31.091693 (+288505us) service_pool.cc:224] Handling call
0709 12:22:31.092021 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:31.092268 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:22:31.731313 (+639045us) log.cc:844] Serialized 14 byte log entry
0709 12:22:32.032279 (+300966us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:32.032350 (+    71us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:32.032393 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:32.032455 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:32.032567 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":835578,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"raft.queue_time_us":6403,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":9005,"thread_start_us":668453,"threads_started":2}
I20260709 12:22:32.105815 16398 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:22:32.106392 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:32.107429 16472 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:22:32.031929 17900 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:32.108371 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:32.104390 16367 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:32.109225 16400 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:32.110908 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:32.141456 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:32.072140 17665 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:32.077459 17874 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:32.154708 16548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 936) took 2221 ms. Trace:
I20260709 12:22:32.162119 16548 rpcz_store.cc:276] 0709 12:22:29.892134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.910676 (+ 18542us) service_pool.cc:224] Handling call
0709 12:22:31.051435 (+1140759us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:31.051686 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:31.051728 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:31.051791 (+    63us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:31.051841 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:31.051920 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:32.113338 (+1061418us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400aa69d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:32.113509 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2622597376}
I20260709 12:22:32.140377 17864 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:32.090668 17799 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:22:32.158658 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 54437us
W20260709 12:22:32.153015 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 84791.6us
I20260709 12:22:32.160630 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:32.110401 18046 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:32.169486 17271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 458) took 2154 ms. Trace:
I20260709 12:22:32.111212 16471 raft_consensus.cc:1275] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:32.178318 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:32.164062 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:32.144186 18087 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:32.172999 16472 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:32.169373 16396 raft_consensus.cc:2393] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 4: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:32.203754 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:32.207769 16398 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:32.094336 18083 consensus_queue.cc:1059] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.106s
I20260709 12:22:32.142887 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:32.172081 16367 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:32.172628 16514 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
W20260709 12:22:32.197484 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 130881us
I20260709 12:22:32.203732 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:32.206516 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:32.076926 18084 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:32.206703 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:32.210099 16545 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:32.201880 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:32.168430 17946 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:32.179647 18073 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:32.170387 18069 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:32.219211 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:32.180148 18070 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:32.212038 16367 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:32.218750 16514 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:32.213049 16477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 675) took 1291 ms. Trace:
I20260709 12:22:32.184677 17875 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:32.230894 16396 raft_consensus.cc:1240] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 5. Current term is 6. Ops: []
W20260709 12:22:32.205614 18076 log.cc:927] Time spent T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 12:22:32.238708 17900 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:22:32.212882 17861 consensus_meta.cc:287] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.560s	user 0.012s	sys 0.000s
I20260709 12:22:32.220949 17829 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:32.240664 16475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 665) took 1378 ms. Trace:
I20260709 12:22:32.241058 17829 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:32.241119 16475 rpcz_store.cc:276] 0709 12:22:30.861786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:31.108793 (+247007us) service_pool.cc:224] Handling call
0709 12:22:31.109134 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:31.115935 (+  6801us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:31.115979 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:31.260751 (+144772us) log.cc:844] Serialized 26 byte log entry
0709 12:22:31.989210 (+728459us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:31.989267 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:31.989375 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:32.240437 (+251062us) raft_consensus.cc:1685] finished
0709 12:22:32.240458 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:32.240604 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":536,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":13756,"thread_start_us":563877,"threads_started":1,"wal-append.queue_time_us":778031}
I20260709 12:22:32.243853 16475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 679) took 1296 ms. Trace:
I20260709 12:22:32.220229 17696 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:32.214040 16471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 162) took 2274 ms. Trace:
I20260709 12:22:32.247303 17696 raft_consensus.cc:697] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:32.247605 16471 rpcz_store.cc:276] 0709 12:22:29.932444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.943776 (+ 11332us) service_pool.cc:224] Handling call
0709 12:22:32.039918 (+2096142us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b5800602a98. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:22:32.061770 (+ 21852us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:32.180001 (+118231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:32.206725 (+ 26724us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4460800}
I20260709 12:22:32.248471 16545 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:22:32.249148 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:32.210958 17142 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:32.199590 18046 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:32.255554 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 43601.8us
I20260709 12:22:32.256415 16367 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:32.168563 17874 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:32.261726 16400 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:32.263147 17874 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:32.201853 16403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 244) took 1542 ms. Trace:
I20260709 12:22:32.263851 16403 rpcz_store.cc:276] 0709 12:22:30.658906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:30.659031 (+   125us) service_pool.cc:224] Handling call
0709 12:22:30.659309 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:30.910279 (+250970us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:32.201426 (+1291147us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b54000d2d54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:32.201699 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":526460160}
I20260709 12:22:32.264432 18046 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:32.266681 18046 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:32.267532 18080 raft_consensus.cc:2749] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:32.170147 17864 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:32.228689 18089 log.cc:927] Time spent T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 12:22:32.229607 17969 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:32.269814 17969 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:32.270223 17969 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:32.270458 17864 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:32.271384 18000 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:32.240012 17861 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:32.274068 17861 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:32.274817 16544 raft_consensus.cc:1240] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 5. Current term is 6. Ops: []
W20260709 12:22:32.277541 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:32.244156 16475 rpcz_store.cc:276] 0709 12:22:30.946997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:32.243085 (+1296088us) service_pool.cc:224] Handling call
0709 12:22:32.243387 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:32.243563 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:32.243588 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:32.243626 (+    38us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:22:32.243654 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:32.243698 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:32.243805 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:32.220127 16547 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
I20260709 12:22:32.222692 16477 rpcz_store.cc:276] 0709 12:22:30.921801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:31.772940 (+851139us) service_pool.cc:224] Handling call
0709 12:22:31.785321 (+ 12381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:31.785581 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:22:32.056910 (+271329us) log.cc:844] Serialized 14 byte log entry
0709 12:22:32.208576 (+151666us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:32.208647 (+    71us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:32.208698 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:32.209556 (+   858us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:32.212998 (+  3442us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":182201,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":60,"raft.queue_time_us":718,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":6655,"thread_start_us":403131,"threads_started":2,"wal-append.queue_time_us":191223}
I20260709 12:22:32.248265 17696 consensus_queue.cc:237] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:32.260601 16473 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:32.257977 17900 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:32.290014 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:32.290664 16403 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:32.235725 16546 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:32.292346 16474 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:32.288719 17900 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:32.293949 17900 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:32.294462 16543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52680 (request call id 164) took 1482 ms. Trace:
I20260709 12:22:32.294840 16543 rpcz_store.cc:276] 0709 12:22:30.811776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:30.813710 (+  1934us) service_pool.cc:224] Handling call
0709 12:22:32.294293 (+1480583us) spinlock_profiling.cc:243] Waited 74.6 ms on lock 0x7b5400a4f454. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:32.294402 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":171612928}
I20260709 12:22:32.297677 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:32.287937 16400 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:32.252296 16396 raft_consensus.cc:1275] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:32.320246 16545 raft_consensus.cc:1275] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:32.320497 16474 raft_consensus.cc:1240] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:32.321313 16472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 677) took 1377 ms. Trace:
I20260709 12:22:32.322465 16474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 681) took 1363 ms. Trace:
I20260709 12:22:32.322929 16474 rpcz_store.cc:276] 0709 12:22:30.959065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:32.319959 (+1360894us) service_pool.cc:224] Handling call
0709 12:22:32.320248 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:32.322233 (+  1985us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:32.322419 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:32.322432 16472 rpcz_store.cc:276] 0709 12:22:30.943529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:32.210312 (+1266783us) service_pool.cc:224] Handling call
0709 12:22:32.210608 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:32.210805 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:22:32.256777 (+ 45972us) log.cc:844] Serialized 14 byte log entry
0709 12:22:32.319546 (+ 62769us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:32.319595 (+    49us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:32.319639 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:32.319725 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:32.321247 (+  1522us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":41788,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":92,"thread_start_us":62365,"threads_started":1}
I20260709 12:22:32.321945 17142 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:32.209965 18087 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:32.328164 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 71867.5us
I20260709 12:22:32.327071 16514 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:32.329658 16367 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:32.331193 16367 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:32.285512 17899 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:32.338748 17899 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:32.339187 18084 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:32.342079 18087 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:32.343374 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:32.341502 17899 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:32.346729 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:32.300101 18083 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:32.348615 18083 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:32.350474 18060 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:32.350841 18083 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:32.351430 18077 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:32.352294 18077 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:32.291754 17829 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:32.348752 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:32.351727 18061 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:32.299921 17992 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:32.353818 16550 raft_consensus.cc:2393] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 3: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:32.352905 18077 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:32.300303 17871 consensus_queue.cc:1048] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 12:22:32.336817 17868 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:32.355118 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:32.355940 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:32.357086 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:32.358019 17255 raft_consensus.cc:1240] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 2. Current term is 4. Ops: []
I20260709 12:22:32.358466 16547 raft_consensus.cc:1275] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:32.358692 17255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 462) took 2345 ms. Trace:
I20260709 12:22:32.337762 16469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 666) took 1475 ms. Trace:
I20260709 12:22:32.359076 17255 rpcz_store.cc:276] 0709 12:22:30.013318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:30.017122 (+  3804us) service_pool.cc:224] Handling call
0709 12:22:32.357679 (+2340557us) spinlock_profiling.cc:243] Waited 4.07 ms on lock 0x7b58003b8298. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64eee 000000000036eb39 00007f287add81e0 00007f287aea10c4 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:22:32.357828 (+   149us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:32.358444 (+   616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:32.358638 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9363840}
I20260709 12:22:32.359568 16469 rpcz_store.cc:276] 0709 12:22:30.862049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:31.352847 (+490798us) service_pool.cc:224] Handling call
0709 12:22:31.491149 (+138302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:31.837712 (+346563us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:31.837758 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:31.844380 (+  6622us) log.cc:844] Serialized 26 byte log entry
0709 12:22:32.168174 (+323794us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:32.168233 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:32.168358 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:32.337564 (+169206us) raft_consensus.cc:1685] finished
0709 12:22:32.337589 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:32.337710 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9095,"raft.queue_time_us":164831,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":9856,"thread_start_us":607233,"threads_started":2,"wal-append.queue_time_us":343427}
I20260709 12:22:32.361616 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:32.219762 17946 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:32.362205 16551 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:32.364586 16551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37242 (request call id 744) took 1101 ms. Trace:
I20260709 12:22:32.364979 16551 rpcz_store.cc:276] 0709 12:22:31.263622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:31.477670 (+214048us) service_pool.cc:224] Handling call
0709 12:22:32.361413 (+883743us) spinlock_profiling.cc:243] Waited 2.45 ms on lock 0x7b58005e9298. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:22:32.364512 (+  3099us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5661952}
I20260709 12:22:32.361685 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:32.335438 16516 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:32.370007 18066 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:32.370343 18000 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:32.370599 18066 raft_consensus.cc:697] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:32.372444 17877 raft_consensus.cc:3055] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:32.372543 18081 consensus_queue.cc:1059] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:32.372997 17877 raft_consensus.cc:740] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:32.373670 18081 raft_consensus.cc:3055] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:32.352083 16395 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:32.374003 17877 consensus_queue.cc:260] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:32.374696 18061 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:32.378504 18001 consensus_queue.cc:1059] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:32.378702 18061 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:32.380177 17665 consensus_queue.cc:1059] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:32.380919 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:32.381407 18061 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:32.356230 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:32.374722 17925 raft_consensus.cc:2804] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:32.374037 18081 raft_consensus.cc:740] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:32.382099 16394 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:32.383116 18081 consensus_queue.cc:260] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:32.384207 17925 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:32.384902 18081 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:32.385914 17877 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:32.376996 17794 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:32.376843 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:32.396209 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:32.334443 17650 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:32.397483 17650 raft_consensus.cc:697] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:32.398379 17650 consensus_queue.cc:237] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:32.221788 18073 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:32.377861 16516 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:32.318866 18080 consensus_queue.cc:1048] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 12:22:32.388573 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:32.389629 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:32.392174 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:32.393136 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:32.380633 18005 raft_consensus.cc:3055] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:32.382223 16548 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:32.371608 18066 consensus_queue.cc:237] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:32.394212 17900 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:32.405220 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:32.366618 17946 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:32.404369 16516 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:32.405411 16398 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:32.408687 17263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45230 (request call id 569) took 1323 ms. Trace:
I20260709 12:22:32.410712 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:32.379941 18077 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:32.416448 16403 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:32.416396 16516 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:32.384902 17925 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:32.421293 16516 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:32.325564 16542 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:32.424827 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:32.425506 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:32.426467 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:32.439419 17258 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:22:32.439997 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:32.426674 18077 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:32.423517 17864 raft_consensus.cc:2804] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:32.427476 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:32.408776 16470 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:32.405966 17556 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:32.415627 17263 rpcz_store.cc:276] 0709 12:22:31.084949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:31.261615 (+176666us) service_pool.cc:224] Handling call
0709 12:22:32.391967 (+1130352us) spinlock_profiling.cc:243] Waited 3 ms on lock 0x7b58003b8298. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:22:32.408593 (+ 16626us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6902528}
I20260709 12:22:32.384943 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:32.406816 18087 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:32.444051 17261 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
W20260709 12:22:32.445288 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:32.412106 18005 raft_consensus.cc:740] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:32.430503 18060 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:32.448661 16473 raft_consensus.cc:2393] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 2: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:32.466176 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:32.467132 17261 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:32.454953 17144 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:32.430346 17665 consensus_queue.cc:1048] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 12:22:32.450352 17899 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:32.457768 16395 raft_consensus.cc:2410] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:22:32.452777 18020 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:32.465857 17255 raft_consensus.cc:2393] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 4: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:32.456667 18060 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:32.475147 16403 raft_consensus.cc:1275] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:22:32.476440 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:32.477066 16403 pending_rounds.cc:85] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 1
I20260709 12:22:32.477514 16403 pending_rounds.cc:107] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:22:32.477804 16403 raft_consensus.cc:2889] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:32.452991 17556 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:32.470903 18019 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:22:32.517951 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 96681.8us
W20260709 12:22:32.519663 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 188569us
I20260709 12:22:32.521981 16367 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:32.480604 18060 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:32.529120 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:32.530489 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:32.531639 18047 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:22:32.534315 18063 log.cc:927] Time spent T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.868s	user 0.001s	sys 0.005s
I20260709 12:22:32.536170 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:32.540385 18077 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:32.541926 17255 raft_consensus.cc:1240] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:32.545142 16471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 758) took 1589 ms. Trace:
I20260709 12:22:32.545567 16471 rpcz_store.cc:276] 0709 12:22:30.955914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:32.248694 (+1292780us) service_pool.cc:224] Handling call
0709 12:22:32.249024 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:32.259411 (+ 10387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:32.259450 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:32.260050 (+   600us) log.cc:844] Serialized 26 byte log entry
0709 12:22:32.332050 (+ 72000us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:32.332742 (+   692us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:32.335521 (+  2779us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:32.522593 (+187072us) raft_consensus.cc:1685] finished
0709 12:22:32.524207 (+  1614us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:32.545082 (+ 20875us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":24165,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":6063,"thread_start_us":58062,"threads_started":1,"wal-append.queue_time_us":158698}
I20260709 12:22:32.547392 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:32.514314 18087 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:32.472261 17872 consensus_queue.cc:1059] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 12:22:32.516896 18081 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:32.550395 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:32.554870 16544 raft_consensus.cc:1275] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:32.548671 18087 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:32.562616 16549 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:32.558358 16367 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:32.531949 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:32.475822 17899 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:32.561322 17269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 471) took 1742 ms. Trace:
I20260709 12:22:32.524058 17696 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:32.518553 17864 raft_consensus.cc:697] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 2 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:32.524644 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:32.478960 18020 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:32.568195 16367 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:32.568650 16471 raft_consensus.cc:2393] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 4: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:32.580778 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:32.522409 17969 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:32.479759 18005 consensus_queue.cc:260] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:32.593381 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:32.557363 18081 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:32.579216 16367 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:32.588016 16441 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:32.570430 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:32.622545 16367 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:32.576296 18106 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:32.628324 16399 raft_consensus.cc:1275] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:32.628834 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:32.573237 17269 rpcz_store.cc:276] 0709 12:22:30.819382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:30.864481 (+ 45099us) service_pool.cc:224] Handling call
0709 12:22:30.865464 (+   983us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:30.916948 (+ 51484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:30.916983 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:30.917512 (+   529us) log.cc:844] Serialized 26 byte log entry
0709 12:22:31.104550 (+187038us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:22:31.104604 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:31.104696 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:32.533748 (+1429052us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:22:32.550895 (+ 17147us) raft_consensus.cc:1685] finished
0709 12:22:32.550922 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:32.561248 (+ 10326us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":84935,"raft.queue_time_us":54203,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":23552,"spinlock_wait_cycles":35495040,"thread_start_us":181284,"threads_started":1,"wal-append.queue_time_us":626313}
I20260709 12:22:32.626783 16367 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:32.630533 16441 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:32.629534 16545 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:32.586220 17790 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:32.638162 16551 raft_consensus.cc:1240] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:32.621181 16396 raft_consensus.cc:2368] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3.
I20260709 12:22:32.586710 17976 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:32.582283 17872 raft_consensus.cc:3055] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:32.633492 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:32.636488 17829 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:32.636584 18071 consensus_queue.cc:1059] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:32.637420 18084 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:32.578020 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:32.639935 16475 raft_consensus.cc:1240] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:32.641225 17790 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:32.653887 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:32.659339 16442 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:32.630174 17954 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:32.626921 17979 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:32.582381 17696 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:32.589100 16473 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:32.631853 17874 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:32.632261 18081 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:32.703008 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:32.692523 17790 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:32.694497 18071 consensus_queue.cc:1243] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer 610f4feda11942739236f2b121a4b9c1 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:22:32.694599 18005 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:32.645517 17872 raft_consensus.cc:740] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:32.641856 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:32.645195 17867 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:32.708073 18077 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:32.631982 18046 consensus_queue.cc:1048] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.066s
I20260709 12:22:32.630115 18106 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:32.690510 18073 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:32.696812 17954 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:32.716391 17868 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:32.702394 16546 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:32.718377 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:32.756062 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 96822us
I20260709 12:22:32.771634 16442 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:32.757850 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 116030us
I20260709 12:22:32.729142 18084 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:32.719045 17979 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:32.761960 18106 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:32.793710 17142 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:32.766436 17954 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:32.763623 17556 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:32.729406 17864 consensus_queue.cc:237] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:32.762645 18077 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:32.789059 17868 raft_consensus.cc:697] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 4 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:32.773976 18084 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:32.828830 18084 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:32.832890 18087 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:32.834528 18109 log.cc:927] Time spent T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
W20260709 12:22:32.841462 18114 log.cc:927] Time spent T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 12:22:32.790261 17966 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:32.844753 17966 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:32.847354 17966 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:32.794844 17872 consensus_queue.cc:260] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:32.847900 17696 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:32.790587 18116 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:32.761719 17867 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:32.848583 17872 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:32.850229 17867 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:32.852066 17979 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:32.852965 17969 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:32.801195 17946 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:32.853760 17946 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:32.854177 17946 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:32.856658 18110 log.cc:927] Time spent T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
W20260709 12:22:32.858124 18008 consensus_queue.cc:1175] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "52ab50b0237346c1a13ede947f6a63c2" responder_term: 3 status { last_received { term: 2 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:22:32.802670 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:32.859691 16473 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:32.809253 17953 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:32.864964 17953 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:32.865280 18060 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:32.866698 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:32.867400 18060 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:32.868300 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:32.868985 18077 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:32.869761 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:32.870620 16475 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:32.871194 17925 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:32.871709 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:32.867758 16548 raft_consensus.cc:1240] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:32.858842 18083 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:32.873382 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:32.873533 17868 consensus_queue.cc:237] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:32.874498 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:32.875591 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:32.876602 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:32.848608 18116 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:32.877101 16475 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:32.882511 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:32.873683 18083 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:32.898814 17925 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:32.910887 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:32.911607 16475 raft_consensus.cc:2393] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 5: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:32.847905 17966 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:32.917620 16402 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:32.867449 17953 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:32.887665 17899 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:32.175705 17271 rpcz_store.cc:276] 0709 12:22:30.011524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:30.011646 (+   122us) service_pool.cc:224] Handling call
0709 12:22:30.011913 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:30.012134 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:22:30.017396 (+  5262us) log.cc:844] Serialized 14 byte log entry
0709 12:22:30.601191 (+583795us) spinlock_profiling.cc:243] Waited 4.78 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:22:32.144118 (+1542927us) spinlock_profiling.cc:243] Waited 6.71 ms on lock 0x7b18001da520. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2875a314bc 00007f2872d7f48b 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a2923a
0709 12:22:32.165960 (+ 21842us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:32.166015 (+    55us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:32.166060 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:32.166134 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:32.166249 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21516,"prepare.queue_time_us":5674,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":4198,"raft.queue_time_us":22077,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":7219,"spinlock_wait_cycles":26440832,"thread_start_us":27693,"threads_started":1,"wal-append.queue_time_us":229313,"wal-append.run_cpu_time_us":2968,"wal-append.run_wall_time_us":1355518}
W20260709 12:22:32.932130 16515 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.004s	sys 0.010s
I20260709 12:22:32.888263 18116 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:32.891065 16394 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:32.950081 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:32.950868 16403 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
W20260709 12:22:32.938875 16515 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.005s	sys 0.010s
W20260709 12:22:32.963238 16397 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 693) took 1906 ms (client timeout 1593 ms). Trace:
W20260709 12:22:32.963754 16397 rpcz_store.cc:269] 0709 12:22:31.056841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:31.078234 (+ 21393us) service_pool.cc:224] Handling call
0709 12:22:32.963032 (+1884798us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54004ec5d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:32.963169 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3047421696}
I20260709 12:22:32.984763 17794 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:32.990970 18084 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:32.991663 18105 log.cc:927] Time spent T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.354s	user 0.001s	sys 0.001s
W20260709 12:22:33.000582 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:33.002712 16516 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:32.931908 17966 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:32.986339 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:33.043711 16545 raft_consensus.cc:1275] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:33.044030 16366 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:32.970654 17790 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:33.049638 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:33.047348 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:33.047472 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:33.052858 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:33.048167 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:33.049484 16401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 253) took 2101 ms. Trace:
I20260709 12:22:33.046536 16366 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:33.052054 16441 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:33.047850 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:33.055228 16366 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:33.057253 16441 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:33.053671 16401 rpcz_store.cc:276] 0709 12:22:30.943095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:30.943295 (+   200us) service_pool.cc:224] Handling call
0709 12:22:30.953218 (+  9923us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:31.932672 (+979454us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:33.044503 (+1111831us) spinlock_profiling.cc:243] Waited 946 ms on lock 0x7b54001e2154. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:33.044652 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2175256832}
I20260709 12:22:33.058233 18128 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:33.060612 18128 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:33.060982 18128 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:33.063346 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 60783.4us
I20260709 12:22:33.063989 17790 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:33.040220 16473 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 3.
I20260709 12:22:33.048964 17925 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:33.067579 18001 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:33.069003 18001 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:33.069459 18001 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:33.058167 18124 log.cc:927] Time spent T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
W20260709 12:22:33.071739 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:33.073434 18127 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:33.074110 17696 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:33.073962 18127 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:33.074882 16399 raft_consensus.cc:1240] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:33.074666 17696 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:33.076522 18004 raft_consensus.cc:3055] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:33.076941 18004 raft_consensus.cc:740] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:33.077014 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:33.077641 16551 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:33.077894 18004 consensus_queue.cc:260] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:33.078972 18004 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:33.079916 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:33.080749 16474 raft_consensus.cc:2393] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 3: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:33.077010 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:33.083077 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:33.083729 16543 raft_consensus.cc:2393] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 5: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:33.084889 16366 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:33.086706 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:33.086997 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:33.087132 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:33.087332 16550 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:33.088050 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:33.088305 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:22:33.089886 18123 log.cc:927] Time spent T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:22:33.086014 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:33.093190 16543 raft_consensus.cc:2393] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 5: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:33.094368 16472 raft_consensus.cc:1240] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:33.095168 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:33.095899 16550 raft_consensus.cc:2393] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 3: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:33.056298 18073 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:33.095844 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:33.058432 17877 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:33.103292 16367 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:33.105526 16474 raft_consensus.cc:1275] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:33.105996 18047 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.076246 18127 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:33.106564 18047 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:33.107335 16514 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:33.108399 18127 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:33.110677 18083 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:33.111438 18112 consensus_queue.cc:1059] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:33.112581 18060 consensus_queue.cc:1048] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:33.118330 18112 raft_consensus.cc:3055] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:33.112823 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:33.118691 18112 raft_consensus.cc:740] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:33.113735 16515 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:33.119743 17994 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:33.120138 17994 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:33.120566 17994 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:33.121061 17900 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:33.122612 16551 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:33.124123 17143 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:33.125319 17143 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:33.125885 18013 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.126473 18013 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:33.129962 18013 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:33.081473 18001 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:33.133945 18001 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:33.133175 16515 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:33.134701 18128 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:33.138218 17953 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:33.138695 17994 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:33.136318 16515 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:33.141394 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:33.141964 16515 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:33.142565 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:33.143771 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:33.144017 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:33.145313 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:33.145677 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:33.145985 16398 raft_consensus.cc:2393] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 3: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:33.147285 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:33.147713 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:33.149058 16469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 184) took 1130 ms. Trace:
I20260709 12:22:33.151466 16515 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:33.153100 16549 raft_consensus.cc:2410] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:22:33.098778 17790 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:33.170822 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 86050.1us
I20260709 12:22:33.168439 18005 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:33.148711 18001 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:33.176990 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:33.161535 18128 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:33.178900 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:33.164434 17780 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:33.153319 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:33.146273 16399 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:33.149437 16469 rpcz_store.cc:276] 0709 12:22:32.018892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:32.361098 (+342206us) service_pool.cc:224] Handling call
0709 12:22:32.361432 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:32.361820 (+   388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:32.361847 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:32.362309 (+   462us) log.cc:844] Serialized 26 byte log entry
0709 12:22:32.523613 (+161304us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:32.569130 (+ 45517us) log.cc:844] Serialized 14 byte log entry
0709 12:22:32.569352 (+   222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:32.569435 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:32.977436 (+408001us) spinlock_profiling.cc:243] Waited 6.94 ms on lock 0x7b18004ce4c0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c522 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:22:32.990102 (+ 12666us) spinlock_profiling.cc:243] Waited 5.25 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:22:33.148861 (+158759us) raft_consensus.cc:1685] finished
0709 12:22:33.148882 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:33.149001 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":41401,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":63,"raft.queue_time_us":571,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":4512,"spinlock_wait_cycles":28039808,"thread_start_us":158625,"threads_started":1,"wal-append.queue_time_us":224432}
I20260709 12:22:33.153955 18111 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:33.092770 17868 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:22:33.166903 18129 log.cc:927] Time spent T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 12:22:33.161916 17799 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:33.160317 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:33.174270 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:33.068006 18125 consensus_queue.cc:1048] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.108s
I20260709 12:22:33.210002 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:33.210609 16543 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
I20260709 12:22:33.211783 18072 raft_consensus.cc:3055] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:33.212229 18072 raft_consensus.cc:740] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:33.213505 16470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34840 (request call id 760) took 1193 ms. Trace:
I20260709 12:22:33.199669 17143 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:33.214082 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:33.214694 17834 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:33.214727 16477 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 3.
I20260709 12:22:33.215772 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:33.114213 17877 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:33.216382 17877 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:33.217953 17806 consensus_queue.cc:1048] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:33.218256 17143 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:33.219419 17143 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:33.220208 17867 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:33.220597 17867 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:33.220947 17867 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:33.222224 18047 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:33.224517 17650 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:33.208837 16474 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:33.202481 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:33.202732 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:33.202900 16367 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:33.226689 16547 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 3.
I20260709 12:22:33.226598 17143 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:33.154459 18077 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:33.210672 17144 leader_election.cc:400] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3.
I20260709 12:22:33.205693 17861 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:33.180099 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:33.226179 16472 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
I20260709 12:22:33.213271 18072 consensus_queue.cc:260] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:33.199519 18090 consensus_queue.cc:1048] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 12:22:33.236146 17966 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:33.237285 17143 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:33.235903 17799 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:33.213940 16470 rpcz_store.cc:276] 0709 12:22:32.020712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:32.025958 (+  5246us) service_pool.cc:224] Handling call
0709 12:22:33.213348 (+1187390us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b54004eff54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:33.213447 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":860078336}
I20260709 12:22:33.241799 17799 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:22:33.206172 18131 log.cc:927] Time spent T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 12:22:33.242532 17899 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:33.243029 17899 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:33.227761 17834 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:33.198793 17780 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:33.244788 17834 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:33.244884 17780 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:33.177932 18005 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:33.198340 18111 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:33.201155 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 49809.5us
I20260709 12:22:33.256565 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:33.257346 17271 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 1.
I20260709 12:22:33.257984 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:33.244095 16470 raft_consensus.cc:1275] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:33.233698 16366 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:33.205789 17875 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:33.266583 17875 raft_consensus.cc:697] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 3 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:33.232600 17925 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:33.234735 18069 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:33.235085 17144 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:33.273079 18069 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:33.263300 17900 consensus_queue.cc:1059] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 12:22:33.274202 17900 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:33.274912 17900 raft_consensus.cc:3055] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:33.275553 17900 raft_consensus.cc:740] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:33.276626 16401 raft_consensus.cc:1275] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:33.206936 18019 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:33.235808 17696 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:33.283800 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:33.284070 17144 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:33.267645 17875 consensus_queue.cc:237] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:33.199867 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:33.203912 16473 raft_consensus.cc:1275] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:33.239348 18084 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:33.244367 17899 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:33.227655 16367 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:22:33.294710 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 67124us
I20260709 12:22:33.294931 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:33.295717 16470 raft_consensus.cc:2393] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 2: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:33.180840 18081 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:33.211828 16543 raft_consensus.cc:1240] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:22:33.227778 16477 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:22:33.259483 16400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46250 (request call id 621) took 1174 ms. Trace:
I20260709 12:22:33.260990 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:33.261353 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:33.263850 16366 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:33.304864 16366 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:33.264627 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:33.305310 16515 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:33.278080 18072 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:33.253010 18111 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:33.265534 18087 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:33.272384 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:33.177830 18106 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:33.238529 17976 consensus_meta.cc:287] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.557s	user 0.009s	sys 0.000s
I20260709 12:22:33.280675 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:33.302138 18002 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:33.179983 17790 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:33.291239 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:33.238997 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:33.242318 17799 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:33.254866 18005 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:33.297912 16442 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:33.228080 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:22:33.323153 18104 log.cc:927] Time spent T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.685s	user 0.001s	sys 0.001s
I20260709 12:22:33.299851 16367 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:33.303798 17271 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:33.231137 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:33.307241 16515 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:33.206501 17868 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:33.311585 17696 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:33.336366 16397 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:33.237056 18112 consensus_queue.cc:260] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:33.338016 16397 raft_consensus.cc:1240] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:33.338101 18112 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:33.339485 16397 raft_consensus.cc:1275] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:33.341614 17868 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:33.342120 16470 raft_consensus.cc:1275] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:33.343142 18084 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:33.343549 18084 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:33.343904 18084 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:33.346508 17780 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:33.315518 17900 consensus_queue.cc:260] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:33.349609 17900 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:33.301685 18090 consensus_queue.cc:1059] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 12:22:33.353040 18090 raft_consensus.cc:3055] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:33.353395 18090 raft_consensus.cc:740] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:33.309808 16395 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:33.357271 17871 consensus_queue.cc:1059] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:33.357450 18008 consensus_queue.cc:1048] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:33.357971 18020 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:33.358546 18020 raft_consensus.cc:3055] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:33.358860 18020 raft_consensus.cc:740] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:33.359783 18020 consensus_queue.cc:260] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:33.360616 18111 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:33.361064 18020 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:33.362413 16516 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:33.362703 18111 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:33.363292 18111 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:33.291909 18019 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:33.365720 18019 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:33.303509 16400 rpcz_store.cc:276] 0709 12:22:32.085219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:32.110102 (+ 24883us) service_pool.cc:224] Handling call
0709 12:22:33.259269 (+1149167us) spinlock_profiling.cc:243] Waited 52.6 ms on lock 0x7b54009bd2d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:33.259411 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":120913152}
I20260709 12:22:33.366986 16223 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59300 (request call id 83) took 6788 ms. Trace:
I20260709 12:22:33.368721 18106 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:33.372440  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:33.373318  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:33.373482 16473 raft_consensus.cc:1275] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:33.263368 16546 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:33.313505 16549 raft_consensus.cc:1275] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:33.377036 17877 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:33.378912 17794 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:33.379321 17877 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:33.379407 17794 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:33.382786 17780 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:22:33.383858 16402 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 691) took 2327 ms (client timeout 1574 ms). Trace:
I20260709 12:22:33.385871 16515 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:33.385653 16224 catalog_manager.cc:5697] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 1 to 3, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 3 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:33.387837 16224 catalog_manager.cc:5697] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 3 to 4, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 4 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:33.392287 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:33.393183 16403 raft_consensus.cc:1240] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:33.263727 17994 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:33.316933 16544 raft_consensus.cc:1240] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:33.242321 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:33.316342 17261 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:33.318473 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:33.405592 16547 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 3.
I20260709 12:22:33.318455 17915 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.323655 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:33.315992 18077 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:33.329068 18128 consensus_queue.cc:1048] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:33.329408 17992 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:33.329613 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:33.313697 18121 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:33.304566 18066 consensus_queue.cc:1048] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 12:22:33.409916 17790 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:33.330865 17953 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:33.331640 17969 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:33.311094 18087 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:33.335526 16550 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:33.292131 17872 consensus_queue.cc:1048] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 12:22:33.393705 16224 catalog_manager.cc:5697] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 3 to 4, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 4 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
W20260709 12:22:33.404295 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 104536us
I20260709 12:22:33.406694 17258 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
W20260709 12:22:33.384550 16402 rpcz_store.cc:269] 0709 12:22:31.056234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:31.070318 (+ 14084us) service_pool.cc:224] Handling call
0709 12:22:33.383605 (+2313287us) spinlock_profiling.cc:243] Waited 1.82 s on lock 0x7b540015e854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:33.383742 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4181097728}
I20260709 12:22:33.411540 16398 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:33.412716 16515 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:33.414469 16477 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:22:33.410488 18121 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:33.444317 18002 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:33.456254 16366 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:33.464665 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 52029.9us
I20260709 12:22:33.367379 16223 rpcz_store.cc:276] 0709 12:22:26.578845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:26.623619 (+ 44774us) service_pool.cc:224] Handling call
0709 12:22:26.818520 (+194901us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b540027e054. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df3090 00007f2875e91f3a 00007f287bc47e9c 00007f287bd4604e 00007f287a2072a4 00007f287a207232 00007f287a207161 00007f287a2070dc 00007f287a206392 00007f28741c448c 00007f28741c38c6
0709 12:22:33.366911 (+6548391us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:22:29.538627 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:22:29.825422 (+286795us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:22:29.833632 (+  8210us) write_op.cc:435] Acquired schema lock
0709 12:22:29.838580 (+  4948us) tablet.cc:662] Decoding operations
0709 12:22:29.957350 (+118770us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:22:29.966363 (+  9013us) write_op.cc:641] Partition lock acquired for write op
0709 12:22:29.973845 (+  7482us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:22:29.974216 (+   371us) tablet.cc:701] Row locks acquired
0709 12:22:29.974236 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:22:29.974408 (+   172us) write_op.cc:270] Start()
0709 12:22:29.974564 (+   156us) write_op.cc:276] Timestamp: P: 1783599749974357 usec, L: 0
0709 12:22:29.974587 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:22:29.975323 (+   736us) log.cc:844] Serialized 1289 byte log entry
0709 12:22:30.403689 (+428366us) op_driver.cc:464] REPLICATION: finished
0709 12:22:30.427642 (+ 23953us) write_op.cc:301] APPLY: starting
0709 12:22:30.427709 (+    67us) tablet.cc:1367] starting BulkCheckPresence
0709 12:22:30.427812 (+   103us) tablet.cc:1370] finished BulkCheckPresence
0709 12:22:30.427901 (+    89us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:22:30.428409 (+   508us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:22:30.428560 (+   151us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 12:22:30.428590 (+    30us) write_op.cc:312] APPLY: finished
0709 12:22:30.428755 (+   165us) log.cc:844] Serialized 34 byte log entry
0709 12:22:30.428964 (+   209us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:30.429057 (+    93us) write_op.cc:454] Released schema lock
0709 12:22:30.429095 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 12:22:30.430109 (+  1014us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:30.430128 (+    19us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":31,"reported_tablets":15,"spinlock_wait_cycles":330525952,"child_traces":[["op",{"apply.queue_time_us":23702,"apply.run_cpu_time_us":1774,"apply.run_wall_time_us":2464,"num_ops":3,"prepare.queue_time_us":445614,"prepare.run_cpu_time_us":10070,"prepare.run_wall_time_us":473894,"raft.queue_time_us":67866,"raft.run_cpu_time_us":7529,"raft.run_wall_time_us":24019,"replication_time_us":428956,"thread_start_us":449732,"threads_started":4,"wal-append.queue_time_us":147583,"wal-append.run_cpu_time_us":57298,"wal-append.run_wall_time_us":1350294}]]}
I20260709 12:22:33.460929 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:33.410610 18047 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:33.462108 18121 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:33.470850 16441 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:33.474215 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:33.469455 16469 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:33.516959 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 46213.4us
I20260709 12:22:33.477388 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:33.518354 16477 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:33.477684 18083 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:33.437150 17861 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:33.479027 18081 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:33.468647 17992 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:33.482007 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:33.486773 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:33.405300 17877 consensus_queue.cc:1048] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.070s
I20260709 12:22:33.493847 17142 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:22:33.542186 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 48439.3us
I20260709 12:22:33.354234 18090 consensus_queue.cc:260] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:33.500927 18136 consensus_queue.cc:1048] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:33.502947 16401 raft_consensus.cc:1240] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:33.503347 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:33.455667 16367 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:33.560499 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 104909us
I20260709 12:22:33.510154 16551 raft_consensus.cc:1240] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:33.411098 18077 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:33.433337 17953 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:33.449146 18087 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:33.524776 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:33.519799 16441 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:22:33.564644 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 44945.1us
I20260709 12:22:33.442600 17915 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:33.532053 17946 consensus_meta.cc:287] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.677s	user 0.011s	sys 0.000s
I20260709 12:22:33.565389 16441 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:33.521981 17976 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:33.530131 17925 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:33.529367 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:33.383985 17867 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:33.458114 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:33.537216 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:33.493278 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:33.411127 18111 consensus_queue.cc:1048] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.104s
I20260709 12:22:33.592128 17270 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
W20260709 12:22:33.524308 18116 consensus_meta.cc:287] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.567s	user 0.006s	sys 0.004s
I20260709 12:22:33.566751 16441 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:33.452057 16224 catalog_manager.cc:5697] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:22:33.611255 16224 catalog_manager.cc:5697] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 1 to 3, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 3 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:33.613659 16469 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:22:33.613493 16224 catalog_manager.cc:5697] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 1 to 3, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 3 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:33.571178 18047 raft_consensus.cc:2804] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:33.567166 16367 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:33.619946 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 52877.8us
I20260709 12:22:33.621110 16224 catalog_manager.cc:5697] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 2 to 4, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 4 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:33.528674 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:33.623512 17263 raft_consensus.cc:2368] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 4. Current term is 5.
I20260709 12:22:33.573531 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:33.575119 16515 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:33.571692 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:33.462430 18002 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:33.564821 17954 consensus_meta.cc:287] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.761s	user 0.012s	sys 0.000s
I20260709 12:22:33.559334 17871 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:33.541095 18083 raft_consensus.cc:697] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:33.552557 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:33.595392 17261 raft_consensus.cc:2393] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 4: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:33.598547 16476 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:33.598660 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:33.598176 17790 consensus_queue.cc:1048] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 12:22:33.571506 17946 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:33.680693 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:33.681900 17946 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:33.682013 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:33.683120 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:33.683743 17261 raft_consensus.cc:2368] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 4. Current term is 5.
I20260709 12:22:33.563656 18077 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:33.684273 18084 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:33.616427 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 49739.3us
I20260709 12:22:33.688915 16441 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:33.449733 17969 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:33.690021 16441 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:33.691068 17969 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:33.573421 18073 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:33.692034 18073 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:33.692561 18073 raft_consensus.cc:697] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:33.692739 17829 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:33.693583 18073 consensus_queue.cc:237] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:33.618536 16468 raft_consensus.cc:1240] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260709 12:22:33.696579 17269 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:33.699656 17915 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:33.700294 18005 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:33.701669 18084 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:33.702462 18005 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:33.710397 16476 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34826 (request call id 688) took 1678 ms (client timeout 1842 ms). Trace:
W20260709 12:22:33.710925 16476 rpcz_store.cc:269] 0709 12:22:32.032016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:32.050541 (+ 18525us) service_pool.cc:224] Handling call
0709 12:22:33.708291 (+1657750us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b5400a690d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:33.710325 (+  2034us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3510781824}
W20260709 12:22:33.716382 18138 log.cc:927] Time spent T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.272s	user 0.005s	sys 0.000s
I20260709 12:22:33.691138 16441 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:33.622783 16224 catalog_manager.cc:5697] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 1 to 3, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 3 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:33.583830 16477 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:33.624850 16442 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:33.722101 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 97361.7us
I20260709 12:22:33.722710 16442 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:33.559216 17780 consensus_queue.cc:1059] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 12:22:33.573213 17966 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:33.627477 17263 raft_consensus.cc:1240] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 5. Current term is 6. Ops: []
W20260709 12:22:33.632597 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 57641.2us
I20260709 12:22:33.738312 16515 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:33.738839 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 74169.9us
I20260709 12:22:33.739475 16442 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:33.739708 16515 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:33.553687 17834 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:33.644222 17271 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:33.604024 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:33.650892 17794 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:33.641033 17954 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:33.640856 17871 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:33.653294 18083 consensus_queue.cc:237] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:33.640578 17556 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:33.656985 17867 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:33.661748 18002 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:33.617779 17976 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:33.628270 18047 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:33.633420 18106 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:33.669627 16549 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:33.574993 17899 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:33.625216 18116 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:33.754045 17871 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:33.724998 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:33.720117 16441 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:33.727998 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:33.797678 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:33.798601 16551 raft_consensus.cc:2393] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 6: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
W20260709 12:22:33.799443 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 79386.3us
I20260709 12:22:33.724291 16224 catalog_manager.cc:5697] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 2 to 4, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 4 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:33.730232 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:33.731803 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:33.734043 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:33.746431 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:33.746932 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:33.748083 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:33.748435 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:33.802811 16224 catalog_manager.cc:5697] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 2 to 5, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 5 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:33.754707 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:33.743817 18087 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:33.755115 18005 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:22:33.750001 16469 raft_consensus.cc:1240] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:33.754312 17264 raft_consensus.cc:1240] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:33.746459 18126 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:33.754333 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:33.761818 16397 raft_consensus.cc:2368] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3.
I20260709 12:22:33.743168 17915 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:33.833158 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:33.743577 18084 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:33.769485 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:33.745947 18125 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:33.774967 16543 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:33.783916 16514 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
W20260709 12:22:33.836124 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 52302.9us
I20260709 12:22:33.750550 17665 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:22:33.794680 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 294998us
I20260709 12:22:33.724682 17269 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:33.805356 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:33.851049 16398 raft_consensus.cc:1240] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:33.800853 16544 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:33.806666 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:33.822256 16224 catalog_manager.cc:5697] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 1 to 2, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 2 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:33.829087 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:33.853125 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:33.853485 16551 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:33.832188 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:33.776551 17790 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:33.821609 17966 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:33.855680 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:33.869612 17263 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:33.795377 18047 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:33.871901 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 284802us
I20260709 12:22:33.861863 16515 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:33.863437 17143 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:33.835670 18125 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:33.881379 16543 raft_consensus.cc:1240] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:33.796131 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:33.867246 17650 consensus_queue.cc:1059] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:22:33.870911 18008 consensus_queue.cc:1048] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:33.910470 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:33.819813 18106 raft_consensus.cc:697] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 4 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:33.751616 18121 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:33.830219 17867 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.821838 18126 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:33.876647 18001 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:33.793882 17969 consensus_queue.cc:1059] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.069s
I20260709 12:22:33.844751 16471 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:33.876835 16551 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:22:33.879245 17143 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:33.931391 17263 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
W20260709 12:22:33.932399 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 53165.1us
I20260709 12:22:33.884407 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:33.936877 16477 raft_consensus.cc:2368] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 3. Current term is 4.
I20260709 12:22:33.842720 18084 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:33.888391 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:33.847951 18066 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:33.875890 16470 raft_consensus.cc:1240] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:33.889981 18116 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:33.890654 18090 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:33.890949 18145 log.cc:927] Time spent T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 12:22:33.891656 18144 log.cc:927] Time spent T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:22:33.897739 17834 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:33.900048 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:33.900911 18087 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:33.901705 18125 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:33.902025 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:33.903990 17954 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:33.904636 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:33.885108 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:33.985975 17269 raft_consensus.cc:1275] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:33.871023 17900 consensus_queue.cc:1059] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:33.988727 17254 raft_consensus.cc:2393] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 4: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:33.913503 17261 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:22:33.819463 17799 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:33.991314 17799 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:22:33.873147 18069 consensus_meta.cc:287] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.600s	user 0.011s	sys 0.000s
I20260709 12:22:33.902160 18013 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:33.991887 18125 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:33.992554 18106 consensus_queue.cc:237] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:33.847883 17829 consensus_queue.cc:1059] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.064s
I20260709 12:22:33.874449 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:33.908352 16396 raft_consensus.cc:1275] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:33.928237 17264 raft_consensus.cc:1275] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:33.981336 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:33.938401 16403 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:22:34.002394 16544 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
I20260709 12:22:33.872020 17953 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:33.939612 17143 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:33.872378 17790 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:33.941779 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:33.926610 18126 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:33.942425 17780 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:33.946905 17650 consensus_queue.cc:1059] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 12:22:33.980726 16394 raft_consensus.cc:1240] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 3 LEADER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:33.955466 16516 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:33.981752 18066 raft_consensus.cc:697] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:22:34.031603 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 92116.4us
I20260709 12:22:33.926091 17867 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:33.978466 16548 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:33.981431 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:34.022306 16551 raft_consensus.cc:2393] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 2: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:33.940516 18084 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:34.032864 17143 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:33.936062 17696 consensus_queue.cc:1059] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.073s
I20260709 12:22:34.042786 17143 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:33.936846 17794 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:22:34.054658 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 430496us
I20260709 12:22:34.001906 18151 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.954838 18127 raft_consensus.cc:3055] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:33.963729 17665 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:34.057636 17143 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:34.027865 17954 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:34.074205 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:34.076462 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:34.079460 16397 raft_consensus.cc:2393] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 4: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:34.078848 17143 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:34.051977 17834 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:34.054621 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:33.992594 18069 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:34.083341 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:34.088884 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:34.092782 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:34.031574 17255 raft_consensus.cc:2376] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6. Re-sending same reply.
I20260709 12:22:34.067654 17925 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:34.070895 17780 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:34.068270 17966 raft_consensus.cc:697] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:34.073958 18127 raft_consensus.cc:740] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:22:34.104885 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 149524us
W20260709 12:22:34.104593 18148 log.cc:927] Time spent T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 12:22:34.120930 16475 raft_consensus.cc:1240] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:34.079353 16473 raft_consensus.cc:1240] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:34.080224 17650 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:34.097373 18008 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:34.094758 18060 raft_consensus.cc:3055] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:34.105171 16543 raft_consensus.cc:2393] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 4: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:34.095737 17665 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:34.099229 17868 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:34.104671 17899 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:34.081666 17954 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:34.115711 16516 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:34.134282 16542 raft_consensus.cc:2393] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 3: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:34.115442 17799 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:22:34.078138 18151 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:34.151031 16477 raft_consensus.cc:1240] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:34.130683 17780 raft_consensus.cc:697] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:22:34.153365 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 316742us
I20260709 12:22:34.155460 16366 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:34.113296 17925 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:34.159269 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:34.136467 18060 raft_consensus.cc:740] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:34.129527 18005 consensus_queue.cc:1059] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 12:22:34.173032 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:34.221891 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:34.222257 16548 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:34.160720 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:34.224699 16397 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:34.224818 17142 leader_election.cc:400] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 3. Current term is 4.
I20260709 12:22:34.225147 17142 leader_election.cc:403] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:22:34.229188 18168 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 3. Current term is 4.
I20260709 12:22:34.229684 17799 consensus_queue.cc:1059] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:34.231011 18166 raft_consensus.cc:3055] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:34.231340 18166 raft_consensus.cc:740] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:34.232323 18166 consensus_queue.cc:260] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:34.152611 17556 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:34.145959 18125 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:34.261912 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:34.262977 16544 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:34.164263 17875 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:34.118752 18066 consensus_queue.cc:237] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:34.161784 17696 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:22:34.247897 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 152133us
I20260709 12:22:34.273979 16367 leader_election.cc:400] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 4. Current term is 5.
I20260709 12:22:34.288260 16367 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:34.148284 17868 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:34.313635 18163 log.cc:927] Time spent T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:22:34.315007 18170 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:34.316165 16367 leader_election.cc:400] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 4. Current term is 5.
W20260709 12:22:34.316316 18152 log.cc:927] Time spent T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.202s	user 0.000s	sys 0.000s
W20260709 12:22:34.317276 18158 log.cc:927] Time spent T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.208s	user 0.000s	sys 0.000s
I20260709 12:22:34.317415 18083 consensus_queue.cc:1059] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:34.318701 17877 raft_consensus.cc:3055] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:34.319114 17877 raft_consensus.cc:740] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:34.320034 17877 consensus_queue.cc:260] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:34.321528 17877 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:34.325104 17790 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:34.325773 18161 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:34.326305 18161 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:34.300249 17875 raft_consensus.cc:697] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:34.330488 16367 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:34.330363 17875 consensus_queue.cc:237] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:34.331781 18072 consensus_queue.cc:1059] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:34.332819 18072 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:34.333555 18072 raft_consensus.cc:3055] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:34.333765 18151 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:34.334192 18072 raft_consensus.cc:740] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:34.333694 16367 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:34.335884 18072 consensus_queue.cc:260] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:34.337332 18072 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:34.338486 18087 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:34.338723 17665 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:34.222285 18153 consensus_queue.cc:1059] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.177s
W20260709 12:22:34.340904 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:34.341296 18153 raft_consensus.cc:3055] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:34.341807 18153 raft_consensus.cc:740] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:34.342831 18153 consensus_queue.cc:260] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:34.260795 17780 consensus_queue.cc:237] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:34.158370 18001 consensus_queue.cc:1059] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.112s
I20260709 12:22:34.350651 16367 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:22:34.368620 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:34.371515 16544 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 3.
W20260709 12:22:34.379664 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:34.323699 17871 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:34.422353 17790 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:34.423133 17871 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:22:34.423830 18156 log.cc:927] Time spent T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.202s	user 0.010s	sys 0.002s
W20260709 12:22:34.424980 18159 log.cc:927] Time spent T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.090s	user 0.001s	sys 0.010s
W20260709 12:22:34.426256 17976 consensus_meta.cc:287] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.640s	user 0.005s	sys 0.005s
I20260709 12:22:34.426661 17976 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:34.222045 17969 consensus_queue.cc:1048] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.080s
I20260709 12:22:34.272531 17953 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:22:34.451942 18149 log.cc:927] Time spent T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.347s	user 0.005s	sys 0.001s
W20260709 12:22:34.452353 18169 log.cc:927] Time spent T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.085s	user 0.006s	sys 0.000s
I20260709 12:22:34.423193 18161 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:34.447283 18153 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:34.457182 17867 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:22:34.470304 18077 consensus_meta.cc:287] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.784s	user 0.006s	sys 0.005s
I20260709 12:22:34.269060 17966 consensus_queue.cc:237] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:34.493517 16475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 701) took 1272 ms. Trace:
I20260709 12:22:34.497833 16475 rpcz_store.cc:276] 0709 12:22:33.221421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:33.243815 (+ 22394us) service_pool.cc:224] Handling call
0709 12:22:33.244083 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:34.129989 (+885906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:34.493284 (+363295us) spinlock_profiling.cc:243] Waited 872 ms on lock 0x7b54002f2e54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:34.493458 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2005775104}
I20260709 12:22:34.324239 16548 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:34.494975 16397 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:34.503165 16516 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:34.320221 16471 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
W20260709 12:22:34.504503 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 278680us
I20260709 12:22:34.506839 16474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 788) took 1096 ms. Trace:
I20260709 12:22:34.507215 16474 rpcz_store.cc:276] 0709 12:22:33.410037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:33.482090 (+ 72053us) service_pool.cc:224] Handling call
0709 12:22:33.482360 (+   270us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:33.482739 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:33.482766 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:33.483204 (+   438us) log.cc:844] Serialized 26 byte log entry
0709 12:22:33.718868 (+235664us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:33.718907 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:33.718977 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:34.506627 (+787650us) raft_consensus.cc:1685] finished
0709 12:22:34.506648 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:34.506783 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1011,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":82,"thread_start_us":235360,"threads_started":1,"wal-append.queue_time_us":587808}
I20260709 12:22:34.509083 18005 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:34.512097 16400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 281) took 1120 ms. Trace:
I20260709 12:22:34.513020 17954 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:34.517325 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:34.518080 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:34.518313 16548 raft_consensus.cc:2393] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 4: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:34.518750 16550 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:34.519676 17868 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:34.520819 16395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 280) took 1234 ms. Trace:
I20260709 12:22:34.520932 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:34.521937 16543 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:34.521382 18069 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:22:34.524601 18047 consensus_meta.cc:287] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.628s	user 0.008s	sys 0.003s
I20260709 12:22:34.525161 18047 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:34.525707 16550 raft_consensus.cc:1240] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:34.527411 18047 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:34.528203 16401 raft_consensus.cc:1240] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:22:34.493366 18077 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:34.534477 18077 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:34.535211 16543 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
W20260709 12:22:34.535733 18090 consensus_meta.cc:287] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.584s	user 0.009s	sys 0.001s
I20260709 12:22:34.521268 16395 rpcz_store.cc:276] 0709 12:22:33.286730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:33.358896 (+ 72166us) service_pool.cc:224] Handling call
0709 12:22:33.359227 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:33.515375 (+156148us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:33.523864 (+  8489us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:33.615506 (+ 91642us) log.cc:844] Serialized 26 byte log entry
0709 12:22:34.112007 (+496501us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:22:34.112054 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:34.520428 (+408374us) spinlock_profiling.cc:243] Waited 51.2 ms on lock 0x7b540015fc54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:34.520488 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:34.520539 (+    51us) raft_consensus.cc:1685] finished
0709 12:22:34.520551 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:34.520762 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13159,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"spinlock_wait_cycles":117867776,"thread_start_us":447394,"threads_started":1,"wal-append.queue_time_us":520268}
I20260709 12:22:34.539075 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:22:34.539338 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:34.539717 16471 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:22:34.540527 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:34.541320 17271 raft_consensus.cc:2393] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 4: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:34.541383 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:34.542305 16474 raft_consensus.cc:2410] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:22:34.542495 17143 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:34.543329 17871 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:34.544381 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:34.550616 18166 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:34.554029 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:34.555006 16394 raft_consensus.cc:2410] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:22:34.555475 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:22:34.562673 18084 consensus_meta.cc:287] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.526s	user 0.011s	sys 0.001s
I20260709 12:22:34.512609 16400 rpcz_store.cc:276] 0709 12:22:33.391402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:33.391932 (+   530us) service_pool.cc:224] Handling call
0709 12:22:33.455995 (+ 64063us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:33.457474 (+  1479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:33.457527 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:33.475422 (+ 17895us) log.cc:844] Serialized 26 byte log entry
0709 12:22:33.754427 (+279005us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:33.754467 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:33.754537 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:34.164721 (+410184us) raft_consensus.cc:1685] finished
0709 12:22:34.164755 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:34.511838 (+347083us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b54005e8dd0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:34.512037 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2258,"raft.queue_time_us":1653,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":13160,"spinlock_wait_cycles":39850240,"thread_start_us":274749,"threads_started":1,"wal-append.queue_time_us":449263}
I20260709 12:22:34.578069 17143 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:34.555095 17144 leader_election.cc:400] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3.
I20260709 12:22:34.579735 17258 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:34.581187 17144 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:34.582425  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:34.591343 17143 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:34.506132 18020 raft_consensus.cc:3055] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:34.603691 17144 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:34.564216 18084 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:34.599565 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 248980us
I20260709 12:22:34.565240 18005 raft_consensus.cc:697] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:22:34.576884 17829 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.011s	sys 0.003s
W20260709 12:22:34.606240 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 448906us
I20260709 12:22:34.605376 16514 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:34.608568 16367 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:34.608852 16441 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:34.609591 16469 raft_consensus.cc:1240] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:34.610862 18116 raft_consensus.cc:2804] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:34.611233 18116 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:34.611617 18116 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:34.613140 18084 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:34.614790 16441 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:34.615840 16441 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:34.622426 16441 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:34.628296 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:34.629438 16551 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:34.632846 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:34.633785 16549 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:34.600051 18090 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:34.648070 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:22:34.715010 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 92693us
I20260709 12:22:34.691097 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:34.604018 18020 raft_consensus.cc:740] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:34.692458 18127 consensus_queue.cc:260] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:34.692847 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 86140.8us
W20260709 12:22:34.621165 17829 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.012s	sys 0.004s
I20260709 12:22:34.695995 16550 raft_consensus.cc:1240] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:34.696578 18172 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:34.697499 16473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 784) took 1374 ms. Trace:
W20260709 12:22:34.697602 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:34.698055 16551 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 3.
I20260709 12:22:34.729897 16473 rpcz_store.cc:276] 0709 12:22:33.323068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:33.376174 (+ 53106us) service_pool.cc:224] Handling call
0709 12:22:33.405999 (+ 29825us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:34.132999 (+727000us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:34.697261 (+564262us) spinlock_profiling.cc:243] Waited 656 ms on lock 0x7b5400a69fd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:34.697444 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1509370112}
I20260709 12:22:34.698751 18153 consensus_queue.cc:1059] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:34.621186 18161 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:34.674158 17665 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:34.704644 17868 consensus_queue.cc:1048] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
W20260709 12:22:34.673620 17915 consensus_meta.cc:287] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.832s	user 0.012s	sys 0.001s
I20260709 12:22:34.708678 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:34.709772 16470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 210) took 1355 ms. Trace:
I20260709 12:22:34.712090 16476 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:39000
I20260709 12:22:34.753372 16476 raft_consensus.cc:606] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
W20260709 12:22:34.605623 18126 consensus_meta.cc:287] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.552s	user 0.011s	sys 0.000s
I20260709 12:22:34.676614 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:34.692166 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 88556.5us
I20260709 12:22:34.715837 16441 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:22:34.703194 18155 log.cc:927] Time spent T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.584s	user 0.002s	sys 0.008s
I20260709 12:22:34.718416 17263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 588) took 1039 ms. Trace:
I20260709 12:22:34.742714 17556 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:34.634815 18084 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:34.758252 17263 rpcz_store.cc:276] 0709 12:22:33.679075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:33.946551 (+267476us) service_pool.cc:224] Handling call
0709 12:22:33.978850 (+ 32299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:33.979092 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:22:34.026449 (+ 47357us) log.cc:844] Serialized 14 byte log entry
0709 12:22:34.717791 (+691342us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:34.717867 (+    76us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:34.717932 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:34.718029 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:34.718343 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52245,"prepare.queue_time_us":47115,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":12606,"raft.queue_time_us":55827,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"thread_start_us":622757,"threads_started":2,"wal-append.queue_time_us":321989}
I20260709 12:22:34.754045 17915 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:22:34.763639 18175 log.cc:927] Time spent T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 12:22:34.764772 16547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 774) took 1253 ms. Trace:
I20260709 12:22:34.765192 16547 rpcz_store.cc:276] 0709 12:22:33.511262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:33.511418 (+   156us) service_pool.cc:224] Handling call
0709 12:22:33.511914 (+   496us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:33.512399 (+   485us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:33.512442 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:33.513021 (+   579us) log.cc:844] Serialized 26 byte log entry
0709 12:22:33.929083 (+416062us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:33.953015 (+ 23932us) log.cc:844] Serialized 14 byte log entry
0709 12:22:33.953304 (+   289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:33.953404 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:34.702317 (+748913us) spinlock_profiling.cc:243] Waited 2.78 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:22:34.764469 (+ 62152us) raft_consensus.cc:1685] finished
0709 12:22:34.764568 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:34.764714 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19933,"prepare.queue_time_us":13765,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":2455,"raft.run_cpu_time_us":343,"raft.run_wall_time_us":40980,"spinlock_wait_cycles":6397824,"thread_start_us":415518,"threads_started":1,"wal-append.queue_time_us":564856}
I20260709 12:22:34.775141 16546 raft_consensus.cc:1275] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:34.760703 18126 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:34.778904 16477 raft_consensus.cc:1240] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:34.753031 16470 rpcz_store.cc:276] 0709 12:22:33.353983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:33.479052 (+125069us) service_pool.cc:224] Handling call
0709 12:22:33.479365 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:33.951894 (+472529us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:34.709502 (+757608us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b5400a68bd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:34.709712 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":848926080}
I20260709 12:22:34.726462 17143 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:34.673367 18047 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:34.782527 16475 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
mode: GRACEFUL
new_leader_uuid: "610f4feda11942739236f2b121a4b9c1"
 from {username='slave'} at 127.0.0.1:39000
I20260709 12:22:34.783275 18047 raft_consensus.cc:697] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:34.784545 16471 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:34.784376 18047 consensus_queue.cc:237] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:34.786516 17829 raft_consensus.cc:993] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2: : Instructing follower ef57beab0e1d40d585983873c55107e8 to start an election
I20260709 12:22:34.787122 17829 raft_consensus.cc:1081] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Signalling peer ef57beab0e1d40d585983873c55107e8 to start an election
I20260709 12:22:34.786782 18020 consensus_queue.cc:260] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:34.722543 17953 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:34.788455 17556 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:34.790541 17925 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:34.761518 18005 consensus_queue.cc:237] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:34.791203 17556 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:34.757944 17665 raft_consensus.cc:697] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:34.794914 17806 raft_consensus.cc:3055] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:34.795469 17806 raft_consensus.cc:740] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:34.795037 17665 consensus_queue.cc:237] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:34.796429 17806 consensus_queue.cc:260] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:34.799911 18184 log.cc:927] Time spent T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:22:34.800422 18121 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:34.758991 18127 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:34.803776 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 77456.4us
I20260709 12:22:34.804565 18172 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:34.805202 18172 raft_consensus.cc:697] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:34.806150 18172 consensus_queue.cc:237] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:34.807402 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:34.808104 16469 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:34.809618 16514 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:34.809865 18073 raft_consensus.cc:3055] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:34.616942 18066 consensus_queue.cc:1059] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.123s
I20260709 12:22:34.751447 16543 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
I20260709 12:22:34.739490 18171 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:34.755585 16441 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
W20260709 12:22:34.835253 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 79784.2us
I20260709 12:22:34.607091 17899 raft_consensus.cc:476] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Not starting pre-election -- already a leader
I20260709 12:22:34.788172 18020 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:34.735289 18176 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:22:34.838292 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:34.839516 16441 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:34.841209 16441 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:34.623860 18077 consensus_queue.cc:1048] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.116s
W20260709 12:22:34.810096 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 49123.7us
I20260709 12:22:34.832129 17556 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:34.820926 18073 raft_consensus.cc:740] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:34.816058 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:34.783303 16475 raft_consensus.cc:606] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Received request to transfer leadership to TS 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:34.856048 16475 raft_consensus.cc:614] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:34.816880 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:34.818634 18001 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:34.818919 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:34.857197 17269 raft_consensus.cc:2393] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 4: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:34.832427 17263 raft_consensus.cc:1275] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:22:34.741994 18178 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:34.759158 17966 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:22:34.810554 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:34.813488 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:34.852788 16544 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:34.846462 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:34.842602 18176 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:34.843606 17915 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:34.865262 17271 raft_consensus.cc:2393] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 4: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:34.847182 17829 raft_consensus.cc:2764] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:22:34.866364 16400 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:34.847414 18161 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:22:34.851603 18164 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.017s	sys 0.022s
I20260709 12:22:34.852041 18121 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:34.859350 18001 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:34.873313 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:34.912638 16401 raft_consensus.cc:1240] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:34.860116 17925 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:34.864842 18151 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:34.912017 18164 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.017s	sys 0.022s
I20260709 12:22:34.857530 17556 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:34.919129 16470 raft_consensus.cc:1240] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:22:34.918418 18001 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:34.915192 17966 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:34.871886 18161 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:22:34.941905 18008 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.790s	user 0.005s	sys 0.021s
I20260709 12:22:34.870683 17868 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:34.982688 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:34.911897 17255 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:34.870215 18176 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:34.870368 17915 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:22:34.973112 18160 log.cc:927] Time spent T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:22:34.867956 17871 consensus_queue.cc:1059] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.117s
I20260709 12:22:35.000000 17915 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:34.984079 18159 log.cc:927] Time spent T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 12:22:35.004827 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 158850us
I20260709 12:22:34.983875 18047 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:22:34.989368 18008 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.840s	user 0.005s	sys 0.021s
I20260709 12:22:34.918217 18121 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:34.969026 18179 consensus_queue.cc:1059] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.092s
I20260709 12:22:34.995358 18161 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:35.010959 17266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 585) took 1696 ms. Trace:
I20260709 12:22:35.007591 16441 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:34.972534 17925 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:34.864846 18178 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:34.980409 18151 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:35.004606 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:35.025205 17266 rpcz_store.cc:276] 0709 12:22:33.315071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:33.315201 (+   130us) service_pool.cc:224] Handling call
0709 12:22:33.340627 (+ 25426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:33.366324 (+ 25697us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:22:33.606540 (+240216us) log.cc:844] Serialized 14 byte log entry
0709 12:22:33.917656 (+311116us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:33.917706 (+    50us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:33.917746 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:33.987251 (+ 69505us) spinlock_profiling.cc:243] Waited 1.16 ms on lock 0x7b18002608c0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d6a2a6 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419
0709 12:22:35.010733 (+1023482us) spinlock_profiling.cc:243] Waited 25.5 ms on lock 0x7b54005f3554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:35.010805 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:35.010916 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":389180,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":397,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":5299,"spinlock_wait_cycles":61240448,"thread_start_us":550177,"threads_started":2,"wal-append.queue_time_us":474975}
I20260709 12:22:35.033234 17270 raft_consensus.cc:1275] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:22:35.033581 17270 pending_rounds.cc:85] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8: Aborting all ops after (but not including) 1
I20260709 12:22:35.033802 17270 pending_rounds.cc:107] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:22:35.034128 17270 raft_consensus.cc:2889] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:34.998610 17966 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:35.001969 17868 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:35.054344 17868 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:35.032932 17556 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:35.062645 18176 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:35.064976 18176 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:35.066584 18126 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:35.016155 18008 raft_consensus.cc:697] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:34.940315 18073 consensus_queue.cc:260] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:35.033145 18188 log.cc:927] Time spent T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
W20260709 12:22:35.040141 18187 log.cc:927] Time spent T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 12:22:35.085083 17799 consensus_queue.cc:1059] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:35.047535 18121 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:35.048738 17966 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:35.049892 18178 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:35.051673 17953 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:35.051816 17925 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:35.084084 18161 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:35.097087 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:35.021260 18047 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:35.022708 18200 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:22:35.104918 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 97660.2us
I20260709 12:22:35.097054 17915 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:35.109040 18161 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:35.106551 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:35.124693 16475 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:22:35.104003 18200 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:35.125957 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:35.126927 16475 raft_consensus.cc:2393] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 2: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:35.127161 18200 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:35.111984 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:35.128700 16514 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:35.112882 16473 raft_consensus.cc:1275] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:35.123214 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:35.115666 16441 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:35.116240 18047 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:35.088069 17650 consensus_queue.cc:1059] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
W20260709 12:22:35.088214 17829 consensus_queue.cc:1175] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ef57beab0e1d40d585983873c55107e8" responder_term: 2 status { last_received { term: 1 index: 1 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:22:35.124843 18116 consensus_meta.cc:287] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.513s	user 0.008s	sys 0.004s
I20260709 12:22:35.115293 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:35.130839 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:35.104012 17642 consensus_queue.cc:1059] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 12:22:35.133013 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:35.132217 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:35.138186 16475 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "95954ea821224e4d962974a83d83ad44"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
mode: GRACEFUL
new_leader_uuid: "610f4feda11942739236f2b121a4b9c1"
 from {username='slave'} at 127.0.0.1:39000
I20260709 12:22:35.138768 16475 raft_consensus.cc:606] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Received request to transfer leadership to TS 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:35.139199 16475 raft_consensus.cc:614] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:35.139319 16477 raft_consensus.cc:1240] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:35.137072 16473 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:22:35.132023 18206 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:35.141040 18206 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:35.141386 18206 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:35.142010 16474 raft_consensus.cc:1275] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:35.142437 18151 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:35.142637 18083 consensus_queue.cc:1059] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:35.135815 18008 consensus_queue.cc:237] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:35.143988 18083 consensus_queue.cc:1048] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:35.135823 18200 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:35.129536 18204 consensus_queue.cc:1059] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 12:22:35.145387 18083 raft_consensus.cc:3055] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:22:35.133574 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 130235us
I20260709 12:22:35.147552 16473 raft_consensus.cc:1275] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:22:35.147893 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:35.149019 16398 raft_consensus.cc:1240] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:35.149386 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:35.150044 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:35.153945 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:35.154700 16473 raft_consensus.cc:2393] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 4: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:35.150663 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:35.150971 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:35.155889 18072 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.156057 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:35.156733 16396 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:35.158044 18151 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:35.158594 18151 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:35.161204 17946 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.161728 17946 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:35.163945 17946 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:35.149757 16366 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:35.165966 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:35.166988 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:35.168463 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:35.169003 16475 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:22:35.171384 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:35.171254 18072 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:35.173254 17868 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:35.146406 18116 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:35.170193 17142 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:35.173369 18072 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:35.177094 16401 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:35.178941 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:35.179215 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:35.179817 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:35.181352 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:35.179767 16468 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:35.181897 16401 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
I20260709 12:22:35.186506 16551 raft_consensus.cc:1240] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:35.190382 18151 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:35.184094 18200 raft_consensus.cc:697] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:35.190557 18204 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:35.183341 17925 consensus_queue.cc:1048] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 12:22:35.191985 18204 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:35.192458 18204 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:35.194794 18209 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:35.173813 17868 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:35.191787 18200 consensus_queue.cc:237] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:35.195953 17868 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:35.197232 18072 consensus_queue.cc:1048] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:35.180586 16471 raft_consensus.cc:2393] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 4: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:35.199506 16471 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a60dbe0067044806b384e90805c877bc"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:39000
I20260709 12:22:35.199574 18206 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:35.200120 16471 raft_consensus.cc:606] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:22:35.200549 16471 raft_consensus.cc:614] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:35.201867 16471 raft_consensus.cc:1240] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:35.205508 16476 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
mode: GRACEFUL
new_leader_uuid: "610f4feda11942739236f2b121a4b9c1"
 from {username='slave'} at 127.0.0.1:39000
I20260709 12:22:35.207131 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:35.208039 16367 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:35.188352 16545 raft_consensus.cc:1275] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:35.200390 18179 raft_consensus.cc:3055] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:35.136590 18001 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:35.165277 18083 raft_consensus.cc:740] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:35.213193 16549 raft_consensus.cc:2368] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 3. Current term is 4.
I20260709 12:22:35.213505 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:35.214788 16396 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6.
I20260709 12:22:35.209916 18208 consensus_queue.cc:1059] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 12:22:35.215639 18083 consensus_queue.cc:260] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:35.216984 18083 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:35.210557 18179 raft_consensus.cc:740] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:35.225137 18179 consensus_queue.cc:260] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:35.226305 18179 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:35.226751 18170 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:35.227931 18170 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:35.228312 18170 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:35.230852 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:35.231647 16551 raft_consensus.cc:2393] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 2: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:35.231519 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:35.232290 16544 raft_consensus.cc:2393] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 3: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:35.232967 18081 raft_consensus.cc:3055] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:35.233569 18081 raft_consensus.cc:740] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:35.234026 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:35.234375 16442 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:35.234877 16545 raft_consensus.cc:2393] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 7: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:35.234624 18081 consensus_queue.cc:260] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:35.235798 18084 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:35.236510 18127 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:35.236960 16442 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:35.237007 18127 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:35.238170 18186 log.cc:927] Time spent T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.481s	user 0.003s	sys 0.001s
I20260709 12:22:35.239400 18180 consensus_queue.cc:1059] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:35.239714 16442 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:35.240443 18164 raft_consensus.cc:3055] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:35.241133 18164 raft_consensus.cc:740] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:35.223827 17868 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:35.242022 18164 consensus_queue.cc:260] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:35.243140 18164 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:35.243840 17868 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:35.240507 18209 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:35.245622 18209 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:35.246048 18209 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:35.246575 17806 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:35.246997 16442 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:35.247864 18180 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:35.248610 18180 raft_consensus.cc:697] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:35.248991 17946 raft_consensus.cc:3055] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:35.249533 17946 raft_consensus.cc:740] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:35.249640 18180 consensus_queue.cc:237] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:35.250640 17946 consensus_queue.cc:260] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:35.246601 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:35.256686 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:35.209050 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:35.189256 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:35.260692 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:35.254465 16442 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:35.267570 16442 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:35.277537 16366 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:35.279781 17270 raft_consensus.cc:2393] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 6: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:35.285393 16476 raft_consensus.cc:606] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Received request to transfer leadership to TS 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:35.286150 16476 raft_consensus.cc:614] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:35.288236 16550 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:35.289350 16442 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:35.290581 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:35.296123 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:35.238225 18084 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:35.296988 16548 raft_consensus.cc:2393] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 7: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:35.280306 18069 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:35.308017 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:35.320529 16475 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:35.288048 18081 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:35.296190 16471 raft_consensus.cc:1275] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:35.290915 18077 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:35.326483 16550 raft_consensus.cc:1275] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:35.295480 18207 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:35.285070 17868 consensus_queue.cc:1048] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 12:22:35.335633 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:35.289958 17642 consensus_queue.cc:1059] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 12:22:35.309216 17946 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:35.324688 17966 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:35.319726 18069 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:35.299487 18178 consensus_queue.cc:1048] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 12:22:35.336412 18206 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:35.342018 16550 raft_consensus.cc:1275] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:35.373049 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:35.373729 16367 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:35.357467 17261 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45230 (request call id 594) took 1265 ms. Trace:
I20260709 12:22:35.378823 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:35.369647 16475 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
I20260709 12:22:35.386173 16399 raft_consensus.cc:1240] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:22:35.371418 16548 raft_consensus.cc:1240] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:35.388556 16367 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:35.366652 18127 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:35.367982 18073 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:35.378616 17261 rpcz_store.cc:276] 0709 12:22:34.092080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:34.092457 (+   377us) service_pool.cc:224] Handling call
0709 12:22:35.357229 (+1264772us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400281754. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:35.357396 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2744687872}
I20260709 12:22:35.345425 17915 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:35.332864 18207 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:35.348208 18060 consensus_queue.cc:260] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:35.338948 18077 raft_consensus.cc:697] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 4 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:35.347931 18069 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:35.353787 17872 raft_consensus.cc:3055] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:35.342795 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:35.338255 18204 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:35.429425 16475 raft_consensus.cc:1240] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:35.393118 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:35.430698 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:35.419857 17144 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:35.349556 18013 raft_consensus.cc:3055] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:35.433462 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:35.420576 16399 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 4. Current term is 5. Ops: []
W20260709 12:22:35.406512 17258 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45230 (request call id 577) took 1838 ms (client timeout 1894 ms). Trace:
W20260709 12:22:35.442816 17258 rpcz_store.cc:269] 0709 12:22:33.568235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:33.568404 (+   169us) service_pool.cc:224] Handling call
0709 12:22:35.406282 (+1837878us) spinlock_profiling.cc:243] Waited 894 ms on lock 0x7b54005f6254. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:35.406438 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2055913216}
I20260709 12:22:35.431816 16401 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:35.445194 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:35.412421 18001 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:35.426287 18002 consensus_queue.cc:1048] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 12:22:35.408392 18207 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:35.415320 17872 raft_consensus.cc:740] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:35.448514 18001 consensus_queue.cc:1059] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:35.448802 17872 consensus_queue.cc:260] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:35.449534 18001 raft_consensus.cc:3055] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:35.449863 17872 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:35.408563 18121 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:35.451344 16473 raft_consensus.cc:1240] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:35.440279 18020 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:35.357601 18197 raft_consensus.cc:3055] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:35.446110 16395 raft_consensus.cc:2393] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 7: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:35.459376 18197 raft_consensus.cc:740] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:35.422902 16396 raft_consensus.cc:1240] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:22:35.460279 18197 consensus_queue.cc:260] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:35.449889 18001 raft_consensus.cc:740] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:35.461724 18197 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:35.462133 18069 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:35.462457 18001 consensus_queue.cc:260] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:35.465420 18001 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:35.465807 18171 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:35.466459 18171 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:35.454885 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:22:35.456432 18214 log.cc:927] Time spent T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:22:35.467495 16399 raft_consensus.cc:2393] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 7: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:35.429620 17915 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:35.468112 17915 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:35.470543 17915 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:35.457685 18127 consensus_queue.cc:1059] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:35.414243 18060 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:35.473474 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:35.458037 18116 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:35.422310 16471 raft_consensus.cc:2393] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 7: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:35.419886 18072 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:35.480656 18072 raft_consensus.cc:697] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 3 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:35.480895 16516 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:35.482271 18204 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:35.481988 18072 consensus_queue.cc:237] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:35.452785 18020 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.483194 18020 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:35.483726 17665 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:22:35.484521 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:35.421355 18066 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.406747 17925 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.458833 18161 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.496695 16472 raft_consensus.cc:1240] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:35.415109 18077 consensus_queue.cc:237] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:35.478704  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:22:35.503202 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:35.504037 16544 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:35.504124  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:35.505254  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:35.441207 18013 raft_consensus.cc:740] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:35.485706 18020 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:35.487978 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:22:35.493001 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:35.494436 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:35.524806 16399 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:35.524909 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:35.493271 16366 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:35.506805 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:35.427526 18083 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:35.521001 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:35.532212 16516 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:35.532752 16401 raft_consensus.cc:2410] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:22:35.525785 16547 raft_consensus.cc:2393] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 4: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:35.525579 16545 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:22:35.535408 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:35.536267 17258 raft_consensus.cc:2376] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5. Re-sending same reply.
I20260709 12:22:35.538852 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:35.541710 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:35.545056 17270 raft_consensus.cc:1240] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:22:35.496924 18121 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:35.548799 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:35.550962 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:35.553967 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:35.557034 17144 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:35.559172 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:35.560043 17263 raft_consensus.cc:2393] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 8: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:35.561393 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:35.562104 17264 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:35.562978 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:35.564621 17269 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb"
dest_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:45224
I20260709 12:22:35.565460 17269 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:35.565990 17269 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:35.548799 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:35.569211 17270 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 3.
I20260709 12:22:35.570535 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:35.572502 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:35.574222 16545 raft_consensus.cc:1275] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:35.575479 16441 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:35.576231 17258 raft_consensus.cc:1240] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:35.578508 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:35.583351 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:35.585207 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:35.586840 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:35.590315 16441 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:35.593107 16441 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:35.602490 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:35.603206 16545 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:35.362622 18209 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:35.422151 18170 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:35.541209 18168 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.546373 18150 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:22:35.613026 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 224551us
I20260709 12:22:35.620515 16367 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:35.543363 18223 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:35.516516 18161 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:35.623140 16545 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:22:35.626621 16544 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:22:35.516894 18066 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:35.516775 17925 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:35.630463 17143 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:35.589277 18008 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:35.641770 16403 raft_consensus.cc:1240] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3. Ops: []
W20260709 12:22:35.639133 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 46795.7us
I20260709 12:22:35.606560 18180 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:35.649843 16441 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:35.651467 16441 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:35.546114 18083 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:35.613121 18116 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:35.619455 17915 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:35.636951 17269 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:35.627707 16395 raft_consensus.cc:1275] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:35.657646 18178 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:35.575559 18121 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:35.632458 18073 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.647537 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:22:35.718806 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 88392.1us
I20260709 12:22:35.657827 18164 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:22:35.653592 18150 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.666021 17794 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.738852 17899 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.739629 17899 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:35.742588 17899 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:35.699766 18206 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:22:35.749820 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:35.622277 18168 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:35.712316 18208 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.725657 17799 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:35.726670 18069 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:35.771162 17258 raft_consensus.cc:1240] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:35.734427 18073 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:35.779320 17264 raft_consensus.cc:2393] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 6: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:35.779768 18073 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:35.782795 18136 raft_consensus.cc:3055] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:35.783316 18136 raft_consensus.cc:740] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:35.786232 18171 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:35.789713 18209 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:35.748066 18150 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:35.792994 18150 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:35.794098 17266 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:35.749603 18179 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.726419 18047 consensus_queue.cc:1059] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 12:22:35.758083 18222 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:35.646564 18013 consensus_queue.cc:260] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:35.744905 17900 consensus_queue.cc:1048] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 12:22:35.748679 17794 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:35.889197 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 266957us
I20260709 12:22:35.989763 16367 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:35.991798 17794 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:35.992678 17794 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.930083 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:35.930549 16476 raft_consensus.cc:1275] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:35.933244 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:35.938874 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:35.939478 16546 raft_consensus.cc:1275] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:35.941062 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:35.746316 17254 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:35.998289 17642 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.957265 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:35.999452 17258 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:35.973534 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:35.993577 17794 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:36.001127 16469 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:36.003553 17794 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:35.976145 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:35.921725 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:36.005245 17271 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:35.999051 17642 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:35.962424 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:36.015841 17799 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.016911 17799 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:36.019320 16547 raft_consensus.cc:1240] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:36.025576 17799 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:36.026535 17799 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:36.027606 17799 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:35.797808 17834 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.034363 17915 consensus_queue.cc:1059] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:36.036114 17263 raft_consensus.cc:1275] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:35.784263 18136 consensus_queue.cc:260] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:35.835041 18161 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:35.834298 18170 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:35.832365 18171 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:35.804062 18179 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:35.757301 18206 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:35.839905 18207 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:35.775954 18208 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:35.832019 18220 log.cc:927] Time spent T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
W20260709 12:22:35.832127 18219 log.cc:927] Time spent T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 12:22:35.809175 18222 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:35.822968 18224 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:36.040099 17861 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.009346 17794 consensus_queue.cc:1048] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
W20260709 12:22:36.095832 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 106392us
I20260709 12:22:36.099380 18244 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:35.817518 18083 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:22:36.110656 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 99309.3us
W20260709 12:22:36.114573 16369 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.009s	sys 0.038s
I20260709 12:22:36.124444 18121 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:36.135460 16469 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:22:36.138484 17271 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:22:36.132115 17834 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:36.145920 17799 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:36.164096 18209 raft_consensus.cc:3055] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:36.171590 18209 raft_consensus.cc:740] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:36.171717 18206 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:36.168694 18208 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:36.168838 17834 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:36.173378 18209 consensus_queue.cc:260] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:36.174708 18222 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:36.177407 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:36.177664 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:36.178516 17258 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:36.178591 17264 raft_consensus.cc:2410] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:22:36.179905 18179 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:36.180495 16441 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:36.181753 17915 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:36.182413 17915 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:36.182840 18207 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:36.182966 17915 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:36.184827 18020 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.185621 18020 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:36.186901 18224 raft_consensus.cc:697] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 4 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:36.188297 18224 consensus_queue.cc:237] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:36.188742 18020 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:36.186012 18171 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:36.183835 16441 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:22:36.164813 16369 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.010s	sys 0.038s
I20260709 12:22:36.155860 18244 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:36.143479 18136 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:36.197887 18209 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:36.202113 18116 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:36.202831 18116 raft_consensus.cc:697] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:36.198899 18244 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:36.183889 18207 consensus_queue.cc:1048] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 12:22:36.213948 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:36.204310 18116 consensus_queue.cc:237] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:36.214838 16548 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:36.215694 16546 raft_consensus.cc:1240] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:36.216130 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:36.206871 18066 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:36.217077 16549 raft_consensus.cc:2393] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 6: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:36.218108 18197 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:36.210485 18171 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:36.218959 18197 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:36.218142 17142 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:36.220592 18204 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:36.210747 17861 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:36.221169 18204 raft_consensus.cc:697] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:36.222502 18204 consensus_queue.cc:237] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:36.223487 17861 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:36.194008 17143 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:36.225399 18166 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:36.225988 18166 raft_consensus.cc:697] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:36.226619 18171 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:22:36.219211 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:36.229907 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:36.231490 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:36.231966 17925 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:36.234176 17642 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:36.232225 17264 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:36.227275 18166 consensus_queue.cc:237] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:36.253316 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 59649.6us
W20260709 12:22:36.256705 16399 consensus_meta.cc:287] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.731s	user 0.014s	sys 0.014s
I20260709 12:22:36.261013 17271 raft_consensus.cc:1275] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:22:36.266968 16441 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.4:44491 timed out after 1.920s (SENT) [suppressed 19 similar messages]
I20260709 12:22:36.267491 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:36.268573 16551 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:22:36.270320 16546 raft_consensus.cc:1275] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:22:36.266449 16398 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.007s	sys 0.010s
W20260709 12:22:36.276309 16398 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.763s	user 0.008s	sys 0.010s
I20260709 12:22:36.272619 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:36.280732 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:36.281606 16547 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:36.282886 18222 consensus_queue.cc:1059] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:36.284490 18222 raft_consensus.cc:3055] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:36.284994 18222 raft_consensus.cc:740] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:36.286284 18222 consensus_queue.cc:260] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:36.291632 16399 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:36.288223 18222 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:36.302744 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:36.303397 16367 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:36.309762 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:22:36.310421 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:36.313303 18047 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:36.314102 18047 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:36.314790 16441 leader_election.cc:400] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 3. Current term is 4.
I20260709 12:22:36.302841 18197 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:36.316928 18197 consensus_queue.cc:1059] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:36.316685 16441 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:22:36.318683 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:36.316272 17915 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:36.327561 18245 log.cc:927] Time spent T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.160s	user 0.002s	sys 0.003s
I20260709 12:22:36.328459 18235 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:36.328871 17271 raft_consensus.cc:2393] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 6: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:36.329209 18235 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:36.329972 17872 consensus_meta.cc:287] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.879s	user 0.005s	sys 0.016s
I20260709 12:22:36.330821 17266 raft_consensus.cc:1275] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:36.346297 17915 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:36.352324 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:36.352875 17665 consensus_queue.cc:1048] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 12:22:36.353461 16402 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:36.355556 18116 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:36.356899 18231 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.357963 18231 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:36.337761 17925 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:36.361001 17925 raft_consensus.cc:697] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 3 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:36.361660 17269 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:36.362378 17925 consensus_queue.cc:237] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:36.364454 18223 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.005s	sys 0.025s
W20260709 12:22:36.364856 18223 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.005s	sys 0.026s
I20260709 12:22:36.365141 18223 raft_consensus.cc:697] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 7 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
W20260709 12:22:36.365855 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 93354.4us
I20260709 12:22:36.366468 18223 consensus_queue.cc:237] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:36.369975 18240 log.cc:927] Time spent T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.201s	user 0.000s	sys 0.002s
W20260709 12:22:36.375442 18001 consensus_meta.cc:287] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.909s	user 0.020s	sys 0.001s
I20260709 12:22:36.376183 16548 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
W20260709 12:22:36.376842 18060 consensus_meta.cc:287] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.904s	user 0.003s	sys 0.019s
I20260709 12:22:36.377776 16550 raft_consensus.cc:2410] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:22:36.377985 18235 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:36.380748 17556 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.381338 17556 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:36.386667 17556 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:22:36.380739 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 73201.9us
I20260709 12:22:36.357098 16516 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:36.367403 16367 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:36.398834 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:36.412642 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:36.413417 16399 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:36.408975 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:36.412050 18170 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:36.421711 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:36.426494 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:36.427385 16395 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:36.426600 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:36.428694 16397 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:22:36.428789 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:36.428941 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:36.429812 16403 raft_consensus.cc:2410] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:22:36.433972 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:36.439245 17868 raft_consensus.cc:3055] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:36.439776 17868 raft_consensus.cc:740] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:36.443235 18013 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:36.444129 17270 raft_consensus.cc:1275] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:36.450111 16402 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
W20260709 12:22:36.442991 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 126514us
I20260709 12:22:36.460071 18231 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:36.462221 18116 consensus_queue.cc:1048] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:36.464684 18116 consensus_queue.cc:1059] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:36.465875 17915 raft_consensus.cc:3055] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:36.466349 17915 raft_consensus.cc:740] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:36.468729 17143 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:22:36.469776 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 58289.6us
I20260709 12:22:36.467619 17915 consensus_queue.cc:260] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:36.478767 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:36.480573 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:36.482225 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:36.483333 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:36.483877 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:36.484622 16473 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:36.486101 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:36.486956 16469 raft_consensus.cc:2393] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 2: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:36.490891 16477 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:22:36.411654 18047 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:36.488587 17143 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:36.499034 16402 raft_consensus.cc:1240] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 3 LEADER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:36.491937 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:36.502571 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:36.493147 18060 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:36.482985 16395 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:22:36.515930 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:36.524652 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:36.519279 17143 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:36.537015 16398 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 1.
I20260709 12:22:36.519451 17966 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:36.545405 17806 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:36.546062 18002 consensus_queue.cc:1048] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 12:22:36.545948 17806 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:36.547103 17806 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:36.549016 17780 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:36.549577 17780 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:36.549988 17780 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:36.554548 18172 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:36.555157 18172 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:36.555596 18172 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:36.536224 17868 consensus_queue.cc:260] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:36.558060 17868 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:36.545148 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 188365us
I20260709 12:22:36.545930 17966 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:36.552448 18019 consensus_queue.cc:1059] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.061s
I20260709 12:22:36.542547 17915 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:36.541923 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:22:36.582033 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 214902us
I20260709 12:22:36.586319 18047 consensus_queue.cc:1048] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:36.588168 16367 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:36.589602 18170 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:36.590148 18170 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:36.590603 18170 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:36.598112 16367 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:36.603829 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:22:36.610639 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:36.611549 17969 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:36.612612 17969 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:36.613756 16471 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:36.614455 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:36.615264 17969 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:22:36.611583 17261 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.987s	user 0.045s	sys 0.042s
W20260709 12:22:36.621559 17261 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.071s	user 0.046s	sys 0.042s
I20260709 12:22:36.627818 16468 raft_consensus.cc:2393] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 5: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:36.655956 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:36.661561 17144 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:36.675209 16515 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:36.673755 16476 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 3.
I20260709 12:22:36.685940 18077 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:36.692526 18020 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:36.692912 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:36.693199 18020 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:36.693773 17270 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:36.695408 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:36.695647 18020 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:36.698295 16516 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
W20260709 12:22:36.689826 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:36.699848 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 161539us
I20260709 12:22:36.666369 17872 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.708702 18001 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:36.709450 18001 raft_consensus.cc:697] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:36.712033 16471 raft_consensus.cc:1240] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 4. Current term is 5. Ops: []
W20260709 12:22:36.718828 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 65813.2us
I20260709 12:22:36.721449 17966 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:36.721665 16516 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:36.723421 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 725567us
I20260709 12:22:36.727098 18136 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:36.727818 18136 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:36.728302 18136 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:36.727103 18153 raft_consensus.cc:3055] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:36.732987 18153 raft_consensus.cc:740] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:22:36.734258 18256 log.cc:927] Time spent T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:22:36.736436 18066 consensus_queue.cc:1048] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:36.737214 16515 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:36.738206 18168 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:22:36.744588 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 146633us
I20260709 12:22:36.747134 16367 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:36.746887 18170 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:36.710702 18001 consensus_queue.cc:237] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:36.748554 17872 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:36.751080 17872 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:36.649106 18244 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:36.752209 16394 raft_consensus.cc:1240] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:36.753347 18244 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:36.754144 18244 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:36.755515 17143 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:36.755890 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:36.757418 16516 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:36.755918 17266 raft_consensus.cc:1275] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:36.702791 17969 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:36.782689 16475 raft_consensus.cc:1240] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:36.784827 16516 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:36.779722 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:36.787147 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:36.789219 16469 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
W20260709 12:22:36.780748 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 287624us
I20260709 12:22:36.782460 17969 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:36.773222 17665 consensus_queue.cc:1048] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 12:22:36.793215 17969 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:36.795629 17261 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 605) took 1257 ms. Trace:
I20260709 12:22:36.796190 17261 rpcz_store.cc:276] 0709 12:22:35.538281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:35.538392 (+   111us) service_pool.cc:224] Handling call
0709 12:22:35.538694 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:35.546889 (+  8195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:35.546928 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:35.550328 (+  3400us) log.cc:844] Serialized 26 byte log entry
0709 12:22:36.331324 (+780996us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2872d7f0ac 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b
0709 12:22:36.673420 (+342096us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:36.683436 (+ 10016us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:36.758531 (+ 75095us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:36.795125 (+ 36594us) raft_consensus.cc:1685] finished
0709 12:22:36.795153 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:36.795541 (+   388us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47890,"raft.queue_time_us":12599,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":31188,"spinlock_wait_cycles":346271104,"thread_start_us":1071369,"threads_started":1,"wal-append.queue_time_us":1125349}
I20260709 12:22:36.798623 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:36.798918 18200 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:36.802580 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:36.803783 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:36.806962 16469 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:36.814340 17925 consensus_queue.cc:1048] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:36.816062 17925 consensus_queue.cc:1048] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:36.817408 17806 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:36.818550 17969 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:36.769340 18206 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:36.821365 17969 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:36.768942 18116 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.824795 18116 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:36.790369 18020 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:36.828972 18020 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:36.829562 18020 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:36.831859 16473 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:36.734277 18153 consensus_queue.cc:260] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:36.786051 16544 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:36.772672 17872 raft_consensus.cc:2804] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:22:36.845008 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:22:36.853895 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:36.884007 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 108332us
I20260709 12:22:36.835640 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:36.799746 18200 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:36.832137 18253 log.cc:927] Time spent T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.204s	user 0.002s	sys 0.000s
W20260709 12:22:36.902134 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 379257us
I20260709 12:22:36.847770 18153 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:36.848553 18179 raft_consensus.cc:3055] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:36.903144 17142 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:36.887566 18066 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:36.857810 17966 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.917762 17144 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:36.890177 17872 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:36.804033 17780 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:36.892022 17861 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.932286 16468 raft_consensus.cc:1240] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:36.923754 17142 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:36.900101 18002 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.936152 16366 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:36.910454 17806 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:36.920753 18116 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:36.965119 18081 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.965992 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:36.966091 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:36.966980 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:36.918628 18179 raft_consensus.cc:740] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:36.970793 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:36.972558 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:36.957028 18002 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:36.950138 17966 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:36.916211 16225 catalog_manager.cc:5697] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 3 to 4, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 4 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:36.981482 16474 raft_consensus.cc:1275] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:22:37.011814 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 53535.8us
I20260709 12:22:36.986392 17780 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:36.946682 18204 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:22:36.988937 18013 consensus_meta.cc:287] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.545s	user 0.016s	sys 0.000s
I20260709 12:22:36.990370 18020 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:36.934713 18197 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:36.940361 17872 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:22:36.992192 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 43846.4us
W20260709 12:22:36.998903 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 62819.4us
W20260709 12:22:37.032945 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:36.940623 18235 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.996068 17143 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:36.947050 18209 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.010689 18019 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.014869 17142 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:37.028808 16547 raft_consensus.cc:2393] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 4: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:36.997870 17799 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:37.029106 18172 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:36.946306 18206 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:37.038475 18020 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:37.044076 17144 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:37.045501 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 49545.4us
I20260709 12:22:37.035252 18204 raft_consensus.cc:697] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:37.029227 18231 raft_consensus.cc:2804] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:37.055284 17266 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:36.958961 17861 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:37.021795 16544 raft_consensus.cc:2393] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 3: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:37.066196 17269 raft_consensus.cc:1240] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:37.044795 17556 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:37.045213 18170 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:37.025040 18081 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:37.054486 18197 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:37.019513 18116 consensus_queue.cc:1048] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 12:22:37.071201 16397 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:22:37.068222 18166 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:37.151687 17270 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:22:37.153612 17263 raft_consensus.cc:1240] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:22:37.062744 18231 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:37.156646 18231 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:37.160167 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:37.068475 18235 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:37.047755 18019 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:37.095543 17556 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:37.097357 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:37.185348 16468 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:37.098115 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:37.098562 16477 raft_consensus.cc:1275] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:22:37.099560 18170 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:37.099896 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:37.100382 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:37.101058 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:37.101410 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:37.102178 16399 raft_consensus.cc:1240] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:37.103070 16367 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:37.105127 18081 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:37.077241 18197 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:37.106850 18002 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:37.109222 18008 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:37.109835 18247 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.129128 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:37.129817 18172 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:37.092897 18223 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:37.136019 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:37.099896 16403 raft_consensus.cc:1240] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:37.138033 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:37.240679 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:37.139757 18200 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:37.140260 18207 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.111670 18204 consensus_queue.cc:237] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:37.140843 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:37.141637 18116 consensus_queue.cc:1243] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer ef57beab0e1d40d585983873c55107e8 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:22:37.143344 17966 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:22:37.146306 18255 log.cc:927] Time spent T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.483s	user 0.000s	sys 0.008s
I20260709 12:22:37.148578 17915 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:37.149097 17144 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:37.149848 17861 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:37.236021 18008 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:37.150656 17868 consensus_meta.cc:287] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.592s	user 0.010s	sys 0.005s
W20260709 12:22:37.262463 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 159493us
I20260709 12:22:37.150960 17665 consensus_queue.cc:1048] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:37.143569 18179 consensus_queue.cc:260] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:37.161711 17270 raft_consensus.cc:2368] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 3. Current term is 4.
I20260709 12:22:37.288607 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:37.290143 17270 raft_consensus.cc:1240] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:37.094269 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:37.191174 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:37.068619 18209 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:37.234925 16475 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:37.235373 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:37.237663 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:22:37.239990 18222 consensus_meta.cc:287] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.921s	user 0.014s	sys 0.000s
I20260709 12:22:37.292152 18223 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:37.295379 16396 raft_consensus.cc:1275] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:22:37.296154 16396 pending_rounds.cc:85] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 2
I20260709 12:22:37.296430 16396 pending_rounds.cc:107] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:22:37.296705 16396 raft_consensus.cc:2889] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:37.293850 16397 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:37.237594 17254 raft_consensus.cc:2393] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 3: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:37.251402 16474 raft_consensus.cc:2393] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 5: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:37.223424 18197 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:37.244764 17799 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:37.253821 17915 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:37.256170 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:37.150108 18013 consensus_queue.cc:1059] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:37.259846 16399 raft_consensus.cc:3055] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:22:37.265980 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 116939us
I20260709 12:22:37.259655 16394 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:22:37.234195 17834 consensus_queue.cc:1048] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:22:37.266877 18019 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:37.267369 18171 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:37.192051 18206 consensus_queue.cc:1059] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 12:22:37.262658 18008 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:37.290961 18207 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:37.303938 17799 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:37.303567 17264 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:37.242563 18170 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:37.306597 17665 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.306711 18179 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:37.308457 17271 raft_consensus.cc:1275] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:22:37.308449 16399 raft_consensus.cc:740] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:37.246938 18224 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:22:37.313598 16550 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.012s	sys 0.026s
W20260709 12:22:37.341887 16550 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.012s	sys 0.026s
I20260709 12:22:37.303941 18235 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:37.258499 18200 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.329841 17263 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:37.330106 16403 raft_consensus.cc:1240] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:37.246662 18247 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:37.331368 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:37.346544 16367 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:37.353365 16402 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:22:37.354341 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:37.354909 16402 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:22:37.306622 17915 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:37.330037 18136 consensus_meta.cc:287] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.601s	user 0.011s	sys 0.006s
I20260709 12:22:37.336970 17271 pending_rounds.cc:85] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8: Aborting all ops after (but not including) 1
I20260709 12:22:37.361054 16515 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:22:37.330595 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:37.361294 17271 pending_rounds.cc:107] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8: Aborting uncommitted NO_OP operation due to leader change: 2.2
W20260709 12:22:37.332763 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:37.362010 17271 raft_consensus.cc:2889] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:37.361945 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:37.341212 18171 raft_consensus.cc:697] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:37.351415 16399 consensus_queue.cc:260] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:37.380227 16399 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:37.381260 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:37.360430 17861 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.368008 18060 raft_consensus.cc:3055] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:37.363399 18235 raft_consensus.cc:2764] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 12:22:37.376878 18244 consensus_meta.cc:287] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.622s	user 0.017s	sys 0.000s
I20260709 12:22:37.405189 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:22:37.400314 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 615525us
I20260709 12:22:37.406167 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:37.354527 17872 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:37.398442 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:37.349527 17665 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:37.376166 18206 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:37.408541 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:37.416175 17270 pending_rounds.cc:85] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8: Aborting all ops after (but not including) 1
I20260709 12:22:37.419791 17270 pending_rounds.cc:107] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:22:37.420136 17270 raft_consensus.cc:2889] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:37.416150 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:37.411926 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:37.355875 18200 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:37.426885 16403 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:37.383749 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:22:37.430259 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 358274us
I20260709 12:22:37.421211 16549 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:37.429369 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:37.368834 18136 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:37.387640 17915 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.397790 18081 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.402529 18060 raft_consensus.cc:740] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:37.435591 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:37.444700 16403 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
W20260709 12:22:37.436501 16472 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.009s	sys 0.035s
W20260709 12:22:37.445914 16472 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.009s	sys 0.035s
I20260709 12:22:37.397630 18066 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:37.409065 18209 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:37.397156 17806 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:37.422943 16441 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:37.417944 18244 raft_consensus.cc:2749] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:22:37.417642 18223 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:37.450103 16547 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:22:37.454298 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:37.427119 18206 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:37.436453 17794 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.433236 18116 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.492228 18116 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:37.495922 18116 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:37.436400 18161 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:37.496939 18161 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:37.499547 18161 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:37.500805 18170 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:37.501636 17806 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:37.502564 17665 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:37.450170 18224 consensus_queue.cc:1048] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 12:22:37.503981 17806 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:37.472851 18081 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:37.480867 16394 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:37.481289 17871 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:37.485031 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:37.486032 17872 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:37.470371 17799 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:37.489317 18207 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:37.490284 18066 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:37.491321 18171 consensus_queue.cc:237] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:37.540289 18060 consensus_queue.cc:260] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:37.504704 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:37.506392 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:22:37.476275 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 143430us
I20260709 12:22:37.519255 18161 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:37.514758 17794 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:37.551491 17794 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:37.508716 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 85916.9us
I20260709 12:22:37.552317 18153 raft_consensus.cc:3055] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:37.546355 16367 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:37.552961 18153 raft_consensus.cc:740] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:37.553968 18153 consensus_queue.cc:260] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:37.554313 18224 consensus_queue.cc:1048] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:37.554637 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:37.555385 16543 raft_consensus.cc:2393] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 4: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:37.555851 18224 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:37.556402 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:37.557706 18222 consensus_queue.cc:1059] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:37.558554 18047 raft_consensus.cc:3055] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:37.558907 18047 raft_consensus.cc:740] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:37.559001 16367 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:37.559185 16546 raft_consensus.cc:1275] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:37.559863 18222 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:37.560420 18222 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:37.560962 18222 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:37.562393 16514 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:37.548534 17871 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:37.566206 16367 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:37.566861 17871 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:37.567549 17925 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:37.568109 17925 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:37.568403 17969 raft_consensus.cc:2804] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:37.568696 17925 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:37.569000 17969 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:37.569555 17969 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:37.429450 17861 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:37.573793 17861 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:37.454686 17915 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:37.577589 17915 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:37.577715 17556 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:37.552347 16441 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:37.527487 16395 raft_consensus.cc:1275] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:37.577589 18081 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:37.580556 18116 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:37.580271 16441 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:37.581334 18116 raft_consensus.cc:697] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:37.559863 18047 consensus_queue.cc:260] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:37.584882 18271 raft_consensus.cc:2749] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:37.585268 18247 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:37.585850 18047 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:37.586426 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:37.476255 18206 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:37.527454 18244 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:22:37.528760 18264 log.cc:927] Time spent T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.420s	user 0.005s	sys 0.000s
I20260709 12:22:37.534576 18200 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:37.536497 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:37.602051 16397 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
W20260709 12:22:37.537499 18268 log.cc:927] Time spent T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:22:37.544572 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:37.556479 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:37.556413 18224 raft_consensus.cc:697] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 5 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:37.526427 16475 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:22:37.540064 18066 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:37.508098 16402 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:37.551180 16400 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6.
I20260709 12:22:37.556573 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:37.614102 16549 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:37.612933 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:37.521010 16398 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
I20260709 12:22:37.583319 18116 consensus_queue.cc:237] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:37.618396 17144 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:37.600088 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:37.546931 18161 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:37.523926 18001 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:37.604956 16514 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:37.607862 16367 leader_election.cc:400] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 3. Current term is 4.
I20260709 12:22:37.625257 16367 leader_election.cc:403] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:37.610255 16399 raft_consensus.cc:1275] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:22:37.613556 16474 raft_consensus.cc:1275] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:22:37.600193 16468 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:37.626744 16474 pending_rounds.cc:85] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 1
I20260709 12:22:37.627080 16474 pending_rounds.cc:107] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:22:37.627334 16474 raft_consensus.cc:2889] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:37.564044 18197 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:37.622797 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:37.628832 17269 raft_consensus.cc:2393] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 4: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:37.631659 16399 pending_rounds.cc:85] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 2
I20260709 12:22:37.631551 17144 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:37.615722 17142 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:37.629833 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:37.633272 17142 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:37.635380 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:37.633638 17258 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:37.637753 16367 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:37.632210 16399 pending_rounds.cc:107] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:22:37.638772 16399 raft_consensus.cc:2889] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:37.636281 17263 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:37.646121 16476 raft_consensus.cc:1275] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:22:37.646418 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:37.616503 17780 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.652181 16402 raft_consensus.cc:1240] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:37.653373 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:37.654440 16367 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:37.638130 17144 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:37.649214 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 46544us
I20260709 12:22:37.544248 18008 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:37.619637 18200 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.652549 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:37.600721 18244 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:37.630046 18161 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:37.660317 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:37.655260 16441 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:37.642797 18168 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:37.640746 18020 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:37.639302 17861 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.663756 17264 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:37.681639 17144 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:37.682799 16441 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:37.685424 16544 raft_consensus.cc:1275] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:22:37.653376 18223 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:37.686393 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:37.620848 18247 consensus_queue.cc:1059] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 12:22:37.649602 18235 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 3. Current term is 4.
I20260709 12:22:37.680555 17665 raft_consensus.cc:2804] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:37.656054 17834 consensus_queue.cc:1048] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.068s
W20260709 12:22:37.734222 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 52689.1us
I20260709 12:22:37.687347 17915 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:37.648264 18001 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:37.738848 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 56125.1us
I20260709 12:22:37.739198 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:37.732554 16547 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:37.687187 18168 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:37.681492 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:37.763253 18168 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:37.745388 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 91701us
I20260709 12:22:37.751505 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:37.725981 18019 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:37.687045 18166 raft_consensus.cc:2804] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:37.760138 18274 consensus_queue.cc:1059] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.097s
I20260709 12:22:37.760569 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:37.761008 17871 consensus_queue.cc:1059] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:37.762247 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:37.751672 16474 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:22:37.661494 17780 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:37.769392 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:37.738642 17665 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:37.745782 17915 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:37.777685 17258 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
W20260709 12:22:37.775035 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 120696us
I20260709 12:22:37.727095 18200 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:37.738251 18197 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:37.768905 18008 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:37.769037 17261 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:37.764447 18284 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:37.753480 18207 raft_consensus.cc:2804] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:37.764295 16516 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:37.804179 18273 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.791664 16441 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:37.807073 17665 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:37.808609 17915 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:37.810252 17266 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 3.
I20260709 12:22:37.794569 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:37.836057 16551 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:37.764459 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:37.778870 18166 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:37.744737 17872 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:37.815026 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:37.789814 18136 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:37.741501 17861 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:37.819402 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:37.823745 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 59875.4us
I20260709 12:22:37.748324 18224 consensus_queue.cc:237] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:37.825618 18286 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:37.800678 16543 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
W20260709 12:22:37.820436 18231 consensus_meta.cc:287] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.663s	user 0.010s	sys 0.005s
I20260709 12:22:37.854554 18223 consensus_queue.cc:1048] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:22:37.882099 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 113069us
I20260709 12:22:37.855818 18292 consensus_queue.cc:1048] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:37.809135 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:37.820169 18207 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:37.872891 16516 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:37.826221 16441 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:37.876894 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:37.835326 18209 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:37.852594 18166 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:37.847842 18284 raft_consensus.cc:697] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:37.861057 16396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 313) took 1523 ms. Trace:
I20260709 12:22:37.887822 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:37.895798 16516 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:37.892953 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 66833.3us
I20260709 12:22:37.892773 16548 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:37.890311 16551 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:37.897652 16441 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:37.901041 17780 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:22:37.904601 18282 log.cc:927] Time spent T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
I20260709 12:22:37.870179 16470 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 3.
I20260709 12:22:37.905290 18001 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:37.904997 18284 consensus_queue.cc:237] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:37.905998 18244 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:37.873625 18168 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:37.906847 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:37.907485 16398 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:37.874835 17872 raft_consensus.cc:697] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:37.909265 17872 consensus_queue.cc:237] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:37.886128 18231 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:37.911929 18231 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:37.912700 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:37.913306 17969 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:37.914927 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:37.912700 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:37.919790 16516 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:37.908627 18272 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:37.938359 16401 raft_consensus.cc:2393] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 4: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:37.933843 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:37.918027 17264 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:37.942096 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:37.943068 17264 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:22:37.935127 16398 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:37.933398 16544 raft_consensus.cc:2393] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 4: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:37.891927 18207 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:37.838207 18273 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:37.948458 17144 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:37.951685 16367 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:37.955276 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:37.888406 18020 raft_consensus.cc:697] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:37.912761 18267 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.934230 18001 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:37.968675 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:37.934528 17969 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:37.936728 18136 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:37.985724 18136 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:37.986187 18136 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:37.986474 16547 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
W20260709 12:22:37.991744 18285 log.cc:927] Time spent T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 12:22:37.997890 16546 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:38.004225 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:38.004980 16546 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 2.
I20260709 12:22:38.005964 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:37.944995 18272 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:38.007068 18272 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:37.968675 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:37.976847 18020 consensus_queue.cc:237] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:38.011847 16398 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:37.978184 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:38.012521 18168 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:38.012902 17271 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:38.013347 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:22:37.980477 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 74456.2us
I20260709 12:22:37.982661 18273 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:37.984692 16551 raft_consensus.cc:2368] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 3. Current term is 4.
I20260709 12:22:37.935484 18247 consensus_queue.cc:1048] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 12:22:37.967530 17861 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:37.976976 16367 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:37.978817 18222 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:38.014676 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:38.013653 17144 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:38.024097 16549 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:22:38.019035 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:38.026928 16442 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
W20260709 12:22:38.068780 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 55196.5us
I20260709 12:22:38.000607 18209 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:37.972115 18231 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:38.074291 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:38.024561 18296 raft_consensus.cc:3055] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:37.997148 18008 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:38.014030 18286 raft_consensus.cc:2764] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
W20260709 12:22:38.082291 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 184731us
I20260709 12:22:38.016305 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:38.007743 18136 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:38.071861 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 45025.4us
I20260709 12:22:38.085052 16441 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:38.086390 16442 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:38.016188 16398 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:38.076488 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:38.076910 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 100018us
I20260709 12:22:38.088181 16549 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
I20260709 12:22:38.088586 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:38.088665 16367 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:38.089856 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:37.917047 16396 rpcz_store.cc:276] 0709 12:22:36.311773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:36.311981 (+   208us) service_pool.cc:224] Handling call
0709 12:22:36.312448 (+   467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:37.297269 (+984821us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:37.810039 (+512770us) spinlock_profiling.cc:243] Waited 981 ms on lock 0x7b54009bdcd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:37.835038 (+ 24999us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2257343232}
I20260709 12:22:37.969547 17925 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:38.092042 16367 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:38.093441 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:38.093746 18223 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:38.093935 16367 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:38.094214 18223 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:38.074436 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:38.094772 18223 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:38.094965 18244 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:38.095053 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:38.095649 18244 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:38.095737 16397 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:38.096086 18244 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:38.077991 16402 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:38.097389 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:38.098479 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:38.099685 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:38.022995 18200 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:38.101429 18200 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:38.101785 18200 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:38.102095 18200 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:38.075350 16551 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:38.084163 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:38.103741 18223 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:38.104851 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:38.104969 16551 raft_consensus.cc:1240] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:22:37.995086 18267 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:38.105937 18223 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:38.107434 16551 raft_consensus.cc:1240] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:22:38.105962 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:38.096213 16396 raft_consensus.cc:2393] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 4: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:38.127959 16400 raft_consensus.cc:2393] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 4: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:38.100282 16402 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:38.095427 16403 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:38.138015 16398 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:22:38.077889 18292 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:38.072284 18209 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:38.111939 16394 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:38.078413 18296 raft_consensus.cc:740] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:38.080890 18116 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:38.094694 17925 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:38.141360 17144 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:38.150566 18267 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:38.151273 18267 raft_consensus.cc:2764] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260709 12:22:38.151593 18222 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:38.146405 18020 consensus_queue.cc:1048] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 12:22:38.109022 18278 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:38.128069 18008 raft_consensus.cc:697] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:38.168457 18290 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:38.167830 18008 consensus_queue.cc:237] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:38.171002 18290 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:38.171492 18290 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:38.103662 18244 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:38.162196 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:22:38.152475 18303 log.cc:927] Time spent T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:22:38.162744 18298 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:38.162849 18060 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:38.163160 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:38.152359 18295 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:38.148200 18209 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:38.203876 18298 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:38.210747 18298 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:38.211023 16546 raft_consensus.cc:1275] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:22:38.210814 18206 consensus_meta.cc:287] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.609s	user 0.008s	sys 0.000s
I20260709 12:22:38.153864 18116 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:38.212540 18206 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:38.213060 17915 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:38.212700 18116 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:38.215160 17915 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:38.161315 16547 raft_consensus.cc:1275] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:38.205829 18207 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:38.222038 18207 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:38.216161 18047 consensus_meta.cc:287] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.630s	user 0.007s	sys 0.006s
I20260709 12:22:38.157277 17872 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:38.182380 16398 raft_consensus.cc:1275] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:22:38.231482 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 146567us
I20260709 12:22:38.232583 16547 raft_consensus.cc:1275] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:22:38.232849 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 146569us
I20260709 12:22:38.212623 18295 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:38.237373 16400 raft_consensus.cc:1275] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:22:38.238098 16545 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:38.223593 18136 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:38.237735 16400 pending_rounds.cc:85] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 2
I20260709 12:22:38.252092 16442 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:38.253062 16400 pending_rounds.cc:107] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:22:38.253479 16400 raft_consensus.cc:2889] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:38.184214 18166 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:38.235417 17872 raft_consensus.cc:697] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:38.224615 18200 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:38.261447 18166 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:38.261835 18200 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:38.262869 18267 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:38.263439 18267 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:38.264642 17261 raft_consensus.cc:2368] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 4. Current term is 5.
I20260709 12:22:38.266333 18244 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:38.267271 18206 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:38.240710 17915 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:38.268812 16545 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 3.
I20260709 12:22:38.238783 18295 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:38.273615 18271 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:38.273972 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:38.257594 18306 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:38.236210 18207 consensus_queue.cc:1048] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
W20260709 12:22:38.278281 18161 consensus_meta.cc:287] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.588s	user 0.007s	sys 0.004s
I20260709 12:22:38.280649 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:38.285012 16441 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:38.286760 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:38.287827 16469 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:38.288951 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:38.289649 16470 raft_consensus.cc:2393] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 5: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:38.286217 18306 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:38.294106 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:38.294786 16402 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:38.295745 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:38.297652 16469 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:38.299712 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:38.300853 17665 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:38.302793 18270 log.cc:927] Time spent T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.774s	user 0.002s	sys 0.002s
I20260709 12:22:38.304855 16549 raft_consensus.cc:1240] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:38.304678 18161 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:38.306847 18313 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:38.289649 16475 raft_consensus.cc:2368] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 4. Current term is 5.
I20260709 12:22:38.265777 18267 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:38.306922 16477 raft_consensus.cc:2368] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 4. Current term is 5.
I20260709 12:22:38.296255 16402 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:38.317065 16442 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:38.319984 16550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 237) took 1684 ms. Trace:
I20260709 12:22:38.278900 17915 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:38.307590 18313 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:38.179167 18296 consensus_queue.cc:260] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:38.330574 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:38.259429 18272 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:38.319963 16396 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:38.334148 18161 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:38.334352 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:38.335074 18306 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:38.336015 16367 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:38.338863 17915 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:38.354367 16367 leader_election.cc:400] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 3. Current term is 4.
I20260709 12:22:38.340159 17872 consensus_queue.cc:237] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:38.343147 18313 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:38.343310 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:38.320732 16550 rpcz_store.cc:276] 0709 12:22:36.626030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:36.630417 (+  4387us) service_pool.cc:224] Handling call
0709 12:22:36.658188 (+ 27771us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:36.662756 (+  4568us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:36.662798 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:36.779843 (+117045us) log.cc:844] Serialized 52 byte log entry
0709 12:22:37.342329 (+562486us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:37.616863 (+274534us) log.cc:844] Serialized 14 byte log entry
0709 12:22:37.618072 (+  1209us) log.cc:844] Serialized 14 byte log entry
0709 12:22:37.618787 (+   715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:37.618998 (+   211us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:38.023609 (+404611us) spinlock_profiling.cc:243] Waited 6.72 ms on lock 0x7b48004bffd8. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287adb2aa1 00007f287ada9b37 00007f287ada99ca 00007f287aea126f 00007f287aea0f37 00007f287aeb555c 00007f287aeb5508 00007f287aeb5488 00007f287aeb5443 00007f287aeb46ec 00007f2879ddea96
0709 12:22:38.275301 (+251692us) spinlock_profiling.cc:243] Waited 5.66 ms on lock 0x7b18004951c0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d6a2a6 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287adb2aa1 00007f287ada9b37 00007f287ada99ca 00007f287aea126f
0709 12:22:38.275751 (+   450us) spinlock_profiling.cc:243] Waited 6.63 ms on lock 0x7b48004bffd8. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287adb2aa1 00007f287ada9b37 00007f287ada99ca 00007f287aea126f 00007f287aea0f37 00007f287aeb555c 00007f287aeb5508 00007f287aeb5488 00007f287aeb5443 00007f287aeb46ec 00007f2879ddea96
0709 12:22:38.301796 (+ 26045us) spinlock_profiling.cc:243] Waited 62.6 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:22:38.310084 (+  8288us) raft_consensus.cc:1685] finished
0709 12:22:38.310121 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:38.310313 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":146714,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":6574,"raft.queue_time_us":31851,"raft.run_cpu_time_us":6903,"raft.run_wall_time_us":1044885,"spinlock_wait_cycles":187745408,"thread_start_us":795528,"threads_started":2,"wal-append.queue_time_us":695865}
I20260709 12:22:38.355382 16367 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:38.348395 18312 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:38.347781 18267 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:38.357920 16516 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:38.345038 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:38.344609 17665 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:22:38.467252 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 109397us
W20260709 12:22:38.435849 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 118868us
I20260709 12:22:38.467409 18081 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:38.424649 16543 raft_consensus.cc:2368] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 3. Current term is 4.
I20260709 12:22:38.468108 18081 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:38.419950 18271 raft_consensus.cc:697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 8 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:38.468802 18081 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:38.391810 18231 consensus_queue.cc:1048] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.078s
I20260709 12:22:38.469918 18271 consensus_queue.cc:237] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:38.392138 18223 consensus_queue.cc:1048] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.080s
I20260709 12:22:38.364725 18244 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:38.372381 18306 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:38.468820 18267 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:38.473613 18306 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:38.474110 18267 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:38.476218 18306 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:38.476848 18312 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:38.468220 16516 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:38.477324 18312 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:38.478883 18081 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:38.422430 18047 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:38.391942 18222 consensus_queue.cc:1048] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.089s
I20260709 12:22:38.480944 18047 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:38.481475 18047 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:38.351079 18207 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:38.482522 18207 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:38.483119 16551 raft_consensus.cc:2410] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 12:22:38.430994 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 150621us
I20260709 12:22:38.364058 16402 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:22:38.392311 16367 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:38.484915 18001 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:22:38.485239 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 93067.4us
I20260709 12:22:38.422585 18247 consensus_queue.cc:1059] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 12:22:38.485532 17144 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:38.485656 18001 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:38.486132 18326 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:38.348826 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:38.486392 18001 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:38.486881 18326 raft_consensus.cc:697] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 3 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:38.488191 18326 consensus_queue.cc:237] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:38.349099 18278 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:38.492295 18278 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:38.345458 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:38.494532 16395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 730) took 1398 ms. Trace:
I20260709 12:22:38.497769 17834 raft_consensus.cc:2804] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:38.498416 17834 raft_consensus.cc:697] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:38.499377 17834 consensus_queue.cc:237] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:38.495028 16395 rpcz_store.cc:276] 0709 12:22:37.096384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:37.096521 (+   137us) service_pool.cc:224] Handling call
0709 12:22:37.096975 (+   454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:37.579030 (+482055us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:38.424435 (+845405us) spinlock_profiling.cc:243] Waited 423 ms on lock 0x7b54008a6bd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:38.494459 (+ 70024us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":973913984}
I20260709 12:22:38.502225 17266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45230 (request call id 612) took 1434 ms. Trace:
I20260709 12:22:38.502661 17266 rpcz_store.cc:276] 0709 12:22:37.067599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:37.067814 (+   215us) service_pool.cc:224] Handling call
0709 12:22:38.502016 (+1434202us) spinlock_profiling.cc:243] Waited 420 ms on lock 0x7b5400a46d54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:38.502133 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":966407552}
I20260709 12:22:38.508023 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:38.509040 16468 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:22:38.510483 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:38.522853 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:38.523783 16475 raft_consensus.cc:2368] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3.
I20260709 12:22:38.528323 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:38.528936 16398 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:38.344271 18209 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:38.495306 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:38.531756 16367 leader_election.cc:400] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 3. Current term is 4.
I20260709 12:22:38.534407 16367 leader_election.cc:403] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:22:38.534853 16469 raft_consensus.cc:1240] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:38.535784 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:38.536520 16468 raft_consensus.cc:2393] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 5: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:38.521552 16395 raft_consensus.cc:1275] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:38.541774 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:38.545406 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:38.545997 16468 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:22:38.547510 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:38.549006 16468 raft_consensus.cc:2393] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 5: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:38.535266 18136 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:38.554215 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:38.554886 17261 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:38.554579 16441 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:38.559084 18272 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:38.560504 18321 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:38.561002 18321 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:38.561336 18321 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:38.561417 18209 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:38.554829 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:38.565059 16470 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:22:38.566584 18296 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:38.569168 18295 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:38.543917 18247 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:38.572403 18273 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:38.577283 18273 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:38.578701 18136 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:38.579591 18273 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:38.580680 16514 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:38.581171 18136 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:38.573973 17142 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:38.576067 16442 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:38.584697 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:38.590291 16401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 734) took 1485 ms. Trace:
I20260709 12:22:38.592535 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:38.593504 16468 raft_consensus.cc:2393] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 6: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:38.598335 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:38.590711 16401 rpcz_store.cc:276] 0709 12:22:37.097740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:37.097860 (+   120us) service_pool.cc:224] Handling call
0709 12:22:38.582640 (+1484780us) spinlock_profiling.cc:243] Waited 732 ms on lock 0x7b54004ecfd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:38.582774 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1684192640}
I20260709 12:22:38.598603 17142 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:38.551927 18306 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:38.599788 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:38.601835 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:38.602586 17142 leader_election.cc:400] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3.
I20260709 12:22:38.605145 17142 leader_election.cc:403] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:22:38.605268 16475 raft_consensus.cc:2368] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4.
I20260709 12:22:38.606583 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:38.606860 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:38.608604 16468 raft_consensus.cc:2393] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 4: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:38.609907 18209 consensus_queue.cc:1048] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:22:38.611829 17269 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45228 (request call id 532) took 1420 ms (client timeout 1647 ms). Trace:
I20260709 12:22:38.625926 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:38.612758 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:38.615283 18323 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:38.616421 18020 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:38.616817 18171 raft_consensus.cc:3055] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:38.618520 16398 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:22:38.626519 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:38.601274 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:22:38.625360 17269 rpcz_store.cc:269] 0709 12:22:37.191278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:37.196230 (+  4952us) service_pool.cc:224] Handling call
0709 12:22:38.611635 (+1415405us) spinlock_profiling.cc:243] Waited 386 ms on lock 0x7b540055f354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:38.611767 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":887579008}
I20260709 12:22:38.630178 16472 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:22:38.561439 18153 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:38.627719 16402 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
W20260709 12:22:38.646677 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 44312.6us
I20260709 12:22:38.597740 18272 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:38.625463 18008 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:38.629035 18020 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:38.638403 18171 raft_consensus.cc:740] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:38.627259 18306 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:38.629734 18323 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:38.634994 18295 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:38.572304 18290 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:38.636467 18116 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:38.601431 18273 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:38.636761 18278 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:38.638418 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 56005.4us
I20260709 12:22:38.638762 16474 raft_consensus.cc:2393] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 4: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:38.666990 16477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34840 (request call id 829) took 1056 ms. Trace:
I20260709 12:22:38.627964 17261 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:38.648226 18179 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:38.649554 17665 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:22:38.693084 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 121191us
I20260709 12:22:38.643518 18013 consensus_queue.cc:1048] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 12:22:38.694291 16477 rpcz_store.cc:276] 0709 12:22:37.599563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:37.599701 (+   138us) service_pool.cc:224] Handling call
0709 12:22:38.656260 (+1056559us) spinlock_profiling.cc:243] Waited 686 ms on lock 0x7b5400a68bd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:38.656423 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1577736448}
I20260709 12:22:38.696663 16515 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:38.668427 18020 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:38.662429 18081 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:38.667258 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:38.702538 16472 raft_consensus.cc:1240] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:38.655157 18272 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:38.704095 18171 consensus_queue.cc:260] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:38.704769 18207 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:38.706478 18272 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:22:38.710916 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:38.712208 16546 raft_consensus.cc:1275] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:38.699996 17665 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:38.714319 16548 raft_consensus.cc:1275] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:38.716008 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:38.716771 16542 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:22:38.719399 16397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58430 (request call id 343) took 1110 ms. Trace:
I20260709 12:22:38.714947 17665 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:38.721212 18306 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:38.723043 18290 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:38.724326 18171 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:38.678294 18323 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:38.716136 16400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 342) took 1107 ms. Trace:
I20260709 12:22:38.729339 16515 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:38.729095 16400 rpcz_store.cc:276] 0709 12:22:37.608712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:38.157015 (+548303us) service_pool.cc:224] Handling call
0709 12:22:38.223463 (+ 66448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:38.256386 (+ 32923us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:38.715865 (+459479us) spinlock_profiling.cc:243] Waited 6.8 ms on lock 0x7b540011fb54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:38.716061 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15642368}
I20260709 12:22:38.719933 16397 rpcz_store.cc:276] 0709 12:22:37.609304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:37.621832 (+ 12528us) service_pool.cc:224] Handling call
0709 12:22:38.719209 (+1097377us) spinlock_profiling.cc:243] Waited 448 ms on lock 0x7b54005e8dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:38.719333 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1030999296}
I20260709 12:22:38.733196 18273 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:38.733743 18273 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:38.734157 18273 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:38.674618 18166 consensus_queue.cc:1059] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
W20260709 12:22:38.740164 18280 log.cc:927] Time spent T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 1.205s	user 0.007s	sys 0.000s
I20260709 12:22:38.740710 18116 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:38.741677 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:38.743140 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:22:38.747426 18283 log.cc:927] Time spent T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.983s	user 0.000s	sys 0.003s
I20260709 12:22:38.741880 18019 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:38.728134 16395 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:38.750038 16470 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:22:38.754989 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:38.750314 18312 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:38.741716 17255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 526) took 1643 ms. Trace:
I20260709 12:22:38.755908 18136 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:38.756717 17255 rpcz_store.cc:276] 0709 12:22:37.098386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:37.102114 (+  3728us) service_pool.cc:224] Handling call
0709 12:22:37.102753 (+   639us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:37.103294 (+   541us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:37.103339 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:37.145454 (+ 42115us) log.cc:844] Serialized 26 byte log entry
0709 12:22:37.402854 (+257400us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:37.414822 (+ 11968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:37.428455 (+ 13633us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:38.738072 (+1309617us) spinlock_profiling.cc:243] Waited 61.6 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:22:38.741390 (+  3318us) raft_consensus.cc:1685] finished
0709 12:22:38.741423 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:38.741652 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":43662,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":135,"spinlock_wait_cycles":141631744,"thread_start_us":230984,"threads_started":1,"wal-append.queue_time_us":387753}
I20260709 12:22:38.758133 18166 raft_consensus.cc:3055] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:38.758327 18136 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:38.758880 18136 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:38.755069 18332 consensus_queue.cc:1048] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:22:38.760787 16468 raft_consensus.cc:1240] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:38.758760 18166 raft_consensus.cc:740] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:38.732189 18306 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:38.766767 16366 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:38.771339 18312 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:38.770807 18267 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:38.772631 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:38.771865 17143 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:38.775516 17780 consensus_queue.cc:1048] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:38.762652 18019 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:38.778010 16400 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:22:38.777675 17142 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:38.750295 18179 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3.
I20260709 12:22:38.776659 16366 leader_election.cc:400] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 4. Current term is 5.
I20260709 12:22:38.782641 16366 leader_election.cc:403] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:22:38.776249 18290 raft_consensus.cc:2804] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:38.799845 16515 leader_election.cc:400] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 4. Current term is 5.
I20260709 12:22:38.748287 18047 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:38.800274 16515 leader_election.cc:403] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:38.808053 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:22:38.834355 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 49641us
I20260709 12:22:38.806669 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:38.839805 17261 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:38.841761 16515 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:38.819422 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:38.843724 17266 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:38.845911 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 68323.8us
I20260709 12:22:38.847241 16400 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:22:38.847496 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:38.848088 16551 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:38.848124 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:38.849169 16400 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:38.857484 17261 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:22:38.861716 16400 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:38.865124 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:38.807058 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:38.780646 18116 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:22:38.818403 18336 log.cc:927] Time spent T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
W20260709 12:22:38.839952 18310 log.cc:927] Time spent T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.679s	user 0.003s	sys 0.003s
I20260709 12:22:38.788713 18292 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:38.871840 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:38.802034 16470 raft_consensus.cc:1240] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:38.891310 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:38.845181 18002 consensus_queue.cc:1048] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
W20260709 12:22:38.893132 18333 log.cc:927] Time spent T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.176s	user 0.003s	sys 0.000s
I20260709 12:22:38.893985 18267 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:38.896829 16546 raft_consensus.cc:1240] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:38.897006 18321 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:38.903489 18273 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:38.904630 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:22:38.905303 18328 log.cc:927] Time spent T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.201s	user 0.004s	sys 0.000s
I20260709 12:22:38.866863 17871 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:38.920109 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:38.773044 18166 consensus_queue.cc:260] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:38.924777 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:38.925602 18020 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:38.927057 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:38.876718 17266 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:22:38.785472 18008 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:38.929085 18008 raft_consensus.cc:697] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:38.930083 18008 consensus_queue.cc:237] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:38.931617 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:38.879019 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:38.932420 16441 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:38.933384 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:38.933725 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:38.934726 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:38.934988 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:38.878727 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:38.935905 16442 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:38.936412 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:38.937199 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:38.939827 17269 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:38.940295 18267 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:38.940832 18267 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:38.941259 18267 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:38.942166 18019 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:38.942700 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:38.876612 16514 leader_election.cc:400] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4.
I20260709 12:22:38.872800 17263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45228 (request call id 541) took 1112 ms. Trace:
I20260709 12:22:38.935995 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:38.806375 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:38.882261 16516 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:38.883193 16396 raft_consensus.cc:2393] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 4: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:38.778014 18306 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:38.818774 16395 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6.
I20260709 12:22:38.885953 16366 leader_election.cc:400] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 4. Current term is 5.
I20260709 12:22:38.945574 17263 rpcz_store.cc:276] 0709 12:22:37.760172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:37.801848 (+ 41676us) service_pool.cc:224] Handling call
0709 12:22:38.854796 (+1052948us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b540005fd54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:38.872751 (+ 17955us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":531789056}
W20260709 12:22:38.953469 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 71023.2us
I20260709 12:22:38.806273 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:38.886281 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:38.961615 16549 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:38.888471 16477 raft_consensus.cc:2368] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4.
I20260709 12:22:38.814136 18001 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:38.965364 16515 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:38.965910 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:38.966936 17266 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:38.887025 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:38.968247 16468 raft_consensus.cc:2368] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 5. Current term is 6.
I20260709 12:22:38.872555 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:22:38.968299 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 86112.8us
I20260709 12:22:38.909703 16469 raft_consensus.cc:2393] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 5: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:38.911075 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:38.913060 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:38.858771 18286 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:22:38.879703 18335 log.cc:927] Time spent T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 12:22:38.981009 16516 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:38.983603 16398 raft_consensus.cc:1240] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3. Ops: []
W20260709 12:22:38.946448 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 76318us
I20260709 12:22:38.947007 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:38.874861 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:38.954214 16442 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:38.993366 16442 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:38.955394 16441 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:38.867547 18290 raft_consensus.cc:697] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 4 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:39.017428 16545 raft_consensus.cc:2393] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 6: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:38.970383 16402 raft_consensus.cc:2393] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 5: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:38.971427 16544 raft_consensus.cc:2393] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 5: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:38.976006 16477 raft_consensus.cc:1240] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 3. Current term is 4. Ops: []
W20260709 12:22:38.936412 18334 log.cc:927] Time spent T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.232s	user 0.003s	sys 0.000s
I20260709 12:22:38.980453 17264 raft_consensus.cc:1275] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:38.900309 18321 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:38.927753 18020 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:39.003579 16398 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:39.007004 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:38.914563 18329 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:38.978929 18332 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:39.013780 18278 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:39.044538 18207 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:39.014302 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:39.015213 17271 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:38.979837 18136 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:39.076911 17255 raft_consensus.cc:2393] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 6: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:39.030861 16549 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:39.051580 16475 raft_consensus.cc:1275] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:39.020804 18235 consensus_queue.cc:1059] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 12:22:38.977161 17871 raft_consensus.cc:697] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:38.882894 18116 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:39.054208 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:39.097172 17269 raft_consensus.cc:2368] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 4. Current term is 5.
I20260709 12:22:39.056572 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:39.058543 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:39.016234 18153 consensus_queue.cc:1048] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:22:39.067919 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 74693.1us
I20260709 12:22:39.016571 18019 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:39.013888 18306 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:39.014151 17144 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:39.047189 18312 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:39.078289 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:38.974934 18209 consensus_queue.cc:1059] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.096s
I20260709 12:22:39.045589 16514 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:39.106523 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 61023.7us
I20260709 12:22:39.079867 18329 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:39.054947 18170 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.054766 18161 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:39.067863 18247 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 3. Current term is 4.
I20260709 12:22:39.101387 18001 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:39.108971 18337 raft_consensus.cc:2749] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:39.109619 18020 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:22:39.101970 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 87893.7us
I20260709 12:22:39.102034 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:39.105262 17258 raft_consensus.cc:2368] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 4. Current term is 5.
I20260709 12:22:39.116255 17271 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:39.108263 17264 raft_consensus.cc:2368] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 3. Current term is 4.
I20260709 12:22:39.109351 18002 raft_consensus.cc:2764] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:22:39.088107 18332 raft_consensus.cc:697] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 3 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:39.114995 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:39.114630 18020 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:39.161262 16403 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:39.103080 18312 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:39.020780 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 65521.6us
I20260709 12:22:39.132542 18279 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:39.133132 18332 consensus_queue.cc:237] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:39.134119 18136 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:39.134716 18001 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:39.134838 18235 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.135193 18273 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:39.135872 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:39.100793 18081 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:22:39.143784 17872 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.036s	sys 0.075s
I20260709 12:22:39.117748 18209 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:39.115392 18170 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:39.145931 18267 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:39.147224 18047 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:39.151995 18306 raft_consensus.cc:3055] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:39.152763 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:39.554510 18306 raft_consensus.cc:740] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:39.153076 18299 raft_consensus.cc:3055] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:39.555299 18299 raft_consensus.cc:740] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:39.156844 16542 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:22:39.157681 16546 raft_consensus.cc:1275] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:39.158099 18290 consensus_queue.cc:237] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:39.158888 16551 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:39.115120 18329 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:39.163142 18020 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:39.116182 17144 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:39.314188 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 297652us
W20260709 12:22:39.186617 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 245296us
I20260709 12:22:39.314790 18116 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:39.272800 17871 consensus_queue.cc:237] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:39.570950 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 454969us
W20260709 12:22:39.330844 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 223525us
I20260709 12:22:39.214262 17271 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:22:39.523590 16441 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:39.574506 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 393041us
I20260709 12:22:39.575534 18161 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:39.576565 18020 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:39.577913 18170 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:39.549185 18235 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:39.555552 18306 consensus_queue.cc:260] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:39.580646 18299 consensus_queue.cc:260] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:39.581197 18235 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:39.581331 18306 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:39.555917 18247 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.581861 18299 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:39.582175 18247 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:39.583273 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:39.583935 17258 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:22:39.555756 18209 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:39.587956 18209 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:39.572852 18267 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:39.551581 18279 raft_consensus.cc:697] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:39.517508 18168 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:39.240355 18271 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:39.515867 18341 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:39.528180 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:39.553543 16549 raft_consensus.cc:1275] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:39.552928 18153 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:39.551589 18292 consensus_meta.cc:287] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.633s	user 0.008s	sys 0.008s
W20260709 12:22:39.553058 18323 consensus_meta.cc:287] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.800s	user 0.016s	sys 0.005s
I20260709 12:22:39.558643 17665 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 4. Current term is 5.
I20260709 12:22:39.623605 18323 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:39.200232 18312 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:39.617746 18271 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:39.625644 16548 raft_consensus.cc:1275] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:39.626173 18323 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:39.600171 17258 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:22:39.602725 16514 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:39.523936 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 558625us
I20260709 12:22:39.569092 18206 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.616585 18247 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:39.622969 18292 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 4. Current term is 5.
I20260709 12:22:39.631861 16471 raft_consensus.cc:2393] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 6: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:39.627357 18153 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:39.631560 16514 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:39.636621 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:39.638074 16514 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:39.642995 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:39.646338 16476 raft_consensus.cc:2393] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 6: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:39.653146 17271 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 8. Current term is 9. Ops: []
W20260709 12:22:39.655751 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:39.657394 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:22:39.657684 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 85592.9us
I20260709 12:22:39.659057 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:39.641209 17665 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:39.646684 18329 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:39.646253 18060 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:39.646723 18271 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:39.618144 18168 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:39.657410 18209 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:39.674875 16515 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:39.676142 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:39.677265 16441 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:39.679829 18295 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:39.642519 18206 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:39.662959 18273 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:39.620877 18341 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:39.690124 16398 raft_consensus.cc:1275] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:39.663643 18020 raft_consensus.cc:697] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:39.691386 16548 raft_consensus.cc:1275] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:22:39.710589 17266 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:39.710989 16548 pending_rounds.cc:85] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1: Aborting all ops after (but not including) 2
I20260709 12:22:39.711315 16548 pending_rounds.cc:107] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:22:39.711576 16548 raft_consensus.cc:2889] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:39.692586 17665 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:39.721378 17665 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:39.721858 17665 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:39.704221 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:39.664966 18347 consensus_queue.cc:1059] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 12:22:39.681300 18060 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:39.685232 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:39.716681 16402 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:39.718880 17255 raft_consensus.cc:1240] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:39.738560 16403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46250 (request call id 692) took 1008 ms. Trace:
I20260709 12:22:39.739063 16403 rpcz_store.cc:276] 0709 12:22:38.730445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:38.919734 (+189289us) service_pool.cc:224] Handling call
0709 12:22:39.738389 (+818655us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b54009bd7d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:39.738499 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":523450240}
I20260709 12:22:39.739714 18171 consensus_queue.cc:1048] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:39.740427 18171 consensus_queue.cc:1243] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer 610f4feda11942739236f2b121a4b9c1 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:22:39.740293 18358 consensus_queue.cc:1048] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:39.742956 16397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 355) took 1015 ms. Trace:
I20260709 12:22:39.743866 18153 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:39.743422 16397 rpcz_store.cc:276] 0709 12:22:38.728005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:38.975289 (+247284us) service_pool.cc:224] Handling call
0709 12:22:39.052798 (+ 77509us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:39.068432 (+ 15634us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:22:39.091348 (+ 22916us) log.cc:844] Serialized 14 byte log entry
0709 12:22:39.146693 (+ 55345us) spinlock_profiling.cc:243] Waited 3.27 ms on lock 0x7b180016a260. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:22:39.368807 (+222114us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:39.536964 (+168157us) log.cc:844] Serialized 26 byte log entry
0709 12:22:39.537408 (+   444us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:39.537488 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:39.537600 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:39.742630 (+205030us) raft_consensus.cc:1685] finished
0709 12:22:39.742744 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:39.742896 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5401,"prepare.queue_time_us":22859,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":10540,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":3696,"spinlock_wait_cycles":7523712,"thread_start_us":250300,"threads_started":1,"wal-append.queue_time_us":592041}
I20260709 12:22:39.696880 18279 consensus_queue.cc:237] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:39.743024 17665 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:39.748395 17868 consensus_queue.cc:1059] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:39.749044 17665 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:39.749590 17868 consensus_queue.cc:1048] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:39.750675 18166 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:39.751418 17868 raft_consensus.cc:3055] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:39.751801 17868 raft_consensus.cc:740] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:39.753690 17868 consensus_queue.cc:260] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:39.755020 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:39.716727 18273 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:39.756441 17868 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:39.763464 16469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34826 (request call id 761) took 1183 ms. Trace:
I20260709 12:22:39.763998 16469 rpcz_store.cc:276] 0709 12:22:38.583234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:38.590840 (+  7606us) service_pool.cc:224] Handling call
0709 12:22:39.763211 (+1172371us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b54002f3354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:39.763384 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":650277120}
I20260709 12:22:39.767127 18273 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:39.719923 18020 consensus_queue.cc:237] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:39.652902 18351 consensus_queue.cc:1059] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 12:22:39.699477 16441 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:39.690909 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:22:39.800375 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 101020us
I20260709 12:22:39.731164 18060 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:39.702867 18209 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:39.772504 16400 raft_consensus.cc:1275] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:39.703148 18271 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:39.719600 18168 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:39.779269 17266 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:39.779142 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:39.756574 16471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53186 (request call id 268) took 1081 ms. Trace:
I20260709 12:22:39.806216 18271 raft_consensus.cc:697] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:39.806470 16471 rpcz_store.cc:276] 0709 12:22:38.674868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:38.739307 (+ 64439us) service_pool.cc:224] Handling call
0709 12:22:39.756326 (+1017019us) spinlock_profiling.cc:243] Waited 871 ms on lock 0x7b5400594554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:39.756514 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2003216768}
W20260709 12:22:39.807375 18355 log.cc:927] Time spent T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 12:22:39.807435 18271 consensus_queue.cc:237] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:39.785459 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:39.810834 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:39.811203 16399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 700) took 1046 ms. Trace:
I20260709 12:22:39.811533 16395 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:22:39.785862 16394 raft_consensus.cc:1240] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:22:39.815459 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:39.815989 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:39.786237 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:39.818034 16397 raft_consensus.cc:1275] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:39.816812 16401 raft_consensus.cc:2393] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 6: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:39.788105 18329 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:39.775029 16545 pending_rounds.cc:85] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1: Aborting all ops after (but not including) 2
I20260709 12:22:39.820667 16545 pending_rounds.cc:107] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:22:39.821051 16545 raft_consensus.cc:2889] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:39.798712 16544 raft_consensus.cc:1275] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:39.704607 18295 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:39.780310 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:39.822054 16548 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:22:39.811671 16399 rpcz_store.cc:276] 0709 12:22:38.764326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.020095 (+255769us) service_pool.cc:224] Handling call
0709 12:22:39.064697 (+ 44602us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:39.088508 (+ 23811us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:39.089382 (+   874us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:39.095230 (+  5848us) log.cc:844] Serialized 26 byte log entry
0709 12:22:39.369583 (+274353us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:39.369631 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:39.369708 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:39.810985 (+441277us) raft_consensus.cc:1685] finished
0709 12:22:39.811011 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:39.811145 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4916,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":80,"thread_start_us":216949,"threads_started":1,"wal-append.queue_time_us":536409}
I20260709 12:22:39.804796 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:39.819041 16441 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:39.833964 16441 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:39.797010 18267 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.835927 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:39.814141 18341 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:39.841529 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:39.113626  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:39.803536 18323 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:39.818528 18351 consensus_queue.cc:1243] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer 52ab50b0237346c1a13ede947f6a63c2 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:22:39.845098 18161 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:39.851977 16225 catalog_manager.cc:5697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 3 to 5, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 5 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:39.852252 17143 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:39.845903 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:39.802192 16546 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:39.805210 16402 raft_consensus.cc:1275] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:39.836903 18116 raft_consensus.cc:2804] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:39.884629 16394 raft_consensus.cc:1275] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:39.885190 18116 raft_consensus.cc:697] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:39.886075 16224 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53636 (request call id 130) took 10588 ms. Trace:
I20260709 12:22:39.886417 18295 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:39.891229 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:39.870831 16225 catalog_manager.cc:5697] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:39.866587 17144 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:39.878651 17780 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:22:39.878883 18357 log.cc:927] Time spent T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:22:39.879359 16395 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:39.879128 18002 consensus_queue.cc:1048] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:39.941356 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:39.944062 18206 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:39.945400 17261 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:39.880059 18360 consensus_queue.cc:1048] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:39.950901 16367 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:39.950930 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:39.952962 16367 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:39.886502 16224 rpcz_store.cc:276] 0709 12:22:29.297855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.298028 (+   173us) service_pool.cc:224] Handling call
0709 12:22:39.885823 (+10587795us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:22:34.275835 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:22:35.125253 (+849418us) spinlock_profiling.cc:243] Waited 84.2 ms on lock 0x7b5c00209b8c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287c07b331 00007f287c07b221 00007f287c075f6c 00007f28798d3d69 00007f28798bf0a6 00007f28798bed58 00007f28798c39f1 00007f28798c39a9 00007f28798c3939 00007f28798c3901 00007f28798c2b2d
0709 12:22:35.125329 (+    76us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:22:35.125362 (+    33us) write_op.cc:435] Acquired schema lock
0709 12:22:35.125383 (+    21us) tablet.cc:662] Decoding operations
0709 12:22:35.126607 (+  1224us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:22:35.126694 (+    87us) write_op.cc:641] Partition lock acquired for write op
0709 12:22:35.127876 (+  1182us) tablet.cc:685] Acquiring locks for 11 operations
0709 12:22:35.225037 (+ 97161us) tablet.cc:701] Row locks acquired
0709 12:22:35.225057 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:22:35.225207 (+   150us) write_op.cc:270] Start()
0709 12:22:35.225398 (+   191us) write_op.cc:276] Timestamp: P: 1783599755225160 usec, L: 0
0709 12:22:35.225422 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:22:35.226095 (+   673us) log.cc:844] Serialized 4322 byte log entry
0709 12:22:35.794138 (+568043us) op_driver.cc:464] REPLICATION: finished
0709 12:22:35.844462 (+ 50324us) write_op.cc:301] APPLY: starting
0709 12:22:35.844556 (+    94us) tablet.cc:1367] starting BulkCheckPresence
0709 12:22:35.844871 (+   315us) tablet.cc:1370] finished BulkCheckPresence
0709 12:22:35.844909 (+    38us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:22:35.846326 (+  1417us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:22:35.846638 (+   312us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 12:22:35.846706 (+    68us) write_op.cc:312] APPLY: finished
0709 12:22:35.847117 (+   411us) log.cc:844] Serialized 82 byte log entry
0709 12:22:35.847308 (+   191us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:35.847485 (+   177us) write_op.cc:454] Released schema lock
0709 12:22:35.847552 (+    67us) write_op.cc:341] FINISH: Updating metrics
0709 12:22:36.125567 (+278015us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:36.125639 (+    72us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":128728,"reported_tablets":21,"child_traces":[["op",{"apply.queue_time_us":49998,"apply.run_cpu_time_us":3346,"apply.run_wall_time_us":3490,"num_ops":11,"prepare.queue_time_us":364979,"prepare.run_cpu_time_us":50463,"prepare.run_wall_time_us":1084917,"raft.queue_time_us":332788,"raft.run_cpu_time_us":12296,"raft.run_wall_time_us":356009,"replication_time_us":568488,"spinlock_wait_cycles":193760128,"thread_start_us":810748,"threads_started":4,"wal-append.queue_time_us":195233,"wal-append.run_cpu_time_us":28594,"wal-append.run_wall_time_us":1785937}]]}
I20260709 12:22:39.954026 16367 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
W20260709 12:22:39.959190 18353 log.cc:927] Time spent T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:22:39.958894 18312 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:39.900972 17780 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:39.960222 16547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 254) took 1378 ms. Trace:
I20260709 12:22:39.960305 17780 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:39.960723 16547 rpcz_store.cc:276] 0709 12:22:38.581398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:38.582161 (+   763us) service_pool.cc:224] Handling call
0709 12:22:38.582557 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:38.847080 (+264523us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:38.847113 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:38.880480 (+ 33367us) log.cc:844] Serialized 26 byte log entry
0709 12:22:39.378874 (+498394us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:39.378925 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:39.379012 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:39.960011 (+580999us) raft_consensus.cc:1685] finished
0709 12:22:39.960039 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:39.960167 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":42974,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":12121,"thread_start_us":655881,"threads_started":2,"wal-append.queue_time_us":877675}
I20260709 12:22:39.961694 18312 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:39.904464 16441 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:39.916230 16225 catalog_manager.cc:5697] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:39.919869 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:39.920687 18295 consensus_queue.cc:1048] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:39.926178 16516 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:39.950345 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:39.933094 18290 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.861052 18153 consensus_queue.cc:1048] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.096s
I20260709 12:22:39.949319 18358 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:39.851662 18267 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:39.861573 18359 consensus_queue.cc:1048] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.118s
I20260709 12:22:39.965678 18222 raft_consensus.cc:2764] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:22:39.934386 17871 consensus_queue.cc:1048] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:39.886803 18116 consensus_queue.cc:237] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:39.880441 18347 raft_consensus.cc:3055] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:40.036643 18347 raft_consensus.cc:740] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:40.043820 18347 consensus_queue.cc:260] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:40.046777 16402 raft_consensus.cc:2393] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 6: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:39.974989 18206 consensus_queue.cc:1059] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.067s
I20260709 12:22:39.972668 18290 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:39.965816 18299 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:39.967813 18136 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:39.957336 18292 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:39.934008 16366 leader_election.cc:400] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 5. Current term is 6.
I20260709 12:22:40.100759 16366 leader_election.cc:403] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
W20260709 12:22:40.101766 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 167925us
I20260709 12:22:39.950961 18161 consensus_queue.cc:1048] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
W20260709 12:22:40.015169 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:40.103417 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:40.103840 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:40.104672 16475 raft_consensus.cc:2393] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 4: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:40.105027 16473 raft_consensus.cc:1240] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 5. Current term is 6. Ops: []
W20260709 12:22:40.033067 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 128694us
I20260709 12:22:39.963577 16367 leader_election.cc:400] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 4. Current term is 5.
I20260709 12:22:40.107311 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:22:40.107615 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 144227us
I20260709 12:22:40.108517 16367 leader_election.cc:400] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 3. Current term is 4.
I20260709 12:22:40.073442 16476 raft_consensus.cc:1275] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:40.077229 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:40.040869 18168 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:40.041584 18312 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:40.116750 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:40.051314 18323 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:22:40.113793 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:40.145529 16367 leader_election.cc:400] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 4. Current term is 5.
I20260709 12:22:40.145987 16367 leader_election.cc:403] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:22:40.147018 16367 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:40.127511 17261 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45224 (request call id 632) took 1246 ms. Trace:
I20260709 12:22:40.148458 17261 rpcz_store.cc:276] 0709 12:22:38.881592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:38.910738 (+ 29146us) service_pool.cc:224] Handling call
0709 12:22:40.127328 (+1216590us) spinlock_profiling.cc:243] Waited 949 ms on lock 0x7b5400210054. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:40.127454 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2183300992}
I20260709 12:22:40.103425 18247 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 5. Current term is 6.
I20260709 12:22:40.152635 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:40.117612 18273 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:40.167770 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 364716us
I20260709 12:22:40.179646 17269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 634) took 1088 ms. Trace:
I20260709 12:22:40.180215 17269 rpcz_store.cc:276] 0709 12:22:39.091369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.931148 (+839779us) service_pool.cc:224] Handling call
0709 12:22:39.931623 (+   475us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:39.931903 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:22:39.939068 (+  7165us) log.cc:844] Serialized 14 byte log entry
0709 12:22:40.178861 (+239793us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:40.178937 (+    76us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:40.179005 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:40.179414 (+   409us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:40.179574 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6952,"prepare.queue_time_us":3438,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"thread_start_us":231923,"threads_started":1}
I20260709 12:22:40.114033 18350 consensus_queue.cc:1059] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 12:22:40.161827 18170 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:40.161053 18298 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 4. Current term is 5.
W20260709 12:22:40.182717 18286 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.106s	user 0.025s	sys 0.041s
I20260709 12:22:40.165781 18360 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.168393 18209 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:40.127108 18323 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:40.144634 18299 consensus_queue.cc:1059] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 12:22:40.122706 18168 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:40.210999 16471 raft_consensus.cc:1275] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:40.169507 18279 consensus_queue.cc:1059] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.050s
W20260709 12:22:40.217535 18286 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.150s	user 0.026s	sys 0.041s
W20260709 12:22:40.236796 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 128787us
I20260709 12:22:40.233872 16514 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:40.177057 18273 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:40.245322 16514 leader_election.cc:400] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4.
I20260709 12:22:40.248237 16514 leader_election.cc:403] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:22:40.217877 18306 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:40.230079 18267 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:40.251613 16546 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:40.248474 18066 raft_consensus.cc:3055] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:40.264513 16549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 261) took 1166 ms. Trace:
I20260709 12:22:40.242341 18286 raft_consensus.cc:697] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:40.248898 18136 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:40.265148 16549 rpcz_store.cc:276] 0709 12:22:39.098512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.127225 (+ 28713us) service_pool.cc:224] Handling call
0709 12:22:39.127633 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:39.613819 (+486186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:40.264192 (+650373us) spinlock_profiling.cc:243] Waited 425 ms on lock 0x7b54007b52d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:40.264412 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":978599808}
I20260709 12:22:40.227434 18360 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:40.258359 18366 log.cc:927] Time spent T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:22:40.257417 17665 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4.
W20260709 12:22:40.311779 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 195809us
I20260709 12:22:40.317309 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:40.318611 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:40.322062 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:40.323020 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:40.323273 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:40.328323 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:40.328699 16396 raft_consensus.cc:1275] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:40.329540 16403 raft_consensus.cc:2393] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 6: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
W20260709 12:22:40.308804 18362 log.cc:927] Time spent T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.278s	user 0.001s	sys 0.000s
I20260709 12:22:40.330525 16469 raft_consensus.cc:2393] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 4: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:40.264935 18066 raft_consensus.cc:740] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:40.327441 17255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 633) took 1244 ms. Trace:
I20260709 12:22:40.255721 18306 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:40.338106 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:40.344170 16397 raft_consensus.cc:2393] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 5: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:40.309213 18267 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:40.322738 18172 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:40.333267 18019 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.354678 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:40.355017 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:40.354743 17258 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:40.328059 17872 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:40.342350 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:22:40.414979 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 161056us
I20260709 12:22:40.416605 16469 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:40.421249 16514 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:22:40.416841 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 70610us
W20260709 12:22:40.482702 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 61541.8us
I20260709 12:22:40.417825 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:40.484875 17254 raft_consensus.cc:2393] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 5: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:40.487129 18168 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:22:40.489080 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 342112us
I20260709 12:22:40.328675 17780 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:40.490823 16367 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:40.489277 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:40.352593 17871 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:40.355232 18312 raft_consensus.cc:697] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:40.502560 16473 raft_consensus.cc:2393] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 7: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:40.483111 17142 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:40.504778 18171 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:40.505604 18370 log.cc:927] Time spent T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.270s	user 0.001s	sys 0.000s
W20260709 12:22:40.506382 18369 log.cc:927] Time spent T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.256s	user 0.001s	sys 0.000s
I20260709 12:22:40.521347 17263 raft_consensus.cc:2393] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 6: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:40.516661 16441 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:40.418248 18290 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:40.501573 18367 log.cc:927] Time spent T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.318s	user 0.000s	sys 0.001s
I20260709 12:22:40.489029 16470 raft_consensus.cc:1240] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:40.356698 18153 consensus_queue.cc:1048] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.151s
I20260709 12:22:40.508368 18332 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:22:40.530442 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 47744us
I20260709 12:22:40.419289 18172 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:40.350659 18267 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:40.551443 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 60738.8us
W20260709 12:22:40.555025 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 67634.7us
I20260709 12:22:40.568990 16473 raft_consensus.cc:1275] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:22:40.425693 18019 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:40.518884 18008 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:40.340085 17255 rpcz_store.cc:276] 0709 12:22:39.083037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.148480 (+ 65443us) service_pool.cc:224] Handling call
0709 12:22:39.148903 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:39.733391 (+584488us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:40.309237 (+575846us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b54005f6254. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:40.327350 (+ 18113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1285155712}
I20260709 12:22:40.556314 16441 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:40.583603 16441 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:40.421145 18380 consensus_queue.cc:1048] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.221s
I20260709 12:22:40.586575 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:40.587333 16546 raft_consensus.cc:2393] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 4: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:40.587169 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:40.571151 18386 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:40.571314 18222 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:40.598170 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:40.514957 17871 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:40.588606 18373 log.cc:927] Time spent T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
I20260709 12:22:40.480149 17872 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:40.605425 16551 raft_consensus.cc:1240] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:22:40.521967 18171 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:40.577373 18323 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:40.585335 18390 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:40.350906 18286 consensus_queue.cc:237] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:40.581453 18271 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:40.594094 16366 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:40.608219 16546 raft_consensus.cc:1240] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:40.583520 18235 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:40.616928 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:40.622229 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:40.602849 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 119285us
W20260709 12:22:40.609520 18377 log.cc:927] Time spent T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.346s	user 0.001s	sys 0.001s
I20260709 12:22:40.603129 18272 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.598006 18279 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:22:40.622963 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:40.602064 18385 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:40.623383 16514 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:40.606663 18273 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:40.611905 18170 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:40.600754 18116 consensus_queue.cc:1048] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 12:22:40.636272 18047 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:40.638814 18047 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:40.639237 18047 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:40.616142 18222 raft_consensus.cc:697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:40.641481 16549 raft_consensus.cc:1275] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:22:40.629010 18060 consensus_meta.cc:287] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.806s	user 0.011s	sys 0.002s
I20260709 12:22:40.624238 18247 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.670535 16514 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:40.677899 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:40.625991 18235 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:40.687325 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 93337.8us
I20260709 12:22:40.605803 16543 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "f99066d13e524238aec6aa2024a8bd50"
 from {username='slave'} at 127.0.0.1:59590
W20260709 12:22:40.641184 18378 log.cc:927] Time spent T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.332s	user 0.001s	sys 0.000s
I20260709 12:22:40.693532 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:40.694495 17266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45224 (request call id 636) took 1166 ms. Trace:
I20260709 12:22:40.709631 17266 rpcz_store.cc:276] 0709 12:22:39.527759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.527907 (+   148us) service_pool.cc:224] Handling call
0709 12:22:40.694346 (+1166439us) spinlock_profiling.cc:243] Waited 96.7 ms on lock 0x7b540005f854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:40.694445 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":222488832}
I20260709 12:22:40.694630 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:40.637327 18172 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:40.669911 18273 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:40.672685 18360 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:22:40.741250 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 46692.2us
I20260709 12:22:40.622304 18312 consensus_queue.cc:237] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:40.746446 16473 raft_consensus.cc:1275] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:40.755312 17142 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:40.768090 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:40.771359 17142 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:40.631295 18272 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:40.636575 18170 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:40.746035 17780 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:40.688134 18247 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:40.781198 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:40.686753 18060 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:40.789784 17142 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:40.793756 16442 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:40.774745 18172 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:40.795126 16442 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:40.782342 18153 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:22:40.800244 18382 log.cc:927] Time spent T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.323s	user 0.000s	sys 0.001s
I20260709 12:22:40.783795 18002 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:40.801946 18396 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:40.781728 17665 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:40.801550 18171 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:40.804358 16442 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:40.798141 18371 log.cc:927] Time spent T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.333s	user 0.000s	sys 0.001s
I20260709 12:22:40.766765 17868 consensus_queue.cc:1048] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 12:22:40.789763 18279 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:40.797889 18341 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.808713 16550 raft_consensus.cc:1240] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:40.794972 18207 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.774609 18360 consensus_queue.cc:1059] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
W20260709 12:22:40.892683 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 50828.3us
W20260709 12:22:40.842118 18368 log.cc:927] Time spent T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.302s	user 0.000s	sys 0.001s
I20260709 12:22:40.844619 18389 consensus_queue.cc:1048] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 12:22:40.642247 18385 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:40.809193 17665 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:40.806730 18395 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:40.625270 18390 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:40.940141 18390 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:40.802109 18002 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:40.940367 18385 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:40.941051 18272 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:40.941751 18390 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.942407 18002 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:40.942301 18390 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:40.801599 18170 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:40.943567 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:40.944451 16548 raft_consensus.cc:2393] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 4: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:40.944844 18390 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:40.845923 18397 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:40.893769 18296 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:40.945631 18397 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:40.945914 18296 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:40.947904 18312 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:22:40.905877 18372 log.cc:927] Time spent T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.597s	user 0.013s	sys 0.006s
I20260709 12:22:40.949146 18397 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:40.951531 18047 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:40.953619 16470 raft_consensus.cc:1240] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:40.956223 18153 raft_consensus.cc:3055] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:40.956768 18153 raft_consensus.cc:740] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:40.957676 18153 consensus_queue.cc:260] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:40.958969 18153 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:40.961236 18296 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:40.966480 18306 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:40.977905 18171 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.978613 18171 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:40.982712 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:40.983664 17266 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:40.796284 18222 consensus_queue.cc:237] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:40.805028 18267 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:40.797000 18386 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:40.810945 18168 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:40.915463 18247 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:40.915844 18402 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.917238 17868 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:40.832427 18279 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:40.846346 17871 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:40.831794 18341 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:40.785053 18292 consensus_queue.cc:1059] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 12:22:40.902037 18323 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:40.918854 18060 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:40.833176 16543 raft_consensus.cc:606] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Received request to transfer leadership to TS f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:40.835027 18207 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:40.923512 18235 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:22:40.925158 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 120913us
I20260709 12:22:40.934736 17665 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:22:40.844705 18375 log.cc:927] Time spent T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.341s	user 0.001s	sys 0.000s
I20260709 12:22:40.968863 18312 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:40.968523 18395 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:41.036616 18247 raft_consensus.cc:3055] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:41.037096 18247 raft_consensus.cc:740] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:41.038144 18395 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:41.038058 18247 consensus_queue.cc:260] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:41.039455 18247 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:41.041481 18279 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:41.004683 18402 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:41.043654 17872 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:40.972617 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:41.044240 18402 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:40.894200 17780 consensus_queue.cc:1048] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:40.986529 16545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 269) took 1212 ms. Trace:
I20260709 12:22:41.027292 18332 consensus_queue.cc:1048] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 12:22:41.046135 16545 rpcz_store.cc:276] 0709 12:22:39.774279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.774432 (+   153us) service_pool.cc:224] Handling call
0709 12:22:39.774814 (+   382us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:39.833771 (+ 58957us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:39.833814 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:39.838259 (+  4445us) log.cc:844] Serialized 52 byte log entry
0709 12:22:40.219730 (+381471us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:40.947625 (+727895us) spinlock_profiling.cc:243] Waited 1.78 ms on lock 0x7b180011b3c0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2875a314bc 00007f2872d7f48b 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d99455 00007f28798813c7 00007f2875e939a7 00007f2875ea30bf
0709 12:22:40.947789 (+   164us) log.cc:844] Serialized 14 byte log entry
0709 12:22:40.948049 (+   260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:40.948132 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:40.948173 (+    41us) raft_consensus.cc:1685] finished
0709 12:22:40.948255 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:40.986475 (+ 38220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5322,"prepare.queue_time_us":254967,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":3792,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":222,"spinlock_wait_cycles":4084096,"thread_start_us":458737,"threads_started":2,"wal-append.queue_time_us":483471}
I20260709 12:22:40.986523 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:40.987763 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:41.048565 18329 raft_consensus.cc:3055] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:40.989341 16474 raft_consensus.cc:1240] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:22:40.990722 18396 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:40.995643 18408 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:40.996727 18171 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:40.922212 18401 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:40.996919 18350 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:40.954679 18047 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:41.009330 16543 raft_consensus.cc:614] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:41.015300 16395 raft_consensus.cc:1240] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:22:40.998283 18271 raft_consensus.cc:697] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:41.029748 16442 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:41.031576 18312 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:41.032151 16515 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:41.032537 16542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 270) took 1257 ms. Trace:
I20260709 12:22:41.064770 18401 raft_consensus.cc:697] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:41.064628 16542 rpcz_store.cc:276] 0709 12:22:39.774663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.780171 (+  5508us) service_pool.cc:224] Handling call
0709 12:22:39.786459 (+  6288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:39.817554 (+ 31095us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:39.817594 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:39.845381 (+ 27787us) log.cc:844] Serialized 26 byte log entry
0709 12:22:40.161369 (+315988us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:40.167519 (+  6150us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:40.178143 (+ 10624us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:41.032233 (+854090us) raft_consensus.cc:1685] finished
0709 12:22:41.032259 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:41.032474 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8898,"raft.queue_time_us":16777,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"thread_start_us":279674,"threads_started":1,"wal-append.queue_time_us":468079}
I20260709 12:22:41.065979 16515 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:41.065837 18401 consensus_queue.cc:237] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:41.048588 17264 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:41.052873 16472 raft_consensus.cc:1240] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 12:22:41.067548 16515 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:41.068439 18348 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:41.054464 18396 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:41.068917 18348 raft_consensus.cc:697] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:41.069150 18286 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:41.068819 16515 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:41.069820 18348 consensus_queue.cc:237] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:41.070597 18273 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:41.072652 18312 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:41.000469 17868 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:41.074229 18404 log.cc:927] Time spent T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.230s	user 0.002s	sys 0.000s
I20260709 12:22:41.075429 17868 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:40.975901 18066 consensus_queue.cc:260] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:41.076670 18066 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:41.072129 18298 raft_consensus.cc:3055] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:41.069847 18286 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:22:41.080008 18380 consensus_queue.cc:1175] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "52ab50b0237346c1a13ede947f6a63c2" responder_term: 6 status { last_received { term: 5 index: 3 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 5. Current term is 6. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:22:41.050944 17665 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:41.070506 16402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 766) took 1153 ms. Trace:
I20260709 12:22:41.058735 18413 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:41.082484 17665 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:41.082513 16402 rpcz_store.cc:276] 0709 12:22:39.917259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.919555 (+  2296us) service_pool.cc:224] Handling call
0709 12:22:41.070324 (+1150769us) spinlock_profiling.cc:243] Waited 82.2 ms on lock 0x7b5400762ad4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:41.070450 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":188976896}
I20260709 12:22:41.084290 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:41.084929 17665 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:41.027050 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:41.084939 16472 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:41.086959 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:41.006359 18394 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:41.049046 18329 raft_consensus.cc:740] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:41.088932 18341 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:41.088768 18394 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:41.062314 16442 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:41.089262 18329 consensus_queue.cc:260] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:41.090446 18329 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:41.090510 16442 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:41.090957 18394 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:41.091761 18413 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:41.062773 18350 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:41.092149 18413 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:41.092648 18413 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:41.063448 18408 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:41.080622 18298 raft_consensus.cc:740] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:41.081773 16471 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
mode: GRACEFUL
new_leader_uuid: "f99066d13e524238aec6aa2024a8bd50"
 from {username='slave'} at 127.0.0.1:39000
I20260709 12:22:41.100863 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:41.101833 16403 raft_consensus.cc:2393] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 3: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:41.087080 16397 raft_consensus.cc:1275] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 12:22:41.102468 16397 pending_rounds.cc:85] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 4
I20260709 12:22:41.102779 16397 pending_rounds.cc:107] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 12:22:41.103066 16397 raft_consensus.cc:2889] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:22:41.087460 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:41.087666 16514 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:41.091930 18279 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:41.060643 18207 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:41.095405 16441 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:41.001389 18386 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:41.095633 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:41.096909 16442 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:41.099068 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:41.105229 16543 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:41.107105 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:41.106496 16399 pending_rounds.cc:85] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 3
I20260709 12:22:41.108711 16399 pending_rounds.cc:107] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:22:41.108980 16399 raft_consensus.cc:2889] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:22:41.097887 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:41.115442 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:41.105657 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:41.113232 16516 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:41.113750 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:41.119488 16543 raft_consensus.cc:2393] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 4: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:41.119681 16472 raft_consensus.cc:1275] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:41.120321 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:41.121181 16551 raft_consensus.cc:2410] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:22:41.124559 16396 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:41.125242 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:41.125960 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:41.124895 16515 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:41.120499 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:41.129508 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:41.130391 17261 raft_consensus.cc:1275] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:41.130750 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:41.131232 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:41.130499 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:41.133255 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:41.070544 18396 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:41.134914 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:22:41.141215 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 45910us
I20260709 12:22:41.141709 17261 raft_consensus.cc:2410] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:22:41.136760 17258 raft_consensus.cc:1240] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:22:41.137971 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:41.143558 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:41.143774 16468 raft_consensus.cc:2468] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
W20260709 12:22:41.143997 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:41.128980 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:41.145028 17263 raft_consensus.cc:2393] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 6: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:41.103700 18116 consensus_queue.cc:1059] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 12:22:41.146982 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:41.114842 17665 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:41.147537 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:41.117514 18207 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:41.154458 17261 raft_consensus.cc:1275] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:41.123965 18332 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:41.152719 17266 raft_consensus.cc:1275] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:41.155915 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:41.156550 17258 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 2.
I20260709 12:22:41.158437 17780 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:41.158607 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:41.163060 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:41.126936 18081 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:41.139958 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:41.176568 18350 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:41.141378 18168 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:22:41.178447 18407 log.cc:927] Time spent T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 12:22:41.178303 18271 consensus_queue.cc:237] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:41.156643 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:41.181396 18386 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:41.167510 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:41.209152 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:41.145004 18286 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:41.185353 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:41.161759 18267 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:41.213647 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:41.183779 18081 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:22:41.222143 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:41.226554 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:41.208799 18047 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:41.208889 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:41.221704 17142 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:41.242882 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:41.244040 16551 raft_consensus.cc:2393] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 4: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:41.164098 17665 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:41.215360 16542 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6.
I20260709 12:22:41.186054 17780 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:41.246142 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:41.160844 18298 consensus_queue.cc:260] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:41.184114 18385 consensus_queue.cc:1059] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 12:22:41.216429 18166 consensus_queue.cc:1048] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:22:41.235666 18081 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:41.255733 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 122207us
I20260709 12:22:41.223779 18286 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:41.238276 18267 raft_consensus.cc:697] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:41.269469 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:41.260953 16515 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:41.219156 18168 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:41.271684 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:41.272256 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:41.255431 18360 raft_consensus.cc:2804] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:41.281720 16515 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:41.267249 17143 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:41.242290 18047 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:41.268136 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:41.268739 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:41.298806 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:41.299317 17258 raft_consensus.cc:2376] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4. Re-sending same reply.
I20260709 12:22:41.249269 18153 consensus_queue.cc:1048] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
W20260709 12:22:41.310855 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 43725.5us
I20260709 12:22:41.303750 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:41.310855 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:41.313097 16397 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:22:41.302598 16515 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:41.308360 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:41.284404 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:41.282172 16398 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:41.313131 16441 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:41.339844 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:41.283089 18360 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:41.268750 18286 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:41.309386 18359 raft_consensus.cc:3055] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:41.308842 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:41.313585 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:41.321619 18332 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:41.319291 18019 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:22:41.321501 18419 log.cc:927] Time spent T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:22:41.349953 16515 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:41.388260 17143 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:41.390100 17143 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:41.313695 18348 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:41.315109 18002 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:41.352737 18161 raft_consensus.cc:3055] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:41.347784 17144 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:41.399080 17258 raft_consensus.cc:1275] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:22:41.399585 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 51892.2us
W20260709 12:22:41.324496 18418 log.cc:927] Time spent T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:22:41.369117 18429 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:22:41.370229 18417 log.cc:927] Time spent T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:22:41.371598 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:41.422907 16470 raft_consensus.cc:1275] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 12:22:41.423563 18431 log.cc:927] Time spent T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 12:22:41.372605 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:41.424705 16397 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:41.372263 18387 consensus_queue.cc:1048] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 12:22:41.373600 18047 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:41.376478 18267 consensus_queue.cc:237] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:41.378125 17665 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:41.379900 18416 log.cc:927] Time spent T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 12:22:41.385623 17780 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:41.346906 18401 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:41.347280 18298 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:41.391911 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:41.322139 18408 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:41.347434 18360 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:41.402130 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:41.428226 18329 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:41.401437 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:41.432307 17871 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:41.398398 18066 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:41.406733 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:41.406934 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:41.406874 18413 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:41.407316 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:41.354164 18359 raft_consensus.cc:740] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:41.397068 18348 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:41.404451 18389 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:41.399260 18161 raft_consensus.cc:740] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:22:41.478314 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:41.408964 16401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 374) took 1360 ms. Trace:
I20260709 12:22:41.432489 18429 consensus_queue.cc:1048] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 12:22:41.410862 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:41.414906 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:41.416515 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:41.417021 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:41.418570 16516 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:41.419188 18081 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:41.409080 16441 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:41.402659 16543 raft_consensus.cc:1275] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:41.405839 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:41.460217 16400 raft_consensus.cc:1275] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:41.487646 16473 raft_consensus.cc:1240] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:22:41.447253 18351 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:41.467651 18395 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:41.465509 18066 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:22:41.475127 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 85155.1us
I20260709 12:22:41.404068 18153 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:41.466503 17871 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:41.454862 18401 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:41.487023 17258 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
W20260709 12:22:41.493032 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 74546.6us
I20260709 12:22:41.479796 16401 rpcz_store.cc:276] 0709 12:22:40.048718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.048873 (+   155us) service_pool.cc:224] Handling call
0709 12:22:40.051403 (+  2530us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:40.127976 (+ 76573us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:40.128008 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:40.969950 (+841942us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b2c0004c948. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:22:40.970760 (+   810us) log.cc:844] Serialized 26 byte log entry
0709 12:22:41.117065 (+146305us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:41.117112 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:41.363579 (+246467us) spinlock_profiling.cc:243] Waited 63.3 ms on lock 0x7b5400a79ed4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:41.363640 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:41.408740 (+ 45100us) raft_consensus.cc:1685] finished
0709 12:22:41.408760 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:41.408906 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6594,"raft.queue_time_us":24740,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":14558,"spinlock_wait_cycles":183679104,"thread_start_us":145840,"threads_started":1,"wal-append.queue_time_us":252576}
I20260709 12:22:41.496230 17143 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:41.476349 18348 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:41.539112 17143 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
W20260709 12:22:41.539923 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 130913us
W20260709 12:22:41.512825 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:41.541930 17143 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:41.465739 18329 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:41.547235 16516 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:41.543989 17143 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:41.501935 18066 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:41.500664 17665 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:41.537617 18401 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:41.503190 17871 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:41.543320 16471 raft_consensus.cc:606] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Received request to transfer leadership to TS f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:41.545853 18209 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:41.547917 17780 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:41.565920 17861 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:41.501148 16394 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:41.502691 18395 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:41.569648 18247 consensus_meta.cc:287] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.530s	user 0.009s	sys 0.005s
I20260709 12:22:41.573382 18013 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:41.633220 18222 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:41.618489 18166 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:41.653647 16397 raft_consensus.cc:1275] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:41.619740 18389 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:41.500847 18351 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:41.633440 18047 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:41.633868 18209 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:41.663559 16396 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:22:41.647310 18408 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:22:41.667706 18426 log.cc:927] Time spent T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 12:22:41.668015 18439 consensus_queue.cc:1048] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.189s
I20260709 12:22:41.646132 17861 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:41.658282 18168 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:41.652932 18222 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:41.664772 18429 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:41.723263 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 231214us
I20260709 12:22:41.668665 18293 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:41.805241 16515 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:41.569741 18161 consensus_queue.cc:260] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:41.660923 18403 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:22:41.762753 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 204732us
I20260709 12:22:41.667107 18209 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:41.764492 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 223819us
I20260709 12:22:41.645526 17780 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:41.729859 18408 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:41.761237 18273 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:41.677785 18047 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:41.814195 16442 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.744s (SENT) [suppressed 4 similar messages]
W20260709 12:22:41.824460 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:41.823046 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:41.825089 16545 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:41.815976 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:41.852015 16477 raft_consensus.cc:1240] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:41.854005 16545 raft_consensus.cc:1275] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:41.881217 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:41.882761 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:41.889505 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:41.890527 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:41.891196 17266 raft_consensus.cc:2376] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4. Re-sending same reply.
I20260709 12:22:41.892222 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:41.815426 18347 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:41.907977 17258 raft_consensus.cc:1275] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:22:41.908520 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 361374us
I20260709 12:22:41.930785 17143 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:41.965732 17271 raft_consensus.cc:1275] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:41.822798 18408 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:41.803537 18429 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:41.993942 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:41.970517 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:41.890300 18421 log.cc:927] Time spent T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:22:41.857543 17868 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:22:41.988008 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 57360us
I20260709 12:22:41.829046 18273 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:41.813584 18403 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:41.965188 18329 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:41.987481 18371 log.cc:927] Time spent T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 12:22:42.018890 18013 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:42.019688 18013 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:41.855378 18457 consensus_queue.cc:1048] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.178s
I20260709 12:22:41.985225 18081 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:41.988411 18455 raft_consensus.cc:2804] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:42.000788 18447 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:42.034744 18433 log.cc:927] Time spent T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.386s	user 0.000s	sys 0.001s
I20260709 12:22:42.012581 18414 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:42.015329 18008 raft_consensus.cc:2804] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:22:42.094298 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:42.095099 18267 consensus_queue.cc:1059] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:42.015944 18395 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:42.087185 18374 log.cc:927] Time spent T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:22:42.047521 18455 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:42.035219 17868 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:42.100404 18008 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:42.152813 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 269999us
I20260709 12:22:42.117386 18341 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:42.163937 16441 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:42.157523 18371 log.cc:927] Time spent T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 12:22:42.157090 18444 log.cc:927] Time spent T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.422s	user 0.001s	sys 0.000s
W20260709 12:22:42.157194 18066 consensus_meta.cc:287] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.538s	user 0.002s	sys 0.015s
I20260709 12:22:42.144544 18455 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:42.163379 18008 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:42.186450 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 276640us
I20260709 12:22:42.162688 18351 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:42.098438 18447 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:42.119486 18414 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:42.169212 17861 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:42.188414 18457 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:42.211110 18466 consensus_queue.cc:1048] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:42.213164 18466 consensus_queue.cc:1048] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:42.207547 16399 raft_consensus.cc:1240] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 12:22:42.209123 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:42.220775 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:42.221686 16441 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:42.221745 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:42.185559 18222 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:42.164749 18341 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:42.192565 18247 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:42.246388 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:42.187873 18066 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:42.283927 16544 raft_consensus.cc:1240] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:42.285524 18066 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:42.287350 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:42.247924 16441 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
W20260709 12:22:42.292451 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 44620.3us
I20260709 12:22:42.250325 18247 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:42.257289 18013 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:22:42.261022 18446 log.cc:927] Time spent T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 12:22:42.262184 18453 log.cc:927] Time spent T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 12:22:42.264518 17868 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:42.265268 18447 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:42.265914 18403 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:42.268234 18429 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:42.190246 18459 consensus_queue.cc:1059] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.068s
I20260709 12:22:42.274642 18414 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:42.522677 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:22:42.278540 17871 consensus_meta.cc:287] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.643s	user 0.005s	sys 0.005s
I20260709 12:22:42.528506 17264 raft_consensus.cc:1240] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:42.530033 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:42.530743 17264 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
W20260709 12:22:42.282507 18360 consensus_meta.cc:287] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.812s	user 0.016s	sys 0.001s
I20260709 12:22:42.248502 17780 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:42.313519 18468 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:42.540638 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:42.542222 17269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 654) took 1300 ms. Trace:
I20260709 12:22:42.541937 16367 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:42.413609 16441 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:42.423784 16549 raft_consensus.cc:1240] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:42.426649 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:42.241587 18350 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:42.430848 16545 raft_consensus.cc:1275] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:22:42.485577 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:42.295264 18267 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:42.523751 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:42.524080 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:42.524664 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:42.524801 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:42.416726 18452 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:42.544426 16545 pending_rounds.cc:85] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1: Aborting all ops after (but not including) 1
I20260709 12:22:42.560997 16545 pending_rounds.cc:107] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:22:42.561333 16545 raft_consensus.cc:2889] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:22:42.549219 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 133185us
I20260709 12:22:42.542732 17269 rpcz_store.cc:276] 0709 12:22:41.241699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.249647 (+  7948us) service_pool.cc:224] Handling call
0709 12:22:41.258819 (+  9172us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:41.314460 (+ 55641us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:41.314506 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:41.314564 (+    58us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:41.314605 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:42.541977 (+1227372us) spinlock_profiling.cc:243] Waited 55.4 ms on lock 0x7b540005df54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:42.542027 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:42.542166 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":127504768}
I20260709 12:22:42.468444 17868 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:42.544824 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:42.544183 16400 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:22:42.558703 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:42.560199 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:42.560413 18395 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:42.519793 18474 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:42.524217 16394 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.003s	sys 0.008s
I20260709 12:22:42.742288 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:42.742570 16394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 396) took 1406 ms. Trace:
I20260709 12:22:42.743072 18207 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:22:42.590021 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:42.691057 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:42.743248 16394 rpcz_store.cc:276] 0709 12:22:41.336454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.661576 (+325122us) service_pool.cc:224] Handling call
0709 12:22:41.756527 (+ 94951us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:41.812012 (+ 55485us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:41.817073 (+  5061us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:41.993487 (+176414us) log.cc:844] Serialized 26 byte log entry
0709 12:22:42.741807 (+748320us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:42.741885 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:42.742013 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:42.742086 (+    73us) raft_consensus.cc:1685] finished
0709 12:22:42.742111 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:42.742509 (+   398us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2504,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":6795,"thread_start_us":726263,"threads_started":1,"wal-append.queue_time_us":725161}
I20260709 12:22:42.699375 17868 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:42.745146 16548 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
I20260709 12:22:42.522305 18429 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:42.745991 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:42.746363 18429 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:42.543080 18457 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:42.747468 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:42.747941 18457 raft_consensus.cc:697] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:42.748229 16394 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:42.748889 18457 consensus_queue.cc:237] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:42.751978 18247 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:42.743700 18207 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:42.668883 18452 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:42.541481 17871 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:42.757278 18410 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:42.758076 18271 consensus_queue.cc:1048] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:42.560621 18350 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:42.519106 18323 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.003s	sys 0.008s
I20260709 12:22:42.697808 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:42.761602 16545 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:22:42.703724 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:42.559686 18459 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:42.563043 18273 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:42.712757 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:42.713748 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:42.763096 18459 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:42.542651 18414 consensus_queue.cc:1048] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.159s
I20260709 12:22:42.716626 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:42.765839 17266 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:42.534807 18222 consensus_queue.cc:1059] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.160s
I20260709 12:22:42.721611 18480 consensus_queue.cc:1059] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:42.670253 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:42.698890 18389 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.056s	sys 0.098s
I20260709 12:22:42.741127 18474 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:42.540467 18360 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:42.720619 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:22:42.855916 18323 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.004s	sys 0.008s
I20260709 12:22:42.762665 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:42.856772 18273 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:42.859686 18222 raft_consensus.cc:3055] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:42.860633 16470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 857) took 1567 ms. Trace:
I20260709 12:22:42.906369 18222 raft_consensus.cc:740] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:42.906441 16470 rpcz_store.cc:276] 0709 12:22:41.271397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.423753 (+152356us) service_pool.cc:224] Handling call
0709 12:22:41.424111 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:41.450441 (+ 26330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:41.457216 (+  6775us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:41.543681 (+ 86465us) log.cc:844] Serialized 26 byte log entry
0709 12:22:41.811795 (+268114us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:22:41.824849 (+ 13054us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:41.853657 (+ 28808us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:42.838642 (+984985us) raft_consensus.cc:1685] finished
0709 12:22:42.838678 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:42.838910 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":21575,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":8761,"thread_start_us":172320,"threads_started":1,"wal-append.queue_time_us":428966}
I20260709 12:22:42.908087 18474 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:42.907789 18222 consensus_queue.cc:260] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:42.909363 18222 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:42.910616 17270 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:22:42.688587 18464 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:22:42.877492 18389 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.856s	user 0.056s	sys 0.098s
I20260709 12:22:42.896124 18161 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:42.902278 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:42.899143 16542 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:42.903463 18207 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:42.903570 17868 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:22:42.904136 18298 consensus_meta.cc:287] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 1.441s	user 0.016s	sys 0.033s
I20260709 12:22:42.904971 18323 raft_consensus.cc:3055] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:42.905689 16394 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:42.932256 17144 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:42.701399 18468 consensus_queue.cc:1048] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.205s
I20260709 12:22:42.897302 17254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 572) took 1103 ms. Trace:
I20260709 12:22:42.945454 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:22:42.946581 17142 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.729s	user 0.115s	sys 0.190s
W20260709 12:22:42.955940 17142 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.115s	sys 0.190s
I20260709 12:22:42.955410 16400 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:22:42.958248 17144 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:42.936149 18403 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:42.906324 18273 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:42.950667 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 380756us
I20260709 12:22:42.940244 18341 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:42.945957 18323 raft_consensus.cc:740] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:42.953979 18358 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:42.953246 18350 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:42.969189 17266 raft_consensus.cc:1275] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:42.971719 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:42.933470 17871 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:42.968302 18439 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:42.975839 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:22:42.972810 16443 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.043s	sys 0.078s
W20260709 12:22:42.982128 16443 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.043s	sys 0.078s
I20260709 12:22:42.981809 17269 raft_consensus.cc:1240] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:42.986568 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:42.963855 18447 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:42.935516 18464 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:42.970253 18358 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:42.954763 18474 consensus_queue.cc:1048] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 12:22:42.970124 18298 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:42.998584 18452 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:43.003075 18429 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:43.005890 18298 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:43.007345 18401 consensus_meta.cc:287] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 1.375s	user 0.011s	sys 0.017s
I20260709 12:22:43.007745 18401 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:43.009748 18401 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:43.015080 18408 consensus_meta.cc:287] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.925s	user 0.011s	sys 0.005s
I20260709 12:22:43.015522 18408 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:42.974305 18414 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:42.969826 18481 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:42.965713 18403 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:42.982190 18410 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:42.974387 18480 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:22:43.034300 18008 consensus_meta.cc:287] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.828s	user 0.013s	sys 0.002s
W20260709 12:22:43.034765 18332 consensus_meta.cc:287] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 1.671s	user 0.010s	sys 0.015s
I20260709 12:22:43.034412 18480 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:43.035220 18332 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:43.020632 18323 consensus_queue.cc:260] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:43.036806 18480 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:43.036844 18323 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:43.034003 18410 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:43.024194 16367 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:42.990235 18447 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:43.008167 18298 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:43.044512 16367 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:43.031672 18273 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:43.032437 18348 consensus_meta.cc:287] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 1.475s	user 0.018s	sys 0.011s
I20260709 12:22:43.001504 18358 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:42.993265 18459 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:43.031126 18490 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:43.047781 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:43.029595 18401 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:43.033710 18247 consensus_queue.cc:1048] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 12:22:43.080768 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:22:43.084069 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 242298us
I20260709 12:22:43.058928 16367 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:43.045547 17868 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:43.033105 18482 consensus_queue.cc:1048] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 12:22:43.085819 17266 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:43.091110 17258 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:42.984632 18439 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:43.040392 18414 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:43.044317 18447 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:43.092638 16367 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:22:43.095120 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 130779us
I20260709 12:22:43.035024 18008 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:43.079478 18298 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:43.088614 18385 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:43.099264 16442 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:43.094697 18286 consensus_meta.cc:287] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 1.730s	user 0.022s	sys 0.009s
W20260709 12:22:43.106047 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 135311us
I20260709 12:22:43.108740 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:43.109215 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:43.109620 16398 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:43.110890 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:43.112175 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:43.113196 16398 raft_consensus.cc:2410] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:22:43.095742 18480 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:43.115744 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:43.109091 17144 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:43.085121 18267 raft_consensus.cc:697] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:43.099898 18360 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:43.118142 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:43.090767 18490 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:43.121622 18490 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:43.087642 18348 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:43.125027 16428 heartbeater.cc:646] Failed to heartbeat to 127.5.58.62:39363 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.5.58.62:39363 timed out after 15.000s (SENT)
I20260709 12:22:43.126158 16551 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:43.126318 18348 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:43.130995 16442 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:43.133975 16442 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:43.134507 17258 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 3.
I20260709 12:22:43.093628 18401 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:43.137645 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:43.139825 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:43.136584 17144 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:43.042925 18481 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:43.138464 18490 raft_consensus.cc:2804] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:43.154791 18490 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:43.155310 18490 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:43.157027 18481 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:43.138099 16442 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:43.135262 18483 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:43.158308 18483 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:43.158676 18483 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:43.160048 18455 consensus_meta.cc:287] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.962s	user 0.008s	sys 0.000s
I20260709 12:22:42.955206 17254 rpcz_store.cc:276] 0709 12:22:41.785945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.805817 (+ 19872us) service_pool.cc:224] Handling call
0709 12:22:41.890178 (+ 84361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:42.023250 (+133072us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:42.034189 (+ 10939us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:42.167619 (+133430us) log.cc:844] Serialized 26 byte log entry
0709 12:22:42.522267 (+354648us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:42.539704 (+ 17437us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:42.559387 (+ 19683us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:42.858125 (+298738us) raft_consensus.cc:1685] finished
0709 12:22:42.858168 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:42.889667 (+ 31499us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23591,"raft.queue_time_us":40115,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":115,"thread_start_us":228159,"threads_started":1,"wal-append.queue_time_us":528534}
I20260709 12:22:43.108399 18491 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:43.165323 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:43.165990 16395 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:22:43.166143 18360 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:43.166779 18360 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:43.167215 18360 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:43.137025 18494 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:43.141616 18457 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:43.168815 16403 raft_consensus.cc:1275] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:43.114239 18385 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:43.172647 18008 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:22:43.162640 18209 consensus_meta.cc:287] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 1.339s	user 0.014s	sys 0.006s
I20260709 12:22:43.135504 18389 consensus_queue.cc:1059] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 12:22:43.150097 18414 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:43.149729 18408 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:43.153957 18401 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:43.180711 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:43.173573 18481 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:43.152235 16394 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:22:43.214375 18447 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:43.184887 18347 consensus_meta.cc:287] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 1.213s	user 0.011s	sys 0.002s
W20260709 12:22:43.216918 18430 log.cc:927] Time spent T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 1.929s	user 0.008s	sys 0.005s
I20260709 12:22:43.217374 16399 raft_consensus.cc:1240] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:43.218196 16472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 293) took 2058 ms. Trace:
W20260709 12:22:43.187614 18438 log.cc:927] Time spent T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 1.464s	user 0.007s	sys 0.005s
I20260709 12:22:43.218602 16472 rpcz_store.cc:276] 0709 12:22:41.159680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.159808 (+   128us) service_pool.cc:224] Handling call
0709 12:22:41.160172 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:41.160646 (+   474us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:41.160687 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:41.182678 (+ 21991us) log.cc:844] Serialized 26 byte log entry
0709 12:22:41.260574 (+ 77896us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:41.260617 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:41.260699 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:43.173320 (+1912621us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:22:43.217905 (+ 44585us) raft_consensus.cc:1685] finished
0709 12:22:43.217935 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:43.218139 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2673,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":20080,"spinlock_wait_cycles":25791872,"thread_start_us":74289,"threads_started":1,"wal-append.queue_time_us":104552}
I20260709 12:22:43.169831 18455 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:43.220212 18060 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:43.220837 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.222198 18455 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:43.220837 18060 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:43.189229 18432 log.cc:927] Time spent T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 1.781s	user 0.017s	sys 0.014s
I20260709 12:22:43.193522 17144 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:43.224689 18455 consensus_queue.cc:1048] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:22:43.198112 18293 consensus_meta.cc:287] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 1.387s	user 0.014s	sys 0.006s
I20260709 12:22:43.235232 16477 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:22:43.238665 18408 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:43.239024 16477 pending_rounds.cc:85] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 5
I20260709 12:22:43.239346 16477 pending_rounds.cc:107] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:22:43.239306 18408 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:43.234083 18455 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:43.240594 18136 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:43.240676 18455 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:43.241321 18136 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:43.242250 18408 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:43.242393 18360 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:43.242913 18408 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:43.243088 18455 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:43.199630 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.209722 17261 raft_consensus.cc:2393] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 7: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:43.206825 18209 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:43.207681 18279 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:22:43.221094 18436 log.cc:927] Time spent T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 1.796s	user 0.013s	sys 0.009s
I20260709 12:22:43.194967 16473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 302) took 1022 ms. Trace:
W20260709 12:22:43.226902 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 94259.7us
I20260709 12:22:43.115018 18286 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:43.226179 18385 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:43.228070 18295 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:43.227967 18347 consensus_queue.cc:1059] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:43.228402 16397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 721) took 1899 ms. Trace:
I20260709 12:22:43.228497 18410 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:43.230060 18439 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:43.231269 18066 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:43.231760 18397 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:43.231685 18358 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:43.189638 18267 consensus_queue.cc:237] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:43.161554 18403 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:43.239869 16477 raft_consensus.cc:2889] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:43.205317 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.235925 18293 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:43.251209 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.261032 16546 raft_consensus.cc:2393] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 4: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:43.260897 18468 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:43.264559 16474 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53186 (request call id 291) took 2125 ms (client timeout 1739 ms). Trace:
I20260709 12:22:43.264649 18468 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:43.264979 16474 rpcz_store.cc:269] 0709 12:22:41.138725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.138873 (+   148us) service_pool.cc:224] Handling call
0709 12:22:43.251051 (+2112178us) spinlock_profiling.cc:243] Waited 1.96 ms on lock 0x7b5800602a98. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:22:43.264472 (+ 13421us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4498816}
I20260709 12:22:43.266513 18060 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:43.267794 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:43.268499 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:43.268621 16550 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
W20260709 12:22:43.269074 18445 log.cc:927] Time spent T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 1.505s	user 0.000s	sys 0.008s
I20260709 12:22:43.243991 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:43.269795 18209 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:43.267885 18468 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:43.271930 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.224164 16544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 845) took 1405 ms. Trace:
I20260709 12:22:43.271272 16397 rpcz_store.cc:276] 0709 12:22:41.329428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.545410 (+215982us) service_pool.cc:224] Handling call
0709 12:22:41.637839 (+ 92429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:41.659817 (+ 21978us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:43.228171 (+1568354us) spinlock_profiling.cc:243] Waited 90.7 ms on lock 0x7b54009bd7d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:43.228319 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208551680}
I20260709 12:22:43.262985 18286 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:43.247210 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.273518 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:43.226429 17144 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:43.274864 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 48516.3us
I20260709 12:22:43.276242 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:43.276940 16549 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:43.278522 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:43.270740 16401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 394) took 1920 ms. Trace:
I20260709 12:22:43.236389 16515 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:43.274086 16475 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:22:43.235368 16476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 288) took 2110 ms. Trace:
I20260709 12:22:43.305703 16472 raft_consensus.cc:1240] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 6. Current term is 7. Ops: []
W20260709 12:22:43.305617 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 185772us
W20260709 12:22:43.306941 17261 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45224 (request call id 656) took 2041 ms (client timeout 2162 ms). Trace:
W20260709 12:22:43.307449 17261 rpcz_store.cc:269] 0709 12:22:41.265424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.268474 (+  3050us) service_pool.cc:224] Handling call
0709 12:22:43.180518 (+1912044us) spinlock_profiling.cc:243] Waited 1 ms on lock 0x7b58003b8298. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:22:43.306884 (+126366us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2303872}
I20260709 12:22:43.272816 16544 rpcz_store.cc:276] 0709 12:22:41.819140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.819293 (+   153us) service_pool.cc:224] Handling call
0709 12:22:41.819575 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:42.284628 (+465053us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:43.223945 (+939317us) spinlock_profiling.cc:243] Waited 464 ms on lock 0x7b54004d7454. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:43.224106 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1067310848}
I20260709 12:22:43.309279 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:43.309554 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:43.309939 17258 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:43.310424 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:43.270787 16473 rpcz_store.cc:276] 0709 12:22:42.172573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:42.172751 (+   178us) service_pool.cc:224] Handling call
0709 12:22:42.189773 (+ 17022us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:42.687431 (+497658us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:42.687475 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:42.711519 (+ 24044us) log.cc:844] Serialized 26 byte log entry
0709 12:22:43.106711 (+395192us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:43.106762 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:43.106888 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:43.194270 (+ 87382us) raft_consensus.cc:1685] finished
0709 12:22:43.194301 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:43.194876 (+   575us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":24941,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":111,"thread_start_us":629719,"threads_started":2,"wal-append.queue_time_us":250093}
I20260709 12:22:43.300693 16401 rpcz_store.cc:276] 0709 12:22:41.320234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.541139 (+220905us) service_pool.cc:224] Handling call
0709 12:22:41.562942 (+ 21803us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:42.088678 (+525736us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:22:42.092877 (+  4199us) log.cc:844] Serialized 14 byte log entry
0709 12:22:42.245721 (+152844us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:42.245769 (+    48us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:42.245806 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:43.240082 (+994276us) spinlock_profiling.cc:243] Waited 525 ms on lock 0x7b54007634d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:43.240134 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:43.240263 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6384,"prepare.queue_time_us":4305,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":48,"raft.queue_time_us":2716,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":9819,"spinlock_wait_cycles":1210224128,"thread_start_us":149878,"threads_started":1,"wal-append.queue_time_us":441700}
I20260709 12:22:43.316309 16516 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:43.320349 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 84076.8us
I20260709 12:22:43.321209 17271 raft_consensus.cc:1240] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:43.322255 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:43.322593 16544 raft_consensus.cc:1240] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:22:43.323542 16549 raft_consensus.cc:1240] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:43.325004 17261 raft_consensus.cc:2368] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4.
I20260709 12:22:43.303777 16476 rpcz_store.cc:276] 0709 12:22:41.125252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.125387 (+   135us) service_pool.cc:224] Handling call
0709 12:22:41.125679 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:41.133062 (+  7383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:41.133095 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:41.146696 (+ 13601us) log.cc:844] Serialized 26 byte log entry
0709 12:22:41.310828 (+164132us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:22:41.310874 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:41.310942 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:43.188708 (+1877766us) spinlock_profiling.cc:243] Waited 21.5 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:22:43.227192 (+ 38484us) raft_consensus.cc:1685] finished
0709 12:22:43.227214 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:43.235244 (+  8030us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":511,"raft.queue_time_us":6842,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":13257,"spinlock_wait_cycles":49373568,"thread_start_us":100390,"threads_started":1,"wal-append.queue_time_us":243186}
I20260709 12:22:43.327312 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:43.328156 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:43.328948 17263 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:43.245096 18360 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:43.329510 16367 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:43.336483 16516 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:43.271134 18295 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:43.251441 18455 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:43.338222 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:43.346465 16549 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:43.243556 18408 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:43.334146 17266 raft_consensus.cc:2393] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 4: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:43.238392 18447 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:43.243587 18136 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:43.237105 18047 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:43.309989 18171 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:43.337041 16367 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:43.237738 18483 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:43.342370 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.343155 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:43.357453 16397 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:43.330895 16515 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:43.359429 16515 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:43.350466 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:43.361616 16401 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:22:43.319118 18081 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:43.232306 18490 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:43.356232 16367 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:43.337165 18380 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:43.310817 18066 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:43.307242 18279 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:43.344733 18295 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:43.344313 18247 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:43.307010 18397 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:43.344663 18206 raft_consensus.cc:2804] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:43.366043 16367 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:43.348181 18286 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:43.366605 18136 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:43.352149 18455 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:43.384626 18483 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:43.401528 18299 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:43.402019 18299 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:43.402155 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:43.402470 18299 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:43.376758 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.383319 17258 raft_consensus.cc:1240] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:22:43.369360 18348 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:43.384552 18273 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:43.366792 18380 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:43.386680 18408 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:43.368813 18347 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:43.387506 16471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:39000 (request call id 17) took 2306 ms. Trace:
I20260709 12:22:43.336829 18491 consensus_queue.cc:1048] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 12:22:43.368993 18171 raft_consensus.cc:697] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:43.390213 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:43.366605 18081 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:43.337010 18414 consensus_queue.cc:1059] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 12:22:43.362090 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:43.444965 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:43.407457 16549 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:43.245453 18359 consensus_queue.cc:260] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:43.369211 18222 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:22:43.389824 18499 log.cc:927] Time spent T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260709 12:22:43.392699 18247 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:43.418488 16545 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:43.441399 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:43.466389 16547 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:43.413004 18380 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:43.461915 17255 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:43.443162 16397 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:22:43.400354 18206 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:43.429569 16401 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:22:43.463356 18323 consensus_queue.cc:1059] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 12:22:43.447916 16441 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:43.456746 18359 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:43.450511 18447 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:43.459466 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:43.409262 18136 raft_consensus.cc:697] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 3 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:43.414597 18273 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:43.403501 18455 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:43.412081 18452 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:43.419314 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:43.414000 18298 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:43.414243 18347 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:43.472383 17144 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:43.473595 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.440860 18386 consensus_queue.cc:1059] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 12:22:43.454221 18222 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:22:43.490211 16551 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52680 (request call id 289) took 2184 ms (client timeout 1958 ms). Trace:
I20260709 12:22:43.469177 18247 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:43.460630 18358 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:43.491693 16399 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:43.537092 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 64863.1us
I20260709 12:22:43.491818 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:43.537917 16401 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:43.536423 16473 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:43.492727 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:22:43.494042 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 46191us
I20260709 12:22:43.497917 16543 raft_consensus.cc:2410] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:22:43.481385 18206 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:43.527868 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:43.419137 16471 rpcz_store.cc:276] 0709 12:22:41.081322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.081558 (+   236us) service_pool.cc:224] Handling call
0709 12:22:43.387254 (+2305696us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b5400a69fd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e83853 00007f287ae5160a 00007f28743d77d4 00007f28743d7762 00007f28743d7691 00007f28743d760c 00007f28743d68c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:43.387397 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":984560512}
I20260709 12:22:43.535617 16547 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:22:43.499462 18410 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:22:43.534755 16551 rpcz_store.cc:269] 0709 12:22:41.301072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.303442 (+  2370us) service_pool.cc:224] Handling call
0709 12:22:43.484997 (+2181555us) spinlock_profiling.cc:243] Waited 1.81 s on lock 0x7b5400980654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:43.485143 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4172092288}
I20260709 12:22:43.461005 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.540477 17144 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:43.414710 18348 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:43.546933 16475 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:22:43.553894 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.537171 18222 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:43.544198 18360 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:43.575183 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.538095 18347 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:43.570755 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.591228 16469 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:43.592458 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:43.551416 18293 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:43.497201 18447 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:43.553093 18505 log.cc:927] Time spent T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 12:22:43.532655 18298 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:43.592783 16401 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:43.569379 18395 raft_consensus.cc:3055] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:43.581715 16399 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:22:43.599673 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:43.501806 18452 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:43.600616 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.562098 18410 raft_consensus.cc:697] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:43.605823 17144 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:43.607553 16544 raft_consensus.cc:1275] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:43.599929 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.617096 16473 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:43.618016 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.614310 16441 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:43.614449 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:43.619688 16545 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:43.619845 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:43.620764 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.613274 16397 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:43.619266 17144 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:43.624761 16441 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:43.625224 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.542450 18171 consensus_queue.cc:237] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:43.598423 18360 raft_consensus.cc:697] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:43.648473 16367 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:43.631407 18323 raft_consensus.cc:3055] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:43.631561 18298 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:43.621902 18481 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:43.632663 18457 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:43.621666 18161 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:43.627014 18401 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:43.619560 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:43.621766 18166 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:43.640877 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:43.626905 18066 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:43.642061 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:43.642156 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:43.643239 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.644810 17665 raft_consensus.cc:3055] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:43.678081 17665 raft_consensus.cc:740] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:43.644976 18464 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:43.648517 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.608148 18293 raft_consensus.cc:697] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:43.627705 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.613425 18395 raft_consensus.cc:740] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:43.618566 18490 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:43.617172 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.635751 18081 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:22:43.661032 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:43.669533 16396 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58430 (request call id 391) took 2504 ms (client timeout 2391 ms). Trace:
W20260709 12:22:43.693383 16396 rpcz_store.cc:269] 0709 12:22:41.151867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.416788 (+264921us) service_pool.cc:224] Handling call
0709 12:22:43.655794 (+2239006us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b54005e8dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:43.655901 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":410660608}
I20260709 12:22:43.670022 16400 raft_consensus.cc:1275] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:43.670955 16543 raft_consensus.cc:1275] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:43.650041 18323 raft_consensus.cc:740] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:43.651072 18358 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:43.624102 18271 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:43.667608 18403 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:43.650429 18279 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:43.605381 18136 consensus_queue.cc:237] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:43.656163 18457 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:43.693411 16473 raft_consensus.cc:2393] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 5: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:43.706143 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.704417 18271 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:43.677311 18401 raft_consensus.cc:697] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:43.667322 18481 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:43.706743 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:22:43.697921 18504 log.cc:927] Time spent T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 12:22:43.673184 18161 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:43.682950 18166 raft_consensus.cc:697] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:43.705639 18047 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:43.707268 18403 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:43.738253 18161 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:43.732466 16471 raft_consensus.cc:2393] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 6: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:43.765710 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.766387 16471 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:22:43.766556 18206 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:43.767117 18481 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:43.767402 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.744253 18447 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:43.731197 18413 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:43.772967 17255 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:43.774705 18206 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:43.745115 18401 consensus_queue.cc:237] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:43.772970 16545 raft_consensus.cc:2368] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 4. Current term is 5.
I20260709 12:22:43.753655 18449 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:43.781383 18449 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:43.715619 18397 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:43.719015 18358 raft_consensus.cc:697] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:43.786221 18358 consensus_queue.cc:237] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:43.768234 16471 raft_consensus.cc:2393] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 6: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:43.790575 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.792793 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.749236 16476 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:43.758373 18323 consensus_queue.cc:260] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:43.801895 18273 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:43.802651 18323 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:43.709784 18457 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:43.817519 18454 log.cc:927] Time spent T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 1.644s	user 0.007s	sys 0.003s
I20260709 12:22:43.821936 18247 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:43.732679 18360 consensus_queue.cc:237] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:43.762845 18293 consensus_queue.cc:237] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:43.853740 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:43.695799 18410 consensus_queue.cc:237] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:43.862068 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.862886 16469 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:22:43.746881 18166 consensus_queue.cc:237] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:43.866681 17868 raft_consensus.cc:3055] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:43.867228 17868 raft_consensus.cc:740] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
W20260709 12:22:43.870713 18508 log.cc:927] Time spent T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.244s	user 0.000s	sys 0.000s
I20260709 12:22:43.759181 18403 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:43.878299 18452 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:43.883414 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:43.885090 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:43.760025 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.748708 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.750645 18348 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:43.824682 18047 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:22:43.834369 16546 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52680 (request call id 281) took 2697 ms (client timeout 2184 ms). Trace:
I20260709 12:22:43.835805 16471 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:43.926856 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.841471 16542 raft_consensus.cc:1240] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:22:43.842367 16468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 299) took 2462 ms. Trace:
W20260709 12:22:43.935139 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 48416.7us
I20260709 12:22:43.916505 17144 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:43.942865 17144 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:43.925642 16546 rpcz_store.cc:269] 0709 12:22:41.137429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.137687 (+   258us) service_pool.cc:224] Handling call
0709 12:22:41.970310 (+832623us) spinlock_profiling.cc:243] Waited 4.01 ms on lock 0x7b58005e9298. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:22:43.833823 (+1863513us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54008c4bd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:43.834199 (+   376us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2907055232}
I20260709 12:22:43.839661 18222 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:43.889729 16544 raft_consensus.cc:1240] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:43.795512 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.891258 16549 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:22:43.958525 16549 raft_consensus.cc:606] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:22:43.959024 16549 raft_consensus.cc:614] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:43.897907 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:43.795209 18299 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:43.779295 18295 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:43.912879 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:43.914597 16477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 867) took 2057 ms. Trace:
I20260709 12:22:43.974110 16477 rpcz_store.cc:276] 0709 12:22:41.857163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.857658 (+   495us) service_pool.cc:224] Handling call
0709 12:22:41.857959 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:43.260684 (+1402725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:43.914328 (+653644us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54002f3854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:43.914521 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3166437120}
I20260709 12:22:43.915738 17271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 641) took 1394 ms. Trace:
I20260709 12:22:43.975561 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:43.916851 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 296102us
I20260709 12:22:43.919373 16442 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:43.909709 16476 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:22:43.924726 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:43.838833 18271 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:43.927809 16471 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:22:43.928045 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:43.951017 16547 raft_consensus.cc:1240] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:43.953524 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:43.933110 16468 rpcz_store.cc:276] 0709 12:22:41.380360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.483261 (+102901us) service_pool.cc:224] Handling call
0709 12:22:41.483633 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:41.484085 (+   452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:41.484125 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:41.538346 (+ 54221us) log.cc:844] Serialized 26 byte log entry
0709 12:22:41.853640 (+315294us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:41.853681 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:41.853743 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:43.816773 (+1963030us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:22:43.841983 (+ 25210us) raft_consensus.cc:1685] finished
0709 12:22:43.842013 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:43.842279 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40672,"raft.queue_time_us":2229,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":4696,"spinlock_wait_cycles":23017216,"thread_start_us":220820,"threads_started":1,"wal-append.queue_time_us":538987}
I20260709 12:22:43.973425 16543 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:22:43.976105 17271 rpcz_store.cc:276] 0709 12:22:42.521327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:42.522993 (+  1666us) service_pool.cc:224] Handling call
0709 12:22:43.252944 (+729951us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b58003b8298. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:22:43.253176 (+   232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:43.321630 (+ 68454us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:43.915441 (+593811us) spinlock_profiling.cc:243] Waited 67.8 ms on lock 0x7b5400281754. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:43.915667 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":189761664}
I20260709 12:22:43.963615 16397 raft_consensus.cc:1240] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:43.965489 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:43.968375 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:43.970700 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:43.921581 17665 consensus_queue.cc:260] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:43.976619 16549 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:22:43.982555 16469 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:43.977231 16514 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:43.989024 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 69749.9us
I20260709 12:22:43.990255 16442 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:43.993028 16395 raft_consensus.cc:2410] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:22:43.995007 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:43.997064 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:44.001806 17143 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:43.931506 18468 raft_consensus.cc:3055] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:44.009256 16403 raft_consensus.cc:1240] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:44.012957 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:44.013875 16396 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:43.943941 18047 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:44.015769 16399 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:22:44.005213 16514 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:44.022349 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:44.022590 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:43.964131 17872 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:43.949152 18414 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:44.022251 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:43.972385 18002 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:44.033931 16514 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:44.028879 16402 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 6: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:44.037030 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:44.035521 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:44.037946 16470 raft_consensus.cc:1240] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:44.040772 17270 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 8. Current term is 9. Ops: [7.5-8.6]
I20260709 12:22:44.041296 16514 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:44.043560 17269 raft_consensus.cc:1240] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:22:44.043606 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:44.045640 16514 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:43.947201 16515 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:44.071131 16469 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:22:44.071163 17266 raft_consensus.cc:1240] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:22:44.071614 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:44.081454 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:22:44.075186 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 150994us
W20260709 12:22:44.092957 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 101053us
I20260709 12:22:44.074860 16514 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:44.098991 16398 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:44.009577 18206 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:22:44.113473 18060 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.075s	sys 0.167s
I20260709 12:22:43.735283 18395 consensus_queue.cc:260] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:44.105341 16543 raft_consensus.cc:2393] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 4: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:44.107381 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:44.107735 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:22:44.108000 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 160889us
I20260709 12:22:44.134250 17264 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:22:44.134495 16515 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:44.071810 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:44.109126 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:44.135084 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:44.109719 16514 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:44.135442 17271 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:43.967294 18290 consensus_queue.cc:1059] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 12:22:44.113068 17266 raft_consensus.cc:1240] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:43.972743 18410 consensus_queue.cc:1048] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.068s
I20260709 12:22:43.972601 18466 consensus_queue.cc:1048] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.068s
I20260709 12:22:44.115587 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:44.011103 18468 raft_consensus.cc:740] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:44.184695 18468 consensus_queue.cc:260] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:44.186326 18468 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:44.189718 17269 raft_consensus.cc:1275] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:44.028373 17780 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:44.029726 18490 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:44.031729 17872 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:44.123355 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:44.032969 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:44.101940 18408 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:44.102522 18464 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:44.003350 18312 raft_consensus.cc:2749] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:44.127703 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:44.107487 18429 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:44.101742 18066 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:44.132081 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:44.040632 18414 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:44.123044 18397 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:44.119220 18455 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:44.142717 16514 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:44.262105 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 119461us
I20260709 12:22:44.121410 18449 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:44.056625 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:22:44.136075 18060 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.075s	sys 0.167s
I20260709 12:22:44.289616 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:44.129609 18413 consensus_queue.cc:1059] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 12:22:44.157325 16367 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:44.143702 17255 raft_consensus.cc:1240] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:22:44.245568 18247 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:44.166100 18299 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:44.299916 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:44.300843 16547 raft_consensus.cc:2410] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:22:44.301681 18047 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:44.303784 18047 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:44.304905 18279 raft_consensus.cc:3055] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:44.305459 18279 raft_consensus.cc:740] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:44.306460 18279 consensus_queue.cc:260] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:44.307701 18299 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:44.138372 18457 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:44.254241 16394 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:44.309952 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:44.310249 18457 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:44.168776 18206 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:44.311124 18206 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:44.311259 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:44.311583 18206 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:44.291592 17861 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:44.312960 17861 raft_consensus.cc:697] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:44.169588 16515 leader_election.cc:400] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4.
I20260709 12:22:44.313867 16515 leader_election.cc:403] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:22:44.169711 18452 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:44.172408 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:44.174045 18222 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:44.182740 18295 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:43.868248 17868 consensus_queue.cc:260] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:44.119719 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:44.179869 17665 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:44.221084 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:44.229671 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:44.317634 16366 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:44.251365 16398 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:44.324168 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:44.255751 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:44.324834 17261 raft_consensus.cc:1240] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:22:44.325218 17271 raft_consensus.cc:2393] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 7: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:44.326272 17861 consensus_queue.cc:237] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:44.327265 18380 consensus_meta.cc:287] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.842s	user 0.005s	sys 0.007s
I20260709 12:22:44.329087 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:44.259895 16396 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:22:44.230895 17872 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:44.265693 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 217997us
I20260709 12:22:44.257622 18429 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:44.112855 18267 consensus_queue.cc:1048] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:22:44.291867 16400 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:44.164974 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:44.247016 18464 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:22:44.307924 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 150710us
I20260709 12:22:44.257892 18066 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:44.316618 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 147158us
I20260709 12:22:44.336143 16515 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:44.336324 16367 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:44.290202 18397 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:44.290030 18455 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:44.319406 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:44.233305 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:44.319865 18332 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:44.327750 18380 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:44.343174 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:44.340399 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:44.347914 16394 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:44.348702 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:44.340389 16399 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:22:44.339839 18066 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:44.342453 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:44.364506 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 46958.1us
I20260709 12:22:44.358683 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:44.371830 16396 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:44.377496 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:44.383376 16546 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:44.383899 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:44.340682 18464 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:44.391778 18380 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:44.392546 18047 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:44.392982 18247 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:44.370229 17868 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:44.394541 16544 raft_consensus.cc:2393] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 5: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:44.404187 18491 raft_consensus.cc:3055] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:44.405577 18456 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:44.406661 18519 log.cc:927] Time spent T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 12:22:44.345609 18397 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:44.403316 16396 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:22:44.419342 16394 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
W20260709 12:22:44.420217 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 326117us
I20260709 12:22:44.348589 18408 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:44.348721 18403 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:44.345530 18455 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:44.432562 16546 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:22:44.393682 18290 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:44.337695 18429 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:44.435281 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:44.363737 18060 consensus_queue.cc:1059] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.074s
W20260709 12:22:44.446149 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 351626us
I20260709 12:22:44.448750 17142 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:44.438593 18247 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:44.429757 16549 raft_consensus.cc:2468] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:44.450150 17144 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:44.463711 16550 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:44.464774 16442 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:44.440306 18491 raft_consensus.cc:740] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:44.463759 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:44.468984 16546 raft_consensus.cc:2393] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 6: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:44.472780 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:44.461920 18408 raft_consensus.cc:697] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:44.470360 18247 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:44.498612 18267 raft_consensus.cc:2749] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:44.474164 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:44.472488 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:44.470809 16398 raft_consensus.cc:2393] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 5: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:44.470881 16442 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:44.536876 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:44.537922 17258 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:44.512745 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:44.479547 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:22:44.538691 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 67893.3us
W20260709 12:22:44.471429 18323 consensus_meta.cc:287] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.668s	user 0.013s	sys 0.000s
I20260709 12:22:44.544399 18401 consensus_queue.cc:1048] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:44.544939 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:44.545367 18206 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:44.542382 18267 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:22:44.547049 18521 log.cc:927] Time spent T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
W20260709 12:22:44.548266 18517 log.cc:927] Time spent T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
I20260709 12:22:44.548442 16515 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:22:44.548908 18524 log.cc:927] Time spent T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:22:44.549893 17665 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:44.550312 17665 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:44.550139 18066 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:44.550817 17665 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:44.551235 18491 consensus_queue.cc:260] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:44.551743 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:44.552642 18491 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:44.552798 18347 consensus_meta.cc:287] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.953s	user 0.010s	sys 0.002s
I20260709 12:22:44.546183 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:44.553928 16547 raft_consensus.cc:2368] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 8. Current term is 9.
I20260709 12:22:44.553089 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:44.498920 18295 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:44.449046 18456 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:44.561374 17665 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:44.509361 17255 raft_consensus.cc:1240] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:44.513944 17264 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:44.566011 16442 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:44.566260 17665 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:44.556005 18206 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:44.568231 17872 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:44.570478 18295 raft_consensus.cc:697] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:44.572594 18267 consensus_queue.cc:1059] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:44.573022 18066 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:44.553283 18347 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:44.597369 18380 consensus_queue.cc:1059] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:44.598637 18347 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:44.599352 16551 raft_consensus.cc:1240] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:44.600556 16516 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:44.601469 18347 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:44.601878 18347 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:44.602295 18347 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:44.602406 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:44.603683 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:44.604261 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:44.574091 18332 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4.
I20260709 12:22:44.574126 18526 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:44.606621 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:44.607405 16547 raft_consensus.cc:2368] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 6. Current term is 7.
I20260709 12:22:44.607141 18526 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:44.609512 18526 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:22:44.559422 18136 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.821s	user 0.023s	sys 0.033s
W20260709 12:22:44.610276 18136 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.024s	sys 0.033s
I20260709 12:22:44.500536 18414 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:44.541848 18060 raft_consensus.cc:3055] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:44.585063 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:44.511698 18171 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:44.611176 18060 raft_consensus.cc:740] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:44.611442 17269 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:44.586623 16442 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:44.565129 17254 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:44.593922 17872 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:44.593859 18295 consensus_queue.cc:237] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:44.594697 17263 raft_consensus.cc:2393] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 5: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:44.594776 18529 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:44.595837 18403 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:44.571731 18247 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:44.596565 18456 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:44.601716 16515 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:44.506953 18360 consensus_queue.cc:1059] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 12:22:44.614220 16442 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:44.614856 16516 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:44.608255 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:44.612931 18414 consensus_queue.cc:1059] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:44.618860 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:44.619072 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:44.618170 18161 raft_consensus.cc:2804] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:44.623618 17269 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:44.500907 18408 consensus_queue.cc:237] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:44.627179 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:44.628466 16515 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:44.629559 17665 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:22:44.639382 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:44.639779 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:44.639420 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:44.640596 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:44.642303 17142 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:44.642911 17665 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:44.610909 18455 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:44.643347 17665 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:44.645344 16394 raft_consensus.cc:1275] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:44.645493 17665 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:44.630743 16516 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:44.647008 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:44.649381 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:44.651268 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:44.651633 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:44.652155 16476 raft_consensus.cc:2410] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:22:44.645435 16515 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:44.656598 17143 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:44.661038 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:44.660375 16516 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:44.662716 16516 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:44.663646 16545 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:44.640995 18136 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:44.665402 16474 raft_consensus.cc:1240] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:44.659279 18518 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:44.646634 18468 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:44.670197 17143 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:44.648034 18161 raft_consensus.cc:697] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:22:44.680719 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:44.617179 18530 consensus_queue.cc:1048] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.070s
I20260709 12:22:44.683303 16549 raft_consensus.cc:1240] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:44.657465 18273 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:44.711234 17143 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:44.612174 18060 consensus_queue.cc:260] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:44.724149 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:44.732306 17665 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:44.732676 16472 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
W20260709 12:22:44.734653 18538 log.cc:927] Time spent T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 12:22:44.677850 18002 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:44.736052 18002 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:44.736434 18002 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:44.738284 18161 consensus_queue.cc:237] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:44.740767 16474 raft_consensus.cc:1275] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:22:44.669790 18360 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:22:44.742302 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:44.745011 16442 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:44.669426 18206 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:44.653122 18385 consensus_queue.cc:1059] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 12:22:44.752506 17143 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:44.666860 17872 consensus_queue.cc:1059] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.082s
I20260709 12:22:44.731417 18136 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:44.718575 18526 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:44.756515 16442 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:44.757202 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:44.749908 18360 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:44.764838 17142 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:44.741222 18414 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:22:44.785604 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 122943us
I20260709 12:22:44.789981 16516 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:44.754395 18410 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:44.771239 18166 raft_consensus.cc:2804] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:44.782083 18060 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:44.746567 17861 consensus_queue.cc:1059] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 12:22:44.820467 17269 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:22:44.821223 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:44.809062 16516 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:44.786021 18360 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:44.823755 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:44.823812 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:44.815513 18267 consensus_queue.cc:1048] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 12:22:44.827278 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:44.827991 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:44.828464 17861 consensus_queue.cc:1243] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer ef57beab0e1d40d585983873c55107e8 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:22:44.828527 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:44.829202 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:44.829916 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:44.830559 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:44.831269 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:44.831640 18247 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:44.831969 16397 raft_consensus.cc:2368] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4.
I20260709 12:22:44.793208 18526 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:44.792029 18136 raft_consensus.cc:697] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:44.805074 18439 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:44.831708 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:44.835107 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:44.843187 17264 raft_consensus.cc:2410] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:22:44.810537 18468 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:44.826531 16516 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:44.822340 18348 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:44.845376 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:44.846845 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:44.847493 17264 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:44.792520 18414 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:44.848045 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:44.850371 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:44.833101 18166 raft_consensus.cc:697] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:44.854529 16367 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:44.825831 18528 consensus_queue.cc:1059] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:22:44.844940 18066 raft_consensus.cc:2764] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:22:44.856492 18515 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:44.866099 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:44.867280 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:44.871186 16367 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:44.871487 16402 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:44.870869 16394 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:44.868218 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 103521us
I20260709 12:22:44.839931 18464 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:44.876995 16367 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:44.877269 16396 raft_consensus.cc:1240] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:22:44.877496 18455 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:44.878526 17142 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:44.858248 18429 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:44.893594 16367 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:44.894371 17142 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:44.875098 18247 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:44.874969 18480 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:22:44.910931 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:44.914389 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:44.916123 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:44.896646 17142 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:44.901684 16550 raft_consensus.cc:1240] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:44.904543 16402 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:44.907330 16394 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:22:44.899032 17264 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:22:44.901167 16367 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:44.874898 18540 raft_consensus.cc:2804] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:44.920683 16441 leader_election.cc:400] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 4. Current term is 5.
I20260709 12:22:44.905344 18047 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:44.923743 18545 consensus_queue.cc:1048] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:44.924319 18047 raft_consensus.cc:2804] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:44.924723 18047 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:44.925096 18047 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:44.925160 18464 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:44.927215 18545 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:44.928217 18545 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:44.892380 18222 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:44.931998 16546 raft_consensus.cc:2368] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 5. Current term is 6.
I20260709 12:22:44.934291 18545 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:44.899925 18455 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:44.937918 18455 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:44.939028 18545 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:44.939249 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:44.940455 18166 consensus_queue.cc:237] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:44.940984 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:44.942121 16543 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:22:44.942689 16543 raft_consensus.cc:606] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:22:44.942768 18047 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:44.943138 16543 raft_consensus.cc:614] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:44.944814 18429 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:44.945142 18047 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:44.917280 18161 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:44.952749 18161 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:44.955032 18161 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:44.960507 18540 raft_consensus.cc:697] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:44.961732 18540 consensus_queue.cc:237] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:44.928961 18293 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:44.905967 17872 consensus_queue.cc:1059] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 12:22:44.965248 18545 raft_consensus.cc:3055] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:44.965636 17264 raft_consensus.cc:1240] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:44.892911 18136 consensus_queue.cc:237] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:44.965781 18545 raft_consensus.cc:740] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:44.967198 18545 consensus_queue.cc:260] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:44.970722 17868 consensus_meta.cc:287] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.528s	user 0.007s	sys 0.003s
I20260709 12:22:44.906704 18528 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:44.971395 16403 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6.
I20260709 12:22:44.971946 18528 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:44.972395 18528 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:44.976172 18395 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:44.980929 18543 log.cc:927] Time spent T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.193s	user 0.002s	sys 0.000s
I20260709 12:22:44.987349 18002 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:44.988911 18397 consensus_meta.cc:287] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.527s	user 0.010s	sys 0.000s
I20260709 12:22:44.989424 18414 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:44.991351 17270 raft_consensus.cc:1240] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:22:44.940042 18455 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:44.991660 18466 consensus_queue.cc:1059] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:44.995318 16441 leader_election.cc:400] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 8. Current term is 9.
I20260709 12:22:44.995782 16441 leader_election.cc:403] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:44.951861 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:44.989434 18397 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:44.998243 16542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1045) took 1765 ms. Trace:
I20260709 12:22:44.998517 16400 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
W20260709 12:22:44.997361 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 61283us
I20260709 12:22:44.955869 18347 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:44.998855 18161 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:22:45.011215 16548 raft_consensus.cc:1240] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 6. Current term is 7. Ops: []
W20260709 12:22:45.011626 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:44.999089 18002 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:44.994024 17254 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:45.024714 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:45.022701 18468 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.053164 18397 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:45.055058 18347 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:45.061573 16470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 876) took 1511 ms. Trace:
I20260709 12:22:45.062148 16470 rpcz_store.cc:276] 0709 12:22:43.549771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:43.926430 (+376659us) service_pool.cc:224] Handling call
0709 12:22:44.037689 (+111259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:44.038557 (+   868us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:45.061270 (+1022713us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b5400a695d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:45.061491 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":254846976}
I20260709 12:22:45.073231 18468 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:45.000240 16542 rpcz_store.cc:276] 0709 12:22:43.232974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:43.233106 (+   132us) service_pool.cc:224] Handling call
0709 12:22:43.841067 (+607961us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:43.951921 (+110854us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:44.997966 (+1046045us) spinlock_profiling.cc:243] Waited 607 ms on lock 0x7b54008c4bd0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:44.998165 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1397105024}
I20260709 12:22:45.100708 16542 raft_consensus.cc:1240] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:22:45.119925 18466 raft_consensus.cc:3055] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:45.121013 18466 raft_consensus.cc:740] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:45.122148 18466 consensus_queue.cc:260] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:45.123595 18466 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:45.144304 16543 raft_consensus.cc:2393] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 6: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:45.145923 16543 raft_consensus.cc:1240] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:45.151842 16475 raft_consensus.cc:1240] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:45.152634 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:45.152909 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:45.153517 16468 raft_consensus.cc:2393] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 5: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:45.153704 16477 raft_consensus.cc:2393] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 6: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:45.155025 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:45.155588 16477 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 3.
I20260709 12:22:45.076066 18468 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:45.158141 18528 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:45.151945 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:45.167771 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:45.160113 18360 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:45.171315 18547 raft_consensus.cc:3055] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:45.171242 18360 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:45.171803 18547 raft_consensus.cc:740] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:45.174718 16546 raft_consensus.cc:1240] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 12:22:45.174875 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.176991 16401 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
I20260709 12:22:45.178124 16401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 790) took 1179 ms. Trace:
I20260709 12:22:45.177940 18547 consensus_queue.cc:260] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:45.178449 16401 rpcz_store.cc:276] 0709 12:22:43.999214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:43.999353 (+   139us) service_pool.cc:224] Handling call
0709 12:22:45.177968 (+1178615us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b540015f754. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:45.178069 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2653151488}
I20260709 12:22:45.179188 18547 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:45.181663 16398 raft_consensus.cc:1275] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:45.182866 18545 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:45.185029 18267 raft_consensus.cc:3055] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:45.185143 16395 raft_consensus.cc:1275] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:45.185488 18267 raft_consensus.cc:740] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:45.186334 18267 consensus_queue.cc:260] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:45.187441 18267 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:45.187500 16395 raft_consensus.cc:1275] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:22:45.189693 16394 raft_consensus.cc:1240] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:45.190639 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:45.192111 16399 raft_consensus.cc:1275] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:22:45.203426 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.208179 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:45.209097 16396 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:22:45.210824 18559 consensus_queue.cc:1048] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:45.213303 18559 consensus_queue.cc:1048] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:45.213598 17144 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:45.215462 18559 consensus_queue.cc:1048] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:45.215647 18172 consensus_queue.cc:1059] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:45.216543 18551 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:45.216761 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:45.219758 16470 raft_consensus.cc:2410] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:22:45.223243 16547 raft_consensus.cc:1240] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:45.193394 17861 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.225080 17861 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:45.226251 16549 raft_consensus.cc:3055] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:45.226626 16549 raft_consensus.cc:740] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:45.227151 17861 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:45.227389 16549 consensus_queue.cc:260] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:45.228931 18449 raft_consensus.cc:3055] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:45.229416 18449 raft_consensus.cc:740] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:45.230156 18449 consensus_queue.cc:260] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:45.231385 16547 raft_consensus.cc:1275] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:45.231503 18449 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:45.232324 16403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 791) took 1232 ms. Trace:
I20260709 12:22:45.232702 16403 rpcz_store.cc:276] 0709 12:22:43.999613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:44.022050 (+ 22437us) service_pool.cc:224] Handling call
0709 12:22:45.232162 (+1210112us) spinlock_profiling.cc:243] Waited 948 ms on lock 0x7b54004ec5d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:45.232249 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2180676736}
W20260709 12:22:45.234998 16544 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52680 (request call id 305) took 1270 ms (client timeout 1585 ms). Trace:
W20260709 12:22:45.235409 16544 rpcz_store.cc:269] 0709 12:22:43.964906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:43.965164 (+   258us) service_pool.cc:224] Handling call
0709 12:22:45.234808 (+1269644us) spinlock_profiling.cc:243] Waited 412 ms on lock 0x7b5400366354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:45.234924 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":948301056}
I20260709 12:22:45.236219 18528 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:45.193799 18491 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.243186 18551 consensus_queue.cc:1048] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:45.243710 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:45.244315 16396 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:45.241475 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:45.246778 16403 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:45.242381 16549 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:45.250536 16551 raft_consensus.cc:1275] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:22:45.252379 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 355768us
I20260709 12:22:45.257115 16441 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:22:45.260464 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 49816.4us
I20260709 12:22:45.261605 16441 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:45.268203 16549 raft_consensus.cc:1275] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:45.274974 16396 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:22:45.278661 16441 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:45.277341 17144 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:45.281292 17263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45230 (request call id 657) took 1247 ms. Trace:
W20260709 12:22:45.281437 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 380339us
I20260709 12:22:45.292713 17263 rpcz_store.cc:276] 0709 12:22:44.033635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:44.036802 (+  3167us) service_pool.cc:224] Handling call
0709 12:22:45.281004 (+1244202us) spinlock_profiling.cc:243] Waited 557 ms on lock 0x7b540005f854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:45.281205 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1280569600}
I20260709 12:22:45.293846 16367 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:45.274290 16542 raft_consensus.cc:1240] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 4. Current term is 5. Ops: []
W20260709 12:22:45.255976 18518 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.009s	sys 0.012s
I20260709 12:22:45.297035 16472 raft_consensus.cc:1240] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:45.314615 16367 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:45.243074 18491 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:45.297180 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:45.297366 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:45.298205 17143 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:45.300565 18209 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:45.306666 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:45.327167 18403 raft_consensus.cc:3055] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:45.327651 18403 raft_consensus.cc:740] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:45.328475 16367 leader_election.cc:400] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 6. Current term is 7.
I20260709 12:22:45.328789 16367 leader_election.cc:403] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:22:45.329030 18168 raft_consensus.cc:3055] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:45.295642 18171 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:45.329398 18168 raft_consensus.cc:740] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:45.329806 18298 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:45.329948 18395 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:45.330607 18298 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 6. Current term is 7.
I20260709 12:22:45.330328 18168 consensus_queue.cc:260] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:45.330734 16441 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:45.331310 18466 consensus_queue.cc:1048] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:45.331683 18168 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:45.332670 18466 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:45.310765 17270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 676) took 1266 ms. Trace:
I20260709 12:22:45.329698 16367 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:45.289827 16515 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:45.262478 18271 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:45.321715 16476 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:45.338999 16367 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:45.343163 18527 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:45.343395 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:45.343859 18527 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:45.344101 17271 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:45.344707 17861 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:45.345266 17861 raft_consensus.cc:697] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:22:45.346324 16225 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53622 (request call id 132) took 18751 ms (client timeout 15000 ms). Trace:
I20260709 12:22:45.346156 17861 consensus_queue.cc:237] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:45.348683 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:22:45.349292 18561 log.cc:927] Time spent T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
W20260709 12:22:45.349431 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 51349.2us
I20260709 12:22:45.350728 18548 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:45.351397 18548 consensus_queue.cc:1048] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:45.352757 18279 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:45.353987 17271 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:45.353801 18403 consensus_queue.cc:260] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:45.355052 18403 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:45.284389 18542 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 12:22:45.361968 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:45.363664 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:45.363795 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:45.364403 16476 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:45.349382 17264 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:22:45.365291 17142 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:45.367102 16402 raft_consensus.cc:1240] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 4. Current term is 5. Ops: []
W20260709 12:22:45.334501 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 44802.1us
I20260709 12:22:45.337472 17261 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 666) took 1302 ms. Trace:
I20260709 12:22:45.368292 17261 rpcz_store.cc:276] 0709 12:22:44.034734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:44.034860 (+   126us) service_pool.cc:224] Handling call
0709 12:22:44.035171 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:44.325305 (+290134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:45.337106 (+1011801us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b54005f2654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:45.337362 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":665585536}
I20260709 12:22:45.333364 17270 rpcz_store.cc:276] 0709 12:22:44.044505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:44.114649 (+ 70144us) service_pool.cc:224] Handling call
0709 12:22:44.673054 (+558405us) spinlock_profiling.cc:243] Waited 7.49 ms on lock 0x7b58003b8298. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:22:44.673281 (+   227us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:44.993843 (+320562us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:45.277710 (+283867us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b540055f854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:45.310684 (+ 32974us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":748087808}
W20260709 12:22:45.323730 18518 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.009s	sys 0.017s
I20260709 12:22:45.358625 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:45.371047 16402 raft_consensus.cc:1240] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:45.359230 16516 leader_election.cc:400] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4.
I20260709 12:22:45.332821 16441 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:22:45.377326 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 44573.7us
W20260709 12:22:45.360489 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.365449 16442 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:45.370374 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:45.382186 16402 raft_consensus.cc:1240] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:22:45.356479 18161 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 8. Current term is 9.
I20260709 12:22:45.384495 16394 raft_consensus.cc:1240] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:45.383869 16402 raft_consensus.cc:1240] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:45.387768 16442 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:45.384989 16395 raft_consensus.cc:1240] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:45.382833 16367 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:45.393057 16367 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:45.393392 16441 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:22:45.394161 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.394315 16367 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:45.395905 16476 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:22:45.397660 16403 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:22:45.393290 16366 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:45.400221 17665 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:45.405514 17142 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:45.395210 16515 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:22:45.346717 16225 rpcz_store.cc:269] 0709 12:22:26.594577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:26.594912 (+   335us) service_pool.cc:224] Handling call
0709 12:22:45.346242 (+18751330us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:22:40.507509 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:22:40.628088 (+120579us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:22:40.637708 (+  9620us) write_op.cc:435] Acquired schema lock
0709 12:22:40.637729 (+    21us) tablet.cc:662] Decoding operations
0709 12:22:40.638546 (+   817us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:22:40.638642 (+    96us) write_op.cc:641] Partition lock acquired for write op
0709 12:22:40.638684 (+    42us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:22:40.639283 (+   599us) tablet.cc:701] Row locks acquired
0709 12:22:40.639335 (+    52us) write_op.cc:260] PREPARE: finished
0709 12:22:40.639519 (+   184us) write_op.cc:270] Start()
0709 12:22:40.639682 (+   163us) write_op.cc:276] Timestamp: P: 1783599760639451 usec, L: 0
0709 12:22:40.639722 (+    40us) op_driver.cc:348] REPLICATION: starting
0709 12:22:40.699579 (+ 59857us) log.cc:844] Serialized 1669 byte log entry
0709 12:22:41.268461 (+568882us) op_driver.cc:464] REPLICATION: finished
0709 12:22:41.574772 (+306311us) write_op.cc:301] APPLY: starting
0709 12:22:41.635449 (+ 60677us) tablet.cc:1367] starting BulkCheckPresence
0709 12:22:41.665564 (+ 30115us) tablet.cc:1370] finished BulkCheckPresence
0709 12:22:41.784456 (+118892us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:22:42.162873 (+378417us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:22:42.238898 (+ 76025us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 12:22:42.238979 (+    81us) write_op.cc:312] APPLY: finished
0709 12:22:42.257521 (+ 18542us) log.cc:844] Serialized 40 byte log entry
0709 12:22:42.710310 (+452789us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:42.858479 (+148169us) write_op.cc:454] Released schema lock
0709 12:22:42.895332 (+ 36853us) write_op.cc:341] FINISH: Updating metrics
0709 12:22:42.956713 (+ 61381us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:42.961538 (+  4825us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3656709,"reported_tablets":19,"child_traces":[["op",{"apply.queue_time_us":223431,"apply.run_cpu_time_us":28580,"apply.run_wall_time_us":1374466,"num_ops":4,"prepare.queue_time_us":292218,"prepare.run_cpu_time_us":17464,"prepare.run_wall_time_us":626866,"raft.queue_time_us":164692,"raft.run_cpu_time_us":8964,"raft.run_wall_time_us":245263,"replication_time_us":605395,"thread_start_us":1001419,"threads_started":5,"wal-append.queue_time_us":792211,"wal-append.run_cpu_time_us":20199,"wal-append.run_wall_time_us":2214959}]]}
I20260709 12:22:45.361481 18171 raft_consensus.cc:697] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:45.412657 16366 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:45.416610 16367 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:22:45.420308 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 43883.2us
I20260709 12:22:45.420928 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:45.422274 16367 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:45.422633 16516 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:45.371119 18518 raft_consensus.cc:697] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:45.426983 16367 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:45.398155 18271 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:45.397915 18527 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:45.460533 18527 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:45.461083 18527 raft_consensus.cc:697] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:45.432303 17665 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:45.461802 17665 raft_consensus.cc:697] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:45.432427 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:45.436163 16515 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:45.435861 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:45.440197 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:45.441545 16402 raft_consensus.cc:1275] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:45.467703 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:45.423610 18290 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:45.468211 16549 raft_consensus.cc:2393] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 3: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:45.431476 18247 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:45.423836 18466 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:45.450207 18563 consensus_queue.cc:1059] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:45.424439 18468 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:45.426589 18401 consensus_queue.cc:1059] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 12:22:45.419230 18413 raft_consensus.cc:2764] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:22:45.433084 16367 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:45.452003 18171 consensus_queue.cc:237] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:45.411769 18410 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:45.455066 17144 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:45.411546 18298 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:45.455632 18529 consensus_queue.cc:1059] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:45.456480 18013 consensus_queue.cc:1059] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:45.411258 18389 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:45.442312 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:45.470211 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:45.490546 17271 raft_consensus.cc:2393] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 6: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:45.490463 16403 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:45.472185 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:45.462759 17665 consensus_queue.cc:237] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:45.483700 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 50674.6us
I20260709 12:22:45.484993 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:45.433811 18359 consensus_queue.cc:1059] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:22:45.498600 16403 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:22:45.465575 18166 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:22:45.501296 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 46332.5us
I20260709 12:22:45.465104 18515 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:45.478068 18273 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:45.509927 16367 leader_election.cc:400] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 5. Current term is 6.
I20260709 12:22:45.533795 16367 leader_election.cc:403] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:22:45.478945 18466 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:45.465315 18558 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:45.536482 16402 raft_consensus.cc:2393] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 7: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:45.508224 16403 raft_consensus.cc:1275] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:45.511494 16516 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:45.549809 17665 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:45.550383 17665 raft_consensus.cc:697] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:45.500072 18290 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:45.551672 17665 consensus_queue.cc:237] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:45.500545 18298 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:22:45.557431 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 61251.8us
I20260709 12:22:45.507208 18166 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:45.511175 18440 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:45.538103 18273 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:45.487489 18066 consensus_queue.cc:1048] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 12:22:45.545296 18466 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:45.577147 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 64882.9us
I20260709 12:22:45.544842 18247 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 5. Current term is 6.
I20260709 12:22:45.561777 18298 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:45.565888 18166 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:45.561838 18558 raft_consensus.cc:697] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:45.566200 18401 raft_consensus.cc:3055] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:45.553795 18515 consensus_queue.cc:1059] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:22:45.578214 18273 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:45.608994 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 71841.1us
I20260709 12:22:45.609779 16367 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:45.617045 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:45.617489 17258 raft_consensus.cc:1240] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:45.599246 18440 raft_consensus.cc:697] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:45.619334 17258 raft_consensus.cc:1275] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:22:45.619868 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:45.624720 17258 raft_consensus.cc:1240] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:45.626298 17266 raft_consensus.cc:1240] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:22:45.603989 18401 raft_consensus.cc:740] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:45.633591 16395 raft_consensus.cc:1275] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:45.641009 16548 raft_consensus.cc:1275] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:45.614499 17780 consensus_queue.cc:1048] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 12:22:45.632964 18557 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:45.577073 18527 consensus_queue.cc:237] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:45.562456 18518 consensus_queue.cc:237] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:45.705799 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 86664.3us
W20260709 12:22:45.710311 17142 leader_election.cc:336] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.739s (SENT)
W20260709 12:22:45.711833 18577 log.cc:927] Time spent T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 12:22:45.715284 17270 raft_consensus.cc:1275] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:45.725016 16475 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:45.691339 18547 consensus_meta.cc:287] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.512s	user 0.009s	sys 0.004s
I20260709 12:22:45.725380 18481 consensus_queue.cc:1059] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:45.726310 18481 raft_consensus.cc:3055] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:45.726704 18481 raft_consensus.cc:740] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:45.727494 18481 consensus_queue.cc:260] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:45.730321 18161 raft_consensus.cc:993] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2: : Instructing follower f99066d13e524238aec6aa2024a8bd50 to start an election
I20260709 12:22:45.731127 18161 raft_consensus.cc:1081] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Signalling peer f99066d13e524238aec6aa2024a8bd50 to start an election
W20260709 12:22:45.694003 18565 log.cc:927] Time spent T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 12:22:45.669046 18066 consensus_queue.cc:1048] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 12:22:45.703526 18209 consensus_queue.cc:1059] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 12:22:45.809798 16475 raft_consensus.cc:1275] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:45.820107 16548 raft_consensus.cc:1275] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:45.684151 18581 consensus_queue.cc:1059] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.137s
I20260709 12:22:45.694646 18295 consensus_queue.cc:1048] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.123s
I20260709 12:22:45.802911 18222 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:45.807710 18481 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:45.803712 18548 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:45.808861 18452 consensus_queue.cc:1048] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 12:22:45.831718 18548 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:45.832454 16469 raft_consensus.cc:2410] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:22:45.833693 18589 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.834205 18589 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:45.834573 18548 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:45.835261 18466 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:45.835805 18395 consensus_queue.cc:1059] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:22:45.836421 18575 log.cc:927] Time spent T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
W20260709 12:22:45.837563 18579 log.cc:927] Time spent T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 12:22:45.837633 18222 raft_consensus.cc:3055] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:45.838575 18222 raft_consensus.cc:740] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:45.840324 18439 raft_consensus.cc:3055] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:45.839793 18222 consensus_queue.cc:260] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:45.840822 18439 raft_consensus.cc:740] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:45.841413 18222 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:45.841694 18439 consensus_queue.cc:260] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:45.843793 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.820494 18480 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:45.844707 18563 raft_consensus.cc:3055] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:45.845136 18563 raft_consensus.cc:740] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:45.822124 18585 consensus_queue.cc:1048] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
W20260709 12:22:45.847047 18564 log.cc:927] Time spent T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 12:22:45.706888 18440 consensus_queue.cc:237] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:45.849975 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.851001 18290 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:45.853021 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:45.853762 16468 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:22:45.857641 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:45.856001 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:45.859608 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:45.861162 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:45.862133 16475 raft_consensus.cc:2393] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 5: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:45.681846 18558 consensus_queue.cc:237] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:45.855623 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:45.685700 18401 consensus_queue.cc:260] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:45.822250 17255 raft_consensus.cc:1275] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:45.870968 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:45.868789 17142 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:45.899464 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:45.901012 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:45.830611 18209 raft_consensus.cc:3055] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:45.976949 18466 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:45.837567 18515 raft_consensus.cc:3055] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:45.863399 18279 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.053375 18589 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:46.054738 18563 consensus_queue.cc:260] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:46.056674 18563 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:45.894640 18581 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:46.093406 16475 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:46.094067 18589 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:46.099486 18589 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:46.100239 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:46.101596 18589 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:46.101770 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:22:46.094952 16477 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.004s	sys 0.015s
W20260709 12:22:46.103276 16477 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.004s	sys 0.015s
I20260709 12:22:45.979498 18209 raft_consensus.cc:740] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:22:46.097920 16441 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.002s	sys 0.017s
I20260709 12:22:46.034858 18588 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:45.982738 18515 raft_consensus.cc:740] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:46.102365 16475 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:22:46.120425 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:46.121270 16477 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:46.118064 17665 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.010s	sys 0.012s
W20260709 12:22:46.110316 16441 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.002s	sys 0.017s
W20260709 12:22:46.123426 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 703111us
W20260709 12:22:46.123754 17665 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.010s	sys 0.012s
I20260709 12:22:46.102392 18589 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.129599 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:46.140062 16475 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:46.121955 17254 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.011s	sys 0.011s
W20260709 12:22:46.104081 18271 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.005s	sys 0.014s
W20260709 12:22:46.116848 17868 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.004s	sys 0.016s
I20260709 12:22:46.106914 18528 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:46.154651 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:46.158618 16546 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:46.094547 18279 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:46.140323 18403 consensus_meta.cc:287] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.785s	user 0.011s	sys 0.003s
I20260709 12:22:46.117208 18588 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:46.172358 16548 raft_consensus.cc:1240] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 4. Current term is 5. Ops: []
W20260709 12:22:46.160518 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 283928us
I20260709 12:22:46.162248 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:46.110157 18581 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:46.147614 17868 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.005s	sys 0.016s
W20260709 12:22:46.147394 18271 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.005s	sys 0.015s
W20260709 12:22:46.147746 17254 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.011s	sys 0.011s
W20260709 12:22:46.169027 16367 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.011s	sys 0.019s
W20260709 12:22:46.186982 16367 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.011s	sys 0.019s
W20260709 12:22:46.187829 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 578075us
I20260709 12:22:46.189006 16367 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:46.177529 16477 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:22:46.144366 18047 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:46.155510 16475 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:22:46.190068 16470 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:22:46.193812 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:46.194875 16367 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:46.191584 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:46.196805 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:46.176422 18588 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:46.144863 18589 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:46.162340 18528 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:46.242607 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:46.243688 16401 raft_consensus.cc:2368] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8.
I20260709 12:22:46.204581 17868 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:46.179961 18466 consensus_queue.cc:1059] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 12:22:46.236786 16398 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa"
dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
 from {username='slave'} at 127.0.0.1:46252
I20260709 12:22:46.235394 18290 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:22:46.284279 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:46.169054 18209 consensus_queue.cc:260] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:46.234607 18271 consensus_queue.cc:1048] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 12:22:46.203832 18047 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:46.309047 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:46.261377 18414 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.193497 18515 consensus_queue.cc:260] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:46.311916 16477 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:22:46.321190 16403 raft_consensus.cc:1275] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:22:46.375128 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 235750us
I20260709 12:22:46.396394 16477 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "defc374629a649a8a47502f058be2c8c"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:39000
I20260709 12:22:46.397436 16477 raft_consensus.cc:606] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:22:46.398041 16477 raft_consensus.cc:614] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:22:46.423079 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:46.470057 16542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 866) took 1559 ms. Trace:
I20260709 12:22:46.470836 16542 rpcz_store.cc:276] 0709 12:22:44.910358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:45.101905 (+191547us) service_pool.cc:224] Handling call
0709 12:22:45.239572 (+137667us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:45.313228 (+ 73656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:46.469703 (+1156475us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b5400366350. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:46.469949 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":310101760}
W20260709 12:22:46.483145 18586 log.cc:927] Time spent T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.378s	user 0.000s	sys 0.000s
W20260709 12:22:46.485301 18573 log.cc:927] Time spent T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.879s	user 0.002s	sys 0.000s
I20260709 12:22:46.501420 18551 raft_consensus.cc:3055] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:46.502187 18551 raft_consensus.cc:740] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:46.503407 18551 consensus_queue.cc:260] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:46.505470 18551 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:46.524309 18410 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:46.525066 18410 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:46.321184 18209 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:46.592864 18521 log.cc:927] Time spent T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:22:46.428936 18414 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:46.585053 18452 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.598109 18279 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:46.610067 18222 consensus_meta.cc:287] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.767s	user 0.011s	sys 0.000s
W20260709 12:22:46.623960 18298 consensus_meta.cc:287] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 1.021s	user 0.010s	sys 0.003s
I20260709 12:22:46.788198 18298 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:46.629299 18452 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:46.792217 18360 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.793569 18360 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:46.795121 18452 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:46.648783 18597 log.cc:927] Time spent T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.349s	user 0.001s	sys 0.000s
I20260709 12:22:46.797338 18360 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:22:46.647891 18604 log.cc:927] Time spent T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.261s	user 0.001s	sys 0.000s
I20260709 12:22:46.798959 16402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 765) took 1158 ms. Trace:
I20260709 12:22:46.678138 18528 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:22:46.750036 16515 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.014s	sys 0.002s
I20260709 12:22:46.680425 18613 consensus_queue.cc:1059] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.121s
W20260709 12:22:46.801342 16515 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.015s	sys 0.002s
I20260709 12:22:46.802250 16394 raft_consensus.cc:1240] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:46.802800 18466 raft_consensus.cc:3055] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:46.803515 18466 raft_consensus.cc:740] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:46.762782 18527 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:22:46.664290 18481 consensus_meta.cc:287] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.817s	user 0.007s	sys 0.002s
I20260709 12:22:46.804968 18527 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:46.750749 16517 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.020s	sys 0.017s
W20260709 12:22:46.806376 16517 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.021s	sys 0.017s
I20260709 12:22:46.805881 18466 consensus_queue.cc:260] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:46.767123 18599 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:22:46.670269 18563 consensus_meta.cc:287] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.613s	user 0.012s	sys 0.000s
I20260709 12:22:46.664839 18581 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:22:46.744580 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:46.818531 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:46.665163 18589 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:46.762202 18385 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.797499 18267 consensus_queue.cc:1059] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:22:46.646982 18610 log.cc:927] Time spent T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 12:22:46.749722 16514 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.009s	sys 0.007s
I20260709 12:22:46.808493 16472 raft_consensus.cc:2410] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:22:46.749293 16469 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53186 (request call id 316) took 2100 ms (client timeout 1739 ms). Trace:
I20260709 12:22:46.750645 16395 raft_consensus.cc:1240] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 12:22:46.748121 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 461140us
W20260709 12:22:46.747572 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:46.746425 16399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 450) took 1429 ms. Trace:
I20260709 12:22:46.811508 18599 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:46.754751 16471 raft_consensus.cc:1275] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:22:46.848922 16471 pending_rounds.cc:85] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 3
I20260709 12:22:46.853510 16471 pending_rounds.cc:107] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 5.4
W20260709 12:22:46.850150 16514 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.010s	sys 0.007s
I20260709 12:22:46.855697 18385 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:46.861236 16442 leader_election.cc:400] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8.
I20260709 12:22:46.861799 16442 leader_election.cc:403] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:22:46.863997 16401 raft_consensus.cc:1275] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:22:46.864529 16401 pending_rounds.cc:85] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 2
I20260709 12:22:46.864895 16401 pending_rounds.cc:107] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:22:46.865228 16401 raft_consensus.cc:2889] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:46.865502 16403 raft_consensus.cc:1275] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:46.866133  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:22:46.868817 18581 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:46.799646 16402 rpcz_store.cc:276] 0709 12:22:45.640282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:45.640658 (+   376us) service_pool.cc:224] Handling call
0709 12:22:45.641071 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:45.641497 (+   426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:45.641530 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:45.642137 (+   607us) log.cc:844] Serialized 26 byte log entry
0709 12:22:46.105351 (+463214us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:46.105398 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:46.105467 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:46.798327 (+692860us) raft_consensus.cc:1685] finished
0709 12:22:46.798389 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:46.798705 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15304,"raft.queue_time_us":49296,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":13762,"thread_start_us":462850,"threads_started":1,"wal-append.queue_time_us":594824}
I20260709 12:22:46.875700 18047 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:46.877542 18632 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:46.878669 18632 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:46.886364 18271 raft_consensus.cc:3055] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:46.887074 18271 raft_consensus.cc:740] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:46.888872 17254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 676) took 1491 ms. Trace:
I20260709 12:22:46.889400 17254 rpcz_store.cc:276] 0709 12:22:45.357613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:45.361234 (+  3621us) service_pool.cc:224] Handling call
0709 12:22:45.414702 (+ 53468us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:45.461019 (+ 46317us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:22:46.186206 (+725187us) log.cc:844] Serialized 14 byte log entry
0709 12:22:46.752749 (+566543us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:46.754542 (+  1793us) log.cc:844] Serialized 52 byte log entry
0709 12:22:46.754915 (+   373us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:46.755844 (+   929us) log.cc:844] Serialized 14 byte log entry
0709 12:22:46.756182 (+   338us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:46.756316 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:46.821192 (+ 64876us) raft_consensus.cc:1685] finished
0709 12:22:46.822603 (+  1411us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:46.848914 (+ 26311us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7813,"prepare.queue_time_us":710188,"prepare.run_cpu_time_us":128,"prepare.run_wall_time_us":129,"raft.queue_time_us":195560,"raft.run_cpu_time_us":571,"raft.run_wall_time_us":11570,"thread_start_us":1122040,"threads_started":2,"wal-append.queue_time_us":630037}
I20260709 12:22:46.894841 18599 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:46.854005 16471 raft_consensus.cc:2889] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:22:46.840533 16469 rpcz_store.cc:269] 0709 12:22:44.649023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:44.649172 (+   149us) service_pool.cc:224] Handling call
0709 12:22:46.496159 (+1846987us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400a68bd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:46.748905 (+252746us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2718600064}
I20260709 12:22:46.840859 18527 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:46.876063 18401 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:46.907120 18298 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:46.911003 18581 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8.
I20260709 12:22:46.841233 16399 rpcz_store.cc:276] 0709 12:22:45.316583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:45.316707 (+   124us) service_pool.cc:224] Handling call
0709 12:22:45.317156 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:45.320282 (+  3126us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:45.320309 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:45.332905 (+ 12596us) log.cc:844] Serialized 26 byte log entry
0709 12:22:45.467543 (+134638us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:45.467597 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:45.467684 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:46.484596 (+1016912us) spinlock_profiling.cc:243] Waited 1.64 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:22:46.745256 (+260660us) raft_consensus.cc:1685] finished
0709 12:22:46.745336 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:46.746065 (+   729us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4751,"raft.queue_time_us":3221,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":77,"spinlock_wait_cycles":3770752,"thread_start_us":134233,"threads_started":1,"wal-append.queue_time_us":220025}
W20260709 12:22:46.923183 16474 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.858s	user 0.005s	sys 0.022s
I20260709 12:22:46.922049 18518 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:46.922257 18627 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.938723 18630 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:46.939242 16396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 766) took 1085 ms. Trace:
I20260709 12:22:46.922204 18563 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:22:46.941121 16474 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.962s	user 0.006s	sys 0.022s
W20260709 12:22:46.943090 16514 leader_election.cc:336] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.714s (SENT)
I20260709 12:22:46.932729 18452 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.946918 16514 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:46.940658 16396 rpcz_store.cc:276] 0709 12:22:45.853296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:45.853440 (+   144us) service_pool.cc:224] Handling call
0709 12:22:45.892126 (+ 38686us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:45.892594 (+   468us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:45.892627 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:45.893240 (+   613us) log.cc:844] Serialized 26 byte log entry
0709 12:22:46.158852 (+265612us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:46.158913 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:46.159010 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:46.937928 (+778918us) raft_consensus.cc:1685] finished
0709 12:22:46.938791 (+   863us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:46.939051 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6404,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":26605,"thread_start_us":265231,"threads_started":1,"wal-append.queue_time_us":474403}
I20260709 12:22:46.946786 18271 consensus_queue.cc:260] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:46.948088 16548 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.000s	sys 0.016s
I20260709 12:22:46.930101 18609 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:46.945815 18385 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:46.937973 18589 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:46.940208 18413 consensus_queue.cc:1048] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:22:46.960901 18589 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:46.958966 18563 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:46.958982 16548 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.001s	sys 0.016s
I20260709 12:22:46.940166 18518 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:46.965550 18385 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:46.965440 18609 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:46.966418 18518 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:46.966605 18385 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:46.958952 18452 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:46.968959 16476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 847) took 1107 ms. Trace:
I20260709 12:22:46.969638 16476 rpcz_store.cc:276] 0709 12:22:45.861063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:46.834332 (+973269us) service_pool.cc:224] Handling call
0709 12:22:46.911567 (+ 77235us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:46.954702 (+ 43135us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:46.954735 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:46.958123 (+  3388us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:46.968436 (+ 10313us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:46.968536 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:46.968700 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:46.971653 18452 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:46.939653 18630 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:46.975759 18635 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:46.975502 18630 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:46.939643 18627 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:46.976784 18481 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.976737 18635 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:46.977497 18481 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:46.978420 18563 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:46.979107 18627 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:46.979506 18635 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:46.979931 18481 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:46.981201 18632 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:22:46.951473 18569 log.cc:927] Time spent T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 1.134s	user 0.000s	sys 0.003s
I20260709 12:22:46.958510 18359 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:46.993716 18161 raft_consensus.cc:3055] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:46.994071 18271 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:46.994544 18161 raft_consensus.cc:740] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:47.000742 18359 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:47.001691 16476 raft_consensus.cc:1275] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:22:47.004868 16474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 826) took 1157 ms. Trace:
I20260709 12:22:47.005903 18609 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:47.009821 16400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 447) took 1770 ms. Trace:
I20260709 12:22:47.005581 16474 rpcz_store.cc:276] 0709 12:22:45.847658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:45.853813 (+  6155us) service_pool.cc:224] Handling call
0709 12:22:45.977859 (+124046us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:45.978113 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:22:45.978418 (+   305us) log.cc:844] Serialized 14 byte log entry
0709 12:22:46.989254 (+1010836us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:46.989401 (+   147us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:46.989565 (+   164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:47.004450 (+ 14885us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:47.004756 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":588,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":31727,"raft.queue_time_us":321,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":1216,"spinlock_wait_cycles":4010880,"thread_start_us":967464,"threads_started":1,"wal-append.queue_time_us":976949}
I20260709 12:22:46.959501 18581 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:47.010437 16400 rpcz_store.cc:276] 0709 12:22:45.239183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:45.239318 (+   135us) service_pool.cc:224] Handling call
0709 12:22:45.239739 (+   421us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:45.240121 (+   382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:45.240184 (+    63us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:45.240897 (+   713us) log.cc:844] Serialized 52 byte log entry
0709 12:22:45.472406 (+231509us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:45.472468 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:45.472551 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:46.950305 (+1477754us) spinlock_profiling.cc:243] Waited 4.38 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:22:47.009129 (+ 58824us) raft_consensus.cc:1685] finished
0709 12:22:47.009153 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:47.009562 (+   409us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4407,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"spinlock_wait_cycles":10064128,"thread_start_us":187424,"threads_started":1,"wal-append.queue_time_us":408686}
I20260709 12:22:47.004092 18359 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:47.020550 18452 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:22:47.024364 16441 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.014s	sys 0.008s
I20260709 12:22:47.033578 18622 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:47.034735 18622 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:47.036777 18286 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:47.037793 18286 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:47.012929 18298 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:47.042497 18527 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.043426 18527 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:47.047096 16477 raft_consensus.cc:1275] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:47.048053 16477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 844) took 1188 ms. Trace:
I20260709 12:22:47.057296 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:47.058794 16400 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:22:47.060920 18599 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.061383 18385 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:47.061713 18599 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:47.051254 16477 rpcz_store.cc:276] 0709 12:22:45.859745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:46.836831 (+977086us) service_pool.cc:224] Handling call
0709 12:22:46.913882 (+ 77051us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:47.047692 (+133810us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:47.047968 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:47.041355 18298 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:47.031306 18161 consensus_queue.cc:260] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:47.068030 16398 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:47.068652 18491 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:47.050958 18589 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:22:47.070487 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 134870us
W20260709 12:22:47.056098 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 674781us
I20260709 12:22:47.071574 16398 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:47.071555 18161 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:47.071326 18286 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:47.070659 18529 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.081171 18599 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:47.073468 18628 log.cc:927] Time spent T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.117s	user 0.000s	sys 0.003s
I20260709 12:22:47.071815 18298 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:47.069749 18637 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.099315 18527 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:47.094542 18491 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.101068 18002 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.066706 18581 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:47.105401 18578 log.cc:927] Time spent T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 1.296s	user 0.002s	sys 0.002s
I20260709 12:22:47.110131 16441 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:47.109498 18002 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:47.111212 18491 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:47.113672 18567 log.cc:927] Time spent T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 1.474s	user 0.003s	sys 0.000s
W20260709 12:22:47.128401 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:47.129252 18002 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:47.128515 18491 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:47.131796 18637 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:47.137233 16475 raft_consensus.cc:2410] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:22:47.138530 16441 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:47.139103 16551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 339) took 1337 ms. Trace:
I20260709 12:22:47.139673 18634 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:47.140841 16472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34826 (request call id 828) took 1292 ms. Trace:
I20260709 12:22:47.140640 18634 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:47.141268 16472 rpcz_store.cc:276] 0709 12:22:45.848303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:45.855321 (+  7018us) service_pool.cc:224] Handling call
0709 12:22:47.140582 (+1285261us) spinlock_profiling.cc:243] Waited 939 ms on lock 0x7b540058cd54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:47.140766 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2160775552}
I20260709 12:22:47.141954 18593 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:47.144265 18589 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:47.148669 16402 raft_consensus.cc:1240] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:47.152464 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:47.152865 18637 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:47.139588 16551 rpcz_store.cc:276] 0709 12:22:45.801925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:45.802192 (+   267us) service_pool.cc:224] Handling call
0709 12:22:47.137866 (+1335674us) spinlock_profiling.cc:243] Waited 1.71 ms on lock 0x7b58005e9298. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:22:47.138249 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:47.138546 (+   297us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:47.138574 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:47.138619 (+    45us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:47.138674 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:47.138734 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:47.138899 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3932416}
I20260709 12:22:47.139730 16475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34826 (request call id 838) took 1281 ms. Trace:
I20260709 12:22:47.159173 18545 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:47.160056 18545 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:47.163196 18545 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:47.164256 18358 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.154546 16224 catalog_manager.cc:5697] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 2 to 4, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 4 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:22:47.166364 18358 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:47.167877 18414 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:47.107333 18599 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:47.169088 18599 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:47.170586 18358 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:47.141830 18611 consensus_queue.cc:1059] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:22:47.172102 18599 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:47.172487 18637 raft_consensus.cc:3055] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:47.165359 18002 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.176744 16546 raft_consensus.cc:1240] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:22:47.186448 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:47.177100 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:47.178105  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:47.177764 18551 consensus_queue.cc:1048] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:47.178426 18360 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.177793 18359 consensus_queue.cc:1048] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:22:47.180254 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:47.201894 16544 raft_consensus.cc:2393] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 5: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:47.194325 18637 raft_consensus.cc:740] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:47.202330  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:47.181275 18622 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:22:47.185441 18636 log.cc:927] Time spent T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:22:47.186363 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:47.158340 16475 rpcz_store.cc:276] 0709 12:22:45.858148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:46.193559 (+335411us) service_pool.cc:224] Handling call
0709 12:22:47.139467 (+945908us) spinlock_profiling.cc:243] Waited 941 ms on lock 0x7b54002f3354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:47.139649 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2164166528}
W20260709 12:22:47.187515 18209 consensus_meta.cc:287] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.609s	user 0.013s	sys 0.006s
I20260709 12:22:47.199214 18360 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:47.217952 18631 log.cc:927] Time spent T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.281s	user 0.002s	sys 0.000s
I20260709 12:22:47.209404 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:47.226753 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:47.204566 16551 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:47.229653 18634 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:47.221678 16441 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:47.230991 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:47.231156 18634 raft_consensus.cc:2804] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:47.231796 18634 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:47.232069 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 44019.1us
I20260709 12:22:47.233973 16397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 803) took 1003 ms. Trace:
I20260709 12:22:47.235505 16514 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:47.212990 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:22:47.285802 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 50576.1us
I20260709 12:22:47.286629 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:47.287022 16514 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:47.216085 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:47.218194 18360 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:47.218958 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:47.288048 18629 raft_consensus.cc:2749] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:47.288187 16396 raft_consensus.cc:1240] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:47.220445 16394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 763) took 1771 ms. Trace:
I20260709 12:22:47.289283 18629 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:47.289811 18629 raft_consensus.cc:697] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:47.289532 16394 rpcz_store.cc:276] 0709 12:22:45.449373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:45.490707 (+ 41334us) service_pool.cc:224] Handling call
0709 12:22:45.491064 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:46.803056 (+1311992us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:47.220097 (+417041us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54008a6bd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:47.220377 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3014036352}
I20260709 12:22:47.219719 16224 catalog_manager.cc:5697] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 1 to 5, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 5 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:47.220676 18626 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:47.221057 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:47.291230 18626 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:47.291850 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:47.232419 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:47.293464 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:47.293591 18626 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:47.292920 16224 catalog_manager.cc:5697] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
W20260709 12:22:47.221807 18410 consensus_meta.cc:287] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.696s	user 0.019s	sys 0.006s
I20260709 12:22:47.294690 18410 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:47.294764 16224 catalog_manager.cc:5697] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 3 to 4, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 4 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:47.296867 18410 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:47.203456 18637 consensus_queue.cc:260] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:47.297613 18626 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:47.157256 18529 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:47.238225 18466 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:47.298418 18626 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:47.302099 17258 raft_consensus.cc:1275] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:47.302397 18626 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:47.298516 18629 consensus_queue.cc:237] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:47.303830 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.232173 18634 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:47.240430 16471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 322) took 2588 ms. Trace:
I20260709 12:22:47.304526 17258 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:47.304638 16471 rpcz_store.cc:276] 0709 12:22:44.651861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:44.652382 (+   521us) service_pool.cc:224] Handling call
0709 12:22:46.753319 (+2100937us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:46.915767 (+162448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:47.240183 (+324416us) spinlock_profiling.cc:243] Waited 2.1 s on lock 0x7b5400a68bd0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:47.240347 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4830595584}
I20260709 12:22:47.307906 16224 catalog_manager.cc:5697] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 3 to 5, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 5 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:22:47.309531 17263 raft_consensus.cc:1275] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:47.309728 16224 catalog_manager.cc:5697] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 3 to 5, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 5 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
W20260709 12:22:47.312595 18273 consensus_meta.cc:287] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 1.582s	user 0.017s	sys 0.009s
I20260709 12:22:47.312690 18529 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:47.313310 18273 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:47.314210 18380 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:47.314770 16549 raft_consensus.cc:2393] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 7: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:47.314970 18380 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:47.240756 16395 raft_consensus.cc:1275] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:47.234606 16397 rpcz_store.cc:276] 0709 12:22:46.230223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:46.232617 (+  2394us) service_pool.cc:224] Handling call
0709 12:22:46.233044 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:46.744728 (+511684us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:22:46.915482 (+170754us) log.cc:844] Serialized 14 byte log entry
0709 12:22:47.029168 (+113686us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:47.029219 (+    51us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:47.033222 (+  4003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:47.233334 (+200112us) spinlock_profiling.cc:243] Waited 511 ms on lock 0x7b54009bebd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:47.233396 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:47.233668 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1808,"prepare.queue_time_us":228263,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":111,"raft.queue_time_us":7524,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":7938,"spinlock_wait_cycles":1174529280,"thread_start_us":246816,"threads_started":2,"wal-append.queue_time_us":164499}
I20260709 12:22:47.200488 18002 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:47.318084 18380 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:22:47.243422 18166 consensus_meta.cc:287] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 1.635s	user 0.015s	sys 0.011s
I20260709 12:22:47.318496 18002 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:47.244836 18439 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:47.318917 18166 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:47.320238 18637 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:47.320585 17261 raft_consensus.cc:1240] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 4. Current term is 5. Ops: []
W20260709 12:22:47.247082 16470 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34826 (request call id 848) took 1385 ms (client timeout 1701 ms). Trace:
I20260709 12:22:47.322261 17261 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 617) took 1839 ms. Trace:
I20260709 12:22:47.322081 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:22:47.322489 16470 rpcz_store.cc:269] 0709 12:22:45.861311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:45.870687 (+  9376us) service_pool.cc:224] Handling call
0709 12:22:47.246692 (+1376005us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b5400b2fcd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:47.246839 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":732127104}
I20260709 12:22:47.322634 17261 rpcz_store.cc:276] 0709 12:22:45.483113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:45.483971 (+   858us) service_pool.cc:224] Handling call
0709 12:22:45.484217 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:47.321104 (+1836887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:47.322008 (+   904us) spinlock_profiling.cc:243] Waited 1.84 s on lock 0x7b54002800d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:47.322206 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4222296320}
I20260709 12:22:47.223594 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:47.326200 18166 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:47.330762 18634 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:47.250250 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:47.253971 17255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 690) took 1634 ms. Trace:
I20260709 12:22:47.260015 18515 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:47.336735 17255 rpcz_store.cc:276] 0709 12:22:45.619250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:45.620927 (+  1677us) service_pool.cc:224] Handling call
0709 12:22:45.638930 (+ 18003us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:45.870985 (+232055us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:47.253386 (+1382401us) spinlock_profiling.cc:243] Waited 97.5 ms on lock 0x7b540063ea54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:47.253765 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":224350592}
W20260709 12:22:47.249583 18640 log.cc:927] Time spent T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:22:47.262463 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:47.262486 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:47.264679 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:47.265952 18480 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.266634 18290 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:47.340523 18380 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:47.341259 18380 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:47.344072 18380 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:47.341323 18290 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:47.270709 16442 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:47.345973 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 75486us
I20260709 12:22:47.348423 18290 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:47.272500 18601 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.273368 16441 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
W20260709 12:22:47.351015 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 77704.5us
I20260709 12:22:47.350831 18601 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:47.351747 16469 raft_consensus.cc:1275] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 12:22:47.352166 16469 pending_rounds.cc:85] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 3
I20260709 12:22:47.352500 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:47.352691 18466 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:47.353367 16470 raft_consensus.cc:2393] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 9: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:47.354152 18466 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:47.354605 18466 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:47.355015 18466 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:47.352478 16469 pending_rounds.cc:107] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:22:47.355595 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:47.356353 16477 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:47.356772 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:47.358676 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:22:47.267374 18588 consensus_meta.cc:287] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 1.027s	user 0.000s	sys 0.021s
I20260709 12:22:47.277695 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:47.359277 16471 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
I20260709 12:22:47.355647 16469 raft_consensus.cc:2889] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:47.283763 18456 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:47.211130 16398 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:47.297749 18410 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.355060 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:47.360561 18410 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:47.361061 16475 raft_consensus.cc:2410] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 12:22:47.361019 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 74133.4us
I20260709 12:22:47.362149 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:47.362252 16514 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:47.362872 18410 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:47.363901 16398 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:47.364265 16469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 849) took 1484 ms. Trace:
W20260709 12:22:47.276619 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 71474.1us
I20260709 12:22:47.244942 18581 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:47.260871 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:47.341791 18480 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:47.356290 18601 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:47.276834 17665 consensus_queue.cc:1048] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:47.359769 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:47.365155 16398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:46252 (request call id 804) took 1134 ms. Trace:
I20260709 12:22:47.316054 18273 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:47.364725 16469 rpcz_store.cc:276] 0709 12:22:45.879923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:46.935907 (+1055984us) service_pool.cc:224] Handling call
0709 12:22:46.945839 (+  9932us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:47.360236 (+414397us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:47.364031 (+  3795us) spinlock_profiling.cc:243] Waited 405 ms on lock 0x7b54000b5c54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:47.364181 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":932599168}
I20260709 12:22:47.364698 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:47.380211 16475 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:47.363996 16514 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:47.311868 16224 catalog_manager.cc:5697] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:22:47.358289 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:47.365410 16473 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:22:47.369189 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:47.384581 16402 raft_consensus.cc:2410] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:22:47.378332 16398 rpcz_store.cc:276] 0709 12:22:46.230625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:46.231329 (+   704us) service_pool.cc:224] Handling call
0709 12:22:47.362623 (+1131294us) spinlock_profiling.cc:243] Waited 779 ms on lock 0x7b54009bebd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e80a3d 00007f287ae50cd0 00007f28743d5254 00007f28743d51e2 00007f28743d5111 00007f28743d508c 00007f28743d4342 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:47.364989 (+  2366us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1791930112}
I20260709 12:22:47.388990 16470 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:22:47.364658 18558 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:47.396584 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:47.404160 16471 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:47.406136 16472 raft_consensus.cc:1275] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:47.406284 18464 consensus_queue.cc:1048] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:47.364759 18456 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:47.408715 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:47.409482 16547 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:47.409854 18456 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:47.410724 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.411758 18480 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:47.379784 18581 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:47.412652 18581 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:47.413213 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 49376.9us
I20260709 12:22:47.412779 16224 catalog_manager.cc:5697] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 3 to 4, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 4 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:47.414932 18581 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:47.415570 16514 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:47.397812 18002 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:47.416145 18439 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:47.416599 18464 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:47.416648 18290 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:47.414628 16224 catalog_manager.cc:5697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 5 to 8, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 8 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:47.387174 18601 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.431030 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:47.416219 18529 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:47.425825 18464 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:47.428928 18480 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.446542 16224 catalog_manager.cc:5697] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 1 to 4, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 4 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
I20260709 12:22:47.463057 17665 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:47.380686 18588 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:47.460711 16548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1066) took 1152 ms. Trace:
I20260709 12:22:47.477707 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:47.428926 18002 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:47.463930 18464 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:47.481093 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:47.481011 16224 catalog_manager.cc:5697] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 1 to 3, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 3 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:22:47.478511 16397 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:22:47.485714 18222 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.486382 18222 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:47.486164 16224 catalog_manager.cc:5697] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 1 to 2, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 2 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:47.489179 18222 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:47.465044 18047 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:47.449450 18601 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:47.457728 18634 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:47.484587 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:47.490137 18047 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:47.488257 16224 catalog_manager.cc:5697] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:22:47.450914 18290 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:47.493995 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:47.460539 18439 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:47.451582 18558 consensus_queue.cc:1048] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 12:22:47.498978 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:47.511734 16546 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:47.512171 16397 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:22:47.499560 18616 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:47.500079 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.501135 18389 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.501616 18166 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.502068 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.494752 18581 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.543805 18634 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:47.544842 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:47.545104 18634 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:47.508821 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 93359.5us
I20260709 12:22:47.545125 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.546195 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.546411 17263 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:47.546602 16514 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:47.510012 18290 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:47.554710 18558 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:47.555004 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:47.555409 18558 raft_consensus.cc:697] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:47.556689 18558 consensus_queue.cc:237] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:47.557565 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:47.558331 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:47.477751 18480 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:47.481669 18529 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:47.466207 16548 rpcz_store.cc:276] 0709 12:22:46.301982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:46.302098 (+   116us) service_pool.cc:224] Handling call
0709 12:22:46.365301 (+ 63203us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:46.365578 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:22:46.992202 (+626624us) log.cc:844] Serialized 14 byte log entry
0709 12:22:47.149953 (+157751us) spinlock_profiling.cc:243] Waited 6.13 ms on lock 0x7b1800075ca0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a2923a 00007f2875e93a2a
0709 12:22:47.382017 (+232064us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:47.392032 (+ 10015us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:47.428276 (+ 36244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:47.438167 (+  9891us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:47.453853 (+ 15686us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6448,"prepare.queue_time_us":621513,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":5424,"raft.queue_time_us":12802,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":3840,"spinlock_wait_cycles":14098304,"thread_start_us":792497,"threads_started":2,"wal-append.queue_time_us":424720}
I20260709 12:22:47.534180 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:47.519819 16398 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
W20260709 12:22:47.538741 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 43618.2us
I20260709 12:22:47.530864 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:47.542361 18166 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:47.582450 18389 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:47.583048 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.584266 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.584422 18166 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:47.584843 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.585451 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:47.585716 18439 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:47.586020 17269 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:47.586119 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.587710 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.544153 18581 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:47.588200 18480 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:47.588840 18480 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:47.588929 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.589370 18480 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:47.590085 18581 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:47.590301 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.590986 17271 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:22:47.591640 18480 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:47.494607 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.504103 16516 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:47.513576 16224 catalog_manager.cc:5697] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 1 to 6. New cstate: current_term: 6 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:22:47.510813 18588 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:47.571115 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.575578 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:47.577369 18529 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:47.515272 18047 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:47.577931 18616 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:47.581378 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.581924 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.569308 16441 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:47.592607 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:47.598037 16396 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
W20260709 12:22:47.599275 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 50913.1us
I20260709 12:22:47.599659 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.584841 18389 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:47.614527 17665 consensus_queue.cc:1048] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:47.600864 16366 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:47.602361 18644 log.cc:927] Time spent T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:22:47.602728 17269 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
I20260709 12:22:47.610110 17258 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:47.610736 17261 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6.
I20260709 12:22:47.612591 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.612941 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.614476 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:47.515650 18002 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:47.564155 18601 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:47.620056 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 116043us
I20260709 12:22:47.575649 18634 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:47.630264 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:22:47.632722 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 138120us
I20260709 12:22:47.643854 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.644987 16366 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:47.632951 16224 catalog_manager.cc:5697] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 1 to 3, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 3 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:22:47.636710 18047 raft_consensus.cc:2764] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:22:47.637015 16442 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:47.636785 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:22:47.637894 18653 log.cc:927] Time spent T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.083s	user 0.000s	sys 0.003s
I20260709 12:22:47.646579 17261 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:47.648149 17255 raft_consensus.cc:1275] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:47.649448 16515 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:47.653690 18066 consensus_queue.cc:1048] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:47.648355 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.655587 17665 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:47.658143 17665 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:47.658519 17665 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:47.621241 18529 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:47.661193 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:47.666498 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.670548 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:47.639447 18629 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:47.677354 16472 raft_consensus.cc:2393] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 10: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:47.684192 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:47.685487 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:47.686501 16470 raft_consensus.cc:2468] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
I20260709 12:22:47.689569 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:47.697880 18540 consensus_queue.cc:1048] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 12:22:47.698535 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:47.699788 18540 consensus_queue.cc:1059] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:47.701025 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:47.701679 16474 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:22:47.704622 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:47.705260 16476 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:22:47.708554 16514 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:47.709327 16468 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
W20260709 12:22:47.709993 18652 log.cc:927] Time spent T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.164s	user 0.000s	sys 0.002s
I20260709 12:22:47.660164 18466 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:47.660669 18581 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:47.691428 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:47.715562 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.712118 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:47.728529 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.738018 17269 raft_consensus.cc:2393] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 5: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:47.714053 16224 catalog_manager.cc:5697] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 1 to 2, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 2 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:47.723572 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.732563 18439 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:47.732775 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:47.733225 16399 raft_consensus.cc:1275] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:47.743443 17258 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:47.733768 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:47.734025 17144 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:47.674700 18529 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:47.734611 18637 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.735112 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:47.734418 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:47.662177 18047 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:47.734405 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:47.748692 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.691836 18629 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:47.774322 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:47.776860 17266 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:47.780982 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:47.782107 16470 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:47.789412 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 55445.8us
W20260709 12:22:47.794194 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:47.797222 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.799764 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:47.797334 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:47.758801 18529 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:47.800633 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:47.811090 16401 raft_consensus.cc:2393] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 5: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:47.757722 18047 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:47.804692 18545 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:47.814810 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.819393 18480 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:47.819844 18290 consensus_queue.cc:1059] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:47.822407 16546 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:22:47.822947 16473 raft_consensus.cc:1275] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:22:47.836283 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:22:47.867805 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 55729.1us
I20260709 12:22:47.823402 18581 consensus_queue.cc:1048] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:47.829387 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:47.829612 18616 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:47.788406 18629 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:47.776199 18637 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:47.809180 18273 consensus_queue.cc:1048] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 12:22:47.849617 17144 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:47.819960 16366 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:47.890565 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:47.899231 16544 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:47.852213 18047 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:47.897312 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 247930us
I20260709 12:22:47.894340 17665 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:47.861358 18290 raft_consensus.cc:3055] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:47.834419 18558 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:47.902431 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 82551.8us
I20260709 12:22:47.797806 18439 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:47.856717 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.832240 18466 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:47.851104 16550 pending_rounds.cc:85] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1: Aborting all ops after (but not including) 2
I20260709 12:22:47.906033 16515 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:47.843855 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.892575 17266 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:22:47.922258 17665 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:47.922900 16366 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:47.934669 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:47.916230 18515 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:47.860806 18480 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:47.944870 17258 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:47.913900 18290 raft_consensus.cc:740] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:47.942363 16366 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:47.953893 16366 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:47.957350 18222 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:47.930140 18166 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:47.958146 18222 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:47.958725 18222 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:47.959642 18637 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:47.945829 18359 consensus_queue.cc:1048] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 12:22:47.961035 16515 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:47.961970 18439 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:47.962452 18582 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:47.965210 16442 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:47.966933 16442 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:47.929104 18601 consensus_queue.cc:1048] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 12:22:47.971052 18222 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:47.972522 18403 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:47.973181 18403 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:47.973681 18403 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:47.976735 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:47.957137 17143 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:22:47.978320 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 128817us
W20260709 12:22:47.981307 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 62757.8us
I20260709 12:22:47.982278 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.982853 16442 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:47.950507 17142 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:47.945127 17665 consensus_queue.cc:1048] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 12:22:47.988924 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:47.943833 18515 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:47.994884 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:48.002314 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:48.001981 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:48.005038 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:47.990754 17261 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:22:47.986070 16475 raft_consensus.cc:2393] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 5: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:48.001998 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:47.995995 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:48.007611 16474 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:22:47.992147 17665 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:47.989895 16550 pending_rounds.cc:107] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:22:47.985978 16442 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:48.005306 17264 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:47.995121 17144 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:48.014628 16442 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:48.009694 16550 raft_consensus.cc:2889] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:47.985405 18637 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:47.948411 17254 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6.
I20260709 12:22:47.955014 17255 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:47.990607 18360 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:47.988293 18634 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:48.033032 16442 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:47.994022 18359 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:47.988075 18466 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:48.011897 18515 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:48.039108 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:48.036619 18273 consensus_queue.cc:1059] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 12:22:48.042356 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:48.042554 16401 raft_consensus.cc:1275] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:48.064637 17270 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:48.035167 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:48.069620 16401 raft_consensus.cc:1240] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:48.036413 18360 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:48.043658 18637 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:48.043035 18290 consensus_queue.cc:260] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:48.045270 18222 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:48.046118 18529 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:48.034112 18410 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:48.080816 16472 raft_consensus.cc:2393] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 6: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:48.048472 18171 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:48.081578 18171 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:48.081943 18273 raft_consensus.cc:3055] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:48.082073 18171 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:48.082377 18273 raft_consensus.cc:740] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:48.083184 18558 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:22:48.050925 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 53396.7us
I20260709 12:22:48.083316 18273 consensus_queue.cc:260] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:48.084425 16441 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:48.079879 18529 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:48.085719 16441 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:48.086869 16441 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:48.088546 18466 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:48.089031 18466 raft_consensus.cc:697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 10 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:48.089848 18466 consensus_queue.cc:237] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:48.072765 18360 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:48.061465 18359 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:48.093531 18359 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:48.096621 18410 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:48.098943 18171 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:48.052119 18464 consensus_meta.cc:287] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.540s	user 0.012s	sys 0.000s
I20260709 12:22:48.011495 18601 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:48.021396 18581 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:48.058781 17665 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:48.076375 18290 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:48.051483 18480 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:48.085728 18529 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:48.120776 18273 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:48.122305 16441 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:48.079934 18403 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:48.152312 17270 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:48.177842 17264 raft_consensus.cc:1240] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:22:48.189044 18528 consensus_queue.cc:1048] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 12:22:48.191265 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:48.191475 18515 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:48.198518 18515 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:48.150807 18464 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:48.204877 18360 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:48.209370 16399 raft_consensus.cc:1240] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:48.210844 18403 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:48.212561 18634 raft_consensus.cc:2804] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:48.213935 18359 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:48.196045 18047 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:48.156718 16470 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:22:48.163044 18480 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:48.220573 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:48.225968 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:48.186431 18637 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:48.235308 16472 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
I20260709 12:22:48.222612 16441 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:22:48.217150 16514 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.033s	sys 0.029s
I20260709 12:22:48.162592 18529 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:48.143635 18581 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:48.231146 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:48.231791 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:48.244019 16471 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:48.242844 16475 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:22:48.216913 18464 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:48.232674 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:48.171201 18385 consensus_queue.cc:1059] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 12:22:48.171706 18601 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:48.241183 17258 raft_consensus.cc:2393] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 6: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
W20260709 12:22:48.240911 16514 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.034s	sys 0.029s
I20260709 12:22:48.250511 16402 raft_consensus.cc:1240] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
W20260709 12:22:48.249061 16516 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.009s	sys 0.046s
W20260709 12:22:48.252061 16516 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.009s	sys 0.046s
I20260709 12:22:48.246748 18529 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:48.246562 18581 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:48.249531 18403 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:48.250684 18588 raft_consensus.cc:3055] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:48.247800 18480 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:48.248934 18637 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:48.280279 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:48.280627 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:48.280942 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:48.281262 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:48.269169 18611 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.021s	sys 0.036s
I20260709 12:22:48.291137 18629 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:48.288067 18611 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.022s	sys 0.036s
I20260709 12:22:48.295631 18629 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:48.297376 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:48.298005 18360 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:48.298595 16542 raft_consensus.cc:2393] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 5: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
W20260709 12:22:48.300429 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 635029us
I20260709 12:22:48.301463 16514 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:48.271620 18588 raft_consensus.cc:740] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:22:48.283206 18540 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.017s	sys 0.017s
I20260709 12:22:48.311515 16471 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:22:48.273679 18403 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:48.281435 18545 consensus_queue.cc:1059] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:22:48.367247 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:48.326177 18540 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.017s	sys 0.017s
W20260709 12:22:48.332860 18389 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.021s	sys 0.021s
I20260709 12:22:48.320175 18515 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:48.320776 18166 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:48.381508 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 80119.1us
I20260709 12:22:48.350800 18360 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:48.378897 18389 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.021s	sys 0.021s
I20260709 12:22:48.399437 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:48.406464 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:48.406975 18659 log.cc:927] Time spent T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:22:48.411069 16401 raft_consensus.cc:1240] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:22:48.414501 18601 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:48.390463 18515 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:48.418398 18515 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:48.419224 18403 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:48.371079 18629 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:48.438360 18359 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:48.390057 18171 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:48.456853 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:48.444203 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:48.450093 18582 raft_consensus.cc:697] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 5 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:48.395793 18166 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:48.473357 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:48.434016 18360 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:48.455755 18047 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:48.479578 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:48.481561 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:48.481844 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:48.421649 18588 consensus_queue.cc:260] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:48.482295 16543 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:48.485749 16544 raft_consensus.cc:1240] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:48.487769 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:48.490425 16398 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.013s	sys 0.011s
W20260709 12:22:48.492247 16398 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.013s	sys 0.011s
I20260709 12:22:48.490311 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:48.474335 16542 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:48.499994 16515 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:48.504560 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:22:48.506611 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 473662us
I20260709 12:22:48.506908 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:48.507381 16442 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:48.509007 17142 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:48.508558 16515 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:48.516163 17264 raft_consensus.cc:1275] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:48.517850 16547 raft_consensus.cc:2393] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 5: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:48.505462 16548 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6.
I20260709 12:22:48.520846 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:48.521590 16548 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 6
W20260709 12:22:48.523129 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 96201us
I20260709 12:22:48.523463 17142 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:48.524578 17142 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:48.526657 16515 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:48.530026 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:48.508749 18588 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:48.503614 18515 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:48.531778 16471 raft_consensus.cc:1275] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:22:48.532156 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 309633us
I20260709 12:22:48.541375 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:48.506255 18480 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:48.515962 18669 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:48.546936 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:48.549278 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:48.544286 16397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 816) took 1001 ms. Trace:
I20260709 12:22:48.532980 16515 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:48.548277 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:48.503793 18528 consensus_queue.cc:1059] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:22:48.531005 18466 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:48.526077 18661 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:48.531507 18002 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:48.523847 18637 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:48.554016 16441 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:48.560082 16397 rpcz_store.cc:276] 0709 12:22:47.529817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:47.539158 (+  9341us) service_pool.cc:224] Handling call
0709 12:22:47.562253 (+ 23095us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:47.597554 (+ 35301us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:22:47.670143 (+ 72589us) log.cc:844] Serialized 14 byte log entry
0709 12:22:48.530976 (+860833us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:48.531045 (+    69us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:48.531104 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:48.531188 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:48.531321 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":59622,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":187918,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":100,"thread_start_us":790914,"threads_started":2,"wal-append.queue_time_us":779073}
I20260709 12:22:48.560251 16477 raft_consensus.cc:2393] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 6: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:48.559823 16547 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:48.560518 16475 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:48.549729 18629 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:48.597117 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:48.527451 18666 consensus_queue.cc:1048] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
W20260709 12:22:48.599026 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 491853us
I20260709 12:22:48.569069 18466 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:48.571208 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:48.571319 18661 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:22:48.631139 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 106621us
I20260709 12:22:48.592223 18637 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:22:48.636345 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:48.553586 18634 raft_consensus.cc:697] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:48.650751 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:48.643312 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:48.618645 18466 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:48.602571 18676 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:48.559042 18515 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:48.683533 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:48.684604 16477 raft_consensus.cc:1240] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:48.632611 18629 raft_consensus.cc:697] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:48.550277 18582 consensus_queue.cc:237] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:48.670961 16442 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.613s (SENT) [suppressed 27 similar messages]
I20260709 12:22:48.631619 18661 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:48.598286 18675 consensus_queue.cc:1048] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.079s
I20260709 12:22:48.640323 18528 raft_consensus.cc:3055] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:48.734908 16475 raft_consensus.cc:1275] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:48.638789 16542 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:22:48.746621 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:48.747496 16401 raft_consensus.cc:2410] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:22:48.747289 16403 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:48.734128 18528 raft_consensus.cc:740] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:22:48.742985 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 155437us
I20260709 12:22:48.745991 18171 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:48.758322 18528 consensus_queue.cc:260] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:48.759620 18528 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:48.731197 18410 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:48.731482 18013 raft_consensus.cc:2804] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:48.760682 18013 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:48.761101 18013 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:48.761190 18410 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:48.761744 18410 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:48.763998 18410 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:48.739104 18360 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:48.768981 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:48.770584 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:48.771766 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:48.773283 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:48.774230 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:48.775153 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:48.775279 16403 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:22:48.786968 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:48.787737 16403 raft_consensus.cc:2393] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 7: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
W20260709 12:22:48.789990 18686 log.cc:927] Time spent T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:22:48.790786 16366 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:48.791934 16366 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:48.792475 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 185840us
I20260709 12:22:48.793174 16367 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:48.794433 16367 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:48.795372 18047 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:48.796514 18689 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:48.797331 18601 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:48.798141 18480 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:48.795455 16367 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:48.698112 18066 raft_consensus.cc:3055] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:48.707859 18676 raft_consensus.cc:697] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:48.783260 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:48.782128 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:48.809559 16548 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6.
I20260709 12:22:48.847746 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:48.816545 16366 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:48.816912 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 163684us
I20260709 12:22:48.799960 16551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 353) took 1161 ms. Trace:
I20260709 12:22:48.825977 16547 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6.
W20260709 12:22:48.776418 18671 log.cc:927] Time spent T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.290s	user 0.001s	sys 0.000s
I20260709 12:22:48.793246 16468 raft_consensus.cc:1275] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:22:48.858450 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:48.860558 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:48.784130 18685 log.cc:927] Time spent T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 12:22:48.778613 18684 log.cc:927] Time spent T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:22:48.835629 16367 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:48.669625 18637 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:48.849308 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 50173.2us
I20260709 12:22:48.867878 17142 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:48.871163 17142 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:48.817276 18689 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:48.858966 16468 pending_rounds.cc:85] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 2
I20260709 12:22:48.885243 16468 pending_rounds.cc:107] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:22:48.885625 16468 raft_consensus.cc:2889] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:48.849295 16551 rpcz_store.cc:276] 0709 12:22:47.638294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:47.652018 (+ 13724us) service_pool.cc:224] Handling call
0709 12:22:47.652412 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:47.929584 (+277172us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:47.929629 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:47.936218 (+  6589us) log.cc:844] Serialized 26 byte log entry
0709 12:22:48.482833 (+546615us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:48.482877 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:48.482969 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:48.799730 (+316761us) raft_consensus.cc:1685] finished
0709 12:22:48.799750 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:48.799906 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":46854,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"thread_start_us":768772,"threads_started":2,"wal-append.queue_time_us":686150}
I20260709 12:22:48.875072 17142 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:48.825415 18410 raft_consensus.cc:2804] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:22:48.883656 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 212721us
I20260709 12:22:48.895256 16514 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:22:48.873602 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 192227us
I20260709 12:22:48.833981 18066 raft_consensus.cc:740] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:48.843474 18403 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:48.904786 17142 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:48.914793 17142 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:48.861709 18166 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:22:48.885537 18581 consensus_meta.cc:287] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.610s	user 0.012s	sys 0.000s
I20260709 12:22:48.826452 18047 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:48.884271 18359 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:22:48.944047 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 108435us
I20260709 12:22:48.902686 18689 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:48.945816 16367 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:48.904700 18480 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:48.904834 18601 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:48.893255 18410 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:48.900307 18698 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:22:48.976220 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:48.952437 16441 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:48.777671 18629 consensus_queue.cc:237] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:48.913170 18664 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:48.963277 16367 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:48.920953 18690 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:48.920722 18403 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:22:48.954262 18684 log.cc:927] Time spent T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 12:22:48.945561 18290 consensus_meta.cc:287] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.788s	user 0.012s	sys 0.000s
I20260709 12:22:48.945390 18166 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:48.937116 18687 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:49.000622 18403 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:48.970454 18480 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:49.045842 16442 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:49.050349 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:48.976895 18410 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:48.944056 18691 consensus_queue.cc:1059] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 12:22:49.121794 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:49.135175 16401 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:49.122856 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:48.942713 18581 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:48.989122 18601 raft_consensus.cc:697] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
W20260709 12:22:49.142264 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 227536us
I20260709 12:22:49.133513 16395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 787) took 1315 ms. Trace:
I20260709 12:22:49.143232 17142 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:49.147137 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:49.148058 16400 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:22:49.143007 16395 rpcz_store.cc:276] 0709 12:22:47.818223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:47.820449 (+  2226us) service_pool.cc:224] Handling call
0709 12:22:47.820796 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:47.821270 (+   474us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:22:47.821300 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:47.821854 (+   554us) log.cc:844] Serialized 26 byte log entry
0709 12:22:48.283165 (+461311us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:48.283213 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:48.283278 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:49.133273 (+849995us) raft_consensus.cc:1685] finished
0709 12:22:49.133297 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:49.133455 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":95,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"thread_start_us":460854,"threads_started":1,"wal-append.queue_time_us":633609}
I20260709 12:22:49.157068 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:49.156009 16367 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
W20260709 12:22:49.164297 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 211988us
I20260709 12:22:49.157961 16396 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:22:49.165889 16441 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:49.166973 16441 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:49.169991 17142 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:49.132433 18480 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:48.921792 18634 consensus_queue.cc:237] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:49.178665 16475 raft_consensus.cc:1275] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:49.186266 17142 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:49.128062 18718 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:49.159085 18290 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:49.150287 18710 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:49.210103 18060 consensus_queue.cc:1048] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.304s
I20260709 12:22:49.042804 18690 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:49.231643 18694 log.cc:927] Time spent T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.387s	user 0.000s	sys 0.001s
I20260709 12:22:49.171204 18699 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:22:49.243773 18693 log.cc:927] Time spent T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:22:49.244946 16516 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:49.219399 18290 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:49.265532 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 79381.3us
W20260709 12:22:49.260423 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 104492us
I20260709 12:22:49.268465 17142 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:49.268981 16367 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:49.204159 18718 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:49.207175 18691 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:49.231206 18601 consensus_queue.cc:237] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:49.265588 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 219778us
I20260709 12:22:49.288636 16442 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:49.275794 16516 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:49.125174 18066 consensus_queue.cc:260] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:49.229086 18515 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:49.224843 18060 consensus_queue.cc:1243] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer 52ab50b0237346c1a13ede947f6a63c2 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 12:22:49.262631 18466 consensus_meta.cc:287] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.565s	user 0.010s	sys 0.000s
I20260709 12:22:49.303719 16367 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:49.113229 18676 consensus_queue.cc:237] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:49.274964 18290 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:49.285885 18540 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:49.293355 18712 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:49.273692 18718 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:49.328517 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 161533us
I20260709 12:22:49.331751 16441 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
W20260709 12:22:49.324440 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 401094us
I20260709 12:22:49.300441 18666 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:22:49.336900 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 68502.3us
I20260709 12:22:49.304455 18679 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:49.310539 18611 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:49.294009 18664 consensus_queue.cc:1048] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 12:22:49.320106 18515 raft_consensus.cc:2749] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:49.344794 17142 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:49.331272 18222 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:49.343845 18695 log.cc:927] Time spent T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.443s	user 0.002s	sys 0.000s
I20260709 12:22:49.322119 18721 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:49.328627 18637 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:49.337934 18047 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:49.311095 18466 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:49.347128 18699 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:22:49.358232 18681 log.cc:927] Time spent T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.373s	user 0.001s	sys 0.001s
I20260709 12:22:49.315891 18691 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:49.370278 18066 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:49.395450 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 106931us
W20260709 12:22:49.401855 18661 consensus_meta.cc:287] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.600s	user 0.010s	sys 0.002s
I20260709 12:22:49.402320 18661 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:49.403565 18683 log.cc:927] Time spent T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.669s	user 0.004s	sys 0.011s
W20260709 12:22:49.404374 18680 log.cc:927] Time spent T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 12:22:49.405128 18731 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:49.406173 17261 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 642) took 1409 ms. Trace:
I20260709 12:22:49.405956 18731 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:49.406582 17261 rpcz_store.cc:276] 0709 12:22:47.997091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:48.019421 (+ 22330us) service_pool.cc:224] Handling call
0709 12:22:48.019916 (+   495us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:48.042914 (+ 22998us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:48.042945 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:48.043432 (+   487us) log.cc:844] Serialized 26 byte log entry
0709 12:22:48.487553 (+444121us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:48.487590 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:48.487655 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:49.405960 (+918305us) raft_consensus.cc:1685] finished
0709 12:22:49.405979 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:49.406118 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8366,"raft.queue_time_us":7950,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":93,"spinlock_wait_cycles":1045376,"thread_start_us":442960,"threads_started":1,"wal-append.queue_time_us":1142263}
I20260709 12:22:49.408471 18724 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:49.408953 18724 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:49.409317 18724 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:49.350973 18611 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:49.374650 18701 consensus_queue.cc:1048] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.025s
W20260709 12:22:49.406178 18704 log.cc:927] Time spent T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 12:22:49.455950 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 142793us
I20260709 12:22:49.461846 18711 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:49.487980 18711 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:49.488550 18711 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:49.489322 18466 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:49.490381 16550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 366) took 1642 ms. Trace:
I20260709 12:22:49.462811 18719 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:49.491250 18719 raft_consensus.cc:697] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 6 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:49.490796 16550 rpcz_store.cc:276] 0709 12:22:47.847883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:47.850530 (+  2647us) service_pool.cc:224] Handling call
0709 12:22:47.850913 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:48.026061 (+175148us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:48.026093 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:48.032088 (+  5995us) log.cc:844] Serialized 26 byte log entry
0709 12:22:48.507236 (+475148us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:48.507270 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:48.507981 (+   711us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:49.403063 (+895082us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:22:49.490066 (+ 87003us) raft_consensus.cc:1685] finished
0709 12:22:49.490088 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:49.490313 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35238272,"thread_start_us":413979,"threads_started":1,"wal-append.queue_time_us":536986}
I20260709 12:22:49.492547 18719 consensus_queue.cc:237] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:49.470132 18705 raft_consensus.cc:3055] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:22:49.488142 18588 consensus_meta.cc:287] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.933s	user 0.004s	sys 0.004s
I20260709 12:22:49.458868 18702 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:49.450773 18731 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:22:49.477595 18688 log.cc:927] Time spent T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.488s	user 0.001s	sys 0.000s
I20260709 12:22:49.481735 18691 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:22:49.485159 18709 log.cc:927] Time spent T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:22:49.462229 18723 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:49.410102 18002 raft_consensus.cc:2804] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:22:49.450178 18681 log.cc:927] Time spent T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:22:49.392501 18699 raft_consensus.cc:697] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:22:49.497287 16468 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.004s	sys 0.015s
W20260709 12:22:49.524435 16468 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.005s	sys 0.015s
I20260709 12:22:49.397974 18692 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:49.470757 16402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 489) took 1437 ms. Trace:
W20260709 12:22:49.512063 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 167362us
I20260709 12:22:49.512790 16516 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:49.526484 17665 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:49.527091 17665 raft_consensus.cc:697] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:49.527817 16516 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:49.525985 16402 rpcz_store.cc:276] 0709 12:22:48.033298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:48.264263 (+230965us) service_pool.cc:224] Handling call
0709 12:22:48.272909 (+  8646us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:48.273158 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:48.273183 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:48.276228 (+  3045us) log.cc:844] Serialized 26 byte log entry
0709 12:22:48.589328 (+313100us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:48.589392 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:48.589496 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:49.470457 (+880961us) raft_consensus.cc:1685] finished
0709 12:22:49.470485 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:49.470683 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":279523,"threads_started":1,"wal-append.queue_time_us":466459}
W20260709 12:22:49.529233 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 225640us
I20260709 12:22:49.527973 17665 consensus_queue.cc:237] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:49.530992 18664 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:49.533114 16367 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:49.534467 16367 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:49.535842 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:49.535637 18735 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:49.537199 18735 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:49.537719 16367 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:49.538720 18687 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:22:49.539129 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:49.539346 18464 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:49.539492 16367 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:49.539870 18464 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:49.514321 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 173325us
W20260709 12:22:49.520697 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 186875us
I20260709 12:22:49.541950 18464 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:49.543511 18687 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:49.545137 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:49.545753 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:49.546677 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:49.547068 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:49.545814 16470 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:49.548262 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:49.549204 16474 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:22:49.545814 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:49.552320 16475 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:22:49.556941 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:49.557818 16469 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:49.559689 16442 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:49.561664 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:49.563736 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:49.564611 16400 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:22:49.565977 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:49.566936 16366 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:49.567154 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:49.567898 16400 raft_consensus.cc:2368] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6.
I20260709 12:22:49.568506 18687 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:49.568984 18687 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:49.569406 18687 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:49.570009 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:49.570902 16396 raft_consensus.cc:2393] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 4: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:49.571171 16470 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:22:49.573220 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:49.573542 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:49.573830 16550 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:49.573823 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:49.574249 16542 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:49.574628 16545 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
I20260709 12:22:49.575517 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:49.575850 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:49.575984 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:49.572103 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:49.580802 18736 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:49.581445 18736 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:49.583662 18738 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:49.584597 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:49.585033 17271 raft_consensus.cc:1240] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:22:49.585476 18736 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:49.576306 16366 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:49.586566 18601 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:49.586678 16366 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:49.587008 18601 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:49.587468 18601 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:49.587548 18735 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:49.588172 18735 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:49.588608 18735 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:49.590938 18738 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:49.595062 16367 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:49.596313 18601 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:49.595360 18738 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:49.598438 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:49.598779 16366 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:49.574601 16400 raft_consensus.cc:1240] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:49.662647 18679 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:49.663084 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:49.599997 17269 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:49.495939 18661 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:49.602207 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:49.602761 17143 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:49.510568 18705 raft_consensus.cc:740] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:49.606328 18667 consensus_queue.cc:1059] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.089s
I20260709 12:22:49.608171 18222 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:49.608652 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:49.665194 18705 consensus_queue.cc:260] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:49.665987 16395 raft_consensus.cc:1275] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:22:49.599889 16550 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:22:49.666676 18705 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:49.613056 18529 consensus_meta.cc:287] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 1.349s	user 0.007s	sys 0.007s
I20260709 12:22:49.604475 16367 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:49.667372 18529 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:49.609611 18699 consensus_queue.cc:237] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:49.668543 18515 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:49.669044 16472 raft_consensus.cc:2393] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 7: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:49.664804 16400 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:22:49.520138 18702 raft_consensus.cc:697] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 6 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:49.670382 18529 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:49.671056 18702 consensus_queue.cc:237] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:49.519662 18002 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:49.671757 16400 raft_consensus.cc:1240] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:49.647511 16473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 914) took 1427 ms. Trace:
W20260709 12:22:49.614146 18273 consensus_meta.cc:287] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 1.439s	user 0.015s	sys 0.000s
W20260709 12:22:49.654803 16476 consensus_meta.cc:287] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.879s	user 0.012s	sys 0.012s
I20260709 12:22:49.663246 18679 raft_consensus.cc:697] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:49.609241 18728 raft_consensus.cc:2804] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:22:49.668005 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 63652.1us
I20260709 12:22:49.672233 18002 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:49.631451 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:49.689191 16545 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:49.663656 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 64961.6us
I20260709 12:22:49.619549 16401 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6.
W20260709 12:22:49.676975 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 74278.6us
I20260709 12:22:49.690714 17143 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:49.677625 18739 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:49.720058 16546 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:22:49.721058 16549 raft_consensus.cc:1240] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:49.680830 18687 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:49.620682 18736 consensus_queue.cc:1048] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 12:22:49.673293 16473 rpcz_store.cc:276] 0709 12:22:48.220145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:48.221484 (+  1339us) service_pool.cc:224] Handling call
0709 12:22:48.231067 (+  9583us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:48.558413 (+327346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:48.558456 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:48.565005 (+  6549us) log.cc:844] Serialized 26 byte log entry
0709 12:22:48.894467 (+329462us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:48.894615 (+   148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:48.894872 (+   257us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:49.647248 (+752376us) raft_consensus.cc:1685] finished
0709 12:22:49.647272 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:49.647444 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":259008,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":5368,"thread_start_us":582774,"threads_started":2,"wal-append.queue_time_us":717397}
I20260709 12:22:49.685611 16476 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
W20260709 12:22:49.686161 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 126548us
I20260709 12:22:49.687937 16367 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
W20260709 12:22:49.689591 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 57298.7us
I20260709 12:22:49.638007 18464 consensus_queue.cc:1059] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
W20260709 12:22:49.685629 18697 log.cc:927] Time spent T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.606s	user 0.000s	sys 0.002s
I20260709 12:22:49.722568 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:49.636363 18718 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:49.739558 16396 raft_consensus.cc:1240] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:49.684458 18171 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:22:49.731446 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 43620.1us
I20260709 12:22:49.688776 18728 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:49.732815 16515 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:49.732043 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:49.737703 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:49.742106 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:49.743053 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:49.749862 17269 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:22:49.750464 16470 raft_consensus.cc:2410] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:22:49.756302 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:49.757381 17270 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:22:49.728312 18166 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:49.752914 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:49.766959 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:49.769134 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:49.770063 16401 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:49.769205 17258 raft_consensus.cc:2393] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 7: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:49.771260 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:49.771282 16516 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:49.772173 17270 raft_consensus.cc:2410] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:22:49.735083 18222 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:49.735110 18637 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:49.785370 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:49.798343 17271 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:49.790184 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 57447.7us
I20260709 12:22:49.799086 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:49.792184 17144 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:49.751219 18728 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:49.795218 18464 raft_consensus.cc:3055] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:49.795575 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:49.753369 18171 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:49.751469 18743 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:49.773730 18661 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:49.807629 16543 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:49.774278 18754 raft_consensus.cc:3055] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:49.819957 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:49.820866 17266 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:22:49.817319 17144 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:49.806253 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:49.822414 16471 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:22:49.802326 18464 raft_consensus.cc:740] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:49.858027 18464 consensus_queue.cc:260] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:49.859519 18464 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:49.864538 18696 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:49.803963 18667 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:49.866497 16441 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:49.808112 18739 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:22:49.817536 18480 consensus_meta.cc:287] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.611s	user 0.007s	sys 0.003s
I20260709 12:22:49.870651 16441 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:49.868364 16367 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:22:49.872151 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 54907.1us
I20260709 12:22:49.891314 18737 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:49.891714 16441 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:49.892649 18737 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:49.894255 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:49.894991 16550 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:49.895520 17258 raft_consensus.cc:1240] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:22:49.866189 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:49.902809 17144 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:49.867823 18667 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:49.889274 18529 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:49.926754 16471 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:22:49.887745 18480 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:49.895143 18222 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:49.895054 18698 raft_consensus.cc:2804] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:49.819375 18754 raft_consensus.cc:740] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:49.810680 18735 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:49.898110 18601 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:49.935496 16545 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:22:49.806834 18679 consensus_queue.cc:237] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:49.940676 16550 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:22:49.919409 18529 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:49.954741 16402 raft_consensus.cc:1240] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:22:49.918718 18667 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:49.898761 18719 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:22:49.867609 18696 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:49.953132 16367 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:49.960268 18601 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:49.941036 18698 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:49.908805 18661 raft_consensus.cc:697] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:49.936744 18710 consensus_queue.cc:1059] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 12:22:49.958300 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:22:49.955596 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:49.960731 18745 log.cc:927] Time spent T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 12:22:49.956638 18529 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:49.955765 18582 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:49.960774 18753 log.cc:927] Time spent T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
W20260709 12:22:49.993196 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:49.960933 18601 raft_consensus.cc:697] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 8 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:49.998378 16367 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:49.968982 18698 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:50.004936 18410 consensus_meta.cc:287] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.849s	user 0.012s	sys 0.000s
W20260709 12:22:50.026018 18724 consensus_meta.cc:287] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.616s	user 0.013s	sys 0.000s
I20260709 12:22:50.026585 18724 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:50.028299 16469 raft_consensus.cc:1240] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 7. Current term is 8. Ops: [5.4-7.5]
W20260709 12:22:50.029222 18755 log.cc:927] Time spent T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
W20260709 12:22:50.033044 18700 log.cc:927] Time spent T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.974s	user 0.000s	sys 0.007s
I20260709 12:22:50.033363 18480 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:50.036990 16399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 495) took 1840 ms. Trace:
I20260709 12:22:50.038710 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:50.037557 16399 rpcz_store.cc:276] 0709 12:22:48.196221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:48.433643 (+237422us) service_pool.cc:224] Handling call
0709 12:22:48.453927 (+ 20284us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:48.462764 (+  8837us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:48.462811 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:48.480118 (+ 17307us) log.cc:844] Serialized 52 byte log entry
0709 12:22:48.635377 (+155259us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:48.635422 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:48.635514 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:50.032160 (+1396646us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:22:50.036529 (+  4369us) raft_consensus.cc:1685] finished
0709 12:22:50.036568 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:50.036904 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3857,"raft.queue_time_us":100,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":5412,"spinlock_wait_cycles":36561792,"thread_start_us":115906,"threads_started":1,"wal-append.queue_time_us":440460}
I20260709 12:22:50.039511 17255 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:50.040740 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:50.045190 17255 raft_consensus.cc:2393] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 5: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:50.062381 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:50.011801 18047 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:22:50.064550 18757 log.cc:927] Time spent T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:22:50.066025 18661 consensus_queue.cc:237] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:50.068429 18410 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:50.070474 18410 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:50.071664 18403 consensus_meta.cc:287] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.939s	user 0.007s	sys 0.004s
W20260709 12:22:50.045251 18713 log.cc:927] Time spent T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.614s	user 0.002s	sys 0.003s
I20260709 12:22:50.010933 18718 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:50.021831 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:50.025547 18730 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:50.098170 16403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 789) took 1430 ms. Trace:
I20260709 12:22:50.098922 16403 rpcz_store.cc:276] 0709 12:22:48.667536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:48.788438 (+120902us) service_pool.cc:224] Handling call
0709 12:22:48.788784 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:48.789182 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:48.789209 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:48.789732 (+   523us) log.cc:844] Serialized 26 byte log entry
0709 12:22:48.939053 (+149321us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:48.939225 (+   172us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:48.939312 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:50.044179 (+1104867us) spinlock_profiling.cc:243] Waited 2.64 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:22:50.097724 (+ 53545us) raft_consensus.cc:1685] finished
0709 12:22:50.097757 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:50.098076 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":198,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":88,"spinlock_wait_cycles":6081408,"thread_start_us":147684,"threads_started":1,"wal-append.queue_time_us":572551}
I20260709 12:22:50.101943 18735 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:50.103433 18696 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:50.079720 18702 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:50.002669 18754 consensus_queue.cc:260] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:50.108560 16367 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:50.118103 17271 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:22:50.121322 16441 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:50.079499 18712 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:49.999275 18582 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:50.133772 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:50.135005 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:50.117218 18480 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:50.126050 18702 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:50.115679 18171 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:50.136708 18712 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:50.140658 18698 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:50.143919 18761 log.cc:927] Time spent T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
W20260709 12:22:50.145570 18528 consensus_meta.cc:287] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 1.386s	user 0.006s	sys 0.015s
I20260709 12:22:50.145917 18002 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:50.150990 17271 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:50.130836 18696 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:50.072242 18403 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:50.147936 18711 consensus_meta.cc:287] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.659s	user 0.002s	sys 0.008s
I20260709 12:22:50.127705 18735 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:50.177425 17261 raft_consensus.cc:1240] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:50.157140 18754 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:50.190567 16400 raft_consensus.cc:1240] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 6. Current term is 7. Ops: []
W20260709 12:22:50.182253 18290 consensus_meta.cc:287] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.845s	user 0.013s	sys 0.000s
I20260709 12:22:50.192603 17271 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:22:50.130369 18601 consensus_queue.cc:237] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:50.155529 18480 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:50.187685 18696 raft_consensus.cc:697] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:50.195858 18724 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:50.199049 18389 raft_consensus.cc:3055] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:50.184873 18711 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:50.201573 18689 consensus_meta.cc:287] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 1.213s	user 0.009s	sys 0.004s
I20260709 12:22:50.194288 18290 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:50.251549 16468 raft_consensus.cc:1275] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:22:50.254726 17266 raft_consensus.cc:1240] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:50.207208 18002 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:50.199545 18667 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:50.259537 18673 log.cc:927] Time spent T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 1.519s	user 0.007s	sys 0.021s
I20260709 12:22:50.260272 18403 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:50.244041 18389 raft_consensus.cc:740] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:50.261385 18667 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:50.257978 18696 consensus_queue.cc:237] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:50.265214 16394 raft_consensus.cc:2368] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 5. Current term is 6.
I20260709 12:22:50.246214 18171 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:50.267637 16398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 783) took 2450 ms. Trace:
I20260709 12:22:50.268046 16398 rpcz_store.cc:276] 0709 12:22:47.816695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:47.889885 (+ 73190us) service_pool.cc:224] Handling call
0709 12:22:47.890292 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:47.896431 (+  6139us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:47.896481 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:47.898918 (+  2437us) log.cc:844] Serialized 26 byte log entry
0709 12:22:48.497434 (+598516us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:48.497468 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:48.497526 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:50.259018 (+1761492us) spinlock_profiling.cc:243] Waited 3.23 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:22:50.263222 (+  4204us) raft_consensus.cc:1685] finished
0709 12:22:50.263238 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:50.266697 (+  3459us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2427,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":14789,"spinlock_wait_cycles":7438080,"thread_start_us":541269,"threads_started":1,"wal-append.queue_time_us":707338}
I20260709 12:22:50.271996 18728 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:50.303076 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:50.303869 16468 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:50.263073 18581 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:50.306066 16542 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:50.269604 18702 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:50.310537 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:50.249183 18689 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:50.314910 17269 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:22:50.304505 18637 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:50.316454 18582 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:50.316454 18712 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:50.318626 18728 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:50.319835 18013 consensus_meta.cc:287] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 1.558s	user 0.014s	sys 0.003s
I20260709 12:22:50.324678 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:50.322117 18711 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:50.325686 17263 raft_consensus.cc:2393] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 7: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:50.325956 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:50.307765 18389 consensus_queue.cc:260] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:50.328819 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:50.321169 18637 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:50.345701 16468 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
I20260709 12:22:50.354821 18611 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:50.357272 18689 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:50.324337 18013 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:50.359270 16474 raft_consensus.cc:1275] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:50.324107 18698 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:50.313274 18529 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:50.360231 18013 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:50.365839 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:50.365542 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:50.381628 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:50.369160 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:50.367164 17264 raft_consensus.cc:1240] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:50.379809 18690 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:22:50.376128 18769 log.cc:927] Time spent T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 12:22:50.393936 16471 raft_consensus.cc:2393] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 6: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:50.394765 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:50.366568 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:50.397744 18529 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:50.383945 18480 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:50.401175 18222 consensus_queue.cc:1048] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:50.401484 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:50.403175 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:50.403939 16468 raft_consensus.cc:1240] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:50.402891 16366 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:50.406550 17255 raft_consensus.cc:2393] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 6: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:50.408842 18389 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:50.417759 16367 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:50.419239 16477 raft_consensus.cc:1240] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 12:22:50.419553 16471 raft_consensus.cc:1275] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:22:50.421713 18582 consensus_queue.cc:1048] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:50.424697 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:50.425431 17254 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:50.425899 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:50.426430 16367 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:50.419436 18222 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:50.429673 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:50.429957 16367 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:22:50.430825 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:50.436187 16442 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:50.431569 18728 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:50.433652 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:50.434033 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:50.434435 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:50.435554 16468 raft_consensus.cc:1275] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:50.454712 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:50.455381 16477 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:22:50.436039 16472 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34826 (request call id 860) took 1666 ms (client timeout 1515 ms). Trace:
W20260709 12:22:50.456149 16472 rpcz_store.cc:269] 0709 12:22:48.769577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:48.770270 (+   693us) service_pool.cc:224] Handling call
0709 12:22:50.435725 (+1665455us) spinlock_profiling.cc:243] Waited 898 ms on lock 0x7b5400b2fcd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:50.435863 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2064273792}
I20260709 12:22:50.436779 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:50.457561 16470 raft_consensus.cc:2393] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 4: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:50.458801 16442 leader_election.cc:400] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6.
W20260709 12:22:50.437368 18733 log.cc:927] Time spent T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a: Append to log took a long time: real 0.353s	user 0.002s	sys 0.002s
I20260709 12:22:50.462536 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:50.441604 16475 raft_consensus.cc:1275] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:22:50.442317 17271 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:22:50.427580 18601 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:22:50.446898 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 399518us
I20260709 12:22:50.469274 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:50.451143 16514 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:50.459324 16442 leader_election.cc:403] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:22:50.452323 18222 raft_consensus.cc:697] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 7 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:50.470988 16442 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:50.462560 17254 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:22:50.471860 16441 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:50.479596 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:50.459223 18728 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:50.464849 18743 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:50.489332 17254 raft_consensus.cc:1275] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:50.479400 18611 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:22:50.495589 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:50.490983 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:50.549903 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:22:50.559602 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 78855.4us
I20260709 12:22:50.562868 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:50.564744 16549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 894) took 2069 ms. Trace:
I20260709 12:22:50.565295 16549 rpcz_store.cc:276] 0709 12:22:48.495406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:48.790820 (+295414us) service_pool.cc:224] Handling call
0709 12:22:48.791128 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:49.723684 (+932556us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:50.564449 (+840765us) spinlock_profiling.cc:243] Waited 930 ms on lock 0x7b5400a4f454. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:50.564668 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2138120960}
I20260709 12:22:50.555476 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:50.479555 18480 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:22:50.570570 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 78699.4us
I20260709 12:22:50.473901 18690 raft_consensus.cc:2749] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6.
I20260709 12:22:50.479588 18601 raft_consensus.cc:697] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:50.582427 16400 raft_consensus.cc:1275] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:50.607052 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:50.479485 18679 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:50.555076 18222 consensus_queue.cc:237] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:50.587246 18705 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:50.589419 17254 raft_consensus.cc:1275] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:22:50.555403 18581 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:22:50.580751 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:50.486349 18734 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:50.598496 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:50.600585 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:50.610294 16477 raft_consensus.cc:2468] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:22:50.612224 16403 raft_consensus.cc:1240] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:50.618184 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:50.481439 18675 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:50.693802 16398 raft_consensus.cc:2393] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 7: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:50.327530  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:50.649899 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:50.621640 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:22:50.646417 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 49846us
I20260709 12:22:50.697765 16548 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:22:50.698302 16550 raft_consensus.cc:2368] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6.
I20260709 12:22:50.644518 16394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46250 (request call id 795) took 1109 ms. Trace:
I20260709 12:22:50.699903 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:50.700661 16394 rpcz_store.cc:276] 0709 12:22:49.535377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:49.535544 (+   167us) service_pool.cc:224] Handling call
0709 12:22:50.644200 (+1108656us) spinlock_profiling.cc:243] Waited 726 ms on lock 0x7b540011f154. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:50.644409 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1670565120}
I20260709 12:22:50.559170 18667 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:50.704082 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:50.649318 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:50.500792 18728 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:50.718421 16543 raft_consensus.cc:1275] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:22:50.719489 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 62337us
I20260709 12:22:50.712677 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:50.720916 16398 raft_consensus.cc:2393] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 7: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:50.724287 16399 raft_consensus.cc:1240] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:50.742086 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:50.638273 18687 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:22:50.767146 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 60465.2us
I20260709 12:22:50.746825 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:50.747776 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:50.751035 16514 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:50.754392 16548 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
I20260709 12:22:50.645694 18719 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:22:50.766906 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:50.650343 18290 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:50.710516 18734 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:50.711028 18273 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:50.828121 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:50.823560 16442 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:50.833001 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 52223.2us
I20260709 12:22:50.822111 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:50.613744 18629 consensus_queue.cc:1048] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.159s
I20260709 12:22:50.651067 18661 consensus_queue.cc:1059] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.122s
I20260709 12:22:50.712751 18736 consensus_queue.cc:1048] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.104s
I20260709 12:22:50.832396 18719 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:50.858893 18736 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:50.859457 18736 raft_consensus.cc:697] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:50.859599 18719 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:50.845415 18389 consensus_queue.cc:1059] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 12:22:50.861059 18667 consensus_queue.cc:1059] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:50.623662 18725 consensus_queue.cc:1059] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.149s
I20260709 12:22:50.633690 18464 consensus_queue.cc:1048] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.139s
I20260709 12:22:50.746354 18540 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:50.863147 16550 raft_consensus.cc:1240] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:50.621621 18666 consensus_queue.cc:1059] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.151s
I20260709 12:22:50.865028 18666 raft_consensus.cc:3055] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:50.865393 18666 raft_consensus.cc:740] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:50.862881 18540 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:50.622143 18679 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:50.716008 18705 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:50.877902 16543 raft_consensus.cc:1275] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:50.870450 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:50.873107 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:50.877594 16367 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:50.830519 18734 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:50.889204 16546 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37242 (request call id 890) took 2408 ms (client timeout 1792 ms). Trace:
I20260709 12:22:50.644246 18601 consensus_queue.cc:237] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:50.719961 18675 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:50.891069 16546 rpcz_store.cc:269] 0709 12:22:48.468996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:48.471193 (+  2197us) service_pool.cc:224] Handling call
0709 12:22:50.877422 (+2406229us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b540082e6d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:50.877540 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2851007872}
I20260709 12:22:50.833686 18690 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:50.902422 16544 raft_consensus.cc:1275] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:50.904301 16367 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:50.905599 17264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 652) took 1321 ms. Trace:
I20260709 12:22:50.871255 18702 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:50.910588 16469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 379) took 1180 ms. Trace:
W20260709 12:22:50.914124 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:50.908942 17264 rpcz_store.cc:276] 0709 12:22:49.584610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:49.589010 (+  4400us) service_pool.cc:224] Handling call
0709 12:22:49.589306 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:50.392614 (+803308us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:50.905356 (+512742us) spinlock_profiling.cc:243] Waited 778 ms on lock 0x7b5400aee054. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:50.905540 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1788322176}
I20260709 12:22:50.914271 16469 rpcz_store.cc:276] 0709 12:22:49.730355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:49.738446 (+  8091us) service_pool.cc:224] Handling call
0709 12:22:49.738839 (+   393us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:50.028943 (+290104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:50.910257 (+881314us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b5400977f54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:50.910521 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":664604032}
I20260709 12:22:50.883109 18687 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:50.918362 17258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 699) took 1099 ms. Trace:
I20260709 12:22:50.842301 18273 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:50.918788 17258 rpcz_store.cc:276] 0709 12:22:49.818570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:49.818927 (+   357us) service_pool.cc:224] Handling call
0709 12:22:49.819231 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:49.895947 (+ 76716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:50.918119 (+1022172us) spinlock_profiling.cc:243] Waited 76 ms on lock 0x7b54005f7b54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:50.918303 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":174896512}
I20260709 12:22:50.912262 18689 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:50.937027 17261 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 650) took 1353 ms. Trace:
I20260709 12:22:50.937491 17261 rpcz_store.cc:276] 0709 12:22:49.583905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:49.588457 (+  4552us) service_pool.cc:224] Handling call
0709 12:22:49.588767 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:50.183774 (+595007us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:50.936776 (+753002us) spinlock_profiling.cc:243] Waited 588 ms on lock 0x7b540005f0d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:50.936966 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1353279232}
I20260709 12:22:50.916841 18735 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:22:50.945768 18778 log.cc:927] Time spent T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
W20260709 12:22:50.946440 18781 log.cc:927] Time spent T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
I20260709 12:22:50.949661 18705 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:50.951050 18679 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:50.951256 18711 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:50.914299 18690 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:50.974880 18690 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:50.979281 18711 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:51.002249 17270 raft_consensus.cc:1240] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:50.909205 18719 consensus_queue.cc:1048] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 12:22:50.954448 17143 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:50.943967 16367 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:50.934902 18702 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:51.002951 17270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 657) took 1183 ms. Trace:
I20260709 12:22:50.952320 18735 raft_consensus.cc:697] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 5 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:50.897087 18736 consensus_queue.cc:237] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:50.922982 18687 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:51.006755 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 62855.6us
I20260709 12:22:50.889425 18666 consensus_queue.cc:260] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:51.007977 17270 rpcz_store.cc:276] 0709 12:22:49.819175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:49.821913 (+  2738us) service_pool.cc:224] Handling call
0709 12:22:51.001817 (+1179904us) spinlock_profiling.cc:243] Waited 4.21 ms on lock 0x7b58003b8298. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:22:51.002049 (+   232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:51.002725 (+   676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:51.002899 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9679104}
W20260709 12:22:50.997216 18637 consensus_meta.cc:287] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.611s	user 0.012s	sys 0.001s
W20260709 12:22:51.009522 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 55182.1us
I20260709 12:22:50.982544 18711 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:50.951575 18728 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:51.037266 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:22:51.011701 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 260784us
I20260709 12:22:51.037077 18540 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:51.010217 18679 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:51.030766 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:51.039366 18728 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:51.011368 18528 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:51.045522 16514 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:51.049201 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 268722us
I20260709 12:22:51.056808 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:51.020922 18725 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:51.032778 18290 consensus_queue.cc:1059] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 12:22:51.067339 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:51.048573 18540 consensus_queue.cc:1059] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 12:22:51.008697 18702 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:51.089708 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:51.041177 18637 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:51.094336 18690 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:51.102061 18273 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:51.105603 18689 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:51.106307 18689 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:51.107513 18676 raft_consensus.cc:3055] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:51.108042 18676 raft_consensus.cc:740] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:51.109073 18676 consensus_queue.cc:260] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:51.110056 18687 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:51.110538 18676 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:51.112815 18711 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:51.112607 18735 consensus_queue.cc:237] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:51.113394 18711 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:51.115607 18711 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:51.117506 16544 raft_consensus.cc:1275] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:51.117964 18389 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:51.118769 18637 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:51.118595 18389 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:51.065191 18725 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:51.123651 18725 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:51.058887 18439 consensus_queue.cc:1059] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 12:22:51.125061 18439 raft_consensus.cc:3055] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:51.125439 18439 raft_consensus.cc:740] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:51.126240 18439 consensus_queue.cc:260] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:51.126768 18740 raft_consensus.cc:3055] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:51.127342 18740 raft_consensus.cc:740] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:51.128751 18702 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:51.057296 18749 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:51.101943 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:51.096508 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:51.092901 16441 leader_election.cc:400] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6.
W20260709 12:22:51.149000 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 67124us
I20260709 12:22:51.148828 16547 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
I20260709 12:22:51.152683 16441 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:51.085456 18675 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:51.172024 16441 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:51.141222 18439 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:51.191614 18734 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:51.194937 18734 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:51.074067 18528 consensus_queue.cc:1048] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.091s
I20260709 12:22:51.157655 18290 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:51.213893 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 380375us
I20260709 12:22:51.214927 16442 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:51.169998 18637 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:51.230182 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:51.231993 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:51.234999 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:51.213634 18689 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:51.200889 18702 consensus_queue.cc:1048] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 12:22:51.213176 18705 consensus_queue.cc:1048] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 12:22:51.217038 18690 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:51.254844 18629 consensus_queue.cc:1059] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:51.257110 16442 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:51.133101 18740 consensus_queue.cc:260] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:51.269363 16442 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:51.213649 18696 consensus_queue.cc:1048] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 12:22:51.243340 18581 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:51.251652 18529 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:51.230962 18290 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:51.266426 18637 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:51.302906 18581 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:51.351006 18013 raft_consensus.cc:3055] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:51.351310 18581 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:51.351876 18735 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:51.352586 18735 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:51.355005 18735 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:51.298015 18629 consensus_queue.cc:1243] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer 52ab50b0237346c1a13ede947f6a63c2 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:22:51.348476 16366 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:22:51.376456 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 332582us
I20260709 12:22:51.377547 17142 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:51.337571 18222 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:51.293473 18690 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:51.389696 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 335497us
W20260709 12:22:51.390311 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 343162us
I20260709 12:22:51.304790 18637 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:51.373008 18581 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:51.373816 18724 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:51.402913 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:51.380702 18740 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:51.362512 18013 raft_consensus.cc:740] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:51.399151 17270 raft_consensus.cc:1275] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:22:51.407732 17263 raft_consensus.cc:1275] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:51.376636 18791 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:51.414044 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:22:51.419068 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 263769us
I20260709 12:22:51.419421 17270 pending_rounds.cc:85] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8: Aborting all ops after (but not including) 5
I20260709 12:22:51.419869 17270 pending_rounds.cc:107] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:22:51.420217 17270 raft_consensus.cc:2889] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:51.389125 18222 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:51.410162 18679 raft_consensus.cc:697] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:51.430289 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:51.430639 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:51.440398 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:51.408226 18735 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:22:51.443476 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:51.447701 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:51.452108 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:51.452996 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:51.451550 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:51.451273 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:51.459623 17270 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
I20260709 12:22:51.458400 17263 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:51.464905 17266 raft_consensus.cc:2393] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 6: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:51.413421 18661 consensus_queue.cc:1048] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:22:51.472986 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:51.450055 18721 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:51.481244 17266 raft_consensus.cc:1240] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:22:51.475435 16441 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:51.417501 18724 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:51.478404 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 209135us
I20260709 12:22:51.420701 18789 consensus_queue.cc:1048] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 12:22:51.523859 17270 raft_consensus.cc:1275] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:51.520022 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:51.521144 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:51.526261 16473 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:51.428162 18791 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:51.552845 16475 raft_consensus.cc:1275] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:22:51.480026 18735 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:51.536563 16367 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:51.530437 18702 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:51.447161 18013 consensus_queue.cc:260] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:51.601747 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 204363us
I20260709 12:22:51.606933 18735 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:51.539345 18601 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:51.611129 18013 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:51.655515 16475 raft_consensus.cc:1240] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 2. Current term is 3. Ops: []
W20260709 12:22:51.612787 18790 log.cc:927] Time spent T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.252s	user 0.000s	sys 0.001s
I20260709 12:22:51.605036 18702 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:51.557446 18696 consensus_queue.cc:1048] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.069s
I20260709 12:22:51.680254 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:22:51.680934 18796 log.cc:927] Time spent T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.207s	user 0.000s	sys 0.000s
W20260709 12:22:51.680254 18794 log.cc:927] Time spent T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.258s	user 0.000s	sys 0.000s
I20260709 12:22:51.713040 16470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 937) took 1255 ms. Trace:
W20260709 12:22:51.632153 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 210194us
I20260709 12:22:51.742656 16470 rpcz_store.cc:276] 0709 12:22:50.457497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:50.458163 (+   666us) service_pool.cc:224] Handling call
0709 12:22:50.478123 (+ 19960us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:50.720797 (+242674us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:50.720824 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:50.740860 (+ 20036us) log.cc:844] Serialized 26 byte log entry
0709 12:22:51.059234 (+318374us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:51.059276 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:51.129295 (+ 70019us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b540058c854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:51.129340 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:51.712759 (+583419us) raft_consensus.cc:1685] finished
0709 12:22:51.712778 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:51.712978 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64728,"raft.queue_time_us":10629,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":15497,"spinlock_wait_cycles":533820160,"thread_start_us":213211,"threads_started":1,"wal-append.queue_time_us":467809}
W20260709 12:22:51.749909 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:51.753166 16544 consensus_meta.cc:287] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.516s	user 0.013s	sys 0.000s
I20260709 12:22:51.753768 16544 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
W20260709 12:22:51.690171 18676 consensus_meta.cc:287] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.579s	user 0.009s	sys 0.005s
W20260709 12:22:51.677551 18805 log.cc:927] Time spent T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
W20260709 12:22:51.669749 18792 log.cc:927] Time spent T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.279s	user 0.001s	sys 0.000s
I20260709 12:22:51.681522 18687 raft_consensus.cc:3055] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:51.627339 18290 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:51.629374 18581 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:51.713790 18702 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:51.677517 18724 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:51.629065 18601 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:51.757920 18637 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:51.681962 18800 consensus_queue.cc:1048] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.095s
I20260709 12:22:51.765556 18690 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:51.686707 18789 consensus_queue.cc:1059] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.100s
I20260709 12:22:51.767071 18812 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:51.770089 18728 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:51.828191 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:51.832370 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:51.777465 18798 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:51.783520 18687 raft_consensus.cc:740] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:51.838660 16543 raft_consensus.cc:2468] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:51.846467 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:51.660967 18679 consensus_queue.cc:237] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:51.817939 18581 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:51.856576 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 244523us
W20260709 12:22:51.862066 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 325593us
I20260709 12:22:51.857467 17142 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:51.788879 18222 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:51.842530 18814 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:51.863657 16367 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:51.898097 16545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 906) took 1336 ms. Trace:
I20260709 12:22:51.898633 16545 rpcz_store.cc:276] 0709 12:22:50.562043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:50.564018 (+  1975us) service_pool.cc:224] Handling call
0709 12:22:50.564424 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:50.745449 (+181025us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:50.745487 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:50.745529 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:50.745565 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:51.897841 (+1152276us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b54008c46d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:51.897892 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:51.898025 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":415796608}
I20260709 12:22:51.848086 18728 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:51.896243 16367 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:51.843243 18812 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:51.906788 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:51.874553 18696 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:51.887157 18734 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:51.919260 16367 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:22:51.920866 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 63440.2us
I20260709 12:22:51.931512 16545 raft_consensus.cc:1275] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:51.893668 18797 consensus_queue.cc:1048] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:22:51.915723 18820 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:51.942842 16367 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:51.939162 17142 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:51.948845 16516 leader_election.cc:400] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 5. Current term is 6.
I20260709 12:22:51.910605 18687 consensus_queue.cc:260] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:51.970930 18811 log.cc:927] Time spent T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 12:22:51.900024 18814 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:51.919185 18829 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:51.973809 16546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 394) took 1032 ms. Trace:
I20260709 12:22:51.978447 16546 rpcz_store.cc:276] 0709 12:22:50.941188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:50.941336 (+   148us) service_pool.cc:224] Handling call
0709 12:22:50.945456 (+  4120us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:50.957484 (+ 12028us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:50.957515 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:50.957562 (+    47us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:50.957594 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:51.973526 (+1015932us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b54005e0954. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:51.973592 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:51.973742 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27187072}
I20260709 12:22:51.929116 18540 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:22:51.984637 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 45596.8us
I20260709 12:22:51.946262 18818 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:51.986322 17142 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
W20260709 12:22:51.941339 18802 log.cc:927] Time spent T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.251s	user 0.000s	sys 0.001s
I20260709 12:22:51.989558 16395 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:22:51.931787 18791 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:51.945154 18810 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:51.990447 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:51.994575 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:51.995442 16401 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:22:51.978039 18829 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:51.939183 18834 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:51.996529 18834 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:51.996924 18834 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:51.997071 18545 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:51.997179 18830 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:51.997599 18844 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:51.997766 18545 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:51.999321 18601 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:51.999907 18545 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:52.000985 18762 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:51.919705 18696 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:51.996196 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:52.002534 17142 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:52.003449 16400 raft_consensus.cc:2393] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 8: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:52.011906 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:52.012228 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:52.012480 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:51.996170 18829 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:51.972694 18832 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:52.013041 16403 raft_consensus.cc:2393] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 7: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:52.015442 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:52.053511 16473 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:52.018196 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:52.055047 16401 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:52.012991 16395 raft_consensus.cc:2393] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 4: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:52.047631 16442 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:51.999913 18601 raft_consensus.cc:2804] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:51.938673 18837 consensus_queue.cc:1048] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.075s
I20260709 12:22:51.990000 18839 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:22:52.013638 18808 log.cc:927] Time spent T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
I20260709 12:22:52.001507 18762 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:52.063231 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:22:52.019080 18815 log.cc:927] Time spent T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 12:22:52.079010 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:52.079994 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:52.060252 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:52.096813 16470 raft_consensus.cc:2410] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:22:52.046037 18403 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:52.084839 16472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 880) took 1030 ms. Trace:
I20260709 12:22:52.089346 16442 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:51.991406 18540 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:52.108043 16401 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
W20260709 12:22:52.090847 18803 log.cc:927] Time spent T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.244s	user 0.000s	sys 0.001s
W20260709 12:22:52.148921 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 59668.5us
I20260709 12:22:52.089782 18529 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:51.975184 18820 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:52.076071 18845 raft_consensus.cc:2804] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:52.081310 18789 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:52.157997 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:52.084295 18840 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:52.095625 18791 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:52.081786 18705 consensus_queue.cc:1059] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 12:22:52.095479 18801 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:52.095381 18545 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:52.151185 16515 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:52.099849 16472 rpcz_store.cc:276] 0709 12:22:51.054074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:51.054505 (+   431us) service_pool.cc:224] Handling call
0709 12:22:51.065746 (+ 11241us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:51.066215 (+   469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:51.066261 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:51.072385 (+  6124us) log.cc:844] Serialized 26 byte log entry
0709 12:22:51.474529 (+402144us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:51.776816 (+302287us) log.cc:844] Serialized 14 byte log entry
0709 12:22:52.083794 (+306978us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:52.084084 (+   290us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:52.084234 (+   150us) raft_consensus.cc:1685] finished
0709 12:22:52.084305 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:52.084750 (+   445us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":384258,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":3593,"raft.queue_time_us":131298,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":19709,"spinlock_wait_cycles":14384000,"thread_start_us":811129,"threads_started":3,"wal-append.queue_time_us":675885}
I20260709 12:22:52.158010 18831 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:52.154424 18828 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:52.155603 18836 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:52.230595 18838 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:52.238829 18838 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:52.230530 18831 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:52.241781 18439 consensus_meta.cc:287] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 1.037s	user 0.010s	sys 0.004s
I20260709 12:22:52.216961 18840 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:52.243494 18831 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:52.091231 18839 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:52.219095 18728 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:52.155043 18823 consensus_queue.cc:1048] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 12:22:52.248060 18823 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:52.148398 18403 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:52.218874 18808 log.cc:927] Time spent T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:22:52.251216 18403 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:52.253480 18849 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:52.252869 18838 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:52.254325 18849 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:52.254999 18828 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:52.192631 18791 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:52.256759 18791 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:52.258317 18828 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:52.261018 18812 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:52.267182 18762 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:52.270638 18637 raft_consensus.cc:3055] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:52.271284 18637 raft_consensus.cc:740] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:52.228227 18827 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:52.273262 18827 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:52.216662 18845 raft_consensus.cc:697] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:52.277933 18820 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:52.277550 18845 consensus_queue.cc:237] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:52.227963 18545 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:52.233989 18843 log.cc:927] Time spent T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 12:22:52.068871 18832 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:52.235071 18853 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:52.194290 18789 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:52.237330 18698 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:52.171864 18581 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:52.235322 18833 log.cc:927] Time spent T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:22:52.179298 18797 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:52.315802 17271 raft_consensus.cc:1240] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:52.317339 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:52.323310 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:52.328989 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:52.329775 17266 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:52.310521 18666 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:52.313040 18803 log.cc:927] Time spent T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 12:22:52.331941 16474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 878) took 1279 ms. Trace:
I20260709 12:22:52.332561 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:52.332758 18849 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:52.333158 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:52.336006 18389 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:52.336970 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:52.337977 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:22:52.339735 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 389557us
I20260709 12:22:52.332273 16474 rpcz_store.cc:276] 0709 12:22:51.053112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:51.054714 (+  1602us) service_pool.cc:224] Handling call
0709 12:22:51.055181 (+   467us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:51.055666 (+   485us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:51.055707 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:51.069869 (+ 14162us) log.cc:844] Serialized 26 byte log entry
0709 12:22:51.383386 (+313517us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:51.585612 (+202226us) log.cc:844] Serialized 14 byte log entry
0709 12:22:51.881007 (+295395us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:51.881110 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:51.971897 (+ 90787us) spinlock_profiling.cc:243] Waited 3.32 ms on lock 0x7b18002df820. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c522 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:22:52.331676 (+359779us) raft_consensus.cc:1685] finished
0709 12:22:52.331773 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:52.331887 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62608,"prepare.queue_time_us":434848,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":65,"raft.queue_time_us":240011,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":11559,"spinlock_wait_cycles":7643904,"thread_start_us":704042,"threads_started":3,"wal-append.queue_time_us":661045}
I20260709 12:22:52.311774 18601 raft_consensus.cc:697] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 7 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:52.343940 16550 raft_consensus.cc:1275] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:52.345559 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:52.346840 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:52.347702 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:52.350454 16544 raft_consensus.cc:1275] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:52.351421 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:52.352548 17271 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:52.221164 18801 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:52.412624 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:52.328263 18828 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:22:52.426152 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 81232.3us
I20260709 12:22:52.426517 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:52.343873 18842 log.cc:927] Time spent T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.130s	user 0.001s	sys 0.001s
I20260709 12:22:52.450372 16551 raft_consensus.cc:2410] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:22:52.451413 16546 raft_consensus.cc:1275] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:22:52.336633 18464 raft_consensus.cc:3055] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:52.455166 16545 raft_consensus.cc:1275] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:52.459090 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:52.461817 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:52.457173 17143 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:52.318457 18698 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:52.458710 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:52.467581 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:52.468739 18545 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:52.469957 18540 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:52.470167 18789 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:52.470880 18801 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:52.471323 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:52.291330 18637 consensus_queue.cc:260] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:52.461242 18464 raft_consensus.cc:740] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:52.477000 18832 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:52.311856 18853 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:52.477766 18834 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:52.320883 18797 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:52.480561 18829 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:52.481925 18839 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:22:52.483237 18848 log.cc:927] Time spent T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
W20260709 12:22:52.486479 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 106123us
I20260709 12:22:52.492022 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:52.526841 17142 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:52.493270 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:52.346566 18581 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:52.515414 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 58360.7us
I20260709 12:22:52.529362 17143 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:52.463692 18740 consensus_meta.cc:287] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 1.050s	user 0.015s	sys 0.001s
W20260709 12:22:52.521960 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 370861us
I20260709 12:22:52.523617 17264 raft_consensus.cc:1275] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:52.527901 16398 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
W20260709 12:22:52.478684 18854 log.cc:927] Time spent T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:22:52.508415 18403 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:52.533931 17142 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:52.466688 18828 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:52.516790 18818 consensus_queue.cc:1048] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 12:22:52.538834 16399 raft_consensus.cc:2393] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 6: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:52.540871 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:52.546001 17263 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:22:52.521653 18545 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:52.538347 17144 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:52.531258 18013 consensus_meta.cc:287] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.863s	user 0.011s	sys 0.006s
I20260709 12:22:52.531608 18637 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:52.563491 16477 raft_consensus.cc:1275] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:52.564657 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:52.557052 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:52.558272 18860 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:52.561321 18853 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:52.548717 18403 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:52.562121 18814 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:52.562865 16476 raft_consensus.cc:1275] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:52.537478 18809 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:52.565838 16471 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:22:52.570354 16515 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:52.522326 18728 consensus_queue.cc:1048] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:22:52.578569 18865 consensus_queue.cc:1048] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:52.580976 18865 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:52.581655 18865 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:52.561350 18795 log.cc:927] Time spent T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 1.149s	user 0.007s	sys 0.000s
I20260709 12:22:52.525228 18856 consensus_queue.cc:1048] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 12:22:52.549136 18839 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:52.604318 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:52.616462 16476 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:52.617106 18855 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:52.617913 18403 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:52.605350 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:52.619258 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:52.620503 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:52.621650 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:52.622856 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:52.624173 18816 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:52.624377 18829 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:52.624851 18816 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:52.565011 18749 consensus_queue.cc:1059] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 12:22:52.628587 18385 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:52.628685 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:52.629299 18385 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:52.631556 18385 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:52.572319 18740 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:52.632755 18740 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:52.633097 18740 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:52.634864 18702 consensus_meta.cc:287] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.774s	user 0.014s	sys 0.006s
I20260709 12:22:52.552076 18849 consensus_queue.cc:1059] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 12:22:52.637609 18828 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:52.638825 18816 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:52.640079 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:52.640645 18839 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:52.641934 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:52.642778 18839 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:52.642961 16399 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:52.643363 18824 raft_consensus.cc:3055] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:52.643863 18824 raft_consensus.cc:740] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:52.552692 18540 consensus_queue.cc:1048] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 12:22:52.644493 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:52.646654 18834 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:52.646968 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:52.574481 18809 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:52.647588 18809 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:52.538230 18581 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:52.649261 18698 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:52.595862 18762 consensus_queue.cc:1059] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 12:22:52.565483 18789 consensus_queue.cc:1059] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.058s
I20260709 12:22:52.553527 18601 consensus_queue.cc:237] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:52.650918 18060 raft_consensus.cc:2804] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:22:52.651280 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 80988.1us
I20260709 12:22:52.654120 18749 raft_consensus.cc:3055] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:52.654663 18749 raft_consensus.cc:740] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:52.655475 18749 consensus_queue.cc:260] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:52.654160 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:52.651326 18060 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:52.671516 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:52.673337 18060 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:52.674206 18865 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:52.674346 18581 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:52.677623 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:52.679736 16549 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:22:52.635324 18702 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:52.685799 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:52.669945 18403 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:52.707847 18403 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:52.666724 18611 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:52.687425 18791 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:52.687777 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:52.682418 18834 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:52.686777 18749 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:52.706638 16546 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
I20260709 12:22:52.714180 16542 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:22:52.719630 16366 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:52.723199 16366 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:52.724320 16468 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
I20260709 12:22:52.726014 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:52.585270 18464 consensus_queue.cc:260] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:52.730564 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:52.711568 18818 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:52.721232 18629 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:52.734169 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 118171us
I20260709 12:22:52.733314 16366 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:52.735998 17271 raft_consensus.cc:1275] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:22:52.737392 18868 log.cc:927] Time spent T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.067s	user 0.000s	sys 0.003s
I20260709 12:22:52.739053 18857 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:52.739696 18857 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:52.737361 18867 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:22:52.741704 18403 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:52.736635 18818 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:52.731981 18829 consensus_queue.cc:1048] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 12:22:52.718474 18611 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:52.742107 18857 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:52.727753 18801 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:52.746595 18611 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:52.746866 18801 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:52.749442 18801 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:52.725293 18834 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:52.744593 16469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 879) took 1691 ms. Trace:
I20260709 12:22:52.729606 18828 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:52.752943 18828 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:52.753425 18828 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:52.753607 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:52.754503 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:52.749815 16441 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:52.755465 18824 consensus_queue.cc:260] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:52.744570 18791 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:52.757074 18791 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:52.757419 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:52.751881 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:52.757093 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:52.759130 17255 raft_consensus.cc:1275] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:52.759541 16395 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:52.751858 18834 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:52.756428 16441 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:52.743386 16396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 805) took 1376 ms. Trace:
I20260709 12:22:52.763334 16366 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:52.699522 18060 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:52.752624 16469 rpcz_store.cc:276] 0709 12:22:51.053681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:51.053822 (+   141us) service_pool.cc:224] Handling call
0709 12:22:51.066610 (+ 12788us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:51.130308 (+ 63698us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:51.130342 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:51.139650 (+  9308us) log.cc:844] Serialized 26 byte log entry
0709 12:22:51.385448 (+245798us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:51.606625 (+221177us) log.cc:844] Serialized 14 byte log entry
0709 12:22:52.659988 (+1053363us) spinlock_profiling.cc:243] Waited 3.23 ms on lock 0x7b180008d160. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2875a314bc 00007f2872d7f48b 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d99455 00007f2875e81060 00007f2875e93716 00007f2875ea4287
0709 12:22:52.660173 (+   185us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:52.744288 (+ 84115us) spinlock_profiling.cc:243] Waited 63.4 ms on lock 0x7b54002e63d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:52.744364 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:52.744404 (+    40us) raft_consensus.cc:1685] finished
0709 12:22:52.744418 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:52.744536 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73304,"prepare.queue_time_us":442229,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":3112,"raft.queue_time_us":235570,"raft.run_wall_time_us":7585,"spinlock_wait_cycles":159113600,"thread_start_us":640653,"threads_started":3,"wal-append.queue_time_us":582427}
I20260709 12:22:52.767247 16441 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:52.763303 16396 rpcz_store.cc:276] 0709 12:22:51.366573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:51.366707 (+   134us) service_pool.cc:224] Handling call
0709 12:22:51.384935 (+ 18228us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:51.469828 (+ 84893us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:51.469923 (+    95us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:51.469966 (+    43us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:51.470012 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:52.743175 (+1273163us) spinlock_profiling.cc:243] Waited 84.4 ms on lock 0x7b54004ec5d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:52.743218 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:52.743331 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":194057984}
I20260709 12:22:52.771136 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:52.772781 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:52.771508 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:52.770531 16441 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:52.776944 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:52.777813 17261 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:22:52.766458 18601 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:52.777012 18728 consensus_queue.cc:1048] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 12:22:52.757524 18791 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:52.780737 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:52.780637 16366 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:52.780061 18867 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:52.781706 18832 consensus_queue.cc:1048] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:22:52.784309 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:52.769575 18611 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:52.771798 18581 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:52.744732 18818 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:52.791620 16441 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:52.798202 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:52.799790 16366 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:52.803038 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:52.804658 16516 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:52.803643 16396 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:52.780191 18529 raft_consensus.cc:2804] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:52.745014 18629 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:52.808772 16441 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:52.809593 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:52.816972 17261 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:22:52.817415 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:52.786808 18791 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:52.807492 18762 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:52.815696 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:52.774451 18666 consensus_queue.cc:1048] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 12:22:52.806875 16516 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:52.794871 18867 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:52.809269 18582 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:52.876449 16476 raft_consensus.cc:1240] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:52.798236 18728 consensus_queue.cc:1048] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.064s
W20260709 12:22:52.865473 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 58715.2us
I20260709 12:22:52.880267 16475 raft_consensus.cc:1240] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:52.868849 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:52.846354 16471 raft_consensus.cc:1240] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:52.840240 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:22:52.863497 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 54838.1us
I20260709 12:22:52.872404 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:52.868780 16398 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:22:52.873775 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:52.809626 18824 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:52.873481 18867 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:52.813283 18690 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:52.889165 18793 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:52.889523 18690 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:52.890027 18793 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:52.890690 18060 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:52.813550 18529 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:52.893885 18529 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:52.899385 18823 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:52.902678 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:52.905154 17255 raft_consensus.cc:2393] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 7: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:52.839246 18273 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:52.909124 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:52.910099 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:52.919126 16471 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:22:52.919256 16477 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
W20260709 12:22:52.910401 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:52.899971 18823 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:52.920475 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:52.813786 18581 raft_consensus.cc:697] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:52.926174 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:52.937403 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:52.937934 17264 raft_consensus.cc:1240] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:52.939771 16546 raft_consensus.cc:1240] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:52.875236 18849 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:52.876354 18825 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:52.942322 16469 raft_consensus.cc:1275] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:52.944162 16366 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:52.998029 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 53944.1us
I20260709 12:22:52.944725 16441 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:52.890008 18690 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:52.945745 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:52.887934 18839 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:52.949878 18823 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:52.953015 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:52.889233 18867 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:52.953996 18702 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:52.956571 17254 raft_consensus.cc:1240] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:22:52.957666 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:52.959033 16516 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:52.959257 18529 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:52.880704 18582 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:52.892230 18793 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:52.965453 18611 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:52.881188 18847 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:52.922798 18480 consensus_queue.cc:1048] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 12:22:52.984344 18740 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:52.903579 18728 consensus_queue.cc:1243] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer 52ab50b0237346c1a13ede947f6a63c2 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:22:52.998872 16366 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:52.936548 18273 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:53.000048 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 55419.9us
I20260709 12:22:53.031603 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:53.000921 18601 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:53.011490 16395 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6.
W20260709 12:22:53.016602 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 57672.3us
I20260709 12:22:53.040110 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:53.040300 16516 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:53.041174 16395 raft_consensus.cc:2393] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 7: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:53.018877 17143 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:53.002740 18825 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:53.018931 18582 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:53.006050 18814 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:52.944284 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:53.034199 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 135217us
I20260709 12:22:53.020450 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:53.022746 18832 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:53.007656 18839 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:53.029783 18849 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:53.061146 17266 raft_consensus.cc:2368] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8.
I20260709 12:22:53.058138 16546 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:22:53.058720 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:53.067910 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:53.074541 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:53.025453 18823 consensus_queue.cc:1059] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 12:22:53.035167 18674 raft_consensus.cc:2804] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:53.045156 18818 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:53.074895 16544 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:22:53.034973 18601 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:53.061987 18666 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:53.026270 18540 consensus_queue.cc:1059] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 12:22:53.065716 16514 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:53.050107 18360 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:53.051707 18825 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:53.060410 18581 consensus_queue.cc:237] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:53.096958 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:53.097447 18529 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:53.099897 18839 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:53.101028 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:53.101449 18791 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:53.103616 18858 raft_consensus.cc:3055] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:53.104076 18858 raft_consensus.cc:740] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:53.074642 18788 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.104740 18809 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:53.105130 18788 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:53.105871 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:53.107357 17255 raft_consensus.cc:1240] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:53.107848 18809 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:53.108595 18540 consensus_queue.cc:1059] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:53.083369 18666 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:53.109417 18666 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:53.109807 18540 raft_consensus.cc:3055] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:53.110191 18540 raft_consensus.cc:740] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:53.111016 18540 consensus_queue.cc:260] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:53.073014 18849 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:53.112851 18849 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:53.086018 18877 consensus_queue.cc:1048] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 12:22:53.080241 18674 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:53.115293 18674 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:53.116022 16401 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:22:53.117764 16401 raft_consensus.cc:1240] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:53.033082 18847 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:53.081856 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:53.058118 18793 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.135625 18793 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:53.057672 18814 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:53.139642 18814 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:53.139780 18791 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:53.140424 18273 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:53.083886 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:53.082894 18601 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:53.096060 18823 consensus_queue.cc:1059] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 12:22:53.083132 18818 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:53.124162 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:53.155462 17269 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:53.075443 16395 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:22:53.079685 16441 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:53.157516 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:53.131248 16401 raft_consensus.cc:1240] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 7 LEADER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:53.071890 18832 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:53.162200 18832 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:53.142313 16548 raft_consensus.cc:2410] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:22:53.134541 18540 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:53.151973 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:53.164366 18818 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:53.154546 18793 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:22:53.167538 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 87918.5us
I20260709 12:22:53.169267 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:53.175513 16441 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:53.179606 16546 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:22:53.188849 16401 raft_consensus.cc:1240] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:53.190184 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:53.194380 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:53.195655 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:53.205359 17271 raft_consensus.cc:2393] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 6: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:53.172876 18809 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.178593 18832 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:53.180133 18816 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.178783 18791 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:53.208036 16366 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:53.214664 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:22:53.248301 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:53.184104 18880 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:53.231071 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:53.257032 18847 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:22:53.259819 18842 log.cc:927] Time spent T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:22:53.263769 16403 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:22:53.265483 16549 raft_consensus.cc:2468] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:53.207544 18788 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:53.211937 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:53.212803 18832 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:53.274297 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:53.219543 18480 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:53.229818 18629 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:53.285622 16515 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:53.293329 16403 raft_consensus.cc:1240] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:53.262503 18880 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:53.297984 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:53.269296 18787 raft_consensus.cc:3055] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:22:53.288890 18749 consensus_meta.cc:287] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.549s	user 0.012s	sys 0.000s
I20260709 12:22:53.303478 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:53.302901 17261 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:22:53.275841 18879 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:53.214409 18809 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:53.306756 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:53.214955 18791 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:53.311347 16468 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:53.312786 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:53.321550 17261 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:22:53.316215 17264 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:53.301421 18847 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:22:53.297673 18669 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.120s	user 0.014s	sys 0.010s
W20260709 12:22:53.336491 18669 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.163s	user 0.014s	sys 0.011s
I20260709 12:22:53.215257 18816 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:53.303289 18880 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:53.339025 18816 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:53.290661 18740 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:53.271760 18611 consensus_queue.cc:1059] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 12:22:53.341061 18858 consensus_queue.cc:260] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:53.290024 18480 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:53.342519 18480 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:53.343173 18809 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:53.289955 18832 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:53.350693 16516 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:53.321205 16515 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:53.352771 16516 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:53.352876 18667 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:53.332623 18825 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:53.355738 16515 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:53.306660 18787 raft_consensus.cc:740] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:53.303475 18690 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:53.373880 18690 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:53.374683 18582 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:53.376188 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:53.376730 18791 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:53.377483 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:53.354149 18847 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:53.377130 18582 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:53.364785 18858 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:53.382483 16477 raft_consensus.cc:1240] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:22:53.361536 18749 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:53.365753 18879 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:53.391476 16515 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:53.384040 18847 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:53.400300 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:53.365108 18822 consensus_queue.cc:1059] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 12:22:53.361428 18674 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:53.363735 18787 consensus_queue.cc:260] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:53.364670 18540 consensus_queue.cc:1059] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:22:53.389067 18749 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:53.390971 18879 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:53.438230 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 46879us
I20260709 12:22:53.438141 16468 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:22:53.393983 18629 raft_consensus.cc:2804] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:53.385353 18667 consensus_queue.cc:1059] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
W20260709 12:22:53.444871 18886 log.cc:927] Time spent T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:22:53.443135 18629 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:53.448756 18629 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:53.453107 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 116578us
I20260709 12:22:53.456037 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:53.456630 16472 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:22:53.452569 18822 consensus_queue.cc:1243] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Peer f99066d13e524238aec6aa2024a8bd50 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:22:53.454595 17142 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:53.457896 16399 raft_consensus.cc:1240] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:53.391144 18849 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:53.458590 16471 raft_consensus.cc:1240] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:53.441809 18669 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:53.441241 18749 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:53.442816 18879 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:53.399524 18867 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:53.465219 17142 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:53.462700 18740 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:53.465618 18669 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:53.500643 18669 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:53.470692 18831 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:53.501505 18831 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:53.502050 18831 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:53.483649 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:53.484375 18721 consensus_queue.cc:1059] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 12:22:53.485886 18814 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:53.486212 18480 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:53.488085 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:53.490072 18849 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:53.494083 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:53.466013 18273 raft_consensus.cc:3055] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:53.531728 16473 raft_consensus.cc:2410] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:22:53.539072 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:53.541792 18480 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:53.541834 18629 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:53.543953 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:53.544220 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:53.544900 16403 raft_consensus.cc:2376] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8. Re-sending same reply.
I20260709 12:22:53.546584 18674 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:53.533752 18787 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:53.532135 18273 raft_consensus.cc:740] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:53.535843 18740 raft_consensus.cc:697] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:22:53.565246 18828 consensus_meta.cc:287] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.809s	user 0.012s	sys 0.002s
I20260709 12:22:53.539094 18666 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:53.573254 18812 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.005s	sys 0.015s
I20260709 12:22:53.561903 18816 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.589569 18816 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:53.587759 18273 consensus_queue.cc:260] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:53.606200 18629 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:53.584668 18828 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:53.621536 18666 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:53.622116 18273 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:53.637204 18816 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:53.639067 18847 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:53.590631 18740 consensus_queue.cc:237] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:53.631676 17263 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45224 (request call id 735) took 2151 ms (client timeout 2137 ms). Trace:
I20260709 12:22:53.632180 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:22:53.640995 18902 log.cc:927] Time spent T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:22:53.641332 16403 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:22:53.633199 18828 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:22:53.633440 18895 log.cc:927] Time spent T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
W20260709 12:22:53.640677 17263 rpcz_store.cc:269] 0709 12:22:51.472561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:51.472715 (+   154us) service_pool.cc:224] Handling call
0709 12:22:53.613366 (+2140651us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400260f54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:53.624021 (+ 10655us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2466014976}
I20260709 12:22:53.647202 18880 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:53.650815 18879 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:53.651620 18831 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:53.585495 18825 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:53.652688 16543 raft_consensus.cc:2368] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 8. Current term is 9.
I20260709 12:22:53.654129 18832 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:53.661284 18827 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:53.661168 18735 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:53.663200 18832 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:53.664252 18735 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.664796 18735 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:53.667457 18735 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:53.656947 18879 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:53.657475 18601 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:53.640820 18814 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:53.669631 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:53.671124 18528 raft_consensus.cc:3055] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:53.671659 18528 raft_consensus.cc:740] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:53.673972 18797 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:53.677336 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:53.677654 18880 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:53.677974 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:53.678015 16403 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:53.678330 16396 raft_consensus.cc:1240] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:22:53.679296 18669 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:53.682461 18669 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:53.682226 18528 consensus_queue.cc:260] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:53.658187 18867 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:53.684414 16442 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:53.694512 16442 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:53.695598 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:53.696084 16442 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:53.696367 16401 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:53.697470 16442 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:53.685482 18528 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:53.588260 18812 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.791s	user 0.006s	sys 0.015s
I20260709 12:22:53.703008 16442 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:53.705600 16403 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:53.706816 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:53.707480 16403 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:53.707850 16442 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:53.718143 16442 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:53.720669 16442 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:53.722134 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:53.722153 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:53.722009 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:53.734447 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 181448us
I20260709 12:22:53.734917 16442 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:53.736243 16396 raft_consensus.cc:1240] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:53.731832 16547 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:53.736243 16442 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:53.737531 16546 raft_consensus.cc:1275] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:22:53.640601 18749 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:53.701614 18825 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:53.743626 16401 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:22:53.740365 16403 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:22:53.701838 18827 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:53.701423 18832 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:53.722327 18837 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:53.722172 18813 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:53.746233 16403 raft_consensus.cc:1240] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:53.669663 18847 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:53.722597 18702 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:53.738971 16546 pending_rounds.cc:85] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1: Aborting all ops after (but not including) 3
I20260709 12:22:53.749893 16546 pending_rounds.cc:107] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:22:53.750214 16546 raft_consensus.cc:2889] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:53.731909 18734 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:53.734934 18719 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:53.748749 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:53.741325 18637 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:53.751443 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:53.751808 16477 raft_consensus.cc:1275] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:53.754184 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:53.755645 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:53.756284 16470 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:53.743136 18581 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:53.759241 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:53.759851 17266 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:22:53.748492 18840 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:53.765084 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:53.771176 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:53.749634 18813 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:53.773656 17255 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:53.778136 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:53.774631 18637 raft_consensus.cc:697] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:53.774899 18812 raft_consensus.cc:3055] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:53.779096 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:53.776173 17258 raft_consensus.cc:1275] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:53.776201 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:53.752779 18787 consensus_queue.cc:1059] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:22:53.818506 18904 consensus_queue.cc:1048] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:53.819027 18787 raft_consensus.cc:3055] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:53.819422 18787 raft_consensus.cc:740] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:53.798154 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:53.820246 18787 consensus_queue.cc:260] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:53.820886 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:53.821319 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:53.821871 16477 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:53.825096 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:53.825917 17261 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 9: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:53.826685 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:53.827190 17261 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:53.828994 18637 consensus_queue.cc:237] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:53.804775 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:53.810148 16546 raft_consensus.cc:2393] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 5: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:53.831903 16547 raft_consensus.cc:2393] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 5: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:53.832381 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:53.832834 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:53.833010 16546 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:22:53.833738 16547 raft_consensus.cc:2410] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:22:53.833892 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:53.834136 17266 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:22:53.834666 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:53.835026 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:53.835178 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:53.835479 16547 raft_consensus.cc:2393] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 7: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:53.779232 18813 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:53.772781 18734 raft_consensus.cc:697] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:53.793952 18812 raft_consensus.cc:740] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:53.837774 18822 raft_consensus.cc:3055] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:53.837606 18734 consensus_queue.cc:237] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:53.838290 18822 raft_consensus.cc:740] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:53.837863 18812 consensus_queue.cc:260] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:53.839005 18812 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:53.839176 18822 consensus_queue.cc:260] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:53.840713 18822 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:53.841111 16515 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:53.841581 18787 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:53.798322 18798 consensus_queue.cc:1059] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 12:22:53.805900 18831 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:53.814931 18749 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:53.760167 18581 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:53.851776 16477 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:22:53.817745 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:53.821463 17264 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:53.843240 17255 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:22:53.857666 17261 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:22:53.814733 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:53.857952 16515 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:53.841953 18661 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:53.859905 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:53.843648 17266 raft_consensus.cc:2393] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 7: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:53.861609 16515 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:53.848863 18581 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:53.867976 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:53.868825 17255 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:53.869935 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:53.872980 17261 raft_consensus.cc:1240] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 5. Current term is 6. Ops: [4.3-5.4]
I20260709 12:22:53.870817 18066 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:53.878422 18066 raft_consensus.cc:697] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:53.878710 18840 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.879410 18840 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:53.881256 17261 raft_consensus.cc:1240] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:53.883800 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:53.885174 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:53.888001 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:53.889789 17271 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 9: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:53.891852 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:53.895098 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:53.896806 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:53.897537 16551 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:53.898494 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:53.899107 16551 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:53.901196 17142 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:53.902029 18813 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:53.902504 18528 raft_consensus.cc:2804] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:53.902609 17142 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:53.903059 18528 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:53.903554 18528 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:53.903709 18873 log.cc:927] Time spent T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.856s	user 0.005s	sys 0.000s
I20260709 12:22:53.904577 18910 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:53.905118 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:53.905421 18910 raft_consensus.cc:697] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:53.905824 17266 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
I20260709 12:22:53.906445 18910 consensus_queue.cc:237] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:53.907769 17270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 673) took 1156 ms. Trace:
I20260709 12:22:53.907851 18385 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.911144 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:53.871202 18661 raft_consensus.cc:697] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:53.911885 18813 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:53.913312 18528 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:53.913610 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:53.865724 18749 raft_consensus.cc:2804] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:53.942773 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:53.915766 16516 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:53.927953 16514 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:53.958029 16516 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:53.936555 17270 rpcz_store.cc:276] 0709 12:22:52.750929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:52.751061 (+   132us) service_pool.cc:224] Handling call
0709 12:22:52.751420 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:52.751817 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:52.751851 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:52.752360 (+   509us) log.cc:844] Serialized 26 byte log entry
0709 12:22:52.844154 (+ 91794us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:52.844196 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:52.844269 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:53.891939 (+1047670us) spinlock_profiling.cc:243] Waited 4.76 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:22:53.907441 (+ 15502us) raft_consensus.cc:1685] finished
0709 12:22:53.907474 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:53.907706 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1215,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":96,"spinlock_wait_cycles":10958720,"thread_start_us":91370,"threads_started":1,"wal-append.queue_time_us":236011}
I20260709 12:22:53.937642 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:53.938319 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:53.940649 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:53.952370 17264 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:22:53.964185 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:53.964941 16469 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:53.941418 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:53.965940 16474 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:53.968210 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:53.968711 16515 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:53.972641 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:53.973503 16544 raft_consensus.cc:1275] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:53.879679 18066 consensus_queue.cc:237] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:53.974126 16542 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52680 (request call id 391) took 3153 ms (client timeout 1811 ms). Trace:
I20260709 12:22:53.978189 16472 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:22:53.976909 16514 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:53.977418 17143 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:53.935897 18696 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:53.982766 16549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37242 (request call id 930) took 1263 ms. Trace:
I20260709 12:22:53.985984 16549 rpcz_store.cc:276] 0709 12:22:52.709682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:52.710349 (+   667us) service_pool.cc:224] Handling call
0709 12:22:53.972965 (+1262616us) spinlock_profiling.cc:243] Waited 539 ms on lock 0x7b540037f354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:53.973088 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1238728064}
W20260709 12:22:53.978547 16542 rpcz_store.cc:269] 0709 12:22:50.821077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:50.827868 (+  6791us) service_pool.cc:224] Handling call
0709 12:22:53.973940 (+3146072us) spinlock_profiling.cc:243] Waited 1.81 s on lock 0x7b5400366354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:53.974050 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4173312896}
W20260709 12:22:53.983067 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:53.965091 18675 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:53.967748 18540 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:53.989611 17143 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:53.967120 18845 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:53.991552 17264 raft_consensus.cc:1275] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:53.994453 17143 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:53.998416 17143 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:53.938951 18702 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:53.995051 16514 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:53.928109 18879 consensus_queue.cc:1059] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 12:22:54.009089 16551 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:53.936841 18385 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:54.001434 17143 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:54.002051 18389 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:54.004101 18013 raft_consensus.cc:2764] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:22:53.974934 18669 consensus_queue.cc:1048] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 12:22:53.993131 18821 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:53.969555 18749 raft_consensus.cc:697] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:53.989005 18861 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:53.990325 18273 raft_consensus.cc:2749] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:53.939749 18661 consensus_queue.cc:237] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:54.011471 17143 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:53.991153 16516 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:53.990550 18667 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:54.009110 16469 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:22:54.014698 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:54.027034 16476 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:54.008503 18675 raft_consensus.cc:697] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:54.008623 18740 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:54.040527 18711 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:54.005985 18845 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:54.042870 18845 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:54.014441 18820 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:54.032179 18581 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:54.043470 18385 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:54.014601 16477 raft_consensus.cc:1240] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:54.043546 18696 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:54.044770 18696 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:54.045157 18581 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:54.016517 18540 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:54.041070 18711 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:54.046437 18840 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:54.046715 18711 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:54.047792 18060 consensus_queue.cc:1059] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 12:22:54.030107 18824 raft_consensus.cc:3055] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:54.049503 18861 raft_consensus.cc:697] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:54.037940 18749 consensus_queue.cc:237] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:54.050699 18736 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:54.051414 18736 raft_consensus.cc:697] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:54.050980 18861 consensus_queue.cc:237] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:54.049798 18824 raft_consensus.cc:740] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:54.056708 16472 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:22:54.029224 17143 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:54.064378 16476 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:22:54.065609 16551 raft_consensus.cc:1240] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:54.070533 16548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52680 (request call id 414) took 1100 ms. Trace:
I20260709 12:22:54.071012 16548 rpcz_store.cc:276] 0709 12:22:52.970270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:53.081654 (+111384us) service_pool.cc:224] Handling call
0709 12:22:54.070258 (+988604us) spinlock_profiling.cc:243] Waited 6.48 ms on lock 0x7b54008c4bd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:54.070453 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14902656}
I20260709 12:22:54.078482 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:54.080585 16547 raft_consensus.cc:1240] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:54.082767 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:54.083442 16477 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:22:54.082281 17142 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:22:54.087280 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:54.047797 18696 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:54.080229 18845 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:54.123584 17143 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:54.161850 16470 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:22:54.158957 16441 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:54.164165 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 133385us
I20260709 12:22:54.165441 16441 leader_election.cc:400] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8.
I20260709 12:22:54.165740 16441 leader_election.cc:403] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:22:54.119263 18911 raft_consensus.cc:2804] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:22:54.232982 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 67674.3us
I20260709 12:22:54.118584 18389 consensus_queue.cc:1059] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.113s
I20260709 12:22:54.235689 16477 raft_consensus.cc:1240] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:22:54.236831 16441 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:54.086210 18711 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:54.077313 18736 consensus_queue.cc:237] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:54.234525 18696 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8.
I20260709 12:22:54.242491 16477 pending_rounds.cc:85] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 2
I20260709 12:22:54.253906 16477 pending_rounds.cc:107] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:22:54.254263 16477 raft_consensus.cc:2889] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:54.254328 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:54.255224 16474 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:54.130515 18812 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:54.256670 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:54.257745 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:54.262274 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:54.267827 16468 raft_consensus.cc:2393] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 8: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:54.239184 18675 consensus_queue.cc:237] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:54.234436 18734 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:54.250847 18637 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:54.280002 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:54.283452 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:54.289301 16476 raft_consensus.cc:1240] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:54.281075 18667 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:54.283516 18820 raft_consensus.cc:2764] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:22:54.294302 18667 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:54.295411 18711 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:22:54.296804 18916 log.cc:927] Time spent T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.216s	user 0.000s	sys 0.001s
I20260709 12:22:54.297076 18667 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:54.299376 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:54.300645 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:54.301074 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:54.270146 18812 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:54.283346 18637 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:54.306087 18637 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:54.299427 18918 log.cc:927] Time spent T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:22:54.302486 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:54.313370 16474 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:22:54.315704 16474 raft_consensus.cc:1240] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 LEADER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:22:54.316975 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:54.317888 16474 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 9: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:54.319459 16441 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:54.321264 16441 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:54.323467 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:54.333374 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:54.328009 18734 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:54.327786 18813 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:54.336316 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:54.332625 16441 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:54.364992 16441 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:54.368937 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 245417us
I20260709 12:22:54.373368 17143 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:54.380537 16441 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:54.395067 16441 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:54.394248 17143 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:54.384790 18734 raft_consensus.cc:2804] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:54.388371 18696 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:54.406581 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:54.389148 18825 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:54.393045 18910 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:54.390239 18813 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:54.402583 18734 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:54.411549 18910 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:54.440013 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:54.429508 18711 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:54.429920 18696 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:22:54.439515 18883 log.cc:927] Time spent T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:22:54.432415 18734 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:54.437706 18910 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:54.425930 18813 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:54.446697 18676 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:54.437518 18911 raft_consensus.cc:697] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:54.471714 18601 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:54.473961 17254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 687) took 1446 ms. Trace:
W20260709 12:22:54.475128 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 80191.2us
I20260709 12:22:54.451435 18926 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:54.461311 18661 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:54.474611 17254 rpcz_store.cc:276] 0709 12:22:53.027918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:53.030042 (+  2124us) service_pool.cc:224] Handling call
0709 12:22:53.053040 (+ 22998us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:53.053583 (+   543us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:53.053627 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:54.309155 (+1255528us) spinlock_profiling.cc:243] Waited 1.88 ms on lock 0x7b2c00000708. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:22:54.309915 (+   760us) log.cc:844] Serialized 52 byte log entry
0709 12:22:54.425786 (+115871us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:22:54.425829 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:54.425928 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:54.473688 (+ 47760us) raft_consensus.cc:1685] finished
0709 12:22:54.473707 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:54.473900 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2079,"raft.queue_time_us":117,"raft.run_cpu_time_us":315,"raft.run_wall_time_us":7517,"spinlock_wait_cycles":4460672,"thread_start_us":115350,"threads_started":1,"wal-append.queue_time_us":157871}
I20260709 12:22:54.478755 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:54.479514 16549 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:22:54.487630 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:54.488485 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:54.489924 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:54.491567 17143 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:54.492789 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:54.492766 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:54.495239 18806 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:22:54.495390 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 581168us
I20260709 12:22:54.495541 17255 raft_consensus.cc:1240] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:54.495924 18806 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:54.503392 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:54.508492 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:54.514730 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:54.476598 18676 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:54.502825 18698 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:54.525198 18698 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:54.510083 18789 consensus_queue.cc:1059] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 12:22:54.527383 18698 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:54.527676 18806 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:54.527804 18789 raft_consensus.cc:3055] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:54.528288 18789 raft_consensus.cc:740] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:54.485265 18661 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:54.530066 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:54.510551 18290 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:54.531783 16366 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:54.509856 18582 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:54.529654 16367 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:54.532050 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:54.510289 18749 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:54.538087 16401 raft_consensus.cc:2393] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 8: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:54.483866 18926 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:54.541756 16366 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:54.508601 18222 consensus_queue.cc:1048] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 12:22:54.542994 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:54.546977 16400 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
W20260709 12:22:54.533582 18924 log.cc:927] Time spent T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 12:22:54.525467 18464 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:54.549180 16402 raft_consensus.cc:1240] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:54.553841 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:54.554648 16398 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 9: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:54.554497 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:54.554752 18728 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:54.555737 18676 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:54.550570 18674 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:54.556947 18800 raft_consensus.cc:3055] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:54.557200 18674 raft_consensus.cc:697] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 5 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:54.558682 18674 consensus_queue.cc:237] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:54.564575 18829 consensus_queue.cc:1059] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:54.546588 18582 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:54.570106 18661 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:54.546769 18601 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:54.571512 18813 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:54.578186 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:54.579535 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:54.580160 17261 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:22:54.544894 18290 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:54.581260 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:54.582491 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:54.532534 18403 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:54.582844 16515 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:54.584244 18847 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:54.584693 18847 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:54.583683 18403 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:54.585279 16402 raft_consensus.cc:1240] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:54.551329 18749 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:54.557542 18800 raft_consensus.cc:740] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:54.589151 18800 consensus_queue.cc:260] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:54.589874 18749 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:54.590806 18800 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:54.592235 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:54.585016 18847 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:54.582823 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:54.583184 16398 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:54.529520 18935 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:54.603969 16476 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:22:54.608880 17271 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
I20260709 12:22:54.500831 18911 consensus_queue.cc:237] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:54.618738 16395 raft_consensus.cc:1240] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:54.614745 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:22:54.619302 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 77655.1us
I20260709 12:22:54.620086 16400 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 9: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:54.621230 16366 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:54.583613 18789 consensus_queue.cc:260] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:54.630704 16366 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:54.633159 16516 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:54.624117 18661 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:22:54.640044 18930 log.cc:927] Time spent T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 12:22:54.640715 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:54.641749 17142 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:54.648079 16472 raft_consensus.cc:1275] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:22:54.649873 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:54.650905 18813 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:54.651654 18925 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:54.652204 18925 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:54.652608 18925 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:54.656057 18809 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:54.657073 18809 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:54.660053 18813 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:54.609724 18637 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:54.663079 18813 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:54.663980 16471 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
I20260709 12:22:54.666553 16550 raft_consensus.cc:1275] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:54.667493 18894 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:54.668170 18894 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:54.641939 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:54.670885 18926 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:54.678223 16367 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:54.686818 16472 raft_consensus.cc:1275] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:22:54.717845 16395 raft_consensus.cc:3055] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:54.722118 16395 raft_consensus.cc:740] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:54.717237 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:54.700551 17142 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:54.629978 18601 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:22:54.743691 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 51388.3us
W20260709 12:22:54.747493 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 47057.3us
I20260709 12:22:54.699591 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:54.765938 16472 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:22:54.610203 18935 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:54.635569 18728 raft_consensus.cc:2804] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:54.789264 16545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1110) took 1085 ms. Trace:
I20260709 12:22:54.790352 16545 rpcz_store.cc:276] 0709 12:22:53.703620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:53.968062 (+264442us) service_pool.cc:224] Handling call
0709 12:22:53.968300 (+   238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:54.002578 (+ 34278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:54.002608 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:54.002651 (+    43us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:54.002683 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:54.788971 (+786288us) spinlock_profiling.cc:243] Waited 34.1 ms on lock 0x7b5400bb5654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:54.789033 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:54.789196 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":78379904}
I20260709 12:22:54.791961 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:54.792788 17142 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:54.792675 18385 consensus_queue.cc:1048] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:54.793655 18740 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:54.794068 18740 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:54.794456 18740 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:22:54.796444 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 103964us
I20260709 12:22:54.796521 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:54.797652 16367 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:54.687175 18676 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:22:54.796000 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 46590.8us
I20260709 12:22:54.666473 18637 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:54.635262 18943 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:54.721455 18667 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:54.799753 16395 consensus_queue.cc:260] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:54.815044 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:54.785267 18926 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:54.694895 18749 consensus_queue.cc:1059] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.115s
I20260709 12:22:54.796870 18927 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:54.816613 16395 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:54.801760 18676 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:54.688862 18661 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:54.720705 18698 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:54.726047 18800 consensus_queue.cc:1048] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.088s
I20260709 12:22:54.836608 18290 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:54.720315 18925 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:54.846812 18728 raft_consensus.cc:697] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:54.847512 18829 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:54.785306 18910 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:54.844072 18676 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:54.839399 18674 consensus_queue.cc:1048] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 12:22:54.873677 16545 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:54.881912 17261 raft_consensus.cc:1275] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:22:54.843070 18222 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:54.896962 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:54.932770 18290 raft_consensus.cc:3055] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:54.933162 18940 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:22:54.934900 18939 log.cc:927] Time spent T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:22:54.935659 18948 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:54.936474 18947 raft_consensus.cc:3055] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:54.815557 18943 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:54.921988 16395 raft_consensus.cc:1275] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 12:22:54.980067 16395 pending_rounds.cc:85] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 3
I20260709 12:22:54.945387 18661 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:54.948107 16470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34826 (request call id 898) took 1011 ms. Trace:
I20260709 12:22:54.950606 18728 consensus_queue.cc:237] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:54.957711 18935 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:54.964660 18910 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:54.867082 18810 consensus_queue.cc:1059] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.109s
I20260709 12:22:54.886659 18698 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:22:54.978220 18734 consensus_meta.cc:287] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.514s	user 0.009s	sys 0.000s
I20260709 12:22:54.826428 18667 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:54.980592 16395 pending_rounds.cc:107] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:22:55.041441 16395 raft_consensus.cc:2889] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:54.994804 16470 rpcz_store.cc:276] 0709 12:22:53.937223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:53.937361 (+   138us) service_pool.cc:224] Handling call
0709 12:22:54.947930 (+1010569us) spinlock_profiling.cc:243] Waited 84.6 ms on lock 0x7b540012b954. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:54.948047 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":194582784}
W20260709 12:22:54.995980 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 193338us
W20260709 12:22:55.015973 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:54.967430 18740 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:54.838467 18926 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:54.888650 18809 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:54.969774 16551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 421) took 1152 ms. Trace:
I20260709 12:22:55.050565 16551 rpcz_store.cc:276] 0709 12:22:53.817219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:54.012187 (+194968us) service_pool.cc:224] Handling call
0709 12:22:54.012536 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:54.066023 (+ 53487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:54.969469 (+903446us) spinlock_profiling.cc:243] Waited 52.9 ms on lock 0x7b54008c5ad4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:54.969698 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":121582336}
I20260709 12:22:55.051713 16400 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
I20260709 12:22:54.975273 18290 raft_consensus.cc:740] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
W20260709 12:22:55.059996 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:55.060595 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:55.060743 16399 raft_consensus.cc:1275] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:22:55.065456 16543 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37242 (request call id 938) took 1763 ms (client timeout 1914 ms). Trace:
W20260709 12:22:55.069089 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 271503us
I20260709 12:22:54.980505 18222 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:55.078759 16551 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:54.983819 18947 raft_consensus.cc:740] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:22:55.078733 16543 rpcz_store.cc:269] 0709 12:22:53.301719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:53.388901 (+ 87182us) service_pool.cc:224] Handling call
0709 12:22:55.065240 (+1676339us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b5400b13354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:55.065388 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":574880128}
I20260709 12:22:55.097081 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:22:55.105010 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:55.105758 16543 raft_consensus.cc:1240] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:54.999118 18948 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:55.106463 16543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 941) took 1016 ms. Trace:
I20260709 12:22:55.106775 16543 rpcz_store.cc:276] 0709 12:22:54.089735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:55.105247 (+1015512us) service_pool.cc:224] Handling call
0709 12:22:55.105589 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:55.106183 (+   594us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:55.106405 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:55.108026 18687 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:55.110193 18921 log.cc:927] Time spent T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.977s	user 0.003s	sys 0.003s
I20260709 12:22:55.111809 16544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 416) took 2097 ms. Trace:
I20260709 12:22:55.112345 16544 rpcz_store.cc:276] 0709 12:22:53.014020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:53.226910 (+212890us) service_pool.cc:224] Handling call
0709 12:22:53.973177 (+746267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:53.978131 (+  4954us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:55.111653 (+1133522us) spinlock_profiling.cc:243] Waited 746 ms on lock 0x7b540037f350. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:55.111760 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1715280128}
W20260709 12:22:55.114516 18847 consensus_meta.cc:287] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.503s	user 0.011s	sys 0.000s
I20260709 12:22:55.104218 16551 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:22:55.122879 18847 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:55.124750 18403 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:55.125166 16397 raft_consensus.cc:2410] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:22:55.125948 18834 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.126439 18834 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:55.125252 16394 raft_consensus.cc:1275] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:22:55.056977 18734 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:55.134325 16367 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:55.136073 16367 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:55.075592 18949 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.138193 16546 raft_consensus.cc:1240] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:22:55.138445 16476 raft_consensus.cc:1240] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:55.140525 16367 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:55.140515 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:55.148687 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:55.145268 16472 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:55.145291 18403 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:55.151940 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:55.145560 18687 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:22:55.153613 17144 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 2.437s (SENT) [suppressed 5 similar messages]
I20260709 12:22:55.128222 18740 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:55.156189 16475 raft_consensus.cc:1240] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:55.158115 16367 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:55.165310 16548 raft_consensus.cc:1240] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:55.165798 16476 raft_consensus.cc:1275] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:22:55.169260 16367 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:55.115192 18947 consensus_queue.cc:260] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:55.165122 18687 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:55.178730 16367 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:55.194345 18687 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:55.186517 18403 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:55.121416 18676 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:55.184106 16477 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:22:55.173336 18962 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:55.186902 16549 raft_consensus.cc:2393] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 7: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:55.148119 18949 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:55.195746 16367 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:55.198489 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:55.205740 16476 raft_consensus.cc:2393] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 4: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:55.205674 16367 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:55.198974 18403 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:55.207588 16367 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:55.211584 16367 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:55.218046 16472 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:22:55.224382 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:55.225823 16551 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:55.230731 17264 raft_consensus.cc:1275] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:22:55.231076 17269 raft_consensus.cc:1275] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:55.201947 18948 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:55.211941 18403 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:55.211024 18687 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:55.247238 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:55.249975 18834 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:55.240286 18667 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:55.244122 18926 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:55.235561 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:55.239110 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:55.234992 18953 consensus_queue.cc:1059] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:22:55.240252 16403 raft_consensus.cc:1240] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 5 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:55.240418 18698 consensus_queue.cc:1059] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:55.243487 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:55.244505 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:55.244686 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:55.244840 17142 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:55.210740 18822 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:22:55.236159 18954 log.cc:927] Time spent T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 12:22:55.275712 18910 raft_consensus.cc:3055] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:55.282037 18910 raft_consensus.cc:740] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:55.259290 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:55.283051 17254 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
I20260709 12:22:55.187062 18958 consensus_queue.cc:1048] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 12:22:55.279546 18879 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.259519 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:55.291965 18822 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:55.252148 18403 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:55.292234 18879 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:55.294719 18822 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:55.224907 18940 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:55.283282 18910 consensus_queue.cc:260] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:55.264061 18824 consensus_queue.cc:260] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:55.296715 18910 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:55.297508 18824 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:55.298061 16471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34840 (request call id 973) took 1010 ms. Trace:
I20260709 12:22:55.284955 18582 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:55.298461 16471 rpcz_store.cc:276] 0709 12:22:54.287921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:54.288086 (+   165us) service_pool.cc:224] Handling call
0709 12:22:55.297855 (+1009769us) spinlock_profiling.cc:243] Waited 363 ms on lock 0x7b5400594554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:55.297971 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":833911680}
I20260709 12:22:55.295558 18879 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:55.300534 16548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 423) took 1323 ms. Trace:
I20260709 12:22:55.301223 16548 rpcz_store.cc:276] 0709 12:22:53.976775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:54.076393 (+ 99618us) service_pool.cc:224] Handling call
0709 12:22:54.076804 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:55.165826 (+1089022us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:55.300149 (+134323us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54005e0954. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:55.300455 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2502799744}
I20260709 12:22:55.258389 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:55.302965 18798 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:55.287403 16547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1121) took 1507 ms. Trace:
I20260709 12:22:55.321034 16547 rpcz_store.cc:276] 0709 12:22:53.779844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:54.008527 (+228683us) service_pool.cc:224] Handling call
0709 12:22:54.017422 (+  8895us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:54.082870 (+ 65448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:55.286975 (+1204105us) spinlock_profiling.cc:243] Waited 63 ms on lock 0x7b540069ffd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:55.287197 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":144851200}
I20260709 12:22:55.259665 16403 raft_consensus.cc:1275] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:55.316123 16395 raft_consensus.cc:1275] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:22:55.316691 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:55.323830 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:55.265767 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:55.325212 16403 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:55.283773 18676 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:55.329429 17143 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:55.330737 18953 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:55.159536 18290 consensus_queue.cc:260] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:55.337328 18290 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:55.264531 16396 raft_consensus.cc:1275] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:22:55.303731 18798 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:55.342306 18798 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:55.260744 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:55.343922 16399 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:55.315946 17261 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:55.346204 18943 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:55.218436 18955 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:55.349674 18847 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:55.228993 18674 consensus_queue.cc:1048] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.121s
I20260709 12:22:55.350783 16546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1108) took 1647 ms. Trace:
W20260709 12:22:55.351207 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:55.259680 18734 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:55.195351 18961 consensus_queue.cc:1059] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 12:22:55.288486 18949 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:55.363032 18961 raft_consensus.cc:3055] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:55.363600 18961 raft_consensus.cc:740] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:55.364816 18961 consensus_queue.cc:260] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:55.365943 18222 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:55.367174 16403 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
W20260709 12:22:55.368536 16397 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58430 (request call id 548) took 3048 ms (client timeout 2437 ms). Trace:
W20260709 12:22:55.369472 16397 rpcz_store.cc:269] 0709 12:22:52.320489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:52.320598 (+   109us) service_pool.cc:224] Handling call
0709 12:22:55.368209 (+3047611us) spinlock_profiling.cc:243] Waited 2.79 s on lock 0x7b540015f254. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:55.368414 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6418743168}
I20260709 12:22:55.282123 18698 raft_consensus.cc:3055] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:55.280067 18966 consensus_queue.cc:1048] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
W20260709 12:22:55.318553 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 73774.6us
I20260709 12:22:55.319455 18800 raft_consensus.cc:3055] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:55.377717 18800 raft_consensus.cc:740] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:55.320194 18967 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:55.301838 16550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1101) took 2955 ms. Trace:
I20260709 12:22:55.387032 16550 rpcz_store.cc:276] 0709 12:22:52.346289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:52.346863 (+   574us) service_pool.cc:224] Handling call
0709 12:22:52.347135 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:54.667228 (+2320093us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:55.301480 (+634252us) spinlock_profiling.cc:243] Waited 2.32 s on lock 0x7b540037f854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:55.301747 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5333176704}
I20260709 12:22:55.323997 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:55.289710 16441 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:55.273919 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:55.389024 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 99409.8us
I20260709 12:22:55.390118 16441 leader_election.cc:400] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 8. Current term is 9.
I20260709 12:22:55.390530 16441 leader_election.cc:403] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:22:55.336354 18935 consensus_queue.cc:1059] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:55.335646 16367 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:55.393533 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 57974.5us
W20260709 12:22:55.267293 18965 log.cc:927] Time spent T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:22:55.357760 16394 raft_consensus.cc:1275] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:22:55.351276 16546 rpcz_store.cc:276] 0709 12:22:53.703047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:53.835837 (+132790us) service_pool.cc:224] Handling call
0709 12:22:54.070409 (+234572us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:55.144655 (+1074246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:55.349649 (+204994us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54008c4bd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:55.350550 (+   901us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b54008c4bd0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:55.350711 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2980389504}
I20260709 12:22:55.229261 18818 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:55.348091 18947 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:55.379019 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 225418us
I20260709 12:22:55.294677 18403 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:55.410738 16516 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:55.396394 16545 raft_consensus.cc:2393] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 6: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:55.380759 16551 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:22:55.376880 17142 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:55.404208 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:55.414368 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:55.415064 16469 raft_consensus.cc:2393] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 6: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:55.415762 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:55.374095 18698 raft_consensus.cc:740] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:55.420254 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:55.420847 17266 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:22:55.410820 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:55.394995 16367 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:55.416399 17271 raft_consensus.cc:2468] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:55.417083 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:55.376096 18861 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:55.376866 18949 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:55.417937 17143 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:55.427685 17255 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:22:55.429404 18810 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:55.400209 18825 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:55.398375 18222 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:55.396023 18967 raft_consensus.cc:697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:55.511647 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:55.398303 18961 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:55.424309 18879 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:55.452646 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 57739us
I20260709 12:22:55.398186 18818 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:55.434139 18711 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:55.424435 18809 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:55.400322 18581 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 8. Current term is 9.
I20260709 12:22:55.482089 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:55.485265 17142 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:55.496887 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:55.499516 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:55.500108 18829 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:55.378723 18800 consensus_queue.cc:260] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:55.387907 18734 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:55.525836 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:55.527228 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:55.528522 16515 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:55.454617 16545 raft_consensus.cc:1275] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:22:55.537101 16367 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:55.560817 16545 pending_rounds.cc:85] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1: Aborting all ops after (but not including) 4
I20260709 12:22:55.561214 16545 pending_rounds.cc:107] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:22:55.453763 18958 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:55.561992 16545 raft_consensus.cc:2889] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:55.498051 18389 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:55.500404 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:55.419157 18831 consensus_queue.cc:1059] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.108s
I20260709 12:22:55.552557 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:55.511094 18222 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:55.561180 16515 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:22:55.555507 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 70367.5us
I20260709 12:22:55.542941 17266 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
W20260709 12:22:55.593755 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 325668us
I20260709 12:22:55.598404 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:55.600646 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:55.464344 18060 consensus_queue.cc:1048] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.084s
I20260709 12:22:55.537742 18818 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:55.608724 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:55.622238 16548 raft_consensus.cc:2368] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 5. Current term is 6.
I20260709 12:22:55.623382 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:55.611214 16367 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:55.626856 16367 leader_election.cc:400] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 5. Current term is 6.
I20260709 12:22:55.627249 16367 leader_election.cc:403] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:22:55.612099 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:55.613876 18689 consensus_queue.cc:1048] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:22:55.552114 18806 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.943s	user 0.010s	sys 0.025s
I20260709 12:22:55.498390 18787 consensus_queue.cc:1048] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 12:22:55.499660 18861 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:55.508724 18810 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:55.429909 18801 consensus_queue.cc:1048] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:55.518257 18969 consensus_queue.cc:1048] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.099s
I20260709 12:22:55.541121 18403 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:55.466938 18949 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:55.634341 16469 raft_consensus.cc:2393] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 7: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
W20260709 12:22:55.660004 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:55.660547 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:55.661630 16367 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:55.629410 18935 consensus_queue.cc:1243] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer f99066d13e524238aec6aa2024a8bd50 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:22:55.552783 18829 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:55.543576 18959 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:22:55.669476 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:55.618762 18222 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:55.540846 18879 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:55.564901 18809 raft_consensus.cc:697] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:55.462885 16550 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:22:55.709372 16550 raft_consensus.cc:606] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
W20260709 12:22:55.635538 18806 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.029s	user 0.010s	sys 0.025s
W20260709 12:22:55.635646 18960 log.cc:927] Time spent T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:22:55.630170 18798 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:55.750236 16476 raft_consensus.cc:1240] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 5. Current term is 6. Ops: []
W20260709 12:22:55.675364 18971 log.cc:927] Time spent T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 12:22:55.635285 18853 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.638265 18674 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:55.679389 18861 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:55.754426 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:55.630568 18928 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:55.681212 16548 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:22:55.548503 18711 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:55.686964 18829 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:55.689862 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 113513us
I20260709 12:22:55.696712 18528 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:55.705348 18959 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:55.635681 18970 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:55.708163 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:22:55.716718 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:55.607581 18734 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:55.736810 16468 raft_consensus.cc:1240] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:55.739782 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:55.650133 18946 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 5. Current term is 6.
I20260709 12:22:55.670449 18582 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:22:55.751261 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 49053.5us
I20260709 12:22:55.679641 18949 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:55.760903 18853 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:55.755718 16476 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
W20260709 12:22:55.758090 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 51971.7us
I20260709 12:22:55.686007 18824 raft_consensus.cc:2804] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:55.635546 18956 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:55.762603 16515 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:55.770759 16471 raft_consensus.cc:1240] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:55.670248 18403 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:55.756599 18669 consensus_queue.cc:1048] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 12:22:55.687110 18967 consensus_queue.cc:237] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:56.053283 16516 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:55.931993 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 256194us
I20260709 12:22:55.829268 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:56.055598 16470 raft_consensus.cc:2410] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:22:55.860365 16475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 433) took 1194 ms. Trace:
I20260709 12:22:56.055802 16442 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:55.947304 16542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1118) took 2201 ms. Trace:
I20260709 12:22:56.056269 16475 rpcz_store.cc:276] 0709 12:22:54.665724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:54.665958 (+   234us) service_pool.cc:224] Handling call
0709 12:22:54.666395 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:55.156663 (+490268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:55.859985 (+703322us) spinlock_profiling.cc:243] Waited 490 ms on lock 0x7b5400b05254. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:55.860256 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1125883136}
I20260709 12:22:55.991415 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:22:56.053382 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 217112us
I20260709 12:22:56.058430 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:56.059696 16395 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:56.057410 16542 rpcz_store.cc:276] 0709 12:22:53.745714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:53.987793 (+242079us) service_pool.cc:224] Handling call
0709 12:22:53.997644 (+  9851us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:54.054166 (+ 56522us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:54.054202 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:54.054661 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 12:22:54.164155 (+109494us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:54.164197 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:55.861037 (+1696840us) spinlock_profiling.cc:243] Waited 56.1 ms on lock 0x7b5400729554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:55.861109 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:55.861177 (+    68us) raft_consensus.cc:1685] finished
0709 12:22:55.861194 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:55.947215 (+ 86021us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":259,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":6570,"spinlock_wait_cycles":129083136,"thread_start_us":109065,"threads_started":1,"wal-append.queue_time_us":223457,"wal-append.run_cpu_time_us":4948,"wal-append.run_wall_time_us":1032130}
W20260709 12:22:56.053238 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 290713us
W20260709 12:22:55.936213 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 159378us
I20260709 12:22:56.061049 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:56.061235 16441 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:56.053706 16474 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 5: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:56.053138 16473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 438) took 1257 ms. Trace:
I20260709 12:22:56.085549 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:56.085739 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:56.063143 16442 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:56.084856 16468 raft_consensus.cc:1240] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:22:55.750656 18798 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:56.087070 16395 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:56.088184 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 168577us
W20260709 12:22:56.095024 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:56.086017 16473 rpcz_store.cc:276] 0709 12:22:54.795705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:54.798156 (+  2451us) service_pool.cc:224] Handling call
0709 12:22:54.798538 (+   382us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:22:54.799873 (+  1335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:54.799903 (+    30us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:22:54.816575 (+ 16672us) log.cc:844] Serialized 78 byte log entry
0709 12:22:55.066827 (+250252us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:55.465344 (+398517us) log.cc:844] Serialized 14 byte log entry
0709 12:22:55.685774 (+220430us) log.cc:844] Serialized 14 byte log entry
0709 12:22:55.825418 (+139644us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:55.825504 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:55.825551 (+    47us) raft_consensus.cc:1685] finished
0709 12:22:55.825563 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:56.053079 (+227516us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35919,"prepare.queue_time_us":729983,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":4729,"raft.queue_time_us":406299,"raft.run_cpu_time_us":378,"raft.run_wall_time_us":8093,"thread_start_us":910486,"threads_started":4,"wal-append.queue_time_us":280848}
I20260709 12:22:56.095873 17261 raft_consensus.cc:2393] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 5: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:56.098484 16542 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:22:56.098371 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:22:55.947540 18290 consensus_meta.cc:287] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.610s	user 0.012s	sys 0.000s
I20260709 12:22:56.107352 16395 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
W20260709 12:22:56.110356 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 679096us
I20260709 12:22:56.111021 16543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 429) took 1620 ms. Trace:
I20260709 12:22:56.113133 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:56.113826 16395 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:22:56.111443 16543 rpcz_store.cc:276] 0709 12:22:54.490853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:55.111610 (+620757us) service_pool.cc:224] Handling call
0709 12:22:55.274511 (+162901us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:55.274735 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:55.274763 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:55.274807 (+    44us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:55.274846 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:55.274901 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:56.110790 (+835889us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b5400bb5150. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:56.110957 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":373914880}
I20260709 12:22:55.783136 18970 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:56.111299 17144 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:55.914206 18812 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:56.059007 18824 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:56.126910 18824 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:56.102658 18389 raft_consensus.cc:697] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:55.833335 18810 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:55.821839 18809 consensus_queue.cc:237] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:56.101750 18952 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:56.084059 18969 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:56.132397 18969 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:56.131757 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:55.835326 18702 raft_consensus.cc:3055] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:56.008874 18985 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.099337 18798 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:55.845429 18816 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:56.088304 18979 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:55.992609 18861 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.122851 16475 raft_consensus.cc:1240] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:56.135149 16400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 873) took 1497 ms. Trace:
I20260709 12:22:56.113904 18736 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:56.146130 16400 rpcz_store.cc:276] 0709 12:22:54.637254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:54.637827 (+   573us) service_pool.cc:224] Handling call
0709 12:22:56.134963 (+1497136us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b54001487d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:56.135081 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":875737344}
I20260709 12:22:56.115551 18801 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.136636 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:56.098358 18981 raft_consensus.cc:2804] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:56.066073 18879 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:22:56.126063 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 64892.2us
I20260709 12:22:55.799592 18825 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:56.138928 18702 raft_consensus.cc:740] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:56.160513 16395 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:22:56.161861 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:56.137719 18982 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:22:56.162432 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:56.163704 16366 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:56.164008 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:56.168448 16441 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:56.132848 18969 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:56.087666 18956 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:56.170467 18812 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.083880 18978 consensus_queue.cc:1048] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.087s
I20260709 12:22:56.101761 18980 consensus_queue.cc:1048] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.083s
I20260709 12:22:56.120540 18950 consensus_queue.cc:1059] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 12:22:56.131956 18952 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:56.172873 18698 consensus_queue.cc:260] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:56.009855 18818 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:56.133445 18970 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:56.195744 16441 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:56.159029 18816 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:56.198243 18702 consensus_queue.cc:260] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:56.163965 18801 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:56.189697 16394 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:56.249745 18928 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:56.204520 18935 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:22:56.209010 16549 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52680 (request call id 426) took 1809 ms (client timeout 2326 ms). Trace:
I20260709 12:22:56.209503 18403 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:56.210304 18908 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:22:56.211537 18968 log.cc:927] Time spent T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.782s	user 0.003s	sys 0.001s
I20260709 12:22:56.169452 18981 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:56.219745 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:22:56.220659 18984 log.cc:927] Time spent T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 12:22:56.221282 16516 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
W20260709 12:22:56.221527 18995 log.cc:927] Time spent T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:22:56.223484 18969 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:56.229136 18986 log.cc:927] Time spent T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:22:56.163304 18861 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:56.237118 18661 raft_consensus.cc:993] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1: : Instructing follower 52ab50b0237346c1a13ede947f6a63c2 to start an election
I20260709 12:22:56.129320 18389 consensus_queue.cc:237] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:56.119194 18949 consensus_queue.cc:1048] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 12:22:56.246220 16399 raft_consensus.cc:1275] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:56.147004 18991 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:22:56.262079 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 150859us
I20260709 12:22:56.199726 18806 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:56.238703 18982 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:56.320578 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:56.260673 18996 consensus_queue.cc:1059] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 12:22:56.321301 17255 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:56.256227 18728 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.196171 18879 consensus_queue.cc:1059] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.066s
I20260709 12:22:56.269989 16398 raft_consensus.cc:2368] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8.
I20260709 12:22:56.142714 18985 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:56.270012 16549 rpcz_store.cc:269] 0709 12:22:54.399889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:54.529833 (+129944us) service_pool.cc:224] Handling call
0709 12:22:56.208814 (+1678981us) spinlock_profiling.cc:243] Waited 653 ms on lock 0x7b5400a80cd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:56.208951 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1502338816}
I20260709 12:22:56.327569 16549 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:22:56.195178 18825 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:56.273393 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 77739.8us
I20260709 12:22:56.330075 16441 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:56.249958 18812 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:56.282611 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 119778us
I20260709 12:22:56.260134 18721 consensus_queue.cc:1059] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 12:22:56.268460 18581 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:22:56.339203 18825 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:56.340407 16441 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:56.341524 16441 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:56.342072 18991 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:56.258082 18829 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:56.293215 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:56.282557 18702 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:56.321432 18734 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:56.283550 18981 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:56.366297 18985 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:22:56.300681 18222 consensus_meta.cc:287] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.548s	user 0.008s	sys 0.004s
I20260709 12:22:56.366675 18676 raft_consensus.cc:3055] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:56.367053 18222 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:56.335582 18980 consensus_queue.cc:1059] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 12:22:56.368685 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:56.369189 18980 raft_consensus.cc:3055] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:56.369539 18980 raft_consensus.cc:740] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:56.370851 18829 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:56.301892 18952 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:56.373215 18894 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:56.303510 18816 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:56.277933 18824 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:56.353972 18935 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:22:56.379305 18935 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:56.379802 18935 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:56.380371 18824 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:56.307255 18950 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:56.381270 18950 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:56.381754 18950 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:56.382433 18969 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:56.308190 18661 raft_consensus.cc:1081] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Signalling peer 52ab50b0237346c1a13ede947f6a63c2 to start an election
I20260709 12:22:56.310048 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:56.384449 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:56.385710 18861 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:56.346724 18978 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:56.311712 18798 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:56.330382 18728 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:56.390934 18798 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:56.265635 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 105216us
I20260709 12:22:56.392010 18728 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:56.392503 16514 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:56.395309 18853 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:56.396483 18999 log.cc:927] Time spent T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:22:56.325798 18879 raft_consensus.cc:3055] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:56.398267 18879 raft_consensus.cc:740] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:56.399111 18879 consensus_queue.cc:260] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:56.318257 18928 consensus_queue.cc:1048] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 12:22:56.401376 18904 raft_consensus.cc:3055] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:56.401808 18904 raft_consensus.cc:740] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:56.402664 18904 consensus_queue.cc:260] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:56.403860 18904 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:56.321542 19003 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:56.405236 19003 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:56.406018 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:56.406723 17254 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:56.407522 19003 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:56.407757 16545 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6.
I20260709 12:22:56.409752 16514 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:56.411765 16394 raft_consensus.cc:1240] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:56.412742 18957 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:56.413401 16515 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:56.414425 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:56.415030 16399 raft_consensus.cc:2376] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6. Re-sending same reply.
I20260709 12:22:56.416901 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:56.417697 16396 raft_consensus.cc:2393] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 8: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:56.418777 17254 raft_consensus.cc:1275] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:56.421234 18853 consensus_queue.cc:1048] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:56.423024 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:56.424180 17254 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:56.425333 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:56.342861 16441 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:56.321424 18979 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:56.432443 18978 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:56.432673 18979 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:56.295835 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 74650.5us
I20260709 12:22:56.433854 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:56.361914 18812 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:56.370359 18980 consensus_queue.cc:260] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:56.300992 18818 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:56.376046 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:56.436378 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:56.360051 17261 raft_consensus.cc:1275] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:56.427279 16550 raft_consensus.cc:1275] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:56.429219 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:56.430401 18950 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:56.433020 18978 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:56.434376 16516 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:56.435468 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:56.448495 16477 raft_consensus.cc:2376] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6. Re-sending same reply.
I20260709 12:22:56.358302 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:56.435902 19003 consensus_queue.cc:1048] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:56.435438 18979 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:56.436051 18581 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:22:56.432013 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 89227us
I20260709 12:22:56.436878 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:56.437388 16515 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:56.438061 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:56.439273 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:56.465121 16396 raft_consensus.cc:2393] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 8: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:56.442337 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:56.466132 16473 raft_consensus.cc:2410] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 12:22:56.443462 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:56.425917 18949 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:56.452675 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:56.477458 16470 raft_consensus.cc:2393] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 4: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:56.454718 17266 raft_consensus.cc:2393] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 8: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:56.478571 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:56.479923 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:56.455873 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:56.456184 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:56.350060 17255 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:22:56.459105 16395 raft_consensus.cc:1240] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:56.460830 17144 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:56.395313 19002 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:56.367069 18676 raft_consensus.cc:740] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:56.476488 16399 raft_consensus.cc:2393] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 5: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:56.467772 16476 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:56.449648 18985 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:22:56.510684 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 76371.5us
I20260709 12:22:56.526234 16551 raft_consensus.cc:1240] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:56.493660 16366 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:56.475541 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:56.512528 16549 raft_consensus.cc:606] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:22:56.529182 16549 raft_consensus.cc:614] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:56.507613 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:22:56.461033 18947 consensus_meta.cc:287] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 1.049s	user 0.008s	sys 0.005s
W20260709 12:22:56.521646 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 49471.6us
I20260709 12:22:56.483484 18969 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:56.451419 16474 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:34826
I20260709 12:22:56.537853 16474 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:56.538373 16474 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:56.539934 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:56.531284 16516 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:56.521775 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 61007.1us
I20260709 12:22:56.477067 18949 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:56.475406 18980 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:56.534924 16366 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:56.547267 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:56.484375 16441 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:56.484539 18991 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:56.555984 17254 raft_consensus.cc:2468] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:56.558066 16515 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:56.476253 18816 consensus_queue.cc:1059] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 12:22:56.529801 18360 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:56.534678 18818 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:56.532235 18661 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:56.482887 18853 consensus_queue.cc:1048] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.061s
W20260709 12:22:56.571362 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 87097.9us
I20260709 12:22:56.509516 18798 consensus_queue.cc:1059] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 12:22:56.514595 18812 consensus_queue.cc:1048] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 12:22:56.513969 18601 consensus_queue.cc:1059] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:22:56.537324 18969 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:56.532429 18985 raft_consensus.cc:697] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:56.482950 18581 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:56.549170 19002 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.548111 18949 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:56.563936 18991 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:56.586306 18676 consensus_queue.cc:260] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:56.689646 18676 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:56.692553 18935 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:56.646672 18711 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:56.696877 18222 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:56.697181 18950 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:56.697835 18829 raft_consensus.cc:3055] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:56.698417 18829 raft_consensus.cc:740] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:56.700718 18679 raft_consensus.cc:3055] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:56.702030 18679 raft_consensus.cc:740] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:56.575449 18762 consensus_queue.cc:1048] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.128s
I20260709 12:22:56.562192 18952 consensus_queue.cc:1059] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.102s
I20260709 12:22:56.705765 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:56.705590 16472 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:22:56.706574 16475 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:56.637006 16469 raft_consensus.cc:1275] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:56.544260 18957 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.556597 18825 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:56.561941 18728 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:22:56.559693 18947 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:56.634464 18389 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:22:56.643921 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 82975us
I20260709 12:22:56.730189 18947 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:56.732076 18947 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:56.734297 18950 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:56.734866 18950 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:56.660476 18661 raft_consensus.cc:697] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:56.631286 18582 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:56.641398 16474 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:56.651211 17263 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:56.730681 18389 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:56.592603 18806 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.736733 18661 consensus_queue.cc:237] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:56.737912 18806 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:56.738581 18389 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:56.739650 18950 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:56.665006 18464 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:56.644047 18403 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.741014 18464 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:56.741247 18403 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:56.743261 18464 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:56.637625 18847 raft_consensus.cc:2804] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:56.736737 18582 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:56.745594 18847 raft_consensus.cc:697] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:56.720453 19002 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:56.746541 18847 consensus_queue.cc:237] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:56.747051 18582 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:56.743324 18403 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:56.747648 19002 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:56.748607 18360 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:22:56.749212 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 92154us
I20260709 12:22:56.750075 18403 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.750749 18403 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:56.752589 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:56.753711 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:56.656282 18969 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:56.754380 18403 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:56.754395 17263 raft_consensus.cc:2393] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 8: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:56.757493 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:56.757068 18981 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:56.727406 18957 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:22:56.661837 18961 consensus_meta.cc:287] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 1.120s	user 0.012s	sys 0.000s
I20260709 12:22:56.667253 18985 consensus_queue.cc:237] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:56.765472 18806 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:56.775748 16551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 440) took 1086 ms. Trace:
I20260709 12:22:56.776247 16551 rpcz_store.cc:276] 0709 12:22:55.689415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:56.385273 (+695858us) service_pool.cc:224] Handling call
0709 12:22:56.385537 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:56.526900 (+141363us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:56.775464 (+248564us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b54003de854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:56.775673 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":323040512}
I20260709 12:22:56.777734 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:56.777778 18806 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.776352 18949 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:56.778412 18806 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:56.780668 18981 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:56.781282 18925 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:56.782754 18806 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:56.785696 18949 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:22:56.788460 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 251507us
I20260709 12:22:56.789348 16514 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:56.790211 18666 raft_consensus.cc:2804] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:56.790836 18666 raft_consensus.cc:697] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:56.699563 18829 consensus_queue.cc:260] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:56.791708 18666 consensus_queue.cc:237] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:56.792448 16471 raft_consensus.cc:2410] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:22:56.792891 18829 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:56.795307 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:56.799057 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:56.800482 16470 raft_consensus.cc:1275] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 12:22:56.801978 16470 pending_rounds.cc:85] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 3
I20260709 12:22:56.802232 16470 pending_rounds.cc:107] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:22:56.802459 16470 raft_consensus.cc:2889] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:56.803090 16474 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:56.800768 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:56.804682 17263 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:22:56.805290 18806 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.806056 18806 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:56.804206 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:56.730221 16475 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:22:56.809406 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:56.810150 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:56.811038 16470 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 5: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:56.704046 18679 consensus_queue.cc:260] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:56.812979 18806 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:56.813938 18679 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:56.814227 18581 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:56.813032 18978 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:56.816754 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:56.817571 16474 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:22:56.796126 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:56.821051 16471 raft_consensus.cc:2393] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 10: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:56.810468 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:22:56.840847 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:56.763679 18403 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:56.841475 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:56.842481 17263 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:22:56.842485 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: true dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:56.851687 16551 raft_consensus.cc:3055] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:56.852099 16551 raft_consensus.cc:740] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:56.852102 16515 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:22:56.800732 18970 consensus_meta.cc:287] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.534s	user 0.006s	sys 0.005s
I20260709 12:22:56.853549 16551 consensus_queue.cc:260] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:56.859210 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:56.859699 16551 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:56.861706 16548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37226 (request call id 1133) took 1329 ms. Trace:
I20260709 12:22:56.862120 16548 rpcz_store.cc:276] 0709 12:22:55.532533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:55.623058 (+ 90525us) service_pool.cc:224] Handling call
0709 12:22:56.861543 (+1238485us) spinlock_profiling.cc:243] Waited 57.1 ms on lock 0x7b54009745d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:56.861649 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":131217152}
I20260709 12:22:56.863431 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:56.868266 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:56.860350 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:56.870812 16548 raft_consensus.cc:2410] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:22:56.871874 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:56.874500 16548 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:56.858946 18894 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:56.884935 16551 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
I20260709 12:22:56.887614 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:56.888301 16551 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
I20260709 12:22:56.860787 18728 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:56.890571 16441 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:56.859913 18403 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:56.889356 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:56.900651 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:56.902477 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:56.895983 18702 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:56.911155 18702 raft_consensus.cc:697] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:56.912161 18702 consensus_queue.cc:237] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:56.868465 18970 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:56.917033 18970 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:56.921489 16542 raft_consensus.cc:2368] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 8. Current term is 9.
I20260709 12:22:56.923403 18947 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:56.924011 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:56.925020 17263 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:22:56.932706 16548 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:22:56.936712 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:56.894394 18894 raft_consensus.cc:697] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:56.867100 18969 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:56.941476 17263 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:22:56.945451 16550 raft_consensus.cc:1275] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:56.945664 16515 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:22:56.937567 18904 consensus_meta.cc:287] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.533s	user 0.008s	sys 0.005s
I20260709 12:22:56.949316 16550 raft_consensus.cc:1240] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:56.894744 18728 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:56.952476 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:56.953827 16551 raft_consensus.cc:1240] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:56.961544 16547 raft_consensus.cc:2368] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6.
I20260709 12:22:56.948058 18949 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:56.981755 18947 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:56.994503 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:56.997174 17255 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:56.997394 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:56.998215 16468 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:22:57.001443 16550 raft_consensus.cc:1240] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:57.001735 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:56.981986 18969 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:56.961349 18978 consensus_queue.cc:1059] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:22:56.995066 18820 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.010504 17264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 745) took 1768 ms. Trace:
I20260709 12:22:57.011807 17255 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:22:57.010927 17264 rpcz_store.cc:276] 0709 12:22:55.242196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:55.244103 (+  1907us) service_pool.cc:224] Handling call
0709 12:22:55.244601 (+   498us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:55.244983 (+   382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:55.245014 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:55.245868 (+   854us) log.cc:844] Serialized 52 byte log entry
0709 12:22:55.529261 (+283393us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:55.730188 (+200927us) log.cc:844] Serialized 14 byte log entry
0709 12:22:55.730673 (+   485us) log.cc:844] Serialized 14 byte log entry
0709 12:22:56.208369 (+477696us) spinlock_profiling.cc:243] Waited 2.91 ms on lock 0x7b1800221440. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c522 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:22:57.007087 (+798718us) spinlock_profiling.cc:243] Waited 7.81 ms on lock 0x7b5400a13720. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e04733 00007f2875e89dbf 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c
0709 12:22:57.010249 (+  3162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:57.010308 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:57.010343 (+    35us) raft_consensus.cc:1685] finished
0709 12:22:57.010349 (+     6us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:57.010456 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35687,"prepare.queue_time_us":386131,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":55,"raft.queue_time_us":44361,"raft.run_cpu_time_us":348,"raft.run_wall_time_us":8915,"spinlock_wait_cycles":24860672,"thread_start_us":466877,"threads_started":2,"wal-append.queue_time_us":385409,"wal-append.run_cpu_time_us":3688,"wal-append.run_wall_time_us":1206120}
I20260709 12:22:57.012885 16515 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:56.999965 18949 raft_consensus.cc:697] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:56.999267 18829 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:57.014693 18820 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:57.030097 18879 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:57.030421 18820 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:57.033483 18403 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:57.035125 18800 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:57.035352 18728 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:57.036397 18959 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:57.020264 18661 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:56.998996 18894 consensus_queue.cc:237] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:57.044880 18949 consensus_queue.cc:237] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:57.049067 18661 raft_consensus.cc:697] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:57.051347 18661 consensus_queue.cc:237] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:57.021548 18829 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:57.058588 18698 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:57.058956 18829 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:57.073024 18935 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:57.077515 18801 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:57.078387 18845 raft_consensus.cc:3055] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:57.077570 18956 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:57.079051 18845 raft_consensus.cc:740] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:57.079794 18845 consensus_queue.cc:260] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:57.080642 18845 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:57.094225 18821 raft_consensus.cc:3055] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:57.096725 18957 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:57.099431 18910 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.100719 18991 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:57.100703 18910 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:57.103440 18982 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:57.103814 18910 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:57.094911 18821 raft_consensus.cc:740] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:57.106887 18821 consensus_queue.cc:260] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:57.108126 18821 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:57.179041 16366 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:57.182744 16366 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:57.184805 16367 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:57.186385 18728 consensus_queue.cc:1048] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:57.188035 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:57.193691 18679 consensus_queue.cc:1048] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:57.196660 18582 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:57.197283 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:57.198798 16471 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:22:57.200207 18403 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:57.201928 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:57.203733 16475 raft_consensus.cc:2393] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 6: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
W20260709 12:22:57.180356 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 150027us
W20260709 12:22:57.220786 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:57.225960 18582 raft_consensus.cc:697] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 8 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:57.252055 18728 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:57.258930 18728 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:57.259586 18728 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:22:57.262356 16542 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52680 (request call id 443) took 1423 ms (client timeout 1848 ms). Trace:
W20260709 12:22:57.263077 16542 rpcz_store.cc:269] 0709 12:22:55.838254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:56.124757 (+286503us) service_pool.cc:224] Handling call
0709 12:22:57.261638 (+1136881us) spinlock_profiling.cc:243] Waited 788 ms on lock 0x7b5400ac8354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:57.262011 (+   373us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1812459904}
I20260709 12:22:57.265607 18582 consensus_queue.cc:237] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:57.266788 17264 raft_consensus.cc:1275] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:57.266980 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:57.262639 17269 raft_consensus.cc:1275] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:57.269042 16544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1132) took 1742 ms. Trace:
I20260709 12:22:57.271293 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:57.273062 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:22:57.281440 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:57.281219 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:57.273687 16544 rpcz_store.cc:276] 0709 12:22:55.526127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:55.765476 (+239349us) service_pool.cc:224] Handling call
0709 12:22:55.819458 (+ 53982us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:56.206314 (+386856us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:56.206347 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:56.206401 (+    54us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:56.206435 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:57.268515 (+1062080us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b5400304b54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:57.268584 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:57.268836 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":889143552}
I20260709 12:22:57.290161 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:57.307772 18949 consensus_queue.cc:1048] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 12:22:57.307745 18736 consensus_queue.cc:1048] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 12:22:57.315387 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:57.319190 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:57.323582 16367 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:57.325127 18679 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:57.333832 18728 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:57.337020 18728 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:57.329198 16367 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:57.342185 16367 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:57.342422 16473 raft_consensus.cc:1240] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:57.334264 16399 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:22:57.344043 18789 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:57.347869 16367 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:57.354518 18961 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:57.355289 18961 raft_consensus.cc:697] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 8 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:57.356616 18961 consensus_queue.cc:237] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:57.364004 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:22:57.365233 16398 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 882) took 2107 ms (client timeout 1730 ms). Trace:
W20260709 12:22:57.365767 16398 rpcz_store.cc:269] 0709 12:22:55.257680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:55.257819 (+   139us) service_pool.cc:224] Handling call
0709 12:22:57.364775 (+2106956us) spinlock_profiling.cc:243] Waited 977 ms on lock 0x7b5400762ad4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:57.365044 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2247323392}
I20260709 12:22:57.366712 18829 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:57.367260 16367 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:57.370149 18829 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:57.367895 16223 catalog_manager.cc:5697] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 3 to 6, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 6 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:57.376085 16397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 847) took 1318 ms. Trace:
W20260709 12:22:57.373353 17271 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45228 (request call id 725) took 1878 ms (client timeout 1960 ms). Trace:
W20260709 12:22:57.377060 17271 rpcz_store.cc:269] 0709 12:22:55.481538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:55.481715 (+   177us) service_pool.cc:224] Handling call
0709 12:22:57.360071 (+1878356us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b540005df54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:57.360248 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3784952704}
W20260709 12:22:57.380057 16224 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53626 (request call id 126) took 16202 ms (client timeout 15000 ms). Trace:
I20260709 12:22:57.370721 18829 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:57.374902 16223 catalog_manager.cc:5697] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 1 to 4, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 4 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:57.376538 16397 rpcz_store.cc:276] 0709 12:22:56.057308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:56.057420 (+   112us) service_pool.cc:224] Handling call
0709 12:22:56.057757 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:56.767850 (+710093us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:22:56.767891 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:56.767939 (+    48us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:56.767981 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:57.375447 (+607466us) spinlock_profiling.cc:243] Waited 709 ms on lock 0x7b540015fc54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:57.375557 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:57.375868 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2944,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":1437,"spinlock_wait_cycles":1631504768}
I20260709 12:22:57.383783 18829 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:57.390470 16547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37242 (request call id 951) took 1042 ms. Trace:
I20260709 12:22:57.390970 16547 rpcz_store.cc:276] 0709 12:22:56.347853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:56.375905 (+ 28052us) service_pool.cc:224] Handling call
0709 12:22:57.390043 (+1014138us) spinlock_profiling.cc:243] Waited 481 ms on lock 0x7b540061eed4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:57.390278 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1105467776}
W20260709 12:22:57.394482 17270 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45230 (request call id 751) took 1867 ms (client timeout 2080 ms). Trace:
W20260709 12:22:57.403468 17270 rpcz_store.cc:269] 0709 12:22:55.526704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:55.526909 (+   205us) service_pool.cc:224] Handling call
0709 12:22:57.393963 (+1867054us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b5400a74c54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:57.394264 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2300809472}
W20260709 12:22:57.395032 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:57.408536 16546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 948) took 2003 ms. Trace:
I20260709 12:22:57.392323 16223 catalog_manager.cc:5697] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 4 to 6, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 6 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:57.408990 16546 rpcz_store.cc:276] 0709 12:22:55.405255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:55.405469 (+   214us) service_pool.cc:224] Handling call
0709 12:22:55.405811 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:55.680254 (+274443us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:22:55.680614 (+   360us) log.cc:844] Serialized 14 byte log entry
0709 12:22:56.510008 (+829394us) log.cc:844] Serialized 14 byte log entry
0709 12:22:56.512170 (+  2162us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:56.512221 (+    51us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:56.512275 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:57.070330 (+558055us) spinlock_profiling.cc:243] Waited 7.11 ms on lock 0x7b18000768a0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c522 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:22:57.407244 (+336914us) spinlock_profiling.cc:243] Waited 1.54 ms on lock 0x7b18000768a0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d6a2a6 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419
0709 12:22:57.408022 (+   778us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b5400272c54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:57.408074 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:57.408324 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":85031,"prepare.queue_time_us":4928,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":83,"raft.queue_time_us":191264,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":1543,"spinlock_wait_cycles":650451456,"thread_start_us":690868,"threads_started":2,"wal-append.queue_time_us":528667}
I20260709 12:22:57.401242 16468 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:22:57.412955 18013 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:57.405973 18978 raft_consensus.cc:3055] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:57.414765 18978 raft_consensus.cc:740] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:57.416402 18978 consensus_queue.cc:260] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:57.418418 18978 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:57.420737 18981 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:57.421296 18845 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.421643 18981 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:57.422161 18845 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:57.425630 18845 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:57.407649 18991 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:57.426829 16548 raft_consensus.cc:1275] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:57.427124 18991 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:57.407891 18935 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.429436 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:57.429265 18935 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:57.432760 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:57.428529 18060 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.434638 18060 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:57.437521 16403 raft_consensus.cc:1240] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:22:57.439764 18060 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:22:57.440201 19038 log.cc:927] Time spent T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:22:57.442116 18845 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:57.443944 18845 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:57.444680 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:57.445700 16442 connection.cc:256] RPC call timeout handler was delayed by -1.1148566770002617s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17300000000000001s for a response
I20260709 12:22:57.446215 18935 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:57.447629 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:57.448280 16476 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 4.
I20260709 12:22:57.448943 17270 raft_consensus.cc:1240] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:57.396987 16515 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:57.453336 18861 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:22:57.449613 19037 log.cc:927] Time spent T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.166s	user 0.001s	sys 0.004s
I20260709 12:22:57.439972 16395 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:22:57.450181 18845 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:57.457198 16548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1136) took 1281 ms. Trace:
I20260709 12:22:57.457727 16398 raft_consensus.cc:1275] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:57.458120 16548 rpcz_store.cc:276] 0709 12:22:56.175386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:56.938517 (+763131us) service_pool.cc:224] Handling call
0709 12:22:56.938931 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:57.427319 (+488388us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:57.428071 (+   752us) spinlock_profiling.cc:243] Waited 487 ms on lock 0x7b5400aa69d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:57.428327 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1120103808}
W20260709 12:22:57.453418 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 56635.4us
I20260709 12:22:57.461223 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:57.369367 18728 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:22:57.401129 19036 log.cc:927] Time spent T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.126s	user 0.000s	sys 0.002s
I20260709 12:22:57.460059 18991 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:57.381315 16224 rpcz_store.cc:269] 0709 12:22:41.177273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.215272 (+ 37999us) service_pool.cc:224] Handling call
0709 12:22:42.119981 (+904709us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b540027e054. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df3090 00007f2875e91f3a 00007f287bc47e9c 00007f287bd4604e 00007f287a2072a4 00007f287a207232 00007f287a207161 00007f287a2070dc 00007f287a206392 00007f28741c448c 00007f28741c38c6
0709 12:22:57.379710 (+15259729us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:22:48.595472 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:22:48.784693 (+189221us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:22:48.784745 (+    52us) write_op.cc:435] Acquired schema lock
0709 12:22:48.784780 (+    35us) tablet.cc:662] Decoding operations
0709 12:22:48.786348 (+  1568us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:22:48.786445 (+    97us) write_op.cc:641] Partition lock acquired for write op
0709 12:22:48.786477 (+    32us) tablet.cc:685] Acquiring locks for 16 operations
0709 12:22:48.787853 (+  1376us) tablet.cc:701] Row locks acquired
0709 12:22:48.787905 (+    52us) write_op.cc:260] PREPARE: finished
0709 12:22:48.788086 (+   181us) write_op.cc:270] Start()
0709 12:22:48.788246 (+   160us) write_op.cc:276] Timestamp: P: 1783599768788015 usec, L: 0
0709 12:22:48.788269 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:22:49.141883 (+353614us) log.cc:844] Serialized 6217 byte log entry
0709 12:22:49.593265 (+451382us) op_driver.cc:464] REPLICATION: finished
0709 12:22:49.755335 (+162070us) write_op.cc:301] APPLY: starting
0709 12:22:49.773099 (+ 17764us) tablet.cc:1367] starting BulkCheckPresence
0709 12:22:49.903830 (+130731us) tablet.cc:1370] finished BulkCheckPresence
0709 12:22:49.941486 (+ 37656us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:22:50.027435 (+ 85949us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:22:50.027870 (+   435us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=16
0709 12:22:50.027920 (+    50us) write_op.cc:312] APPLY: finished
0709 12:22:50.046283 (+ 18363us) log.cc:844] Serialized 112 byte log entry
0709 12:22:50.096420 (+ 50137us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:50.096614 (+   194us) write_op.cc:454] Released schema lock
0709 12:22:50.096698 (+    84us) write_op.cc:341] FINISH: Updating metrics
0709 12:22:50.097253 (+   555us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:50.097292 (+    39us) write_op.cc:454] Released schema lock
0709 12:22:50.428842 (+331550us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b20001d7c78. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875a37b91 00007f2875a59a19 00007f2875a248c3 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169 00007f2875a2d131 00007f2875a2c42d 0000000000388634
Metrics: {"mutex_wait_us":10449,"reported_tablets":36,"spinlock_wait_cycles":461629696,"child_traces":[["op",{"apply.queue_time_us":95598,"apply.run_cpu_time_us":6252,"apply.run_wall_time_us":364357,"num_ops":16,"prepare.queue_time_us":523790,"prepare.run_cpu_time_us":31248,"prepare.run_wall_time_us":794043,"raft.queue_time_us":109571,"raft.run_cpu_time_us":31205,"raft.run_wall_time_us":61748,"replication_time_us":804813,"spinlock_wait_cycles":26304384,"thread_start_us":712791,"threads_started":4,"wal-append.queue_time_us":310115,"wal-append.run_cpu_time_us":16794,"wal-append.run_wall_time_us":2037536}]]}
I20260709 12:22:57.484088 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:57.508698 16514 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:22:57.509516 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:57.510154 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 74065us
I20260709 12:22:57.511650 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:57.512881 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:57.413894 18013 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:57.487866 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:57.491570 18979 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:57.492663 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:57.493103 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:57.470592 18728 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:57.481052 18060 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.494292 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:57.495061 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:57.498225 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:57.517518 17263 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6.
I20260709 12:22:57.500701 16223 catalog_manager.cc:5697] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 4 to 5, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 5 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:57.503428 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:57.507606 18740 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:57.507845 18698 consensus_queue.cc:1048] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:57.485487 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:57.503324 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:57.523510 16395 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:57.528822 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:57.521965 18728 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:57.540617 18749 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:57.540851 18060 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:57.546587 16403 raft_consensus.cc:2393] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 6: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:57.541313 16367 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:57.539021 16397 raft_consensus.cc:1240] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:57.542778 17270 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
I20260709 12:22:57.551115 16223 catalog_manager.cc:5697] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:57.552412 18979 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:57.553459 16223 catalog_manager.cc:5697] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:57.555060 18979 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:57.555557 16441 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:57.556533 16551 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:22:57.557788 16441 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:57.558749 16514 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:57.561976 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:57.562739 17269 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:22:57.564222 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:57.564821 16469 raft_consensus.cc:1275] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:57.565548 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:57.546635 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: true dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:57.580492 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:57.582000 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:57.579355 16548 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:22:57.583981 16396 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:22:57.574532 16514 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:57.537091 18981 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:57.590118 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:57.584542 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:57.595635 16550 raft_consensus.cc:2393] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 7: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:57.599089 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:57.588022 16223 catalog_manager.cc:5697] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 4 to 5, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 5 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:57.605968 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:57.608180 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 161736us
I20260709 12:22:57.596621 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:57.607847 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:57.610210 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:57.610049 16544 raft_consensus.cc:1275] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:57.620976 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:57.621378 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:57.624902 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:57.596199 18935 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:57.597198 18979 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:57.627100 16551 raft_consensus.cc:1240] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:57.621898 16403 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6.
I20260709 12:22:57.640130 16542 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:22:57.655467 16398 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:57.615437 18981 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:57.629908 18979 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:57.662345 17271 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:22:57.672847 18987 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:57.673625 18987 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:57.674082 18987 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:57.676257 18982 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.676961 18982 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:57.677556 18013 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:57.678210 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:57.679034 17263 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:22:57.679757 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:57.680629 16549 raft_consensus.cc:2393] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 8: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:57.686815 17144 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:57.689116 19042 log.cc:927] Time spent T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.176s	user 0.000s	sys 0.002s
W20260709 12:22:57.691229 19043 log.cc:927] Time spent T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
W20260709 12:22:57.692632 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 132426us
I20260709 12:22:57.693385 18861 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:57.693811 16514 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:57.695758 18958 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:57.696285 18060 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:57.696900 18060 consensus_queue.cc:1048] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:57.698448 17258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 727) took 1340 ms. Trace:
I20260709 12:22:57.701320 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:57.701707 17261 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 741) took 1043 ms. Trace:
I20260709 12:22:57.702066 16396 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:57.702140 18736 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:57.703042 18927 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:57.703264 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:57.703549 18927 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:57.702821 18736 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:57.704108 18927 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:57.705979 18736 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:57.706267 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:57.702174 17261 rpcz_store.cc:276] 0709 12:22:56.658463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:56.658612 (+   149us) service_pool.cc:224] Handling call
0709 12:22:56.659038 (+   426us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:56.659437 (+   399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:56.659470 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:56.761067 (+101597us) log.cc:844] Serialized 52 byte log entry
0709 12:22:57.334677 (+573610us) spinlock_profiling.cc:243] Waited 21.6 ms on lock 0x7b18001e0880. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:22:57.607131 (+272454us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:57.622974 (+ 15843us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:57.632075 (+  9101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:57.690340 (+ 58265us) spinlock_profiling.cc:243] Waited 2.93 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:22:57.701326 (+ 10986us) raft_consensus.cc:1685] finished
0709 12:22:57.701347 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:57.701527 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6711,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":33496,"spinlock_wait_cycles":56341760,"thread_start_us":208031,"threads_started":1,"wal-append.queue_time_us":849390}
I20260709 12:22:57.707819 16472 raft_consensus.cc:3055] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:57.709304 16475 raft_consensus.cc:1275] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:57.710503 16471 raft_consensus.cc:1275] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 12:22:57.712796 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 171622us
I20260709 12:22:57.693909 18861 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:57.692243 18981 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:57.676124 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:22:57.716367 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:57.648320 18403 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:57.714788 16472 raft_consensus.cc:740] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:22:57.719362 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 161733us
I20260709 12:22:57.712000 16223 catalog_manager.cc:5697] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 3 to 6, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 6 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:57.670858 18979 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:57.728826 18403 raft_consensus.cc:697] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 5 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:57.735306 18403 consensus_queue.cc:237] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:57.737764 18789 consensus_queue.cc:1048] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:57.740693 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:57.741112 16470 raft_consensus.cc:1240] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:57.741636 16475 raft_consensus.cc:2393] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 4: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:57.698865 17258 rpcz_store.cc:276] 0709 12:22:56.357582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:56.358582 (+  1000us) service_pool.cc:224] Handling call
0709 12:22:56.358910 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:56.374780 (+ 15870us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:56.374813 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:57.356740 (+981927us) spinlock_profiling.cc:243] Waited 1.5 ms on lock 0x7b2c00000708. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:22:57.357405 (+   665us) log.cc:844] Serialized 26 byte log entry
0709 12:22:57.482608 (+125203us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:57.496435 (+ 13827us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:57.496542 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:57.697947 (+201405us) raft_consensus.cc:1685] finished
0709 12:22:57.697965 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:57.698241 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15408,"raft.queue_time_us":21338,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":87,"spinlock_wait_cycles":3447680,"thread_start_us":117248,"threads_started":1,"wal-append.queue_time_us":219485}
I20260709 12:22:57.740751 16472 consensus_queue.cc:260] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:57.744649 16472 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:57.744565 18661 consensus_queue.cc:1048] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:57.679476 18982 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:57.746809 16441 leader_election.cc:400] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6.
I20260709 12:22:57.747143 16441 leader_election.cc:403] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:57.748368 18982 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6.
I20260709 12:22:57.748607 18879 consensus_queue.cc:1059] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:57.749917 18789 raft_consensus.cc:3055] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:57.750449 18789 raft_consensus.cc:740] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:57.749958 16223 catalog_manager.cc:5697] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 2 to 5, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 5 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:57.751237 18789 consensus_queue.cc:260] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:57.752568 18789 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:57.719043 18861 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:57.752179 16223 catalog_manager.cc:5697] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 1 to 5, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 5 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:57.755012 16395 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:22:57.762524 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:57.770788 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:57.760830 16398 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:22:57.758996 16542 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:22:57.777689 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:57.779224 16396 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
W20260709 12:22:57.763099 19041 log.cc:927] Time spent T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 12:22:57.787000 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:57.783818 16403 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 11. Current term is 12. Ops: [10.7-11.8]
I20260709 12:22:57.786391 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:57.764350 16223 catalog_manager.cc:5697] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 2 to 4, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 4 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
I20260709 12:22:57.786123 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:57.770475 18749 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:57.801757 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:57.802165 16398 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:57.802772 16542 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:22:57.803679 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:57.819541 17261 raft_consensus.cc:2468] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:57.764936 18728 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:57.763434 18829 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:57.833762 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:57.839112 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:57.779919 18927 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:57.853508 16441 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:57.855702 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:22:57.856002 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:57.859515 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:57.862480 16400 raft_consensus.cc:1240] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:57.834946 18740 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.866371 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:57.867269 16396 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:22:57.866530 16395 raft_consensus.cc:2393] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 8: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
W20260709 12:22:57.850901 18679 consensus_meta.cc:287] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.525s	user 0.011s	sys 0.003s
W20260709 12:22:57.878424 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:57.878659 18702 raft_consensus.cc:2804] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:22:57.879034 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:57.880365 19045 log.cc:927] Time spent T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:22:57.884171 19000 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:57.885174 19000 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:57.888935 16516 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:57.890617 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 128598us
I20260709 12:22:57.891719 16516 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:57.893227 19048 log.cc:927] Time spent T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 12:22:57.893567 18749 raft_consensus.cc:2749] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:57.895888 18927 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:57.902130 18829 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:57.904150 18728 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:22:57.904644 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 67983.3us
I20260709 12:22:57.909907 18679 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:57.911224 17264 raft_consensus.cc:1275] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:22:57.911931 17270 raft_consensus.cc:2368] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 8. Current term is 9.
I20260709 12:22:57.918115 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:22:57.918905 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 115222us
I20260709 12:22:57.924602 18812 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:22:57.936132 19052 log.cc:927] Time spent T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.089s	user 0.003s	sys 0.000s
I20260709 12:22:57.874956 18740 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:57.871284 18698 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:57.879060 18702 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:57.937922 19050 log.cc:927] Time spent T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 12:22:57.873396 16223 catalog_manager.cc:5697] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 3 to 5, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 5 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:57.942648 18809 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:57.944216 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:57.948994 18981 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:57.927505 16400 raft_consensus.cc:1240] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:57.969308 16472 raft_consensus.cc:1275] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:22:57.927436 16515 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
W20260709 12:22:57.987252 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 59896us
I20260709 12:22:57.986344 16472 pending_rounds.cc:85] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 3
I20260709 12:22:57.959069 19000 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:57.973440 18812 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:57.995076 16401 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
I20260709 12:22:58.002246 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:58.008157 17258 raft_consensus.cc:1275] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:22:58.008490 19051 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:57.987900 18809 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:58.009296 19051 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:57.993358 16471 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:22:57.993983 16472 pending_rounds.cc:107] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:22:58.014271 16472 raft_consensus.cc:2889] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:57.984143 18702 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:57.993104 16367 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:58.018002 16400 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:22:57.998102 18661 raft_consensus.cc:2804] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:58.029954 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:58.026255 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:58.034722 16542 raft_consensus.cc:1240] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:57.989476 16398 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:22:58.037603 17144 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:58.013787 18812 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:58.054154 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:58.055068 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:58.056188 16397 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:22:58.057927 16398 raft_consensus.cc:1275] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:22:58.060788 17261 raft_consensus.cc:1275] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:22:58.065433 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:58.065680 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:58.067282 16476 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:22:57.997502 18698 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:58.029584 18661 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:58.049829 18829 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:22:58.088658 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 197094us
I20260709 12:22:58.090858 16472 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:58.092728 18675 consensus_queue.cc:1048] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 12:22:58.094857 18676 consensus_queue.cc:1048] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 12:22:58.075476 18927 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:58.102921 16516 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:58.080600 18066 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:22:58.110731 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 117730us
I20260709 12:22:58.113113 16398 raft_consensus.cc:1275] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:22:58.084975 18661 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:58.101552 18809 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:58.081158 18979 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:58.111261 18927 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:58.142987 16366 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:22:58.143258 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 53559.3us
I20260709 12:22:58.119657 18829 raft_consensus.cc:697] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:58.100090 18947 consensus_queue.cc:1048] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.083s
I20260709 12:22:58.128355 18676 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:58.153276 18927 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:58.139183 19051 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:58.199659 16516 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:58.173251 18961 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:58.226500 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:58.238499 16472 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
W20260709 12:22:58.229780 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 151257us
W20260709 12:22:58.249233 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 336676us
I20260709 12:22:58.250227 17143 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:58.232483 18961 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:58.256808 18961 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:58.262331 19000 raft_consensus.cc:2764] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:22:58.263860 16441 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:58.266929 18861 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:58.269075 19051 consensus_queue.cc:1048] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:22:58.220702 18987 consensus_meta.cc:287] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.546s	user 0.015s	sys 0.000s
W20260709 12:22:58.281189 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:58.282178 19057 log.cc:927] Time spent T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:22:58.228076 18949 raft_consensus.cc:2749] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:58.301004 16472 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:22:58.222855 18981 consensus_queue.cc:1059] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 12:22:58.313656 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:22:58.352306 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 201579us
I20260709 12:22:58.353490 16516 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:58.282608 18987 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:58.404410 19070 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.328903 18403 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:58.351202 18740 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:58.416215 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:22:58.438822 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 85446.8us
I20260709 12:22:58.435973 16442 leader_election.cc:400] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8.
I20260709 12:22:58.434788 16543 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
W20260709 12:22:58.445398 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 251148us
I20260709 12:22:58.446448 16471 raft_consensus.cc:2393] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 10: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:58.321204 18829 consensus_queue.cc:237] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:58.452235 16516 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:58.437568 18949 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.437881 18698 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:58.444828 18789 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:22:58.470170 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:58.426326 18403 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:58.448661 18861 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:58.436690 19070 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:58.464250 18979 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:58.501173 16367 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
W20260709 12:22:58.502600 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:58.481361 18789 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:22:58.503953 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:58.509115 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 245284us
I20260709 12:22:58.506716 18789 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:58.501259 18879 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:22:58.514382 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 74626.4us
I20260709 12:22:58.481837 18949 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:58.505424 18728 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:58.518298 16441 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:58.523257 18812 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:58.524636 16472 raft_consensus.cc:1240] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:58.527617 18812 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:58.536751 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 175828us
W20260709 12:22:58.548019 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 297947us
I20260709 12:22:58.553001 18947 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:58.553617 18403 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:58.554108 18879 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.554680 18879 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:58.561803 19070 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:58.562608 18987 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:58.553645 18947 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:58.567682 18948 consensus_queue.cc:1048] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:58.569880 18947 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:58.554152 16516 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:58.573442 19064 log.cc:927] Time spent T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 12:22:58.573717 16516 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:58.575280 18540 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:58.575548 16516 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:58.584733 18661 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:58.633843 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:58.635771 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:58.635968 18661 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:22:58.577663 19061 log.cc:927] Time spent T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:22:58.576052 18540 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:58.583369 18961 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:58.638564 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:58.639686 18961 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:58.597684 18879 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:58.598433 18949 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:22:58.560004 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 463928us
W20260709 12:22:58.602581 19059 log.cc:927] Time spent T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:22:58.614066 19076 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:58.614861 18979 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.645714 18979 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:22:58.648073 18702 consensus_meta.cc:287] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.599s	user 0.012s	sys 0.000s
I20260709 12:22:58.647940 18979 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:58.645735 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:22:58.621949 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 111641us
I20260709 12:22:58.622169 18812 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:58.649477 18812 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:58.649489 17261 raft_consensus.cc:2393] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 8: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:22:58.650007 18812 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:58.650430 16441 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:58.614952 19084 raft_consensus.cc:3055] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:58.651501 19084 raft_consensus.cc:740] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:58.625365 18991 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.625980 18935 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.652612 18991 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:58.653241 16366 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:58.627545 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:58.654925 18991 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:22:58.656126 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 58140.4us
I20260709 12:22:58.626204 19083 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.652593 18979 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:58.657008 19083 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:58.654940 18961 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:58.657720 17143 leader_election.cc:400] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 8. Current term is 9.
I20260709 12:22:58.658150 17143 leader_election.cc:403] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:58.658179 18961 raft_consensus.cc:697] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:58.629570 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:58.659298 18961 consensus_queue.cc:237] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:58.659960 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:58.657280 18979 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:22:58.660637 17264 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:22:58.628728 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:22:58.582619 16397 consensus_meta.cc:287] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.526s	user 0.010s	sys 0.000s
I20260709 12:22:58.661849 16397 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:22:58.662297 19081 consensus_queue.cc:1059] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:22:58.669720 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 130592us
I20260709 12:22:58.625741 16401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 891) took 1131 ms. Trace:
I20260709 12:22:58.653061 18935 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:58.661294 18910 consensus_queue.cc:1048] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:58.630499 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:58.701978 16471 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:22:58.648547 18702 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:58.668435 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:22:58.671236 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 95759.5us
I20260709 12:22:58.706627 16516 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:58.667244 16474 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:22:58.694887 18927 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:58.691220 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:58.693305 18935 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:58.675889 18676 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:58.719426 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:58.697026 19051 consensus_queue.cc:1048] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:58.690425 19084 consensus_queue.cc:260] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:58.698362 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:58.698908 17142 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:58.670374 16401 rpcz_store.cc:276] 0709 12:22:57.494624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:57.494720 (+    96us) service_pool.cc:224] Handling call
0709 12:22:58.625526 (+1130806us) spinlock_profiling.cc:243] Waited 472 ms on lock 0x7b540015f754. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:22:58.625652 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1086415104}
I20260709 12:22:58.659299 19083 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:58.709795 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:58.712296 16472 raft_consensus.cc:2393] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 4: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:58.660897 18979 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:58.714979 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:58.717590 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:58.729643 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:58.732076 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:58.733340 16468 raft_consensus.cc:1240] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:58.734108 16542 raft_consensus.cc:1275] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:22:58.738188 16470 raft_consensus.cc:2393] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 8: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:58.786123 16546 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:58.786295 16442 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:58.793005 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:58.793265 16473 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:22:58.790657 17264 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:22:58.803432 19083 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:58.803948 16472 raft_consensus.cc:1275] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:22:58.804107 18702 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:58.806649 16401 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d"
dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
mode: GRACEFUL
new_leader_uuid: "610f4feda11942739236f2b121a4b9c1"
 from {username='slave'} at 127.0.0.1:60492
I20260709 12:22:58.807466 16401 raft_consensus.cc:606] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Received request to transfer leadership to TS 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:58.807828 16401 raft_consensus.cc:614] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:58.808485 19046 consensus_queue.cc:1059] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.100s
I20260709 12:22:58.809051 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:58.810515 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:58.811949 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:58.813073 16396 raft_consensus.cc:2376] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12. Re-sending same reply.
I20260709 12:22:58.813452 18812 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:58.814694 16472 raft_consensus.cc:1240] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:58.814563 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:58.815937 18812 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:58.816561 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:22:58.816901 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 110375us
I20260709 12:22:58.834892 16516 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:58.817582 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:58.819510 16367 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:58.832348 16514 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:58.772480 18679 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.820405 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:58.821659 18910 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:58.822428 16396 raft_consensus.cc:1275] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:22:58.823069 16442 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:58.816594 18698 consensus_queue.cc:1048] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 12:22:58.738476 18991 consensus_queue.cc:1059] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.101s
I20260709 12:22:58.733722 18948 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.819293 18979 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:58.890650 16469 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
I20260709 12:22:58.891410 17261 raft_consensus.cc:2468] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:22:58.794023 18661 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:58.688454 18789 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:58.895455 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:58.895153 16403 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
W20260709 12:22:58.935639 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 236818us
W20260709 12:22:58.939065 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:58.880702 18676 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:58.940738 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:58.891196 18879 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:58.950235 16548 raft_consensus.cc:2410] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:22:58.941237 16476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 458) took 1047 ms. Trace:
I20260709 12:22:58.949260 16543 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:22:58.957950 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:58.953980 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 263007us
I20260709 12:22:58.964378 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:22:58.965560 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 84891.8us
W20260709 12:22:58.966678 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 143720us
I20260709 12:22:58.965749 16395 raft_consensus.cc:1240] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 8 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:58.975819 16366 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:58.965579 16549 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:58.989949 16546 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:22:58.992872 17142 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:58.992606 16471 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:22:58.957520 16476 rpcz_store.cc:276] 0709 12:22:57.893844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.080588 (+186744us) service_pool.cc:224] Handling call
0709 12:22:58.092441 (+ 11853us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:58.093760 (+  1319us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:22:58.343100 (+249340us) log.cc:844] Serialized 14 byte log entry
0709 12:22:58.940859 (+597759us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:58.940927 (+    68us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:58.940979 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:58.941043 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:58.941178 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":161582,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":67577,"thread_start_us":719656,"threads_started":2}
I20260709 12:22:58.948777 18676 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:59.000357 16545 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:58.956770 18013 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 8. Current term is 9.
I20260709 12:22:58.961800 18812 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.982573 18979 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:22:59.025489 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 57125.3us
W20260709 12:22:59.027284 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:58.981534 18927 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:59.030794 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:22:59.036952 16473 raft_consensus.cc:2393] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 6: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:58.984711 18879 raft_consensus.cc:697] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:58.981488 18789 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:58.968854 19076 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:58.993821 18947 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.948534 18679 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:59.012701 18676 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:59.067368 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 57173.7us
W20260709 12:22:59.054467 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 61693.3us
I20260709 12:22:59.012418 18698 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:59.063679 16476 raft_consensus.cc:3055] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:59.071702 16476 raft_consensus.cc:740] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:22:59.055313 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 385047us
I20260709 12:22:58.959568 18948 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:58.972625 18981 consensus_queue.cc:1048] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 12:22:59.012759 18949 consensus_queue.cc:1048] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 12:22:59.077538 17143 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:59.082316 17142 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:59.040617 18812 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:59.103401 17144 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:59.052011 19046 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:22:59.121968 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 50738.9us
I20260709 12:22:59.073541 18861 consensus_queue.cc:1048] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.037s
W20260709 12:22:59.124366 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 88657.6us
I20260709 12:22:59.125381 16442 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:59.066447 18947 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:59.130965 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 311547us
W20260709 12:22:59.134598 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 57141.5us
I20260709 12:22:59.137959 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:59.112419 19000 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:59.130614 16476 consensus_queue.cc:260] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:59.142153 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:59.109805 19103 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:59.120568 19046 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:59.112138 18987 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:59.134020 18749 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:59.149827 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:59.150281 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:59.150543 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:22:59.153129 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 70869.1us
I20260709 12:22:59.164376 17142 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:59.157384 16476 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:59.157570 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 54275.3us
I20260709 12:22:59.105525 19078 consensus_queue.cc:1048] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 12:22:59.168916 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:59.170397 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:59.135723 19102 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:59.171612 16396 raft_consensus.cc:1240] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:59.144138 19000 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:59.143391 19086 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:59.132133 18698 consensus_queue.cc:1059] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:22:59.157989 19046 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:59.176215 17264 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:22:59.186616 17261 raft_consensus.cc:1240] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:59.186677 16442 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:59.189431 16367 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:59.190791 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:59.151044 19101 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:59.202996 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:59.203550 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:22:59.205235 19093 log.cc:927] Time spent T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:22:59.206724 19086 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:59.207494 19086 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:59.210449 19086 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:59.180450 18927 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:59.214847 18948 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:22:59.217362 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 77310.9us
I20260709 12:22:59.219256 16442 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:22:59.173056 18987 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:59.221055 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:59.222015 18987 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:59.222348 17270 raft_consensus.cc:2410] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:22:59.224613 19094 log.cc:927] Time spent T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a: Append to log took a long time: real 0.114s	user 0.002s	sys 0.000s
W20260709 12:22:59.232131 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:59.232728 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:59.232793 19098 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:59.233860 18927 raft_consensus.cc:697] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:22:59.237109 19097 log.cc:927] Time spent T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:22:59.237141 18698 raft_consensus.cc:3055] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:22:59.163713 19103 raft_consensus.cc:697] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 7 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
W20260709 12:22:59.231863 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 67596.9us
I20260709 12:22:59.240793 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:59.189071 19000 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:59.246950 19095 log.cc:927] Time spent T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:22:59.249084 17142 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:59.253856 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:59.256783 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:59.241890 17264 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
W20260709 12:22:59.264533 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 276883us
I20260709 12:22:59.264986 16549 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6.
I20260709 12:22:59.267059 16548 raft_consensus.cc:2393] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 7: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:59.238538 18698 raft_consensus.cc:740] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:59.270946 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:59.272109 17269 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:59.235138 18927 consensus_queue.cc:237] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:59.150846 18879 consensus_queue.cc:237] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:59.201090 19101 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:59.276082 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:59.289454 16549 raft_consensus.cc:1275] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:59.290398 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:59.279117 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:59.339493 17261 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:59.186980 18749 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:59.188432 19102 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:59.334439 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:22:59.344828 16546 raft_consensus.cc:2393] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 8: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:59.338153 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:59.343397 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:59.345198 16476 raft_consensus.cc:1275] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
W20260709 12:22:59.345582 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 96620.5us
I20260709 12:22:59.347488 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:59.348202 16547 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:22:59.345976 16548 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
W20260709 12:22:59.349632 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 130498us
W20260709 12:22:59.325964 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 54555.6us
I20260709 12:22:59.350666 16442 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:59.347064 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:22:59.352870 19092 log.cc:927] Time spent T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.271s	user 0.000s	sys 0.001s
I20260709 12:22:59.354179 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:59.343401 19101 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:59.356516 18581 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:59.357100 18581 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:59.357573 18581 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:59.359651 19102 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:59.363189 19080 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:59.363807 19080 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:59.364773 19107 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:59.364773 19111 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:59.365862 19100 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:22:59.366180 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 176876us
I20260709 12:22:59.365903 19111 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:59.366561 19100 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:59.367409 16367 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
W20260709 12:22:59.368448 19089 log.cc:927] Time spent T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.245s	user 0.000s	sys 0.001s
I20260709 12:22:59.368467 19111 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:59.369372 19115 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:59.368495 17255 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 3.
W20260709 12:22:59.370321 19091 log.cc:927] Time spent T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:22:59.371809 16475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 959) took 1276 ms. Trace:
I20260709 12:22:59.372224 16475 rpcz_store.cc:276] 0709 12:22:58.095712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.409048 (+313336us) service_pool.cc:224] Handling call
0709 12:22:58.409445 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:58.506491 (+ 97046us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:58.506536 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:58.513533 (+  6997us) log.cc:844] Serialized 26 byte log entry
0709 12:22:58.903808 (+390275us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:58.903863 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:58.904794 (+   931us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:59.371578 (+466784us) raft_consensus.cc:1685] finished
0709 12:22:59.371599 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:59.371748 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":205692,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":201,"thread_start_us":457700,"threads_started":2,"wal-append.queue_time_us":539682}
I20260709 12:22:59.334991 18948 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:59.375679 18948 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:59.378810 19084 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:59.307199 19086 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:22:59.380610 18728 raft_consensus.cc:3055] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:59.381042 19086 raft_consensus.cc:697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:59.381141 18728 raft_consensus.cc:740] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:22:59.381824 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:59.382186 19086 consensus_queue.cc:237] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:59.382941 16549 raft_consensus.cc:2393] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 7: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:22:59.369848 19115 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:22:59.384608 19115 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:59.385139 19106 log.cc:927] Time spent T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 12:22:59.386787 19051 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:59.387570 19051 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:59.389636 18698 consensus_queue.cc:260] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:59.394732 16547 raft_consensus.cc:1240] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:22:59.395371 19076 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:22:59.387588 19050 log.cc:927] Time spent T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.418s	user 0.005s	sys 0.000s
I20260709 12:22:59.402348 16367 leader_election.cc:400] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 8. Current term is 9.
I20260709 12:22:59.382186 18728 consensus_queue.cc:260] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:59.404367 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 58011.6us
I20260709 12:22:59.405860 19105 consensus_queue.cc:1059] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:22:59.405134 17142 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:59.409092 18812 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:59.410487 16402 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:22:59.347546 18679 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:59.411520 18698 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:59.346040 16476 pending_rounds.cc:85] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 4
I20260709 12:22:59.414769 16476 pending_rounds.cc:107] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:22:59.402828 16367 leader_election.cc:403] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:59.415066 16476 raft_consensus.cc:2889] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:59.424698 17269 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
W20260709 12:22:59.433753 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 81879.9us
I20260709 12:22:59.433920 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:59.435895 16473 raft_consensus.cc:2393] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 6: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:22:59.439615 17261 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:59.435025 16441 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:59.448130 16547 raft_consensus.cc:1240] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:59.432204 18812 raft_consensus.cc:2804] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:59.459200 16468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 461) took 1553 ms. Trace:
W20260709 12:22:59.431756 19089 log.cc:927] Time spent T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.062s	user 0.001s	sys 0.001s
I20260709 12:22:59.459802 16468 rpcz_store.cc:276] 0709 12:22:57.895505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.092894 (+197389us) service_pool.cc:224] Handling call
0709 12:22:58.093270 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:58.791981 (+698711us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:59.448059 (+656078us) spinlock_profiling.cc:243] Waited 640 ms on lock 0x7b540080b1d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:59.448273 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1471431040}
W20260709 12:22:59.463434 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 112887us
I20260709 12:22:59.442662 16514 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:22:59.462886 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:59.475991 16367 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:59.436834 18679 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 8. Current term is 9.
I20260709 12:22:59.442444 18013 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:59.462524 19076 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:59.489228 16441 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:22:59.491617 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:59.497465 16442 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:22:59.506006 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:22:59.508260 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 242685us
I20260709 12:22:59.477662 18675 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:59.494410 16441 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:22:59.490120 18991 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:59.490582 18702 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:59.372027 19103 consensus_queue.cc:237] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:59.525730 16549 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
I20260709 12:22:59.495944 19051 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:59.488477 19076 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:59.474699 18812 raft_consensus.cc:697] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:22:59.474604 19111 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:59.508168 18935 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:59.488341 19099 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:59.569988 16395 raft_consensus.cc:1275] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:59.546984 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:59.549129 16551 raft_consensus.cc:1275] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:22:59.552608 16403 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 11. Current term is 12. Ops: [10.7-11.8]
I20260709 12:22:59.553197 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:59.524924 18679 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:59.527407 19083 raft_consensus.cc:2764] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 LEADER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:22:59.510726 19100 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:59.566433 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:22:59.566496 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:59.575253 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:59.578922 16549 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 2.
I20260709 12:22:59.558696 19120 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:59.568145 16394 pending_rounds.cc:85] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 3
I20260709 12:22:59.543852 19113 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:59.581331 16394 pending_rounds.cc:107] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:22:59.581712 16394 raft_consensus.cc:2889] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:59.565223 19076 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:59.547951 18981 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:59.580220 16401 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:22:59.582885 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:22:59.576861 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 79510us
I20260709 12:22:59.597628 16442 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:22:59.548982 18991 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:59.605226 16516 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:59.608433 16396 raft_consensus.cc:2393] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 8: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:22:59.578949 18675 raft_consensus.cc:697] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:59.610394 16442 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:59.614341 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 77542.2us
I20260709 12:22:59.583772 19123 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:59.584410 19100 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:59.588531 18981 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:59.572842 19111 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:59.587344 19120 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:59.635157 19120 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:59.623615 18981 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:59.626179 18935 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:22:59.637260 19112 log.cc:927] Time spent T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 12:22:59.626358 19100 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:59.639572 19100 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:22:59.644878 16476 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:39000
I20260709 12:22:59.645776 16476 raft_consensus.cc:606] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:22:59.646234 16476 raft_consensus.cc:614] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:59.616462 19083 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:59.653644 16400 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:22:59.661758 18749 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:59.667518 18829 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:22:59.611161 19105 consensus_queue.cc:1059] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 12:22:59.597900 19113 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:59.610289 19114 consensus_queue.cc:1048] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 12:22:59.686251 16516 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:22:59.693588 16472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1013) took 1002 ms. Trace:
I20260709 12:22:59.623080 18861 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:59.680217 18829 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:22:59.701541 18749 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:59.740108 18749 raft_consensus.cc:697] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:22:59.740160 18948 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:59.739887 18581 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:59.741230 18749 consensus_queue.cc:237] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:59.742493 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:59.743116 16475 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:22:59.702164 18679 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:59.702929 18991 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:59.758427 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:59.749156 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:59.703069 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:22:59.703428 18675 consensus_queue.cc:237] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:59.696149 16472 rpcz_store.cc:276] 0709 12:22:58.691565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.804813 (+113248us) service_pool.cc:224] Handling call
0709 12:22:58.805143 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:58.815426 (+ 10283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:59.693236 (+877810us) spinlock_profiling.cc:243] Waited 9.37 ms on lock 0x7b540010efd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:22:59.693502 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21552384}
I20260709 12:22:59.704761 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:59.721519 16470 raft_consensus.cc:1240] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:59.768119 16367 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:59.721832 18981 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:59.722723 19111 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:22:59.723456 19099 raft_consensus.cc:3055] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:59.677935 18812 consensus_queue.cc:237] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:22:59.725762 19080 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:59.727905 18947 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:22:59.730301 16477 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:22:59.730516 18740 raft_consensus.cc:3055] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:59.732641 19113 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:22:59.735364 18861 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:59.767298 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 137731us
I20260709 12:22:59.833617 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:59.768887 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:59.834384 16397 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:22:59.785120 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:59.627588 19123 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:22:59.767804 19046 consensus_meta.cc:287] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.523s	user 0.010s	sys 0.002s
I20260709 12:22:59.835587 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:22:59.836741 16516 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:59.838056 16442 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:22:59.841400 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 296244us
I20260709 12:22:59.885316 17261 raft_consensus.cc:1275] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:22:59.885552 17144 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:59.771000 18679 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:22:59.889335 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 51455.4us
I20260709 12:22:59.892167 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:59.893657 17269 raft_consensus.cc:2410] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 12:22:59.888546 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 462217us
I20260709 12:22:59.895604 17261 raft_consensus.cc:1275] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:59.898664 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:59.737866 19000 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:59.900349 16468 raft_consensus.cc:1240] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:59.785307 19099 raft_consensus.cc:740] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:59.838639 18861 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:59.780702 19107 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:59.907948 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:59.773002 19078 consensus_queue.cc:1048] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.135s
I20260709 12:22:59.909802 17271 raft_consensus.cc:1275] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:22:59.910765 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:22:59.912595 17264 raft_consensus.cc:1275] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:22:59.841104 18740 raft_consensus.cc:740] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:22:59.914285 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:59.778116 19051 consensus_queue.cc:1059] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.123s
I20260709 12:22:59.887302 18947 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:59.786453 18991 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:59.887210 18809 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:59.929932 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:59.931219 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:59.931600 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:59.838907 19122 consensus_queue.cc:1048] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.082s
W20260709 12:22:59.937966 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 163445us
I20260709 12:22:59.939649 17269 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
I20260709 12:22:59.907008 19129 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:59.942540 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:59.945762 16549 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:22:59.944434 16367 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:22:59.951472 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:59.952033 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:59.952616 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:22:59.789103 19115 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:22:59.952107 16543 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:22:59.919180 19107 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:59.955456 19107 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:59.952872 17261 raft_consensus.cc:2468] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:22:59.868690 19046 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:59.911723 19084 consensus_queue.cc:1048] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:22:59.978480 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:22:59.938375 18947 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:22:59.982920 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:22:59.985724 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 99118.1us
I20260709 12:22:59.983592 17258 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
W20260709 12:23:00.009461 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 76491.2us
I20260709 12:23:00.010128 16469 raft_consensus.cc:1275] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:22:59.974567 19131 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:59.962209 18809 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:22:59.954317 18991 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:00.050976 19101 consensus_meta.cc:287] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.695s	user 0.009s	sys 0.004s
I20260709 12:23:00.051241 18581 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:00.051566 19101 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:00.026186 18676 consensus_meta.cc:287] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.941s	user 0.015s	sys 0.000s
I20260709 12:22:59.987798 18947 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:59.985095 19051 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:00.024251 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:00.052024 19140 consensus_queue.cc:1048] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 12:23:00.069669 17254 raft_consensus.cc:1240] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 6. Current term is 7. Ops: []
W20260709 12:23:00.070756 19139 log.cc:927] Time spent T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:23:00.073208 19132 log.cc:927] Time spent T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:23:00.073303 17266 raft_consensus.cc:2368] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 8. Current term is 9.
I20260709 12:23:00.074965 19101 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:00.090137 18676 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:00.091078 19076 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:00.107424 16471 raft_consensus.cc:1240] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:59.974372 19099 consensus_queue.cc:260] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:00.108781 19099 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:00.112798 19123 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:00.120548 19126 log.cc:927] Time spent T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 12:23:00.075399 19131 raft_consensus.cc:697] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 10 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:00.121469 19046 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:00.122586 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:00.122195 19131 consensus_queue.cc:237] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:00.123266 16550 raft_consensus.cc:2468] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
W20260709 12:23:00.124464 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:00.125388 16544 raft_consensus.cc:2393] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 9: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:00.051337 19125 consensus_queue.cc:1059] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.111s
I20260709 12:23:00.089051 19051 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:00.128998 19051 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:00.134130 19100 raft_consensus.cc:3055] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:00.134737 19100 raft_consensus.cc:740] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:00.011943 17143 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:23:00.054206 18698 consensus_meta.cc:287] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.642s	user 0.008s	sys 0.004s
W20260709 12:23:00.080585 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 128609us
I20260709 12:23:00.089103 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:00.123318 18809 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:00.007350 19114 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:00.142344 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 130465us
I20260709 12:23:00.141753 19080 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:00.142927 19080 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:00.142642 19114 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:00.143285 19080 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:00.144618 16475 raft_consensus.cc:1275] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:23:00.144928 19114 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:00.137900 18981 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:00.145759 18829 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:00.145861 18809 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:00.146528 18809 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:00.148141 19123 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:00.148594 18809 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:00.148737 19123 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:00.151237 19123 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:00.148777 19098 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:00.152768 19098 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:00.154541 18679 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:00.155066 19098 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:00.155983 19000 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:00.157079 19108 consensus_queue.cc:1175] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ef57beab0e1d40d585983873c55107e8" responder_term: 6 status { last_received { term: 6 index: 4 } last_committed_idx: 4 last_received_current_leader { term: 6 index: 4 } }
I20260709 12:23:00.094897 19122 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.158860 19122 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:00.161231 19122 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:00.135895 19100 consensus_queue.cc:260] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:00.149179 19115 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:00.189025 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:00.189500 17143 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:00.191354 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:00.193176 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:00.193785 16395 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:23:00.194013 18861 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:00.195066 16551 raft_consensus.cc:1240] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:23:00.194813 18740 consensus_queue.cc:260] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:00.193034 19115 consensus_queue.cc:1048] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:00.197242 16367 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:00.204576 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:00.171674 19107 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:00.206786 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:00.207643 16396 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:00.207513 16395 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:00.208969 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:00.209069 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:00.212383 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 315814us
I20260709 12:23:00.216650 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:00.220129 17143 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:00.222956 16400 raft_consensus.cc:1240] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:00.209983 19000 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:00.206017 18740 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:00.238317 17144 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:00.241117 16403 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:00.205717 18961 consensus_queue.cc:1048] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:23:00.235070 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:00.237246 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:00.242053 16468 raft_consensus.cc:2368] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 10. Current term is 11.
I20260709 12:23:00.242347 16472 raft_consensus.cc:2393] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 4: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
W20260709 12:23:00.247098 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:00.247356 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:00.248010 16469 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:00.248281 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:00.259054 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:00.258931 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:00.259807 16395 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:00.238063 19140 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:00.252759 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:00.262198 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:00.234333 19122 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:00.233830 19000 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:00.264813 16471 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:23:00.270253 16475 raft_consensus.cc:1275] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:00.266600 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:00.265599 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:00.271751 16472 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:23:00.264125 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 114668us
W20260709 12:23:00.266719 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 44149.7us
I20260709 12:23:00.275038 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:00.275713 17261 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:23:00.277995 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:00.278746 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:00.279984 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:00.280514 19101 consensus_queue.cc:1059] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:00.280965 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:00.281520 17143 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:00.284067 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:00.284505 16396 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:23:00.253466 18698 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:00.268730 18809 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:00.289426 16515 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:00.295501 19114 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:00.295822 19151 raft_consensus.cc:3055] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:00.295956 19114 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:00.270177 19123 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:00.296289 19151 raft_consensus.cc:740] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:00.296545 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:00.285257 19051 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:00.296448 19114 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:00.300349 17143 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:00.273001 19000 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:00.304104 18698 raft_consensus.cc:697] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:00.309649 19081 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:00.297315 19151 consensus_queue.cc:260] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:00.313099 16469 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:00.301920 16515 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:00.300031 18676 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:00.330482 18675 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:00.331020 18675 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:00.299265 18809 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:00.314568 18981 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:00.332263 18981 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:00.333534 18809 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:00.331434 18675 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:00.334616 18981 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:00.338515 18728 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:23:00.340950 19138 log.cc:927] Time spent T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.272s	user 0.000s	sys 0.005s
I20260709 12:23:00.315186 16515 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:00.304517 16395 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
I20260709 12:23:00.310936 19081 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:00.354418 18676 raft_consensus.cc:2804] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:00.314374 19150 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:00.361340 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:00.362442 16472 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:00.308925 19140 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:23:00.373498 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 80795.2us
I20260709 12:23:00.300997 18947 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:00.296669 19123 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:00.370333 18675 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:00.372776 18809 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:00.363278 18698 consensus_queue.cc:237] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:00.379827 18981 consensus_queue.cc:1059] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:00.381510 16442 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:00.382603 16367 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.1:41805 timed out after 1.528s (SENT) [suppressed 7 similar messages]
I20260709 12:23:00.373299 19140 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:00.387610 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:00.401316 17261 raft_consensus.cc:1240] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:00.381886 18676 raft_consensus.cc:697] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:00.399047 19140 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:00.417562 16515 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:00.373394 19150 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:00.387985 18861 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:00.421272 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:00.422664 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:00.424556 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:00.395012 19151 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:00.421526 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:00.428906 17270 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:00.430549 16442 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:00.457799 16544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52680 (request call id 456) took 1180 ms. Trace:
I20260709 12:23:00.458324 16544 rpcz_store.cc:276] 0709 12:22:59.277205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:59.278482 (+  1277us) service_pool.cc:224] Handling call
0709 12:23:00.457577 (+1179095us) spinlock_profiling.cc:243] Waited 773 ms on lock 0x7b54008c5ad4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:00.457731 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1777232768}
I20260709 12:23:00.409260 18879 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:00.477031 18981 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:00.481652 18981 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:00.393842 18809 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:00.428143 18812 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:00.425557 19150 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:23:00.493525 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 110940us
I20260709 12:23:00.506407 16367 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:23:00.510105 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:00.510490 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:00.511598 17271 raft_consensus.cc:2368] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 8. Current term is 9.
I20260709 12:23:00.512844 16397 raft_consensus.cc:1240] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:00.475204 18066 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:00.478773 18702 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:23:00.517760 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 97920us
I20260709 12:23:00.485666 19051 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:00.516769 18066 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:00.524667 18066 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:00.522437 16515 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:00.517480 17254 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:23:00.522691 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:00.514310 18981 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:00.514120 18812 raft_consensus.cc:697] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:00.535907 16477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53186 (request call id 471) took 1909 ms. Trace:
I20260709 12:23:00.522127 16367 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:00.534368 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:00.536697 16477 rpcz_store.cc:276] 0709 12:22:58.624651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.628276 (+  3625us) service_pool.cc:224] Handling call
0709 12:23:00.533406 (+1905130us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400594f54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:00.533503 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2457378688}
I20260709 12:23:00.520005 19115 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:23:00.545120 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:00.504824 19080 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:00.547284 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:00.494773 18879 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:00.527503 18702 raft_consensus.cc:697] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:00.512701 19122 consensus_queue.cc:1048] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 12:23:00.549849 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:00.550340 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:00.553429 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:00.556653 19115 consensus_queue.cc:1059] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:00.525969 18961 consensus_queue.cc:1048] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
W20260709 12:23:00.560572 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 104205us
I20260709 12:23:00.563189 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:00.563014 18581 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:00.572257 18581 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:00.572899 19103 raft_consensus.cc:3055] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:00.573467 19103 raft_consensus.cc:740] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:00.574627 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:00.574488 19103 consensus_queue.cc:260] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:00.576702 18661 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:00.575251 17266 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:23:00.556681 19163 log.cc:927] Time spent T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 12:23:00.550050 18812 consensus_queue.cc:237] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:00.578416 19107 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:00.550086 18675 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:00.531495 18927 consensus_queue.cc:1059] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
W20260709 12:23:00.595291 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 46550.9us
I20260709 12:23:00.550452 19051 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:00.598361 17270 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
W20260709 12:23:00.560123 19160 log.cc:927] Time spent T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260709 12:23:00.598498 16515 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:00.611357 16367 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:00.524549 18676 consensus_queue.cc:237] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:00.614327 16366 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
W20260709 12:23:00.615659 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 52800.4us
I20260709 12:23:00.608573 18066 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:00.618186 16366 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:00.620379 16542 raft_consensus.cc:1240] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:00.622934 18991 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:00.624617 19086 raft_consensus.cc:3055] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:00.625003 19086 raft_consensus.cc:740] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:00.625787 19086 consensus_queue.cc:260] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:00.627146 17266 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6.
I20260709 12:23:00.627836 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:00.631120 17263 raft_consensus.cc:1240] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:23:00.633671 19114 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:00.637247 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:00.637463 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:00.617944 16514 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:00.605976 19103 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:00.640906 17142 leader_election.cc:400] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 10. Current term is 11.
I20260709 12:23:00.645555 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:00.605641 19051 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:00.605130 18403 raft_consensus.cc:3055] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:00.648372 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:00.649827 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:00.652156 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:00.646930 17142 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:00.654848 16394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 884) took 1107 ms. Trace:
I20260709 12:23:00.616848 19046 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.657445 16397 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:00.636142 19114 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:00.659299 19165 log.cc:927] Time spent T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 12:23:00.639326 19147 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:00.642097 19117 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.667233 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:00.622990 19115 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:00.670193 18702 consensus_queue.cc:237] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:00.671898 19080 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:00.644874 19086 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:00.659109 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:00.677999 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 60098.2us
I20260709 12:23:00.608278 19107 consensus_queue.cc:1059] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 12:23:00.642949 19120 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:00.639446 18987 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:00.656556 18066 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:00.656090 18403 raft_consensus.cc:740] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:00.706459 17143 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:00.760610 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 54216.3us
I20260709 12:23:00.705612 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:00.721299 18927 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:00.676719 19046 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:00.772917 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:00.675797 19150 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:00.712085 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:23:00.695319 19157 log.cc:927] Time spent T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.277s	user 0.001s	sys 0.000s
I20260709 12:23:00.754153 19168 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:00.655635 19051 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:23:00.728749 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:00.729527 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 82690us
I20260709 12:23:00.678689 19081 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:00.696884 18879 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:00.721299 18675 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:00.666891 19078 consensus_queue.cc:1059] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
W20260709 12:23:00.730314 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 131904us
I20260709 12:23:00.737190 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:00.761699 16399 raft_consensus.cc:2393] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 9: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:00.655311 16394 rpcz_store.cc:276] 0709 12:22:59.547305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:59.547440 (+   135us) service_pool.cc:224] Handling call
0709 12:22:59.567878 (+ 20438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:59.602983 (+ 35105us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:22:59.918396 (+315413us) log.cc:844] Serialized 14 byte log entry
0709 12:23:00.206677 (+288281us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:00.211470 (+  4793us) log.cc:844] Serialized 26 byte log entry
0709 12:23:00.211601 (+   131us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:00.211813 (+   212us) log.cc:844] Serialized 14 byte log entry
0709 12:23:00.222914 (+ 11101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:00.223001 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:00.335052 (+112051us) spinlock_profiling.cc:243] Waited 6.85 ms on lock 0x7b180016a560. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d16c0a 00007f287c456219 00007f2879d5d58f 00007f2872d9c571 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd
0709 12:23:00.654600 (+319548us) raft_consensus.cc:1685] finished
0709 12:23:00.654631 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:00.654786 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4192,"prepare.queue_time_us":339511,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":112,"raft.queue_time_us":12564,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":9110,"spinlock_wait_cycles":47160704,"thread_start_us":548924,"threads_started":2,"wal-append.queue_time_us":339142}
I20260709 12:23:00.806532 16515 leader_election.cc:400] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 8. Current term is 9.
I20260709 12:23:00.763538 16395 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:23:00.764868 18066 consensus_queue.cc:1048] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:23:00.805965 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 47804us
I20260709 12:23:00.778980 16551 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:23:00.765892 18698 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:00.774506 18581 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:00.809823 16401 raft_consensus.cc:2393] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 9: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:00.810027 18066 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.800897 16550 raft_consensus.cc:1275] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:00.729125 19153 consensus_queue.cc:1048] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 12:23:00.801455 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:00.721625 19117 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:00.803853 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:00.804055 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:00.729589 19156 consensus_queue.cc:1059] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 12:23:00.761703 17143 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:00.807569 16551 raft_consensus.cc:606] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:00.808810 16544 raft_consensus.cc:1240] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:23:00.813208 16546 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:23:00.804780 18675 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:00.811764 18879 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.798449 17142 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:00.814965 18879 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:00.816140 17142 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:00.817118 18879 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:00.817255 17142 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:00.811295 16400 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:23:00.818568 19080 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:00.818614 17142 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:00.821384 19131 raft_consensus.cc:3055] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:00.821900 19131 raft_consensus.cc:740] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:00.822882 19131 consensus_queue.cc:260] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:00.802264 19168 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:00.826167 19168 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:00.826411 18947 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:00.829730 16543 raft_consensus.cc:2393] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 7: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:00.789402 19079 consensus_queue.cc:1059] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 12:23:00.832607 19080 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:00.834249 18740 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:00.835893 19130 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:00.814178 18675 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:00.857492 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 95901.2us
I20260709 12:23:00.860272 17142 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:00.861284 18740 consensus_queue.cc:1048] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:00.862576 19076 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:00.862686 18698 consensus_queue.cc:1059] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:00.863716 19150 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:00.863792 18698 raft_consensus.cc:3055] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:00.864355 18698 raft_consensus.cc:740] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:00.865300 18698 consensus_queue.cc:260] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:00.866386 16441 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:00.867204 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:00.867730 16441 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:00.867970 18581 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:00.868999 18581 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:00.869386 18581 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:00.869336 19098 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:00.870240 19098 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:00.868410 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:00.859155 19125 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:00.873715 19125 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:00.874126 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:00.874753 16547 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:00.877250 18066 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:00.880867 16441 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:00.866529 18698 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:00.884979 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:00.886083 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:00.887725 16441 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:00.888298 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:00.888676 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:00.891302 18991 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:00.891436 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:00.888936 18740 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:00.892057 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:00.893018 16477 raft_consensus.cc:1240] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 6. Current term is 7. Ops: []
W20260709 12:23:00.869311 19177 log.cc:927] Time spent T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:23:00.898363 16470 raft_consensus.cc:1240] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:23:00.894011 16472 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:00.898777 16441 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:00.905396 16441 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:00.905795 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:00.876088 19125 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:00.912276 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:00.918058 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:00.918537 16470 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:23:00.921152 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:00.921043 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:00.920855 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:00.876711 19098 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:00.885571 19111 raft_consensus.cc:3055] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:00.924770 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:00.925732 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:00.926371 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:00.928614 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:00.917060 19084 raft_consensus.cc:2804] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:23:00.910884 18991 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:00.894448 19076 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:00.929502 18991 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:00.930655 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:00.898551 19131 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:00.942929 16470 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:00.929893 18991 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:00.947311 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:00.910517 19123 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:00.911363 18740 raft_consensus.cc:697] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:00.951977 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:00.922313 18981 raft_consensus.cc:2804] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:00.961124 18981 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:00.961680 18981 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:23:00.966037 19182 log.cc:927] Time spent T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 12:23:00.967936 19078 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.968598 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:23:00.968931 19000 consensus_meta.cc:287] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.656s	user 0.009s	sys 0.007s
I20260709 12:23:00.969163 19117 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:00.969347 19000 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:00.973634 19170 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.974210 19170 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:00.935657 19099 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:00.977224 19099 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:00.982065 18740 consensus_queue.cc:237] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:00.954026 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:00.985471 17263 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:00.931787 19111 raft_consensus.cc:740] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:00.956557 16472 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:23:00.924028 19046 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:00.957479 16471 raft_consensus.cc:1240] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 LEADER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:00.958076 19175 consensus_queue.cc:1059] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:00.954840 18676 consensus_queue.cc:1059] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 12:23:00.987437 17266 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:23:01.004148 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 92030.8us
I20260709 12:23:01.006263 16441 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:00.993758 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:01.009269 19083 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:01.012488 16441 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:01.007364 19190 consensus_queue.cc:1059] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:01.014034 19083 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:01.014786 19083 raft_consensus.cc:697] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:01.016112 19083 consensus_queue.cc:237] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:01.022904 18702 raft_consensus.cc:3055] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:01.023583 18702 raft_consensus.cc:740] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:01.025194 18702 consensus_queue.cc:260] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:01.026681 18702 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:01.033560 16441 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:01.040548 19190 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:01.041394 19190 raft_consensus.cc:697] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:01.042896 19190 consensus_queue.cc:237] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:01.048441 16441 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:23:00.990444 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:01.053889 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:01.057121 18702 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:01.060866 16441 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:01.063601 18702 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:01.066855 18702 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:01.067765 18702 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:01.065964 16441 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:01.073856 19193 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:23:01.074829 19193 raft_consensus.cc:697] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 6 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:01.078275 19193 consensus_queue.cc:237] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:01.092855 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:01.093953 16468 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:01.096089 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:01.105345 16468 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
I20260709 12:23:01.106757 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:01.109262 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:00.950222 19140 consensus_meta.cc:287] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.536s	user 0.011s	sys 0.002s
I20260709 12:23:00.979959 19099 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:01.123325 16468 raft_consensus.cc:1275] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:01.110395 18702 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:01.121193 16542 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:23:00.955367 18581 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:01.125211 16474 raft_consensus.cc:1275] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:23:01.125900 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 132423us
I20260709 12:23:01.129896 17266 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:23:01.186896 16476 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:23:01.129958 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:00.949631 19084 raft_consensus.cc:697] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:01.140772 19000 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:01.146653 16468 raft_consensus.cc:1275] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:23:01.125119 18676 consensus_queue.cc:1243] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer 52ab50b0237346c1a13ede947f6a63c2 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:23:01.135667 17263 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:23:01.192914 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:01.193969 17263 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:23:00.968658 19078 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:01.146118 17142 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:01.117731 19105 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.203766 18661 consensus_queue.cc:1048] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 12:23:01.204191 19105 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:01.205305 19100 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:01.153312 19170 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:01.210035 18698 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:01.185885 19076 raft_consensus.cc:697] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:01.210824 18698 raft_consensus.cc:697] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:01.211717 19076 consensus_queue.cc:237] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:01.214293 19194 log.cc:927] Time spent T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:23:01.124276 19140 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:01.216732 17271 raft_consensus.cc:2393] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 7: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:01.219931 19078 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:01.155393 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:01.188676 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:01.158425 18675 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:01.183297 16547 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:01.197448 17269 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:23:01.198791 19084 consensus_queue.cc:237] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:01.160657 19186 log.cc:927] Time spent T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 12:23:01.202096 16469 raft_consensus.cc:1240] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260709 12:23:01.203018 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 56933.1us
I20260709 12:23:01.228863 16550 raft_consensus.cc:2368] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 6. Current term is 7.
I20260709 12:23:01.235005 17271 raft_consensus.cc:1240] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:01.236052 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:01.243065 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:01.244798 16545 raft_consensus.cc:1240] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:01.212191 18698 consensus_queue.cc:237] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:01.259895 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:01.154227 19111 consensus_queue.cc:260] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:01.265643 16550 raft_consensus.cc:2393] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 7: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:01.268415 17263 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 12:23:01.268852 17263 pending_rounds.cc:85] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8: Aborting all ops after (but not including) 7
I20260709 12:23:01.269109 17263 pending_rounds.cc:107] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 12:23:01.269317 17263 raft_consensus.cc:2889] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:01.267132 16514 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
W20260709 12:23:01.275787 19187 log.cc:927] Time spent T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:23:01.278198 19086 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:01.278966 19086 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:01.287075 19111 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:01.272660 18812 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.297238 18812 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:01.285245 19140 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:01.301460 17258 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:23:01.301509 19051 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:01.302891 19120 raft_consensus.cc:3055] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:01.304097 19120 raft_consensus.cc:740] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:01.305162 19120 consensus_queue.cc:260] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:01.306233 19120 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:23:01.306849 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:01.307605 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:01.308589 19122 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:01.309237 19122 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:01.310571 18728 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:01.311113 18728 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:01.312134 19181 log.cc:927] Time spent T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.487s	user 0.000s	sys 0.001s
I20260709 12:23:01.313122 18728 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:01.318965 18947 raft_consensus.cc:3055] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:01.319453 18947 raft_consensus.cc:740] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:01.319756 18987 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.320268 18987 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:01.320739 18675 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:01.319801 19122 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:01.322417 19105 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:01.323177 18987 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:01.323982 19051 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:01.325659 18981 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:01.280274 18749 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:01.330302 18749 raft_consensus.cc:697] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:01.330361 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:01.331506 18749 consensus_queue.cc:237] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:01.334153 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:01.334718 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:01.334813 16396 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:23:01.335383 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:01.335505 16394 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:23:01.336162 16396 raft_consensus.cc:1275] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:23:01.336598 18702 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:01.351495 18987 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:01.351750 18991 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:01.352171 18987 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:01.354264 18987 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:01.354934 16475 raft_consensus.cc:2368] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 4. Current term is 5.
I20260709 12:23:01.355510 19154 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:01.359414 16401 raft_consensus.cc:1240] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:01.360852 18581 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:01.353919 18991 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:01.348686 18812 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:01.363700 16515 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
W20260709 12:23:01.381637 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:01.383031 16515 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:01.399750 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:01.338887 18947 consensus_queue.cc:260] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:01.381102 18675 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:01.364653 19154 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:01.401836 18540 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:01.407119 16547 raft_consensus.cc:1275] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:23:01.408156 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:01.414573 18981 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:01.414624 19079 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:01.417578 17144 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:01.427591 18403 consensus_queue.cc:260] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:01.419188 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:01.420635 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:01.421000 16516 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:01.422070 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:01.422593 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:01.425910 18991 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:01.451485 16516 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:01.453107 19183 log.cc:927] Time spent T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.621s	user 0.004s	sys 0.000s
I20260709 12:23:01.453598 18987 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:01.453441 19079 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:01.454698 19150 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:01.454348 18987 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:01.427121 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:01.426270 16442 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:01.427682 18809 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:01.432150 17270 raft_consensus.cc:1240] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:01.430056 16544 raft_consensus.cc:1240] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:01.418036 19100 consensus_queue.cc:1059] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:01.448596 17144 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:01.453050 16516 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:01.433653 16367 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:01.432404 18675 raft_consensus.cc:697] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:01.471493 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:01.475513 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 49319.8us
I20260709 12:23:01.475682 17258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45228 (request call id 786) took 1201 ms. Trace:
I20260709 12:23:01.468941 16473 raft_consensus.cc:1240] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:01.474728 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:01.468240 16477 raft_consensus.cc:1240] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:23:01.476106 17258 rpcz_store.cc:276] 0709 12:23:00.273828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:00.273959 (+   131us) service_pool.cc:224] Handling call
0709 12:23:01.475523 (+1201564us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54002800d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:01.475625 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2347376384}
I20260709 12:23:01.466758 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:01.461341 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:01.473887 17255 raft_consensus.cc:1240] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:23:01.472864 16545 raft_consensus.cc:1275] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:23:01.481014 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:01.477124 16550 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
I20260709 12:23:01.475697 18675 consensus_queue.cc:237] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:01.485980 18981 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:01.485471 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:01.487440 16516 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:01.488663 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:01.489704 17255 raft_consensus.cc:2410] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 13 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 12:23:01.498142 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 49637us
I20260709 12:23:01.434329 19051 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.442971 18540 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:01.499528 17144 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:01.499995 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 66425.9us
I20260709 12:23:01.504935 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:01.506412 16367 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:01.456154 19086 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:01.470912 18740 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.511615 16367 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:01.456233 19079 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:01.527217 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:01.529063 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:01.500515 18987 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:01.531749 16397 raft_consensus.cc:1275] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:23:01.541029 16400 raft_consensus.cc:1275] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:01.506989 19080 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:01.509316 19150 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:01.510064 18540 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:01.508481 19108 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:01.591372 16397 raft_consensus.cc:1275] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:01.509496 19184 raft_consensus.cc:2804] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:01.594009 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:01.548240 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:01.517213 18910 consensus_queue.cc:1048] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:23:01.509172 18809 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:01.595850 18809 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:01.596230 18809 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:01.508733 18991 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:01.601202 19079 consensus_queue.cc:1048] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:01.603529 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:01.600414 18991 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:01.604156 16549 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:01.605671 17261 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 787) took 1205 ms. Trace:
I20260709 12:23:01.607175 16441 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:01.594233 19184 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:01.538353 19120 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:01.537470 18403 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:01.606071 17261 rpcz_store.cc:276] 0709 12:23:00.400688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:00.411590 (+ 10902us) service_pool.cc:224] Handling call
0709 12:23:00.412043 (+   453us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:00.423938 (+ 11895us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:00.423962 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:01.357381 (+933419us) spinlock_profiling.cc:243] Waited 3.15 ms on lock 0x7b2c00000708. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:23:01.357757 (+   376us) log.cc:844] Serialized 26 byte log entry
0709 12:23:01.494935 (+137178us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:01.494970 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:01.495034 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:01.605349 (+110315us) raft_consensus.cc:1685] finished
0709 12:23:01.605370 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:01.605607 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":96,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":7797,"spinlock_wait_cycles":8232448,"thread_start_us":119116,"threads_started":1,"wal-append.queue_time_us":234962}
I20260709 12:23:01.531569 19122 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:01.612926 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:01.601259 16367 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:01.641409 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:01.545944 18861 raft_consensus.cc:3055] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:01.520164 18812 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 12:23:01.518705 18879 consensus_queue.cc:1059] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 12:23:01.649299 16367 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:01.636713 16441 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:01.531554 19151 consensus_queue.cc:1059] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.083s
I20260709 12:23:01.520017 19076 consensus_queue.cc:1059] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.096s
I20260709 12:23:01.655961 19111 consensus_queue.cc:1059] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 12:23:01.663748 19079 raft_consensus.cc:3055] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:01.664527 19079 raft_consensus.cc:740] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:01.665838 19079 consensus_queue.cc:260] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:01.667943 19079 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:01.651219 18961 raft_consensus.cc:2804] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:01.681221 18961 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:01.681856 18961 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:23:01.686380 19204 log.cc:927] Time spent T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:23:01.614794 19084 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.510319 19051 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:01.633477 19184 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:01.641842 19190 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:01.634267 19120 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:23:01.699769 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 50661.8us
I20260709 12:23:01.705267 16367 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:01.550995 19150 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:01.644930 19108 raft_consensus.cc:697] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 7 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:01.645848 18861 raft_consensus.cc:740] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:01.714205 16442 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:01.714522 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:01.718111 16549 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:23:01.694519 18581 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:01.732569 18581 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:01.733042 18581 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:01.733114 16401 raft_consensus.cc:1275] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:23:01.734022 18702 raft_consensus.cc:2749] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:01.737669 18809 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:01.738653 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 302666us
I20260709 12:23:01.742767 18991 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:01.744237 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:01.745194 19080 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:01.745695 19080 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:01.701486 19084 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:01.746696 17264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 792) took 1127 ms. Trace:
I20260709 12:23:01.748200 19084 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:01.747087 17264 rpcz_store.cc:276] 0709 12:23:00.619444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:00.630225 (+ 10781us) service_pool.cc:224] Handling call
0709 12:23:00.630555 (+   330us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:00.694981 (+ 64426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:00.695021 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:01.321840 (+626819us) spinlock_profiling.cc:243] Waited 6.94 ms on lock 0x7b2c00000708. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:23:01.322374 (+   534us) log.cc:844] Serialized 52 byte log entry
0709 12:23:01.497440 (+175066us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:01.518233 (+ 20793us) log.cc:844] Serialized 14 byte log entry
0709 12:23:01.518460 (+   227us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:01.518584 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:01.746423 (+227839us) raft_consensus.cc:1685] finished
0709 12:23:01.746513 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:01.746642 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24227,"prepare.queue_time_us":22863,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":58,"raft.queue_time_us":41194,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"spinlock_wait_cycles":44752896,"thread_start_us":164773,"threads_started":1,"wal-append.queue_time_us":333256}
I20260709 12:23:01.740139 16366 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:01.655229 19191 consensus_queue.cc:1048] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 12:23:01.694630 19192 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:23:01.753162 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:01.737506 18861 consensus_queue.cc:260] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:01.539903 18740 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:01.722098 19140 raft_consensus.cc:3055] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:01.711614 19120 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:01.717839 19079 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:01.716488 19150 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:01.704049 19122 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:23:01.802996 18066 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:01.818562 17271 raft_consensus.cc:1240] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:01.771872 19078 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:01.826828 16441 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:01.784025 19140 raft_consensus.cc:740] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:01.843962  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:23:01.773715 18728 consensus_queue.cc:1048] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
W20260709 12:23:01.848459 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 134336us
I20260709 12:23:01.821532 18861 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:01.771706 19192 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:01.829329 19086 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.829180 18809 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:01.869531 16544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1172) took 1069 ms. Trace:
I20260709 12:23:01.850054 18981 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:01.875306 16544 rpcz_store.cc:276] 0709 12:23:00.799844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:00.814949 (+ 15105us) service_pool.cc:224] Handling call
0709 12:23:01.429746 (+614797us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:01.487039 (+ 57293us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:01.869238 (+382199us) spinlock_profiling.cc:243] Waited 614 ms on lock 0x7b54007b52d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:01.869440 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1413195136}
I20260709 12:23:01.877197 16442 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:01.857224 19213 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:01.877116 19086 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:01.879012 18702 consensus_queue.cc:1048] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
W20260709 12:23:01.887106 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:01.889632 19213 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:01.904922 19208 log.cc:927] Time spent T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:23:01.909543 18702 consensus_queue.cc:1048] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:23:01.910785 19207 log.cc:927] Time spent T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:23:01.918850 19080 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:01.924578 18702 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:01.926594 19080 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:01.937919 19154 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:01.896230 19140 consensus_queue.cc:260] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:01.942620 19140 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:01.942479 19120 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:01.944715 19120 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:01.946776 18947 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:23:01.961396 19210 log.cc:927] Time spent T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 12:23:01.962466 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:01.964476 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 493458us
I20260709 12:23:01.965499 17143 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:01.970351 19000 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:01.974560 17254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 796) took 1036 ms. Trace:
I20260709 12:23:01.976962 17270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 798) took 1038 ms. Trace:
I20260709 12:23:01.977448 17270 rpcz_store.cc:276] 0709 12:23:00.938847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:00.938979 (+   132us) service_pool.cc:224] Handling call
0709 12:23:00.951536 (+ 12557us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:01.477635 (+526099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:01.976660 (+499025us) spinlock_profiling.cc:243] Waited 480 ms on lock 0x7b540055f854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:01.976891 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1104788224}
W20260709 12:23:01.981158 19211 log.cc:927] Time spent T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:23:01.985080 16476 raft_consensus.cc:1240] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:01.990799 18740 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:01.938818 16395 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58430 (request call id 642) took 1592 ms (client timeout 1739 ms). Trace:
I20260709 12:23:01.996186 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:23:01.996498 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 199399us
I20260709 12:23:02.001466 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:01.967000 17143 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:02.006428 16549 raft_consensus.cc:1275] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:01.855049 19108 consensus_queue.cc:237] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:02.013341 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 253870us
W20260709 12:23:01.992919 16395 rpcz_store.cc:269] 0709 12:23:00.346166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:00.361036 (+ 14870us) service_pool.cc:224] Handling call
0709 12:23:01.938624 (+1577588us) spinlock_profiling.cc:243] Waited 391 ms on lock 0x7b540011f154. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:01.938751 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":900008192}
W20260709 12:23:02.017009 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 139885us
I20260709 12:23:02.018553 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:02.018693 16366 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:02.020012 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:02.021080 16472 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:23:02.002416 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:01.879196 19084 consensus_queue.cc:1048] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
W20260709 12:23:02.026254 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 217232us
W20260709 12:23:02.028594 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 61761.5us
I20260709 12:23:02.029938 17143 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:02.032299 16366 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:02.034366 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:02.034777 16396 raft_consensus.cc:1240] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:01.975174 17254 rpcz_store.cc:276] 0709 12:23:00.937847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:00.938144 (+   297us) service_pool.cc:224] Handling call
0709 12:23:00.955050 (+ 16906us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:00.987942 (+ 32892us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:23:01.355854 (+367912us) log.cc:844] Serialized 14 byte log entry
0709 12:23:01.536739 (+180885us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:01.536838 (+    99us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:01.537003 (+   165us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:01.974247 (+437244us) spinlock_profiling.cc:243] Waited 32.6 ms on lock 0x7b54002e95d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:01.974311 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:01.974481 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":384933,"prepare.queue_time_us":368703,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":23759,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":24705,"spinlock_wait_cycles":75040000,"thread_start_us":161155,"threads_started":1,"wal-append.queue_time_us":363838}
I20260709 12:23:02.035665 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:02.036772 16402 raft_consensus.cc:2410] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:23:02.037598 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:02.038335 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:02.039902 18066 consensus_queue.cc:1059] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:02.039013 16394 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:23:02.043114 16394 raft_consensus.cc:1240] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:02.023001 18809 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:02.035211 19212 log.cc:927] Time spent T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.146s	user 0.004s	sys 0.000s
I20260709 12:23:02.044646 16514 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:02.054127 18066 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:02.054723 18066 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:02.055253 18066 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:02.043995 18581 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:02.060314 17143 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:02.038014 18740 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:02.065024 17143 leader_election.cc:400] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 6. Current term is 7.
I20260709 12:23:02.083421 17143 leader_election.cc:403] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:23:02.065399 16395 raft_consensus.cc:1275] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:23:02.039647 19120 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:23:02.084880 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:02.086010 16394 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:23:02.088871 17143 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:02.092288 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:02.046708 19081 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:02.094393 16394 raft_consensus.cc:1240] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:02.059803 18661 raft_consensus.cc:3055] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:23:02.098768 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:02.113307 17270 raft_consensus.cc:1275] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 12:23:02.115585 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:02.117225 16395 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:23:02.118203 17270 pending_rounds.cc:85] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8: Aborting all ops after (but not including) 6
I20260709 12:23:02.118629 17270 pending_rounds.cc:107] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:23:02.118932 17270 raft_consensus.cc:2889] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:02.044025 19078 consensus_queue.cc:1059] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 12:23:02.084971 19151 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:02.121776 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:02.096961 18728 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:02.143656 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:02.096954 19100 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 6. Current term is 7.
I20260709 12:23:02.147622 17266 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:02.146296 17261 raft_consensus.cc:1275] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:23:02.089011 19051 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:02.063809 18809 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:02.100045 18661 raft_consensus.cc:740] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:02.100903 18013 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:23:02.157516 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:02.096094 19120 raft_consensus.cc:697] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
W20260709 12:23:02.144697 19200 log.cc:927] Time spent T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:23:02.163980 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:02.167945 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 124807us
I20260709 12:23:02.121645 18981 raft_consensus.cc:697] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:02.174062 18066 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:02.159652 19083 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.138s	user 0.027s	sys 0.052s
W20260709 12:23:02.182128 19083 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.160s	user 0.027s	sys 0.053s
I20260709 12:23:02.182228 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:02.190076 17254 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:23:02.191154 16474 raft_consensus.cc:1240] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:02.100732 19105 consensus_queue.cc:1048] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 12:23:02.183063 18809 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:02.166147 19220 consensus_queue.cc:1059] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:23:02.202414 19120 consensus_queue.cc:237] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:02.209652 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:02.222404 17266 raft_consensus.cc:2468] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:23:02.222568 18728 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.227517 17269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45228 (request call id 797) took 1289 ms. Trace:
I20260709 12:23:02.227953 17269 rpcz_store.cc:276] 0709 12:23:00.938498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:00.938681 (+   183us) service_pool.cc:224] Handling call
0709 12:23:02.227326 (+1288645us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b5400bc4dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:02.227457 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":439409024}
I20260709 12:23:02.225770 18661 consensus_queue.cc:260] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:02.237766 18661 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:02.238734 16473 raft_consensus.cc:1275] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:02.240914 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:02.242491 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:02.242972 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:02.244004 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:02.244876 17269 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:02.248095 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:02.250332 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:23:02.256911 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:02.229900 18981 consensus_queue.cc:237] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:02.252831 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:02.252007 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:02.262003 16395 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:23:02.263234 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:02.263609 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:02.264015 16395 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 3.
I20260709 12:23:02.266551 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:02.264322 16544 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 2.
W20260709 12:23:02.233837 19184 consensus_meta.cc:287] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.516s	user 0.013s	sys 0.000s
I20260709 12:23:02.272195 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:02.269169 17143 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:02.233590 19228 raft_consensus.cc:3055] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:02.278031 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:02.228699 18728 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:02.255712 18403 consensus_meta.cc:287] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.613s	user 0.009s	sys 0.002s
I20260709 12:23:02.285624 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:02.290371 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:02.255504 19151 raft_consensus.cc:697] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:02.284307 19228 raft_consensus.cc:740] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:02.301764 19228 consensus_queue.cc:260] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:02.302798 19131 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:02.303107 19228 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:23:02.304787 19219 log.cc:927] Time spent T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 12:23:02.307029 19213 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:02.269596 19078 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:02.303603 19131 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:02.314777 17261 raft_consensus.cc:1275] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 12:23:02.353852 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:02.289857 19140 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.323282 19122 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:02.377612 18728 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:02.312759 19232 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.381150 19232 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:02.382054 19131 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:02.383507 19232 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:02.377308 19122 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:02.283601 19184 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:02.369858 18961 consensus_meta.cc:287] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.687s	user 0.010s	sys 0.005s
I20260709 12:23:02.397320 16367 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:02.410341 16367 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:02.406008 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:02.384497 19100 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:02.413784 16367 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:02.415179 16367 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:02.415759 17255 raft_consensus.cc:2410] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:23:02.420614 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:02.433005 16367 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:02.422094 17271 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:23:02.438333 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:02.439806 16402 raft_consensus.cc:1275] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:02.393844 18676 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:23:02.448621 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:02.374622 19140 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:02.452450 16402 raft_consensus.cc:1275] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:02.401301 19131 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:02.386119 19078 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:02.417376 18728 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:02.469642 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:02.511296 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:02.416867 19226 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:02.428316 18013 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:02.417013 19217 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:23:02.418633 19100 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:02.416785 19080 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.416491 18403 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:02.521512 17144 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:02.545116 16401 raft_consensus.cc:1275] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:23:02.408213 18961 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:02.564018 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:02.566154 16401 pending_rounds.cc:85] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 2
I20260709 12:23:02.570562 16401 pending_rounds.cc:107] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:23:02.570916 16401 raft_consensus.cc:2889] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:02.562094 16477 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:23:02.511983 18947 consensus_meta.cc:287] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.565s	user 0.010s	sys 0.001s
W20260709 12:23:02.573155 18540 consensus_meta.cc:287] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.873s	user 0.004s	sys 0.009s
I20260709 12:23:02.573853 18540 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:02.576016 16545 raft_consensus.cc:1275] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:23:02.576485 16545 pending_rounds.cc:85] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1: Aborting all ops after (but not including) 4
I20260709 12:23:02.576720 16545 pending_rounds.cc:107] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:23:02.576973 16545 raft_consensus.cc:2889] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:02.513906 18675 consensus_queue.cc:1048] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 12:23:02.565682 19100 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:02.578094 17269 raft_consensus.cc:1240] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 4. Current term is 5. Ops: []
W20260709 12:23:02.580036 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:02.582316 16403 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:23:02.463445 18676 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:02.511715 19237 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:02.517843 19213 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.512884 19076 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:23:02.546610 18861 consensus_meta.cc:287] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.674s	user 0.006s	sys 0.006s
I20260709 12:23:02.514166 18661 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:02.562763 18740 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:02.517518 19184 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:02.615389 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:02.616379 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:02.522393 19131 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:02.587844 19150 consensus_meta.cc:287] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.796s	user 0.012s	sys 0.000s
I20260709 12:23:02.600183 16395 raft_consensus.cc:1240] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 12:23:02.616066 17271 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
W20260709 12:23:02.597522 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 76082.7us
I20260709 12:23:02.596562 19243 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:02.611660 18066 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:02.613260 16474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1039) took 1188 ms. Trace:
I20260709 12:23:02.619370 16400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 659) took 1020 ms. Trace:
I20260709 12:23:02.588563 19195 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:02.621104 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:02.624312 16474 rpcz_store.cc:276] 0709 12:23:01.431650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:01.474747 (+ 43097us) service_pool.cc:224] Handling call
0709 12:23:01.475021 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:02.196681 (+721660us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:02.613074 (+416393us) spinlock_profiling.cc:243] Waited 708 ms on lock 0x7b540010efd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:02.613214 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1629220096}
I20260709 12:23:02.457955 19151 consensus_queue.cc:237] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:02.594066 19237 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:02.602234 18961 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:02.606683 18403 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.580557 19080 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:02.658434 19209 log.cc:927] Time spent T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.856s	user 0.001s	sys 0.014s
I20260709 12:23:02.631958 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:02.715976 16549 raft_consensus.cc:2393] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 6: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:02.581559 18013 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:02.634079 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:02.720425 16550 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:02.634054 18540 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:02.722823 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 95847.4us
I20260709 12:23:02.627696 19150 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:02.620314 18661 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:02.619115 18740 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:02.610610 19076 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:02.634500 19195 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:02.646559 16477 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:23:02.647286 19078 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:02.610661 19191 consensus_queue.cc:1048] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 12:23:02.677949 19237 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:02.607065 19213 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:02.616683 19083 consensus_queue.cc:1048] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 12:23:02.757949 18740 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:02.759864 18661 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:02.652475 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:02.659760 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:02.761185 17258 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:23:02.757958 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:02.656862 16545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 994) took 1054 ms. Trace:
I20260709 12:23:02.715628 18403 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:02.755519 19080 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:02.762029 16545 rpcz_store.cc:276] 0709 12:23:01.602111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:01.602831 (+   720us) service_pool.cc:224] Handling call
0709 12:23:01.603111 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:02.577440 (+974329us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:02.656501 (+ 79061us) spinlock_profiling.cc:243] Waited 972 ms on lock 0x7b5400304b54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:02.656762 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2236525440}
I20260709 12:23:02.664919 17271 raft_consensus.cc:1240] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:02.661584 17270 raft_consensus.cc:1275] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:02.661036 17142 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
W20260709 12:23:02.766228 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 105267us
I20260709 12:23:02.767318 17142 leader_election.cc:400] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 4. Current term is 5.
I20260709 12:23:02.671525  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:02.623939 19243 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:02.624439 16400 rpcz_store.cc:276] 0709 12:23:01.598932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:01.599138 (+   206us) service_pool.cc:224] Handling call
0709 12:23:01.599676 (+   538us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:01.661895 (+ 62219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:01.661944 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:01.663018 (+  1074us) log.cc:844] Serialized 26 byte log entry
0709 12:23:01.793585 (+130567us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:01.793728 (+   143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:02.612030 (+818302us) spinlock_profiling.cc:243] Waited 56.7 ms on lock 0x7b5400bb80d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:02.612067 (+    37us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:02.612103 (+    36us) raft_consensus.cc:1685] finished
0709 12:23:02.612112 (+     9us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:02.619301 (+  7189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4951,"raft.queue_time_us":6696,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":109,"spinlock_wait_cycles":130359296,"thread_start_us":129647,"threads_started":1,"wal-append.queue_time_us":190143}
W20260709 12:23:02.662261 19231 log.cc:927] Time spent T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.428s	user 0.001s	sys 0.002s
I20260709 12:23:02.632961 16516 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:02.721840 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:02.763983 18403 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:02.649555 19122 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:02.629392 17144 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:02.675738 18728 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.735567 17263 raft_consensus.cc:1240] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 6. Current term is 7. Ops: []
W20260709 12:23:02.775444 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 142592us
I20260709 12:23:02.645836 18676 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:02.741540 19249 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:02.622453 19222 consensus_queue.cc:1048] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:23:02.745024 16549 raft_consensus.cc:1240] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 12:23:02.746142 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:02.689226 19000 consensus_queue.cc:1048] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 12:23:02.753747 19191 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:02.754225 19131 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:02.755502 19213 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:02.648237 19140 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:02.770143 17142 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:02.812505 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:02.813207 17270 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:23:02.813743 17142 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:02.813918 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:02.773840 16514 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:02.781181 16403 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:23:02.781697 16516 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:02.775449 16441 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:23:02.782920 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 153593us
I20260709 12:23:02.796176 16544 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:23:02.772817 18809 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:02.804244 16471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1037) took 1380 ms. Trace:
I20260709 12:23:02.774711 19238 consensus_queue.cc:1059] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:23:02.784430 18661 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:23:02.807312 19195 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:02.774240 19243 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:02.826609 17144 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:02.795329 19241 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:02.812745 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:02.821532 17258 raft_consensus.cc:1275] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:23:02.832578 17258 pending_rounds.cc:85] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8: Aborting all ops after (but not including) 2
I20260709 12:23:02.832829 17258 pending_rounds.cc:107] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:23:02.832684 17271 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:23:02.833096 17258 raft_consensus.cc:2889] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:02.786016 18728 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:02.821364 16471 rpcz_store.cc:276] 0709 12:23:01.423887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:01.468934 (+ 45047us) service_pool.cc:224] Handling call
0709 12:23:01.469272 (+   338us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:01.469673 (+   401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:01.469702 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:01.470275 (+   573us) log.cc:844] Serialized 52 byte log entry
0709 12:23:01.703023 (+232748us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:01.792637 (+ 89614us) log.cc:844] Serialized 14 byte log entry
0709 12:23:01.792919 (+   282us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:01.793027 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:02.657717 (+864690us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:23:02.798315 (+140598us) raft_consensus.cc:1685] finished
0709 12:23:02.803920 (+  5605us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:02.804144 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":139525,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":54,"raft.queue_time_us":5627,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":185,"spinlock_wait_cycles":33906432,"thread_start_us":307919,"threads_started":2,"wal-append.queue_time_us":331874}
I20260709 12:23:02.827989 16516 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:02.740862 19223 consensus_queue.cc:1059] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.080s
W20260709 12:23:02.905428 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 78895.4us
W20260709 12:23:02.911193 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 83265.5us
I20260709 12:23:02.913551 16516 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:02.917032 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:02.823413 18676 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:23:02.924968 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 149619us
I20260709 12:23:02.922528 16516 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:02.923255 18066 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:02.823849 19140 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:02.926090 16477 raft_consensus.cc:1275] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:23:02.929173 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:02.823231 18740 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:02.826292 18540 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:02.932267 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:02.933041 16471 raft_consensus.cc:1275] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:23:02.934989 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:02.935428 16516 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:02.936964 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:02.831948 19241 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:02.967914 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:02.817363 19191 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:02.920315 19184 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:02.919170 18809 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:02.904417 19247 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:02.905198 19122 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:02.919610 19130 raft_consensus.cc:3055] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:23:03.011389 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 249673us
I20260709 12:23:03.010188 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:02.935813 16473 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:23:03.013803 16367 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:03.014125 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 96554.7us
W20260709 12:23:02.976701 19220 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.024s	sys 0.033s
I20260709 12:23:03.041247 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:03.015192 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 89457.6us
I20260709 12:23:03.042289 16441 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:02.910250 18987 consensus_queue.cc:1048] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.082s
I20260709 12:23:02.930786 19078 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:02.934595 18540 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:02.798707 19249 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:02.935194 19250 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.013913 16550 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:23:02.937182 19111 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:02.971345 19241 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:02.916185 19236 consensus_queue.cc:1059] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.096s
I20260709 12:23:03.038098 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:03.042316 16549 raft_consensus.cc:1275] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 12:23:03.015638 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 77465us
I20260709 12:23:03.011938 18675 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:03.015093 18861 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:02.920491 19244 consensus_queue.cc:1059] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.113s
I20260709 12:23:02.994786 19076 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:03.018142 19210 log.cc:927] Time spent T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.081s	user 0.004s	sys 0.000s
I20260709 12:23:03.053471 18540 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:03.033398 19222 consensus_queue.cc:1048] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:23:03.033083 18740 consensus_queue.cc:1048] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:23:03.061002 19111 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:03.048794 16545 raft_consensus.cc:1275] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:23:03.081487 16545 pending_rounds.cc:85] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1: Aborting all ops after (but not including) 5
I20260709 12:23:03.081898 16545 pending_rounds.cc:107] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:23:03.082298 16545 raft_consensus.cc:2889] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:03.049607 16473 raft_consensus.cc:2393] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 9: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:03.083266 19111 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:03.047220 16367 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:03.084499 19223 consensus_queue.cc:1059] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:03.083977 19213 consensus_queue.cc:1048] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:03.088090 18403 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:03.051406 16515 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:03.016073 19184 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:03.056689 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:03.017967 18812 raft_consensus.cc:3055] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:03.018482 19084 raft_consensus.cc:3055] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:03.032892 19131 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:03.041674 19220 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.811s	user 0.024s	sys 0.034s
I20260709 12:23:03.033267 18676 raft_consensus.cc:697] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:03.035857 19235 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.035373 19150 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:03.036182 19130 raft_consensus.cc:740] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:03.066860 16441 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:03.068876 17143 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:03.052593 19230 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:03.065434 19100 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:03.043292 19122 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:03.097258 19235 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:03.091897 19254 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:03.109792 19122 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:03.110039 19235 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:03.110282 19238 consensus_queue.cc:1048] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:03.111863 19100 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:03.112052 19130 consensus_queue.cc:260] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:03.098289 19230 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:03.113296 19130 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:03.113484 19230 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:03.114986 19237 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:03.093335 19184 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:03.116422 19140 consensus_queue.cc:1048] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:23:03.118480 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 51704.5us
W20260709 12:23:03.118808 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 50051.4us
I20260709 12:23:03.119843 19223 raft_consensus.cc:2804] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:03.120226 16441 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:03.120747 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:03.122637 16545 raft_consensus.cc:1275] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:23:03.123068 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:03.047221 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 132353us
I20260709 12:23:03.093143 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:03.032159 19247 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:03.095388 17142 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:03.128870 17142 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:23:03.082283 19246 log.cc:927] Time spent T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:23:03.099786 18676 consensus_queue.cc:237] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:03.101840 19249 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:03.103618 19191 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:03.153383 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 62721.6us
I20260709 12:23:03.153739 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:03.154132 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:03.155010 16542 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:03.156467 17271 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:03.058216 19078 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:03.063527 19250 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:03.064661 19239 consensus_queue.cc:1059] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 12:23:03.105532 19086 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:03.107087 19192 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:03.098018 19131 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:03.091240 19084 raft_consensus.cc:740] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:03.017091 19195 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:23:03.129546 16545 raft_consensus.cc:1240] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:03.147011 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:03.091857 18812 raft_consensus.cc:740] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:03.154215 16442 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:03.096145 19251 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.139957 19236 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:03.091337 18403 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:03.140497 19081 raft_consensus.cc:2804] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:03.149745 18740 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:03.165300 19222 raft_consensus.cc:3055] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:03.148145 19242 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.147794 19080 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.168260 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:03.225847 16551 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:23:03.149780 18947 consensus_queue.cc:1048] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 12:23:03.202399 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:03.227240 16401 raft_consensus.cc:2393] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 5: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:03.179358 19000 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:03.165995 19258 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:03.206086 19237 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:03.206581 18728 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:03.209144 18812 consensus_queue.cc:260] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:03.170599 19076 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:03.220420 19086 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:03.214113 16474 raft_consensus.cc:2393] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 6: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:03.214804 16395 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:23:03.216514 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:03.229650 17266 raft_consensus.cc:2393] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 7: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:03.140753 19213 consensus_queue.cc:1048] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 12:23:03.225442 19222 raft_consensus.cc:740] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:03.189607 19122 raft_consensus.cc:3055] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:03.240689 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:03.169855 19150 consensus_queue.cc:1048] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 12:23:03.252277 16542 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:23:03.174722 19100 consensus_queue.cc:1048] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
W20260709 12:23:03.260093 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 193822us
I20260709 12:23:03.220682 19081 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:03.214278 19236 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:03.242923 19258 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:03.282866 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:03.258817 19120 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.960s	user 0.013s	sys 0.021s
I20260709 12:23:03.247941 19080 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:03.295433 19086 raft_consensus.cc:697] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:03.264729 19076 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:03.228374 19000 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:03.279944 18812 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:03.285972 19122 raft_consensus.cc:740] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:03.290825 19223 raft_consensus.cc:697] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:23:03.322131 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 140389us
W20260709 12:23:03.309037 19120 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.018s	user 0.013s	sys 0.021s
I20260709 12:23:03.302191 19081 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:03.242058 19242 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:03.349536 19263 log.cc:927] Time spent T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 12:23:03.350155 19100 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:03.350775 19242 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:03.302086 19236 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:03.351806 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:03.332919 18581 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:03.371143 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:03.336141 18403 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:03.337411 19223 consensus_queue.cc:237] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:03.285364 19084 consensus_queue.cc:260] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:03.380278 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:03.379654 16545 raft_consensus.cc:2393] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 7: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:03.223316 19251 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:03.354839 19176 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.309720 19258 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:23:03.405789 19245 log.cc:927] Time spent T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.675s	user 0.000s	sys 0.003s
I20260709 12:23:03.407079 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:03.407619 16367 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:03.407773 17261 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:03.374626 16551 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:23:03.379828 17255 raft_consensus.cc:1275] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:03.412730 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:03.381649 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:03.383612 16549 raft_consensus.cc:2368] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12.
I20260709 12:23:03.368367 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:03.321453 19076 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:03.385371 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:03.350782 19100 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:03.389117 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:03.396458 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.397836 19086 consensus_queue.cc:237] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:03.401053 19084 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:03.401561 19000 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:03.416639 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:03.416611 17143 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:23:03.386299 19219 log.cc:927] Time spent T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:23:03.419991 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:03.424043 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:03.451155 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:03.425922 16367 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:03.413565 19237 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:23:03.450019 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 329833us
I20260709 12:23:03.420070 19122 consensus_queue.cc:260] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:03.459610 16548 raft_consensus.cc:2393] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 7: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:03.459342 19000 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:03.460714 17271 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:23:03.455857 17261 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:23:03.461639 16367 leader_election.cc:400] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12.
I20260709 12:23:03.462160 16367 leader_election.cc:403] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:03.465106 16367 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:03.448539 19080 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:03.470511 19250 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:03.471379 16475 raft_consensus.cc:1240] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:23:03.464774 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:03.473115 16543 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:23:03.475602 19235 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:03.476963 19122 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:03.464079 19220 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:03.478538 19220 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:03.478901 19220 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:03.468154 19256 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12.
W20260709 12:23:03.485965 19233 log.cc:927] Time spent T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 1.199s	user 0.005s	sys 0.001s
I20260709 12:23:03.489447 19100 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:03.490656 19230 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:03.427309 19272 consensus_queue.cc:1048] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 12:23:03.468695 19000 raft_consensus.cc:697] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:03.460989 19078 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:03.412506 19176 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:03.424315 19184 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:03.499505 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:03.499155 17143 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:03.502492 16548 raft_consensus.cc:1240] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:03.504858 16367 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:03.505738 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:03.466640 19253 consensus_queue.cc:1048] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 12:23:03.506943 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:03.512302 19078 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:03.512777 19258 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:03.514202 17258 raft_consensus.cc:2393] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 6: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:03.514760 18675 raft_consensus.cc:3055] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:03.515175 18675 raft_consensus.cc:740] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:03.516206 19078 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.516883 19078 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:03.519632 19078 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:03.521131 16402 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:03.521417 19000 consensus_queue.cc:237] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:03.463634 19081 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:03.525200 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:03.525126 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:03.526435 17255 raft_consensus.cc:2393] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 7: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:03.534929 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:03.535772 16403 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:23:03.539812 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:03.539834 16548 raft_consensus.cc:1240] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:03.521181 19251 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:03.469357 19236 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:03.533054 19243 consensus_meta.cc:287] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.622s	user 0.011s	sys 0.000s
I20260709 12:23:03.529471 19131 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:03.552873 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:03.515918 18675 consensus_queue.cc:260] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:03.543495 19140 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:03.543216 19081 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:03.553632 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:03.561761 16543 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:03.556056 16441 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:03.557380 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:03.555712 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:03.558009 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:03.554605 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:03.555711 16398 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:03.563182 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:03.563552 16403 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:03.564249 16548 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
I20260709 12:23:03.568609 16397 raft_consensus.cc:2393] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 8: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:03.570930 16367 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:03.575246 16441 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:03.579770 19176 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:03.552588 19130 consensus_queue.cc:1059] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 12:23:03.544720 19220 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:03.581647 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:03.582286 17271 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:03.582954 19220 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:03.583820 19213 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:23:03.579797 19280 log.cc:927] Time spent T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
W20260709 12:23:03.585609 19241 consensus_meta.cc:287] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.520s	user 0.011s	sys 0.000s
I20260709 12:23:03.586201 19241 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:03.558485 19140 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:03.588615 19140 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:03.588783 16399 raft_consensus.cc:1240] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:23:03.555800 19230 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:03.577656 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:03.580870 19130 raft_consensus.cc:3055] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:23:03.591912 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:03.556586 19243 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:03.598260 16398 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:23:03.602243 16441 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:23:03.603384 19154 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.361s	user 0.008s	sys 0.004s
I20260709 12:23:03.598845 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:03.600063 16403 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:23:03.601326 19078 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:03.604923 17271 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:23:03.613698 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:03.613715 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:03.614749 16542 raft_consensus.cc:2368] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 7. Current term is 8.
I20260709 12:23:03.616489 17143 leader_election.cc:400] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 7. Current term is 8.
I20260709 12:23:03.616896 17143 leader_election.cc:403] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:23:03.617653 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:03.619444 16367 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:03.621769 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:03.625584 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:03.626754 16542 raft_consensus.cc:1275] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:23:03.592561 18675 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:03.566454 19253 consensus_queue.cc:1048] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:23:03.630863 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:03.562937 18540 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:03.616482 19154 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.374s	user 0.008s	sys 0.005s
I20260709 12:23:03.635164 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:03.635694 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:03.602028 19130 raft_consensus.cc:740] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:03.650754 17271 raft_consensus.cc:2393] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 9: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:03.652675 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:03.610697 19078 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:03.604708 19279 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:03.569911 19256 consensus_queue.cc:1059] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 12:23:03.609947 18581 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:03.661160 19213 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:03.627580 19083 consensus_queue.cc:1048] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:23:03.655257 19241 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:03.664311 18540 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:03.664552 19262 log.cc:927] Time spent T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.374s	user 0.003s	sys 0.000s
W20260709 12:23:03.667356 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.609654 19243 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:03.630085 19230 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:03.669486 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:03.669775 16470 raft_consensus.cc:1275] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:03.630704 19081 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:03.731158 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:03.688732 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:03.714224 19122 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:03.714744 19190 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:23:03.653606 19108 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.366s	user 0.013s	sys 0.035s
I20260709 12:23:03.673926 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:03.714000 16543 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:23:03.671067 19078 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:03.720093 16441 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:03.636060 19251 consensus_queue.cc:1048] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 12:23:03.712409 19256 consensus_queue.cc:1243] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer 610f4feda11942739236f2b121a4b9c1 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 12:23:03.703657 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.687182 19076 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:03.738070 19108 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.450s	user 0.019s	sys 0.038s
I20260709 12:23:03.739106 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:03.728837 19236 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:03.738278 16472 raft_consensus.cc:2410] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:23:03.744513 17143 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:03.761169 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:03.720822 19213 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:03.740052 19184 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:03.737643 19230 raft_consensus.cc:697] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:03.759881 16441 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:03.767652 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:03.757596 18961 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:03.756599 19084 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:03.768471 19131 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.737038 19122 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:03.765501 19243 raft_consensus.cc:2804] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:03.737574 19190 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:03.802102 19190 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:03.802987 19230 consensus_queue.cc:237] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:02.806106  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:03.809625 19122 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:03.771697 19269 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:03.810357 19269 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:03.810752 19269 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:23:03.811643 16223 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59300 (request call id 84) took 22570 ms (client timeout 15000 ms). Trace:
I20260709 12:23:03.763309 19083 consensus_queue.cc:1048] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 12:23:03.823375 18961 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:03.825018 17254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 808) took 1421 ms. Trace:
I20260709 12:23:03.825629 17254 rpcz_store.cc:276] 0709 12:23:02.403541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:02.403704 (+   163us) service_pool.cc:224] Handling call
0709 12:23:02.427896 (+ 24192us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:02.638419 (+210523us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:02.638472 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:02.638533 (+    61us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:02.638603 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:03.824680 (+1186077us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b5400be2dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:03.824746 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:03.824926 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":483463552}
I20260709 12:23:03.780491 16441 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:23:03.782358 19225 log.cc:927] Time spent T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 1.550s	user 0.001s	sys 0.003s
I20260709 12:23:03.773541 19131 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:03.838053 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:03.786666 19076 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:03.787405 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:03.786761 19222 consensus_queue.cc:260] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:03.789702 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:03.797210 18581 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:03.798282 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:03.799738 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:03.800029 17271 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:23:03.735047 19130 consensus_queue.cc:260] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:03.815766 19122 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:03.817584 19168 raft_consensus.cc:3055] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:03.840442 19213 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:03.841032 18013 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:03.844627 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 64380.3us
I20260709 12:23:03.868042 16441 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:03.847496 19154 raft_consensus.cc:3055] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:03.847668 19243 raft_consensus.cc:697] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:03.850952 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:03.843339 19051 raft_consensus.cc:3055] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:03.860605 17261 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
I20260709 12:23:03.848135 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:03.875527 16441 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:03.831331 18961 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:03.882469 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:03.892210 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:03.866251 19168 raft_consensus.cc:740] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:03.894958 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:03.899885 16543 raft_consensus.cc:1240] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 12:23:03.853323 19140 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:03.872531 19154 raft_consensus.cc:740] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:03.872680 19222 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:03.875905 19213 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:03.874447 19051 raft_consensus.cc:740] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:03.939494 18961 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:03.943989 19051 consensus_queue.cc:260] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:03.945315 19051 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:03.861299 19122 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:03.828667 16223 rpcz_store.cc:269] 0709 12:22:41.258107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.264078 (+  5971us) service_pool.cc:224] Handling call
0709 12:22:43.627588 (+2363510us) spinlock_profiling.cc:243] Waited 85.6 ms on lock 0x7b540027e054. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df3090 00007f2875e91f3a 00007f287bc47e9c 00007f287bd4604e 00007f287a2072a4 00007f287a207232 00007f287a207161 00007f287a2070dc 00007f287a206392 00007f28741c448c 00007f28741c38c6
0709 12:23:03.811483 (+20183895us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:22:58.612753 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:22:58.613676 (+   923us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:22:58.613744 (+    68us) write_op.cc:435] Acquired schema lock
0709 12:22:58.613771 (+    27us) tablet.cc:662] Decoding operations
0709 12:22:58.615285 (+  1514us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:22:58.615382 (+    97us) write_op.cc:641] Partition lock acquired for write op
0709 12:22:58.615432 (+    50us) tablet.cc:685] Acquiring locks for 12 operations
0709 12:22:58.616920 (+  1488us) tablet.cc:701] Row locks acquired
0709 12:22:58.616973 (+    53us) write_op.cc:260] PREPARE: finished
0709 12:22:58.617182 (+   209us) write_op.cc:270] Start()
0709 12:22:58.617355 (+   173us) write_op.cc:276] Timestamp: P: 1783599778617102 usec, L: 0
0709 12:22:58.617383 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 12:22:58.618331 (+   948us) log.cc:844] Serialized 4701 byte log entry
0709 12:22:59.603278 (+984947us) op_driver.cc:464] REPLICATION: finished
0709 12:23:00.100468 (+497190us) write_op.cc:301] APPLY: starting
0709 12:23:00.103682 (+  3214us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:00.104024 (+   342us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:00.104209 (+   185us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:00.105917 (+  1708us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:00.106298 (+   381us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=12
0709 12:23:00.106353 (+    55us) write_op.cc:312] APPLY: finished
0709 12:23:00.106774 (+   421us) log.cc:844] Serialized 88 byte log entry
0709 12:23:00.116790 (+ 10016us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:00.116978 (+   188us) write_op.cc:454] Released schema lock
0709 12:23:00.117085 (+   107us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:00.181659 (+ 64574us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:00.181699 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":840170,"reported_tablets":37,"spinlock_wait_cycles":196941184,"child_traces":[["op",{"apply.queue_time_us":440450,"apply.run_cpu_time_us":4395,"apply.run_wall_time_us":57486,"num_ops":12,"prepare.queue_time_us":353763,"prepare.run_cpu_time_us":16529,"prepare.run_wall_time_us":346565,"raft.queue_time_us":316579,"raft.run_cpu_time_us":14176,"raft.run_wall_time_us":412972,"replication_time_us":953638,"thread_start_us":984496,"threads_started":4,"wal-append.queue_time_us":420523,"wal-append.run_cpu_time_us":53602,"wal-append.run_wall_time_us":2060333}]]}
W20260709 12:23:03.921941 19291 log.cc:927] Time spent T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:23:03.881424 19130 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:03.925913 19084 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:03.927625 19154 consensus_queue.cc:260] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:03.929685 19131 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:03.934010 19168 consensus_queue.cc:260] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:03.936510 19247 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:03.938912 19258 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:03.941936 19078 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:03.866546 19269 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:03.956053 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:03.961038 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:03.906530 18581 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:03.962725 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:03.971009 16399 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:23:03.969282 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.971009 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:03.980727 16476 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:23:03.961858 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:03.970237 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:03.954880 19232 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:03.950287 19122 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:03.961489 19239 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:03.978583 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:03.995286 16475 raft_consensus.cc:2368] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 6. Current term is 7.
I20260709 12:23:03.978652 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:03.980424 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:03.977388 16403 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:23:03.980644 19168 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:23:03.986876 19289 log.cc:927] Time spent T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:23:03.989428 18961 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:03.972877 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:03.996567 16397 raft_consensus.cc:2393] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 8: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:03.977304 19154 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:04.012739 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:04.033521 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:04.021692 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 63726.7us
I20260709 12:23:04.022017 16550 raft_consensus.cc:1240] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:23:03.998180 19232 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:04.026850 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:04.038497 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:04.034719 16367 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:04.019263 19226 raft_consensus.cc:3055] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:23:04.038770 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:04.032305 17144 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:04.039198 16547 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:23:04.058238 17255 raft_consensus.cc:1240] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
W20260709 12:23:04.044088 17264 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45224 (request call id 813) took 1952 ms (client timeout 2559 ms). Trace:
I20260709 12:23:04.059726 17144 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
W20260709 12:23:04.059754 17264 rpcz_store.cc:269] 0709 12:23:02.091673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:02.092018 (+   345us) service_pool.cc:224] Handling call
0709 12:23:04.043893 (+1951875us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54005f3554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:04.044021 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2708428672}
I20260709 12:23:04.062002 16367 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:04.049973 16400 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:23:04.011816 19269 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:04.042768 16474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53186 (request call id 504) took 1116 ms. Trace:
I20260709 12:23:04.064246 16474 rpcz_store.cc:276] 0709 12:23:02.926666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:02.926800 (+   134us) service_pool.cc:224] Handling call
0709 12:23:04.042561 (+1115761us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b54006c7ad4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:04.042704 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":407910144}
I20260709 12:23:04.066099 16366 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:04.067760 16442 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:04.078629 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:04.078773 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:04.085832 17254 raft_consensus.cc:1275] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 12:23:04.086300 17254 pending_rounds.cc:85] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8: Aborting all ops after (but not including) 3
I20260709 12:23:04.086555 17254 pending_rounds.cc:107] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:23:04.087056 17254 raft_consensus.cc:2889] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:04.087288 17144 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:04.085556 18581 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:23:04.090852 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:04.099476 17263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 812) took 1679 ms. Trace:
W20260709 12:23:04.102063 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:04.057206 19081 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 7. Current term is 8.
I20260709 12:23:04.097472 16549 raft_consensus.cc:1275] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 12:23:04.117252 16549 pending_rounds.cc:85] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1: Aborting all ops after (but not including) 3
I20260709 12:23:04.117668 16549 pending_rounds.cc:107] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:23:04.118018 16549 raft_consensus.cc:2889] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:04.089154 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:04.100024 17263 rpcz_store.cc:276] 0709 12:23:02.420267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:02.420416 (+   149us) service_pool.cc:224] Handling call
0709 12:23:02.457605 (+ 37189us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:02.781268 (+323663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:04.099142 (+1317874us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b5400a46d54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:04.099397 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":638496000}
I20260709 12:23:04.120482 16473 raft_consensus.cc:1275] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:23:04.110432 19241 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:04.057309 19226 raft_consensus.cc:740] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:03.978956 19243 consensus_queue.cc:237] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:04.074275 19108 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:04.078327 19230 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:04.074863 19256 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:04.074939 18740 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:23:04.031060 19239 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:04.189733 19239 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:04.202209 17142 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:04.175146 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:04.203352 16548 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:04.203644 17269 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:23:04.073980 18961 consensus_queue.cc:1059] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.097s
I20260709 12:23:04.176968 16394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 947) took 1082 ms. Trace:
I20260709 12:23:04.116899 19261 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:04.209023 19261 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:04.209591 19261 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:04.098508 18581 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:04.212565 18581 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:04.212980 16548 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:23:04.215510 16469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1020) took 1060 ms. Trace:
I20260709 12:23:04.126468 19266 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.167651 17263 raft_consensus.cc:1240] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:23:04.208855 16394 rpcz_store.cc:276] 0709 12:23:03.094616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:03.094757 (+   141us) service_pool.cc:224] Handling call
0709 12:23:03.095079 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:03.247013 (+151934us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:03.247064 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:03.247129 (+    65us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:03.247230 (+   101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:04.176708 (+929478us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b5400543154. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:04.176757 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:04.176907 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":348824960}
I20260709 12:23:04.208236 19232 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:04.219146 16224 catalog_manager.cc:5697] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 2 to 8. New cstate: current_term: 8 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:23:04.202368 19078 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:04.215905 16469 rpcz_store.cc:276] 0709 12:23:03.155639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:03.156441 (+   802us) service_pool.cc:224] Handling call
0709 12:23:03.156876 (+   435us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:03.176121 (+ 19245us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:03.176168 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:03.183011 (+  6843us) log.cc:844] Serialized 52 byte log entry
0709 12:23:03.788122 (+605111us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x7b180046eb80. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:23:03.960859 (+172737us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:03.980096 (+ 19237us) log.cc:844] Serialized 14 byte log entry
0709 12:23:03.980347 (+   251us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:03.980430 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:04.193388 (+212958us) spinlock_profiling.cc:243] Waited 3.18 ms on lock 0x7b180046eb80. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:23:04.215295 (+ 21907us) raft_consensus.cc:1685] finished
0709 12:23:04.215317 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:04.215454 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59403,"prepare.queue_time_us":10717,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":142,"raft.queue_time_us":19177,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":10987,"spinlock_wait_cycles":43550976,"thread_start_us":172444,"threads_started":1,"wal-append.queue_time_us":976709}
I20260709 12:23:04.224146 16224 catalog_manager.cc:5697] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:23:04.226747 16224 catalog_manager.cc:5697] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
W20260709 12:23:04.229161 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 44289.4us
W20260709 12:23:04.235926 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:04.241477 16224 catalog_manager.cc:5697] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 2 to 7, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 7 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:04.245406 16514 leader_election.cc:400] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 6. Current term is 7.
I20260709 12:23:04.245923 16514 leader_election.cc:403] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:23:04.243268 16224 catalog_manager.cc:5697] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 1 to 9, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 9 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:04.249631 16224 catalog_manager.cc:5697] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:04.251133 16224 catalog_manager.cc:5697] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 4 to 5, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 5 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:04.261229 16224 catalog_manager.cc:5697] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 2 to 5, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 5 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:04.215914 19122 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.269922 16515 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:04.268939 16224 catalog_manager.cc:5697] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:04.270965 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:04.272732 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:04.222674 19239 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.332907 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:04.339079 16475 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:04.186028 19176 consensus_queue.cc:1048] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
W20260709 12:23:04.348497 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 103268us
I20260709 12:23:04.348832 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:23:04.351164 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 265923us
I20260709 12:23:04.340724 16224 catalog_manager.cc:5697] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 1 to 5, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 5 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:23:04.354180 16472 raft_consensus.cc:1240] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:23:04.357359 16394 raft_consensus.cc:1240] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:04.222290 19266 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:04.265601 19215 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:04.357226 16224 catalog_manager.cc:5697] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 6 to 9, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 9 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } }
W20260709 12:23:04.329480 19276 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.032s	sys 0.047s
I20260709 12:23:04.369966 17142 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:04.373207 16224 catalog_manager.cc:5697] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 3 to 6, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 6 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:23:04.359313 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:04.375566 16224 catalog_manager.cc:5697] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 5 to 6, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 6 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:04.265188 19081 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:23:04.367812 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 280585us
W20260709 12:23:04.360565 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 274311us
I20260709 12:23:04.365836 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:04.353660 19113 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 6. Current term is 7.
I20260709 12:23:04.355742 16366 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:04.350231 19084 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:23:04.376030 19276 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.032s	sys 0.048s
W20260709 12:23:04.577805 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:04.576128 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 220473us
I20260709 12:23:04.579077 16366 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:04.331681 19122 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:04.381027 19100 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:04.419991 16475 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:23:04.355002 19239 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:04.430508 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 55438.2us
I20260709 12:23:04.434362 16399 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:23:04.456072 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:04.467082 16400 raft_consensus.cc:1240] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:04.473606 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:04.605353 16397 raft_consensus.cc:2368] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6.
W20260709 12:23:04.479056 17271 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.006s	sys 0.014s
I20260709 12:23:04.488394 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:04.502348 19230 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:04.505048 16477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 512) took 1311 ms. Trace:
I20260709 12:23:04.573484 19272 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:04.515991 19226 consensus_queue.cc:260] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:04.575587 18961 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:04.562310 16401 raft_consensus.cc:1275] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:23:04.568673 16224 catalog_manager.cc:5697] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 2 to 7, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 7 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:04.564671 17144 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:04.566335 17143 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:04.563651 19184 consensus_queue.cc:1048] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 12:23:04.572714 18675 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:04.580835 19084 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:04.606541 16477 rpcz_store.cc:276] 0709 12:23:03.193605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:03.194356 (+   751us) service_pool.cc:224] Handling call
0709 12:23:03.194701 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:03.424284 (+229583us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:03.424317 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:03.448856 (+ 24539us) log.cc:844] Serialized 26 byte log entry
0709 12:23:03.913951 (+465095us) spinlock_profiling.cc:243] Waited 1.33 ms on lock 0x7b18002f03e0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:23:04.243720 (+329769us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:04.243763 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:04.354926 (+111163us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b5400594f54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:04.354996 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:04.504782 (+149786us) raft_consensus.cc:1685] finished
0709 12:23:04.504803 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:04.504977 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7033,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":10762,"spinlock_wait_cycles":521998208,"thread_start_us":329569,"threads_started":1,"wal-append.queue_time_us":1043201}
I20260709 12:23:04.513039 19241 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:23:04.575884 19301 log.cc:927] Time spent T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:23:04.613934 19308 raft_consensus.cc:2764] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:23:04.543381 17258 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:23:04.612222 17142 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:23:04.394738 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 219203us
W20260709 12:23:04.637496 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:04.620278 16395 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
I20260709 12:23:04.623924 18961 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:23:04.615639 19051 consensus_meta.cc:287] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.670s	user 0.013s	sys 0.001s
W20260709 12:23:04.598289 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 248779us
I20260709 12:23:04.620771 16224 catalog_manager.cc:5697] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
W20260709 12:23:04.628830 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:04.630551 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 64329.3us
I20260709 12:23:04.577162 19223 consensus_queue.cc:1059] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.043s
W20260709 12:23:04.630839 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 66267us
I20260709 12:23:04.617537 19086 consensus_queue.cc:1059] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 12:23:04.618242 19272 raft_consensus.cc:697] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:23:04.632174 19222 consensus_meta.cc:287] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.677s	user 0.013s	sys 0.000s
I20260709 12:23:04.628683 19213 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.610772 16366 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:04.781975 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:04.782598 17261 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:04.656244 16224 catalog_manager.cc:5697] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 4 to 6, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 6 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
I20260709 12:23:04.683804 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:04.783442 16547 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:04.682317 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:04.706360 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 436510us
I20260709 12:23:04.640664 19120 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.639595 19100 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:04.778515 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:04.796316 16549 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:04.777021 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:04.620678 19261 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:04.779155 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 97722us
I20260709 12:23:04.797818 17144 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:04.782513 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 171818us
I20260709 12:23:04.798872 16366 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:04.616456 18581 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:04.785651 16515 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:04.658561 18961 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:04.820165 19315 log.cc:927] Time spent T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 12:23:04.821956 17255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 829) took 1164 ms. Trace:
I20260709 12:23:04.822153 18961 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:04.795620 19307 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:04.627215 18675 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:04.822369 17255 rpcz_store.cc:276] 0709 12:23:03.657724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:04.059000 (+401276us) service_pool.cc:224] Handling call
0709 12:23:04.059292 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:04.106533 (+ 47241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:04.106582 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:04.129020 (+ 22438us) log.cc:844] Serialized 26 byte log entry
0709 12:23:04.457804 (+328784us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:23:04.457853 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:04.457939 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:04.821569 (+363630us) raft_consensus.cc:1685] finished
0709 12:23:04.821590 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:04.821868 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15063,"raft.queue_time_us":31141,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":12701,"thread_start_us":255795,"threads_started":1,"wal-append.queue_time_us":445070}
I20260709 12:23:04.825084 18675 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:04.689356 18013 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:23:04.633452 19238 raft_consensus.cc:2764] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:23:04.825770 18013 raft_consensus.cc:697] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:04.826203 19238 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:04.826668 19238 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:04.827039 19168 consensus_meta.cc:287] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.790s	user 0.007s	sys 0.002s
I20260709 12:23:04.794351 16224 catalog_manager.cc:5697] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 6 to 7, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 7 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
W20260709 12:23:04.827744 19154 consensus_meta.cc:287] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.771s	user 0.009s	sys 0.000s
W20260709 12:23:04.792680 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 99119.6us
I20260709 12:23:04.798239 19258 consensus_queue.cc:1059] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
W20260709 12:23:04.816758 19313 log.cc:927] Time spent T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 12:23:04.817658 19084 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:04.817902 19309 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:04.818765 19122 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:04.819088 16366 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:04.819860 19311 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:04.826697 18013 consensus_queue.cc:237] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:04.828208 16224 catalog_manager.cc:5697] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:04.859681 16224 catalog_manager.cc:5697] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:04.829159 17144 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:04.829638 17271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 836) took 1333 ms. Trace:
I20260709 12:23:04.829888 19150 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:23:04.801815 17261 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:23:04.864807 16549 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:23:04.830206 17270 raft_consensus.cc:1240] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:04.817718 18581 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:04.811221 16547 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
W20260709 12:23:04.832021 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 46841.4us
I20260709 12:23:04.836024 16544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1010) took 1057 ms. Trace:
I20260709 12:23:04.850253 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:23:04.829682 19304 log.cc:927] Time spent T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.182s	user 0.000s	sys 0.000s
I20260709 12:23:04.805939 19100 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:04.793757 19308 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:04.861864 16366 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:04.806277 19120 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:04.913776 17254 raft_consensus.cc:1240] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 12:23:04.864861 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:04.864264 16224 catalog_manager.cc:5697] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 5 to 6, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 6 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:04.915122 17263 raft_consensus.cc:2393] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 7: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:04.831090 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:04.867151 17143 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:04.916796 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:04.853502 19310 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.806391 19213 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:04.893935 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:04.915454 16224 catalog_manager.cc:5697] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 3 to 7, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 7 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:23:04.917711 16400 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:23:04.918529 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:04.905848 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:04.861585 17271 rpcz_store.cc:276] 0709 12:23:03.496000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:03.895633 (+399633us) service_pool.cc:224] Handling call
0709 12:23:03.895926 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:03.896867 (+   941us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:03.896913 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:03.905737 (+  8824us) log.cc:844] Serialized 26 byte log entry
0709 12:23:04.609728 (+703991us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:04.609771 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:04.609840 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:04.829415 (+219575us) raft_consensus.cc:1685] finished
0709 12:23:04.829440 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:04.829578 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1743,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":4469,"thread_start_us":638538,"threads_started":1,"wal-append.queue_time_us":676949}
I20260709 12:23:04.920262 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:04.918268 16224 catalog_manager.cc:5697] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 5 to 8, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 8 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:04.928802 16401 raft_consensus.cc:2393] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 8: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:04.930112 16224 catalog_manager.cc:5697] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 2 to 3, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 3 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:04.934618 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:23:04.935909 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 52237.3us
I20260709 12:23:04.867569 18809 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.937212 16515 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:23:04.958813 16403 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.030s	sys 0.029s
I20260709 12:23:04.852886 19084 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:23:04.939065 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 72002.3us
W20260709 12:23:04.981655 16403 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.032s	sys 0.030s
I20260709 12:23:04.950670 16399 raft_consensus.cc:1240] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:04.867695 18675 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:23:04.978936 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:04.987460 16400 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:23:04.990926 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:04.987605 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:23:04.929279 19320 log.cc:927] Time spent T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:23:05.004065 16398 raft_consensus.cc:2393] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 6: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:04.868502 19122 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:04.992225 16400 raft_consensus.cc:1240] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:04.884192 16544 rpcz_store.cc:276] 0709 12:23:03.778535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:03.778927 (+   392us) service_pool.cc:224] Handling call
0709 12:23:03.779247 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:03.779619 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:03.779645 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:03.780657 (+  1012us) log.cc:844] Serialized 26 byte log entry
0709 12:23:03.915174 (+134517us) spinlock_profiling.cc:243] Waited 33.9 ms on lock 0x7b18004b1ae0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:23:04.244138 (+328964us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:04.329015 (+ 84877us) log.cc:844] Serialized 14 byte log entry
0709 12:23:04.543812 (+214797us) spinlock_profiling.cc:243] Waited 4 ms on lock 0x7b18004b1ae0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d16c0a 00007f287c456219 00007f2872d9ea40 00007f2872d994e4 00007f2872d99455 00007f2875e81060 00007f2875e93716 00007f2875ea4287 00007f2875ea4208
0709 12:23:04.835675 (+291863us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:04.835755 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:04.835813 (+    58us) raft_consensus.cc:1685] finished
0709 12:23:04.835825 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:04.835960 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23061,"prepare.queue_time_us":102909,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":2899,"raft.queue_time_us":243831,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":4672,"spinlock_wait_cycles":87174656,"thread_start_us":614062,"threads_started":2,"wal-append.queue_time_us":657075}
W20260709 12:23:04.999936 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 57281us
W20260709 12:23:05.009267 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 72123.1us
I20260709 12:23:05.012367 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:05.013160 16399 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:05.014781 17264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 832) took 1356 ms. Trace:
I20260709 12:23:05.015192 17264 rpcz_store.cc:276] 0709 12:23:03.658614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:04.061331 (+402717us) service_pool.cc:224] Handling call
0709 12:23:04.061703 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:04.061916 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:04.061939 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:04.092214 (+ 30275us) log.cc:844] Serialized 26 byte log entry
0709 12:23:04.478295 (+386081us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:04.478428 (+   133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:04.478567 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:05.014595 (+536028us) raft_consensus.cc:1685] finished
0709 12:23:05.014614 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:05.014727 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":301287,"threads_started":1,"wal-append.queue_time_us":404917}
I20260709 12:23:05.005937 16367 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:04.892827 19100 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:05.025504 16515 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:04.936419 19000 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:04.854801 19309 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:04.978814 19316 log.cc:927] Time spent T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:23:05.033488 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:04.931972 19311 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:04.993499 19243 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:05.043491 16402 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
I20260709 12:23:05.018867 19168 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:05.060931 16472 raft_consensus.cc:1240] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:05.032392 19176 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.032840 19108 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:05.048211 19307 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:23:05.098539 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 260926us
I20260709 12:23:05.047658 19318 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:05.008620 18809 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:04.940517 19310 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:05.099994 16367 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:04.931406 19308 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:05.091475 16399 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:23:05.057698 19239 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:05.010757 19084 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:04.936053 19272 consensus_queue.cc:237] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:05.040097 19250 consensus_queue.cc:1059] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 12:23:05.061659 19083 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.067745 19243 raft_consensus.cc:697] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:23:05.115197 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 286100us
I20260709 12:23:05.121315 19223 consensus_queue.cc:1243] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer f99066d13e524238aec6aa2024a8bd50 log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:23:05.102182 19318 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:05.157122 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:05.163571 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:05.164315 16402 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:05.114286 19122 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:05.169126 19122 raft_consensus.cc:697] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 8 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:05.170243 19122 consensus_queue.cc:237] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:05.073825 19168 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:05.098438 19176 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:05.182150 16402 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:23:05.183462 16546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1014) took 1291 ms. Trace:
I20260709 12:23:05.101341 19108 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:05.185685 16546 rpcz_store.cc:276] 0709 12:23:03.891971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:03.892248 (+   277us) service_pool.cc:224] Handling call
0709 12:23:03.892644 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:03.903728 (+ 11084us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:03.903762 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:03.916507 (+ 12745us) log.cc:844] Serialized 26 byte log entry
0709 12:23:04.238689 (+322182us) spinlock_profiling.cc:243] Waited 36.9 ms on lock 0x7b180001fd20. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:23:04.659789 (+421100us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:23:04.659851 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:04.659960 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:05.175388 (+515428us) raft_consensus.cc:1685] finished
0709 12:23:05.175406 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:05.183364 (+  7958us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":196,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":9075,"spinlock_wait_cycles":84951424,"thread_start_us":420823,"threads_started":1,"wal-append.queue_time_us":972883}
I20260709 12:23:05.161397 18740 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.160795 19261 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:05.159981 19000 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:05.191655 19220 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:05.192559 19223 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:05.192857 18809 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:05.193353 19108 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:05.174554 19276 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.174093 19318 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:05.154105 19083 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:05.208914 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:05.217432 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:05.216652 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 179666us
I20260709 12:23:05.197656 19260 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:05.185736 19086 consensus_queue.cc:1059] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:23:05.197468 19330 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:05.208252 19321 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:05.218860 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 184703us
I20260709 12:23:05.213050 19120 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:05.224364 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 199354us
I20260709 12:23:05.213014 19108 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:05.220615 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:05.201690 18740 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:05.201464 19334 consensus_queue.cc:1048] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 12:23:05.248370 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:05.223182 19213 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:05.207504 19220 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:05.207765 19223 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:05.204216 19000 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:05.215986 19276 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:05.254323 19250 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:05.261931 19308 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:05.267786 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:05.279825 19331 log.cc:927] Time spent T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 12:23:05.291733 16400 raft_consensus.cc:2410] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:23:05.217140 19243 consensus_queue.cc:237] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:05.262706 19250 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:05.296452 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:05.280758 19086 raft_consensus.cc:3055] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:05.297266 16516 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:05.298173 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 48756.8us
W20260709 12:23:05.300233 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 79826.4us
I20260709 12:23:05.302322 16441 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:05.299363 16402 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:05.280522 19168 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:05.229295 19260 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:05.289160 19309 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:05.252457 19108 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:05.293419 19310 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:05.326375 19350 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:05.327090 19176 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:05.327607 16441 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:05.328660 18740 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:05.330089 19000 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:05.330334 19220 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:05.330646 19347 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:05.317098 19168 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.243741 19330 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:05.331282 19084 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:05.346071 19250 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:05.244000 19321 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:05.343482 19361 raft_consensus.cc:2749] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:23:05.299310 19086 raft_consensus.cc:740] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:23:05.372526 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 59867.4us
I20260709 12:23:05.375200  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:05.330704 19335 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.380442 19000 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:05.360818 18740 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.416070 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:05.443686 16442 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.530s (SENT) [suppressed 8 similar messages]
I20260709 12:23:05.407948 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:05.417681 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:05.394881 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:05.523653 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:05.394042 16515 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:05.502113 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:05.407096 16543 raft_consensus.cc:1275] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 12:23:05.526405 16476 raft_consensus.cc:2410] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:23:05.417917 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:05.485167 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:05.407413 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:05.423007 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:05.529498 16547 raft_consensus.cc:2468] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:23:05.361366 19350 raft_consensus.cc:697] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:05.383111 19220 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.473701 19340 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:05.387635 19321 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:05.396380 19310 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:05.491531 19084 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:05.409045 19168 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:05.381700 19318 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:05.617391 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 45441.3us
I20260709 12:23:05.565471 17143 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:23:05.384377 19342 log.cc:927] Time spent T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:23:05.627897 19318 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:05.628749 16546 raft_consensus.cc:1275] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:05.529211 16544 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
W20260709 12:23:05.568472 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 174508us
I20260709 12:23:05.528744 16395 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
W20260709 12:23:05.396871 19333 log.cc:927] Time spent T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 12:23:05.386668 19307 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:05.385458 18581 raft_consensus.cc:3055] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:05.630756 18581 raft_consensus.cc:740] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:05.528363 19000 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:05.631703 18581 consensus_queue.cc:260] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:05.527166 18740 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:05.633064 18581 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:05.634590 18740 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:05.636003 19000 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:05.385289 19330 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:05.490811 19108 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:05.398782 19309 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:05.680081 19309 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:05.683310 19309 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:05.524371 19347 consensus_queue.cc:1059] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.132s
W20260709 12:23:05.684494 17258 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45228 (request call id 840) took 2027 ms (client timeout 2495 ms). Trace:
I20260709 12:23:05.685019 19330 raft_consensus.cc:3055] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:23:05.684955 17258 rpcz_store.cc:269] 0709 12:23:03.657353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:03.871092 (+213739us) service_pool.cc:224] Handling call
0709 12:23:05.684282 (+1813190us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b540005fd54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:05.684428 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":675223296}
I20260709 12:23:05.685451 19330 raft_consensus.cc:740] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:05.518792 19250 consensus_queue.cc:1048] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.104s
I20260709 12:23:05.474050 19276 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:05.408535 19223 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:05.388032 19100 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:05.405249 19083 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:05.385023 19222 raft_consensus.cc:3055] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:23:05.624454 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 59143.5us
I20260709 12:23:05.386750 19086 consensus_queue.cc:260] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:05.627724 19168 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:05.388952 19130 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:05.636636 19000 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.527134 19261 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:05.623850 19310 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:05.617812 19353 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:23:05.661931 19326 log.cc:927] Time spent T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.538s	user 0.002s	sys 0.000s
I20260709 12:23:05.662058 19260 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:05.665164 19361 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:05.758729 19276 consensus_queue.cc:1048] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:23:05.761353 19361 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:05.762760 19100 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:05.669241 19366 consensus_queue.cc:1059] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:05.725322 19222 raft_consensus.cc:740] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:05.637457 19357 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:05.613400 19348 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:05.677071 19350 consensus_queue.cc:237] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:05.586239 19355 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:05.686280 19330 consensus_queue.cc:260] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:05.569976 19220 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:05.701041 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 404314us
W20260709 12:23:05.703415 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 353938us
W20260709 12:23:05.704629 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 260987us
I20260709 12:23:05.566712 19345 consensus_queue.cc:1059] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.185s
I20260709 12:23:05.529412 19335 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:05.707832 19364 log.cc:927] Time spent T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.070s	user 0.003s	sys 0.003s
I20260709 12:23:05.687798 19105 raft_consensus.cc:2804] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:05.749918 17263 raft_consensus.cc:2393] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 8: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:05.753966 17143 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:05.625872 19340 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:05.768918 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:05.754415 19108 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.763494 19361 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:05.759905 19130 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:05.807164 19108 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:05.797771 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:05.802328 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:05.810693 17271 raft_consensus.cc:2410] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:23:05.768764 19086 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:05.801034 16476 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
W20260709 12:23:05.813390 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:05.810964 16400 raft_consensus.cc:1217] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Deduplicated request from leader. Original: 9.6->[11.7-11.7]   Dedup: 11.7->[]
I20260709 12:23:05.814702 16400 raft_consensus.cc:1240] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:23:05.817159 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:05.817893 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:05.818645 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:05.817989 16514 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:05.828720 16396 raft_consensus.cc:1275] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:05.830578 16477 raft_consensus.cc:2393] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 8: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:05.764110 19261 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:05.806273 19260 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:05.799603 19105 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:05.842449 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:05.835281 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 47948.4us
I20260709 12:23:05.759594 19000 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:05.845114 17258 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:23:05.837625 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:05.838670 17254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 837) took 2178 ms. Trace:
W20260709 12:23:05.838866 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 87827.2us
I20260709 12:23:05.839051 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:05.767379 19353 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:05.811589 19330 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:05.858538 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:05.799108 19250 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:05.831105 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:05.840798 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:23:05.860780 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:05.836880 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:05.773017 19357 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:05.839530 19084 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:05.847699 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:05.847452 19260 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:05.851672 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:05.852123 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:05.852862 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:05.849618 16471 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:05.872228 16549 raft_consensus.cc:2393] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 11: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
W20260709 12:23:05.873454 19367 log.cc:927] Time spent T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:23:05.873911 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:05.855405 19348 raft_consensus.cc:2804] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:05.854418 19368 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:05.855566 16516 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:05.864080 16542 raft_consensus.cc:2393] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 8: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:05.839907 19223 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.846697 19345 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.847699 17255 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:23:05.849220 17254 rpcz_store.cc:276] 0709 12:23:03.660312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:04.194788 (+534476us) service_pool.cc:224] Handling call
0709 12:23:04.195249 (+   461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:04.914410 (+719161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:05.838442 (+924032us) spinlock_profiling.cc:243] Waited 701 ms on lock 0x7b540005f0d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:05.838617 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1612794112}
I20260709 12:23:05.846037 19105 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:05.870193 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:05.868625 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:05.842625 19369 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:05.854359 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:05.873312 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:05.862144 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:05.876206 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:05.847482 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:05.892925 16470 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:05.868177 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:05.788825 19355 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:05.862713 17271 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:23:05.884135 16441 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:05.892952 16550 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:23:05.893972 16400 raft_consensus.cc:2368] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6.
I20260709 12:23:05.890337 16473 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:23:05.891664 16543 raft_consensus.cc:1275] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:23:05.871622 19130 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:05.868970 19243 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:05.877878 19260 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:05.924790 19360 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:05.927924 16441 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:05.928022 16516 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:05.928747 19385 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:05.928985 16516 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:05.929261 19385 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:05.929196 19360 consensus_queue.cc:1048] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:05.929868 19385 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:05.930955 19371 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:05.931609 19371 raft_consensus.cc:697] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:05.932694 19318 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:05.932492 19371 consensus_queue.cc:237] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:05.933415 19340 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:05.891100 19368 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:05.934414 16472 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:23:05.881143 19348 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:05.886180 18581 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:05.863745 19250 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:05.866670 19376 consensus_queue.cc:1059] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 12:23:05.896571 19140 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:05.947321 16476 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
I20260709 12:23:05.950558 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:05.909859 19168 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:23:05.877440 19084 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:05.938004 19368 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:05.945874 19348 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:05.901732 19223 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:05.948158 19108 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:05.935802 19243 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:05.900887 19345 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:05.908977 19369 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:06.017228 19310 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:06.028266 19372 log.cc:927] Time spent T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.274s	user 0.001s	sys 0.000s
I20260709 12:23:06.040762 17270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 823) took 2415 ms. Trace:
I20260709 12:23:06.049039 17270 rpcz_store.cc:276] 0709 12:23:03.628970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:03.939935 (+310965us) service_pool.cc:224] Handling call
0709 12:23:04.830040 (+890105us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:04.866647 (+ 36607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:06.040475 (+1173828us) spinlock_profiling.cc:243] Waited 890 ms on lock 0x7b540063d650. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:06.040672 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2046000384}
I20260709 12:23:05.952036 18581 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:06.069339 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:06.053622 19369 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:06.061575 19335 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:06.096977 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:06.072228 19261 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:06.076009 19220 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:06.111022 18581 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:06.114053 19238 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:06.117007 19357 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:06.112128 19150 raft_consensus.cc:3055] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:06.120837 19150 raft_consensus.cc:740] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:06.121206 19355 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:06.121855 19355 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:06.122046 19150 consensus_queue.cc:260] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:06.090401 19353 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:06.122416 19355 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:06.100962 19000 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:06.129738 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:06.124280 19150 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:06.106005 19385 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:06.140256 19385 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:06.141088 17263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45230 (request call id 821) took 1291 ms. Trace:
I20260709 12:23:06.141280 16548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 518) took 1290 ms. Trace:
I20260709 12:23:06.141520 17263 rpcz_store.cc:276] 0709 12:23:04.850125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:04.928480 (+ 78355us) service_pool.cc:224] Handling call
0709 12:23:06.140900 (+1212420us) spinlock_profiling.cc:243] Waited 758 ms on lock 0x7b5400665dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:06.141034 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1743243648}
I20260709 12:23:06.141630 16548 rpcz_store.cc:276] 0709 12:23:04.851262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:04.851505 (+   243us) service_pool.cc:224] Handling call
0709 12:23:04.851825 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:04.852049 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:23:04.868092 (+ 16043us) log.cc:844] Serialized 14 byte log entry
0709 12:23:06.016895 (+1148803us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b180020c980. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a2923a 00007f2875e93a2a
0709 12:23:06.140937 (+124042us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:06.140986 (+    49us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:06.141047 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:06.141125 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:06.141230 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15935,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":6164,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":84,"spinlock_wait_cycles":31440256,"thread_start_us":115590,"threads_started":1}
I20260709 12:23:06.144764 19328 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:06.146024 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:06.146265 19084 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:06.146679 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:06.146745 16547 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6.
I20260709 12:23:06.147878 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:06.147955 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:06.148655 16547 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:06.149775 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:06.150656 16544 raft_consensus.cc:2410] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:23:06.108134 19250 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:06.154026 16441 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:06.156622 16547 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
I20260709 12:23:06.152149 17144 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:06.160112 16441 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:06.165463 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:06.165877 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:06.166785 16476 raft_consensus.cc:2393] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 9: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:06.168754 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:06.169878 16474 raft_consensus.cc:2410] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
W20260709 12:23:06.170740 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:06.162555 19238 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:06.162595 19385 raft_consensus.cc:2804] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:06.172097 19130 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.179366 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:06.190724 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:06.177134 19385 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:06.177315 19355 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:06.192292 19340 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:06.195040 19386 log.cc:927] Time spent T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:23:06.196660 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:06.204715 19000 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:06.205435 19000 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:06.207723 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:06.208228 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:06.212534 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:06.214077 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:06.193686 19385 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:06.179271 19250 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:06.217082 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:06.219159 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:06.220419 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:06.221138 16470 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:06.219376 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:06.222273 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:06.192793 19238 raft_consensus.cc:697] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:06.206346 19243 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:06.191588 19376 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:06.200079 19309 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:06.177340 19130 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:06.232527 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:06.233197 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:06.234354 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:23:06.241760 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:06.242292 19100 consensus_queue.cc:1059] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:06.243783 19243 raft_consensus.cc:3055] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:06.244263 19243 raft_consensus.cc:740] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:06.245525 19243 consensus_queue.cc:260] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:06.246483 17263 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:23:06.247429 19243 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:06.195061 19340 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:06.252920 19250 raft_consensus.cc:697] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:06.254004 19250 consensus_queue.cc:237] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:06.257707 19238 consensus_queue.cc:237] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:06.261617 16441 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:23:06.262413 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:06.262804 16441 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:06.205432 19260 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:06.266237 19260 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:06.250299 19223 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:06.268512 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:06.278270 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:06.278839 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:06.280236 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:06.248862 19309 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:06.281383 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:06.281451 19337 raft_consensus.cc:3055] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:06.282037 19337 raft_consensus.cc:740] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:06.251266 19384 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.282722 19309 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:06.282840 19384 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:06.283581 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:06.284469 16547 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:23:06.285104 19353 consensus_queue.cc:1059] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:06.286490 19370 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:23:06.286880 19389 log.cc:927] Time spent T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:23:06.287106 19370 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:06.287523 19370 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:06.285909 19388 log.cc:927] Time spent T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.169s	user 0.000s	sys 0.004s
I20260709 12:23:06.290715 19340 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:06.291405 17258 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:23:06.292521 16542 raft_consensus.cc:1240] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 12:23:06.247566 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:06.300246 17263 raft_consensus.cc:2376] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8. Re-sending same reply.
I20260709 12:23:06.299561 19356 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.303089 19353 consensus_queue.cc:1243] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer ef57beab0e1d40d585983873c55107e8 log is divergent from this leader: its last log entry 7.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:23:06.303596 19356 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:06.306191 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:06.307140 17143 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:06.299914 16515 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:06.310912 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:06.311693 17271 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:23:06.313222 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:06.285756 19384 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:06.323597 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:06.326283 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:06.326479 17271 raft_consensus.cc:2468] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
I20260709 12:23:06.327528 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:06.328491 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:06.328897 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:06.329562 16551 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:23:06.324015 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:06.331373 16441 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:06.329128 17271 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:23:06.334156 16515 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:06.333916 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:06.231030 19376 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:06.337625 19376 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:06.309051 19330 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:06.340577 19330 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:06.340967 19330 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:06.341710 19391 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:06.342198 19391 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:06.342574 19391 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:06.345100 19000 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:06.349818 19356 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:06.325399 19340 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:06.350776 19340 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:06.351197 19340 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:06.320955 19390 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:06.354092 19390 raft_consensus.cc:697] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:06.358415 19266 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:06.358639 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:06.360685 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:06.362010 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:06.364296 19105 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:06.365489 17264 raft_consensus.cc:1240] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:06.369871 16441 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:06.370699 19353 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:06.371070 19353 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:06.371428 19353 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:06.372663 16441 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:06.374279 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:06.375172 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:06.376813 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:06.378751 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:06.332525 17143 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:06.379567 16469 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 7: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
W20260709 12:23:06.379637 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 47187.1us
I20260709 12:23:06.372758 16442 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:06.380632 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:06.377949 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:06.385257 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:06.390323 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:06.391315 17271 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:23:06.394407 16550 raft_consensus.cc:1217] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Deduplicated request from leader. Original: 6.4->[7.5-7.5]   Dedup: 7.5->[]
I20260709 12:23:06.395022 16550 raft_consensus.cc:1240] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:23:06.395991 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:06.397351 17263 raft_consensus.cc:2410] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:23:06.402148 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:06.403113 16469 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:23:06.405966 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:06.411914 16367 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:06.404964 16514 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:06.413302 16550 raft_consensus.cc:1240] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:06.373263 19344 raft_consensus.cc:3055] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:06.367254 19391 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:06.347689 19348 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:06.426544 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:06.439116 16441 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:06.428061 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:06.424813 19184 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:06.429164 19369 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:06.430981 16442 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:06.392637 19000 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:06.355063 19390 consensus_queue.cc:237] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:06.381372 19361 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:06.392482 19223 raft_consensus.cc:2804] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:06.407200 19243 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.447288 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:06.449700 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:06.450706 17271 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:06.441286 17264 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:06.448673 19361 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:06.444411 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:06.452719 19355 consensus_queue.cc:1059] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:06.453480 19330 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:06.454365 19361 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:06.444715 19223 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:06.455703 19223 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:06.454985 16472 raft_consensus.cc:1240] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:06.456554 19243 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:06.457120 19226 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:06.433534 18581 raft_consensus.cc:2749] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:06.458045 19391 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:06.458496 18581 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:06.459331 19105 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:06.459472 16546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 522) took 1054 ms. Trace:
I20260709 12:23:06.459929 16546 rpcz_store.cc:276] 0709 12:23:05.405163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:05.405560 (+   397us) service_pool.cc:224] Handling call
0709 12:23:05.405998 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:05.629159 (+223161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:06.459249 (+830090us) spinlock_profiling.cc:243] Waited 222 ms on lock 0x7b54008c41d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:06.459416 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":511467264}
I20260709 12:23:06.460373 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:06.429524 19344 raft_consensus.cc:740] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:06.461503 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:06.463846 19105 consensus_queue.cc:1048] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:06.464205 17264 raft_consensus.cc:2393] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 6: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:06.467036 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:06.467231 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:06.456204 19330 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:06.469960 19130 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:06.470861 17266 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45224 (request call id 839) took 1783 ms (client timeout 1695 ms). Trace:
I20260709 12:23:06.471402 19223 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:06.473085 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:06.474001 16543 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:23:06.474442 16515 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:06.475034 16550 raft_consensus.cc:2393] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 9: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:06.482836 16549 raft_consensus.cc:2393] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 5: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:06.475478 17143 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:23:06.471280 17266 rpcz_store.cc:269] 0709 12:23:04.687206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:04.687391 (+   185us) service_pool.cc:224] Handling call
0709 12:23:06.470663 (+1783272us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b540005ebd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:06.470788 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2681913088}
I20260709 12:23:06.442999 19184 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:06.474510 16442 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:06.479547 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:06.492679 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:06.444280 19000 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:06.498677 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:06.499579 17266 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:06.501259 17271 raft_consensus.cc:2393] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 13: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:06.494920 19184 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:06.503223 16442 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:06.485185 19334 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:23:06.472645 19215 consensus_queue.cc:1048] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 12:23:06.484450 19243 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:06.508831 16442 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:06.510753 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:06.516271 16477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34826 (request call id 1041) took 1014 ms. Trace:
I20260709 12:23:06.512409 16396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 688) took 1377 ms. Trace:
I20260709 12:23:06.519470 16477 rpcz_store.cc:276] 0709 12:23:05.501581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:05.501755 (+   174us) service_pool.cc:224] Handling call
0709 12:23:06.515997 (+1014242us) spinlock_profiling.cc:243] Waited 267 ms on lock 0x7b54002e63d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:06.516194 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":613071744}
I20260709 12:23:06.516358 16403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 693) took 1379 ms. Trace:
I20260709 12:23:06.517931 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:06.499260 19000 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:06.523175 16403 rpcz_store.cc:276] 0709 12:23:05.136841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:05.569082 (+432241us) service_pool.cc:224] Handling call
0709 12:23:05.630441 (+ 61359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:05.630634 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:05.630657 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:05.630703 (+    46us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:05.630748 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:05.630802 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:06.516175 (+885373us) spinlock_profiling.cc:243] Waited 60.8 ms on lock 0x7b54004eded0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:06.516275 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":139852032}
I20260709 12:23:06.532063 17264 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:06.500579 19105 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:06.532629 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:06.517874 16398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 966) took 1513 ms. Trace:
I20260709 12:23:06.533758 16398 rpcz_store.cc:276] 0709 12:23:05.004836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:05.037731 (+ 32895us) service_pool.cc:224] Handling call
0709 12:23:05.038192 (+   461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:05.060421 (+ 22229us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:05.060461 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:05.060516 (+    55us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:05.060578 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:06.517676 (+1457098us) spinlock_profiling.cc:243] Waited 21.9 ms on lock 0x7b5400543154. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:06.517719 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:06.517825 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":50483072}
I20260709 12:23:06.519701 16396 rpcz_store.cc:276] 0709 12:23:05.135070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:05.564358 (+429288us) service_pool.cc:224] Handling call
0709 12:23:05.565064 (+   706us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:05.829288 (+264224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:06.512114 (+682826us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b54004ebbd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:06.512301 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":582696192}
I20260709 12:23:06.536237 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:06.537782 16442 leader_election.cc:400] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6.
I20260709 12:23:06.538141 16442 leader_election.cc:403] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:23:06.541527 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:06.537135 17269 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:23:06.504714 19250 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:06.479250 19370 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:06.548713 16442 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:06.516073 19369 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:06.500003 19384 consensus_queue.cc:1059] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:23:06.576612 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:06.577450 17255 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:23:06.579957 17255 raft_consensus.cc:1240] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:06.533082 19105 raft_consensus.cc:697] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:06.515791 19368 consensus_queue.cc:1059] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 11.7, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 12:23:06.548036 19334 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6.
W20260709 12:23:06.588274 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 149232us
I20260709 12:23:06.550452 19250 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:06.571166 19369 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:06.563575 19407 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:06.575543 19355 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:06.574065 19230 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:06.604110 16441 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:06.618374 16441 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:06.626533 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:23:06.594805 19398 log.cc:927] Time spent T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
W20260709 12:23:06.634739 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 222961us
I20260709 12:23:06.634867 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:06.665823 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:06.595988 19250 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:06.668421 16396 raft_consensus.cc:2393] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 7: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:06.670048 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:06.668695 16399 raft_consensus.cc:2393] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 10: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:06.638072 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:06.637079 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:06.648077 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:06.673733 16547 raft_consensus.cc:1275] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:06.672242 16398 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:06.674685 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:06.675386 16403 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:23:06.675441 16394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 687) took 1540 ms. Trace:
I20260709 12:23:06.522507 19344 consensus_queue.cc:260] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:06.689073 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:06.601658 19368 consensus_queue.cc:1243] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer f99066d13e524238aec6aa2024a8bd50 log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:23:06.676010 16394 rpcz_store.cc:276] 0709 12:23:05.134665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:05.531095 (+396430us) service_pool.cc:224] Handling call
0709 12:23:05.531476 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:05.531748 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:23:05.582456 (+ 50708us) log.cc:844] Serialized 14 byte log entry
0709 12:23:06.257315 (+674859us) spinlock_profiling.cc:243] Waited 3.14 ms on lock 0x7b18000197e0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a2923a 00007f2875e93a2a
0709 12:23:06.675015 (+417700us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:06.675069 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:06.675129 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:06.675261 (+   132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:06.675370 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":159427,"prepare.queue_time_us":50572,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":3490,"raft.queue_time_us":175734,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":9818,"spinlock_wait_cycles":7222528,"thread_start_us":264175,"threads_started":2,"wal-append.queue_time_us":902805}
I20260709 12:23:06.691190 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:06.622115 19230 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:06.702250 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:06.635954 19086 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:06.693346 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:06.614612 19369 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:06.696229 16403 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:23:06.613195 19220 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.703740 16394 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:23:06.723663 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:06.724962 17255 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:06.726145 16441 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:06.711649 16398 raft_consensus.cc:1275] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:23:06.618569 19409 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:06.674108 19391 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:06.704553 19350 consensus_queue.cc:1059] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 12:23:06.746228 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:06.747747 16397 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 7: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:06.722618 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:06.621553 19239 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:06.733727 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:06.754839 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:06.757460 16396 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:23:06.693152 19223 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:06.709085 19230 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:23:06.750485 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:06.712587 19344 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:06.732731 19409 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:06.758108 16397 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:23:06.767122 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:06.768290 16394 raft_consensus.cc:2393] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 7: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:06.722565 19348 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:06.781776 16397 raft_consensus.cc:1275] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:23:06.786509 17144 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:06.790803 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:06.792577 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:06.793454 16477 raft_consensus.cc:2410] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:23:06.794606 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:06.761929 19330 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:23:06.793562 17144 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:06.780085 19385 consensus_meta.cc:287] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.553s	user 0.011s	sys 0.000s
I20260709 12:23:06.801806 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:06.801674 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:06.757288 16367 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:06.803194 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:06.803872 16396 raft_consensus.cc:2376] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7. Re-sending same reply.
I20260709 12:23:06.776461 19241 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:06.748591 19340 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:06.814705 17264 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:06.816284 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:06.817131 17264 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:06.811728 16394 raft_consensus.cc:2393] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 8: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:06.812161 16397 raft_consensus.cc:2410] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:23:06.813406 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:06.834240 16402 raft_consensus.cc:1275] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:23:06.791543 19215 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:06.794091 19260 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:06.716017 19105 consensus_queue.cc:237] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:06.840148 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:06.752955 19220 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:06.842468 17264 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
W20260709 12:23:06.842868 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 49402.8us
I20260709 12:23:06.777207 19410 consensus_queue.cc:1059] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 12:23:06.842707 16397 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 4.
I20260709 12:23:06.794308 19081 consensus_queue.cc:1048] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 12:23:06.776273 19239 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:06.811364 19308 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:06.848843 17264 raft_consensus.cc:1275] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:23:06.855363 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:06.839354 19122 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.861689 16397 raft_consensus.cc:2393] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 8: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:06.851903 17261 raft_consensus.cc:2393] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 9: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:06.848901 17144 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:06.811408 19385 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:06.841357 19108 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.898095 17144 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:06.853788 19238 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:23:06.847617 19370 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:06.923707 16476 raft_consensus.cc:2393] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 7: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
W20260709 12:23:06.915637 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 158369us
I20260709 12:23:06.925046 16367 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:06.897608 19308 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:06.910611 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:06.937732 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:06.938684 16394 raft_consensus.cc:2410] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:23:06.928397 19086 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:06.939268 19086 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:06.940599 19220 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:06.941280 19220 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:06.941468 19086 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:06.941751 19220 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:06.942203 19220 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:06.944649 19238 raft_consensus.cc:2804] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:06.927412 16367 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:06.907342 19150 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:06.844111 19226 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:06.924645 19410 raft_consensus.cc:3055] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:06.848919 19184 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:06.962159 19150 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:06.934983 19415 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.990753 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:06.847061 19241 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:06.992583 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 65356.2us
I20260709 12:23:06.936604 19360 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:06.955255 19238 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:06.998839 19238 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:06.964434 19226 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:07.006281 16394 raft_consensus.cc:1275] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:23:06.969965 19410 raft_consensus.cc:740] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:06.992947 19150 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:06.953552 19108 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:07.014953 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:23:07.016021 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 118017us
I20260709 12:23:07.017509 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:07.017601 17144 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:07.005956 19051 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:07.020797 19051 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:07.022941 19051 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:07.023037 19308 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:07.004048 19419 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:07.025439 19419 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:06.910483 19122 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:07.026507 19419 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:07.032941 16473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1051) took 1095 ms. Trace:
I20260709 12:23:07.033366 16473 rpcz_store.cc:276] 0709 12:23:05.937191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:05.937360 (+   169us) service_pool.cc:224] Handling call
0709 12:23:05.937676 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:06.051823 (+114147us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:06.051868 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:06.051930 (+    62us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:06.051979 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:07.032702 (+980723us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b5400594a54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:07.032754 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:07.032878 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":261960064}
I20260709 12:23:07.034256 19150 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:07.036171 16475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1036) took 2216 ms. Trace:
I20260709 12:23:07.036348 16396 raft_consensus.cc:1240] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 6. Current term is 7. Ops: []
W20260709 12:23:07.037317 19413 log.cc:927] Time spent T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.199s	user 0.000s	sys 0.000s
I20260709 12:23:07.036788 16475 rpcz_store.cc:276] 0709 12:23:04.819416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:04.819960 (+   544us) service_pool.cc:224] Handling call
0709 12:23:04.824157 (+  4197us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:04.824563 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:04.824607 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:06.766138 (+1941531us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b2c0017a228. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:23:06.766717 (+   579us) log.cc:844] Serialized 52 byte log entry
0709 12:23:06.933639 (+166922us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:06.933681 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:06.933768 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:07.035866 (+102098us) raft_consensus.cc:1685] finished
0709 12:23:07.035886 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:07.036113 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17278,"raft.queue_time_us":103827,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":5726,"spinlock_wait_cycles":25155328,"thread_start_us":114290,"threads_started":1,"wal-append.queue_time_us":217941}
I20260709 12:23:07.038614 16396 raft_consensus.cc:1240] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:07.037750 19150 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:07.040046 19184 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:07.041986 19122 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:07.042043 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:07.043254 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:07.052839 17258 raft_consensus.cc:2410] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:23:07.056875 17255 raft_consensus.cc:2468] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
W20260709 12:23:07.058656 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:07.009856 19226 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:07.036079 19391 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:07.066135 17269 raft_consensus.cc:1240] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:23:06.965416 19222 consensus_queue.cc:260] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:07.069334 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:07.067608 17144 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:07.061085 19150 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:07.004483 19360 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:07.059885 19409 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:07.011762 19250 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:07.081110 17258 raft_consensus.cc:1240] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:07.001307 19415 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:07.074483 19340 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:07.096709 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 83996.3us
I20260709 12:23:07.098721 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:07.099519 16475 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:07.072839 19150 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:07.102520 16473 raft_consensus.cc:1240] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:07.065865 19417 consensus_queue.cc:1048] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
W20260709 12:23:07.100461 19353 consensus_meta.cc:287] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.729s	user 0.006s	sys 0.009s
I20260709 12:23:07.129521 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:07.093770 19385 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:07.131455 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 63894.7us
I20260709 12:23:07.134459 17144 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:07.127076 16441 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:07.134488 16394 raft_consensus.cc:2393] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 10: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:07.119570 16475 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:23:07.106829 19150 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:07.136476 16514 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:07.103268 19421 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:07.141404 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:07.080749 19410 consensus_queue.cc:260] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:07.119674 19239 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:07.117702 19330 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:07.144026 19081 consensus_queue.cc:1048] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:07.144886 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:07.146881 19108 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:07.133431 19353 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:07.151327 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:07.141417 19220 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:07.163308 16394 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:07.150138 16366 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
W20260709 12:23:07.147939 19000 consensus_meta.cc:287] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.623s	user 0.007s	sys 0.004s
I20260709 12:23:07.167393 19000 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:07.148604 19329 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:07.149312 19309 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:07.169492 16366 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:07.148769 19421 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:07.133267 19370 consensus_queue.cc:1048] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:23:07.180984 16366 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:07.135214 19348 consensus_queue.cc:1059] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:23:07.183805 16477 raft_consensus.cc:1240] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:07.184484 19309 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:07.187616 16395 raft_consensus.cc:2393] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 10: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:07.190614 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:07.198143 16394 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:07.183331 17258 raft_consensus.cc:1275] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:07.202121 17144 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:07.203190 16366 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:07.201368 16545 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:07.173969 19239 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:07.205726 16366 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:07.213263 16366 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:07.183058 19108 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:07.175787 19423 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:23:07.221910 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:07.164105 19330 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:07.210654 16441 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:07.220577 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:07.205267 19309 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:07.188827 19421 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:07.201172 19329 raft_consensus.cc:697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:07.219722 19409 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:07.212178 19424 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:07.212489 19412 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:07.199432 19150 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:07.224504 19241 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:07.190668 19419 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:07.236297 19239 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:07.246486 19108 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:07.237448 19423 raft_consensus.cc:697] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:07.241748 19086 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:07.275710 19250 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:07.259908 19105 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.287541 16397 raft_consensus.cc:1240] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:07.241490 19409 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:07.248548 19412 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:07.269419 19424 raft_consensus.cc:697] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:23:07.301620 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 91043.4us
I20260709 12:23:07.258946 19226 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:07.276574 19108 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:07.284413 19086 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:07.312906 17261 raft_consensus.cc:1275] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:07.285782 19239 raft_consensus.cc:697] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 7 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:07.296816 19409 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:07.318990 17261 raft_consensus.cc:1240] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:23:07.296787 19412 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:07.328634 17261 raft_consensus.cc:1275] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:23:07.329849 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:07.330614 16547 raft_consensus.cc:2393] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 11: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:07.314090 19108 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:07.316999 19086 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:07.351120 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:07.326586 19420 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.338195 19260 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:07.338071 19238 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:07.338380 19424 consensus_queue.cc:237] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:07.338833 19239 consensus_queue.cc:237] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:07.339897 19226 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:07.317229 19415 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:07.341010 19150 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:07.305855 19329 consensus_queue.cc:237] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:07.297504 19105 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:07.336580 19423 consensus_queue.cc:237] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:07.333024 19381 log.cc:927] Time spent T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 1.235s	user 0.008s	sys 0.000s
W20260709 12:23:07.365010 19426 log.cc:927] Time spent T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 12:23:07.357427 19353 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:07.376585 19411 log.cc:927] Time spent T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.464s	user 0.003s	sys 0.000s
I20260709 12:23:07.377879 19337 consensus_queue.cc:260] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:07.379578 19344 consensus_meta.cc:287] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.611s	user 0.009s	sys 0.000s
I20260709 12:23:07.387985 19419 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:07.392526 19369 consensus_meta.cc:287] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.639s	user 0.007s	sys 0.005s
I20260709 12:23:07.392719 19330 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:07.393030 19369 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:07.365278 19420 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:07.395128 19369 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:07.395429 19360 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:07.396476 19420 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:07.405258 19421 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:07.410705 19409 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:07.413508 19421 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:07.414096 19414 log.cc:927] Time spent T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.468s	user 0.002s	sys 0.000s
I20260709 12:23:07.414505 19409 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:07.406528 19412 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:07.410190 19345 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:07.418241 19410 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:07.419193 19412 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:07.467298 16396 raft_consensus.cc:2393] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 9: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
W20260709 12:23:07.468950 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:07.470899 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:07.351162 16545 raft_consensus.cc:1240] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:07.490021 17258 raft_consensus.cc:1240] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:07.357003 19000 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:07.360957 19220 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:07.505329 19238 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:07.512486 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:07.513849 16400 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:07.519897 16546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 547) took 1049 ms. Trace:
I20260709 12:23:07.527346 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:07.521212 16546 rpcz_store.cc:276] 0709 12:23:06.470107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:06.470345 (+   238us) service_pool.cc:224] Handling call
0709 12:23:06.470672 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:06.500596 (+ 29924us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:06.500634 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:06.501092 (+   458us) log.cc:844] Serialized 26 byte log entry
0709 12:23:06.833699 (+332607us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:23:06.833743 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:06.833821 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:07.413449 (+579628us) spinlock_profiling.cc:243] Waited 8.33 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:23:07.518449 (+105000us) raft_consensus.cc:1685] finished
0709 12:23:07.518969 (+   520us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:07.519436 (+   467us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29549,"raft.queue_time_us":29749,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":11531,"spinlock_wait_cycles":19167104,"thread_start_us":280449,"threads_started":1,"wal-append.queue_time_us":343662}
W20260709 12:23:07.527786 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 314433us
I20260709 12:23:07.568832 16366 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:07.572479 19105 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:07.577260 16475 raft_consensus.cc:1240] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:07.583424 19407 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:07.584659 19407 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:07.586501 16546 raft_consensus.cc:1240] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:07.587939 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:07.595749 19329 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.597292 19329 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:07.600495 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:07.604218 16399 raft_consensus.cc:2410] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:23:07.607295 19220 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:07.607465 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:07.608536 19220 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:07.609091 19407 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:07.615136 19355 consensus_queue.cc:1059] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:07.615914 19424 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:07.616935 19424 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:07.571899 19409 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:23:07.604111 19105 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.625910 19105 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:07.629976 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:07.630934 16399 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:07.631386 19356 consensus_queue.cc:1059] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:07.634183 16542 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6.
W20260709 12:23:07.641305 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 72831.5us
I20260709 12:23:07.640590 19105 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:07.619747 19424 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:07.642689 16366 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:07.647060 19239 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:07.630282 19329 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:07.653079 16401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 929) took 2751 ms. Trace:
I20260709 12:23:07.653613 16401 rpcz_store.cc:276] 0709 12:23:04.901402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:04.937283 (+ 35881us) service_pool.cc:224] Handling call
0709 12:23:05.412537 (+475254us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:05.412988 (+   451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:05.413656 (+   668us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:05.414440 (+   784us) log.cc:844] Serialized 26 byte log entry
0709 12:23:05.752630 (+338190us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:05.752977 (+   347us) log.cc:844] Serialized 14 byte log entry
0709 12:23:06.509162 (+756185us) spinlock_profiling.cc:243] Waited 1.2 ms on lock 0x7b180030c700. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d16c0a 00007f287c456219 00007f2872d9ea40 00007f2872d994e4 00007f2872d99455 00007f2875e81060 00007f2875e93716 00007f2875ea4287 00007f2875ea4208
0709 12:23:06.509330 (+   168us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:06.509405 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:06.509446 (+    41us) raft_consensus.cc:1685] finished
0709 12:23:06.509527 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:07.652689 (+1143162us) spinlock_profiling.cc:243] Waited 475 ms on lock 0x7b54004ec5d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:07.652870 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6863,"prepare.queue_time_us":861,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":822414,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":19311,"spinlock_wait_cycles":1094695680,"thread_start_us":225163,"threads_started":1,"wal-append.queue_time_us":275068,"wal-append.run_cpu_time_us":3457,"wal-append.run_wall_time_us":1158253}
I20260709 12:23:07.658077 16441 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:07.658334 19086 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:07.659889 19355 raft_consensus.cc:2804] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:07.660796 19355 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:07.661485 19355 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:07.652029 19407 consensus_queue.cc:1059] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 12:23:07.658203 16402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 939) took 2588 ms. Trace:
I20260709 12:23:07.676789 16402 rpcz_store.cc:276] 0709 12:23:05.070227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:05.303733 (+233506us) service_pool.cc:224] Handling call
0709 12:23:05.304124 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:06.834761 (+1530637us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:07.657637 (+822876us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b540015e854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:07.657995 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3518054144}
I20260709 12:23:07.677532 16367 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:07.678457 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:07.680816 19239 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:07.676283 16470 raft_consensus.cc:1240] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 4. Current term is 5. Ops: []
W20260709 12:23:07.683060 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:07.685293 16367 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:07.672612 19220 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:07.690838 16400 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
W20260709 12:23:07.693881 19230 consensus_meta.cc:287] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.913s	user 0.007s	sys 0.025s
I20260709 12:23:07.693723 19423 consensus_queue.cc:1059] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:07.694805 16477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1060) took 1354 ms. Trace:
I20260709 12:23:07.695235 16477 rpcz_store.cc:276] 0709 12:23:06.340517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:06.943492 (+602975us) service_pool.cc:224] Handling call
0709 12:23:06.943789 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:07.200487 (+256698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:07.694299 (+493812us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b5400a681d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:07.694615 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":551568256}
I20260709 12:23:07.698137 16441 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:07.695302 19423 raft_consensus.cc:3055] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:23:07.701125 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:07.701398 19423 raft_consensus.cc:740] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:07.702152 16402 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:07.700670 19360 raft_consensus.cc:2804] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:07.694574 19230 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:07.704418 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:07.707978 19220 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:07.710539 19360 raft_consensus.cc:697] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:07.713235 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:07.714138 16394 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6.
I20260709 12:23:07.700681 19385 consensus_queue.cc:1059] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 12:23:07.715212 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:07.716233 17144 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:07.717044 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:07.717656 17258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 865) took 1232 ms. Trace:
I20260709 12:23:07.717871 16470 raft_consensus.cc:1240] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:07.718168 17258 rpcz_store.cc:276] 0709 12:23:06.484926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:07.202557 (+717631us) service_pool.cc:224] Handling call
0709 12:23:07.218888 (+ 16331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:07.490877 (+271989us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:07.717288 (+226411us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b54005f1754. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:07.717521 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":620922624}
I20260709 12:23:07.721602 19105 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:07.722553 19105 raft_consensus.cc:697] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:07.726578 19356 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.727388 19356 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:07.728012 16542 raft_consensus.cc:1275] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:23:07.723917 19105 consensus_queue.cc:237] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:07.716081 16394 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:07.734530 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:07.739719 16399 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:07.730556 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:07.742172 19086 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:07.742494 16545 raft_consensus.cc:2410] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:23:07.744159 16542 raft_consensus.cc:1240] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:07.746342 16515 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:07.736932 19407 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.748932 19407 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:07.749205 17271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45230 (request call id 833) took 1338 ms. Trace:
I20260709 12:23:07.748984 19184 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:07.749955 19184 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:07.749812 17271 rpcz_store.cc:276] 0709 12:23:06.410370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:06.507703 (+ 97333us) service_pool.cc:224] Handling call
0709 12:23:07.748725 (+1241022us) spinlock_profiling.cc:243] Waited 563 ms on lock 0x7b540018a954. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:07.749003 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1294971648}
I20260709 12:23:07.750485 19184 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:07.751353 19407 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:07.752705 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:07.757614 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 72551.5us
I20260709 12:23:07.758586 18581 raft_consensus.cc:3055] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:07.759258 18581 raft_consensus.cc:740] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:07.760401 16401 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
W20260709 12:23:07.765964 19309 consensus_meta.cc:287] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.527s	user 0.015s	sys 0.015s
I20260709 12:23:07.767233 16474 raft_consensus.cc:1240] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:07.767616 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:07.768419 16394 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
I20260709 12:23:07.769440 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:07.769778 16474 raft_consensus.cc:1275] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:23:07.769711 17144 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:07.747831 16515 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:07.771325 18013 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:07.771791 18013 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:07.773852 19390 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:07.774268 16403 raft_consensus.cc:2393] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 6: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:07.775735 19230 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:07.775620 19384 consensus_queue.cc:1059] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:07.777199 16515 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:07.774297 19390 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:07.762331 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:07.778668 19390 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:07.778802 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:07.779708 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:07.780469 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:07.782099 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:07.782972 16547 raft_consensus.cc:2393] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 8: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:07.783879 19108 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:07.737676 19360 consensus_queue.cc:237] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:07.782243 19423 consensus_queue.cc:260] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:07.788196 19356 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:07.789218 19423 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:07.766693 19309 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:07.790510 18581 consensus_queue.cc:260] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:07.791640 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:07.772210 18013 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:07.791980 19309 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:07.792447 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:07.798022 16402 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:23:07.798981 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:07.799364 16401 raft_consensus.cc:1240] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:07.817180 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:07.808655 19425 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:07.818023 16542 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:23:07.812518 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:07.819823 17144 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:07.809703 19108 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:07.823014 16543 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:07.827183 16515 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:07.828490 19425 raft_consensus.cc:697] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:07.828547 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:07.824486 16474 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:07.829916 19425 consensus_queue.cc:237] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:07.830638 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:07.831616 16545 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:07.832722 19356 consensus_queue.cc:1048] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:07.837579 16546 raft_consensus.cc:1275] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:07.840374 19220 consensus_queue.cc:1048] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:23:07.844807 16476 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34826 (request call id 1055) took 1670 ms (client timeout 1870 ms). Trace:
W20260709 12:23:07.845362 16476 rpcz_store.cc:269] 0709 12:23:06.174310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:06.178922 (+  4612us) service_pool.cc:224] Handling call
0709 12:23:07.844251 (+1665329us) spinlock_profiling.cc:243] Waited 732 ms on lock 0x7b54004c0254. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:07.844549 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1683797376}
W20260709 12:23:07.847718 16549 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37242 (request call id 1022) took 1982 ms (client timeout 1830 ms). Trace:
W20260709 12:23:07.848316 16549 rpcz_store.cc:269] 0709 12:23:05.865270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:05.873015 (+  7745us) service_pool.cc:224] Handling call
0709 12:23:07.847167 (+1974152us) spinlock_profiling.cc:243] Waited 590 ms on lock 0x7b5400bb5654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:07.847461 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1356494592}
I20260709 12:23:07.848637 19184 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:07.855963 17143 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:07.868980 16470 raft_consensus.cc:1240] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:07.844897 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:07.847932 16551 raft_consensus.cc:1240] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:07.869608 16546 raft_consensus.cc:1240] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:09.546440 16546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1057) took 1700 ms. Trace:
I20260709 12:23:09.546831 16546 rpcz_store.cc:276] 0709 12:23:07.846437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:07.869003 (+ 22566us) service_pool.cc:224] Handling call
0709 12:23:07.869345 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:09.546143 (+1676798us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:09.546381 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:09.548090 19213 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.548740 19213 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:07.860195 19184 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:07.886399 19355 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:07.963388 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:09.625293 16396 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 4.
W20260709 12:23:09.625887 16396 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 991) took 1787 ms (client timeout 1988 ms). Trace:
W20260709 12:23:09.626645 16396 rpcz_store.cc:269] 0709 12:23:07.838573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:07.962986 (+124413us) service_pool.cc:224] Handling call
0709 12:23:09.625819 (+1662833us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:09.627048 19355 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:07.969049 16550 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37226 (request call id 1214) took 1544 ms (client timeout 1573 ms). Trace:
W20260709 12:23:09.630605 16550 rpcz_store.cc:269] 0709 12:23:06.425962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:06.426200 (+   238us) service_pool.cc:224] Handling call
0709 12:23:07.968354 (+1542154us) spinlock_profiling.cc:243] Waited 36.6 ms on lock 0x7b5400201054. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:07.968790 (+   436us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":84206336}
I20260709 12:23:07.988175 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:07.992983 19454 log.cc:927] Time spent T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.052s	user 0.002s	sys 0.000s
W20260709 12:23:08.016182 19455 log.cc:927] Time spent T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.122s	user 0.000s	sys 0.002s
W20260709 12:23:08.505970 16545 consensus_meta.cc:287] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.674s	user 0.016s	sys 0.004s
I20260709 12:23:07.825027 18581 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:23:08.539932 16403 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58430 (request call id 706) took 1818 ms (client timeout 1990 ms). Trace:
I20260709 12:23:09.632891 16545 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
W20260709 12:23:09.633035 16403 rpcz_store.cc:269] 0709 12:23:06.721884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:06.722161 (+   277us) service_pool.cc:224] Handling call
0709 12:23:08.539574 (+1817413us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400a79ed4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:08.539832 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2356035840}
W20260709 12:23:08.951870 16257 debug-util.cc:398] Leaking SignalData structure 0x7b08001af5e0 after lost signal to thread 5352
W20260709 12:23:09.634501 16257 debug-util.cc:398] Leaking SignalData structure 0x7b080033e3a0 after lost signal to thread 5355
I20260709 12:23:08.968212 19122 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:23:09.209841 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 1.35408e+06us
W20260709 12:23:09.229476 16441 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.444s	user 0.016s	sys 0.046s
W20260709 12:23:09.279794 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:09.276117 19360 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.386s	user 0.050s	sys 0.067s
I20260709 12:23:09.285218 16400 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
I20260709 12:23:09.297305 19391 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:23:09.312981 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:09.637017 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:09.319595 16397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 951) took 2294 ms. Trace:
I20260709 12:23:09.321203 19130 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:23:09.346033 19269 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:09.637943 19355 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:09.638430 19269 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:09.638700 19355 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:09.639717 19347 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:23:09.363065 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 66619us
W20260709 12:23:09.640712 16542 consensus_meta.cc:287] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 1.796s	user 0.018s	sys 0.007s
I20260709 12:23:09.640467 19347 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:09.641319 16542 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:09.641990 19266 raft_consensus.cc:2764] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:23:09.643057 19385 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:09.643378 19347 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:09.390769 19337 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:23:09.427349 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 1.62681e+06us
I20260709 12:23:09.430639 19000 raft_consensus.cc:2804] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:09.451404 19272 raft_consensus.cc:3055] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:09.465481 16398 raft_consensus.cc:1240] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:09.467963 19239 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:09.471443 19412 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:09.486454 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:09.487139 16516 connection.cc:256] RPC call timeout handler was delayed by -1.7091797359998964s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18700000000000003s for a response
W20260709 12:23:09.498286 16543 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52680 (request call id 553) took 1667 ms (client timeout 1718 ms). Trace:
W20260709 12:23:09.498440 19390 consensus_meta.cc:287] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 1.658s	user 0.026s	sys 0.020s
W20260709 12:23:09.508078 19051 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.321s	user 0.029s	sys 0.057s
I20260709 12:23:09.512036 19463 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:23:09.635804 16441 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.851s	user 0.017s	sys 0.046s
I20260709 12:23:09.644135 19347 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:09.699086 19463 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:09.699564 19347 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:09.701591 19463 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:09.527336 19465 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.531435 16471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 530) took 3373 ms. Trace:
I20260709 12:23:09.548990 16551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1215) took 3116 ms. Trace:
I20260709 12:23:09.549448 19215 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.689141 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:09.703155 19215 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:09.550829 19213 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:07.825223 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:09.774653 16402 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 10: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
W20260709 12:23:09.775180 16402 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 988) took 1982 ms (client timeout 2196 ms). Trace:
I20260709 12:23:09.553246 19466 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:09.776610 19466 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:09.779294 19466 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:09.553362 19410 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:09.780192 19410 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:09.783416 19410 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:09.554561 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:09.784303 16476 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:23:09.785748 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:09.786749 16476 raft_consensus.cc:2393] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 7: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:09.556408 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:09.787797 16472 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:09.558272 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:09.793084 16473 raft_consensus.cc:2410] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:23:09.559108 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:09.559906 19467 consensus_queue.cc:1059] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:09.796274 19467 raft_consensus.cc:3055] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:09.796718 19467 raft_consensus.cc:740] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:09.797555 16477 raft_consensus.cc:1240] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:09.797739 19467 consensus_queue.cc:260] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:09.799226 19467 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:09.562018 16442 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
W20260709 12:23:09.563301 16367 connection.cc:256] RPC call timeout handler was delayed by -1.7757764529997075s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1573s for a response
I20260709 12:23:09.569249 19361 raft_consensus.cc:3055] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:09.801337 19361 raft_consensus.cc:740] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:23:09.801555 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:09.572254 19184 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:09.802349 19361 consensus_queue.cc:260] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:09.802619 19184 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:09.803543 19361 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:09.577659 19108 consensus_queue.cc:1048] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:09.804929 19184 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:09.583213 17254 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.841s	user 0.008s	sys 0.017s
W20260709 12:23:09.806932 17254 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.065s	user 0.009s	sys 0.017s
I20260709 12:23:09.807881 17254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 848) took 3574 ms. Trace:
I20260709 12:23:09.808255 17254 rpcz_store.cc:276] 0709 12:23:06.233429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:06.269554 (+ 36125us) service_pool.cc:224] Handling call
0709 12:23:06.269778 (+   224us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:06.269964 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:23:06.309941 (+ 39977us) log.cc:844] Serialized 14 byte log entry
0709 12:23:07.741435 (+1431494us) spinlock_profiling.cc:243] Waited 1.6 ms on lock 0x7b18001e4c00. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a2923a 00007f2875e93a2a
0709 12:23:09.807387 (+2065952us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:09.807454 (+    67us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:09.807562 (+   108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:09.807669 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:09.807827 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":39832,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":3927,"raft.queue_time_us":86,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":158,"spinlock_wait_cycles":3669248,"thread_start_us":2065654,"threads_started":1,"wal-append.queue_time_us":1582427}
I20260709 12:23:09.810428 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:09.811105 17254 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:09.811477 19464 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:23:09.606985 19113 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.701s	user 0.010s	sys 0.005s
I20260709 12:23:09.616192 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:09.618969 16470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1076) took 1881 ms. Trace:
W20260709 12:23:09.632536 16395 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 990) took 1794 ms (client timeout 2156 ms). Trace:
I20260709 12:23:09.633479 16545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52680 (request call id 554) took 1804 ms. Trace:
I20260709 12:23:09.283732 16401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 728) took 1666 ms. Trace:
W20260709 12:23:09.813841 19113 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.695s	user 0.013s	sys 0.014s
I20260709 12:23:09.813877 16401 rpcz_store.cc:276] 0709 12:23:07.617462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:07.780640 (+163178us) service_pool.cc:224] Handling call
0709 12:23:07.780971 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:07.816304 (+ 35333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:09.283375 (+1467071us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b54001e2154. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:09.283623 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26603904}
I20260709 12:23:09.815534 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:23:09.815809 16395 rpcz_store.cc:269] 0709 12:23:07.838011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:07.987651 (+149640us) service_pool.cc:224] Handling call
0709 12:23:09.632475 (+1644824us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:09.815217 16470 rpcz_store.cc:276] 0709 12:23:07.737774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:07.737924 (+   150us) service_pool.cc:224] Handling call
0709 12:23:07.868279 (+130355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:09.493398 (+1625119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:09.618685 (+125287us) spinlock_profiling.cc:243] Waited 95.5 ms on lock 0x7b54004c0250. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:09.618871 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":219595520}
I20260709 12:23:09.817111 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:09.818131 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:09.818411 19113 raft_consensus.cc:3055] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:09.818167 16395 raft_consensus.cc:2393] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 10: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:09.819015 19113 raft_consensus.cc:740] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:09.820097 16395 raft_consensus.cc:1275] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:09.820747 16395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 729) took 2203 ms. Trace:
I20260709 12:23:09.820272 19113 consensus_queue.cc:260] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:09.821130 19466 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:09.821264 16545 rpcz_store.cc:276] 0709 12:23:07.829276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:07.830345 (+  1069us) service_pool.cc:224] Handling call
0709 12:23:09.633418 (+1803073us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:09.644163 19266 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:23:09.697085 16257 debug-util.cc:398] Leaking SignalData structure 0x7b0800293460 after lost signal to thread 16260
I20260709 12:23:09.821862 19466 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:09.823014 19466 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:09.823640 16257 debug-util.cc:398] Leaking SignalData structure 0x7b08001965a0 after lost signal to thread 16427
W20260709 12:23:09.824949 16257 debug-util.cc:398] Leaking SignalData structure 0x7b08002faba0 after lost signal to thread 16501
W20260709 12:23:09.825253 16257 debug-util.cc:398] Leaking SignalData structure 0x7b08002fab80 after lost signal to thread 16502
W20260709 12:23:09.775701 16402 rpcz_store.cc:269] 0709 12:23:07.793374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:07.818461 (+ 25087us) service_pool.cc:224] Handling call
0709 12:23:09.775119 (+1956658us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:09.658465 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:09.826268 17269 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6.
I20260709 12:23:09.826212 19266 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:09.827629 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:09.821094 16395 rpcz_store.cc:276] 0709 12:23:07.617794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:09.819428 (+2201634us) service_pool.cc:224] Handling call
0709 12:23:09.819716 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:09.820465 (+   749us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:09.820670 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:09.829181 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:09.830338 17269 raft_consensus.cc:2393] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 10: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:09.640740 19355 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:09.660218 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:09.832305 17263 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:23:09.833446 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:09.657006 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:09.834990 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:09.835868 17271 raft_consensus.cc:2368] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 13. Current term is 14.
I20260709 12:23:09.660897 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:09.836701 17258 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:23:09.838004 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:09.838888 17258 raft_consensus.cc:2393] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 7: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:09.839712 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:09.697934 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:09.702945 16551 rpcz_store.cc:276] 0709 12:23:06.432445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:06.432922 (+   477us) service_pool.cc:224] Handling call
0709 12:23:07.847438 (+1414516us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:09.500219 (+1652781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:09.548790 (+ 48571us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b5400bb5650. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:09.548913 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3252100480}
I20260709 12:23:09.703706 16396 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:23:09.830458 16395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 731) took 2201 ms. Trace:
I20260709 12:23:09.550004 19370 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:09.840818 16396 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 992) took 2001 ms (client timeout 2233 ms). Trace:
W20260709 12:23:09.704095 16543 rpcz_store.cc:269] 0709 12:23:07.828877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:07.844461 (+ 15584us) service_pool.cc:224] Handling call
0709 12:23:09.496309 (+1651848us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:09.704618 19465 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:09.841224 19370 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:09.704979 19412 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:09.652616 19122 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:09.843465 17263 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
W20260709 12:23:09.844254 16396 rpcz_store.cc:269] 0709 12:23:07.839044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:09.628710 (+1789666us) service_pool.cc:224] Handling call
0709 12:23:09.840741 (+212031us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:09.844779 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:09.844955 19122 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:09.636639 16400 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 979) took 2026 ms (client timeout 2057 ms). Trace:
I20260709 12:23:09.705859 19000 raft_consensus.cc:697] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:23:09.636231 19360 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.746s	user 0.053s	sys 0.067s
I20260709 12:23:09.706418 19272 raft_consensus.cc:740] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:09.840816 16395 rpcz_store.cc:276] 0709 12:23:07.629443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:09.829245 (+2199802us) service_pool.cc:224] Handling call
0709 12:23:09.829515 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:09.829844 (+   329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:09.829877 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:09.829956 (+    79us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:09.830013 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:09.830064 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:09.830163 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:09.637084 19391 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:09.847558 19391 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:09.669606 16542 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52680 (request call id 552) took 1860 ms (client timeout 1650 ms). Trace:
I20260709 12:23:09.707182 19239 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:09.707463 19230 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:09.659477 17255 raft_consensus.cc:1217] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Deduplicated request from leader. Original: 3.2->[6.3-6.3]   Dedup: 6.3->[]
I20260709 12:23:09.848598 17255 raft_consensus.cc:1240] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 12:23:09.848474 19230 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:09.709420 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 1.92435e+06us
I20260709 12:23:09.708488 19409 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:09.850143 19409 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:09.848026 16473 raft_consensus.cc:1240] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:09.657732 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:09.851679 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:09.852715 17270 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:09.853088 16470 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:09.853250 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:09.693590 19366 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.854151 19366 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:09.709153 19317 consensus_queue.cc:1059] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:23:09.711383 19344 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.711722 19215 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:09.856729 19344 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:09.857290 19215 consensus_queue.cc:1048] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:09.858959 19344 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:09.643764 19385 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:09.859773 16473 raft_consensus.cc:2410] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:23:09.692015 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:23:09.712632 18013 consensus_meta.cc:287] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 1.904s	user 0.009s	sys 0.024s
I20260709 12:23:09.860870 16399 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:09.860958 18013 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:09.685907 17264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 856) took 3449 ms. Trace:
I20260709 12:23:09.845450 17271 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:09.863371 18013 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:09.713040 19051 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.536s	user 0.038s	sys 0.061s
I20260709 12:23:09.862419 17264 rpcz_store.cc:276] 0709 12:23:06.235917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:06.863778 (+627861us) service_pool.cc:224] Handling call
0709 12:23:06.864211 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:06.864522 (+   311us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:23:06.911661 (+ 47139us) log.cc:844] Serialized 14 byte log entry
0709 12:23:07.722957 (+811296us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b180029c6e0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a2923a 00007f2875e93a2a
0709 12:23:09.685039 (+1962082us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:09.685160 (+   121us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:09.685333 (+   173us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:09.685460 (+   127us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:09.685679 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":46914,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":1766,"raft.queue_time_us":7499,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":110,"spinlock_wait_cycles":23458688,"thread_start_us":85980,"threads_started":1,"wal-append.queue_time_us":981178}
I20260709 12:23:09.707947 19390 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:09.864626 16401 raft_consensus.cc:2410] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:23:09.865046 19000 consensus_queue.cc:237] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:09.688493 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:09.637959 16397 rpcz_store.cc:276] 0709 12:23:07.024904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:07.025376 (+   472us) service_pool.cc:224] Handling call
0709 12:23:07.026336 (+   960us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:07.291430 (+265094us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:09.318940 (+2027510us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b54009be1d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:09.319390 (+   450us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":600170880}
I20260709 12:23:09.868158 17258 raft_consensus.cc:2393] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 9: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:09.640740 19269 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:09.708953 16516 leader_election.cc:336] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.504s (SENT)
I20260709 12:23:09.711437 16398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 955) took 2669 ms. Trace:
I20260709 12:23:09.709070 16471 rpcz_store.cc:276] 0709 12:23:06.158253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:06.158377 (+   124us) service_pool.cc:224] Handling call
0709 12:23:06.366116 (+207739us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:06.366319 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:06.366345 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:06.366388 (+    43us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:06.366428 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:06.366486 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:09.531029 (+3164543us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b540011add0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:09.531248 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":477008640}
I20260709 12:23:09.717702 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:09.718245 17144 connection.cc:256] RPC call timeout handler was delayed by -1.7162589760000628s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.199s for a response
I20260709 12:23:09.870055 17261 raft_consensus.cc:2393] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 5: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
W20260709 12:23:09.869843 16400 rpcz_store.cc:269] 0709 12:23:07.610297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:07.707210 (+ 96913us) service_pool.cc:224] Handling call
0709 12:23:09.636578 (+1929368us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:09.871776 17144 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:09.701907 19347 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:09.691195 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 76772.6us
W20260709 12:23:09.725535 16394 consensus_meta.cc:287] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 1.955s	user 0.019s	sys 0.006s
W20260709 12:23:09.679726 19425 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.841s	user 0.050s	sys 0.086s
W20260709 12:23:09.876287 19425 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.038s	user 0.051s	sys 0.088s
I20260709 12:23:09.728742 19463 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:09.876971 19471 consensus_queue.cc:1059] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:09.877430 19463 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:09.878225 19424 raft_consensus.cc:3055] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:09.878563 19424 raft_consensus.cc:740] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:09.525657 17142 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:09.733385 16548 raft_consensus.cc:1275] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:23:09.737186 19469 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:09.879437 19424 consensus_queue.cc:260] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:09.734575 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:09.738003 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:09.880106 19469 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:09.880517 19424 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:09.739666 16469 raft_consensus.cc:1275] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 3. (term mismatch)
I20260709 12:23:09.881433 16469 pending_rounds.cc:85] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 2
W20260709 12:23:09.750270 19423 consensus_meta.cc:287] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 1.960s	user 0.018s	sys 0.021s
I20260709 12:23:09.758351 19417 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.759523 19243 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.882921 19243 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:09.760532 19105 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:09.884965 19105 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:09.880545 19272 consensus_queue.cc:260] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:09.761458 19100 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.886720 19272 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:09.886425 19100 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:09.762472 19130 consensus_queue.cc:1048] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:09.889554 19130 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:23:09.800869 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 239141us
I20260709 12:23:09.812280 19464 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:09.890849 16442 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:09.891670 19347 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:09.892068 19347 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:09.892604 19347 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:09.877439 16395 raft_consensus.cc:1275] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:23:09.893385 16395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 983) took 2134 ms. Trace:
I20260709 12:23:09.893867 16395 rpcz_store.cc:276] 0709 12:23:07.759924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:09.876777 (+2116853us) service_pool.cc:224] Handling call
0709 12:23:09.877139 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:09.893087 (+ 15948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:09.893295 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:09.652508 19461 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.904762 19461 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:09.866102 19390 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:09.907549 19471 consensus_queue.cc:1048] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:23:09.848111 16542 rpcz_store.cc:269] 0709 12:23:07.809018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:07.809182 (+   164us) service_pool.cc:224] Handling call
0709 12:23:09.669489 (+1860307us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:09.876173 16394 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
W20260709 12:23:09.909178 16394 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 986) took 2147 ms (client timeout 2038 ms). Trace:
W20260709 12:23:09.909715 16394 rpcz_store.cc:269] 0709 12:23:07.761766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:07.769133 (+  7367us) service_pool.cc:224] Handling call
0709 12:23:09.909005 (+2139872us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:09.825521 19113 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:09.881712 16469 pending_rounds.cc:107] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 6.3
I20260709 12:23:09.911518 16469 raft_consensus.cc:2889] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:09.869427 16398 rpcz_store.cc:276] 0709 12:23:07.043172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:07.043401 (+   229us) service_pool.cc:224] Handling call
0709 12:23:09.465132 (+2421731us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:09.709393 (+244261us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:09.711252 (+  1859us) spinlock_profiling.cc:243] Waited 2.42 s on lock 0x7b5400a79ed0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:09.711385 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5567385472}
I20260709 12:23:09.913292 16469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 538) took 3707 ms. Trace:
I20260709 12:23:09.752646 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:09.913769 16469 rpcz_store.cc:276] 0709 12:23:06.205499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:06.413466 (+207967us) service_pool.cc:224] Handling call
0709 12:23:06.413731 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:09.912131 (+3498400us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:09.912953 (+   822us) spinlock_profiling.cc:243] Waited 3.33 s on lock 0x7b540080b1d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:09.913217 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7647653120}
I20260709 12:23:09.731841 19222 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:09.915493 19384 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.916695 19223 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.916805 19384 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:09.917408 19223 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:09.899011 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:09.918936 16471 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 5.
I20260709 12:23:09.913851 16398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 737) took 2141 ms. Trace:
I20260709 12:23:09.915102 16475 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:23:09.921801 16398 rpcz_store.cc:276] 0709 12:23:07.772007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:09.912848 (+2140841us) service_pool.cc:224] Handling call
0709 12:23:09.913253 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:09.913494 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:09.913521 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:09.913586 (+    65us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:09.913620 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:09.913681 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:09.913781 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:09.925312 16394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 984) took 2151 ms. Trace:
I20260709 12:23:09.928040 16394 rpcz_store.cc:276] 0709 12:23:07.760776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:09.910645 (+2149869us) service_pool.cc:224] Handling call
0709 12:23:09.911518 (+   873us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:09.911831 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:09.911872 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:09.911924 (+    52us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:09.912007 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:09.912094 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:09.912273 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:09.931174 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:09.932138 16394 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:09.908149 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:09.961150 16394 raft_consensus.cc:1275] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:23:09.996749 19461 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:09.882248 19417 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:09.927001 16398 raft_consensus.cc:1240] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 8. Current term is 9. Ops: []
W20260709 12:23:09.999097 16474 consensus_meta.cc:287] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 2.169s	user 0.019s	sys 0.003s
W20260709 12:23:09.996436 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 105671us
I20260709 12:23:10.003788 16442 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:09.831542 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:23:10.009279 16257 debug-util.cc:398] Leaking SignalData structure 0x7b08000a6000 after lost signal to thread 16575
W20260709 12:23:10.009752 16257 debug-util.cc:398] Leaking SignalData structure 0x7b0800020220 after lost signal to thread 16576
W20260709 12:23:10.018054 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 309145us
W20260709 12:23:10.021622 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 263594us
W20260709 12:23:10.023793 16257 debug-util.cc:398] Leaking SignalData structure 0x7b0800218520 after lost signal to thread 17299
W20260709 12:23:10.024531 16257 debug-util.cc:398] Leaking SignalData structure 0x7b08003b67c0 after lost signal to thread 17300
W20260709 12:23:10.025247 16257 debug-util.cc:398] Leaking SignalData structure 0x7b08003174c0 after lost signal to thread 19276
I20260709 12:23:10.000319 16474 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
W20260709 12:23:10.037966 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:10.018236 16395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 989) took 2183 ms. Trace:
I20260709 12:23:10.039587 16395 rpcz_store.cc:276] 0709 12:23:07.814143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:09.923969 (+2109826us) service_pool.cc:224] Handling call
0709 12:23:09.997036 (+ 73067us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:09.997273 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:09.997298 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:09.997342 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:09.997401 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:09.997472 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:09.997774 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:23:10.044760 16257 debug-util.cc:398] Leaking SignalData structure 0x7b08003aef80 after lost signal to thread 19340
W20260709 12:23:10.045154 16257 debug-util.cc:398] Leaking SignalData structure 0x7b080038dde0 after lost signal to thread 19348
W20260709 12:23:10.045460 16257 debug-util.cc:398] Leaking SignalData structure 0x7b080032e400 after lost signal to thread 19363
W20260709 12:23:10.045718 16257 debug-util.cc:398] Leaking SignalData structure 0x7b08000a1f60 after lost signal to thread 19371
W20260709 12:23:10.045997 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 167380us
W20260709 12:23:10.048756 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:10.049705 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:10.051429 16474 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53186 (request call id 552) took 2232 ms (client timeout 2432 ms). Trace:
W20260709 12:23:10.054123 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 45436.1us
W20260709 12:23:10.057399 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:10.059616 16257 debug-util.cc:398] Leaking SignalData structure 0x7b080032b580 after lost signal to thread 19402
W20260709 12:23:10.060014 16257 debug-util.cc:398] Leaking SignalData structure 0x7b080035b780 after lost signal to thread 19403
I20260709 12:23:10.068307 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:10.069763 16257 debug-util.cc:398] Leaking SignalData structure 0x7b080028f5e0 after lost signal to thread 19415
W20260709 12:23:10.069811 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:10.070570 16257 debug-util.cc:398] Leaking SignalData structure 0x7b080018b400 after lost signal to thread 19419
I20260709 12:23:10.077333 16472 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
W20260709 12:23:10.053948 16474 rpcz_store.cc:269] 0709 12:23:07.796072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:07.800245 (+  4173us) service_pool.cc:224] Handling call
0709 12:23:10.027949 (+2227704us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:23:10.094092 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 293195us
I20260709 12:23:10.103175 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:10.104555 16399 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
W20260709 12:23:10.105538 16399 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58430 (request call id 738) took 2313 ms (client timeout 1659 ms). Trace:
W20260709 12:23:10.108796 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 105116us
I20260709 12:23:10.109647 16442 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:10.110692 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:10.106034 16399 rpcz_store.cc:269] 0709 12:23:07.792622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:07.801128 (+  8506us) service_pool.cc:224] Handling call
0709 12:23:10.105448 (+2304320us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:10.111125 16395 raft_consensus.cc:1240] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:10.113099 16472 raft_consensus.cc:2393] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 10: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:10.044523 19461 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:23:10.104673 16548 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:23:10.118041 16542 raft_consensus.cc:3055] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:10.111707 16476 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:10.119036 16542 raft_consensus.cc:740] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:10.120048 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:10.052814 19100 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:10.125504 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:10.129511 16471 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:23:10.123189 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:23:10.126538 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 601121us
W20260709 12:23:10.127991 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 76617.9us
I20260709 12:23:10.131814 16477 raft_consensus.cc:2393] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 10: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
W20260709 12:23:10.133708 16367 connection.cc:256] RPC call timeout handler was delayed by -1.2510764529997687s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2156s for a response
I20260709 12:23:10.134229 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:10.141041 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:10.118388 19355 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.125130 19122 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.162431 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:10.150363 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:10.159269 16397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 735) took 2508 ms. Trace:
I20260709 12:23:10.135459 19243 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:10.163188 16402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 730) took 2545 ms. Trace:
W20260709 12:23:10.218178 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 58078.5us
W20260709 12:23:10.175648 16516 leader_election.cc:336] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.543s (SENT)
I20260709 12:23:10.143447 19360 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:10.164669 16542 consensus_queue.cc:260] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:10.232264 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:10.219777 16397 rpcz_store.cc:276] 0709 12:23:07.650418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:09.867281 (+2216863us) service_pool.cc:224] Handling call
0709 12:23:09.867617 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:09.867883 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:23:10.098318 (+230435us) log.cc:844] Serialized 14 byte log entry
0709 12:23:10.158652 (+ 60334us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:10.158807 (+   155us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:10.158853 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:10.159103 (+   250us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:10.159214 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":56198,"thread_start_us":267028,"threads_started":2}
I20260709 12:23:10.158159 19105 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:10.163584 19108 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:10.221714 19409 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:10.164964 19465 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:10.163188 19370 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:10.245791 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 136231us
I20260709 12:23:10.161867 19230 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:10.237802 19366 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:10.252687 19384 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:10.255577 19223 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:10.248497 17143 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:10.223882 19464 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:10.259457 16542 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:10.234208 19463 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:10.262180 16475 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 10: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:10.165676 19469 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:10.231822 19412 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:10.269778 16399 raft_consensus.cc:1275] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:23:10.177349 19122 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:10.162618 19355 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:10.240551 19385 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:10.286998 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:10.313629 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 236103us
W20260709 12:23:10.337646 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 174010us
I20260709 12:23:10.240170 19266 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:10.321801 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:10.319361 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:10.323005 16514 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:10.324895 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:10.332595 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:10.333598 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:10.317823 19108 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:10.258790 19105 consensus_queue.cc:1048] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.071s
I20260709 12:23:10.335938 16399 raft_consensus.cc:1275] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:23:10.345703 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:10.352988 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 81570.2us
W20260709 12:23:10.355608 17270 consensus_meta.cc:287] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.502s	user 0.006s	sys 0.009s
W20260709 12:23:10.355482 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 199760us
I20260709 12:23:10.356385 17270 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6.
I20260709 12:23:10.357775 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:10.358392 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:10.360538 16546 raft_consensus.cc:2393] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 8: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
W20260709 12:23:10.362241 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 204938us
I20260709 12:23:10.370407 16367 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:10.368640 19239 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:10.368398 19100 consensus_queue.cc:1059] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 12:23:10.384735 17142 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:10.388780 19355 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:10.393236 19477 log.cc:927] Time spent T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:23:10.394074 19384 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.394644 19384 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:10.396766 19384 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:10.381524 16516 leader_election.cc:336] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.547s (SENT)
I20260709 12:23:10.397641 16516 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:10.383770 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:10.382136 19480 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:23:10.400038 17254 consensus_meta.cc:287] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.588s	user 0.016s	sys 0.002s
I20260709 12:23:10.398741 16546 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:23:10.401054 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:10.400929 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:10.402647 17254 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:23:10.403424 18013 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:10.404191 16475 raft_consensus.cc:2393] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 7: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:10.378899 19473 raft_consensus.cc:2804] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:10.407274 16474 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:23:10.411811 19272 consensus_meta.cc:287] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.525s	user 0.008s	sys 0.008s
W20260709 12:23:10.420776 19222 consensus_meta.cc:287] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.505s	user 0.008s	sys 0.009s
I20260709 12:23:10.412830 19473 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:10.424650 19473 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:10.424742 19417 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:10.425030 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 45779.2us
I20260709 12:23:10.425918 17142 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:10.416199 17143 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:10.412632 19464 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:23:10.421366 19467 consensus_meta.cc:287] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.622s	user 0.013s	sys 0.005s
I20260709 12:23:10.419143 19243 consensus_queue.cc:1048] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:10.434020 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:10.219108 16402 rpcz_store.cc:276] 0709 12:23:07.618138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:09.826192 (+2208054us) service_pool.cc:224] Handling call
0709 12:23:09.826487 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:09.826884 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:23:10.108518 (+281634us) log.cc:844] Serialized 14 byte log entry
0709 12:23:10.152497 (+ 43979us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:10.152554 (+    57us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:23:10.162913 (+ 10359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:10.162993 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:10.163122 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51819,"prepare.queue_time_us":306303,"prepare.run_wall_time_us":51,"raft.queue_time_us":4992,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":3353,"spinlock_wait_cycles":28114688,"thread_start_us":324097,"threads_started":2}
I20260709 12:23:10.437753 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:10.438460 16402 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:23:10.434672 16398 raft_consensus.cc:2376] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8. Re-sending same reply.
I20260709 12:23:10.433717 19465 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:10.431368 17142 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:10.440155 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:10.402429 19480 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:10.444286 16542 raft_consensus.cc:1275] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:23:10.444819 16542 pending_rounds.cc:85] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1: Aborting all ops after (but not including) 5
I20260709 12:23:10.445179 16542 pending_rounds.cc:107] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:23:10.445519 16542 raft_consensus.cc:2889] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:10.443384 16399 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:23:10.447063 16468 raft_consensus.cc:1275] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:10.431924 19417 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:10.432214 19266 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:10.449146 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:10.457206 16475 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 10: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:10.443946 19465 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:10.463888 19335 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:10.464532 19361 consensus_meta.cc:287] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.661s	user 0.012s	sys 0.005s
I20260709 12:23:10.436371 19481 consensus_queue.cc:1048] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:23:10.468276 19473 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:10.470192 19347 consensus_meta.cc:287] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.577s	user 0.011s	sys 0.009s
I20260709 12:23:10.447757 19480 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:10.474510 16474 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
I20260709 12:23:10.484499 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:10.487723 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 110182us
W20260709 12:23:10.491019 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 51627.6us
W20260709 12:23:10.483722 19483 log.cc:927] Time spent T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:23:10.477267 19465 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:10.474337 19105 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:10.497717 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:10.500916 16548 raft_consensus.cc:1240] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 7. Current term is 8. Ops: []
W20260709 12:23:10.491451 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 106829us
W20260709 12:23:10.518770 16442 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 2.038s (SENT) [suppressed 20 similar messages]
W20260709 12:23:10.519493 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 91245us
I20260709 12:23:10.520534 17142 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:10.483067 19335 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:10.521994 17142 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
W20260709 12:23:10.487794 19113 consensus_meta.cc:287] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.576s	user 0.013s	sys 0.002s
I20260709 12:23:10.501694 17144 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:10.528699 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:23:10.533193 16516 leader_election.cc:336] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.609s (SENT)
I20260709 12:23:10.533346 16474 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:23:10.533089 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:10.534073 16473 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:10.535140 17270 raft_consensus.cc:1240] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:10.532733 16549 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:23:10.537798 16516 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:10.491722 19347 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:10.544757 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:10.581782 16403 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:23:10.542129 17144 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
W20260709 12:23:10.546499 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:10.548543 17261 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:23:10.532370 19473 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:10.531213 19495 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:10.543568 19481 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:10.590565 19347 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:10.537464 19122 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:10.593992 19480 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:10.595242 16473 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:23:10.537140 19108 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:10.547690 19424 consensus_meta.cc:287] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.666s	user 0.017s	sys 0.000s
I20260709 12:23:10.596885 19480 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:10.602180 19410 consensus_queue.cc:1048] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:10.604291 17269 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
W20260709 12:23:10.608062 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 66016.3us
I20260709 12:23:10.608620 19347 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:10.542816 19105 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:10.609206 19417 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:10.609782 19417 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:10.609519 19347 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:10.585209 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 54864.7us
I20260709 12:23:10.588539 19113 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:10.610255 19417 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:10.589118 17258 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6.
W20260709 12:23:10.610504 16516 connection.cc:256] RPC call timeout handler was delayed by -1.5706643510002323s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1738s for a response
I20260709 12:23:10.610723 16468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1067) took 4773 ms. Trace:
W20260709 12:23:10.610909 16516 connection.cc:256] RPC call timeout handler was delayed by -1.5683378399999128s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21880000000000002s for a response
W20260709 12:23:10.611232 16516 connection.cc:256] RPC call timeout handler was delayed by -1.534601979999934s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1834s for a response
I20260709 12:23:10.611142 16468 rpcz_store.cc:276] 0709 12:23:05.837741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:05.838929 (+  1188us) service_pool.cc:224] Handling call
0709 12:23:05.839199 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:10.453767 (+4614568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:10.610440 (+156673us) spinlock_profiling.cc:243] Waited 4.59 s on lock 0x7b540021cd54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:10.610648 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10568122240}
W20260709 12:23:10.554721 19466 consensus_meta.cc:287] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.731s	user 0.011s	sys 0.003s
W20260709 12:23:10.611657 16516 connection.cc:256] RPC call timeout handler was delayed by -1.5332378399998561s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2227s for a response
I20260709 12:23:10.609856 19105 raft_consensus.cc:697] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:23:10.612195 16516 connection.cc:256] RPC call timeout handler was delayed by -1.482946401999925s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1683s for a response
I20260709 12:23:10.612941 19105 consensus_queue.cc:237] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:10.614442 16514 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:10.615257 19409 consensus_queue.cc:1059] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:10.617686 19409 raft_consensus.cc:3055] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:10.618149 19409 raft_consensus.cc:740] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:10.617513 16225 catalog_manager.cc:5697] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 5 to 6, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 6 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:10.619227 19409 consensus_queue.cc:260] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:10.621215 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 255090us
W20260709 12:23:10.622277 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 84058.5us
W20260709 12:23:10.622225 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 100337us
W20260709 12:23:10.583994 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:10.617728 16468 raft_consensus.cc:1275] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:23:10.627126 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:10.628647 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:10.621204 16401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 981) took 2980 ms. Trace:
I20260709 12:23:10.632046 16401 rpcz_store.cc:276] 0709 12:23:07.640589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:09.865857 (+2225268us) service_pool.cc:224] Handling call
0709 12:23:09.866214 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:09.866592 (+   378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:09.866619 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:09.867206 (+   587us) log.cc:844] Serialized 26 byte log entry
0709 12:23:10.101081 (+233875us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:10.366410 (+265329us) log.cc:844] Serialized 14 byte log entry
0709 12:23:10.620830 (+254420us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:10.620910 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:10.620953 (+    43us) raft_consensus.cc:1685] finished
0709 12:23:10.620964 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:10.621118 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":254130,"prepare.queue_time_us":152208,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":98,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":189,"thread_start_us":498241,"threads_started":2,"wal-append.queue_time_us":374854}
I20260709 12:23:10.634615 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:10.634917 19335 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:10.629575 16543 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
W20260709 12:23:10.622825 16441 connection.cc:256] RPC call timeout handler was delayed by -1.0995959729998503s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20710000000000003s for a response
W20260709 12:23:10.638350 16441 connection.cc:256] RPC call timeout handler was delayed by -1.0950959729998431s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2076s for a response
W20260709 12:23:10.638901 16441 connection.cc:256] RPC call timeout handler was delayed by -1.0791444989995398s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1988s for a response
W20260709 12:23:10.549803 19490 log.cc:927] Time spent T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.150s	user 0.004s	sys 0.000s
I20260709 12:23:10.675472 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:10.635883 19239 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:23:10.645331 16516 connection.cc:256] RPC call timeout handler was delayed by -1.0599464020001506s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21530000000000002s for a response
I20260709 12:23:10.555444 19130 raft_consensus.cc:3055] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:10.646603 19347 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:10.647190 19480 consensus_queue.cc:1048] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:10.612205 19466 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:10.649845 19113 raft_consensus.cc:2804] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:10.653620 16394 raft_consensus.cc:1275] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:23:10.654101 16403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46250 (request call id 963) took 1198 ms. Trace:
I20260709 12:23:10.700290 16403 rpcz_store.cc:276] 0709 12:23:09.455634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:09.634041 (+178407us) service_pool.cc:224] Handling call
0709 12:23:09.734381 (+100340us) spinlock_profiling.cc:243] Waited 77.7 ms on lock 0x7b5800125198. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:23:10.653120 (+918739us) spinlock_profiling.cc:243] Waited 557 ms on lock 0x7b54008a6bd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:10.654039 (+   919us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1460076800}
I20260709 12:23:10.656607 17142 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:10.658784 16366 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:10.660027 16400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 736) took 2907 ms. Trace:
I20260709 12:23:10.660403 19465 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:10.662138 16396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 732) took 3032 ms. Trace:
I20260709 12:23:10.664588 19423 consensus_queue.cc:1048] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:10.665256 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:10.666368 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:10.666604 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:10.667369 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:10.707227 17254 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
I20260709 12:23:10.669420 16549 raft_consensus.cc:1275] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:23:10.668715 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:10.670495 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:10.693648 16441 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:10.708554 17255 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:23:10.698503 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:10.709472 16546 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:10.710651 17258 raft_consensus.cc:2410] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:23:10.707419 17271 raft_consensus.cc:2468] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6.
W20260709 12:23:10.711732 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 53009us
I20260709 12:23:10.702247 16400 rpcz_store.cc:276] 0709 12:23:07.752234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:09.870742 (+2118508us) service_pool.cc:224] Handling call
0709 12:23:09.871068 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:09.871327 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:23:10.163704 (+292377us) log.cc:844] Serialized 14 byte log entry
0709 12:23:10.659665 (+495961us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:10.659724 (+    59us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:10.659779 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:10.659848 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:10.659971 (+   123us) inbound_call.cc:177] Queueing success response
0709 12:23:10.693115 (+ 33144us) spinlock_profiling.cc:243] Waited 18.1 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
Metrics: {"mutex_wait_us":345076,"prepare.queue_time_us":381658,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":45,"raft.queue_time_us":345195,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":115,"spinlock_wait_cycles":41611648,"thread_start_us":438454,"threads_started":2,"wal-append.queue_time_us":205174}
I20260709 12:23:10.697964 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:10.669449 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:10.713723 17270 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:23:10.714136 18013 consensus_queue.cc:1059] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:10.710240 17143 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:10.713053 16550 raft_consensus.cc:2376] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9. Re-sending same reply.
I20260709 12:23:10.718473 18013 raft_consensus.cc:3055] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:10.718883 18013 raft_consensus.cc:740] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:23:10.719956 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:10.692842 19130 raft_consensus.cc:740] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:10.723512 19465 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:10.723773 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 67288.8us
I20260709 12:23:10.724547 16516 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:10.725564 19409 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:10.721824 16367 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:10.725998 16516 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:10.728574 19410 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:10.728971 19410 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:10.729297 19410 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:10.705099 19473 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.725471 17142 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:10.730692 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:10.713617 17264 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:23:10.723121 19130 consensus_queue.cc:260] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:10.705790 16396 rpcz_store.cc:276] 0709 12:23:07.629796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:09.845263 (+2215467us) service_pool.cc:224] Handling call
0709 12:23:09.865649 (+ 20386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:09.865866 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:23:10.060293 (+194427us) log.cc:844] Serialized 14 byte log entry
0709 12:23:10.608507 (+548214us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:10.608571 (+    64us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:10.608628 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:10.608704 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:10.661973 (+ 53269us) spinlock_profiling.cc:243] Waited 20 ms on lock 0x7b54009bc8d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:10.662081 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":386358,"prepare.queue_time_us":293903,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":2306,"raft.queue_time_us":390475,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":85,"spinlock_wait_cycles":46035840,"thread_start_us":284992,"threads_started":2,"wal-append.queue_time_us":291903}
I20260709 12:23:10.733381 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:10.736636 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:10.736745 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:10.731226 18013 consensus_queue.cc:260] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:10.739472 16549 raft_consensus.cc:2393] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 5: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:10.716140 19487 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:10.714052 19113 raft_consensus.cc:697] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:23:10.745324 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 51800.7us
W20260709 12:23:10.745980 19488 log.cc:927] Time spent T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.365s	user 0.002s	sys 0.000s
I20260709 12:23:10.746381 16441 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:10.716260 19239 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:10.741770 17254 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6.
I20260709 12:23:10.721743 19243 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:10.748545 19355 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:10.729804 19347 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:10.749920 18013 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:10.745995 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:10.754577 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:10.755137 16550 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:10.755259 16403 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:10.751753 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:10.748203 16441 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:10.735693 19266 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:10.737133 19184 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:23:10.757119 19184 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:10.757450 17269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45230 (request call id 847) took 1027 ms. Trace:
I20260709 12:23:10.753697 16548 raft_consensus.cc:1275] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:23:10.756698 16401 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:23:10.758539 19385 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:10.758980 19385 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:10.759063 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:10.759361 19385 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:10.759720 16401 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6.
I20260709 12:23:10.757865 17269 rpcz_store.cc:276] 0709 12:23:09.730383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:09.831176 (+100793us) service_pool.cc:224] Handling call
0709 12:23:10.757289 (+926113us) spinlock_profiling.cc:243] Waited 591 ms on lock 0x7b54005f6254. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:10.757393 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1360100736}
I20260709 12:23:10.759580 16441 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:10.763602 16394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 968) took 1276 ms. Trace:
I20260709 12:23:10.764163 19222 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:23:10.764999 19222 raft_consensus.cc:697] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 6 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:10.765383 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:10.765965 16397 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:10.765895 19222 consensus_queue.cc:237] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:10.769132 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:10.769972 16396 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:23:10.771339 19223 consensus_queue.cc:1059] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:10.772899 19223 raft_consensus.cc:3055] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:10.773238 17143 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:10.764159 16394 rpcz_store.cc:276] 0709 12:23:09.487643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:09.962326 (+474683us) service_pool.cc:224] Handling call
0709 12:23:10.653255 (+690929us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:10.699173 (+ 45918us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:10.763346 (+ 64173us) spinlock_profiling.cc:243] Waited 690 ms on lock 0x7b54008a6bd0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:10.763527 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1587649920}
I20260709 12:23:10.762174 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:10.737076 19481 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:10.724167 19465 consensus_queue.cc:1048] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
W20260709 12:23:10.783566 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:10.744153 19000 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:10.795415 16403 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:23:10.749138 19239 raft_consensus.cc:697] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 7 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:10.749215 19466 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:10.776983 19184 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:10.802801 17144 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:10.752403 19243 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:10.754361 19347 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:10.753798 19355 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:10.743934 19473 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:10.773258 19223 raft_consensus.cc:740] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:10.780972 19105 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:10.812805 16549 raft_consensus.cc:3055] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:10.806017 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:10.818331 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:10.828909 16549 raft_consensus.cc:740] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:10.784718 19266 consensus_queue.cc:1048] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
W20260709 12:23:10.829591 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 104279us
I20260709 12:23:10.850545 16396 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:10.752672 19487 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:10.824323 19424 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:10.849749 17263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 848) took 1093 ms. Trace:
I20260709 12:23:10.823433 19347 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:10.816823 19243 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:23:10.869902 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 95744.5us
I20260709 12:23:10.856805 16548 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:23:10.857935 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:10.875520 17270 raft_consensus.cc:2410] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 8 because replica has last-logged OpId of term: 7 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:23:10.875355 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:10.865411 19499 log.cc:927] Time spent T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:23:10.823993 19355 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:10.811785 19417 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:10.879206 17263 rpcz_store.cc:276] 0709 12:23:09.730690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:09.846451 (+115761us) service_pool.cc:224] Handling call
0709 12:23:09.846810 (+   359us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:09.847273 (+   463us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:09.847302 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:10.044384 (+197082us) log.cc:844] Serialized 52 byte log entry
0709 12:23:10.313198 (+268814us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:10.313267 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:10.313383 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:10.549067 (+235684us) spinlock_profiling.cc:243] Waited 3.9 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:23:10.814031 (+264964us) raft_consensus.cc:1685] finished
0709 12:23:10.814067 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:10.823727 (+  9660us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4837,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":138,"spinlock_wait_cycles":8959744,"thread_start_us":187714,"threads_started":1,"wal-append.queue_time_us":254998}
I20260709 12:23:10.870513 19105 raft_consensus.cc:697] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:10.887934 16549 consensus_queue.cc:260] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:10.908545 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:10.829917 19466 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 12:23:10.913050 19223 consensus_queue.cc:260] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:10.914387 19223 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:10.914511 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:10.917096 17142 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:10.917529 19105 consensus_queue.cc:237] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:10.918571 17142 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:10.920567 19000 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:10.880648 19113 consensus_queue.cc:237] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:10.921221 19000 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:10.922043 16549 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:10.927026 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:10.901713 19385 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:10.930658 19385 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:10.931255 16516 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:10.936681 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:10.937544 16402 raft_consensus.cc:2376] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8. Re-sending same reply.
I20260709 12:23:10.938930 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:10.939944 16394 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 10: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:10.951915 16402 raft_consensus.cc:1275] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:23:10.936204 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:10.935544 16225 catalog_manager.cc:5697] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:23:10.957374 16550 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:23:10.964761 16396 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:10.896615 19424 raft_consensus.cc:697] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:10.969645 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:10.982625 16547 raft_consensus.cc:2468] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6.
I20260709 12:23:10.860312 19239 consensus_queue.cc:237] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:10.995445 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:10.970562 17142 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:11.001698 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:11.002713 16547 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
I20260709 12:23:11.008155 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:10.969906 19266 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:23:11.024444 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 51528.9us
I20260709 12:23:11.026033 18013 consensus_queue.cc:1048] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:11.027879 16225 catalog_manager.cc:5697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 8 to 9, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 9 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
W20260709 12:23:11.035995 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 115648us
I20260709 12:23:11.044081 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:11.045015 16396 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:11.049315 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:11.049531 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:11.029388 19495 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:11.050278 16395 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
W20260709 12:23:11.072211 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 77101.2us
I20260709 12:23:11.073918 16225 catalog_manager.cc:5697] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 2 to 6, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 6 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:23:11.088927 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:11.099339 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:11.105602 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:11.150539 16395 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:23:11.141122 16468 raft_consensus.cc:2468] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:23:11.152200 16225 catalog_manager.cc:5697] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 2 to 7, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 7 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:23:11.156790 16397 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:23:11.156733 16396 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:11.159449 16549 raft_consensus.cc:1275] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:23:11.159650 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:11.160861 16225 catalog_manager.cc:5697] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 1 to 4, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 4 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
W20260709 12:23:11.155395 19486 log.cc:927] Time spent T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:23:11.169611 16514 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:11.180377 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:11.175487 17142 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:11.091506 19424 consensus_queue.cc:237] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:11.216619 16546 consensus_meta.cc:287] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.507s	user 0.010s	sys 0.002s
I20260709 12:23:11.218169 16546 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:11.218616 16225 catalog_manager.cc:5697] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:23:11.229183 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:11.230154 16395 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:23:11.204931 19272 raft_consensus.cc:2804] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:23:11.236124 19507 log.cc:927] Time spent T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
W20260709 12:23:11.235956 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 70353.2us
W20260709 12:23:11.238998 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:11.238070 16225 catalog_manager.cc:5697] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
I20260709 12:23:11.208869 19473 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:11.248059 18013 consensus_queue.cc:1048] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:11.233556 19464 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:23:11.249836 19000 consensus_queue.cc:1059] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
W20260709 12:23:11.253618 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 364315us
I20260709 12:23:11.237195 19272 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:11.247882 19508 log.cc:927] Time spent T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.155s	user 0.000s	sys 0.005s
I20260709 12:23:11.257510 16441 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:11.245012 19113 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.262290 16441 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:11.267057 16441 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:11.267964 16441 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:11.268823 16441 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:11.269703 19335 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:11.270133 19335 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:11.270519 19335 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:11.270613 19222 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:23:11.271003 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:11.271108 19222 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:11.271629 19222 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:11.269860 16441 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:11.274091 19517 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:11.274639 19517 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:11.274997 19517 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:23:11.266077 19513 log.cc:927] Time spent T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:23:11.278198 19505 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:11.278999 19505 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:11.279465 19505 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:11.281054 19506 raft_consensus.cc:2804] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:11.281543 19506 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:11.273844 16441 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:23:11.284519 16367 connection.cc:256] RPC call timeout handler was delayed by -1.5946546319996742s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2227s for a response
I20260709 12:23:11.261459 19272 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:23:11.285437 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 173700us
I20260709 12:23:11.281903 19506 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:23:11.289314 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:11.285138 16367 connection.cc:256] RPC call timeout handler was delayed by -1.5802072930000577s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23550000000000001s for a response
I20260709 12:23:11.272555 19466 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:23:11.295056 16224 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53636 (request call id 131) took 12554 ms (client timeout 15000 ms). Trace:
I20260709 12:23:11.297407 16548 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:23:11.315167 19487 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:11.291298 19522 raft_consensus.cc:2804] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:11.316143 19522 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:11.316632 19522 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:11.314922 19105 consensus_queue.cc:1048] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:23:11.317291 19501 log.cc:927] Time spent T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
W20260709 12:23:11.321143 19516 log.cc:927] Time spent T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:23:11.324954 19517 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:11.327641 19517 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:11.331532 19335 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:11.334184 19410 consensus_meta.cc:287] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.598s	user 0.010s	sys 0.002s
I20260709 12:23:11.334530 19335 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:11.346433 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 100480us
W20260709 12:23:11.316773 19512 log.cc:927] Time spent T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 12:23:11.334993 19410 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:11.355587 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 146369us
I20260709 12:23:11.277153 19113 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:11.305341 19466 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:11.376147 16398 raft_consensus.cc:2368] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 11. Current term is 12.
I20260709 12:23:11.370959 17142 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:11.378988 16516 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:11.380573 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 87317.8us
I20260709 12:23:11.382102 17143 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:11.354120 19473 consensus_queue.cc:1048] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:23:11.402595 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:11.368108 19466 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:11.369396 19487 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:23:11.403539 19510 log.cc:927] Time spent T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.299s	user 0.002s	sys 0.000s
I20260709 12:23:11.402956 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:11.403172 19481 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:11.429852 19417 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:11.406539 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:11.408074 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 48907.2us
I20260709 12:23:11.447686 17254 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:11.447999 17142 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:11.408505 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:11.410440 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:11.410606 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:11.411057 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:11.404423 19130 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:11.436612 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 78699.7us
I20260709 12:23:11.419497 16395 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6.
W20260709 12:23:11.438233 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 92124.5us
I20260709 12:23:11.400557 19464 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:11.439967 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:11.439916 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:11.458011 16403 raft_consensus.cc:2376] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6. Re-sending same reply.
W20260709 12:23:11.406577 19520 log.cc:927] Time spent T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 12:23:11.428052 19410 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:11.373855 19522 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:11.436131 19266 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:11.400182 19335 consensus_queue.cc:1048] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 12:23:11.396818 19525 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:11.436811 19347 consensus_meta.cc:287] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.531s	user 0.010s	sys 0.005s
I20260709 12:23:11.411985 19113 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:11.457939 16468 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:11.460263 16442 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:11.464425 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:11.298334 16224 rpcz_store.cc:269] 0709 12:22:58.579377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.579632 (+   255us) service_pool.cc:224] Handling call
0709 12:23:11.098911 (+12519279us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:05.675556 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:05.676178 (+   622us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:05.676222 (+    44us) write_op.cc:435] Acquired schema lock
0709 12:23:05.676241 (+    19us) tablet.cc:662] Decoding operations
0709 12:23:05.678089 (+  1848us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:05.678142 (+    53us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:05.678168 (+    26us) tablet.cc:685] Acquiring locks for 23 operations
0709 12:23:05.707179 (+ 29011us) tablet.cc:701] Row locks acquired
0709 12:23:05.707213 (+    34us) write_op.cc:260] PREPARE: finished
0709 12:23:05.751214 (+ 44001us) write_op.cc:270] Start()
0709 12:23:05.751412 (+   198us) write_op.cc:276] Timestamp: P: 1783599785748529 usec, L: 0
0709 12:23:05.751437 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:23:05.752495 (+  1058us) log.cc:844] Serialized 8866 byte log entry
0709 12:23:06.399462 (+646967us) op_driver.cc:464] REPLICATION: finished
0709 12:23:06.510122 (+110660us) write_op.cc:301] APPLY: starting
0709 12:23:06.510208 (+    86us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:06.510600 (+   392us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:06.510719 (+   119us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:06.512948 (+  2229us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:06.513309 (+   361us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=23
0709 12:23:06.513362 (+    53us) write_op.cc:312] APPLY: finished
0709 12:23:06.513757 (+   395us) log.cc:844] Serialized 157 byte log entry
0709 12:23:06.513941 (+   184us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:06.514130 (+   189us) write_op.cc:454] Released schema lock
0709 12:23:06.514188 (+    58us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:06.514629 (+   441us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:06.514655 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":885902,"reported_tablets":40,"child_traces":[["op",{"apply.queue_time_us":110382,"apply.run_cpu_time_us":4477,"apply.run_wall_time_us":4478,"num_ops":23,"prepare.queue_time_us":523915,"prepare.run_cpu_time_us":53801,"prepare.run_wall_time_us":257612,"raft.queue_time_us":344868,"raft.run_cpu_time_us":29354,"raft.run_wall_time_us":118273,"replication_time_us":647811,"thread_start_us":924514,"threads_started":4,"wal-append.queue_time_us":197528,"wal-append.run_cpu_time_us":63699,"wal-append.run_wall_time_us":1630248}]]}
W20260709 12:23:11.390825 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:11.466486 19355 consensus_meta.cc:287] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.559s	user 0.014s	sys 0.000s
I20260709 12:23:11.449746 19481 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:11.403190 19505 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:11.466697 19417 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:11.470469 19266 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:23:11.466840 19464 raft_consensus.cc:697] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:11.482692 19522 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:11.449406 19487 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:11.484817 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:11.503240 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:11.503952 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:11.504523 19487 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:11.465061 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 179923us
I20260709 12:23:11.490571 16514 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:11.490437 16442 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:11.482043 19506 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:11.499557 19464 consensus_queue.cc:237] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:11.515511 17144 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:11.493782 19223 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:11.514393 16477 raft_consensus.cc:1240] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:11.500262 19309 raft_consensus.cc:3055] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:11.493413 19481 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:11.523389 16442 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:11.526698 19184 consensus_meta.cc:287] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.703s	user 0.011s	sys 0.000s
I20260709 12:23:11.527220 19184 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:11.527992 19530 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:11.528352 19417 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:11.528733 19530 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:11.529017 19417 raft_consensus.cc:697] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:11.529229 19530 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:11.530223 19505 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:11.522897 19473 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.531219 19473 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:11.532747 16442 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:11.533272 16468 raft_consensus.cc:1275] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:23:11.533437 19506 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:11.534297 19370 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:11.534480 16442 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:11.536432 17142 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:11.537528 18013 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:11.537864 18013 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:11.538288 18013 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:11.538780 17142 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:11.528455 16546 raft_consensus.cc:1240] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:11.529959 19417 consensus_queue.cc:237] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:11.541903 16398 raft_consensus.cc:1275] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
W20260709 12:23:11.543720 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:11.475528 19347 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:11.517390 19266 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:11.545841 16442 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:11.493003 19355 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:11.473415 19525 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:11.521080 19223 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:11.557431 16442 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:11.563730 16367 connection.cc:256] RPC call timeout handler was delayed by -1.288607293000041s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26789999999999997s for a response
W20260709 12:23:11.565126 16367 connection.cc:256] RPC call timeout handler was delayed by -1.288607293000041s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26789999999999997s for a response
I20260709 12:23:11.520090 19309 raft_consensus.cc:740] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:11.563174 19384 consensus_queue.cc:1048] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 12:23:11.573997 19410 consensus_queue.cc:1048] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:11.543071 19473 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:11.576721 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 205440us
I20260709 12:23:11.587146 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:11.580808 18013 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:11.575193 19113 consensus_queue.cc:1059] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:23:11.582598 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 44034.6us
I20260709 12:23:11.546061 19506 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:11.559267 19266 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:11.592156 19243 consensus_meta.cc:287] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.687s	user 0.003s	sys 0.012s
I20260709 12:23:11.592298 19495 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:11.592566 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:11.592712 19243 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:11.593223 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:11.593659 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:11.594029 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:11.594300 17258 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:11.594333 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:11.594846 19243 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:11.595067 17255 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:11.595654 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:11.596565 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:11.555470 19461 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:11.598227 19461 raft_consensus.cc:697] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 9 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:11.592217 19266 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:11.599347 19461 consensus_queue.cc:237] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:11.600978 19309 consensus_queue.cc:260] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:11.602681 19309 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:11.563247 19505 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:11.604578 19505 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:11.604924 19505 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:11.605098 16543 raft_consensus.cc:1240] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:11.605638 19355 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:11.605523 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:11.607349 17263 raft_consensus.cc:1240] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 8. Current term is 9. Ops: [6.5-8.6]
I20260709 12:23:11.590682 16367 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:11.607589 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:11.564316 19424 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:11.610311 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:11.607961 17266 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:23:11.612885 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:11.613314 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:11.613649 17269 raft_consensus.cc:1275] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:23:11.576154 19223 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:11.614012 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:11.619467 17270 raft_consensus.cc:2393] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 9: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:11.581845 19522 raft_consensus.cc:2804] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:11.620108 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:11.624372 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:23:11.625735 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 59772.1us
I20260709 12:23:11.627130 17254 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:11.627522 16367 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:11.628470 16468 raft_consensus.cc:2368] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 8. Current term is 9.
I20260709 12:23:11.563405 19335 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:23:11.647253 19272 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:11.636912 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:11.637352 19184 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:11.637558 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:11.638146 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:11.638667 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:11.640445 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:11.641244 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:11.641865 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:11.662292 16472 raft_consensus.cc:2393] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 10: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:11.642490 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:11.632982 19424 raft_consensus.cc:697] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 10 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:11.648294 17254 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
I20260709 12:23:11.649785 19466 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:11.576879 19527 consensus_queue.cc:1059] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 12:23:11.653482 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:11.655153 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:11.669193 16474 raft_consensus.cc:1240] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:11.671084 16470 raft_consensus.cc:2368] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 5. Current term is 6.
I20260709 12:23:11.663332 16475 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:11.671896 17266 raft_consensus.cc:1240] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:11.633741 18013 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:11.606163 19113 raft_consensus.cc:3055] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:11.659876 17258 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6.
I20260709 12:23:11.660983 17255 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:23:11.665550 16471 raft_consensus.cc:2393] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 10: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:11.673899 17270 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:23:11.675055 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 84715.1us
I20260709 12:23:11.676537 17254 raft_consensus.cc:1240] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:11.670641 16468 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:11.589404 19465 consensus_queue.cc:1048] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 12:23:11.681951 19466 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:11.684618 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:11.684937 16475 raft_consensus.cc:1240] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:11.676877 19113 raft_consensus.cc:740] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:11.686251 19113 consensus_queue.cc:260] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:11.693864 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:11.694530 16395 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 9
W20260709 12:23:11.699644 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 72207.2us
I20260709 12:23:11.700441 17254 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:11.700529 16367 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:11.709605 16395 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:11.711601 16547 raft_consensus.cc:2393] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 7: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:11.712729 19222 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:11.714078 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:11.714668 19525 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:11.715862 17263 raft_consensus.cc:1240] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:23:11.716670 16475 raft_consensus.cc:1240] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:11.718647 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:11.719549 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:11.725663 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:11.720415 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:11.659341 19522 raft_consensus.cc:697] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:11.676963 19410 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:11.728010 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:11.729027 16472 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:11.721772 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:11.727950 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:11.727744 17144 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:11.731524 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:11.732144 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:11.734149 17269 raft_consensus.cc:1240] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:11.737303 19525 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:11.699168 19239 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:11.748757 16472 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:23:11.751816 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:11.752620 16472 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:23:11.759817 16398 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:23:11.764747 17266 raft_consensus.cc:1240] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:11.774830 17270 raft_consensus.cc:2468] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:23:11.664125 19532 consensus_queue.cc:1059] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.083s
I20260709 12:23:11.763963 16441 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:11.717921 19184 consensus_queue.cc:1059] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 12:23:11.778394 19410 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:11.779038 19410 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:11.781544 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:11.786145 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:11.785598 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:11.761876 19409 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:23:11.801396 17144 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:11.801985 19409 raft_consensus.cc:697] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:11.754978 19239 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:11.805176 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:11.808837 19184 raft_consensus.cc:3055] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:11.809270 19184 raft_consensus.cc:740] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:11.808550 19522 consensus_queue.cc:237] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:11.810096 19184 consensus_queue.cc:260] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:11.811316 19184 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:23:11.813176 19535 log.cc:927] Time spent T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:23:11.812951 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:11.818681 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 54850.9us
I20260709 12:23:11.752315 19525 raft_consensus.cc:697] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:23:11.801746 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 71363.8us
I20260709 12:23:11.728726 19423 consensus_queue.cc:1048] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
I20260709 12:23:11.820673 19239 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:11.844708 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:11.844969 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:11.824183 19532 raft_consensus.cc:3055] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:11.827013 19531 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:11.824229 17254 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:23:11.849303 16441 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:11.855520 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:11.860046 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:11.861192 17254 raft_consensus.cc:2393] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 7: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:11.840761 19355 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:11.867200 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:11.843647 19410 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:11.847441 19409 consensus_queue.cc:237] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:11.851188 19531 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:11.865489 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:11.874032 17263 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:23:11.865275 19530 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:11.866677 17255 raft_consensus.cc:1240] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:11.848601 19532 raft_consensus.cc:740] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:11.870124 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:11.833832 19473 consensus_queue.cc:1048] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 12:23:11.874908 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:11.903553 16472 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:23:11.887694 19223 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:11.877005 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:11.851100 19347 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:11.878208 17254 raft_consensus.cc:1240] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:11.867897 16398 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:23:11.885483 19222 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:11.886137 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:11.889585 16400 raft_consensus.cc:1240] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:23:11.890254 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:11.891036 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:11.891608 19384 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:11.892088 16469 raft_consensus.cc:1240] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:11.877822 19531 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:11.838143 19424 consensus_queue.cc:237] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:11.892753 19266 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:11.894523 19272 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:11.895226 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:11.895503 17258 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:11.896011 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:11.892138 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:11.899888 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:11.905340 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:23:11.875540 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 175096us
I20260709 12:23:11.912092 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:11.890676 19481 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:11.893393 19532 consensus_queue.cc:260] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:11.930820 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:11.978391 16472 raft_consensus.cc:2393] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 9: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:11.941780 17142 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:11.981742 17142 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:11.944793 16473 raft_consensus.cc:1240] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:11.906736 19335 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:23:11.968741 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 119557us
I20260709 12:23:11.970000 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:23:11.988658 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:11.906736 19540 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.972117 16476 raft_consensus.cc:2368] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 8. Current term is 9.
I20260709 12:23:11.975077 16403 raft_consensus.cc:2393] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 7: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:11.976542 16394 raft_consensus.cc:1240] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:23:12.046850 16546 raft_consensus.cc:1240] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:11.946943 19384 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:12.050077 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:12.050839 16476 raft_consensus.cc:2368] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 8. Current term is 9.
I20260709 12:23:12.051580 16470 raft_consensus.cc:2393] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 7: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:12.058517 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:12.046839 16367 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:12.061858 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 85070.8us
I20260709 12:23:12.061930 16441 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:12.063182 16367 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:12.056123 19471 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:12.064325 19544 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:11.939649 17254 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2"
dest_uuid: "ef57beab0e1d40d585983873c55107e8"
mode: GRACEFUL
new_leader_uuid: "610f4feda11942739236f2b121a4b9c1"
 from {username='slave'} at 127.0.0.1:45120
I20260709 12:23:12.066277 17254 raft_consensus.cc:606] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Received request to transfer leadership to TS 610f4feda11942739236f2b121a4b9c1
W20260709 12:23:12.066444 19548 log.cc:927] Time spent T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 12:23:12.068245 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:12.070447 17261 raft_consensus.cc:1240] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 6. Current term is 7. Ops: []
W20260709 12:23:12.072921 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 91294.2us
I20260709 12:23:12.074680 16441 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:23:12.076027 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:12.079105 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:12.081895 17263 raft_consensus.cc:2393] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 7: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:12.080013 16476 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
W20260709 12:23:12.085865 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:12.096819 16470 raft_consensus.cc:1240] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:12.097908 16514 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:12.082839 19000 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:12.107618 16366 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:12.069687 19409 raft_consensus.cc:2764] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:23:12.110481 16542 raft_consensus.cc:1240] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:23:12.105470 19384 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:12.151997 19537 log.cc:927] Time spent T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:23:12.185966 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:12.181726 19266 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:12.176693 19384 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:12.183301 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:12.177233 19526 consensus_queue.cc:1059] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:12.154587 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:12.186062 19113 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:12.108251 19471 raft_consensus.cc:697] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:12.185175 19481 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:12.114590 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:12.179621 19223 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:12.110512 19532 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:12.152642 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:12.068395 19347 consensus_queue.cc:1048] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:23:11.937839 19525 consensus_queue.cc:237] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:12.108831 19461 consensus_queue.cc:1059] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.048s
W20260709 12:23:12.160705 16276 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 46081.4us
I20260709 12:23:12.186128 19530 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:12.195379 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:12.196969 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 85050.8us
I20260709 12:23:12.205567 16547 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:23:12.212883 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:12.265717 17261 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:12.096995 19222 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:12.097070 19495 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:12.071131 19540 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:12.228848 16545 raft_consensus.cc:1275] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:23:12.229285 16475 raft_consensus.cc:1275] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:23:12.089830 19487 consensus_queue.cc:1059] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.123s
I20260709 12:23:12.097882 19410 consensus_queue.cc:1048] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.124s
I20260709 12:23:12.214970 19385 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:12.219285 19409 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:12.249076 16551 raft_consensus.cc:3055] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:12.304123 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:12.229902 19466 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:12.228906 19384 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:23:12.264479 19505 consensus_meta.cc:287] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.659s	user 0.012s	sys 0.000s
I20260709 12:23:12.268656 19223 raft_consensus.cc:3055] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:12.357895 19223 raft_consensus.cc:740] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:12.358978 19223 consensus_queue.cc:260] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:12.260934 19481 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:12.360131 19481 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:12.362435 19481 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:12.304180 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:12.302913 16551 raft_consensus.cc:740] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:12.363008 16515 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:12.329223 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:12.307237 19487 consensus_queue.cc:1243] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer ef57beab0e1d40d585983873c55107e8 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:23:12.267854 19239 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:12.329062 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:12.363932 16551 consensus_queue.cc:260] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:12.365376 16551 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:12.366139 19552 consensus_queue.cc:1059] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:12.312438 19384 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:12.367424 19552 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:12.312551 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:12.331434 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:12.339802 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:12.368921 17143 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:12.369187 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:12.369855 17263 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:12.372586 19243 consensus_queue.cc:1048] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:12.373466 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:12.374004 19243 consensus_queue.cc:1059] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:12.374370 16546 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:23:12.374871 19243 raft_consensus.cc:3055] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:12.375253 19243 raft_consensus.cc:740] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:23:12.335425 19184 consensus_meta.cc:287] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.524s	user 0.007s	sys 0.005s
I20260709 12:23:12.340674 19494 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:12.340273 19547 log.cc:927] Time spent T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 12:23:12.376156 19494 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:12.331614 19505 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:12.354189 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:12.354452 16442 leader_election.cc:400] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 11. Current term is 12.
I20260709 12:23:12.385274 16442 leader_election.cc:403] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:23:12.280362 17254 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:12.264889 19424 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:12.363369 17266 raft_consensus.cc:2368] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 5. Current term is 6.
I20260709 12:23:12.364208 19223 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:12.366664 19239 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:12.329180 19471 consensus_queue.cc:237] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:12.342583 19540 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:12.364486 17271 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac"
dest_uuid: "ef57beab0e1d40d585983873c55107e8"
mode: GRACEFUL
new_leader_uuid: "610f4feda11942739236f2b121a4b9c1"
 from {username='slave'} at 127.0.0.1:45120
I20260709 12:23:12.368108 19552 raft_consensus.cc:3055] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:12.356952 19531 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:12.387197 17261 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:12.411933 16551 raft_consensus.cc:1275] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 12:23:12.381415 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:12.401294 17271 raft_consensus.cc:606] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Received request to transfer leadership to TS 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:12.279004 19495 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:12.409337 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:12.412518 16551 pending_rounds.cc:85] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1: Aborting all ops after (but not including) 3
W20260709 12:23:12.404001 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:12.429031 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:12.433598 16551 pending_rounds.cc:107] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:23:12.434091 16551 raft_consensus.cc:2889] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:12.437325 16474 raft_consensus.cc:2368] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 8. Current term is 9.
I20260709 12:23:12.308597 19385 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:12.442922 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:12.452417 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:12.451707 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 195452us
I20260709 12:23:12.455391 17264 raft_consensus.cc:2368] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 7. Current term is 8.
I20260709 12:23:12.456789 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:12.457361 19562 log.cc:927] Time spent T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:23:12.458003 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:12.458581 16549 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:12.458921 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 104658us
I20260709 12:23:12.459699 16442 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:12.411826 19552 raft_consensus.cc:740] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:12.428171 19465 raft_consensus.cc:2764] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
I20260709 12:23:12.467257 16441 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:12.467294 16544 raft_consensus.cc:1240] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 8. Current term is 9. Ops: [6.5-8.6]
I20260709 12:23:12.457947 19495 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:12.472990 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:12.473295 16441 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:12.479671 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:12.478282 16549 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:23:12.447576 19461 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 11. Current term is 12.
I20260709 12:23:12.481580 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:12.376113 19243 consensus_queue.cc:260] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:12.485908 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:12.485247 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:12.470937 19355 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:23:12.466935 19000 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:12.489157 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:12.487092 16544 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:23:12.400839 19424 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:12.489827 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 126818us
I20260709 12:23:12.490408 16545 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:23:12.490408 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:12.509500 16515 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:12.486325 19464 raft_consensus.cc:3055] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:23:12.492072 19391 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.894s	user 0.015s	sys 0.015s
I20260709 12:23:12.516106 16441 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:12.519483 16516 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:12.489089 19309 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:12.516911 17263 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
I20260709 12:23:12.523607 17269 raft_consensus.cc:1240] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:12.523762 16515 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:12.494369 19461 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:12.527539 17255 raft_consensus.cc:1275] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 12:23:12.524169 19391 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.924s	user 0.015s	sys 0.016s
I20260709 12:23:12.530692 16515 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:12.529783 16516 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:12.535570 16367 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:12.518025 19464 raft_consensus.cc:740] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:12.524433 19554 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:12.519985 19355 raft_consensus.cc:697] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:12.539357 19546 consensus_queue.cc:1059] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:12.540906 19568 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:12.530112 19540 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:12.539268 16367 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:12.542196 19335 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:12.539064 19409 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:12.545918 19531 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:12.550038 16516 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:12.554692 17143 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:12.555227 16515 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:12.531891 19272 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:12.561957 16473 raft_consensus.cc:1240] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 9 LEADER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 8. Current term is 9. Ops: []
W20260709 12:23:12.572822 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 56811.7us
I20260709 12:23:12.532948 19494 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:12.562889 19563 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:12.525573 19384 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:12.563145 19464 consensus_queue.cc:260] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:12.563954 16367 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:12.564667 19385 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:12.555356 19391 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:12.529309 19309 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:12.576647 16543 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:23:12.574798 16441 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:12.585183 16515 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:12.560698 19113 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:12.591630 19505 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:12.577433 19000 consensus_queue.cc:1048] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 12:23:12.594211 19355 consensus_queue.cc:237] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:12.597359 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:12.523435 19552 consensus_queue.cc:260] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:12.600162 19552 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:12.601552 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:12.602253 16550 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:23:12.604177 19544 raft_consensus.cc:3055] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:12.604688 19544 raft_consensus.cc:740] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:12.585255 16367 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:12.555739 19568 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:12.586870 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:12.590325 16473 raft_consensus.cc:1240] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:12.577781 19417 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:12.590866 19239 consensus_queue.cc:1059] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:12.591214 19540 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:23:12.585798 19409 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:12.577430 19272 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:12.607750 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:12.607694 16441 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:12.608781 16542 raft_consensus.cc:1240] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:12.610103 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:12.610579 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:12.611297 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:12.656831 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:12.657684 16403 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:23:12.658793 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:12.538671 19575 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:12.613729 16545 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:23:12.615526 17143 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:23:12.671625 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 56198us
I20260709 12:23:12.672389 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:12.673058 16550 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:12.613567 19222 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:12.613365 19391 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:23:12.617141 19568 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:12.580256 19424 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:12.620857 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:12.735569 16395 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:12.622841 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:23:12.622722 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 63889.7us
W20260709 12:23:12.624187 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 74235.3us
I20260709 12:23:12.624320 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:12.736807 16398 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:12.659462 16403 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:23:12.618340 19384 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:12.670146 16399 raft_consensus.cc:2393] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 9: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:12.675258 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:12.678409 16400 raft_consensus.cc:1275] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 12:23:12.690589 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:12.613561 19113 consensus_queue.cc:1059] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 12:23:12.728935 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:12.618559 19546 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:12.732569 17271 raft_consensus.cc:1240] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 8. Current term is 9. Ops: [6.5-8.6]
I20260709 12:23:12.627781 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:12.624420 19417 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:12.757009 19417 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:12.758538 16399 raft_consensus.cc:1275] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:23:12.759904 19410 consensus_queue.cc:1059] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:23:12.726327 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 118700us
I20260709 12:23:12.668700 19272 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:12.703724 19527 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:12.606392 19532 consensus_queue.cc:1059] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.075s
I20260709 12:23:12.751192 16401 raft_consensus.cc:1275] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:23:12.807852 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:12.671617 19566 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:12.763077 16441 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:12.746371 17270 raft_consensus.cc:1275] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:23:12.776355 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:12.779134 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:12.781509 19569 log.cc:927] Time spent T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.175s	user 0.000s	sys 0.000s
I20260709 12:23:12.738656 19222 raft_consensus.cc:697] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 8 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:12.811365 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:12.812225 16544 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:12.812485 19222 consensus_queue.cc:237] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:12.815260 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:12.816222 16402 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:12.817951 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:12.818813 16545 raft_consensus.cc:2376] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6. Re-sending same reply.
I20260709 12:23:12.753494 19385 raft_consensus.cc:2804] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:12.821209 19385 raft_consensus.cc:697] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:12.822304 19385 consensus_queue.cc:237] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:12.782980 19572 consensus_queue.cc:1059] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:23:12.792582 19424 consensus_queue.cc:1048] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:12.824715 19505 raft_consensus.cc:2804] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:12.793522 19544 consensus_queue.cc:260] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:12.822299 16366 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.044s	sys 0.055s
W20260709 12:23:12.825809 16366 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.044s	sys 0.055s
I20260709 12:23:12.794119 16474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34840 (request call id 1108) took 1166 ms. Trace:
I20260709 12:23:12.826308 16474 rpcz_store.cc:276] 0709 12:23:11.627885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:11.678980 (+ 51095us) service_pool.cc:224] Handling call
0709 12:23:12.793918 (+1114938us) spinlock_profiling.cc:243] Waited 711 ms on lock 0x7b540058c854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:12.794055 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1634446592}
I20260709 12:23:12.794934 19563 raft_consensus.cc:3055] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:12.801245 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:12.801487 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:12.803197 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:12.826853 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 63871.1us
I20260709 12:23:12.833592 16401 raft_consensus.cc:2393] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 11: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:12.834425 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:12.839406 18013 consensus_queue.cc:1048] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:12.845682 16472 raft_consensus.cc:1240] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:12.848910 16403 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:12.800467 19527 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:12.861279 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 753710us
I20260709 12:23:12.854349 16395 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:23:12.825758 19566 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:12.873951 16394 raft_consensus.cc:1240] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:23:12.840094 19113 consensus_queue.cc:1243] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer ef57beab0e1d40d585983873c55107e8 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:23:12.875813 16401 raft_consensus.cc:1240] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:23:12.839174 19505 raft_consensus.cc:697] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:12.834805 19563 raft_consensus.cc:740] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:12.876024 16516 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:12.880420 16516 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:12.874984 16366 leader_election.cc:400] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 8. Current term is 9.
I20260709 12:23:12.882949 16366 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:12.877118 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:12.891886 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:12.860018 16550 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:12.910960 16402 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:23:12.868595 19527 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:12.914366 16516 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:12.876825 19544 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:12.876641 19540 raft_consensus.cc:3055] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:12.878386 19566 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:12.912964 19113 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:23:12.932910 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:12.912930 19409 consensus_queue.cc:1059] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 12:23:12.938789 19272 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:12.943491 19417 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:12.954947 19417 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:12.922588 19572 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:12.924747 19578 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:12.944798 19563 consensus_queue.cc:260] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:12.965342 16398 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
W20260709 12:23:12.966481 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 81352.9us
I20260709 12:23:12.970039 16399 raft_consensus.cc:1240] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:12.970687 16544 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:12.876679 19184 consensus_queue.cc:1059] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 12:23:12.927608 19540 raft_consensus.cc:740] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:12.976125 19546 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:12.980118 19184 consensus_queue.cc:1243] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer 52ab50b0237346c1a13ede947f6a63c2 log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:23:12.978147 19239 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:12.983187 19546 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:13.005286 19546 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:13.012506 19464 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:13.004184 19564 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:13.016882 19564 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:13.016556 19540 consensus_queue.cc:260] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:13.018453 19540 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:12.990064 19239 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:12.982025 19505 consensus_queue.cc:237] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:13.041280 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 312298us
I20260709 12:23:13.044997 16367 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:13.045512 19272 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:12.990289 19527 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:13.069432 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 230688us
W20260709 12:23:13.069655 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 207968us
I20260709 12:23:13.072705 16441 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:13.093152 16472 raft_consensus.cc:1240] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:13.094183 19552 consensus_queue.cc:1059] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:13.095237 19552 raft_consensus.cc:3055] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:13.095561 19552 raft_consensus.cc:740] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:13.096997 16441 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:13.097141 19466 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:13.100152 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:13.105050 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:13.111138 16476 raft_consensus.cc:2393] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 8: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
W20260709 12:23:13.113428 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:13.115118 17263 raft_consensus.cc:2393] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 11: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:13.116055 16469 raft_consensus.cc:1275] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:23:13.117619 19113 consensus_queue.cc:1059] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:13.118407 19585 raft_consensus.cc:3055] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:13.119025 19585 raft_consensus.cc:740] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:13.113054 19473 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:13.120168 19585 consensus_queue.cc:260] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:13.121023 19473 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:13.121649 19585 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:13.119371 19113 raft_consensus.cc:3055] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:13.123236 19113 raft_consensus.cc:740] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:13.123750 19487 raft_consensus.cc:3055] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:23:13.125730 17269 raft_consensus.cc:1275] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:23:13.102246 19355 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:13.168172 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:13.111683 19466 raft_consensus.cc:697] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:13.100041 19272 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:23:13.172307 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 202730us
I20260709 12:23:13.192864 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:13.124221 19487 raft_consensus.cc:740] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:13.205513 16551 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:23:13.203086 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:13.206591 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:13.132383 19564 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:13.124140 19113 consensus_queue.cc:260] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:13.098225 19552 consensus_queue.cc:260] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:13.205515 16366 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:13.207190 16551 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:13.268051 16544 raft_consensus.cc:1240] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:13.269374 19409 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:23:13.270854 16442 leader_election.cc:336] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.654s (SENT)
W20260709 12:23:13.273618 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 66250.7us
I20260709 12:23:13.277058 19473 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:13.315032 16441 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:23:13.346325 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 140886us
I20260709 12:23:13.347860 16366 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
W20260709 12:23:13.360586 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 315645us
I20260709 12:23:13.361816 16367 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:13.371232 16551 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:13.247052 19222 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:13.247108 19417 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:23:13.363560 19568 consensus_meta.cc:287] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.609s	user 0.013s	sys 0.000s
W20260709 12:23:13.382481 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 67561.5us
I20260709 12:23:13.380180 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:13.280133 19355 raft_consensus.cc:697] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:23:13.386612 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:13.372713 19409 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:13.361646 19113 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:23:13.412182 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 50528.2us
W20260709 12:23:13.454851 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 107097us
I20260709 12:23:13.454901 16367 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:13.277063 19272 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:13.455920 16366 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:13.447054 19599 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:13.407143 19577 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:13.446270 19409 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:13.417301 19575 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:23:13.409479 19417 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:13.384301 19222 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:13.447433 16441 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:13.414623 19568 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:13.470458 19581 log.cc:927] Time spent T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.420s	user 0.002s	sys 0.000s
W20260709 12:23:13.501477 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 54162.6us
I20260709 12:23:13.466162 19596 consensus_queue.cc:1059] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:23:13.407087 19466 consensus_queue.cc:237] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:13.512516 19417 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:13.509585 16477 raft_consensus.cc:1240] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:13.513085 19603 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:13.514288 19603 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:13.514987 19603 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:13.515573 19603 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:13.504420 16441 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:13.517033 19575 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:13.517527 19575 raft_consensus.cc:697] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:13.518409 19575 consensus_queue.cc:237] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:13.527837 19561 log.cc:927] Time spent T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 1.184s	user 0.006s	sys 0.000s
W20260709 12:23:13.529685 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:13.594317 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:13.530385 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:13.533179 19595 log.cc:927] Time spent T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:23:13.459182 19487 consensus_queue.cc:260] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:13.543919 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:13.553362 19309 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:13.481627 19577 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:13.521332 16551 raft_consensus.cc:3055] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:13.610430 16551 raft_consensus.cc:740] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:13.611425 16551 consensus_queue.cc:260] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:13.611920 19577 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:13.571226 16544 raft_consensus.cc:1275] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:23:13.573173 16550 raft_consensus.cc:1275] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:23:13.573951 19601 log.cc:927] Time spent T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.068s	user 0.001s	sys 0.001s
W20260709 12:23:13.615731 19544 consensus_meta.cc:287] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.688s	user 0.012s	sys 0.000s
I20260709 12:23:13.575234 19568 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:13.576092 19588 log.cc:927] Time spent T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.370s	user 0.000s	sys 0.002s
I20260709 12:23:13.616854 16400 raft_consensus.cc:1275] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:23:13.617244 16400 pending_rounds.cc:85] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 4
I20260709 12:23:13.617493 16400 pending_rounds.cc:107] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:23:13.617744 16400 raft_consensus.cc:2889] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:13.618075 16400 pending_rounds.cc:107] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:23:13.618332 16400 raft_consensus.cc:2889] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:23:13.576493 19566 consensus_meta.cc:287] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.604s	user 0.012s	sys 0.001s
I20260709 12:23:13.619222 19566 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:13.621915 19566 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:13.623134 16545 raft_consensus.cc:1240] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 12. Current term is 13. Ops: []
W20260709 12:23:13.576632 19590 log.cc:927] Time spent T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.203s	user 0.001s	sys 0.001s
I20260709 12:23:13.626260 16548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 588) took 1718 ms. Trace:
I20260709 12:23:13.505076 19355 consensus_queue.cc:237] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:13.577967 19222 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:13.578824 19576 raft_consensus.cc:2804] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:13.628865 19576 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:13.629276 19576 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:13.504402 19527 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:13.539726 19594 consensus_queue.cc:1059] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:23:13.626906 16548 rpcz_store.cc:276] 0709 12:23:11.907918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:11.908139 (+   221us) service_pool.cc:224] Handling call
0709 12:23:11.908595 (+   456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:12.820940 (+912345us) spinlock_profiling.cc:243] Waited 81.5 ms on lock 0x7b180048e2c0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2875a314bc 00007f2872d7f48b 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d99455 00007f2875e81060 00007f2875e81bfb 00007f2875e8c64b
0709 12:23:12.821080 (+   140us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:12.821108 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:12.821611 (+   503us) log.cc:844] Serialized 26 byte log entry
0709 12:23:12.992158 (+170547us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:12.992198 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:12.994217 (+  2019us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:13.625720 (+631503us) raft_consensus.cc:1685] finished
0709 12:23:13.625740 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:13.625938 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":184277,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":7781,"spinlock_wait_cycles":187511040,"thread_start_us":343825,"threads_started":2,"wal-append.queue_time_us":346516}
I20260709 12:23:13.633776 19594 raft_consensus.cc:3055] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:13.634260 19594 raft_consensus.cc:740] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:13.580379 19461 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:13.636080 19461 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:13.638489 19461 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:13.635165 19594 consensus_queue.cc:260] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:13.602605 19272 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:13.569769 19409 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:13.628182 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:13.644929 19527 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:13.645638 19527 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:13.648978 19417 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:13.649821 19417 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:13.650413 19563 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:13.652917 19539 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:13.653398 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:13.653566 19539 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:13.655833 19539 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:13.656214 17255 raft_consensus.cc:1240] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:13.656600 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:13.657946 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:13.658599 17269 raft_consensus.cc:1275] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:23:13.652189 19417 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:13.676945 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:23:13.619189 19546 consensus_meta.cc:287] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.609s	user 0.009s	sys 0.003s
I20260709 12:23:13.681067 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:13.683596 17255 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:13.683521 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:13.684311 17266 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:13.684362 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:13.682960 17261 raft_consensus.cc:1240] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:23:13.689033 19384 raft_consensus.cc:3055] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:13.689538 19384 raft_consensus.cc:740] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:13.693604 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:13.697932 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 243086us
I20260709 12:23:13.697611 19384 consensus_queue.cc:260] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:13.706357 17142 leader_election.cc:400] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 8. Current term is 9.
I20260709 12:23:13.708264 17142 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:13.708277 17255 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:23:13.714406 16367 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:23:13.715086 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:13.716137 17142 leader_election.cc:400] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 8. Current term is 9.
W20260709 12:23:13.717900 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 262046us
I20260709 12:23:13.720036 16366 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:13.715726 19384 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:13.726732 16367 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:13.692448 19546 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:13.714185 19494 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:13.726121 19113 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:23:13.763984 16543 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52680 (request call id 589) took 1852 ms (client timeout 1824 ms). Trace:
W20260709 12:23:13.787108 16543 rpcz_store.cc:269] 0709 12:23:11.911675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:11.911858 (+   183us) service_pool.cc:224] Handling call
0709 12:23:13.763733 (+1851875us) spinlock_profiling.cc:243] Waited 604 ms on lock 0x7b5400304654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:13.763892 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1389879680}
I20260709 12:23:13.770834 16401 raft_consensus.cc:1240] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:13.777650 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:13.790022 16402 raft_consensus.cc:2393] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 11: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:13.778934 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:13.779526 16399 raft_consensus.cc:1275] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:23:13.772569 16475 raft_consensus.cc:1240] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:13.782763 16395 raft_consensus.cc:1275] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:23:13.786332 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 66407.2us
I20260709 12:23:13.796131 16366 leader_election.cc:400] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 5. Current term is 6.
I20260709 12:23:13.796581 16366 leader_election.cc:403] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:13.776614 16477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1120) took 1219 ms. Trace:
W20260709 12:23:13.726902 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 224770us
I20260709 12:23:13.718726 19409 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:13.752560 19580 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:13.761518 16367 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
W20260709 12:23:13.717046 19540 consensus_meta.cc:287] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.696s	user 0.015s	sys 0.001s
W20260709 12:23:13.843727 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 57094.3us
I20260709 12:23:13.770469 16398 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:23:13.845276 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 90399.3us
I20260709 12:23:13.797854 16477 rpcz_store.cc:276] 0709 12:23:12.557183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:12.557327 (+   144us) service_pool.cc:224] Handling call
0709 12:23:12.557716 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:13.513660 (+955944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:13.776314 (+262654us) spinlock_profiling.cc:243] Waited 950 ms on lock 0x7b54000b5c54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:13.776538 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2185321856}
I20260709 12:23:13.824648 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:13.728690 19544 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:13.826647 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:13.756608 19607 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:13.785035 19113 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:13.762029 19539 consensus_queue.cc:1059] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 12:23:13.756700 19239 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:13.758725 19582 log.cc:927] Time spent T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.781s	user 0.011s	sys 0.027s
I20260709 12:23:13.770125 19546 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:13.839304 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:13.756325 19576 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:14.371646 19575 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:13.840554 19580 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:13.842259 19599 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:23:13.842893 19604 log.cc:927] Time spent T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.362s	user 0.002s	sys 0.000s
W20260709 12:23:13.843035 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 81599.7us
I20260709 12:23:13.781582 16400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 782) took 1107 ms. Trace:
W20260709 12:23:13.863026 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:14.385381 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:13.917124 16398 raft_consensus.cc:1275] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:23:14.385106 16400 rpcz_store.cc:276] 0709 12:23:12.673764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:12.760296 (+ 86532us) service_pool.cc:224] Handling call
0709 12:23:12.760628 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:13.618818 (+858190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:13.781375 (+162557us) spinlock_profiling.cc:243] Waited 856 ms on lock 0x7b54009bd7d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:13.781528 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1968059136}
I20260709 12:23:13.938073 16476 raft_consensus.cc:2393] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 8: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:13.909785 16471 raft_consensus.cc:1275] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:23:14.385430 16367 leader_election.cc:400] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 5 pre-election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 5. Current term is 6.
I20260709 12:23:13.939658 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:13.941530 16469 raft_consensus.cc:1275] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:23:14.388290 16473 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:23:13.911507 19335 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:13.844027 19370 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:14.109891 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 264006us
W20260709 12:23:14.087649 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 291662us
W20260709 12:23:14.085073 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 176370us
W20260709 12:23:14.165766 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:14.359159 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:13.935364 19522 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:13.970618 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:13.943277 19494 consensus_queue.cc:1048] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.062s
W20260709 12:23:14.359939 16546 consensus_meta.cc:287] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.516s	user 0.013s	sys 0.001s
I20260709 12:23:13.983731 19266 consensus_queue.cc:1059] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
W20260709 12:23:14.204372 17143 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.005s	sys 0.021s
W20260709 12:23:14.428740 17143 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.008s	sys 0.021s
I20260709 12:23:14.374799 19607 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 5. Current term is 6.
I20260709 12:23:13.937022 19578 consensus_queue.cc:1059] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 12:23:14.356910 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:14.204826 19355 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.020s	sys 0.011s
I20260709 12:23:13.982909 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:14.236836 19527 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:14.203686 19596 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.028s	sys 0.008s
W20260709 12:23:14.460235 19596 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.949s	user 0.030s	sys 0.009s
W20260709 12:23:14.444665 19355 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.812s	user 0.022s	sys 0.012s
I20260709 12:23:14.460999 19355 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:14.461517 19355 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:14.462769 19552 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:14.463902 19355 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:14.465296 19580 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:14.421595 19546 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:14.468796 19546 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:14.470373 16551 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:14.471511 19546 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:14.450471 19599 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:14.473784 19599 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:14.475857 19599 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:14.356977 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:14.369585 19409 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.372278 19576 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:14.357340 19526 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.477373 19409 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:14.477702 19526 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:14.479826 19409 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:14.363601 19585 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:23:14.415750 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:14.482316 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:14.415977 17142 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:14.483377 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 67473.1us
I20260709 12:23:14.485080 16545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1093) took 1295 ms. Trace:
I20260709 12:23:14.485488 16545 rpcz_store.cc:276] 0709 12:23:13.189788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:13.191873 (+  2085us) service_pool.cc:224] Handling call
0709 12:23:13.267862 (+ 75989us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:13.623660 (+355798us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:14.484842 (+861182us) spinlock_profiling.cc:243] Waited 355 ms on lock 0x7b5400896a54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:14.485018 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":815530624}
I20260709 12:23:14.360467 19540 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:14.488652 19540 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:14.490927 19540 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:14.491650 19585 raft_consensus.cc:697] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
W20260709 12:23:14.417546 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:14.421319 16395 raft_consensus.cc:1240] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:14.423403 16546 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:23:14.372167 19575 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:14.427155 17144 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
W20260709 12:23:14.429097 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 980980us
W20260709 12:23:14.388397 19563 consensus_meta.cc:287] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.737s	user 0.012s	sys 0.011s
I20260709 12:23:13.933921 17263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45230 (request call id 873) took 1260 ms. Trace:
I20260709 12:23:14.384747 19572 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.437599 16399 raft_consensus.cc:2410] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:23:14.385586 19554 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:23:14.443228 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 57948.4us
I20260709 12:23:14.406250 19544 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.420562 19335 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:14.457396 19384 consensus_meta.cc:287] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.706s	user 0.009s	sys 0.015s
I20260709 12:23:14.458616 19113 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:14.459331 19527 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.517266 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:14.493634 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:14.501591 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:14.521281 16515 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:14.448310 16366 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:14.504719 17263 rpcz_store.cc:276] 0709 12:23:12.673552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:12.674835 (+  1283us) service_pool.cc:224] Handling call
0709 12:23:13.933604 (+1258769us) spinlock_profiling.cc:243] Waited 371 ms on lock 0x7b54005f1754. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:13.933828 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":852668160}
I20260709 12:23:14.480005 19526 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:14.511408 17254 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825"
dest_uuid: "ef57beab0e1d40d585983873c55107e8"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:45120
I20260709 12:23:14.522799 17254 raft_consensus.cc:606] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:14.450011 19239 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.522975 16548 raft_consensus.cc:1275] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:23:14.523269 17254 raft_consensus.cc:614] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:14.523509 16472 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:23:14.522864 19266 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.529958 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:14.510087 16551 raft_consensus.cc:1275] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:23:14.524070 16473 raft_consensus.cc:2393] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 8: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:14.526535 16367 leader_election.cc:400] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 7. Current term is 8.
I20260709 12:23:14.498930 19494 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:23:14.429637 19522 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:14.531952 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 83792.6us
I20260709 12:23:14.531983 16367 leader_election.cc:403] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:23:14.529203 19527 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:14.529491 19266 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:14.524062 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:14.534068 19526 consensus_queue.cc:1048] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:14.534448 19624 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:14.517729 19554 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:14.535069 19391 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 12:23:14.431340 19370 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:14.553568 16469 raft_consensus.cc:1275] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:23:14.553975 16548 raft_consensus.cc:1275] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:23:14.532171 19384 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:14.554342 16548 pending_rounds.cc:85] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1: Aborting all ops after (but not including) 4
I20260709 12:23:14.556278 16548 pending_rounds.cc:107] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:23:14.556671 16548 raft_consensus.cc:2889] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:14.559861 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:14.535523 16551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1097) took 1014 ms. Trace:
I20260709 12:23:14.535393 19627 consensus_queue.cc:1048] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:14.511675 19540 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.516528 19572 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:14.538792 17143 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:14.523025 19544 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:14.547044 16515 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:14.561990 16548 raft_consensus.cc:1275] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:23:14.566972 16442 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:14.561020 16367 connection.cc:256] RPC call timeout handler was delayed by -1.0613419399996928s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1573s for a response
I20260709 12:23:14.536870 16366 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:14.549678 19409 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:14.571228 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:14.561429 16551 rpcz_store.cc:276] 0709 12:23:13.520711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:13.520878 (+   167us) service_pool.cc:224] Handling call
0709 12:23:13.521181 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:14.470005 (+948824us) spinlock_profiling.cc:243] Waited 774 ms on lock 0x7b58006b81d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875de50aa 00007f2875e67447 00007f2875e7ffe7 00007f2875e817de 00007f2875e8bb49 00007f2875e8c416 00007f2875e88d8f 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4
0709 12:23:14.532149 (+ 62144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:14.535455 (+  3306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1781101440}
I20260709 12:23:14.549546 19113 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:14.582108 16366 leader_election.cc:400] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 8. Current term is 9.
I20260709 12:23:14.582496 16366 leader_election.cc:403] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:23:14.568017 16367 connection.cc:256] RPC call timeout handler was delayed by -1.0415419399996608s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1595s for a response
I20260709 12:23:14.553799 19563 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 7. Current term is 8.
I20260709 12:23:14.590708 17255 raft_consensus.cc:1275] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:23:14.591428 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:14.592640 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:14.593628 16400 raft_consensus.cc:1275] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 12:23:14.579314 16546 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37242 (request call id 1086) took 1464 ms (client timeout 1843 ms). Trace:
W20260709 12:23:14.557638 19539 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.044s	sys 0.028s
I20260709 12:23:14.580530 16548 raft_consensus.cc:2393] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 11: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:14.584949 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:14.585304 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:14.555209 19624 raft_consensus.cc:697] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:14.597199 16398 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
I20260709 12:23:14.582705 19576 consensus_queue.cc:1048] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 12:23:14.588536 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:14.574889 17254 raft_consensus.cc:1240] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:14.589340 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:14.536940 19239 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:14.594254 16546 rpcz_store.cc:269] 0709 12:23:13.115178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:13.128365 (+ 13187us) service_pool.cc:224] Handling call
0709 12:23:14.579099 (+1450734us) spinlock_profiling.cc:243] Waited 662 ms on lock 0x7b540037ee54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:14.579233 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1523424640}
I20260709 12:23:14.595507 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:14.549041 19625 consensus_queue.cc:1048] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 12:23:14.595818 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:14.592137 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:14.586443 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:14.650887 16394 raft_consensus.cc:2393] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 10: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:14.598079 17255 raft_consensus.cc:1240] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:14.650305 16402 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:23:14.598762 16367 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:14.599866 17269 raft_consensus.cc:2393] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 7: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:14.619223 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:14.653200 16400 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:14.539494 19585 consensus_queue.cc:237] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:14.580475 19355 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:23:14.666129 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 84139.9us
I20260709 12:23:14.564070 19384 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:14.589502 19410 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:23:14.601153 19539 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.044s	sys 0.028s
W20260709 12:23:14.693853 19617 log.cc:927] Time spent T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.217s	user 0.002s	sys 0.000s
I20260709 12:23:14.695433 19575 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:14.696843 19632 consensus_queue.cc:1048] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 12:23:14.697695 19572 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:14.697826 19391 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:14.698313 19632 consensus_queue.cc:1048] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:14.698477 19391 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:14.678210 16399 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6.
I20260709 12:23:14.594580 19409 raft_consensus.cc:697] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:14.701814 19409 consensus_queue.cc:237] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:14.669709 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 71016.3us
I20260709 12:23:14.588969 19599 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:14.670619 17261 raft_consensus.cc:1275] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:14.705418 19410 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:14.673683 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:14.566675 19612 consensus_queue.cc:1048] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 12:23:14.667217 16366 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:14.704028 19599 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:14.566569 19540 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:14.691113 19633 consensus_queue.cc:1048] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:14.698885 19385 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.705528 19544 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:14.588969 19546 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.706367 16546 raft_consensus.cc:2468] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:23:14.679515 19563 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:14.706588 16367 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:14.596535 19113 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:14.720911 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 53807us
I20260709 12:23:14.698316 19572 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:14.581390 19494 consensus_queue.cc:1059] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
W20260709 12:23:14.668226 19602 log.cc:927] Time spent T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 1.184s	user 0.000s	sys 0.003s
I20260709 12:23:14.733785 19596 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:23:14.732733 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:14.745515 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 318426us
I20260709 12:23:14.581768 19309 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:14.772982 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:14.747591 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:14.773694 16546 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:23:14.748835 16400 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
W20260709 12:23:14.748772 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 243248us
I20260709 12:23:14.706061 19410 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:23:14.753880 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 47355.3us
I20260709 12:23:14.650362 19607 raft_consensus.cc:2749] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 8. Current term is 9.
W20260709 12:23:14.746116 19621 log.cc:927] Time spent T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:23:14.775003 17142 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:14.749953 19563 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:14.798195 19563 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:14.798491 19527 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:14.799464 19522 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:14.680003 19355 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:14.752682 19629 raft_consensus.cc:993] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8: : Instructing follower 610f4feda11942739236f2b121a4b9c1 to start an election
I20260709 12:23:14.752769 19546 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:14.787175 19391 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:14.787590 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:14.788561 19335 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:14.755221 19572 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:14.817461 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:14.823199 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:14.841317 16397 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 4.
I20260709 12:23:14.790838 19603 raft_consensus.cc:3055] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:14.761839 19628 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:14.766499 19113 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:14.773495 19624 consensus_queue.cc:237] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:14.762545 19544 consensus_queue.cc:1048] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:23:14.764652 19613 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:14.750444 19385 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:14.795079 19559 raft_consensus.cc:3055] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:14.781865 19576 consensus_queue.cc:1059] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 12:23:14.825919 19630 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.827976 19384 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:14.843494 19572 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:14.796821 16395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 804) took 1015 ms. Trace:
I20260709 12:23:14.777829 19578 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:14.797335 19554 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:14.842271 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 56285.6us
I20260709 12:23:14.867570 16367 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:14.795550 19494 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:23:14.850113 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 269996us
I20260709 12:23:14.771195 19309 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:14.856142 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 289229us
I20260709 12:23:14.800181 19522 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:14.857901 16401 raft_consensus.cc:1275] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 12:23:14.858769 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 311789us
I20260709 12:23:14.815088 19487 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:14.815706 19626 raft_consensus.cc:3055] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:14.800254 19243 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:14.778185 19631 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:14.796397 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:14.879345 19309 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:14.916889 16551 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:23:14.814363 19184 raft_consensus.cc:3055] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:14.918231 19184 raft_consensus.cc:740] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:14.918831 16515 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:14.919317 19184 consensus_queue.cc:260] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:14.864063 16395 rpcz_store.cc:276] 0709 12:23:13.781546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:14.359920 (+578374us) service_pool.cc:224] Handling call
0709 12:23:14.360216 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:14.493020 (+132804us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:14.796592 (+303572us) spinlock_profiling.cc:243] Waited 60.9 ms on lock 0x7b5400bb80d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:14.796760 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":140092672}
I20260709 12:23:14.864311 19629 consensus_queue.cc:1048] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 12:23:14.920747 19633 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:14.921285 19633 raft_consensus.cc:697] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:14.921449 19184 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:14.922122 19633 consensus_queue.cc:237] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:14.865523 17144 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:14.843948 19603 raft_consensus.cc:740] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:14.922935 19629 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.923454 19629 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:14.923691 19603 consensus_queue.cc:260] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:14.925107 19603 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:14.859774 19559 raft_consensus.cc:740] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:14.890965 19522 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:14.849671 19539 consensus_queue.cc:1048] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:23:14.926528 19000 raft_consensus.cc:3055] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:14.926997 19000 raft_consensus.cc:740] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:14.927469 19539 raft_consensus.cc:1081] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Signalling peer 610f4feda11942739236f2b121a4b9c1 to start an election
W20260709 12:23:14.885653 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:14.905443 19599 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:14.853613 19527 consensus_queue.cc:1059] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:23:14.857563 19464 consensus_queue.cc:1048] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 12:23:14.909062 19637 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.893499 19626 raft_consensus.cc:740] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:14.877310 19494 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:14.880151 19578 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:14.911852 19410 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:14.929589 16442 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:14.930611 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:14.931173 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 65845.6us
I20260709 12:23:14.955628 16396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 793) took 1187 ms. Trace:
I20260709 12:23:14.958205 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:14.958714 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:14.959128 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:14.959417 16473 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:14.960680 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:14.961053 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:14.956089 16396 rpcz_store.cc:276] 0709 12:23:13.768703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:13.768960 (+   257us) service_pool.cc:224] Handling call
0709 12:23:13.770934 (+  1974us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:13.771269 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:13.771306 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:13.771789 (+   483us) log.cc:844] Serialized 26 byte log entry
0709 12:23:14.901980 (+1130191us) spinlock_profiling.cc:243] Waited 1.08 ms on lock 0x7b18001b16c0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2875a314bc 00007f2872d7f48b 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1
0709 12:23:14.902212 (+   232us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:14.902267 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:14.902367 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:14.902426 (+    59us) raft_consensus.cc:1685] finished
0709 12:23:14.902438 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:14.955567 (+ 53129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":230186,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":16599,"spinlock_wait_cycles":2474880,"thread_start_us":314608,"threads_started":1,"wal-append.queue_time_us":649432}
I20260709 12:23:14.964445 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:14.965301 16474 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:14.967595 16471 raft_consensus.cc:1240] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 7. Current term is 8. Ops: []
W20260709 12:23:14.970327 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:14.860285 19526 consensus_queue.cc:1048] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 12:23:14.956316 16442 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:14.974087 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:14.978927 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:14.897307 19612 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:14.852811 19628 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:14.891671 19370 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:14.992396 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:14.993121 16548 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:14.994953 19410 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:14.995930 19628 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:14.986002 16543 raft_consensus.cc:1275] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:23:14.930658 19594 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:14.999555 19611 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:15.000056 19611 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:15.003087 19611 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:14.945091 19552 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.007385 19552 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:15.008716 16551 raft_consensus.cc:1275] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:15.009865 19552 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:15.010052 17258 raft_consensus.cc:2410] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:23:14.932744 19494 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:15.011556 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:15.015769 16548 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
I20260709 12:23:14.932567 19626 consensus_queue.cc:260] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:15.017300 19626 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:14.930051 19527 raft_consensus.cc:3055] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:15.020555 19527 raft_consensus.cc:740] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:15.021452 19527 consensus_queue.cc:260] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:15.027948 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:14.929363 19540 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:14.933893 19576 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.932374 19643 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:14.929698 19266 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:14.933420 19546 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:14.935766 19580 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.932044 19355 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:15.033483 16546 raft_consensus.cc:1275] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:23:14.931285 19631 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:14.934247 19578 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:14.930981 19638 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.930868 19466 consensus_queue.cc:1048] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.105s
I20260709 12:23:15.030601 19417 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.030857 19335 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.026144 19572 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:15.033874 19239 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:14.866384 19630 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:14.975221 19222 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:14.928287 19000 consensus_queue.cc:260] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:15.092283 19370 consensus_queue.cc:1048] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.016s
W20260709 12:23:15.033447 19464 consensus_queue.cc:1175] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f99066d13e524238aec6aa2024a8bd50" responder_term: 10 status { last_received { term: 9 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 9. Current term is 10. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:23:15.094419 19335 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:15.101529 19239 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.102475 19239 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:15.105556 19239 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:15.086328 19638 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:15.111588 19540 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.112918 19540 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:15.114099 19630 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:15.117125 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:15.047077 16473 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:15.051097 19643 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:14.931903 19637 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:15.047289 19612 raft_consensus.cc:697] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:15.119903 16472 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
W20260709 12:23:15.132236 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:15.124631 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:15.137779 16471 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:23:15.110487 19638 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:15.101421 19000 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:15.121812 19113 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:15.120154 19522 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:15.149528 19612 consensus_queue.cc:237] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:15.100054 19222 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:15.152705 19222 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:15.153083 19639 raft_consensus.cc:3055] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:15.153533 19639 raft_consensus.cc:740] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:15.154323 19639 consensus_queue.cc:260] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:15.155761 19639 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:15.050360 19576 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:15.158316 19526 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:15.174007 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 399071us
I20260709 12:23:15.174608 16545 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
I20260709 12:23:15.181562 17142 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:15.119489 19540 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:15.228768 17258 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45228 (request call id 895) took 1573 ms (client timeout 2056 ms). Trace:
W20260709 12:23:15.229295 17258 rpcz_store.cc:269] 0709 12:23:13.655157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:13.656271 (+  1114us) service_pool.cc:224] Handling call
0709 12:23:15.213461 (+1557190us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b540005df54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:15.228691 (+ 15230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3061160320}
I20260709 12:23:15.228556 16514 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:15.230921 16514 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:15.235011 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:15.236723 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:15.238934 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:15.239661 16394 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:15.241220 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:15.137328 19643 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:15.084283 19580 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:15.092495 19417 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:15.269167 16401 raft_consensus.cc:1240] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:23:15.271695 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:15.214509 19615 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.233937 19113 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.235991 19635 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:15.084467 19631 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:15.288921 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:15.289387 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:15.285915 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 398594us
I20260709 12:23:15.248713 19636 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:15.324136 16543 raft_consensus.cc:1275] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:23:15.251338 19540 raft_consensus.cc:2804] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:15.148180 19563 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:15.065841 19559 consensus_queue.cc:260] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:15.215793 19526 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:15.258904 19629 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:15.329358 19540 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:15.344897 16476 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:15.329059 19636 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:15.348115 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 380398us
I20260709 12:23:15.374878 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:15.343853 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 387630us
I20260709 12:23:15.377089 16442 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:15.377432 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:15.368819 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 430355us
I20260709 12:23:15.378021 16442 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:15.383746 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:15.349263 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:15.348855 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:15.387198 16402 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:23:15.393929 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:15.394800 16549 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:23:15.400825 16471 raft_consensus.cc:2393] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 10: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:15.339541 19385 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:15.405174 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:15.427065 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:15.389312 16366 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:15.433172 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:15.434544 16546 raft_consensus.cc:2393] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 10: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:15.327617 19113 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:15.431224 16366 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:15.438139 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:15.445375 16398 raft_consensus.cc:2410] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:23:15.370034 19645 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.449735 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:15.450953 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:15.379060 19636 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:15.455325 16398 raft_consensus.cc:2376] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7. Re-sending same reply.
I20260709 12:23:15.401724 19522 raft_consensus.cc:2804] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:15.460304 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:15.418941 19540 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:15.463065 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:15.472529 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:15.473278 16472 raft_consensus.cc:1240] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:15.480027 16402 raft_consensus.cc:2376] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8. Re-sending same reply.
I20260709 12:23:15.324467 19615 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:15.479164 16442 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:15.474000 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:15.436422 19355 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:15.483958 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:15.488521 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:15.442672 19391 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:15.400933 19638 consensus_queue.cc:1048] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 12:23:15.459231 19559 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:15.458892 19546 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:15.540405 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:15.518801  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:23:15.518472 19632 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:23:15.451849 19585 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:15.472417 19641 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.427312 19611 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:15.562023 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:15.570976 16398 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
I20260709 12:23:15.448155 19385 consensus_queue.cc:1048] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 12:23:15.562625 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:15.576733 16402 raft_consensus.cc:2376] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8. Re-sending same reply.
I20260709 12:23:15.493788 19355 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:23:15.577737 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:15.506084 19522 raft_consensus.cc:697] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 8 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:23:15.588464 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:15.496238 16442 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:15.587992 16403 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
I20260709 12:23:15.596944 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:15.603618 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 210212us
I20260709 12:23:15.558287 19653 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:23:15.608676 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 112446us
I20260709 12:23:15.526851 19391 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:15.640345 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:15.578784 19647 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:15.636708 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:15.636858 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:15.583283 19355 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:15.641290 16403 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
I20260709 12:23:15.463773 19645 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:15.630100 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:15.583403 19576 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:15.661106 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 49846.3us
I20260709 12:23:15.662868 19659 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:15.663003 19632 consensus_queue.cc:1048] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:15.600116 19585 raft_consensus.cc:697] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:15.674808 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:15.702792 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:15.706192 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:15.706883 17254 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:23:15.707684 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:15.707793 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:15.705860 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:15.721357 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:15.723830 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:15.722625 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:15.576407 19641 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:15.727432 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:15.733316 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:15.722713 17266 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:15.735222 17254 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:23:15.747895 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:15.751791 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:15.767438 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:15.671204 19563 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:15.772507 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:15.781468 16367 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:15.782069 17142 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.633s (SENT) [suppressed 33 similar messages]
W20260709 12:23:15.773424 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 307527us
I20260709 12:23:15.765056 17270 raft_consensus.cc:2468] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:23:15.775290 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:15.666394 19637 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:15.780125 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 69268.6us
I20260709 12:23:15.789393 16516 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:15.797819 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:15.799321 17264 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:23:15.715369 19629 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.807452 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:15.702703 19612 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:15.815773 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:15.820573 16516 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:15.818840 16546 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 5.
I20260709 12:23:15.834815 16516 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:15.630842 19653 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:15.786988 16542 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
 from {username='slave'} at 127.0.0.1:52680
I20260709 12:23:15.894937 16542 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:15.895401 16542 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:15.724752 19638 consensus_queue.cc:1048] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.110s
I20260709 12:23:15.876690 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:15.878144 16516 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:15.908056 17271 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.017s	sys 0.056s
I20260709 12:23:15.909590 16516 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:15.666708 19647 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:15.908555 17271 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.017s	sys 0.056s
I20260709 12:23:15.897608 16400 raft_consensus.cc:1275] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:23:15.929677 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:15.770859 19522 consensus_queue.cc:237] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:15.901662 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 119691us
W20260709 12:23:15.876669 19494 consensus_meta.cc:287] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.865s	user 0.012s	sys 0.000s
I20260709 12:23:15.833847 19563 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:15.953704 17271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 913) took 1331 ms. Trace:
W20260709 12:23:15.886646 19616 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.020s	sys 0.033s
I20260709 12:23:15.835781 19526 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:15.830591 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:16.005684 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:16.024504 16546 raft_consensus.cc:2410] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:23:15.893536 19637 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:15.922642 19552 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:15.885375 19532 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:23:15.985148 19184 consensus_meta.cc:287] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 1.063s	user 0.003s	sys 0.009s
I20260709 12:23:15.977644 17271 rpcz_store.cc:276] 0709 12:23:14.587513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:14.587645 (+   132us) service_pool.cc:224] Handling call
0709 12:23:14.592706 (+  5061us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:14.593064 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:23:14.850795 (+257731us) log.cc:844] Serialized 14 byte log entry
0709 12:23:15.917146 (+1066351us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:15.917230 (+    84us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:15.917299 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:15.917400 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:15.918839 (+  1439us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1377,"prepare.queue_time_us":509348,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":196923,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":32344,"spinlock_wait_cycles":52736,"thread_start_us":1321002,"threads_started":3,"wal-append.queue_time_us":873246}
W20260709 12:23:16.031337 16544 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.008s	sys 0.000s
W20260709 12:23:16.033958 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:16.053134 16544 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.008s	sys 0.000s
W20260709 12:23:16.035853 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 202869us
I20260709 12:23:16.037683 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:16.005511 19616 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.020s	sys 0.034s
W20260709 12:23:16.038848 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 129302us
I20260709 12:23:16.062047 19627 consensus_queue.cc:1048] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:16.052175 19532 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:16.028839 19239 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.983511 19563 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:15.876376 19629 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:16.092669 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:16.219544 17271 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
I20260709 12:23:15.885308 19612 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:15.930546 19576 consensus_queue.cc:1048] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.102s
W20260709 12:23:16.118172 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 336741us
W20260709 12:23:16.176820 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 387440us
I20260709 12:23:16.058059 17264 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
W20260709 12:23:16.039433 19309 consensus_meta.cc:287] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 1.116s	user 0.007s	sys 0.006s
I20260709 12:23:16.138126 17143 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:16.236572 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:16.238134 16546 raft_consensus.cc:1275] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:23:16.241178 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:16.235924 19646 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.276046 17264 raft_consensus.cc:1275] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:15.983307 19494 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:16.260612 16441 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:16.016162 19632 consensus_queue.cc:1048] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.196s
I20260709 12:23:16.066727 19526 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.303671 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:16.305248 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:16.326325 16470 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:16.188966 19532 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:16.343222 17266 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
I20260709 12:23:16.346690 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:16.355780 17264 raft_consensus.cc:1275] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:23:16.357295 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 51483.4us
I20260709 12:23:16.242583 19563 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:16.376827 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:16.377887 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:16.391475 17263 raft_consensus.cc:1240] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13. Ops: []
W20260709 12:23:16.295434 19000 consensus_meta.cc:287] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 1.147s	user 0.010s	sys 0.002s
I20260709 12:23:16.011744 19585 consensus_queue.cc:237] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:16.256960 19113 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:16.261363 19611 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:16.212088 19184 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:16.242228 19552 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.317754 19646 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:16.202385 19385 consensus_queue.cc:1048] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.158s
I20260709 12:23:16.345014 19638 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:23:16.394770 17258 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.034s	sys 0.042s
W20260709 12:23:16.457607 17258 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.034s	sys 0.043s
W20260709 12:23:16.391506 19659 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.066s	sys 0.135s
I20260709 12:23:16.413442 17264 raft_consensus.cc:1275] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:23:16.413465 16544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1113) took 1384 ms. Trace:
I20260709 12:23:16.276456 19309 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:16.253939 19239 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:16.294696 19661 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:23:16.439287 16542 consensus_meta.cc:287] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.542s	user 0.005s	sys 0.009s
I20260709 12:23:16.446347 17270 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
I20260709 12:23:16.355412 19694 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:16.359917 19526 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:16.498293 19000 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.542089 16398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 1048) took 1084 ms. Trace:
I20260709 12:23:16.542603 16398 rpcz_store.cc:276] 0709 12:23:15.457984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:15.459624 (+  1640us) service_pool.cc:224] Handling call
0709 12:23:16.541886 (+1082262us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b54004ecfd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:16.542017 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":234971520}
W20260709 12:23:16.404408 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:16.477970 16544 rpcz_store.cc:276] 0709 12:23:15.029455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:15.029571 (+   116us) service_pool.cc:224] Handling call
0709 12:23:15.049359 (+ 19788us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:15.049572 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:23:15.332200 (+282628us) log.cc:844] Serialized 14 byte log entry
0709 12:23:16.412921 (+1080721us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:16.412999 (+    78us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:16.413072 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:16.413186 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:16.413365 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":90437,"prepare.queue_time_us":581785,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":404203,"thread_start_us":1256234,"threads_started":3,"wal-append.queue_time_us":986791}
I20260709 12:23:16.502205 19699 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:16.491665 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 353656us
I20260709 12:23:16.456445 19638 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.455652 19184 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:16.485757 19659 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.067s	sys 0.136s
W20260709 12:23:16.494259 19670 log.cc:927] Time spent T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.697s	user 0.001s	sys 0.001s
I20260709 12:23:16.521005 17269 raft_consensus.cc:1240] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:23:16.469437 19113 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.483335 19641 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:16.460654 19552 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:16.463057 19675 consensus_queue.cc:1048] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.070s
W20260709 12:23:16.440438 19672 log.cc:927] Time spent T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.726s	user 0.000s	sys 0.002s
I20260709 12:23:16.499779 19661 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:16.536787 19696 log.cc:927] Time spent T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
W20260709 12:23:16.538010 19679 log.cc:927] Time spent T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.299s	user 0.000s	sys 0.001s
W20260709 12:23:16.539281 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:16.670105 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:16.496528 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:16.532055 16542 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:16.538829 19688 log.cc:927] Time spent T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 12:23:16.554468 16543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1109) took 1527 ms. Trace:
I20260709 12:23:16.524434 19539 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:23:16.558540 19647 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:16.536938 19691 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.657079 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:16.699862 16403 raft_consensus.cc:2393] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 8: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:16.539830 19645 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:16.671710 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 264373us
W20260709 12:23:16.674552 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 54414.2us
I20260709 12:23:16.711730 17143 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:16.680908 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:16.659192 19666 log.cc:927] Time spent T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.447s	user 0.001s	sys 0.001s
I20260709 12:23:16.615231 19000 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:16.692845 16470 raft_consensus.cc:1240] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:16.737429 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:16.691058 16546 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:16.529520 19694 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:16.699581 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:16.622570 19638 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:16.665673 19661 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:16.719367 19691 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:16.699651 16515 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:16.711585 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:16.690024 16543 rpcz_store.cc:276] 0709 12:23:15.027330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:15.027495 (+   165us) service_pool.cc:224] Handling call
0709 12:23:15.027863 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:15.329179 (+301316us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:16.540706 (+1211527us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b54008c55d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:16.554386 (+ 13680us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":680503552}
I20260709 12:23:16.626207 19659 raft_consensus.cc:697] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:16.665288 19706 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.645164 19653 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:16.775250 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:16.680154 19637 raft_consensus.cc:2764] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 12:23:16.742740 19678 log.cc:927] Time spent T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.521s	user 0.000s	sys 0.001s
I20260709 12:23:16.745054 19694 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:16.745664 16468 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 8: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:16.755487 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:16.757051 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:16.760888 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 61316.4us
I20260709 12:23:16.799072 16515 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:16.660224 19113 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:16.766690 17271 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
W20260709 12:23:16.748998 19689 log.cc:927] Time spent T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:23:16.697243 19335 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:16.718672 19676 log.cc:927] Time spent T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.584s	user 0.004s	sys 0.006s
I20260709 12:23:16.798058 16543 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:16.772074 16548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1108) took 1745 ms. Trace:
W20260709 12:23:16.763870 19355 consensus_meta.cc:287] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 1.101s	user 0.006s	sys 0.003s
W20260709 12:23:16.840853 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 129207us
W20260709 12:23:16.750440 19652 log.cc:927] Time spent T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.864s	user 0.007s	sys 0.001s
I20260709 12:23:16.845191 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:16.631249 19699 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:16.753677 19681 log.cc:927] Time spent T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:23:16.847838 16548 rpcz_store.cc:276] 0709 12:23:15.026973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:15.049248 (+ 22275us) service_pool.cc:224] Handling call
0709 12:23:15.049511 (+   263us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:15.049859 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:15.049886 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:15.085520 (+ 35634us) log.cc:844] Serialized 26 byte log entry
0709 12:23:15.379412 (+293892us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:15.379468 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:15.379536 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:16.771858 (+1392322us) raft_consensus.cc:1685] finished
0709 12:23:16.771878 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:16.772014 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9589,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":21479,"thread_start_us":275470,"threads_started":1,"wal-append.queue_time_us":560756}
I20260709 12:23:16.804093 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:16.733757 19612 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:16.767308 19540 consensus_meta.cc:287] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 1.245s	user 0.010s	sys 0.002s
I20260709 12:23:16.761068 19384 raft_consensus.cc:2804] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:23:16.753011 19701 log.cc:927] Time spent T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.222s	user 0.000s	sys 0.000s
I20260709 12:23:16.748107 19700 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.757366 19646 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:16.761291 19409 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:16.861832 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:16.917557 16543 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:16.917704 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:16.876157 17143 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:16.877245 16400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 1046) took 1425 ms. Trace:
I20260709 12:23:16.881304 16547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1111) took 1853 ms. Trace:
I20260709 12:23:16.749675 19184 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:16.760643 19687 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.752969 19684 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.777034 19494 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:16.755111 19691 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:16.893985 16544 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
W20260709 12:23:16.894344 16550 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.880s	user 0.003s	sys 0.007s
W20260709 12:23:16.925499 16550 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.916s	user 0.004s	sys 0.008s
I20260709 12:23:16.747426 19532 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:16.901064 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:16.902287 16402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 1038) took 1464 ms. Trace:
I20260709 12:23:16.761758 19487 consensus_queue.cc:1048] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.119s
I20260709 12:23:16.908542 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:16.929349 17271 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
W20260709 12:23:16.886893 19222 consensus_meta.cc:287] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 1.734s	user 0.014s	sys 0.002s
I20260709 12:23:16.933312 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:16.851023 19645 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:16.923097 16400 rpcz_store.cc:276] 0709 12:23:15.452266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:16.018276 (+566010us) service_pool.cc:224] Handling call
0709 12:23:16.188975 (+170699us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:16.189249 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:23:16.660205 (+470956us) log.cc:844] Serialized 14 byte log entry
0709 12:23:16.876821 (+216616us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:16.876879 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:16.876920 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:16.876970 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:16.877176 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1132,"prepare.queue_time_us":713817,"raft.queue_time_us":5383,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":14035,"thread_start_us":465911,"threads_started":2}
I20260709 12:23:16.923146 16547 rpcz_store.cc:276] 0709 12:23:15.027907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:15.028042 (+   135us) service_pool.cc:224] Handling call
0709 12:23:15.028320 (+   278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:15.028729 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:15.028769 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:15.029343 (+   574us) log.cc:844] Serialized 26 byte log entry
0709 12:23:15.442185 (+412842us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:15.442231 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:15.442296 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:16.881020 (+1438724us) raft_consensus.cc:1685] finished
0709 12:23:16.881044 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:16.881228 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5607,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":8492,"thread_start_us":354974,"threads_started":1,"wal-append.queue_time_us":927356}
I20260709 12:23:16.930464 16550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 619) took 1170 ms. Trace:
I20260709 12:23:16.929224 16402 rpcz_store.cc:276] 0709 12:23:15.438135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:15.720097 (+281962us) service_pool.cc:224] Handling call
0709 12:23:15.781904 (+ 61807us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:16.202296 (+420392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:16.202335 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:16.276277 (+ 73942us) log.cc:844] Serialized 52 byte log entry
0709 12:23:16.656445 (+380168us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:16.887037 (+230592us) log.cc:844] Serialized 14 byte log entry
0709 12:23:16.887506 (+   469us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:16.898850 (+ 11344us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:16.898960 (+   110us) raft_consensus.cc:1685] finished
0709 12:23:16.898981 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:16.902201 (+  3220us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":192283,"raft.run_cpu_time_us":410,"raft.run_wall_time_us":51207,"thread_start_us":934160,"threads_started":3,"wal-append.queue_time_us":377356}
I20260709 12:23:16.935462 16550 rpcz_store.cc:276] 0709 12:23:15.760214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:15.760387 (+   173us) service_pool.cc:224] Handling call
0709 12:23:15.760728 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:15.760958 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:23:15.836270 (+ 75312us) log.cc:844] Serialized 14 byte log entry
0709 12:23:16.930128 (+1093858us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:16.930187 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:16.930244 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:16.930305 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:16.930413 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4084,"prepare.queue_time_us":124949,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":40,"thread_start_us":916480,"threads_started":1}
W20260709 12:23:16.931890 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 55815.2us
I20260709 12:23:16.766667 19638 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:16.762519 19661 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:16.903493 19384 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:16.928612 19409 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:16.950027 19705 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.953816 19705 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:16.955408 19409 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:16.918751 19694 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.956152 19694 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:16.956600 19705 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:16.950392 19657 log.cc:927] Time spent T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.389s	user 0.000s	sys 0.001s
I20260709 12:23:16.957366 19715 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:16.942884 19695 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:16.958415 19694 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:16.952720 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:16.953117 19384 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:16.856741 19709 consensus_queue.cc:1048] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.252s
W20260709 12:23:16.961448 19635 consensus_meta.cc:287] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 1.633s	user 0.011s	sys 0.003s
I20260709 12:23:16.961880 19635 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:16.962389 19552 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:16.962908 19526 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:16.963842 19113 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:16.964735 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:16.964753 16516 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:16.965410 16402 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:23:16.966305 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:16.967521 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:16.757750 19659 consensus_queue.cc:237] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:16.933109 19684 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:16.968708 19000 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:16.970382 19684 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:16.919910 19700 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:16.970585 19532 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:16.970923 19684 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:16.909500 19540 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:16.933429 19687 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:16.758237 19706 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:16.973301 16548 raft_consensus.cc:2410] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:23:16.973574 19687 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:16.974169 19706 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:16.974401 16551 raft_consensus.cc:2393] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 6: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:16.933573 19184 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.975102 19184 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:16.975952 16474 raft_consensus.cc:2393] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 9: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:16.976881 19661 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:16.977027 19184 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:16.947535 19222 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:16.979394 19222 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:16.980605 16469 raft_consensus.cc:1240] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:23:16.981949 19384 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:16.854584 19563 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:16.984177 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:16.982308 19699 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:16.985858 16544 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
I20260709 12:23:16.986534 16543 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:16.989972 16516 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:16.991254 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:16.949542 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:16.996770 16403 raft_consensus.cc:2410] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 12:23:16.980075 19639 consensus_meta.cc:287] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 1.824s	user 0.016s	sys 0.000s
I20260709 12:23:17.002363 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:16.942485 17143 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:16.996567 16550 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
I20260709 12:23:16.968149 16402 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 5.
I20260709 12:23:16.883525 19355 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:16.944959 19309 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:17.030161 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 87735.7us
W20260709 12:23:17.031574 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 297203us
I20260709 12:23:16.945329 19691 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:16.973572 19540 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:16.972247 19700 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:17.085804 17143 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:17.007800 19662 log.cc:927] Time spent T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.056s	user 0.000s	sys 0.003s
I20260709 12:23:17.023075 19113 raft_consensus.cc:2804] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:17.090710 16473 raft_consensus.cc:1240] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:17.091615 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:17.076262 19643 consensus_meta.cc:287] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 1.816s	user 0.010s	sys 0.005s
W20260709 12:23:17.037038 19711 log.cc:927] Time spent T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.076s	user 0.000s	sys 0.003s
I20260709 12:23:17.074421 19494 raft_consensus.cc:2804] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:23:17.085503 19638 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:17.101352 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 302376us
I20260709 12:23:17.102173 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:23:17.103546 19698 log.cc:927] Time spent T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.231s	user 0.000s	sys 0.003s
I20260709 12:23:17.096985 19113 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.104880 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:17.028013 19222 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.032452 19385 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.101095 19674 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:17.089712 19384 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:17.100122 16402 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:23:17.111493 16515 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:17.118750 16515 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:17.104442 17143 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:17.121750 16515 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:17.122658 19677 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:23:17.119961 19636 consensus_meta.cc:287] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 1.634s	user 0.009s	sys 0.006s
I20260709 12:23:17.123235 19718 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.123637 19636 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:17.122936 16476 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34840 (request call id 1121) took 2163 ms (client timeout 1924 ms). Trace:
I20260709 12:23:17.109782 19494 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:17.090937 19691 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:17.128561 16476 rpcz_store.cc:269] 0709 12:23:14.959071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:14.960417 (+  1346us) service_pool.cc:224] Handling call
0709 12:23:17.122730 (+2162313us) spinlock_profiling.cc:243] Waited 383 ms on lock 0x7b5400a68bd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:17.122851 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":882013952}
I20260709 12:23:17.129104 16515 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:17.132215 16395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 810) took 2185 ms. Trace:
I20260709 12:23:17.131006 17143 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:17.115438 19113 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:17.132807 16395 rpcz_store.cc:276] 0709 12:23:14.947191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:14.948612 (+  1421us) service_pool.cc:224] Handling call
0709 12:23:14.949023 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:14.949561 (+   538us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:14.949611 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:14.950221 (+   610us) log.cc:844] Serialized 26 byte log entry
0709 12:23:14.998229 (+ 48008us) spinlock_profiling.cc:243] Waited 5.03 ms on lock 0x7b4800001a58. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287adb2aa1 00007f287ada9b37 00007f287ada99ca 00007f287aea126f 00007f287aea0f37 00007f287aeb555c 00007f287aeb5508 00007f287aeb5488 00007f287aeb5443 00007f287aeb46ec 00007f2879ddea96
0709 12:23:15.347994 (+349765us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:15.348055 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:15.348148 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:17.129297 (+1781149us) raft_consensus.cc:1685] finished
0709 12:23:17.129319 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:17.132152 (+  2833us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":118,"raft.run_cpu_time_us":2071,"raft.run_wall_time_us":43485,"spinlock_wait_cycles":11577600,"thread_start_us":355941,"threads_started":1,"wal-append.queue_time_us":1972819}
I20260709 12:23:17.123175 19677 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:17.140764 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:17.141955 16403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 1050) took 1663 ms. Trace:
I20260709 12:23:17.142294 16403 rpcz_store.cc:276] 0709 12:23:15.478477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:17.128703 (+1650226us) service_pool.cc:224] Handling call
0709 12:23:17.141630 (+ 12927us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:17.141746 (+   116us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:17.141761 (+    15us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:17.141786 (+    25us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:17.141804 (+    18us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:17.141835 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:17.141907 (+    72us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:17.129827 19540 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:23:17.145535 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:17.145182 16515 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:17.144604 17258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 882) took 1486 ms. Trace:
I20260709 12:23:17.137051 19674 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:17.132241 19494 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:17.148545 17258 rpcz_store.cc:276] 0709 12:23:15.658744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:15.659026 (+   282us) service_pool.cc:224] Handling call
0709 12:23:15.714498 (+ 55472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:15.720751 (+  6253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:15.720801 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:15.749171 (+ 28370us) log.cc:844] Serialized 26 byte log entry
0709 12:23:16.458074 (+708903us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:16.458123 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:16.458209 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:17.144323 (+686114us) raft_consensus.cc:1685] finished
0709 12:23:17.144417 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:17.144548 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5684,"raft.queue_time_us":9064,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":13453,"thread_start_us":582005,"threads_started":1,"wal-append.queue_time_us":823985}
I20260709 12:23:17.110108 19643 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:17.150018 16395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 819) took 1657 ms. Trace:
I20260709 12:23:17.130779 19641 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:17.151137 16515 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:17.144466 19677 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:17.152272 19384 raft_consensus.cc:2804] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:17.152710 19563 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:17.157454 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:17.158913 16403 raft_consensus.cc:1240] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:17.159873 16397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 813) took 2211 ms. Trace:
I20260709 12:23:17.233223 16548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37226 (request call id 1255) took 1123 ms. Trace:
I20260709 12:23:17.164752 16395 rpcz_store.cc:276] 0709 12:23:15.492479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:17.138378 (+1645899us) service_pool.cc:224] Handling call
0709 12:23:17.149602 (+ 11224us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:17.149769 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:17.149784 (+    15us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:17.149813 (+    29us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:17.149840 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:17.149874 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:17.149944 (+    70us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:17.233599 16548 rpcz_store.cc:276] 0709 12:23:16.109827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:16.928857 (+819030us) service_pool.cc:224] Handling call
0709 12:23:17.187610 (+258753us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b54009745d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:17.233182 (+ 45572us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30563584}
I20260709 12:23:17.233716 16403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 820) took 1706 ms. Trace:
I20260709 12:23:17.120772 19222 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:17.159546 19355 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:17.123787 19718 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:17.170825 19697 log.cc:927] Time spent T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 12:23:17.172101 19559 consensus_meta.cc:287] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 1.560s	user 0.012s	sys 0.002s
I20260709 12:23:17.175165 16515 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:17.175402 19540 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:17.170750 19704 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:17.178524 19631 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:17.174329 19384 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.193583 19239 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:17.203426 19572 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:17.231035 16546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37226 (request call id 1257) took 1120 ms. Trace:
I20260709 12:23:17.207419 19724 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:23:17.231652 16551 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37242 (request call id 1116) took 1854 ms (client timeout 1636 ms). Trace:
I20260709 12:23:17.231565 17270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45224 (request call id 928) took 1399 ms. Trace:
I20260709 12:23:17.231801 16544 raft_consensus.cc:1240] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 4. Current term is 6. Ops: []
I20260709 12:23:17.232156 17255 raft_consensus.cc:1240] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 7. Current term is 8. Ops: []
W20260709 12:23:17.555110 16551 rpcz_store.cc:269] 0709 12:23:15.376773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:15.377201 (+   428us) service_pool.cc:224] Handling call
0709 12:23:17.191365 (+1814164us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b5400bb5654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:17.231584 (+ 40219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3670822144}
I20260709 12:23:17.232306 16469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 586) took 1786 ms. Trace:
I20260709 12:23:17.232652 17271 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
I20260709 12:23:17.233708 16543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37226 (request call id 1260) took 1122 ms. Trace:
I20260709 12:23:17.556070 16469 rpcz_store.cc:276] 0709 12:23:15.445655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:15.446319 (+   664us) service_pool.cc:224] Handling call
0709 12:23:15.446601 (+   282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:16.981060 (+1534459us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:17.189832 (+208772us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b5400594554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:17.232237 (+ 42405us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3527208704}
I20260709 12:23:17.556811 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:17.234942 16403 rpcz_store.cc:276] 0709 12:23:15.527551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:17.142895 (+1615344us) service_pool.cc:224] Handling call
0709 12:23:17.158763 (+ 15868us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:17.233456 (+ 74693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:17.233664 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:17.233541 16397 rpcz_store.cc:276] 0709 12:23:14.948449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:14.948584 (+   135us) service_pool.cc:224] Handling call
0709 12:23:14.979084 (+ 30500us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:14.979473 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:14.979521 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:15.061184 (+ 81663us) log.cc:844] Serialized 26 byte log entry
0709 12:23:15.449698 (+388514us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:15.449736 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:15.449810 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:17.159660 (+1709850us) raft_consensus.cc:1685] finished
0709 12:23:17.159683 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:17.159829 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13234,"raft.queue_time_us":372,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":11450,"thread_start_us":346169,"threads_started":1,"wal-append.queue_time_us":2027008}
I20260709 12:23:17.237639 17254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 914) took 1530 ms. Trace:
I20260709 12:23:17.546938 17263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 887) took 1790 ms. Trace:
I20260709 12:23:17.558314 17263 rpcz_store.cc:276] 0709 12:23:15.756938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:15.797386 (+ 40448us) service_pool.cc:224] Handling call
0709 12:23:15.797683 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:16.413341 (+615658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:17.263901 (+850560us) spinlock_profiling.cc:243] Waited 593 ms on lock 0x7b540080c854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:17.546883 (+282982us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1365032192}
I20260709 12:23:17.558053 17254 rpcz_store.cc:276] 0709 12:23:15.707575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:15.768663 (+ 61088us) service_pool.cc:224] Handling call
0709 12:23:15.811456 (+ 42793us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:15.811976 (+   520us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:15.812022 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:15.868960 (+ 56938us) log.cc:844] Serialized 26 byte log entry
0709 12:23:16.470568 (+601608us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:16.481271 (+ 10703us) log.cc:844] Serialized 14 byte log entry
0709 12:23:16.486799 (+  5528us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:16.486905 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:17.237461 (+750556us) raft_consensus.cc:1685] finished
0709 12:23:17.237483 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:17.237595 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2088,"prepare.queue_time_us":10765,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":6816,"raft.queue_time_us":279691,"raft.run_cpu_time_us":403,"raft.run_wall_time_us":56102,"thread_start_us":272026,"threads_started":1,"wal-append.queue_time_us":1100018}
I20260709 12:23:17.238955 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:17.241137 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 66075.1us
I20260709 12:23:17.385406 19682 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:17.238651 19641 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:17.387080 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:17.387281 19222 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:17.254736 19384 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:17.388190 19695 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:17.388474 16477 raft_consensus.cc:1275] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:23:17.590639 16477 pending_rounds.cc:85] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 4
I20260709 12:23:17.591099 16477 pending_rounds.cc:107] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 6.5
W20260709 12:23:17.548995 19720 log.cc:927] Time spent T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.310s	user 0.000s	sys 0.002s
I20260709 12:23:17.591387 16477 raft_consensus.cc:2889] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:23:17.389096 19671 log.cc:927] Time spent T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.888s	user 0.007s	sys 0.011s
I20260709 12:23:17.388916 19718 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:17.389600 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:17.596658 16400 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 5.
W20260709 12:23:17.275142 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:17.390066 19580 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:17.276877 19673 log.cc:927] Time spent T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 1.352s	user 0.001s	sys 0.009s
W20260709 12:23:17.390429 19685 log.cc:927] Time spent T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 1.460s	user 0.000s	sys 0.015s
I20260709 12:23:17.390861 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:17.391336 19691 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:17.609321 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:17.610001 16469 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:17.563594 19724 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:17.391891 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:17.394145 19636 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:17.394156 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:17.395257 19633 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:17.396222 17144 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:17.616253 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 220077us
I20260709 12:23:17.616504 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:17.397153 19677 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:17.245474 19629 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:17.433662 16473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 587) took 1987 ms. Trace:
I20260709 12:23:17.260794 16472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 590) took 1488 ms. Trace:
I20260709 12:23:17.317232 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:17.621347 16548 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:23:17.385522 16549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 621) took 1609 ms. Trace:
I20260709 12:23:17.541189 19391 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:17.621982 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:17.542224 19699 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.626034 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:17.545523 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:23:17.546114 16442 leader_election.cc:336] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.814s (SENT)
I20260709 12:23:17.242197 16546 rpcz_store.cc:276] 0709 12:23:16.110398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:16.291958 (+181560us) service_pool.cc:224] Handling call
0709 12:23:17.208809 (+916851us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b5400bb5154. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:17.230458 (+ 21649us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":642457344}
I20260709 12:23:17.546540 16545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1115) took 2170 ms. Trace:
I20260709 12:23:17.547510 19527 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:17.547874 19546 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:17.548797 16542 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:17.549471  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:17.551888 17269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 880) took 1905 ms. Trace:
I20260709 12:23:17.554330 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:17.556223 16543 rpcz_store.cc:276] 0709 12:23:16.111245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:16.928805 (+817560us) service_pool.cc:224] Handling call
0709 12:23:17.188241 (+259436us) spinlock_profiling.cc:243] Waited 52.5 ms on lock 0x7b54000663d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:17.233649 (+ 45408us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":120678656}
I20260709 12:23:17.120692 19385 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:17.555241 17270 rpcz_store.cc:276] 0709 12:23:15.832595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:16.236209 (+403614us) service_pool.cc:224] Handling call
0709 12:23:17.202477 (+966268us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b54001a0254. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:17.231511 (+ 29034us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":432802560}
I20260709 12:23:17.557524 16551 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:23:17.561978 16515 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:17.574705 16544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1251) took 1505 ms. Trace:
I20260709 12:23:17.579536 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:17.581408 16441 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:17.583329 16474 raft_consensus.cc:2393] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 9: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:17.582377 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:17.588048 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:17.588238 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:17.588711 17255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 881) took 1938 ms. Trace:
I20260709 12:23:17.664966 16547 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
W20260709 12:23:17.665333 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 103424us
I20260709 12:23:17.666184 16515 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:17.627204 16398 raft_consensus.cc:2393] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 7: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:17.666597 17255 rpcz_store.cc:276] 0709 12:23:15.650179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:15.650354 (+   175us) service_pool.cc:224] Handling call
0709 12:23:15.650698 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:17.555781 (+1905083us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:17.576849 (+ 21068us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b54007053d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:17.588643 (+ 11794us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3560726400}
I20260709 12:23:17.666491 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:17.545430 19668 log.cc:927] Time spent T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 1.640s	user 0.013s	sys 0.003s
I20260709 12:23:17.595078 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:17.668815 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:17.606964 17142 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:17.384684 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:17.580659 19722 log.cc:927] Time spent T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.420s	user 0.003s	sys 0.008s
I20260709 12:23:17.669806 16395 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
I20260709 12:23:17.611699 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:17.666672 17269 rpcz_store.cc:276] 0709 12:23:15.646908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:15.647091 (+   183us) service_pool.cc:224] Handling call
0709 12:23:15.688967 (+ 41876us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:16.647511 (+958544us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:17.414936 (+767425us) spinlock_profiling.cc:243] Waited 832 ms on lock 0x7b540055f354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:17.551725 (+136789us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1912825600}
I20260709 12:23:17.670691 16400 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:23:17.671723 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:17.627378 16442 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:17.391788 17266 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45224 (request call id 924) took 1561 ms (client timeout 1897 ms). Trace:
I20260709 12:23:17.582360 19487 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.582204 19239 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:17.611860 19615 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:17.617719 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:17.575356 19585 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:23:17.606153 19655 log.cc:927] Time spent T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 2.023s	user 0.010s	sys 0.013s
I20260709 12:23:17.618741 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:17.589818 19641 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:17.620747 16473 rpcz_store.cc:276] 0709 12:23:15.446143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:15.446266 (+   123us) service_pool.cc:224] Handling call
0709 12:23:15.446614 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:17.095677 (+1649063us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:17.431849 (+336172us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b54005e6354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:17.433546 (+  1697us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3780452352}
I20260709 12:23:17.622160 16549 rpcz_store.cc:276] 0709 12:23:15.776182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:15.776351 (+   169us) service_pool.cc:224] Handling call
0709 12:23:15.776727 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:15.776984 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:23:15.875309 (+ 98325us) log.cc:844] Serialized 14 byte log entry
0709 12:23:17.249877 (+1374568us) spinlock_profiling.cc:243] Waited 71.9 ms on lock 0x7b1800296b60. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2875a314bc 00007f2872d7f48b 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a2923a
0709 12:23:17.385116 (+135239us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:17.385168 (+    52us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:17.385224 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:17.385313 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:17.385448 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34102,"prepare.queue_time_us":109675,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":115,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":108,"spinlock_wait_cycles":165277568,"thread_start_us":102944,"threads_started":1,"wal-append.queue_time_us":1207705}
W20260709 12:23:17.673091 17266 rpcz_store.cc:269] 0709 12:23:15.831147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:15.929385 (+ 98238us) service_pool.cc:224] Handling call
0709 12:23:17.247405 (+1318020us) spinlock_profiling.cc:243] Waited 411 ms on lock 0x7b54003f0554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:17.391735 (+144330us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":946344320}
I20260709 12:23:17.620824 16472 rpcz_store.cc:276] 0709 12:23:15.771963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:15.772855 (+   892us) service_pool.cc:224] Handling call
0709 12:23:17.123353 (+1350498us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:17.123561 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:17.123592 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:17.123640 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:17.123685 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:17.123744 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:17.260530 (+136786us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b5400a68bd0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:17.260710 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2964217088}
I20260709 12:23:17.618664 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:17.672825 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:17.676824 19641 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:17.670156 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:17.673369 19239 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:17.677840 16403 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:23:17.666853 16544 rpcz_store.cc:276] 0709 12:23:16.069490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:17.024479 (+954989us) service_pool.cc:224] Handling call
0709 12:23:17.231601 (+207122us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:17.555395 (+323794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:17.574484 (+ 19089us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b5400bb5650. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:17.574642 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":353411328}
I20260709 12:23:17.673805 19487 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:17.679068 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:17.679126 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:17.680281 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:17.680513 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:17.673645 19585 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:17.681458 16544 raft_consensus.cc:2410] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:23:17.682132 16403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 821) took 2042 ms. Trace:
I20260709 12:23:17.682236 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:17.677860 16400 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:17.674377 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: true dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:17.683934 16395 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:17.684927 19239 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:17.682816 19585 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:17.673645 16475 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:23:17.686277 16398 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:17.682972 16544 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
I20260709 12:23:17.690651 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:17.691979 16544 raft_consensus.cc:2393] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 13: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:17.672875 16545 rpcz_store.cc:276] 0709 12:23:15.376485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:15.410104 (+ 33619us) service_pool.cc:224] Handling call
0709 12:23:15.453179 (+ 43075us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:15.688943 (+235764us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:15.688979 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:15.689041 (+    62us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:15.689074 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:17.413426 (+1724352us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b54004d7454. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:17.546235 (+132809us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:17.546460 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":541653376}
I20260709 12:23:17.680454 19487 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:17.692140 16514 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:17.695247 19641 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:17.695279 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:17.695927 19641 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.696311 19641 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:17.609237 19000 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.682694 16403 rpcz_store.cc:276] 0709 12:23:15.639505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:17.681306 (+2041801us) service_pool.cc:224] Handling call
0709 12:23:17.681619 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:17.681800 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:17.681829 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:17.681876 (+    47us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:17.681912 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:17.681966 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:17.682077 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:17.707535 17143 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:17.693210 16395 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:17.694088 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:17.697770 17255 raft_consensus.cc:1275] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:23:17.705615 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:17.697618 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:17.701433 16515 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:17.669414 16477 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:17.710268 16474 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:17.705119 16542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:52680 (request call id 622) took 1890 ms. Trace:
I20260709 12:23:17.710706 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:17.711669 17264 raft_consensus.cc:2393] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 7: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:17.711143 16542 rpcz_store.cc:276] 0709 12:23:15.786406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:15.786848 (+   442us) service_pool.cc:224] Handling call
0709 12:23:17.418663 (+1631815us) spinlock_profiling.cc:243] Waited 30.2 ms on lock 0x7b4c0033bf80. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874155a20 00007f2874166a29 00007f287416713f 00007f287412b2ab 00007f2874158f3c 00007f28741593fe 00007f28743e688b 00007f2875de5495 00007f2875e3630f 00007f2875e80a55 00007f287ae50cd0
0709 12:23:17.667261 (+248598us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":69541120}
W20260709 12:23:17.706779 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 99888.1us
I20260709 12:23:17.713409 17142 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:17.714097 16402 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:23:17.715551 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:17.708712 16544 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:17.660914 19266 consensus_queue.cc:1048] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 12:23:17.721987 16400 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:23:17.723675 16395 raft_consensus.cc:1240] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:17.723780 16400 raft_consensus.cc:1275] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:23:17.615235 19724 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:17.672652 19546 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:17.724956 19266 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:17.725373 19674 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:17.725396 19266 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.725980 19674 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.726078 19266 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:17.726464 19674 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:17.709369 19687 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:17.728826 19687 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:17.733184 19630 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:23:17.734112 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 152794us
I20260709 12:23:17.732784 19642 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:23:17.738220 16441 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:17.739907 19630 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:17.740672 19630 consensus_queue.cc:1048] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:17.742926 16441 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:17.744096 19642 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:17.745251 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:17.746001 16395 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:23:17.746656 16474 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:23:17.746785 16441 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:17.746924 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:17.747985 19642 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:17.748432 19642 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.748780 19642 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:17.749879 19643 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:17.627149 19624 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:17.751047 19624 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:17.751754 19624 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:17.603811 19494 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:17.758244 19724 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:17.758590 19494 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:17.759519 19000 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:17.761162 17263 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
W20260709 12:23:17.760856 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 214762us
I20260709 12:23:17.762792 19000 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:17.763139 16442 leader_election.cc:336] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.891s (SENT)
I20260709 12:23:17.763752 16442 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:17.764237 16514 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:17.764608 19640 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:23:17.765386 19633 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:17.765522 16514 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:17.765851 19633 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.766237 19724 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:17.766367 19580 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:17.766259 19633 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:17.766976 19580 raft_consensus.cc:697] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:17.767722 16396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 815) took 2657 ms. Trace:
I20260709 12:23:17.769009 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:17.768574 19580 consensus_queue.cc:237] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:17.770516 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:17.771313 17258 raft_consensus.cc:2393] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 11: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:17.772658 19640 consensus_queue.cc:1048] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:23:17.774102 19686 log.cc:927] Time spent T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 1.236s	user 0.006s	sys 0.013s
I20260709 12:23:17.775290 19640 consensus_queue.cc:1048] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:17.775970 19633 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:17.776929 19540 consensus_meta.cc:287] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.535s	user 0.013s	sys 0.000s
I20260709 12:23:17.777793 19540 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:17.697340 19699 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:17.781489 16471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 584) took 2336 ms. Trace:
I20260709 12:23:17.782104 16471 rpcz_store.cc:276] 0709 12:23:15.444918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:15.445062 (+   144us) service_pool.cc:224] Handling call
0709 12:23:15.445421 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:15.445829 (+   408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:15.445860 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:15.475888 (+ 30028us) log.cc:844] Serialized 26 byte log entry
0709 12:23:15.697303 (+221415us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:15.697364 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:15.697465 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:17.244550 (+1547085us) spinlock_profiling.cc:243] Waited 7.28 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:23:17.781104 (+536554us) raft_consensus.cc:1685] finished
0709 12:23:17.781137 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:17.781410 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13553,"raft.queue_time_us":2362,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":30374,"spinlock_wait_cycles":16748928,"thread_start_us":137834,"threads_started":1,"wal-append.queue_time_us":1018802}
I20260709 12:23:17.784092 16545 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:17.782853 19633 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:17.789626 19222 consensus_queue.cc:1048] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:17.790493 19677 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:17.792021 16441 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:17.793150 19222 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:17.793608 19222 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.794009 19222 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:17.799167 19113 consensus_meta.cc:287] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.655s	user 0.006s	sys 0.003s
I20260709 12:23:17.800848 17271 raft_consensus.cc:1240] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:23:17.590101 19639 consensus_queue.cc:1059] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 12:23:17.766846 16399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 812) took 2818 ms. Trace:
I20260709 12:23:17.803517 16441 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:17.805716 16399 rpcz_store.cc:276] 0709 12:23:14.948025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:14.948206 (+   181us) service_pool.cc:224] Handling call
0709 12:23:14.982085 (+ 33879us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:15.032411 (+ 50326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:15.032442 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:15.052162 (+ 19720us) log.cc:844] Serialized 26 byte log entry
0709 12:23:15.369773 (+317611us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:15.369822 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:15.369893 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:17.246975 (+1877082us) spinlock_profiling.cc:243] Waited 49.5 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:23:17.766536 (+519561us) raft_consensus.cc:1685] finished
0709 12:23:17.766558 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:17.766784 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13237,"raft.queue_time_us":112,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":11484,"spinlock_wait_cycles":113826176,"thread_start_us":266494,"threads_started":1,"wal-append.queue_time_us":629561}
I20260709 12:23:17.783149 19266 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:17.817713 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:17.818660 16542 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:17.820643 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:17.821322 16548 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:17.706010 19391 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:17.709844 19578 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:17.799854 19113 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:17.841311 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:17.842298 16546 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:17.843535 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:17.851892 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:17.816699 19629 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.852670 16546 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:23:17.855716 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:17.858990 16550 raft_consensus.cc:1240] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8. Ops: [5.5-7.6]
I20260709 12:23:17.859328 16546 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:23:17.867638 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:17.870312 19723 log.cc:927] Time spent T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.486s	user 0.003s	sys 0.000s
I20260709 12:23:17.861618 19642 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:17.874425 19385 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:17.874581 19642 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:17.875212 19642 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:17.875615 19642 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:17.876006 19642 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:17.879606 19647 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.880200 19647 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:17.879688 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:17.882251 19266 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:17.882437 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:17.883126 19266 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.883646 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:17.884348 16543 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
I20260709 12:23:17.884490 19222 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:17.883186 16549 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:23:17.885290 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:17.885908 16543 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:17.887872 19222 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:17.853475 19700 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.038s	sys 0.072s
I20260709 12:23:17.888695 19624 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:17.883884 19266 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:17.855897 19391 raft_consensus.cc:697] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 10 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:17.894387 19527 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:17.857285 19629 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:17.900245 19700 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.038s	sys 0.073s
I20260709 12:23:17.906088 17270 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:17.935065 17261 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:17.905709 19527 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:17.903268 19391 consensus_queue.cc:237] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:17.936518 17261 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45230 (request call id 877) took 2412 ms (client timeout 2018 ms). Trace:
W20260709 12:23:17.936928 17261 rpcz_store.cc:269] 0709 12:23:15.524475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:15.524653 (+   178us) service_pool.cc:224] Handling call
0709 12:23:17.936268 (+2411615us) spinlock_profiling.cc:243] Waited 2.32 s on lock 0x7b54005f7b54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:17.936411 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5341846272}
I20260709 12:23:17.942226 19546 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:17.950419 19527 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:17.954272 19113 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:17.956633 19624 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:17.962421 19266 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:17.966900 19629 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:17.985464 19546 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:17.962517 19417 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:17.966667 19642 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:17.995508 19641 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:18.014493 19699 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:18.012156 19647 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:18.079906 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:18.082293 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 367468us
I20260709 12:23:18.086156 16367 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:18.089813 16367 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:18.093832 16367 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:17.768152 16396 rpcz_store.cc:276] 0709 12:23:15.110094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:15.110246 (+   152us) service_pool.cc:224] Handling call
0709 12:23:15.110659 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:15.111393 (+   734us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:15.111444 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:15.112153 (+   709us) log.cc:844] Serialized 26 byte log entry
0709 12:23:15.448649 (+336496us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:15.448700 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:15.448786 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:17.197238 (+1748452us) spinlock_profiling.cc:243] Waited 1.37 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:23:17.767416 (+570178us) raft_consensus.cc:1685] finished
0709 12:23:17.767439 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:17.767661 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":95,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":96,"spinlock_wait_cycles":3160320,"thread_start_us":335998,"threads_started":1,"wal-append.queue_time_us":1340613}
I20260709 12:23:18.097690 16398 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:18.098124 16394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 1044) took 2656 ms. Trace:
I20260709 12:23:18.064986  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:18.132833 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:18.133174 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:18.136575 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:18.137030 16402 raft_consensus.cc:1240] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:18.141990 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:18.141752 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:18.143643 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:18.143992 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:18.145002 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:18.145733 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:17.862592 16545 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:23:18.150020 16546 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:18.150704 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:18.153784 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 440257us
W20260709 12:23:18.157258 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 449556us
I20260709 12:23:18.158263 19113 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.159699 19585 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:18.328573 16397 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:23:18.166806 19641 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:17.996424 19384 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:18.172770 19642 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:18.175704 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:18.031718 19674 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:18.190999 16542 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 9. Current term is 10. Ops: []
W20260709 12:23:18.202521 16469 consensus_meta.cc:287] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.550s	user 0.004s	sys 0.023s
I20260709 12:23:18.204649 19385 raft_consensus.cc:3055] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:18.202536 19730 consensus_queue.cc:1059] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 12:23:18.200079 16550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1120) took 1506 ms. Trace:
I20260709 12:23:18.218722 16548 raft_consensus.cc:1275] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:18.221019 16551 raft_consensus.cc:1275] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:23:18.222745 16226 catalog_manager.cc:5697] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:23:18.229568 19699 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.231508 16516 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:18.305316 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 211751us
I20260709 12:23:18.313294 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:18.315073 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:18.316344 19700 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:18.278741 16394 rpcz_store.cc:276] 0709 12:23:15.441666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:15.805235 (+363569us) service_pool.cc:224] Handling call
0709 12:23:17.403935 (+1598700us) spinlock_profiling.cc:243] Waited 23.8 ms on lock 0x7b5800125198. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:23:17.540936 (+137001us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:17.541224 (+   288us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:23:17.596605 (+ 55381us) log.cc:844] Serialized 14 byte log entry
0709 12:23:18.096748 (+500143us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:18.096905 (+   157us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:18.097061 (+   156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:18.097348 (+   287us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:18.097759 (+   411us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26282,"prepare.queue_time_us":47710,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":64012,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":277,"spinlock_wait_cycles":54839680,"thread_start_us":499283,"threads_started":1,"wal-append.queue_time_us":416544}
I20260709 12:23:18.320538 16468 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:18.321269 16543 raft_consensus.cc:1275] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:23:18.323596 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:18.325389 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:18.351043 16396 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
I20260709 12:23:18.324800 19113 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:18.324901 16477 raft_consensus.cc:2393] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 9: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:18.355911 19736 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.336912 16550 rpcz_store.cc:276] 0709 12:23:16.693121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:17.073091 (+379970us) service_pool.cc:224] Handling call
0709 12:23:17.073575 (+   484us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:17.859812 (+786237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:18.199227 (+339415us) spinlock_profiling.cc:243] Waited 785 ms on lock 0x7b5400285354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:18.199825 (+   598us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1805368576}
I20260709 12:23:18.367107 16543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 625) took 1231 ms. Trace:
I20260709 12:23:18.363386 16469 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:23:18.368706 16473 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:18.356941 16542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 633) took 1182 ms. Trace:
I20260709 12:23:18.369297 16542 rpcz_store.cc:276] 0709 12:23:17.157086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:18.189954 (+1032868us) service_pool.cc:224] Handling call
0709 12:23:18.190557 (+   603us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:18.336637 (+146080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:18.339017 (+  2380us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:23:18.374457 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 51513.9us
I20260709 12:23:18.370455 16472 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:23:18.367771 16543 rpcz_store.cc:276] 0709 12:23:17.135592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:17.886496 (+750904us) service_pool.cc:224] Handling call
0709 12:23:18.320579 (+434083us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:18.348565 (+ 27986us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:18.366645 (+ 18080us) spinlock_profiling.cc:243] Waited 424 ms on lock 0x7b5400304650. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:18.366985 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":975135488}
I20260709 12:23:18.335947 19385 raft_consensus.cc:740] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:18.357596 19700 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:18.385668 16402 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:23:18.386178 16367 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:18.388548 19585 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:18.389012 19585 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:18.389430 19585 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:18.393357 16398 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:18.400189 19699 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:18.404493 19113 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:18.410789 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:18.392057 16403 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 9. Current term is 10. Ops: []
W20260709 12:23:18.409968 16401 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.031s	sys 0.025s
I20260709 12:23:18.383522 16475 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
W20260709 12:23:18.415329 16401 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.032s	sys 0.025s
I20260709 12:23:18.412777 16367 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:18.368810 19736 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:18.418977 19639 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:18.419521 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 46223.3us
I20260709 12:23:18.419866 19639 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:18.421314 19736 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:18.421895 16514 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:18.422690 19639 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:18.423182 16514 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:18.424237 19580 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:18.424702 19580 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.425138 19580 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:18.429199 19385 consensus_queue.cc:260] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:18.430569 19730 raft_consensus.cc:3055] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:18.431236 19730 raft_consensus.cc:740] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:18.431417 19385 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:23:18.432926 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 201853us
I20260709 12:23:18.434278 16516 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:18.436785 19682 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:18.437343 19682 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.437780 19682 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:23:18.440518 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 59998.1us
I20260709 12:23:18.441268 19633 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:18.447386 16468 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
I20260709 12:23:18.410538 19700 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:18.427939 19674 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:18.449046 19546 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:18.464821 19546 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:18.465250 19546 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:18.466589 16403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 822) took 1697 ms. Trace:
I20260709 12:23:18.467025 16403 rpcz_store.cc:276] 0709 12:23:16.769074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:17.718403 (+949329us) service_pool.cc:224] Handling call
0709 12:23:18.332454 (+614051us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:18.417731 (+ 85277us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:18.466284 (+ 48553us) spinlock_profiling.cc:243] Waited 560 ms on lock 0x7b540015e350. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:18.466447 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1287967104}
W20260709 12:23:18.448866 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 116038us
I20260709 12:23:18.455916 19239 raft_consensus.cc:2804] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:18.454643 19495 consensus_queue.cc:1048] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 12:23:18.472059 19417 consensus_queue.cc:1059] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:18.476934 16398 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:18.425144 19578 raft_consensus.cc:2804] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:23:18.479274 16473 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:18.483366 19578 raft_consensus.cc:697] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:23:18.485132 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 96881.5us
I20260709 12:23:18.481290 16395 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:23:18.493083 16516 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:18.492234 16397 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:23:18.501719 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:18.502497 16441 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.004s	sys 0.015s
W20260709 12:23:18.503837 16442 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.007s	sys 0.011s
I20260709 12:23:18.503501 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:18.505539 16514 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:18.509943 16548 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:23:18.510797 16441 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.005s	sys 0.015s
W20260709 12:23:18.512297 16442 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.746s	user 0.008s	sys 0.012s
I20260709 12:23:18.514995 16551 raft_consensus.cc:1240] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:23:18.483989 19730 consensus_queue.cc:260] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:18.519239 16226 catalog_manager.cc:5697] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:23:18.528276 16545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52680 (request call id 628) took 1380 ms. Trace:
I20260709 12:23:18.489977 19578 consensus_queue.cc:237] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:18.529196 19391 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:18.532205 16545 rpcz_store.cc:276] 0709 12:23:17.137997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:17.694830 (+556833us) service_pool.cc:224] Handling call
0709 12:23:18.500322 (+805492us) spinlock_profiling.cc:243] Waited 87.8 ms on lock 0x7b54005e0954. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:18.517867 (+ 17545us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":201894272}
I20260709 12:23:18.515942 19239 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:23:18.545244 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 779261us
W20260709 12:23:18.539104 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 735569us
I20260709 12:23:18.530638 17144 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:18.543545 16367 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:18.538111 19633 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:18.542125 19239 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:18.556181 19487 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:18.556763 19487 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:18.557153 19487 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:18.558279 16514 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:18.554965 19633 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.569715 16549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1121) took 1870 ms. Trace:
I20260709 12:23:18.579061 19624 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:18.571391 19633 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:18.582001 16514 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:18.579618 19624 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:18.576802 19738 log.cc:927] Time spent T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.204s	user 0.004s	sys 0.002s
I20260709 12:23:18.571473 16549 rpcz_store.cc:276] 0709 12:23:16.693454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:17.885982 (+1192528us) service_pool.cc:224] Handling call
0709 12:23:17.886303 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:17.887255 (+   952us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:17.887287 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:17.887824 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 12:23:18.433178 (+545354us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:18.433268 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:18.433364 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:18.558136 (+124772us) raft_consensus.cc:1685] finished
0709 12:23:18.558159 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:18.563789 (+  5630us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":412,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":84,"thread_start_us":493153,"threads_started":1,"wal-append.queue_time_us":634589}
I20260709 12:23:18.582888 19699 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:18.586905 16544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 629) took 1442 ms. Trace:
I20260709 12:23:18.576694 19113 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:18.589932 19730 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:18.588598 16544 rpcz_store.cc:276] 0709 12:23:17.138289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:18.153915 (+1015626us) service_pool.cc:224] Handling call
0709 12:23:18.154971 (+  1056us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:18.156004 (+  1033us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:18.156072 (+    68us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:18.213233 (+ 57161us) log.cc:844] Serialized 26 byte log entry
0709 12:23:18.357187 (+143954us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:18.357238 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:18.357300 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:18.580277 (+222977us) raft_consensus.cc:1685] finished
0709 12:23:18.580297 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:18.580937 (+   640us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6925,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":284,"thread_start_us":127183,"threads_started":1,"wal-append.queue_time_us":291808}
W20260709 12:23:18.591248 16225 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53622 (request call id 133) took 29380 ms (client timeout 15000 ms). Trace:
I20260709 12:23:18.594026 19113 raft_consensus.cc:697] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:18.593919 16441 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:23:18.599015 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:18.600651 19113 consensus_queue.cc:237] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:18.586135 19624 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:23:18.616658 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 159306us
W20260709 12:23:18.618940 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 47675.1us
I20260709 12:23:18.625808 19580 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:18.630417 19580 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:18.638635 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:18.640950 19417 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.642575 19740 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:18.643118 19740 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.643740 19740 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:18.643103 19417 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:18.647344 19417 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:18.648629 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:18.689670 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:18.690866 17261 raft_consensus.cc:2393] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 6: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:18.649936 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:18.650547 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: true dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:18.693249 17263 raft_consensus.cc:2410] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:23:18.694461 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:18.695149 17263 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:18.651152 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:18.696247 17266 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:18.697332 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:18.698269 16401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 1051) took 2457 ms. Trace:
I20260709 12:23:18.698697 16401 rpcz_store.cc:276] 0709 12:23:16.241335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:17.709125 (+1467790us) service_pool.cc:224] Handling call
0709 12:23:17.718277 (+  9152us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:17.718719 (+   442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:17.718750 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:17.719382 (+   632us) log.cc:844] Serialized 52 byte log entry
0709 12:23:18.455218 (+735836us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:18.455301 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:18.455470 (+   169us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:18.698017 (+242547us) raft_consensus.cc:1685] finished
0709 12:23:18.698037 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:18.698189 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9817,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":90,"thread_start_us":696090,"threads_started":1,"wal-append.queue_time_us":854517}
I20260709 12:23:18.700723 17266 raft_consensus.cc:2393] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 9: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:18.702277 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:18.703006 17266 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 8
W20260709 12:23:18.640928 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 55082.5us
I20260709 12:23:18.651552 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:18.652346 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:18.652752 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:18.709542 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:18.710312 17271 raft_consensus.cc:2393] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 9: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:18.652995 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:18.711071 17269 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
I20260709 12:23:18.653726 16226 catalog_manager.cc:5697] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 1 to 6. New cstate: current_term: 6 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
I20260709 12:23:18.713276 16226 catalog_manager.cc:5697] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 4 to 5, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 5 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:18.656531 19695 consensus_queue.cc:1059] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:18.717468 19695 raft_consensus.cc:3055] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:18.717953 19695 raft_consensus.cc:740] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:18.656880 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:18.720121 19695 consensus_queue.cc:260] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:18.721124 17255 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:23:18.658366 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:18.658984 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:18.722538 16544 raft_consensus.cc:2393] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 9: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:18.660816 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:18.715814 16226 catalog_manager.cc:5697] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:23:18.722100 16545 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:18.725117 16226 catalog_manager.cc:5697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 9 to 12, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 12 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:18.660182 16546 raft_consensus.cc:1240] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:18.691918 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:18.666916 19649 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.717496 19487 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:18.728418 17261 raft_consensus.cc:2393] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 6: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:18.728717 19649 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:18.722134 19695 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:18.667856 19495 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.730715 19495 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:18.731381 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:18.732412 17261 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:23:18.733291 19495 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:18.668471 19384 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:18.669960 19742 log.cc:927] Time spent T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:23:18.737581 19633 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:18.671296 19243 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:18.672663 17144 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:18.739005 19243 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:18.739444 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 97712.1us
I20260709 12:23:18.739848 19633 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:18.740835 17144 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:18.741197 19243 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:18.741902 19243 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:18.742028 19730 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:23:18.742449 19243 raft_consensus.cc:697] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:18.742614 19730 raft_consensus.cc:697] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:18.743770 16401 raft_consensus.cc:1240] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:18.743512 19243 consensus_queue.cc:237] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:18.640928 19699 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:18.682248 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:18.745210 19699 raft_consensus.cc:697] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:18.661625 19674 raft_consensus.cc:3055] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:23:18.682325 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:18.591775 16225 rpcz_store.cc:269] 0709 12:22:49.217028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:49.231756 (+ 14728us) service_pool.cc:224] Handling call
0709 12:23:18.590979 (+29359223us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:11.544368 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:11.648993 (+104625us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:11.649061 (+    68us) write_op.cc:435] Acquired schema lock
0709 12:23:11.649086 (+    25us) tablet.cc:662] Decoding operations
0709 12:23:11.650324 (+  1238us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:11.650427 (+   103us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:11.650467 (+    40us) tablet.cc:685] Acquiring locks for 8 operations
0709 12:23:11.651412 (+   945us) tablet.cc:701] Row locks acquired
0709 12:23:11.651459 (+    47us) write_op.cc:260] PREPARE: finished
0709 12:23:11.651669 (+   210us) write_op.cc:270] Start()
0709 12:23:11.651857 (+   188us) write_op.cc:276] Timestamp: P: 1783599791651575 usec, L: 0
0709 12:23:11.651881 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:23:11.765459 (+113578us) log.cc:844] Serialized 3181 byte log entry
0709 12:23:12.605292 (+839833us) op_driver.cc:464] REPLICATION: finished
0709 12:23:12.932951 (+327659us) write_op.cc:301] APPLY: starting
0709 12:23:12.933095 (+   144us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:12.933367 (+   272us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:12.933544 (+   177us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:12.934796 (+  1252us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:12.935184 (+   388us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 12:23:12.935242 (+    58us) write_op.cc:312] APPLY: finished
0709 12:23:12.935646 (+   404us) log.cc:844] Serialized 64 byte log entry
0709 12:23:12.937119 (+  1473us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:12.937265 (+   146us) write_op.cc:454] Released schema lock
0709 12:23:12.937342 (+    77us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:12.937864 (+   522us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:12.937898 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":854726,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":327335,"apply.run_cpu_time_us":3650,"apply.run_wall_time_us":4879,"num_ops":8,"prepare.queue_time_us":118338,"prepare.run_cpu_time_us":17308,"prepare.run_wall_time_us":396417,"raft.queue_time_us":209501,"raft.run_cpu_time_us":78816,"raft.run_wall_time_us":280072,"replication_time_us":953223,"thread_start_us":643440,"threads_started":4,"wal-append.queue_time_us":378728,"wal-append.run_cpu_time_us":31873,"wal-append.run_wall_time_us":1726345}]]}
I20260709 12:23:18.684700 19700 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:18.678150 19546 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:18.684255 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 166081us
I20260709 12:23:18.685870 16516 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:18.653805 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:18.767351 17254 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:18.722200 16474 raft_consensus.cc:1240] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 10. Current term is 11. Ops: []
W20260709 12:23:18.683509 16442 connection.cc:256] RPC call timeout handler was delayed by -1.055300361999798s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23410000000000003s for a response
I20260709 12:23:18.790361 16548 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
W20260709 12:23:18.790966 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 105326us
I20260709 12:23:18.754055 19674 raft_consensus.cc:740] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:18.792129 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:18.792833 16473 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:18.794293 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:18.794919 16468 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:18.795261 17263 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
I20260709 12:23:18.797896 16400 raft_consensus.cc:1240] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:18.757694 19699 consensus_queue.cc:237] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:18.745549 19730 consensus_queue.cc:237] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:18.794323 19384 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:18.808636 19384 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:18.809116 19384 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:18.813580 19546 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:18.789873 19487 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:18.754463 19239 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:18.815110 16367 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:18.806748 16226 catalog_manager.cc:5697] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
I20260709 12:23:18.816983 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:18.835779 16473 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:18.839036 16468 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
I20260709 12:23:18.832515 16547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 627) took 1629 ms. Trace:
I20260709 12:23:18.792065 19385 consensus_queue.cc:1059] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:23:18.834394 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:18.849380 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:18.842754 16547 rpcz_store.cc:276] 0709 12:23:17.137704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:17.888061 (+750357us) service_pool.cc:224] Handling call
0709 12:23:17.888443 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:17.888845 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:23:18.396259 (+507414us) log.cc:844] Serialized 14 byte log entry
0709 12:23:18.588249 (+191990us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:18.592118 (+  3869us) log.cc:844] Serialized 26 byte log entry
0709 12:23:18.592411 (+   293us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:18.616450 (+ 24039us) log.cc:844] Serialized 14 byte log entry
0709 12:23:18.620483 (+  4033us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:18.752823 (+132340us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:18.752887 (+    64us) raft_consensus.cc:1685] finished
0709 12:23:18.752909 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:18.766994 (+ 14085us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3721,"prepare.queue_time_us":430774,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":113,"raft.queue_time_us":10892,"raft.run_cpu_time_us":440,"raft.run_wall_time_us":20034,"thread_start_us":660071,"threads_started":2,"wal-append.queue_time_us":225206}
I20260709 12:23:18.855316 19385 raft_consensus.cc:3055] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:18.855880 19385 raft_consensus.cc:740] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:18.856204 19674 consensus_queue.cc:260] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:18.860122 19649 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:18.860108 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:18.869604 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:18.857335 19487 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:18.928557 17266 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:23:18.929472 16545 raft_consensus.cc:2468] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:23:18.929633 17271 raft_consensus.cc:1275] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:23:18.932127 17254 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:23:18.934732 17258 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:18.938117 16514 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:18.945158 17263 raft_consensus.cc:1275] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:23:18.940440 16516 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:18.948369 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:18.949339 17266 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:18.950254 17264 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:18.954135 16514 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
W20260709 12:23:18.961081 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 59432.6us
I20260709 12:23:18.934304 19487 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:23:18.973984 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 160156us
I20260709 12:23:18.954982 19391 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:23:18.977885 19682 consensus_meta.cc:287] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.540s	user 0.013s	sys 0.001s
I20260709 12:23:18.978534 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:18.934592 19113 consensus_queue.cc:1059] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:23:18.982024 17266 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
W20260709 12:23:18.968367 16366 leader_election.cc:336] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.924s (SENT)
I20260709 12:23:18.986781 16366 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:18.971848 19674 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:23:18.986222 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 184919us
I20260709 12:23:18.974807 19487 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:18.972870 19580 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:18.999758 17271 raft_consensus.cc:1275] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:23:19.000573 19700 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:19.020912 19239 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:18.998355 19266 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:19.022073 19266 raft_consensus.cc:697] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:19.015472 16226 catalog_manager.cc:5697] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 2 to 5, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 5 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:23:19.032284 16226 catalog_manager.cc:5697] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 5 to 10, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 10 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:18.998620 19391 raft_consensus.cc:697] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:23:19.043361 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 75085.4us
W20260709 12:23:19.048694 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:19.014302 19641 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:19.054508 19749 log.cc:927] Time spent T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:23:19.053494 16226 catalog_manager.cc:5697] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
W20260709 12:23:19.055733 19751 log.cc:927] Time spent T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:23:18.986308 19682 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:19.025213 19546 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:18.981696 19385 consensus_queue.cc:260] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:19.015511 17143 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:23:19.050086 19585 consensus_meta.cc:287] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.660s	user 0.009s	sys 0.005s
W20260709 12:23:19.088497 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:19.089376 19748 log.cc:927] Time spent T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.155s	user 0.001s	sys 0.001s
I20260709 12:23:19.093103 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:19.085521 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 94054.4us
I20260709 12:23:19.090757 16515 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:19.090844 16442 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:19.097843 16395 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:23:19.106884 16401 raft_consensus.cc:1240] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:19.023725 19266 consensus_queue.cc:237] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:19.112663 16367 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:19.078389 19580 raft_consensus.cc:697] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:19.078753 19417 consensus_queue.cc:1059] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 12:23:19.118160 16400 raft_consensus.cc:1240] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 7. Current term is 8. Ops: []
W20260709 12:23:19.125089 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 60182.2us
I20260709 12:23:19.128196 19391 consensus_queue.cc:237] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:19.129009 19762 log.cc:927] Time spent T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
W20260709 12:23:19.131695 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:19.074096 19641 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:19.087090 19585 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:19.140640 16468 raft_consensus.cc:1275] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:19.096972 19649 consensus_queue.cc:1059] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:23:19.149850 16367 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:19.141940 19546 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:19.151579 17143 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:19.122066 19642 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:19.108623 19740 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:19.151152 19756 raft_consensus.cc:3055] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:19.154214 19647 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:19.155117 19764 consensus_queue.cc:1059] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:23:19.113936 19624 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:19.162741 16395 raft_consensus.cc:1240] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:19.128213 19766 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:19.152611 19385 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:23:19.181084 16366 leader_election.cc:336] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.879s (SENT)
I20260709 12:23:19.179116 17144 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:19.184999 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 155252us
I20260709 12:23:19.159389 19546 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:23:19.189388 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 91035.4us
I20260709 12:23:19.170256 19647 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.198360 16366 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:19.166344 19763 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:23:19.202036 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 118537us
I20260709 12:23:19.163975 19756 raft_consensus.cc:740] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:19.228557 17264 raft_consensus.cc:2468] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:23:19.190834 19546 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:19.227563 19760 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:19.186419 19700 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:19.232198 16367 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:19.180764 19113 consensus_queue.cc:1048] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:23:19.209465 19647 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:19.198211 19243 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:23:19.243922 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 229331us
I20260709 12:23:19.239596 19700 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:19.233677 19760 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.257638 16515 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:23:19.258731 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 140191us
I20260709 12:23:19.189800 19766 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:19.243814 19764 raft_consensus.cc:3055] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:19.269651 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:19.249899 19243 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.273097 19527 consensus_queue.cc:1059] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:23:19.274367 19527 raft_consensus.cc:3055] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:19.274775 19527 raft_consensus.cc:740] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:19.258350 19700 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:19.239985 19580 consensus_queue.cc:237] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:19.309962 19768 log.cc:927] Time spent T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:23:19.310758 19740 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:19.312965 19769 log.cc:927] Time spent T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.122s	user 0.002s	sys 0.000s
I20260709 12:23:19.314862 19756 consensus_queue.cc:260] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:19.316295 19756 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:19.317370 17143 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:19.318630 19641 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:19.321050 19585 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:19.321945 19585 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:23:19.322770 19585 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:19.276001 19527 consensus_queue.cc:260] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:19.324146 19527 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:19.263615 19760 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:19.323156 19585 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.276912 19757 raft_consensus.cc:3055] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:19.271586 19764 raft_consensus.cc:740] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:19.271988 19682 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:19.278721 19243 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:19.339848 16400 raft_consensus.cc:1240] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 9. Current term is 10. Ops: []
W20260709 12:23:19.349687 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:19.357100 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 176046us
W20260709 12:23:19.334546 19384 consensus_meta.cc:287] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.525s	user 0.009s	sys 0.004s
W20260709 12:23:19.360736 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 163456us
W20260709 12:23:19.367655 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:19.346400 19585 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:19.341178 19641 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:19.347901 19417 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:23:19.352150 19695 consensus_meta.cc:287] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.622s	user 0.011s	sys 0.003s
I20260709 12:23:19.345800 19757 raft_consensus.cc:740] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:23:19.386586 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 156297us
W20260709 12:23:19.366401 19699 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.007s	sys 0.016s
W20260709 12:23:19.396399 19774 log.cc:927] Time spent T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:23:19.397944 17143 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:23:19.399252 19765 log.cc:927] Time spent T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.311s	user 0.002s	sys 0.000s
W20260709 12:23:19.400801 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 199693us
W20260709 12:23:19.401604 19758 log.cc:927] Time spent T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.344s	user 0.000s	sys 0.001s
I20260709 12:23:19.402258 16515 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:19.404553 19682 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:19.408543 19740 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.383646 19417 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.410152 16515 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:19.367023 19384 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:19.394677 19699 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.007s	sys 0.017s
W20260709 12:23:19.426842 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 161859us
I20260709 12:23:19.416507 19417 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:19.432559 19740 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:19.435873 19740 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:19.436414 19771 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:19.436195 19647 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:19.448858 19764 consensus_queue.cc:260] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:19.450955 19764 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:19.394945 19546 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:19.454980 19546 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:19.457317 16516 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:19.459026 16516 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:19.438687 19766 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:19.383441 19641 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:19.443861 19391 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:19.463863 19782 consensus_queue.cc:1048] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:23:19.464725 19759 log.cc:927] Time spent T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.351s	user 0.002s	sys 0.000s
I20260709 12:23:19.466704 19699 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:23:19.466864 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 62598.6us
I20260709 12:23:19.468575 19682 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:23:19.468962 19647 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:19.482972 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 97103us
I20260709 12:23:19.469175 19695 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:19.491633 19787 consensus_queue.cc:1059] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:19.492956 19787 raft_consensus.cc:3055] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:19.493294 19787 raft_consensus.cc:740] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:19.483650 19642 consensus_queue.cc:1048] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 12:23:19.494205 19787 consensus_queue.cc:260] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:19.495913 19787 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:19.496953 17143 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:19.501293 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:19.516445 19785 consensus_queue.cc:1059] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:19.529989 16515 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:19.518924 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:19.486086 19695 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.482719 19391 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:19.546049 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:19.518365 19730 raft_consensus.cc:3055] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:19.552373 16468 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 11: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
W20260709 12:23:19.574656 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:19.500346 19699 raft_consensus.cc:697] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 10 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:19.590312 16473 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:19.591076 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:19.518436 19786 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:19.521266 19384 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:19.520046 19641 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:19.520560 19417 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:19.605804 16475 raft_consensus.cc:2393] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 11: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:19.553709 19391 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:19.594362 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 95005.9us
I20260709 12:23:19.593571 19786 raft_consensus.cc:697] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:19.595661 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:19.597446 16441 connection.cc:256] RPC call timeout handler was delayed by -1.860972608999873s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1636s for a response
W20260709 12:23:19.610085 16441 connection.cc:256] RPC call timeout handler was delayed by -1.1553726090000964s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.242s for a response
I20260709 12:23:19.545868 19695 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:23:19.594452 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 64556.5us
W20260709 12:23:19.604970 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 104276us
I20260709 12:23:19.554415 19766 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:23:19.611249 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 79962.6us
I20260709 12:23:19.552650 19730 raft_consensus.cc:740] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:19.640612 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:19.632676 19624 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:19.640509 19578 raft_consensus.cc:3055] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:19.645067 16468 raft_consensus.cc:1240] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:19.645445 19578 raft_consensus.cc:740] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:19.641759 19546 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:19.647903 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:19.650031 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:19.653141 16515 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:19.649266 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:19.649559 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:19.660701 16472 raft_consensus.cc:2368] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 7. Current term is 8.
I20260709 12:23:19.661149 16475 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 8: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:19.661439 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:19.661779 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:19.662439 16476 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:19.653788 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:19.658973 16471 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:23:19.649715 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:19.650746 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:19.665231 16477 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:23:19.665416 16470 raft_consensus.cc:2393] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 9: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:19.666419 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:19.667147 19730 consensus_queue.cc:260] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:19.667788 16400 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:19.668653 19730 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:19.668879 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:19.656637 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:19.646258 19417 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:19.672621 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:19.662551 16472 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:23:19.673166 16395 raft_consensus.cc:1240] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:19.662500 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:19.676990 19779 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.677796 19779 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:19.680191 19779 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:19.681439 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 48576us
I20260709 12:23:19.679960 16473 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
I20260709 12:23:19.694002 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:19.486634 19757 consensus_queue.cc:260] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:19.698509 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:19.699375 16468 raft_consensus.cc:2368] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 13. Current term is 14.
W20260709 12:23:19.696012 19487 consensus_meta.cc:287] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.684s	user 0.012s	sys 0.002s
I20260709 12:23:19.706115 19777 consensus_queue.cc:1059] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:19.706728 19487 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:19.707545 19777 raft_consensus.cc:3055] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:19.707932 19777 raft_consensus.cc:740] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:19.708755 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:19.708956 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 55912.6us
I20260709 12:23:19.709236 19487 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:19.709609 16403 raft_consensus.cc:2393] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 6: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:19.709880 16515 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:19.710083 16397 raft_consensus.cc:1240] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:19.646678 19578 consensus_queue.cc:260] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:19.711388 16515 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:19.712091 19578 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:19.712615 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:19.713238 16395 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:23:19.709775 19243 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:19.715161 19760 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:19.715730 19243 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:19.718655 19384 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:19.719806 16515 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:19.687737 19736 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.762894 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:19.720603 19641 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:19.720901 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:19.764155 16398 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:19.721330 19699 consensus_queue.cc:237] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:19.725726 17143 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:19.726640 19700 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:19.727744 19757 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:19.652289 19546 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:19.729506 19791 log.cc:927] Time spent T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
W20260709 12:23:19.732436 19756 consensus_queue.cc:1175] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "610f4feda11942739236f2b121a4b9c1" responder_term: 9 status { last_received { term: 9 index: 5 } last_committed_idx: 4 last_received_current_leader { term: 9 index: 5 } }
I20260709 12:23:19.735733 16400 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
W20260709 12:23:19.785615 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 65919.4us
I20260709 12:23:19.786084 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:19.709272 19771 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:19.743613 19760 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:19.744626 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:19.744051 19585 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:19.747020 19385 consensus_meta.cc:287] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.556s	user 0.009s	sys 0.000s
I20260709 12:23:19.747424 16475 raft_consensus.cc:1240] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:19.748051 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:19.748546 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 49501.6us
I20260709 12:23:19.666370 19786 consensus_queue.cc:237] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:19.750859 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:19.798010 16469 raft_consensus.cc:2410] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:23:19.757500 16468 raft_consensus.cc:1275] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
W20260709 12:23:19.760171 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:19.764700 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:19.733927 19674 consensus_meta.cc:287] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.735s	user 0.011s	sys 0.000s
I20260709 12:23:19.740048 19764 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:19.775581 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:19.809581 16395 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:19.779927 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:19.780323 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:19.780995 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 55356.6us
I20260709 12:23:19.810236 16545 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:19.783532 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:19.681411 19779 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.813308 16399 raft_consensus.cc:1240] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:19.814272 19779 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:19.807869 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:19.815092 16475 raft_consensus.cc:1275] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:23:19.815459 16549 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:23:19.796972 19788 log.cc:927] Time spent T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a: Append to log took a long time: real 0.143s	user 0.000s	sys 0.003s
W20260709 12:23:19.790670 16442 connection.cc:256] RPC call timeout handler was delayed by -1.1148193959998025s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1636s for a response
I20260709 12:23:19.779059 19580 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:19.793278 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:19.822134 16546 raft_consensus.cc:2410] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:23:19.822607 19779 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:19.822813 19546 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:19.823033 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:19.824025 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:19.813566 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:19.770787 19736 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:19.795967 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:19.799896 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:19.803362 16398 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
I20260709 12:23:19.804651 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:19.806272 16402 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:19.795889 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:19.790664 19417 raft_consensus.cc:2804] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:23:19.808706 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:19.807238 16547 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:19.813236 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:19.828414 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:19.820161 16472 raft_consensus.cc:2393] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 13: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:19.829484 16551 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:19.767447 19641 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.829988 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:19.821542 16442 connection.cc:256] RPC call timeout handler was delayed by -1.0950193959997705s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1658s for a response
I20260709 12:23:19.832043 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:19.828974 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:19.833020 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:19.826996 16543 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:19.834447 16477 raft_consensus.cc:2393] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 11: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:19.835510 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:19.832432 19641 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:19.836141 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:19.836460 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:19.836584 19682 consensus_queue.cc:1048] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:19.836946 16470 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:19.837366 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:19.838501 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:19.819379 19766 consensus_queue.cc:1048] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 12:23:19.839449 16547 raft_consensus.cc:2393] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 8: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:19.840974 16542 raft_consensus.cc:2393] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 9: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:19.836263 17263 raft_consensus.cc:2393] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 8: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:19.841390 16548 raft_consensus.cc:2393] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 8: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:19.841688 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:19.843384 16397 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:23:19.809397 19764 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.833472 16550 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 6.
I20260709 12:23:19.865176 17270 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:19.819096 19385 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.847008 19700 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:19.847262 17269 raft_consensus.cc:1240] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260709 12:23:19.847589 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:19.786134 19777 consensus_queue.cc:260] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:19.848500 19417 raft_consensus.cc:697] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:19.849862 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:19.854517 16545 raft_consensus.cc:2393] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 8: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:19.854779 19752 consensus_queue.cc:1059] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:19.858196 16367 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:19.859895 19585 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:19.861074 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:19.864177 16516 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:19.880561 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:19.883412 16546 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:19.884275 17254 raft_consensus.cc:2393] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 11: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:19.861291 16551 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:19.865064 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:19.866348 17255 raft_consensus.cc:2393] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 9: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:19.868506 17266 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:23:19.868898 16395 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:23:19.871825 17144 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:19.866868 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:19.871567 16549 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:19.887549 16403 raft_consensus.cc:2393] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 8: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
W20260709 12:23:19.887998 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 43682.7us
I20260709 12:23:19.890179 16516 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:19.898940 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:19.825214 19760 consensus_queue.cc:1059] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 12:23:19.899305 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:19.884462 19641 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:19.915095 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:19.919112 19641 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:19.924031 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:19.873646 19764 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:19.926151 17144 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:19.923280 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:23:19.934926 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 65175.9us
I20260709 12:23:19.935858 17142 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:19.874504 19546 consensus_queue.cc:1048] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:23:19.942025 16546 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:23:19.943565 17255 raft_consensus.cc:2393] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 11: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:19.889261 19785 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.919970 19700 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:19.973071 16367 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:19.921149 19580 raft_consensus.cc:2804] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:19.922792 19682 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:19.920895 19787 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:19.860181 19391 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:19.982704 17255 raft_consensus.cc:1240] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:23:19.988055 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:19.883611 19695 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:19.876819 19385 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:20.004393 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 78393.9us
W20260709 12:23:20.009929 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 129419us
W20260709 12:23:20.016034 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 80251.9us
I20260709 12:23:20.017025 16516 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:19.985013 19580 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.018491 16473 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:20.017447 19760 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:19.989037 19756 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:19.980651 19700 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:19.974812 19674 consensus_queue.cc:1059] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 12:23:20.025316 17263 raft_consensus.cc:1240] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:23:20.002961 19777 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:20.002882 19585 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:20.024293 17255 raft_consensus.cc:1240] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:23:20.028323 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:20.037406 16473 raft_consensus.cc:1240] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:19.920742 19417 consensus_queue.cc:237] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:20.016230 19787 raft_consensus.cc:697] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:20.034838 17142 leader_election.cc:400] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 7. Current term is 8.
I20260709 12:23:20.034708 16442 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:20.038951 17142 leader_election.cc:403] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:20.037498 17266 raft_consensus.cc:2368] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11.
W20260709 12:23:20.027849 19761 log.cc:927] Time spent T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 12:23:20.032877 16402 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:23:20.025825 19580 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:19.981339 19785 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:20.031677 19700 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:20.073849 17271 raft_consensus.cc:2410] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 12:23:20.047852 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 78438us
I20260709 12:23:20.047317 16395 raft_consensus.cc:1275] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:23:20.051990 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:20.053160 19787 consensus_queue.cc:237] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:20.054358 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:20.055136 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:20.055821 19385 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:20.058794 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:20.092152 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:20.092978 17255 raft_consensus.cc:2368] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11.
I20260709 12:23:20.093983 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:20.062839 19740 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.063581 17142 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:20.064278 19495 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 7. Current term is 8.
I20260709 12:23:20.069995 19733 raft_consensus.cc:3055] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:20.070387 19391 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:20.096721 19740 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:20.097326 19733 raft_consensus.cc:740] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:23:20.098577 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:20.098266 19733 consensus_queue.cc:260] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:20.099259 19740 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:20.099368 19804 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:20.099718 19733 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:20.100121 19642 raft_consensus.cc:2764] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:23:20.100106 19804 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:20.100867 19736 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:20.101740 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:20.102380 19804 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:20.073470 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:20.053071 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:20.104364 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:23:20.105583 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:20.106078 19806 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:20.106284 19495 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:20.106639 19806 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:20.106849 19495 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.107188 19806 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:20.107338 19495 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:20.049547 16516 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:20.109848 19779 raft_consensus.cc:3055] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:20.086129 16403 raft_consensus.cc:1275] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:23:20.090384 16515 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:20.110332 19779 raft_consensus.cc:740] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
W20260709 12:23:20.110041 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 60612.2us
I20260709 12:23:20.111454 16515 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:20.111310 19779 consensus_queue.cc:260] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:20.112646 16367 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:20.113101 16403 raft_consensus.cc:1275] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:23:20.036885 19585 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.113530 19750 consensus_queue.cc:1048] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:20.113906 19585 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:20.113977 16367 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:20.072448 19695 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:20.115384 19239 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:20.115567 19546 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:20.116040 19546 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.044065 19760 raft_consensus.cc:697] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:20.116250 19239 consensus_queue.cc:1048] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:20.117142 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:20.117231 19760 consensus_queue.cc:237] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:20.117954 16399 raft_consensus.cc:2393] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 14: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:20.115111 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:20.117950 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:20.120348 19546 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:20.120720 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:20.121454 16394 raft_consensus.cc:2393] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 11: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:20.122253 19785 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:20.122450 19633 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:20.123034 19633 raft_consensus.cc:697] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:20.123266 17255 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:20.123878 19633 consensus_queue.cc:237] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:20.128654 16470 raft_consensus.cc:2393] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 7: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
W20260709 12:23:20.129750 16366 leader_election.cc:336] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.511s (SENT)
I20260709 12:23:20.130182 16366 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:20.125101 16515 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
W20260709 12:23:20.131904 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 46215.2us
W20260709 12:23:20.133018 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 69724us
I20260709 12:23:20.133507 17258 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:20.141530 17143 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:20.142661 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:20.145049 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:20.146102 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:20.147094 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:20.147154 17143 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:20.153048 17142 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:20.156663 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:20.143751 16394 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:23:20.153647 19487 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:20.161293 19779 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:23:20.165460 16366 leader_election.cc:336] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.552s (SENT)
I20260709 12:23:20.169162 19792 raft_consensus.cc:3055] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:20.169848 19792 raft_consensus.cc:740] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:20.155431 19750 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:20.170604 19750 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.170976 19750 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:20.171027 19792 consensus_queue.cc:260] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:20.153816 19695 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.174115 19695 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:20.175452 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:20.177783 17270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45228 (request call id 924) took 1536 ms. Trace:
I20260709 12:23:20.178221 17270 rpcz_store.cc:276] 0709 12:23:18.646896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:18.650133 (+  3237us) service_pool.cc:224] Handling call
0709 12:23:20.177602 (+1527469us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54003f02d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:20.177703 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2563616000}
I20260709 12:23:20.179737 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:20.180538 16545 raft_consensus.cc:2393] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 9: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:20.181403 17271 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:20.133934 19391 raft_consensus.cc:2804] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:20.153359 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:20.194123 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:20.195143 16477 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:20.222203 16398 raft_consensus.cc:1240] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:20.135798 19699 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:23:20.226765 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 44111.9us
I20260709 12:23:20.220988 17144 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:20.234366 16366 leader_election.cc:336] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.696s (SENT)
I20260709 12:23:20.234710 16366 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:20.235622 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:20.236815 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 89878.7us
I20260709 12:23:20.237658 17143 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:20.239460 17143 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:20.242184 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:20.248526 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 95534.9us
I20260709 12:23:20.226346 19391 raft_consensus.cc:697] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:23:20.306491 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 72147.9us
I20260709 12:23:20.300823 16396 raft_consensus.cc:1240] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:23:20.149628 19113 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:20.287133 19766 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:20.313166 19793 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:20.197598 19266 raft_consensus.cc:2804] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:20.298780 17142 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:20.281805 19730 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:20.319717 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:20.287312 19243 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:20.282202 19756 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:20.191196 19805 consensus_queue.cc:1048] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.146s
I20260709 12:23:20.287623 19649 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:20.365504 16442 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:20.323165 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:20.337687 19792 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:23:20.329303 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 204272us
I20260709 12:23:20.332993 16477 raft_consensus.cc:1275] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:23:20.299365 19239 raft_consensus.cc:2749] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:20.240199 19695 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:20.344972 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:20.346117 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:20.376133 16547 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:20.349345 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:20.351289 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:20.377394 17255 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:20.350979 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:20.353575 17270 raft_consensus.cc:1275] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:20.352974 16394 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:23:20.361517 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:20.354544 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:20.323482 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 169153us
I20260709 12:23:20.368782 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:20.355783 19730 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.372473 16396 raft_consensus.cc:1240] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:20.371511 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:20.352408 19800 log.cc:927] Time spent T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:23:20.382575 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:20.380712 16470 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
I20260709 12:23:20.396624 16542 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:23:20.402467 16366 leader_election.cc:336] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 2.188s (SENT)
W20260709 12:23:20.375216 19797 log.cc:927] Time spent T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.288s	user 0.002s	sys 0.000s
W20260709 12:23:20.395062 19578 consensus_meta.cc:287] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.682s	user 0.007s	sys 0.005s
I20260709 12:23:20.351765 19113 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:20.350042 19766 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:20.420259 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:20.422842 16543 raft_consensus.cc:2410] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:23:20.425565 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:20.433310 16477 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:20.401698 19730 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:23:20.428663 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 72581.9us
I20260709 12:23:20.428984 16367 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:20.398846 19649 raft_consensus.cc:697] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:20.375041 19487 consensus_queue.cc:1048] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:23:20.406412 19815 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:20.408648 19385 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:23:20.441954 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 71652.1us
I20260709 12:23:20.415987 19239 consensus_queue.cc:1048] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:23:20.412173 19546 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:20.473726 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:20.449657 19752 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:20.478986 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:23:20.479799 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 122956us
I20260709 12:23:20.481222 16548 raft_consensus.cc:2393] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 9: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
W20260709 12:23:20.482266 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 62443.7us
I20260709 12:23:20.412170 19391 consensus_queue.cc:237] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:20.527467 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:20.528156 16476 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:20.529241 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:20.484406 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:20.531127 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:20.486972 19808 log.cc:927] Time spent T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
W20260709 12:23:20.489135 19764 consensus_meta.cc:287] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.518s	user 0.008s	sys 0.002s
I20260709 12:23:20.488837 19649 consensus_queue.cc:237] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:20.490106 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:20.546581 16542 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:20.498718 19546 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:20.501788 19578 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:20.443022 19585 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:20.504913 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:20.482513 19752 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:20.455431 19384 consensus_queue.cc:1048] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:23:20.452069 19806 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:20.442668 19812 consensus_queue.cc:1048] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 12:23:20.517570 16367 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:20.519676 16366 leader_election.cc:336] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.798s (SENT)
W20260709 12:23:20.557597 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 258912us
I20260709 12:23:20.520725 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:20.522387 19766 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:20.524390 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:20.524896 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:20.525322 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:23:20.525595 19810 log.cc:927] Time spent T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.274s	user 0.000s	sys 0.001s
I20260709 12:23:20.526288 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:20.526676 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:20.536023 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 59706.4us
I20260709 12:23:20.561687 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:20.547266 16547 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:20.561969 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:20.548065 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:20.475236 19385 raft_consensus.cc:697] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:20.552610 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:20.562903 16470 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:20.508888 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:20.549783 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:20.555996 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:20.543321 17144 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:20.591116 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:23:20.581712 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 64254us
I20260709 12:23:20.582361 16400 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 9: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:20.548410 19764 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:20.599500 19495 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:20.558331 19752 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:20.599515 16367 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:20.601279 19777 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:20.602046 17269 raft_consensus.cc:1240] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:23:20.569865 16475 raft_consensus.cc:2368] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 13. Current term is 14.
I20260709 12:23:20.564743 16542 raft_consensus.cc:2368] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11.
I20260709 12:23:20.603580 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:20.555739 19578 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:20.593418 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:20.602046 16367 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:20.592976 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:23:20.582119 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 62460.4us
I20260709 12:23:20.605592 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:20.606189 19777 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:23:20.606345 16366 leader_election.cc:336] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.858s (SENT)
I20260709 12:23:20.606611 19777 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:20.606931 16366 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:20.607390 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:20.604094 19495 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:20.570547 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:20.607177 19777 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:20.609198 17263 raft_consensus.cc:2393] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 9: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:20.565683 19580 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:20.610951 16545 raft_consensus.cc:2393] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 9: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:20.571054 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:20.611799 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:20.573465 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:20.613467 19580 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:20.583094 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:20.564144 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:20.614192 17263 raft_consensus.cc:2393] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 10: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:20.606544 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:20.598901 17255 raft_consensus.cc:2393] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 8: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
W20260709 12:23:20.618866 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 75672us
I20260709 12:23:20.620498 17264 raft_consensus.cc:1275] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:23:20.622519 16549 raft_consensus.cc:1240] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:20.614012 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:20.596215 19239 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:20.626335 19239 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:20.626518 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:20.626718 19239 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:20.604974 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:20.657869 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:20.687167 17263 raft_consensus.cc:1275] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 12:23:20.687228 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 73898.7us
I20260709 12:23:20.689600 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:20.700110 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 93701.1us
W20260709 12:23:20.700803 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 120034us
I20260709 12:23:20.632575 19750 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:20.613783 19578 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:20.711717 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 96891.6us
I20260709 12:23:20.623989 19546 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:23:20.719131 19757 consensus_meta.cc:287] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.935s	user 0.011s	sys 0.000s
I20260709 12:23:20.702260 19495 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:23:20.723603 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:20.735821 16515 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:20.748545 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 132776us
I20260709 12:23:20.748409 16515 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:20.751523 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:20.755033 16367 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:20.756474 16477 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:20.760800 17266 raft_consensus.cc:1240] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:20.712101 19266 raft_consensus.cc:697] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:20.779918 16515 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:23:20.753659 19779 consensus_meta.cc:287] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.592s	user 0.010s	sys 0.002s
I20260709 12:23:20.716004 19819 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:20.790500 17255 raft_consensus.cc:1275] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:23:20.792691 16515 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:20.794456 16515 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:20.801251 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 46344us
I20260709 12:23:20.802523 16367 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:20.763278 19384 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:20.813091 16367 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:20.829722 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 63811.2us
I20260709 12:23:20.774886 19113 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:20.832079 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 90913.9us
I20260709 12:23:20.787210 19578 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:23:20.801831 19803 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.007s	sys 0.010s
W20260709 12:23:20.803725 19700 consensus_meta.cc:287] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.679s	user 0.002s	sys 0.010s
I20260709 12:23:20.782744 19833 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:23:20.831655 17142 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.778s (SENT) [suppressed 33 similar messages]
I20260709 12:23:20.803448 19804 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:20.770416 19495 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:20.801891 19641 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:20.795444 19819 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.852748 16367 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:20.823031 19812 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:23:20.826846 19779 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:20.830668 19546 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:23:20.847566 19803 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.008s	sys 0.016s
I20260709 12:23:20.781924 19385 consensus_queue.cc:237] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:20.903607 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 180740us
I20260709 12:23:20.914849 16469 raft_consensus.cc:1275] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:23:20.915791 19826 log.cc:927] Time spent T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:23:20.916643 19833 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:23:20.915028 19828 log.cc:927] Time spent T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:23:20.917243 19833 raft_consensus.cc:697] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 8 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:20.918299 19833 consensus_queue.cc:237] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:20.919595 16397 raft_consensus.cc:1275] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:23:20.922086 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 204183us
I20260709 12:23:20.922260 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:20.898085 19792 consensus_meta.cc:287] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.526s	user 0.007s	sys 0.004s
W20260709 12:23:20.909787 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 55753.5us
I20260709 12:23:20.920809 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:20.929881 17254 raft_consensus.cc:2393] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 9: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:20.864248 19819 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:20.854277 19700 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:20.870368 19730 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:20.938232 16442 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:20.963292 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:20.964172 17254 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:20.926091 19495 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:20.931948 19585 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:20.968484 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 115840us
I20260709 12:23:20.974292 16367 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:20.985795 19682 consensus_meta.cc:287] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.996s	user 0.015s	sys 0.000s
I20260709 12:23:20.986619 19682 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:20.986769 19700 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:20.967864 19546 raft_consensus.cc:697] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:20.988408 19546 consensus_queue.cc:237] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:20.989073 19841 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.989706 19841 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:20.938215 19580 consensus_queue.cc:1048] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
W20260709 12:23:20.997642 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:21.002887 16474 raft_consensus.cc:3055] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:21.003412 16474 raft_consensus.cc:740] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:21.004419 16474 consensus_queue.cc:260] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:21.005532 16474 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:21.009094 19842 consensus_queue.cc:1048] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:21.011082 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:21.012068 16394 raft_consensus.cc:2393] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 9: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:20.974285 19417 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.908806 19266 consensus_queue.cc:237] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:21.023034 17261 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:21.013631 16397 raft_consensus.cc:1275] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:23:21.028002 17271 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:21.025378 16397 pending_rounds.cc:85] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 4
I20260709 12:23:21.032042 19750 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:21.035833 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 101426us
I20260709 12:23:21.046334 16551 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:23:21.054998 17254 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:23:21.000353 19787 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:21.032349 16397 pending_rounds.cc:107] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 6.5
W20260709 12:23:21.033792 19733 consensus_meta.cc:287] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.934s	user 0.014s	sys 0.000s
I20260709 12:23:21.058974 16397 raft_consensus.cc:2889] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:20.996917 19757 consensus_queue.cc:1048] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:23:21.023727 19391 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.035295 19849 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:21.074297 16401 raft_consensus.cc:1240] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:21.057080 19822 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:23:21.085431 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 161256us
I20260709 12:23:21.058730 19841 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:21.024297 19417 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:21.100723 19847 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:23:21.102090 19845 log.cc:927] Time spent T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:23:21.104501 19417 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:21.110002 19752 consensus_meta.cc:287] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.509s	user 0.010s	sys 0.000s
I20260709 12:23:21.110584 19752 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:21.114733 16399 raft_consensus.cc:1240] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:21.115897 17258 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
I20260709 12:23:21.110041  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:21.065057 19787 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:21.085100 19849 raft_consensus.cc:697] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:21.130301 16474 raft_consensus.cc:1275] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 12:23:21.130945 16474 pending_rounds.cc:85] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 5
I20260709 12:23:21.131256 16474 pending_rounds.cc:107] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 12:23:21.131582 16474 raft_consensus.cc:2889] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:23:21.133831 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 159639us
I20260709 12:23:21.134938 16367 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:21.136816 16367 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:21.076663 19391 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:21.144083 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:21.139443 19239 consensus_meta.cc:287] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.512s	user 0.011s	sys 0.000s
I20260709 12:23:21.094893 19822 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:21.114674 19642 consensus_queue.cc:1059] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
W20260709 12:23:21.148656 19777 consensus_meta.cc:287] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.540s	user 0.011s	sys 0.000s
I20260709 12:23:21.158802 19777 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:21.161290 19777 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:21.162830 16403 raft_consensus.cc:1240] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:21.140229 19580 consensus_queue.cc:1048] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 12:23:21.164122 19787 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:21.150141 19239 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:21.151677 19851 consensus_queue.cc:1048] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:21.148898 19730 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:21.167537 19239 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:21.152061 19861 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:21.147322 19854 consensus_queue.cc:1059] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:23:21.142447 19546 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:21.154400 19822 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:21.156960 19391 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:21.156651 19849 consensus_queue.cc:237] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:21.192404 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:21.199281 16548 raft_consensus.cc:2393] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 9: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:21.194832 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:21.196419 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:21.200596 16397 raft_consensus.cc:2393] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 9: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:21.202898 16546 raft_consensus.cc:1240] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:23:21.203919 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:21.204676 16400 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:23:21.206751 16400 raft_consensus.cc:1240] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:21.208563 16367 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:21.210956 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:21.212585 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:23:21.212970 19809 log.cc:927] Time spent T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.906s	user 0.004s	sys 0.009s
I20260709 12:23:21.213994 19752 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:21.214342 16472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1199) took 1369 ms. Trace:
I20260709 12:23:21.214890 16472 rpcz_store.cc:276] 0709 12:23:19.844751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:19.844900 (+   149us) service_pool.cc:224] Handling call
0709 12:23:19.872872 (+ 27972us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:19.898314 (+ 25442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:19.898369 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:19.960189 (+ 61820us) log.cc:844] Serialized 26 byte log entry
0709 12:23:20.093731 (+133542us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:23:20.093772 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:20.093841 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:21.212113 (+1118272us) spinlock_profiling.cc:243] Waited 8.32 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:23:21.213957 (+  1844us) raft_consensus.cc:1685] finished
0709 12:23:21.213987 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:21.214243 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":43239,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":13610,"spinlock_wait_cycles":19144064,"thread_start_us":66017,"threads_started":1,"wal-append.queue_time_us":206774}
I20260709 12:23:21.232905 16550 raft_consensus.cc:1240] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:23:21.183038 19861 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:21.241591 17255 raft_consensus.cc:1275] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:23:21.241488 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:21.243229 17270 raft_consensus.cc:1275] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:23:21.210023 16367 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:21.247646 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:21.247954 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:21.248718 16472 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 11: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:21.233901 19239 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:21.251009 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:21.231043 19546 raft_consensus.cc:697] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:21.253489 16477 raft_consensus.cc:1275] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:23:21.255251 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:21.256090 16477 raft_consensus.cc:2393] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 11: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:21.256238 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:21.256839 17255 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:21.251986 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:21.248631 16474 raft_consensus.cc:2376] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13. Re-sending same reply.
I20260709 12:23:21.257606 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:21.258834 19755 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.259449 19755 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:21.261829 19837 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.262446 19837 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:21.265057 16477 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:23:21.266449 16468 raft_consensus.cc:2393] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 11: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:21.267270 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:21.247104 19730 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.271236 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:21.246732 19851 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:21.269894 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:21.245568 19861 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:21.273360 19487 consensus_queue.cc:1059] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:21.273963 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:21.250917 19865 raft_consensus.cc:3055] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:21.288727 19865 raft_consensus.cc:740] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:21.293864 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:21.300400 16477 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:23:21.278213 16367 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:21.278051 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:21.274884 16550 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:23:21.302407 16550 raft_consensus.cc:606] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:21.302827 16550 raft_consensus.cc:614] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:21.264681 19239 raft_consensus.cc:697] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:21.303213 19865 consensus_queue.cc:260] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:21.304702 19865 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:21.304401 19239 consensus_queue.cc:237] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:21.276793 19730 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:21.307371 19546 consensus_queue.cc:237] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:21.312217 16468 raft_consensus.cc:2393] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 11: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:21.318274 17270 raft_consensus.cc:1240] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:23:21.322641 17255 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:21.276046 19819 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:21.325789 17142 leader_election.cc:336] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.525s (SENT)
I20260709 12:23:21.326671 17142 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:21.287870 19851 raft_consensus.cc:697] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 8 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
W20260709 12:23:21.326344 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 181460us
I20260709 12:23:21.316577 19833 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:21.308037 19487 raft_consensus.cc:3055] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:23:21.353641 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 84482.4us
I20260709 12:23:21.346604 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:21.348107 19733 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:21.346539 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:21.357501 16477 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:23:21.359241 17270 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2"
dest_uuid: "ef57beab0e1d40d585983873c55107e8"
mode: GRACEFUL
new_leader_uuid: "f99066d13e524238aec6aa2024a8bd50"
 from {username='slave'} at 127.0.0.1:45120
I20260709 12:23:21.363221 16400 raft_consensus.cc:1240] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:21.360020 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:21.364053 16474 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:21.363289 17270 raft_consensus.cc:606] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Received request to transfer leadership to TS f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:21.365329 17270 raft_consensus.cc:614] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:21.341693 19787 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:21.368947 16397 raft_consensus.cc:1275] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:23:21.352865 19837 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:21.370465 17142 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:21.353868 19487 raft_consensus.cc:740] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:21.373363 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:21.376015 16474 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:23:21.362262 19266 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:21.379424 16396 raft_consensus.cc:2393] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 7: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:21.379671 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:21.385186 17269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 912) took 1033 ms. Trace:
W20260709 12:23:21.434314 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:21.442961 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 72593.1us
W20260709 12:23:21.453297 19871 log.cc:927] Time spent T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:23:21.467118 19849 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:21.479971 16401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 876) took 1114 ms. Trace:
I20260709 12:23:21.455245 19806 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:21.456540 19832 log.cc:927] Time spent T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
W20260709 12:23:21.456549 17258 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45230 (request call id 906) took 1320 ms (client timeout 1629 ms). Trace:
I20260709 12:23:21.417363 19851 consensus_queue.cc:237] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:21.380573 19820 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:21.458482 16550 raft_consensus.cc:1275] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:23:21.461678 17266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 947) took 1417 ms. Trace:
I20260709 12:23:21.479358 17269 rpcz_store.cc:276] 0709 12:23:20.352046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:20.357569 (+  5523us) service_pool.cc:224] Handling call
0709 12:23:20.357975 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:20.602810 (+244835us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:21.384958 (+782148us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b54005f7b54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:21.385133 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":560717184}
I20260709 12:23:21.383044 19266 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:21.376214 19787 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:21.485119 17258 rpcz_store.cc:269] 0709 12:23:20.135849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:20.153019 (+ 17170us) service_pool.cc:224] Handling call
0709 12:23:21.456322 (+1303303us) spinlock_profiling.cc:243] Waited 875 ms on lock 0x7b5400a74c54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:21.456480 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2012544896}
I20260709 12:23:21.468824 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:21.490893 16400 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:21.473295 16544 raft_consensus.cc:1240] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:21.464084 16367 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:21.475239 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:21.493775 16402 raft_consensus.cc:1275] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:21.373972 19733 consensus_queue.cc:1048] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 12:23:21.492769 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:21.385154 16441 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:21.480266 16545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37226 (request call id 1290) took 1130 ms. Trace:
I20260709 12:23:21.565987 16545 rpcz_store.cc:276] 0709 12:23:20.343246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:20.352247 (+  9001us) service_pool.cc:224] Handling call
0709 12:23:21.472935 (+1120688us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b5400aa69d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:21.473125 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":492704128}
I20260709 12:23:21.481098 16401 rpcz_store.cc:276] 0709 12:23:20.356699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:20.356834 (+   135us) service_pool.cc:224] Handling call
0709 12:23:21.072382 (+715548us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:21.075797 (+  3415us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:21.470662 (+394865us) spinlock_profiling.cc:243] Waited 696 ms on lock 0x7b5400ad34d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:21.470836 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1600932736}
I20260709 12:23:21.480736 17143 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:21.463287 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:21.567999 16396 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:21.488629 16548 raft_consensus.cc:1275] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:23:21.465263 17261 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45230 (request call id 903) took 1409 ms. Trace:
I20260709 12:23:21.486577 17266 rpcz_store.cc:276] 0709 12:23:20.044618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:20.086521 (+ 41903us) service_pool.cc:224] Handling call
0709 12:23:20.086937 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:20.761597 (+674660us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:21.461399 (+699802us) spinlock_profiling.cc:243] Waited 674 ms on lock 0x7b5400be2dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:21.461591 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1549141248}
W20260709 12:23:21.544790 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 64944.9us
I20260709 12:23:21.571030 17142 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:21.552650 19730 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:21.554592 19266 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:21.493976 16367 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:21.572816 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 78936.2us
I20260709 12:23:21.556476 16398 raft_consensus.cc:2393] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 10: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:21.549007 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:21.575919 16542 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:21.562462 16397 raft_consensus.cc:1275] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:23:21.562755 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 201256us
W20260709 12:23:21.546891 19872 log.cc:927] Time spent T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:23:21.564877 16543 raft_consensus.cc:1275] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:21.569418 17261 rpcz_store.cc:276] 0709 12:23:20.056311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:20.104072 (+ 47761us) service_pool.cc:224] Handling call
0709 12:23:21.465064 (+1360992us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b5400665dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:21.465174 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1481767808}
W20260709 12:23:21.572953 19779 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.029s	sys 0.020s
W20260709 12:23:21.582664 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 197558us
I20260709 12:23:21.585489 16442 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:21.599138 16442 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:21.600245 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:21.601100 16402 raft_consensus.cc:2393] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 9: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:21.584857 19585 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:21.611688 19868 consensus_queue.cc:1059] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
W20260709 12:23:21.584869 19779 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.029s	sys 0.020s
I20260709 12:23:21.614655 19779 raft_consensus.cc:697] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:21.615515 19779 consensus_queue.cc:237] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:21.597257 19266 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.559082 19833 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:21.555701 19239 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.631703 16441 leader_election.cc:400] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11.
I20260709 12:23:21.632093 16441 leader_election.cc:403] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:21.633337 16400 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:23:21.597385 19806 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.629313 19391 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.629339 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:21.623608 16542 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:23:21.488489 19849 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:21.646065 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 47031.2us
I20260709 12:23:21.647272 16402 raft_consensus.cc:1275] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:23:21.647927 19849 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:21.648553 19852 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:21.651140 16543 raft_consensus.cc:1275] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:21.584882 19853 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:21.641161 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:21.652922 16545 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:23:21.653053 19853 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11.
I20260709 12:23:21.637771 19239 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:21.655431 19239 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:21.656581 16545 raft_consensus.cc:1240] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:21.657153 16543 raft_consensus.cc:1240] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:21.658483 19391 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:21.658807 19819 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:21.644958 19873 log.cc:927] Time spent T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:23:21.660923 19391 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:21.645339 19806 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:21.662643 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:21.663218 19838 log.cc:927] Time spent T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.730s	user 0.002s	sys 0.000s
I20260709 12:23:21.664436 19806 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:21.674970 19764 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:21.677631 19787 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:21.680894 16551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37242 (request call id 1127) took 1197 ms. Trace:
I20260709 12:23:21.682346 16547 raft_consensus.cc:1240] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13. Ops: [10.6-12.7]
I20260709 12:23:21.681420 16551 rpcz_store.cc:276] 0709 12:23:20.483764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:20.484116 (+   352us) service_pool.cc:224] Handling call
0709 12:23:21.680702 (+1196586us) spinlock_profiling.cc:243] Waited 462 ms on lock 0x7b5400a4f954. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:21.680828 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1063698304}
I20260709 12:23:21.616968 19266 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:21.691090 16551 raft_consensus.cc:1240] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:21.630897 19858 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:21.697916 16551 raft_consensus.cc:1240] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 5. Current term is 6. Ops: []
I20260709 12:23:21.699919 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:21.648937 19385 consensus_queue.cc:1059] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 12:23:21.711375 16543 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:21.696077 19858 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:21.715758 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:21.716638 16397 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 9: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
W20260709 12:23:21.717482 16441 connection.cc:256] RPC call timeout handler was delayed by -1.3425317619999078s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17830000000000001s for a response
W20260709 12:23:21.717921 16441 connection.cc:256] RPC call timeout handler was delayed by -1.327231761999883s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18000000000000002s for a response
W20260709 12:23:21.718246 16441 connection.cc:256] RPC call timeout handler was delayed by -1.0932317619999594s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20600000000000002s for a response
I20260709 12:23:21.573841 19487 consensus_queue.cc:260] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:21.634030 19863 consensus_queue.cc:1059] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.081s
I20260709 12:23:21.736047 16545 raft_consensus.cc:1240] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:21.644047 19861 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:21.720839 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:21.738197 16402 raft_consensus.cc:2393] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 10: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:21.725786 16397 raft_consensus.cc:1240] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:21.727627 16398 raft_consensus.cc:1240] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:21.715144 19239 consensus_queue.cc:1048] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
W20260709 12:23:21.735450 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:21.718891 16441 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:21.692293 19837 consensus_queue.cc:1059] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:23:21.719211 19868 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.720072 19858 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:21.749624 16441 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:21.750814 16403 raft_consensus.cc:1240] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:23:21.745054 19863 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:21.720271 19585 raft_consensus.cc:697] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:21.760725 16402 raft_consensus.cc:1240] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:21.747193 19868 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:21.771610 19868 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:21.773550 19266 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:21.774230 19266 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.774871 19806 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.775279 19755 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:21.775635 19806 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:21.778682 19806 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:21.774748 19266 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:21.735963 19546 consensus_queue.cc:1175] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ef57beab0e1d40d585983873c55107e8" responder_term: 12 status { last_received { term: 9 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: [11.7-11.7]" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:23:21.794647 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:21.805160 16366 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:21.811110 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:21.859942 16396 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:21.813515 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:21.861163 16399 raft_consensus.cc:1240] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:21.861919 16402 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:23:21.761328 19771 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.863914 16394 raft_consensus.cc:2393] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 9: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:21.865943 16441 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:21.787137 19266 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:21.787038 19852 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:21.799304 19835 raft_consensus.cc:3055] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:21.802345 19861 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:21.858646 19777 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.888098 19851 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:21.795225 19779 consensus_queue.cc:1059] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 12:23:21.799603 19752 consensus_queue.cc:1048] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 12:23:21.868654 19863 consensus_queue.cc:1059] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 12:23:21.795902 19880 consensus_queue.cc:1059] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.093s
I20260709 12:23:21.876794 19239 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:21.878597 19835 raft_consensus.cc:740] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:21.889189 19868 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:21.866498 19771 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:21.900074 16441 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:21.803252 19585 consensus_queue.cc:237] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:21.918628 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:21.919931 16441 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:21.896417 19777 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:21.920710 19854 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:21.921283 19854 raft_consensus.cc:697] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:21.922115 19777 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:21.922348 19842 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:21.902529 19239 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:21.925325 19239 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:21.928610 19771 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:21.930357 19835 consensus_queue.cc:260] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:21.931779 19835 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:21.932035 19642 raft_consensus.cc:3055] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:21.932514 19642 raft_consensus.cc:740] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:21.933740 19642 consensus_queue.cc:260] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:21.935352 19642 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:21.922159 19854 consensus_queue.cc:237] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:21.938467 19876 log.cc:927] Time spent T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.290s	user 0.000s	sys 0.000s
I20260709 12:23:21.939514 19867 raft_consensus.cc:3055] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:21.939996 19867 raft_consensus.cc:740] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:21.940290 19863 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:21.940848 19868 consensus_queue.cc:1059] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:21.952994 16441 leader_election.cc:400] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11.
I20260709 12:23:21.953395 16441 leader_election.cc:403] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:23:21.948837 16516 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:21.958858 19699 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:21.959470 16516 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:21.960669 19764 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:21.962926 19764 consensus_queue.cc:1048] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:21.966262 19764 consensus_queue.cc:1048] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:23:21.982463 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 177388us
I20260709 12:23:21.985030 16441 leader_election.cc:400] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11.
I20260709 12:23:21.984630 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:22.001789 16550 raft_consensus.cc:1275] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:23:22.002775 16441 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:22.003489 16545 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
I20260709 12:23:21.981386 19847 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11.
I20260709 12:23:22.000828 19863 consensus_queue.cc:1059] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:23:22.009534 16366 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:22.010871 16366 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:22.013875 16441 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:22.015563 19853 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:22.016489 19853 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:22.019703 19847 consensus_queue.cc:1059] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:22.020843 19863 raft_consensus.cc:3055] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:22.021327 19863 raft_consensus.cc:740] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:22.022336 19863 consensus_queue.cc:260] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:22.023524 19863 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:22.024507 19842 raft_consensus.cc:3055] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:21.983143 19865 raft_consensus.cc:3055] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:22.024938 19842 raft_consensus.cc:740] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:22.001294 19867 consensus_queue.cc:260] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:22.025992 19858 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:22.026685 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:22.027918 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:22.028091 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:22.029387 16470 raft_consensus.cc:3055] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:22.030903 16542 raft_consensus.cc:1240] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:22.036722 16367 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:22.032267 16551 raft_consensus.cc:1240] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:23:22.032444 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:22.040056 16548 raft_consensus.cc:2393] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 7: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:22.032624 19585 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:23:22.033458 19806 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.029381 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:22.040863 16472 raft_consensus.cc:2393] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 11: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:22.033303 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:21.970175 19771 consensus_queue.cc:1048] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:23:22.033843 19880 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:22.034854 19865 raft_consensus.cc:740] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:22.038038 16470 raft_consensus.cc:740] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:22.045311 16366 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:22.053452 16470 consensus_queue.cc:260] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:22.054478 16470 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:22.053231 19806 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:22.057636 16546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1125) took 1581 ms. Trace:
I20260709 12:23:22.058071 16546 rpcz_store.cc:276] 0709 12:23:20.476710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:20.483903 (+  7193us) service_pool.cc:224] Handling call
0709 12:23:20.503871 (+ 19968us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:21.203610 (+699739us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:22.057362 (+853752us) spinlock_profiling.cc:243] Waited 699 ms on lock 0x7b540037f354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:22.057570 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1606800256}
I20260709 12:23:22.046665 19892 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:22.059119 19892 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:22.059463 19892 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:22.062765 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:22.063624 16471 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:22.064919 19842 consensus_queue.cc:260] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:22.066102 19842 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:22.066787 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:22.069181 19867 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:23:22.070977 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 604068us
I20260709 12:23:22.071313 19821 consensus_queue.cc:1048] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:22.073167 19821 consensus_queue.cc:1059] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:22.073410 17144 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:22.080497 16545 raft_consensus.cc:1240] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8. Ops: [5.5-7.6]
I20260709 12:23:22.083711 19820 raft_consensus.cc:697] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:23:22.084236 19889 log.cc:927] Time spent T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 12:23:22.085318 19385 consensus_queue.cc:1059] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:22.085215 19820 consensus_queue.cc:237] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:22.097052 16470 raft_consensus.cc:1275] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 12:23:22.108660 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 627974us
I20260709 12:23:22.120388 16516 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:22.087841 19880 raft_consensus.cc:2749] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:22.122303 16471 raft_consensus.cc:1275] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:23:22.050714 19777 consensus_queue.cc:1048] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 12:23:22.094097 19771 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.094038 19756 raft_consensus.cc:3055] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:22.094116 19806 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:22.127472 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:22.124866 16516 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:22.127388 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:22.131379 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:22.134311 16471 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:22.136129 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:22.120194 19757 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:22.139117 19585 consensus_queue.cc:1059] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:22.139611 19757 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:22.140187 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:22.126698 19865 consensus_queue.cc:260] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:22.141527 19865 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:22.128247 19764 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.140069 19585 raft_consensus.cc:3055] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:22.094135 19821 consensus_queue.cc:1048] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:23:22.093802 19816 consensus_queue.cc:1059] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 12:23:22.140323 19699 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:22.143532 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:22.150548 16544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1131) took 1653 ms. Trace:
W20260709 12:23:22.150751 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 77444.2us
I20260709 12:23:22.120674 19891 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:22.183154 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 612169us
I20260709 12:23:22.193964 17142 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:22.175065 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:22.186785 16398 raft_consensus.cc:2410] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:23:22.200886 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:22.201984 17271 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:22.184942 16544 rpcz_store.cc:276] 0709 12:23:20.496811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:20.596165 (+ 99354us) service_pool.cc:224] Handling call
0709 12:23:21.473107 (+876942us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:21.498829 (+ 25722us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:22.150319 (+651490us) spinlock_profiling.cc:243] Waited 836 ms on lock 0x7b5400aa69d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:22.150466 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1922126720}
I20260709 12:23:22.145422 19585 raft_consensus.cc:740] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:22.190848 19495 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:22.203660 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:22.149960 16474 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
I20260709 12:23:22.206897 17142 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:22.123647 19730 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:22.223523 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:22.234009 17254 raft_consensus.cc:1240] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:22.247030 17266 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:22.235510 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:22.248454 16471 raft_consensus.cc:2410] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:23:22.224694 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:22.249560 17258 raft_consensus.cc:2393] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 11: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:22.233310 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:22.150204 19816 raft_consensus.cc:3055] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:22.132015 19266 raft_consensus.cc:2804] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:22.253957 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:22.150650 19391 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:22.255937 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:22.256795 16472 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:22.234344 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:22.258072 17263 raft_consensus.cc:2393] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 7: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:22.255246 16470 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
W20260709 12:23:22.196511 19839 log.cc:927] Time spent T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 1.189s	user 0.011s	sys 0.000s
I20260709 12:23:22.191871 19674 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:22.270780 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:22.272267 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:22.272593 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:22.130365 19756 raft_consensus.cc:740] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:22.266373 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:22.274849 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 144582us
I20260709 12:23:22.275465 16475 raft_consensus.cc:2410] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:23:22.275507 17264 raft_consensus.cc:2393] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 9: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:22.142166 19757 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:22.273402 16471 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:23:22.221586 19695 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:22.210832 19755 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:22.222617 19733 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.184024 19642 consensus_queue.cc:1059] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 12:23:22.284230 16468 raft_consensus.cc:3055] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:22.285233 16468 raft_consensus.cc:740] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:22.261466 19816 raft_consensus.cc:740] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:22.261783 19391 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:22.261977 19787 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:22.299069 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:22.290998 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:22.292846 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:22.301507 17263 raft_consensus.cc:2376] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7. Re-sending same reply.
I20260709 12:23:22.142980 19764 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:22.294404 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:22.303001 17255 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:22.136276 19771 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:22.297806 16547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1295) took 1110 ms. Trace:
I20260709 12:23:22.307660 16547 rpcz_store.cc:276] 0709 12:23:21.187144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:21.187307 (+   163us) service_pool.cc:224] Handling call
0709 12:23:21.682055 (+494748us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:21.682834 (+   779us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:22.297599 (+614765us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b5400a4f950. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:22.297740 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1136690944}
I20260709 12:23:22.286281 16468 consensus_queue.cc:260] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:22.226419 19821 consensus_queue.cc:1059] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 12:23:22.305989 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:22.310951 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:22.311148 16398 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:23:22.312665 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:22.313086 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:22.313752 17271 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:22.274374 19266 raft_consensus.cc:697] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:22.320833 19585 consensus_queue.cc:260] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:22.322966 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:22.271723 19854 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:22.309835 16468 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:22.320588 19852 raft_consensus.cc:3055] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:22.319422 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:22.324450 16471 raft_consensus.cc:2393] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 11: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:22.318104 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:22.309206 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:22.284098 19674 raft_consensus.cc:697] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:22.321578 19266 consensus_queue.cc:237] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:22.312383 19895 log.cc:927] Time spent T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 12:23:22.211340 19891 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:22.435231 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:22.436293 17255 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:22.326829 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:22.334735 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:22.324844 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:22.323868 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:22.327992 19852 raft_consensus.cc:740] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:22.294731 19733 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:22.322027 19585 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:22.416733 17258 raft_consensus.cc:1275] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:23:22.308197 19391 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:22.289052 19755 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:22.352423 19756 consensus_queue.cc:260] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:22.420585 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:22.421571 16547 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:22.423713 16475 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:23:22.427277 17144 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:22.427963 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 110149us
I20260709 12:23:22.336527 19854 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:22.432417 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 118638us
W20260709 12:23:22.519625 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:22.494055 19819 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.521081 16550 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:23:22.521270 19699 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:22.521725 16550 raft_consensus.cc:606] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:22.521546 19819 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:22.524055 19819 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:22.521809 19699 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:22.517405 19695 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:22.526042 19695 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:22.527112 19699 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:22.530061 19891 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:22.533352 19896 log.cc:927] Time spent T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:23:22.533926 19487 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:22.442724 16397 raft_consensus.cc:2393] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 7: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:22.442026 16545 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:22.443491 16546 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:22.590909 16367 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:22.447707 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:22.470799 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:22.593010 17269 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
I20260709 12:23:22.471720 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:22.420011 16548 raft_consensus.cc:1275] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 12:23:22.475034 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 115717us
I20260709 12:23:22.451386 16468 raft_consensus.cc:1275] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 12:23:22.594466 16468 pending_rounds.cc:85] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 5
I20260709 12:23:22.594767 16468 pending_rounds.cc:107] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 12:23:22.595330 16468 raft_consensus.cc:2889] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:22.439778 19787 raft_consensus.cc:697] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:22.442018 17271 raft_consensus.cc:2376] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10. Re-sending same reply.
I20260709 12:23:22.542333 19730 consensus_queue.cc:1048] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:23:22.633374 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:22.543520 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:22.633704 16397 raft_consensus.cc:1240] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:22.528232 19695 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:22.545331 19854 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:22.639873 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 45302.6us
I20260709 12:23:22.640411 16472 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:23:22.640769 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:22.641757 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:22.546224 19757 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:23:22.546120 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 118916us
I20260709 12:23:22.548379 16470 raft_consensus.cc:3055] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:22.644256 16470 raft_consensus.cc:740] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:22.556968 19833 raft_consensus.cc:3055] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:22.560743 19674 consensus_queue.cc:237] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:22.562672 19816 consensus_queue.cc:260] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:22.566685 19890 log.cc:927] Time spent T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.460s	user 0.000s	sys 0.000s
W20260709 12:23:22.570436 19888 log.cc:927] Time spent T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.566s	user 0.003s	sys 0.000s
I20260709 12:23:22.518319 19858 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:22.573935 16542 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "f99066d13e524238aec6aa2024a8bd50"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:23:22.576505 17270 raft_consensus.cc:1275] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:23:22.578855 16549 raft_consensus.cc:1275] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:23:22.579772 16543 raft_consensus.cc:1275] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:23:22.580047 19819 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:22.580932 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:22.675028 16550 raft_consensus.cc:2376] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13. Re-sending same reply.
I20260709 12:23:22.539567 16514 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:22.600001 16547 raft_consensus.cc:1275] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:23:22.591820 16367 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:22.677537 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 85780.8us
I20260709 12:23:22.542371 17143 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:22.699301 19880 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:22.632845 19777 raft_consensus.cc:2804] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:22.504621 19642 consensus_queue.cc:1059] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.153s
I20260709 12:23:22.707441 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:22.713613 16468 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
I20260709 12:23:22.714608 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:22.710819 19777 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:22.715250 16468 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
I20260709 12:23:22.715485 19777 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:22.716104 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:22.716704 16468 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:22.717922 16468 raft_consensus.cc:1275] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:22.642354 19764 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:22.490281 19892 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:22.653301 19757 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:22.637838 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:22.634101 16475 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:22.656947 19833 raft_consensus.cc:740] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:22.480211 19755 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:22.646184 16470 consensus_queue.cc:260] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:22.451118 19752 consensus_queue.cc:1059] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.161s
I20260709 12:23:22.443666 19785 consensus_queue.cc:1048] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.193s
I20260709 12:23:22.452005 19385 consensus_queue.cc:1059] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.194s
I20260709 12:23:22.662191 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:22.730975 16471 raft_consensus.cc:1275] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:23:22.546170 19893 log.cc:927] Time spent T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.587s	user 0.005s	sys 0.000s
I20260709 12:23:22.670344 19756 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:22.762380 16398 raft_consensus.cc:1240] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:23:22.633049 19825 consensus_queue.cc:1048] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 12:23:22.728550 19847 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.769670 16472 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:22.664618 19806 consensus_queue.cc:1048] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.082s
I20260709 12:23:22.735733 19757 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:22.761780 19858 consensus_queue.cc:1059] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 12:23:22.741997 19816 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:22.700304 19880 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:22.713294 19819 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:22.775537 19385 consensus_queue.cc:1243] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer 610f4feda11942739236f2b121a4b9c1 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:23:22.789985 16403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 1034) took 1427 ms. Trace:
I20260709 12:23:22.789916 16472 raft_consensus.cc:1240] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:23:22.774406 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:22.792874 19833 consensus_queue.cc:260] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:22.721261 19900 consensus_queue.cc:1059] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
I20260709 12:23:22.777933 19699 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:22.703611 19894 consensus_queue.cc:1048] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.078s
I20260709 12:23:22.802039 19858 consensus_queue.cc:1243] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Peer ef57beab0e1d40d585983873c55107e8 log is divergent from this leader: its last log entry 6.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:23:22.802706 19777 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:22.753471 19733 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:22.779807 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:22.817736 16394 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:22.817248 16403 rpcz_store.cc:276] 0709 12:23:21.362146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:21.362356 (+   210us) service_pool.cc:224] Handling call
0709 12:23:21.362707 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:21.751433 (+388726us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:22.789694 (+1038261us) spinlock_profiling.cc:243] Waited 388 ms on lock 0x7b540011fb54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:22.789926 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":891705216}
I20260709 12:23:22.782262 19868 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:22.790453 19821 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:22.806126 19880 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:22.803726 19892 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:22.821463 19771 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:22.786928 19847 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:22.841341 16366 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:22.838847 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:22.840853 16395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 890) took 1356 ms. Trace:
I20260709 12:23:22.851661 16472 raft_consensus.cc:1240] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
W20260709 12:23:22.851727 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 264662us
I20260709 12:23:22.824970 19699 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:22.837430 19833 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:22.855155 16395 rpcz_store.cc:276] 0709 12:23:21.484280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:21.494998 (+ 10718us) service_pool.cc:224] Handling call
0709 12:23:21.495384 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:21.495666 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:23:21.555428 (+ 59762us) log.cc:844] Serialized 14 byte log entry
0709 12:23:22.183252 (+627824us) spinlock_profiling.cc:243] Waited 8.53 ms on lock 0x7b1800476ce0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d16c0a 00007f287c456219 00007f2872d9ea40 00007f2872d994e4 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a2923a 00007f2875e93a2a
0709 12:23:22.840420 (+657168us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:22.840466 (+    46us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:22.840520 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:22.840622 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:22.840776 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23798,"prepare.queue_time_us":65153,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":51,"raft.queue_time_us":100898,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"spinlock_wait_cycles":19625728,"thread_start_us":641445,"threads_started":2,"wal-append.queue_time_us":941670}
I20260709 12:23:22.860199 19699 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:22.842593 19385 raft_consensus.cc:3055] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:23:22.852754 19867 consensus_meta.cc:287] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.783s	user 0.008s	sys 0.005s
I20260709 12:23:22.789729 19787 consensus_queue.cc:237] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:22.869253 16471 raft_consensus.cc:1275] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 12:23:22.870916 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 207186us
I20260709 12:23:22.859992 16515 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:22.854557 19821 raft_consensus.cc:697] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:22.902343 16472 raft_consensus.cc:1275] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:23:22.906258 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:22.910209 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 50318.1us
I20260709 12:23:22.844866 19868 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:22.845575 19391 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:22.922376 19906 log.cc:927] Time spent T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:23:22.872063 19892 consensus_queue.cc:1048] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 12:23:22.909330 19891 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:22.926126 19905 log.cc:927] Time spent T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:23:22.926857 19852 consensus_queue.cc:260] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:22.927873 19868 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:22.929411 19904 log.cc:927] Time spent T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:23:22.929979 19495 raft_consensus.cc:3055] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:22.930696 19495 raft_consensus.cc:740] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:22.934746 19861 raft_consensus.cc:3055] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:22.935322 19861 raft_consensus.cc:740] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:22.934952 19821 consensus_queue.cc:237] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:22.872289 19385 raft_consensus.cc:740] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:22.871569 19546 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:22.911180 16515 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:23:22.947896 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 405651us
I20260709 12:23:22.942168 16394 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:23:22.951858 16542 raft_consensus.cc:606] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Received request to transfer leadership to TS f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:22.952771 16542 raft_consensus.cc:614] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:22.955049 17143 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:22.955283 19730 consensus_queue.cc:1048] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:22.956645 17143 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:22.956785 19730 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:22.957643 19730 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:22.957780 19755 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:22.958110 19730 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:22.915351 19825 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:22.955099 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:22.944687 19852 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:22.963176 19771 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:22.926280 19891 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:22.996206 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:23.001199 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 361614us
I20260709 12:23:23.007743 16366 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
W20260709 12:23:23.009733 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:23.011819 19766 consensus_queue.cc:1048] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:23.016119 16366 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
W20260709 12:23:23.030115 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 490637us
I20260709 12:23:23.035311 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:23.036036 16474 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:23.031368 16514 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:23.037638 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:23.043301 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:23.043955 16471 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:23.032752 19849 raft_consensus.cc:2804] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:23.057305 16471 raft_consensus.cc:1275] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:23:23.041216 16441 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:23.059870 19920 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:23.060397 19920 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:23.060786 19920 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:22.993532 19825 consensus_queue.cc:1048] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:23:23.063915 16441 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:23.064875 19868 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:23.064525 19837 consensus_queue.cc:1059] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:23.065285 19825 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:23.066040 16441 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:23.065380 19868 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:23.067823 19868 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:23.064627 16470 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:23.070099 16441 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:23.004983 19771 raft_consensus.cc:697] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 12 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:23.073520 16473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1219) took 1014 ms. Trace:
I20260709 12:23:23.102061 16441 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:23.100826 17142 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.002s	sys 0.045s
W20260709 12:23:23.103144 17142 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.002s	sys 0.045s
I20260709 12:23:23.068148 19854 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:23:23.103539 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 518167us
I20260709 12:23:23.107136 16441 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:23.101137 16473 rpcz_store.cc:276] 0709 12:23:22.058884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:22.065488 (+  6604us) service_pool.cc:224] Handling call
0709 12:23:22.065820 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:22.112530 (+ 46710us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:23:22.239712 (+127182us) log.cc:844] Serialized 14 byte log entry
0709 12:23:22.969294 (+729582us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:22.969880 (+   586us) log.cc:844] Serialized 26 byte log entry
0709 12:23:22.970050 (+   170us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:22.970099 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:22.970196 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:23.073219 (+103023us) raft_consensus.cc:1685] finished
0709 12:23:23.073248 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:23.073445 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":106721,"prepare.queue_time_us":112380,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":157008,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":14828,"thread_start_us":634271,"threads_started":3,"wal-append.queue_time_us":554883}
I20260709 12:23:22.978259 19495 consensus_queue.cc:260] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:23.029690 19912 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:23.029279 19785 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:23.105841 17142 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:23.113878 19858 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:23.129359 16470 raft_consensus.cc:1275] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:23:23.104489 19837 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:23.116179 16366 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:23.037853 19819 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:23.122290 17142 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:23.124401 16401 raft_consensus.cc:2410] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:23:23.124963 19785 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:23.127425 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 68885.1us
W20260709 12:23:23.111315 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 162544us
I20260709 12:23:23.109375 19849 raft_consensus.cc:697] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 8 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:23.135615 16366 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:23.109784 19867 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:23:23.162768 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:23.167022 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:23.169705 16550 raft_consensus.cc:1275] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:23:23.142419 17142 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:23.111521 19892 consensus_queue.cc:1048] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:23:23.198846 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:23.143481 19487 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:23.145282 19787 raft_consensus.cc:2764] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:23:23.214516 16544 raft_consensus.cc:1240] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:23:23.161551 19908 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:23:23.203416 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 57425.3us
I20260709 12:23:23.216954 17255 raft_consensus.cc:1240] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260709 12:23:23.160986 19854 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.217972 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:23.218115 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:23.218647 17266 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
I20260709 12:23:23.222430 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:23.224021 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:23.226152 19926 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:23.226734 19926 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:23.227547 19926 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:23.229300 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:23.204741 19916 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.230858 19917 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:23.231133 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:23.231267 19913 log.cc:927] Time spent T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 12:23:23.231745 17269 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
I20260709 12:23:23.232678 16400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 894) took 1010 ms. Trace:
W20260709 12:23:23.232618 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 90278.3us
I20260709 12:23:23.233759 17142 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:23.234103 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:23.204675 19819 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.235112 17142 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:23.239549 17254 raft_consensus.cc:1275] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:23:23.240535 17261 raft_consensus.cc:1275] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:23:23.230012 17264 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:23.224769 19835 raft_consensus.cc:3055] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:23.216713 19391 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:23.225020 19853 consensus_meta.cc:287] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 1.208s	user 0.012s	sys 0.005s
I20260709 12:23:23.221583 19908 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:23.250895 17142 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:23.198290 19931 consensus_queue.cc:1048] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.067s
I20260709 12:23:23.220054 19820 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:23.243158 19922 log.cc:927] Time spent T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 12:23:23.298334 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:23:23.265305 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 60134.1us
I20260709 12:23:23.219653 19487 raft_consensus.cc:697] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 7 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
W20260709 12:23:23.235584 19585 consensus_meta.cc:287] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.727s	user 0.009s	sys 0.004s
I20260709 12:23:23.204891 19771 consensus_queue.cc:237] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:23.244624 19835 raft_consensus.cc:740] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:23.233289 16400 rpcz_store.cc:276] 0709 12:23:22.222202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:22.313051 (+ 90849us) service_pool.cc:224] Handling call
0709 12:23:22.313355 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:22.324021 (+ 10666us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:22.324052 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:22.334984 (+ 10932us) log.cc:844] Serialized 26 byte log entry
0709 12:23:22.839742 (+504758us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:22.839798 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:22.839890 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:23.232475 (+392585us) raft_consensus.cc:1685] finished
0709 12:23:23.232492 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:23.232623 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10259,"raft.queue_time_us":10582,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":17924,"thread_start_us":333768,"threads_started":1,"wal-append.queue_time_us":526397}
I20260709 12:23:23.272593 19917 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:23.307689 17264 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
W20260709 12:23:23.243748 19923 log.cc:927] Time spent T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:23:23.231169 19239 consensus_queue.cc:1048] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 12:23:23.278848 19816 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:23.297698 19764 raft_consensus.cc:993] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1: : Instructing follower 52ab50b0237346c1a13ede947f6a63c2 to start an election
I20260709 12:23:23.298333 17254 raft_consensus.cc:1240] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:23:23.259769 19880 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:23.301083 16514 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:23.285948 17142 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:23.254859 19819 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:23.223363 19854 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:23.276906 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 127488us
I20260709 12:23:23.321198 17261 raft_consensus.cc:1275] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:23:23.305471 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:23.271324 19787 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:23.300064 19756 consensus_queue.cc:1048] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:23:23.328502 16514 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:23.293005 19820 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:23.330017 16514 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:23.331190 19820 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:23.331986 17258 raft_consensus.cc:1275] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:23:23.241986 19916 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:23.312878 16441 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:23.309654 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:23.333992 19916 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:23.304106 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 44603.8us
I20260709 12:23:23.279072 19853 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:23.335542 19777 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:23.293758 19757 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:23.336656 17255 raft_consensus.cc:1275] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:23:23.337303 19915 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:23.337625 19935 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:23.337767 19915 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:23.338142 19935 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:23.338239 19915 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:23.338353 19757 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:23.338588 19935 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:23.302899 19785 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:23.279518 19908 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:23.318624 19487 consensus_queue.cc:237] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:23.324734 19897 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 12:23:23.320752 16515 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:23.326664 19854 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:23.327126 19755 consensus_queue.cc:1048] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:23.248432 19932 consensus_queue.cc:1048] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 12:23:23.332777 19835 consensus_queue.cc:260] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:23.319190 17270 raft_consensus.cc:1240] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:23.340529 16514 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:23.314555 19946 consensus_queue.cc:1048] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:23:23.342408 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:23.342846 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:23.371526 16545 raft_consensus.cc:1240] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:23:23.373977 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:23.374840 16514 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:23.340133 16441 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:23.293215 19733 consensus_queue.cc:1048] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 12:23:23.344535 19941 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:23:23.320462 19842 consensus_meta.cc:287] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 1.254s	user 0.019s	sys 0.000s
I20260709 12:23:23.353325 16546 raft_consensus.cc:1240] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:23.354251 16469 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:23.316105 19821 consensus_queue.cc:1059] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:23:23.355145 19914 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:23.356024 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:23.358717 19495 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:23.360688 19940 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:23:23.367123 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:23.356051 19938 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:23.384387 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 44273.5us
I20260709 12:23:23.310497 19912 consensus_queue.cc:1048] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 12:23:23.270352 19849 consensus_queue.cc:237] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:23.327471 19868 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:23.388782 19897 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.337329 19764 raft_consensus.cc:1081] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Signalling peer 52ab50b0237346c1a13ede947f6a63c2 to start an election
I20260709 12:23:23.389394 19854 raft_consensus.cc:2804] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:23:23.396477 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 77316us
I20260709 12:23:23.441757 16367 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
W20260709 12:23:23.425830 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:23.399170 19835 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:23.443707 16515 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:23.401420 19819 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:23.406381 16441 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:23.406584 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:23.406921 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:23.415132 19954 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.415160 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:23.449187 16470 raft_consensus.cc:2468] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 6.
I20260709 12:23:23.416160 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:23.449923 16367 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:23.394531 19940 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:23.386513 19806 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:23.446640 16545 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
I20260709 12:23:23.447362 16473 raft_consensus.cc:1240] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:23.450253 16474 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:23:23.391018 19931 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:23.407734 19585 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.416319 19239 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:23.458876 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:23.445202 19854 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:23.468521 16515 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:23.464764 19752 consensus_queue.cc:1048] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:23:23.471172 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:23.476665 19928 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:23.477599 19948 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.471180 19239 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:23.478140 19948 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:23.480518 19932 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:23.480437 19881 consensus_queue.cc:1048] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 12:23:23.482016 19771 consensus_queue.cc:1048] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 12:23:23.469175 19642 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:23.485000 19642 raft_consensus.cc:697] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:23.485162 17269 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:23:23.485903 19642 consensus_queue.cc:237] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:23.486649 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:23.471944 19941 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:23.488552 19880 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:23.455065 19940 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:23.460667 19931 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:23.502746 17269 raft_consensus.cc:1240] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
W20260709 12:23:23.498468 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:23.457154 19546 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:23.456311 19806 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:23.475953 19699 consensus_meta.cc:287] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.586s	user 0.011s	sys 0.000s
I20260709 12:23:23.460377 19833 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:23.480068 19239 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:23.404002 19914 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:23.505159 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 98853.6us
I20260709 12:23:23.471036 19816 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:23.512941 19546 raft_consensus.cc:697] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 7 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:23.508316 19806 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:23.507794 19730 consensus_meta.cc:287] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.549s	user 0.013s	sys 0.000s
I20260709 12:23:23.516407 19833 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:23.468132 19585 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:23.410686 19938 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:23.472445 19854 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:23.500047 19868 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:23.502296 19880 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.541079 19880 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:23.543418 19880 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:23.523965 16441 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:23.517493 19699 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:23.546762 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:23.547118 19699 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:23.547430 17258 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:23.506212 19931 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:23.550081 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:23.551231 19695 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.517817 19947 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:23:23.558863 19852 consensus_meta.cc:287] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.580s	user 0.010s	sys 0.000s
I20260709 12:23:23.553691 19695 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:23.560461 16468 raft_consensus.cc:1275] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:23:23.560832 16468 pending_rounds.cc:85] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 4
I20260709 12:23:23.561065 16468 pending_rounds.cc:107] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:23:23.561308 16468 raft_consensus.cc:2889] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:23:23.521723 19863 consensus_meta.cc:287] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 1.498s	user 0.014s	sys 0.004s
I20260709 12:23:23.455744 19954 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:23.529369 19833 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:23.505748 19881 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:23.446664 19897 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:23.529722 19730 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:23.584532 16476 raft_consensus.cc:1275] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:23:23.586611 16476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1217) took 1528 ms. Trace:
I20260709 12:23:23.535328 19938 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:23.589016 19487 consensus_queue.cc:1048] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:23.589377 19938 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.589819 19938 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:23.590443 19487 consensus_queue.cc:1048] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:23.591846 19938 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:23.591691 19487 consensus_queue.cc:1048] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:23.591440 19854 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:23.532073 19842 consensus_queue.cc:1059] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 12:23:23.592846 19842 consensus_queue.cc:1243] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer 610f4feda11942739236f2b121a4b9c1 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:23:23.593411 19266 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.594018 19914 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:23.593897 19266 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:23.594909 19854 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:23.596378 16398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 898) took 1291 ms. Trace:
I20260709 12:23:23.596813 16398 rpcz_store.cc:276] 0709 12:23:22.305346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:22.475465 (+170119us) service_pool.cc:224] Handling call
0709 12:23:22.521834 (+ 46369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:22.762912 (+241078us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:23.596111 (+833199us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b5400a79ed4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:23.596300 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":552602368}
I20260709 12:23:23.598065 19730 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:23.600503 19931 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:23.601193 19585 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:23.507438 19932 consensus_queue.cc:1059] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:23:23.460216 19937 consensus_queue.cc:1048] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 12:23:23.602190 17266 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:23.544351 19546 consensus_queue.cc:237] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:23.602984 19931 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:23.604657 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:23.480087 19948 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:23.557103 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:23.527154 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:23.562885 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:23.562930 16366 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:23.605422 17266 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:23.607625 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 44767.3us
I20260709 12:23:23.495743 19915 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:23.580159 19695 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:23.580458 19847 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:23.625455 16398 raft_consensus.cc:1275] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:23:23.626050 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:23.574205 19880 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:23.580955 19954 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:23.581634 16468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1228) took 1295 ms. Trace:
I20260709 12:23:23.601370 19266 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:23.607218 17258 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:23.606923 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:23.606180 16401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46250 (request call id 1051) took 1470 ms. Trace:
I20260709 12:23:23.536660 19926 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:23.587095 16476 rpcz_store.cc:276] 0709 12:23:22.058259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:22.061610 (+  3351us) service_pool.cc:224] Handling call
0709 12:23:22.061951 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:23.584933 (+1522982us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:23.586341 (+  1408us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b540012b954. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:23.586542 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3484475264}
I20260709 12:23:23.617830 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:23.624722 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:23.626674 16545 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:23.602223 19932 raft_consensus.cc:3055] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:23.630448 16550 raft_consensus.cc:1240] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13. Ops: [10.6-12.7]
I20260709 12:23:23.632310 17266 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
I20260709 12:23:23.658506 17263 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:23.667191 16470 raft_consensus.cc:1240] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:23:23.658398 16401 rpcz_store.cc:276] 0709 12:23:22.135718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:22.135842 (+   124us) service_pool.cc:224] Handling call
0709 12:23:23.605954 (+1470112us) spinlock_profiling.cc:243] Waited 987 ms on lock 0x7b54004ec0d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:23.606098 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2269753728}
I20260709 12:23:23.659484 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:23.656888 16468 rpcz_store.cc:276] 0709 12:23:22.286354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:22.719407 (+433053us) service_pool.cc:224] Handling call
0709 12:23:22.720039 (+   632us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:23.574816 (+854777us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:23.581340 (+  6524us) spinlock_profiling.cc:243] Waited 840 ms on lock 0x7b5400b05254. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:23.581560 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1931606400}
I20260709 12:23:23.628958 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:23.669131 16398 raft_consensus.cc:2393] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 10: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:23.664908 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:23.669662 16545 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:23.665508 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:23.664093 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:23.682293 16514 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:23.688685 16396 raft_consensus.cc:2393] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 8: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:23.689610 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:23.615914 19777 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:23.693526 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:23.695853 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:23.696323 17266 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:23:23.660241 19920 consensus_meta.cc:287] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.599s	user 0.009s	sys 0.004s
I20260709 12:23:23.697293 16514 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:23.695242 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:23.616954 19854 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:23.619218 19853 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:23.633623 19842 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:23.700675 16543 raft_consensus.cc:2393] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 8: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:23.626760 19863 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:23.699973 17271 raft_consensus.cc:2393] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 8: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:23.705924 17258 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:23.706907 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:23.665741 19929 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.666198 19932 raft_consensus.cc:740] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:23.713910 17263 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:23.585007 19881 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:23.693017 19849 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.717420 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:23.722782 19947 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.741665 19947 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:23.708717 19854 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:23.743983 19947 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:23.728471 16545 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:23.728684 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:23.746567 17258 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:23.747613 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:23.748286 17258 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:23.733307 19940 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:23.749212 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:23.750406 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:23.751020 19940 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:23.711022 17266 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:23.753055 19941 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:23.753579 19926 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:23.754365 17258 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:23.694804 19585 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:23.702570 19766 raft_consensus.cc:2804] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:23.740811 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:23.724848 19912 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:23.771776 19912 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:23.774308 19912 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:23.712853 19898 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.775765 19898 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:23.699788 19868 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:23.704655 19847 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:23.778589 19847 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:23.658407 19880 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:23.764771 16546 raft_consensus.cc:2393] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 10: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:23.755252 16549 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:23.786441 19880 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:23.771180 19766 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:23.788853 19766 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:23.791128 19385 consensus_queue.cc:260] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:23.793932 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:23.793764 19585 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:23.769407 17263 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:23:23.801121 17258 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:23.707825 19914 consensus_queue.cc:1059] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 12:23:23.703711 19920 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:23.735551 19391 consensus_queue.cc:1059] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.121s
I20260709 12:23:23.716817 19842 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:23.826895 19941 consensus_queue.cc:1048] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 12:23:23.711928 19853 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:23.902994 17255 raft_consensus.cc:1240] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:23.898988 17263 raft_consensus.cc:1240] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:23:23.898113 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:23.933180 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:23.729568 19863 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:23.825160 19779 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:23.823400 19858 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:23.874729 19385 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:23.876693 19959 consensus_queue.cc:1048] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.104s
I20260709 12:23:23.999251 16474 raft_consensus.cc:1240] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:23:23.731889 19929 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:23.956789 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:24.003144 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:23.764967 19935 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:24.005839 16395 raft_consensus.cc:1240] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:24.016371 16400 raft_consensus.cc:2393] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 10: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:23.903086 19852 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.901372 19881 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:23.877478 19699 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:23.766561 19849 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:23.935842 19837 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.936758 19854 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:23.894739 19585 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:24.004669 16476 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:24.011700 19937 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:24.014680 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:24.020749 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 295196us
I20260709 12:23:24.008042 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:23.958333 19908 consensus_meta.cc:287] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.587s	user 0.005s	sys 0.005s
I20260709 12:23:23.902345 19897 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:23.974138 19940 raft_consensus.cc:3055] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:24.044222 19932 consensus_queue.cc:260] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:24.064884 16399 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:23:24.045732 19842 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:24.049212 19929 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:24.049801 19863 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:24.050083 17270 raft_consensus.cc:1240] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:24.051304 19764 raft_consensus.cc:3055] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:24.051587 19853 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:23.889683 19806 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:23.929450 19880 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:23.888226 19766 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:24.015359 19911 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:24.064613 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:24.097455 17261 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:23.989791 19858 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:24.045478 19757 raft_consensus.cc:3055] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:24.010579 19833 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:24.074843 16516 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:24.033186 19816 consensus_queue.cc:1059] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 12:23:24.035280 19852 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:24.022825 19779 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:24.111660 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:24.096180 19764 raft_consensus.cc:740] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:24.230432 19861 consensus_queue.cc:260] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:24.230747 19764 consensus_queue.cc:260] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:24.231587 19861 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:24.073163 19940 raft_consensus.cc:740] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:24.233001 19940 consensus_queue.cc:260] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:24.234404 19940 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:24.235647 17261 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:24.120275 19941 consensus_queue.cc:1048] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.116s
W20260709 12:23:24.238698 19909 log.cc:927] Time spent T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 1.272s	user 0.003s	sys 0.002s
I20260709 12:23:24.239295 19849 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:24.240325 19920 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:24.066084 19937 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:24.241317 19928 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:24.241439 19858 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:24.241901 19928 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:24.243835 19937 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:24.072590 19908 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:24.245273 19935 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:24.123618 19921 log.cc:927] Time spent T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:23:24.246819 19699 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:24.247387 19699 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:24.249143 19833 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:24.253060 19585 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:24.253785 19585 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:24.255540 19954 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:23:24.255795 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 181013us
W20260709 12:23:24.173344 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:24.256964 19891 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:24.256247 19585 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:24.232029 19764 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:24.228802 17270 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
W20260709 12:23:24.254144 19963 log.cc:927] Time spent T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.190s	user 0.000s	sys 0.000s
I20260709 12:23:24.246030 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:24.054616 19785 consensus_queue.cc:1059] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:23:24.228657 16473 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:24.060287 19837 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:24.096385 19932 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:24.285665 16542 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:23:24.286152 16551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1309) took 1969 ms. Trace:
I20260709 12:23:24.293001 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:24.321331 16394 raft_consensus.cc:2393] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 10: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
W20260709 12:23:24.286185 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 263879us
I20260709 12:23:24.319325 16551 rpcz_store.cc:276] 0709 12:23:22.316945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:22.317721 (+   776us) service_pool.cc:224] Handling call
0709 12:23:22.318042 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:22.318511 (+   469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:22.318542 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:22.319088 (+   546us) log.cc:844] Serialized 26 byte log entry
0709 12:23:22.807193 (+488105us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:22.807230 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:22.807299 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:24.238131 (+1430832us) spinlock_profiling.cc:243] Waited 3.09 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:23:24.260414 (+ 22283us) raft_consensus.cc:1685] finished
0709 12:23:24.260438 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:24.286086 (+ 25648us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4334,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":11698,"spinlock_wait_cycles":7110016,"thread_start_us":356456,"threads_started":1,"wal-append.queue_time_us":560934}
I20260709 12:23:24.297855 16514 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:24.101424 19757 raft_consensus.cc:740] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:24.246852 19908 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:24.318517 16542 raft_consensus.cc:606] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:24.190850 19868 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:24.338821 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:24.341394 16472 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
W20260709 12:23:24.329042 19787 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.011s	sys 0.029s
I20260709 12:23:24.374464 17270 raft_consensus.cc:1275] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:23:24.374310 17144 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:24.374300 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:24.379343 16468 raft_consensus.cc:2410] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:23:24.120661 19911 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:24.120929 19880 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:24.285712 19858 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:23:24.390264 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:24.393730 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:24.346434 19816 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.363153 19914 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:24.373765 19868 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:24.416716 16514 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:24.393170 19947 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:24.362097 19833 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.383962 19585 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:23:24.421691 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 367637us
W20260709 12:23:24.406400 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:24.442699 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:24.399549 19978 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.447000 16367 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:24.465363 16442 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
W20260709 12:23:24.495440 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 48595.5us
I20260709 12:23:24.479467 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:24.497015 16473 raft_consensus.cc:2393] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 10: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:24.498827 16543 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:23:24.499492 16543 raft_consensus.cc:606] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:24.499975 16543 raft_consensus.cc:614] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:24.559760 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:24.560770 16473 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:23:24.441109 19941 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:24.411780 19914 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:24.539093 16367 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:24.479246 19865 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.458734 19920 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.449126 19908 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:24.532968 19937 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:24.575596  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:23:24.591161 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:24.420469 19816 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:24.515556 19978 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:24.620601 16473 raft_consensus.cc:2393] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 9: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:24.479804 19787 consensus_queue.cc:1048] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.131s
I20260709 12:23:24.554147 19853 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:23:24.623579 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 84605us
I20260709 12:23:24.460891 19585 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:24.629307 16367 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:24.589362 19975 log.cc:927] Time spent T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.294s	user 0.000s	sys 0.001s
W20260709 12:23:24.600796 19385 consensus_meta.cc:287] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.603s	user 0.008s	sys 0.004s
I20260709 12:23:24.405622 19858 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:24.458266 19833 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:24.577324 19914 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:24.575955 19969 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:24.617142 19937 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:24.448145 19757 consensus_queue.cc:260] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:24.627686 19973 log.cc:927] Time spent T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.367s	user 0.000s	sys 0.001s
W20260709 12:23:24.638027 19965 log.cc:927] Time spent T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.374s	user 0.001s	sys 0.001s
I20260709 12:23:24.627851 19861 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:23:24.628749 19852 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:24.636595 19766 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:24.696321 17254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 945) took 1070 ms. Trace:
I20260709 12:23:24.611703 19865 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:24.670677 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:24.638206 19849 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:24.699357 17254 rpcz_store.cc:276] 0709 12:23:23.625671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:24.038831 (+413160us) service_pool.cc:224] Handling call
0709 12:23:24.039167 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:24.039946 (+   779us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:24.039988 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:24.064601 (+ 24613us) log.cc:844] Serialized 26 byte log entry
0709 12:23:24.421058 (+356457us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:24.421101 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:24.421175 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:24.696009 (+274834us) raft_consensus.cc:1685] finished
0709 12:23:24.696036 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:24.696241 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8640,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"thread_start_us":319956,"threads_started":1,"wal-append.queue_time_us":546585}
I20260709 12:23:24.660058 19853 raft_consensus.cc:697] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:24.626914 19920 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:24.626263 19908 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:24.709946 19880 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:24.710316 19847 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:24.711326 19968 log.cc:927] Time spent T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.664s	user 0.002s	sys 0.000s
I20260709 12:23:24.713778 19858 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:24.715830  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:24.716753  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:24.717553 16550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1320) took 1025 ms. Trace:
I20260709 12:23:24.718127 19978 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:24.720293 17255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 937) took 1347 ms. Trace:
I20260709 12:23:24.720702 17255 rpcz_store.cc:276] 0709 12:23:23.373187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:23.373426 (+   239us) service_pool.cc:224] Handling call
0709 12:23:23.902773 (+529347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:23.926053 (+ 23280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:24.720079 (+794026us) spinlock_profiling.cc:243] Waited 529 ms on lock 0x7b54005f6250. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:24.720236 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1216380672}
I20260709 12:23:24.725416 19816 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:24.725680 19819 raft_consensus.cc:3055] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:24.726194 19819 raft_consensus.cc:740] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:24.727499 17263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 941) took 1229 ms. Trace:
I20260709 12:23:24.727126 19819 consensus_queue.cc:260] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:24.727876 17263 rpcz_store.cc:276] 0709 12:23:23.498339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:23.937141 (+438802us) service_pool.cc:224] Handling call
0709 12:23:23.994468 (+ 57327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:24.011279 (+ 16811us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:24.011318 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:24.018187 (+  6869us) log.cc:844] Serialized 26 byte log entry
0709 12:23:24.402450 (+384263us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:24.402496 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:24.402584 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:24.727313 (+324729us) raft_consensus.cc:1685] finished
0709 12:23:24.727333 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:24.727445 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16268,"raft.queue_time_us":16844,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":9812,"thread_start_us":383862,"threads_started":1,"wal-append.queue_time_us":676581}
I20260709 12:23:24.730276 19865 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:24.730924 19849 raft_consensus.cc:697] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:24.731324 19585 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:24.731709 16474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1186) took 1186 ms. Trace:
I20260709 12:23:24.731925 19849 consensus_queue.cc:237] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:24.733341 19853 consensus_queue.cc:237] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:24.734856 19806 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:24.735776 19920 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:24.736799 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:24.707968 19971 log.cc:927] Time spent T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.343s	user 0.001s	sys 0.000s
I20260709 12:23:24.741139 16470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1187) took 1195 ms. Trace:
I20260709 12:23:24.741788 19943 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.742340 19943 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:24.743311 16475 raft_consensus.cc:1240] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 8. Current term is 9. Ops: [7.4-8.5]
W20260709 12:23:24.742652 16469 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34840 (request call id 1170) took 2716 ms (client timeout 1945 ms). Trace:
I20260709 12:23:24.744875 16549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37226 (request call id 1319) took 1056 ms. Trace:
I20260709 12:23:24.745143 16398 raft_consensus.cc:1240] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:24.741595 16470 rpcz_store.cc:276] 0709 12:23:23.545934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:23.546061 (+   127us) service_pool.cc:224] Handling call
0709 12:23:23.546377 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:23.667770 (+121393us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:24.740866 (+1073096us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b540080c0d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:24.741076 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":277089536}
I20260709 12:23:24.745474 16549 rpcz_store.cc:276] 0709 12:23:23.688906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:23.689390 (+   484us) service_pool.cc:224] Handling call
0709 12:23:24.744698 (+1055308us) spinlock_profiling.cc:243] Waited 64.9 ms on lock 0x7b5400304654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:24.744812 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":149249280}
I20260709 12:23:24.746348 19585 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:24.746908 19585 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:24.746367 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:24.749023 16470 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:24.751163 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:24.751799 17255 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:23:24.752794 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:24.753955 19924 log.cc:927] Time spent T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 1.505s	user 0.005s	sys 0.000s
W20260709 12:23:24.695199 19982 log.cc:927] Time spent T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 12:23:24.754989 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:24.744550 16469 rpcz_store.cc:269] 0709 12:23:22.026294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:22.026425 (+   131us) service_pool.cc:224] Handling call
0709 12:23:24.742396 (+2715971us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b5400974354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:24.742541 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2984368896}
I20260709 12:23:24.765645 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:24.766366 17254 raft_consensus.cc:2393] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 10: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:24.753860 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:24.755681 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:24.755848 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:24.771051 16398 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:24.767365 19819 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:24.772883 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:24.774322 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:24.774855 17255 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
I20260709 12:23:24.775893 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:24.759080 16547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1321) took 1064 ms. Trace:
W20260709 12:23:24.780947 19976 log.cc:927] Time spent T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.200s	user 0.000s	sys 0.001s
I20260709 12:23:24.732117 16474 rpcz_store.cc:276] 0709 12:23:23.545646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:23.549293 (+  3647us) service_pool.cc:224] Handling call
0709 12:23:23.549666 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:23.999744 (+450078us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:24.730782 (+731038us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b540028b020. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e00dc0 00007f2875e8ca02 00007f2875e88dd6 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:23:24.731462 (+   680us) spinlock_profiling.cc:243] Waited 449 ms on lock 0x7b5400a686d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:24.731654 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1066019840}
I20260709 12:23:24.702029 19779 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:24.764369 16470 raft_consensus.cc:2468] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:23:24.717921 16550 rpcz_store.cc:276] 0709 12:23:23.692686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:23.692930 (+   244us) service_pool.cc:224] Handling call
0709 12:23:23.693257 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:23.693628 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:23.693659 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:23.707543 (+ 13884us) log.cc:844] Serialized 26 byte log entry
0709 12:23:23.869803 (+162260us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:23.869870 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:23.869985 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:24.710660 (+840675us) spinlock_profiling.cc:243] Waited 5.83 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:23:24.717286 (+  6626us) raft_consensus.cc:1685] finished
0709 12:23:24.717309 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:24.717498 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":737,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":8832,"spinlock_wait_cycles":13401984,"thread_start_us":145221,"threads_started":1,"wal-append.queue_time_us":231305}
I20260709 12:23:24.767168 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:24.804271 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:24.767309 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:24.767195 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:24.805861 16514 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
W20260709 12:23:24.762130 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:24.769271 19868 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:24.770387 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:24.776855 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:24.777736 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:24.811168 17270 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:24.811514 16469 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:24.719775 19699 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:24.781068 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:24.775970 16403 raft_consensus.cc:1275] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 12:23:24.758397 16473 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:24.813390 16403 pending_rounds.cc:85] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 6
I20260709 12:23:24.684859 19969 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:24.783497 16398 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:23:24.696213 19837 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:24.786257 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:24.832103 16474 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:23:24.832576 17254 raft_consensus.cc:2393] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 12: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:24.833009 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:24.763330 17144 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:24.834149 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 70945us
I20260709 12:23:24.834812 17144 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:24.784358 16547 rpcz_store.cc:276] 0709 12:23:23.694992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:23.695173 (+   181us) service_pool.cc:224] Handling call
0709 12:23:23.695518 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:23.695947 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:23:23.712624 (+ 16677us) log.cc:844] Serialized 14 byte log entry
0709 12:23:24.009738 (+297114us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:24.385835 (+376097us) spinlock_profiling.cc:243] Waited 1.89 ms on lock 0x7b18003c04c0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:23:24.690850 (+305015us) spinlock_profiling.cc:243] Waited 1.81 ms on lock 0x7b2c000ab488. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:23:24.696508 (+  5658us) log.cc:844] Serialized 26 byte log entry
0709 12:23:24.696706 (+   198us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:24.696755 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:24.696847 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:24.758813 (+ 61966us) raft_consensus.cc:1685] finished
0709 12:23:24.758844 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:24.759012 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67948,"prepare.queue_time_us":16516,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":1795,"raft.queue_time_us":84579,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":18229,"spinlock_wait_cycles":9421056,"thread_start_us":216325,"threads_started":1,"wal-append.queue_time_us":365747}
I20260709 12:23:24.788080 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:24.793006 16402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 900) took 2472 ms. Trace:
I20260709 12:23:24.836172 16469 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:23:24.764019 19920 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:24.805081 16470 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 11: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:24.805971 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:24.768098 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:24.687788 19937 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:24.844614 19764 consensus_meta.cc:287] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.558s	user 0.007s	sys 0.004s
I20260709 12:23:24.848599 19819 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:24.768683 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:24.849076 19819 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:24.849466 19819 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:24.694312 19959 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:24.813810 16403 pending_rounds.cc:107] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:23:24.850051 16403 raft_consensus.cc:2889] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:24.850010 19959 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:24.829775 19585 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:24.848680 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:24.854794 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:24.855043 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:24.855623 16396 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:24.796643 19943 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:24.820788 19978 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:24.821561 19858 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:24.889591 19858 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:24.892874 17266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 978) took 1429 ms. Trace:
I20260709 12:23:24.894058 19969 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:24.893316 17266 rpcz_store.cc:276] 0709 12:23:23.463613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:23.752135 (+288522us) service_pool.cc:224] Handling call
0709 12:23:23.752478 (+   343us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:23.752917 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:23.752948 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:23.753601 (+   653us) log.cc:844] Serialized 52 byte log entry
0709 12:23:23.996195 (+242594us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:23.996264 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:23.997319 (+  1055us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:24.892653 (+895334us) raft_consensus.cc:1685] finished
0709 12:23:24.892673 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:24.892814 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1664,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":12250,"thread_start_us":242157,"threads_started":1,"wal-append.queue_time_us":311336}
I20260709 12:23:24.822613 19847 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.896278 19847 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:24.899080 19847 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:24.900270 16476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34840 (request call id 1193) took 1107 ms. Trace:
I20260709 12:23:24.900774 16476 rpcz_store.cc:276] 0709 12:23:23.793327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:23.793613 (+   286us) service_pool.cc:224] Handling call
0709 12:23:24.900054 (+1106441us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b540080a7d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:24.900179 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":410873600}
I20260709 12:23:24.901391 16401 raft_consensus.cc:1275] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:23:24.901820 19868 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:24.902040 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:24.902892 16476 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 7.
I20260709 12:23:24.903607 16476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34826 (request call id 1250) took 1074 ms. Trace:
W20260709 12:23:24.826705 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:24.836840 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:24.905694 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:24.830492 19941 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.857823 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:24.880781 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:24.881345 16473 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:23:24.888541 19978 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:24.813099 19932 raft_consensus.cc:2804] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:23:24.856549 19982 log.cc:927] Time spent T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:23:24.906790 16514 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:24.836252 16402 rpcz_store.cc:276] 0709 12:23:22.320364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:22.483449 (+163085us) service_pool.cc:224] Handling call
0709 12:23:22.483883 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:22.805893 (+322010us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:22.805923 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:22.819124 (+ 13201us) log.cc:844] Serialized 26 byte log entry
0709 12:23:23.002773 (+183649us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:23.002833 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:23.053123 (+ 50290us) spinlock_profiling.cc:243] Waited 33.7 ms on lock 0x7b5400bb80d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:23.053197 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:24.753337 (+1700140us) spinlock_profiling.cc:243] Waited 17.2 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:23:24.792773 (+ 39436us) raft_consensus.cc:1685] finished
0709 12:23:24.792794 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:24.792942 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6251,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":77,"spinlock_wait_cycles":116931584,"thread_start_us":436620,"threads_started":2,"wal-append.queue_time_us":343962}
I20260709 12:23:24.909570 16474 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:24.916195 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:24.917738 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:24.918453 16474 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:24.909570 16469 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:24.853880 19787 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:24.904163 16476 rpcz_store.cc:276] 0709 12:23:23.829241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:24.901740 (+1072499us) service_pool.cc:224] Handling call
0709 12:23:24.903531 (+  1791us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:24.905194 19833 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:24.933599 16470 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:24.892167 19858 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:24.938848 16514 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:24.903636 19816 consensus_queue.cc:1059] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:23:24.877935 19920 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:24.958592 19764 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:24.959120 19764 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:24.959483 19764 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:24.961099 19908 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:24.913947 19941 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:24.966616 19941 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:24.963806 19819 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:24.931830 19969 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:24.931939 19932 raft_consensus.cc:697] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:24.974596 16469 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:23:24.972725 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:24.987305 16474 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
W20260709 12:23:24.943783 19928 consensus_meta.cc:287] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.698s	user 0.011s	sys 0.003s
W20260709 12:23:24.990489 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 329100us
I20260709 12:23:24.995838 16474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34826 (request call id 1251) took 1150 ms. Trace:
I20260709 12:23:24.998375 16474 rpcz_store.cc:276] 0709 12:23:23.844997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:24.917436 (+1072439us) service_pool.cc:224] Handling call
0709 12:23:24.995482 (+ 78046us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:25.000712 16514 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:25.001299 16469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1239) took 1199 ms. Trace:
I20260709 12:23:25.001865 16469 rpcz_store.cc:276] 0709 12:23:23.801764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:25.000197 (+1198433us) service_pool.cc:224] Handling call
0709 12:23:25.000611 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:25.000853 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:25.000891 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:25.000963 (+    72us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:23:25.001029 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:25.001104 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:25.001229 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:25.012612 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:25.004683 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:25.014783 16549 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:25.003798 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:24.979759 19816 raft_consensus.cc:3055] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:24.979251 19978 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:24.998529 19787 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.974269 19819 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:24.972311 19937 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:24.999658 19969 raft_consensus.cc:697] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 12 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:25.024649 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:25.008241 19806 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:25.029356 16550 raft_consensus.cc:1275] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:23:25.033572 19920 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:25.034236 19932 consensus_queue.cc:237] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:25.037424 16474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1240) took 1235 ms. Trace:
I20260709 12:23:24.993639 19917 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:25.040856 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:25.026015 19940 consensus_meta.cc:287] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.784s	user 0.016s	sys 0.000s
I20260709 12:23:25.057884 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:25.046653 17266 raft_consensus.cc:1275] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:23:25.046962 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:25.048350 16474 rpcz_store.cc:276] 0709 12:23:23.802237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:25.002108 (+1199871us) service_pool.cc:224] Handling call
0709 12:23:25.036618 (+ 34510us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:25.036902 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:25.036945 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:25.037013 (+    68us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:25.037077 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:25.037162 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:25.037325 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:25.054703 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:25.019719 19816 raft_consensus.cc:740] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:25.060915 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:24.999442 19928 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:25.057206 16543 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:23:25.085011 16550 raft_consensus.cc:1275] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:25.078653 16395 raft_consensus.cc:2393] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 11: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:25.078673 16469 raft_consensus.cc:1275] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:23:25.086212 16469 pending_rounds.cc:85] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 4
I20260709 12:23:25.086475 16469 pending_rounds.cc:107] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:23:25.086747 16469 raft_consensus.cc:2889] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:25.079742 16549 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:25.087693 16402 raft_consensus.cc:1240] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:25.089956 16477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1191) took 1542 ms. Trace:
I20260709 12:23:25.081631 17255 raft_consensus.cc:1275] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:25.082350 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:25.085593 16397 raft_consensus.cc:1240] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:23:25.094895 16401 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 6.
I20260709 12:23:25.090540 16477 rpcz_store.cc:276] 0709 12:23:23.547557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:23.552030 (+  4473us) service_pool.cc:224] Handling call
0709 12:23:23.552291 (+   261us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:23.787515 (+235224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:23.787560 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:23.788095 (+   535us) log.cc:844] Serialized 26 byte log entry
0709 12:23:23.979237 (+191142us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:23.979302 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:25.077802 (+1098500us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b540080b6d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:25.077856 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:25.077903 (+    47us) raft_consensus.cc:1685] finished
0709 12:23:25.077916 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:25.089854 (+ 11938us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5714,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":121,"spinlock_wait_cycles":539865344,"thread_start_us":190674,"threads_started":1,"wal-append.queue_time_us":663459}
I20260709 12:23:25.097249 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:25.099871 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:25.017940 19914 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:25.061035 19806 raft_consensus.cc:697] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 10 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:25.110774 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:25.101626 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:25.111493 19806 consensus_queue.cc:237] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:25.112598 19914 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:25.101740 16469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1241) took 1299 ms. Trace:
I20260709 12:23:25.057116 19943 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.113740 16469 rpcz_store.cc:276] 0709 12:23:23.802506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:25.042627 (+1240121us) service_pool.cc:224] Handling call
0709 12:23:25.078173 (+ 35546us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:25.101487 (+ 23314us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:25.101684 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:25.114331 19943 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:25.115105 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:25.115684 16469 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:25.117713 19928 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:25.118530 19969 consensus_queue.cc:237] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:25.099047 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:25.122768 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:25.123697 16549 raft_consensus.cc:2410] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:23:25.103680 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:25.127074 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:25.111500 16401 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:25.104672 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:25.056457 19908 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.107537 19816 consensus_queue.cc:260] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:25.109652 16367 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:25.077527 19917 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:25.042907 19787 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:25.140815 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 183405us
I20260709 12:23:25.142002 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:23:25.142786 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 171392us
I20260709 12:23:25.139264 16367 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:25.152930 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:25.153589 17269 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14.
I20260709 12:23:25.129109 16477 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:34826
I20260709 12:23:25.166930 16477 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 12:23:25.167229 16441 leader_election.cc:336] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491): Timed out: RequestConsensusVote RPC to 127.5.58.4:44491 timed out after 1.595s (SENT)
I20260709 12:23:25.172003 16477 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:25.144404 19891 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:25.177686 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:25.190220 17255 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:25.168917 19816 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:25.185883 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:25.194368 17269 raft_consensus.cc:2368] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12.
I20260709 12:23:25.129046 19928 consensus_queue.cc:1048] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
W20260709 12:23:25.187161 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 47960.9us
W20260709 12:23:25.185104 19994 log.cc:927] Time spent T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:23:25.135356 19908 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:25.170511 19943 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:25.198364 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:25.205529 16469 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
I20260709 12:23:25.218449 16514 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:25.192315 19849 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:25.192534 19891 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:25.220892 16367 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:25.203914 19914 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:25.239459 16401 raft_consensus.cc:1275] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 12:23:25.240124 16401 pending_rounds.cc:85] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 6
I20260709 12:23:25.243260 16401 pending_rounds.cc:107] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:23:25.243644 16401 raft_consensus.cc:2889] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:25.243497 16477 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:25.247512 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:25.224920 19941 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:25.255630 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:25.257289 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:25.257444 16399 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:23:25.257967 17264 raft_consensus.cc:2376] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10. Re-sending same reply.
I20260709 12:23:25.224354 19849 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:25.258694 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:25.253985 16367 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:25.259634 17266 raft_consensus.cc:2393] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 11: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:25.228123 19891 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:25.267616 16469 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:25.271859 16477 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:25.275105 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:25.275429 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:25.277366 16469 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:23:25.278162 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:25.279269 16546 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37242 (request call id 1153) took 1873 ms (client timeout 1595 ms). Trace:
W20260709 12:23:25.279742 16546 rpcz_store.cc:269] 0709 12:23:23.406048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:23.406213 (+   165us) service_pool.cc:224] Handling call
0709 12:23:25.279058 (+1872845us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b54005eea54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:25.279195 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":709588352}
I20260709 12:23:25.282629 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:25.283257 16546 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 7.
I20260709 12:23:25.288905 16441 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:25.310099 16549 raft_consensus.cc:2410] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:23:25.311374 16441 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:25.278699 16514 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:25.228325 19943 consensus_queue.cc:1048] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:23:25.309551 19941 raft_consensus.cc:697] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:25.258729 19880 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:25.319686 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:25.324373 16549 raft_consensus.cc:1240] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 12:23:25.324895 16477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:34826 (request call id 1242) took 1470 ms. Trace:
W20260709 12:23:25.325707 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:25.325477 16477 rpcz_store.cc:276] 0709 12:23:23.826605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:25.097402 (+1270797us) service_pool.cc:224] Handling call
0709 12:23:25.273070 (+175668us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:25.262459 19849 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:25.242748 19914 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:25.326514 16441 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:25.272552 19833 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:25.265296 19819 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.334519 16441 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:25.273037 19937 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:25.337800 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:25.338999 19999 log.cc:927] Time spent T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:23:25.339278 16543 raft_consensus.cc:1240] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:23:25.338757 19941 consensus_queue.cc:237] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:25.330817 19920 consensus_queue.cc:1059] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 12:23:25.341883 19914 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:25.344781 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:25.346323 17143 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:25.347612 16550 raft_consensus.cc:2410] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:23:25.320801 19853 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:25.323693 19931 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:25.350490 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:25.354517 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:25.326184 19978 raft_consensus.cc:2804] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:25.355376 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:25.274371 19892 consensus_queue.cc:1048] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 12:23:25.352109 16441 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:25.327724 19880 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:25.366571 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:25.369498 16542 raft_consensus.cc:2368] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 10. Current term is 11.
I20260709 12:23:25.371330 16477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1247) took 1543 ms. Trace:
I20260709 12:23:25.374503 16477 rpcz_store.cc:276] 0709 12:23:23.828272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:25.353333 (+1525061us) service_pool.cc:224] Handling call
0709 12:23:25.370583 (+ 17250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:25.370849 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:25.370893 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:25.370961 (+    68us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:25.371015 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:25.371110 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:25.371252 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:25.373307 16441 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:25.343771 19898 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:25.377468 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 116528us
I20260709 12:23:25.379240 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:25.374126 16544 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:25.384258 16542 raft_consensus.cc:2410] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 12:23:25.380653 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:25.381870 19847 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:25.391117 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:25.392019 16551 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:25.354823 19858 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:25.337026 19833 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:25.343048 19865 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:25.357409 19920 consensus_queue.cc:1243] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer ef57beab0e1d40d585983873c55107e8 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:23:25.349933 19819 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:25.396600 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:25.377087 19880 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:25.408510 19833 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:25.621430 20003 log.cc:927] Time spent T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.244s	user 0.000s	sys 0.005s
I20260709 12:23:25.390223 19847 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:25.392508 19931 raft_consensus.cc:697] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 12 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:25.453121 19865 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:25.376225 19978 raft_consensus.cc:697] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:25.394372 19787 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:25.631873 16367 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:25.357949 19979 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:25.602818 17143 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:25.638264 19979 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:25.603073 20006 log.cc:927] Time spent T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.226s	user 0.010s	sys 0.000s
I20260709 12:23:25.603202 16474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1243) took 1776 ms. Trace:
I20260709 12:23:25.639387 16474 rpcz_store.cc:276] 0709 12:23:23.826981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:25.101179 (+1274198us) service_pool.cc:224] Handling call
0709 12:23:25.108956 (+  7777us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:25.109181 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:23:25.217212 (+108031us) log.cc:844] Serialized 14 byte log entry
0709 12:23:25.539256 (+322044us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:25.602886 (+ 63630us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:25.602949 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:25.603011 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:25.603146 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":245980,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":87,"raft.queue_time_us":13880,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":14263,"thread_start_us":429171,"threads_started":3,"wal-append.queue_time_us":396483}
I20260709 12:23:25.643220 16474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1252) took 1768 ms. Trace:
I20260709 12:23:25.603760 16548 raft_consensus.cc:1240] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:25.604744 19914 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.606217 19819 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:25.644920 19914 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:25.645303 19819 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:25.643900 16474 rpcz_store.cc:276] 0709 12:23:23.875067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:25.641744 (+1766677us) service_pool.cc:224] Handling call
0709 12:23:25.642384 (+   640us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:25.642687 (+   303us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:25.642734 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:25.642801 (+    67us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:25.642857 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:25.642940 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:25.643128 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:23:25.610493 19997 log.cc:927] Time spent T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.338s	user 0.009s	sys 0.001s
I20260709 12:23:25.395591 19898 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:25.646839 19898 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:25.647264 19898 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:25.647796 19914 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:25.616642 16545 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 11
W20260709 12:23:25.652920 19764 consensus_meta.cc:287] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.693s	user 0.017s	sys 0.000s
W20260709 12:23:25.617338 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:25.653391 19764 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:25.655689 19764 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:25.624585 19847 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:25.656740 20009 consensus_queue.cc:1048] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:23:25.659273 19757 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:25.661075 19978 consensus_queue.cc:237] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:25.665632 19911 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:25.670934 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:25.675387 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 44441.7us
I20260709 12:23:25.638929 19979 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:23:25.606258 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:25.608165 19908 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:25.609995 19917 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:25.680985 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 307721us
W20260709 12:23:25.700079 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 334761us
W20260709 12:23:25.681803 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 50423.4us
I20260709 12:23:25.702517 19853 raft_consensus.cc:697] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:25.702553 19764 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.703066 20012 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.704499 19764 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:25.705107 20012 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:25.707679 19764 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:25.707731 16472 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34826 (request call id 1246) took 1879 ms (client timeout 1698 ms). Trace:
W20260709 12:23:25.708482 16472 rpcz_store.cc:269] 0709 12:23:23.827943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:23.932814 (+104871us) service_pool.cc:224] Handling call
0709 12:23:25.707416 (+1774602us) spinlock_profiling.cc:243] Waited 371 ms on lock 0x7b540058c854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:25.707623 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":853417216}
I20260709 12:23:25.710067 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:25.710948 16472 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
W20260709 12:23:25.683036 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 51294.6us
I20260709 12:23:25.707840 20012 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:25.718575 16471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1237) took 2302 ms. Trace:
I20260709 12:23:25.730952 19837 raft_consensus.cc:3055] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:25.731701 19837 raft_consensus.cc:740] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:25.742247 19959 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:25.744542 19915 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:25.746152 16473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53186 (request call id 652) took 1282 ms. Trace:
I20260709 12:23:25.746791 16473 rpcz_store.cc:276] 0709 12:23:24.463958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:24.635588 (+171630us) service_pool.cc:224] Handling call
0709 12:23:25.745910 (+1110322us) spinlock_profiling.cc:243] Waited 55.4 ms on lock 0x7b5400594554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:25.746072 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":127318784}
I20260709 12:23:25.719401 16471 rpcz_store.cc:276] 0709 12:23:23.415743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:23.415881 (+   138us) service_pool.cc:224] Handling call
0709 12:23:23.416265 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:23.657591 (+241326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:23.657626 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:23.668822 (+ 11196us) log.cc:844] Serialized 26 byte log entry
0709 12:23:23.956999 (+288177us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:23.957068 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:25.717447 (+1760379us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b5400a69ad4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:25.717524 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:25.717599 (+    75us) raft_consensus.cc:1685] finished
0709 12:23:25.717624 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:25.718341 (+   717us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25882,"raft.queue_time_us":29918,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":23355,"spinlock_wait_cycles":494322944,"thread_start_us":241135,"threads_started":1,"wal-append.queue_time_us":489935,"wal-append.run_cpu_time_us":3230,"wal-append.run_wall_time_us":1536407}
W20260709 12:23:25.751873 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:25.753983 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:25.754887 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 52006us
I20260709 12:23:25.756875 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:25.757305 16367 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:25.758384 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:25.752977 17144 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:25.672639 19865 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:25.762666 16367 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:25.764811 19779 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:25.765345 16366 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
W20260709 12:23:25.700711 19816 consensus_meta.cc:287] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.503s	user 0.012s	sys 0.001s
I20260709 12:23:25.768784 17144 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:25.651947 19891 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:25.700088 19911 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:25.681167 19943 consensus_queue.cc:1048] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.089s
W20260709 12:23:25.782444 16468 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34826 (request call id 1244) took 1955 ms (client timeout 1884 ms). Trace:
W20260709 12:23:25.782908 16468 rpcz_store.cc:269] 0709 12:23:23.827267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:24.245719 (+418452us) service_pool.cc:224] Handling call
0709 12:23:25.782219 (+1536500us) spinlock_profiling.cc:243] Waited 89 ms on lock 0x7b5400b2fcd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:25.782369 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":204650752}
W20260709 12:23:25.783468 19987 log.cc:927] Time spent T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 1.084s	user 0.002s	sys 0.001s
I20260709 12:23:25.784157 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:25.774001 19785 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:25.784970 19785 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:25.785328 19785 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:25.790802 17258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 980) took 2294 ms. Trace:
I20260709 12:23:25.791420 17258 rpcz_store.cc:276] 0709 12:23:23.496181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:23.855515 (+359334us) service_pool.cc:224] Handling call
0709 12:23:23.856055 (+   540us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:23.878696 (+ 22641us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:23.878949 (+   253us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:25.272653 (+1393704us) spinlock_profiling.cc:243] Waited 30.5 ms on lock 0x7b2c00000708. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:23:25.315415 (+ 42762us) log.cc:844] Serialized 52 byte log entry
0709 12:23:25.601179 (+285764us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:23:25.601239 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:25.601356 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:25.790377 (+189021us) raft_consensus.cc:1685] finished
0709 12:23:25.790405 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:25.790715 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44238,"raft.queue_time_us":59960,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":16997,"spinlock_wait_cycles":70164864,"thread_start_us":250216,"threads_started":1,"wal-append.queue_time_us":363096}
I20260709 12:23:25.752717 19995 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.661958 19849 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:25.779770 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:25.780113 19911 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:25.794644 16475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 642) took 3504 ms. Trace:
W20260709 12:23:25.774272 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 361253us
I20260709 12:23:25.803182 17143 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:25.751925 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:25.801800 16475 rpcz_store.cc:276] 0709 12:23:22.290416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:22.730640 (+440224us) service_pool.cc:224] Handling call
0709 12:23:24.743053 (+2012413us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:24.743737 (+   684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:25.794387 (+1050650us) spinlock_profiling.cc:243] Waited 2.01 s on lock 0x7b5400974350. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:25.794560 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4627357056}
I20260709 12:23:25.778779 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:25.692378 16545 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:25.805994 16472 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:25.770205 19917 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:25.765372 19937 consensus_queue.cc:1059] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:23:25.766415 19908 consensus_queue.cc:1048] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 12:23:25.807194 16471 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:25.797694 19779 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:25.805176 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:25.825220 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:25.847785 16475 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:25.843354 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:25.850878 16471 raft_consensus.cc:2410] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:23:25.772110 19787 consensus_queue.cc:1048] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:23:25.804072 19837 consensus_queue.cc:260] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:25.860898 19937 raft_consensus.cc:3055] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:25.861598 19937 raft_consensus.cc:740] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:25.846843 19779 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:25.867269 19779 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:25.874166 16441 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.4:44491 timed out after 1.731s (SENT) [suppressed 20 similar messages]
I20260709 12:23:25.778803 19819 consensus_queue.cc:1048] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 12:23:25.893005 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:25.903141 16471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1256) took 1646 ms. Trace:
I20260709 12:23:25.905611 16471 rpcz_store.cc:276] 0709 12:23:24.257234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:25.898733 (+1641499us) service_pool.cc:224] Handling call
0709 12:23:25.899204 (+   471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:25.902615 (+  3411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:25.902661 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:25.902731 (+    70us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:25.902783 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:25.902873 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:25.903053 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:25.911247 16471 raft_consensus.cc:1240] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:25.834760 19585 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:25.856124 19837 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:25.852022 20013 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.806351 19995 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:25.862859 19937 consensus_queue.cc:260] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:25.991863 19937 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:25.997500 16474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1254) took 1936 ms. Trace:
I20260709 12:23:25.997983 16474 rpcz_store.cc:276] 0709 12:23:24.061389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:25.648008 (+1586619us) service_pool.cc:224] Handling call
0709 12:23:25.648539 (+   531us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:25.649361 (+   822us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:25.649402 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:25.650046 (+   644us) log.cc:844] Serialized 26 byte log entry
0709 12:23:25.774781 (+124735us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:25.774834 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:25.774905 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:25.997249 (+222344us) raft_consensus.cc:1685] finished
0709 12:23:25.997270 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:25.997427 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":141,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":138,"thread_start_us":123483,"threads_started":1,"wal-append.queue_time_us":269678}
I20260709 12:23:25.794515 19853 consensus_queue.cc:237] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:25.808107 19931 consensus_queue.cc:237] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:25.963655 16471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 648) took 1506 ms. Trace:
I20260709 12:23:26.004205 16471 rpcz_store.cc:276] 0709 12:23:24.457730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:25.910551 (+1452821us) service_pool.cc:224] Handling call
0709 12:23:25.910978 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:25.963365 (+ 52387us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:25.963591 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:26.005097 19847 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:26.005822 16471 raft_consensus.cc:1240] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 11. Current term is 12. Ops: [7.5-11.7]
I20260709 12:23:26.006567 16471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 654) took 1542 ms. Trace:
I20260709 12:23:26.006889 16471 rpcz_store.cc:276] 0709 12:23:24.464634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:26.005126 (+1540492us) service_pool.cc:224] Handling call
0709 12:23:26.005547 (+   421us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:23:26.006276 (+   729us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:26.006508 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:26.007359 19847 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:26.008621 16471 raft_consensus.cc:1275] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:23:26.008766 19785 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:26.009505 16471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1196) took 1185 ms. Trace:
I20260709 12:23:26.009869 16471 rpcz_store.cc:276] 0709 12:23:24.823785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:26.007776 (+1183991us) service_pool.cc:224] Handling call
0709 12:23:26.008253 (+   477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:26.009268 (+  1015us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:26.009455 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:25.984755 19819 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:25.984433 19585 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:26.012869 19585 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:25.988474 20013 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:26.013981 19699 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:26.014891 20013 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:25.990957 19995 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:26.011013 19785 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:26.015766 19865 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:26.019754 16470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34840 (request call id 1201) took 1194 ms. Trace:
I20260709 12:23:26.020424 16470 rpcz_store.cc:276] 0709 12:23:24.825520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:24.854507 (+ 28987us) service_pool.cc:224] Handling call
0709 12:23:26.019492 (+1164985us) spinlock_profiling.cc:243] Waited 78.2 ms on lock 0x7b54002e5c54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:26.019674 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":179835136}
I20260709 12:23:26.022387 19849 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:26.024083 20016 log.cc:927] Time spent T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:23:26.026175 16476 raft_consensus.cc:1240] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:26.033069 19891 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:26.040469 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:26.040649 16477 raft_consensus.cc:1240] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:26.040619 16471 raft_consensus.cc:1240] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:26.046346 16474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 653) took 1582 ms. Trace:
I20260709 12:23:26.047199 16474 rpcz_store.cc:276] 0709 12:23:24.464300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:25.999690 (+1535390us) service_pool.cc:224] Handling call
0709 12:23:26.000113 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:26.000319 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:26.045946 (+ 45627us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:26.046002 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:26.046071 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:26.046148 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:26.046282 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:25.983608 19779 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:26.019572 19833 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:26.064323 19981 log.cc:927] Time spent T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 1.443s	user 0.004s	sys 0.001s
I20260709 12:23:26.068729 19915 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:26.069629 19915 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:26.070451 20019 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.071041 20019 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:26.073294 19915 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:26.076078 19915 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.077011 19915 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:26.079699 19915 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:26.081416 19858 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:26.082016 19858 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:26.084237 19858 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:26.090133 19898 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:26.095822 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:26.117525 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:26.113323 16442 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:26.138748 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 344570us
W20260709 12:23:26.140446 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 375143us
I20260709 12:23:26.145892 17271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 939) took 2642 ms. Trace:
I20260709 12:23:26.146306 16395 raft_consensus.cc:2410] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:23:26.139436 20019 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:26.147919 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 344792us
I20260709 12:23:26.153667 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:26.154825 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:26.155963 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:26.157582 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:26.158205 17264 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:26.158464 16476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1238) took 2362 ms. Trace:
I20260709 12:23:26.159026 16476 rpcz_store.cc:276] 0709 12:23:23.796266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:24.938397 (+1142131us) service_pool.cc:224] Handling call
0709 12:23:26.025702 (+1087305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:26.026738 (+  1036us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:26.158201 (+131463us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54002e5c50. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:26.158377 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2456583552}
I20260709 12:23:26.160156 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:26.161231 17143 leader_election.cc:400] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 10. Current term is 11.
I20260709 12:23:26.161602 17143 leader_election.cc:403] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:23:26.161727 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:26.162539 17266 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:26.162894 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 400298us
I20260709 12:23:26.165653 19995 consensus_queue.cc:1059] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:26.166492 19891 consensus_queue.cc:1048] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:26.166765 19995 consensus_queue.cc:1048] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:26.148003 16515 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:26.143663 16366 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
W20260709 12:23:26.168696 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:26.169539 19755 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 10. Current term is 11.
I20260709 12:23:26.169943 16477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1249) took 2341 ms. Trace:
I20260709 12:23:26.170308 16515 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:26.171172 19764 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:26.171312 16515 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:26.149704 17271 rpcz_store.cc:276] 0709 12:23:23.497564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:23.900635 (+403071us) service_pool.cc:224] Handling call
0709 12:23:23.900977 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:23.936358 (+ 35381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:23.936394 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:23.955937 (+ 19543us) log.cc:844] Serialized 26 byte log entry
0709 12:23:24.305037 (+349100us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:24.305083 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:24.305159 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:26.064009 (+1758850us) spinlock_profiling.cc:243] Waited 53.3 ms on lock 0x7b240051e618. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872c229c4 00007f2872c20acb 00007f2872ddbdc9 00007f2872ddba55 00007f2872ddb09d 00007f2875a0d10e 00007f2875a2488c 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221
0709 12:23:26.116199 (+ 52190us) raft_consensus.cc:1685] finished
0709 12:23:26.116233 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:26.139860 (+ 23627us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34959,"raft.queue_time_us":35523,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":21407,"spinlock_wait_cycles":122523904,"thread_start_us":278249,"threads_started":1,"wal-append.queue_time_us":487555}
I20260709 12:23:26.174564 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:26.172228 17143 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:26.170324 16477 rpcz_store.cc:276] 0709 12:23:23.828936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:25.391029 (+1562093us) service_pool.cc:224] Handling call
0709 12:23:25.606088 (+215059us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:26.041687 (+435599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:26.169689 (+128002us) spinlock_profiling.cc:243] Waited 434 ms on lock 0x7b54002729d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:26.169883 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":998922112}
I20260709 12:23:26.178100 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:26.178939 17255 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:26.179931 16515 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:26.166242 16367 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:26.182556 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:26.183507 17255 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:26.139834 19785 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.138476 19699 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.211750 16367 leader_election.cc:400] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12.
I20260709 12:23:26.212221 16367 leader_election.cc:403] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:23:26.187136 17266 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
W20260709 12:23:26.194185 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 50611.2us
I20260709 12:23:26.146931 19819 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:26.173063 19764 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:26.208441 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:26.186758 17143 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:26.215510 16367 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:26.100858 19880 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:26.172994 19915 raft_consensus.cc:2804] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:26.228929 19915 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:26.229382 19915 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:26.217181 16366 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:26.196034 17144 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:26.178543 19970 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.232990 19865 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:26.233907 16366 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:26.211473 19983 raft_consensus.cc:2764] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:23:26.236578 19585 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:26.214623 19785 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:26.192471 19730 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:26.216392 19699 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:26.237356 19585 raft_consensus.cc:697] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:26.232862 19970 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:26.140930 19979 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:26.240352 17258 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:26.249084 16545 raft_consensus.cc:1240] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:26.271788 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:26.276302 16545 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:26.269582 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:26.279453 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:26.224975 19764 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:26.280264 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:26.280809 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:26.284365 16549 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:23:26.256203 19730 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:26.266757 19943 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:26.253364 16401 raft_consensus.cc:1240] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:26.253561 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:26.254452 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:26.313197 17264 raft_consensus.cc:2393] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 8: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:26.254822 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:26.255156 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:26.255534 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:26.255826 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 59876.8us
I20260709 12:23:26.314599 16394 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:26.240360 16366 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:26.257804 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 69943.9us
I20260709 12:23:26.151597 19757 consensus_queue.cc:1059] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 12:23:26.321295 17258 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:23:26.321583 16550 raft_consensus.cc:1275] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:23:26.213449 19891 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:26.236033 19892 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:26.215827 19995 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:26.213982 19849 raft_consensus.cc:3055] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:26.268316 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:26.222869 20009 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12.
W20260709 12:23:26.275504 16226 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53626 (request call id 127) took 25176 ms (client timeout 15000 ms). Trace:
W20260709 12:23:26.249086 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 62408.3us
I20260709 12:23:26.294430 16545 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:26.261862 19926 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:23:26.307281 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:26.251751 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:26.315817 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 75609.5us
I20260709 12:23:26.333050 16400 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
I20260709 12:23:26.333534 17144 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:26.280704 19787 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:26.239588 19819 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:26.339682 16551 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:26.284245 19764 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:26.341262 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: true dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:23:26.375990 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 60086.1us
I20260709 12:23:26.320956 19806 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:26.391592 16401 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:26.312180 19730 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:26.392086 16515 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:26.393215 16395 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
I20260709 12:23:26.410423 16394 raft_consensus.cc:1275] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:23:26.407531 16442 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:26.336259 19892 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:26.334129 19908 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:26.373724 16469 raft_consensus.cc:1240] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 LEADER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 8. Current term is 9. Ops: []
W20260709 12:23:26.432222 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 98797.6us
I20260709 12:23:26.453759 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:26.460754 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 53319.6us
I20260709 12:23:26.462060 16442 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:26.463927 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:26.464634 16398 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:23:26.312536 19898 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:26.336812 19891 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:26.304557 19917 raft_consensus.cc:697] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:26.342681 19755 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.342593 19849 raft_consensus.cc:740] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:26.339387 19995 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:26.437000 19892 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:26.392524 16224 catalog_manager.cc:5697] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 7 to 8, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 8 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:23:26.434365 19787 raft_consensus.cc:697] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:26.468765 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:26.502345 16476 raft_consensus.cc:1240] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:23:26.502743 16394 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:26.470841 16551 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:26.477025 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:26.393934 20022 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.473572 17144 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:26.336865 19943 raft_consensus.cc:697] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:26.523927 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:26.492566 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:26.525089 16544 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:23:26.430812 20028 raft_consensus.cc:3055] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:26.483596 19891 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:26.391909 16366 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:26.434533 20019 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:26.434508 19978 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:26.431875 20013 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.322937 19932 consensus_queue.cc:1059] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.165s
I20260709 12:23:26.498255 16403 raft_consensus.cc:2393] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 12: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:26.471113 16545 raft_consensus.cc:1240] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:26.501029 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:26.376083 20021 consensus_queue.cc:1059] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.153s
W20260709 12:23:26.520958 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 47533.3us
I20260709 12:23:26.474908 19858 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:26.468827 19908 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:26.527734 16469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1245) took 2700 ms. Trace:
I20260709 12:23:26.509099 16224 catalog_manager.cc:5697] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 9 to 10, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 10 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:26.563021 19849 consensus_queue.cc:260] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:26.563680 16224 catalog_manager.cc:5697] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:23:26.564688 19849 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:26.565547 16224 catalog_manager.cc:5697] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:23:26.491197 20023 consensus_queue.cc:1059] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.084s
I20260709 12:23:26.567322 20023 raft_consensus.cc:3055] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:26.567158 16224 catalog_manager.cc:5697] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 6 to 7, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 7 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:23:26.567776 20023 raft_consensus.cc:740] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:23:26.491592 19969 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.009s	sys 0.045s
W20260709 12:23:26.568544 19969 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.009s	sys 0.045s
I20260709 12:23:26.568571 20033 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.484261 20027 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.569352 20033 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:26.569554 20027 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:26.570988 19819 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:26.571882 19819 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:26.493819 19926 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:26.572211 19819 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:26.490036 19779 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:26.495637 19833 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:26.529160 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:26.437387 19585 consensus_queue.cc:237] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:26.504607 19853 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.006s	sys 0.032s
W20260709 12:23:26.573691 19853 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.006s	sys 0.033s
W20260709 12:23:26.541977 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 150136us
I20260709 12:23:26.573897 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:23:26.548384 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:26.523022 19931 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.010s	sys 0.031s
W20260709 12:23:26.574855 19931 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.011s	sys 0.032s
I20260709 12:23:26.574442 19943 consensus_queue.cc:237] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:26.574959 19908 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:26.575323 19880 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:26.575676 16546 raft_consensus.cc:1240] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 12 LEADER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:26.497959 19995 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:26.574476 19917 consensus_queue.cc:237] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:26.576548 19970 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:26.576735 16475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1255) took 2480 ms. Trace:
I20260709 12:23:26.577170 19970 raft_consensus.cc:3055] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:26.514171 19941 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:26.577697 19970 raft_consensus.cc:740] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:26.577396 19787 consensus_queue.cc:237] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:26.578001 19941 raft_consensus.cc:697] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 10 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:26.578755 19970 consensus_queue.cc:260] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:26.572503 19926 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:26.511183 19898 consensus_queue.cc:1048] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 12:23:26.580767 19979 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:26.554921 19978 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:26.582015 19978 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:26.494402 19755 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:26.558525 20009 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:26.572620 19785 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:26.578931 19941 consensus_queue.cc:237] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:26.577171 16475 rpcz_store.cc:276] 0709 12:23:24.096894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:25.848721 (+1751827us) service_pool.cc:224] Handling call
0709 12:23:25.849195 (+   474us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:25.849702 (+   507us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:23:25.911946 (+ 62244us) log.cc:844] Serialized 14 byte log entry
0709 12:23:26.287217 (+375271us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:26.304446 (+ 17229us) log.cc:844] Serialized 26 byte log entry
0709 12:23:26.304651 (+   205us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:23:26.304690 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:26.304770 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:26.571947 (+267177us) raft_consensus.cc:1685] finished
0709 12:23:26.571969 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:26.576672 (+  4703us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27826,"prepare.queue_time_us":62351,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":67,"raft.queue_time_us":279,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":28545,"thread_start_us":340019,"threads_started":1,"wal-append.queue_time_us":600949}
I20260709 12:23:26.562584 16469 rpcz_store.cc:276] 0709 12:23:23.827612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:25.332511 (+1504899us) service_pool.cc:224] Handling call
0709 12:23:25.335055 (+  2544us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:26.467881 (+1132826us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:26.527143 (+ 59262us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b540080b1d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:26.527545 (+   402us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2388065152}
I20260709 12:23:26.596359 16366 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:26.591900 16224 catalog_manager.cc:5697] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 5 to 7, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 7 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:26.530506 20028 raft_consensus.cc:740] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:26.609066 16394 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:23:26.468473 20029 consensus_queue.cc:1059] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 12:23:26.613957 16366 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:26.613957 16367 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:26.616607 17144 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:26.619473 16224 catalog_manager.cc:5697] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 5 to 7, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 7 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:23:26.572679 19819 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:26.627976 16366 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
W20260709 12:23:26.609737 19816 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.042s	sys 0.037s
I20260709 12:23:26.630793 16367 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:26.632215 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:26.632763 16366 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:26.638070 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:26.639137 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:26.639977 16402 raft_consensus.cc:2410] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:23:26.643365 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:26.644152 16394 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
I20260709 12:23:26.610917 19699 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:26.377689 16226 rpcz_store.cc:269] 0709 12:23:01.099030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:01.099304 (+   274us) service_pool.cc:224] Handling call
0709 12:23:26.275285 (+25175981us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:19.261234 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:19.311090 (+ 49856us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:19.311146 (+    56us) write_op.cc:435] Acquired schema lock
0709 12:23:19.311166 (+    20us) tablet.cc:662] Decoding operations
0709 12:23:19.312858 (+  1692us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:19.313019 (+   161us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:19.313060 (+    41us) tablet.cc:685] Acquiring locks for 10 operations
0709 12:23:19.314022 (+   962us) tablet.cc:701] Row locks acquired
0709 12:23:19.314081 (+    59us) write_op.cc:260] PREPARE: finished
0709 12:23:19.314245 (+   164us) write_op.cc:270] Start()
0709 12:23:19.314669 (+   424us) write_op.cc:276] Timestamp: P: 1783599799314179 usec, L: 0
0709 12:23:19.314718 (+    49us) op_driver.cc:348] REPLICATION: starting
0709 12:23:19.315504 (+   786us) log.cc:844] Serialized 3943 byte log entry
0709 12:23:19.731174 (+415670us) spinlock_profiling.cc:243] Waited 39.7 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:23:20.134506 (+403332us) op_driver.cc:464] REPLICATION: finished
0709 12:23:20.631145 (+496639us) spinlock_profiling.cc:243] Waited 3.74 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2872d7f0ac 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f28798bfef0
0709 12:23:21.767363 (+1136218us) spinlock_profiling.cc:243] Waited 21.8 ms on lock 0x7b18003a6f00. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:23:21.767609 (+   246us) write_op.cc:301] APPLY: starting
0709 12:23:21.767727 (+   118us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:21.768009 (+   282us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:21.768048 (+    39us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:21.769512 (+  1464us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:21.792844 (+ 23332us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 12:23:21.812456 (+ 19612us) write_op.cc:312] APPLY: finished
0709 12:23:21.812887 (+   431us) log.cc:844] Serialized 76 byte log entry
0709 12:23:21.892781 (+ 79894us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:21.892937 (+   156us) write_op.cc:454] Released schema lock
0709 12:23:21.899273 (+  6336us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:21.904377 (+  5104us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:21.904427 (+    50us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":503311,"reported_tablets":43,"child_traces":[["op",{"apply.queue_time_us":772353,"apply.run_cpu_time_us":14240,"apply.run_wall_time_us":135935,"num_ops":10,"prepare.queue_time_us":108740,"prepare.run_cpu_time_us":56759,"prepare.run_wall_time_us":368281,"raft.queue_time_us":167288,"raft.run_cpu_time_us":26477,"raft.run_wall_time_us":882420,"replication_time_us":723874,"spinlock_wait_cycles":150163328,"thread_start_us":1221466,"threads_started":5,"wal-append.queue_time_us":399141,"wal-append.run_cpu_time_us":22995,"wal-append.run_wall_time_us":2551664}]]}
I20260709 12:23:26.650517 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:26.652292 16475 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:26.655349 16224 catalog_manager.cc:5697] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
W20260709 12:23:26.656154 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 394563us
I20260709 12:23:26.626444 20009 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:23:26.634898 19816 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.042s	sys 0.038s
I20260709 12:23:26.530650 20022 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:26.631872 19880 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:26.605643 20013 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:26.676558 16546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 687) took 1333 ms. Trace:
W20260709 12:23:26.662307 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:26.677107 16546 rpcz_store.cc:276] 0709 12:23:25.343297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:25.343557 (+   260us) service_pool.cc:224] Handling call
0709 12:23:25.343908 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:26.576095 (+1232187us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:26.676219 (+100124us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400a80cd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:26.676486 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2832345856}
I20260709 12:23:26.665416 16224 catalog_manager.cc:5697] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:23:26.666046 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:26.679324 16471 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:26.667574 16476 raft_consensus.cc:1275] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:26.637754 19779 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:26.669046 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:26.674609 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:26.642383 20029 consensus_queue.cc:1243] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer 610f4feda11942739236f2b121a4b9c1 log is divergent from this leader: its last log entry 7.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:23:26.651469 19487 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:26.680995 16470 raft_consensus.cc:2376] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10. Re-sending same reply.
I20260709 12:23:26.637844 20025 raft_consensus.cc:2749] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:26.682667 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:26.692204 16470 raft_consensus.cc:2393] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 10: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:26.681226 16472 raft_consensus.cc:2393] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 9: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:26.684092 16476 raft_consensus.cc:1275] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:23:26.696873 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:26.666072 19996 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:26.663887 20009 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:26.708083 16474 raft_consensus.cc:1275] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:23:26.669824 19979 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:26.639202 19894 consensus_queue.cc:1059] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:23:26.672621 19880 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:26.704545 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:26.706758 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:26.681953 19779 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:26.726346 16442 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:26.727774 16470 raft_consensus.cc:2410] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:23:26.712162 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:26.713929 16224 catalog_manager.cc:5697] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 3 to 7, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 7 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:23:26.710224 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:26.731346 16472 raft_consensus.cc:2393] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 9: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:26.714248 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:26.709281 16514 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:26.731876 16471 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:23:26.732851 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:26.733649 16470 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 11: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:26.733382 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:26.637653 20023 consensus_queue.cc:260] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:26.735966 16471 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:26.702919 20033 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:26.735339 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 343313us
I20260709 12:23:26.730229 16442 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:26.742849 16401 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
W20260709 12:23:26.749692 20034 log.cc:927] Time spent T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:23:26.738492 19785 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.750367 20015 raft_consensus.cc:2764] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:23:26.750988 16473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 651) took 2287 ms. Trace:
I20260709 12:23:26.750758 19785 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:26.751936 20015 consensus_queue.cc:1059] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:26.751446 16473 rpcz_store.cc:276] 0709 12:23:24.463639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:25.916554 (+1452915us) service_pool.cc:224] Handling call
0709 12:23:25.917063 (+   509us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:25.917500 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:25.917541 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:25.929941 (+ 12400us) log.cc:844] Serialized 26 byte log entry
0709 12:23:26.284004 (+354063us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:26.284056 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:26.284140 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:26.750782 (+466642us) raft_consensus.cc:1685] finished
0709 12:23:26.750803 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:26.750928 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1506,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":13001,"thread_start_us":353685,"threads_started":1,"wal-append.queue_time_us":674870}
I20260709 12:23:26.755051 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:26.715916 20027 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:26.755723 16473 raft_consensus.cc:2376] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12. Re-sending same reply.
I20260709 12:23:26.756633 19853 raft_consensus.cc:3055] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:26.757045 19853 raft_consensus.cc:740] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:26.757886 19853 consensus_queue.cc:260] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:26.759370 19926 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:26.760110 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:26.761248 16473 raft_consensus.cc:2393] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 11: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:26.763211 20049 consensus_queue.cc:1048] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:23:26.764743 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 55558.4us
I20260709 12:23:26.765365 20049 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:26.682462 19833 consensus_queue.cc:1048] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:23:26.713482 20009 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:26.762567 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:26.767401 16514 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:26.774006 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:26.775703 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:26.768289 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:26.776651 16472 raft_consensus.cc:2393] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 12: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:26.769286 16471 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:23:26.767292 16442 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:26.765913 20049 raft_consensus.cc:697] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 10 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:26.777520 19847 consensus_queue.cc:1048] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:23:26.777814 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:26.778841 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:26.778434 20049 consensus_queue.cc:237] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:26.780915 19837 consensus_meta.cc:287] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.796s	user 0.010s	sys 0.001s
W20260709 12:23:26.781469 20037 log.cc:927] Time spent T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 12:23:26.781826 16468 raft_consensus.cc:2410] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:23:26.783659 16515 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:26.784555 20042 raft_consensus.cc:2804] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:26.784929 20042 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:26.792591 19932 consensus_queue.cc:1059] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:23:26.781858 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:26.793712 19932 consensus_queue.cc:1243] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Peer 52ab50b0237346c1a13ede947f6a63c2 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:23:26.734333 19898 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:26.794648 19935 consensus_queue.cc:1048] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:26.785205 16515 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:26.796726 16515 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:26.797632 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:26.787319 17144 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:26.799996 16394 raft_consensus.cc:2410] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:23:26.800637 16545 raft_consensus.cc:1217] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Deduplicated request from leader. Original: 6.4->[8.5-8.5]   Dedup: 8.5->[]
I20260709 12:23:26.801290 16545 raft_consensus.cc:1240] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:23:26.801610 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:26.720639 19880 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:26.785974 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:26.729857 19779 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:23:26.817406 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:26.805884 17142 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:26.724041 19979 raft_consensus.cc:697] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 10 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:26.910329 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:26.829807 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:26.916255 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:26.916599 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:26.770533 19894 raft_consensus.cc:3055] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:26.917586 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:26.824747 16367 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:26.904516 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:26.806903 16224 catalog_manager.cc:5697] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
W20260709 12:23:26.834903 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:26.826964 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:26.927837 17264 raft_consensus.cc:2410] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:23:26.849112 16472 raft_consensus.cc:1240] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:26.807626 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:26.788210 20019 raft_consensus.cc:3055] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:26.827260 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:26.918744 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:26.905414 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:26.882274 17255 raft_consensus.cc:1240] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:23:26.991083 17270 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:26.921046 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:26.991377 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:26.992461 16551 raft_consensus.cc:2410] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:23:26.992614 16547 raft_consensus.cc:2410] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
W20260709 12:23:26.909149 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 103367us
I20260709 12:23:26.806527 16546 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:26.818446 19699 raft_consensus.cc:2764] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:23:26.785266 20042 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:26.931161 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:26.930721 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:26.996577 16224 catalog_manager.cc:5697] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 8 to 9, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 9 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:23:26.961880 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:26.960647 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:26.973796 16468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34840 (request call id 1207) took 1262 ms. Trace:
I20260709 12:23:26.993882 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:27.008472 16468 rpcz_store.cc:276] 0709 12:23:25.711419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:25.783896 (+ 72477us) service_pool.cc:224] Handling call
0709 12:23:26.973391 (+1189495us) spinlock_profiling.cc:243] Waited 997 ms on lock 0x7b54005e6354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:26.973725 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2292345216}
I20260709 12:23:26.996265 16471 raft_consensus.cc:1240] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 11. Current term is 12. Ops: [7.5-11.7]
I20260709 12:23:27.010833 16468 raft_consensus.cc:1275] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:26.974886 16469 raft_consensus.cc:1275] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:23:27.011757 16469 pending_rounds.cc:85] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 4
I20260709 12:23:27.012113 16469 pending_rounds.cc:107] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:23:27.012509 16469 raft_consensus.cc:2889] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:26.786855 19969 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:27.017956 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:26.791551 19959 raft_consensus.cc:3055] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:23:26.985869 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:26.882476 20029 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:26.905148 19898 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.832506 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:26.993608 17142 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:26.811097 19983 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:26.997655 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:26.921990 19894 raft_consensus.cc:740] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:26.969439 16470 raft_consensus.cc:1240] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 12:23:27.013926 17264 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:27.065161 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 71679.3us
I20260709 12:23:27.003193 16543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1340) took 1396 ms. Trace:
I20260709 12:23:26.707625 20028 consensus_queue.cc:260] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:26.778939 19847 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:26.797214 19755 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:27.097627 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:27.030184 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:26.811326 20027 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:27.038810 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:26.932408 19911 consensus_queue.cc:1059] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.133s
W20260709 12:23:26.963618 19937 consensus_meta.cc:287] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.971s	user 0.001s	sys 0.014s
I20260709 12:23:27.007323 17258 raft_consensus.cc:2393] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 8: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:26.989885 20019 raft_consensus.cc:740] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:27.016202 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:27.037431 16224 catalog_manager.cc:5697] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:23:27.022166 19959 raft_consensus.cc:740] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:27.088851 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:27.116240 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:26.919041 19926 consensus_queue.cc:1059] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.121s
I20260709 12:23:26.993387 19806 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:27.123751 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:27.065817 16543 rpcz_store.cc:276] 0709 12:23:25.606688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:25.606844 (+   156us) service_pool.cc:224] Handling call
0709 12:23:25.796002 (+189158us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:25.796269 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:23:26.192548 (+396279us) log.cc:844] Serialized 14 byte log entry
0709 12:23:26.606385 (+413837us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:26.606469 (+    84us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:26.606541 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:26.606629 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:26.792670 (+186041us) spinlock_profiling.cc:243] Waited 1.02 ms on lock 0x7b18003b9140. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d6a2a6 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4
0709 12:23:27.002919 (+210249us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b5400bb5150. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:27.003104 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1033,"prepare.queue_time_us":200201,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"raft.queue_time_us":1159,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":4632,"spinlock_wait_cycles":436358784,"thread_start_us":808086,"threads_started":2,"wal-append.queue_time_us":520761}
I20260709 12:23:27.050540 17261 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:27.125427 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:23:27.125861 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 481938us
I20260709 12:23:27.127193 16366 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:27.097167 19847 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:26.993965 19487 raft_consensus.cc:697] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:27.102794 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:27.106832 16224 catalog_manager.cc:5697] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 6 to 12. New cstate: current_term: 12 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:27.003680 19785 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:27.051144 20029 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:27.114962 17264 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
I20260709 12:23:27.143194 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:27.187384 17258 raft_consensus.cc:2393] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 11: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:27.187285 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:27.188176 17264 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:27.118165 16398 raft_consensus.cc:2410] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:23:27.116876 17271 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:27.054158 19969 raft_consensus.cc:697] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:27.119530 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:27.064196 19983 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:27.204489 19983 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:27.121320 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 354061us
I20260709 12:23:27.122510 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:27.205282 19969 consensus_queue.cc:237] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:27.206645 20053 log.cc:927] Time spent T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:23:27.207376 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:27.208186 16543 raft_consensus.cc:2393] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 9: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
W20260709 12:23:27.209276 20047 log.cc:927] Time spent T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
I20260709 12:23:27.130625 16400 raft_consensus.cc:1240] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:27.101702 17270 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
W20260709 12:23:27.215916 16441 leader_election.cc:336] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Timed out: RequestConsensusVote RPC to 127.5.58.1:41805 timed out after 1.514s (SENT)
I20260709 12:23:27.140606 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:27.141417 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 316933us
I20260709 12:23:27.219110 16471 raft_consensus.cc:1240] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:27.220597 17270 raft_consensus.cc:1275] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:23:27.108063 20050 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:23:27.116870 20028 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:27.249264 19983 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:27.251052 19730 consensus_meta.cc:287] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.766s	user 0.008s	sys 0.000s
I20260709 12:23:27.128672 19806 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:27.202813 16224 catalog_manager.cc:5697] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 6 to 7, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 7 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:27.270103 16546 raft_consensus.cc:1240] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:27.204005 17258 raft_consensus.cc:1275] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:23:27.205644 16442 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:27.271759 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 66196.9us
I20260709 12:23:27.101658 19943 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:27.277055 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 52096.7us
W20260709 12:23:27.218009 19849 consensus_meta.cc:287] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.653s	user 0.011s	sys 0.000s
W20260709 12:23:27.216638 16441 leader_election.cc:336] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491): Timed out: RequestConsensusVote RPC to 127.5.58.4:44491 timed out after 1.514s (SENT)
I20260709 12:23:27.274912 16442 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:27.221884 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:27.064114 19898 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:27.182574 19757 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:27.232091 19853 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:27.233008 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:27.219985 19915 consensus_meta.cc:287] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.990s	user 0.011s	sys 0.002s
I20260709 12:23:27.233911 20023 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:27.220166 19926 raft_consensus.cc:3055] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:27.243219 20045 consensus_queue.cc:1048] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:27.279361 20015 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:27.218066 19894 consensus_queue.cc:260] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:27.245023 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:27.252405 20022 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:27.253129 20040 log.cc:927] Time spent T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.518s	user 0.001s	sys 0.000s
W20260709 12:23:27.253978 20044 log.cc:927] Time spent T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.320s	user 0.000s	sys 0.001s
I20260709 12:23:27.256738 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:27.256942 20046 log.cc:927] Time spent T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.500s	user 0.003s	sys 0.000s
I20260709 12:23:27.257529 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: true dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:27.259387 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:27.142997 19847 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:27.262645 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:27.201100 20029 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:27.119364 19755 consensus_queue.cc:1059] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.106s
I20260709 12:23:27.115901 20027 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:27.278061 16441 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:27.279482 16224 catalog_manager.cc:5697] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:23:27.227191 17271 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:23:27.285099 16470 raft_consensus.cc:1275] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 12:23:27.355055 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 138423us
I20260709 12:23:27.287665 20021 consensus_queue.cc:1048] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:23:27.258491 20013 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:27.368037 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 302052us
I20260709 12:23:27.298734 19915 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:27.289446 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:27.279747 19764 consensus_meta.cc:287] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.848s	user 0.013s	sys 0.000s
W20260709 12:23:27.281932 19978 consensus_meta.cc:287] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.695s	user 0.012s	sys 0.000s
I20260709 12:23:27.228137 17264 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
W20260709 12:23:27.300516 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 373844us
I20260709 12:23:27.278023 19937 raft_consensus.cc:2764] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
W20260709 12:23:27.244673 20041 log.cc:927] Time spent T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.473s	user 0.000s	sys 0.001s
I20260709 12:23:27.395221 17264 raft_consensus.cc:1240] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:23:27.396694 17264 raft_consensus.cc:1240] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:23:27.303145 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:27.282940 19585 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:27.306222 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:27.385627 19928 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.007s	sys 0.001s
W20260709 12:23:27.298285 19891 consensus_meta.cc:287] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.730s	user 0.010s	sys 0.002s
I20260709 12:23:27.283874 20055 raft_consensus.cc:3055] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:27.319684 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:27.321828 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 497173us
I20260709 12:23:27.299345 19926 raft_consensus.cc:740] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:27.303131 19858 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:27.334865 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:27.421689 16477 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:23:27.226210 20019 consensus_queue.cc:260] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:27.419759 16367 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:27.283782 19943 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:27.336710 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:27.337464 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:27.279078 19730 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:27.338050 19487 consensus_queue.cc:237] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:27.338845 19892 consensus_meta.cc:287] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.816s	user 0.008s	sys 0.003s
I20260709 12:23:27.283993 19959 consensus_queue.cc:260] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:27.310173 19995 consensus_meta.cc:287] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.734s	user 0.012s	sys 0.005s
I20260709 12:23:27.310024 20015 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:27.287314 19983 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:27.348670 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:27.351770  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:27.285075 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:27.358300 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:27.355322 16542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1342) took 1110 ms. Trace:
I20260709 12:23:27.361207 16441 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:27.365513 19819 consensus_meta.cc:287] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.724s	user 0.010s	sys 0.000s
I20260709 12:23:27.392602 19978 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:27.365836 20027 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:27.365955 16224 catalog_manager.cc:5697] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:23:27.366967 20059 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:23:27.276639 20039 log.cc:927] Time spent T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.541s	user 0.000s	sys 0.001s
W20260709 12:23:27.370996 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:27.378027 17142 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:27.381223 16469 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:27.390801 17143 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:27.472245 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 81540.9us
I20260709 12:23:27.119985 19979 consensus_queue.cc:237] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:27.406772 16471 raft_consensus.cc:1240] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:27.411221 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:27.418361 19928 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.007s	sys 0.001s
I20260709 12:23:27.429988 16367 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:27.405539 19787 raft_consensus.cc:3055] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:27.441509 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:27.479661 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:23:27.436714 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 309590us
I20260709 12:23:27.421682 19937 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:27.419081 20055 raft_consensus.cc:740] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:27.429708 19970 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:27.488637 16514 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:27.391974 19764 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:27.365278 19914 consensus_queue.cc:1048] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:27.457594 16542 rpcz_store.cc:276] 0709 12:23:26.245047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:26.246507 (+  1460us) service_pool.cc:224] Handling call
0709 12:23:26.247055 (+   548us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:23:26.247660 (+   605us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:26.247705 (+    45us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:23:26.267405 (+ 19700us) log.cc:844] Serialized 78 byte log entry
0709 12:23:26.649104 (+381699us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:26.649188 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:26.649297 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:27.256361 (+607064us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:23:27.342994 (+ 86633us) raft_consensus.cc:1685] finished
0709 12:23:27.343524 (+   530us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:27.355261 (+ 11737us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2437,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":15523,"spinlock_wait_cycles":29207936,"thread_start_us":328117,"threads_started":1,"wal-append.queue_time_us":439533}
I20260709 12:23:27.476745 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:27.469465 16474 raft_consensus.cc:1275] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:23:27.457177 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:27.466681 17261 raft_consensus.cc:2393] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 11: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:27.559279 16549 raft_consensus.cc:2393] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 9: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
W20260709 12:23:27.471411 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 93480.6us
I20260709 12:23:27.567190 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:27.562144 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 73632.5us
W20260709 12:23:27.458366 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 97260.2us
I20260709 12:23:27.418818 19891 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:27.559943 16224 catalog_manager.cc:5697] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 5 to 8, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 8 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:27.450266 20052 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:23:27.567544 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 137625us
I20260709 12:23:27.570528 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:27.571338 16441 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:27.571081 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:27.565099 17142 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:27.446749 19892 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:27.438478 19858 raft_consensus.cc:697] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:27.577805 16475 raft_consensus.cc:1240] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:27.555550 19787 raft_consensus.cc:740] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:27.578994 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:27.605931 16542 raft_consensus.cc:1275] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:23:27.606980 16546 raft_consensus.cc:1275] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:23:27.608628 16547 raft_consensus.cc:2393] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 10: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:27.579403 16366 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:27.459879 20019 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:23:27.474247 19931 consensus_queue.cc:1175] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "52ab50b0237346c1a13ede947f6a63c2" responder_term: 10 status { last_received { term: 9 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 9. Current term is 10. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:23:27.471431 19959 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:27.454385 20066 consensus_queue.cc:1048] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.134s
I20260709 12:23:27.578320 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:27.602029 16224 catalog_manager.cc:5697] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 5 to 6, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 6 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:27.641657 16224 catalog_manager.cc:5697] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 1 to 8, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 8 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:23:27.468271 20015 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:27.604074 16367 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:27.613972 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:27.466192 19837 raft_consensus.cc:2749] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:23:27.655612 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 76295.3us
I20260709 12:23:27.574648 19983 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:27.472980 19819 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:27.599965 20052 raft_consensus.cc:697] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:27.467578 19995 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:27.585986 19996 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:27.661803 16400 raft_consensus.cc:1240] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:27.660414 19898 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:27.585096 20065 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:23:27.566094 19937 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:27.651229 16399 raft_consensus.cc:1240] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:23:27.676887 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:27.597401 20025 consensus_queue.cc:1059] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 12:23:27.669500 19983 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:27.686020 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:27.726779 16398 raft_consensus.cc:2393] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 7: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:27.730408 19937 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:27.678053 20091 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:23:27.737282 20082 log.cc:927] Time spent T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:23:27.620455 20074 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:27.605396 19941 consensus_queue.cc:1048] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 12:23:27.480419 19926 consensus_queue.cc:260] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:27.660966 20054 log.cc:927] Time spent T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.057s	user 0.001s	sys 0.001s
I20260709 12:23:27.669198 20070 consensus_queue.cc:1048] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.082s
W20260709 12:23:27.685439 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 114179us
W20260709 12:23:27.686018 20068 log.cc:927] Time spent T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.257s	user 0.000s	sys 0.000s
I20260709 12:23:27.747028 16550 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "f99066d13e524238aec6aa2024a8bd50"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:23:27.749330 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:27.681208 20058 log.cc:927] Time spent T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.358s	user 0.001s	sys 0.000s
I20260709 12:23:27.738597 16402 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 14. Current term is 15. Ops: []
W20260709 12:23:27.724085 20062 log.cc:927] Time spent T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.326s	user 0.001s	sys 0.000s
I20260709 12:23:27.822052 16441 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:27.828955 16397 raft_consensus.cc:2393] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 10: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:27.829770 16441 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:27.824347 16550 raft_consensus.cc:606] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Received request to transfer leadership to TS f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:27.831135 16550 raft_consensus.cc:614] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:23:27.833227 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 229424us
I20260709 12:23:27.798501 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:27.837133 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:27.837926 16542 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:27.683139 20048 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:27.841825 16441 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:27.738204 19915 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:27.866340 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:27.866219 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:27.739283 19983 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:27.753458 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:23:27.862859 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:27.856649 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:27.851763 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:27.886315 16544 raft_consensus.cc:2393] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 11: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
W20260709 12:23:27.773222 20042 consensus_meta.cc:287] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.701s	user 0.010s	sys 0.000s
I20260709 12:23:27.840292 20091 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:27.851778 19937 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:27.857360 19898 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:23:27.911470 17144 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.043s	sys 0.084s
W20260709 12:23:27.911851 17144 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.043s	sys 0.084s
I20260709 12:23:27.907198 16542 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
W20260709 12:23:27.898903 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 57146us
I20260709 12:23:27.914405 16396 raft_consensus.cc:1240] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:27.914765 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:27.905548 16544 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "191e052863dd48fe92ebe151197330a8"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "f99066d13e524238aec6aa2024a8bd50"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:23:27.914934 16441 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:27.917328 16394 raft_consensus.cc:1275] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 12:23:27.919597 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:27.887900 20025 raft_consensus.cc:3055] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:27.922410 16401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 1115) took 1120 ms. Trace:
I20260709 12:23:27.923933 16401 rpcz_store.cc:276] 0709 12:23:26.802042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:26.911301 (+109259us) service_pool.cc:224] Handling call
0709 12:23:26.911763 (+   462us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:26.919679 (+  7916us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:23:26.919711 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:26.920200 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 12:23:27.279167 (+358967us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:27.279214 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:27.279297 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:27.922177 (+642880us) raft_consensus.cc:1685] finished
0709 12:23:27.922204 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:27.922333 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74272,"raft.queue_time_us":228666,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":40760,"thread_start_us":257086,"threads_started":1,"wal-append.queue_time_us":449915}
W20260709 12:23:27.906728 20062 log.cc:927] Time spent T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:23:27.896279 19978 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:27.929252 17142 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:27.694861 19787 consensus_queue.cc:260] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:27.904182 19837 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:27.930264 17143 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:27.937626 17142 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:27.904556 19730 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:27.900743 19915 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:27.946166 17142 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:27.926008 20013 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:27.939589 16398 raft_consensus.cc:1240] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:27.906543 19943 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:27.906267 19996 raft_consensus.cc:697] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:27.843992 19858 consensus_queue.cc:237] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:27.955252 20075 log.cc:927] Time spent T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.346s	user 0.001s	sys 0.000s
I20260709 12:23:27.676787 20055 consensus_queue.cc:260] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:27.955828 20072 log.cc:927] Time spent T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.288s	user 0.000s	sys 0.001s
W20260709 12:23:27.956403 20077 log.cc:927] Time spent T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.339s	user 0.001s	sys 0.000s
I20260709 12:23:27.958930 16545 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:27.928260 19699 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:27.926625 20025 raft_consensus.cc:740] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
W20260709 12:23:27.962920 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 128896us
I20260709 12:23:27.939383 16441 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:27.900492 20042 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:27.933912 19819 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:27.887058 20048 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:28.018846 19880 consensus_meta.cc:287] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 1.089s	user 0.010s	sys 0.002s
I20260709 12:23:27.941905 20066 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:27.944617 20065 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:27.950117 19891 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:27.939919 19898 consensus_queue.cc:1048] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:23:27.912698 20091 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:27.974946 19730 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:28.022924 20095 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:28.048183 16442 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:28.020892 19806 consensus_meta.cc:287] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.738s	user 0.011s	sys 0.000s
I20260709 12:23:27.976258 19787 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:23:28.047844 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 108858us
W20260709 12:23:28.063043 16441 leader_election.cc:336] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Timed out: RequestConsensusVote RPC to 127.5.58.1:41805 timed out after 1.948s (SENT)
I20260709 12:23:28.063651 16441 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:28.051160 16542 raft_consensus.cc:1275] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 12:23:28.065217 17264 raft_consensus.cc:1275] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:23:28.065546 16542 pending_rounds.cc:85] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1: Aborting all ops after (but not including) 4
I20260709 12:23:28.065786 16542 pending_rounds.cc:107] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 12:23:28.066006 16542 raft_consensus.cc:2889] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:28.066354 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:28.067154 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:28.067572 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:28.051425 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:27.968662 19892 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:27.945909 19837 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:27.889760 20052 consensus_queue.cc:237] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:28.056244 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 110157us
I20260709 12:23:28.057049 16544 raft_consensus.cc:606] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Received request to transfer leadership to TS f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:28.025568 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:28.081171 16544 raft_consensus.cc:614] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:28.081452 17142 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:27.979816 19764 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:28.081753 16550 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:28.068696 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:28.068293 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:28.053467 20055 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:28.054738 20073 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:28.053507 20013 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:23:28.080850 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:28.072248 19908 consensus_meta.cc:287] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 1.497s	user 0.012s	sys 0.003s
I20260709 12:23:28.050740 19995 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:28.091281 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:28.048920 19880 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:28.093457 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:28.037205 19699 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:28.059695 20065 raft_consensus.cc:697] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:28.076900 19978 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:28.098287 17264 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
I20260709 12:23:28.059615 20067 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:28.078233 19730 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:28.109031 17142 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:28.113433 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:28.113796 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 50804.9us
I20260709 12:23:28.127584 16367 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:28.071054 19806 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:28.132723 17142 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:28.134204 16441 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:28.135494 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:28.136464 17264 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:28.122426 20095 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:28.113006 20013 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:23:28.140668 20092 log.cc:927] Time spent T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
W20260709 12:23:28.140753 20096 log.cc:927] Time spent T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 12:23:28.143606 20067 raft_consensus.cc:697] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:28.144590 20095 consensus_queue.cc:1048] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:23:28.146050 20084 log.cc:927] Time spent T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.222s	user 0.001s	sys 0.000s
I20260709 12:23:28.183882 20067 consensus_queue.cc:237] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:28.149655 20009 consensus_meta.cc:287] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 1.334s	user 0.006s	sys 0.006s
I20260709 12:23:28.097431 19908 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:28.186357 20009 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:28.186497 20065 consensus_queue.cc:237] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:28.186992 20095 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:28.187582 20095 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:28.149564 19837 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:28.188591 16401 raft_consensus.cc:1240] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:28.189939 20095 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:28.150549 20085 log.cc:927] Time spent T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.241s	user 0.001s	sys 0.001s
I20260709 12:23:28.150701 20105 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:23:28.151082 20083 log.cc:927] Time spent T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.352s	user 0.000s	sys 0.001s
I20260709 12:23:28.080210 20025 consensus_queue.cc:260] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:28.152503 19880 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:28.153738 20048 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:28.114806 20094 consensus_queue.cc:1048] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.217s
I20260709 12:23:28.156755 19806 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:28.161791 17142 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
W20260709 12:23:28.224557 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 62886.5us
I20260709 12:23:28.225579 17142 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:28.174556 20091 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:28.159678 20066 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:23:28.178368 20015 consensus_meta.cc:287] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.500s	user 0.006s	sys 0.003s
I20260709 12:23:28.235579 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:28.095063 20073 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:28.134745 19730 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:28.182889 19847 consensus_meta.cc:287] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.831s	user 0.006s	sys 0.006s
I20260709 12:23:28.185710 20110 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:28.193055 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:28.194991 16367 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
W20260709 12:23:28.184084 19779 consensus_meta.cc:287] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 1.259s	user 0.004s	sys 0.007s
W20260709 12:23:28.196463 17270 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.008s	sys 0.012s
I20260709 12:23:28.161144 19978 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:28.221429 16469 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.018s	sys 0.036s
I20260709 12:23:28.234030 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:28.135458 19979 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:28.195976 19941 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:28.236275 19908 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:28.197324 19943 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:23:28.131263 19996 consensus_queue.cc:237] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:28.244247 16473 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 11: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
W20260709 12:23:28.248914 20086 log.cc:927] Time spent T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:23:28.246816 16546 raft_consensus.cc:2410] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 12:23:28.247303 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 52637.3us
W20260709 12:23:28.253049 17270 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.009s	sys 0.013s
I20260709 12:23:28.272672 17270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 961) took 1424 ms. Trace:
I20260709 12:23:28.202493 19892 consensus_queue.cc:1048] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
W20260709 12:23:28.254801 16469 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.018s	sys 0.036s
I20260709 12:23:28.228118 20105 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:28.254717 20100 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:28.238596 20050 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:23:28.261597 20029 consensus_meta.cc:287] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.905s	user 0.011s	sys 0.001s
I20260709 12:23:28.244371 20025 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:23:28.242962 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 108845us
I20260709 12:23:28.245219 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:28.325809 17266 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:28.270049 17261 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45230 (request call id 959) took 1422 ms. Trace:
I20260709 12:23:28.326277 16441 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:28.247622 19847 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:28.247473 20009 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:28.273046 17270 rpcz_store.cc:276] 0709 12:23:26.848734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:27.221414 (+372680us) service_pool.cc:224] Handling call
0709 12:23:27.263321 (+ 41907us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:27.574468 (+311147us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:27.574501 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:27.593366 (+ 18865us) log.cc:844] Serialized 26 byte log entry
0709 12:23:28.272283 (+678917us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:28.272324 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:28.272449 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:28.272493 (+    44us) raft_consensus.cc:1685] finished
0709 12:23:28.272504 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:28.272618 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":77248,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"thread_start_us":934628,"threads_started":2,"wal-append.queue_time_us":662697}
I20260709 12:23:28.274852 16476 raft_consensus.cc:1275] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:28.302469 16367 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:28.179396 19699 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:28.274437 19908 consensus_queue.cc:1059] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:23:28.305632 19880 consensus_queue.cc:1048] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:23:28.307042 19959 consensus_meta.cc:287] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.653s	user 0.007s	sys 0.003s
I20260709 12:23:28.265329 19941 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:28.309113 17258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 964) took 1459 ms. Trace:
I20260709 12:23:28.256193 20104 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:28.311519 19585 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:28.258769 19978 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:28.314925 20118 log.cc:927] Time spent T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:23:28.318262 16398 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:28.242635 20015 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:28.322707 16469 raft_consensus.cc:1275] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:23:28.323652 16516 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:28.423573 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 100046us
I20260709 12:23:28.424396 20015 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:28.424853 20101 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:28.425454 19699 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:28.425479 20101 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:28.426489 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:28.326833 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:28.427165 17269 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
I20260709 12:23:28.427629 17266 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:28.428179 20101 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:28.428752 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:28.269621 20074 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:28.326514 17261 rpcz_store.cc:276] 0709 12:23:26.848087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:27.125209 (+277122us) service_pool.cc:224] Handling call
0709 12:23:28.269825 (+1144616us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b540005fd54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:28.269984 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":782978432}
I20260709 12:23:28.327594 16441 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:28.433148 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 105663us
I20260709 12:23:28.433204 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:28.433841 17261 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
I20260709 12:23:28.433923 16441 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:28.434674 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:28.435237 16441 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:28.275506 19917 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:28.435665 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:28.316582 19853 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:28.254706 19779 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:28.363372 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 61041.2us
I20260709 12:23:28.328577 20021 consensus_queue.cc:1059] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 12:23:28.319836 20050 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:28.353324 20094 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:28.351137 20100 raft_consensus.cc:697] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:28.406292 16515 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:28.396653 17258 rpcz_store.cc:276] 0709 12:23:26.849716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:27.275908 (+426192us) service_pool.cc:224] Handling call
0709 12:23:27.287993 (+ 12085us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:27.288788 (+   795us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:27.288816 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:27.299050 (+ 10234us) log.cc:844] Serialized 26 byte log entry
0709 12:23:27.478828 (+179778us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:27.654248 (+175420us) log.cc:844] Serialized 14 byte log entry
0709 12:23:27.669499 (+ 15251us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:27.669574 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:28.308888 (+639314us) raft_consensus.cc:1685] finished
0709 12:23:28.308907 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:28.309045 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15018,"prepare.queue_time_us":422805,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":91743,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":126,"thread_start_us":230625,"threads_started":2,"wal-append.queue_time_us":651172}
I20260709 12:23:28.321496 20042 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:28.391999 19943 raft_consensus.cc:2804] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:28.408325 16543 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
I20260709 12:23:28.328269 20029 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:28.408505 20105 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:28.414338 19978 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:28.414541 19941 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:28.415942 19730 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:28.410899 19585 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:28.493577 19928 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:28.497056 19928 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:28.497484 19928 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:28.462411 20013 consensus_queue.cc:1048] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:23:28.422037 19908 raft_consensus.cc:3055] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:28.501997 19908 raft_consensus.cc:740] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:28.503216 19908 consensus_queue.cc:260] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:28.325717 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:28.505321 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:28.441430 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:28.428166 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:28.506675 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:28.507294 19585 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:28.507902 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:28.446995 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:28.511734 16516 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:28.448577 17261 raft_consensus.cc:2393] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 10: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:28.327351 20055 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:28.466682 20021 consensus_queue.cc:1243] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer f99066d13e524238aec6aa2024a8bd50 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:23:28.523341 16542 raft_consensus.cc:1240] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:28.414007 19847 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:28.489635 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 83467us
I20260709 12:23:28.448395 20050 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:28.449169 20110 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:28.449443 19917 consensus_queue.cc:1048] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 12:23:28.555316 16515 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:28.556545 20110 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:28.557065 20110 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:28.557333 20097 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:28.461372 19853 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:28.558136 20097 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:28.559238 19853 raft_consensus.cc:697] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:28.560232 19853 consensus_queue.cc:237] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:28.560766 19941 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:28.561234 19941 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:28.561587 19941 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:28.462672 19837 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:28.565057 19837 raft_consensus.cc:697] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:28.566454 19837 consensus_queue.cc:237] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:28.443783 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:28.512106 17263 raft_consensus.cc:1240] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 14. Current term is 15. Ops: []
W20260709 12:23:28.515870 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 68969.3us
I20260709 12:23:28.439867 19779 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:28.505910 16477 raft_consensus.cc:2376] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10. Re-sending same reply.
I20260709 12:23:28.573303 16477 raft_consensus.cc:1275] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:23:28.517616 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:28.468186 19943 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:28.574908 19943 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:28.576620 16468 raft_consensus.cc:1240] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:28.525208 17266 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:28.578644 20097 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:28.578943 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:28.579267 20097 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:28.580232 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:28.581251 20121 log.cc:927] Time spent T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:23:28.581956 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:28.583389 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:28.525633 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:23:28.512473 20117 log.cc:927] Time spent T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 12:23:28.529114 16546 raft_consensus.cc:1240] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:28.595291 16515 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:28.650771 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:28.650910 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:28.524506 19908 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:28.495791 20105 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:28.554350 20042 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:28.579769 20097 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:28.526623 19978 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:28.656486 17269 raft_consensus.cc:1240] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:23:28.411911 20104 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:28.549556 20021 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:28.579887 20029 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:28.528134 20055 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:28.678839 19978 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:28.685586 19959 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:28.725051 19983 consensus_meta.cc:287] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.811s	user 0.013s	sys 0.000s
I20260709 12:23:28.762804 20021 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:28.679173 20042 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:28.769603 19978 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:28.790227 20021 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:28.782796 19959 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:28.783529 19983 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:28.808462 20122 log.cc:927] Time spent T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.280s	user 0.001s	sys 0.000s
I20260709 12:23:28.802748 20021 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:28.679983 20100 consensus_queue.cc:237] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:28.837387 20019 consensus_meta.cc:287] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 1.196s	user 0.009s	sys 0.005s
I20260709 12:23:28.844735 20055 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:28.848549 20042 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:28.850461 20097 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:28.854357 19959 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:28.855865 20104 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:28.873276 19983 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:28.922993 20057 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.255s	user 0.025s	sys 0.004s
I20260709 12:23:28.978389 20110 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:28.971236 20057 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.317s	user 0.025s	sys 0.004s
I20260709 12:23:28.987666 20050 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:29.009930 20110 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:28.524313 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:29.063954 16403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46250 (request call id 1090) took 1014 ms. Trace:
I20260709 12:23:29.065877 16403 rpcz_store.cc:276] 0709 12:23:28.049124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:28.523994 (+474870us) service_pool.cc:224] Handling call
0709 12:23:29.063564 (+539570us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:28.514213 16394 raft_consensus.cc:2393] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 7: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:29.073871 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:29.075522 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:29.077286 16403 raft_consensus.cc:2393] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 12: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:29.078599 16402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 1114) took 2276 ms. Trace:
I20260709 12:23:29.079190 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:29.079162 16402 rpcz_store.cc:276] 0709 12:23:26.801657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:26.802369 (+   712us) service_pool.cc:224] Handling call
0709 12:23:26.921462 (+119093us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:27.827369 (+905907us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:29.077544 (+1250175us) spinlock_profiling.cc:243] Waited 804 ms on lock 0x7b54001487d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:29.078256 (+   712us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1849678592}
I20260709 12:23:28.555316 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:29.096587 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:29.097185 16473 raft_consensus.cc:2410] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:23:29.098686 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:29.099198 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:29.096710 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:29.076116 16394 raft_consensus.cc:2393] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 9: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:29.109084 16402 raft_consensus.cc:2393] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 10: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:29.110265 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:29.112485 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:29.113569 16474 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:29.114857 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:29.116760 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:29.112066 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:29.118074 16474 raft_consensus.cc:2393] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 9: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:29.119706 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:29.110460 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:28.586043 16542 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:28.878791 19943 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:29.119750 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:29.259827 16403 raft_consensus.cc:2410] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
W20260709 12:23:29.275157 16549 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.005s	sys 0.005s
I20260709 12:23:29.275437 19779 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:29.275969 16549 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.006s	sys 0.005s
I20260709 12:23:29.276648 20013 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:29.277973 20013 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:29.275695 19853 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.006s	sys 0.009s
W20260709 12:23:29.280781 16470 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.840s	user 0.002s	sys 0.012s
I20260709 12:23:29.281664 19816 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:29.285166 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:29.287012 16399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 931) took 2824 ms. Trace:
I20260709 12:23:29.289158 16548 raft_consensus.cc:2368] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11.
I20260709 12:23:29.290639 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:29.276708 19779 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:29.306181 19853 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.007s	sys 0.009s
W20260709 12:23:29.311578 16470 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.872s	user 0.003s	sys 0.012s
I20260709 12:23:29.316975 17271 raft_consensus.cc:2410] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:23:29.319082 17258 raft_consensus.cc:1240] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:29.334802 20013 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:29.318150 19779 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:29.336561 20013 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.337453 20013 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:29.347002 20013 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:29.344523 19928 consensus_meta.cc:287] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.846s	user 0.012s	sys 0.009s
W20260709 12:23:29.347707 20079 log.cc:927] Time spent T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 1.692s	user 0.119s	sys 0.205s
I20260709 12:23:29.347975 17258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 1005) took 1102 ms. Trace:
I20260709 12:23:29.348539 17258 rpcz_store.cc:276] 0709 12:23:28.245154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:28.453630 (+208476us) service_pool.cc:224] Handling call
0709 12:23:28.493044 (+ 39414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:29.346591 (+853547us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:29.347530 (+   939us) spinlock_profiling.cc:243] Waited 691 ms on lock 0x7b54002800d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:29.347819 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1589712640}
I20260709 12:23:29.360721 16403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46250 (request call id 1087) took 1383 ms. Trace:
I20260709 12:23:29.361379 16403 rpcz_store.cc:276] 0709 12:23:27.977769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.284522 (+1306753us) service_pool.cc:224] Handling call
0709 12:23:29.360611 (+ 76089us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:29.363229 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:29.365353 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:29.367461 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:29.376729 17271 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45224 (request call id 993) took 1763 ms (client timeout 1526 ms). Trace:
I20260709 12:23:29.377624 17264 raft_consensus.cc:1275] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
W20260709 12:23:29.378254 17266 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.014s	sys 0.025s
I20260709 12:23:29.378449 17264 pending_rounds.cc:85] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8: Aborting all ops after (but not including) 5
W20260709 12:23:29.378793 17266 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.014s	sys 0.025s
I20260709 12:23:29.378890 17264 pending_rounds.cc:107] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 12:23:29.379496 17264 raft_consensus.cc:2889] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:29.381204 16402 raft_consensus.cc:2368] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 10. Current term is 11.
I20260709 12:23:29.383036 16402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46250 (request call id 1093) took 1235 ms. Trace:
I20260709 12:23:29.304011 16399 rpcz_store.cc:276] 0709 12:23:26.462798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:26.462911 (+   113us) service_pool.cc:224] Handling call
0709 12:23:26.463210 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:27.686900 (+1223690us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:29.286015 (+1599115us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b540015f254. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:29.286783 (+   768us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2731454720}
I20260709 12:23:29.386638 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:29.387763 16399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46250 (request call id 1097) took 1239 ms. Trace:
W20260709 12:23:29.377512 17271 rpcz_store.cc:269] 0709 12:23:27.613592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:27.613716 (+   124us) service_pool.cc:224] Handling call
0709 12:23:29.376272 (+1762556us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b5400be2dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:29.376573 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3528764672}
I20260709 12:23:29.388310 16399 rpcz_store.cc:276] 0709 12:23:28.148700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.386138 (+1237438us) service_pool.cc:224] Handling call
0709 12:23:29.387657 (+  1519us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:29.406829 16473 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:29.406965 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:29.399200 17271 raft_consensus.cc:1275] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:29.409052 17263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 986) took 2578 ms. Trace:
I20260709 12:23:29.392674 16403 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:29.407172 16546 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.000s	sys 0.012s
I20260709 12:23:29.403939 17258 raft_consensus.cc:1275] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:29.391985 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:29.480567 16394 raft_consensus.cc:2368] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 10. Current term is 11.
I20260709 12:23:29.383769 16402 rpcz_store.cc:276] 0709 12:23:28.147224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.119270 (+972046us) service_pool.cc:224] Handling call
0709 12:23:29.382614 (+263344us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b54007634d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:29.382888 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":469715840}
W20260709 12:23:29.392151 16515 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.007s	sys 0.004s
W20260709 12:23:29.489542 16546 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.813s	user 0.000s	sys 0.013s
I20260709 12:23:29.387955 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:29.491974 16469 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:23:29.492431 16515 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.837s	user 0.007s	sys 0.004s
I20260709 12:23:29.485188 17258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 1013) took 1225 ms. Trace:
I20260709 12:23:29.493278 17258 rpcz_store.cc:276] 0709 12:23:28.258034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.402494 (+1144460us) service_pool.cc:224] Handling call
0709 12:23:29.403310 (+   816us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:29.470572 (+ 67262us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:29.483419 (+ 12847us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:29.484225 17271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 1012) took 1226 ms. Trace:
I20260709 12:23:29.496146 17271 rpcz_store.cc:276] 0709 12:23:28.253899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.397717 (+1143818us) service_pool.cc:224] Handling call
0709 12:23:29.398413 (+   696us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:29.467558 (+ 69145us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:29.480547 (+ 12989us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:29.490895 16394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46250 (request call id 1100) took 1165 ms. Trace:
I20260709 12:23:29.471766 17263 rpcz_store.cc:276] 0709 12:23:26.830791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:26.831579 (+   788us) service_pool.cc:224] Handling call
0709 12:23:26.921564 (+ 89985us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:28.569029 (+1647465us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:29.408584 (+839555us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b540080c854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:29.408878 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3657260288}
I20260709 12:23:29.499924 16394 rpcz_store.cc:276] 0709 12:23:28.325918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.406481 (+1080563us) service_pool.cc:224] Handling call
0709 12:23:29.490811 (+ 84330us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:29.508535 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:29.390975 19943 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:29.396713 19996 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.415331 20100 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:29.347786 19928 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:29.405197 19816 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:29.537205 20141 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:29.537506 19996 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:29.538201 20141 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:29.540936 20100 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:29.552135 20127 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:29.552482 16474 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:29.553130 20127 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:29.554594 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:29.559317 19928 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:29.561118 20033 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.561826 20033 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:29.517993 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 181421us
I20260709 12:23:29.566010 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:29.567291 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:29.536315 16514 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.005s	sys 0.004s
I20260709 12:23:29.571677 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:29.572412 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:23:29.535794 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 940534us
W20260709 12:23:29.574901 16514 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.005s	user 0.006s	sys 0.004s
W20260709 12:23:29.578471 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 389579us
I20260709 12:23:29.544062 19816 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:29.577689 16399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46250 (request call id 1101) took 1148 ms. Trace:
I20260709 12:23:29.582340 16399 rpcz_store.cc:276] 0709 12:23:28.429286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.489915 (+1060629us) service_pool.cc:224] Handling call
0709 12:23:29.577637 (+ 87722us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:23:29.571401 19917 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.890s	user 0.009s	sys 0.012s
W20260709 12:23:29.590271 16442 leader_election.cc:336] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.516s (SENT)
I20260709 12:23:29.574540 20127 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:29.610140 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:29.584108 20100 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:29.612636 16442 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:29.613191 16403 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
I20260709 12:23:29.579159 20073 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:29.615186 16403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 1124) took 1149 ms. Trace:
I20260709 12:23:29.616983 16403 rpcz_store.cc:276] 0709 12:23:28.465936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.362461 (+896525us) service_pool.cc:224] Handling call
0709 12:23:29.615123 (+252662us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:23:29.610520 19917 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.940s	user 0.009s	sys 0.013s
I20260709 12:23:29.618152 16394 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:29.622354 16515 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:23:29.618649 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 227777us
I20260709 12:23:29.634929 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:29.624039 20105 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:29.624032 20129 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:29.644776 20129 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:29.645452 20129 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:29.645314 16366 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:29.624725 19996 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:29.646251 20142 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:29.646978 19996 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:29.634148 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:29.647609 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:29.647302 20142 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:29.648272 16474 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:29.643069 16516 leader_election.cc:336] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.755s (SENT)
I20260709 12:23:29.638425 17271 raft_consensus.cc:1275] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:29.650772 17271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 1014) took 1391 ms. Trace:
I20260709 12:23:29.651302 17271 rpcz_store.cc:276] 0709 12:23:28.259130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.534248 (+1275118us) service_pool.cc:224] Handling call
0709 12:23:29.618397 (+ 84149us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:29.650276 (+ 31879us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:29.650683 (+   407us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:29.618871 19816 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.652989 19816 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:29.657714 20067 consensus_queue.cc:1059] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:29.658202 20073 consensus_queue.cc:1059] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:29.659507 20073 raft_consensus.cc:3055] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:29.659711 20057 raft_consensus.cc:3055] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:29.648319 16400 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:29.660270 20057 raft_consensus.cc:740] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:29.661387 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:29.661620 20057 consensus_queue.cc:260] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:29.662340 17271 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:29.663338 20057 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:29.664969 19858 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:23:29.631526 17261 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.970s	user 0.011s	sys 0.008s
W20260709 12:23:29.666443 17261 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.071s	user 0.012s	sys 0.008s
I20260709 12:23:29.649152 16516 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:29.659984 20073 raft_consensus.cc:740] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:29.669340 20022 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:29.669193 20073 consensus_queue.cc:260] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:29.648638 16542 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:29.671988 20073 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:29.679327 16468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 690) took 1086 ms. Trace:
I20260709 12:23:29.631211 20033 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:29.685190 16546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1363) took 1160 ms. Trace:
W20260709 12:23:29.685454 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:29.685662 16546 rpcz_store.cc:276] 0709 12:23:28.525060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:28.652234 (+127174us) service_pool.cc:224] Handling call
0709 12:23:28.652548 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:28.652787 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:23:29.527379 (+874592us) log.cc:844] Serialized 14 byte log entry
0709 12:23:29.678643 (+151264us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:29.684380 (+  5737us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:29.684515 (+   135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:29.684840 (+   325us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:29.685067 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":717514,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":86,"raft.queue_time_us":3292,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":135,"thread_start_us":884611,"threads_started":2}
I20260709 12:23:29.687033 17144 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:29.688581 19755 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.689291 19755 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:29.692286 19755 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:29.692902 19755 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:29.696601 19935 raft_consensus.cc:3055] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:29.696848 16366 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:29.697118 19935 raft_consensus.cc:740] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:29.698859 16366 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:29.699832 19996 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:29.700896 19996 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:23:29.701959 16547 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37242 (request call id 1166) took 2644 ms (client timeout 1541 ms). Trace:
W20260709 12:23:29.702474 16547 rpcz_store.cc:269] 0709 12:23:27.057623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:27.057903 (+   280us) service_pool.cc:224] Handling call
0709 12:23:29.701329 (+2643426us) spinlock_profiling.cc:243] Waited 503 ms on lock 0x7b54008c55d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:29.701617 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1156545280}
I20260709 12:23:29.704730 19816 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:29.706048 19926 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:29.679839 16468 rpcz_store.cc:276] 0709 12:23:28.592784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.099715 (+506931us) service_pool.cc:224] Handling call
0709 12:23:29.100288 (+   573us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:29.270869 (+170581us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:29.270917 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:29.287588 (+ 16671us) log.cc:844] Serialized 52 byte log entry
0709 12:23:29.527170 (+239582us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:29.635217 (+108047us) log.cc:844] Serialized 14 byte log entry
0709 12:23:29.635449 (+   232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:29.635526 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:29.677054 (+ 41528us) raft_consensus.cc:1685] finished
0709 12:23:29.677082 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:29.679259 (+  2177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47049,"raft.queue_time_us":214345,"raft.run_cpu_time_us":415,"raft.run_wall_time_us":8067,"thread_start_us":443549,"threads_started":3,"wal-append.queue_time_us":341711}
I20260709 12:23:29.710444 19585 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:29.704114 16400 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
I20260709 12:23:29.630872 20127 consensus_queue.cc:1048] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 12:23:29.674981 20141 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:29.698498 19935 consensus_queue.cc:260] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:29.680972 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 51950.1us
I20260709 12:23:29.712522 16400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 1121) took 1272 ms. Trace:
W20260709 12:23:29.651366 20025 consensus_meta.cc:287] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 1.301s	user 0.011s	sys 0.009s
I20260709 12:23:29.720942 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:29.723970 16400 rpcz_store.cc:276] 0709 12:23:28.440154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.616585 (+1176431us) service_pool.cc:224] Handling call
0709 12:23:29.712451 (+ 95866us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:29.726212 16399 raft_consensus.cc:2410] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:23:29.726863 16399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 1123) took 1264 ms. Trace:
I20260709 12:23:29.727246 16399 rpcz_store.cc:276] 0709 12:23:28.462147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.598364 (+1136217us) service_pool.cc:224] Handling call
0709 12:23:29.726781 (+128417us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:29.728157 16515 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:29.729202 19858 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.730000 20048 raft_consensus.cc:2804] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:29.730787 20048 raft_consensus.cc:697] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 10 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:29.733314 17264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 1003) took 1489 ms. Trace:
W20260709 12:23:29.733646 19941 consensus_meta.cc:287] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 1.171s	user 0.009s	sys 0.017s
I20260709 12:23:29.733896 17264 rpcz_store.cc:276] 0709 12:23:28.244447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:28.244602 (+   155us) service_pool.cc:224] Handling call
0709 12:23:29.376734 (+1132132us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:29.439536 (+ 62802us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:29.733012 (+293476us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b5400be2dd0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:29.733212 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2575138688}
I20260709 12:23:29.734212 19941 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:29.735584 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:29.738814 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:29.741974 19959 consensus_queue.cc:1048] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:29.743288 16474 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:29.740916 16542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37226 (request call id 1361) took 1196 ms. Trace:
W20260709 12:23:29.734609 19908 consensus_meta.cc:287] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 1.076s	user 0.012s	sys 0.003s
I20260709 12:23:29.741086 19779 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.748672 19779 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:29.749850 17261 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 1010) took 1496 ms. Trace:
W20260709 12:23:29.755299 19978 consensus_meta.cc:287] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.959s	user 0.007s	sys 0.013s
I20260709 12:23:29.721837 19816 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:29.756742 20022 consensus_queue.cc:1059] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:29.757045 20142 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:29.756985 19816 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:29.745396 16542 rpcz_store.cc:276] 0709 12:23:28.524609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:28.525426 (+   817us) service_pool.cc:224] Handling call
0709 12:23:29.712274 (+1186848us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:23:29.760699 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 62008.6us
I20260709 12:23:29.784976 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:29.792017 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 202044us
W20260709 12:23:29.792763 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 149760us
I20260709 12:23:29.755867 19978 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:29.796445 16394 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:29.803746 16394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 1126) took 1287 ms. Trace:
I20260709 12:23:29.804186 16394 rpcz_store.cc:276] 0709 12:23:28.516462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.520844 (+1004382us) service_pool.cc:224] Handling call
0709 12:23:29.803664 (+282820us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:23:29.799530 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 67644.5us
W20260709 12:23:29.801877 16442 leader_election.cc:336] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.531s (SENT)
I20260709 12:23:29.801720 20157 consensus_queue.cc:1059] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:29.796285 16549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1359) took 1264 ms. Trace:
I20260709 12:23:29.802956 20033 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.809945 16396 raft_consensus.cc:1240] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 8. Current term is 9. Ops: []
W20260709 12:23:29.810366 16516 leader_election.cc:336] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.902s (SENT)
I20260709 12:23:29.810931 19928 raft_consensus.cc:3055] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:29.811390 19928 raft_consensus.cc:740] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:29.810781 16442 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:29.811875 19787 consensus_meta.cc:287] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 1.740s	user 0.013s	sys 0.010s
I20260709 12:23:29.812739 16366 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
W20260709 12:23:29.813179 16442 leader_election.cc:336] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.538s (SENT)
I20260709 12:23:29.813752 16442 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:29.814620 16366 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:29.814679 20141 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:29.815768 20141 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:29.816213 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:29.816941 16400 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:29.817103 16394 raft_consensus.cc:1240] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:29.817757 16394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 1089) took 1791 ms. Trace:
I20260709 12:23:29.818038 16394 rpcz_store.cc:276] 0709 12:23:28.026547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.816416 (+1789869us) service_pool.cc:224] Handling call
0709 12:23:29.816848 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:29.817507 (+   659us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:29.817683 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:29.819245 19816 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:29.744577 20048 consensus_queue.cc:237] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:29.823932 16469 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:29.824582 16399 raft_consensus.cc:2410] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:23:29.814903 19978 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:29.820240 17269 raft_consensus.cc:1275] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:23:29.810359 20033 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:29.826447 17269 pending_rounds.cc:85] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8: Aborting all ops after (but not including) 4
I20260709 12:23:29.828680 16477 raft_consensus.cc:1240] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:23:29.828815 17269 pending_rounds.cc:107] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:23:29.829188 17269 raft_consensus.cc:2889] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:29.730038 19858 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:29.830227 20029 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.830711 20029 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:29.828353 16516 leader_election.cc:400] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 10. Current term is 11.
I20260709 12:23:29.832391 19959 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.750357 17261 rpcz_store.cc:276] 0709 12:23:28.253487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:28.516507 (+263020us) service_pool.cc:224] Handling call
0709 12:23:28.516898 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:28.517168 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:23:28.595201 (+ 78033us) log.cc:844] Serialized 14 byte log entry
0709 12:23:29.749040 (+1153839us) log.cc:844] Serialized 14 byte log entry
0709 12:23:29.749269 (+   229us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:29.749342 (+    73us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:29.749457 (+   115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:29.749591 (+   134us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:29.749769 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83469,"prepare.queue_time_us":216258,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":78,"raft.queue_time_us":3697,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":266,"thread_start_us":1071449,"threads_started":1,"wal-append.queue_time_us":1049174}
I20260709 12:23:29.832986 19959 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:29.833204 16516 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:29.833063 19928 consensus_queue.cc:260] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:29.807255 16549 rpcz_store.cc:276] 0709 12:23:28.521880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:28.522679 (+   799us) service_pool.cc:224] Handling call
0709 12:23:28.570553 (+ 47874us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:28.570787 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:23:29.276804 (+706017us) log.cc:844] Serialized 14 byte log entry
0709 12:23:29.606119 (+329315us) log.cc:844] Serialized 14 byte log entry
0709 12:23:29.625914 (+ 19795us) log.cc:844] Serialized 14 byte log entry
0709 12:23:29.628471 (+  2557us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:29.631961 (+  3490us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:29.720738 (+ 88777us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:29.723885 (+  3147us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:29.785913 (+ 62028us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6634,"prepare.queue_time_us":825139,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":100,"raft.queue_time_us":14473,"raft.run_cpu_time_us":494,"raft.run_wall_time_us":31478,"spinlock_wait_cycles":9160448,"thread_start_us":939369,"threads_started":2,"wal-append.queue_time_us":401450}
I20260709 12:23:29.834673 19928 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:29.833528 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:29.835382 19959 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:29.825129 16400 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:23:29.836094 16400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 1120) took 1396 ms. Trace:
I20260709 12:23:29.873298 16400 rpcz_store.cc:276] 0709 12:23:28.439872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.767231 (+1327359us) service_pool.cc:224] Handling call
0709 12:23:29.836041 (+ 68810us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:23:29.873883 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 139087us
I20260709 12:23:29.874349 20158 consensus_queue.cc:1048] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:29.848147 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:29.842571 17254 raft_consensus.cc:1275] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:29.877316 20158 consensus_queue.cc:1048] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:29.878387 16542 raft_consensus.cc:2393] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 9: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:29.880012 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:29.841503 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:29.848549 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:29.879576 20158 consensus_queue.cc:1048] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:23:29.848436 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 161770us
I20260709 12:23:29.836867 16394 raft_consensus.cc:1240] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:29.839102 16474 raft_consensus.cc:1240] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:29.895594 16468 raft_consensus.cc:1240] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:29.836678 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:29.837934 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:29.839581 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:29.886584 16399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46250 (request call id 1096) took 1738 ms. Trace:
I20260709 12:23:29.844126 19779 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:29.897300 19730 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.897686 17266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 1011) took 1644 ms. Trace:
I20260709 12:23:29.907004 16399 rpcz_store.cc:276] 0709 12:23:28.148408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.728133 (+1579725us) service_pool.cc:224] Handling call
0709 12:23:29.886486 (+158353us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:29.900821 19978 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:29.836061 19959 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:29.908928 16469 raft_consensus.cc:2410] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:23:29.910858 19730 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:29.912554 20142 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:29.913184 20142 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:29.913439 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:29.914855 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:29.915863 20142 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:29.907407 17266 rpcz_store.cc:276] 0709 12:23:28.253701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:28.653480 (+399779us) service_pool.cc:224] Handling call
0709 12:23:28.653749 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:28.653960 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:23:29.379465 (+725505us) log.cc:844] Serialized 14 byte log entry
0709 12:23:29.743574 (+364109us) log.cc:844] Serialized 14 byte log entry
0709 12:23:29.761832 (+ 18258us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:29.790059 (+ 28227us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:29.803260 (+ 13201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:29.859818 (+ 56558us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:29.897623 (+ 37805us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20683,"prepare.queue_time_us":240732,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":91,"raft.queue_time_us":7691,"raft.run_cpu_time_us":396,"raft.run_wall_time_us":72688,"thread_start_us":893694,"threads_started":2,"wal-append.queue_time_us":311568}
I20260709 12:23:29.918107 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:29.918742 17266 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:29.920025 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:29.837641 20033 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:29.887765 20127 raft_consensus.cc:2749] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:29.928444 20025 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:29.929176 20025 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:29.929606 20025 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:29.931559 20129 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:29.921645 17255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 963) took 3072 ms. Trace:
I20260709 12:23:29.921722 19941 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:29.922809 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:29.933398 20141 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:29.933799 20021 consensus_meta.cc:287] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 1.114s	user 0.011s	sys 0.015s
I20260709 12:23:29.894814 20105 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.934561 20050 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:29.935127 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:29.935350 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:29.935643 16475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1262) took 3278 ms. Trace:
W20260709 12:23:29.933794 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 119297us
W20260709 12:23:29.920979 20147 log.cc:927] Time spent T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:23:29.916110 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:29.910218 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:29.953547 17258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 982) took 1418 ms. Trace:
I20260709 12:23:29.954047 20141 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:29.939471 16549 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:29.957222 16475 rpcz_store.cc:276] 0709 12:23:26.657463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:26.657652 (+   189us) service_pool.cc:224] Handling call
0709 12:23:26.658007 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:27.599763 (+941756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:29.935151 (+2335388us) spinlock_profiling.cc:243] Waited 916 ms on lock 0x7b540080acd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:29.935444 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2106820480}
I20260709 12:23:29.935390 19858 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:29.975436 20021 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:30.028537 20074 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:30.029249 20074 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:29.980696 17261 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:29.987701 17266 raft_consensus.cc:2393] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 11: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:29.986382 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:29.857007 20132 consensus_queue.cc:1059] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.115s
I20260709 12:23:29.959560 17258 rpcz_store.cc:276] 0709 12:23:28.519682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.556604 (+1036922us) service_pool.cc:224] Handling call
0709 12:23:29.557231 (+   627us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:29.557831 (+   600us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:29.557868 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:29.558510 (+   642us) log.cc:844] Serialized 26 byte log entry
0709 12:23:29.626627 (+ 68117us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:29.626672 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:29.626741 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:29.938179 (+311438us) raft_consensus.cc:1685] finished
0709 12:23:29.938210 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:29.938379 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2404,"raft.run_cpu_time_us":380,"raft.run_wall_time_us":381,"thread_start_us":67602,"threads_started":1,"wal-append.queue_time_us":185531}
I20260709 12:23:29.992817 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:29.975142 19978 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:29.999485 20088 log.cc:927] Time spent T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 2.098s	user 0.021s	sys 0.039s
I20260709 12:23:30.002555 20073 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.002851 16366 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:30.003674 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:30.005976 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:30.039700 16469 raft_consensus.cc:2410] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:23:30.040470 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:30.007401 16474 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
I20260709 12:23:30.010706 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:29.982247 20105 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:30.011664 20050 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:30.029914 20021 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:30.013878 20129 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:30.051364 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 48653.8us
I20260709 12:23:30.016299 16367 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:30.017071 20128 log.cc:927] Time spent T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.700s	user 0.000s	sys 0.002s
I20260709 12:23:30.019239 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:30.054140 16473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 691) took 1454 ms. Trace:
I20260709 12:23:30.053216 17270 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:30.021145 17271 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
W20260709 12:23:30.026423 16550 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37226 (request call id 1355) took 1975 ms (client timeout 2384 ms). Trace:
I20260709 12:23:29.989778 20141 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:30.033265 16403 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
W20260709 12:23:30.037102 16551 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37242 (request call id 1176) took 2805 ms (client timeout 1514 ms). Trace:
I20260709 12:23:30.041424 17261 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
I20260709 12:23:30.045285 16547 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:30.013753 16546 raft_consensus.cc:1240] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:30.048043 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:30.054380 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:30.056483 16550 rpcz_store.cc:269] 0709 12:23:28.051015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:28.051150 (+   135us) service_pool.cc:224] Handling call
0709 12:23:30.025943 (+1974793us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b540069ffd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:30.026190 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30004608}
I20260709 12:23:30.061585 19911 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:30.057670 20050 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:30.062116 19911 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:30.062415 16547 raft_consensus.cc:1240] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 12:23:30.062630 19911 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:30.062947 19959 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.063838 20050 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:30.063570 19959 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:30.058910 16551 rpcz_store.cc:269] 0709 12:23:27.231252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:27.232624 (+  1372us) service_pool.cc:224] Handling call
0709 12:23:29.992492 (+2759868us) spinlock_profiling.cc:243] Waited 28.8 ms on lock 0x7b58005e9298. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64eee 000000000036eb39 00007f287add81e0 00007f287aea10c4 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:23:30.037017 (+ 44525us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":66196736}
I20260709 12:23:30.068907 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:30.060689 17258 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:23:30.070825 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:30.064193 16547 raft_consensus.cc:1275] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:30.071666 17258 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:30.074932 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:30.075812 17270 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:30.076090 19978 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:29.952617 17255 rpcz_store.cc:276] 0709 12:23:26.849405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:27.253034 (+403629us) service_pool.cc:224] Handling call
0709 12:23:27.253361 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:27.253733 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:27.253763 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:27.254267 (+   504us) log.cc:844] Serialized 26 byte log entry
0709 12:23:27.387603 (+133336us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:27.556235 (+168632us) log.cc:844] Serialized 14 byte log entry
0709 12:23:27.557904 (+  1669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:27.557994 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:29.730521 (+2172527us) spinlock_profiling.cc:243] Waited 8.6 ms on lock 0x7b18001e9b80. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:23:29.859435 (+128914us) raft_consensus.cc:1685] finished
0709 12:23:29.909242 (+ 49807us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:29.921382 (+ 12140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19496,"prepare.queue_time_us":484759,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":39611,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":17379,"spinlock_wait_cycles":19769088,"thread_start_us":274950,"threads_started":2,"wal-append.queue_time_us":311082}
I20260709 12:23:30.077062 16399 raft_consensus.cc:2393] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 9: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:30.078416 20074 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:30.066160 20073 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:30.079684 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:30.080582 17261 raft_consensus.cc:2393] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 11: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:30.080912 20073 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:30.084975 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 68827.1us
I20260709 12:23:30.090273 16542 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:30.092499 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 272406us
I20260709 12:23:30.109338 16402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 1084) took 2170 ms. Trace:
W20260709 12:23:30.111986 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:30.115597 16475 raft_consensus.cc:1275] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:30.111559 16402 rpcz_store.cc:276] 0709 12:23:27.924325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.740093 (+1815768us) service_pool.cc:224] Handling call
0709 12:23:29.745406 (+  5313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:29.795527 (+ 50121us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:23:29.909290 (+113763us) log.cc:844] Serialized 14 byte log entry
0709 12:23:30.093680 (+184390us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:30.093736 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:30.093787 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.093856 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:30.094735 (+   879us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":81608,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":60,"raft.queue_time_us":425,"thread_start_us":263248,"threads_started":2}
I20260709 12:23:30.117110 16514 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:30.123524 16475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1220) took 1024 ms. Trace:
I20260709 12:23:30.128527 16475 rpcz_store.cc:276] 0709 12:23:29.092226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:30.051103 (+958877us) service_pool.cc:224] Handling call
0709 12:23:30.092016 (+ 40913us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:30.116142 (+ 24126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.116445 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:30.132750 16474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1218) took 1041 ms. Trace:
I20260709 12:23:30.133234 16474 rpcz_store.cc:276] 0709 12:23:29.091498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:30.047041 (+955543us) service_pool.cc:224] Handling call
0709 12:23:30.095849 (+ 48808us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:30.127518 (+ 31669us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:30.127538 (+    20us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:30.132445 (+  4907us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:30.132492 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.132557 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:30.132689 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:30.130174 16367 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:30.118765 20141 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:30.123297 19978 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:30.144763 19928 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:30.145116 20144 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:30.145835 20144 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:30.148461 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:30.149667 20024 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:30.142891 19858 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:30.150408 20024 raft_consensus.cc:697] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:30.140671 17271 raft_consensus.cc:1275] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:23:30.150884 19858 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:30.158278 19858 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:30.159521 16366 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:30.161460 16366 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:30.163619 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:30.164024 16400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 1116) took 1725 ms. Trace:
I20260709 12:23:30.164615 17261 raft_consensus.cc:2468] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
I20260709 12:23:30.164572 16400 rpcz_store.cc:276] 0709 12:23:28.438771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.874564 (+1435793us) service_pool.cc:224] Handling call
0709 12:23:29.875161 (+   597us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:29.876213 (+  1052us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:29.876262 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:29.882128 (+  5866us) log.cc:844] Serialized 26 byte log entry
0709 12:23:30.045410 (+163282us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:30.045497 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.045592 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:30.163781 (+118189us) raft_consensus.cc:1685] finished
0709 12:23:30.163802 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:30.163961 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":140,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":183,"thread_start_us":108880,"threads_started":1,"wal-append.queue_time_us":203962}
I20260709 12:23:30.162356 19880 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:23:30.162725 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 69048.9us
I20260709 12:23:30.150344 16544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:59590 (request call id 13) took 2245 ms. Trace:
I20260709 12:23:30.169617 19959 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:30.171602 16545 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37242 (request call id 1161) took 3364 ms (client timeout 1624 ms). Trace:
I20260709 12:23:30.172454 20142 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:30.175263 16477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1291) took 1737 ms. Trace:
I20260709 12:23:30.175817 16442 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:30.176425 17269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 1017) took 1872 ms. Trace:
I20260709 12:23:30.178129 20105 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:30.178583 20141 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:30.232825 17269 rpcz_store.cc:276] 0709 12:23:28.303740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.530908 (+1227168us) service_pool.cc:224] Handling call
0709 12:23:29.545395 (+ 14487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:29.829848 (+284453us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.176146 (+346298us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b54005f3554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:30.176363 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":631011072}
I20260709 12:23:30.178903 20029 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:30.179411 19978 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:30.142607 20057 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:30.180981 16472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1278) took 2901 ms. Trace:
I20260709 12:23:30.249132 16469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1219) took 1154 ms. Trace:
I20260709 12:23:30.162415 19779 consensus_queue.cc:1059] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 12:23:30.181010 20024 consensus_queue.cc:237] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:30.143765 20025 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:30.249809 16469 rpcz_store.cc:276] 0709 12:23:29.091868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:30.048838 (+956970us) service_pool.cc:224] Handling call
0709 12:23:30.072516 (+ 23678us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:30.073084 (+   568us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:30.073118 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:30.073696 (+   578us) log.cc:844] Serialized 26 byte log entry
0709 12:23:30.132163 (+ 58467us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:30.132206 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.132281 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:30.245607 (+113326us) raft_consensus.cc:1685] finished
0709 12:23:30.245638 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:30.245838 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1534,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":97,"thread_start_us":57978,"threads_started":1,"wal-append.queue_time_us":94526}
I20260709 12:23:30.185442 20094 consensus_queue.cc:1059] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:23:30.186977 17254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 974) took 2122 ms. Trace:
I20260709 12:23:30.154162 20144 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:30.188520 19487 consensus_queue.cc:1059] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
W20260709 12:23:30.232506 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:30.240396 16542 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14.
I20260709 12:23:30.232106 16477 rpcz_store.cc:276] 0709 12:23:28.437424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:28.574106 (+136682us) service_pool.cc:224] Handling call
0709 12:23:28.574422 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:29.829173 (+1254751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.174878 (+345705us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b5400a690d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:30.175185 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2865313024}
I20260709 12:23:30.240895 17270 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:30.168530 16544 rpcz_store.cc:276] 0709 12:23:27.905236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:27.905397 (+   161us) service_pool.cc:224] Handling call
0709 12:23:30.149940 (+2244543us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b540069ffd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e83853 00007f287ae5160a 00007f28743d77d4 00007f28743d7762 00007f28743d7691 00007f28743d760c 00007f28743d68c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:30.150113 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":325492480}
W20260709 12:23:30.233707 16545 rpcz_store.cc:269] 0709 12:23:26.806887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:26.807343 (+   456us) service_pool.cc:224] Handling call
0709 12:23:30.171249 (+3363906us) spinlock_profiling.cc:243] Waited 999 ms on lock 0x7b540061eed4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:30.171421 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2298160896}
W20260709 12:23:30.258234 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:30.185681 16475 raft_consensus.cc:1275] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:23:30.253121 17255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 998) took 1651 ms. Trace:
I20260709 12:23:30.258841 17255 rpcz_store.cc:276] 0709 12:23:28.599082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:30.077309 (+1478227us) service_pool.cc:224] Handling call
0709 12:23:30.077775 (+   466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:30.078238 (+   463us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:30.078266 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:30.078871 (+   605us) log.cc:844] Serialized 26 byte log entry
0709 12:23:30.125721 (+ 46850us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:30.125772 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.125869 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:30.250074 (+124205us) raft_consensus.cc:1685] finished
0709 12:23:30.250094 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:30.250232 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":36692,"threads_started":1,"wal-append.queue_time_us":102124}
W20260709 12:23:30.264921 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 103573us
I20260709 12:23:30.266640 20055 consensus_queue.cc:1048] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 12:23:30.267980 20025 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:30.268226 20142 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:30.250937 20057 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:30.270035 20057 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:30.271535 20166 raft_consensus.cc:3055] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:30.272027 20166 raft_consensus.cc:740] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:30.249066 16472 rpcz_store.cc:276] 0709 12:23:27.279362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:27.280301 (+   939us) service_pool.cc:224] Handling call
0709 12:23:27.280668 (+   367us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:27.290652 (+  9984us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:27.290683 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:27.309410 (+ 18727us) log.cc:844] Serialized 52 byte log entry
0709 12:23:27.442937 (+133527us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:27.442997 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:27.443088 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:29.998476 (+2555388us) spinlock_profiling.cc:243] Waited 7.75 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:23:30.180424 (+181948us) raft_consensus.cc:1685] finished
0709 12:23:30.180455 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:30.180798 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9586,"raft.queue_time_us":10159,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":7469,"spinlock_wait_cycles":17828096,"thread_start_us":75357,"threads_started":1,"wal-append.queue_time_us":506733}
I20260709 12:23:30.256604 17254 rpcz_store.cc:276] 0709 12:23:28.064239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:28.064409 (+   170us) service_pool.cc:224] Handling call
0709 12:23:28.064722 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:29.877412 (+1812690us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.186671 (+309259us) spinlock_profiling.cc:243] Waited 1.78 s on lock 0x7b54005f4454. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:30.186883 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4087281920}
I20260709 12:23:30.272990 20166 consensus_queue.cc:260] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:30.273927 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:30.274636 20166 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:30.276136 16548 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37242 (request call id 1187) took 2941 ms (client timeout 1883 ms). Trace:
I20260709 12:23:30.247661 19941 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.269791 16394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 1102) took 1840 ms. Trace:
I20260709 12:23:30.278282 16394 rpcz_store.cc:276] 0709 12:23:28.429627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.818811 (+1389184us) service_pool.cc:224] Handling call
0709 12:23:29.836627 (+ 17816us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:29.888860 (+ 52233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.269585 (+380725us) spinlock_profiling.cc:243] Waited 17.4 ms on lock 0x7b54004ecfd0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:30.269728 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40056064}
W20260709 12:23:30.280825 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 169065us
W20260709 12:23:30.276700 16548 rpcz_store.cc:269] 0709 12:23:27.334763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:27.442144 (+107381us) service_pool.cc:224] Handling call
0709 12:23:30.275826 (+2833682us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b540037f854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:30.276047 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3724984704}
I20260709 12:23:30.272984 20094 raft_consensus.cc:3055] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:30.287209 20094 raft_consensus.cc:740] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:30.283010 19941 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:30.290011 20094 consensus_queue.cc:260] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:30.287956 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 112316us
I20260709 12:23:30.285615 20141 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:30.292212 16442 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:30.263794 20073 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:30.295329 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:30.296936 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:30.297122 17258 raft_consensus.cc:1275] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:30.291285 20094 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:30.296030 16475 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:30.239709 19911 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:30.260416 19730 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:30.304414 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:30.308101 16471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1277) took 3486 ms. Trace:
I20260709 12:23:30.288218 20067 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.313659 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:30.315102 16394 raft_consensus.cc:1240] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:30.311085 16471 rpcz_store.cc:276] 0709 12:23:26.790563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:27.225728 (+435165us) service_pool.cc:224] Handling call
0709 12:23:27.226295 (+   567us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:27.473473 (+247178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.191994 (+2718521us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b540080bbd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:30.276570 (+ 84576us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":414485376}
I20260709 12:23:30.325009 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:30.325476 16367 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:30.326078 16400 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:30.337319 17263 raft_consensus.cc:1275] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:23:30.333271 16442 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:30.312279 19969 consensus_queue.cc:1059] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 12:23:30.344326 19969 raft_consensus.cc:3055] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:30.345239 19969 raft_consensus.cc:740] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:30.346359 19969 consensus_queue.cc:260] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:30.341684 17263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 1016) took 2055 ms. Trace:
I20260709 12:23:30.347963 17263 rpcz_store.cc:276] 0709 12:23:28.285970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:30.314897 (+2028927us) service_pool.cc:224] Handling call
0709 12:23:30.336819 (+ 21922us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:30.339767 (+  2948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.341611 (+  1844us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:23:30.334331 20176 log.cc:927] Time spent T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:23:30.327929 19941 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:30.353760 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:30.354581 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 66585.6us
I20260709 12:23:30.332244 20173 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:30.357811 16441 leader_election.cc:336] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491): Timed out: RequestConsensusVote RPC to 127.5.58.4:44491 timed out after 1.750s (SENT)
I20260709 12:23:30.335707 19911 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:30.361249 19969 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:30.352126 19837 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:30.364698 19837 raft_consensus.cc:697] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:30.365974 19837 consensus_queue.cc:237] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:30.352198 19849 raft_consensus.cc:2764] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:23:30.328505 20067 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:30.370393 16397 raft_consensus.cc:1275] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:23:30.371467 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:30.372895 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:30.373128 16402 raft_consensus.cc:1240] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:30.374176 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:30.380852 16399 raft_consensus.cc:2393] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 15: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:30.379305 19858 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:30.383522 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:30.382929 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:30.384418 16475 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:30.386591 16402 raft_consensus.cc:2410] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 12:23:30.387187 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 66547.5us
I20260709 12:23:30.388222 16367 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:30.389990 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:30.382123 19858 raft_consensus.cc:697] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 9 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:30.396713 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:30.405298 16400 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:30.401880 16366 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:30.415776 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:30.418025 16394 raft_consensus.cc:2393] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 12: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:30.413671 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:30.420980 16398 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46250 (request call id 1088) took 2396 ms (client timeout 1554 ms). Trace:
W20260709 12:23:30.421711 16398 rpcz_store.cc:269] 0709 12:23:27.978162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:28.090767 (+112605us) service_pool.cc:224] Handling call
0709 12:23:30.372189 (+2281422us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b540015ed54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:30.374162 (+  1973us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":430593792}
W20260709 12:23:30.428876 20175 log.cc:927] Time spent T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 12:23:30.432291 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 81331.8us
W20260709 12:23:30.434468 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 57842.9us
W20260709 12:23:30.435547 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:30.436276 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:30.438035 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:30.440584 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:30.443594 16402 raft_consensus.cc:2410] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:23:30.414340 19911 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:30.446714 16400 raft_consensus.cc:1275] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:23:30.380939 20173 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:30.451310 20180 log.cc:927] Time spent T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 12:23:30.465157 19730 consensus_queue.cc:1059] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:30.437191 20143 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:30.470556 16366 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:30.456498 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:30.447683 20073 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:30.487960 16544 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 6.
I20260709 12:23:30.484236 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:30.503435 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:30.504532 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:30.503103 16366 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:30.485460 19911 raft_consensus.cc:697] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 14 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
W20260709 12:23:30.518510 20182 log.cc:927] Time spent T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:23:30.518560 19487 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:30.519395 19487 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:30.519235 19911 consensus_queue.cc:237] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:30.517761 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:30.521276 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:30.522027 16400 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:23:30.522976 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:30.523533 16400 raft_consensus.cc:2376] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7. Re-sending same reply.
I20260709 12:23:30.513967 19849 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:30.525457 19858 consensus_queue.cc:237] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:30.525945 16514 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:30.519912 19487 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:23:30.529034 20131 log.cc:927] Time spent T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.978s	user 0.004s	sys 0.000s
I20260709 12:23:30.529626 16367 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:30.531739 20055 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:30.532385 20055 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:30.532846 20055 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:30.533428 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:30.534008 20173 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:30.534930 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:30.538175 16470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1282) took 3182 ms. Trace:
W20260709 12:23:30.538296 16441 leader_election.cc:336] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491): Timed out: RequestConsensusVote RPC to 127.5.58.4:44491 timed out after 1.858s (SENT)
I20260709 12:23:30.538829 16441 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:30.489928 20025 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.538823 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:30.545436 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:30.546483 16400 raft_consensus.cc:2393] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 9: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:30.540191 16514 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:30.517462 20070 consensus_queue.cc:1059] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:23:30.536216 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:30.550091 16474 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:23:30.547859 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:30.551178 16442 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:30.515920 20073 raft_consensus.cc:697] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:30.568919 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:30.579546 16397 raft_consensus.cc:1275] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 12:23:30.538682 16470 rpcz_store.cc:276] 0709 12:23:27.355643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:27.358933 (+  3290us) service_pool.cc:224] Handling call
0709 12:23:27.359219 (+   286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:27.359527 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:27.359555 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:28.439234 (+1079679us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b2c0017a228. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:23:28.439659 (+   425us) log.cc:844] Serialized 26 byte log entry
0709 12:23:29.320231 (+880572us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:29.320307 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:29.320432 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:30.528412 (+1207980us) spinlock_profiling.cc:243] Waited 26.1 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:23:30.537889 (+  9477us) raft_consensus.cc:1685] finished
0709 12:23:30.537912 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:30.538116 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5485,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":99,"spinlock_wait_cycles":83523072,"thread_start_us":879938,"threads_started":1,"wal-append.queue_time_us":1110850}
I20260709 12:23:30.584486 16544 raft_consensus.cc:1240] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:30.580498 16397 pending_rounds.cc:85] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 5
I20260709 12:23:30.585886 16397 pending_rounds.cc:107] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 12:23:30.586225 16397 raft_consensus.cc:2889] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:30.569872 16477 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
W20260709 12:23:30.585206 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 82186.7us
W20260709 12:23:30.589766 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:30.590445 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:30.591943 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:30.609915 16476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 683) took 2334 ms. Trace:
I20260709 12:23:30.562809 20025 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:30.579121 20048 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:30.614583 20048 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:30.615018 20048 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:30.616219 19487 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:30.610455 16476 rpcz_store.cc:276] 0709 12:23:28.233954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:28.263394 (+ 29440us) service_pool.cc:224] Handling call
0709 12:23:28.271612 (+  8218us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:28.350577 (+ 78965us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.506421 (+2155844us) spinlock_profiling.cc:243] Waited 7.78 ms on lock 0x7b5400977f50. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:30.568519 (+ 62098us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17891200}
I20260709 12:23:30.569039 20143 raft_consensus.cc:697] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:30.579497 20108 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:30.624032 20070 consensus_queue.cc:1059] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:30.626585 16470 raft_consensus.cc:1240] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:30.626338 20189 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:30.610016 19931 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:30.565305 19941 consensus_queue.cc:1059] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
W20260709 12:23:30.646646 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 95595.7us
W20260709 12:23:30.662175 16516 leader_election.cc:336] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 2.740s (SENT)
I20260709 12:23:30.662809 16516 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:30.664844 20073 consensus_queue.cc:237] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:30.666141 19959 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:30.669343 19941 raft_consensus.cc:3055] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:30.669652 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:30.669970 19941 raft_consensus.cc:740] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:30.670567 20189 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:30.671027 19941 consensus_queue.cc:260] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:30.673419 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:30.671068 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:30.639591 19931 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:30.679600 16399 raft_consensus.cc:1240] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 14 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:30.651468 16442 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:30.681254 16397 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:30.692008 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:30.672297 17144 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:30.671198 20189 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:30.633992 20057 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:30.637449 20108 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:30.674314 19487 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:30.690269 19931 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:30.672595 20029 consensus_queue.cc:1059] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:23:30.640273 19730 consensus_queue.cc:1048] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 12:23:30.672904 20105 consensus_queue.cc:1059] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
W20260709 12:23:30.723452 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 75968.4us
I20260709 12:23:30.734079 16394 raft_consensus.cc:2393] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 12: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
W20260709 12:23:30.737160 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 198900us
I20260709 12:23:30.705143 20189 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:30.742462 20055 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:30.744845 20199 raft_consensus.cc:2804] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:30.745178 19941 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:30.757568 20143 consensus_queue.cc:237] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:30.775279 20187 log.cc:927] Time spent T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:23:30.777959 20108 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:30.778863 20108 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:30.782955 17264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 983) took 2263 ms. Trace:
I20260709 12:23:30.783607 17264 rpcz_store.cc:276] 0709 12:23:28.520015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.791574 (+1271559us) service_pool.cc:224] Handling call
0709 12:23:29.802199 (+ 10625us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:29.810319 (+  8120us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:29.810361 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:29.810988 (+   627us) log.cc:844] Serialized 26 byte log entry
0709 12:23:30.523015 (+712027us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b1800441e20. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:23:30.635366 (+112351us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:30.635426 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.636734 (+  1308us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:30.782539 (+145805us) raft_consensus.cc:1685] finished
0709 12:23:30.782569 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:30.782869 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4044,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":3779,"spinlock_wait_cycles":25314176,"thread_start_us":99458,"threads_started":1,"wal-append.queue_time_us":918028}
W20260709 12:23:30.793064 16441 leader_election.cc:336] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491): Timed out: RequestConsensusVote RPC to 127.5.58.4:44491 timed out after 1.958s (SENT)
I20260709 12:23:30.793561 20025 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:30.795265 16441 leader_election.cc:336] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491): Timed out: RequestConsensusVote RPC to 127.5.58.4:44491 timed out after 2.145s (SENT)
I20260709 12:23:30.798449 20048 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:30.801421 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:30.804199 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:30.809990 16397 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
W20260709 12:23:30.863317 20195 log.cc:927] Time spent T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:23:30.800829 20199 raft_consensus.cc:697] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:30.865676 20199 consensus_queue.cc:237] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:30.869458 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:30.870332 16475 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:30.873682 20021 consensus_queue.cc:1059] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:30.874943 20021 raft_consensus.cc:3055] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:30.875311 20021 raft_consensus.cc:740] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:30.875505 20057 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:30.876570 20021 consensus_queue.cc:260] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:30.878254 20021 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:30.880733 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:30.881405 16397 raft_consensus.cc:2376] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11. Re-sending same reply.
I20260709 12:23:30.882314 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:30.883390 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:23:30.889547 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:30.745635 20197 consensus_queue.cc:1048] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.241s
W20260709 12:23:30.996162 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:31.024358 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 352087us
W20260709 12:23:31.031657 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:31.054598 16397 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
I20260709 12:23:31.066485 16549 raft_consensus.cc:1240] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 12:23:31.067577 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:31.069109 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:31.075552 16367 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.1:41805 timed out after 2.080s (SENT) [suppressed 43 similar messages]
W20260709 12:23:31.076452 20088 log.cc:927] Time spent T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:23:31.088457 16544 raft_consensus.cc:1240] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:31.088816 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:31.089242 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:31.004673 20166 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:31.097146 20166 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:31.093359 16515 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.235s	user 0.011s	sys 0.020s
W20260709 12:23:31.100688 16515 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.242s	user 0.012s	sys 0.020s
W20260709 12:23:31.101058 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 1.24256e+06us
I20260709 12:23:31.077142 20073 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.095485 20198 consensus_queue.cc:1048] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:23:31.096521 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:31.103053 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:31.088425 20048 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:31.100229 20166 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:31.113899 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:31.116402 20201 log.cc:927] Time spent T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 12:23:31.116740 20029 consensus_queue.cc:1048] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:31.117066 20055 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:31.109454 20073 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:31.118484 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:31.120580 19699 consensus_queue.cc:1048] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:31.121389 20048 consensus_queue.cc:1048] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:30.054711 16473 rpcz_store.cc:276] 0709 12:23:28.593114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.627596 (+1034482us) service_pool.cc:224] Handling call
0709 12:23:29.722208 (+ 94612us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:29.885786 (+163578us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:23:29.925159 (+ 39373us) log.cc:844] Serialized 14 byte log entry
0709 12:23:30.040235 (+115076us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:30.040285 (+    50us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:30.047056 (+  6771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.047156 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:30.047281 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":19578,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":55,"raft.queue_time_us":3438,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":6497,"thread_start_us":112280,"threads_started":1}
I20260709 12:23:31.126277 19978 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:31.121392 19764 consensus_queue.cc:1048] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 12:23:31.131654 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:31.133181 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:31.139758 16544 raft_consensus.cc:2393] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 11: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:31.145681 16516 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:31.148345 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:31.167004 16549 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:31.166999 16394 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:31.172293 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:31.185292 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:31.145166 19959 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:31.192889 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:31.193022 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:31.202333 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:31.209419 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:31.183482 20094 consensus_meta.cc:287] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.875s	user 0.007s	sys 0.008s
I20260709 12:23:31.214170 16473 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:31.215641 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:31.216521 16473 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:23:31.218045 16366 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:31.223215 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 158099us
I20260709 12:23:31.222959 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:31.224478 16473 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:31.227175 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:31.194430 19858 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.005s	sys 0.032s
I20260709 12:23:31.219345 19764 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:23:31.235188 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 92476.8us
I20260709 12:23:31.225729 20025 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:31.236227 17144 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:31.239950 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 118059us
I20260709 12:23:31.242554 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:23:31.232898 19858 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.005s	sys 0.032s
I20260709 12:23:31.243788 16477 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:31.237334 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:31.248807 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 173273us
I20260709 12:23:31.251073 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:31.251852 16470 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:23:31.239501 20025 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:31.260385 16516 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:31.268615 16400 raft_consensus.cc:1240] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:23:31.247516 19764 raft_consensus.cc:697] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:23:31.271754 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:31.259953 20025 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:31.276607 16516 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:31.277931 16400 raft_consensus.cc:1240] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 10. Current term is 11. Ops: []
W20260709 12:23:31.279110 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 48745.1us
I20260709 12:23:31.282785 17144 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:31.286403 20204 log.cc:927] Time spent T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:23:31.262663 20205 consensus_queue.cc:1048] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.130s
I20260709 12:23:31.261881 20105 consensus_queue.cc:1059] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 12:23:31.293300 16400 raft_consensus.cc:1240] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:31.275776 20021 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:23:31.308699 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:31.302476 16516 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:31.275370 20217 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:31.309718 19806 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:23:31.316722 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 170631us
I20260709 12:23:31.318120 20021 raft_consensus.cc:697] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:31.319456 19764 consensus_queue.cc:237] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:31.324764 20138 log.cc:927] Time spent T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 1.773s	user 0.000s	sys 0.014s
I20260709 12:23:31.329877 16475 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
W20260709 12:23:31.369180 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 91081.9us
W20260709 12:23:31.328718 20211 log.cc:927] Time spent T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
W20260709 12:23:31.370756 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 91062.4us
I20260709 12:23:31.373289 16473 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:23:31.372481 16366 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:31.376093 16366 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:31.380084 17144 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:31.382673 17142 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:31.310420 19779 consensus_queue.cc:1059] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 12:23:31.390690 16475 raft_consensus.cc:1275] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:23:31.382853 20055 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:31.435041 16468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1213) took 2346 ms. Trace:
W20260709 12:23:31.411803 16403 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46250 (request call id 1098) took 3087 ms (client timeout 2225 ms). Trace:
W20260709 12:23:31.480090 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:31.490321 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:31.542286 16403 rpcz_store.cc:269] 0709 12:23:28.324626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.628121 (+1303495us) service_pool.cc:224] Handling call
0709 12:23:31.411537 (+1783416us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b54009bc8d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:31.411688 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":631113472}
I20260709 12:23:31.511245 20073 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:31.512028 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:31.512104 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:31.512730 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:31.513283 19730 raft_consensus.cc:3055] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:31.513545 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:31.514358 17263 raft_consensus.cc:1275] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:23:31.514271 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:31.451606 19943 raft_consensus.cc:3055] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:31.463115 16396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 1085) took 3524 ms. Trace:
I20260709 12:23:31.445406 16401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 943) took 3523 ms. Trace:
I20260709 12:23:31.519477 16516 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:31.551765 17264 raft_consensus.cc:2410] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:23:31.578837 17269 raft_consensus.cc:2468] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
W20260709 12:23:31.408725 16543 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37242 (request call id 1174) took 4178 ms (client timeout 1958 ms). Trace:
W20260709 12:23:31.426337 20113 log.cc:927] Time spent T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 1.157s	user 0.009s	sys 0.005s
W20260709 12:23:31.509578 20189 consensus_meta.cc:287] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.729s	user 0.008s	sys 0.008s
I20260709 12:23:31.543702 16468 rpcz_store.cc:276] 0709 12:23:29.089339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.708288 (+618949us) service_pool.cc:224] Handling call
0709 12:23:29.740586 (+ 32298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:29.907975 (+167389us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:31.434795 (+1526820us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b5400a69fd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:31.434981 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":274624256}
I20260709 12:23:31.543994 17254 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 7.
I20260709 12:23:31.444192 20028 raft_consensus.cc:3055] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:31.544986 17271 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
I20260709 12:23:31.592247 17271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45224 (request call id 1010) took 1414 ms. Trace:
I20260709 12:23:31.592751 17271 rpcz_store.cc:276] 0709 12:23:30.177736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:31.512462 (+1334726us) service_pool.cc:224] Handling call
0709 12:23:31.592170 (+ 79708us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:31.575636 16396 rpcz_store.cc:276] 0709 12:23:27.938768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.750042 (+1811274us) service_pool.cc:224] Handling call
0709 12:23:29.805943 (+ 55901us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:29.810448 (+  4505us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:31.462882 (+1652434us) spinlock_profiling.cc:243] Waited 53.2 ms on lock 0x7b54004ec5d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:31.463024 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":122256640}
I20260709 12:23:31.394372 20203 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:23:31.441650 20169 log.cc:927] Time spent T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 1.176s	user 0.004s	sys 0.006s
W20260709 12:23:31.661573 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 66751.6us
W20260709 12:23:31.440246 20177 log.cc:927] Time spent T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 1.095s	user 0.007s	sys 0.005s
W20260709 12:23:31.580089 16543 rpcz_store.cc:269] 0709 12:23:27.230467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:27.256437 (+ 25970us) service_pool.cc:224] Handling call
0709 12:23:31.408189 (+4151752us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400b13354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:31.408516 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2493168512}
I20260709 12:23:31.446041 20108 raft_consensus.cc:3055] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:31.576543 16401 rpcz_store.cc:276] 0709 12:23:27.921784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:27.925462 (+  3678us) service_pool.cc:224] Handling call
0709 12:23:27.940736 (+ 15274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:28.189051 (+248315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:31.444822 (+3255771us) spinlock_profiling.cc:243] Waited 248 ms on lock 0x7b54005e8dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:31.445176 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":569413376}
I20260709 12:23:31.393289 19935 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:31.403797 20188 log.cc:927] Time spent T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.783s	user 0.002s	sys 0.009s
I20260709 12:23:31.436952 19779 consensus_queue.cc:1059] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.229s
I20260709 12:23:31.419816 19908 raft_consensus.cc:3055] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:31.394083 19937 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:31.431898 20067 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:31.424701 19858 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:31.443466 20208 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:31.575330 19943 raft_consensus.cc:740] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:31.374752 20160 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:31.578194 20055 raft_consensus.cc:697] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:31.670188 20108 raft_consensus.cc:740] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:31.418939 19894 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:31.584987 19730 raft_consensus.cc:740] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:23:31.688187 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 305572us
I20260709 12:23:31.660884 20028 raft_consensus.cc:740] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:31.729203 17142 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:23:31.696115 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:31.678097 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 158725us
I20260709 12:23:31.674880 20203 raft_consensus.cc:697] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:31.722833 16477 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:31.414114 20021 consensus_queue.cc:237] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:31.723680 16549 consensus_meta.cc:287] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.547s	user 0.006s	sys 0.005s
I20260709 12:23:31.736524 16549 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
W20260709 12:23:31.726312 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 352499us
I20260709 12:23:31.687641 19908 raft_consensus.cc:740] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
W20260709 12:23:31.739082 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 354447us
I20260709 12:23:31.730804 20105 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:31.658748 20189 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:31.741268 16367 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:31.719131 20160 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:31.741254 16514 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
W20260709 12:23:31.704156 19931 consensus_meta.cc:287] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.967s	user 0.008s	sys 0.005s
I20260709 12:23:31.756608 19771 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:31.747239 20220 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:31.757050 16403 raft_consensus.cc:1275] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:23:31.703384 19937 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:31.764814 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:31.784150 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:31.784759 16549 raft_consensus.cc:2410] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:23:31.723044 19858 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:31.787820 16402 raft_consensus.cc:1275] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:23:31.793169 19730 consensus_queue.cc:260] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:31.795042 19730 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:31.796547 19853 consensus_queue.cc:1059] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:31.796656 19943 consensus_queue.cc:260] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:31.797550 19853 consensus_queue.cc:1243] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer f99066d13e524238aec6aa2024a8bd50 log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:23:31.797988 19943 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:31.798115 20141 raft_consensus.cc:3055] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:31.798579 20141 raft_consensus.cc:740] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:31.799539 20141 consensus_queue.cc:260] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:31.747601 20160 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:31.751768 19779 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:31.756657 19699 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:31.729265 20208 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:31.813659 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:23:31.815373 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 77890.8us
I20260709 12:23:31.838205 16543 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:23:31.818831 20220 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:31.818461 20108 consensus_queue.cc:260] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:31.833010 19699 raft_consensus.cc:697] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 10 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:31.760042 19931 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:31.844666 16550 raft_consensus.cc:1275] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:23:31.847568 16367 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:31.856855 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:31.839395 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: true dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:31.784471 20048 consensus_queue.cc:1059] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 12:23:31.862039 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:31.804664 20229 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:31.897804 16549 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:31.843598 17266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 997) took 3245 ms. Trace:
I20260709 12:23:31.815274 20141 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:31.881397 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 43870us
I20260709 12:23:31.903137 16548 raft_consensus.cc:1240] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:31.917305 17144 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:31.917838 16542 raft_consensus.cc:1275] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:23:31.834129 20055 consensus_queue.cc:237] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:31.906265 19858 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:31.907604 19915 consensus_queue.cc:1048] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:31.931854 20228 consensus_queue.cc:1048] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 12:23:31.909013 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:31.933779 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:31.934446 16550 raft_consensus.cc:2393] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 10: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:31.934782 20208 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:31.909854 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:31.859324 20189 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:31.935469 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:31.936915 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:31.814344 20167 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:31.915360 17266 rpcz_store.cc:276] 0709 12:23:28.598722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:30.065716 (+1466994us) service_pool.cc:224] Handling call
0709 12:23:30.070448 (+  4732us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:30.070982 (+   534us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:30.071013 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:30.071804 (+   791us) log.cc:844] Serialized 52 byte log entry
0709 12:23:30.123022 (+ 51218us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:30.123074 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.123147 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:31.439404 (+1316257us) spinlock_profiling.cc:243] Waited 4.54 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:23:31.817230 (+377826us) raft_consensus.cc:1685] finished
0709 12:23:31.817262 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:31.843535 (+ 26273us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8202,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":136,"spinlock_wait_cycles":10446208,"thread_start_us":50765,"threads_started":1,"wal-append.queue_time_us":228639}
I20260709 12:23:31.940842 20167 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:31.845010 19891 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.922696 20229 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:31.948132 20167 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:31.948757 19891 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:31.951088 19891 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:31.936233 16550 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:31.953578 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:31.863461 20227 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:31.955085 20227 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:31.876271 20217 consensus_meta.cc:287] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.507s	user 0.008s	sys 0.004s
I20260709 12:23:31.800431 19908 consensus_queue.cc:260] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:31.931666 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:31.932103 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:23:31.861261 19941 consensus_meta.cc:287] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 1.095s	user 0.007s	sys 0.007s
W20260709 12:23:31.923785 16470 consensus_meta.cc:287] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.668s	user 0.008s	sys 0.003s
I20260709 12:23:32.005225 16470 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:31.935602 20208 consensus_queue.cc:1048] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:23:31.845148 20067 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.910593 19931 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:31.940605 16546 raft_consensus.cc:1240] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:23:32.024626 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:31.846153 20220 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:31.948876 20229 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:32.025696 16550 raft_consensus.cc:2393] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 10: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
W20260709 12:23:32.006341 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 158865us
I20260709 12:23:31.855515 20108 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:32.016781 20217 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:32.017146 20157 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:32.022059 19891 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:23:32.078123 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 337105us
I20260709 12:23:32.084657 16549 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:32.105643 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:31.838485 20028 consensus_queue.cc:260] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:31.912034 20203 consensus_queue.cc:237] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:32.112818 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:32.029898 20158 raft_consensus.cc:3055] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:32.033377 20220 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:32.031179 20067 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:32.130090 17142 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:32.131808 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:32.114176 19891 raft_consensus.cc:697] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:32.124042 20217 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:32.132602 16550 raft_consensus.cc:2376] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9. Re-sending same reply.
I20260709 12:23:31.858189 19699 consensus_queue.cc:237] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:32.146124 20029 consensus_queue.cc:1059] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:32.149147 16543 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:32.104321 20157 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:32.155573 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:32.157481 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:32.136369 20025 consensus_meta.cc:287] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.851s	user 0.006s	sys 0.007s
W20260709 12:23:32.162722 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 244153us
I20260709 12:23:32.129971 20158 raft_consensus.cc:740] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:32.209302 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:32.210065 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 144769us
I20260709 12:23:32.215477 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:32.216423 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:32.215166 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:32.219202 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:32.224318 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:32.224586 16395 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
W20260709 12:23:32.225912 20233 log.cc:927] Time spent T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 12:23:32.228554 20234 log.cc:927] Time spent T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.116s	user 0.001s	sys 0.001s
I20260709 12:23:32.229192 20028 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:32.233209 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:32.233935 17255 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:32.206964 20227 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:32.206245 20025 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:32.228775 19891 consensus_queue.cc:237] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:32.234364 17266 raft_consensus.cc:1275] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:23:32.236830 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:32.228300 17264 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2"
dest_uuid: "ef57beab0e1d40d585983873c55107e8"
mode: GRACEFUL
new_leader_uuid: "f99066d13e524238aec6aa2024a8bd50"
 from {username='slave'} at 127.0.0.1:45120
I20260709 12:23:32.240551 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:32.241039 16472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1244) took 1043 ms. Trace:
I20260709 12:23:32.246307 16472 rpcz_store.cc:276] 0709 12:23:31.197794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:31.464541 (+266747us) service_pool.cc:224] Handling call
0709 12:23:31.464906 (+   365us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:31.465411 (+   505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:31.465443 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:31.466169 (+   726us) log.cc:844] Serialized 52 byte log entry
0709 12:23:31.751835 (+285666us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:31.751881 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:31.751963 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:32.240798 (+488835us) raft_consensus.cc:1685] finished
0709 12:23:32.240822 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:32.240980 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":125,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":101,"thread_start_us":285257,"threads_started":1,"wal-append.queue_time_us":584372}
I20260709 12:23:32.245947 17264 raft_consensus.cc:606] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Received request to transfer leadership to TS f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:32.249024 17264 raft_consensus.cc:614] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:32.252988 20236 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:32.253139 19892 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:32.254312 16547 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
I20260709 12:23:32.258675 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:32.259712 16547 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:32.236569 19937 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:32.257371 20024 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:32.270960 19908 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:32.246524 20025 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:32.272064 19892 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:32.286105 17261 raft_consensus.cc:1275] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:23:32.236624 20227 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:32.276221 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:32.267951 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:32.279924 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:32.294181 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:32.241281 20217 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:32.295323 16543 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:32.295578 20217 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:32.296101 20217 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:32.297401 16543 raft_consensus.cc:1275] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:23:32.275105 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.238031 20157 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:32.290509 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:23:32.262516 20235 log.cc:927] Time spent T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:23:32.248206 20208 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:32.304033 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:32.262210 20142 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:32.302843 17266 raft_consensus.cc:3055] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:32.294199 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:32.294770 16549 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 7.
W20260709 12:23:32.292078 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 442225us
I20260709 12:23:32.327287 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:32.311988 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:32.329556 16547 raft_consensus.cc:2410] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:23:32.329722 17266 raft_consensus.cc:740] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:32.304919 17261 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
W20260709 12:23:32.320657 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:32.335363 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:32.340420 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.345227 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:32.337260 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:23:32.312124 19935 consensus_meta.cc:287] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.626s	user 0.010s	sys 0.001s
I20260709 12:23:32.350934 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:32.327378 19880 raft_consensus.cc:3055] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:32.305989 19937 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:32.365286 16400 raft_consensus.cc:1240] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:32.309145 20230 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:32.352203 16399 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:32.263139 20236 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:32.367739 16545 raft_consensus.cc:1275] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:23:32.352813 20160 consensus_meta.cc:287] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.531s	user 0.010s	sys 0.000s
I20260709 12:23:32.377723 16549 raft_consensus.cc:1275] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:23:32.378618 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:32.284125 20024 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:32.383769 16516 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:32.382321 19969 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:32.304086 20227 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:32.392616 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:32.328269 20208 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:32.413895 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 75342.7us
W20260709 12:23:32.380643 19943 consensus_meta.cc:287] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.582s	user 0.010s	sys 0.002s
I20260709 12:23:32.344902 20142 consensus_queue.cc:1048] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:23:32.382098 19937 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:32.367640 20116 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:32.367537 19880 raft_consensus.cc:740] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:32.397413 20173 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:32.392793 20067 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:32.327545 20158 consensus_queue.cc:260] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:32.415973 19931 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:32.443912 19931 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:32.446679 16550 raft_consensus.cc:1240] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:32.447170 19931 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:32.448279 20249 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:32.448882 20243 log.cc:927] Time spent T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 12:23:32.449141 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:32.448972 20249 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:32.450699 20209 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:32.451371 20209 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:32.453652 20227 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:32.454903 16367 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:32.456521 19894 consensus_queue.cc:1048] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:32.457811 19943 consensus_queue.cc:1048] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:32.384629 20230 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:32.460932 19892 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:32.461947 20230 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:32.415030 19969 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:32.462168 16545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1378) took 2024 ms. Trace:
I20260709 12:23:32.462677 16545 rpcz_store.cc:276] 0709 12:23:30.437621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:30.438005 (+   384us) service_pool.cc:224] Handling call
0709 12:23:30.438293 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:32.377600 (+1939307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:32.461911 (+ 84311us) spinlock_profiling.cc:243] Waited 1.92 s on lock 0x7b5400ac8354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:32.462100 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4423302528}
W20260709 12:23:32.448349 16395 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46250 (request call id 1076) took 4772 ms (client timeout 1601 ms). Trace:
W20260709 12:23:32.469404 16395 rpcz_store.cc:269] 0709 12:23:27.676305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:27.676577 (+   272us) service_pool.cc:224] Handling call
0709 12:23:32.448093 (+4771516us) spinlock_profiling.cc:243] Waited 4.17 s on lock 0x7b54009bebd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:32.448249 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9597205376}
I20260709 12:23:32.398128 19935 consensus_queue.cc:1059] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:23:32.508929 19935 raft_consensus.cc:3055] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:32.509506 19935 raft_consensus.cc:740] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:32.494027 17263 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:32.498795 17264 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:32.501665 17255 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:32.502611 20067 consensus_queue.cc:1048] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:32.503834 20024 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:32.456830 19891 raft_consensus.cc:2804] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:32.504472 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:32.504635 20229 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:32.452008 20249 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:32.505436 20209 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:32.442070 20173 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:32.393047 20160 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:32.531086 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:32.532016 17270 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:32.534876 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:32.535841 17258 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:32.519259 16367 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:32.523064 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:32.540606 17261 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:32.510675 19935 consensus_queue.cc:260] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:32.522987 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:32.541842 17255 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:32.528021 16547 raft_consensus.cc:1275] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:23:32.537549 19969 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:32.547302 16547 raft_consensus.cc:1240] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 12:23:32.536630 19891 raft_consensus.cc:697] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:32.549409 19891 consensus_queue.cc:237] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:32.519781 19943 consensus_queue.cc:1048] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:23:32.558843 20160 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:32.560655 20208 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:32.563644 17263 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:32.440232 20116 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:32.543298 19935 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:32.566126 20116 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:32.527936 20230 consensus_queue.cc:1048] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:23:32.531045 20173 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:32.570778 19937 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:32.564061 20220 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:32.578377 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.578787 17269 raft_consensus.cc:2393] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 12: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:32.583040 16543 raft_consensus.cc:1240] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:23:32.585860 20073 raft_consensus.cc:3055] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:32.586455 20073 raft_consensus.cc:740] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:32.587378 20073 consensus_queue.cc:260] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:32.590689 20236 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:32.591481 20108 consensus_meta.cc:287] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.527s	user 0.009s	sys 0.000s
I20260709 12:23:32.593683 20220 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:32.559043 20217 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:32.611344 17261 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:23:32.627053 17255 raft_consensus.cc:2410] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:23:32.578419 20229 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:32.643167 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 123986us
W20260709 12:23:32.648397 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.649791 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:32.604808 20217 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:32.650832 17261 raft_consensus.cc:2393] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 15: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:32.652917 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:32.611696 19892 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:23:32.656391 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.648008 20229 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:32.611991 19969 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:32.607813 20073 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:32.662642 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:32.665228 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:32.665833 17254 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:32.669910 17270 raft_consensus.cc:1240] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:23:32.673710 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:23:32.674623 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.689580 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:32.692875 16397 raft_consensus.cc:1240] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:32.692955 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:32.694145 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:32.694796 16472 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:32.695777 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.700578 16471 raft_consensus.cc:1240] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 LEADER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:32.700937 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:32.702610 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:32.663149 19937 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:32.708715 16402 raft_consensus.cc:1240] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:23:32.663358 19943 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:32.673127 20067 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:32.712080 16403 raft_consensus.cc:1275] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:23:32.720288 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:32.706089 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:32.707943 16367 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:32.672755 20144 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:32.666335 20220 consensus_queue.cc:1059] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:23:32.748286 16476 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
I20260709 12:23:32.750834 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:23:32.751377 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.668377 20229 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:32.747330 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:32.777359 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:32.721316 17254 raft_consensus.cc:2376] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9. Re-sending same reply.
I20260709 12:23:32.721755 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:32.672160 19969 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:32.776096 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 68227.2us
I20260709 12:23:32.779214 17255 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:32.722515 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:32.784474 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:32.785681 17263 raft_consensus.cc:2393] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 12: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:32.780614 16367 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:32.879531 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 90296.9us
I20260709 12:23:32.880118 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:32.810284 16367 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:32.716502 20214 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:23:32.884631 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 74464.6us
I20260709 12:23:32.710632 19894 consensus_queue.cc:1048] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 12:23:32.712244 20230 consensus_queue.cc:1048] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.168s
I20260709 12:23:32.898072 16367 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:32.759367 20144 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:32.720598 19937 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:32.721042 19943 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:32.900667 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:32.913370 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:32.914443 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:32.883152 19892 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:23:32.927033 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.913652 17258 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
I20260709 12:23:32.776648 20203 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:32.898280 19891 raft_consensus.cc:2804] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:32.744251 20067 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:33.006227 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 108357us
I20260709 12:23:33.007586 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:33.009927 16367 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:32.884361 20236 consensus_queue.cc:1059] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.115s
I20260709 12:23:33.013878 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:32.887687 20108 consensus_queue.cc:1059] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.114s
I20260709 12:23:33.021912 17254 raft_consensus.cc:1240] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:23:32.998122 20203 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:33.023584 20203 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:33.006842 19891 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:33.029381 19891 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:23:33.031222 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:33.005797 19892 raft_consensus.cc:697] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 8 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:32.899119 20237 consensus_queue.cc:1048] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.109s
I20260709 12:23:33.015952 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:33.038067 16395 raft_consensus.cc:2393] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 12: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:33.009466 17258 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 15.
I20260709 12:23:32.897030 20214 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:33.046062 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:33.043205 20236 consensus_queue.cc:1243] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer f99066d13e524238aec6aa2024a8bd50 log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:23:33.066219 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:33.058528 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:33.071398 17254 raft_consensus.cc:2410] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:23:33.047277 20108 consensus_queue.cc:1243] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer f99066d13e524238aec6aa2024a8bd50 log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:23:33.058068 19894 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:33.101822 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:33.102617 17254 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
I20260709 12:23:33.103535 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:33.108137 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:33.088742 16441 connection.cc:256] RPC call timeout handler was delayed by -1.6241313320001609s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20520000000000002s for a response
I20260709 12:23:33.081159 20220 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:33.115669 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:33.119041 17258 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:33.119544 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 109701us
W20260709 12:23:33.120822 16441 leader_election.cc:336] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491): Timed out: RequestConsensusVote RPC to 127.5.58.4:44491 timed out after 1.883s (SENT)
I20260709 12:23:33.121330 16441 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:33.120429 16367 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:33.117321 20258 log.cc:927] Time spent T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:23:33.123318 16367 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:33.133512 17269 raft_consensus.cc:1240] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:33.138381 17269 raft_consensus.cc:1240] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:33.106209 19892 consensus_queue.cc:237] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:33.127522 20229 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:33.123395 20220 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:33.158480 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:33.203397 20239 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:23:33.206494 16472 consensus_meta.cc:287] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.511s	user 0.012s	sys 0.000s
I20260709 12:23:33.207504 16472 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
I20260709 12:23:33.191542 19943 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:33.209230 20067 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:33.209457 20265 log.cc:927] Time spent T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 12:23:33.130359 19849 consensus_queue.cc:1048] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 12:23:33.215474 20220 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:33.216447 20214 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:33.185293 20262 log.cc:927] Time spent T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:23:33.186972 20108 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:23:33.243144 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 119880us
W20260709 12:23:33.242054 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:33.249032 16471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1308) took 2815 ms. Trace:
I20260709 12:23:33.249205 16550 raft_consensus.cc:1240] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:33.249584 16471 rpcz_store.cc:276] 0709 12:23:30.433409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:30.612328 (+178919us) service_pool.cc:224] Handling call
0709 12:23:30.620390 (+  8062us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:32.712498 (+2092108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:33.248765 (+536267us) spinlock_profiling.cc:243] Waited 2.08 s on lock 0x7b540058c854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:33.248965 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4783327488}
I20260709 12:23:33.253737 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:33.199549 19894 raft_consensus.cc:697] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:33.266973 16550 raft_consensus.cc:1240] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:33.231714 20267 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:33.201416 20273 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:23:33.268997 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 156276us
W20260709 12:23:33.301173 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 98655.3us
I20260709 12:23:33.242157 19937 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:33.241961 19943 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.308493 16550 raft_consensus.cc:1240] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 6. Current term is 8. Ops: []
W20260709 12:23:33.270771 20266 log.cc:927] Time spent T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 12:23:33.267215 20144 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:33.305485 20173 consensus_meta.cc:287] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.735s	user 0.013s	sys 0.000s
W20260709 12:23:33.323493 16515 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.021s	sys 0.026s
I20260709 12:23:33.297600 20267 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:33.325886 16515 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.021s	sys 0.026s
W20260709 12:23:33.328401 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 519714us
W20260709 12:23:33.322281 20073 consensus_meta.cc:287] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.651s	user 0.006s	sys 0.006s
I20260709 12:23:33.265128 19849 consensus_queue.cc:1059] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 12:23:33.454291 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:33.263195 20108 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:33.297329 20067 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:33.473390 17271 raft_consensus.cc:2393] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 12: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
W20260709 12:23:33.481674 20268 log.cc:927] Time spent T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.368s	user 0.002s	sys 0.000s
I20260709 12:23:33.483984 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:33.485281 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:33.495029 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 382096us
W20260709 12:23:33.496012 16441 connection.cc:256] RPC call timeout handler was delayed by -1.5072356440000476s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1526s for a response
W20260709 12:23:33.496836 16441 connection.cc:256] RPC call timeout handler was delayed by -1.4756313319999208s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2217s for a response
W20260709 12:23:33.497450 16441 connection.cc:256] RPC call timeout handler was delayed by -1.3866532590000133s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1809s for a response
W20260709 12:23:33.497942 16441 connection.cc:256] RPC call timeout handler was delayed by -1.3866532590000133s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1809s for a response
W20260709 12:23:33.498360 16441 connection.cc:256] RPC call timeout handler was delayed by -1.3696523010003148s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15480000000000002s for a response
W20260709 12:23:33.498786 16441 connection.cc:256] RPC call timeout handler was delayed by -1.324652301000242s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15980000000000003s for a response
W20260709 12:23:33.499187 16441 connection.cc:256] RPC call timeout handler was delayed by -1.3209512950002136s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18380000000000002s for a response
W20260709 12:23:33.499607 16441 connection.cc:256] RPC call timeout handler was delayed by -1.3209512950002136s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18380000000000002s for a response
W20260709 12:23:33.500015 16441 connection.cc:256] RPC call timeout handler was delayed by -1.3127620460004437s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2037s for a response
W20260709 12:23:33.502664 16441 connection.cc:256] RPC call timeout handler was delayed by -1.3127620460004437s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2037s for a response
W20260709 12:23:33.503338 16441 connection.cc:256] RPC call timeout handler was delayed by -1.2992620460004218s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20520000000000002s for a response
I20260709 12:23:33.508656 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:33.513173 16441 connection.cc:256] RPC call timeout handler was delayed by -1.2319523010000921s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17010000000000003s for a response
W20260709 12:23:33.514252 16441 connection.cc:256] RPC call timeout handler was delayed by -1.200589852000121s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.211s for a response
W20260709 12:23:33.514739 16441 connection.cc:256] RPC call timeout handler was delayed by -1.1492975679998381s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19510000000000002s for a response
W20260709 12:23:33.515224 16441 connection.cc:256] RPC call timeout handler was delayed by -1.1492975679998381s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19510000000000002s for a response
W20260709 12:23:33.515691 16441 connection.cc:256] RPC call timeout handler was delayed by -1.1053001119998953s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16770000000000002s for a response
I20260709 12:23:33.517819 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:33.335704 20267 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:33.531467 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:33.331161 20275 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:33.543680 20275 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:33.544206 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:33.545148 17255 raft_consensus.cc:2410] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:23:33.534982 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:33.535801 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:33.546149 17271 raft_consensus.cc:2376] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10. Re-sending same reply.
I20260709 12:23:33.546173 16398 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
W20260709 12:23:33.546844 16441 connection.cc:256] RPC call timeout handler was delayed by -1.0816523010003039s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18680000000000002s for a response
I20260709 12:23:33.549160 16398 raft_consensus.cc:1240] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:33.322036 19943 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:33.550621 16441 connection.cc:256] RPC call timeout handler was delayed by -1.0330523010002253s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1922s for a response
I20260709 12:23:33.565850 16468 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:23:33.563509 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:33.567516 17255 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:23:33.339594 20173 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:33.528177 20144 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:33.592684 16548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1377) took 3148 ms. Trace:
I20260709 12:23:33.593297 16548 rpcz_store.cc:276] 0709 12:23:30.434043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:31.459510 (+1025467us) service_pool.cc:224] Handling call
0709 12:23:31.459927 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:31.929667 (+469740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:33.581929 (+1652262us) spinlock_profiling.cc:243] Waited 443 ms on lock 0x7b54009745d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:33.582196 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1018686208}
W20260709 12:23:33.582227 19891 consensus_meta.cc:287] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.552s	user 0.009s	sys 0.004s
I20260709 12:23:33.595414 16399 raft_consensus.cc:2393] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 11: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:33.597581 17255 raft_consensus.cc:3055] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:33.598157 17255 raft_consensus.cc:740] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:33.599156 17255 consensus_queue.cc:260] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:33.600703 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:33.601284 20245 log.cc:927] Time spent T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 1.198s	user 0.007s	sys 0.010s
I20260709 12:23:33.601567 16550 raft_consensus.cc:2410] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:23:33.607566 17263 raft_consensus.cc:3055] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:23:33.609145 20274 log.cc:927] Time spent T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.304s	user 0.001s	sys 0.000s
I20260709 12:23:33.610028 16367 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:33.611284 17261 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:33.578291 20203 consensus_meta.cc:287] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.554s	user 0.009s	sys 0.002s
I20260709 12:23:33.597817 17266 consensus_queue.cc:260] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:33.608091 17263 raft_consensus.cc:740] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:33.616211 16441 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:33.616587 17255 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:33.574420 19937 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.510931 20067 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:33.620900 16441 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:23:33.624886 20278 log.cc:927] Time spent T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 12:23:33.624521 17263 consensus_queue.cc:260] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:33.626065 20284 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.627139 16549 raft_consensus.cc:1240] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:33.624135 19915 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:33.629741 20290 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.629920 20282 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:33.629343 16441 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:33.634845 17263 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:33.618191 20259 log.cc:927] Time spent T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.487s	user 0.001s	sys 0.001s
I20260709 12:23:33.649250 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:33.649607 17261 raft_consensus.cc:1275] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:33.662181 16441 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:33.636168 20073 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:33.666522 16441 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:33.678113 20279 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:33.681929 20279 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:33.567088 19894 consensus_queue.cc:237] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:33.617542 19891 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:33.682415 20279 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:33.691326 17258 consensus_meta.cc:287] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.572s	user 0.009s	sys 0.001s
I20260709 12:23:33.701800 17263 raft_consensus.cc:1275] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:23:33.616026 20144 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:33.635461 19937 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:33.637423 20284 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:33.695034 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 90031.3us
I20260709 12:23:33.666749 19915 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:33.701267 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:33.655535 20282 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:33.702492 17258 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:33.708665 16542 raft_consensus.cc:2393] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 10: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:33.627831 20203 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:33.706380 16550 raft_consensus.cc:1240] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 13. Current term is 14. Ops: [10.6-13.7]
I20260709 12:23:33.708056 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:33.678360 20073 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:33.692898 20273 raft_consensus.cc:697] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:33.656983 20290 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:33.710877 16515 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:33.718674 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 107487us
I20260709 12:23:33.719522 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:33.729601 16546 raft_consensus.cc:1275] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:23:33.717904 20073 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:33.705291 19931 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:33.723265 16367 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:33.720283 17255 raft_consensus.cc:1275] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 12:23:33.734409 17255 pending_rounds.cc:85] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8: Aborting all ops after (but not including) 7
I20260709 12:23:33.734787 17255 pending_rounds.cc:107] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 12:23:33.735088 17255 raft_consensus.cc:2889] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:33.731245 19915 raft_consensus.cc:2764] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:23:33.741235 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:33.741726 20293 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:33.742239 16551 raft_consensus.cc:2410] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:23:33.742875 16367 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:33.741334 19931 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:33.748523 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:33.751345 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 84909.1us
I20260709 12:23:33.709379 20282 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:33.704574 20015 consensus_queue.cc:1059] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.105s
I20260709 12:23:33.769695 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:33.763824 20015 consensus_queue.cc:1243] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer ef57beab0e1d40d585983873c55107e8 log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:23:33.787032 16468 raft_consensus.cc:1275] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:33.788005 16468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1326) took 1120 ms. Trace:
I20260709 12:23:33.788503 16468 rpcz_store.cc:276] 0709 12:23:32.667578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:33.519523 (+851945us) service_pool.cc:224] Handling call
0709 12:23:33.565707 (+ 46184us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:33.787639 (+221932us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:33.787927 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:33.792183 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:33.768766 19928 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.777314 20297 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:33.795987 17254 raft_consensus.cc:1275] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:23:33.803503 19928 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:33.804760 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:33.805449 16546 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:23:33.805934 19928 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:33.765208 20295 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:23:33.808843 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 99058.1us
I20260709 12:23:33.811848 16515 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:33.812840 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 57663.5us
I20260709 12:23:33.791158 20173 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:33.793357 20296 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.786567 19943 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:33.817322 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:33.819945 16403 raft_consensus.cc:1240] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:33.809221 20295 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:33.813975 20291 log.cc:927] Time spent T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 12:23:33.828701 17258 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:33.826725 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:33.832353 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 70003.4us
I20260709 12:23:33.809831 20275 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:33.834517 16441 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:33.844828 16514 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:33.845417 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:33.855129 16515 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:33.830178 20295 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:33.842150 20298 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.848507 20297 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:23:33.866628 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:33.796305 20273 consensus_queue.cc:237] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:33.881307 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:33.881946 20244 log.cc:927] Time spent T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 1.498s	user 0.022s	sys 0.036s
I20260709 12:23:33.882542 16551 raft_consensus.cc:2393] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 12: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:33.882751 20158 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:33.887089 20067 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:33.825256 20296 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:33.896008 20257 log.cc:927] Time spent T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.999s	user 0.000s	sys 0.008s
I20260709 12:23:33.896687 20144 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:33.897044 20256 raft_consensus.cc:3055] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:33.897266 20296 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:33.897562 20256 raft_consensus.cc:740] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:33.897666 16474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 711) took 1530 ms. Trace:
I20260709 12:23:33.897996 20296 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:33.898507 20290 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:33.898680 20296 raft_consensus.cc:697] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 12 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:33.846977 20173 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.898442 20256 consensus_queue.cc:260] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:33.899885 20173 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:33.899843 20249 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 12:23:33.847322 19891 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:33.901109 20256 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:33.898083 16474 rpcz_store.cc:276] 0709 12:23:32.367019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:32.367179 (+   160us) service_pool.cc:224] Handling call
0709 12:23:32.391763 (+ 24584us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:23:32.428704 (+ 36941us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:32.428747 (+    43us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:23:32.442514 (+ 13767us) log.cc:844] Serialized 78 byte log entry
0709 12:23:32.675857 (+233343us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:23:32.675923 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:32.676069 (+   146us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:33.895331 (+1219262us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:23:33.897334 (+  2003us) raft_consensus.cc:1685] finished
0709 12:23:33.897354 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:33.897607 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6347,"raft.queue_time_us":49176,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":27286,"spinlock_wait_cycles":28113152,"thread_start_us":135441,"threads_started":1,"wal-append.queue_time_us":224140}
I20260709 12:23:33.903545 19880 consensus_queue.cc:260] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:33.904712 19880 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:33.847913 20302 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:33.907541 20302 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:33.907932 20302 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:33.893118 20158 consensus_queue.cc:1048] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:33.892323 17144 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:33.909272 20249 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:33.909675 20249 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:33.910025 20249 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:33.873214 20203 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:33.878368 20297 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:33.912943 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:33.913374 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:33.914271 16548 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:33.878575 20298 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:33.915391 16549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1226) took 1931 ms. Trace:
I20260709 12:23:33.915899 16549 rpcz_store.cc:276] 0709 12:23:31.984124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:32.397973 (+413849us) service_pool.cc:224] Handling call
0709 12:23:32.422010 (+ 24037us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:33.635879 (+1213869us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:33.915114 (+279235us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54000663d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:33.915325 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2770990976}
I20260709 12:23:33.919023 20277 raft_consensus.cc:3055] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:33.919479 20277 raft_consensus.cc:740] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:33.920264 20277 consensus_queue.cc:260] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:33.899796 20296 consensus_queue.cc:237] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:33.876572 19931 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:33.925200 20173 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:33.927345 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:33.927897 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:33.928402 16402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 1146) took 1229 ms. Trace:
I20260709 12:23:33.928675 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:33.928864 16402 rpcz_store.cc:276] 0709 12:23:32.698764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:32.718466 (+ 19702us) service_pool.cc:224] Handling call
0709 12:23:32.762252 (+ 43786us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:33.006533 (+244281us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:23:33.112828 (+106295us) log.cc:844] Serialized 14 byte log entry
0709 12:23:33.909354 (+796526us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b180011aee0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2875a314bc 00007f2872d7f48b 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a2923a
0709 12:23:33.927022 (+ 17668us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:33.927084 (+    62us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:33.927148 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:33.928174 (+  1026us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b54009be6d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:33.928222 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:33.928352 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17412,"prepare.queue_time_us":178418,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":1335,"raft.queue_time_us":18326,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"spinlock_wait_cycles":588370560,"thread_start_us":177412,"threads_started":2,"wal-append.queue_time_us":157481}
I20260709 12:23:33.930335 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:33.931434 20073 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:33.931885 20205 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.913595 20297 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:33.932209 20277 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:33.932965 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:33.934633 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:33.935454 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:33.936182 19937 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:33.947409 16470 raft_consensus.cc:1275] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:23:33.936578 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:33.936866 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:33.937772 20300 raft_consensus.cc:3055] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:33.940124 16400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 964) took 1266 ms. Trace:
I20260709 12:23:33.951179 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:33.948815 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:33.951939 16544 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:33.952075 16542 raft_consensus.cc:1275] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:23:33.949720 20300 raft_consensus.cc:740] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:23:33.944651 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 98008.1us
I20260709 12:23:33.942623 19908 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:33.954290 16545 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:23:33.949079 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:33.955130 16441 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:33.955480 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:33.949921 16400 rpcz_store.cc:276] 0709 12:23:32.673846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:32.674831 (+   985us) service_pool.cc:224] Handling call
0709 12:23:32.675252 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:32.691143 (+ 15891us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:32.691176 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:32.713259 (+ 22083us) log.cc:844] Serialized 26 byte log entry
0709 12:23:33.011327 (+298068us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:33.116799 (+105472us) log.cc:844] Serialized 14 byte log entry
0709 12:23:33.481098 (+364299us) spinlock_profiling.cc:243] Waited 1.41 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:23:33.939628 (+458530us) spinlock_profiling.cc:243] Waited 3.41 ms on lock 0x7b18000ab0a0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2875a314bc 00007f2872d7f48b 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d99455 00007f2875e81060 00007f2875e93716 00007f2875ea4287
0709 12:23:33.939761 (+   133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:33.939864 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:33.939905 (+    41us) raft_consensus.cc:1685] finished
0709 12:23:33.939917 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:33.940046 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5173,"prepare.queue_time_us":189007,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":37238,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":13434,"spinlock_wait_cycles":11080704,"thread_start_us":468353,"threads_started":3,"wal-append.queue_time_us":310942}
I20260709 12:23:33.954511 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:33.956458 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:33.958019 16468 raft_consensus.cc:2410] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:23:33.957329 16550 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:33.960898 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:33.956693 16545 raft_consensus.cc:606] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:33.958657 16542 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:33.964836 16549 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:33.973486 16543 raft_consensus.cc:1275] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:23:33.973851 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:33.974040 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:33.973888 16543 pending_rounds.cc:85] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1: Aborting all ops after (but not including) 5
I20260709 12:23:33.974686 16543 pending_rounds.cc:107] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:23:33.974969 16543 raft_consensus.cc:2889] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:33.975857 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:33.976732 16542 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:23:33.974517 16548 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:33.979032 16471 raft_consensus.cc:1240] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:33.979261 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:33.982821 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:33.984936 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:33.983565 16468 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:33.985581 16476 raft_consensus.cc:2376] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11. Re-sending same reply.
I20260709 12:23:33.990391 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:33.963093 19853 raft_consensus.cc:2804] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:33.943945 20205 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:33.986708 16367 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:33.995358 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:33.995653 16441 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:33.997483 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:33.998337 16470 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:23:34.000941 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:34.004644 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:34.004671 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:33.949501 20173 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.006485 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:33.985534 20217 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.001820 16401 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:34.009977 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:34.012718 16542 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:34.012633 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:34.013363 16550 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:34.011324 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:34.015550 16468 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:23:34.015017 17255 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:34.016317 16549 raft_consensus.cc:2410] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 12:23:34.019929 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:33.994145 19853 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:34.002173 19891 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:34.027652 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:34.022709 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:34.041739 17264 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:34.032966 16396 raft_consensus.cc:1240] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:34.033130 20305 raft_consensus.cc:2804] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:23:34.053015 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:34.041766 16548 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:34.022397 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:34.034483 20214 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:33.989965 19892 consensus_queue.cc:1048] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:23:34.033001 17263 raft_consensus.cc:1275] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:23:34.025255 16514 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:34.022337 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:34.048635 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:34.049933 20306 raft_consensus.cc:993] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1: : Instructing follower 52ab50b0237346c1a13ede947f6a63c2 to start an election
I20260709 12:23:34.027091 19853 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:33.994454 20203 consensus_queue.cc:1048] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:23:33.992357 20293 consensus_queue.cc:1048] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:23:34.053483 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:34.053830 20305 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:34.076858 20305 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:34.077745 20205 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:34.021185 20173 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:34.060866 16543 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:23:34.080698 20173 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:34.081349 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:34.082556 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:34.083295 16550 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:34.083683 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:34.084607 20173 consensus_queue.cc:1059] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:34.050915 20073 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:34.064569 16542 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
W20260709 12:23:34.067057 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:34.061661 20279 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:34.087313 16401 raft_consensus.cc:1240] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 10. Current term is 11. Ops: [8.5-10.6]
I20260709 12:23:34.087738 20279 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:34.062310 17143 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:34.088366 16394 raft_consensus.cc:1240] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
W20260709 12:23:34.090235 20267 consensus_meta.cc:287] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.548s	user 0.010s	sys 0.000s
I20260709 12:23:34.023341 20157 consensus_queue.cc:1048] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 12:23:34.041081 19891 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:34.084826 16514 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:34.051878 20282 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:34.070643 20306 raft_consensus.cc:1081] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Signalling peer 52ab50b0237346c1a13ede947f6a63c2 to start an election
I20260709 12:23:34.068485 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:23:34.092535 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 96965.3us
I20260709 12:23:34.096371 16543 raft_consensus.cc:2393] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 11: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:34.097988 17264 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:34.099045 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:34.105834 16514 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:34.099862 16550 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:23:34.040182 20298 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:34.057454 20214 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:34.099741 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:34.110239 16441 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:34.099087 19891 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:34.098707 19931 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:34.022585 20217 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:34.113912 16550 raft_consensus.cc:606] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:34.114627 16514 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:34.129750 16550 raft_consensus.cc:614] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:34.121078 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:34.059990 20229 consensus_queue.cc:1059] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:23:34.131395 20229 consensus_queue.cc:1243] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Peer f99066d13e524238aec6aa2024a8bd50 log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:23:34.132268 20218 consensus_queue.cc:1048] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:23:34.135625 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 76742.1us
I20260709 12:23:34.112974 20276 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:34.129199 16441 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:34.137897 16514 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:34.125852 20290 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:34.124286 16545 raft_consensus.cc:2393] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 10: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:34.065490 20302 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:34.138183 16441 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:34.145859 16395 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 1145) took 1447 ms (client timeout 1836 ms). Trace:
W20260709 12:23:34.167536 16395 rpcz_store.cc:269] 0709 12:23:32.698447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:32.702250 (+  3803us) service_pool.cc:224] Handling call
0709 12:23:34.145691 (+1443441us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b540015f254. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:34.145806 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":588626688}
W20260709 12:23:34.148707 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:34.099227 20267 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:34.140323 16514 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
W20260709 12:23:34.116003 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:34.176990 16477 raft_consensus.cc:1240] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:34.143260 20021 raft_consensus.cc:2804] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:34.139117 20158 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:34.126637 20279 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:23:34.185801 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:34.112625 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 50402.1us
I20260709 12:23:34.126096 20214 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:34.145565 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:34.135708 20275 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.178239 16545 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:23:34.195067 16545 raft_consensus.cc:606] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
W20260709 12:23:34.195361 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 340313us
I20260709 12:23:34.195562 16545 raft_consensus.cc:614] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:34.141944 20218 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:34.163120 16396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 962) took 1489 ms. Trace:
I20260709 12:23:34.144085 20290 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:34.192366 16398 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:23:34.196738 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:34.165625 20311 log.cc:927] Time spent T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 12:23:34.203354 16396 rpcz_store.cc:276] 0709 12:23:32.673256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:32.674723 (+  1467us) service_pool.cc:224] Handling call
0709 12:23:32.709684 (+ 34961us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:34.032827 (+1323143us) spinlock_profiling.cc:243] Waited 3.23 ms on lock 0x7b5400a314a0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e00dc0 00007f2875e8b1c0 00007f2875e8c231 00007f2875e88d8f 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c
0709 12:23:34.053542 (+ 20715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:34.162815 (+109273us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b5400a314a0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e00dc0 00007f2875e8ca02 00007f2875e88dd6 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:23:34.163043 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55568896}
I20260709 12:23:34.191668 16468 raft_consensus.cc:1240] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:34.204316 16546 raft_consensus.cc:2393] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 11: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:34.205885 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:34.212523 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 74424.7us
I20260709 12:23:34.206140 16515 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:34.157696 20282 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:34.159641 20284 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:34.178293 20217 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:34.217267 17143 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:34.184504 20296 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:34.188598 20021 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:34.228799 20021 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:34.229452 20217 consensus_queue.cc:1059] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:34.230809 20217 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:34.209110 20275 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:34.193660 16477 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:34826
I20260709 12:23:34.222469 16515 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:34.213502 20290 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:34.220577 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:34.243445 16473 raft_consensus.cc:2393] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 12: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:34.204092 20116 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:23:34.248714 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 107635us
I20260709 12:23:34.208829 20218 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:23:34.219880 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 62986.3us
I20260709 12:23:34.207386 20158 raft_consensus.cc:697] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:34.253348 16468 raft_consensus.cc:1240] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 11. Current term is 12. Ops: [10.7-11.8]
I20260709 12:23:34.255123 16398 raft_consensus.cc:1275] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:34.255244 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:34.257208 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:34.257707 16475 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:34.191236 20249 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:34.259465 16441 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:34.220158 20296 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:34.262406 17144 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:34.260128 16515 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:34.263247 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:34.263916 16473 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:23:34.265020 17271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 1045) took 1239 ms. Trace:
I20260709 12:23:34.268033 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:34.251103 20157 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:34.266701 20296 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:34.271219 16515 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:34.271976 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:34.218616 20256 consensus_queue.cc:1059] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 12:23:34.254853 20218 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:34.276683 17269 raft_consensus.cc:2393] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 12: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:34.277045 20249 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:34.280580 20305 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:34.267521 20295 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:34.281277 20158 consensus_queue.cc:237] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:34.264246 20306 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:34.285379 20142 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:34.285466 16441 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:34.300503 16515 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:34.329756 16515 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:34.265425 17271 rpcz_store.cc:276] 0709 12:23:33.025412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:33.546927 (+521515us) service_pool.cc:224] Handling call
0709 12:23:34.037926 (+490999us) spinlock_profiling.cc:243] Waited 16.1 ms on lock 0x7b58003b8298. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:23:34.038134 (+   208us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:34.038571 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:34.038606 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:34.039170 (+   564us) log.cc:844] Serialized 52 byte log entry
0709 12:23:34.175793 (+136623us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:34.203356 (+ 27563us) log.cc:844] Serialized 14 byte log entry
0709 12:23:34.259687 (+ 56331us) log.cc:844] Serialized 14 byte log entry
0709 12:23:34.259857 (+   170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:34.259954 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:34.264828 (+  4874us) raft_consensus.cc:1685] finished
0709 12:23:34.264846 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:34.264972 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8462,"prepare.queue_time_us":23940,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":28952,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":8054,"spinlock_wait_cycles":37019264,"thread_start_us":183396,"threads_started":2,"wal-append.queue_time_us":160550}
I20260709 12:23:34.305487 16475 raft_consensus.cc:2410] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
W20260709 12:23:34.306545 20311 log.cc:927] Time spent T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:23:34.272352 20237 raft_consensus.cc:3055] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:34.266431 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:34.261672 20205 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:34.304819 19935 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:23:34.318377 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 101193us
I20260709 12:23:34.308883 20320 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:34.349125 16472 raft_consensus.cc:1240] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:34.348995 17143 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
W20260709 12:23:34.334306 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 71964.4us
I20260709 12:23:34.337668 16515 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:34.350772 16441 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:34.337507 16396 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
I20260709 12:23:34.350775 17143 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:34.337774 20157 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:34.336854 20256 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:34.354712 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:34.355124 20256 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:34.355491 20256 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:34.356779 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:34.357280 20249 raft_consensus.cc:2804] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:34.357802 20249 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:34.357429 16398 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:34.327636 20142 raft_consensus.cc:697] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:34.355563 16396 raft_consensus.cc:2410] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:23:34.360234 17143 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:34.361528 16515 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:34.335619 19915 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:23:34.358474 20312 log.cc:927] Time spent T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
I20260709 12:23:34.345460 20306 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:34.371673 17143 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:34.337275 20237 raft_consensus.cc:740] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:34.373027 16441 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:34.373297 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 206713us
I20260709 12:23:34.351974 20320 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:34.363592 20249 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:34.365686 19935 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:34.363101 20116 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:34.397271 17143 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:34.368592 19699 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:34.368199 19915 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:34.372721 20276 consensus_queue.cc:1059] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:23:34.389142 20279 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.364311 20323 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:34.394372 20329 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:34.405122 20237 consensus_queue.cc:260] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:34.409922 20302 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:34.398533 20320 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:34.392017 19969 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:34.413719 16516 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:34.420193 20305 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:34.421343 19915 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:34.422276 20276 raft_consensus.cc:3055] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:23:34.428439 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 55504.8us
I20260709 12:23:34.408084 17143 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:34.401729 20326 raft_consensus.cc:2804] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:23:34.447376 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 125617us
I20260709 12:23:34.372550 20314 consensus_queue.cc:1059] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 12:23:34.420089  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:23:34.423710 20308 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:23:34.456524 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 88903.5us
I20260709 12:23:34.476744 17142 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:34.426738 20267 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:34.426319 20323 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:23:34.461424 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:34.453591 20277 consensus_meta.cc:287] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.504s	user 0.008s	sys 0.004s
W20260709 12:23:34.473280 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:34.476751 16441 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:34.477128 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 69134.4us
W20260709 12:23:34.458539 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 44941.4us
I20260709 12:23:34.483901 17143 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:34.428884 20279 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:34.489267 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:34.493564 16441 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:34.425796 20158 consensus_queue.cc:1048] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 12:23:34.462697 20329 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:34.476811 20302 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:34.498030 20333 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:34.498225 19853 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:34.476064 20276 raft_consensus.cc:740] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:34.499495 17142 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:34.535549 20329 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:34.500092 20332 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:34.501296 20108 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:34.537883 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:34.476828 19891 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:34.506135 20205 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:34.489485 20305 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:34.511881 20217 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:34.486160 19969 consensus_queue.cc:1048] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:23:34.516701 16547 raft_consensus.cc:1240] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:23:34.548889 20276 consensus_queue.cc:260] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:34.550384 20276 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:34.553161 17270 raft_consensus.cc:1275] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:23:34.555071 19969 consensus_queue.cc:1048] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:34.517583 20284 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:34.479182 20237 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:34.477385 20326 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:34.531757 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 197186us
I20260709 12:23:34.539175 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:34.539858 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 46336.2us
I20260709 12:23:34.574896 17142 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:34.541102 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:34.480559 20323 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:34.538589 17269 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:23:34.544236 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:34.575361 16441 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:34.545514 20330 log.cc:927] Time spent T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:23:34.547825 17263 raft_consensus.cc:1240] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:34.457182 20142 consensus_queue.cc:237] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:34.572576 20021 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:34.584357 19891 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:34.584872 20021 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:34.579154 20308 raft_consensus.cc:3055] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:34.585808 20308 raft_consensus.cc:740] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:34.586884 20256 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:34.587893 20295 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:34.589246 20256 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:34.589836 20160 raft_consensus.cc:3055] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:34.590299 20160 raft_consensus.cc:740] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:34.591235 20160 consensus_queue.cc:260] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:34.562305 16469 raft_consensus.cc:1240] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:34.573217 20251 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.597293 20251 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:34.599707 16515 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:34.601950 19969 raft_consensus.cc:2764] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:23:34.602119 20279 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:34.599709 20251 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:34.576867 20326 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:34.605425 19915 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:34.608230 19915 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:34.610039 20300 consensus_queue.cc:260] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:34.611348 20300 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:34.614840 20297 consensus_meta.cc:287] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.667s	user 0.007s	sys 0.004s
I20260709 12:23:34.615334 20297 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:34.627689 20116 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:23:34.628767 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 53515.1us
I20260709 12:23:34.629880 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:23:34.537711 20331 log.cc:927] Time spent T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:23:34.525928 20302 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:34.567453 20327 log.cc:927] Time spent T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 12:23:34.576732 17261 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:34.574296 17254 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:23:34.631317 16515 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:34.566522 20282 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:34.635058 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:34.636623 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 152784us
I20260709 12:23:34.641705 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:34.642387 17143 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:34.643831 17143 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:34.644212 16470 raft_consensus.cc:1240] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:34.567903 20284 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:34.641494 16468 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:23:34.653053 17143 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:34.631300 20116 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:34.655377 20256 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:34.656805 20116 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:34.657600 19853 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:34.658782 20208 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:34.658834 16515 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:34.662506 19969 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:34.663035 19969 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:34.663532 16515 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:34.663499 19969 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:34.664417 20276 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:34.664834 20276 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:34.665220 20276 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:34.665971 20297 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:34.641304 20279 consensus_queue.cc:1048] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:23:34.671753 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:34.672402 16473 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:34.638203 20240 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:34.674000 20240 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:34.674365 20240 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:34.674242 20276 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:34.676465 20276 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:34.641103 20158 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:34.681886 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:34.682638 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:34.683131 20305 consensus_queue.cc:1048] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:34.647434 20282 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:34.684864 20282 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:34.687615 20275 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:34.688719 20328 log.cc:927] Time spent T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.302s	user 0.004s	sys 0.000s
I20260709 12:23:34.689217 17143 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:34.690302 20229 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:34.691450 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:34.692106 17261 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:34.692710 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:34.694096 17254 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:34.698103 20214 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:34.630314 20217 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:34.655195 20160 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:34.697934 20290 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:34.710899 17261 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:34.692271 17143 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:34.717886 17254 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:23:34.724841 16473 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
W20260709 12:23:34.729765 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:34.639628 20302 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:34.646333 20325 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:34.652220 20314 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:34.748147 16474 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:34.755335 17143 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:34.740103 20251 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:34.761998 20338 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.762580 20338 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:34.763911 20205 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:34.766022 16399 raft_consensus.cc:3055] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:34.766376 16399 raft_consensus.cc:740] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:34.766832 17254 raft_consensus.cc:1275] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:23:34.767053 20290 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:34.767143 16399 consensus_queue.cc:260] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:34.742377 20325 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:34.768493 16399 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:34.748013 19699 consensus_queue.cc:1059] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:23:34.734191 20218 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:34.805964 20296 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:34.787354 20284 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:34.777536 20282 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:34.800037 20326 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:34.812147 20320 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:34.812939 16399 raft_consensus.cc:1275] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 12:23:34.802201 20338 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:34.813347 20326 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:34.815055 20320 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:34.820622 16399 pending_rounds.cc:85] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 7
I20260709 12:23:35.654865 16399 pending_rounds.cc:107] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 12:23:35.655227 16399 raft_consensus.cc:2889] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:23:35.659008 16517 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.904s	user 0.016s	sys 0.024s
W20260709 12:23:35.659468 16517 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.905s	user 0.016s	sys 0.024s
W20260709 12:23:34.919482 20343 log.cc:927] Time spent T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
W20260709 12:23:35.005482 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:35.663594 16366 leader_election.cc:336] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 2.001s (SENT)
I20260709 12:23:35.043270 20282 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:35.667695 20275 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:35.668416 20275 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:35.670909 20275 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:35.169867 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:35.671798 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:35.672612 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:35.177264 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:35.674805 16396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46250 (request call id 1142) took 1058 ms. Trace:
I20260709 12:23:35.675284 16396 rpcz_store.cc:276] 0709 12:23:34.617306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:35.176973 (+559667us) service_pool.cc:224] Handling call
0709 12:23:35.674744 (+497771us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:35.676442 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:35.678108 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:34.760578 20214 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:35.211023 20350 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:35.679608 20350 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:35.215458 20296 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:35.218604 17266 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:23:35.558660 20276 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.791s	user 0.020s	sys 0.030s
W20260709 12:23:35.689443 20276 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.922s	user 0.021s	sys 0.030s
W20260709 12:23:35.561936 17269 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.932s	user 0.019s	sys 0.022s
W20260709 12:23:35.690413 17269 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.060s	user 0.020s	sys 0.023s
I20260709 12:23:35.691206 17269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 1060) took 1090 ms. Trace:
I20260709 12:23:35.691567 17269 rpcz_store.cc:276] 0709 12:23:34.600397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:34.628522 (+ 28125us) service_pool.cc:224] Handling call
0709 12:23:34.628907 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:34.629327 (+   420us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:34.629353 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:34.629856 (+   503us) log.cc:844] Serialized 26 byte log entry
0709 12:23:35.690804 (+1060948us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:35.690849 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:35.690925 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:35.690974 (+    49us) raft_consensus.cc:1685] finished
0709 12:23:35.690990 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:35.691143 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4282,"raft.queue_time_us":6781,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":8383,"thread_start_us":1060533,"threads_started":1,"wal-append.queue_time_us":277621}
W20260709 12:23:35.590260 16475 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.919s	user 0.008s	sys 0.013s
W20260709 12:23:35.693481 16475 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.022s	user 0.010s	sys 0.014s
I20260709 12:23:35.694289 16475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1351) took 1025 ms. Trace:
I20260709 12:23:35.694651 16475 rpcz_store.cc:276] 0709 12:23:34.668920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:34.669756 (+   836us) service_pool.cc:224] Handling call
0709 12:23:34.670194 (+   438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:34.670521 (+   327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:34.670546 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:34.670980 (+   434us) log.cc:844] Serialized 26 byte log entry
0709 12:23:35.693886 (+1022906us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:35.693934 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:35.694013 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:35.694071 (+    58us) raft_consensus.cc:1685] finished
0709 12:23:35.694087 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:35.694230 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3048,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":108,"thread_start_us":1022518,"threads_started":1,"wal-append.queue_time_us":245459}
I20260709 12:23:35.592911 16469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1349) took 1415 ms. Trace:
I20260709 12:23:35.696465 16469 rpcz_store.cc:276] 0709 12:23:34.177490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:34.178043 (+   553us) service_pool.cc:224] Handling call
0709 12:23:34.194533 (+ 16490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:34.596666 (+402133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:35.592618 (+995952us) spinlock_profiling.cc:243] Waited 367 ms on lock 0x7b5400594a54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:35.592841 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":845200256}
W20260709 12:23:35.602214 19935 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.841s	user 0.007s	sys 0.007s
W20260709 12:23:35.698935 19935 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.937s	user 0.008s	sys 0.008s
W20260709 12:23:35.699172 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 943855us
I20260709 12:23:35.621555 16400 raft_consensus.cc:1240] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 11. Current term is 12. Ops: []
W20260709 12:23:35.630228 20142 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.044s	user 0.020s	sys 0.008s
W20260709 12:23:35.622884 20240 consensus_meta.cc:287] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.948s	user 0.011s	sys 0.009s
I20260709 12:23:35.700671 20240 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:35.646212 20237 consensus_meta.cc:287] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 1.070s	user 0.010s	sys 0.007s
I20260709 12:23:35.702316 20237 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:35.702893 20237 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:35.703796 20339 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.704393 20339 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:35.706879 20339 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:35.707455 20339 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.676280 20348 consensus_queue.cc:1048] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:35.709906 17271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 1056) took 1169 ms. Trace:
W20260709 12:23:35.710039 20142 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.114s	user 0.020s	sys 0.009s
I20260709 12:23:35.683477 20267 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.711207 20267 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:35.713501 20267 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:35.716545 20217 raft_consensus.cc:3055] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:35.717103 20217 raft_consensus.cc:740] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:23:35.717937 16398 consensus_meta.cc:287] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 1.354s	user 0.010s	sys 0.007s
I20260709 12:23:35.683014 20350 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:35.718184 20237 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:35.718420 16398 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:23:35.699194 20338 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:35.719074 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:35.719745 16399 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:35.710364 17271 rpcz_store.cc:276] 0709 12:23:34.540234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:34.540394 (+   160us) service_pool.cc:224] Handling call
0709 12:23:34.564032 (+ 23638us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:34.564356 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:23:34.579587 (+ 15231us) log.cc:844] Serialized 14 byte log entry
0709 12:23:35.187252 (+607665us) log.cc:844] Serialized 14 byte log entry
0709 12:23:35.709530 (+522278us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:35.709588 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:35.709649 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:35.709723 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:35.709846 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":541166,"prepare.queue_time_us":458178,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":99,"raft.queue_time_us":1483696,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":192,"thread_start_us":162007,"threads_started":1,"wal-append.queue_time_us":265776}
W20260709 12:23:35.726083 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 62530us
I20260709 12:23:35.718012 20217 consensus_queue.cc:260] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:35.728516 20217 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:35.730743 16472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1337) took 1774 ms. Trace:
I20260709 12:23:35.703219 20240 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:34.786541 19699 raft_consensus.cc:3055] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:35.733294 19699 raft_consensus.cc:740] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:35.734227 19699 consensus_queue.cc:260] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:35.679864 20325 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:35.683116 16441 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.033s	user 0.006s	sys 0.036s
W20260709 12:23:35.736380 16441 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.087s	user 0.007s	sys 0.037s
I20260709 12:23:35.735682 19699 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:35.736716 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 1.08719e+06us
I20260709 12:23:35.737793 16441 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:35.738927 16441 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:35.740029 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:35.734241 17258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 1059) took 1181 ms. Trace:
I20260709 12:23:35.741267 17258 rpcz_store.cc:276] 0709 12:23:34.552474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:34.554124 (+  1650us) service_pool.cc:224] Handling call
0709 12:23:34.554437 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:34.554772 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:34.554804 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:34.555312 (+   508us) log.cc:844] Serialized 26 byte log entry
0709 12:23:34.761154 (+205842us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:34.761192 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:34.761273 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:35.733880 (+972607us) raft_consensus.cc:1685] finished
0709 12:23:35.733906 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:35.734171 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1542,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":7393,"thread_start_us":205506,"threads_started":1,"wal-append.queue_time_us":350333}
I20260709 12:23:35.731778 16472 rpcz_store.cc:276] 0709 12:23:33.955830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:33.955954 (+   124us) service_pool.cc:224] Handling call
0709 12:23:33.956227 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:34.349865 (+393638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:35.730471 (+1380606us) spinlock_profiling.cc:243] Waited 393 ms on lock 0x7b5400b05254. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:35.730677 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":903138560}
W20260709 12:23:35.740245 16474 consensus_meta.cc:287] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.991s	user 0.011s	sys 0.003s
I20260709 12:23:35.743818 16474 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:23:35.744946 16547 raft_consensus.cc:2368] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 11. Current term is 12.
I20260709 12:23:35.747010 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:35.747659 16396 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:35.719095 16398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58430 (request call id 985) took 1362 ms. Trace:
I20260709 12:23:35.747701 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:35.748777 16543 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:35.748585 16398 rpcz_store.cc:276] 0709 12:23:34.356393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:34.356539 (+   146us) service_pool.cc:224] Handling call
0709 12:23:35.719035 (+1362496us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:35.749423 16469 raft_consensus.cc:1275] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:23:35.747730 16399 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:23:35.751111 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:35.751716 16547 raft_consensus.cc:1240] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 9. Current term is 10. Ops: [7.5-9.6]
I20260709 12:23:35.752153 16398 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:35.753866 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:35.754300 17143 leader_election.cc:400] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 11. Current term is 12.
I20260709 12:23:35.707897 20339 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:35.755410 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:35.755604 16475 raft_consensus.cc:1240] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:35.755518 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:35.740561 20214 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:35.756413 16473 raft_consensus.cc:1275] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:23:35.756529 16551 raft_consensus.cc:2393] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 12: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:35.740749 20142 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:35.754973 17143 leader_election.cc:403] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:23:35.757552 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:35.757704 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:35.758226 19931 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.973s	user 0.005s	sys 0.001s
I20260709 12:23:35.758728 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:35.754413 16547 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:35.759502 16473 raft_consensus.cc:2393] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 10: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:35.758759 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:35.760592 17266 raft_consensus.cc:1275] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:23:35.760766 16548 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:23:35.761195 17266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 1003) took 3511 ms. Trace:
I20260709 12:23:35.758673 20339 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:35.762182 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:35.750233 20240 consensus_queue.cc:1048] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 12:23:35.763213 20285 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 11. Current term is 12.
I20260709 12:23:35.761518 17266 rpcz_store.cc:276] 0709 12:23:32.249874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:32.260312 (+ 10438us) service_pool.cc:224] Handling call
0709 12:23:32.285767 (+ 25455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:33.597427 (+1311660us) spinlock_profiling.cc:243] Waited 26.3 ms on lock 0x7b4c000143c0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874155a20 00007f2874166a29 00007f287412c644 00007f2874153f2d 00007f2874153b93 00007f2875e8418b 00007f2875e84cb5 00007f2875e7ff30 00007f2875e817de 00007f2875e8bb49 00007f2875e8c416
0709 12:23:35.218269 (+1620842us) spinlock_profiling.cc:243] Waited 940 ms on lock 0x7b580069e9d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875de50aa 00007f2875e67447 00007f2875e7ffe7 00007f2875e817de 00007f2875e8bb49 00007f2875e8c416 00007f2875e88d8f 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4
0709 12:23:35.760974 (+542705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:35.761147 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2222188672}
W20260709 12:23:35.761950 19931 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.976s	user 0.006s	sys 0.002s
I20260709 12:23:35.767336 16476 raft_consensus.cc:1275] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:23:35.767552 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:35.765383 16516 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:35.744685 20325 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:35.768927 20173 consensus_queue.cc:1059] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 12:23:35.771963 20325 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:35.773810 20323 consensus_meta.cc:287] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 1.165s	user 0.014s	sys 0.002s
I20260709 12:23:35.774149 20325 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:35.766952 20296 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:35.793664 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:35.795286 17143 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:35.795229 16473 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:35.800926 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:35.801000 16543 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:23:35.802685 16547 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:23:35.804066 17143 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:35.749581 20282 consensus_queue.cc:1059] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:23:35.775107 20296 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:35.807615 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:35.806712 17143 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:35.808710 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:35.809523 16476 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 15.
W20260709 12:23:35.805068 20160 consensus_meta.cc:287] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 1.098s	user 0.007s	sys 0.011s
I20260709 12:23:35.774494 20323 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:35.816748 20323 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:35.803740 20348 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:35.812806 20339 raft_consensus.cc:2804] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:35.817387 20348 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:35.817595 20339 raft_consensus.cc:697] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:35.817809 20348 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:35.818560 20339 consensus_queue.cc:237] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:35.819953 16424 debug-util.cc:398] Leaking SignalData structure 0x7b080031f800 after lost signal to thread 5352
I20260709 12:23:35.804036 20325 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:23:35.820438 16424 debug-util.cc:398] Leaking SignalData structure 0x7b080035ac20 after lost signal to thread 5355
I20260709 12:23:35.807420 20285 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:35.822875 20338 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:35.823127 20285 raft_consensus.cc:697] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:35.823597 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:35.824214 16476 raft_consensus.cc:2376] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9. Re-sending same reply.
I20260709 12:23:35.824587 20285 consensus_queue.cc:237] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:35.826195 20296 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:35.828302 20271 consensus_queue.cc:1048] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:23:35.830672 20282 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:35.831020 20271 consensus_queue.cc:1243] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer 610f4feda11942739236f2b121a4b9c1 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:23:35.831261 20282 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:35.829499 20267 consensus_queue.cc:1048] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:23:35.833597 20282 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:35.830139 20284 consensus_queue.cc:1048] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 12:23:35.835120 16396 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:35.835691 16367 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:35.836948 16516 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:35.838652 20284 raft_consensus.cc:2764] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:23:35.840304 16367 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:35.841158 20284 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:35.838155 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:35.843178 16367 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:35.846236 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:35.846735 16367 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:35.846905 16547 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:35.848492 20271 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:35.848073 16367 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:35.850361 16367 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:35.850982 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:35.853598 16397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46250 (request call id 1146) took 1060 ms. Trace:
I20260709 12:23:35.853957 16397 rpcz_store.cc:276] 0709 12:23:34.793418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:35.850605 (+1057187us) service_pool.cc:224] Handling call
0709 12:23:35.853530 (+  2925us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:35.848933 20271 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:35.854691 20271 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:35.845655 20296 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:35.844894 19931 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.861775 16476 raft_consensus.cc:1275] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:35.844866 20284 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:35.861572 20108 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.865908 20275 raft_consensus.cc:2764] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:23:35.865772 20108 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:35.866775 20275 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:35.867274 20275 raft_consensus.cc:697] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:35.857918 20296 raft_consensus.cc:697] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 10 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:35.859916 20338 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.870770 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:35.874253 16543 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:23:35.864818 19931 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:35.875620 17143 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:35.846297 20267 consensus_queue.cc:1048] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 12:23:35.868046 20108 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:35.878562 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:35.864219 20284 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:35.868108 20275 consensus_queue.cc:237] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:35.877432 20286 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:35.887706 16473 raft_consensus.cc:1275] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:23:35.880977 16547 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:23:35.893700 20286 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:35.894178 20286 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:35.894968 17143 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:35.895881 20217 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:35.896507 20217 raft_consensus.cc:697] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 10 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:35.896986 16472 raft_consensus.cc:1240] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:23:35.896782 20296 consensus_queue.cc:237] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:35.898020 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:35.897513 20217 consensus_queue.cc:237] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:35.898893 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:35.899117 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:35.899631 16473 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:35.898285 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:35.900643 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:35.900915 16476 raft_consensus.cc:2393] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 15: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:35.901437 16475 raft_consensus.cc:2410] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:23:35.900136 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:35.905561 20348 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:35.906364 19931 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:35.907868 20267 raft_consensus.cc:2804] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:35.888247 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:35.908499 20267 raft_consensus.cc:697] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 8 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
W20260709 12:23:35.908969 19969 consensus_meta.cc:287] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 1.245s	user 0.002s	sys 0.016s
I20260709 12:23:35.909392 16473 raft_consensus.cc:1240] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:35.909631 20267 consensus_queue.cc:237] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:35.910915 16475 raft_consensus.cc:1275] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:23:35.908991 16470 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:35.911465 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:35.921442 16397 raft_consensus.cc:2393] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 11: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:35.924070 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:35.924152 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:35.924835 16475 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:35.925130 16476 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:35.928645 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:35.929322 16472 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:35.930198 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:35.931201 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:35.931727 16472 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
I20260709 12:23:35.923198 19969 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:35.936734 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:35.938024 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:35.939043 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:35.939749 16470 raft_consensus.cc:2410] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:23:35.939841 20280 consensus_queue.cc:1059] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:35.940749 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:35.948747 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:35.949291 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:35.950333 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:35.950502 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:35.951020 17254 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:35.950914 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:35.951301 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:35.952051 16473 raft_consensus.cc:2393] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 10: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:35.953061 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:35.954116 16546 raft_consensus.cc:2393] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 9: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
W20260709 12:23:35.943303 20225 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 12:23:35.947289 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:35.968111 17266 raft_consensus.cc:2393] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 9: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:35.948065 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:35.957369 16468 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:35.959082 16475 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:23:35.961087 17143 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:35.971304 17255 raft_consensus.cc:1240] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:23:35.961982 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:35.966094 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:35.971402 17254 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:23:35.972805 16470 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 8.
W20260709 12:23:35.976890 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 125162us
I20260709 12:23:35.979827 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:35.980592 17264 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:35.877266 20338 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:35.982774 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:35.983006 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:35.983799 17271 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:35.982822 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:35.986248 17270 raft_consensus.cc:2410] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 13 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:23:35.982190 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:35.987149 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:35.987547 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:35.987609 16546 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:35.987908 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:35.988930 16543 raft_consensus.cc:2393] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 11: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:35.987833 16547 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:23:35.992046 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:35.992787 17270 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
I20260709 12:23:35.967348 20286 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:35.993705 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:35.995463 20286 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:35.981595 20015 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.976805 20285 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:36.002933 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:36.013314 16543 raft_consensus.cc:2393] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 10: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:36.004035 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:36.005147 16441 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:36.007974 20275 raft_consensus.cc:3055] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:36.018453 16543 raft_consensus.cc:1240] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:23:36.008649 20338 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:36.009092 16548 raft_consensus.cc:1240] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:36.009786 16551 raft_consensus.cc:1240] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:23:36.010145 16544 pending_rounds.cc:85] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1: Aborting all ops after (but not including) 6
I20260709 12:23:36.015045 16441 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:35.999238 20015 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:36.044453 16544 pending_rounds.cc:107] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:23:35.985438 19969 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:36.046406 16544 raft_consensus.cc:2889] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:36.063337 17264 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:23:36.065169 16546 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:36.056614 20214 raft_consensus.cc:2804] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:36.068887 20361 consensus_queue.cc:1048] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 12:23:36.069020 20214 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:36.070001 20214 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:36.070324 20015 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:36.071182 20348 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:36.071734 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:36.072518 16545 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
W20260709 12:23:36.075384 20208 consensus_meta.cc:287] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 1.416s	user 0.014s	sys 0.005s
I20260709 12:23:36.075856 20208 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:36.076552 17263 raft_consensus.cc:1240] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11. Ops: []
W20260709 12:23:36.076658 16514 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.319s	user 0.008s	sys 0.020s
W20260709 12:23:36.077387 16514 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.320s	user 0.008s	sys 0.020s
W20260709 12:23:36.077701 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 1.32559e+06us
I20260709 12:23:36.078317 16514 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:36.022737 20282 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:36.080101 20282 raft_consensus.cc:697] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:36.081072 20282 consensus_queue.cc:237] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:36.017004 20275 raft_consensus.cc:740] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:36.084431 19699 raft_consensus.cc:2804] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:36.084954 19699 raft_consensus.cc:697] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:36.085355 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:36.086037 16471 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:36.085810 19699 consensus_queue.cc:237] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:36.086411 19969 consensus_queue.cc:1059] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:36.086992 20350 consensus_queue.cc:1059] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:36.087365 16473 raft_consensus.cc:1275] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:36.081453 20308 consensus_queue.cc:260] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:36.088507 16475 raft_consensus.cc:1240] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:36.088887 20160 raft_consensus.cc:3055] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:36.084971 20275 consensus_queue.cc:260] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:36.092562 16468 raft_consensus.cc:1275] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:23:36.095382 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:36.097981 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:36.101147 16475 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:36.097479 17258 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:23:36.090220 20350 raft_consensus.cc:3055] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:36.108433 17271 raft_consensus.cc:2468] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:36.108269 20320 consensus_queue.cc:1048] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:36.117427 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:36.118535 17271 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:23:36.108688 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:36.110358 16476 raft_consensus.cc:2410] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:23:36.075966 20284 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:36.093035 20160 raft_consensus.cc:740] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:36.114949 17264 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
I20260709 12:23:36.110357 20308 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:36.113858 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 144767us
W20260709 12:23:36.134447 16366 leader_election.cc:336] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.508s (SENT)
W20260709 12:23:36.135519 16366 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 3.335s (SENT) [suppressed 39 similar messages]
I20260709 12:23:36.115715 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:36.136677 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:36.123347 16475 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:36.113683 20275 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:36.139268 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:36.140808 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:36.148836 16468 raft_consensus.cc:2393] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 15: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:36.149036 17271 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:36.149832 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:36.150063 20284 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:36.143529 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:36.152702 16399 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:36.154429 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 45040.5us
I20260709 12:23:36.167320 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:36.125957 20368 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.116446 20350 raft_consensus.cc:740] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:36.170814  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:36.181695 17258 raft_consensus.cc:1240] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:36.185834 16399 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:36.182494 16367 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:36.190608 17271 raft_consensus.cc:2468] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
I20260709 12:23:36.164852 20361 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:36.191938 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:36.193042 17271 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:36.205602 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:36.206851 16550 raft_consensus.cc:2410] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:23:36.207726 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:36.182025 16471 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
I20260709 12:23:36.165946 20271 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:36.125377 20160 consensus_queue.cc:260] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:36.220132 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:36.223726 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:36.224488 17261 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
W20260709 12:23:36.225690 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:36.227082 17143 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:36.225466 16441 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:36.228720 17143 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:36.229699 16441 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:36.189828 20368 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:36.233134 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:36.233619 16471 raft_consensus.cc:1240] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:36.237226 17261 raft_consensus.cc:2368] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 10. Current term is 11.
I20260709 12:23:36.231241 16441 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:36.193940 20214 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:36.238435 17271 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:23:36.238474 16441 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:36.243089 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:36.244109 17261 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:36.245188 16441 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:36.228041 20015 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:36.248998 20015 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:36.245983 20160 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:36.234356 20208 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:36.249940 20208 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:36.250356 20208 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:36.251173 20015 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:36.251729 20015 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:23:36.252475 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:36.252168 20015 raft_consensus.cc:697] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:36.254559 20015 consensus_queue.cc:237] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:36.256757 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:36.260780 17271 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:36.195333 20361 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:36.205008 20372 consensus_queue.cc:1048] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:23:36.234333 20285 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:36.238866 20142 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:36.234596 20338 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:36.283066 16470 raft_consensus.cc:1240] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:36.271359 19908 consensus_queue.cc:1048] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 12:23:36.239681 20367 consensus_queue.cc:1048] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:23:36.219125 20350 consensus_queue.cc:260] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:36.289794 19908 raft_consensus.cc:2764] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:23:36.325774 20350 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:36.325944 19853 raft_consensus.cc:2804] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:36.326630 20295 consensus_queue.cc:1048] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:36.267902 20284 consensus_queue.cc:1048] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:23:36.325820 16471 raft_consensus.cc:1275] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 12:23:36.290262 20249 consensus_meta.cc:287] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 1.871s	user 0.016s	sys 0.007s
I20260709 12:23:36.341693 17261 raft_consensus.cc:2410] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:23:36.273092 20373 consensus_queue.cc:1048] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.100s
I20260709 12:23:36.287724 20338 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:36.356274 20338 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:36.358363 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:36.360431 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 113272us
I20260709 12:23:36.360773 19853 raft_consensus.cc:697] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:36.340147 20249 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:36.404070 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 211282us
I20260709 12:23:36.405990 16477 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 12:23:36.411617 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 265919us
I20260709 12:23:36.373450 20383 raft_consensus.cc:2764] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:23:36.441075 20286 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.443051 19853 consensus_queue.cc:237] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:36.445182 20286 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:36.446415 16222 catalog_manager.cc:5697] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 7 to 9, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 9 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:36.440691 16477 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:36.450767 20286 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:36.457103 20383 consensus_queue.cc:1059] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:36.458420 20021 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.459106 20021 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:36.249317  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:36.463541 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:36.464378 16550 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:36.458252 20383 raft_consensus.cc:2804] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:36.465714 17143 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:36.477218 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:36.484535 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:36.487031 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:36.488365 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:36.490047 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:36.490794 16547 raft_consensus.cc:1240] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:23:36.492504 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:36.498392 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:36.499312 16396 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:23:36.499878 16549 raft_consensus.cc:2393] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 11: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:36.487246 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:36.500597 16397 raft_consensus.cc:1240] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:36.502790 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:36.510098 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:36.508773 16477 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:36.511765 17143 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:36.512065 16547 raft_consensus.cc:1240] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11. Ops: [8.5-10.6]
I20260709 12:23:36.472819 16222 catalog_manager.cc:5697] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 4 to 6, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 6 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:23:36.516721 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:36.517510 16548 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:36.458168 20251 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:36.520718 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:23:36.521904 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:36.524979 17269 raft_consensus.cc:1240] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:36.530658 16366 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:36.531539 16397 raft_consensus.cc:1275] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:23:36.500418 20306 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.506897 20286 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:36.535467 16397 raft_consensus.cc:1240] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:23:36.531965 16366 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:36.540850 17258 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:36.519626 20324 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:36.510140 20296 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.541817 16366 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:36.519786 19969 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.523731 20157 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:36.533653 16542 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:23:36.521207 20142 raft_consensus.cc:697] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:36.583479 16366 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:36.535696 20361 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:36.535831 20271 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:36.549314 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:36.550920 16549 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:23:36.519941 19931 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.584519 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:36.536069 20286 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:36.597131 16366 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:36.585083 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:36.599023 16366 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:36.591830 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:36.601557 16403 raft_consensus.cc:2393] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 12: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:36.514654 20208 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:36.591147 19969 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:36.604600 17261 raft_consensus.cc:1240] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:36.605537 19969 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:36.606622 19931 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:36.541066 20368 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:36.603935 20208 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:36.613798 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:36.534978 19908 consensus_queue.cc:1048] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:23:36.550350 20214 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:36.626441 16395 raft_consensus.cc:2393] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 12: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:36.628373 17258 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:36.531616 20380 consensus_queue.cc:1059] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 12:23:36.542359 20306 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:36.552389 20296 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:36.583734 20157 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:36.612438 20286 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:36.609400 20372 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:36.653178 20383 raft_consensus.cc:697] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:36.643944 20242 raft_consensus.cc:3055] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:36.730505 20242 raft_consensus.cc:740] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:36.731184 20157 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:36.731914 20383 consensus_queue.cc:237] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:36.711898 20142 consensus_queue.cc:237] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:36.677295 16441 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:36.587800 20388 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:36.733067 20242 consensus_queue.cc:260] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:36.587064 20392 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.684157 16400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 1138) took 2687 ms. Trace:
W20260709 12:23:36.634285 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 96049.5us
I20260709 12:23:36.609722 20361 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:36.748179 16400 rpcz_store.cc:276] 0709 12:23:33.996455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:33.996821 (+   366us) service_pool.cc:224] Handling call
0709 12:23:33.997163 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:35.700252 (+1703089us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:36.683849 (+983597us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b54009bd7d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:36.684088 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3735134080}
W20260709 12:23:36.755779 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 78633.4us
W20260709 12:23:36.753124 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 154183us
I20260709 12:23:36.657145 20367 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:36.759078 16366 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:36.764184 16545 raft_consensus.cc:2468] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:36.760488 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:36.733083 20380 raft_consensus.cc:3055] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:36.765478 17271 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
I20260709 12:23:36.652546 20271 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:36.658993 20015 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.761770 16441 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:36.766443 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:36.644220 20368 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:23:36.759537 20376 log.cc:927] Time spent T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.281s	user 0.001s	sys 0.000s
I20260709 12:23:36.770020 16366 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:36.771000 16546 raft_consensus.cc:2468] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
I20260709 12:23:36.768757 20367 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:36.772455 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:36.747959 20388 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:36.788028 16545 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:36.774418 16441 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
W20260709 12:23:36.776397 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:36.776834 20308 consensus_meta.cc:287] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.630s	user 0.013s	sys 0.004s
I20260709 12:23:36.777467 19849 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:36.777640 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:36.795723 17271 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:23:36.778851 20389 log.cc:927] Time spent T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
W20260709 12:23:36.779588 20387 log.cc:927] Time spent T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
I20260709 12:23:36.759248 20386 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:36.757643 20396 raft_consensus.cc:3055] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:36.814236 20396 raft_consensus.cc:740] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:36.815359 20396 consensus_queue.cc:260] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:36.733664 20306 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:36.785027 20015 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:36.789862 20368 raft_consensus.cc:697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:36.818919 20015 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:36.820045 20368 consensus_queue.cc:237] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:36.821355 20395 log.cc:927] Time spent T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
W20260709 12:23:36.822355 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:36.790701 20367 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:36.784072 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:36.782128 20324 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:36.799979 20240 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.825260 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:36.825161 20240 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:36.826709 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:23:36.782215 20382 log.cc:927] Time spent T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
I20260709 12:23:36.782063 20249 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:36.775995 20242 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:36.772114 19969 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.799865 16545 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:36.770788 20296 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:36.794255 20385 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:36.801249 20390 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:36.805011 19908 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:23:36.790036 20384 log.cc:927] Time spent T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.262s	user 0.000s	sys 0.001s
W20260709 12:23:36.837957 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 247267us
I20260709 12:23:36.838338 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:36.822683 20218 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:36.791612 16366 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:36.827819 17271 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
W20260709 12:23:36.794839 20379 log.cc:927] Time spent T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.275s	user 0.000s	sys 0.001s
I20260709 12:23:36.827982 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:36.839607 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:36.840147 17271 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:36.762053 20392 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:36.770380 20380 raft_consensus.cc:740] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:36.833482 17143 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:36.830888 20225 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.782091 20361 consensus_queue.cc:1048] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:23:36.790197 20388 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:36.852797 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:36.852308 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:36.854307 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:36.831029 16548 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
W20260709 12:23:36.841480 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 49920us
I20260709 12:23:36.855122 16398 raft_consensus.cc:2393] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 9: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:36.856144 16548 raft_consensus.cc:1240] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:36.845698 16441 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:36.847302 17271 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:23:36.847761 16543 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:36.858798 16548 raft_consensus.cc:1240] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:36.835193 20396 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:36.853602 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:36.855165 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:36.858820 16441 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:36.842403 16545 raft_consensus.cc:1240] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:23:36.859565 17144 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:36.836199 20390 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:36.858398 16516 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:36.861315 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:36.832253 19969 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:36.863137 16472 raft_consensus.cc:1240] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:23:36.863626 16543 raft_consensus.cc:1240] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:23:36.863862 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:36.864741 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:36.865348 16397 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:23:36.865721 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:36.866164 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:36.867942 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:23:36.850026 20275 consensus_meta.cc:287] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.680s	user 0.011s	sys 0.002s
I20260709 12:23:36.871210 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:36.840397 20385 raft_consensus.cc:697] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:36.872164 16400 raft_consensus.cc:3055] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:36.877661 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:36.878216 16400 raft_consensus.cc:740] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:36.849838 20249 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:36.875417 16471 raft_consensus.cc:2376] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16. Re-sending same reply.
I20260709 12:23:36.876149 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:36.844389 19908 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:36.855618 20324 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.881971 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:36.883632 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:36.888147 17254 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:36.856635 19931 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:36.866314 16402 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:23:36.879341 16400 consensus_queue.cc:260] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:36.863652 20386 raft_consensus.cc:2804] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:36.864430 20390 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:36.866991 20296 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:36.889168 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:36.871912 20218 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:36.869031 20251 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:36.868682 20229 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:36.899258 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:36.896522 16400 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:36.912011 17254 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 8.
I20260709 12:23:36.866835 20276 consensus_queue.cc:1059] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:23:36.857275 20225 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:36.922510 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:36.923264 16475 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
W20260709 12:23:36.928506 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 69008.4us
I20260709 12:23:36.929049 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:36.930048 16471 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:23:36.940465 16397 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
W20260709 12:23:36.943730 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:36.945493 16366 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:36.949283 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:36.901486 20386 raft_consensus.cc:697] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:36.948987 16366 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:36.885011 20401 consensus_queue.cc:1059] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 12:23:36.920855 20296 raft_consensus.cc:697] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 10 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:36.893651 20324 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:36.929224 20218 raft_consensus.cc:697] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 10 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:36.906826 20251 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:37.013824 20208 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.906656 20229 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:37.016167 20408 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:36.955184 20275 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:36.974216 20385 consensus_queue.cc:237] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:36.966173 20402 log.cc:927] Time spent T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
W20260709 12:23:37.018193 20302 consensus_meta.cc:287] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 2.276s	user 0.019s	sys 0.002s
I20260709 12:23:37.094817 20208 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:37.157140 20302 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:37.158149 20229 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:37.158674 20208 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:37.026124 20361 raft_consensus.cc:3055] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:37.159256 20302 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:37.159341 20361 raft_consensus.cc:740] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:37.018098 20392 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:37.019243 20406 log.cc:927] Time spent T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:23:37.160404 20361 consensus_queue.cc:260] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:36.963446 20386 consensus_queue.cc:237] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:37.019810 20296 consensus_queue.cc:237] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:37.161798 20361 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:37.160199 19908 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:37.023327 20390 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:36.888559 20380 consensus_queue.cc:260] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:37.165294 20380 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:37.166158 20390 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:37.166389 19969 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:37.032295 17264 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:23:37.027675 20251 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:37.033569 20286 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:37.091684 20249 raft_consensus.cc:697] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:37.170228 17271 raft_consensus.cc:2368] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 9. Current term is 10.
I20260709 12:23:37.164255 20408 raft_consensus.cc:697] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:37.165036 20218 consensus_queue.cc:237] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:37.026131 20324 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:37.160538 16550 raft_consensus.cc:1240] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:36.963496 19849 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:23:37.220692 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 289940us
I20260709 12:23:37.225493 17144 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:37.197189 20372 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:37.222904 19849 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:37.116513 20275 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:37.238162 20286 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:37.240818 20324 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:37.254837 20334 raft_consensus.cc:3055] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:37.281729 20334 raft_consensus.cc:740] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:37.282891 20334 consensus_queue.cc:260] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:37.256573 20249 consensus_queue.cc:237] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:37.262279 20367 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:37.263952 20408 consensus_queue.cc:237] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:37.289598 20388 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:37.288544 20275 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:37.284179 20334 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:37.279440 19849 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:37.289741 20367 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:37.242969 17144 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:37.298242 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.298238 16400 raft_consensus.cc:1275] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 12:23:37.298982 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:37.298436 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 55550.2us
I20260709 12:23:37.292090 20388 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:37.305670 20286 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:37.323100 20286 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:37.336915 20286 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:37.347916 20242 consensus_meta.cc:287] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.516s	user 0.012s	sys 0.000s
I20260709 12:23:37.354873 20275 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:37.359977 19849 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:37.362329 19849 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:37.364297 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:37.365643 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:37.366173 20286 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:37.369096 17258 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45230 (request call id 1048) took 1202 ms (client timeout 1547 ms). Trace:
I20260709 12:23:37.367038 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:37.371518 17258 rpcz_store.cc:269] 0709 12:23:36.167114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.192985 (+ 25871us) service_pool.cc:224] Handling call
0709 12:23:37.368972 (+1175987us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b5400bc4dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:37.369053 (+    81us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":764646144}
I20260709 12:23:37.372651 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:37.372592 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:37.373353 17258 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:37.374025 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:37.374157 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:37.374716 16545 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:37.369680 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:37.375694 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:37.376202 16548 raft_consensus.cc:1275] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:23:37.377645 16548 raft_consensus.cc:1275] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:23:37.366688 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:37.379132 17254 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:37.370810 20240 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:37.370369 20275 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:37.383002 16545 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:23:37.387634 20240 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:37.388260 20240 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:37.390506 20240 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:37.368498 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:37.384371 20407 consensus_queue.cc:1048] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 12:23:37.384428 20408 consensus_queue.cc:1048] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
W20260709 12:23:37.383431 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:37.387660 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.400524 16441 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:37.401829 16441 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:37.400569 17143 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:37.403388 20408 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:37.403889 20408 raft_consensus.cc:697] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 10 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:37.404896 20408 consensus_queue.cc:237] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:37.368395 20286 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:37.416242 16367 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:37.419800 20240 raft_consensus.cc:2764] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
I20260709 12:23:37.421679 20271 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:37.422091 20271 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:37.422406 20271 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:37.422472 16468 raft_consensus.cc:1240] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:37.427755 16367 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
W20260709 12:23:37.428452 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.429184 20275 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:37.430260 16224 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53622 (request call id 134) took 19711 ms (client timeout 15000 ms). Trace:
W20260709 12:23:37.430631 16224 rpcz_store.cc:269] 0709 12:23:17.718274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:17.729434 (+ 11160us) service_pool.cc:224] Handling call
0709 12:23:37.429839 (+19700405us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:28.150117 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:28.150823 (+   706us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:28.150867 (+    44us) write_op.cc:435] Acquired schema lock
0709 12:23:28.150885 (+    18us) tablet.cc:662] Decoding operations
0709 12:23:28.181408 (+ 30523us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:28.181472 (+    64us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:28.181502 (+    30us) tablet.cc:685] Acquiring locks for 20 operations
0709 12:23:28.183111 (+  1609us) tablet.cc:701] Row locks acquired
0709 12:23:28.183134 (+    23us) write_op.cc:260] PREPARE: finished
0709 12:23:28.183289 (+   155us) write_op.cc:270] Start()
0709 12:23:28.183421 (+   132us) write_op.cc:276] Timestamp: P: 1783599808183237 usec, L: 0
0709 12:23:28.183443 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:23:28.184208 (+   765us) log.cc:844] Serialized 7729 byte log entry
0709 12:23:28.800431 (+616223us) op_driver.cc:464] REPLICATION: finished
0709 12:23:29.556854 (+756423us) write_op.cc:301] APPLY: starting
0709 12:23:29.557061 (+   207us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:29.557650 (+   589us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:29.557807 (+   157us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:29.571018 (+ 13211us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:29.664755 (+ 93737us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=20
0709 12:23:29.664815 (+    60us) write_op.cc:312] APPLY: finished
0709 12:23:29.665576 (+   761us) log.cc:844] Serialized 138 byte log entry
0709 12:23:29.665896 (+   320us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:29.784465 (+118569us) write_op.cc:454] Released schema lock
0709 12:23:29.784568 (+   103us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:29.825855 (+ 41287us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:29.825916 (+    61us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":698617,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":731055,"apply.run_cpu_time_us":7947,"apply.run_wall_time_us":269013,"num_ops":20,"prepare.queue_time_us":184658,"prepare.run_cpu_time_us":25021,"prepare.run_wall_time_us":103276,"raft.queue_time_us":360226,"raft.run_cpu_time_us":24485,"raft.run_wall_time_us":523401,"replication_time_us":610899,"thread_start_us":865175,"threads_started":4,"wal-append.queue_time_us":57734,"wal-append.run_cpu_time_us":55383,"wal-append.run_wall_time_us":2425568}]]}
I20260709 12:23:37.437340 17258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 1047) took 1301 ms. Trace:
W20260709 12:23:37.437641 20300 consensus_meta.cc:287] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 2.826s	user 0.015s	sys 0.011s
I20260709 12:23:37.438540 20314 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:37.439203 20314 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:37.437791 17258 rpcz_store.cc:276] 0709 12:23:36.137235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.380820 (+1243585us) service_pool.cc:224] Handling call
0709 12:23:37.381154 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:37.384523 (+  3369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:37.384558 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:37.405802 (+ 21244us) log.cc:844] Serialized 26 byte log entry
0709 12:23:37.424288 (+ 18486us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:37.424997 (+   709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.425264 (+   267us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:37.437153 (+ 11889us) raft_consensus.cc:1685] finished
0709 12:23:37.437172 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:37.437285 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2573,"raft.queue_time_us":3023,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"thread_start_us":17996,"threads_started":1,"wal-append.queue_time_us":13164}
I20260709 12:23:37.442291 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:37.443696 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:37.444144 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:37.445636 17264 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:37.444290 17258 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:37.446661 17264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 1051) took 1251 ms. Trace:
I20260709 12:23:37.446661 17254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 1046) took 1324 ms. Trace:
I20260709 12:23:37.447292 17264 rpcz_store.cc:276] 0709 12:23:36.195845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.445149 (+1249304us) service_pool.cc:224] Handling call
0709 12:23:37.445456 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.446457 (+  1001us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.446605 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:37.448887 17264 raft_consensus.cc:1275] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:23:37.437412 17261 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 1045) took 1322 ms. Trace:
I20260709 12:23:37.449640 17264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 1052) took 1247 ms. Trace:
I20260709 12:23:37.449436 17261 rpcz_store.cc:276] 0709 12:23:36.114619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.344029 (+229410us) service_pool.cc:224] Handling call
0709 12:23:36.346369 (+  2340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:36.605123 (+258754us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.437206 (+832083us) spinlock_profiling.cc:243] Waited 258 ms on lock 0x7b540018a954. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:37.437362 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":593263360}
I20260709 12:23:37.447561 17254 rpcz_store.cc:276] 0709 12:23:36.122560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.379772 (+1257212us) service_pool.cc:224] Handling call
0709 12:23:37.380035 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.380232 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:23:37.395248 (+ 15016us) log.cc:844] Serialized 14 byte log entry
0709 12:23:37.445912 (+ 50664us) log.cc:844] Serialized 14 byte log entry
0709 12:23:37.446329 (+   417us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:37.446377 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:37.446429 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.446491 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:37.446600 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":14572,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":82,"raft.queue_time_us":185,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":201,"spinlock_wait_cycles":33152,"thread_start_us":63640,"threads_started":2,"wal-append.queue_time_us":49340}
I20260709 12:23:37.450320 17264 rpcz_store.cc:276] 0709 12:23:36.202364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.448246 (+1245882us) service_pool.cc:224] Handling call
0709 12:23:37.448581 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.449305 (+   724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.449569 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:37.451108 17261 raft_consensus.cc:1240] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:37.451776 17264 raft_consensus.cc:1240] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:37.451999 16544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1422) took 1447 ms. Trace:
I20260709 12:23:37.453574 17264 raft_consensus.cc:1240] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:37.452452 16544 rpcz_store.cc:276] 0709 12:23:36.005072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.005194 (+   122us) service_pool.cc:224] Handling call
0709 12:23:36.009882 (+  4688us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:36.064883 (+ 55001us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:36.064917 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:36.076990 (+ 12073us) log.cc:844] Serialized 52 byte log entry
0709 12:23:36.189527 (+112537us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:36.206742 (+ 17215us) log.cc:844] Serialized 14 byte log entry
0709 12:23:36.207149 (+   407us) log.cc:844] Serialized 14 byte log entry
0709 12:23:36.207397 (+   248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:36.207495 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:36.820422 (+612927us) spinlock_profiling.cc:243] Waited 2.1 ms on lock 0x7b180019cfc0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c522 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:23:37.449571 (+629149us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b180019cfc0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:23:37.451802 (+  2231us) raft_consensus.cc:1685] finished
0709 12:23:37.451824 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:37.451943 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22013,"prepare.queue_time_us":52881,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":5987,"raft.queue_time_us":9030,"raft.run_cpu_time_us":388,"raft.run_wall_time_us":12062,"spinlock_wait_cycles":29615232,"thread_start_us":112117,"threads_started":1,"wal-append.queue_time_us":159457}
I20260709 12:23:37.455168 17264 raft_consensus.cc:1240] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:37.455746 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:37.459251 20240 consensus_queue.cc:1059] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:37.460824 20240 raft_consensus.cc:3055] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:37.461169 20240 raft_consensus.cc:740] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:37.461205 20228 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:37.461738 20228 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:37.462021 20240 consensus_queue.cc:260] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:37.463198 20240 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:23:37.466989 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:37.467578 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.469130 17270 raft_consensus.cc:2393] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 12: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:37.470476 17270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45224 (request call id 1034) took 1488 ms. Trace:
I20260709 12:23:37.470880 17270 rpcz_store.cc:276] 0709 12:23:35.985223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:35.993383 (+  8160us) service_pool.cc:224] Handling call
0709 12:23:37.470299 (+1476916us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b54001a0254. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:37.470423 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3391515392}
I20260709 12:23:37.474613 16470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1360) took 1389 ms. Trace:
I20260709 12:23:37.475019 16470 rpcz_store.cc:276] 0709 12:23:36.085309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.089707 (+  4398us) service_pool.cc:224] Handling call
0709 12:23:36.089981 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:36.284929 (+194948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.474356 (+1189427us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b540012b954. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:37.474528 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":433630976}
I20260709 12:23:37.478202 17255 raft_consensus.cc:1240] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
W20260709 12:23:37.498994 16474 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34826 (request call id 1354) took 2816 ms (client timeout 2199 ms). Trace:
W20260709 12:23:37.500371 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:37.501039 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 43688.6us
I20260709 12:23:37.501729 19931 raft_consensus.cc:3055] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:37.502185 19931 raft_consensus.cc:740] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:23:37.499765 16474 rpcz_store.cc:269] 0709 12:23:34.682280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:34.682407 (+   127us) service_pool.cc:224] Handling call
0709 12:23:37.498654 (+2816247us) spinlock_profiling.cc:243] Waited 64.9 ms on lock 0x7b5400a690d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:37.498839 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":149267712}
I20260709 12:23:37.503163 19931 consensus_queue.cc:260] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:37.504688 19931 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:37.508780 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 81112.4us
I20260709 12:23:37.510138 20373 consensus_queue.cc:1059] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:37.510587 17258 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:37.511210 20373 raft_consensus.cc:3055] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:37.520483 20373 raft_consensus.cc:740] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
W20260709 12:23:37.522620 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.521818 20373 consensus_queue.cc:260] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:37.524313 16469 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 15.
I20260709 12:23:37.524824 20373 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:37.525619 20275 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:37.527012 17254 raft_consensus.cc:1240] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:37.527701 16441 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:37.528600 20300 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:37.529116 20300 raft_consensus.cc:697] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:37.529253 17254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 1067) took 1046 ms. Trace:
I20260709 12:23:37.529759 17254 rpcz_store.cc:276] 0709 12:23:36.482562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.450943 (+968381us) service_pool.cc:224] Handling call
0709 12:23:37.526823 (+ 75880us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.527358 (+   535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.529045 (+  1687us) spinlock_profiling.cc:243] Waited 75.6 ms on lock 0x7b54005f3550. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:37.529189 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":173833472}
I20260709 12:23:37.530102 20300 consensus_queue.cc:237] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:37.515673 20334 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:37.531473 17254 raft_consensus.cc:1240] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:37.531562 20334 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:37.516775 17261 raft_consensus.cc:1240] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 10 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:37.480050 17255 raft_consensus.cc:1275] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:23:37.526583 16469 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34840 (request call id 1250) took 1780 ms (client timeout 1697 ms). Trace:
I20260709 12:23:37.534900 17255 raft_consensus.cc:1275] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 12:23:37.534713 16469 rpcz_store.cc:269] 0709 12:23:35.746200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:35.750398 (+  4198us) service_pool.cc:224] Handling call
0709 12:23:37.526337 (+1775939us) spinlock_profiling.cc:243] Waited 1.76 s on lock 0x7b540021cd54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:37.526476 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4050978176}
I20260709 12:23:37.512146 16367 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:37.543270 20284 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:37.543928 20284 raft_consensus.cc:697] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:37.545111 20284 consensus_queue.cc:237] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:37.548517 19849 consensus_queue.cc:1059] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:37.549584 19849 raft_consensus.cc:3055] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:37.550004 19849 raft_consensus.cc:740] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:37.551056 19849 consensus_queue.cc:260] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:37.526495 17270 raft_consensus.cc:1275] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:23:37.569262 19849 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:37.571684 20279 consensus_queue.cc:1059] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:37.574059 20279 consensus_queue.cc:1048] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:37.576915 20334 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:37.586762 20271 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:37.589421 19898 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:37.590103 19898 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:37.592273 19898 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:37.595206 20267 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.586s	user 0.010s	sys 0.008s
W20260709 12:23:37.595721 20267 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.587s	user 0.010s	sys 0.008s
I20260709 12:23:37.612998 17270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 1057) took 1024 ms. Trace:
I20260709 12:23:37.613627 17270 rpcz_store.cc:276] 0709 12:23:36.588468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.471948 (+883480us) service_pool.cc:224] Handling call
0709 12:23:37.472285 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.596128 (+123843us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.612919 (+ 16791us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:37.616542 16399 raft_consensus.cc:1240] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 8 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:23:37.617951 20228 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:37.622365 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.623167 20267 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:37.623737 20267 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:37.625921 20267 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:37.627213 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:37.627926 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.630540 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:37.632234 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:37.633082 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:37.633467 16397 raft_consensus.cc:2393] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 14: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:37.633761 16474 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:37.543310 16367 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:37.636134 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:37.636289 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:37.636849 16394 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:37.637221 16397 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:37.641659 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:37.642324 16476 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:23:37.641834 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:37.643302 20373 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:37.648222 16403 raft_consensus.cc:1240] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:37.631448 16400 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:23:37.648749 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:37.649374 16469 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:37.650996 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:37.651546 16424 debug-util.cc:398] Leaking SignalData structure 0x7b080031be40 after lost signal to thread 16260
I20260709 12:23:37.651818 16468 raft_consensus.cc:2410] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:23:37.653185 16403 raft_consensus.cc:1240] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:37.653285 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:37.643903 20373 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:37.648707 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:37.654884 16470 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:37.655385 16474 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14.
I20260709 12:23:37.659399 16469 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:37.660669 16397 raft_consensus.cc:2410] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:23:37.661996 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:37.662817 16469 raft_consensus.cc:2410] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 12:23:37.664566 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 121321us
I20260709 12:23:37.666033 16394 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
I20260709 12:23:37.669677 16366 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:37.670476 16367 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:37.671049 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:37.671936 16474 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:37.673942 16367 leader_election.cc:400] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 10. Current term is 11.
I20260709 12:23:37.674315 16367 leader_election.cc:403] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:37.670938 16366 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:37.678362 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:37.678736 16367 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:37.723611 16476 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:37.736752 16366 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:37.755770 16470 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:23:37.746089 16366 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:37.790773 20361 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:37.791343 20361 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:37.791731 20361 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:38.241606 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:38.316537 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 637806us
I20260709 12:23:38.345960 16366 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:38.535211 20437 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:38.536546 20437 raft_consensus.cc:697] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 14 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:38.537791 20437 consensus_queue.cc:237] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:38.539215 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 221423us
I20260709 12:23:38.742962 20439 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:38.557284 20438 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:38.744645 20440 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:23:38.557997 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 212202us
I20260709 12:23:38.745002 20439 raft_consensus.cc:697] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:38.745313 20440 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:38.745827 20440 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:38.744946 20438 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:38.746249 20439 consensus_queue.cc:237] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:38.747046 20438 raft_consensus.cc:697] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:38.748275 20438 consensus_queue.cc:237] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:38.722869 20277 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 10. Current term is 11.
W20260709 12:23:38.723343 16424 debug-util.cc:398] Leaking SignalData structure 0x7b08004085a0 after lost signal to thread 16427
I20260709 12:23:38.754401 20442 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.724788 20361 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:38.755913 20440 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:38.759691 20361 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:38.760159 20440 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:38.761147 20440 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:38.757154 16396 raft_consensus.cc:1240] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 10. Current term is 11. Ops: [8.5-10.6]
I20260709 12:23:38.762187 16396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 994) took 1908 ms. Trace:
I20260709 12:23:38.762566 16396 rpcz_store.cc:276] 0709 12:23:36.853613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:38.756356 (+1902743us) service_pool.cc:224] Handling call
0709 12:23:38.756831 (+   475us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:38.761826 (+  4995us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:38.762119 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:38.764464 20442 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:38.764124 20440 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:38.755970 20277 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.766855 20440 raft_consensus.cc:697] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:38.767025 20277 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:38.768056 20440 consensus_queue.cc:237] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:38.773098 20280 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.773720 20280 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:38.775822 20280 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:38.770009 20439 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.776937 20439 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:38.779719 20439 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:38.780413 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:38.785092 20437 consensus_queue.cc:1059] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:23:38.785804 20361 consensus_queue.cc:1059] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:38.786790 20225 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:38.787101 20361 consensus_queue.cc:1048] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:38.788939 20361 raft_consensus.cc:3055] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:38.789440 20361 raft_consensus.cc:740] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:38.789974 20280 raft_consensus.cc:3055] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:38.791373 20361 consensus_queue.cc:260] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:38.792119 20280 raft_consensus.cc:740] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:38.803215 20280 consensus_queue.cc:260] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:38.769359 20277 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:38.804811 20280 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:38.805064 20361 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:38.773108 20442 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:38.799774 20271 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:38.801506 16396 raft_consensus.cc:1240] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 11. Current term is 12. Ops: []
W20260709 12:23:38.793234 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:38.807767 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:38.808269 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:38.808458 16542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1241) took 2313 ms. Trace:
I20260709 12:23:38.808844 16542 rpcz_store.cc:276] 0709 12:23:36.494767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.607581 (+112814us) service_pool.cc:224] Handling call
0709 12:23:36.607864 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:36.729823 (+121959us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:23:37.441978 (+712155us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b1800446860. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2875a314bc 00007f2872d7f48b 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d99455 00007f28798813c7 00007f2875e939a7 00007f2875ea30bf
0709 12:23:37.442137 (+   159us) log.cc:844] Serialized 14 byte log entry
0709 12:23:37.590441 (+148304us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:37.590498 (+    57us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:37.590550 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:38.808224 (+1217674us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b54000884d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:38.808272 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:38.808405 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":28008,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":2539,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"spinlock_wait_cycles":282468352,"thread_start_us":206431,"threads_started":2,"wal-append.queue_time_us":146970,"wal-append.run_cpu_time_us":1139,"wal-append.run_wall_time_us":1001149}
I20260709 12:23:38.810596 16367 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:38.811087 20401 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.811579 20401 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:38.811899 16367 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:38.786635 20437 consensus_queue.cc:1048] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:38.814405 16396 raft_consensus.cc:1240] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:38.816109 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:38.815362 20293 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:38.816545 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:38.811592 20271 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.819151 20271 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:38.813189 20438 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.820155 20438 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:38.821303 20271 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:38.815585 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:38.822316 20438 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:38.815677 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:38.826100 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:38.827008 16394 raft_consensus.cc:2410] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:23:38.827641 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:38.824606 20442 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:38.830171 16397 raft_consensus.cc:2393] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 10: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:38.831027 20401 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:38.838222 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:38.839149 16542 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:38.841506 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:38.842543 16396 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:23:38.815578 20446 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.852517 16542 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:23:38.853395 16551 raft_consensus.cc:1275] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:23:38.854377 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:38.853930 20446 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:38.857050 16396 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:38.858628 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:38.878229 16396 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:23:38.879328 16367 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:38.882084 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:38.884577 20280 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:38.870457 20446 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:38.890524 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:38.891906 16468 raft_consensus.cc:2410] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:23:38.897459 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:38.898783 16476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34826 (request call id 1371) took 1254 ms. Trace:
I20260709 12:23:38.899258 16542 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:38.899256 16476 rpcz_store.cc:276] 0709 12:23:37.644675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.724945 (+ 80270us) service_pool.cc:224] Handling call
0709 12:23:38.897243 (+1172298us) spinlock_profiling.cc:243] Waited 504 ms on lock 0x7b5800602a98. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:23:38.898689 (+  1446us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1158603648}
I20260709 12:23:38.900701 20440 consensus_queue.cc:1048] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:38.900746 20280 raft_consensus.cc:697] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:38.903786 20280 consensus_queue.cc:237] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:38.913769 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:38.915639 16468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34826 (request call id 1373) took 1270 ms. Trace:
I20260709 12:23:38.916460 16468 rpcz_store.cc:276] 0709 12:23:37.645298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.741682 (+ 96384us) service_pool.cc:224] Handling call
0709 12:23:38.890297 (+1148615us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b5800602a98. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:23:38.915546 (+ 25249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23099136}
I20260709 12:23:38.928366 16547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1240) took 2434 ms. Trace:
I20260709 12:23:38.930836 16547 rpcz_store.cc:276] 0709 12:23:36.493844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.493961 (+   117us) service_pool.cc:224] Handling call
0709 12:23:36.511735 (+ 17774us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:36.512717 (+   982us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:38.928026 (+2415309us) spinlock_profiling.cc:243] Waited 17.4 ms on lock 0x7b54008c55d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:38.928252 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39959808}
I20260709 12:23:38.931856 16545 raft_consensus.cc:1275] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:23:38.932507 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 50559.7us
I20260709 12:23:38.933493 16367 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:38.933741 16468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1378) took 1286 ms. Trace:
I20260709 12:23:38.933916 20441 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.934156 16468 rpcz_store.cc:276] 0709 12:23:37.646917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:38.919591 (+1272674us) service_pool.cc:224] Handling call
0709 12:23:38.933235 (+ 13644us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:38.933414 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:38.933440 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:38.933489 (+    49us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:38.933532 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:38.933587 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:38.933686 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:38.935485 20361 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:38.935910 20361 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:38.936239 20361 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:38.936296 16468 raft_consensus.cc:1275] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:23:38.936765 20275 consensus_queue.cc:1048] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 12:23:38.947834 16468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1379) took 1300 ms. Trace:
I20260709 12:23:38.948215 16468 rpcz_store.cc:276] 0709 12:23:37.647216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:38.935596 (+1288380us) service_pool.cc:224] Handling call
0709 12:23:38.935986 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:38.947566 (+ 11580us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:38.947778 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:38.936664 20334 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.949573 20334 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:38.951884 20334 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:38.934533 20441 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:38.952530 20334 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.953090 20334 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:38.956282 20334 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:38.954730 20441 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:38.957795 20334 consensus_queue.cc:1059] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:38.959437 20334 consensus_queue.cc:1059] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:38.939003 16516 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:38.963212 16400 raft_consensus.cc:1275] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:23:38.963845 16403 raft_consensus.cc:1275] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:23:38.965643 17263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 1027) took 3018 ms. Trace:
I20260709 12:23:38.960266 20334 consensus_queue.cc:1243] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer f99066d13e524238aec6aa2024a8bd50 log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:23:38.966037 17263 rpcz_store.cc:276] 0709 12:23:35.946745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:35.972427 (+ 25682us) service_pool.cc:224] Handling call
0709 12:23:35.981434 (+  9007us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:36.076990 (+ 95556us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:38.965434 (+2888444us) spinlock_profiling.cc:243] Waited 94.9 ms on lock 0x7b5400210054. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:38.965586 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":218333440}
I20260709 12:23:38.967212 20334 raft_consensus.cc:3055] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:38.967648 20334 raft_consensus.cc:740] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:38.968284 16394 raft_consensus.cc:1275] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:23:38.970203 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:38.970839 20225 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:38.970976 16547 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:38.972592 16367 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:38.973618 20441 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:38.974076 20441 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:38.974484 20441 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:38.962324 20361 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:38.983626 20361 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:38.941658 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:38.971583 20225 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:38.989801 20441 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:38.989853 16544 raft_consensus.cc:2410] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:23:38.984833 20334 consensus_queue.cc:260] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:38.994894 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:38.995472 16547 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:38.997092 20021 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:38.997902 20021 raft_consensus.cc:3055] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:38.998309 20021 raft_consensus.cc:740] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:38.999310 20021 consensus_queue.cc:260] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:39.001160 19969 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:39.001757 19969 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:39.002267 19969 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:39.004206 16547 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:39.006155 16367 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:39.007054 20280 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:39.007599 20280 raft_consensus.cc:697] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:39.008533 20280 consensus_queue.cc:237] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:39.011060 20302 consensus_queue.cc:1048] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:39.014853 20256 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:39.018939 20441 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:39.015623 20256 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:39.031339 20256 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:39.032903 16395 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58430 (request call id 1000) took 2162 ms (client timeout 1536 ms). Trace:
W20260709 12:23:39.033753 16395 rpcz_store.cc:269] 0709 12:23:36.870777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.876859 (+  6082us) service_pool.cc:224] Handling call
0709 12:23:39.032034 (+2155175us) spinlock_profiling.cc:243] Waited 2.13 s on lock 0x7b54005e8dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:39.032824 (+   790us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4904908160}
I20260709 12:23:39.035383 17269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 1022) took 3090 ms. Trace:
W20260709 12:23:39.057227 16424 debug-util.cc:398] Leaking SignalData structure 0x7b0800230640 after lost signal to thread 16501
I20260709 12:23:39.057842 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:39.058070 16424 debug-util.cc:398] Leaking SignalData structure 0x7b08002ab3a0 after lost signal to thread 16502
W20260709 12:23:39.060374 16424 debug-util.cc:398] Leaking SignalData structure 0x7b0800300b60 after lost signal to thread 16575
W20260709 12:23:39.062412 16424 debug-util.cc:398] Leaking SignalData structure 0x7b08002b3e40 after lost signal to thread 17299
W20260709 12:23:39.062795 16424 debug-util.cc:398] Leaking SignalData structure 0x7b0800116440 after lost signal to thread 17300
W20260709 12:23:39.063190 16424 debug-util.cc:398] Leaking SignalData structure 0x7b0800221fe0 after lost signal to thread 19880
W20260709 12:23:39.063225 20284 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.444s	user 0.013s	sys 0.002s
W20260709 12:23:39.063629 16424 debug-util.cc:398] Leaking SignalData structure 0x7b08003ddfa0 after lost signal to thread 19892
W20260709 12:23:39.064026 20284 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.444s	user 0.013s	sys 0.002s
W20260709 12:23:39.084255 17258 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.555s	user 0.043s	sys 0.049s
W20260709 12:23:39.084921 17258 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.556s	user 0.043s	sys 0.049s
I20260709 12:23:39.086004 17258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 1063) took 2495 ms. Trace:
I20260709 12:23:39.089026 20021 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:39.094642 16550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1233) took 2608 ms. Trace:
I20260709 12:23:39.095054 16550 rpcz_store.cc:276] 0709 12:23:36.486176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.504835 (+ 18659us) service_pool.cc:224] Handling call
0709 12:23:36.505108 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:37.218312 (+713204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:39.094392 (+1876080us) spinlock_profiling.cc:243] Waited 655 ms on lock 0x7b5400729554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:39.094584 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1506868480}
I20260709 12:23:39.086403 17258 rpcz_store.cc:276] 0709 12:23:36.590242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.527086 (+936844us) service_pool.cc:224] Handling call
0709 12:23:37.527521 (+   435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:37.527880 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:37.527913 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:37.528466 (+   553us) log.cc:844] Serialized 26 byte log entry
0709 12:23:39.085523 (+1557057us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:39.085588 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:39.085686 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:39.085745 (+    59us) raft_consensus.cc:1685] finished
0709 12:23:39.085762 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:39.085937 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1604,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":72,"thread_start_us":1556574,"threads_started":1,"wal-append.queue_time_us":250347,"wal-append.run_cpu_time_us":1704,"wal-append.run_wall_time_us":1008409}
I20260709 12:23:39.099318 19928 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:39.099506 19699 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:39.099992 19699 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:38.992095 20225 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:39.101580 20225 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.103504 20225 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:39.099910 19928 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:39.106012 20225 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:39.106534 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.106798 19928 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:39.107290 19928 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.107698 19928 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:39.109440 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:39.110214 16403 raft_consensus.cc:2393] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 11: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:39.113960 19928 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:39.125284 16498 debug-util.cc:398] Leaking SignalData structure 0x7b08000de240 after lost signal to thread 5355
I20260709 12:23:39.031894 20256 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.132943 20256 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:39.136169 20256 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:39.136911 20256 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.137562 20256 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:39.140805 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:39.141942 16474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34826 (request call id 1372) took 1497 ms. Trace:
I20260709 12:23:39.142364 16474 rpcz_store.cc:276] 0709 12:23:37.644994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.682811 (+ 37817us) service_pool.cc:224] Handling call
0709 12:23:39.140579 (+1457768us) spinlock_profiling.cc:243] Waited 9 ms on lock 0x7b5800602a98. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:23:39.141852 (+  1273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20691200}
W20260709 12:23:39.064213 16424 debug-util.cc:398] Leaking SignalData structure 0x7b0800333540 after lost signal to thread 19937
I20260709 12:23:39.071826 20158 consensus_queue.cc:1048] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:39.146826 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:39.077210 20290 consensus_queue.cc:1048] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:39.150733 20229 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.151389 20388 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.151348 20229 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:39.151952 20388 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:39.154140 20388 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:39.155725 20373 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:39.079981 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.080765 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:39.157100 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:39.151046 16222 catalog_manager.cc:5697] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 3 to 9, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 9 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:39.035969 17269 rpcz_store.cc:276] 0709 12:23:35.945417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:35.949857 (+  4440us) service_pool.cc:224] Handling call
0709 12:23:35.950142 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:36.529492 (+579350us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:39.035085 (+2505593us) spinlock_profiling.cc:243] Waited 575 ms on lock 0x7b540005f0d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:39.035312 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1321451264}
I20260709 12:23:39.157523 16544 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:39.081275 20300 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.532s	user 0.004s	sys 0.008s
W20260709 12:23:39.159004 20300 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.610s	user 0.005s	sys 0.008s
I20260709 12:23:39.159564 16399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 1149) took 2754 ms. Trace:
I20260709 12:23:39.159922 16476 raft_consensus.cc:1240] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 LEADER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:39.158005 16548 raft_consensus.cc:2410] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:23:39.163148 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:39.163854 16546 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:39.164691 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:39.165344 16469 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:23:39.165750 16469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34826 (request call id 1375) took 1519 ms. Trace:
I20260709 12:23:39.166746 16472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1292) took 2303 ms. Trace:
W20260709 12:23:39.168555 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.167138 16472 rpcz_store.cc:276] 0709 12:23:36.864163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.872197 (+  8034us) service_pool.cc:224] Handling call
0709 12:23:36.872686 (+   489us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:36.873176 (+   490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:36.873247 (+    71us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:36.880969 (+  7722us) log.cc:844] Serialized 26 byte log entry
0709 12:23:37.582100 (+701131us) spinlock_profiling.cc:243] Waited 4.71 ms on lock 0x7b1800328a20. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:23:37.668179 (+ 86079us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:37.668490 (+   311us) log.cc:844] Serialized 14 byte log entry
0709 12:23:39.166358 (+1497868us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:39.166435 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:39.166476 (+    41us) raft_consensus.cc:1685] finished
0709 12:23:39.166574 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:39.166691 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1497577,"prepare.queue_time_us":158,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":60,"raft.queue_time_us":7315,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":1317,"spinlock_wait_cycles":10835328,"thread_start_us":85909,"threads_started":1,"wal-append.queue_time_us":786175,"wal-append.run_cpu_time_us":2670,"wal-append.run_wall_time_us":1156814}
I20260709 12:23:39.170627 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:39.171227 16473 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:39.172135 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:39.172861 20218 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:39.173013 16473 raft_consensus.cc:2410] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:23:39.173403 20218 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:39.174045 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:39.174615 16473 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 15.
I20260709 12:23:39.175371 20218 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:39.175933 16424 debug-util.cc:398] Leaking SignalData structure 0x7b08002d0a20 after lost signal to thread 19943
I20260709 12:23:39.159958 16399 rpcz_store.cc:276] 0709 12:23:36.405381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.496434 (+ 91053us) service_pool.cc:224] Handling call
0709 12:23:36.496820 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:37.617063 (+1120243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:39.159267 (+1542204us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54004ecfd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:39.159497 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2528195328}
W20260709 12:23:39.178232 16515 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.724s	user 0.046s	sys 0.054s
I20260709 12:23:39.178093 16366 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
W20260709 12:23:39.178953 16515 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.724s	user 0.046s	sys 0.054s
W20260709 12:23:39.179286 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 1.72489e+06us
W20260709 12:23:39.176537 16424 debug-util.cc:398] Leaking SignalData structure 0x7b0800255200 after lost signal to thread 20073
W20260709 12:23:39.180795 16424 debug-util.cc:398] Leaking SignalData structure 0x7b08003a1840 after lost signal to thread 20278
I20260709 12:23:39.183411 20229 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:39.183694 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:39.187775 16468 raft_consensus.cc:2410] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:23:39.187911 20116 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.188594 20116 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:39.189023 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:39.190611 16468 raft_consensus.cc:1275] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 12:23:39.190943 16468 pending_rounds.cc:85] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 3
I20260709 12:23:39.191192 16468 pending_rounds.cc:107] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:23:39.191030 20116 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:39.191643 16468 raft_consensus.cc:2889] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:39.193549 20380 consensus_queue.cc:1048] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:39.187846 20218 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.197140 20396 consensus_queue.cc:1059] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:39.197566 20218 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:39.198276 20396 raft_consensus.cc:3055] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:39.198719 20396 raft_consensus.cc:740] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:39.199766 20218 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:39.184485 16366 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:39.201244 16469 rpcz_store.cc:276] 0709 12:23:37.645876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.665508 (+ 19632us) service_pool.cc:224] Handling call
0709 12:23:39.128149 (+1462641us) spinlock_profiling.cc:243] Waited 56.5 ms on lock 0x7b5800602a98. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:23:39.165700 (+ 37551us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":129970560}
I20260709 12:23:39.202275 20295 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:39.202690 20295 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:39.203068 20295 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:39.204694 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:39.168145 20408 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.662s	user 0.018s	sys 0.018s
I20260709 12:23:39.205399 16547 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
W20260709 12:23:39.205739 20408 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.700s	user 0.018s	sys 0.018s
I20260709 12:23:39.206362 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:39.207021 16547 raft_consensus.cc:2468] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:39.207202 20208 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:39.175396 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:39.207908 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:39.212661 19699 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:39.213732 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:39.186947 20390 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:39.199584 20396 consensus_queue.cc:260] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:39.214805 16468 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:23:39.237551 16546 raft_consensus.cc:1275] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:23:39.228300 16498 debug-util.cc:398] Leaking SignalData structure 0x7b08003f6ae0 after lost signal to thread 16260
I20260709 12:23:39.237622 20390 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:39.239029 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:39.239490 16498 debug-util.cc:398] Leaking SignalData structure 0x7b080036bb40 after lost signal to thread 16427
I20260709 12:23:39.239753 16546 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
W20260709 12:23:39.240806 16498 debug-util.cc:398] Leaking SignalData structure 0x7b08004021c0 after lost signal to thread 16501
I20260709 12:23:39.217468 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:39.241233 16498 debug-util.cc:398] Leaking SignalData structure 0x7b0800312880 after lost signal to thread 16502
I20260709 12:23:39.241961 16366 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:39.227227 20314 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:39.207765 20208 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:39.243104 20267 raft_consensus.cc:2749] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:39.243872 20267 consensus_queue.cc:1048] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:39.245056 20208 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:39.231887 16476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1370) took 1587 ms. Trace:
I20260709 12:23:39.245965 16476 rpcz_store.cc:276] 0709 12:23:37.644323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:38.919157 (+1274834us) service_pool.cc:224] Handling call
0709 12:23:38.921994 (+  2837us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:39.160537 (+238543us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:39.231605 (+ 71068us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b5400185954. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:39.231823 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":546570496}
I20260709 12:23:39.214972 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:39.247469 16543 raft_consensus.cc:2393] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 13: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:39.248608 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:39.249676 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:39.250542 16477 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:39.250669 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:39.251180 16477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:34826 (request call id 1350) took 5073 ms. Trace:
I20260709 12:23:39.251547 16477 rpcz_store.cc:276] 0709 12:23:34.184174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:34.193420 (+  9246us) service_pool.cc:224] Handling call
0709 12:23:39.249382 (+5055962us) spinlock_profiling.cc:243] Waited 2.17 s on lock 0x7b5400a681d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e80a3d 00007f287ae50cd0 00007f28743d5254 00007f28743d51e2 00007f28743d5111 00007f28743d508c 00007f28743d4342 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:39.251092 (+  1710us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4996714880}
W20260709 12:23:39.255098 16498 debug-util.cc:398] Leaking SignalData structure 0x7b080027cc00 after lost signal to thread 16575
I20260709 12:23:39.255997 16469 raft_consensus.cc:1275] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 12:23:39.256639 16498 debug-util.cc:398] Leaking SignalData structure 0x7b0800145b60 after lost signal to thread 17299
W20260709 12:23:39.257025 16498 debug-util.cc:398] Leaking SignalData structure 0x7b08002d4480 after lost signal to thread 17300
I20260709 12:23:39.257174 20334 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:39.257905 19969 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:39.258679 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.259668 16367 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:39.261814 20367 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:39.262269 20367 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:39.262650 20367 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:39.265467 20392 raft_consensus.cc:3055] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:39.265969 20392 raft_consensus.cc:740] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:39.267264 16470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1305) took 1626 ms. Trace:
I20260709 12:23:39.266925 20392 consensus_queue.cc:260] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:39.268276 20392 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:39.256093 20390 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:39.277925 20396 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:39.284842 20390 consensus_queue.cc:1059] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:39.286278 20390 consensus_queue.cc:1243] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer 610f4feda11942739236f2b121a4b9c1 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:23:39.286849 20116 raft_consensus.cc:3055] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:39.287380 20116 raft_consensus.cc:740] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:39.208536 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:39.209520 20173 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.288283 20173 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:39.288476 20116 consensus_queue.cc:260] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:39.288360 20373 consensus_queue.cc:1048] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:39.289520 20116 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:39.289770 20295 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:39.290810 20173 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:39.291447 16474 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:39.210301 16516 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:39.211076 17264 raft_consensus.cc:1240] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 10 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 8. Current term is 10. Ops: []
I20260709 12:23:39.186476 20239 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.292968 20239 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:39.211735 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:39.295225 17264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 1056) took 2709 ms. Trace:
I20260709 12:23:39.295738 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:39.295642 17264 rpcz_store.cc:276] 0709 12:23:36.586256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.456115 (+869859us) service_pool.cc:224] Handling call
0709 12:23:37.576321 (+120206us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:39.292772 (+1716451us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:39.294343 (+  1571us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b540028fd54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:39.295025 (+   682us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b540028fd50. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:39.295168 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4010920704}
I20260709 12:23:39.297914 16548 raft_consensus.cc:1275] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:23:39.300348 20239 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:39.300529 16550 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:23:39.301314 16548 raft_consensus.cc:1240] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 10 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:23:39.287109 20267 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:23:39.305691 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.306425 16222 catalog_manager.cc:5697] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 1 to 5, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 5 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
W20260709 12:23:39.311686 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 261169us
W20260709 12:23:39.313056 16514 connection.cc:256] RPC call timeout handler was delayed by -1.3348518100001456s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1814s for a response
W20260709 12:23:39.313627 16514 connection.cc:256] RPC call timeout handler was delayed by -1.2484518100000059s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.191s for a response
I20260709 12:23:39.314070 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:39.314662 16476 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:39.318578 20284 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:39.321094 20284 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:39.321462 20284 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:39.325167 20242 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:23:39.325209 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 65638.7us
I20260709 12:23:39.326119 20242 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:39.326511 20242 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:39.331207 16471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1297) took 2464 ms. Trace:
I20260709 12:23:39.331647 16471 rpcz_store.cc:276] 0709 12:23:36.867208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.931903 (+ 64695us) service_pool.cc:224] Handling call
0709 12:23:36.955685 (+ 23782us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:36.956104 (+   419us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:36.956129 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:39.195059 (+2238930us) log.cc:844] Serialized 26 byte log entry
0709 12:23:39.324584 (+129525us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:39.324645 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:39.324743 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:39.330986 (+  6243us) raft_consensus.cc:1685] finished
0709 12:23:39.331007 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:39.331141 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":104,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":11805,"thread_start_us":129048,"threads_started":1,"wal-append.queue_time_us":128714}
I20260709 12:23:39.334882 20463 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.335486 20463 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:39.267683 16470 rpcz_store.cc:276] 0709 12:23:37.640930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.765104 (+124174us) service_pool.cc:224] Handling call
0709 12:23:37.781080 (+ 15976us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:39.167237 (+1386157us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:23:39.167266 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:39.167855 (+   589us) log.cc:844] Serialized 52 byte log entry
0709 12:23:39.266831 (+ 98976us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:39.266875 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:39.266944 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:39.266990 (+    46us) raft_consensus.cc:1685] finished
0709 12:23:39.267001 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:39.267208 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1385586,"raft.queue_time_us":1405642,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"thread_start_us":98600,"threads_started":1,"wal-append.queue_time_us":95566}
I20260709 12:23:39.338593 20463 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:39.208222 16473 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:39.341943 20479 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.342482 20479 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:39.342988 20290 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:39.343593 20290 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:39.251856 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:39.347384 16543 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
I20260709 12:23:39.348668 16544 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:23:39.310961 16222 catalog_manager.cc:5697] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 4 to 7, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 7 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:39.352155 16476 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:23:39.356426 16472 raft_consensus.cc:1275] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:23:39.324079 16474 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
W20260709 12:23:39.366537 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 156310us
I20260709 12:23:39.292048 20295 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:39.395578 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.403649 16549 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:23:39.371331 16367 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:39.345209 20479 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:39.359553 16366 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:39.403841 16475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1294) took 2537 ms. Trace:
I20260709 12:23:39.409580 16470 raft_consensus.cc:1275] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:23:39.410887 16547 raft_consensus.cc:2410] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
W20260709 12:23:39.411823 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.347157 20256 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:39.413399 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:39.413870 16515 connection.cc:256] RPC call timeout handler was delayed by -1.36151355300035s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15480000000000002s for a response
I20260709 12:23:39.409754 16475 rpcz_store.cc:276] 0709 12:23:36.866379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.931356 (+ 64977us) service_pool.cc:224] Handling call
0709 12:23:36.931795 (+   439us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:36.946911 (+ 15116us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:36.946965 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:36.954860 (+  7895us) log.cc:844] Serialized 52 byte log entry
0709 12:23:39.400173 (+2445313us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:39.400817 (+   644us) log.cc:844] Serialized 14 byte log entry
0709 12:23:39.403461 (+  2644us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:39.403554 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:39.403604 (+    50us) raft_consensus.cc:1685] finished
0709 12:23:39.403621 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:39.403774 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14475,"prepare.queue_time_us":194,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":19716,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":10007,"thread_start_us":444583,"threads_started":1,"wal-append.queue_time_us":2394953}
W20260709 12:23:39.414909 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 43682.5us
W20260709 12:23:39.416646 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:39.417260 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.357762 19969 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:39.421777 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 62331.8us
I20260709 12:23:39.422760 16366 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:39.428478 16515 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:39.408138 20173 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:39.436676 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:39.437623 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 67009us
I20260709 12:23:39.437429 16394 raft_consensus.cc:1275] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:39.436076 16396 raft_consensus.cc:1240] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 10. Current term is 11. Ops: []
W20260709 12:23:39.439498 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.440297 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:39.440886 16442 connection.cc:256] RPC call timeout handler was delayed by -3.3737031529999513s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2135s for a response
I20260709 12:23:39.443310 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:39.444568 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:39.451555 16442 connection.cc:256] RPC call timeout handler was delayed by -3.3530031529999178s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2158s for a response
W20260709 12:23:39.452086 16442 connection.cc:256] RPC call timeout handler was delayed by -3.1302861730000586s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1836s for a response
W20260709 12:23:39.452567 16442 connection.cc:256] RPC call timeout handler was delayed by -3.1176861730000383s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18500000000000003s for a response
W20260709 12:23:39.446281 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.409714 20290 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:39.449209 16472 raft_consensus.cc:1240] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
W20260709 12:23:39.452894 16442 connection.cc:256] RPC call timeout handler was delayed by -2.1969861729999138s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28730000000000006s for a response
I20260709 12:23:39.453624 20225 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.454217 17144 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:39.461267 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:39.456171 16403 raft_consensus.cc:1275] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:23:39.461912 16474 raft_consensus.cc:2376] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12. Re-sending same reply.
I20260709 12:23:39.458225 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:39.457957 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:39.419266 20437 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:39.463201 16472 raft_consensus.cc:2410] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:23:39.415256 19699 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:39.420462 20348 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.447417 20479 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.467272 20479 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:39.472043 19699 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:39.437350 19849 consensus_queue.cc:1048] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:23:39.437543 20380 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:39.474797 20380 raft_consensus.cc:697] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:39.460290 20251 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.475977 20380 consensus_queue.cc:237] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:39.476672 20251 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:39.478950 20251 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:39.479529 20251 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.462265 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:39.481585 16401 raft_consensus.cc:2393] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 13: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:39.460641 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:39.459043 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:39.448022 20463 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.430610 20276 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.483876 20463 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:39.484431 20173 raft_consensus.cc:697] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 12 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:39.484393 20276 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:39.485519 20173 consensus_queue.cc:237] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:39.486752 20276 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:39.487082 20463 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:39.463347 16476 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:39.487726 20463 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.458792 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:39.488451 20437 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:39.488873 16471 raft_consensus.cc:1240] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:39.489799 20409 consensus_queue.cc:1059] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:39.490121 19699 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:39.473464 20348 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:39.490998 20409 raft_consensus.cc:3055] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:39.491377 20409 raft_consensus.cc:740] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:39.430094 20256 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.447729 20282 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.459308 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:39.496717 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.429910 20477 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.437680 20229 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.457218 16515 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:39.501091 16475 raft_consensus.cc:1240] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:39.503769 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:39.453934 20408 consensus_queue.cc:1048] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 12:23:39.505307 16476 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:23:39.502784 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:39.447533 20386 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.509399 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:39.508675 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:39.506102 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 164637us
I20260709 12:23:39.446877 20475 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:39.506862 16475 raft_consensus.cc:2410] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:23:39.510216 16474 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:23:39.454629 20465 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.492300 20348 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:39.470522 20479 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:39.463807 20302 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:39.492614 20208 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:39.465350 20225 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:39.484040 20407 consensus_queue.cc:1059] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 12:23:39.488915 20437 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:39.507354 20290 consensus_queue.cc:1059] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:23:39.446417 20367 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:39.508168 20229 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:39.480057 20251 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:39.488229 20463 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:39.495426 20284 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:39.492208 20409 consensus_queue.cc:260] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:39.506403 20282 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:39.510748 20475 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:39.568499 20409 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:39.691628 20475 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:39.618304 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:39.692992 16395 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:23:39.509496 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:39.694510 20284 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:39.694968 16472 raft_consensus.cc:2393] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 10: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:39.459767 16399 raft_consensus.cc:1275] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 12:23:39.629221 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:39.696369 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:23:39.696919 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 272912us
W20260709 12:23:39.698204 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 275308us
W20260709 12:23:39.516000 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 58865.4us
I20260709 12:23:39.632899 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:39.698844 16367 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:39.637301 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:39.615710 16222 catalog_manager.cc:5697] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
W20260709 12:23:39.649569 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 121981us
I20260709 12:23:39.652748 16442 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:39.655084 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:39.655758 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:39.656363 17269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 1046) took 2058 ms. Trace:
I20260709 12:23:39.657102 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:39.657298 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:39.658254 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:39.699504 20478 consensus_queue.cc:1048] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 12:23:39.671420 20116 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.528977 20208 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:39.715746 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:39.717200 16474 raft_consensus.cc:2410] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:23:39.716912 20116 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:39.718382 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:39.719273 16474 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:23:39.719918 16399 pending_rounds.cc:85] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 5
I20260709 12:23:39.720484 16399 pending_rounds.cc:107] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 12:23:39.720968 16399 raft_consensus.cc:2889] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:39.721055 16515 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:39.723387 20296 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:39.723877 20296 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:39.513564 20386 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:39.506057 20256 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:39.685169 20408 consensus_queue.cc:1048] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:39.687484 20251 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:39.520829 20465 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:39.746559 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:39.687945 20407 consensus_queue.cc:1059] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:39.748555 17263 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:39.748811 20407 consensus_queue.cc:1243] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer f99066d13e524238aec6aa2024a8bd50 log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:23:39.749466 20407 raft_consensus.cc:3055] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:39.749792 20407 raft_consensus.cc:740] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:39.698264 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:39.507505 20477 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:39.755836 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:39.757234 16401 raft_consensus.cc:2393] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 12: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:39.757668 20477 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:39.759192 20225 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:39.716284 20208 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:39.645002 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:39.664721 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:39.691434 20282 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:39.708978 17271 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:39.713547 17264 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:39.687485 20367 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:39.683717 20242 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:39.733196 20296 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:39.759629 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:39.634342 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:39.760711 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 199142us
I20260709 12:23:39.761543 17144 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:39.763002 20229 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:39.763895 19928 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:39.740747 20116 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:39.764914 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:39.765421 20386 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:39.724318 16515 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:39.766342 20143 raft_consensus.cc:2764] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
W20260709 12:23:39.766479 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 67853.5us
I20260709 12:23:39.767576 19849 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:39.794525 19849 raft_consensus.cc:697] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:39.794965 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:39.795702 16469 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:23:39.796715 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:39.782626 20275 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:39.797562 20275 raft_consensus.cc:697] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:39.797133 19849 consensus_queue.cc:237] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:39.799011 20275 consensus_queue.cc:237] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:39.797132 16468 raft_consensus.cc:2393] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 10: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
W20260709 12:23:39.767733 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 115608us
I20260709 12:23:39.751858 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:39.748230 16475 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:23:39.770687 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:23:39.772109 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.797585 16469 raft_consensus.cc:2410] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 12:23:39.774348 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.729943 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:39.728648 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:39.631393 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:39.777323 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.776238 20256 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:39.804298 16471 raft_consensus.cc:2393] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 9: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:39.729959 20302 raft_consensus.cc:697] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 12 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:39.801426 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:39.810340 16477 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:23:39.792915 20242 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:39.725672 16222 catalog_manager.cc:5697] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 4 to 5, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 5 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:39.816946 20465 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:39.817351 16475 raft_consensus.cc:1275] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 12:23:39.818109 16475 pending_rounds.cc:85] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 6
I20260709 12:23:39.818430 16475 pending_rounds.cc:107] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:23:39.818813 16475 raft_consensus.cc:2889] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:39.820662 20302 consensus_queue.cc:237] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:39.825385 20493 log.cc:927] Time spent T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.062s	user 0.000s	sys 0.002s
W20260709 12:23:39.833086 20488 log.cc:927] Time spent T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.284s	user 0.003s	sys 0.008s
I20260709 12:23:39.833513 16442 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:39.835275 20396 consensus_queue.cc:1059] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:39.701104 17269 rpcz_store.cc:276] 0709 12:23:37.598093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:39.192470 (+1594377us) service_pool.cc:224] Handling call
0709 12:23:39.192817 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:39.314356 (+121539us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:39.314401 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:39.314953 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 12:23:39.415440 (+100487us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:39.415475 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:39.415535 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:39.655814 (+240279us) raft_consensus.cc:1685] finished
0709 12:23:39.655848 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:39.656131 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":40591,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":91,"thread_start_us":221278,"threads_started":2,"wal-append.queue_time_us":212279}
I20260709 12:23:39.837687 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:39.838294 16547 raft_consensus.cc:1275] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:23:39.838469 17269 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:39.776083 19908 consensus_queue.cc:1048] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 12:23:39.845831 16547 raft_consensus.cc:1275] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:23:39.847358 16396 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:23:39.848704 16547 raft_consensus.cc:1240] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:39.849004 17261 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:23:39.850072 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:39.850942 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:39.853408 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:39.854107 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:39.854326 16549 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:23:39.855436 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:39.851464 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:39.852072 20463 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:39.851293 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:39.859575 17266 raft_consensus.cc:2468] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:39.858016 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:39.860726 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:39.832193 17142 leader_election.cc:336] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.969s (SENT)
I20260709 12:23:39.861549 16547 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:23:39.751225 20407 consensus_queue.cc:260] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:39.872102 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 128861us
I20260709 12:23:39.873971 20256 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:39.874480 20256 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:39.874917 20256 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:39.874507 20463 consensus_queue.cc:1048] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:39.878428 20479 raft_consensus.cc:3055] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:23:39.878698 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 154667us
I20260709 12:23:39.879005 20479 raft_consensus.cc:740] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:39.879202 19969 raft_consensus.cc:3055] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:39.879652 19969 raft_consensus.cc:740] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:39.880368 20334 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:39.880926 19969 consensus_queue.cc:260] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:39.882349 20290 consensus_queue.cc:1059] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:23:39.884325 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.886638 16544 raft_consensus.cc:1275] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:23:39.888139 20242 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.888955 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:39.888833 20242 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:39.889807 16542 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:39.891893 20158 consensus_queue.cc:1048] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:39.893116 16514 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:39.893465 20334 consensus_queue.cc:1059] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:39.894713 17144 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:39.894992 20407 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:39.895080 20334 raft_consensus.cc:3055] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:39.895826 20334 raft_consensus.cc:740] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:39.869391 17142 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:39.904727 20158 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:39.904044 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:23:39.910521 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 74177.8us
I20260709 12:23:39.916705 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:39.920579 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:39.922070 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:39.923333 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:39.924499 17266 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:23:39.927322 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.928972 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:39.930327 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:39.931316 16545 raft_consensus.cc:2393] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 12: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:39.909399 20218 raft_consensus.cc:3055] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:39.932932 20218 raft_consensus.cc:740] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:39.933256 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:39.932302 16549 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:23:39.898916 20242 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:39.944981 17263 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14.
I20260709 12:23:39.934224 20218 consensus_queue.cc:260] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:39.905732 20396 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:39.947381 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:39.951174 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:39.942745 20463 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:39.957159 17271 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:23:39.956398 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:39.958053 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 66397.9us
I20260709 12:23:39.955602 17269 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:23:39.959039 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:39.967706 16547 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
W20260709 12:23:39.969044 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:39.969406 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 137306us
W20260709 12:23:39.970013 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.880219 20479 consensus_queue.cc:260] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:39.957849 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:39.980674 16470 raft_consensus.cc:1240] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:39.981668 16475 raft_consensus.cc:1275] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:23:40.005209 16545 raft_consensus.cc:1275] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:23:40.004403 17264 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:39.972571 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:40.012310 17266 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:23:39.977375 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:39.956429 19969 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:39.958652 20463 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:39.976667 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:40.011242 16395 raft_consensus.cc:2376] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13. Re-sending same reply.
I20260709 12:23:40.024570 16542 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:23:39.979317 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:40.025650 16515 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:39.977115 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:39.973523 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:39.973618 20218 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:39.972747 20251 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:39.978964 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:40.025790 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:40.038802 16473 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:40.033213 17261 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:40.041601 16468 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:40.041191 20396 consensus_queue.cc:1059] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:40.042263 17143 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:40.043243 20396 raft_consensus.cc:3055] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:40.043557 17143 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:40.044471 20478 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:40.044879 20478 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:40.045262 20478 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:40.045871 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:40.049022 17144 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:40.051565 16515 leader_election.cc:400] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 9. Current term is 10.
I20260709 12:23:40.051935 16515 leader_election.cc:403] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:23:40.052064 20477 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:40.052523 16514 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:40.052706 20477 raft_consensus.cc:697] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:40.053107 20021 consensus_queue.cc:1059] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:40.054178 20021 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:40.053849 20477 consensus_queue.cc:237] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:40.054476 20470 raft_consensus.cc:3055] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:40.054898 16514 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:40.055428 20457 consensus_queue.cc:1048] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:40.055855 16402 raft_consensus.cc:2376] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11. Re-sending same reply.
I20260709 12:23:40.056679 20173 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:39.972211 20296 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:40.058372 16549 raft_consensus.cc:1240] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:40.059438 20296 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:40.060207 17254 raft_consensus.cc:2376] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12. Re-sending same reply.
I20260709 12:23:40.030871 20463 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:40.043694 20396 raft_consensus.cc:740] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:40.064131 17266 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:23:40.064832 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:40.079821 20479 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:40.080304 16515 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:40.081360 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:40.082782 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:40.083940 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:40.052616 20465 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:40.084784 20021 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 9. Current term is 10.
I20260709 12:23:40.086790 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:40.066570 16367 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:23:40.098042 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 54633.9us
W20260709 12:23:40.100597 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:40.054996 20470 raft_consensus.cc:740] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:40.108075 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:40.116523 16395 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:23:40.117086 19908 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:40.117615 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:40.079836 20208 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:40.104511 20229 raft_consensus.cc:2804] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:40.118803 20229 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:40.119160 20229 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:40.120232 20208 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:40.120885 20158 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:39.984760 20334 consensus_queue.cc:260] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:40.116292 16442 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:40.124042 16401 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:40.127228 17143 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:40.133724 17261 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:23:40.136322 20380 consensus_queue.cc:1048] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:40.138134 19908 consensus_queue.cc:1059] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:23:40.139042 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 72848.6us
I20260709 12:23:40.139210 19908 raft_consensus.cc:3055] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:40.139686 19908 raft_consensus.cc:740] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:40.141927 19908 consensus_queue.cc:260] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:40.140300 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:40.145292  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:40.145422 20302 consensus_queue.cc:1059] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 12:23:40.148953 16515 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:40.149813 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:23:40.150869 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 53508.4us
I20260709 12:23:40.153506 16396 raft_consensus.cc:1240] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:40.121613 20158 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:40.164748 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:40.170676 16402 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 11: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:40.175630 16468 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:23:40.174364 17261 raft_consensus.cc:1275] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 12:23:40.180294 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 57600.5us
I20260709 12:23:40.185381 17143 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:40.183542 16366 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:40.167655 20158 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:40.189339 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:40.198273 17270 raft_consensus.cc:2410] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:23:40.208036 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:40.209234 20367 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:40.209836 20367 raft_consensus.cc:697] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:40.180307 20409 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:40.210934 20367 consensus_queue.cc:237] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:40.204619 20208 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:40.205098 20290 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:40.213794 17143 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:40.216643 16516 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:40.215150 20229 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:40.228129 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:40.229698 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:40.231031 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:40.218609 20173 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:40.237927 16473 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:40.232142 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:40.235697 17258 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
W20260709 12:23:40.267349 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:40.286145 17254 raft_consensus.cc:1275] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:23:40.295849 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:40.305114 20229 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:40.305720 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:40.306998 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:40.152938 20470 consensus_queue.cc:260] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:40.309274 20470 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:23:40.310670 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 186033us
I20260709 12:23:40.305105 17144 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:40.311673 16515 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:40.312137 20229 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:40.300585 20463 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:40.312924 16515 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:40.313675 19969 raft_consensus.cc:2804] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:40.314201 19969 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:40.314594 19969 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:40.315021 16515 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:40.315306 20465 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:40.316723 17144 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:40.315019 20463 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:40.308174 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:23:40.318020 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:40.270821 20208 consensus_queue.cc:1048] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:23:40.296602 20251 consensus_queue.cc:1048] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 12:23:40.294950 19908 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:40.294945 20320 consensus_queue.cc:1059] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:23:40.238308 20173 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:40.323256 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:40.368399 16398 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:40.301872 17143 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:40.327843 20334 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:23:40.328578 20437 consensus_meta.cc:287] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.771s	user 0.007s	sys 0.010s
W20260709 12:23:40.328905 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 272530us
I20260709 12:23:40.302886 20396 consensus_queue.cc:260] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:40.330096 20158 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:40.330528 20478 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:40.331184 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 127095us
I20260709 12:23:40.335637 20256 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:40.350628 16401 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
W20260709 12:23:40.370229 20482 log.cc:927] Time spent T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.078s	user 0.002s	sys 0.000s
I20260709 12:23:40.373860 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:40.286407 20409 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:40.379634 16396 raft_consensus.cc:2393] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 10: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
W20260709 12:23:40.379460 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 64511.5us
W20260709 12:23:40.398798 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 163151us
W20260709 12:23:40.406450 17142 leader_election.cc:336] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 2.726s (SENT)
I20260709 12:23:40.380674 20173 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:40.339661 19969 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:40.418038 20267 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.390372 20437 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:40.418849 20267 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:40.415207 20302 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:40.420853 20437 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:40.421188 20302 raft_consensus.cc:697] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 12 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:40.422256 20478 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:40.422565 20302 consensus_queue.cc:237] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:40.425806 20158 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:40.426630 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:40.446231 16442 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:40.448748 20479 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:40.457234 17255 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 12: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:40.408516 20290 raft_consensus.cc:3055] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:40.464025 16397 raft_consensus.cc:1240] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:40.456750 20218 raft_consensus.cc:2749] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:40.457015 20463 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:40.455528 20479 raft_consensus.cc:697] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:23:40.480970 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 81144.4us
I20260709 12:23:40.468355 20290 raft_consensus.cc:740] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:40.497521 17143 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:40.482718 20463 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:40.482091 20396 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:40.505107 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 188462us
W20260709 12:23:40.510406 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 104137us
I20260709 12:23:40.518878 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:40.519752 16394 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:40.522558 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:40.527168 16542 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:40.483125 19849 consensus_queue.cc:1048] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.026s
W20260709 12:23:40.528183 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 44365.6us
I20260709 12:23:40.530439 17143 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:40.513681 20368 consensus_queue.cc:1048] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 12:23:40.494509 20437 consensus_queue.cc:1059] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:23:40.507856 20463 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:40.534111 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RunLeaderElection -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 45040.3us
I20260709 12:23:40.538005 20409 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:40.538816 16544 raft_consensus.cc:1275] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:23:40.540346 20208 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:40.541007 20208 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:40.541571 20208 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:40.544622 20479 consensus_queue.cc:237] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:40.547173 20290 consensus_queue.cc:260] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:40.548568 20290 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:40.548888 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:40.553678 20461 log.cc:927] Time spent T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.236s	user 0.002s	sys 0.000s
I20260709 12:23:40.556499 20218 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.557112 20218 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:40.559217 20218 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:40.563349 20242 consensus_queue.cc:1048] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:40.565623 20242 consensus_queue.cc:1048] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:23:40.570585 20500 log.cc:927] Time spent T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.465s	user 0.000s	sys 0.002s
I20260709 12:23:40.577469 20256 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:40.578961 20501 log.cc:927] Time spent T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.443s	user 0.004s	sys 0.001s
I20260709 12:23:40.580854 19969 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:40.584813 20267 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:40.585112 16397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 1016) took 1150 ms. Trace:
I20260709 12:23:40.585577 16397 rpcz_store.cc:276] 0709 12:23:39.434694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:39.434846 (+   152us) service_pool.cc:224] Handling call
0709 12:23:39.435104 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:40.464606 (+1029502us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:40.584795 (+120189us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540011fb54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:40.585030 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2337328000}
I20260709 12:23:40.586539 20208 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:40.588513 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 142452us
I20260709 12:23:40.588882 16514 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:40.590556 16442 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:40.592593 20021 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:40.592859 20437 raft_consensus.cc:3055] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:40.593273 20437 raft_consensus.cc:740] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:40.594180 20437 consensus_queue.cc:260] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:40.595403 20437 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:40.595669 20208 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:40.548666 20506 log.cc:927] Time spent T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:23:40.600261 16398 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
W20260709 12:23:40.599004 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 68680.3us
I20260709 12:23:40.599457 16442 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:40.592085 16514 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:40.609417 17143 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:23:40.614121 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:40.620689 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:40.633098 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:40.636652 17271 raft_consensus.cc:2393] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 13: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:40.638993 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: true dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:40.639711 16544 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:23:40.644582 20517 log.cc:927] Time spent T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.323s	user 0.005s	sys 0.000s
I20260709 12:23:40.645484 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:40.646287 20229 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:40.646878 20229 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:40.647329 20229 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:40.645530 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 44065.8us
I20260709 12:23:40.648375 17143 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:40.649395 17143 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:40.651381 20225 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:40.651999 20225 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:40.652469 20225 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:40.652189 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:40.655114 20477 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:40.620376 20320 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:40.623844 20218 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:40.657377 17264 raft_consensus.cc:1275] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:23:40.657989 20218 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:40.644455 20208 consensus_queue.cc:1059] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:40.659092 20208 raft_consensus.cc:3055] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:40.659489 20208 raft_consensus.cc:740] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:23:40.659755 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:40.660193 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 44739.4us
I20260709 12:23:40.660476 20208 consensus_queue.cc:260] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:40.661880 20208 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:40.634495 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:40.663233 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:40.669453 16397 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:40.673224 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:40.671113 17144 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:40.676023 16402 raft_consensus.cc:2410] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:23:40.683844 16366 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:40.686017 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:40.686471 16542 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:23:40.688361 16394 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
I20260709 12:23:40.688939 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:40.687359 16516 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:40.695957 17258 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 16.
I20260709 12:23:40.662441 20284 consensus_queue.cc:1048] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 12:23:40.705143 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:40.710024 17261 raft_consensus.cc:1275] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:23:40.710901 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:40.711524 17254 raft_consensus.cc:2393] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 16: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:40.712180 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:40.715231 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:40.716688 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:40.717859 17264 raft_consensus.cc:2393] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 13: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:40.695182 20256 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:23:40.720265 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 50969.7us
I20260709 12:23:40.720921 16544 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:40.721346 17270 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:40.723531 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: true dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:40.728554 20320 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:40.728950 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:40.728574 17263 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:40.720577 17144 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:40.729684 16547 raft_consensus.cc:1275] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:23:40.729664 17261 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:23:40.738314 16397 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:23:40.688037 20173 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:40.740141 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 148148us
I20260709 12:23:40.749449 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:40.752868 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:40.759690 16543 raft_consensus.cc:2410] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:23:40.764961 16402 raft_consensus.cc:1275] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:40.742235 20320 raft_consensus.cc:697] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:23:40.766841 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:40.767437 20320 consensus_queue.cc:237] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:40.768415 20267 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.768992 20267 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:40.773073 16551 raft_consensus.cc:1275] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:40.774040 17271 raft_consensus.cc:1240] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 8. Current term is 9. Ops: []
W20260709 12:23:40.775238 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:40.775338 16550 raft_consensus.cc:1275] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:23:40.777949 17270 raft_consensus.cc:1240] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:40.766696 16547 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:40.781767 20173 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:40.783270 16514 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:40.783802 20225 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:40.792992 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:40.797408 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:40.793975 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:40.799286 17258 raft_consensus.cc:1240] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:40.793785 17271 raft_consensus.cc:1240] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:40.797989 17270 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
I20260709 12:23:40.795660 20267 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:40.801971 20528 log.cc:927] Time spent T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:23:40.804878 17255 raft_consensus.cc:2393] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 14: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:40.807020 17254 raft_consensus.cc:1240] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:40.810101 20225 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:40.816071 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:40.816845 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 154838us
I20260709 12:23:40.738205 20373 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:40.816421 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:40.741605 20477 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:40.820226 16547 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
I20260709 12:23:40.824097 16542 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:40.833707 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:40.831339 17263 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
W20260709 12:23:40.832791 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 49583.7us
I20260709 12:23:40.819268 20284 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.836639 16402 raft_consensus.cc:2393] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 13: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:40.837517 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:40.839453 17264 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:40.841063 16544 raft_consensus.cc:1275] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:23:40.818979 20475 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:40.819072 20290 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:40.825085 20437 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:40.857070 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 295180us
I20260709 12:23:40.837177 20373 raft_consensus.cc:697] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:40.869848 16441 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:40.871352 20396 raft_consensus.cc:2764] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:23:40.880273 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:40.880013 20229 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:40.842857 20477 raft_consensus.cc:697] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:40.835538 20173 consensus_queue.cc:1048] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:23:40.886106 16550 raft_consensus.cc:2393] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 10: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:40.895082 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:23:40.901723 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 181210us
I20260709 12:23:40.884198 20251 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.862835 20284 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:40.869434 20437 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:40.919714 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:40.908471 16441 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:40.922961 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 68229.6us
I20260709 12:23:40.924255 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:40.927067 17263 raft_consensus.cc:1275] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:23:40.931041 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:40.931691 17264 raft_consensus.cc:1275] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:23:40.932286 17261 raft_consensus.cc:1275] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:23:40.918653 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:40.940315 17270 raft_consensus.cc:1275] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:23:40.943689 16402 raft_consensus.cc:1275] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:23:40.944605 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:40.950126 17255 raft_consensus.cc:1275] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:23:40.955021 17254 raft_consensus.cc:1275] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:23:40.945623 17144 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:40.961359 20256 raft_consensus.cc:697] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:40.966325 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:40.967520 17263 raft_consensus.cc:2393] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 12: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:40.968483 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:40.969660 16551 raft_consensus.cc:2393] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 13: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:40.971014 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:40.982234 16551 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:40.997605 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 91904.5us
I20260709 12:23:40.998210 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:40.911543 20251 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:40.998857 16548 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:41.021246 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:41.024608 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 432315us
I20260709 12:23:41.003952 20534 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:41.008265 20229 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:40.983759 20535 consensus_queue.cc:1048] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 12:23:41.033957 20540 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:41.006754 20477 consensus_queue.cc:237] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:41.047936 20533 log.cc:927] Time spent T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 12:23:41.048059 16399 raft_consensus.cc:1275] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:41.051282 16477 raft_consensus.cc:1275] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
W20260709 12:23:41.051697 20532 log.cc:927] Time spent T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:23:41.051820 20437 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:41.051762 16477 pending_rounds.cc:85] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 5
I20260709 12:23:41.052832 16477 pending_rounds.cc:107] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:23:41.053108 16477 raft_consensus.cc:2889] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:41.056787 16548 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
I20260709 12:23:41.060952 16551 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:23:41.063656 20479 consensus_queue.cc:1048] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:41.063786 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:41.066114 16367 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:40.984174 20373 consensus_queue.cc:237] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:41.033461 20534 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:41.092918 17143 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:41.093904 17143 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:41.094547 16441 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:41.072160 20267 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:41.073138 16543 raft_consensus.cc:1240] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 12. Current term is 13. Ops: [10.5-12.6]
W20260709 12:23:41.074278 20536 log.cc:927] Time spent T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
W20260709 12:23:41.078472 20463 consensus_meta.cc:287] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.505s	user 0.009s	sys 0.001s
I20260709 12:23:41.078603 20021 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:41.060838 20368 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:41.085059 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:41.061031 20396 consensus_queue.cc:1048] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:23:41.068977 20540 raft_consensus.cc:697] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:41.153261 16442 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:41.113309 20225 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:41.152837 16547 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:41.026170 20256 consensus_queue.cc:237] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:41.155257 20539 log.cc:927] Time spent T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:23:41.156682 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:41.168052 16548 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:41.157686 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:41.169322 16474 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:41.117811 20267 raft_consensus.cc:697] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:41.097724 20534 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:41.161311 20225 raft_consensus.cc:697] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:41.189498 20225 consensus_queue.cc:237] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:41.192497 20242 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:41.197150 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:41.198549 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:41.198417 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 132389us
I20260709 12:23:41.179888 16547 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:41.182703 20267 consensus_queue.cc:237] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:41.183768 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:41.202172 16477 raft_consensus.cc:2368] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 10. Current term is 11.
I20260709 12:23:41.184739 20540 consensus_queue.cc:237] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:41.187294 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 119397us
I20260709 12:23:41.199141 16545 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
I20260709 12:23:41.206990 16548 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:23:41.209892 17271 raft_consensus.cc:2410] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
W20260709 12:23:41.210021 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 104014us
I20260709 12:23:41.153815 20478 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:41.167757 20218 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:41.217453 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:41.150729 20463 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:41.219799 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:41.230357 16515 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:41.236951 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:41.237972 17254 raft_consensus.cc:2393] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 10: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:41.239181 16549 raft_consensus.cc:1240] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
W20260709 12:23:41.240482 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:41.241242 16548 raft_consensus.cc:1240] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:23:41.235932 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:41.245060 17266 raft_consensus.cc:2393] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 13: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:41.245481 16550 raft_consensus.cc:2393] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 12: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:41.245469 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:41.168783 20368 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:41.241729 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:23:41.248236 17142 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.890s (SENT) [suppressed 41 similar messages]
W20260709 12:23:41.252810 20546 log.cc:927] Time spent T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a: Append to log took a long time: real 0.073s	user 0.001s	sys 0.001s
I20260709 12:23:41.256465 20284 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:41.256616 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:41.258956 16442 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:41.260543 20290 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:41.260097 16515 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:41.261582 16441 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:23:41.291234 17142 leader_election.cc:336] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.501s (SENT)
I20260709 12:23:41.233011 20478 raft_consensus.cc:697] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:41.308853 16514 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:41.360909 16475 raft_consensus.cc:1240] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:23:41.375432 16469 raft_consensus.cc:1275] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:41.339331 16474 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:41.370405 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:23:41.346323 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:41.322407 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:41.364167 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:41.372036 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:23:41.321427 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 61399.1us
I20260709 12:23:41.396780 16394 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:41.424414 16515 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:41.395206 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:41.315543 16441 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:41.421865 16547 raft_consensus.cc:1275] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:41.423290 16542 raft_consensus.cc:2393] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 13: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:41.316056 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:41.381433 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:41.432634 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 132565us
W20260709 12:23:41.357803 20334 consensus_meta.cc:287] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.978s	user 0.012s	sys 0.000s
I20260709 12:23:41.264994 20470 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:41.305342 20475 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:41.244571 20396 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:41.434682 16471 raft_consensus.cc:2376] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11. Re-sending same reply.
I20260709 12:23:41.325364 20290 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:41.461329 16546 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:41.353452 20463 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:41.358669 20368 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:41.431707 20320 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:41.265216 20242 consensus_queue.cc:1059] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.196s
I20260709 12:23:41.431957 20173 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:41.351999 20251 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:41.489737 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 65443.7us
I20260709 12:23:41.510195 16515 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:41.266822 20218 consensus_queue.cc:1059] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.218s
I20260709 12:23:41.307137 20535 consensus_queue.cc:1059] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.164s
I20260709 12:23:41.485088 20267 consensus_queue.cc:1048] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 12:23:41.496344 16367 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:41.489038 20158 consensus_queue.cc:1059] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 12:23:41.293156 20302 consensus_queue.cc:1048] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.193s
I20260709 12:23:41.329777 20542 consensus_queue.cc:1059] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.196s
I20260709 12:23:41.471247 20396 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:41.464115 20475 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:23:41.525000 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 227656us
I20260709 12:23:41.315673 20527 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.197s
I20260709 12:23:41.488976 20470 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:41.521648 19969 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:41.532927 20368 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:41.533344 20475 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:41.550154 20563 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:23:41.538892 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 223441us
I20260709 12:23:41.573443 16441 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:41.533730 20396 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:41.584915 20566 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:41.585770 20566 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:41.587296 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 328414us
I20260709 12:23:41.530799 20534 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:41.590710 20534 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:41.591549 20534 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:23:41.559480 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 63288.2us
I20260709 12:23:41.560214 20470 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:41.566933 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:41.599917 20540 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:41.600626 20540 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:41.539994 20021 consensus_queue.cc:1059] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
W20260709 12:23:41.613624 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 148627us
I20260709 12:23:41.598402 20534 raft_consensus.cc:697] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 9 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:41.607300 20470 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:41.622325 20566 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:41.556414 20478 consensus_queue.cc:237] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:41.672645 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 142941us
I20260709 12:23:41.674321 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:41.650377 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:41.675644 16471 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:41.678864 16514 leader_election.cc:400] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 10. Current term is 11.
I20260709 12:23:41.679167 16474 raft_consensus.cc:1275] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:23:41.676784 16477 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:41.671542 20535 consensus_queue.cc:1243] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer ef57beab0e1d40d585983873c55107e8 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:23:41.674779 20470 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:41.671223 20475 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:41.680194 20540 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:41.674507 20267 consensus_queue.cc:1048] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 12:23:41.695235 16472 raft_consensus.cc:1275] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:23:41.697649 20368 consensus_queue.cc:1048] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:41.699433 20527 consensus_queue.cc:1048] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:41.700207 20527 consensus_queue.cc:1243] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer ef57beab0e1d40d585983873c55107e8 log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:23:41.702368 16514 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:41.703341 20569 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:41.713528 16367 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:41.713660 16514 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:41.710078 20470 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:41.719202 20158 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:23:41.719686 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:41.720295 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:41.720960 16471 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:41.722136 16469 raft_consensus.cc:1275] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:23:41.722780 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:41.716542 20529 consensus_queue.cc:1048] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 12:23:41.724160 16477 raft_consensus.cc:1240] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:41.725736 20475 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:41.729254 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:41.727383 20373 consensus_queue.cc:1048] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 12:23:41.730391 20373 consensus_queue.cc:1243] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer ef57beab0e1d40d585983873c55107e8 log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:23:41.732937 16514 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:41.743503 16367 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:41.746464 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:41.747303 16542 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:41.748651 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:41.749557 16547 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:41.752285 16550 raft_consensus.cc:1275] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:23:41.752794 16544 raft_consensus.cc:1240] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:41.753165 17143 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:41.756243 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:41.756852 20567 log.cc:927] Time spent T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:23:41.757650 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:41.758239 17261 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:41.741664 20470 consensus_queue.cc:1048] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 12:23:41.759159 20470 consensus_queue.cc:1243] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer 52ab50b0237346c1a13ede947f6a63c2 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:23:41.759487 20320 raft_consensus.cc:3055] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:41.759857 20320 raft_consensus.cc:740] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:41.744253 19908 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:41.758736 16544 raft_consensus.cc:1275] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:23:41.760672 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:41.761373 17255 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:41.764602 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:41.765295 16542 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:41.743223 19969 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:41.761055 20373 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:41.728559 20368 consensus_queue.cc:1048] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:23:41.732722 20275 consensus_queue.cc:1048] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:23:41.770331 20396 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:41.772470 20477 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:41.772934 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:41.773579 16550 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:41.764015 20251 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:41.773936 16403 raft_consensus.cc:1275] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:23:41.774236 20251 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:41.776201 16550 raft_consensus.cc:1275] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:23:41.760638 20320 consensus_queue.cc:260] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:41.777254 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:41.777585 20320 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:41.776201 16398 raft_consensus.cc:1275] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:23:41.780115 20561 log.cc:927] Time spent T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.222s	user 0.000s	sys 0.000s
I20260709 12:23:41.780439 20437 consensus_queue.cc:1048] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:41.768131 20373 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:41.777657 16546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52680 (request call id 780) took 1868 ms. Trace:
I20260709 12:23:41.784691 16546 rpcz_store.cc:276] 0709 12:23:39.909257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:39.928501 (+ 19244us) service_pool.cc:224] Handling call
0709 12:23:41.777478 (+1848977us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b54008c50d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:41.777586 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3522203392}
I20260709 12:23:41.806281 16547 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
I20260709 12:23:41.781589 20373 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:41.812824 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 49590.6us
I20260709 12:23:41.814577 16548 raft_consensus.cc:1275] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:23:41.830786 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 48649.5us
I20260709 12:23:41.794888 20477 raft_consensus.cc:697] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:23:41.833724 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 93084.1us
I20260709 12:23:41.832970 17143 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:41.718366 20534 consensus_queue.cc:237] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:41.839313 16514 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:41.838164 20396 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:41.839990 17143 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:41.838658 20208 consensus_queue.cc:1048] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 12:23:41.851277 20576 consensus_queue.cc:1048] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:23:41.851696 16441 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:41.847868 17266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45228 (request call id 1091) took 1221 ms. Trace:
I20260709 12:23:41.852733 20576 consensus_queue.cc:1048] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:41.852954 17266 rpcz_store.cc:276] 0709 12:23:40.626042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:40.626201 (+   159us) service_pool.cc:224] Handling call
0709 12:23:41.847614 (+1221413us) spinlock_profiling.cc:243] Waited 599 ms on lock 0x7b540063ea54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:41.847769 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1376835840}
I20260709 12:23:41.854040 20573 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:41.854611 20573 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:41.854763 16441 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:41.855803 20542 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:41.856544 20542 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:41.856972 20542 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:41.857239 20554 log.cc:927] Time spent T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.524s	user 0.000s	sys 0.004s
I20260709 12:23:41.857316 20542 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:41.857713 17255 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
I20260709 12:23:41.851578 20367 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:41.858871 20367 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:41.859565 20563 raft_consensus.cc:3055] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:41.859982 20563 raft_consensus.cc:740] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:41.861232 20367 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:41.862075 16441 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:41.862107 20367 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:41.865011 20463 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:41.864897 20367 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:41.848755 20477 consensus_queue.cc:237] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:41.865640 20463 raft_consensus.cc:697] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:41.866673 20463 consensus_queue.cc:237] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:41.840958 20540 consensus_queue.cc:1048] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 2, Time since last communication: 0.009s
W20260709 12:23:41.865497 20557 log.cc:927] Time spent T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.540s	user 0.000s	sys 0.004s
I20260709 12:23:41.829898 20225 consensus_queue.cc:1048] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 12:23:41.837432 20251 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:41.836371 17261 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 17.
I20260709 12:23:41.874967 16441 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:41.868903 20562 log.cc:927] Time spent T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.321s	user 0.002s	sys 0.000s
I20260709 12:23:41.880672 20302 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:41.881395 20302 raft_consensus.cc:697] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:41.869522 17143 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:41.887522 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:41.888572 17261 raft_consensus.cc:2410] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:23:41.896245 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:41.897734 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:41.898506 17266 raft_consensus.cc:2376] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12. Re-sending same reply.
I20260709 12:23:41.903847 16542 raft_consensus.cc:1275] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:23:41.906026 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:41.906889 17255 raft_consensus.cc:2376] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9. Re-sending same reply.
I20260709 12:23:41.908347 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:41.901299 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:41.917277 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:41.917140 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:41.918771 17266 raft_consensus.cc:2376] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14. Re-sending same reply.
W20260709 12:23:41.919546 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:41.920534 17266 raft_consensus.cc:1240] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:23:41.925159 16546 raft_consensus.cc:2393] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 10: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:41.926389 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:41.927171 17269 raft_consensus.cc:2376] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12. Re-sending same reply.
I20260709 12:23:41.872872 20396 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:23:41.945343 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 70466us
I20260709 12:23:41.946686 16441 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:41.947800 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:41.951042 17254 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
W20260709 12:23:41.973759 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 137422us
I20260709 12:23:41.974875 16441 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:23:41.992194 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 155120us
I20260709 12:23:41.925312 20576 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:41.998337 17269 pending_rounds.cc:85] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8: Aborting all ops after (but not including) 4
I20260709 12:23:41.998766 17269 pending_rounds.cc:107] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 12:23:41.999044 17269 raft_consensus.cc:2889] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:42.004122 16515 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:41.957062 20573 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:42.013934 19908 raft_consensus.cc:697] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:42.006374 20208 raft_consensus.cc:2804] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:42.029400 20576 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.016242 20566 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:42.051813 16473 raft_consensus.cc:1275] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:23:42.018524 20334 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:42.030016 20576 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:42.070449 16473 raft_consensus.cc:1217] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Deduplicated request from leader. Original: 7.5->[9.6-12.8]   Dedup: 9.6->[10.7-12.8]
I20260709 12:23:42.071120 16473 pending_rounds.cc:85] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 6
I20260709 12:23:42.071426 16473 pending_rounds.cc:107] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:23:42.071695 16473 raft_consensus.cc:2889] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:42.012593 20535 consensus_queue.cc:1059] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:23:42.049853 20251 consensus_queue.cc:1048] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.034s
W20260709 12:23:42.084786 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 110018us
I20260709 12:23:42.085515 16549 raft_consensus.cc:1275] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 12:23:42.086054 16549 pending_rounds.cc:85] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1: Aborting all ops after (but not including) 8
I20260709 12:23:42.086326 16549 pending_rounds.cc:107] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 12:23:42.086592 16549 raft_consensus.cc:2889] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:23:42.094326 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 65451.8us
W20260709 12:23:42.129504 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 114353us
I20260709 12:23:42.083339 20479 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:42.018080 20302 consensus_queue.cc:237] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:42.164137 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 160151us
I20260709 12:23:42.165768 16515 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:23:42.180811 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 94762.5us
I20260709 12:23:42.145473 20334 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.146081 20559 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:23:42.204360 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 71820.2us
I20260709 12:23:42.158957 20173 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:42.126127 20534 consensus_queue.cc:1048] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
W20260709 12:23:42.282078 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 53799.5us
W20260709 12:23:42.288919 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:42.305039 20591 log.cc:927] Time spent T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:23:42.232501 20173 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:42.307683 20535 consensus_queue.cc:1059] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:23:42.309006 20579 log.cc:927] Time spent T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.253s	user 0.001s	sys 0.000s
W20260709 12:23:42.309734 20595 log.cc:927] Time spent T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:23:42.210124 20572 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:42.162034 19908 consensus_queue.cc:237] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:42.238014 20367 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:42.221473 20334 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:42.334950 20173 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:42.227568 20559 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:42.353514 20477 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.354424 20572 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:42.353570 20463 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.231942 20571 consensus_queue.cc:1059] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.395s
I20260709 12:23:42.431166 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:42.432523 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 219730us
I20260709 12:23:42.432971 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:42.437559 16398 raft_consensus.cc:1240] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 12. Current term is 13. Ops: []
W20260709 12:23:42.445335 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:42.446609 16399 raft_consensus.cc:1240] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:42.443003 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:42.448200 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:42.460201 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:42.461017 16542 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
W20260709 12:23:42.438716 20593 log.cc:927] Time spent T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.231s	user 0.001s	sys 0.000s
I20260709 12:23:42.472213 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:42.487952 16394 raft_consensus.cc:1275] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:23:42.463063 16472 raft_consensus.cc:2393] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 10: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:42.431253 20173 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:42.477763 20320 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:42.440994 20477 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:42.430444 20572 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:42.507299 20615 log.cc:927] Time spent T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:23:42.510641 17255 pending_rounds.cc:85] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8: Aborting all ops after (but not including) 8
I20260709 12:23:42.545879 17255 pending_rounds.cc:107] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:23:42.546232 17255 raft_consensus.cc:2889] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:42.512632 20242 raft_consensus.cc:3055] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:23:42.513686 20585 log.cc:927] Time spent T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.242s	user 0.000s	sys 0.001s
I20260709 12:23:42.513974 20534 raft_consensus.cc:3055] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:42.514308 20600 consensus_queue.cc:1048] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.353s
I20260709 12:23:42.518893 20576 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:42.442240 20463 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:42.545315 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 81885.5us
I20260709 12:23:42.597086 16472 raft_consensus.cc:1240] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 12. Current term is 13. Ops: []
W20260709 12:23:42.570187 17264 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.001s	sys 0.011s
W20260709 12:23:42.524247 20373 consensus_meta.cc:287] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.690s	user 0.007s	sys 0.003s
W20260709 12:23:42.603565 17264 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.002s	sys 0.011s
I20260709 12:23:42.525498 20470 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.526124 20320 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:42.593801 20630 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:42.549403 20242 raft_consensus.cc:740] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:42.525110 20158 consensus_queue.cc:1048] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.089s
W20260709 12:23:42.641633 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 140860us
I20260709 12:23:42.579048 20534 raft_consensus.cc:740] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:23:42.648662 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 464708us
I20260709 12:23:42.524681 20173 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:42.661036 16547 raft_consensus.cc:1240] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
W20260709 12:23:42.641268 20620 log.cc:927] Time spent T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:23:42.637527 20218 raft_consensus.cc:3055] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:42.629595 20463 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:42.684834 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:42.709213 20612 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:42.690307 20603 consensus_queue.cc:1059] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 12:23:42.709973 20612 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:42.713155 20606 log.cc:927] Time spent T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 12:23:42.714509 16441 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:42.715847 16441 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:42.717631 20631 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:42.718454 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:42.721035 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:42.746745 17269 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
I20260709 12:23:42.721601 20559 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:42.732197 20612 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:42.733026 20534 consensus_queue.cc:260] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:42.651587 20630 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:42.734714 20334 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:42.622984 20470 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:42.651170 20320 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:42.674916 20218 raft_consensus.cc:740] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:42.614720 20373 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:42.752205 16402 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
W20260709 12:23:42.744455 20275 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.015s	sys 0.046s
I20260709 12:23:42.763690 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:42.684473 20478 consensus_queue.cc:1048] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 12:23:42.758653 20631 raft_consensus.cc:697] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:42.776594 20373 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:42.778654 20320 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:42.778880 20633 raft_consensus.cc:3055] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:42.781026 20470 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:42.781265 20208 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:42.822973 17258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 1066) took 1071 ms. Trace:
I20260709 12:23:42.823293 20208 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:42.823442 17258 rpcz_store.cc:276] 0709 12:23:41.751608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:41.755434 (+  3826us) service_pool.cc:224] Handling call
0709 12:23:41.755715 (+   281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:41.780826 (+ 25111us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:41.780863 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:41.781330 (+   467us) log.cc:844] Serialized 26 byte log entry
0709 12:23:42.058168 (+276838us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:42.058208 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:42.058292 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:42.822690 (+764398us) raft_consensus.cc:1685] finished
0709 12:23:42.822719 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:42.822886 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24728,"raft.queue_time_us":24834,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"thread_start_us":276485,"threads_started":1,"wal-append.queue_time_us":1006107}
I20260709 12:23:42.826090 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:42.827168 17258 raft_consensus.cc:2393] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 10: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:42.783785 20617 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.784714 16400 raft_consensus.cc:2393] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 12: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:42.779701 20218 consensus_queue.cc:260] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:42.828593 20437 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.829877 20218 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:42.830080 20437 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:42.838642 20576 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.839241 20576 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:42.838717 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:42.841039 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:42.842244 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:42.843302 20475 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.843972 20475 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:42.845897 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:42.840513 20613 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.847324 20613 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:42.850172 20613 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:42.848176 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:42.846710 16542 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
I20260709 12:23:42.853406 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:42.854192 16542 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:42.855299 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:42.855540 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:42.850101 20576 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:42.860172 20572 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:42.861124 16548 raft_consensus.cc:1275] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:23:42.786407 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:42.786624 20534 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:42.788906 20477 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:42.790028 20568 raft_consensus.cc:3055] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:42.790262 16367 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:42.754822 20600 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:23:42.759440 20242 consensus_queue.cc:260] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:42.804709 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:42.875192 20572 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:42.875432 20600 raft_consensus.cc:697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 17 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:42.876309 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:42.806609 20479 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:42.809947 16472 raft_consensus.cc:1275] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:42.810619 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:42.810681 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:42.811616 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:42.813969 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:23:42.873530 20542 consensus_meta.cc:287] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 1.015s	user 0.014s	sys 0.000s
I20260709 12:23:42.815961 17269 raft_consensus.cc:1240] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
W20260709 12:23:42.774394 20275 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.016s	sys 0.047s
I20260709 12:23:42.820595 17263 raft_consensus.cc:1240] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:23:42.917510 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:42.865970 17258 raft_consensus.cc:1240] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
W20260709 12:23:42.866642 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 44232.9us
I20260709 12:23:42.867086 17271 raft_consensus.cc:1240] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:23:42.868489 16542 raft_consensus.cc:2410] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 14 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:23:42.790069 20630 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:42.869781 16545 raft_consensus.cc:2410] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:23:42.874534 20437 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:42.926270 16477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1348) took 1198 ms. Trace:
I20260709 12:23:42.821250 20633 raft_consensus.cc:740] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:42.926779 16477 rpcz_store.cc:276] 0709 12:23:41.719494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:41.719608 (+   114us) service_pool.cc:224] Handling call
0709 12:23:41.719888 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:41.724582 (+  4694us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:42.917525 (+1192943us) spinlock_profiling.cc:243] Waited 4.08 ms on lock 0x7b54006c7ad4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:42.917713 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9388416}
W20260709 12:23:42.896260 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:42.898161 16544 raft_consensus.cc:1240] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:23:42.864980 20475 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:42.814723 17261 raft_consensus.cc:1240] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:42.898546 20373 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:42.899853 17254 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:42.904872 16474 raft_consensus.cc:2393] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 12: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:42.904125 16515 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:42.939940 16545 raft_consensus.cc:1275] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 12:23:42.903406 17142 leader_election.cc:336] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.536s (SENT)
I20260709 12:23:42.925276 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:42.871799 20568 raft_consensus.cc:740] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:42.894444 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:42.932598 16544 raft_consensus.cc:1275] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:23:42.934541 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:42.924103 17143 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:42.942061 17142 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:42.900281 20479 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:42.947913 16515 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:42.950331 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:42.953689 20229 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:42.959973 20229 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:42.960445 20229 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:42.960216 20568 consensus_queue.cc:260] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:42.961328 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:42.962517 16477 raft_consensus.cc:2376] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10. Re-sending same reply.
I20260709 12:23:42.962728 20475 consensus_queue.cc:1059] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:42.963656 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:42.964413 20527 consensus_queue.cc:1048] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:23:42.964798 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 174637us
I20260709 12:23:42.922962 20542 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:42.965572 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:42.966368 16472 raft_consensus.cc:2393] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 16: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:42.966687 20633 consensus_queue.cc:260] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:42.967365 20542 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:42.967998 20633 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:42.957579 16545 raft_consensus.cc:1275] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:23:42.968034 20566 consensus_meta.cc:287] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.911s	user 0.013s	sys 0.001s
I20260709 12:23:42.969447 20566 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:42.973951 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:42.929996 20367 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:42.975766 20367 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:42.977138 20636 log.cc:927] Time spent T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:23:42.969501 16476 raft_consensus.cc:2393] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 12: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:42.876523 20600 consensus_queue.cc:237] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:42.982221 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:42.869689 20617 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:42.900552 20477 consensus_queue.cc:1048] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 12:23:42.899816 20573 consensus_queue.cc:1059] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:23:42.962006 20568 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:42.959625 20613 consensus_queue.cc:1059] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:42.987972 16471 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:23:42.959503 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:42.992583 17261 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
I20260709 12:23:42.962534 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:42.991627 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:42.937177 20559 consensus_queue.cc:1048] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 12:23:42.998435 20613 consensus_queue.cc:1243] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer ef57beab0e1d40d585983873c55107e8 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:23:42.999174 20613 raft_consensus.cc:3055] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:42.999559 20613 raft_consensus.cc:740] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:23:43.000754 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 97401.7us
I20260709 12:23:43.000592 20613 consensus_queue.cc:260] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:43.001431 17142 leader_election.cc:336] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.691s (SENT)
I20260709 12:23:42.959424 20627 consensus_queue.cc:1048] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
W20260709 12:23:43.002862 17142 leader_election.cc:336] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.827s (SENT)
I20260709 12:23:43.003258 17142 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:43.004048 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:43.004824 16542 raft_consensus.cc:2393] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 12: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:43.004680 20563 consensus_queue.cc:260] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:43.005910 20463 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:43.006141 20563 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:42.997933 20626 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:43.013588 17270 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.905s	user 0.160s	sys 0.164s
I20260709 12:23:43.009284 20477 consensus_queue.cc:1048] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:23:43.031742 17270 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.923s	user 0.160s	sys 0.172s
I20260709 12:23:42.986589 20631 consensus_queue.cc:237] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:43.027946 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 61837.8us
I20260709 12:23:43.029315 16441 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:43.033619 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:43.008554 20540 consensus_queue.cc:1048] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:23:43.036567 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 47203.1us
I20260709 12:23:43.041491 20535 raft_consensus.cc:2804] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:43.038899 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:43.042686 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:43.043361 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:43.040120 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:43.013656 20437 consensus_queue.cc:1059] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:23:43.046151 16472 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
I20260709 12:23:42.955354 20630 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:23:43.048976 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:43.050365 16475 raft_consensus.cc:2393] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 10: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:43.052139 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:43.054165 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:43.054430 16472 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:23:43.055262 16474 raft_consensus.cc:2410] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 13 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:23:43.037928 20559 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:43.063259 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:43.048375 20535 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:43.065166 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:43.066671 16550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 803) took 1169 ms. Trace:
I20260709 12:23:43.041671 20284 consensus_queue.cc:1048] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 12:23:43.051045 20630 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:43.039050 20275 consensus_queue.cc:1059] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:23:43.140395 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:43.137876 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:43.143216 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:43.090747 16367 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:23:43.106674 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:43.111441 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 71254.1us
I20260709 12:23:43.164647 16366 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
W20260709 12:23:43.115545 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 112691us
W20260709 12:23:43.169479 17142 leader_election.cc:336] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.918s (SENT)
I20260709 12:23:43.116412 16514 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
W20260709 12:23:43.172355 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 56019.1us
I20260709 12:23:43.172148 16477 raft_consensus.cc:2410] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:23:43.172258 17142 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:43.040957 20613 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:43.173568 16514 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:43.179311 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:43.118894 20367 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:43.092875 20535 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:43.192492 16468 raft_consensus.cc:2410] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:23:43.198897 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:43.199747 16476 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:43.062029 20208 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:43.145996 20617 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:43.173415 20630 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:43.086689 16550 rpcz_store.cc:276] 0709 12:23:41.897283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:41.897452 (+   169us) service_pool.cc:224] Handling call
0709 12:23:41.925708 (+ 28256us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:23:41.957523 (+ 31815us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:41.957560 (+    37us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:23:42.009409 (+ 51849us) log.cc:844] Serialized 78 byte log entry
0709 12:23:42.190843 (+181434us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:42.330223 (+139380us) log.cc:844] Serialized 14 byte log entry
0709 12:23:42.372989 (+ 42766us) log.cc:844] Serialized 14 byte log entry
0709 12:23:42.858680 (+485691us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b18003fc820. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d99455 00007f2875e81060 00007f2875e93716 00007f2875ea4287 00007f2875ea4208
0709 12:23:42.979052 (+120372us) spinlock_profiling.cc:243] Waited 4.47 ms on lock 0x7b18003fc820. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c522 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:23:43.066419 (+ 87367us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:43.066472 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:43.066502 (+    30us) raft_consensus.cc:1685] finished
0709 12:23:43.066510 (+     8us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:43.066610 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6732,"prepare.queue_time_us":113465,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":89,"raft.queue_time_us":353862,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":27686,"spinlock_wait_cycles":35628160,"thread_start_us":416868,"threads_started":3,"wal-append.queue_time_us":502057}
I20260709 12:23:43.189168 20158 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:43.188817 20612 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:43.189898 19908 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:43.212019 16366 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:43.218700 20367 consensus_queue.cc:1048] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:23:43.220017 16470 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:43.226334 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:23:43.228596 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 137908us
I20260709 12:23:43.243870 17263 raft_consensus.cc:1240] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:43.200295 20275 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:43.239647 20229 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:43.208325 20621 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:43.214375 20158 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:43.282436 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 70528us
I20260709 12:23:43.279542 20645 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.292691 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:43.293219 20638 log.cc:927] Time spent T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 12:23:43.293699 16477 raft_consensus.cc:2410] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:23:43.241202 20612 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:43.294884 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:43.295373 20612 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:43.300577 20208 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:43.300853 16475 raft_consensus.cc:2410] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:23:43.274730 20284 consensus_queue.cc:1059] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 12:23:43.303582 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:43.294960 20320 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:43.305316 16475 raft_consensus.cc:2393] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 12: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:43.304004 20368 raft_consensus.cc:3055] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:43.317533 17270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 1115) took 1401 ms. Trace:
I20260709 12:23:43.317991 17270 rpcz_store.cc:276] 0709 12:23:41.950303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:41.957323 (+  7020us) service_pool.cc:224] Handling call
0709 12:23:42.006289 (+ 48966us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:42.024223 (+ 17934us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:42.024271 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:42.051970 (+ 27699us) log.cc:844] Serialized 52 byte log entry
0709 12:23:43.032154 (+980184us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:43.046245 (+ 14091us) log.cc:844] Serialized 14 byte log entry
0709 12:23:43.051429 (+  5184us) log.cc:844] Serialized 14 byte log entry
0709 12:23:43.051667 (+   238us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:43.051768 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:43.298871 (+247103us) spinlock_profiling.cc:243] Waited 1.78 ms on lock 0x7b180042b440. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:23:43.317292 (+ 18421us) raft_consensus.cc:1685] finished
0709 12:23:43.317322 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:43.317474 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33597,"prepare.queue_time_us":13903,"prepare.run_wall_time_us":33,"raft.queue_time_us":8494,"raft.run_cpu_time_us":408,"raft.run_wall_time_us":9854,"spinlock_wait_cycles":7168640,"thread_start_us":923717,"threads_started":1,"wal-append.queue_time_us":1096197}
I20260709 12:23:43.319568 20626 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:43.307538 20158 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:43.324473 20612 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:43.293238 20645 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:43.327982 20612 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:43.328488 20645 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:43.329739 20320 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:43.333873 20367 consensus_queue.cc:1048] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:43.336169 16546 raft_consensus.cc:1275] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:23:43.336524 20373 consensus_queue.cc:1059] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:43.337211 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:43.337803 16546 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:43.341224 20437 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:43.341837 20470 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:43.341782 20437 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:43.342798 20470 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:43.344525 20284 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:43.345158 20587 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:43.345435 20284 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:43.313910 17254 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:23:43.348641 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:43.345767 20587 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:43.349303 20470 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:43.307641 16476 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:23:43.349408 20527 raft_consensus.cc:3055] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:23:43.357609 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:43.359264 16476 raft_consensus.cc:2410] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:23:43.277892 20275 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:43.365734 16399 raft_consensus.cc:1275] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:23:43.354763 20527 raft_consensus.cc:740] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:43.379295 20587 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:43.379926 20527 consensus_queue.cc:260] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:43.380518 20630 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:43.381317 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:43.381703 20527 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:43.386348 20630 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:43.392931 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 223478us
I20260709 12:23:43.351909 20617 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:43.415815 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:43.336215 20367 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.316797 20368 raft_consensus.cc:740] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:43.319959 20479 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:43.413839 16546 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:23:43.401661 17255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 1119) took 1432 ms. Trace:
I20260709 12:23:43.418350 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:43.418874 17143 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:43.419353 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:43.418073 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:43.418051 17255 rpcz_store.cc:276] 0709 12:23:41.968921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:42.510047 (+541126us) service_pool.cc:224] Handling call
0709 12:23:42.510424 (+   377us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:42.587532 (+ 77108us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:42.587562 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:42.971406 (+383844us) spinlock_profiling.cc:243] Waited 7.71 ms on lock 0x7b18003bc860. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d6a2a6 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a
0709 12:23:42.971878 (+   472us) spinlock_profiling.cc:243] Waited 941 us on lock 0x7b2c00000708. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:23:42.985070 (+ 13192us) log.cc:844] Serialized 52 byte log entry
0709 12:23:43.209436 (+224366us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:43.225251 (+ 15815us) log.cc:844] Serialized 14 byte log entry
0709 12:23:43.225502 (+   251us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:43.225625 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:43.401376 (+175751us) raft_consensus.cc:1685] finished
0709 12:23:43.401407 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:43.401600 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26116,"prepare.queue_time_us":15603,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":28356,"raft.run_cpu_time_us":286,"raft.run_wall_time_us":20268,"spinlock_wait_cycles":19887232,"thread_start_us":223932,"threads_started":1,"wal-append.queue_time_us":413808}
I20260709 12:23:43.394294 20275 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:43.403610 20284 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:43.413815 20630 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.446533 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:43.451268 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:43.460948 17261 raft_consensus.cc:1240] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:43.460325 16476 raft_consensus.cc:2376] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12. Re-sending same reply.
W20260709 12:23:43.465255 17142 leader_election.cc:336] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.600s (SENT)
I20260709 12:23:43.470162 16395 raft_consensus.cc:1275] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:23:43.423465 20621 raft_consensus.cc:2804] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:43.473031 16549 raft_consensus.cc:2410] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:23:43.423771 20437 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:43.457794 20627 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:43.485087 20627 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:43.487144 20627 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:43.483218 17142 leader_election.cc:336] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.788s (SENT)
I20260709 12:23:43.488101 17142 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:43.463024 19969 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:43.458904 20540 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:43.490581 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:43.494199 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:43.494674 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:43.420650 20367 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:43.498881 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:43.517700 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:43.471453 20630 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:43.482583 20621 raft_consensus.cc:697] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 10 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:43.520864 16477 raft_consensus.cc:2410] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:23:43.482038 20368 consensus_queue.cc:260] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:43.523504 16400 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:23:43.495776 20627 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:43.524448 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:43.527122 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:43.467697 17255 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159"
dest_uuid: "ef57beab0e1d40d585983873c55107e8"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:45120
I20260709 12:23:43.535414 17255 raft_consensus.cc:606] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:43.535897 17255 raft_consensus.cc:614] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:43.536293 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:43.544646 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:43.545876 20320 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:43.531505 20251 consensus_queue.cc:1048] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 12:23:43.548731 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:43.552971 16399 raft_consensus.cc:1240] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:43.552752 20621 consensus_queue.cc:237] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:43.553043 20158 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:43.554131 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:43.555308 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:43.556239 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:43.556649 20368 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:43.497148 19969 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:43.559181 19969 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:43.561961 16516 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:43.563535 16516 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:43.565083 20302 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:43.497529 20540 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:43.569219 20158 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:43.565075 20470 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:23:43.574096 20649 log.cc:927] Time spent T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 12:23:43.589131 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:43.589918 16545 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
W20260709 12:23:43.575026 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 91829.8us
W20260709 12:23:43.593228 17142 leader_election.cc:336] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 2.099s (SENT)
I20260709 12:23:43.593724 17142 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:43.575639 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:43.576347 20367 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:43.582409 20275 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:43.583537 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:43.595955 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:43.596657 16472 raft_consensus.cc:2393] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 12: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:43.585790 16516 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:43.594878 16477 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:23:43.596751 16545 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:43.591926 17143 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:43.601706 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:43.605377 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:43.608074 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 312238us
I20260709 12:23:43.600312 20559 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:43.599996 20470 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:43.621127 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:43.636456 16367 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:43.603842 20621 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:43.601202 20367 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.639142 16367 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:43.640264 20633 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:43.646674 20633 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:43.640455 20367 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:43.651520 20367 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:43.643181 20535 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:43.655486 20535 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:43.643560 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 50365.1us
W20260709 12:23:43.657604 17142 leader_election.cc:336] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 2.186s (SENT)
I20260709 12:23:43.658114 17142 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:43.660612 20478 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:43.638556 20621 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:43.661966 20621 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:43.659683 17142 leader_election.cc:336] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 2.190s (SENT)
I20260709 12:23:43.665141 17142 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:43.667196 20633 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:43.672786 20367 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:43.673034 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:43.677045 20367 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:43.677742 20478 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:43.681975 20367 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:43.679349 17142 leader_election.cc:336] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 2.206s (SENT)
I20260709 12:23:43.685065 17142 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
W20260709 12:23:43.637949 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 100461us
I20260709 12:23:43.692106 20478 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:43.694502 17144 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:43.697585 17144 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:43.700217 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 133127us
W20260709 12:23:43.695931 17142 leader_election.cc:336] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 2.230s (SENT)
I20260709 12:23:43.702301 17142 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:43.706719 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:43.636132 19969 consensus_queue.cc:1059] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 12:23:43.724702 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:43.725077 19969 consensus_queue.cc:1243] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer f99066d13e524238aec6aa2024a8bd50 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:23:43.725688 19969 raft_consensus.cc:3055] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:43.725965 16547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1285) took 1060 ms. Trace:
I20260709 12:23:43.726104 19969 raft_consensus.cc:740] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:43.727209 19969 consensus_queue.cc:260] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:43.729079 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:43.636839 20158 consensus_queue.cc:1048] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 12:23:43.725778 16477 raft_consensus.cc:2410] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:23:43.726472 16547 rpcz_store.cc:276] 0709 12:23:42.665320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:42.666133 (+   813us) service_pool.cc:224] Handling call
0709 12:23:42.666561 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:43.056005 (+389444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:43.056051 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:43.059220 (+  3169us) log.cc:844] Serialized 26 byte log entry
0709 12:23:43.456841 (+397621us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:43.456906 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:43.456997 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:43.725717 (+268720us) raft_consensus.cc:1685] finished
0709 12:23:43.725737 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:43.725900 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":66733,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":93,"thread_start_us":678921,"threads_started":2,"wal-append.queue_time_us":526060}
I20260709 12:23:43.738130 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:43.739090 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:43.717258 20559 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:43.739754 16477 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:23:43.743381 16543 raft_consensus.cc:1240] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:23:43.747704 16477 raft_consensus.cc:1275] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:43.732931 20470 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.753058 16477 raft_consensus.cc:1275] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:23:43.747651 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 108588us
I20260709 12:23:43.739792 20251 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:43.737808 20663 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:43.743181 20173 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:43.770704 16473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1413) took 1174 ms. Trace:
I20260709 12:23:43.765878 16367 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:43.771140 16473 rpcz_store.cc:276] 0709 12:23:42.596054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:42.596224 (+   170us) service_pool.cc:224] Handling call
0709 12:23:42.622280 (+ 26056us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:42.953316 (+331036us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:42.953376 (+    60us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:42.953453 (+    77us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:42.953499 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:43.770456 (+816957us) spinlock_profiling.cc:243] Waited 331 ms on lock 0x7b54002729d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:43.770527 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:43.770651 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":760648576}
I20260709 12:23:43.754818 20559 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:43.783624 20669 consensus_queue.cc:1048] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:23:43.784433 20654 log.cc:927] Time spent T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:23:43.784582 20669 consensus_queue.cc:1243] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer 610f4feda11942739236f2b121a4b9c1 log is divergent from this leader: its last log entry 12.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:23:43.786094 20669 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:23:43.797031 20655 log.cc:927] Time spent T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:23:43.802399 20218 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:43.758935 20470 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:43.789266 20173 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.791224 20630 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:43.798434 20669 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:43.813376 16545 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
I20260709 12:23:43.800040 20566 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:43.786966 20368 consensus_queue.cc:1048] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:23:43.817654 20669 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:43.821161 20613 consensus_meta.cc:287] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.630s	user 0.012s	sys 0.001s
I20260709 12:23:43.806313 20275 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:43.889331 16470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34840 (request call id 1357) took 1079 ms. Trace:
I20260709 12:23:43.889871 16475 raft_consensus.cc:2410] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:23:43.788995 20559 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:43.826419 20470 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:43.827777 20242 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:43.885800 16442 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:43.886004 20396 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:43.886473 20576 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.886960 20600 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:43.887449 20622 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:43.887938 20568 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:43.888598 20570 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.889770 16470 rpcz_store.cc:276] 0709 12:23:42.809405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:42.810415 (+  1010us) service_pool.cc:224] Handling call
0709 12:23:43.889127 (+1078712us) spinlock_profiling.cc:243] Waited 321 ms on lock 0x7b54002e5c54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:43.889246 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":737585920}
I20260709 12:23:43.902530 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:43.900547 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:43.805979 20218 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:43.903174 16545 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:43.903309 17258 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:43.897787 20396 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:43.910351 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 144551us
I20260709 12:23:43.820125 20173 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:43.909526 20650 log.cc:927] Time spent T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 12:23:43.890458 20566 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:43.901844 20535 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:43.966717 16367 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:43.955415 20622 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:43.959865 20631 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:23:43.967680 20662 log.cc:927] Time spent T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 12:23:43.908931 20600 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:43.957191 20568 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:43.983139 20367 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:43.902180 20576 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:43.959029 20621 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:43.985857 20672 log.cc:927] Time spent T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
W20260709 12:23:43.992842 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:43.995105 16545 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
I20260709 12:23:43.966915 20570 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:43.997499 17258 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
W20260709 12:23:43.999985 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:44.009841 20396 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:44.010181 20173 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:44.010727 20660 log.cc:927] Time spent T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.222s	user 0.001s	sys 0.001s
I20260709 12:23:44.011585 17261 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 1076) took 1154 ms. Trace:
I20260709 12:23:44.013062 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:44.013794 16473 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:44.017796 20670 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:44.012099 17261 rpcz_store.cc:276] 0709 12:23:42.857231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:43.031400 (+174169us) service_pool.cc:224] Handling call
0709 12:23:43.031757 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:43.461542 (+429785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:44.011323 (+549781us) spinlock_profiling.cc:243] Waited 429 ms on lock 0x7b54005f3554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:44.011515 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":986446080}
I20260709 12:23:44.023348 20621 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:44.027596 16473 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
I20260709 12:23:43.983342 20535 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:44.038156 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 71531.6us
I20260709 12:23:44.007344 20667 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:23:44.066165 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 370233us
W20260709 12:23:44.067193 17142 leader_election.cc:336] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 2.276s (SENT)
I20260709 12:23:44.067685 17142 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:44.072293 20302 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:44.018553 20670 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:44.088550 20676 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:44.125895 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:44.133158 20570 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:44.117672 20396 consensus_queue.cc:1048] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:23:44.133177 20568 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:44.135661 17261 raft_consensus.cc:1275] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:23:44.142760 20535 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:44.136874 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:44.139837 16542 raft_consensus.cc:1240] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:44.142124 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:44.145648 20566 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:44.150555 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:44.151723 17269 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:44.153420 20622 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:44.152590 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:44.155874 20540 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:44.161139 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 93985.2us
I20260709 12:23:44.126720 20559 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:44.120675 20367 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:44.114284 20302 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:44.365374 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:44.367498 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:44.227414 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:44.374192 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 195340us
I20260709 12:23:44.384486 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:44.385304 16475 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:44.386550 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:44.388026 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:44.401895 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:44.405390 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:44.388705 16542 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:44.408457 16471 raft_consensus.cc:1275] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 12:23:44.408876 16471 pending_rounds.cc:85] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 7
I20260709 12:23:44.409116 16471 pending_rounds.cc:107] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:23:44.409332 16473 raft_consensus.cc:1275] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:23:44.411288 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:44.411469 16469 raft_consensus.cc:2410] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:23:44.409374 16471 raft_consensus.cc:2889] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:23:44.407219 17142 leader_election.cc:336] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 2.653s (SENT)
I20260709 12:23:44.412778 17142 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:44.403729 16468 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53186 (request call id 753) took 1564 ms (client timeout 1501 ms). Trace:
I20260709 12:23:44.410392 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:44.418221 17261 raft_consensus.cc:1240] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:44.387059 16475 raft_consensus.cc:2376] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12. Re-sending same reply.
I20260709 12:23:44.420171 16474 raft_consensus.cc:1275] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 12:23:44.420706 16474 pending_rounds.cc:85] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 7
I20260709 12:23:44.420982 16474 pending_rounds.cc:107] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:23:44.421279 16474 raft_consensus.cc:2889] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:23:44.416473 16468 rpcz_store.cc:269] 0709 12:23:42.839239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:42.841929 (+  2690us) service_pool.cc:224] Handling call
0709 12:23:44.373084 (+1531155us) spinlock_profiling.cc:243] Waited 349 ms on lock 0x7b54004eff54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:44.403654 (+ 30570us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":802517248}
I20260709 12:23:44.427892 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:44.432180 16470 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
I20260709 12:23:44.433667 16542 raft_consensus.cc:1240] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 13. Current term is 14. Ops: [11.6-13.7]
I20260709 12:23:44.439589 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:44.440495 17258 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:44.423920 20559 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:44.446748 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:44.447250 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:44.447953 16473 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:44.450726 20302 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:44.443656 20242 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:44.461421 20242 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:44.464537 20535 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:44.465268 20535 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:44.464635 20670 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:44.473853 20669 consensus_meta.cc:287] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.584s	user 0.009s	sys 0.000s
I20260709 12:23:44.474599 20669 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:44.475224 20535 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:44.479024 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:44.480494 20568 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:44.481081 20568 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:44.483457 20568 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:44.486555 20396 consensus_queue.cc:1048] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:44.487751 20572 consensus_queue.cc:1048] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:44.490939 16469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53186 (request call id 756) took 1650 ms. Trace:
I20260709 12:23:44.491225 16476 raft_consensus.cc:1275] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:23:44.491797 16476 pending_rounds.cc:85] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 8
I20260709 12:23:44.492410 16476 pending_rounds.cc:107] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:23:44.492709 16476 raft_consensus.cc:2889] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:44.489967 20535 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:44.499571 20535 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:44.514186 20535 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:44.515463 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:44.516326 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:44.516274 17258 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
I20260709 12:23:44.519083 17266 raft_consensus.cc:1240] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:44.498533 16441 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:44.523773 20622 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:44.524760 20622 consensus_queue.cc:1048] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:44.528015 16470 raft_consensus.cc:2410] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:23:44.528947 20242 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:44.531930 16441 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:44.491415 16469 rpcz_store.cc:276] 0709 12:23:42.840108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:42.924921 (+ 84813us) service_pool.cc:224] Handling call
0709 12:23:44.490660 (+1565739us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54002e63d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:44.490864 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3366218496}
I20260709 12:23:44.553079 16469 raft_consensus.cc:1240] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13. Ops: [11.8-12.9]
I20260709 12:23:44.557783 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:44.558784 20622 raft_consensus.cc:2804] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:44.559195 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:44.559396 20622 raft_consensus.cc:697] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:44.560271 17255 raft_consensus.cc:2393] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 13: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:44.560262 20568 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:44.561745 20568 raft_consensus.cc:697] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:44.561246 20622 consensus_queue.cc:237] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:44.562840 20568 consensus_queue.cc:237] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:44.565510 17255 raft_consensus.cc:1240] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:44.569089 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:44.569957 16548 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:23:44.571100 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:44.571687 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:44.571945 17264 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:44.572662 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:44.572937 20535 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:44.573333 17255 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:44.573887 20396 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:44.574252 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:44.574923 17255 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:44.574533 20396 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:44.575749 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:44.577240 20396 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:44.578042 20535 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:44.578830 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:44.580083 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:44.580437 17264 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
I20260709 12:23:44.581283 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:44.582031 17266 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
I20260709 12:23:44.583032 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:44.583719 17266 raft_consensus.cc:2468] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:44.580943 20535 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:44.558799 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:44.586169 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:44.586510 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:44.587026 16548 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:23:44.586174 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:44.590057 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:44.593204 20479 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:44.597986 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:44.598699 16472 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:44.601445 20242 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:44.602236 20242 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:44.605553 20576 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:44.615051 16543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1283) took 1954 ms. Trace:
I20260709 12:23:44.615835 16548 raft_consensus.cc:1240] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:23:44.615592 16543 rpcz_store.cc:276] 0709 12:23:42.660388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:42.660573 (+   185us) service_pool.cc:224] Handling call
0709 12:23:43.366045 (+705472us) spinlock_profiling.cc:243] Waited 1.71 ms on lock 0x7b58005e9298. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:23:43.743139 (+377094us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:43.746968 (+  3829us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:44.614760 (+867792us) spinlock_profiling.cc:243] Waited 377 ms on lock 0x7b5400a4f450. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:44.614969 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":870554240}
W20260709 12:23:44.613050 20681 log.cc:927] Time spent T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.468s	user 0.002s	sys 0.001s
I20260709 12:23:44.621102 19969 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:44.625238 16471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 787) took 1083 ms. Trace:
I20260709 12:23:44.627097 16476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 780) took 1152 ms. Trace:
I20260709 12:23:44.627513 16476 rpcz_store.cc:276] 0709 12:23:43.474555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:43.479494 (+  4939us) service_pool.cc:224] Handling call
0709 12:23:44.490827 (+1011333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:44.493149 (+  2322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:44.626916 (+133767us) spinlock_profiling.cc:243] Waited 995 ms on lock 0x7b54002e63d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:44.627043 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2287354752}
I20260709 12:23:44.625936 16471 rpcz_store.cc:276] 0709 12:23:43.542183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:43.748795 (+206612us) service_pool.cc:224] Handling call
0709 12:23:44.373365 (+624570us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:44.422752 (+ 49387us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:44.624946 (+202194us) spinlock_profiling.cc:243] Waited 624 ms on lock 0x7b54004c4ad0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:44.625155 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1435608960}
I20260709 12:23:44.605921 20669 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:44.608269 16474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 772) took 1406 ms. Trace:
I20260709 12:23:44.601642 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:44.633785 16468 raft_consensus.cc:2410] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
W20260709 12:23:44.609918 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 82788.6us
W20260709 12:23:44.635520 16551 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.025s	sys 0.061s
I20260709 12:23:44.632679 16474 rpcz_store.cc:276] 0709 12:23:43.202045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:43.202235 (+   190us) service_pool.cc:224] Handling call
0709 12:23:44.373866 (+1171631us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:44.436081 (+ 62215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:44.608056 (+171975us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54004eff50. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:44.608190 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2693591936}
I20260709 12:23:44.643617 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:44.646687 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:44.649161 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:44.651614 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:44.652568 16470 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:44.589475 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:44.654357 16441 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:44.710210 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:44.712069 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:44.664255 20535 consensus_queue.cc:1059] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 12:23:44.711293 17255 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
I20260709 12:23:44.719501 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:44.719913 17142 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:44.727320 17266 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:23:44.737012 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:44.738613 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:44.768160 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:44.768638 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 48847.6us
I20260709 12:23:44.776443 17266 raft_consensus.cc:1275] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:23:44.777927 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:44.781587 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:44.779013 17266 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:23:44.706063 20334 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:44.790575 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:44.789093 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:44.789438 16477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 789) took 1246 ms. Trace:
I20260709 12:23:44.724277 20396 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:44.797444 20218 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:44.804586 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:44.807449 17142 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:44.838653 17271 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:23:44.839558 17142 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:44.808442 16472 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:44.811700 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:44.779822 20689 log.cc:927] Time spent T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:23:44.767338 20535 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:44.813764 17269 raft_consensus.cc:1275] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:44.799793 20576 raft_consensus.cc:3055] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:44.817008 16477 rpcz_store.cc:276] 0709 12:23:43.542754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:43.762347 (+219593us) service_pool.cc:224] Handling call
0709 12:23:43.762704 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:43.898161 (+135457us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:23:43.958546 (+ 60385us) log.cc:844] Serialized 14 byte log entry
0709 12:23:44.109940 (+151394us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:44.110558 (+   618us) log.cc:844] Serialized 26 byte log entry
0709 12:23:44.110930 (+   372us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:44.116779 (+  5849us) log.cc:844] Serialized 14 byte log entry
0709 12:23:44.116999 (+   220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:44.117084 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:44.487305 (+370221us) spinlock_profiling.cc:243] Waited 1.59 ms on lock 0x7b1800139a20. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:23:44.596093 (+108788us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:23:44.789210 (+193117us) raft_consensus.cc:1685] finished
0709 12:23:44.789232 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:44.789364 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":183677,"prepare.queue_time_us":63515,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":79,"raft.queue_time_us":136705,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":11337,"spinlock_wait_cycles":15530880,"thread_start_us":128225,"threads_started":1,"wal-append.queue_time_us":175732}
W20260709 12:23:44.832345 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:44.863183 17263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 1081) took 1831 ms. Trace:
I20260709 12:23:44.863761 17263 rpcz_store.cc:276] 0709 12:23:43.031398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:43.049888 (+ 18490us) service_pool.cc:224] Handling call
0709 12:23:43.050199 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:43.261812 (+211613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:44.862846 (+1601034us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b540005e454. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:44.863098 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":444818176}
I20260709 12:23:44.869136 20600 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:44.874323 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:44.876150 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:44.877331 16468 raft_consensus.cc:2410] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:23:44.878517 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:44.880373 16469 raft_consensus.cc:1240] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:44.806742 20334 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:44.967789 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 128316us
I20260709 12:23:44.840220 20396 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:44.976120 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:44.977315 16470 raft_consensus.cc:2410] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:23:44.858407 20576 raft_consensus.cc:740] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:44.994056 17142 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:44.974622 20396 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:45.025440 17142 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:44.859195 20242 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:45.042781 20676 consensus_meta.cc:287] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.908s	user 0.005s	sys 0.008s
I20260709 12:23:44.860137 20535 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:45.040982 20540 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:44.999691 20701 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:45.046233 20229 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:45.080250 20568 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:45.080343 20540 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:23:45.120846 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 466561us
I20260709 12:23:45.068189 20208 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:23:45.122129 16545 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.019s	sys 0.061s
W20260709 12:23:45.147181 16545 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.020s	sys 0.061s
I20260709 12:23:45.144879 16441 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
W20260709 12:23:45.148103 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 122754us
I20260709 12:23:45.143661 20540 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:45.065014 20627 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:45.080929 20622 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:45.147192 20691 log.cc:927] Time spent T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.152s	user 0.001s	sys 0.001s
I20260709 12:23:45.078444 20676 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:23:45.143844 20670 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.044s	sys 0.118s
W20260709 12:23:45.157567 19969 consensus_meta.cc:287] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.536s	user 0.010s	sys 0.002s
I20260709 12:23:45.046926 20576 consensus_queue.cc:260] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:45.200224 16544 raft_consensus.cc:2410] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:23:45.204077 17254 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:45.176728 20670 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.045s	sys 0.119s
I20260709 12:23:45.210418 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:45.213279 16549 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37242 (request call id 1280) took 2540 ms (client timeout 2051 ms). Trace:
I20260709 12:23:45.213604 17254 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:45.168686 20701 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:45.146708 20568 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:45.213912 16549 rpcz_store.cc:269] 0709 12:23:42.657863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:42.658323 (+   460us) service_pool.cc:224] Handling call
0709 12:23:42.973787 (+315464us) spinlock_profiling.cc:243] Waited 2.82 ms on lock 0x7b58005e9298. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:23:45.193627 (+2219840us) spinlock_profiling.cc:243] Waited 495 ms on lock 0x7b540082e6d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:45.198292 (+  4665us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1145606784}
I20260709 12:23:45.219805 20676 consensus_queue.cc:1048] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:23:45.178117 20229 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:45.164109 20627 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:45.212144 16215 maintenance_manager.cc:419] P 9e68af265dd04372955944ded6c1437a: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033350
I20260709 12:23:45.206950 19969 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:45.170048 20622 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:45.205480 20706 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:45.263808 20576 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:45.289629 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 140306us
I20260709 12:23:45.292987 16551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1489) took 1173 ms. Trace:
I20260709 12:23:45.266532 20229 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:45.205265 20707 consensus_queue.cc:1048] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
W20260709 12:23:45.268173 20692 log.cc:927] Time spent T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.280s	user 0.001s	sys 0.000s
I20260709 12:23:45.269217 20708 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:45.316401 17142 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:45.314776 16551 rpcz_store.cc:276] 0709 12:23:44.119594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:44.119765 (+   171us) service_pool.cc:224] Handling call
0709 12:23:44.125646 (+  5881us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:44.125982 (+   336us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:23:44.636094 (+510112us) log.cc:844] Serialized 14 byte log entry
0709 12:23:45.292531 (+656437us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:45.292624 (+    93us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:45.292704 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:45.292775 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:45.292914 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":474136,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"thread_start_us":1165343,"threads_started":3,"wal-append.queue_time_us":459264}
I20260709 12:23:45.264868 20242 consensus_queue.cc:1059] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 12:23:45.292073 20334 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:45.274861 20706 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:45.343189 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 169576us
I20260709 12:23:45.325481 20229 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:45.339032 20700 log.cc:927] Time spent T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.283s	user 0.002s	sys 0.000s
I20260709 12:23:45.274399 19969 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:45.349349 17142 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:45.352559 20600 consensus_queue.cc:1048] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:23:45.357084 20697 log.cc:927] Time spent T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:23:45.364025 17142 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:45.337381 20713 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:45.344318 20706 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:45.370900 16441 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:45.365501 20334 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:45.368124 20701 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:45.370224 20535 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:45.349494 20709 consensus_queue.cc:1048] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:23:45.368963 20711 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:45.339781 20708 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:45.369524 20713 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:45.391275 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:45.393921 16547 raft_consensus.cc:2410] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:23:45.364197 20242 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.021s
W20260709 12:23:45.395124 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:45.378013 20670 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:45.400045 16551 raft_consensus.cc:1275] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:23:45.396449 16441 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:23:45.403781 16550 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.008s	sys 0.008s
I20260709 12:23:45.388583 20701 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:45.388734 20566 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:23:45.404210 16550 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.008s	sys 0.008s
W20260709 12:23:45.398239 20658 log.cc:927] Time spent T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 1.621s	user 0.005s	sys 0.020s
I20260709 12:23:45.386654 20714 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:45.406426 16441 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:45.397576 20470 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:45.387822 20707 consensus_queue.cc:1048] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 12:23:45.396579 20712 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:23:45.438515 20710 log.cc:927] Time spent T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:23:45.397230 20713 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:45.451656 19969 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:45.453437 20708 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:45.443548 20470 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:45.445264 20572 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:45.456612 20627 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:45.446213 20535 consensus_queue.cc:1048] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 12:23:45.448726 20705 consensus_queue.cc:1059] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.011s
W20260709 12:23:45.462354 20683 log.cc:927] Time spent T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 1.026s	user 0.010s	sys 0.021s
I20260709 12:23:45.463074 20568 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:45.468103 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:45.469154 16395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 1065) took 1946 ms. Trace:
I20260709 12:23:45.407050 20566 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:45.393083 20334 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:45.443401 20693 log.cc:927] Time spent T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 12:23:45.472031 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:45.486836 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:45.406886 20701 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:45.408381 20600 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:45.469643 16395 rpcz_store.cc:276] 0709 12:23:43.522428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:43.523352 (+   924us) service_pool.cc:224] Handling call
0709 12:23:43.540315 (+ 16963us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:43.540575 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:23:44.115821 (+575246us) log.cc:844] Serialized 14 byte log entry
0709 12:23:45.468743 (+1352922us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:45.468803 (+    60us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:45.468859 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:45.468981 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:45.469100 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1502371,"prepare.queue_time_us":451789,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"thread_start_us":416444,"threads_started":2,"wal-append.queue_time_us":489807}
I20260709 12:23:45.484731 20657 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:45.484795 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:45.486512 16396 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
W20260709 12:23:45.492873 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 90607.8us
I20260709 12:23:45.397445 20711 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:45.436184 20670 consensus_queue.cc:1048] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 12:23:45.435766 20717 consensus_queue.cc:1048] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
W20260709 12:23:45.493346 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 50831us
I20260709 12:23:45.527578 16515 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:45.534339 16515 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:45.477485 20566 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:45.539043 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 94553.6us
I20260709 12:23:45.490904 20617 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:45.491828 20470 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:45.550887 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:45.579576 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:45.550305 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:45.580403 16473 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:45.581537 17261 raft_consensus.cc:1275] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 12:23:45.524367 20535 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:45.582669 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:45.578459 16366 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:45.548267 17255 raft_consensus.cc:1275] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:23:45.583375 17271 raft_consensus.cc:2468] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
I20260709 12:23:45.543713 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:45.525997 20267 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:45.597342 20535 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:45.443759 20714 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:45.449625 20712 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:45.538030 20722 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:45.531723 20674 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:45.541544 20709 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:45.541255 20667 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:45.620136 16473 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:23:45.617164 16366 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:45.542661 19969 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:45.582052 20622 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:45.609300 20693 log.cc:927] Time spent T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:23:45.613127 20669 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:45.634440 20368 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:23:45.642522 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 106201us
I20260709 12:23:45.582134 20617 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:45.643601 16515 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:45.627151 19969 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:23:45.640923 20698 log.cc:927] Time spent T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
W20260709 12:23:45.661441 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 90852.5us
I20260709 12:23:45.624814 20275 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:45.664216 16515 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:45.611191 20267 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:45.671809 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:45.675521 16515 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:45.672595 16441 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:45.672711 16542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1503) took 1166 ms. Trace:
I20260709 12:23:45.639612 20669 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:45.678483 16515 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:45.679491 16515 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:45.629180 20667 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:45.659112 20720 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:45.676645 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:45.677449 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:45.625970 20709 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:45.622941 20674 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:45.663668 20368 raft_consensus.cc:697] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:45.672854 19969 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:45.676662 16542 rpcz_store.cc:276] 0709 12:23:44.506329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:44.521326 (+ 14997us) service_pool.cc:224] Handling call
0709 12:23:44.521871 (+   545us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:44.522782 (+   911us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:44.522905 (+   123us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:44.524246 (+  1341us) log.cc:844] Serialized 52 byte log entry
0709 12:23:45.001426 (+477180us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:45.046791 (+ 45365us) log.cc:844] Serialized 14 byte log entry
0709 12:23:45.267577 (+220786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:45.267946 (+   369us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:45.672407 (+404461us) raft_consensus.cc:1685] finished
0709 12:23:45.672427 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:45.672649 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17561,"prepare.queue_time_us":52524,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":320120,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":3495,"thread_start_us":696754,"threads_started":2,"wal-append.queue_time_us":639831}
I20260709 12:23:45.671602 20721 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:45.687624 16394 raft_consensus.cc:2393] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 12: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:45.680842 20669 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:45.682307 20242 raft_consensus.cc:2804] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:45.701406 16545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1300) took 1110 ms. Trace:
I20260709 12:23:45.674033 20275 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:45.703481 16545 rpcz_store.cc:276] 0709 12:23:44.590869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:44.590985 (+   116us) service_pool.cc:224] Handling call
0709 12:23:44.591296 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:44.591681 (+   385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:44.591712 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:44.592855 (+  1143us) log.cc:844] Serialized 26 byte log entry
0709 12:23:45.147853 (+554998us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:45.147897 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:45.147969 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:45.701171 (+553202us) raft_consensus.cc:1685] finished
0709 12:23:45.701197 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:45.701341 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1047,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":88,"thread_start_us":554568,"threads_started":1,"wal-append.queue_time_us":680466}
I20260709 12:23:45.709623 16515 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:45.712599 17254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45230 (request call id 1136) took 1196 ms. Trace:
I20260709 12:23:45.691802 20721 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:45.713167 17254 rpcz_store.cc:276] 0709 12:23:44.515928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:44.516111 (+   183us) service_pool.cc:224] Handling call
0709 12:23:45.712323 (+1196212us) spinlock_profiling.cc:243] Waited 685 ms on lock 0x7b540005e6d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:45.712516 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1575713536}
I20260709 12:23:45.684894 20622 consensus_queue.cc:1048] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 12:23:45.701781 20470 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:23:45.710872 20396 consensus_meta.cc:287] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.651s	user 0.012s	sys 0.000s
I20260709 12:23:45.703162 20242 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:45.706648 20540 consensus_meta.cc:287] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.556s	user 0.014s	sys 0.000s
I20260709 12:23:45.687572 20720 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:45.732443 20242 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:45.732620 20540 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:45.735522 20540 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:45.706413 20677 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:45.690443 20729 consensus_queue.cc:1048] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:23:45.741765 20732 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:45.741807 20734 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:45.744199 20734 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:45.744781 20734 raft_consensus.cc:697] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:45.737900 20708 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:45.745867 20708 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:45.746269 20708 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:45.719801 20721 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:45.748729 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:45.732669 20707 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:45.749925 20707 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:45.750636 20734 consensus_queue.cc:237] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:45.752756 20739 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:45.737903 20334 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:45.753501 20739 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:45.728984 20670 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:45.756735 20670 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:45.759821 20670 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:45.741667 20368 consensus_queue.cc:237] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:45.727141 20728 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:45.752071 20707 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:45.728046 20470 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:23:45.768206 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 58673.3us
I20260709 12:23:45.780122 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:45.706496 20675 consensus_queue.cc:1059] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.182s
I20260709 12:23:45.745313 20674 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:45.796931 16515 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:45.796782 16474 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:45.753517 20275 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:45.768816 20711 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:45.736984 20396 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:45.807437 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:45.795601 20470 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:45.816833 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:45.829708 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 289241us
I20260709 12:23:45.826138 16515 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:45.837598 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:45.833551 16515 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:45.853322 16476 raft_consensus.cc:1240] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:45.811808 20674 raft_consensus.cc:3055] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:45.855754 16515 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:45.811519 20667 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:45.829319 20712 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:45.768206 20677 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:45.870225 16401 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 17.
I20260709 12:23:45.872596 20617 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:45.855860 20674 raft_consensus.cc:740] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:45.873004 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:45.871798 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:45.839437 20714 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:45.863873 20334 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:45.903148 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:45.912739 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:45.796393 20728 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:45.904157 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:45.824747 20566 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:45.855377 20746 raft_consensus.cc:2804] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:45.853523 20737 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:45.916354 16473 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:23:45.916939 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:23:45.926699 20736 log.cc:927] Time spent T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.195s	user 0.001s	sys 0.001s
W20260709 12:23:45.964041 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 108040us
I20260709 12:23:45.945222 20631 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:45.946094 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:45.946292 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:45.969607 16475 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
I20260709 12:23:45.946995 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:45.947773 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:45.872804 20739 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:45.950740 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:45.914285 16399 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
I20260709 12:23:45.971323 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:45.971666 16396 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:45.972666 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:45.961621 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:45.942023 20576 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:45.854394 20721 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:45.965103 16514 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:45.911342 20334 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:45.952642 20229 consensus_meta.cc:287] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.585s	user 0.008s	sys 0.003s
W20260709 12:23:46.038950 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 183273us
I20260709 12:23:45.965819 20740 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:45.985016 16471 raft_consensus.cc:1240] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:46.035331 20470 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:45.956418 20709 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:45.955638 20746 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:45.987329 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:23:46.039476 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 74442.7us
I20260709 12:23:46.040365 16472 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
I20260709 12:23:46.040858 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:45.984998 20208 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:46.049633 16396 raft_consensus.cc:2393] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 12: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:46.050067 16401 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
I20260709 12:23:46.080125 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:46.083671 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:46.042830 20334 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:46.036891 20663 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:46.049696 17144 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:46.081188 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:46.091634 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 49686.8us
I20260709 12:23:46.092227 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:46.092649 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:46.094178 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:46.103540 16514 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:46.080004 20713 consensus_meta.cc:287] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.622s	user 0.010s	sys 0.003s
I20260709 12:23:46.108450 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:46.116529 16548 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
W20260709 12:23:46.116823 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 61041.5us
I20260709 12:23:46.112875 16550 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:46.046059 20712 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:23:46.113349 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 63710us
I20260709 12:23:46.123631 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:46.113621 16542 raft_consensus.cc:2410] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:23:46.111385 16441 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:46.114074 16396 raft_consensus.cc:2410] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:23:46.106945 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:46.048806 20746 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:46.126246 16442 leader_election.cc:336] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.740s (SENT)
I20260709 12:23:46.039893 20576 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:46.091045 20711 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:46.059195 20568 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:46.143869 16548 raft_consensus.cc:1275] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 6. (term mismatch)
I20260709 12:23:46.144049 16442 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:46.144419 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:46.160678 16548 pending_rounds.cc:85] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1: Aborting all ops after (but not including) 5
I20260709 12:23:46.160986 16548 pending_rounds.cc:107] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:23:46.161254 16548 raft_consensus.cc:2889] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:46.161669 16548 pending_rounds.cc:107] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:23:46.161928 16548 raft_consensus.cc:2889] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:46.091297 20737 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:46.044888 20229 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:46.038028 20631 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:46.151659 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:46.159137 16396 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:23:46.103430 20739 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:46.036651 20674 consensus_queue.cc:260] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:46.103780 20732 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:46.163861 20752 raft_consensus.cc:2749] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:46.164104 20758 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:46.164363 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:46.215991 16401 raft_consensus.cc:2376] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11. Re-sending same reply.
I20260709 12:23:46.129627 20707 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:23:46.181682 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 55474us
I20260709 12:23:46.191836 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:46.217588 16472 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
W20260709 12:23:46.218052 16442 leader_election.cc:336] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.863s (SENT)
I20260709 12:23:46.218557 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:46.192660 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:46.111385 20208 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:46.214092 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:46.225939 16551 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
I20260709 12:23:46.130327 20713 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:46.215087 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 82750.2us
I20260709 12:23:46.124724 20745 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:46.233309 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:46.123396 20663 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:46.163515 20712 raft_consensus.cc:697] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 13 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:46.238732 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:46.221617 20762 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:23:46.222886 20747 log.cc:927] Time spent T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 12:23:46.222663 20729 consensus_queue.cc:1048] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:23:46.223803 20749 log.cc:927] Time spent T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:23:46.224891 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:46.168025 20714 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:46.234905 20674 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:46.234210 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:46.236788 16514 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:46.236938 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:46.219204 16442 leader_election.cc:336] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.927s (SENT)
W20260709 12:23:46.280583 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 61381.1us
I20260709 12:23:46.272668 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:46.219841 20707 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:46.274866 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:46.162992 20711 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:46.165923 20568 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:46.280755 16470 raft_consensus.cc:2393] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 12: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
W20260709 12:23:46.281277 16442 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 2.033s (SENT) [suppressed 28 similar messages]
I20260709 12:23:46.216912 20396 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:46.281438 16514 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:46.282635 16441 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
W20260709 12:23:46.342615 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 60086.5us
I20260709 12:23:46.213697 19969 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:46.287955 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:46.289717 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:46.284209 20707 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:46.297289 20755 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:46.298441 20568 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:46.291092 20714 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:46.300158 20760 log.cc:927] Time spent T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:23:46.305195 20566 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:46.307010 20208 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:46.246630 20709 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:23:46.313203 20242 consensus_meta.cc:287] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.580s	user 0.009s	sys 0.002s
W20260709 12:23:46.314294 20750 log.cc:927] Time spent T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 12:23:46.226435 20758 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:46.280279 20677 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:46.317915 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:46.369415 16551 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:23:46.318838 20470 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:46.320297 20713 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:46.220016 20732 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:46.298441 20762 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:46.325733 20728 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:46.326970 20669 consensus_meta.cc:287] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.620s	user 0.009s	sys 0.001s
W20260709 12:23:46.334945 20706 consensus_meta.cc:287] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.955s	user 0.010s	sys 0.000s
I20260709 12:23:46.335054 20746 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:46.324173 20708 consensus_meta.cc:287] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.577s	user 0.008s	sys 0.002s
I20260709 12:23:46.336648 20631 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:46.337296 20722 consensus_meta.cc:287] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.710s	user 0.010s	sys 0.002s
I20260709 12:23:46.228549 20739 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:46.189177 20737 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:46.339080 20712 consensus_queue.cc:237] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:46.336668 20576 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:46.340246 20711 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:46.414515 20739 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:46.345278 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 63917.2us
I20260709 12:23:46.417829 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:46.419076 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:46.419269 16514 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:46.419654 16394 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:46.349996 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:46.351838 16550 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:23:46.425271 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:46.425967 16550 raft_consensus.cc:2376] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14. Re-sending same reply.
I20260709 12:23:46.426863 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:46.427477 16550 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
I20260709 12:23:46.428539 20752 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:46.432590 20722 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:46.434545 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:46.438454 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:46.439152 16468 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
I20260709 12:23:46.439317 20758 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:46.369855 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:46.353855 20755 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:46.441490 20755 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:46.441535 16543 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
I20260709 12:23:46.441851 19969 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:46.349574 20559 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:46.442991 16394 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
I20260709 12:23:46.442883 20559 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:46.444088 20714 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:46.238448 20745 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:46.444993 20559 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:46.445256 16394 raft_consensus.cc:1240] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:23:46.446728 20745 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:46.440039 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:46.448963 20732 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:46.450223 20229 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:46.450757 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:46.452207 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:46.453565 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:46.454833 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:46.459370 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:46.460584 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:46.383189 20479 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:46.389549 20669 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:46.470384 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:46.416648 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:46.461120 20663 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:46.493314 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:46.462603 16542 raft_consensus.cc:1275] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:46.493901 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:46.405390 20706 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:46.468286 20479 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:46.366237 20709 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:46.496671 20479 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:46.471355 16400 raft_consensus.cc:2368] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11.
I20260709 12:23:46.411157 20754 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:46.388396 20721 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:46.392273 17143 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:46.473672 16514 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
W20260709 12:23:46.500797 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 108634us
I20260709 12:23:46.501829 17143 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:46.502283 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:46.383168 20768 raft_consensus.cc:2764] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:23:46.475982 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:46.503487 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:46.456106 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:46.478657 16548 raft_consensus.cc:1275] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:23:46.482911 16545 raft_consensus.cc:2393] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 11: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:46.368959 20242 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:46.470554 20669 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:46.419191 20737 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:46.493997 16395 raft_consensus.cc:2376] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11. Re-sending same reply.
I20260709 12:23:46.503657 16551 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:23:46.502868 17143 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:46.489949 20600 consensus_queue.cc:1048] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 12:23:46.525465 17255 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d"
dest_uuid: "ef57beab0e1d40d585983873c55107e8"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:45120
W20260709 12:23:46.550944 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 48141.3us
I20260709 12:23:46.551632 17255 raft_consensus.cc:606] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:46.552011 17255 raft_consensus.cc:614] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:46.517338 20709 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:46.552554 20758 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:46.552977 20758 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:46.553339 20758 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:46.554190 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:46.555128 16551 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:46.557830 20242 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:46.558995 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:46.559633 16395 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
W20260709 12:23:46.545394 20701 consensus_meta.cc:287] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 1.019s	user 0.012s	sys 0.000s
I20260709 12:23:46.410207 20708 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:46.560748 20701 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:46.562193 20708 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:46.562611 20208 consensus_queue.cc:1048] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:46.563009 20701 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:46.519455 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:46.564489 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:46.564760 20334 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:46.565192 20208 consensus_queue.cc:1048] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:46.565900 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:46.566661 16398 raft_consensus.cc:2393] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 12: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:46.566509 17143 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:46.390796 20762 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:46.560669 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:46.567579 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:46.575292 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:46.560197 17142 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:46.568543 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:46.566869 20334 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:46.569276 20762 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:46.537246 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:46.517053 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:46.571997 16442 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:46.514194 20767 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:46.516731 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:46.537930 16366 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:46.581619 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:46.582042 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:46.582329 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:46.582527 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:46.583199 20479 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:46.583546 20701 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:46.586068 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:46.504123 16396 raft_consensus.cc:2376] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 17. Re-sending same reply.
I20260709 12:23:46.600970 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:46.612262 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 52108.9us
I20260709 12:23:46.641036 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:46.579838 20715 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.922s	user 0.000s	sys 0.015s
I20260709 12:23:46.642210 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:23:46.642680 20715 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.001s	user 0.000s	sys 0.019s
I20260709 12:23:46.642910 17266 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:23:46.643721 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:46.603361 16367 connection.cc:256] RPC call timeout handler was delayed by -1.0551310490000105s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1574s for a response
W20260709 12:23:46.645344 16367 connection.cc:256] RPC call timeout handler was delayed by -1.0200310489999538s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1613s for a response
W20260709 12:23:46.638185 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 100345us
I20260709 12:23:46.646637 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:46.647425 16394 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:23:46.637766 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:46.615521 16442 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:46.591946 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:46.588217 16551 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
I20260709 12:23:46.648451 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:46.530911 20535 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:46.496868 20706 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:46.650231 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:46.667747 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:46.650705 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:46.681471 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:46.683841 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:46.685649 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:46.686002 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:46.657783 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:46.668368 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:46.688000 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:46.688622 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:46.550542 20746 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:46.689961 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:46.614138 20675 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:46.650902 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:46.514204 20663 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:46.633780 20762 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:46.694525 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:46.678035 17254 raft_consensus.cc:1240] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 10 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:46.671460 16441 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:46.694648 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:46.694806 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:46.697033 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:46.514891 20717 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:46.679234 16366 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:46.600987 20732 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:23:46.692893 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 77503.4us
I20260709 12:23:46.708456 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:46.701037 17258 raft_consensus.cc:1240] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:46.702611 16394 raft_consensus.cc:1275] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:46.706696 16395 raft_consensus.cc:3055] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:46.709883 16395 raft_consensus.cc:740] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:46.710211 16550 raft_consensus.cc:2393] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 14: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:46.643793 20669 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:46.637810 20701 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:46.716650 16442 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:46.713001 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:46.716918 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:46.721230 16366 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:46.725806 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:46.726676 16550 raft_consensus.cc:2376] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13. Re-sending same reply.
I20260709 12:23:46.730784 16442 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:46.728040 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:46.733428 16544 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37226 (request call id 1490) took 2608 ms (client timeout 2207 ms). Trace:
I20260709 12:23:46.733645 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:46.734195 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:46.737496 17258 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:46.739506 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:46.734922 16442 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:46.733975 16544 rpcz_store.cc:269] 0709 12:23:44.125239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:44.125555 (+   316us) service_pool.cc:224] Handling call
0709 12:23:46.733199 (+2607644us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400283a54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:46.733348 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2463895424}
I20260709 12:23:46.741982 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:46.743548 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:46.747310 20373 consensus_queue.cc:1059] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:46.748669 20373 consensus_queue.cc:1243] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer ef57beab0e1d40d585983873c55107e8 log is divergent from this leader: its last log entry 12.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:23:46.753969 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:46.754861 17263 raft_consensus.cc:1275] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:23:46.701524 20663 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:46.756067 16442 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:46.754899 16472 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:23:46.752436 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:46.757737 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:46.717447 20767 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:46.759753 16543 raft_consensus.cc:2376] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10. Re-sending same reply.
I20260709 12:23:46.761650 16471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1375) took 1218 ms. Trace:
I20260709 12:23:46.723456 20701 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:46.763491 16544 raft_consensus.cc:2393] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 14: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:46.765520 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:46.771139 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:46.771929 20704 raft_consensus.cc:3055] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:46.779269 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:46.785423 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 64285.4us
I20260709 12:23:46.787745 16471 rpcz_store.cc:276] 0709 12:23:45.543531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.581502 (+ 37971us) service_pool.cc:224] Handling call
0709 12:23:45.608893 (+ 27391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:46.044791 (+435898us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:46.761323 (+716532us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b5400594554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:46.761558 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":864477056}
I20260709 12:23:46.762099 20720 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:46.789374 17266 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:46.791250 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:46.732540 20717 raft_consensus.cc:697] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:46.715097 20437 consensus_queue.cc:1048] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 12:23:46.808017 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:46.738312 20773 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:46.811173 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:46.800886 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:46.802201 17263 raft_consensus.cc:2393] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 11: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:46.803292 16545 raft_consensus.cc:1240] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:23:46.812265 16551 raft_consensus.cc:2393] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 14: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:46.805321 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:46.811839 17254 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:46.813100 16468 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:46.759382 20677 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:46.815681 20677 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:46.816097 20677 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:46.816507 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:46.816582 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:46.739421 20242 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:46.820381 20529 raft_consensus.cc:3055] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:46.795003 20704 raft_consensus.cc:740] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:46.820838 20529 raft_consensus.cc:740] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:46.821367 20267 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:46.799837 20767 raft_consensus.cc:697] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:46.821735 20529 consensus_queue.cc:260] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:46.822613 20767 consensus_queue.cc:237] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:46.823127 20732 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:46.823661 20732 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:46.826711 16402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 1054) took 3487 ms. Trace:
I20260709 12:23:46.827214 16402 rpcz_store.cc:276] 0709 12:23:43.339576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:43.349400 (+  9824us) service_pool.cc:224] Handling call
0709 12:23:43.349734 (+   334us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:43.429122 (+ 79388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:43.429162 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:43.458351 (+ 29189us) log.cc:844] Serialized 52 byte log entry
0709 12:23:43.708765 (+250414us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:44.111380 (+402615us) log.cc:844] Serialized 14 byte log entry
0709 12:23:45.435761 (+1324381us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:46.811732 (+1375971us) spinlock_profiling.cc:243] Waited 78.9 ms on lock 0x7b540015f254. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:46.811782 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:46.811829 (+    47us) raft_consensus.cc:1685] finished
0709 12:23:46.826428 (+ 14599us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:46.826618 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1669038,"prepare.queue_time_us":274431,"prepare.run_cpu_time_us":23,"prepare.run_wall_time_us":23,"raft.queue_time_us":1331985,"raft.run_cpu_time_us":358,"raft.run_wall_time_us":15031,"spinlock_wait_cycles":181493632,"thread_start_us":336814,"threads_started":2,"wal-append.queue_time_us":263935,"wal-append.run_cpu_time_us":3664,"wal-append.run_wall_time_us":1398434}
I20260709 12:23:46.829484 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:46.830145 16475 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:46.831072 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:46.831349 20761 log.cc:927] Time spent T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.666s	user 0.001s	sys 0.004s
I20260709 12:23:46.831698 16475 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:23:46.836787 16403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 1075) took 1366 ms. Trace:
W20260709 12:23:46.837406 20696 log.cc:927] Time spent T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 1.677s	user 0.002s	sys 0.003s
I20260709 12:23:46.837294 16403 rpcz_store.cc:276] 0709 12:23:45.470413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.470822 (+   409us) service_pool.cc:224] Handling call
0709 12:23:45.493851 (+ 23029us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:45.682402 (+188551us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:23:45.700856 (+ 18454us) log.cc:844] Serialized 14 byte log entry
0709 12:23:45.947132 (+246276us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b1800018a60. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a2923a 00007f2875e93a2a
0709 12:23:46.049448 (+102316us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:46.049497 (+    49us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:46.049538 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:46.836113 (+786575us) spinlock_profiling.cc:243] Waited 8.49 ms on lock 0x7b1800018a60. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d6a2a6 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419
0709 12:23:46.836572 (+   459us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b5400ad34d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:46.836611 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:46.836727 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15356,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":34,"raft.queue_time_us":595,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":30344,"spinlock_wait_cycles":480332928,"thread_start_us":117626,"threads_started":2,"wal-append.queue_time_us":444137}
I20260709 12:23:46.839286 16395 consensus_queue.cc:260] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:46.841477 16395 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:46.805337 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:46.807088 17258 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:23:46.759351 20663 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:46.709339 20675 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:46.758291 20746 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:46.786980 20600 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:46.731348 20535 consensus_queue.cc:1059] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 12:23:46.787372 20762 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:46.723098 20706 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 12:23:46.814785 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:23:46.815066 20777 log.cc:927] Time spent T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:23:46.829672 16476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1376) took 1284 ms. Trace:
I20260709 12:23:46.918516 16476 rpcz_store.cc:276] 0709 12:23:45.545521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.546049 (+   528us) service_pool.cc:224] Handling call
0709 12:23:45.594561 (+ 48512us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:45.853956 (+259395us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:46.829446 (+975490us) spinlock_profiling.cc:243] Waited 258 ms on lock 0x7b5400a686d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:46.829615 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":594462464}
I20260709 12:23:46.842554 16473 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:23:46.809338 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:23:46.843135 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 126382us
I20260709 12:23:46.920677 17266 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:46.923031 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:46.924296 17270 raft_consensus.cc:2393] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 10: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:46.843499 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:23:46.850143 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:46.893337 16546 raft_consensus.cc:1240] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:46.805860 20674 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:46.903338 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:46.935891 16477 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:46.836871 20717 consensus_queue.cc:237] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:46.906455 16548 raft_consensus.cc:1240] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 9. Current term is 10. Ops: []
W20260709 12:23:46.907192 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 64348.4us
W20260709 12:23:46.845836 20742 log.cc:927] Time spent T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:23:46.911147 16471 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:46.928893 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:46.951614 16475 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:46.952682 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:46.932320 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:46.929736 16441 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:46.943620 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:46.956221 17261 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:46.944892 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:46.957510 16473 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
W20260709 12:23:46.959996 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:46.943054 17254 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:23:46.939742 16468 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:23:46.907873 20657 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:46.917155 20572 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:46.968636 20572 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:46.974515 16468 raft_consensus.cc:1240] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:46.918418 20725 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:23:46.948786 20752 consensus_meta.cc:287] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.520s	user 0.010s	sys 0.001s
I20260709 12:23:46.964043 16475 raft_consensus.cc:2393] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 14: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:46.960811 17143 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:46.935516 20674 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:23:46.968156 20791 log.cc:927] Time spent T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:23:46.970624 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:46.993278 16476 raft_consensus.cc:2393] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 11: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:46.976462 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:46.977901 16468 raft_consensus.cc:1240] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13. Ops: [11.8-12.9]
I20260709 12:23:46.993491 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:46.995445 16477 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:46.948303 17266 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
I20260709 12:23:46.959553 16367 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:46.896294 20785 consensus_queue.cc:1059] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 12:23:46.821731 20704 consensus_queue.cc:260] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:46.908228 20746 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:46.974313 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 44502.2us
I20260709 12:23:46.984309 17143 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:46.978559 20755 consensus_meta.cc:287] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.536s	user 0.010s	sys 0.000s
I20260709 12:23:47.010614 16441 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:23:47.001744 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:46.970556 20657 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:47.025180 20781 log.cc:927] Time spent T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
W20260709 12:23:47.027180 20780 log.cc:927] Time spent T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
W20260709 12:23:47.030884 20723 log.cc:927] Time spent T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.509s	user 0.002s	sys 0.006s
W20260709 12:23:47.031618 20741 log.cc:927] Time spent T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 1.163s	user 0.003s	sys 0.004s
I20260709 12:23:47.009449 17261 raft_consensus.cc:2468] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
I20260709 12:23:47.019409 20267 raft_consensus.cc:2764] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:23:47.021584 20746 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:47.021250 20755 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:47.033087 16397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 1220) took 1387 ms. Trace:
I20260709 12:23:47.016021 17143 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:46.993641 20674 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:47.041972 20631 consensus_queue.cc:1175] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ef57beab0e1d40d585983873c55107e8" responder_term: 10 status { last_received { term: 10 index: 8 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 9. Current term is 10. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:23:47.047345 20707 consensus_meta.cc:287] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.689s	user 0.010s	sys 0.001s
I20260709 12:23:47.047811 20707 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:47.049978 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:47.050786 17261 raft_consensus.cc:2393] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 14: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:47.050895 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:47.051465 17269 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:47.052726 20674 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:47.049952 20707 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:47.053920 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:47.055557 20674 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:47.056202 20674 consensus_queue.cc:1059] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:23:47.042935 16397 rpcz_store.cc:276] 0709 12:23:45.645856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.645995 (+   139us) service_pool.cc:224] Handling call
0709 12:23:45.669286 (+ 23291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:45.670281 (+   995us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:45.670312 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:45.674066 (+  3754us) log.cc:844] Serialized 26 byte log entry
0709 12:23:45.776689 (+102623us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:45.776735 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:45.776807 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:47.030990 (+1254183us) spinlock_profiling.cc:243] Waited 4.44 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:23:47.032765 (+  1775us) raft_consensus.cc:1685] finished
0709 12:23:47.032788 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:47.033021 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22235,"raft.queue_time_us":14140,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":6322,"spinlock_wait_cycles":10218240,"thread_start_us":65915,"threads_started":1,"wal-append.queue_time_us":115626}
I20260709 12:23:47.016814 20670 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:47.066324 20675 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:47.067143 20675 raft_consensus.cc:3055] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:47.067595 20675 raft_consensus.cc:740] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:47.038007 20768 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:47.068904 20768 raft_consensus.cc:697] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:47.069921 20768 consensus_queue.cc:237] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:46.991101 20752 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:23:47.075392 20758 consensus_meta.cc:287] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.522s	user 0.005s	sys 0.003s
I20260709 12:23:47.038492 20704 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:47.079332 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 63406.4us
I20260709 12:23:47.080295 17143 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:47.093748 17269 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
I20260709 12:23:47.096002 20796 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:47.096698 20796 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:47.085644 20559 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:47.100746 20559 raft_consensus.cc:697] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 10 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:47.102015 20796 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:47.101876 20559 consensus_queue.cc:237] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:46.981151 20725 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:47.107337 16441 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:47.107668 20725 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:47.108709 20535 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:47.109310 20535 raft_consensus.cc:697] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:47.110369 20677 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:47.110419 20535 consensus_queue.cc:237] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:47.117076 16550 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
W20260709 12:23:47.118043 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.118911 16441 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:47.120595 16471 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:23:47.121928 16477 raft_consensus.cc:1240] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:47.127427 16395 raft_consensus.cc:1275] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 12:23:47.126020 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:47.132710 16395 pending_rounds.cc:85] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 5
I20260709 12:23:47.133034 16395 pending_rounds.cc:107] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 12:23:47.133219 16548 raft_consensus.cc:2368] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 10. Current term is 11.
I20260709 12:23:47.038695 20709 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:47.129123 16542 raft_consensus.cc:1240] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 14. Current term is 15. Ops: []
W20260709 12:23:47.129702 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 184878us
W20260709 12:23:47.082856 20747 log.cc:927] Time spent T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:23:47.147037 17142 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:47.086786 20676 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:47.154029 20572 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:47.129218 20600 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:23:47.154827 20786 log.cc:927] Time spent T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:23:47.133255 16395 raft_consensus.cc:2889] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:47.158020 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:47.158887 16551 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:23:47.076076 20758 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:47.131880 20770 raft_consensus.cc:2804] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:47.131194 20657 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:47.179010 17264 raft_consensus.cc:1240] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 13. Current term is 14. Ops: [13.6-13.6]
W20260709 12:23:47.186224 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.189610 20758 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:47.157387 16395 pending_rounds.cc:107] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 12:23:47.191957 16395 raft_consensus.cc:2889] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:47.176084 20770 raft_consensus.cc:697] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:23:47.215255 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 200012us
I20260709 12:23:47.179229 20677 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:47.221714 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:47.224861 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:47.162060 20675 consensus_queue.cc:260] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:47.227522 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:47.236404 16366 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:47.256451 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:47.266458 20529 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:47.224014 20669 raft_consensus.cc:697] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:47.249974 20677 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.255407 20790 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:23:47.284060 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 286827us
I20260709 12:23:47.260051 20667 consensus_queue.cc:1059] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
W20260709 12:23:47.309922 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 162993us
I20260709 12:23:47.284727 20790 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:47.317096 16441 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:47.342911 16515 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:23:47.354620 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.280550 20677 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:47.310408 20755 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:47.371784 16545 raft_consensus.cc:1240] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 13 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:47.372534 16515 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:23:47.372987 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 244355us
I20260709 12:23:47.379052 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:47.319443 20790 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:47.320127 20788 log.cc:927] Time spent T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 12:23:47.380997 16515 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:47.373306 17142 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:47.389977 17143 leader_election.cc:400] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 10. Current term is 11.
I20260709 12:23:47.284437 20770 consensus_queue.cc:237] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:47.399271 17143 leader_election.cc:403] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:23:47.372874 20708 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:23:47.399737 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 88809.5us
I20260709 12:23:47.400631 17143 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:47.394531 16515 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:47.373065 20762 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:47.400751 17142 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:47.400290 20708 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:47.401911 20762 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:47.387423 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 245073us
I20260709 12:23:47.402489 20762 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:47.403746 20715 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:47.402422 17142 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:47.407050 16515 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:47.436882 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:47.439087 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.389614 20712 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:47.383791 20755 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:47.376405 20773 consensus_queue.cc:1059] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 12:23:47.405237 20708 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:47.443635 16515 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:47.447222 17142 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:47.405275 20762 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:47.450095 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 191382us
I20260709 12:23:47.392010 20805 raft_consensus.cc:3055] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:47.450775 20805 raft_consensus.cc:740] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:47.451711 20805 consensus_queue.cc:260] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:47.452193 20773 raft_consensus.cc:3055] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:47.452718 20773 raft_consensus.cc:740] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:47.453064 20805 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:47.453547 20773 consensus_queue.cc:260] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:47.455629 20701 consensus_meta.cc:287] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.649s	user 0.006s	sys 0.005s
I20260709 12:23:47.456091 20701 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:47.458218 20701 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:47.458779 20701 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:47.459112 20701 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:23:47.463511 20732 consensus_meta.cc:287] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.639s	user 0.008s	sys 0.001s
I20260709 12:23:47.425910 20676 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:47.466353 20810 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:47.466782 20810 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:47.467167 20810 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:47.467738 20670 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:47.381978 20669 consensus_queue.cc:237] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:47.468799 20746 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:47.459465 20701 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:47.469574 20677 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:47.393658 20657 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:47.388870 20785 consensus_queue.cc:1059] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
W20260709 12:23:47.477600 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.474279 20773 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:47.475090 16366 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:47.463997 20732 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:47.484915 16399 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46250 (request call id 1226) took 1742 ms (client timeout 2144 ms). Trace:
I20260709 12:23:47.448899 20740 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:47.441064 20803 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 10. Current term is 11.
W20260709 12:23:47.486750 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 116489us
I20260709 12:23:47.485440 20715 consensus_queue.cc:1048] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:23:47.486730 16399 rpcz_store.cc:269] 0709 12:23:45.741087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.748440 (+  7353us) service_pool.cc:224] Handling call
0709 12:23:47.483366 (+1734926us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b54009bd7d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:47.483515 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":379105664}
I20260709 12:23:47.505590 20709 raft_consensus.cc:697] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:23:47.506704 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 59564.2us
W20260709 12:23:47.520506 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.525120 17142 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:47.526577 17142 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:47.527468 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 124660us
I20260709 12:23:47.533563 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:47.534372 16399 raft_consensus.cc:2368] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 10. Current term is 11.
I20260709 12:23:47.535723 17142 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:47.538699 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 138121us
I20260709 12:23:47.495249 20559 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:47.536935 16515 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:47.550920 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:47.551607 16475 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:23:47.546993 16515 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:47.544407 20790 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:47.564424 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:47.528007 20677 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:47.484141 20712 consensus_queue.cc:1059] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 12:23:47.576507 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:23:47.607120 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 290114us
I20260709 12:23:47.573479 16366 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:47.593765 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:47.603511 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:47.614303 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:47.615207 16551 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
W20260709 12:23:47.615658 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:47.621088 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 58690.9us
I20260709 12:23:47.623369 17144 leader_election.cc:400] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 10. Current term is 11.
I20260709 12:23:47.546435 20559 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:47.594770 20734 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:47.594652 20708 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:23:47.636247 20807 log.cc:927] Time spent T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:23:47.637293 20816 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:47.637820 20816 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:47.638163 20816 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:47.637202 20709 consensus_queue.cc:237] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:47.623143 20677 raft_consensus.cc:697] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:47.625900 20809 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:47.623960 20670 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:47.644934 20734 raft_consensus.cc:697] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:47.649186 20790 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:47.649101 20734 consensus_queue.cc:237] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:47.652848 16400 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 1192) took 1543 ms (client timeout 1740 ms). Trace:
W20260709 12:23:47.653294 16400 rpcz_store.cc:269] 0709 12:23:46.109877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:46.114381 (+  4504us) service_pool.cc:224] Handling call
0709 12:23:47.652643 (+1538262us) spinlock_profiling.cc:243] Waited 347 ms on lock 0x7b54001e2154. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:47.652791 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":797919488}
W20260709 12:23:47.653744 20704 consensus_meta.cc:287] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.577s	user 0.006s	sys 0.006s
I20260709 12:23:47.630034 20559 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:47.627743 20755 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:47.663030 20803 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:47.663908 16476 raft_consensus.cc:1275] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:23:47.680554 16544 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "f99066d13e524238aec6aa2024a8bd50"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:23:47.681181 16544 raft_consensus.cc:606] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Received request to transfer leadership to TS f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:47.681496 16544 raft_consensus.cc:614] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:23:47.686496 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 128207us
I20260709 12:23:47.687033 16516 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:47.646422 20833 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:47.688861 16441 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:47.655622 20670 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:23:47.690258 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 154603us
I20260709 12:23:47.689100 17263 raft_consensus.cc:1275] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:23:47.697487 20833 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:47.702021 20833 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:47.698679 16516 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:47.703590 20755 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:47.704064 20755 raft_consensus.cc:697] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:47.704882 20823 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:47.705376 20823 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:47.705519 20732 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:47.705823 20823 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:47.706370 20677 consensus_queue.cc:237] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:47.707252 16516 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:47.708425 20803 raft_consensus.cc:697] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 10 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:47.708521 20732 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:47.712348 20803 consensus_queue.cc:237] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:47.711195 16516 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:47.716763 17255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45228 (request call id 1133) took 1001 ms. Trace:
I20260709 12:23:47.717270 17255 rpcz_store.cc:276] 0709 12:23:46.715442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:46.716655 (+  1213us) service_pool.cc:224] Handling call
0709 12:23:47.716561 (+999906us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b54003f02d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:47.716689 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1219950848}
W20260709 12:23:47.705372 20663 consensus_meta.cc:287] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.800s	user 0.006s	sys 0.006s
I20260709 12:23:47.720840 16516 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:47.751005 16516 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:47.698252 20670 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:47.760819 20779 log.cc:927] Time spent T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 1.003s	user 0.008s	sys 0.004s
W20260709 12:23:47.761631 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.755501 20715 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:47.751504 20732 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:23:47.781841 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 158783us
I20260709 12:23:47.705727 20755 consensus_queue.cc:237] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:47.789968 16474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1386) took 1938 ms. Trace:
I20260709 12:23:47.788524 17142 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:47.781075 20732 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:47.756400 20663 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:47.755584 20701 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:47.801338 20701 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:47.796494 16367 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:47.787094 20828 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:47.804051 16474 rpcz_store.cc:276] 0709 12:23:45.851668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.851817 (+   149us) service_pool.cc:224] Handling call
0709 12:23:45.852132 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:45.852549 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:23:45.958795 (+106246us) log.cc:844] Serialized 14 byte log entry
0709 12:23:45.960638 (+  1843us) log.cc:844] Serialized 14 byte log entry
0709 12:23:47.560294 (+1599656us) spinlock_profiling.cc:243] Waited 36.4 ms on lock 0x7b1800209560. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d16c0a 00007f287c456219 00007f2875a3fd70 00007f2875a33293 00007f2875a28cea 00007f2875a2923a 00007f2875e93a2a 00007f2875ea30bf 00007f2875ea3038
0709 12:23:47.789506 (+229212us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:47.789570 (+    64us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:47.789636 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:47.789744 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:47.789898 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":84630,"prepare.queue_time_us":398467,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":82,"raft.queue_time_us":96371,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":15684,"spinlock_wait_cycles":83722368,"thread_start_us":320067,"threads_started":2}
I20260709 12:23:47.790602 20218 raft_consensus.cc:2749] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:23:47.815778 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:47.818608 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.820814 17142 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:47.821252 16474 raft_consensus.cc:1275] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:23:47.803568 20732 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:47.824177 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:23:47.825984 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 200289us
I20260709 12:23:47.811758 20701 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:47.832727 16395 raft_consensus.cc:1240] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:47.837970 16515 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:47.840237 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:23:47.843230 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 92328.1us
I20260709 12:23:47.847620 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:47.847262 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:47.848377 16515 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:47.844928 20796 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:47.842505 20842 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:47.851629 20842 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:47.851992 20842 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:47.855067 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:47.856443 20712 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:47.865532 16544 raft_consensus.cc:1275] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:23:47.866206 16516 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:47.868837 16515 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:47.874590 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:47.847579 20830 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:23:47.891681 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.868119 20535 consensus_queue.cc:1048] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.023s
W20260709 12:23:47.908876 20806 log.cc:927] Time spent T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.503s	user 0.002s	sys 0.000s
I20260709 12:23:47.884719 20821 consensus_queue.cc:1059] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 12:23:47.911403 20821 raft_consensus.cc:3055] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:47.911805 20821 raft_consensus.cc:740] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:47.912660 20821 consensus_queue.cc:260] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:47.913807 20821 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:47.875718 20828 consensus_queue.cc:1048] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:23:47.889257 20725 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:47.921404 20725 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:47.891295 20830 raft_consensus.cc:697] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:47.923167 20842 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:47.923950 20663 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:47.923715 20830 consensus_queue.cc:237] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:47.923255 20715 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:47.930474 20675 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:47.903144 20712 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:47.900357 20848 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:47.927948 20823 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:47.944607 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:47.946731 16548 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:47.948597 20823 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:47.949334 20810 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:47.950652 17258 raft_consensus.cc:2393] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 12: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:47.952637 16550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37226 (request call id 1505) took 1397 ms. Trace:
I20260709 12:23:47.953231 17264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 1107) took 1280 ms. Trace:
I20260709 12:23:47.953619 17264 rpcz_store.cc:276] 0709 12:23:46.672925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:46.673199 (+   274us) service_pool.cc:224] Handling call
0709 12:23:46.673619 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:47.180484 (+506865us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:47.952987 (+772503us) spinlock_profiling.cc:243] Waited 505 ms on lock 0x7b54002800d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:47.953175 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1161442560}
I20260709 12:23:47.955829 20810 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:47.953052 16550 rpcz_store.cc:276] 0709 12:23:46.555436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:46.741301 (+185865us) service_pool.cc:224] Handling call
0709 12:23:47.952461 (+1211160us) spinlock_profiling.cc:243] Waited 329 ms on lock 0x7b5400b13354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:47.952573 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":757118720}
I20260709 12:23:47.958763 20559 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:47.959838 20716 raft_consensus.cc:3055] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:47.960798 20559 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:47.957683 16548 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
I20260709 12:23:47.964766 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:47.971630 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:47.960292 20716 raft_consensus.cc:740] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:47.981400 20716 consensus_queue.cc:260] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:47.982940 20716 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:47.984213 20816 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:47.987092 20816 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:47.991636 20798 log.cc:927] Time spent T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.621s	user 0.000s	sys 0.013s
I20260709 12:23:47.993675 16441 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:47.999548 20535 consensus_queue.cc:1048] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:47.946352 20848 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:48.001514 20848 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:48.001554 20535 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:48.002259 20732 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:48.003592 20657 raft_consensus.cc:3055] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:48.004148 20657 raft_consensus.cc:740] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
W20260709 12:23:48.004611 20856 log.cc:927] Time spent T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:23:48.005299 20657 consensus_queue.cc:260] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:48.007344 20657 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:47.965408 20842 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:48.010447 20670 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:48.010792 16472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34826 (request call id 1451) took 1508 ms. Trace:
I20260709 12:23:48.011526 16472 rpcz_store.cc:276] 0709 12:23:46.502178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:46.502705 (+   527us) service_pool.cc:224] Handling call
0709 12:23:48.010547 (+1507842us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b54002f2e54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:48.010708 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":273910528}
I20260709 12:23:48.102138 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:48.111310 16542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 835) took 1878 ms. Trace:
I20260709 12:23:48.111816 16542 rpcz_store.cc:276] 0709 12:23:46.232969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:46.494415 (+261446us) service_pool.cc:224] Handling call
0709 12:23:46.494672 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:47.141402 (+646730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:48.111049 (+969647us) spinlock_profiling.cc:243] Waited 634 ms on lock 0x7b5400a80cd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:48.111238 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1458511616}
I20260709 12:23:48.114290 20763 raft_consensus.cc:3055] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:48.114779 20763 raft_consensus.cc:740] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:48.116345 20732 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:48.115741 20763 consensus_queue.cc:260] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:48.117980 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:48.118574 16469 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:47.963420 20833 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:48.122588 20833 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:48.125643 16442 leader_election.cc:336] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.526s (SENT)
I20260709 12:23:48.126950 16469 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
W20260709 12:23:48.127522 16442 leader_election.cc:336] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.631s (SENT)
I20260709 12:23:48.129901 16542 raft_consensus.cc:1275] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:23:48.128211 16442 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:48.081342 20535 raft_consensus.cc:697] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:48.128700 16514 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:48.142350 17269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45228 (request call id 1128) took 1457 ms. Trace:
I20260709 12:23:48.160583 17269 rpcz_store.cc:276] 0709 12:23:46.684830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:46.694436 (+  9606us) service_pool.cc:224] Handling call
0709 12:23:48.142097 (+1447661us) spinlock_profiling.cc:243] Waited 355 ms on lock 0x7b5400a74c54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:48.142263 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":815780608}
I20260709 12:23:48.159571 16477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1460) took 1141 ms. Trace:
I20260709 12:23:48.162176 16477 rpcz_store.cc:276] 0709 12:23:47.018564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:47.018679 (+   115us) service_pool.cc:224] Handling call
0709 12:23:47.121699 (+103020us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:47.122390 (+   691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:48.159319 (+1036929us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b54004100d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:48.159499 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":236179200}
W20260709 12:23:48.135315 20798 log.cc:927] Time spent T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:23:48.174134 20267 raft_consensus.cc:3055] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:48.174728 20267 raft_consensus.cc:740] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:48.114326 20670 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:48.181020 16476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 820) took 1129 ms. Trace:
I20260709 12:23:48.181660 16476 rpcz_store.cc:276] 0709 12:23:47.052117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:47.053167 (+  1050us) service_pool.cc:224] Handling call
0709 12:23:47.053485 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:47.666970 (+613485us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:48.180745 (+513775us) spinlock_profiling.cc:243] Waited 601 ms on lock 0x7b540012b954. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:48.180958 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1383381888}
I20260709 12:23:48.191015 20535 consensus_queue.cc:237] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:48.201004 20844 log.cc:927] Time spent T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:23:48.209244 20763 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:48.145068 20334 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:48.175712 20267 consensus_queue.cc:260] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:48.142982 20600 consensus_queue.cc:1048] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 12:23:48.181622 20732 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:48.157508 20773 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:23:48.213227 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:48.211859 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 75305.2us
I20260709 12:23:48.218384 16545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1507) took 1662 ms. Trace:
I20260709 12:23:48.225922 16545 rpcz_store.cc:276] 0709 12:23:46.555879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:46.812716 (+256837us) service_pool.cc:224] Handling call
0709 12:23:46.812999 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:47.377409 (+564410us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:48.218091 (+840682us) spinlock_profiling.cc:243] Waited 558 ms on lock 0x7b54009745d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:48.218316 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1284544768}
W20260709 12:23:48.227679 16442 leader_election.cc:336] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.745s (SENT)
I20260709 12:23:48.218268 20267 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:48.219561 20621 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:48.220271 20732 raft_consensus.cc:697] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 13 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:48.217443 20334 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:48.236594 20773 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:48.240219 20621 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:48.278980 20600 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:48.286595 20773 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:48.294785 20732 consensus_queue.cc:237] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:48.293407 20600 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:48.325788 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 529346us
I20260709 12:23:48.325923 20600 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:48.326184 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 498093us
I20260709 12:23:48.326933 16401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 1080) took 2351 ms. Trace:
I20260709 12:23:48.328262 20621 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:48.328768 20669 consensus_queue.cc:1048] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:48.329823 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:48.330992 16367 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:23:48.331336 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.332109 16367 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:48.327389 16401 rpcz_store.cc:276] 0709 12:23:45.975272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:46.291849 (+316577us) service_pool.cc:224] Handling call
0709 12:23:47.483533 (+1191684us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:47.483808 (+   275us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:23:47.692549 (+208741us) log.cc:844] Serialized 14 byte log entry
0709 12:23:47.803864 (+111315us) log.cc:844] Serialized 14 byte log entry
0709 12:23:48.325072 (+521208us) spinlock_profiling.cc:243] Waited 3.08 ms on lock 0x7b18002a2740. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d16c0a 00007f287c456219 00007f2872d9ea40 00007f2872d994e4 00007f2872d99455 00007f2875e81060 00007f2875e93716 00007f2875ea4287 00007f2875ea4208
0709 12:23:48.325241 (+   169us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:48.325293 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:48.325352 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:48.325425 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:48.326748 (+  1323us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54009bd7d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:48.326869 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3111,"prepare.queue_time_us":235957,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":2187,"raft.queue_time_us":521580,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":3159,"spinlock_wait_cycles":2635729024,"thread_start_us":242134,"threads_started":2,"wal-append.queue_time_us":239907}
I20260709 12:23:48.332876 20819 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:48.333521 20819 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.333936 20819 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:48.335175 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:48.330864 20675 consensus_queue.cc:1059] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:48.336010 16395 raft_consensus.cc:1240] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:48.336953 16399 raft_consensus.cc:1240] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:48.340960 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:48.336826 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:48.337179 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:48.342857 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:48.343139 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:48.337749 16401 raft_consensus.cc:1275] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:23:48.343748 16398 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:48.344410 16397 raft_consensus.cc:1240] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:48.345822 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:48.346692 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:48.347280 16551 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:48.294220 20848 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:48.336863 20857 log.cc:927] Time spent T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:23:48.325961 20800 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:48.353844 16367 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:48.355597 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:48.357232 16394 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:23:48.361493 16548 raft_consensus.cc:1240] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 13. Current term is 14. Ops: []
W20260709 12:23:48.361652 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:48.362218 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.367231 16398 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
I20260709 12:23:48.368739 16367 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:48.370095 16551 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
W20260709 12:23:48.371049 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.356806 20669 raft_consensus.cc:2749] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:48.382335 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:48.382983 16551 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:48.383687 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:48.384229 16547 raft_consensus.cc:1240] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:48.384981 16550 raft_consensus.cc:1240] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:48.384267 16544 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:48.388942 16551 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
I20260709 12:23:48.390215 16542 raft_consensus.cc:1240] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:48.389174 17144 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:48.374696 20716 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.397078 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:48.373523 20816 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:48.400167 16545 raft_consensus.cc:1275] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:23:48.388049 20535 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.413653 16398 raft_consensus.cc:1275] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:23:48.391988 20800 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:48.377835 20670 consensus_queue.cc:1059] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
W20260709 12:23:48.425119 20866 log.cc:927] Time spent T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 12:23:48.379463 20208 consensus_queue.cc:1059] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:23:48.427459 16544 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:23:48.391563 20621 consensus_queue.cc:1048] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 12:23:48.406628 20825 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:48.405228 20559 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:48.430603 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.408078 20810 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:48.408010 20663 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:48.421639 20819 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:48.406255 20716 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:48.419232 20758 consensus_queue.cc:1059] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.023s
W20260709 12:23:48.465723 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 97067.6us
I20260709 12:23:48.435635 20621 consensus_queue.cc:1243] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer f99066d13e524238aec6aa2024a8bd50 log is divergent from this leader: its last log entry 13.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:23:48.467255 16367 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:48.435456 20825 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:48.471130 16367 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:48.422091 20535 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:48.478117 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.463920 20773 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:48.441009 20803 consensus_queue.cc:1048] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:23:48.468742 20850 raft_consensus.cc:3055] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:48.441283 20559 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:48.506971 16367 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:48.441243 20810 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:48.501868 20437 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:48.489039 20825 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:48.511970 16367 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:48.491570 20763 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:48.514927 16367 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:23:48.513494 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 83158.3us
I20260709 12:23:48.518042 16367 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:48.516784 17143 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:48.522856 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 74100.2us
I20260709 12:23:48.506632 20850 raft_consensus.cc:740] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:48.525494 17143 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:48.530167 16442 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:48.513875 20848 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:48.517709 20373 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:48.517400 20677 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:48.519104 20838 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:48.523120 20816 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:48.449820 20663 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:48.524588 20809 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:48.540505 20535 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:48.540633 20600 consensus_queue.cc:1048] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:48.520277 20763 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:48.544691 20836 raft_consensus.cc:3055] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:48.545153 20836 raft_consensus.cc:740] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:48.546165 20763 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:48.545918 20836 consensus_queue.cc:260] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:48.547266 20836 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:48.530813 20670 consensus_queue.cc:1059] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:23:48.536180 20773 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:48.549703 20621 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:48.549976 20716 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:48.551957 20559 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:48.557761 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:48.558485 16398 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
I20260709 12:23:48.560220 16398 raft_consensus.cc:1240] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:23:48.567431 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:48.562225 20787 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:48.568509 16394 raft_consensus.cc:2393] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 14: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:48.569834 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:48.562441 20850 consensus_queue.cc:260] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:48.568495 20809 raft_consensus.cc:697] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:48.573658 20850 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:48.573858 20809 consensus_queue.cc:237] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:48.573657 20621 raft_consensus.cc:697] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:48.562628 20208 consensus_queue.cc:1059] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 12:23:48.576151 20621 consensus_queue.cc:237] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:48.577018 20208 raft_consensus.cc:3055] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:48.577364 20208 raft_consensus.cc:740] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:48.578140 20208 consensus_queue.cc:260] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:48.573035 20670 raft_consensus.cc:3055] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:48.579342 20208 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:48.579879 20670 raft_consensus.cc:740] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:48.580555 20787 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:48.580757 20670 consensus_queue.cc:260] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:48.581727 20670 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:48.582687 20787 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:48.567094 20657 consensus_meta.cc:287] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.559s	user 0.008s	sys 0.004s
I20260709 12:23:48.583420 20663 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:48.569321 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:48.588256 20773 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:48.590447 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:48.590613 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:48.593187 17143 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:48.594211 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:48.594800 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:48.595403 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:48.588917 20773 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:48.598685 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:48.600502 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:48.602140 17264 raft_consensus.cc:1240] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:48.602635 17255 raft_consensus.cc:1240] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 10 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:48.602650 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:48.601524 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:48.604449 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:48.605428 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:48.606514 17264 raft_consensus.cc:1240] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:48.607270 16398 raft_consensus.cc:2393] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 11: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:48.608513 16545 raft_consensus.cc:2393] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 10: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:48.611770 20470 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:48.612357 20470 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:48.614198 20716 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:23:48.615008 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.614893 20716 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:48.620172 20773 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:48.620903 20716 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:48.621524 17263 raft_consensus.cc:1240] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 13. Current term is 14. Ops: [11.6-13.7]
W20260709 12:23:48.623159 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.625191 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:48.626384 16477 raft_consensus.cc:2410] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:23:48.626833 16472 raft_consensus.cc:1275] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:23:48.627820 16401 raft_consensus.cc:1240] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:48.622510 20825 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:48.604640 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:48.628311 16400 raft_consensus.cc:1217] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Deduplicated request from leader. Original: 8.4->[10.5-14.7]   Dedup: 10.5->[12.6-14.7]
I20260709 12:23:48.630268 17271 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:48.647483 16398 raft_consensus.cc:1240] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:48.649026 16400 pending_rounds.cc:85] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 5
I20260709 12:23:48.649350 16400 pending_rounds.cc:107] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 13.6
I20260709 12:23:48.649595 16400 raft_consensus.cc:2889] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:48.650974 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:48.651875 16477 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
I20260709 12:23:48.653873 16396 raft_consensus.cc:1275] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:48.655252 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:48.631454 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:48.657686 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:48.658159 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:48.666447 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:48.614802 20470 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:48.597956 20850 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:48.670547 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:48.671373 16403 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:48.672763 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:48.673396 16477 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:48.687404 17261 raft_consensus.cc:1240] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:48.689396 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:48.690127 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:48.691087 16472 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 8.
I20260709 12:23:48.700824 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:48.701602 16549 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14.
I20260709 12:23:48.703261 20845 consensus_queue.cc:1059] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:48.704401 20845 raft_consensus.cc:3055] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:48.704720 20845 raft_consensus.cc:740] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:48.705500 20845 consensus_queue.cc:260] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:48.597259 20836 consensus_queue.cc:1048] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 12:23:48.711269 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:23:48.712206 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 55491.2us
I20260709 12:23:48.712996 20600 consensus_queue.cc:1059] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:48.713821 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:48.714084 20470 consensus_queue.cc:1048] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:48.714926 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:48.630770 20825 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:48.718067 16514 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:48.715982 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:48.719916 17264 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:23:48.720247 16468 raft_consensus.cc:1275] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:23:48.688884 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:48.594162 20663 consensus_queue.cc:1059] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
W20260709 12:23:48.665200 20867 log.cc:927] Time spent T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 12:23:48.725932 17264 raft_consensus.cc:1240] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:48.729444 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:48.734309 16397 raft_consensus.cc:2393] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 11: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:48.737120 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:48.742965 17261 raft_consensus.cc:1240] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:48.743970 16515 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:48.745184 16515 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:48.736116 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:48.747213 16477 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14.
I20260709 12:23:48.731143 16403 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
I20260709 12:23:48.751549 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:48.753968 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:48.761193 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:48.768111 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:48.769376 16543 raft_consensus.cc:2410] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:23:48.713835 20600 raft_consensus.cc:3055] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:48.772538 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:48.773288 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:48.774142 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:48.780277 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:48.782229 16469 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:23:48.776695 16547 raft_consensus.cc:2393] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 14: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:48.725674 20762 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.790810 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:48.783531 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:48.793098 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:48.794209 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:48.795324 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:48.795949 16403 raft_consensus.cc:2393] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 12: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:48.796572 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:48.796810 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:48.797749 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:48.797888 17261 raft_consensus.cc:1240] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:48.798794 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:48.769785 20708 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:48.800800 20725 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:48.724937 20850 raft_consensus.cc:697] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:48.793154 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:48.803809 16398 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
W20260709 12:23:48.805862 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 60729us
I20260709 12:23:48.809098 16397 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:23:48.810251 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:48.735841 20746 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:48.804100 17144 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:48.816318 16398 raft_consensus.cc:1275] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:23:48.821189 16403 raft_consensus.cc:3055] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:48.823019 16403 raft_consensus.cc:740] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:48.816565 16442 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:48.824901 16442 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:48.826045 17263 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:23:48.824072 16403 consensus_queue.cc:260] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:48.719424 20851 consensus_queue.cc:1059] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 12:23:48.687300 20870 consensus_queue.cc:1059] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.123s
I20260709 12:23:48.820940 20470 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:48.827152 20773 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:48.847690 16551 raft_consensus.cc:1240] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:48.849571 16551 raft_consensus.cc:1240] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:23:48.780676 20600 raft_consensus.cc:740] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:48.827193 20768 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:48.874362 17261 raft_consensus.cc:1240] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:48.813493 20796 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.877064 20796 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:48.879643 20796 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:48.830332 17263 raft_consensus.cc:1240] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:48.829638 16515 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:48.879297 20871 consensus_queue.cc:1048] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:48.880649 16223 catalog_manager.cc:5697] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 3 to 8. New cstate: current_term: 8 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:48.881464 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:48.882652 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:48.884938 20600 consensus_queue.cc:260] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:48.885970 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.886109 20600 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:48.889034 20875 log.cc:927] Time spent T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 12:23:48.900774 20851 raft_consensus.cc:3055] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:48.901438 20851 raft_consensus.cc:740] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:23:48.902640 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.902468 20851 consensus_queue.cc:260] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:48.905589 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:48.906803 16550 raft_consensus.cc:2393] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 15: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:48.816532 20762 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:48.822844 20746 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:48.908386 20746 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:48.912298 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 82744.6us
I20260709 12:23:48.914819 20708 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:48.915505 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:48.924288 16547 raft_consensus.cc:1275] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:48.883435 16223 catalog_manager.cc:5697] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:48.927412 16549 raft_consensus.cc:1240] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:48.923976 20470 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.835417 20830 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:23:48.950012 17142 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:48.900887 20670 raft_consensus.cc:3055] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:48.826525 16442 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:48.959860 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 133446us
I20260709 12:23:48.961133 16442 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:48.973137 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:48.957001 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.937726 20851 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:48.931162 20715 consensus_queue.cc:1175] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "52ab50b0237346c1a13ede947f6a63c2" responder_term: 13 status { last_received { term: 9 index: 7 } last_committed_idx: 7 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 10. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:23:48.961547 16366 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:49.012629 17142 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:48.930222 20470 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:48.958220 20670 raft_consensus.cc:740] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:23:49.026997 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.959283 20830 raft_consensus.cc:697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:48.958622 20437 consensus_queue.cc:1048] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:23:49.027706 17271 raft_consensus.cc:1240] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 13. Current term is 14. Ops: []
W20260709 12:23:49.025055 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 50389.3us
I20260709 12:23:48.965397 20763 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:23:49.031252 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 67770us
W20260709 12:23:49.029340 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 68311.1us
I20260709 12:23:48.986730 20758 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:23:49.036765 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 75312.6us
I20260709 12:23:49.040530 16442 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:49.041075 17142 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:49.023692 20850 consensus_queue.cc:237] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:49.050673 20830 consensus_queue.cc:237] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:49.054764 20763 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:49.056491 20706 consensus_queue.cc:1048] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:49.059456 20529 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:49.066335 17271 raft_consensus.cc:1240] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:49.069136 17142 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:49.060086 20529 raft_consensus.cc:697] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 14 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:49.073063 17142 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:49.078090 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:49.078524 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:49.035354 20768 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:49.080647 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:49.034752 20809 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:49.081391 17266 raft_consensus.cc:2376] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11. Re-sending same reply.
I20260709 12:23:49.085012 16515 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:49.082305 17270 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6474b10917204b6994269be17387f38d"
dest_uuid: "ef57beab0e1d40d585983873c55107e8"
mode: GRACEFUL
new_leader_uuid: "610f4feda11942739236f2b121a4b9c1"
 from {username='slave'} at 127.0.0.1:45120
I20260709 12:23:49.086253 17270 raft_consensus.cc:606] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Received request to transfer leadership to TS 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:49.082783 17254 raft_consensus.cc:1275] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:23:49.064728 20663 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:49.090205 17266 raft_consensus.cc:1275] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:49.094583 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:49.082058 20829 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:49.086462 20836 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:49.090274 20708 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:49.121892 17271 raft_consensus.cc:1240] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:49.081085 20825 consensus_queue.cc:1048] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:23:49.125478 17266 raft_consensus.cc:1275] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:23:49.128993 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:49.131278 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:49.139621 16367 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:49.141217 17263 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
I20260709 12:23:49.149910 16549 raft_consensus.cc:1275] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:49.118757 20836 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:49.155036 16367 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:49.157928 16547 raft_consensus.cc:1275] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:23:49.180667 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:49.181917 17263 raft_consensus.cc:2393] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 14: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:49.185653 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:49.188568 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:49.192912 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:49.193938 17263 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
I20260709 12:23:49.195067 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:49.242218 17266 raft_consensus.cc:1275] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:23:49.267498 17254 raft_consensus.cc:2393] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 12: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:49.268524 17263 raft_consensus.cc:1275] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:49.273643 20470 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:49.159796 20805 consensus_queue.cc:1059] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.115s
I20260709 12:23:49.151279 20529 consensus_queue.cc:237] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:49.175011 20833 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:49.276360 20805 raft_consensus.cc:3055] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:49.276947 20805 raft_consensus.cc:740] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:49.276832 20833 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:49.278975 20833 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:49.150971 20768 consensus_queue.cc:1059] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.133s
I20260709 12:23:49.280048 20768 raft_consensus.cc:3055] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:23:49.280409 20768 raft_consensus.cc:740] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:49.158140 20836 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:49.281198 20768 consensus_queue.cc:260] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:49.282413 20768 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:49.158926 20816 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:49.104128 20670 consensus_queue.cc:260] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:49.148670 20677 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:23:49.287503 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 76248.3us
I20260709 12:23:49.350812 16441 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:49.353173 16441 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:49.367750 16441 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:49.372920 16441 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:49.244760 20708 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:49.159457 16367 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
W20260709 12:23:49.378671 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 219271us
I20260709 12:23:49.299602 20720 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:49.398010 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:49.398967 16549 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
I20260709 12:23:49.369364 20706 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:49.380012 20877 log.cc:927] Time spent T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 12:23:49.375450 20535 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:49.379842 20600 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:49.417495 20600 consensus_queue.cc:1048] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:23:49.419466 20746 consensus_meta.cc:287] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.510s	user 0.007s	sys 0.003s
I20260709 12:23:49.419905 20746 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:49.380719 20816 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:49.426718 20746 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:49.394243 20708 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:49.403193 20720 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:49.416215 20677 consensus_queue.cc:1059] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
W20260709 12:23:49.450241 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:49.432804 20816 raft_consensus.cc:697] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
W20260709 12:23:49.455508 16546 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.002s	sys 0.017s
W20260709 12:23:49.455919 16546 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.002s	sys 0.017s
W20260709 12:23:49.456938 16399 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.042s	sys 0.054s
W20260709 12:23:49.457494 16399 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.042s	sys 0.054s
I20260709 12:23:49.277933 20805 consensus_queue.cc:260] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:49.475674 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 435159us
W20260709 12:23:49.477128 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 427493us
I20260709 12:23:49.485015 16442 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:49.550343 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 65423.3us
I20260709 12:23:49.551400 16442 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:49.553187 17255 raft_consensus.cc:1240] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:49.453758 20667 raft_consensus.cc:3055] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:49.563133 16442 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:49.554359 20763 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:49.549516 20746 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:49.548480 20881 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:49.565908 20667 raft_consensus.cc:740] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:49.574582 20850 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:49.592398 20810 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:49.596117 20850 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:49.598400 20816 consensus_queue.cc:237] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:49.604542 20821 raft_consensus.cc:3055] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:49.605012 20821 raft_consensus.cc:740] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:49.605782 20821 consensus_queue.cc:260] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:49.606346 20851 consensus_meta.cc:287] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.571s	user 0.007s	sys 0.002s
I20260709 12:23:49.607220 20821 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:49.607743 20708 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:49.610970 20773 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:49.611678 20773 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:49.613619 20805 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:49.617548 20905 raft_consensus.cc:3055] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:49.618093 20905 raft_consensus.cc:740] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:49.618993 20905 consensus_queue.cc:260] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:49.620898 20839 consensus_queue.cc:1059] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:23:49.623930 20675 raft_consensus.cc:3055] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:49.624392 20675 raft_consensus.cc:740] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:49.624985 16549 raft_consensus.cc:1275] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:23:49.626760 20720 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:49.628818 16398 raft_consensus.cc:3055] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:49.629200 16398 raft_consensus.cc:740] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:49.629884 16398 consensus_queue.cc:260] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:49.630894 16403 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:49.631557 20879 log.cc:927] Time spent T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
I20260709 12:23:49.632180 16395 raft_consensus.cc:3055] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:49.632647 16395 raft_consensus.cc:740] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:49.633136 20850 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:49.634367 16396 raft_consensus.cc:3055] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:49.634807 16396 raft_consensus.cc:740] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:49.635529 16396 consensus_queue.cc:260] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:49.633338 16395 consensus_queue.cc:260] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:49.633843 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:49.633450 17269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 1122) took 1038 ms. Trace:
I20260709 12:23:49.639708 16396 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:49.642650 17269 rpcz_store.cc:276] 0709 12:23:48.594772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.594926 (+   154us) service_pool.cc:224] Handling call
0709 12:23:48.595252 (+   326us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:48.595672 (+   420us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:48.595703 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:48.598451 (+  2748us) log.cc:844] Serialized 52 byte log entry
0709 12:23:49.014475 (+416024us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:49.014532 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:49.014630 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:49.633234 (+618604us) raft_consensus.cc:1685] finished
0709 12:23:49.633252 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:49.633395 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1563,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":118,"thread_start_us":326401,"threads_started":1,"wal-append.queue_time_us":589655}
I20260709 12:23:49.648622 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:49.649972 16516 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:49.655088 16394 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.027s	sys 0.044s
W20260709 12:23:49.655536 16394 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.028s	sys 0.045s
I20260709 12:23:49.582996 20535 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:23:49.658774 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:49.658980 16395 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:49.662448 16403 raft_consensus.cc:1275] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 12:23:49.663130 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:49.663064 16403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 1256) took 1009 ms. Trace:
W20260709 12:23:49.636292 20208 consensus_meta.cc:287] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 1.056s	user 0.007s	sys 0.003s
I20260709 12:23:49.664484 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:23:49.636809 20884 log.cc:927] Time spent T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:23:49.665531 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:49.645702 20871 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:49.668737 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:49.668390 16403 rpcz_store.cc:276] 0709 12:23:48.653645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.802860 (+149215us) service_pool.cc:224] Handling call
0709 12:23:48.803168 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:48.894544 (+ 91376us) spinlock_profiling.cc:243] Waited 65.1 ms on lock 0x7b18004d9ca0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875ec4646 00007f2875dfe58e 00007f2875e7ff9e 00007f2875e817de 00007f2875e8bb49 00007f2875e8c416 00007f2875e88d8f 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4
0709 12:23:49.630737 (+736193us) spinlock_profiling.cc:243] Waited 735 ms on lock 0x7b58000e48d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875de50aa 00007f2875e67447 00007f2875e7ffe7 00007f2875e817de 00007f2875e8bb49 00007f2875e8c416 00007f2875e88d8f 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4
0709 12:23:49.662822 (+ 32085us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:49.663013 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1841069184}
W20260709 12:23:49.661073 20880 log.cc:927] Time spent T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.316s	user 0.000s	sys 0.001s
I20260709 12:23:49.676648 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:49.693396 17258 raft_consensus.cc:2410] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:23:49.664359 20905 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:49.661908 20708 raft_consensus.cc:2749] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:49.704118 16550 raft_consensus.cc:1240] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: []
W20260709 12:23:49.709774 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 335299us
I20260709 12:23:49.647435 20677 consensus_queue.cc:1059] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 12:23:49.642053 20746 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:49.638863 20881 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:49.717960 16395 raft_consensus.cc:1275] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:23:49.722568 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 246994us
I20260709 12:23:49.724665 16367 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:49.725255 16549 raft_consensus.cc:1240] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: []
W20260709 12:23:49.725785 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:49.667059 20836 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:49.729326 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 207652us
I20260709 12:23:49.647666 20675 consensus_queue.cc:260] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:49.702733 20892 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:49.737608 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:49.637310 20667 consensus_queue.cc:260] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:49.741218 16550 raft_consensus.cc:1275] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:23:49.720044 20773 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:49.721482 20706 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:49.744133 16397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 1261) took 1026 ms. Trace:
I20260709 12:23:49.745668 16515 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:49.748030 16397 rpcz_store.cc:276] 0709 12:23:48.718035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.812304 (+ 94269us) service_pool.cc:224] Handling call
0709 12:23:48.813169 (+   865us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:48.813474 (+   305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:48.813501 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:48.875869 (+ 62368us) log.cc:844] Serialized 26 byte log entry
0709 12:23:49.153220 (+277351us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:49.153259 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:49.153319 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:49.743930 (+590611us) raft_consensus.cc:1685] finished
0709 12:23:49.743952 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:49.744074 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1243,"spinlock_wait_cycles":7664896,"thread_start_us":276995,"threads_started":1,"wal-append.queue_time_us":573115}
I20260709 12:23:49.755771 16544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1338) took 1145 ms. Trace:
I20260709 12:23:49.704984 20871 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:49.717545 20535 consensus_queue.cc:1048] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:23:49.756384 16544 rpcz_store.cc:276] 0709 12:23:48.610156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.637447 (+ 27291us) service_pool.cc:224] Handling call
0709 12:23:48.637782 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:48.638131 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:48.638152 (+    21us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:48.669279 (+ 31127us) log.cc:844] Serialized 26 byte log entry
0709 12:23:48.963051 (+293772us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:48.963094 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:48.963162 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:49.755461 (+792299us) raft_consensus.cc:1685] finished
0709 12:23:49.755493 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:49.755689 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15106,"raft.queue_time_us":18141,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":10794,"thread_start_us":256126,"threads_started":1,"wal-append.queue_time_us":432204}
I20260709 12:23:49.760432 20675 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:49.764581 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:49.765347 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:49.771968 16399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 1245) took 1429 ms. Trace:
I20260709 12:23:49.753319 20667 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:49.733789 20810 consensus_queue.cc:1048] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:23:49.779079 16542 raft_consensus.cc:1275] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:49.779738 16542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1339) took 1169 ms. Trace:
I20260709 12:23:49.780110 16542 rpcz_store.cc:276] 0709 12:23:48.610440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.611937 (+  1497us) service_pool.cc:224] Handling call
0709 12:23:49.778532 (+1166595us) spinlock_profiling.cc:243] Waited 25 ms on lock 0x7b58005e9298. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:23:49.778733 (+   201us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:49.779489 (+   756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:49.779681 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57389824}
I20260709 12:23:49.770006 20823 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:49.743726 20892 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:49.769634 20787 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:49.773589 16399 rpcz_store.cc:276] 0709 12:23:48.342262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.343410 (+  1148us) service_pool.cc:224] Handling call
0709 12:23:48.343648 (+   238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:48.343824 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:23:48.352981 (+  9157us) log.cc:844] Serialized 14 byte log entry
0709 12:23:48.914897 (+561916us) spinlock_profiling.cc:243] Waited 2 ms on lock 0x7b180019b760. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a2923a 00007f2875e93a2a
0709 12:23:49.771530 (+856633us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:49.771606 (+    76us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:49.771663 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:49.771753 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:49.771898 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":9048,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":153146,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"spinlock_wait_cycles":4608896,"thread_start_us":848619,"threads_started":2,"wal-append.queue_time_us":1310605}
I20260709 12:23:49.789798 16542 raft_consensus.cc:1275] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:23:49.793871 20907 consensus_queue.cc:1048] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:49.796546 20871 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:49.796940 20896 log.cc:927] Time spent T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
W20260709 12:23:49.798024 20894 log.cc:927] Time spent T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 12:23:49.798542 17271 raft_consensus.cc:1275] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:49.774313 20851 consensus_queue.cc:1048] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
W20260709 12:23:49.802518 20895 log.cc:927] Time spent T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:23:49.805225 20845 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:49.805419 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:49.806531 16544 raft_consensus.cc:1240] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:49.773605 20868 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:49.810185 16396 raft_consensus.cc:1275] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:23:49.820318 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:49.823206 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:49.823648 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:49.824384 16367 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:49.781632 20334 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:49.784453 20908 consensus_queue.cc:1059] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 12:23:49.827888 20816 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:49.826238 20706 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:49.819407 20834 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:49.821535 20787 raft_consensus.cc:697] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
W20260709 12:23:49.847970 20887 log.cc:927] Time spent T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 12:23:49.855275 20740 consensus_queue.cc:1059] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:49.847198 20706 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:49.863116 20732 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:49.863703 20732 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:49.864832 20706 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:49.865481 20706 consensus_queue.cc:1048] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:49.866035 20732 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:49.832895 20727 consensus_queue.cc:1048] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:23:49.871623 16474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1408) took 1199 ms. Trace:
I20260709 12:23:49.790081 20848 consensus_queue.cc:1048] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 12:23:49.855909 16367 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:49.859447 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:49.834506 20816 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:49.883239 17254 raft_consensus.cc:2376] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14. Re-sending same reply.
I20260709 12:23:49.872102 16474 rpcz_store.cc:276] 0709 12:23:48.672055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.674338 (+  2283us) service_pool.cc:224] Handling call
0709 12:23:48.674799 (+   461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:48.694418 (+ 19619us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:48.694470 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:49.599950 (+905480us) spinlock_profiling.cc:243] Waited 24.5 ms on lock 0x7b2c0017a228. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:23:49.600419 (+   469us) log.cc:844] Serialized 26 byte log entry
0709 12:23:49.736428 (+136009us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:49.736492 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:49.736569 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:49.871284 (+134715us) raft_consensus.cc:1685] finished
0709 12:23:49.871308 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:49.871557 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19018,"raft.queue_time_us":37889,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":99,"spinlock_wait_cycles":56414976,"thread_start_us":133840,"threads_started":1,"wal-append.queue_time_us":232424}
I20260709 12:23:49.872511 20834 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:49.875260 16514 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:49.873489 20904 log.cc:927] Time spent T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:23:49.897185 20746 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:49.897636 20823 raft_consensus.cc:2804] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:49.898092 20787 consensus_queue.cc:237] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:49.899945 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:49.900499 20892 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:49.885749 20816 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:49.901206 20903 log.cc:927] Time spent T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:23:49.902740 20834 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:49.905161 20740 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:49.907387 20437 raft_consensus.cc:3055] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:49.912088 20763 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:49.912503 20881 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:49.912168 20836 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:49.914177 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:49.930245 17254 raft_consensus.cc:2393] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 14: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:49.923403 20823 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:49.931375 20763 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:49.928256 20437 raft_consensus.cc:740] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:49.928714 20740 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:49.941133 20823 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:50.017976 16400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 1120) took 1392 ms. Trace:
I20260709 12:23:50.021435 20740 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:50.037367 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:50.034757 16400 rpcz_store.cc:276] 0709 12:23:48.624859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.627725 (+  2866us) service_pool.cc:224] Handling call
0709 12:23:48.628108 (+   383us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:23:48.650275 (+ 22167us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:23:48.650309 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:48.653030 (+  2721us) log.cc:844] Serialized 52 byte log entry
0709 12:23:49.598684 (+945654us) spinlock_profiling.cc:243] Waited 7.08 ms on lock 0x7b1800453220. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:23:49.740901 (+142217us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:49.740939 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:49.741009 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:50.016719 (+275710us) raft_consensus.cc:1685] finished
0709 12:23:50.016802 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:50.017605 (+   803us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2456,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":10697,"spinlock_wait_cycles":16283520,"thread_start_us":142025,"threads_started":1,"wal-append.queue_time_us":1132842}
I20260709 12:23:50.029995 16401 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
W20260709 12:23:50.031371 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 459104us
I20260709 12:23:50.035015 20437 consensus_queue.cc:260] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:50.072887 16442 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:50.073678 20437 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:50.077458 20868 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:50.078366 20892 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:50.078925 20868 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:23:50.026923 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 170959us
I20260709 12:23:50.077461 16442 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:50.071062 16223 catalog_manager.cc:5697] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:50.079598 20892 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:50.104943 20901 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:50.106036 20901 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:50.106391 20816 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:50.108542 20923 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:23:50.109041 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 246949us
I20260709 12:23:50.111356 16469 raft_consensus.cc:1275] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:23:50.115069 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:50.121594 20901 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:50.122282 20868 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:50.125538 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 48583.6us
I20260709 12:23:50.127724 16545 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:50.129392 20919 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:50.133100 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:50.133360 20923 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:50.137339 20715 raft_consensus.cc:3055] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:50.137639 20816 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:50.152509 20715 raft_consensus.cc:740] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:50.153002 20816 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:50.153059 20669 consensus_queue.cc:1048] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:50.152952 16548 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
I20260709 12:23:50.154126 20816 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:50.154440 20715 consensus_queue.cc:260] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:50.155918 20715 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:50.141940 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:50.142355 20746 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:50.190796 20746 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:50.193533 20535 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:50.103055 20892 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:50.194819 20535 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:50.171888 16442 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:50.198380 17258 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45224 (request call id 1123) took 1603 ms (client timeout 2131 ms). Trace:
I20260709 12:23:50.141602 20762 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:50.199998 20762 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:50.200416 20919 raft_consensus.cc:2749] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:50.200953 20762 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:50.156576 20816 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:50.161832 20803 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:50.205560 20803 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:50.162940 20831 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:50.141391 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:50.208913 20831 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:50.211813 20831 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:50.212990 20803 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:50.169726 16469 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:50.169873 20923 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:50.219235 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:50.245491 20923 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:50.224617 20529 consensus_queue.cc:1175] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ef57beab0e1d40d585983873c55107e8" responder_term: 13 status { last_received { term: 13 index: 8 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:23:50.175220 20868 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:50.178488 20763 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:50.180826 20921 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:50.181317 16441 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:50.181862 16515 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:50.183843 20674 raft_consensus.cc:3055] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:50.185177 20893 consensus_queue.cc:1048] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:50.187952 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:50.207804 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:50.213414 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:50.214052 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:50.215219 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:50.216897 16223 catalog_manager.cc:5697] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:23:50.139694 20823 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:50.225557 16396 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:23:50.238997 16442 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:50.241988 16545 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37226 (request call id 1514) took 1604 ms (client timeout 1858 ms). Trace:
W20260709 12:23:50.199090 17258 rpcz_store.cc:269] 0709 12:23:48.595083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.595193 (+   110us) service_pool.cc:224] Handling call
0709 12:23:50.197689 (+1602496us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54005f3554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:50.198051 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2522831616}
W20260709 12:23:50.255059 16545 rpcz_store.cc:269] 0709 12:23:48.637430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.658537 (+ 21107us) service_pool.cc:224] Handling call
0709 12:23:50.241587 (+1583050us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b540082b4d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:50.241895 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3006340864}
I20260709 12:23:50.240128 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:50.250631 20674 raft_consensus.cc:740] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:50.261538 20871 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:50.262285 20674 consensus_queue.cc:260] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:50.262786 17255 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:50.264001 20674 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:50.226687 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:50.266146 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 84941.6us
I20260709 12:23:50.234247 20535 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:50.228384 20746 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:50.259483 16474 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
I20260709 12:23:50.269433 16399 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
I20260709 12:23:50.231915 20773 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:50.274801 16472 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
I20260709 12:23:50.275949 16548 raft_consensus.cc:1240] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:50.260023 20762 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:50.280716 20830 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:50.270453 20600 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:23:50.288542 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:50.296548 20670 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:50.298044 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:50.300756 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:50.258584 20921 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:50.292526 20746 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:50.302718 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:50.305269 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:50.262338 20871 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:50.301791 17266 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
I20260709 12:23:50.294518 20535 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:50.314000 17144 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:50.319890 20707 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:50.320153 20893 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:50.320497 20707 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:50.321007 20707 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:50.326251 17255 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:23:50.326740 17261 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 1147) took 1249 ms. Trace:
I20260709 12:23:50.318969 16469 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
I20260709 12:23:50.327167 17261 rpcz_store.cc:276] 0709 12:23:49.077022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:49.077198 (+   176us) service_pool.cc:224] Handling call
0709 12:23:50.173866 (+1096668us) spinlock_profiling.cc:243] Waited 2.37 ms on lock 0x7b58003b8298. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:23:50.174369 (+   503us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:50.252227 (+ 77858us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:50.252259 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:50.252307 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:50.256019 (+  3712us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:50.326383 (+ 70364us) spinlock_profiling.cc:243] Waited 73.3 ms on lock 0x7b540055f854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:50.326426 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:50.326567 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":173952640}
I20260709 12:23:50.328004 20921 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:50.328804 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 89978.6us
I20260709 12:23:50.329914 16442 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:50.330530 17144 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:50.331751 16442 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:50.331557 20621 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:50.286908 20830 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:50.339359 20927 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:50.343425 16546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 868) took 1744 ms. Trace:
I20260709 12:23:50.350941 16472 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
W20260709 12:23:50.356057 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 174585us
I20260709 12:23:50.357551 20871 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:50.358006 20821 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:50.358494 20821 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:50.358930 20821 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:50.360021 20830 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:50.337360 16367 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
W20260709 12:23:50.361881 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 67714.8us
I20260709 12:23:50.354450 20669 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:50.364634 20907 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:50.365442 20907 raft_consensus.cc:697] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:50.365964 16545 raft_consensus.cc:1240] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:50.366817 16398 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:50.364821 20921 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:50.367818 20921 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:50.370105 20669 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:50.371124 20921 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:50.373186 20921 consensus_queue.cc:1059] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:50.373569 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:50.374764 20677 raft_consensus.cc:3055] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:50.375216 20677 raft_consensus.cc:740] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:50.375514 17255 raft_consensus.cc:1240] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:50.376817 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:50.376650 20677 consensus_queue.cc:260] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:50.378623 20677 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:50.332445 20746 raft_consensus.cc:697] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:50.382110 20746 consensus_queue.cc:237] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:50.385165 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:50.385627 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 47555.9us
I20260709 12:23:50.386377 16547 raft_consensus.cc:2393] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 14: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:50.411662 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:50.390623 20600 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:50.332746 20773 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:50.321022 20893 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:50.426589 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:50.437357 16396 raft_consensus.cc:1240] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:50.417225 20823 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:50.430918 20600 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:50.430611 20763 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:50.456310 16399 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
W20260709 12:23:50.466054 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 78171.9us
W20260709 12:23:50.468597 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 137026us
I20260709 12:23:50.455698 20669 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:50.471181 16398 raft_consensus.cc:1275] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:23:50.367125 20907 consensus_queue.cc:237] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:50.477720 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:50.481734 20707 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:50.471400 20834 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:50.495256 20845 consensus_queue.cc:1048] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 12:23:50.478721 20803 consensus_queue.cc:1059] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 12:23:50.516515 16442 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:50.516462 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 110683us
W20260709 12:23:50.520022 20932 log.cc:927] Time spent T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
W20260709 12:23:50.520653 20934 log.cc:927] Time spent T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:23:50.524752 20763 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:50.529114 20805 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:50.529829 20805 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:50.532641 20803 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:50.533421 20803 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:50.344030 16546 rpcz_store.cc:276] 0709 12:23:48.598936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.599077 (+   141us) service_pool.cc:224] Handling call
0709 12:23:48.845307 (+246230us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b58005e9298. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:23:48.845568 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:48.845868 (+   300us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:23:49.469751 (+623883us) log.cc:844] Serialized 14 byte log entry
0709 12:23:50.341556 (+871805us) spinlock_profiling.cc:243] Waited 1.73 ms on lock 0x7b180022b160. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d16c0a 00007f287c456219 00007f2872d9ea40 00007f2872d994e4 00007f2872d99455 00007f28798813c7 00007f2875e939a7 00007f2875ea30bf 00007f2875ea3038
0709 12:23:50.342148 (+   592us) log.cc:844] Serialized 14 byte log entry
0709 12:23:50.342530 (+   382us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:50.342661 (+   131us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:50.342834 (+   173us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:50.342968 (+   134us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:50.343184 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16272,"prepare.queue_time_us":1313532,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":83,"raft.queue_time_us":15742,"raft.run_cpu_time_us":335,"raft.run_wall_time_us":13216,"spinlock_wait_cycles":36954624,"thread_start_us":798179,"threads_started":2,"wal-append.queue_time_us":332581}
I20260709 12:23:50.539602 20921 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:50.540308 20921 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:50.540858 16398 raft_consensus.cc:1275] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:23:50.541276 16398 pending_rounds.cc:85] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 5
I20260709 12:23:50.541536 16398 pending_rounds.cc:107] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:23:50.541802 16398 raft_consensus.cc:2889] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:50.543207 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:50.547282 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 68225.9us
W20260709 12:23:50.555872 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 198386us
I20260709 12:23:50.529152 20773 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:50.546792 20803 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:50.560050 20935 log.cc:927] Time spent T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 12:23:50.559788 20848 consensus_queue.cc:1048] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 12:23:50.562558 20893 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:50.564538 16514 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:50.564690 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:50.565676 16398 raft_consensus.cc:2393] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 11: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:50.517990 17266 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070"
dest_uuid: "ef57beab0e1d40d585983873c55107e8"
mode: GRACEFUL
new_leader_uuid: "f99066d13e524238aec6aa2024a8bd50"
 from {username='slave'} at 127.0.0.1:45120
I20260709 12:23:50.566290 16514 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:50.566756 17266 raft_consensus.cc:606] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Received request to transfer leadership to TS f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:50.567435 20732 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:50.567512 16514 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:50.547525 20529 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:23:50.559777 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 134687us
I20260709 12:23:50.567914 20732 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:50.570665 20732 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:50.571436 20823 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:50.572261 16475 raft_consensus.cc:1275] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:50.572121 20823 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:50.573454  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:23:50.574312 20921 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:50.571153 20529 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:50.559842 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:50.575088 20921 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:50.584918 20734 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:50.584797 20600 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:50.586113 20936 log.cc:927] Time spent T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 12:23:50.587450 20600 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:50.588943 20529 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:50.590049 20707 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:50.590245 17258 raft_consensus.cc:1275] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 12:23:50.590699 17258 pending_rounds.cc:85] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8: Aborting all ops after (but not including) 6
I20260709 12:23:50.590962 17258 pending_rounds.cc:107] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:23:50.591195 17258 raft_consensus.cc:2889] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:50.591128 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:50.593242 17255 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
I20260709 12:23:50.594440 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:50.596091 17142 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:50.596148 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:50.598677 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:50.613848 17144 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:50.615048 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:50.615736 16398 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:50.613940 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 46586us
W20260709 12:23:50.617983 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 48288.4us
I20260709 12:23:50.613929 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:50.624116 16514 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:50.574313 20823 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:50.633744 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:50.634881 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:50.640435 16547 raft_consensus.cc:1275] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 12:23:50.640870 16547 pending_rounds.cc:85] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1: Aborting all ops after (but not including) 9
I20260709 12:23:50.641125 16547 pending_rounds.cc:107] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 12:23:50.641362 16547 raft_consensus.cc:2889] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:50.644150 20796 consensus_queue.cc:1048] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:50.648869 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:50.649539 20927 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:50.651136 20830 raft_consensus.cc:993] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8: : Instructing follower f99066d13e524238aec6aa2024a8bd50 to start an election
I20260709 12:23:50.651479 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:50.652074 20670 raft_consensus.cc:1081] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Signalling peer f99066d13e524238aec6aa2024a8bd50 to start an election
I20260709 12:23:50.653169 20758 raft_consensus.cc:3055] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:50.653577 20758 raft_consensus.cc:740] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:50.654657 20758 consensus_queue.cc:260] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:50.656478 17264 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:50.657109 20921 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:50.658563 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:50.619729 20707 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:50.656102 20758 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:50.620586 20803 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:50.669239 20946 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:50.669701 16474 raft_consensus.cc:1275] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:23:50.669991 17264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45228 (request call id 1148) took 1592 ms. Trace:
I20260709 12:23:50.626183 20848 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:50.631940 20893 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:50.701364 20893 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:50.703996 20893 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:50.691286 17269 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 13: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:50.691165 20946 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:50.693276 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:50.712044 16545 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:23:50.696864 16475 raft_consensus.cc:1275] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:23:50.697009 17143 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:50.692992 20848 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:50.697449 20830 consensus_queue.cc:1059] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:50.678253 20803 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:50.628281 20734 raft_consensus.cc:697] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:50.689368 17264 rpcz_store.cc:276] 0709 12:23:49.077287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:49.077392 (+   105us) service_pool.cc:224] Handling call
0709 12:23:49.664281 (+586889us) spinlock_profiling.cc:243] Waited 2.32 ms on lock 0x7b58003b8298. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:23:50.669787 (+1005506us) spinlock_profiling.cc:243] Waited 830 ms on lock 0x7b540018a954. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:50.669932 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1915265536}
I20260709 12:23:50.708449 16398 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:23:50.714643 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:50.705886 20939 log.cc:927] Time spent T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 12:23:50.720245 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:50.726167 16543 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:50.727603 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:50.723695 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:50.732270 16400 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 13: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:50.733665 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:50.740841 20707 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:50.718616 20848 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:50.751210 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:50.730978 20669 consensus_queue.cc:1048] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:23:50.759123 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:50.769527 20941 log.cc:927] Time spent T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 12:23:50.752737 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:50.755023 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:50.754842 20734 consensus_queue.cc:237] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:50.758210 20921 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:50.758392 20715 consensus_queue.cc:1048] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:50.759348 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:50.729024 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:50.758387 16516 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:50.760548 20805 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:50.761849 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:50.763253 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 66355.9us
I20260709 12:23:50.721702 20803 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:50.763919 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:50.716892 20946 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:50.777172 16402 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:50.785085 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:50.773862 17144 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:50.787603 16469 raft_consensus.cc:1240] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:50.791836  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:50.802616 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:50.809232 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:50.810570 16551 raft_consensus.cc:2393] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 12: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:50.813872 20740 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:50.814184 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:50.815323 17270 raft_consensus.cc:2376] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10. Re-sending same reply.
I20260709 12:23:50.809460 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:50.815852 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:50.816799 16515 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:50.817660 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:50.817974 20823 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:50.818727 20823 raft_consensus.cc:697] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 10 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:50.814692 20740 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:50.819906 20823 consensus_queue.cc:237] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:50.822088 20740 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:50.824185 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:50.792282 16223 catalog_manager.cc:5697] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 9 to 10, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 10 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:50.837697 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:50.847075 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:50.852913 16549 raft_consensus.cc:2393] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 15: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:50.851701 16441 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:50.823042 20830 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:50.852615 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:50.864374 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:50.865238 16395 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:23:50.865983 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:50.867077 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:50.868101 16473 raft_consensus.cc:2410] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:23:50.876780 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:50.878676 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 120407us
I20260709 12:23:50.882323 20836 raft_consensus.cc:3055] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:50.882802 20836 raft_consensus.cc:740] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:50.904654 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:50.853022 16396 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070"
dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
 from {username='slave'} at 127.0.0.1:58430
I20260709 12:23:50.907941 16396 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:50.908321 16396 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:50.896371 20941 log.cc:927] Time spent T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
W20260709 12:23:50.914716 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 290648us
I20260709 12:23:50.916493 16543 raft_consensus.cc:1275] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:50.897243 20600 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:50.883780 20836 consensus_queue.cc:260] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:50.917889 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:50.919349 16394 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:23:50.875888 20740 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:50.848563 20821 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:50.933638 16469 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
I20260709 12:23:50.935168 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:50.945456 20836 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:50.923399 20600 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:50.951717 20600 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:50.952768 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:50.953491 16475 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:23:50.960597 16514 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:50.996937 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:51.004242 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:51.008641 17266 raft_consensus.cc:1240] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 14. Current term is 15. Ops: []
W20260709 12:23:51.011603 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 180641us
I20260709 12:23:51.025880 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:51.021189 20830 raft_consensus.cc:697] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:51.041532 20830 consensus_queue.cc:237] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:51.042366 16470 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
I20260709 12:23:51.048918 20768 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:51.049955 20768 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:51.053819 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 98870.1us
I20260709 12:23:51.056028 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:51.056412 20740 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:51.056775 16470 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:51.056962 20740 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:51.057418 20740 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:51.059713 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:51.064394 20937 log.cc:927] Time spent T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.254s	user 0.003s	sys 0.003s
I20260709 12:23:51.068966 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:51.069516 16470 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:51.070222 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:51.071302 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:51.071588 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:51.072705 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:51.072945 16470 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:51.073920 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:51.074450 16473 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:51.074659 16475 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
W20260709 12:23:51.075865 20732 consensus_meta.cc:287] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.505s	user 0.005s	sys 0.005s
I20260709 12:23:51.080777 16468 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:23:51.081996 20707 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:51.082496 20707 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:51.083240 16473 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
W20260709 12:23:51.084724 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:51.086946 20881 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:51.087651 20881 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:51.090327 20881 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:51.090969 20881 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:51.091490 20881 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:51.092548 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 66188.1us
I20260709 12:23:51.092658 20831 consensus_queue.cc:1048] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:51.094549 20621 consensus_queue.cc:1048] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:51.094738 20670 consensus_queue.cc:1048] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:51.100265 20707 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:51.103521 20881 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:51.104207 20946 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:51.106858 20946 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:51.107228 20848 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:51.109386 20943 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:51.109849 20715 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:51.116668 17271 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
I20260709 12:23:51.111980 20600 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:51.129134 16542 raft_consensus.cc:2393] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 12: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:51.117156 20946 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:51.125286 20821 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:51.137252 16477 raft_consensus.cc:1240] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:51.137809 16442 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:51.118155 20732 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:51.142829 16442 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:51.144090 20927 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:51.149777 20600 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:51.183408 16396 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:51.152913 20768 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:51.158461 20954 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:51.161950 20848 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:51.133574 20946 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:51.122308 20943 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:51.127794 20715 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:51.206499 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 295630us
I20260709 12:23:51.130074 20803 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:51.236096 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 93365.4us
I20260709 12:23:51.232556 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:51.236779 16396 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:51.237118 16542 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:51.237047 16442 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:51.239852 16442 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:51.239096 16516 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:51.242202 20954 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:51.241750 20768 raft_consensus.cc:2764] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:23:51.250377 20773 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:51.250908 20773 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:51.251356 20773 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:51.255213 20951 log.cc:927] Time spent T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.233s	user 0.002s	sys 0.000s
W20260709 12:23:51.257412 20950 log.cc:927] Time spent T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.428s	user 0.002s	sys 0.000s
I20260709 12:23:51.257808 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:51.258649 16401 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
I20260709 12:23:50.885344  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:51.266036 16223 catalog_manager.cc:5697] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 4 to 5, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 5 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
I20260709 12:23:51.231266 20923 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:51.246326 16516 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:51.275036 20945 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:51.281678 20945 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:51.291575 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 45363.3us
I20260709 12:23:51.293694 16516 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:51.271880 20848 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:51.320688 20529 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:51.323799 16222 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59300 (request call id 85) took 49099 ms (client timeout 15000 ms). Trace:
W20260709 12:23:51.330022 20952 log.cc:927] Time spent T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.501s	user 0.002s	sys 0.000s
I20260709 12:23:51.333566 16223 catalog_manager.cc:5697] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
I20260709 12:23:51.336535 20674 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:51.337646 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:51.337352 20674 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:51.344314 16366 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:51.284258 20945 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:51.300527 16442 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:51.353826 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 53420.6us
W20260709 12:23:51.353636 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 267145us
I20260709 12:23:51.298105 20746 raft_consensus.cc:2804] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:51.285400 20954 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:51.364656 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 271280us
I20260709 12:23:51.348610 16223 catalog_manager.cc:5697] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 3 to 5, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 5 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
I20260709 12:23:51.391395 16223 catalog_manager.cc:5697] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:23:51.360061 20836 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:51.380478 20923 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:51.361513 20674 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:51.403405 17142 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:51.399541 20927 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:51.432615 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:51.324435 16222 rpcz_store.cc:269] 0709 12:23:02.231975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:02.289162 (+ 57187us) service_pool.cc:224] Handling call
0709 12:23:51.323493 (+49034331us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:40.518037 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:40.549671 (+ 31634us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:40.549775 (+   104us) write_op.cc:435] Acquired schema lock
0709 12:23:40.549810 (+    35us) tablet.cc:662] Decoding operations
0709 12:23:40.550747 (+   937us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:40.550817 (+    70us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:40.550843 (+    26us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:23:40.551628 (+   785us) tablet.cc:701] Row locks acquired
0709 12:23:40.551684 (+    56us) write_op.cc:260] PREPARE: finished
0709 12:23:40.551851 (+   167us) write_op.cc:270] Start()
0709 12:23:40.552007 (+   156us) write_op.cc:276] Timestamp: P: 1783599820551786 usec, L: 0
0709 12:23:40.552032 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:23:40.707199 (+155167us) log.cc:844] Serialized 2806 byte log entry
0709 12:23:40.851134 (+143935us) spinlock_profiling.cc:243] Waited 34.4 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2872d7f0ac 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b
0709 12:23:41.560124 (+708990us) op_driver.cc:464] REPLICATION: finished
0709 12:23:41.696529 (+136405us) write_op.cc:301] APPLY: starting
0709 12:23:41.696644 (+   115us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:41.696857 (+   213us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:41.696899 (+    42us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:41.698006 (+  1107us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:41.698347 (+   341us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 12:23:41.698395 (+    48us) write_op.cc:312] APPLY: finished
0709 12:23:41.698747 (+   352us) log.cc:844] Serialized 58 byte log entry
0709 12:23:41.698944 (+   197us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:41.699075 (+   131us) write_op.cc:454] Released schema lock
0709 12:23:41.699147 (+    72us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:41.740990 (+ 41843us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:41.741022 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":6155889,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":127823,"apply.run_cpu_time_us":3154,"apply.run_wall_time_us":21739,"num_ops":7,"prepare.queue_time_us":332418,"prepare.run_cpu_time_us":24229,"prepare.run_wall_time_us":444187,"raft.queue_time_us":206311,"raft.run_cpu_time_us":21972,"raft.run_wall_time_us":121213,"replication_time_us":1007903,"spinlock_wait_cycles":79106304,"thread_start_us":598538,"threads_started":4,"wal-append.queue_time_us":423637,"wal-append.run_cpu_time_us":19960,"wal-append.run_wall_time_us":1569261}]]}
I20260709 12:23:51.399819 20945 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:51.361013 20529 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:51.455282 16223 catalog_manager.cc:5697] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:51.400807 20746 raft_consensus.cc:697] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:51.400139 20921 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:51.429862 20803 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:51.469645 16542 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:23:51.475504 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:51.426709 20923 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:51.466181 20871 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:51.462165 20927 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:51.481679 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:51.484686 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:51.505283 16399 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:51.485196 20943 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:51.490110 17143 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:51.491783 20707 consensus_queue.cc:1048] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:51.428543 20836 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:51.411604 20740 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:51.478680 16441 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:51.503731 16402 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
I20260709 12:23:51.497987 20529 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:51.509488 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 122950us
W20260709 12:23:51.523427 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 44844.4us
I20260709 12:23:51.484750 20677 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:51.464421 20674 consensus_queue.cc:1048] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 12:23:51.505559 20923 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:51.505306 20715 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:51.508492 20946 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:51.520557 20823 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.008s	sys 0.027s
I20260709 12:23:51.508832 20732 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:51.522755 20803 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:51.500702 20921 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:51.562388 17142 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:51.531379 20763 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:23:51.556236 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 66304.2us
W20260709 12:23:51.547678 20823 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.009s	sys 0.027s
W20260709 12:23:51.591130 16214 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.530s	user 0.018s	sys 0.021s
I20260709 12:23:51.591454 20836 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:51.595456 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 45399.6us
I20260709 12:23:51.596879 20797 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:51.614588 20621 consensus_queue.cc:1048] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:51.510023 20746 consensus_queue.cc:237] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:51.618875 20927 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:51.630493 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 246849us
I20260709 12:23:51.629282 17144 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:51.632934 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:51.653183 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:51.653648 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:51.654737 16472 raft_consensus.cc:2393] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 12: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:51.655820 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:51.630689 20763 raft_consensus.cc:697] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:51.659780 16441 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:51.660508 16469 raft_consensus.cc:1275] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:23:51.660521 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:51.661516 16468 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:51.631114 20845 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:51.655257 20669 consensus_queue.cc:1048] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 12:23:51.678726 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:51.680490 20972 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:51.681071 20972 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:51.683557 20972 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:51.630369 20831 consensus_queue.cc:1059] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:23:51.685559 20740 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:51.687479 17142 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:51.694069 20763 consensus_queue.cc:237] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:51.705657 20921 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:51.706634 20921 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:51.658437 16514 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:51.656673 16474 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
W20260709 12:23:51.710314 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 51948.9us
W20260709 12:23:51.712819 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 368746us
I20260709 12:23:51.725867 16366 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:51.642566 20621 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:51.662496 20823 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:51.750461 16366 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:51.728643 20972 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:51.756976 16514 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:51.756237 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:51.732686 20621 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:51.727289 20836 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:23:51.763211 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 118130us
I20260709 12:23:51.707383 20921 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:51.774216 16441 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:51.800151 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 49818.9us
I20260709 12:23:51.753455 20707 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:51.752184 20529 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:51.727679 20773 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:51.870884 20773 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:51.754187 20674 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:51.871963 20674 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:51.753233 20669 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:51.709132 20845 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:51.803617 20706 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.007s	sys 0.017s
I20260709 12:23:51.867825 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:51.732929 20907 consensus_queue.cc:1048] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:23:51.805987 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:51.806408 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 119065us
I20260709 12:23:51.881600 16366 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:51.802734 16367 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.1:41805 timed out after 1.579s (SENT) [suppressed 24 similar messages]
I20260709 12:23:51.831614 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:51.766187 20621 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:51.884987 16548 raft_consensus.cc:1240] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:51.877619 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:51.803752 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:51.769673 20970 raft_consensus.cc:3055] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:51.907388 16549 raft_consensus.cc:2368] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 10. Current term is 11.
I20260709 12:23:51.912158 16468 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
I20260709 12:23:51.750128 20823 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:51.769363 20927 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:23:51.884488 20706 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.934s	user 0.011s	sys 0.039s
I20260709 12:23:51.763643 20943 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:51.832767 20740 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:51.918805 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 160284us
I20260709 12:23:51.923916 16546 raft_consensus.cc:2410] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 15 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:23:51.769023 20670 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:51.802006 20892 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:51.926267 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:51.891240 20893 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:51.928076 20943 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:51.936921 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:51.937479 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:51.939980 16551 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:23:51.762180 20715 consensus_queue.cc:1048] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.156s
I20260709 12:23:51.942325 17258 raft_consensus.cc:1240] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:51.947901 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:51.917032 20970 raft_consensus.cc:740] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:51.922221 20773 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:51.964401 16514 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:51.961890 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:51.962093 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:51.971453 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:51.972260 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:51.973272 17271 raft_consensus.cc:2393] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 15: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:51.801548 20836 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:51.889992 20669 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:51.978282 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 175563us
I20260709 12:23:51.919821 20927 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:51.941838 20893 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:51.981101 16549 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:52.000792 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:52.026305 16551 raft_consensus.cc:1240] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:23:52.023783 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:51.967253 20768 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:52.034170 20674 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:52.034274 20975 log.cc:927] Time spent T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
W20260709 12:23:52.035508 20974 log.cc:927] Time spent T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
I20260709 12:23:52.035604 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:52.070804 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 189240us
W20260709 12:23:52.036005 20960 log.cc:927] Time spent T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:23:52.036296 20980 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:52.036494 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:51.979110 20746 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:51.978963 20600 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:52.032251 16367 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
W20260709 12:23:52.083876 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 51780.4us
I20260709 12:23:52.024088 20901 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:51.946667 20892 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:51.950595 20670 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:51.981755 20981 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:52.022277 20893 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:52.071100 17258 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14.
I20260709 12:23:52.080807 17142 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:52.081471 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 96339.3us
I20260709 12:23:51.966928 20740 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:51.944934 20943 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:52.025158 20927 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:51.979524 20773 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:52.091398 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:52.103030 17255 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
W20260709 12:23:52.117401 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 352050us
I20260709 12:23:52.088145 20600 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:52.120878 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:52.092106 20674 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:52.160293 16441 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:52.093472 20746 raft_consensus.cc:697] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:52.167393 16545 raft_consensus.cc:1275] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:23:52.069151 20768 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:52.172837 16471 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
I20260709 12:23:52.189787 17255 raft_consensus.cc:1275] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:52.153833 20600 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:52.227622 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:52.162940 20994 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:52.228737 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:52.386479 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:23:52.229362 20993 log.cc:927] Time spent T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:23:52.163161 20982 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:23:52.230417 20988 log.cc:927] Time spent T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260709 12:23:52.230688 20746 consensus_queue.cc:237] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:52.231570 20985 log.cc:927] Time spent T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
I20260709 12:23:52.328677 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:52.409749 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:52.333691 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 261693us
W20260709 12:23:52.330104 16399 consensus_meta.cc:287] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.807s	user 0.016s	sys 0.001s
I20260709 12:23:52.412616 16399 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:23:52.184715 20921 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:52.361969 20707 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.166950 20796 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:52.099458 20901 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:52.341001 20706 consensus_queue.cc:1048] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.090s
I20260709 12:23:52.461316 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:52.462553 20669 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:52.462965 20901 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:52.387946 17261 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 15.
I20260709 12:23:52.369097 20845 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:52.464219 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:52.465322 20670 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:52.465502 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:23:52.465930 20990 log.cc:927] Time spent T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.440s	user 0.000s	sys 0.022s
I20260709 12:23:52.392932 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:52.378762 20871 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:52.402354 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:52.103353 20970 consensus_queue.cc:260] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:52.415225 17142 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:52.378595 20674 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:52.401950 16394 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:52.418507 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:52.421340 20984 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.422003 20946 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:23:52.422878 20976 log.cc:927] Time spent T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.352s	user 0.001s	sys 0.001s
I20260709 12:23:52.102641 20981 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:52.457813 16367 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:52.391997 17270 raft_consensus.cc:1275] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:52.467165 20740 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:52.438704 20972 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.469714 20995 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.473487 20768 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:52.476311 20823 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:52.484546 20677 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.476799 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:52.529258 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 114485us
I20260709 12:23:52.530376 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:52.486212 17255 raft_consensus.cc:3055] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:52.401542 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:52.483050 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:52.484087 16398 raft_consensus.cc:1275] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:23:52.532558 16395 raft_consensus.cc:2393] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 15: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:52.512941 16396 raft_consensus.cc:1275] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:52.509207 20797 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.496367 20991 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.480420 20983 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:52.513768 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:52.505822 20871 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:52.518760 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 61027.9us
I20260709 12:23:52.508100 20763 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.519589 20836 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:52.531603 17255 raft_consensus.cc:740] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:23:52.536283 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:52.410187 20982 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:52.511386 20715 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:52.538928 20881 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:52.540068 20677 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:52.541136 20991 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:52.548738 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:52.486438 20707 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:52.547413 20923 consensus_meta.cc:287] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.997s	user 0.011s	sys 0.003s
I20260709 12:23:52.586947 20991 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:52.520782 20921 raft_consensus.cc:2749] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:52.596943 17254 raft_consensus.cc:2368] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11.
I20260709 12:23:52.597604 17264 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:23:52.543522 20983 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:52.510645 20674 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:52.573422 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:52.534063 20706 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:52.387111 16214 maintenance_manager.cc:643] P 9e68af265dd04372955944ded6c1437a: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 7.028s	user 0.194s	sys 0.054s Metrics: {"bytes_written":28870,"cfile_init":1,"dirs.queue_time_us":120827,"dirs.run_cpu_time_us":447,"dirs.run_wall_time_us":1393,"drs_written":1,"lbm_read_time_us":450,"lbm_reads_lt_1ms":4,"lbm_write_time_us":53477,"lbm_writes_1-10_ms":1,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":63,"thread_start_us":416927,"threads_started":2,"wal-append.queue_time_us":481255}
I20260709 12:23:52.577276 20901 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:52.577378 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:52.545220 20972 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:52.457068 20994 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:52.579880 20677 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:52.579638 17263 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
I20260709 12:23:52.512106 20984 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:52.583220 17142 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:52.584132 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:52.568681 20992 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:52.536624 20970 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:52.515239 20946 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:52.615782 20983 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:52.640770 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 83869.6us
I20260709 12:23:52.589035 17269 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14.
I20260709 12:23:52.614132 20773 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:52.618745 16395 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
W20260709 12:23:52.589344 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 429159us
I20260709 12:23:52.547233 20797 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:52.574486 17255 consensus_queue.cc:260] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:52.646953 17255 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:52.630424 16367 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:52.538170 20995 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:52.556677 20529 consensus_queue.cc:1059] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
W20260709 12:23:52.633985 20997 log.cc:927] Time spent T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 12:23:52.648629 17142 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:52.615382 17143 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:52.587193 20923 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:52.649464 16441 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:52.668843 16441 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
W20260709 12:23:52.671229 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 55977.1us
I20260709 12:23:52.589692 20715 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:52.621488 20892 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:52.565945 20763 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:52.651495 20670 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:52.676680 20803 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:52.688248 20892 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:52.626703 20901 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:52.696920 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 266837us
W20260709 12:23:52.698076 16400 consensus_meta.cc:287] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.665s	user 0.008s	sys 0.000s
I20260709 12:23:52.698611 16400 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:23:52.697830 17144 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:52.649372 20677 consensus_queue.cc:1059] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:23:52.706139 17143 leader_election.cc:400] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 10 election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 10. Current term is 11.
I20260709 12:23:52.707489 20707 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:52.707646 20871 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:52.653190 20881 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:23:52.640048 20992 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:52.640664 20706 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:52.714512 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 307313us
I20260709 12:23:52.697193 20831 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:52.704041 20669 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:52.721783 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:52.734508 16400 raft_consensus.cc:1240] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 12. Current term is 13. Ops: []
W20260709 12:23:52.713565 20998 log.cc:927] Time spent T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.319s	user 0.001s	sys 0.000s
I20260709 12:23:52.704473 20670 raft_consensus.cc:697] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 10 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:23:52.741482 16549 consensus_meta.cc:287] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.710s	user 0.009s	sys 0.005s
I20260709 12:23:52.742125 16549 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:23:52.745363 16470 raft_consensus.cc:1275] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:23:52.709659 21007 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:52.751055 16551 raft_consensus.cc:1275] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:52.745698 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:52.717268 20707 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:23:52.749547 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 76885.3us
I20260709 12:23:52.805771 17143 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:52.806182 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:52.806835 16468 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:52.749752 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:52.807704 16398 raft_consensus.cc:2393] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 13: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
W20260709 12:23:52.750159 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 101625us
I20260709 12:23:52.752985 16543 raft_consensus.cc:2393] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 12: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:52.746542 16514 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:52.735476 20669 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:52.806922 17143 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:52.811534 20669 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:52.745656 20600 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:52.825174 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 78699us
I20260709 12:23:52.820793 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:52.823242 16472 raft_consensus.cc:1275] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:52.813316 16215 maintenance_manager.cc:419] P 9e68af265dd04372955944ded6c1437a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7126 bytes on disk
I20260709 12:23:52.811980 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:52.833148 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:52.827476 16514 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
W20260709 12:23:52.819303 21005 log.cc:927] Time spent T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 12:23:52.839241 16549 raft_consensus.cc:1275] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:23:52.841585 20669 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:52.741307 20831 raft_consensus.cc:697] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:23:52.812269 20893 consensus_meta.cc:287] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.698s	user 0.009s	sys 0.001s
I20260709 12:23:52.820683 21004 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:52.819682 21000 log.cc:927] Time spent T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
W20260709 12:23:52.844527 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 99449.6us
I20260709 12:23:52.816246 20984 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:52.816938 20674 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:52.816893 20982 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:52.805162 20740 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.870775 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:52.852125 16396 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
W20260709 12:23:52.820083 20999 log.cc:927] Time spent T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
I20260709 12:23:52.835093 17143 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:52.858415 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:52.860726 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:52.863933 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:52.851549 16477 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
I20260709 12:23:52.817992 20763 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:52.833716 20797 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:52.824599 20707 raft_consensus.cc:697] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:52.844319 20981 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:52.819041 20973 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:23:52.844815 20621 consensus_meta.cc:287] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.931s	user 0.005s	sys 0.007s
I20260709 12:23:52.821897 20881 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:52.820207 20600 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:52.865187 20892 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:52.828189 21007 raft_consensus.cc:697] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:52.847358 20823 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:52.872740 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:52.872522 16214 maintenance_manager.cc:643] P 9e68af265dd04372955944ded6c1437a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s	user 0.001s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":356,"lbm_reads_lt_1ms":4}
I20260709 12:23:52.873400 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:52.873675 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:52.853428 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:52.873708 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:52.890915 16398 raft_consensus.cc:2410] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:23:52.892580 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:52.871173 21006 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:52.940618 16476 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:52.814105 20983 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:52.845743 20836 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:52.873400 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:52.865935 20972 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:52.865655 20768 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:52.868125 20991 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:52.906387 20881 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:52.918543 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:52.849370 20732 consensus_queue.cc:1048] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 12:23:52.938750 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:52.895690 20982 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.941390 16470 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:23:52.941162 16472 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
W20260709 12:23:52.953541 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 101038us
I20260709 12:23:52.941658 16468 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:52.941452 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:52.954385 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 324039us
I20260709 12:23:52.816844 20670 consensus_queue.cc:237] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:52.902532 20674 raft_consensus.cc:3055] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:52.891027 20740 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:52.910440  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:52.984120 16441 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:53.021082 16367 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:52.980347 20994 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:52.997881 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:53.044436 20669 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:52.873265 20893 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:53.049784 20893 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:52.910341 20981 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:53.050768 20981 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:53.053633 20981 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:53.054492 16403 raft_consensus.cc:1240] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 13. Current term is 14. Ops: [11.6-13.7]
W20260709 12:23:53.000309 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 165338us
I20260709 12:23:53.057241 17143 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:53.003249 16396 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
I20260709 12:23:52.891440 21004 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:53.062955 21004 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:52.980993 20968 log.cc:927] Time spent T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a: Append to log took a long time: real 0.359s	user 0.001s	sys 0.001s
I20260709 12:23:52.969615 20946 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:53.066032 20946 consensus_queue.cc:1048] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:52.909528 20797 consensus_queue.cc:1059] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.091s
I20260709 12:23:53.013701 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:53.068208 16470 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:23:52.970906 20706 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:52.989498 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 129452us
I20260709 12:23:53.000713 20881 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:53.039913 17266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 1170) took 1205 ms. Trace:
I20260709 12:23:52.973244 21008 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:23:53.071054 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 201083us
I20260709 12:23:52.909442 20621 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:53.075119 16516 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:52.965960 20892 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:53.088582 16548 raft_consensus.cc:1275] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:23:53.010535 20823 raft_consensus.cc:697] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:53.010870 20674 raft_consensus.cc:740] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:52.939764 20973 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:52.988240 20768 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:53.086138 16546 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
I20260709 12:23:53.098426 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:53.120543 16468 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:23:52.973126 21006 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:53.004247 20991 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:53.122558 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:53.080042 20669 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:53.036362 20715 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:53.125388 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 104451us
I20260709 12:23:53.122116 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:53.086138 21008 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:53.011967 20982 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:53.073453 17266 rpcz_store.cc:276] 0709 12:23:51.879677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:51.889798 (+ 10121us) service_pool.cc:224] Handling call
0709 12:23:51.907197 (+ 17399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:51.925325 (+ 18128us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:51.925356 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:51.949283 (+ 23927us) log.cc:844] Serialized 26 byte log entry
0709 12:23:52.408834 (+459551us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:52.408904 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:52.408991 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:53.039626 (+630635us) raft_consensus.cc:1685] finished
0709 12:23:53.039658 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:53.039846 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":135,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":6954,"spinlock_wait_cycles":1862912,"thread_start_us":345931,"threads_started":1,"wal-append.queue_time_us":620308}
I20260709 12:23:53.137998 17258 raft_consensus.cc:2393] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 11: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:53.094319 20995 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:53.086813 20893 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:23:53.141752 16394 consensus_meta.cc:287] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.632s	user 0.005s	sys 0.007s
I20260709 12:23:53.144939 20797 raft_consensus.cc:3055] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:53.150120 20892 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:53.151049 21009 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:53.073403 20529 consensus_queue.cc:1048] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 12:23:53.154258 20973 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:53.155295 20674 consensus_queue.cc:260] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:53.130873 20803 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:53.156368 21006 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:53.159461 16367 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:53.132031 20901 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:53.071189 20831 consensus_queue.cc:237] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:53.094099 20707 consensus_queue.cc:237] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:53.201175 21011 log.cc:927] Time spent T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:23:53.198128 20674 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:53.209817 20983 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:53.198421 20991 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:53.198879 20927 consensus_meta.cc:287] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 1.046s	user 0.010s	sys 0.002s
W20260709 12:23:53.220742 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 61515.1us
I20260709 12:23:53.169700 21008 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:53.184710 20984 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:23:53.227519 17255 consensus_meta.cc:287] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.580s	user 0.008s	sys 0.004s
I20260709 12:23:53.176455 20797 raft_consensus.cc:740] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:53.238291 17255 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
W20260709 12:23:53.228207 21005 log.cc:927] Time spent T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:23:53.226824 20529 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:53.275806 16367 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:53.250541 17255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45230 (request call id 1181) took 1025 ms. Trace:
W20260709 12:23:53.252753 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 177162us
I20260709 12:23:53.256986 20982 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:53.258391 21018 log.cc:927] Time spent T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:23:53.261031 20995 consensus_queue.cc:1048] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:53.184973 20893 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:53.300614 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:53.158581 21007 consensus_queue.cc:237] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:53.286170 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 302181us
I20260709 12:23:53.238009 20927 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:53.297391 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 274035us
I20260709 12:23:53.302184 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:53.303301 16394 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
I20260709 12:23:53.282552 17255 rpcz_store.cc:276] 0709 12:23:52.225360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:52.334816 (+109456us) service_pool.cc:224] Handling call
0709 12:23:53.250398 (+915582us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:53.304601 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:53.313633 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:53.313762 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:53.314116 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:53.317693 16547 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 15.
I20260709 12:23:53.346791 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:53.347080 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:53.347605 16399 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
I20260709 12:23:53.348830 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:53.390910 16469 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
I20260709 12:23:53.326335 16514 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:53.312037 21038 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.319121 20669 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:53.402180 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 75915.3us
I20260709 12:23:53.312549 21019 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.303189 20732 consensus_queue.cc:1048] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.088s
W20260709 12:23:53.398730 21018 log.cc:927] Time spent T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:23:53.414381 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:53.248513 20823 consensus_queue.cc:237] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:53.390331 20995 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:23:53.413275 20943 consensus_meta.cc:287] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 1.260s	user 0.007s	sys 0.008s
I20260709 12:23:53.424806 16544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1544) took 1593 ms. Trace:
I20260709 12:23:53.414414 20973 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:53.410504 20621 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:53.410840 20715 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.428442 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:53.429450 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:53.403700 20529 consensus_queue.cc:1048] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:23:53.437845 17270 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:23:53.438920 17258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45230 (request call id 1183) took 1212 ms. Trace:
I20260709 12:23:53.389798 20797 consensus_queue.cc:260] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:53.441390 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:53.441881 20797 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:53.440809 17258 rpcz_store.cc:276] 0709 12:23:52.226216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:52.334624 (+108408us) service_pool.cc:224] Handling call
0709 12:23:53.438696 (+1104072us) spinlock_profiling.cc:243] Waited 744 ms on lock 0x7b540005f854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:53.438863 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1711264128}
I20260709 12:23:53.443804 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:53.444425 21027 consensus_queue.cc:1048] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.141s
W20260709 12:23:53.444862 20970 consensus_meta.cc:287] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.804s	user 0.012s	sys 0.000s
I20260709 12:23:53.446014 17261 raft_consensus.cc:2410] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:23:53.446203 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:53.447368 17270 raft_consensus.cc:2393] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 15: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:53.433010 20768 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:53.433225 20995 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:53.449782 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:53.423868 20669 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:53.450469 17270 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:23:53.424050 20881 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:53.452294 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:53.452455 21020 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:53.453312 21020 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:53.453857 21020 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:53.456928 17255 raft_consensus.cc:2393] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 14: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:53.429342 16544 rpcz_store.cc:276] 0709 12:23:51.831448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:51.833188 (+  1740us) service_pool.cc:224] Handling call
0709 12:23:51.890285 (+ 57097us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:51.921950 (+ 31665us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:51.921992 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:51.967804 (+ 45812us) log.cc:844] Serialized 26 byte log entry
0709 12:23:52.187627 (+219823us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:52.187690 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:52.187785 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:53.401314 (+1213529us) spinlock_profiling.cc:243] Waited 61.1 ms on lock 0x7b240026f378. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872c229c4 00007f2872c20acb 00007f2872ddbdc9 00007f2872ddba55 00007f2872ddafb0 00007f2875a51488 00007f2875a51be8 00007f2875a29398 00007f2875a2476a 00007f2875a23bb6
0709 12:23:53.424516 (+ 23202us) raft_consensus.cc:1685] finished
0709 12:23:53.424538 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:53.424743 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32304,"raft.queue_time_us":26301,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":9055,"spinlock_wait_cycles":140465408,"thread_start_us":67641,"threads_started":1,"wal-append.queue_time_us":525429}
I20260709 12:23:53.425758 21019 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:53.412510 21024 consensus_queue.cc:1059] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 12:23:53.473500 20881 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:53.473860 21024 raft_consensus.cc:3055] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:53.475500 21024 raft_consensus.cc:740] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:53.424012 21038 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:53.457005 20715 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:53.479735 16542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 895) took 1609 ms. Trace:
I20260709 12:23:53.433311 20973 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:53.476279 20995 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:53.488008 16542 rpcz_store.cc:276] 0709 12:23:51.869920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:51.972459 (+102539us) service_pool.cc:224] Handling call
0709 12:23:51.972987 (+   528us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:52.056001 (+ 83014us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:52.056038 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:52.091714 (+ 35676us) log.cc:844] Serialized 26 byte log entry
0709 12:23:52.513020 (+421306us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:52.971005 (+457985us) log.cc:844] Serialized 14 byte log entry
0709 12:23:53.479077 (+508072us) spinlock_profiling.cc:243] Waited 35.2 ms on lock 0x7b5400ba0f20. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e04733 00007f2875e89dbf 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c
0709 12:23:53.479154 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:53.479270 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:53.479333 (+    63us) raft_consensus.cc:1685] finished
0709 12:23:53.479468 (+   135us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:53.479657 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7791,"prepare.queue_time_us":613572,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":87780,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":13857,"spinlock_wait_cycles":81099904,"thread_start_us":858642,"threads_started":3,"wal-append.queue_time_us":550203}
I20260709 12:23:53.475431 21019 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:53.476522 21024 consensus_queue.cc:260] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:53.497521 21024 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:53.497758 20893 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:53.435206 20943 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:53.495983 20927 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:53.503118 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 54243.9us
I20260709 12:23:53.507527 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:53.507808 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:53.509580 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:53.510118 20677 raft_consensus.cc:3055] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:53.510345 16398 raft_consensus.cc:2368] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11.
I20260709 12:23:53.510603 20677 raft_consensus.cc:740] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:53.511724 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:53.511544 20677 consensus_queue.cc:260] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:53.511737 17266 raft_consensus.cc:1275] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:23:53.518947 21038 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:53.501988 20973 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:53.529821 20740 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:53.514294 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:53.536394 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 552279us
I20260709 12:23:53.537382 21029 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:53.538009 21029 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:53.538630 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:53.540295 21029 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:53.541873 16471 raft_consensus.cc:1240] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:23:53.543650 20773 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:53.544557 20901 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.545104 20901 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:53.546845 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:53.547132 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:53.547636 17255 raft_consensus.cc:2368] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11.
I20260709 12:23:53.547742 16397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 1275) took 1311 ms. Trace:
I20260709 12:23:53.547968 17263 raft_consensus.cc:2393] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 10: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:53.548506 21014 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:53.548650 21030 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:53.548717 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:53.549106 21014 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:53.549636 17264 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 16.
I20260709 12:23:53.550822 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:53.549263 21030 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:53.551571 17269 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:23:53.541098 20600 consensus_queue.cc:1048] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 12:23:53.552816 20901 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:53.548193 16397 rpcz_store.cc:276] 0709 12:23:52.236796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:52.361672 (+124876us) service_pool.cc:224] Handling call
0709 12:23:52.391980 (+ 30308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:52.556897 (+164917us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:23:52.902931 (+346034us) log.cc:844] Serialized 14 byte log entry
0709 12:23:53.011722 (+108791us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:53.011913 (+   191us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:53.026999 (+ 15086us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:53.547525 (+520526us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b540011b054. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:53.547573 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:53.547688 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":50362,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":5570,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":51352,"spinlock_wait_cycles":363406208,"thread_start_us":338136,"threads_started":2,"wal-append.queue_time_us":282696}
I20260709 12:23:53.555083 21030 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:53.558357 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:53.572661 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:53.573081 21030 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:53.573191 17144 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:53.573915 21030 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:53.575059 20621 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:53.575518 20621 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:53.575913 20621 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:53.551596 21014 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:53.540954 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:53.540654 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:53.576289 21022 log.cc:927] Time spent T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.443s	user 0.000s	sys 0.011s
I20260709 12:23:53.561125 20992 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:53.561488 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:53.577872 16398 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:23:53.581243 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:53.541168 20669 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:53.582872 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:53.583621 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:53.569113 20677 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:53.595032 16403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 1277) took 1232 ms. Trace:
I20260709 12:23:53.569694 16472 raft_consensus.cc:1275] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:53.565595 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:53.571009 16516 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:53.576932 17144 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:53.582916 16395 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 13: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:53.567662 16442 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:53.578382 16400 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
I20260709 12:23:53.567248 17261 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45224 (request call id 1150) took 1339 ms. Trace:
I20260709 12:23:53.522637 21008 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:53.596124 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:53.598212 17261 rpcz_store.cc:276] 0709 12:23:52.228137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:52.465211 (+237074us) service_pool.cc:224] Handling call
0709 12:23:53.567017 (+1101806us) spinlock_profiling.cc:243] Waited 979 ms on lock 0x7b540063d654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:53.567178 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2252696448}
W20260709 12:23:53.598788 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:53.608254 16367 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:53.595510 16403 rpcz_store.cc:276] 0709 12:23:52.362690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:52.362828 (+   138us) service_pool.cc:224] Handling call
0709 12:23:52.363244 (+   416us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:53.055079 (+691835us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:53.594778 (+539699us) spinlock_profiling.cc:243] Waited 691 ms on lock 0x7b5400543154. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:53.594972 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1588675968}
I20260709 12:23:53.611585 16397 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:53.611534 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 43995.9us
I20260709 12:23:53.613081 16442 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:53.597800 17266 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 15.
I20260709 12:23:53.615782 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:53.623479 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:53.624593 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:53.625437 17264 raft_consensus.cc:2376] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 15. Re-sending same reply.
I20260709 12:23:53.625295 16442 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:53.615597 16516 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:53.627277 20901 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:53.634222 16367 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:53.634370 16442 leader_election.cc:400] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11.
I20260709 12:23:53.635044 16442 leader_election.cc:403] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:23:53.639213 16367 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:53.639430 16442 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:53.610296 21014 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:53.643265 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:53.641331 16442 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:53.615161 20946 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:53.646420 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:53.631521 20992 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:53.663455 16542 raft_consensus.cc:1240] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:53.658838 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:53.637025 20740 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:53.635185 20600 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:53.622093 20669 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:53.649649 20674 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:53.659394 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:53.666005 16398 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:23:53.668491 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:53.667608 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:53.640360 21027 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:53.672134 16472 raft_consensus.cc:1275] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:53.638317 20901 raft_consensus.cc:697] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:53.691489 20901 consensus_queue.cc:237] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:53.668783 20732 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:53.692734 20732 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:53.695134 20732 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:53.701206 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:53.661201 20892 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:53.679332 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:53.643719 20529 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:53.665352 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:53.647472 21004 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:53.681793 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:53.682677 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:53.683841 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:53.644361 20763 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11.
I20260709 12:23:53.645807 21014 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:53.688834 20674 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:53.676584 20972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.004s	sys 0.020s
I20260709 12:23:53.689534 21042 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:23:53.702929 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:53.677142 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:53.705323 16548 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
I20260709 12:23:53.708658 17254 raft_consensus.cc:2393] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 15: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:53.662253 20992 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:53.684132 20600 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:53.709769 16442 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:53.677893 20740 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:53.685000 20669 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:53.718530 16550 raft_consensus.cc:2393] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 12: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:53.640628 20978 consensus_queue.cc:1059] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:23:53.714815 16551 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:23:53.685348 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:53.730131 21014 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:53.717193 21004 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:53.732834 20943 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:53.733763 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:53.734297 20923 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:53.732931 20715 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:53.735353 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:53.775650 16367 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:53.785938 17255 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14.
I20260709 12:23:53.757987 20600 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:53.738099 20621 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:53.738919 20707 consensus_queue.cc:1048] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:53.738256 17143 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:53.750867 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:53.705123 20892 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:53.758159 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:53.775461 21004 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:53.771346 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:53.771761 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:53.757247 20992 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:53.756170 21042 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:53.735947 21030 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:53.791610 20981 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:53.760557 20740 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:53.790556 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:53.765304 20669 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:53.796684 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 58505.5us
I20260709 12:23:53.748750 20674 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:53.705799 21027 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:53.819597 21027 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:53.820109 17143 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:53.799403 20892 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:53.823361 20831 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:53.823874 20831 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:53.824460 20831 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:53.758726 21028 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:53.821137 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:53.829641 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:53.829242 21028 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:53.802345 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:53.830274 17254 raft_consensus.cc:2376] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13. Re-sending same reply.
I20260709 12:23:53.831712 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:53.832448 17254 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:23:53.833750 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:53.843016 20831 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:53.852711 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:53.844730 16367 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:53.800042 20830 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:53.848140 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:53.791805 21020 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:53.847988 16475 raft_consensus.cc:1240] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:53.855109 17263 raft_consensus.cc:2393] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 12: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:53.815379 20981 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:53.856376 16367 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:53.866835 21020 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:53.867537 17258 raft_consensus.cc:1275] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:23:53.868243 16394 raft_consensus.cc:1240] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 14. Current term is 15. Ops: []
W20260709 12:23:53.869755 21052 log.cc:927] Time spent T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:23:53.857786 21038 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:53.845633 21008 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:53.848570 21028 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:53.890975 16367 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:53.906556 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:53.858398 21047 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.862855 20981 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:53.908012 16367 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:53.915934 16367 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:53.908952 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:53.907577 17254 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
I20260709 12:23:53.923765 16367 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:53.882838 21038 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:53.937927 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:53.909881 20529 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:53.909799 21020 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:53.952594 16367 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:53.954057 16397 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:23:53.930192 21050 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:53.957958 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:53.935418 21038 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:53.963050 17254 raft_consensus.cc:3055] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:53.965957 17254 raft_consensus.cc:740] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:53.984874 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:53.955128 21019 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:53.969400 17254 consensus_queue.cc:260] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:53.987042 17254 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:53.960212 20529 raft_consensus.cc:697] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 15 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:53.919934 21047 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:53.994892 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:54.004839 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:54.004912 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:53.982759 20921 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:54.009548 17269 raft_consensus.cc:2393] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 14: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:54.009600 17261 raft_consensus.cc:2393] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 15: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:54.010640 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:54.011579 16475 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
W20260709 12:23:54.012130 21070 log.cc:927] Time spent T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
W20260709 12:23:54.015949 21059 log.cc:927] Time spent T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 12:23:54.016111 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:54.028977 21042 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:54.029134 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:54.043743 16395 raft_consensus.cc:1275] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:54.044993 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:54.027180 20674 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:54.047648 20621 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:54.050176 21047 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:54.051286 20992 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:54.054891 20529 consensus_queue.cc:237] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:54.056501 20927 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:54.057035 20927 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:54.058202 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 348532us
I20260709 12:23:54.059806 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:54.060426 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:54.039851 17144 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:54.009310 20921 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:54.065064 16397 raft_consensus.cc:1240] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 13. Current term is 14. Ops: [11.6-13.7]
I20260709 12:23:54.065510 20921 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:54.066350 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:54.067178 16547 raft_consensus.cc:2368] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 11. Current term is 12.
I20260709 12:23:54.065585 20923 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:54.067961 20923 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:54.068392 20923 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:54.069638 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:54.071130 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:54.071306 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:54.071964 17255 raft_consensus.cc:2393] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 12: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:54.071898 20972 raft_consensus.cc:3055] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:54.072535 20972 raft_consensus.cc:740] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:54.073105 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:54.073551 20972 consensus_queue.cc:260] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:54.074332 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:54.074646 20972 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:54.075472 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:54.076802 17258 raft_consensus.cc:1275] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:23:54.021936 20892 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:54.079223 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:54.079454 20892 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:54.079624 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:54.018023 21044 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:54.080605 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:54.080696 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:54.081190 20715 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:54.066525 21007 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:54.081748 20715 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:54.080273 16476 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:23:54.082038 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:54.083385 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:54.083832 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:54.018703 21060 log.cc:927] Time spent T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:23:54.084933 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:54.085764 17270 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
I20260709 12:23:54.084954 20831 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:54.087332 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:54.088251 17263 raft_consensus.cc:2410] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:23:54.088419 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:54.088979 16469 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 15.
I20260709 12:23:54.076701 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:54.090039 16542 raft_consensus.cc:2393] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 13: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:54.091370 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:54.068168 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:54.081918 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:54.095325 17261 raft_consensus.cc:1240] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:54.096901 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:54.099186 16471 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:54.102649 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:54.103818 16473 raft_consensus.cc:2393] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 12: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:54.105446 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:54.106850 17263 raft_consensus.cc:2393] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 13: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:54.109211 17271 raft_consensus.cc:2393] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 15: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:54.089005 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:54.110621 17269 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:54.111516 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:54.112190 16549 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
W20260709 12:23:54.112895 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 160401us
I20260709 12:23:54.114002 16367 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:54.114168 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:54.109050 16477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 868) took 1230 ms. Trace:
I20260709 12:23:54.118894 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:54.121129 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:54.125298 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:54.126406 17255 raft_consensus.cc:2368] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 11. Current term is 12.
W20260709 12:23:54.119879 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 80137.5us
I20260709 12:23:54.126293 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:54.118391 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:54.130833 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:54.131250 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:54.125604 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:54.048043 20674 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:54.103025 20973 consensus_meta.cc:287] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.577s	user 0.012s	sys 0.000s
I20260709 12:23:54.091270 20715 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:54.144236 16474 raft_consensus.cc:1240] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:54.144862 17266 raft_consensus.cc:2393] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 15: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:54.138192 16515 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:54.146312 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:54.007473 21014 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:54.155984 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:54.137311 20707 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:54.158244 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:54.158557 20707 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:54.160487 20796 consensus_queue.cc:1048] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 12:23:54.157806 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:54.163058 16476 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:23:54.132143 20621 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:54.153617 20677 consensus_queue.cc:1059] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 12:23:54.117900 16477 rpcz_store.cc:276] 0709 12:23:52.859949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:53.024078 (+164129us) service_pool.cc:224] Handling call
0709 12:23:53.076481 (+ 52403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:53.541269 (+464788us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:53.541298 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:53.541820 (+   522us) log.cc:844] Serialized 26 byte log entry
0709 12:23:53.652083 (+110263us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:23:53.708253 (+ 56170us) log.cc:844] Serialized 14 byte log entry
0709 12:23:54.031568 (+323315us) spinlock_profiling.cc:243] Waited 4.56 ms on lock 0x7b180028ba00. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:23:54.056191 (+ 24623us) spinlock_profiling.cc:243] Waited 1.48 ms on lock 0x7b180028ba00. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d16c0a 00007f287c456219 00007f2872d9ea40 00007f2872d994e4 00007f2872d99455 00007f2875e81060 00007f2875e93716 00007f2875ea4287 00007f2875ea4208
0709 12:23:54.063274 (+  7083us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:54.089775 (+ 26501us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b54002f3854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:54.089824 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:54.089872 (+    48us) raft_consensus.cc:1685] finished
0709 12:23:54.089887 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:54.090031 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":362180,"prepare.queue_time_us":56596,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":697269,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":166,"spinlock_wait_cycles":282883968,"thread_start_us":100703,"threads_started":1,"wal-append.queue_time_us":163594}
I20260709 12:23:54.164227 17254 raft_consensus.cc:1275] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 12:23:54.171222 17254 pending_rounds.cc:85] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8: Aborting all ops after (but not including) 6
I20260709 12:23:54.171563 17254 pending_rounds.cc:107] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:23:54.171940 17254 raft_consensus.cc:2889] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:54.176192 17269 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
W20260709 12:23:54.176892 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:54.183398 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:54.189036 16476 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
I20260709 12:23:54.193590 20746 consensus_queue.cc:1059] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:54.082077 21007 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:54.176556 21024 consensus_meta.cc:287] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.672s	user 0.013s	sys 0.000s
I20260709 12:23:54.168090 20927 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:54.164244 20621 raft_consensus.cc:697] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:54.224318 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:54.225529 17255 raft_consensus.cc:2368] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 13. Current term is 14.
I20260709 12:23:54.229470 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:54.233923 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:54.182449 20732 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:54.181025 20796 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:54.244805 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 110837us
I20260709 12:23:54.166843 21050 consensus_queue.cc:1048] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 12:23:54.182694 20991 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:54.149381 20973 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:54.167723 21039 consensus_queue.cc:1048] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 12:23:54.256184 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:54.259618 16515 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:54.277714 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:54.185359 20669 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:54.261065 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:54.270073 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:54.247357 16469 raft_consensus.cc:1275] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:54.286407 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:54.284643 16470 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:23:54.286059 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:54.290390 16473 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:54.261065 17142 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:54.281800 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:54.282003 17266 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2"
dest_uuid: "ef57beab0e1d40d585983873c55107e8"
mode: GRACEFUL
new_leader_uuid: "610f4feda11942739236f2b121a4b9c1"
 from {username='slave'} at 127.0.0.1:45120
I20260709 12:23:54.293849 16475 raft_consensus.cc:2393] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 12: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:54.282506 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:54.296854 16551 raft_consensus.cc:1240] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:54.301580 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:54.301602 20734 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:54.320613 20732 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:54.337805 21014 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:54.341817 21007 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:54.344915 20992 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:54.346741 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:54.356730 20621 consensus_queue.cc:237] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:54.374918 20973 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:54.254701 20796 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:54.385247 17266 raft_consensus.cc:606] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Received request to transfer leadership to TS 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:54.385656 17266 raft_consensus.cc:614] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:54.288110 20707 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:54.388233 20740 consensus_meta.cc:287] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.540s	user 0.009s	sys 0.000s
I20260709 12:23:54.388842 20740 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:54.391618 20740 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:54.393882 20981 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:54.396052 20981 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:54.396450 21044 raft_consensus.cc:2764] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 12:23:54.223399 20600 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:54.403815 20669 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:54.404706 16396 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 14: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:54.278128 20991 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:54.406883 21072 consensus_queue.cc:1048] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:54.416839 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:54.441991 16397 raft_consensus.cc:2393] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 15: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
W20260709 12:23:54.423190 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 162191us
I20260709 12:23:54.443994 17142 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:54.446106 17142 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:54.453603 16471 raft_consensus.cc:1240] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:23:54.430562 20836 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:54.470028 16476 raft_consensus.cc:1275] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:23:54.456214 17142 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:54.473589 16514 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:54.460220 21047 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:54.462859 20830 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:54.478940 21047 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:54.479118 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:54.479420 21047 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:54.479933 20836 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:54.480433 20836 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:54.480858 20836 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:54.483378 16473 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
I20260709 12:23:54.492955 20763 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:54.493583 20763 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:54.494967 20991 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:54.495651 20763 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:54.496539 20923 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:54.485605 17142 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
W20260709 12:23:54.486346 21004 consensus_meta.cc:287] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.673s	user 0.007s	sys 0.004s
I20260709 12:23:54.501085 16514 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:54.502424 16514 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:54.507299 21004 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:54.507782 21008 raft_consensus.cc:2749] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:54.495827 20991 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:54.511776 16514 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:54.504266 20740 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:54.510372 20707 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:54.508893 20973 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:23:54.542805 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 283222us
W20260709 12:23:54.530256 21019 consensus_meta.cc:287] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.539s	user 0.012s	sys 0.001s
I20260709 12:23:54.546665 16515 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:23:54.547202 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 368987us
I20260709 12:23:54.551687 16366 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:54.554060 21078 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:54.540469 20715 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:54.515555 20921 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:54.556838 16366 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:54.551295 21042 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:54.551185 21019 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:54.538477 20991 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:54.561702 16515 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:54.573349 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:54.587064 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:54.588145 20600 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:54.573218 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 63706us
W20260709 12:23:54.571331 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 315180us
W20260709 12:23:54.591562 16441 leader_election.cc:336] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Timed out: RequestConsensusVote RPC to 127.5.58.1:41805 timed out after 2.131s (SENT)
I20260709 12:23:54.561847 21008 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:54.568135 20823 raft_consensus.cc:3055] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:54.596710 20823 raft_consensus.cc:740] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:54.597718 20823 consensus_queue.cc:260] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:54.598851 20823 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:54.579584 20529 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:54.575592 21042 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:54.601266 21042 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:54.576203 21038 consensus_meta.cc:287] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.593s	user 0.010s	sys 0.003s
I20260709 12:23:54.603741 21038 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:54.605574 20921 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:54.605858 21038 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:54.590597 17142 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:54.610388 16366 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:54.612038 16366 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:23:54.642257 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:54.666110 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 54197.6us
I20260709 12:23:54.667297 16366 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:54.687721 16366 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
W20260709 12:23:54.688848 21020 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.040s	sys 0.040s
W20260709 12:23:54.689452 21020 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.040s	sys 0.040s
W20260709 12:23:54.691402 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 129783us
W20260709 12:23:54.692833 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 102336us
W20260709 12:23:54.609082 21074 log.cc:927] Time spent T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:23:54.589200 20923 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:54.561594 20715 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:54.552175 21047 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:54.694890 17142 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:54.698810 16367 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.039s	sys 0.041s
I20260709 12:23:54.665196 21038 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:54.675451 20921 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:54.675889 20529 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:54.694911 16515 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:54.700675 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 109139us
W20260709 12:23:54.693796 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 119128us
I20260709 12:23:54.709079 17142 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:54.710565 16441 leader_election.cc:400] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11.
I20260709 12:23:54.711894 16441 leader_election.cc:400] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11.
I20260709 12:23:54.713456 16441 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:54.714048 17255 raft_consensus.cc:1240] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17. Ops: []
W20260709 12:23:54.696280 21077 log.cc:927] Time spent T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
W20260709 12:23:54.701797 16367 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.040s	sys 0.042s
I20260709 12:23:54.720312 20674 raft_consensus.cc:3055] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:54.720877 20674 raft_consensus.cc:740] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:54.699615 21004 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:54.721841 20674 consensus_queue.cc:260] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:54.708148 20621 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:54.723637 21028 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:54.724658 20734 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:54.725421 20734 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:54.725862 20734 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:54.727963 20734 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:54.664947 21019 consensus_queue.cc:1059] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 12:23:54.712181 20715 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:54.735652 16441 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:54.697257 21088 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:54.695600 20973 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:54.705173 21080 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:54.755394 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:54.756485 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:54.757244 16398 raft_consensus.cc:2368] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11.
I20260709 12:23:54.724668 21037 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:54.754550 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:54.691635 21053 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.014s	sys 0.003s
I20260709 12:23:54.714946 17143 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:54.772936 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:54.765712 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:54.767544 21083 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:54.767715 16402 raft_consensus.cc:1240] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:54.768265 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:54.751953 20892 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:54.770148 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:54.713356 20921 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:54.798856 16394 raft_consensus.cc:2368] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 15. Current term is 16.
I20260709 12:23:54.740115 21004 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:54.800158 16403 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:54.755759 21080 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:54.800858 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:54.804298 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:54.802399 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 92483.7us
W20260709 12:23:54.773028 21053 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.015s	sys 0.003s
I20260709 12:23:54.804968 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:54.810729 21085 consensus_queue.cc:1048] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:54.811914 21098 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:54.759573 20946 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:54.821312 16442 leader_election.cc:400] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11.
I20260709 12:23:54.850524 16367 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:54.857537 16442 leader_election.cc:403] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:23:54.852120 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:54.759346 20763 consensus_queue.cc:1048] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:23:54.858131 16543 raft_consensus.cc:2468] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 8.
I20260709 12:23:54.862200 16442 leader_election.cc:400] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 15. Current term is 16.
I20260709 12:23:54.772516 21037 raft_consensus.cc:697] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:54.806149 20892 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:54.853511 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:54.862275 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:54.862990 16442 leader_election.cc:403] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:23:54.857769 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:54.872816 17269 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:54.868404 16367 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:54.801115 21100 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:54.874492 16367 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:54.876288 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:54.878090 17254 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:54.880185 20992 consensus_queue.cc:1059] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:23:54.881302 20992 raft_consensus.cc:3055] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:54.881670 20992 raft_consensus.cc:740] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:54.881759 17143 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:54.882771 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:54.882565 20992 consensus_queue.cc:260] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:54.817060 21004 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:54.887953 17266 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 14: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:54.822717 21080 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:54.849117 21008 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:54.805361 20923 consensus_queue.cc:1059] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 12:23:54.885057 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:54.889144 16398 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:23:54.900038 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:54.889788 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 180767us
I20260709 12:23:54.902583 17254 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:23:54.857148 21098 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:54.865674 16401 raft_consensus.cc:1275] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:23:54.870925 21007 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:54.876984 20892 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:54.805963 21083 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:54.918104 17142 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:54.808702 20836 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:54.886899 20972 consensus_meta.cc:287] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.803s	user 0.013s	sys 0.001s
I20260709 12:23:54.877209 20995 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:54.885895 21078 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:23:54.888665 21020 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:54.922823 20621 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:54.908960 21028 raft_consensus.cc:2804] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:54.864769 20946 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:54.928633 21096 log.cc:927] Time spent T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260709 12:23:54.919538 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 160832us
I20260709 12:23:54.893741 21004 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:54.932258 17142 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:54.915426 16442 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:54.986637 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 71369.5us
I20260709 12:23:54.989626 16441 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:54.934686 20600 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:54.945506 16403 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:23:54.926368 17263 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14.
I20260709 12:23:54.905714 21098 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:54.953786 17270 raft_consensus.cc:1240] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:54.940637 17269 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14.
I20260709 12:23:54.947240 20831 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:54.945060 20991 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:54.953665 20992 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:55.007591 21103 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:55.008602 20621 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:55.008487 21103 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:55.009227 20621 raft_consensus.cc:697] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:55.010164 20621 consensus_queue.cc:237] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:55.011621 21103 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:54.992760 21047 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:55.034819 20836 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:55.040602 21037 consensus_queue.cc:237] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:55.054829 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:54.895375 21100 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:55.057740 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:55.068166 16544 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:23:54.983883 21028 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:23:55.089254 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 153967us
I20260709 12:23:55.040587 20981 raft_consensus.cc:3055] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:55.096004 16367 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:55.103509 16367 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:55.104453 20893 raft_consensus.cc:2764] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election decision vote started in defunct term 14: won
I20260709 12:23:55.106814 20796 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:55.107626 21044 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:55.108202 21044 raft_consensus.cc:697] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:55.109385 16367 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:55.109284 21044 consensus_queue.cc:237] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:55.110571 21019 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:55.111114 21019 raft_consensus.cc:697] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 14 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:55.073118 21105 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:55.112002 21019 consensus_queue.cc:237] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:55.113154 21105 consensus_queue.cc:1048] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:23:55.117087 21100 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:55.118594 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:55.119010 20893 consensus_queue.cc:1059] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:23:55.120247 20677 raft_consensus.cc:3055] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:23:55.120662 20677 raft_consensus.cc:740] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:55.121518 20677 consensus_queue.cc:260] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:55.122995 20677 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:55.125099 21083 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:55.129344 16468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1528) took 1049 ms. Trace:
I20260709 12:23:55.094849 20670 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:55.130472 20670 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:55.129773 16468 rpcz_store.cc:276] 0709 12:23:54.080221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:54.084807 (+  4586us) service_pool.cc:224] Handling call
0709 12:23:54.085143 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:54.131278 (+ 46135us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:54.131313 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:54.153246 (+ 21933us) log.cc:844] Serialized 26 byte log entry
0709 12:23:54.816229 (+662983us) spinlock_profiling.cc:243] Waited 7.33 ms on lock 0x7b180026c860. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:23:55.004134 (+187905us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:55.004783 (+   649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:55.004868 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:55.129026 (+124158us) raft_consensus.cc:1685] finished
0709 12:23:55.129046 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:55.129278 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52517,"raft.queue_time_us":46295,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":8311,"spinlock_wait_cycles":16849280,"thread_start_us":187659,"threads_started":1,"wal-append.queue_time_us":942376}
I20260709 12:23:55.129133 21008 consensus_queue.cc:1059] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:23:55.132138 21008 raft_consensus.cc:3055] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:55.132567 21008 raft_consensus.cc:740] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:55.133582 21008 consensus_queue.cc:260] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:55.134830 21008 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:55.140882 17255 raft_consensus.cc:1275] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:23:55.144035 17143 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:55.034864 20674 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:55.094297 21028 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:55.066092 21095 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:55.149703 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:55.153004 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:55.155740 16544 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
I20260709 12:23:55.154946 21080 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:55.160197 20921 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:55.099435 20981 raft_consensus.cc:740] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:55.162178 21080 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:55.161931 20981 consensus_queue.cc:260] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:55.163473 20981 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:55.164405 21095 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:55.166823 21095 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:55.156986 20796 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:55.093946 20991 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:55.168432 21020 consensus_queue.cc:1048] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:55.170202 20796 raft_consensus.cc:697] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:55.171244 20796 consensus_queue.cc:237] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:55.168694 20991 raft_consensus.cc:697] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:23:55.172740 21042 consensus_meta.cc:287] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.571s	user 0.006s	sys 0.005s
I20260709 12:23:55.173084 20991 consensus_queue.cc:237] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:55.173691 21004 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:55.176844 16397 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 14: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:55.178561 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:55.179737 20946 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:55.184821 20670 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:55.185665 21028 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:55.186779 16394 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 17.
I20260709 12:23:55.188095 16442 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:55.159162 17143 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:55.193980 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:55.183755 21098 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:55.197485 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:55.199257 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:55.199976 16544 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:23:55.203894 16551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1560) took 1432 ms. Trace:
I20260709 12:23:55.204402 16551 rpcz_store.cc:276] 0709 12:23:53.771889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:53.772055 (+   166us) service_pool.cc:224] Handling call
0709 12:23:53.782560 (+ 10505us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:54.408552 (+625992us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:55.203656 (+795104us) spinlock_profiling.cc:243] Waited 514 ms on lock 0x7b54009745d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:55.203837 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1182164864}
I20260709 12:23:55.100005 21053 consensus_queue.cc:1048] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.088s
I20260709 12:23:55.176230 21004 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:55.212536 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 222959us
I20260709 12:23:55.214110 16441 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:55.215771 16441 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:55.218933 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:55.203992 21105 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:23:55.224727 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 292577us
I20260709 12:23:55.255286 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:55.238600 16441 leader_election.cc:400] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 11. Current term is 12.
I20260709 12:23:55.256148 16441 leader_election.cc:403] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:55.255400 16515 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:55.229291 16394 raft_consensus.cc:1240] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:55.201278 21042 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:55.258066 16515 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:55.263060 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:55.260849 16515 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:55.228317 21004 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:55.268975 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:55.269620 16469 raft_consensus.cc:1275] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:55.260195 20923 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:55.259405 20946 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:55.274454 16474 raft_consensus.cc:1240] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:55.264786 16441 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:55.368034 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:55.261737 21105 raft_consensus.cc:697] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:55.262393 20978 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:55.369766 20892 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:55.429425 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:55.430263 20734 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:55.432034 16471 raft_consensus.cc:1240] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:55.435076 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:55.435956 16468 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:55.437005 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 59917.3us
I20260709 12:23:55.419736 21093 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:55.438021 17143 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:55.374007 21038 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:55.372964 21004 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 11. Current term is 12.
W20260709 12:23:55.401638 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 140854us
I20260709 12:23:55.453663 16515 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:55.415009 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 150299us
I20260709 12:23:55.457521 16441 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:55.421060 21088 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:55.433213 21007 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11.
I20260709 12:23:55.494351 16441 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:55.495668 16515 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:55.514087 20978 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:55.482096 21053 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:55.488889 21028 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:55.515089 20600 consensus_meta.cc:287] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.516s	user 0.009s	sys 0.002s
I20260709 12:23:55.523712 20670 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:55.529323 21098 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:55.611935 21115 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:55.529727 21007 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:23:55.466670 21093 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:55.554056 20978 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:55.627521 20978 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:55.627074 21115 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:55.630324 21115 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:55.630756 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:55.634270 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 196394us
I20260709 12:23:55.544590 20621 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:23:55.638347 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:55.676513 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:55.680532 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:55.574129 21004 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:55.681510 16544 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:55.491528 21088 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:55.734660 21088 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:55.515674 21095 consensus_queue.cc:1048] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.118s
W20260709 12:23:55.701128 20674 consensus_meta.cc:287] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.546s	user 0.011s	sys 0.000s
I20260709 12:23:55.706697 21042 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:55.691320 20992 consensus_meta.cc:287] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.684s	user 0.006s	sys 0.005s
W20260709 12:23:55.720187 21008 consensus_meta.cc:287] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.585s	user 0.012s	sys 0.000s
I20260709 12:23:55.641530 20892 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:55.627094 21093 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:55.480319 21038 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:55.637550 21020 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:55.690529 20600 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 15. Current term is 16.
I20260709 12:23:55.688261 20621 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:23:55.834586 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 340333us
I20260709 12:23:55.840515 16544 raft_consensus.cc:2468] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 9.
I20260709 12:23:55.841192 16515 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:55.846644 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:55.847008 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:55.856259 17255 raft_consensus.cc:2376] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16. Re-sending same reply.
I20260709 12:23:55.857146 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:55.857524 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:55.848539 17263 raft_consensus.cc:1240] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:55.860095 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:55.860805 17263 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:23:55.849131 16441 leader_election.cc:400] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 13. Current term is 14.
I20260709 12:23:55.861739 16441 leader_election.cc:403] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:23:55.851217 16468 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
I20260709 12:23:55.852974 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:55.622571 21105 consensus_queue.cc:237] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:55.858527 17255 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
W20260709 12:23:55.871989 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 357496us
W20260709 12:23:55.835090 21044 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.054s	sys 0.085s
I20260709 12:23:55.875245 16441 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:55.876307 16515 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:55.883556 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:55.883755 17142 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:55.884766 17258 raft_consensus.cc:2393] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 9: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:55.837002 20621 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:55.881839 16515 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:55.896938 16515 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:55.885439 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:55.902143 17266 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:55.881778 21044 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.055s	sys 0.086s
I20260709 12:23:55.900681 16441 leader_election.cc:400] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 11. Current term is 12.
W20260709 12:23:55.964836 16472 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.786s	user 0.116s	sys 0.257s
W20260709 12:23:55.948163 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:56.016279 16472 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.837s	user 0.119s	sys 0.271s
W20260709 12:23:56.019289 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 118799us
I20260709 12:23:55.880741 21042 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:55.833783 21020 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:55.881572 20600 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 13. Current term is 14.
I20260709 12:23:55.887097 21008 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:55.882880 20803 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:55.897589 20992 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:55.870421 21116 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:55.899426 21027 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:55.892686 21053 consensus_queue.cc:1048] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.138s
W20260709 12:23:55.959966 16576 heartbeater.cc:646] Failed to heartbeat to 127.5.58.62:39363 (1 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.5.58.62:39363 timed out after 15.000s (SENT)
I20260709 12:23:55.913270 20797 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.020259 16441 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:56.021679 20740 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:56.065549 17270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 1209) took 1214 ms. Trace:
W20260709 12:23:56.068058 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 47912.4us
I20260709 12:23:56.069259 16441 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:56.066080 17270 rpcz_store.cc:276] 0709 12:23:54.850862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:54.851029 (+   167us) service_pool.cc:224] Handling call
0709 12:23:54.851344 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:54.996024 (+144680us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:56.065266 (+1069242us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b54007053d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:56.065477 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":234966272}
I20260709 12:23:56.038772 21129 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:55.948975 20978 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:55.946942 20734 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:56.076387 16441 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:56.037822 20992 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:56.081025 16441 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:56.044009 21102 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:23:56.067109 21024 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.012s	sys 0.025s
W20260709 12:23:56.084858 21024 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.012s	sys 0.025s
W20260709 12:23:56.084897 20981 consensus_meta.cc:287] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.921s	user 0.015s	sys 0.000s
I20260709 12:23:56.074690 21132 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:56.072083 21004 raft_consensus.cc:2749] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:56.042577 20803 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:56.091022 16477 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.023s	sys 0.067s
I20260709 12:23:56.077793 21133 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:56.100303 16477 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.023s	sys 0.067s
W20260709 12:23:56.102726 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:56.088763 16441 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:56.082231 20992 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:56.084371 20600 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:56.084295 20734 raft_consensus.cc:697] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:56.074190 20797 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:56.131459 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 331470us
I20260709 12:23:56.114874 16441 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:56.075461 20740 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:56.136113 16544 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:23:56.136907 16544 raft_consensus.cc:606] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:56.112309 17266 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:23:56.087749 20892 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:56.136240 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:56.139400 16551 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:56.145443 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:23:56.154734 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:56.155295 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:56.079252 21129 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:56.157858 16542 raft_consensus.cc:1240] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:56.162806 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:56.164644 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:56.166263 16549 raft_consensus.cc:2368] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 14. Current term is 15.
I20260709 12:23:56.163475 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:56.167068 16397 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:23:56.167363 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:56.167950 16401 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:23:56.168663 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:56.169062 16550 raft_consensus.cc:1240] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:56.137954 21132 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:56.170504 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:56.176185 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:56.179908 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:56.153586 21007 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:56.152806 21044 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:56.139035 20981 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:56.153579 21072 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.189280 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:56.189996 16545 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:23:56.190652 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:56.116559 21133 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:56.191247 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:56.165114 21053 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.171674 21038 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:56.202579 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:56.182168 21088 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:56.227702 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:56.185467 21079 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:23:56.221282 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:56.229231 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 345519us
I20260709 12:23:56.231110 17266 raft_consensus.cc:1275] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:23:56.235862 17270 raft_consensus.cc:1240] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:56.236905 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:56.238391 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:56.239287 17255 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:23:56.241444 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:56.242197 17255 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:23:56.243454 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:56.244441 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:23:56.250052 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 64178.2us
I20260709 12:23:56.253983 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:56.254771 17270 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
I20260709 12:23:56.254844 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:56.256071 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:56.257004 17266 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:56.254446 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:56.269281 17255 raft_consensus.cc:2393] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 13: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:56.196870 20981 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:56.193476 21044 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:56.243531 21038 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:23:56.282546 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 84960.3us
I20260709 12:23:56.308038 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:56.227530 21053 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:23:56.310117 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 120187us
I20260709 12:23:56.310904 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:56.198101 21072 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:56.272298 21030 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:56.320230 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:56.321019 17255 raft_consensus.cc:2368] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 12. Current term is 13.
I20260709 12:23:56.260440 21093 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:56.243817 21088 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:56.276367 21130 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.249078 21079 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:56.254498 21102 consensus_queue.cc:1048] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 12:23:56.266592 21019 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:56.332834 21019 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:56.317912 21030 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:56.348264 16516 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:56.348520 17255 raft_consensus.cc:1240] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:56.243568 20734 consensus_queue.cc:237] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:56.307318 21038 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:56.357180 16516 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:56.401582 21154 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:56.405542 21154 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:56.356003 20797 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:56.327836 21138 consensus_queue.cc:1059] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:23:56.409251 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:56.358246 20746 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:56.327211 21093 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:56.417680 16550 raft_consensus.cc:2410] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:23:56.363050 21024 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:56.418983 20670 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:56.396003 20803 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:56.425734 20670 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:56.417773 21129 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:56.432086 21030 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:56.422434 21140 log.cc:927] Time spent T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.234s	user 0.003s	sys 0.000s
W20260709 12:23:56.423394 21127 log.cc:927] Time spent T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.249s	user 0.000s	sys 0.002s
W20260709 12:23:56.470072 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 187233us
I20260709 12:23:56.432577 20740 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:56.332099 21130 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:56.492691 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:23:56.464674 21134 log.cc:927] Time spent T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 12:23:56.493718 17254 raft_consensus.cc:2393] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 17: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:56.494741 16551 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14.
I20260709 12:23:56.498466 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:56.501134 16475 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 14: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
W20260709 12:23:56.502919 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 256793us
I20260709 12:23:56.443766 20981 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:56.503750 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:56.508608 16544 raft_consensus.cc:1240] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:56.499668 20892 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:56.487619 21129 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:56.511292 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:23:56.508780 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 147143us
I20260709 12:23:56.516700 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:56.511224 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:56.460853 21148 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:56.507616 16515 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:56.533753 16474 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:56.533775 16516 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:56.567039 16366 leader_election.cc:336] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 2.094s (SENT)
I20260709 12:23:56.534134 16441 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:56.562359 16515 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:56.503008 21152 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:56.578346 21159 consensus_queue.cc:1059] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 12:23:56.578646 16441 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:56.567807 16366 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:56.585144 21152 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:56.496912 20991 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.591423 16516 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:56.557920 21044 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:56.596293 16475 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:23:56.561856 21095 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:56.568972 21121 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:56.563764 17266 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:23:56.598779 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:56.599965 16515 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:56.608963 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:56.609892 16475 raft_consensus.cc:2376] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16. Re-sending same reply.
I20260709 12:23:56.612602 16441 leader_election.cc:400] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 12. Current term is 13.
I20260709 12:23:56.612926 16441 leader_election.cc:403] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:23:56.536267 20892 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:56.618155 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:56.614711 16477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1551) took 1620 ms. Trace:
I20260709 12:23:56.619072 16477 rpcz_store.cc:276] 0709 12:23:54.994165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:54.994355 (+   190us) service_pool.cc:224] Handling call
0709 12:23:55.004812 (+ 10457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:55.468796 (+463984us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:55.468844 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:55.481478 (+ 12634us) log.cc:844] Serialized 26 byte log entry
0709 12:23:56.105251 (+623773us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:56.105296 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:56.105380 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:56.614450 (+509070us) raft_consensus.cc:1685] finished
0709 12:23:56.614479 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:56.614639 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107727,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":99,"thread_start_us":1010575,"threads_started":2,"wal-append.queue_time_us":670020}
I20260709 12:23:56.622509 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:56.560154 21148 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:56.604521 21095 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:56.626634 16474 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 14: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:56.592238 21145 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:56.609529 21121 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:56.628712 20746 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:56.597368 21149 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:56.615187 21146 raft_consensus.cc:2804] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:56.634814 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:56.633599 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:56.635921 16473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53186 (request call id 885) took 1361 ms. Trace:
I20260709 12:23:56.594544 21129 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:56.652221 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 166387us
I20260709 12:23:56.608315 20991 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:56.614444 21088 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:56.658149 16473 rpcz_store.cc:276] 0709 12:23:55.268062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.268435 (+   373us) service_pool.cc:224] Handling call
0709 12:23:56.629100 (+1360665us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540021cd54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:56.629381 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2415766400}
I20260709 12:23:56.658962 21167 consensus_queue.cc:1059] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.024s
W20260709 12:23:56.662478 21157 log.cc:927] Time spent T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 12:23:56.662571 20621 consensus_meta.cc:287] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.722s	user 0.010s	sys 0.000s
I20260709 12:23:56.662914 21093 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:56.663545 21168 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:56.663949 21163 log.cc:927] Time spent T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:23:56.694581 20892 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:56.694295 21168 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:56.694996 21165 log.cc:927] Time spent T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:23:56.695353 20892 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:56.664433 21160 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:56.696233 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:56.697053 21160 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.629204 21095 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:56.697722 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:56.638288 21121 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:56.676618 16366 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:56.677696 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:56.678853 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:56.657649 21019 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:56.640771 20746 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:23:56.666611 20992 consensus_meta.cc:287] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.529s	user 0.014s	sys 0.001s
W20260709 12:23:56.662585 21027 consensus_meta.cc:287] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.604s	user 0.003s	sys 0.006s
I20260709 12:23:56.666996 21138 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:56.693449 16477 raft_consensus.cc:1240] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:56.693682 21169 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:56.673815 16474 raft_consensus.cc:1240] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:23:56.698478 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:56.700908 17263 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:23:56.680212 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:56.656905 21146 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:56.658254 21145 raft_consensus.cc:697] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:56.716665 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:56.702466 21167 raft_consensus.cc:3055] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:56.667945 21038 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:56.666805 21149 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:23:56.708350 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 108498us
I20260709 12:23:56.676421 21088 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.719998 16473 raft_consensus.cc:1240] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:23:56.706863 20746 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:56.728262 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:56.731456 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 140116us
I20260709 12:23:56.703605 16367 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:56.711287 21019 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:23:56.737236 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 124756us
I20260709 12:23:56.713410 21027 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:56.697580 21160 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:56.713928 20992 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:56.817683 16548 raft_consensus.cc:1275] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:23:56.735966 21168 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:56.819358 16469 raft_consensus.cc:2393] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 12: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:56.742851 21028 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.819830 21129 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:56.750514 21175 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:56.820048 21028 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:56.820776 21175 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:56.822284 21028 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:56.823289 20992 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:56.750723 21156 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:23:56.753738 21136 log.cc:927] Time spent T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.560s	user 0.000s	sys 0.001s
I20260709 12:23:56.755959 21079 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:56.757311 20978 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:56.761360 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:56.763443 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:56.721932 21167 raft_consensus.cc:740] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:56.825284 21151 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:56.766208 21072 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:56.769124 16543 raft_consensus.cc:1275] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:23:56.770773 21114 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:56.771540 21133 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:56.771451 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 67945.7us
I20260709 12:23:56.772408 21173 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:56.827824 21151 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:56.830200 21173 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:56.773260 21038 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.831430 21038 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:56.772907 21044 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:56.832603 21044 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:56.828423 21114 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:56.833053 21044 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:56.833722 21038 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:56.773777 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:56.834736 16398 raft_consensus.cc:2393] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 16: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:56.834791 21114 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:56.774035 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:56.835880 16549 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:23:56.775782 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:56.836865 21173 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:56.825937 21175 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:56.819859 21160 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:56.837926 16397 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
I20260709 12:23:56.829663 16367 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:56.777623 21053 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:56.837217 21151 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:56.826772 16547 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:23:56.826473 16551 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:23:56.843477 21044 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:56.737538 21088 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:56.799228 21145 consensus_queue.cc:237] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:56.846650 21088 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:56.848353 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:56.849018 17254 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:56.849354 17264 raft_consensus.cc:1240] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:56.828981 16543 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "f99066d13e524238aec6aa2024a8bd50"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:23:56.850528 16543 raft_consensus.cc:606] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Received request to transfer leadership to TS f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:56.850983 16543 raft_consensus.cc:614] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:56.718768 21138 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:56.715710 21169 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:23:56.784402 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:56.746114 21137 log.cc:927] Time spent T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.501s	user 0.000s	sys 0.001s
I20260709 12:23:56.705808 21093 consensus_queue.cc:1048] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:23:56.790946 20892 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 12. Current term is 13.
I20260709 12:23:56.791388 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:56.795490 21152 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:56.717845 21146 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:56.699050 20621 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:56.777827 20991 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:56.801246 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:56.806463 20715 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.807399 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:56.873322 21169 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:56.874643 21154 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:56.854447 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:56.877635 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:56.879266 17261 raft_consensus.cc:2393] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 12: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:56.880069 16477 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
I20260709 12:23:56.880791 20715 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:56.803926 21130 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:56.764384 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:56.827576 21167 consensus_queue.cc:260] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:56.733012 21149 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:56.880841 17261 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45228 (request call id 1211) took 1028 ms. Trace:
I20260709 12:23:56.890733 21159 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:56.903731 17261 rpcz_store.cc:276] 0709 12:23:55.852456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.852628 (+   172us) service_pool.cc:224] Handling call
0709 12:23:56.880652 (+1028024us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54005f3554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:56.880786 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2337639808}
I20260709 12:23:56.904728 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:56.904870 21159 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:56.906780 16470 raft_consensus.cc:1240] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:23:56.907100 21159 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:56.899293 21020 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:56.853883 21148 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:56.908797 21167 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:56.841194 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:56.837282 16549 raft_consensus.cc:1275] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:56.911832 21162 raft_consensus.cc:3055] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:56.912322 21162 raft_consensus.cc:740] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:56.852200 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:56.913182 21162 consensus_queue.cc:260] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:56.913925 21138 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:56.914579 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:56.914873 16545 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:56.859047 21143 log.cc:927] Time spent T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.607s	user 0.005s	sys 0.000s
I20260709 12:23:56.909034 17271 raft_consensus.cc:1240] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:56.915345 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:56.918412 20621 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:56.920920 21044 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:56.885290 16474 raft_consensus.cc:3055] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:56.922055 16546 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:56.896903 16514 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:56.847378 21088 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:56.825340 20992 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.924307 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:56.924950 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:56.925714 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:23:56.927402 16367 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.4:44491 timed out after 1.532s (SENT) [suppressed 18 similar messages]
I20260709 12:23:56.914592 16471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1553) took 1854 ms. Trace:
I20260709 12:23:56.938431 16545 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
I20260709 12:23:56.923355 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:57.011257 17261 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
W20260709 12:23:56.922446 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:56.926350 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:23:56.948760 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 51974.4us
I20260709 12:23:56.940022 16474 raft_consensus.cc:740] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:57.017241 16468 raft_consensus.cc:1275] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:57.017066 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:57.018604 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:57.009647 16471 rpcz_store.cc:276] 0709 12:23:55.060440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.060631 (+   191us) service_pool.cc:224] Handling call
0709 12:23:55.091296 (+ 30665us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:55.432589 (+341293us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:56.914374 (+1481785us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b54004c4ad4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:56.914534 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":783030144}
I20260709 12:23:57.009825 16442 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:57.023211 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:57.018357 16542 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:23:56.739761 20763 raft_consensus.cc:3055] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:57.027647 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:57.029517 16468 raft_consensus.cc:1275] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:57.013926 16474 consensus_queue.cc:260] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:57.037515 16474 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:57.040032 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:57.044209 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:57.049387 16545 raft_consensus.cc:2393] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 14: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:57.039187 16514 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:56.949227 20674 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:57.054692 17261 raft_consensus.cc:1240] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:23:57.056274 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:56.949271 21088 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:56.940161 20621 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.058658 17264 raft_consensus.cc:1275] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:23:56.999636 21053 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:57.063985 21053 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:57.064451 21053 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:57.074050 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:56.851212 20670 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:57.081017 20621 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:57.081862 21095 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:57.082752 20670 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:57.083202 20621 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:57.026299 20992 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:57.085933 20600 consensus_meta.cc:287] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.949s	user 0.009s	sys 0.004s
I20260709 12:23:57.086090 20992 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:57.086377 20600 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:57.004815 20892 consensus_queue.cc:1059] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.082s
I20260709 12:23:57.087101 16475 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:23:57.087445 21146 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:57.087798 20892 raft_consensus.cc:3055] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:57.038267 20991 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:23:57.089154 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:57.110918 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 380421us
I20260709 12:23:57.100037 17143 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:57.113507 16366 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:57.089586 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:57.089663 16476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1557) took 1657 ms. Trace:
I20260709 12:23:57.115765 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:57.090384 16470 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:23:57.116854 16542 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:57.090512 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:57.117877 16545 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:23:57.091117 17266 raft_consensus.cc:1240] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:57.091224 21180 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:57.083858 21088 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:57.094254 21162 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:57.096689 16550 raft_consensus.cc:2393] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 16: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:57.092252 21042 consensus_queue.cc:1059] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 12:23:57.097568 21095 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:57.129436 16223 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53636 (request call id 132) took 38317 ms (client timeout 15000 ms). Trace:
I20260709 12:23:57.130493 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:57.132118 16474 raft_consensus.cc:1275] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 12:23:57.132635 16398 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:57.132728 16474 pending_rounds.cc:85] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 9
I20260709 12:23:57.133122 16471 raft_consensus.cc:1275] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:23:57.133208 16474 pending_rounds.cc:107] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 12:23:57.105427 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:57.133756 16474 raft_consensus.cc:2889] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:57.036067 20763 raft_consensus.cc:740] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:57.109879 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:57.119062 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:57.136047 16544 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
I20260709 12:23:57.115710 16476 rpcz_store.cc:276] 0709 12:23:55.432443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.441836 (+  9393us) service_pool.cc:224] Handling call
0709 12:23:55.479329 (+ 37493us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:55.479722 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:55.479756 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:55.487041 (+  7285us) log.cc:844] Serialized 26 byte log entry
0709 12:23:56.041006 (+553965us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:56.041054 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:56.041130 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:56.745159 (+704029us) spinlock_profiling.cc:243] Waited 20.4 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:23:57.061254 (+316095us) raft_consensus.cc:1685] finished
0709 12:23:57.061276 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:57.089574 (+ 28298us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":48555,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":23164,"spinlock_wait_cycles":46987520,"thread_start_us":426410,"threads_started":1,"wal-append.queue_time_us":637283}
I20260709 12:23:57.126322 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:57.129541 16469 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
I20260709 12:23:57.153748 16549 raft_consensus.cc:1240] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:57.145187 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:57.138823 16397 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
I20260709 12:23:57.161113 17143 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:57.163858 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:57.165850 20715 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:57.102052 20674 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:57.170812 16551 raft_consensus.cc:1275] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:23:57.171548 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:57.172688 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:57.173908 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:57.174928 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:57.175990 16398 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:23:57.176477 16477 raft_consensus.cc:2410] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:23:57.146315 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:57.173360 16470 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:23:57.178965 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:57.180325 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:57.114894 20892 raft_consensus.cc:740] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:57.188971 16545 raft_consensus.cc:1240] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:57.171383 21141 consensus_queue.cc:1048] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 12:23:57.183284 20674 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:57.194319 21169 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:57.172112 21138 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:57.171844 21116 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 12:23:57.199625 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:57.200575 16476 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
I20260709 12:23:57.127892 20621 raft_consensus.cc:2804] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:57.186753 16547 raft_consensus.cc:2393] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 15: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:57.187801 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:57.179953 17143 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:57.191232 16543 raft_consensus.cc:2393] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 12: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:57.126271 21044 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:57.192121 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:57.171872 21167 consensus_queue.cc:1059] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 12:23:57.197921 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:57.198645 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:57.128819 21174 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:57.192090 21130 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:57.183480 20746 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:57.184917 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:57.201850 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:57.196794 21138 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:57.205749 17143 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:57.178393 20978 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.199262 16542 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:23:57.230980 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:57.217775 16475 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:57.232367 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:57.218762 16471 raft_consensus.cc:2393] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 12: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:57.216325 16470 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:23:57.233956 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:57.234650 16470 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:23:57.224165 20674 raft_consensus.cc:3055] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:57.223925 20763 consensus_queue.cc:260] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:57.131393 16223 rpcz_store.cc:269] 0709 12:23:18.816092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:18.856484 (+ 40392us) service_pool.cc:224] Handling call
0709 12:23:57.129337 (+38272853us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:52.122181 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:52.493747 (+371566us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:52.493796 (+    49us) write_op.cc:435] Acquired schema lock
0709 12:23:52.493813 (+    17us) tablet.cc:662] Decoding operations
0709 12:23:52.495207 (+  1394us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:52.495322 (+   115us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:52.495364 (+    42us) tablet.cc:685] Acquiring locks for 10 operations
0709 12:23:52.534566 (+ 39202us) tablet.cc:701] Row locks acquired
0709 12:23:52.534631 (+    65us) write_op.cc:260] PREPARE: finished
0709 12:23:52.538738 (+  4107us) write_op.cc:270] Start()
0709 12:23:52.538918 (+   180us) write_op.cc:276] Timestamp: P: 1783599832534839 usec, L: 0
0709 12:23:52.538940 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:23:52.539632 (+   692us) log.cc:844] Serialized 3943 byte log entry
0709 12:23:53.256965 (+717333us) op_driver.cc:464] REPLICATION: finished
0709 12:23:53.507159 (+250194us) write_op.cc:301] APPLY: starting
0709 12:23:53.507280 (+   121us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:53.690455 (+183175us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:53.690606 (+   151us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:54.361596 (+670990us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:54.361988 (+   392us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=0,mrs_lookups=0
0709 12:23:54.362063 (+    75us) write_op.cc:312] APPLY: finished
0709 12:23:54.362550 (+   487us) log.cc:844] Serialized 96 byte log entry
0709 12:23:54.593482 (+230932us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:54.597024 (+  3542us) write_op.cc:454] Released schema lock
0709 12:23:54.597103 (+    79us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:54.597642 (+   539us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:54.597675 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2373025,"reported_tablets":41,"child_traces":[["op",{"apply.queue_time_us":234238,"apply.run_cpu_time_us":32825,"apply.run_wall_time_us":1090430,"cfile_cache_hit":38,"cfile_cache_hit_bytes":39748,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6219,"cfile_init":1,"lbm_read_time_us":825,"lbm_reads_lt_1ms":8,"num_ops":10,"prepare.queue_time_us":486300,"prepare.run_cpu_time_us":15247,"prepare.run_wall_time_us":442400,"replication_time_us":717781,"thread_start_us":649434,"threads_started":3,"wal-append.queue_time_us":352748,"wal-append.run_cpu_time_us":1202,"wal-append.run_wall_time_us":1006681}]]}
I20260709 12:23:57.237876 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:57.226486 21146 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:57.227515 21121 consensus_meta.cc:287] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.516s	user 0.008s	sys 0.004s
I20260709 12:23:57.216976 20621 raft_consensus.cc:697] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:57.215749 21027 raft_consensus.cc:993] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1: : Instructing follower 52ab50b0237346c1a13ede947f6a63c2 to start an election
I20260709 12:23:57.229033 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:57.234862 16366 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:57.231597 16477 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 18.
I20260709 12:23:57.217423 20831 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:57.223816 21169 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:57.197244 21053 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:57.250921 16441 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:57.252352 16366 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:57.268911 16366 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:57.269377 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:57.269472 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:57.270057 17269 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:57.270453 17264 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
I20260709 12:23:57.273597 17143 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:57.273834 16366 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:57.236101 20674 raft_consensus.cc:740] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:57.236583 21130 raft_consensus.cc:697] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:57.265383 20803 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:57.266029 20763 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:57.264920 20831 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:57.268406 21044 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:57.286652 16441 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:57.303500 20992 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:57.305110 16441 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:57.306401 20992 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:57.308413 16441 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:57.295158 16366 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:57.229290 21174 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:57.237231 20978 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:57.272670 21167 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:57.290489 17269 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:23:57.286412 20803 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:57.279120 21177 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:57.278005 21141 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:57.276077 21093 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:23:57.327031 17255 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.019s	sys 0.025s
I20260709 12:23:57.257345 21149 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:57.250324 21121 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:57.280221 21027 raft_consensus.cc:1081] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Signalling peer 52ab50b0237346c1a13ede947f6a63c2 to start an election
I20260709 12:23:57.297410 21148 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:57.354156 21121 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:57.297636 20831 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:57.356642 16473 raft_consensus.cc:1240] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:57.358327 16472 raft_consensus.cc:1240] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:57.313637 21044 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:57.359160 21044 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:57.346340 17255 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.020s	sys 0.026s
I20260709 12:23:57.284933 21169 consensus_queue.cc:1059] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
W20260709 12:23:57.349783 16368 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.003s	sys 0.033s
I20260709 12:23:57.343624 20674 consensus_queue.cc:260] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:57.366523 20978 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:57.396376 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 176284us
I20260709 12:23:57.395017 21004 consensus_queue.cc:1048] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
W20260709 12:23:57.437358 17144 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.800s	user 0.025s	sys 0.066s
I20260709 12:23:57.416400 16473 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:34826
I20260709 12:23:57.386539 20991 consensus_queue.cc:1059] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 12:23:57.352066 21149 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:57.407819 16472 raft_consensus.cc:1240] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 11. Current term is 12. Ops: []
W20260709 12:23:57.387821 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 92726.9us
W20260709 12:23:57.439572 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:57.348658 20981 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:57.395268 16441 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:23:57.442143 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 46987.9us
I20260709 12:23:57.428985 16514 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
W20260709 12:23:57.432514 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 504244us
I20260709 12:23:57.337653 20992 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:57.346582 20803 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:57.434491 16368 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.831s	user 0.004s	sys 0.042s
W20260709 12:23:57.371553 21082 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.011s	sys 0.027s
I20260709 12:23:57.413710 21174 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:57.440649 17144 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.026s	sys 0.066s
W20260709 12:23:57.462098 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 828829us
I20260709 12:23:57.446233 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:57.431437 21145 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.008s	sys 0.007s
I20260709 12:23:57.449916 16473 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:57.455078 16366 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:57.393252 21130 consensus_queue.cc:237] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:57.457468 17269 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7fe8595f696842c8821880524389c68b"
dest_uuid: "ef57beab0e1d40d585983873c55107e8"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:45120
I20260709 12:23:57.374140 21141 raft_consensus.cc:697] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 16 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:57.462512 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:57.430986 20746 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:57.463881 17142 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:57.465862 17269 raft_consensus.cc:606] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:57.465307 16473 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:57.471889 17269 raft_consensus.cc:614] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:57.466734 16403 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
I20260709 12:23:57.466315 17144 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:57.461355 21082 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.012s	sys 0.034s
I20260709 12:23:57.483383 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:57.484709 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:57.459643 21169 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:57.453760 20981 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:57.471156 16441 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:57.432026 20715 consensus_queue.cc:1048] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
W20260709 12:23:57.471081 21145 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.008s	sys 0.007s
I20260709 12:23:57.463924 21184 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.463784 21042 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:57.486790 16395 raft_consensus.cc:1275] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:23:57.485874 16398 raft_consensus.cc:2410] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:23:57.481885 20674 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:57.511795 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:57.491015 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:57.506613 20746 consensus_queue.cc:1059] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 12:23:57.474819 21053 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:57.482125 21174 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:57.527782 16366 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:57.514941 16472 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:23:57.498008 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:57.507814 16516 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:57.486104 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:57.464627 20670 consensus_queue.cc:1059] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:23:57.515101 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:57.529894 16516 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:57.521356 21042 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:57.492797 21167 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.493558 21169 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:57.484838 21138 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:57.493839 21148 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:57.537853 20797 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:57.512760 20981 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:57.525866 16473 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:57.522639 21180 consensus_queue.cc:1059] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 12:23:57.469995 21153 consensus_queue.cc:1048] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:23:57.554143 17261 raft_consensus.cc:1275] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:23:57.554769 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:57.555672 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:23:57.566097 21179 log.cc:927] Time spent T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 12:23:57.566438 20991 consensus_queue.cc:1059] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:23:57.600420 21169 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:57.566936 21145 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:57.600111 20992 raft_consensus.cc:2764] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 12:23:57.571695 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:57.560660 20892 consensus_queue.cc:260] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:57.576452 21141 consensus_queue.cc:237] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:57.589584 20670 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:23:57.567201 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:57.615729 16396 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:57.564180 21151 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.565876 21042 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:57.537451 21192 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:57.537659 21174 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:57.494417 21024 consensus_queue.cc:1059] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:23:57.516146 21184 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:57.543768 21027 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:57.608877 20740 raft_consensus.cc:3055] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:57.575172 21181 raft_consensus.cc:3055] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:57.637079 20740 raft_consensus.cc:740] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:57.589828 17269 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:57.638228 20740 consensus_queue.cc:260] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:57.639532 20740 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:57.573380 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:57.618300 16401 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
W20260709 12:23:57.568929 21186 log.cc:927] Time spent T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 12:23:57.607774 17255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 1189) took 1208 ms. Trace:
I20260709 12:23:57.642349 17255 rpcz_store.cc:276] 0709 12:23:56.399882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.400327 (+   445us) service_pool.cc:224] Handling call
0709 12:23:56.764859 (+364532us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:56.765210 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:56.765241 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:56.765711 (+   470us) log.cc:844] Serialized 26 byte log entry
0709 12:23:57.361198 (+595487us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:57.361400 (+   202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.361497 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:57.585898 (+224401us) raft_consensus.cc:1685] finished
0709 12:23:57.585917 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:57.607504 (+ 21587us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b5400210050. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:23:57.607711 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5180,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"spinlock_wait_cycles":837132544,"thread_start_us":595093,"threads_started":1,"wal-append.queue_time_us":818615}
I20260709 12:23:57.628810 16551 raft_consensus.cc:1240] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:57.627708 16472 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:23:57.604343 21145 raft_consensus.cc:697] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:57.647324 21145 consensus_queue.cc:237] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:57.647760 20797 consensus_queue.cc:1059] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:57.581394 21148 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:57.648799 21148 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:57.648957 20978 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:57.649776 20978 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:57.580767 21138 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:57.652876 21138 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:57.653323 21138 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:57.624750 21151 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:57.654244 20831 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:57.656697 21151 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:57.568202 21167 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:57.657415 17266 raft_consensus.cc:1240] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 14. Current term is 16. Ops: []
I20260709 12:23:57.657485 16399 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:23:57.658548 21167 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:57.659627 16402 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:23:57.660269 21174 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:57.654240 20991 consensus_queue.cc:1048] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 12:23:57.660898 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:57.662977 16514 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:57.663982 21191 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:57.664499 21191 raft_consensus.cc:697] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:57.665335 21191 consensus_queue.cc:237] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:57.676622 16396 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:23:57.610494 16398 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:57.566295 21004 consensus_queue.cc:1048] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 12:23:57.637194 21181 raft_consensus.cc:740] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:57.624181 21116 raft_consensus.cc:3055] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:57.621295 21192 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:57.697455 21192 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:57.648797 20797 raft_consensus.cc:3055] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:57.700855 20797 raft_consensus.cc:740] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:57.700971 21177 raft_consensus.cc:3055] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:57.701401 21177 raft_consensus.cc:740] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:57.701809 20797 consensus_queue.cc:260] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:57.702276 21177 consensus_queue.cc:260] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:57.703295 20797 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:57.703933 21177 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:57.706621 20978 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:57.709242 21042 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:57.710125 20803 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:57.693593 21088 consensus_meta.cc:287] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.561s	user 0.012s	sys 0.000s
I20260709 12:23:57.713696 21088 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:57.694743 17269 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:23:57.693440 16400 raft_consensus.cc:1275] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:23:57.694327 16473 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:57.692554 21182 raft_consensus.cc:3055] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:57.717521 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:57.706611 16398 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
I20260709 12:23:57.696863 21024 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:57.699644 21181 consensus_queue.cc:260] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:57.677362 21027 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:57.720170 16441 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:57.693799 21116 raft_consensus.cc:740] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:57.696120 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:57.723557 17255 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:57.721228 16472 raft_consensus.cc:1240] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:57.590909 20621 consensus_queue.cc:237] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:57.718891 21182 raft_consensus.cc:740] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:23:57.724109 21159 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.013s	sys 0.037s
W20260709 12:23:57.739176 21159 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.013s	sys 0.037s
I20260709 12:23:57.739410 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:57.742225 16551 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:57.739192 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:57.744166 16545 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:57.718622 21044 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:57.767536 20803 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:57.733199 20600 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:57.734866 20831 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:57.739916 21181 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:57.787649 20981 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:57.723148 21024 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:57.790521 17255 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:23:57.741830 20740 consensus_queue.cc:1059] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 12:23:57.794486 21044 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:57.801276 16403 raft_consensus.cc:2393] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 14: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:57.804945 21088 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:57.806859 21027 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:57.807662 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:57.812018 16545 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:23:57.830273 20981 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:57.833792 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:57.846029 16551 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
W20260709 12:23:57.846546 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:57.847812 16441 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:57.849602 17143 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:57.854099 17143 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:57.854525 16394 raft_consensus.cc:2410] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:23:57.837142 20600 raft_consensus.cc:697] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:23:57.858031 21181 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:57.861094 21181 raft_consensus.cc:697] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:57.856828 21121 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:57.856786 20831 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:57.882616 21024 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:57.888482 21181 consensus_queue.cc:237] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:23:57.898262 16476 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.008s	sys 0.032s
W20260709 12:23:57.906566 16476 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.008s	sys 0.032s
W20260709 12:23:57.905614 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 439334us
W20260709 12:23:57.961189 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 498013us
I20260709 12:23:57.962709 16400 raft_consensus.cc:1275] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:57.968252 16398 raft_consensus.cc:1275] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:23:57.881753 21121 raft_consensus.cc:697] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:23:57.973160 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:57.962700 17142 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:57.893350 20831 raft_consensus.cc:697] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 16 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:57.985653 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:57.991274 16401 raft_consensus.cc:2393] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 16: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:57.986824 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:57.992313 16400 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
I20260709 12:23:57.997284 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:57.999058 16400 raft_consensus.cc:1240] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:58.000828 21004 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.001534 21004 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:58.002760 21042 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:57.995972 21153 consensus_queue.cc:1048] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 12:23:57.981225 21155 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:58.004263 21004 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:58.005654 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:58.006829 17255 raft_consensus.cc:2410] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:23:57.914187 21184 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:58.030879 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:57.982092 21102 consensus_queue.cc:1059] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:23:58.033174 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:58.034194 16476 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:58.039215 21116 consensus_queue.cc:260] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:58.040629 21116 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:58.046043 21182 consensus_queue.cc:260] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:58.047755 21182 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:58.052605 20831 consensus_queue.cc:237] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:58.015825 20600 consensus_queue.cc:237] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:58.055891 20892 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:58.058152 21148 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:58.061358 16402 raft_consensus.cc:1240] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:58.055859 16476 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
W20260709 12:23:58.071725 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 92815.2us
I20260709 12:23:58.073930 17142 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:23:58.081866 16514 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:58.092701 16551 raft_consensus.cc:1275] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:58.093544 21102 raft_consensus.cc:3055] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:58.093973 21102 raft_consensus.cc:740] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:23:58.094782 21102 consensus_queue.cc:260] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:58.093040 21121 consensus_queue.cc:237] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:58.099417 17142 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:58.105111 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:58.106609 16394 raft_consensus.cc:2368] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 11. Current term is 12.
W20260709 12:23:58.108278 16367 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.016s	sys 0.020s
W20260709 12:23:58.108707 16367 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.016s	sys 0.020s
W20260709 12:23:58.109098 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 534974us
I20260709 12:23:58.113837 16394 raft_consensus.cc:1275] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:23:58.124464 16367 leader_election.cc:400] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 14. Current term is 15.
I20260709 12:23:58.104638 20670 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:23:58.127497 16471 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.013s	sys 0.025s
W20260709 12:23:58.127907 16515 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.018s	sys 0.023s
W20260709 12:23:58.134124 16515 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.018s	sys 0.023s
W20260709 12:23:58.134086 16471 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.013s	sys 0.026s
I20260709 12:23:58.107777 21146 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:23:58.134799 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 599210us
I20260709 12:23:58.141237 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:23:58.143973 16366 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.000s	sys 0.043s
W20260709 12:23:58.144485 16366 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.000s	sys 0.043s
W20260709 12:23:58.144830 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 617082us
W20260709 12:23:58.136662 21082 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.011s	sys 0.027s
W20260709 12:23:58.149020 21082 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.011s	sys 0.027s
I20260709 12:23:58.149255 16402 raft_consensus.cc:1240] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:58.111248 21199 consensus_queue.cc:1048] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 12:23:58.157704 16476 raft_consensus.cc:1240] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:58.105336 21192 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:58.160264 16546 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:58.161404 21192 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:23:58.162518 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:58.163435 16366 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:58.134353 20670 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:58.165251 20670 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:58.166064 21138 consensus_meta.cc:287] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.512s	user 0.011s	sys 0.000s
I20260709 12:23:58.166574 21138 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:58.167893 16395 raft_consensus.cc:2393] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 13: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:58.169189 21184 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:23:58.169761 21184 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:58.170059 21198 consensus_queue.cc:1048] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:58.170624 21184 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:58.171595 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:58.174834 20674 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.175515 20674 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:23:58.176833 21169 consensus_meta.cc:287] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.576s	user 0.013s	sys 0.000s
I20260709 12:23:58.177335 21169 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:58.179533 21169 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:58.178040 20674 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:58.184245 16397 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.011s	sys 0.034s
W20260709 12:23:58.186218 16397 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.011s	sys 0.034s
I20260709 12:23:58.180684 21138 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:23:58.183765 21195 log.cc:927] Time spent T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:23:58.144192 21148 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:58.235919 16475 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.011s	sys 0.033s
W20260709 12:23:59.240175 16475 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.607s	user 0.016s	sys 0.033s
W20260709 12:23:58.528352 21216 log.cc:927] Time spent T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 12:23:58.902801 17266 raft_consensus.cc:1240] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:59.241166 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:23:58.909706 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:59.004652 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:59.023025 21219 log.cc:927] Time spent T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.539s	user 0.005s	sys 0.000s
I20260709 12:23:59.243872 21233 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:58.137351 16367 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:59.045154 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:59.053525 16397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 1302) took 1530 ms. Trace:
I20260709 12:23:59.244454 21233 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:59.058317 21231 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:59.245294 16402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58430 (request call id 1240) took 1075 ms. Trace:
I20260709 12:23:59.063457 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:59.246001 16476 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:23:59.064872 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:59.066916 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:59.247066 16472 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:59.077433 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:59.078418 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:23:59.083680 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:59.249775 17271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 1208) took 1696 ms. Trace:
W20260709 12:23:59.083875 17142 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.968s	user 0.005s	sys 0.017s
I20260709 12:23:59.093649 21232 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.095779 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:59.097373 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:23:59.098289 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:59.099372 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:59.099831 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:59.267575 16469 raft_consensus.cc:2393] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 13: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:23:59.102155 17264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 1213) took 1437 ms. Trace:
I20260709 12:23:59.250188 17271 rpcz_store.cc:276] 0709 12:23:57.553479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.564943 (+ 11464us) service_pool.cc:224] Handling call
0709 12:23:57.565270 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:57.587324 (+ 22054us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:57.587356 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:57.587829 (+   473us) log.cc:844] Serialized 26 byte log entry
0709 12:23:58.030235 (+442406us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:58.030304 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:58.030535 (+   231us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:59.249584 (+1219049us) raft_consensus.cc:1685] finished
0709 12:23:59.249604 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:59.249723 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21640,"raft.queue_time_us":22803,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":121,"thread_start_us":441892,"threads_started":1,"wal-append.queue_time_us":653000}
I20260709 12:23:59.268756 17264 rpcz_store.cc:276] 0709 12:23:57.664500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.668445 (+  3945us) service_pool.cc:224] Handling call
0709 12:23:57.668832 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:57.669205 (+   373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:57.669236 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:57.669854 (+   618us) log.cc:844] Serialized 26 byte log entry
0709 12:23:58.111432 (+441578us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:58.111473 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:58.111536 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:59.101929 (+990393us) raft_consensus.cc:1685] finished
0709 12:23:59.101950 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:59.102095 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2306,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"thread_start_us":441122,"threads_started":1,"wal-append.queue_time_us":619753}
W20260709 12:23:59.115411 17145 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.980s	user 0.020s	sys 0.061s
W20260709 12:23:59.116391 17143 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.983s	user 0.011s	sys 0.009s
I20260709 12:23:59.129523 16546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 954) took 1525 ms. Trace:
I20260709 12:23:59.271631 16546 rpcz_store.cc:276] 0709 12:23:57.604085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.604697 (+   612us) service_pool.cc:224] Handling call
0709 12:23:57.605053 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:58.160771 (+555718us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:59.129191 (+968420us) spinlock_profiling.cc:243] Waited 555 ms on lock 0x7b540066e9d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:59.129443 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1276345088}
I20260709 12:23:59.103557 20978 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:59.355309 20978 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:59.104913 21053 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:59.363747 21053 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:59.365865 21053 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:59.089687 21093 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.367067 21093 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:59.369128 21093 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:59.369551 21240 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:59.370179 21240 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:59.094370 21102 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:23:58.238657 21130 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.042s	sys 0.035s
W20260709 12:23:59.375011 21130 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.842s	user 0.043s	sys 0.036s
I20260709 12:23:58.294041 20670 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:59.391010 20670 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:59.084071 21027 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:59.144500 21138 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.393146 21138 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:59.394650 21241 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.395448 21138 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:59.143824 21148 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:59.397398 21148 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:59.400429 21027 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:59.402878 21027 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:58.184190 21155 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:58.203299 21169 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.405048 21169 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:59.407377 21169 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:59.408509 21169 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.409183 21169 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:59.411070 21169 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:59.411495 21169 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:59.084587 21175 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.413187 21175 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:59.415230 21175 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:59.039705 21177 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:59.416389 21177 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:59.418553 21177 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:58.533002 21184 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:59.421059 21184 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:59.104651 21191 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.422433 21191 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:58.219115 21192 consensus_queue.cc:1059] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.049s
W20260709 12:23:58.525264 21200 log.cc:927] Time spent T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
W20260709 12:23:58.527285 21201 log.cc:927] Time spent T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
W20260709 12:23:58.426759 21207 log.cc:927] Time spent T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 12:23:58.527139 21214 log.cc:927] Time spent T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
W20260709 12:23:59.126052 21215 log.cc:927] Time spent T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.759s	user 0.005s	sys 0.004s
W20260709 12:23:58.204620 21141 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.019s	sys 0.015s
W20260709 12:23:59.449396 21141 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.807s	user 0.020s	sys 0.016s
W20260709 12:23:58.220479 21145 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.019s	sys 0.016s
W20260709 12:23:58.486158 21208 log.cc:927] Time spent T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.122s	user 0.001s	sys 0.001s
W20260709 12:23:58.488173 21213 log.cc:927] Time spent T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:23:59.154184 21182 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:59.189060 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:59.189999 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:59.456817 16468 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:59.456966 16477 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:23:59.458130 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:59.191819 16543 raft_consensus.cc:1240] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:59.207168 21235 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.459625 21235 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:59.210566 21095 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.235661 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:59.245736 21231 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:59.461171 16473 raft_consensus.cc:2393] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 14: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:23:59.460990 21095 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:59.462222 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:59.462574 21231 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:59.252300 16542 raft_consensus.cc:1240] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:59.241178 16475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1484) took 2022 ms. Trace:
I20260709 12:23:59.245733 16402 rpcz_store.cc:276] 0709 12:23:58.169619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:58.184724 (+ 15105us) service_pool.cc:224] Handling call
0709 12:23:59.245211 (+1060487us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:59.463471 21095 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:59.262038 16551 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
W20260709 12:23:59.262672 17142 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.147s	user 0.007s	sys 0.019s
I20260709 12:23:59.465590 16468 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
W20260709 12:23:59.277523 17145 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.142s	user 0.022s	sys 0.063s
I20260709 12:23:59.278402 21232 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:59.280982 16472 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:23:59.463311 16475 rpcz_store.cc:276] 0709 12:23:57.218940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.239921 (+ 20981us) service_pool.cc:224] Handling call
0709 12:23:57.240223 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:57.240635 (+   412us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:23:57.632991 (+392356us) log.cc:844] Serialized 14 byte log entry
0709 12:23:59.240800 (+1607809us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:59.240874 (+    74us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:59.240920 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:59.240984 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:59.241120 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":842,"prepare.queue_time_us":740536,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":11765,"raft.queue_time_us":2866,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":122,"thread_start_us":1998128,"threads_started":2,"wal-append.queue_time_us":761349}
W20260709 12:23:59.283560 17143 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.150s	user 0.011s	sys 0.009s
I20260709 12:23:59.470459 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:59.470620 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 1.33718e+06us
I20260709 12:23:59.471572 21232 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:23:59.284828 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 1.14748e+06us
I20260709 12:23:59.473681 16367 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:23:59.474452 21232 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:59.475477 21232 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:23:59.475998 21232 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:59.455478 21155 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:23:59.290484 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 54221.4us
I20260709 12:23:59.479326 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:59.261162 20763 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:59.301872 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:59.305968 21238 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:59.306916 21004 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:59.482793 21184 consensus_queue.cc:1048] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:59.484390 21004 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:59.313791 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:59.485797 21231 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.486122 16394 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
I20260709 12:23:59.314162 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:59.486322 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:59.486974 16395 raft_consensus.cc:2393] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 14: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:23:59.486435 21231 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:59.314886 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:59.321857 21239 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.316849 21233 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:59.315922 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:59.333025 21246 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.488255 21239 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:59.333091 16366 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:59.489363 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:59.489738 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 156763us
I20260709 12:23:59.491046 16441 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:23:59.339484 21121 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.491098 16366 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:59.347890 16476 raft_consensus.cc:1275] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:23:59.345130 21236 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.494645 21198 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.495195 21198 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:59.494946 21236 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:59.496181 21141 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.495234 21249 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:59.496968 21141 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:59.346583 21247 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.499459 21141 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:59.361972 20978 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:59.361856 16221 catalog_manager.cc:5697] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 8 to 11, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 11 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:23:59.353843 21243 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.500196 21121 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:59.501004 21243 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:59.501067 16221 catalog_manager.cc:5697] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
W20260709 12:23:59.386834 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:59.387524 21250 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.493175 21251 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.489365 21246 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:59.395220 21241 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:58.147313 21146 raft_consensus.cc:697] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:59.383330 21245 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.399892 21148 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:59.429286 21191 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:59.411911 21169 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:59.434336 21082 raft_consensus.cc:3055] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:59.441417 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:59.384939 21248 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:59.447535 16471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1580) took 1863 ms. Trace:
W20260709 12:23:59.451372 21145 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.797s	user 0.020s	sys 0.017s
I20260709 12:23:59.456655 21182 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:59.213539 20734 raft_consensus.cc:3055] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:59.503887 21121 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:59.505596 21095 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.515355 20734 raft_consensus.cc:740] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:59.515120 21169 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:59.516163 21095 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:59.516669 21182 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:59.503015 21250 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:59.465646 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 1.36618e+06us
I20260709 12:23:59.510825 21192 consensus_queue.cc:1048] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 12:23:59.245081 16397 rpcz_store.cc:276] 0709 12:23:57.523250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.523403 (+   153us) service_pool.cc:224] Handling call
0709 12:23:57.523818 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:57.524109 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:23:57.567569 (+ 43460us) log.cc:844] Serialized 14 byte log entry
0709 12:23:59.053093 (+1485524us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:59.053163 (+    70us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:59.053224 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:59.053307 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:59.053455 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":41885,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":522,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":131,"thread_start_us":1485003,"threads_started":2,"wal-append.queue_time_us":870999}
I20260709 12:23:59.481268 16470 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:59.518656 21095 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:59.503141 21251 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:59.510635 21245 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:59.520315 17142 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:59.521502 21245 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:59.511785 16473 raft_consensus.cc:2468] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 9.
I20260709 12:23:59.476186 16367 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
W20260709 12:23:59.523041 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 46946.5us
I20260709 12:23:59.512887 21248 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:59.523840 16367 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:59.524152 21004 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:59.511962 21082 raft_consensus.cc:740] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:23:59.524890 21138 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:59.525727 16367 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:59.525272 21082 consensus_queue.cc:260] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:59.483700 21130 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.485580 20763 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:59.325614 17266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 1215) took 1611 ms. Trace:
I20260709 12:23:59.343757 16469 raft_consensus.cc:1240] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 11. Current term is 13. Ops: []
I20260709 12:23:59.490477 16475 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
W20260709 12:23:59.493712 16515 leader_election.cc:336] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Timed out: RequestConsensusVote RPC to 127.5.58.1:41805 timed out after 2.147s (SENT)
I20260709 12:23:59.494256 21238 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:59.494385 21233 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:59.494769 21239 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:59.499070 21247 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:59.372464 21240 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:59.495985 21231 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:59.566954 16515 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:23:59.261797 17144 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.145s	user 0.023s	sys 0.006s
I20260709 12:23:59.522028 21251 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:59.511624 21198 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:59.569972 21198 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:59.570429 21198 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:59.570901 21247 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:59.571969 21240 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:59.572600 21240 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:59.572935 21198 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:59.525775 21248 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:59.573424 21238 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:59.556648 21236 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:59.573868 21238 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:59.585227 17144 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.468s	user 0.023s	sys 0.007s
I20260709 12:23:59.480754 21235 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:23:59.481938 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:59.527247 16367 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:59.513475 16471 rpcz_store.cc:276] 0709 12:23:57.583714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.588442 (+  4728us) service_pool.cc:224] Handling call
0709 12:23:57.620267 (+ 31825us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:57.621148 (+   881us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:57.621193 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:57.621903 (+   710us) log.cc:844] Serialized 26 byte log entry
0709 12:23:58.135122 (+513219us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:58.135167 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:58.135787 (+   620us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:59.447296 (+1311509us) raft_consensus.cc:1685] finished
0709 12:23:59.447319 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:59.447471 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":109,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":4962,"thread_start_us":512867,"threads_started":1,"wal-append.queue_time_us":696362}
I20260709 12:23:59.529430 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:59.494670 16366 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:59.514954 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 167274us
I20260709 12:23:59.605796 16473 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
W20260709 12:23:59.605887 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 111284us
I20260709 12:23:59.607018 16366 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:59.608867 21257 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:59.609212 21239 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:59.530454 21082 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:59.512928 21184 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:59.565136 16470 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:23:59.525282 21138 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:59.533419 16514 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:59.380740 21242 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.565757 17266 rpcz_store.cc:276] 0709 12:23:57.714482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.714606 (+   124us) service_pool.cc:224] Handling call
0709 12:23:57.714926 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:59.241601 (+1526675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:59.325256 (+ 83655us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400be2dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:23:59.325523 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2731072896}
I20260709 12:23:59.504756 20978 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:23:59.581481 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 87770.4us
I20260709 12:23:59.609529 21257 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:59.526463 16221 catalog_manager.cc:5697] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 7 to 12, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 12 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:59.737805 16221 catalog_manager.cc:5697] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:23:59.629822 16543 raft_consensus.cc:1275] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:23:59.507347 21246 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:59.647805 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:59.559582 21093 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:59.615654 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:59.624974 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:59.617051 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:59.560498 20670 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:59.526019 16477 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:23:59.560148 21148 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:59.509953 21241 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:23:59.612514 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:23:59.621407 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 91345.1us
I20260709 12:23:59.507928 21177 consensus_queue.cc:1059] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.107s
I20260709 12:23:59.624698 16394 raft_consensus.cc:1240] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:59.650856 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:59.626777 16397 raft_consensus.cc:1275] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:23:59.635954 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:59.507091 21243 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:59.641880 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:59.650507 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:59.651489 16366 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:59.635288 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:23:59.744980 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 93660.9us
I20260709 12:23:59.634879 16399 raft_consensus.cc:1275] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 12:23:59.733668 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 206472us
I20260709 12:23:59.740286 16221 catalog_manager.cc:5697] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 4 to 11, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 11 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
W20260709 12:23:59.746286 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:59.746145 16367 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:23:59.747025 16221 catalog_manager.cc:5697] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 5 to 9, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 9 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:23:59.636072 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:59.749512 16395 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:23:59.749271 16221 catalog_manager.cc:5697] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:23:59.635697 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:59.747958 16367 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:23:59.641304 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:59.751281 16549 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:23:59.752425 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:59.752878 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:59.753429 16549 raft_consensus.cc:2393] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 16: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:23:59.754711 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:59.667455 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:59.624416 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:59.757694 16473 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14.
I20260709 12:23:59.733664 21263 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:59.738631 21257 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:23:59.606446 21236 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:59.765558 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:59.766490 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:59.767411 16471 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:23:59.740540 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:23:59.768772 16543 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
I20260709 12:23:59.741585 16476 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:23:59.741958 16470 raft_consensus.cc:2376] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13. Re-sending same reply.
I20260709 12:23:59.616315 21184 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:59.619709 21138 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:59.775154 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:59.787117 16470 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14.
I20260709 12:23:59.516674 20734 consensus_queue.cc:260] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:59.777297 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:23:59.792253 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 51362.5us
I20260709 12:23:59.792575 16476 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:59.741159 16546 raft_consensus.cc:2468] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 9.
I20260709 12:23:59.742544 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:59.744315 16550 raft_consensus.cc:2410] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:23:59.743583 16398 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:23:59.743937 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:59.782500 21262 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:59.741029 21246 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:59.783172 21266 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:59.784179 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:59.826864 16471 raft_consensus.cc:2368] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 12. Current term is 13.
I20260709 12:23:59.758948 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:23:59.574244 21130 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:23:59.832967 21130 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:23:59.835108 21130 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:23:59.509771 21146 consensus_queue.cc:237] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:23:59.837162 16473 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:59.835775 21130 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:23:59.843393 21130 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:23:59.785701 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:23:59.795255 16441 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:59.793745 16366 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:59.849478 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 55809.3us
I20260709 12:23:59.850340 16366 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:59.771138 21253 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:59.741252 21093 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:59.855214 16477 raft_consensus.cc:1275] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:59.786887 21184 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:59.776257 20670 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:23:59.787686 21257 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:23:59.860087 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 81730.4us
I20260709 12:23:59.742762 21148 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:59.861799 16473 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:23:59.729133 20978 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:59.863809 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 91724.9us
W20260709 12:23:59.858770 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:59.871825 16476 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 19.
I20260709 12:23:59.847762 21182 raft_consensus.cc:2804] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:59.824084 21263 raft_consensus.cc:697] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:59.872577 16514 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:59.879093 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:59.866708 16516 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:59.873853 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 78674.2us
I20260709 12:23:59.874183 16469 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:23:59.881964 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:59.882608 16476 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
W20260709 12:23:59.874795 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 126930us
I20260709 12:23:59.862021 21093 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:59.876722 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:23:59.883850 16399 raft_consensus.cc:2368] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 12. Current term is 13.
I20260709 12:23:59.877290 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:59.884480 16394 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
I20260709 12:23:59.879988 16514 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:59.882409 16441 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:59.850015 21239 raft_consensus.cc:697] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:23:59.877527 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:23:59.864099 21246 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:59.871006 21148 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:59.728937 21242 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:59.889758 16545 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:23:59.891567 16441 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:59.890120 16516 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:23:59.903719 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:23:59.898375 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:23:59.905359 17266 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:59.908130 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:59.901544 16514 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:59.856767 21243 consensus_queue.cc:1048] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:23:59.891027 16403 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:23:59.904642 16476 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
I20260709 12:23:59.887296 21192 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:59.866317 21235 consensus_queue.cc:1059] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:23:59.883560 21241 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:59.918006 16514 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:23:59.926156 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:59.957715 17255 raft_consensus.cc:2410] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:23:59.915982 16442 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:23:59.953567 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:59.954546 17270 raft_consensus.cc:1275] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:23:59.892407 21095 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:59.891225 21256 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:59.889561 21261 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:59.965587 17266 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:23:59.973675 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:59.975416 16472 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:23:59.894826 21246 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:23:59.870479 20670 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:23:59.974812 21256 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:59.967355 21095 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:59.968699 21241 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:59.958460 21269 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:59.906610 21251 raft_consensus.cc:2804] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:59.902508 21236 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:59.864735 21253 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:59.980965 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:59.986845 17271 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:23:59.980286 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:59.875916 21257 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:23:59.986361 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 136112us
I20260709 12:23:59.906728 21260 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:00.002640 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:59.955116 21245 raft_consensus.cc:2804] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:59.956305 21192 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:59.980976 21175 raft_consensus.cc:2804] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:59.978310 21268 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.981078 21177 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:00.037717 17270 raft_consensus.cc:1275] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:24:00.027599 16366 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:23:59.979084 16367 connection.cc:256] RPC call timeout handler was delayed by -1.6408175090000441s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15600000000000003s for a response
I20260709 12:23:59.992937 21261 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:00.030742 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:23:59.980568 21235 consensus_queue.cc:1243] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Peer 52ab50b0237346c1a13ede947f6a63c2 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:23:59.998593 21246 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:24:00.039351 16367 connection.cc:256] RPC call timeout handler was delayed by -1.275417508999908s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1966s for a response
I20260709 12:24:00.040288 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:00.007848 21095 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:00.044548 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:00.044560 17266 raft_consensus.cc:1240] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 11. Current term is 13. Ops: [11.7-11.7]
I20260709 12:24:00.008072 21241 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:00.005158 21250 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:00.004477 21256 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:00.076390 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:00.012730 21251 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:00.092659 17270 raft_consensus.cc:1240] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:00.012924 21236 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:00.066934 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:00.093328 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:00.067138 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:00.067793 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:00.039856 16366 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:00.094643 17258 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:24:00.095005 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 55251.7us
I20260709 12:24:00.038034 21267 consensus_queue.cc:1048] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.066s
W20260709 12:24:00.068375 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:00.072770 16542 raft_consensus.cc:1275] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:24:00.075057 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:00.068671 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 152793us
I20260709 12:24:00.109372 16542 raft_consensus.cc:1275] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:24:00.075703 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:00.076676 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:00.108093 17254 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
I20260709 12:24:00.110772 16548 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:24:00.077368 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:00.077800 21262 raft_consensus.cc:697] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
W20260709 12:24:00.070464 16367 connection.cc:256] RPC call timeout handler was delayed by -1.1539175090001663s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2101s for a response
I20260709 12:24:00.071312 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:00.071751 16543 raft_consensus.cc:1275] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:24:00.113766 16367 connection.cc:256] RPC call timeout handler was delayed by -1.0936175090000688s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21680000000000002s for a response
I20260709 12:24:00.008217 21269 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:00.033636 21192 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:00.091032 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:00.118669 17269 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:24:00.118803 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:00.119037 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:00.119586 17271 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:24:00.042743 21141 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:00.120954 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:00.026795 21245 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:00.047968 21261 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:00.096788 21272 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:00.097793 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:00.100648 21276 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:00.102397 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 184456us
I20260709 12:24:00.104387 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:00.152544 17266 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
W20260709 12:24:00.153527 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:00.153784 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:00.153993 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:00.154871 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:00.047210 21266 raft_consensus.cc:2804] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:00.155864 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:00.156167 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:00.156872 16469 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:24:00.108847 16549 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:24:00.096159 16366 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:00.110275 16550 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
W20260709 12:24:00.157977 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 61897.5us
I20260709 12:24:00.096570 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:00.033085 21175 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:00.159798 21175 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:59.974458 21239 consensus_queue.cc:237] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:00.119627 17254 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:24:00.149685 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:00.151660 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:00.156359 21266 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:00.115674 16367 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:00.161175 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:00.159898 16515 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:00.154137 16514 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:00.160444 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:00.156826 17266 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:24:00.068089 21271 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:00.160312 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:00.173123 17255 raft_consensus.cc:2393] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 12: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:00.155913 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:00.106186 21251 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:00.101857 21236 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:00.112828 21250 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:24:00.180560 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 64993.3us
I20260709 12:24:00.165009 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:00.165187 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:00.169333 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:00.181493 16367 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:00.165246 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:00.161448 17261 raft_consensus.cc:2376] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14. Re-sending same reply.
I20260709 12:24:00.173678 17263 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 15.
I20260709 12:24:00.174906 16514 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:00.172495 21266 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:59.897881 16221 catalog_manager.cc:5697] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 9 to 13, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 13 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:00.176890 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:00.127094 21245 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:00.177203 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:00.173708 21258 log.cc:927] Time spent T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.303s	user 0.001s	sys 0.000s
I20260709 12:24:00.183701 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:00.166535 17296 debug-util.cc:398] Leaking SignalData structure 0x7b080013cca0 after lost signal to thread 5355
I20260709 12:24:00.160530 21283 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:00.150619 21272 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:00.188051 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 160625us
I20260709 12:24:00.177924 21271 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:00.036048 21260 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:00.186338 21250 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:00.201363 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:00.200254 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:00.176708 21278 consensus_queue.cc:1048] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 12:24:00.213913 21266 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:00.215704 21232 consensus_meta.cc:287] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.739s	user 0.010s	sys 0.000s
I20260709 12:24:00.160640 21259 consensus_queue.cc:1048] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 12:24:00.219281 21093 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:00.042063 21268 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:00.200203 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:00.009734 21263 consensus_queue.cc:237] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:00.228088 17296 debug-util.cc:398] Leaking SignalData structure 0x7b0800291e20 after lost signal to thread 16260
I20260709 12:24:00.119441 21269 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:24:00.239951 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:00.245638 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 64280.5us
I20260709 12:24:00.126528 21243 consensus_queue.cc:1048] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
W20260709 12:24:00.245021 21274 log.cc:927] Time spent T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:24:00.246237 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:00.245661 21275 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:00.253747 21242 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:00.254243 20978 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:00.151984 21141 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:00.170665 21155 consensus_queue.cc:1059] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 12:24:00.262679 20734 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:00.263545 21287 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:00.247560 21262 consensus_queue.cc:237] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:00.273576 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:00.166175 21276 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:00.275753 21182 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:00.277441 21282 consensus_queue.cc:1048] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:24:00.277818 21238 consensus_meta.cc:287] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.663s	user 0.010s	sys 0.000s
I20260709 12:24:00.278466 21102 consensus_queue.cc:1048] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:00.224148 21265 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:00.348765 21265 raft_consensus.cc:697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 19 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:00.349702 21265 consensus_queue.cc:237] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:00.294534 17296 debug-util.cc:398] Leaking SignalData structure 0x7b080029f480 after lost signal to thread 16427
W20260709 12:24:00.353556 17296 debug-util.cc:398] Leaking SignalData structure 0x7b08004053e0 after lost signal to thread 16501
W20260709 12:24:00.354061 17296 debug-util.cc:398] Leaking SignalData structure 0x7b080037b840 after lost signal to thread 16502
I20260709 12:24:00.353703 21266 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:00.355062 21271 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:00.355432 17296 debug-util.cc:398] Leaking SignalData structure 0x7b08003553e0 after lost signal to thread 16575
I20260709 12:24:00.357996 21240 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:00.229043 21281 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:00.300716 21283 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:00.286958 21191 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:00.296577 21250 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:00.292682 21272 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:00.340548 21261 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:00.341893 21241 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:00.344995 21253 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:00.345599 21257 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:00.319686 21287 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:00.337888 21260 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:00.370070 21281 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:00.370654 21286 consensus_queue.cc:1048] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.094s
I20260709 12:24:00.425823 21287 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:00.432631 21283 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:00.442086 21272 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:00.213340 16221 catalog_manager.cc:5697] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 6 to 9, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 9 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:24:00.478884 16221 catalog_manager.cc:5697] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 9 to 14, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 14 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:00.482692 16221 catalog_manager.cc:5697] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 7 to 12, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 12 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:00.485229 16221 catalog_manager.cc:5697] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 6 to 9, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 9 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:00.488300 16221 catalog_manager.cc:5697] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 6 to 14. New cstate: current_term: 14 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:00.490861 16221 catalog_manager.cc:5697] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 7 to 9, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 9 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:00.293143 16367 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:00.496022 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 202769us
I20260709 12:24:00.517719 16221 catalog_manager.cc:5697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 12 to 16, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 16 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:00.523180 16221 catalog_manager.cc:5697] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:00.527676 16221 catalog_manager.cc:5697] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 9 to 11, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 11 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
W20260709 12:24:00.536653 16470 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.006s	sys 0.002s
W20260709 12:24:00.538257 16470 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.007s	sys 0.003s
I20260709 12:24:00.538839 16221 catalog_manager.cc:5697] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 9 to 11, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 11 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
W20260709 12:24:00.539716 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:00.544071 16472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34826 (request call id 1601) took 1077 ms. Trace:
W20260709 12:24:00.302997 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 385135us
I20260709 12:24:00.553309 16472 rpcz_store.cc:276] 0709 12:23:59.466925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.470006 (+  3081us) service_pool.cc:224] Handling call
0709 12:23:59.489191 (+ 19185us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b5800602a98. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:24:00.543554 (+1054363us) spinlock_profiling.cc:243] Waited 485 ms on lock 0x7b54004c4ad4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:00.543752 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1141903616}
I20260709 12:24:00.555109 16551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 963) took 1077 ms. Trace:
W20260709 12:24:00.313129 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 138284us
I20260709 12:24:00.557153 16221 catalog_manager.cc:5697] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 8 to 9, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 9 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:00.552796 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 392448us
I20260709 12:24:00.562781 16221 catalog_manager.cc:5697] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 11 to 12, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 12 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:00.555588 16551 rpcz_store.cc:276] 0709 12:23:59.477258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.477404 (+   146us) service_pool.cc:224] Handling call
0709 12:23:59.477723 (+   319us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:59.607019 (+129296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:59.607051 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:59.607628 (+   577us) log.cc:844] Serialized 52 byte log entry
0709 12:24:00.077968 (+470340us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:00.078014 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:00.554099 (+476085us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b54005eea54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:00.554174 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:00.554277 (+   103us) raft_consensus.cc:1685] finished
0709 12:24:00.554294 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:00.554787 (+   493us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":99,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":7217,"spinlock_wait_cycles":296433024,"thread_start_us":304923,"threads_started":1,"wal-append.queue_time_us":441746}
I20260709 12:24:00.572563 16221 catalog_manager.cc:5697] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:00.575601 16516 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:00.577278 16221 catalog_manager.cc:5697] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 6 to 9, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 9 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:00.279495 17143 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:00.310945 17144 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:00.300691 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:00.313985 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:00.300782 17261 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:24:00.312233 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:00.330258 17269 raft_consensus.cc:2410] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:24:00.314697 17271 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:00.324962 17270 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:24:00.342659 20670 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:00.313553 21027 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:24:00.342721 21082 consensus_meta.cc:287] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.732s	user 0.010s	sys 0.000s
I20260709 12:24:00.346395 21148 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:00.337886 21141 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:00.347455 21192 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:00.652942 21302 log.cc:927] Time spent T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 12:24:00.653228 21276 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:00.601435 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 321869us
I20260709 12:24:00.338789 21095 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:00.371656 21191 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:00.657512 21191 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:00.657002 21270 consensus_queue.cc:1059] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:24:00.427675 21248 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.035s	sys 0.029s
I20260709 12:24:00.347007 21169 raft_consensus.cc:3055] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:24:00.658946 21248 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.036s	sys 0.029s
I20260709 12:24:00.659351 21169 raft_consensus.cc:740] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:00.332267 21182 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:00.587564 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:00.580253 16516 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:00.602303 17255 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 12
W20260709 12:24:00.612476 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 293342us
I20260709 12:24:00.622032 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:24:00.618755 17296 debug-util.cc:398] Leaking SignalData structure 0x7b0800086560 after lost signal to thread 17299
I20260709 12:24:00.635131 21027 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:00.590385 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:00.620155 20763 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:00.623189 20670 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:00.623714 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:00.583321 16221 catalog_manager.cc:5697] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 1 to 11, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 11 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:24:00.585363 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:00.586889 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:00.616719 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:00.630337 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 126065us
I20260709 12:24:00.601122 17254 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:00.626983 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:00.647464 21121 consensus_queue.cc:1048] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 12:24:00.600100 17142 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:00.658838 21270 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:00.300427 21232 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:00.664129 21257 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:00.605788 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:00.665213 21272 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:00.671414 21279 raft_consensus.cc:3055] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:00.680637 21240 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:00.685885 21239 consensus_queue.cc:1059] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:00.687440 21145 consensus_queue.cc:1048] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:00.606344 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:00.757004 17296 debug-util.cc:398] Leaking SignalData structure 0x7b0800307de0 after lost signal to thread 17300
I20260709 12:24:00.701810 16542 raft_consensus.cc:1240] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:00.660588 16515 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:00.597232 16514 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:00.656095 21192 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:00.688246 21175 consensus_meta.cc:287] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.518s	user 0.008s	sys 0.000s
I20260709 12:24:00.659000 21095 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:00.778133 21053 consensus_queue.cc:1048] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:24:00.782419 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:00.783872 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:00.760588 21279 raft_consensus.cc:740] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:00.785213 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:00.786227 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:00.792796 17270 raft_consensus.cc:2393] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 13: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:00.755661 20670 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:00.795248 20670 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:00.798888 16367 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:00.801028 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 221188us
I20260709 12:24:00.801960 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:00.765633 21027 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:00.804931 21270 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:00.805562 21270 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:00.806021 21270 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:00.806250 20763 consensus_queue.cc:1048] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:00.795325 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:00.813094 21261 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:00.776265 21256 consensus_meta.cc:287] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.563s	user 0.011s	sys 0.001s
W20260709 12:24:00.815030 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 178277us
I20260709 12:24:00.816473 21148 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:00.816864 21192 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:00.817564 21192 raft_consensus.cc:697] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 13 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:00.818360 21169 consensus_queue.cc:260] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:00.818925 21192 consensus_queue.cc:237] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:00.786610 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:00.816056 16515 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:00.821596 21146 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:24:00.822345 17269 raft_consensus.cc:2393] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 13: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:00.827767 17269 raft_consensus.cc:1275] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:24:00.694582 21093 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:00.429095 21281 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:00.695124 21241 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:00.761188 21257 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:00.688387 17143 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:00.813997 21256 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:00.713292 21268 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:00.820237 21169 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:00.691465 21141 consensus_queue.cc:1048] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.142s
W20260709 12:24:00.842897 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 44228.3us
I20260709 12:24:00.370002 21238 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:00.832721 21257 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:00.785961 21279 consensus_queue.cc:260] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:00.690816 21269 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:00.851994 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 255342us
I20260709 12:24:00.833433 21175 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:00.864825 21298 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:00.873999 16549 raft_consensus.cc:1240] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:00.874675 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:00.875931 16398 raft_consensus.cc:2393] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 14: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:00.877097 21279 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:00.882020 21256 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:00.884124 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:00.885334 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 197362us
I20260709 12:24:00.885704 21271 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:00.886210 21271 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:00.886478 17143 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:00.886674 21271 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:00.887738 21130 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:00.888615 21130 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:00.888924 21298 raft_consensus.cc:697] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:00.889269 21130 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:00.890630 21298 consensus_queue.cc:237] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:00.897838 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:00.899442 16403 raft_consensus.cc:2393] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 14: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:00.922236 16397 raft_consensus.cc:1275] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:24:00.912752 16543 raft_consensus.cc:1240] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 13. Current term is 14. Ops: []
W20260709 12:24:00.916929 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 78915us
W20260709 12:24:00.919008 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 48376.3us
I20260709 12:24:00.901208 17271 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:24:00.908892 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:00.968106 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 104110us
I20260709 12:24:00.920478 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:00.868366 21146 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:00.981395 16394 raft_consensus.cc:2393] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 16: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:00.915122 21261 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:24:00.915158 21184 consensus_meta.cc:287] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 1.045s	user 0.007s	sys 0.012s
W20260709 12:24:00.921567 20734 consensus_meta.cc:287] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.598s	user 0.007s	sys 0.015s
I20260709 12:24:00.927896 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:00.921700 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:00.909622 21232 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:01.000625 16401 raft_consensus.cc:2376] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13. Re-sending same reply.
W20260709 12:24:01.002285 21246 consensus_meta.cc:287] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.890s	user 0.000s	sys 0.018s
I20260709 12:24:01.029913 21246 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:00.918387 21238 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:00.986899 16545 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
W20260709 12:24:00.995306 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.004957 21102 raft_consensus.cc:3055] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:24:01.010856 21245 consensus_meta.cc:287] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.765s	user 0.016s	sys 0.000s
W20260709 12:24:01.011416 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 412115us
I20260709 12:24:00.901124 16221 catalog_manager.cc:5697] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 12 to 14, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 14 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
W20260709 12:24:01.013468 17258 consensus_meta.cc:287] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.918s	user 0.004s	sys 0.020s
I20260709 12:24:01.016433 17261 raft_consensus.cc:1275] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:24:01.017585 21184 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:01.018385 17255 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:24:01.019995 17271 raft_consensus.cc:1240] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:24:01.022312 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:01.024566 16441 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:00.989779 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.025817 16550 raft_consensus.cc:1240] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:00.987581 21261 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:01.030542 21270 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:01.031997 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:01.032637 21292 raft_consensus.cc:3055] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:01.033062 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:01.033356 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:01.034178 16400 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 15: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:00.912228 21269 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:00.905388 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:24:01.035315 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 50295.2us
I20260709 12:24:01.037078 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:01.046344 16544 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:01.047181 17142 leader_election.cc:400] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 12. Current term is 13.
I20260709 12:24:01.057049 17258 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:24:01.057659 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:01.079488 17142 leader_election.cc:403] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:24:01.059134 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:01.059725 21236 consensus_meta.cc:287] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.827s	user 0.011s	sys 0.008s
I20260709 12:24:01.066001 16394 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
I20260709 12:24:01.066551 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:01.067433 16396 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:24:01.053618 21261 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:01.063038 16401 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 16.
I20260709 12:24:01.073601 17144 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:01.076431 21251 consensus_meta.cc:287] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.847s	user 0.015s	sys 0.000s
I20260709 12:24:01.042343 21146 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:01.042126 21102 raft_consensus.cc:740] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:01.084676 16397 raft_consensus.cc:2368] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 15. Current term is 16.
I20260709 12:24:01.090238 16542 raft_consensus.cc:1240] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:01.087086 17254 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:24:01.032249 20670 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:01.062143 21292 raft_consensus.cc:740] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:01.088294 17270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 1226) took 1015 ms. Trace:
I20260709 12:24:01.099890 20763 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:01.100162 16395 raft_consensus.cc:1240] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:01.100654 16399 raft_consensus.cc:2393] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 13: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:01.102815 17258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45228 (request call id 1223) took 1035 ms. Trace:
I20260709 12:24:01.102341 16221 catalog_manager.cc:5697] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 8 to 9, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 9 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:01.103278 17258 rpcz_store.cc:276] 0709 12:24:00.067468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.067573 (+   105us) service_pool.cc:224] Handling call
0709 12:24:01.095536 (+1027963us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:01.095410 16403 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:24:01.104655 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:01.105106 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 111571us
I20260709 12:24:01.105636 16394 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
I20260709 12:24:01.105307 21102 consensus_queue.cc:260] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:01.107256 21102 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:01.107414 16441 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
W20260709 12:24:01.107959 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 61123.2us
I20260709 12:24:01.056058 21245 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:01.108701 21282 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 12. Current term is 13.
I20260709 12:24:01.110680 20670 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:01.111354 17263 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:24:01.112099 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:01.112855 21138 consensus_meta.cc:287] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 1.267s	user 0.009s	sys 0.011s
I20260709 12:24:01.113912 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:01.177609 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:01.177802 16442 leader_election.cc:336] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.564s (SENT)
I20260709 12:24:01.178387 16551 raft_consensus.cc:2393] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 12: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:01.178297 16442 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:01.115345 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:01.179342 21184 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:01.116691 21095 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.180581 21248 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:01.180708 21095 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:01.181789 17263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45228 (request call id 1243) took 1105 ms. Trace:
I20260709 12:24:01.182969 21095 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:01.117563 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:01.183517 17266 raft_consensus.cc:1240] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:01.184435 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:01.185158 16550 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:24:01.185811 17266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 1230) took 1113 ms. Trace:
I20260709 12:24:01.118407 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:01.118809 16398 raft_consensus.cc:1240] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:01.186267 17266 rpcz_store.cc:276] 0709 12:24:00.072233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.833273 (+761040us) service_pool.cc:224] Handling call
0709 12:24:01.183257 (+349984us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:01.184079 (+   822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.185537 (+  1458us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b540063ea50. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:01.185737 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":750208768}
I20260709 12:24:01.187645 21138 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:01.120082 17261 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 1243) took 1037 ms. Trace:
I20260709 12:24:01.120723 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:01.189219 21270 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:01.189721 16545 raft_consensus.cc:2393] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 16: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:01.184578 21268 consensus_queue.cc:1059] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 12:24:01.190667 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:01.122290 21292 consensus_queue.cc:260] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:01.191015 16395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 1244) took 1315 ms. Trace:
I20260709 12:24:01.121381 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:01.191901 21292 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:01.192443 16543 raft_consensus.cc:2368] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 14. Current term is 15.
I20260709 12:24:01.090504 21146 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:01.191596 16395 rpcz_store.cc:276] 0709 12:23:59.876017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.876164 (+   147us) service_pool.cc:224] Handling call
0709 12:23:59.891647 (+ 15483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:01.100726 (+1209079us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.176233 (+ 75507us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b540015f254. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:01.190916 (+ 14683us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2746684160}
I20260709 12:24:01.193360 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:01.194097 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:01.124493 21250 consensus_meta.cc:287] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.666s	user 0.016s	sys 0.001s
I20260709 12:24:01.133279 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:01.194649 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:01.194911 21250 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:01.195470 16470 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:01.195932 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:01.135771 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:24:01.136894 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 63452.1us
I20260709 12:24:01.139379 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:01.139946 21287 consensus_meta.cc:287] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.707s	user 0.016s	sys 0.005s
I20260709 12:24:01.207255 16395 raft_consensus.cc:2393] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 13: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:01.207484 21287 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:01.128863 21191 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:01.208391 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:01.209244 16395 raft_consensus.cc:2368] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 12. Current term is 13.
I20260709 12:24:01.210680 21191 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:01.212105 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:01.212517 20734 consensus_queue.cc:1048] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:24:01.213379 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:01.214087 16475 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:24:01.214874 21287 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:01.216686 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:01.218462 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:01.219206 16469 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:01.216774 16396 raft_consensus.cc:1240] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:24:01.225749 16514 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:01.164662 16544 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
W20260709 12:24:01.167296 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 60007.1us
I20260709 12:24:01.168056 21169 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:01.229254 17144 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:01.171748 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:01.170349 21282 consensus_queue.cc:1048] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:01.230237 16403 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:01.231280 21095 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:01.172199 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:01.231885 21095 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:01.232302 21095 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:01.232509 16399 raft_consensus.cc:2393] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 10: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:01.174384 21245 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:01.176105 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:01.188153 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:01.119583 21182 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:01.180267 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:01.190752 21268 raft_consensus.cc:3055] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:01.172493 16473 raft_consensus.cc:1240] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:01.234699 21268 raft_consensus.cc:740] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:01.235798 21268 consensus_queue.cc:260] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:01.132925 21281 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:01.237497 16473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 916) took 1075 ms. Trace:
I20260709 12:24:01.191975 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:01.238184 16473 rpcz_store.cc:276] 0709 12:24:00.161700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.161878 (+   178us) service_pool.cc:224] Handling call
0709 12:24:00.177984 (+ 16106us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:01.234877 (+1056893us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.236947 (+  2070us) spinlock_profiling.cc:243] Waited 994 ms on lock 0x7b5400594a54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:01.237281 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2285538560}
I20260709 12:24:01.093822 21236 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:01.239343 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:01.239899 21281 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:01.241487 21236 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:01.091717 21251 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:01.242251 21175 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:01.243647 21251 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:01.190688 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:01.244663 16394 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:01.245713 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:01.205602 21246 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:01.246599 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:01.182548 17263 rpcz_store.cc:276] 0709 12:24:00.076044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.604536 (+528492us) service_pool.cc:224] Handling call
0709 12:24:01.181430 (+576894us) spinlock_profiling.cc:243] Waited 358 ms on lock 0x7b540063ea54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:01.181702 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":823061760}
I20260709 12:24:01.179379 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:01.248077 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:01.206977 16542 raft_consensus.cc:2393] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 13: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:01.249631 16472 raft_consensus.cc:2393] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 11: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:01.250190 16542 raft_consensus.cc:1240] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 14. Current term is 15. Ops: [12.7-14.8]
I20260709 12:24:01.251516 16548 raft_consensus.cc:2393] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 16: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:01.251624 16542 raft_consensus.cc:1240] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:01.229377 21169 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:01.236521 21182 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:01.240548 20670 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:01.188977 17261 rpcz_store.cc:276] 0709 12:24:00.082151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.078381 (+996230us) service_pool.cc:224] Handling call
0709 12:24:01.119343 (+ 40962us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:01.119667 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:01.119695 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:01.119743 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:01.119805 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.119871 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:01.119982 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:01.197456 21250 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:01.241417 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:01.200098 16402 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:24:01.206107 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:01.248255 16551 raft_consensus.cc:2393] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 16: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:01.186444 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:01.111686 21130 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:01.287760 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:01.096937 17270 rpcz_store.cc:276] 0709 12:24:00.068257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.814076 (+745819us) service_pool.cc:224] Handling call
0709 12:24:00.814581 (+   505us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:00.815038 (+   457us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:24:00.988205 (+173167us) log.cc:844] Serialized 14 byte log entry
0709 12:24:01.083684 (+ 95479us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:01.083732 (+    48us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:01.083784 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.083846 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:01.083970 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":615,"prepare.queue_time_us":221690,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":2117,"raft.queue_time_us":5817,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":4836,"thread_start_us":204020,"threads_started":2}
I20260709 12:24:01.266645 17269 raft_consensus.cc:1240] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 10. Current term is 11. Ops: []
W20260709 12:24:01.275077 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:01.292135 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.238184 21268 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:01.281159 17254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45230 (request call id 1239) took 1195 ms. Trace:
I20260709 12:24:01.282171 16397 raft_consensus.cc:2393] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 14: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:01.287751 16470 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
I20260709 12:24:01.297884 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:01.293304 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:01.265760 21169 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:01.300153 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:01.295135 17254 rpcz_store.cc:276] 0709 12:24:00.080859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.160943 (+ 80084us) service_pool.cc:224] Handling call
0709 12:24:01.227522 (+1066579us) spinlock_profiling.cc:243] Waited 434 ms on lock 0x7b540018a954. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:01.276122 (+ 48600us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":997141760}
I20260709 12:24:01.306540 16545 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:01.320441 16514 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:01.324038 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:01.314949 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:01.318222 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:01.252724 17263 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773"
dest_uuid: "ef57beab0e1d40d585983873c55107e8"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:45120
I20260709 12:24:01.287915 21298 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.330899 16551 raft_consensus.cc:2393] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 16: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:01.339567 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:01.340201 16545 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
W20260709 12:24:01.348322 16515 leader_election.cc:336] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Timed out: RequestConsensusVote RPC to 127.5.58.1:41805 timed out after 1.761s (SENT)
W20260709 12:24:01.342145 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 113008us
I20260709 12:24:01.293756 21276 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:01.344913 16542 raft_consensus.cc:2393] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 14: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:01.345320 16550 raft_consensus.cc:2393] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 13: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:01.352651 16475 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:01.228356 21271 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:01.348517 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:01.228641 21027 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:01.340821 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:01.356114 16470 raft_consensus.cc:2393] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 10: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:01.353524 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:01.329259 17263 raft_consensus.cc:606] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:01.358741 17263 raft_consensus.cc:614] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:01.355146 16515 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:01.369411 16472 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:01.418617 21245 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.419709 16546 raft_consensus.cc:1240] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 12. Current term is 13. Ops: []
W20260709 12:24:01.422065 16402 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46250 (request call id 1355) took 1772 ms (client timeout 2249 ms). Trace:
I20260709 12:24:01.419721 21251 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:01.383955 21316 log.cc:927] Time spent T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.078s	user 0.002s	sys 0.000s
I20260709 12:24:01.420547 16442 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:01.428759 16477 raft_consensus.cc:1275] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:24:01.433269 16473 raft_consensus.cc:1275] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:24:01.377625 21138 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:01.419291 21245 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:01.431795 16402 rpcz_store.cc:269] 0709 12:23:59.649971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.650178 (+   207us) service_pool.cc:224] Handling call
0709 12:24:01.421682 (+1771504us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54009bebd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:01.421868 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3347276160}
I20260709 12:24:01.446065 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:01.436980 16551 raft_consensus.cc:1275] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:24:01.359834 21298 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:01.452273 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:24:01.456467 21257 consensus_meta.cc:287] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.583s	user 0.009s	sys 0.003s
I20260709 12:24:01.456924 17144 leader_election.cc:400] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 11. Current term is 12.
I20260709 12:24:01.457538 17144 leader_election.cc:403] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:01.457015 21251 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:01.459470 21235 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:01.460163 21235 raft_consensus.cc:697] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 13 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:01.461184 21235 consensus_queue.cc:237] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:01.461818 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:24:01.463347 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 131661us
I20260709 12:24:01.464006 17264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 1236) took 1389 ms. Trace:
W20260709 12:24:01.465847 21311 log.cc:927] Time spent T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:24:01.466813 17269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 1233) took 1393 ms. Trace:
I20260709 12:24:01.467181 17269 rpcz_store.cc:276] 0709 12:24:00.073288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.834602 (+761314us) service_pool.cc:224] Handling call
0709 12:24:01.263291 (+428689us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:01.299948 (+ 36657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.466645 (+166697us) spinlock_profiling.cc:243] Waited 420 ms on lock 0x7b540018a950. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:01.466759 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":966786432}
I20260709 12:24:01.470091 21298 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:01.471704 21271 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:01.444839 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:01.464598 16550 raft_consensus.cc:1240] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:24:01.464578 17264 rpcz_store.cc:276] 0709 12:24:00.074249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.877543 (+803294us) service_pool.cc:224] Handling call
0709 12:24:01.079753 (+202210us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:01.108403 (+ 28650us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:24:01.108786 (+   383us) log.cc:844] Serialized 14 byte log entry
0709 12:24:01.281953 (+173167us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:01.282015 (+    62us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:01.293105 (+ 11090us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.293198 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:01.463792 (+170594us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b5400281750. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:01.463935 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8028,"prepare.queue_time_us":787,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":17533,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":10776,"spinlock_wait_cycles":458729344,"thread_start_us":172671,"threads_started":1,"wal-append.queue_time_us":263936}
I20260709 12:24:01.477975 16469 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
I20260709 12:24:01.478833 16475 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14.
W20260709 12:24:01.482366 16544 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37242 (request call id 1417) took 2407 ms (client timeout 1678 ms). Trace:
I20260709 12:24:01.483234 16394 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:24:01.486586 16472 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
W20260709 12:24:01.482846 16544 rpcz_store.cc:269] 0709 12:23:59.074500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.078167 (+  3667us) service_pool.cc:224] Handling call
0709 12:24:01.472767 (+2394600us) spinlock_profiling.cc:243] Waited 1.78 s on lock 0x7b54006b08d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:01.482264 (+  9497us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4105428736}
I20260709 12:24:01.488438 16545 raft_consensus.cc:1240] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:01.486694 16403 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
I20260709 12:24:01.489315 16514 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:01.492097 20763 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:24:01.493511 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 73097.3us
I20260709 12:24:01.494451 16442 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:01.440117 21182 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:01.495457 20763 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:01.496969 21191 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:01.497613 21191 raft_consensus.cc:697] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 13 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:01.497820 16547 raft_consensus.cc:1275] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:24:01.498739 21282 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 11. Current term is 12.
I20260709 12:24:01.499809 16551 raft_consensus.cc:1240] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 12:24:01.501217 16549 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:24:01.486776 16367 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:01.505486 21298 consensus_queue.cc:1048] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:24:01.506258 21315 log.cc:927] Time spent T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:24:01.505580 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:24:01.507719 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.513127 16470 raft_consensus.cc:1275] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:24:01.516224 16442 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:01.528616 21027 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:01.517822 21251 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:01.530746 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 85733.2us
I20260709 12:24:01.531021 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:01.535267 21130 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:01.537115 16547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1425) took 1902 ms. Trace:
I20260709 12:24:01.537571 16547 rpcz_store.cc:276] 0709 12:23:59.634888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.635013 (+   125us) service_pool.cc:224] Handling call
0709 12:24:01.497324 (+1862311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:01.498248 (+   924us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.536913 (+ 38665us) spinlock_profiling.cc:243] Waited 1.84 s on lock 0x7b54006b08d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:01.537058 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4226056448}
I20260709 12:24:01.538583 21138 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:01.538307 21261 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:01.540148 16395 raft_consensus.cc:1240] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:24:01.488111 21146 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:01.546443 16543 raft_consensus.cc:2368] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 15. Current term is 16.
I20260709 12:24:01.532137 16367 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:01.498646 21191 consensus_queue.cc:237] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:01.558521 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 61066.7us
I20260709 12:24:01.567893 16367 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:01.570130 21292 raft_consensus.cc:2749] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:01.570998 21292 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:01.570005 16367 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:01.573021 16401 raft_consensus.cc:1240] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:01.456970 21257 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:01.580317 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:01.587473 16403 raft_consensus.cc:2393] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 13: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:01.583323 16514 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:01.557726 21248 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:01.601238 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:01.599578 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:01.610168 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:01.560865 20763 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:01.588681 21320 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.612881 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:01.614138 16395 raft_consensus.cc:1275] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:24:01.624102 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:01.627161 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:01.629110 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:01.629257 21246 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:24:01.629818 17266 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
I20260709 12:24:01.630955 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:01.587402 21169 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:01.631821 17270 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 16.
I20260709 12:24:01.632577 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:01.633328 21261 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:01.636297 16474 raft_consensus.cc:2393] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 13: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:01.573998 21095 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:01.637943 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:01.638787 21095 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:01.638127 21192 consensus_queue.cc:1048] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 12:24:01.639007 17271 raft_consensus.cc:2376] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13. Re-sending same reply.
I20260709 12:24:01.640383 16471 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:01.639247 16546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 978) took 1343 ms. Trace:
I20260709 12:24:01.589061 21130 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.643875 16542 raft_consensus.cc:1240] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:24:01.641285 16546 rpcz_store.cc:276] 0709 12:24:00.296381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.297004 (+   623us) service_pool.cc:224] Handling call
0709 12:24:00.316357 (+ 19353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:01.450881 (+1134524us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.638963 (+188082us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54008c55d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:01.639190 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2455230208}
I20260709 12:24:01.629110 17261 raft_consensus.cc:1240] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:01.633365 17255 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:01.615211 21053 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:01.658986 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:01.661177 16470 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:01.662097 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:01.662858 16474 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:01.656834 21130 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:01.665462 17144 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:01.666132  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:24:01.666474 21276 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:01.666491 21130 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:01.662313 21146 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:01.668007 17144 leader_election.cc:400] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 12. Current term is 13.
I20260709 12:24:01.697326 17143 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:01.682374 16401 raft_consensus.cc:1275] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:24:01.692500 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:01.732641 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 268488us
W20260709 12:24:01.676651 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 106809us
I20260709 12:24:01.668624 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:01.748399 16403 raft_consensus.cc:2393] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 16: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:01.696560 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:01.685321 16516 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:01.684883 21318 log.cc:927] Time spent T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.169s	user 0.002s	sys 0.000s
I20260709 12:24:01.766672 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:01.678158 21319 log.cc:927] Time spent T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.162s	user 0.002s	sys 0.000s
I20260709 12:24:01.762135 17255 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
W20260709 12:24:01.763254 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 66000.9us
I20260709 12:24:01.651588 21141 consensus_queue.cc:1059] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.113s
W20260709 12:24:01.761309 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 76048.8us
I20260709 12:24:01.686232 21053 raft_consensus.cc:697] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:01.676223 21248 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:01.775053 16516 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:01.675282 21169 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:01.673280 21245 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:01.767014 21323 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:01.789467 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:01.788825 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:01.697674 21268 consensus_queue.cc:1059] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.077s
I20260709 12:24:01.777089 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:01.782569 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:01.696506 21276 consensus_queue.cc:1059] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.087s
I20260709 12:24:01.642132 21320 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:01.788784 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 272641us
I20260709 12:24:01.696978 20670 consensus_queue.cc:1059] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 12:24:01.673555 21257 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:01.784420 16470 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 17.
I20260709 12:24:01.798388 16367 leader_election.cc:400] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 14. Current term is 15.
I20260709 12:24:01.678507 21155 consensus_queue.cc:1059] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.103s
I20260709 12:24:01.805742 17261 raft_consensus.cc:1240] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:01.770717 21146 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:01.782024 21248 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:01.800041 16471 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
W20260709 12:24:01.802035 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 134240us
I20260709 12:24:01.808203 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:01.805450 16514 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:01.789683 21323 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:01.792990 21292 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:01.797735 21182 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:01.816834 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:01.815861 16516 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:01.825524 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.829293 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:01.829982 17254 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:01.832072 21182 raft_consensus.cc:697] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:01.833720 21182 consensus_queue.cc:237] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:01.836144 21279 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:01.810874 21248 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:01.803202 21145 consensus_queue.cc:1059] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:24:01.815349 21323 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:01.862815 16442 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:01.820525 21250 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:24:01.860514 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 50362.7us
I20260709 12:24:01.810491 21095 consensus_queue.cc:1059] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 12:24:01.815532 21270 consensus_queue.cc:1048] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
W20260709 12:24:01.874320 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 47932.8us
I20260709 12:24:01.859076 21298 raft_consensus.cc:3055] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:24:01.878760 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:01.882903 21325 log.cc:927] Time spent T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
W20260709 12:24:01.884145 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 107990us
I20260709 12:24:01.895496 16515 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:01.877228 21298 raft_consensus.cc:740] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:01.880199 21053 consensus_queue.cc:237] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:01.883453 21281 raft_consensus.cc:3055] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:24:01.906466 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 52603.6us
I20260709 12:24:01.896602 21250 raft_consensus.cc:697] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 17 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:24:01.925302 21102 consensus_meta.cc:287] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.763s	user 0.011s	sys 0.002s
W20260709 12:24:01.998041 21330 log.cc:927] Time spent T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:24:02.026007 21326 raft_consensus.cc:3055] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:02.026721 21326 raft_consensus.cc:740] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:01.905761 21281 raft_consensus.cc:740] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:02.029584 21281 consensus_queue.cc:260] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:02.031217 21281 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:02.031316 21250 consensus_queue.cc:237] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:02.035151 21279 raft_consensus.cc:3055] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:02.035660 21279 raft_consensus.cc:740] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:24:02.038821 16441 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.013s	sys 0.034s
W20260709 12:24:02.039218 16441 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.013s	sys 0.034s
W20260709 12:24:02.039726 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 540771us
I20260709 12:24:01.995973 21146 raft_consensus.cc:697] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:01.905616 21268 consensus_queue.cc:1059] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.090s
I20260709 12:24:01.900485 21169 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:02.007458 16442 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
W20260709 12:24:02.008464 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 122889us
I20260709 12:24:02.013188  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:02.015394 17254 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:24:02.018261 21320 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:02.019354 21145 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:02.066462 21145 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:02.067122 21145 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:24:02.020304 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 115504us
I20260709 12:24:02.021171 21271 raft_consensus.cc:3055] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:02.070815 21271 raft_consensus.cc:740] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:01.904500 21257 consensus_queue.cc:1048] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 4, Time since last communication: 0.089s
I20260709 12:24:02.072053 21271 consensus_queue.cc:260] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:02.075668 16471 raft_consensus.cc:1275] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:24:02.041563 21298 consensus_queue.cc:260] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:02.044799 21130 raft_consensus.cc:3055] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:02.048565 21282 raft_consensus.cc:3055] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:02.051534 21102 consensus_queue.cc:1059] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:24:01.995059 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 179260us
I20260709 12:24:02.091166 16474 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:24:02.092943 21192 consensus_queue.cc:1048] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:02.092855 21292 consensus_queue.cc:1059] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:24:02.095829 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 197554us
I20260709 12:24:02.091193 16397 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:24:02.063488 21169 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:02.111042 17143 leader_election.cc:400] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 15. Current term is 16.
I20260709 12:24:02.081311 21130 raft_consensus.cc:740] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:02.111771 17143 leader_election.cc:403] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:24:02.036700 21279 consensus_queue.cc:260] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:02.083647 21282 raft_consensus.cc:740] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:02.132997 16367 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:02.133071 21282 consensus_queue.cc:260] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:02.133781 21146 consensus_queue.cc:237] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:02.134321 21282 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:02.134532 16367 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:02.135469 21279 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:02.136893 21320 raft_consensus.cc:3055] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:02.136830 21292 consensus_queue.cc:1059] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:02.137450 21320 raft_consensus.cc:740] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:02.112721 16514 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:02.116549 16515 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:02.117172 16394 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:02.117339 21298 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:02.107620 21169 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:02.118121 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 110806us
I20260709 12:24:02.093245 21323 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:02.028193 21326 consensus_queue.cc:260] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:02.124679 21145 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:02.126514 21271 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:02.146786 21145 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:02.149295 16515 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:02.141832 21138 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:02.152841 16442 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:02.154500 16442 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:02.155304 21245 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:02.155707 21245 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:02.156095 21245 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:02.156509 21329 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:02.110694 21257 consensus_queue.cc:1059] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:24:02.157307 21329 consensus_queue.cc:1059] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:02.158058 21257 raft_consensus.cc:3055] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:02.104688 21338 consensus_queue.cc:1048] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:24:02.109607 16441 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:02.129575 21248 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:02.102056 21251 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:02.132035 21261 raft_consensus.cc:3055] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:24:02.168443 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:02.142812 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.153056 21138 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:02.126930 21130 consensus_queue.cc:260] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:02.152215 16515 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:02.161134 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 48478.5us
I20260709 12:24:02.163154 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:02.165937 21248 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:02.166483 21251 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:02.158336 20763 raft_consensus.cc:3055] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:02.174383 16397 raft_consensus.cc:2393] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 16: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:02.158408 21141 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 15. Current term is 16.
I20260709 12:24:02.181036 16516 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:02.181596 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:02.158408 21257 raft_consensus.cc:740] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:02.182160 21145 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:02.167212 21191 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:02.213157 21257 consensus_queue.cc:260] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:02.214660 21257 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:02.215013  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:24:02.216817 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 134773us
I20260709 12:24:02.218901 21326 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:02.220435 21327 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:02.220937 21327 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:02.221266 21327 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:02.221515 21235 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:24:02.168615 21261 raft_consensus.cc:740] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:02.224115 16401 raft_consensus.cc:2393] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 16: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:02.182266 21251 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:02.231415 21095 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:24:02.243005 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 86696.4us
I20260709 12:24:02.243897 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:02.244122 16442 leader_election.cc:400] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 12. Current term is 13.
I20260709 12:24:02.244524 16442 leader_election.cc:403] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:24:02.245110 21323 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:02.246349 21130 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:24:02.239513 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.182850 20763 raft_consensus.cc:740] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:02.240001 16394 raft_consensus.cc:1275] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:24:02.235881 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:02.188755 21053 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:02.259939 17258 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14.
I20260709 12:24:02.264037 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:02.269636 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:02.271238 17261 raft_consensus.cc:1275] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 12:24:02.270895 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 136470us
I20260709 12:24:02.269788 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:02.279515 16542 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:02.282446 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:02.283120 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.285542 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:02.270071 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:02.189170 21268 consensus_queue.cc:1059] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 12:24:02.292268 21268 raft_consensus.cc:3055] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:02.292809 16367 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:02.292856 21268 raft_consensus.cc:740] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:02.295194 21268 consensus_queue.cc:260] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:02.296096 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:02.296460 21268 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:02.296922 17264 raft_consensus.cc:2393] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 14: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:02.299010 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:02.300451 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:02.301030 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:02.302178 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:02.302558 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.302630 16545 raft_consensus.cc:1240] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:02.187558 20670 consensus_queue.cc:1059] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 12:24:02.304960 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:02.254982 21191 raft_consensus.cc:697] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:02.309206 21191 consensus_queue.cc:237] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:02.313211 16395 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 19 LEADER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:02.314764 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:02.315706 16395 raft_consensus.cc:2393] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 11: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:02.317046 16402 raft_consensus.cc:1275] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:24:02.316805 20763 consensus_queue.cc:260] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:02.318272 20763 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:02.319473 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:02.320757 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:02.278600 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:02.305032 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:24:02.342628 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:02.330256 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.255905 21248 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 12. Current term is 13.
I20260709 12:24:02.352615 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:24:02.353449 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.369172 17270 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:24:02.369962 21251 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:02.371315 16476 raft_consensus.cc:1240] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:02.375974 16403 raft_consensus.cc:1240] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 12:24:02.377912 16401 raft_consensus.cc:1240] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:02.380725 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:02.381260 17144 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:02.383553 21257 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:02.384893 21298 consensus_queue.cc:1048] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:24:02.386653 17271 raft_consensus.cc:1275] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:24:02.363191 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:02.305035 20670 raft_consensus.cc:3055] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:02.387166 17144 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:02.423399 16548 raft_consensus.cc:2410] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 14 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:24:02.418227 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:02.433626 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:02.452708 16542 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:02.232795 21320 consensus_queue.cc:260] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:02.492611 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:02.495328 16551 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:24:02.419759 20670 raft_consensus.cc:740] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:02.499794 21346 consensus_queue.cc:1048] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 12:24:02.501343 16394 raft_consensus.cc:1240] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:02.502537 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:02.503298 16398 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
I20260709 12:24:02.500190 16400 raft_consensus.cc:1240] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:24:02.484687 21095 raft_consensus.cc:697] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:02.509107 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:02.508285 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:02.387671 21261 consensus_queue.cc:260] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:02.481876 21329 consensus_queue.cc:1059] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 12:24:02.516822 16401 raft_consensus.cc:1240] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 10. Current term is 11. Ops: []
W20260709 12:24:02.545902 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.518740 16395 raft_consensus.cc:2393] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 14: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
W20260709 12:24:02.523228 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 203305us
W20260709 12:24:02.525681 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.550647 16403 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:02.552758 16398 raft_consensus.cc:1240] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:02.526067 17143 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:02.555126 17143 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:02.550128 16366 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:02.533120 16400 raft_consensus.cc:1240] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 13. Current term is 14. Ops: []
W20260709 12:24:02.539470 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 387287us
I20260709 12:24:02.533318 21261 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:02.540217 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 153135us
I20260709 12:24:02.541966 21138 raft_consensus.cc:3055] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:02.559304 21138 raft_consensus.cc:740] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:02.559414 17144 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:02.538671 21268 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.560223 21323 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:02.560403 21268 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:02.560822 21323 raft_consensus.cc:697] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:02.517232 21251 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:02.563108 21268 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:02.515758 21347 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:02.492725 21292 consensus_queue.cc:1048] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 12:24:02.563130 21251 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.565265 21145 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:02.565758 21251 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:02.567500 16397 raft_consensus.cc:1275] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 12:24:02.567981 16397 pending_rounds.cc:85] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 7
I20260709 12:24:02.537154 21146 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.568302 16397 pending_rounds.cc:107] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 12:24:02.575798 16397 raft_consensus.cc:2889] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:02.576671 16515 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:02.579244 16366 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:02.583030 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:02.584036 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 291288us
I20260709 12:24:02.546609 21053 raft_consensus.cc:697] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:02.535162 21235 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:24:02.597308 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 351349us
I20260709 12:24:02.594769 16542 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:24:02.598592 16367 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:02.598333 16442 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:02.600975 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.574748 21339 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:02.609097 16476 raft_consensus.cc:1275] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:24:02.568256 21251 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:02.580526 21102 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:02.632186 16441 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
W20260709 12:24:02.629792 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 50617.3us
I20260709 12:24:02.538296 21245 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:02.640183 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 63614.7us
I20260709 12:24:02.647370 21292 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:24:02.649205 21349 consensus_queue.cc:1048] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 12:24:02.651057 21095 consensus_queue.cc:237] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:02.572927 21169 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:02.645277 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 46776.1us
I20260709 12:24:02.593950 21347 raft_consensus.cc:697] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:02.602660 21320 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:02.530659 21327 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:02.577374 21146 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:02.597703 21351 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:02.629575 21346 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:02.628002 21360 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:24:02.703366 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.638967 21339 raft_consensus.cc:697] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:24:02.708504 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.706063 16471 raft_consensus.cc:1240] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:02.708920 17144 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:02.655905 21251 raft_consensus.cc:3055] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:02.710292 21251 raft_consensus.cc:740] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:24:02.710943 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.711819 21251 consensus_queue.cc:260] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:02.713217 21251 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:02.725342 17144 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:02.629655 21235 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:02.724150 21281 consensus_meta.cc:287] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.692s	user 0.012s	sys 0.001s
I20260709 12:24:02.612663 20670 consensus_queue.cc:260] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:02.692667 21338 consensus_queue.cc:1059] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 12:24:02.761402 16403 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:02.710955 21365 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:02.674764 21349 consensus_queue.cc:1048] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 12:24:02.719561 21346 raft_consensus.cc:697] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:02.735694 21276 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:24:02.735579 21351 raft_consensus.cc:697] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 14 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
W20260709 12:24:02.779755 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 120650us
I20260709 12:24:02.740386 21102 consensus_queue.cc:1059] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:24:02.783576 16401 raft_consensus.cc:1275] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:24:02.786299 16367 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:02.669149 21323 consensus_queue.cc:237] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:02.778618 21282 consensus_meta.cc:287] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.644s	user 0.006s	sys 0.005s
I20260709 12:24:02.795997 21358 raft_consensus.cc:2764] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:24:02.769384 21345 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.797108 21327 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:02.798213 21245 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:02.782027 21102 consensus_queue.cc:1059] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 12:24:02.778560 21141 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:02.763521 21236 consensus_queue.cc:1048] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:24:02.784965 21130 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:02.791623 21292 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.802371 21347 consensus_queue.cc:237] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:02.819582 16550 raft_consensus.cc:1240] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 13. Current term is 14. Ops: []
W20260709 12:24:02.843623 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.844732 21146 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:02.848922 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:02.901922 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.846988 21339 consensus_queue.cc:237] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:02.815307 20670 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:02.887008 16367 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:02.845640 21138 consensus_queue.cc:260] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:02.844952 21360 raft_consensus.cc:3055] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:02.783066 21326 consensus_queue.cc:1059] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
W20260709 12:24:02.804894 21271 consensus_meta.cc:287] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.660s	user 0.011s	sys 0.000s
I20260709 12:24:02.751904 21053 consensus_queue.cc:237] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:02.899060 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 62395.6us
I20260709 12:24:02.817418 21261 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:02.820281 21141 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:02.907132 21369 raft_consensus.cc:3055] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:02.907683 21235 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:02.908491 21366 log.cc:927] Time spent T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 12:24:02.845024 21351 consensus_queue.cc:237] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:02.908915 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:02.902632 17142 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.533s (SENT) [suppressed 26 similar messages]
W20260709 12:24:02.917675 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 47350.6us
W20260709 12:24:02.905141 21370 log.cc:927] Time spent T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 12:24:02.808562 21345 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:02.901671 21292 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:02.931658 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.034036 17143 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:24:02.932126 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.899682 21245 consensus_queue.cc:1059] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
W20260709 12:24:02.961930 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 166532us
I20260709 12:24:03.037559 21323 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:02.919080 21360 raft_consensus.cc:740] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:03.037990 21323 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:03.038371 21323 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:03.038738 21360 consensus_queue.cc:260] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:03.040846 21360 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:03.002204 16400 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
I20260709 12:24:02.910308 21276 consensus_queue.cc:1059] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 12:24:03.045671 21276 raft_consensus.cc:3055] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:03.046168 21276 raft_consensus.cc:740] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:03.047210 21276 consensus_queue.cc:260] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:03.005724 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.050311 16515 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:02.972093 21141 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:02.930083 20763 consensus_meta.cc:287] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.611s	user 0.012s	sys 0.002s
I20260709 12:24:03.001235 21138 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:02.962702 21369 raft_consensus.cc:740] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:24:02.924842 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 217019us
I20260709 12:24:03.053393 16516 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:02.947376 21169 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:03.067796 17264 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
I20260709 12:24:02.963550 21261 raft_consensus.cc:697] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:03.070434 16403 raft_consensus.cc:1240] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:02.930915 21352 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:24:03.074450 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 187587us
I20260709 12:24:03.002972 21271 consensus_queue.cc:1048] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.106s
I20260709 12:24:03.079780 21352 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:03.081720 21102 raft_consensus.cc:3055] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:03.082285 16367 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:03.002099 21146 consensus_queue.cc:1048] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.073s
I20260709 12:24:03.067231 21279 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:24:03.095763 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 497449us
W20260709 12:24:03.099249 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 61312us
I20260709 12:24:03.076802 21383 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:03.109426 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:03.160987 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 123932us
I20260709 12:24:03.070967 21282 consensus_queue.cc:1059] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:24:03.163045 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:03.164091 16471 raft_consensus.cc:2368] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13.
I20260709 12:24:03.097043 21102 raft_consensus.cc:740] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:03.164710 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:03.163039 16442 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:03.195818 16399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 1383) took 1081 ms. Trace:
I20260709 12:24:03.202454 16399 rpcz_store.cc:276] 0709 12:24:02.114326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.114544 (+   218us) service_pool.cc:224] Handling call
0709 12:24:02.119766 (+  5222us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:02.324743 (+204977us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:02.324780 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:02.357559 (+ 32779us) log.cc:844] Serialized 26 byte log entry
0709 12:24:02.610171 (+252612us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:02.610219 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:02.610286 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:03.195552 (+585266us) raft_consensus.cc:1685] finished
0709 12:24:03.195589 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:03.195752 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32868,"raft.queue_time_us":236571,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":28699,"thread_start_us":408390,"threads_started":2,"wal-append.queue_time_us":336403}
I20260709 12:24:03.160696 21245 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:03.204980 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:03.163463 21355 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:24:03.205614 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 123483us
I20260709 12:24:03.206486 16367 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:03.207077 21102 consensus_queue.cc:260] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:03.207734 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:03.208443 21102 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:03.209373 21329 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:03.086764 21339 consensus_queue.cc:1059] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 12:24:03.209295 16441 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:03.210688 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:03.205018 21245 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:03.211194 16441 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:03.211352 17254 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 19.
W20260709 12:24:03.109767 21257 consensus_meta.cc:287] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.726s	user 0.010s	sys 0.003s
I20260709 12:24:03.206581 21292 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:03.218597 16367 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:03.096836 21352 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:03.227839 16367 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:03.228549 21292 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:24:03.219756 21355 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:03.165717 21383 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:03.109519 21192 consensus_queue.cc:1059] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.123s
I20260709 12:24:03.230017 21292 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:24:03.271565 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:03.247695 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 345057us
I20260709 12:24:02.916991 21346 consensus_queue.cc:237] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:03.305066 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 50637.3us
I20260709 12:24:03.306079 16441 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:03.243640 21323 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:03.252732 20763 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:03.247097 21257 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:03.252368 21169 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:03.194116 21385 consensus_queue.cc:1048] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.138s
I20260709 12:24:03.300464 21388 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:24:03.331364 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 112741us
W20260709 12:24:03.332753 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 233167us
I20260709 12:24:03.318729 20763 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:03.336925 17258 raft_consensus.cc:2393] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 13: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
W20260709 12:24:03.338161 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:03.341936 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:03.342491 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:03.342952 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.342612 16442 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:03.345158 21372 log.cc:927] Time spent T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.258s	user 0.000s	sys 0.001s
W20260709 12:24:03.346901 21387 log.cc:927] Time spent T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
W20260709 12:24:03.326272 21251 consensus_meta.cc:287] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.613s	user 0.011s	sys 0.002s
W20260709 12:24:03.348300 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 120591us
I20260709 12:24:03.320523 21248 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:03.342013 20763 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:03.406574 21382 log.cc:927] Time spent T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.351s	user 0.000s	sys 0.001s
I20260709 12:24:03.337524 21378 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:03.404872 21192 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.451640  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:03.233155 21369 consensus_queue.cc:260] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:03.454778 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 203048us
I20260709 12:24:03.496819 16515 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:03.449457 21292 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:03.476004 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.318928 21261 consensus_queue.cc:237] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:03.407634 21326 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:03.446161 21248 raft_consensus.cc:697] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:24:03.525835 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 202133us
I20260709 12:24:03.527871 16515 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:03.450533 21378 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:03.535567 16441 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:03.340603 21388 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:03.575778 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 48017us
I20260709 12:24:03.577149 16515 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:03.530900 21279 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:03.562943 21369 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:03.568467 21358 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:03.606484 21250 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:24:03.610955 21250 consensus_queue.cc:1059] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:03.517613 21192 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:03.541234 21378 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:03.529342 21292 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:03.580866 21338 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:24:03.540082 21326 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:03.628506 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:03.632862 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:03.648075 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 153820us
W20260709 12:24:03.652686 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 117188us
I20260709 12:24:03.629444 21245 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:03.667380 21326 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:03.668514 21397 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.669256 21397 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:03.670713 21138 consensus_meta.cc:287] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.595s	user 0.005s	sys 0.005s
I20260709 12:24:03.639358 21347 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.676685 16367 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:03.682365 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 84124.9us
W20260709 12:24:03.687565 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.687316 16515 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:03.689801 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:03.690342 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:03.690888 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.690927 21292 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:03.692061 21406 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:03.692813 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:03.692739 21406 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:03.693912 17254 raft_consensus.cc:2410] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 14 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
W20260709 12:24:03.694499 21390 log.cc:927] Time spent T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.394s	user 0.000s	sys 0.001s
W20260709 12:24:03.696733 21389 log.cc:927] Time spent T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 12:24:03.698093 16396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 1394) took 1041 ms. Trace:
I20260709 12:24:03.698478 16396 rpcz_store.cc:276] 0709 12:24:02.656256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.750824 (+ 94568us) service_pool.cc:224] Handling call
0709 12:24:02.773936 (+ 23112us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:02.783975 (+ 10039us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:02.784010 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:02.816985 (+ 32975us) log.cc:844] Serialized 26 byte log entry
0709 12:24:03.163555 (+346570us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:03.163590 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.163663 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:03.697839 (+534176us) raft_consensus.cc:1685] finished
0709 12:24:03.697866 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:03.698003 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":320,"raft.queue_time_us":18941,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":4981,"thread_start_us":251957,"threads_started":1,"wal-append.queue_time_us":430749}
I20260709 12:24:03.700735 21404 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.701649 21404 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:03.700816 21374 log.cc:927] Time spent T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.391s	user 0.001s	sys 0.001s
W20260709 12:24:03.703961 21393 log.cc:927] Time spent T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.251s	user 0.000s	sys 0.001s
W20260709 12:24:03.705093 21396 log.cc:927] Time spent T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.172s	user 0.000s	sys 0.000s
I20260709 12:24:03.702101 21392 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.705843 21392 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:03.709168 21395 log.cc:927] Time spent T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:24:03.709403 21404 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:03.710832 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:03.683900 21383 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:03.714110 21138 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:03.714429 21192 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:03.715623 16367 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:03.719012 21141 consensus_meta.cc:287] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.643s	user 0.000s	sys 0.008s
I20260709 12:24:03.719344 16367 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:03.719944 21323 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:03.720594 21397 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:03.696041 17261 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159"
dest_uuid: "ef57beab0e1d40d585983873c55107e8"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:45120
I20260709 12:24:03.777096 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:03.781092 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 93905.3us
I20260709 12:24:03.779431 17261 raft_consensus.cc:606] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:03.743403 16367 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:03.785655 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 128951us
W20260709 12:24:03.787498 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.687642 21245 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:24:03.793028 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.731884 21351 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:03.808830 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:03.807854 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:03.810897 16469 raft_consensus.cc:2368] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13.
W20260709 12:24:03.812243 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:03.814189 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:03.845773 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 102450us
I20260709 12:24:03.847980 16367 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:03.744234 21292 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:03.851122 16550 raft_consensus.cc:1240] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:03.792055 21383 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.792577 21182 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:24:03.854813 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:03.776702 21169 raft_consensus.cc:697] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:03.803078 21245 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:03.812772 21365 consensus_meta.cc:287] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 1.017s	user 0.008s	sys 0.005s
I20260709 12:24:03.722201 21347 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:03.726143 21141 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:03.687009 21248 consensus_queue.cc:237] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:03.862255 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:03.869459 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:03.872251 16542 raft_consensus.cc:2393] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 10: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:03.780231 21355 consensus_queue.cc:1048] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.077s
W20260709 12:24:03.856640 20670 consensus_meta.cc:287] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.934s	user 0.010s	sys 0.003s
W20260709 12:24:03.862471 21361 log.cc:927] Time spent T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:24:03.874881 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:03.880086 16550 raft_consensus.cc:2393] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 11: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:03.863590 21245 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:03.863106 21397 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.881582 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:03.883531 17270 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 15: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:03.862792 21323 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.867241 21365 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:03.866377 21392 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:03.865811 21351 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:03.888293 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:03.863248 21383 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:03.893404 21169 consensus_queue.cc:237] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:03.905606 20763 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:03.906513 17264 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45224 (request call id 1243) took 1608 ms (client timeout 1815 ms). Trace:
I20260709 12:24:03.865639 21292 raft_consensus.cc:697] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 14 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:03.914220 16551 raft_consensus.cc:2393] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 13: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:03.927443 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:03.932112 16543 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:24:03.884209 20670 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:03.929340 17264 rpcz_store.cc:269] 0709 12:24:02.298308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.356544 (+ 58236us) service_pool.cc:224] Handling call
0709 12:24:03.906337 (+1549793us) spinlock_profiling.cc:243] Waited 422 ms on lock 0x7b540005e454. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:03.906448 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":971597696}
I20260709 12:24:03.940968 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:24:03.921661 21102 consensus_meta.cc:287] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.713s	user 0.002s	sys 0.012s
I20260709 12:24:03.935478 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:03.948799 16546 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:24:03.953091 21383 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:03.957340 21292 consensus_queue.cc:237] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:03.960927 16544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 1007) took 1307 ms. Trace:
I20260709 12:24:03.961401 16544 rpcz_store.cc:276] 0709 12:24:02.654056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.654261 (+   205us) service_pool.cc:224] Handling call
0709 12:24:02.654726 (+   465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:02.727673 (+ 72947us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:24:02.728012 (+   339us) log.cc:844] Serialized 14 byte log entry
0709 12:24:03.779912 (+1051900us) spinlock_profiling.cc:243] Waited 2.07 ms on lock 0x7b1800125f20. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d16c0a 00007f287c456219 00007f2872d9ea40 00007f2872d994e4 00007f2872d99455 00007f28798813c7 00007f2875e939a7 00007f2875ea30bf 00007f2875ea3038
0709 12:24:03.780139 (+   227us) log.cc:844] Serialized 14 byte log entry
0709 12:24:03.780325 (+   186us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:03.780394 (+    69us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:03.780446 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.960691 (+180245us) spinlock_profiling.cc:243] Waited 72.6 ms on lock 0x7b54003de854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:03.960741 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:03.960871 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":114042,"prepare.queue_time_us":1014647,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":6731,"raft.queue_time_us":57478,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":13892,"spinlock_wait_cycles":171847296}
I20260709 12:24:03.964514 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:03.918787 21355 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:24:03.971086 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 178514us
I20260709 12:24:03.971006 16441 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:04.002887 16543 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:24:04.005343 16441 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
W20260709 12:24:04.011808 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:04.017853 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 684060us
I20260709 12:24:04.018734 16366 leader_election.cc:400] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13.
I20260709 12:24:04.019492 16366 leader_election.cc:403] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:24:04.020459 16441 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:04.022238 21416 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:04.029816 21416 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:04.030297 21416 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:04.034389 21276 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:03.918545 21323 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:04.039386 21323 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:04.022002 21245 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:04.041719 16469 raft_consensus.cc:1275] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:24:03.918447 21397 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:04.050541 21413 log.cc:927] Time spent T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 12:24:04.009557 21358 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:04.008847 21175 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.016649 21365 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:24:04.016886 21027 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:24:04.058688 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 210984us
I20260709 12:24:04.042462 21268 raft_consensus.cc:3055] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:04.099372 16367 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:03.969030 20670 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:04.055809 21360 consensus_meta.cc:287] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 1.014s	user 0.011s	sys 0.003s
I20260709 12:24:04.042548 21355 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13.
I20260709 12:24:04.048686 21323 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.052831 21369 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.052544 21102 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:04.112968 21369 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:04.098361 21392 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.120249 16367 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:04.103904 21268 raft_consensus.cc:740] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:04.128903 20763 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:04.117265 21323 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:04.132463 21415 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.112435 21383 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:04.142148 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:04.155381 16543 raft_consensus.cc:2410] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:24:04.145452 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:04.117238 21358 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.103421 21175 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:04.106421 21378 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:04.124110 21400 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:04.106251 21027 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:04.167587 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:04.173604 16542 raft_consensus.cc:1240] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:04.178416 16550 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:24:04.178833 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:04.117116 21102 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:04.179142 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:04.179009 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:04.184819 16544 raft_consensus.cc:2410] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:24:04.132488 21403 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:04.139200 21292 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:24:04.192375 17271 raft_consensus.cc:1275] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:24:04.192821 17271 pending_rounds.cc:85] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8: Aborting all ops after (but not including) 8
I20260709 12:24:04.193068 17271 pending_rounds.cc:107] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:24:04.193300 17271 raft_consensus.cc:2889] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:04.126946 21392 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:04.187029 20763 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:04.203507 20763 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:04.204347 21102 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:04.205822 20763 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:04.199901 21365 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.206699 21365 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:04.206753 21416 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:04.176556 21358 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:04.209128 16398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 1270) took 1256 ms. Trace:
I20260709 12:24:04.209234 21365 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:04.210294 21358 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:04.181070 21169 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:04.211122 21169 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:04.213635 21169 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:04.184558 21369 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:04.195533 21403 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:04.216884 21323 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:04.217904 21182 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.218096 21397 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:04.194715 21320 consensus_meta.cc:287] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 1.500s	user 0.002s	sys 0.013s
I20260709 12:24:04.218425 21182 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:04.218914 21423 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:24:04.191970 16442 connection.cc:256] RPC call timeout handler was delayed by -1.430530557000111s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1549s for a response
W20260709 12:24:04.219956 16442 connection.cc:256] RPC call timeout handler was delayed by -1.3783587919997444s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1579s for a response
I20260709 12:24:04.220958 21392 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:04.222038 16469 raft_consensus.cc:1240] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:04.223254 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:04.223912 16477 raft_consensus.cc:1240] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:04.224769 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:04.225275 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:04.225687 16468 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 15: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:04.224879 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:04.226135 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:04.226414 16477 raft_consensus.cc:2368] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 15. Current term is 16.
I20260709 12:24:04.226508 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:04.220976 21424 log.cc:927] Time spent T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:24:04.223919 16471 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:24:04.228673 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:04.200592 21268 consensus_queue.cc:260] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:04.230384 21169 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.231348 21378 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:04.186290 21347 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:04.231657 21169 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:04.233188 16472 raft_consensus.cc:1240] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:04.233635 21416 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:04.235020 21352 consensus_meta.cc:287] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 1.006s	user 0.006s	sys 0.008s
I20260709 12:24:04.235378 21169 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:04.236819 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:04.165416 21383 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:04.157123 21351 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 12:24:04.231870 16469 raft_consensus.cc:1240] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:04.232779 21027 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:04.235553 21352 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:04.242988 17255 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14.
I20260709 12:24:04.224681 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:04.253629 16469 raft_consensus.cc:1217] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Deduplicated request from leader. Original: 10.7->[11.8-13.9]   Dedup: 13.9->[]
I20260709 12:24:04.253923 16474 raft_consensus.cc:1240] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:04.255244 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:04.256685 16477 raft_consensus.cc:1240] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:04.256812 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:04.257146 16468 raft_consensus.cc:1240] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:04.257620 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:04.257874 21169 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:04.258447 21169 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:04.258873 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:04.253940 16472 raft_consensus.cc:1240] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:04.258128 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:04.260675 21169 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:04.260068 16469 raft_consensus.cc:1240] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 13. Current term is 14. Ops: [11.8-13.9]
I20260709 12:24:04.261793 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:04.262558 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:04.264173 16471 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:24:04.263242 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:04.262849 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:04.196597 21400 raft_consensus.cc:697] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:04.157548 21415 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:04.209599 16398 rpcz_store.cc:276] 0709 12:24:02.952574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.953135 (+   561us) service_pool.cc:224] Handling call
0709 12:24:03.008094 (+ 54959us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:03.079295 (+ 71201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:03.079326 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:03.101140 (+ 21814us) log.cc:844] Serialized 26 byte log entry
0709 12:24:03.319843 (+218703us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:03.319910 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:04.208738 (+888828us) spinlock_profiling.cc:243] Waited 66.6 ms on lock 0x7b540011fb54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:04.208785 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:04.208832 (+    47us) raft_consensus.cc:1685] finished
0709 12:24:04.208845 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:04.209069 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13879,"raft.queue_time_us":6062,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":6584,"spinlock_wait_cycles":153246592,"thread_start_us":100970,"threads_started":1,"wal-append.queue_time_us":210874}
I20260709 12:24:04.230518 21268 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:04.268702 16477 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 15: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:04.269485 17258 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:24:04.274217 16474 raft_consensus.cc:2410] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:24:04.271603 16473 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
I20260709 12:24:04.271158 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:04.238077 21141 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:04.276544 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:04.280578 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:04.270732 21169 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:04.328055 21415 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:04.328896 16475 raft_consensus.cc:2393] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 15: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
W20260709 12:24:04.329303 16442 leader_election.cc:336] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.549s (SENT)
I20260709 12:24:04.329574 21400 consensus_queue.cc:237] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:04.264822 21404 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.331286 21027 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.273344 21420 consensus_queue.cc:1048] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 12:24:04.333781 21175 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:04.284780 16400 raft_consensus.cc:1240] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:04.337982 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:04.353642 16472 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:24:04.277289 16477 raft_consensus.cc:2468] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
W20260709 12:24:04.339293 16367 connection.cc:256] RPC call timeout handler was delayed by -1.0552989640000305s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21660000000000001s for a response
I20260709 12:24:04.355824 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:04.357836 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:04.358060 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:04.358882 16397 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:04.350111 16469 raft_consensus.cc:1275] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:24:04.345407 21404 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:04.356012 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:04.361199 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:04.347309 16442 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:04.379328 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:04.442317 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:04.440907 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:04.421582 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:04.369169 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:04.385576 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:04.412827 16397 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:24:04.379666 16398 raft_consensus.cc:1240] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:04.380134 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:04.415251 17270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45224 (request call id 1250) took 1213 ms. Trace:
I20260709 12:24:04.497567 16400 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:04.448168 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:04.392517 16470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1518) took 1289 ms. Trace:
I20260709 12:24:04.440066 16473 raft_consensus.cc:1275] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:24:04.502648 16544 raft_consensus.cc:2393] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 13: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
W20260709 12:24:04.425984 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:04.500278 17270 rpcz_store.cc:276] 0709 12:24:03.202223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.202371 (+   148us) service_pool.cc:224] Handling call
0709 12:24:04.414967 (+1212596us) spinlock_profiling.cc:243] Waited 678 ms on lock 0x7b54003f0554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:04.415167 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1558481792}
I20260709 12:24:04.504868 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:04.504997 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:04.505667 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:04.499752 16399 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:24:04.507145 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:04.508662 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:04.510481 21329 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:04.449992 21356 raft_consensus.cc:3055] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:04.446352 21376 raft_consensus.cc:3055] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:04.501184 16470 rpcz_store.cc:276] 0709 12:24:03.103573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.160445 (+ 56872us) service_pool.cc:224] Handling call
0709 12:24:03.194395 (+ 33950us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:03.499459 (+305064us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:24:03.652543 (+153084us) log.cc:844] Serialized 14 byte log entry
0709 12:24:03.857494 (+204951us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:03.867659 (+ 10165us) log.cc:844] Serialized 52 byte log entry
0709 12:24:03.867866 (+   207us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:03.958529 (+ 90663us) spinlock_profiling.cc:243] Waited 6.64 ms on lock 0x7b18002ac280. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c522 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:24:04.105305 (+146776us) log.cc:844] Serialized 14 byte log entry
0709 12:24:04.177194 (+ 71889us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:04.392119 (+214925us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b540011add4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:04.392177 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:04.392227 (+    50us) raft_consensus.cc:1685] finished
0709 12:24:04.392241 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:04.392449 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54860,"prepare.queue_time_us":205629,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":2198,"raft.queue_time_us":192251,"raft.run_cpu_time_us":399,"raft.run_wall_time_us":14024,"spinlock_wait_cycles":320769280,"thread_start_us":740940,"threads_started":5,"wal-append.queue_time_us":228413}
I20260709 12:24:04.530658 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:04.532680 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:04.533404 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:04.532225 16514 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:04.534650 17255 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45230 (request call id 1266) took 2241 ms (client timeout 2166 ms). Trace:
I20260709 12:24:04.575753 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:04.437963 21175 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:04.443432 21338 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:04.402174 21270 raft_consensus.cc:3055] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:04.448949 21146 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.580523 16398 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
I20260709 12:24:04.405902 20670 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:04.361805 21245 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:04.404592 21383 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:04.410902 21352 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:04.387686 21345 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:04.595865 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:04.352362 21169 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:04.597359 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:04.598049 21169 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:04.598507 16543 raft_consensus.cc:2393] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 13: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
W20260709 12:24:04.601279 21437 log.cc:927] Time spent T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:24:04.601410 21360 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:04.601995 21360 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:04.602380 21360 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:04.583495 21146 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:04.607594 21146 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:04.609520 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:04.447700 21388 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:04.576519 16544 raft_consensus.cc:2468] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 10.
W20260709 12:24:04.576246 17255 rpcz_store.cc:269] 0709 12:24:02.292852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.352208 (+ 59356us) service_pool.cc:224] Handling call
0709 12:24:04.534409 (+2182201us) spinlock_profiling.cc:243] Waited 1.87 s on lock 0x7b54005f7b54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:04.534545 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4304071424}
W20260709 12:24:04.578110 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 45973.8us
I20260709 12:24:04.613664 16400 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
W20260709 12:24:04.615193 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:04.615747 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:04.616012 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:04.616197 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:04.437109 21095 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.355798 21027 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:04.581723 16394 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
W20260709 12:24:04.583609 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 254325us
I20260709 12:24:04.541517 21356 raft_consensus.cc:740] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:04.582557 21338 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:04.553220 21376 raft_consensus.cc:740] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:04.580921 21270 raft_consensus.cc:740] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:24:04.625658 16442 leader_election.cc:336] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.579s (SENT)
I20260709 12:24:04.649458 16442 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:04.637038 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 216408us
I20260709 12:24:04.640820 21401 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.642872 16542 raft_consensus.cc:1275] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:24:04.643905 21027 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:04.625344 21276 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.666193 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:04.583276 21175 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:04.671007 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:04.674540 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:04.674695 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:04.682562 16548 raft_consensus.cc:2393] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 15: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:04.683517 16549 raft_consensus.cc:1275] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 12:24:04.685470 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 59851.2us
I20260709 12:24:04.691413 21356 consensus_queue.cc:260] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:04.693188 21356 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:04.673230 21130 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.697005 21130 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:04.699738 21130 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:04.699635 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:04.669127 21276 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:04.700759 16471 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 15: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:04.702152 21276 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:04.703326 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:04.704671 21400 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.705216 21400 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:04.707325 21400 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:04.707602 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:04.708474 16471 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:24:04.709736 16547 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "f99066d13e524238aec6aa2024a8bd50"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:24:04.710198 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:04.710351 16547 raft_consensus.cc:606] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Received request to transfer leadership to TS f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:04.710984 16547 raft_consensus.cc:614] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:04.711472 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:04.695684 21360 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:04.712538 21276 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.713100 21276 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:04.713887 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:04.714548 16471 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:04.715656 21276 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:04.718556 21175 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:04.675463 21251 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:04.720670 21360 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:04.721302 21379 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.721869 21379 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:04.724696 21338 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:04.724927 16469 raft_consensus.cc:2410] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:24:04.727131 21182 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:04.728451 21406 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:04.729288 21403 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:04.724061 21379 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:04.669471 21401 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:04.733893 21027 consensus_queue.cc:1048] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:24:04.734372 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:04.735384 21409 consensus_queue.cc:1059] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:04.736495 21409 raft_consensus.cc:3055] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:04.736855 21409 raft_consensus.cc:740] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:04.689247 21388 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.737625 16402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 1257) took 2426 ms. Trace:
I20260709 12:24:04.737918 21388 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:04.738240 16402 rpcz_store.cc:276] 0709 12:24:02.311146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.318032 (+  6886us) service_pool.cc:224] Handling call
0709 12:24:02.318430 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:02.589771 (+271341us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:02.589821 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:02.624206 (+ 34385us) log.cc:844] Serialized 26 byte log entry
0709 12:24:02.696571 (+ 72365us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:02.696608 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:02.696677 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:04.733317 (+2036640us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b240054b618. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872c229c4 00007f2872c20acb 00007f2872ddbdc9 00007f2872ddba55 00007f2872ddafb0 00007f2875a51488 00007f2875a51be8 00007f2875a29398 00007f2875a2476a 00007f2875a23bb6
0709 12:24:04.737193 (+  3876us) raft_consensus.cc:1685] finished
0709 12:24:04.737226 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:04.737524 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":218755,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":74,"spinlock_wait_cycles":23854848,"thread_start_us":318886,"threads_started":2,"wal-append.queue_time_us":99474}
I20260709 12:24:04.741489 16475 raft_consensus.cc:1275] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:24:04.741535 17142 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:04.688097 17143 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:04.644820 21095 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:04.744982 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 56958.8us
I20260709 12:24:04.744585 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:04.750350 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:04.750553 17143 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:04.751051 16477 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:24:04.740175 21388 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:04.754487 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:04.757687 16473 raft_consensus.cc:1240] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:04.758644 16477 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
W20260709 12:24:04.764410 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:04.771447 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:04.772272 16472 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:04.744999 21027 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.744580 21339 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:04.773947 16477 raft_consensus.cc:1275] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:24:04.781250 16473 raft_consensus.cc:1240] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:04.781376 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:04.783015 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:04.748279 21379 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:04.784010 16543 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
I20260709 12:24:04.757517 21369 raft_consensus.cc:2804] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:04.789371 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:04.789714 17142 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:04.785199 16441 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:04.792192 21169 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:04.792896 21169 raft_consensus.cc:697] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:04.793983 21169 consensus_queue.cc:237] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:04.737663 21409 consensus_queue.cc:260] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:04.783418 21420 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.800366 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:04.806977 16547 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:24:04.802536 21406 raft_consensus.cc:2804] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:04.805486 21401 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:04.785820 21130 consensus_queue.cc:1059] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:24:04.791009 21369 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:04.780357 21276 consensus_queue.cc:1059] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 12:24:04.782024 21027 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:04.756878 21376 consensus_queue.cc:260] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:04.815397 17142 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:04.817643 21409 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:04.808636 21406 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:04.820716 17142 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:04.812521 21369 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:04.824651 16551 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
I20260709 12:24:04.834012 17142 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:04.838258 17142 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:04.841176 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:04.842156 16474 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:04.821938 21379 raft_consensus.cc:3055] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:04.845088 21379 raft_consensus.cc:740] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:04.809828 21420 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:04.848990 21379 consensus_queue.cc:260] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:04.822734 21130 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:04.822799 21406 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:04.857097 21379 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:04.860951 21095 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:04.861029 21404 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:04.845909 21339 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:04.863732 21420 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:04.864379 16473 raft_consensus.cc:1275] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:24:04.854473 21412 raft_consensus.cc:3055] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:04.865505 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:04.838033 21326 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:24:04.865579 21412 raft_consensus.cc:740] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:04.867861 21412 consensus_queue.cc:260] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:04.840085 21276 consensus_queue.cc:1048] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:24:04.874995 16473 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
W20260709 12:24:04.876354 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:04.873813 21027 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:04.844054 21271 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:04.883535 16514 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:04.841965 17142 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:24:04.899930 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 58083.6us
I20260709 12:24:04.904716 21192 raft_consensus.cc:2804] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:04.882812 21412 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:04.905356 21192 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:04.905980 21192 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:04.905366 21369 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:04.915346 21369 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:04.884891 21250 consensus_queue.cc:1048] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:24:04.916031 21276 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:04.916540 21276 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:04.916927 21276 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:04.921366 16442 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:04.900254 16514 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:04.936118 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:04.938905 16514 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:04.940618 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:04.944736 17255 raft_consensus.cc:1275] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:24:04.945113 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:04.938426 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:04.946949 16542 raft_consensus.cc:2393] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 10: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:04.945201 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:04.948254 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:04.948541 16442 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:04.948861 17266 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:04.949764 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:04.950295 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:04.949785 16442 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:04.952073 16442 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:04.953140 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:04.953388 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:04.954193 16550 raft_consensus.cc:2410] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 18 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:24:04.958886 16474 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 17.
I20260709 12:24:04.959403 16442 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:04.960369 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:04.961167 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:04.962780 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:04.962878 16441 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:04.963717 16543 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:04.958379 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:04.898594 21271 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:04.965930 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:04.971700 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:04.967746 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:04.960129 16514 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:04.966866 16551 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:04.961016 17261 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
I20260709 12:24:04.980819 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:04.984392 21270 consensus_queue.cc:260] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:04.950882 21410 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:04.985579 21270 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:04.967267 21409 consensus_queue.cc:1048] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:24:04.938119 21250 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:04.984683 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:04.994529 16401 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:24:04.994560 16442 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:05.001528 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:05.001902 17264 raft_consensus.cc:1275] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 12:24:05.003521 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 43490.7us
W20260709 12:24:05.004827 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:05.005501 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:05.009728 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:05.010375 17269 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
I20260709 12:24:04.972963 21248 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:05.011193 21406 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:05.012698 21236 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:05.013507 21406 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:05.014101 16442 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:05.020359 21245 consensus_queue.cc:1059] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 12:24:05.021212 21236 raft_consensus.cc:697] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 17 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:05.021488 16442 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:05.022195 21236 consensus_queue.cc:237] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:05.023170 21400 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:05.014648 21435 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:05.024211 21435 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:05.024701 21435 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:05.025415 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:05.026124 17258 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14.
I20260709 12:24:04.993257 21271 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:04.997320 21250 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:05.036818 21250 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:04.966009 21095 raft_consensus.cc:2804] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:05.040657 21095 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:05.040869 16474 raft_consensus.cc:1240] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:05.041285 21095 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:04.992847 21251 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:05.048684 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:05.049417 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:05.049748 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:05.050280 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:05.050876 17269 raft_consensus.cc:2376] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11. Re-sending same reply.
I20260709 12:24:05.053849 21436 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:24:05.054010 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 91184.9us
I20260709 12:24:04.968250 21346 consensus_queue.cc:1059] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.087s
I20260709 12:24:04.968739 21329 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:05.049830 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:05.021552 21245 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:05.059744 17261 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
I20260709 12:24:05.065192 16441 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:05.066542 16551 raft_consensus.cc:3055] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:05.022765 16442 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:05.068771 16551 raft_consensus.cc:740] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:24:05.069558 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 46858.1us
I20260709 12:24:05.070554 16442 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:04.938261 21442 consensus_queue.cc:1048] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 12:24:04.968617 21385 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:24:05.074316 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:05.081473 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:05.083894 17266 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:24:05.087738 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:04.954790 21352 consensus_queue.cc:1059] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.111s
I20260709 12:24:05.078872 16551 consensus_queue.cc:260] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:04.960673 21345 consensus_queue.cc:1048] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.112s
I20260709 12:24:05.097229 16551 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:05.090801 16396 raft_consensus.cc:1275] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:24:05.097608 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:05.099607 16547 raft_consensus.cc:1275] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:24:05.096148 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:05.103921 16394 raft_consensus.cc:2393] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 11: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:05.079162 21436 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:05.109761 16543 raft_consensus.cc:2468] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
W20260709 12:24:05.109519 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 44443.3us
W20260709 12:24:05.117252 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 46853.2us
I20260709 12:24:05.117308 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:05.118139 17266 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:24:05.118194 16442 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:05.125008 17269 raft_consensus.cc:1240] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 11. Current term is 12. Ops: []
W20260709 12:24:05.140651 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:05.156584 16402 raft_consensus.cc:1275] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:24:05.117058 21436 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:05.161732 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:05.126291 21385 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:05.175360 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:05.177201 16394 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:24:05.179718 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:05.179857 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 232112us
I20260709 12:24:05.183187 17258 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
I20260709 12:24:05.139218 21169 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:05.184013 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:05.186461 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:05.189914 16398 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:24:05.210084 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:05.164234 21436 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:05.210837 17270 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
W20260709 12:24:05.219838 21459 log.cc:927] Time spent T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:24:05.223331 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:05.224114 17258 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:24:05.233203 16546 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:24:05.187510 21169 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:05.234992 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:05.241248 17264 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:05.164633 21410 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.191748 21442 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:05.247668 21442 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:05.193626 21385 raft_consensus.cc:697] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:05.257680 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:05.289363 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:05.250921 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:05.301043 17258 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
I20260709 12:24:05.299724 16394 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 10.
I20260709 12:24:05.319398 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:05.319898 16398 raft_consensus.cc:1275] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:24:05.245590 21169 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:05.333220 16516 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:05.336862 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:05.341536 16516 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:05.099383 21027 consensus_queue.cc:1175] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "52ab50b0237346c1a13ede947f6a63c2" responder_term: 15 status { last_received { term: 15 index: 10 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 14. Current term is 15. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:24:05.349167 17270 raft_consensus.cc:1240] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:24:05.355630 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:05.356199 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 153201us
I20260709 12:24:05.357896 17271 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 16.
I20260709 12:24:05.236519 21276 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:05.300716 21345 consensus_queue.cc:1059] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 12:24:05.344749 21236 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:05.367362 21236 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:05.367854 21236 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:05.368497 21281 consensus_queue.cc:1048] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:05.359663 21270 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:05.373402 21442 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:05.376119 21345 raft_consensus.cc:3055] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:05.301017 21435 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:05.382311 17271 raft_consensus.cc:1275] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 8. (term mismatch)
I20260709 12:24:05.358435 16367 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:05.390370 17271 pending_rounds.cc:85] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8: Aborting all ops after (but not including) 7
I20260709 12:24:05.390662 17271 pending_rounds.cc:107] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 12:24:05.391021 17271 raft_consensus.cc:2889] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:05.388656 17258 raft_consensus.cc:1275] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:24:05.393654 17270 raft_consensus.cc:1275] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:24:05.342250 16398 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
I20260709 12:24:05.395432 17264 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:24:05.397873 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:05.399235 17258 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:24:05.401849 17261 raft_consensus.cc:1240] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:05.376654 21345 raft_consensus.cc:740] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:05.311735 21410 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:05.422534 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:05.423888 17264 raft_consensus.cc:2410] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:24:05.408807 21442 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:05.428789 21442 raft_consensus.cc:697] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:05.429709 21442 consensus_queue.cc:237] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:05.405503 21447 consensus_queue.cc:1048] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:24:05.436645 21345 consensus_queue.cc:260] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:05.437978 21345 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:05.438140 21276 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:05.439147 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:05.441466 17258 raft_consensus.cc:1275] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 12:24:05.441915 17258 pending_rounds.cc:85] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8: Aborting all ops after (but not including) 10
I20260709 12:24:05.442152 17258 pending_rounds.cc:107] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 12:24:05.442394 17258 raft_consensus.cc:2889] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:05.443500 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:05.445374 17143 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:05.446543 17143 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:05.447700 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:05.418982 16516 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:05.449498 17143 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:05.448451 17258 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
I20260709 12:24:05.453544 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:05.456267 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:05.459643 17263 raft_consensus.cc:1275] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:24:05.460414 21410 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:05.458103 21276 raft_consensus.cc:2804] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:05.461548 21276 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:05.461872 21276 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:05.467412 17264 raft_consensus.cc:1240] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:05.452453 21355 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.464237 17269 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:24:05.447252 21369 consensus_queue.cc:1048] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 12:24:05.457093 21406 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:05.455155 21270 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:05.420353 21271 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:05.458567 16516 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:05.473256 21412 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:05.454847 21385 consensus_queue.cc:237] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:05.477761 17270 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7fe8595f696842c8821880524389c68b"
dest_uuid: "ef57beab0e1d40d585983873c55107e8"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:45120
I20260709 12:24:05.495080 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:05.495285 16398 raft_consensus.cc:1240] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:05.484563 21248 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:05.497709 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:05.492946 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:05.510881 17270 raft_consensus.cc:606] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:05.511166 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:05.517369 16402 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
W20260709 12:24:05.511587 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 53863us
I20260709 12:24:05.513561 16396 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:05.520745 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:05.524009 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:05.485661 21355 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:05.526088 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:05.526710 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:05.528551 16398 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 15: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:05.513110 21412 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:05.530915 17254 raft_consensus.cc:1240] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:24:05.532624 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:05.535046 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:05.536130 16401 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
I20260709 12:24:05.500221 21406 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:05.537278 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 87766us
W20260709 12:24:05.540501 21250 consensus_meta.cc:287] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.503s	user 0.014s	sys 0.000s
W20260709 12:24:05.542770 21192 consensus_meta.cc:287] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.636s	user 0.015s	sys 0.000s
I20260709 12:24:05.548693 21192 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:05.549371 21236 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:05.550900 21192 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:05.551527 21236 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:05.551764 16545 raft_consensus.cc:2393] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 13: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
W20260709 12:24:05.544364 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:05.540107 21406 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:05.552482 21479 consensus_queue.cc:1048] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:24:05.553002 21469 log.cc:927] Time spent T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:24:05.546280 17143 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:05.520598 21447 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:05.555567 16394 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
I20260709 12:24:05.509310 21270 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:05.557444 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:05.523893 21473 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:24:05.560237 21468 log.cc:927] Time spent T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:24:05.560978 21471 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.561859 21471 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:05.563329 21192 consensus_queue.cc:1048] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:05.563932 21471 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:05.565340 21271 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:05.566846 21169 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:05.569694 21436 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:05.567113 16516 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:05.573213 21355 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:05.585573 16441 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:05.547602 21281 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:05.574308 21095 consensus_meta.cc:287] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.532s	user 0.006s	sys 0.006s
I20260709 12:24:05.558024 21250 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:05.537765 21412 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:05.584939 16474 raft_consensus.cc:1275] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:24:05.589200 16441 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:05.591307 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:05.592417 16400 raft_consensus.cc:1275] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:24:05.600543 21346 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:05.601123 21346 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:05.587862 21146 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:05.604751 21146 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:05.605089 21146 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:05.606091 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:05.558419 21447 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:05.608636 21447 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:05.558544 21270 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:24:05.613236 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:05.614991 16441 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:05.606262 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:05.596333 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:05.617157 16398 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
I20260709 12:24:05.605299 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:05.619334 16400 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:24:05.620658 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:05.621482 17270 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:05.605252 21346 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:05.587129 21479 raft_consensus.cc:2804] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:05.625375 16399 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
I20260709 12:24:05.589593 21471 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:05.636817 16441 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:05.639719 16549 raft_consensus.cc:2393] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 16: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:05.596807 21400 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:05.642854 16441 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:05.644754 17254 raft_consensus.cc:1275] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:24:05.646914 16398 raft_consensus.cc:1275] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:24:05.647117 16367 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:05.682242 16367 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:05.685639 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:05.645217 21276 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:05.687851 21476 consensus_queue.cc:1048] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:24:05.687387 16367 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:05.698014 16367 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:05.635722 21248 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:05.586760 21473 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:05.700011 21456 log.cc:927] Time spent T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a: Append to log took a long time: real 0.599s	user 0.005s	sys 0.003s
I20260709 12:24:05.694073 21475 consensus_queue.cc:1048] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:05.680925 21472 raft_consensus.cc:2804] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:05.698709 21476 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:05.701366 21473 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:05.692016 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:05.703653 21376 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:05.643857 21471 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:05.638203 21383 raft_consensus.cc:2804] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:05.597122 21281 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:05.618330 21095 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:05.763440 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 206030us
I20260709 12:24:05.736192 21473 consensus_queue.cc:1048] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:05.739761 21250 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:05.641566 21479 raft_consensus.cc:697] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:05.680897 21369 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.748807 17270 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
I20260709 12:24:05.750478 21420 raft_consensus.cc:3055] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:05.754145 21436 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:05.756146 21435 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:05.685869 21470 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:05.739933 21276 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:05.731375 21472 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:05.788651 16470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1524) took 1042 ms. Trace:
I20260709 12:24:05.869364 16470 rpcz_store.cc:276] 0709 12:24:04.745867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.746018 (+   151us) service_pool.cc:224] Handling call
0709 12:24:04.975809 (+229791us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:04.976016 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:24:04.981001 (+  4985us) log.cc:844] Serialized 14 byte log entry
0709 12:24:05.478476 (+497475us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:05.478541 (+    65us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:05.478617 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.478704 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:05.788366 (+309662us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b5400a68bd0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:05.788557 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3594,"prepare.queue_time_us":667,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":26368,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":8784,"spinlock_wait_cycles":537566464,"thread_start_us":493299,"threads_started":2,"wal-append.queue_time_us":357525}
I20260709 12:24:05.689925 21415 consensus_queue.cc:1048] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 12:24:05.839778 16402 raft_consensus.cc:2410] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:24:05.758703 21476 raft_consensus.cc:697] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:05.840778 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:05.768754 21401 raft_consensus.cc:993] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8: : Instructing follower 52ab50b0237346c1a13ede947f6a63c2 to start an election
W20260709 12:24:05.826467 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 141979us
I20260709 12:24:05.846908 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:05.847088 21479 consensus_queue.cc:237] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:05.847927 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:05.882711 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:05.857555 17144 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
W20260709 12:24:05.840421 21352 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.002s	sys 0.008s
I20260709 12:24:05.872303 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:05.888407 16470 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
I20260709 12:24:05.840154 21345 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:05.885962 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:05.883430 16441 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:05.881882 16473 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
I20260709 12:24:05.843873 21346 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:05.895002 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:05.848616 21399 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:05.844594 21383 raft_consensus.cc:697] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:05.897897 17144 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:05.899317 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:05.903129 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:05.903363 16473 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:05.904166 16474 raft_consensus.cc:2368] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 14. Current term is 15.
W20260709 12:24:05.889580 21352 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.002s	sys 0.010s
I20260709 12:24:05.854120 21420 raft_consensus.cc:740] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:05.857708 21470 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:05.881431 21401 raft_consensus.cc:1081] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Signalling peer 52ab50b0237346c1a13ede947f6a63c2 to start an election
I20260709 12:24:05.852423 21369 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:05.909771 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 211834us
I20260709 12:24:05.885658 21472 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:05.913993 21383 consensus_queue.cc:237] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:05.916687 16367 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:05.915145 21470 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:05.922550 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:05.923043 21447 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:05.898766 21276 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:05.923921 21420 consensus_queue.cc:260] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:05.942041 21276 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:05.898588 21410 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:05.942526 21276 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:05.942545 21420 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:05.943526 21463 log.cc:927] Time spent T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.542s	user 0.001s	sys 0.001s
I20260709 12:24:05.910358 21399 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:05.946606 21399 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:05.947688 16549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37242 (request call id 1464) took 1012 ms. Trace:
I20260709 12:24:05.926646 21369 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:05.930305 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:05.949502 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:05.950534 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:05.950619 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:05.948212 16549 rpcz_store.cc:276] 0709 12:24:04.934868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.966119 (+ 31251us) service_pool.cc:224] Handling call
0709 12:24:05.947443 (+981324us) spinlock_profiling.cc:243] Waited 657 ms on lock 0x7b540082e6d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:05.947614 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1510965120}
I20260709 12:24:05.951555 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:05.952579 21447 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:05.952880 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:05.953966 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:05.954216 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:05.958668 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:05.959314 17266 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:05.959743 21146 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:05.937932 16476 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 16: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:05.929545 21095 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:05.937618 16477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1675) took 1074 ms. Trace:
I20260709 12:24:05.925009 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:05.936081 17144 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:05.885520 21406 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:05.909214 21169 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:05.967329 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:05.915660 21250 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.983731 16473 raft_consensus.cc:2468] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:24:05.973070 17266 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 16.
I20260709 12:24:05.980108 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:05.934412 21481 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.881219 21476 consensus_queue.cc:237] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:05.963382 21271 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:05.969311 16477 rpcz_store.cc:276] 0709 12:24:04.862739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.866314 (+  3575us) service_pool.cc:224] Handling call
0709 12:24:04.866631 (+   317us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:04.866987 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:04.867016 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:04.867603 (+   587us) log.cc:844] Serialized 52 byte log entry
0709 12:24:05.164949 (+297346us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:05.192581 (+ 27632us) log.cc:844] Serialized 14 byte log entry
0709 12:24:05.481002 (+288421us) log.cc:844] Serialized 14 byte log entry
0709 12:24:05.482110 (+  1108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.482221 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:05.854007 (+371786us) spinlock_profiling.cc:243] Waited 2.72 ms on lock 0x7b1800273040. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:24:05.878194 (+ 24187us) spinlock_profiling.cc:243] Waited 6.7 ms on lock 0x7b1800273040. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:24:05.937394 (+ 59200us) raft_consensus.cc:1685] finished
0709 12:24:05.937416 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:05.937557 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2577,"prepare.queue_time_us":28522,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":66,"raft.queue_time_us":67144,"raft.run_cpu_time_us":539,"raft.run_wall_time_us":66034,"spinlock_wait_cycles":21673472,"thread_start_us":582568,"threads_started":2,"wal-append.queue_time_us":432674}
W20260709 12:24:05.995404 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:05.987825 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 51868.6us
I20260709 12:24:05.996371 16515 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:05.994036 16549 raft_consensus.cc:1240] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 17 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:05.996732 17144 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:05.984943 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 68370.4us
I20260709 12:24:06.006713 17144 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:06.009486 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:06.010309 16544 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:06.011615 16550 raft_consensus.cc:1240] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 12:24:05.968181 21281 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:06.019281 16515 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:06.019943 21146 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:06.022848 21482 log.cc:927] Time spent T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:24:05.965818 21251 consensus_queue.cc:1048] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:24:06.024255 21192 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:06.024366 21494 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:06.025027 21494 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:24:06.025497 21480 log.cc:927] Time spent T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 12:24:06.026265 21271 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:06.025453 21494 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:06.028486 16442 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:06.028545 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:06.030350 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:06.036103 17266 raft_consensus.cc:2376] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14. Re-sending same reply.
I20260709 12:24:06.031188 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:05.993505 21410 raft_consensus.cc:697] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:05.995497 21369 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:06.009085 21095 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.042608 16442 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:06.040588 16477 raft_consensus.cc:1240] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:06.050462 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:06.045809 16549 raft_consensus.cc:1275] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:24:06.046162 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:06.051630 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:06.046607 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:06.047230 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:06.052294 17264 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:24:06.040082 17255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 1281) took 1070 ms. Trace:
I20260709 12:24:06.053987 16477 raft_consensus.cc:1240] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:06.055086 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:06.055431 16394 raft_consensus.cc:1275] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:24:06.034403 21271 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:05.993489 21250 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:06.051227 16547 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:24:06.060941 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:06.064365 16398 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:06.036150 21401 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:06.056563 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:06.043272 21281 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.069824 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:06.074453 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:06.064942 16441 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:06.076566 16402 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:05.999289 21481 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:06.052691 21415 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:06.084164 21400 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:06.042996 21251 consensus_queue.cc:1048] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:24:06.091908 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:06.041715 21352 consensus_queue.cc:1059] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:24:06.095299 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:06.061206 21329 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:06.065024 21271 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:06.115990 16548 raft_consensus.cc:1275] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 12:24:06.116654 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 97483.6us
I20260709 12:24:06.053894 17255 rpcz_store.cc:276] 0709 12:24:04.963176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.963293 (+   117us) service_pool.cc:224] Handling call
0709 12:24:04.963580 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:04.963900 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:04.963926 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:04.971489 (+  7563us) log.cc:844] Serialized 26 byte log entry
0709 12:24:05.213072 (+241583us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:05.629751 (+416679us) log.cc:844] Serialized 14 byte log entry
0709 12:24:06.033615 (+403864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.033692 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:06.033735 (+    43us) raft_consensus.cc:1685] finished
0709 12:24:06.033748 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:06.033880 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56322,"prepare.queue_time_us":416665,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":135908,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":12021,"thread_start_us":964224,"threads_started":3,"wal-append.queue_time_us":399126}
I20260709 12:24:06.102936 16543 raft_consensus.cc:1275] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:24:06.100472 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:06.126487 16544 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
W20260709 12:24:06.100904 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 88181.1us
I20260709 12:24:06.072135 21355 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:06.060758 21095 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:06.109280 16403 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:06.109742 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:06.070969 21369 consensus_queue.cc:1048] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 12:24:06.084312 21489 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:06.128540 17144 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:06.088721 21248 consensus_queue.cc:1048] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 12:24:06.150197 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:06.151814 16394 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
W20260709 12:24:06.152323 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:06.091112 21479 consensus_queue.cc:1059] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 12:24:06.152839 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:06.153528 16394 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:06.085490 21281 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:06.106279 21420 consensus_queue.cc:1048] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:24:06.092445 21496 consensus_queue.cc:1059] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.084s
W20260709 12:24:06.176427 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 56843.1us
I20260709 12:24:06.177870 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:06.188393 16515 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:06.176050 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 47565.4us
I20260709 12:24:06.111824 21478 consensus_queue.cc:1048] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.097s
I20260709 12:24:06.190737 17144 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:06.088884 21410 consensus_queue.cc:237] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:06.192003 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 153356us
I20260709 12:24:06.205385 16475 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:24:06.205726 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:06.161003 21169 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:06.176905 21473 raft_consensus.cc:3055] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:06.208101 17143 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:06.204378 16515 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:06.236580 16515 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:06.153586 21489 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:06.203222 21270 consensus_meta.cc:287] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.566s	user 0.012s	sys 0.000s
I20260709 12:24:06.221324 16403 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
I20260709 12:24:06.195777 21146 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:06.258347 21497 consensus_queue.cc:1048] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 12:24:06.211457 21406 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:06.210611 21192 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:06.213750 21473 raft_consensus.cc:740] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:06.207770 21496 consensus_queue.cc:1243] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer 610f4feda11942739236f2b121a4b9c1 log is divergent from this leader: its last log entry 17.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:24:06.249437 21447 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:06.259604 16394 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:24:06.252013 16515 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:06.249828 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:06.276551 21369 raft_consensus.cc:2764] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
W20260709 12:24:06.310525 21500 log.cc:927] Time spent T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 12:24:06.310885 21426 raft_consensus.cc:3055] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:06.340552 21426 raft_consensus.cc:740] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:06.310678 21470 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:06.341570 21426 consensus_queue.cc:260] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:06.286119 21487 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:06.308152 16402 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 16.
I20260709 12:24:06.343058 21426 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:06.218779 21169 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:06.320806 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:06.325119 21507 raft_consensus.cc:2804] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:06.326830 21508 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:06.277817 21270 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:06.286279 21491 consensus_queue.cc:1048] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 12:24:06.336613 16470 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:06.331645 16398 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
W20260709 12:24:06.336299 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 84386.9us
I20260709 12:24:06.299140 21492 consensus_queue.cc:1048] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 12:24:06.320279 21510 raft_consensus.cc:3055] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:24:06.332634 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 124613us
I20260709 12:24:06.305677 21442 consensus_queue.cc:1059] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 12:24:06.302537 21447 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:06.348722 16442 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:06.325409 21250 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:06.347074 21487 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:06.356670 21505 log.cc:927] Time spent T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 12:24:06.346608 21497 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:06.385456 16542 raft_consensus.cc:1240] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:06.391531 17142 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:06.385228 21508 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:06.368628 21481 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:06.393584 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 202942us
I20260709 12:24:06.400098 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:06.442252 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:06.443435 16545 raft_consensus.cc:2410] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:24:06.409571 16544 raft_consensus.cc:1275] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:24:06.444924 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:06.443066 17266 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:24:06.411248 16547 raft_consensus.cc:1240] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:06.417569 17142 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:06.448910 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:06.441382 17144 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:06.451619 17142 leader_election.cc:400] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 14. Current term is 15.
I20260709 12:24:06.451939 17142 leader_election.cc:403] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:24:06.328431 21515 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:06.435456 16476 raft_consensus.cc:1240] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:06.437311 16549 raft_consensus.cc:1240] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:06.456478 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:06.412720 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 64123.9us
I20260709 12:24:06.452826 17142 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:06.454192 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:06.455615 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:06.420348 16400 raft_consensus.cc:3055] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:06.461447 16400 raft_consensus.cc:740] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:06.462184 16367 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:06.421721 16470 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
I20260709 12:24:06.437234 16441 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:06.468930 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:06.469547 16550 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:06.385398 21507 raft_consensus.cc:697] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:06.468302 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:24:06.463980 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 361411us
I20260709 12:24:06.462174 16400 consensus_queue.cc:260] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:06.513186 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:06.473675 16367 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:06.516322 16400 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:06.412855 21245 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:06.412935 21487 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:06.406175 21471 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:06.519345 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 82194.2us
I20260709 12:24:06.417737 21369 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:06.412261 21447 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:06.400900 21270 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:06.524029 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:06.450089 21470 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:06.464576 21515 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.415038 21497 raft_consensus.cc:697] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:06.402031 21436 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:24:06.537097 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 69781.8us
I20260709 12:24:06.521899 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:06.413255 21281 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:06.397744 21510 raft_consensus.cc:740] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:06.464730 21027 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:06.514870 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:24:06.540304 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 68012.5us
I20260709 12:24:06.418959 21276 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:06.584928 16367 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:06.396184 21095 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:06.417574 21508 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:06.415494 21489 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:06.469545 21346 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:06.558775 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:06.559949 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:06.562007 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:06.590101 16396 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:24:06.564949 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:06.568813 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:06.438663 21472 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:06.556831 21471 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:06.553920 21245 raft_consensus.cc:697] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 16 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:06.557876 16516 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:06.398097 21146 consensus_queue.cc:1048] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.127s
W20260709 12:24:06.580559 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 58779.6us
I20260709 12:24:06.604444 16551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1468) took 1647 ms. Trace:
I20260709 12:24:06.580766 16550 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
I20260709 12:24:06.407493 21498 consensus_queue.cc:1059] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.171s
I20260709 12:24:06.582903 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:06.552915 21475 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:24:06.614545 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 56862.2us
I20260709 12:24:06.582105 21369 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:06.570545 16400 raft_consensus.cc:1275] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 12:24:06.579429 21406 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 14. Current term is 15.
I20260709 12:24:06.593163 21479 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:06.575253 21515 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:06.607556 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 141714us
I20260709 12:24:06.579296 21251 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:06.609716 17254 raft_consensus.cc:1240] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:24:06.610937 16470 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:06.588084 16367 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:06.586084 21470 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:06.539606 21501 consensus_queue.cc:1048] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
I20260709 12:24:06.615324 16516 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:06.584090 21514 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:06.605144 16551 rpcz_store.cc:276] 0709 12:24:04.957124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.998424 (+ 41300us) service_pool.cc:224] Handling call
0709 12:24:05.066367 (+ 67943us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.387548 (+1321181us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b54009040a0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e04858 00007f2875e88e9d 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c
0709 12:24:06.387638 (+    90us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:24:06.519255 (+131617us) log.cc:844] Serialized 14 byte log entry
0709 12:24:06.603973 (+ 84718us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:06.604045 (+    72us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:06.604114 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.604191 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:06.604380 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":882,"raft.queue_time_us":74,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":8351,"spinlock_wait_cycles":480963328,"thread_start_us":195115,"threads_started":2}
I20260709 12:24:06.581461 21520 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:06.625550 16441 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:06.594350 21027 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:06.631778 21498 raft_consensus.cc:3055] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:06.619407 21508 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:06.632912 16468 raft_consensus.cc:2368] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 12. Current term is 13.
I20260709 12:24:06.583943 21522 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:06.516023 21473 consensus_queue.cc:260] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:06.639410 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:06.605053 21426 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:06.582044 21436 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:06.640630 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 52616us
I20260709 12:24:06.641994 16516 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:06.689370 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 47453.5us
I20260709 12:24:06.689270 16367 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:06.603677 21471 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.619905 21507 consensus_queue.cc:237] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:06.645906 21516 log.cc:927] Time spent T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.258s	user 0.001s	sys 0.000s
W20260709 12:24:06.647634 21517 log.cc:927] Time spent T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:24:06.617581 21527 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:24:06.634207 21509 log.cc:927] Time spent T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.260s	user 0.001s	sys 0.000s
I20260709 12:24:06.654134 16473 raft_consensus.cc:1275] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:24:06.654474 16441 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:06.707089 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 52745.3us
I20260709 12:24:06.708243 16441 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:06.619184 16400 pending_rounds.cc:85] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 10
I20260709 12:24:06.709662 16400 pending_rounds.cc:107] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 12:24:06.710055 16400 raft_consensus.cc:2889] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:06.652640 21478 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:06.636200 21475 raft_consensus.cc:697] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:06.613039 21481 consensus_queue.cc:1048] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:24:06.678396 16470 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
I20260709 12:24:06.635725 21376 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:06.614886 21248 consensus_queue.cc:1048] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:24:06.653277 21192 raft_consensus.cc:2804] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:06.695077 16367 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:06.713099 21412 consensus_meta.cc:287] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 1.088s	user 0.016s	sys 0.001s
I20260709 12:24:06.655812 21476 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:06.653147 21491 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:06.670063 21498 raft_consensus.cc:740] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:06.638504 21251 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:06.679920 21508 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.633292 21523 consensus_queue.cc:1059] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.089s
I20260709 12:24:06.699111 21426 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:06.691375 21471 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:06.713984 21399 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:24:06.704005 21420 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:06.664947 21514 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:06.758570 21528 log.cc:927] Time spent T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 12:24:06.758241 21498 consensus_queue.cc:260] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:06.759591 21498 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:06.760061 21514 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:06.732018 21412 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:06.649031 21510 consensus_queue.cc:260] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:06.762247 21412 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:06.670125 21497 consensus_queue.cc:237] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:06.764140 21510 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:06.766386 21475 consensus_queue.cc:237] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:06.717566 21245 consensus_queue.cc:237] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:06.770234 21399 raft_consensus.cc:2749] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:06.728238 21376 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:06.733891 21248 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:06.732636 16367 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:06.734637 21027 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:06.744742 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 53874.7us
I20260709 12:24:06.722271 21522 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:06.723973 21478 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:06.725888 16470 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:53186
I20260709 12:24:06.774667 16470 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:06.775025 16470 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:24:06.752995 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 279398us
W20260709 12:24:06.698149 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 245351us
I20260709 12:24:06.670363 21520 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:06.730171 21501 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:06.715041 21499 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:06.778457 17142 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:06.777222 16516 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:06.782614 16547 raft_consensus.cc:1275] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:24:06.735361 21192 raft_consensus.cc:697] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:24:06.788703 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 56140.9us
I20260709 12:24:06.797646 16367 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:06.798081 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 89990.3us
I20260709 12:24:06.753518 21508 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:06.799291 17142 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:06.754280 21476 raft_consensus.cc:697] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 14 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:06.774129 21376 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:06.792829 16470 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:06.807653 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:24:06.794781 21271 consensus_meta.cc:287] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.653s	user 0.012s	sys 0.000s
W20260709 12:24:06.804847 21529 log.cc:927] Time spent T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 12:24:06.737754 21491 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:06.804991 21496 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:06.831961 17142 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:06.834573 16470 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:06.806962 21412 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:06.819907 21276 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:06.818871 21251 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:06.818928 21250 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:06.852645 21470 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:06.853986 16397 raft_consensus.cc:1275] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:24:06.828869 21472 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:06.855422 21496 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.824123 21271 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:06.811591 21499 raft_consensus.cc:697] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:06.774040 21522 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:06.866533 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:06.854681 21487 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:06.830475 21527 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.831367 21502 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:06.832566 21523 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.834698 21510 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:06.875444 17142 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:06.865093 21250 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.874439 21496 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:06.874609 21401 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:06.924459 21412 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:06.929026 16551 raft_consensus.cc:1240] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:24:06.901255 21523 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:06.967300 21523 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:06.895263 21527 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:06.894445 21250 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:06.970178 21527 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:06.974643 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:06.975399 16548 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:24:06.936430 21476 consensus_queue.cc:237] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:06.981709 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 106336us
I20260709 12:24:07.029897 17142 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:06.915927 21436 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:06.874471 21192 consensus_queue.cc:237] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:07.032733 21544 log.cc:927] Time spent T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 12:24:07.034940 21271 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:06.892028 21472 consensus_queue.cc:1048] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.091s
I20260709 12:24:07.042080 21522 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:07.057065 21520 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:07.057451 21426 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:07.048686 16367 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:07.063903 17142 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:07.068701 21447 consensus_meta.cc:287] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.513s	user 0.008s	sys 0.004s
W20260709 12:24:07.031009 21494 consensus_meta.cc:287] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.982s	user 0.012s	sys 0.002s
I20260709 12:24:07.080798 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:07.027386 21470 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:07.057157 21478 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:07.077955 21523 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:07.082003 21522 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.086344 21527 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:07.099290 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:07.100255 17271 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:07.100577 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:07.101063 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:07.102990 17255 raft_consensus.cc:1240] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:24:07.101263 17266 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:24:07.105552 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:07.106920 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:07.107285 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:07.107899 17264 raft_consensus.cc:2410] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 12:24:07.108199 17269 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
I20260709 12:24:07.109364 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:07.110592 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:07.111626 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:07.113221 17258 raft_consensus.cc:2393] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 15: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:07.113865 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:07.089298 21494 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:07.089901 21470 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:07.088240 21447 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:07.091737 21533 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:07.078544 21499 consensus_queue.cc:237] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:07.116295 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:07.135224 17254 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:07.135948 21447 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:07.136036 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:07.136904 16470 raft_consensus.cc:2393] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 12: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:07.138418 21027 consensus_queue.cc:1048] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:07.146389 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:07.146993 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:07.147075 16544 raft_consensus.cc:1240] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:07.121534 16546 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:24:07.123503 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:07.125566 16468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53186 (request call id 973) took 1252 ms. Trace:
I20260709 12:24:07.126921 16400 raft_consensus.cc:1240] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:07.130743 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:07.120584 17255 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78"
dest_uuid: "ef57beab0e1d40d585983873c55107e8"
mode: GRACEFUL
new_leader_uuid: "610f4feda11942739236f2b121a4b9c1"
 from {username='slave'} at 127.0.0.1:45120
I20260709 12:24:07.120859 16542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 1036) took 1136 ms. Trace:
I20260709 12:24:07.130308 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:07.151770 17255 raft_consensus.cc:606] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Received request to transfer leadership to TS 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:07.152453 17255 raft_consensus.cc:614] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:07.149063 16468 rpcz_store.cc:276] 0709 12:24:05.873096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.950420 (+ 77324us) service_pool.cc:224] Handling call
0709 12:24:07.125390 (+1174970us) spinlock_profiling.cc:243] Waited 672 ms on lock 0x7b540080b1d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:07.125513 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1544844032}
I20260709 12:24:07.151726 16542 rpcz_store.cc:276] 0709 12:24:05.984620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.984743 (+   123us) service_pool.cc:224] Handling call
0709 12:24:05.985046 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.415717 (+430671us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:07.120558 (+704841us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b54008c41d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:07.120779 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":865188096}
I20260709 12:24:07.111455 21508 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:07.164716  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:24:07.178036 17254 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
I20260709 12:24:07.116679 21494 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:07.118994 21470 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:07.122126 21491 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:07.190769 21481 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:07.208827 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.185632 21442 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.208851 21481 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:07.209940 21442 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:07.211102 16471 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53186 (request call id 966) took 1358 ms (client timeout 1681 ms). Trace:
W20260709 12:24:07.211488 16471 rpcz_store.cc:269] 0709 12:24:05.852262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.894662 (+ 42400us) service_pool.cc:224] Handling call
0709 12:24:07.210903 (+1316241us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b540012b954. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:07.211041 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2738850176}
I20260709 12:24:07.211836 21481 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:07.213196 21545 log.cc:927] Time spent T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:24:07.214205 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:07.214597 21553 log.cc:927] Time spent T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:24:07.215858 21169 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:07.216753 16543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1621) took 1187 ms. Trace:
I20260709 12:24:07.217170 16543 rpcz_store.cc:276] 0709 12:24:06.029702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.032821 (+  3119us) service_pool.cc:224] Handling call
0709 12:24:06.033159 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.124787 (+ 91628us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:07.216486 (+1091699us) spinlock_profiling.cc:243] Waited 69.3 ms on lock 0x7b540037ee54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:07.216675 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":159396224}
I20260709 12:24:07.200672 21496 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:07.218945 16398 raft_consensus.cc:2368] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 11. Current term is 12.
I20260709 12:24:07.219779 21496 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:07.220793 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.221038 16516 leader_election.cc:400] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 11. Current term is 12.
I20260709 12:24:07.221465 16516 leader_election.cc:403] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:24:07.161202 21502 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:07.223299 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:07.226177 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:07.224038 17142 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:07.224164 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:07.227916 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:07.224982 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:07.224169 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:07.228994 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:07.179847 21420 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:07.224777 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:07.101768 21522 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:07.234576 17142 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:07.212064 21442 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:07.240412 17142 leader_election.cc:400] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 12. Current term is 13.
I20260709 12:24:07.241087 17142 leader_election.cc:403] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:24:07.199318 17271 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 17.
I20260709 12:24:07.161358 21489 consensus_queue.cc:1059] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 12:24:07.180266 21447 consensus_queue.cc:1059] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 12:24:07.134240 21515 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:07.236826 21420 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:07.233511 21491 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:07.248754 21478 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.261273 16471 raft_consensus.cc:1240] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:07.236937 21401 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:24:07.247995 21523 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 12. Current term is 13.
I20260709 12:24:07.248971 21533 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:07.268824 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:07.265926 21478 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:07.266228 21436 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.283252 16400 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:07.263960 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:07.290834 16366 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:07.305011 16476 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:24:07.262537 21489 consensus_queue.cc:1243] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer 610f4feda11942739236f2b121a4b9c1 log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:24:07.233217 21502 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:07.251359 21494 consensus_queue.cc:1048] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 12:24:07.376322 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:07.376367 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 85639.7us
I20260709 12:24:07.377343 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:07.390780 21476 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:07.382881 16400 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:24:07.352635 21236 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:07.392812 21236 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:07.395780 21236 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:07.398399 21250 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:07.405260 21548 log.cc:927] Time spent T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 12:24:07.271937 21508 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:07.378333 21436 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:07.423593 21533 raft_consensus.cc:697] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:07.457587 21250 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:07.461658 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 72705.1us
I20260709 12:24:07.464566 21524 raft_consensus.cc:3055] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:07.465145 21524 raft_consensus.cc:740] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:07.466287 21524 consensus_queue.cc:260] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:07.470136 21539 consensus_queue.cc:1059] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:07.470487 21192 consensus_queue.cc:1048] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:07.433461 21383 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:24:07.441726 21471 consensus_meta.cc:287] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.683s	user 0.011s	sys 0.001s
I20260709 12:24:07.468137 21524 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:07.478937 16514 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:07.440348 21481 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:07.488315 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:07.489159 16471 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:07.495203 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:07.479285 21236 raft_consensus.cc:3055] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:07.501056 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:07.509050 16472 raft_consensus.cc:1275] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:24:07.508917 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:07.510740 16550 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14.
W20260709 12:24:07.517124 16475 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34840 (request call id 1523) took 2759 ms (client timeout 2616 ms). Trace:
I20260709 12:24:07.516090 16469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1683) took 1351 ms. Trace:
I20260709 12:24:07.522693 16469 rpcz_store.cc:276] 0709 12:24:06.165240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.168245 (+  3005us) service_pool.cc:224] Handling call
0709 12:24:06.193125 (+ 24880us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.384110 (+190985us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:06.384145 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:06.384193 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:06.384231 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:07.515749 (+1131518us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b54000a32d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:07.515817 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:07.516003 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":383881088}
W20260709 12:24:07.525678 21352 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260709 12:24:07.526047 21522 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:07.527626 21515 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:07.530350 21502 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:07.531986 21420 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:07.513906 21533 consensus_queue.cc:237] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:07.517733 16475 rpcz_store.cc:269] 0709 12:24:04.743182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.743699 (+   517us) service_pool.cc:224] Handling call
0709 12:24:07.502376 (+2758677us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b5400a686d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:07.502485 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3345779456}
I20260709 12:24:07.487939 21491 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 11. Current term is 12.
I20260709 12:24:07.477133 21027 consensus_queue.cc:1059] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:24:07.547721 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:07.550554 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:07.502904 21236 raft_consensus.cc:740] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:07.554788 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:07.556870 16544 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:24:07.552465 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:07.485405 21471 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:07.559595 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:07.510308 21520 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.563957 16469 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:07.484800 21383 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:07.569834 16442 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:07.571846 16475 raft_consensus.cc:3055] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:07.572243 16475 raft_consensus.cc:740] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:07.544540 21515 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:07.581400 21271 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:07.556259 21385 consensus_queue.cc:1059] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:24:07.582161 16475 consensus_queue.cc:260] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:07.620118 16475 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:07.587656 21489 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:24:07.591853 21563 log.cc:927] Time spent T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.057s	user 0.003s	sys 0.000s
I20260709 12:24:07.623018 21271 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:07.595562 21487 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:07.604875 21558 log.cc:927] Time spent T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 12:24:07.574469 21520 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:07.615787 21523 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.555603 21401 consensus_queue.cc:1059] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:24:07.605571 21435 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:07.613659 16546 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:24:07.608884 16367 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:07.617995 16394 raft_consensus.cc:2393] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 14: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:07.498589 21481 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:07.641238 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:07.554252 21522 consensus_queue.cc:1048] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 12:24:07.641987 16397 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
W20260709 12:24:07.631834 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 242403us
I20260709 12:24:07.648583 16442 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:07.627089 21436 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:07.657600 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:07.658502 16400 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:24:07.656123 21435 raft_consensus.cc:697] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:07.634260 21514 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.622192 21515 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:07.647034 21236 consensus_queue.cc:260] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:07.653530 21420 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:07.690035 16468 raft_consensus.cc:1275] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:24:07.630218 16471 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 17.
I20260709 12:24:07.667156 21514 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:07.655814 21385 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:07.701733 16514 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:07.627866 21489 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:07.701552 16442 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:07.653450 21502 consensus_queue.cc:1059] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:24:07.645478 21523 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:07.710290 16475 raft_consensus.cc:1275] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 12:24:07.748860 16475 pending_rounds.cc:85] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 8
I20260709 12:24:07.697917 21236 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:07.749230 16475 pending_rounds.cc:107] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 12:24:07.765224 16475 raft_consensus.cc:2889] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:07.721158 16400 raft_consensus.cc:1275] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 12:24:07.771193 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 59707.9us
I20260709 12:24:07.699220 21420 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:07.700119 21383 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:07.773187 16366 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:07.704502 21346 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:07.711879 21385 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:07.775084 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 73613.1us
I20260709 12:24:07.781981 21520 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:07.783010 16475 raft_consensus.cc:1275] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:24:07.718096 21487 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:07.784446 21487 raft_consensus.cc:697] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 17 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:07.784164 16399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 1438) took 1697 ms. Trace:
I20260709 12:24:07.784987 16399 rpcz_store.cc:276] 0709 12:24:06.086551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.087385 (+   834us) service_pool.cc:224] Handling call
0709 12:24:06.087724 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:06.088157 (+   433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:06.088198 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:07.362016 (+1273818us) spinlock_profiling.cc:243] Waited 5.33 ms on lock 0x7b2c0004c948. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:24:07.391861 (+ 29845us) log.cc:844] Serialized 26 byte log entry
0709 12:24:07.630625 (+238764us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:07.630664 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:07.630730 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:07.783875 (+153145us) raft_consensus.cc:1685] finished
0709 12:24:07.783894 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:07.784107 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":714,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":121,"spinlock_wait_cycles":12247808,"thread_start_us":235661,"threads_started":1,"wal-append.queue_time_us":380519}
I20260709 12:24:07.785401 21487 consensus_queue.cc:237] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:07.785362 21401 raft_consensus.cc:3055] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:07.790727 21401 raft_consensus.cc:740] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:07.760692 21447 raft_consensus.cc:2749] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:07.804451 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:07.805773 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:07.806423 16471 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:24:07.777148 21420 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:07.792855 21329 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.778864 16469 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:24:07.792481 21385 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:07.746788 21027 consensus_queue.cc:1048] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 12:24:07.784168 21472 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:07.815243 16397 raft_consensus.cc:1275] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:24:07.820178 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:07.824084 16399 raft_consensus.cc:1275] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:24:07.853107 17271 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:07.824187 16403 raft_consensus.cc:1275] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 12:24:07.853865 16403 pending_rounds.cc:85] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 9
I20260709 12:24:07.854240 16403 pending_rounds.cc:107] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 12:24:07.854571 16403 raft_consensus.cc:2889] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:07.830049 21329 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:07.836500 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:07.831686 16475 raft_consensus.cc:1240] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:07.841415 16477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1686) took 1675 ms. Trace:
I20260709 12:24:07.845605 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:07.847043 17270 raft_consensus.cc:1275] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 12:24:07.857316 17269 raft_consensus.cc:2393] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 13: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:07.835675 21442 raft_consensus.cc:3055] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:07.857262 17270 pending_rounds.cc:85] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8: Aborting all ops after (but not including) 8
I20260709 12:24:07.858268 17270 pending_rounds.cc:107] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:24:07.858639 17270 raft_consensus.cc:2889] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:07.856737 16477 rpcz_store.cc:276] 0709 12:24:06.166354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.167176 (+   822us) service_pool.cc:224] Handling call
0709 12:24:06.188695 (+ 21519us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:07.262955 (+1074260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:07.262993 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:07.263035 (+    42us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:07.263069 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:07.841093 (+578024us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54002f2954. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:07.841162 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:07.841334 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2470067456}
W20260709 12:24:07.860817 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.826745 16366 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:07.849820 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:07.862246 17255 raft_consensus.cc:3055] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:07.862717 17255 raft_consensus.cc:740] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:07.853684 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:07.865123 16400 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:24:07.864099 17271 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 17.
I20260709 12:24:07.871470 17263 raft_consensus.cc:2393] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 17: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
W20260709 12:24:07.872145 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 45529.4us
W20260709 12:24:07.873569 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 88678.4us
I20260709 12:24:07.863682 17255 consensus_queue.cc:260] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:07.876319 17255 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:07.792730 21346 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:07.791797 21401 consensus_queue.cc:260] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:07.880854 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:07.816296 21383 raft_consensus.cc:3055] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:07.875815 17142 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:07.878772 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:07.891239 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:07.892122 17269 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
I20260709 12:24:07.856388 21472 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:07.892069 16468 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:24:07.856999 21573 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:24:07.904458 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.906363 17270 raft_consensus.cc:1275] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:24:07.870225 21442 raft_consensus.cc:740] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:07.909379 16403 raft_consensus.cc:1275] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:24:07.913957 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:07.914781 16403 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 17.
W20260709 12:24:07.921712 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 46194.3us
I20260709 12:24:07.923142 17255 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:24:07.927304 16400 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 17.
W20260709 12:24:07.932216 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 47354.4us
I20260709 12:24:07.935508 17254 raft_consensus.cc:1275] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 12:24:07.936872 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.929494 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:07.893100 21271 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:07.889987 21383 raft_consensus.cc:740] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:07.940825 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:07.811203 21435 consensus_queue.cc:237] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:07.946091 17269 raft_consensus.cc:1275] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:24:07.939664 17263 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:24:07.882064 21524 consensus_queue.cc:1048] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:24:07.905561 21472 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:07.943377 16442 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:07.955140 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:07.897853 21539 consensus_queue.cc:1048] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 12:24:07.937022 21504 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.955756 17263 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:24:07.905865 21487 consensus_queue.cc:1048] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 12:24:07.905589 21572 consensus_queue.cc:1048] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 12:24:07.936204 21415 consensus_queue.cc:1048] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 12:24:07.957537 21401 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:07.951778 21580 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:07.962424 21520 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:07.963621 21271 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:07.950388 21475 consensus_queue.cc:1048] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 12:24:07.952833 21497 consensus_queue.cc:1048] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 12:24:07.955006 21442 consensus_queue.cc:260] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:08.000622 21271 raft_consensus.cc:697] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:08.000815 21527 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:08.001561 21527 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:08.001744 21271 consensus_queue.cc:237] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:08.005666 21489 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:08.005937 21525 log.cc:927] Time spent T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 1.388s	user 0.004s	sys 0.005s
I20260709 12:24:07.985388 21346 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:07.963415 21504 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:07.987509 21383 consensus_queue.cc:260] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:07.989068 21329 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:07.968572 21515 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:07.981057 21520 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.010641 21520 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:08.011698 21555 log.cc:927] Time spent T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.658s	user 0.003s	sys 0.000s
I20260709 12:24:08.012882 21504 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:08.013895 21442 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:07.992750 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:08.017128 17258 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 16.
I20260709 12:24:08.020587 21471 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:08.037731 21523 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:08.039523 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:08.040205 17263 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:24:08.041069 17261 raft_consensus.cc:1275] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:24:07.974061 21580 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:08.041905 21580 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:08.044977 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:08.048705 17266 raft_consensus.cc:2393] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 15: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:08.016114 21514 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:08.056730 17258 raft_consensus.cc:1240] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:24:08.069487 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:08.070586 17258 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 15.
I20260709 12:24:08.043010 21574 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:24:08.075440 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 117325us
I20260709 12:24:08.078166 21574 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:08.080575 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:08.091856 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:08.091935 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:08.093492 17264 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 15.
I20260709 12:24:08.101163 21574 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:08.103798 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:08.104352 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:08.104955 17263 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:24:08.105468 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:08.105635 17261 raft_consensus.cc:1240] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:08.106341 17254 raft_consensus.cc:2393] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 12: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:08.020623 21385 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:08.107896 16474 raft_consensus.cc:2393] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 13: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:08.110323 21385 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:08.104808 17266 raft_consensus.cc:2393] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 14: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:08.112146 21385 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:08.112903 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:08.113723 16543 raft_consensus.cc:2393] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 12: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
W20260709 12:24:08.115891 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:08.116412 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.116032 21527 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:08.116930 21539 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:08.117444 21539 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:08.118085 21527 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:08.119493 21539 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:08.112980 21385 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:08.122411 21385 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:08.104729 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:08.129840 17264 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:08.138648 16550 raft_consensus.cc:1240] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
W20260709 12:24:08.223109 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 81601.2us
W20260709 12:24:08.223492 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.223610 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:08.224704 17143 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:08.226132 17143 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:08.224973 16550 raft_consensus.cc:2393] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 14: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:08.228055 16441 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:08.234786 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:08.236850 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:08.237994 17255 raft_consensus.cc:2393] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 13: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
W20260709 12:24:08.238663 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 284592us
I20260709 12:24:08.240428 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:08.255028 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:08.258204 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:08.259815 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:08.261054 17263 raft_consensus.cc:1240] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:08.266211 17258 raft_consensus.cc:1240] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:08.269778 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:08.271330 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:08.272017 17263 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:24:08.276387 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:08.223114 21522 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:08.277298 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:08.233306 21515 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:08.239326 21572 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:08.281911 17269 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:08.295981 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:08.281651 17261 raft_consensus.cc:1275] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:24:08.304248 17271 raft_consensus.cc:1275] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:24:08.313472 17264 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
W20260709 12:24:08.315423 16515 connection.cc:256] RPC call timeout handler was delayed by -1.1538894150003216s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1525s for a response
I20260709 12:24:08.281428 21597 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:08.298889 21572 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:08.332055 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 89163.4us
I20260709 12:24:08.281608 21436 consensus_queue.cc:1048] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 12:24:08.297919 21522 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:08.362844 16441 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:08.326486 21584 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.372730 21584 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:08.371213 21551 log.cc:927] Time spent T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 1.334s	user 0.005s	sys 0.000s
I20260709 12:24:08.362046 21587 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.371711 17269 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:24:08.337407 21572 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:08.389362 16441 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:08.342129 21597 raft_consensus.cc:697] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:08.346458 21590 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:08.375303 21584 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:08.385502 21475 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:24:08.419381 21595 log.cc:927] Time spent T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:24:08.419425 21475 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:08.420948 21475 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:24:08.422163 21420 consensus_meta.cc:287] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.589s	user 0.010s	sys 0.000s
I20260709 12:24:08.422576 21420 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:08.384649 21415 consensus_queue.cc:1048] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
W20260709 12:24:08.424860 21596 log.cc:927] Time spent T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 12:24:08.426568 21598 log.cc:927] Time spent T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:24:08.395670 21385 consensus_queue.cc:1048] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:24:08.408494 21590 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:08.388036 21582 log.cc:927] Time spent T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.312s	user 0.003s	sys 0.007s
I20260709 12:24:08.417712 16547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1629) took 1595 ms. Trace:
I20260709 12:24:08.430792 16547 rpcz_store.cc:276] 0709 12:24:06.822286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.822435 (+   149us) service_pool.cc:224] Handling call
0709 12:24:06.822904 (+   469us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:06.831025 (+  8121us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:06.831079 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:06.833828 (+  2749us) log.cc:844] Serialized 26 byte log entry
0709 12:24:06.927034 (+ 93206us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:06.927136 (+   102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.927239 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:08.278152 (+1350913us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:24:08.417432 (+139280us) raft_consensus.cc:1685] finished
0709 12:24:08.417453 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:08.417649 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7992,"raft.queue_time_us":8360,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":6100,"spinlock_wait_cycles":32319872,"thread_start_us":43271,"threads_started":1,"wal-append.queue_time_us":173477}
I20260709 12:24:08.418587 21522 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:08.433817 21562 log.cc:927] Time spent T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.859s	user 0.000s	sys 0.002s
I20260709 12:24:08.436024 17263 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
I20260709 12:24:08.436533 21601 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:08.454950 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:08.456996 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:08.442293 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:08.460451 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:08.443486 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:08.462340 16397 raft_consensus.cc:2410] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:24:08.447538 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:08.471065 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:08.471781 16398 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
W20260709 12:24:08.471616 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 155295us
I20260709 12:24:08.482522 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:08.481164 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:08.486012 16397 raft_consensus.cc:1240] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:08.475250 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:08.505395 16551 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 17.
W20260709 12:24:08.475725 21501 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.408s	user 0.000s	sys 0.012s
I20260709 12:24:08.481956 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:08.473534 21401 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.508600 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:08.489565 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:08.398679 21435 consensus_queue.cc:1048] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.083s
W20260709 12:24:08.501758 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 91364.9us
I20260709 12:24:08.514432 16441 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:08.502187 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:08.517338 16550 raft_consensus.cc:2393] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 13: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:08.506304 16477 raft_consensus.cc:2393] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 20: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:08.460041 21601 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:08.505683 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:08.399454 21520 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:08.520592 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:08.521338 16398 raft_consensus.cc:2393] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 15: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:08.522444 17144 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:08.511921 21401 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:08.522723 21597 consensus_queue.cc:237] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:08.395725 21587 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:08.524117 17144 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:08.541788 17144 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:08.536013 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:08.543709 16469 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
W20260709 12:24:08.508600 21501 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.441s	user 0.000s	sys 0.014s
I20260709 12:24:08.522790 16397 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:24:08.523267 16395 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 20.
I20260709 12:24:08.545585 16543 pending_rounds.cc:85] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1: Aborting all ops after (but not including) 9
I20260709 12:24:08.546008 16543 pending_rounds.cc:107] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 12:24:08.546252 16543 raft_consensus.cc:2889] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:08.509490 21420 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:08.528847 21329 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:08.524298 21522 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:08.536722 21520 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:08.566892 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 55366.3us
W20260709 12:24:08.559198 21533 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.994s	user 0.008s	sys 0.022s
I20260709 12:24:08.542800 21601 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:08.555210 21385 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:08.551994 21581 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:08.581210 21385 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:08.581907 21581 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:08.582335 21581 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:08.583428 21385 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:08.571182 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:08.584555 16548 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 17.
I20260709 12:24:08.577476 21329 raft_consensus.cc:697] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:08.586181 21329 consensus_queue.cc:237] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:08.573732 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:08.588774 16397 raft_consensus.cc:1275] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 12:24:08.589288 21472 consensus_meta.cc:287] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.628s	user 0.008s	sys 0.004s
I20260709 12:24:08.589807 21472 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:08.592199 21498 raft_consensus.cc:3055] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:08.571498 21524 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.592648 21498 raft_consensus.cc:740] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:08.592911 21524 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:08.593565 21498 consensus_queue.cc:260] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:08.594556 21498 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:08.595053 21524 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:08.575224 16395 raft_consensus.cc:1240] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:08.565395 21523 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:24:08.559757 21580 consensus_meta.cc:287] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.517s	user 0.012s	sys 0.000s
W20260709 12:24:08.580699 21603 log.cc:927] Time spent T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:24:08.580278 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:08.569191 21590 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:08.581419 21499 raft_consensus.cc:3055] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:08.597424 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:08.536568 21401 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:08.603516 16550 raft_consensus.cc:1240] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:08.607100 21590 consensus_queue.cc:1048] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:24:08.622319 21481 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:08.623027 21481 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:08.623538 21481 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:08.624938 21560 log.cc:927] Time spent T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 1.120s	user 0.000s	sys 0.003s
I20260709 12:24:08.577298 21520 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:08.626188 16402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 1460) took 1478 ms. Trace:
I20260709 12:24:08.626601 21520 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:08.628017 16546 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37226 (request call id 1635) took 1400 ms (client timeout 1700 ms). Trace:
I20260709 12:24:08.626698 16402 rpcz_store.cc:276] 0709 12:24:07.147999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.148325 (+   326us) service_pool.cc:224] Handling call
0709 12:24:07.183076 (+ 34751us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:07.191109 (+  8033us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:07.191157 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:07.191769 (+   612us) log.cc:844] Serialized 26 byte log entry
0709 12:24:07.462832 (+271063us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:07.462873 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:07.476459 (+ 13586us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:08.588196 (+1111737us) spinlock_profiling.cc:243] Waited 84.6 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:24:08.625854 (+ 37658us) raft_consensus.cc:1685] finished
0709 12:24:08.625890 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:08.626125 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25109,"raft.queue_time_us":8877,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":11853,"spinlock_wait_cycles":194621184,"thread_start_us":269293,"threads_started":1,"wal-append.queue_time_us":313136}
I20260709 12:24:08.628803 21520 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:08.631135 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:08.632414 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:08.632838 16551 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:08.633085 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:08.633518 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:08.605010 21415 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:08.634545 21415 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:08.605526 21580 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:08.576726 21533 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.011s	user 0.009s	sys 0.022s
I20260709 12:24:08.633868 16515 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:08.573776 21587 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:08.628454 16546 rpcz_store.cc:269] 0709 12:24:07.227639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.228707 (+  1068us) service_pool.cc:224] Handling call
0709 12:24:08.627824 (+1399117us) spinlock_profiling.cc:243] Waited 329 ms on lock 0x7b54008c4bd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:08.627932 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":757228800}
I20260709 12:24:08.637578 16515 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:08.597821 21472 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:08.657728 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.658967 16551 raft_consensus.cc:1275] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:24:08.649287 16515 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:08.660785 21270 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:08.662992 16515 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:08.667171 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:08.606141 21499 raft_consensus.cc:740] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:08.573868 21572 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:08.667930 16515 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:08.670488 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.645107 21608 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:08.677366 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:08.664748 21236 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:08.700315 16515 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:08.724085 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:08.666996 21520 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.672852 21270 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:08.726264 16515 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:08.673431 21415 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:08.727468 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:08.701547 21236 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.739617 16471 raft_consensus.cc:1275] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:24:08.742801 16476 raft_consensus.cc:1240] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:08.742969 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:08.744035 16473 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34826 (request call id 1696) took 1476 ms (client timeout 1958 ms). Trace:
W20260709 12:24:08.744521 16473 rpcz_store.cc:269] 0709 12:24:07.267913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.292164 (+ 24251us) service_pool.cc:224] Handling call
0709 12:24:08.742828 (+1450664us) spinlock_profiling.cc:243] Waited 1.19 ms on lock 0x7b5800602a98. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64eee 000000000036eb39 00007f287add81e0 00007f287aea10c4 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:24:08.743965 (+  1137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2771968}
I20260709 12:24:08.723932 21146 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:08.728571 17255 raft_consensus.cc:2368] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 13. Current term is 14.
I20260709 12:24:08.732774 21584 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:08.731251 21608 raft_consensus.cc:2804] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:08.746032 21236 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:08.750540 21385 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.768437 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:08.767412 21572 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:08.731753 21520 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:08.784832 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 56637.3us
W20260709 12:24:08.803737 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 66384.7us
I20260709 12:24:08.788626 21426 raft_consensus.cc:3055] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:08.756861 21475 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:08.789291 21584 raft_consensus.cc:697] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 17 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:08.764521 21499 consensus_queue.cc:260] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:08.789110 21608 raft_consensus.cc:697] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 13 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:08.786094 21385 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:08.808305 21426 raft_consensus.cc:740] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:08.823076 21481 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:08.843449 21539 consensus_queue.cc:1048] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:24:08.845104 16394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46250 (request call id 1461) took 1563 ms. Trace:
I20260709 12:24:08.845621 16394 rpcz_store.cc:276] 0709 12:24:07.281395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.281576 (+   181us) service_pool.cc:224] Handling call
0709 12:24:08.844878 (+1563302us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b5400a79ed4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:08.845033 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":522280320}
I20260709 12:24:08.853402 21608 consensus_queue.cc:237] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:08.856539 21385 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:08.853855 17142 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:24:08.857986 21556 log.cc:927] Time spent T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 1.392s	user 0.004s	sys 0.001s
W20260709 12:24:08.869302 21613 log.cc:927] Time spent T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.141s	user 0.001s	sys 0.001s
I20260709 12:24:08.857223 21584 consensus_queue.cc:237] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:08.868968 21570 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:08.903203 21570 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.903672 21570 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:08.873728 21478 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:08.882154 21619 log.cc:927] Time spent T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 12:24:08.882828 21618 log.cc:927] Time spent T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 12:24:08.907733 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:08.884899 21543 log.cc:927] Time spent T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 1.828s	user 0.004s	sys 0.000s
W20260709 12:24:08.892899 21617 log.cc:927] Time spent T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 12:24:08.897915 21475 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:08.914273 16472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1670) took 4142 ms. Trace:
I20260709 12:24:08.914736 16472 rpcz_store.cc:276] 0709 12:24:04.771480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.774771 (+  3291us) service_pool.cc:224] Handling call
0709 12:24:07.502607 (+2727836us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:07.509788 (+  7181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:08.914048 (+1404260us) spinlock_profiling.cc:243] Waited 2.73 s on lock 0x7b5400a686d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:08.914180 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6272598400}
I20260709 12:24:08.916267 16468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34840 (request call id 1552) took 1038 ms. Trace:
I20260709 12:24:08.920744 16468 rpcz_store.cc:276] 0709 12:24:07.878272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.878468 (+   196us) service_pool.cc:224] Handling call
0709 12:24:08.916114 (+1037646us) spinlock_profiling.cc:243] Waited 1.54 ms on lock 0x7b5400a686d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:08.916214 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3536128}
W20260709 12:24:08.899063 21621 log.cc:927] Time spent T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:24:08.869510 21426 consensus_queue.cc:260] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:08.921978 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.922235 21426 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:24:08.923418 21561 log.cc:927] Time spent T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 1.368s	user 0.003s	sys 0.000s
I20260709 12:24:08.901836 21520 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:08.925041 21581 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:08.919020 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.919981  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:08.927412 21581 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:08.928267 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:08.930023 17255 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 17.
I20260709 12:24:08.932616 21473 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:08.929008 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:08.938344 21593 log.cc:927] Time spent T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.563s	user 0.000s	sys 0.002s
I20260709 12:24:08.940112 16468 raft_consensus.cc:1275] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 12:24:08.941016 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 399270us
I20260709 12:24:08.942478 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:08.943727 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:08.945418 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:08.941913 17144 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:08.947173 17263 raft_consensus.cc:1240] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:24:08.940821 21481 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:08.949179 17266 raft_consensus.cc:2393] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 14: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:08.891578 21601 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:08.949568 17271 raft_consensus.cc:2393] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 14: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:08.950642 17144 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:08.952239 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:08.952610 16401 raft_consensus.cc:1240] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:08.952930 17255 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:24:08.954267 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:08.954891 16543 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:08.960999 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:08.962426 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:08.963037 16399 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:08.964179 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:08.967720 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:08.968472 16400 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:24:08.968775 21601 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:08.969766 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 180039us
I20260709 12:24:08.957221 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:08.973268 16401 raft_consensus.cc:1240] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:08.975345 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:08.976657 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:08.978057 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:08.978806 16543 raft_consensus.cc:1275] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:24:08.978992 16551 raft_consensus.cc:2393] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 16: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:08.979524 16515 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:08.979804 16547 raft_consensus.cc:2393] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 17: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:08.980422 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:08.980633 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:08.981189 16546 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:08.981592 16515 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:08.986835 16550 raft_consensus.cc:1240] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:24:08.951522 21501 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:08.988862 16515 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:08.987491 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:09.001657 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.988257 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:08.999300 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:09.000083 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.017510 16401 raft_consensus.cc:2410] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
W20260709 12:24:09.028178 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 78560.1us
I20260709 12:24:09.003077 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:09.029362 17144 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:09.030457 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:09.030582 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:09.031194 17144 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:09.032136 16515 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:09.029372 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:09.045560 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:09.059568 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 61826.5us
I20260709 12:24:09.061949 16367 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:09.064033 16400 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:24:09.068477 17144 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:08.986765 21250 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:08.951834 21581 consensus_queue.cc:1048] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.083s
I20260709 12:24:08.986541 21633 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:08.971191 21236 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:09.049823 21561 log.cc:927] Time spent T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:24:08.981272 21570 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:08.974666 21481 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:09.033275 21605 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:09.099905 16515 leader_election.cc:400] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 13. Current term is 14.
W20260709 12:24:09.106336 16441 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.037s	sys 0.059s
I20260709 12:24:09.026849 21498 consensus_queue.cc:1048] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 12:24:09.106992 16515 leader_election.cc:403] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:24:09.112452 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:09.113255 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 281830us
I20260709 12:24:09.071691 21515 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:24:09.107333 16441 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.037s	sys 0.059s
I20260709 12:24:09.113229 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:09.067030 21401 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:24:09.126319 16543 raft_consensus.cc:2368] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 14. Current term is 15.
I20260709 12:24:09.127743 16514 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:09.129667 16515 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:09.116209 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:09.138322 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:09.147521 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.095235 21522 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:09.037591 21501 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:09.188578 16515 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:09.049420 21626 consensus_queue.cc:1048] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 12:24:09.099591 21271 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:09.097544 21447 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:24:09.198239 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.195674 16546 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
W20260709 12:24:09.279585 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 218126us
W20260709 12:24:09.238868 21600 log.cc:927] Time spent T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
W20260709 12:24:09.289975 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 228127us
I20260709 12:24:09.240571 21476 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:09.291785 17143 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
W20260709 12:24:09.298574 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 230145us
I20260709 12:24:09.310359 17144 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:09.195760 21632 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:24:09.297698 16367 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:09.220129 21472 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:09.236937 21608 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:09.313468 17144 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:09.191762 21625 consensus_queue.cc:1059] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.098s
I20260709 12:24:09.288566 21584 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:09.273839 21522 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:09.284946 21447 raft_consensus.cc:697] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:24:09.340385 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 207203us
I20260709 12:24:09.326974 21281 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:09.322059 21633 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:09.360877 16475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1549) took 1557 ms. Trace:
I20260709 12:24:09.361544 16475 rpcz_store.cc:276] 0709 12:24:07.803545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.803669 (+   124us) service_pool.cc:224] Handling call
0709 12:24:07.817145 (+ 13476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:07.856566 (+ 39421us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:09.360562 (+1503996us) spinlock_profiling.cc:243] Waited 9.6 ms on lock 0x7b54004eff50. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:09.360775 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22084480}
I20260709 12:24:09.307852 21515 consensus_queue.cc:1048] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
W20260709 12:24:09.398870 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 56044.2us
W20260709 12:24:09.403070 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.354586 21581 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:09.334270 21472 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:09.394121 21600 log.cc:927] Time spent T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:24:09.341734 21522 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:09.425302 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 111922us
I20260709 12:24:09.426359 17144 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:09.433173 16399 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 18.
I20260709 12:24:09.408913 21587 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:09.411412 21633 raft_consensus.cc:2764] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:24:09.416034 21524 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:09.409929 21646 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:09.417361 21645 raft_consensus.cc:2764] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:24:09.452947 21581 raft_consensus.cc:697] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:09.462152 21645 consensus_queue.cc:1048] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:24:09.462342 21581 consensus_queue.cc:237] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:09.463272 21587 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:09.435266 21523 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:09.436563 21420 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:09.422554 21636 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:09.449798 21541 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 13. Current term is 14.
I20260709 12:24:09.453495 21281 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:24:09.517365 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 88360.8us
I20260709 12:24:09.498692 21570 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:09.448417 21629 consensus_queue.cc:1048] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.075s
W20260709 12:24:09.532979 21643 log.cc:927] Time spent T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 12:24:09.543479 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:09.556059 17264 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:09.498920 21481 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:09.541167 21426 consensus_meta.cc:287] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.618s	user 0.013s	sys 0.002s
I20260709 12:24:09.464552 21524 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:09.434924 21447 consensus_queue.cc:237] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:09.572908 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 121146us
I20260709 12:24:09.579645 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:09.581918 16403 raft_consensus.cc:3055] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:09.582477 16403 raft_consensus.cc:740] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:09.584444 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:09.586685 17144 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:09.588037 16469 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.009s	sys 0.017s
W20260709 12:24:09.589569 16469 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.010s	sys 0.017s
I20260709 12:24:09.497709 21646 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:09.516745 21420 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:09.583288 16403 consensus_queue.cc:260] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:09.593967 17144 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:09.598605 16403 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:09.512975 21523 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:09.594900 16548 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.003s	sys 0.015s
W20260709 12:24:09.604035 16548 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.003s	sys 0.015s
I20260709 12:24:09.556727 21501 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:09.590755 17264 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
W20260709 12:24:09.612629 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.607740 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:09.564268 21660 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:09.599754 21570 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:09.514125 21636 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:09.541898 21281 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:09.583200 21515 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:09.592650 21633 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:24:09.605953 21616 log.cc:927] Time spent T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.805s	user 0.002s	sys 0.008s
W20260709 12:24:09.627012 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.617522 21660 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:24:09.643337 21635 log.cc:927] Time spent T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.288s	user 0.001s	sys 0.000s
W20260709 12:24:09.643654 21647 log.cc:927] Time spent T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.127s	user 0.001s	sys 0.001s
W20260709 12:24:09.646077 21642 log.cc:927] Time spent T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.267s	user 0.001s	sys 0.000s
I20260709 12:24:09.647488 21420 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:09.649008 21522 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:09.649494 21646 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:09.651199 21472 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:09.654553 21524 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:09.656958 21649 log.cc:927] Time spent T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
I20260709 12:24:09.658097 21281 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:09.622990 21656 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:09.635089 21633 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:09.662497 21656 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:09.663209 21656 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:09.663759 21633 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:09.666282 21636 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:09.668601 21660 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:09.647851 21640 log.cc:927] Time spent T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
I20260709 12:24:09.674528 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:09.676772 16542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1478) took 1233 ms. Trace:
I20260709 12:24:09.677248 16542 rpcz_store.cc:276] 0709 12:24:08.443104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.459151 (+ 16047us) service_pool.cc:224] Handling call
0709 12:24:08.459466 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:08.459835 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:08.459867 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:08.490957 (+ 31090us) log.cc:844] Serialized 26 byte log entry
0709 12:24:08.651087 (+160130us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:08.651145 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:08.651240 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:09.550867 (+899627us) spinlock_profiling.cc:243] Waited 939 us on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:24:09.676409 (+125542us) raft_consensus.cc:1685] finished
0709 12:24:09.676434 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:09.676680 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":505,"raft.queue_time_us":11376,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":1613,"spinlock_wait_cycles":2158976,"thread_start_us":127013,"threads_started":1,"wal-append.queue_time_us":239897}
W20260709 12:24:09.680128 21473 consensus_meta.cc:287] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.747s	user 0.005s	sys 0.005s
I20260709 12:24:09.680219 21271 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:09.681018 16398 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:24:09.681478 21355 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:09.681950 21355 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:09.686386 21383 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:09.632982 21515 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:09.691038 21515 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:09.710232 21523 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:09.712756 16471 raft_consensus.cc:1240] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:09.713902 21587 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:09.715749 21498 raft_consensus.cc:3055] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:09.716199 21498 raft_consensus.cc:740] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:09.717082 21498 consensus_queue.cc:260] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:09.718135 21498 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:09.719408 16471 raft_consensus.cc:1275] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:24:09.723994 17261 raft_consensus.cc:1240] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:09.727877 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:09.728574 17266 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:24:09.729667 17261 raft_consensus.cc:1275] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:24:09.731205 21499 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:09.734194 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:09.666972 21656 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:09.742875 16515 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:09.743839 17144 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:09.746869 21656 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:09.748818 21590 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:09.749648 21590 raft_consensus.cc:697] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:09.750844 21590 consensus_queue.cc:237] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:09.757932 21236 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:09.767727 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:09.768182 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:09.768492 16470 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 20.
I20260709 12:24:09.769183 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:09.771062 21580 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:09.775833 16477 raft_consensus.cc:1240] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:08.928200  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:09.780359 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:09.781025 16477 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:09.783078 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:09.784771 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:09.795542 16366 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.528s (SENT) [suppressed 18 similar messages]
I20260709 12:24:09.796432 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:09.797215 16470 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
I20260709 12:24:09.798512 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:09.799259 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:09.728535 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:09.801548 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:09.802526 16470 raft_consensus.cc:2410] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
W20260709 12:24:09.806140 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 62380.8us
I20260709 12:24:09.769239 16471 raft_consensus.cc:2393] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 18: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
W20260709 12:24:09.808964 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 46695.5us
I20260709 12:24:09.804553 17142 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:24:09.816077 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.817273 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:09.819272 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:09.820425 16400 raft_consensus.cc:2410] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:24:09.820816 16403 raft_consensus.cc:1275] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 12:24:09.824481 16403 pending_rounds.cc:85] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 8
I20260709 12:24:09.824813 16403 pending_rounds.cc:107] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:24:09.825155 16403 raft_consensus.cc:2889] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:09.865134 16471 raft_consensus.cc:1240] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: []
W20260709 12:24:09.867998 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 57188.1us
I20260709 12:24:09.869474 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:09.872764 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 61321.6us
I20260709 12:24:09.873854 16515 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:09.879772 16471 raft_consensus.cc:1240] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 13. Current term is 14. Ops: []
W20260709 12:24:09.823698 21572 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.002s	sys 0.011s
W20260709 12:24:09.883512 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 56815us
I20260709 12:24:09.858569 21539 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:09.901213 16469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 995) took 1032 ms. Trace:
I20260709 12:24:09.901450 16515 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:09.901640 17144 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:09.904507 21523 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:24:09.913820 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:09.916991 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.919276 17271 raft_consensus.cc:1275] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 12:24:09.894797 21572 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.003s	sys 0.011s
I20260709 12:24:09.923967 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:09.896371 21639 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:24:09.927841 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.900709 21656 raft_consensus.cc:2804] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:09.928673 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:09.937552 21523 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:24:09.949127 16367 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.006s	sys 0.015s
I20260709 12:24:09.952214 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:09.896179 21487 consensus_queue.cc:1048] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:24:09.954053 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:09.919909 21541 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:09.881872 21236 consensus_queue.cc:1059] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 12:24:09.949299 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:09.900895 21587 consensus_queue.cc:1048] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
W20260709 12:24:09.957813 21476 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.006s	sys 0.016s
W20260709 12:24:09.958030 16367 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.007s	sys 0.016s
W20260709 12:24:09.963186 17255 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.005s	sys 0.017s
W20260709 12:24:09.966611 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 632711us
W20260709 12:24:09.966926 17255 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.005s	sys 0.018s
I20260709 12:24:09.943749 21656 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:09.934319 21639 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:24:09.962059 17143 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.007s	sys 0.016s
I20260709 12:24:09.912094 21660 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:09.968693 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.915666 21539 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:09.971834 17143 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.010s	sys 0.016s
W20260709 12:24:09.976835 17145 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.008s	sys 0.016s
W20260709 12:24:09.991216 17145 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.008s	sys 0.017s
W20260709 12:24:09.976527 16394 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.007s	sys 0.018s
W20260709 12:24:09.991952 16394 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.007s	sys 0.019s
W20260709 12:24:09.966420 21476 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.006s	sys 0.016s
I20260709 12:24:09.992996 16394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 1385) took 1035 ms. Trace:
I20260709 12:24:09.998001 16394 rpcz_store.cc:276] 0709 12:24:08.957804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.959610 (+  1806us) service_pool.cc:224] Handling call
0709 12:24:08.978260 (+ 18650us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:08.978832 (+   572us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:08.978860 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:09.002423 (+ 23563us) log.cc:844] Serialized 52 byte log entry
0709 12:24:09.290523 (+288100us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:09.335630 (+ 45107us) log.cc:844] Serialized 14 byte log entry
0709 12:24:09.992637 (+657007us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:09.992718 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:09.992763 (+    45us) raft_consensus.cc:1685] finished
0709 12:24:09.992778 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:09.992932 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38010,"prepare.queue_time_us":45000,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":126869,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":14269,"thread_start_us":853418,"threads_started":2,"wal-append.queue_time_us":376157}
I20260709 12:24:09.962054 21027 consensus_queue.cc:1048] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:24:10.007066 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:10.014285 16547 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:09.975399 21639 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:10.015813 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:09.953279 21523 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:10.018088 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:09.995334 21476 raft_consensus.cc:697] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:10.019575 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:10.020565 16545 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
I20260709 12:24:10.020733 21355 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:09.961270 21250 consensus_queue.cc:1048] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:24:10.021485 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:10.023635 21660 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:10.025786 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:10.027407 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:10.028144 16545 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
I20260709 12:24:10.029276 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:10.030460 16545 raft_consensus.cc:2410] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:24:09.902262 16469 rpcz_store.cc:276] 0709 12:24:08.868515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.871501 (+  2986us) service_pool.cc:224] Handling call
0709 12:24:08.871911 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:08.872196 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:24:09.592758 (+720562us) log.cc:844] Serialized 14 byte log entry
0709 12:24:09.900737 (+307979us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:09.900810 (+    73us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:09.900894 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:09.901000 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:09.901154 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6670,"prepare.queue_time_us":36249,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":7935,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":93,"thread_start_us":1011671,"threads_started":2}
I20260709 12:24:10.019544 21476 consensus_queue.cc:237] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:10.035605 16477 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:10.036103 21523 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:10.038614 16550 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:24:10.046406 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:10.047257 16469 raft_consensus.cc:2393] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 20: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:10.039511 21539 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:10.041592 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:10.048854 16472 raft_consensus.cc:2368] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 17. Current term is 18.
I20260709 12:24:10.045542 17258 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 20.
I20260709 12:24:09.971686 21656 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:10.045689 16367 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:10.054651 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:10.050866 16547 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:10.059796 16542 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:24:10.090381 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:10.091444 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 45965.7us
I20260709 12:24:10.092235 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:10.093204 17271 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:24:10.092787 16367 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:10.101969 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:10.102793 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:10.104928 16398 raft_consensus.cc:2368] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 17. Current term is 18.
I20260709 12:24:10.107131 16367 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
W20260709 12:24:10.105072 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 67695.8us
I20260709 12:24:10.110141 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:10.113297 21575 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:10.113806 21575 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:10.114178 21575 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:10.114677 21497 raft_consensus.cc:3055] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:10.115097 21497 raft_consensus.cc:740] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:10.115393 17254 raft_consensus.cc:1275] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:24:10.117411 21662 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.090399 21426 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:10.117908 21662 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:10.118491 21426 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:10.120018 21662 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:10.120471 21426 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:10.096896 21355 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:10.085173 21498 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:24:10.121390 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.052407 21522 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:10.127849 16543 raft_consensus.cc:1275] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:24:10.131883 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:10.051954 21646 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.124045 17258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45230 (request call id 1311) took 1194 ms. Trace:
I20260709 12:24:10.148694 17142 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:10.159073 17258 rpcz_store.cc:276] 0709 12:24:08.929361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.941252 (+ 11891us) service_pool.cc:224] Handling call
0709 12:24:09.734118 (+792866us) spinlock_profiling.cc:243] Waited 8.97 ms on lock 0x7b5c001e6b8c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f28797f3161 00007f2879994a79 00007f287ae4e9b6 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:10.123891 (+389773us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b54005f6c54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:10.123989 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":734706304}
I20260709 12:24:10.152226 17144 leader_election.cc:400] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 17. Current term is 18.
I20260709 12:24:10.161581 17144 leader_election.cc:403] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:10.153226 17143 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:10.163635 17142 leader_election.cc:400] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 17. Current term is 18.
W20260709 12:24:10.162616 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 367078us
I20260709 12:24:10.166595 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:10.154206 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 47185.1us
I20260709 12:24:10.096892 21648 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:10.097007 21271 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:10.170538 16367 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:10.171451 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:10.174285 17254 raft_consensus.cc:1275] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:24:10.105798 21655 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:10.173720 17271 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 18.
I20260709 12:24:10.099472 21587 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:10.176980 16397 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:24:10.183642 17143 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:10.115914 21497 consensus_queue.cc:260] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:10.168641 21426 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:10.182835 21581 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:10.227548 16394 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 20.
W20260709 12:24:10.230479 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 60062.9us
I20260709 12:24:10.232632 16367 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:10.184489 21515 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 17. Current term is 18.
I20260709 12:24:10.199532 21487 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.199079 21633 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:10.170522 21522 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:10.160660 21498 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:10.254431 16441 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:10.247413 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 56817.9us
I20260709 12:24:10.238296 16366 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
W20260709 12:24:10.257416 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 73853.2us
I20260709 12:24:10.259799 17143 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:10.170475 21646 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:10.218266 21426 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:10.261348 16441 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:10.279747 16441 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:10.236825 21645 raft_consensus.cc:2764] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:24:10.206209 21587 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:10.190797 21271 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:10.284678 16367 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:10.277040 17143 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:10.201706 21655 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:10.245301 21383 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:10.294653 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 56516.6us
I20260709 12:24:10.187263 21648 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:10.301982 17143 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:10.299015 16366 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:10.279647 21499 consensus_meta.cc:287] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.548s	user 0.010s	sys 0.005s
I20260709 12:24:10.256464 21581 raft_consensus.cc:697] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:10.266625 21497 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:10.345741 16546 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:24:10.265084 21673 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:10.276547 21478 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:10.282405 21426 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:10.308552 16367 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:10.381562 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 73116.8us
I20260709 12:24:10.353912 17270 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:24:10.272819 21674 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:10.382665 16367 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:10.376612 21399 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:24:10.376960 21251 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:10.439707 16543 raft_consensus.cc:1275] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:24:10.343796 21639 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:10.409143 21645 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:10.308898 21383 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:10.319135 21355 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:10.330158 21580 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:10.296514 21146 consensus_queue.cc:1059] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.094s
I20260709 12:24:10.407313 21587 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:10.325903 21668 log.cc:927] Time spent T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:24:10.307411 21671 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:10.260946 21487 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:10.326747 21615 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:10.335256 21524 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:10.454011 16549 raft_consensus.cc:2393] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 13: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:10.322391 21660 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:10.431699 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:10.471248 16542 raft_consensus.cc:2468] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 11.
I20260709 12:24:10.337908 21646 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:10.320685 21522 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:10.322170 16224 catalog_manager.cc:5697] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 8 to 9, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 9 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
I20260709 12:24:10.472816 17143 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:10.390640 21674 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:10.444065 21580 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:10.448602 21675 consensus_queue.cc:1048] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 12:24:10.455456 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:10.443189 21355 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:10.496824 21674 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:10.500402 21355 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:10.443080 21251 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:10.481570 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.504492 21523 raft_consensus.cc:2804] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:10.504971 21523 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:10.505321 21523 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:24:10.438879 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 56334.4us
I20260709 12:24:10.440357 21476 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:10.498953 17266 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:10.507494 16367 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:10.512025 21476 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:10.512462 21476 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:10.512858 21476 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:10.512782 16367 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:10.512184 21355 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:10.515687 21355 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:10.449409 21671 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:10.516610 21671 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:10.519440 21383 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:10.509011 21498 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:10.497607 21673 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:10.521323 21673 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:10.524593 21426 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:10.525413 21476 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:10.527103 21426 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:10.525871 21355 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:24:10.531926 17266 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:24:10.533522 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:10.545667 17271 raft_consensus.cc:2393] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 14: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:10.454584 21615 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:10.510730 21674 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:10.454169 21524 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:10.547744 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:10.485728 21587 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.559492 21587 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:10.561563 21678 log.cc:927] Time spent T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:24:10.562642 21587 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:10.551642 21615 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:10.565369 21655 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:10.566766 21672 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.444638 21581 consensus_queue.cc:237] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:10.567595 21672 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:10.568552 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:10.569161 17263 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:10.488421 21645 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:10.497439 21580 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:10.558490 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:10.556559 17271 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:24:10.576705 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:10.584785 16542 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:24:10.577683 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:10.567006 21648 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:10.588698 17143 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:10.581346 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:10.601884 17263 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:24:10.570780 21524 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:24:10.604436 16551 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.053s	sys 0.085s
W20260709 12:24:10.611299 16551 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.053s	sys 0.085s
I20260709 12:24:10.619431 21476 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:10.598459 21587 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:10.629488 21673 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:10.596468 21645 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:10.631953 21645 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:10.571537 21656 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:10.632907 21584 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.633446 21584 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:10.634157 21656 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:10.635995 17258 raft_consensus.cc:1240] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:24:10.636601 21271 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:10.636128 21575 consensus_meta.cc:287] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.522s	user 0.012s	sys 0.000s
W20260709 12:24:10.647182 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.628525 21587 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:10.659518 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 339161us
W20260709 12:24:10.662483 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 363534us
I20260709 12:24:10.659976 21587 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:10.644258 21575 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:10.662138 21633 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:24:10.658344 21590 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.016s	sys 0.028s
I20260709 12:24:10.669595 16366 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:10.696417 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:10.700814 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:10.702276 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:10.703228 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:10.684912 21236 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.714910 21636 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:10.729113 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 350830us
W20260709 12:24:10.754201 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 84907.8us
I20260709 12:24:10.757145 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:10.761529 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:10.766330 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:10.766556 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:10.767153 16403 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:10.767983 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:10.771031 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:10.782963 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:10.791553 16400 raft_consensus.cc:1275] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:24:10.798867 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:24:10.751368 21590 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.017s	sys 0.028s
W20260709 12:24:10.801290 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 521589us
W20260709 12:24:10.753675 21669 log.cc:927] Time spent T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.759s	user 0.003s	sys 0.003s
I20260709 12:24:10.804682 16394 raft_consensus.cc:1275] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:24:10.806269 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:10.807030 16441 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:10.817276 16473 raft_consensus.cc:2410] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:24:10.807395 16477 raft_consensus.cc:3055] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:10.807663 16403 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:24:10.817953 16477 raft_consensus.cc:740] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:10.808998 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:10.812441 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:10.822503 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:10.749984 21674 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:10.824896 16469 raft_consensus.cc:2393] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 12: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:10.827837 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:10.828465 16396 raft_consensus.cc:1275] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:24:10.829172 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:10.829995 16403 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:24:10.830865 16395 raft_consensus.cc:1275] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:24:10.782449 21645 raft_consensus.cc:2749] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:24:10.792023 21648 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.819036 16477 consensus_queue.cc:260] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:10.837472 16477 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:10.827415 16442 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:10.836730 16403 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:10.843235 16394 raft_consensus.cc:1240] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 15. Current term is 16. Ops: []
W20260709 12:24:10.826018 21684 log.cc:927] Time spent T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.177s	user 0.000s	sys 0.002s
I20260709 12:24:10.793088 21476 consensus_queue.cc:1059] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:24:10.848621 16398 raft_consensus.cc:2393] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 14: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:10.930846 21675 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.882190 21575 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:10.833395 21601 consensus_queue.cc:1059] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 12:24:10.881075 21691 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:10.911475 21615 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:10.882180 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:10.893270 21685 log.cc:927] Time spent T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:24:10.830900 21355 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:10.850239 21648 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:10.891870 21524 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:10.927933 21581 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.928707 21671 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:10.916808 21674 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.889323 21639 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:10.904819 21515 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.792949 21636 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:10.898880 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:10.791348 21236 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:10.930743 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:10.850606 21645 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:10.896924 21478 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:10.952128 16477 raft_consensus.cc:1275] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:24:10.975293 21615 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:11.002351 21270 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:10.945340 21673 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.014457 21270 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:10.953979 21675 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:10.980911 21355 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:11.016748 16551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 1061) took 1009 ms. Trace:
I20260709 12:24:11.017340 16551 rpcz_store.cc:276] 0709 12:24:10.007573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.007784 (+   211us) service_pool.cc:224] Handling call
0709 12:24:10.008323 (+   539us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:10.008702 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:10.008740 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:10.009673 (+   933us) log.cc:844] Serialized 26 byte log entry
0709 12:24:10.611888 (+602215us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:10.611949 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.612062 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:11.016418 (+404356us) raft_consensus.cc:1685] finished
0709 12:24:11.016451 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:11.016663 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":601384,"threads_started":1,"wal-append.queue_time_us":739618}
I20260709 12:24:11.018766 21355 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:10.981402 16543 raft_consensus.cc:1240] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:10.958356 21601 raft_consensus.cc:3055] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:10.983664 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:10.986982 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 53510.2us
I20260709 12:24:10.995388 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:10.991946 21515 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:10.984686 21497 consensus_meta.cc:287] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.605s	user 0.010s	sys 0.002s
I20260709 12:24:10.994578 17143 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:11.009038 16548 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:11.014518 21673 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:10.977157 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:11.026579 21270 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:10.999732 21636 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:11.020015 21601 raft_consensus.cc:740] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:24:11.022765 21523 consensus_meta.cc:287] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.517s	user 0.012s	sys 0.001s
I20260709 12:24:11.000953 21487 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:11.027123 21478 raft_consensus.cc:697] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:10.989326 21581 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:11.035897 21580 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:11.040597 17269 raft_consensus.cc:1275] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:24:11.045408 21673 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:11.047698 21236 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:11.070674 21671 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:11.021760 21645 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:11.070900 17271 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:24:11.072424 21581 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:11.042899 21670 log.cc:927] Time spent T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.744s	user 0.001s	sys 0.001s
I20260709 12:24:10.992753 21674 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:11.063800 17264 raft_consensus.cc:1275] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:24:11.031786 21695 log.cc:927] Time spent T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:24:11.068110 21522 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.082245 21645 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:11.083287 17261 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 17.
I20260709 12:24:11.084754 21674 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:11.085916 21515 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:11.086512 21601 consensus_queue.cc:260] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:11.087059 21648 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:11.077853 21522 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:11.065788 21523 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:11.089802 21522 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:11.087877 21601 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:11.099817 21478 consensus_queue.cc:237] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:11.100853 21584 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:11.090126 21523 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:11.130997 21692 raft_consensus.cc:3055] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:11.131675 21692 raft_consensus.cc:740] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:11.138134 21251 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:11.142364 21675 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:11.165519 21499 raft_consensus.cc:3055] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:11.165817 21580 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:11.186235 21672 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:11.176467 21499 raft_consensus.cc:740] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:11.191917 21499 consensus_queue.cc:260] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:11.193497 21499 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:11.188465 21524 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:11.199298 21587 consensus_meta.cc:287] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.522s	user 0.011s	sys 0.001s
I20260709 12:24:11.200115 21587 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:11.207126 21587 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:11.154029 21692 consensus_queue.cc:260] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:11.209061 21692 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:10.938526 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:10.958899 16395 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
W20260709 12:24:10.946348 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.253185 16400 raft_consensus.cc:1240] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 16. Current term is 17. Ops: []
W20260709 12:24:11.257100 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 429558us
W20260709 12:24:11.260674 16476 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.005s	sys 0.045s
W20260709 12:24:11.261242 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 454150us
W20260709 12:24:10.977717 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.011157 16477 pending_rounds.cc:85] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 8
I20260709 12:24:11.267059 16477 pending_rounds.cc:107] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:24:11.267561 16477 raft_consensus.cc:2889] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:10.886348 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:11.270582 16403 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 17.
W20260709 12:24:11.261297 16476 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.006s	sys 0.045s
I20260709 12:24:11.021127 16516 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:11.259749 16442 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
W20260709 12:24:11.290161 17266 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.727s	user 0.033s	sys 0.031s
I20260709 12:24:11.290482 17255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 1304) took 2341 ms. Trace:
I20260709 12:24:11.291111 16543 raft_consensus.cc:1275] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:24:11.293236 21708 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:11.293963 21708 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:11.294577 21708 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:24:11.290875 17266 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.034s	sys 0.031s
W20260709 12:24:11.298702 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 277505us
W20260709 12:24:11.322446 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 325756us
W20260709 12:24:11.325623 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 496800us
I20260709 12:24:11.327576 16367 leader_election.cc:400] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 14. Current term is 15.
I20260709 12:24:11.328166 16367 leader_election.cc:403] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:24:11.330295 21707 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 14. Current term is 15.
I20260709 12:24:11.332504 21590 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:11.335548 21707 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:11.336637 21707 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:11.341677 21707 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:11.342243 21590 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:11.343813 21590 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:11.345602 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:24:11.352347 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 93047.3us
I20260709 12:24:11.291218 17255 rpcz_store.cc:276] 0709 12:24:08.954167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.972102 (+ 17935us) service_pool.cc:224] Handling call
0709 12:24:08.972532 (+   430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:09.044973 (+ 72441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:09.045023 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:09.093554 (+ 48531us) log.cc:844] Serialized 26 byte log entry
0709 12:24:09.967348 (+873794us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:09.988701 (+ 21353us) log.cc:844] Serialized 14 byte log entry
0709 12:24:09.988958 (+   257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:09.989050 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:11.042312 (+1053262us) spinlock_profiling.cc:243] Waited 5.46 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:24:11.289433 (+247121us) raft_consensus.cc:1685] finished
0709 12:24:11.289659 (+   226us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:11.290100 (+   441us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73114,"prepare.queue_time_us":19968,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":121703,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":64234,"spinlock_wait_cycles":12554624,"thread_start_us":779669,"threads_started":1,"wal-append.queue_time_us":1119854}
I20260709 12:24:11.360121 17266 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:11.307574 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:11.369359 16551 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
W20260709 12:24:11.369915 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 49069.1us
I20260709 12:24:11.371035 21656 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:24:11.371387 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 99105.5us
I20260709 12:24:11.372915 16441 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:11.374348 21691 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:11.375993 21611 consensus_queue.cc:1048] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:11.388288 17270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45224 (request call id 1313) took 1296 ms. Trace:
I20260709 12:24:11.401394 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:11.391911 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:11.418921 16551 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:11.419130 16546 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:11.409283 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:11.414229 17270 rpcz_store.cc:276] 0709 12:24:10.091252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.096631 (+  5379us) service_pool.cc:224] Handling call
0709 12:24:11.387470 (+1290839us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b54007053d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:11.387938 (+   468us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":558767488}
I20260709 12:24:11.413154 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:11.392449 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:11.401543 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:11.388953 21524 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:11.408455 21587 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.428941 16542 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:11.431071 21587 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:11.432794 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:11.433537 17143 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:11.434518 21587 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:11.434659 21633 consensus_queue.cc:1048] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:11.434973 21524 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:11.437726 16549 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:11.438464 16545 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:11.438596 21523 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:11.416528 21712 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.451440 21707 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:11.451967 17144 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:11.455230 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:11.442807 21523 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.459797 21523 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:11.460915 17144 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:11.461045 21498 consensus_queue.cc:1048] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:11.462129 21498 consensus_queue.cc:1243] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer f99066d13e524238aec6aa2024a8bd50 log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:24:11.455440 21426 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:11.464284 21522 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:11.467633 17258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 1328) took 1296 ms. Trace:
I20260709 12:24:11.468446 17258 rpcz_store.cc:276] 0709 12:24:10.171082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.171790 (+   708us) service_pool.cc:224] Handling call
0709 12:24:10.172164 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:10.637110 (+464946us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:11.466726 (+829616us) spinlock_profiling.cc:243] Waited 463 ms on lock 0x7b540005f0d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:11.467335 (+   609us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1065808768}
W20260709 12:24:11.472870 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.465363 21522 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.474752 21522 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:11.475711 21672 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.476816 21672 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:11.478865 21675 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.479697 21672 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:11.479617 21675 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:11.454276 21611 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.480954 21611 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:11.482607 21675 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:11.483750 21611 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:11.473425 17143 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:11.485474 21633 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:11.486402 21633 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.486923 21633 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:11.475073 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 102437us
I20260709 12:24:11.490041 21712 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:11.497459 16516 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:11.498560 21250 consensus_queue.cc:1059] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:11.498943 16516 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:11.455710 21524 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:11.500818 16476 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
I20260709 12:24:11.501875 21250 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:11.502640 21250 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.503001 21250 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:11.502398 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:11.464452 21426 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:11.456043 21707 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:11.506214 21707 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:11.507339 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:11.511200 21146 raft_consensus.cc:3055] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:11.511657 21584 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:11.506235 21426 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:11.518949 17266 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 21.
I20260709 12:24:11.519739 21655 raft_consensus.cc:2764] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader election decision vote started in defunct term 20: won
I20260709 12:24:11.522536 21704 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:11.243904  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:11.527760 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:11.528187 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:11.528584 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:11.528963 17143 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:11.529749 16473 raft_consensus.cc:2410] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:24:11.530041 21655 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:11.530449 21655 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.530786 21655 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:11.512507 21584 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.531636 16472 raft_consensus.cc:1275] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:24:11.532013 21584 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:11.523439 21704 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:11.492008 16441 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:11.512233 21146 raft_consensus.cc:740] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:24:11.532464 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 44175.2us
I20260709 12:24:11.534638 21704 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:11.577380 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:11.577929 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:11.578114 17266 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:24:11.578351 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:11.537721 21478 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:24:11.537719 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 45912.1us
I20260709 12:24:11.580097 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:11.580613 16441 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:11.551808 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:11.582345 16547 raft_consensus.cc:2410] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
W20260709 12:24:11.551599 21719 log.cc:927] Time spent T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:24:11.555990 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:11.585741 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:11.586519 16400 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
I20260709 12:24:11.583570 16549 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
I20260709 12:24:11.556756 16546 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
W20260709 12:24:11.562268 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:11.538435 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 60139.6us
I20260709 12:24:11.552202 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:11.587843 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:11.562750 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:11.573472 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 73432.1us
I20260709 12:24:11.556486 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:11.559645 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:11.601543 16550 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:11.601995 16394 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:11.560025 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:11.603492 16542 raft_consensus.cc:2468] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:24:11.556249 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:11.560760 16548 consensus_meta.cc:287] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.530s	user 0.013s	sys 0.004s
I20260709 12:24:11.558451 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:24:11.569314 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 55413.2us
I20260709 12:24:11.557435 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:11.556268 21146 consensus_queue.cc:260] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:11.607779 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 61052.4us
I20260709 12:24:11.609233 21146 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:11.609359 16514 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:11.610183 21478 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:11.558707 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:11.557853 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:11.611932 21636 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:11.612428 21636 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:11.559477 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:11.568347 21720 log.cc:927] Time spent T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 12:24:11.611230 21656 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:11.612843 21636 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:11.544631 16224 catalog_manager.cc:5697] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 9 to 10, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 10 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
W20260709 12:24:11.617316 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.611943 16395 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:11.565388 21712 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:11.569056 21590 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:11.623942 16403 raft_consensus.cc:2410] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:24:11.627475 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:11.519755 21708 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:11.625617 16548 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
I20260709 12:24:11.624550 16402 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:11.628530 16401 raft_consensus.cc:2410] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:24:11.626754 16396 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
I20260709 12:24:11.630210 16516 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:11.532279 21523 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:11.626926 16224 catalog_manager.cc:5697] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:11.657617 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:11.663156 16394 raft_consensus.cc:2410] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:24:11.654219 17143 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:11.669687 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:11.670825 16476 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 17.
I20260709 12:24:11.673075 16398 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:11.673331 21639 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:11.681133 17258 raft_consensus.cc:2368] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17.
I20260709 12:24:11.674619 16514 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:11.677596 16367 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:11.684934 16441 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:11.688090 21712 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:24:11.691975 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 76036.5us
W20260709 12:24:11.697211 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 43598.5us
I20260709 12:24:11.681183 17142 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:11.704114 17143 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:11.694679 21674 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:11.694622 21712 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.675729 21655 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:11.716756 21580 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:11.717846 21580 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.719039 21580 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:11.717819 21655 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:11.720676 21704 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:11.723027 21633 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:11.724424 21478 consensus_queue.cc:1048] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:11.726409 16367 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:11.728681 17144 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:11.729923 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 71955.1us
I20260709 12:24:11.730293 17144 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:11.732534 21656 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:11.732686 21655 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:11.733389 21655 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.733875 21655 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:11.734665 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:11.736748 21707 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:11.737154 21498 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:11.701759 21639 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:11.739464 16367 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:11.746390 16224 catalog_manager.cc:5697] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
W20260709 12:24:11.741182 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 122378us
I20260709 12:24:11.748508 21672 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:11.748868 21674 raft_consensus.cc:697] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:11.749352 21672 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:11.749861 21672 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:11.750263 21674 consensus_queue.cc:237] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:11.754068 16442 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:11.713846 21712 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:11.756541 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 51432.1us
I20260709 12:24:11.757651 21704 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:11.758076 21704 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.758462 21704 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:11.743611 21639 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:11.747735 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:11.759222 16397 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:11.760052 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 80936.7us
I20260709 12:24:11.760792 21648 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:11.761206 21399 consensus_queue.cc:1048] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:11.762936 21478 consensus_queue.cc:1059] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:24:11.763995 21478 raft_consensus.cc:3055] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:11.764820 21478 raft_consensus.cc:740] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:11.765957 21478 consensus_queue.cc:260] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:11.772758 16221 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53622 (request call id 135) took 30005 ms (client timeout 15000 ms). Trace:
I20260709 12:24:11.772951 21478 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:11.775287 16397 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
I20260709 12:24:11.744017 21633 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:11.784034 21580 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:11.761088 17142 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
W20260709 12:24:11.805644 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 44748.7us
I20260709 12:24:11.806574 21580 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:11.806788 21250 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:11.790407 16441 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:11.790812 21655 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:11.794243 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:11.797809 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:11.800045 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:11.800963 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:11.801875 16367 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:11.789727 21497 consensus_queue.cc:1059] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:11.808238 21524 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:24:11.804558 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:11.777048 16224 catalog_manager.cc:5697] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:11.813268 16400 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
W20260709 12:24:11.820847 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 126417us
I20260709 12:24:11.822830 17143 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:11.828033 21656 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:11.824061 16515 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:11.829128 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 96951.3us
I20260709 12:24:11.810736 21523 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:11.823995 21611 raft_consensus.cc:3055] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:11.841516 21399 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:11.830185 21587 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:11.843048 16395 raft_consensus.cc:1275] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 12:24:11.843845 21723 log.cc:927] Time spent T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.107s	user 0.000s	sys 0.003s
I20260709 12:24:11.844849 16398 raft_consensus.cc:1275] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:24:11.846046 21655 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:11.823663 21704 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:11.848875 16224 catalog_manager.cc:5697] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 11 to 12, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 12 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
W20260709 12:24:11.850119 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 60837.2us
I20260709 12:24:11.811862 21633 consensus_queue.cc:1048] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
W20260709 12:24:11.877365 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.878095 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:11.880498 16397 raft_consensus.cc:1240] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 16. Current term is 17. Ops: []
W20260709 12:24:11.882884 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 81110.9us
I20260709 12:24:11.883703 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:11.879777 21707 raft_consensus.cc:697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 21 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:11.860455 21399 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:11.885773 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:11.860162 21587 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.855093 21611 raft_consensus.cc:740] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:11.889075 17143 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:11.868397 21708 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:11.889935 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:11.895643 16400 raft_consensus.cc:2410] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:24:11.895684 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:11.896759 16546 raft_consensus.cc:2368] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17.
I20260709 12:24:11.899407 16224 catalog_manager.cc:5697] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 8 to 11, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 11 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:24:11.900701 16441 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:11.901674 21497 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:24:11.901396 16224 catalog_manager.cc:5697] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 8 to 10, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 10 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:11.903566 21704 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:11.904426 21704 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:11.905015 16550 raft_consensus.cc:1240] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:11.905848 21696 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:11.904080 16224 catalog_manager.cc:5697] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 5 to 10, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 10 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:11.900599 21691 consensus_queue.cc:1048] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:24:11.773393 16221 rpcz_store.cc:269] 0709 12:23:41.767669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:41.770501 (+  2832us) service_pool.cc:224] Handling call
0709 12:24:11.772464 (+30001963us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:01.557204 (+     0us) spinlock_profiling.cc:243] Waited 9.02 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2872d7f0ac 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d99455 00007f28798be16c
0709 12:24:02.001649 (+444445us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:02.014209 (+ 12560us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:02.014258 (+    49us) write_op.cc:435] Acquired schema lock
0709 12:24:02.014291 (+    33us) tablet.cc:662] Decoding operations
0709 12:24:02.027665 (+ 13374us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:02.027816 (+   151us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:02.027857 (+    41us) tablet.cc:685] Acquiring locks for 25 operations
0709 12:24:02.030954 (+  3097us) tablet.cc:701] Row locks acquired
0709 12:24:02.031020 (+    66us) write_op.cc:260] PREPARE: finished
0709 12:24:02.031288 (+   268us) write_op.cc:270] Start()
0709 12:24:02.031761 (+   473us) write_op.cc:276] Timestamp: P: 1783599842031189 usec, L: 0
0709 12:24:02.031821 (+    60us) op_driver.cc:348] REPLICATION: starting
0709 12:24:02.036843 (+  5022us) log.cc:844] Serialized 9628 byte log entry
0709 12:24:02.800532 (+763689us) op_driver.cc:464] REPLICATION: finished
0709 12:24:03.043887 (+243355us) write_op.cc:301] APPLY: starting
0709 12:24:03.044007 (+   120us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:04.101389 (+1057382us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:04.110644 (+  9255us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:04.739592 (+628948us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:04.740045 (+   453us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=50,key_file_lookups=50,delta_file_lookups=0,mrs_lookups=0
0709 12:24:04.740109 (+    64us) write_op.cc:312] APPLY: finished
0709 12:24:04.800128 (+ 60019us) log.cc:844] Serialized 219 byte log entry
0709 12:24:04.946399 (+146271us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:04.973314 (+ 26915us) write_op.cc:454] Released schema lock
0709 12:24:04.986128 (+ 12814us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:04.986719 (+   591us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:04.986762 (+    43us) write_op.cc:454] Released schema lock
0709 12:24:05.699307 (+712545us) spinlock_profiling.cc:243] Waited 38.9 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
Metrics: {"mutex_wait_us":888586,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":239010,"apply.run_cpu_time_us":91155,"apply.run_wall_time_us":1942769,"cfile_cache_hit":102,"cfile_cache_hit_bytes":108727,"num_ops":25,"prepare.queue_time_us":447384,"prepare.run_cpu_time_us":52370,"prepare.run_wall_time_us":223042,"raft.queue_time_us":315834,"raft.run_cpu_time_us":8126,"raft.run_wall_time_us":199573,"replication_time_us":734631,"spinlock_wait_cycles":110229120,"thread_start_us":746059,"threads_started":5,"wal-append.queue_time_us":371431,"wal-append.run_cpu_time_us":60198,"wal-append.run_wall_time_us":3225727}]]}
I20260709 12:24:11.889743 21399 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:11.860273 21648 consensus_queue.cc:1048] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:24:11.913996 17144 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:11.889952 21587 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:11.916796 21633 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:11.853686 21636 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:11.866705 21584 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:11.887629 21656 raft_consensus.cc:697] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:11.926862 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:11.930107 21498 raft_consensus.cc:2749] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:11.932631 21648 raft_consensus.cc:2804] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:11.932992 16400 pending_rounds.cc:85] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 8
I20260709 12:24:11.933389 16400 pending_rounds.cc:107] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 12:24:11.933491 21648 raft_consensus.cc:697] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:11.937170 21584 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:11.938813 21611 consensus_queue.cc:260] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:11.940966 21611 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:11.939193 21672 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:11.941327 16442 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:11.941881 21696 consensus_queue.cc:1059] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:24:11.942175 16548 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:24:11.941974 17144 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:11.933634 16400 raft_consensus.cc:2889] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:11.943338 21250 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:11.925823 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 58623.3us
I20260709 12:24:11.943756 21648 consensus_queue.cc:237] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:11.945291 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:11.957409 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:11.905028 21704 raft_consensus.cc:697] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:24:11.960310 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 74882.3us
I20260709 12:24:11.961113 21691 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.962044 17144 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:11.963249 21633 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:11.964010 21601 raft_consensus.cc:2749] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:24:11.948091 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 129752us
I20260709 12:24:11.965855 16401 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:24:11.969834 16403 raft_consensus.cc:2368] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 13. Current term is 14.
I20260709 12:24:11.970865 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:11.973070 16548 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:24:11.973631 16396 raft_consensus.cc:1240] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:11.974264 17143 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:11.945696 21522 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:11.976679 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:11.969668 21691 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:11.978483 16442 leader_election.cc:400] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 13. Current term is 14.
I20260709 12:24:11.981135 21523 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:11.981758 17144 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:11.989241 16442 leader_election.cc:403] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:24:11.989185 16224 catalog_manager.cc:5697] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 5 to 8, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 8 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:11.973862 21712 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:11.971638 21633 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:12.013572 21633 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:12.014227 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:24:12.014496 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 114025us
I20260709 12:24:12.015435 16441 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:11.999377 21601 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:12.003708 21426 raft_consensus.cc:3055] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:11.991283 21523 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:12.006044 16367 leader_election.cc:400] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17.
I20260709 12:24:12.020735 16367 leader_election.cc:403] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:24:12.006413 21691 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:12.007702 21727 log.cc:927] Time spent T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 12:24:12.009378 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:12.010988 17144 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:12.011475 21515 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:12.012437 21655 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:12.031329 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:24:12.023463 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.020835 16224 catalog_manager.cc:5697] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 9 to 10, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 10 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:24:12.025763 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:12.028777 17144 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:11.970139 21478 consensus_queue.cc:1048] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 12:24:12.033460 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:12.033694 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:12.034106 16546 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:12.034686 16550 raft_consensus.cc:2393] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 18: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
W20260709 12:24:12.035259 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 55025.8us
I20260709 12:24:12.037900 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:12.036151 17144 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:12.033321 21696 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:12.040709 21675 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:12.041213 21675 raft_consensus.cc:697] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:12.042042 21675 consensus_queue.cc:237] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:12.044431 21524 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:12.020072 21270 consensus_queue.cc:1048] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 12:24:12.048708 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:12.049834 16543 raft_consensus.cc:2393] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 14: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:12.051826 16401 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
I20260709 12:24:12.052480 21656 consensus_queue.cc:237] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:12.056447 21692 consensus_meta.cc:287] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.739s	user 0.013s	sys 0.005s
I20260709 12:24:12.057245 16399 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:24:12.023650 21707 consensus_queue.cc:237] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:12.062841 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 58516.6us
I20260709 12:24:12.063501 16547 raft_consensus.cc:2393] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 17: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:12.022646 21426 raft_consensus.cc:740] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:12.062304 16442 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:12.064775 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.025096 21523 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:12.072752 16224 catalog_manager.cc:5697] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 6 to 12, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 12 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:12.078444 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:12.036808 21674 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:12.080408 16477 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:12.052227 21696 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17.
I20260709 12:24:12.064646 21691 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:12.088567 21691 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:12.088280 21426 consensus_queue.cc:260] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:12.089003 21691 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:12.090509 21587 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:12.087644 17258 raft_consensus.cc:1240] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:12.074600 21515 raft_consensus.cc:697] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:12.096688 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:12.098186 21636 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:12.121461 21633 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:12.098459 21672 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:12.122190 21522 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:12.123566 21633 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:12.109531 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:12.124882 16472 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:12.098251 21730 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:12.099850 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:12.126549 17266 raft_consensus.cc:2393] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 18: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:12.081471 21674 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:12.103298 21426 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:12.103993 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:12.035899 21704 consensus_queue.cc:237] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:12.081486 21655 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:12.105713 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:12.106055 21601 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 13. Current term is 14.
I20260709 12:24:12.106600 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:12.114583 17269 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:24:12.119448 21399 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:12.120735 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:12.160357 17254 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
I20260709 12:24:12.102667 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:12.120752 16367 leader_election.cc:400] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17.
I20260709 12:24:12.165524 16542 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 17.
W20260709 12:24:12.171651 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 109415us
I20260709 12:24:12.173063 16224 catalog_manager.cc:5697] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 8 to 11, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 11 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:12.152624 21674 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:12.162195 17261 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:24:12.176563 16477 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 18.
I20260709 12:24:12.177516 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:12.183230 16547 raft_consensus.cc:1240] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:12.184751 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:12.180452 17270 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 17.
I20260709 12:24:12.177016 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:24:12.186655 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 66066.5us
W20260709 12:24:12.187633 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 54057us
I20260709 12:24:12.182473 21633 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.197045 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:12.197634 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:12.200842 16401 raft_consensus.cc:2393] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 18: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:12.203078 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:12.180598 21587 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:12.212106 16516 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:12.213891 17269 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 15.
I20260709 12:24:12.215941 21738 consensus_queue.cc:1059] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 12:24:12.180318 21730 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:12.220371 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:12.223016 16546 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
W20260709 12:24:12.228659 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.218880 21730 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:12.232048 21712 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:12.233147 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:12.233625 21399 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:12.234118 17254 raft_consensus.cc:2393] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 17: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:12.234684 21399 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:12.217664 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:12.236282 16514 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:12.241459 21515 consensus_queue.cc:237] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:12.243455 21734 log.cc:927] Time spent T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 12:24:12.184530 21655 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:12.245091 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 53422.4us
W20260709 12:24:12.247107 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 44816.2us
I20260709 12:24:12.270074 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:12.271171 16542 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:12.251699 21712 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:12.276095 21712 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:12.276242 17261 raft_consensus.cc:1240] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:12.276858 21712 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:12.252278 21399 raft_consensus.cc:697] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:12.250888 21523 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:12.282366 21523 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:12.261246 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:12.202402 21696 consensus_queue.cc:1059] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
W20260709 12:24:12.262199 21737 log.cc:927] Time spent T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
I20260709 12:24:12.263626 16399 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 17.
I20260709 12:24:12.244575 21633 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:12.264163 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:12.270119 16367 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:12.273392 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:12.304435 17270 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:12.291673 16472 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
W20260709 12:24:12.295653 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 259547us
I20260709 12:24:12.216280 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:12.307521 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:12.307726 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:12.316644 17255 raft_consensus.cc:2393] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 14: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:12.283802 21584 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:12.321352 17266 raft_consensus.cc:1240] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:12.308274 17258 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14.
I20260709 12:24:12.283568 21696 raft_consensus.cc:3055] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:12.321419 16514 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:12.322331 16516 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:12.342943 17143 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:12.284009 21611 consensus_queue.cc:1048] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:24:12.351032 16542 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
W20260709 12:24:12.354692 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 84664.2us
I20260709 12:24:12.352908 17143 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:12.328287 21696 raft_consensus.cc:740] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:12.356475 17144 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:12.358561 21648 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:12.362748 21648 raft_consensus.cc:697] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:12.396709 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:12.341044 21692 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:12.350103 21707 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:12.354316 21584 raft_consensus.cc:697] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 18 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:12.410732 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:12.411995 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 64665.6us
W20260709 12:24:12.408075 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 89058.5us
I20260709 12:24:12.415678 21541 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:12.413213 21648 consensus_queue.cc:237] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:12.416225 21655 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:12.417348 21270 consensus_queue.cc:1048] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:12.417799 21027 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:12.418445 21027 raft_consensus.cc:697] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:12.419342 21692 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:12.407426 21730 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:12.420400 21743 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:12.420954 21730 raft_consensus.cc:697] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:12.421025 21743 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:12.421620 21541 raft_consensus.cc:697] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 17 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:12.422068 21730 consensus_queue.cc:237] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:12.422698 21541 consensus_queue.cc:237] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:12.423941 21741 raft_consensus.cc:3055] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:12.424434 21741 raft_consensus.cc:740] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:12.424263 21696 consensus_queue.cc:260] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:12.425557 21741 consensus_queue.cc:260] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:12.426294 16515 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:12.426882 21741 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:12.427600 16515 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:12.419556 21027 consensus_queue.cc:237] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:12.428640 21236 consensus_queue.cc:1059] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:12.430173 16515 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:12.413539 16367 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:12.445338 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 86398.7us
I20260709 12:24:12.423336 21743 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:12.460613 21711 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:12.447676 21270 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:12.461458 21711 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:12.444105 21399 consensus_queue.cc:237] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:12.463778 21711 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:12.470613 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:12.425722 21696 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:24:12.485077 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 136242us
I20260709 12:24:12.491217 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:12.443771 21250 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:12.487326 16367 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:12.506982 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 77056.6us
I20260709 12:24:12.508081 16515 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:12.472098 21726 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:12.470561 21711 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:24:12.519749 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 201628us
I20260709 12:24:12.443882 21524 raft_consensus.cc:2764] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:24:12.502142 21250 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:12.519599 21726 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:12.544409 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 495267us
W20260709 12:24:12.545734 16366 leader_election.cc:336] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.528s (SENT)
I20260709 12:24:12.537989 21250 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:12.567348 16515 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:12.546622 21639 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:12.586995 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.544169 21726 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:12.552287 21711 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:12.601480 21748 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:12.601934 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:12.628623 17270 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14.
W20260709 12:24:12.602336 21751 log.cc:927] Time spent T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 12:24:12.602722 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:12.602898 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:12.630777 16401 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:12.603470 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:12.603799 21524 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:24:12.638794 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 151538us
I20260709 12:24:12.606914 21675 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:12.608558 21522 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:12.609292 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:12.610389 21747 log.cc:927] Time spent T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:24:12.587870 21633 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:12.618660 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:12.619760 21426 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:12.621198 17263 raft_consensus.cc:1240] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:24:12.565387 21584 consensus_queue.cc:237] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:12.624471 21692 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:12.624570 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 87688us
I20260709 12:24:12.546087 21672 consensus_queue.cc:1059] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 12:24:12.555402 21523 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:12.638720 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:24:12.741326 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.741641 16396 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:24:12.605499 21731 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:12.640738 16441 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:12.648978 16400 raft_consensus.cc:2393] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 17: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:12.550494 21674 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:12.654237 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:12.616797 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:12.760244 17254 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:12.626045 21611 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:24:12.739740 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.743681 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:12.751545 16401 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
I20260709 12:24:12.766585 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:12.746951 21691 consensus_meta.cc:287] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.657s	user 0.011s	sys 0.002s
W20260709 12:24:12.769330 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 278821us
I20260709 12:24:12.633297 21712 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:12.638453 21707 raft_consensus.cc:697] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:12.628800 21639 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:12.776808 17264 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 17: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:12.759871 21692 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.797652 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:12.679467 21524 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:12.739089 21522 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:12.804661 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:12.660136 21748 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:12.808477 16549 raft_consensus.cc:2368] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 17. Current term is 18.
I20260709 12:24:12.798424 17144 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:12.813941 17144 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:12.816493 17144 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:12.747292 21675 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:12.818559 16547 raft_consensus.cc:1240] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:24:12.823585 17144 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:12.824640 17144 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:12.826079 17266 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:24:12.827797 16545 raft_consensus.cc:1240] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:12.773053 21523 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:12.777261 21691 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:12.772563 21731 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:12.812786 21762 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:12.828428 21672 raft_consensus.cc:3055] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:12.812978 21692 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:12.875715 21674 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:12.876134 21757 log.cc:927] Time spent T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 12:24:12.877898 21752 log.cc:927] Time spent T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
I20260709 12:24:12.879290 21648 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:24:12.879591 21758 log.cc:927] Time spent T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:24:12.825368 16549 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "025780567e194437bc14e26465b690e3"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:24:12.855595 21672 raft_consensus.cc:740] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:12.880730 21712 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:12.861685 21426 raft_consensus.cc:697] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 17 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:12.881263 21524 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:12.881099 21672 consensus_queue.cc:260] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:12.881796 21675 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:12.882390 21672 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:12.881983 21426 consensus_queue.cc:237] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:12.883692 21639 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:12.870136 21730 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:12.880008 21648 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:12.859062 21754 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:24:12.891161 17254 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 18.
I20260709 12:24:12.892802 16398 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
I20260709 12:24:12.870394 21770 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:24:12.934489 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:12.960503 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 138535us
W20260709 12:24:12.970974 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 124798us
W20260709 12:24:12.974131 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 149496us
I20260709 12:24:12.857151 21762 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:12.958527 21758 log.cc:927] Time spent T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:24:12.954367 21648 raft_consensus.cc:697] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:12.984210 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:12.988503 17263 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:12.986243 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:12.989229 16394 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:12.886641 21707 consensus_queue.cc:237] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:13.001013 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:12.996551 21741 consensus_meta.cc:287] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.569s	user 0.014s	sys 0.000s
I20260709 12:24:13.004747 21611 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.010051 21611 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:13.010778 21761 log.cc:927] Time spent T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 12:24:13.023078 21781 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.023779 21781 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:13.026645 21781 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:13.012159 21611 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:13.033493 16367 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:13.055459 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 47284.9us
I20260709 12:24:13.056818 16367 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:13.058905 16396 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
I20260709 12:24:13.059873 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:13.066419 21692 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:13.069965 21270 raft_consensus.cc:3055] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:13.070611 21270 raft_consensus.cc:740] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:13.071623 21270 consensus_queue.cc:260] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:13.073657 21270 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:13.064697 21691 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:13.082906 21773 log.cc:927] Time spent T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 12:24:13.083932 16547 raft_consensus.cc:1275] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:24:13.085080 21762 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:13.050257 21236 consensus_queue.cc:1059] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 12:24:13.085956 21498 raft_consensus.cc:3055] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:13.086369 21498 raft_consensus.cc:740] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:13.087330 21498 consensus_queue.cc:260] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:13.088934 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:13.089370 21498 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:13.089545 16551 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:24:13.000937 21648 consensus_queue.cc:237] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:13.065234 16396 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
I20260709 12:24:13.097723 17263 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:24:13.100602 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:13.101293 16396 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 17: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:13.079864 21250 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:13.105825 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:13.065095 21707 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:13.107745 16551 raft_consensus.cc:1275] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:24:13.109709 16396 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 17.
I20260709 12:24:13.113086 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:13.113624 16396 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
I20260709 12:24:13.102154 21731 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:13.116660 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 59929.6us
I20260709 12:24:13.118964 21764 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:13.121052 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:13.122237 16367 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:13.109259 21707 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:13.123109 16396 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 16: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:13.136473 16399 raft_consensus.cc:1275] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:24:13.133040 17144 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:13.135108 16394 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:24:13.141029 17144 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:13.108973 21726 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:13.150707 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 506404us
I20260709 12:24:13.152040 21782 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:24:13.152685 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.153404 16475 raft_consensus.cc:3055] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:24:13.153518 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.153815 16475 raft_consensus.cc:740] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:13.154639 16475 consensus_queue.cc:260] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:13.155241 16477 raft_consensus.cc:3055] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:13.155632 16477 raft_consensus.cc:740] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:13.156358 16477 consensus_queue.cc:260] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:13.157661 16477 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:13.141775 21731 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:13.159327 16469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 1034) took 1133 ms. Trace:
I20260709 12:24:13.160629 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:13.164783 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:13.165644 21250 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:13.166234 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:13.166337 21696 consensus_meta.cc:287] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.669s	user 0.012s	sys 0.001s
W20260709 12:24:13.167232 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.167502 21788 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:13.168202 16403 raft_consensus.cc:1275] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 12:24:13.168730 16403 pending_rounds.cc:85] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 7
I20260709 12:24:13.168998 16403 pending_rounds.cc:107] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:24:13.169299 16403 raft_consensus.cc:2889] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:13.170919 21478 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:24:13.181932 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 59729.3us
I20260709 12:24:13.155632 16475 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:13.173440 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:13.180548 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.188019 21726 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:13.183032 16367 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:13.137277 21707 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:24:13.155917 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 529608us
I20260709 12:24:13.191495 16472 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:13.184548 16473 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:13.197131 16367 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:13.195204 21782 consensus_queue.cc:1048] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 12:24:13.213811 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:13.214939 16471 raft_consensus.cc:2393] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 15: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:13.215503 16367 leader_election.cc:400] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 17. Current term is 18.
I20260709 12:24:13.215859 16367 leader_election.cc:403] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:13.217638 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:13.219211 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:13.220122 16546 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:13.201821 21696 consensus_queue.cc:1059] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 12:24:13.225436 21738 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:13.226342 21738 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:13.228309 16441 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:13.192361 21731 consensus_queue.cc:1048] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 12:24:13.195386 21726 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:13.234879 21726 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:13.235380 21726 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:13.240566 21738 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:13.241598 21738 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:13.243451 21691 consensus_queue.cc:1048] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:13.251003 16396 raft_consensus.cc:1275] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:24:13.255016 16477 raft_consensus.cc:1275] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:24:13.256124 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:13.262529 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.264034 16546 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:24:13.274828 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:13.283067 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:13.284132 16472 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:24:13.298517 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:13.287806 16367 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:13.314380 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:13.254348 21672 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:13.248265 21741 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:13.320511 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:13.256457 21764 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 17. Current term is 18.
I20260709 12:24:13.322847 16399 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:24:13.259306 21426 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:13.324069 16514 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:13.325667 21692 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:24:13.352886 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 121744us
I20260709 12:24:13.304622 21731 consensus_queue.cc:1048] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
W20260709 12:24:13.370385 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.337303 21672 raft_consensus.cc:697] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 14 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:13.345077 21250 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:13.337075 21741 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:13.417233 21672 consensus_queue.cc:237] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:13.417753 21478 consensus_queue.cc:1048] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:13.418584 16441 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:13.419590 21744 log.cc:927] Time spent T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 1.006s	user 0.011s	sys 0.000s
W20260709 12:24:13.424109 21789 log.cc:927] Time spent T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.282s	user 0.001s	sys 0.001s
W20260709 12:24:13.425654 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.340653 21426 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:13.365796 21696 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.455991 16399 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
I20260709 12:24:13.388767 21692 raft_consensus.cc:697] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:13.391571 21250 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.458071 16442 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:13.465147 16549 raft_consensus.cc:606] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 18 LEADER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:13.465903 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:13.466902 17263 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 18.
I20260709 12:24:13.468447 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:13.472981 16551 raft_consensus.cc:1275] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:24:13.478349 16545 raft_consensus.cc:1275] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:24:13.479526 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:13.480106 17263 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:13.462327 21250 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:13.483383 16546 raft_consensus.cc:1275] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 12:24:13.484863 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 66396.6us
I20260709 12:24:13.489390 21783 raft_consensus.cc:3055] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:13.489890 21783 raft_consensus.cc:740] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:13.159719 16469 rpcz_store.cc:276] 0709 12:24:12.025893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.025997 (+   104us) service_pool.cc:224] Handling call
0709 12:24:12.026286 (+   289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:12.212631 (+186345us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:12.212671 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:12.235918 (+ 23247us) log.cc:844] Serialized 26 byte log entry
0709 12:24:12.543890 (+307972us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:12.563676 (+ 19786us) log.cc:844] Serialized 14 byte log entry
0709 12:24:13.148893 (+585217us) spinlock_profiling.cc:243] Waited 900 us on lock 0x7b180024e9e0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d16c0a 00007f287c456219 00007f2872d9ea40 00007f2872d994e4 00007f2872d99455 00007f2875e81060 00007f2875e93716 00007f2875ea4287 00007f2875ea4208
0709 12:24:13.149162 (+   269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.149258 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:13.149319 (+    61us) raft_consensus.cc:1685] finished
0709 12:24:13.159089 (+  9770us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:13.159236 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29133,"prepare.queue_time_us":20111,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":4946,"raft.queue_time_us":726146,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":6914,"spinlock_wait_cycles":2070272,"thread_start_us":346312,"threads_started":2,"wal-append.queue_time_us":459198}
I20260709 12:24:13.499991 16547 raft_consensus.cc:1275] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:24:13.500866 16548 raft_consensus.cc:1275] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:24:13.492120 16475 raft_consensus.cc:1275] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:24:13.483398 17143 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:13.471545 21738 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:13.505656 17254 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:24:13.469216 21696 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:13.515162 16367 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:13.518234 21762 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:13.518836 21762 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.519274 21762 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:13.505213 21764 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.517920 21777 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:13.474829 21730 consensus_queue.cc:1048] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:24:13.500883 21800 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:13.538691 17263 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:24:13.543570 16441 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:13.490831 21783 consensus_queue.cc:260] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:13.543362 21777 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:13.558116 21522 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:13.558601 21777 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:13.514420 21738 raft_consensus.cc:697] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:13.560474 21800 raft_consensus.cc:697] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:24:13.569839 21794 log.cc:927] Time spent T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:24:13.572417 21692 consensus_queue.cc:237] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:13.556900 21782 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.576588 21601 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:13.576639 21782 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:13.577402 21601 raft_consensus.cc:697] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:13.578400 21601 consensus_queue.cc:237] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:13.579348 21782 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:13.580819 21796 log.cc:927] Time spent T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.211s	user 0.001s	sys 0.000s
I20260709 12:24:13.585947 21236 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.586737 21236 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:13.591001 21236 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:13.591132 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:13.571170 21800 consensus_queue.cc:237] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:13.524369 21523 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:13.517019 21731 consensus_queue.cc:1048] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
W20260709 12:24:13.601465 21798 log.cc:927] Time spent T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 12:24:13.607774 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:13.613407 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:13.615379 16547 raft_consensus.cc:2393] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 16: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:13.527300 21774 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.079s
I20260709 12:24:13.542600 21764 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:13.628533 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.608055 21524 consensus_queue.cc:1048] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
W20260709 12:24:13.634043 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 117359us
I20260709 12:24:13.635278 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:13.638391 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:13.640569 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.607872 21784 consensus_queue.cc:1048] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:24:13.611799 21782 consensus_queue.cc:1048] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 12:24:13.598486 21777 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:13.618366 21523 consensus_queue.cc:1048] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 12:24:13.626607 21587 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:13.631229 21741 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:13.669428 16516 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:13.674469 16516 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:13.656934 21777 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:13.661612 21782 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:13.663771 21587 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:13.677587 21541 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:13.688391 21775 consensus_queue.cc:1048] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.081s
I20260709 12:24:13.679781 21426 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:13.695650 16516 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:13.706879 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.685463 21236 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:13.694229 21541 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.659889 21738 consensus_queue.cc:237] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:13.706087 21696 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:13.685791 21782 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:13.714105 21236 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:13.752640 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.758749 16441 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:13.742913 21270 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.739225 21541 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:13.760836 21236 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:13.754114 21807 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:13.781011 21764 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:13.781605 21587 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:13.780718 21750 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:13.782552 21750 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:13.783010 21750 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:13.783903 21539 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:13.774183 21270 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:13.784562 21539 raft_consensus.cc:697] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:13.785707 21539 consensus_queue.cc:237] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:13.786458 21270 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:13.780666 21807 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:13.789100 21807 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:13.789633 21250 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:13.790544 21726 consensus_meta.cc:287] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.555s	user 0.012s	sys 0.000s
I20260709 12:24:13.791072 21726 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:13.792354 16396 raft_consensus.cc:1240] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:13.793641 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:13.794343 16469 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:24:13.792186 21250 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:13.807037 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:13.808377 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:13.810354 16469 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 18.
W20260709 12:24:13.812529 21806 log.cc:927] Time spent T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 12:24:13.811757 21750 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:13.813184 16514 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:13.814281 21027 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:13.814795 21027 raft_consensus.cc:697] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 18 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:13.815699 21027 consensus_queue.cc:237] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:13.818763 21750 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:13.821986 21726 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:13.826046 16549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:59590 (request call id 21) took 1013 ms. Trace:
I20260709 12:24:13.826573 16549 rpcz_store.cc:276] 0709 12:24:12.818263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.825241 (+  6978us) service_pool.cc:224] Handling call
0709 12:24:13.825801 (+1000560us) spinlock_profiling.cc:243] Waited 584 ms on lock 0x7b5400a4f454. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e83853 00007f287ae5160a 00007f28743d77d4 00007f28743d7762 00007f28743d7691 00007f28743d760c 00007f28743d68c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:13.825917 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1344292736}
I20260709 12:24:13.830821 21783 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:13.834921 21750 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.835486 21750 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:13.835424 21762 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:13.838738 16394 raft_consensus.cc:1240] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
W20260709 12:24:13.840299 21707 consensus_meta.cc:287] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.587s	user 0.011s	sys 0.000s
I20260709 12:24:13.840493 21762 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:13.840586 21541 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:13.841365 21750 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:13.811691 16395 raft_consensus.cc:1275] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:24:13.821863 21782 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:13.862529 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:13.863402 16395 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:13.865171 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 129573us
I20260709 12:24:13.866637 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:13.872900 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:13.867457 16402 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
I20260709 12:24:13.873965 16549 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:24:13.874614 16549 raft_consensus.cc:606] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:24:13.875034 16549 raft_consensus.cc:614] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:13.848886 21707 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:13.884986 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:13.886045 17263 raft_consensus.cc:2368] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 17. Current term is 18.
I20260709 12:24:13.879698 16550 raft_consensus.cc:1240] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:13.869575 21236 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:13.879645 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:13.880028 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:13.882565 16548 raft_consensus.cc:1240] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:13.884188 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:13.883422 16546 raft_consensus.cc:1240] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:13.888723 17144 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:13.895394 16515 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:13.898880 16395 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 18.
I20260709 12:24:13.895350 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:13.901073 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:13.901947 16477 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 20.
I20260709 12:24:13.899690 21731 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:13.897457 17271 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:24:13.903786 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:13.905085 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:13.903004 16473 raft_consensus.cc:3055] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:13.906618 16473 raft_consensus.cc:740] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:13.910017 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:13.913980 16472 raft_consensus.cc:2393] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 15: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:13.905872 16471 raft_consensus.cc:3055] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:13.914752 16471 raft_consensus.cc:740] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:13.911249 16468 raft_consensus.cc:3055] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:13.898094 21522 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:13.921886 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:13.921763 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:13.920192 16468 raft_consensus.cc:740] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:13.915323 21775 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:13.930801 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:13.931977 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:13.905272 21731 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.932802 17266 raft_consensus.cc:2368] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 13. Current term is 14.
I20260709 12:24:13.940716 16550 raft_consensus.cc:1240] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:24:13.930214 16549 raft_consensus.cc:1240] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:13.908126 21787 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:13.949098 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:13.959923 17258 raft_consensus.cc:2393] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 14: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:13.934016 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:13.960955 17269 raft_consensus.cc:1240] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:13.931748 21784 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.961798 21784 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:13.930747 21730 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.968135 21730 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:13.963518 16366 leader_election.cc:336] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.808s (SENT)
I20260709 12:24:13.969199 16366 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:13.963482 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:13.967059 16543 raft_consensus.cc:1240] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:13.964447 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:13.931818 21522 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:13.966115 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:13.974661 16549 raft_consensus.cc:2393] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 15: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:13.968596 21707 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:13.940413 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:13.970952 16544 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:24:13.976495 17264 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:13.962653 21787 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:13.981212 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:13.971433 16468 consensus_queue.cc:260] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:13.983029 17255 raft_consensus.cc:1240] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
W20260709 12:24:13.983451 21824 log.cc:927] Time spent T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:24:13.990365 21522 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:13.991640 17255 raft_consensus.cc:1240] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:13.948498 21731 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:14.002398 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:14.003265 16469 raft_consensus.cc:2368] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 17. Current term is 18.
I20260709 12:24:14.002482 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:14.003896 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:14.005909 16544 raft_consensus.cc:2393] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 17: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:13.979372 21787 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:14.007004 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.007256 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:14.006124 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:14.010416 17142 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:13.933887 21775 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:14.005208 21250 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:14.014271 21250 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:14.014935 21541 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:14.015074 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:14.005760 21812 log.cc:927] Time spent T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.211s	user 0.000s	sys 0.003s
W20260709 12:24:14.019806 21801 log.cc:927] Time spent T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.527s	user 0.001s	sys 0.000s
I20260709 12:24:14.021032 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:24:14.022359 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:14.022799 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.016458 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:14.027869 17255 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:14.027835 17142 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:14.030566 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.033172 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.035386 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:14.038623 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:14.036854 17142 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:14.039777 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.038861 16469 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 17: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:14.041726 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.028025 21819 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.032928 21770 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:14.059301 21819 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:14.048272 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.035060 21515 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:14.071609 21515 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:14.072222 21515 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:14.073071 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:14.074664 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:14.060756 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:14.053184 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.068842 21524 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:14.125430 17264 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
W20260709 12:24:14.129488 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:14.076279 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:14.076658 16474 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:14.127813 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:14.143949 17264 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:24:14.151715 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:14.155093 17255 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
I20260709 12:24:14.157511 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.158631 16477 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 17.
I20260709 12:24:14.034797 21522 consensus_queue.cc:1048] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:24:14.161432 16514 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:14.044147 21648 consensus_queue.cc:1048] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.115s
I20260709 12:24:14.165272 17261 raft_consensus.cc:1275] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:24:14.167773 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:14.163395 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.177549 16477 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:14.183274 17255 raft_consensus.cc:1240] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:14.090054 21524 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:14.174330 21822 log.cc:927] Time spent T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:24:14.213615 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:14.185705 21541 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:14.223150 16477 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 17.
I20260709 12:24:14.235342 16474 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 21.
I20260709 12:24:14.238471 21748 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:14.196869 21524 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:14.281138 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:14.281525 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:14.282016 16474 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 17: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:14.242823 16469 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:24:14.215848 21236 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:14.290416 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.291416 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.225697 21541 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:14.291566 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.299736 16474 raft_consensus.cc:1240] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:14.300390 16477 raft_consensus.cc:1240] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 14. Current term is 15. Ops: []
W20260709 12:24:14.303257 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 241332us
I20260709 12:24:14.305115 16474 raft_consensus.cc:1275] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:24:14.307441 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.296512 21541 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:14.298077 21784 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:14.307305 21770 raft_consensus.cc:697] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:14.315183 17255 raft_consensus.cc:1240] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:24:14.305370 21730 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:14.320205 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:24:14.320624 21826 log.cc:927] Time spent T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 12:24:14.321045 21819 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:14.321107 17255 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:14.325403 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:14.326785 16474 raft_consensus.cc:2368] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 17. Current term is 18.
I20260709 12:24:14.327303 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.328261 16477 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
I20260709 12:24:14.351218 17255 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
I20260709 12:24:14.329900 16514 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:14.357620 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.314107 21236 consensus_queue.cc:1048] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:24:14.368659 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.377070 21711 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.377849 21711 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:14.380568 16515 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:14.384671 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.388702 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.387156 16470 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14.
I20260709 12:24:14.359844 21691 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.367183 21636 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:14.360662 21819 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.421736 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:14.422468 17255 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
I20260709 12:24:14.423488 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:14.424243 17255 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:24:14.425877 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.380252 21711 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:14.390754 21539 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:14.430887 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.421627 21636 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:14.426493 21811 log.cc:927] Time spent T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.632s	user 0.003s	sys 0.017s
I20260709 12:24:14.434509 16469 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:24:14.440155 17255 raft_consensus.cc:1275] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:24:14.447049 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.426961 21819 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:14.477021 21828 log.cc:927] Time spent T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 12:24:14.477383 21819 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:14.484066 16475 raft_consensus.cc:1275] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:24:14.454092 21775 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:14.437242 21601 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.438524 21636 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:14.499603 21837 log.cc:927] Time spent T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 12:24:14.502466 21839 log.cc:927] Time spent T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 12:24:14.439252 21633 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.504601 21633 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:14.505848 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:14.473430 21835 log.cc:927] Time spent T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.086s	user 0.001s	sys 0.001s
I20260709 12:24:14.473982 21539 raft_consensus.cc:697] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:14.507787 21601 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:14.507414 21539 consensus_queue.cc:237] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:14.434051 21770 consensus_queue.cc:237] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:14.424635 21691 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:14.513967 21731 consensus_meta.cc:287] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.505s	user 0.006s	sys 0.004s
I20260709 12:24:14.498792 17271 raft_consensus.cc:1275] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:24:14.525707 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.526757 21691 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:14.527099 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.528155 16475 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 17: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
W20260709 12:24:14.530673 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 101462us
I20260709 12:24:14.531680 16515 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:24:14.548617 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:14.550323 16515 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:14.551011 21844 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:14.551548 21844 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:14.552472 21539 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:14.552937 21539 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:14.553229 21539 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:14.555723 21270 consensus_queue.cc:1048] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:24:14.502914 21515 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:14.562193 21515 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:14.573772 16441 leader_election.cc:400] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 17. Current term is 18.
I20260709 12:24:14.574208 16441 leader_election.cc:403] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:24:14.575043 21750 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 17. Current term is 18.
I20260709 12:24:14.576211 16441 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:14.514483 21731 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:14.576874 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:14.578368 16475 raft_consensus.cc:2376] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 21. Re-sending same reply.
I20260709 12:24:14.581125 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.581732 16472 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 17.
I20260709 12:24:14.545835 21738 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:14.574271 21636 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:14.583165 16441 leader_election.cc:400] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 13. Current term is 14.
I20260709 12:24:14.551923 21844 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:14.584700 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.585155 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.587707 21539 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:14.588876 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.589854 16471 consensus_queue.cc:260] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:14.583231 16514 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:14.614068 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.606606 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.607563 21636 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:14.597271 17142 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:14.546284 21787 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:14.609879 21731 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:14.599118 21738 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:14.610488 16394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 1414) took 1407 ms. Trace:
I20260709 12:24:14.582383 21750 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:14.613821 16476 raft_consensus.cc:2376] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14. Re-sending same reply.
I20260709 12:24:14.652405 21819 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:14.619201 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.700634 16394 rpcz_store.cc:276] 0709 12:24:13.203381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.203516 (+   135us) service_pool.cc:224] Handling call
0709 12:24:13.617727 (+414211us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:13.839141 (+221414us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.609320 (+770179us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b5400bb80d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:14.610241 (+   921us) spinlock_profiling.cc:243] Waited 343 ms on lock 0x7b5400bb80d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:14.610394 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1297669760}
W20260709 12:24:14.701927 21838 log.cc:927] Time spent T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.275s	user 0.002s	sys 0.000s
W20260709 12:24:14.702414 21816 log.cc:927] Time spent T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.907s	user 0.000s	sys 0.003s
I20260709 12:24:14.703372 16472 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14.
I20260709 12:24:14.703933 21539 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:14.632902 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:14.618227 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:14.703787 21750 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:14.707607 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:14.708581 16476 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:14.706588 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 109399us
I20260709 12:24:14.665017 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:14.707139 16474 raft_consensus.cc:2376] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 17. Re-sending same reply.
I20260709 12:24:14.710676 17142 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:14.710949 16475 raft_consensus.cc:2376] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16. Re-sending same reply.
I20260709 12:24:14.716501 16551 raft_consensus.cc:1240] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:14.652607 21755 consensus_queue.cc:1059] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.092s
I20260709 12:24:14.720671 21755 consensus_queue.cc:1243] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Peer ef57beab0e1d40d585983873c55107e8 log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:24:14.706277 21524 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:14.707710 21522 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:14.722158 21787 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:14.709764 21750 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:14.728395 21522 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:14.714224 21833 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.659499 21270 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:24:14.729043 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 136950us
I20260709 12:24:14.705866 21541 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:14.724434 16476 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:24:14.744067 21857 consensus_queue.cc:1059] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:14.745148 16468 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:14.731817 21833 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:14.750669 21833 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:14.732621 21763 raft_consensus.cc:993] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1: : Instructing follower 52ab50b0237346c1a13ede947f6a63c2 to start an election
W20260709 12:24:14.756507 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 792950us
I20260709 12:24:14.756850 21762 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:14.739295 21731 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.757860 21731 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:14.737640 21270 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:14.760665 21270 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:14.760900 21539 raft_consensus.cc:1081] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 18 LEADER]: Signalling peer 52ab50b0237346c1a13ede947f6a63c2 to start an election
I20260709 12:24:14.761515 21783 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:14.762015 21783 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:14.764000 21783 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:14.765069 21764 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.765576 21764 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:14.766686 21776 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:14.766925 16395 raft_consensus.cc:1275] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:24:14.767228 21776 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:14.768934 21601 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:14.769553 21776 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:14.770355 21776 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:14.760077 21731 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:14.700657 21819 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:14.767620 21764 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:14.772532 16468 raft_consensus.cc:1275] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:24:14.773506 16543 raft_consensus.cc:1240] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:14.773118 16514 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:14.776003 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:14.776746 16400 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
I20260709 12:24:14.777781 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:14.730278 21524 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:14.779019 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:14.778920 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:14.781596 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:14.790511 16395 raft_consensus.cc:2393] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 19: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:14.792282 16441 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:14.789119 16396 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
I20260709 12:24:14.794726 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:14.795207 16398 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 16: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:14.808923 21778 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:14.809566 21778 raft_consensus.cc:697] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:14.809965 21541 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:14.810508 21778 consensus_queue.cc:237] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:14.732720 21844 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:14.813985 21844 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:14.778357 21776 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:14.817730 21819 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:14.818121 21776 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:14.820223 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:14.826879 16471 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:14.828349 17143 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:14.848974 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:14.851353 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:14.852242 16399 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:14.862828 17144 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:14.865000 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:14.865134 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:14.865669 17143 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:14.865916 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:14.871861 17144 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:14.872952 16401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 1542) took 1788 ms. Trace:
I20260709 12:24:14.794224 16468 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "025780567e194437bc14e26465b690e3"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:34826
I20260709 12:24:14.876763 16468 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:14.877287 16468 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:14.874089 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:14.826362 21764 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.882148 16473 consensus_queue.cc:260] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:14.889148 17144 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:14.892557 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:14.893862 17143 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:14.826125 21707 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:14.873404 16401 rpcz_store.cc:276] 0709 12:24:13.084749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.165737 (+ 80988us) service_pool.cc:224] Handling call
0709 12:24:13.166094 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:13.845450 (+679356us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:24:13.920852 (+ 75402us) log.cc:844] Serialized 14 byte log entry
0709 12:24:14.701440 (+780588us) spinlock_profiling.cc:243] Waited 1.78 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:24:14.779828 (+ 78388us) log.cc:844] Serialized 14 byte log entry
0709 12:24:14.779979 (+   151us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:14.780018 (+    39us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:14.780063 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.872746 (+ 92683us) spinlock_profiling.cc:243] Waited 679 ms on lock 0x7b54009bd7d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:14.872790 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.872900 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":480629,"prepare.queue_time_us":104094,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":72,"raft.queue_time_us":482227,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":131,"spinlock_wait_cycles":1565909632,"thread_start_us":450130,"threads_started":2,"wal-append.queue_time_us":446175}
I20260709 12:24:14.849231 21819 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:14.904435 16551 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:14.853120 21776 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:14.905393 21858 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:14.905251 21776 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:14.906175 16394 raft_consensus.cc:1275] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:24:14.906966 21522 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:14.906828 21858 consensus_queue.cc:1059] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:24:14.907325 21776 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:14.908070 21750 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:14.909075 21785 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:14.910852 16471 raft_consensus.cc:1275] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 12:24:14.911216 16471 pending_rounds.cc:85] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 10
I20260709 12:24:14.911489 16471 pending_rounds.cc:107] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 12:24:14.911021 21764 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:14.911720 16471 raft_consensus.cc:2889] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:14.912501 21707 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:14.912722 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:14.913513 21764 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:14.912722 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:14.914630 21707 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:14.914839 21270 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:14.915307 21782 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:14.915432 16471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 1053) took 1022 ms. Trace:
I20260709 12:24:14.915822 21782 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:14.889098 21730 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:14.916303 21782 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:14.916640 21730 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:14.888044 21784 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:14.917222 21730 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:14.917510 21784 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:14.918008 21784 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:14.918547 16398 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:24:14.919828 21819 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:14.921736 21515 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:14.918593 21236 consensus_queue.cc:1048] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 12:24:14.923642 16403 raft_consensus.cc:1275] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:24:14.898494 21639 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.925026 21639 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:14.913061 21856 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.926388 21856 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:14.928537 16544 raft_consensus.cc:1240] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:14.899279 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:14.929790 16546 raft_consensus.cc:2393] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 14: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:14.934319 21633 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:14.935128 21819 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:14.941097 21270 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:14.915887 16471 rpcz_store.cc:276] 0709 12:24:13.893531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.893688 (+   157us) service_pool.cc:224] Handling call
0709 12:24:13.893990 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:14.589499 (+695509us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b4c000e55c0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874155a20 00007f2874166a29 00007f287412c644 00007f2874153f2d 00007f2874153b93 00007f2875e8418b 00007f2875e84cb5 00007f2875e7ff30 00007f2875e817de 00007f2875e8bb49 00007f2875e8c416
0709 12:24:14.826454 (+236955us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b58004a09d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875de50aa 00007f2875e67447 00007f2875e7ffe7 00007f2875e817de 00007f2875e8bb49 00007f2875e8c416 00007f2875e88d8f 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4
0709 12:24:14.912303 (+ 85849us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.915379 (+  3076us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":266545664}
W20260709 12:24:14.943714 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 49979.8us
I20260709 12:24:14.944664 17143 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:14.945827 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:14.946218 16399 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:14.946848 16476 raft_consensus.cc:2393] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 13: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:14.948637 17143 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:14.949981 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:14.950317 16397 raft_consensus.cc:1275] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:24:14.950546 17143 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:14.950568 16472 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:14.957064 16468 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:14.963121 16468 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:14.953120 21587 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:14.866916 21731 consensus_queue.cc:1048] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.081s
I20260709 12:24:14.916458 21820 raft_consensus.cc:3055] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:14.978976 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:14.985565 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 210993us
I20260709 12:24:14.988602 17271 raft_consensus.cc:1275] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 12:24:14.990314 16366 leader_election.cc:336] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.843s (SENT)
I20260709 12:24:14.966077 21785 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:14.991000 21639 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:14.991514 21856 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:14.951313 21270 consensus_queue.cc:1048] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:24:14.993305 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:14.994877 21730 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:14.988744 21524 consensus_queue.cc:1048] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 12:24:14.985956 21250 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:14.996599 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:14.997841 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:14.998627 17255 raft_consensus.cc:2393] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 16: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:15.001950 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:15.003382 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:15.004482 16551 raft_consensus.cc:2393] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 14: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:14.983275 21820 raft_consensus.cc:740] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:15.008952 21820 consensus_queue.cc:260] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:15.011001 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:14.959059 21633 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:15.027868 16476 raft_consensus.cc:1275] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 12:24:15.038621 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 48338.6us
I20260709 12:24:15.039762 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:15.035869 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:15.042261 16551 raft_consensus.cc:2410] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:24:14.941756 21738 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:15.049285 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:15.050828 16366 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:15.057862 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:15.055387 21775 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.070570 17258 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 12:24:15.069605 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:15.071761 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:15.071841 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:15.072638 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:15.073958 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:15.070263 21775 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:15.074990 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:15.074819 21782 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:15.075680 17269 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:24:15.047101 21777 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.076203 21777 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:15.076781 21775 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:15.073969 21840 log.cc:927] Time spent T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.638s	user 0.005s	sys 0.000s
I20260709 12:24:15.078320 21777 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:15.066565 21270 consensus_queue.cc:1048] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 12:24:15.080278 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:15.081998 16473 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:15.081871 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:15.082835 16398 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 12.
I20260709 12:24:15.078018 17255 raft_consensus.cc:2393] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 13: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:15.004103 21834 consensus_queue.cc:1059] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:24:15.084789 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:15.085516 21834 raft_consensus.cc:3055] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:15.085951 17270 raft_consensus.cc:1240] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:15.087006 17261 raft_consensus.cc:1275] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:24:15.087467 17270 raft_consensus.cc:1275] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:24:15.055660 21522 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:15.089553 17144 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:15.093111 17254 raft_consensus.cc:1240] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:24:15.095918 17258 raft_consensus.cc:2393] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 19: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:15.106989 17143 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:15.129146 16399 raft_consensus.cc:1275] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:24:15.085952 21834 raft_consensus.cc:740] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:15.139323 17264 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:15.138936 21834 consensus_queue.cc:260] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:15.140590 17261 raft_consensus.cc:1240] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
W20260709 12:24:15.160422 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 109691us
I20260709 12:24:15.161489 17264 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 17.
I20260709 12:24:15.163462 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:15.151470 16472 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 18.
I20260709 12:24:15.164144 17271 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:24:15.142709 21648 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:15.155505 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:15.165683 16514 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:15.170043 16398 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:15.149472 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 303380us
I20260709 12:24:15.177696 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:15.100445 21692 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:24:15.186692 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.121322 21775 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:15.197316 16441 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:15.203354 16441 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:15.205482 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:15.206554 16476 raft_consensus.cc:2410] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:24:15.122102 21778 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:15.209611 17263 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:15.211716 16441 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:15.213443 16441 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:15.219515 16366 leader_election.cc:400] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 17. Current term is 18.
I20260709 12:24:15.220007 16366 leader_election.cc:403] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:24:15.221551 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:15.222594 16476 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 16: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
W20260709 12:24:15.223460 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.223608 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:15.224644 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:15.225473 16471 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:15.231242 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:15.232111 16546 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 15.
I20260709 12:24:15.239740 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:15.240582 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:15.242760 17261 raft_consensus.cc:2393] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 15: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:15.243515 17254 raft_consensus.cc:1240] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:15.243671 16515 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:15.245795 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:15.192625 21775 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:15.192222 21601 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:15.247123 16366 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:15.249995 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:15.251251 16471 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 22.
I20260709 12:24:15.252614 16515 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:15.263705 16546 raft_consensus.cc:1275] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:24:15.264959 17270 raft_consensus.cc:1240] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:15.217115 21755 raft_consensus.cc:2804] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:15.189496 21707 consensus_queue.cc:1059] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 12:24:15.274580 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:15.275658 17264 raft_consensus.cc:1240] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:15.190135 21784 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:15.285833 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:15.255014 16367 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:15.287432 16397 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 17.
I20260709 12:24:15.276319 17271 raft_consensus.cc:1240] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:15.241653 21834 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:15.208361 21856 consensus_queue.cc:1059] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:24:15.201835 21738 consensus_queue.cc:1059] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 12:24:15.282559 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:15.246021 21730 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:15.292717 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:15.246345 21858 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:24:15.245939 21764 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 17. Current term is 18.
I20260709 12:24:15.256451 21775 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:24:15.278887 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:15.305797 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 58765.3us
I20260709 12:24:15.312969 16366 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:15.305975 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:15.307113 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:15.233834 21776 consensus_queue.cc:1059] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 12:24:15.262315 21636 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:15.315312 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:15.226352 21859 consensus_queue.cc:1059] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.084s
W20260709 12:24:15.316759 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 61844.4us
I20260709 12:24:15.227666 21778 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:15.262282 21867 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:15.326752 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:15.337781 17266 raft_consensus.cc:2393] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 14: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:15.270767 21755 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:15.326671 16473 raft_consensus.cc:1275] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:24:15.300513 21763 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:15.353936 21755 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:15.354697 16473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 1054) took 1460 ms. Trace:
I20260709 12:24:15.355069 16473 rpcz_store.cc:276] 0709 12:24:13.893900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.894094 (+   194us) service_pool.cc:224] Handling call
0709 12:24:13.894339 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:14.881809 (+987470us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b4c000e55c0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874155a20 00007f2874166a29 00007f287412c644 00007f2874153f2d 00007f2874153b93 00007f2875e8418b 00007f2875e84cb5 00007f2875e7ff30 00007f2875e817de 00007f2875e8bb49 00007f2875e8c416
0709 12:24:15.081843 (+200034us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b580013f4d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875de50aa 00007f2875e67447 00007f2875e7ffe7 00007f2875e817de 00007f2875e8bb49 00007f2875e8c416 00007f2875e88d8f 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4
0709 12:24:15.354437 (+272594us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.354646 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":320437248}
I20260709 12:24:15.356880 21866 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:15.357736 21784 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:15.357836 21866 consensus_queue.cc:1059] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:15.262355 21762 consensus_queue.cc:1059] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.053s
I20260709 12:24:15.291831 21813 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:15.362246 16398 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:24:15.301316 21833 consensus_queue.cc:1048] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 12:24:15.366441 17144 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:15.351840 21836 log.cc:927] Time spent T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 12:24:15.375770 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 111278us
I20260709 12:24:15.378922 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:15.380040 16397 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:15.338758 21426 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:24:15.389562 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.308223 21711 consensus_queue.cc:1048] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 12:24:15.398808 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:15.404525 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:15.370745 21707 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:15.379312 21784 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:24:15.366182 21857 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 12:24:15.392433 21499 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:15.388291 21426 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:15.412624 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:24:15.421072 21849 log.cc:927] Time spent T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:24:15.424621 21707 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.425551 21867 raft_consensus.cc:3055] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:15.426611 21868 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:15.421159 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:15.393896 21636 consensus_queue.cc:1059] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:24:15.443542 16441 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:15.454443 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 141539us
I20260709 12:24:15.455943 16366 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:15.456147 16441 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:15.442662 16397 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
I20260709 12:24:15.449213 16473 raft_consensus.cc:2376] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14. Re-sending same reply.
I20260709 12:24:15.405395 21762 raft_consensus.cc:3055] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:15.453083 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:15.413489 21859 raft_consensus.cc:3055] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:15.444366 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:15.417330 21782 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:15.447665 16442 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:15.434162 21426 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:15.442631 21499 raft_consensus.cc:697] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 18 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:15.442462 21867 raft_consensus.cc:740] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:24:15.488090 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 137709us
W20260709 12:24:15.491494 21881 log.cc:927] Time spent T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:24:15.496677 21787 raft_consensus.cc:3055] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:15.457294 21868 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:15.448498 21707 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:15.481539 21859 raft_consensus.cc:740] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:15.383416 16515 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:24:15.524500 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 68440.8us
I20260709 12:24:15.556236 16441 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:15.479701 21762 raft_consensus.cc:740] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
W20260709 12:24:15.557384 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 174076us
I20260709 12:24:15.562207 16515 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:15.552763 21787 raft_consensus.cc:740] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:15.564659 21711 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:15.588922 16366 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:15.585172 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:15.593679 16546 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:24:15.599799 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:15.603641 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:15.574620 21888 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:15.576836 21769 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:24:15.612985 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 165457us
I20260709 12:24:15.614595 21769 raft_consensus.cc:697] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 17 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:15.616041 21769 consensus_queue.cc:237] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:15.502208 21499 consensus_queue.cc:237] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:15.596156 21711 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:15.652884 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.670246 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:15.647109 21691 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:15.649447 21888 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:15.661604 21866 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:24:15.664649 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 49942.6us
W20260709 12:24:15.678264 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 172669us
I20260709 12:24:15.604213 21887 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:15.671098 16546 raft_consensus.cc:3055] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:15.684664 16546 raft_consensus.cc:740] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:15.682837 16543 raft_consensus.cc:1240] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:15.664758 21884 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:15.678716 16542 raft_consensus.cc:1275] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 12:24:15.701821 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 145668us
I20260709 12:24:15.687709 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:15.687084 21890 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:15.689478 21691 consensus_queue.cc:1048] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:15.690282 21868 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:15.579671 21762 consensus_queue.cc:260] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:15.689993 21887 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:15.695770 21784 raft_consensus.cc:697] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:15.589257 21867 consensus_queue.cc:260] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:15.697664 21707 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:15.679560 21881 log.cc:927] Time spent T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:24:15.671644 21711 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:15.701004 21884 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:15.679924 21866 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:15.720849 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.699812 21888 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:15.685689 16546 consensus_queue.cc:260] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:15.742903 16546 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:15.742928 21692 consensus_meta.cc:287] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.551s	user 0.005s	sys 0.007s
I20260709 12:24:15.598477 21893 consensus_queue.cc:1059] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.125s
I20260709 12:24:15.633368 21883 consensus_queue.cc:1048] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.113s
I20260709 12:24:15.744139 21897 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:15.735044 21887 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:15.760818 21887 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:15.744227 21692 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:15.687175 21898 consensus_queue.cc:1048] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 12:24:15.687767 21813 consensus_queue.cc:1059] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 12:24:15.769783 16546 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 19.
I20260709 12:24:15.670976 21787 consensus_queue.cc:260] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:15.622573 21882 consensus_queue.cc:1048] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.119s
I20260709 12:24:15.721695 21775 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:15.734019 21731 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:15.734259 21890 raft_consensus.cc:697] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:15.784935 16477 raft_consensus.cc:1275] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:24:15.745878 21866 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:15.713804 21776 consensus_queue.cc:1048] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:24:15.608457 21859 consensus_queue.cc:260] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:15.743109 21884 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:15.788090 17142 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:15.788375 21894 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.794971 21775 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:15.802243 21878 log.cc:927] Time spent T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.305s	user 0.002s	sys 0.000s
I20260709 12:24:15.809091 21858 raft_consensus.cc:3055] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:15.814533 21738 raft_consensus.cc:3055] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:24:15.818755 21891 log.cc:927] Time spent T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 12:24:15.822487 17258 raft_consensus.cc:1240] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:15.734437 21750 consensus_queue.cc:1048] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 12:24:15.729992 21895 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.055s
W20260709 12:24:15.839157 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 157201us
I20260709 12:24:15.764698 21707 consensus_queue.cc:1048] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
W20260709 12:24:15.844269 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 57184.7us
I20260709 12:24:15.848439 21894 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:15.846457 21648 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:24:15.867856 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 131366us
I20260709 12:24:15.873865 17142 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:15.809222 21755 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:15.877087 17142 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:15.850335 21858 raft_consensus.cc:740] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:15.879999 16367 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:15.853089 21738 raft_consensus.cc:740] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:15.829676 21731 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:15.881608 17142 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:15.841430 21868 consensus_queue.cc:1059] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:24:15.889214 17142 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:15.880739 21775 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:15.882393 21524 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:15.889214 16367 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:15.839553 21784 consensus_queue.cc:237] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:15.892889 21901 consensus_queue.cc:1059] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 12:24:15.898311 21901 raft_consensus.cc:3055] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:15.895428 21882 consensus_queue.cc:1059] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 12:24:15.898756 21901 raft_consensus.cc:740] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:15.899569 21901 consensus_queue.cc:260] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:15.900761 21901 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:15.895618 21587 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:15.892590 17142 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:15.906320 16442 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:15.879206 21898 consensus_queue.cc:1059] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 12:24:15.908820 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:15.912230 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:15.904521 21897 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:15.915360 21897 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:15.915730 21897 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:15.916610 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:15.916743 21882 consensus_queue.cc:1059] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:15.915987 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:15.918864 21820 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:15.921998 21731 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:15.923300 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.924432 21904 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.924808 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:15.924988 21904 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:15.925493 16399 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:15.924432 21764 raft_consensus.cc:3055] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:15.926488 21764 raft_consensus.cc:740] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:15.927672 21904 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:15.927417 21764 consensus_queue.cc:260] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:15.928380 21524 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:15.886348 21893 consensus_queue.cc:1048] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:24:15.925204 16403 raft_consensus.cc:1275] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:24:15.888007 21903 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:15.895537 21648 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:15.930150 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:15.925271 21755 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:15.933632 16441 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:15.934171 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:15.888278 21883 consensus_queue.cc:1059] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 12:24:15.936869 17144 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:15.939388 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:15.928541 21764 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:15.943342 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:15.949355 16542 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:24:15.913149 21886 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:15.908134 21763 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.950871 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:15.945683 21524 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:15.937274 21858 consensus_queue.cc:260] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:15.964190 21524 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:15.965276 21858 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:15.963161 16399 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
I20260709 12:24:15.895923 21885 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:15.956660 21426 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:15.974458 21894 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:15.968271 21900 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:15.975462 21894 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:15.975935 21894 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:15.929665 21782 consensus_queue.cc:1048] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:24:15.976539 21524 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:15.975628 21900 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:15.975730 21426 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:15.969652 21763 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:15.976629 16401 raft_consensus.cc:2393] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 13: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:15.911777 21888 consensus_queue.cc:1048] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:24:15.989331 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:15.991233 16397 raft_consensus.cc:2393] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 15: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
W20260709 12:24:15.959924 21889 log.cc:927] Time spent T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 12:24:15.989876 21894 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:15.967921 21731 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:15.999363 21897 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:15.942932 21738 consensus_queue.cc:260] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:15.960578 21886 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:15.957300 21648 raft_consensus.cc:697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 22 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:15.914184 21890 consensus_queue.cc:237] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:15.950564 21866 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:15.950366 21692 consensus_queue.cc:1059] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 12:24:16.013015 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:16.011289 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:16.010545 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:16.014389 17144 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:15.999994 21731 raft_consensus.cc:697] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:15.979979 21763 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:15.979847 21900 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:16.049244 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:16.051553 21738 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:15.992426 21885 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:16.043735 21499 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:16.052678 21648 consensus_queue.cc:237] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:16.054155 21524 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:16.055030 21897 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:16.056717 21866 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:16.017263 21711 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:15.995584 21539 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:16.044737 21886 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:16.060992 21731 consensus_queue.cc:237] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:16.068457 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.070122 17263 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
I20260709 12:24:16.052951 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:16.083550 16397 raft_consensus.cc:2393] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 16: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:16.007050 21515 consensus_queue.cc:1048] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 12:24:16.088425 16515 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:16.080514 21524 raft_consensus.cc:697] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:16.076815 21499 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:16.023667 21894 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:16.095310 16397 raft_consensus.cc:1240] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:16.085805 21770 consensus_queue.cc:1048] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 12:24:16.101789 21885 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:16.104303 17255 raft_consensus.cc:1275] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:24:16.096932 17144 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:16.082649 21539 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:16.093235 21636 raft_consensus.cc:2804] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:24:16.124596 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:16.117142 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 183576us
I20260709 12:24:16.131062 16402 raft_consensus.cc:1240] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:24:16.131699 16441 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:16.103674 21897 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:24:16.178694 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 47118.8us
I20260709 12:24:16.103991 21900 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:16.111490 21846 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:16.118072 21770 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:16.194586 16441 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:16.198658 16402 raft_consensus.cc:1240] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:24:16.128590 21636 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:16.211484 16441 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
W20260709 12:24:16.208945 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.109650 21884 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:16.129026 21769 consensus_queue.cc:1048] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 12:24:16.181282 21890 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:16.180790 21885 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:16.225997 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.226249 16441 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:16.227733 21894 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:16.237476 21499 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:16.238207 16366 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.016s	sys 0.023s
W20260709 12:24:16.252812 16366 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.018s	sys 0.024s
I20260709 12:24:16.253036 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:16.253386 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 664544us
I20260709 12:24:16.254158 16366 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:16.238119 21250 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:16.212801 21866 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:16.235837 21885 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:16.214884 21691 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:16.224213 21893 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:16.246253 21539 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:16.251931 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:16.212984 21636 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:16.275367 21896 consensus_queue.cc:1059] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:16.253659 16402 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 16.
I20260709 12:24:16.193648 21897 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:16.277284 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:16.278354 16366 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:16.278956 21897 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:16.268975 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:16.272109 21910 consensus_queue.cc:1059] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 12:24:16.280222 21910 raft_consensus.cc:3055] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:16.280751 21910 raft_consensus.cc:740] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:16.281193 21763 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:16.281602 21910 consensus_queue.cc:260] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:16.281780 21763 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:16.283699 16395 raft_consensus.cc:1240] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:16.282817 21910 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:16.286625 16471 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:24:16.196959 21900 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:16.263268 21250 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:16.273581 21893 raft_consensus.cc:697] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:24:16.273880 21873 log.cc:927] Time spent T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.710s	user 0.002s	sys 0.007s
I20260709 12:24:16.275094 21885 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:16.278462 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:16.201828 21882 consensus_queue.cc:1048] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 12:24:16.287556 17255 raft_consensus.cc:2393] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 14: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:16.272511 21691 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:16.300704 16402 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:24:16.221414 21524 consensus_queue.cc:237] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:16.302065 17254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 1342) took 1231 ms. Trace:
I20260709 12:24:16.302752 21900 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:16.289634 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.290429 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:16.298646 21884 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:16.304884 16542 raft_consensus.cc:2368] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 15. Current term is 16.
I20260709 12:24:16.299129 21893 consensus_queue.cc:237] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:16.299470 21711 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:16.306257 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.294361 16366 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
W20260709 12:24:16.307129 16516 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.511s (SENT) [suppressed 19 similar messages]
I20260709 12:24:16.328490 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:16.308833 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.319621 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:16.308066 17261 raft_consensus.cc:1275] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:24:16.331228 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:16.331912 16551 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:16.327145 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:16.332880 16546 raft_consensus.cc:2410] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:24:16.210917 21846 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:16.333966 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:16.254271 21883 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:24:16.334810 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 45766.4us
I20260709 12:24:16.302460 17254 rpcz_store.cc:276] 0709 12:24:15.071049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.097508 (+ 26459us) service_pool.cc:224] Handling call
0709 12:24:15.163482 (+ 65974us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.244276 (+ 80794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.301828 (+1057552us) spinlock_profiling.cc:243] Waited 67.5 ms on lock 0x7b54005f3554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:16.302009 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":155166464}
I20260709 12:24:16.338678 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:16.339910 16366 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:16.341037 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:16.338927 16367 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:16.303293 21707 raft_consensus.cc:3055] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:16.353015 16402 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6474b10917204b6994269be17387f38d"
dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:59912
I20260709 12:24:16.353461 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:16.353894 16402 raft_consensus.cc:606] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:24:16.354434 16402 raft_consensus.cc:614] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:16.355016 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:16.343978 16441 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:16.305307 21897 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:16.359252 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:16.293504 21888 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:16.360800 16367 leader_election.cc:400] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 15. Current term is 16.
I20260709 12:24:16.361425 16367 leader_election.cc:403] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:24:16.272542 21894 consensus_queue.cc:1059] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 12:24:16.367274 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:16.364061 16367 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:16.369524 21922 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.370100 21922 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:16.337339 21900 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:16.372231 16367 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:16.375540 16366 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:16.379535 16441 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:16.352713 21707 raft_consensus.cc:740] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:16.363550 21884 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:16.367930 21764 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:16.367529 21901 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:16.377444 21921 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.367515 21763 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:16.393908 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:16.420912 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.371537 21426 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:16.377015 21898 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:16.423704 16366 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:16.442468 16366 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:16.386282 21883 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:24:16.431777 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.385046 21776 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:16.431213 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:16.422955 21901 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:16.389479 21897 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:16.390130 21764 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:16.474838 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:16.456915 21901 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:16.461272 21764 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:16.487811 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:16.488607 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:16.489393 17255 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:16.492062 16549 raft_consensus.cc:2393] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 14: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:16.440810 21921 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:16.494059 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 51693.2us
I20260709 12:24:16.505041 16366 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:16.494486 17266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45228 (request call id 1374) took 1506 ms. Trace:
W20260709 12:24:16.502295 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 117595us
I20260709 12:24:16.455396 21636 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:16.507009 16366 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:16.483682 21897 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:16.505887 17266 rpcz_store.cc:276] 0709 12:24:14.988521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.039491 (+ 50970us) service_pool.cc:224] Handling call
0709 12:24:16.494193 (+1454702us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b5400a74c54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:16.494390 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":658665344}
I20260709 12:24:16.520540 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:16.550226 21897 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:16.516513 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:16.551280 21897 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:16.486145 21898 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:16.548623 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:16.555178 17266 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 16.
I20260709 12:24:16.555588 16542 raft_consensus.cc:2368] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 14. Current term is 15.
I20260709 12:24:16.483160 21707 consensus_queue.cc:260] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:16.554546 16402 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:16.562419 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:16.563978 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:16.566890 16397 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:16.569221 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:16.571511 17266 raft_consensus.cc:1240] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:24:16.571856 21770 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:16.574290 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:16.575340 21250 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:16.575500 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:16.577498 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:16.577883 21921 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:16.578287 16398 raft_consensus.cc:2410] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:24:16.517688 21648 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.586050 16551 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
I20260709 12:24:16.589331 16550 raft_consensus.cc:1240] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:16.591866 17255 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:24:16.596264 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:16.597412 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 290283us
I20260709 12:24:16.603103 17269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45230 (request call id 1351) took 1533 ms. Trace:
I20260709 12:24:16.603717 17269 rpcz_store.cc:276] 0709 12:24:15.069364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.093094 (+ 23730us) service_pool.cc:224] Handling call
0709 12:24:16.596058 (+1502964us) spinlock_profiling.cc:243] Waited 1.25 ms on lock 0x7b58003b8298. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:24:16.603022 (+  6964us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2867456}
I20260709 12:24:16.581171 21707 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:16.607962 16398 raft_consensus.cc:1275] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:24:16.612164 16515 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:16.591490 21250 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:16.642295 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:16.592634 21922 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:16.644924 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:16.669934 16542 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:59590
W20260709 12:24:16.668010 21931 log.cc:927] Time spent T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
I20260709 12:24:16.672067 17258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 1346) took 1395 ms. Trace:
I20260709 12:24:16.672700 17258 rpcz_store.cc:276] 0709 12:24:15.276416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.276566 (+   150us) service_pool.cc:224] Handling call
0709 12:24:15.303424 (+ 26858us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.867639 (+564215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.671733 (+804094us) spinlock_profiling.cc:243] Waited 489 ms on lock 0x7b54005f2654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:16.671986 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1124172032}
I20260709 12:24:16.685606 21762 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:24:16.695020 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.610574 21846 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:16.674125 17258 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
W20260709 12:24:16.702190 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.592737 21648 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:16.642990 21539 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:16.709411 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:16.712940 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:16.713325 16542 raft_consensus.cc:606] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:16.716509 16542 raft_consensus.cc:614] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:16.652433 21920 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:16.719858 16397 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:24:16.724798 16398 raft_consensus.cc:1275] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 12:24:16.735967 21886 consensus_meta.cc:287] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.651s	user 0.013s	sys 0.000s
W20260709 12:24:16.748557 21738 consensus_meta.cc:287] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.678s	user 0.012s	sys 0.000s
I20260709 12:24:16.727093 21539 raft_consensus.cc:697] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:16.750093 21539 consensus_queue.cc:237] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:16.751302 21691 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:16.736511 21886 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:16.753945 21886 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:16.760756 21925 log.cc:927] Time spent T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.270s	user 0.001s	sys 0.000s
I20260709 12:24:16.763262 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:16.764539 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:16.771864 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:16.772879 16469 raft_consensus.cc:1240] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:24:16.773718 21648 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:16.782886 21755 raft_consensus.cc:3055] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:16.783450 21755 raft_consensus.cc:740] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:16.785044 17261 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 1379) took 1504 ms. Trace:
I20260709 12:24:16.785490 17261 rpcz_store.cc:276] 0709 12:24:15.280907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.312254 (+ 31347us) service_pool.cc:224] Handling call
0709 12:24:16.303394 (+991140us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:16.331591 (+ 28197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.784826 (+453235us) spinlock_profiling.cc:243] Waited 991 ms on lock 0x7b54005f3550. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:16.784980 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2278527872}
I20260709 12:24:16.787075 21636 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:16.737877 21920 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:16.792613 21601 raft_consensus.cc:3055] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:16.793068 21601 raft_consensus.cc:740] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:16.793514 21920 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:16.793948 21601 consensus_queue.cc:260] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:16.795049 21601 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:16.795433 21892 log.cc:927] Time spent T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.921s	user 0.006s	sys 0.002s
I20260709 12:24:16.725286 21587 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.797257 21587 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:16.799268 21587 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:16.801900 17270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 1341) took 1731 ms. Trace:
I20260709 12:24:16.802601 17270 rpcz_store.cc:276] 0709 12:24:15.070774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.094273 (+ 23499us) service_pool.cc:224] Handling call
0709 12:24:15.190616 (+ 96343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.265516 (+ 74900us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.801620 (+1536104us) spinlock_profiling.cc:243] Waited 74.1 ms on lock 0x7b5400aee054. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:16.801829 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":170440960}
I20260709 12:24:16.804644 16397 raft_consensus.cc:2393] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 14: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:16.804503 16475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 1080) took 1385 ms. Trace:
I20260709 12:24:16.805606 21778 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:16.806958 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:16.807986 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:16.808737 16397 raft_consensus.cc:2410] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:24:16.809371 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:16.810256 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:16.813609 16401 raft_consensus.cc:1240] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
W20260709 12:24:16.814076 21944 log.cc:927] Time spent T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 12:24:16.816998 16400 raft_consensus.cc:1240] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:24:16.817422 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:16.817816 21924 consensus_queue.cc:1048] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:16.819026 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:16.787168 17261 raft_consensus.cc:1240] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:24:16.820641 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:16.821396 17263 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:16.827139 21906 log.cc:927] Time spent T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.744s	user 0.000s	sys 0.009s
I20260709 12:24:16.827656 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:16.828591 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:16.829741 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:16.830093 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:16.830610 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:16.830892 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.833905 16442 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:16.835351 16442 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:16.649891 21250 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:16.837494 17142 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:16.848037 17263 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:24:16.854744 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:16.805150 16475 rpcz_store.cc:276] 0709 12:24:15.414822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.415080 (+   258us) service_pool.cc:224] Handling call
0709 12:24:15.424985 (+  9905us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:15.425351 (+   366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:15.425391 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:15.425912 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 12:24:15.804995 (+379083us) spinlock_profiling.cc:243] Waited 6.19 ms on lock 0x7b1800057820. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2875a314bc 00007f2872d7f48b 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1
0709 12:24:15.805367 (+   372us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:15.805418 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.805517 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:16.794717 (+989200us) spinlock_profiling.cc:243] Waited 3.46 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:24:16.800099 (+  5382us) raft_consensus.cc:1685] finished
0709 12:24:16.800129 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:16.800270 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12549,"raft.queue_time_us":139762,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":13664,"spinlock_wait_cycles":22191744,"thread_start_us":162871,"threads_started":1,"wal-append.queue_time_us":366088}
I20260709 12:24:16.838757 17144 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:16.873544 16400 raft_consensus.cc:1240] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:24:16.870019 16515 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:16.888234 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:16.890748 16477 raft_consensus.cc:1240] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 14. Current term is 15. Ops: []
W20260709 12:24:16.908450 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 69791.8us
I20260709 12:24:16.898944 17271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 1383) took 1119 ms. Trace:
I20260709 12:24:16.907104 16400 raft_consensus.cc:1240] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:24:16.913712 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:16.921941 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:16.900916 21784 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.889s	user 0.019s	sys 0.039s
W20260709 12:24:16.897738 21910 consensus_meta.cc:287] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.613s	user 0.004s	sys 0.007s
I20260709 12:24:16.861877 21691 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:16.861928 21893 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:16.899996 21738 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:16.917326 21639 raft_consensus.cc:2804] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:24:16.931577 21784 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.920s	user 0.020s	sys 0.039s
I20260709 12:24:16.919080 21587 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:16.909191 17271 rpcz_store.cc:276] 0709 12:24:15.779747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.783697 (+  3950us) service_pool.cc:224] Handling call
0709 12:24:15.784094 (+   397us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:15.785666 (+  1572us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:15.785706 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:15.789924 (+  4218us) log.cc:844] Serialized 52 byte log entry
0709 12:24:15.938916 (+148992us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:15.997656 (+ 58740us) log.cc:844] Serialized 14 byte log entry
0709 12:24:16.041723 (+ 44067us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.041839 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:16.826481 (+784642us) spinlock_profiling.cc:243] Waited 1.07 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:24:16.862692 (+ 36211us) raft_consensus.cc:1685] finished
0709 12:24:16.898662 (+ 35970us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:16.898854 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":22564,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":75,"raft.queue_time_us":1139,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":6741,"spinlock_wait_cycles":2464640,"thread_start_us":148470,"threads_started":1,"wal-append.queue_time_us":260724}
I20260709 12:24:16.943809 21884 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:16.947242 21691 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:16.922590 21522 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:24:16.949083 21783 raft_consensus.cc:3055] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:16.952541 21738 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:16.854363 21755 consensus_queue.cc:260] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:16.986672 21899 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:16.987011 21755 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:16.987430 21899 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:16.988090 21899 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:16.990154 17145 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.000s	sys 0.052s
W20260709 12:24:16.990607 17145 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.000s	sys 0.052s
I20260709 12:24:16.983202 21738 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:16.992358 21912 consensus_queue.cc:1059] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 12:24:16.993714 21912 raft_consensus.cc:3055] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:16.994154 21912 raft_consensus.cc:740] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:24:16.996563 21908 consensus_queue.cc:1175] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f99066d13e524238aec6aa2024a8bd50" responder_term: 12 status { last_received { term: 12 index: 8 } last_committed_idx: 8 last_received_current_leader { term: 12 index: 8 } }
I20260709 12:24:16.968425 21884 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:16.947507 21524 consensus_queue.cc:1059] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:24:17.002333 21884 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:17.004312 21941 log.cc:927] Time spent T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 12:24:16.957964 21587 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:16.957669 21639 raft_consensus.cc:697] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:17.005362 21886 consensus_queue.cc:1059] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:24:17.013630 21904 raft_consensus.cc:3055] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:24:17.016510 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.983814 21731 consensus_queue.cc:1059] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:24:16.979437 21783 raft_consensus.cc:740] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
W20260709 12:24:17.021238 21901 consensus_meta.cc:287] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.530s	user 0.010s	sys 0.001s
I20260709 12:24:16.952256 21893 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:17.010803 21426 consensus_meta.cc:287] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.566s	user 0.009s	sys 0.002s
I20260709 12:24:17.039064 21731 raft_consensus.cc:3055] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:17.022012 21587 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:16.995666 21912 consensus_queue.cc:260] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:17.054467 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:17.056280 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:17.059926 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:17.060351 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.028692 21904 raft_consensus.cc:740] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:17.054749 16516 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:17.055611 16402 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
W20260709 12:24:17.063668 21946 log.cc:927] Time spent T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:24:17.062816 21639 consensus_queue.cc:237] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:17.066635 16397 raft_consensus.cc:1240] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:17.060433 21426 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 15. Current term is 16.
W20260709 12:24:17.066167 21764 consensus_meta.cc:287] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.573s	user 0.009s	sys 0.002s
I20260709 12:24:17.078717 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:17.056269 21731 raft_consensus.cc:740] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:24:17.085912 17143 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.014s	sys 0.000s
I20260709 12:24:17.058285 21250 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:17.087668 17143 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.014s	sys 0.000s
I20260709 12:24:17.090539 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:17.098241 16398 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:24:17.098580 16470 raft_consensus.cc:2393] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 15: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:17.051915 21901 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:17.079610 21776 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.002s	sys 0.010s
I20260709 12:24:17.050913 21927 consensus_queue.cc:1048] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:24:17.104432 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:17.104493 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:17.073571 21775 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.064s	user 0.010s	sys 0.022s
I20260709 12:24:17.061672 21918 consensus_queue.cc:1048] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
W20260709 12:24:17.069836 21888 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.003s	sys 0.009s
I20260709 12:24:17.085330 21912 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:17.130352 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:17.134509 21901 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:17.134670 16394 raft_consensus.cc:1240] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:17.088376 21764 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:17.137219 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.138420 21691 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:17.140754 21904 consensus_queue.cc:260] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:17.142129 21904 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:17.136287 16395 raft_consensus.cc:2393] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 15: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:17.107704 16402 raft_consensus.cc:2376] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15. Re-sending same reply.
I20260709 12:24:17.083462 21846 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:24:17.123674 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:17.128731 21949 log.cc:927] Time spent T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:24:17.137269 21764 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:17.151202 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:17.160427 16398 raft_consensus.cc:2410] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:24:17.162036 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:17.163448 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 185655us
W20260709 12:24:17.151793 21942 log.cc:927] Time spent T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:24:17.152899 21893 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:17.148424 21888 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.711s	user 0.003s	sys 0.010s
I20260709 12:24:17.163443 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:24:17.117666 21775 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.108s	user 0.010s	sys 0.022s
W20260709 12:24:17.117807 21776 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.003s	sys 0.010s
I20260709 12:24:17.179198 16474 raft_consensus.cc:1240] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 LEADER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:17.179682 16473 raft_consensus.cc:1275] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:24:17.179795 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:17.146245 21731 consensus_queue.cc:260] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:17.183776 16470 raft_consensus.cc:1275] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:24:17.106528 21522 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:17.185873 16396 raft_consensus.cc:2410] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
W20260709 12:24:17.185930 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 131398us
I20260709 12:24:17.200467 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:17.201516 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:17.203023 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:17.206974 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:17.207952 16542 raft_consensus.cc:2393] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 18: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
W20260709 12:24:17.208958 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.211305 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:17.213567 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:17.214546 16470 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
I20260709 12:24:17.101991 21783 consensus_queue.cc:260] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:17.216706 21783 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:17.223134 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:17.229954 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:17.231163 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.232793 16468 raft_consensus.cc:1240] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:17.238183 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:17.244481 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:17.246506 16472 raft_consensus.cc:1275] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:24:17.249734 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:17.250908 17270 raft_consensus.cc:1240] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:24:17.252316 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:17.273284 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.326397 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:17.189486 21776 raft_consensus.cc:697] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
W20260709 12:24:17.327535 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.330659 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:24:17.330057 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 283534us
I20260709 12:24:17.332690 16442 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:17.333421 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:17.334079 16468 raft_consensus.cc:3060] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:17.330462 16396 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 14. (term mismatch)
W20260709 12:24:17.335870 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.336153 17266 raft_consensus.cc:1275] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 12:24:17.328032 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 92805.2us
I20260709 12:24:17.340535 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:17.343708 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.340386 16396 pending_rounds.cc:85] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 13
I20260709 12:24:17.345510 21899 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:17.346293 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:17.347395 17270 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 18.
I20260709 12:24:17.354111 21738 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:17.355661 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:17.357012 21888 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.357636 21888 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:17.357909 21899 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:17.359447 16469 raft_consensus.cc:1240] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 14. Current term is 15. Ops: []
W20260709 12:24:17.360287 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.343189 21886 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:17.363302 21886 raft_consensus.cc:697] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:17.364444 21886 consensus_queue.cc:237] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:17.359336 16396 pending_rounds.cc:107] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 21.14
W20260709 12:24:17.361686 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.376077 21762 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.376634 21856 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.376766 21762 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:17.377214 21856 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:17.382334 17258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 1393) took 1044 ms. Trace:
I20260709 12:24:17.389559 21953 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.390383 21953 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:17.204411 21522 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:17.372836 16396 raft_consensus.cc:2889] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:17.410403 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:17.326555 21893 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:17.382875 17258 rpcz_store.cc:276] 0709 12:24:16.337830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.762554 (+424724us) service_pool.cc:224] Handling call
0709 12:24:16.762895 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:16.819541 (+ 56646us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:16.819574 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:16.820061 (+   487us) log.cc:844] Serialized 26 byte log entry
0709 12:24:17.103098 (+283037us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:17.103135 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.381916 (+278781us) spinlock_profiling.cc:243] Waited 56.3 ms on lock 0x7b54005f5354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:17.381982 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:17.382028 (+    46us) raft_consensus.cc:1685] finished
0709 12:24:17.382040 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:17.382260 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1888,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":24822,"spinlock_wait_cycles":129450368,"thread_start_us":282650,"threads_started":1,"wal-append.queue_time_us":402956}
I20260709 12:24:17.416769 21893 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:17.428529 21893 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:17.429147 21691 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:17.429726 21691 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:17.431720 21691 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:17.432722 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.425525 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:17.417037 21883 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.438591 17271 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 15.
W20260709 12:24:17.444368 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.445713 16367 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:17.451996 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:17.456063 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:17.457542 16441 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:17.456859 21764 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:17.423880 21910 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:17.461144 16441 leader_election.cc:400] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 14. Current term is 15.
I20260709 12:24:17.461705 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:17.461993 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:17.462687 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:17.465802 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:17.468406 16548 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:17.466024 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:17.469348 16399 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 18.
I20260709 12:24:17.437988 21524 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:24:17.459676 21893 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.440758 21762 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:17.470780 21893 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:17.467772 16542 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
I20260709 12:24:17.437922 21952 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.467303 16441 leader_election.cc:403] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
W20260709 12:24:17.480242 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.480466 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:17.440052 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:17.459533 21958 log.cc:927] Time spent T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:24:17.444233 21738 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:17.494062 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:17.476218 21903 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.000s	sys 0.022s
W20260709 12:24:17.476112 21707 consensus_meta.cc:287] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.843s	user 0.008s	sys 0.007s
I20260709 12:24:17.473024 21601 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:17.496310 16441 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:17.498459 21908 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.456787 21883 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:17.505689 21762 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.456650 21691 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:17.460055 21899 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:17.473208 21764 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.527091 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:17.512765 16442 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:17.528004 16473 raft_consensus.cc:2368] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 17. Current term is 18.
I20260709 12:24:17.522668 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:17.523283 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:17.531322 16473 raft_consensus.cc:1240] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:24:17.531322 16472 raft_consensus.cc:1240] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:17.486527 21866 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.524901 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:17.493768 21639 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.520593 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:17.523598 16476 raft_consensus.cc:1240] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:17.489886 21893 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:17.511453 21903 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.001s	sys 0.022s
I20260709 12:24:17.537405 16403 raft_consensus.cc:2410] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:24:17.538031 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:17.535970 16468 raft_consensus.cc:2468] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:24:17.539888 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:17.540172 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:17.514640 21776 consensus_queue.cc:237] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:17.570350 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:17.543761 16441 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:17.544768 16366 leader_election.cc:400] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 17. Current term is 18.
I20260709 12:24:17.571590 16366 leader_election.cc:403] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:24:17.479547 21910 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:17.556679 17269 raft_consensus.cc:1240] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:17.573239 16546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 1152) took 1200 ms. Trace:
I20260709 12:24:17.556823 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:17.546983 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:17.557667 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 364190us
I20260709 12:24:17.558581 16472 raft_consensus.cc:1240] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:17.489452 21952 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:17.544818 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:17.565804 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:17.579952 16366 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:17.532406 21738 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:17.529949 21953 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:17.529490 21601 raft_consensus.cc:697] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:17.546068 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:17.536582 21857 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:17.581997 16476 raft_consensus.cc:2410] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:24:17.580705 16469 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:24:17.587316 16516 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:17.527658 21908 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:17.584870 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 374165us
I20260709 12:24:17.585206 16474 raft_consensus.cc:2393] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 15: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
W20260709 12:24:17.585606 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:17.602437 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.532943 21762 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:17.544931 21866 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:17.536543 21764 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:17.576902 21884 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 14. Current term is 15.
I20260709 12:24:17.576480 21957 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:17.537801 21899 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:17.589376 21707 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:17.538508 21691 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:17.588200 21857 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:17.583196 21904 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:24:17.687167 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 75216.3us
W20260709 12:24:17.701843 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.621611 16516 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:17.544996 21639 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:17.590750 21953 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.609696 21856 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:17.722923 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 101401us
I20260709 12:24:17.723794 16516 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:17.726729 16516 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:17.612521 21426 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:17.609185 21888 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:17.677352 21942 log.cc:927] Time spent T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:24:17.611173 21901 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.700904 21778 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:17.731588 21921 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:17.751186 21921 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:17.728583 16473 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 17.
I20260709 12:24:17.722980 16514 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:17.737752 21952 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:17.738796 21962 log.cc:927] Time spent T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:24:17.739907 21883 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:17.715644 21857 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:17.741117 21967 log.cc:927] Time spent T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
W20260709 12:24:17.773380 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 50479.2us
I20260709 12:24:17.743325 21866 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:17.748637 21901 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:17.752707 21539 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:24:17.766916 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 47651us
W20260709 12:24:17.771353 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.736505 21636 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:17.759153 16516 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:17.735824 21250 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:17.714856 21957 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:17.789516 16441 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:17.803493 16441 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:17.762388 21888 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:17.778012 21920 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:24:17.820546 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.804638 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:17.793768 21883 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:17.777724 21426 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:17.806130 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 196578us
I20260709 12:24:17.732934 21953 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:17.801020 21867 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:17.756815 21952 consensus_queue.cc:1059] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 12:24:17.828575 16366 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:17.803911 21539 raft_consensus.cc:697] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:24:17.849433 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.805229 21856 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:17.800225 21707 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 17. Current term is 18.
I20260709 12:24:17.755470 21601 consensus_queue.cc:237] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:17.817294 21636 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:17.811131 21904 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:17.868432 21953 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:17.573773 16546 rpcz_store.cc:276] 0709 12:24:16.367823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.368675 (+   852us) service_pool.cc:224] Handling call
0709 12:24:16.368950 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:16.369150 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:24:17.140788 (+771638us) spinlock_profiling.cc:243] Waited 2.41 ms on lock 0x7b18004b5b60. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2875a314bc 00007f2872d7f48b 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d99455 00007f28798813c7 00007f2875e939a7 00007f2875ea30bf
0709 12:24:17.140940 (+   152us) log.cc:844] Serialized 14 byte log entry
0709 12:24:17.565307 (+424367us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:17.568003 (+  2696us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:17.568077 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.568164 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:17.568312 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15940,"prepare.queue_time_us":184504,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":74,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":7238,"spinlock_wait_cycles":5539584,"thread_start_us":584110,"threads_started":2}
I20260709 12:24:17.872982 21764 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:17.875587 21639 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:17.932358 21979 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:17.933076 21979 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:17.935606 21979 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:17.849584 21903 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:17.936478 21903 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:17.938863 21903 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:17.938472 21539 consensus_queue.cc:237] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:17.829708 21883 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:17.942301 21883 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:17.947368 21981 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:17.948181 21981 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:17.882697 21963 log.cc:927] Time spent T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.358s	user 0.003s	sys 0.000s
I20260709 12:24:17.885994 17269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 1361) took 1552 ms. Trace:
I20260709 12:24:17.959838 17269 rpcz_store.cc:276] 0709 12:24:16.334060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.736537 (+402477us) service_pool.cc:224] Handling call
0709 12:24:16.736886 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:17.572720 (+835834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.885607 (+312887us) spinlock_profiling.cc:243] Waited 819 ms on lock 0x7b54005f3a54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:17.885908 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1884825344}
I20260709 12:24:17.887111 21787 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:17.962503 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:17.887765 21783 consensus_meta.cc:287] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.671s	user 0.005s	sys 0.004s
I20260709 12:24:17.807386 21886 consensus_queue.cc:1048] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 12:24:17.893006 21908 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:17.893158 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:17.895918 21980 raft_consensus.cc:3055] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:17.897050 21921 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:17.897455 16395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46250 (request call id 1559) took 1329 ms. Trace:
I20260709 12:24:17.974383 16395 rpcz_store.cc:276] 0709 12:24:16.573812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.573946 (+   134us) service_pool.cc:224] Handling call
0709 12:24:17.897182 (+1323236us) spinlock_profiling.cc:243] Waited 561 ms on lock 0x7b54004eb6d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:17.897347 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1289808128}
I20260709 12:24:17.898326 16401 raft_consensus.cc:1275] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:24:17.899763 21901 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:17.900449 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:17.901386 21899 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:17.901862 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 73416.2us
I20260709 12:24:17.844771 21426 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:17.907398 21912 consensus_meta.cc:287] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.778s	user 0.010s	sys 0.000s
W20260709 12:24:17.908015 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.827503 21738 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:17.910183 21691 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:17.911106 16403 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46250 (request call id 1566) took 1159 ms (client timeout 1511 ms). Trace:
I20260709 12:24:17.916066 16468 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:24:17.919049 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:17.826066 21910 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:17.925741 16400 raft_consensus.cc:1275] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 12:24:18.065258 16400 pending_rounds.cc:85] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 10
I20260709 12:24:18.065572 16400 pending_rounds.cc:107] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 12:24:18.065814 16400 raft_consensus.cc:2889] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:17.927626 16442 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:17.877883 21762 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:17.961530 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 185640us
I20260709 12:24:17.824777 21888 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:18.048755 21738 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:17.924916 21952 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:24:17.970642 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 191357us
I20260709 12:24:17.951035 21981 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:17.972500 16402 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
W20260709 12:24:17.963217 21587 consensus_meta.cc:287] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.899s	user 0.013s	sys 0.000s
W20260709 12:24:17.972137 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 146909us
W20260709 12:24:17.976897 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 217857us
I20260709 12:24:17.961675 21522 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:17.982256 16366 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:17.973774 21833 raft_consensus.cc:3055] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:17.990455 21957 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:17.971305 21980 raft_consensus.cc:740] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:17.993763 16401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 1576) took 1239 ms. Trace:
I20260709 12:24:17.997292 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:17.977285 21859 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:18.010744 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:18.080861 16401 rpcz_store.cc:276] 0709 12:24:16.754797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.815851 (+ 61054us) service_pool.cc:224] Handling call
0709 12:24:17.897846 (+1081995us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:17.975631 (+ 77785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.993434 (+ 17803us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54004eb6d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:17.993663 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2487695616}
W20260709 12:24:18.049291 16403 rpcz_store.cc:269] 0709 12:24:16.751644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.827415 (+ 75771us) service_pool.cc:224] Handling call
0709 12:24:17.910839 (+1083424us) spinlock_profiling.cc:243] Waited 709 ms on lock 0x7b540015e354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:17.911027 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1630267776}
I20260709 12:24:18.082841 16396 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:24:18.083868 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:18.072906 17143 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:18.081907 16398 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
W20260709 12:24:18.077448 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 95285.8us
W20260709 12:24:18.084573 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 303359us
I20260709 12:24:18.086961 16366 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:18.066690 16477 raft_consensus.cc:2410] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260709 12:24:18.067117 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 139587us
I20260709 12:24:18.071902 16514 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:18.088393 16366 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:18.088881 16442 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:17.993197 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:18.084800 16396 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 19.
I20260709 12:24:18.095995 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:18.096526 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:18.096743 16396 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:24:18.097733 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:18.098021 16442 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:18.084774 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:18.099694 16401 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 17.
I20260709 12:24:18.100597 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:18.101351 16401 raft_consensus.cc:2410] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:24:18.106357 17143 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:18.107683 17143 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:18.098292 16396 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:24:18.104813 17144 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:18.109863 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:18.110924 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:18.111627 16476 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 22.
I20260709 12:24:18.113183 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:18.113791 16468 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14.
I20260709 12:24:18.115664 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:18.068073 21738 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:18.066958 21901 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.123418 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:18.124433 16401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 1414) took 1103 ms. Trace:
I20260709 12:24:18.120812 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:18.121922 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:18.068418 21978 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:18.067416 21691 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:18.126766 16401 rpcz_store.cc:276] 0709 12:24:17.021218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.123523 (+1102305us) service_pool.cc:224] Handling call
0709 12:24:18.123844 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.124051 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:18.124083 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:18.124124 (+    41us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:18.124159 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.124218 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:18.124376 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:18.133576 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:18.069200 21888 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:18.125380 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 53543.1us
I20260709 12:24:18.108906 21764 raft_consensus.cc:2749] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:18.073715 21981 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.078123 21783 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.081554 21522 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.116490 21750 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.072659 21587 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:18.128307 21957 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:18.130158 21524 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:18.125356 17144 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:18.085510 21952 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:18.129539 16403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 1590) took 1102 ms. Trace:
I20260709 12:24:18.077625 21833 raft_consensus.cc:740] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:18.068346 21762 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:18.114940 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:18.171006 16472 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 18.
I20260709 12:24:18.135375 16396 raft_consensus.cc:1240] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:18.107805 21908 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.109231 21912 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:18.140177 17143 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:18.176826 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:18.071641 21639 consensus_queue.cc:1048] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 12:24:18.111599 21903 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:18.071815 21920 consensus_queue.cc:1048] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 12:24:18.127265 16442 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:18.182935 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 55738.9us
I20260709 12:24:18.161545 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:24:18.184587 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 88762.1us
I20260709 12:24:18.164757 21784 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:18.156410 21750 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:18.163117 17144 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:18.165506 16366 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:18.133047 21982 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:18.166878 16403 rpcz_store.cc:276] 0709 12:24:17.026967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.128571 (+1101604us) service_pool.cc:224] Handling call
0709 12:24:18.128880 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.129087 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:18.129119 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:18.129162 (+    43us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:18.129204 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.129364 (+   160us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:18.129487 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:18.148877 21783 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:18.147109 21989 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:18.154534 21922 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:18.174710 16396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 1415) took 1097 ms. Trace:
I20260709 12:24:18.154171 21250 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:18.160422 21867 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:18.179339 21970 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.132455 21901 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:18.186236 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:18.386175 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 200969us
I20260709 12:24:18.183560 16442 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:18.167637 16514 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:18.209649 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:18.351781 16397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 1569) took 1599 ms. Trace:
W20260709 12:24:18.387003 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 203503us
I20260709 12:24:18.146204 21981 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:18.155252 21522 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:18.360421 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:18.360524 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 173965us
I20260709 12:24:18.168737 21524 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:18.267171 16399 raft_consensus.cc:1275] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:24:18.286479 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:18.135514 21978 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:18.369953 16396 rpcz_store.cc:276] 0709 12:24:17.076932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.132873 (+1055941us) service_pool.cc:224] Handling call
0709 12:24:18.133310 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.174404 (+ 41094us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.174641 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:18.179734 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:18.174033 21904 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:18.373337 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 210256us
I20260709 12:24:18.183075 21633 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:18.387220 16397 rpcz_store.cc:276] 0709 12:24:16.752680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.810990 (+ 58310us) service_pool.cc:224] Handling call
0709 12:24:17.066385 (+255395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:17.077972 (+ 11587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.351550 (+1273578us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b54008a6bd0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:18.351714 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":586667392}
I20260709 12:24:18.369776 21922 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.391590 16548 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 17.
I20260709 12:24:18.187564 21784 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:18.394059 16399 raft_consensus.cc:1275] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 12:24:18.384117 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 219576us
I20260709 12:24:18.388067 16550 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 18.
I20260709 12:24:18.181866 21903 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:18.360466 21601 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:24:18.395512 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 227931us
I20260709 12:24:18.176275 21762 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:18.262629 21957 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.361373 21707 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:18.401292 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:18.402518 21879 consensus_queue.cc:1048] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:18.402927 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:18.370019 21989 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.403255 21904 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:18.403775 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.405689 21524 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:18.182312 21857 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:18.387696 16442 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:18.369628 21764 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:18.371685 21982 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.392447 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:18.414610 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:18.417228 16551 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:24:18.378099 21987 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.425431 16401 raft_consensus.cc:1275] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:24:18.417109 17271 raft_consensus.cc:1275] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:24:18.440253 16397 raft_consensus.cc:1275] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:24:18.568794 16397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 1421) took 1075 ms. Trace:
I20260709 12:24:18.569257 16397 rpcz_store.cc:276] 0709 12:24:17.493670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.416986 (+923316us) service_pool.cc:224] Handling call
0709 12:24:18.417288 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.568545 (+151257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.568735 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:18.395342 21899 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:18.570837 21899 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.508217 16394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 1411) took 1682 ms. Trace:
I20260709 12:24:18.571122 21857 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:18.187318 21778 consensus_queue.cc:1059] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.222s
I20260709 12:24:18.446437 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:18.421696 16366 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:18.571818 21764 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:18.572810 21987 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:18.574455 21764 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:18.575120 21764 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:18.575528 21764 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.567605 21867 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:18.576474 21867 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:18.576912 21867 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:18.575845 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:18.578361 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:18.579494 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:18.567848 21982 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:18.571285 21899 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:18.582172 16471 raft_consensus.cc:2393] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 19: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:18.385501 21953 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:18.582793 21953 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.583191 21953 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:18.584371 21901 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:18.584751 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:18.575888 21764 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:18.389693 21776 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:18.588907 21776 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.589373 21776 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:18.589699 21978 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:18.591038 16543 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:24:18.419679 16395 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 16.
I20260709 12:24:18.592103 21648 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.408142 21986 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:18.519752 16514 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:18.417376 21922 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:18.592967 21648 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:18.412144 21950 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:18.596647 21950 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:18.539286 16400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 1579) took 1584 ms. Trace:
I20260709 12:24:18.598686 16400 rpcz_store.cc:276] 0709 12:24:16.954668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.956000 (+  1332us) service_pool.cc:224] Handling call
0709 12:24:17.913602 (+957602us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.066317 (+152715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.535170 (+468853us) spinlock_profiling.cc:243] Waited 957 ms on lock 0x7b540015e350. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:18.539210 (+  4040us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2201530112}
I20260709 12:24:18.441931 21784 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:18.440999 21903 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:18.373217 21833 consensus_queue.cc:260] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:18.506692 21888 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:18.563025 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 175396us
I20260709 12:24:18.615326 16442 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:18.391487 21970 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:18.565745 16551 raft_consensus.cc:606] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:18.618494 16551 raft_consensus.cc:614] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:18.178539 21908 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:18.441174 21601 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:18.621546 16516 leader_election.cc:336] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.770s (SENT)
I20260709 12:24:18.571478 16394 rpcz_store.cc:276] 0709 12:24:16.826224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.857113 (+ 30889us) service_pool.cc:224] Handling call
0709 12:24:16.857418 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:17.135312 (+277894us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.507836 (+1372524us) spinlock_profiling.cc:243] Waited 277 ms on lock 0x7b540015fc54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:18.508122 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":636986240}
I20260709 12:24:18.603549 21899 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:18.610334 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:18.538863 21952 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:18.619602 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 99954.4us
I20260709 12:24:18.551723 21989 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:18.638721 21908 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:18.625989 16516 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:18.627441 16394 raft_consensus.cc:1275] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:24:18.641645 16548 raft_consensus.cc:2393] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 19: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:18.621762 16400 raft_consensus.cc:1275] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 12:24:18.643100 16400 pending_rounds.cc:85] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 10
I20260709 12:24:18.643374 16400 pending_rounds.cc:107] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 12:24:18.643795 16400 raft_consensus.cc:2889] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:18.631197 16514 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:18.632913 21952 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:18.645080 21904 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:18.645629 21762 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:18.631186 21587 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:18.628472 21910 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.646972 21952 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:18.620816 21857 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:18.647284 21910 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:18.648871 21426 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:18.539189 21957 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:18.616431 21250 consensus_queue.cc:1048] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:24:18.625429 21833 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:18.617851 21986 raft_consensus.cc:3055] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:18.571156 21981 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:18.620622 21888 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:18.629294 21601 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:18.640218 21970 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:18.548671 21707 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:18.707696 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:18.665220 16402 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46250 (request call id 1587) took 1665 ms (client timeout 1770 ms). Trace:
I20260709 12:24:18.670007 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:18.671809 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:18.715662 16473 raft_consensus.cc:2410] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:24:18.706506 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:18.716714 17261 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 16.
I20260709 12:24:18.672001 16514 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:18.673434 16477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34840 (request call id 1608) took 1153 ms. Trace:
I20260709 12:24:18.717995 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:18.677274 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:18.710816 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:18.720491 17269 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:18.714166 16543 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:24:18.654479 16401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 1416) took 1497 ms. Trace:
W20260709 12:24:18.713145 16402 rpcz_store.cc:269] 0709 12:24:16.999507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.163220 (+163713us) service_pool.cc:224] Handling call
0709 12:24:18.665035 (+1501815us) spinlock_profiling.cc:243] Waited 782 ms on lock 0x7b54009be6d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:18.665155 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1798171392}
W20260709 12:24:18.671064 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 49551.3us
I20260709 12:24:18.721444 16401 rpcz_store.cc:276] 0709 12:24:17.156968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.137571 (+980603us) service_pool.cc:224] Handling call
0709 12:24:18.137868 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.567841 (+429973us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.654224 (+ 86383us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b540011b054. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:18.654416 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":660298112}
I20260709 12:24:18.671167 16476 raft_consensus.cc:1240] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:24:18.672649 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:18.711844 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:18.724778 17270 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14.
I20260709 12:24:18.725432 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:18.725934 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:18.718674 16477 rpcz_store.cc:276] 0709 12:24:17.520071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.520236 (+   165us) service_pool.cc:224] Handling call
0709 12:24:18.673222 (+1152986us) spinlock_profiling.cc:243] Waited 475 ms on lock 0x7b5400a68bd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:18.673354 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1091387776}
W20260709 12:24:18.728435 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 56515.8us
I20260709 12:24:18.670660 21857 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:18.735853 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:18.737543 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:18.738469 17255 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:24:18.732445 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:18.731376 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:18.700973 16403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 1453) took 1280 ms. Trace:
I20260709 12:24:18.734280 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:18.741887 17261 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 15.
I20260709 12:24:18.743005 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:18.743480 17263 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:18.740312 16514 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:18.745226 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:18.682796 21859 consensus_meta.cc:287] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.586s	user 0.005s	sys 0.003s
I20260709 12:24:18.772243 16550 raft_consensus.cc:1275] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:24:18.743220  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:24:18.773413 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:18.672202 21778 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:18.783259 16468 raft_consensus.cc:1275] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 12:24:18.784200 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:18.789991 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:18.672201 21899 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:18.667940 21888 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:18.760829 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:18.672799 21522 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:18.672382 21636 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:18.662307 21910 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:18.823141 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:18.824775 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:18.825423 16469 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
I20260709 12:24:18.786178 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:18.826550 17261 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:24:18.827345 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:18.789866 16468 pending_rounds.cc:85] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 9
I20260709 12:24:18.792263 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:18.828087 16468 pending_rounds.cc:107] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 12:24:18.667141 21587 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.794530 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:18.796962 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:18.747615 21857 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:18.800633 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:18.771485 21879 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:18.665831 21691 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:18.749452 21983 consensus_queue.cc:1059] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 12:24:18.673794 21426 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:18.728686 21973 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:18.659258 21986 raft_consensus.cc:740] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:18.674863 21950 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:18.664618 21996 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:18.809031 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:18.790565 16475 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:18.668407 21904 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:18.755424 16441 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:18.813460 17263 raft_consensus.cc:1275] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:24:18.817858 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:18.818504 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 65302.3us
I20260709 12:24:18.818424 16543 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 18.
I20260709 12:24:18.820108 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:18.821103 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 399465us
I20260709 12:24:18.731737 21970 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:18.821894 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:18.877381 16402 raft_consensus.cc:2393] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 15: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:18.790977 16474 raft_consensus.cc:2410] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:24:18.662704 21957 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:18.828455 16468 raft_consensus.cc:2889] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:18.881053 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:18.881209 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:18.881958 16474 raft_consensus.cc:2393] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 17: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:18.882298 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:18.831960 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:18.658989 21953 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:18.672763 21783 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:18.690496 21764 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:18.837061 16472 raft_consensus.cc:1240] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:18.683632 21867 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:18.893615 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:18.651520 21776 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:18.670521 21989 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:18.664847 21922 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:18.746212 16403 rpcz_store.cc:276] 0709 12:24:17.420308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.362772 (+942464us) service_pool.cc:224] Handling call
0709 12:24:18.363141 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.363393 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:24:18.511483 (+148090us) log.cc:844] Serialized 14 byte log entry
0709 12:24:18.700569 (+189086us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:18.700648 (+    79us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:18.700694 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.700768 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:18.700903 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":262024,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":1890,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":15164,"thread_start_us":312538,"threads_started":2}
I20260709 12:24:18.869306 21868 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.894527 16395 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:18.901381 21998 consensus_queue.cc:1048] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:18.871456 21648 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:18.855470 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:18.856849 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:18.815389 21859 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:18.747282 21755 consensus_queue.cc:1048] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.087s
I20260709 12:24:18.917644 21522 consensus_queue.cc:1048] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:18.867265 21999 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.059s
I20260709 12:24:18.864735 21426 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:18.922144 16471 raft_consensus.cc:1240] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:24:18.884955 21899 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:18.864110 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 190825us
I20260709 12:24:18.926743 16400 raft_consensus.cc:2393] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 15: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:18.652458 21980 consensus_queue.cc:260] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:18.860954 21787 consensus_meta.cc:287] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.896s	user 0.007s	sys 0.006s
I20260709 12:24:18.861682 21691 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:18.878266 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 122965us
I20260709 12:24:18.939749 16441 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:18.892349 16474 raft_consensus.cc:1275] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:24:18.839677 21587 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:18.905259 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 94799.8us
I20260709 12:24:18.907451 17266 raft_consensus.cc:1240] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
W20260709 12:24:18.626554 21775 consensus_queue.cc:1175] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f99066d13e524238aec6aa2024a8bd50" responder_term: 18 status { last_received { term: 17 index: 10 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 17. Current term is 18. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:24:18.913419 21857 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:18.931138 16396 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 23 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 12:24:18.878973 21903 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:18.908280 21868 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:18.966044 21903 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:18.966879 21868 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:18.967762 21707 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:18.883531 21904 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:18.973870 21587 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:18.974457 21904 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:18.935076 21986 consensus_queue.cc:260] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:18.976292 21986 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:18.950276 21886 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:18.938570 21899 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:18.982398 21867 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:18.874886 16366 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:18.983456 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:24:18.983623 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 108781us
I20260709 12:24:18.984140 16397 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:18.987965 16401 raft_consensus.cc:2410] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
W20260709 12:24:18.988976 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 78772.9us
I20260709 12:24:18.989343 21691 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:18.990269 21960 log.cc:927] Time spent T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 1.451s	user 0.006s	sys 0.008s
I20260709 12:24:18.990727 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:18.924113 21426 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:18.991665 17142 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:18.991947 21426 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:18.992383 21426 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.992772 21426 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:18.993815 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 52320.3us
I20260709 12:24:18.995095 16475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53186 (request call id 1102) took 2082 ms. Trace:
W20260709 12:24:18.995929 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 51273.2us
I20260709 12:24:18.996973 17144 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:18.998457 21957 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:18.998890 17144 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:18.983506 21991 log.cc:927] Time spent T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.592s	user 0.002s	sys 0.000s
I20260709 12:24:18.990764 16399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 1423) took 1177 ms. Trace:
I20260709 12:24:19.004144 16399 rpcz_store.cc:276] 0709 12:24:17.813820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.508841 (+695021us) service_pool.cc:224] Handling call
0709 12:24:18.509204 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.606334 (+ 97130us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:18.606364 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:18.606404 (+    40us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:18.606434 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.990564 (+384130us) spinlock_profiling.cc:243] Waited 96.9 ms on lock 0x7b540049f054. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:18.990612 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:18.990716 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":222864768}
I20260709 12:24:19.005990 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:19.006068 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:19.006841 16399 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:19.006940 16395 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
W20260709 12:24:19.009552 21992 log.cc:927] Time spent T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.459s	user 0.000s	sys 0.002s
I20260709 12:24:19.013070 21775 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:19.013540 21775 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.013857 21775 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:18.995713 16475 rpcz_store.cc:276] 0709 12:24:16.913214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.913402 (+   188us) service_pool.cc:224] Handling call
0709 12:24:18.994855 (+2081453us) spinlock_profiling.cc:243] Waited 1.88 s on lock 0x7b54000b5c54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:18.995008 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4313558912}
W20260709 12:24:19.018345 21994 log.cc:927] Time spent T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.499s	user 0.000s	sys 0.002s
I20260709 12:24:19.020124 21601 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:19.022100 16470 raft_consensus.cc:2393] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 15: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:19.021687 16223 catalog_manager.cc:5697] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 12 to 14, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 14 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:24:19.023417 16223 catalog_manager.cc:5697] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 8 to 15, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 15 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:24:19.024917 17143 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:19.025494 16223 catalog_manager.cc:5697] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 9 to 11, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 11 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:19.026590 22002 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:19.027124 16223 catalog_manager.cc:5697] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 7 to 14. New cstate: current_term: 14 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:18.931931 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:19.028664 16398 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 19.
I20260709 12:24:19.029002 17143 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:19.029620 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:19.030238 16398 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:19.031201 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:18.889139 21988 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:19.031877 16402 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
I20260709 12:24:19.032121 21988 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:19.034142 16549 raft_consensus.cc:1275] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:24:18.885396 22008 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:19.034797 21988 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:19.035220 22008 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.849340 16477 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:24:18.942142 16474 raft_consensus.cc:1275] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 12:24:19.036283 16474 pending_rounds.cc:85] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 9
I20260709 12:24:18.943593 17269 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 17.
I20260709 12:24:18.947046 16441 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
W20260709 12:24:19.036614 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 45020.3us
W20260709 12:24:19.037092 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 90209.5us
I20260709 12:24:18.949643 16472 raft_consensus.cc:1240] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:18.930567 16367 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:19.039515 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 109092us
W20260709 12:24:19.039713 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.932832 16403 raft_consensus.cc:1275] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:24:18.948863 21770 consensus_queue.cc:1059] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 12:24:18.952708 21980 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:19.057268 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:19.058455 16398 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:19.059005 16399 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 16.
I20260709 12:24:19.060611 16398 raft_consensus.cc:1240] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:24:18.953321 21764 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:18.953186 21969 consensus_queue.cc:1059] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 12:24:18.953519 21750 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:18.953846 21250 consensus_queue.cc:1048] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:18.954622 21515 consensus_queue.cc:1048] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:19.070680 21927 consensus_queue.cc:1059] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:19.072072 21927 raft_consensus.cc:3055] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:19.072467 21927 raft_consensus.cc:740] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:18.956279 21776 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 23 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:18.957628 21953 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:18.958141 21922 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:18.958916 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:18.959029 21784 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:18.959683 21989 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:18.962757 21738 consensus_meta.cc:287] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.827s	user 0.009s	sys 0.000s
I20260709 12:24:18.905716 21834 consensus_queue.cc:1048] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 12:24:18.919567 21859 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:18.963661 21982 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:19.020704  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:19.022389 21601 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:18.937846 16547 raft_consensus.cc:2410] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 12:24:19.036553 16474 pending_rounds.cc:107] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 12:24:19.110472 16474 raft_consensus.cc:2889] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:18.945107 17254 raft_consensus.cc:2393] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 19: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:19.041347 16367 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:19.042893 21903 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:19.043045 22008 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:18.867537 21973 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:19.046195 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:19.129827 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.049873 17261 raft_consensus.cc:1240] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:19.048274 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:19.043358 21867 consensus_queue.cc:1048] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 12:24:19.050760 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:19.051287 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:19.052381 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:19.044140 17144 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:19.137897 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 93814.2us
I20260709 12:24:19.139431 17144 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:19.053552 21731 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:19.062531 17142 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:19.066584 16477 raft_consensus.cc:1240] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:24:19.165223 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:19.166189 16477 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:19.067102 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:19.049880 16366 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
W20260709 12:24:19.172726 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 122925us
I20260709 12:24:19.066283 16397 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 19.
W20260709 12:24:19.082867 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 53932.3us
I20260709 12:24:19.175236 16402 raft_consensus.cc:1240] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 12:24:19.081840 16394 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:24:19.085173 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:19.177170 16470 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:24:19.177469 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:19.178388 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:19.092072 16551 raft_consensus.cc:1275] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:24:19.111831 21924 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.095705 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:19.066371 21691 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:19.188406 16472 raft_consensus.cc:2393] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 15: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:19.099242 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:19.100623 21784 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:19.101856 21993 log.cc:927] Time spent T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.590s	user 0.003s	sys 0.000s
I20260709 12:24:19.102245 21636 raft_consensus.cc:3055] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:19.091722 21988 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:19.072477 21778 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:19.113883 21782 consensus_queue.cc:1048] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 12:24:19.112517 16396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 1420) took 1640 ms. Trace:
I20260709 12:24:19.202915 16396 rpcz_store.cc:276] 0709 12:24:17.471619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.397120 (+925501us) service_pool.cc:224] Handling call
0709 12:24:18.397440 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:18.949814 (+552374us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.112200 (+162386us) spinlock_profiling.cc:243] Waited 533 ms on lock 0x7b54001487d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:19.112437 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1226822016}
W20260709 12:24:19.114792 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 73511us
I20260709 12:24:19.080736 21987 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:19.077519 16441 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:19.132637 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:19.134410 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:19.050853 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:19.093174 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:19.205708 16548 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 16.
I20260709 12:24:19.205966 16473 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 9.
I20260709 12:24:19.206640 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:24:19.179682 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 97742us
I20260709 12:24:19.184719 17270 raft_consensus.cc:2410] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
W20260709 12:24:19.184579 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 45279.8us
W20260709 12:24:19.185585 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 123127us
I20260709 12:24:19.207988 17144 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:19.187227 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:19.189275 16400 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:19.209201 16470 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 15.
I20260709 12:24:19.129927 21776 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:19.104259 21738 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:19.194592 16477 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 23.
I20260709 12:24:19.205190 16367 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:19.183125 21922 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:19.209862 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:19.105358 21927 consensus_queue.cc:260] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:19.196307 21636 raft_consensus.cc:740] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:19.188001 21833 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:19.253666 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:19.190272 21770 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:19.217244 17142 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:19.220530 21859 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:19.217329 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:19.220162 21899 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:19.224251 21973 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:19.227029 21738 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:19.227756 16468 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 18.
I20260709 12:24:19.229090 21426 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:19.143122 21787 consensus_queue.cc:1048] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 12:24:19.190505 21691 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:19.233220 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:24:19.235005 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 157596us
I20260709 12:24:19.274333 16544 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:19.275697 16441 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:19.242005 16470 raft_consensus.cc:1240] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:19.242628 16474 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:24:19.242981 16367 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:19.244956 16472 raft_consensus.cc:2393] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 15: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:19.187073 21903 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:19.246572 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:19.247514 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:19.249679 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:19.244957 16473 raft_consensus.cc:1240] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:19.251101 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:19.254875 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:19.257858 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:19.279671 16441 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:19.259157 17143 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:19.262982 17258 raft_consensus.cc:2393] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 17: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:19.192022 21924 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:19.213753 21776 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:19.265026 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:19.244426 21522 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:19.240500 21922 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.244633 21764 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:19.251535 21867 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:19.264189 21778 consensus_queue.cc:1048] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 12:24:19.263545 21846 consensus_queue.cc:1048] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 12:24:19.279348 16367 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:19.264760 22000 consensus_queue.cc:1059] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 12:24:19.267102 21970 consensus_queue.cc:1048] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:24:19.277611 21691 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:19.283169 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 66037.4us
I20260709 12:24:19.285290 17270 raft_consensus.cc:2393] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 20: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:19.281457 17269 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
I20260709 12:24:19.293747 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:19.345755 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 86659.1us
I20260709 12:24:19.279294 21648 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:19.284776 21859 consensus_queue.cc:1059] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 12:24:19.280030 21903 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:19.316609 21921 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:19.289096 21776 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:19.311448 21820 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:19.329794 17142 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:19.334280 16542 raft_consensus.cc:1275] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
I20260709 12:24:19.367686 16542 pending_rounds.cc:85] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1: Aborting all ops after (but not including) 8
I20260709 12:24:19.368067 16542 pending_rounds.cc:107] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1: Aborting uncommitted NO_OP operation due to leader change: 17.9
W20260709 12:24:19.368276 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 84577.5us
I20260709 12:24:19.339038 21250 raft_consensus.cc:3055] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:19.294889 21426 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:19.285070 21738 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:24:19.333813 16547 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52680 (request call id 1158) took 2954 ms (client timeout 2531 ms). Trace:
I20260709 12:24:19.342796 21927 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:19.328871 21922 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:19.343695 22000 raft_consensus.cc:3055] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:19.348731 21953 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:19.344601 21707 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:19.276844 21995 consensus_queue.cc:1059] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 12:24:19.291718 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:19.366475 21859 raft_consensus.cc:3055] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:19.349423 16544 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 19.
W20260709 12:24:19.350353 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 82841.8us
I20260709 12:24:19.295450 21763 consensus_queue.cc:1059] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 12:24:19.351969 21601 consensus_queue.cc:1048] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:19.352928 16400 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 20.
I20260709 12:24:19.368400 16542 raft_consensus.cc:2889] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:19.281558 21770 raft_consensus.cc:697] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 16 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:19.300247 21982 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:19.317641 21867 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:19.368405 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 44077.6us
W20260709 12:24:19.384405 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 96083.3us
I20260709 12:24:19.379107 21820 raft_consensus.cc:697] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 17 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:19.333233 21587 raft_consensus.cc:3055] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:19.407644 21587 raft_consensus.cc:740] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:19.408514 21587 consensus_queue.cc:260] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:19.409801 21587 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:19.377728 16475 raft_consensus.cc:1240] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:19.406827 17143 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:19.336418 21924 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:19.375154 16547 rpcz_store.cc:269] 0709 12:24:16.379664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.384795 (+  5131us) service_pool.cc:224] Handling call
0709 12:24:19.333612 (+2948817us) spinlock_profiling.cc:243] Waited 2.49 s on lock 0x7b54008c41d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:19.333742 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5718278016}
I20260709 12:24:19.413204 21770 consensus_queue.cc:237] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:19.404515 21867 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:19.402220 21973 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:19.418309 21973 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:19.389168 21707 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.421135 21707 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:19.393496 21859 raft_consensus.cc:740] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:24:19.378489 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 48773.9us
I20260709 12:24:19.429131 21859 consensus_queue.cc:260] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:19.285231 21888 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:19.382198 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:19.414224 21633 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:19.431031 21633 raft_consensus.cc:697] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 19 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:19.431823 16515 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:19.382495 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:19.373071 21908 consensus_queue.cc:1048] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:24:19.441609 22008 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:19.382658 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:19.384837 22000 raft_consensus.cc:740] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:19.285912 21899 consensus_queue.cc:1048] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.084s
I20260709 12:24:19.446875 16548 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:19.354661 21903 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:19.447290 22000 consensus_queue.cc:260] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:19.289278 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:24:19.392503 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 111880us
I20260709 12:24:19.400434 17144 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:19.369095 21250 raft_consensus.cc:740] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:19.401491 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:19.403321 16514 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:19.448225 21648 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:19.463795 21648 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:19.464681 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 61441.6us
I20260709 12:24:19.451538 21820 consensus_queue.cc:237] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:19.448448 22000 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:19.451503 21636 consensus_queue.cc:260] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:19.431845 21633 consensus_queue.cc:237] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:19.465999 21918 raft_consensus.cc:2764] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:24:19.469503 21987 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:19.470733 21250 consensus_queue.cc:260] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:19.478798 21859 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:19.404999 16367 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:19.356206 16441 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:19.389375 21784 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:24:19.481575 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 49866.5us
I20260709 12:24:19.472350 21250 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:24:19.482461 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 82152.6us
I20260709 12:24:19.472180 16542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 1169) took 1997 ms. Trace:
I20260709 12:24:19.197090  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:24:19.488440 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 84551.8us
I20260709 12:24:19.406817 16366 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:19.497114 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 91134.3us
I20260709 12:24:19.496165 16367 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:19.498251 17270 raft_consensus.cc:1275] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:24:19.504494 17261 raft_consensus.cc:1240] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:19.494903 17144 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:19.497975 16366 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:19.495185 16542 rpcz_store.cc:276] 0709 12:24:17.474520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.474693 (+   173us) service_pool.cc:224] Handling call
0709 12:24:19.333949 (+1859256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:19.397991 (+ 64042us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.471981 (+ 73990us) spinlock_profiling.cc:243] Waited 1.86 s on lock 0x7b54008c41d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:19.472118 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4274979584}
I20260709 12:24:19.427698 21922 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:19.508910 17264 raft_consensus.cc:2393] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 14: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:19.510314 21922 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:19.514714 16224 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53636 (request call id 133) took 38952 ms (client timeout 15000 ms). Trace:
I20260709 12:24:19.517791 16367 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:19.530006 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:19.458199 21973 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:19.527395 16366 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:19.550033 16223 catalog_manager.cc:5697] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 7 to 13, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 13 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
W20260709 12:24:19.551775 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 56994.3us
W20260709 12:24:19.537225 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 181088us
I20260709 12:24:19.552191 16223 catalog_manager.cc:5697] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 10 to 13, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 13 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:19.478834 21899 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:19.492249 22008 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:19.571063 16548 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:19.506093 21639 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:24:19.580123 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 111391us
I20260709 12:24:19.582636 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:19.577308 17143 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:19.505434 21888 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:19.516477 22003 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:19.517664 16224 rpcz_store.cc:269] 0709 12:23:40.562904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:40.564675 (+  1771us) service_pool.cc:224] Handling call
0709 12:23:41.156044 (+591369us) spinlock_profiling.cc:243] Waited 288 ms on lock 0x7b540027e054. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df3090 00007f2875e91f3a 00007f287bc47e9c 00007f287bd4604e 00007f287a2072a4 00007f287a207232 00007f287a207161 00007f287a2070dc 00007f287a206392 00007f28741c448c 00007f28741c38c6
0709 12:24:19.514588 (+38358544us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:12.815342 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:12.990147 (+174805us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:13.003423 (+ 13276us) write_op.cc:435] Acquired schema lock
0709 12:24:13.005413 (+  1990us) tablet.cc:662] Decoding operations
0709 12:24:13.069929 (+ 64516us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:13.070013 (+    84us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:13.070042 (+    29us) tablet.cc:685] Acquiring locks for 13 operations
0709 12:24:13.071344 (+  1302us) tablet.cc:701] Row locks acquired
0709 12:24:13.071391 (+    47us) write_op.cc:260] PREPARE: finished
0709 12:24:13.071574 (+   183us) write_op.cc:270] Start()
0709 12:24:13.071716 (+   142us) write_op.cc:276] Timestamp: P: 1783599853071507 usec, L: 0
0709 12:24:13.071737 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:24:13.082651 (+ 10914us) log.cc:844] Serialized 5080 byte log entry
0709 12:24:13.334586 (+251935us) op_driver.cc:464] REPLICATION: finished
0709 12:24:13.558451 (+223865us) write_op.cc:301] APPLY: starting
0709 12:24:13.558590 (+   139us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:13.800407 (+241817us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:13.800521 (+   114us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:13.820853 (+ 20332us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:13.821381 (+   528us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=0,mrs_lookups=0
0709 12:24:13.821461 (+    80us) write_op.cc:312] APPLY: finished
0709 12:24:13.822001 (+   540us) log.cc:844] Serialized 120 byte log entry
0709 12:24:13.822250 (+   249us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:13.822467 (+   217us) write_op.cc:454] Released schema lock
0709 12:24:13.822569 (+   102us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:13.860378 (+ 37809us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:13.861845 (+  1467us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1349156,"reported_tablets":44,"spinlock_wait_cycles":662390528,"child_traces":[["op",{"apply.queue_time_us":223511,"apply.run_cpu_time_us":23571,"apply.run_wall_time_us":290465,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58519,"num_ops":13,"prepare.queue_time_us":341743,"prepare.run_cpu_time_us":15412,"prepare.run_wall_time_us":330557,"raft.queue_time_us":120883,"raft.run_cpu_time_us":55215,"raft.run_wall_time_us":151998,"replication_time_us":262623,"thread_start_us":316586,"threads_started":4,"wal-append.queue_time_us":106921,"wal-append.run_cpu_time_us":16709,"wal-append.run_wall_time_us":1633099}]]}
I20260709 12:24:19.585939 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:19.603392 16475 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:19.587154 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:19.604921 16474 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:19.606228 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:19.588466 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:19.609458 16477 raft_consensus.cc:2410] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:24:19.572600 21782 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:19.593122 16476 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 22. Current term is 23. Ops: [19.13-22.14]
I20260709 12:24:19.517809 21784 raft_consensus.cc:697] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:19.621588 21784 consensus_queue.cc:237] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:19.593355 16223 catalog_manager.cc:5697] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:19.577154 22023 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.601027 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:19.537092 21783 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:24:19.550113 21764 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:19.558548 21601 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:19.518307 21426 raft_consensus.cc:2804] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:19.611076 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:19.482102 21691 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:19.619863 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:19.529378 21775 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:19.632767 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 55538.7us
I20260709 12:24:19.635135 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:19.671727 16548 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:19.517272 21970 consensus_queue.cc:1059] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.114s
W20260709 12:24:19.556710 21980 consensus_meta.cc:287] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.501s	user 0.009s	sys 0.002s
I20260709 12:24:19.637161 16441 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:19.604718 21636 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:19.633988 21764 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.636395 16546 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 19.
I20260709 12:24:19.634140 16223 catalog_manager.cc:5697] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 9 to 11, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 11 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:24:19.640996 21922 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:19.687920 16474 raft_consensus.cc:2410] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:24:19.701483 16403 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:24:19.708938 16475 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 19.
I20260709 12:24:19.582333 21899 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:19.711489 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 74458.4us
I20260709 12:24:19.711799 16476 raft_consensus.cc:1240] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:19.602254 21950 consensus_queue.cc:1048] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 12:24:19.714382 16223 catalog_manager.cc:5697] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 5 to 12, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 12 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:19.692314 16469 raft_consensus.cc:2393] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 15: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:19.712280 16441 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:19.717501 16441 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:19.639794 21601 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.635823 21888 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:19.716152 16548 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:24:19.721853 21750 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:24:19.722919 22024 log.cc:927] Time spent T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 12:24:19.723467 22022 log.cc:927] Time spent T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 12:24:19.724426 21776 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:19.718371 21980 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.670184 21426 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:19.616575 22003 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:19.714500 21908 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:19.731304 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:19.639990 21782 raft_consensus.cc:697] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 20 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:19.734228 21601 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:19.710472 21783 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:24:19.753789 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 236054us
I20260709 12:24:19.753947 16223 catalog_manager.cc:5697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 16 to 19. New cstate: current_term: 19 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:19.710692 21711 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:19.640766 22023 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:19.677991 21987 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.763145 16441 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:19.712355 21764 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:19.752256 21980 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:19.733176 21750 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.783859 16367 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:19.735930 21922 raft_consensus.cc:697] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:19.755409 21738 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:19.756878 21426 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:19.765494 21888 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:24:19.833882 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:19.840946 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 57202.7us
I20260709 12:24:19.842164 16367 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:19.845352 16223 catalog_manager.cc:5697] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:19.789012 21776 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:19.797349 21908 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:19.854880 16367 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:19.823297 21783 raft_consensus.cc:697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 23 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:19.780077 21903 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:19.857795 16223 catalog_manager.cc:5697] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 12 to 17, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 17 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:24:19.828802 21750 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:19.871133 16223 catalog_manager.cc:5697] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 6 to 14, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 14 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:24:19.876871 21980 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:19.877678 21980 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:19.878119 21980 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:19.878551 21980 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:19.880692 21691 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:19.884688 21782 consensus_queue.cc:237] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:19.799362 21711 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:19.849781 21920 consensus_queue.cc:1059] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 12:24:19.889096 21920 raft_consensus.cc:3055] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:19.889436 21903 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:19.822530 21987 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:19.874207 21776 raft_consensus.cc:2749] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:19.889453 21920 raft_consensus.cc:740] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:19.891440 21987 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:19.892138 21920 consensus_queue.cc:260] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:19.893286 22003 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:19.894411 21899 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:19.897953 21783 consensus_queue.cc:237] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:19.905922 21921 consensus_meta.cc:287] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.536s	user 0.010s	sys 0.001s
I20260709 12:24:19.906920 21921 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:19.837328 21270 consensus_queue.cc:1048] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:24:19.852569 21738 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:19.929421 16223 catalog_manager.cc:5697] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 10 to 12, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 12 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:19.848912 21888 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:19.948046 16223 catalog_manager.cc:5697] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 3 to 12. New cstate: current_term: 12 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:19.922099 22029 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:24:19.938670 22026 log.cc:927] Time spent T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.182s	user 0.001s	sys 0.004s
I20260709 12:24:19.954407 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:19.963505 17255 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
I20260709 12:24:19.987305 16223 catalog_manager.cc:5697] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:19.958323 21738 raft_consensus.cc:697] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:24:19.988966 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 222089us
W20260709 12:24:19.986171 21731 consensus_meta.cc:287] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.810s	user 0.017s	sys 0.001s
I20260709 12:24:19.999588 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:20.035059 16397 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:24:20.003032 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.039333 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:20.007993 22036 log.cc:927] Time spent T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:24:20.014588 21426 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:20.045259 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:20.015082 21888 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:19.949349 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 421993us
I20260709 12:24:20.031291 21522 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:20.026540 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 171863us
I20260709 12:24:20.026775 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:20.047837 16473 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:20.028649 16223 catalog_manager.cc:5697] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 9 to 11, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 11 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:24:20.027640 16396 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:20.047371 16366 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:20.016072 21707 consensus_meta.cc:287] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.594s	user 0.010s	sys 0.000s
I20260709 12:24:20.050868 17144 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
W20260709 12:24:20.034108 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:20.002144 22032 log.cc:927] Time spent T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:24:20.041921 17266 raft_consensus.cc:1275] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:24:20.068883 16546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37242 (request call id 1549) took 1030 ms. Trace:
I20260709 12:24:20.069765 16546 rpcz_store.cc:276] 0709 12:24:19.038716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.038854 (+   138us) service_pool.cc:224] Handling call
0709 12:24:20.068675 (+1029821us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b5400896a54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:20.068817 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":848764288}
I20260709 12:24:20.072854 21524 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.073529 21524 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:20.075557 21750 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:20.034418 21775 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:20.064002 21782 consensus_queue.cc:1059] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:24:19.986346 21922 consensus_queue.cc:237] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:20.042862 16397 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
W20260709 12:24:20.027833 22000 consensus_meta.cc:287] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.562s	user 0.009s	sys 0.002s
I20260709 12:24:20.047973 16367 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:20.056216 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:20.056988 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:20.058096 16394 raft_consensus.cc:1240] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:24:20.057680 16396 raft_consensus.cc:1275] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 12:24:20.124411 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 64939.7us
I20260709 12:24:20.090075 22041 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.056530 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:20.125855 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:20.035780 22042 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.092525 21980 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:20.139274 22042 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:20.140251 16223 catalog_manager.cc:5697] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:24:20.141099 21980 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:20.090449 22003 consensus_queue.cc:1059] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:24:20.141714 22042 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:20.076200 21764 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:20.143934 21764 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:20.096786 21250 consensus_meta.cc:287] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.580s	user 0.011s	sys 0.000s
I20260709 12:24:20.092643 21707 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:20.146493 16398 raft_consensus.cc:2393] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 17: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:20.096771 21833 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:20.146844 22023 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:20.147205 21833 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:20.147670 21833 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:20.149267 21711 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:20.151947 21859 consensus_meta.cc:287] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.673s	user 0.008s	sys 0.004s
I20260709 12:24:20.092239 21920 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:20.060593 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:20.156674 16469 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 15.
I20260709 12:24:20.157490 16403 raft_consensus.cc:1240] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:20.061661 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:20.027582 22029 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:20.158722 16475 raft_consensus.cc:2393] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 18: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
W20260709 12:24:20.096864 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 46085.4us
I20260709 12:24:20.132157 16394 raft_consensus.cc:1240] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:20.149137 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:20.158654 22029 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:20.061661 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:20.125874 22041 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:20.127135 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:20.160552 16472 raft_consensus.cc:2410] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:24:20.167141 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:20.093753 21648 consensus_meta.cc:287] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.629s	user 0.012s	sys 0.000s
W20260709 12:24:20.096889 21867 consensus_meta.cc:287] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.680s	user 0.008s	sys 0.002s
W20260709 12:24:20.180230 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 52029.2us
I20260709 12:24:20.188319 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:24:20.089707 21587 consensus_meta.cc:287] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.680s	user 0.011s	sys 0.000s
I20260709 12:24:20.060815 21888 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:20.079231 22020 raft_consensus.cc:3055] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:20.183172 16366 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:20.190143 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:20.193750 16470 raft_consensus.cc:2393] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 16: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:20.190824 16441 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:20.087560 21426 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:20.199466 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 151569us
I20260709 12:24:20.200286 16367 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:20.205036 16473 raft_consensus.cc:2410] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:24:20.207693 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:20.161655 21776 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.214793 16473 raft_consensus.cc:1275] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:24:20.146863 21707 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:20.218480 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:20.053893 21522 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:20.222401 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:20.194204 21908 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.223160 17144 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:20.240773 16396 raft_consensus.cc:1240] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:20.081634 21601 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:20.226639 16367 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:20.255188 17264 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
I20260709 12:24:20.256412 16367 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:20.200533 22020 raft_consensus.cc:740] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:20.188225 21763 consensus_queue.cc:1048] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 12:24:20.193035 21648 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:20.259774 16367 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:20.237695 21764 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:20.213762 22039 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:20.208843 21918 consensus_queue.cc:1059] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 12:24:20.260285 21707 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:20.222121 21888 raft_consensus.cc:697] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:20.213189 21859 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:20.076979 21738 consensus_queue.cc:237] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:20.242962 21426 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:20.197407 21867 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:20.270424 16477 raft_consensus.cc:2393] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 17: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:20.213150 22044 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:20.270551 21922 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:20.265695 17144 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:20.280215 17263 raft_consensus.cc:1240] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:20.237920 21776 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:20.275731 21980 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:20.249893 21784 consensus_queue.cc:1048] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:24:20.268193 22003 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:20.273070 21764 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:20.254715 21908 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:20.308499 16516 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:20.307510 17144 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:20.318198 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.327493 16471 raft_consensus.cc:2393] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 17: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:20.307569 21707 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.328894 16394 raft_consensus.cc:1275] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:24:20.329336 16399 raft_consensus.cc:1240] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:20.334578 16473 raft_consensus.cc:1240] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:20.304843 21903 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.304939 21980 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:20.307670 21922 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:20.320766 21972 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:20.306612 22023 consensus_queue.cc:1059] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:24:20.326432 21764 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:20.348536 21522 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:20.349802 22049 log.cc:927] Time spent T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 12:24:20.356884 22048 log.cc:927] Time spent T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 12:24:20.360703 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:20.330746 21782 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:20.345166 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:20.345439 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:20.346249 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:20.368772 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:20.342285 21980 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:20.373185 22008 consensus_queue.cc:1048] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:20.375698 21867 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:20.309538 22044 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:20.374670 21764 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:20.341228 22058 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:20.383154 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:20.386260 16476 raft_consensus.cc:1275] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 12:24:20.342484 21922 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:20.339720 21903 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:20.457655 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:20.431344 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:20.425453 21776 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:20.388746 22041 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:20.419888 22044 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:20.399255 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:20.438561 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:20.472308 16472 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 18.
I20260709 12:24:20.438830 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:20.336930 21707 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:20.363054 21972 raft_consensus.cc:697] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:24:20.440500 22038 log.cc:927] Time spent T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.430s	user 0.002s	sys 0.004s
I20260709 12:24:20.442301 21764 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:20.445901 22051 log.cc:927] Time spent T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 12:24:20.446731 17255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45224 (request call id 1383) took 1193 ms. Trace:
I20260709 12:24:20.446527 22029 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:20.448225 21648 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:20.452863 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:20.453156 21927 raft_consensus.cc:3055] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:20.454566 16397 raft_consensus.cc:1240] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:24:20.456898 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:20.387925 21888 consensus_queue.cc:237] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:20.457368 16476 pending_rounds.cc:85] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 9
I20260709 12:24:20.463052 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:20.363317 21763 consensus_queue.cc:1059] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.082s
W20260709 12:24:20.463826 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 172290us
I20260709 12:24:20.465286 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:20.478758 16366 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:20.480218 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:20.486171 16367 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:20.486646 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:20.487118 17255 rpcz_store.cc:276] 0709 12:24:19.253108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.276356 (+ 23248us) service_pool.cc:224] Handling call
0709 12:24:20.446532 (+1170176us) spinlock_profiling.cc:243] Waited 438 ms on lock 0x7b540005f854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:20.446667 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1007095040}
I20260709 12:24:20.487550 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:20.490356 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:20.507218 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.521934 16401 raft_consensus.cc:2393] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 15: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:20.522017 16399 raft_consensus.cc:2393] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 18: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:20.491356 16476 pending_rounds.cc:107] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 15.10
W20260709 12:24:20.522646 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 44028.9us
I20260709 12:24:20.523192 16476 raft_consensus.cc:2889] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:20.522521 16474 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:20.524224 16442 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:20.480765 21707 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:20.526324 16367 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:20.527127 22039 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:20.527956 21972 consensus_queue.cc:237] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:20.528786 21778 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:20.529294 21778 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:20.529693 21833 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:20.531402 21778 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:20.532506 21833 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:20.532529 16400 raft_consensus.cc:1240] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:20.488116 21776 consensus_queue.cc:1059] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:24:20.533651 21769 consensus_queue.cc:1059] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:20.535622 16544 raft_consensus.cc:2393] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 14: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:20.538141 16367 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:20.538859 22041 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:20.539527 22041 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:20.540722 16367 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:20.541425 22041 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:20.542830 16367 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
W20260709 12:24:20.544941 22059 log.cc:927] Time spent T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:24:20.546559 21908 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:20.546801 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:20.547840 16402 raft_consensus.cc:2393] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 17: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:20.549007 22041 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:20.461874 21980 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:20.554225 21764 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:20.554785 21764 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:20.555166 21764 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:20.558241 22041 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:20.562929 16474 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:24:20.581302 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:20.582726 16396 raft_consensus.cc:2393] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 14: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
W20260709 12:24:20.584509 22070 log.cc:927] Time spent T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:24:20.586403 21763 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.586939 21763 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:20.589114 21763 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:20.483187 21750 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:20.635258 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:20.484073 21859 raft_consensus.cc:697] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:20.636135 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:20.645159 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:20.633482 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 90707.8us
I20260709 12:24:20.660703 16477 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:24:20.613054 22043 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:20.618571 21922 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:20.661099 21859 consensus_queue.cc:237] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:20.661932 16514 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:20.662663 21999 raft_consensus.cc:3055] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:20.663123 21922 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:20.663170 16367 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:20.664101 16547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1562) took 1105 ms. Trace:
I20260709 12:24:20.589556 22045 raft_consensus.cc:3055] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:20.664992 22045 raft_consensus.cc:740] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:20.665675 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:20.666024 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:20.667047 21964 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:20.667495 21964 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:20.667855 21964 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:20.668354 17266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 1382) took 1420 ms. Trace:
I20260709 12:24:20.668519 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:20.668751 17266 rpcz_store.cc:276] 0709 12:24:19.252732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.265076 (+ 12344us) service_pool.cc:224] Handling call
0709 12:24:19.265378 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.064263 (+798885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.668127 (+603864us) spinlock_profiling.cc:243] Waited 776 ms on lock 0x7b54005f5854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:20.668285 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1783893888}
I20260709 12:24:20.670994 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:20.671671 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:20.663123 21999 raft_consensus.cc:740] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:20.672899 22041 raft_consensus.cc:2749] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:20.664446 16547 rpcz_store.cc:276] 0709 12:24:19.558278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.558408 (+   130us) service_pool.cc:224] Handling call
0709 12:24:19.595535 (+ 37127us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:19.845122 (+249587us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:19.845154 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:20.085753 (+240599us) spinlock_profiling.cc:243] Waited 3.34 ms on lock 0x7b2c000ab488. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:24:20.086185 (+   432us) log.cc:844] Serialized 26 byte log entry
0709 12:24:20.194545 (+108360us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:20.194602 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.663789 (+469187us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b54008c55d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:20.663825 (+    36us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:20.663856 (+    31us) raft_consensus.cc:1685] finished
0709 12:24:20.663865 (+     9us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:20.664048 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1942,"raft.queue_time_us":424,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":9036,"spinlock_wait_cycles":580825600,"thread_start_us":108001,"threads_started":1,"wal-append.queue_time_us":189850}
I20260709 12:24:20.678256 21984 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:20.657356 16366 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:20.678787 21984 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:20.680373 22041 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:20.653288 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:20.489789 21927 raft_consensus.cc:740] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:20.694242 16366 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
W20260709 12:24:20.636269 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 112093us
W20260709 12:24:20.685736 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.687932 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:20.696943 16477 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:24:20.699018 21731 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.698942 21515 consensus_queue.cc:1059] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:24:20.699776 21731 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:20.700155 21515 raft_consensus.cc:3055] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:20.700624 21515 raft_consensus.cc:740] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:20.701375 21515 consensus_queue.cc:260] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:20.702288 16471 raft_consensus.cc:1240] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 17 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:24:20.702472 21515 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:20.703289 16399 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:24:20.705807 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:20.697932 16442 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:20.680884 22041 raft_consensus.cc:697] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:20.591645 22003 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:20.589423 22029 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:20.494059 21601 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:20.720330 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:20.718135 16442 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:20.699359 22043 raft_consensus.cc:697] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:20.719638 21987 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.710094 21922 consensus_queue.cc:1048] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:24:20.724932 16403 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:24:20.724932 16396 raft_consensus.cc:1240] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 15. Current term is 16. Ops: []
W20260709 12:24:20.740088 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.740993 16547 raft_consensus.cc:2393] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 17: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:20.748890 16515 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:20.754566 16442 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:20.760628 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:20.764701 16441 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:24:20.840167 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 85703.7us
I20260709 12:24:20.722379 21750 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:20.842203 16442 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:20.852483 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 87891.9us
I20260709 12:24:20.768997 16515 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:20.839250 21270 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:20.834237 21778 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:20.843813 21763 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:20.893594 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 186759us
I20260709 12:24:20.748136 22045 consensus_queue.cc:260] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:20.887748 21601 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:20.732985 21903 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:20.747817 21776 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:20.898828 16396 raft_consensus.cc:1240] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:20.747786 22008 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:24:20.903625 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 134721us
I20260709 12:24:20.747766 21987 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:20.849030 21738 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:20.896250 22056 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:24:20.929067 22073 log.cc:927] Time spent T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a: Append to log took a long time: real 0.086s	user 0.002s	sys 0.000s
I20260709 12:24:20.750399 22003 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.162s
I20260709 12:24:20.732019 22041 consensus_queue.cc:237] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:20.898141 21524 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:20.912050 21846 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:20.934412 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:20.938472 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:20.938732 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:20.901741 21750 consensus_queue.cc:1048] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
W20260709 12:24:20.912969 21783 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.895s	user 0.013s	sys 0.025s
I20260709 12:24:20.940694 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:20.941587 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:20.995802 17255 raft_consensus.cc:2410] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 23 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:24:20.839433 21927 consensus_queue.cc:260] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:20.942889 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:20.944104 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:20.943542 16477 raft_consensus.cc:1240] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:20.939935 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:20.900977 21270 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:20.989596 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:20.990249 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:20.734248 21999 consensus_queue.cc:260] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:20.993798 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:20.998189 17258 raft_consensus.cc:2393] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 15: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:20.999889 17270 raft_consensus.cc:2393] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 13: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:21.003118 17264 raft_consensus.cc:2393] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 16: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:20.915382 21763 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:21.031910 17254 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 15.
I20260709 12:24:21.032147 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:21.036252 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:20.994855 21783 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.992s	user 0.014s	sys 0.026s
I20260709 12:24:21.041790 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:21.042207 16545 raft_consensus.cc:1275] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:24:21.042513 16551 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
I20260709 12:24:20.987178 22056 raft_consensus.cc:697] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:21.050127 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:21.050748 17270 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 18.
I20260709 12:24:21.044804 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:21.029520 21996 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:20.996290 21738 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.922538 22043 consensus_queue.cc:237] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:21.049039 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:20.989768 22023 consensus_queue.cc:1059] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 12:24:20.988998 21846 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:21.052384 16395 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 23 LEADER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 22. Current term is 23. Ops: [22.14-22.14]
W20260709 12:24:20.989248 21903 consensus_queue.cc:1175] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "52ab50b0237346c1a13ede947f6a63c2" responder_term: 17 status { last_received { term: 17 index: 10 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17. Ops: [16.10-16.10]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:24:21.053285 17264 raft_consensus.cc:2393] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 17: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:21.044554 21927 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:21.059206 22077 log.cc:927] Time spent T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:24:21.073125 21846 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:21.066262 17266 raft_consensus.cc:2410] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:24:21.039253 21524 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:21.075923 21524 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:21.064981 22072 log.cc:927] Time spent T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:24:21.058728 21738 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:21.065544 21522 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:21.078581 21522 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:21.077975 22067 log.cc:927] Time spent T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.181s	user 0.000s	sys 0.003s
I20260709 12:24:21.068081 17254 raft_consensus.cc:1240] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:21.064874 21987 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:21.066596 21922 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:21.076753 22056 consensus_queue.cc:237] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:21.080951 21922 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:21.081354 21524 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:21.058440 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:21.083237 21987 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.089174 21987 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:21.089577 21964 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:21.091588 21987 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:21.091765 21964 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:21.091974 21999 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:21.067445 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:21.092949 17269 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
I20260709 12:24:21.079706 21738 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:21.066804 21996 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:21.095468 21996 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:21.096212 17270 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:24:21.098013 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 247434us
I20260709 12:24:21.098826 16442 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:21.099438 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.077967 21763 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:21.100925 16542 raft_consensus.cc:2393] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 16: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:21.102079 21846 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:21.105309 16400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 1451) took 1120 ms. Trace:
I20260709 12:24:21.105713 16400 rpcz_store.cc:276] 0709 12:24:19.985124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.985302 (+   178us) service_pool.cc:224] Handling call
0709 12:24:19.985649 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.533202 (+547553us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.105109 (+571907us) spinlock_profiling.cc:243] Waited 547 ms on lock 0x7b54004ec0d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:21.105256 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1257207552}
I20260709 12:24:21.106221 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:21.082901 21922 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:21.108637 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:21.109220 16474 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:21.110402 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:21.107013 22042 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:21.114214 21846 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:21.115988 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:21.125190 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:21.181169 21270 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:21.184608 21522 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:21.186229 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:21.186877 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:21.169909 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:21.188099 16398 raft_consensus.cc:2393] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 17: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
W20260709 12:24:21.196913 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 50559.7us
I20260709 12:24:21.198823 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:21.201562 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 72724.6us
I20260709 12:24:21.138266 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:21.129776 17270 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
W20260709 12:24:21.151479 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.205475 16400 raft_consensus.cc:2393] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 20: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:21.128306 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:21.206926 16398 raft_consensus.cc:2393] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 14: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:21.056767 21859 consensus_queue.cc:1048] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 12:24:21.209550 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:21.207665 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:21.215004 16400 raft_consensus.cc:2393] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 16: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:21.135921 16474 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:24:21.217816 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:21.220026 17258 raft_consensus.cc:1275] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:24:21.220702 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:21.218020 16514 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:21.214818 16396 raft_consensus.cc:2376] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 19. Re-sending same reply.
I20260709 12:24:21.216585 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:21.223784 16400 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
I20260709 12:24:21.229184 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:21.240806 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.255614 16515 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:21.256816 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:21.286077 16515 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:21.206903 22068 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:21.290261 16515 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:21.296739 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:21.297868 17258 raft_consensus.cc:2410] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:24:21.297927 22042 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:21.298542  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:21.300441 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:21.300529 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:21.301401 17270 raft_consensus.cc:2393] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 17: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:21.288781 21964 consensus_queue.cc:1059] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 12:24:21.302217 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:21.286216 21769 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:21.303282 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:21.306593 17263 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:24:21.213979 21738 consensus_queue.cc:1059] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 12:24:21.303388 21769 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:21.307570 21769 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:21.307896 21738 raft_consensus.cc:3055] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:21.308293 21738 raft_consensus.cc:740] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:21.309136 21738 consensus_queue.cc:260] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:21.310526 21738 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:21.304136 22068 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:21.313730 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:21.298576 22042 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:21.316922 16367 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:21.222134 21601 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:21.262868 21270 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:21.324671 16367 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:21.332756 16367 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:21.334611 16476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 1153) took 1120 ms. Trace:
I20260709 12:24:21.336062 16476 rpcz_store.cc:276] 0709 12:24:20.214205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.214966 (+   761us) service_pool.cc:224] Handling call
0709 12:24:20.385537 (+170571us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.523737 (+138200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.334332 (+810595us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b540080b6d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:21.334529 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":373745408}
I20260709 12:24:21.344033 16367 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:21.340376 21764 consensus_meta.cc:287] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.784s	user 0.005s	sys 0.008s
I20260709 12:24:21.286073 21996 consensus_queue.cc:1059] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
W20260709 12:24:21.349634 16369 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.011s	sys 0.030s
W20260709 12:24:21.369670 16369 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.011s	sys 0.030s
I20260709 12:24:21.372784 17143 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:21.349076 16367 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:21.341905 21426 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:21.377830 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:21.346549 21972 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.816s	user 0.001s	sys 0.021s
I20260709 12:24:21.335278 21711 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:21.388688 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:21.345225 22041 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:21.332561 21601 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:21.397759 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 48800.9us
I20260709 12:24:21.404117 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:21.340072 21270 raft_consensus.cc:697] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:21.368178 22023 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:21.360945 21707 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:24:21.385758 21972 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.855s	user 0.002s	sys 0.021s
I20260709 12:24:21.350371 22003 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:24:21.428483 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 56745.4us
I20260709 12:24:21.382244 21903 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.369928 21764 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:21.394398 21648 consensus_queue.cc:1059] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.089s
I20260709 12:24:21.410509 16367 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:21.434036 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:21.676548 16396 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
W20260709 12:24:21.679701 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 269143us
I20260709 12:24:21.680184 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:21.682323 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:21.682157 16367 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:21.691488 16402 raft_consensus.cc:1240] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:24:21.697434 16402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 1629) took 1017 ms. Trace:
I20260709 12:24:21.698037 16402 rpcz_store.cc:276] 0709 12:24:20.679635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.896485 (+216850us) service_pool.cc:224] Handling call
0709 12:24:20.914734 (+ 18249us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:21.692926 (+778192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.696882 (+  3956us) spinlock_profiling.cc:243] Waited 776 ms on lock 0x7b5400762fd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:21.697291 (+   409us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1784588672}
W20260709 12:24:21.706884 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.433218 21996 raft_consensus.cc:3055] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:21.755366 21996 raft_consensus.cc:740] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:21.474746 21711 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.758313 21711 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:21.428867 22068 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:21.385790 21987 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:24:21.762266 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:21.783000 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.857663 22039 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:21.858742 22039 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:21.861129 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:21.862381 16472 raft_consensus.cc:2368] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 15. Current term is 16.
I20260709 12:24:21.863828 22041 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:21.864552 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:21.758692 21996 consensus_queue.cc:260] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:21.866500 22029 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:21.867051 21996 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:21.868567 22089 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.869824 22089 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:21.857681 16401 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 19: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:21.871997 21783 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:21.864778 21764 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:21.780766 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:21.864567 16476 raft_consensus.cc:1240] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:21.759393 22008 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:24:21.771335 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:21.572750 22085 log.cc:927] Time spent T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.140s	user 0.002s	sys 0.000s
I20260709 12:24:21.872692 16401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 1480) took 1389 ms. Trace:
I20260709 12:24:21.856504 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:21.778689 21601 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:21.759521 22088 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:21.600740 22033 log.cc:927] Time spent T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 1.588s	user 0.005s	sys 0.003s
I20260709 12:24:21.855213 16367 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:21.598414 21999 consensus_meta.cc:287] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.506s	user 0.009s	sys 0.000s
W20260709 12:24:21.599855 22086 log.cc:927] Time spent T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:24:21.783547 22023 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:21.887603 16517 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.010s	sys 0.029s
W20260709 12:24:21.887847 16400 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.007s	sys 0.004s
I20260709 12:24:21.892694 16477 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
I20260709 12:24:21.893224 22041 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.893935 16475 raft_consensus.cc:1275] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 12:24:21.895203 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:21.896667 22029 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:21.899315 16476 raft_consensus.cc:1275] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:24:21.899570 21783 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:21.588186 22045 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:21.763304 21950 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.880561 16401 rpcz_store.cc:276] 0709 12:24:20.483553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.644787 (+161234us) service_pool.cc:224] Handling call
0709 12:24:21.872317 (+1227530us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b5400ad34d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:21.872573 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2597602176}
I20260709 12:24:21.475641 16550 raft_consensus.cc:1240] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:21.768491 22087 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:21.780673 21707 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:21.770620 21648 raft_consensus.cc:3055] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:24:21.915338 16517 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.010s	sys 0.029s
I20260709 12:24:21.913543 16475 pending_rounds.cc:85] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 10
W20260709 12:24:21.919291 16400 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.007s	sys 0.006s
I20260709 12:24:21.915516 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:21.618448 21984 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:21.520992 17143 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:21.949338 21950 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:21.955505 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:21.957764 16402 raft_consensus.cc:1240] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:21.959409 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:21.960671 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:21.961269 21927 raft_consensus.cc:3055] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:21.961604 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:21.961863 21927 raft_consensus.cc:740] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:21.961930 22089 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:21.436002 21524 consensus_queue.cc:1059] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.489s
I20260709 12:24:21.922899 22008 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:21.963647 16402 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:24:21.965317 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:21.966318 16402 raft_consensus.cc:2393] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 19: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:21.966684 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:21.967864 16403 raft_consensus.cc:1275] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:24:21.968446 22100 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:21.969400 22100 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:21.941577 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 251134us
I20260709 12:24:21.479344 22042 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:21.972961 16367 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:21.974001 21731 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:21.975436 16367 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:21.976437 22100 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:21.976869 22039 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:21.977090 22100 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:21.977506 22039 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:21.934695 21636 raft_consensus.cc:3055] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:21.978168 21636 raft_consensus.cc:740] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:21.979704 22029 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:21.982712 22029 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:21.989300 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:21.940879 21707 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:21.993700 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:21.995489 21707 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:21.789502 21903 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:21.995836 21972 raft_consensus.cc:3055] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:21.996368 16395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 1447) took 2048 ms. Trace:
I20260709 12:24:21.996454 21972 raft_consensus.cc:740] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:21.996884 16395 rpcz_store.cc:276] 0709 12:24:19.948003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.950178 (+  2175us) service_pool.cc:224] Handling call
0709 12:24:19.984443 (+ 34265us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:21.070864 (+1086421us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.995648 (+924784us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54001487d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:21.996164 (+   516us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2455228672}
I20260709 12:24:21.965615 22008 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:21.998456 21903 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:21.942392 22087 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:21.999539 22000 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:22.000422 22000 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:22.001116 22087 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:21.943435 21764 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:21.944509 21908 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:22.002686 21908 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:22.005926 16550 raft_consensus.cc:1275] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 12:24:22.005820 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 484977us
I20260709 12:24:22.006400 21522 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:22.006996 21522 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.007536 21522 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:21.938289 16475 pending_rounds.cc:107] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 12:24:22.009281 16475 raft_consensus.cc:2889] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:22.010066 17142 leader_election.cc:400] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 15. Current term is 16.
I20260709 12:24:22.010432 17142 leader_election.cc:403] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:24:22.011515 17144 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:21.921327 22041 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:22.012517 21587 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 15. Current term is 16.
I20260709 12:24:21.959580 16547 raft_consensus.cc:1275] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:24:21.596211 21980 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:21.998282 21972 consensus_queue.cc:260] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:22.024574 21908 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:21.936468 21750 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:21.930106 21711 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:21.998504 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:22.025254 22023 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:22.026628 16395 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
I20260709 12:24:22.028502 16551 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:22.025842 21750 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:22.033231 21750 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:21.936667 16441 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:22.009227 21987 consensus_queue.cc:1048] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 12:24:21.926805 21783 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:22.057775 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:22.057950 21922 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:21.938081 21648 raft_consensus.cc:740] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:22.004132 22000 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:22.063306 16515 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.017s	sys 0.012s
W20260709 12:24:22.070323 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 135838us
W20260709 12:24:22.071458 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 60230.8us
W20260709 12:24:22.073144 16515 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.018s	sys 0.012s
I20260709 12:24:22.034564 22041 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:22.065891 21750 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:22.077843 21927 consensus_queue.cc:260] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:22.078949 16395 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:22.085007 21648 consensus_queue.cc:260] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:22.086905 21648 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:22.088526 22087 consensus_queue.cc:1048] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:24:22.090476 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 800203us
I20260709 12:24:22.088462 17144 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:22.093858 22087 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:22.095438 22087 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:22.099870 22088 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:22.094758 21927 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:22.102927 22088 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:22.075142 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 61678us
I20260709 12:24:22.106355 17142 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:22.108253 22087 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:22.108413 21769 consensus_meta.cc:287] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.799s	user 0.016s	sys 0.013s
I20260709 12:24:22.110541 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:22.111682 16474 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 23.
I20260709 12:24:22.096539 22098 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:22.112715 22098 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:22.115869 22098 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:22.111279 21769 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:22.117429 21636 consensus_queue.cc:260] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:22.118836 21636 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:22.119606 21972 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:22.120426 16399 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.034s	sys 0.050s
I20260709 12:24:22.120527 21769 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:22.120829 16399 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.803s	user 0.035s	sys 0.050s
I20260709 12:24:22.122002 21426 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:22.123018 16399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 1624) took 1663 ms. Trace:
I20260709 12:24:21.570184 22020 consensus_queue.cc:260] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:22.123553 16399 rpcz_store.cc:276] 0709 12:24:20.459571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.704727 (+245156us) service_pool.cc:224] Handling call
0709 12:24:20.706833 (+  2106us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.707064 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:24:20.707748 (+   684us) log.cc:844] Serialized 14 byte log entry
0709 12:24:21.312283 (+604535us) spinlock_profiling.cc:243] Waited 1.34 ms on lock 0x7b1800064720. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d99455 00007f2875e81060 00007f2875e93716 00007f2875ea4287 00007f2875ea4208
0709 12:24:22.121376 (+809093us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:22.121468 (+    92us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:22.121607 (+   139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:22.122748 (+  1141us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:22.122957 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":116,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":2048,"raft.queue_time_us":41304,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":72084,"spinlock_wait_cycles":3087104,"thread_start_us":808808,"threads_started":1}
I20260709 12:24:22.125387 22020 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:22.127204 16399 raft_consensus.cc:1240] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:22.128167 21995 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:22.128997 21995 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:22.132704 21999 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:22.133258 21999 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:22.133671 21999 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:22.134214 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 61925.2us
I20260709 12:24:22.134536 16402 raft_consensus.cc:1240] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:22.135648 16441 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:22.135715 21950 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:22.137283 22087 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:22.138048 16441 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:22.139704 17271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 1422) took 2643 ms. Trace:
I20260709 12:24:22.141726 21922 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:22.140220 17271 rpcz_store.cc:276] 0709 12:24:19.495764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.495893 (+   129us) service_pool.cc:224] Handling call
0709 12:24:19.496262 (+   369us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:19.496655 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:19.496686 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:19.516802 (+ 20116us) log.cc:844] Serialized 52 byte log entry
0709 12:24:19.837909 (+321107us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:19.856595 (+ 18686us) log.cc:844] Serialized 14 byte log entry
0709 12:24:19.856878 (+   283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.856987 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:21.600064 (+1743077us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:24:22.139015 (+538951us) raft_consensus.cc:1685] finished
0709 12:24:22.139150 (+   135us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:22.139419 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5504,"prepare.queue_time_us":18276,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":12200,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":26229,"spinlock_wait_cycles":37996928,"thread_start_us":236543,"threads_started":1,"wal-append.queue_time_us":422661}
I20260709 12:24:22.145785 22087 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:24:22.146597 22003 consensus_meta.cc:287] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.672s	user 0.007s	sys 0.015s
I20260709 12:24:22.147306 22003 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:22.148794 16399 raft_consensus.cc:1240] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:21.535812 21270 consensus_queue.cc:237] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:22.168495 16397 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:22.168911 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:22.151775 22023 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:22.163615 21950 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:22.170728 21950 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:22.140085 16441 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:22.164880 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:22.173556 21950 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:22.175776 22087 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:22.176282 22087 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.176751 22087 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:22.183948 21995 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:22.184442 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 44631.6us
I20260709 12:24:22.184827 17142 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:22.188796 21950 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:22.189340 21950 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.189754 21950 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:22.204530 21731 consensus_queue.cc:1048] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:22.205792 21587 consensus_queue.cc:1048] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:22.204665 21922 consensus_queue.cc:1059] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:22.207944 17144 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:22.209611 16395 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:24:22.224432 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:22.229928 16551 raft_consensus.cc:1275] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:24:22.245460 16442 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:22.259280 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:22.272748 21833 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:22.273308 21833 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.273655 21833 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:22.278347 17261 raft_consensus.cc:1275] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:24:22.289116 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:22.254343 21426 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.286478 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:22.286686 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:22.292192 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:22.284451 17269 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
W20260709 12:24:22.264199 16441 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.1:41805 timed out after 1.717s (SENT) [suppressed 16 similar messages]
I20260709 12:24:22.280014 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:22.277127 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:22.291632 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:22.319095 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 93035.9us
I20260709 12:24:22.291023 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:22.286906 21899 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:22.279026 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:22.293473 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:22.284560 21738 consensus_meta.cc:287] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.932s	user 0.008s	sys 0.016s
I20260709 12:24:22.331981 16543 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 19.
I20260709 12:24:22.319540 21764 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:22.276181 21522 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:22.340770 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 99452.4us
I20260709 12:24:22.332688 21927 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:22.345880 21927 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:22.351749 21927 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:22.353216 21764 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:22.357376 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:22.358690 16546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 1207) took 1602 ms. Trace:
W20260709 12:24:22.358965 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 72146.5us
I20260709 12:24:22.359210 16546 rpcz_store.cc:276] 0709 12:24:20.755742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.755924 (+   182us) service_pool.cc:224] Handling call
0709 12:24:20.840476 (+ 84552us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:20.889505 (+ 49029us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.889535 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:20.889991 (+   456us) log.cc:844] Serialized 26 byte log entry
0709 12:24:21.860050 (+970059us) spinlock_profiling.cc:243] Waited 3.45 ms on lock 0x7b18001ab240. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:24:22.118237 (+258187us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:22.118283 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:22.118417 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:22.358167 (+239750us) raft_consensus.cc:1685] finished
0709 12:24:22.358188 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:22.358477 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":48818,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":4321,"spinlock_wait_cycles":7944576,"thread_start_us":160383,"threads_started":1,"wal-append.queue_time_us":1362360}
I20260709 12:24:22.359902 16516 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:22.367607 16516 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:22.370107 17266 raft_consensus.cc:1275] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:24:22.339123 21426 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:22.376933 21927 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:22.377519 21927 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.377920 21927 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:22.380688 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:22.380868 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:22.384121 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:22.386278 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:22.365175 22058 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:24:22.387054 21996 consensus_meta.cc:287] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.519s	user 0.015s	sys 0.000s
I20260709 12:24:22.387483 21764 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:22.387540 22058 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:22.390996 22058 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:22.394032 17258 raft_consensus.cc:2368] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 14. Current term is 15.
I20260709 12:24:22.395056 21522 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:22.396757 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:22.397943 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:22.398646 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:22.399132 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 24 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:22.399629 22087 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:22.400408 17266 raft_consensus.cc:1275] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:24:22.400918 21950 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:22.401423 21999 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:22.411680 21999 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:22.415728 17271 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 19: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
W20260709 12:24:22.402480 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 157164us
I20260709 12:24:22.404423 16442 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:22.405014 21888 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.404417 22058 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:22.425814 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:22.426497 21995 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:22.426951 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:22.427392 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:22.429806 16400 raft_consensus.cc:1275] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:24:22.431143 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:22.431872 17143 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:22.432922 16395 raft_consensus.cc:2410] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 16 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:24:22.426308 21950 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:22.426831 16397 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:22.441403 21987 raft_consensus.cc:3055] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:22.441941 21987 raft_consensus.cc:740] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:22.442588 16394 raft_consensus.cc:1275] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 12:24:22.443109 21987 consensus_queue.cc:260] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:22.446755 16396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 1467) took 1061 ms. Trace:
I20260709 12:24:22.447274 21999 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:24:22.448073 17269 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45224 (request call id 1389) took 1323 ms (client timeout 1717 ms). Trace:
W20260709 12:24:22.448601 17269 rpcz_store.cc:269] 0709 12:24:21.124744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.124889 (+   145us) service_pool.cc:224] Handling call
0709 12:24:22.447850 (+1322961us) spinlock_profiling.cc:243] Waited 893 ms on lock 0x7b540055f854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:22.447992 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2054666496}
I20260709 12:24:22.449858 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:22.456476 21601 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:22.423959 21636 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:24:22.455665 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 114056us
I20260709 12:24:22.457697 21636 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:22.458789 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:22.458925 16442 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:22.460425 22087 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:22.447276 16396 rpcz_store.cc:276] 0709 12:24:21.385147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.938732 (+553585us) service_pool.cc:224] Handling call
0709 12:24:21.979262 (+ 40530us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:22.009853 (+ 30591us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:22.009895 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:22.010470 (+   575us) log.cc:844] Serialized 26 byte log entry
0709 12:24:22.313935 (+303465us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:22.313984 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:22.446000 (+132016us) spinlock_profiling.cc:243] Waited 30 ms on lock 0x7b54004ecad4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:22.446053 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:22.446101 (+    48us) raft_consensus.cc:1685] finished
0709 12:24:22.446118 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:22.446685 (+   567us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4776,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":43324,"spinlock_wait_cycles":69000448,"thread_start_us":298532,"threads_started":1,"wal-append.queue_time_us":386240}
W20260709 12:24:22.462956 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 95518.8us
I20260709 12:24:22.464967 22058 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:22.465698 21987 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:22.467384 22058 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:22.443770 16394 pending_rounds.cc:85] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 9
I20260709 12:24:22.470393 16394 pending_rounds.cc:107] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:24:22.470746 16394 raft_consensus.cc:2889] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:22.468068 21426 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:22.484256 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:22.488700 17269 raft_consensus.cc:2393] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 15: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
W20260709 12:24:22.494076 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.478873 21636 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:22.451858 21833 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:22.502949 16397 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:24:22.506049 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:22.506234 16548 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 18.
I20260709 12:24:22.506130 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:22.507211 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:22.508011 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 243854us
W20260709 12:24:22.509865 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.490890 21601 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:22.495607 22098 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:22.516049 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 96584.9us
I20260709 12:24:22.519464 17264 raft_consensus.cc:2393] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 17: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
W20260709 12:24:22.519548 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 60797.6us
I20260709 12:24:22.514838 16515 leader_election.cc:400] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 14. Current term is 15.
I20260709 12:24:22.523478 16515 leader_election.cc:403] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:24:22.521063 21426 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:22.524927 21426 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:22.527155 21426 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:22.434626 21888 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:22.531306 16396 raft_consensus.cc:1275] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:24:22.536789 16397 raft_consensus.cc:1275] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:24:22.524107 21922 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.512727 22023 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:22.528270 22098 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:22.547606 22020 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.552691 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:22.553844 16469 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:22.558681 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:22.560818 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:22.561954 16472 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 15.
I20260709 12:24:22.562752 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:22.562989 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:22.563416 16477 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:24:22.563882 16475 raft_consensus.cc:2393] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 15: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:22.564515 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:22.564726 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:22.564893 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:22.566005 16470 raft_consensus.cc:1275] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:24:22.571321 16394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 1482) took 1430 ms. Trace:
I20260709 12:24:22.571756 16394 rpcz_store.cc:276] 0709 12:24:21.140692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.140877 (+   185us) service_pool.cc:224] Handling call
0709 12:24:21.202754 (+ 61877us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:22.489581 (+1286827us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:22.571041 (+ 81460us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54004eded4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:22.571243 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2830799232}
I20260709 12:24:22.574342 17143 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
W20260709 12:24:22.576570 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 61939.8us
I20260709 12:24:22.579944 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:22.585620 21984 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:22.597600 21833 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:22.583822 17144 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:22.584848 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:22.618427 16472 raft_consensus.cc:1240] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:22.620025 16516 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:22.626983 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:22.627736 16399 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:22.629436 16442 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:22.630108 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:22.628401 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:22.638437 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 91733.7us
I20260709 12:24:22.639206 16396 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 19: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:22.637012 17263 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 19.
I20260709 12:24:22.579041 22020 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:22.612926 21833 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:22.645560 16394 raft_consensus.cc:2393] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 19: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:22.621078 21999 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:22.650934 22058 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:22.654637 22058 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:22.606012 21922 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:22.658005 16548 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37226 (request call id 1712) took 2312 ms (client timeout 2239 ms). Trace:
I20260709 12:24:22.605899 22023 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:22.658455 16548 rpcz_store.cc:269] 0709 12:24:20.345424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.346015 (+   591us) service_pool.cc:224] Handling call
0709 12:24:22.657755 (+2311740us) spinlock_profiling.cc:243] Waited 2.07 s on lock 0x7b5400283a54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:22.657899 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4753843584}
I20260709 12:24:22.660542 22023 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:22.662230 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:22.663612 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 81543.9us
I20260709 12:24:22.665445 16442 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:22.669260 21750 raft_consensus.cc:2764] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:24:22.677765 16442 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:22.685778 22092 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:24:22.686036 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 66156.1us
I20260709 12:24:22.691273 17143 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:22.692314 21950 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:22.706174 22045 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.706789 22045 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:22.694481 16366 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:22.703007 22058 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:22.646554 21738 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:22.717772 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:22.720072 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 139596us
W20260709 12:24:22.722285 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 71269.7us
I20260709 12:24:22.718639 17271 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 18.
I20260709 12:24:22.702350 21950 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:22.633716 21636 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:24:22.744648 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 92433.7us
I20260709 12:24:22.739513 16366 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:22.736512 22003 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:22.692910 21426 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:22.760857 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:22.761971 16469 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 19.
I20260709 12:24:22.652532 22087 consensus_queue.cc:1048] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.107s
I20260709 12:24:22.769217 16515 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:22.696583 21899 consensus_queue.cc:1048] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 12:24:22.752612 21764 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:22.769745 16397 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 19.
I20260709 12:24:22.777139 16515 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:22.794340 21996 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:22.794821 21996 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:22.795222 21996 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:22.780762 16441 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:22.763062 22003 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:22.813619 21750 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:24:22.814687 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 79226.2us
I20260709 12:24:22.749464 21927 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:22.765470 22103 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:22.836752 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:22.837772 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:22.818953 21750 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:22.800498 21522 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:22.787707 22023 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:22.787048 21999 raft_consensus.cc:697] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:22.880306 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:22.880584 16546 raft_consensus.cc:1240] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:22.827987 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:22.791426 22020 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:22.867810 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 176651us
I20260709 12:24:22.868877 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:22.846609 16399 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:22.766992 21636 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:22.772850 21426 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:22.898793 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:22.904597 16402 raft_consensus.cc:2393] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 19: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:22.899447 21980 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:22.900213 17144 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:22.874646 21750 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:22.895253 16403 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:22.903113 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:22.895855 16396 raft_consensus.cc:2368] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 14. Current term is 15.
I20260709 12:24:22.897650 21636 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:22.876698 22023 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:22.844919 21764 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:22.822135 22003 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:22.934742 16400 raft_consensus.cc:2393] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 17: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:22.935276 22110 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:22.900395 22020 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.936143 21899 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.952520 21899 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:22.936510 21984 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 14. Current term is 15.
I20260709 12:24:22.955001 21899 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:22.955749 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:22.956663 16400 raft_consensus.cc:2393] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 18: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:22.956553 21524 consensus_queue.cc:1048] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:22.958348 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:22.960371 21927 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:22.961861 16396 raft_consensus.cc:1275] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:24:22.963671 21522 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:22.964521 21522 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:22.967001 21426 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:22.967543 21426 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:22.967847 21522 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:22.930263 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 340589us
I20260709 12:24:22.944913 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:22.941952 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:22.943475 22023 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:22.939643 21601 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:22.948643 21995 consensus_meta.cc:287] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.522s	user 0.010s	sys 0.004s
W20260709 12:24:22.950091 22116 log.cc:927] Time spent T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 12:24:22.968686 21950 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:22.995172 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 302245us
I20260709 12:24:22.972451 22110 raft_consensus.cc:697] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:24:23.008841 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 319609us
I20260709 12:24:23.002447 17144 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:22.968057 21426 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:22.979887 21899 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:23.011952 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:23.013556 21522 consensus_queue.cc:1048] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:24:23.013967 22087 consensus_queue.cc:1048] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:23.015506 21738 consensus_queue.cc:1048] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:23.003355 16514 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:22.993947 21995 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:23.013538 21950 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:22.978456 22020 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:23.037483 22115 log.cc:927] Time spent T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
W20260709 12:24:23.039502 22117 log.cc:927] Time spent T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
W20260709 12:24:23.040315 22114 log.cc:927] Time spent T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.224s	user 0.001s	sys 0.001s
I20260709 12:24:23.024175 21950 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.045738 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:23.048765 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 170438us
I20260709 12:24:23.055631 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:23.056502 16548 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 18.
I20260709 12:24:22.991828 21601 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:23.068521 21995 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:24:23.076015 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 109427us
I20260709 12:24:23.055130 21950 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:23.078750 16441 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:23.060000 21524 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:23.069187 21995 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:23.096740 16441 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:23.097369 22128 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:23.101013 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:23.104743 22119 log.cc:927] Time spent T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:24:23.083334 22120 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:23.054591 22087 consensus_queue.cc:1048] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:24:23.053074 21899 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:23.088577 21738 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:23.121490 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:23.124470 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:23.123839 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:23.122361 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:23.051765 21999 consensus_queue.cc:237] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:23.128285 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 50456.1us
I20260709 12:24:23.136296 16366 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:23.134449 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:23.137552 16548 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:23.139815 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:23.142148 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 78641.4us
I20260709 12:24:23.126505 21738 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.121868 21980 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:23.145697 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:23.147305 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:23.149924 16367 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:23.151176 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:23.151365 17264 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:23.147876 17261 raft_consensus.cc:2376] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15. Re-sending same reply.
W20260709 12:24:23.157150 22121 log.cc:927] Time spent T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:24:23.158519 21908 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:23.158422 22126 consensus_queue.cc:1048] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 12:24:23.162508 21750 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:23.163838 16469 raft_consensus.cc:2410] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:24:23.150148 21980 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:23.166707 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:23.134547 22133 consensus_queue.cc:1059] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:24:23.167749 21995 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:23.168490 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:23.169490 21601 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:23.122884 21524 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:23.177557 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:23.178936 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:23.148608 21738 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:23.182013 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:23.183166 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:23.183992 16472 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 19: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:23.184230 21899 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:23.140499 21996 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:23.128803 22128 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:23.196388 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:23.197057 16472 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:23.167785 21764 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:23.198748 17270 pending_rounds.cc:85] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8: Aborting all ops after (but not including) 13
I20260709 12:24:23.125366 22110 consensus_queue.cc:237] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:23.197547 16475 pending_rounds.cc:85] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 8
I20260709 12:24:23.202365 16475 pending_rounds.cc:107] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 12:24:23.202692 16475 raft_consensus.cc:2889] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:23.199945 17270 pending_rounds.cc:107] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8: Aborting uncommitted NO_OP operation due to leader change: 22.14
I20260709 12:24:23.203451 17270 raft_consensus.cc:2889] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:23.200603 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 103950us
I20260709 12:24:23.208503 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:23.209182 16441 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:23.212179 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:23.216621 21972 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.217211 21972 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:23.217967 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:23.218323 22132 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.218835 22132 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:23.219456 21972 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:23.180104 22126 consensus_queue.cc:1059] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:24:23.221079 22126 raft_consensus.cc:3055] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:23.221436 22126 raft_consensus.cc:740] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:23.222535 21888 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:23.222366 22126 consensus_queue.cc:260] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:23.223713 22126 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:23.224471 22131 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:23.225078 22131 raft_consensus.cc:697] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 19 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:23.225143 17264 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:24:23.226075 22131 consensus_queue.cc:237] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:23.191248 22045 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:23.239221 16546 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:24:23.251467 22132 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:23.256584 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 120403us
I20260709 12:24:23.210245 22133 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:23.228628 22088 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:23.262949 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:23.228875 21426 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:23.268227 16548 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:23.224738 21922 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:23.257764 21764 raft_consensus.cc:3055] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:23.260108 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:23.277483 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:23.275425 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 66367.4us
I20260709 12:24:23.278497 16441 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:23.279803 16514 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:23.284405 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:23.285405 16546 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:24:23.286509 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:23.292351 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.293180 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:23.285661 16401 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:23.295163 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:23.286567 16548 raft_consensus.cc:2393] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 14: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:23.297320 21524 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:23.300521 21964 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:23.301467 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:23.309767 17143 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:23.303416 21738 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:23.276279 21764 raft_consensus.cc:740] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:23.299710 21950 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:23.307029 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 149844us
I20260709 12:24:23.292910 22088 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:23.311830 16441 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:23.295648 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:23.318279 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:23.304934 22098 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:23.313848 16473 raft_consensus.cc:1275] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:24:23.314553 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:23.290987 21750 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:23.325274 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:23.308688 16514 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:23.324165 16441 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:23.330816 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:23.331753 16476 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:23.336896 22128 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:23.325663 22136 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:23.325768 22133 consensus_queue.cc:1048] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 12:24:23.342283 21950 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:23.336807 21996 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:23.343271 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:23.450369 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 126245us
I20260709 12:24:23.451077 16469 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:23.451431 16441 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:23.452107 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:23.451467 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:23.353104 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 24 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:23.460418 22003 consensus_meta.cc:287] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.503s	user 0.011s	sys 0.000s
I20260709 12:24:23.461156 22003 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:23.352725 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:23.453173 16542 raft_consensus.cc:2376] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 18. Re-sending same reply.
I20260709 12:24:23.467664 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:23.468771 16546 raft_consensus.cc:2410] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
W20260709 12:24:23.470254 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 160558us
I20260709 12:24:23.474123 22111 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:23.476999 22103 consensus_queue.cc:1048] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:24:23.479374 22142 log.cc:927] Time spent T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 12:24:23.481103 22134 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:23.350268 22120 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:23.482915 16367 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:23.445714 22023 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:23.485411 16367 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:23.496628 16367 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:23.464051 22003 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:23.484346 22133 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:23.488767 22128 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.348280 21738 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:23.521550 16516 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:23.523979 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:23.528865 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:23.522380 16367 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:23.535789 16400 raft_consensus.cc:2393] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 19: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:23.513087 22126 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:23.539487 22136 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:23.526739 16516 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:23.523949 22131 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:23.537775 16367 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:23.540452 22133 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:23.540918 22126 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:23.540918 21769 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:23.542526 16516 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:23.543413 16367 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:23.518607 22128 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:23.548979 22131 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:23.550794 22128 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:23.551189 22131 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:23.521137 22120 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.551854 22128 raft_consensus.cc:2804] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:23.552269 22128 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.552021 22120 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:23.552805 22128 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:23.553469 16516 leader_election.cc:400] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 14. Current term is 15.
I20260709 12:24:23.554391 22120 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:23.526110 22098 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:23.537943 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:23.556632 16395 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 16.
I20260709 12:24:23.556919 21980 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:23.557585 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:23.558355 16395 raft_consensus.cc:2393] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 18: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:23.559268 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:23.559785 16395 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 18.
I20260709 12:24:23.559902 21980 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:23.545727 21769 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:23.537412 16441 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:23.555990 16516 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:23.561785 16441 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:23.562040 21769 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:23.563987 16367 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:23.564532 21738 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:23.565191 21738 raft_consensus.cc:697] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:23.566196 21738 consensus_queue.cc:237] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:23.566972 21769 consensus_queue.cc:1048] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:23.568517 22088 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:23.568553 22131 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:23.569231 22088 raft_consensus.cc:697] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:23.569940 16367 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:23.569990 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:23.357237 21764 consensus_queue.cc:260] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:23.571244 21764 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:23.539137 21987 consensus_queue.cc:1048] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 12:24:23.572213 22131 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:23.571374 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:23.573482 17143 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:23.574801 21964 consensus_queue.cc:1048] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:24:23.577057 21950 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:23.577194 17142 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:23.577366 17143 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:23.578117 21522 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:23.578541 21522 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:23.578893 21522 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:23.579192 17142 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:23.581589 16516 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:23.573535 16542 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
I20260709 12:24:23.562739 22128 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:23.561698 22023 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:23.609287 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:23.623661 16546 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:23.619141 16367 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:23.619387 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:23.604619 22138 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:23.649387 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:23.651803 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:23.646463 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:23.653808 16468 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:24:23.608521 21524 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:23.642149 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:23.638275 22020 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:23.644179 16403 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:23.645638 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:23.673558 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:23.675076 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:23.642529 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:23.651556 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:23.682052 16394 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:23.704643 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:23.640575 21922 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:23.747714 16548 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 19.
I20260709 12:24:23.753553 16403 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 19.
I20260709 12:24:23.759461 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:23.764806 16394 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 17.
I20260709 12:24:23.766006 16398 raft_consensus.cc:2410] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:24:23.697476 22131 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.767783 16395 raft_consensus.cc:1240] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
W20260709 12:24:23.768791 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 167265us
I20260709 12:24:23.770594 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:23.570214 22088 consensus_queue.cc:237] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:23.776181 16546 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 19.
I20260709 12:24:23.814479 16367 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:23.708201 21522 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:23.765982 21980 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:23.770799 22125 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:23.856877 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:23.856877 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:23.858422 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:23.864213 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:23.865425 16394 raft_consensus.cc:2393] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 19: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
W20260709 12:24:23.867062 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 287913us
I20260709 12:24:23.869110 16367 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:23.783552 21996 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:23.878445 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:23.881757 17142 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:23.890547 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:23.860787 22126 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:23.777014 22131 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:23.913834 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:23.914549 16546 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:23.893329 16398 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 17.
I20260709 12:24:23.916157 17142 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
W20260709 12:24:23.930066 22150 log.cc:927] Time spent T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 12:24:23.925161 16548 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:59590
W20260709 12:24:23.929782 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 76427.6us
I20260709 12:24:23.899048 22098 raft_consensus.cc:2764] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
W20260709 12:24:23.935108 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 88253.6us
I20260709 12:24:23.935720 16367 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:23.931294 16548 raft_consensus.cc:606] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:23.938622 17142 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:23.937137 16366 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:23.947810 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:23.927369 21648 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:23.968526 16547 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 15.
I20260709 12:24:23.985623 22131 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:24.002897 21996 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:24.003655 21996 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:24.017241 21522 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 24 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:24.034178 16546 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:24:24.034755 17142 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:24:24.043787 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 86693.3us
I20260709 12:24:23.958395 22003 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:24.018214 21833 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:23.977552 22120 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:24.047834 16516 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:24.032927 22098 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:23.958256 22157 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:24.068486 17255 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:24.053027 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:24.047391 21524 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:24.067477 22098 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:24.048425 21996 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:24.106412 22159 log.cc:927] Time spent T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 12:24:24.096485 21524 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:24.124719 16546 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:24:24.134788 16546 raft_consensus.cc:606] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:24.135267 16546 raft_consensus.cc:614] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:24.140980 22137 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:24.141695 22137 raft_consensus.cc:697] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 17 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:24:24.143113 22149 log.cc:927] Time spent T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.372s	user 0.004s	sys 0.004s
I20260709 12:24:24.142686 22137 consensus_queue.cc:237] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:24.132478 22098 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:24.146876 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 195997us
I20260709 12:24:24.078207 22120 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:24.150228 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 213220us
I20260709 12:24:24.137253 21524 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:24.148957 16367 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:24.151618 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:24.151665 16366 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:24.155611 16367 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:24.159374 16366 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:24.166061 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:24.171057 16366 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:24.152990 22166 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:24.158710 22003 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:24.160892 21980 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:24.161305 21764 raft_consensus.cc:2804] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:24.209270 21764 raft_consensus.cc:697] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:24.209408 21980 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:24.210366 21764 consensus_queue.cc:237] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:24.210871 22020 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.211436 22020 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:24.178905 16366 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:24.180737 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:24.185738 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:24.186575 21996 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:24.190470 16546 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:24:24.222340 16546 raft_consensus.cc:606] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:24.222918 16546 raft_consensus.cc:614] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:24.191411 21922 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:24:24.192305 22155 log.cc:927] Time spent T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:24:24.193436 22166 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:24.195333 22120 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:24.200582 22162 consensus_queue.cc:1059] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 12:24:24.201870 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:24.172935 22131 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:24.223990 17254 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 18.
I20260709 12:24:24.179147 22003 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:24.226507 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:24:24.235375 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 56532.7us
I20260709 12:24:24.262472 16442 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:24.235424 17255 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 19.
I20260709 12:24:24.224786 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:24.241035 17142 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:24.265028 16516 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:24.222147 21999 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:24.275431 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:24.279132 22162 consensus_queue.cc:1243] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer 610f4feda11942739236f2b121a4b9c1 log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:24:24.279183 22087 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:24.286475 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:24.286495 22087 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:24.287488 16474 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:24.290171 22087 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:24.280293 22120 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:24.293848 22120 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:24.294230 22120 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:24.295599 21950 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.296218 21950 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:24.297760 16544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1591) took 1234 ms. Trace:
W20260709 12:24:24.299028 22143 log.cc:927] Time spent T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.818s	user 0.004s	sys 0.014s
I20260709 12:24:24.244438 22169 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:24.297945 17270 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 18.
I20260709 12:24:24.282440 16442 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:24.280977 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:24.274487 22003 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:24.327847 16402 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709"
dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:59912
I20260709 12:24:24.298264 16544 rpcz_store.cc:276] 0709 12:24:23.063157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.063304 (+   147us) service_pool.cc:224] Handling call
0709 12:24:23.068176 (+  4872us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:23.257363 (+189187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:23.257392 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:23.548026 (+290634us) spinlock_profiling.cc:243] Waited 2.11 ms on lock 0x7b2c000ab488. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:24:23.548512 (+   486us) log.cc:844] Serialized 26 byte log entry
0709 12:24:23.778508 (+229996us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:23.778632 (+   124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:23.778755 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:24.297447 (+518692us) raft_consensus.cc:1685] finished
0709 12:24:24.297469 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:24.297690 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":69301,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":11822,"spinlock_wait_cycles":4844416,"thread_start_us":383306,"threads_started":2,"wal-append.queue_time_us":489094}
I20260709 12:24:24.328635 16402 raft_consensus.cc:606] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:24:24.329339 16402 raft_consensus.cc:614] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:24.302207 22160 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:24:24.334084 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 51724.4us
I20260709 12:24:24.299877 22170 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:24.335947 16442 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:24.343139 22168 log.cc:927] Time spent T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 12:24:24.337165 22160 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:24.300419 21999 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:24.347581 16442 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:24.337448 22088 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:24.348219 21738 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:24.348110 21980 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:24.357437 22162 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:24.389113 16394 raft_consensus.cc:1240] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:24.389530 21738 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:24.395602 16397 raft_consensus.cc:1275] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:24:24.395709 21738 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:24.396382 16397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 1498) took 1068 ms. Trace:
W20260709 12:24:24.397164 22164 log.cc:927] Time spent T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 12:24:24.397750 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:24.389726 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:24.400070 16399 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:24.346737 21524 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:24.396793 16397 rpcz_store.cc:276] 0709 12:24:23.328204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.328355 (+   151us) service_pool.cc:224] Handling call
0709 12:24:24.391970 (+1063615us) spinlock_profiling.cc:243] Waited 2.97 ms on lock 0x7b5800125198. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:24:24.392169 (+   199us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:24.396111 (+  3942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:24.396303 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6838144}
I20260709 12:24:24.339891 22170 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:24.362923 22160 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:24.367954 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:24.402067 16442 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:24.388581 22173 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:24:24.455830 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 82350.3us
I20260709 12:24:24.438563 22131 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.401813 21980 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:24.447501 22136 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:24.479897 16399 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
W20260709 12:24:24.494987 22180 log.cc:927] Time spent T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:24:24.495658 22170 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:24.497120 21524 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:24.499832 21980 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:24.482028 22166 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:24.484957 22136 raft_consensus.cc:697] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:24.477855 22131 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:24.546152 16441 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:24.537660 21524 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:24:24.553434 22177 log.cc:927] Time spent T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
W20260709 12:24:24.554553 22175 log.cc:927] Time spent T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 12:24:24.555349 22144 log.cc:927] Time spent T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 1.079s	user 0.000s	sys 0.006s
I20260709 12:24:24.557956 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:24.558239 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:24.562760 22183 log.cc:927] Time spent T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:24:24.559214 21524 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:24.565480 22023 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:24.568873 22136 consensus_queue.cc:237] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:24.596184 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 50121.2us
I20260709 12:24:24.550868 21999 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:24.723984 22185 log.cc:927] Time spent T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.222s	user 0.000s	sys 0.000s
W20260709 12:24:24.724910 22181 log.cc:927] Time spent T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.257s	user 0.001s	sys 0.000s
I20260709 12:24:24.715631 22126 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.726809 22126 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:24.727342 22179 log.cc:927] Time spent T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.279s	user 0.000s	sys 0.000s
I20260709 12:24:24.730835 21738 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:24.741755 22120 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:24.718815 16441 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:24.541265 22189 consensus_queue.cc:1059] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.174s
I20260709 12:24:24.712067 22087 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:24.744145 22120 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:24.767925 21750 raft_consensus.cc:3055] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:24.784298 22020 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:24.828215 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:24.785774 16472 raft_consensus.cc:1275] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:24:24.829753 22092 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:24.787075 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:24.788553 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:24.830404 22092 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:24.762354 22125 raft_consensus.cc:993] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1: : Instructing follower 52ab50b0237346c1a13ede947f6a63c2 to start an election
I20260709 12:24:24.832830 22125 raft_consensus.cc:1081] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Signalling peer 52ab50b0237346c1a13ede947f6a63c2 to start an election
I20260709 12:24:24.828698 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:24.834038 16403 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:24.744458 22023 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:24.830931 16470 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:24:24.845017 22128 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:24.845324 22126 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:24.845862 22126 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.846302 22126 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:24.846558 22161 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:24.805918 21636 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:24.847393 22161 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:24.848050 21636 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:24.848690 21833 consensus_queue.cc:1059] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:24.849799 21648 raft_consensus.cc:3055] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:24.850154 21636 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:24.807578 22160 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:24.715269 21980 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:24.851287 17144 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:24.851622 21980 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:24.852650 22157 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:24.850167 21648 raft_consensus.cc:740] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:24.853889 21980 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:24.854511 21648 consensus_queue.cc:260] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:24.859536 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:24.811075 16474 consensus_meta.cc:287] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.523s	user 0.010s	sys 0.000s
I20260709 12:24:24.817051 16400 raft_consensus.cc:1275] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:24:24.860531 16474 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:24.817503 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:24.819126 16394 raft_consensus.cc:1275] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 12:24:24.799189 22175 log.cc:927] Time spent T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 12:24:24.826220 21750 raft_consensus.cc:740] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:24.841110 16396 raft_consensus.cc:1240] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:24.846627 22134 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:24:24.805200 22171 log.cc:927] Time spent T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.576s	user 0.008s	sys 0.006s
I20260709 12:24:24.811342 22167 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:24.856053 21648 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:24.760087 22173 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:24.876238 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:24.876380 22173 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:24.878716 22173 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:24.871039 22134 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:24.879545 16476 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14.
I20260709 12:24:24.878774 22131 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:24.881260 22134 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:24.882294 21950 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:24.882385 21750 consensus_queue.cc:260] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:24.883682 22161 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:24.887151 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:24.887796 16396 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:24:24.797400 22087 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:24.888653 22087 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:24.889492 21524 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:24.893463 21738 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:24.895756 22020 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:24.896287 22020 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:24.896780 16546 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:24:24.897352 16546 raft_consensus.cc:606] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:24.898394 22020 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:24.898571 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:24.894837 16394 raft_consensus.cc:1275] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:24:24.899197 16548 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 18.
W20260709 12:24:24.871304 22181 log.cc:927] Time spent T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:24:24.898612 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:24.901024 16544 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14.
I20260709 12:24:24.901865 16477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1669) took 1294 ms. Trace:
I20260709 12:24:24.907104 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:24.907801 16398 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
I20260709 12:24:24.852152 22160 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:24.875100 16515 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:24.873584 22167 raft_consensus.cc:697] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:24.887984 22131 consensus_queue.cc:1048] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:24.909180 17144 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:24.910920 16515 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:24.915024 16515 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:24.911131 16366 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:24.921069 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 437566us
I20260709 12:24:24.918395 16396 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 19.
I20260709 12:24:24.926561 16403 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
I20260709 12:24:24.902315 16477 rpcz_store.cc:276] 0709 12:24:23.607029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.607164 (+   135us) service_pool.cc:224] Handling call
0709 12:24:23.646678 (+ 39514us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:24.069252 (+422574us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:24:24.099330 (+ 30078us) log.cc:844] Serialized 14 byte log entry
0709 12:24:24.242692 (+143362us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:24.330384 (+ 87692us) log.cc:844] Serialized 52 byte log entry
0709 12:24:24.330579 (+   195us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:24.331710 (+  1131us) log.cc:844] Serialized 14 byte log entry
0709 12:24:24.344381 (+ 12671us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:24.344492 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:24.495768 (+151276us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b180011f800. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:24:24.901614 (+405846us) raft_consensus.cc:1685] finished
0709 12:24:24.901641 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:24.901784 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33492,"prepare.queue_time_us":30232,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":75,"raft.queue_time_us":64248,"raft.run_cpu_time_us":373,"raft.run_wall_time_us":35943,"spinlock_wait_cycles":42043776,"thread_start_us":437756,"threads_started":2,"wal-append.queue_time_us":282102}
I20260709 12:24:24.944200 22202 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:24.945488 22202 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:24:24.946027 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 48695.2us
I20260709 12:24:24.947554 16516 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:24.948865 22202 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:24.952306 16515 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:24.871678 22120 consensus_queue.cc:1059] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 12:24:24.920370 22161 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:24.958240 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:24.958976 17255 raft_consensus.cc:2393] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 15: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:24.958485 22161 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:24.956933 16470 raft_consensus.cc:1275] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:24:24.913784 21636 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:24.957239 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:24.960649 17270 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 16.
I20260709 12:24:24.962210 22186 consensus_queue.cc:1048] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:24.922048 21999 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:24:24.960249 21522 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:24.969393 21522 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:24.974376 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:24.975317 16394 raft_consensus.cc:2393] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 16: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:24.954751 21980 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:24:24.981606 22194 log.cc:927] Time spent T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.103s	user 0.003s	sys 0.000s
I20260709 12:24:24.983176 21980 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:24.983770 21980 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:24.988720 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:24.932353 22023 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:24.959998 22003 consensus_meta.cc:287] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.626s	user 0.007s	sys 0.003s
I20260709 12:24:25.004534 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:25.007720 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 55486.1us
I20260709 12:24:25.009663 16515 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:24.972124 21522 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:24.930008 22110 consensus_queue.cc:1048] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 12:24:24.951759 22134 consensus_queue.cc:1048] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
I20260709 12:24:25.019162 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:25.019970 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.018028 16515 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:25.018792 22023 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:25.012858 22003 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:25.064364 22157 raft_consensus.cc:697] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:25.066272 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:25.066638 16544 raft_consensus.cc:1240] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 19. Current term is 20. Ops: []
W20260709 12:24:25.017403 22098 consensus_meta.cc:287] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.860s	user 0.007s	sys 0.004s
I20260709 12:24:25.064011 16401 raft_consensus.cc:1240] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:25.068060 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:25.070641 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:25.075873 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:25.078085 17144 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:25.090042 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:25.093080 17269 raft_consensus.cc:1275] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:24:25.081926 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:25.016901 22195 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:25.098482 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:25.098855 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:25.099345 16547 raft_consensus.cc:2368] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11.
I20260709 12:24:25.094842 16545 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 19.
I20260709 12:24:25.106933 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:25.107775 17271 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:25.110096 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:25.110103 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:25.110894 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 156582us
I20260709 12:24:25.112229 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:25.112884 16543 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 17.
I20260709 12:24:25.114555 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:25.115073 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:25.115666 17261 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:25.119091 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:25.081292 22023 raft_consensus.cc:697] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 18 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:25.080796 16476 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:34826
I20260709 12:24:25.120442 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:25.120842 16476 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:25.121203 16476 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:25.081111 22206 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:25.096407 21999 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:25.075385 21524 consensus_queue.cc:1048] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 12:24:25.105116 22138 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.083727 22098 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:25.097257 22160 consensus_queue.cc:1059] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
W20260709 12:24:25.154390 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.160547 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:25.151273 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:25.113976 22195 raft_consensus.cc:697] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 19 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:25.158767 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:25.163998 16470 raft_consensus.cc:2468] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
I20260709 12:24:25.120096 21636 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:25.170794 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:25.162050 16441 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:25.172439 16441 leader_election.cc:400] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11.
I20260709 12:24:25.172704 16441 leader_election.cc:403] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:25.127760 22161 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:25.066042 22157 consensus_queue.cc:237] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:25.169603 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 49252.3us
W20260709 12:24:25.173794 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 53290.6us
I20260709 12:24:25.182102 16402 raft_consensus.cc:2368] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 16. Current term is 17.
I20260709 12:24:25.186185 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:25.186311 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:25.096573 22167 consensus_queue.cc:237] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:25.180272 22161 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11.
I20260709 12:24:25.173439 22126 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:25.176966 22173 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:25.173875 22088 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:25.199649 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:25.179283 21999 raft_consensus.cc:697] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:25.211390 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:25.176651 21636 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:25.212621 22204 log.cc:927] Time spent T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:24:25.199162 22088 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:25.196457 22174 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:25.233462 16470 raft_consensus.cc:2368] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 16. Current term is 17.
I20260709 12:24:25.227679 22195 consensus_queue.cc:237] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:25.235551 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:25.196839 22023 consensus_queue.cc:237] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:25.196473 22087 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:25.201509 22173 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:25.229805 17261 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 19.
I20260709 12:24:25.214336 22122 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:25.214805 22190 consensus_queue.cc:1059] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.111s
I20260709 12:24:25.234254 16516 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:25.159225 22138 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:25.238687 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 109725us
W20260709 12:24:25.210839 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 45862.4us
I20260709 12:24:25.250353 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:25.248536 16476 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:25.244397 17271 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 17.
I20260709 12:24:25.252907 17254 raft_consensus.cc:1275] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:24:25.253402 22087 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:25.250218 16468 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:24:25.254279 21769 raft_consensus.cc:3055] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:25.254741 21769 raft_consensus.cc:740] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:25.256783 22202 consensus_queue.cc:1048] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:24:25.256894 21999 consensus_queue.cc:237] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:25.264984 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:25.235338 22088 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:25.262491 16516 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:25.273557 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:25.248260 22173 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:25.277091 16367 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:25.282824 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:25.283406 16476 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:25.283623 17254 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 18.
I20260709 12:24:25.289860 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:25.291631 16367 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:25.255671 21769 consensus_queue.cc:260] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:25.305702 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:25.301710 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:25.268221 22207 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:25.312605 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:25.309841 17269 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 17.
I20260709 12:24:25.279233 22087 raft_consensus.cc:3055] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:25.244868 22174 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:25.312851 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: true dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:25.281994 22208 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:25.309041 16543 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 19.
I20260709 12:24:25.322299 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:25.319804 16394 raft_consensus.cc:1275] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:24:25.319865 16545 raft_consensus.cc:3055] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:25.320624 16476 raft_consensus.cc:2393] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 17: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:25.278448 22202 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:25.323526 16545 raft_consensus.cc:740] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:25.260880 22122 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:25.323519 16394 pending_rounds.cc:85] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 8
I20260709 12:24:25.314023 22133 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:25.347712 16394 pending_rounds.cc:107] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:24:25.346267 16545 consensus_queue.cc:260] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:25.351079 16394 raft_consensus.cc:2889] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:25.315855 22087 raft_consensus.cc:740] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:25.348974 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:25.358094 22212 log.cc:927] Time spent T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:24:25.357895 22087 consensus_queue.cc:260] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:25.359061 22087 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:25.311874 22207 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:25.360574 22207 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:25.321277 21636 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:25.352317 16472 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
I20260709 12:24:25.358291 17254 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 24 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:25.315589 22190 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:25.331545 22128 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:25.356788 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: true dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:25.380650 16394 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:25.349407 16546 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:24:25.386322 22190 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:25.329752 22138 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:25.331802 22202 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:25.361920 22131 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:25.476212 22131 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:25.477929 22207 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:25.337204 22208 raft_consensus.cc:697] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 19 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:25.371930 16545 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:25.479281 22208 consensus_queue.cc:237] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:25.350916 22133 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:25.482820 16396 raft_consensus.cc:1275] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:24:25.482985 22133 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:25.485980 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:25.486670 16477 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:25.486888 16545 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
I20260709 12:24:25.489279 16402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58430 (request call id 1499) took 1121 ms. Trace:
I20260709 12:24:25.489892 16402 rpcz_store.cc:276] 0709 12:24:24.367472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.367689 (+   217us) service_pool.cc:224] Handling call
0709 12:24:25.489022 (+1121333us) spinlock_profiling.cc:243] Waited 738 ms on lock 0x7b54005e8dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:25.489210 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1697172864}
I20260709 12:24:25.495630 22190 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:25.500897 22207 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:25.400652 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:25.364611 21980 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:25.354063 22098 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:25.379694 22128 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:25.462448 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:25.520388 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:25.468709 16546 raft_consensus.cc:606] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:25.521082 16546 raft_consensus.cc:614] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:25.475214 21750 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:25.475675 22202 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:25.478338 22131 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:25.368316 22219 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.507894 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:25.525024 16441 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:25.535547 17269 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 19.
I20260709 12:24:25.508741 17254 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14.
I20260709 12:24:25.509732 16547 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
I20260709 12:24:25.523723 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:25.526230 16394 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 10.
I20260709 12:24:25.528530 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:25.529094 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:25.537559 16442 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:25.541479 16367 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:25.544441 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 227894us
I20260709 12:24:25.548530 16545 raft_consensus.cc:1275] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:24:25.549043 16367 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:25.499728 16225 catalog_manager.cc:5697] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 9 to 11, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 11 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
I20260709 12:24:25.548873 16514 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:25.554306 16367 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:25.529004 22128 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:25.577181 16514 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:25.583473 16548 raft_consensus.cc:1275] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:24:25.585917 16367 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:25.594231 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:25.595561 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:25.544256 22231 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.550232 22131 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:25.610725 22122 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:25.611639 22122 consensus_queue.cc:1048] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:24:25.586524 22202 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:25.600957 16367 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:25.603451 21922 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:25.569228 22098 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:25.569715 21999 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.550789 22162 raft_consensus.cc:2804] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:25.550822 22103 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:25.608924 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:25.613948 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:25.614419 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:25.596529 17271 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 19.
I20260709 12:24:25.613741 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:25.548089 22189 consensus_queue.cc:1048] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:24:25.588774 22020 raft_consensus.cc:2804] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:25.624305 17261 raft_consensus.cc:2393] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 17: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:25.625483 17269 raft_consensus.cc:2393] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 15: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
W20260709 12:24:25.628157 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 46128.7us
I20260709 12:24:25.606002 22186 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:25.606731 22126 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:25.589607 22170 consensus_queue.cc:1048] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:24:25.640995 16441 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:25.639096 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.583501 22133 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:25.646878 21980 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:25.547777 22219 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:25.648977 22232 log.cc:927] Time spent T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:24:25.637209 22020 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:25.626564 21999 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:25.654325 21648 raft_consensus.cc:3055] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:25.623349 21922 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:25.644815 22162 raft_consensus.cc:697] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:25.615197 22231 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:25.648138 22220 log.cc:927] Time spent T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
I20260709 12:24:25.623799 22169 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:24:25.674678 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 54872.4us
I20260709 12:24:25.684120 16441 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:25.663101 16477 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
I20260709 12:24:25.637053 22230 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:25.642632 22120 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:25.687976 16367 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:25.669236 22160 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:25.697474 17254 raft_consensus.cc:1275] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:24:25.677035 22020 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:25.641903 22131 consensus_queue.cc:1059] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 12:24:25.681190 21648 raft_consensus.cc:740] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:24:25.704068 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 143933us
I20260709 12:24:25.701885 16441 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:25.642887 22167 consensus_queue.cc:1048] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 12:24:25.686349 22087 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:25.692934 22195 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.698732 21738 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:25.710923 16367 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:25.713837 16367 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:25.714304 16441 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:25.705854 22169 raft_consensus.cc:697] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 17 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:25.713837 22126 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:25.708402 22161 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:25.708518 21980 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.700975 22120 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:25.704600 22160 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:25.699798 22230 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:25.724591 22170 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:25.725802 22173 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:25.731419 21999 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:25.732631 22157 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:25.732921 22120 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:25.739420 21922 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:25.749608 22227 log.cc:927] Time spent T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.355s	user 0.003s	sys 0.000s
I20260709 12:24:25.751784 22231 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:25.752287 22161 raft_consensus.cc:697] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:25.752566 21738 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:25.752792 22174 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:25.753757 22229 log.cc:927] Time spent T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 12:24:25.757400 22088 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:25.834288 22088 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:25.765038 22169 consensus_queue.cc:237] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:25.836542 16474 raft_consensus.cc:1240] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:25.766579 22219 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:25.713919 22195 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:25.816640 22186 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.838877 22186 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:25.839447 22195 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:25.767760 22133 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:25.841347 22186 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:25.729172 21980 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:25.769171 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.770571 22230 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:25.772612 22160 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:25.725971 22162 consensus_queue.cc:237] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:25.720041 22237 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:25.844813 22237 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:25.845348 22237 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:25.845660 22003 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:25.774626 21769 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:25.737813 22170 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:25.786983 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.848892 22170 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:25.795825 22128 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:25.797986 22157 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:25.799264 22226 log.cc:927] Time spent T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 12:24:25.729395 21648 consensus_queue.cc:260] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:25.814724 16470 raft_consensus.cc:1240] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:25.785624 22201 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.855599 22173 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:25.996883 22201 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:25.927682 22161 consensus_queue.cc:237] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:25.999366 22201 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:25.905553 22223 log.cc:927] Time spent T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.433s	user 0.004s	sys 0.002s
I20260709 12:24:26.000614 16516 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:26.000924 17142 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:25.883633 16477 raft_consensus.cc:1275] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:24:25.875581 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:25.938102 16223 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53622 (request call id 136) took 20961 ms (client timeout 15000 ms). Trace:
I20260709 12:24:25.996461 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:25.996483 22157 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:25.992321 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 278093us
W20260709 12:24:26.013012 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 299313us
I20260709 12:24:26.015646 16366 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:25.953513 16225 catalog_manager.cc:5697] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:26.026842 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:26.031033 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:26.031083 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:26.033599 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:26.033991 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:26.034366 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:26.004163 22169 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.038550 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:26.045020 16544 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:24:26.045588 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:26.045950 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:26.046206 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:26.046808 17258 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:26.047781 16550 raft_consensus.cc:1275] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:24:26.003813 22195 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:26.045327 16542 raft_consensus.cc:1275] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:24:26.050263 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:26.050668 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:26.043135 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:26.043996 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:26.051347 16547 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:24:26.048837 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:26.039203 16225 catalog_manager.cc:5697] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 6 to 8, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 8 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
W20260709 12:24:26.057497 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 56656.2us
I20260709 12:24:26.058403 17142 leader_election.cc:400] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 16. Current term is 17.
I20260709 12:24:26.057794 16225 catalog_manager.cc:5697] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 5 to 6, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 6 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:26.056445 16551 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 19.
I20260709 12:24:26.052806 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:26.058712 17142 leader_election.cc:403] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:24:25.923992 22087 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:26.064579 22206 raft_consensus.cc:993] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1: : Instructing follower 52ab50b0237346c1a13ede947f6a63c2 to start an election
I20260709 12:24:26.065528 21980 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:26.066427 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:26.067736 16542 raft_consensus.cc:1275] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 12:24:26.068018 22225 log.cc:927] Time spent T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.699s	user 0.006s	sys 0.005s
I20260709 12:24:26.068723 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:26.036227 22020 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:26.072751 22243 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:26.073729 22128 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:26.074577 22092 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:26.056437 22201 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:26.099433 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:26.078603 17142 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:26.029649 22186 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:26.087321 16551 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:26.038285 22173 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:26.088258 16441 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
W20260709 12:24:26.097229 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 328999us
I20260709 12:24:26.079916 17258 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 19.
I20260709 12:24:26.099433 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:26.103562 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:26.104570 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:26.118754 16545 raft_consensus.cc:1275] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:24:26.081275 21522 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 16. Current term is 17.
I20260709 12:24:26.117158 16549 raft_consensus.cc:1240] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:26.118847 16225 catalog_manager.cc:5697] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:26.119807 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:26.135773 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:26.136646 16545 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:24:26.099359 22137 raft_consensus.cc:1081] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Signalling peer 52ab50b0237346c1a13ede947f6a63c2 to start an election
I20260709 12:24:26.127769 22246 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:26.099524 22202 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.060679 22169 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:26.106094 22200 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:26.104209 22242 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:26.100489 22237 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:26.148864 16441 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:26.137458 21522 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:26.148038 21738 consensus_queue.cc:1048] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 12:24:26.154979 21522 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:26.157961 21522 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:26.160362 16477 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:34826
I20260709 12:24:26.161108 16477 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:26.161545 16477 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:26.149700 22122 consensus_queue.cc:1048] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 12:24:26.100219 22243 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:26.168902 22237 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:26.171291 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:26.169610 22020 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:26.169956 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:26.004186 16223 rpcz_store.cc:269] 0709 12:24:04.977063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.977383 (+   320us) service_pool.cc:224] Handling call
0709 12:24:25.937741 (+20960358us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:20.572620 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:20.573461 (+   841us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:20.573523 (+    62us) write_op.cc:435] Acquired schema lock
0709 12:24:20.573553 (+    30us) tablet.cc:662] Decoding operations
0709 12:24:20.575434 (+  1881us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:20.575522 (+    88us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:20.575567 (+    45us) tablet.cc:685] Acquiring locks for 18 operations
0709 12:24:20.623357 (+ 47790us) tablet.cc:701] Row locks acquired
0709 12:24:20.623388 (+    31us) write_op.cc:260] PREPARE: finished
0709 12:24:20.623600 (+   212us) write_op.cc:270] Start()
0709 12:24:20.623747 (+   147us) write_op.cc:276] Timestamp: P: 1783599860623542 usec, L: 0
0709 12:24:20.623775 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 12:24:20.624695 (+   920us) log.cc:844] Serialized 6975 byte log entry
0709 12:24:21.536471 (+911776us) op_driver.cc:464] REPLICATION: finished
0709 12:24:21.611051 (+ 74580us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2872d7f0ac 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f28798bfef0
0709 12:24:21.635466 (+ 24415us) write_op.cc:301] APPLY: starting
0709 12:24:21.635590 (+   124us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:21.985500 (+349910us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:21.985562 (+    62us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:22.160193 (+174631us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:22.229098 (+ 68905us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=0,mrs_lookups=0
0709 12:24:22.229155 (+    57us) write_op.cc:312] APPLY: finished
0709 12:24:22.229710 (+   555us) log.cc:844] Serialized 163 byte log entry
0709 12:24:22.229927 (+   217us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:22.230412 (+   485us) write_op.cc:454] Released schema lock
0709 12:24:22.230501 (+    89us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:22.237767 (+  7266us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:22.237877 (+   110us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":975193,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":98597,"apply.run_cpu_time_us":32736,"apply.run_wall_time_us":595798,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79439,"num_ops":18,"prepare.queue_time_us":152476,"prepare.run_cpu_time_us":11924,"prepare.run_wall_time_us":155478,"raft.queue_time_us":494936,"raft.run_cpu_time_us":10562,"raft.run_wall_time_us":245563,"replication_time_us":906907,"spinlock_wait_cycles":26297216,"thread_start_us":687482,"threads_started":4,"wal-append.queue_time_us":215300,"wal-append.run_cpu_time_us":39400,"wal-append.run_wall_time_us":2405155}]]}
I20260709 12:24:26.138872 22201 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:26.149993 22202 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:26.179625 22201 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:26.181197 22202 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:26.181942 22201 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:26.173045 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:26.104744 22231 consensus_queue.cc:1048] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 12:24:26.183177 22169 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:26.183534 22231 consensus_queue.cc:1243] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer 52ab50b0237346c1a13ede947f6a63c2 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:24:26.150360 22200 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:26.172346 22242 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:26.187217 22242 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:26.152606 22246 raft_consensus.cc:697] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:26.189354 22246 consensus_queue.cc:237] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:26.137014 22173 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:26.174099 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:26.172080 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:26.193483 16547 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
I20260709 12:24:26.194730 22173 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:26.184813 22200 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:26.174091 22088 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:26.196834 22088 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:26.197402 22088 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:26.199416 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:26.207546 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:26.209271 16475 pending_rounds.cc:85] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 9
I20260709 12:24:26.210556 16475 pending_rounds.cc:107] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:24:26.211234 16475 raft_consensus.cc:2889] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:26.229859 17143 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:26.228478 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:26.234666 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 127949us
I20260709 12:24:26.243454 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:26.245491 22217 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:24:26.246353 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 128055us
I20260709 12:24:26.251930 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:26.252871 16547 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 17.
I20260709 12:24:26.248193 16477 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:26.257414 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:26.258100 16399 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 20.
I20260709 12:24:26.257452 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:26.261267 16366 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:26.255023 16441 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:26.262302 22023 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.862s	user 0.010s	sys 0.017s
W20260709 12:24:26.262867 22023 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.863s	user 0.010s	sys 0.017s
I20260709 12:24:26.263461 22169 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:26.272658 16547 raft_consensus.cc:2393] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 15: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:26.273408 16394 raft_consensus.cc:1275] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:24:26.259967 16367 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:26.176695 22243 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:26.230738 22020 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.283921 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:26.284509 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:26.285507 16545 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:26.281841 16442 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:26.291730 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:26.305786 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:26.301153 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:26.308049 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: true dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:26.308743 16403 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:26.279323 22254 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:26.293107 22003 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:26.334671 16403 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:24:26.254678 22197 consensus_queue.cc:1048] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 12:24:26.337782 16477 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:26.293735 22134 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:26.310304 21738 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:26.246093 22217 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:26.309849 22166 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:26.343389 16401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 1511) took 2100 ms. Trace:
I20260709 12:24:26.339223 16442 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:26.296800 22242 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:24:26.395226 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 152058us
I20260709 12:24:26.364806 16401 rpcz_store.cc:276] 0709 12:24:24.236938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.237115 (+   177us) service_pool.cc:224] Handling call
0709 12:24:24.258394 (+ 21279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:25.084051 (+825657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.336909 (+1252858us) spinlock_profiling.cc:243] Waited 805 ms on lock 0x7b54008a6bd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:26.337101 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1852028800}
I20260709 12:24:26.293465 22020 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:26.289672 22259 consensus_queue.cc:1048] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 12:24:26.400427 17143 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
W20260709 12:24:26.400473 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 61409.5us
I20260709 12:24:26.313230 21750 consensus_queue.cc:1048] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 12:24:26.399580 16545 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 20.
W20260709 12:24:26.404687 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 169092us
I20260709 12:24:26.361402 21738 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:26.408915 16551 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 18.
I20260709 12:24:26.336194 22170 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:26.407305 17143 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:26.361195 22003 raft_consensus.cc:697] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 19 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:26.413041 16542 raft_consensus.cc:1240] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:26.415167 22126 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:26.421456 22126 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:26.421653 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:26.422330 17261 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:26.423758 22126 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:26.410372 22161 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:26.410996 21524 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:26.414249 21738 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:26.408736 22237 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:26.458113 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:26.459757 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:26.340960 22254 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:26.396587 22134 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:26.462574 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:26.462752 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:26.408929 22243 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:26.413483 22201 consensus_queue.cc:1048] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
W20260709 12:24:26.575404 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 160294us
I20260709 12:24:26.576494 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:26.576963 17143 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:26.586766 22122 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:26.589079 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:26.590131 17269 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 19.
I20260709 12:24:26.587428 22122 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:26.554111 22161 raft_consensus.cc:697] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:26.585114 17261 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:24:26.609805 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:26.562582 22255 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:26.568614 22237 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:26.620798 21750 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.577720 22243 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:26.651229 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:26.670133 17261 raft_consensus.cc:1240] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:26.649540 17255 raft_consensus.cc:1275] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:24:26.750001 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:26.640198 22020 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:26.742044 22244 log.cc:927] Time spent T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.411s	user 0.001s	sys 0.000s
I20260709 12:24:26.641142 22170 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:26.762192 17258 raft_consensus.cc:1275] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:24:26.761193 22266 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:24:26.771802 16546 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.004s	sys 0.004s
I20260709 12:24:26.595340 22003 consensus_queue.cc:237] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:26.776350 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 183700us
I20260709 12:24:26.757107 22126 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:26.729259 21750 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:26.805328 22170 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:24:26.817809 22267 log.cc:927] Time spent T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:24:26.817981 22170 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:26.819697 22270 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:24:26.813551 16472 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34826 (request call id 1935) took 1740 ms (client timeout 2069 ms). Trace:
W20260709 12:24:26.820758 16472 rpcz_store.cc:269] 0709 12:24:25.067637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.067742 (+   105us) service_pool.cc:224] Handling call
0709 12:24:26.807738 (+1739996us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b54002729d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:26.807860 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":651585792}
I20260709 12:24:26.821903 22170 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:26.826090 17143 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:26.818363 22254 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:26.831508 17143 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:24:26.812484 17144 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.003s	sys 0.011s
W20260709 12:24:26.838615 17144 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.003s	sys 0.011s
W20260709 12:24:26.839071 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 591988us
W20260709 12:24:26.838207 16546 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.009s	sys 0.005s
I20260709 12:24:26.762091 22258 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:26.764072 22197 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:24:26.814218 22200 consensus_meta.cc:287] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.619s	user 0.011s	sys 0.001s
I20260709 12:24:26.817507 22266 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:26.922740 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 74146.1us
W20260709 12:24:26.815989 22263 log.cc:927] Time spent T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:24:26.783708 22217 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:26.805609 22242 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:26.816174 22256 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:26.844866 22201 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:26.983043 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:26.984189 16394 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:26.985191 16398 raft_consensus.cc:1275] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:24:26.921937 22087 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:27.003516 16401 raft_consensus.cc:1240] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 14. Current term is 15. Ops: []
W20260709 12:24:26.999176 22088 consensus_meta.cc:287] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.801s	user 0.011s	sys 0.003s
I20260709 12:24:26.921032 22258 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:26.927057 22266 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:26.994836 16403 raft_consensus.cc:1275] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:24:27.004253 17254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 1424) took 1395 ms. Trace:
I20260709 12:24:26.844514 22126 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:26.851796 22270 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:27.071557 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:26.999248 22273 log.cc:927] Time spent T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:24:27.075132 17254 rpcz_store.cc:276] 0709 12:24:25.592692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.592896 (+   204us) service_pool.cc:224] Handling call
0709 12:24:25.615840 (+ 22944us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:25.700238 (+ 84398us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.987901 (+1287663us) spinlock_profiling.cc:243] Waited 81.3 ms on lock 0x7b540055f854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:26.988073 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":186880896}
I20260709 12:24:26.807052 22161 consensus_queue.cc:237] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:27.152781 16399 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:27.181833 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:27.198251 22126 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:27.199196 16403 raft_consensus.cc:2368] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 16. Current term is 17.
I20260709 12:24:27.157066 22270 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:27.200232 22270 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:26.927199 22200 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:27.202860 22200 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:27.155921 22258 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:27.206851 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:27.207139 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:27.207882 16401 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
W20260709 12:24:27.214473 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.989154 22242 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:26.979581 22197 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:27.233160 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 303569us
I20260709 12:24:27.234261 16515 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:27.237390 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:27.238092 16397 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:24:27.241130 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:27.241833 16394 raft_consensus.cc:2393] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 15: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:27.244029 16402 raft_consensus.cc:1240] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:27.074110 22088 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:27.250821 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:27.171797 22136 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:27.254791 16515 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:27.258728 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:27.262665 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:27.264784  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:24:27.264910 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:27.265990 16395 raft_consensus.cc:2393] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 20: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:27.267773  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:24:27.267911 22280 log.cc:927] Time spent T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:24:27.269142 16546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1620) took 1038 ms. Trace:
I20260709 12:24:27.075698 22087 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:27.074793 22255 consensus_queue.cc:1059] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.162s
I20260709 12:24:27.269682 16546 rpcz_store.cc:276] 0709 12:24:26.231246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.231818 (+   572us) service_pool.cc:224] Handling call
0709 12:24:26.232170 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:26.245098 (+ 12928us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:26.245139 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:26.245689 (+   550us) log.cc:844] Serialized 26 byte log entry
0709 12:24:26.852716 (+607027us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:26.852780 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.852905 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:27.268610 (+415705us) raft_consensus.cc:1685] finished
0709 12:24:27.268642 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:27.269046 (+   404us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":417,"raft.run_cpu_time_us":47,"raft.run_wall_time_us":47,"spinlock_wait_cycles":22784,"thread_start_us":593834,"threads_started":1,"wal-append.queue_time_us":606597}
I20260709 12:24:27.278618 16397 raft_consensus.cc:1240] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:27.230367 22237 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:27.180982 22134 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:27.068500 22256 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:27.148980 22278 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:27.285385 16476 raft_consensus.cc:1240] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:27.289851 22276 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:27.289938  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:27.293310 22242 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:27.298745 16395 raft_consensus.cc:1275] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:24:27.270439 22243 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:27.302668 22256 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:27.303814 21738 consensus_meta.cc:287] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.728s	user 0.007s	sys 0.000s
I20260709 12:24:27.305161 22197 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:27.305437 22088 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:27.307134 22087 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:27.270539 22126 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.307919 22126 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:27.309871 22126 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:27.315065 22126 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.316221 22266 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:27.315872 22126 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:27.313405 22270 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:27.318089 22278 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:27.321138 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 66538.4us
I20260709 12:24:27.321789 16515 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:27.281268 22246 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:27.342212 16515 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:27.318997 22126 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:27.349543 22291 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:27.276360 22136 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:27.366976 22270 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:27.349625 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:27.368546 16395 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 19.
I20260709 12:24:27.369304 22136 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:27.351768 22287 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.370330 22287 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:27.372716 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:27.372864 16516 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:27.373466 16397 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 18.
I20260709 12:24:27.374040 16516 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:27.339718 22246 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:27.352607 22274 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:27.353273 16470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 1200) took 2283 ms. Trace:
I20260709 12:24:27.351038 22098 raft_consensus.cc:3055] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:24:27.359822 16515 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:27.367096 22278 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:27.304253 21738 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:27.372782 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:27.392824 16470 rpcz_store.cc:276] 0709 12:24:25.069773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.278861 (+209088us) service_pool.cc:224] Handling call
0709 12:24:25.279124 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.000548 (+721424us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.353028 (+1352480us) spinlock_profiling.cc:243] Waited 518 ms on lock 0x7b54000b5c54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:27.353212 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1191109888}
I20260709 12:24:27.374439 22275 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:27.401430 16515 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:27.395423 22098 raft_consensus.cc:740] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:27.368577 22266 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:27.408550 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:27.409991 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:27.410669 17261 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14.
I20260709 12:24:27.411336 22305 raft_consensus.cc:2764] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:24:27.397084 22291 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:27.394361 22287 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:27.417330 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:27.417145 22098 consensus_queue.cc:260] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:27.402702 22126 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:27.419883 16515 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:27.431691 17269 raft_consensus.cc:2393] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 15: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:27.433040 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:27.436933 17255 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:27.392477 22217 consensus_queue.cc:1059] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 12:24:27.405254 22290 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:24:27.495548 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 80984us
I20260709 12:24:27.533339 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:24:27.495384 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 75582.5us
I20260709 12:24:27.534036 16549 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:27.536610 22293 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.537215 21738 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:27.495384 22003 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.538312 22003 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:27.544795 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:27.545523 16551 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:27.548107 22255 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.548300 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:27.548631 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:27.549168 16548 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:27.548767 22255 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:27.537309 22293 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:27.551301 22255 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:27.552654 22293 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:27.553573 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:27.554538 22291 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:27.554792 17258 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 24 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:27.555094 22003 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:27.437284 22276 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:27.397854 22274 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:27.555096 22291 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:27.557467 22246 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:27.558529 22274 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:27.558720 22291 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:27.559926 22098 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:27.560676 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:27.561923 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:27.562712 16396 raft_consensus.cc:2410] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 15 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:24:27.556496 22278 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:27.555262 22211 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:24:27.585763 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 51146.4us
I20260709 12:24:27.566457 17255 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 20.
I20260709 12:24:27.576537 16516 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:27.586855 16515 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:27.563853 22278 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:27.595765 16516 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:27.683039 16548 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 19.
I20260709 12:24:27.690057 16551 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:27.529212 22231 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:27.552732 22283 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:27.714883 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 175536us
I20260709 12:24:27.718490 17144 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:27.532312 22126 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:27.569430 22020 consensus_queue.cc:1059] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.139s
I20260709 12:24:27.751819 17144 leader_election.cc:400] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 16. Current term is 17.
I20260709 12:24:27.705331 22217 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:27.789233 17144 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:27.619422 22275 consensus_queue.cc:1048] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.124s
I20260709 12:24:27.718240 22231 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:24:27.810228 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 56401.4us
I20260709 12:24:27.811810 17144 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
W20260709 12:24:27.793136 22302 log.cc:927] Time spent T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.257s	user 0.000s	sys 0.001s
I20260709 12:24:27.752946 22243 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.717900 22283 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:27.790771 22237 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:27.804399 22290 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:27.823362 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:27.792346 22217 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:27.828442 22292 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.827735 22288 consensus_queue.cc:1048] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 12:24:27.796900 22270 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:27.802920 22291 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:27.811442 22020 raft_consensus.cc:3055] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:27.855070 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:27.872069 16396 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 24.
I20260709 12:24:27.869789 22270 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:27.856137 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:27.873173 22270 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:27.837628 22283 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:27.818181 22201 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:27.874353 22201 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:27.865796 22276 raft_consensus.cc:697] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:27.874917 22201 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:27.875623 22276 consensus_queue.cc:237] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:27.819842 22266 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:27.878289 22266 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:27.879251 22266 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:27.843509 22297 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:27.859498 22217 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:27.803118 22293 consensus_queue.cc:1059] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 12:24:27.806653 22300 consensus_queue.cc:1048] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 12:24:27.852001 22290 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:27.883834 16474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 1214) took 2583 ms. Trace:
I20260709 12:24:27.865674 22274 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:27.890597 16474 rpcz_store.cc:276] 0709 12:24:25.287446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.485743 (+198297us) service_pool.cc:224] Handling call
0709 12:24:25.486270 (+   527us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:25.837070 (+350800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.870287 (+2033217us) spinlock_profiling.cc:243] Waited 350 ms on lock 0x7b5400a690d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:27.870549 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":805085952}
I20260709 12:24:27.841449 22243 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:27.858197 22237 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:27.880848 22020 raft_consensus.cc:740] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:27.864430 22292 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:27.892572 22290 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:27.921710 16366 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.736s (SENT) [suppressed 3 similar messages]
I20260709 12:24:27.925275 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:27.927989 16395 raft_consensus.cc:2393] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 20: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:27.929350 22299 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:27.930012 22299 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:27.934595 22299 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:27.935493 22126 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:27.941532 22237 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:27.942489 22278 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:27.902251 22274 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:27.944977 22274 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:27.945622 22020 consensus_queue.cc:260] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:27.946975 22020 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:27.947345 22292 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:27.948009 16549 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:27.889417 22297 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:27.949461 22243 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:27.954861 22261 log.cc:927] Time spent T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 1.542s	user 0.002s	sys 0.007s
I20260709 12:24:27.884482 22291 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:27.963400 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 48929.8us
I20260709 12:24:27.974570 16400 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 1516) took 2211 ms. Trace:
I20260709 12:24:27.975214 17144 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:27.975045 16400 rpcz_store.cc:276] 0709 12:24:25.763297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.764137 (+   840us) service_pool.cc:224] Handling call
0709 12:24:25.766851 (+  2714us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:25.842613 (+ 75762us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:25.842644 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:25.843125 (+   481us) log.cc:844] Serialized 26 byte log entry
0709 12:24:25.993304 (+150179us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:25.993349 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.969353 (+1976004us) spinlock_profiling.cc:243] Waited 75.4 ms on lock 0x7b54004ee3d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:27.969434 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:27.969501 (+    67us) raft_consensus.cc:1685] finished
0709 12:24:27.969519 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:27.974506 (+  4987us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1217,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":142,"spinlock_wait_cycles":173347072,"thread_start_us":129885,"threads_started":1,"wal-append.queue_time_us":156343,"wal-append.run_cpu_time_us":1692,"wal-append.run_wall_time_us":1003590}
W20260709 12:24:27.994333 22301 log.cc:927] Time spent T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.191s	user 0.002s	sys 0.000s
W20260709 12:24:27.977861 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 391086us
W20260709 12:24:27.986863 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 390906us
I20260709 12:24:28.004027 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:27.987673 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:28.004815 16544 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 19.
I20260709 12:24:28.004742 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:28.010275 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:28.011220 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 89558.8us
I20260709 12:24:27.992111 22313 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:24:28.018277 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.018661 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:28.019129 17143 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:28.027168 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:28.030273 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.030982 17143 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:28.034451 22274 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:28.012881 21750 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:28.036789 21648 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:28.043323 22291 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:28.012086 21996 consensus_queue.cc:1048] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:24:28.047130 22221 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.003144 22270 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:28.050138 22270 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:28.043550 22217 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:28.047722 22221 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:28.054857 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:28.055122 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:28.056488 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:28.056758 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:28.054294 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:28.060410 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:28.060845 16397 raft_consensus.cc:1240] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:28.061592 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:28.062403 16403 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:24:28.027838 22313 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.078238 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:28.081344 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:28.083096 16551 raft_consensus.cc:2393] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 20: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:28.012454 22315 consensus_queue.cc:1059] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 12:24:28.051077 22297 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:28.053540 21769 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:28.088464 22239 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.091428 22239 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:28.092499 22221 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:28.077384 22314 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:28.095485 22314 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:28.096654 22315 raft_consensus.cc:3055] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:28.101855 22314 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:28.103492 22262 log.cc:927] Time spent T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 1.548s	user 0.024s	sys 0.051s
I20260709 12:24:28.104579 22173 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:28.097256 22315 raft_consensus.cc:740] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:28.105182 22173 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:28.107710 22173 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:28.107070 22315 consensus_queue.cc:260] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:28.088686 21750 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.076946 22313 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:28.108790 21750 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:28.110972 21750 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:28.112066 22243 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:28.112798 16475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 1957) took 1908 ms. Trace:
I20260709 12:24:28.112924 22243 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:28.113278 16475 rpcz_store.cc:276] 0709 12:24:26.204593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.208551 (+  3958us) service_pool.cc:224] Handling call
0709 12:24:26.209012 (+   461us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:26.212408 (+  3396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:26.212441 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:26.236124 (+ 23683us) log.cc:844] Serialized 52 byte log entry
0709 12:24:26.334892 (+ 98768us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:26.616506 (+281614us) log.cc:844] Serialized 14 byte log entry
0709 12:24:26.633772 (+ 17266us) log.cc:844] Serialized 14 byte log entry
0709 12:24:26.634166 (+   394us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.634296 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:28.049821 (+1415525us) spinlock_profiling.cc:243] Waited 8.91 ms on lock 0x7b1800175420. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:24:28.102851 (+ 53030us) spinlock_profiling.cc:243] Waited 4.72 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:24:28.112582 (+  9731us) raft_consensus.cc:1685] finished
0709 12:24:28.112610 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:28.112744 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24060,"prepare.queue_time_us":333272,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":50,"raft.queue_time_us":22273,"raft.run_cpu_time_us":391,"raft.run_wall_time_us":20947,"spinlock_wait_cycles":31340032,"thread_start_us":319002,"threads_started":2,"wal-append.queue_time_us":166484}
I20260709 12:24:28.114204 22254 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:28.077167 22274 raft_consensus.cc:697] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 19 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:28.117031 22274 consensus_queue.cc:237] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:28.115549 22254 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:28.082386 22157 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:28.111716 21769 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:28.125068 17144 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:28.120946 16367 connection.cc:256] RPC call timeout handler was delayed by -1.0342234089998783s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2388s for a response
W20260709 12:24:28.135862 16367 connection.cc:256] RPC call timeout handler was delayed by -1.0103624379999019s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16970000000000002s for a response
I20260709 12:24:28.136560 17143 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:28.137051 16367 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:28.142316 16367 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:28.152789 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:28.155570 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:28.156138 16547 raft_consensus.cc:2368] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 18. Current term is 19.
I20260709 12:24:28.156235 16550 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 14.
I20260709 12:24:28.123313 22217 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:28.085170 22290 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:28.154630 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:28.123659 22243 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:28.161404 16400 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:28.161572 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:28.159227 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:28.159376 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:28.159403 16367 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:28.160037 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:28.164064 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:28.143348 22126 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.165673 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:28.166208 16547 raft_consensus.cc:2393] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 20: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:28.145249 22157 raft_consensus.cc:697] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 16 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:28.167779 16367 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:28.157923 22131 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:28.167443 16544 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 15.
I20260709 12:24:28.169878 16543 raft_consensus.cc:1240] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:28.166208 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:28.164937 22098 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:28.138633 22314 consensus_queue.cc:1059] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:24:28.136539 22201 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:28.160619 22259 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:28.180500 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:28.190752 16547 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 10.
I20260709 12:24:28.167382 22243 raft_consensus.cc:2749] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:28.189401 21769 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:28.188861 22307 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:28.171362 22131 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.194176 16400 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 20.
I20260709 12:24:28.204519 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:28.206027 16397 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 19.
I20260709 12:24:28.203415 22131 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:28.190215 22259 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:28.211067 22259 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:28.175819 22003 consensus_queue.cc:1048] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:24:28.169400 22126 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:28.218899 22289 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.212105 22307 raft_consensus.cc:697] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 16 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:24:28.225580 22322 log.cc:927] Time spent T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:24:28.235703 22290 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:28.237133 22254 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:28.238170 22157 consensus_queue.cc:237] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:28.214649 22304 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:28.239907 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:28.209329 22243 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:28.245203 22289 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:28.248900 16396 raft_consensus.cc:2393] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 16: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:28.249681 22307 consensus_queue.cc:237] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:28.267307 16516 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:28.292151 16396 raft_consensus.cc:1240] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:28.293195 22201 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:28.294996 22239 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:28.297561 22313 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:28.298630 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:28.299234 22259 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:28.299830 22313 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 25 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:28.301960 16549 raft_consensus.cc:2410] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:24:28.305238 22259 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:28.300212 16397 raft_consensus.cc:1240] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 19 LEADER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 18. Current term is 19. Ops: []
W20260709 12:24:28.311671 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 175219us
I20260709 12:24:28.300714 22211 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:28.291983 22266 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:28.297588 22126 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:28.313962 17143 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:28.315734 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:28.309859 22098 raft_consensus.cc:697] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 20 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:28.255141 22304 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:28.317600 17143 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:28.313584 22211 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.320307 17254 raft_consensus.cc:1240] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:28.311980 22313 consensus_queue.cc:1059] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
W20260709 12:24:28.322678 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 154974us
I20260709 12:24:28.318338 22255 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:28.325650 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:28.326897 16395 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:28.322816 22211 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:28.333956 16367 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
W20260709 12:24:28.329345 22251 log.cc:927] Time spent T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 2.016s	user 0.016s	sys 0.018s
I20260709 12:24:28.337857 16367 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:28.340232 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:28.343359 16367 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:28.345978 16475 raft_consensus.cc:2410] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:24:28.347321 16367 leader_election.cc:400] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 18. Current term is 19.
I20260709 12:24:28.354633 16367 leader_election.cc:403] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:28.356309 16547 raft_consensus.cc:1240] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:24:28.357403 16544 raft_consensus.cc:1275] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:24:28.345579 22330 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:28.359421 22133 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:28.359549 22330 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:28.359997 22330 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:28.326655 22255 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:28.362598 22255 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:28.325879 22332 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:28.366673 22285 raft_consensus.cc:3055] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:28.367167 22285 raft_consensus.cc:740] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:28.364877 22020 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:28.369014 16367 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:28.373569 16514 connection.cc:256] RPC call timeout handler was delayed by -1.433407576999798s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2069s for a response
I20260709 12:24:28.375254 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:28.379719 16514 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:28.379912 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:28.364051 22266 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:28.390813 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: true dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:28.392513 16543 raft_consensus.cc:3055] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:28.388749 16367 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:28.394482 16367 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:28.396198 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:28.392809 16543 raft_consensus.cc:740] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:28.353896 22126 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:28.397665 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:28.403400 16474 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 18.
I20260709 12:24:28.407600 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:28.398661 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:28.400125 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:28.402168 16469 raft_consensus.cc:1275] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:24:28.400614 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:28.401798 16470 raft_consensus.cc:1275] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:24:28.411219 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:28.372152 22133 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:24:28.414178 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:28.408663 16475 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
I20260709 12:24:28.414939 17266 raft_consensus.cc:2393] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 17: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:28.397598 16543 consensus_queue.cc:260] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:28.417603 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:28.419000 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:28.420739 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:28.418243 17263 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 19.
I20260709 12:24:28.424211 16477 raft_consensus.cc:1240] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:28.425761 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:28.425980 16395 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:28.427618 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:28.401687 22243 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:28.437386 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:28.442776 16474 raft_consensus.cc:1240] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:28.437462 22243 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:28.444288 22243 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:28.444406 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:28.445540 17264 raft_consensus.cc:1240] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:28.445854 22330 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:28.446872 17269 raft_consensus.cc:1275] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:24:28.447945 22330 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:28.391693 22239 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:28.450294 16543 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:28.405192 22254 raft_consensus.cc:2804] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:28.454804 22254 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.455214 22254 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:28.459925 22304 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:28.453469 16542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 1261) took 2334 ms. Trace:
I20260709 12:24:28.368103 22285 consensus_queue.cc:260] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:28.407305 22126 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:28.391378 22201 consensus_queue.cc:1059] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 12:24:28.466115 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:28.467311 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:28.469085 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:28.448004 22131 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:28.396456 22161 consensus_queue.cc:1059] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 12:24:28.476732 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:28.500386 16474 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 20.
I20260709 12:24:28.479007 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:28.424541 22003 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.481422 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:28.511346 16475 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:28.482240 22255 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:28.385962 22098 consensus_queue.cc:237] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:28.483232 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:28.418941 22230 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:28.487363 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:28.413296 22266 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.385494 22020 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:28.488791 16542 rpcz_store.cc:276] 0709 12:24:26.114646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.114779 (+   133us) service_pool.cc:224] Handling call
0709 12:24:26.412857 (+298078us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.415359 (+  2502us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:28.448597 (+2033238us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b5400366350. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:28.448714 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":656136064}
I20260709 12:24:28.397835 22334 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:28.490267 22256 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:28.494681 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:28.496117 16548 raft_consensus.cc:2393] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 12: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:28.498883 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:28.513062 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:28.513669 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:28.525663 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:28.498332 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:28.483968 22239 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:28.498792 21750 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:28.516422 16468 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:28.527870 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:28.522306 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:28.492594 16514 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:28.493960 22126 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:28.499621 22241 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.533787 16543 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:24:28.522186 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:28.522536 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:28.536352 17255 raft_consensus.cc:1240] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:28.526671 16395 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:24:28.545082 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 52575.1us
I20260709 12:24:28.546909 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:28.542791 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:28.544081 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:28.546404 16470 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 19.
I20260709 12:24:28.515829 22230 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:28.552381 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:28.557080 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:28.557785 17271 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 16.
I20260709 12:24:28.555140 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:28.558681 16401 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 16.
I20260709 12:24:28.559180 16548 raft_consensus.cc:1240] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:28.559723 16442 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:28.459954 22243 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:28.560588 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:28.561510 16476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 1206) took 3404 ms. Trace:
I20260709 12:24:28.561874 16476 rpcz_store.cc:276] 0709 12:24:25.157475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.354739 (+197264us) service_pool.cc:224] Handling call
0709 12:24:26.807861 (+1453122us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:27.288995 (+481134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:28.438325 (+1149330us) spinlock_profiling.cc:243] Waited 477 ms on lock 0x7b54002729d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:28.561337 (+123012us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b54002729d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:28.561455 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4438958592}
I20260709 12:24:28.558164 22161 raft_consensus.cc:3055] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:28.563830 22161 raft_consensus.cc:740] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:28.564728 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:28.567324 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:28.524139 22334 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:28.569402 16548 raft_consensus.cc:1275] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:24:28.568367 16401 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 19.
I20260709 12:24:28.573073 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:28.542209 21750 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:28.574522 16442 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:28.575462 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:28.547382 22314 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:28.578794 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:28.515937 22003 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:28.576764 16514 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:28.566825 22230 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:28.609848 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:28.568813 22243 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:28.545627 22256 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:28.588410 22020 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:28.590142 22239 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:28.593873 22255 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:28.599116 22131 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:28.600152 16477 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 23.
I20260709 12:24:28.601297 21750 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:28.584944 22305 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 12:24:28.606179 16514 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:28.631630 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:28.529469 22266 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:28.581071 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:28.609970 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:28.584807 16395 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 25.
I20260709 12:24:28.617820 22161 consensus_queue.cc:260] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:28.618819 22316 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.631340 22239 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:28.636294 22266 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:28.612682 17258 raft_consensus.cc:1275] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:24:28.563287 22254 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:28.636767 22314 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:28.648906 22314 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:28.649338 22314 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:28.649259 22201 consensus_queue.cc:1059] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:28.650244 22254 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:28.615758 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:28.650648 22201 raft_consensus.cc:3055] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:28.651139 22201 raft_consensus.cc:740] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:28.615782 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:28.617439 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:28.550504 22241 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:28.639583 22161 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:28.637991 16401 raft_consensus.cc:2393] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 15: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:28.653627 16398 raft_consensus.cc:1240] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:28.616783 22216 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:28.654490 22241 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:28.577833 22334 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:28.621493 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:28.624447 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:28.655421 22329 consensus_queue.cc:1059] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:28.656317 17261 raft_consensus.cc:2393] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 17: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:28.656590 22329 raft_consensus.cc:3055] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:28.641153 22256 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:28.657037 22329 raft_consensus.cc:740] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:28.641746 22003 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:28.654940 22216 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:28.638811 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:28.658771 22216 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:28.644919 22239 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.659462 22239 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:28.644110 22133 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 18. Current term is 19.
I20260709 12:24:28.630523 16402 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 18.
I20260709 12:24:28.654838 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:28.652310 17269 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:28.662568 16401 raft_consensus.cc:2368] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 24. Current term is 25.
I20260709 12:24:28.663486 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:28.642630 22316 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:28.668568 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:28.669669 17266 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:24:28.672490 17142 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:28.674431 16442 leader_election.cc:400] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 24 pre-election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 24. Current term is 25.
I20260709 12:24:28.674841 16442 leader_election.cc:403] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:28.628923 22342 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:28.675595 17142 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:28.677011 17266 raft_consensus.cc:1275] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:24:28.694657 22283 raft_consensus.cc:3055] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:28.694805 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:28.695171 22283 raft_consensus.cc:740] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:28.652086 22201 consensus_queue.cc:260] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:28.694044 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:28.691530 22241 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:24:28.701624 16395 raft_consensus.cc:1240] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 19. Current term is 20. Ops: [18.11-19.12]
I20260709 12:24:28.708568 17142 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:28.714723 16395 raft_consensus.cc:1275] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:24:28.717581 16395 raft_consensus.cc:1240] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:28.719395 16395 raft_consensus.cc:1240] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:24:28.721037 16395 raft_consensus.cc:1275] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:24:28.726173 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:28.726671 16395 raft_consensus.cc:1240] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:28.729259 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:28.730227 22340 consensus_queue.cc:1059] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 12:24:28.691829 22342 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.733500 17142 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:28.735603 22298 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:28.735390 22211 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:28.747516 17269 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 20.
I20260709 12:24:28.739420 22314 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:28.741669 22316 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:28.742240 22201 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:28.672765 22270 consensus_queue.cc:1048] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 12:24:28.676247 22266 consensus_queue.cc:1048] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 12:24:28.754662 16550 raft_consensus.cc:2393] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 21: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:28.725163 22297 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:28.756608 21922 consensus_queue.cc:1059] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:28.757462 22211 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:28.758234 21922 raft_consensus.cc:3055] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:28.758646 21922 raft_consensus.cc:740] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:28.758569 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:28.759727 21922 consensus_queue.cc:260] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:28.761142 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:28.761445 21922 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:28.749148 22298 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.764295 22298 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:28.774199 22314 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:28.677866 22307 consensus_queue.cc:1059] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.075s
I20260709 12:24:28.658273 22329 consensus_queue.cc:260] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:28.778574 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 113335us
I20260709 12:24:28.778779 17271 raft_consensus.cc:1240] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:28.776888 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:28.777735 17261 raft_consensus.cc:1240] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:28.785223 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:28.793928 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:28.796041 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.749545 22342 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:28.732249 22173 consensus_queue.cc:1048] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 12:24:28.775676 22297 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:24:28.828001 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 101297us
I20260709 12:24:28.794157 22126 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:28.799582 22216 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:28.815668 21750 consensus_queue.cc:1059] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 12:24:28.813773 22329 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:28.814642 22297 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:28.837807 17143 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:28.861378 17143 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:28.862231 22340 raft_consensus.cc:3055] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:28.866034 22340 raft_consensus.cc:740] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:28.872174 22255 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:28.864307 22266 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:28.872588 22340 consensus_queue.cc:260] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:28.874555 22340 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:28.875419 22289 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:28.877475 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.877938 22348 consensus_queue.cc:1048] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:28.879561 22348 consensus_queue.cc:1059] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:24:28.881028 22343 log.cc:927] Time spent T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 12:24:28.881556 22122 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:28.882716 22315 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:28.884632 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 53802.9us
I20260709 12:24:28.887778 16441 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:28.888265 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:28.885941 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:28.890129 16476 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:28.892695 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 63358.1us
I20260709 12:24:28.895295 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:28.884418 22126 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:28.893076 22289 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:28.904763 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:28.899266 22216 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:28.903087 22351 raft_consensus.cc:3055] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:28.912985 22351 raft_consensus.cc:740] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:28.913707 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:28.887859 22350 consensus_queue.cc:1048] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:24:28.911585 22239 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:28.917449 16472 raft_consensus.cc:2393] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 21: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:28.919502 22239 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:28.923331 22195 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.923930 22195 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:28.864953 22346 consensus_queue.cc:1059] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:24:28.925608 22346 raft_consensus.cc:3055] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:28.925815 16441 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:28.926041 22346 raft_consensus.cc:740] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:28.927382 16441 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:28.928727 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.933398 16476 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:24:28.934191 16469 raft_consensus.cc:1240] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:28.937037 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:28.938760 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:28.939570 16468 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 18.
I20260709 12:24:28.939774 22354 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:28.926218 22195 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:28.971148 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:28.971706 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:28.972317 16551 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:28.927076 22346 consensus_queue.cc:260] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:28.977295 22346 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:28.920053 22230 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:28.959664 16399 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58430 (request call id 1526) took 1888 ms (client timeout 1542 ms). Trace:
I20260709 12:24:28.946066 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:28.988741 16399 rpcz_store.cc:269] 0709 12:24:27.071151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.071293 (+   142us) service_pool.cc:224] Handling call
0709 12:24:28.959384 (+1888091us) spinlock_profiling.cc:243] Waited 2.3 ms on lock 0x7b540015e354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:28.959562 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5290880}
I20260709 12:24:28.991093 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:28.943387 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 49301.3us
I20260709 12:24:28.991240 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:28.993115 16398 raft_consensus.cc:3060] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Advancing to term 21
W20260709 12:24:28.996420 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 53273.3us
I20260709 12:24:28.989914 16542 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:24:28.999617 16399 raft_consensus.cc:2393] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 20: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:28.944064 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:24:29.001199 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 65264.8us
I20260709 12:24:29.001736 16548 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:29.002740 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:29.003932 16468 raft_consensus.cc:2393] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 15: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:28.940310 22354 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:29.006917 16441 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:28.944008 22349 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:29.011648 16475 raft_consensus.cc:3055] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:29.021958 16475 raft_consensus.cc:740] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:29.022099 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:29.012697 21750 consensus_queue.cc:1059] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:29.023345 16551 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 11.
I20260709 12:24:28.960309 22241 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 24. Current term is 25.
I20260709 12:24:29.005167 16514 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:28.963321 22298 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:28.988215 22230 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:29.026791 22350 consensus_queue.cc:1048] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 12:24:29.012678 22354 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:29.050685 16545 raft_consensus.cc:1275] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 12:24:29.051362 16545 pending_rounds.cc:85] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1: Aborting all ops after (but not including) 10
I20260709 12:24:29.051641 16545 pending_rounds.cc:107] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 12:24:29.051476 16403 raft_consensus.cc:1275] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:24:29.027858 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:29.022930 16475 consensus_queue.cc:260] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:29.029157 22355 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:29.032701 16514 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:29.031916 16473 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:24:28.942185 22297 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:29.053542 22314 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:24:29.056953 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 264945us
I20260709 12:24:29.076768 22314 raft_consensus.cc:697] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:29.077507 22297 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:29.052062 16545 raft_consensus.cc:2889] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:29.048172 22321 log.cc:927] Time spent T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.810s	user 0.003s	sys 0.000s
I20260709 12:24:29.077749 22314 consensus_queue.cc:237] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:28.936579 22353 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:29.090385 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:29.067996 16475 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:29.082454 16394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 1703) took 1049 ms. Trace:
I20260709 12:24:29.082626 17254 raft_consensus.cc:1240] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:29.074064 16549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52680 (request call id 1285) took 1087 ms. Trace:
I20260709 12:24:29.092983 16549 rpcz_store.cc:276] 0709 12:24:27.986515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.986678 (+   163us) service_pool.cc:224] Handling call
0709 12:24:29.073823 (+1087145us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b5400201054. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:29.073965 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":681794304}
I20260709 12:24:29.092424 16394 rpcz_store.cc:276] 0709 12:24:28.033506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.033644 (+   138us) service_pool.cc:224] Handling call
0709 12:24:28.033969 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:28.034346 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:28.034393 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:28.034954 (+   561us) log.cc:844] Serialized 26 byte log entry
0709 12:24:28.134937 (+ 99983us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:28.134983 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:28.135239 (+   256us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:29.047534 (+912295us) spinlock_profiling.cc:243] Waited 2.31 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:24:29.082159 (+ 34625us) raft_consensus.cc:1685] finished
0709 12:24:29.082180 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:29.082377 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14346,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"spinlock_wait_cycles":5302656,"thread_start_us":99535,"threads_started":1,"wal-append.queue_time_us":144235}
I20260709 12:24:29.095315 16396 raft_consensus.cc:1240] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:29.073762 16514 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:29.103080 16394 raft_consensus.cc:1240] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:29.104522 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:29.105700 16468 raft_consensus.cc:3055] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:29.074334 16398 raft_consensus.cc:2468] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 21.
I20260709 12:24:29.107743 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:29.107928 16516 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:29.090452 16548 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 20.
I20260709 12:24:29.097648 16367 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:29.106400 16468 raft_consensus.cc:740] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:24:29.115262 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 108423us
I20260709 12:24:29.113000 16468 consensus_queue.cc:260] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:29.117826 16441 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:29.118361 16468 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:29.071911 22216 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:29.058270 22298 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:29.060289 22288 raft_consensus.cc:3055] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:29.136134 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:29.063464 22304 raft_consensus.cc:3055] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:29.178810 16474 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:29.057137 22126 consensus_queue.cc:1048] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 12:24:29.131915 16367 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:29.059988 22334 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:29.068176 22351 consensus_queue.cc:260] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:29.052273 22342 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:29.110659 22217 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:29.135406 22216 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:29.153278 22288 raft_consensus.cc:740] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:29.186359 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:29.203047 16399 raft_consensus.cc:1240] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:29.203903 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:29.138902 22230 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:29.138299 22241 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:29.194350 16516 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:29.210880 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:29.195137 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 63341us
I20260709 12:24:29.215451 16367 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:29.216300 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:29.218726 16395 raft_consensus.cc:1240] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:29.227540 16474 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:29.200723 22271 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:29.235219 16367 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:29.114514 22355 consensus_queue.cc:1059] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.123s
I20260709 12:24:29.194795 22304 raft_consensus.cc:740] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:29.202571 22131 raft_consensus.cc:2804] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:29.231930 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:29.066985 22283 consensus_queue.cc:260] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:29.253677 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:29.253922 22283 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:29.237025 16471 raft_consensus.cc:2393] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 17: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:29.228603 22124 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:29.239773 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:29.258605 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:29.259868 16469 raft_consensus.cc:2410] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:24:29.240288 16367 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:29.202371 22217 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:29.208694 22216 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:29.262106 22217 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:29.262521 16367 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:29.214944 22351 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:29.240506 17142 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:29.270097 22186 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:29.217676 21999 consensus_queue.cc:1059] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:24:29.248946 22350 raft_consensus.cc:3055] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:29.249812 22304 consensus_queue.cc:260] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:29.242033 22169 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:29.251602 22131 raft_consensus.cc:697] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:29.241617 22335 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:29.228516 22258 consensus_queue.cc:1048] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:24:29.269263 22288 consensus_queue.cc:260] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:29.270840 16367 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:29.265316 22128 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:29.295156 22186 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:29.270921 22271 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:29.246459 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:29.295558 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:29.309363 16399 raft_consensus.cc:1275] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:24:29.314507 16367 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
W20260709 12:24:29.317876 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:29.357601 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.317986 22269 consensus_queue.cc:1048] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 12:24:29.319188 22131 consensus_queue.cc:237] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:29.320293 22186 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:29.320565 22341 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:29.299234 22350 raft_consensus.cc:740] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:24:29.341233 22360 log.cc:927] Time spent T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 12:24:29.345547 22342 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:29.347858 22288 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:29.349725 22334 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:29.359949 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 155375us
I20260709 12:24:29.360409 16475 raft_consensus.cc:1275] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
W20260709 12:24:29.363040 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 48654.1us
I20260709 12:24:29.369596 16475 pending_rounds.cc:85] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 9
I20260709 12:24:29.370105 16475 pending_rounds.cc:107] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 12:24:29.370620 16475 raft_consensus.cc:2889] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:29.363447 16468 raft_consensus.cc:1275] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
W20260709 12:24:29.365939 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 123477us
I20260709 12:24:29.366376 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:29.368382 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:29.366004 22241 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:29.372030 16468 pending_rounds.cc:85] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 10
I20260709 12:24:29.305619 22230 raft_consensus.cc:697] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 20 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:29.369195 22341 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:24:29.379197 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.377137 16516 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:29.381480 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:29.382838 16468 pending_rounds.cc:107] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 12:24:29.395018 16468 raft_consensus.cc:2889] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:29.395803 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:29.360956 22271 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:29.383155 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:29.391275 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:29.396481 16546 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:29.394310 22350 consensus_queue.cc:260] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:29.410737 16470 raft_consensus.cc:2410] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:24:29.375871 22169 consensus_queue.cc:1059] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 12:24:29.413957 16546 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 18.
I20260709 12:24:29.396821 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:29.396231 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:29.396528 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:29.397383 22211 raft_consensus.cc:2804] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:29.414685 16367 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:29.377717 22342 consensus_queue.cc:1059] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:24:29.419270 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:29.420009 22211 raft_consensus.cc:697] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 21 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:29.422971 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:29.422554 22237 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:29.423802 22237 raft_consensus.cc:697] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 20 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:29.424245 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:29.423903 22211 consensus_queue.cc:237] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:29.424784 22237 consensus_queue.cc:237] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:29.425336 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:29.422744 22334 raft_consensus.cc:3055] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:29.426723 22334 raft_consensus.cc:740] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:29.427640 22315 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:29.427662 22334 consensus_queue.cc:260] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:29.428310 22315 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:29.429230 22334 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:29.418654 22350 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:29.431211 22315 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:29.431649 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:24:29.428910 22364 log.cc:927] Time spent T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:24:29.435319 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:29.435321 17255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 1438) took 1025 ms. Trace:
I20260709 12:24:29.437018 17255 rpcz_store.cc:276] 0709 12:24:28.409725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.410084 (+   359us) service_pool.cc:224] Handling call
0709 12:24:28.438462 (+ 28378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:28.547430 (+108968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:29.435123 (+887693us) spinlock_profiling.cc:243] Waited 97.6 ms on lock 0x7b540063ea54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:29.435267 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":224508672}
I20260709 12:24:29.420698 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:29.441327 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:29.445288 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:29.446523 16442 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:29.447743 22315 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.447896 22346 consensus_queue.cc:1059] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:29.448446 16367 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:29.448287 22315 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:29.449409 22169 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:29.449915 22169 raft_consensus.cc:697] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 18 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:29.450223 21750 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:29.450634 21750 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:29.451085 21750 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:29.450786 22169 consensus_queue.cc:237] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:29.452766 22346 raft_consensus.cc:3055] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:29.453222 22346 raft_consensus.cc:740] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:29.454208 22346 consensus_queue.cc:260] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:29.455755 22346 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:29.459450 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:29.460654 21750 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:29.463481 21750 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:29.471419 22211 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.472283 22211 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:29.474083 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.418781 22341 raft_consensus.cc:697] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 20 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:29.380877 22369 consensus_queue.cc:1048] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 12:24:29.403788 22230 consensus_queue.cc:237] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:29.481493 16394 raft_consensus.cc:1240] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:24:29.485550 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:29.446408 22349 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:24:29.512990 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.518481 16473 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:29.523495 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:29.420701 22354 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:29.536577 16476 raft_consensus.cc:1240] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:29.538385 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:29.532476 22216 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:29.547479 22216 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:29.549701 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:29.556795 16468 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:29.491299 22217 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:29.557863 22365 log.cc:927] Time spent T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 12:24:29.558415 17266 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 15.
I20260709 12:24:29.564515 22335 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:29.582167 22354 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:29.583267 22341 consensus_queue.cc:237] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:29.586426 16477 raft_consensus.cc:2410] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
W20260709 12:24:29.590379 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 983200us
I20260709 12:24:29.592384 22314 consensus_queue.cc:1059] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:24:29.594174 22124 raft_consensus.cc:3055] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:29.594566 22124 raft_consensus.cc:740] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:29.595468 22124 consensus_queue.cc:260] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:29.596987 22124 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:29.597685 22285 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:29.601357 22255 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:29.602054 22255 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:29.603359 22186 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:29.605965 22334 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:29.606616 22334 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:29.610146 22334 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:29.559473 22315 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:29.624398 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:29.624830 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:29.624903 22216 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.625631 22378 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.625886 22216 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:29.626241 22378 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:29.628069 22216 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:29.626253 22173 raft_consensus.cc:3055] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:29.628808 16549 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 19.
I20260709 12:24:29.628860 22173 raft_consensus.cc:740] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:29.629040 16545 raft_consensus.cc:1240] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:29.630030 22173 consensus_queue.cc:260] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:29.630678 16550 raft_consensus.cc:1275] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:24:29.632691 16550 raft_consensus.cc:1275] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:24:29.592034 16515 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:29.635784 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 43839.9us
I20260709 12:24:29.625641 16546 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:29.638150 22219 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:29.638824 16548 raft_consensus.cc:1275] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:24:29.639384 16550 raft_consensus.cc:1240] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:29.641077 22378 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:29.641919 16474 raft_consensus.cc:1240] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:29.642151 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:29.642421 16441 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:29.643039 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:29.644213 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:29.646924 16515 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:29.648824 22315 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:29.649482 22368 consensus_queue.cc:1059] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:29.650617 22368 consensus_queue.cc:1059] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:24:29.651372 22216 raft_consensus.cc:3055] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:29.651678 22219 raft_consensus.cc:3055] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:29.652012 22216 raft_consensus.cc:740] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:29.652261 22219 raft_consensus.cc:740] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:29.653095 22216 consensus_queue.cc:260] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:29.653254 22219 consensus_queue.cc:260] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:29.654399 16400 raft_consensus.cc:1240] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:29.654462 22216 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:29.661046 17264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 1452) took 1117 ms. Trace:
W20260709 12:24:29.662945 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 422491us
W20260709 12:24:29.664780 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.665191 22304 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:29.666400 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.667542 22307 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:29.667927 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:29.668099 22307 raft_consensus.cc:697] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:29.668198 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:29.668525 22298 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:29.669075 22298 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:29.669154 22307 consensus_queue.cc:237] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:29.619867 22122 raft_consensus.cc:3055] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:29.670264 22122 raft_consensus.cc:740] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:29.671377 22298 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:29.671217 22122 consensus_queue.cc:260] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:29.671954 22298 raft_consensus.cc:3055] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:29.672942 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:29.675158 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.678277 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:29.679819 16367 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:29.683422 16548 raft_consensus.cc:1240] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 18. Current term is 19. Ops: []
W20260709 12:24:29.691735 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 49410us
I20260709 12:24:29.684609 17266 raft_consensus.cc:1275] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:24:29.680884 16396 raft_consensus.cc:2393] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 17: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:29.686933 16471 raft_consensus.cc:2368] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 20. Current term is 21.
W20260709 12:24:29.692055 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.661511 17264 rpcz_store.cc:276] 0709 12:24:28.543654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.547927 (+  4273us) service_pool.cc:224] Handling call
0709 12:24:28.568955 (+ 21028us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:28.569329 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:28.569361 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:28.569932 (+   571us) log.cc:844] Serialized 52 byte log entry
0709 12:24:29.058027 (+488095us) spinlock_profiling.cc:243] Waited 17.8 ms on lock 0x7b18002a85c0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:24:29.201283 (+143256us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:29.215395 (+ 14112us) log.cc:844] Serialized 14 byte log entry
0709 12:24:29.660500 (+445105us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b18002a85c0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d16c0a 00007f287c456219 00007f2872d9ea40 00007f2872d994e4 00007f2872d99455 00007f2875e81060 00007f2875e93716 00007f2875ea4287 00007f2875ea4208
0709 12:24:29.660698 (+   198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:29.660787 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:29.660837 (+    50us) raft_consensus.cc:1685] finished
0709 12:24:29.660852 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:29.660985 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2847,"prepare.queue_time_us":16455,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":4151,"raft.queue_time_us":447808,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":9009,"spinlock_wait_cycles":66555392,"thread_start_us":143059,"threads_started":1,"wal-append.queue_time_us":679985}
I20260709 12:24:29.696542 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:29.680226 22298 raft_consensus.cc:740] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:24:29.681607 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 202170us
I20260709 12:24:29.703603 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:29.708038 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:29.709729 16468 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 19.
I20260709 12:24:29.711054 16514 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:29.695411 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:29.713510 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:29.713932 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:29.714773 16473 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
I20260709 12:24:29.707322 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:29.721495 16546 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 20.
I20260709 12:24:29.722090 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:29.724205 16397 raft_consensus.cc:2393] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 16: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:29.716744 17144 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
W20260709 12:24:29.748494 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:29.731702 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:29.774031 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.714174 16399 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 19.
W20260709 12:24:29.775122 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 57006.7us
I20260709 12:24:29.765836 16476 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 15.
I20260709 12:24:29.777710 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:29.778460 16548 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:24:29.776494 16441 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:29.684921 21522 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:29.781436 16441 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:29.783219 16441 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:29.784405 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:29.785305 16441 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:29.790812 16441 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:29.793045 16441 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:29.715369 22289 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.794900 22355 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:29.795133 22241 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:29.795334 22342 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:29.800006 22239 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:29.805150 22258 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:29.809108 22351 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:29.810503 16472 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 20.
W20260709 12:24:29.813091 16366 leader_election.cc:336] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.736s (SENT)
I20260709 12:24:29.704370 22269 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:29.838174 22269 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:29.838726 22269 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:29.728449 22283 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:24:29.814096 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 134954us
W20260709 12:24:29.814641 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 129331us
W20260709 12:24:29.815558 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 98867.3us
I20260709 12:24:29.846689 17144 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:29.796782 16367 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:29.847795 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 51261.3us
I20260709 12:24:29.849339 16469 raft_consensus.cc:1240] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:29.786481 21750 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:29.774490 22202 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:29.849565 17143 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:29.716068 22353 raft_consensus.cc:2764] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 12:24:29.830323 21999 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:29.852846 17144 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
W20260709 12:24:29.861972 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.856057 16548 raft_consensus.cc:1275] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:24:29.860205 17143 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:29.823786 22297 consensus_queue.cc:1048] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:24:29.872783 22259 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:29.873427 22300 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:24:29.851256 22334 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:29.873544 22255 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:29.874410 21999 raft_consensus.cc:697] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 16 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:29.836604 22298 consensus_queue.cc:260] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:29.829531 22289 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:29.880522 17270 raft_consensus.cc:2468] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:24:29.837631 22258 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:29.884068 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.851670 22313 consensus_queue.cc:1048] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 12:24:29.715845 22359 consensus_queue.cc:1059] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.150s
W20260709 12:24:29.904173 16366 leader_election.cc:336] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.825s (SENT)
I20260709 12:24:29.855224 22169 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:29.861927 22316 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:29.941043 22255 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:29.942085 21999 consensus_queue.cc:237] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:29.867087 22329 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:29.885376 22186 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:29.888998 16550 raft_consensus.cc:1275] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:24:29.946229 22300 raft_consensus.cc:697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 25 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:24:29.951081 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.953027 16366 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:29.899546 22202 raft_consensus.cc:697] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 19 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:29.861080 22283 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:29.950708 22230 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:29.898921 22259 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:30.004690 22230 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:30.005317 22259 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:29.996619 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 92494us
I20260709 12:24:29.999359 22237 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:30.009817 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:30.000043 22202 consensus_queue.cc:237] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:30.010756 16473 raft_consensus.cc:2393] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 20: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:30.013252 22283 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:29.954239 22169 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:29.911770 22334 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:29.954044 22124 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:30.036201 22219 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:30.042675 22289 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:30.051424 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.984191 22377 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:30.008085 22230 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:30.048125 22258 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:30.049433 22217 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:30.032481 22169 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:29.954514 22285 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:30.062773 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 209984us
I20260709 12:24:30.032497 22348 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:30.099850 22348 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:30.071033 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.094691 22186 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:30.102550 22230 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:30.102885 22186 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:30.103080 22230 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:30.103622 17270 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45224 (request call id 1444) took 1692 ms (client timeout 1644 ms). Trace:
W20260709 12:24:30.104087 17270 rpcz_store.cc:269] 0709 12:24:28.411402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.425448 (+ 14046us) service_pool.cc:224] Handling call
0709 12:24:30.103441 (+1677993us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b5400be2dd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:30.103550 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3343415040}
I20260709 12:24:30.105264 22230 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:30.105592 22383 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:30.106142 22383 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:30.106865 22186 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:30.106986 22315 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.107534 22315 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:30.061018 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 200907us
W20260709 12:24:30.112689 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 319713us
W20260709 12:24:30.083813 22379 log.cc:927] Time spent T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.088s	user 0.002s	sys 0.000s
W20260709 12:24:30.113546 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 256826us
I20260709 12:24:30.113924 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:30.116547 16515 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:30.116235 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:30.111965 22285 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:30.118441 22241 consensus_queue.cc:1048] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:24:30.119488 22285 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:30.105772 22230 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:30.108162 22383 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:30.119990 22359 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:30.066109 22124 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:30.123553 16469 raft_consensus.cc:1240] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:30.104573 22348 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:30.140312 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:30.141111 16476 raft_consensus.cc:2393] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 17: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:30.126080 16366 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:30.143014 16366 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:30.144187 16366 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:30.127277 17144 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:30.127442 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:30.129704 22385 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:24:30.130546 22380 log.cc:927] Time spent T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 12:24:30.131798 22334 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:30.134748 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:30.136742 22289 consensus_queue.cc:1048] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:30.137090 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:30.116636 22304 consensus_queue.cc:1059] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:24:30.138031 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:30.138833 22241 consensus_queue.cc:1059] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:30.144891 16476 raft_consensus.cc:1275] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:24:30.158718 16474 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 20: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:30.069324 22377 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:30.159420 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:30.153321 22186 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:30.160880 22329 consensus_queue.cc:1048] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:30.161480 22124 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:30.149688 16366 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:30.164752 16441 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:30.149531 16468 raft_consensus.cc:1275] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 12:24:30.166093 16468 pending_rounds.cc:85] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 8
I20260709 12:24:30.166448 16468 pending_rounds.cc:107] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:24:30.166797 16468 raft_consensus.cc:2889] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:30.154054 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:30.150364 22237 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:30.156787 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:30.139110 16548 raft_consensus.cc:1240] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 19. Current term is 20. Ops: []
W20260709 12:24:30.148933 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.174551 16366 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:30.165010 17144 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:30.176326 16366 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:30.178573 22186 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:30.179677 22335 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:30.150925 22341 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:30.126319 22230 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:30.161393 22241 raft_consensus.cc:3055] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:30.188714 17143 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:30.141803 22305 consensus_queue.cc:1048] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 12:24:30.186856 16366 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:30.194164 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:30.194959 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:30.195376 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:30.196000 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:30.196611 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:24:30.181514 22271 consensus_meta.cc:287] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.764s	user 0.015s	sys 0.001s
I20260709 12:24:30.124850 22300 consensus_queue.cc:237] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:30.233125 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:30.229570 16475 raft_consensus.cc:1275] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:24:30.231328 17266 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:30.229779 17144 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:30.233314 17254 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 20.
I20260709 12:24:30.213827 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:30.242048 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 55280.7us
I20260709 12:24:30.243095 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:30.199608 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:30.243969 16366 leader_election.cc:400] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 20. Current term is 21.
I20260709 12:24:30.171533 22316 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:30.247952 16366 leader_election.cc:403] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:30.251443 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:30.200768 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:30.254279 17258 raft_consensus.cc:2468] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:24:30.254823 16366 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:30.225066 22313 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:30.256518 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:30.232671 22304 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:30.260782 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:30.277092 17258 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 20.
I20260709 12:24:30.267410 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:30.269304 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:30.259722 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 71101.9us
I20260709 12:24:30.278811 16476 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 20.
I20260709 12:24:30.192639 22186 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:30.192456 22359 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:30.192022 22283 consensus_queue.cc:1048] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 12:24:30.264698 17255 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:24:30.281497 17254 raft_consensus.cc:1275] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:24:30.284101 17258 raft_consensus.cc:1240] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:24:30.288192 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:30.289906 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:30.294440 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:30.294592 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:30.325963 17270 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 19.
I20260709 12:24:30.295025 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:30.283217 16515 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:30.249691 22341 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:30.198107 22334 consensus_queue.cc:1059] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 12:24:30.247094 22241 raft_consensus.cc:740] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:30.260587 22307 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:30.426833 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:30.427826 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:30.232371 22390 consensus_queue.cc:1059] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.088s
I20260709 12:24:30.485298 17271 raft_consensus.cc:1240] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:30.279457 22335 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:30.429172 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:30.492723 17255 raft_consensus.cc:2393] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 16: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:30.257401 22237 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 20. Current term is 21.
W20260709 12:24:30.473753 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 206251us
I20260709 12:24:30.272306 22400 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:30.478237 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:30.499491 16543 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 19.
I20260709 12:24:30.240516 22271 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:30.295074 22316 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:30.280247 22313 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:30.431265 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 148115us
W20260709 12:24:30.469815 22269 consensus_meta.cc:287] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.587s	user 0.008s	sys 0.004s
I20260709 12:24:30.507385 16475 raft_consensus.cc:1275] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:24:30.513155 16515 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:30.486835 22385 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:30.484876 22286 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.519527 16515 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:30.513150 16366 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:30.515563 22316 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:30.404126 22369 consensus_queue.cc:1048] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.109s
I20260709 12:24:30.533109 16515 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
W20260709 12:24:30.538679 22259 consensus_meta.cc:287] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.532s	user 0.012s	sys 0.000s
I20260709 12:24:30.539330 22259 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:30.540760 22122 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:30.520747 22269 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:30.519781 22399 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.516427 22335 raft_consensus.cc:697] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 15 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:30.502698 22315 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:30.497125 22255 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:30.542863 16394 raft_consensus.cc:1240] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:30.582019 16515 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:30.586611 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:30.540803 17269 raft_consensus.cc:2393] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 20: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:30.548926 16366 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:30.496872 22186 consensus_queue.cc:1059] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 12:24:30.516798 22400 raft_consensus.cc:697] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 20 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:30.526451 22195 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:24:30.472970 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 187188us
W20260709 12:24:30.618577 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.542092 22259 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:30.547894 22391 log.cc:927] Time spent T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.254s	user 0.000s	sys 0.000s
I20260709 12:24:30.580404 22385 raft_consensus.cc:2804] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:30.551913 22386 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:30.546844 22269 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:30.580701 22211 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:30.619443 22202 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:30.629541 22385 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:30.654062 22385 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:30.547758 22399 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:30.656481 22269 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:30.658440 22399 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:30.551427 22286 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:30.622807 22359 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:30.589299 22255 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:30.618402 22195 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:30.641804 22259 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.656497 22258 raft_consensus.cc:2749] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:30.642575 22377 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:30.429591 22270 consensus_queue.cc:1048] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.240s
I20260709 12:24:30.548930 22241 consensus_queue.cc:260] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:30.548735 22335 consensus_queue.cc:237] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:30.644987 22202 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:30.674867 22195 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:30.717811 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:30.718747 16469 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:30.667300 16471 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:24:30.627203 22404 consensus_queue.cc:1048] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 12:24:30.660050 22316 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:30.670291 22230 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:30.711812 21999 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.019s	sys 0.038s
I20260709 12:24:30.690512 22259 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:30.752197 21999 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.020s	sys 0.038s
I20260709 12:24:30.770432 22120 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:24:30.770941 22169 consensus_meta.cc:287] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.665s	user 0.005s	sys 0.005s
I20260709 12:24:30.771124 22120 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:30.771373 22169 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:30.772588 22271 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:30.718640 22202 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:30.773300 22120 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:30.773736 17142 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:30.777832 22259 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:30.778793 16394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 1564) took 1113 ms. Trace:
I20260709 12:24:30.779251 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:30.779214 16394 rpcz_store.cc:276] 0709 12:24:29.665382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:29.666761 (+  1379us) service_pool.cc:224] Handling call
0709 12:24:29.667195 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:30.580077 (+912882us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:30.778575 (+198498us) spinlock_profiling.cc:243] Waited 867 ms on lock 0x7b540015ed54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:30.778730 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1993858944}
I20260709 12:24:30.780010 16543 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
W20260709 12:24:30.781952 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.788414 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:30.790040 22219 consensus_meta.cc:287] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.725s	user 0.010s	sys 0.000s
I20260709 12:24:30.791195 22404 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.791885 22404 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:30.794318 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:30.795205 22404 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:30.797309 22131 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.798043 22341 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:30.797971 22131 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:30.798663 22341 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:30.798955 22399 consensus_queue.cc:1048] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:30.676589 22359 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:30.800408 22399 consensus_queue.cc:1048] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:24:30.801966 17254 raft_consensus.cc:3055] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:30.802399 17254 raft_consensus.cc:740] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:30.802784 22354 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.792289 16473 raft_consensus.cc:1275] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:24:30.712118 22400 consensus_queue.cc:237] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:30.803244 17254 consensus_queue.cc:260] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:30.803288 22354 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:30.808316 17254 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:30.811208 17266 consensus_meta.cc:287] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.572s	user 0.008s	sys 0.002s
I20260709 12:24:30.811890 17266 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 19.
I20260709 12:24:30.813331 17255 raft_consensus.cc:1240] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 16 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:24:30.814967 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:30.816761 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:30.817235 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:30.818295 17266 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 20: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:30.818419 17261 raft_consensus.cc:1240] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 16 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:30.819129 21522 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.819669 21522 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:30.821013 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:30.821583 16548 raft_consensus.cc:1275] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:24:30.821962 16546 raft_consensus.cc:2393] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 19: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:30.823426 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:30.823608 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:30.826754 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:30.827587 16475 raft_consensus.cc:2393] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 15: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:30.827477 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:30.830601 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:30.831056 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:30.815789 17255 raft_consensus.cc:2468] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:24:30.832209 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:30.835682 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:24:30.838692 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 65059.6us
I20260709 12:24:30.839462 17142 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:30.838552 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:30.833870 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:30.796159 22404 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:30.848894 16473 raft_consensus.cc:1275] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 12:24:30.851301 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.802273 22359 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:30.849313 22390 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:30.808519 22353 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.807739 22241 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:30.854557 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:30.854460 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:30.855819 17255 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:30.858896 17261 raft_consensus.cc:1240] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 16 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:30.862427 21522 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:30.866683 17261 raft_consensus.cc:1275] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:24:30.851475 22259 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:30.888753 16514 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:30.849150 22313 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:30.838619 22355 consensus_queue.cc:1059] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.030s
W20260709 12:24:30.897732 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 51810.1us
I20260709 12:24:30.898499 21522 consensus_queue.cc:1059] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:30.899027 17142 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:30.903354 17255 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:24:30.893615 22354 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:30.855640 22404 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:30.912628 17143 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:30.895490 22131 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:30.901423 22120 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:30.922441 22217 consensus_queue.cc:1048] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 12:24:30.926405 22120 raft_consensus.cc:697] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:30.927218 22237 raft_consensus.cc:3055] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:30.927708 22237 raft_consensus.cc:740] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:30.927366 22120 consensus_queue.cc:237] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:30.924610 16468 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:24:30.902771 22359 raft_consensus.cc:3055] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:30.862165 22353 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:30.935458 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 319931us
I20260709 12:24:30.938529 16515 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:30.911787 22259 raft_consensus.cc:697] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:30.942824 16441 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:30.943571 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 156821us
I20260709 12:24:30.945374 16472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34840 (request call id 1711) took 1547 ms. Trace:
I20260709 12:24:30.945976 16472 rpcz_store.cc:276] 0709 12:24:29.397748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:29.431374 (+ 33626us) service_pool.cc:224] Handling call
0709 12:24:30.945165 (+1513791us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b54002f2e54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:30.945309 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":471607680}
I20260709 12:24:30.948311 21522 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:30.902323 22377 consensus_queue.cc:1048] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 12:24:30.950963 22354 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:30.936764 22393 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:30.956926 22411 log.cc:927] Time spent T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:24:30.956825 22266 raft_consensus.cc:3055] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:24:30.957989 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 45512.2us
I20260709 12:24:30.958618 22304 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:30.959937 17142 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:30.915592 22305 consensus_queue.cc:1048] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
W20260709 12:24:30.993275 22410 log.cc:927] Time spent T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:24:30.969295 22286 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:30.925051 22341 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:30.956094 17254 raft_consensus.cc:1275] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:24:30.998369 22304 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.013726 22304 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:30.975304 17258 raft_consensus.cc:1240] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:24:31.014402 16396 raft_consensus.cc:1275] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:24:31.015417 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:31.016625 22404 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:30.994673 22393 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:30.975896 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:31.018838 22393 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:31.019373 22329 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 12:24:30.944697 22385 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:30.915812 22270 consensus_queue.cc:1048] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:24:31.021160 22400 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:30.933828 22359 raft_consensus.cc:740] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:31.022209 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:30.950268 22219 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:31.000869 17144 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:31.033061 16544 raft_consensus.cc:1275] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:24:31.032900 17144 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:30.973907 22169 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:31.007386 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:31.007303 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:31.037557 17261 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 20.
W20260709 12:24:31.008786 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 459917us
I20260709 12:24:31.009819 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:31.010032 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:31.021454 22385 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:31.021656 22400 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:31.022668 22359 consensus_queue.cc:260] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:30.996028 17143 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:31.025686 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:31.028095 16400 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:31.026193 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:31.048349 16472 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 20.
I20260709 12:24:31.029223 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:31.028131 16442 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:31.029597 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:31.049624 17258 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:30.968955 22255 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:30.994918 22266 raft_consensus.cc:740] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:31.038769 16402 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 20.
I20260709 12:24:31.044377 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:31.038708 16367 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:31.053593 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:24:31.055210 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 59299.5us
I20260709 12:24:31.055612 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:31.056602 16471 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 17.
I20260709 12:24:31.057843 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:31.059772 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:31.061240 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:31.060778 16402 raft_consensus.cc:1275] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:24:31.061851 16398 raft_consensus.cc:1275] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:24:31.062860 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:31.016650 22418 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:31.071082 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:31.067879 16367 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:31.072749 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:31.077536 16367 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:31.077709 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:31.077868 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:31.077602 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:31.045939 22409 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:31.046646 22400 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:31.046754 22219 raft_consensus.cc:697] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:31.134094 22219 consensus_queue.cc:237] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:31.136181 22305 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:31.120298 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 92247.6us
I20260709 12:24:31.137188 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:31.050624 22169 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:31.126307 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 56358.1us
I20260709 12:24:31.141228 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:31.076668 16441 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:31.141512 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:31.131955 17258 raft_consensus.cc:2468] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
I20260709 12:24:31.079066 16546 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:31.138357 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:31.065447 22354 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:31.146862 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:31.144150 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:31.138667 16543 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:24:31.138381 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:31.139330 16542 raft_consensus.cc:1240] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:31.072299 22173 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:31.150974 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:31.151149 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:31.056926 22404 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:31.065362 22329 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:31.138541 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:31.143160 16396 raft_consensus.cc:2393] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 16: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:31.072675 22131 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:31.065440 22341 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:31.151933 17258 raft_consensus.cc:2393] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 16: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:31.147100 17143 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:31.148567 22359 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:30.999395 22237 consensus_queue.cc:260] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:31.159629 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:31.165917 17255 raft_consensus.cc:1275] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:24:31.079231 22334 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:31.157322 16366 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:31.175745 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:31.193725 16472 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 25.
I20260709 12:24:31.072799 22258 consensus_queue.cc:1048] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.092s
I20260709 12:24:31.136765 22305 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:31.140991 22270 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.152994 22241 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.143744 22412 raft_consensus.cc:2804] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:31.201422 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:31.177098 22131 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:24:31.178934 22417 log.cc:927] Time spent T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 12:24:31.072749 22408 consensus_queue.cc:1059] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.117s
I20260709 12:24:31.167045 22286 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:31.208738 16472 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:31.046578 22259 consensus_queue.cc:237] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:31.144045 22271 consensus_queue.cc:1048] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 12:24:31.075331 22418 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:31.147847 22169 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:31.229982 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 84135us
I20260709 12:24:31.173703 22409 consensus_queue.cc:1048] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 12:24:31.204588 22401 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:31.271396 17144 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:31.199029 22355 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:31.194133 22186 consensus_queue.cc:1059] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 12:24:31.208258 22412 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:31.174226 22404 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:31.214527 22131 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:31.281164 22274 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:31.324494 22258 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:31.325350 22258 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:31.327587 22258 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:31.270500 22315 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.329031 22315 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:31.329877 22195 consensus_meta.cc:287] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.592s	user 0.010s	sys 0.000s
I20260709 12:24:31.330331 22404 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:31.331187 22315 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:31.190212 22341 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:31.336711 22341 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:31.228757 22202 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:31.339696 22428 log.cc:927] Time spent T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.160s	user 0.003s	sys 0.000s
I20260709 12:24:31.341180 22202 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:31.345280 22423 log.cc:927] Time spent T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 12:24:31.278703 17144 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:31.348183 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 69547us
I20260709 12:24:31.349947 22283 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:31.350502 22283 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.350898 22283 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:30.726356  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:31.358736 22269 raft_consensus.cc:3055] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:31.359220 22269 raft_consensus.cc:740] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:31.361796 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:31.362643 16475 raft_consensus.cc:2393] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 20: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:31.364704 16477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 1262) took 1220 ms. Trace:
I20260709 12:24:31.365129 16477 rpcz_store.cc:276] 0709 12:24:30.144419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.144598 (+   179us) service_pool.cc:224] Handling call
0709 12:24:30.144967 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:30.145375 (+   408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:30.145413 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:30.145881 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 12:24:30.655194 (+509313us) spinlock_profiling.cc:243] Waited 2.47 ms on lock 0x7b18003f2080. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:24:30.857157 (+201963us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:30.857213 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:30.857300 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:31.364398 (+507098us) raft_consensus.cc:1685] finished
0709 12:24:31.364419 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:31.364635 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":100,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":5444,"spinlock_wait_cycles":5687040,"thread_start_us":201747,"threads_started":1,"wal-append.queue_time_us":797661}
I20260709 12:24:31.372942 22353 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:31.383980 16514 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
W20260709 12:24:31.289671 22421 log.cc:927] Time spent T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.209s	user 0.000s	sys 0.000s
W20260709 12:24:31.290573 22427 log.cc:927] Time spent T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
W20260709 12:24:31.291428 22426 log.cc:927] Time spent T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 12:24:31.398378 16468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34826 (request call id 1995) took 1185 ms. Trace:
W20260709 12:24:31.292304 22425 log.cc:927] Time spent T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:24:31.398836 16468 rpcz_store.cc:276] 0709 12:24:30.213324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.213508 (+   184us) service_pool.cc:224] Handling call
0709 12:24:31.398210 (+1184702us) spinlock_profiling.cc:243] Waited 684 ms on lock 0x7b5400a695d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:31.398320 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1572291840}
W20260709 12:24:31.293002 22420 log.cc:927] Time spent T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.241s	user 0.000s	sys 0.001s
I20260709 12:24:31.296649 22399 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:31.305341 22305 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:31.208117 22270 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:31.401716 22399 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.401569 22304 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:31.403790 22270 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:31.212843 22241 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:31.404194 22298 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:31.406481 22241 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:31.285995 22412 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:24:31.393036 22307 consensus_meta.cc:287] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.906s	user 0.010s	sys 0.003s
I20260709 12:24:31.408697 22392 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:31.409189 22392 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.409569 22392 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:31.408715 22307 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:31.277448 22401 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.215461 22334 consensus_queue.cc:1048] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 12:24:31.316444 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:31.317523 22169 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:31.417109 17254 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 17.
I20260709 12:24:31.319397 16474 raft_consensus.cc:1275] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:24:31.319545 22418 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:31.180109 22266 consensus_queue.cc:260] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:31.330374 16515 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:31.330338 22195 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:31.415153 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:31.473759 16396 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:31.286748 16472 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
W20260709 12:24:31.473798 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 56825.9us
W20260709 12:24:31.462180 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 304922us
W20260709 12:24:31.484587 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 154269us
I20260709 12:24:31.483072 17144 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:31.489099 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 342062us
I20260709 12:24:31.413501 22237 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:31.497028 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:31.486110 16515 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:31.416756 22401 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:31.402380 22399 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:31.501569 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:31.502969 16472 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 21.
W20260709 12:24:31.499621 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 422894us
I20260709 12:24:31.505214 16549 raft_consensus.cc:1240] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:31.502070 17270 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 17.
I20260709 12:24:31.360090 22269 consensus_queue.cc:260] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:31.506767 16441 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:31.474210 22393 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:31.517184 16441 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:31.519033 16547 raft_consensus.cc:1240] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:31.521417 16542 raft_consensus.cc:1275] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:24:31.522179 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:31.522814 16543 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
W20260709 12:24:31.526228 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 383906us
I20260709 12:24:31.527462 16367 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:31.526139 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:31.489760 22314 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:31.529098 16441 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:31.529979 16401 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:24:31.531215 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:31.536211 17143 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:31.536262 16441 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:31.534461 22355 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:31.534432 22122 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:24:31.543256 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 57212.6us
I20260709 12:24:31.564309 16515 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:31.565661 16515 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:31.543885 22137 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:31.548947 22387 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:31.502537 22353 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:31.548097 16225 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59300 (request call id 86) took 66209 ms (client timeout 15000 ms). Trace:
I20260709 12:24:31.533509 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:31.507300 22307 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:31.533107 22393 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:31.534987 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 59168.1us
I20260709 12:24:31.562630 16516 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:31.490825 22131 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:31.568817 22137 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:31.531600 22354 raft_consensus.cc:2804] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:31.568917 22269 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:31.644523 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 108407us
I20260709 12:24:31.622821 16514 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
W20260709 12:24:31.644346 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 78759.6us
I20260709 12:24:31.645251 17263 raft_consensus.cc:1275] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 12:24:31.630585 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 95811us
I20260709 12:24:31.704704 16515 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:31.709724 16367 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:31.734081 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 111303us
I20260709 12:24:31.736210 16441 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:31.645116 22289 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:31.749119 16367 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:31.728029 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 165476us
I20260709 12:24:31.756134 16516 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:31.531243 22330 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.002s	sys 0.016s
I20260709 12:24:31.758303 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:31.736984 22378 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:31.758942 16401 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 20.
I20260709 12:24:31.620510 22202 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:31.701267 22259 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.766161 16366 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:31.777417 16366 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:31.767458 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:31.658167 22137 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:31.699630 22435 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:31.618412 22195 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:31.645207 22368 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:24:31.766578 22330 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.003s	sys 0.023s
I20260709 12:24:31.780910 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:31.782552 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:31.783538 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:31.784480 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:31.658201 22355 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:31.785921 16366 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:31.746366 22219 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:31.804281 22219 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:31.804446 22435 raft_consensus.cc:697] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:31.804780 22219 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:31.805451 22435 consensus_queue.cc:237] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:31.806633 22258 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:24:31.807188 22258 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 25 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:31.807621 22258 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:31.705730 22354 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:31.808580 22354 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:31.809705 22195 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:31.811486 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:31.812520 22271 consensus_queue.cc:1048] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:31.706916 22390 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.814078 22390 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:31.620762 22307 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.815083 22307 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:31.573580 22216 consensus_queue.cc:1048] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.207s
I20260709 12:24:31.750510 22304 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:31.746655 22169 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:31.785241 22173 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.763814 21922 consensus_queue.cc:1059] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 12:24:31.645767 22387 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:31.620556 22211 consensus_queue.cc:1048] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.217s
I20260709 12:24:31.625226 22400 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:31.786835 22259 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:31.753480 22383 raft_consensus.cc:2804] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:31.812125 16473 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:31.818694 16442 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:31.763959 22335 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:31.773576 22122 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:24:31.571825 16225 rpcz_store.cc:269] 0709 12:23:25.338723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:25.339781 (+  1058us) service_pool.cc:224] Handling call
0709 12:24:31.547988 (+66208207us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:26.235744 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:26.236649 (+   905us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:26.236720 (+    71us) write_op.cc:435] Acquired schema lock
0709 12:24:26.236749 (+    29us) tablet.cc:662] Decoding operations
0709 12:24:26.237711 (+   962us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:26.237832 (+   121us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:26.237872 (+    40us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:24:26.238710 (+   838us) tablet.cc:701] Row locks acquired
0709 12:24:26.238769 (+    59us) write_op.cc:260] PREPARE: finished
0709 12:24:26.239013 (+   244us) write_op.cc:270] Start()
0709 12:24:26.239192 (+   179us) write_op.cc:276] Timestamp: P: 1783599866238922 usec, L: 0
0709 12:24:26.239217 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:24:26.283305 (+ 44088us) log.cc:844] Serialized 2044 byte log entry
0709 12:24:27.435384 (+1152079us) op_driver.cc:464] REPLICATION: finished
0709 12:24:27.797879 (+362495us) write_op.cc:301] APPLY: starting
0709 12:24:27.813662 (+ 15783us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:27.896613 (+ 82951us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:27.905390 (+  8777us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:27.937413 (+ 32023us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:27.937653 (+   240us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=0,mrs_lookups=0
0709 12:24:27.937707 (+    54us) write_op.cc:312] APPLY: finished
0709 12:24:27.937993 (+   286us) log.cc:844] Serialized 56 byte log entry
0709 12:24:27.939372 (+  1379us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:27.939516 (+   144us) write_op.cc:454] Released schema lock
0709 12:24:27.939572 (+    56us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:27.944651 (+  5079us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:27.944829 (+   178us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":417728,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":278554,"apply.run_cpu_time_us":10444,"apply.run_wall_time_us":191973,"cfile_cache_hit":22,"cfile_cache_hit_bytes":25047,"num_ops":5,"prepare.queue_time_us":43850,"prepare.run_cpu_time_us":18698,"prepare.run_wall_time_us":541305,"raft.queue_time_us":165746,"raft.run_cpu_time_us":6211,"raft.run_wall_time_us":256570,"replication_time_us":1195949,"thread_start_us":846410,"threads_started":4,"wal-append.queue_time_us":642856,"wal-append.run_cpu_time_us":22915,"wal-append.run_wall_time_us":1960121}]]}
I20260709 12:24:31.789229 22341 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:31.707988 22412 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:31.753325 22408 consensus_queue.cc:1048] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
W20260709 12:24:31.821650 22410 log.cc:927] Time spent T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:24:31.699278 22353 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:31.839619 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:31.924929 16545 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:24:31.860900 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 56852.5us
I20260709 12:24:31.789352 22377 consensus_queue.cc:1048] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 12:24:31.820510 22355 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:31.765283 22401 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:31.863924 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 60385.9us
I20260709 12:24:31.831012 22271 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.786571 22286 consensus_queue.cc:1048] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
W20260709 12:24:31.839695 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 77715.2us
I20260709 12:24:31.834919 22169 raft_consensus.cc:697] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
W20260709 12:24:31.942477 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:31.944790 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 94549.6us
I20260709 12:24:31.942276 17142 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:31.871775 22383 raft_consensus.cc:697] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 13 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:31.978621 16397 raft_consensus.cc:1240] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:31.979562 16473 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 21.
I20260709 12:24:31.834837 22173 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:31.979735 16515 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:24:31.995256 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 250440us
I20260709 12:24:32.001071 16477 raft_consensus.cc:1275] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:24:32.001739 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:32.002615 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:32.008074 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 65939.4us
I20260709 12:24:32.013325 16475 raft_consensus.cc:1275] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:24:32.018793 16545 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 21.
I20260709 12:24:32.020494 22436 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:31.975612 22122 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:24:32.029711 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 273671us
I20260709 12:24:31.996176 22369 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:32.042747 22131 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:32.004917 22217 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:32.043061 22458 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.063618 22401 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:32.021517 22416 consensus_queue.cc:1059] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.277s
I20260709 12:24:32.064678 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:31.996376 22434 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:24:32.086076 22446 log.cc:927] Time spent T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.238s	user 0.000s	sys 0.000s
W20260709 12:24:32.087426 22441 log.cc:927] Time spent T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.261s	user 0.001s	sys 0.001s
I20260709 12:24:31.984079 22271 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:32.090709 22444 log.cc:927] Time spent T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 12:24:32.121500 22434 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:32.092078 22412 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:32.093420 22173 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:32.095364 22283 consensus_meta.cc:287] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.744s	user 0.008s	sys 0.002s
I20260709 12:24:32.022078 22448 consensus_queue.cc:1048] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.349s
W20260709 12:24:32.069042 22359 consensus_meta.cc:287] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.873s	user 0.009s	sys 0.003s
I20260709 12:24:32.098599 22369 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.116765 22300 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.111435 22458 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:32.132798 22314 consensus_meta.cc:287] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.597s	user 0.010s	sys 0.001s
I20260709 12:24:32.133241 22314 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:32.133570 22458 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:32.134176 22387 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:32.135697 22454 log.cc:927] Time spent T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
W20260709 12:24:32.138746 22392 consensus_meta.cc:287] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.729s	user 0.013s	sys 0.000s
I20260709 12:24:31.972572 22355 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:32.059238 22457 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.139678 22457 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:32.140743 22355 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:32.142082 22457 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:32.059597 22419 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:32.059374 22438 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:32.139271 22392 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:32.061920 22400 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:32.136818 22378 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:32.142939 22438 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:32.079272 22383 consensus_queue.cc:237] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:32.148757 22392 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:32.130548 22369 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:32.149885 22438 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:32.150045 22390 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:32.143570 22399 consensus_meta.cc:287] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.628s	user 0.006s	sys 0.000s
I20260709 12:24:32.205838 16399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 1582) took 1196 ms. Trace:
W20260709 12:24:32.219931 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 401204us
I20260709 12:24:32.207623 16399 rpcz_store.cc:276] 0709 12:24:31.009579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.009681 (+   102us) service_pool.cc:224] Handling call
0709 12:24:31.009949 (+   268us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:31.010442 (+   493us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:31.010470 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:31.010906 (+   436us) log.cc:844] Serialized 26 byte log entry
0709 12:24:31.531421 (+520515us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:31.531461 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:31.531535 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:32.205129 (+673594us) raft_consensus.cc:1685] finished
0709 12:24:32.205175 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:32.205533 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":142,"raft.queue_time_us":2662,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":115,"thread_start_us":472907,"threads_started":1,"wal-append.queue_time_us":794967}
I20260709 12:24:32.227993 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:32.229409 16402 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:32.223016 16442 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:32.226317 16469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1735) took 1184 ms. Trace:
I20260709 12:24:32.237540 16469 rpcz_store.cc:276] 0709 12:24:31.041302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.346971 (+305669us) service_pool.cc:224] Handling call
0709 12:24:31.347346 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:31.414541 (+ 67195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:31.414574 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:31.415099 (+   525us) log.cc:844] Serialized 26 byte log entry
0709 12:24:31.776257 (+361158us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:31.776302 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:31.867717 (+ 91415us) spinlock_profiling.cc:243] Waited 66.8 ms on lock 0x7b54002f2954. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:31.867769 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:32.225375 (+357606us) raft_consensus.cc:1685] finished
0709 12:24:32.225422 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:32.225986 (+   564us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":729,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":13763,"spinlock_wait_cycles":153574784,"thread_start_us":360424,"threads_started":1,"wal-append.queue_time_us":443684}
I20260709 12:24:32.090556 17254 raft_consensus.cc:1275] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:24:32.083271 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:32.066246 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:32.255715 17266 raft_consensus.cc:2368] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 20. Current term is 21.
I20260709 12:24:32.107288 22217 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:32.112181 17255 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 18.
I20260709 12:24:32.265045 22334 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.266387 22334 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:32.136437 22300 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:32.043398 21922 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:32.269047 22399 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:32.269510 21922 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:32.270394 22300 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:32.270784 22334 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:32.271353 17258 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:32.144296 22354 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:32.040304 22169 consensus_queue.cc:237] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:32.061161 22258 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:32.273500 16395 raft_consensus.cc:1240] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
W20260709 12:24:32.281543 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 58890.9us
I20260709 12:24:32.272324 21922 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:32.280496 22399 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:32.281956 22354 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:32.275496 22137 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:32.329958 16403 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 26 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 25. Current term is 26. Ops: []
I20260709 12:24:32.127489 22283 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:32.336102 17271 raft_consensus.cc:2368] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 21. Current term is 22.
W20260709 12:24:32.336870 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 267388us
I20260709 12:24:32.338222 22314 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.338364 17143 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:32.339114 22314 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:32.285352 22219 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:32.340940 22315 consensus_queue.cc:1048] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:24:32.343405 22219 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:32.343506 22315 consensus_queue.cc:1048] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:32.317248 22334 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.354264 22259 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:32.345384 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:32.356410 16397 raft_consensus.cc:2393] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 15: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:32.357492 22258 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 26 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:32.129583 22271 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:32.362920 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 104786us
I20260709 12:24:32.365657 16515 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:32.340975 22200 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:32.368640 22200 raft_consensus.cc:697] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 21 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:32.369381 22400 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:32.370235 22400 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:32.371658 22334 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:32.374261 22334 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:32.345156 22353 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:32.375836 22283 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:32.381084 22400 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:32.381371 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:32.370072 22200 consensus_queue.cc:237] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:32.385990 17263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 1528) took 2151 ms. Trace:
I20260709 12:24:32.386796 17263 rpcz_store.cc:276] 0709 12:24:30.234609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.234732 (+   123us) service_pool.cc:224] Handling call
0709 12:24:30.247526 (+ 12794us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:31.705383 (+1457857us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.385254 (+679871us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b5400040bd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:32.385728 (+   474us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3179094784}
I20260709 12:24:32.391016 17270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45230 (request call id 1463) took 2109 ms. Trace:
I20260709 12:24:32.391701 17270 rpcz_store.cc:276] 0709 12:24:30.281593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.427533 (+145940us) service_pool.cc:224] Handling call
0709 12:24:32.390446 (+1962913us) spinlock_profiling.cc:243] Waited 936 ms on lock 0x7b54005f6254. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:32.390753 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2152747904}
I20260709 12:24:32.404827 16402 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
I20260709 12:24:32.409677 22137 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:32.412563 16515 leader_election.cc:400] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 20. Current term is 21.
I20260709 12:24:32.413007 16515 leader_election.cc:403] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:24:32.326936 22217 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:32.344211 22434 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:32.425385 17258 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 21.
I20260709 12:24:32.419379 22173 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:24:32.426370 16441 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.008s	sys 0.010s
I20260709 12:24:32.430011 22482 consensus_queue.cc:1048] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:24:32.432662 22216 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:32.433393 22216 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:32.434747 22480 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 20. Current term is 21.
I20260709 12:24:32.435870 22480 consensus_queue.cc:1048] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:24:32.438846 16441 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.008s	sys 0.010s
I20260709 12:24:32.441584 22354 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.442415 22354 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:32.444381 22314 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:32.444959 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:32.445724 16469 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:24:32.413260 22334 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.446924 22334 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:32.447212 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:32.448792 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:32.449800 16474 raft_consensus.cc:2393] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 21: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:32.449903 22334 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:32.454185 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:32.454600 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:32.455515 16544 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:32.455938 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:32.456844 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:32.457051 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:32.457872 16547 raft_consensus.cc:2393] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 21: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:32.458042 16551 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:32.461273 22353 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:32.462021 22353 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:32.462878 16472 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 25 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
W20260709 12:24:32.424449 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 291982us
I20260709 12:24:32.463992 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:32.470178 16476 raft_consensus.cc:1240] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:24:32.479496 16476 raft_consensus.cc:1275] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:24:32.480870 22266 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:32.482671 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 65247.8us
I20260709 12:24:32.464315 22353 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:32.486166 22332 consensus_queue.cc:1059] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:32.487922 22415 consensus_queue.cc:1059] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:24:32.486407 16471 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:24:32.490624 16474 raft_consensus.cc:1275] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:24:32.464908 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:32.498957 16475 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:32.500433 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 88194.6us
I20260709 12:24:32.469977 16516 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:32.504817 22392 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:32.505525 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:32.506551 16472 raft_consensus.cc:2410] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:24:32.507652 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:32.509193 16476 raft_consensus.cc:1275] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:24:32.510537 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:32.511132 16476 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:32.511529 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:32.512490 16468 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:24:32.464774 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:32.513450 16477 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:24:32.489207 22471 raft_consensus.cc:3055] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:24:32.432188 22482 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:32.438922 22330 raft_consensus.cc:3055] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 12:24:32.533339 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 47717.2us
I20260709 12:24:32.534109 22334 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:32.468283 21999 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:32.526898 22471 raft_consensus.cc:740] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:32.540105 22398 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.540800 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:32.541406 17258 raft_consensus.cc:1240] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:32.559273 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:32.533452 22482 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.550678 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:32.525995 22449 log.cc:927] Time spent T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.644s	user 0.005s	sys 0.000s
I20260709 12:24:32.492882 22216 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:32.563493 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:32.568198 16472 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:32.572956 22335 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:32.573931 22335 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:32.577503 22335 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:32.582376 16472 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 22.
I20260709 12:24:32.586627 22369 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:32.593349 22388 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.594189 22388 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:32.594594 22173 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:32.595161 22173 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:32.595458 22274 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:24:32.597173 22300 consensus_queue.cc:1048] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:32.598206 22393 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:32.600970 22388 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:32.602061 22398 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:32.598157 16471 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34840 (request call id 1733) took 1557 ms (client timeout 1720 ms). Trace:
W20260709 12:24:32.603049 16471 rpcz_store.cc:269] 0709 12:24:31.040637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.057515 (+ 16878us) service_pool.cc:224] Handling call
0709 12:24:32.597871 (+1540356us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54000a32d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:32.598080 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2666607872}
I20260709 12:24:32.604403 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:32.605218 21522 raft_consensus.cc:3055] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:32.607911 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:32.608388 21522 raft_consensus.cc:740] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:32.605700 16543 raft_consensus.cc:1240] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:32.609946 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:32.609335 16219 catalog_manager.cc:5697] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 6 to 12, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 12 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:24:32.609372 21522 consensus_queue.cc:260] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:32.612269 22334 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:32.612278 22369 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:32.615290 22334 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:32.616613 16477 raft_consensus.cc:2410] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:24:32.618187 21522 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:32.618217 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:32.618294 22369 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:32.620746 21999 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:32.622598 22369 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:32.626420 22412 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:24:32.598088 22489 log.cc:927] Time spent T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.060s	user 0.001s	sys 0.001s
I20260709 12:24:32.627449 22412 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:32.628567 22131 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:32.621201 17266 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 19.
I20260709 12:24:32.617666 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:32.547901 22330 raft_consensus.cc:740] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:32.641297 16468 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 20.
I20260709 12:24:32.646658 16366 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:32.650015 22369 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:32.649561 22471 consensus_queue.cc:260] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:32.651633 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:32.652050 22443 log.cc:927] Time spent T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.696s	user 0.000s	sys 0.002s
I20260709 12:24:32.652223 22412 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:32.652578 16468 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 20.
I20260709 12:24:32.629308 22131 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:32.653460 22304 raft_consensus.cc:3055] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:32.653988 22304 raft_consensus.cc:740] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:32.655695 22131 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:32.640564 16551 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
I20260709 12:24:32.655723 22304 consensus_queue.cc:260] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:32.632218 22482 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:32.670228 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:32.645290 22398 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:32.598062 22270 consensus_queue.cc:1048] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.067s
I20260709 12:24:32.691135 16471 raft_consensus.cc:2393] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 13: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
W20260709 12:24:32.698719 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 52541.1us
I20260709 12:24:32.707116 22131 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:32.707587 22131 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.707994 22131 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:32.708604 22269 consensus_meta.cc:287] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.978s	user 0.020s	sys 0.004s
I20260709 12:24:32.708520 22435 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:32.709520 22435 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:32.710182 16401 raft_consensus.cc:1275] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 12:24:32.710609 16401 pending_rounds.cc:85] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 10
I20260709 12:24:32.710839 16401 pending_rounds.cc:107] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 12:24:32.711108 16401 raft_consensus.cc:2889] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:32.664245 22354 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:32.667666 21999 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:32.584105 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:32.714556 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:32.611860 16219 catalog_manager.cc:5697] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:32.731657 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:32.729213 17144 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:32.739338 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 195442us
I20260709 12:24:32.704505 22412 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:32.728147 16366 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:32.724953 22269 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.747082 17142 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:32.733248 22435 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:32.741081 22304 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:32.745113 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:32.757459 16441 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:32.766256 16477 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
W20260709 12:24:32.768208 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 171925us
I20260709 12:24:32.756179 22471 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:32.764592 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:32.769090 17142 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:24:32.777863 22307 consensus_meta.cc:287] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.935s	user 0.015s	sys 0.007s
I20260709 12:24:32.778478 22307 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:32.780059 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:32.780665 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:32.785108 16475 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
I20260709 12:24:32.786064 16471 raft_consensus.cc:1240] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:32.773074 22398 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:32.787747 22398 raft_consensus.cc:697] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:32.785035 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:32.770946 22415 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:32.751396 22269 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:32.796669 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:32.772449 22435 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:24:32.798254 22484 log.cc:927] Time spent T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.234s	user 0.003s	sys 0.000s
I20260709 12:24:32.765044 22412 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:32.804841 17264 raft_consensus.cc:2368] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 17. Current term is 18.
I20260709 12:24:32.805598 22477 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:32.806623 22412 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:32.819341 22269 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:32.793171 22415 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:32.820509 22415 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:32.824775 22487 log.cc:927] Time spent T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:24:32.790628 22330 consensus_queue.cc:260] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:32.831349 22330 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:32.793462 16402 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:24:32.839298 16476 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 26.
W20260709 12:24:32.846905 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 94255.6us
I20260709 12:24:32.789227 22270 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:24:32.859305 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:32.863111 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 195536us
W20260709 12:24:32.868713 22297 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.987s	user 0.010s	sys 0.019s
I20260709 12:24:32.868943 22270 raft_consensus.cc:697] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 22 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:24:32.869751 22297 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.116s	user 0.011s	sys 0.019s
I20260709 12:24:32.869066 22398 consensus_queue.cc:237] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:32.870404 22270 consensus_queue.cc:237] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:32.872915 17142 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:32.869400 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:32.874317 22266 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:32.874879 22266 raft_consensus.cc:697] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:32.876670 16515 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:32.877929 22195 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:32.878592 22195 raft_consensus.cc:697] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 21 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:32.880753 22480 consensus_queue.cc:1059] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:32.882378 21922 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.883121 22480 raft_consensus.cc:3055] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:32.883594 22480 raft_consensus.cc:740] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:32.883986 17144 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:32.882902 21922 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:32.876073 22266 consensus_queue.cc:237] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:32.884316 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:32.886492 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:32.907054 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:32.891886 21522 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:32.891667 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:32.922966 16475 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 26 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 25. Current term is 26. Ops: [25.15-25.15]
I20260709 12:24:32.887125 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:32.885962 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:32.930508 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:32.931456 16470 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:32.907168 22434 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.944995 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:32.890868 22307 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:32.944763 16394 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 19.
I20260709 12:24:32.944973 16476 raft_consensus.cc:2393] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 19: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:32.886202 22298 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:32.949491 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:32.930291 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:32.953805 16402 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 22.
I20260709 12:24:32.879812 22195 consensus_queue.cc:237] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:32.958118 16366 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.720s (SENT) [suppressed 11 similar messages]
I20260709 12:24:32.977852 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:32.983844 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:32.984395 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:32.985196 16472 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 17.
I20260709 12:24:32.985332 22389 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.986042 22389 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:32.989033 16396 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:24:32.991560 22173 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:32.994589 22173 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:32.968686 16515 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:32.997116 22298 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:32.999720 22298 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:33.001354 16551 raft_consensus.cc:1240] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:33.002130 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:33.002795 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:33.004617 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:33.005116 16542 raft_consensus.cc:1275] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:24:33.002985 16544 raft_consensus.cc:2368] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 17. Current term is 18.
I20260709 12:24:33.006637 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:33.007436 16551 raft_consensus.cc:2393] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 19: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:33.010809 16548 raft_consensus.cc:1240] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:24:33.022787 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:33.004663 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:32.976904 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:33.032275 17255 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 19.
I20260709 12:24:33.044106 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:33.052850 16547 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:24:33.053611 16395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 1589) took 1684 ms. Trace:
I20260709 12:24:33.053854 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:33.054023 16395 rpcz_store.cc:276] 0709 12:24:31.368762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.375066 (+  6304us) service_pool.cc:224] Handling call
0709 12:24:31.375433 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:32.311810 (+936377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.053263 (+741453us) spinlock_profiling.cc:243] Waited 897 ms on lock 0x7b54000d2d54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:33.053519 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2063732096}
I20260709 12:24:32.962824 22353 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:24:33.061405 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 146941us
I20260709 12:24:33.056756 22353 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:32.936371 22482 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:33.070562 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:33.069047 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:33.079610 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:33.080590 16545 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:33.091493 22412 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:33.092237 22412 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:33.074028 22216 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:33.093144 22216 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:33.093472 22216 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:32.950525 16219 catalog_manager.cc:5697] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 8 to 13, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 13 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:24:33.101854 17142 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:33.068392 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:33.134269 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:32.972999 22434 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:33.151800 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 50023.9us
I20260709 12:24:33.153349 16549 raft_consensus.cc:1240] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:24:33.082104 22389 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:33.148414 16543 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:24:33.158087 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:33.157411 17142 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:33.165529 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:33.034417 22480 consensus_queue.cc:260] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:33.138259 16514 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:33.166520 16469 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 20.
I20260709 12:24:33.174427 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:33.179245 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:33.096148 22415 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:33.156411 22402 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:33.209892 16476 raft_consensus.cc:3060] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:24:33.218451 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 65563us
I20260709 12:24:33.227938 16396 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
I20260709 12:24:33.229379 17255 raft_consensus.cc:2410] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:24:33.240325 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:24:33.251472 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 136796us
I20260709 12:24:33.252588 16514 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
W20260709 12:24:33.254813 22377 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.006s	sys 0.020s
I20260709 12:24:33.212683 22402 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:33.256440 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:33.264263 17142 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:33.277156 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:33.281764 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:33.261523 22377 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.006s	sys 0.020s
I20260709 12:24:33.254983 22287 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:24:33.301539 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.287492 16514 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
W20260709 12:24:33.280882 22131 consensus_meta.cc:287] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.572s	user 0.013s	sys 0.002s
I20260709 12:24:33.273815 22402 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:33.296356 22389 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:33.302179 22480 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:33.336560 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 49096us
I20260709 12:24:33.296770 22297 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.347409 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:33.257588 22269 consensus_queue.cc:1059] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 12:24:33.307617 22412 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:33.353268 22211 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:33.353788 17254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 1549) took 1062 ms. Trace:
I20260709 12:24:33.356693 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:33.334908 22389 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:33.369462 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:33.373293 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 93246.2us
I20260709 12:24:33.352566 22369 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:33.315760 22131 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:33.409511 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:33.415443 22496 log.cc:927] Time spent T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.166s	user 0.001s	sys 0.001s
I20260709 12:24:33.389966 22269 consensus_queue.cc:1243] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer 610f4feda11942739236f2b121a4b9c1 log is divergent from this leader: its last log entry 19.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 12:24:33.419903 22304 consensus_meta.cc:287] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.662s	user 0.009s	sys 0.000s
I20260709 12:24:33.426695 17263 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:24:33.359145 17254 rpcz_store.cc:276] 0709 12:24:32.261055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.282369 (+ 21314us) service_pool.cc:224] Handling call
0709 12:24:32.283176 (+   807us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:32.284797 (+  1621us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:32.284838 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:32.285681 (+   843us) log.cc:844] Serialized 26 byte log entry
0709 12:24:32.324797 (+ 39116us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:32.468047 (+143250us) log.cc:844] Serialized 14 byte log entry
0709 12:24:32.468260 (+   213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.468358 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:33.322802 (+854444us) raft_consensus.cc:1685] finished
0709 12:24:33.322833 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.323023 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":140045,"prepare.run_cpu_time_us":194,"prepare.run_wall_time_us":197,"raft.queue_time_us":3508,"raft.run_cpu_time_us":466,"raft.run_wall_time_us":467,"thread_start_us":36520,"threads_started":1,"wal-append.queue_time_us":746743}
I20260709 12:24:33.380882 22389 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:33.380527 22287 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:33.475199 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:33.484028 22330 raft_consensus.cc:3055] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:33.484575 22330 raft_consensus.cc:740] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:33.485538 22330 consensus_queue.cc:260] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:33.487058 22330 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:33.421200 22369 raft_consensus.cc:697] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:24:33.461477 17264 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45230 (request call id 1478) took 1679 ms (client timeout 1826 ms). Trace:
I20260709 12:24:33.490370 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:33.494095 17264 rpcz_store.cc:269] 0709 12:24:31.782088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.782228 (+   140us) service_pool.cc:224] Handling call
0709 12:24:33.429054 (+1646826us) spinlock_profiling.cc:243] Waited 995 ms on lock 0x7b54009722d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:33.461382 (+ 32328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2289445632}
I20260709 12:24:33.510325 17263 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 19.
I20260709 12:24:33.476198 22385 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:33.512452 22385 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:33.512996 22385 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:33.515220 17144 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:33.516736 22354 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:33.512465 22202 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:33.517401 22354 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:33.517592 22202 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:33.517685 22398 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:33.519616 16476 raft_consensus.cc:1275] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:24:33.373322 22297 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:33.532673 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:33.533566 17254 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:24:33.499641 22434 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:33.509328 22477 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:33.548259 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:33.522424 22398 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:33.547015 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:33.518321 22304 consensus_queue.cc:1048] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:24:33.518316 22313 consensus_queue.cc:1059] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 12:24:33.565970 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:33.576092 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.492797 22287 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:33.540652 22459 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:33.584759 17254 raft_consensus.cc:1240] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:24:33.594432 17263 raft_consensus.cc:1240] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
W20260709 12:24:33.594852 22454 log.cc:927] Time spent T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
W20260709 12:24:33.595481 22444 log.cc:927] Time spent T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
I20260709 12:24:33.597060 16542 raft_consensus.cc:1275] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:24:33.597643 22297 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:33.551141 22173 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:33.598454 22173 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:33.582808 22477 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:33.600708 16367 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:33.602684 22477 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:33.567574 16219 catalog_manager.cc:5697] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 7 to 12, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 12 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
I20260709 12:24:33.605901 22287 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:33.606266 16219 catalog_manager.cc:5697] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 7 to 12, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 12 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
I20260709 12:24:33.607841 22202 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:33.594300 22369 consensus_queue.cc:237] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:33.612780 22269 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:33.561574 22398 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:33.616046 16474 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 2033) took 1001 ms. Trace:
I20260709 12:24:33.616618 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:33.617224 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:33.616536 16474 rpcz_store.cc:276] 0709 12:24:32.614226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.741538 (+127312us) service_pool.cc:224] Handling call
0709 12:24:32.752839 (+ 11301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:32.969602 (+216763us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:32.969635 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:33.011459 (+ 41824us) spinlock_profiling.cc:243] Waited 6.37 ms on lock 0x7b180048a240. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:24:33.094262 (+ 82803us) log.cc:844] Serialized 26 byte log entry
0709 12:24:33.349981 (+255719us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:33.350173 (+   192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.350449 (+   276us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:33.615760 (+265311us) raft_consensus.cc:1685] finished
0709 12:24:33.615779 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.615989 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6266,"raft.queue_time_us":54206,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"spinlock_wait_cycles":14656768,"thread_start_us":311856,"threads_started":2,"wal-append.queue_time_us":450789}
I20260709 12:24:33.617314 22297 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:33.621224 22354 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:33.623365 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:33.623736 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:33.624632 16549 raft_consensus.cc:2393] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 18: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:33.625456 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:33.626122 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:33.626490 22459 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:33.627259 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:33.627673 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:33.627867 21922 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:33.628364 17270 raft_consensus.cc:2368] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 12. Current term is 13.
I20260709 12:24:33.628371 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:33.628724 22459 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:33.626448 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:33.633831 22385 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:33.636806 22415 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:33.630755 16219 catalog_manager.cc:5697] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 5 to 13, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 13 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
I20260709 12:24:33.642285 17143 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:33.648453 21999 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:33.648926 21999 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:33.649184 22131 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:33.649613 17143 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:33.650820 22385 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:33.651312 22353 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:33.646785 16219 catalog_manager.cc:5697] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
I20260709 12:24:33.655361 22270 raft_consensus.cc:3055] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:33.655751 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:33.656174 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:33.657641 16474 raft_consensus.cc:1240] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:33.659240 22173 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:33.660491 22471 consensus_meta.cc:287] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.885s	user 0.013s	sys 0.000s
I20260709 12:24:33.551553 22435 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:24:33.665035 22485 log.cc:927] Time spent T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.668s	user 0.000s	sys 0.005s
I20260709 12:24:33.655820 22270 raft_consensus.cc:740] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:33.667371 22270 consensus_queue.cc:260] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:33.668740 22270 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:33.620889 22297 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:33.685662 21922 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:24:33.687811 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.688934 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:33.683327 16545 consensus_meta.cc:287] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.585s	user 0.006s	sys 0.003s
I20260709 12:24:33.649302 21999 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:33.692720 16545 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:24:33.693512 16550 raft_consensus.cc:1240] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:33.694412 17258 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 26.
W20260709 12:24:33.700237 16366 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.013s	sys 0.010s
W20260709 12:24:33.700700 16366 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.013s	sys 0.010s
I20260709 12:24:33.701479 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:33.702376 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 184930us
W20260709 12:24:33.700994 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 743341us
I20260709 12:24:33.713990 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:33.702131 16549 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
I20260709 12:24:33.704504 17144 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:33.737272 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 87754.2us
I20260709 12:24:33.738230 17143 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:33.746462 16441 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:33.699162 22435 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:33.767925 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:33.770380 16397 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 19.
I20260709 12:24:33.747769 22353 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.753916 22471 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:33.758842 22173 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:33.791474 22173 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:33.791923 22173 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:33.797266 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:33.758451 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:33.753711 22287 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:33.815824 16397 raft_consensus.cc:2393] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 13: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:33.792845 22353 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:33.827448 16398 raft_consensus.cc:1240] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:33.827595 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:33.834578 22435 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:33.826287 22287 raft_consensus.cc:697] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:33.835883 22287 consensus_queue.cc:237] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:33.841117 22353 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:33.857674 22297 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:33.879344 22216 consensus_meta.cc:287] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.727s	user 0.009s	sys 0.001s
I20260709 12:24:33.882992 16401 raft_consensus.cc:2393] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 16: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:33.683939 16219 catalog_manager.cc:5697] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 8 to 11, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 11 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:33.905408 16441 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:33.907193 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:33.900846 22200 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.904470 22415 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.919802 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:33.907003 22353 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:33.925747 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:33.925752 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:33.898829 22216 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:33.928695 16544 raft_consensus.cc:2410] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:24:33.929893 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:33.931018 16441 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:33.936456 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.938637 16549 raft_consensus.cc:2393] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 21: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:33.944396 16551 raft_consensus.cc:1240] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:34.016672 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:34.018671 16398 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:34.020797 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:34.022230 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:34.022850 22241 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:34.023628 22241 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:33.923041 22200 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:34.027026 22241 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:34.020011 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:34.021077 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:34.022251 16394 raft_consensus.cc:2410] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:24:34.064636 16402 raft_consensus.cc:2393] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 15: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:34.042022 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:34.029906 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:34.065727 16397 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 20.
I20260709 12:24:34.044553 16548 raft_consensus.cc:2393] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 20: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:34.033010 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:34.077121 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:34.085223 16403 raft_consensus.cc:2410] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
W20260709 12:24:34.086119 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 155389us
W20260709 12:24:34.103940 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:34.114915 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:34.027969 22354 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:34.064652 16398 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
I20260709 12:24:34.140676 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:33.926555 22415 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:34.150113 16441 leader_election.cc:400] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 12. Current term is 13.
W20260709 12:24:34.152868 22266 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.253s	user 0.049s	sys 0.101s
I20260709 12:24:34.155649 16441 leader_election.cc:403] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
W20260709 12:24:34.156524 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 68843us
I20260709 12:24:34.157094 22415 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:34.157297 16441 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:34.035785 22216 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:34.153088 16367 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.028s	sys 0.045s
I20260709 12:24:33.955083 22353 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:34.159560 16395 raft_consensus.cc:2393] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 18: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
W20260709 12:24:34.160593 16367 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.029s	sys 0.045s
I20260709 12:24:34.160808 16396 raft_consensus.cc:1275] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
W20260709 12:24:34.173944 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 571739us
I20260709 12:24:34.029457 22471 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.028141 22200 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:34.180928 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:34.160813 17254 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.025s	sys 0.053s
W20260709 12:24:34.182969 17254 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.025s	sys 0.053s
I20260709 12:24:34.208658 16401 raft_consensus.cc:2393] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 13: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
W20260709 12:24:34.172920 22335 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.744s	user 0.045s	sys 0.048s
I20260709 12:24:34.048559 22457 consensus_queue.cc:1048] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.240s
W20260709 12:24:34.202899 22266 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.293s	user 0.049s	sys 0.101s
I20260709 12:24:34.171238 22241 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 12. Current term is 13.
W20260709 12:24:34.218798 22335 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.828s	user 0.045s	sys 0.049s
W20260709 12:24:34.215458 21999 consensus_meta.cc:287] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.513s	user 0.010s	sys 0.003s
W20260709 12:24:34.202876 22412 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.018s	sys 0.024s
I20260709 12:24:34.220290 22353 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:34.241430 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:34.182037 22354 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:34.238652 22412 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.019s	sys 0.024s
I20260709 12:24:34.208053 22471 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:34.250090 16367 leader_election.cc:400] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 17. Current term is 18.
I20260709 12:24:34.245477 22522 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.254573 22523 consensus_queue.cc:1059] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:24:34.266359 16367 leader_election.cc:403] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:34.259685 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:34.223242 22330 consensus_queue.cc:1048] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:24:34.257150 22520 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:34.042420 16219 catalog_manager.cc:5697] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 6 to 8, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 8 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:34.243155 21999 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:34.285637 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 46883.6us
I20260709 12:24:34.304073 16401 raft_consensus.cc:1275] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:24:34.279278 22459 raft_consensus.cc:3055] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:24:34.322708 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 293902us
W20260709 12:24:34.338788 22402 consensus_meta.cc:287] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.992s	user 0.009s	sys 0.004s
I20260709 12:24:34.339422 22402 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:34.342470 22402 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:34.342535 22482 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:34.349233 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:34.350859 16401 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:34.272399 22522 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:34.324373 22398 consensus_meta.cc:287] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.708s	user 0.011s	sys 0.000s
I20260709 12:24:34.336849 17144 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:34.359279 16400 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.009s	sys 0.021s
W20260709 12:24:34.363152 16400 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.009s	sys 0.021s
I20260709 12:24:34.311034 22459 raft_consensus.cc:740] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:24:34.369387 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 43581.6us
W20260709 12:24:34.375404 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 88960us
W20260709 12:24:34.385372 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 317296us
W20260709 12:24:34.397953 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 240722us
I20260709 12:24:34.301713 22520 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:34.399024 16441 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:34.407829 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:34.408809 16477 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:24:34.384204 22307 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:34.414525 16516 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:34.387571 22269 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:24:34.416991 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 410808us
I20260709 12:24:34.419955 16514 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:34.420962 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:34.431319 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:34.395932 22412 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:34.401636 22353 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.433609 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:34.378544 22398 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:34.480935 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:34.404611 22241 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:34.415306 22211 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:34.416900 22313 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:34.429659 21922 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:34.417816 22330 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:34.511137 16403 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:34.418637 22307 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:34.427753 16219 catalog_manager.cc:5697] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 8 to 11, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 11 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:34.537525 22313 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:34.542023 16219 catalog_manager.cc:5697] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 11 to 14, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 14 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:34.543208 22313 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:34.537703 22211 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:34.544296 16219 catalog_manager.cc:5697] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 5 to 13, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 13 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:34.546705 22211 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:34.547298 22533 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:34.498628 22269 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:34.547729 16219 catalog_manager.cc:5697] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 14 to 15, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 15 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:34.549643 22269 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:34.501430 22524 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 17. Current term is 18.
I20260709 12:24:34.550254 22269 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.537834 21922 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:34.525451 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 70613.4us
I20260709 12:24:34.553195 21999 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:34.554513 21922 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:34.520112 22531 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:34.555160 22531 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:34.555730 22398 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:34.494465 22412 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:34.528697 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 95660.9us
W20260709 12:24:34.521152 22525 log.cc:927] Time spent T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
W20260709 12:24:34.531111 22519 log.cc:927] Time spent T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.194s	user 0.000s	sys 0.000s
I20260709 12:24:34.532470 22195 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:34.517494 22535 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:24:34.538708 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 128375us
I20260709 12:24:34.538758 16401 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 20.
I20260709 12:24:34.550604 16219 catalog_manager.cc:5697] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:24:34.439088 22459 consensus_queue.cc:260] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:34.551571 22540 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:34.500274 22353 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:34.522248 22526 log.cc:927] Time spent T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
I20260709 12:24:34.528893 22354 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:34.558115 16367 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:34.504827 22335 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:34.560443 17144 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:34.566660 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 152267us
I20260709 12:24:34.576606 16516 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:34.557655 22307 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:34.558668 22471 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:34.547804 22533 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:34.563972 22531 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:34.510133 22518 consensus_queue.cc:1059] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.061s
W20260709 12:24:34.566640 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 137831us
I20260709 12:24:34.583616 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:34.584553 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:34.587006 17144 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:34.575978 22459 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:34.558182 22330 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:34.599017 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:34.603607 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:34.624634 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:34.612253 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:34.616880 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:34.617262 16551 raft_consensus.cc:1275] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:24:34.574843 22398 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:34.617831 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:34.617918 16477 raft_consensus.cc:1275] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:24:34.567948 22195 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:34.586159 21922 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:34.552964 22269 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:34.592134 16516 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:34.641212 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:34.628777 17254 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
I20260709 12:24:34.642165 16545 raft_consensus.cc:2393] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 21: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:34.598289 22533 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:34.622236 17270 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:24:34.624010 16549 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 19.
I20260709 12:24:34.632751 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:34.634052 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:34.629410 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:34.636236 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:34.648386 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 56385.5us
I20260709 12:24:34.648913 16546 raft_consensus.cc:2393] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 19: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:34.649195 16516 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:34.645653 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:34.650380 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:34.599491 22535 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:34.623081 22531 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:34.636080 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:34.650995 16471 raft_consensus.cc:2376] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13. Re-sending same reply.
I20260709 12:24:34.608099 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:34.589674 21999 consensus_queue.cc:1048] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:24:34.573588 22540 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:34.660084 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:34.653466 16394 raft_consensus.cc:2393] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 20: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:34.599120 22211 consensus_queue.cc:1059] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 12:24:34.661064 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:34.658913 16476 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 19.
I20260709 12:24:34.664675 16516 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:34.647796 22480 consensus_meta.cc:287] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 1.286s	user 0.008s	sys 0.006s
I20260709 12:24:34.669956 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:34.670145 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:34.670917 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:34.673581 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:34.674650 16548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52680 (request call id 1350) took 1110 ms. Trace:
I20260709 12:24:34.642848 21922 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.676956 17254 raft_consensus.cc:1275] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:24:34.693847 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:34.694806 16545 raft_consensus.cc:2393] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 20: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:34.689522 16548 rpcz_store.cc:276] 0709 12:24:33.549182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.549465 (+   283us) service_pool.cc:224] Handling call
0709 12:24:34.659409 (+1109944us) spinlock_profiling.cc:243] Waited 447 ms on lock 0x7b54008c41d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:34.659555 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1027562880}
I20260709 12:24:34.629899 22304 consensus_queue.cc:1048] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:24:34.601040 22335 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:34.656688 22482 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:34.698012 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:34.649427 22534 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:34.690785 22457 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:34.657554 22531 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.719710 22531 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:34.726218 22542 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:34.727316 22530 log.cc:927] Time spent T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 12:24:34.727021 22542 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:34.727898 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:34.729224 22353 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:34.729663 22335 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:34.730232 22545 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:34.731231 22545 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:34.732561 22412 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:34.699631 21922 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:34.736457 22330 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:34.737977 22522 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:34.742377 22389 consensus_meta.cc:287] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 1.241s	user 0.013s	sys 0.000s
I20260709 12:24:34.742906 22389 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:34.744210 17264 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 21: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:34.635736 22398 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:34.745535 22173 consensus_meta.cc:287] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.934s	user 0.010s	sys 0.002s
I20260709 12:24:34.746004 22173 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:34.675786 16219 catalog_manager.cc:5697] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 2 to 11, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 11 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:34.716060 17269 raft_consensus.cc:1275] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:24:34.709167 16547 raft_consensus.cc:606] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:34.709038 16477 raft_consensus.cc:1275] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:24:34.676261 22392 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:24:34.727494 22529 log.cc:927] Time spent T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.223s	user 0.002s	sys 0.002s
I20260709 12:24:34.729357 22520 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:34.736598 22536 log.cc:927] Time spent T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 12:24:34.715258 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:34.746330 22398 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:34.748236 22173 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:34.828795 22547 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:34.785669 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:34.750708 22534 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:34.776818 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:34.838622 17271 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 20.
I20260709 12:24:34.774004 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:34.706725 22538 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:34.784716 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:34.840250 17255 raft_consensus.cc:2393] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 20: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:34.840382 16547 raft_consensus.cc:614] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:24:34.780431 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:34.841249 16550 raft_consensus.cc:1275] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:24:34.841764 22377 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.842232 22377 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:34.842932 22459 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:34.774094 17264 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45230 (request call id 1490) took 1605 ms (client timeout 2078 ms). Trace:
W20260709 12:24:34.843966 17264 rpcz_store.cc:269] 0709 12:24:33.168369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.529768 (+361399us) service_pool.cc:224] Handling call
0709 12:24:34.773887 (+1244119us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b540005da54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:34.774033 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2727822592}
I20260709 12:24:34.773082 22540 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:34.844466 22459 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:34.796882 22542 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:34.732015 22524 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:34.789565 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:34.803148 22545 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:34.785205 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:34.847939 22524 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.748669 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:34.848520 22524 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:34.839970 22538 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:34.809194 16544 raft_consensus.cc:1275] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:24:34.850726 22538 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:34.769250 22335 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:34.851559 22335 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:34.853940 22335 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:34.854017 22547 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:34.835021 22535 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:34.831629 16394 raft_consensus.cc:1275] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:24:34.754436 22202 raft_consensus.cc:3055] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:34.874505 22547 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:34.874800 22202 raft_consensus.cc:740] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:34.856752 17142 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:34.879639 22459 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:34.778546 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:34.751461 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:34.875946 22202 consensus_queue.cc:260] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:34.881090 16542 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:34.881307 22202 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:34.861835 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 303791us
I20260709 12:24:34.895375 22535 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.895978 22535 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:34.769045 22541 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:34.898552 22541 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:34.900413 22535 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:34.749886 22195 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:34.761646 22269 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:34.901276 16471 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 11.
I20260709 12:24:34.776683 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:34.838925 22307 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:34.867069 16547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:59590 (request call id 30) took 1907 ms. Trace:
I20260709 12:24:34.867818 22389 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:34.868652 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:34.913374 16400 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:34.866462 16468 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:24:34.840257 16474 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
I20260709 12:24:34.881654 22516 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:34.885095 21999 consensus_queue.cc:1048] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:34.839052 22534 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:34.901631 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 280954us
I20260709 12:24:34.855281 22377 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:34.911707 16476 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:34.905198 16367 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:34.918036 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 251460us
I20260709 12:24:34.938961 17143 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:34.918279 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:34.924911 16473 raft_consensus.cc:1275] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:24:34.911903 16547 rpcz_store.cc:276] 0709 12:24:32.975239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.038767 (+ 63528us) service_pool.cc:224] Handling call
0709 12:24:34.866815 (+1828048us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b54008c46d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e83853 00007f287ae5160a 00007f28743d77d4 00007f28743d7762 00007f28743d7691 00007f28743d760c 00007f28743d68c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:34.866950 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3807005056}
I20260709 12:24:34.930718 16542 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:34.944634 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:34.944784 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:34.945726 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:34.931411 16515 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:34.951516 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:34.953378 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:34.953905 16475 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 21.
I20260709 12:24:34.955533 17143 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:34.955677 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:34.956311 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:34.956663 16473 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:34.958384 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:34.946573 16474 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:34.963099 16515 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:34.965709 16514 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:34.965304 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 60219.1us
I20260709 12:24:34.967640 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:34.972576 22521 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:34.973412 22304 consensus_queue.cc:1048] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:34.974050 22412 raft_consensus.cc:3055] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:34.974337 22516 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:34.974517 22412 raft_consensus.cc:740] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:34.965763 22541 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:34.977694 22385 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:34.967875 22216 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:34.980470 22550 raft_consensus.cc:2804] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:34.980836 22216 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.981205 22550 raft_consensus.cc:697] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:34.982295 22550 consensus_queue.cc:237] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:34.982924 22521 consensus_queue.cc:1048] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:34.912920 22392 consensus_queue.cc:1048] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 12:24:34.985679 22289 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:34.986665 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:34.987339 16542 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:24:34.987555 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:34.944769 22518 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.988369 16544 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:34.981289 22216 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:34.894866 22524 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:34.998335 17143 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:34.940267 22297 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:24:35.001753 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.002063 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:35.001983 22297 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:35.002882 16550 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 16.
I20260709 12:24:34.988824 22518 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:35.006242 22518 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:35.005999 22412 consensus_queue.cc:260] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:35.007475 22412 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:34.998139 16514 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:34.994985 17142 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:34.945665 22547 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:35.010242 16514 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:35.011758 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:35.000317 16476 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:35.012459 16366 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
W20260709 12:24:35.017584 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 285326us
I20260709 12:24:35.016136 16550 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 21.
I20260709 12:24:34.925380 22335 consensus_queue.cc:1059] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 12:24:35.025126 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:34.965334 22471 consensus_queue.cc:1059] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 12:24:35.023001 22541 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:35.022478 22547 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:35.029922 22541 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:35.031715 22547 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:35.033720 22534 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:35.034968 22389 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:35.025921 16550 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
I20260709 12:24:35.037382 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:35.038100 22553 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:35.023506 17142 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:35.049974 16400 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 21.
I20260709 12:24:35.039300 16366 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:35.053834 22270 raft_consensus.cc:3055] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:35.054311 22270 raft_consensus.cc:740] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:35.055133 22270 consensus_queue.cc:260] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:35.056710 22553 raft_consensus.cc:3055] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:35.057134 22553 raft_consensus.cc:740] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:35.057669 17143 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:35.057986 22553 consensus_queue.cc:260] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:35.057792 16516 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:35.059135 17143 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:35.059630 16516 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:35.059957 22388 consensus_queue.cc:1048] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:34.969699 21922 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:35.072645 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:35.073321 17258 raft_consensus.cc:2393] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 20: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:35.022812 22459 raft_consensus.cc:2804] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:35.074522 16515 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:35.074769 22459 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:35.032114 22541 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:35.075225 22459 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:35.076452 22270 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:35.076656 22402 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:35.077231 22402 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:35.077610 22402 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:35.061058 22313 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:35.061102 21999 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:35.059293 22553 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:35.056861 22377 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:35.071751 16441 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:34.953289 22533 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:35.062752 22307 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:35.060182 22520 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:35.074556 21922 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:35.060679 22534 raft_consensus.cc:697] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:35.068125 22297 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:35.061154 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 44003.7us
W20260709 12:24:35.069044 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 45637.2us
I20260709 12:24:35.102735 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:35.103761 16398 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:24:35.080935 22289 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:35.085465 16544 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 16.
I20260709 12:24:35.085625 17144 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:35.070086 16516 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:35.110772 16516 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:35.112583 22521 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:35.040570 22518 consensus_queue.cc:1048] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 12:24:35.088689 22457 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:35.090047 16400 raft_consensus.cc:1275] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:24:35.096690 22541 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:35.060565 17143 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:35.076467 22523 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:35.050454 22547 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.124403 22547 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:35.090864 16441 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:35.125838 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:35.097451 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:35.093144 16366 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:35.100553 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:35.127975 16366 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:35.129911 16366 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:35.131669 16441 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:35.136529 16366 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:35.133800 21999 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:35.137909 22551 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:35.138478 22335 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:35.138593 22551 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:35.137923 22330 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:35.139225 22551 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:35.098088 22216 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:35.140579 16367 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:35.138615 22538 raft_consensus.cc:2804] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:35.141368 22538 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:35.141816 22538 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:35.142360 16367 leader_election.cc:400] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 pre-election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 21. Current term is 22.
I20260709 12:24:35.142491 22216 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:35.142765 16367 leader_election.cc:403] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:35.144660 22546 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:35.145224 22546 raft_consensus.cc:697] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 17 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:35.146149 22546 consensus_queue.cc:237] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:35.148566 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.149058 22548 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:24:35.148686 22534 consensus_queue.cc:237] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:35.149375 21999 raft_consensus.cc:697] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 21 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:35.149796 22548 raft_consensus.cc:697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 26 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:35.150512 21999 consensus_queue.cc:237] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:35.150825 22548 consensus_queue.cc:237] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.15, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:35.151629 16441 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:35.149115 22388 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:35.136976 22353 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:35.152654 22542 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:35.152953 22353 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:35.153223 22542 raft_consensus.cc:697] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:35.153470 22353 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:35.154222 22542 consensus_queue.cc:237] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:35.156167 16394 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 19.
W20260709 12:24:35.148944 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 88455.9us
I20260709 12:24:35.144050 16367 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:35.158316 17143 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:35.148036 16398 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 20.
I20260709 12:24:35.105584 17142 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:35.089874 22545 raft_consensus.cc:2764] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:24:35.089731 22377 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:35.123245 22559 consensus_queue.cc:1048] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:35.126958 22547 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:35.087502 16473 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 21.
I20260709 12:24:35.170476 17143 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:35.130692 22520 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:35.130358 22541 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:35.174017 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:35.179167 16441 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:35.107707 22313 consensus_queue.cc:1048] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 12:24:35.180616 16367 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:35.175871 22287 consensus_queue.cc:1048] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 12:24:35.182564 16367 leader_election.cc:400] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 17. Current term is 18.
I20260709 12:24:35.183496 16367 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:35.184012 22389 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:35.184501 22415 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:35.184651 22389 raft_consensus.cc:697] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:35.185134 22415 raft_consensus.cc:697] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:35.169226 22377 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:35.193881 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:35.187881 16441 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:35.187232 16367 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:35.164371 22388 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:35.165179 22335 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:35.206406 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:35.186184 22415 consensus_queue.cc:237] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:35.208642 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:35.219363 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 43970.9us
W20260709 12:24:35.211047 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.148900 22269 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 21. Current term is 22.
I20260709 12:24:35.172776 22559 consensus_queue.cc:1243] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer f99066d13e524238aec6aa2024a8bd50 log is divergent from this leader: its last log entry 20.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:24:35.174665 22553 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:35.171527 22307 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:35.212473 17144 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:35.176240 22541 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:35.185727 22389 consensus_queue.cc:237] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:35.222041 22524 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:35.171610 22270 consensus_queue.cc:1048] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:24:35.186570 22289 consensus_queue.cc:1048] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 12:24:35.199673 22330 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:24:35.226943 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 121436us
I20260709 12:24:35.199296 22549 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:35.199904 22412 raft_consensus.cc:2764] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:24:35.218680 22388 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:35.228978 22200 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:35.227815 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:35.294682 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:35.295318 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.229377 17144 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:35.219486 22335 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:35.297915 16548 raft_consensus.cc:2393] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 19: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:35.277273 16366 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:35.224699 22547 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:35.224579 22545 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:35.223987 22459 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:35.287109 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:35.291251 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:35.277292 22533 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:35.291447 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:35.291563 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:35.293113 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:35.295642 17254 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 18.
I20260709 12:24:35.302376 22388 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:35.304533 22547 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:35.305079 22518 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:35.313328 16473 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:35.314200 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:35.319645 16366 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:35.292637 22549 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:35.330729 16542 raft_consensus.cc:2393] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 12: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:35.306027 22545 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:24:35.336158 22569 log.cc:927] Time spent T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 12:24:35.320451 22547 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:35.328135 22531 consensus_meta.cc:287] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.606s	user 0.009s	sys 0.002s
I20260709 12:24:35.291337 22551 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:35.333981 22574 log.cc:927] Time spent T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:24:35.326449 22549 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:35.353343 16471 raft_consensus.cc:1240] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 21. Current term is 22. Ops: [21.11-21.11]
I20260709 12:24:35.355036 22551 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:35.342123 22553 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:35.357542 22553 raft_consensus.cc:697] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:35.358969 22553 consensus_queue.cc:237] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:35.333213 22266 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:35.360770 22266 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:35.361244 22266 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:35.315400 22524 consensus_queue.cc:1048] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:24:35.338410 22545 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:35.356429 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:35.342355 22531 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:35.371047 22531 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:35.338369 22555 raft_consensus.cc:2764] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader election decision vote started in defunct term 19: won
I20260709 12:24:35.342496 22402 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:35.405969 17263 raft_consensus.cc:2368] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 19. Current term is 20.
I20260709 12:24:35.407124 22402 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:35.423679 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 123196us
I20260709 12:24:35.428659 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:35.429979 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:35.430272 16473 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 20.
I20260709 12:24:35.431354 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:35.432839 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:35.437732 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:35.445091 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.467487 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:35.468182 16475 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:35.468849 16469 raft_consensus.cc:1240] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:24:35.494778 16400 raft_consensus.cc:1240] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 12:24:35.497664 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:35.503646 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:35.504225 17258 raft_consensus.cc:1240] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:35.504501 17254 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:35.507315 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:35.515486 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:35.517366 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:35.518388 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:35.589507 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:35.618289 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 388950us
W20260709 12:24:35.619303 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 341086us
W20260709 12:24:35.658999 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.669813 16403 raft_consensus.cc:1240] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:35.671586 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:35.677474 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:35.696583 16398 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 20.
I20260709 12:24:35.702004 17254 raft_consensus.cc:2468] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 21.
W20260709 12:24:35.705364 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 269972us
W20260709 12:24:35.706328 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 230419us
I20260709 12:24:35.709604 16403 pending_rounds.cc:85] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 10
I20260709 12:24:35.714663 16403 pending_rounds.cc:107] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 12:24:35.714984 16403 raft_consensus.cc:2889] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:35.715140 16475 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 17.
I20260709 12:24:35.726418 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:35.727908 16396 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:35.736739 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:35.722818 22202 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:35.723182 22548 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.001s	sys 0.017s
W20260709 12:24:35.722993 22353 consensus_meta.cc:287] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.002s
W20260709 12:24:35.754665 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 78368.5us
W20260709 12:24:35.756211 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 136030us
I20260709 12:24:35.755549 16516 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:35.748456 22412 consensus_queue.cc:1059] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
W20260709 12:24:35.790843 22377 consensus_meta.cc:287] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.572s	user 0.010s	sys 0.001s
I20260709 12:24:35.794291 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:24:35.800062 22590 log.cc:927] Time spent T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 12:24:35.800446 22545 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:35.801080 22581 log.cc:927] Time spent T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.298s	user 0.001s	sys 0.000s
I20260709 12:24:35.782554 22200 raft_consensus.cc:697] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 20 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:35.787200 22459 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:35.807914 17144 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:35.899413 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 143958us
I20260709 12:24:35.781751 22335 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:35.898991 22548 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.013s	sys 0.017s
W20260709 12:24:35.925211 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.934820 17143 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:35.964733 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 158330us
I20260709 12:24:35.969727 16401 raft_consensus.cc:1240] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:35.911213 22392 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:24:35.936690 22266 consensus_meta.cc:287] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.575s	user 0.009s	sys 0.002s
W20260709 12:24:35.978233 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 43534.2us
I20260709 12:24:35.982879 17143 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:35.979036 22586 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:35.985808 16397 raft_consensus.cc:1240] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:35.974792 22354 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.908819 22353 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:35.985745 17143 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:35.935897 22585 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.974946 22304 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:35.942611 22287 consensus_queue.cc:1059] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:24:35.910878 22377 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:35.994261 16516 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:36.023334 16516 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:36.005996 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.995908 16396 raft_consensus.cc:3055] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:24:35.987128 22541 consensus_meta.cc:287] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.707s	user 0.010s	sys 0.002s
I20260709 12:24:36.025002 16396 raft_consensus.cc:740] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:35.915808 22202 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:36.023025 22195 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:24:36.058996 16367 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.847s	user 0.006s	sys 0.036s
I20260709 12:24:36.059309 16545 raft_consensus.cc:1240] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 21. Current term is 22. Ops: [21.11-21.11]
I20260709 12:24:35.997299 22392 raft_consensus.cc:697] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 20 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:36.005489 22304 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:35.984066 22266 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:36.062918 16367 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.851s	user 0.007s	sys 0.036s
W20260709 12:24:36.068117 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 66430.3us
I20260709 12:24:36.053740 22289 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:24:36.070266 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 883039us
I20260709 12:24:36.055968 22522 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:35.996167 22354 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:36.073504 16367 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:36.062048 22542 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.038s	sys 0.023s
I20260709 12:24:36.056214 22523 consensus_queue.cc:1059] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 12:24:36.062188 22415 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:36.087765 16542 raft_consensus.cc:1240] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:36.022738 22585 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:36.069880 22304 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:36.086303 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:24:36.079510 22542 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.038s	sys 0.023s
I20260709 12:24:36.117631 22542 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:36.118291 22542 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:36.120625 16473 raft_consensus.cc:1240] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:24:36.056847 22541 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:36.126434 22541 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:36.133250 16475 raft_consensus.cc:1240] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 16. Current term is 17. Ops: []
W20260709 12:24:36.134955 22538 consensus_meta.cc:287] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.993s	user 0.013s	sys 0.000s
I20260709 12:24:36.135598 22538 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:36.138576 16396 consensus_queue.cc:260] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:36.139706 16396 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:36.139202 22392 consensus_queue.cc:237] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:36.141175 16470 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 26 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:24:36.142833 16474 raft_consensus.cc:1275] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:24:36.107076 16550 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:36.146345 22266 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:36.107961 22547 consensus_meta.cc:287] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.759s	user 0.011s	sys 0.000s
I20260709 12:24:36.148365 22353 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:36.148726 22547 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:36.151085 22547 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:36.151203 22335 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:36.151791 16468 raft_consensus.cc:1240] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:36.108764 22545 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:36.102622 22549 consensus_meta.cc:287] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.750s	user 0.007s	sys 0.006s
I20260709 12:24:36.111236 22585 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:36.111603 22575 log.cc:927] Time spent T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.424s	user 0.004s	sys 0.000s
I20260709 12:24:36.113673 22202 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:36.057971 22200 consensus_queue.cc:237] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:36.116551 22354 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:36.152971 16475 raft_consensus.cc:1240] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: []
W20260709 12:24:36.158378 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.161334 16470 raft_consensus.cc:1240] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:36.162391 16441 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:36.175714 16367 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:36.174173 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:36.176199 16471 raft_consensus.cc:1240] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:24:36.135627 22522 raft_consensus.cc:3055] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:36.151213 22555 raft_consensus.cc:3055] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:36.183604 16546 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:36.188443 16441 leader_election.cc:400] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 19. Current term is 20.
I20260709 12:24:36.191200 16441 leader_election.cc:403] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:24:36.175865 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 235529us
I20260709 12:24:36.138672 22538 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:36.196657 22354 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:24:36.173424 22377 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 22 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:36.203565 16366 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:36.206287 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:36.212541 16550 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 20.
W20260709 12:24:36.182323 22389 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.760s	user 0.027s	sys 0.010s
I20260709 12:24:36.189728 22547 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:36.186013 22522 raft_consensus.cc:740] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:36.189109 22555 raft_consensus.cc:740] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:36.227059 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:36.179890 22549 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:36.229991 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:36.239886 16469 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:36.230712 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:36.240175 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:36.241206 16474 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 21.
I20260709 12:24:36.233775 16472 raft_consensus.cc:1240] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:36.232465 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:36.222828 22389 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.798s	user 0.027s	sys 0.010s
I20260709 12:24:36.251236 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:36.252180 16472 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 26.
I20260709 12:24:36.208634 22545 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:24:36.248991 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 255698us
I20260709 12:24:36.249835 16470 raft_consensus.cc:2393] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 22: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:36.216367 22550 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:36.250432 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:36.262797 16472 raft_consensus.cc:1275] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:24:36.224597 22547 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:36.270820 16476 raft_consensus.cc:1240] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:36.272143 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:36.272436 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:36.235778 22541 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:36.278739 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:36.283744 17142 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:36.217913 22548 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.044s
I20260709 12:24:36.261255 22402 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.272469 22547 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:36.301596 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 255519us
I20260709 12:24:36.249946 22377 consensus_queue.cc:1059] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 12:24:36.259371 22518 consensus_queue.cc:1048] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:24:36.304236 16396 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 19.
I20260709 12:24:36.306646 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:36.271898 22542 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:36.270925 22550 raft_consensus.cc:697] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 17 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:36.319303 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:36.320052 16396 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
I20260709 12:24:36.323601 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:36.334569 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:36.302040 22545 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:36.337183 16550 raft_consensus.cc:2393] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 19: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:36.292599 22541 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:36.340926 22392 consensus_queue.cc:1059] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:36.341094 21922 consensus_queue.cc:1048] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 12:24:36.342314 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:36.343197 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 27 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:36.298144 22335 consensus_queue.cc:1059] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 12:24:36.289180 22593 consensus_queue.cc:1059] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 12:24:36.345211 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:36.315881 22389 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:24:36.364090 22389 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:36.302172 22402 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:36.306342 22600 consensus_queue.cc:1059] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 12:24:36.369616 22389 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:36.302430 22555 consensus_queue.cc:260] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:36.376821 16472 raft_consensus.cc:1240] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 21. Current term is 22. Ops: [21.11-21.11]
W20260709 12:24:36.391919 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 49418.2us
W20260709 12:24:36.392060 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 140790us
I20260709 12:24:36.394960 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:36.395691 16397 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:36.400467 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:36.403399 16442 leader_election.cc:336] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.569s (SENT)
I20260709 12:24:36.349088 22545 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:36.414156 22545 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:36.403918 16442 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:36.406955 22597 log.cc:927] Time spent T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
W20260709 12:24:36.407789 22592 log.cc:927] Time spent T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 12:24:36.432858 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.400847 22555 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:36.409837 22402 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:36.403401 22550 consensus_queue.cc:237] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:36.424522 16469 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
I20260709 12:24:36.407945 22354 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 19. Current term is 20.
I20260709 12:24:36.443751 16472 raft_consensus.cc:1240] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:24:36.444221 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:36.434275 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.449165 16473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1771) took 1015 ms. Trace:
I20260709 12:24:36.453049 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:36.449726 16473 rpcz_store.cc:276] 0709 12:24:35.434198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.434942 (+   744us) service_pool.cc:224] Handling call
0709 12:24:35.439850 (+  4908us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:36.121131 (+681281us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.448789 (+327658us) spinlock_profiling.cc:243] Waited 680 ms on lock 0x7b54002e5c54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:36.449055 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1564930816}
I20260709 12:24:36.421980 22541 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:36.465869 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 62502.6us
I20260709 12:24:36.467500 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:36.467836 16550 raft_consensus.cc:1275] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:24:36.471062 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:36.471711 16543 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:24:36.480060 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 106977us
I20260709 12:24:36.481133 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:36.482335 16442 leader_election.cc:336] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.647s (SENT)
I20260709 12:24:36.483254 16442 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:36.480293 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 47131.1us
I20260709 12:24:36.488655 22335 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.489274 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:36.491066 17142 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:36.455879 22603 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:36.456012 22538 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:36.500205 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:36.502218 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:36.503315 16396 raft_consensus.cc:2393] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 20: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:36.503456 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:36.461099 22523 consensus_queue.cc:1059] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 12:24:36.466168 22545 consensus_queue.cc:1059] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
W20260709 12:24:36.512180 16442 leader_election.cc:336] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.691s (SENT)
I20260709 12:24:36.490229 22549 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:36.513492 16442 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:36.513665 16546 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:24:36.515929 17144 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:36.518388 17270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 1584) took 1015 ms. Trace:
I20260709 12:24:36.517319 16477 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 20.
I20260709 12:24:36.489250 22335 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:36.522578 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:36.549901 16550 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 17.
I20260709 12:24:36.524032 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:36.527908 22545 consensus_queue.cc:1243] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer 52ab50b0237346c1a13ede947f6a63c2 log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:24:36.503933 22538 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:36.519341 17270 rpcz_store.cc:276] 0709 12:24:35.502845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.502981 (+   136us) service_pool.cc:224] Handling call
0709 12:24:35.503343 (+   362us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:35.704693 (+201350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:35.704723 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:35.722222 (+ 17499us) log.cc:844] Serialized 52 byte log entry
0709 12:24:36.068286 (+346064us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:36.068326 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.068410 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:36.518146 (+449736us) raft_consensus.cc:1685] finished
0709 12:24:36.518172 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:36.518331 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":200890,"raft.queue_time_us":215774,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":27338,"spinlock_wait_cycles":112896,"thread_start_us":294486,"threads_started":1,"wal-append.queue_time_us":482376}
I20260709 12:24:36.554888 22538 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:36.398592 22330 consensus_queue.cc:1175] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "52ab50b0237346c1a13ede947f6a63c2" responder_term: 17 status { last_received { term: 14 index: 9 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17. Ops: [16.10-16.10]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:24:36.543098 16515 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:36.507141 22542 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:36.544464 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:36.485757 22304 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:36.560818 17254 raft_consensus.cc:1240] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 17. Current term is 18. Ops: []
W20260709 12:24:36.561209 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 49059.1us
I20260709 12:24:36.560197 22304 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:36.524077 22603 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:36.549042 16397 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 21.
I20260709 12:24:36.546028 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:36.568854 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:36.512672 22353 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:36.569780 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:36.517747 22541 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:24:36.567629 22611 log.cc:927] Time spent T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260709 12:24:36.566238 22610 log.cc:927] Time spent T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:24:36.544731 22621 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:36.584647 16473 raft_consensus.cc:1275] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:24:36.587815 22541 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:36.591116 22541 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:36.593860 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:36.545300 22388 raft_consensus.cc:3055] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:36.600281 16401 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 21.
W20260709 12:24:36.610845 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 59346.2us
I20260709 12:24:36.520325 22600 consensus_queue.cc:1059] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.078s
I20260709 12:24:36.616928 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:36.626013 16514 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
W20260709 12:24:36.654778 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 289244us
I20260709 12:24:36.598887 22620 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:36.609534 22412 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:36.603099 22388 raft_consensus.cc:740] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:36.693222 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:36.695122 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:36.695161 16397 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
W20260709 12:24:36.702165 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 78783.8us
I20260709 12:24:36.599463 22353 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:36.720073 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:36.719604 16397 raft_consensus.cc:1275] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:24:36.717836 17143 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:36.693308 22603 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:24:36.737080 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 111198us
I20260709 12:24:36.748481 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:36.715068 22625 raft_consensus.cc:3055] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:36.794807 16397 pending_rounds.cc:85] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 9
I20260709 12:24:36.795310 16397 pending_rounds.cc:107] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:24:36.795631 16397 raft_consensus.cc:2889] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:36.755265 16442 leader_election.cc:336] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.891s (SENT)
I20260709 12:24:36.756798 22620 raft_consensus.cc:697] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 19 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:36.751677 22369 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:36.731014 22626 consensus_queue.cc:1059] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 12:24:36.730211 22619 consensus_queue.cc:1059] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:24:36.754462 22216 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:36.763828 16543 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
W20260709 12:24:36.764051 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 60624.3us
W20260709 12:24:36.765774 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 48024.1us
I20260709 12:24:36.766453 22600 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:36.767191 16403 raft_consensus.cc:1240] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 17 LEADER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:24:36.704864 22621 raft_consensus.cc:697] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:36.797998 16442 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:36.736826 22627 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.854135 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:36.836584 22596 log.cc:927] Time spent T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 12:24:36.836099 22354 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:36.910804 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:36.911625 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 156397us
I20260709 12:24:36.851373 22628 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:36.777792 22625 raft_consensus.cc:740] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:24:36.909284 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:36.973321 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.976521 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:36.856231 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 313191us
I20260709 12:24:36.859656 22216 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:37.010198 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 134716us
I20260709 12:24:37.019335 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:36.921737 22541 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:36.755007 22603 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:36.988687 22549 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:24:37.011579 22216 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:36.994139 22600 raft_consensus.cc:697] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 20 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:24:37.025959 22547 consensus_meta.cc:287] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.707s	user 0.010s	sys 0.002s
I20260709 12:24:37.009933 22402 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:37.062083 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:36.986907 22354 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:37.071642 16398 raft_consensus.cc:2393] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 18: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
W20260709 12:24:37.049862 22389 consensus_meta.cc:287] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.657s	user 0.010s	sys 0.001s
I20260709 12:24:36.911976 22627 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:37.078032 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:37.038329 22550 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:37.024685 22313 consensus_queue.cc:1048] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 12:24:37.037626 22335 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:37.008270 22628 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:37.086297 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 91922.9us
I20260709 12:24:37.059592 22547 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:37.113190 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:37.072033 22402 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:37.125902 22624 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.126807 22624 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:37.127815 22634 log.cc:927] Time spent T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:24:37.129061 16545 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 1374) took 1935 ms. Trace:
I20260709 12:24:37.129599 16545 rpcz_store.cc:276] 0709 12:24:35.194277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.195606 (+  1329us) service_pool.cc:224] Handling call
0709 12:24:35.196046 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:36.061930 (+865884us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.128732 (+1066802us) spinlock_profiling.cc:243] Waited 863 ms on lock 0x7b540082e6d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:37.128943 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1984522112}
I20260709 12:24:37.130450 22353 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:37.131829 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:37.132633 16398 raft_consensus.cc:2376] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 21. Re-sending same reply.
I20260709 12:24:37.133697 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:37.135106 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:37.135185 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:37.139816 22547 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:37.142078 22628 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:37.079681 22389 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:37.147022 17263 raft_consensus.cc:2393] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 27: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:37.148094 22389 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 27 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:37.150246 22603 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:37.089238 22550 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:37.153438 22550 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:37.117206 22631 log.cc:927] Time spent T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.352s	user 0.001s	sys 0.000s
W20260709 12:24:37.181882 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 123769us
I20260709 12:24:37.060272 22620 consensus_queue.cc:237] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:37.205075 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:37.206511 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:24:37.206900 22555 consensus_meta.cc:287] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.766s	user 0.007s	sys 0.004s
I20260709 12:24:37.208038 16395 raft_consensus.cc:1275] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 12:24:37.208614 16395 pending_rounds.cc:85] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 9
I20260709 12:24:37.208657 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:37.209088 16395 pending_rounds.cc:107] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:24:37.077828 22625 consensus_queue.cc:260] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:37.088017 22621 consensus_queue.cc:237] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:37.117066 22600 consensus_queue.cc:237] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:37.207234 17269 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:37.210099 16545 raft_consensus.cc:1275] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:24:37.221969 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:37.434610 22216 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:37.227178 22402 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:37.413335 22550 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:37.452396 22624 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:37.209697 16395 raft_consensus.cc:2889] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:37.209758 16542 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 13.
W20260709 12:24:37.484510 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 313569us
I20260709 12:24:37.230865 22354 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:37.523806 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:37.488324 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 391835us
I20260709 12:24:37.210305 22643 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:37.341293 22555 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:37.505599 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:37.526428 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:37.363392 22648 log.cc:927] Time spent T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.189s	user 0.000s	sys 0.000s
I20260709 12:24:37.521685 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:37.521798 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:37.523252 22550 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:37.504320 21922 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.527413 22647 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.529769 22643 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:37.529865 22555 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:37.531018 22601 log.cc:927] Time spent T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 1.197s	user 0.006s	sys 0.015s
I20260709 12:24:37.536741 22402 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.546815 22516 consensus_queue.cc:1059] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:37.549086 22548 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:24:37.649242 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.747308 16477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53186 (request call id 1338) took 1517 ms. Trace:
I20260709 12:24:37.746510 17254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 1491) took 1424 ms. Trace:
I20260709 12:24:37.764075 17254 rpcz_store.cc:276] 0709 12:24:36.322122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.324693 (+  2571us) service_pool.cc:224] Handling call
0709 12:24:36.325056 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:36.561355 (+236299us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.746141 (+1184786us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b5400654354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:37.746402 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":541694848}
I20260709 12:24:37.554917 17263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45224 (request call id 1493) took 1232 ms. Trace:
I20260709 12:24:37.765195 17263 rpcz_store.cc:276] 0709 12:24:36.322769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.322904 (+   135us) service_pool.cc:224] Handling call
0709 12:24:37.554686 (+1231782us) spinlock_profiling.cc:243] Waited 803 ms on lock 0x7b54005f6c54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:37.554845 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1846362496}
I20260709 12:24:37.763192 16477 rpcz_store.cc:276] 0709 12:24:36.229939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.230478 (+   539us) service_pool.cc:224] Handling call
0709 12:24:37.747084 (+1516606us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b54002e63d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:37.747231 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":530619264}
I20260709 12:24:37.553169 17143 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:37.741997 16475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 1339) took 1511 ms. Trace:
I20260709 12:24:37.762388 17269 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 16.
I20260709 12:24:37.768517 16475 rpcz_store.cc:276] 0709 12:24:36.230297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.243846 (+ 13549us) service_pool.cc:224] Handling call
0709 12:24:36.244246 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:36.342956 (+ 98710us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:36.342992 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:36.343051 (+    59us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:36.343109 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.741362 (+1398253us) spinlock_profiling.cc:243] Waited 98.4 ms on lock 0x7b5400594554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:37.741434 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:37.741824 (+   390us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":226419584}
W20260709 12:24:37.780393 16442 leader_election.cc:336] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.733s (SENT)
W20260709 12:24:37.631664 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 110259us
I20260709 12:24:37.552505 16515 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:37.792272 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:37.793514 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 240425us
I20260709 12:24:37.548044 22354 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.794194 16475 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 20.
I20260709 12:24:37.795325 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:37.805630 17269 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
I20260709 12:24:37.773007 21922 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:37.743573 22289 raft_consensus.cc:3055] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:37.802332 16476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 2076) took 2365 ms. Trace:
I20260709 12:24:37.803194 16472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1780) took 1373 ms. Trace:
I20260709 12:24:37.803943 16468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 2073) took 2465 ms. Trace:
I20260709 12:24:37.792788 22545 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:37.811486 16477 raft_consensus.cc:1240] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:37.836596 22545 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:37.839025 22545 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:37.839443 16401 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 1587) took 1325 ms. Trace:
I20260709 12:24:37.840483 17266 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 18.
I20260709 12:24:37.807396 22548 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:37.845387 22548 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:37.814811 16470 raft_consensus.cc:1240] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:24:37.816143 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:37.848174 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:37.813020 22289 raft_consensus.cc:740] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:37.813450 16472 rpcz_store.cc:276] 0709 12:24:36.430319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.430741 (+   422us) service_pool.cc:224] Handling call
0709 12:24:36.443405 (+ 12664us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:36.455666 (+ 12261us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.803021 (+1347355us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b540011add0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:37.803138 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26178816}
I20260709 12:24:37.849704 22289 consensus_queue.cc:260] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:37.851130 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:37.851416 22289 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:37.814247 16468 rpcz_store.cc:276] 0709 12:24:35.338873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.339029 (+   156us) service_pool.cc:224] Handling call
0709 12:24:35.339384 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:36.152276 (+812892us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.803703 (+1651427us) spinlock_profiling.cc:243] Waited 812 ms on lock 0x7b54004100d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:37.803871 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1867871488}
I20260709 12:24:37.820449 22522 consensus_queue.cc:260] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:37.813144 22620 raft_consensus.cc:2804] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:37.821514 22654 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.852836 22620 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.819877 21922 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:37.853084 16395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 1666) took 1216 ms. Trace:
I20260709 12:24:37.823853 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:37.853405 22620 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:37.853267 22654 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:37.854368 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:37.855022 22546 raft_consensus.cc:3055] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:37.822383 16403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 1800) took 1378 ms. Trace:
I20260709 12:24:37.855372 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:37.855657 16403 rpcz_store.cc:276] 0709 12:24:36.443814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.444920 (+  1106us) service_pool.cc:224] Handling call
0709 12:24:36.445217 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:36.871911 (+426694us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.822148 (+950237us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b54004ec5d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:37.822327 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":739725184}
I20260709 12:24:37.824843 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:37.823860 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:37.856199 17258 raft_consensus.cc:2393] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 17: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:37.859193 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:37.859607 22518 raft_consensus.cc:3055] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:37.859954 17258 raft_consensus.cc:2410] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:24:37.855443 22546 raft_consensus.cc:740] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:37.813313 16476 rpcz_store.cc:276] 0709 12:24:35.437167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.468224 (+ 31057us) service_pool.cc:224] Handling call
0709 12:24:35.468577 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:36.271334 (+802757us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.802068 (+1530734us) spinlock_profiling.cc:243] Waited 802 ms on lock 0x7b5400a681d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:37.802248 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1844498688}
I20260709 12:24:37.830523 22547 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:37.861956 22547 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:37.862288 16395 rpcz_store.cc:276] 0709 12:24:36.636704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.687044 (+ 50340us) service_pool.cc:224] Handling call
0709 12:24:36.700853 (+ 13809us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:37.520663 (+819810us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.852817 (+332154us) spinlock_profiling.cc:243] Waited 507 ms on lock 0x7b5400a104d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:37.853013 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1165262592}
W20260709 12:24:37.863121 22609 log.cc:927] Time spent T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 1.392s	user 0.016s	sys 0.021s
I20260709 12:24:37.831130 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:37.825304 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:37.832365 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:37.864743 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:37.829010 17269 raft_consensus.cc:1240] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:37.832872 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:37.835945 22614 raft_consensus.cc:3055] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:37.614410 22287 consensus_queue.cc:1048] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.180s
I20260709 12:24:37.738065 22593 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.824177 22524 raft_consensus.cc:3055] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:37.869763 17271 raft_consensus.cc:2393] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 21: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:37.853242 16398 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 11.
I20260709 12:24:37.860781 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:37.864087 17255 raft_consensus.cc:2393] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 18: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
W20260709 12:24:37.796403 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 240794us
I20260709 12:24:37.865945 16468 raft_consensus.cc:1240] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:24:37.832466 22603 consensus_queue.cc:1059] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:37.839869 16401 rpcz_store.cc:276] 0709 12:24:36.513944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.523736 (+  9792us) service_pool.cc:224] Handling call
0709 12:24:37.839251 (+1315515us) spinlock_profiling.cc:243] Waited 44.5 ms on lock 0x7b5400543154. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:37.839360 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":102358400}
I20260709 12:24:37.865442 16396 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 21.
W20260709 12:24:37.867652 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.867801 17269 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 27 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 26. Current term is 27. Ops: []
I20260709 12:24:37.807116 22402 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:37.831859 16441 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:24:37.901141 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 69370.6us
I20260709 12:24:37.549521 22550 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:37.676160 22377 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:37.916849 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:37.871600 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:37.852804 22522 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:37.873145 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:37.917729 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 45403.7us
I20260709 12:24:37.904522 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:37.904893 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:37.638439 22627 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:37.897833 16366 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:37.921656 16515 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:37.910631 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:37.868218 16368 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.033s	sys 0.077s
I20260709 12:24:37.859997 22518 raft_consensus.cc:740] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
W20260709 12:24:37.923103 16368 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.034s	sys 0.077s
I20260709 12:24:37.923163 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:37.924052 16468 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 18.
I20260709 12:24:37.927767 16468 raft_consensus.cc:1275] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:24:37.929867 17254 raft_consensus.cc:1240] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:37.919284 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:37.926635 16441 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:37.866556 22641 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.815191 22653 raft_consensus.cc:2804] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:37.866595 22614 raft_consensus.cc:740] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:37.942613 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:37.855494 22654 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:37.949002 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.959419 16367 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:37.946264 17270 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 21.
I20260709 12:24:37.869503 22524 raft_consensus.cc:740] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
W20260709 12:24:37.963285 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.955075 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:37.965054 16545 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 21.
I20260709 12:24:37.794210 22647 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:37.916193 22542 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:37.983659 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:37.941845 22304 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.990180 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:37.967214 17143 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:37.863826 22354 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:37.931278 22645 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.937541 22548 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:37.977278 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 27 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:37.948624 22550 consensus_queue.cc:1059] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 12:24:37.995556 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:37.997656 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:37.896932 22593 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:37.956209 22216 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 22 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:37.930049 22377 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:37.936910 22412 consensus_queue.cc:1059] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 12:24:37.968035 22531 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:38.008566 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:38.027395 16476 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:37.969133 22653 raft_consensus.cc:697] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 21 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:37.941485 22606 consensus_queue.cc:1048] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 12:24:38.012880 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:37.991317 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:38.013724 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:37.970067 22586 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:37.861980 22546 consensus_queue.cc:260] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:38.016534 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:38.016103 22523 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.988490 22629 consensus_queue.cc:1059] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 12:24:38.051129 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:38.018501 22657 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.989451 22297 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:38.024000 16549 raft_consensus.cc:2393] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 17: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:38.053581 21922 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:38.110586 22629 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:38.110939 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:38.111327 22629 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.111631 22643 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:38.111887 22629 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:37.989702 22270 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:38.041194 22531 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:38.112847 22270 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:38.113981 22629 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:38.114260 22531 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:38.114547 22629 raft_consensus.cc:3055] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:38.025336 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:38.114825 22270 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:37.969235 22627 consensus_queue.cc:1059] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:24:38.115662 22297 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:38.040957 22482 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:38.116780 22482 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:38.118121 22297 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:37.955919 22641 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:37.997291 22471 consensus_queue.cc:1048] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:24:38.030735 17142 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:38.032415 16545 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 18.
I20260709 12:24:38.134361 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:38.015879 22548 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:38.045161 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 78069.8us
I20260709 12:24:38.048166 16544 raft_consensus.cc:1240] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:37.987609 22547 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:38.104692 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:38.000092 22304 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:38.103287 16396 raft_consensus.cc:2393] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 18: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:38.109001 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:38.109279 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:38.112200 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:38.115962 22627 raft_consensus.cc:3055] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:37.998456 22521 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:38.036623 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 255198us
W20260709 12:24:38.174201 16442 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 2.202s (SENT) [suppressed 16 similar messages]
I20260709 12:24:38.027526 22550 consensus_queue.cc:1243] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer 52ab50b0237346c1a13ede947f6a63c2 log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:24:38.015666 22645 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:38.123764 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:38.176810 16468 raft_consensus.cc:2393] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 17: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:38.177642 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:38.179622 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 253090us
I20260709 12:24:38.127264 22620 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:38.110786 21922 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.132671 22482 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:38.133684 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 174390us
I20260709 12:24:38.184196 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:38.184983 16367 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:38.008414 22545 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:38.042742 22552 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:38.149861 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:38.154232 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:38.192848 16401 raft_consensus.cc:3060] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:38.168800 16549 raft_consensus.cc:2410] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:24:38.114954 22629 raft_consensus.cc:740] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
W20260709 12:24:38.173167 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 142515us
I20260709 12:24:38.194518 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:38.181442 16366 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:38.185712 16550 raft_consensus.cc:1240] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:38.187222 16476 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 22.
I20260709 12:24:38.149926 22618 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:24:38.159029 22548 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:38.150117 22612 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:38.198761 16514 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:38.198892 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:38.175196 22547 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:38.169215 22627 raft_consensus.cc:740] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:38.240756 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:38.242034 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:38.203701 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:38.206645 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:38.243808 16550 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 15.
I20260709 12:24:38.206815 17269 raft_consensus.cc:1240] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:38.180584 22555 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:38.208257 22482 raft_consensus.cc:2804] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:38.250877 22482 raft_consensus.cc:697] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 22 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:38.251202 17263 raft_consensus.cc:1275] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:24:38.251861 22482 consensus_queue.cc:237] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:38.253116 22645 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:38.200055 22653 consensus_queue.cc:237] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:38.149700 22622 consensus_queue.cc:1059] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.247s
I20260709 12:24:38.256008 22622 raft_consensus.cc:3055] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:38.256407 22622 raft_consensus.cc:740] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:38.256990 22669 consensus_queue.cc:1048] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:24:38.179618 22521 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:38.208849 22671 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:38.258742 22671 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:38.259184 22671 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:38.259785 22521 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:38.210251 22670 raft_consensus.cc:3055] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:38.260814 16547 raft_consensus.cc:2393] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 21: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:38.260506 22670 raft_consensus.cc:740] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:38.262792 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:38.214213 22608 log.cc:927] Time spent T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.477s	user 0.004s	sys 0.000s
I20260709 12:24:38.210815 22641 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:38.263199 22670 consensus_queue.cc:260] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:38.216802 22546 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:38.266791 22670 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:38.268414 16548 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 1386) took 2183 ms. Trace:
I20260709 12:24:38.270341 22647 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:38.271463 22626 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.269233 16548 rpcz_store.cc:276] 0709 12:24:36.085021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.089906 (+  4885us) service_pool.cc:224] Handling call
0709 12:24:36.090366 (+   460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:36.090879 (+   513us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:36.090937 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:36.091634 (+   697us) log.cc:844] Serialized 26 byte log entry
0709 12:24:36.318327 (+226693us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:36.386754 (+ 68427us) log.cc:844] Serialized 14 byte log entry
0709 12:24:36.386990 (+   236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.387070 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:37.690454 (+1303384us) spinlock_profiling.cc:243] Waited 21.4 ms on lock 0x7b1800325d80. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:24:37.828045 (+137591us) spinlock_profiling.cc:243] Waited 6.77 ms on lock 0x7b1800325d80. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d6a2a6 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1
0709 12:24:37.828552 (+   507us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b1800325d80. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:24:38.213349 (+384797us) spinlock_profiling.cc:243] Waited 3.73 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:24:38.268131 (+ 54782us) raft_consensus.cc:1685] finished
0709 12:24:38.268156 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:38.268306 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":87392,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":4433,"raft.queue_time_us":57441,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":28013,"spinlock_wait_cycles":121811328,"thread_start_us":258882,"threads_started":2,"wal-append.queue_time_us":341831}
I20260709 12:24:38.272068 22626 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:38.273031 16548 raft_consensus.cc:1240] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:24:38.274298 16548 raft_consensus.cc:1240] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:38.274446 16401 raft_consensus.cc:2468] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 22.
I20260709 12:24:38.219656 16516 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:38.275471 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:38.220441 22616 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:38.276207 22388 consensus_queue.cc:260] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:38.276908 16542 raft_consensus.cc:2410] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:24:38.276700 22616 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:38.277650 22388 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:38.278954 22616 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:38.279320 16542 raft_consensus.cc:1240] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:38.280599 22647 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:24:38.276261 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 56713.8us
I20260709 12:24:38.281926 16542 raft_consensus.cc:1275] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 12:24:38.283692 22672 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:38.284682 22667 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:38.284288 22672 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:38.285266 22667 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:38.287352 22672 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:38.287622 22667 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:38.287979 22672 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.211531 22550 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:38.137328 22657 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:38.276129 16401 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 20.
I20260709 12:24:38.224050 22354 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:38.225857 22304 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:38.182492 22297 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:38.242439 22548 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:38.210850 22385 raft_consensus.cc:3055] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:38.226524 22612 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:38.042888 22586 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:38.025295 22614 consensus_queue.cc:260] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:38.205998 22547 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:38.333062 22547 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:38.284250 22626 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:38.305560 22550 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.234100 22377 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:38.235579 22593 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:38.200526 22552 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:38.336615 22550 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:38.339262 22552 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:38.236702 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:38.340478 16549 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:38.340926 22671 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:38.343021 22522 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.288820 22672 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:38.343684 22522 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:38.237818 16545 raft_consensus.cc:1240] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:24:38.345381 22672 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:38.346001 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:38.347354 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:38.197295 22545 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:38.239537 22629 consensus_queue.cc:260] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:38.241492 22627 consensus_queue.cc:260] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:38.244625 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:38.245599 16367 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:38.203502 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:38.170521 22542 raft_consensus.cc:697] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 16 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:38.257308 22622 consensus_queue.cc:260] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:38.260494 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:38.333259 22614 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:38.221719 22660 log.cc:927] Time spent T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:24:38.288846 16542 pending_rounds.cc:85] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1: Aborting all ops after (but not including) 9
I20260709 12:24:38.364306 16542 pending_rounds.cc:107] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:24:38.364728 16542 raft_consensus.cc:2889] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:38.290984 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:38.291474 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:38.297489 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:38.367112 16401 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 19.
I20260709 12:24:38.368265 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:38.369370 16401 raft_consensus.cc:2410] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:24:38.294957 17144 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:38.370456 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 75590.4us
I20260709 12:24:38.104001 22523 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:38.306063 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.307919 16548 raft_consensus.cc:1275] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:24:38.366433 16399 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 18.
I20260709 12:24:38.186983 21922 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:38.307049 22553 consensus_queue.cc:1059] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.012s
I20260709 12:24:38.316238 22385 raft_consensus.cc:740] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:38.382105 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:38.384298 16401 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 19.
I20260709 12:24:38.385289 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:38.386477 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:38.387081 16401 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 21.
I20260709 12:24:38.324322 22612 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:38.325052 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:38.325896 16551 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 19.
I20260709 12:24:38.329241 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:38.331486 22586 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:38.332199 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:38.288297 22667 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:38.333412 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:38.216228 17271 raft_consensus.cc:1240] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 16. Current term is 17. Ops: []
W20260709 12:24:38.361646 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 116099us
I20260709 12:24:38.393256 16367 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:38.374055 16399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 1611) took 1217 ms. Trace:
I20260709 12:24:38.376272 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:38.381155 16542 raft_consensus.cc:1275] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:24:38.346043 22522 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:38.350390 22657 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:38.383297 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:38.388046 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:38.411369 16403 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
I20260709 12:24:38.388490 17263 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 17.
I20260709 12:24:38.352258 22672 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:38.390213 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:38.394158 16399 rpcz_store.cc:276] 0709 12:24:37.156345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.160969 (+  4624us) service_pool.cc:224] Handling call
0709 12:24:38.291329 (+1130360us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b5800125198. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:24:38.374001 (+ 82672us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25108224}
I20260709 12:24:38.424307 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:38.426229 17266 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 21.
I20260709 12:24:38.426364 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:38.400066 16549 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 17.
I20260709 12:24:38.403017 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:38.425267 16367 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:38.411392 16401 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:24:38.388568 17144 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:38.428653 16401 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90"
dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:59912
I20260709 12:24:38.429069 17144 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:38.412757 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:38.429975 16546 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 21.
I20260709 12:24:38.376072 22629 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:38.425239 16441 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:38.376806 22619 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:38.431738 16399 raft_consensus.cc:2410] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:24:38.374672 22668 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:24:38.437335 16442 leader_election.cc:336] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 1.770s (SENT)
I20260709 12:24:38.440140 16442 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:38.439198 16441 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:38.410092 22297 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.408658 22521 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:38.383580 22377 consensus_queue.cc:1175] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ef57beab0e1d40d585983873c55107e8" responder_term: 16 status { last_received { term: 15 index: 10 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 15. Current term is 16. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:24:38.433732 22335 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:38.430577 22548 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:38.435217 22304 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:38.431665 22674 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.455016 16401 raft_consensus.cc:606] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:38.455228 16441 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:38.439245 22603 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:38.392472 22632 consensus_queue.cc:1048] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:24:38.437152 22619 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.436796 22553 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:38.433668 22672 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:38.439576 22616 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:38.431320 22667 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:38.458105 16442 leader_election.cc:336] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 2.020s (SENT)
I20260709 12:24:38.439121 22200 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:38.439714 22657 consensus_queue.cc:1059] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 12:24:38.459015 16515 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:38.490808 22200 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:38.455508 22521 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:38.492115 22672 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:38.468281 22670 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:38.469118 22619 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:38.493569 22200 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:38.494096 22521 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:38.494482 22657 raft_consensus.cc:3055] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:38.494735 22521 raft_consensus.cc:2764] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:24:38.495029 22657 raft_consensus.cc:740] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:38.460242 16401 raft_consensus.cc:614] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:38.457653 22304 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:38.466336 16394 raft_consensus.cc:1275] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:24:38.496804 22304 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:38.455617 22297 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:38.499507 22297 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:38.474949 22545 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:38.485895 22523 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:38.466138 22674 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:38.472368 22616 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:38.501124 22616 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:38.501763 22620 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:38.502429 22674 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:38.461741 22542 consensus_queue.cc:237] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:38.491210 16515 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:38.435452 22354 consensus_queue.cc:1048] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 12:24:38.473708 16442 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:38.506487 16515 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:38.496217 22657 consensus_queue.cc:260] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:38.466307 22603 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:24:38.527587 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 69509.9us
W20260709 12:24:38.527374 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 75835.4us
I20260709 12:24:38.511075 22678 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:38.541240 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:38.510577 22523 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:38.530696 16476 raft_consensus.cc:1275] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:24:38.510519 22607 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:24:38.541237 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 86074.9us
I20260709 12:24:38.545034 16396 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:24:38.556190 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:38.455907 22385 consensus_queue.cc:260] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:38.557130 22385 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:38.528962 22657 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:38.559356 16396 raft_consensus.cc:1240] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 18. Current term is 19. Ops: []
W20260709 12:24:38.560408 22633 consensus_queue.cc:1175] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ef57beab0e1d40d585983873c55107e8" responder_term: 17 status { last_received { term: 16 index: 10 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 16. Current term is 17. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:24:38.563562 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.514595 22603 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:38.515354 22402 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:38.564707 16396 raft_consensus.cc:1275] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:24:38.565454 22518 consensus_queue.cc:260] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:38.566777 22518 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:38.568913 22553 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:38.569317 16396 raft_consensus.cc:1240] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:38.569518 22553 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:38.571504 22553 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:38.525215 22620 consensus_queue.cc:1048] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:24:38.573312 16441 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:38.551544 22625 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:38.552237 22335 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:38.553248 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:38.554162 22680 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:38.547183 22678 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:38.549655 22523 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.550900 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:38.575757 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:38.577790 16473 raft_consensus.cc:1240] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
W20260709 12:24:38.578930 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.579147 16475 raft_consensus.cc:1240] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 21. Current term is 22. Ops: [21.11-21.11]
I20260709 12:24:38.609494 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:38.581604 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:38.583014 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:38.611366 16468 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 17.
I20260709 12:24:38.612528 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:38.544418 22674 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:38.582451 22670 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:38.635238 17255 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 17.
I20260709 12:24:38.577646 17144 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:38.592110 22547 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:38.593763 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:38.564231 22667 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:38.614634 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:38.649410 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:38.650283 16543 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 19.
I20260709 12:24:38.615336 16516 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:38.610994 16401 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
W20260709 12:24:38.536649 22656 log.cc:927] Time spent T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.784s	user 0.004s	sys 0.002s
I20260709 12:24:38.584965 16471 raft_consensus.cc:1240] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:38.654310 16516 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:38.606415 22678 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:38.617102 22627 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:38.610347 16475 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 17.
W20260709 12:24:38.636376 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 58843.2us
I20260709 12:24:38.638149 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:38.635460 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:38.642704 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.645190 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:38.645483 16396 raft_consensus.cc:1275] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:24:38.591903 21922 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:38.657007 16441 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:38.659176 16516 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:38.665504 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:38.708230 16549 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:24:38.715793 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 58938.4us
I20260709 12:24:38.716144 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:38.716878 16476 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 20.
I20260709 12:24:38.636634 22605 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:38.618854 16468 raft_consensus.cc:2410] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:24:38.710410 16477 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 17.
W20260709 12:24:38.720929 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.721163 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:38.722107 16468 raft_consensus.cc:2393] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 15: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:38.722769 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:24:38.721992 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 62830.1us
I20260709 12:24:38.723577 16477 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Advancing to term 22
W20260709 12:24:38.672557 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.723819 16516 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:38.708936 16473 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
I20260709 12:24:38.727049 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:38.708122 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:38.714860 16401 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 17.
I20260709 12:24:38.729585 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:38.721885 16366 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:38.731885 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:38.621214 22523 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:24:38.739876 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.667879 22624 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:38.651718 22670 raft_consensus.cc:697] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:38.737869 16471 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
I20260709 12:24:38.737308 16366 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:38.655454 22619 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:38.740942 17258 raft_consensus.cc:2393] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 22: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:38.789297 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:38.822340 16397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 1671) took 2039 ms. Trace:
W20260709 12:24:38.784221 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.777650 16475 raft_consensus.cc:1240] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 15. Current term is 16. Ops: []
W20260709 12:24:38.779124 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.662308 22392 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:38.780897 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:38.817799 16395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 1683) took 1013 ms. Trace:
I20260709 12:24:38.826556 16477 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 22.
I20260709 12:24:38.724876 22605 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:38.740262 22626 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:38.736519 22680 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:38.733606 22667 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:38.833777 16395 rpcz_store.cc:276] 0709 12:24:37.804471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.503517 (+699046us) service_pool.cc:224] Handling call
0709 12:24:38.503792 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:38.509652 (+  5860us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:24:38.657917 (+148265us) log.cc:844] Serialized 14 byte log entry
0709 12:24:38.817330 (+159413us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:38.817403 (+    73us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:38.817470 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.817571 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:38.817719 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4438,"thread_start_us":300726,"threads_started":2}
W20260709 12:24:38.834926 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 97695.9us
I20260709 12:24:38.853221 16473 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 20.
I20260709 12:24:38.853899 16395 raft_consensus.cc:1240] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:38.753448 16549 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 22.
I20260709 12:24:38.825593 16397 rpcz_store.cc:276] 0709 12:24:36.783105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.794079 (+ 10974us) service_pool.cc:224] Handling call
0709 12:24:36.794563 (+   484us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:36.796251 (+  1688us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:36.796296 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:36.834193 (+ 37897us) log.cc:844] Serialized 26 byte log entry
0709 12:24:37.197627 (+363434us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:37.197668 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.197741 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:38.535838 (+1338097us) spinlock_profiling.cc:243] Waited 3.7 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:24:38.813405 (+277567us) raft_consensus.cc:1685] finished
0709 12:24:38.813436 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:38.822265 (+  8829us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8518400,"thread_start_us":320803,"threads_started":1,"wal-append.queue_time_us":721903}
I20260709 12:24:38.789836 17144 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:38.845906 16396 raft_consensus.cc:2393] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 22: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:38.731258 22647 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:38.852295 16514 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:38.853858 16366 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:38.859391 16514 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:38.862874 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 73114.9us
I20260709 12:24:38.740314 22553 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:38.867146 16477 raft_consensus.cc:1275] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:24:38.872170 17266 raft_consensus.cc:1240] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 20. Current term is 21. Ops: [20.10-20.10]
I20260709 12:24:38.901685 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:38.904707 16395 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:38.875175 21922 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:38.909518 16468 raft_consensus.cc:1240] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:24:38.843374 22392 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:38.842914 22605 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:38.847194 22626 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:38.920058 16398 raft_consensus.cc:2410] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:24:38.901667 22684 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:38.920989 22684 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:38.921679 16396 raft_consensus.cc:1275] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:24:38.790841 22531 consensus_queue.cc:1059] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.123s
I20260709 12:24:38.850907 22542 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.773828 22548 consensus_queue.cc:1059] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.133s
I20260709 12:24:38.874276 22624 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:38.874909 22628 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:38.876869 22553 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:38.875118 22545 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:38.949769 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:38.813199 22621 consensus_queue.cc:1048] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.094s
I20260709 12:24:38.939563 22547 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:38.904037 22402 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:38.864706 22647 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:38.860745 22667 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:38.783583 22304 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:38.938438 22687 log.cc:927] Time spent T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 12:24:38.936811 22624 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.925717 22392 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:38.919162 22335 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:38.982484 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:38.966336 22545 raft_consensus.cc:697] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 22 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:38.939218 22670 consensus_queue.cc:237] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:38.945209 16396 raft_consensus.cc:1275] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:24:38.945533 16442 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
W20260709 12:24:38.947785 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.925997 22626 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:38.907253 22522 consensus_queue.cc:1059] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 12:24:39.005007 22522 raft_consensus.cc:3055] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:39.005602 22522 raft_consensus.cc:740] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:39.006724 22522 consensus_queue.cc:260] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:38.944788 22628 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.873777 22680 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:39.008625 22628 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:39.010320 22680 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:38.947372 22553 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:38.981314 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:38.934504 22542 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:38.998277 16544 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 18.
W20260709 12:24:38.999452 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 54077.8us
I20260709 12:24:39.000106 22547 raft_consensus.cc:2804] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:39.000026 22545 consensus_queue.cc:237] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:39.001379 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:38.968722 22667 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:38.945466 22616 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:38.979137 22624 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:39.016426 16400 raft_consensus.cc:2393] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 17: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:38.971161 22647 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:39.018592 16442 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
W20260709 12:24:39.028267 16400 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 1607) took 1894 ms (client timeout 1996 ms). Trace:
I20260709 12:24:39.013862 22522 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:39.019276 22547 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:39.018101 22369 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.029799 17271 raft_consensus.cc:1240] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:39.029114 16442 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:38.964272 22402 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.030520 16468 raft_consensus.cc:2376] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 20. Re-sending same reply.
W20260709 12:24:39.028756 16400 rpcz_store.cc:269] 0709 12:24:37.133602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.136320 (+  2718us) service_pool.cc:224] Handling call
0709 12:24:37.757649 (+621329us) spinlock_profiling.cc:243] Waited 7.61 ms on lock 0x7b5800125198. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:24:38.981136 (+1223487us) spinlock_profiling.cc:243] Waited 31.7 ms on lock 0x7b5800125198. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64e9c 00007f2872d64a5b 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae4f0a0 00007f287ae4e984 00007f28743c96d4 00007f28743c9662
0709 12:24:39.028204 (+ 47068us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":90399872}
I20260709 12:24:39.033382 22680 raft_consensus.cc:2764] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:24:39.041662 16442 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:39.043560 22616 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:39.036120 22547 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:39.048769 22524 consensus_queue.cc:260] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:39.050065 22524 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:39.044559 22402 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:39.053606 16398 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 1616) took 1233 ms. Trace:
I20260709 12:24:39.054024 16398 rpcz_store.cc:276] 0709 12:24:37.820140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.025057 (+204917us) service_pool.cc:224] Handling call
0709 12:24:39.053372 (+1028315us) spinlock_profiling.cc:243] Waited 804 ms on lock 0x7b54009be6d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:39.053546 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1849822592}
I20260709 12:24:39.052376 22593 raft_consensus.cc:3055] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:24:39.056092 16442 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:39.062850 16442 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:39.044178 22674 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:39.069787 17142 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:39.077384 17142 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:39.070567 22680 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:39.073720 22616 consensus_queue.cc:1048] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:24:39.057207 22672 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:39.078457 22680 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:39.080896 22531 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:39.068042 22354 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:39.057454 22593 raft_consensus.cc:740] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:39.089206 22593 consensus_queue.cc:260] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:39.090226 22593 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:39.093323 22678 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:39.095296 22671 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:39.097237 16472 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 2101) took 1063 ms. Trace:
I20260709 12:24:39.065557 22619 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:39.076584 22621 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:24:39.044518 22369 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:39.084669 22542 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:39.085090 22680 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:39.083035 17142 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:39.044864 22603 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:39.147915 22622 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:39.115825 22550 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:39.147975 22523 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:39.116950 22552 raft_consensus.cc:3055] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:39.117228 16401 raft_consensus.cc:3060] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:39.114193 22621 consensus_queue.cc:1048] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:24:39.098088 22304 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:39.097406 22678 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:39.129436 16516 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:39.217312 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 87968.5us
I20260709 12:24:39.218369 16516 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:39.133405 22651 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:39.118281 16473 raft_consensus.cc:1275] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 12:24:39.134363 22682 log.cc:927] Time spent T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.536s	user 0.004s	sys 0.000s
I20260709 12:24:39.135740 16476 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 17.
I20260709 12:24:39.135560 22603 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:39.137472 22680 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:39.221925 17255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45224 (request call id 1506) took 1038 ms. Trace:
I20260709 12:24:39.222419 17255 rpcz_store.cc:276] 0709 12:24:38.183362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.185849 (+  2487us) service_pool.cc:224] Handling call
0709 12:24:39.189045 (+1003196us) spinlock_profiling.cc:243] Waited 433 ms on lock 0x7b540055f854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:39.221872 (+ 32827us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":995836672}
W20260709 12:24:39.107488 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:39.223824 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.136070 22335 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:39.157943 22624 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:39.158195 16395 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 22.
I20260709 12:24:39.146158 16544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37226 (request call id 1824) took 1009 ms. Trace:
I20260709 12:24:39.160738 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:39.236099 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:39.160837 22605 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:39.161612 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:39.236629 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:39.242864 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:39.235520 16544 rpcz_store.cc:276] 0709 12:24:38.136968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.170232 (+ 33264us) service_pool.cc:224] Handling call
0709 12:24:39.145982 (+975750us) spinlock_profiling.cc:243] Waited 762 ms on lock 0x7b5400729554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:39.146100 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1751860992}
I20260709 12:24:39.161734 22628 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:39.213657 22552 raft_consensus.cc:740] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:39.163232 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:39.251806 17144 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:39.254519 16396 raft_consensus.cc:2393] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 20: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
W20260709 12:24:39.165304 22685 log.cc:927] Time spent T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.581s	user 0.002s	sys 0.000s
I20260709 12:24:39.165817 22684 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:39.166610 16402 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 1603) took 2034 ms. Trace:
I20260709 12:24:39.166513 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:39.145056 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:39.147706 22647 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:39.172760 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 555531us
I20260709 12:24:39.259071 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:39.177834 22626 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:39.186976 22553 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:39.195920 22369 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:39.236404 22482 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:24:39.200613 16514 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.201538 22667 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:39.202951 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:39.283026 16516 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:39.204422 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:39.208811 16367 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:39.209311 16477 raft_consensus.cc:1275] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:24:39.211233 22354 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:24:39.287575 16546 raft_consensus.cc:1240] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:39.282743 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:24:39.133494 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 50591.8us
I20260709 12:24:39.097652 16472 rpcz_store.cc:276] 0709 12:24:38.034258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.034405 (+   147us) service_pool.cc:224] Handling call
0709 12:24:39.096542 (+1062137us) spinlock_profiling.cc:243] Waited 6.1 ms on lock 0x7b5800602a98. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:24:39.096738 (+   196us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:39.096909 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:39.096934 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:39.096973 (+    39us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:39.097031 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.097086 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:39.097181 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14020480}
I20260709 12:24:39.288551 16396 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:39.233565 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:39.244791 16469 raft_consensus.cc:1240] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 LEADER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:39.249651 16470 raft_consensus.cc:1240] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:39.289455 16473 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 20.
I20260709 12:24:39.162361 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:39.250118 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:39.291260 16400 raft_consensus.cc:2393] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 19: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:39.291373 16403 raft_consensus.cc:2393] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 17: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
W20260709 12:24:39.251921 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:39.292050 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.253224 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:39.292840 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:39.293151 16544 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:24:39.260629 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.293946 16543 raft_consensus.cc:1240] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:39.255647 16402 rpcz_store.cc:276] 0709 12:24:37.132375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.136744 (+  4369us) service_pool.cc:224] Handling call
0709 12:24:38.376510 (+1239766us) spinlock_profiling.cc:243] Waited 680 ms on lock 0x7b5800125198. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872d64b71 00007f2872d64a22 00007f287bc7d8d9 00007f287add7ee0 00007f287aea10ab 00007f287aea1131 00007f287ae62fc0 00007f287ae4d929 00007f287ae4d3d2 00007f28743c4bd4
0709 12:24:39.055631 (+679121us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:39.055945 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:24:39.055974 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:39.056017 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:39.056055 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.056122 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:39.166413 (+110291us) spinlock_profiling.cc:243] Waited 679 ms on lock 0x7b54009be6d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:39.166548 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1245,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":4546,"spinlock_wait_cycles":3124956800}
I20260709 12:24:39.223037 22651 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.287854 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:39.294570 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:39.267220 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:39.235773 22335 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:39.268541 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:39.297551 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:39.268784 22389 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:39.278357 16401 raft_consensus.cc:1275] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:24:39.278873 16475 raft_consensus.cc:1240] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:24:39.228778 22385 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:39.200001 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:39.288414 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.200026 22392 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:39.292071 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 83343.4us
I20260709 12:24:39.295670 16550 raft_consensus.cc:2376] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12. Re-sending same reply.
I20260709 12:24:39.327416 16367 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:39.272411 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:39.282421 22627 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.305438 16476 raft_consensus.cc:1275] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:24:39.306782 22647 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:39.307330 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:39.307996 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:39.239126 22547 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:39.309011 16516 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:39.309341 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:39.314311 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:39.314793 17258 raft_consensus.cc:1275] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:24:39.316392 21922 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:39.341289 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:39.342118 21922 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:39.342473 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:39.342813 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:39.343082 17258 raft_consensus.cc:2376] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 18. Re-sending same reply.
I20260709 12:24:39.310631 16399 raft_consensus.cc:2393] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 18: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:39.310527 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:39.312062 17142 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:39.344085 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:39.344305 16548 raft_consensus.cc:2393] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 22: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:39.312704 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:39.328738 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:39.345156 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:39.327906 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:39.345831 16548 raft_consensus.cc:2393] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 18: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:39.297262 22524 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:39.349599 17270 raft_consensus.cc:1240] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:39.331923 16515 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:39.330848 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.350807 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:39.332015 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:39.351524 16551 raft_consensus.cc:2368] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 14. Current term is 15.
I20260709 12:24:39.333685 16477 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 27 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 26. Current term is 27. Ops: []
I20260709 12:24:39.296748 22651 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:39.308749 22653 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:39.338457 16547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 1399) took 1008 ms. Trace:
I20260709 12:24:39.338615 17255 raft_consensus.cc:1240] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:39.339401 16367 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:39.346482 16403 raft_consensus.cc:2410] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:24:39.329631 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:39.344383 17258 raft_consensus.cc:1275] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:24:39.357743 16367 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:39.341295 16394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 1678) took 1816 ms. Trace:
I20260709 12:24:39.337522 16544 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
I20260709 12:24:39.333027 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:39.352890 16542 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:39.288599 22369 consensus_queue.cc:1048] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 12:24:39.348963 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:39.348502 22626 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:39.347347 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:39.360733 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:39.351334 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:39.361505 16548 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:39.332587 22545 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:39.355806 22630 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:39.374229 22630 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.374644 22630 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:39.374782 22628 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:39.348693 22388 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.375792 22388 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:39.376720 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:39.378166 22388 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:39.326236 22552 consensus_queue.cc:260] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:39.355103 22389 raft_consensus.cc:697] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 22 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:39.363607 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:39.380723 22389 consensus_queue.cc:237] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:39.356267 22653 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.383559 22653 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:39.353194 22522 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:39.385579 16476 raft_consensus.cc:1240] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:39.387128 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:39.390874 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:39.391687 16477 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 17.
I20260709 12:24:39.391773 16367 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:39.392091 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:39.364638 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:39.364650 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:39.364945 16474 raft_consensus.cc:1240] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:39.362792 22671 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:39.355818 22612 consensus_queue.cc:1048] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.013s
I20260709 12:24:39.370218 22553 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:39.370900 22392 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:39.373167 22593 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:39.402246 16367 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:39.356506 16547 rpcz_store.cc:276] 0709 12:24:38.330543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.351418 (+ 20875us) service_pool.cc:224] Handling call
0709 12:24:38.363400 (+ 11982us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:38.381880 (+ 18480us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:38.381923 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:38.382525 (+   602us) log.cc:844] Serialized 26 byte log entry
0709 12:24:38.508687 (+126162us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:38.508719 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.508768 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:39.133357 (+624589us) spinlock_profiling.cc:243] Waited 6.87 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:24:39.309505 (+176148us) raft_consensus.cc:1685] finished
0709 12:24:39.309528 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:39.338380 (+ 28852us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21837,"raft.queue_time_us":27002,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":17774,"spinlock_wait_cycles":15794560,"thread_start_us":64170,"threads_started":1,"wal-append.queue_time_us":144484}
I20260709 12:24:39.403682 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:39.405412 22550 raft_consensus.cc:2749] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:39.413396 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:39.410708 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:39.358358 16394 rpcz_store.cc:276] 0709 12:24:37.525108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.499593 (+974485us) service_pool.cc:224] Handling call
0709 12:24:38.499902 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:38.503852 (+  3950us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:38.503878 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:38.504631 (+   753us) log.cc:844] Serialized 26 byte log entry
0709 12:24:38.539402 (+ 34771us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:38.539456 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.539534 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:39.164670 (+625136us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:24:39.341038 (+176368us) raft_consensus.cc:1685] finished
0709 12:24:39.341059 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:39.341237 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3369,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"spinlock_wait_cycles":32197888,"thread_start_us":34015,"threads_started":1,"wal-append.queue_time_us":79242}
I20260709 12:24:39.416890 16367 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:39.422983 21922 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:39.423862 22671 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:39.426545 16548 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 22.
I20260709 12:24:39.427901 16548 raft_consensus.cc:1275] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:24:39.428076 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:39.427824 16441 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:39.436187 16396 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 21.
I20260709 12:24:39.437665 16441 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:39.440214 16441 leader_election.cc:400] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 14. Current term is 15.
I20260709 12:24:39.440603 16441 leader_election.cc:403] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:24:39.447925 16550 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 16.
I20260709 12:24:39.347044 22627 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:39.452530 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.462616 16366 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:39.458842 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:39.390133 22624 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:39.464388 16547 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
W20260709 12:24:39.470886 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 54318.2us
I20260709 12:24:39.472102 16367 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:39.412731 22402 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:39.477430 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:24:39.481287 22695 log.cc:927] Time spent T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:24:39.482040 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:39.483536 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:39.484503 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:39.484694 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:39.485169 16474 raft_consensus.cc:3060] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:39.488684 16514 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:39.488948 22550 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:39.489637 22550 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:39.490142 22550 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:39.490592 22392 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:39.491106 22392 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.491495 22392 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:39.465637 22684 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:39.493496 22522 consensus_queue.cc:1059] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:39.494797 22522 raft_consensus.cc:3055] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:39.495209 22522 raft_consensus.cc:740] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:39.496364 22518 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:39.496490 22670 consensus_queue.cc:1048] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:39.454008 22385 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:39.496173 22522 consensus_queue.cc:260] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:39.502209 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.467181 22354 raft_consensus.cc:2804] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:39.504851 22354 raft_consensus.cc:697] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:39.505776 22354 consensus_queue.cc:237] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:39.466719 22642 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:39.508891 22642 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:39.409236 22542 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.503419 16223 catalog_manager.cc:5697] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 12 to 19, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 19 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:39.512106 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:39.519769 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:39.521844 16544 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 20.
W20260709 12:24:39.528565 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 175660us
I20260709 12:24:39.533752 16474 raft_consensus.cc:2468] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 22.
I20260709 12:24:39.535369 16514 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:39.533138 16441 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:39.530614 16367 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:39.519120 16223 catalog_manager.cc:5697] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 12 to 17, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 17 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:24:39.552182 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:24:39.542549 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 192018us
I20260709 12:24:39.553601 16367 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:39.553960 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:39.549746 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:39.554905 16394 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 18.
I20260709 12:24:39.554991 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:39.550099 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:39.560674 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:39.564294 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:39.565213 16396 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:39.566205 16550 raft_consensus.cc:1275] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:24:39.521018 22552 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:39.468817 22697 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:39.553766 17143 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:39.550629 17144 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:39.573529 16399 raft_consensus.cc:1240] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 20. Current term is 21. Ops: [20.10-20.10]
W20260709 12:24:39.565209 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:39.543972 22696 log.cc:927] Time spent T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 12:24:39.459508 22628 consensus_queue.cc:1059] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.092s
I20260709 12:24:39.458606 22626 consensus_queue.cc:1059] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.092s
I20260709 12:24:39.595161 21922 consensus_queue.cc:1048] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:24:39.599541 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 48995.2us
I20260709 12:24:39.541738 22671 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:39.579476 16367 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:39.550908 16441 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:39.585662 16396 raft_consensus.cc:2410] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:24:39.526643 22542 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:39.551286 22680 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:39.549939 22641 raft_consensus.cc:2804] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:39.550500 22593 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:39.553449 22670 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:39.583822 22642 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:39.549993 22369 consensus_queue.cc:1048] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.080s
I20260709 12:24:39.553632 22385 consensus_queue.cc:1048] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.076s
W20260709 12:24:39.635716 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 84875.6us
I20260709 12:24:39.582252 22522 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:24:39.641500 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 44828.8us
W20260709 12:24:39.604506 22698 log.cc:927] Time spent T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 12:24:39.729354 22620 consensus_queue.cc:1048] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:39.577804 22523 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 22 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:39.528743 22518 consensus_queue.cc:1059] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.084s
I20260709 12:24:39.583487 22553 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:39.607373 22398 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:39.655841 22521 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:39.602077 22524 consensus_queue.cc:1048] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 12:24:39.613657 22613 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:39.685482 22670 raft_consensus.cc:697] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 16 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:39.676100 16396 raft_consensus.cc:1275] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 12:24:39.676810 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:39.761334 16396 pending_rounds.cc:85] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 10
I20260709 12:24:39.761518 16399 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:39.761621 16396 pending_rounds.cc:107] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 12:24:39.679446 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:39.634502 17143 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:39.762050 16396 raft_consensus.cc:2889] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:39.614884 22605 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:39.703612 17142 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:39.707631 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:39.708942 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:39.711395 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:39.713522 22701 log.cc:927] Time spent T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
W20260709 12:24:39.714434 22693 log.cc:927] Time spent T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
I20260709 12:24:39.722394 22542 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:39.726255 22622 consensus_meta.cc:287] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.578s	user 0.007s	sys 0.003s
I20260709 12:24:39.726665 22641 raft_consensus.cc:697] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 22 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:39.681603 22658 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:39.647401 22697 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:39.609952 22545 consensus_queue.cc:1048] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 12:24:39.656276 22377 raft_consensus.cc:3055] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:39.674222 22627 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:39.682181 22402 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:39.666424 22547 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:39.703850 22667 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:39.813930 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 179501us
I20260709 12:24:39.868659 17143 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:39.814049 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 110517us
W20260709 12:24:39.829094 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.835646 16399 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
I20260709 12:24:39.836504 22518 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.821250 22398 raft_consensus.cc:2804] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:39.758846 22521 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:39.847995 17270 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 19.
I20260709 12:24:39.822719 16545 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:24:39.756882 22523 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.872133 17142 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
W20260709 12:24:39.823626 22707 log.cc:927] Time spent T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:24:39.883528 16223 catalog_manager.cc:5697] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:24:39.818773 22388 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:39.816875 22613 raft_consensus.cc:697] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 21 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:39.843804 22550 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:39.823364 22697 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.861613 22377 raft_consensus.cc:740] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:39.897732 22616 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:39.902129 22658 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:39.887008 22518 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:39.893507 22552 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:24:39.921497 22521 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:39.922051 22605 consensus_queue.cc:1059] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:24:39.898348 22523 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:39.888677 22651 consensus_meta.cc:287] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.517s	user 0.011s	sys 0.000s
I20260709 12:24:39.940112 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:39.941238 16544 raft_consensus.cc:1275] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 12:24:39.987088 16442 leader_election.cc:336] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 2.161s (SENT)
W20260709 12:24:39.942855 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 363411us
I20260709 12:24:39.943056 16397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 1700) took 1359 ms. Trace:
I20260709 12:24:39.988061 16397 rpcz_store.cc:276] 0709 12:24:38.584114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.908722 (+324608us) service_pool.cc:224] Handling call
0709 12:24:38.928369 (+ 19647us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:38.928738 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:38.928771 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:38.931381 (+  2610us) log.cc:844] Serialized 52 byte log entry
0709 12:24:39.140051 (+208670us) spinlock_profiling.cc:243] Waited 2.56 ms on lock 0x7b18004a9ce0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:24:39.341462 (+201411us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:39.341497 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.341552 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:39.942577 (+601025us) raft_consensus.cc:1685] finished
0709 12:24:39.942611 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:39.942963 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":96,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":5186,"spinlock_wait_cycles":5888256,"thread_start_us":185298,"threads_started":1,"wal-append.queue_time_us":516322}
I20260709 12:24:39.989100 16367 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:39.961705 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:39.963768 16516 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:39.990231 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 55779.7us
I20260709 12:24:39.976312 16471 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34840 (request call id 1797) took 1328 ms. Trace:
I20260709 12:24:39.944648 16399 raft_consensus.cc:1240] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:39.992044 16471 rpcz_store.cc:276] 0709 12:24:38.647518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.869941 (+222423us) service_pool.cc:224] Handling call
0709 12:24:38.870296 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:38.870706 (+   410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:38.870745 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:38.901592 (+ 30847us) log.cc:844] Serialized 26 byte log entry
0709 12:24:39.281897 (+380305us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:39.281954 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.282034 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:39.976071 (+694037us) raft_consensus.cc:1685] finished
0709 12:24:39.976094 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:39.976253 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3913,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":26505,"thread_start_us":341138,"threads_started":1,"wal-append.queue_time_us":484424}
I20260709 12:24:40.003043 16367 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:39.938730 22697 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:39.941962 22658 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:39.981701 22624 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:40.023705 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 60014.9us
I20260709 12:24:40.024699 16442 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:40.026942 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 129104us
W20260709 12:24:40.003321 22335 consensus_meta.cc:287] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.670s	user 0.006s	sys 0.006s
W20260709 12:24:40.005959 22653 consensus_meta.cc:287] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.622s	user 0.014s	sys 0.000s
I20260709 12:24:40.041714 16442 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:40.019867 22684 consensus_meta.cc:287] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.527s	user 0.010s	sys 0.002s
I20260709 12:24:40.044054 17143 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:40.048032 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 44805us
I20260709 12:24:40.048583 16442 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:40.051855 16397 raft_consensus.cc:1275] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:24:40.055589 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:40.057356 16399 raft_consensus.cc:2393] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 18: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:40.058396 16442 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:40.019956 22711 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:39.986552 22621 consensus_queue.cc:1059] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 12:24:40.023733 22616 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.026772 22545 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:24:40.042137 22706 log.cc:927] Time spent T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 12:24:40.067723 16442 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:39.975288 22670 consensus_queue.cc:237] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:40.054194 22628 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:40.037847 22642 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:40.084578 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:40.085734 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:40.086247 16399 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 18.
I20260709 12:24:40.036638 22553 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.986783 22641 consensus_queue.cc:237] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:40.003118 22651 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:40.045475 22680 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:40.077708 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:24:40.084864 22630 consensus_meta.cc:287] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.710s	user 0.015s	sys 0.000s
I20260709 12:24:40.061519 22684 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 14. Current term is 15.
I20260709 12:24:40.063503 22542 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:40.066861 22547 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:40.101424 22630 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:40.073098 22619 consensus_queue.cc:1048] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.020s
W20260709 12:24:40.108355 22705 log.cc:927] Time spent T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
W20260709 12:24:40.113830 22710 log.cc:927] Time spent T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 12:24:40.086728 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:40.114836 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:40.115603 16544 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 12.
I20260709 12:24:40.050096 22653 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:40.074018 17144 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:40.074983 22531 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:40.075109 22711 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:40.040753 22624 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:40.046684 22335 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:40.072547 22398 raft_consensus.cc:697] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:40.081792 22616 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:40.128456 16551 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:24:40.093885 16442 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:40.094502 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:40.094702 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:40.000797 22388 consensus_queue.cc:1048] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 12:24:40.092854 22717 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:40.097630 22621 raft_consensus.cc:3055] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:40.090510 22628 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:40.070923 22304 consensus_queue.cc:1059] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 12:24:40.063278 22703 consensus_queue.cc:1048] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:24:40.104784 17264 raft_consensus.cc:2393] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 19: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:40.105073 22518 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:40.068491 22678 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:40.160377 16547 raft_consensus.cc:1240] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:24:40.119369 16367 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:40.161935 22614 consensus_queue.cc:1048] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:24:40.122146 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 48243.8us
I20260709 12:24:40.126775 22630 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:40.124809 16542 raft_consensus.cc:1240] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:24:40.049369 22377 consensus_queue.cc:260] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:40.133726 16543 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:24:40.135020 22398 consensus_queue.cc:237] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:40.136204 22542 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:40.137738 16442 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:40.137984 16549 raft_consensus.cc:2368] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 17. Current term is 18.
I20260709 12:24:40.059571 22613 consensus_queue.cc:237] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:40.098263 16397 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:40.093850 22553 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:40.152697 16395 raft_consensus.cc:1240] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:40.125088 22711 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:40.127331 22605 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:40.155354 22550 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:40.156133 22651 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:40.123762 16544 raft_consensus.cc:1275] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:24:40.170305 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:40.172649 16543 raft_consensus.cc:606] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 19 LEADER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:40.172708 16548 raft_consensus.cc:1240] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:40.180590 16473 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:24:40.172654 22518 raft_consensus.cc:2804] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:24:40.182296 22392 consensus_meta.cc:287] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.690s	user 0.011s	sys 0.003s
I20260709 12:24:40.160146 22335 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:40.184186 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 46540.2us
I20260709 12:24:40.146737 22621 raft_consensus.cc:740] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:40.206761 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:40.223462 16442 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:40.209474 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 212693us
I20260709 12:24:40.209738 16396 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:40.173172 22531 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.229406 16547 raft_consensus.cc:2393] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 16: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
W20260709 12:24:40.216130 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 53747.5us
W20260709 12:24:40.181303 22354 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.011s	sys 0.011s
I20260709 12:24:40.205646 22678 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:40.236600 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 190785us
I20260709 12:24:40.212710 22518 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.233314 16367 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:40.185437 22545 raft_consensus.cc:697] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 18 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:40.213025 22605 raft_consensus.cc:697] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:40.240178 16367 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:40.243381 16396 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 19.
I20260709 12:24:40.204751 22716 consensus_queue.cc:1059] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 12:24:40.251238 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:40.257469 16547 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 22.
I20260709 12:24:40.250950 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:40.258497 16544 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:40.258868 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:24:40.235512 22354 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.012s	sys 0.012s
I20260709 12:24:40.252897 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:40.259850 16397 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 12.
I20260709 12:24:40.220813 22392 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:40.262223 17258 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 27 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 26. Current term is 27. Ops: [26.16-26.16]
I20260709 12:24:40.255578 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:40.256254 16396 pending_rounds.cc:85] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 8
I20260709 12:24:40.256064 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:40.256412 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:40.264014 16396 pending_rounds.cc:107] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:24:40.229811 22542 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:40.264304 16396 raft_consensus.cc:2889] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:40.263643 17254 raft_consensus.cc:2368] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 21. Current term is 22.
I20260709 12:24:40.261428 16442 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:40.272938 22545 consensus_queue.cc:237] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:40.278035 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:40.241804 22518 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:40.263644 17261 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 18.
I20260709 12:24:40.267204 17143 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:40.281863 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:40.282619 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:40.282091 22621 consensus_queue.cc:260] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:40.290581 22621 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:40.293462 22553 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:40.253477 22335 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:40.250141 22724 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:40.236624 22531 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:40.237169 22388 consensus_queue.cc:1048] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 12:24:40.310674 17263 raft_consensus.cc:2393] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 20: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
W20260709 12:24:40.313424 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 80805.3us
W20260709 12:24:40.311854 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 44944.2us
W20260709 12:24:40.318037 22727 log.cc:927] Time spent T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.102s	user 0.000s	sys 0.005s
I20260709 12:24:40.320998 22392 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:40.323900 22616 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:40.309240 22630 consensus_queue.cc:1059] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 12:24:40.308601 22729 consensus_queue.cc:1059] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 12:24:40.325825 22523 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:40.266346 22542 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.327293 16547 raft_consensus.cc:1275] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:24:40.327843 17271 raft_consensus.cc:1275] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 12:24:40.328886 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 148106us
W20260709 12:24:40.349469 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.350879 16544 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
I20260709 12:24:40.353839 22731 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.354537 22717 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:40.296285 22697 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:40.355064 22731 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:40.357389 22697 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:40.283352 22670 raft_consensus.cc:2804] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:40.361094 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:40.361219 16367 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:40.361819 17258 raft_consensus.cc:2376] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 19. Re-sending same reply.
I20260709 12:24:40.363246 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:40.363937 16547 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:40.366110 16367 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:40.362910 22335 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:40.366837 22732 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:40.367606 22732 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.368052 22732 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:40.379844 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:40.358341 22542 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:40.382154 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:40.379241 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:40.358266 22354 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:24:40.391146 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 51135.4us
I20260709 12:24:40.360193 22724 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:40.299255 22658 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:40.417747 16367 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:40.425171 16367 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:40.377681 22670 raft_consensus.cc:697] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:40.423167 17144 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:40.385082 22553 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:40.299631 22605 consensus_queue.cc:237] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:40.432991 22518 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:40.452406 22658 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:40.437107 22550 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:40.454382 16397 raft_consensus.cc:1240] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:40.426455 22724 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:40.441416 22553 raft_consensus.cc:697] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 19 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:40.467026 22658 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.467741 22658 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:40.470046 22658 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:40.467056 22518 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:40.473316 22531 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:40.474262 22531 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:40.474864 22531 raft_consensus.cc:697] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 19 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:40.500684 22542 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:40.422106 22354 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:40.489530 22524 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:40.503021 22697 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.546015 22354 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:40.553259 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 116706us
I20260709 12:24:40.555083 22389 raft_consensus.cc:3055] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:40.558854 22377 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:24:40.562093 16545 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52680 (request call id 1400) took 2231 ms (client timeout 2113 ms). Trace:
W20260709 12:24:40.570274 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 119878us
I20260709 12:24:40.577479 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:40.578457 16545 rpcz_store.cc:269] 0709 12:24:38.330898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.362892 (+ 31994us) service_pool.cc:224] Handling call
0709 12:24:40.561444 (+2198552us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b54005eea54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:40.561794 (+   350us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3212646272}
I20260709 12:24:40.579115 16402 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 16.
I20260709 12:24:40.475966 22531 consensus_queue.cc:237] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:40.579646 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:40.580597 16442 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:40.582759 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 248815us
I20260709 12:24:40.580732 17143 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:40.580857 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:40.543668 22671 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:40.566816 22593 raft_consensus.cc:3055] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:40.589303 17269 raft_consensus.cc:1240] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:40.578218 22389 raft_consensus.cc:740] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:40.602346 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:40.602665 22522 consensus_queue.cc:1059] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:40.603724 22522 raft_consensus.cc:3055] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:40.605870 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:40.606487 16471 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:40.604099 22522 raft_consensus.cc:740] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:40.581388 22553 consensus_queue.cc:237] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:40.588788 22354 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:40.539736 22670 consensus_queue.cc:237] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:40.632543 22354 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:40.600723 22731 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:40.633087 22354 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:40.637820 22724 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:40.566960 22697 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:40.616915 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:40.640515 22697 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:40.587532 22671 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:40.641196 16469 raft_consensus.cc:2393] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 22: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:40.620972 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:40.607592 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:40.618338 16470 raft_consensus.cc:1240] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:24:40.615770 16471 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 19.
I20260709 12:24:40.630362 22542 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:40.589200 22593 raft_consensus.cc:740] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:40.612993 22717 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:40.637601 22732 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:40.642450 16474 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 22.
I20260709 12:24:40.645120 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:40.621662 22680 consensus_meta.cc:287] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.505s	user 0.013s	sys 0.000s
I20260709 12:24:40.659816 16403 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
W20260709 12:24:40.654572 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 58488.3us
I20260709 12:24:40.622453 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:40.658216 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:40.665642 21922 consensus_queue.cc:1048] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:40.660454 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:40.652693 16400 raft_consensus.cc:1240] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:40.663390 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:40.665975 16399 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:40.661136 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:40.667753 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:40.668886 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:40.669044 22732 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 23 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:40.669667 17258 raft_consensus.cc:2393] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 22: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:40.670485 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:40.671070 17258 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 17.
I20260709 12:24:40.671509 17143 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:40.654496 22398 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:40.673095 22736 log.cc:927] Time spent T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:24:40.673004 22398 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:40.673514 22754 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:40.674085 22754 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.674491 22754 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:40.604589 22632 consensus_queue.cc:1048] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 12:24:40.606933 22605 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:40.681353 17254 raft_consensus.cc:1240] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:40.704021 17142 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:40.677899 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:40.666518 17144 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:40.673127 22737 log.cc:927] Time spent T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:24:40.662498 22697 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:40.724854 16399 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 19.
I20260709 12:24:40.718130 17142 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
W20260709 12:24:40.741117 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 74666.5us
I20260709 12:24:40.741979 21922 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:40.739389 16442 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:40.742403 21922 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.742919 21922 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:40.688691 22680 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:40.744823 17266 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:24:40.746225 16470 raft_consensus.cc:1240] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:40.752699 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:40.704736 22398 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:40.655249 22713 consensus_queue.cc:1059] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.074s
I20260709 12:24:40.763933 16471 raft_consensus.cc:2393] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 19: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:40.607707 22522 consensus_queue.cc:260] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:40.728578 22734 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:40.715817 22542 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:40.663787 22717 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:40.774955 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 56950.8us
I20260709 12:24:40.777027 17142 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:40.779259 21922 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:40.783682 16515 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:40.789381 16515 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:40.779820 22711 consensus_meta.cc:287] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.567s	user 0.014s	sys 0.001s
I20260709 12:24:40.681665 22593 consensus_queue.cc:260] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:40.790663 21922 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:40.784824 22552 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:40.702680 22389 consensus_queue.cc:260] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:40.782003 22398 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:24:40.754522 22697 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:40.806874 16395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 1710) took 1555 ms. Trace:
I20260709 12:24:40.797784 16515 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:40.805579 22522 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:40.796983 22684 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:40.810230 22593 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:40.782506 22542 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:40.844895 22335 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:40.847162 22717 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:40.847393 22697 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:40.848105 22753 log.cc:927] Time spent T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:24:40.796700 22613 consensus_queue.cc:1059] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 12:24:40.800587 22711 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:40.801782 22713 consensus_queue.cc:1059] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 12:24:40.795306 22712 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:24:40.851119 22678 consensus_meta.cc:287] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2: Time spent flushing consensus metadata: real 0.608s	user 0.012s	sys 0.000s
W20260709 12:24:40.853271 22751 log.cc:927] Time spent T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:24:40.797336 22625 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:24:40.862783 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 123353us
I20260709 12:24:40.859305 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:40.894323 16470 raft_consensus.cc:2393] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 19: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:40.864351 22552 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:40.898309 22711 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:40.898181 22552 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:40.895057 22625 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.899336 22678 raft_consensus.cc:2749] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:40.899487 22625 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:40.892220 22377 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:40.867908 22628 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:40.875461 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:40.902300 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.903266 16516 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:40.866559 22714 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:40.904488 22628 consensus_queue.cc:1059] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:40.904623 22714 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:40.905575 22628 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:40.906188 22628 raft_consensus.cc:3055] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:40.906503 22628 raft_consensus.cc:740] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:40.906665 22714 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:40.885128 17263 raft_consensus.cc:1275] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:24:40.891157 16398 raft_consensus.cc:1240] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:40.907325 22628 consensus_queue.cc:260] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:40.909660 22628 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:40.897464 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:40.910341 16470 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:40.900687 22552 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:40.921499 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:40.903842 22542 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:40.877439 16395 rpcz_store.cc:276] 0709 12:24:39.251193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.352706 (+101513us) service_pool.cc:224] Handling call
0709 12:24:39.382964 (+ 30258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:40.185684 (+802720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.806562 (+620878us) spinlock_profiling.cc:243] Waited 769 ms on lock 0x7b5400a104d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:40.806798 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1769576192}
I20260709 12:24:40.908507 16516 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:40.887918 22684 raft_consensus.cc:697] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:24:40.895912 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 133892us
I20260709 12:24:40.926374 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:40.928639 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:40.928656 16548 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "766cb112f8024db5af150d6be611a597"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:24:40.928917 16543 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 15.
I20260709 12:24:40.929412 16548 raft_consensus.cc:606] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:40.929798 16548 raft_consensus.cc:614] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:40.927170 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:40.930361 16544 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:40.930671 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:40.931211 16545 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:40.929809 22759 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:40.943535 16544 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 20.
I20260709 12:24:40.925311 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:40.913053 22697 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.946307 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:40.946841 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:40.950480 16472 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:40.948766 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:40.949657 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:40.950384 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:40.951610 16548 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:40.951807 16544 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:24:40.889065 22671 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:40.953933 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:40.947862 16545 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
W20260709 12:24:40.956264 16366 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.906584 22354 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:40.957053 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 83953.6us
I20260709 12:24:40.959306 22553 consensus_queue.cc:1059] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:40.959972 16469 raft_consensus.cc:1275] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:24:40.974408 16543 raft_consensus.cc:2393] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 16: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:40.959864 22684 consensus_queue.cc:237] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:40.953069 22398 consensus_queue.cc:1059] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 12:24:40.962574 22742 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.970521 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:40.971910 16470 raft_consensus.cc:2468] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 19.
I20260709 12:24:40.980835 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:40.982350 16401 raft_consensus.cc:1240] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:40.982817 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:40.983815 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:40.991200 16396 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:40.991235 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 193553us
I20260709 12:24:40.984781 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:40.991827 16544 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:24:40.992477 16544 raft_consensus.cc:606] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
W20260709 12:24:40.986514 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.992920 16544 raft_consensus.cc:614] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:24:40.994534 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.986532 16545 raft_consensus.cc:1275] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:24:40.928051 22754 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:40.995530 16548 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 23.
I20260709 12:24:40.990006 16367 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:40.996719 16476 raft_consensus.cc:1240] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:41.002347 16472 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 23.
I20260709 12:24:41.010159 16367 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:41.012108 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:40.956602 22697 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:40.979491 22518 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:41.019529 16396 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 18.
I20260709 12:24:40.985497 22545 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:41.021792 22732 raft_consensus.cc:3055] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:41.037659 22518 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:41.007025 22335 consensus_queue.cc:1059] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:24:41.041476 22335 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:41.042028 22335 raft_consensus.cc:697] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 20 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:41.009503 22711 consensus_queue.cc:1048] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 12:24:41.045275 22711 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:41.045831 22711 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:41.046067 22354 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:41.047108 22724 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:41.047679 22724 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:41.048215 22724 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:41.042973 22335 consensus_queue.cc:237] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:41.048108 22711 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:41.063289 22711 raft_consensus.cc:3055] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:41.064147 22711 raft_consensus.cc:740] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:41.002506 22605 consensus_queue.cc:1059] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 12:24:41.022073 22553 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:41.040223 22518 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:41.086820 22402 raft_consensus.cc:3055] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:41.066053 22711 consensus_queue.cc:260] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:41.035737 22732 raft_consensus.cc:740] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:24:41.081523 22721 log.cc:927] Time spent T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:24:41.001272 22742 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:41.101828 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 98145.4us
I20260709 12:24:41.090368 22765 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:41.112051 16516 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:41.055579 22545 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:41.096217 22402 raft_consensus.cc:740] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:41.114578 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:41.121426 17143 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:41.096109 22553 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:41.122601 16516 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:41.117039 22765 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:41.124696 22671 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:41.125046 22605 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:41.113675 22772 raft_consensus.cc:3055] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:41.145545 16395 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:24:41.125289 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:41.125496 22377 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:41.126852 22754 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:41.110481 22759 consensus_queue.cc:1048] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 12:24:41.114714 22711 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:41.169839 17144 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:41.154206 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 144085us
I20260709 12:24:41.159545 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:41.167181 16402 raft_consensus.cc:1275] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:24:41.168512 16403 raft_consensus.cc:1275] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 12:24:41.171835 16396 raft_consensus.cc:3055] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:41.172132 16399 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:41.172277 16396 raft_consensus.cc:740] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:41.171039 16367 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:41.173293 16396 consensus_queue.cc:260] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:41.174641 16396 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:41.177623 22553 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:41.178992 22377 raft_consensus.cc:2804] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:41.179631 22377 raft_consensus.cc:697] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 18 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:41.179762 22778 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:41.180392 22778 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:41.180579 22402 consensus_queue.cc:260] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:41.181856 22402 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:41.183525 16394 raft_consensus.cc:2393] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 19: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:41.148324 22772 raft_consensus.cc:740] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:41.145233 22605 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:41.188098 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:41.194116 16396 raft_consensus.cc:1275] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 12:24:41.194713 16396 pending_rounds.cc:85] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 6
I20260709 12:24:41.195133 16396 pending_rounds.cc:107] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 12:24:41.194793 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:41.195518 16396 raft_consensus.cc:2889] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:41.199139 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:41.202207 22625 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:41.130636 22732 consensus_queue.cc:260] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:41.205384 22625 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:41.206159 22735 raft_consensus.cc:3055] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:41.206264 22625 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.206712 22735 raft_consensus.cc:740] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:41.206900 22625 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:41.184161 22771 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.208415 22771 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:41.210495 22772 consensus_queue.cc:260] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:41.211906 22772 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:41.212585 22377 consensus_queue.cc:237] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:41.213407 22545 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:41.217618 16550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37242 (request call id 1707) took 1511 ms. Trace:
I20260709 12:24:41.205224 22732 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:41.218001 16550 rpcz_store.cc:276] 0709 12:24:39.706711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.708271 (+  1560us) service_pool.cc:224] Handling call
0709 12:24:41.217416 (+1509145us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b5400a4f454. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:41.217559 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2593439616}
W20260709 12:24:41.219816 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.185978 22754 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:41.221344 22765 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:41.225502 22742 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:41.226667 22697 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:41.190282 22605 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:41.190835 22593 raft_consensus.cc:2804] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:41.229566 22593 raft_consensus.cc:697] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:41.230535 22593 consensus_queue.cc:237] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:41.231899 16514 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:41.232426 16545 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:41.233846 16544 raft_consensus.cc:1275] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 10. (term mismatch)
I20260709 12:24:41.216133 16223 catalog_manager.cc:5697] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:24:41.234277 16544 pending_rounds.cc:85] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1: Aborting all ops after (but not including) 9
I20260709 12:24:41.234632 16544 pending_rounds.cc:107] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1: Aborting uncommitted NO_OP operation due to leader change: 20.10
W20260709 12:24:41.228843 16549 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37242 (request call id 1704) took 1921 ms (client timeout 1847 ms). Trace:
I20260709 12:24:41.234895 16544 raft_consensus.cc:2889] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:41.198421 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:24:41.235011 16549 rpcz_store.cc:269] 0709 12:24:39.307242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.331758 (+ 24516us) service_pool.cc:224] Handling call
0709 12:24:41.228643 (+1896885us) spinlock_profiling.cc:243] Waited 786 ms on lock 0x7b540037ee54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:41.228770 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1807519616}
I20260709 12:24:41.207688 22735 consensus_queue.cc:260] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:41.247530 22735 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:41.187529 22627 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.865s	user 0.000s	sys 0.023s
I20260709 12:24:41.234947 17143 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:41.261940 16550 raft_consensus.cc:1275] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:24:41.265273 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:41.266966 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:41.254037 16223 catalog_manager.cc:5697] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 11 to 18, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 18 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:24:41.270642 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:41.273389 16470 raft_consensus.cc:1240] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:41.274502 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:41.275419 16472 raft_consensus.cc:2410] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 28 because replica has last-logged OpId of term: 26 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 15.
I20260709 12:24:41.248783 22522 consensus_queue.cc:1048] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.016s
I20260709 12:24:41.278095 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:41.276458 22627 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.954s	user 0.000s	sys 0.029s
I20260709 12:24:41.210577 22771 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:41.304359 16474 raft_consensus.cc:1240] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:24:41.318935 22626 raft_consensus.cc:3055] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:41.319454 22626 raft_consensus.cc:740] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:41.320788 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:41.321961 16549 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:41.324020 16551 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37242 (request call id 1708) took 1617 ms (client timeout 1500 ms). Trace:
I20260709 12:24:41.304672 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:41.326176 17266 raft_consensus.cc:2393] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 19: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
W20260709 12:24:41.324745 16551 rpcz_store.cc:269] 0709 12:24:39.707085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.707276 (+   191us) service_pool.cc:224] Handling call
0709 12:24:41.323805 (+1616529us) spinlock_profiling.cc:243] Waited 297 ms on lock 0x7b5400bb5654. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:41.323943 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":684158336}
I20260709 12:24:41.306388 17254 raft_consensus.cc:1275] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:24:41.334587 22388 consensus_queue.cc:1048] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:41.307126 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:41.336388 22388 consensus_queue.cc:1059] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:24:41.338075 22709 log.cc:927] Time spent T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 1.274s	user 0.009s	sys 0.000s
I20260709 12:24:41.310403 22553 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:41.339622 22553 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:41.340231 22723 raft_consensus.cc:3055] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:41.340683 22723 raft_consensus.cc:740] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:41.341455 22723 consensus_queue.cc:260] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:41.341833 22553 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:41.313941 22658 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 12:24:41.314772 22392 raft_consensus.cc:3055] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:41.342558 22723 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:41.304850 16397 raft_consensus.cc:2410] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 28 because replica has last-logged OpId of term: 26 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 15.
I20260709 12:24:41.328431 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:41.338774 16468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34826 (request call id 2130) took 1260 ms. Trace:
I20260709 12:24:41.342665 22392 raft_consensus.cc:740] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:41.343794 16468 rpcz_store.cc:276] 0709 12:24:40.078096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.078267 (+   171us) service_pool.cc:224] Handling call
0709 12:24:40.078588 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:40.126931 (+ 48343us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:40.126954 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:40.126985 (+    31us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:40.127006 (+    21us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:41.338563 (+1211557us) spinlock_profiling.cc:243] Waited 48.1 ms on lock 0x7b540058cd54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:41.338610 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:41.338722 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":110728448}
I20260709 12:24:41.345034 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:41.320313 22626 consensus_queue.cc:260] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:41.308178 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.276422 22726 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:41.352419 17258 raft_consensus.cc:2410] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:24:41.354193 16367 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
W20260709 12:24:41.345841 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.338828 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:41.350535 17270 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 20.
W20260709 12:24:41.306138 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 309771us
I20260709 12:24:41.344187 17271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 1520) took 1089 ms. Trace:
I20260709 12:24:41.264694 22628 raft_consensus.cc:2804] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:24:41.306648 16441 connection.cc:256] RPC call timeout handler was delayed by -1.2529176279999774s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19930000000000003s for a response
I20260709 12:24:41.355491 16472 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:24:41.355566 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:41.359596 16550 raft_consensus.cc:2393] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 20: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:41.355779 17271 rpcz_store.cc:276] 0709 12:24:40.254596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.254785 (+   189us) service_pool.cc:224] Handling call
0709 12:24:40.269607 (+ 14822us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:40.339713 (+ 70106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:41.343983 (+1004270us) spinlock_profiling.cc:243] Waited 40.9 ms on lock 0x7b54005f3554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:41.344131 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":94019328}
I20260709 12:24:41.357080 16223 catalog_manager.cc:5697] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 12 to 17. New cstate: current_term: 17 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:24:41.367511 16474 raft_consensus.cc:1240] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:41.368914 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:41.372491 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:41.262450 22621 consensus_queue.cc:1048] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.095s
I20260709 12:24:41.375761 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:41.326776 22782 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:41.395361 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:41.308297 22778 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:41.395983 22782 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:41.396615 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:41.397794 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:41.397825 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:41.398463 22782 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:41.398913 16397 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 27.
I20260709 12:24:41.377985 17142 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:41.376622 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:41.402711 17142 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:41.381886 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:41.382189 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:41.375640 22726 raft_consensus.cc:697] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 23 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:41.383713 22746 consensus_queue.cc:1048] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:41.384434 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:41.385022 22731 consensus_queue.cc:1059] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:41.389497 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:41.328379 22653 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:41.393960 16544 raft_consensus.cc:1240] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:41.394764 16546 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1698) took 2101 ms. Trace:
I20260709 12:24:41.394917 22624 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:41.398849 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:41.395395 22392 consensus_queue.cc:260] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:41.345368 22754 consensus_queue.cc:1048] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 12:24:41.373450 22354 consensus_queue.cc:1059] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 12:24:41.374485 22628 raft_consensus.cc:697] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 19 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:41.364023 22717 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.403760 22616 raft_consensus.cc:3055] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:41.368721 22658 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:41.520064 22354 raft_consensus.cc:3055] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:41.402799 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:41.522806 22726 consensus_queue.cc:237] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:41.523859 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:24:41.406180 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.495275 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:41.525106 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:41.525758 16544 raft_consensus.cc:2393] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 23: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:41.525869 16477 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 27.
I20260709 12:24:41.526875 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:41.526939 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:41.374316 16549 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
I20260709 12:24:41.402803 22630 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:41.413199 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:41.420769 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:41.420085 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.419860 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:41.422991 17142 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:41.419476 22724 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:41.534422 22630 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:41.536274 22724 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:41.526206 22717 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:41.533126 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:41.539402 16477 raft_consensus.cc:2393] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 19: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:41.539300 16474 raft_consensus.cc:1240] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:41.539832 16469 raft_consensus.cc:1275] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:24:41.495536 16546 rpcz_store.cc:276] 0709 12:24:39.293186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.424921 (+131735us) service_pool.cc:224] Handling call
0709 12:24:39.425232 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:39.425570 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:39.425597 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:39.426030 (+   433us) log.cc:844] Serialized 26 byte log entry
0709 12:24:39.784086 (+358056us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:39.784149 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.784255 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:41.337517 (+1553262us) spinlock_profiling.cc:243] Waited 6.29 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:24:41.394371 (+ 56854us) raft_consensus.cc:1685] finished
0709 12:24:41.394394 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:41.394697 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":29069,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":14598,"spinlock_wait_cycles":14468352,"thread_start_us":357591,"threads_started":1,"wal-append.queue_time_us":559429}
W20260709 12:24:41.539481 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 122198us
I20260709 12:24:41.412068 16472 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 18.
I20260709 12:24:41.486953 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:41.420872 16442 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:41.534508 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 111635us
W20260709 12:24:41.467643 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 65597.9us
I20260709 12:24:41.420300 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:41.516377 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 100157us
I20260709 12:24:41.516767 22616 raft_consensus.cc:740] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:24:41.520128 22783 log.cc:927] Time spent T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.288s	user 0.001s	sys 0.000s
I20260709 12:24:41.520887 22658 raft_consensus.cc:697] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:41.380148 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:41.522251 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:41.551576 16470 raft_consensus.cc:3060] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:41.551690 16475 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:24:41.529191 16549 raft_consensus.cc:1275] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:24:41.552881 16474 raft_consensus.cc:1275] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:24:41.470170 16397 raft_consensus.cc:1275] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:24:41.541198 16515 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:41.549893 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.546514 16514 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:41.554062 16441 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:41.460695 22377 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:41.553638 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 132901us
I20260709 12:24:41.495229 22553 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:41.470682 22760 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.557952 17142 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:41.558988 22684 raft_consensus.cc:2764] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:24:41.473352 22402 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:41.559093 22732 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:24:41.471421 22625 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:41.561662 16441 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:41.563917 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:41.564410 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:41.527700 22354 raft_consensus.cc:740] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:41.532653 22754 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:41.583022 17271 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 27.
I20260709 12:24:41.591305 16470 raft_consensus.cc:2468] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 18.
I20260709 12:24:41.592836 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:41.593161 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 47564.7us
I20260709 12:24:41.584565 16441 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:41.602137 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:41.603080 17263 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:24:41.564986 22607 raft_consensus.cc:2804] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:24:41.608861 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 67761.3us
I20260709 12:24:41.563107 22678 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:24:41.581776 22760 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:41.620078 16549 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:24:41.620761 16549 raft_consensus.cc:606] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:41.621093 16549 raft_consensus.cc:614] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:41.621798 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:41.622123 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:41.622912 16544 raft_consensus.cc:2393] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 15: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:41.582609 22717 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:41.622790 16549 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:41.625185 16473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34826 (request call id 2121) took 2133 ms. Trace:
I20260709 12:24:41.628121 16473 rpcz_store.cc:276] 0709 12:24:39.479928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.483274 (+  3346us) service_pool.cc:224] Handling call
0709 12:24:41.613579 (+2130305us) spinlock_profiling.cc:243] Waited 637 ms on lock 0x7b54000a32d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:41.613735 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1464674048}
W20260709 12:24:41.629680 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 45226.1us
I20260709 12:24:41.628628 16514 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:41.631028 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:41.630584 16441 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:41.640321 16515 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:41.565284 22377 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:41.580796 22553 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:41.674257 22760 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:41.672752 22801 raft_consensus.cc:2764] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 12:24:41.675448 22377 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:41.647296  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:24:41.676384 22801 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:41.618837 22624 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:41.657289 22354 consensus_queue.cc:260] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:41.660748 22522 raft_consensus.cc:2804] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:41.617501 22547 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:41.605396 22788 consensus_queue.cc:1059] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 12:24:41.679999 22741 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.685185 22788 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:41.685240 22741 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:41.685843 22788 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:41.686426 22788 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:41.687948 22741 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:41.608606 22628 consensus_queue.cc:237] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:41.689284 22789 log.cc:927] Time spent T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:24:41.689740 22747 consensus_queue.cc:1048] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:24:41.664615 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.659798 22658 consensus_queue.cc:237] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:41.647301 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:24:41.667068 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.672714 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:41.616536 22678 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:41.650105 16514 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:41.600845 22754 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:41.678720 16549 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:24:41.645046 22607 raft_consensus.cc:697] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 18 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:41.683208 22786 raft_consensus.cc:993] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1: : Instructing follower 52ab50b0237346c1a13ede947f6a63c2 to start an election
W20260709 12:24:41.692428 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 52292.7us
I20260709 12:24:41.695899 17263 raft_consensus.cc:3060] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:41.675451 22553 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:41.702549 17143 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:41.707877 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:41.708227 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 58220.7us
I20260709 12:24:41.692011 22522 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:41.707113 16515 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:41.702438 22552 consensus_queue.cc:1048] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 12:24:41.698153 22678 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:24:41.715430 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:41.718302 17264 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45230 (request call id 1511) took 2357 ms (client timeout 1573 ms). Trace:
I20260709 12:24:41.719066 22627 raft_consensus.cc:2764] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
W20260709 12:24:41.718770 17264 rpcz_store.cc:269] 0709 12:24:39.355044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.355143 (+    99us) service_pool.cc:224] Handling call
0709 12:24:41.712754 (+2357611us) spinlock_profiling.cc:243] Waited 727 ms on lock 0x7b540005e6d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:41.712898 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1672093056}
I20260709 12:24:41.721791 17269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 1517) took 1464 ms. Trace:
I20260709 12:24:41.722126 22627 raft_consensus.cc:1081] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 19 LEADER]: Signalling peer 52ab50b0237346c1a13ede947f6a63c2 to start an election
I20260709 12:24:41.722606 22388 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:41.723153 22388 raft_consensus.cc:697] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:41.691782 22605 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:41.725307 17271 raft_consensus.cc:1240] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:41.725442 22605 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:41.726761 22389 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:41.722169 17269 rpcz_store.cc:276] 0709 12:24:40.257611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.257751 (+   140us) service_pool.cc:224] Handling call
0709 12:24:40.261719 (+  3968us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:40.590922 (+329203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:41.721592 (+1130670us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b540063e554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:41.721740 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":752920960}
I20260709 12:24:41.730070 16515 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:41.730866 16403 raft_consensus.cc:1275] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:24:41.722333 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:41.733006 17263 raft_consensus.cc:2468] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 18.
I20260709 12:24:41.736776 22607 consensus_queue.cc:237] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:41.713835 22522 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:41.745302 22632 consensus_queue.cc:1048] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:41.745956 22653 consensus_queue.cc:1059] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:41.747282 22684 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:41.748075 22627 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:41.747856 22684 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:41.728256 16469 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:34826
I20260709 12:24:41.749924 16469 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:41.749770 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:41.750561 16469 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:41.729468 22802 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:41.732911 22778 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:41.739768 22786 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.724010 22388 consensus_queue.cc:237] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:41.786088 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 45965us
W20260709 12:24:41.800815 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.805425 22624 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:41.815186 16469 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:41.821266 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 86813.2us
W20260709 12:24:41.822094 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.823181 16515 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:41.784438 22786 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:41.834347 16469 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:41.779196 22802 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:41.855785 16469 raft_consensus.cc:1240] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 18. Current term is 19. Ops: []
W20260709 12:24:41.861500 22794 log.cc:927] Time spent T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.179s	user 0.003s	sys 0.000s
I20260709 12:24:41.867861 16551 raft_consensus.cc:1240] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 19 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:41.868722 22402 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:41.870148 17254 raft_consensus.cc:1275] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:24:41.870586 22392 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:41.875134 22754 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:41.877141 22802 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:41.881596 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:41.882398 16396 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 14.
I20260709 12:24:41.883559 16516 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:41.886307 22522 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:41.886837 22798 log.cc:927] Time spent T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:24:41.855239 22684 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:41.867973 22786 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:41.888689 22795 log.cc:927] Time spent T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.232s	user 0.000s	sys 0.002s
W20260709 12:24:41.888700 22800 log.cc:927] Time spent T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:24:41.889439 22678 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:41.894754 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:41.901221 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:41.902842 17261 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:24:41.903966 22788 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:41.951068 16469 raft_consensus.cc:2393] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 15: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:41.904323 22354 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:41.971393 21922 raft_consensus.cc:3055] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:41.971941 21922 raft_consensus.cc:740] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:41.949800 22754 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.972810 21922 consensus_queue.cc:260] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:41.975947 21922 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:41.968319 22553 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.944905 22605 consensus_queue.cc:1059] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
W20260709 12:24:42.001884 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.977864 22754 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:42.009229 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.968595 22807 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:42.021616 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:42.022601 22678 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 28 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:42.021447 22788 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:42.013962 22522 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:42.030035 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 180210us
W20260709 12:24:42.032992 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 402467us
I20260709 12:24:42.021615 22553 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:42.035779 16441 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:42.039062 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:42.043118 16441 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:42.037863 22553 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:42.049642 16441 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:42.051193 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:42.052031 17270 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:42.041286 22678 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:42.046419 22684 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:42.054740 22717 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:42.055079 22760 consensus_queue.cc:1048] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:24:42.088867 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:42.095312 22522 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.095947 22522 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:42.098430 22522 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:42.099069 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 49549.3us
I20260709 12:24:42.100976 16441 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:42.107457 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:42.108522 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:42.110807 16396 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:24:42.111869 16516 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:42.113332 16441 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:42.115176 16441 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:42.110080 22810 log.cc:927] Time spent T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:24:42.118211 16441 leader_election.cc:400] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 21. Current term is 22.
I20260709 12:24:42.118636 16441 leader_election.cc:403] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:24:42.118803 17270 raft_consensus.cc:2468] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 17.
I20260709 12:24:42.121450 22717 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 21. Current term is 22.
I20260709 12:24:42.116184 22784 raft_consensus.cc:2804] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:42.128360 22717 consensus_queue.cc:1048] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:42.118806 22804 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:42.116945 22765 raft_consensus.cc:2804] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:42.116097 22354 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:42.136564 22765 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.137252 22765 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:42.126925 22760 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:42.145051 17270 raft_consensus.cc:1240] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:42.146101 22784 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:42.147643 22784 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:42.150177 16515 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:42.150799 16472 raft_consensus.cc:1275] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:24:42.151327 16515 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:42.155023 16441 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:42.157685 16441 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:42.160099 22786 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:42.157364 22605 consensus_queue.cc:1048] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 12:24:42.163506 17261 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45228 (request call id 1621) took 1503 ms. Trace:
I20260709 12:24:42.159559 22547 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:42.160821 22766 raft_consensus.cc:2804] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:24:42.167207 22790 log.cc:927] Time spent T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.861s	user 0.004s	sys 0.000s
I20260709 12:24:42.167619 22766 raft_consensus.cc:697] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 17 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:42.164131 17261 rpcz_store.cc:276] 0709 12:24:40.659716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.663117 (+  3401us) service_pool.cc:224] Handling call
0709 12:24:42.163239 (+1500122us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400a74c54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:42.163419 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2820002688}
I20260709 12:24:42.175693 16441 leader_election.cc:400] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 17. Current term is 18.
I20260709 12:24:42.178310 16441 leader_election.cc:403] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:24:42.181922 16395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46250 (request call id 1845) took 1201 ms. Trace:
I20260709 12:24:42.159852 22354 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:42.182363 16395 rpcz_store.cc:276] 0709 12:24:40.980750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.982482 (+  1732us) service_pool.cc:224] Handling call
0709 12:24:42.181756 (+1199274us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b54004ed4d4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:42.181864 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":355359104}
I20260709 12:24:42.182356 22804 consensus_queue.cc:1048] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 12:24:42.187320 22717 raft_consensus.cc:2749] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 17. Current term is 18.
I20260709 12:24:42.187471 22766 consensus_queue.cc:237] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:42.188853 22684 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:42.189564 22684 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:42.186865 16441 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:42.201385 16517 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.208132 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:42.208740 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:42.209394 17255 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 17.
I20260709 12:24:42.209726 17261 raft_consensus.cc:2393] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 22: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:42.211432 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:42.212563 17261 raft_consensus.cc:2393] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 15: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:42.212610 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:42.215582 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:42.216682 16542 raft_consensus.cc:1240] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 19. Current term is 20. Ops: [18.11-19.12]
I20260709 12:24:42.208299 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:42.201720 22354 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:42.241918 17270 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 27.
I20260709 12:24:42.223404 16441 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:42.201507 22808 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.241783  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:42.254750 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:42.265451 16544 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:24:42.265204 17143 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:42.271037 21922 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:42.272078 17143 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
W20260709 12:24:42.283757 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:42.285282 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 151834us
I20260709 12:24:42.286490 21922 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:42.287071 21922 raft_consensus.cc:697] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:42.266945 22553 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.261801 22802 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:42.256639 22816 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:42.293984 21922 consensus_queue.cc:237] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:42.300206 22626 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:42.300356 22784 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:42.302884 22754 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:42.304510 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:42.305223 16544 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:24:42.305696 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:42.307799 17143 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:24:42.309466 22824 log.cc:927] Time spent T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:24:42.300206 17258 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45224 (request call id 1524) took 1036 ms. Trace:
I20260709 12:24:42.261587 22808 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:42.311223 17258 rpcz_store.cc:276] 0709 12:24:41.263959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.265037 (+  1078us) service_pool.cc:224] Handling call
0709 12:24:42.300010 (+1034973us) spinlock_profiling.cc:243] Waited 86.2 ms on lock 0x7b540005df54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:42.300143 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":198153600}
I20260709 12:24:42.284811 22804 consensus_queue.cc:1048] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 12:24:42.318686 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:42.319384 16397 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:42.313109 22784 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:42.326577 16473 raft_consensus.cc:1275] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:24:42.327868 16397 raft_consensus.cc:2468] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:24:42.328260 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:42.329063 16475 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:42.331024 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:42.338057 16516 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:42.315440 22619 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:42.293772 22553 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:42.347720 16473 raft_consensus.cc:1275] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:24:42.336911 22684 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:42.341814 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:42.337159 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 73880us
I20260709 12:24:42.360540 16366 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:42.346951 22814 raft_consensus.cc:2804] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:42.344952 22619 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.368615 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:42.368778 16476 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 1386) took 1749 ms. Trace:
I20260709 12:24:42.368636 16475 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
I20260709 12:24:42.369606 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:42.369220 16476 rpcz_store.cc:276] 0709 12:24:40.618987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.619114 (+   127us) service_pool.cc:224] Handling call
0709 12:24:40.619402 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:41.008858 (+389456us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.368564 (+1359706us) spinlock_profiling.cc:243] Waited 377 ms on lock 0x7b540010efd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:42.368726 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":867267328}
I20260709 12:24:42.371620 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:42.368099 22619 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:42.372433 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: true dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:42.373211 16545 raft_consensus.cc:3055] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:42.373394 22808 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:42.373539 16545 raft_consensus.cc:740] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:42.373970 22735 consensus_queue.cc:1059] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:42.374305 16545 consensus_queue.cc:260] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:42.374970 22735 raft_consensus.cc:3055] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:42.375288 22735 raft_consensus.cc:740] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:42.370402 16542 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
I20260709 12:24:42.375756 16545 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:42.376277 22735 consensus_queue.cc:260] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:42.376819 22524 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:42.394733 17255 raft_consensus.cc:1240] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:24:42.393079 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:42.385275 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: true dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:42.396081 17258 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:42.396888 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:42.397122 16546 raft_consensus.cc:1240] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 17 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:42.388180 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:42.351640 22754 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.392589 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:42.395648 17261 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:42.406642 17254 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 21.
I20260709 12:24:42.396937 17270 raft_consensus.cc:1275] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:24:42.385834 16473 raft_consensus.cc:1275] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:24:42.398344 16544 raft_consensus.cc:2393] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 18: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
W20260709 12:24:42.380764 22825 log.cc:927] Time spent T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
W20260709 12:24:42.410066 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 49640.4us
I20260709 12:24:42.408629 16547 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 22.
I20260709 12:24:42.410668 16549 raft_consensus.cc:1240] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:24:42.409147 16545 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 20.
I20260709 12:24:42.420001 16366 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:42.368271 22766 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.428689 17261 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 28.
I20260709 12:24:42.431775 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:42.434036 16395 raft_consensus.cc:2410] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:24:42.407688 22735 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:42.431875 16514 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:42.433806 16366 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:42.439484 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:42.440521 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:42.440457 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:42.442551 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:42.444700 17258 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 20.
I20260709 12:24:42.446365 16475 raft_consensus.cc:1275] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 12:24:42.493352 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:42.569201 16475 pending_rounds.cc:85] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 9
I20260709 12:24:42.569741 16475 pending_rounds.cc:107] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 12:24:42.419524 22831 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.464193 22616 consensus_queue.cc:260] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:42.579890 17261 raft_consensus.cc:1275] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:24:42.562178 22815 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.584709 16551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1849) took 1228 ms. Trace:
I20260709 12:24:42.585296 16551 rpcz_store.cc:276] 0709 12:24:41.356234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.357035 (+   801us) service_pool.cc:224] Handling call
0709 12:24:41.357334 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:41.868451 (+511117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.584392 (+715941us) spinlock_profiling.cc:243] Waited 510 ms on lock 0x7b5400729554. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:42.584619 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1173578496}
I20260709 12:24:42.450613 22354 consensus_queue.cc:1048] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:24:42.516067 22786 raft_consensus.cc:2804] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:42.454262 22524 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:24:42.513577 22828 log.cc:927] Time spent T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 12:24:42.475468 22553 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:42.465647 22829 log.cc:927] Time spent T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 12:24:42.418519 22628 consensus_queue.cc:1048] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 12:24:42.562367 17263 raft_consensus.cc:1275] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:24:42.437002 22836 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:42.560482 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:42.618619 17270 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 19.
I20260709 12:24:42.594259 22747 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.432715 22684 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.517398 22626 consensus_queue.cc:1048] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:42.411080 22754 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:42.410441 22814 raft_consensus.cc:697] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:42.561877 22697 consensus_queue.cc:1048] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 12:24:42.445399 22766 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:42.613709 22786 raft_consensus.cc:697] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 13 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:42.406698 22607 consensus_queue.cc:1048] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 12:24:42.621176 22553 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:42.619426 22746 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.596050 22815 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:42.605418 22765 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:42.633142 22616 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:42.632843 22684 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:42.789497 22765 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:42.643828 22746 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:42.648777 22786 consensus_queue.cc:237] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:42.629184 22747 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:42.562356 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.662284 22754 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:42.668963 22766 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:42.444923 16441 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:42.454713 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:42.813979 22388 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.815425 22388 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:42.588693 16475 raft_consensus.cc:2889] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:42.569820 16476 raft_consensus.cc:2410] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
W20260709 12:24:42.740965 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 146114us
I20260709 12:24:42.565261 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:42.832229 22833 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:42.566619 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:42.834992 16545 raft_consensus.cc:2393] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 13: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:42.835322 22697 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.558554 16549 raft_consensus.cc:1275] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:24:42.836493 22697 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:42.501367 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.749804 16542 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 18.
I20260709 12:24:42.761982 22392 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:42.762703 17266 raft_consensus.cc:1240] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:24:42.611155 22524 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.853785 22524 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:24:42.664893 22797 log.cc:927] Time spent T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 1.018s	user 0.004s	sys 0.003s
I20260709 12:24:42.449358 16366 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:42.857620 22684 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:42.485940 16401 raft_consensus.cc:1240] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 20 LEADER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:42.650463 22814 consensus_queue.cc:237] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:42.665351 22827 log.cc:927] Time spent T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.404s	user 0.003s	sys 0.001s
I20260709 12:24:42.577088 22831 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:42.666983 22815 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:42.554772 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:42.868803 22388 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:42.805208 16473 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:42.869907 22831 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:42.642580 22619 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:42.873649 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:42.874439 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 425055us
I20260709 12:24:42.809489 22836 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.816773 22552 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.875662 22552 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:42.875660 22836 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:42.878484 22697 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:42.879204 22552 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:42.796316 22747 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:42.798859 22839 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.880649 22839 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:42.468734 16547 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37226 (request call id 1853) took 1067 ms. Trace:
I20260709 12:24:42.882339 16476 raft_consensus.cc:3055] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:42.882886 16476 raft_consensus.cc:740] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:42.882670 16547 rpcz_store.cc:276] 0709 12:24:41.401787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.419573 (+ 17786us) service_pool.cc:224] Handling call
0709 12:24:42.468495 (+1048922us) spinlock_profiling.cc:243] Waited 874 ms on lock 0x7b5400201054. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:42.468658 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2011238656}
I20260709 12:24:42.848286 22392 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:42.885710 22839 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:42.886390 22839 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:42.886453 22392 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:42.851022 16399 raft_consensus.cc:1275] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:24:42.849977 16223 catalog_manager.cc:5697] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:42.887717 22839 consensus_queue.cc:1059] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:42.888392 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:42.889333 16469 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 19.
W20260709 12:24:42.890142 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.889537 16223 catalog_manager.cc:5697] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 13 to 19. New cstate: current_term: 19 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
W20260709 12:24:42.851904 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 406867us
I20260709 12:24:42.892258 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:42.893210 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:42.894229 16469 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:42.894392 16441 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:42.852533 22377 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.266577  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:42.896324 22760 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:42.896816 22760 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:42.897125 16471 raft_consensus.cc:1275] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:24:42.777318 22553 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:42.442739 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:42.897930 22697 consensus_queue.cc:1048] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:42.899894 16441 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:42.900528 16223 catalog_manager.cc:5697] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:24:42.850399 22746 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:42.897222 22760 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:42.904079 16223 catalog_manager.cc:5697] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 9 to 11, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 11 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:42.880254 22836 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:42.907210 22836 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:42.874598 22619 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:42.893224 16468 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 19.
I20260709 12:24:42.895771 22377 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:42.874936 16475 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 17.
I20260709 12:24:42.900166 22553 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:42.914336 22553 raft_consensus.cc:3055] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:42.914778 22553 raft_consensus.cc:740] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:42.923144 22377 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:42.916118 22553 consensus_queue.cc:260] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:42.925318 16476 consensus_queue.cc:260] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:42.926694 16476 raft_consensus.cc:3060] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:42.927104 22524 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:42.908284 22836 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:42.907137 16223 catalog_manager.cc:5697] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 6 to 16. New cstate: current_term: 16 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:24:42.933485 16473 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:24:42.947172 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:42.950531 16402 raft_consensus.cc:2393] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 18: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:42.933974 16474 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 13.
I20260709 12:24:42.943344 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:42.943508 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 233940us
I20260709 12:24:42.947302 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:42.990814 17263 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:24:42.994668 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:42.983714 22833 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:24:42.989506 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:42.995658 16441 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:24:42.997611 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 116530us
I20260709 12:24:42.998397 16400 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 18.
W20260709 12:24:42.931340 22684 consensus_queue.cc:1175] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ef57beab0e1d40d585983873c55107e8" responder_term: 17 status { last_received { term: 15 index: 10 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 16. Current term is 17. Ops: [16.11-16.11]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:24:43.007321 22697 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:43.013494 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:43.014652 16398 raft_consensus.cc:2393] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 28: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:43.008142 22697 raft_consensus.cc:697] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:43.023301 16402 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 14.
I20260709 12:24:43.023535 17270 raft_consensus.cc:1275] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:24:43.007330 16223 catalog_manager.cc:5697] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 16 to 20, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 20 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:24:43.024094 17261 raft_consensus.cc:1275] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 12:24:43.025578 16219 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53626 (request call id 129) took 54066 ms (client timeout 15000 ms). Trace:
I20260709 12:24:43.025974 22524 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 28 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:43.026302 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:43.027489 17258 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 21.
I20260709 12:24:43.029695 22377 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:43.030124 22836 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:43.010959 22833 raft_consensus.cc:697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 28 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:43.030551 22377 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:43.032244 22833 consensus_queue.cc:237] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:43.032900 22392 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:43.033174 22814 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.033771 22392 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:43.033869 22814 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:43.036844 16396 raft_consensus.cc:1275] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:24:43.010632 16514 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:43.042358 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:43.041993 16223 catalog_manager.cc:5697] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 12 to 16, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 16 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:24:43.043582 16403 raft_consensus.cc:2393] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 21: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:43.045105 22524 consensus_queue.cc:1048] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:43.046936 22836 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.047672 22836 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:43.051782 22836 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:43.058830 16547 raft_consensus.cc:1240] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:43.060611 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:43.063055 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:43.053053 16394 raft_consensus.cc:1240] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 12:24:43.043979 16223 catalog_manager.cc:5697] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 9 to 15, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 15 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:24:43.074090 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:43.075323 16542 raft_consensus.cc:2393] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 16: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:43.078289 17144 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:43.078418 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:43.081362 22818 log.cc:927] Time spent T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:24:43.041363 22392 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:43.083029 16223 catalog_manager.cc:5697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 19 to 23. New cstate: current_term: 23 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:43.084829 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:43.085590 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:43.086112 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:43.087168 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:43.087708 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:43.089257 16551 raft_consensus.cc:1240] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:43.102208 17263 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:24:43.102432 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:43.102587 22814 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:43.103173 16550 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:43.103415 16396 raft_consensus.cc:2393] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 16: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:43.104275 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:43.105114 17261 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 20.
I20260709 12:24:43.106508 22697 consensus_queue.cc:237] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:43.108632 16546 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
W20260709 12:24:43.110100 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 118704us
I20260709 12:24:43.110972 16549 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:24:43.112016 16543 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 19.
I20260709 12:24:43.112746 22392 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:43.113188 22392 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.097496 16542 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:43.113579 22392 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:43.116757 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.117837 16223 catalog_manager.cc:5697] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 8 to 11, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 11 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:24:43.120355 16223 catalog_manager.cc:5697] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 11 to 17, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 17 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:24:43.123315 17142 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:43.125295 16476 raft_consensus.cc:1275] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:24:43.092432 16441 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
W20260709 12:24:43.136688 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 126869us
I20260709 12:24:43.141016 17142 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:43.142795 22772 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:43.099042 22760 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:43.145287 22833 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:43.152279 16550 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:43.157088 22531 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:43.157302 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:43.157694 22531 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.158126 22531 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:43.159060 22553 raft_consensus.cc:3060] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:43.026245 16219 rpcz_store.cc:269] 0709 12:23:49.024396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:49.056712 (+ 32316us) service_pool.cc:224] Handling call
0709 12:24:43.025247 (+53968535us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:35.037510 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:35.038309 (+   799us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:35.038377 (+    68us) write_op.cc:435] Acquired schema lock
0709 12:24:35.038415 (+    38us) tablet.cc:662] Decoding operations
0709 12:24:35.080940 (+ 42525us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:35.081046 (+   106us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:35.081079 (+    33us) tablet.cc:685] Acquiring locks for 15 operations
0709 12:24:35.082433 (+  1354us) tablet.cc:701] Row locks acquired
0709 12:24:35.082486 (+    53us) write_op.cc:260] PREPARE: finished
0709 12:24:35.082662 (+   176us) write_op.cc:270] Start()
0709 12:24:35.082816 (+   154us) write_op.cc:276] Timestamp: P: 1783599875082592 usec, L: 0
0709 12:24:35.082838 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:24:35.083638 (+   800us) log.cc:844] Serialized 5830 byte log entry
0709 12:24:35.466055 (+382417us) op_driver.cc:464] REPLICATION: finished
0709 12:24:35.932606 (+466551us) write_op.cc:301] APPLY: starting
0709 12:24:35.932811 (+   205us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:36.691627 (+758816us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:36.705885 (+ 14258us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:37.717371 (+1011486us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:37.731251 (+ 13880us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=30,key_file_lookups=30,delta_file_lookups=0,mrs_lookups=0
0709 12:24:37.731337 (+    86us) write_op.cc:312] APPLY: finished
0709 12:24:37.736268 (+  4931us) log.cc:844] Serialized 138 byte log entry
0709 12:24:38.160197 (+423929us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:38.180391 (+ 20194us) write_op.cc:454] Released schema lock
0709 12:24:38.195199 (+ 14808us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:38.212656 (+ 17457us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:38.212707 (+    51us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":479694,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":324166,"apply.run_cpu_time_us":54345,"apply.run_wall_time_us":2313769,"cfile_cache_hit":62,"cfile_cache_hit_bytes":66887,"num_ops":15,"prepare.queue_time_us":134697,"prepare.run_cpu_time_us":11863,"prepare.run_wall_time_us":183362,"raft.queue_time_us":255120,"raft.run_cpu_time_us":63315,"raft.run_wall_time_us":234783,"replication_time_us":382989,"thread_start_us":1009125,"threads_started":5,"wal-append.queue_time_us":561334,"wal-append.run_cpu_time_us":18875,"wal-append.run_wall_time_us":2297293}]]}
I20260709 12:24:43.160594 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:43.164469 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:43.164530 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:43.166771 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:43.167066 22786 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.167697 16472 raft_consensus.cc:2393] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 21: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:43.167614 22786 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:43.170939 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:43.171679 16469 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 22.
I20260709 12:24:43.172924 22786 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:43.174067 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 101814us
W20260709 12:24:43.172155 22735 consensus_meta.cc:287] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.726s	user 0.015s	sys 0.006s
I20260709 12:24:43.158143 16402 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
I20260709 12:24:43.149202 22833 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.180819 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:43.182518 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:43.182793 22377 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:43.183699 22760 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:43.201202 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 140947us
I20260709 12:24:43.186828 22766 consensus_queue.cc:1048] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:43.187932 17143 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:43.166793 16223 catalog_manager.cc:5697] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 11 to 15, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 15 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:24:43.193849 17142 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:43.195667 16398 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:24:43.196393 16397 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:24:43.201454 22848 consensus_queue.cc:1048] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:24:43.207268 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 130857us
I20260709 12:24:43.196635 22833 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:24:43.212296 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 58848.6us
I20260709 12:24:43.215248 17143 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:43.209197 16223 catalog_manager.cc:5697] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 17 to 18, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 18 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:24:43.211773 22735 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:43.222110 22845 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:43.225713 22845 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.226099 22845 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:43.226543 16367 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:43.228819 16367 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:43.229946 22842 consensus_queue.cc:1059] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:24:43.230829 22842 consensus_queue.cc:1243] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer 610f4feda11942739236f2b121a4b9c1 log is divergent from this leader: its last log entry 15.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:24:43.231343 22842 raft_consensus.cc:3055] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:43.231658 22842 raft_consensus.cc:740] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:43.214113 22854 consensus_queue.cc:1048] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 12:24:43.232995 22388 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:43.233438 22388 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.233769 22388 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:43.246232 17143 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:43.253748 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:43.254058 16367 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:43.254788 17261 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 20.
I20260709 12:24:43.195744 22746 consensus_queue.cc:1048] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 12:24:43.256234 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:43.224203 22735 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.257680 22836 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:43.258137 22735 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:43.260236 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.268585 16441 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:43.270639 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 46548.4us
I20260709 12:24:43.264081 16223 catalog_manager.cc:5697] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 12 to 14, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 14 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
W20260709 12:24:43.271986 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 47106.1us
I20260709 12:24:43.249228 22616 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:43.279989 22843 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:43.281028 22843 raft_consensus.cc:697] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 20 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
W20260709 12:24:43.281503 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 66882.4us
I20260709 12:24:43.282682 22843 consensus_queue.cc:237] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:43.283929 22760 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:43.283631 16223 catalog_manager.cc:5697] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:43.284807 22760 consensus_queue.cc:1059] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:43.285163 16545 raft_consensus.cc:1275] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:24:43.286923 16550 raft_consensus.cc:1275] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:24:43.285830 22684 raft_consensus.cc:3055] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:43.285650 16223 catalog_manager.cc:5697] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 12 to 17, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 17 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:24:43.287636 22684 raft_consensus.cc:740] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:43.288503 22616 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.289023 22616 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:43.288815 22684 consensus_queue.cc:260] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:43.289463 16514 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:43.290021 22684 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:43.292052 22377 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:43.292542 22377 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:43.292891 22377 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:43.293047 22392 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:43.295403 22392 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:43.300112 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.276834 22524 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:43.310210 22524 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:43.318677 22524 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:43.320715 16398 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:43.312788 16223 catalog_manager.cc:5697] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 11 to 14, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 14 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:24:43.264199 22772 consensus_queue.cc:1048] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 12:24:43.308946 22552 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:24:43.334739 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 80806.3us
W20260709 12:24:43.382074 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 74751.5us
I20260709 12:24:43.376632 22552 raft_consensus.cc:697] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 16 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:43.386178 22552 consensus_queue.cc:237] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:43.388022 16543 raft_consensus.cc:1275] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:24:43.329465 22766 consensus_queue.cc:1048] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 12:24:43.387029 22626 consensus_queue.cc:1048] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 12:24:43.381191 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:43.405751 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 117185us
I20260709 12:24:43.383803 16223 catalog_manager.cc:5697] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 11 to 17. New cstate: current_term: 17 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:24:43.415166 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:24:43.438683 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 216291us
I20260709 12:24:43.454605 16473 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:24:43.383219 16515 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:24:43.412935 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.306501 22842 consensus_queue.cc:260] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:43.425973 22388 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:43.491638 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 158034us
I20260709 12:24:43.492548 16515 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:43.440299 22619 consensus_queue.cc:1048] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 12:24:43.496616 16223 catalog_manager.cc:5697] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 16 to 19, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 19 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:24:43.516599 16441 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:43.523468 16515 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:43.529206 16472 raft_consensus.cc:1275] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:24:43.530953 16223 catalog_manager.cc:5697] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:24:43.538290 22788 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:43.538830 22788 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.539207 22788 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:43.539126 16223 catalog_manager.cc:5697] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 10 to 15. New cstate: current_term: 15 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:24:43.542171 22392 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:43.544364 22392 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.544811 22392 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:43.544735 22354 consensus_queue.cc:1059] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:43.535135 16515 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:43.546267 16515 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:43.520838 22766 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:24:43.545706 22859 log.cc:927] Time spent T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.248s	user 0.002s	sys 0.000s
I20260709 12:24:43.569168 16515 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:43.547099 22607 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:43.545076 22732 consensus_queue.cc:1048] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.128s
I20260709 12:24:43.585417 16366 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:43.599501 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 82179.6us
W20260709 12:24:43.598694 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 125595us
I20260709 12:24:43.588080 16473 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 19.
I20260709 12:24:43.573400 22786 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:43.565248 22766 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:43.545209 22841 consensus_queue.cc:1059] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.124s
I20260709 12:24:43.587291 22607 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.590365 22765 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.589632 22626 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:24:43.626746 22866 log.cc:927] Time spent T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:24:43.630904 22841 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.613401 22786 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:43.637401 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 87936.4us
I20260709 12:24:43.605324 22760 raft_consensus.cc:2764] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:24:43.620647 22843 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:43.613750 22766 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:43.665154 22388 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:43.675637 22553 consensus_meta.cc:287] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.516s	user 0.011s	sys 0.003s
I20260709 12:24:43.672140 22841 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:43.680523 22854 consensus_queue.cc:1048] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:24:43.641223 22607 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:43.671975 22786 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:43.698967 22833 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:43.685544 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 124678us
I20260709 12:24:43.695572 22616 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:43.688665 16516 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:43.675778 22856 log.cc:927] Time spent T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.166s	user 0.000s	sys 0.004s
I20260709 12:24:43.605696 22377 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:43.729744 22845 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:43.735070 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 46665.7us
I20260709 12:24:43.736603 22377 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 23 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:43.737094 22647 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:43.737653 22855 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:43.700757 22735 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:43.738200 22553 consensus_queue.cc:1048] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:24:43.740291 22735 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:43.742210 22841 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:43.702088 22765 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:43.746493 22765 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:43.747822 22524 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:43.740057 22553 consensus_queue.cc:1048] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.012s
W20260709 12:24:43.752578 22531 consensus_meta.cc:287] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.594s	user 0.014s	sys 0.000s
I20260709 12:24:43.753834 17266 raft_consensus.cc:2393] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 20: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:43.753355 22531 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:43.756182 22616 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:43.671105 22848 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:43.786907 16475 raft_consensus.cc:1240] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:43.705626 22388 consensus_queue.cc:1048] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 12:24:43.756297 22531 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:43.758358 22845 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:43.807997 16398 raft_consensus.cc:1275] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 12. (term mismatch)
I20260709 12:24:43.808713 16398 pending_rounds.cc:85] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 11
I20260709 12:24:43.738303 22855 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:43.827795 16398 pending_rounds.cc:107] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 12:24:43.838737 16398 raft_consensus.cc:2889] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:43.791685 22848 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:43.855832 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:43.855949 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:43.856735 16395 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:24:43.833825 22843 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:43.858656 22843 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:43.861330 22843 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:43.866076 22842 raft_consensus.cc:3060] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:43.870263 22778 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.870376 22848 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:43.835876 22845 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:43.844820 22836 consensus_meta.cc:287] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.586s	user 0.013s	sys 0.001s
I20260709 12:24:43.905316 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:43.871097 22778 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:43.902046 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:24:43.912372 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 175871us
W20260709 12:24:43.916558 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 245711us
I20260709 12:24:43.915891 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:43.903887 22760 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.924070 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:43.924750 16396 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:43.907574 22836 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:43.930510 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:43.933252 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:43.933107 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 296040us
I20260709 12:24:43.919639 22354 raft_consensus.cc:3055] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:43.928128 16514 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:43.934278 16397 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 18.
I20260709 12:24:43.903548 22845 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:43.942471 22815 consensus_queue.cc:1048] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
W20260709 12:24:43.944100 22873 log.cc:927] Time spent T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:24:43.944319 22845 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:43.932245 16367 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:43.946542 16367 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:43.948010 16367 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:43.968680 22848 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:43.974086 22848 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.974589 22848 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:43.975680 22765 raft_consensus.cc:2804] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:43.976576 22765 raft_consensus.cc:697] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 19 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:43.976866 22845 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.977496 22845 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:43.978380 16367 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:43.978037 22765 consensus_queue.cc:237] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:43.979601 22845 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:43.972564 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 44497.6us
I20260709 12:24:43.986948 22843 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:43.993542 22843 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:43.994459 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:43.995214 16398 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 21.
W20260709 12:24:44.000754 22867 log.cc:927] Time spent T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.319s	user 0.005s	sys 0.000s
I20260709 12:24:43.939994 22354 raft_consensus.cc:740] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:44.005477 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:44.004611 17144 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:44.006366 16397 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 19.
I20260709 12:24:44.015295 16395 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:43.933085 22760 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:43.971835 22836 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:44.014315 17144 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:44.029345 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:44.030159 17144 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:43.987481 22833 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:44.036305 22747 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:44.020876 22772 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:44.020747 22815 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.033385 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:44.055997 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:44.036113 22746 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:44.035786 22887 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:44.055500 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:44.048915 22772 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.065364 22778 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:44.062904 22747 raft_consensus.cc:697] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:44.099280 22747 consensus_queue.cc:237] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:44.067890 22392 consensus_meta.cc:287] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.522s	user 0.008s	sys 0.003s
I20260709 12:24:44.102025 22392 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:44.061897 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:44.070667 16546 raft_consensus.cc:1275] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 12:24:44.072599 22875 log.cc:927] Time spent T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.078s	user 0.001s	sys 0.001s
I20260709 12:24:44.073254 22855 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:44.073698 22815 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:44.074589 16442 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.2:41367 timed out after 2.053s (SENT) [suppressed 18 similar messages]
I20260709 12:24:44.098142 22772 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:44.104192 16545 raft_consensus.cc:2393] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 17: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:44.104633 16551 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:44.115717 16396 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 19.
I20260709 12:24:44.061899 22746 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.061987 22743 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.104259 22392 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:44.109632 22766 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:44.134022 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:44.126084 22892 log.cc:927] Time spent T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:24:44.145323 16395 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:24:44.119628 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 141277us
I20260709 12:24:44.145733 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:24:44.171564 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.209139 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:44.209782 16475 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:44.143592 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:44.211163 16398 raft_consensus.cc:3060] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:44.144678 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:44.217437 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:44.218719 22890 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.218875 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:44.219352 22890 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:44.220072 16471 raft_consensus.cc:2410] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:24:44.221308 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:44.221598 22890 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:44.222095 16472 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:24:44.143757 16400 raft_consensus.cc:1275] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:24:44.164222 16516 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:44.126084 22879 log.cc:927] Time spent T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 12:24:44.223845 22392 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:24:44.223598 22786 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:44.204448 16403 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 20.
W20260709 12:24:44.223663 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 59545.3us
I20260709 12:24:44.231812 16475 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 23.
I20260709 12:24:44.232811 22743 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:44.235590 22766 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:44.236557 22743 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:44.169615 22897 consensus_queue.cc:1048] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 12:24:44.144436 16551 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 20.
W20260709 12:24:44.168159 22788 consensus_meta.cc:287] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.628s	user 0.008s	sys 0.006s
I20260709 12:24:44.238701 17143 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:24:44.239557 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 217879us
I20260709 12:24:44.240722 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:44.241436 16396 raft_consensus.cc:2410] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 17 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:24:44.146822 22886 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.201072 22746 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:44.127883 22760 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:44.245870 16398 raft_consensus.cc:2468] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 19.
I20260709 12:24:44.247331 16395 raft_consensus.cc:1275] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:24:44.248744 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:44.259579 16551 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 20.
I20260709 12:24:44.168777 22815 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:44.257633 16367 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:44.146560 22531 consensus_queue.cc:1048] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.097s
I20260709 12:24:44.270227 16514 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:44.272979 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:44.173658 22848 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:44.274386 16475 raft_consensus.cc:2393] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 21: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:44.244853 22735 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:44.120502 22607 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:44.280015 17263 raft_consensus.cc:1275] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:24:44.255367 22886 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:44.282950 22895 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:44.283737 22895 raft_consensus.cc:697] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:44.284931 22895 consensus_queue.cc:237] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:44.286023 16367 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:44.287452 16367 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:44.288560 22836 consensus_queue.cc:1048] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:24:44.289690 22843 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:44.290335 22836 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:44.126190 22887 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:44.294566 22886 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:44.299786 22887 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:44.259974 22874 consensus_queue.cc:1059] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 12:24:44.300547 22607 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:44.301048 22874 raft_consensus.cc:3055] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:44.301688 22874 raft_consensus.cc:740] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:44.277779 22890 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.303231 22890 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:44.302992 22874 consensus_queue.cc:260] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:44.305961 22890 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:44.243492 22788 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:44.306900 22848 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:44.307343 16549 raft_consensus.cc:2393] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 17: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:44.308432 22788 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:44.128089 22354 consensus_queue.cc:260] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:44.309777 22735 raft_consensus.cc:697] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 20 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:44.311074 22735 consensus_queue.cc:237] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:44.313527 22524 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:44.313874 16441 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.786s	user 0.011s	sys 0.009s
W20260709 12:24:44.314282 16441 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.012s	sys 0.009s
W20260709 12:24:44.314608 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 786536us
I20260709 12:24:44.315572 16441 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:44.316720 22876 raft_consensus.cc:2804] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:44.317142 22876 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.317507 22876 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:24:44.325858 22872 log.cc:927] Time spent T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.652s	user 0.001s	sys 0.001s
I20260709 12:24:44.351135 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:44.351459 16476 raft_consensus.cc:1275] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 12:24:44.261838 22772 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:44.351797 16476 pending_rounds.cc:85] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 10
I20260709 12:24:44.354769 16476 pending_rounds.cc:107] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 12:24:44.355043 16476 raft_consensus.cc:2889] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:44.358870 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:44.360922 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:44.362315 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 168250us
I20260709 12:24:44.363209 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:44.363808 16475 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:24:44.365417 17258 raft_consensus.cc:2368] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 21. Current term is 22.
I20260709 12:24:44.365715 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:44.366989 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:44.367230 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:44.367929 16396 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:44.368747 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:44.368842 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:44.369524 16398 raft_consensus.cc:2393] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 17: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:44.370151 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:44.370993 16471 raft_consensus.cc:2393] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 19: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:44.357617 22874 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:44.374970 16469 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:44.374854 17261 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 28 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:24:44.380276 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:44.384572 17266 raft_consensus.cc:1275] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:24:44.384863 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:44.385457 16470 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:44.390626 22786 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:44.392796 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 105445us
I20260709 12:24:44.395187 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:44.396122 16472 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:44.398166 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:44.399703 22772 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:44.399717 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:44.410454 16396 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 21.
I20260709 12:24:44.411212 16475 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 19.
I20260709 12:24:44.413646 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:44.416085 22878 log.cc:927] Time spent T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.720s	user 0.003s	sys 0.000s
I20260709 12:24:44.391791 22836 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:44.424350 22895 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:44.436935 16367 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:44.441256 22746 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:44.440297 22876 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:44.477905 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 207780us
I20260709 12:24:44.447970 22895 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:44.488754 16514 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:44.490355 22392 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:44.490734 22895 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:44.490908 22392 raft_consensus.cc:697] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 23 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:44.448195 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:44.493621 16397 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:44.496650 22392 consensus_queue.cc:237] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:44.498003 16549 raft_consensus.cc:1275] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:24:44.500558 17270 raft_consensus.cc:2393] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 20: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:44.457043 16471 raft_consensus.cc:1240] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 12:24:44.460152 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:44.461349 17255 raft_consensus.cc:2410] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:24:44.466485 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:44.485160 16468 raft_consensus.cc:1240] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
W20260709 12:24:44.474447 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 205281us
I20260709 12:24:44.503975 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:44.450556 16398 raft_consensus.cc:1275] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:24:44.506108 16550 raft_consensus.cc:2393] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 21: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:44.443944 16470 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 20.
I20260709 12:24:44.507557 16476 raft_consensus.cc:1240] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:24:44.476161 17143 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:44.473672 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:44.460702 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:44.442499 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:44.464236 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:44.503944 17266 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 20.
I20260709 12:24:44.505139 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:44.511335 16475 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 19.
W20260709 12:24:44.511561 17144 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.505100 16472 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 20.
I20260709 12:24:44.512225 17144 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:44.515417 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:44.520275 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:44.521760 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:24:44.472586 22909 log.cc:927] Time spent T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.080s	user 0.004s	sys 0.000s
I20260709 12:24:44.524685 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:24:44.533033 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 56985.2us
I20260709 12:24:44.531639 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:44.536470 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:44.537619 16475 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 21.
I20260709 12:24:44.479449 22647 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:44.534258 16545 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 20.
I20260709 12:24:44.541375 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:44.539492 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:44.463382 22836 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:44.463160 22354 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:44.540900 22647 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.541771 22732 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.547638 22647 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:44.532143 22772 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:44.547886 22732 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:44.550473 22732 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:44.551841 16441 leader_election.cc:400] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 21. Current term is 22.
I20260709 12:24:44.552253 16441 leader_election.cc:403] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:24:44.552665 22893 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:24:44.554214 22893 consensus_queue.cc:1048] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:44.555800 22893 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:24:44.556123 16367 connection.cc:256] RPC call timeout handler was delayed by -1.8874504739997064s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:24:44.554242 22883 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:44.486842 22876 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:44.557889 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:44.567569 16477 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:44.558216 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:44.563150 22553 consensus_queue.cc:1059] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:24:44.563875 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 29 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:44.541999 16471 raft_consensus.cc:2376] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16. Re-sending same reply.
I20260709 12:24:44.480257 22746 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:44.563876 22895 consensus_queue.cc:1059] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 12:24:44.570648 22843 consensus_queue.cc:1059] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:24:44.573688 17143 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:44.584545 16397 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 22.
I20260709 12:24:44.587815 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:44.600731 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:44.564886 22836 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:44.606089 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:44.607787 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:44.610309 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:44.567480 22883 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.653802 17261 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 21.
W20260709 12:24:44.654124 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 69161.3us
I20260709 12:24:44.657459 16550 raft_consensus.cc:1275] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:24:44.654587 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:44.576002 22772 raft_consensus.cc:697] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 19 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:44.659484 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:44.599009 22886 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:44.662307 16549 raft_consensus.cc:2368] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17.
W20260709 12:24:44.662328 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 76231.4us
W20260709 12:24:44.664407 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 58307.1us
I20260709 12:24:44.665541 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:44.603365 22765 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:44.579007 22897 consensus_queue.cc:1048] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 12:24:44.666339 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:44.666874 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:44.666260 16395 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 19.
I20260709 12:24:44.673960 16551 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:24:44.673628 16397 raft_consensus.cc:2368] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 21. Current term is 22.
I20260709 12:24:44.664268 16442 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:44.675104 17266 raft_consensus.cc:2393] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 17: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:44.652906 22883 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:44.678015 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:44.652977 22836 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:44.681788 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:44.682811 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:44.683518 16398 raft_consensus.cc:2368] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 21. Current term is 22.
I20260709 12:24:44.677002 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:44.685359 17270 raft_consensus.cc:3060] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:44.686417 16402 raft_consensus.cc:2393] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 28: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:44.663290 22697 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.691484 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:44.691157 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:44.694926 16396 raft_consensus.cc:2368] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 18. Current term is 19.
I20260709 12:24:44.695055 16551 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:44.686947 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:44.659598 22766 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.686931 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:44.697108 16401 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 28 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:24:44.691905 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:24:44.713102 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 47976.9us
I20260709 12:24:44.689045 22843 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.687317 22760 consensus_queue.cc:1048] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 12:24:44.723420 22760 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:44.715186 17263 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:44.667022 22735 consensus_queue.cc:1048] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:24:44.726706 17143 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:44.726053 22843 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:44.729087 22912 raft_consensus.cc:3055] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:44.724964 22766 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:44.729580 22912 raft_consensus.cc:740] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:44.730571 22912 consensus_queue.cc:260] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:44.732878 22766 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:44.737046 16477 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 20.
I20260709 12:24:44.737112 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:44.732930 17144 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:44.713231 17142 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:44.699159 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:44.723842 22760 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:44.738624 16551 raft_consensus.cc:2468] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
W20260709 12:24:44.745808 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.688234 22765 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:44.732156 22912 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:44.697690 22697 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:44.754153 16551 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:44.759660 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 29 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:44.769874 16396 raft_consensus.cc:2410] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 29 because replica has last-logged OpId of term: 26 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 15.
I20260709 12:24:44.761196 22732 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:44.762156 22553 raft_consensus.cc:3055] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:44.762427 22874 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:44.766270 22743 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:44.767064 22848 raft_consensus.cc:3055] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:44.767460 16367 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:44.789033 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:24:44.770706 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 57565.7us
I20260709 12:24:44.789654 16399 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:44.773526 17270 raft_consensus.cc:2468] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 21.
I20260709 12:24:44.776558 22743 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.775008 22553 raft_consensus.cc:740] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:44.775331 22874 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.750195 22760 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:44.751870 22772 consensus_queue.cc:237] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:44.697013 22882 consensus_queue.cc:1048] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.143s
I20260709 12:24:44.797557 16549 raft_consensus.cc:1240] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:24:44.798386 22765 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:44.798763 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:44.798434 22647 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:44.799716 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:44.800776 22788 consensus_queue.cc:1048] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:44.827031 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:44.726150 22886 consensus_queue.cc:1048] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:24:44.827445 22647 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:44.788854 22848 raft_consensus.cc:740] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:44.829061 22848 consensus_queue.cc:260] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:44.830248 22893 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 21. Current term is 22.
I20260709 12:24:44.830394 22848 raft_consensus.cc:3060] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:44.831283 22836 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:44.810037 16441 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:44.803344 22743 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:44.808498 16516 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:44.819057 22925 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:44.826215 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:44.788941 22732 raft_consensus.cc:697] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 22 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:44.833560 16441 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:44.813927 16545 raft_consensus.cc:1240] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 23 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:44.824656 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:44.799315 22697 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:44.812295 22874 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:44.848902 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:44.849362  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:24:44.840608 16367 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.815881 22766 consensus_queue.cc:1048] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 12:24:44.815562 22890 raft_consensus.cc:2764] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:24:44.815840 22854 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:44.843003 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:44.853101 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:44.853890 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 45481.9us
I20260709 12:24:44.858769 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:44.857381 16516 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:44.838145 22882 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:24:44.864166 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 203241us
I20260709 12:24:44.860157 16476 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 22.
W20260709 12:24:44.871743 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 75732.8us
I20260709 12:24:44.874169 17142 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:44.877238 17269 raft_consensus.cc:1240] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:24:44.872789 22854 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:44.878607 16472 raft_consensus.cc:1240] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:24:44.879467 16468 raft_consensus.cc:1275] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:24:44.886153 22886 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:44.898273 22524 consensus_queue.cc:1048] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:24:44.901860 22874 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:44.905915 22836 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:44.875934 22732 consensus_queue.cc:237] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:44.893805 16515 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:44.907871 16515 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:44.908056 22897 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:44.908605 22897 raft_consensus.cc:697] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 21 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:44.909061 16515 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:44.906353 16442 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:44.909482 22897 consensus_queue.cc:237] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:44.912025 16442 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:44.912786 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:44.913547 17269 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 22.
I20260709 12:24:44.913544 16442 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:44.864648 22893 raft_consensus.cc:2804] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:44.915977 16442 leader_election.cc:400] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 21. Current term is 22.
I20260709 12:24:44.864498 22778 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:44.938047 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:44.952572 16442 leader_election.cc:400] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 21. Current term is 22.
I20260709 12:24:44.933909 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:44.956959 16396 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 22.
I20260709 12:24:44.959041 16367 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:44.852651 22553 consensus_queue.cc:260] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:44.971000 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:44.969349 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:44.973131 17264 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:44.973755 16367 leader_election.cc:400] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17.
I20260709 12:24:44.974115 16367 leader_election.cc:403] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:44.881804 22788 raft_consensus.cc:2804] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:44.975093 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:44.979918 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:44.914460 22882 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:44.868525 22925 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:44.950325 22377 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:44.926224 22924 raft_consensus.cc:2764] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:24:44.968533 22893 raft_consensus.cc:697] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:44.950690 22886 raft_consensus.cc:697] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 20 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:44.955592 22765 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:44.970425 22927 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.971932 22836 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:45.013253 16442 leader_election.cc:400] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for earlier term 18. Current term is 19.
I20260709 12:24:44.985203 22874 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:45.026087 17255 raft_consensus.cc:1240] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:24:45.026778 17263 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 21.
I20260709 12:24:44.982726 22524 consensus_queue.cc:1059] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 12:24:45.043702 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:45.002110 22788 raft_consensus.cc:697] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 21 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:45.047384 17142 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:45.000881 22616 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:24:45.049110 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 81071.4us
I20260709 12:24:45.017055 22890 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.052817 16367 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:45.026443 22848 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17.
I20260709 12:24:45.034586 22354 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:45.056756 17142 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:45.058306 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:45.060278 16367 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:45.061154 16441 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:45.071758 16367 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:45.083273 17271 raft_consensus.cc:1240] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 20. Current term is 21. Ops: []
W20260709 12:24:45.083918 22929 log.cc:927] Time spent T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.208s	user 0.003s	sys 0.000s
I20260709 12:24:45.084466 16547 raft_consensus.cc:1275] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:24:45.084719 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:45.085800 16548 raft_consensus.cc:1275] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:24:45.086827 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:45.055547 22616 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:45.092928 22937 log.cc:927] Time spent T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.088s	user 0.000s	sys 0.007s
I20260709 12:24:45.093832 22930 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:24:45.094623 22930 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:45.095135 22930 raft_consensus.cc:697] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 20 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:45.032428 22927 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:45.100235 16399 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 20.
I20260709 12:24:45.101794 17271 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:45.101186 17264 raft_consensus.cc:2468] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 18.
I20260709 12:24:45.113335 16367 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:45.081024 22848 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:45.057636 22890 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:45.123437 22940 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:45.113019 22836 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:45.133677 22836 consensus_queue.cc:1059] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:45.134516 22836 raft_consensus.cc:2749] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:45.135115 22836 raft_consensus.cc:3055] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:45.135428 22836 raft_consensus.cc:740] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:45.112038 22886 consensus_queue.cc:237] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:45.130975 22940 consensus_queue.cc:1059] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:45.125238 22848 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:45.141017 22848 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:45.141119 22940 raft_consensus.cc:3055] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:45.141374 22927 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:45.141634 22940 raft_consensus.cc:740] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:45.142027 22854 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:45.117394 22883 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:45.142493 22940 consensus_queue.cc:260] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:45.116614 22893 consensus_queue.cc:237] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:45.165670 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 104611us
I20260709 12:24:45.140311 22890 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:45.096184 22930 consensus_queue.cc:237] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:45.137943 22743 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:45.123492 22778 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:45.171633 17258 raft_consensus.cc:2393] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 21: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:45.184753 17261 raft_consensus.cc:1240] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 12:24:45.186827 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:45.187502 16468 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:45.165689 22945 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:45.178864 22940 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:45.222779 17271 raft_consensus.cc:2410] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:24:45.180363 22778 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:45.243428 22836 consensus_queue.cc:260] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:45.246042 22925 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:45.341996 16369 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.313815 16516 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:45.345124 22836 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:45.347401 22945 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:24:45.331158 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 117215us
I20260709 12:24:45.222395 16477 raft_consensus.cc:1275] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:24:45.327986 16468 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
W20260709 12:24:45.337728 22912 consensus_meta.cc:287] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.563s	user 0.013s	sys 0.007s
W20260709 12:24:45.333624 22935 log.cc:927] Time spent T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.201s	user 0.000s	sys 0.004s
W20260709 12:24:45.339388 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 152379us
W20260709 12:24:45.339828 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 226592us
W20260709 12:24:45.320768 22936 log.cc:927] Time spent T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.175s	user 0.002s	sys 0.002s
I20260709 12:24:45.126950 22788 consensus_queue.cc:237] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:45.345249 22626 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:24:45.335765 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 267136us
I20260709 12:24:45.374706 16367 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:45.375895 22626 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.377331 16399 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47"
dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:59912
I20260709 12:24:45.377665 22626 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:45.340454 22760 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:45.357712 22883 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:45.382416 16367 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:45.357946 22874 consensus_queue.cc:1048] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:24:45.384182 16367 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:45.387145 22944 consensus_queue.cc:1059] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:45.388549 22947 consensus_queue.cc:1048] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:45.389951 22778 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:45.390874 22942 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:45.393618 16367 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:45.394697 22942 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:45.386252 22912 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:45.384459 22890 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:24:45.397559 16367 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:45.400792 22958 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:45.348228 22945 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:45.406347 22945 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:45.410300 22760 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:45.412748 22943 consensus_queue.cc:1048] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:45.428076 17269 raft_consensus.cc:1240] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 12:24:45.422397 22760 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.452319 22760 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:45.425148 22848 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:45.467965 16367 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:45.470429 16401 raft_consensus.cc:1240] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 12:24:45.470777 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:45.465696 17142 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:45.480908 16474 raft_consensus.cc:1275] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:24:45.481078 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:45.483054 16542 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:45.486887 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:45.481386 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:45.491355 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:45.492905 16398 raft_consensus.cc:1240] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:45.488901 16514 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:45.479867 22524 raft_consensus.cc:2804] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:45.504935 22524 raft_consensus.cc:697] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 18 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:45.505784 22616 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:45.506021 22524 consensus_queue.cc:237] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:45.506913 22626 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:45.508368 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:45.509088 22626 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 23 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:45.488680 22874 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:45.485726 22947 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:45.531188 22760 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:45.511566 22960 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:45.532837 22874 raft_consensus.cc:697] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:45.533946 22874 consensus_queue.cc:237] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:45.489171 22886 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:45.544847 16396 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 18.
I20260709 12:24:45.589090 16542 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 21.
I20260709 12:24:45.598083 16515 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:45.561380 22947 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:45.614602 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:45.619361 16515 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:45.598856 22765 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:45.622567 16545 raft_consensus.cc:1240] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:45.623421 16515 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:45.610000 22843 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:45.627683 22958 consensus_queue.cc:1059] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:45.632009 16515 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:45.631927 16551 raft_consensus.cc:1240] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:45.630041 16367 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:45.634398 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:45.630972 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:45.637013 16547 raft_consensus.cc:2368] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 22. Current term is 23.
I20260709 12:24:45.635346 16548 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 18.
I20260709 12:24:45.588115 22886 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:45.622494 22815 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:45.649114 22880 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:24:45.650686 22880 consensus_queue.cc:1059] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:45.651609 22880 consensus_queue.cc:1059] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:45.635892 22814 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:45.652352 22836 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:45.624616 22607 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:45.648191 16442 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:45.656478 22848 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:45.629583 22765 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:45.658967 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:45.660879 22765 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:45.661640 22760 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:45.662156 22760 raft_consensus.cc:697] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 18 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:45.663022 22760 consensus_queue.cc:237] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:45.665035 16367 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:45.656118 22746 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.665591 22815 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:45.666065 16367 leader_election.cc:400] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 election: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 22. Current term is 23.
I20260709 12:24:45.665786 22746 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:45.666400 16367 leader_election.cc:403] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:24:45.667451 22848 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:45.667820 22848 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.668316 22836 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:45.668560 22950 raft_consensus.cc:3055] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:45.669093 22950 raft_consensus.cc:740] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:45.669991 22950 consensus_queue.cc:260] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:45.670951 22743 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:45.671278 22950 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:45.660799 22893 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.668159 22848 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:45.672950 22893 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:45.675606 22893 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:45.680176 16399 raft_consensus.cc:606] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:24:45.680676 16399 raft_consensus.cc:614] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:45.668121 22746 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:45.655767 22854 raft_consensus.cc:3055] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:45.684778 22854 raft_consensus.cc:740] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:45.685752 22854 consensus_queue.cc:260] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:45.689280 22947 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:45.693241 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:45.693701 22607 raft_consensus.cc:697] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 21 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:45.694024 16398 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:24:45.694703 22607 consensus_queue.cc:237] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:45.687002 22854 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:45.697662 22836 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491) with higher term. Message: Invalid argument: T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 22. Current term is 23.
I20260709 12:24:45.701357 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:45.702869 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:45.706039 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:45.705219 16442 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:45.706888 17264 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 15.
I20260709 12:24:45.702162 16472 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:45.709558 17142 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:45.710644 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:45.711504 16542 raft_consensus.cc:2393] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 18: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:45.712535 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:45.713655 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:45.713757 17258 raft_consensus.cc:3060] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:45.715832 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:45.716791 22970 log.cc:927] Time spent T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:24:45.716980 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:45.717115 22848 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:45.716599 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:45.657197 22843 consensus_queue.cc:1059] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 12:24:45.720281 22843 raft_consensus.cc:3055] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:45.720674 22843 raft_consensus.cc:740] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:45.721113 22848 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:45.721477 22843 consensus_queue.cc:260] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:45.722685 22843 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:45.733374 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:45.734355 16542 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:45.746196 17258 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 23.
I20260709 12:24:45.716614 16548 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 23.
I20260709 12:24:45.748420 16397 raft_consensus.cc:1275] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:24:45.755523 17258 raft_consensus.cc:1275] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:24:45.758900 16402 raft_consensus.cc:1275] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 23 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:24:45.712040 22880 raft_consensus.cc:2804] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:45.737733 22874 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:45.761253 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:45.770875 16542 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 19.
I20260709 12:24:45.766140 22626 consensus_queue.cc:1048] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:24:45.767961 22933 log.cc:927] Time spent T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.718s	user 0.002s	sys 0.000s
I20260709 12:24:45.772620 16367 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:45.771137 22874 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.776717 22886 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:45.785039 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:45.787840 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:45.790884 16394 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:24:45.794435 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:45.795567 16400 raft_consensus.cc:2393] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 20: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:45.770808 22880 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.796618 22940 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:45.797290 22940 raft_consensus.cc:697] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 19 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:45.798221 22815 consensus_queue.cc:1048] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:45.798333 22940 consensus_queue.cc:237] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:45.797089 22880 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:45.804937 16398 raft_consensus.cc:2468] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 23.
W20260709 12:24:45.808704 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.807487 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:45.811306 16400 raft_consensus.cc:1275] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:24:45.812615 16397 raft_consensus.cc:1240] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:45.814796 16516 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:45.821064 22760 consensus_queue.cc:1059] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:24:45.821496 22786 raft_consensus.cc:2804] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:45.822813 22786 raft_consensus.cc:697] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 23 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:45.790199 22874 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:45.823796 22786 consensus_queue.cc:237] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:45.845157 16477 raft_consensus.cc:1275] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:24:45.850821 16471 raft_consensus.cc:1275] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:24:45.855656 16470 raft_consensus.cc:1275] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:24:45.854607 22947 consensus_queue.cc:1048] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 12:24:45.860706 22388 consensus_queue.cc:1048] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 12:24:45.883749 16398 raft_consensus.cc:1275] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 12:24:45.884298 16398 pending_rounds.cc:85] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 9
I20260709 12:24:45.884577 16398 pending_rounds.cc:107] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 12:24:45.862958 22943 consensus_queue.cc:1048] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 12:24:45.884967 16398 raft_consensus.cc:2889] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:45.887738 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.890858 22354 raft_consensus.cc:3055] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:45.891388 22354 raft_consensus.cc:740] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:45.884482 17254 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 1551) took 1053 ms. Trace:
I20260709 12:24:45.893183 17254 rpcz_store.cc:276] 0709 12:24:44.818492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.819025 (+   533us) service_pool.cc:224] Handling call
0709 12:24:44.819401 (+   376us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:44.837590 (+ 18189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:44.837628 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:44.876270 (+ 38642us) log.cc:844] Serialized 52 byte log entry
0709 12:24:44.950821 (+ 74551us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:44.950873 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.950944 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:45.766835 (+815891us) spinlock_profiling.cc:243] Waited 2.43 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:24:45.868922 (+102087us) raft_consensus.cc:1685] finished
0709 12:24:45.868943 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:45.871649 (+  2706us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16360,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":15083,"spinlock_wait_cycles":5588864,"thread_start_us":73982,"threads_started":1,"wal-append.queue_time_us":103457}
I20260709 12:24:45.919533 16394 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
I20260709 12:24:45.927910 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:45.928786 17269 raft_consensus.cc:1240] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:45.928889 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:45.938560 17255 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45230 (request call id 1545) took 1065 ms. Trace:
I20260709 12:24:45.940790 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:45.949460 17263 raft_consensus.cc:2368] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 22. Current term is 23.
I20260709 12:24:45.941401 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:45.942201 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:45.951495 16547 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:24:45.943624 17143 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:45.948093 17255 rpcz_store.cc:276] 0709 12:24:44.873302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.873461 (+   159us) service_pool.cc:224] Handling call
0709 12:24:45.017989 (+144528us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:45.041008 (+ 23019us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.938317 (+897309us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b5400a46d50. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:45.938493 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":325451520}
I20260709 12:24:45.913045 22893 consensus_queue.cc:1048] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 12:24:45.967346 22616 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:45.968000 22616 raft_consensus.cc:697] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 21 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:45.969152 22616 consensus_queue.cc:237] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:45.973048 16367 leader_election.cc:400] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 22. Current term is 23.
I20260709 12:24:45.984318 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:45.985369 16543 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 15.
I20260709 12:24:45.990995 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:46.006225 22874 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:46.013309 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:46.014989 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:46.015842 16477 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:46.016621 22553 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:46.017740 16469 raft_consensus.cc:1275] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:24:46.020889 22874 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:46.025720 16395 raft_consensus.cc:2410] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:24:46.030789 16477 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:46.032495 22880 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:45.915841 22354 consensus_queue.cc:260] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:46.035691 22354 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:46.040218 22607 consensus_queue.cc:1048] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:46.051265 17271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45230 (request call id 1551) took 1001 ms. Trace:
I20260709 12:24:46.051772 17271 rpcz_store.cc:276] 0709 12:24:45.054464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.084406 (+ 29942us) service_pool.cc:224] Handling call
0709 12:24:46.051052 (+966646us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b5400654354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:46.051181 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37536000}
I20260709 12:24:46.056092 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:46.056836 16401 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:46.056939 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:46.060683 17142 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:46.061708 22902 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:46.062372 22902 raft_consensus.cc:697] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:46.063318 22902 consensus_queue.cc:237] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:46.071607 16473 raft_consensus.cc:1275] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:24:46.071807 16475 raft_consensus.cc:1275] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:24:46.076671 22814 consensus_queue.cc:1048] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:46.078020 16550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1732) took 1416 ms. Trace:
I20260709 12:24:46.078394 16550 rpcz_store.cc:276] 0709 12:24:44.661551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.661673 (+   122us) service_pool.cc:224] Handling call
0709 12:24:44.661954 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:44.790768 (+128814us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:24:44.813047 (+ 22279us) log.cc:844] Serialized 14 byte log entry
0709 12:24:44.951157 (+138110us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:44.951214 (+    57us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:44.951264 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.083359 (+132095us) spinlock_profiling.cc:243] Waited 17 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:24:46.077763 (+994404us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b540061eed4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:46.077811 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:46.077926 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":78435,"prepare.queue_time_us":22593,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":9415,"raft.queue_time_us":78835,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":14210,"spinlock_wait_cycles":334691456,"thread_start_us":59211,"threads_started":1,"wal-append.queue_time_us":61992}
I20260709 12:24:46.079689 22950 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:46.084363 16400 raft_consensus.cc:1275] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:24:46.081930 16515 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:46.088027 22950 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:46.088315 16398 raft_consensus.cc:1275] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 12:24:46.090107 16398 pending_rounds.cc:85] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 12
I20260709 12:24:46.090516 22950 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:46.090919 16398 pending_rounds.cc:107] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 12:24:46.091336 16398 raft_consensus.cc:2889] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:46.091686 22747 consensus_queue.cc:1048] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:24:46.094077 22971 log.cc:927] Time spent T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.286s	user 0.002s	sys 0.000s
I20260709 12:24:46.099715 16515 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:46.106678 17271 raft_consensus.cc:1275] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:24:46.118386 22960 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:46.120743 22960 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:46.121213 22960 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:46.076179 16401 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:24:46.124521 22647 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:46.128295 16442 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:46.127113 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:46.143015 22950 raft_consensus.cc:2804] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:46.143841 22950 raft_consensus.cc:697] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:46.144174 22880 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:46.144976 22950 consensus_queue.cc:237] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:46.149487 22960 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:46.116156 22848 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:46.113511 22842 consensus_queue.cc:1048] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:24:46.142480 22607 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.246219 22714 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:46.247068 22714 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:46.167572 22848 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:46.221300 22607 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:46.258354 22960 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:46.290779 22714 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:46.308141 16441 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.310472 22945 consensus_queue.cc:1059] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:46.314821 16441 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:46.318353 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:46.319315 22684 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:46.319893 22684 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:46.320148 22902 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:46.320468 22684 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:46.320824 22902 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:46.325189 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:46.311364 22945 raft_consensus.cc:3055] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:46.327838 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:46.331468 16477 raft_consensus.cc:2368] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 17. Current term is 18.
I20260709 12:24:46.351501 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:46.317463 22941 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:46.356262 16366 leader_election.cc:400] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 17. Current term is 18.
I20260709 12:24:46.357046 16366 leader_election.cc:403] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:46.326995 22945 raft_consensus.cc:740] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:46.361902 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:46.317802 22882 consensus_queue.cc:1059] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:24:46.339803 22902 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:46.357357 22553 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:46.373919 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 260683us
I20260709 12:24:46.387264 16403 raft_consensus.cc:1275] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:24:46.388737 16399 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:59912 (request call id 3) took 1017 ms. Trace:
I20260709 12:24:46.389348 16399 rpcz_store.cc:276] 0709 12:24:45.374252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.374407 (+   155us) service_pool.cc:224] Handling call
0709 12:24:46.388474 (+1014067us) spinlock_profiling.cc:243] Waited 302 ms on lock 0x7b540015e354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e83853 00007f287ae5160a 00007f28743d77d4 00007f28743d7762 00007f28743d7691 00007f28743d760c 00007f28743d68c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:46.388625 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":694413184}
W20260709 12:24:46.395095 16442 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.398197 16471 raft_consensus.cc:3055] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:46.398626 16471 raft_consensus.cc:740] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:46.366261 22854 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 17. Current term is 18.
I20260709 12:24:46.399210 16471 consensus_queue.cc:260] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:46.404520 16471 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:46.405915 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 284277us
W20260709 12:24:46.408949 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 309327us
W20260709 12:24:46.429701 22987 log.cc:927] Time spent T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:24:46.431283 22949 consensus_queue.cc:1048] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:46.408869 22945 consensus_queue.cc:260] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:46.437633 22945 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:46.444911 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:46.446285 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.452734 16401 raft_consensus.cc:1275] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:24:46.397454 22553 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:46.466352 22553 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:46.420835 22607 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:46.456053 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:46.460986 16399 raft_consensus.cc:1240] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 18 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:46.485680 16469 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 17.
I20260709 12:24:46.507200 17144 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:46.527398 16399 raft_consensus.cc:1275] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:24:46.499589 22553 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.479732 22684 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:46.545428 22949 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:46.526490 22616 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:46.539101 22553 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:46.564548 16472 raft_consensus.cc:1240] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 24 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:46.547538 22354 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.538031 22945 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:46.558760 22949 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:46.575971 22684 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 24 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:46.576750 22882 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:46.633533 22882 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:46.635561 22943 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.578727 22553 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:46.583431 16477 raft_consensus.cc:3055] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:46.636724 16477 raft_consensus.cc:740] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:24:46.638682 22973 log.cc:927] Time spent T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.829s	user 0.005s	sys 0.000s
I20260709 12:24:46.638181 16477 consensus_queue.cc:260] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:46.602438 16367 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:46.604641 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 193230us
I20260709 12:24:46.641904 22949 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:46.606892 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:46.643628 16551 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:24:46.607613 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:46.644964 22553 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:46.608043 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:46.646837 16545 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:46.648049 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:46.654762 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:46.615545 22354 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:46.655332 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:46.655601 16546 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 13.
I20260709 12:24:46.599700 22778 consensus_queue.cc:1048] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
W20260709 12:24:46.654881 22975 log.cc:927] Time spent T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.846s	user 0.002s	sys 0.000s
I20260709 12:24:46.615494 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:46.623775 16400 raft_consensus.cc:1275] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 12:24:46.658231 16400 pending_rounds.cc:85] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 12
I20260709 12:24:46.658483 16400 pending_rounds.cc:107] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 12:24:46.658740 16400 raft_consensus.cc:2889] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:46.659672 16546 raft_consensus.cc:1275] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:24:46.659965 16395 raft_consensus.cc:2393] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 14: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:46.630947 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:46.632740 17266 raft_consensus.cc:1240] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:46.661419 16546 raft_consensus.cc:1240] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:46.662846 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:46.663379 16546 raft_consensus.cc:3060] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:46.664808 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:46.666472 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:46.661419 22945 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:46.640156 16477 raft_consensus.cc:3060] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:46.673302 16545 raft_consensus.cc:1240] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:46.674198 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:46.672308 16546 raft_consensus.cc:2468] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
I20260709 12:24:46.675006 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:46.678434 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 267257us
I20260709 12:24:46.676070 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:46.689457 17255 raft_consensus.cc:1275] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:24:46.682276 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:46.722841 16400 raft_consensus.cc:2393] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 17: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:46.695859 17143 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:46.725144 16399 raft_consensus.cc:1275] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:24:46.732880 16549 raft_consensus.cc:1275] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:24:46.734511 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:24:46.736212 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.677804 22684 consensus_queue.cc:1048] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 12:24:46.734609 17143 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:46.636264 22943 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:46.740001 22885 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:46.740902 17271 raft_consensus.cc:2410] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:24:46.744148 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:46.673985 22945 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:46.760264 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:46.727458 22880 consensus_queue.cc:1059] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 12:24:46.741135 22854 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.695153 22814 consensus_queue.cc:1048] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 12:24:46.677999 23007 consensus_queue.cc:1048] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.080s
I20260709 12:24:46.755766 16551 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
W20260709 12:24:46.788727 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 261615us
I20260709 12:24:46.795058 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:46.786314 22882 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:46.817061 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 293109us
I20260709 12:24:46.817807 16472 raft_consensus.cc:1275] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:24:46.787727 22854 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:46.823472 22885 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:46.837477 22874 consensus_queue.cc:1059] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:46.857935 22354 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:46.841755 16477 raft_consensus.cc:1275] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:24:46.848084 22876 raft_consensus.cc:3055] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:46.862934 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:46.849236 22940 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.016s	sys 0.008s
I20260709 12:24:46.858240 22885 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:24:46.908743 22940 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.016s	sys 0.008s
W20260709 12:24:46.863332 22924 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 12:24:46.861852 22876 raft_consensus.cc:740] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:46.869423 22943 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:46.928661 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 189440us
I20260709 12:24:46.916465 22885 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:24:46.993506 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:47.059640 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 49305us
W20260709 12:24:47.065421 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:47.082652 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 257047us
I20260709 12:24:47.084036 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:24:47.015791 23005 log.cc:927] Time spent T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:24:47.113926 17263 raft_consensus.cc:1275] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:24:47.061636 22882 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:47.009507 22354 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:47.060354 22945 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:24:47.129693 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 190841us
I20260709 12:24:47.130494 17143 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:47.105470 16514 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:47.137516 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 275513us
I20260709 12:24:47.149188 16514 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:47.149916 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 54080.4us
I20260709 12:24:47.162503 16515 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:47.132980 22945 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:47.117327 22876 consensus_queue.cc:260] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:47.129108 22354 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:47.129238 22882 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:47.184540 17144 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:47.190917 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:47.191336 22876 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:47.192478 22377 consensus_queue.cc:1048] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:47.194581 23025 consensus_queue.cc:1048] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:47.196554 22607 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:47.197827 23030 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.198462 23030 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:47.187527 22945 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:47.187306 23012 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:47.196290 23025 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.205350 23025 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:47.206185 16397 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46250 (request call id 1911) took 1402 ms. Trace:
I20260709 12:24:47.208201 23025 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:47.209446 23030 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:47.214864 17143 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:47.225769 23026 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:47.226293 23026 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:47.226334 23033 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:47.226801 23026 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:47.233399 23004 log.cc:927] Time spent T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.722s	user 0.004s	sys 0.000s
I20260709 12:24:47.208521 23000 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:47.237494 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:47.242595 22553 consensus_queue.cc:1048] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:47.244306 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:47.254943 16473 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 1490) took 1142 ms. Trace:
I20260709 12:24:47.206749 16397 rpcz_store.cc:276] 0709 12:24:45.803337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.831820 (+ 28483us) service_pool.cc:224] Handling call
0709 12:24:45.832234 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:45.832722 (+   488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:45.832765 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:45.852862 (+ 20097us) log.cc:844] Serialized 26 byte log entry
0709 12:24:46.664233 (+811371us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b1800451c60. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:24:46.915511 (+251278us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:46.915553 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.915627 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:47.184183 (+268556us) raft_consensus.cc:1685] finished
0709 12:24:47.184205 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:47.206072 (+ 21867us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9417,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":6068,"spinlock_wait_cycles":2562816,"thread_start_us":251092,"threads_started":1,"wal-append.queue_time_us":1326272}
I20260709 12:24:47.251533 22842 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.174327 22854 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:47.183720 22977 log.cc:927] Time spent T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 1.422s	user 0.002s	sys 0.003s
I20260709 12:24:47.256767 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:47.256049 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 71609.8us
I20260709 12:24:47.255358 16473 rpcz_store.cc:276] 0709 12:24:46.105779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.112751 (+  6972us) service_pool.cc:224] Handling call
0709 12:24:46.129277 (+ 16526us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:46.129723 (+   446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:46.129765 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:46.143409 (+ 13644us) log.cc:844] Serialized 26 byte log entry
0709 12:24:46.257286 (+113877us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b18000d2940. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a290d1 00007f2875e44a0f
0709 12:24:46.405416 (+148130us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:46.405461 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.405533 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:47.232472 (+826939us) spinlock_profiling.cc:243] Waited 9.3 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:24:47.248130 (+ 15658us) raft_consensus.cc:1685] finished
0709 12:24:47.248163 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:47.248315 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11242,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":4838,"spinlock_wait_cycles":51472512,"thread_start_us":147935,"threads_started":1,"wal-append.queue_time_us":354778}
I20260709 12:24:47.262104 22945 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:47.262405 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:47.262656 22945 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:47.260680 22747 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:47.264104 16548 raft_consensus.cc:1275] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:24:47.205235 23038 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:47.253196 23021 log.cc:927] Time spent T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
I20260709 12:24:47.274088 16396 raft_consensus.cc:1275] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 23 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:24:47.257766 23012 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.259209 22924 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:24:47.211625 23034 consensus_queue.cc:1048] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 12:24:47.296940 22848 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:47.322893 23037 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:47.323554 23037 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:47.329417 16441 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:47.332018 17261 raft_consensus.cc:1275] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:24:47.332427 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:47.333244 16366 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:47.338750 16547 raft_consensus.cc:2393] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 17: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
W20260709 12:24:47.338778 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 59866.4us
I20260709 12:24:47.340979 16441 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:47.264695 22842 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:47.318315 22354 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:47.326045 22854 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:47.354849 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:47.358703 16473 pending_rounds.cc:85] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 10
I20260709 12:24:47.359272 16473 pending_rounds.cc:107] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 12:24:47.359616 16473 raft_consensus.cc:2889] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:47.364694 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:47.344497 22924 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:47.349413 22945 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:47.348176 22880 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:47.275700 23038 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:47.364704 22616 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.006s	sys 0.001s
I20260709 12:24:47.350042 22876 consensus_queue.cc:1048] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:24:47.363517 23040 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:47.367050 23037 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:47.309083 23012 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:47.354405 22960 consensus_queue.cc:1048] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 12:24:47.369985 22553 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:47.400067 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:47.397127 23020 log.cc:927] Time spent T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 12:24:47.397236 22882 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:47.369958 22848 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:47.367808 23034 consensus_queue.cc:1048] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 12:24:47.368748 22854 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:47.416865 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:47.417821 22854 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:47.418530 23042 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:47.419147 23042 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:47.419812 23038 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:47.372026 23035 consensus_queue.cc:1048] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:24:47.421509 23042 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:47.422117 23012 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:47.424310 23026 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:47.426981 23033 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:47.427803 16441 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:47.429224 22882 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:47.430239 23026 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:47.431480 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:47.435135 23013 raft_consensus.cc:3055] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:24:47.404394 23040 raft_consensus.cc:697] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 17 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:47.435652 23013 raft_consensus.cc:740] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:47.436290 23040 consensus_queue.cc:237] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:47.381448 22616 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.006s	sys 0.001s
I20260709 12:24:47.377647 23030 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.406402 22553 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:47.394917 22874 consensus_queue.cc:1048] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 12:24:47.412600 22848 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:47.413463 22842 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:47.443454 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:47.450609 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:47.464088 16547 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:47.453675 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:47.468864 16551 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
W20260709 12:24:47.469148 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 87205.6us
I20260709 12:24:47.454643 22876 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:47.449829 23007 consensus_queue.cc:1048] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 12:24:47.465494 23038 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:47.480304 22848 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:47.480924 22522 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:47.481421 22522 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:47.483508 22522 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:47.478330 22876 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:47.485090 22553 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:47.486693 22876 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:47.455834 23030 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:47.487203 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:47.478513 22885 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:47.491751 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:47.495298 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:47.496268 16398 raft_consensus.cc:2410] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:24:47.497661 16399 raft_consensus.cc:1275] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:24:47.497903 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:47.500046 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:47.504433 23030 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:47.504922 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:47.506839 22885 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:47.514186 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:47.520726 16477 raft_consensus.cc:2410] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:24:47.521303 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:47.522490 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:47.524724 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:47.529317 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:47.529567 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:47.530185 17270 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 17.
I20260709 12:24:47.533217 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:47.533960 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:47.534914 16551 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 17.
I20260709 12:24:47.538228 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:47.541944 16551 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:24:47.544080 16550 raft_consensus.cc:1275] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:24:47.544030 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:47.545740 16545 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 18.
I20260709 12:24:47.490989 23038 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:47.552902 22786 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:47.554234 22786 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:47.558281 23049 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:47.559226 23049 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:47.560518 22765 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:47.561139 22765 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:47.562903 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:47.563843 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:47.565255 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:47.565412 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:47.569093 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.572265 16367 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:47.575580 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:47.577034 16551 raft_consensus.cc:2393] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 16: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:47.576896 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:47.577574 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:47.578275 16544 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 21.
I20260709 12:24:47.581326 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:47.582026 16546 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 18.
I20260709 12:24:47.491200 22616 consensus_queue.cc:1048] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.084s
I20260709 12:24:47.465703 23013 consensus_queue.cc:260] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:47.562341 23028 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:47.599866 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:47.582702 16367 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:47.584656 22848 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:47.577796 22943 consensus_queue.cc:1048] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 12:24:47.620772 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:47.649286 22876 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.650043 22876 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:47.653496 22876 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:47.685989 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:47.698593 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:47.698930 16549 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:24:47.699779 16549 raft_consensus.cc:606] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:24:47.700150 16549 raft_consensus.cc:614] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:47.606405 23031 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:47.606439 23040 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:47.619904 22848 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.725997 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:47.743285 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:47.726363 22553 consensus_queue.cc:1048] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:24:47.750852 23049 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:47.606521 23028 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:47.755515 23028 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:47.740515 23040 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:47.740839 22876 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:47.757695 22876 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:47.740506 23031 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:47.761251 22786 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:47.761616 23031 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:47.764474 23040 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:47.759792 22876 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:47.713325 23013 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:24:47.772789 23023 log.cc:927] Time spent T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:24:47.804900 16397 raft_consensus.cc:1275] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 12:24:47.805414 16397 pending_rounds.cc:85] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 12
I20260709 12:24:47.805694 16397 pending_rounds.cc:107] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 12:24:47.805963 16397 raft_consensus.cc:2889] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:47.796559 16547 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:47.740100 22848 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:47.780486 23068 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:47.820606 16543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37242 (request call id 1751) took 1020 ms. Trace:
I20260709 12:24:47.821120 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:47.823845 17255 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 21.
I20260709 12:24:47.823850 17143 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:47.747695 23032 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:47.827324 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:47.829339 16544 raft_consensus.cc:1275] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:24:47.834043 17143 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:47.823381 22765 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:47.840526 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:47.843425 17271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45224 (request call id 1558) took 1209 ms. Trace:
I20260709 12:24:47.823473 16543 rpcz_store.cc:276] 0709 12:24:46.800450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.800596 (+   146us) service_pool.cc:224] Handling call
0709 12:24:46.800897 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:47.010706 (+209809us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:24:47.082516 (+ 71810us) log.cc:844] Serialized 14 byte log entry
0709 12:24:47.339436 (+256920us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:47.369143 (+ 29707us) log.cc:844] Serialized 26 byte log entry
0709 12:24:47.372992 (+  3849us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:47.373036 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.373107 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:47.756008 (+382901us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b240026f378. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872c229c4 00007f2872c20acb 00007f2872ddbdc9 00007f2872ddba55 00007f2872ddafb0 00007f2875a51488 00007f2875a51be8 00007f2875a29398 00007f2875a2476a 00007f2875a23bb6
0709 12:24:47.816553 (+ 60545us) raft_consensus.cc:1685] finished
0709 12:24:47.820407 (+  3854us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:47.820546 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1381,"prepare.queue_time_us":71767,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":2099,"raft.queue_time_us":32178,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":11821,"spinlock_wait_cycles":31801088,"thread_start_us":424838,"threads_started":2,"wal-append.queue_time_us":268167}
I20260709 12:24:47.827780 22842 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:47.845494 17271 rpcz_store.cc:276] 0709 12:24:46.628735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.630636 (+  1901us) service_pool.cc:224] Handling call
0709 12:24:47.831146 (+1200510us) spinlock_profiling.cc:243] Waited 79.8 ms on lock 0x7b540005df54. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:47.837756 (+  6610us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":183622528}
I20260709 12:24:47.858471 17143 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:47.865100 23026 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:47.871166 23026 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.871650 23026 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:47.871840 23055 log.cc:927] Time spent T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 12:24:47.874079 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:47.874533 23031 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:47.874718 16400 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 18.
I20260709 12:24:47.875139 23031 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:47.876884 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:47.877712 23031 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:47.837174 23039 consensus_queue.cc:1048] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.041s
W20260709 12:24:47.880093 23054 log.cc:927] Time spent T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:24:47.832226 23068 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:47.890712 23068 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:47.835647 23038 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:47.856079 23034 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:47.897588 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:47.897900 16543 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:24:47.900561 16543 raft_consensus.cc:606] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 13 LEADER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:47.898583 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:47.865082 17143 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:24:47.899426 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 290402us
I20260709 12:24:47.902444 16516 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:47.904412 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:47.905571 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:47.906414 16367 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:47.830096 23032 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:47.922082 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:47.908579 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:47.925380 16394 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 17.
I20260709 12:24:47.907608 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:47.925503 16367 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:47.929586 16367 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:47.930148 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:47.935940 23014 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:24:47.938844 23064 log.cc:927] Time spent T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:24:47.943751 17264 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 24.
I20260709 12:24:47.932665 23038 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:47.945385 23038 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:47.945778 23038 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:47.948451 23040 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:47.948853 23040 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.949206 23040 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:47.929153 22743 consensus_queue.cc:1048] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:24:47.948014 23032 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:47.907652 16441 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:47.959728 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:47.904870 23067 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.935504 16367 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:47.962102 16516 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:47.966846 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:47.968240 16545 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:24:47.969135 16545 raft_consensus.cc:606] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:47.969623 16545 raft_consensus.cc:614] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:47.969425 16515 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:47.909808 23031 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:47.973014 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:47.976003 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:47.956951 23068 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:47.983073 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:47.997298 23072 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:48.001565 23042 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:48.002233 23042 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:48.002532 23074 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:47.958225 23014 raft_consensus.cc:697] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:47.974314 23028 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:47.978296 23031 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:48.008422 23074 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:48.023226 23028 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:48.039779 23028 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:48.026480 23068 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:48.036026 23031 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:48.044348 23074 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:48.095021 23059 log.cc:927] Time spent T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 12:24:48.104941 16367 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:48.099658 23026 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:48.116514 23022 log.cc:927] Time spent T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.951s	user 0.004s	sys 0.001s
I20260709 12:24:48.116909 23073 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:47.972764 23067 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:48.122133 23014 consensus_queue.cc:237] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:48.123647 23067 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:48.118288 22848 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:48.127918 23026 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:48.129424 22848 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:48.131039 16547 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d46e3072dec049b18b323d65470603ec"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:24:48.131819 16547 raft_consensus.cc:606] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 19 LEADER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:48.080235 23068 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:48.142042 17266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 1562) took 1475 ms. Trace:
I20260709 12:24:48.152812 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:48.153486 16548 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:48.157343 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 47692.2us
I20260709 12:24:48.158195 16367 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:48.162245 16367 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:48.163341 22940 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:48.174119 23042 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:48.183794 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.196697 16367 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:48.218043 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 100793us
I20260709 12:24:48.219391 16367 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:48.222885 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 67507.8us
I20260709 12:24:48.227300 16367 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:48.232584 16367 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:48.233601 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 325994us
I20260709 12:24:48.235927 16367 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:48.223116 23084 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:48.251089 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:48.251637 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:48.260468 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:48.238991 23073 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:48.226713 22940 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:48.252988 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:48.261605 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:48.265200 17270 raft_consensus.cc:2393] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 18: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:48.242311 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:48.267714 16397 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 22.
I20260709 12:24:48.267385 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:48.265440 16396 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
I20260709 12:24:48.185139 22848 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:48.269003 16367 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:48.244704 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:48.269748 16550 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:24:48.261090 16399 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 17.
I20260709 12:24:48.229847 23067 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:48.272074 16550 raft_consensus.cc:606] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:48.277276 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:48.264596 23066 log.cc:927] Time spent T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.383s	user 0.003s	sys 0.000s
I20260709 12:24:48.237372 23012 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:48.260865 23084 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.266907 22854 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:48.266490 22626 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.263391 23085 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.262639 23036 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.263185 23068 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:48.270923 23040 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:48.271698 23072 consensus_queue.cc:1048] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 12:24:48.301080 23093 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.301299 23040 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:48.286728 22626 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:48.284112 23086 log.cc:927] Time spent T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:24:48.278661 23073 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:48.302759 16547 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 17.
I20260709 12:24:48.302700 23073 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:48.278456 23082 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:48.301664 23093 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:48.314944 16548 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
I20260709 12:24:48.319499 17143 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:48.290227 23084 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:48.292593 23012 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:48.342075 22746 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:48.317981 23085 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:48.327952 23039 consensus_queue.cc:1048] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:24:48.372001 23012 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:48.321800 23036 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:48.384873 16442 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:48.360867 23089 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:48.394479 23101 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:48.395210 23101 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:48.405817 22377 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:48.383780 23073 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:48.409298 23094 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:48.412063 22377 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:48.415071 22377 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:48.419540 23037 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:48.421530 23037 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:48.421775 23085 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:48.422034 23037 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:48.422416 23085 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:48.422925 23085 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:48.428966 23088 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:48.412681 23094 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:48.441422 23101 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:48.459128 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 118341us
W20260709 12:24:48.491518 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 106688us
I20260709 12:24:48.407102 23089 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:48.471558 23101 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:48.429618 23088 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:48.489425 22626 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:48.517171 23101 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:48.531989 23098 log.cc:927] Time spent T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.171s	user 0.002s	sys 0.000s
W20260709 12:24:48.537957 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 52891.6us
I20260709 12:24:48.526973 23101 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:48.522100 23085 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:48.522362 23093 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:48.568246 23089 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:48.568907  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:24:48.572892 23036 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:48.573668 23104 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:48.574216 23104 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:48.568584 22746 raft_consensus.cc:697] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:48.524240 23102 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:48.564551 23074 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:48.146857 17266 rpcz_store.cc:276] 0709 12:24:46.666258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.667719 (+  1461us) service_pool.cc:224] Handling call
0709 12:24:46.668039 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:46.668257 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:24:46.841811 (+173554us) log.cc:844] Serialized 14 byte log entry
0709 12:24:47.950782 (+1108971us) spinlock_profiling.cc:243] Waited 7.96 ms on lock 0x7b1800143140. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2875a314bc 00007f2872d7f48b 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d99455 00007f2875e81060 00007f2875e93716 00007f2875ea4287
0709 12:24:48.047335 (+ 96553us) log.cc:844] Serialized 14 byte log entry
0709 12:24:48.141540 (+ 94205us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:48.141613 (+    73us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:24:48.141692 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.141797 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:48.141964 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68712,"prepare.queue_time_us":134375,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":6365,"raft.queue_time_us":332602,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":4489,"spinlock_wait_cycles":18300672,"thread_start_us":604060,"threads_started":4}
I20260709 12:24:48.583927 16516 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:48.581008 16366 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:48.594646 23094 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:48.595669 23100 log.cc:927] Time spent T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 12:24:48.576511 23085 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:48.596354 23085 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:48.597245 23028 consensus_meta.cc:287] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.556s	user 0.009s	sys 0.000s
I20260709 12:24:48.582901 23074 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:48.597713 23028 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:48.598441 23085 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:48.591150 23030 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:48.599730 23028 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:48.576407 22746 consensus_queue.cc:237] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:48.600948 23084 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:48.568167 23088 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:48.587391 22876 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.010s	sys 0.027s
I20260709 12:24:48.570951 23101 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:48.568322 23108 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:48.607277 23103 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:48.581143 23073 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:48.587695 22377 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:48.576323 23104 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:48.581074 23102 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:48.612567 22876 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.010s	sys 0.027s
I20260709 12:24:48.619695 16515 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:48.629724 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:48.630636 17258 raft_consensus.cc:2393] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 21: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:48.631193 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:48.631524 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 108759us
I20260709 12:24:48.640895 16394 raft_consensus.cc:2410] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:24:48.636673 16441 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:48.653057 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:48.660187 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:48.661228 16398 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:48.623883 23084 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:48.663909 16441 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:48.629665 23036 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:48.630532 22997 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:48.614558 23101 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:48.615442 23108 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:48.685073 23113 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:48.685932 23113 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:48.686276 23108 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:48.686590 23113 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:48.664769 23103 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:48.670516 23036 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:48.687844 23103 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:48.689551 23036 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:48.690368 23103 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:48.691198 23085 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:48.691699 23085 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:48.693835 23085 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:48.694339 23085 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:48.694905 23085 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:48.626398 23073 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:48.705710 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:48.679394 23031 consensus_meta.cc:287] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1: Time spent flushing consensus metadata: real 0.594s	user 0.011s	sys 0.003s
W20260709 12:24:48.680953 23038 consensus_meta.cc:287] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.735s	user 0.011s	sys 0.002s
I20260709 12:24:48.696797 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:48.693847 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:48.714188 16469 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 18.
I20260709 12:24:48.676486 23114 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:48.715667 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:48.717877 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:48.718407 17258 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 20.
I20260709 12:24:48.720031 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:48.721139 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:48.723407 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:48.723611 16398 raft_consensus.cc:2468] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 18.
I20260709 12:24:48.725265 16516 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:48.726394 16470 raft_consensus.cc:2393] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 24: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
W20260709 12:24:48.726977 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.727790 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:48.728324 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:48.728698 16475 raft_consensus.cc:2468] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:24:48.729590 17258 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:48.669198 22997 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:48.732649 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:48.745615 16471 raft_consensus.cc:2368] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 21. Current term is 22.
W20260709 12:24:48.730906 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 238178us
I20260709 12:24:48.738579 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:48.740833 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:48.757624 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:48.758976 16544 raft_consensus.cc:1240] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:48.760497 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:48.761411 16474 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 19.
I20260709 12:24:48.711552 23103 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:48.756887 22876 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:48.773113 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:48.766175 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:48.776023 16366 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:48.776504 17258 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 22.
I20260709 12:24:48.777002 16545 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 17.
I20260709 12:24:48.777592 16442 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:48.778061 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:48.778748 16474 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 22.
I20260709 12:24:48.780398 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:48.781069 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:48.766889 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:48.785075 16474 raft_consensus.cc:1275] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:24:48.776113 16441 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:48.788069 23094 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:48.788544 23094 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.788993 23094 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:48.788784 23123 consensus_queue.cc:1048] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:48.744560 23042 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:48.796066 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:48.796066 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:48.792958 16442 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:48.758028 22997 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:48.804103 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:48.830252 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:48.725994 23038 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:48.833657 16544 raft_consensus.cc:3060] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:24:48.775686 23101 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:48.727685 23031 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:48.792819 22940 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:48.784813 23104 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.843073 16441 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:48.842188 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:48.793458 23108 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:24:48.834100 23100 log.cc:927] Time spent T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:24:48.800959 16442 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
W20260709 12:24:48.856443 23116 log.cc:927] Time spent T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 12:24:48.857388 23085 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:48.859978 16403 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58430 (request call id 1846) took 1366 ms. Trace:
I20260709 12:24:48.860517 23085 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:48.861166 23085 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:48.861601 23085 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:24:48.864769 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 63913.9us
I20260709 12:24:48.865795 16442 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:48.860415 16403 rpcz_store.cc:276] 0709 12:24:47.494048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.494192 (+   144us) service_pool.cc:224] Handling call
0709 12:24:47.497225 (+  3033us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:47.531605 (+ 34380us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:24:47.710142 (+178537us) log.cc:844] Serialized 14 byte log entry
0709 12:24:48.620739 (+910597us) spinlock_profiling.cc:243] Waited 1.16 ms on lock 0x7b18000e9e00. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d7ee07 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f2875a2320b 00007f2875a28d23 00007f2875a2923a 00007f2875e93a2a
0709 12:24:48.708640 (+ 87901us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:48.726772 (+ 18132us) log.cc:844] Serialized 26 byte log entry
0709 12:24:48.727050 (+   278us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:48.732237 (+  5187us) log.cc:844] Serialized 14 byte log entry
0709 12:24:48.732495 (+   258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.732563 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:48.859750 (+127187us) raft_consensus.cc:1685] finished
0709 12:24:48.859773 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:48.859917 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67778,"prepare.queue_time_us":174639,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":27786,"raft.queue_time_us":35779,"raft.run_cpu_time_us":378,"raft.run_wall_time_us":29752,"spinlock_wait_cycles":57876608,"thread_start_us":217063,"threads_started":2,"wal-append.queue_time_us":1051587}
I20260709 12:24:48.786356 22876 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:48.849658 22940 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:48.868263 23113 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:48.926447 22940 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:48.911967 23111 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:48.868181 23114 raft_consensus.cc:2804] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:48.863114 23104 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:49.054307 16544 raft_consensus.cc:2468] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 22.
I20260709 12:24:48.800720 23037 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:49.027678 16551 raft_consensus.cc:1240] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:49.027891 22765 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.031068 22876 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:49.032269 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:49.061686 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:49.062282 22765 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:49.062810 23037 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:49.038604 23073 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:49.032609 23101 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:49.033263 23031 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:48.944388 23001 raft_consensus.cc:2804] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:48.930189 23119 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:49.030421 23094 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:49.076866 16477 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:48.783299 23103 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:49.079938 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:49.033443 22997 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:49.033437 23102 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:49.086398 23030 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:49.039599 23074 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.093232 23038 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:49.078641 23101 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:49.075824 23114 raft_consensus.cc:697] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:49.083096 23037 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:49.029965 23085 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:49.102115 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:49.104997 16442 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:49.110036 23103 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:49.075475 23111 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:49.097529 23025 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.132397 23037 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:49.137876 23025 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:49.111078 22848 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:49.095564 22765 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:49.057914 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 182997us
I20260709 12:24:49.124135 16514 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:49.127962 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:49.130353 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:49.144111 16477 raft_consensus.cc:2393] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 21: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:49.146749 16514 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:49.119122 23001 raft_consensus.cc:697] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 18 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:49.121076 23101 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:49.140126 23037 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:49.156381 16474 raft_consensus.cc:1240] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:49.160317 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:49.160732 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:49.141299 23111 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:49.106318 23119 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:49.128957 23038 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:49.162784 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:49.140211 23025 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:49.156100 22815 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:49.161437 23113 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.120493 23030 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:49.170670 16401 raft_consensus.cc:2393] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 19: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:49.120739 23074 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:49.171008 16544 raft_consensus.cc:1275] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:24:49.154120 23103 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:49.175315 22815 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:49.158578 23006 raft_consensus.cc:2804] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:49.176887 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:49.187811 16475 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 16.
I20260709 12:24:49.174733 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:49.177016 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:49.147279 22848 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:49.197830 22848 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:49.196203 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:49.199690 23074 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:49.202610 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:49.184224 23103 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:49.207160 23103 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:49.181099 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:49.177778 23038 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:49.159435 23088 raft_consensus.cc:493] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:24:49.193082 23126 log.cc:927] Time spent T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:24:49.194453 23030 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:49.191448 22815 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:49.214322 23088 raft_consensus.cc:515] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:49.216982 23088 leader_election.cc:290] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:49.190016 23006 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:49.225358 23006 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:49.181952 23001 consensus_queue.cc:237] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:49.235903 23038 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:49.224797 23115 log.cc:927] Time spent T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 12:24:49.236991 22997 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 24 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:49.223529 16469 raft_consensus.cc:1275] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:24:49.246160 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:49.252657 16401 raft_consensus.cc:1240] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:49.179960 23113 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:49.262188 16403 raft_consensus.cc:1240] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:49.263057 23094 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:49.263976 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:49.264739 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:49.277863 16396 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 28.
I20260709 12:24:49.265153 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:49.279394 16399 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 20.
I20260709 12:24:49.266340 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:49.266512 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:49.290697 16395 raft_consensus.cc:2393] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 18: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:49.183068 23114 consensus_queue.cc:237] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:49.322697 23097 log.cc:927] Time spent T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.973s	user 0.004s	sys 0.003s
I20260709 12:24:49.327365 23085 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:49.329770 16477 raft_consensus.cc:2393] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 24: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:49.337265 23113 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:49.282305 23102 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:49.328622 22815 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:49.329867 23104 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:49.349889 23113 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:24:49.375619 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 46852.7us
I20260709 12:24:49.376832 16367 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:49.378469 16367 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:49.384969 16395 raft_consensus.cc:1240] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:49.386655 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:49.387264 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:49.389031 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:49.377435 23113 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:49.400626 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:49.410316 23113 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:49.410434 23111 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:49.392838 23025 consensus_queue.cc:1048] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 12:24:49.414676 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:49.393713 22848 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:49.385540 23074 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:49.420640 17144 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:49.422449 17144 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:49.424161 16475 raft_consensus.cc:1240] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:49.424683 17144 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:49.426262 23034 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:24:49.426733 23034 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:49.393491 23110 consensus_queue.cc:1059] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 12:24:49.419807 22814 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:49.427124 23034 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:24:49.420662 22848 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:49.440497 22531 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:49.461270 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:49.432358 23031 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:49.429124 22885 consensus_queue.cc:1059] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 12:24:49.461385 22848 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:49.464311 22531 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:49.444275 22940 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:49.481133 23101 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:49.490573 23101 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:49.490787 16401 raft_consensus.cc:1240] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:49.491602 16473 raft_consensus.cc:2410] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:24:49.490598 22940 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:49.493851 23006 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:49.484660 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:49.465327 23119 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:49.465525 22886 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:49.499953 17261 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 1719) took 1273 ms. Trace:
I20260709 12:24:49.501927 22815 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:49.460033 22814 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:49.502385 23111 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:49.487897 22531 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:49.470779 23031 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:49.470345 23110 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:49.519586 23110 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:49.521687 23110 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:49.505084 17261 rpcz_store.cc:276] 0709 12:24:48.226991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.227575 (+   584us) service_pool.cc:224] Handling call
0709 12:24:48.259227 (+ 31652us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:48.259592 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:48.259619 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:48.270324 (+ 10705us) log.cc:844] Serialized 26 byte log entry
0709 12:24:48.353880 (+ 83556us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:48.380461 (+ 26581us) log.cc:844] Serialized 14 byte log entry
0709 12:24:48.533982 (+153521us) spinlock_profiling.cc:243] Waited 885 us on lock 0x7b1800403240. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d16c0a 00007f287c456219 00007f2872d9ea40 00007f2872d994e4 00007f2872d99455 00007f2875e81060 00007f2875e93716 00007f2875ea4287 00007f2875ea4208
0709 12:24:48.534157 (+   175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.534238 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:49.497851 (+963613us) spinlock_profiling.cc:243] Waited 1.88 ms on lock 0x7b1800403240. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:24:49.499744 (+  1893us) raft_consensus.cc:1685] finished
0709 12:24:49.499765 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:49.499892 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2808,"prepare.queue_time_us":28444,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":157029,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":159,"spinlock_wait_cycles":6360320,"thread_start_us":41381,"threads_started":1,"wal-append.queue_time_us":126360}
I20260709 12:24:49.518195 23031 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:49.518293 22814 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:49.516284 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:49.529099 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:49.524859 23006 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:49.534217 16442 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:49.535466 16442 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:49.535823 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:49.536506 23112 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:49.537286 23112 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:49.537549 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:49.538610 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:49.539052 16400 raft_consensus.cc:3060] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:49.541195 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:49.542088 22967 raft_consensus.cc:3055] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:49.542338 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:49.542552 22967 raft_consensus.cc:740] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:49.543382 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:49.543488 22967 consensus_queue.cc:260] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:49.544744 22531 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:49.545282 17263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 1700) took 2427 ms. Trace:
I20260709 12:24:49.545357 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:49.530133 23105 log.cc:927] Time spent T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.882s	user 0.001s	sys 0.001s
I20260709 12:24:49.532116 22946 raft_consensus.cc:993] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1: : Instructing follower 52ab50b0237346c1a13ede947f6a63c2 to start an election
I20260709 12:24:49.547843 17263 rpcz_store.cc:276] 0709 12:24:47.118351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.118546 (+   195us) service_pool.cc:224] Handling call
0709 12:24:47.831809 (+713263us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:47.832479 (+   670us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:47.832525 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:48.861733 (+1029208us) spinlock_profiling.cc:243] Waited 980 us on lock 0x7b2c00000708. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2874782a11 00007f2874792372 00007f2875ec4a01 00007f2875e8943a 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2
0709 12:24:48.862188 (+   455us) log.cc:844] Serialized 26 byte log entry
0709 12:24:48.984881 (+122693us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:48.984942 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.985056 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:49.215029 (+229973us) raft_consensus.cc:1685] finished
0709 12:24:49.215049 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:49.545101 (+330052us) spinlock_profiling.cc:243] Waited 704 ms on lock 0x7b540005df50. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e888ad 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:49.545207 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1301,"raft.queue_time_us":423,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":5747,"spinlock_wait_cycles":1622035968,"thread_start_us":122184,"threads_started":1,"wal-append.queue_time_us":216149}
I20260709 12:24:49.552860 16549 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37226 (request call id 1891) took 1311 ms. Trace:
I20260709 12:24:49.554659 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:49.553287 16549 rpcz_store.cc:276] 0709 12:24:48.241648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.241856 (+   208us) service_pool.cc:224] Handling call
0709 12:24:48.242191 (+   335us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:48.259478 (+ 17287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:48.259509 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:48.269126 (+  9617us) log.cc:844] Serialized 52 byte log entry
0709 12:24:48.520558 (+251432us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:48.529194 (+  8636us) log.cc:844] Serialized 14 byte log entry
0709 12:24:48.530710 (+  1516us) log.cc:844] Serialized 14 byte log entry
0709 12:24:48.531200 (+   490us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.531357 (+   157us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:49.529499 (+998142us) spinlock_profiling.cc:243] Waited 7.25 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:24:49.552656 (+ 23157us) raft_consensus.cc:1685] finished
0709 12:24:49.552680 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:49.552802 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16846,"prepare.queue_time_us":8031,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":67,"raft.queue_time_us":22147,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":237,"spinlock_wait_cycles":16663936,"thread_start_us":199407,"threads_started":1,"wal-append.queue_time_us":309645}
I20260709 12:24:49.558094 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:49.558348 16442 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:49.559494 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:49.560966 16442 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:49.566125 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:49.567229 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:49.566769 16394 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 19.
I20260709 12:24:49.569386 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:49.569684 16442 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:49.576094 22848 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:49.577173 16401 raft_consensus.cc:2393] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 20: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:49.578733 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:49.579176 16399 raft_consensus.cc:1240] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:24:49.579348 16401 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 22.
I20260709 12:24:49.580389 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:49.586858 16550 raft_consensus.cc:1275] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:24:49.587651 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:49.588668 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:49.588984 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:49.566975 23123 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:49.589830 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a79e67586f842a7ba4c25dfd7cdf4d5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:49.588796 23133 log.cc:927] Time spent T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 12:24:49.592504 22967 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:49.594398 22531 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:49.594928 16396 raft_consensus.cc:1240] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:49.595568 22848 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:49.597651 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:49.560812 22946 raft_consensus.cc:1081] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 13 LEADER]: Signalling peer 52ab50b0237346c1a13ede947f6a63c2 to start an election
I20260709 12:24:49.598714 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:49.599442 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:49.605381 16400 raft_consensus.cc:2468] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 21.
I20260709 12:24:49.605672 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:49.607398 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:49.607512 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:49.600184 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:49.580395 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:49.608928 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:49.598749 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:49.601943 16514 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.58.3:33107 timed out after 1.727s (SENT) [suppressed 3 similar messages]
I20260709 12:24:49.602787 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:49.610497 16542 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:49.601897 16442 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:49.612295 16442 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:49.613425 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:49.613651 17271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45228 (request call id 1708) took 1718 ms. Trace:
I20260709 12:24:49.613955 16549 raft_consensus.cc:1240] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:49.614173 17271 rpcz_store.cc:276] 0709 12:24:47.895479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.895733 (+   254us) service_pool.cc:224] Handling call
0709 12:24:47.896155 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:49.233658 (+1337503us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:49.233700 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:49.233754 (+    54us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:49.233795 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.613339 (+379544us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b5400040bd4. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:49.613413 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:49.613573 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3075294464}
I20260709 12:24:49.616194 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:49.616963 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:49.617558 16547 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 24.
I20260709 12:24:49.621809 16547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:49.623122 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:49.623353 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:49.624073 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:49.588212 23119 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:49.565047 23114 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:49.633117 17254 raft_consensus.cc:2393] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 21: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:49.635547 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:49.639717 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:49.645298 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:49.647017 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:49.650924 16542 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 18.
I20260709 12:24:49.656752 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:49.659628 16401 raft_consensus.cc:2393] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 21: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:49.640477 22747 consensus_queue.cc:1059] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 12:24:49.661820 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:49.662860 16475 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:24:49.662066 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:49.654929 23013 consensus_queue.cc:1059] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 12:24:49.651290 16367 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:49.630324 23026 consensus_queue.cc:1059] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 12:24:49.624673 23112 raft_consensus.cc:2804] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:49.620138 23123 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:49.667317 16367 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:49.665735 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:49.701387 16367 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:49.698668 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:49.672456  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:49.666685 23025 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:24:49.616073 22531 consensus_queue.cc:1059] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 12:24:49.630407 23119 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:49.707446 16441 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:49.710407 16367 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:49.721005 16441 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:49.724629 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:49.727025 16441 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:49.732600 22842 consensus_queue.cc:1048] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:49.734457 17270 raft_consensus.cc:1240] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:49.735239 17270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45224 (request call id 1576) took 1012 ms. Trace:
I20260709 12:24:49.737017 23039 raft_consensus.cc:3055] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:49.737468 23039 raft_consensus.cc:740] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:49.738270 23039 consensus_queue.cc:260] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:49.739292 23039 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:49.739754 16551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52680 (request call id 1558) took 1450 ms. Trace:
I20260709 12:24:49.743259 16366 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:49.744177 23102 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:49.744753 23102 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:49.745175 23102 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:49.747730 23030 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:49.700289 23123 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:49.735667 17270 rpcz_store.cc:276] 0709 12:24:48.722691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.722870 (+   179us) service_pool.cc:224] Handling call
0709 12:24:48.723185 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:49.734264 (+1011079us) spinlock_profiling.cc:243] Waited 3.34 ms on lock 0x7b54008bde20. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e00dc0 00007f2875e8b1c0 00007f2875e8c231 00007f2875e88d8f 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c
0709 12:24:49.734953 (+   689us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.735171 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7674624}
I20260709 12:24:49.731444 16367 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:49.751003 16441 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:49.708333 23074 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:49.755662 16441 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:49.740145 16551 rpcz_store.cc:276] 0709 12:24:48.289885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.306426 (+ 16541us) service_pool.cc:224] Handling call
0709 12:24:48.306745 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:49.060446 (+753701us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.739523 (+679077us) spinlock_profiling.cc:243] Waited 599 ms on lock 0x7b54005e0954. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e89e01 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c 00007f28743c3cc2 00007f28741c448c 00007f28741c38c6
0709 12:24:49.739695 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1378842496}
I20260709 12:24:49.757318 16441 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:49.709864 23112 raft_consensus.cc:697] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:49.716791 23119 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:49.713653 23025 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:49.765707 16366 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
W20260709 12:24:49.790814 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 59477us
W20260709 12:24:49.794085 17142 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.794694 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:49.795185 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:49.751785 23078 raft_consensus.cc:3055] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:49.698840 23034 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:49.758602 22553 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:49.804786 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:49.785360 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:49.724882 23114 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:49.793079 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:49.764111 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:49.753294 23111 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:49.760635 23123 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:49.799687 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:49.759624 23089 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:49.758431 23101 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:49.765709 23093 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:49.831439 17264 raft_consensus.cc:1240] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 29 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 28. Current term is 29. Ops: []
I20260709 12:24:49.803833 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:49.832302 16394 raft_consensus.cc:2393] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 22: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:49.727876 23103 consensus_queue.cc:1059] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.077s
I20260709 12:24:49.827711 22607 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.829070 17142 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:49.785009 23038 consensus_queue.cc:1048] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 12:24:49.857390 17142 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
W20260709 12:24:49.859596 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 257672us
I20260709 12:24:49.859819 17142 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:49.760816 16441 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:49.832818 16402 raft_consensus.cc:2376] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 21. Re-sending same reply.
W20260709 12:24:49.863765 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 103040us
I20260709 12:24:49.865710 16441 leader_election.cc:304] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:49.801875 23030 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:49.800412 23078 raft_consensus.cc:740] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:49.827591 22886 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:49.861884 23039 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:49.871518 22927 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:49.872294 23101 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:49.874696 16441 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:49.810909 22553 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:49.881384 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:49.883579 16441 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:49.889542 16441 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:49.874459 23013 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:49.858866 22607 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:49.880368 23093 raft_consensus.cc:2749] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:49.893867 16366 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:49.891638 16441 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:49.883388 22553 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:49.883509 22886 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:49.903411 22607 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:49.885213 23030 raft_consensus.cc:697] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 18 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:49.905112 23030 consensus_queue.cc:237] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:49.885640 23104 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:49.909004 23104 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:49.885521 23113 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:49.885213 23114 raft_consensus.cc:2749] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:49.910341 16366 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:49.917837 16366 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:49.897512 23101 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:49.914783 23042 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:49.914209 23113 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:49.920590 23038 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:24:49.978281 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 251372us
I20260709 12:24:49.881162 23112 consensus_queue.cc:237] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:49.982916 17143 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:49.926467 22940 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:24:49.990406 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 98840.4us
I20260709 12:24:49.991182 16441 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:49.930358 23078 consensus_queue.cc:260] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:49.992771 23078 raft_consensus.cc:3060] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:49.994472 23139 log.cc:927] Time spent T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:24:49.931643 23042 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:49.977932 23113 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:50.010308 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:50.000742 16441 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:50.001077 17269 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 19.
W20260709 12:24:50.002619 23142 log.cc:927] Time spent T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:24:50.008322 23103 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:50.018538 16367 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:50.015509 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:49.990236 23019 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:50.031675 16367 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:50.028225 16551 raft_consensus.cc:1240] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:50.030289 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:50.030575 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:50.030309 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:50.033381 17258 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 24.
I20260709 12:24:50.034003 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:50.034451 16542 raft_consensus.cc:1240] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 18 LEADER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:50.034590 16476 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:50.036814 23074 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:24:50.010475 23042 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:50.024992 23137 log.cc:927] Time spent T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
I20260709 12:24:50.042266 16550 raft_consensus.cc:1275] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:24:50.035853 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:49.989866 22616 consensus_queue.cc:1059] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 12:24:50.028838 23093 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:50.043857 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:50.045567 16542 raft_consensus.cc:1275] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 12:24:50.046430 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 162285us
I20260709 12:24:50.042516 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:50.037500 23038 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:50.084864 16366 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:50.037642 23034 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 29 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:50.087952 16441 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:50.097054 23114 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:50.100062 16441 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:50.101780 16441 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:50.104506 16476 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 18.
I20260709 12:24:50.105682 23114 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:50.108156 23093 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:24:50.108996 23145 log.cc:927] Time spent T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:24:50.107678 16441 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:50.048709 22927 raft_consensus.cc:697] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 16 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:50.041631 23102 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:50.103609 23130 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:50.095198 22531 consensus_queue.cc:1059] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 12:24:50.147487 16471 raft_consensus.cc:1240] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:24:50.117388 23093 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:50.100203 23103 consensus_queue.cc:1048] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
W20260709 12:24:50.154033 23143 log.cc:927] Time spent T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.245s	user 0.001s	sys 0.000s
I20260709 12:24:50.154644 22553 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:50.155862 23144 log.cc:927] Time spent T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:24:50.156032 22531 raft_consensus.cc:3055] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:50.163369 16514 leader_election.cc:400] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 21. Current term is 22.
I20260709 12:24:50.163719 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:50.164942 16400 raft_consensus.cc:1240] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:24:50.166607 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:50.177189 16514 leader_election.cc:403] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:50.167778 16396 raft_consensus.cc:1240] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:50.168722 16477 raft_consensus.cc:3055] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:50.102695 23150 consensus_queue.cc:1059] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
W20260709 12:24:50.172390 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 85201us
W20260709 12:24:50.176685 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 90163us
I20260709 12:24:50.178143 16477 raft_consensus.cc:740] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:50.183466 16441 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:50.181397 16514 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:50.176787 22531 raft_consensus.cc:740] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:50.182654 23028 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 21. Current term is 22.
I20260709 12:24:50.193073 22531 consensus_queue.cc:260] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:50.194621 22531 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:50.190339 23093 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:50.191227 22887 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:50.196269 23093 consensus_queue.cc:1059] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 28.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:24:50.196897 22887 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.197304 22887 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:50.187898 23150 consensus_queue.cc:1243] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer ef57beab0e1d40d585983873c55107e8 log is divergent from this leader: its last log entry 17.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:24:50.201529 22927 consensus_queue.cc:237] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:50.198628 23025 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:50.230281 23078 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:50.230736 22553 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 23 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:50.231112 23078 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:50.233399 23078 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:50.235549 23101 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:50.236392 23013 consensus_queue.cc:1048] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:50.236773 23114 raft_consensus.cc:3055] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:24:50.237272 23114 raft_consensus.cc:740] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:50.225833 23026 consensus_queue.cc:1048] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 12:24:50.238600 23114 consensus_queue.cc:260] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:50.239939 23150 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:50.241046 16514 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:50.241206 23042 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:50.243161 23119 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:50.245307 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 62604.1us
W20260709 12:24:50.245719 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 63152.5us
I20260709 12:24:50.270569 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:50.253271 23102 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:50.273242 23042 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:50.274283 23119 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:50.275763 23123 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:50.277976 23123 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:50.267544 23150 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:50.283386 23150 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:50.270666 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 82360.7us
I20260709 12:24:50.287103 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:50.291774 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:50.268555 23006 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:50.294159 22887 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:50.291234 23042 consensus_queue.cc:1059] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 12:24:50.297272 22887 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:50.297319 23156 raft_consensus.cc:3055] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:50.297945 23156 raft_consensus.cc:740] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:50.298841 23156 consensus_queue.cc:260] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:50.300361 23156 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:50.315307 16402 raft_consensus.cc:1240] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:50.279027 23113 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:50.318540 23113 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 25 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:50.321384 17144 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:50.322269 23026 consensus_queue.cc:1059] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:50.329146 23157 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:50.329679 16402 raft_consensus.cc:1275] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:24:50.331116 17142 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:50.332681 23113 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:50.333390 23113 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:50.335521 16402 raft_consensus.cc:1275] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 12:24:50.335943 16402 pending_rounds.cc:85] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 9
I20260709 12:24:50.336236 16402 pending_rounds.cc:107] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:24:50.336395 23113 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:50.337503 16402 raft_consensus.cc:2889] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:50.337960 23162 consensus_queue.cc:1059] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:50.336628 22814 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:50.338908 23162 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:50.338955 22814 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:50.339565 23162 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:50.342633 22814 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:50.342738 23162 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:50.352900 23078 raft_consensus.cc:3055] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:50.353319 23078 raft_consensus.cc:740] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:50.354110 23078 consensus_queue.cc:260] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:50.355276 23078 raft_consensus.cc:3060] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:24:50.356923 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.358667 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:50.359447 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:50.360409 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:50.361425 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:50.362005 16546 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:50.363305 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:50.323457 23026 consensus_queue.cc:1059] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:50.374045 16470 raft_consensus.cc:2393] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 23: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
W20260709 12:24:50.371450 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 47911.3us
I20260709 12:24:50.377135 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:50.381508 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:50.377199 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:50.383905 22885 raft_consensus.cc:3055] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:50.384295 22885 raft_consensus.cc:740] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:50.385061 22885 consensus_queue.cc:260] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:50.386231 22885 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:50.388515 22902 raft_consensus.cc:3055] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:50.388958 22902 raft_consensus.cc:740] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:24:50.389469 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 50645.8us
I20260709 12:24:50.389823 22902 consensus_queue.cc:260] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:50.391705 22842 consensus_queue.cc:1048] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:50.394467 16366 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:50.388043 16546 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 25.
I20260709 12:24:50.397244 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
W20260709 12:24:50.476298 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 81015.2us
W20260709 12:24:50.477048 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 79891.9us
I20260709 12:24:50.392094 23155 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:50.377589 23039 raft_consensus.cc:3055] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:50.495060 23039 raft_consensus.cc:740] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:50.476969 23042 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:50.497282 23110 raft_consensus.cc:3055] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:50.497748 23110 raft_consensus.cc:740] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:50.498546 23110 consensus_queue.cc:260] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:50.499789 23110 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:50.502310 22886 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:50.388532 22967 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:50.373500 22814 consensus_queue.cc:1059] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.115s
I20260709 12:24:50.376607 23006 consensus_queue.cc:1048] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.132s
I20260709 12:24:50.484154 23104 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:24:50.523954 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:24:50.507496 23148 consensus_queue.cc:1175] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f99066d13e524238aec6aa2024a8bd50" responder_term: 15 status { last_received { term: 15 index: 10 } last_committed_idx: 10 last_received_current_leader { term: 15 index: 10 } }
I20260709 12:24:50.537722 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:50.542289 16472 raft_consensus.cc:1275] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:24:50.496104 23039 consensus_queue.cc:260] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:50.508549 23155 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:50.548851 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:50.549764 17254 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 22.
I20260709 12:24:50.549985 23155 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:50.532297 23104 raft_consensus.cc:697] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 25 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:50.561795 23104 consensus_queue.cc:237] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:50.579995 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 210641us
I20260709 12:24:50.582600 17266 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 24.
I20260709 12:24:50.606139 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:50.583254 23130 consensus_queue.cc:1048] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 12:24:50.615970 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:50.618309 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:50.618587 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:50.622009 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:50.622615 16470 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:50.628851 16472 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:34826
I20260709 12:24:50.629755 16472 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:50.630198 16472 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:24:50.635463 23166 log.cc:927] Time spent T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:24:50.642649 16470 raft_consensus.cc:2468] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 17.
I20260709 12:24:50.644709 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:50.644909 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:50.646452 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aea54881468a40aaabad162d385fc6d9" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:50.652109 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.655216 17255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:50.655563 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:50.656033 17271 raft_consensus.cc:1275] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:24:50.657384 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:50.656289 17255 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 21.
I20260709 12:24:50.668874 17263 raft_consensus.cc:2468] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
I20260709 12:24:50.671531 17254 raft_consensus.cc:1275] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:24:50.672587 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:50.678172 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:50.679021 17261 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:50.680441 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:50.646037 23146 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
W20260709 12:24:50.695086 23165 log.cc:927] Time spent T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:24:50.702751 17261 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 25.
I20260709 12:24:50.706276 17258 raft_consensus.cc:1275] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:24:50.707432 23019 raft_consensus.cc:3055] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:50.707872 23019 raft_consensus.cc:740] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:50.709076 23019 consensus_queue.cc:260] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:50.710383 23019 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:50.691493 16472 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:50.716223 16472 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:50.714344 16441 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:50.717994 16441 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:50.714324 23111 consensus_queue.cc:1048] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:50.720911 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: true dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:50.721520 16548 raft_consensus.cc:3055] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:50.721807 16548 raft_consensus.cc:740] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:50.717701 16442 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:50.726316 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:50.728478 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:50.729739 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:50.730139 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:50.730967 16395 raft_consensus.cc:2410] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:24:50.722565 16548 consensus_queue.cc:260] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:50.738101 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: true dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:50.740167 16403 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:50.741102 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:50.742753 16400 raft_consensus.cc:1275] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:24:50.719492 23103 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:50.732025 23093 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:50.753093 23093 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:50.753687 23093 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:50.756675 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:50.762490 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:50.789328 16515 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:50.790652 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:50.794513 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:50.731694 23112 consensus_queue.cc:1048] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.073s
I20260709 12:24:50.811710 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
W20260709 12:24:50.796574 23166 log.cc:927] Time spent T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:24:50.812644 16515 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:50.820292 16400 raft_consensus.cc:1240] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:50.810853 16477 consensus_queue.cc:260] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:50.720104 23146 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:50.798643 23030 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:50.803637 23119 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:50.829536 16515 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:50.865067 16515 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:50.811246 23028 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:50.869325 17269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45230 (request call id 1575) took 1270 ms. Trace:
I20260709 12:24:50.870466 16515 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:50.869750 17269 rpcz_store.cc:276] 0709 12:24:49.599264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.599886 (+   622us) service_pool.cc:224] Handling call
0709 12:24:50.869166 (+1269280us) spinlock_profiling.cc:243] Waited 277 ms on lock 0x7b540055f354. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:50.869269 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":637338880}
I20260709 12:24:50.811082 23161 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:50.860986 22778 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:24:50.889863 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 63979.5us
I20260709 12:24:50.871968 22788 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:50.871688 22392 raft_consensus.cc:2764] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 25 FOLLOWER]: Leader election decision vote started in defunct term 24: won
W20260709 12:24:50.901557 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 72139.4us
I20260709 12:24:50.893044 23028 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:50.873087 23119 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:50.919713 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 49365.1us
I20260709 12:24:50.920521 17142 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:50.866386 23030 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:50.920560 16515 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
W20260709 12:24:50.901837 23078 consensus_meta.cc:287] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8: Time spent flushing consensus metadata: real 0.546s	user 0.013s	sys 0.001s
I20260709 12:24:50.923573 16367 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:50.900473 22788 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:50.930832 16442 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:50.932271 16367 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:50.935263 16442 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:50.941128 16442 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:50.941798 17142 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:50.944541 17264 raft_consensus.cc:2468] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 23.
I20260709 12:24:50.929301 22788 raft_consensus.cc:3060] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:50.929716 22553 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:50.935091 22778 raft_consensus.cc:2804] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:50.934914 23123 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:24:50.966509 23186 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:50.975044 17142 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:50.949288 23078 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:24:50.977886 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 46322.4us
I20260709 12:24:50.955879 23113 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:50.956002 22607 consensus_queue.cc:1048] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:24:50.989894 22747 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:24:50.990161 23179 log.cc:927] Time spent T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 12:24:50.994871 17145 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.996410 23119 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:50.977792 22778 raft_consensus.cc:697] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 13 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:51.001946 22778 consensus_queue.cc:237] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:51.004271 17143 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.965279 23110 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:51.009742 16403 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
W20260709 12:24:51.011952 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 69593.5us
I20260709 12:24:50.976366 23183 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:24:51.045984 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 56613.2us
I20260709 12:24:51.046324 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:51.052281 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:51.062232 22902 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:51.063611 22788 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:51.066272 23030 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:51.067215 22788 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 22 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:51.068840 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:51.072134 23111 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:51.072724 23111 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:51.074765 23111 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:51.010298 23123 consensus_queue.cc:1048] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 12:24:51.075999 23039 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:51.077138 16473 raft_consensus.cc:1240] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:51.080371 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:51.082630 17258 raft_consensus.cc:2468] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 22.
I20260709 12:24:51.085824 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:51.094913 16400 raft_consensus.cc:2393] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 22: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:51.102762 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:51.104709 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:51.052204 23146 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:51.107184 16542 raft_consensus.cc:2393] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 19: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:51.105465 17254 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 19.
I20260709 12:24:51.102061 16515 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:51.133724 16441 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
W20260709 12:24:51.137087 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 163298us
I20260709 12:24:51.138722 16442 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:51.100626 23093 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:51.151782 16443 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.153610 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:51.153771 16367 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:51.159467 16442 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:51.161253 22946 raft_consensus.cc:3055] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:51.172879 22946 raft_consensus.cc:740] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:51.173134 22939 raft_consensus.cc:2804] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:51.171360 23186 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:51.173839 23093 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:51.174162 23186 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:51.176465 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 74638.9us
I20260709 12:24:51.171813 23188 raft_consensus.cc:2804] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:51.158341 23123 raft_consensus.cc:2804] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:51.150888 23183 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:51.173772 22939 raft_consensus.cc:697] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 22 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:24:51.182603 23173 log.cc:927] Time spent T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.421s	user 0.004s	sys 0.000s
I20260709 12:24:51.194794 16542 raft_consensus.cc:1275] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 12:24:51.195303 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 93312.9us
I20260709 12:24:51.183480 23183 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:51.185544 23123 raft_consensus.cc:697] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:51.186858 23188 raft_consensus.cc:697] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 23 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:51.185050 23093 raft_consensus.cc:2749] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:51.202991 23183 raft_consensus.cc:3060] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:51.199028 23042 consensus_queue.cc:1048] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:24:51.210843 23030 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:51.224987 22939 consensus_queue.cc:237] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:51.242298 23188 consensus_queue.cc:237] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:51.245240 23030 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:51.254492 23114 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:24:51.254969 16400 raft_consensus.cc:1240] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:51.258190 23198 consensus_queue.cc:1059] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:51.260650 23123 consensus_queue.cc:237] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:51.224233 23195 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:51.269806 23195 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:51.300624 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:51.270938 23030 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:51.295208 23186 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:51.324075 23194 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:51.336755 23194 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:51.339207 23194 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:51.340484 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:51.341251 16395 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 18.
I20260709 12:24:51.342620 16442 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:51.343824 23195 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:51.345577 23194 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:51.346108 23194 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:51.346546 23194 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:51.347852 23183 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:51.350251 23183 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:24:51.351090 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 218499us
I20260709 12:24:51.352981 22747 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.364593 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
W20260709 12:24:51.365279 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 261741us
I20260709 12:24:51.366060 16542 raft_consensus.cc:1240] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:51.355666 22747 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:51.369184 22747 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:24:51.372160 23206 log.cc:927] Time spent T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:24:51.375900 16474 raft_consensus.cc:1240] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:51.378347 17143 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:51.382498 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:51.401579 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:51.408174 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:51.409770 23039 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:51.410580 23039 consensus_queue.cc:1059] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:51.411391 22927 raft_consensus.cc:3055] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:51.411794 22927 raft_consensus.cc:740] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:51.413937 16548 raft_consensus.cc:3060] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:51.413169 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:51.416199 16542 raft_consensus.cc:2393] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 19: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:51.417726 16514 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:51.422712 23164 log.cc:927] Time spent T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.808s	user 0.001s	sys 0.003s
I20260709 12:24:51.424223 22814 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:51.427520 16548 raft_consensus.cc:2468] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 14.
W20260709 12:24:51.428229 23202 log.cc:927] Time spent T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 12:24:51.431689 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:51.432266 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:51.434976 16475 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 29.
I20260709 12:24:51.436246 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:51.436106 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:51.435619 16514 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:24:51.437279 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:51.451433 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
W20260709 12:24:51.456746 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 56955.1us
I20260709 12:24:51.457304 22927 consensus_queue.cc:260] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:51.458784 22927 raft_consensus.cc:3060] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:51.459597 23194 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:51.461799 23034 consensus_queue.cc:1059] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:51.462888 23014 raft_consensus.cc:3055] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:51.463271 23014 raft_consensus.cc:740] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:51.463389 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:51.464025 23014 consensus_queue.cc:260] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:51.468503 23148 consensus_queue.cc:1175] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "610f4feda11942739236f2b121a4b9c1" responder_term: 17 status { last_received { term: 15 index: 9 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 16. Current term is 17. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:24:51.472174 23194 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:51.474715 16473 raft_consensus.cc:2393] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 19: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
W20260709 12:24:51.475499 16516 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.478792 16367 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:51.479601 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 58867.8us
I20260709 12:24:51.480624 16514 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:51.486636 16441 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:51.486867 17143 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:51.485997 23028 raft_consensus.cc:2804] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:51.487178 23042 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:51.485975 23014 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:51.488651 23197 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:51.486610 23114 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:51.497606 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:51.496757 23028 raft_consensus.cc:697] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 17 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:51.516832 17254 raft_consensus.cc:2468] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 19.
I20260709 12:24:51.497377 23187 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:24:51.496994 23013 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:51.545874 16441 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:51.546859 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:24:51.547398 23216 log.cc:927] Time spent T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:24:51.554416 16471 raft_consensus.cc:1240] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:51.510070 23114 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:51.572665 22553 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:51.570832 23034 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.512234 23028 consensus_queue.cc:237] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:51.571225 23042 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.610301 23042 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:51.601486 23034 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:51.600147 22553 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:51.618033 23034 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:51.607841 23114 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:51.621978 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:51.622160 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:51.622923 16396 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 20.
W20260709 12:24:51.622943 23212 log.cc:927] Time spent T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 12:24:51.624256 17144 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:51.625610 23157 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:51.626073 23157 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.626639 23157 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:51.570660 23194 raft_consensus.cc:2804] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:51.620195 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:51.629962 23194 raft_consensus.cc:697] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:51.630003 16471 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 20.
I20260709 12:24:51.630342 22553 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:51.631207 23194 consensus_queue.cc:237] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:51.642005 16471 raft_consensus.cc:1240] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:51.638628 23184 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:51.672646 23072 consensus_queue.cc:1059] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:24:51.673662 23033 raft_consensus.cc:3055] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:51.674023 23033 raft_consensus.cc:740] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:51.674773 23033 consensus_queue.cc:260] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:51.675362 22553 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.675968 23033 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:51.675889 22553 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:51.678323 22553 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:51.683378 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:51.684408 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:51.696118 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:51.719341 23185 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:51.707279 23184 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:51.724597 23185 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:51.738565 23184 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:51.739643 23185 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:51.741055 23157 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:51.744107 23157 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:51.744450 23042 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:51.788861 16366 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:51.791636 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 86527.3us
I20260709 12:24:51.797780 23042 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:51.798985 16366 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:51.800874 22940 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:51.806998 22940 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:49.703613  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:51.853354 16221 catalog_manager.cc:5697] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 11 to 13, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 13 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
I20260709 12:24:51.855461 16221 catalog_manager.cc:5697] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:24:51.874130 23104 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:51.874889 23104 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:51.876135 23155 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:51.876705 23155 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:51.877806 23104 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:51.878894 23155 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:51.942411 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:51.942531 17263 raft_consensus.cc:1275] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:24:51.943265 17258 raft_consensus.cc:2393] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 29: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:51.944254 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:51.945024 17264 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 19.
I20260709 12:24:51.945941 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:51.946765 17254 raft_consensus.cc:2410] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 23 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:24:51.955917 23104 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:51.956636 23104 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:51.958655 23104 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:51.959790 23104 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.960539 23104 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:51.965942 22940 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:51.990633 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:51.990509 22940 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:51.991322 16398 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
I20260709 12:24:51.992781 22940 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:51.995464 23104 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:51.996466 16367 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:51.969518 16474 raft_consensus.cc:1240] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:51.978901 17264 raft_consensus.cc:1275] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:24:51.983604 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:51.998347 17263 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 19.
W20260709 12:24:51.998534 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 282869us
I20260709 12:24:51.986126 23042 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.999521 16367 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:51.999449 23042 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:52.000084 22553 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:24:52.001209 22885 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.001698 22885 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:52.002074 23042 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:52.002719 23042 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:52.003108 23042 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.003520 23042 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:51.971686 16442 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:51.997092 22940 consensus_queue.cc:1048] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:51.984997 17254 raft_consensus.cc:1275] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:24:52.011247 23104 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:52.016252 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:52.019173 23151 consensus_queue.cc:1059] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.015s
I20260709 12:24:52.016680 23203 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:52.035461 23203 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:52.036021 22885 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:52.037614 16475 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53186 (request call id 1527) took 1279 ms. Trace:
I20260709 12:24:52.037856 23203 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:52.037983 16475 rpcz_store.cc:276] 0709 12:24:50.758481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.770148 (+ 11667us) service_pool.cc:224] Handling call
0709 12:24:52.037458 (+1267310us) spinlock_profiling.cc:243] Waited 604 ms on lock 0x7b54002f3854. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e8d2fa 00007f287ae4eb17 00007f28743c96d4 00007f28743c9662 00007f28743c9591 00007f28743c950c 00007f28743c87c2 00007f28741c448c 00007f28741c38c6 00007f28741c326f
0709 12:24:52.037542 (+    84us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1390078336}
I20260709 12:24:52.039789 22531 raft_consensus.cc:3055] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:52.040217 22531 raft_consensus.cc:740] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:52.041144 22531 consensus_queue.cc:260] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:52.042644 22531 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:52.043151 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:52.043828 16546 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 16.
I20260709 12:24:52.045071 17143 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:52.045665 17142 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:52.046403 22885 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:52.048043 16472 raft_consensus.cc:1240] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:52.048350 22885 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.048780 22885 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:52.051623 23155 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 29 FOLLOWER]: Leader election won for term 29
W20260709 12:24:52.055665 16515 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.052292 23155 raft_consensus.cc:697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 29 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:52.062673 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:52.056819 23151 consensus_queue.cc:1059] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 12:24:52.063184 16395 raft_consensus.cc:1275] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:24:52.064320 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:52.065307 16403 raft_consensus.cc:2393] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 17: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:52.057123 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:52.066917 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:52.067723 16398 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 18.
I20260709 12:24:52.069057 23155 consensus_queue.cc:237] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:52.073730 16402 raft_consensus.cc:1240] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:52.074717 22553 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:24:52.056874 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 57444.1us
I20260709 12:24:52.059798 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:52.076577 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:52.077248 16403 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:52.076794 16468 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:52.081619 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:52.085556 16515 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:52.090449 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:52.098351 16403 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 21.
I20260709 12:24:52.104261 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:52.105062 17254 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:24:52.105969 16468 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 21.
I20260709 12:24:52.103420 16442 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:52.108054 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:52.108489 16367 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:52.109827 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:52.110368 16551 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 18.
W20260709 12:24:52.111160 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.111817 16403 raft_consensus.cc:1240] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:52.111922 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:52.113610 17144 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:52.106922 22998 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:52.115202 16516 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:52.118170 17142 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:52.117779 22842 consensus_queue.cc:1059] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 12:24:52.121539 23113 raft_consensus.cc:493] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:52.122056 23113 raft_consensus.cc:515] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:52.123767 23028 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:52.111121 23183 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:52.124215 23028 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:52.124581 23183 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:52.125073 23183 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:52.125278 22946 consensus_queue.cc:260] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:52.124643 23028 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:52.109522 22940 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.111567 23146 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.124060 23113 leader_election.cc:290] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:52.131484 23114 raft_consensus.cc:493] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:52.133401 22553 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:52.133668 23114 raft_consensus.cc:515] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:52.116816 22392 consensus_queue.cc:1048] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 12:24:52.129123 22747 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:52.132225 23218 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:52.158064 23218 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:52.138067 22946 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:52.167553 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 59145.3us
I20260709 12:24:52.132773 22927 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:52.132741 23146 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:52.176286 22927 raft_consensus.cc:697] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 21 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:52.177227 22927 consensus_queue.cc:237] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:52.177649 23146 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:52.179736 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:52.180794 16551 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 19.
I20260709 12:24:52.182425 17143 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:52.136297 23114 leader_election.cc:290] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:52.188489 22967 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:52.189052 22967 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.189378 22967 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:52.193619 22885 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:52.196127 22885 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:52.197237 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:52.197948 16551 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:24:52.174885 23042 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:52.199674 23042 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:52.200850 16400 raft_consensus.cc:2368] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 19. Current term is 20.
I20260709 12:24:52.132390 22940 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:52.203162 22940 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:52.203889 16401 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 17.
I20260709 12:24:52.204267 16403 raft_consensus.cc:1240] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:52.204994 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:52.205461 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:52.205626 16396 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 19.
I20260709 12:24:52.206421 22885 consensus_queue.cc:1059] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:52.173172 23218 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:52.216039 23183 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:52.219362 16472 raft_consensus.cc:1240] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:52.222260 23183 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:52.230289 23183 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:52.200389 23028 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:52.449676 23028 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:52.233212 16551 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 18.
W20260709 12:24:52.242986 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 125686us
I20260709 12:24:52.452505 16515 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:52.454797 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:52.243360 16476 raft_consensus.cc:1275] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 12:24:52.243531 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 128351us
I20260709 12:24:52.244374 23230 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.246850 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:52.456894 16475 raft_consensus.cc:2393] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 18: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:52.456640 23230 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:52.458748 23230 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:52.248826 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:52.250021 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:52.250620 16403 raft_consensus.cc:1275] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:24:52.250844 16474 raft_consensus.cc:3060] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:52.252192 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:52.464939 16402 raft_consensus.cc:2393] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 17: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:52.252794 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:52.254281 23148 raft_consensus.cc:3055] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:52.467290 23148 raft_consensus.cc:740] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:52.468222 23148 consensus_queue.cc:260] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:52.469504 23148 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:52.470103 16441 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:52.471679 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:52.252516 16367 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:52.472430 16550 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 18.
W20260709 12:24:52.472738 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 220320us
I20260709 12:24:52.255455 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:52.475334 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:52.476115 16473 raft_consensus.cc:2393] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 16: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:52.255674 16468 raft_consensus.cc:1275] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:24:52.310066 22967 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:52.477610 23223 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:52.478055 23223 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:52.478416 23223 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:52.478588 22967 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:52.327368 23159 raft_consensus.cc:493] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:52.335685 16477 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:52.478856 23006 consensus_queue.cc:1048] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:24:52.372314 17261 raft_consensus.cc:1275] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:24:52.373634 17269 raft_consensus.cc:1275] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 25 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:24:52.480406 23006 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:52.256609 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:52.481326 22885 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.481472 16471 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 18.
I20260709 12:24:52.481927 22885 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:52.481508 23072 consensus_queue.cc:1048] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:52.377409 17254 raft_consensus.cc:1240] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:52.387377 22842 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.401825 16472 raft_consensus.cc:1240] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:52.488611 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:52.440474 23210 consensus_queue.cc:1059] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:52.490399 23210 raft_consensus.cc:3055] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:52.446566 23183 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:52.464800 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:52.491206 23113 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:52.490772 23210 raft_consensus.cc:740] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:52.493393 23210 consensus_queue.cc:260] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:52.496469 23210 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:52.479423 23159 raft_consensus.cc:515] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:52.487340 16477 raft_consensus.cc:1275] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 12:24:52.497664 23089 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260709 12:24:52.498255 16477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53186 (request call id 1524) took 2703 ms. Trace:
I20260709 12:24:52.498692 23159 leader_election.cc:290] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:52.494238 23150 consensus_queue.cc:1048] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:24:52.498644 16477 rpcz_store.cc:276] 0709 12:24:49.794449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.794556 (+   107us) service_pool.cc:224] Handling call
0709 12:24:49.794809 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:50.168545 (+373736us) spinlock_profiling.cc:243] Waited 81.4 ms on lock 0x7b54003a23a0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e00dc0 00007f2875e8b1c0 00007f2875e8c231 00007f2875e88d8f 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62 00007f28743c4a91 00007f28743c4a0c
0709 12:24:50.810531 (+641986us) spinlock_profiling.cc:243] Waited 41.5 ms on lock 0x7b54003a23a0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875e0706f 00007f2875e7ff60 00007f2875e817de 00007f2875e8bb49 00007f2875e8c416 00007f2875e88d8f 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4 00007f28743c4b62
0709 12:24:51.613391 (+802860us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b18002acb40. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875ec4646 00007f2875dfe58e 00007f2875e7ff9e 00007f2875e817de 00007f2875e8bb49 00007f2875e8c416 00007f2875e88d8f 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4
0709 12:24:52.052753 (+439362us) spinlock_profiling.cc:243] Waited 438 ms on lock 0x7b5800500fd0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875de50aa 00007f2875e67447 00007f2875e7ffe7 00007f2875e817de 00007f2875e8bb49 00007f2875e8c416 00007f2875e88d8f 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4
0709 12:24:52.335496 (+282743us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b58009041d0. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2875df2571 00007f2875de50aa 00007f2875e67447 00007f2875e7ffe7 00007f2875e817de 00007f2875e8bb49 00007f2875e8c416 00007f2875e88d8f 00007f2875e8883e 00007f287ae4d419 00007f28743c4bd4
0709 12:24:52.497387 (+161891us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:52.498171 (+   784us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3174902912}
I20260709 12:24:52.500708 23194 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.501070 23123 consensus_queue.cc:1048] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:52.501233 23194 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:52.501699 23150 consensus_queue.cc:1048] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:52.503998 16398 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 18.
I20260709 12:24:52.476156 17258 raft_consensus.cc:2368] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 17. Current term is 18.
I20260709 12:24:52.493400 17270 raft_consensus.cc:1275] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 12:24:52.505295 17270 pending_rounds.cc:85] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8: Aborting all ops after (but not including) 9
I20260709 12:24:52.505532 17270 pending_rounds.cc:107] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:24:52.505767 17270 raft_consensus.cc:2889] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:52.494803 23183 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:52.496374 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:52.508097 17254 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:52.480782 23006 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:52.509555 23006 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:52.381042 23157 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:52.487250 22885 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:52.512620 23157 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:52.495352 22842 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:52.515185 23157 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:52.558616 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:52.559659 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 147821us
I20260709 12:24:52.562803 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:52.567137 23194 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:52.567824 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57fce16c9dc745ce8d5a631936c11e00" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:52.570554 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:52.571455 16545 raft_consensus.cc:2393] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 18: Already voted for candidate ef57beab0e1d40d585983873c55107e8 in this term.
I20260709 12:24:52.515653 22842 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:52.518996 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:52.572610 16477 raft_consensus.cc:2393] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 18: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:52.512995 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:52.573918 16474 raft_consensus.cc:1275] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:24:52.520826 23203 consensus_queue.cc:1048] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:24:52.521209 17263 raft_consensus.cc:2393] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 13: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:24:52.534628 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:52.575728 16551 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:52.584364 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07c17e01135a4bae850c5038c62d2ba2" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:52.597430 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:52.524236 17254 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 20.
I20260709 12:24:52.533999 22927 consensus_queue.cc:1048] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 12:24:52.531448 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:52.619541 17270 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 16.
I20260709 12:24:52.540437 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:52.542847 17143 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:52.547049 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:52.620850 16550 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 25.
I20260709 12:24:52.621116 16548 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 16.
I20260709 12:24:52.546010 16221 catalog_manager.cc:5697] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 10 to 13, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 13 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:52.549638 16547 raft_consensus.cc:1275] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 12:24:52.622484 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 55405.2us
I20260709 12:24:52.550035 16542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:52.551115 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 55274.6us
I20260709 12:24:52.623843 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:52.601037 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 55894.5us
I20260709 12:24:52.584096 16551 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 20.
I20260709 12:24:52.607173 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:52.609549 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 47808.6us
I20260709 12:24:52.634490 16543 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 17.
I20260709 12:24:52.623458 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:52.618450 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:52.626410 16549 raft_consensus.cc:1275] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:24:52.631558 16550 raft_consensus.cc:1240] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:52.631754 16547 raft_consensus.cc:1275] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 25 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:24:52.624433 16367 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:52.639004 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:52.657222 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:52.643846 16221 catalog_manager.cc:5697] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 12 to 15, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 15 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:52.658550 16394 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:52.645063 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 102296us
I20260709 12:24:52.660758 16548 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 16.
I20260709 12:24:52.661433 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:52.663018 23181 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:52.673273 16221 catalog_manager.cc:5697] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
I20260709 12:24:52.674481 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:52.676158 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:52.588111 23223 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:52.679638 16221 catalog_manager.cc:5697] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 11 to 12, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 12 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
W20260709 12:24:52.690490 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 57344.3us
I20260709 12:24:52.690793 17143 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:52.696277 16515 leader_election.cc:400] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 17. Current term is 18.
I20260709 12:24:52.708761 16515 leader_election.cc:403] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:52.700291 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:52.650998 22553 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:52.703986 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0e81993a5641399dd0ca4ded8b4037" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:52.705550 16223 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53622 (request call id 137) took 23624 ms (client timeout 15000 ms). Trace:
I20260709 12:24:52.637460 22940 consensus_queue.cc:1048] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 12:24:52.676842 23181 raft_consensus.cc:697] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:52.712123 16441 leader_election.cc:304] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:52.725498 16440 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.730970 16398 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 29 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:24:52.733971 17143 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:52.725134 16516 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:52.735184 17143 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:52.735411 16441 leader_election.cc:304] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:52.741200 16394 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 20.
I20260709 12:24:52.643540 23211 consensus_queue.cc:1048] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
W20260709 12:24:52.743917 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 105479us
I20260709 12:24:52.745316 16441 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:52.741789 16221 catalog_manager.cc:5697] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 12 to 15, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 15 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:52.748262 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:52.742954 16400 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 16.
I20260709 12:24:52.754340 17144 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:52.716367 22553 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:52.716038 16223 rpcz_store.cc:269] 0709 12:24:29.081269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:29.082680 (+  1411us) service_pool.cc:224] Handling call
0709 12:24:52.705441 (+23622761us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:44.064979 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:44.065741 (+   762us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:44.065801 (+    60us) write_op.cc:435] Acquired schema lock
0709 12:24:44.065820 (+    19us) tablet.cc:662] Decoding operations
0709 12:24:44.135420 (+ 69600us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:44.135588 (+   168us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:44.135619 (+    31us) tablet.cc:685] Acquiring locks for 27 operations
0709 12:24:44.160527 (+ 24908us) tablet.cc:701] Row locks acquired
0709 12:24:44.160581 (+    54us) write_op.cc:260] PREPARE: finished
0709 12:24:44.160747 (+   166us) write_op.cc:270] Start()
0709 12:24:44.160895 (+   148us) write_op.cc:276] Timestamp: P: 1783599884160679 usec, L: 0
0709 12:24:44.160915 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:24:44.237330 (+ 76415us) log.cc:844] Serialized 10386 byte log entry
0709 12:24:44.403705 (+166375us) spinlock_profiling.cc:243] Waited 2.04 ms on lock 0x7b1800008560. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f287bc64ab1 00007f287bc71281 00007f2872d9c3f3 00007f2872d9e7c1 00007f2872d9e779 00007f2872d9e709 00007f2872d9e6d1 00007f2872d9d9cd 0000000000388634 0000000000388479 00007f2872d7f83a
0709 12:24:44.566991 (+163286us) op_driver.cc:464] REPLICATION: finished
0709 12:24:44.716231 (+149240us) write_op.cc:301] APPLY: starting
0709 12:24:44.717811 (+  1580us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:45.171285 (+453474us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:45.171340 (+    55us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:45.696301 (+524961us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:45.725053 (+ 28752us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=54,key_file_lookups=54,delta_file_lookups=0,mrs_lookups=0
0709 12:24:45.725103 (+    50us) write_op.cc:312] APPLY: finished
0709 12:24:45.725692 (+   589us) log.cc:844] Serialized 235 byte log entry
0709 12:24:45.773318 (+ 47626us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:45.773502 (+   184us) write_op.cc:454] Released schema lock
0709 12:24:45.773625 (+   123us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:45.833162 (+ 59537us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:45.841550 (+  8388us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3859228,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":96669,"apply.run_cpu_time_us":57244,"apply.run_wall_time_us":1109428,"cfile_cache_hit":110,"cfile_cache_hit_bytes":117095,"num_ops":27,"prepare.queue_time_us":89974,"prepare.run_cpu_time_us":31307,"prepare.run_wall_time_us":281770,"raft.queue_time_us":137800,"raft.run_cpu_time_us":9259,"raft.run_wall_time_us":45687,"replication_time_us":405844,"spinlock_wait_cycles":4698880,"thread_start_us":328019,"threads_started":5,"wal-append.queue_time_us":167965,"wal-append.run_cpu_time_us":20722,"wal-append.run_wall_time_us":2177522}]]}
I20260709 12:24:52.757421 16441 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:52.759044 16221 catalog_manager.cc:5697] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 7 to 13, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 13 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:52.730168 23185 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 17. Current term is 18.
I20260709 12:24:52.774173 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:52.746757 22885 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:24:52.786363 22885 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.786839 22885 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:52.741914 23089 raft_consensus.cc:2749] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:52.776678 23104 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:52.793218 23104 raft_consensus.cc:697] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 16 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:52.794189 23104 consensus_queue.cc:237] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:52.751701 23230 consensus_queue.cc:1048] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:24:52.801565 23223 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:52.732828 23072 consensus_queue.cc:1048] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
W20260709 12:24:52.811185 23232 log.cc:927] Time spent T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.118s	user 0.001s	sys 0.001s
I20260709 12:24:52.766129 22553 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:52.750430 23157 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:52.703428 23006 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:52.760978 23218 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:52.772589 22747 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.006s
I20260709 12:24:52.761165 23183 consensus_queue.cc:1048] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 12:24:52.817883 23006 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:52.776090 22887 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:52.780756 23225 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.781411 16471 raft_consensus.cc:1275] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:24:52.828647 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:52.767669 22927 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:52.747032 23159 raft_consensus.cc:2749] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:52.818362 16546 raft_consensus.cc:3060] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:52.761369 23150 consensus_queue.cc:1048] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 12:24:52.819159 23181 consensus_queue.cc:237] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:52.819782 16442 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:52.820816 23157 raft_consensus.cc:697] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 20 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:52.822151 16367 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:52.823309 16441 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:52.827031 16516 leader_election.cc:400] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Vote denied by peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367) with higher term. Message: Invalid argument: T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 19. Current term is 20.
I20260709 12:24:52.842391 16397 raft_consensus.cc:1275] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:24:52.760685 16221 catalog_manager.cc:5697] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 13 to 16, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 16 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
I20260709 12:24:52.850244 16441 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:52.852072 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 84888.5us
I20260709 12:24:52.853739 17269 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
I20260709 12:24:52.860584 16441 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:52.829207 23218 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:52.874349 16367 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:52.876263 16221 catalog_manager.cc:5697] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 8 to 12, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 12 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
I20260709 12:24:52.878942 16367 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:52.887625 16546 raft_consensus.cc:2468] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 20.
I20260709 12:24:52.843997 22887 raft_consensus.cc:697] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 18 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:52.859701 23042 raft_consensus.cc:2804] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:52.858732 23183 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:52.863008 23159 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:52.875156 23101 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:52.865195 23218 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:52.876950 23089 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:52.842779 23225 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:52.896059 23006 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:52.941378 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 219101us
I20260709 12:24:52.895613 23113 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:52.948566 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 179398us
I20260709 12:24:52.900774 23026 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:52.901126 23223 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:52.901003 23034 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:52.879652 23169 consensus_queue.cc:1048] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.066s
I20260709 12:24:52.905149 23183 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:52.942312 23232 log.cc:927] Time spent T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:24:52.935369 23114 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.966392 23183 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:52.958550 23185 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.977778 23042 raft_consensus.cc:697] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 20 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:52.959142 23238 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:52.983680 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 242521us
I20260709 12:24:52.952005 23113 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:52.954093 23006 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:52.964215 23034 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:52.932876 23157 consensus_queue.cc:237] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:52.971279 23223 raft_consensus.cc:697] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:53.019304 23042 consensus_queue.cc:237] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:52.992275 22842 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:53.037194 22842 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:52.979277 23148 consensus_queue.cc:1048] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 12:24:52.971863 23114 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:52.984524 23185 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:52.986068 22887 consensus_queue.cc:237] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:24:53.068840 23234 log.cc:927] Time spent T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 12:24:53.072237 23113 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:53.071676 23223 consensus_queue.cc:237] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:53.089282 23225 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:53.094450 23034 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:53.104880 23114 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:53.106057 23006 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:53.107926 23185 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:53.109171 22885 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:53.111949 22885 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 26 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:53.145294 23218 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:53.153761 23238 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:53.167297 23218 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:53.228981 22842 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:53.198716 16221 catalog_manager.cc:5697] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 12 to 13, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 13 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:53.236794 16221 catalog_manager.cc:5697] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 14 to 17, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 17 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:53.240051 17271 raft_consensus.cc:1240] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 26 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 25. Current term is 26. Ops: [25.13-25.13]
I20260709 12:24:53.135111 23183 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:53.241905 23110 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.243002 23110 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:53.242044 16572 debug-util.cc:398] Leaking SignalData structure 0x7b0800271300 after lost signal to thread 5355
W20260709 12:24:53.248020 16572 debug-util.cc:398] Leaking SignalData structure 0x7b08003254e0 after lost signal to thread 16260
I20260709 12:24:53.261858 22927 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.249068 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:53.264644 23114 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.264637 22927 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:53.250326 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:53.258795 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
W20260709 12:24:53.274277 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 66316.6us
W20260709 12:24:53.263960 16572 debug-util.cc:398] Leaking SignalData structure 0x7b0800130460 after lost signal to thread 16427
W20260709 12:24:53.291277 16572 debug-util.cc:398] Leaking SignalData structure 0x7b0800074820 after lost signal to thread 16501
I20260709 12:24:53.325182 16474 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 29 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
W20260709 12:24:53.326540 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 84478.2us
I20260709 12:24:53.328107 23110 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:53.330293 22940 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:53.331182 22940 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:53.331813 23042 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:53.332540 23042 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:53.335112 23042 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:53.265623 23114 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:53.354295 16572 debug-util.cc:398] Leaking SignalData structure 0x7b0800289ec0 after lost signal to thread 16575
I20260709 12:24:53.357499 23183 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:53.382859 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:53.383113 16402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:53.383899 16402 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 19.
I20260709 12:24:53.367412 16221 catalog_manager.cc:5697] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
I20260709 12:24:53.391139 22940 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:53.392765 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:53.393954 16545 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 17.
I20260709 12:24:53.393478 16221 catalog_manager.cc:5697] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 8 to 11, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 11 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
I20260709 12:24:53.411149 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6d1babdc37d40e4a71ffde9b310feaa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:53.426224 23114 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:53.427675 22927 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:53.432008 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:53.431342 16401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:53.432864 16400 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 19.
I20260709 12:24:53.433262 16401 raft_consensus.cc:2393] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 17: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:53.434337 16400 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:53.440163 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:53.441013 17271 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 20.
W20260709 12:24:53.442430 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 302143us
I20260709 12:24:53.445492 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:53.445359 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:53.449891 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:53.455739 16572 debug-util.cc:398] Leaking SignalData structure 0x7b080027fa80 after lost signal to thread 17299
I20260709 12:24:53.455801 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:53.457006 17264 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 23.
W20260709 12:24:53.458240 23245 log.cc:927] Time spent T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.438s	user 0.014s	sys 0.022s
I20260709 12:24:53.459393 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:53.478626 23146 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:53.479624 23146 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:53.485409 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:53.488363 23225 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.489144 23225 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:53.490120 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:53.491312 16546 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:53.491698 23225 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:53.494747 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:53.415601 16221 catalog_manager.cc:5697] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:53.497649 16367 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:24:53.498844 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 48031.7us
I20260709 12:24:53.502463 23223 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:53.503044 23223 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:53.503402 23223 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:53.499176 16221 catalog_manager.cc:5697] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 14 to 17, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 17 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
I20260709 12:24:53.503800 23114 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.512601 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:53.514197 23114 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:53.524627 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:53.525419 17264 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 20.
I20260709 12:24:53.527462 23169 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:24:53.517035 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:53.523804 23181 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:53.530721 23181 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:53.532693 23146 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:53.533391 23181 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:53.535158 23019 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:53.535918 23019 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:53.538470 23019 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:53.530164 16470 raft_consensus.cc:2393] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 22: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:53.542544 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:53.557870 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:53.558964 16471 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 20.
W20260709 12:24:53.576844 16572 debug-util.cc:398] Leaking SignalData structure 0x7b0800054960 after lost signal to thread 23103
I20260709 12:24:53.580724 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:53.581808 16477 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 17.
I20260709 12:24:53.607466 16397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:53.618907 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:53.610977 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:53.622012 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:53.625653 23255 raft_consensus.cc:493] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:53.626504 23255 raft_consensus.cc:515] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:53.623764 22940 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:53.631592 22940 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:53.639366 16550 raft_consensus.cc:1275] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 12:24:53.641024 22553 consensus_meta.cc:287] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50: Time spent flushing consensus metadata: real 0.827s	user 0.012s	sys 0.019s
I20260709 12:24:53.642763 16473 raft_consensus.cc:2368] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17.
I20260709 12:24:53.640549 17264 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:53.654060 23255 leader_election.cc:290] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:53.654597 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:53.654820 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 50814.5us
I20260709 12:24:53.655169 22553 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:53.628993 23114 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:53.655607 16477 raft_consensus.cc:2393] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 20: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:53.659174 22940 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:53.656013 16442 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:53.665380 22553 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:53.665813 23194 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:53.666322 23194 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:53.666689 23194 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:53.673287  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:53.637142 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:53.641553 17144 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:53.660789 16550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:53.687414 23252 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:53.692413 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:24:53.692572 23267 log.cc:927] Time spent T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.076s	user 0.000s	sys 0.003s
I20260709 12:24:53.693354 17263 raft_consensus.cc:2368] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 19. Current term is 20.
I20260709 12:24:53.692662 23252 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:53.694253 23252 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:53.694962 16548 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 22.
I20260709 12:24:53.696553 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:53.699875 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:53.702579 16546 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 26.
I20260709 12:24:53.703112 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:53.709692 16472 raft_consensus.cc:1240] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17. Ops: []
W20260709 12:24:53.710182 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 169471us
I20260709 12:24:53.713147 16472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:53.715160 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:53.716274 16477 raft_consensus.cc:2393] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 23: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:53.711892 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:53.720115 16471 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 19.
I20260709 12:24:53.704053 16516 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:53.721632 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 156077us
I20260709 12:24:53.723418 23243 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:53.728366 17143 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:53.729678 23250 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:53.730679 17143 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:53.731040 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:53.734436 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:53.735139 16543 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 22.
I20260709 12:24:53.740675 16550 raft_consensus.cc:1240] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 26 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 25. Current term is 26. Ops: [25.13-25.13]
I20260709 12:24:53.740723 23146 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:24:53.686694 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 64118.1us
I20260709 12:24:53.743108 16400 raft_consensus.cc:1275] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:24:53.752400 16366 leader_election.cc:400] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 16 election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17.
I20260709 12:24:53.762537 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:53.769598 17143 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:53.775250 16515 leader_election.cc:400] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 19. Current term is 20.
I20260709 12:24:53.775699 16515 leader_election.cc:403] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:24:53.777050 16477 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:53.781436 16367 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:53.785596 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:53.786581 16367 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:53.787143 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:53.787599 22887 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 for earlier term 19. Current term is 20.
I20260709 12:24:53.795637 22940 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:53.803146 22940 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:53.803604 22940 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:53.805994 17142 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:53.811194 23218 consensus_queue.cc:1048] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 12:24:53.811349 17143 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:53.791903 23252 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:53.822547 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 70765.7us
I20260709 12:24:53.795363 23255 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:53.772500 23169 consensus_queue.cc:1048] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
W20260709 12:24:53.835961 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 70637.1us
I20260709 12:24:53.833303 23157 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:53.838617 23157 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:53.839057 23157 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:53.844420 22967 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:53.845976 16549 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 17.
I20260709 12:24:53.846061 17142 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:53.847424 16515 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:53.848376 16477 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 19.
W20260709 12:24:53.861310 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 48176.4us
I20260709 12:24:53.865613 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:53.868101 22553 consensus_queue.cc:1059] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:53.873646 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46e3072dec049b18b323d65470603ec" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:53.848613 23019 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:24:53.880100 16441 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:24:53.882457 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 88917.5us
I20260709 12:24:53.882799 23183 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:53.883206 16396 raft_consensus.cc:1240] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:53.885502 16549 raft_consensus.cc:1275] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:24:53.883929 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:53.888706 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:53.889421 17270 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 23.
I20260709 12:24:53.861603 22967 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:53.887907 17263 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 19.
I20260709 12:24:53.894083 23252 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:24:53.893024 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 95948.8us
I20260709 12:24:53.893114 23042 raft_consensus.cc:3055] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:53.897114 23042 raft_consensus.cc:740] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:53.898447 23042 consensus_queue.cc:260] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:53.900071 23042 raft_consensus.cc:3060] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:53.889107 23194 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:53.894743 17269 raft_consensus.cc:1275] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:24:53.903679 23194 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:53.904181 23006 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:24:53.905471 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 58151.2us
I20260709 12:24:53.901024 16441 leader_election.cc:304] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:53.911326 23183 raft_consensus.cc:2749] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:53.914075 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:53.914175 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:53.933828 16441 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:53.930258 17144 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:53.939918 17264 raft_consensus.cc:2468] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 17.
I20260709 12:24:53.903957 23019 raft_consensus.cc:697] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 26 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:53.960589 22788 consensus_queue.cc:1059] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:24:53.972882 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 87611us
I20260709 12:24:53.948506 23194 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:53.978384 17261 raft_consensus.cc:1240] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 16. Current term is 17. Ops: []
W20260709 12:24:53.979835 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 50538.1us
I20260709 12:24:53.980031 22788 raft_consensus.cc:3055] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:53.980551 22788 raft_consensus.cc:740] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:53.981611 22788 consensus_queue.cc:260] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:53.976392 23183 consensus_queue.cc:1059] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 12:24:53.982971 22967 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:53.982223 23157 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:53.998183 16402 raft_consensus.cc:1275] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:24:53.988880 16367 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:53.967384 23225 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.999799 23225 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:54.002072 23225 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:54.002549 16368 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.003856 16367 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:54.004523 23225 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:53.977097 23194 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:54.014057 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 194686us
I20260709 12:24:54.022782 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:54.014281 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:54.023887 16367 leader_election.cc:304] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:54.027928 16471 raft_consensus.cc:2393] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 in current term 21: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:24:54.025643 17269 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 22.
I20260709 12:24:54.024534 23255 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:54.031214 23148 consensus_queue.cc:1048] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:54.031649 23255 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.034348 23255 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:53.960605 23019 consensus_queue.cc:237] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:54.013293 23225 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:24:54.039342 23225 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:54.033895 23114 consensus_queue.cc:1059] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:54.013056 23194 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:54.032040 23183 raft_consensus.cc:3055] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:24:54.070542 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:54.071110 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:54.084686 22553 consensus_queue.cc:1048] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:54.086419 22553 raft_consensus.cc:2749] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:54.088893 16549 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 22.
I20260709 12:24:54.090323 17143 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8; no voters: 
I20260709 12:24:54.091578 23169 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:54.092105 23169 raft_consensus.cc:697] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 22 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:54.093120 23169 consensus_queue.cc:237] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:54.093459 22788 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:54.098382 16546 raft_consensus.cc:1275] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:24:54.079509 23183 raft_consensus.cc:740] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:53.990828 23223 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:54.101819 23252 consensus_queue.cc:1048] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:54.101682 23183 consensus_queue.cc:260] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:54.104859 23223 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:54.080228 23157 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:24:54.171979 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 151690us
W20260709 12:24:54.179400 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 148180us
I20260709 12:24:54.185638 16548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:54.188419 16548 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:54.194372 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:54.200135 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:54.215386 16548 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 18.
I20260709 12:24:54.220758 16367 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:54.222486 22553 raft_consensus.cc:2804] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:54.223048 22553 raft_consensus.cc:697] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 18 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:54.223979 22553 consensus_queue.cc:237] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:54.225468 23194 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:54.229702 16477 raft_consensus.cc:2368] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 22. Current term is 23.
I20260709 12:24:54.188905 23282 consensus_queue.cc:1048] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:24:54.241847 16549 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 16.
W20260709 12:24:54.242136 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 164489us
I20260709 12:24:54.175318 22940 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:54.251688 16400 raft_consensus.cc:1275] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 12:24:54.245008 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 58658.6us
I20260709 12:24:54.258464 16396 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:54.264571 17142 leader_election.cc:400] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate ef57beab0e1d40d585983873c55107e8 for earlier term 22. Current term is 23.
I20260709 12:24:54.285187 16441 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:54.244984 23255 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:54.263846 23225 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:54.306531 23287 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:54.314028 23252 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
W20260709 12:24:54.338704 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 81979us
I20260709 12:24:54.321089 23194 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:54.323262 23287 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.348582 23026 consensus_queue.cc:1048] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 12:24:54.361058 22940 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:54.351938 23287 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:54.348213 23252 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:54.373898 23255 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:54.372092 16441 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:54.381983 16367 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:54.389637 16396 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef57beab0e1d40d585983873c55107e8 in term 21.
I20260709 12:24:54.388859 16441 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:54.391666 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 64381us
I20260709 12:24:54.373898 23225 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 23 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:54.391335 23114 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:24:54.405264 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:54.406669 17271 raft_consensus.cc:2368] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 20. Current term is 21.
I20260709 12:24:54.410763 17264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:54.392134 22940 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:54.416296 22940 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:54.408089 23218 raft_consensus.cc:2804] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:54.417387 23218 raft_consensus.cc:697] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:54.418480 23218 consensus_queue.cc:237] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:54.421495 16366 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:54.416762 22940 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:54.425813 23289 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:54.426574 23289 raft_consensus.cc:697] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 19 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:54.434231 23183 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:54.438011 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:54.438675 16546 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:54.427678 23289 consensus_queue.cc:237] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:54.419843 17261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:54.448736 17261 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:54.454665 17264 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 23.
W20260709 12:24:54.460464 17143 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 53008.6us
I20260709 12:24:54.458913 16471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:54.469630 16471 raft_consensus.cc:3060] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:54.469699 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:54.480477 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:54.428143 23252 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:54.481129 23104 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:54.516252 16546 raft_consensus.cc:2468] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 23.
W20260709 12:24:54.521165 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 49032.2us
W20260709 12:24:54.521234 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 49525.9us
I20260709 12:24:54.524973 16551 raft_consensus.cc:1275] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:24:54.532678 16471 raft_consensus.cc:2468] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 18.
I20260709 12:24:54.534759 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:54.537328 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:54.533649 16441 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:54.539028 16473 raft_consensus.cc:2368] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 20. Current term is 21.
W20260709 12:24:54.554324 23295 log.cc:927] Time spent T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.144s	user 0.000s	sys 0.005s
I20260709 12:24:54.559074 16367 leader_election.cc:400] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 election: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 20. Current term is 21.
I20260709 12:24:54.558877 17144 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2
I20260709 12:24:54.559476 16367 leader_election.cc:403] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:24:54.570348 16367 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:54.586697 16475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:54.587461 16475 raft_consensus.cc:3060] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:54.594362 16551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:54.597966 17261 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 20.
W20260709 12:24:54.598675 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 65157.6us
I20260709 12:24:54.560725 23194 raft_consensus.cc:2764] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
I20260709 12:24:54.598351 23282 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805) with higher term. Message: Invalid argument: T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 20. Current term is 21.
I20260709 12:24:54.533408 23104 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:54.538967 23287 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:54.623075 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 83295.8us
W20260709 12:24:54.627329 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 77578.2us
I20260709 12:24:54.603317 23252 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:54.650576 16477 raft_consensus.cc:2368] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 23. Current term is 24.
I20260709 12:24:54.622236 23223 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:54.685151 16441 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
W20260709 12:24:54.672708 23303 log.cc:927] Time spent T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a: Append to log took a long time: real 0.275s	user 0.004s	sys 0.002s
W20260709 12:24:54.693395 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 53749.1us
I20260709 12:24:54.696269 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:54.696731 17271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
W20260709 12:24:54.688500 23314 log.cc:927] Time spent T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:24:54.701071 16367 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:54.675784 23223 raft_consensus.cc:697] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 23 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:54.721439 23283 raft_consensus.cc:2804] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:54.722319 23283 raft_consensus.cc:697] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:54.723430 23283 consensus_queue.cc:237] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:54.727538 23104 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:54.739691 16475 raft_consensus.cc:2468] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 20.
I20260709 12:24:54.700825 23252 raft_consensus.cc:697] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 21 LEADER]: Becoming Leader. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
W20260709 12:24:54.742849 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 48226.7us
W20260709 12:24:54.759413 23313 log.cc:927] Time spent T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 12:24:54.767478 23223 consensus_queue.cc:237] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:54.732860 23225 raft_consensus.cc:2804] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:54.771237 23225 raft_consensus.cc:697] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 20 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
W20260709 12:24:54.771740 23311 log.cc:927] Time spent T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 12:24:54.772763 23225 consensus_queue.cc:237] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:24:54.781226 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 222425us
W20260709 12:24:54.779258 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 94187.4us
I20260709 12:24:54.793041 17258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:54.793829 17258 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:54.793498 23194 consensus_queue.cc:1048] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:54.792922 17263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:54.800992 16366 leader_election.cc:400] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 20 election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 20. Current term is 21.
I20260709 12:24:54.776427 23287 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 24 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:54.819077 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:54.840708 16471 raft_consensus.cc:1240] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 LEADER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:54.877087 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
W20260709 12:24:54.881459 23312 log.cc:927] Time spent T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:24:54.867472 22967 consensus_queue.cc:1059] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.015s
I20260709 12:24:54.885396 17258 raft_consensus.cc:2468] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 24.
I20260709 12:24:54.896785 22940 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:54.904747 16403 raft_consensus.cc:1240] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Rejecting Update request from peer 52ab50b0237346c1a13ede947f6a63c2 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:54.906311 22940 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:54.916365 22967 consensus_queue.cc:1243] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer 52ab50b0237346c1a13ede947f6a63c2 log is divergent from this leader: its last log entry 23.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
W20260709 12:24:54.919338 17144 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 62872.3us
I20260709 12:24:54.939985 17143 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:54.920266 16548 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0b9c348d235e432d92d8f8f20f1f1317"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "f99066d13e524238aec6aa2024a8bd50"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:24:54.942380 16548 raft_consensus.cc:606] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Received request to transfer leadership to TS f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:54.942792 16548 raft_consensus.cc:614] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:54.947371 23325 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:54.964782 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:54.965752 17258 raft_consensus.cc:1275] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:24:54.967027 16366 leader_election.cc:400] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 23 election: Vote denied by peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107) with higher term. Message: Invalid argument: T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for earlier term 23. Current term is 24.
I20260709 12:24:54.968787 16474 raft_consensus.cc:1275] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 12:24:54.970224 23329 log.cc:927] Time spent T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.179s	user 0.002s	sys 0.000s
I20260709 12:24:54.895994 23252 consensus_queue.cc:237] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:54.984824 22940 consensus_queue.cc:1048] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 12:24:55.000856 16548 raft_consensus.cc:1275] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 26 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
W20260709 12:24:54.991360 23328 log.cc:927] Time spent T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.197s	user 0.002s	sys 0.001s
I20260709 12:24:55.012616 16548 raft_consensus.cc:1240] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Rejecting Update request from peer ef57beab0e1d40d585983873c55107e8 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:55.014555 16441 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:55.028978 16470 raft_consensus.cc:1275] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:24:55.030349 23297 consensus_queue.cc:1048] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 12:24:55.024753 23218 raft_consensus.cc:2804] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:24:55.035986 23218 raft_consensus.cc:697] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 24 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:55.037088 23218 consensus_queue.cc:237] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:55.038921 23026 consensus_queue.cc:1059] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:24:55.054267 22553 consensus_queue.cc:1048] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:24:55.065268 22616 raft_consensus.cc:3055] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:55.065847 22616 raft_consensus.cc:740] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: ef57beab0e1d40d585983873c55107e8, State: Running, Role: LEADER
I20260709 12:24:55.066831 22616 consensus_queue.cc:260] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:55.072170 16442 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:55.081887 23321 consensus_queue.cc:1048] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 12:24:55.084513 23310 consensus_queue.cc:1059] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 12:24:55.086318 23310 raft_consensus.cc:3055] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:55.086835 23310 raft_consensus.cc:740] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:55.108273 16551 raft_consensus.cc:1275] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:24:55.108505 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:55.110818 16543 raft_consensus.cc:1275] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 12:24:55.111315 16543 pending_rounds.cc:85] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1: Aborting all ops after (but not including) 10
I20260709 12:24:55.111569 16543 pending_rounds.cc:107] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 12:24:55.111896 16543 raft_consensus.cc:2889] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:55.087925 23310 consensus_queue.cc:260] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:55.115067 23310 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:55.116984 16544 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "f99066d13e524238aec6aa2024a8bd50"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:24:55.117638 16544 raft_consensus.cc:606] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 21 LEADER]: Received request to transfer leadership to TS f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:55.079857 23104 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:55.120059 16367 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:55.120689 22940 consensus_queue.cc:1048] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:55.121308 23291 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:24:55.123577 23335 log.cc:927] Time spent T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:24:55.079545 22616 raft_consensus.cc:3060] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:24:55.132930 23331 log.cc:927] Time spent T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 12:24:55.143787 16544 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "f99066d13e524238aec6aa2024a8bd50"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:24:55.147203 23289 consensus_queue.cc:1048] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:24:55.158912 16544 raft_consensus.cc:606] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Received request to transfer leadership to TS f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:55.159428 16544 raft_consensus.cc:614] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:55.166698 16544 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "ef57beab0e1d40d585983873c55107e8"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:24:55.167430 16544 raft_consensus.cc:606] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Received request to transfer leadership to TS ef57beab0e1d40d585983873c55107e8
I20260709 12:24:55.167838 16394 raft_consensus.cc:1275] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 12:24:55.167874 16544 raft_consensus.cc:614] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:55.168277 16394 pending_rounds.cc:85] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 10
I20260709 12:24:55.175784 16394 pending_rounds.cc:107] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 12:24:55.176100 16394 raft_consensus.cc:2889] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:55.179804 23297 consensus_queue.cc:1048] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:55.176116 16544 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "95954ea821224e4d962974a83d83ad44"
dest_uuid: "610f4feda11942739236f2b121a4b9c1"
mode: GRACEFUL
new_leader_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:59590
I20260709 12:24:55.186066 16544 raft_consensus.cc:606] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 23 LEADER]: Received request to transfer leadership to TS 52ab50b0237346c1a13ede947f6a63c2
W20260709 12:24:55.197723 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 337777us
I20260709 12:24:55.217902  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:24:55.224692  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:55.249142 16441 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:55.298480 23334 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:55.249542 16474 raft_consensus.cc:1275] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 26 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 13. (term mismatch)
I20260709 12:24:55.334141 16474 pending_rounds.cc:85] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2: Aborting all ops after (but not including) 12
W20260709 12:24:55.339634 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 90596.9us
I20260709 12:24:55.334903 16474 pending_rounds.cc:107] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2: Aborting uncommitted NO_OP operation due to leader change: 25.13
I20260709 12:24:55.342433 16474 raft_consensus.cc:2889] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:55.364173 23323 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:55.368000 16476 raft_consensus.cc:1240] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 24 LEADER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:55.375265 22553 consensus_queue.cc:1059] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:24:55.376133 22553 consensus_queue.cc:1243] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Peer 52ab50b0237346c1a13ede947f6a63c2 log is divergent from this leader: its last log entry 24.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:24:55.376991 23289 raft_consensus.cc:3055] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:24:55.377355 23289 raft_consensus.cc:740] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:55.349067 23334 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:55.385795 23334 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:55.393471 16549 raft_consensus.cc:1275] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:24:55.382424 23323 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.400311 17254 raft_consensus.cc:1275] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:24:55.405716 17270 raft_consensus.cc:1240] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 24 FOLLOWER]: Rejecting Update request from peer f99066d13e524238aec6aa2024a8bd50 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:55.423270 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:55.400440 23310 consensus_queue.cc:1048] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.014s
I20260709 12:24:55.378182 23289 consensus_queue.cc:260] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:55.432535 16403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:55.412943 23323 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:55.455855 23289 raft_consensus.cc:3060] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:24:55.469363 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 61595.4us
I20260709 12:24:55.450131 22842 consensus_queue.cc:1048] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:24:55.488976 23282 consensus_queue.cc:1048] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:55.494485 23323 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:55.526408 17143 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:55.538735 23334 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:24:55.559540 17142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 113314us
W20260709 12:24:55.564512 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 107472us
I20260709 12:24:55.574671 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 27 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:55.578799 17254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 27 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "ef57beab0e1d40d585983873c55107e8" is_pre_election: true
I20260709 12:24:55.619369 17271 raft_consensus.cc:1275] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 14. (term mismatch)
I20260709 12:24:55.619838 17271 pending_rounds.cc:85] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8: Aborting all ops after (but not including) 13
I20260709 12:24:55.620107 17271 pending_rounds.cc:107] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8: Aborting uncommitted NO_OP operation due to leader change: 22.14
I20260709 12:24:55.620329 17271 raft_consensus.cc:2889] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:55.641098 23211 raft_consensus.cc:993] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1: : Instructing follower f99066d13e524238aec6aa2024a8bd50 to start an election
I20260709 12:24:55.641887 23211 raft_consensus.cc:1081] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 21 LEADER]: Signalling peer f99066d13e524238aec6aa2024a8bd50 to start an election
W20260709 12:24:55.651609 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 71287.4us
W20260709 12:24:55.669919 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 121801us
W20260709 12:24:55.670517 23342 log.cc:927] Time spent T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.382s	user 0.006s	sys 0.000s
W20260709 12:24:55.672030 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 104513us
I20260709 12:24:55.680887 23226 raft_consensus.cc:993] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1: : Instructing follower 52ab50b0237346c1a13ede947f6a63c2 to start an election
I20260709 12:24:55.681217 16395 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47"
dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
 from {username='slave'} at 127.0.0.1:46250
I20260709 12:24:55.681890 23211 raft_consensus.cc:1081] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 23 LEADER]: Signalling peer 52ab50b0237346c1a13ede947f6a63c2 to start an election
I20260709 12:24:55.681931 16395 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:55.683611 16395 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:55.687878 17269 raft_consensus.cc:1275] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 24 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:24:55.690521 16474 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "95954ea821224e4d962974a83d83ad44"
dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
 from {username='slave'} at 127.0.0.1:34826
I20260709 12:24:55.691190 16474 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:55.691577 16474 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:55.709710 16395 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:55.716831 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:24:55.711989 16395 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:24:55.718873 23299 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:24:55.719003 23114 consensus_queue.cc:1048] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:24:55.721491 16476 raft_consensus.cc:1275] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:24:55.398875  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:55.737072 16474 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 24 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:55.748099 23282 consensus_queue.cc:1048] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:24:55.777241 16474 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 24 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:55.793888 23114 consensus_queue.cc:1048] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:24:55.799957 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: true dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:55.800638 16549 raft_consensus.cc:3055] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:24:55.800980 16549 raft_consensus.cc:740] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:55.801820 16549 consensus_queue.cc:260] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:55.803030 16549 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:55.814074 16400 raft_consensus.cc:1275] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:24:55.822191 16396 raft_consensus.cc:1275] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 24 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 15. (term mismatch)
I20260709 12:24:55.822597 16396 pending_rounds.cc:85] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50: Aborting all ops after (but not including) 14
I20260709 12:24:55.822826 16396 pending_rounds.cc:107] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50: Aborting uncommitted NO_OP operation due to leader change: 23.15
I20260709 12:24:55.823082 16396 raft_consensus.cc:2889] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:55.823153 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95954ea821224e4d962974a83d83ad44" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: true dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:55.824048 16394 raft_consensus.cc:3060] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:55.826561 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: true dest_uuid: "ef57beab0e1d40d585983873c55107e8"
I20260709 12:24:55.827292 17269 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:55.831087 23319 consensus_queue.cc:1048] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:55.843204 16474 raft_consensus.cc:1275] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Refusing update from remote peer ef57beab0e1d40d585983873c55107e8: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:24:55.843489 16549 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 24.
I20260709 12:24:55.868741 16441 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:24:55.881189 17269 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 22.
I20260709 12:24:55.884090 16367 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:24:55.888041 23282 raft_consensus.cc:2804] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:55.888746 23282 raft_consensus.cc:697] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 22 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:55.888526 16223 catalog_manager.cc:5697] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:55.889750 23282 consensus_queue.cc:237] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:55.890311 16223 catalog_manager.cc:5697] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 17 to 23. New cstate: current_term: 23 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:55.885149 23319 consensus_queue.cc:1048] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 12:24:55.892145 16223 catalog_manager.cc:5697] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:55.902802 23104 consensus_queue.cc:1048] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:24:55.921053 23369 log.cc:927] Time spent T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:24:55.887404 23299 raft_consensus.cc:2804] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:24:55.922142 23299 raft_consensus.cc:697] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 24 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:24:55.941535 16394 raft_consensus.cc:2468] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 24.
I20260709 12:24:55.900211 16223 catalog_manager.cc:5697] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 11 to 18, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 18 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:24:55.942790 16223 catalog_manager.cc:5697] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 15 to 22, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 22 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:55.944476 16223 catalog_manager.cc:5697] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 15 to 22. New cstate: current_term: 22 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
W20260709 12:24:55.951763 16221 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53626 (request call id 130) took 42001 ms (client timeout 15000 ms). Trace:
W20260709 12:24:55.952253 16221 rpcz_store.cc:269] 0709 12:24:13.950098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.950360 (+   262us) service_pool.cc:224] Handling call
0709 12:24:55.951140 (+42000780us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:53.992001 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:53.992712 (+   711us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:53.992766 (+    54us) write_op.cc:435] Acquired schema lock
0709 12:24:53.992799 (+    33us) tablet.cc:662] Decoding operations
0709 12:24:53.994317 (+  1518us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:53.994531 (+   214us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:53.994558 (+    27us) tablet.cc:685] Acquiring locks for 16 operations
0709 12:24:53.996017 (+  1459us) tablet.cc:701] Row locks acquired
0709 12:24:53.996084 (+    67us) write_op.cc:260] PREPARE: finished
0709 12:24:53.996260 (+   176us) write_op.cc:270] Start()
0709 12:24:54.029053 (+ 32793us) write_op.cc:276] Timestamp: P: 1783599893996177 usec, L: 0
0709 12:24:54.051337 (+ 22284us) op_driver.cc:348] REPLICATION: starting
0709 12:24:54.196202 (+144865us) log.cc:844] Serialized 6213 byte log entry
0709 12:24:54.526089 (+329887us) spinlock_profiling.cc:243] Waited 5.55 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2875a2486e 00007f2875a23bb6 00007f2875a237d8 00007f2875a2d221 00007f2875a2d1d9 00007f2875a2d169
0709 12:24:54.795684 (+269595us) op_driver.cc:464] REPLICATION: finished
0709 12:24:54.974139 (+178455us) spinlock_profiling.cc:243] Waited 7.5 ms on lock 0x7b4400022b4c. stack: 00007f2872d6a343 00007f2872d6a240 00007f28726f91d4 00007f287c437304 00007f287c437299 00007f2872c24db1 00007f2872cc2c85 00007f2872cc36a2 00007f2875a351db 00007f2875a30aaf 00007f2872d7f0ac 00007f287bbd99a9 00007f2872d9b69b 00007f2872d9993f 00007f2872d9bc02 00007f28798bfef0
0709 12:24:55.065009 (+ 90870us) write_op.cc:301] APPLY: starting
0709 12:24:55.065148 (+   139us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:55.149831 (+ 84683us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:55.149870 (+    39us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:55.288952 (+139082us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:55.289379 (+   427us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=0,mrs_lookups=0
0709 12:24:55.289466 (+    87us) write_op.cc:312] APPLY: finished
0709 12:24:55.290112 (+   646us) log.cc:844] Serialized 147 byte log entry
0709 12:24:55.290369 (+   257us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:55.290610 (+   241us) write_op.cc:454] Released schema lock
0709 12:24:55.290712 (+   102us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:55.291351 (+   639us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:55.303175 (+ 11824us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1773610,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":266223,"apply.run_cpu_time_us":24912,"apply.run_wall_time_us":226351,"cfile_cache_hit":66,"cfile_cache_hit_bytes":71071,"num_ops":16,"prepare.queue_time_us":213372,"prepare.run_cpu_time_us":20378,"prepare.run_wall_time_us":358444,"raft.queue_time_us":33242,"raft.run_cpu_time_us":10377,"raft.run_wall_time_us":219684,"replication_time_us":744166,"spinlock_wait_cycles":30026368,"thread_start_us":376251,"threads_started":4,"wal-append.queue_time_us":150714}]]}
I20260709 12:24:55.963382 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: true dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:55.964115 16543 raft_consensus.cc:3055] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:55.923659 23299 consensus_queue.cc:237] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:55.964432 16543 raft_consensus.cc:740] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:55.948699 16223 catalog_manager.cc:5697] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 13 to 17, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 17 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
I20260709 12:24:55.965197 16543 consensus_queue.cc:260] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:55.974262 16223 catalog_manager.cc:5697] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 13 to 17, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 17 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:55.981268 16223 catalog_manager.cc:5697] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 16 to 22. New cstate: current_term: 22 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:55.983000 16223 catalog_manager.cc:5697] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 17 to 22, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 22 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:55.989496 16543 raft_consensus.cc:3060] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:55.990989 16223 catalog_manager.cc:5697] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 16 to 18, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 18 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:24:55.993383 16223 catalog_manager.cc:5697] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 13 to 18, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 18 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:56.009495 16223 catalog_manager.cc:5697] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 11 to 17, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 17 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:24:56.017750 16223 catalog_manager.cc:5697] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 9 to 14, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 14 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:24:56.048244 16223 catalog_manager.cc:5697] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 11 to 13, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 13 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
W20260709 12:24:56.099241 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 45572.5us
I20260709 12:24:56.099824 16223 catalog_manager.cc:5697] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:24:56.108116 16223 catalog_manager.cc:5697] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 12 to 15, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 15 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:56.110430 16223 catalog_manager.cc:5697] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:56.114135 16223 catalog_manager.cc:5697] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 11 to 13, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 13 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:56.115435 16223 catalog_manager.cc:5697] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:56.116693 16223 catalog_manager.cc:5697] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 11 to 14, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 14 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:24:56.118175 16223 catalog_manager.cc:5697] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 14 to 17, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 17 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
I20260709 12:24:56.119647 16223 catalog_manager.cc:5697] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 10 to 19, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 19 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:24:56.121292 16223 catalog_manager.cc:5697] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 19 to 21, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 21 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:56.124828 16223 catalog_manager.cc:5697] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 13 to 18, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 18 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:56.129087 16223 catalog_manager.cc:5697] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 8 to 22. New cstate: current_term: 22 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:24:56.130453 16223 catalog_manager.cc:5697] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:24:56.131845 16223 catalog_manager.cc:5697] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 19 to 21, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 21 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:56.134861 16223 catalog_manager.cc:5697] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 17 to 20, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 20 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
I20260709 12:24:56.136951 16223 catalog_manager.cc:5697] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 16 to 18, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 18 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:24:56.138504 16223 catalog_manager.cc:5697] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:56.140388 16223 catalog_manager.cc:5697] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 18 to 21. New cstate: current_term: 21 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:24:56.141819 16223 catalog_manager.cc:5697] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 9 to 23. New cstate: current_term: 23 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:56.143500 16223 catalog_manager.cc:5697] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 11 to 21. New cstate: current_term: 21 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:56.175092 16543 raft_consensus.cc:2468] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 22.
I20260709 12:24:56.357647 23361 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:56.358314 23361 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:56.362859 23361 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:56.364842 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:56.365770 16394 raft_consensus.cc:2393] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 17: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:56.375491 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:56.376358 16468 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 17.
I20260709 12:24:56.405519 16514 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:56.410332 23361 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:56.410760 23361 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:56.414048 23361 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:56.483583 16545 raft_consensus.cc:1275] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:24:56.486821 22553 consensus_queue.cc:1048] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:24:56.496912 17270 raft_consensus.cc:1275] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:24:56.500952 22553 consensus_queue.cc:1048] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:24:56.500682 16223 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53636 (request call id 136) took 3797 ms. Trace:
I20260709 12:24:56.510385 16398 raft_consensus.cc:1275] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 24 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:24:56.508507 16223 rpcz_store.cc:276] 0709 12:24:52.701208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.825733 (+124525us) service_pool.cc:224] Handling call
0709 12:24:56.498159 (+3672426us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:56.272933 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:56.273466 (+   533us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:56.273499 (+    33us) write_op.cc:435] Acquired schema lock
0709 12:24:56.273514 (+    15us) tablet.cc:662] Decoding operations
0709 12:24:56.276154 (+  2640us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:56.276231 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:56.276257 (+    26us) tablet.cc:685] Acquiring locks for 34 operations
0709 12:24:56.326290 (+ 50033us) tablet.cc:701] Row locks acquired
0709 12:24:56.326310 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:24:56.326622 (+   312us) write_op.cc:270] Start()
0709 12:24:56.326740 (+   118us) write_op.cc:276] Timestamp: P: 1783599896326559 usec, L: 0
0709 12:24:56.326763 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:24:56.327409 (+   646us) log.cc:844] Serialized 13035 byte log entry
0709 12:24:56.354159 (+ 26750us) op_driver.cc:464] REPLICATION: finished
0709 12:24:56.355632 (+  1473us) write_op.cc:301] APPLY: starting
0709 12:24:56.355719 (+    87us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:56.388302 (+ 32583us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:56.388490 (+   188us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:56.424988 (+ 36498us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:56.425438 (+   450us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=68,key_file_lookups=68,delta_file_lookups=0,mrs_lookups=0
0709 12:24:56.425495 (+    57us) write_op.cc:312] APPLY: finished
0709 12:24:56.426112 (+   617us) log.cc:844] Serialized 291 byte log entry
0709 12:24:56.426298 (+   186us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:56.426587 (+   289us) write_op.cc:454] Released schema lock
0709 12:24:56.426670 (+    83us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:56.430875 (+  4205us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:56.430917 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":81192,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":1227,"apply.run_cpu_time_us":42956,"apply.run_wall_time_us":75201,"cfile_cache_hit":138,"cfile_cache_hit_bytes":146383,"num_ops":34,"prepare.queue_time_us":40455,"prepare.run_cpu_time_us":7845,"prepare.run_wall_time_us":74897,"raft.queue_time_us":8970,"raft.run_cpu_time_us":953,"raft.run_wall_time_us":14371,"replication_time_us":27232,"thread_start_us":72222,"threads_started":4,"wal-append.queue_time_us":6157}]]}
I20260709 12:24:56.511379 23361 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:56.512130 23218 consensus_queue.cc:1048] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:24:56.512528 16219 catalog_manager.cc:5697] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 19 to 21, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 21 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:24:56.514034 16219 catalog_manager.cc:5697] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 17 to 18, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 18 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:56.515516 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:24:56.516963 16394 raft_consensus.cc:2393] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 610f4feda11942739236f2b121a4b9c1 in current term 17: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
I20260709 12:24:56.518864 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:56.513863 23361 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:56.523516 16219 catalog_manager.cc:5697] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 17 to 20, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 20 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:24:56.565594 16476 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 17.
I20260709 12:24:56.566840 16514 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:56.573736 23006 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:56.574355 23006 raft_consensus.cc:697] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 17 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:56.575261 23006 consensus_queue.cc:237] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:56.586421 16219 catalog_manager.cc:5697] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 20 to 23, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 23 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:56.590072 16219 catalog_manager.cc:5697] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 15 to 20, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 20 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:24:56.598134 16219 catalog_manager.cc:5697] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 12 to 15, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 15 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:24:56.624688 16219 catalog_manager.cc:5697] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 15 to 18, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 18 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:24:56.669895 16219 catalog_manager.cc:5697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 23 to 28, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 28 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:24:56.697391 16545 raft_consensus.cc:1275] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 24 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:24:56.733748 16219 catalog_manager.cc:5697] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 11 to 19, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 19 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:24:56.735728 16219 catalog_manager.cc:5697] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 18 to 22. New cstate: current_term: 22 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:24:56.745003 23299 consensus_queue.cc:1048] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:24:56.743736 16219 catalog_manager.cc:5697] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 14 to 17, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 17 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
W20260709 12:24:56.756618 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 45255.1us
I20260709 12:24:56.789947 16219 catalog_manager.cc:5697] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 13 to 18, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 18 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:24:56.959641 22553 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.960218 22553 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:56.962877 22553 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:56.967007 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:56.967027 16221 catalog_manager.cc:5697] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 14 to 17, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 17 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
I20260709 12:24:56.970781 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:56.977545 16476 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 17.
I20260709 12:24:56.979100 16366 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:56.978638 16221 catalog_manager.cc:5697] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 11 to 14, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 14 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:56.980363 23385 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:56.980829 23385 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:56.981181 23385 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:56.981556 16219 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53622 (request call id 138) took 4951 ms. Trace:
I20260709 12:24:56.984277 16221 catalog_manager.cc:5697] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 11 to 21. New cstate: current_term: 21 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:56.982877 16219 rpcz_store.cc:276] 0709 12:24:52.029785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.030857 (+  1072us) service_pool.cc:224] Handling call
0709 12:24:56.981425 (+4950568us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:56.819051 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:56.819564 (+   513us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:56.819613 (+    49us) write_op.cc:435] Acquired schema lock
0709 12:24:56.819632 (+    19us) tablet.cc:662] Decoding operations
0709 12:24:56.821631 (+  1999us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:56.821679 (+    48us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:56.821704 (+    25us) tablet.cc:685] Acquiring locks for 12 operations
0709 12:24:56.822722 (+  1018us) tablet.cc:701] Row locks acquired
0709 12:24:56.822760 (+    38us) write_op.cc:260] PREPARE: finished
0709 12:24:56.822887 (+   127us) write_op.cc:270] Start()
0709 12:24:56.822997 (+   110us) write_op.cc:276] Timestamp: P: 1783599896822845 usec, L: 0
0709 12:24:56.823012 (+    15us) op_driver.cc:348] REPLICATION: starting
0709 12:24:56.823650 (+   638us) log.cc:844] Serialized 4701 byte log entry
0709 12:24:56.826854 (+  3204us) op_driver.cc:464] REPLICATION: finished
0709 12:24:56.827376 (+   522us) write_op.cc:301] APPLY: starting
0709 12:24:56.827477 (+   101us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:56.841789 (+ 14312us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:56.841889 (+   100us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:56.869689 (+ 27800us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:56.869981 (+   292us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=0,mrs_lookups=0
0709 12:24:56.870036 (+    55us) write_op.cc:312] APPLY: finished
0709 12:24:56.870392 (+   356us) log.cc:844] Serialized 112 byte log entry
0709 12:24:56.874283 (+  3891us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:56.884569 (+ 10286us) write_op.cc:454] Released schema lock
0709 12:24:56.884643 (+    74us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:56.885134 (+   491us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:56.885175 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":376724,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":283,"apply.run_cpu_time_us":17000,"apply.run_wall_time_us":57695,"cfile_cache_hit":48,"cfile_cache_hit_bytes":50208,"num_ops":12,"prepare.queue_time_us":130,"prepare.run_cpu_time_us":4115,"prepare.run_wall_time_us":4869,"replication_time_us":3676}]]}
I20260709 12:24:56.994041 16221 catalog_manager.cc:5697] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 13 to 19. New cstate: current_term: 19 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:57.005626 16221 catalog_manager.cc:5697] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 13 to 25. New cstate: current_term: 25 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:57.080433 23385 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:57.090982 16399 raft_consensus.cc:1240] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Rejecting Update request from peer 610f4feda11942739236f2b121a4b9c1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:57.094961 23028 consensus_queue.cc:1059] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:57.095860 23028 raft_consensus.cc:3055] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:57.096177 23028 raft_consensus.cc:740] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:24:57.097396 23028 consensus_queue.cc:260] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:57.098757 23028 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:57.100505  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:57.119755 23385 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:57.120575 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:24:57.121186 16476 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:57.127583 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:24:57.140687 16543 raft_consensus.cc:2410] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:24:57.176317 16476 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 18.
I20260709 12:24:57.178237 16366 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:57.179325 23385 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:57.179888 23385 raft_consensus.cc:697] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 18 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:24:57.180821 23385 consensus_queue.cc:237] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:24:57.267112 16221 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53626 (request call id 132) took 3517 ms. Trace:
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:493: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 6
I20260709 12:24:57.272754  5352 tablet_server.cc:179] TabletServer@127.5.58.1:41805 shutting down...
I20260709 12:24:57.273717 16221 rpcz_store.cc:276] 0709 12:24:53.750230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.029844 (+2279614us) service_pool.cc:224] Handling call
0709 12:24:57.267023 (+1237179us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:57.051697 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:57.052305 (+   608us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:57.052395 (+    90us) write_op.cc:435] Acquired schema lock
0709 12:24:57.052419 (+    24us) tablet.cc:662] Decoding operations
0709 12:24:57.053199 (+   780us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:57.053261 (+    62us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:57.053295 (+    34us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:24:57.053929 (+   634us) tablet.cc:701] Row locks acquired
0709 12:24:57.053952 (+    23us) write_op.cc:260] PREPARE: finished
0709 12:24:57.054105 (+   153us) write_op.cc:270] Start()
0709 12:24:57.054231 (+   126us) write_op.cc:276] Timestamp: P: 1783599897054059 usec, L: 0
0709 12:24:57.054256 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:24:57.055025 (+   769us) log.cc:844] Serialized 2044 byte log entry
0709 12:24:57.057684 (+  2659us) op_driver.cc:464] REPLICATION: finished
0709 12:24:57.058051 (+   367us) write_op.cc:301] APPLY: starting
0709 12:24:57.058133 (+    82us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:57.061036 (+  2903us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:57.061132 (+    96us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:57.073981 (+ 12849us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:57.074177 (+   196us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=0,mrs_lookups=0
0709 12:24:57.074221 (+    44us) write_op.cc:312] APPLY: finished
0709 12:24:57.074450 (+   229us) log.cc:844] Serialized 56 byte log entry
0709 12:24:57.076209 (+  1759us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:57.076313 (+   104us) write_op.cc:454] Released schema lock
0709 12:24:57.076383 (+    70us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:57.100561 (+ 24178us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:57.100598 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":11414,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":118,"apply.run_cpu_time_us":8253,"apply.run_wall_time_us":42483,"cfile_cache_hit":20,"cfile_cache_hit_bytes":20920,"num_ops":5,"prepare.queue_time_us":143,"prepare.run_cpu_time_us":3682,"prepare.run_wall_time_us":6514,"replication_time_us":3287}]]}
W20260709 12:24:57.281879 16441 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 1 similar messages]
I20260709 12:24:57.333274 16219 catalog_manager.cc:5697] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 14 to 19, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 19 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
I20260709 12:24:57.335623 16219 catalog_manager.cc:5697] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 16 to 20, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 20 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
W20260709 12:24:57.336673 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 55248.3us
I20260709 12:24:57.337261 16219 catalog_manager.cc:5697] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 17 to 23. New cstate: current_term: 23 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
I20260709 12:24:57.351969 16219 catalog_manager.cc:5697] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 18 to 20, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 20 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:24:57.355607 16219 catalog_manager.cc:5697] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 16 to 18, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 18 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
W20260709 12:24:57.366928 16515 consensus_peers.cc:597] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:57.367847 16219 catalog_manager.cc:5697] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 25 to 26, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 26 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
I20260709 12:24:57.371694 16219 catalog_manager.cc:5697] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:57.373543 16219 catalog_manager.cc:5697] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
I20260709 12:24:57.384136 16219 catalog_manager.cc:5697] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 21 to 22, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 22 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } }
W20260709 12:24:57.409472 16441 consensus_peers.cc:597] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:57.412424 16441 consensus_peers.cc:597] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:57.412424 16515 consensus_peers.cc:597] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:57.421330 16441 consensus_peers.cc:597] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:57.425434 16225 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53636 (request call id 135) took 22908 ms (client timeout 15000 ms). Trace:
W20260709 12:24:57.425966 16225 rpcz_store.cc:269] 0709 12:24:34.516703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.517100 (+   397us) service_pool.cc:224] Handling call
0709 12:24:57.425219 (+22908119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17759,"reported_tablets":44}
W20260709 12:24:57.434820 16367 consensus_peers.cc:597] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:57.436369 16367 consensus_peers.cc:597] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:57.450484 16515 consensus_peers.cc:597] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:57.484565 16441 consensus_peers.cc:597] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:57.496907 16515 consensus_peers.cc:597] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:57.550160 16221 catalog_manager.cc:5697] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 23 to 24, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 24 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } }
W20260709 12:24:57.563506 16515 consensus_peers.cc:597] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:57.563278 16221 catalog_manager.cc:5697] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 15 to 17, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 17 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:24:57.565388 16221 catalog_manager.cc:5697] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 22 to 24, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 24 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
W20260709 12:24:57.566313 16515 consensus_peers.cc:597] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:57.567119 16221 catalog_manager.cc:5697] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 16 to 21, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 21 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:24:57.568943 16221 catalog_manager.cc:5697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 28 to 29, leader changed from 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3) to ef57beab0e1d40d585983873c55107e8 (127.5.58.1). New cstate: current_term: 29 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:24:57.578056 16221 catalog_manager.cc:5697] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "ef57beab0e1d40d585983873c55107e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } }
I20260709 12:24:57.579566 16221 catalog_manager.cc:5697] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 18 to 20, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 20 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
W20260709 12:24:57.583989 16515 consensus_peers.cc:597] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:57.599857 16441 consensus_peers.cc:597] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:57.610931 16367 consensus_peers.cc:597] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:57.623669 16515 consensus_peers.cc:597] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:57.635573 16515 consensus_peers.cc:597] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:57.637506 16515 consensus_peers.cc:597] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:57.705399 16367 consensus_peers.cc:597] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:57.705464 16515 consensus_peers.cc:597] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:57.721724 16367 consensus_peers.cc:597] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:57.733007 16441 consensus_peers.cc:597] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:57.734022 16515 consensus_peers.cc:597] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:57.737259 16468 raft_consensus.cc:1275] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:24:57.740816 23291 consensus_queue.cc:1048] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:57.794600 16545 raft_consensus.cc:1275] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 12. (term mismatch)
I20260709 12:24:57.795162 16545 pending_rounds.cc:85] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1: Aborting all ops after (but not including) 11
I20260709 12:24:57.795437 16545 pending_rounds.cc:107] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1: Aborting uncommitted NO_OP operation due to leader change: 17.12
I20260709 12:24:57.795789 16545 raft_consensus.cc:2889] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:57.805481 22553 consensus_queue.cc:1048] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:57.817902 16221 catalog_manager.cc:5697] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 17 to 18, leader changed from 610f4feda11942739236f2b121a4b9c1 (127.5.58.4) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 18 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:57.824833 16515 consensus_peers.cc:597] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:57.927040 16222 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53636 (request call id 134) took 47793 ms (client timeout 15000 ms). Trace:
W20260709 12:24:57.934532 16222 rpcz_store.cc:269] 0709 12:24:10.150497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.207571 (+ 57074us) service_pool.cc:224] Handling call
0709 12:24:57.926733 (+47719162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1123048,"reported_tablets":44}
I20260709 12:24:58.217772 23344 raft_consensus.cc:493] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:58.218411 23344 raft_consensus.cc:515] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:58.221231 23344 leader_election.cc:290] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:58.222201 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:58.221390 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cd596d6e69a4b58804e76d2dd4f51f1" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:58.227952 17144 leader_election.cc:304] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:58.228755 23344 raft_consensus.cc:2749] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:24:58.236227 16226 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53626 (request call id 128) took 90773 ms (client timeout 15000 ms). Trace:
I20260709 12:24:58.254740 23344 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:58.268141 23344 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:58.271376 23344 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:58.272037 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:58.272346 16226 rpcz_store.cc:269] 0709 12:23:27.366698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:27.367781 (+  1083us) service_pool.cc:224] Handling call
0709 12:24:58.139619 (+90771838us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27449,"reported_tablets":44}
I20260709 12:24:58.273056 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7889f3ab253b473eafb86002e9612a25" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:58.275518 17143 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:58.276382 22927 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:58.322827 22927 raft_consensus.cc:493] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:58.323395 22927 raft_consensus.cc:515] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:58.326026 22927 leader_election.cc:290] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:58.326467 23344 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:58.327250 23344 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:58.327348 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:58.329927 23344 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:58.331421 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:58.342176 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f9bc0dbb7e4c6f94aaf8146768ed79" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:58.345810 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701eedfc29544bbeafd695f04ccea545" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:58.348531 17143 leader_election.cc:304] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:58.349792 22927 raft_consensus.cc:2749] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:58.350291 17143 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:58.351155 23344 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:58.416208 22927 raft_consensus.cc:493] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:58.416836 22927 raft_consensus.cc:515] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:58.421245 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:58.423583 23344 raft_consensus.cc:493] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:58.424259 23344 raft_consensus.cc:515] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:58.426584 23344 leader_election.cc:290] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:58.429068 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bb7e1a96a94612b2f6759156f5f7c0" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:58.432243 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:58.435446 22927 leader_election.cc:290] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:58.436501 17143 leader_election.cc:304] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:58.438165 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b76873ad2645ebb97d6f5b03655773" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:58.438735 23400 raft_consensus.cc:2749] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:58.439489 23400 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:58.439940 23400 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:58.440011 17142 leader_election.cc:304] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:58.445420 23400 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:58.443924 23344 raft_consensus.cc:2749] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:24:58.447407 16220 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59300 (request call id 87) took 69849 ms (client timeout 15000 ms). Trace:
I20260709 12:24:58.448096 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:24:58.448041 16220 rpcz_store.cc:269] 0709 12:23:48.597968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.739438 (+141470us) service_pool.cc:224] Handling call
0709 12:24:58.447284 (+69707846us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":860511,"reported_tablets":45}
I20260709 12:24:58.450462 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2546c2ffdb52493c972a8bf000c78ed5" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:58.453191 17142 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:58.454533 23344 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:24:58.463406 16224 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53626 (request call id 131) took 23492 ms (client timeout 15000 ms). Trace:
I20260709 12:24:58.463515 22927 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:58.464407 22927 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:24:58.464233 16224 rpcz_store.cc:269] 0709 12:24:34.970908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.972526 (+  1618us) service_pool.cc:224] Handling call
0709 12:24:58.463195 (+23490669us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80737,"reported_tablets":44}
I20260709 12:24:58.466722 22927 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:58.467650 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:58.470003 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2476e68a0d4ae695099a41865c99c3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:58.472537 17143 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:58.472755 23344 raft_consensus.cc:493] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:58.473366 22927 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:58.473295 23344 raft_consensus.cc:515] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:58.476238 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:58.475570 23344 leader_election.cc:290] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:58.482461 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "162cebc248914fb5aa57de3a383a4159" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:58.484133 17142 leader_election.cc:304] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:58.488507 23344 raft_consensus.cc:2749] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:58.500763 22927 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:58.501410 22927 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:58.506649 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:58.526432 23400 raft_consensus.cc:493] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:58.527072 23400 raft_consensus.cc:515] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:58.529341 23400 leader_election.cc:290] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:58.530265 22927 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:58.531126 22927 raft_consensus.cc:493] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:58.531762 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:58.533352 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f88a73c287745feb816089d0e08d10b" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:58.532680 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c8566780d1400ab2e42d9529d97ceb" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:58.534256 22927 raft_consensus.cc:515] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:58.536062 17142 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:24:58.536705 16218 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59300 (request call id 88) took 47290 ms (client timeout 15000 ms). Trace:
W20260709 12:24:58.537148 16218 rpcz_store.cc:269] 0709 12:24:11.246583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.254971 (+  8388us) service_pool.cc:224] Handling call
0709 12:24:58.536607 (+47281636us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7136,"reported_tablets":45}
I20260709 12:24:58.538479 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:58.540437 23400 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:58.542565 17144 leader_election.cc:304] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:58.544063 23400 raft_consensus.cc:2749] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:58.545181 23344 raft_consensus.cc:493] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:58.545682 23344 raft_consensus.cc:515] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:58.548053 23344 leader_election.cc:290] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:58.553188 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:58.571347 22927 leader_election.cc:290] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:58.573046 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da151f033444a308804c713dd5ab691" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:58.574785 17143 leader_election.cc:304] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:58.576283 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "191e052863dd48fe92ebe151197330a8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:58.578317 23405 raft_consensus.cc:2749] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:58.578886 23400 raft_consensus.cc:493] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:58.579468 23400 raft_consensus.cc:515] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:58.583175 23400 leader_election.cc:290] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:58.584281 17144 leader_election.cc:304] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:58.584595 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:58.586117 23400 raft_consensus.cc:2749] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:24:58.587563  5352 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 17300)
I20260709 12:24:58.587378 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48bd85a0339b41fba1759ca092d6e95e" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:58.592849 22927 raft_consensus.cc:493] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:58.594061 17142 leader_election.cc:304] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:58.595603 23405 raft_consensus.cc:2749] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:58.593510 22927 raft_consensus.cc:515] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:24:58.610481 16217 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59300 (request call id 89) took 21581 ms (client timeout 15000 ms). Trace:
I20260709 12:24:58.610785 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:24:58.610986 16217 rpcz_store.cc:269] 0709 12:24:37.035054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.088291 (+ 53237us) service_pool.cc:224] Handling call
0709 12:24:58.610396 (+21522105us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16157,"reported_tablets":45}
I20260709 12:24:58.613922 22927 leader_election.cc:290] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:58.615772 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6474b10917204b6994269be17387f38d" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:58.618023 17144 leader_election.cc:304] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:58.618884 22927 raft_consensus.cc:2749] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:58.636766 22927 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:58.637341 22927 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:58.643635 22927 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:58.644117 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:58.646113 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a154d032654a179da41ad8f10fa339" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:58.648967 17142 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:58.650271 23405 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:58.686122 22927 raft_consensus.cc:493] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:58.686789 22927 raft_consensus.cc:515] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:58.702877 23400 raft_consensus.cc:493] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:58.703531 23400 raft_consensus.cc:515] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:58.705708 23400 leader_election.cc:290] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:58.708413 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:58.710697 16223 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59300 (request call id 90) took 4653 ms. Trace:
I20260709 12:24:58.711133 16223 rpcz_store.cc:276] 0709 12:24:54.057775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.529862 (+2472087us) service_pool.cc:224] Handling call
0709 12:24:58.710640 (+2180778us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20931,"reported_tablets":45}
I20260709 12:24:58.711956 23344 raft_consensus.cc:493] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:58.712700 23344 raft_consensus.cc:515] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:58.714986 23344 leader_election.cc:290] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:58.714771 23405 raft_consensus.cc:493] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:58.716933 23405 raft_consensus.cc:515] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:58.721457 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:58.723908 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edff15255cdf492dbc9c1189b889aa78" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:58.734977 23406 raft_consensus.cc:493] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:24:58.737424 23406 raft_consensus.cc:515] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:58.744864 22927 leader_election.cc:290] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:58.747143 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:58.719389 23405 leader_election.cc:290] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:58.755496 17144 leader_election.cc:304] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:58.756467 22927 raft_consensus.cc:2749] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:58.762617 22927 raft_consensus.cc:493] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 610f4feda11942739236f2b121a4b9c1)
I20260709 12:24:58.763167 22927 raft_consensus.cc:515] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:24:58.765170 22927 leader_election.cc:290] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:24:58.774827 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "766cb112f8024db5af150d6be611a597" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:58.774998 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:58.776749 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:58.776788 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:58.775319 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "025780567e194437bc14e26465b690e3" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:58.779028  5352 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:24:58.780827 22927 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:58.782274  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:58.783237 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de44585291c44bfe9a6f7168d04169b8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:58.781351 22927 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:24:58.784899 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81a29889843548dd6ed149807ecc8" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:58.785202 22927 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:58.783468  5352 raft_consensus.cc:2243] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:24:58.788321 16545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:24:58.789211 23344 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:24:58.789855 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434028125a2d48998f4f5106c59a9d47" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:58.789737 23344 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:24:58.793195 23344 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:24:58.795984 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:24:58.800094  5352 raft_consensus.cc:2272] T 7fe8595f696842c8821880524389c68b P ef57beab0e1d40d585983873c55107e8 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:58.773008 23406 leader_election.cc:290] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:24:58.813087  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:58.813112 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d860a1605704f96b47bc65033809793" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:58.814635 17143 leader_election.cc:304] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:58.816619 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e768efb7fcef4624b08d1e122c6dc825" candidate_uuid: "ef57beab0e1d40d585983873c55107e8" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:24:58.817389 23344 raft_consensus.cc:2749] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:58.818416  5352 raft_consensus.cc:2243] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:58.819216  5352 raft_consensus.cc:2272] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:58.817077 17142 leader_election.cc:304] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:58.824962 23406 raft_consensus.cc:2749] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:58.826325 17143 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:58.827203 23406 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:58.835449 17142 leader_election.cc:304] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:58.836478 23406 raft_consensus.cc:2749] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:58.838963 17142 leader_election.cc:304] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:58.840317 23406 raft_consensus.cc:2749] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:58.843752 17142 leader_election.cc:304] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1
I20260709 12:24:58.844650 23406 raft_consensus.cc:2749] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:58.844887  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:58.846014  5352 raft_consensus.cc:2243] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:58.846055 17142 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef57beab0e1d40d585983873c55107e8; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:24:58.846836  5352 raft_consensus.cc:2272] T 2546c2ffdb52493c972a8bf000c78ed5 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:58.847062 23406 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:58.867304  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:58.868152  5352 raft_consensus.cc:2243] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:24:58.869334  5352 raft_consensus.cc:2272] T ef53202d665e46c39113572380be6c36 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:58.872550  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:58.873394  5352 raft_consensus.cc:2243] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:24:58.876525  5352 raft_consensus.cc:2272] T 33037c31b4434e838d90b2a2065ef52d P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:58.879755  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:58.880759  5352 raft_consensus.cc:2243] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:24:58.892190  5352 raft_consensus.cc:2272] T 3c4fd014cdf249c996641fe7793f42a5 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:58.902447  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:58.903438  5352 raft_consensus.cc:2243] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:24:58.904714  5352 raft_consensus.cc:2272] T a60dbe0067044806b384e90805c877bc P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:58.923291  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:58.924538  5352 raft_consensus.cc:2243] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:24:58.927933  5352 raft_consensus.cc:2272] T 7117f6656fa14807ac801392259e843a P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:58.951778  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:58.952718  5352 raft_consensus.cc:2243] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:58.953308  5352 raft_consensus.cc:2272] T ad2476e68a0d4ae695099a41865c99c3 P ef57beab0e1d40d585983873c55107e8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:58.967233  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:58.968165  5352 raft_consensus.cc:2243] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:24:58.969461  5352 raft_consensus.cc:2272] T aadc86310ab642c69b7dbbf94be01070 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:58.989697  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:58.990618  5352 raft_consensus.cc:2243] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:24:58.991707  5352 raft_consensus.cc:2272] T 3851c76367ba4b3eaf3a1b5833ef4835 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.005082  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.006268  5352 raft_consensus.cc:2243] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:59.006964  5352 raft_consensus.cc:2272] T 5f88a73c287745feb816089d0e08d10b P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.010803  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.011786  5352 raft_consensus.cc:2243] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 12 LEADER]: Raft consensus shutting down.
I20260709 12:24:59.031698  5352 raft_consensus.cc:2272] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P ef57beab0e1d40d585983873c55107e8 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.040994  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.041934  5352 raft_consensus.cc:2243] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:59.042626  5352 raft_consensus.cc:2272] T 2da151f033444a308804c713dd5ab691 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.072590  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.073681  5352 raft_consensus.cc:2243] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:59.074524  5352 raft_consensus.cc:2272] T 95b81a29889843548dd6ed149807ecc8 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.087653  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.093370  5352 raft_consensus.cc:2243] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:59.094422  5352 raft_consensus.cc:2272] T a4bb7e1a96a94612b2f6759156f5f7c0 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.103230  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.104048  5352 raft_consensus.cc:2243] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:59.110210  5352 raft_consensus.cc:2272] T 9cd596d6e69a4b58804e76d2dd4f51f1 P ef57beab0e1d40d585983873c55107e8 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.117797  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.118635  5352 raft_consensus.cc:2243] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:59.125775  5352 raft_consensus.cc:2272] T edff15255cdf492dbc9c1189b889aa78 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.133767  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.134943  5352 raft_consensus.cc:2243] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:24:59.135998  5352 raft_consensus.cc:2272] T 44b9e8076a1d43319b0c86d4b356ac68 P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.147704  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.161897  5352 raft_consensus.cc:2243] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:59.162622  5352 raft_consensus.cc:2272] T 701eedfc29544bbeafd695f04ccea545 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.171927  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.178303  5352 raft_consensus.cc:2243] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:59.179041  5352 raft_consensus.cc:2272] T c0b76873ad2645ebb97d6f5b03655773 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.203276  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.209944  5352 raft_consensus.cc:2243] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:59.210561  5352 raft_consensus.cc:2272] T e768efb7fcef4624b08d1e122c6dc825 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.234934  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.235952  5352 raft_consensus.cc:2243] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:59.236769  5352 raft_consensus.cc:2272] T de44585291c44bfe9a6f7168d04169b8 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.271708  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.272943  5352 raft_consensus.cc:2243] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:24:59.279072  5352 raft_consensus.cc:2272] T 3ba84aa22298489eb2845bec782a34f2 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.285176  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.285985  5352 raft_consensus.cc:2243] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:59.286571  5352 raft_consensus.cc:2272] T 025780567e194437bc14e26465b690e3 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.305621  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.306455  5352 raft_consensus.cc:2243] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:24:59.307654  5352 raft_consensus.cc:2272] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.310796  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.311808  5352 raft_consensus.cc:2243] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:59.312608  5352 raft_consensus.cc:2272] T 48bd85a0339b41fba1759ca092d6e95e P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.316227  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.317200  5352 raft_consensus.cc:2243] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:24:59.319618  5352 raft_consensus.cc:2272] T defc374629a649a8a47502f058be2c8c P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.323343  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.324275  5352 raft_consensus.cc:2243] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:59.324962  5352 raft_consensus.cc:2272] T 162cebc248914fb5aa57de3a383a4159 P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.329166  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.330104  5352 raft_consensus.cc:2243] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:59.330804  5352 raft_consensus.cc:2272] T c1a154d032654a179da41ad8f10fa339 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.334542  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.335445  5352 raft_consensus.cc:2243] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:59.336163  5352 raft_consensus.cc:2272] T 6474b10917204b6994269be17387f38d P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.339468  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.340428  5352 raft_consensus.cc:2243] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:24:59.341477  5352 raft_consensus.cc:2272] T 472c1e509cf2401e8153946529cfba2e P ef57beab0e1d40d585983873c55107e8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.345171  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.346069  5352 raft_consensus.cc:2243] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:59.346809  5352 raft_consensus.cc:2272] T 05c8566780d1400ab2e42d9529d97ceb P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.350171  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.351305  5352 raft_consensus.cc:2243] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:59.352401  5352 raft_consensus.cc:2272] T 191e052863dd48fe92ebe151197330a8 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.356083  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.356988  5352 raft_consensus.cc:2243] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:59.357616  5352 raft_consensus.cc:2272] T 766cb112f8024db5af150d6be611a597 P ef57beab0e1d40d585983873c55107e8 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.360554  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.361380  5352 raft_consensus.cc:2243] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:59.362010  5352 raft_consensus.cc:2272] T 434028125a2d48998f4f5106c59a9d47 P ef57beab0e1d40d585983873c55107e8 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.364991  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.365794  5352 raft_consensus.cc:2243] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:24:59.367338  5352 raft_consensus.cc:2272] T 4777f1999cdc462c8526206c2f39fe0b P ef57beab0e1d40d585983873c55107e8 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.371165  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.371982  5352 raft_consensus.cc:2243] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:24:59.373073  5352 raft_consensus.cc:2272] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P ef57beab0e1d40d585983873c55107e8 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.376452  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.377260  5352 raft_consensus.cc:2243] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:24:59.420265  5352 raft_consensus.cc:2272] T 1215b509464e4efe977922ffdeb0b064 P ef57beab0e1d40d585983873c55107e8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.437489  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.438287  5352 raft_consensus.cc:2243] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Raft consensus shutting down.
W20260709 12:24:59.444612 23344 consensus_queue.cc:1175] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f99066d13e524238aec6aa2024a8bd50" responder_term: 18 status { last_received { term: 18 index: 9 } last_committed_idx: 9 last_received_current_leader { term: 18 index: 9 } }
I20260709 12:24:59.445519  5352 raft_consensus.cc:2272] T dbdcb11d24644ca0bef26ea87647f4ac P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.454728  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.455767  5352 raft_consensus.cc:2243] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:59.456651  5352 raft_consensus.cc:2272] T 6d860a1605704f96b47bc65033809793 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.465397  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.466302  5352 raft_consensus.cc:2243] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:24:59.467501  5352 raft_consensus.cc:2272] T 14d8123994de43ec8e36a4e02961bc90 P ef57beab0e1d40d585983873c55107e8 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.470875  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.471722  5352 raft_consensus.cc:2243] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:59.478677  5352 raft_consensus.cc:2272] T 7889f3ab253b473eafb86002e9612a25 P ef57beab0e1d40d585983873c55107e8 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.481792  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.482604  5352 raft_consensus.cc:2243] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:24:59.483783  5352 raft_consensus.cc:2272] T 6a90f33738a74f2c9fbf18bb0199dccf P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.486872  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:24:59.487633  5352 raft_consensus.cc:2243] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:24:59.492002  5352 raft_consensus.cc:2272] T 49b3f77bde864e0ab25c63401793c709 P ef57beab0e1d40d585983873c55107e8 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:59.600418 16441 connection.cc:573] client connection to 127.5.58.1:41805 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 12:24:59.662796 16441 consensus_peers.cc:597] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:59.716009 16441 consensus_peers.cc:597] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:59.750949 16367 consensus_peers.cc:597] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:59.797930 16441 consensus_peers.cc:597] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:59.799857 16367 consensus_peers.cc:597] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:59.923408 16515 consensus_peers.cc:597] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:59.944162 16515 consensus_peers.cc:597] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:59.952638 16515 consensus_peers.cc:597] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:24:59.984833 16515 consensus_peers.cc:597] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:00.002499 23398 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.003741 23398 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:25:00.017311 16367 leader_election.cc:336] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.021134 23409 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.022806 23398 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:25:00.022388 23409 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:25:00.025810 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:25:00.028146 16366 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:25:00.029824 23291 raft_consensus.cc:2749] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:00.035328 23409 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:25:00.043565 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:25:00.050084 16515 consensus_peers.cc:597] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:00.052850 16441 leader_election.cc:336] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.060523 23409 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.060556 16441 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:25:00.061296 23409 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:00.062376 23299 raft_consensus.cc:2749] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:00.065371 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:25:00.066054 16394 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 21.
I20260709 12:25:00.067194 16442 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:25:00.068395 23299 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:00.068502 23409 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:25:00.070540 23299 raft_consensus.cc:493] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.071014 23398 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.071350 23299 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:00.071702 23398 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:00.074877 23398 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:25:00.066303 16515 consensus_peers.cc:597] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:00.078986 16367 leader_election.cc:336] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.080538 23299 raft_consensus.cc:515] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:00.082074 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:25:00.083630 23299 leader_election.cc:290] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:25:00.083804 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c4fd014cdf249c996641fe7793f42a5" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:25:00.084574 16394 raft_consensus.cc:3060] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:25:00.091372 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 55228.6us
I20260709 12:25:00.093693 16394 raft_consensus.cc:2468] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 22.
I20260709 12:25:00.095100 16442 leader_election.cc:304] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:25:00.095711 23410 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.096369 23299 raft_consensus.cc:2804] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:00.098235 23299 raft_consensus.cc:697] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:25:00.099520 23299 consensus_queue.cc:237] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:25:00.083390 16441 consensus_peers.cc:597] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:00.103886 16441 leader_election.cc:336] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.106354 23410 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:25:00.108983 23410 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:25:00.114344 23410 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.115042 23410 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:25:00.120676 16366 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:25:00.123553 23398 raft_consensus.cc:2749] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:25:00.124892 16515 consensus_peers.cc:597] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:00.132230 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:25:00.140399 23410 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:25:00.144923 16441 leader_election.cc:336] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:25:00.145998 16367 consensus_peers.cc:597] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:00.146993 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:25:00.152210 23226 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.152747 23226 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:25:00.154441 23226 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:25:00.156785 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:25:00.157394 16474 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 17.
W20260709 12:25:00.157832 16515 consensus_peers.cc:597] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:00.158365 16514 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
W20260709 12:25:00.158993 16515 consensus_peers.cc:597] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:00.160051 16515 leader_election.cc:336] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.160452 16515 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:00.161372 23226 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:00.159755 23416 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:00.162532 23416 raft_consensus.cc:493] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.163038 23416 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:25:00.164845 16515 leader_election.cc:336] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.165324 16515 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:00.166023 23226 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:25:00.172009 16515 leader_election.cc:336] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.175899 23416 raft_consensus.cc:515] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:25:00.178042 16223 catalog_manager.cc:5697] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 21 to 22, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 22 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:00.191927 16441 consensus_peers.cc:597] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:00.200111 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60dbe0067044806b384e90805c877bc" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:25:00.200834 16474 raft_consensus.cc:3060] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:00.200362 23416 leader_election.cc:290] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:25:00.209249 16474 raft_consensus.cc:2468] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 18.
I20260709 12:25:00.210731 16514 leader_election.cc:304] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:25:00.212293 23416 raft_consensus.cc:2804] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:00.212863 23416 raft_consensus.cc:697] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 18 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:25:00.214085 23416 consensus_queue.cc:237] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:25:00.222477 16515 consensus_peers.cc:597] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:00.223119 16515 leader_election.cc:336] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.223549 23394 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.224045 23409 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.224189 23394 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:25:00.224622 23409 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:00.226419 23394 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:25:00.226584 23409 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:25:00.228206 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:25:00.228673 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:25:00.231866 23398 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.232615 23398 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:25:00.232987 16441 leader_election.cc:336] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.233460 16441 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
W20260709 12:25:00.234337 16441 leader_election.cc:336] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.235430 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:25:00.235592 16441 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:25:00.236291 23394 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:00.237128 23394 raft_consensus.cc:2749] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:00.237954 23398 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:25:00.243310 23291 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:25:00.243395 16367 leader_election.cc:336] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.243912 23291 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:25:00.244551 16367 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:25:00.245939 23398 raft_consensus.cc:2749] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:00.246130 23291 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:25:00.266955 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:25:00.267707 16549 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 17.
I20260709 12:25:00.267704 16221 catalog_manager.cc:5697] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 17 to 18, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 18 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:00.273465 23398 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.274101 23398 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:00.277352 23398 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:25:00.278311 23394 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.279099 23394 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:00.282850 23394 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:25:00.286624 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:25:00.287403 16394 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 17.
I20260709 12:25:00.288488 16367 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:25:00.292353 23398 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:00.292799 23398 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.293224 23398 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:00.294553 23421 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.295233 23421 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:25:00.296885 16367 leader_election.cc:336] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.297042 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:25:00.298890 23291 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.299535 23291 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:00.301759 23291 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:25:00.297029 16441 leader_election.cc:336] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.306840 23421 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:25:00.310869 16442 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:25:00.312021 23409 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:00.312441 23409 raft_consensus.cc:493] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.312870 23409 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:00.320041 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:25:00.328536 16441 consensus_peers.cc:597] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:00.330288 23398 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:25:00.333837 23394 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:25:00.335644 16515 consensus_peers.cc:597] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:00.334739 23394 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:00.339823 23394 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:25:00.342572 23226 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.343149 23226 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:25:00.345311 23226 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:25:00.347338 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
W20260709 12:25:00.348026 16441 leader_election.cc:336] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.348201 23226 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.348742 23226 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:25:00.348078 16474 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 26.
I20260709 12:25:00.351811 16514 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:25:00.352952 23299 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.353542 23299 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:25:00.354025 23415 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.349035 23398 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:25:00.354657 23415 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:25:00.358814 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:25:00.360558 23415 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:25:00.353011 23416 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:00.365389 23416 raft_consensus.cc:493] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 26 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.366586 23416 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:00.371141 23409 raft_consensus.cc:515] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:00.363767 23415 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.372310 23415 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:25:00.365483 23226 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:25:00.365382 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:25:00.373756 16394 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 18.
I20260709 12:25:00.375097 16516 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:25:00.375404 23415 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:25:00.376252 23414 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:00.376763 23414 raft_consensus.cc:493] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.377156 23414 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:00.381378 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:25:00.364784 23299 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:25:00.382164 16474 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 12.
I20260709 12:25:00.383901 16514 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:25:00.385696 23410 raft_consensus.cc:2804] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:00.386103 23410 raft_consensus.cc:493] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.386508 23410 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:00.391373 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:25:00.392132 16549 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 21.
I20260709 12:25:00.395021 23414 raft_consensus.cc:515] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:25:00.399256 16515 leader_election.cc:336] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.399667 23410 raft_consensus.cc:515] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:25:00.402694 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef53202d665e46c39113572380be6c36" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:25:00.403566 16549 raft_consensus.cc:3060] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:00.404440 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:25:00.405103 16474 raft_consensus.cc:3060] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:00.408859 16441 leader_election.cc:336] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.410995 16549 raft_consensus.cc:2468] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 18.
I20260709 12:25:00.421838 23410 leader_election.cc:290] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:25:00.423425 16474 raft_consensus.cc:2468] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 13.
I20260709 12:25:00.391160 23416 raft_consensus.cc:515] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:25:00.416283 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7117f6656fa14807ac801392259e843a" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:25:00.426222 16394 raft_consensus.cc:3060] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:00.428085 23416 leader_election.cc:290] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 27 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:25:00.434717 16441 leader_election.cc:336] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.435822 16394 raft_consensus.cc:2468] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 19.
I20260709 12:25:00.416221 23299 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:25:00.400089 16515 leader_election.cc:336] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.436602 23299 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:25:00.437177 16515 leader_election.cc:336] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.437647 16515 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:00.438781 23299 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:25:00.439270 23410 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:00.417268 23409 leader_election.cc:290] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:25:00.419775 23414 leader_election.cc:290] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:25:00.424741 16514 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:25:00.441756 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fe8595f696842c8821880524389c68b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:25:00.442337 16469 raft_consensus.cc:3060] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:00.444655 16516 leader_election.cc:304] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:25:00.446156 23416 raft_consensus.cc:2804] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:25:00.446477 16441 leader_election.cc:336] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.446645 23416 raft_consensus.cc:697] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 13 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:25:00.447584 23416 consensus_queue.cc:237] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:25:00.413331 16367 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:25:00.454243 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:25:00.455120 16398 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 20.
I20260709 12:25:00.460014 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:25:00.462396 16396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33037c31b4434e838d90b2a2065ef52d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:25:00.456163 23398 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:00.463188 23398 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.463812 23398 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:00.468532 23415 raft_consensus.cc:2804] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:00.463124 16396 raft_consensus.cc:3060] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:25:00.470358 16367 leader_election.cc:336] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.471673 16469 raft_consensus.cc:2468] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 27.
I20260709 12:25:00.469164 23415 raft_consensus.cc:697] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 19 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:25:00.473464 23226 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.474560 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c02e60dcbc14b2fa3a2c11a214a93a6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:25:00.474282 23226 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:25:00.473774 23415 consensus_queue.cc:237] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:25:00.476739 23394 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.481639 23398 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:25:00.482286 23394 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:00.484643 23421 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.477090 23226 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:25:00.485363 23421 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:25:00.486287 16515 leader_election.cc:336] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.487459 23394 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:25:00.478174 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:25:00.471211 16367 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:25:00.490653 16441 leader_election.cc:304] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:25:00.491046 23421 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:25:00.491884 23291 raft_consensus.cc:2749] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:00.492063 23299 raft_consensus.cc:2749] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:00.495545 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:25:00.496089 16469 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 21.
I20260709 12:25:00.497213 16366 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:25:00.497469 16515 leader_election.cc:336] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.499426 16367 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:25:00.499457 16515 leader_election.cc:336] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.504232 16396 raft_consensus.cc:2468] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 18.
I20260709 12:25:00.501142 16514 leader_election.cc:304] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
W20260709 12:25:00.505841 16515 leader_election.cc:336] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.506464 23226 raft_consensus.cc:2804] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:25:00.506613 16515 consensus_peers.cc:597] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:00.506953 23226 raft_consensus.cc:697] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 27 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
W20260709 12:25:00.507146 16515 leader_election.cc:336] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.508512 16515 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:00.513461 23291 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:00.514068 23291 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.514544 23291 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:00.511029 23226 consensus_queue.cc:237] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.13, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:25:00.534516 16442 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
W20260709 12:25:00.540798 16441 leader_election.cc:336] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.523532 23414 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:25:00.567188 16441 consensus_peers.cc:597] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:00.568811 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 71722.6us
I20260709 12:25:00.539851 23398 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:25:00.564663 23394 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:00.574893 23421 raft_consensus.cc:2804] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:00.577095 23421 raft_consensus.cc:697] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 18 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:25:00.569118 23426 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.581287 23433 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.598861 23426 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:25:00.599200 23433 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:00.575544 23394 raft_consensus.cc:493] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.600688 23394 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:00.601809 23433 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:25:00.602790 16221 catalog_manager.cc:5697] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 12 to 13, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 13 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:00.604801 23421 consensus_queue.cc:237] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:25:00.604681 16221 catalog_manager.cc:5697] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 18 to 19, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 19 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:00.607345 23226 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.607992 23226 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:25:00.632191  5352 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 17296)
I20260709 12:25:00.623479 23291 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:00.636912 23291 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:25:00.638485 23426 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:25:00.645687 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 66746.4us
I20260709 12:25:00.650019 23430 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.650751 23430 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:25:00.659898 16515 consensus_peers.cc:597] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:00.661162 16515 leader_election.cc:336] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:25:00.664319 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 164945us
I20260709 12:25:00.665371 16442 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:00.667443 23226 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:25:00.667309 23427 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:00.670171 23430 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:25:00.671784 16442 leader_election.cc:304] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:25:00.674001 23430 raft_consensus.cc:2804] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:00.674528 23430 raft_consensus.cc:697] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 18 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:25:00.675438 23430 consensus_queue.cc:237] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:25:00.680217 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 103134us
I20260709 12:25:00.681757 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:25:00.682441 16474 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 29.
I20260709 12:25:00.685664 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44b9e8076a1d43319b0c86d4b356ac68" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:25:00.686321 16474 raft_consensus.cc:3060] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:25:00.697795 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 91206.2us
I20260709 12:25:00.697956 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:25:00.699666 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadc86310ab642c69b7dbbf94be01070" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:25:00.700242 16549 raft_consensus.cc:3060] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:00.711498 16366 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:25:00.716744 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:25:00.722056 23299 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.722757 23299 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
W20260709 12:25:00.724292 16441 leader_election.cc:336] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.678049 23427 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.726569 16549 raft_consensus.cc:2468] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 22.
I20260709 12:25:00.730639 23226 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.731137 23226 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:25:00.731673 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:25:00.732935 16474 raft_consensus.cc:2468] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 22.
I20260709 12:25:00.734220 16469 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 17.
W20260709 12:25:00.736533 16515 leader_election.cc:336] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.737480 16515 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:00.724972 23299 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:25:00.735755 16514 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:25:00.744300 16394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:25:00.746536 16396 raft_consensus.cc:1275] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:25:00.749140 23394 raft_consensus.cc:515] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:00.749387 23437 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.750317 23437 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:25:00.751843 23291 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:00.752358 23291 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 29 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.752791 23291 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:00.753286 23437 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:25:00.758453 16367 leader_election.cc:336] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.759328 16367 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:25:00.760267 23421 raft_consensus.cc:2749] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:25:00.761148 16367 consensus_peers.cc:597] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:00.762917 16367 leader_election.cc:336] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:25:00.768925 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 57506.4us
W20260709 12:25:00.767546 16367 leader_election.cc:336] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:25:00.770905 16367 consensus_peers.cc:597] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:00.786490 16367 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:25:00.786607 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 50910.3us
I20260709 12:25:00.798923 23440 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:00.799350 23433 raft_consensus.cc:2804] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:00.799894 23433 raft_consensus.cc:697] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 22 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:25:00.736138 23427 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:00.798276 23426 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:00.808733 23426 raft_consensus.cc:493] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.809158 23426 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:00.785880 23410 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.812839 23410 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:25:00.815316 23410 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:25:00.829391 23433 consensus_queue.cc:237] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:25:00.835954 23438 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.836766 23438 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:25:00.838887 23438 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:25:00.842988 23438 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.843526 23438 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:00.847929 23426 raft_consensus.cc:515] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:25:00.858734 16474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:25:00.859675 16474 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 20.
I20260709 12:25:00.859740 23428 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
W20260709 12:25:00.864383 23445 log.cc:927] Time spent T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:25:00.866387 23299 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.867030 23299 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:25:00.869184 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 132739us
I20260709 12:25:00.868152 16366 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:25:00.873256 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:25:00.874138 16549 raft_consensus.cc:2393] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 18: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:25:00.874380 23226 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:25:00.876863 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 90432.4us
I20260709 12:25:00.860466 23428 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:00.874467 23421 raft_consensus.cc:2804] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:00.884758 23421 raft_consensus.cc:697] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 22 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:25:00.888571 23444 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.889456 23299 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:25:00.890005 23299 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:00.890470 23299 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:00.891479 23394 leader_election.cc:290] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:25:00.892545 23299 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:25:00.889170 23444 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:25:00.896648 23428 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:25:00.897853 23421 consensus_queue.cc:237] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:00.903173 16221 catalog_manager.cc:5697] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 17 to 18, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 18 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:00.917541 23426 leader_election.cc:290] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:25:00.921877 23438 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:25:00.923395 23427 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:25:00.938800 16515 consensus_peers.cc:597] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:00.939455 16367 leader_election.cc:336] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:25:00.939630 16515 leader_election.cc:336] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:25:00.940320 16367 leader_election.cc:336] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:25:00.941259 16367 leader_election.cc:336] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.915482 23291 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:25:00.942133 16367 leader_election.cc:336] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:00.947988 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:25:00.948395 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3851c76367ba4b3eaf3a1b5833ef4835" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:25:00.948834 16395 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 15.
I20260709 12:25:00.949137 16398 raft_consensus.cc:3060] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:00.950176 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:25:00.950822 16395 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 20.
I20260709 12:25:00.953781 16544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:25:00.954459 16544 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 22.
I20260709 12:25:00.958266 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:25:00.957957 16217 catalog_manager.cc:5697] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 26 to 27, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 27 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:00.960878 23444 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:25:00.952730 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:25:00.965447 16442 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:00.977305 16395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:25:00.978061 16395 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 20.
I20260709 12:25:00.962834 16399 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 21.
I20260709 12:25:00.985560 16516 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:25:00.944196 23291 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 30 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:25:00.992676 16546 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:25:00.993360 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4777f1999cdc462c8526206c2f39fe0b" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:25:00.994040 16476 raft_consensus.cc:3060] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:00.993856 16223 catalog_manager.cc:5697] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 17 to 18, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 18 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:00.995493 16474 raft_consensus.cc:1275] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:25:00.982254 23427 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:01.005285 23452 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:01.006081 23452 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:25:00.988142 23299 consensus_queue.cc:1048] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 12:25:01.008503 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:25:01.009423 16469 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 21.
I20260709 12:25:01.013846 16367 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:25:00.989243 23426 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:01.033025 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:25:01.034245 16514 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:25:01.034456 16549 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 18.
W20260709 12:25:01.029325 16515 leader_election.cc:336] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:01.047932 16476 raft_consensus.cc:2468] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 18.
I20260709 12:25:01.022554 23421 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:01.051062 23421 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:01.051563 23421 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:01.049793 16543 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 18.
I20260709 12:25:01.050398 16367 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:25:01.056761 23438 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:01.056571 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:25:01.057201 23438 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:01.057451 16469 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 18.
I20260709 12:25:01.057654 23438 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:01.036168 23452 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:25:01.059974 16367 leader_election.cc:336] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:25:01.060773 16367 leader_election.cc:336] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:25:01.061621 16367 leader_election.cc:336] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:25:01.070178 16367 leader_election.cc:336] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:01.070470 16441 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
W20260709 12:25:01.070869 16367 leader_election.cc:336] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:01.074759 16469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff57aebc0fb4c6dbb6077f83ac8a60e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:25:01.075464 16469 raft_consensus.cc:3060] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:01.043720 23443 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:01.077135 23443 raft_consensus.cc:493] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:01.077499 23443 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:01.080830 23447 raft_consensus.cc:2749] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:01.082726 23421 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:25:01.026607 23426 raft_consensus.cc:493] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:01.084610 23426 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:25:01.086709 16367 leader_election.cc:336] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:25:01.087426 16367 leader_election.cc:336] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:25:01.091220 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 49616.7us
I20260709 12:25:01.092149 16367 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:25:01.092478 16398 raft_consensus.cc:2468] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 21.
I20260709 12:25:01.095094 23291 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:01.095611 23291 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:01.095979 23291 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:01.095822 23438 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:25:01.100148 16543 raft_consensus.cc:1275] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:25:01.100407 16399 raft_consensus.cc:1275] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:25:01.105155 23421 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 23 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:25:01.107816 16515 leader_election.cc:336] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:25:01.108738 16515 consensus_peers.cc:597] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:01.109822 16469 raft_consensus.cc:2468] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 30.
I20260709 12:25:01.110065 16366 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
W20260709 12:25:01.110075 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 125407us
W20260709 12:25:01.113564 16441 leader_election.cc:336] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:01.113979 16441 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:25:01.114732 23447 raft_consensus.cc:2749] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:01.114717 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbdcb11d24644ca0bef26ea87647f4ac" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:25:01.115259 23456 consensus_queue.cc:1048] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:01.115623 16543 raft_consensus.cc:3060] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:25:01.116067 16441 leader_election.cc:336] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:01.116178 16366 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
W20260709 12:25:01.116631 16441 leader_election.cc:336] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:01.116751 23444 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:01.117203 23444 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:01.117645 23444 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:01.118891 23450 consensus_queue.cc:1048] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:25:01.119556 23437 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:01.112836 16442 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
W20260709 12:25:01.121721 16441 leader_election.cc:336] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:01.120056 23437 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:01.124965 23437 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:01.121371 16514 leader_election.cc:304] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:25:01.132558 23450 raft_consensus.cc:2804] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:01.133258 23450 raft_consensus.cc:697] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 18 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:25:01.134245 23450 consensus_queue.cc:237] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:25:01.136479 23447 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:01.136950 23447 raft_consensus.cc:493] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:01.137322 23447 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:01.140120 16474 raft_consensus.cc:1275] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 27 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 26 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 12:25:01.122937 16442 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:25:01.141081 16442 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:25:01.144373 16474 raft_consensus.cc:1275] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:25:01.146971 23427 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:01.147408 23427 raft_consensus.cc:493] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:01.147815 23427 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:01.148540 23438 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:25:01.147840 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba84aa22298489eb2845bec782a34f2" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:25:01.150573 16549 raft_consensus.cc:3060] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:01.145172 16543 raft_consensus.cc:2468] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 19.
I20260709 12:25:01.160315 23440 consensus_queue.cc:1048] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:25:01.111654 23421 consensus_queue.cc:1048] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 12:25:01.164220 16367 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:25:01.167325 23452 raft_consensus.cc:2804] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:01.167819 23452 raft_consensus.cc:697] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 19 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:25:01.168706 23452 consensus_queue.cc:237] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:25:01.169996 16441 leader_election.cc:336] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:01.173103 16442 leader_election.cc:304] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:25:01.179919 16366 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:25:01.180766 16549 raft_consensus.cc:2468] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 23.
W20260709 12:25:01.186439 16515 consensus_peers.cc:597] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:01.177485 23426 raft_consensus.cc:515] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:25:01.147876 23291 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:25:01.187841 23428 raft_consensus.cc:2804] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:25:01.202541 23432 consensus_queue.cc:1048] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 12:25:01.211208 16367 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:25:01.193694 23394 raft_consensus.cc:2804] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:01.172226 23444 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:01.221323 16546 raft_consensus.cc:2393] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 21: Already voted for candidate 610f4feda11942739236f2b121a4b9c1 in this term.
I20260709 12:25:01.244954 23443 raft_consensus.cc:515] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:25:01.247305 23443 leader_election.cc:290] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 election: Requested vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:25:01.265995 23437 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:01.195087 23458 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:01.291517 16468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215b509464e4efe977922ffdeb0b064" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:25:01.292277 16468 raft_consensus.cc:3060] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:01.261273 23436 raft_consensus.cc:2804] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:01.295994 23436 raft_consensus.cc:697] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 23 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:25:01.297358 23436 consensus_queue.cc:237] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:25:01.258348 23394 raft_consensus.cc:697] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:25:01.302698 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 94725.7us
I20260709 12:25:01.267874 23428 raft_consensus.cc:697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 30 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:25:01.307925 16399 raft_consensus.cc:1275] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:25:01.271266 23458 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader ef57beab0e1d40d585983873c55107e8)
I20260709 12:25:01.318655 23291 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Requested vote from peers 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:25:01.320997 23428 consensus_queue.cc:237] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.18, Last appended by leader: 18, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:01.348275 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3f77bde864e0ab25c63401793c709" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:25:01.348923 16549 raft_consensus.cc:3060] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:25:01.350732 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 100221us
W20260709 12:25:01.351496 16367 leader_election.cc:336] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:25:01.353019 16367 leader_election.cc:336] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:01.356685 23409 consensus_queue.cc:1048] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:25:01.360669 23427 raft_consensus.cc:515] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:01.314474 23458 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:01.365419 16468 raft_consensus.cc:2468] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 21.
I20260709 12:25:01.371539 16549 raft_consensus.cc:2468] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 19.
I20260709 12:25:01.379603 23426 leader_election.cc:290] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:25:01.393450 23447 raft_consensus.cc:515] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:01.440647 23427 leader_election.cc:290] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:25:01.441310 23437 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:25:01.443025 16367 leader_election.cc:336] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:01.378367 23394 consensus_queue.cc:237] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:01.471457 23447 leader_election.cc:290] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:25:01.474159 16515 consensus_peers.cc:597] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:01.478183 16515 leader_election.cc:336] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:25:01.485814 16515 leader_election.cc:336] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:01.516085 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "defc374629a649a8a47502f058be2c8c" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:25:01.516906 16398 raft_consensus.cc:3060] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:25:01.521332 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 268500us
I20260709 12:25:01.526309 16398 raft_consensus.cc:2468] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 21.
I20260709 12:25:01.532881 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14d8123994de43ec8e36a4e02961bc90" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:25:01.533459 16549 raft_consensus.cc:1275] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:25:01.533712 16400 raft_consensus.cc:3060] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:01.547298 23444 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:25:01.555888 23462 log.cc:927] Time spent T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:25:01.558005 16400 raft_consensus.cc:2468] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 16.
I20260709 12:25:01.531565 16442 leader_election.cc:304] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:25:01.573849 16474 raft_consensus.cc:1275] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:25:01.574393 16442 leader_election.cc:304] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:25:01.575522 23430 raft_consensus.cc:2804] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:01.587412 16474 raft_consensus.cc:1275] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:25:01.591413 16476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:25:01.586313 23446 raft_consensus.cc:2804] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:01.596046 16470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a90f33738a74f2c9fbf18bb0199dccf" candidate_uuid: "f99066d13e524238aec6aa2024a8bd50" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:25:01.598111 16470 raft_consensus.cc:3060] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:01.611877 16514 leader_election.cc:304] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:25:01.620675 23443 raft_consensus.cc:2804] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:01.621284 23443 raft_consensus.cc:697] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 21 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:25:01.622330 23443 consensus_queue.cc:237] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:25:01.608992 23458 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:25:01.634253  5352 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 17296)
I20260709 12:25:01.635502 16476 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f99066d13e524238aec6aa2024a8bd50 in current term 22: Already voted for candidate 52ab50b0237346c1a13ede947f6a63c2 in this term.
I20260709 12:25:01.636325 16399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:25:01.641795 16399 raft_consensus.cc:2393] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 52ab50b0237346c1a13ede947f6a63c2 in current term 22: Already voted for candidate f99066d13e524238aec6aa2024a8bd50 in this term.
W20260709 12:25:01.647853 16441 consensus_peers.cc:597] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:01.609997 23446 raft_consensus.cc:697] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:25:01.633569 23432 consensus_queue.cc:1048] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 12:25:01.641712 23458 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:25:01.610877 23438 consensus_queue.cc:1048] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
W20260709 12:25:01.694586 16515 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 171900us
W20260709 12:25:01.696902 23472 log.cc:927] Time spent T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
W20260709 12:25:01.698542 23474 log.cc:927] Time spent T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
W20260709 12:25:01.715247 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 81610.1us
I20260709 12:25:01.678285 23430 raft_consensus.cc:697] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 21 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
W20260709 12:25:01.767921 16441 leader_election.cc:336] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:25:01.769193 16441 leader_election.cc:336] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:25:01.785135 23477 log.cc:927] Time spent T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 12:25:01.795518 16470 raft_consensus.cc:2468] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99066d13e524238aec6aa2024a8bd50 in term 19.
W20260709 12:25:01.825771 23482 log.cc:927] Time spent T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 12:25:01.836540 23430 consensus_queue.cc:237] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:01.797284 23446 consensus_queue.cc:237] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:01.843848 16217 catalog_manager.cc:5697] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 17 to 18, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 18 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:01.870321 16441 leader_election.cc:336] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:01.893106 16366 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:25:01.895063 23398 raft_consensus.cc:2804] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:01.895715 23398 raft_consensus.cc:697] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 19 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:25:01.896816 23398 consensus_queue.cc:237] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:25:01.904798 23490 log.cc:927] Time spent T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1: Append to log took a long time: real 0.060s	user 0.003s	sys 0.000s
W20260709 12:25:01.910786 23469 log.cc:927] Time spent T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2: Append to log took a long time: real 0.421s	user 0.004s	sys 0.000s
W20260709 12:25:01.914186 16367 consensus_peers.cc:597] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:01.916952 16515 consensus_peers.cc:597] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:01.921031 16398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e0c5dbd539e4fa1a5f1e3fac0802b81" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:25:01.921934 16398 raft_consensus.cc:3060] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:25:01.927279 16367 leader_election.cc:336] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:01.931039 16546 raft_consensus.cc:1275] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:25:01.951828 16398 raft_consensus.cc:2468] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 21.
I20260709 12:25:01.972193 16367 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 610f4feda11942739236f2b121a4b9c1, ef57beab0e1d40d585983873c55107e8
I20260709 12:25:02.003548 16442 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:02.019294 16516 leader_election.cc:304] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
W20260709 12:25:02.019915 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 47882.1us
I20260709 12:25:02.020458 23440 raft_consensus.cc:2804] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:02.020843 16367 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:25:02.021993 23487 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:25:02.020406 23467 raft_consensus.cc:2749] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:02.024992 23440 raft_consensus.cc:697] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 21 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:25:02.025125 23452 raft_consensus.cc:2804] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:02.025892 23452 raft_consensus.cc:697] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 19 LEADER]: Becoming Leader. State: Replica: f99066d13e524238aec6aa2024a8bd50, State: Running, Role: LEADER
I20260709 12:25:02.026284 23440 consensus_queue.cc:237] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:25:02.026777 23452 consensus_queue.cc:237] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:25:02.035463 23468 consensus_queue.cc:1048] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:25:02.039891 16441 consensus_peers.cc:597] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:02.084240 16396 raft_consensus.cc:1275] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:25:02.093101 16546 raft_consensus.cc:1275] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:25:02.073380 23468 consensus_queue.cc:1048] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 12:25:02.147594 16221 catalog_manager.cc:5697] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 20 to 21, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 21 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:02.159857 16221 catalog_manager.cc:5697] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 15 to 16, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 16 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:02.163300 23398 consensus_queue.cc:1048] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
W20260709 12:25:02.198611 23502 log.cc:927] Time spent T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:25:02.166880 23484 consensus_queue.cc:1048] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 12:25:02.215448 16221 catalog_manager.cc:5697] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 20 to 21, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 21 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:02.234875 23501 log.cc:927] Time spent T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 12:25:02.259827 16468 raft_consensus.cc:1275] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:25:02.261759 23440 consensus_queue.cc:1048] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:25:02.286607 16515 consensus_peers.cc:597] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:02.292630 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 138229us
W20260709 12:25:02.303169 16515 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111) [suppressed 308 similar messages]
W20260709 12:25:02.308822 16367 consensus_peers.cc:597] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:02.309468 16367 consensus_peers.cc:597] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:02.310019 16367 leader_election.cc:336] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:02.313647 16367 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f99066d13e524238aec6aa2024a8bd50; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
W20260709 12:25:02.314541 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 217788us
I20260709 12:25:02.318689 23421 raft_consensus.cc:2749] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:25:02.320014 16367 consensus_peers.cc:597] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:02.320248 16468 raft_consensus.cc:1275] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 30 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 29 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
W20260709 12:25:02.320561 16367 consensus_peers.cc:597] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:02.316447 16221 catalog_manager.cc:5697] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 21 to 22, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 22 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:02.332856 16221 catalog_manager.cc:5697] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 18 to 19, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 19 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:02.335110 16221 catalog_manager.cc:5697] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 29 to 30, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 30 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:02.337203 16221 catalog_manager.cc:5697] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 21 to 22, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 22 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:02.341672 16221 catalog_manager.cc:5697] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 22 to 23, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 23 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:02.341535 23505 consensus_queue.cc:1048] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:25:02.343740 16221 catalog_manager.cc:5697] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 18 to 19, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 19 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:02.345470 16221 catalog_manager.cc:5697] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 reported cstate change: term changed from 18 to 19, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to f99066d13e524238aec6aa2024a8bd50 (127.5.58.2). New cstate: current_term: 19 leader_uuid: "f99066d13e524238aec6aa2024a8bd50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:02.359021 16441 consensus_peers.cc:597] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:02.363552 16396 raft_consensus.cc:1275] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 12:25:02.364774 16367 consensus_peers.cc:597] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:02.378388 16470 raft_consensus.cc:1275] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:25:02.378499 23427 consensus_queue.cc:1048] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 12:25:02.397867 16217 catalog_manager.cc:5697] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 20 to 21, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 21 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } }
I20260709 12:25:02.399822 16395 raft_consensus.cc:1275] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 12:25:02.413929 16441 consensus_peers.cc:597] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:02.420674 16515 consensus_peers.cc:597] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:02.431232 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 50241.8us
I20260709 12:25:02.432476 23467 consensus_queue.cc:1048] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:25:02.446964 23427 consensus_queue.cc:1048] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:25:02.490762 16441 consensus_peers.cc:597] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:02.495028 16515 consensus_peers.cc:597] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:25:02.512889 16394 raft_consensus.cc:1275] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Refusing update from remote peer 610f4feda11942739236f2b121a4b9c1: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:25:02.524123 23435 consensus_queue.cc:1048] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 12:25:02.529004 16545 raft_consensus.cc:1275] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Refusing update from remote peer f99066d13e524238aec6aa2024a8bd50: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:25:02.538254 23507 consensus_queue.cc:1048] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:25:02.542160 16367 consensus_peers.cc:597] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:02.550918 16441 consensus_peers.cc:597] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:02.557435 16515 consensus_peers.cc:597] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:02.562958 16515 consensus_peers.cc:597] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:02.572542 16441 consensus_peers.cc:597] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:02.573594 16367 consensus_peers.cc:597] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:02.634816  5352 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 17296)
I20260709 12:25:02.651511 16217 catalog_manager.cc:5697] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 20 to 21, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 21 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:02.666302 16441 consensus_peers.cc:597] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:02.677248 16515 consensus_peers.cc:597] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:02.678090 16515 consensus_peers.cc:597] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:02.744082 16367 consensus_peers.cc:597] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:02.787088 16441 consensus_peers.cc:597] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:02.922598 16515 consensus_peers.cc:597] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:02.928201 16515 consensus_peers.cc:597] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:02.960695 16515 consensus_peers.cc:597] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:02.971091 16441 consensus_peers.cc:597] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:03.045240 16515 consensus_peers.cc:597] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:03.045888 16515 consensus_peers.cc:597] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:03.116174 16515 consensus_peers.cc:597] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:03.221176 16441 consensus_peers.cc:597] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:03.224553 16367 consensus_peers.cc:597] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:03.342414 16367 consensus_peers.cc:597] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:03.401124 16515 consensus_peers.cc:597] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:03.571599 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 44031.9us
W20260709 12:25:03.602173 16367 consensus_peers.cc:597] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:03.635622 16515 consensus_peers.cc:597] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:03.635641  5352 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 17296)
I20260709 12:25:03.650810 23522 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.651979 23522 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:03.667189 23522 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:25:03.710050 16515 consensus_peers.cc:597] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:03.747083 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50" is_pre_election: true
I20260709 12:25:03.749194 16400 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 22.
W20260709 12:25:03.804457 16515 consensus_peers.cc:597] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:03.816258 16442 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: 
I20260709 12:25:03.818858 23486 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:03.819362 23486 raft_consensus.cc:493] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.819715 23486 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:03.866955 23486 raft_consensus.cc:515] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
W20260709 12:25:03.877861 16441 leader_election.cc:336] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:25:03.879112 16441 consensus_peers.cc:597] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:03.881018 23486 leader_election.cc:290] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 election: Requested vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:25:03.915735 16515 consensus_peers.cc:597] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:03.939594 16441 leader_election.cc:336] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:25:03.954392 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 90059.2us
W20260709 12:25:03.957093 16367 consensus_peers.cc:597] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:25:03.993889 16400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472c1e509cf2401e8153946529cfba2e" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f99066d13e524238aec6aa2024a8bd50"
I20260709 12:25:03.994647 16400 raft_consensus.cc:3060] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:04.104728 16400 raft_consensus.cc:2468] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 23.
W20260709 12:25:04.123107 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 44648.8us
W20260709 12:25:04.174445 16515 consensus_peers.cc:597] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:04.177796 16367 consensus_peers.cc:597] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:04.194371 16515 consensus_peers.cc:597] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:04.224583 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 64468.4us
I20260709 12:25:04.229527 16442 leader_election.cc:304] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50; no voters: ef57beab0e1d40d585983873c55107e8
I20260709 12:25:04.231999 23486 raft_consensus.cc:2804] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:04.232637 23486 raft_consensus.cc:697] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 23 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:25:04.233929 23486 consensus_queue.cc:237] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:04.300626 16223 catalog_manager.cc:5697] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 21 to 23, leader changed from ef57beab0e1d40d585983873c55107e8 (127.5.58.1) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 23 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:04.394374 16367 consensus_peers.cc:597] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:04.398075 16367 consensus_peers.cc:597] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:04.621002 16441 consensus_peers.cc:597] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:04.638906  5352 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 17296)
W20260709 12:25:04.638864 16515 consensus_peers.cc:597] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:04.643743 16515 consensus_peers.cc:597] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:04.735770 16441 consensus_peers.cc:597] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:04.757350 16441 consensus_peers.cc:597] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:04.757357 16367 consensus_peers.cc:597] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:04.763130 16400 raft_consensus.cc:1275] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 23 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:25:04.764636 23528 consensus_queue.cc:1048] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:25:04.836038 16441 consensus_peers.cc:597] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:04.860637 16367 consensus_peers.cc:597] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:04.894429 16441 consensus_peers.cc:597] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:04.903844 16367 consensus_peers.cc:597] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:04.922260 16441 consensus_peers.cc:597] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:04.974851 16367 consensus_peers.cc:597] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:05.141592 16367 consensus_peers.cc:597] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:05.150707 16441 consensus_peers.cc:597] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:05.213593 16441 consensus_peers.cc:597] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:05.334611 16441 consensus_peers.cc:597] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:05.444484 16367 consensus_peers.cc:597] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:05.524453 16441 consensus_peers.cc:597] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:05.574764 16367 consensus_peers.cc:597] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:05.639905  5352 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 17296)
W20260709 12:25:05.698989 16441 consensus_peers.cc:597] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:05.738811 16515 consensus_peers.cc:597] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:05.740020 16515 consensus_peers.cc:597] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:05.741549 16515 consensus_peers.cc:597] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:05.751605 16515 consensus_peers.cc:597] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:05.753216 16515 consensus_peers.cc:597] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:05.792452 16515 consensus_peers.cc:597] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:05.794162 16367 consensus_peers.cc:597] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:05.984434 16367 consensus_peers.cc:597] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:06.068913 16515 consensus_peers.cc:597] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:06.098510 16367 consensus_peers.cc:597] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:06.108598 16515 consensus_peers.cc:597] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:06.145298 16441 consensus_peers.cc:597] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:06.243768 16515 consensus_peers.cc:597] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:06.348455 16515 consensus_peers.cc:597] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:06.385238 16515 consensus_peers.cc:597] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:06.427832 16515 consensus_peers.cc:597] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:06.561967 16367 consensus_peers.cc:597] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:06.567516 16515 consensus_peers.cc:597] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:06.640637  5352 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 17296)
W20260709 12:25:06.742640 16515 consensus_peers.cc:597] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:06.778368 16515 consensus_peers.cc:597] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:06.916298 16441 consensus_peers.cc:597] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:06.951074 16367 consensus_peers.cc:597] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:06.964191 16515 consensus_peers.cc:597] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:06.977020 16367 consensus_peers.cc:597] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:07.069628 16441 consensus_peers.cc:597] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:07.141607 16515 consensus_peers.cc:597] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:07.256462 16515 consensus_peers.cc:597] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:07.279881 16515 consensus_peers.cc:597] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:07.307912 16367 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111) [suppressed 429 similar messages]
W20260709 12:25:07.314396 16367 consensus_peers.cc:597] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:07.336820 16441 consensus_peers.cc:597] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:07.422021 16441 consensus_peers.cc:597] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:07.437515 16367 consensus_peers.cc:597] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:07.442049 16367 consensus_peers.cc:597] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:07.450459 16441 consensus_peers.cc:597] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:07.471948 16441 consensus_peers.cc:597] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:07.532799 16515 consensus_peers.cc:597] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:07.589389 16367 consensus_peers.cc:597] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:07.593427 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 146809us
W20260709 12:25:07.641268  5352 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 17296)
W20260709 12:25:07.702399 16441 consensus_peers.cc:597] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:07.714267 16441 consensus_peers.cc:597] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:07.734606 16441 consensus_peers.cc:597] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:07.792936 16367 consensus_peers.cc:597] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:07.830535 16515 consensus_peers.cc:597] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:07.887914 16441 consensus_peers.cc:597] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:07.898049 16515 consensus_peers.cc:597] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:07.990945 16515 consensus_peers.cc:597] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:08.011445 16367 consensus_peers.cc:597] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:08.068135 16367 consensus_peers.cc:597] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:08.074577 16515 consensus_peers.cc:597] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:08.097221 16441 consensus_peers.cc:597] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:08.164264 16515 consensus_peers.cc:597] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:08.281816 16367 consensus_peers.cc:597] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:08.304358 16515 consensus_peers.cc:597] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:08.364018 16441 consensus_peers.cc:597] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:08.605002 16515 consensus_peers.cc:597] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:08.641808  5352 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 17296)
W20260709 12:25:08.643361 16367 consensus_peers.cc:597] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:08.662424 16367 consensus_peers.cc:597] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:08.683794 16515 consensus_peers.cc:597] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:08.725914 16515 consensus_peers.cc:597] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:08.726505 16515 consensus_peers.cc:597] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:08.833389 16515 consensus_peers.cc:597] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:08.915997 16515 consensus_peers.cc:597] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:08.947343 16515 consensus_peers.cc:597] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:09.100674 16515 consensus_peers.cc:597] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:09.196267 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 107978us
W20260709 12:25:09.211772 16515 consensus_peers.cc:597] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:09.235504 16367 consensus_peers.cc:597] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:09.335834 16441 consensus_peers.cc:597] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:09.357278 16367 consensus_peers.cc:597] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:09.458374 16367 consensus_peers.cc:597] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:09.459535 16515 consensus_peers.cc:597] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:09.466241 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 48543.4us
W20260709 12:25:09.537052 16441 consensus_peers.cc:597] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:09.567763 16367 consensus_peers.cc:597] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:09.612618 16515 consensus_peers.cc:597] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:09.642311  5352 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 17296)
W20260709 12:25:09.748097 16515 consensus_peers.cc:597] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:09.793268 16441 consensus_peers.cc:597] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:09.943333 16367 consensus_peers.cc:597] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:09.976720 16441 consensus_peers.cc:597] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:09.976847 16367 consensus_peers.cc:597] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:09.992619 16515 consensus_peers.cc:597] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:10.024273 16441 consensus_peers.cc:597] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:10.041427 16515 consensus_peers.cc:597] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:10.137072 16367 consensus_peers.cc:597] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:10.215058 16441 consensus_peers.cc:597] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:10.232851 16441 consensus_peers.cc:597] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:10.234273 16515 consensus_peers.cc:597] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:10.333850 16515 consensus_peers.cc:597] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:10.383278 16441 consensus_peers.cc:597] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:10.417045 16441 consensus_peers.cc:597] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:10.432885 16515 consensus_peers.cc:597] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:10.525712 16367 consensus_peers.cc:597] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:10.547147 16441 consensus_peers.cc:597] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:10.605922 16515 consensus_peers.cc:597] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:10.642870  5352 thread.cc:527] Waited for 11000ms trying to join with diag-logger (tid 17296)
W20260709 12:25:10.654183 16441 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 46599.5us
W20260709 12:25:10.648429 16367 consensus_peers.cc:597] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:10.669965 16441 consensus_peers.cc:597] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:10.702860 16442 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 67117.2us
W20260709 12:25:10.748660 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 53275.3us
W20260709 12:25:10.761336 16367 consensus_peers.cc:597] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:10.789801 16515 consensus_peers.cc:597] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:10.801335 16516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:41367, user_credentials={real_user=slave}} blocked reactor thread for 51089.2us
W20260709 12:25:10.923952 16367 consensus_peers.cc:597] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:10.962009 16367 consensus_peers.cc:597] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:10.968159 16515 consensus_peers.cc:597] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:10.999714 16515 consensus_peers.cc:597] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:11.010982 16515 consensus_peers.cc:597] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:11.026198 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 47729.9us
W20260709 12:25:11.100893 16441 consensus_peers.cc:597] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:11.157437 16367 consensus_peers.cc:597] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:11.187835 16515 consensus_peers.cc:597] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:11.269905 16515 consensus_peers.cc:597] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:11.436260 16515 consensus_peers.cc:597] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:11.443090 16515 consensus_peers.cc:597] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:11.604197 16367 consensus_peers.cc:597] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:11.631214 16515 consensus_peers.cc:597] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:11.643967  5352 thread.cc:527] Waited for 12000ms trying to join with diag-logger (tid 17296)
W20260709 12:25:11.649691 16515 consensus_peers.cc:597] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:11.787115 16515 consensus_peers.cc:597] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:11.790781 16441 consensus_peers.cc:597] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:25:11.824575 16367 consensus_peers.cc:597] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:11.912946 16367 consensus_peers.cc:597] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:11.969249 16367 consensus_peers.cc:597] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:12.041328 16515 consensus_peers.cc:597] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:12.196041 16441 consensus_peers.cc:597] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:12.244366 16515 consensus_peers.cc:597] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:12.259866 16515 consensus_peers.cc:597] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:12.316578 16367 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111) [suppressed 444 similar messages]
W20260709 12:25:12.324637 16515 consensus_peers.cc:597] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:12.330947 16367 consensus_peers.cc:597] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:12.354130 16441 consensus_peers.cc:597] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:12.483922 16441 consensus_peers.cc:597] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:12.492709 16441 consensus_peers.cc:597] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:12.520246 16367 consensus_peers.cc:597] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:12.613149 16515 consensus_peers.cc:597] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:12.640318 16441 consensus_peers.cc:597] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:12.645933  5352 thread.cc:527] Waited for 13000ms trying to join with diag-logger (tid 17296)
W20260709 12:25:12.762358 16515 consensus_peers.cc:597] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:12.938594 16441 consensus_peers.cc:597] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:12.965751 16515 consensus_peers.cc:597] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:12.984812 16441 consensus_peers.cc:597] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:12.998041 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 50914.3us
W20260709 12:25:13.136905 16515 consensus_peers.cc:597] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:13.137593 16515 consensus_peers.cc:597] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:13.239667 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.4:44491, user_credentials={real_user=slave}} blocked reactor thread for 81413.3us
W20260709 12:25:13.242522 16367 consensus_peers.cc:597] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:13.243324 16367 consensus_peers.cc:597] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:13.261927 16441 consensus_peers.cc:597] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:13.278929 16441 consensus_peers.cc:597] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:13.290572 16441 consensus_peers.cc:597] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:13.297984 16367 consensus_peers.cc:597] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:13.318634 16367 consensus_peers.cc:597] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:13.403712 16515 consensus_peers.cc:597] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:13.411849 16367 consensus_peers.cc:597] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:13.430194 16367 consensus_peers.cc:597] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:13.442911 16367 consensus_peers.cc:597] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:13.465013 16515 consensus_peers.cc:597] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:13.528646 16515 consensus_peers.cc:597] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:13.574361 16515 consensus_peers.cc:597] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:13.624202 16515 consensus_peers.cc:597] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:13.652717  5352 thread.cc:527] Waited for 14000ms trying to join with diag-logger (tid 17296)
W20260709 12:25:13.789106 16441 consensus_peers.cc:597] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:13.846637 16366 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 52571.2us
W20260709 12:25:13.866235 16515 consensus_peers.cc:597] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:13.873190 16515 consensus_peers.cc:597] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:13.896358 16515 consensus_peers.cc:597] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:14.076463 16515 consensus_peers.cc:597] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:14.135406 16441 consensus_peers.cc:597] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:25:14.153944 16515 consensus_peers.cc:597] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:14.196156 16367 consensus_peers.cc:597] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:14.255780 16367 consensus_peers.cc:597] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:14.405694 16514 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:33107, user_credentials={real_user=slave}} blocked reactor thread for 61817.8us
W20260709 12:25:14.435665 16367 consensus_peers.cc:597] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:14.454392 16515 consensus_peers.cc:597] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:14.459578 16515 consensus_peers.cc:597] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:14.510195 16515 consensus_peers.cc:597] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:14.584944 16367 consensus_peers.cc:597] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:14.654021  5352 thread.cc:527] Waited for 15000ms trying to join with diag-logger (tid 17296)
W20260709 12:25:14.683926 16367 consensus_peers.cc:597] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:14.741139 16441 consensus_peers.cc:597] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:14.824594 16441 consensus_peers.cc:597] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:14.880677 16515 consensus_peers.cc:597] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:14.902603 16441 consensus_peers.cc:597] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:14.957442 16441 consensus_peers.cc:597] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:14.969832 16515 consensus_peers.cc:597] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:15.004005 16367 consensus_peers.cc:597] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:15.041564 16515 consensus_peers.cc:597] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:15.127274 16441 consensus_peers.cc:597] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:15.227953 16515 consensus_peers.cc:597] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:15.277887 16367 consensus_peers.cc:597] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:15.284586 16367 consensus_peers.cc:597] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:15.323046 16441 consensus_peers.cc:597] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:15.324748 16515 consensus_peers.cc:597] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:15.429303 16515 consensus_peers.cc:597] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:15.432983 16441 consensus_peers.cc:597] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:15.501123 16441 consensus_peers.cc:597] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:15.655517  5352 thread.cc:527] Waited for 16000ms trying to join with diag-logger (tid 17296)
W20260709 12:25:15.714533 16367 consensus_peers.cc:597] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:15.715232 16367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:41805, user_credentials={real_user=slave}} blocked reactor thread for 50646.3us
W20260709 12:25:15.828356 16515 consensus_peers.cc:597] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:15.833987 16367 consensus_peers.cc:597] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:15.851631 16441 consensus_peers.cc:597] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:15.941103 16367 consensus_peers.cc:597] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:15.946403 16367 consensus_peers.cc:597] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:15.977746 16515 consensus_peers.cc:597] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:15.994232 16515 consensus_peers.cc:597] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:16.001688 16515 consensus_peers.cc:597] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:16.006106 16515 consensus_peers.cc:597] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:16.044253 16515 consensus_peers.cc:597] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:16.054118 16367 consensus_peers.cc:597] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:16.112757 16441 consensus_peers.cc:597] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:16.298207 16515 consensus_peers.cc:597] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:16.371286 16441 consensus_peers.cc:597] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:16.424731 16515 consensus_peers.cc:597] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:16.470690 16515 consensus_peers.cc:597] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:16.530350 16515 consensus_peers.cc:597] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:16.656198  5352 thread.cc:527] Waited for 17000ms trying to join with diag-logger (tid 17296)
W20260709 12:25:16.667680 16257 debug-util.cc:398] Leaking SignalData structure 0x7b08001a0580 after lost signal to thread 5355
W20260709 12:25:16.669323 16257 debug-util.cc:398] Leaking SignalData structure 0x7b0800094860 after lost signal to thread 16260
W20260709 12:25:16.670923 16257 debug-util.cc:398] Leaking SignalData structure 0x7b0800350cc0 after lost signal to thread 16427
W20260709 12:25:16.672426 16257 debug-util.cc:398] Leaking SignalData structure 0x7b08000e3180 after lost signal to thread 16501
W20260709 12:25:16.673782 16257 debug-util.cc:398] Leaking SignalData structure 0x7b0800302480 after lost signal to thread 16575
W20260709 12:25:16.717674 16515 consensus_peers.cc:597] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:16.825147 16367 consensus_peers.cc:597] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:16.837327 16515 consensus_peers.cc:597] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:16.862922 16515 consensus_peers.cc:597] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:17.008481 16367 consensus_peers.cc:597] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:17.009032 16367 consensus_peers.cc:597] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:17.009466 16367 consensus_peers.cc:597] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:17.012918 16441 consensus_peers.cc:597] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:25:17.057821 16515 consensus_peers.cc:597] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:17.151041 16515 consensus_peers.cc:597] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:25:17.206977 16441 consensus_peers.cc:597] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:17.299237 16367 consensus_peers.cc:597] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:17.327538 16367 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111) [suppressed 446 similar messages]
W20260709 12:25:17.408088 16441 consensus_peers.cc:597] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:17.416884 16441 consensus_peers.cc:597] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:17.418896 16515 consensus_peers.cc:597] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:25:17.555285 16367 consensus_peers.cc:597] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:17.567749 16441 consensus_peers.cc:597] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:25:17.581387 16515 consensus_peers.cc:597] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:17.656898  5352 thread.cc:527] Waited for 18000ms trying to join with diag-logger (tid 17296)
W20260709 12:25:17.689268 16441 consensus_peers.cc:597] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:25:17.711833 16441 consensus_peers.cc:597] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:25:17.729090 16515 consensus_peers.cc:597] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:17.733906 16367 consensus_peers.cc:597] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:25:17.881788 16515 consensus_peers.cc:597] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:25:17.932209 16367 consensus_peers.cc:597] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:17.932438 16424 debug-util.cc:398] Leaking SignalData structure 0x7b08001cbb80 after lost signal to thread 5355
W20260709 12:25:17.933633 16424 debug-util.cc:398] Leaking SignalData structure 0x7b0800166a80 after lost signal to thread 16260
W20260709 12:25:17.934970 16424 debug-util.cc:398] Leaking SignalData structure 0x7b080027d7a0 after lost signal to thread 16427
W20260709 12:25:17.936177 16424 debug-util.cc:398] Leaking SignalData structure 0x7b08001733c0 after lost signal to thread 16501
W20260709 12:25:17.938863 16424 debug-util.cc:398] Leaking SignalData structure 0x7b08003eb4c0 after lost signal to thread 16575
W20260709 12:25:18.080749 16441 consensus_peers.cc:597] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:25:18.094950 16515 consensus_peers.cc:597] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:25:18.234314 16367 consensus_peers.cc:597] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:25:18.242530 16515 consensus_peers.cc:597] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:25:18.260749 16367 consensus_peers.cc:597] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:25:18.261420 16441 consensus_peers.cc:597] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:25:18.337409 16515 consensus_peers.cc:597] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:25:18.374235 16367 consensus_peers.cc:597] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:18.404908 16367 consensus_peers.cc:597] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:25:18.492965 16441 consensus_peers.cc:597] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:18.537279 16441 consensus_peers.cc:597] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:25:18.546304 16515 consensus_peers.cc:597] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:25:18.548971 16367 consensus_peers.cc:597] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:25:18.612560 16515 consensus_peers.cc:597] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:25:18.619508 16515 consensus_peers.cc:597] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:18.657485  5352 thread.cc:527] Waited for 19000ms trying to join with diag-logger (tid 17296)
W20260709 12:25:18.672220 16515 consensus_peers.cc:597] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:18.740053 16441 consensus_peers.cc:597] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:19.012266 16515 consensus_peers.cc:597] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:25:19.143989 16367 consensus_peers.cc:597] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:19.154592 16515 consensus_peers.cc:597] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:25:19.217137 16515 consensus_peers.cc:597] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:25:19.258749 16515 consensus_peers.cc:597] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:19.368088 16515 consensus_peers.cc:597] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:25:19.406041 16515 consensus_peers.cc:597] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:19.420694 16515 consensus_peers.cc:597] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:19.433419 16515 consensus_peers.cc:597] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:25:19.533998 16367 consensus_peers.cc:597] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:19.556136 16441 consensus_peers.cc:597] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:25:19.593312 16367 consensus_peers.cc:597] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:19.664734 16367 consensus_peers.cc:597] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:19.768184 16441 consensus_peers.cc:597] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:19.796530 16441 consensus_peers.cc:597] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:19.799502 16515 consensus_peers.cc:597] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:25:19.841742 16441 consensus_peers.cc:597] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:25:19.854502  5352 tablet_server.cc:196] TabletServer@127.5.58.1:41805 shutdown complete.
W20260709 12:25:19.879227 16515 consensus_peers.cc:597] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:19.941854 16515 consensus_peers.cc:597] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:25:19.976856 16367 consensus_peers.cc:597] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:25:19.991432  5352 tablet_server.cc:179] TabletServer@127.5.58.2:0 shutting down...
W20260709 12:25:20.021437 16442 consensus_peers.cc:597] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:20.028368 16442 consensus_peers.cc:597] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:20.030731 16442 consensus_peers.cc:597] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:20.034690 16516 consensus_peers.cc:597] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:20.085294 16442 consensus_peers.cc:597] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:20.105276 16516 consensus_peers.cc:597] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:20.108467  5352 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:25:20.110260  5352 tablet_replica.cc:333] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.113060  5352 raft_consensus.cc:2243] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:20.116047  5352 raft_consensus.cc:2272] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P f99066d13e524238aec6aa2024a8bd50 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.123250  5352 tablet_replica.cc:333] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.124655  5352 raft_consensus.cc:2243] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Raft consensus shutting down.
W20260709 12:25:20.125174 16515 consensus_peers.cc:597] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:25:20.125656  5352 raft_consensus.cc:2272] T 6474b10917204b6994269be17387f38d P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.129458  5352 tablet_replica.cc:333] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.131552  5352 raft_consensus.cc:2243] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 18 LEADER]: Raft consensus shutting down.
W20260709 12:25:20.132494 16442 consensus_peers.cc:597] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:20.132983  5352 raft_consensus.cc:2272] T 05c8566780d1400ab2e42d9529d97ceb P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.136276  5352 tablet_replica.cc:333] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.137305  5352 raft_consensus.cc:2243] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:25:20.138773  5352 raft_consensus.cc:2272] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.141992  5352 tablet_replica.cc:333] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.142975  5352 raft_consensus.cc:2243] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:25:20.144230  5352 raft_consensus.cc:2272] T 701eedfc29544bbeafd695f04ccea545 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.147737  5352 tablet_replica.cc:333] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.148746  5352 raft_consensus.cc:2243] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:20.149407  5352 raft_consensus.cc:2272] T b6d1babdc37d40e4a71ffde9b310feaa P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.152984  5352 tablet_replica.cc:333] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.154299  5352 raft_consensus.cc:2243] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:25:20.156589  5352 raft_consensus.cc:2272] T c1a154d032654a179da41ad8f10fa339 P f99066d13e524238aec6aa2024a8bd50 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.159685  5352 tablet_replica.cc:333] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.160573  5352 raft_consensus.cc:2243] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:20.161253  5352 raft_consensus.cc:2272] T 7117f6656fa14807ac801392259e843a P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.164419  5352 tablet_replica.cc:333] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.165345  5352 raft_consensus.cc:2243] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:25:20.166766  5352 raft_consensus.cc:2272] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P f99066d13e524238aec6aa2024a8bd50 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.170928  5352 tablet_replica.cc:333] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.171929  5352 raft_consensus.cc:2243] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:20.172974  5352 raft_consensus.cc:2272] T aea54881468a40aaabad162d385fc6d9 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.176784  5352 tablet_replica.cc:333] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.177764  5352 raft_consensus.cc:2243] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:20.178797  5352 raft_consensus.cc:2272] T a4bb7e1a96a94612b2f6759156f5f7c0 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:20.179203 16516 consensus_peers.cc:597] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:20.182467  5352 tablet_replica.cc:333] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.183585  5352 raft_consensus.cc:2243] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:20.184489  5352 raft_consensus.cc:2272] T 07c17e01135a4bae850c5038c62d2ba2 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.187734  5352 tablet_replica.cc:333] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.188642  5352 raft_consensus.cc:2243] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:25:20.189879  5352 raft_consensus.cc:2272] T d1faa3c7bc3e4073bb65968a654e815d P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:20.192296 16516 consensus_peers.cc:597] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:20.193215  5352 tablet_replica.cc:333] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.194284  5352 raft_consensus.cc:2243] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:25:20.196534  5352 raft_consensus.cc:2272] T e768efb7fcef4624b08d1e122c6dc825 P f99066d13e524238aec6aa2024a8bd50 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.200603  5352 tablet_replica.cc:333] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.201524  5352 raft_consensus.cc:2243] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:25:20.202729  5352 raft_consensus.cc:2272] T b7b4960094cc44ceb337b594b1c269e6 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.206125  5352 tablet_replica.cc:333] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.207050  5352 raft_consensus.cc:2243] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:25:20.208372  5352 raft_consensus.cc:2272] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.211644  5352 tablet_replica.cc:333] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.212610  5352 raft_consensus.cc:2243] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:20.213390  5352 raft_consensus.cc:2272] T 3851c76367ba4b3eaf3a1b5833ef4835 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.216660  5352 tablet_replica.cc:333] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.217595  5352 raft_consensus.cc:2243] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:20.218302  5352 raft_consensus.cc:2272] T 4a0e81993a5641399dd0ca4ded8b4037 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:20.218556 16516 consensus_peers.cc:597] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:20.222527  5352 tablet_replica.cc:333] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.223603  5352 raft_consensus.cc:2243] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:20.224434  5352 raft_consensus.cc:2272] T 5f88a73c287745feb816089d0e08d10b P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.227947  5352 tablet_replica.cc:333] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.228899  5352 raft_consensus.cc:2243] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:25:20.230623  5352 raft_consensus.cc:2272] T 49b3f77bde864e0ab25c63401793c709 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.234395  5352 tablet_replica.cc:333] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.235574  5352 raft_consensus.cc:2243] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:20.236464  5352 raft_consensus.cc:2272] T 57fce16c9dc745ce8d5a631936c11e00 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.239913  5352 tablet_replica.cc:333] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.240947  5352 raft_consensus.cc:2243] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:25:20.242617  5352 raft_consensus.cc:2272] T 6a90f33738a74f2c9fbf18bb0199dccf P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.246822  5352 tablet_replica.cc:333] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.247761  5352 raft_consensus.cc:2243] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:25:20.250483  5352 raft_consensus.cc:2272] T dbdcb11d24644ca0bef26ea87647f4ac P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.254120  5352 tablet_replica.cc:333] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
W20260709 12:25:20.254113 16442 consensus_peers.cc:597] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:20.255168 16442 consensus_peers.cc:597] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:20.255251  5352 raft_consensus.cc:2243] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Raft consensus shutting down.
W20260709 12:25:20.255293 16367 consensus_peers.cc:597] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:20.255970 16442 consensus_peers.cc:597] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:20.256268  5352 raft_consensus.cc:2272] T d46e3072dec049b18b323d65470603ec P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.259654  5352 tablet_replica.cc:333] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.260636  5352 raft_consensus.cc:2243] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:20.261448  5352 raft_consensus.cc:2272] T 3c4fd014cdf249c996641fe7793f42a5 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.264518  5352 tablet_replica.cc:333] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.265385  5352 raft_consensus.cc:2243] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:20.266047  5352 raft_consensus.cc:2272] T 472c1e509cf2401e8153946529cfba2e P f99066d13e524238aec6aa2024a8bd50 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.269266  5352 tablet_replica.cc:333] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.270179  5352 raft_consensus.cc:2243] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:20.270977  5352 raft_consensus.cc:2272] T edff15255cdf492dbc9c1189b889aa78 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:20.272142 16516 consensus_peers.cc:597] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:20.274070  5352 tablet_replica.cc:333] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.275033  5352 raft_consensus.cc:2243] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:25:20.276350  5352 raft_consensus.cc:2272] T 44b9e8076a1d43319b0c86d4b356ac68 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.279480  5352 tablet_replica.cc:333] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.280983  5352 raft_consensus.cc:2243] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:20.281626  5352 raft_consensus.cc:2272] T 9cd596d6e69a4b58804e76d2dd4f51f1 P f99066d13e524238aec6aa2024a8bd50 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.284639  5352 tablet_replica.cc:333] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.285532  5352 raft_consensus.cc:2243] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:20.286334  5352 raft_consensus.cc:2272] T 191e052863dd48fe92ebe151197330a8 P f99066d13e524238aec6aa2024a8bd50 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:20.286399 16442 consensus_peers.cc:597] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:20.289381  5352 tablet_replica.cc:333] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.290263  5352 raft_consensus.cc:2243] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:25:20.291462  5352 raft_consensus.cc:2272] T 434028125a2d48998f4f5106c59a9d47 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.294680  5352 tablet_replica.cc:333] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.295509  5352 raft_consensus.cc:2243] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:20.296311  5352 raft_consensus.cc:2272] T 95b81a29889843548dd6ed149807ecc8 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.299769  5352 tablet_replica.cc:333] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.300649  5352 raft_consensus.cc:2243] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:25:20.301820  5352 raft_consensus.cc:2272] T 3ba84aa22298489eb2845bec782a34f2 P f99066d13e524238aec6aa2024a8bd50 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.304596  5352 tablet_replica.cc:333] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.305478  5352 raft_consensus.cc:2243] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:25:20.306638  5352 raft_consensus.cc:2272] T e88d061c8b85425aa7478ab3874c4be4 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:20.309039 16516 consensus_peers.cc:597] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:20.309855  5352 tablet_replica.cc:333] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.311352  5352 raft_consensus.cc:2243] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:20.312248  5352 raft_consensus.cc:2272] T defc374629a649a8a47502f058be2c8c P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.315543  5352 tablet_replica.cc:333] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.316417  5352 raft_consensus.cc:2243] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:20.317236  5352 raft_consensus.cc:2272] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P f99066d13e524238aec6aa2024a8bd50 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.320431  5352 tablet_replica.cc:333] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.321309  5352 raft_consensus.cc:2243] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:20.322082  5352 raft_consensus.cc:2272] T 162cebc248914fb5aa57de3a383a4159 P f99066d13e524238aec6aa2024a8bd50 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:20.323196 16516 consensus_peers.cc:597] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:20.325516  5352 tablet_replica.cc:333] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.326503  5352 raft_consensus.cc:2243] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:20.327294  5352 raft_consensus.cc:2272] T 33037c31b4434e838d90b2a2065ef52d P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.330678  5352 tablet_replica.cc:333] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.331638  5352 raft_consensus.cc:2243] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:20.332552  5352 raft_consensus.cc:2272] T 14d8123994de43ec8e36a4e02961bc90 P f99066d13e524238aec6aa2024a8bd50 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:20.335016 16441 consensus_peers.cc:597] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:25:20.336308  5352 tablet_replica.cc:333] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.337198  5352 raft_consensus.cc:2243] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:20.338044  5352 raft_consensus.cc:2272] T 95954ea821224e4d962974a83d83ad44 P f99066d13e524238aec6aa2024a8bd50 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.341023  5352 tablet_replica.cc:333] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
W20260709 12:25:20.341144 16516 consensus_peers.cc:597] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:20.341787  5352 raft_consensus.cc:2243] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 22 LEADER]: Raft consensus shutting down.
W20260709 12:25:20.342831 16442 consensus_peers.cc:597] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:20.343194  5352 raft_consensus.cc:2272] T aadc86310ab642c69b7dbbf94be01070 P f99066d13e524238aec6aa2024a8bd50 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:20.344650 16441 consensus_peers.cc:597] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:25:20.345916 16442 consensus_peers.cc:597] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:20.346464  5352 tablet_replica.cc:333] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.347348  5352 raft_consensus.cc:2243] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:20.348196  5352 raft_consensus.cc:2272] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P f99066d13e524238aec6aa2024a8bd50 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.351379  5352 tablet_replica.cc:333] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.352136  5352 raft_consensus.cc:2243] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 18 LEADER]: Raft consensus shutting down.
W20260709 12:25:20.352919 16516 consensus_peers.cc:597] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:20.353430  5352 raft_consensus.cc:2272] T ef53202d665e46c39113572380be6c36 P f99066d13e524238aec6aa2024a8bd50 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.356619  5352 tablet_replica.cc:333] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50: stopping tablet replica
I20260709 12:25:20.357368  5352 raft_consensus.cc:2243] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:20.358033  5352 raft_consensus.cc:2272] T 0b9c348d235e432d92d8f8f20f1f1317 P f99066d13e524238aec6aa2024a8bd50 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:20.403239 16442 consensus_peers.cc:597] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:20.435832 16442 consensus_peers.cc:597] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:20.445988 16516 consensus_peers.cc:597] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:20.459317 16516 consensus_peers.cc:597] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:20.490813 16516 consensus_peers.cc:597] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:20.545344 16515 consensus_peers.cc:597] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:25:20.586791 16441 consensus_peers.cc:597] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:25:20.595063 16441 consensus_peers.cc:597] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:25:20.630743 16441 consensus_peers.cc:597] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:25:20.769484 16515 consensus_peers.cc:597] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:25:20.774530 16515 consensus_peers.cc:597] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:25:20.801285 16441 consensus_peers.cc:597] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:25:20.864372 16515 consensus_peers.cc:597] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:25:20.965942 16515 consensus_peers.cc:597] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:25:21.034476 16515 consensus_peers.cc:597] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:25:21.135584 23605 raft_consensus.cc:493] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:25:21.136188 23605 raft_consensus.cc:515] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:21.139391 23605 leader_election.cc:290] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:25:21.139599 16441 consensus_peers.cc:597] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:25:21.144763 16442 leader_election.cc:336] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
W20260709 12:25:21.145269 16441 consensus_peers.cc:597] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:25:21.149034 16441 leader_election.cc:336] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:21.150533 16441 leader_election.cc:304] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:21.151312 23605 raft_consensus.cc:2749] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:21.177343 23605 raft_consensus.cc:493] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:25:21.177949 23605 raft_consensus.cc:515] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:21.180276 23605 leader_election.cc:290] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:25:21.186832 16441 leader_election.cc:336] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:25:21.200810 16442 leader_election.cc:336] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
I20260709 12:25:21.201577 16442 leader_election.cc:304] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:21.202952 23605 raft_consensus.cc:2749] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:25:21.234223 16515 consensus_peers.cc:597] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:25:21.244544 16515 consensus_peers.cc:597] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:25:21.268491 23571 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:25:21.269167 23571 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:25:21.271610 23571 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:25:21.273324 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:25:21.284204 23571 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:25:21.284739 23571 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:25:21.291563 16515 leader_election.cc:336] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:21.286516 23571 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:25:21.292917 16516 leader_election.cc:336] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
I20260709 12:25:21.293445 16516 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:21.294410 23571 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:25:21.296640 16516 leader_election.cc:336] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
I20260709 12:25:21.297111 16516 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:21.297875 23571 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:21.301501 23605 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:25:21.302107 23605 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:25:21.303607 23587 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:25:21.304142 23587 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:25:21.304643 23605 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:25:21.306310 23587 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:25:21.309465 23599 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:25:21.309653 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:25:21.310108 23599 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:25:21.311429 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:25:21.312397 23599 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:25:21.321545 16515 leader_election.cc:336] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:25:21.323740 16516 leader_election.cc:336] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
I20260709 12:25:21.324169 16516 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:21.324882 23599 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:25:21.326884 16442 leader_election.cc:336] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
I20260709 12:25:21.327399 16442 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
W20260709 12:25:21.328279 16442 leader_election.cc:336] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
I20260709 12:25:21.328372 23587 raft_consensus.cc:2749] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:21.328733 16442 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:21.332383 23605 raft_consensus.cc:2749] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:21.335790 23599 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:25:21.336472 23599 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:25:21.339059 23599 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:25:21.346658 16515 leader_election.cc:336] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:25:21.347115 16516 leader_election.cc:336] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
I20260709 12:25:21.348771 16516 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:21.350435 23599 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:21.364862 23571 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:25:21.365494 23571 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:25:21.368454 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:25:21.369057 16477 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 19.
I20260709 12:25:21.370301 16514 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
W20260709 12:25:21.371693 16516 leader_election.cc:336] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
I20260709 12:25:21.371249 23599 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:21.378947 23599 raft_consensus.cc:493] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:25:21.379434 23599 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:21.372432 23605 raft_consensus.cc:493] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:25:21.380935 23605 raft_consensus.cc:515] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:21.382966 23605 leader_election.cc:290] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:25:21.385090 23606 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:25:21.386030 23606 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:25:21.388192 23606 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
W20260709 12:25:21.390769 16441 leader_election.cc:336] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:21.372557 23571 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:25:21.408915 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:25:21.409744 16543 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 18.
W20260709 12:25:21.412547 16442 leader_election.cc:336] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
I20260709 12:25:21.416603 16442 leader_election.cc:304] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:21.412654 16441 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:25:21.417444 23587 raft_consensus.cc:2749] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:21.418175 23587 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:21.418512 23587 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:25:21.418859 23587 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:21.415072 23599 raft_consensus.cc:515] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:25:21.422539 23599 leader_election.cc:290] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:25:21.422813 16477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba50aa1e4bd43928d6d7e4a7d6a60d1" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:25:21.423462 16477 raft_consensus.cc:3060] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:25:21.430677 16516 leader_election.cc:336] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
I20260709 12:25:21.440723 16477 raft_consensus.cc:2468] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 20.
W20260709 12:25:21.448112 16442 leader_election.cc:336] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
I20260709 12:25:21.450024 16514 leader_election.cc:304] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:21.451594 23599 raft_consensus.cc:2804] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:25:21.443439 23587 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:25:21.453812 23599 raft_consensus.cc:697] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 20 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:25:21.456424 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7b4960094cc44ceb337b594b1c269e6" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1"
I20260709 12:25:21.455379 23599 consensus_queue.cc:237] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:25:21.457070 16543 raft_consensus.cc:3060] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:21.466179 16543 raft_consensus.cc:2468] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ab50b0237346c1a13ede947f6a63c2 in term 19.
W20260709 12:25:21.467659 16515 consensus_peers.cc:597] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:25:21.470341 16441 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:25:21.471247 23605 raft_consensus.cc:2804] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:21.471791 23605 raft_consensus.cc:697] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 LEADER]: Becoming Leader. State: Replica: 52ab50b0237346c1a13ede947f6a63c2, State: Running, Role: LEADER
I20260709 12:25:21.472149 23571 raft_consensus.cc:493] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:25:21.472987 23571 raft_consensus.cc:515] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:25:21.472811 23605 consensus_queue.cc:237] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:25:21.478649 23571 leader_election.cc:290] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:25:21.496477 23587 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:25:21.497474 23571 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:25:21.498018 23571 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:25:21.498867 16442 leader_election.cc:336] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
W20260709 12:25:21.499677 16516 leader_election.cc:336] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
I20260709 12:25:21.500051 23571 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:25:21.495929 16221 catalog_manager.cc:5697] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 reported cstate change: term changed from 18 to 19, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3). New cstate: current_term: 19 leader_uuid: "52ab50b0237346c1a13ede947f6a63c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:21.497045 16515 consensus_peers.cc:597] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:25:21.505995 16516 leader_election.cc:336] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
W20260709 12:25:21.503654 16515 leader_election.cc:336] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:21.507251 16515 leader_election.cc:304] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:21.517776 23587 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:25:21.518386 23587 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:25:21.520530 23587 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:25:21.521289 23612 raft_consensus.cc:2749] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:21.527194 16543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
I20260709 12:25:21.527832 16223 catalog_manager.cc:5697] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 19 to 20, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 20 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:21.535823 16515 leader_election.cc:336] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:21.536425 16515 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:21.538296 23599 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:25:21.541769 16515 consensus_peers.cc:597] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:25:21.543179 16442 leader_election.cc:336] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
I20260709 12:25:21.546268 23587 raft_consensus.cc:493] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:25:21.546074 16442 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:21.546784 23587 raft_consensus.cc:515] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:21.547544 23605 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:21.551317 23587 leader_election.cc:290] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:25:21.553448 16441 leader_election.cc:336] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:21.556252  5352 tablet_server.cc:196] TabletServer@127.5.58.2:0 shutdown complete.
W20260709 12:25:21.564287 16442 leader_election.cc:336] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
I20260709 12:25:21.566437 16442 leader_election.cc:304] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:21.567256 23587 raft_consensus.cc:2749] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:21.598985 23571 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:25:21.599682 23571 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:25:21.602843 23571 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:25:21.607913 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1faa3c7bc3e4073bb65968a654e815d" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:25:21.608826 16473 raft_consensus.cc:2468] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 19.
I20260709 12:25:21.610474 16514 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
W20260709 12:25:21.615760 16516 leader_election.cc:336] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
I20260709 12:25:21.619592 23599 raft_consensus.cc:2804] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:21.620116 23599 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:25:21.620563 23599 raft_consensus.cc:3060] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:21.620635 23571 raft_consensus.cc:493] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:25:21.621248 23571 raft_consensus.cc:515] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:25:21.623441 23571 leader_election.cc:290] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:25:21.627279 16515 leader_election.cc:336] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:25:21.629515 16516 leader_election.cc:336] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
I20260709 12:25:21.629963 16516 leader_election.cc:304] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:21.630277 23599 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:25:21.631494 23571 raft_consensus.cc:2749] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:21.640311 23611 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:25:21.640954 23611 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:25:21.643126 23611 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:25:21.645589 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2" is_pre_election: true
I20260709 12:25:21.646291 16473 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 18.
I20260709 12:25:21.650590 16514 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:25:21.652814 23605 raft_consensus.cc:493] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:25:21.653492 23605 raft_consensus.cc:515] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:21.656635 23597 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:21.657114 23597 raft_consensus.cc:493] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:25:21.657488 23597 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:21.665170 23597 raft_consensus.cc:515] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:25:21.668416 16473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca74e9d2a1f4ef6bc63f8a1deeb05fa" candidate_uuid: "610f4feda11942739236f2b121a4b9c1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52ab50b0237346c1a13ede947f6a63c2"
I20260709 12:25:21.669057 16473 raft_consensus.cc:3060] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:21.656116 23605 leader_election.cc:290] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:25:21.670853 16442 leader_election.cc:336] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
W20260709 12:25:21.671694 16441 leader_election.cc:336] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:21.672094 16441 leader_election.cc:304] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:21.673213 23605 raft_consensus.cc:2749] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:25:21.677242 16515 consensus_peers.cc:597] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:25:21.682515 23605 raft_consensus.cc:493] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:25:21.683174 23605 raft_consensus.cc:515] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } }
I20260709 12:25:21.685467 16473 raft_consensus.cc:2468] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 610f4feda11942739236f2b121a4b9c1 in term 19.
I20260709 12:25:21.687800 16514 leader_election.cc:304] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2, 610f4feda11942739236f2b121a4b9c1; no voters: 
I20260709 12:25:21.688759 23612 raft_consensus.cc:2804] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:21.689373 23612 raft_consensus.cc:697] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 19 LEADER]: Becoming Leader. State: Replica: 610f4feda11942739236f2b121a4b9c1, State: Running, Role: LEADER
I20260709 12:25:21.690409 23612 consensus_queue.cc:237] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
W20260709 12:25:21.692849 16442 leader_election.cc:336] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
W20260709 12:25:21.697113 16516 leader_election.cc:336] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
I20260709 12:25:21.685554 23605 leader_election.cc:290] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:25:21.686491 23597 leader_election.cc:290] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:25:21.701182 23599 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 election: Requested vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:25:21.701929 16441 leader_election.cc:336] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:21.702468 16441 leader_election.cc:304] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:21.703236 23605 raft_consensus.cc:2749] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:25:21.703783 16516 leader_election.cc:336] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
W20260709 12:25:21.706327 16516 leader_election.cc:336] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
I20260709 12:25:21.719178  5352 tablet_server.cc:179] TabletServer@127.5.58.3:0 shutting down...
I20260709 12:25:21.734701 16223 catalog_manager.cc:5697] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 reported cstate change: term changed from 18 to 19, leader changed from f99066d13e524238aec6aa2024a8bd50 (127.5.58.2) to 610f4feda11942739236f2b121a4b9c1 (127.5.58.4). New cstate: current_term: 19 leader_uuid: "610f4feda11942739236f2b121a4b9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:21.737430 23606 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:25:21.738555 23606 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:25:21.740872 23606 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 610f4feda11942739236f2b121a4b9c1 (127.5.58.4:44491)
I20260709 12:25:21.744819 16549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e88d061c8b85425aa7478ab3874c4be4" candidate_uuid: "52ab50b0237346c1a13ede947f6a63c2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "610f4feda11942739236f2b121a4b9c1" is_pre_election: true
W20260709 12:25:21.751658 16515 consensus_peers.cc:597] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:25:21.754597 16514 leader_election.cc:336] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:25:21.755122 16514 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
W20260709 12:25:21.756240 16514 consensus_peers.cc:597] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 -> Peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Couldn't send request to peer 52ab50b0237346c1a13ede947f6a63c2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:21.756902 23612 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:25:21.764369 16442 leader_election.cc:336] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
I20260709 12:25:21.772255 16442 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52ab50b0237346c1a13ede947f6a63c2; no voters: 610f4feda11942739236f2b121a4b9c1, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:21.768018 23618 raft_consensus.cc:493] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:25:21.774338 23605 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:21.774261 23618 raft_consensus.cc:515] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:25:21.777088 23618 leader_election.cc:290] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
I20260709 12:25:21.764529 23612 raft_consensus.cc:493] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
W20260709 12:25:21.794260 16516 leader_election.cc:336] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
I20260709 12:25:21.780203 23612 raft_consensus.cc:515] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
W20260709 12:25:21.799000 16514 leader_election.cc:336] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:25:21.799443 16514 leader_election.cc:304] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:21.800380 23618 raft_consensus.cc:2749] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:25:21.801805 16514 consensus_peers.cc:597] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 -> Peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Couldn't send request to peer 52ab50b0237346c1a13ede947f6a63c2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:21.803476 23612 leader_election.cc:290] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:25:21.805260 16515 leader_election.cc:336] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:25:21.806939 16514 consensus_peers.cc:597] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 -> Peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Couldn't send request to peer 52ab50b0237346c1a13ede947f6a63c2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:21.815544 16516 leader_election.cc:336] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
I20260709 12:25:21.816058 16516 leader_election.cc:304] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:21.817950 23618 raft_consensus.cc:2749] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:25:21.854765 16515 consensus_peers.cc:597] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:25:21.860365 16549 raft_consensus.cc:1275] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Refusing update from remote peer 52ab50b0237346c1a13ede947f6a63c2: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:25:21.861970 23605 consensus_queue.cc:1048] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:25:21.863305 16514 consensus_peers.cc:597] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 -> Peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Couldn't send request to peer 52ab50b0237346c1a13ede947f6a63c2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:21.864889  5352 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:25:21.872303  5352 tablet_replica.cc:333] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:21.873286  5352 raft_consensus.cc:2243] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:25:21.876114  5352 raft_consensus.cc:2272] T 2546c2ffdb52493c972a8bf000c78ed5 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:21.879570  5352 tablet_replica.cc:333] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:21.880946  5352 raft_consensus.cc:2243] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:21.881659  5352 raft_consensus.cc:2272] T 05c8566780d1400ab2e42d9529d97ceb P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:21.884996  5352 tablet_replica.cc:333] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:21.886159  5352 raft_consensus.cc:2243] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:25:21.887849  5352 raft_consensus.cc:2272] T ad2476e68a0d4ae695099a41865c99c3 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:21.891894  5352 tablet_replica.cc:333] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:21.893330  5352 raft_consensus.cc:2243] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:21.894213  5352 raft_consensus.cc:2272] T c1a154d032654a179da41ad8f10fa339 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:21.898612  5352 tablet_replica.cc:333] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:21.899780  5352 raft_consensus.cc:2243] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 LEADER]: Raft consensus shutting down.
W20260709 12:25:21.900839 16442 consensus_peers.cc:597] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:21.901473  5352 raft_consensus.cc:2272] T 7889f3ab253b473eafb86002e9612a25 P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:21.905448  5352 tablet_replica.cc:333] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:21.906430  5352 raft_consensus.cc:2243] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:21.907168  5352 raft_consensus.cc:2272] T 07c17e01135a4bae850c5038c62d2ba2 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:21.908306 16514 consensus_peers.cc:597] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 -> Peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Couldn't send request to peer 52ab50b0237346c1a13ede947f6a63c2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:21.910622 16514 consensus_peers.cc:597] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 -> Peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Couldn't send request to peer 52ab50b0237346c1a13ede947f6a63c2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:21.910638  5352 tablet_replica.cc:333] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:21.913718  5352 raft_consensus.cc:2243] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:21.914568  5352 raft_consensus.cc:2272] T d1faa3c7bc3e4073bb65968a654e815d P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:21.917588  5352 tablet_replica.cc:333] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:21.919047  5352 raft_consensus.cc:2243] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:21.919878  5352 raft_consensus.cc:2272] T 6d860a1605704f96b47bc65033809793 P 52ab50b0237346c1a13ede947f6a63c2 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:21.923426  5352 tablet_replica.cc:333] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:21.924501  5352 raft_consensus.cc:2243] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:25:21.925794  5352 raft_consensus.cc:2272] T b6d1babdc37d40e4a71ffde9b310feaa P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:21.929060  5352 tablet_replica.cc:333] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:21.929972  5352 raft_consensus.cc:2243] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:21.930748  5352 raft_consensus.cc:2272] T c0b76873ad2645ebb97d6f5b03655773 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:21.931586 16514 consensus_peers.cc:597] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 -> Peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Couldn't send request to peer 52ab50b0237346c1a13ede947f6a63c2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:21.934122  5352 tablet_replica.cc:333] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:21.935039  5352 raft_consensus.cc:2243] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:21.935724  5352 raft_consensus.cc:2272] T de44585291c44bfe9a6f7168d04169b8 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:21.938830  5352 tablet_replica.cc:333] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
W20260709 12:25:21.938968 16514 consensus_peers.cc:597] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 -> Peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Couldn't send request to peer 52ab50b0237346c1a13ede947f6a63c2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:21.939586  5352 raft_consensus.cc:2243] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:21.940526  5352 raft_consensus.cc:2272] T e768efb7fcef4624b08d1e122c6dc825 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:21.943377  5352 tablet_replica.cc:333] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:21.944232  5352 raft_consensus.cc:2243] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:21.945086  5352 raft_consensus.cc:2272] T 7fe8595f696842c8821880524389c68b P 52ab50b0237346c1a13ede947f6a63c2 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:21.948099  5352 tablet_replica.cc:333] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:21.948864  5352 raft_consensus.cc:2243] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:21.949542  5352 raft_consensus.cc:2272] T 57fce16c9dc745ce8d5a631936c11e00 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:21.951583  5352 tablet_replica.cc:333] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:21.952627  5352 raft_consensus.cc:2243] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:21.953235  5352 raft_consensus.cc:2272] T 6a90f33738a74f2c9fbf18bb0199dccf P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:21.955770  5352 tablet_replica.cc:333] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:21.956544  5352 raft_consensus.cc:2243] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:21.957106  5352 raft_consensus.cc:2272] T 1215b509464e4efe977922ffdeb0b064 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:21.960521  5352 tablet_replica.cc:333] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:21.961975  5352 raft_consensus.cc:2243] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:25:21.963284  5352 raft_consensus.cc:2272] T b7b4960094cc44ceb337b594b1c269e6 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:21.963806 16514 consensus_peers.cc:597] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 -> Peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Couldn't send request to peer 52ab50b0237346c1a13ede947f6a63c2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:21.966868  5352 tablet_replica.cc:333] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:21.967780  5352 raft_consensus.cc:2243] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:25:21.969143  5352 raft_consensus.cc:2272] T 472c1e509cf2401e8153946529cfba2e P 52ab50b0237346c1a13ede947f6a63c2 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:21.972096  5352 tablet_replica.cc:333] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:21.973178  5352 raft_consensus.cc:2243] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:21.973959  5352 raft_consensus.cc:2272] T 44b9e8076a1d43319b0c86d4b356ac68 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:21.976994  5352 tablet_replica.cc:333] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:21.977861  5352 raft_consensus.cc:2243] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:25:21.978971  5352 raft_consensus.cc:2272] T 9cd596d6e69a4b58804e76d2dd4f51f1 P 52ab50b0237346c1a13ede947f6a63c2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:21.982301  5352 tablet_replica.cc:333] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:21.983116  5352 raft_consensus.cc:2243] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:21.983770  5352 raft_consensus.cc:2272] T 766cb112f8024db5af150d6be611a597 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:21.986598  5352 tablet_replica.cc:333] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:21.987327  5352 raft_consensus.cc:2243] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:21.987933  5352 raft_consensus.cc:2272] T d46e3072dec049b18b323d65470603ec P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:21.989282 16514 consensus_peers.cc:597] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 -> Peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Couldn't send request to peer 52ab50b0237346c1a13ede947f6a63c2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:21.990662  5352 tablet_replica.cc:333] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:21.991626  5352 raft_consensus.cc:2243] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:25:21.992684  5352 raft_consensus.cc:2272] T 95b81a29889843548dd6ed149807ecc8 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:21.995122 16441 consensus_peers.cc:597] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:25:21.996246  5352 tablet_replica.cc:333] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:21.997308  5352 raft_consensus.cc:2243] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:21.998087  5352 raft_consensus.cc:2272] T 2da151f033444a308804c713dd5ab691 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:22.001246  5352 tablet_replica.cc:333] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:22.002089  5352 raft_consensus.cc:2243] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Raft consensus shutting down.
W20260709 12:25:22.002723 16516 consensus_peers.cc:597] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:22.003219  5352 raft_consensus.cc:2272] T 0b9c348d235e432d92d8f8f20f1f1317 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:22.007084 16514 consensus_peers.cc:597] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 -> Peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Couldn't send request to peer 52ab50b0237346c1a13ede947f6a63c2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:22.007438  5352 tablet_replica.cc:333] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:22.008381  5352 raft_consensus.cc:2243] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:22.008973  5352 raft_consensus.cc:2272] T 7ff57aebc0fb4c6dbb6077f83ac8a60e P 52ab50b0237346c1a13ede947f6a63c2 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:22.011662  5352 tablet_replica.cc:333] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:22.012418  5352 raft_consensus.cc:2243] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:25:22.013463  5352 raft_consensus.cc:2272] T aea54881468a40aaabad162d385fc6d9 P 52ab50b0237346c1a13ede947f6a63c2 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:22.016769  5352 tablet_replica.cc:333] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:22.017665  5352 raft_consensus.cc:2243] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:22.018275  5352 raft_consensus.cc:2272] T 4777f1999cdc462c8526206c2f39fe0b P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:22.021703  5352 tablet_replica.cc:333] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:22.022472  5352 raft_consensus.cc:2243] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:25:22.023605  5352 raft_consensus.cc:2272] T defc374629a649a8a47502f058be2c8c P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:22.024139 16514 consensus_peers.cc:597] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 -> Peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Couldn't send request to peer 52ab50b0237346c1a13ede947f6a63c2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:22.026273  5352 tablet_replica.cc:333] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:22.027125  5352 raft_consensus.cc:2243] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:22.027772  5352 raft_consensus.cc:2272] T 025780567e194437bc14e26465b690e3 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:22.030476  5352 tablet_replica.cc:333] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:22.031245  5352 raft_consensus.cc:2243] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:25:22.032274  5352 raft_consensus.cc:2272] T 162cebc248914fb5aa57de3a383a4159 P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:22.035348  5352 tablet_replica.cc:333] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:22.036226  5352 raft_consensus.cc:2243] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:22.036975  5352 raft_consensus.cc:2272] T 48bd85a0339b41fba1759ca092d6e95e P 52ab50b0237346c1a13ede947f6a63c2 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:22.038183 16514 consensus_peers.cc:597] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 -> Peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Couldn't send request to peer 52ab50b0237346c1a13ede947f6a63c2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:22.039836  5352 tablet_replica.cc:333] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:22.040817  5352 raft_consensus.cc:2243] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:25:22.042138  5352 raft_consensus.cc:2272] T 3c4fd014cdf249c996641fe7793f42a5 P 52ab50b0237346c1a13ede947f6a63c2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:22.045246  5352 tablet_replica.cc:333] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:22.046264  5352 raft_consensus.cc:2243] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:22.047070  5352 raft_consensus.cc:2272] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 52ab50b0237346c1a13ede947f6a63c2 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:22.049739  5352 tablet_replica.cc:333] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:22.050540  5352 raft_consensus.cc:2243] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:22.051234  5352 raft_consensus.cc:2272] T e88d061c8b85425aa7478ab3874c4be4 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:22.053918  5352 tablet_replica.cc:333] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:22.054735  5352 raft_consensus.cc:2243] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:25:22.055879  5352 raft_consensus.cc:2272] T 33037c31b4434e838d90b2a2065ef52d P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:22.058599  5352 tablet_replica.cc:333] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:22.059293  5352 raft_consensus.cc:2243] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:22.059899  5352 raft_consensus.cc:2272] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 52ab50b0237346c1a13ede947f6a63c2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:22.062534  5352 tablet_replica.cc:333] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:22.063261  5352 raft_consensus.cc:2243] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:22.063829  5352 raft_consensus.cc:2272] T a60dbe0067044806b384e90805c877bc P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:22.066495  5352 tablet_replica.cc:333] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:22.067193  5352 raft_consensus.cc:2243] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:25:22.068413  5352 raft_consensus.cc:2272] T 14d8123994de43ec8e36a4e02961bc90 P 52ab50b0237346c1a13ede947f6a63c2 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:22.072686  5352 tablet_replica.cc:333] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:22.073522  5352 raft_consensus.cc:2243] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:25:22.075812  5352 raft_consensus.cc:2272] T 95954ea821224e4d962974a83d83ad44 P 52ab50b0237346c1a13ede947f6a63c2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:22.079150  5352 tablet_replica.cc:333] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:22.079988  5352 raft_consensus.cc:2243] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:22.080840  5352 raft_consensus.cc:2272] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:22.083962  5352 tablet_replica.cc:333] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:22.084885  5352 raft_consensus.cc:2243] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:22.085685  5352 raft_consensus.cc:2272] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 52ab50b0237346c1a13ede947f6a63c2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:22.088619  5352 tablet_replica.cc:333] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:22.089361  5352 raft_consensus.cc:2243] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 LEADER]: Raft consensus shutting down.
W20260709 12:25:22.089375 16515 consensus_peers.cc:597] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:25:22.090656  5352 raft_consensus.cc:2272] T 3851c76367ba4b3eaf3a1b5833ef4835 P 52ab50b0237346c1a13ede947f6a63c2 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:22.092052 16515 consensus_peers.cc:597] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:25:22.093786  5352 tablet_replica.cc:333] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2: stopping tablet replica
I20260709 12:25:22.094581  5352 raft_consensus.cc:2243] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:22.095263  5352 raft_consensus.cc:2272] T 4a0e81993a5641399dd0ca4ded8b4037 P 52ab50b0237346c1a13ede947f6a63c2 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:22.111853 16514 consensus_peers.cc:597] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 -> Peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Couldn't send request to peer 52ab50b0237346c1a13ede947f6a63c2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:22.113480 16514 consensus_peers.cc:597] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 -> Peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Couldn't send request to peer 52ab50b0237346c1a13ede947f6a63c2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:22.154066 16514 consensus_peers.cc:597] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 -> Peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Couldn't send request to peer 52ab50b0237346c1a13ede947f6a63c2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:22.219478 16514 consensus_peers.cc:597] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 -> Peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Couldn't send request to peer 52ab50b0237346c1a13ede947f6a63c2. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.3:33107: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:22.222368 16516 consensus_peers.cc:597] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:22.225098 16514 consensus_peers.cc:597] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 -> Peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Couldn't send request to peer 52ab50b0237346c1a13ede947f6a63c2. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.3:33107: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:22.225594 16514 consensus_peers.cc:597] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 -> Peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Couldn't send request to peer 52ab50b0237346c1a13ede947f6a63c2. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.3:33107: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:22.335022 16514 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.58.3:33107: connect: Connection refused (error 111) [suppressed 567 similar messages]
W20260709 12:25:22.358512 16515 consensus_peers.cc:597] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:25:22.482918 16515 consensus_peers.cc:597] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:25:22.531322 16516 consensus_peers.cc:597] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:22.574577 16515 consensus_peers.cc:597] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:25:22.628611 16516 consensus_peers.cc:597] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:22.629168 16516 consensus_peers.cc:597] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:22.697074 16516 consensus_peers.cc:597] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:22.726941 16516 consensus_peers.cc:597] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:22.739112 16515 consensus_peers.cc:597] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:25:22.766352 16516 consensus_peers.cc:597] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:22.813118 16516 consensus_peers.cc:597] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:22.822949 16516 consensus_peers.cc:597] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:22.839601 16516 consensus_peers.cc:597] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:22.905637 23625 raft_consensus.cc:493] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:25:22.906180 23625 raft_consensus.cc:515] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:25:22.908305 23625 leader_election.cc:290] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:25:22.912766 16515 leader_election.cc:336] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:25:22.915170 16516 leader_election.cc:336] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
I20260709 12:25:22.915844 16516 leader_election.cc:304] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:22.918802 23625 raft_consensus.cc:2749] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:25:22.921377 16516 consensus_peers.cc:597] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:22.986608 16516 consensus_peers.cc:597] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:23.031527 16516 consensus_peers.cc:597] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:23.091060 16516 consensus_peers.cc:597] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 -> Peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Couldn't send request to peer f99066d13e524238aec6aa2024a8bd50. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:25:23.093961 23625 raft_consensus.cc:493] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:25:23.094504 23625 raft_consensus.cc:515] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:25:23.096632 23625 leader_election.cc:290] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:25:23.101178 16516 leader_election.cc:336] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
W20260709 12:25:23.101625 16514 leader_election.cc:336] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:33107: connect: Connection refused (error 111)
I20260709 12:25:23.102053 16514 leader_election.cc:304] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:23.102752 23625 raft_consensus.cc:2749] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:23.135476 23623 raft_consensus.cc:493] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:25:23.135874 23625 raft_consensus.cc:493] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:25:23.136034 23623 raft_consensus.cc:515] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:25:23.136611 23625 raft_consensus.cc:515] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:25:23.138231 23623 leader_election.cc:290] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
I20260709 12:25:23.138716 23625 leader_election.cc:290] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:25:23.144965 16515 leader_election.cc:336] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:25:23.145301 16516 leader_election.cc:336] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
W20260709 12:25:23.145654 16514 leader_election.cc:336] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:33107: connect: Connection refused (error 111)
I20260709 12:25:23.146152 16514 leader_election.cc:304] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:25:23.146829 23623 raft_consensus.cc:2749] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:25:23.146974 16514 leader_election.cc:336] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:33107: connect: Connection refused (error 111)
I20260709 12:25:23.147498 16514 leader_election.cc:304] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:23.148160 23623 raft_consensus.cc:2749] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:23.151182 23625 raft_consensus.cc:493] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:25:23.151721 23625 raft_consensus.cc:515] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:25:23.153795 23625 leader_election.cc:290] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:25:23.160830 16516 leader_election.cc:336] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
W20260709 12:25:23.161453 16514 leader_election.cc:336] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:33107: connect: Connection refused (error 111)
I20260709 12:25:23.161980 16514 leader_election.cc:304] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:23.162679 23625 raft_consensus.cc:2749] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:25:23.163187 16515 consensus_peers.cc:597] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:25:23.217692 23623 raft_consensus.cc:493] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:25:23.218254 23623 raft_consensus.cc:515] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:25:23.220162 23623 leader_election.cc:290] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:25:23.224414 16515 leader_election.cc:336] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
W20260709 12:25:23.225055 16516 leader_election.cc:336] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
I20260709 12:25:23.225459 16516 leader_election.cc:304] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:23.226131 23623 raft_consensus.cc:2749] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:23.245793 23625 raft_consensus.cc:493] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:25:23.246351 23625 raft_consensus.cc:515] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:25:23.248703 23625 leader_election.cc:290] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:25:23.252408 16514 leader_election.cc:336] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:33107: connect: Connection refused (error 111)
W20260709 12:25:23.253113 16515 leader_election.cc:336] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:23.253469 16515 leader_election.cc:304] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:25:23.254086 23625 raft_consensus.cc:2749] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:23.261658 23625 raft_consensus.cc:493] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:25:23.262176 23625 raft_consensus.cc:515] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:25:23.264485 23625 leader_election.cc:290] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
W20260709 12:25:23.268694 16516 leader_election.cc:336] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
W20260709 12:25:23.269217 16515 leader_election.cc:336] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:23.269626 16515 leader_election.cc:304] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:23.270319 23625 raft_consensus.cc:2749] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:23.270843 23623 raft_consensus.cc:493] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:25:23.271401 23623 raft_consensus.cc:515] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } }
I20260709 12:25:23.273269 23623 leader_election.cc:290] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107), ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805)
W20260709 12:25:23.277387 16514 leader_election.cc:336] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:33107: connect: Connection refused (error 111)
W20260709 12:25:23.277900 16515 leader_election.cc:336] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:23.278352 16515 leader_election.cc:304] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, ef57beab0e1d40d585983873c55107e8
I20260709 12:25:23.278988 23623 raft_consensus.cc:2749] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:23.290772 23623 raft_consensus.cc:493] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:25:23.291332 23623 raft_consensus.cc:515] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:25:23.294320 23623 leader_election.cc:290] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:25:23.297484 16516 leader_election.cc:336] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
I20260709 12:25:23.298159  5352 tablet_server.cc:196] TabletServer@127.5.58.3:0 shutdown complete.
W20260709 12:25:23.300261 16514 leader_election.cc:336] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:33107: connect: Connection refused (error 111)
I20260709 12:25:23.300899 16514 leader_election.cc:304] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:23.301764 23623 raft_consensus.cc:2749] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:23.310135 23623 raft_consensus.cc:493] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.310613 23623 raft_consensus.cc:515] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:25:23.312487 23623 leader_election.cc:290] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:25:23.316118 16516 leader_election.cc:336] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
W20260709 12:25:23.316839 16514 leader_election.cc:336] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:33107: connect: Connection refused (error 111)
I20260709 12:25:23.317205 16514 leader_election.cc:304] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:23.317867 23623 raft_consensus.cc:2749] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:25:23.369236 16515 consensus_peers.cc:597] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:25:23.381446  5352 tablet_server.cc:179] TabletServer@127.5.58.4:0 shutting down...
I20260709 12:25:23.387142 23625 raft_consensus.cc:493] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 52ab50b0237346c1a13ede947f6a63c2)
I20260709 12:25:23.387759 23625 raft_consensus.cc:515] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } } peers { permanent_uuid: "52ab50b0237346c1a13ede947f6a63c2" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 33107 } } peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } }
I20260709 12:25:23.390175 23625 leader_election.cc:290] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367), 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107)
W20260709 12:25:23.396210 16516 leader_election.cc:336] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
W20260709 12:25:23.398931 16514 leader_election.cc:336] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 52ab50b0237346c1a13ede947f6a63c2 (127.5.58.3:33107): Network error: Client connection negotiation failed: client connection to 127.5.58.3:33107: connect: Connection refused (error 111)
I20260709 12:25:23.399997 16514 leader_election.cc:304] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: 52ab50b0237346c1a13ede947f6a63c2, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:23.401691 23625 raft_consensus.cc:2749] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:25:23.430866 16515 consensus_peers.cc:597] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:25:23.457549  5352 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:25:23.458773  5352 tablet_replica.cc:333] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
W20260709 12:25:23.459285 16515 consensus_peers.cc:597] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 -> Peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Couldn't send request to peer ef57beab0e1d40d585983873c55107e8. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:25:23.459789  5352 raft_consensus.cc:2243] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:23.460588  5352 raft_consensus.cc:2272] T 2546c2ffdb52493c972a8bf000c78ed5 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.463519  5352 tablet_replica.cc:333] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.464162  5352 raft_consensus.cc:2243] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:25:23.465094  5352 raft_consensus.cc:2272] T 14f9bc0dbb7e4c6f94aaf8146768ed79 P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.467177  5352 tablet_replica.cc:333] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.467846  5352 raft_consensus.cc:2243] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:23.468422  5352 raft_consensus.cc:2272] T ad2476e68a0d4ae695099a41865c99c3 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.471596  5352 tablet_replica.cc:333] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.472313  5352 raft_consensus.cc:2243] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:25:23.473801  5352 raft_consensus.cc:2272] T 6474b10917204b6994269be17387f38d P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.476924  5352 tablet_replica.cc:333] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.477871  5352 raft_consensus.cc:2243] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:25:23.478973  5352 raft_consensus.cc:2272] T de44585291c44bfe9a6f7168d04169b8 P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.481871  5352 tablet_replica.cc:333] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.482662  5352 raft_consensus.cc:2243] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:23.483320  5352 raft_consensus.cc:2272] T 7889f3ab253b473eafb86002e9612a25 P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.485543  5352 tablet_replica.cc:333] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.486202  5352 raft_consensus.cc:2243] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:23.486737  5352 raft_consensus.cc:2272] T b7b4960094cc44ceb337b594b1c269e6 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.489289  5352 tablet_replica.cc:333] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.490082  5352 raft_consensus.cc:2243] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:25:23.491272  5352 raft_consensus.cc:2272] T 7117f6656fa14807ac801392259e843a P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.494040  5352 tablet_replica.cc:333] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.494740  5352 raft_consensus.cc:2243] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:25:23.495589  5352 raft_consensus.cc:2272] T a4bb7e1a96a94612b2f6759156f5f7c0 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.498224  5352 tablet_replica.cc:333] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.498993  5352 raft_consensus.cc:2243] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:25:23.500087  5352 raft_consensus.cc:2272] T 07c17e01135a4bae850c5038c62d2ba2 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.502365  5352 tablet_replica.cc:333] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.502902  5352 raft_consensus.cc:2243] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:23.503346  5352 raft_consensus.cc:2272] T d1faa3c7bc3e4073bb65968a654e815d P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.505304  5352 tablet_replica.cc:333] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.505929  5352 raft_consensus.cc:2243] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:23.506425  5352 raft_consensus.cc:2272] T 701eedfc29544bbeafd695f04ccea545 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.508576  5352 tablet_replica.cc:333] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.509104  5352 raft_consensus.cc:2243] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:23.509507  5352 raft_consensus.cc:2272] T b6d1babdc37d40e4a71ffde9b310feaa P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.511423  5352 tablet_replica.cc:333] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.511948  5352 raft_consensus.cc:2243] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:25:23.512828  5352 raft_consensus.cc:2272] T c0b76873ad2645ebb97d6f5b03655773 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.513351 23625 raft_consensus.cc:493] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f99066d13e524238aec6aa2024a8bd50)
I20260709 12:25:23.513789 23625 raft_consensus.cc:515] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610f4feda11942739236f2b121a4b9c1" member_type: VOTER last_known_addr { host: "127.5.58.4" port: 44491 } } peers { permanent_uuid: "ef57beab0e1d40d585983873c55107e8" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 41805 } } peers { permanent_uuid: "f99066d13e524238aec6aa2024a8bd50" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 41367 } }
I20260709 12:25:23.515379  5352 tablet_replica.cc:333] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.515450 23625 leader_election.cc:290] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805), f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367)
I20260709 12:25:23.516502  5352 raft_consensus.cc:2243] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:25:23.517712  5352 raft_consensus.cc:2272] T 6d860a1605704f96b47bc65033809793 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:23.519380 16516 leader_election.cc:336] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f99066d13e524238aec6aa2024a8bd50 (127.5.58.2:41367): Network error: Client connection negotiation failed: client connection to 127.5.58.2:41367: connect: Connection refused (error 111)
W20260709 12:25:23.520182 16515 leader_election.cc:336] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ef57beab0e1d40d585983873c55107e8 (127.5.58.1:41805): Network error: Client connection negotiation failed: client connection to 127.5.58.1:41805: connect: Connection refused (error 111)
I20260709 12:25:23.520717  5352 tablet_replica.cc:333] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.520658 16515 leader_election.cc:304] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 610f4feda11942739236f2b121a4b9c1; no voters: ef57beab0e1d40d585983873c55107e8, f99066d13e524238aec6aa2024a8bd50
I20260709 12:25:23.521432 23625 raft_consensus.cc:2749] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:23.521709  5352 raft_consensus.cc:2243] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:25:23.522765  5352 raft_consensus.cc:2272] T 8c02e60dcbc14b2fa3a2c11a214a93a6 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.525115  5352 tablet_replica.cc:333] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.525744  5352 raft_consensus.cc:2243] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:23.526376  5352 raft_consensus.cc:2272] T e88d061c8b85425aa7478ab3874c4be4 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.528508  5352 tablet_replica.cc:333] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.529107  5352 raft_consensus.cc:2243] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:25:23.530273  5352 raft_consensus.cc:2272] T 7fe8595f696842c8821880524389c68b P 610f4feda11942739236f2b121a4b9c1 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.532930  5352 tablet_replica.cc:333] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.533579  5352 raft_consensus.cc:2243] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:23.534116  5352 raft_consensus.cc:2272] T dbdcb11d24644ca0bef26ea87647f4ac P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.536237  5352 tablet_replica.cc:333] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.536897  5352 raft_consensus.cc:2243] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:25:23.537806  5352 raft_consensus.cc:2272] T 1215b509464e4efe977922ffdeb0b064 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.539921  5352 tablet_replica.cc:333] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.540663  5352 raft_consensus.cc:2243] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:25:23.541700  5352 raft_consensus.cc:2272] T 48bd85a0339b41fba1759ca092d6e95e P 610f4feda11942739236f2b121a4b9c1 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.544004  5352 tablet_replica.cc:333] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.544694  5352 raft_consensus.cc:2243] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:25:23.545568  5352 raft_consensus.cc:2272] T 025780567e194437bc14e26465b690e3 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.547747  5352 tablet_replica.cc:333] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.548425  5352 raft_consensus.cc:2243] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:25:23.549327  5352 raft_consensus.cc:2272] T 4e0c5dbd539e4fa1a5f1e3fac0802b81 P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.551367  5352 tablet_replica.cc:333] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.551951  5352 raft_consensus.cc:2243] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:25:23.552894  5352 raft_consensus.cc:2272] T d46e3072dec049b18b323d65470603ec P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.555114  5352 tablet_replica.cc:333] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.555950  5352 raft_consensus.cc:2243] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:25:23.557098  5352 raft_consensus.cc:2272] T 2da151f033444a308804c713dd5ab691 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.559494  5352 tablet_replica.cc:333] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.560158  5352 raft_consensus.cc:2243] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:23.561002  5352 raft_consensus.cc:2272] T 49b3f77bde864e0ab25c63401793c709 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.563220  5352 tablet_replica.cc:333] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.564139  5352 raft_consensus.cc:2243] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:25:23.565431  5352 raft_consensus.cc:2272] T 57fce16c9dc745ce8d5a631936c11e00 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.567894  5352 tablet_replica.cc:333] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.568608  5352 raft_consensus.cc:2243] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:23.569113  5352 raft_consensus.cc:2272] T ef53202d665e46c39113572380be6c36 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.571347  5352 tablet_replica.cc:333] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.572062  5352 raft_consensus.cc:2243] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:23.572600  5352 raft_consensus.cc:2272] T 0b9c348d235e432d92d8f8f20f1f1317 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.574468  5352 tablet_replica.cc:333] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.575026  5352 raft_consensus.cc:2243] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:25:23.575778  5352 raft_consensus.cc:2272] T 766cb112f8024db5af150d6be611a597 P 610f4feda11942739236f2b121a4b9c1 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.577662  5352 tablet_replica.cc:333] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.578430  5352 raft_consensus.cc:2243] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:25:23.579203  5352 raft_consensus.cc:2272] T 191e052863dd48fe92ebe151197330a8 P 610f4feda11942739236f2b121a4b9c1 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.581264  5352 tablet_replica.cc:333] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.581799  5352 raft_consensus.cc:2243] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:23.582226  5352 raft_consensus.cc:2272] T 434028125a2d48998f4f5106c59a9d47 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.583967  5352 tablet_replica.cc:333] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.584537  5352 raft_consensus.cc:2243] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:23.584965  5352 raft_consensus.cc:2272] T 95954ea821224e4d962974a83d83ad44 P 610f4feda11942739236f2b121a4b9c1 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.586769  5352 tablet_replica.cc:333] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.587308  5352 raft_consensus.cc:2243] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:23.587734  5352 raft_consensus.cc:2272] T aadc86310ab642c69b7dbbf94be01070 P 610f4feda11942739236f2b121a4b9c1 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.589617  5352 tablet_replica.cc:333] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.590150  5352 raft_consensus.cc:2243] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:25:23.590891  5352 raft_consensus.cc:2272] T 1a79e67586f842a7ba4c25dfd7cdf4d5 P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.592758  5352 tablet_replica.cc:333] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.593401  5352 raft_consensus.cc:2243] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:23.593940  5352 raft_consensus.cc:2272] T aea54881468a40aaabad162d385fc6d9 P 610f4feda11942739236f2b121a4b9c1 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.596181  5352 tablet_replica.cc:333] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.596817  5352 raft_consensus.cc:2243] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:25:23.597704  5352 raft_consensus.cc:2272] T 4777f1999cdc462c8526206c2f39fe0b P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.599820  5352 tablet_replica.cc:333] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.600425  5352 raft_consensus.cc:2243] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:25:23.601269  5352 pending_rounds.cc:70] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1: Trying to abort 1 pending ops.
I20260709 12:25:23.601693  5352 pending_rounds.cc:77] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1: Aborting op as it isn't in flight: id { term: 19 index: 13 } timestamp: 7305625279263145984 op_type: NO_OP noop_request { }
I20260709 12:25:23.602123  5352 raft_consensus.cc:2889] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 19 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:25:23.602447  5352 raft_consensus.cc:2272] T 7ca74e9d2a1f4ef6bc63f8a1deeb05fa P 610f4feda11942739236f2b121a4b9c1 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.604664  5352 tablet_replica.cc:333] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.605268  5352 raft_consensus.cc:2243] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:25:23.606489  5352 raft_consensus.cc:2272] T 5f88a73c287745feb816089d0e08d10b P 610f4feda11942739236f2b121a4b9c1 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.608786  5352 tablet_replica.cc:333] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.609321  5352 raft_consensus.cc:2243] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:25:23.610100  5352 raft_consensus.cc:2272] T 4a0e81993a5641399dd0ca4ded8b4037 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.612016  5352 tablet_replica.cc:333] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.612579  5352 raft_consensus.cc:2243] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:25:23.613291  5352 raft_consensus.cc:2272] T edff15255cdf492dbc9c1189b889aa78 P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.615150  5352 tablet_replica.cc:333] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.615646  5352 raft_consensus.cc:2243] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:25:23.616482  5352 pending_rounds.cc:70] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1: Trying to abort 1 pending ops.
I20260709 12:25:23.616686  5352 pending_rounds.cc:77] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1: Aborting op as it isn't in flight: id { term: 20 index: 12 } timestamp: 7305625278391193600 op_type: NO_OP noop_request { }
I20260709 12:25:23.616986  5352 raft_consensus.cc:2889] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 20 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:25:23.617259  5352 raft_consensus.cc:2272] T 4ba50aa1e4bd43928d6d7e4a7d6a60d1 P 610f4feda11942739236f2b121a4b9c1 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.619020  5352 tablet_replica.cc:333] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.619498  5352 raft_consensus.cc:2243] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:25:23.620510  5352 raft_consensus.cc:2272] T a60dbe0067044806b384e90805c877bc P 610f4feda11942739236f2b121a4b9c1 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:23.622684  5352 tablet_replica.cc:333] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1: stopping tablet replica
I20260709 12:25:23.623159  5352 raft_consensus.cc:2243] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:23.623684  5352 raft_consensus.cc:2272] T 3ba84aa22298489eb2845bec782a34f2 P 610f4feda11942739236f2b121a4b9c1 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:24.303730  5352 tablet_server.cc:196] TabletServer@127.5.58.4:0 shutdown complete.
I20260709 12:25:24.436290  5352 master.cc:562] Master@127.5.58.62:39363 shutting down...
I20260709 12:25:24.488021  5352 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:25:24.488930  5352 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:24.489312  5352 tablet_replica.cc:333] T 00000000000000000000000000000000 P 9e68af265dd04372955944ded6c1437a: stopping tablet replica
I20260709 12:25:24.671681  5352 master.cc:584] Master@127.5.58.62:39363 shutdown complete.
I20260709 12:25:24.728868  5352 test_util.cc:182] -----------------------------------------------
I20260709 12:25:24.729121  5352 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599341860723-5352-0
[  FAILED  ] LeaderRebalancerTest.RestartTserver (219716 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 12:25:24.734081  5352 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.58.62:43883
I20260709 12:25:24.735353  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:25:24.742055 23627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:25:24.742033 23628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:25:24.743058 23630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:25:24.743878  5352 server_base.cc:1061] running on GCE node
I20260709 12:25:24.747189  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:25:24.747416  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:25:24.747601  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599924747576 us; error 0 us; skew 500 ppm
I20260709 12:25:24.748131  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:25:24.756803  5352 webserver.cc:533] Webserver started at http://127.5.58.62:36449/ using document root <none> and password file <none>
I20260709 12:25:24.757324  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:25:24.757550  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:25:24.757839  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:25:24.759088  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599341860723-5352-0/minicluster-data/master-0-root/instance:
uuid: "7fc82a5999f34967bbc914905d87549d"
format_stamp: "Formatted at 2026-07-09 12:25:24 on dist-test-slave-xs2d"
I20260709 12:25:24.763875  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.003s
I20260709 12:25:24.767880 23635 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:25:24.768961  5352 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:25:24.769376  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599341860723-5352-0/minicluster-data/master-0-root
uuid: "7fc82a5999f34967bbc914905d87549d"
format_stamp: "Formatted at 2026-07-09 12:25:24 on dist-test-slave-xs2d"
I20260709 12:25:24.769730  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599341860723-5352-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599341860723-5352-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599341860723-5352-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:25:24.804026  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:25:24.805308  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:25:24.850026  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.62:43883
I20260709 12:25:24.850142 23686 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.62:43883 every 8 connection(s)
I20260709 12:25:24.854382 23687 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:24.866772 23687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fc82a5999f34967bbc914905d87549d: Bootstrap starting.
I20260709 12:25:24.872407 23687 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7fc82a5999f34967bbc914905d87549d: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:24.877025 23687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fc82a5999f34967bbc914905d87549d: No bootstrap required, opened a new log
I20260709 12:25:24.879362 23687 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7fc82a5999f34967bbc914905d87549d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fc82a5999f34967bbc914905d87549d" member_type: VOTER }
I20260709 12:25:24.879839 23687 raft_consensus.cc:385] T 00000000000000000000000000000000 P 7fc82a5999f34967bbc914905d87549d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:24.880086 23687 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7fc82a5999f34967bbc914905d87549d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fc82a5999f34967bbc914905d87549d, State: Initialized, Role: FOLLOWER
I20260709 12:25:24.880795 23687 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7fc82a5999f34967bbc914905d87549d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fc82a5999f34967bbc914905d87549d" member_type: VOTER }
I20260709 12:25:24.881356 23687 raft_consensus.cc:399] T 00000000000000000000000000000000 P 7fc82a5999f34967bbc914905d87549d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:25:24.881592 23687 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7fc82a5999f34967bbc914905d87549d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:25:24.881899 23687 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7fc82a5999f34967bbc914905d87549d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:24.887044 23687 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7fc82a5999f34967bbc914905d87549d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fc82a5999f34967bbc914905d87549d" member_type: VOTER }
I20260709 12:25:24.887588 23687 leader_election.cc:304] T 00000000000000000000000000000000 P 7fc82a5999f34967bbc914905d87549d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7fc82a5999f34967bbc914905d87549d; no voters: 
I20260709 12:25:24.888828 23687 leader_election.cc:290] T 00000000000000000000000000000000 P 7fc82a5999f34967bbc914905d87549d [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:25:24.889339 23690 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7fc82a5999f34967bbc914905d87549d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:24.891284 23690 raft_consensus.cc:697] T 00000000000000000000000000000000 P 7fc82a5999f34967bbc914905d87549d [term 1 LEADER]: Becoming Leader. State: Replica: 7fc82a5999f34967bbc914905d87549d, State: Running, Role: LEADER
I20260709 12:25:24.892151 23687 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7fc82a5999f34967bbc914905d87549d [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:25:24.892138 23690 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7fc82a5999f34967bbc914905d87549d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fc82a5999f34967bbc914905d87549d" member_type: VOTER }
I20260709 12:25:24.899718 23692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fc82a5999f34967bbc914905d87549d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7fc82a5999f34967bbc914905d87549d. Latest consensus state: current_term: 1 leader_uuid: "7fc82a5999f34967bbc914905d87549d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fc82a5999f34967bbc914905d87549d" member_type: VOTER } }
I20260709 12:25:24.899770 23691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fc82a5999f34967bbc914905d87549d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7fc82a5999f34967bbc914905d87549d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fc82a5999f34967bbc914905d87549d" member_type: VOTER } }
I20260709 12:25:24.900786 23692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fc82a5999f34967bbc914905d87549d [sys.catalog]: This master's current role is: LEADER
I20260709 12:25:24.900882 23691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fc82a5999f34967bbc914905d87549d [sys.catalog]: This master's current role is: LEADER
I20260709 12:25:24.902859 23699 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:25:24.908474 23699 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:25:24.912730  5352 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:25:24.918408 23699 catalog_manager.cc:1383] Generated new cluster ID: 153d175bc89b4e068f239115aa67e163
I20260709 12:25:24.918740 23699 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:25:24.939064 23699 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:25:24.940696 23699 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:25:24.956645 23699 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 7fc82a5999f34967bbc914905d87549d: Generated new TSK 0
I20260709 12:25:24.957355 23699 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:25:24.980062  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:25:24.987655 23708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:25:24.989140 23709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:25:24.991597 23711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:25:24.992208  5352 server_base.cc:1061] running on GCE node
I20260709 12:25:24.994410  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:25:24.994693  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:25:24.994892  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599924994876 us; error 0 us; skew 500 ppm
I20260709 12:25:24.995566  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:25:24.999441  5352 webserver.cc:533] Webserver started at http://127.5.58.1:33487/ using document root <none> and password file <none>
I20260709 12:25:24.999954  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:25:25.000447  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:25:25.000875  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:25:25.003508  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599341860723-5352-0/minicluster-data/ts-0-root/instance:
uuid: "cc755c49216847f4a9ead01695cd2ab1"
format_stamp: "Formatted at 2026-07-09 12:25:24 on dist-test-slave-xs2d"
I20260709 12:25:25.013257  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.009s	user 0.007s	sys 0.001s
I20260709 12:25:25.018870 23716 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:25:25.019942  5352 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:25:25.020365  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599341860723-5352-0/minicluster-data/ts-0-root
uuid: "cc755c49216847f4a9ead01695cd2ab1"
format_stamp: "Formatted at 2026-07-09 12:25:24 on dist-test-slave-xs2d"
I20260709 12:25:25.020933  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599341860723-5352-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599341860723-5352-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599341860723-5352-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:25:25.043233  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:25:25.044857  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:25:25.046475  5352 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:25:25.049502  5352 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:25:25.049809  5352 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:25:25.050149  5352 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:25:25.050392  5352 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:25:25.106249  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.1:40445
I20260709 12:25:25.106459 23779 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.1:40445 every 8 connection(s)
I20260709 12:25:25.112439  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:25:25.120479 23784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:25:25.122880 23785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:25:25.127343 23787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:25:25.128953 23780 heartbeater.cc:344] Connected to a master server at 127.5.58.62:43883
I20260709 12:25:25.129367 23780 heartbeater.cc:461] Registering TS with master...
I20260709 12:25:25.130075  5352 server_base.cc:1061] running on GCE node
I20260709 12:25:25.130172 23780 heartbeater.cc:507] Master 127.5.58.62:43883 requested a full tablet report, sending...
I20260709 12:25:25.134549 23652 ts_manager.cc:194] Registered new tserver with Master: cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445)
I20260709 12:25:25.135548  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:25:25.135915  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:25:25.136160  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599925136139 us; error 0 us; skew 500 ppm
I20260709 12:25:25.137089  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:25:25.137465 23652 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51928
I20260709 12:25:25.140672  5352 webserver.cc:533] Webserver started at http://127.5.58.2:40423/ using document root <none> and password file <none>
I20260709 12:25:25.141249  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:25:25.141414  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:25:25.141727  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:25:25.143024  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599341860723-5352-0/minicluster-data/ts-1-root/instance:
uuid: "50493539c1954244889d4f3e6d55f396"
format_stamp: "Formatted at 2026-07-09 12:25:25 on dist-test-slave-xs2d"
I20260709 12:25:25.148170  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.004s
I20260709 12:25:25.152493 23792 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:25:25.153551  5352 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:25:25.153836  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599341860723-5352-0/minicluster-data/ts-1-root
uuid: "50493539c1954244889d4f3e6d55f396"
format_stamp: "Formatted at 2026-07-09 12:25:25 on dist-test-slave-xs2d"
I20260709 12:25:25.154115  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599341860723-5352-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599341860723-5352-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599341860723-5352-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:25:25.171438  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:25:25.172680  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:25:25.174131  5352 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:25:25.176764  5352 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:25:25.176996  5352 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:25:25.177235  5352 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:25:25.177420  5352 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:25:25.223311  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.2:38837
I20260709 12:25:25.223419 23854 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.2:38837 every 8 connection(s)
I20260709 12:25:25.228516  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:25:25.235658 23858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:25:25.238229 23859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:25:25.241990 23861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:25:25.242031  5352 server_base.cc:1061] running on GCE node
I20260709 12:25:25.242681 23855 heartbeater.cc:344] Connected to a master server at 127.5.58.62:43883
I20260709 12:25:25.243234 23855 heartbeater.cc:461] Registering TS with master...
I20260709 12:25:25.243412  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:25:25.243701  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:25:25.243908  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599925243887 us; error 0 us; skew 500 ppm
I20260709 12:25:25.244161 23855 heartbeater.cc:507] Master 127.5.58.62:43883 requested a full tablet report, sending...
I20260709 12:25:25.244761  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:25:25.246979 23652 ts_manager.cc:194] Registered new tserver with Master: 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837)
I20260709 12:25:25.248209  5352 webserver.cc:533] Webserver started at http://127.5.58.3:37907/ using document root <none> and password file <none>
I20260709 12:25:25.248773 23652 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51942
I20260709 12:25:25.248871  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:25:25.249433  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:25:25.249766  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:25:25.251480  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599341860723-5352-0/minicluster-data/ts-2-root/instance:
uuid: "565673cccffc4dbbb2d7431552ba280f"
format_stamp: "Formatted at 2026-07-09 12:25:25 on dist-test-slave-xs2d"
I20260709 12:25:25.256577  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:25:25.260133 23866 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:25:25.261209  5352 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:25:25.261646  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599341860723-5352-0/minicluster-data/ts-2-root
uuid: "565673cccffc4dbbb2d7431552ba280f"
format_stamp: "Formatted at 2026-07-09 12:25:25 on dist-test-slave-xs2d"
I20260709 12:25:25.261945  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599341860723-5352-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599341860723-5352-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599341860723-5352-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:25:25.279191  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:25:25.280951  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:25:25.282617  5352 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:25:25.285506  5352 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:25:25.285703  5352 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:25:25.285995  5352 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:25:25.286532  5352 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
I20260709 12:25:25.338979  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.3:35373
I20260709 12:25:25.339286 23928 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.3:35373 every 8 connection(s)
I20260709 12:25:25.355953 23929 heartbeater.cc:344] Connected to a master server at 127.5.58.62:43883
I20260709 12:25:25.356506 23929 heartbeater.cc:461] Registering TS with master...
I20260709 12:25:25.357226 23929 heartbeater.cc:507] Master 127.5.58.62:43883 requested a full tablet report, sending...
I20260709 12:25:25.359215 23652 ts_manager.cc:194] Registered new tserver with Master: 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373)
I20260709 12:25:25.359894  5352 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.01730754s
I20260709 12:25:25.360698 23652 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51946
I20260709 12:25:25.389657 23652 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51962:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:25:25.393561 23652 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:25:25.486670 23820 tablet_service.cc:1511] Processing CreateTablet for tablet 797d4300bcc7475ba608835b5d9addc2 (DEFAULT_TABLE table=test-workload [id=933040d0bb8b433c8bdb8b124e79c854]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:25:25.488430 23820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 797d4300bcc7475ba608835b5d9addc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:25.496836 23745 tablet_service.cc:1511] Processing CreateTablet for tablet 797d4300bcc7475ba608835b5d9addc2 (DEFAULT_TABLE table=test-workload [id=933040d0bb8b433c8bdb8b124e79c854]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:25:25.498546 23745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 797d4300bcc7475ba608835b5d9addc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:25.498857 23741 tablet_service.cc:1511] Processing CreateTablet for tablet 8085f6a4e929411097ff3c58f115ed80 (DEFAULT_TABLE table=test-workload [id=933040d0bb8b433c8bdb8b124e79c854]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:25:25.500470 23741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8085f6a4e929411097ff3c58f115ed80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:25.505424 23744 tablet_service.cc:1511] Processing CreateTablet for tablet 63fdfbf5f20f4c1ea9db0d4707c4861b (DEFAULT_TABLE table=test-workload [id=933040d0bb8b433c8bdb8b124e79c854]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:25:25.506145 23742 tablet_service.cc:1511] Processing CreateTablet for tablet 3ae7d028bd6548359867825588960bcb (DEFAULT_TABLE table=test-workload [id=933040d0bb8b433c8bdb8b124e79c854]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:25:25.507167 23744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63fdfbf5f20f4c1ea9db0d4707c4861b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:25.508064 23894 tablet_service.cc:1511] Processing CreateTablet for tablet 797d4300bcc7475ba608835b5d9addc2 (DEFAULT_TABLE table=test-workload [id=933040d0bb8b433c8bdb8b124e79c854]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:25:25.509812 23894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 797d4300bcc7475ba608835b5d9addc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:25.510138 23819 tablet_service.cc:1511] Processing CreateTablet for tablet 63fdfbf5f20f4c1ea9db0d4707c4861b (DEFAULT_TABLE table=test-workload [id=933040d0bb8b433c8bdb8b124e79c854]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:25:25.514246 23739 tablet_service.cc:1511] Processing CreateTablet for tablet e5a6d6eb60bb40cd83ba7787fc127421 (DEFAULT_TABLE table=test-workload [id=933040d0bb8b433c8bdb8b124e79c854]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:25:25.515450 23893 tablet_service.cc:1511] Processing CreateTablet for tablet 63fdfbf5f20f4c1ea9db0d4707c4861b (DEFAULT_TABLE table=test-workload [id=933040d0bb8b433c8bdb8b124e79c854]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:25:25.517395 23893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63fdfbf5f20f4c1ea9db0d4707c4861b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:25.517947 23740 tablet_service.cc:1511] Processing CreateTablet for tablet ee29c9afcbb94cadae2778d1f0d2fcd2 (DEFAULT_TABLE table=test-workload [id=933040d0bb8b433c8bdb8b124e79c854]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:25:25.523483 23818 tablet_service.cc:1511] Processing CreateTablet for tablet 3ae7d028bd6548359867825588960bcb (DEFAULT_TABLE table=test-workload [id=933040d0bb8b433c8bdb8b124e79c854]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:25:25.508177 23742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ae7d028bd6548359867825588960bcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:25.527464 23739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5a6d6eb60bb40cd83ba7787fc127421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:25.528851 23740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee29c9afcbb94cadae2778d1f0d2fcd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:25.530170 23819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63fdfbf5f20f4c1ea9db0d4707c4861b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:25.531867 23818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ae7d028bd6548359867825588960bcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:25.561827 23738 tablet_service.cc:1511] Processing CreateTablet for tablet 09fbbea470544d64b22e97e7f74a305b (DEFAULT_TABLE table=test-workload [id=933040d0bb8b433c8bdb8b124e79c854]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:25:25.563575 23738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09fbbea470544d64b22e97e7f74a305b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:25.565959 23892 tablet_service.cc:1511] Processing CreateTablet for tablet 3ae7d028bd6548359867825588960bcb (DEFAULT_TABLE table=test-workload [id=933040d0bb8b433c8bdb8b124e79c854]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:25:25.567643 23892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ae7d028bd6548359867825588960bcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:25.569612 23817 tablet_service.cc:1511] Processing CreateTablet for tablet 8085f6a4e929411097ff3c58f115ed80 (DEFAULT_TABLE table=test-workload [id=933040d0bb8b433c8bdb8b124e79c854]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:25:25.580075 23812 tablet_service.cc:1511] Processing CreateTablet for tablet 7101b7cd32744ca79f6c50b5e31f246a (DEFAULT_TABLE table=test-workload [id=933040d0bb8b433c8bdb8b124e79c854]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:25:25.580466 23811 tablet_service.cc:1511] Processing CreateTablet for tablet 2fedce3e0a6d41aa8954693aab3796cd (DEFAULT_TABLE table=test-workload [id=933040d0bb8b433c8bdb8b124e79c854]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:25:25.570864 23813 tablet_service.cc:1511] Processing CreateTablet for tablet 5bcd8bf7d5d04504b30ddbb04f6bc7fd (DEFAULT_TABLE table=test-workload [id=933040d0bb8b433c8bdb8b124e79c854]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:25:25.571555 23816 tablet_service.cc:1511] Processing CreateTablet for tablet ee29c9afcbb94cadae2778d1f0d2fcd2 (DEFAULT_TABLE table=test-workload [id=933040d0bb8b433c8bdb8b124e79c854]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:25:25.573644 23815 tablet_service.cc:1511] Processing CreateTablet for tablet e5a6d6eb60bb40cd83ba7787fc127421 (DEFAULT_TABLE table=test-workload [id=933040d0bb8b433c8bdb8b124e79c854]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:25:25.574604 23814 tablet_service.cc:1511] Processing CreateTablet for tablet 09fbbea470544d64b22e97e7f74a305b (DEFAULT_TABLE table=test-workload [id=933040d0bb8b433c8bdb8b124e79c854]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:25:25.580232 23817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8085f6a4e929411097ff3c58f115ed80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:25.585165 23813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bcd8bf7d5d04504b30ddbb04f6bc7fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:25.587255 23811 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fedce3e0a6d41aa8954693aab3796cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:25.588716 23815 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5a6d6eb60bb40cd83ba7787fc127421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:25.594806 23737 tablet_service.cc:1511] Processing CreateTablet for tablet 5bcd8bf7d5d04504b30ddbb04f6bc7fd (DEFAULT_TABLE table=test-workload [id=933040d0bb8b433c8bdb8b124e79c854]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:25:25.598311 23816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee29c9afcbb94cadae2778d1f0d2fcd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:25.600814 23814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09fbbea470544d64b22e97e7f74a305b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:25.600391 23891 tablet_service.cc:1511] Processing CreateTablet for tablet 8085f6a4e929411097ff3c58f115ed80 (DEFAULT_TABLE table=test-workload [id=933040d0bb8b433c8bdb8b124e79c854]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:25:25.602108 23891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8085f6a4e929411097ff3c58f115ed80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:25.600445 23737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bcd8bf7d5d04504b30ddbb04f6bc7fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:25.614156 23885 tablet_service.cc:1511] Processing CreateTablet for tablet 2fedce3e0a6d41aa8954693aab3796cd (DEFAULT_TABLE table=test-workload [id=933040d0bb8b433c8bdb8b124e79c854]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:25:25.615849 23885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fedce3e0a6d41aa8954693aab3796cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:25.617304 23736 tablet_service.cc:1511] Processing CreateTablet for tablet 7101b7cd32744ca79f6c50b5e31f246a (DEFAULT_TABLE table=test-workload [id=933040d0bb8b433c8bdb8b124e79c854]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:25:25.618769 23736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7101b7cd32744ca79f6c50b5e31f246a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:25.606619 23890 tablet_service.cc:1511] Processing CreateTablet for tablet ee29c9afcbb94cadae2778d1f0d2fcd2 (DEFAULT_TABLE table=test-workload [id=933040d0bb8b433c8bdb8b124e79c854]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:25:25.622222 23890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee29c9afcbb94cadae2778d1f0d2fcd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:25.608083 23887 tablet_service.cc:1511] Processing CreateTablet for tablet 5bcd8bf7d5d04504b30ddbb04f6bc7fd (DEFAULT_TABLE table=test-workload [id=933040d0bb8b433c8bdb8b124e79c854]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:25:25.624464 23887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bcd8bf7d5d04504b30ddbb04f6bc7fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:25.608289 23812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7101b7cd32744ca79f6c50b5e31f246a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:25.633487 23735 tablet_service.cc:1511] Processing CreateTablet for tablet 2fedce3e0a6d41aa8954693aab3796cd (DEFAULT_TABLE table=test-workload [id=933040d0bb8b433c8bdb8b124e79c854]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:25:25.634997 23735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fedce3e0a6d41aa8954693aab3796cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:25.612154 23886 tablet_service.cc:1511] Processing CreateTablet for tablet 7101b7cd32744ca79f6c50b5e31f246a (DEFAULT_TABLE table=test-workload [id=933040d0bb8b433c8bdb8b124e79c854]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:25:25.641572 23886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7101b7cd32744ca79f6c50b5e31f246a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:25.609474 23888 tablet_service.cc:1511] Processing CreateTablet for tablet 09fbbea470544d64b22e97e7f74a305b (DEFAULT_TABLE table=test-workload [id=933040d0bb8b433c8bdb8b124e79c854]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:25:25.645639 23888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09fbbea470544d64b22e97e7f74a305b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:25.610737 23889 tablet_service.cc:1511] Processing CreateTablet for tablet e5a6d6eb60bb40cd83ba7787fc127421 (DEFAULT_TABLE table=test-workload [id=933040d0bb8b433c8bdb8b124e79c854]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:25:25.649689 23889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5a6d6eb60bb40cd83ba7787fc127421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:25.705662 23945 tablet_bootstrap.cc:492] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f: Bootstrap starting.
I20260709 12:25:25.749253 23945 tablet_bootstrap.cc:654] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:25.767659 23944 tablet_bootstrap.cc:492] T 3ae7d028bd6548359867825588960bcb P 50493539c1954244889d4f3e6d55f396: Bootstrap starting.
I20260709 12:25:25.776770 23945 tablet_bootstrap.cc:492] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f: No bootstrap required, opened a new log
I20260709 12:25:25.778813 23945 ts_tablet_manager.cc:1403] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f: Time spent bootstrapping tablet: real 0.074s	user 0.014s	sys 0.006s
I20260709 12:25:25.782500 23944 tablet_bootstrap.cc:654] T 3ae7d028bd6548359867825588960bcb P 50493539c1954244889d4f3e6d55f396: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:25.782596 23945 raft_consensus.cc:359] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:25.785135 23946 tablet_bootstrap.cc:492] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1: Bootstrap starting.
I20260709 12:25:25.784296 23945 raft_consensus.cc:385] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:25.789357 23945 raft_consensus.cc:740] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 565673cccffc4dbbb2d7431552ba280f, State: Initialized, Role: FOLLOWER
I20260709 12:25:25.790180 23945 consensus_queue.cc:260] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:25.813418 23945 ts_tablet_manager.cc:1434] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.015s
I20260709 12:25:25.814432 23929 heartbeater.cc:499] Master 127.5.58.62:43883 was elected leader, sending a full tablet report...
I20260709 12:25:25.814936 23945 tablet_bootstrap.cc:492] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f: Bootstrap starting.
I20260709 12:25:25.822891 23944 tablet_bootstrap.cc:492] T 3ae7d028bd6548359867825588960bcb P 50493539c1954244889d4f3e6d55f396: No bootstrap required, opened a new log
I20260709 12:25:25.823585 23944 ts_tablet_manager.cc:1403] T 3ae7d028bd6548359867825588960bcb P 50493539c1954244889d4f3e6d55f396: Time spent bootstrapping tablet: real 0.070s	user 0.015s	sys 0.021s
I20260709 12:25:25.824080 23945 tablet_bootstrap.cc:654] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:25.826974 23944 raft_consensus.cc:359] T 3ae7d028bd6548359867825588960bcb P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:25.828104 23944 raft_consensus.cc:385] T 3ae7d028bd6548359867825588960bcb P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:25.828568 23944 raft_consensus.cc:740] T 3ae7d028bd6548359867825588960bcb P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50493539c1954244889d4f3e6d55f396, State: Initialized, Role: FOLLOWER
I20260709 12:25:25.830585 23945 tablet_bootstrap.cc:492] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f: No bootstrap required, opened a new log
I20260709 12:25:25.829753 23944 consensus_queue.cc:260] T 3ae7d028bd6548359867825588960bcb P 50493539c1954244889d4f3e6d55f396 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:25.831176 23946 tablet_bootstrap.cc:654] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:25.833262 23855 heartbeater.cc:499] Master 127.5.58.62:43883 was elected leader, sending a full tablet report...
I20260709 12:25:25.833887 23944 ts_tablet_manager.cc:1434] T 3ae7d028bd6548359867825588960bcb P 50493539c1954244889d4f3e6d55f396: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.006s
I20260709 12:25:25.831215 23945 ts_tablet_manager.cc:1403] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f: Time spent bootstrapping tablet: real 0.017s	user 0.008s	sys 0.004s
I20260709 12:25:25.835094 23944 tablet_bootstrap.cc:492] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396: Bootstrap starting.
I20260709 12:25:25.836796 23945 raft_consensus.cc:359] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:25.837576 23945 raft_consensus.cc:385] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:25.837870 23945 raft_consensus.cc:740] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 565673cccffc4dbbb2d7431552ba280f, State: Initialized, Role: FOLLOWER
I20260709 12:25:25.838716 23946 tablet_bootstrap.cc:492] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1: No bootstrap required, opened a new log
I20260709 12:25:25.839366 23946 ts_tablet_manager.cc:1403] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1: Time spent bootstrapping tablet: real 0.055s	user 0.010s	sys 0.005s
I20260709 12:25:25.838788 23945 consensus_queue.cc:260] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:25.841208 23945 ts_tablet_manager.cc:1434] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 12:25:25.842212 23945 tablet_bootstrap.cc:492] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f: Bootstrap starting.
I20260709 12:25:25.842455 23944 tablet_bootstrap.cc:654] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396: Neither blocks nor log segments found. Creating new log.
W20260709 12:25:25.843698 23930 tablet.cc:2406] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:25:25.842509 23946 raft_consensus.cc:359] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:25.847596 23946 raft_consensus.cc:385] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:25.847997 23946 raft_consensus.cc:740] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc755c49216847f4a9ead01695cd2ab1, State: Initialized, Role: FOLLOWER
I20260709 12:25:25.849001 23946 consensus_queue.cc:260] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:25.851089 23945 tablet_bootstrap.cc:654] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:25.873606 23949 raft_consensus.cc:493] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.874457 23949 raft_consensus.cc:515] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:25.875629 23780 heartbeater.cc:499] Master 127.5.58.62:43883 was elected leader, sending a full tablet report...
I20260709 12:25:25.879971 23949 leader_election.cc:290] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837)
I20260709 12:25:25.886886 23946 ts_tablet_manager.cc:1434] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1: Time spent starting tablet: real 0.047s	user 0.015s	sys 0.019s
I20260709 12:25:25.888613 23946 tablet_bootstrap.cc:492] T 3ae7d028bd6548359867825588960bcb P cc755c49216847f4a9ead01695cd2ab1: Bootstrap starting.
I20260709 12:25:25.898615 23945 tablet_bootstrap.cc:492] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f: No bootstrap required, opened a new log
I20260709 12:25:25.898736 23944 tablet_bootstrap.cc:492] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396: No bootstrap required, opened a new log
I20260709 12:25:25.904844 23944 ts_tablet_manager.cc:1403] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396: Time spent bootstrapping tablet: real 0.070s	user 0.020s	sys 0.017s
I20260709 12:25:25.905324 23954 raft_consensus.cc:493] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.905972 23954 raft_consensus.cc:515] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:25.901279 23946 tablet_bootstrap.cc:654] T 3ae7d028bd6548359867825588960bcb P cc755c49216847f4a9ead01695cd2ab1: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:25.904836 23945 ts_tablet_manager.cc:1403] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f: Time spent bootstrapping tablet: real 0.063s	user 0.029s	sys 0.026s
I20260709 12:25:25.909158 23944 raft_consensus.cc:359] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:25.910099 23944 raft_consensus.cc:385] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:25.910418 23944 raft_consensus.cc:740] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50493539c1954244889d4f3e6d55f396, State: Initialized, Role: FOLLOWER
I20260709 12:25:25.911317 23944 consensus_queue.cc:260] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:25.912762 23945 raft_consensus.cc:359] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:25.913511 23945 raft_consensus.cc:385] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:25.913811 23945 raft_consensus.cc:740] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 565673cccffc4dbbb2d7431552ba280f, State: Initialized, Role: FOLLOWER
I20260709 12:25:25.914713 23945 consensus_queue.cc:260] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:25.917277 23945 ts_tablet_manager.cc:1434] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f: Time spent starting tablet: real 0.008s	user 0.001s	sys 0.005s
I20260709 12:25:25.918303 23945 tablet_bootstrap.cc:492] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f: Bootstrap starting.
I20260709 12:25:25.920166 23830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ae7d028bd6548359867825588960bcb" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50493539c1954244889d4f3e6d55f396" is_pre_election: true
I20260709 12:25:25.921064 23830 raft_consensus.cc:2468] T 3ae7d028bd6548359867825588960bcb P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 565673cccffc4dbbb2d7431552ba280f in term 0.
I20260709 12:25:25.922695 23869 leader_election.cc:304] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50493539c1954244889d4f3e6d55f396, 565673cccffc4dbbb2d7431552ba280f; no voters: 
I20260709 12:25:25.924444 23949 raft_consensus.cc:2804] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:25:25.924817 23949 raft_consensus.cc:493] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.925079 23945 tablet_bootstrap.cc:654] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:25.925333 23949 raft_consensus.cc:3060] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:25.927636 23954 leader_election.cc:290] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837), 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373)
I20260709 12:25:25.936264 23949 raft_consensus.cc:515] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
W20260709 12:25:25.941022 23700 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:25:25.917284 23944 ts_tablet_manager.cc:1434] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 12:25:25.957581 23944 tablet_bootstrap.cc:492] T e5a6d6eb60bb40cd83ba7787fc127421 P 50493539c1954244889d4f3e6d55f396: Bootstrap starting.
I20260709 12:25:25.966643 23754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ae7d028bd6548359867825588960bcb" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc755c49216847f4a9ead01695cd2ab1"
I20260709 12:25:25.966982 23755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ae7d028bd6548359867825588960bcb" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc755c49216847f4a9ead01695cd2ab1" is_pre_election: true
W20260709 12:25:25.969154 23870 leader_election.cc:343] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:25.969285 23962 raft_consensus.cc:493] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.969777 23830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ae7d028bd6548359867825588960bcb" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50493539c1954244889d4f3e6d55f396"
W20260709 12:25:25.969964 23870 leader_election.cc:343] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:25.970471 23830 raft_consensus.cc:3060] T 3ae7d028bd6548359867825588960bcb P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:25.969909 23962 raft_consensus.cc:515] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:25.974059 23946 tablet_bootstrap.cc:492] T 3ae7d028bd6548359867825588960bcb P cc755c49216847f4a9ead01695cd2ab1: No bootstrap required, opened a new log
I20260709 12:25:25.974773 23946 ts_tablet_manager.cc:1403] T 3ae7d028bd6548359867825588960bcb P cc755c49216847f4a9ead01695cd2ab1: Time spent bootstrapping tablet: real 0.087s	user 0.039s	sys 0.020s
I20260709 12:25:25.974905 23962 leader_election.cc:290] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837)
I20260709 12:25:25.976895 23755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fedce3e0a6d41aa8954693aab3796cd" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc755c49216847f4a9ead01695cd2ab1" is_pre_election: true
W20260709 12:25:25.978890 23870 leader_election.cc:343] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:25.979091 23946 raft_consensus.cc:359] T 3ae7d028bd6548359867825588960bcb P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:25.980203 23946 raft_consensus.cc:385] T 3ae7d028bd6548359867825588960bcb P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:25.980726 23946 raft_consensus.cc:740] T 3ae7d028bd6548359867825588960bcb P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc755c49216847f4a9ead01695cd2ab1, State: Initialized, Role: FOLLOWER
I20260709 12:25:25.981209 23829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fedce3e0a6d41aa8954693aab3796cd" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50493539c1954244889d4f3e6d55f396" is_pre_election: true
I20260709 12:25:25.981868 23946 consensus_queue.cc:260] T 3ae7d028bd6548359867825588960bcb P cc755c49216847f4a9ead01695cd2ab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
W20260709 12:25:25.982969 23869 leader_election.cc:343] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:25.983543 23830 raft_consensus.cc:2468] T 3ae7d028bd6548359867825588960bcb P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 565673cccffc4dbbb2d7431552ba280f in term 1.
I20260709 12:25:25.983693 23869 leader_election.cc:304] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 565673cccffc4dbbb2d7431552ba280f; no voters: 50493539c1954244889d4f3e6d55f396, cc755c49216847f4a9ead01695cd2ab1
I20260709 12:25:25.985373 23869 leader_election.cc:304] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50493539c1954244889d4f3e6d55f396, 565673cccffc4dbbb2d7431552ba280f; no voters: cc755c49216847f4a9ead01695cd2ab1
I20260709 12:25:25.985498 23946 ts_tablet_manager.cc:1434] T 3ae7d028bd6548359867825588960bcb P cc755c49216847f4a9ead01695cd2ab1: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:25:25.985811 23962 raft_consensus.cc:2749] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:25:25.986939 23946 tablet_bootstrap.cc:492] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1: Bootstrap starting.
I20260709 12:25:25.987021 23962 raft_consensus.cc:2804] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:25.989477 23949 leader_election.cc:290] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 election: Requested vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837)
I20260709 12:25:25.989857 23945 tablet_bootstrap.cc:492] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f: No bootstrap required, opened a new log
I20260709 12:25:25.990327 23949 raft_consensus.cc:493] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.991060 23949 raft_consensus.cc:515] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:25.999960 23946 tablet_bootstrap.cc:654] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:25.990520 23945 ts_tablet_manager.cc:1403] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f: Time spent bootstrapping tablet: real 0.072s	user 0.019s	sys 0.025s
I20260709 12:25:26.006611 23962 raft_consensus.cc:697] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f [term 1 LEADER]: Becoming Leader. State: Replica: 565673cccffc4dbbb2d7431552ba280f, State: Running, Role: LEADER
I20260709 12:25:26.008620 23962 consensus_queue.cc:237] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.015502 23946 tablet_bootstrap.cc:492] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1: No bootstrap required, opened a new log
I20260709 12:25:26.016541 23946 ts_tablet_manager.cc:1403] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1: Time spent bootstrapping tablet: real 0.030s	user 0.023s	sys 0.004s
I20260709 12:25:26.018327 23949 leader_election.cc:290] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837)
I20260709 12:25:26.019241 23755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63fdfbf5f20f4c1ea9db0d4707c4861b" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc755c49216847f4a9ead01695cd2ab1" is_pre_election: true
I20260709 12:25:26.020780 23944 tablet_bootstrap.cc:654] T e5a6d6eb60bb40cd83ba7787fc127421 P 50493539c1954244889d4f3e6d55f396: Neither blocks nor log segments found. Creating new log.
W20260709 12:25:26.020987 23870 leader_election.cc:343] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:26.021793 23946 raft_consensus.cc:359] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.023196 23946 raft_consensus.cc:385] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:26.024099 23946 raft_consensus.cc:740] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc755c49216847f4a9ead01695cd2ab1, State: Initialized, Role: FOLLOWER
I20260709 12:25:26.025280 23946 consensus_queue.cc:260] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.027822 23946 ts_tablet_manager.cc:1434] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:25:26.029327 23945 raft_consensus.cc:359] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.030141 23830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "797d4300bcc7475ba608835b5d9addc2" candidate_uuid: "cc755c49216847f4a9ead01695cd2ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50493539c1954244889d4f3e6d55f396" is_pre_election: true
I20260709 12:25:26.030252 23945 raft_consensus.cc:385] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:26.031122 23945 raft_consensus.cc:740] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 565673cccffc4dbbb2d7431552ba280f, State: Initialized, Role: FOLLOWER
I20260709 12:25:26.031201 23830 raft_consensus.cc:2468] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc755c49216847f4a9ead01695cd2ab1 in term 0.
I20260709 12:25:26.032006 23945 consensus_queue.cc:260] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.033143 23719 leader_election.cc:304] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50493539c1954244889d4f3e6d55f396, cc755c49216847f4a9ead01695cd2ab1; no voters: 
I20260709 12:25:26.029011 23946 tablet_bootstrap.cc:492] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1: Bootstrap starting.
I20260709 12:25:26.034960 23954 raft_consensus.cc:2804] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:25:26.035326 23954 raft_consensus.cc:493] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.035732 23954 raft_consensus.cc:3060] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:25:26.044751 23869 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:38837, user_credentials={real_user=slave}} blocked reactor thread for 59469.2us
I20260709 12:25:26.048386 23830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63fdfbf5f20f4c1ea9db0d4707c4861b" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50493539c1954244889d4f3e6d55f396" is_pre_election: true
W20260709 12:25:26.050906 23869 leader_election.cc:343] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:26.051441 23869 leader_election.cc:304] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 565673cccffc4dbbb2d7431552ba280f; no voters: 50493539c1954244889d4f3e6d55f396, cc755c49216847f4a9ead01695cd2ab1
I20260709 12:25:26.054112 23962 raft_consensus.cc:2749] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:25:26.057022 23946 tablet_bootstrap.cc:654] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:26.057447 23954 raft_consensus.cc:515] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.058291 23944 tablet_bootstrap.cc:492] T e5a6d6eb60bb40cd83ba7787fc127421 P 50493539c1954244889d4f3e6d55f396: No bootstrap required, opened a new log
I20260709 12:25:26.058815 23944 ts_tablet_manager.cc:1403] T e5a6d6eb60bb40cd83ba7787fc127421 P 50493539c1954244889d4f3e6d55f396: Time spent bootstrapping tablet: real 0.102s	user 0.012s	sys 0.018s
I20260709 12:25:26.053012 23945 ts_tablet_manager.cc:1434] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f: Time spent starting tablet: real 0.048s	user 0.009s	sys 0.000s
I20260709 12:25:26.062384 23830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "797d4300bcc7475ba608835b5d9addc2" candidate_uuid: "cc755c49216847f4a9ead01695cd2ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50493539c1954244889d4f3e6d55f396"
I20260709 12:25:26.062614 23904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "797d4300bcc7475ba608835b5d9addc2" candidate_uuid: "cc755c49216847f4a9ead01695cd2ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "565673cccffc4dbbb2d7431552ba280f" is_pre_election: true
I20260709 12:25:26.062803 23945 tablet_bootstrap.cc:492] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f: Bootstrap starting.
I20260709 12:25:26.063197 23830 raft_consensus.cc:3060] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:26.065029 23969 raft_consensus.cc:493] T 3ae7d028bd6548359867825588960bcb P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:26.065387 23719 leader_election.cc:343] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:26.065856 23969 raft_consensus.cc:515] T 3ae7d028bd6548359867825588960bcb P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.065917 23650 catalog_manager.cc:5697] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f reported cstate change: term changed from 0 to 1, leader changed from <none> to 565673cccffc4dbbb2d7431552ba280f (127.5.58.3). New cstate: current_term: 1 leader_uuid: "565673cccffc4dbbb2d7431552ba280f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:26.069321 23969 leader_election.cc:290] T 3ae7d028bd6548359867825588960bcb P cc755c49216847f4a9ead01695cd2ab1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837), 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373)
I20260709 12:25:26.073108 23829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ae7d028bd6548359867825588960bcb" candidate_uuid: "cc755c49216847f4a9ead01695cd2ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50493539c1954244889d4f3e6d55f396" is_pre_election: true
I20260709 12:25:26.074550 23829 raft_consensus.cc:2393] T 3ae7d028bd6548359867825588960bcb P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cc755c49216847f4a9ead01695cd2ab1 in current term 1: Already voted for candidate 565673cccffc4dbbb2d7431552ba280f in this term.
I20260709 12:25:26.076184 23954 leader_election.cc:290] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [CANDIDATE]: Term 1 election: Requested vote from peers 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837), 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373)
I20260709 12:25:26.078374 23944 raft_consensus.cc:359] T e5a6d6eb60bb40cd83ba7787fc127421 P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.079367 23944 raft_consensus.cc:385] T e5a6d6eb60bb40cd83ba7787fc127421 P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:26.079139 23903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "797d4300bcc7475ba608835b5d9addc2" candidate_uuid: "cc755c49216847f4a9ead01695cd2ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "565673cccffc4dbbb2d7431552ba280f"
I20260709 12:25:26.079771 23945 tablet_bootstrap.cc:654] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:26.079859 23944 raft_consensus.cc:740] T e5a6d6eb60bb40cd83ba7787fc127421 P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50493539c1954244889d4f3e6d55f396, State: Initialized, Role: FOLLOWER
I20260709 12:25:26.080940 23944 consensus_queue.cc:260] T e5a6d6eb60bb40cd83ba7787fc127421 P 50493539c1954244889d4f3e6d55f396 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.078253 23904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ae7d028bd6548359867825588960bcb" candidate_uuid: "cc755c49216847f4a9ead01695cd2ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "565673cccffc4dbbb2d7431552ba280f" is_pre_election: true
W20260709 12:25:26.081404 23719 leader_election.cc:343] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:26.083578 23944 ts_tablet_manager.cc:1434] T e5a6d6eb60bb40cd83ba7787fc127421 P 50493539c1954244889d4f3e6d55f396: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 12:25:26.082942 23830 raft_consensus.cc:2468] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc755c49216847f4a9ead01695cd2ab1 in term 1.
I20260709 12:25:26.084067 23719 leader_election.cc:304] T 3ae7d028bd6548359867825588960bcb P cc755c49216847f4a9ead01695cd2ab1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc755c49216847f4a9ead01695cd2ab1; no voters: 50493539c1954244889d4f3e6d55f396, 565673cccffc4dbbb2d7431552ba280f
I20260709 12:25:26.084854 23944 tablet_bootstrap.cc:492] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396: Bootstrap starting.
I20260709 12:25:26.085202 23954 raft_consensus.cc:3060] T 3ae7d028bd6548359867825588960bcb P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:26.085815 23719 leader_election.cc:304] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50493539c1954244889d4f3e6d55f396, cc755c49216847f4a9ead01695cd2ab1; no voters: 565673cccffc4dbbb2d7431552ba280f
I20260709 12:25:26.089166 23946 tablet_bootstrap.cc:492] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1: No bootstrap required, opened a new log
I20260709 12:25:26.089771 23946 ts_tablet_manager.cc:1403] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1: Time spent bootstrapping tablet: real 0.061s	user 0.023s	sys 0.006s
I20260709 12:25:26.092813 23944 tablet_bootstrap.cc:654] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:26.093056 23946 raft_consensus.cc:359] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.093809 23954 raft_consensus.cc:2749] T 3ae7d028bd6548359867825588960bcb P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:25:26.094614 23954 raft_consensus.cc:2804] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:26.094996 23954 raft_consensus.cc:697] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [term 1 LEADER]: Becoming Leader. State: Replica: cc755c49216847f4a9ead01695cd2ab1, State: Running, Role: LEADER
I20260709 12:25:26.095633 23954 consensus_queue.cc:237] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.093840 23946 raft_consensus.cc:385] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:26.096796 23946 raft_consensus.cc:740] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc755c49216847f4a9ead01695cd2ab1, State: Initialized, Role: FOLLOWER
I20260709 12:25:26.097611 23946 consensus_queue.cc:260] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.099246 23944 tablet_bootstrap.cc:492] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396: No bootstrap required, opened a new log
I20260709 12:25:26.099808 23944 ts_tablet_manager.cc:1403] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396: Time spent bootstrapping tablet: real 0.015s	user 0.005s	sys 0.009s
I20260709 12:25:26.101413 23946 ts_tablet_manager.cc:1434] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.004s
I20260709 12:25:26.102057 23945 tablet_bootstrap.cc:492] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f: No bootstrap required, opened a new log
I20260709 12:25:26.102587 23946 tablet_bootstrap.cc:492] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1: Bootstrap starting.
I20260709 12:25:26.102591 23945 ts_tablet_manager.cc:1403] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f: Time spent bootstrapping tablet: real 0.040s	user 0.016s	sys 0.000s
I20260709 12:25:26.103170 23944 raft_consensus.cc:359] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.103976 23944 raft_consensus.cc:385] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:26.104353 23944 raft_consensus.cc:740] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50493539c1954244889d4f3e6d55f396, State: Initialized, Role: FOLLOWER
I20260709 12:25:26.105248 23944 consensus_queue.cc:260] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.106289 23945 raft_consensus.cc:359] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.107120 23945 raft_consensus.cc:385] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:26.107429 23945 raft_consensus.cc:740] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 565673cccffc4dbbb2d7431552ba280f, State: Initialized, Role: FOLLOWER
I20260709 12:25:26.107824 23944 ts_tablet_manager.cc:1434] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:25:26.108284 23945 consensus_queue.cc:260] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.108948 23944 tablet_bootstrap.cc:492] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396: Bootstrap starting.
I20260709 12:25:26.110945 23945 ts_tablet_manager.cc:1434] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:25:26.111405 23946 tablet_bootstrap.cc:654] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:26.112097 23945 tablet_bootstrap.cc:492] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f: Bootstrap starting.
I20260709 12:25:26.117033 23944 tablet_bootstrap.cc:654] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:26.127378 23945 tablet_bootstrap.cc:654] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:26.140272 23651 catalog_manager.cc:5697] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "cc755c49216847f4a9ead01695cd2ab1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:26.152308 23944 tablet_bootstrap.cc:492] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396: No bootstrap required, opened a new log
I20260709 12:25:26.153021 23944 ts_tablet_manager.cc:1403] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396: Time spent bootstrapping tablet: real 0.044s	user 0.014s	sys 0.024s
I20260709 12:25:26.153728 23954 raft_consensus.cc:493] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.154323 23954 raft_consensus.cc:515] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.156505 23944 raft_consensus.cc:359] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.156986 23954 leader_election.cc:290] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837), 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373)
I20260709 12:25:26.157735 23830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fedce3e0a6d41aa8954693aab3796cd" candidate_uuid: "cc755c49216847f4a9ead01695cd2ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50493539c1954244889d4f3e6d55f396" is_pre_election: true
I20260709 12:25:26.158372 23946 tablet_bootstrap.cc:492] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1: No bootstrap required, opened a new log
I20260709 12:25:26.158949 23946 ts_tablet_manager.cc:1403] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1: Time spent bootstrapping tablet: real 0.057s	user 0.018s	sys 0.027s
W20260709 12:25:26.159575 23719 leader_election.cc:343] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:26.159950 23945 tablet_bootstrap.cc:492] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f: No bootstrap required, opened a new log
I20260709 12:25:26.160563 23945 ts_tablet_manager.cc:1403] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f: Time spent bootstrapping tablet: real 0.049s	user 0.016s	sys 0.000s
I20260709 12:25:26.160889 23904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fedce3e0a6d41aa8954693aab3796cd" candidate_uuid: "cc755c49216847f4a9ead01695cd2ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "565673cccffc4dbbb2d7431552ba280f" is_pre_election: true
I20260709 12:25:26.161574 23904 raft_consensus.cc:2468] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc755c49216847f4a9ead01695cd2ab1 in term 0.
I20260709 12:25:26.162393 23946 raft_consensus.cc:359] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.162796 23719 leader_election.cc:304] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 565673cccffc4dbbb2d7431552ba280f, cc755c49216847f4a9ead01695cd2ab1; no voters: 50493539c1954244889d4f3e6d55f396
I20260709 12:25:26.163614 23944 raft_consensus.cc:385] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:26.164004 23944 raft_consensus.cc:740] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50493539c1954244889d4f3e6d55f396, State: Initialized, Role: FOLLOWER
I20260709 12:25:26.164189 23954 raft_consensus.cc:2804] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:25:26.163244 23946 raft_consensus.cc:385] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:26.164650 23954 raft_consensus.cc:493] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.165136 23946 raft_consensus.cc:740] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc755c49216847f4a9ead01695cd2ab1, State: Initialized, Role: FOLLOWER
I20260709 12:25:26.165087 23944 consensus_queue.cc:260] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.163879 23945 raft_consensus.cc:359] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.171372 23945 raft_consensus.cc:385] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:26.171765 23945 raft_consensus.cc:740] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 565673cccffc4dbbb2d7431552ba280f, State: Initialized, Role: FOLLOWER
I20260709 12:25:26.165189 23954 raft_consensus.cc:3060] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:26.173573 23944 ts_tablet_manager.cc:1434] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396: Time spent starting tablet: real 0.020s	user 0.010s	sys 0.003s
I20260709 12:25:26.173827 23946 consensus_queue.cc:260] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.174782 23944 tablet_bootstrap.cc:492] T 8085f6a4e929411097ff3c58f115ed80 P 50493539c1954244889d4f3e6d55f396: Bootstrap starting.
I20260709 12:25:26.176828 23946 ts_tablet_manager.cc:1434] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.006s
I20260709 12:25:26.178352 23951 raft_consensus.cc:493] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.178859 23951 raft_consensus.cc:515] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.180835 23954 raft_consensus.cc:515] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.182822 23944 tablet_bootstrap.cc:654] T 8085f6a4e929411097ff3c58f115ed80 P 50493539c1954244889d4f3e6d55f396: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:26.183252 23951 leader_election.cc:290] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373)
I20260709 12:25:26.184113 23830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fedce3e0a6d41aa8954693aab3796cd" candidate_uuid: "cc755c49216847f4a9ead01695cd2ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50493539c1954244889d4f3e6d55f396"
W20260709 12:25:26.185896 23719 leader_election.cc:343] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:26.186945 23945 consensus_queue.cc:260] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.195353 23946 tablet_bootstrap.cc:492] T 7101b7cd32744ca79f6c50b5e31f246a P cc755c49216847f4a9ead01695cd2ab1: Bootstrap starting.
I20260709 12:25:26.197985 23945 ts_tablet_manager.cc:1434] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.000s
I20260709 12:25:26.199581 23945 tablet_bootstrap.cc:492] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f: Bootstrap starting.
I20260709 12:25:26.207556 23904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fedce3e0a6d41aa8954693aab3796cd" candidate_uuid: "cc755c49216847f4a9ead01695cd2ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "565673cccffc4dbbb2d7431552ba280f"
I20260709 12:25:26.208469 23904 raft_consensus.cc:3060] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:26.216473 23954 leader_election.cc:290] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [CANDIDATE]: Term 1 election: Requested vote from peers 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837), 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373)
I20260709 12:25:26.218791 23964 raft_consensus.cc:493] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.219811 23962 raft_consensus.cc:493] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.219537 23964 raft_consensus.cc:515] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.220300 23962 raft_consensus.cc:515] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.222430 23962 leader_election.cc:290] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837)
I20260709 12:25:26.223225 23946 tablet_bootstrap.cc:654] T 7101b7cd32744ca79f6c50b5e31f246a P cc755c49216847f4a9ead01695cd2ab1: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:26.224313 23755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bcd8bf7d5d04504b30ddbb04f6bc7fd" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc755c49216847f4a9ead01695cd2ab1" is_pre_election: true
I20260709 12:25:26.226104 23830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a6d6eb60bb40cd83ba7787fc127421" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50493539c1954244889d4f3e6d55f396" is_pre_election: true
I20260709 12:25:26.227711 23830 raft_consensus.cc:2468] T e5a6d6eb60bb40cd83ba7787fc127421 P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 565673cccffc4dbbb2d7431552ba280f in term 0.
W20260709 12:25:26.228214 23870 leader_election.cc:343] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:26.228650 23964 leader_election.cc:290] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837)
I20260709 12:25:26.228981 23754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a6d6eb60bb40cd83ba7787fc127421" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc755c49216847f4a9ead01695cd2ab1" is_pre_election: true
I20260709 12:25:26.229537 23755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09fbbea470544d64b22e97e7f74a305b" candidate_uuid: "50493539c1954244889d4f3e6d55f396" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc755c49216847f4a9ead01695cd2ab1" is_pre_election: true
I20260709 12:25:26.231369 23869 leader_election.cc:304] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50493539c1954244889d4f3e6d55f396, 565673cccffc4dbbb2d7431552ba280f; no voters: 
I20260709 12:25:26.231632 23904 raft_consensus.cc:2468] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc755c49216847f4a9ead01695cd2ab1 in term 1.
I20260709 12:25:26.232246 23830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bcd8bf7d5d04504b30ddbb04f6bc7fd" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50493539c1954244889d4f3e6d55f396" is_pre_election: true
I20260709 12:25:26.233124 23830 raft_consensus.cc:2468] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 565673cccffc4dbbb2d7431552ba280f in term 0.
I20260709 12:25:26.233346 23964 raft_consensus.cc:2804] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:25:26.232952 23796 leader_election.cc:343] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:25:26.233604 23870 leader_election.cc:343] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:26.234126 23964 raft_consensus.cc:493] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.234886 23964 raft_consensus.cc:3060] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:26.235682 23869 leader_election.cc:304] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50493539c1954244889d4f3e6d55f396, 565673cccffc4dbbb2d7431552ba280f; no voters: cc755c49216847f4a9ead01695cd2ab1
I20260709 12:25:26.236682 23962 raft_consensus.cc:2804] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:25:26.237218 23962 raft_consensus.cc:493] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.237634 23962 raft_consensus.cc:3060] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:26.239817 23719 leader_election.cc:304] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 565673cccffc4dbbb2d7431552ba280f, cc755c49216847f4a9ead01695cd2ab1; no voters: 50493539c1954244889d4f3e6d55f396
I20260709 12:25:26.241263 23954 raft_consensus.cc:2804] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:26.242792 23954 raft_consensus.cc:697] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [term 1 LEADER]: Becoming Leader. State: Replica: cc755c49216847f4a9ead01695cd2ab1, State: Running, Role: LEADER
I20260709 12:25:26.234298 23945 tablet_bootstrap.cc:654] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:26.244658 23954 consensus_queue.cc:237] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.251031 23944 tablet_bootstrap.cc:492] T 8085f6a4e929411097ff3c58f115ed80 P 50493539c1954244889d4f3e6d55f396: No bootstrap required, opened a new log
I20260709 12:25:26.251199 23964 raft_consensus.cc:515] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.251989 23944 ts_tablet_manager.cc:1403] T 8085f6a4e929411097ff3c58f115ed80 P 50493539c1954244889d4f3e6d55f396: Time spent bootstrapping tablet: real 0.077s	user 0.038s	sys 0.016s
I20260709 12:25:26.253926 23904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09fbbea470544d64b22e97e7f74a305b" candidate_uuid: "50493539c1954244889d4f3e6d55f396" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "565673cccffc4dbbb2d7431552ba280f" is_pre_election: true
W20260709 12:25:26.256575 23795 leader_election.cc:343] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:26.257313 23795 leader_election.cc:304] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50493539c1954244889d4f3e6d55f396; no voters: 565673cccffc4dbbb2d7431552ba280f, cc755c49216847f4a9ead01695cd2ab1
I20260709 12:25:26.257834 23755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a6d6eb60bb40cd83ba7787fc127421" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc755c49216847f4a9ead01695cd2ab1"
I20260709 12:25:26.259326 23964 leader_election.cc:290] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 election: Requested vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837)
W20260709 12:25:26.259665 23870 leader_election.cc:343] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:26.259392 23951 raft_consensus.cc:2749] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:25:26.260195 23944 raft_consensus.cc:359] T 8085f6a4e929411097ff3c58f115ed80 P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.260777 23830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a6d6eb60bb40cd83ba7787fc127421" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50493539c1954244889d4f3e6d55f396"
I20260709 12:25:26.261066 23944 raft_consensus.cc:385] T 8085f6a4e929411097ff3c58f115ed80 P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:26.261473 23944 raft_consensus.cc:740] T 8085f6a4e929411097ff3c58f115ed80 P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50493539c1954244889d4f3e6d55f396, State: Initialized, Role: FOLLOWER
I20260709 12:25:26.261552 23830 raft_consensus.cc:3060] T e5a6d6eb60bb40cd83ba7787fc127421 P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:26.262303 23944 consensus_queue.cc:260] T 8085f6a4e929411097ff3c58f115ed80 P 50493539c1954244889d4f3e6d55f396 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.266937 23946 tablet_bootstrap.cc:492] T 7101b7cd32744ca79f6c50b5e31f246a P cc755c49216847f4a9ead01695cd2ab1: No bootstrap required, opened a new log
I20260709 12:25:26.267537 23946 ts_tablet_manager.cc:1403] T 7101b7cd32744ca79f6c50b5e31f246a P cc755c49216847f4a9ead01695cd2ab1: Time spent bootstrapping tablet: real 0.090s	user 0.013s	sys 0.017s
I20260709 12:25:26.268581 23962 raft_consensus.cc:515] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.271095 23946 raft_consensus.cc:359] T 7101b7cd32744ca79f6c50b5e31f246a P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.271931 23946 raft_consensus.cc:385] T 7101b7cd32744ca79f6c50b5e31f246a P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:26.272689 23755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bcd8bf7d5d04504b30ddbb04f6bc7fd" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc755c49216847f4a9ead01695cd2ab1"
W20260709 12:25:26.274444 23870 leader_election.cc:343] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:26.272255 23946 raft_consensus.cc:740] T 7101b7cd32744ca79f6c50b5e31f246a P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc755c49216847f4a9ead01695cd2ab1, State: Initialized, Role: FOLLOWER
I20260709 12:25:26.275651 23954 raft_consensus.cc:493] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.277244 23954 raft_consensus.cc:515] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.277444 23946 consensus_queue.cc:260] T 7101b7cd32744ca79f6c50b5e31f246a P cc755c49216847f4a9ead01695cd2ab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.278033 23944 ts_tablet_manager.cc:1434] T 8085f6a4e929411097ff3c58f115ed80 P 50493539c1954244889d4f3e6d55f396: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.002s
I20260709 12:25:26.279402 23944 tablet_bootstrap.cc:492] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396: Bootstrap starting.
I20260709 12:25:26.279903 23946 ts_tablet_manager.cc:1434] T 7101b7cd32744ca79f6c50b5e31f246a P cc755c49216847f4a9ead01695cd2ab1: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:25:26.280637 23829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63fdfbf5f20f4c1ea9db0d4707c4861b" candidate_uuid: "cc755c49216847f4a9ead01695cd2ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50493539c1954244889d4f3e6d55f396" is_pre_election: true
I20260709 12:25:26.281024 23946 tablet_bootstrap.cc:492] T e5a6d6eb60bb40cd83ba7787fc127421 P cc755c49216847f4a9ead01695cd2ab1: Bootstrap starting.
W20260709 12:25:26.282368 23719 leader_election.cc:343] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:26.284864 23904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63fdfbf5f20f4c1ea9db0d4707c4861b" candidate_uuid: "cc755c49216847f4a9ead01695cd2ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "565673cccffc4dbbb2d7431552ba280f" is_pre_election: true
I20260709 12:25:26.285594 23904 raft_consensus.cc:2468] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc755c49216847f4a9ead01695cd2ab1 in term 0.
I20260709 12:25:26.286365 23945 tablet_bootstrap.cc:492] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f: No bootstrap required, opened a new log
I20260709 12:25:26.286962 23945 ts_tablet_manager.cc:1403] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f: Time spent bootstrapping tablet: real 0.088s	user 0.012s	sys 0.011s
I20260709 12:25:26.287084 23719 leader_election.cc:304] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 565673cccffc4dbbb2d7431552ba280f, cc755c49216847f4a9ead01695cd2ab1; no voters: 50493539c1954244889d4f3e6d55f396
I20260709 12:25:26.288139 23969 raft_consensus.cc:2804] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:25:26.288583 23969 raft_consensus.cc:493] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.288857 23946 tablet_bootstrap.cc:654] T e5a6d6eb60bb40cd83ba7787fc127421 P cc755c49216847f4a9ead01695cd2ab1: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:26.289031 23969 raft_consensus.cc:3060] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:26.290259 23945 raft_consensus.cc:359] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.289685 23650 catalog_manager.cc:5697] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "cc755c49216847f4a9ead01695cd2ab1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:26.293582 23962 leader_election.cc:290] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 election: Requested vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837)
I20260709 12:25:26.291064 23945 raft_consensus.cc:385] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:26.295248 23830 raft_consensus.cc:2468] T e5a6d6eb60bb40cd83ba7787fc127421 P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 565673cccffc4dbbb2d7431552ba280f in term 1.
I20260709 12:25:26.295161 23829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bcd8bf7d5d04504b30ddbb04f6bc7fd" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50493539c1954244889d4f3e6d55f396"
I20260709 12:25:26.302582 23945 raft_consensus.cc:740] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 565673cccffc4dbbb2d7431552ba280f, State: Initialized, Role: FOLLOWER
I20260709 12:25:26.302917 23829 raft_consensus.cc:3060] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:26.303481 23869 leader_election.cc:304] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50493539c1954244889d4f3e6d55f396, 565673cccffc4dbbb2d7431552ba280f; no voters: cc755c49216847f4a9ead01695cd2ab1
I20260709 12:25:26.304638 23962 raft_consensus.cc:2804] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:26.298380 23944 tablet_bootstrap.cc:654] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:26.305150 23962 raft_consensus.cc:697] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f [term 1 LEADER]: Becoming Leader. State: Replica: 565673cccffc4dbbb2d7431552ba280f, State: Running, Role: LEADER
I20260709 12:25:26.299136 23954 leader_election.cc:290] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837), 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373)
I20260709 12:25:26.298336 23969 raft_consensus.cc:515] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.306195 23962 consensus_queue.cc:237] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.302338 23946 tablet_bootstrap.cc:492] T e5a6d6eb60bb40cd83ba7787fc127421 P cc755c49216847f4a9ead01695cd2ab1: No bootstrap required, opened a new log
I20260709 12:25:26.309954 23946 ts_tablet_manager.cc:1403] T e5a6d6eb60bb40cd83ba7787fc127421 P cc755c49216847f4a9ead01695cd2ab1: Time spent bootstrapping tablet: real 0.029s	user 0.016s	sys 0.005s
I20260709 12:25:26.311237 23904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63fdfbf5f20f4c1ea9db0d4707c4861b" candidate_uuid: "cc755c49216847f4a9ead01695cd2ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "565673cccffc4dbbb2d7431552ba280f"
I20260709 12:25:26.311959 23904 raft_consensus.cc:3060] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:26.312803 23830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63fdfbf5f20f4c1ea9db0d4707c4861b" candidate_uuid: "cc755c49216847f4a9ead01695cd2ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50493539c1954244889d4f3e6d55f396"
I20260709 12:25:26.313511 23946 raft_consensus.cc:359] T e5a6d6eb60bb40cd83ba7787fc127421 P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.314345 23946 raft_consensus.cc:385] T e5a6d6eb60bb40cd83ba7787fc127421 P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:26.314457 23829 raft_consensus.cc:2468] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 565673cccffc4dbbb2d7431552ba280f in term 1.
I20260709 12:25:26.314846 23946 raft_consensus.cc:740] T e5a6d6eb60bb40cd83ba7787fc127421 P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc755c49216847f4a9ead01695cd2ab1, State: Initialized, Role: FOLLOWER
I20260709 12:25:26.315783 23869 leader_election.cc:304] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50493539c1954244889d4f3e6d55f396, 565673cccffc4dbbb2d7431552ba280f; no voters: cc755c49216847f4a9ead01695cd2ab1
W20260709 12:25:26.314383 23719 leader_election.cc:343] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:26.315826 23946 consensus_queue.cc:260] T e5a6d6eb60bb40cd83ba7787fc127421 P cc755c49216847f4a9ead01695cd2ab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.316799 23964 raft_consensus.cc:2804] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:26.317353 23964 raft_consensus.cc:697] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [term 1 LEADER]: Becoming Leader. State: Replica: 565673cccffc4dbbb2d7431552ba280f, State: Running, Role: LEADER
I20260709 12:25:26.319062 23946 ts_tablet_manager.cc:1434] T e5a6d6eb60bb40cd83ba7787fc127421 P cc755c49216847f4a9ead01695cd2ab1: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.003s
I20260709 12:25:26.318724 23964 consensus_queue.cc:237] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.320802 23946 tablet_bootstrap.cc:492] T 8085f6a4e929411097ff3c58f115ed80 P cc755c49216847f4a9ead01695cd2ab1: Bootstrap starting.
I20260709 12:25:26.322803 23944 tablet_bootstrap.cc:492] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396: No bootstrap required, opened a new log
I20260709 12:25:26.323359 23944 ts_tablet_manager.cc:1403] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396: Time spent bootstrapping tablet: real 0.044s	user 0.011s	sys 0.003s
I20260709 12:25:26.326491 23944 raft_consensus.cc:359] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.327191 23944 raft_consensus.cc:385] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:26.327504 23944 raft_consensus.cc:740] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50493539c1954244889d4f3e6d55f396, State: Initialized, Role: FOLLOWER
I20260709 12:25:26.329051 23946 tablet_bootstrap.cc:654] T 8085f6a4e929411097ff3c58f115ed80 P cc755c49216847f4a9ead01695cd2ab1: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:26.309247 23969 leader_election.cc:290] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1 [CANDIDATE]: Term 1 election: Requested vote from peers 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837), 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373)
I20260709 12:25:26.328626 23944 consensus_queue.cc:260] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.331405 23944 ts_tablet_manager.cc:1434] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 12:25:26.332584 23944 tablet_bootstrap.cc:492] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396: Bootstrap starting.
I20260709 12:25:26.303550 23945 consensus_queue.cc:260] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.334769 23904 raft_consensus.cc:2468] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc755c49216847f4a9ead01695cd2ab1 in term 1.
I20260709 12:25:26.336349 23969 raft_consensus.cc:493] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.337018 23969 raft_consensus.cc:515] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.337481 23719 leader_election.cc:304] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 565673cccffc4dbbb2d7431552ba280f, cc755c49216847f4a9ead01695cd2ab1; no voters: 50493539c1954244889d4f3e6d55f396
I20260709 12:25:26.338362 23954 raft_consensus.cc:2804] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:26.338867 23945 ts_tablet_manager.cc:1434] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f: Time spent starting tablet: real 0.052s	user 0.009s	sys 0.000s
I20260709 12:25:26.339413 23969 leader_election.cc:290] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837), 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373)
I20260709 12:25:26.342162 23829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee29c9afcbb94cadae2778d1f0d2fcd2" candidate_uuid: "cc755c49216847f4a9ead01695cd2ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50493539c1954244889d4f3e6d55f396" is_pre_election: true
I20260709 12:25:26.342504 23903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee29c9afcbb94cadae2778d1f0d2fcd2" candidate_uuid: "cc755c49216847f4a9ead01695cd2ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "565673cccffc4dbbb2d7431552ba280f" is_pre_election: true
I20260709 12:25:26.343178 23829 raft_consensus.cc:2468] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc755c49216847f4a9ead01695cd2ab1 in term 0.
I20260709 12:25:26.344894 23719 leader_election.cc:304] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50493539c1954244889d4f3e6d55f396, cc755c49216847f4a9ead01695cd2ab1; no voters: 
I20260709 12:25:26.345803 23969 raft_consensus.cc:2804] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:25:26.345973 23719 leader_election.cc:343] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:26.346258 23969 raft_consensus.cc:493] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.346673 23969 raft_consensus.cc:3060] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:26.347651 23944 tablet_bootstrap.cc:654] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:26.355409 23954 raft_consensus.cc:697] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1 [term 1 LEADER]: Becoming Leader. State: Replica: cc755c49216847f4a9ead01695cd2ab1, State: Running, Role: LEADER
I20260709 12:25:26.355715 23969 raft_consensus.cc:515] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.356720 23954 consensus_queue.cc:237] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.340186 23945 tablet_bootstrap.cc:492] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f: Bootstrap starting.
I20260709 12:25:26.359334 23829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee29c9afcbb94cadae2778d1f0d2fcd2" candidate_uuid: "cc755c49216847f4a9ead01695cd2ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50493539c1954244889d4f3e6d55f396"
I20260709 12:25:26.359965 23829 raft_consensus.cc:3060] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:26.360836 23904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee29c9afcbb94cadae2778d1f0d2fcd2" candidate_uuid: "cc755c49216847f4a9ead01695cd2ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "565673cccffc4dbbb2d7431552ba280f"
W20260709 12:25:26.362290 23719 leader_election.cc:343] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:26.362488 23944 tablet_bootstrap.cc:492] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396: No bootstrap required, opened a new log
I20260709 12:25:26.363157 23944 ts_tablet_manager.cc:1403] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396: Time spent bootstrapping tablet: real 0.031s	user 0.011s	sys 0.001s
I20260709 12:25:26.363085 23969 leader_election.cc:290] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [CANDIDATE]: Term 1 election: Requested vote from peers 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837), 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373)
I20260709 12:25:26.366381 23944 raft_consensus.cc:359] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.368230 23944 raft_consensus.cc:385] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:26.368549 23944 raft_consensus.cc:740] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50493539c1954244889d4f3e6d55f396, State: Initialized, Role: FOLLOWER
I20260709 12:25:26.368700 23829 raft_consensus.cc:2468] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc755c49216847f4a9ead01695cd2ab1 in term 1.
I20260709 12:25:26.369289 23944 consensus_queue.cc:260] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.369810 23945 tablet_bootstrap.cc:654] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:26.370118 23719 leader_election.cc:304] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50493539c1954244889d4f3e6d55f396, cc755c49216847f4a9ead01695cd2ab1; no voters: 565673cccffc4dbbb2d7431552ba280f
I20260709 12:25:26.371492 23969 raft_consensus.cc:2804] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:26.372208 23969 raft_consensus.cc:697] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [term 1 LEADER]: Becoming Leader. State: Replica: cc755c49216847f4a9ead01695cd2ab1, State: Running, Role: LEADER
I20260709 12:25:26.373155 23969 consensus_queue.cc:237] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.374006 23944 ts_tablet_manager.cc:1434] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:25:26.373481 23651 catalog_manager.cc:5697] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f reported cstate change: term changed from 0 to 1, leader changed from <none> to 565673cccffc4dbbb2d7431552ba280f (127.5.58.3). New cstate: current_term: 1 leader_uuid: "565673cccffc4dbbb2d7431552ba280f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:26.374924 23946 tablet_bootstrap.cc:492] T 8085f6a4e929411097ff3c58f115ed80 P cc755c49216847f4a9ead01695cd2ab1: No bootstrap required, opened a new log
I20260709 12:25:26.375824 23946 ts_tablet_manager.cc:1403] T 8085f6a4e929411097ff3c58f115ed80 P cc755c49216847f4a9ead01695cd2ab1: Time spent bootstrapping tablet: real 0.055s	user 0.016s	sys 0.003s
I20260709 12:25:26.376580 23962 raft_consensus.cc:493] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.377140 23962 raft_consensus.cc:515] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.374993 23944 tablet_bootstrap.cc:492] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396: Bootstrap starting.
I20260709 12:25:26.378986 23946 raft_consensus.cc:359] T 8085f6a4e929411097ff3c58f115ed80 P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.379766 23946 raft_consensus.cc:385] T 8085f6a4e929411097ff3c58f115ed80 P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:26.380064 23946 raft_consensus.cc:740] T 8085f6a4e929411097ff3c58f115ed80 P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc755c49216847f4a9ead01695cd2ab1, State: Initialized, Role: FOLLOWER
I20260709 12:25:26.380959 23946 consensus_queue.cc:260] T 8085f6a4e929411097ff3c58f115ed80 P cc755c49216847f4a9ead01695cd2ab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.381449 23755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8085f6a4e929411097ff3c58f115ed80" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc755c49216847f4a9ead01695cd2ab1" is_pre_election: true
I20260709 12:25:26.382411 23755 raft_consensus.cc:2468] T 8085f6a4e929411097ff3c58f115ed80 P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 565673cccffc4dbbb2d7431552ba280f in term 0.
I20260709 12:25:26.383101 23829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8085f6a4e929411097ff3c58f115ed80" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50493539c1954244889d4f3e6d55f396" is_pre_election: true
I20260709 12:25:26.383368 23946 ts_tablet_manager.cc:1434] T 8085f6a4e929411097ff3c58f115ed80 P cc755c49216847f4a9ead01695cd2ab1: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.007s
I20260709 12:25:26.383870 23870 leader_election.cc:304] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 565673cccffc4dbbb2d7431552ba280f, cc755c49216847f4a9ead01695cd2ab1; no voters: 
I20260709 12:25:26.385116 23946 tablet_bootstrap.cc:492] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1: Bootstrap starting.
I20260709 12:25:26.385303 23964 raft_consensus.cc:2804] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:25:26.385762 23964 raft_consensus.cc:493] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.386123 23964 raft_consensus.cc:3060] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:26.389285 23944 tablet_bootstrap.cc:654] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:26.381456 23962 leader_election.cc:290] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837)
I20260709 12:25:26.383903 23829 raft_consensus.cc:2468] T 8085f6a4e929411097ff3c58f115ed80 P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 565673cccffc4dbbb2d7431552ba280f in term 0.
I20260709 12:25:26.397444 23946 tablet_bootstrap.cc:654] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:26.397303 23964 raft_consensus.cc:515] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.402294 23650 catalog_manager.cc:5697] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "cc755c49216847f4a9ead01695cd2ab1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:26.403743 23755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8085f6a4e929411097ff3c58f115ed80" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc755c49216847f4a9ead01695cd2ab1"
I20260709 12:25:26.404731 23755 raft_consensus.cc:3060] T 8085f6a4e929411097ff3c58f115ed80 P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:26.429308 23650 catalog_manager.cc:5697] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f reported cstate change: term changed from 0 to 1, leader changed from <none> to 565673cccffc4dbbb2d7431552ba280f (127.5.58.3). New cstate: current_term: 1 leader_uuid: "565673cccffc4dbbb2d7431552ba280f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:26.431934 23944 tablet_bootstrap.cc:492] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396: No bootstrap required, opened a new log
I20260709 12:25:26.432516 23945 tablet_bootstrap.cc:492] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f: No bootstrap required, opened a new log
I20260709 12:25:26.432514 23944 ts_tablet_manager.cc:1403] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396: Time spent bootstrapping tablet: real 0.058s	user 0.011s	sys 0.024s
I20260709 12:25:26.433802 23964 leader_election.cc:290] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 election: Requested vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837)
I20260709 12:25:26.433002 23755 raft_consensus.cc:2468] T 8085f6a4e929411097ff3c58f115ed80 P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 565673cccffc4dbbb2d7431552ba280f in term 1.
I20260709 12:25:26.435778 23829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8085f6a4e929411097ff3c58f115ed80" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50493539c1954244889d4f3e6d55f396"
I20260709 12:25:26.436405 23829 raft_consensus.cc:3060] T 8085f6a4e929411097ff3c58f115ed80 P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:26.438202 23946 tablet_bootstrap.cc:492] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1: No bootstrap required, opened a new log
I20260709 12:25:26.438781 23946 ts_tablet_manager.cc:1403] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1: Time spent bootstrapping tablet: real 0.054s	user 0.017s	sys 0.013s
I20260709 12:25:26.439122 23944 raft_consensus.cc:359] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.439869 23944 raft_consensus.cc:385] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:26.440196 23944 raft_consensus.cc:740] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50493539c1954244889d4f3e6d55f396, State: Initialized, Role: FOLLOWER
I20260709 12:25:26.441057 23870 leader_election.cc:304] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 565673cccffc4dbbb2d7431552ba280f, cc755c49216847f4a9ead01695cd2ab1; no voters: 
I20260709 12:25:26.440995 23944 consensus_queue.cc:260] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.441918 23946 raft_consensus.cc:359] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.442804 23946 raft_consensus.cc:385] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:26.443140 23946 raft_consensus.cc:740] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc755c49216847f4a9ead01695cd2ab1, State: Initialized, Role: FOLLOWER
I20260709 12:25:26.442593 23651 catalog_manager.cc:5697] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "cc755c49216847f4a9ead01695cd2ab1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:26.444049 23946 consensus_queue.cc:260] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.446588 23946 ts_tablet_manager.cc:1434] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:25:26.446938 23829 raft_consensus.cc:2468] T 8085f6a4e929411097ff3c58f115ed80 P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 565673cccffc4dbbb2d7431552ba280f in term 1.
I20260709 12:25:26.447667 23946 tablet_bootstrap.cc:492] T 09fbbea470544d64b22e97e7f74a305b P cc755c49216847f4a9ead01695cd2ab1: Bootstrap starting.
I20260709 12:25:26.448860 23964 raft_consensus.cc:2804] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:26.449636 23964 raft_consensus.cc:697] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f [term 1 LEADER]: Becoming Leader. State: Replica: 565673cccffc4dbbb2d7431552ba280f, State: Running, Role: LEADER
I20260709 12:25:26.451093 23944 ts_tablet_manager.cc:1434] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 12:25:26.450655 23964 consensus_queue.cc:237] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.433502 23945 ts_tablet_manager.cc:1403] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f: Time spent bootstrapping tablet: real 0.094s	user 0.011s	sys 0.005s
I20260709 12:25:26.454682 23946 tablet_bootstrap.cc:654] T 09fbbea470544d64b22e97e7f74a305b P cc755c49216847f4a9ead01695cd2ab1: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:26.456187 23969 raft_consensus.cc:493] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.456748 23969 raft_consensus.cc:515] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.459090 23969 leader_election.cc:290] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837), 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373)
I20260709 12:25:26.459201 23945 raft_consensus.cc:359] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.460000 23945 raft_consensus.cc:385] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:26.460278 23945 raft_consensus.cc:740] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 565673cccffc4dbbb2d7431552ba280f, State: Initialized, Role: FOLLOWER
I20260709 12:25:26.461038 23945 consensus_queue.cc:260] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.462134 23944 tablet_bootstrap.cc:492] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396: Bootstrap starting.
I20260709 12:25:26.463335 23904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bcd8bf7d5d04504b30ddbb04f6bc7fd" candidate_uuid: "cc755c49216847f4a9ead01695cd2ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "565673cccffc4dbbb2d7431552ba280f" is_pre_election: true
I20260709 12:25:26.463596 23945 ts_tablet_manager.cc:1434] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 12:25:26.466114 23829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bcd8bf7d5d04504b30ddbb04f6bc7fd" candidate_uuid: "cc755c49216847f4a9ead01695cd2ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50493539c1954244889d4f3e6d55f396" is_pre_election: true
I20260709 12:25:26.467150 23829 raft_consensus.cc:2393] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cc755c49216847f4a9ead01695cd2ab1 in current term 1: Already voted for candidate 565673cccffc4dbbb2d7431552ba280f in this term.
I20260709 12:25:26.468721 23719 leader_election.cc:304] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc755c49216847f4a9ead01695cd2ab1; no voters: 50493539c1954244889d4f3e6d55f396, 565673cccffc4dbbb2d7431552ba280f
I20260709 12:25:26.466173 23945 tablet_bootstrap.cc:492] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f: Bootstrap starting.
I20260709 12:25:26.470161 23969 raft_consensus.cc:3060] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:26.474515 23944 tablet_bootstrap.cc:654] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:26.478545 23962 consensus_queue.cc:1048] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:25:26.480124 23969 raft_consensus.cc:2749] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:25:26.489559 23969 raft_consensus.cc:493] T e5a6d6eb60bb40cd83ba7787fc127421 P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.490211 23969 raft_consensus.cc:515] T e5a6d6eb60bb40cd83ba7787fc127421 P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.497605 23969 leader_election.cc:290] T e5a6d6eb60bb40cd83ba7787fc127421 P cc755c49216847f4a9ead01695cd2ab1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837), 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373)
I20260709 12:25:26.499538 23829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a6d6eb60bb40cd83ba7787fc127421" candidate_uuid: "cc755c49216847f4a9ead01695cd2ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50493539c1954244889d4f3e6d55f396" is_pre_election: true
I20260709 12:25:26.500867 23829 raft_consensus.cc:2393] T e5a6d6eb60bb40cd83ba7787fc127421 P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cc755c49216847f4a9ead01695cd2ab1 in current term 1: Already voted for candidate 565673cccffc4dbbb2d7431552ba280f in this term.
I20260709 12:25:26.502084 23951 raft_consensus.cc:493] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.503046 23951 raft_consensus.cc:515] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.504134 23945 tablet_bootstrap.cc:654] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:26.505532 23904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a6d6eb60bb40cd83ba7787fc127421" candidate_uuid: "cc755c49216847f4a9ead01695cd2ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "565673cccffc4dbbb2d7431552ba280f" is_pre_election: true
I20260709 12:25:26.508951 23719 leader_election.cc:304] T e5a6d6eb60bb40cd83ba7787fc127421 P cc755c49216847f4a9ead01695cd2ab1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc755c49216847f4a9ead01695cd2ab1; no voters: 50493539c1954244889d4f3e6d55f396, 565673cccffc4dbbb2d7431552ba280f
I20260709 12:25:26.510614 23969 raft_consensus.cc:3060] T e5a6d6eb60bb40cd83ba7787fc127421 P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:26.518565 23951 leader_election.cc:290] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373)
I20260709 12:25:26.519393 23755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7101b7cd32744ca79f6c50b5e31f246a" candidate_uuid: "50493539c1954244889d4f3e6d55f396" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc755c49216847f4a9ead01695cd2ab1" is_pre_election: true
I20260709 12:25:26.519908 23904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7101b7cd32744ca79f6c50b5e31f246a" candidate_uuid: "50493539c1954244889d4f3e6d55f396" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "565673cccffc4dbbb2d7431552ba280f" is_pre_election: true
I20260709 12:25:26.520491 23755 raft_consensus.cc:2468] T 7101b7cd32744ca79f6c50b5e31f246a P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50493539c1954244889d4f3e6d55f396 in term 0.
W20260709 12:25:26.521346 23795 leader_election.cc:343] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:26.521991 23796 leader_election.cc:304] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50493539c1954244889d4f3e6d55f396, cc755c49216847f4a9ead01695cd2ab1; no voters: 565673cccffc4dbbb2d7431552ba280f
I20260709 12:25:26.523298 23951 raft_consensus.cc:2804] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:25:26.523712 23951 raft_consensus.cc:493] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.524166 23951 raft_consensus.cc:3060] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:26.524024 23652 catalog_manager.cc:5697] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f reported cstate change: term changed from 0 to 1, leader changed from <none> to 565673cccffc4dbbb2d7431552ba280f (127.5.58.3). New cstate: current_term: 1 leader_uuid: "565673cccffc4dbbb2d7431552ba280f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:26.525938 23964 consensus_queue.cc:1048] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f [LEADER]: Connected to new peer: Peer: permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:25:26.530471 23969 raft_consensus.cc:2749] T e5a6d6eb60bb40cd83ba7787fc127421 P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:25:26.535022 23944 tablet_bootstrap.cc:492] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396: No bootstrap required, opened a new log
I20260709 12:25:26.535679 23944 ts_tablet_manager.cc:1403] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396: Time spent bootstrapping tablet: real 0.074s	user 0.020s	sys 0.037s
I20260709 12:25:26.535785 23951 raft_consensus.cc:515] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.539384 23755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7101b7cd32744ca79f6c50b5e31f246a" candidate_uuid: "50493539c1954244889d4f3e6d55f396" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc755c49216847f4a9ead01695cd2ab1"
I20260709 12:25:26.539266 23944 raft_consensus.cc:359] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.540177 23755 raft_consensus.cc:3060] T 7101b7cd32744ca79f6c50b5e31f246a P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:26.540215 23944 raft_consensus.cc:385] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:26.546870 23944 raft_consensus.cc:740] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50493539c1954244889d4f3e6d55f396, State: Initialized, Role: FOLLOWER
I20260709 12:25:26.546275 23951 leader_election.cc:290] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 1 election: Requested vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373)
I20260709 12:25:26.548219 23944 consensus_queue.cc:260] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.550406 23904 raft_consensus.cc:3060] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:26.550859 23944 ts_tablet_manager.cc:1434] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396: Time spent starting tablet: real 0.015s	user 0.010s	sys 0.000s
I20260709 12:25:26.554585 23951 raft_consensus.cc:493] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.555173 23951 raft_consensus.cc:515] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.555440 23755 raft_consensus.cc:2468] T 7101b7cd32744ca79f6c50b5e31f246a P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50493539c1954244889d4f3e6d55f396 in term 1.
I20260709 12:25:26.556113 23903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7101b7cd32744ca79f6c50b5e31f246a" candidate_uuid: "50493539c1954244889d4f3e6d55f396" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "565673cccffc4dbbb2d7431552ba280f"
I20260709 12:25:26.556694 23796 leader_election.cc:304] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50493539c1954244889d4f3e6d55f396, cc755c49216847f4a9ead01695cd2ab1; no voters: 
W20260709 12:25:26.558481 23795 leader_election.cc:343] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:26.558511 24003 raft_consensus.cc:2804] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:26.560902 23951 leader_election.cc:290] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373)
I20260709 12:25:26.565830 23946 tablet_bootstrap.cc:492] T 09fbbea470544d64b22e97e7f74a305b P cc755c49216847f4a9ead01695cd2ab1: No bootstrap required, opened a new log
I20260709 12:25:26.566404 23946 ts_tablet_manager.cc:1403] T 09fbbea470544d64b22e97e7f74a305b P cc755c49216847f4a9ead01695cd2ab1: Time spent bootstrapping tablet: real 0.119s	user 0.017s	sys 0.042s
I20260709 12:25:26.569667 23755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fedce3e0a6d41aa8954693aab3796cd" candidate_uuid: "50493539c1954244889d4f3e6d55f396" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc755c49216847f4a9ead01695cd2ab1" is_pre_election: true
I20260709 12:25:26.570830 23946 raft_consensus.cc:359] T 09fbbea470544d64b22e97e7f74a305b P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.572232 23946 raft_consensus.cc:385] T 09fbbea470544d64b22e97e7f74a305b P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:26.572710 23946 raft_consensus.cc:740] T 09fbbea470544d64b22e97e7f74a305b P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc755c49216847f4a9ead01695cd2ab1, State: Initialized, Role: FOLLOWER
I20260709 12:25:26.573717 23946 consensus_queue.cc:260] T 09fbbea470544d64b22e97e7f74a305b P cc755c49216847f4a9ead01695cd2ab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.575663 23903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fedce3e0a6d41aa8954693aab3796cd" candidate_uuid: "50493539c1954244889d4f3e6d55f396" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "565673cccffc4dbbb2d7431552ba280f" is_pre_election: true
I20260709 12:25:26.576316 23946 ts_tablet_manager.cc:1434] T 09fbbea470544d64b22e97e7f74a305b P cc755c49216847f4a9ead01695cd2ab1: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:25:26.576638 23903 raft_consensus.cc:2393] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 50493539c1954244889d4f3e6d55f396 in current term 1: Already voted for candidate cc755c49216847f4a9ead01695cd2ab1 in this term.
I20260709 12:25:26.578101 23795 leader_election.cc:304] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50493539c1954244889d4f3e6d55f396; no voters: 565673cccffc4dbbb2d7431552ba280f, cc755c49216847f4a9ead01695cd2ab1
I20260709 12:25:26.579739 23951 raft_consensus.cc:3060] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:26.579906 23969 consensus_queue.cc:1048] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:25:26.590577 23951 raft_consensus.cc:2749] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:25:26.593673 24003 raft_consensus.cc:697] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 [term 1 LEADER]: Becoming Leader. State: Replica: 50493539c1954244889d4f3e6d55f396, State: Running, Role: LEADER
I20260709 12:25:26.603459 24003 consensus_queue.cc:237] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.596418 23945 tablet_bootstrap.cc:492] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f: No bootstrap required, opened a new log
I20260709 12:25:26.606767 23945 ts_tablet_manager.cc:1403] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f: Time spent bootstrapping tablet: real 0.141s	user 0.024s	sys 0.025s
I20260709 12:25:26.610378 23945 raft_consensus.cc:359] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.611629 23945 raft_consensus.cc:385] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:26.612013 23945 raft_consensus.cc:740] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 565673cccffc4dbbb2d7431552ba280f, State: Initialized, Role: FOLLOWER
I20260709 12:25:26.613101 23945 consensus_queue.cc:260] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.615525 23945 ts_tablet_manager.cc:1434] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:25:26.618479 23969 consensus_queue.cc:1048] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:26.618884 23945 tablet_bootstrap.cc:492] T 7101b7cd32744ca79f6c50b5e31f246a P 565673cccffc4dbbb2d7431552ba280f: Bootstrap starting.
I20260709 12:25:26.618643 23652 catalog_manager.cc:5697] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50493539c1954244889d4f3e6d55f396 (127.5.58.2). New cstate: current_term: 1 leader_uuid: "50493539c1954244889d4f3e6d55f396" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:26.625403 23945 tablet_bootstrap.cc:654] T 7101b7cd32744ca79f6c50b5e31f246a P 565673cccffc4dbbb2d7431552ba280f: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:26.645033 23945 tablet_bootstrap.cc:492] T 7101b7cd32744ca79f6c50b5e31f246a P 565673cccffc4dbbb2d7431552ba280f: No bootstrap required, opened a new log
I20260709 12:25:26.645534 23945 ts_tablet_manager.cc:1403] T 7101b7cd32744ca79f6c50b5e31f246a P 565673cccffc4dbbb2d7431552ba280f: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.008s
I20260709 12:25:26.648289 23945 raft_consensus.cc:359] T 7101b7cd32744ca79f6c50b5e31f246a P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.649011 23945 raft_consensus.cc:385] T 7101b7cd32744ca79f6c50b5e31f246a P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:26.649290 23945 raft_consensus.cc:740] T 7101b7cd32744ca79f6c50b5e31f246a P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 565673cccffc4dbbb2d7431552ba280f, State: Initialized, Role: FOLLOWER
I20260709 12:25:26.650142 23945 consensus_queue.cc:260] T 7101b7cd32744ca79f6c50b5e31f246a P 565673cccffc4dbbb2d7431552ba280f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.652591 23945 ts_tablet_manager.cc:1434] T 7101b7cd32744ca79f6c50b5e31f246a P 565673cccffc4dbbb2d7431552ba280f: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:25:26.702006 23964 consensus_queue.cc:1048] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:26.702385 23954 consensus_queue.cc:1048] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:25:26.703385 23969 consensus_queue.cc:1048] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:26.717413 23949 consensus_queue.cc:1048] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [LEADER]: Connected to new peer: Peer: permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:26.744170 23949 raft_consensus.cc:493] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.744762 23949 raft_consensus.cc:515] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.747042 23949 leader_election.cc:290] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837)
I20260709 12:25:26.747759 23754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09fbbea470544d64b22e97e7f74a305b" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc755c49216847f4a9ead01695cd2ab1" is_pre_election: true
I20260709 12:25:26.746802 23962 consensus_queue.cc:1048] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:25:26.748571 23754 raft_consensus.cc:2468] T 09fbbea470544d64b22e97e7f74a305b P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 565673cccffc4dbbb2d7431552ba280f in term 0.
I20260709 12:25:26.748767 23829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09fbbea470544d64b22e97e7f74a305b" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50493539c1954244889d4f3e6d55f396" is_pre_election: true
I20260709 12:25:26.749557 23829 raft_consensus.cc:2468] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 565673cccffc4dbbb2d7431552ba280f in term 0.
I20260709 12:25:26.750175 23870 leader_election.cc:304] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 565673cccffc4dbbb2d7431552ba280f, cc755c49216847f4a9ead01695cd2ab1; no voters: 
I20260709 12:25:26.751191 23962 raft_consensus.cc:2804] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:25:26.751619 23962 raft_consensus.cc:493] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.751974 23962 raft_consensus.cc:3060] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:26.760047 23962 raft_consensus.cc:515] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.768743 23962 leader_election.cc:290] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 election: Requested vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837)
I20260709 12:25:26.769359 23962 consensus_queue.cc:1048] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f [LEADER]: Connected to new peer: Peer: permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:26.769922 23829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09fbbea470544d64b22e97e7f74a305b" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50493539c1954244889d4f3e6d55f396"
I20260709 12:25:26.770553 23829 raft_consensus.cc:3060] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:26.771333 23754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09fbbea470544d64b22e97e7f74a305b" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc755c49216847f4a9ead01695cd2ab1"
I20260709 12:25:26.771994 23754 raft_consensus.cc:3060] T 09fbbea470544d64b22e97e7f74a305b P cc755c49216847f4a9ead01695cd2ab1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:26.780711 23754 raft_consensus.cc:2468] T 09fbbea470544d64b22e97e7f74a305b P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 565673cccffc4dbbb2d7431552ba280f in term 1.
I20260709 12:25:26.781759 23870 leader_election.cc:304] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 565673cccffc4dbbb2d7431552ba280f, cc755c49216847f4a9ead01695cd2ab1; no voters: 
I20260709 12:25:26.782547 23964 raft_consensus.cc:2804] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:26.783041 23964 raft_consensus.cc:697] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f [term 1 LEADER]: Becoming Leader. State: Replica: 565673cccffc4dbbb2d7431552ba280f, State: Running, Role: LEADER
I20260709 12:25:26.783934 23964 consensus_queue.cc:237] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.786939 23829 raft_consensus.cc:2468] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 565673cccffc4dbbb2d7431552ba280f in term 1.
I20260709 12:25:26.795408 23651 catalog_manager.cc:5697] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f reported cstate change: term changed from 0 to 1, leader changed from <none> to 565673cccffc4dbbb2d7431552ba280f (127.5.58.3). New cstate: current_term: 1 leader_uuid: "565673cccffc4dbbb2d7431552ba280f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:26.812568 23904 raft_consensus.cc:3060] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:26.819538 23991 consensus_queue.cc:1048] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:25:26.822649 23991 consensus_queue.cc:1048] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:26.835251 23829 raft_consensus.cc:3060] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:26.844223 23954 consensus_queue.cc:1048] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:25:26.848639 23954 consensus_queue.cc:1048] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:25:26.851454 24003 raft_consensus.cc:493] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.852089 24003 raft_consensus.cc:515] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:26.855973 23903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63fdfbf5f20f4c1ea9db0d4707c4861b" candidate_uuid: "50493539c1954244889d4f3e6d55f396" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "565673cccffc4dbbb2d7431552ba280f" is_pre_election: true
I20260709 12:25:26.857218 23754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63fdfbf5f20f4c1ea9db0d4707c4861b" candidate_uuid: "50493539c1954244889d4f3e6d55f396" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc755c49216847f4a9ead01695cd2ab1" is_pre_election: true
I20260709 12:25:26.858281 24003 leader_election.cc:290] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373)
I20260709 12:25:26.863968 23796 leader_election.cc:304] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50493539c1954244889d4f3e6d55f396; no voters: 565673cccffc4dbbb2d7431552ba280f, cc755c49216847f4a9ead01695cd2ab1
I20260709 12:25:26.867700 24003 raft_consensus.cc:2749] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:26.978068  5352 ts_manager.cc:295] Set tserver state for cc755c49216847f4a9ead01695cd2ab1 to MAINTENANCE_MODE
I20260709 12:25:26.978998  5352 tablet_server.cc:179] TabletServer@127.5.58.1:0 shutting down...
I20260709 12:25:27.001282  5352 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:25:27.002382  5352 tablet_replica.cc:333] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1: stopping tablet replica
I20260709 12:25:27.003096  5352 raft_consensus.cc:2243] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:27.003505  5352 raft_consensus.cc:2272] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:27.006131  5352 tablet_replica.cc:333] T 8085f6a4e929411097ff3c58f115ed80 P cc755c49216847f4a9ead01695cd2ab1: stopping tablet replica
I20260709 12:25:27.006711  5352 raft_consensus.cc:2243] T 8085f6a4e929411097ff3c58f115ed80 P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:27.007113  5352 raft_consensus.cc:2272] T 8085f6a4e929411097ff3c58f115ed80 P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:27.009114  5352 tablet_replica.cc:333] T e5a6d6eb60bb40cd83ba7787fc127421 P cc755c49216847f4a9ead01695cd2ab1: stopping tablet replica
I20260709 12:25:27.009651  5352 raft_consensus.cc:2243] T e5a6d6eb60bb40cd83ba7787fc127421 P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:27.010144  5352 raft_consensus.cc:2272] T e5a6d6eb60bb40cd83ba7787fc127421 P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:27.012753 23870 consensus_peers.cc:597] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:27.013211  5352 tablet_replica.cc:333] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1: stopping tablet replica
I20260709 12:25:27.013895  5352 raft_consensus.cc:2243] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:25:27.014734  5352 raft_consensus.cc:2272] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:27.015821 23964 raft_consensus.cc:493] T 7101b7cd32744ca79f6c50b5e31f246a P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.016407 23964 raft_consensus.cc:515] T 7101b7cd32744ca79f6c50b5e31f246a P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:27.017246  5352 tablet_replica.cc:333] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1: stopping tablet replica
I20260709 12:25:27.017990  5352 raft_consensus.cc:2243] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:25:27.018307 23964 leader_election.cc:290] T 7101b7cd32744ca79f6c50b5e31f246a P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837)
I20260709 12:25:27.019037  5352 raft_consensus.cc:2272] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:27.019618 23830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7101b7cd32744ca79f6c50b5e31f246a" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50493539c1954244889d4f3e6d55f396" is_pre_election: true
W20260709 12:25:27.022732 23870 leader_election.cc:336] T 7101b7cd32744ca79f6c50b5e31f246a P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:25:27.022848  5352 tablet_replica.cc:333] T 09fbbea470544d64b22e97e7f74a305b P cc755c49216847f4a9ead01695cd2ab1: stopping tablet replica
I20260709 12:25:27.023229 23870 leader_election.cc:304] T 7101b7cd32744ca79f6c50b5e31f246a P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 565673cccffc4dbbb2d7431552ba280f; no voters: 50493539c1954244889d4f3e6d55f396, cc755c49216847f4a9ead01695cd2ab1
I20260709 12:25:27.023656  5352 raft_consensus.cc:2243] T 09fbbea470544d64b22e97e7f74a305b P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:27.024114  5352 raft_consensus.cc:2272] T 09fbbea470544d64b22e97e7f74a305b P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:27.024014 23964 raft_consensus.cc:2749] T 7101b7cd32744ca79f6c50b5e31f246a P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:25:27.026371  5352 tablet_replica.cc:333] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1: stopping tablet replica
I20260709 12:25:27.026955  5352 raft_consensus.cc:2243] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:25:27.027714  5352 raft_consensus.cc:2272] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:27.029884  5352 tablet_replica.cc:333] T 3ae7d028bd6548359867825588960bcb P cc755c49216847f4a9ead01695cd2ab1: stopping tablet replica
I20260709 12:25:27.030387  5352 raft_consensus.cc:2243] T 3ae7d028bd6548359867825588960bcb P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:27.030788  5352 raft_consensus.cc:2272] T 3ae7d028bd6548359867825588960bcb P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:27.032914  5352 tablet_replica.cc:333] T 7101b7cd32744ca79f6c50b5e31f246a P cc755c49216847f4a9ead01695cd2ab1: stopping tablet replica
I20260709 12:25:27.033439  5352 raft_consensus.cc:2243] T 7101b7cd32744ca79f6c50b5e31f246a P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:27.033829  5352 raft_consensus.cc:2272] T 7101b7cd32744ca79f6c50b5e31f246a P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:27.035488  5352 tablet_replica.cc:333] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1: stopping tablet replica
I20260709 12:25:27.035945  5352 raft_consensus.cc:2243] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:25:27.036783  5352 raft_consensus.cc:2272] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:27.057348 23903 raft_consensus.cc:3060] T 7101b7cd32744ca79f6c50b5e31f246a P 565673cccffc4dbbb2d7431552ba280f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:27.065174 24003 consensus_queue.cc:1048] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 [LEADER]: Connected to new peer: Peer: permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:25:27.067507  5352 tablet_server.cc:196] TabletServer@127.5.58.1:0 shutdown complete.
I20260709 12:25:27.077270 23964 consensus_queue.cc:1048] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f [LEADER]: Connected to new peer: Peer: permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:25:27.096064 23870 consensus_peers.cc:597] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:27.096979 23796 consensus_peers.cc:597] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:27.168790 23870 consensus_peers.cc:597] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:27.233587 23870 consensus_peers.cc:597] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:27.245303 23870 consensus_peers.cc:597] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:27.333510 23949 consensus_queue.cc:1048] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f [LEADER]: Connected to new peer: Peer: permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:25:27.472669 23870 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111) [suppressed 166 similar messages]
W20260709 12:25:27.956314 23700 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:25:28.133317 24019 raft_consensus.cc:493] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cc755c49216847f4a9ead01695cd2ab1)
I20260709 12:25:28.133898 24019 raft_consensus.cc:515] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:28.135937 24019 leader_election.cc:290] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837), cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445)
I20260709 12:25:28.136857 23829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "797d4300bcc7475ba608835b5d9addc2" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "50493539c1954244889d4f3e6d55f396" is_pre_election: true
W20260709 12:25:28.140129 23870 leader_election.cc:336] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111)
I20260709 12:25:28.140590 23870 leader_election.cc:304] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 565673cccffc4dbbb2d7431552ba280f; no voters: 50493539c1954244889d4f3e6d55f396, cc755c49216847f4a9ead01695cd2ab1
I20260709 12:25:28.141175 24019 raft_consensus.cc:2749] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:28.150239 24007 raft_consensus.cc:493] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cc755c49216847f4a9ead01695cd2ab1)
I20260709 12:25:28.150954 24007 raft_consensus.cc:515] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:28.153040 24007 leader_election.cc:290] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373)
I20260709 12:25:28.154570 23903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "797d4300bcc7475ba608835b5d9addc2" candidate_uuid: "50493539c1954244889d4f3e6d55f396" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "565673cccffc4dbbb2d7431552ba280f" is_pre_election: true
I20260709 12:25:28.155333 23903 raft_consensus.cc:2468] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50493539c1954244889d4f3e6d55f396 in term 1.
I20260709 12:25:28.156723 23795 leader_election.cc:304] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50493539c1954244889d4f3e6d55f396, 565673cccffc4dbbb2d7431552ba280f; no voters: 
I20260709 12:25:28.157686 24007 raft_consensus.cc:2804] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:28.158252 24007 raft_consensus.cc:493] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cc755c49216847f4a9ead01695cd2ab1)
W20260709 12:25:28.158341 23796 leader_election.cc:336] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111)
I20260709 12:25:28.158813 24007 raft_consensus.cc:3060] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:28.166395 24007 raft_consensus.cc:515] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:28.168363 24007 leader_election.cc:290] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 2 election: Requested vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373)
I20260709 12:25:28.169787 23903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "797d4300bcc7475ba608835b5d9addc2" candidate_uuid: "50493539c1954244889d4f3e6d55f396" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "565673cccffc4dbbb2d7431552ba280f"
I20260709 12:25:28.170503 23903 raft_consensus.cc:3060] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:28.172804 23796 leader_election.cc:336] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111)
I20260709 12:25:28.177918 23903 raft_consensus.cc:2468] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50493539c1954244889d4f3e6d55f396 in term 2.
I20260709 12:25:28.179472 23795 leader_election.cc:304] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50493539c1954244889d4f3e6d55f396, 565673cccffc4dbbb2d7431552ba280f; no voters: cc755c49216847f4a9ead01695cd2ab1
I20260709 12:25:28.180321 24007 raft_consensus.cc:2804] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:28.181300 24007 raft_consensus.cc:697] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [term 2 LEADER]: Becoming Leader. State: Replica: 50493539c1954244889d4f3e6d55f396, State: Running, Role: LEADER
I20260709 12:25:28.182327 24007 consensus_queue.cc:237] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:28.189376 23650 catalog_manager.cc:5697] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 reported cstate change: term changed from 1 to 2, leader changed from cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1) to 50493539c1954244889d4f3e6d55f396 (127.5.58.2). New cstate: current_term: 2 leader_uuid: "50493539c1954244889d4f3e6d55f396" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:28.225797 24007 raft_consensus.cc:493] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cc755c49216847f4a9ead01695cd2ab1)
I20260709 12:25:28.226263 24007 raft_consensus.cc:515] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:28.228003 24007 leader_election.cc:290] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373)
I20260709 12:25:28.229142 23903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fedce3e0a6d41aa8954693aab3796cd" candidate_uuid: "50493539c1954244889d4f3e6d55f396" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "565673cccffc4dbbb2d7431552ba280f" is_pre_election: true
I20260709 12:25:28.232038 24019 raft_consensus.cc:493] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cc755c49216847f4a9ead01695cd2ab1)
I20260709 12:25:28.232617 24019 raft_consensus.cc:515] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
W20260709 12:25:28.232942 23796 leader_election.cc:336] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111)
I20260709 12:25:28.233493 23796 leader_election.cc:304] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 50493539c1954244889d4f3e6d55f396; no voters: 565673cccffc4dbbb2d7431552ba280f, cc755c49216847f4a9ead01695cd2ab1
I20260709 12:25:28.234380 24007 raft_consensus.cc:2749] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:28.235020 24019 leader_election.cc:290] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837)
I20260709 12:25:28.236246 23829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fedce3e0a6d41aa8954693aab3796cd" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "50493539c1954244889d4f3e6d55f396" is_pre_election: true
I20260709 12:25:28.237004 23829 raft_consensus.cc:2468] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 565673cccffc4dbbb2d7431552ba280f in term 1.
I20260709 12:25:28.238080 23869 leader_election.cc:304] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50493539c1954244889d4f3e6d55f396, 565673cccffc4dbbb2d7431552ba280f; no voters: 
W20260709 12:25:28.238713 23870 leader_election.cc:336] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111)
I20260709 12:25:28.238866 24019 raft_consensus.cc:2804] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:28.239500 24019 raft_consensus.cc:493] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Starting leader election (detected failure of leader cc755c49216847f4a9ead01695cd2ab1)
I20260709 12:25:28.239854 24019 raft_consensus.cc:3060] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:28.245534 24019 raft_consensus.cc:515] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:28.247388 24019 leader_election.cc:290] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 2 election: Requested vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837)
I20260709 12:25:28.248601 23829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fedce3e0a6d41aa8954693aab3796cd" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "50493539c1954244889d4f3e6d55f396"
I20260709 12:25:28.249212 23829 raft_consensus.cc:3060] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:28.250901 23870 leader_election.cc:336] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111)
I20260709 12:25:28.255046 23829 raft_consensus.cc:2468] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 565673cccffc4dbbb2d7431552ba280f in term 2.
I20260709 12:25:28.256531 23869 leader_election.cc:304] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50493539c1954244889d4f3e6d55f396, 565673cccffc4dbbb2d7431552ba280f; no voters: cc755c49216847f4a9ead01695cd2ab1
I20260709 12:25:28.257319 24019 raft_consensus.cc:2804] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:28.257855 24019 raft_consensus.cc:697] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [term 2 LEADER]: Becoming Leader. State: Replica: 565673cccffc4dbbb2d7431552ba280f, State: Running, Role: LEADER
I20260709 12:25:28.258620 24019 consensus_queue.cc:237] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:28.267182 23650 catalog_manager.cc:5697] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f reported cstate change: term changed from 1 to 2, leader changed from cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1) to 565673cccffc4dbbb2d7431552ba280f (127.5.58.3). New cstate: current_term: 2 leader_uuid: "565673cccffc4dbbb2d7431552ba280f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:28.331602 24019 raft_consensus.cc:493] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cc755c49216847f4a9ead01695cd2ab1)
I20260709 12:25:28.332288 24019 raft_consensus.cc:515] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:28.334303 24019 leader_election.cc:290] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837)
I20260709 12:25:28.334579 23949 raft_consensus.cc:493] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cc755c49216847f4a9ead01695cd2ab1)
I20260709 12:25:28.335362 23949 raft_consensus.cc:515] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:28.335767 23829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee29c9afcbb94cadae2778d1f0d2fcd2" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "50493539c1954244889d4f3e6d55f396" is_pre_election: true
I20260709 12:25:28.337567 23949 leader_election.cc:290] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837)
I20260709 12:25:28.338712 23829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63fdfbf5f20f4c1ea9db0d4707c4861b" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "50493539c1954244889d4f3e6d55f396" is_pre_election: true
W20260709 12:25:28.339370 23870 leader_election.cc:336] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111)
I20260709 12:25:28.339978 23870 leader_election.cc:304] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 565673cccffc4dbbb2d7431552ba280f; no voters: 50493539c1954244889d4f3e6d55f396, cc755c49216847f4a9ead01695cd2ab1
I20260709 12:25:28.341151 23949 raft_consensus.cc:2749] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:25:28.344164 23870 leader_election.cc:336] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111)
I20260709 12:25:28.344844 23870 leader_election.cc:304] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 565673cccffc4dbbb2d7431552ba280f; no voters: 50493539c1954244889d4f3e6d55f396, cc755c49216847f4a9ead01695cd2ab1
I20260709 12:25:28.345978 23949 raft_consensus.cc:2749] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:28.349066 24007 raft_consensus.cc:493] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cc755c49216847f4a9ead01695cd2ab1)
I20260709 12:25:28.349735 24007 raft_consensus.cc:515] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:28.351761 24007 leader_election.cc:290] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373)
I20260709 12:25:28.353050 23903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee29c9afcbb94cadae2778d1f0d2fcd2" candidate_uuid: "50493539c1954244889d4f3e6d55f396" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "565673cccffc4dbbb2d7431552ba280f" is_pre_election: true
I20260709 12:25:28.353716 23903 raft_consensus.cc:2468] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50493539c1954244889d4f3e6d55f396 in term 1.
I20260709 12:25:28.354686 23795 leader_election.cc:304] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50493539c1954244889d4f3e6d55f396, 565673cccffc4dbbb2d7431552ba280f; no voters: 
I20260709 12:25:28.355612 24007 raft_consensus.cc:2804] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:25:28.355904 23796 leader_election.cc:336] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111)
I20260709 12:25:28.355985 24007 raft_consensus.cc:493] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cc755c49216847f4a9ead01695cd2ab1)
I20260709 12:25:28.356555 24007 raft_consensus.cc:3060] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:28.361644 24007 raft_consensus.cc:515] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:28.363267 24007 leader_election.cc:290] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 2 election: Requested vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373)
I20260709 12:25:28.365865 23903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee29c9afcbb94cadae2778d1f0d2fcd2" candidate_uuid: "50493539c1954244889d4f3e6d55f396" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "565673cccffc4dbbb2d7431552ba280f"
I20260709 12:25:28.366798 23903 raft_consensus.cc:3060] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:28.368026 23796 leader_election.cc:336] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111)
I20260709 12:25:28.372189 23903 raft_consensus.cc:2468] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50493539c1954244889d4f3e6d55f396 in term 2.
I20260709 12:25:28.373528 23795 leader_election.cc:304] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50493539c1954244889d4f3e6d55f396, 565673cccffc4dbbb2d7431552ba280f; no voters: cc755c49216847f4a9ead01695cd2ab1
I20260709 12:25:28.374266 24007 raft_consensus.cc:2804] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:28.374711 24007 raft_consensus.cc:697] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [term 2 LEADER]: Becoming Leader. State: Replica: 50493539c1954244889d4f3e6d55f396, State: Running, Role: LEADER
I20260709 12:25:28.375983 24007 consensus_queue.cc:237] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:28.384512 23650 catalog_manager.cc:5697] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 reported cstate change: term changed from 1 to 2, leader changed from cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1) to 50493539c1954244889d4f3e6d55f396 (127.5.58.2). New cstate: current_term: 2 leader_uuid: "50493539c1954244889d4f3e6d55f396" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:28.579195 23903 raft_consensus.cc:1275] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Refusing update from remote peer 50493539c1954244889d4f3e6d55f396: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:28.580891 24007 consensus_queue.cc:1048] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [LEADER]: Connected to new peer: Peer: permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:25:28.599010 23796 consensus_peers.cc:597] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:28.678124 23829 raft_consensus.cc:1275] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [term 2 FOLLOWER]: Refusing update from remote peer 565673cccffc4dbbb2d7431552ba280f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:28.680320 23949 consensus_queue.cc:1048] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [LEADER]: Connected to new peer: Peer: permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:25:28.697990 23870 consensus_peers.cc:597] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:28.795455 24047 raft_consensus.cc:493] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cc755c49216847f4a9ead01695cd2ab1)
I20260709 12:25:28.795994 24047 raft_consensus.cc:515] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:28.798082 24047 leader_election.cc:290] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373)
I20260709 12:25:28.799425 23903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63fdfbf5f20f4c1ea9db0d4707c4861b" candidate_uuid: "50493539c1954244889d4f3e6d55f396" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "565673cccffc4dbbb2d7431552ba280f" is_pre_election: true
W20260709 12:25:28.799584 23796 consensus_peers.cc:597] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:28.800129 23903 raft_consensus.cc:2468] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50493539c1954244889d4f3e6d55f396 in term 1.
I20260709 12:25:28.801156 23795 leader_election.cc:304] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50493539c1954244889d4f3e6d55f396, 565673cccffc4dbbb2d7431552ba280f; no voters: 
I20260709 12:25:28.801949 24047 raft_consensus.cc:2804] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:25:28.803445 23796 leader_election.cc:336] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111)
I20260709 12:25:28.802342 24047 raft_consensus.cc:493] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cc755c49216847f4a9ead01695cd2ab1)
I20260709 12:25:28.804656 24047 raft_consensus.cc:3060] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:28.810580 24047 raft_consensus.cc:515] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:28.812557 24047 leader_election.cc:290] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 2 election: Requested vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373)
I20260709 12:25:28.813833 23903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63fdfbf5f20f4c1ea9db0d4707c4861b" candidate_uuid: "50493539c1954244889d4f3e6d55f396" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "565673cccffc4dbbb2d7431552ba280f"
I20260709 12:25:28.814502 23903 raft_consensus.cc:3060] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:28.816879 23796 leader_election.cc:336] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111)
I20260709 12:25:28.820201 23903 raft_consensus.cc:2468] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50493539c1954244889d4f3e6d55f396 in term 2.
I20260709 12:25:28.821369 23795 leader_election.cc:304] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50493539c1954244889d4f3e6d55f396, 565673cccffc4dbbb2d7431552ba280f; no voters: cc755c49216847f4a9ead01695cd2ab1
I20260709 12:25:28.822240 24047 raft_consensus.cc:2804] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:28.822749 24047 raft_consensus.cc:697] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396 [term 2 LEADER]: Becoming Leader. State: Replica: 50493539c1954244889d4f3e6d55f396, State: Running, Role: LEADER
I20260709 12:25:28.823632 24047 consensus_queue.cc:237] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:28.831346 23650 catalog_manager.cc:5697] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396 reported cstate change: term changed from 1 to 2, leader changed from cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1) to 50493539c1954244889d4f3e6d55f396 (127.5.58.2). New cstate: current_term: 2 leader_uuid: "50493539c1954244889d4f3e6d55f396" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:28.932399 23903 raft_consensus.cc:1275] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Refusing update from remote peer 50493539c1954244889d4f3e6d55f396: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:28.933929 24047 consensus_queue.cc:1048] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [LEADER]: Connected to new peer: Peer: permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:25:28.957446 23700 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:25:29.216284 23903 raft_consensus.cc:1275] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Refusing update from remote peer 50493539c1954244889d4f3e6d55f396: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:29.218128 24053 consensus_queue.cc:1048] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396 [LEADER]: Connected to new peer: Peer: permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:25:29.239050 23796 consensus_peers.cc:597] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396 -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:29.568216 23870 consensus_peers.cc:597] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:29.606869 23870 consensus_peers.cc:597] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:29.628823 23796 consensus_peers.cc:597] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:29.701820 23870 consensus_peers.cc:597] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:29.745237 23870 consensus_peers.cc:597] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:29.801083 23870 consensus_peers.cc:597] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:29.958590 23700 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:25:30.959614 23700 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:25:31.188256 23870 consensus_peers.cc:597] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:31.213811 23796 consensus_peers.cc:597] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:31.467370 23796 consensus_peers.cc:597] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:31.720398 23796 consensus_peers.cc:597] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396 -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:31.960616 23700 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:25:31.965889 23870 consensus_peers.cc:597] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:32.017748 23870 consensus_peers.cc:597] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:32.052261 23870 consensus_peers.cc:597] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:25:32.080531 23870 consensus_peers.cc:597] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:25:32.108040  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:25:32.115396 24057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:25:32.116386 24058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:25:32.117177 24060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:25:32.120275  5352 server_base.cc:1061] running on GCE node
I20260709 12:25:32.121723  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:25:32.121951  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:25:32.122190  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599932122169 us; error 0 us; skew 500 ppm
I20260709 12:25:32.122757  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:25:32.125187  5352 webserver.cc:533] Webserver started at http://127.5.58.1:33487/ using document root <none> and password file <none>
I20260709 12:25:32.125675  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:25:32.125854  5352 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:25:32.129961  5352 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:25:32.134370 24065 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:25:32.135381  5352 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:25:32.135741  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599341860723-5352-0/minicluster-data/ts-0-root
uuid: "cc755c49216847f4a9ead01695cd2ab1"
format_stamp: "Formatted at 2026-07-09 12:25:24 on dist-test-slave-xs2d"
I20260709 12:25:32.136080  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599341860723-5352-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599341860723-5352-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599341860723-5352-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:25:32.159382  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:25:32.160794  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:25:32.162441  5352 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:25:32.166663 24072 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
W20260709 12:25:32.173096 23870 consensus_peers.cc:597] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:25:32.228097  5352 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 12:25:32.228509  5352 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.063s	user 0.002s	sys 0.001s
I20260709 12:25:32.228870  5352 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 12:25:32.233446 24072 tablet_bootstrap.cc:492] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1: Bootstrap starting.
I20260709 12:25:32.252897 24072 tablet_bootstrap.cc:492] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:32.253913 24072 tablet_bootstrap.cc:492] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1: Bootstrap complete.
I20260709 12:25:32.254587 24072 ts_tablet_manager.cc:1403] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.004s
I20260709 12:25:32.257340 24072 raft_consensus.cc:359] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:32.258090 24072 raft_consensus.cc:740] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc755c49216847f4a9ead01695cd2ab1, State: Initialized, Role: FOLLOWER
I20260709 12:25:32.258868 24072 consensus_queue.cc:260] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:32.262676 24072 ts_tablet_manager.cc:1434] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.004s
I20260709 12:25:32.263691 24072 tablet_bootstrap.cc:492] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1: Bootstrap starting.
I20260709 12:25:32.281715 24072 tablet_bootstrap.cc:492] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:32.282968 24072 tablet_bootstrap.cc:492] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1: Bootstrap complete.
I20260709 12:25:32.283943 24072 ts_tablet_manager.cc:1403] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1: Time spent bootstrapping tablet: real 0.020s	user 0.004s	sys 0.012s
I20260709 12:25:32.286847 24072 raft_consensus.cc:359] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:32.287583 24072 raft_consensus.cc:740] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc755c49216847f4a9ead01695cd2ab1, State: Initialized, Role: FOLLOWER
I20260709 12:25:32.288482 24072 consensus_queue.cc:260] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:32.290748 24072 ts_tablet_manager.cc:1434] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:25:32.291944 24072 tablet_bootstrap.cc:492] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1: Bootstrap starting.
I20260709 12:25:32.298830  5352 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 12:25:32.299218  5352 ts_tablet_manager.cc:595] Time spent register tablets: real 0.070s	user 0.064s	sys 0.004s
I20260709 12:25:32.312660 24072 tablet_bootstrap.cc:492] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:32.313889 24072 tablet_bootstrap.cc:492] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1: Bootstrap complete.
I20260709 12:25:32.314764 24072 ts_tablet_manager.cc:1403] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1: Time spent bootstrapping tablet: real 0.023s	user 0.019s	sys 0.000s
I20260709 12:25:32.318162 24072 raft_consensus.cc:359] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:32.319067 24072 raft_consensus.cc:740] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc755c49216847f4a9ead01695cd2ab1, State: Initialized, Role: FOLLOWER
I20260709 12:25:32.320001 24072 consensus_queue.cc:260] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:32.322458 24072 ts_tablet_manager.cc:1434] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:25:32.323796 24072 tablet_bootstrap.cc:492] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1: Bootstrap starting.
I20260709 12:25:32.344161 24072 tablet_bootstrap.cc:492] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:32.345310 24072 tablet_bootstrap.cc:492] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1: Bootstrap complete.
I20260709 12:25:32.346047 24072 ts_tablet_manager.cc:1403] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1: Time spent bootstrapping tablet: real 0.023s	user 0.020s	sys 0.000s
I20260709 12:25:32.349339 24072 raft_consensus.cc:359] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:32.350545 24072 raft_consensus.cc:740] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc755c49216847f4a9ead01695cd2ab1, State: Initialized, Role: FOLLOWER
I20260709 12:25:32.351580 24072 consensus_queue.cc:260] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:32.354278 24072 ts_tablet_manager.cc:1434] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:25:32.355315 24072 tablet_bootstrap.cc:492] T 8085f6a4e929411097ff3c58f115ed80 P cc755c49216847f4a9ead01695cd2ab1: Bootstrap starting.
W20260709 12:25:32.357522 23796 consensus_peers.cc:597] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:25:32.372879 24072 tablet_bootstrap.cc:492] T 8085f6a4e929411097ff3c58f115ed80 P cc755c49216847f4a9ead01695cd2ab1: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:32.374073 24072 tablet_bootstrap.cc:492] T 8085f6a4e929411097ff3c58f115ed80 P cc755c49216847f4a9ead01695cd2ab1: Bootstrap complete.
I20260709 12:25:32.374759  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.1:40445
I20260709 12:25:32.374970 24072 ts_tablet_manager.cc:1403] T 8085f6a4e929411097ff3c58f115ed80 P cc755c49216847f4a9ead01695cd2ab1: Time spent bootstrapping tablet: real 0.020s	user 0.017s	sys 0.000s
I20260709 12:25:32.375059 24135 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.1:40445 every 8 connection(s)
I20260709 12:25:32.377867 24072 raft_consensus.cc:359] T 8085f6a4e929411097ff3c58f115ed80 P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:32.378785 24072 raft_consensus.cc:740] T 8085f6a4e929411097ff3c58f115ed80 P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc755c49216847f4a9ead01695cd2ab1, State: Initialized, Role: FOLLOWER
I20260709 12:25:32.379982 24072 consensus_queue.cc:260] T 8085f6a4e929411097ff3c58f115ed80 P cc755c49216847f4a9ead01695cd2ab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:32.380501  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:32.382030 24072 ts_tablet_manager.cc:1434] T 8085f6a4e929411097ff3c58f115ed80 P cc755c49216847f4a9ead01695cd2ab1: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:25:32.382964 24072 tablet_bootstrap.cc:492] T 7101b7cd32744ca79f6c50b5e31f246a P cc755c49216847f4a9ead01695cd2ab1: Bootstrap starting.
I20260709 12:25:32.405030 24072 tablet_bootstrap.cc:492] T 7101b7cd32744ca79f6c50b5e31f246a P cc755c49216847f4a9ead01695cd2ab1: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:32.406688 24072 tablet_bootstrap.cc:492] T 7101b7cd32744ca79f6c50b5e31f246a P cc755c49216847f4a9ead01695cd2ab1: Bootstrap complete.
I20260709 12:25:32.407763 24072 ts_tablet_manager.cc:1403] T 7101b7cd32744ca79f6c50b5e31f246a P cc755c49216847f4a9ead01695cd2ab1: Time spent bootstrapping tablet: real 0.025s	user 0.020s	sys 0.003s
I20260709 12:25:32.412554 24072 raft_consensus.cc:359] T 7101b7cd32744ca79f6c50b5e31f246a P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:32.413331 24072 raft_consensus.cc:740] T 7101b7cd32744ca79f6c50b5e31f246a P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc755c49216847f4a9ead01695cd2ab1, State: Initialized, Role: FOLLOWER
I20260709 12:25:32.414543 24072 consensus_queue.cc:260] T 7101b7cd32744ca79f6c50b5e31f246a P cc755c49216847f4a9ead01695cd2ab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:32.416528 24072 ts_tablet_manager.cc:1434] T 7101b7cd32744ca79f6c50b5e31f246a P cc755c49216847f4a9ead01695cd2ab1: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:25:32.417459 24072 tablet_bootstrap.cc:492] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1: Bootstrap starting.
I20260709 12:25:32.422695 23903 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "09fbbea470544d64b22e97e7f74a305b"
dest_uuid: "565673cccffc4dbbb2d7431552ba280f"
mode: GRACEFUL
new_leader_uuid: "50493539c1954244889d4f3e6d55f396"
 from {username='slave'} at 127.0.0.1:34924
I20260709 12:25:32.423626 23903 raft_consensus.cc:606] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f [term 1 LEADER]: Received request to transfer leadership to TS 50493539c1954244889d4f3e6d55f396
I20260709 12:25:32.424002 24136 heartbeater.cc:344] Connected to a master server at 127.5.58.62:43883
I20260709 12:25:32.424824 24136 heartbeater.cc:461] Registering TS with master...
I20260709 12:25:32.425757 24136 heartbeater.cc:507] Master 127.5.58.62:43883 requested a full tablet report, sending...
I20260709 12:25:32.427157 23903 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5bcd8bf7d5d04504b30ddbb04f6bc7fd"
dest_uuid: "565673cccffc4dbbb2d7431552ba280f"
mode: GRACEFUL
new_leader_uuid: "50493539c1954244889d4f3e6d55f396"
 from {username='slave'} at 127.0.0.1:34924
I20260709 12:25:32.427754 23903 raft_consensus.cc:606] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [term 1 LEADER]: Received request to transfer leadership to TS 50493539c1954244889d4f3e6d55f396
I20260709 12:25:32.430130  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:25:32.430835  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:32.431535  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:32.433346 23651 ts_manager.cc:194] Re-registered known tserver with Master: cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445)
I20260709 12:25:32.433722 24072 tablet_bootstrap.cc:492] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:32.434500 24072 tablet_bootstrap.cc:492] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1: Bootstrap complete.
I20260709 12:25:32.435037 24072 ts_tablet_manager.cc:1403] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.004s
I20260709 12:25:32.436932 24072 raft_consensus.cc:359] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:32.437476 24072 raft_consensus.cc:740] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc755c49216847f4a9ead01695cd2ab1, State: Initialized, Role: FOLLOWER
I20260709 12:25:32.438139 24072 consensus_queue.cc:260] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:32.439760 24072 ts_tablet_manager.cc:1434] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:25:32.440502 24072 tablet_bootstrap.cc:492] T 3ae7d028bd6548359867825588960bcb P cc755c49216847f4a9ead01695cd2ab1: Bootstrap starting.
I20260709 12:25:32.446640 23651 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51964
I20260709 12:25:32.450434 24119 raft_consensus.cc:993] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f: : Instructing follower 50493539c1954244889d4f3e6d55f396 to start an election
I20260709 12:25:32.450973 24136 heartbeater.cc:499] Master 127.5.58.62:43883 was elected leader, sending a full tablet report...
I20260709 12:25:32.450971 24019 raft_consensus.cc:1081] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [term 1 LEADER]: Signalling peer 50493539c1954244889d4f3e6d55f396 to start an election
I20260709 12:25:32.453447 23829 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5bcd8bf7d5d04504b30ddbb04f6bc7fd"
dest_uuid: "50493539c1954244889d4f3e6d55f396"
 from {username='slave'} at 127.0.0.1:49984
I20260709 12:25:32.454092 23829 raft_consensus.cc:493] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:32.454403 23829 raft_consensus.cc:3060] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:32.457028 24072 tablet_bootstrap.cc:492] T 3ae7d028bd6548359867825588960bcb P cc755c49216847f4a9ead01695cd2ab1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:32.457872 24072 tablet_bootstrap.cc:492] T 3ae7d028bd6548359867825588960bcb P cc755c49216847f4a9ead01695cd2ab1: Bootstrap complete.
I20260709 12:25:32.458431 24072 ts_tablet_manager.cc:1403] T 3ae7d028bd6548359867825588960bcb P cc755c49216847f4a9ead01695cd2ab1: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.006s
I20260709 12:25:32.460179 24072 raft_consensus.cc:359] T 3ae7d028bd6548359867825588960bcb P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:32.460760 24072 raft_consensus.cc:740] T 3ae7d028bd6548359867825588960bcb P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc755c49216847f4a9ead01695cd2ab1, State: Initialized, Role: FOLLOWER
I20260709 12:25:32.461372 24072 consensus_queue.cc:260] T 3ae7d028bd6548359867825588960bcb P cc755c49216847f4a9ead01695cd2ab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:32.461588 23829 raft_consensus.cc:515] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:32.463060 24072 ts_tablet_manager.cc:1434] T 3ae7d028bd6548359867825588960bcb P cc755c49216847f4a9ead01695cd2ab1: Time spent starting tablet: real 0.004s	user 0.003s	sys 0.001s
I20260709 12:25:32.463925 24072 tablet_bootstrap.cc:492] T 09fbbea470544d64b22e97e7f74a305b P cc755c49216847f4a9ead01695cd2ab1: Bootstrap starting.
I20260709 12:25:32.464551 23829 leader_election.cc:290] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 2 election: Requested vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373)
I20260709 12:25:32.465868 23903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bcd8bf7d5d04504b30ddbb04f6bc7fd" candidate_uuid: "50493539c1954244889d4f3e6d55f396" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "565673cccffc4dbbb2d7431552ba280f"
I20260709 12:25:32.466521 23903 raft_consensus.cc:3055] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:32.466848 23903 raft_consensus.cc:740] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 565673cccffc4dbbb2d7431552ba280f, State: Running, Role: LEADER
I20260709 12:25:32.467773 23903 consensus_queue.cc:260] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:32.469282 23903 raft_consensus.cc:3060] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:32.476650 24109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bcd8bf7d5d04504b30ddbb04f6bc7fd" candidate_uuid: "50493539c1954244889d4f3e6d55f396" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cc755c49216847f4a9ead01695cd2ab1"
I20260709 12:25:32.477591 24109 raft_consensus.cc:3060] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:32.477928 23903 raft_consensus.cc:2468] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50493539c1954244889d4f3e6d55f396 in term 2.
I20260709 12:25:32.479213 23795 leader_election.cc:304] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50493539c1954244889d4f3e6d55f396, 565673cccffc4dbbb2d7431552ba280f; no voters: 
I20260709 12:25:32.480129 24047 raft_consensus.cc:2804] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:32.481467 24047 raft_consensus.cc:697] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396 [term 2 LEADER]: Becoming Leader. State: Replica: 50493539c1954244889d4f3e6d55f396, State: Running, Role: LEADER
I20260709 12:25:32.481946 24072 tablet_bootstrap.cc:492] T 09fbbea470544d64b22e97e7f74a305b P cc755c49216847f4a9ead01695cd2ab1: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:32.482335 24047 consensus_queue.cc:237] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:32.482926 24072 tablet_bootstrap.cc:492] T 09fbbea470544d64b22e97e7f74a305b P cc755c49216847f4a9ead01695cd2ab1: Bootstrap complete.
I20260709 12:25:32.483700 24072 ts_tablet_manager.cc:1403] T 09fbbea470544d64b22e97e7f74a305b P cc755c49216847f4a9ead01695cd2ab1: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.004s
I20260709 12:25:32.485525 24109 raft_consensus.cc:2468] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50493539c1954244889d4f3e6d55f396 in term 2.
I20260709 12:25:32.486253 24072 raft_consensus.cc:359] T 09fbbea470544d64b22e97e7f74a305b P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:32.486979 24072 raft_consensus.cc:740] T 09fbbea470544d64b22e97e7f74a305b P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc755c49216847f4a9ead01695cd2ab1, State: Initialized, Role: FOLLOWER
I20260709 12:25:32.487689 24072 consensus_queue.cc:260] T 09fbbea470544d64b22e97e7f74a305b P cc755c49216847f4a9ead01695cd2ab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:32.489799 24072 ts_tablet_manager.cc:1434] T 09fbbea470544d64b22e97e7f74a305b P cc755c49216847f4a9ead01695cd2ab1: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:25:32.490535 24072 tablet_bootstrap.cc:492] T e5a6d6eb60bb40cd83ba7787fc127421 P cc755c49216847f4a9ead01695cd2ab1: Bootstrap starting.
I20260709 12:25:32.493672 23650 catalog_manager.cc:5697] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396 reported cstate change: term changed from 1 to 2, leader changed from 565673cccffc4dbbb2d7431552ba280f (127.5.58.3) to 50493539c1954244889d4f3e6d55f396 (127.5.58.2). New cstate: current_term: 2 leader_uuid: "50493539c1954244889d4f3e6d55f396" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:32.504583 24072 tablet_bootstrap.cc:492] T e5a6d6eb60bb40cd83ba7787fc127421 P cc755c49216847f4a9ead01695cd2ab1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:32.505518 24072 tablet_bootstrap.cc:492] T e5a6d6eb60bb40cd83ba7787fc127421 P cc755c49216847f4a9ead01695cd2ab1: Bootstrap complete.
I20260709 12:25:32.506276 24072 ts_tablet_manager.cc:1403] T e5a6d6eb60bb40cd83ba7787fc127421 P cc755c49216847f4a9ead01695cd2ab1: Time spent bootstrapping tablet: real 0.016s	user 0.009s	sys 0.004s
I20260709 12:25:32.508231 24109 raft_consensus.cc:3060] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:32.508816 24072 raft_consensus.cc:359] T e5a6d6eb60bb40cd83ba7787fc127421 P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:32.509536 24072 raft_consensus.cc:740] T e5a6d6eb60bb40cd83ba7787fc127421 P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc755c49216847f4a9ead01695cd2ab1, State: Initialized, Role: FOLLOWER
I20260709 12:25:32.510236 24072 consensus_queue.cc:260] T e5a6d6eb60bb40cd83ba7787fc127421 P cc755c49216847f4a9ead01695cd2ab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:32.512260 24072 ts_tablet_manager.cc:1434] T e5a6d6eb60bb40cd83ba7787fc127421 P cc755c49216847f4a9ead01695cd2ab1: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:25:32.716506 24119 raft_consensus.cc:993] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f: : Instructing follower 50493539c1954244889d4f3e6d55f396 to start an election
I20260709 12:25:32.716984 24019 raft_consensus.cc:1081] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f [term 1 LEADER]: Signalling peer 50493539c1954244889d4f3e6d55f396 to start an election
I20260709 12:25:32.718454 23829 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "09fbbea470544d64b22e97e7f74a305b"
dest_uuid: "50493539c1954244889d4f3e6d55f396"
 from {username='slave'} at 127.0.0.1:49984
I20260709 12:25:32.719015 23829 raft_consensus.cc:493] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:32.719290 23829 raft_consensus.cc:3060] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:32.724290 23829 raft_consensus.cc:515] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:32.725780 23829 leader_election.cc:290] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 2 election: Requested vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373)
I20260709 12:25:32.726913 23903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09fbbea470544d64b22e97e7f74a305b" candidate_uuid: "50493539c1954244889d4f3e6d55f396" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "565673cccffc4dbbb2d7431552ba280f"
I20260709 12:25:32.727049 24109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09fbbea470544d64b22e97e7f74a305b" candidate_uuid: "50493539c1954244889d4f3e6d55f396" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cc755c49216847f4a9ead01695cd2ab1"
I20260709 12:25:32.727566 23903 raft_consensus.cc:3055] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:32.727715 24109 raft_consensus.cc:3060] T 09fbbea470544d64b22e97e7f74a305b P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:32.727890 23903 raft_consensus.cc:740] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 565673cccffc4dbbb2d7431552ba280f, State: Running, Role: LEADER
I20260709 12:25:32.728772 23903 consensus_queue.cc:260] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:32.730032 23903 raft_consensus.cc:3060] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:32.733035 24109 raft_consensus.cc:2468] T 09fbbea470544d64b22e97e7f74a305b P cc755c49216847f4a9ead01695cd2ab1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50493539c1954244889d4f3e6d55f396 in term 2.
I20260709 12:25:32.734290 23796 leader_election.cc:304] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50493539c1954244889d4f3e6d55f396, cc755c49216847f4a9ead01695cd2ab1; no voters: 
I20260709 12:25:32.735314 24047 raft_consensus.cc:2804] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:32.735939 24047 raft_consensus.cc:697] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396 [term 2 LEADER]: Becoming Leader. State: Replica: 50493539c1954244889d4f3e6d55f396, State: Running, Role: LEADER
I20260709 12:25:32.736554 23903 raft_consensus.cc:2468] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50493539c1954244889d4f3e6d55f396 in term 2.
I20260709 12:25:32.736866 24047 consensus_queue.cc:237] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:32.740192 24109 raft_consensus.cc:3060] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:32.748658 23650 catalog_manager.cc:5697] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396 reported cstate change: term changed from 1 to 2, leader changed from 565673cccffc4dbbb2d7431552ba280f (127.5.58.3) to 50493539c1954244889d4f3e6d55f396 (127.5.58.2). New cstate: current_term: 2 leader_uuid: "50493539c1954244889d4f3e6d55f396" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:32.753751 24109 raft_consensus.cc:3060] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:32.805682 24109 raft_consensus.cc:3060] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:32.961709 23700 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:25:32.964488 24107 raft_consensus.cc:1275] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1 [term 2 FOLLOWER]: Refusing update from remote peer 50493539c1954244889d4f3e6d55f396: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:32.966154 24157 consensus_queue.cc:1048] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:32.981627 23903 raft_consensus.cc:1275] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Refusing update from remote peer 50493539c1954244889d4f3e6d55f396: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:32.984411 24047 consensus_queue.cc:1048] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396 [LEADER]: Connected to new peer: Peer: permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:33.150624 24107 raft_consensus.cc:1275] T 09fbbea470544d64b22e97e7f74a305b P cc755c49216847f4a9ead01695cd2ab1 [term 2 FOLLOWER]: Refusing update from remote peer 50493539c1954244889d4f3e6d55f396: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:33.151993 24157 consensus_queue.cc:1048] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:33.165722 23903 raft_consensus.cc:1275] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Refusing update from remote peer 50493539c1954244889d4f3e6d55f396: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:33.168243 24147 consensus_queue.cc:1048] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396 [LEADER]: Connected to new peer: Peer: permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:33.443748  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:33.462011 23829 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5bcd8bf7d5d04504b30ddbb04f6bc7fd"
dest_uuid: "50493539c1954244889d4f3e6d55f396"
mode: GRACEFUL
new_leader_uuid: "565673cccffc4dbbb2d7431552ba280f"
 from {username='slave'} at 127.0.0.1:50014
I20260709 12:25:33.462664 23829 raft_consensus.cc:606] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396 [term 2 LEADER]: Received request to transfer leadership to TS 565673cccffc4dbbb2d7431552ba280f
I20260709 12:25:33.466216 23829 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "797d4300bcc7475ba608835b5d9addc2"
dest_uuid: "50493539c1954244889d4f3e6d55f396"
mode: GRACEFUL
new_leader_uuid: "565673cccffc4dbbb2d7431552ba280f"
 from {username='slave'} at 127.0.0.1:50014
I20260709 12:25:33.466867 23829 raft_consensus.cc:606] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [term 2 LEADER]: Received request to transfer leadership to TS 565673cccffc4dbbb2d7431552ba280f
I20260709 12:25:33.468317  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:25:33.469130  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:33.469924  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:33.624203 24157 raft_consensus.cc:993] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396: : Instructing follower 565673cccffc4dbbb2d7431552ba280f to start an election
I20260709 12:25:33.624872 24147 raft_consensus.cc:1081] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [term 2 LEADER]: Signalling peer 565673cccffc4dbbb2d7431552ba280f to start an election
I20260709 12:25:33.626608 23903 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "797d4300bcc7475ba608835b5d9addc2"
dest_uuid: "565673cccffc4dbbb2d7431552ba280f"
 from {username='slave'} at 127.0.0.1:34922
I20260709 12:25:33.627143 23903 raft_consensus.cc:493] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:33.627482 23903 raft_consensus.cc:3060] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:33.633888 23903 raft_consensus.cc:515] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:33.635751 23903 leader_election.cc:290] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 3 election: Requested vote from peers 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837), cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445)
I20260709 12:25:33.636741 23829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "797d4300bcc7475ba608835b5d9addc2" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "50493539c1954244889d4f3e6d55f396"
I20260709 12:25:33.636921 24109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "797d4300bcc7475ba608835b5d9addc2" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cc755c49216847f4a9ead01695cd2ab1"
I20260709 12:25:33.637447 23829 raft_consensus.cc:3055] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:33.637639 24109 raft_consensus.cc:3060] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:33.637957 23829 raft_consensus.cc:740] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 50493539c1954244889d4f3e6d55f396, State: Running, Role: LEADER
I20260709 12:25:33.638826 23829 consensus_queue.cc:260] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:33.639698 23829 raft_consensus.cc:3060] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:33.643905 24109 raft_consensus.cc:2468] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 565673cccffc4dbbb2d7431552ba280f in term 3.
I20260709 12:25:33.646041 23870 leader_election.cc:304] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 565673cccffc4dbbb2d7431552ba280f, cc755c49216847f4a9ead01695cd2ab1; no voters: 
I20260709 12:25:33.646901 24168 raft_consensus.cc:2804] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:33.647620 24168 raft_consensus.cc:697] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [term 3 LEADER]: Becoming Leader. State: Replica: 565673cccffc4dbbb2d7431552ba280f, State: Running, Role: LEADER
I20260709 12:25:33.647848 23829 raft_consensus.cc:2468] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 565673cccffc4dbbb2d7431552ba280f in term 3.
I20260709 12:25:33.648491 24168 consensus_queue.cc:237] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:33.655136 23650 catalog_manager.cc:5697] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f reported cstate change: term changed from 2 to 3, leader changed from 50493539c1954244889d4f3e6d55f396 (127.5.58.2) to 565673cccffc4dbbb2d7431552ba280f (127.5.58.3). New cstate: current_term: 3 leader_uuid: "565673cccffc4dbbb2d7431552ba280f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:33.907218 24157 raft_consensus.cc:993] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396: : Instructing follower 565673cccffc4dbbb2d7431552ba280f to start an election
I20260709 12:25:33.908658 24157 raft_consensus.cc:1081] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396 [term 2 LEADER]: Signalling peer 565673cccffc4dbbb2d7431552ba280f to start an election
I20260709 12:25:33.912010 23903 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5bcd8bf7d5d04504b30ddbb04f6bc7fd"
dest_uuid: "565673cccffc4dbbb2d7431552ba280f"
 from {username='slave'} at 127.0.0.1:34922
I20260709 12:25:33.913545 23903 raft_consensus.cc:493] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:33.914183 23903 raft_consensus.cc:3060] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:33.925119 23903 raft_consensus.cc:515] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:33.930697 23903 leader_election.cc:290] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 3 election: Requested vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837)
I20260709 12:25:33.933436 24107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bcd8bf7d5d04504b30ddbb04f6bc7fd" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cc755c49216847f4a9ead01695cd2ab1"
I20260709 12:25:33.934536 23829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bcd8bf7d5d04504b30ddbb04f6bc7fd" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "50493539c1954244889d4f3e6d55f396"
I20260709 12:25:33.935058 24107 raft_consensus.cc:3060] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:33.935350 23829 raft_consensus.cc:3055] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:33.935839 23829 raft_consensus.cc:740] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 50493539c1954244889d4f3e6d55f396, State: Running, Role: LEADER
I20260709 12:25:33.937619 23829 consensus_queue.cc:260] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:33.940317 23829 raft_consensus.cc:3060] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:33.945233 24107 raft_consensus.cc:2468] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 565673cccffc4dbbb2d7431552ba280f in term 3.
I20260709 12:25:33.946877 23870 leader_election.cc:304] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 565673cccffc4dbbb2d7431552ba280f, cc755c49216847f4a9ead01695cd2ab1; no voters: 
I20260709 12:25:33.948122 24119 raft_consensus.cc:2804] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:33.949563 23829 raft_consensus.cc:2468] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 565673cccffc4dbbb2d7431552ba280f in term 3.
I20260709 12:25:33.951267 24119 raft_consensus.cc:697] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [term 3 LEADER]: Becoming Leader. State: Replica: 565673cccffc4dbbb2d7431552ba280f, State: Running, Role: LEADER
I20260709 12:25:33.953732 24119 consensus_queue.cc:237] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
W20260709 12:25:33.963493 23700 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:25:33.966516 23650 catalog_manager.cc:5697] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f reported cstate change: term changed from 2 to 3, leader changed from 50493539c1954244889d4f3e6d55f396 (127.5.58.2) to 565673cccffc4dbbb2d7431552ba280f (127.5.58.3). New cstate: current_term: 3 leader_uuid: "565673cccffc4dbbb2d7431552ba280f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:34.056824 23829 raft_consensus.cc:1275] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [term 3 FOLLOWER]: Refusing update from remote peer 565673cccffc4dbbb2d7431552ba280f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:34.058302 24177 consensus_queue.cc:1048] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [LEADER]: Connected to new peer: Peer: permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:34.072747 24107 raft_consensus.cc:1275] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [term 3 FOLLOWER]: Refusing update from remote peer 565673cccffc4dbbb2d7431552ba280f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:34.074766 24119 consensus_queue.cc:1048] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:34.356149 23829 raft_consensus.cc:1275] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396 [term 3 FOLLOWER]: Refusing update from remote peer 565673cccffc4dbbb2d7431552ba280f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:34.358100 24177 consensus_queue.cc:1048] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [LEADER]: Connected to new peer: Peer: permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:34.370699 24107 raft_consensus.cc:1275] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1 [term 3 FOLLOWER]: Refusing update from remote peer 565673cccffc4dbbb2d7431552ba280f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:34.373227 24119 consensus_queue.cc:1048] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:34.478242  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:34.489410 23903 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fedce3e0a6d41aa8954693aab3796cd"
dest_uuid: "565673cccffc4dbbb2d7431552ba280f"
mode: GRACEFUL
new_leader_uuid: "50493539c1954244889d4f3e6d55f396"
 from {username='slave'} at 127.0.0.1:34924
I20260709 12:25:34.490382 23903 raft_consensus.cc:606] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [term 2 LEADER]: Received request to transfer leadership to TS 50493539c1954244889d4f3e6d55f396
I20260709 12:25:34.494122 23903 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5a6d6eb60bb40cd83ba7787fc127421"
dest_uuid: "565673cccffc4dbbb2d7431552ba280f"
mode: GRACEFUL
new_leader_uuid: "50493539c1954244889d4f3e6d55f396"
 from {username='slave'} at 127.0.0.1:34924
I20260709 12:25:34.494870 23903 raft_consensus.cc:606] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f [term 1 LEADER]: Received request to transfer leadership to TS 50493539c1954244889d4f3e6d55f396
I20260709 12:25:34.496518  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:25:34.497299  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:34.498133  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:34.514812 24168 raft_consensus.cc:993] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f: : Instructing follower 50493539c1954244889d4f3e6d55f396 to start an election
I20260709 12:25:34.515364 24119 raft_consensus.cc:1081] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [term 2 LEADER]: Signalling peer 50493539c1954244889d4f3e6d55f396 to start an election
I20260709 12:25:34.517164 23829 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2fedce3e0a6d41aa8954693aab3796cd"
dest_uuid: "50493539c1954244889d4f3e6d55f396"
 from {username='slave'} at 127.0.0.1:49984
I20260709 12:25:34.517583 23829 raft_consensus.cc:493] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:34.517807 23829 raft_consensus.cc:3060] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:34.524461 23829 raft_consensus.cc:515] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:34.527923 23829 leader_election.cc:290] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 3 election: Requested vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373)
I20260709 12:25:34.529304 24107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fedce3e0a6d41aa8954693aab3796cd" candidate_uuid: "50493539c1954244889d4f3e6d55f396" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cc755c49216847f4a9ead01695cd2ab1"
I20260709 12:25:34.529320 23903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fedce3e0a6d41aa8954693aab3796cd" candidate_uuid: "50493539c1954244889d4f3e6d55f396" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "565673cccffc4dbbb2d7431552ba280f"
I20260709 12:25:34.530102 24107 raft_consensus.cc:3060] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:34.530159 23903 raft_consensus.cc:3055] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:34.530589 23903 raft_consensus.cc:740] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 565673cccffc4dbbb2d7431552ba280f, State: Running, Role: LEADER
I20260709 12:25:34.531387 23903 consensus_queue.cc:260] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:34.532513 23903 raft_consensus.cc:3060] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:34.536749 24107 raft_consensus.cc:2468] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50493539c1954244889d4f3e6d55f396 in term 3.
I20260709 12:25:34.537736 23903 raft_consensus.cc:2468] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50493539c1954244889d4f3e6d55f396 in term 3.
I20260709 12:25:34.538652 23796 leader_election.cc:304] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50493539c1954244889d4f3e6d55f396, cc755c49216847f4a9ead01695cd2ab1; no voters: 
I20260709 12:25:34.540863 24147 raft_consensus.cc:2804] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:34.542270 24147 raft_consensus.cc:697] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [term 3 LEADER]: Becoming Leader. State: Replica: 50493539c1954244889d4f3e6d55f396, State: Running, Role: LEADER
I20260709 12:25:34.543617 24147 consensus_queue.cc:237] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:34.553153 23651 catalog_manager.cc:5697] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 reported cstate change: term changed from 2 to 3, leader changed from 565673cccffc4dbbb2d7431552ba280f (127.5.58.3) to 50493539c1954244889d4f3e6d55f396 (127.5.58.2). New cstate: current_term: 3 leader_uuid: "50493539c1954244889d4f3e6d55f396" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:34.936686 24107 raft_consensus.cc:1275] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [term 3 FOLLOWER]: Refusing update from remote peer 50493539c1954244889d4f3e6d55f396: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:34.938102 24147 consensus_queue.cc:1048] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:34.949352 23903 raft_consensus.cc:1275] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [term 3 FOLLOWER]: Refusing update from remote peer 50493539c1954244889d4f3e6d55f396: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:34.952983 24186 consensus_queue.cc:1048] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [LEADER]: Connected to new peer: Peer: permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:25:34.965238 23700 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:25:35.031684 24119 raft_consensus.cc:993] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f: : Instructing follower 50493539c1954244889d4f3e6d55f396 to start an election
I20260709 12:25:35.032250 24119 raft_consensus.cc:1081] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f [term 1 LEADER]: Signalling peer 50493539c1954244889d4f3e6d55f396 to start an election
I20260709 12:25:35.033980 23829 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e5a6d6eb60bb40cd83ba7787fc127421"
dest_uuid: "50493539c1954244889d4f3e6d55f396"
 from {username='slave'} at 127.0.0.1:49984
I20260709 12:25:35.034462 23829 raft_consensus.cc:493] T e5a6d6eb60bb40cd83ba7787fc127421 P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:35.034771 23829 raft_consensus.cc:3060] T e5a6d6eb60bb40cd83ba7787fc127421 P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:35.040324 23829 raft_consensus.cc:515] T e5a6d6eb60bb40cd83ba7787fc127421 P 50493539c1954244889d4f3e6d55f396 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:35.041837 23829 leader_election.cc:290] T e5a6d6eb60bb40cd83ba7787fc127421 P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 2 election: Requested vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373)
I20260709 12:25:35.042906 24107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a6d6eb60bb40cd83ba7787fc127421" candidate_uuid: "50493539c1954244889d4f3e6d55f396" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cc755c49216847f4a9ead01695cd2ab1"
I20260709 12:25:35.042924 23903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5a6d6eb60bb40cd83ba7787fc127421" candidate_uuid: "50493539c1954244889d4f3e6d55f396" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "565673cccffc4dbbb2d7431552ba280f"
I20260709 12:25:35.043602 24107 raft_consensus.cc:3060] T e5a6d6eb60bb40cd83ba7787fc127421 P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:35.043696 23903 raft_consensus.cc:3055] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:35.044134 23903 raft_consensus.cc:740] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 565673cccffc4dbbb2d7431552ba280f, State: Running, Role: LEADER
I20260709 12:25:35.044893 23903 consensus_queue.cc:260] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:35.045866 23903 raft_consensus.cc:3060] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:35.049331 24107 raft_consensus.cc:2468] T e5a6d6eb60bb40cd83ba7787fc127421 P cc755c49216847f4a9ead01695cd2ab1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50493539c1954244889d4f3e6d55f396 in term 2.
I20260709 12:25:35.050696 23796 leader_election.cc:304] T e5a6d6eb60bb40cd83ba7787fc127421 P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50493539c1954244889d4f3e6d55f396, cc755c49216847f4a9ead01695cd2ab1; no voters: 
I20260709 12:25:35.051493 24196 raft_consensus.cc:2804] T e5a6d6eb60bb40cd83ba7787fc127421 P 50493539c1954244889d4f3e6d55f396 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:35.051903 24196 raft_consensus.cc:697] T e5a6d6eb60bb40cd83ba7787fc127421 P 50493539c1954244889d4f3e6d55f396 [term 2 LEADER]: Becoming Leader. State: Replica: 50493539c1954244889d4f3e6d55f396, State: Running, Role: LEADER
I20260709 12:25:35.051999 23903 raft_consensus.cc:2468] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50493539c1954244889d4f3e6d55f396 in term 2.
I20260709 12:25:35.052911 24196 consensus_queue.cc:237] T e5a6d6eb60bb40cd83ba7787fc127421 P 50493539c1954244889d4f3e6d55f396 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:35.061846 23650 catalog_manager.cc:5697] T e5a6d6eb60bb40cd83ba7787fc127421 P 50493539c1954244889d4f3e6d55f396 reported cstate change: term changed from 1 to 2, leader changed from 565673cccffc4dbbb2d7431552ba280f (127.5.58.3) to 50493539c1954244889d4f3e6d55f396 (127.5.58.2). New cstate: current_term: 2 leader_uuid: "50493539c1954244889d4f3e6d55f396" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:35.506294  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:35.514007 23829 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7101b7cd32744ca79f6c50b5e31f246a"
dest_uuid: "50493539c1954244889d4f3e6d55f396"
mode: GRACEFUL
new_leader_uuid: "565673cccffc4dbbb2d7431552ba280f"
 from {username='slave'} at 127.0.0.1:50014
I20260709 12:25:35.514606 23829 raft_consensus.cc:606] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 [term 1 LEADER]: Received request to transfer leadership to TS 565673cccffc4dbbb2d7431552ba280f
I20260709 12:25:35.517657 23829 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee29c9afcbb94cadae2778d1f0d2fcd2"
dest_uuid: "50493539c1954244889d4f3e6d55f396"
mode: GRACEFUL
new_leader_uuid: "565673cccffc4dbbb2d7431552ba280f"
 from {username='slave'} at 127.0.0.1:50014
I20260709 12:25:35.518210 23829 raft_consensus.cc:606] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [term 2 LEADER]: Received request to transfer leadership to TS 565673cccffc4dbbb2d7431552ba280f
I20260709 12:25:35.519423  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:25:35.519912  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:35.520670  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:35.601454 24107 raft_consensus.cc:1275] T e5a6d6eb60bb40cd83ba7787fc127421 P cc755c49216847f4a9ead01695cd2ab1 [term 2 FOLLOWER]: Refusing update from remote peer 50493539c1954244889d4f3e6d55f396: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:35.602875 24186 consensus_queue.cc:1048] T e5a6d6eb60bb40cd83ba7787fc127421 P 50493539c1954244889d4f3e6d55f396 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:35.614205 23903 raft_consensus.cc:1275] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Refusing update from remote peer 50493539c1954244889d4f3e6d55f396: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:35.615717 24196 consensus_queue.cc:1048] T e5a6d6eb60bb40cd83ba7787fc127421 P 50493539c1954244889d4f3e6d55f396 [LEADER]: Connected to new peer: Peer: permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:35.621034 24196 raft_consensus.cc:993] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396: : Instructing follower 565673cccffc4dbbb2d7431552ba280f to start an election
I20260709 12:25:35.621568 24196 raft_consensus.cc:1081] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [term 2 LEADER]: Signalling peer 565673cccffc4dbbb2d7431552ba280f to start an election
I20260709 12:25:35.625434 23904 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ee29c9afcbb94cadae2778d1f0d2fcd2"
dest_uuid: "565673cccffc4dbbb2d7431552ba280f"
 from {username='slave'} at 127.0.0.1:34922
I20260709 12:25:35.626714 23904 raft_consensus.cc:493] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:35.627229 23904 raft_consensus.cc:3060] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:35.636890 23904 raft_consensus.cc:515] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:35.639027 23904 leader_election.cc:290] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 3 election: Requested vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837)
I20260709 12:25:35.639582 24107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee29c9afcbb94cadae2778d1f0d2fcd2" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cc755c49216847f4a9ead01695cd2ab1"
I20260709 12:25:35.640300 23829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee29c9afcbb94cadae2778d1f0d2fcd2" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "50493539c1954244889d4f3e6d55f396"
I20260709 12:25:35.640352 24107 raft_consensus.cc:3060] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:35.640861 23829 raft_consensus.cc:3055] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:35.641127 23829 raft_consensus.cc:740] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 50493539c1954244889d4f3e6d55f396, State: Running, Role: LEADER
I20260709 12:25:35.641760 23829 consensus_queue.cc:260] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:35.642588 23829 raft_consensus.cc:3060] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:35.647171 24107 raft_consensus.cc:2468] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 565673cccffc4dbbb2d7431552ba280f in term 3.
I20260709 12:25:35.648514 23870 leader_election.cc:304] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 565673cccffc4dbbb2d7431552ba280f, cc755c49216847f4a9ead01695cd2ab1; no voters: 
I20260709 12:25:35.649230 24119 raft_consensus.cc:2804] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:35.649365 23829 raft_consensus.cc:2468] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 565673cccffc4dbbb2d7431552ba280f in term 3.
I20260709 12:25:35.650976 24119 raft_consensus.cc:697] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f [term 3 LEADER]: Becoming Leader. State: Replica: 565673cccffc4dbbb2d7431552ba280f, State: Running, Role: LEADER
I20260709 12:25:35.652099 24119 consensus_queue.cc:237] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:35.660848 23650 catalog_manager.cc:5697] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f reported cstate change: term changed from 2 to 3, leader changed from 50493539c1954244889d4f3e6d55f396 (127.5.58.2) to 565673cccffc4dbbb2d7431552ba280f (127.5.58.3). New cstate: current_term: 3 leader_uuid: "565673cccffc4dbbb2d7431552ba280f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:35.894451 24186 raft_consensus.cc:993] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396: : Instructing follower 565673cccffc4dbbb2d7431552ba280f to start an election
I20260709 12:25:35.895004 24196 raft_consensus.cc:1081] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 [term 1 LEADER]: Signalling peer 565673cccffc4dbbb2d7431552ba280f to start an election
I20260709 12:25:35.896538 23904 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7101b7cd32744ca79f6c50b5e31f246a"
dest_uuid: "565673cccffc4dbbb2d7431552ba280f"
 from {username='slave'} at 127.0.0.1:34922
I20260709 12:25:35.897049 23904 raft_consensus.cc:493] T 7101b7cd32744ca79f6c50b5e31f246a P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:35.897369 23904 raft_consensus.cc:3060] T 7101b7cd32744ca79f6c50b5e31f246a P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:35.903160 23904 raft_consensus.cc:515] T 7101b7cd32744ca79f6c50b5e31f246a P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:35.904976 23904 leader_election.cc:290] T 7101b7cd32744ca79f6c50b5e31f246a P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 2 election: Requested vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837)
I20260709 12:25:35.905874 24109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7101b7cd32744ca79f6c50b5e31f246a" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cc755c49216847f4a9ead01695cd2ab1"
I20260709 12:25:35.905943 23829 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7101b7cd32744ca79f6c50b5e31f246a" candidate_uuid: "565673cccffc4dbbb2d7431552ba280f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "50493539c1954244889d4f3e6d55f396"
I20260709 12:25:35.906755 24109 raft_consensus.cc:3060] T 7101b7cd32744ca79f6c50b5e31f246a P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:35.906832 23829 raft_consensus.cc:3055] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:35.907313 23829 raft_consensus.cc:740] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 50493539c1954244889d4f3e6d55f396, State: Running, Role: LEADER
I20260709 12:25:35.908236 23829 consensus_queue.cc:260] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:35.909452 23829 raft_consensus.cc:3060] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:35.914566 24109 raft_consensus.cc:2468] T 7101b7cd32744ca79f6c50b5e31f246a P cc755c49216847f4a9ead01695cd2ab1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 565673cccffc4dbbb2d7431552ba280f in term 2.
I20260709 12:25:35.914876 23829 raft_consensus.cc:2468] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 565673cccffc4dbbb2d7431552ba280f in term 2.
I20260709 12:25:35.915721 23870 leader_election.cc:304] T 7101b7cd32744ca79f6c50b5e31f246a P 565673cccffc4dbbb2d7431552ba280f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 565673cccffc4dbbb2d7431552ba280f, cc755c49216847f4a9ead01695cd2ab1; no voters: 
I20260709 12:25:35.916689 24119 raft_consensus.cc:2804] T 7101b7cd32744ca79f6c50b5e31f246a P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:35.917145 24119 raft_consensus.cc:697] T 7101b7cd32744ca79f6c50b5e31f246a P 565673cccffc4dbbb2d7431552ba280f [term 2 LEADER]: Becoming Leader. State: Replica: 565673cccffc4dbbb2d7431552ba280f, State: Running, Role: LEADER
I20260709 12:25:35.918082 24119 consensus_queue.cc:237] T 7101b7cd32744ca79f6c50b5e31f246a P 565673cccffc4dbbb2d7431552ba280f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:35.924817 23650 catalog_manager.cc:5697] T 7101b7cd32744ca79f6c50b5e31f246a P 565673cccffc4dbbb2d7431552ba280f reported cstate change: term changed from 1 to 2, leader changed from 50493539c1954244889d4f3e6d55f396 (127.5.58.2) to 565673cccffc4dbbb2d7431552ba280f (127.5.58.3). New cstate: current_term: 2 leader_uuid: "565673cccffc4dbbb2d7431552ba280f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:35.966439 23700 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:25:36.220963 24109 raft_consensus.cc:1275] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [term 3 FOLLOWER]: Refusing update from remote peer 565673cccffc4dbbb2d7431552ba280f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:36.222332 24119 consensus_queue.cc:1048] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:36.232632 23829 raft_consensus.cc:1275] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [term 3 FOLLOWER]: Refusing update from remote peer 565673cccffc4dbbb2d7431552ba280f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:36.234408 24119 consensus_queue.cc:1048] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f [LEADER]: Connected to new peer: Peer: permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:36.319362 24109 raft_consensus.cc:1275] T 7101b7cd32744ca79f6c50b5e31f246a P cc755c49216847f4a9ead01695cd2ab1 [term 2 FOLLOWER]: Refusing update from remote peer 565673cccffc4dbbb2d7431552ba280f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:36.320756 24119 consensus_queue.cc:1048] T 7101b7cd32744ca79f6c50b5e31f246a P 565673cccffc4dbbb2d7431552ba280f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:36.331537 23829 raft_consensus.cc:1275] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 [term 2 FOLLOWER]: Refusing update from remote peer 565673cccffc4dbbb2d7431552ba280f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:36.333726 24209 consensus_queue.cc:1048] T 7101b7cd32744ca79f6c50b5e31f246a P 565673cccffc4dbbb2d7431552ba280f [LEADER]: Connected to new peer: Peer: permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:36.527208  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:36.535113 23904 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ae7d028bd6548359867825588960bcb"
dest_uuid: "565673cccffc4dbbb2d7431552ba280f"
mode: GRACEFUL
new_leader_uuid: "50493539c1954244889d4f3e6d55f396"
 from {username='slave'} at 127.0.0.1:34924
I20260709 12:25:36.535637 23904 raft_consensus.cc:606] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f [term 1 LEADER]: Received request to transfer leadership to TS 50493539c1954244889d4f3e6d55f396
I20260709 12:25:36.538821 23904 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "797d4300bcc7475ba608835b5d9addc2"
dest_uuid: "565673cccffc4dbbb2d7431552ba280f"
mode: GRACEFUL
new_leader_uuid: "50493539c1954244889d4f3e6d55f396"
 from {username='slave'} at 127.0.0.1:34924
I20260709 12:25:36.539336 23904 raft_consensus.cc:606] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [term 3 LEADER]: Received request to transfer leadership to TS 50493539c1954244889d4f3e6d55f396
I20260709 12:25:36.540733  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:25:36.541296  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:36.541929  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:36.730548 24119 raft_consensus.cc:993] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f: : Instructing follower 50493539c1954244889d4f3e6d55f396 to start an election
I20260709 12:25:36.731187 24209 raft_consensus.cc:1081] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [term 3 LEADER]: Signalling peer 50493539c1954244889d4f3e6d55f396 to start an election
I20260709 12:25:36.733219 23829 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "797d4300bcc7475ba608835b5d9addc2"
dest_uuid: "50493539c1954244889d4f3e6d55f396"
 from {username='slave'} at 127.0.0.1:49984
I20260709 12:25:36.733754 23829 raft_consensus.cc:493] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:36.734037 23829 raft_consensus.cc:3060] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:36.740854 23829 raft_consensus.cc:515] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:36.742364 23829 leader_election.cc:290] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 4 election: Requested vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373)
I20260709 12:25:36.743589 24109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "797d4300bcc7475ba608835b5d9addc2" candidate_uuid: "50493539c1954244889d4f3e6d55f396" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "cc755c49216847f4a9ead01695cd2ab1"
I20260709 12:25:36.743673 23904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "797d4300bcc7475ba608835b5d9addc2" candidate_uuid: "50493539c1954244889d4f3e6d55f396" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "565673cccffc4dbbb2d7431552ba280f"
I20260709 12:25:36.744177 24109 raft_consensus.cc:3060] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:36.744613 23904 raft_consensus.cc:3055] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:36.745011 23904 raft_consensus.cc:740] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 565673cccffc4dbbb2d7431552ba280f, State: Running, Role: LEADER
I20260709 12:25:36.745879 23904 consensus_queue.cc:260] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:36.746938 23904 raft_consensus.cc:3060] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:36.749814 24109 raft_consensus.cc:2468] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50493539c1954244889d4f3e6d55f396 in term 4.
I20260709 12:25:36.751250 23796 leader_election.cc:304] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50493539c1954244889d4f3e6d55f396, cc755c49216847f4a9ead01695cd2ab1; no voters: 
I20260709 12:25:36.751965 24196 raft_consensus.cc:2804] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:36.752429 24196 raft_consensus.cc:697] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [term 4 LEADER]: Becoming Leader. State: Replica: 50493539c1954244889d4f3e6d55f396, State: Running, Role: LEADER
I20260709 12:25:36.752662 23904 raft_consensus.cc:2468] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50493539c1954244889d4f3e6d55f396 in term 4.
I20260709 12:25:36.753275 24196 consensus_queue.cc:237] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:36.760623 23652 catalog_manager.cc:5697] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 reported cstate change: term changed from 3 to 4, leader changed from 565673cccffc4dbbb2d7431552ba280f (127.5.58.3) to 50493539c1954244889d4f3e6d55f396 (127.5.58.2). New cstate: current_term: 4 leader_uuid: "50493539c1954244889d4f3e6d55f396" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:36.967502 23700 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:25:36.975131 24119 raft_consensus.cc:993] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f: : Instructing follower 50493539c1954244889d4f3e6d55f396 to start an election
I20260709 12:25:36.975677 24119 raft_consensus.cc:1081] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f [term 1 LEADER]: Signalling peer 50493539c1954244889d4f3e6d55f396 to start an election
I20260709 12:25:36.977412 23829 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3ae7d028bd6548359867825588960bcb"
dest_uuid: "50493539c1954244889d4f3e6d55f396"
 from {username='slave'} at 127.0.0.1:49984
I20260709 12:25:36.978009 23829 raft_consensus.cc:493] T 3ae7d028bd6548359867825588960bcb P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:36.978367 23829 raft_consensus.cc:3060] T 3ae7d028bd6548359867825588960bcb P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:36.983614 23829 raft_consensus.cc:515] T 3ae7d028bd6548359867825588960bcb P 50493539c1954244889d4f3e6d55f396 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:36.985143 23829 leader_election.cc:290] T 3ae7d028bd6548359867825588960bcb P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 2 election: Requested vote from peers cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445), 565673cccffc4dbbb2d7431552ba280f (127.5.58.3:35373)
I20260709 12:25:36.986258 24109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ae7d028bd6548359867825588960bcb" candidate_uuid: "50493539c1954244889d4f3e6d55f396" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cc755c49216847f4a9ead01695cd2ab1"
I20260709 12:25:36.986258 23904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ae7d028bd6548359867825588960bcb" candidate_uuid: "50493539c1954244889d4f3e6d55f396" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "565673cccffc4dbbb2d7431552ba280f"
I20260709 12:25:36.986961 24109 raft_consensus.cc:3060] T 3ae7d028bd6548359867825588960bcb P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:36.987092 23904 raft_consensus.cc:3055] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:36.987545 23904 raft_consensus.cc:740] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 565673cccffc4dbbb2d7431552ba280f, State: Running, Role: LEADER
I20260709 12:25:36.988425 23904 consensus_queue.cc:260] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:36.989730 23904 raft_consensus.cc:3060] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:36.993814 24109 raft_consensus.cc:2468] T 3ae7d028bd6548359867825588960bcb P cc755c49216847f4a9ead01695cd2ab1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50493539c1954244889d4f3e6d55f396 in term 2.
I20260709 12:25:36.995245 23796 leader_election.cc:304] T 3ae7d028bd6548359867825588960bcb P 50493539c1954244889d4f3e6d55f396 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50493539c1954244889d4f3e6d55f396, cc755c49216847f4a9ead01695cd2ab1; no voters: 
I20260709 12:25:36.995824 23904 raft_consensus.cc:2468] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50493539c1954244889d4f3e6d55f396 in term 2.
I20260709 12:25:36.996172 24186 raft_consensus.cc:2804] T 3ae7d028bd6548359867825588960bcb P 50493539c1954244889d4f3e6d55f396 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:36.996693 24186 raft_consensus.cc:697] T 3ae7d028bd6548359867825588960bcb P 50493539c1954244889d4f3e6d55f396 [term 2 LEADER]: Becoming Leader. State: Replica: 50493539c1954244889d4f3e6d55f396, State: Running, Role: LEADER
I20260709 12:25:36.997668 24186 consensus_queue.cc:237] T 3ae7d028bd6548359867825588960bcb P 50493539c1954244889d4f3e6d55f396 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } }
I20260709 12:25:37.005297 23652 catalog_manager.cc:5697] T 3ae7d028bd6548359867825588960bcb P 50493539c1954244889d4f3e6d55f396 reported cstate change: term changed from 1 to 2, leader changed from 565673cccffc4dbbb2d7431552ba280f (127.5.58.3) to 50493539c1954244889d4f3e6d55f396 (127.5.58.2). New cstate: current_term: 2 leader_uuid: "50493539c1954244889d4f3e6d55f396" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50493539c1954244889d4f3e6d55f396" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:37.198930 23904 raft_consensus.cc:1275] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [term 4 FOLLOWER]: Refusing update from remote peer 50493539c1954244889d4f3e6d55f396: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:25:37.200305 24186 consensus_queue.cc:1048] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [LEADER]: Connected to new peer: Peer: permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:37.211582 24109 raft_consensus.cc:1275] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [term 4 FOLLOWER]: Refusing update from remote peer 50493539c1954244889d4f3e6d55f396: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:25:37.213203 24186 consensus_queue.cc:1048] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:37.534157 23904 raft_consensus.cc:1275] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Refusing update from remote peer 50493539c1954244889d4f3e6d55f396: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:37.535872 24186 consensus_queue.cc:1048] T 3ae7d028bd6548359867825588960bcb P 50493539c1954244889d4f3e6d55f396 [LEADER]: Connected to new peer: Peer: permanent_uuid: "565673cccffc4dbbb2d7431552ba280f" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 35373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:37.545639 24109 raft_consensus.cc:1275] T 3ae7d028bd6548359867825588960bcb P cc755c49216847f4a9ead01695cd2ab1 [term 2 FOLLOWER]: Refusing update from remote peer 50493539c1954244889d4f3e6d55f396: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:37.547317 24237 consensus_queue.cc:1048] T 3ae7d028bd6548359867825588960bcb P 50493539c1954244889d4f3e6d55f396 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc755c49216847f4a9ead01695cd2ab1" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 40445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:37.549118  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:37.594321  5352 tablet_server.cc:179] TabletServer@127.5.58.1:40445 shutting down...
I20260709 12:25:37.625989  5352 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:25:37.627086  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:37.628070  5352 raft_consensus.cc:2243] T e5a6d6eb60bb40cd83ba7787fc127421 P cc755c49216847f4a9ead01695cd2ab1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:37.628662  5352 raft_consensus.cc:2272] T e5a6d6eb60bb40cd83ba7787fc127421 P cc755c49216847f4a9ead01695cd2ab1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:37.632128  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:37.632879  5352 raft_consensus.cc:2243] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:37.633410  5352 raft_consensus.cc:2272] T 63fdfbf5f20f4c1ea9db0d4707c4861b P cc755c49216847f4a9ead01695cd2ab1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:37.638147  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:37.639145  5352 raft_consensus.cc:2243] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1 [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 12:25:37.639429 23796 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 101 similar messages]
I20260709 12:25:37.639816  5352 raft_consensus.cc:2272] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P cc755c49216847f4a9ead01695cd2ab1 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:37.645454 23796 consensus_peers.cc:597] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:37.645771  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:37.646512  5352 raft_consensus.cc:2243] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:37.646953  5352 raft_consensus.cc:2272] T 2fedce3e0a6d41aa8954693aab3796cd P cc755c49216847f4a9ead01695cd2ab1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:37.649580  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:37.650163  5352 raft_consensus.cc:2243] T 8085f6a4e929411097ff3c58f115ed80 P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:37.650573  5352 raft_consensus.cc:2272] T 8085f6a4e929411097ff3c58f115ed80 P cc755c49216847f4a9ead01695cd2ab1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:37.653434  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:37.654424  5352 raft_consensus.cc:2243] T 7101b7cd32744ca79f6c50b5e31f246a P cc755c49216847f4a9ead01695cd2ab1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:37.655089  5352 raft_consensus.cc:2272] T 7101b7cd32744ca79f6c50b5e31f246a P cc755c49216847f4a9ead01695cd2ab1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:37.658412  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:37.659090  5352 raft_consensus.cc:2243] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:37.659575  5352 raft_consensus.cc:2272] T ee29c9afcbb94cadae2778d1f0d2fcd2 P cc755c49216847f4a9ead01695cd2ab1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:37.662024  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:37.662626  5352 raft_consensus.cc:2243] T 09fbbea470544d64b22e97e7f74a305b P cc755c49216847f4a9ead01695cd2ab1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:37.663175  5352 raft_consensus.cc:2272] T 09fbbea470544d64b22e97e7f74a305b P cc755c49216847f4a9ead01695cd2ab1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:37.666386  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:37.666983  5352 raft_consensus.cc:2243] T 3ae7d028bd6548359867825588960bcb P cc755c49216847f4a9ead01695cd2ab1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:37.667469  5352 raft_consensus.cc:2272] T 3ae7d028bd6548359867825588960bcb P cc755c49216847f4a9ead01695cd2ab1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:37.670578  5352 tablet_replica.cc:333] stopping tablet replica
I20260709 12:25:37.671213  5352 raft_consensus.cc:2243] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:37.671854  5352 raft_consensus.cc:2272] T 797d4300bcc7475ba608835b5d9addc2 P cc755c49216847f4a9ead01695cd2ab1 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:37.709317 23796 consensus_peers.cc:597] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:37.711045  5352 tablet_server.cc:196] TabletServer@127.5.58.1:40445 shutdown complete.
W20260709 12:25:37.744923 23870 consensus_peers.cc:597] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:37.747731  5352 tablet_server.cc:179] TabletServer@127.5.58.2:0 shutting down...
W20260709 12:25:37.775111 23796 consensus_peers.cc:597] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396 -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:37.789940  5352 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:25:37.790854  5352 tablet_replica.cc:333] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396: stopping tablet replica
I20260709 12:25:37.791577  5352 raft_consensus.cc:2243] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:25:37.792593  5352 raft_consensus.cc:2272] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 50493539c1954244889d4f3e6d55f396 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:37.795914  5352 tablet_replica.cc:333] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396: stopping tablet replica
I20260709 12:25:37.797646  5352 raft_consensus.cc:2243] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:25:37.798800  5352 raft_consensus.cc:2272] T 2fedce3e0a6d41aa8954693aab3796cd P 50493539c1954244889d4f3e6d55f396 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:37.801710 23870 consensus_peers.cc:597] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:37.802244  5352 tablet_replica.cc:333] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396: stopping tablet replica
I20260709 12:25:37.803260  5352 raft_consensus.cc:2243] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:37.803893  5352 raft_consensus.cc:2272] T 7101b7cd32744ca79f6c50b5e31f246a P 50493539c1954244889d4f3e6d55f396 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:37.806578  5352 tablet_replica.cc:333] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396: stopping tablet replica
I20260709 12:25:37.807449  5352 raft_consensus.cc:2243] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:37.808014  5352 raft_consensus.cc:2272] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 50493539c1954244889d4f3e6d55f396 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:37.810563  5352 tablet_replica.cc:333] T 8085f6a4e929411097ff3c58f115ed80 P 50493539c1954244889d4f3e6d55f396: stopping tablet replica
I20260709 12:25:37.811242  5352 raft_consensus.cc:2243] T 8085f6a4e929411097ff3c58f115ed80 P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:37.811770  5352 raft_consensus.cc:2272] T 8085f6a4e929411097ff3c58f115ed80 P 50493539c1954244889d4f3e6d55f396 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:37.815376  5352 tablet_replica.cc:333] T 3ae7d028bd6548359867825588960bcb P 50493539c1954244889d4f3e6d55f396: stopping tablet replica
I20260709 12:25:37.816419  5352 raft_consensus.cc:2243] T 3ae7d028bd6548359867825588960bcb P 50493539c1954244889d4f3e6d55f396 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:25:37.817324  5352 raft_consensus.cc:2272] T 3ae7d028bd6548359867825588960bcb P 50493539c1954244889d4f3e6d55f396 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:37.821139  5352 tablet_replica.cc:333] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396: stopping tablet replica
I20260709 12:25:37.821923  5352 raft_consensus.cc:2243] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:25:37.822777  5352 raft_consensus.cc:2272] T 797d4300bcc7475ba608835b5d9addc2 P 50493539c1954244889d4f3e6d55f396 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:37.825871  5352 tablet_replica.cc:333] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396: stopping tablet replica
I20260709 12:25:37.826500  5352 raft_consensus.cc:2243] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:25:37.827440  5352 raft_consensus.cc:2272] T 09fbbea470544d64b22e97e7f74a305b P 50493539c1954244889d4f3e6d55f396 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:37.830212  5352 tablet_replica.cc:333] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396: stopping tablet replica
I20260709 12:25:37.830824  5352 raft_consensus.cc:2243] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:37.831283  5352 raft_consensus.cc:2272] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 50493539c1954244889d4f3e6d55f396 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:37.833562  5352 tablet_replica.cc:333] T e5a6d6eb60bb40cd83ba7787fc127421 P 50493539c1954244889d4f3e6d55f396: stopping tablet replica
I20260709 12:25:37.834101  5352 raft_consensus.cc:2243] T e5a6d6eb60bb40cd83ba7787fc127421 P 50493539c1954244889d4f3e6d55f396 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:25:37.834908  5352 raft_consensus.cc:2272] T e5a6d6eb60bb40cd83ba7787fc127421 P 50493539c1954244889d4f3e6d55f396 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:37.870430 23869 consensus_peers.cc:597] T 7101b7cd32744ca79f6c50b5e31f246a P 565673cccffc4dbbb2d7431552ba280f -> Peer 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837): Couldn't send request to peer 50493539c1954244889d4f3e6d55f396. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:38837: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:37.871145  5352 tablet_server.cc:196] TabletServer@127.5.58.2:0 shutdown complete.
W20260709 12:25:37.876199 23869 consensus_peers.cc:597] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f -> Peer 50493539c1954244889d4f3e6d55f396 (127.5.58.2:38837): Couldn't send request to peer 50493539c1954244889d4f3e6d55f396. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:38837: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:37.900019  5352 tablet_server.cc:179] TabletServer@127.5.58.3:0 shutting down...
I20260709 12:25:37.941591  5352 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:25:37.942256 23870 consensus_peers.cc:597] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f -> Peer cc755c49216847f4a9ead01695cd2ab1 (127.5.58.1:40445): Couldn't send request to peer cc755c49216847f4a9ead01695cd2ab1. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:40445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:37.943027  5352 tablet_replica.cc:333] T 7101b7cd32744ca79f6c50b5e31f246a P 565673cccffc4dbbb2d7431552ba280f: stopping tablet replica
I20260709 12:25:37.943900  5352 raft_consensus.cc:2243] T 7101b7cd32744ca79f6c50b5e31f246a P 565673cccffc4dbbb2d7431552ba280f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:25:37.944862  5352 raft_consensus.cc:2272] T 7101b7cd32744ca79f6c50b5e31f246a P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:37.947326  5352 tablet_replica.cc:333] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f: stopping tablet replica
I20260709 12:25:37.948283  5352 raft_consensus.cc:2243] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:25:37.949259  5352 raft_consensus.cc:2272] T ee29c9afcbb94cadae2778d1f0d2fcd2 P 565673cccffc4dbbb2d7431552ba280f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:37.951581  5352 tablet_replica.cc:333] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f: stopping tablet replica
I20260709 12:25:37.952167  5352 raft_consensus.cc:2243] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:25:37.953140  5352 raft_consensus.cc:2272] T 8085f6a4e929411097ff3c58f115ed80 P 565673cccffc4dbbb2d7431552ba280f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:37.955500  5352 tablet_replica.cc:333] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f: stopping tablet replica
I20260709 12:25:37.956107  5352 raft_consensus.cc:2243] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:25:37.956992  5352 raft_consensus.cc:2272] T 5bcd8bf7d5d04504b30ddbb04f6bc7fd P 565673cccffc4dbbb2d7431552ba280f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:37.959499  5352 tablet_replica.cc:333] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f: stopping tablet replica
I20260709 12:25:37.960155  5352 raft_consensus.cc:2243] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:37.960647  5352 raft_consensus.cc:2272] T e5a6d6eb60bb40cd83ba7787fc127421 P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:37.963095  5352 tablet_replica.cc:333] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f: stopping tablet replica
I20260709 12:25:37.963728  5352 raft_consensus.cc:2243] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:37.964417  5352 raft_consensus.cc:2272] T 63fdfbf5f20f4c1ea9db0d4707c4861b P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:37.966881  5352 tablet_replica.cc:333] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f: stopping tablet replica
I20260709 12:25:37.967648  5352 raft_consensus.cc:2243] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:37.968108  5352 raft_consensus.cc:2272] T 2fedce3e0a6d41aa8954693aab3796cd P 565673cccffc4dbbb2d7431552ba280f [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:37.968724 23700 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:25:37.970475  5352 tablet_replica.cc:333] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f: stopping tablet replica
I20260709 12:25:37.971127  5352 raft_consensus.cc:2243] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:37.971724  5352 raft_consensus.cc:2272] T 09fbbea470544d64b22e97e7f74a305b P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:37.973873  5352 tablet_replica.cc:333] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f: stopping tablet replica
I20260709 12:25:37.974402  5352 raft_consensus.cc:2243] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:37.974819  5352 raft_consensus.cc:2272] T 797d4300bcc7475ba608835b5d9addc2 P 565673cccffc4dbbb2d7431552ba280f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:37.978250  5352 tablet_replica.cc:333] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f: stopping tablet replica
I20260709 12:25:37.979009  5352 raft_consensus.cc:2243] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:37.979835  5352 raft_consensus.cc:2272] T 3ae7d028bd6548359867825588960bcb P 565673cccffc4dbbb2d7431552ba280f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:38.012526  5352 tablet_server.cc:196] TabletServer@127.5.58.3:0 shutdown complete.
I20260709 12:25:38.043854  5352 master.cc:562] Master@127.5.58.62:43883 shutting down...
I20260709 12:25:38.071017  5352 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 7fc82a5999f34967bbc914905d87549d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:25:38.071668  5352 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 7fc82a5999f34967bbc914905d87549d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:38.072007  5352 tablet_replica.cc:333] T 00000000000000000000000000000000 P 7fc82a5999f34967bbc914905d87549d: stopping tablet replica
I20260709 12:25:38.094305  5352 master.cc:584] Master@127.5.58.62:43883 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (13411 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 12:25:38.143888  5352 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.58.62:41707
I20260709 12:25:38.145143  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:25:38.153153 24246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:25:38.153205 24248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:25:38.153602  5352 server_base.cc:1061] running on GCE node
W20260709 12:25:38.153306 24245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:25:38.155153  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:25:38.155339  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:25:38.155452  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599938155436 us; error 0 us; skew 500 ppm
I20260709 12:25:38.155905  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:25:38.158305  5352 webserver.cc:533] Webserver started at http://127.5.58.62:34289/ using document root <none> and password file <none>
I20260709 12:25:38.158764  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:25:38.158927  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:25:38.159143  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:25:38.160224  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599341860723-5352-0/minicluster-data/master-0-root/instance:
uuid: "520ce73ea270491d8da29cfdc2c6cfe5"
format_stamp: "Formatted at 2026-07-09 12:25:38 on dist-test-slave-xs2d"
I20260709 12:25:38.164548  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 12:25:38.168045 24253 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:25:38.168967  5352 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:25:38.169284  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599341860723-5352-0/minicluster-data/master-0-root
uuid: "520ce73ea270491d8da29cfdc2c6cfe5"
format_stamp: "Formatted at 2026-07-09 12:25:38 on dist-test-slave-xs2d"
I20260709 12:25:38.169543  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599341860723-5352-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599341860723-5352-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599341860723-5352-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:25:38.194629  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:25:38.196017  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:25:38.242436  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.62:41707
I20260709 12:25:38.242549 24304 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.62:41707 every 8 connection(s)
I20260709 12:25:38.246946 24305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:38.259117 24305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 520ce73ea270491d8da29cfdc2c6cfe5: Bootstrap starting.
I20260709 12:25:38.264547 24305 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 520ce73ea270491d8da29cfdc2c6cfe5: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:38.269398 24305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 520ce73ea270491d8da29cfdc2c6cfe5: No bootstrap required, opened a new log
I20260709 12:25:38.271798 24305 raft_consensus.cc:359] T 00000000000000000000000000000000 P 520ce73ea270491d8da29cfdc2c6cfe5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "520ce73ea270491d8da29cfdc2c6cfe5" member_type: VOTER }
I20260709 12:25:38.272182 24305 raft_consensus.cc:385] T 00000000000000000000000000000000 P 520ce73ea270491d8da29cfdc2c6cfe5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:38.272392 24305 raft_consensus.cc:740] T 00000000000000000000000000000000 P 520ce73ea270491d8da29cfdc2c6cfe5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 520ce73ea270491d8da29cfdc2c6cfe5, State: Initialized, Role: FOLLOWER
I20260709 12:25:38.272961 24305 consensus_queue.cc:260] T 00000000000000000000000000000000 P 520ce73ea270491d8da29cfdc2c6cfe5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "520ce73ea270491d8da29cfdc2c6cfe5" member_type: VOTER }
I20260709 12:25:38.273360 24305 raft_consensus.cc:399] T 00000000000000000000000000000000 P 520ce73ea270491d8da29cfdc2c6cfe5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:25:38.273557 24305 raft_consensus.cc:493] T 00000000000000000000000000000000 P 520ce73ea270491d8da29cfdc2c6cfe5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:25:38.273773 24305 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 520ce73ea270491d8da29cfdc2c6cfe5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:38.278724 24305 raft_consensus.cc:515] T 00000000000000000000000000000000 P 520ce73ea270491d8da29cfdc2c6cfe5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "520ce73ea270491d8da29cfdc2c6cfe5" member_type: VOTER }
I20260709 12:25:38.279209 24305 leader_election.cc:304] T 00000000000000000000000000000000 P 520ce73ea270491d8da29cfdc2c6cfe5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 520ce73ea270491d8da29cfdc2c6cfe5; no voters: 
I20260709 12:25:38.281086 24305 leader_election.cc:290] T 00000000000000000000000000000000 P 520ce73ea270491d8da29cfdc2c6cfe5 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:25:38.281569 24308 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 520ce73ea270491d8da29cfdc2c6cfe5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:38.283420 24308 raft_consensus.cc:697] T 00000000000000000000000000000000 P 520ce73ea270491d8da29cfdc2c6cfe5 [term 1 LEADER]: Becoming Leader. State: Replica: 520ce73ea270491d8da29cfdc2c6cfe5, State: Running, Role: LEADER
I20260709 12:25:38.284162 24308 consensus_queue.cc:237] T 00000000000000000000000000000000 P 520ce73ea270491d8da29cfdc2c6cfe5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "520ce73ea270491d8da29cfdc2c6cfe5" member_type: VOTER }
I20260709 12:25:38.284873 24305 sys_catalog.cc:565] T 00000000000000000000000000000000 P 520ce73ea270491d8da29cfdc2c6cfe5 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:25:38.289358 24310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 520ce73ea270491d8da29cfdc2c6cfe5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 520ce73ea270491d8da29cfdc2c6cfe5. Latest consensus state: current_term: 1 leader_uuid: "520ce73ea270491d8da29cfdc2c6cfe5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "520ce73ea270491d8da29cfdc2c6cfe5" member_type: VOTER } }
I20260709 12:25:38.290237 24310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 520ce73ea270491d8da29cfdc2c6cfe5 [sys.catalog]: This master's current role is: LEADER
I20260709 12:25:38.289939 24309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 520ce73ea270491d8da29cfdc2c6cfe5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "520ce73ea270491d8da29cfdc2c6cfe5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "520ce73ea270491d8da29cfdc2c6cfe5" member_type: VOTER } }
I20260709 12:25:38.290880 24309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 520ce73ea270491d8da29cfdc2c6cfe5 [sys.catalog]: This master's current role is: LEADER
I20260709 12:25:38.294042 24316 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:25:38.299809 24316 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:25:38.301761  5352 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:25:38.309690 24316 catalog_manager.cc:1383] Generated new cluster ID: 6c7b08f83fdf499699dc5e546d3d3a19
I20260709 12:25:38.310006 24316 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:25:38.329636 24316 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:25:38.331017 24316 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:25:38.343035 24316 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 520ce73ea270491d8da29cfdc2c6cfe5: Generated new TSK 0
I20260709 12:25:38.343766 24316 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:25:38.368573  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:25:38.375670 24326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:25:38.376147 24327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:25:38.380028 24329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:25:38.380054  5352 server_base.cc:1061] running on GCE node
I20260709 12:25:38.381664  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:25:38.381870  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:25:38.382122  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599938382084 us; error 0 us; skew 500 ppm
I20260709 12:25:38.382740  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:25:38.385856  5352 webserver.cc:533] Webserver started at http://127.5.58.1:33451/ using document root <none> and password file <none>
I20260709 12:25:38.386409  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:25:38.386617  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:25:38.386930  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:25:38.388125  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599341860723-5352-0/minicluster-data/ts-0-root/instance:
uuid: "464091b79ebd46ab97acf1f8605de50c"
format_stamp: "Formatted at 2026-07-09 12:25:38 on dist-test-slave-xs2d"
I20260709 12:25:38.393342  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.001s
I20260709 12:25:38.397723 24334 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:25:38.398800  5352 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:25:38.399221  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599341860723-5352-0/minicluster-data/ts-0-root
uuid: "464091b79ebd46ab97acf1f8605de50c"
format_stamp: "Formatted at 2026-07-09 12:25:38 on dist-test-slave-xs2d"
I20260709 12:25:38.399545  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599341860723-5352-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599341860723-5352-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599341860723-5352-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:25:38.434509  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:25:38.435688  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:25:38.437279  5352 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:25:38.439877  5352 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:25:38.440126  5352 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:25:38.440505  5352 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:25:38.440783  5352 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:25:38.491544  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.1:43139
I20260709 12:25:38.491726 24396 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.1:43139 every 8 connection(s)
I20260709 12:25:38.497188  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:25:38.505468 24401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:25:38.507884 24402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:25:38.511080  5352 server_base.cc:1061] running on GCE node
W20260709 12:25:38.512217 24404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:25:38.513300  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:25:38.513549  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:25:38.513763  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599938513744 us; error 0 us; skew 500 ppm
I20260709 12:25:38.514253 24397 heartbeater.cc:344] Connected to a master server at 127.5.58.62:41707
I20260709 12:25:38.514400  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:25:38.514801 24397 heartbeater.cc:461] Registering TS with master...
I20260709 12:25:38.515931 24397 heartbeater.cc:507] Master 127.5.58.62:41707 requested a full tablet report, sending...
I20260709 12:25:38.519007 24270 ts_manager.cc:194] Registered new tserver with Master: 464091b79ebd46ab97acf1f8605de50c (127.5.58.1:43139)
I20260709 12:25:38.520082  5352 webserver.cc:533] Webserver started at http://127.5.58.2:44689/ using document root <none> and password file <none>
I20260709 12:25:38.520709  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:25:38.520928  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:25:38.521198  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:25:38.521562 24270 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60232
I20260709 12:25:38.522526  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599341860723-5352-0/minicluster-data/ts-1-root/instance:
uuid: "d27b530f2dca4661b6c3949e255a222b"
format_stamp: "Formatted at 2026-07-09 12:25:38 on dist-test-slave-xs2d"
I20260709 12:25:38.527446  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:25:38.531029 24409 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:25:38.531965  5352 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:25:38.532291  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599341860723-5352-0/minicluster-data/ts-1-root
uuid: "d27b530f2dca4661b6c3949e255a222b"
format_stamp: "Formatted at 2026-07-09 12:25:38 on dist-test-slave-xs2d"
I20260709 12:25:38.532653  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599341860723-5352-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599341860723-5352-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599341860723-5352-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:25:38.545176  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:25:38.546420  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:25:38.547899  5352 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:25:38.550118  5352 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:25:38.550345  5352 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:25:38.550585  5352 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:25:38.550756  5352 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:25:38.593858  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.2:40557
I20260709 12:25:38.593987 24471 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.2:40557 every 8 connection(s)
I20260709 12:25:38.598892  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:25:38.607165 24475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:25:38.610142 24476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:25:38.612142 24478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:25:38.612396  5352 server_base.cc:1061] running on GCE node
I20260709 12:25:38.613425  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:25:38.613579 24472 heartbeater.cc:344] Connected to a master server at 127.5.58.62:41707
W20260709 12:25:38.613770  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:25:38.614008  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599938613989 us; error 0 us; skew 500 ppm
I20260709 12:25:38.614033 24472 heartbeater.cc:461] Registering TS with master...
I20260709 12:25:38.614912  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:25:38.615085 24472 heartbeater.cc:507] Master 127.5.58.62:41707 requested a full tablet report, sending...
I20260709 12:25:38.617590  5352 webserver.cc:533] Webserver started at http://127.5.58.3:44953/ using document root <none> and password file <none>
I20260709 12:25:38.617631 24270 ts_manager.cc:194] Registered new tserver with Master: d27b530f2dca4661b6c3949e255a222b (127.5.58.2:40557)
I20260709 12:25:38.618306  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:25:38.618568  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:25:38.618853  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:25:38.619386 24270 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60246
I20260709 12:25:38.620118  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599341860723-5352-0/minicluster-data/ts-2-root/instance:
uuid: "2a5808db2a594f479695d8a649074c78"
format_stamp: "Formatted at 2026-07-09 12:25:38 on dist-test-slave-xs2d"
I20260709 12:25:38.625584  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.000s	sys 0.004s
I20260709 12:25:38.629412 24483 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:25:38.630298  5352 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.004s	sys 0.000s
I20260709 12:25:38.630688  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599341860723-5352-0/minicluster-data/ts-2-root
uuid: "2a5808db2a594f479695d8a649074c78"
format_stamp: "Formatted at 2026-07-09 12:25:38 on dist-test-slave-xs2d"
I20260709 12:25:38.631034  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599341860723-5352-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599341860723-5352-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599341860723-5352-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:25:38.656757  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:25:38.657966  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:25:38.659504  5352 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:25:38.661798  5352 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:25:38.662029  5352 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:25:38.662273  5352 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:25:38.662458  5352 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:25:38.713616  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.3:34471
I20260709 12:25:38.713729 24545 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.3:34471 every 8 connection(s)
I20260709 12:25:38.728899 24546 heartbeater.cc:344] Connected to a master server at 127.5.58.62:41707
I20260709 12:25:38.729347 24546 heartbeater.cc:461] Registering TS with master...
I20260709 12:25:38.730193 24546 heartbeater.cc:507] Master 127.5.58.62:41707 requested a full tablet report, sending...
I20260709 12:25:38.732446 24270 ts_manager.cc:194] Registered new tserver with Master: 2a5808db2a594f479695d8a649074c78 (127.5.58.3:34471)
I20260709 12:25:38.733552  5352 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016755244s
I20260709 12:25:38.734292 24270 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60258
I20260709 12:25:38.762600 24270 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60268:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:25:38.764860 24270 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:25:38.813494 24362 tablet_service.cc:1511] Processing CreateTablet for tablet e47a9d14818741abb2f13398733b83cc (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=9f71121678964604a6345439e8b2b78b]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:25:38.814167 24361 tablet_service.cc:1511] Processing CreateTablet for tablet cad6990a5f8c4c20b1d4459b5f4c69ce (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=9f71121678964604a6345439e8b2b78b]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:25:38.815040 24362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e47a9d14818741abb2f13398733b83cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:38.815843 24361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cad6990a5f8c4c20b1d4459b5f4c69ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:38.820885 24511 tablet_service.cc:1511] Processing CreateTablet for tablet e47a9d14818741abb2f13398733b83cc (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=9f71121678964604a6345439e8b2b78b]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:25:38.822408 24511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e47a9d14818741abb2f13398733b83cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:38.823798 24510 tablet_service.cc:1511] Processing CreateTablet for tablet cad6990a5f8c4c20b1d4459b5f4c69ce (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=9f71121678964604a6345439e8b2b78b]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:25:38.825229 24510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cad6990a5f8c4c20b1d4459b5f4c69ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:38.842803 24437 tablet_service.cc:1511] Processing CreateTablet for tablet e47a9d14818741abb2f13398733b83cc (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=9f71121678964604a6345439e8b2b78b]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:25:38.844144 24561 tablet_bootstrap.cc:492] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78: Bootstrap starting.
I20260709 12:25:38.844424 24437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e47a9d14818741abb2f13398733b83cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:38.847666 24436 tablet_service.cc:1511] Processing CreateTablet for tablet cad6990a5f8c4c20b1d4459b5f4c69ce (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=9f71121678964604a6345439e8b2b78b]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:25:38.848977 24436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cad6990a5f8c4c20b1d4459b5f4c69ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:38.850466 24560 tablet_bootstrap.cc:492] T cad6990a5f8c4c20b1d4459b5f4c69ce P 464091b79ebd46ab97acf1f8605de50c: Bootstrap starting.
I20260709 12:25:38.856974 24560 tablet_bootstrap.cc:654] T cad6990a5f8c4c20b1d4459b5f4c69ce P 464091b79ebd46ab97acf1f8605de50c: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:38.858436 24561 tablet_bootstrap.cc:654] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:38.868141 24560 tablet_bootstrap.cc:492] T cad6990a5f8c4c20b1d4459b5f4c69ce P 464091b79ebd46ab97acf1f8605de50c: No bootstrap required, opened a new log
I20260709 12:25:38.868665 24560 ts_tablet_manager.cc:1403] T cad6990a5f8c4c20b1d4459b5f4c69ce P 464091b79ebd46ab97acf1f8605de50c: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.006s
I20260709 12:25:38.871763 24560 raft_consensus.cc:359] T cad6990a5f8c4c20b1d4459b5f4c69ce P 464091b79ebd46ab97acf1f8605de50c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:38.873205 24560 raft_consensus.cc:385] T cad6990a5f8c4c20b1d4459b5f4c69ce P 464091b79ebd46ab97acf1f8605de50c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:38.873781 24560 raft_consensus.cc:740] T cad6990a5f8c4c20b1d4459b5f4c69ce P 464091b79ebd46ab97acf1f8605de50c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 464091b79ebd46ab97acf1f8605de50c, State: Initialized, Role: FOLLOWER
I20260709 12:25:38.874985 24561 tablet_bootstrap.cc:492] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78: No bootstrap required, opened a new log
I20260709 12:25:38.874913 24560 consensus_queue.cc:260] T cad6990a5f8c4c20b1d4459b5f4c69ce P 464091b79ebd46ab97acf1f8605de50c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:38.875574 24561 ts_tablet_manager.cc:1403] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78: Time spent bootstrapping tablet: real 0.032s	user 0.014s	sys 0.007s
I20260709 12:25:38.876230 24564 tablet_bootstrap.cc:492] T e47a9d14818741abb2f13398733b83cc P d27b530f2dca4661b6c3949e255a222b: Bootstrap starting.
I20260709 12:25:38.879735 24397 heartbeater.cc:499] Master 127.5.58.62:41707 was elected leader, sending a full tablet report...
I20260709 12:25:38.880580 24560 ts_tablet_manager.cc:1434] T cad6990a5f8c4c20b1d4459b5f4c69ce P 464091b79ebd46ab97acf1f8605de50c: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.005s
I20260709 12:25:38.880148 24561 raft_consensus.cc:359] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } }
I20260709 12:25:38.881125 24561 raft_consensus.cc:385] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:38.881446 24561 raft_consensus.cc:740] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a5808db2a594f479695d8a649074c78, State: Initialized, Role: FOLLOWER
I20260709 12:25:38.882429 24560 tablet_bootstrap.cc:492] T e47a9d14818741abb2f13398733b83cc P 464091b79ebd46ab97acf1f8605de50c: Bootstrap starting.
I20260709 12:25:38.882218 24561 consensus_queue.cc:260] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } }
I20260709 12:25:38.883919 24564 tablet_bootstrap.cc:654] T e47a9d14818741abb2f13398733b83cc P d27b530f2dca4661b6c3949e255a222b: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:38.892249 24560 tablet_bootstrap.cc:654] T e47a9d14818741abb2f13398733b83cc P 464091b79ebd46ab97acf1f8605de50c: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:38.896214 24564 tablet_bootstrap.cc:492] T e47a9d14818741abb2f13398733b83cc P d27b530f2dca4661b6c3949e255a222b: No bootstrap required, opened a new log
I20260709 12:25:38.897435 24564 ts_tablet_manager.cc:1403] T e47a9d14818741abb2f13398733b83cc P d27b530f2dca4661b6c3949e255a222b: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.004s
I20260709 12:25:38.900614 24564 raft_consensus.cc:359] T e47a9d14818741abb2f13398733b83cc P d27b530f2dca4661b6c3949e255a222b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } }
I20260709 12:25:38.901409 24564 raft_consensus.cc:385] T e47a9d14818741abb2f13398733b83cc P d27b530f2dca4661b6c3949e255a222b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:38.901734 24564 raft_consensus.cc:740] T e47a9d14818741abb2f13398733b83cc P d27b530f2dca4661b6c3949e255a222b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d27b530f2dca4661b6c3949e255a222b, State: Initialized, Role: FOLLOWER
I20260709 12:25:38.902640 24564 consensus_queue.cc:260] T e47a9d14818741abb2f13398733b83cc P d27b530f2dca4661b6c3949e255a222b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } }
I20260709 12:25:38.906227 24546 heartbeater.cc:499] Master 127.5.58.62:41707 was elected leader, sending a full tablet report...
I20260709 12:25:38.906518 24561 ts_tablet_manager.cc:1434] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78: Time spent starting tablet: real 0.030s	user 0.019s	sys 0.009s
I20260709 12:25:38.906356 24472 heartbeater.cc:499] Master 127.5.58.62:41707 was elected leader, sending a full tablet report...
I20260709 12:25:38.906893 24564 ts_tablet_manager.cc:1434] T e47a9d14818741abb2f13398733b83cc P d27b530f2dca4661b6c3949e255a222b: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:25:38.908075 24564 tablet_bootstrap.cc:492] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b: Bootstrap starting.
I20260709 12:25:38.908905 24561 tablet_bootstrap.cc:492] T cad6990a5f8c4c20b1d4459b5f4c69ce P 2a5808db2a594f479695d8a649074c78: Bootstrap starting.
I20260709 12:25:38.913269 24560 tablet_bootstrap.cc:492] T e47a9d14818741abb2f13398733b83cc P 464091b79ebd46ab97acf1f8605de50c: No bootstrap required, opened a new log
I20260709 12:25:38.913988 24560 ts_tablet_manager.cc:1403] T e47a9d14818741abb2f13398733b83cc P 464091b79ebd46ab97acf1f8605de50c: Time spent bootstrapping tablet: real 0.032s	user 0.017s	sys 0.007s
I20260709 12:25:38.915560 24561 tablet_bootstrap.cc:654] T cad6990a5f8c4c20b1d4459b5f4c69ce P 2a5808db2a594f479695d8a649074c78: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:38.915977 24564 tablet_bootstrap.cc:654] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:38.917349 24560 raft_consensus.cc:359] T e47a9d14818741abb2f13398733b83cc P 464091b79ebd46ab97acf1f8605de50c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } }
I20260709 12:25:38.918231 24560 raft_consensus.cc:385] T e47a9d14818741abb2f13398733b83cc P 464091b79ebd46ab97acf1f8605de50c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:38.918648 24560 raft_consensus.cc:740] T e47a9d14818741abb2f13398733b83cc P 464091b79ebd46ab97acf1f8605de50c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 464091b79ebd46ab97acf1f8605de50c, State: Initialized, Role: FOLLOWER
I20260709 12:25:38.920300 24560 consensus_queue.cc:260] T e47a9d14818741abb2f13398733b83cc P 464091b79ebd46ab97acf1f8605de50c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } }
I20260709 12:25:38.924160 24560 ts_tablet_manager.cc:1434] T e47a9d14818741abb2f13398733b83cc P 464091b79ebd46ab97acf1f8605de50c: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.003s
I20260709 12:25:38.934257 24564 tablet_bootstrap.cc:492] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b: No bootstrap required, opened a new log
I20260709 12:25:38.934275 24561 tablet_bootstrap.cc:492] T cad6990a5f8c4c20b1d4459b5f4c69ce P 2a5808db2a594f479695d8a649074c78: No bootstrap required, opened a new log
I20260709 12:25:38.935225 24564 ts_tablet_manager.cc:1403] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.014s
I20260709 12:25:38.935694 24561 ts_tablet_manager.cc:1403] T cad6990a5f8c4c20b1d4459b5f4c69ce P 2a5808db2a594f479695d8a649074c78: Time spent bootstrapping tablet: real 0.027s	user 0.016s	sys 0.009s
I20260709 12:25:38.938956 24561 raft_consensus.cc:359] T cad6990a5f8c4c20b1d4459b5f4c69ce P 2a5808db2a594f479695d8a649074c78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:38.939718 24561 raft_consensus.cc:385] T cad6990a5f8c4c20b1d4459b5f4c69ce P 2a5808db2a594f479695d8a649074c78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:38.939432 24564 raft_consensus.cc:359] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:38.940687 24564 raft_consensus.cc:385] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:38.940151 24561 raft_consensus.cc:740] T cad6990a5f8c4c20b1d4459b5f4c69ce P 2a5808db2a594f479695d8a649074c78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a5808db2a594f479695d8a649074c78, State: Initialized, Role: FOLLOWER
I20260709 12:25:38.941146 24564 raft_consensus.cc:740] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d27b530f2dca4661b6c3949e255a222b, State: Initialized, Role: FOLLOWER
I20260709 12:25:38.942099 24564 consensus_queue.cc:260] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:38.942099 24561 consensus_queue.cc:260] T cad6990a5f8c4c20b1d4459b5f4c69ce P 2a5808db2a594f479695d8a649074c78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:38.944864 24564 ts_tablet_manager.cc:1434] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:25:38.945827 24561 ts_tablet_manager.cc:1434] T cad6990a5f8c4c20b1d4459b5f4c69ce P 2a5808db2a594f479695d8a649074c78: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:25:38.991770 24569 raft_consensus.cc:493] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.992403 24569 raft_consensus.cc:515] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:38.995003 24569 leader_election.cc:290] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 464091b79ebd46ab97acf1f8605de50c (127.5.58.1:43139), 2a5808db2a594f479695d8a649074c78 (127.5.58.3:34471)
I20260709 12:25:39.005384 24566 raft_consensus.cc:493] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.006012 24566 raft_consensus.cc:515] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } }
I20260709 12:25:39.007470 24372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cad6990a5f8c4c20b1d4459b5f4c69ce" candidate_uuid: "d27b530f2dca4661b6c3949e255a222b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "464091b79ebd46ab97acf1f8605de50c" is_pre_election: true
I20260709 12:25:39.008265 24372 raft_consensus.cc:2468] T cad6990a5f8c4c20b1d4459b5f4c69ce P 464091b79ebd46ab97acf1f8605de50c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d27b530f2dca4661b6c3949e255a222b in term 0.
I20260709 12:25:39.009882 24566 leader_election.cc:290] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 464091b79ebd46ab97acf1f8605de50c (127.5.58.1:43139), d27b530f2dca4661b6c3949e255a222b (127.5.58.2:40557)
I20260709 12:25:39.010180 24521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cad6990a5f8c4c20b1d4459b5f4c69ce" candidate_uuid: "d27b530f2dca4661b6c3949e255a222b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a5808db2a594f479695d8a649074c78" is_pre_election: true
I20260709 12:25:39.010833 24412 leader_election.cc:304] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 464091b79ebd46ab97acf1f8605de50c, d27b530f2dca4661b6c3949e255a222b; no voters: 
I20260709 12:25:39.011159 24521 raft_consensus.cc:2468] T cad6990a5f8c4c20b1d4459b5f4c69ce P 2a5808db2a594f479695d8a649074c78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d27b530f2dca4661b6c3949e255a222b in term 0.
I20260709 12:25:39.011986 24569 raft_consensus.cc:2804] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:25:39.012771 24569 raft_consensus.cc:493] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.013326 24569 raft_consensus.cc:3060] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:39.023474 24372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e47a9d14818741abb2f13398733b83cc" candidate_uuid: "2a5808db2a594f479695d8a649074c78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "464091b79ebd46ab97acf1f8605de50c" is_pre_election: true
I20260709 12:25:39.024178 24372 raft_consensus.cc:2468] T e47a9d14818741abb2f13398733b83cc P 464091b79ebd46ab97acf1f8605de50c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a5808db2a594f479695d8a649074c78 in term 0.
I20260709 12:25:39.023834 24569 raft_consensus.cc:515] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:39.025791 24486 leader_election.cc:304] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a5808db2a594f479695d8a649074c78, 464091b79ebd46ab97acf1f8605de50c; no voters: 
I20260709 12:25:39.027387 24566 raft_consensus.cc:2804] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:25:39.027803 24566 raft_consensus.cc:493] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.027897 24569 leader_election.cc:290] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b [CANDIDATE]: Term 1 election: Requested vote from peers 464091b79ebd46ab97acf1f8605de50c (127.5.58.1:43139), 2a5808db2a594f479695d8a649074c78 (127.5.58.3:34471)
I20260709 12:25:39.028108 24372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cad6990a5f8c4c20b1d4459b5f4c69ce" candidate_uuid: "d27b530f2dca4661b6c3949e255a222b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "464091b79ebd46ab97acf1f8605de50c"
I20260709 12:25:39.028247 24566 raft_consensus.cc:3060] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:39.028847 24372 raft_consensus.cc:3060] T cad6990a5f8c4c20b1d4459b5f4c69ce P 464091b79ebd46ab97acf1f8605de50c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:39.029630 24521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cad6990a5f8c4c20b1d4459b5f4c69ce" candidate_uuid: "d27b530f2dca4661b6c3949e255a222b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a5808db2a594f479695d8a649074c78"
I20260709 12:25:39.030418 24521 raft_consensus.cc:3060] T cad6990a5f8c4c20b1d4459b5f4c69ce P 2a5808db2a594f479695d8a649074c78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:39.038702 24372 raft_consensus.cc:2468] T cad6990a5f8c4c20b1d4459b5f4c69ce P 464091b79ebd46ab97acf1f8605de50c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d27b530f2dca4661b6c3949e255a222b in term 1.
I20260709 12:25:39.038285 24447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e47a9d14818741abb2f13398733b83cc" candidate_uuid: "2a5808db2a594f479695d8a649074c78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d27b530f2dca4661b6c3949e255a222b" is_pre_election: true
I20260709 12:25:39.039240 24566 raft_consensus.cc:515] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } }
I20260709 12:25:39.039752 24447 raft_consensus.cc:2468] T e47a9d14818741abb2f13398733b83cc P d27b530f2dca4661b6c3949e255a222b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a5808db2a594f479695d8a649074c78 in term 0.
I20260709 12:25:39.040107 24412 leader_election.cc:304] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 464091b79ebd46ab97acf1f8605de50c, d27b530f2dca4661b6c3949e255a222b; no voters: 
I20260709 12:25:39.041198 24569 raft_consensus.cc:2804] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:39.041787 24566 leader_election.cc:290] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [CANDIDATE]: Term 1 election: Requested vote from peers 464091b79ebd46ab97acf1f8605de50c (127.5.58.1:43139), d27b530f2dca4661b6c3949e255a222b (127.5.58.2:40557)
I20260709 12:25:39.042953 24569 raft_consensus.cc:697] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b [term 1 LEADER]: Becoming Leader. State: Replica: d27b530f2dca4661b6c3949e255a222b, State: Running, Role: LEADER
I20260709 12:25:39.043545 24447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e47a9d14818741abb2f13398733b83cc" candidate_uuid: "2a5808db2a594f479695d8a649074c78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d27b530f2dca4661b6c3949e255a222b"
I20260709 12:25:39.044265 24447 raft_consensus.cc:3060] T e47a9d14818741abb2f13398733b83cc P d27b530f2dca4661b6c3949e255a222b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:39.044763 24372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e47a9d14818741abb2f13398733b83cc" candidate_uuid: "2a5808db2a594f479695d8a649074c78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "464091b79ebd46ab97acf1f8605de50c"
I20260709 12:25:39.044194 24569 consensus_queue.cc:237] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:39.045454 24372 raft_consensus.cc:3060] T e47a9d14818741abb2f13398733b83cc P 464091b79ebd46ab97acf1f8605de50c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:39.045994 24521 raft_consensus.cc:2468] T cad6990a5f8c4c20b1d4459b5f4c69ce P 2a5808db2a594f479695d8a649074c78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d27b530f2dca4661b6c3949e255a222b in term 1.
I20260709 12:25:39.053367 24447 raft_consensus.cc:2468] T e47a9d14818741abb2f13398733b83cc P d27b530f2dca4661b6c3949e255a222b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a5808db2a594f479695d8a649074c78 in term 1.
I20260709 12:25:39.054781 24372 raft_consensus.cc:2468] T e47a9d14818741abb2f13398733b83cc P 464091b79ebd46ab97acf1f8605de50c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a5808db2a594f479695d8a649074c78 in term 1.
I20260709 12:25:39.054776 24486 leader_election.cc:304] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a5808db2a594f479695d8a649074c78, d27b530f2dca4661b6c3949e255a222b; no voters: 
I20260709 12:25:39.055851 24566 raft_consensus.cc:2804] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:39.057322 24566 raft_consensus.cc:697] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [term 1 LEADER]: Becoming Leader. State: Replica: 2a5808db2a594f479695d8a649074c78, State: Running, Role: LEADER
I20260709 12:25:39.058300 24566 consensus_queue.cc:237] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } }
I20260709 12:25:39.058079 24268 catalog_manager.cc:5697] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b reported cstate change: term changed from 0 to 1, leader changed from <none> to d27b530f2dca4661b6c3949e255a222b (127.5.58.2). New cstate: current_term: 1 leader_uuid: "d27b530f2dca4661b6c3949e255a222b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:39.067727 24267 catalog_manager.cc:5697] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a5808db2a594f479695d8a649074c78 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "2a5808db2a594f479695d8a649074c78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:39.119803 24267 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60280:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:25:39.122350 24267 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:25:39.152933 24362 tablet_service.cc:1511] Processing CreateTablet for tablet 66bf786ed4d245398e9645dfa35b056f (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=15b994f4198e4ba7b64685178d6f2971]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:25:39.154467 24362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66bf786ed4d245398e9645dfa35b056f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:39.155426 24437 tablet_service.cc:1511] Processing CreateTablet for tablet 66bf786ed4d245398e9645dfa35b056f (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=15b994f4198e4ba7b64685178d6f2971]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:25:39.156898 24437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66bf786ed4d245398e9645dfa35b056f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:39.158390 24510 tablet_service.cc:1511] Processing CreateTablet for tablet 66bf786ed4d245398e9645dfa35b056f (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=15b994f4198e4ba7b64685178d6f2971]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:25:39.161695 24510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66bf786ed4d245398e9645dfa35b056f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:39.163883 24361 tablet_service.cc:1511] Processing CreateTablet for tablet 8d60fa948011446da3fd7cf40a6cfc10 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=15b994f4198e4ba7b64685178d6f2971]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:25:39.165760 24361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d60fa948011446da3fd7cf40a6cfc10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:39.167802 24436 tablet_service.cc:1511] Processing CreateTablet for tablet 8d60fa948011446da3fd7cf40a6cfc10 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=15b994f4198e4ba7b64685178d6f2971]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:25:39.168561 24511 tablet_service.cc:1511] Processing CreateTablet for tablet 8d60fa948011446da3fd7cf40a6cfc10 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=15b994f4198e4ba7b64685178d6f2971]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:25:39.169229 24436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d60fa948011446da3fd7cf40a6cfc10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:39.169804 24511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d60fa948011446da3fd7cf40a6cfc10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:39.184948 24561 tablet_bootstrap.cc:492] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78: Bootstrap starting.
I20260709 12:25:39.191874 24564 tablet_bootstrap.cc:492] T 66bf786ed4d245398e9645dfa35b056f P d27b530f2dca4661b6c3949e255a222b: Bootstrap starting.
I20260709 12:25:39.193799 24560 tablet_bootstrap.cc:492] T 8d60fa948011446da3fd7cf40a6cfc10 P 464091b79ebd46ab97acf1f8605de50c: Bootstrap starting.
I20260709 12:25:39.193948 24561 tablet_bootstrap.cc:654] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:39.202023 24560 tablet_bootstrap.cc:654] T 8d60fa948011446da3fd7cf40a6cfc10 P 464091b79ebd46ab97acf1f8605de50c: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:39.208705 24564 tablet_bootstrap.cc:654] T 66bf786ed4d245398e9645dfa35b056f P d27b530f2dca4661b6c3949e255a222b: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:39.210584 24561 tablet_bootstrap.cc:492] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78: No bootstrap required, opened a new log
I20260709 12:25:39.211074 24560 tablet_bootstrap.cc:492] T 8d60fa948011446da3fd7cf40a6cfc10 P 464091b79ebd46ab97acf1f8605de50c: No bootstrap required, opened a new log
I20260709 12:25:39.211233 24561 ts_tablet_manager.cc:1403] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78: Time spent bootstrapping tablet: real 0.027s	user 0.008s	sys 0.009s
I20260709 12:25:39.211611 24560 ts_tablet_manager.cc:1403] T 8d60fa948011446da3fd7cf40a6cfc10 P 464091b79ebd46ab97acf1f8605de50c: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.003s
I20260709 12:25:39.215013 24561 raft_consensus.cc:359] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:39.215919 24561 raft_consensus.cc:385] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:39.216243 24561 raft_consensus.cc:740] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a5808db2a594f479695d8a649074c78, State: Initialized, Role: FOLLOWER
I20260709 12:25:39.216581 24560 raft_consensus.cc:359] T 8d60fa948011446da3fd7cf40a6cfc10 P 464091b79ebd46ab97acf1f8605de50c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:39.217406 24560 raft_consensus.cc:385] T 8d60fa948011446da3fd7cf40a6cfc10 P 464091b79ebd46ab97acf1f8605de50c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:39.217945 24560 raft_consensus.cc:740] T 8d60fa948011446da3fd7cf40a6cfc10 P 464091b79ebd46ab97acf1f8605de50c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 464091b79ebd46ab97acf1f8605de50c, State: Initialized, Role: FOLLOWER
I20260709 12:25:39.217332 24561 consensus_queue.cc:260] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:39.218909 24560 consensus_queue.cc:260] T 8d60fa948011446da3fd7cf40a6cfc10 P 464091b79ebd46ab97acf1f8605de50c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:39.219525 24564 tablet_bootstrap.cc:492] T 66bf786ed4d245398e9645dfa35b056f P d27b530f2dca4661b6c3949e255a222b: No bootstrap required, opened a new log
I20260709 12:25:39.220152 24564 ts_tablet_manager.cc:1403] T 66bf786ed4d245398e9645dfa35b056f P d27b530f2dca4661b6c3949e255a222b: Time spent bootstrapping tablet: real 0.029s	user 0.011s	sys 0.006s
I20260709 12:25:39.221518 24560 ts_tablet_manager.cc:1434] T 8d60fa948011446da3fd7cf40a6cfc10 P 464091b79ebd46ab97acf1f8605de50c: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.001s
I20260709 12:25:39.222719 24560 tablet_bootstrap.cc:492] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c: Bootstrap starting.
I20260709 12:25:39.223798 24564 raft_consensus.cc:359] T 66bf786ed4d245398e9645dfa35b056f P d27b530f2dca4661b6c3949e255a222b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:39.224686 24564 raft_consensus.cc:385] T 66bf786ed4d245398e9645dfa35b056f P d27b530f2dca4661b6c3949e255a222b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:39.225107 24564 raft_consensus.cc:740] T 66bf786ed4d245398e9645dfa35b056f P d27b530f2dca4661b6c3949e255a222b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d27b530f2dca4661b6c3949e255a222b, State: Initialized, Role: FOLLOWER
I20260709 12:25:39.226027 24564 consensus_queue.cc:260] T 66bf786ed4d245398e9645dfa35b056f P d27b530f2dca4661b6c3949e255a222b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:39.230479 24561 ts_tablet_manager.cc:1434] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78: Time spent starting tablet: real 0.019s	user 0.002s	sys 0.009s
I20260709 12:25:39.230639 24564 ts_tablet_manager.cc:1434] T 66bf786ed4d245398e9645dfa35b056f P d27b530f2dca4661b6c3949e255a222b: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 12:25:39.231786 24561 tablet_bootstrap.cc:492] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78: Bootstrap starting.
I20260709 12:25:39.231940 24564 tablet_bootstrap.cc:492] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b: Bootstrap starting.
I20260709 12:25:39.239800 24561 tablet_bootstrap.cc:654] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:39.241230 24564 tablet_bootstrap.cc:654] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:39.244045 24560 tablet_bootstrap.cc:654] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:39.253646 24561 tablet_bootstrap.cc:492] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78: No bootstrap required, opened a new log
I20260709 12:25:39.254186 24561 ts_tablet_manager.cc:1403] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78: Time spent bootstrapping tablet: real 0.023s	user 0.004s	sys 0.015s
I20260709 12:25:39.255354 24560 tablet_bootstrap.cc:492] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c: No bootstrap required, opened a new log
I20260709 12:25:39.256108 24560 ts_tablet_manager.cc:1403] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c: Time spent bootstrapping tablet: real 0.034s	user 0.014s	sys 0.004s
I20260709 12:25:39.257861 24561 raft_consensus.cc:359] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:39.258751 24561 raft_consensus.cc:385] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:39.259104 24561 raft_consensus.cc:740] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a5808db2a594f479695d8a649074c78, State: Initialized, Role: FOLLOWER
I20260709 12:25:39.259861 24560 raft_consensus.cc:359] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:39.260154 24561 consensus_queue.cc:260] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:39.260823 24560 raft_consensus.cc:385] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:39.261174 24560 raft_consensus.cc:740] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 464091b79ebd46ab97acf1f8605de50c, State: Initialized, Role: FOLLOWER
I20260709 12:25:39.262109 24564 tablet_bootstrap.cc:492] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b: No bootstrap required, opened a new log
I20260709 12:25:39.262034 24560 consensus_queue.cc:260] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:39.262735 24564 ts_tablet_manager.cc:1403] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b: Time spent bootstrapping tablet: real 0.031s	user 0.016s	sys 0.012s
I20260709 12:25:39.263034 24561 ts_tablet_manager.cc:1434] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:25:39.265522 24560 ts_tablet_manager.cc:1434] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:25:39.266367 24564 raft_consensus.cc:359] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:39.267261 24564 raft_consensus.cc:385] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:39.267606 24564 raft_consensus.cc:740] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d27b530f2dca4661b6c3949e255a222b, State: Initialized, Role: FOLLOWER
I20260709 12:25:39.268640 24564 consensus_queue.cc:260] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:39.271211 24564 ts_tablet_manager.cc:1434] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
W20260709 12:25:39.296540 24318 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:25:39.343534 24569 raft_consensus.cc:493] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.344218 24569 raft_consensus.cc:515] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:39.346210 24569 leader_election.cc:290] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 464091b79ebd46ab97acf1f8605de50c (127.5.58.1:43139), 2a5808db2a594f479695d8a649074c78 (127.5.58.3:34471)
I20260709 12:25:39.347097 24372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d60fa948011446da3fd7cf40a6cfc10" candidate_uuid: "d27b530f2dca4661b6c3949e255a222b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "464091b79ebd46ab97acf1f8605de50c" is_pre_election: true
I20260709 12:25:39.347306 24521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d60fa948011446da3fd7cf40a6cfc10" candidate_uuid: "d27b530f2dca4661b6c3949e255a222b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a5808db2a594f479695d8a649074c78" is_pre_election: true
I20260709 12:25:39.347733 24372 raft_consensus.cc:2468] T 8d60fa948011446da3fd7cf40a6cfc10 P 464091b79ebd46ab97acf1f8605de50c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d27b530f2dca4661b6c3949e255a222b in term 0.
I20260709 12:25:39.348021 24521 raft_consensus.cc:2468] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d27b530f2dca4661b6c3949e255a222b in term 0.
I20260709 12:25:39.348949 24412 leader_election.cc:304] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 464091b79ebd46ab97acf1f8605de50c, d27b530f2dca4661b6c3949e255a222b; no voters: 
I20260709 12:25:39.349997 24569 raft_consensus.cc:2804] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:25:39.350417 24569 raft_consensus.cc:493] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.351132 24569 raft_consensus.cc:3060] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:39.360232 24569 raft_consensus.cc:515] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:39.362249 24569 leader_election.cc:290] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [CANDIDATE]: Term 1 election: Requested vote from peers 464091b79ebd46ab97acf1f8605de50c (127.5.58.1:43139), 2a5808db2a594f479695d8a649074c78 (127.5.58.3:34471)
I20260709 12:25:39.363235 24372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d60fa948011446da3fd7cf40a6cfc10" candidate_uuid: "d27b530f2dca4661b6c3949e255a222b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "464091b79ebd46ab97acf1f8605de50c"
I20260709 12:25:39.363530 24521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d60fa948011446da3fd7cf40a6cfc10" candidate_uuid: "d27b530f2dca4661b6c3949e255a222b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a5808db2a594f479695d8a649074c78"
I20260709 12:25:39.363823 24372 raft_consensus.cc:3060] T 8d60fa948011446da3fd7cf40a6cfc10 P 464091b79ebd46ab97acf1f8605de50c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:39.364179 24521 raft_consensus.cc:3060] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:39.369242 24372 raft_consensus.cc:2468] T 8d60fa948011446da3fd7cf40a6cfc10 P 464091b79ebd46ab97acf1f8605de50c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d27b530f2dca4661b6c3949e255a222b in term 1.
I20260709 12:25:39.369242 24521 raft_consensus.cc:2468] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d27b530f2dca4661b6c3949e255a222b in term 1.
I20260709 12:25:39.370539 24412 leader_election.cc:304] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 464091b79ebd46ab97acf1f8605de50c, d27b530f2dca4661b6c3949e255a222b; no voters: 
I20260709 12:25:39.371513 24569 raft_consensus.cc:2804] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:39.372033 24569 raft_consensus.cc:697] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [term 1 LEADER]: Becoming Leader. State: Replica: d27b530f2dca4661b6c3949e255a222b, State: Running, Role: LEADER
I20260709 12:25:39.372831 24569 consensus_queue.cc:237] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:39.380743 24268 catalog_manager.cc:5697] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b reported cstate change: term changed from 0 to 1, leader changed from <none> to d27b530f2dca4661b6c3949e255a222b (127.5.58.2). New cstate: current_term: 1 leader_uuid: "d27b530f2dca4661b6c3949e255a222b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:39.463562 24569 consensus_queue.cc:1048] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:39.479005 24569 consensus_queue.cc:1048] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b [LEADER]: Connected to new peer: Peer: permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:39.499058 24566 consensus_queue.cc:1048] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:39.509891 24576 raft_consensus.cc:493] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.510455 24576 raft_consensus.cc:515] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:39.512472 24576 leader_election.cc:290] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 464091b79ebd46ab97acf1f8605de50c (127.5.58.1:43139), d27b530f2dca4661b6c3949e255a222b (127.5.58.2:40557)
I20260709 12:25:39.516053 24372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66bf786ed4d245398e9645dfa35b056f" candidate_uuid: "2a5808db2a594f479695d8a649074c78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "464091b79ebd46ab97acf1f8605de50c" is_pre_election: true
I20260709 12:25:39.516677 24576 consensus_queue.cc:1048] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:25:39.516893 24372 raft_consensus.cc:2468] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a5808db2a594f479695d8a649074c78 in term 0.
I20260709 12:25:39.518493 24447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66bf786ed4d245398e9645dfa35b056f" candidate_uuid: "2a5808db2a594f479695d8a649074c78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d27b530f2dca4661b6c3949e255a222b" is_pre_election: true
I20260709 12:25:39.519556 24447 raft_consensus.cc:2468] T 66bf786ed4d245398e9645dfa35b056f P d27b530f2dca4661b6c3949e255a222b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a5808db2a594f479695d8a649074c78 in term 0.
I20260709 12:25:39.519371 24486 leader_election.cc:304] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a5808db2a594f479695d8a649074c78, 464091b79ebd46ab97acf1f8605de50c; no voters: 
I20260709 12:25:39.522497 24596 raft_consensus.cc:2804] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:25:39.523062 24596 raft_consensus.cc:493] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.523531 24596 raft_consensus.cc:3060] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:39.533036 24596 raft_consensus.cc:515] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:39.535434 24596 leader_election.cc:290] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [CANDIDATE]: Term 1 election: Requested vote from peers 464091b79ebd46ab97acf1f8605de50c (127.5.58.1:43139), d27b530f2dca4661b6c3949e255a222b (127.5.58.2:40557)
I20260709 12:25:39.536319 24372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66bf786ed4d245398e9645dfa35b056f" candidate_uuid: "2a5808db2a594f479695d8a649074c78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "464091b79ebd46ab97acf1f8605de50c"
I20260709 12:25:39.537185 24372 raft_consensus.cc:3060] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:39.537413 24447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66bf786ed4d245398e9645dfa35b056f" candidate_uuid: "2a5808db2a594f479695d8a649074c78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d27b530f2dca4661b6c3949e255a222b"
I20260709 12:25:39.538115 24447 raft_consensus.cc:3060] T 66bf786ed4d245398e9645dfa35b056f P d27b530f2dca4661b6c3949e255a222b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:39.545984 24372 raft_consensus.cc:2468] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a5808db2a594f479695d8a649074c78 in term 1.
I20260709 12:25:39.546226 24447 raft_consensus.cc:2468] T 66bf786ed4d245398e9645dfa35b056f P d27b530f2dca4661b6c3949e255a222b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a5808db2a594f479695d8a649074c78 in term 1.
I20260709 12:25:39.547636 24486 leader_election.cc:304] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a5808db2a594f479695d8a649074c78, 464091b79ebd46ab97acf1f8605de50c; no voters: 
I20260709 12:25:39.548861 24596 raft_consensus.cc:2804] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:39.549692 24596 raft_consensus.cc:697] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [term 1 LEADER]: Becoming Leader. State: Replica: 2a5808db2a594f479695d8a649074c78, State: Running, Role: LEADER
I20260709 12:25:39.550467 24596 consensus_queue.cc:237] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:39.558322 24266 catalog_manager.cc:5697] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a5808db2a594f479695d8a649074c78 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "2a5808db2a594f479695d8a649074c78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:39.620772 24521 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e47a9d14818741abb2f13398733b83cc"
dest_uuid: "2a5808db2a594f479695d8a649074c78"
mode: GRACEFUL
new_leader_uuid: "464091b79ebd46ab97acf1f8605de50c"
 from {username='slave'} at 127.0.0.1:33404
I20260709 12:25:39.621662 24521 raft_consensus.cc:606] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [term 1 LEADER]: Received request to transfer leadership to TS 464091b79ebd46ab97acf1f8605de50c
I20260709 12:25:39.638263 24447 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cad6990a5f8c4c20b1d4459b5f4c69ce"
dest_uuid: "d27b530f2dca4661b6c3949e255a222b"
mode: GRACEFUL
new_leader_uuid: "2a5808db2a594f479695d8a649074c78"
 from {username='slave'} at 127.0.0.1:37850
I20260709 12:25:39.638784 24447 raft_consensus.cc:606] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b [term 1 LEADER]: Received request to transfer leadership to TS 2a5808db2a594f479695d8a649074c78
I20260709 12:25:39.646013 24521 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "66bf786ed4d245398e9645dfa35b056f"
dest_uuid: "2a5808db2a594f479695d8a649074c78"
mode: GRACEFUL
new_leader_uuid: "464091b79ebd46ab97acf1f8605de50c"
 from {username='slave'} at 127.0.0.1:33404
I20260709 12:25:39.646631 24521 raft_consensus.cc:606] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [term 1 LEADER]: Received request to transfer leadership to TS 464091b79ebd46ab97acf1f8605de50c
I20260709 12:25:39.653254 24447 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8d60fa948011446da3fd7cf40a6cfc10"
dest_uuid: "d27b530f2dca4661b6c3949e255a222b"
mode: GRACEFUL
new_leader_uuid: "464091b79ebd46ab97acf1f8605de50c"
 from {username='slave'} at 127.0.0.1:37850
I20260709 12:25:39.653905 24447 raft_consensus.cc:606] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [term 1 LEADER]: Received request to transfer leadership to TS 464091b79ebd46ab97acf1f8605de50c
I20260709 12:25:39.816587 24569 consensus_queue.cc:1048] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [LEADER]: Connected to new peer: Peer: permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:25:39.826568 24574 raft_consensus.cc:993] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b: : Instructing follower 464091b79ebd46ab97acf1f8605de50c to start an election
I20260709 12:25:39.827697 24569 raft_consensus.cc:1081] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [term 1 LEADER]: Signalling peer 464091b79ebd46ab97acf1f8605de50c to start an election
I20260709 12:25:39.836615 24372 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8d60fa948011446da3fd7cf40a6cfc10"
dest_uuid: "464091b79ebd46ab97acf1f8605de50c"
 from {username='slave'} at 127.0.0.1:43544
I20260709 12:25:39.837314 24372 raft_consensus.cc:493] T 8d60fa948011446da3fd7cf40a6cfc10 P 464091b79ebd46ab97acf1f8605de50c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:39.837715 24372 raft_consensus.cc:3060] T 8d60fa948011446da3fd7cf40a6cfc10 P 464091b79ebd46ab97acf1f8605de50c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:39.843408 24372 raft_consensus.cc:515] T 8d60fa948011446da3fd7cf40a6cfc10 P 464091b79ebd46ab97acf1f8605de50c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:39.844295 24574 consensus_queue.cc:1048] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:25:39.851317 24372 leader_election.cc:290] T 8d60fa948011446da3fd7cf40a6cfc10 P 464091b79ebd46ab97acf1f8605de50c [CANDIDATE]: Term 2 election: Requested vote from peers d27b530f2dca4661b6c3949e255a222b (127.5.58.2:40557), 2a5808db2a594f479695d8a649074c78 (127.5.58.3:34471)
I20260709 12:25:39.864591 24447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d60fa948011446da3fd7cf40a6cfc10" candidate_uuid: "464091b79ebd46ab97acf1f8605de50c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d27b530f2dca4661b6c3949e255a222b"
I20260709 12:25:39.865309 24447 raft_consensus.cc:3055] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:39.865718 24447 raft_consensus.cc:740] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d27b530f2dca4661b6c3949e255a222b, State: Running, Role: LEADER
I20260709 12:25:39.865626 24521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d60fa948011446da3fd7cf40a6cfc10" candidate_uuid: "464091b79ebd46ab97acf1f8605de50c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2a5808db2a594f479695d8a649074c78"
I20260709 12:25:39.866493 24521 raft_consensus.cc:3060] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:39.866854 24447 consensus_queue.cc:260] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:39.868386 24447 raft_consensus.cc:3060] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:39.874653 24521 raft_consensus.cc:2468] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 464091b79ebd46ab97acf1f8605de50c in term 2.
I20260709 12:25:39.876252 24335 leader_election.cc:304] T 8d60fa948011446da3fd7cf40a6cfc10 P 464091b79ebd46ab97acf1f8605de50c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a5808db2a594f479695d8a649074c78, 464091b79ebd46ab97acf1f8605de50c; no voters: 
I20260709 12:25:39.876276 24447 raft_consensus.cc:2468] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 464091b79ebd46ab97acf1f8605de50c in term 2.
I20260709 12:25:39.877437 24565 raft_consensus.cc:2804] T 8d60fa948011446da3fd7cf40a6cfc10 P 464091b79ebd46ab97acf1f8605de50c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:39.878991 24565 raft_consensus.cc:697] T 8d60fa948011446da3fd7cf40a6cfc10 P 464091b79ebd46ab97acf1f8605de50c [term 2 LEADER]: Becoming Leader. State: Replica: 464091b79ebd46ab97acf1f8605de50c, State: Running, Role: LEADER
I20260709 12:25:39.879993 24565 consensus_queue.cc:237] T 8d60fa948011446da3fd7cf40a6cfc10 P 464091b79ebd46ab97acf1f8605de50c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:39.889256 24266 catalog_manager.cc:5697] T 8d60fa948011446da3fd7cf40a6cfc10 P 464091b79ebd46ab97acf1f8605de50c reported cstate change: term changed from 1 to 2, leader changed from d27b530f2dca4661b6c3949e255a222b (127.5.58.2) to 464091b79ebd46ab97acf1f8605de50c (127.5.58.1). New cstate: current_term: 2 leader_uuid: "464091b79ebd46ab97acf1f8605de50c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:39.956260 24604 raft_consensus.cc:993] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b: : Instructing follower 2a5808db2a594f479695d8a649074c78 to start an election
I20260709 12:25:39.956863 24574 raft_consensus.cc:1081] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b [term 1 LEADER]: Signalling peer 2a5808db2a594f479695d8a649074c78 to start an election
I20260709 12:25:39.958482 24521 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cad6990a5f8c4c20b1d4459b5f4c69ce"
dest_uuid: "2a5808db2a594f479695d8a649074c78"
 from {username='slave'} at 127.0.0.1:33392
I20260709 12:25:39.959057 24521 raft_consensus.cc:493] T cad6990a5f8c4c20b1d4459b5f4c69ce P 2a5808db2a594f479695d8a649074c78 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:39.959475 24521 raft_consensus.cc:3060] T cad6990a5f8c4c20b1d4459b5f4c69ce P 2a5808db2a594f479695d8a649074c78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:39.966491 24521 raft_consensus.cc:515] T cad6990a5f8c4c20b1d4459b5f4c69ce P 2a5808db2a594f479695d8a649074c78 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:39.966971 24596 raft_consensus.cc:993] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78: : Instructing follower 464091b79ebd46ab97acf1f8605de50c to start an election
I20260709 12:25:39.967605 24566 raft_consensus.cc:1081] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [term 1 LEADER]: Signalling peer 464091b79ebd46ab97acf1f8605de50c to start an election
I20260709 12:25:39.968945 24521 leader_election.cc:290] T cad6990a5f8c4c20b1d4459b5f4c69ce P 2a5808db2a594f479695d8a649074c78 [CANDIDATE]: Term 2 election: Requested vote from peers 464091b79ebd46ab97acf1f8605de50c (127.5.58.1:43139), d27b530f2dca4661b6c3949e255a222b (127.5.58.2:40557)
I20260709 12:25:39.969537 24372 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e47a9d14818741abb2f13398733b83cc"
dest_uuid: "464091b79ebd46ab97acf1f8605de50c"
 from {username='slave'} at 127.0.0.1:43546
I20260709 12:25:39.970197 24371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cad6990a5f8c4c20b1d4459b5f4c69ce" candidate_uuid: "2a5808db2a594f479695d8a649074c78" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "464091b79ebd46ab97acf1f8605de50c"
I20260709 12:25:39.970341 24447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cad6990a5f8c4c20b1d4459b5f4c69ce" candidate_uuid: "2a5808db2a594f479695d8a649074c78" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d27b530f2dca4661b6c3949e255a222b"
I20260709 12:25:39.970242 24372 raft_consensus.cc:493] T e47a9d14818741abb2f13398733b83cc P 464091b79ebd46ab97acf1f8605de50c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:39.970947 24371 raft_consensus.cc:3060] T cad6990a5f8c4c20b1d4459b5f4c69ce P 464091b79ebd46ab97acf1f8605de50c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:39.970993 24447 raft_consensus.cc:3055] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:39.971025 24372 raft_consensus.cc:3060] T e47a9d14818741abb2f13398733b83cc P 464091b79ebd46ab97acf1f8605de50c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:39.971473 24447 raft_consensus.cc:740] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d27b530f2dca4661b6c3949e255a222b, State: Running, Role: LEADER
I20260709 12:25:39.972512 24447 consensus_queue.cc:260] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:39.974138 24447 raft_consensus.cc:3060] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:39.978164 24372 raft_consensus.cc:515] T e47a9d14818741abb2f13398733b83cc P 464091b79ebd46ab97acf1f8605de50c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } }
I20260709 12:25:39.981283 24372 leader_election.cc:290] T e47a9d14818741abb2f13398733b83cc P 464091b79ebd46ab97acf1f8605de50c [CANDIDATE]: Term 2 election: Requested vote from peers 2a5808db2a594f479695d8a649074c78 (127.5.58.3:34471), d27b530f2dca4661b6c3949e255a222b (127.5.58.2:40557)
I20260709 12:25:39.982098 24446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e47a9d14818741abb2f13398733b83cc" candidate_uuid: "464091b79ebd46ab97acf1f8605de50c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d27b530f2dca4661b6c3949e255a222b"
I20260709 12:25:39.982314 24371 raft_consensus.cc:2468] T cad6990a5f8c4c20b1d4459b5f4c69ce P 464091b79ebd46ab97acf1f8605de50c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a5808db2a594f479695d8a649074c78 in term 2.
I20260709 12:25:39.982810 24446 raft_consensus.cc:3060] T e47a9d14818741abb2f13398733b83cc P d27b530f2dca4661b6c3949e255a222b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:39.983778 24521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e47a9d14818741abb2f13398733b83cc" candidate_uuid: "464091b79ebd46ab97acf1f8605de50c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2a5808db2a594f479695d8a649074c78"
I20260709 12:25:39.984485 24521 raft_consensus.cc:3055] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:39.984843 24521 raft_consensus.cc:740] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2a5808db2a594f479695d8a649074c78, State: Running, Role: LEADER
I20260709 12:25:39.984879 24486 leader_election.cc:304] T cad6990a5f8c4c20b1d4459b5f4c69ce P 2a5808db2a594f479695d8a649074c78 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a5808db2a594f479695d8a649074c78, 464091b79ebd46ab97acf1f8605de50c; no voters: 
I20260709 12:25:39.986035 24566 raft_consensus.cc:2804] T cad6990a5f8c4c20b1d4459b5f4c69ce P 2a5808db2a594f479695d8a649074c78 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:39.986704 24566 raft_consensus.cc:697] T cad6990a5f8c4c20b1d4459b5f4c69ce P 2a5808db2a594f479695d8a649074c78 [term 2 LEADER]: Becoming Leader. State: Replica: 2a5808db2a594f479695d8a649074c78, State: Running, Role: LEADER
I20260709 12:25:39.987823 24447 raft_consensus.cc:2468] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a5808db2a594f479695d8a649074c78 in term 2.
I20260709 12:25:39.987622 24566 consensus_queue.cc:237] T cad6990a5f8c4c20b1d4459b5f4c69ce P 2a5808db2a594f479695d8a649074c78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:39.985801 24521 consensus_queue.cc:260] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } }
I20260709 12:25:39.991019 24576 consensus_queue.cc:1048] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:39.992221 24521 raft_consensus.cc:3060] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:39.998623 24446 raft_consensus.cc:2468] T e47a9d14818741abb2f13398733b83cc P d27b530f2dca4661b6c3949e255a222b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 464091b79ebd46ab97acf1f8605de50c in term 2.
I20260709 12:25:40.001063 24337 leader_election.cc:304] T e47a9d14818741abb2f13398733b83cc P 464091b79ebd46ab97acf1f8605de50c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 464091b79ebd46ab97acf1f8605de50c, d27b530f2dca4661b6c3949e255a222b; no voters: 
I20260709 12:25:40.003047 24565 raft_consensus.cc:2804] T e47a9d14818741abb2f13398733b83cc P 464091b79ebd46ab97acf1f8605de50c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:40.002286 24268 catalog_manager.cc:5697] T cad6990a5f8c4c20b1d4459b5f4c69ce P 2a5808db2a594f479695d8a649074c78 reported cstate change: term changed from 1 to 2, leader changed from d27b530f2dca4661b6c3949e255a222b (127.5.58.2) to 2a5808db2a594f479695d8a649074c78 (127.5.58.3). New cstate: current_term: 2 leader_uuid: "2a5808db2a594f479695d8a649074c78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:40.003387 24521 raft_consensus.cc:2468] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 464091b79ebd46ab97acf1f8605de50c in term 2.
I20260709 12:25:40.003849 24565 raft_consensus.cc:697] T e47a9d14818741abb2f13398733b83cc P 464091b79ebd46ab97acf1f8605de50c [term 2 LEADER]: Becoming Leader. State: Replica: 464091b79ebd46ab97acf1f8605de50c, State: Running, Role: LEADER
I20260709 12:25:40.005344 24565 consensus_queue.cc:237] T e47a9d14818741abb2f13398733b83cc P 464091b79ebd46ab97acf1f8605de50c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } }
I20260709 12:25:40.022277 24576 raft_consensus.cc:993] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78: : Instructing follower 464091b79ebd46ab97acf1f8605de50c to start an election
I20260709 12:25:40.023259 24596 raft_consensus.cc:1081] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [term 1 LEADER]: Signalling peer 464091b79ebd46ab97acf1f8605de50c to start an election
I20260709 12:25:40.025874 24372 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "66bf786ed4d245398e9645dfa35b056f"
dest_uuid: "464091b79ebd46ab97acf1f8605de50c"
 from {username='slave'} at 127.0.0.1:43546
I20260709 12:25:40.027863 24372 raft_consensus.cc:493] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:40.028364 24372 raft_consensus.cc:3060] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:40.030396 24566 consensus_queue.cc:1048] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:40.030803 24268 catalog_manager.cc:5697] T e47a9d14818741abb2f13398733b83cc P 464091b79ebd46ab97acf1f8605de50c reported cstate change: term changed from 1 to 2, leader changed from 2a5808db2a594f479695d8a649074c78 (127.5.58.3) to 464091b79ebd46ab97acf1f8605de50c (127.5.58.1). New cstate: current_term: 2 leader_uuid: "464091b79ebd46ab97acf1f8605de50c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:40.041440 24372 raft_consensus.cc:515] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:40.050761 24372 leader_election.cc:290] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c [CANDIDATE]: Term 2 election: Requested vote from peers d27b530f2dca4661b6c3949e255a222b (127.5.58.2:40557), 2a5808db2a594f479695d8a649074c78 (127.5.58.3:34471)
I20260709 12:25:40.052416 24521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66bf786ed4d245398e9645dfa35b056f" candidate_uuid: "464091b79ebd46ab97acf1f8605de50c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2a5808db2a594f479695d8a649074c78"
I20260709 12:25:40.052516 24446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66bf786ed4d245398e9645dfa35b056f" candidate_uuid: "464091b79ebd46ab97acf1f8605de50c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d27b530f2dca4661b6c3949e255a222b"
I20260709 12:25:40.053141 24521 raft_consensus.cc:3055] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:40.053277 24446 raft_consensus.cc:3060] T 66bf786ed4d245398e9645dfa35b056f P d27b530f2dca4661b6c3949e255a222b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:40.053556 24521 raft_consensus.cc:740] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2a5808db2a594f479695d8a649074c78, State: Running, Role: LEADER
I20260709 12:25:40.055109 24521 consensus_queue.cc:260] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:40.057204 24521 raft_consensus.cc:3060] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:40.064543 24446 raft_consensus.cc:2468] T 66bf786ed4d245398e9645dfa35b056f P d27b530f2dca4661b6c3949e255a222b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 464091b79ebd46ab97acf1f8605de50c in term 2.
I20260709 12:25:40.066123 24337 leader_election.cc:304] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 464091b79ebd46ab97acf1f8605de50c, d27b530f2dca4661b6c3949e255a222b; no voters: 
I20260709 12:25:40.067193 24521 raft_consensus.cc:2468] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 464091b79ebd46ab97acf1f8605de50c in term 2.
I20260709 12:25:40.067260 24565 raft_consensus.cc:2804] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:40.068306 24565 raft_consensus.cc:697] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c [term 2 LEADER]: Becoming Leader. State: Replica: 464091b79ebd46ab97acf1f8605de50c, State: Running, Role: LEADER
I20260709 12:25:40.069461 24565 consensus_queue.cc:237] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:40.084841 24268 catalog_manager.cc:5697] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c reported cstate change: term changed from 1 to 2, leader changed from 2a5808db2a594f479695d8a649074c78 (127.5.58.3) to 464091b79ebd46ab97acf1f8605de50c (127.5.58.1). New cstate: current_term: 2 leader_uuid: "464091b79ebd46ab97acf1f8605de50c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:40.165758  5352 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 12:25:40.168545  5352 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 12:25:40.169593  5352 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 12:25:40.183207 24372 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "66bf786ed4d245398e9645dfa35b056f"
dest_uuid: "464091b79ebd46ab97acf1f8605de50c"
mode: GRACEFUL
new_leader_uuid: "d27b530f2dca4661b6c3949e255a222b"
 from {username='slave'} at 127.0.0.1:43560
I20260709 12:25:40.183837 24372 raft_consensus.cc:606] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c [term 2 LEADER]: Received request to transfer leadership to TS d27b530f2dca4661b6c3949e255a222b
I20260709 12:25:40.185271  5352 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 12:25:40.375268 24521 raft_consensus.cc:1275] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78 [term 2 FOLLOWER]: Refusing update from remote peer 464091b79ebd46ab97acf1f8605de50c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:40.375514 24446 raft_consensus.cc:1275] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [term 2 FOLLOWER]: Refusing update from remote peer 464091b79ebd46ab97acf1f8605de50c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:40.376955 24608 consensus_queue.cc:1048] T 8d60fa948011446da3fd7cf40a6cfc10 P 464091b79ebd46ab97acf1f8605de50c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:40.377543 24565 consensus_queue.cc:1048] T 8d60fa948011446da3fd7cf40a6cfc10 P 464091b79ebd46ab97acf1f8605de50c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:40.440479 24446 raft_consensus.cc:1275] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b [term 2 FOLLOWER]: Refusing update from remote peer 2a5808db2a594f479695d8a649074c78: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:40.441964 24596 consensus_queue.cc:1048] T cad6990a5f8c4c20b1d4459b5f4c69ce P 2a5808db2a594f479695d8a649074c78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:40.450562 24372 raft_consensus.cc:1275] T cad6990a5f8c4c20b1d4459b5f4c69ce P 464091b79ebd46ab97acf1f8605de50c [term 2 FOLLOWER]: Refusing update from remote peer 2a5808db2a594f479695d8a649074c78: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:40.452221 24596 consensus_queue.cc:1048] T cad6990a5f8c4c20b1d4459b5f4c69ce P 2a5808db2a594f479695d8a649074c78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:40.496228 24521 raft_consensus.cc:1275] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [term 2 FOLLOWER]: Refusing update from remote peer 464091b79ebd46ab97acf1f8605de50c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:40.497814 24565 consensus_queue.cc:1048] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:40.506202 24446 raft_consensus.cc:1275] T 66bf786ed4d245398e9645dfa35b056f P d27b530f2dca4661b6c3949e255a222b [term 2 FOLLOWER]: Refusing update from remote peer 464091b79ebd46ab97acf1f8605de50c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:40.508225 24608 consensus_queue.cc:1048] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:40.514598 24613 raft_consensus.cc:993] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c: : Instructing follower d27b530f2dca4661b6c3949e255a222b to start an election
I20260709 12:25:40.515106 24613 raft_consensus.cc:1081] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c [term 2 LEADER]: Signalling peer d27b530f2dca4661b6c3949e255a222b to start an election
I20260709 12:25:40.516777 24446 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "66bf786ed4d245398e9645dfa35b056f"
dest_uuid: "d27b530f2dca4661b6c3949e255a222b"
 from {username='slave'} at 127.0.0.1:37854
I20260709 12:25:40.517271 24446 raft_consensus.cc:493] T 66bf786ed4d245398e9645dfa35b056f P d27b530f2dca4661b6c3949e255a222b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:40.517647 24446 raft_consensus.cc:3060] T 66bf786ed4d245398e9645dfa35b056f P d27b530f2dca4661b6c3949e255a222b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:40.522933 24446 raft_consensus.cc:515] T 66bf786ed4d245398e9645dfa35b056f P d27b530f2dca4661b6c3949e255a222b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:40.524724 24446 leader_election.cc:290] T 66bf786ed4d245398e9645dfa35b056f P d27b530f2dca4661b6c3949e255a222b [CANDIDATE]: Term 3 election: Requested vote from peers 464091b79ebd46ab97acf1f8605de50c (127.5.58.1:43139), 2a5808db2a594f479695d8a649074c78 (127.5.58.3:34471)
I20260709 12:25:40.525781 24521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66bf786ed4d245398e9645dfa35b056f" candidate_uuid: "d27b530f2dca4661b6c3949e255a222b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2a5808db2a594f479695d8a649074c78"
I20260709 12:25:40.526185 24372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66bf786ed4d245398e9645dfa35b056f" candidate_uuid: "d27b530f2dca4661b6c3949e255a222b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "464091b79ebd46ab97acf1f8605de50c"
I20260709 12:25:40.526413 24521 raft_consensus.cc:3060] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:40.526842 24372 raft_consensus.cc:3055] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:40.527159 24372 raft_consensus.cc:740] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 464091b79ebd46ab97acf1f8605de50c, State: Running, Role: LEADER
I20260709 12:25:40.527878 24372 consensus_queue.cc:260] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:40.528928 24372 raft_consensus.cc:3060] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:40.532218 24521 raft_consensus.cc:2468] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d27b530f2dca4661b6c3949e255a222b in term 3.
I20260709 12:25:40.533715 24410 leader_election.cc:304] T 66bf786ed4d245398e9645dfa35b056f P d27b530f2dca4661b6c3949e255a222b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a5808db2a594f479695d8a649074c78, d27b530f2dca4661b6c3949e255a222b; no voters: 
I20260709 12:25:40.534695 24574 raft_consensus.cc:2804] T 66bf786ed4d245398e9645dfa35b056f P d27b530f2dca4661b6c3949e255a222b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:40.535070 24372 raft_consensus.cc:2468] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d27b530f2dca4661b6c3949e255a222b in term 3.
I20260709 12:25:40.535733 24574 raft_consensus.cc:697] T 66bf786ed4d245398e9645dfa35b056f P d27b530f2dca4661b6c3949e255a222b [term 3 LEADER]: Becoming Leader. State: Replica: d27b530f2dca4661b6c3949e255a222b, State: Running, Role: LEADER
I20260709 12:25:40.536705 24574 consensus_queue.cc:237] T 66bf786ed4d245398e9645dfa35b056f P d27b530f2dca4661b6c3949e255a222b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:40.543751 24268 catalog_manager.cc:5697] T 66bf786ed4d245398e9645dfa35b056f P d27b530f2dca4661b6c3949e255a222b reported cstate change: term changed from 2 to 3, leader changed from 464091b79ebd46ab97acf1f8605de50c (127.5.58.1) to d27b530f2dca4661b6c3949e255a222b (127.5.58.2). New cstate: current_term: 3 leader_uuid: "d27b530f2dca4661b6c3949e255a222b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:40.557982 24521 raft_consensus.cc:1275] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [term 2 FOLLOWER]: Refusing update from remote peer 464091b79ebd46ab97acf1f8605de50c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:40.559343 24613 consensus_queue.cc:1048] T e47a9d14818741abb2f13398733b83cc P 464091b79ebd46ab97acf1f8605de50c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:40.569500 24446 raft_consensus.cc:1275] T e47a9d14818741abb2f13398733b83cc P d27b530f2dca4661b6c3949e255a222b [term 2 FOLLOWER]: Refusing update from remote peer 464091b79ebd46ab97acf1f8605de50c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:40.572794 24608 consensus_queue.cc:1048] T e47a9d14818741abb2f13398733b83cc P 464091b79ebd46ab97acf1f8605de50c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:40.823226  5352 tablet_server.cc:179] TabletServer@127.5.58.1:0 shutting down...
W20260709 12:25:41.072783 24412 consensus_peers.cc:597] T 66bf786ed4d245398e9645dfa35b056f P d27b530f2dca4661b6c3949e255a222b -> Peer 464091b79ebd46ab97acf1f8605de50c (127.5.58.1:43139): Couldn't send request to peer 464091b79ebd46ab97acf1f8605de50c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:41.074697 24486 consensus_peers.cc:597] T cad6990a5f8c4c20b1d4459b5f4c69ce P 2a5808db2a594f479695d8a649074c78 -> Peer 464091b79ebd46ab97acf1f8605de50c (127.5.58.1:43139): Couldn't send request to peer 464091b79ebd46ab97acf1f8605de50c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:41.090103 24521 raft_consensus.cc:1275] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [term 3 FOLLOWER]: Refusing update from remote peer d27b530f2dca4661b6c3949e255a222b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:41.093008 24574 consensus_queue.cc:1048] T 66bf786ed4d245398e9645dfa35b056f P d27b530f2dca4661b6c3949e255a222b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:41.127779  5352 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:25:41.128584  5352 tablet_replica.cc:333] T 8d60fa948011446da3fd7cf40a6cfc10 P 464091b79ebd46ab97acf1f8605de50c: stopping tablet replica
I20260709 12:25:41.129504  5352 raft_consensus.cc:2243] T 8d60fa948011446da3fd7cf40a6cfc10 P 464091b79ebd46ab97acf1f8605de50c [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:25:41.130954  5352 raft_consensus.cc:2272] T 8d60fa948011446da3fd7cf40a6cfc10 P 464091b79ebd46ab97acf1f8605de50c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:41.134348  5352 tablet_replica.cc:333] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c: stopping tablet replica
I20260709 12:25:41.134963  5352 raft_consensus.cc:2243] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:41.135462  5352 raft_consensus.cc:2272] T 66bf786ed4d245398e9645dfa35b056f P 464091b79ebd46ab97acf1f8605de50c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:41.137952  5352 tablet_replica.cc:333] T e47a9d14818741abb2f13398733b83cc P 464091b79ebd46ab97acf1f8605de50c: stopping tablet replica
I20260709 12:25:41.138538  5352 raft_consensus.cc:2243] T e47a9d14818741abb2f13398733b83cc P 464091b79ebd46ab97acf1f8605de50c [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:25:41.139420  5352 raft_consensus.cc:2272] T e47a9d14818741abb2f13398733b83cc P 464091b79ebd46ab97acf1f8605de50c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:41.141995  5352 tablet_replica.cc:333] T cad6990a5f8c4c20b1d4459b5f4c69ce P 464091b79ebd46ab97acf1f8605de50c: stopping tablet replica
I20260709 12:25:41.142511  5352 raft_consensus.cc:2243] T cad6990a5f8c4c20b1d4459b5f4c69ce P 464091b79ebd46ab97acf1f8605de50c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:41.142994  5352 raft_consensus.cc:2272] T cad6990a5f8c4c20b1d4459b5f4c69ce P 464091b79ebd46ab97acf1f8605de50c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:25:41.919951 24393 debug-util.cc:398] Leaking SignalData structure 0x7b080023d0c0 after lost signal to thread 5355
W20260709 12:25:41.921465 24393 debug-util.cc:398] Leaking SignalData structure 0x7b08001a83e0 after lost signal to thread 24304
W20260709 12:25:41.923095 24393 debug-util.cc:398] Leaking SignalData structure 0x7b08000e3e60 after lost signal to thread 24471
W20260709 12:25:41.924197 24393 debug-util.cc:398] Leaking SignalData structure 0x7b0800131260 after lost signal to thread 24545
I20260709 12:25:42.077113 24619 raft_consensus.cc:493] T e47a9d14818741abb2f13398733b83cc P d27b530f2dca4661b6c3949e255a222b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 464091b79ebd46ab97acf1f8605de50c)
I20260709 12:25:42.077900 24619 raft_consensus.cc:515] T e47a9d14818741abb2f13398733b83cc P d27b530f2dca4661b6c3949e255a222b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } }
I20260709 12:25:42.080550 24619 leader_election.cc:290] T e47a9d14818741abb2f13398733b83cc P d27b530f2dca4661b6c3949e255a222b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 464091b79ebd46ab97acf1f8605de50c (127.5.58.1:43139), 2a5808db2a594f479695d8a649074c78 (127.5.58.3:34471)
I20260709 12:25:42.081733 24521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e47a9d14818741abb2f13398733b83cc" candidate_uuid: "d27b530f2dca4661b6c3949e255a222b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2a5808db2a594f479695d8a649074c78" is_pre_election: true
W20260709 12:25:42.085515 24412 leader_election.cc:336] T e47a9d14818741abb2f13398733b83cc P d27b530f2dca4661b6c3949e255a222b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 464091b79ebd46ab97acf1f8605de50c (127.5.58.1:43139): Network error: Client connection negotiation failed: client connection to 127.5.58.1:43139: connect: Connection refused (error 111)
I20260709 12:25:42.085944 24412 leader_election.cc:304] T e47a9d14818741abb2f13398733b83cc P d27b530f2dca4661b6c3949e255a222b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d27b530f2dca4661b6c3949e255a222b; no voters: 2a5808db2a594f479695d8a649074c78, 464091b79ebd46ab97acf1f8605de50c
I20260709 12:25:42.086675 24619 raft_consensus.cc:2749] T e47a9d14818741abb2f13398733b83cc P d27b530f2dca4661b6c3949e255a222b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:25:42.166796  5352 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 24393)
I20260709 12:25:42.285007 24619 raft_consensus.cc:493] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 464091b79ebd46ab97acf1f8605de50c)
I20260709 12:25:42.285778 24619 raft_consensus.cc:515] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:42.288467 24619 leader_election.cc:290] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 464091b79ebd46ab97acf1f8605de50c (127.5.58.1:43139), 2a5808db2a594f479695d8a649074c78 (127.5.58.3:34471)
I20260709 12:25:42.290045 24521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d60fa948011446da3fd7cf40a6cfc10" candidate_uuid: "d27b530f2dca4661b6c3949e255a222b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2a5808db2a594f479695d8a649074c78" is_pre_election: true
W20260709 12:25:42.293777 24412 leader_election.cc:336] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 464091b79ebd46ab97acf1f8605de50c (127.5.58.1:43139): Network error: Client connection negotiation failed: client connection to 127.5.58.1:43139: connect: Connection refused (error 111)
I20260709 12:25:42.294248 24412 leader_election.cc:304] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d27b530f2dca4661b6c3949e255a222b; no voters: 2a5808db2a594f479695d8a649074c78, 464091b79ebd46ab97acf1f8605de50c
I20260709 12:25:42.294987 24619 raft_consensus.cc:2749] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:42.630328 24596 raft_consensus.cc:493] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 464091b79ebd46ab97acf1f8605de50c)
I20260709 12:25:42.631027 24596 raft_consensus.cc:515] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } }
I20260709 12:25:42.633630 24596 leader_election.cc:290] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 464091b79ebd46ab97acf1f8605de50c (127.5.58.1:43139), d27b530f2dca4661b6c3949e255a222b (127.5.58.2:40557)
I20260709 12:25:42.634486 24596 raft_consensus.cc:493] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 464091b79ebd46ab97acf1f8605de50c)
I20260709 12:25:42.635087 24596 raft_consensus.cc:515] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:42.635754 24446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e47a9d14818741abb2f13398733b83cc" candidate_uuid: "2a5808db2a594f479695d8a649074c78" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d27b530f2dca4661b6c3949e255a222b" is_pre_election: true
I20260709 12:25:42.636519 24446 raft_consensus.cc:2468] T e47a9d14818741abb2f13398733b83cc P d27b530f2dca4661b6c3949e255a222b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a5808db2a594f479695d8a649074c78 in term 2.
I20260709 12:25:42.637420 24596 leader_election.cc:290] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 464091b79ebd46ab97acf1f8605de50c (127.5.58.1:43139), d27b530f2dca4661b6c3949e255a222b (127.5.58.2:40557)
I20260709 12:25:42.639626 24446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d60fa948011446da3fd7cf40a6cfc10" candidate_uuid: "2a5808db2a594f479695d8a649074c78" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d27b530f2dca4661b6c3949e255a222b" is_pre_election: true
I20260709 12:25:42.640113 24486 leader_election.cc:304] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a5808db2a594f479695d8a649074c78, d27b530f2dca4661b6c3949e255a222b; no voters: 
I20260709 12:25:42.640283 24446 raft_consensus.cc:2468] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a5808db2a594f479695d8a649074c78 in term 2.
I20260709 12:25:42.641127 24596 raft_consensus.cc:2804] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:25:42.641158 24486 leader_election.cc:336] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 464091b79ebd46ab97acf1f8605de50c (127.5.58.1:43139): Network error: Client connection negotiation failed: client connection to 127.5.58.1:43139: connect: Connection refused (error 111)
I20260709 12:25:42.641839 24596 raft_consensus.cc:493] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 464091b79ebd46ab97acf1f8605de50c)
W20260709 12:25:42.642205 24486 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.58.1:43139: connect: Connection refused (error 111) [suppressed 18 similar messages]
I20260709 12:25:42.642333 24596 raft_consensus.cc:3060] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:42.643602 24486 leader_election.cc:304] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a5808db2a594f479695d8a649074c78, d27b530f2dca4661b6c3949e255a222b; no voters: 
I20260709 12:25:42.645311 24626 raft_consensus.cc:2804] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:42.645815 24626 raft_consensus.cc:493] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 464091b79ebd46ab97acf1f8605de50c)
I20260709 12:25:42.646268 24626 raft_consensus.cc:3060] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:42.647358 24486 leader_election.cc:336] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 464091b79ebd46ab97acf1f8605de50c (127.5.58.1:43139): Network error: Client connection negotiation failed: client connection to 127.5.58.1:43139: connect: Connection refused (error 111)
I20260709 12:25:42.651631 24596 raft_consensus.cc:515] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } }
I20260709 12:25:42.654610 24596 leader_election.cc:290] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [CANDIDATE]: Term 3 election: Requested vote from peers 464091b79ebd46ab97acf1f8605de50c (127.5.58.1:43139), d27b530f2dca4661b6c3949e255a222b (127.5.58.2:40557)
I20260709 12:25:42.655104 24626 raft_consensus.cc:515] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:42.657473 24626 leader_election.cc:290] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78 [CANDIDATE]: Term 3 election: Requested vote from peers 464091b79ebd46ab97acf1f8605de50c (127.5.58.1:43139), d27b530f2dca4661b6c3949e255a222b (127.5.58.2:40557)
I20260709 12:25:42.657774 24446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e47a9d14818741abb2f13398733b83cc" candidate_uuid: "2a5808db2a594f479695d8a649074c78" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d27b530f2dca4661b6c3949e255a222b"
I20260709 12:25:42.658459 24446 raft_consensus.cc:3060] T e47a9d14818741abb2f13398733b83cc P d27b530f2dca4661b6c3949e255a222b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:42.660027 24447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d60fa948011446da3fd7cf40a6cfc10" candidate_uuid: "2a5808db2a594f479695d8a649074c78" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d27b530f2dca4661b6c3949e255a222b"
I20260709 12:25:42.660763 24447 raft_consensus.cc:3060] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:42.661454 24486 leader_election.cc:336] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 464091b79ebd46ab97acf1f8605de50c (127.5.58.1:43139): Network error: Client connection negotiation failed: client connection to 127.5.58.1:43139: connect: Connection refused (error 111)
W20260709 12:25:42.665400 24486 leader_election.cc:336] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 464091b79ebd46ab97acf1f8605de50c (127.5.58.1:43139): Network error: Client connection negotiation failed: client connection to 127.5.58.1:43139: connect: Connection refused (error 111)
I20260709 12:25:42.666467 24446 raft_consensus.cc:2468] T e47a9d14818741abb2f13398733b83cc P d27b530f2dca4661b6c3949e255a222b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a5808db2a594f479695d8a649074c78 in term 3.
I20260709 12:25:42.667768 24486 leader_election.cc:304] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a5808db2a594f479695d8a649074c78, d27b530f2dca4661b6c3949e255a222b; no voters: 464091b79ebd46ab97acf1f8605de50c
I20260709 12:25:42.668640 24447 raft_consensus.cc:2468] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a5808db2a594f479695d8a649074c78 in term 3.
I20260709 12:25:42.668704 24626 raft_consensus.cc:2804] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:42.669466 24626 raft_consensus.cc:697] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [term 3 LEADER]: Becoming Leader. State: Replica: 2a5808db2a594f479695d8a649074c78, State: Running, Role: LEADER
I20260709 12:25:42.670351 24626 consensus_queue.cc:237] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } }
I20260709 12:25:42.670675 24486 leader_election.cc:304] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a5808db2a594f479695d8a649074c78, d27b530f2dca4661b6c3949e255a222b; no voters: 464091b79ebd46ab97acf1f8605de50c
I20260709 12:25:42.671635 24596 raft_consensus.cc:2804] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:42.673345 24596 raft_consensus.cc:697] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78 [term 3 LEADER]: Becoming Leader. State: Replica: 2a5808db2a594f479695d8a649074c78, State: Running, Role: LEADER
I20260709 12:25:42.674155 24596 consensus_queue.cc:237] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } }
I20260709 12:25:42.683390 24267 catalog_manager.cc:5697] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 reported cstate change: term changed from 2 to 3, leader changed from 464091b79ebd46ab97acf1f8605de50c (127.5.58.1) to 2a5808db2a594f479695d8a649074c78 (127.5.58.3). New cstate: current_term: 3 leader_uuid: "2a5808db2a594f479695d8a649074c78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:42.706486 24267 catalog_manager.cc:5697] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78 reported cstate change: term changed from 2 to 3, leader changed from 464091b79ebd46ab97acf1f8605de50c (127.5.58.1) to 2a5808db2a594f479695d8a649074c78 (127.5.58.3). New cstate: current_term: 3 leader_uuid: "2a5808db2a594f479695d8a649074c78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464091b79ebd46ab97acf1f8605de50c" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 43139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a5808db2a594f479695d8a649074c78" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 34471 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:43.147218 24447 raft_consensus.cc:1275] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [term 3 FOLLOWER]: Refusing update from remote peer 2a5808db2a594f479695d8a649074c78: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:43.148794 24626 consensus_queue.cc:1048] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:25:43.165313 24486 consensus_peers.cc:597] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78 -> Peer 464091b79ebd46ab97acf1f8605de50c (127.5.58.1:43139): Couldn't send request to peer 464091b79ebd46ab97acf1f8605de50c. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:43139: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:43.167377  5352 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 24393)
W20260709 12:25:43.178186 24486 consensus_peers.cc:597] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 -> Peer 464091b79ebd46ab97acf1f8605de50c (127.5.58.1:43139): Couldn't send request to peer 464091b79ebd46ab97acf1f8605de50c. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:43139: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:43.182425 24447 raft_consensus.cc:1275] T e47a9d14818741abb2f13398733b83cc P d27b530f2dca4661b6c3949e255a222b [term 3 FOLLOWER]: Refusing update from remote peer 2a5808db2a594f479695d8a649074c78: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:43.183915 24596 consensus_queue.cc:1048] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d27b530f2dca4661b6c3949e255a222b" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 40557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:25:43.261569 24486 consensus_peers.cc:597] T cad6990a5f8c4c20b1d4459b5f4c69ce P 2a5808db2a594f479695d8a649074c78 -> Peer 464091b79ebd46ab97acf1f8605de50c (127.5.58.1:43139): Couldn't send request to peer 464091b79ebd46ab97acf1f8605de50c. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:43139: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:43.471470 24412 consensus_peers.cc:597] T 66bf786ed4d245398e9645dfa35b056f P d27b530f2dca4661b6c3949e255a222b -> Peer 464091b79ebd46ab97acf1f8605de50c (127.5.58.1:43139): Couldn't send request to peer 464091b79ebd46ab97acf1f8605de50c. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:43139: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:25:44.168107  5352 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 24393)
W20260709 12:25:45.168859  5352 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 24393)
I20260709 12:25:45.243132  5352 tablet_server.cc:196] TabletServer@127.5.58.1:0 shutdown complete.
I20260709 12:25:45.274720  5352 tablet_server.cc:179] TabletServer@127.5.58.2:0 shutting down...
I20260709 12:25:45.303681  5352 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:25:45.304704  5352 tablet_replica.cc:333] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b: stopping tablet replica
I20260709 12:25:45.305588  5352 raft_consensus.cc:2243] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:45.306443  5352 raft_consensus.cc:2272] T 8d60fa948011446da3fd7cf40a6cfc10 P d27b530f2dca4661b6c3949e255a222b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:45.309600  5352 tablet_replica.cc:333] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b: stopping tablet replica
I20260709 12:25:45.310176  5352 raft_consensus.cc:2243] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:45.310784  5352 raft_consensus.cc:2272] T cad6990a5f8c4c20b1d4459b5f4c69ce P d27b530f2dca4661b6c3949e255a222b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:45.314127  5352 tablet_replica.cc:333] T 66bf786ed4d245398e9645dfa35b056f P d27b530f2dca4661b6c3949e255a222b: stopping tablet replica
I20260709 12:25:45.314797  5352 raft_consensus.cc:2243] T 66bf786ed4d245398e9645dfa35b056f P d27b530f2dca4661b6c3949e255a222b [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:25:45.316188  5352 raft_consensus.cc:2272] T 66bf786ed4d245398e9645dfa35b056f P d27b530f2dca4661b6c3949e255a222b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:45.320050  5352 tablet_replica.cc:333] T e47a9d14818741abb2f13398733b83cc P d27b530f2dca4661b6c3949e255a222b: stopping tablet replica
I20260709 12:25:45.320758  5352 raft_consensus.cc:2243] T e47a9d14818741abb2f13398733b83cc P d27b530f2dca4661b6c3949e255a222b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:45.321398  5352 raft_consensus.cc:2272] T e47a9d14818741abb2f13398733b83cc P d27b530f2dca4661b6c3949e255a222b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:45.355170  5352 tablet_server.cc:196] TabletServer@127.5.58.2:0 shutdown complete.
W20260709 12:25:45.357069 24486 consensus_peers.cc:597] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78 -> Peer d27b530f2dca4661b6c3949e255a222b (127.5.58.2:40557): Couldn't send request to peer d27b530f2dca4661b6c3949e255a222b. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:40557: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:45.377957  5352 tablet_server.cc:179] TabletServer@127.5.58.3:0 shutting down...
I20260709 12:25:45.416869  5352 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:25:45.417968  5352 tablet_replica.cc:333] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78: stopping tablet replica
I20260709 12:25:45.419162  5352 raft_consensus.cc:2243] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:25:45.420499  5352 raft_consensus.cc:2272] T 8d60fa948011446da3fd7cf40a6cfc10 P 2a5808db2a594f479695d8a649074c78 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:45.423167  5352 tablet_replica.cc:333] T cad6990a5f8c4c20b1d4459b5f4c69ce P 2a5808db2a594f479695d8a649074c78: stopping tablet replica
I20260709 12:25:45.423717  5352 raft_consensus.cc:2243] T cad6990a5f8c4c20b1d4459b5f4c69ce P 2a5808db2a594f479695d8a649074c78 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:25:45.424686  5352 raft_consensus.cc:2272] T cad6990a5f8c4c20b1d4459b5f4c69ce P 2a5808db2a594f479695d8a649074c78 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:45.427332  5352 tablet_replica.cc:333] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78: stopping tablet replica
I20260709 12:25:45.427820  5352 raft_consensus.cc:2243] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:45.429201  5352 raft_consensus.cc:2272] T 66bf786ed4d245398e9645dfa35b056f P 2a5808db2a594f479695d8a649074c78 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:45.432899  5352 tablet_replica.cc:333] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78: stopping tablet replica
I20260709 12:25:45.433394  5352 raft_consensus.cc:2243] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:25:45.434343  5352 raft_consensus.cc:2272] T e47a9d14818741abb2f13398733b83cc P 2a5808db2a594f479695d8a649074c78 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:45.462554  5352 tablet_server.cc:196] TabletServer@127.5.58.3:0 shutdown complete.
I20260709 12:25:45.485684  5352 master.cc:562] Master@127.5.58.62:41707 shutting down...
I20260709 12:25:45.510412  5352 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 520ce73ea270491d8da29cfdc2c6cfe5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:25:45.511157  5352 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 520ce73ea270491d8da29cfdc2c6cfe5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:45.511533  5352 tablet_replica.cc:333] T 00000000000000000000000000000000 P 520ce73ea270491d8da29cfdc2c6cfe5: stopping tablet replica
I20260709 12:25:45.535485  5352 master.cc:584] Master@127.5.58.62:41707 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (7438 ms)
[----------] 6 tests from LeaderRebalancerTest (603549 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 12:25:45.583245  5352 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.58.62:37581
I20260709 12:25:45.584515  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:25:45.591317 24645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:25:45.592186 24648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:25:45.592931  5352 server_base.cc:1061] running on GCE node
W20260709 12:25:45.592262 24646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:25:45.594305  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:25:45.594592  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:25:45.594774  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599945594758 us; error 0 us; skew 500 ppm
I20260709 12:25:45.595377  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:25:45.598311  5352 webserver.cc:533] Webserver started at http://127.5.58.62:33259/ using document root <none> and password file <none>
I20260709 12:25:45.599148  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:25:45.599395  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:25:45.599692  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:25:45.601045  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599341860723-5352-0/minicluster-data/master-0-root/instance:
uuid: "54ac5f2af4044ee5ac6d51afda1bade2"
format_stamp: "Formatted at 2026-07-09 12:25:45 on dist-test-slave-xs2d"
I20260709 12:25:45.606356  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:25:45.610764 24653 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:25:45.611773  5352 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:25:45.612146  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599341860723-5352-0/minicluster-data/master-0-root
uuid: "54ac5f2af4044ee5ac6d51afda1bade2"
format_stamp: "Formatted at 2026-07-09 12:25:45 on dist-test-slave-xs2d"
I20260709 12:25:45.612568  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599341860723-5352-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599341860723-5352-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599341860723-5352-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:25:45.632535  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:25:45.634476  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:25:45.681365  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.62:37581
I20260709 12:25:45.681515 24704 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.62:37581 every 8 connection(s)
I20260709 12:25:45.685794 24705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:45.701620 24705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54ac5f2af4044ee5ac6d51afda1bade2: Bootstrap starting.
I20260709 12:25:45.707551 24705 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54ac5f2af4044ee5ac6d51afda1bade2: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:45.712932 24705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54ac5f2af4044ee5ac6d51afda1bade2: No bootstrap required, opened a new log
I20260709 12:25:45.715656 24705 raft_consensus.cc:359] T 00000000000000000000000000000000 P 54ac5f2af4044ee5ac6d51afda1bade2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54ac5f2af4044ee5ac6d51afda1bade2" member_type: VOTER }
I20260709 12:25:45.716144 24705 raft_consensus.cc:385] T 00000000000000000000000000000000 P 54ac5f2af4044ee5ac6d51afda1bade2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:45.716401 24705 raft_consensus.cc:740] T 00000000000000000000000000000000 P 54ac5f2af4044ee5ac6d51afda1bade2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54ac5f2af4044ee5ac6d51afda1bade2, State: Initialized, Role: FOLLOWER
I20260709 12:25:45.716984 24705 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54ac5f2af4044ee5ac6d51afda1bade2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54ac5f2af4044ee5ac6d51afda1bade2" member_type: VOTER }
I20260709 12:25:45.717450 24705 raft_consensus.cc:399] T 00000000000000000000000000000000 P 54ac5f2af4044ee5ac6d51afda1bade2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:25:45.717661 24705 raft_consensus.cc:493] T 00000000000000000000000000000000 P 54ac5f2af4044ee5ac6d51afda1bade2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:25:45.717911 24705 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 54ac5f2af4044ee5ac6d51afda1bade2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:45.723615 24705 raft_consensus.cc:515] T 00000000000000000000000000000000 P 54ac5f2af4044ee5ac6d51afda1bade2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54ac5f2af4044ee5ac6d51afda1bade2" member_type: VOTER }
I20260709 12:25:45.724179 24705 leader_election.cc:304] T 00000000000000000000000000000000 P 54ac5f2af4044ee5ac6d51afda1bade2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54ac5f2af4044ee5ac6d51afda1bade2; no voters: 
I20260709 12:25:45.725612 24705 leader_election.cc:290] T 00000000000000000000000000000000 P 54ac5f2af4044ee5ac6d51afda1bade2 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:25:45.726111 24708 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 54ac5f2af4044ee5ac6d51afda1bade2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:45.727669 24708 raft_consensus.cc:697] T 00000000000000000000000000000000 P 54ac5f2af4044ee5ac6d51afda1bade2 [term 1 LEADER]: Becoming Leader. State: Replica: 54ac5f2af4044ee5ac6d51afda1bade2, State: Running, Role: LEADER
I20260709 12:25:45.728554 24708 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54ac5f2af4044ee5ac6d51afda1bade2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54ac5f2af4044ee5ac6d51afda1bade2" member_type: VOTER }
I20260709 12:25:45.728693 24705 sys_catalog.cc:565] T 00000000000000000000000000000000 P 54ac5f2af4044ee5ac6d51afda1bade2 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:25:45.733969 24710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54ac5f2af4044ee5ac6d51afda1bade2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54ac5f2af4044ee5ac6d51afda1bade2. Latest consensus state: current_term: 1 leader_uuid: "54ac5f2af4044ee5ac6d51afda1bade2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54ac5f2af4044ee5ac6d51afda1bade2" member_type: VOTER } }
I20260709 12:25:45.733915 24709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54ac5f2af4044ee5ac6d51afda1bade2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54ac5f2af4044ee5ac6d51afda1bade2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54ac5f2af4044ee5ac6d51afda1bade2" member_type: VOTER } }
I20260709 12:25:45.734751 24710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54ac5f2af4044ee5ac6d51afda1bade2 [sys.catalog]: This master's current role is: LEADER
I20260709 12:25:45.734786 24709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54ac5f2af4044ee5ac6d51afda1bade2 [sys.catalog]: This master's current role is: LEADER
I20260709 12:25:45.738932 24716 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:25:45.746369 24716 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:25:45.747763  5352 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:25:45.757290 24716 catalog_manager.cc:1383] Generated new cluster ID: da3d802502df4d53a6532181e9f8a6e7
I20260709 12:25:45.757699 24716 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:25:45.779747 24716 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:25:45.781190 24716 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:25:45.798552 24716 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 54ac5f2af4044ee5ac6d51afda1bade2: Generated new TSK 0
I20260709 12:25:45.799197 24716 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:25:45.815102  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:25:45.822573 24726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:25:45.823539 24727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:25:45.825668  5352 server_base.cc:1061] running on GCE node
W20260709 12:25:45.826118 24729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:25:45.827629  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:25:45.827847  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:25:45.828015  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599945828002 us; error 0 us; skew 500 ppm
I20260709 12:25:45.828629  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:25:45.831323  5352 webserver.cc:533] Webserver started at http://127.5.58.1:43997/ using document root <none> and password file <none>
I20260709 12:25:45.831807  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:25:45.831988  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:25:45.832228  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:25:45.833542  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599341860723-5352-0/minicluster-data/ts-0-root/instance:
uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
format_stamp: "Formatted at 2026-07-09 12:25:45 on dist-test-slave-xs2d"
I20260709 12:25:45.839632  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:25:45.847477 24734 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:25:45.848731  5352 fs_manager.cc:730] Time spent opening block manager: real 0.006s	user 0.003s	sys 0.004s
I20260709 12:25:45.849231  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599341860723-5352-0/minicluster-data/ts-0-root
uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
format_stamp: "Formatted at 2026-07-09 12:25:45 on dist-test-slave-xs2d"
I20260709 12:25:45.849709  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599341860723-5352-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599341860723-5352-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599341860723-5352-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:25:45.871085  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:25:45.872453  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:25:45.874068  5352 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:25:45.876570  5352 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:25:45.876749  5352 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:25:45.877046  5352 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:25:45.877225  5352 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:25:45.929901  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.1:44285
I20260709 12:25:45.930059 24796 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.1:44285 every 8 connection(s)
I20260709 12:25:45.935382  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:25:45.945364 24802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:25:45.946802 24801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:25:45.946959 24804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:25:45.948529  5352 server_base.cc:1061] running on GCE node
I20260709 12:25:45.950697  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:25:45.950961  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:25:45.951210  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599945951187 us; error 0 us; skew 500 ppm
I20260709 12:25:45.951371 24797 heartbeater.cc:344] Connected to a master server at 127.5.58.62:37581
I20260709 12:25:45.951882 24797 heartbeater.cc:461] Registering TS with master...
I20260709 12:25:45.952207  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:25:45.953016 24797 heartbeater.cc:507] Master 127.5.58.62:37581 requested a full tablet report, sending...
I20260709 12:25:45.955780  5352 webserver.cc:533] Webserver started at http://127.5.58.2:42529/ using document root <none> and password file <none>
I20260709 12:25:45.955955 24670 ts_manager.cc:194] Registered new tserver with Master: 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285)
I20260709 12:25:45.956570  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:25:45.956969  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:25:45.957458  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:25:45.958421 24670 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42272
I20260709 12:25:45.958956  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599341860723-5352-0/minicluster-data/ts-1-root/instance:
uuid: "0b1901a1549f476696340f2fc61af194"
format_stamp: "Formatted at 2026-07-09 12:25:45 on dist-test-slave-xs2d"
I20260709 12:25:45.964807  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 12:25:45.968601 24809 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:25:45.969597  5352 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:25:45.969949  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599341860723-5352-0/minicluster-data/ts-1-root
uuid: "0b1901a1549f476696340f2fc61af194"
format_stamp: "Formatted at 2026-07-09 12:25:45 on dist-test-slave-xs2d"
I20260709 12:25:45.970295  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599341860723-5352-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599341860723-5352-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599341860723-5352-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:25:45.998677  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:25:46.000005  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:25:46.001613  5352 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:25:46.004000  5352 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:25:46.004257  5352 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:25:46.004572  5352 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:25:46.004750  5352 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:25:46.050577  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.2:38549
I20260709 12:25:46.050681 24871 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.2:38549 every 8 connection(s)
I20260709 12:25:46.055610  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:25:46.063925 24876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:25:46.064558 24875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:25:46.067499 24872 heartbeater.cc:344] Connected to a master server at 127.5.58.62:37581
I20260709 12:25:46.067992 24872 heartbeater.cc:461] Registering TS with master...
W20260709 12:25:46.068375 24878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:25:46.068634  5352 server_base.cc:1061] running on GCE node
I20260709 12:25:46.069046 24872 heartbeater.cc:507] Master 127.5.58.62:37581 requested a full tablet report, sending...
I20260709 12:25:46.069715  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:25:46.070003  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:25:46.070163  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599946070149 us; error 0 us; skew 500 ppm
I20260709 12:25:46.070797  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:25:46.071348 24670 ts_manager.cc:194] Registered new tserver with Master: 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:25:46.073374 24670 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42288
I20260709 12:25:46.073629  5352 webserver.cc:533] Webserver started at http://127.5.58.3:40649/ using document root <none> and password file <none>
I20260709 12:25:46.074272  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:25:46.074512  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:25:46.074851  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:25:46.076447  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599341860723-5352-0/minicluster-data/ts-2-root/instance:
uuid: "2807a439752a4b0ab39c82cdef37f499"
format_stamp: "Formatted at 2026-07-09 12:25:46 on dist-test-slave-xs2d"
I20260709 12:25:46.081508  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.001s
I20260709 12:25:46.085106 24883 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:25:46.086066  5352 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:25:46.086385  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599341860723-5352-0/minicluster-data/ts-2-root
uuid: "2807a439752a4b0ab39c82cdef37f499"
format_stamp: "Formatted at 2026-07-09 12:25:46 on dist-test-slave-xs2d"
I20260709 12:25:46.086715  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599341860723-5352-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599341860723-5352-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599341860723-5352-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:25:46.106365  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:25:46.107681  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:25:46.109376  5352 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:25:46.111624  5352 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:25:46.111809  5352 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:25:46.111987  5352 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:25:46.112113  5352 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:25:46.162881  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.3:45247
I20260709 12:25:46.163295 24945 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.3:45247 every 8 connection(s)
I20260709 12:25:46.179294 24946 heartbeater.cc:344] Connected to a master server at 127.5.58.62:37581
I20260709 12:25:46.179864 24946 heartbeater.cc:461] Registering TS with master...
I20260709 12:25:46.181140 24946 heartbeater.cc:507] Master 127.5.58.62:37581 requested a full tablet report, sending...
I20260709 12:25:46.183584 24670 ts_manager.cc:194] Registered new tserver with Master: 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:46.184610  5352 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018430149s
I20260709 12:25:46.185242 24670 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42300
I20260709 12:25:46.217658 24670 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42316:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:25:46.220976 24670 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:25:46.302207 24911 tablet_service.cc:1511] Processing CreateTablet for tablet 3f69b65186ed4f6c8fe7c50f6b0133d3 (DEFAULT_TABLE table=test-workload [id=e45954aede7b4ff6bd408627994a4351]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:25:46.303277 24910 tablet_service.cc:1511] Processing CreateTablet for tablet 97a81d7f4ccb459ab2e722f09c10d9a1 (DEFAULT_TABLE table=test-workload [id=e45954aede7b4ff6bd408627994a4351]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:25:46.304067 24911 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f69b65186ed4f6c8fe7c50f6b0133d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:46.304905 24910 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97a81d7f4ccb459ab2e722f09c10d9a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:46.305534 24761 tablet_service.cc:1511] Processing CreateTablet for tablet 97a81d7f4ccb459ab2e722f09c10d9a1 (DEFAULT_TABLE table=test-workload [id=e45954aede7b4ff6bd408627994a4351]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:25:46.306411 24760 tablet_service.cc:1511] Processing CreateTablet for tablet f1d7d9ea11c24a428a3ca7a20b11e81c (DEFAULT_TABLE table=test-workload [id=e45954aede7b4ff6bd408627994a4351]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:25:46.307013 24761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97a81d7f4ccb459ab2e722f09c10d9a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:46.308104 24760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1d7d9ea11c24a428a3ca7a20b11e81c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:46.309130 24762 tablet_service.cc:1511] Processing CreateTablet for tablet 3f69b65186ed4f6c8fe7c50f6b0133d3 (DEFAULT_TABLE table=test-workload [id=e45954aede7b4ff6bd408627994a4351]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:25:46.309629 24909 tablet_service.cc:1511] Processing CreateTablet for tablet f1d7d9ea11c24a428a3ca7a20b11e81c (DEFAULT_TABLE table=test-workload [id=e45954aede7b4ff6bd408627994a4351]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:25:46.310621 24762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f69b65186ed4f6c8fe7c50f6b0133d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:46.311018 24909 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1d7d9ea11c24a428a3ca7a20b11e81c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:46.313932 24908 tablet_service.cc:1511] Processing CreateTablet for tablet 3f803a6e48f74d26b826aa57f9233ee9 (DEFAULT_TABLE table=test-workload [id=e45954aede7b4ff6bd408627994a4351]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:25:46.315507 24908 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f803a6e48f74d26b826aa57f9233ee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:46.322161 24759 tablet_service.cc:1511] Processing CreateTablet for tablet 3f803a6e48f74d26b826aa57f9233ee9 (DEFAULT_TABLE table=test-workload [id=e45954aede7b4ff6bd408627994a4351]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:25:46.322925 24758 tablet_service.cc:1511] Processing CreateTablet for tablet 36fd425a2ae24bba8cc5934237f465b6 (DEFAULT_TABLE table=test-workload [id=e45954aede7b4ff6bd408627994a4351]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:25:46.323768 24759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f803a6e48f74d26b826aa57f9233ee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:46.324522 24758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36fd425a2ae24bba8cc5934237f465b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:46.346369 24907 tablet_service.cc:1511] Processing CreateTablet for tablet 36fd425a2ae24bba8cc5934237f465b6 (DEFAULT_TABLE table=test-workload [id=e45954aede7b4ff6bd408627994a4351]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:25:46.348026 24907 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36fd425a2ae24bba8cc5934237f465b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:46.349571 24962 tablet_bootstrap.cc:492] T 3f803a6e48f74d26b826aa57f9233ee9 P 6f825b5c64844f7fb020b60f13e6fdb4: Bootstrap starting.
I20260709 12:25:46.352195 24836 tablet_service.cc:1511] Processing CreateTablet for tablet 97a81d7f4ccb459ab2e722f09c10d9a1 (DEFAULT_TABLE table=test-workload [id=e45954aede7b4ff6bd408627994a4351]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:25:46.353449 24833 tablet_service.cc:1511] Processing CreateTablet for tablet 36fd425a2ae24bba8cc5934237f465b6 (DEFAULT_TABLE table=test-workload [id=e45954aede7b4ff6bd408627994a4351]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:25:46.355789 24835 tablet_service.cc:1511] Processing CreateTablet for tablet f1d7d9ea11c24a428a3ca7a20b11e81c (DEFAULT_TABLE table=test-workload [id=e45954aede7b4ff6bd408627994a4351]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:25:46.356359 24834 tablet_service.cc:1511] Processing CreateTablet for tablet 3f803a6e48f74d26b826aa57f9233ee9 (DEFAULT_TABLE table=test-workload [id=e45954aede7b4ff6bd408627994a4351]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:25:46.357414 24962 tablet_bootstrap.cc:654] T 3f803a6e48f74d26b826aa57f9233ee9 P 6f825b5c64844f7fb020b60f13e6fdb4: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:46.358906 24837 tablet_service.cc:1511] Processing CreateTablet for tablet 3f69b65186ed4f6c8fe7c50f6b0133d3 (DEFAULT_TABLE table=test-workload [id=e45954aede7b4ff6bd408627994a4351]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:25:46.363540 24759 tablet_service.cc:1511] Processing CreateTablet for tablet 89fd2ebc3fe5461c93eac233d095a6df (DEFAULT_TABLE table=test-workload [id=e45954aede7b4ff6bd408627994a4351]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:25:46.353992 24836 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97a81d7f4ccb459ab2e722f09c10d9a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:46.365110 24759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89fd2ebc3fe5461c93eac233d095a6df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:46.369022 24835 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1d7d9ea11c24a428a3ca7a20b11e81c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:46.369796 24833 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36fd425a2ae24bba8cc5934237f465b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:46.372690 24837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f69b65186ed4f6c8fe7c50f6b0133d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:46.372814 24910 tablet_service.cc:1511] Processing CreateTablet for tablet 89fd2ebc3fe5461c93eac233d095a6df (DEFAULT_TABLE table=test-workload [id=e45954aede7b4ff6bd408627994a4351]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:25:46.374185 24910 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89fd2ebc3fe5461c93eac233d095a6df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:46.375485 24832 tablet_service.cc:1511] Processing CreateTablet for tablet 89fd2ebc3fe5461c93eac233d095a6df (DEFAULT_TABLE table=test-workload [id=e45954aede7b4ff6bd408627994a4351]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:25:46.376681 24834 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f803a6e48f74d26b826aa57f9233ee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:46.380283 24832 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89fd2ebc3fe5461c93eac233d095a6df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:46.392455 24831 tablet_service.cc:1511] Processing CreateTablet for tablet 1df8831adbf14c3594dea358a98bfff3 (DEFAULT_TABLE table=test-workload [id=e45954aede7b4ff6bd408627994a4351]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:25:46.394048 24831 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1df8831adbf14c3594dea358a98bfff3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:46.402103 24908 tablet_service.cc:1511] Processing CreateTablet for tablet 1df8831adbf14c3594dea358a98bfff3 (DEFAULT_TABLE table=test-workload [id=e45954aede7b4ff6bd408627994a4351]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:25:46.403592 24908 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1df8831adbf14c3594dea358a98bfff3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:46.393620 24762 tablet_service.cc:1511] Processing CreateTablet for tablet 1df8831adbf14c3594dea358a98bfff3 (DEFAULT_TABLE table=test-workload [id=e45954aede7b4ff6bd408627994a4351]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:25:46.409312 24762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1df8831adbf14c3594dea358a98bfff3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:46.415297 24830 tablet_service.cc:1511] Processing CreateTablet for tablet 8f0eec1401e842eeb0541c2dd6621e58 (DEFAULT_TABLE table=test-workload [id=e45954aede7b4ff6bd408627994a4351]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:25:46.416841 24909 tablet_service.cc:1511] Processing CreateTablet for tablet 8f0eec1401e842eeb0541c2dd6621e58 (DEFAULT_TABLE table=test-workload [id=e45954aede7b4ff6bd408627994a4351]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:25:46.418278 24909 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f0eec1401e842eeb0541c2dd6621e58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:46.420477 24830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f0eec1401e842eeb0541c2dd6621e58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:46.341943 24961 tablet_bootstrap.cc:492] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499: Bootstrap starting.
I20260709 12:25:46.430831 24962 tablet_bootstrap.cc:492] T 3f803a6e48f74d26b826aa57f9233ee9 P 6f825b5c64844f7fb020b60f13e6fdb4: No bootstrap required, opened a new log
I20260709 12:25:46.431418 24962 ts_tablet_manager.cc:1403] T 3f803a6e48f74d26b826aa57f9233ee9 P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent bootstrapping tablet: real 0.082s	user 0.012s	sys 0.000s
I20260709 12:25:46.436792 24961 tablet_bootstrap.cc:654] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:46.449429 24761 tablet_service.cc:1511] Processing CreateTablet for tablet 8f0eec1401e842eeb0541c2dd6621e58 (DEFAULT_TABLE table=test-workload [id=e45954aede7b4ff6bd408627994a4351]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:25:46.457351 24758 tablet_service.cc:1511] Processing CreateTablet for tablet 357a8c2b04ca46349006b7487ed1409e (DEFAULT_TABLE table=test-workload [id=e45954aede7b4ff6bd408627994a4351]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:25:46.458067 24836 tablet_service.cc:1511] Processing CreateTablet for tablet 357a8c2b04ca46349006b7487ed1409e (DEFAULT_TABLE table=test-workload [id=e45954aede7b4ff6bd408627994a4351]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:25:46.458743 24758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 357a8c2b04ca46349006b7487ed1409e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:46.459424 24761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f0eec1401e842eeb0541c2dd6621e58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:46.459780 24836 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 357a8c2b04ca46349006b7487ed1409e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:46.466770 24911 tablet_service.cc:1511] Processing CreateTablet for tablet 357a8c2b04ca46349006b7487ed1409e (DEFAULT_TABLE table=test-workload [id=e45954aede7b4ff6bd408627994a4351]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:25:46.468283 24911 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 357a8c2b04ca46349006b7487ed1409e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:46.475564 24962 raft_consensus.cc:359] T 3f803a6e48f74d26b826aa57f9233ee9 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.476450 24962 raft_consensus.cc:385] T 3f803a6e48f74d26b826aa57f9233ee9 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:46.476770 24962 raft_consensus.cc:740] T 3f803a6e48f74d26b826aa57f9233ee9 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Initialized, Role: FOLLOWER
I20260709 12:25:46.477604 24962 consensus_queue.cc:260] T 3f803a6e48f74d26b826aa57f9233ee9 P 6f825b5c64844f7fb020b60f13e6fdb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.478991 24961 tablet_bootstrap.cc:492] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499: No bootstrap required, opened a new log
I20260709 12:25:46.479768 24961 ts_tablet_manager.cc:1403] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499: Time spent bootstrapping tablet: real 0.138s	user 0.021s	sys 0.015s
I20260709 12:25:46.483945 24961 raft_consensus.cc:359] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } }
I20260709 12:25:46.485054 24961 raft_consensus.cc:385] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:46.485496 24961 raft_consensus.cc:740] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Initialized, Role: FOLLOWER
I20260709 12:25:46.486680 24961 consensus_queue.cc:260] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } }
I20260709 12:25:46.503863 24965 tablet_bootstrap.cc:492] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194: Bootstrap starting.
I20260709 12:25:46.508497 24797 heartbeater.cc:499] Master 127.5.58.62:37581 was elected leader, sending a full tablet report...
I20260709 12:25:46.511574 24965 tablet_bootstrap.cc:654] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:46.527945 24946 heartbeater.cc:499] Master 127.5.58.62:37581 was elected leader, sending a full tablet report...
I20260709 12:25:46.528532 24961 ts_tablet_manager.cc:1434] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499: Time spent starting tablet: real 0.048s	user 0.015s	sys 0.025s
I20260709 12:25:46.529891 24961 tablet_bootstrap.cc:492] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499: Bootstrap starting.
I20260709 12:25:46.525852 24962 ts_tablet_manager.cc:1434] T 3f803a6e48f74d26b826aa57f9233ee9 P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent starting tablet: real 0.094s	user 0.009s	sys 0.000s
I20260709 12:25:46.536020 24965 tablet_bootstrap.cc:492] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194: No bootstrap required, opened a new log
I20260709 12:25:46.536695 24965 ts_tablet_manager.cc:1403] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194: Time spent bootstrapping tablet: real 0.033s	user 0.003s	sys 0.015s
I20260709 12:25:46.538156 24961 tablet_bootstrap.cc:654] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:46.539844 24962 tablet_bootstrap.cc:492] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4: Bootstrap starting.
I20260709 12:25:46.540472 24965 raft_consensus.cc:359] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.541620 24965 raft_consensus.cc:385] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:46.542058 24965 raft_consensus.cc:740] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Initialized, Role: FOLLOWER
I20260709 12:25:46.543133 24965 consensus_queue.cc:260] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.547978 24962 tablet_bootstrap.cc:654] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:46.551551 24961 tablet_bootstrap.cc:492] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499: No bootstrap required, opened a new log
I20260709 12:25:46.552316 24961 ts_tablet_manager.cc:1403] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499: Time spent bootstrapping tablet: real 0.023s	user 0.018s	sys 0.000s
I20260709 12:25:46.558339 24965 ts_tablet_manager.cc:1434] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.012s
I20260709 12:25:46.557986 24961 raft_consensus.cc:359] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.558854 24961 raft_consensus.cc:385] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:46.559196 24961 raft_consensus.cc:740] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Initialized, Role: FOLLOWER
I20260709 12:25:46.559985 24961 consensus_queue.cc:260] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.560760 24872 heartbeater.cc:499] Master 127.5.58.62:37581 was elected leader, sending a full tablet report...
I20260709 12:25:46.562306 24961 ts_tablet_manager.cc:1434] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:25:46.563527 24961 tablet_bootstrap.cc:492] T 3f803a6e48f74d26b826aa57f9233ee9 P 2807a439752a4b0ab39c82cdef37f499: Bootstrap starting.
I20260709 12:25:46.572748 24961 tablet_bootstrap.cc:654] T 3f803a6e48f74d26b826aa57f9233ee9 P 2807a439752a4b0ab39c82cdef37f499: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:46.574790 24965 tablet_bootstrap.cc:492] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 0b1901a1549f476696340f2fc61af194: Bootstrap starting.
I20260709 12:25:46.585387 24965 tablet_bootstrap.cc:654] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 0b1901a1549f476696340f2fc61af194: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:46.589169 24961 tablet_bootstrap.cc:492] T 3f803a6e48f74d26b826aa57f9233ee9 P 2807a439752a4b0ab39c82cdef37f499: No bootstrap required, opened a new log
I20260709 12:25:46.589787 24961 ts_tablet_manager.cc:1403] T 3f803a6e48f74d26b826aa57f9233ee9 P 2807a439752a4b0ab39c82cdef37f499: Time spent bootstrapping tablet: real 0.026s	user 0.020s	sys 0.005s
I20260709 12:25:46.594197 24961 raft_consensus.cc:359] T 3f803a6e48f74d26b826aa57f9233ee9 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.595335 24961 raft_consensus.cc:385] T 3f803a6e48f74d26b826aa57f9233ee9 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:46.595711 24961 raft_consensus.cc:740] T 3f803a6e48f74d26b826aa57f9233ee9 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Initialized, Role: FOLLOWER
I20260709 12:25:46.596807 24961 consensus_queue.cc:260] T 3f803a6e48f74d26b826aa57f9233ee9 P 2807a439752a4b0ab39c82cdef37f499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.598757 24965 tablet_bootstrap.cc:492] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 0b1901a1549f476696340f2fc61af194: No bootstrap required, opened a new log
I20260709 12:25:46.599385 24965 ts_tablet_manager.cc:1403] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 0b1901a1549f476696340f2fc61af194: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.004s
I20260709 12:25:46.601549 24961 ts_tablet_manager.cc:1434] T 3f803a6e48f74d26b826aa57f9233ee9 P 2807a439752a4b0ab39c82cdef37f499: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.004s
I20260709 12:25:46.602814 24961 tablet_bootstrap.cc:492] T 89fd2ebc3fe5461c93eac233d095a6df P 2807a439752a4b0ab39c82cdef37f499: Bootstrap starting.
I20260709 12:25:46.602785 24965 raft_consensus.cc:359] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.603860 24965 raft_consensus.cc:385] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:46.604204 24965 raft_consensus.cc:740] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Initialized, Role: FOLLOWER
I20260709 12:25:46.605264 24962 tablet_bootstrap.cc:492] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4: No bootstrap required, opened a new log
I20260709 12:25:46.606014 24962 ts_tablet_manager.cc:1403] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent bootstrapping tablet: real 0.066s	user 0.031s	sys 0.026s
I20260709 12:25:46.605363 24965 consensus_queue.cc:260] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 0b1901a1549f476696340f2fc61af194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.608213 24965 ts_tablet_manager.cc:1434] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 0b1901a1549f476696340f2fc61af194: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 12:25:46.609301 24965 tablet_bootstrap.cc:492] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194: Bootstrap starting.
I20260709 12:25:46.609938 24962 raft_consensus.cc:359] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.610646 24961 tablet_bootstrap.cc:654] T 89fd2ebc3fe5461c93eac233d095a6df P 2807a439752a4b0ab39c82cdef37f499: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:46.610877 24962 raft_consensus.cc:385] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:46.611382 24962 raft_consensus.cc:740] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Initialized, Role: FOLLOWER
I20260709 12:25:46.612548 24962 consensus_queue.cc:260] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.617776 24962 ts_tablet_manager.cc:1434] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:25:46.617847 24965 tablet_bootstrap.cc:654] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:46.619204 24962 tablet_bootstrap.cc:492] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4: Bootstrap starting.
I20260709 12:25:46.625344 24961 tablet_bootstrap.cc:492] T 89fd2ebc3fe5461c93eac233d095a6df P 2807a439752a4b0ab39c82cdef37f499: No bootstrap required, opened a new log
I20260709 12:25:46.626156 24961 ts_tablet_manager.cc:1403] T 89fd2ebc3fe5461c93eac233d095a6df P 2807a439752a4b0ab39c82cdef37f499: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.011s
I20260709 12:25:46.627976 24962 tablet_bootstrap.cc:654] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:46.630251 24961 raft_consensus.cc:359] T 89fd2ebc3fe5461c93eac233d095a6df P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.631412 24961 raft_consensus.cc:385] T 89fd2ebc3fe5461c93eac233d095a6df P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:46.631799 24961 raft_consensus.cc:740] T 89fd2ebc3fe5461c93eac233d095a6df P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Initialized, Role: FOLLOWER
I20260709 12:25:46.633119 24961 consensus_queue.cc:260] T 89fd2ebc3fe5461c93eac233d095a6df P 2807a439752a4b0ab39c82cdef37f499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.636585 24961 ts_tablet_manager.cc:1434] T 89fd2ebc3fe5461c93eac233d095a6df P 2807a439752a4b0ab39c82cdef37f499: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.004s
I20260709 12:25:46.637970 24961 tablet_bootstrap.cc:492] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499: Bootstrap starting.
I20260709 12:25:46.641969 24962 tablet_bootstrap.cc:492] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4: No bootstrap required, opened a new log
I20260709 12:25:46.642879 24962 ts_tablet_manager.cc:1403] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent bootstrapping tablet: real 0.024s	user 0.019s	sys 0.003s
I20260709 12:25:46.644109 24965 tablet_bootstrap.cc:492] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194: No bootstrap required, opened a new log
I20260709 12:25:46.644924 24965 ts_tablet_manager.cc:1403] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194: Time spent bootstrapping tablet: real 0.036s	user 0.023s	sys 0.009s
I20260709 12:25:46.647918 24962 raft_consensus.cc:359] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } }
I20260709 12:25:46.648694 24961 tablet_bootstrap.cc:654] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:46.649289 24962 raft_consensus.cc:385] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:46.648672 24965 raft_consensus.cc:359] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.649811 24962 raft_consensus.cc:740] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Initialized, Role: FOLLOWER
I20260709 12:25:46.650022 24965 raft_consensus.cc:385] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:46.650434 24965 raft_consensus.cc:740] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Initialized, Role: FOLLOWER
I20260709 12:25:46.650831 24962 consensus_queue.cc:260] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } }
I20260709 12:25:46.651517 24965 consensus_queue.cc:260] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.653404 24962 ts_tablet_manager.cc:1434] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:25:46.654734 24962 tablet_bootstrap.cc:492] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4: Bootstrap starting.
I20260709 12:25:46.665408 24962 tablet_bootstrap.cc:654] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:46.667505 24965 ts_tablet_manager.cc:1434] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 12:25:46.670328 24970 raft_consensus.cc:493] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.671015 24967 raft_consensus.cc:493] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.671147 24970 raft_consensus.cc:515] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.671573 24967 raft_consensus.cc:515] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.676266 24970 leader_election.cc:290] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:46.678381 24967 leader_election.cc:290] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:25:46.684999 24965 tablet_bootstrap.cc:492] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 0b1901a1549f476696340f2fc61af194: Bootstrap starting.
I20260709 12:25:46.701262 24970 raft_consensus.cc:493] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.702363 24970 raft_consensus.cc:515] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.721408 24961 tablet_bootstrap.cc:492] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499: No bootstrap required, opened a new log
I20260709 12:25:46.721907 24965 tablet_bootstrap.cc:654] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 0b1901a1549f476696340f2fc61af194: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:46.722090 24970 leader_election.cc:290] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:46.722252 24961 ts_tablet_manager.cc:1403] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499: Time spent bootstrapping tablet: real 0.085s	user 0.032s	sys 0.033s
I20260709 12:25:46.724980 24847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a81d7f4ccb459ab2e722f09c10d9a1" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194" is_pre_election: true
I20260709 12:25:46.726089 24847 raft_consensus.cc:2468] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 0.
I20260709 12:25:46.726580 24961 raft_consensus.cc:359] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.727586 24961 raft_consensus.cc:385] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:46.727908 24961 raft_consensus.cc:740] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Initialized, Role: FOLLOWER
I20260709 12:25:46.721408 24962 tablet_bootstrap.cc:492] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4: No bootstrap required, opened a new log
I20260709 12:25:46.728976 24962 ts_tablet_manager.cc:1403] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent bootstrapping tablet: real 0.075s	user 0.021s	sys 0.004s
I20260709 12:25:46.730769 24772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a81d7f4ccb459ab2e722f09c10d9a1" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" is_pre_election: true
I20260709 12:25:46.733270 24962 raft_consensus.cc:359] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.733927 24887 leader_election.cc:304] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 2807a439752a4b0ab39c82cdef37f499; no voters: 
I20260709 12:25:46.734982 24961 consensus_queue.cc:260] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.734279 24962 raft_consensus.cc:385] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:46.736043 24962 raft_consensus.cc:740] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Initialized, Role: FOLLOWER
I20260709 12:25:46.737355 24962 consensus_queue.cc:260] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
W20260709 12:25:46.741583 24718 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:25:46.742091 24962 ts_tablet_manager.cc:1434] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent starting tablet: real 0.013s	user 0.011s	sys 0.000s
I20260709 12:25:46.743059 24772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df8831adbf14c3594dea358a98bfff3" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" is_pre_election: true
I20260709 12:25:46.744791 24962 tablet_bootstrap.cc:492] T 1df8831adbf14c3594dea358a98bfff3 P 6f825b5c64844f7fb020b60f13e6fdb4: Bootstrap starting.
I20260709 12:25:46.745519 24772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36fd425a2ae24bba8cc5934237f465b6" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" is_pre_election: true
I20260709 12:25:46.746260 24772 raft_consensus.cc:2468] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b1901a1549f476696340f2fc61af194 in term 0.
W20260709 12:25:46.747614 24813 leader_election.cc:343] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:46.748641 24813 leader_election.cc:304] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:46.749715 24970 raft_consensus.cc:2804] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:25:46.750123 24970 raft_consensus.cc:493] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.750490 24970 raft_consensus.cc:3060] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:46.760876 24962 tablet_bootstrap.cc:654] T 1df8831adbf14c3594dea358a98bfff3 P 6f825b5c64844f7fb020b60f13e6fdb4: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:46.761126 24970 raft_consensus.cc:515] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.765580 24961 ts_tablet_manager.cc:1434] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.001s
I20260709 12:25:46.766808 24967 raft_consensus.cc:2804] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:25:46.767349 24967 raft_consensus.cc:493] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.767758 24967 raft_consensus.cc:3060] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:46.769177 24961 tablet_bootstrap.cc:492] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499: Bootstrap starting.
I20260709 12:25:46.769227 24921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df8831adbf14c3594dea358a98bfff3" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499" is_pre_election: true
W20260709 12:25:46.774101 24810 leader_election.cc:343] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:46.774637 24810 leader_election.cc:304] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0b1901a1549f476696340f2fc61af194; no voters: 2807a439752a4b0ab39c82cdef37f499, 6f825b5c64844f7fb020b60f13e6fdb4
I20260709 12:25:46.769275 24920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36fd425a2ae24bba8cc5934237f465b6" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499" is_pre_election: true
I20260709 12:25:46.776368 24772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36fd425a2ae24bba8cc5934237f465b6" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
I20260709 12:25:46.777289 24772 raft_consensus.cc:3060] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:46.780733 24967 raft_consensus.cc:515] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.783165 24967 leader_election.cc:290] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 election: Requested vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
W20260709 12:25:46.788704 24887 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:38549, user_credentials={real_user=slave}} blocked reactor thread for 55146.2us
I20260709 12:25:46.789745 24970 leader_election.cc:290] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 election: Requested vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:46.790522 24970 raft_consensus.cc:2749] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:25:46.791683 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a81d7f4ccb459ab2e722f09c10d9a1" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
I20260709 12:25:46.792521 24847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a81d7f4ccb459ab2e722f09c10d9a1" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194"
W20260709 12:25:46.792872 24887 leader_election.cc:343] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:46.792532 24771 raft_consensus.cc:3060] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:46.793901 24772 raft_consensus.cc:2468] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b1901a1549f476696340f2fc61af194 in term 1.
I20260709 12:25:46.792773 24984 raft_consensus.cc:493] T 89fd2ebc3fe5461c93eac233d095a6df P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.795574 24813 leader_election.cc:304] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:46.796559 24970 raft_consensus.cc:2804] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:46.795141 24984 raft_consensus.cc:515] T 89fd2ebc3fe5461c93eac233d095a6df P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.797257 24970 raft_consensus.cc:697] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [term 1 LEADER]: Becoming Leader. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Running, Role: LEADER
I20260709 12:25:46.793287 24847 raft_consensus.cc:3060] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:46.798352 24965 tablet_bootstrap.cc:492] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 0b1901a1549f476696340f2fc61af194: No bootstrap required, opened a new log
I20260709 12:25:46.798981 24984 leader_election.cc:290] T 89fd2ebc3fe5461c93eac233d095a6df P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:25:46.799192 24965 ts_tablet_manager.cc:1403] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 0b1901a1549f476696340f2fc61af194: Time spent bootstrapping tablet: real 0.114s	user 0.024s	sys 0.034s
I20260709 12:25:46.798359 24970 consensus_queue.cc:237] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.801841 24772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89fd2ebc3fe5461c93eac233d095a6df" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" is_pre_election: true
I20260709 12:25:46.803635 24965 raft_consensus.cc:359] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } }
I20260709 12:25:46.805085 24965 raft_consensus.cc:385] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:46.805441 24965 raft_consensus.cc:740] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Initialized, Role: FOLLOWER
W20260709 12:25:46.806155 24887 leader_election.cc:343] T 89fd2ebc3fe5461c93eac233d095a6df P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:46.806280 24846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89fd2ebc3fe5461c93eac233d095a6df" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194" is_pre_election: true
I20260709 12:25:46.806471 24965 consensus_queue.cc:260] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 0b1901a1549f476696340f2fc61af194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } }
W20260709 12:25:46.810212 24887 leader_election.cc:343] T 89fd2ebc3fe5461c93eac233d095a6df P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:46.811987 24887 leader_election.cc:304] T 89fd2ebc3fe5461c93eac233d095a6df P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2807a439752a4b0ab39c82cdef37f499; no voters: 0b1901a1549f476696340f2fc61af194, 6f825b5c64844f7fb020b60f13e6fdb4
I20260709 12:25:46.812670 24771 raft_consensus.cc:2468] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 1.
I20260709 12:25:46.813434 24961 tablet_bootstrap.cc:654] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:46.813462 24984 raft_consensus.cc:493] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.815711 24967 raft_consensus.cc:2749] T 89fd2ebc3fe5461c93eac233d095a6df P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:25:46.816177 24887 leader_election.cc:304] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2807a439752a4b0ab39c82cdef37f499, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:46.816229 24984 raft_consensus.cc:515] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } }
I20260709 12:25:46.817121 24967 raft_consensus.cc:2804] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:46.814260 24847 raft_consensus.cc:2468] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 1.
I20260709 12:25:46.819851 24920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36fd425a2ae24bba8cc5934237f465b6" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
I20260709 12:25:46.821870 24965 ts_tablet_manager.cc:1434] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 0b1901a1549f476696340f2fc61af194: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.004s
I20260709 12:25:46.823447 24965 tablet_bootstrap.cc:492] T 3f803a6e48f74d26b826aa57f9233ee9 P 0b1901a1549f476696340f2fc61af194: Bootstrap starting.
I20260709 12:25:46.831881 24962 tablet_bootstrap.cc:492] T 1df8831adbf14c3594dea358a98bfff3 P 6f825b5c64844f7fb020b60f13e6fdb4: No bootstrap required, opened a new log
I20260709 12:25:46.838013 24962 ts_tablet_manager.cc:1403] T 1df8831adbf14c3594dea358a98bfff3 P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent bootstrapping tablet: real 0.095s	user 0.006s	sys 0.017s
I20260709 12:25:46.839341 24966 raft_consensus.cc:493] T 3f803a6e48f74d26b826aa57f9233ee9 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.832547 24847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f69b65186ed4f6c8fe7c50f6b0133d3" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194" is_pre_election: true
I20260709 12:25:46.835533 24967 raft_consensus.cc:697] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Becoming Leader. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Running, Role: LEADER
I20260709 12:25:46.830736 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f69b65186ed4f6c8fe7c50f6b0133d3" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" is_pre_election: true
I20260709 12:25:46.840766 24847 raft_consensus.cc:2468] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 0.
I20260709 12:25:46.840890 24771 raft_consensus.cc:2468] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 0.
W20260709 12:25:46.829998 24810 leader_election.cc:343] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:46.839924 24966 raft_consensus.cc:515] T 3f803a6e48f74d26b826aa57f9233ee9 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.842478 24962 raft_consensus.cc:359] T 1df8831adbf14c3594dea358a98bfff3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.841964 24670 catalog_manager.cc:5697] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b1901a1549f476696340f2fc61af194 (127.5.58.2). New cstate: current_term: 1 leader_uuid: "0b1901a1549f476696340f2fc61af194" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:46.843254 24962 raft_consensus.cc:385] T 1df8831adbf14c3594dea358a98bfff3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:46.843644 24962 raft_consensus.cc:740] T 1df8831adbf14c3594dea358a98bfff3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Initialized, Role: FOLLOWER
W20260709 12:25:46.842913 24810 leader_election.cc:343] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:46.823558 24984 leader_election.cc:290] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:25:46.844640 24962 consensus_queue.cc:260] T 1df8831adbf14c3594dea358a98bfff3 P 6f825b5c64844f7fb020b60f13e6fdb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.843528 24887 leader_election.cc:304] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 2807a439752a4b0ab39c82cdef37f499; no voters: 
I20260709 12:25:46.851447 24966 leader_election.cc:290] T 3f803a6e48f74d26b826aa57f9233ee9 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:46.854597 24961 tablet_bootstrap.cc:492] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499: No bootstrap required, opened a new log
I20260709 12:25:46.855192 24961 ts_tablet_manager.cc:1403] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499: Time spent bootstrapping tablet: real 0.087s	user 0.014s	sys 0.006s
I20260709 12:25:46.860133 24984 raft_consensus.cc:2804] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:25:46.860697 24984 raft_consensus.cc:493] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.861037 24984 raft_consensus.cc:3060] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:46.861430 24967 consensus_queue.cc:237] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.860215 24961 raft_consensus.cc:359] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.863497 24961 raft_consensus.cc:385] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:46.865551 24961 raft_consensus.cc:740] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Initialized, Role: FOLLOWER
I20260709 12:25:46.866587 24961 consensus_queue.cc:260] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.869166 24965 tablet_bootstrap.cc:654] T 3f803a6e48f74d26b826aa57f9233ee9 P 0b1901a1549f476696340f2fc61af194: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:46.873958 24961 ts_tablet_manager.cc:1434] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 12:25:46.877435 24961 tablet_bootstrap.cc:492] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499: Bootstrap starting.
I20260709 12:25:46.881117 24984 raft_consensus.cc:515] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } }
I20260709 12:25:46.884246 24669 catalog_manager.cc:5697] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "2807a439752a4b0ab39c82cdef37f499" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:46.885558 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f69b65186ed4f6c8fe7c50f6b0133d3" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
I20260709 12:25:46.886189 24771 raft_consensus.cc:3060] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:46.887842 24961 tablet_bootstrap.cc:654] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:46.895731 24771 raft_consensus.cc:2468] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 1.
I20260709 12:25:46.897083 24920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f803a6e48f74d26b826aa57f9233ee9" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499" is_pre_election: true
I20260709 12:25:46.898195 24920 raft_consensus.cc:2468] T 3f803a6e48f74d26b826aa57f9233ee9 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f825b5c64844f7fb020b60f13e6fdb4 in term 0.
I20260709 12:25:46.900363 24735 leader_election.cc:304] T 3f803a6e48f74d26b826aa57f9233ee9 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2807a439752a4b0ab39c82cdef37f499, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:46.901738 24966 raft_consensus.cc:2804] T 3f803a6e48f74d26b826aa57f9233ee9 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:25:46.902235 24966 raft_consensus.cc:493] T 3f803a6e48f74d26b826aa57f9233ee9 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.902683 24966 raft_consensus.cc:3060] T 3f803a6e48f74d26b826aa57f9233ee9 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:46.915933 24887 leader_election.cc:304] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2807a439752a4b0ab39c82cdef37f499, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:46.918371 24967 raft_consensus.cc:2804] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:46.919028 24967 raft_consensus.cc:697] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Becoming Leader. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Running, Role: LEADER
I20260709 12:25:46.919991 24984 leader_election.cc:290] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 election: Requested vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:25:46.919926 24967 consensus_queue.cc:237] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } }
I20260709 12:25:46.922456 24847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f69b65186ed4f6c8fe7c50f6b0133d3" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194"
I20260709 12:25:46.923305 24847 raft_consensus.cc:3060] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:46.914659 24966 raft_consensus.cc:515] T 3f803a6e48f74d26b826aa57f9233ee9 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.934100 24966 leader_election.cc:290] T 3f803a6e48f74d26b826aa57f9233ee9 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 1 election: Requested vote from peers 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:46.935595 24920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f803a6e48f74d26b826aa57f9233ee9" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
I20260709 12:25:46.936480 24920 raft_consensus.cc:3060] T 3f803a6e48f74d26b826aa57f9233ee9 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:46.941601 24847 raft_consensus.cc:2468] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 1.
I20260709 12:25:46.948457 24920 raft_consensus.cc:2468] T 3f803a6e48f74d26b826aa57f9233ee9 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f825b5c64844f7fb020b60f13e6fdb4 in term 1.
I20260709 12:25:46.950443 24735 leader_election.cc:304] T 3f803a6e48f74d26b826aa57f9233ee9 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2807a439752a4b0ab39c82cdef37f499, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:46.951535 24965 tablet_bootstrap.cc:492] T 3f803a6e48f74d26b826aa57f9233ee9 P 0b1901a1549f476696340f2fc61af194: No bootstrap required, opened a new log
I20260709 12:25:46.951864 24996 raft_consensus.cc:2804] T 3f803a6e48f74d26b826aa57f9233ee9 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:46.952327 24965 ts_tablet_manager.cc:1403] T 3f803a6e48f74d26b826aa57f9233ee9 P 0b1901a1549f476696340f2fc61af194: Time spent bootstrapping tablet: real 0.129s	user 0.042s	sys 0.029s
I20260709 12:25:46.953634 24996 raft_consensus.cc:697] T 3f803a6e48f74d26b826aa57f9233ee9 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 LEADER]: Becoming Leader. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Running, Role: LEADER
I20260709 12:25:46.954912 24996 consensus_queue.cc:237] T 3f803a6e48f74d26b826aa57f9233ee9 P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.955981 24846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f803a6e48f74d26b826aa57f9233ee9" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194"
W20260709 12:25:46.957589 24738 leader_election.cc:343] T 3f803a6e48f74d26b826aa57f9233ee9 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:46.957984 24965 raft_consensus.cc:359] T 3f803a6e48f74d26b826aa57f9233ee9 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.958773 24965 raft_consensus.cc:385] T 3f803a6e48f74d26b826aa57f9233ee9 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:46.959198 24962 ts_tablet_manager.cc:1434] T 1df8831adbf14c3594dea358a98bfff3 P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent starting tablet: real 0.121s	user 0.007s	sys 0.015s
I20260709 12:25:46.959388 24965 raft_consensus.cc:740] T 3f803a6e48f74d26b826aa57f9233ee9 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Initialized, Role: FOLLOWER
I20260709 12:25:46.958739 24847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f803a6e48f74d26b826aa57f9233ee9" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194" is_pre_election: true
I20260709 12:25:46.960855 24962 tablet_bootstrap.cc:492] T f1d7d9ea11c24a428a3ca7a20b11e81c P 6f825b5c64844f7fb020b60f13e6fdb4: Bootstrap starting.
I20260709 12:25:46.960580 24965 consensus_queue.cc:260] T 3f803a6e48f74d26b826aa57f9233ee9 P 0b1901a1549f476696340f2fc61af194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.963307 24965 ts_tablet_manager.cc:1434] T 3f803a6e48f74d26b826aa57f9233ee9 P 0b1901a1549f476696340f2fc61af194: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.004s
I20260709 12:25:46.963222 24847 raft_consensus.cc:2468] T 3f803a6e48f74d26b826aa57f9233ee9 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f825b5c64844f7fb020b60f13e6fdb4 in term 0.
I20260709 12:25:46.964578 24965 tablet_bootstrap.cc:492] T f1d7d9ea11c24a428a3ca7a20b11e81c P 0b1901a1549f476696340f2fc61af194: Bootstrap starting.
I20260709 12:25:46.974789 24962 tablet_bootstrap.cc:654] T f1d7d9ea11c24a428a3ca7a20b11e81c P 6f825b5c64844f7fb020b60f13e6fdb4: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:46.975579 24965 tablet_bootstrap.cc:654] T f1d7d9ea11c24a428a3ca7a20b11e81c P 0b1901a1549f476696340f2fc61af194: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:46.985891 24961 tablet_bootstrap.cc:492] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499: No bootstrap required, opened a new log
I20260709 12:25:46.986436 24961 ts_tablet_manager.cc:1403] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499: Time spent bootstrapping tablet: real 0.111s	user 0.035s	sys 0.034s
I20260709 12:25:46.986670 24962 tablet_bootstrap.cc:492] T f1d7d9ea11c24a428a3ca7a20b11e81c P 6f825b5c64844f7fb020b60f13e6fdb4: No bootstrap required, opened a new log
I20260709 12:25:46.987244 24962 ts_tablet_manager.cc:1403] T f1d7d9ea11c24a428a3ca7a20b11e81c P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent bootstrapping tablet: real 0.027s	user 0.016s	sys 0.002s
I20260709 12:25:46.987874 24965 tablet_bootstrap.cc:492] T f1d7d9ea11c24a428a3ca7a20b11e81c P 0b1901a1549f476696340f2fc61af194: No bootstrap required, opened a new log
I20260709 12:25:46.988494 24965 ts_tablet_manager.cc:1403] T f1d7d9ea11c24a428a3ca7a20b11e81c P 0b1901a1549f476696340f2fc61af194: Time spent bootstrapping tablet: real 0.024s	user 0.005s	sys 0.014s
I20260709 12:25:46.990499 24961 raft_consensus.cc:359] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.990933 24962 raft_consensus.cc:359] T f1d7d9ea11c24a428a3ca7a20b11e81c P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.991312 24961 raft_consensus.cc:385] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:46.991680 24962 raft_consensus.cc:385] T f1d7d9ea11c24a428a3ca7a20b11e81c P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:46.991727 24961 raft_consensus.cc:740] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Initialized, Role: FOLLOWER
I20260709 12:25:46.992100 24962 raft_consensus.cc:740] T f1d7d9ea11c24a428a3ca7a20b11e81c P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Initialized, Role: FOLLOWER
I20260709 12:25:46.992661 24965 raft_consensus.cc:359] T f1d7d9ea11c24a428a3ca7a20b11e81c P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.993041 24961 consensus_queue.cc:260] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.992964 24962 consensus_queue.cc:260] T f1d7d9ea11c24a428a3ca7a20b11e81c P 6f825b5c64844f7fb020b60f13e6fdb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.993589 24965 raft_consensus.cc:385] T f1d7d9ea11c24a428a3ca7a20b11e81c P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:46.994083 24965 raft_consensus.cc:740] T f1d7d9ea11c24a428a3ca7a20b11e81c P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Initialized, Role: FOLLOWER
I20260709 12:25:46.995723 24961 ts_tablet_manager.cc:1434] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.007s
I20260709 12:25:46.995075 24965 consensus_queue.cc:260] T f1d7d9ea11c24a428a3ca7a20b11e81c P 0b1901a1549f476696340f2fc61af194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:46.996291 24962 ts_tablet_manager.cc:1434] T f1d7d9ea11c24a428a3ca7a20b11e81c P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:25:46.996850 24961 tablet_bootstrap.cc:492] T 357a8c2b04ca46349006b7487ed1409e P 2807a439752a4b0ab39c82cdef37f499: Bootstrap starting.
I20260709 12:25:46.997447 24962 tablet_bootstrap.cc:492] T 89fd2ebc3fe5461c93eac233d095a6df P 6f825b5c64844f7fb020b60f13e6fdb4: Bootstrap starting.
I20260709 12:25:46.998278 24965 ts_tablet_manager.cc:1434] T f1d7d9ea11c24a428a3ca7a20b11e81c P 0b1901a1549f476696340f2fc61af194: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:25:46.999047 24967 raft_consensus.cc:493] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.999442 24965 tablet_bootstrap.cc:492] T 8f0eec1401e842eeb0541c2dd6621e58 P 0b1901a1549f476696340f2fc61af194: Bootstrap starting.
I20260709 12:25:46.999639 24967 raft_consensus.cc:515] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.002084 24967 leader_election.cc:290] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:25:47.001205 24669 catalog_manager.cc:5697] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "2807a439752a4b0ab39c82cdef37f499" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:47.003834 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df8831adbf14c3594dea358a98bfff3" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" is_pre_election: true
I20260709 12:25:47.004539 24771 raft_consensus.cc:2468] T 1df8831adbf14c3594dea358a98bfff3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 0.
I20260709 12:25:47.005807 24887 leader_election.cc:304] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2807a439752a4b0ab39c82cdef37f499, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:47.006678 24967 raft_consensus.cc:2804] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:25:47.005991 24666 catalog_manager.cc:5697] T 3f803a6e48f74d26b826aa57f9233ee9 P 6f825b5c64844f7fb020b60f13e6fdb4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:47.007364 24967 raft_consensus.cc:493] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.007807 24967 raft_consensus.cc:3060] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:47.010875 24965 tablet_bootstrap.cc:654] T 8f0eec1401e842eeb0541c2dd6621e58 P 0b1901a1549f476696340f2fc61af194: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:47.012253 24962 tablet_bootstrap.cc:654] T 89fd2ebc3fe5461c93eac233d095a6df P 6f825b5c64844f7fb020b60f13e6fdb4: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:47.017345 24847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df8831adbf14c3594dea358a98bfff3" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194" is_pre_election: true
I20260709 12:25:47.018428 24847 raft_consensus.cc:2468] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 0.
I20260709 12:25:47.020057 24967 raft_consensus.cc:515] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.017408 24961 tablet_bootstrap.cc:654] T 357a8c2b04ca46349006b7487ed1409e P 2807a439752a4b0ab39c82cdef37f499: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:47.024091 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df8831adbf14c3594dea358a98bfff3" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
I20260709 12:25:47.024796 24771 raft_consensus.cc:3060] T 1df8831adbf14c3594dea358a98bfff3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:47.025813 24965 tablet_bootstrap.cc:492] T 8f0eec1401e842eeb0541c2dd6621e58 P 0b1901a1549f476696340f2fc61af194: No bootstrap required, opened a new log
I20260709 12:25:47.026336 24965 ts_tablet_manager.cc:1403] T 8f0eec1401e842eeb0541c2dd6621e58 P 0b1901a1549f476696340f2fc61af194: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.007s
I20260709 12:25:47.028775 24967 leader_election.cc:290] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 election: Requested vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:25:47.029587 24965 raft_consensus.cc:359] T 8f0eec1401e842eeb0541c2dd6621e58 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.030467 24965 raft_consensus.cc:385] T 8f0eec1401e842eeb0541c2dd6621e58 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:47.030814 24965 raft_consensus.cc:740] T 8f0eec1401e842eeb0541c2dd6621e58 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Initialized, Role: FOLLOWER
I20260709 12:25:47.031742 24965 consensus_queue.cc:260] T 8f0eec1401e842eeb0541c2dd6621e58 P 0b1901a1549f476696340f2fc61af194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.034579 24965 ts_tablet_manager.cc:1434] T 8f0eec1401e842eeb0541c2dd6621e58 P 0b1901a1549f476696340f2fc61af194: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:25:47.035607 24965 tablet_bootstrap.cc:492] T 89fd2ebc3fe5461c93eac233d095a6df P 0b1901a1549f476696340f2fc61af194: Bootstrap starting.
I20260709 12:25:47.037422 24962 tablet_bootstrap.cc:492] T 89fd2ebc3fe5461c93eac233d095a6df P 6f825b5c64844f7fb020b60f13e6fdb4: No bootstrap required, opened a new log
I20260709 12:25:47.037418 24847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df8831adbf14c3594dea358a98bfff3" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194"
I20260709 12:25:47.040817 24847 raft_consensus.cc:3060] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:47.042512 24965 tablet_bootstrap.cc:654] T 89fd2ebc3fe5461c93eac233d095a6df P 0b1901a1549f476696340f2fc61af194: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:47.039203 24771 raft_consensus.cc:2468] T 1df8831adbf14c3594dea358a98bfff3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 1.
I20260709 12:25:47.044164 24887 leader_election.cc:304] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2807a439752a4b0ab39c82cdef37f499, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:47.045358 24967 raft_consensus.cc:2804] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:47.040247 24962 ts_tablet_manager.cc:1403] T 89fd2ebc3fe5461c93eac233d095a6df P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent bootstrapping tablet: real 0.043s	user 0.010s	sys 0.012s
I20260709 12:25:47.045899 24967 raft_consensus.cc:697] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Becoming Leader. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Running, Role: LEADER
I20260709 12:25:47.053057 24967 consensus_queue.cc:237] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.054100 24965 tablet_bootstrap.cc:492] T 89fd2ebc3fe5461c93eac233d095a6df P 0b1901a1549f476696340f2fc61af194: No bootstrap required, opened a new log
I20260709 12:25:47.054636 24965 ts_tablet_manager.cc:1403] T 89fd2ebc3fe5461c93eac233d095a6df P 0b1901a1549f476696340f2fc61af194: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.003s
I20260709 12:25:47.056934 24962 raft_consensus.cc:359] T 89fd2ebc3fe5461c93eac233d095a6df P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.058203 24962 raft_consensus.cc:385] T 89fd2ebc3fe5461c93eac233d095a6df P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:47.059281 24962 raft_consensus.cc:740] T 89fd2ebc3fe5461c93eac233d095a6df P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Initialized, Role: FOLLOWER
I20260709 12:25:47.060207 24962 consensus_queue.cc:260] T 89fd2ebc3fe5461c93eac233d095a6df P 6f825b5c64844f7fb020b60f13e6fdb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.060720 24961 tablet_bootstrap.cc:492] T 357a8c2b04ca46349006b7487ed1409e P 2807a439752a4b0ab39c82cdef37f499: No bootstrap required, opened a new log
I20260709 12:25:47.067418 24961 ts_tablet_manager.cc:1403] T 357a8c2b04ca46349006b7487ed1409e P 2807a439752a4b0ab39c82cdef37f499: Time spent bootstrapping tablet: real 0.071s	user 0.019s	sys 0.015s
I20260709 12:25:47.070119 24962 ts_tablet_manager.cc:1434] T 89fd2ebc3fe5461c93eac233d095a6df P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:25:47.070314 24961 raft_consensus.cc:359] T 357a8c2b04ca46349006b7487ed1409e P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.071098 24961 raft_consensus.cc:385] T 357a8c2b04ca46349006b7487ed1409e P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:47.071233 24962 tablet_bootstrap.cc:492] T 357a8c2b04ca46349006b7487ed1409e P 6f825b5c64844f7fb020b60f13e6fdb4: Bootstrap starting.
I20260709 12:25:47.071395 24961 raft_consensus.cc:740] T 357a8c2b04ca46349006b7487ed1409e P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Initialized, Role: FOLLOWER
I20260709 12:25:47.060570 24965 raft_consensus.cc:359] T 89fd2ebc3fe5461c93eac233d095a6df P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.066496 24847 raft_consensus.cc:2468] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 1.
I20260709 12:25:47.072104 24965 raft_consensus.cc:385] T 89fd2ebc3fe5461c93eac233d095a6df P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:47.072494 24965 raft_consensus.cc:740] T 89fd2ebc3fe5461c93eac233d095a6df P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Initialized, Role: FOLLOWER
I20260709 12:25:47.073256 24965 consensus_queue.cc:260] T 89fd2ebc3fe5461c93eac233d095a6df P 0b1901a1549f476696340f2fc61af194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.074484 24668 catalog_manager.cc:5697] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "2807a439752a4b0ab39c82cdef37f499" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:47.076083 24961 consensus_queue.cc:260] T 357a8c2b04ca46349006b7487ed1409e P 2807a439752a4b0ab39c82cdef37f499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.078302 24961 ts_tablet_manager.cc:1434] T 357a8c2b04ca46349006b7487ed1409e P 2807a439752a4b0ab39c82cdef37f499: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 12:25:47.079203 24961 tablet_bootstrap.cc:492] T 8f0eec1401e842eeb0541c2dd6621e58 P 2807a439752a4b0ab39c82cdef37f499: Bootstrap starting.
I20260709 12:25:47.079660 24962 tablet_bootstrap.cc:654] T 357a8c2b04ca46349006b7487ed1409e P 6f825b5c64844f7fb020b60f13e6fdb4: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:47.075426 24965 ts_tablet_manager.cc:1434] T 89fd2ebc3fe5461c93eac233d095a6df P 0b1901a1549f476696340f2fc61af194: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.002s
I20260709 12:25:47.085059 24961 tablet_bootstrap.cc:654] T 8f0eec1401e842eeb0541c2dd6621e58 P 2807a439752a4b0ab39c82cdef37f499: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:47.085588 24965 tablet_bootstrap.cc:492] T 357a8c2b04ca46349006b7487ed1409e P 0b1901a1549f476696340f2fc61af194: Bootstrap starting.
I20260709 12:25:47.098133 24965 tablet_bootstrap.cc:654] T 357a8c2b04ca46349006b7487ed1409e P 0b1901a1549f476696340f2fc61af194: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:47.114022 24965 tablet_bootstrap.cc:492] T 357a8c2b04ca46349006b7487ed1409e P 0b1901a1549f476696340f2fc61af194: No bootstrap required, opened a new log
I20260709 12:25:47.114641 24965 ts_tablet_manager.cc:1403] T 357a8c2b04ca46349006b7487ed1409e P 0b1901a1549f476696340f2fc61af194: Time spent bootstrapping tablet: real 0.030s	user 0.014s	sys 0.009s
I20260709 12:25:47.115309 24961 tablet_bootstrap.cc:492] T 8f0eec1401e842eeb0541c2dd6621e58 P 2807a439752a4b0ab39c82cdef37f499: No bootstrap required, opened a new log
I20260709 12:25:47.115374 24962 tablet_bootstrap.cc:492] T 357a8c2b04ca46349006b7487ed1409e P 6f825b5c64844f7fb020b60f13e6fdb4: No bootstrap required, opened a new log
I20260709 12:25:47.115859 24961 ts_tablet_manager.cc:1403] T 8f0eec1401e842eeb0541c2dd6621e58 P 2807a439752a4b0ab39c82cdef37f499: Time spent bootstrapping tablet: real 0.037s	user 0.014s	sys 0.016s
I20260709 12:25:47.115911 24962 ts_tablet_manager.cc:1403] T 357a8c2b04ca46349006b7487ed1409e P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent bootstrapping tablet: real 0.045s	user 0.025s	sys 0.016s
I20260709 12:25:47.117610 24965 raft_consensus.cc:359] T 357a8c2b04ca46349006b7487ed1409e P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.118430 24965 raft_consensus.cc:385] T 357a8c2b04ca46349006b7487ed1409e P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:47.118714 24965 raft_consensus.cc:740] T 357a8c2b04ca46349006b7487ed1409e P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Initialized, Role: FOLLOWER
I20260709 12:25:47.118804 24962 raft_consensus.cc:359] T 357a8c2b04ca46349006b7487ed1409e P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.119014 24961 raft_consensus.cc:359] T 8f0eec1401e842eeb0541c2dd6621e58 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.119565 24962 raft_consensus.cc:385] T 357a8c2b04ca46349006b7487ed1409e P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:47.119709 24961 raft_consensus.cc:385] T 8f0eec1401e842eeb0541c2dd6621e58 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:47.119483 24965 consensus_queue.cc:260] T 357a8c2b04ca46349006b7487ed1409e P 0b1901a1549f476696340f2fc61af194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.120203 24962 raft_consensus.cc:740] T 357a8c2b04ca46349006b7487ed1409e P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Initialized, Role: FOLLOWER
I20260709 12:25:47.120287 24961 raft_consensus.cc:740] T 8f0eec1401e842eeb0541c2dd6621e58 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Initialized, Role: FOLLOWER
I20260709 12:25:47.121093 24961 consensus_queue.cc:260] T 8f0eec1401e842eeb0541c2dd6621e58 P 2807a439752a4b0ab39c82cdef37f499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.121093 24962 consensus_queue.cc:260] T 357a8c2b04ca46349006b7487ed1409e P 6f825b5c64844f7fb020b60f13e6fdb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.123205 24962 ts_tablet_manager.cc:1434] T 357a8c2b04ca46349006b7487ed1409e P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:25:47.123205 24961 ts_tablet_manager.cc:1434] T 8f0eec1401e842eeb0541c2dd6621e58 P 2807a439752a4b0ab39c82cdef37f499: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:25:47.124235 24962 tablet_bootstrap.cc:492] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4: Bootstrap starting.
I20260709 12:25:47.125146 24965 ts_tablet_manager.cc:1434] T 357a8c2b04ca46349006b7487ed1409e P 0b1901a1549f476696340f2fc61af194: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:25:47.130956 24962 tablet_bootstrap.cc:654] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:47.134009 24967 raft_consensus.cc:493] T 357a8c2b04ca46349006b7487ed1409e P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.134616 24967 raft_consensus.cc:515] T 357a8c2b04ca46349006b7487ed1409e P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.137087 24967 leader_election.cc:290] T 357a8c2b04ca46349006b7487ed1409e P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:25:47.138440 24847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "357a8c2b04ca46349006b7487ed1409e" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194" is_pre_election: true
I20260709 12:25:47.139055 24847 raft_consensus.cc:2468] T 357a8c2b04ca46349006b7487ed1409e P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 0.
I20260709 12:25:47.139961 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "357a8c2b04ca46349006b7487ed1409e" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" is_pre_election: true
I20260709 12:25:47.140143 24887 leader_election.cc:304] T 357a8c2b04ca46349006b7487ed1409e P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 2807a439752a4b0ab39c82cdef37f499; no voters: 
I20260709 12:25:47.140767 24771 raft_consensus.cc:2468] T 357a8c2b04ca46349006b7487ed1409e P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 0.
I20260709 12:25:47.146687 24967 raft_consensus.cc:2804] T 357a8c2b04ca46349006b7487ed1409e P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:25:47.147142 24967 raft_consensus.cc:493] T 357a8c2b04ca46349006b7487ed1409e P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.147478 24967 raft_consensus.cc:3060] T 357a8c2b04ca46349006b7487ed1409e P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:47.147909 24962 tablet_bootstrap.cc:492] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4: No bootstrap required, opened a new log
I20260709 12:25:47.148380 24962 ts_tablet_manager.cc:1403] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.011s
I20260709 12:25:47.151248 24962 raft_consensus.cc:359] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.151902 24962 raft_consensus.cc:385] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:47.152158 24962 raft_consensus.cc:740] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Initialized, Role: FOLLOWER
I20260709 12:25:47.152804 24962 consensus_queue.cc:260] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.153656 24967 raft_consensus.cc:515] T 357a8c2b04ca46349006b7487ed1409e P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.154559 24962 ts_tablet_manager.cc:1434] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:25:47.155692 24967 leader_election.cc:290] T 357a8c2b04ca46349006b7487ed1409e P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 election: Requested vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:25:47.156807 24847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "357a8c2b04ca46349006b7487ed1409e" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194"
I20260709 12:25:47.156716 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "357a8c2b04ca46349006b7487ed1409e" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
I20260709 12:25:47.157464 24847 raft_consensus.cc:3060] T 357a8c2b04ca46349006b7487ed1409e P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:47.157533 24771 raft_consensus.cc:3060] T 357a8c2b04ca46349006b7487ed1409e P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:47.160517 24970 raft_consensus.cc:493] T 89fd2ebc3fe5461c93eac233d095a6df P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.161038 24970 raft_consensus.cc:515] T 89fd2ebc3fe5461c93eac233d095a6df P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.163393 24970 leader_election.cc:290] T 89fd2ebc3fe5461c93eac233d095a6df P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:47.164115 24772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89fd2ebc3fe5461c93eac233d095a6df" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" is_pre_election: true
I20260709 12:25:47.164849 24772 raft_consensus.cc:2468] T 89fd2ebc3fe5461c93eac233d095a6df P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b1901a1549f476696340f2fc61af194 in term 0.
I20260709 12:25:47.165063 24920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89fd2ebc3fe5461c93eac233d095a6df" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499" is_pre_election: true
I20260709 12:25:47.165474 24847 raft_consensus.cc:2468] T 357a8c2b04ca46349006b7487ed1409e P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 1.
I20260709 12:25:47.165771 24920 raft_consensus.cc:2468] T 89fd2ebc3fe5461c93eac233d095a6df P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b1901a1549f476696340f2fc61af194 in term 0.
I20260709 12:25:47.166142 24813 leader_election.cc:304] T 89fd2ebc3fe5461c93eac233d095a6df P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:47.166705 24887 leader_election.cc:304] T 357a8c2b04ca46349006b7487ed1409e P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 2807a439752a4b0ab39c82cdef37f499; no voters: 
I20260709 12:25:47.167153 24970 raft_consensus.cc:2804] T 89fd2ebc3fe5461c93eac233d095a6df P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:25:47.167609 24970 raft_consensus.cc:493] T 89fd2ebc3fe5461c93eac233d095a6df P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.167675 24967 raft_consensus.cc:2804] T 357a8c2b04ca46349006b7487ed1409e P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:47.168071 24970 raft_consensus.cc:3060] T 89fd2ebc3fe5461c93eac233d095a6df P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:47.168282 24967 raft_consensus.cc:697] T 357a8c2b04ca46349006b7487ed1409e P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Becoming Leader. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Running, Role: LEADER
I20260709 12:25:47.169936 24967 consensus_queue.cc:237] T 357a8c2b04ca46349006b7487ed1409e P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.174860 24771 raft_consensus.cc:2468] T 357a8c2b04ca46349006b7487ed1409e P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 1.
I20260709 12:25:47.177126 24970 raft_consensus.cc:515] T 89fd2ebc3fe5461c93eac233d095a6df P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.179375 24970 leader_election.cc:290] T 89fd2ebc3fe5461c93eac233d095a6df P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 election: Requested vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:47.179942 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89fd2ebc3fe5461c93eac233d095a6df" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
I20260709 12:25:47.180621 24771 raft_consensus.cc:3060] T 89fd2ebc3fe5461c93eac233d095a6df P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:47.180575 24920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89fd2ebc3fe5461c93eac233d095a6df" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
I20260709 12:25:47.181381 24920 raft_consensus.cc:3060] T 89fd2ebc3fe5461c93eac233d095a6df P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:47.181432 24669 catalog_manager.cc:5697] T 357a8c2b04ca46349006b7487ed1409e P 2807a439752a4b0ab39c82cdef37f499 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "2807a439752a4b0ab39c82cdef37f499" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:47.188658 24920 raft_consensus.cc:2468] T 89fd2ebc3fe5461c93eac233d095a6df P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b1901a1549f476696340f2fc61af194 in term 1.
I20260709 12:25:47.189934 24810 leader_election.cc:304] T 89fd2ebc3fe5461c93eac233d095a6df P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 2807a439752a4b0ab39c82cdef37f499; no voters: 
I20260709 12:25:47.190727 24970 raft_consensus.cc:2804] T 89fd2ebc3fe5461c93eac233d095a6df P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:47.191179 24970 raft_consensus.cc:697] T 89fd2ebc3fe5461c93eac233d095a6df P 0b1901a1549f476696340f2fc61af194 [term 1 LEADER]: Becoming Leader. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Running, Role: LEADER
I20260709 12:25:47.192015 24771 raft_consensus.cc:2468] T 89fd2ebc3fe5461c93eac233d095a6df P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b1901a1549f476696340f2fc61af194 in term 1.
I20260709 12:25:47.192461 24970 consensus_queue.cc:237] T 89fd2ebc3fe5461c93eac233d095a6df P 0b1901a1549f476696340f2fc61af194 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.200618 24669 catalog_manager.cc:5697] T 89fd2ebc3fe5461c93eac233d095a6df P 0b1901a1549f476696340f2fc61af194 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b1901a1549f476696340f2fc61af194 (127.5.58.2). New cstate: current_term: 1 leader_uuid: "0b1901a1549f476696340f2fc61af194" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:47.220997 24920 raft_consensus.cc:3060] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:47.227306 24970 consensus_queue.cc:1048] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:47.229260 24967 raft_consensus.cc:493] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.229885 24967 raft_consensus.cc:515] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.231967 24967 leader_election.cc:290] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:25:47.232970 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36fd425a2ae24bba8cc5934237f465b6" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" is_pre_election: true
I20260709 12:25:47.233168 24847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36fd425a2ae24bba8cc5934237f465b6" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194" is_pre_election: true
I20260709 12:25:47.233870 24771 raft_consensus.cc:2468] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 1.
I20260709 12:25:47.235204 24887 leader_election.cc:304] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2807a439752a4b0ab39c82cdef37f499, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 0b1901a1549f476696340f2fc61af194
I20260709 12:25:47.235953 24967 raft_consensus.cc:2804] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:47.236369 24967 raft_consensus.cc:493] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0b1901a1549f476696340f2fc61af194)
I20260709 12:25:47.236763 24967 raft_consensus.cc:3060] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:47.244082 24987 consensus_queue.cc:1048] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:25:47.246404 24967 raft_consensus.cc:515] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.249985 24772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36fd425a2ae24bba8cc5934237f465b6" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
I20260709 12:25:47.250228 24920 raft_consensus.cc:1240] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Rejecting Update request from peer 0b1901a1549f476696340f2fc61af194 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:47.254649 24967 leader_election.cc:290] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 2 election: Requested vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:25:47.254555 24987 consensus_queue.cc:1059] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:47.255216 24847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36fd425a2ae24bba8cc5934237f465b6" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194"
I20260709 12:25:47.255558 24987 raft_consensus.cc:3055] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:47.255959 24987 raft_consensus.cc:740] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Running, Role: LEADER
I20260709 12:25:47.256892 24987 consensus_queue.cc:260] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.257493 24967 raft_consensus.cc:493] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.257680 24887 leader_election.cc:304] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2807a439752a4b0ab39c82cdef37f499; no voters: 0b1901a1549f476696340f2fc61af194, 6f825b5c64844f7fb020b60f13e6fdb4
I20260709 12:25:47.257908 24987 raft_consensus.cc:3060] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:47.258718 24984 raft_consensus.cc:2749] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:47.258069 24967 raft_consensus.cc:515] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.262638 24847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1d7d9ea11c24a428a3ca7a20b11e81c" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194" is_pre_election: true
I20260709 12:25:47.262930 24985 raft_consensus.cc:493] T f1d7d9ea11c24a428a3ca7a20b11e81c P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.261471 24967 leader_election.cc:290] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:25:47.262676 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1d7d9ea11c24a428a3ca7a20b11e81c" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" is_pre_election: true
I20260709 12:25:47.263726 24985 raft_consensus.cc:515] T f1d7d9ea11c24a428a3ca7a20b11e81c P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.264119 24771 raft_consensus.cc:2468] T f1d7d9ea11c24a428a3ca7a20b11e81c P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 0.
I20260709 12:25:47.264822 24847 raft_consensus.cc:2468] T f1d7d9ea11c24a428a3ca7a20b11e81c P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 0.
I20260709 12:25:47.265473 24887 leader_election.cc:304] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2807a439752a4b0ab39c82cdef37f499, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:47.267860 24967 raft_consensus.cc:2804] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:25:47.268074 24920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1d7d9ea11c24a428a3ca7a20b11e81c" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499" is_pre_election: true
I20260709 12:25:47.268364 24967 raft_consensus.cc:493] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.268730 24985 leader_election.cc:290] T f1d7d9ea11c24a428a3ca7a20b11e81c P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:47.268841 24967 raft_consensus.cc:3060] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:47.273962 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1d7d9ea11c24a428a3ca7a20b11e81c" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" is_pre_election: true
I20260709 12:25:47.274632 24771 raft_consensus.cc:2468] T f1d7d9ea11c24a428a3ca7a20b11e81c P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b1901a1549f476696340f2fc61af194 in term 0.
I20260709 12:25:47.275708 24813 leader_election.cc:304] T f1d7d9ea11c24a428a3ca7a20b11e81c P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:47.276324 24967 raft_consensus.cc:515] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.276867 24987 raft_consensus.cc:2804] T f1d7d9ea11c24a428a3ca7a20b11e81c P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:25:47.277383 24987 raft_consensus.cc:493] T f1d7d9ea11c24a428a3ca7a20b11e81c P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.277633 24920 raft_consensus.cc:2393] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0b1901a1549f476696340f2fc61af194 in current term 1: Already voted for candidate 2807a439752a4b0ab39c82cdef37f499 in this term.
I20260709 12:25:47.277772 24987 raft_consensus.cc:3060] T f1d7d9ea11c24a428a3ca7a20b11e81c P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:47.278508 24967 leader_election.cc:290] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 election: Requested vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:25:47.279821 24847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1d7d9ea11c24a428a3ca7a20b11e81c" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194"
I20260709 12:25:47.279891 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1d7d9ea11c24a428a3ca7a20b11e81c" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
I20260709 12:25:47.280614 24771 raft_consensus.cc:3060] T f1d7d9ea11c24a428a3ca7a20b11e81c P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:47.285461 24987 raft_consensus.cc:515] T f1d7d9ea11c24a428a3ca7a20b11e81c P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.285926 24771 raft_consensus.cc:2468] T f1d7d9ea11c24a428a3ca7a20b11e81c P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 1.
I20260709 12:25:47.286669 24847 raft_consensus.cc:2393] T f1d7d9ea11c24a428a3ca7a20b11e81c P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2807a439752a4b0ab39c82cdef37f499 in current term 1: Already voted for candidate 0b1901a1549f476696340f2fc61af194 in this term.
I20260709 12:25:47.287480 24887 leader_election.cc:304] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2807a439752a4b0ab39c82cdef37f499, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:47.288475 24967 raft_consensus.cc:2804] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:47.288414 24987 leader_election.cc:290] T f1d7d9ea11c24a428a3ca7a20b11e81c P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 election: Requested vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:47.288779 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1d7d9ea11c24a428a3ca7a20b11e81c" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
I20260709 12:25:47.288914 24920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1d7d9ea11c24a428a3ca7a20b11e81c" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
I20260709 12:25:47.289285 24967 raft_consensus.cc:697] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Becoming Leader. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Running, Role: LEADER
I20260709 12:25:47.291136 24771 raft_consensus.cc:2393] T f1d7d9ea11c24a428a3ca7a20b11e81c P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0b1901a1549f476696340f2fc61af194 in current term 1: Already voted for candidate 2807a439752a4b0ab39c82cdef37f499 in this term.
I20260709 12:25:47.291320 24967 consensus_queue.cc:237] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.293401 24996 raft_consensus.cc:493] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.293927 24996 raft_consensus.cc:515] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.296265 24996 leader_election.cc:290] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:47.297111 24810 leader_election.cc:304] T f1d7d9ea11c24a428a3ca7a20b11e81c P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0b1901a1549f476696340f2fc61af194; no voters: 2807a439752a4b0ab39c82cdef37f499, 6f825b5c64844f7fb020b60f13e6fdb4
I20260709 12:25:47.298045 24987 raft_consensus.cc:2749] T f1d7d9ea11c24a428a3ca7a20b11e81c P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:25:47.298110 24847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0eec1401e842eeb0541c2dd6621e58" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194" is_pre_election: true
I20260709 12:25:47.298563 24921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0eec1401e842eeb0541c2dd6621e58" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499" is_pre_election: true
I20260709 12:25:47.298969 24847 raft_consensus.cc:2468] T 8f0eec1401e842eeb0541c2dd6621e58 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f825b5c64844f7fb020b60f13e6fdb4 in term 0.
I20260709 12:25:47.299245 24921 raft_consensus.cc:2468] T 8f0eec1401e842eeb0541c2dd6621e58 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f825b5c64844f7fb020b60f13e6fdb4 in term 0.
I20260709 12:25:47.300401 24738 leader_election.cc:304] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:47.301791 24996 raft_consensus.cc:2804] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:25:47.302251 24996 raft_consensus.cc:493] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.302592 24996 raft_consensus.cc:3060] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:47.301911 24670 catalog_manager.cc:5697] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "2807a439752a4b0ab39c82cdef37f499" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:47.309751 24996 raft_consensus.cc:515] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.312548 24847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0eec1401e842eeb0541c2dd6621e58" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194"
I20260709 12:25:47.313133 24847 raft_consensus.cc:3060] T 8f0eec1401e842eeb0541c2dd6621e58 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:47.313889 24921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0eec1401e842eeb0541c2dd6621e58" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
I20260709 12:25:47.313946 24996 leader_election.cc:290] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 1 election: Requested vote from peers 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:47.314538 24921 raft_consensus.cc:3060] T 8f0eec1401e842eeb0541c2dd6621e58 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:47.316874 24967 consensus_queue.cc:1048] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:47.320654 24847 raft_consensus.cc:2468] T 8f0eec1401e842eeb0541c2dd6621e58 P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f825b5c64844f7fb020b60f13e6fdb4 in term 1.
I20260709 12:25:47.321708 24738 leader_election.cc:304] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:47.322593 24996 raft_consensus.cc:2804] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:47.323316 24996 raft_consensus.cc:697] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 LEADER]: Becoming Leader. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Running, Role: LEADER
I20260709 12:25:47.323496 24921 raft_consensus.cc:2468] T 8f0eec1401e842eeb0541c2dd6621e58 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f825b5c64844f7fb020b60f13e6fdb4 in term 1.
I20260709 12:25:47.324218 24996 consensus_queue.cc:237] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.333251 24967 consensus_queue.cc:1048] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:25:47.339977 24669 catalog_manager.cc:5697] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:47.367421 24996 consensus_queue.cc:1048] T 3f803a6e48f74d26b826aa57f9233ee9 P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:47.369917 24984 consensus_queue.cc:1048] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:47.387287 24847 raft_consensus.cc:3060] T 3f803a6e48f74d26b826aa57f9233ee9 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:47.387163 24988 consensus_queue.cc:1048] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:25:47.401331 24996 consensus_queue.cc:1048] T 3f803a6e48f74d26b826aa57f9233ee9 P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:47.439742 24988 consensus_queue.cc:1048] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:47.452032 24967 consensus_queue.cc:1048] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:25:47.486481 24921 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3f69b65186ed4f6c8fe7c50f6b0133d3"
dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
mode: GRACEFUL
new_leader_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
 from {username='slave'} at 127.0.0.1:57276
I20260709 12:25:47.487180 24921 raft_consensus.cc:606] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Received request to transfer leadership to TS 6f825b5c64844f7fb020b60f13e6fdb4
I20260709 12:25:47.493484 24921 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97a81d7f4ccb459ab2e722f09c10d9a1"
dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
mode: GRACEFUL
new_leader_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
 from {username='slave'} at 127.0.0.1:57276
I20260709 12:25:47.494061 24921 raft_consensus.cc:606] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Received request to transfer leadership to TS 6f825b5c64844f7fb020b60f13e6fdb4
I20260709 12:25:47.499521 24921 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f1d7d9ea11c24a428a3ca7a20b11e81c"
dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
mode: GRACEFUL
new_leader_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
 from {username='slave'} at 127.0.0.1:57276
I20260709 12:25:47.500100 24921 raft_consensus.cc:606] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Received request to transfer leadership to TS 6f825b5c64844f7fb020b60f13e6fdb4
I20260709 12:25:47.515164 24921 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1df8831adbf14c3594dea358a98bfff3"
dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
mode: GRACEFUL
new_leader_uuid: "0b1901a1549f476696340f2fc61af194"
 from {username='slave'} at 127.0.0.1:57276
I20260709 12:25:47.515697 24921 raft_consensus.cc:606] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Received request to transfer leadership to TS 0b1901a1549f476696340f2fc61af194
I20260709 12:25:47.530957 24771 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f0eec1401e842eeb0541c2dd6621e58"
dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
mode: GRACEFUL
new_leader_uuid: "0b1901a1549f476696340f2fc61af194"
 from {username='slave'} at 127.0.0.1:41292
I20260709 12:25:47.531639 24771 raft_consensus.cc:606] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 LEADER]: Received request to transfer leadership to TS 0b1901a1549f476696340f2fc61af194
I20260709 12:25:47.596109 24967 consensus_queue.cc:1048] T 357a8c2b04ca46349006b7487ed1409e P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:47.598053 24987 consensus_queue.cc:1048] T 89fd2ebc3fe5461c93eac233d095a6df P 0b1901a1549f476696340f2fc61af194 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:47.612010 24967 consensus_queue.cc:1048] T 357a8c2b04ca46349006b7487ed1409e P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:25:47.619529 24970 consensus_queue.cc:1048] T 89fd2ebc3fe5461c93eac233d095a6df P 0b1901a1549f476696340f2fc61af194 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:47.690492 25038 consensus_queue.cc:1048] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:25:47.706458 24988 consensus_queue.cc:1048] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:25:47.712631 24996 consensus_queue.cc:1048] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:47.720268 24988 raft_consensus.cc:993] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499: : Instructing follower 6f825b5c64844f7fb020b60f13e6fdb4 to start an election
I20260709 12:25:47.720768 24984 raft_consensus.cc:1081] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Signalling peer 6f825b5c64844f7fb020b60f13e6fdb4 to start an election
I20260709 12:25:47.722554 24771 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f1d7d9ea11c24a428a3ca7a20b11e81c"
dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
 from {username='slave'} at 127.0.0.1:41260
I20260709 12:25:47.723446 24771 raft_consensus.cc:493] T f1d7d9ea11c24a428a3ca7a20b11e81c P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:47.723815 24771 raft_consensus.cc:3060] T f1d7d9ea11c24a428a3ca7a20b11e81c P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:47.728448 24966 consensus_queue.cc:1048] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:25:47.734472 24771 raft_consensus.cc:515] T f1d7d9ea11c24a428a3ca7a20b11e81c P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.738981 24966 raft_consensus.cc:993] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4: : Instructing follower 0b1901a1549f476696340f2fc61af194 to start an election
I20260709 12:25:47.739614 24996 raft_consensus.cc:1081] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 LEADER]: Signalling peer 0b1901a1549f476696340f2fc61af194 to start an election
I20260709 12:25:47.742473 24846 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8f0eec1401e842eeb0541c2dd6621e58"
dest_uuid: "0b1901a1549f476696340f2fc61af194"
 from {username='slave'} at 127.0.0.1:47936
I20260709 12:25:47.743455 24846 raft_consensus.cc:493] T 8f0eec1401e842eeb0541c2dd6621e58 P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:47.744102 24846 raft_consensus.cc:3060] T 8f0eec1401e842eeb0541c2dd6621e58 P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:47.744305 24771 leader_election.cc:290] T f1d7d9ea11c24a428a3ca7a20b11e81c P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 2 election: Requested vote from peers 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:47.745143 24921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1d7d9ea11c24a428a3ca7a20b11e81c" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
I20260709 12:25:47.745843 24921 raft_consensus.cc:3055] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:47.741961 24847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1d7d9ea11c24a428a3ca7a20b11e81c" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0b1901a1549f476696340f2fc61af194"
I20260709 12:25:47.746179 24921 raft_consensus.cc:740] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Running, Role: LEADER
I20260709 12:25:47.746578 24847 raft_consensus.cc:3060] T f1d7d9ea11c24a428a3ca7a20b11e81c P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:47.747161 24921 consensus_queue.cc:260] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.749327 24921 raft_consensus.cc:3060] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:47.754290 24846 raft_consensus.cc:515] T 8f0eec1401e842eeb0541c2dd6621e58 P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.757791 24847 raft_consensus.cc:2468] T f1d7d9ea11c24a428a3ca7a20b11e81c P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f825b5c64844f7fb020b60f13e6fdb4 in term 2.
I20260709 12:25:47.757791 24921 raft_consensus.cc:2468] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f825b5c64844f7fb020b60f13e6fdb4 in term 2.
I20260709 12:25:47.759620 24735 leader_election.cc:304] T f1d7d9ea11c24a428a3ca7a20b11e81c P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2807a439752a4b0ab39c82cdef37f499, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:47.760680 24996 raft_consensus.cc:2804] T f1d7d9ea11c24a428a3ca7a20b11e81c P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:47.761212 24996 raft_consensus.cc:697] T f1d7d9ea11c24a428a3ca7a20b11e81c P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 LEADER]: Becoming Leader. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Running, Role: LEADER
I20260709 12:25:47.762149 24996 consensus_queue.cc:237] T f1d7d9ea11c24a428a3ca7a20b11e81c P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.764901 24846 leader_election.cc:290] T 8f0eec1401e842eeb0541c2dd6621e58 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 2 election: Requested vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:47.765937 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0eec1401e842eeb0541c2dd6621e58" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
I20260709 12:25:47.766201 24921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0eec1401e842eeb0541c2dd6621e58" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
I20260709 12:25:47.766647 24771 raft_consensus.cc:3055] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:47.767050 24771 raft_consensus.cc:740] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Running, Role: LEADER
I20260709 12:25:47.767117 24921 raft_consensus.cc:3060] T 8f0eec1401e842eeb0541c2dd6621e58 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:47.767926 24771 consensus_queue.cc:260] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.769311 24771 raft_consensus.cc:3060] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:47.774550 24771 raft_consensus.cc:2468] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b1901a1549f476696340f2fc61af194 in term 2.
I20260709 12:25:47.773989 24666 catalog_manager.cc:5697] T f1d7d9ea11c24a428a3ca7a20b11e81c P 6f825b5c64844f7fb020b60f13e6fdb4 reported cstate change: term changed from 1 to 2, leader changed from 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3) to 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1). New cstate: current_term: 2 leader_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:47.775967 24813 leader_election.cc:304] T 8f0eec1401e842eeb0541c2dd6621e58 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:47.776505 24921 raft_consensus.cc:2468] T 8f0eec1401e842eeb0541c2dd6621e58 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b1901a1549f476696340f2fc61af194 in term 2.
I20260709 12:25:47.776737 24970 raft_consensus.cc:2804] T 8f0eec1401e842eeb0541c2dd6621e58 P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:47.777366 24970 raft_consensus.cc:697] T 8f0eec1401e842eeb0541c2dd6621e58 P 0b1901a1549f476696340f2fc61af194 [term 2 LEADER]: Becoming Leader. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Running, Role: LEADER
I20260709 12:25:47.778173 24970 consensus_queue.cc:237] T 8f0eec1401e842eeb0541c2dd6621e58 P 0b1901a1549f476696340f2fc61af194 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.784951 24666 catalog_manager.cc:5697] T 8f0eec1401e842eeb0541c2dd6621e58 P 0b1901a1549f476696340f2fc61af194 reported cstate change: term changed from 1 to 2, leader changed from 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1) to 0b1901a1549f476696340f2fc61af194 (127.5.58.2). New cstate: current_term: 2 leader_uuid: "0b1901a1549f476696340f2fc61af194" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:47.812538 24988 raft_consensus.cc:993] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499: : Instructing follower 6f825b5c64844f7fb020b60f13e6fdb4 to start an election
I20260709 12:25:47.813082 24984 raft_consensus.cc:1081] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Signalling peer 6f825b5c64844f7fb020b60f13e6fdb4 to start an election
I20260709 12:25:47.815097 24771 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3f69b65186ed4f6c8fe7c50f6b0133d3"
dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
 from {username='slave'} at 127.0.0.1:41260
I20260709 12:25:47.815672 24771 raft_consensus.cc:493] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:47.816246 24771 raft_consensus.cc:3060] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:47.822939 24771 raft_consensus.cc:515] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } }
I20260709 12:25:47.824838 24771 leader_election.cc:290] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 2 election: Requested vote from peers 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:25:47.826097 24921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f69b65186ed4f6c8fe7c50f6b0133d3" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
I20260709 12:25:47.826119 24846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f69b65186ed4f6c8fe7c50f6b0133d3" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0b1901a1549f476696340f2fc61af194"
I20260709 12:25:47.826753 24921 raft_consensus.cc:3055] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:47.827139 24921 raft_consensus.cc:740] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Running, Role: LEADER
I20260709 12:25:47.826972 24846 raft_consensus.cc:3060] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:47.828011 24921 consensus_queue.cc:260] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } }
I20260709 12:25:47.829144 24921 raft_consensus.cc:3060] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:47.833256 24846 raft_consensus.cc:2468] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f825b5c64844f7fb020b60f13e6fdb4 in term 2.
I20260709 12:25:47.834522 24921 raft_consensus.cc:2468] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f825b5c64844f7fb020b60f13e6fdb4 in term 2.
I20260709 12:25:47.834578 24738 leader_election.cc:304] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:47.835626 24996 raft_consensus.cc:2804] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:47.836112 24996 raft_consensus.cc:697] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 LEADER]: Becoming Leader. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Running, Role: LEADER
I20260709 12:25:47.836916 24996 consensus_queue.cc:237] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } }
I20260709 12:25:47.842612 24984 raft_consensus.cc:993] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499: : Instructing follower 6f825b5c64844f7fb020b60f13e6fdb4 to start an election
I20260709 12:25:47.843160 24988 raft_consensus.cc:1081] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Signalling peer 6f825b5c64844f7fb020b60f13e6fdb4 to start an election
I20260709 12:25:47.844954 24771 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "97a81d7f4ccb459ab2e722f09c10d9a1"
dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
 from {username='slave'} at 127.0.0.1:41260
I20260709 12:25:47.844192 24666 catalog_manager.cc:5697] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 reported cstate change: term changed from 1 to 2, leader changed from 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3) to 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1). New cstate: current_term: 2 leader_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:47.845629 24771 raft_consensus.cc:493] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:47.846011 24771 raft_consensus.cc:3060] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:47.854710 24771 raft_consensus.cc:515] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.857583 24771 leader_election.cc:290] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 2 election: Requested vote from peers 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:47.857887 24846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a81d7f4ccb459ab2e722f09c10d9a1" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0b1901a1549f476696340f2fc61af194"
I20260709 12:25:47.858382 24921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a81d7f4ccb459ab2e722f09c10d9a1" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
I20260709 12:25:47.858641 24846 raft_consensus.cc:3060] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:47.859225 24921 raft_consensus.cc:3055] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:47.859527 24921 raft_consensus.cc:740] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Running, Role: LEADER
I20260709 12:25:47.860299 24921 consensus_queue.cc:260] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.861390 24921 raft_consensus.cc:3060] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:47.864535 24846 raft_consensus.cc:2468] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f825b5c64844f7fb020b60f13e6fdb4 in term 2.
I20260709 12:25:47.865722 24738 leader_election.cc:304] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:47.866490 24996 raft_consensus.cc:2804] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:47.867076 24996 raft_consensus.cc:697] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 LEADER]: Becoming Leader. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Running, Role: LEADER
I20260709 12:25:47.867081 24921 raft_consensus.cc:2468] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f825b5c64844f7fb020b60f13e6fdb4 in term 2.
I20260709 12:25:47.868294 24996 consensus_queue.cc:237] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.875563 24666 catalog_manager.cc:5697] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4 reported cstate change: term changed from 1 to 2, leader changed from 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3) to 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1). New cstate: current_term: 2 leader_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:47.975628 24988 raft_consensus.cc:993] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499: : Instructing follower 0b1901a1549f476696340f2fc61af194 to start an election
I20260709 12:25:47.976158 24984 raft_consensus.cc:1081] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Signalling peer 0b1901a1549f476696340f2fc61af194 to start an election
I20260709 12:25:47.977759 24846 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1df8831adbf14c3594dea358a98bfff3"
dest_uuid: "0b1901a1549f476696340f2fc61af194"
 from {username='slave'} at 127.0.0.1:47934
I20260709 12:25:47.978242 24846 raft_consensus.cc:493] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:47.978536 24846 raft_consensus.cc:3060] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:47.983888 24846 raft_consensus.cc:515] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.985551 24846 leader_election.cc:290] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 2 election: Requested vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:47.986640 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df8831adbf14c3594dea358a98bfff3" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
I20260709 12:25:47.986814 24921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df8831adbf14c3594dea358a98bfff3" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
I20260709 12:25:47.987272 24771 raft_consensus.cc:3060] T 1df8831adbf14c3594dea358a98bfff3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:47.987473 24921 raft_consensus.cc:3055] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:47.987874 24921 raft_consensus.cc:740] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Running, Role: LEADER
I20260709 12:25:47.988772 24921 consensus_queue.cc:260] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:47.989827 24921 raft_consensus.cc:3060] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:47.993136 24771 raft_consensus.cc:2468] T 1df8831adbf14c3594dea358a98bfff3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b1901a1549f476696340f2fc61af194 in term 2.
I20260709 12:25:47.994423 24813 leader_election.cc:304] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:47.995186 24970 raft_consensus.cc:2804] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:47.995754 24970 raft_consensus.cc:697] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194 [term 2 LEADER]: Becoming Leader. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Running, Role: LEADER
I20260709 12:25:47.996906 24921 raft_consensus.cc:2468] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b1901a1549f476696340f2fc61af194 in term 2.
I20260709 12:25:47.996663 24970 consensus_queue.cc:237] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:48.003994 24666 catalog_manager.cc:5697] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194 reported cstate change: term changed from 1 to 2, leader changed from 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3) to 0b1901a1549f476696340f2fc61af194 (127.5.58.2). New cstate: current_term: 2 leader_uuid: "0b1901a1549f476696340f2fc61af194" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:48.191160 24921 raft_consensus.cc:1275] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Refusing update from remote peer 6f825b5c64844f7fb020b60f13e6fdb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:48.192618 24996 consensus_queue.cc:1048] T f1d7d9ea11c24a428a3ca7a20b11e81c P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:48.200548 24846 raft_consensus.cc:1275] T f1d7d9ea11c24a428a3ca7a20b11e81c P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Refusing update from remote peer 6f825b5c64844f7fb020b60f13e6fdb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:48.202625 25027 consensus_queue.cc:1048] T f1d7d9ea11c24a428a3ca7a20b11e81c P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:48.318430 24921 raft_consensus.cc:1275] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Refusing update from remote peer 6f825b5c64844f7fb020b60f13e6fdb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:48.319585 24846 raft_consensus.cc:1275] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Refusing update from remote peer 6f825b5c64844f7fb020b60f13e6fdb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:48.319965 25027 consensus_queue.cc:1048] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:48.321135 24966 consensus_queue.cc:1048] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:48.334241 24847 raft_consensus.cc:1275] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Refusing update from remote peer 6f825b5c64844f7fb020b60f13e6fdb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:48.335377 24921 raft_consensus.cc:1275] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Refusing update from remote peer 6f825b5c64844f7fb020b60f13e6fdb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:48.337535 25045 consensus_queue.cc:1048] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:48.339699 24996 consensus_queue.cc:1048] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:48.382800 24920 raft_consensus.cc:1275] T 8f0eec1401e842eeb0541c2dd6621e58 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Refusing update from remote peer 0b1901a1549f476696340f2fc61af194: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:48.384349 24970 consensus_queue.cc:1048] T 8f0eec1401e842eeb0541c2dd6621e58 P 0b1901a1549f476696340f2fc61af194 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:48.393301 24771 raft_consensus.cc:1275] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Refusing update from remote peer 0b1901a1549f476696340f2fc61af194: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:48.395784 24985 consensus_queue.cc:1048] T 8f0eec1401e842eeb0541c2dd6621e58 P 0b1901a1549f476696340f2fc61af194 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:48.459120 24771 raft_consensus.cc:1275] T 1df8831adbf14c3594dea358a98bfff3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Refusing update from remote peer 0b1901a1549f476696340f2fc61af194: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:48.460494 24985 consensus_queue.cc:1048] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:48.469437 24920 raft_consensus.cc:1275] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Refusing update from remote peer 0b1901a1549f476696340f2fc61af194: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:48.471601 24987 consensus_queue.cc:1048] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:48.748219 25027 raft_consensus.cc:493] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0b1901a1549f476696340f2fc61af194)
I20260709 12:25:48.748881 25027 raft_consensus.cc:515] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:48.750871 25027 leader_election.cc:290] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:48.752249 24847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36fd425a2ae24bba8cc5934237f465b6" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194" is_pre_election: true
I20260709 12:25:48.752277 24920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36fd425a2ae24bba8cc5934237f465b6" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499" is_pre_election: true
I20260709 12:25:48.753032 24847 raft_consensus.cc:2468] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f825b5c64844f7fb020b60f13e6fdb4 in term 2.
I20260709 12:25:48.753194 24920 raft_consensus.cc:2393] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6f825b5c64844f7fb020b60f13e6fdb4 in current term 2: Already voted for candidate 2807a439752a4b0ab39c82cdef37f499 in this term.
I20260709 12:25:48.754719 24738 leader_election.cc:304] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 2807a439752a4b0ab39c82cdef37f499
I20260709 12:25:48.755548 25027 raft_consensus.cc:2804] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:48.755846 25027 raft_consensus.cc:493] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0b1901a1549f476696340f2fc61af194)
I20260709 12:25:48.756219 25027 raft_consensus.cc:3060] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:48.761951 25027 raft_consensus.cc:515] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:48.763634 25027 leader_election.cc:290] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 2 election: Requested vote from peers 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:48.764844 24847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36fd425a2ae24bba8cc5934237f465b6" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194"
I20260709 12:25:48.764889 24920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36fd425a2ae24bba8cc5934237f465b6" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
I20260709 12:25:48.765868 24920 raft_consensus.cc:2393] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6f825b5c64844f7fb020b60f13e6fdb4 in current term 2: Already voted for candidate 2807a439752a4b0ab39c82cdef37f499 in this term.
I20260709 12:25:48.771170 24847 raft_consensus.cc:2468] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f825b5c64844f7fb020b60f13e6fdb4 in term 2.
I20260709 12:25:48.772378 24738 leader_election.cc:304] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 2807a439752a4b0ab39c82cdef37f499
I20260709 12:25:48.773073 25027 raft_consensus.cc:2804] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:48.773563 25027 raft_consensus.cc:697] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 LEADER]: Becoming Leader. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Running, Role: LEADER
I20260709 12:25:48.774597 25027 consensus_queue.cc:237] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:48.782430 24669 catalog_manager.cc:5697] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 reported cstate change: term changed from 1 to 2, leader changed from 0b1901a1549f476696340f2fc61af194 (127.5.58.2) to 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1). New cstate: current_term: 2 leader_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:49.026057 24988 raft_consensus.cc:493] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.026588 24988 raft_consensus.cc:515] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:49.029045 24988 leader_election.cc:290] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:25:49.029951 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36fd425a2ae24bba8cc5934237f465b6" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" is_pre_election: true
I20260709 12:25:49.030371 24847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36fd425a2ae24bba8cc5934237f465b6" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194" is_pre_election: true
I20260709 12:25:49.031687 24847 raft_consensus.cc:2468] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 2.
I20260709 12:25:49.033183 24887 leader_election.cc:304] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 2807a439752a4b0ab39c82cdef37f499; no voters: 6f825b5c64844f7fb020b60f13e6fdb4
I20260709 12:25:49.034080 24988 raft_consensus.cc:2804] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:49.034505 24988 raft_consensus.cc:493] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:49.034868 24988 raft_consensus.cc:3060] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:49.042219 24988 raft_consensus.cc:515] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:49.044005 24988 leader_election.cc:290] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 3 election: Requested vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:25:49.045274 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36fd425a2ae24bba8cc5934237f465b6" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
I20260709 12:25:49.045499 24847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36fd425a2ae24bba8cc5934237f465b6" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194"
I20260709 12:25:49.046128 24847 raft_consensus.cc:3060] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:49.053077 24847 raft_consensus.cc:2468] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 3.
I20260709 12:25:49.054193 24887 leader_election.cc:304] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 2807a439752a4b0ab39c82cdef37f499; no voters: 6f825b5c64844f7fb020b60f13e6fdb4
I20260709 12:25:49.055172 24988 raft_consensus.cc:2804] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:49.056630 24988 raft_consensus.cc:697] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [term 3 LEADER]: Becoming Leader. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Running, Role: LEADER
I20260709 12:25:49.057569 24988 consensus_queue.cc:237] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:49.065646 24669 catalog_manager.cc:5697] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 reported cstate change: term changed from 2 to 3, leader changed from 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1) to 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3). New cstate: current_term: 3 leader_uuid: "2807a439752a4b0ab39c82cdef37f499" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:49.176726 24920 raft_consensus.cc:1240] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [term 3 LEADER]: Rejecting Update request from peer 6f825b5c64844f7fb020b60f13e6fdb4 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:49.178232 25027 consensus_queue.cc:1059] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:49.178866 25027 consensus_queue.cc:1243] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Peer 2807a439752a4b0ab39c82cdef37f499 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20260709 12:25:49.179316 24996 raft_consensus.cc:3055] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:49.179600 24996 raft_consensus.cc:740] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Running, Role: LEADER
I20260709 12:25:49.180274 24996 consensus_queue.cc:260] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:49.181216 24996 raft_consensus.cc:3060] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:49.498330 24771 raft_consensus.cc:1275] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 3 FOLLOWER]: Refusing update from remote peer 2807a439752a4b0ab39c82cdef37f499: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:25:49.498636 24771 pending_rounds.cc:85] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4: Aborting all ops after (but not including) 1
I20260709 12:25:49.498769 24771 pending_rounds.cc:107] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:25:49.498955 24771 raft_consensus.cc:2889] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:49.500379 24988 consensus_queue.cc:1048] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:49.510594 24847 raft_consensus.cc:1275] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [term 3 FOLLOWER]: Refusing update from remote peer 2807a439752a4b0ab39c82cdef37f499: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:49.512785 25056 consensus_queue.cc:1048] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:50.568537 24669 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42322:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:25:50.570993 24669 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:25:50.622987 24761 tablet_service.cc:1511] Processing CreateTablet for tablet 65f6dd3a57e04c6a880d0fecbd65401f (DEFAULT_TABLE table=soft_deleted_table [id=c3590c8078c04b8c80ea4d2e16f47c35]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:25:50.624431 24761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65f6dd3a57e04c6a880d0fecbd65401f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:50.624260 24836 tablet_service.cc:1511] Processing CreateTablet for tablet 65f6dd3a57e04c6a880d0fecbd65401f (DEFAULT_TABLE table=soft_deleted_table [id=c3590c8078c04b8c80ea4d2e16f47c35]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:25:50.625666 24836 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65f6dd3a57e04c6a880d0fecbd65401f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:50.625403 24911 tablet_service.cc:1511] Processing CreateTablet for tablet 65f6dd3a57e04c6a880d0fecbd65401f (DEFAULT_TABLE table=soft_deleted_table [id=c3590c8078c04b8c80ea4d2e16f47c35]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:25:50.626639 24911 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65f6dd3a57e04c6a880d0fecbd65401f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:50.628675 24758 tablet_service.cc:1511] Processing CreateTablet for tablet bb7e15e532104619bfc9dfcc15baceea (DEFAULT_TABLE table=soft_deleted_table [id=c3590c8078c04b8c80ea4d2e16f47c35]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:25:50.629962 24758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb7e15e532104619bfc9dfcc15baceea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:50.632236 24759 tablet_service.cc:1511] Processing CreateTablet for tablet f379878d1a0c474aab696ddb355df430 (DEFAULT_TABLE table=soft_deleted_table [id=c3590c8078c04b8c80ea4d2e16f47c35]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:25:50.633702 24759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f379878d1a0c474aab696ddb355df430. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:50.638276 24760 tablet_service.cc:1511] Processing CreateTablet for tablet 804a1b4315914cf9895b192eace7f7d5 (DEFAULT_TABLE table=soft_deleted_table [id=c3590c8078c04b8c80ea4d2e16f47c35]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:25:50.639714 24760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 804a1b4315914cf9895b192eace7f7d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:50.641067 24908 tablet_service.cc:1511] Processing CreateTablet for tablet f379878d1a0c474aab696ddb355df430 (DEFAULT_TABLE table=soft_deleted_table [id=c3590c8078c04b8c80ea4d2e16f47c35]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:25:50.641350 24909 tablet_service.cc:1511] Processing CreateTablet for tablet bb7e15e532104619bfc9dfcc15baceea (DEFAULT_TABLE table=soft_deleted_table [id=c3590c8078c04b8c80ea4d2e16f47c35]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:25:50.641601 24831 tablet_service.cc:1511] Processing CreateTablet for tablet bb7e15e532104619bfc9dfcc15baceea (DEFAULT_TABLE table=soft_deleted_table [id=c3590c8078c04b8c80ea4d2e16f47c35]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:25:50.642486 24908 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f379878d1a0c474aab696ddb355df430. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:50.643164 24909 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb7e15e532104619bfc9dfcc15baceea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:50.643213 24831 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb7e15e532104619bfc9dfcc15baceea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:50.643659 24762 tablet_service.cc:1511] Processing CreateTablet for tablet f51f7a5450c04dc9a93b0efbda38b7ea (DEFAULT_TABLE table=soft_deleted_table [id=c3590c8078c04b8c80ea4d2e16f47c35]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:25:50.644980 24762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f51f7a5450c04dc9a93b0efbda38b7ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:50.645493 24830 tablet_service.cc:1511] Processing CreateTablet for tablet f379878d1a0c474aab696ddb355df430 (DEFAULT_TABLE table=soft_deleted_table [id=c3590c8078c04b8c80ea4d2e16f47c35]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:25:50.646934 24830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f379878d1a0c474aab696ddb355df430. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:50.647423 24832 tablet_service.cc:1511] Processing CreateTablet for tablet 804a1b4315914cf9895b192eace7f7d5 (DEFAULT_TABLE table=soft_deleted_table [id=c3590c8078c04b8c80ea4d2e16f47c35]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:25:50.648824 24832 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 804a1b4315914cf9895b192eace7f7d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:50.653612 24907 tablet_service.cc:1511] Processing CreateTablet for tablet 804a1b4315914cf9895b192eace7f7d5 (DEFAULT_TABLE table=soft_deleted_table [id=c3590c8078c04b8c80ea4d2e16f47c35]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:25:50.655077 24907 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 804a1b4315914cf9895b192eace7f7d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:50.662180 24835 tablet_service.cc:1511] Processing CreateTablet for tablet f51f7a5450c04dc9a93b0efbda38b7ea (DEFAULT_TABLE table=soft_deleted_table [id=c3590c8078c04b8c80ea4d2e16f47c35]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:25:50.663578 24835 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f51f7a5450c04dc9a93b0efbda38b7ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:50.675515 24910 tablet_service.cc:1511] Processing CreateTablet for tablet f51f7a5450c04dc9a93b0efbda38b7ea (DEFAULT_TABLE table=soft_deleted_table [id=c3590c8078c04b8c80ea4d2e16f47c35]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:25:50.677534 24910 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f51f7a5450c04dc9a93b0efbda38b7ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:50.698313 25071 tablet_bootstrap.cc:492] T f379878d1a0c474aab696ddb355df430 P 6f825b5c64844f7fb020b60f13e6fdb4: Bootstrap starting.
I20260709 12:25:50.717371 24761 tablet_service.cc:1511] Processing CreateTablet for tablet 959e5d187b7445d4ad05d5ddd448d101 (DEFAULT_TABLE table=soft_deleted_table [id=c3590c8078c04b8c80ea4d2e16f47c35]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:25:50.719018 24761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 959e5d187b7445d4ad05d5ddd448d101. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:50.728260 24833 tablet_service.cc:1511] Processing CreateTablet for tablet 959e5d187b7445d4ad05d5ddd448d101 (DEFAULT_TABLE table=soft_deleted_table [id=c3590c8078c04b8c80ea4d2e16f47c35]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:25:50.729666 24833 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 959e5d187b7445d4ad05d5ddd448d101. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:50.733616 25070 tablet_bootstrap.cc:492] T f379878d1a0c474aab696ddb355df430 P 0b1901a1549f476696340f2fc61af194: Bootstrap starting.
I20260709 12:25:50.738977 24906 tablet_service.cc:1511] Processing CreateTablet for tablet 959e5d187b7445d4ad05d5ddd448d101 (DEFAULT_TABLE table=soft_deleted_table [id=c3590c8078c04b8c80ea4d2e16f47c35]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:25:50.740497 24906 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 959e5d187b7445d4ad05d5ddd448d101. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:50.745693 24905 tablet_service.cc:1511] Processing CreateTablet for tablet 8484b63eb29b4275b24ccd0cd68b87ce (DEFAULT_TABLE table=soft_deleted_table [id=c3590c8078c04b8c80ea4d2e16f47c35]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:25:50.746810 24834 tablet_service.cc:1511] Processing CreateTablet for tablet 8484b63eb29b4275b24ccd0cd68b87ce (DEFAULT_TABLE table=soft_deleted_table [id=c3590c8078c04b8c80ea4d2e16f47c35]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:25:50.747269 24905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8484b63eb29b4275b24ccd0cd68b87ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:50.754043 24837 tablet_service.cc:1511] Processing CreateTablet for tablet a55055527e0941eeb03b4f81ea2a6777 (DEFAULT_TABLE table=soft_deleted_table [id=c3590c8078c04b8c80ea4d2e16f47c35]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:25:50.755461 24837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a55055527e0941eeb03b4f81ea2a6777. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:50.757369 24904 tablet_service.cc:1511] Processing CreateTablet for tablet a55055527e0941eeb03b4f81ea2a6777 (DEFAULT_TABLE table=soft_deleted_table [id=c3590c8078c04b8c80ea4d2e16f47c35]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:25:50.758095 24757 tablet_service.cc:1511] Processing CreateTablet for tablet ebfd2dfa84b74747bd5e4c3cf31902bf (DEFAULT_TABLE table=soft_deleted_table [id=c3590c8078c04b8c80ea4d2e16f47c35]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:25:50.760109 24757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebfd2dfa84b74747bd5e4c3cf31902bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:50.760581 24829 tablet_service.cc:1511] Processing CreateTablet for tablet ebfd2dfa84b74747bd5e4c3cf31902bf (DEFAULT_TABLE table=soft_deleted_table [id=c3590c8078c04b8c80ea4d2e16f47c35]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:25:50.762984 24903 tablet_service.cc:1511] Processing CreateTablet for tablet ebfd2dfa84b74747bd5e4c3cf31902bf (DEFAULT_TABLE table=soft_deleted_table [id=c3590c8078c04b8c80ea4d2e16f47c35]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:25:50.764700 24834 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8484b63eb29b4275b24ccd0cd68b87ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:50.777575 24829 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebfd2dfa84b74747bd5e4c3cf31902bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:50.765872 24762 tablet_service.cc:1511] Processing CreateTablet for tablet 8484b63eb29b4275b24ccd0cd68b87ce (DEFAULT_TABLE table=soft_deleted_table [id=c3590c8078c04b8c80ea4d2e16f47c35]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:25:50.758756 24904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a55055527e0941eeb03b4f81ea2a6777. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:50.768450 24759 tablet_service.cc:1511] Processing CreateTablet for tablet a55055527e0941eeb03b4f81ea2a6777 (DEFAULT_TABLE table=soft_deleted_table [id=c3590c8078c04b8c80ea4d2e16f47c35]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:25:50.785490 24762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8484b63eb29b4275b24ccd0cd68b87ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:50.785528 24903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebfd2dfa84b74747bd5e4c3cf31902bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:50.794673 25071 tablet_bootstrap.cc:654] T f379878d1a0c474aab696ddb355df430 P 6f825b5c64844f7fb020b60f13e6fdb4: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:50.795687 24759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a55055527e0941eeb03b4f81ea2a6777. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:25:50.802773 24718 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:25:50.796828 25070 tablet_bootstrap.cc:654] T f379878d1a0c474aab696ddb355df430 P 0b1901a1549f476696340f2fc61af194: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:50.810295 25072 tablet_bootstrap.cc:492] T f379878d1a0c474aab696ddb355df430 P 2807a439752a4b0ab39c82cdef37f499: Bootstrap starting.
I20260709 12:25:50.825326 25072 tablet_bootstrap.cc:654] T f379878d1a0c474aab696ddb355df430 P 2807a439752a4b0ab39c82cdef37f499: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:50.835865 25070 tablet_bootstrap.cc:492] T f379878d1a0c474aab696ddb355df430 P 0b1901a1549f476696340f2fc61af194: No bootstrap required, opened a new log
I20260709 12:25:50.836313 25070 ts_tablet_manager.cc:1403] T f379878d1a0c474aab696ddb355df430 P 0b1901a1549f476696340f2fc61af194: Time spent bootstrapping tablet: real 0.103s	user 0.003s	sys 0.009s
I20260709 12:25:50.839432 25070 raft_consensus.cc:359] T f379878d1a0c474aab696ddb355df430 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:50.844079 25070 raft_consensus.cc:385] T f379878d1a0c474aab696ddb355df430 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:50.844787 25070 raft_consensus.cc:740] T f379878d1a0c474aab696ddb355df430 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Initialized, Role: FOLLOWER
I20260709 12:25:50.845556 25070 consensus_queue.cc:260] T f379878d1a0c474aab696ddb355df430 P 0b1901a1549f476696340f2fc61af194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:50.847010 24970 raft_consensus.cc:493] T f379878d1a0c474aab696ddb355df430 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.847771 24970 raft_consensus.cc:515] T f379878d1a0c474aab696ddb355df430 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:50.852480 24970 leader_election.cc:290] T f379878d1a0c474aab696ddb355df430 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:50.854519 24772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379878d1a0c474aab696ddb355df430" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" is_pre_election: true
I20260709 12:25:50.855176 25070 ts_tablet_manager.cc:1434] T f379878d1a0c474aab696ddb355df430 P 0b1901a1549f476696340f2fc61af194: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.002s
I20260709 12:25:50.856266 25070 tablet_bootstrap.cc:492] T 65f6dd3a57e04c6a880d0fecbd65401f P 0b1901a1549f476696340f2fc61af194: Bootstrap starting.
I20260709 12:25:50.854595 24920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379878d1a0c474aab696ddb355df430" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499" is_pre_election: true
W20260709 12:25:50.859462 24810 leader_election.cc:343] T f379878d1a0c474aab696ddb355df430 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:25:50.862548 24813 leader_election.cc:343] T f379878d1a0c474aab696ddb355df430 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:50.863267 24813 leader_election.cc:304] T f379878d1a0c474aab696ddb355df430 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0b1901a1549f476696340f2fc61af194; no voters: 2807a439752a4b0ab39c82cdef37f499, 6f825b5c64844f7fb020b60f13e6fdb4
I20260709 12:25:50.869284 25070 tablet_bootstrap.cc:654] T 65f6dd3a57e04c6a880d0fecbd65401f P 0b1901a1549f476696340f2fc61af194: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:50.869745 24970 raft_consensus.cc:2749] T f379878d1a0c474aab696ddb355df430 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:25:50.874501 25072 tablet_bootstrap.cc:492] T f379878d1a0c474aab696ddb355df430 P 2807a439752a4b0ab39c82cdef37f499: No bootstrap required, opened a new log
I20260709 12:25:50.875257 25072 ts_tablet_manager.cc:1403] T f379878d1a0c474aab696ddb355df430 P 2807a439752a4b0ab39c82cdef37f499: Time spent bootstrapping tablet: real 0.067s	user 0.009s	sys 0.008s
I20260709 12:25:50.878326 25071 tablet_bootstrap.cc:492] T f379878d1a0c474aab696ddb355df430 P 6f825b5c64844f7fb020b60f13e6fdb4: No bootstrap required, opened a new log
I20260709 12:25:50.878862 25071 ts_tablet_manager.cc:1403] T f379878d1a0c474aab696ddb355df430 P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent bootstrapping tablet: real 0.181s	user 0.021s	sys 0.005s
I20260709 12:25:50.879294 25072 raft_consensus.cc:359] T f379878d1a0c474aab696ddb355df430 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:50.880817 25072 raft_consensus.cc:385] T f379878d1a0c474aab696ddb355df430 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:50.881300 25072 raft_consensus.cc:740] T f379878d1a0c474aab696ddb355df430 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Initialized, Role: FOLLOWER
I20260709 12:25:50.881498 25071 raft_consensus.cc:359] T f379878d1a0c474aab696ddb355df430 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:50.882726 25071 raft_consensus.cc:385] T f379878d1a0c474aab696ddb355df430 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:50.882401 25072 consensus_queue.cc:260] T f379878d1a0c474aab696ddb355df430 P 2807a439752a4b0ab39c82cdef37f499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:50.883065 25071 raft_consensus.cc:740] T f379878d1a0c474aab696ddb355df430 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Initialized, Role: FOLLOWER
I20260709 12:25:50.883962 25071 consensus_queue.cc:260] T f379878d1a0c474aab696ddb355df430 P 6f825b5c64844f7fb020b60f13e6fdb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:50.884541 25070 tablet_bootstrap.cc:492] T 65f6dd3a57e04c6a880d0fecbd65401f P 0b1901a1549f476696340f2fc61af194: No bootstrap required, opened a new log
I20260709 12:25:50.885000 25072 ts_tablet_manager.cc:1434] T f379878d1a0c474aab696ddb355df430 P 2807a439752a4b0ab39c82cdef37f499: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.001s
I20260709 12:25:50.885279 25070 ts_tablet_manager.cc:1403] T 65f6dd3a57e04c6a880d0fecbd65401f P 0b1901a1549f476696340f2fc61af194: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.009s
I20260709 12:25:50.886314 25072 tablet_bootstrap.cc:492] T 804a1b4315914cf9895b192eace7f7d5 P 2807a439752a4b0ab39c82cdef37f499: Bootstrap starting.
I20260709 12:25:50.888991 25070 raft_consensus.cc:359] T 65f6dd3a57e04c6a880d0fecbd65401f P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:50.889803 25070 raft_consensus.cc:385] T 65f6dd3a57e04c6a880d0fecbd65401f P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:50.890146 25070 raft_consensus.cc:740] T 65f6dd3a57e04c6a880d0fecbd65401f P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Initialized, Role: FOLLOWER
I20260709 12:25:50.891090 25070 consensus_queue.cc:260] T 65f6dd3a57e04c6a880d0fecbd65401f P 0b1901a1549f476696340f2fc61af194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:50.894775 25072 tablet_bootstrap.cc:654] T 804a1b4315914cf9895b192eace7f7d5 P 2807a439752a4b0ab39c82cdef37f499: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:50.897187 25070 ts_tablet_manager.cc:1434] T 65f6dd3a57e04c6a880d0fecbd65401f P 0b1901a1549f476696340f2fc61af194: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 12:25:50.898416 25070 tablet_bootstrap.cc:492] T f51f7a5450c04dc9a93b0efbda38b7ea P 0b1901a1549f476696340f2fc61af194: Bootstrap starting.
I20260709 12:25:50.902035 25072 tablet_bootstrap.cc:492] T 804a1b4315914cf9895b192eace7f7d5 P 2807a439752a4b0ab39c82cdef37f499: No bootstrap required, opened a new log
I20260709 12:25:50.902673 25072 ts_tablet_manager.cc:1403] T 804a1b4315914cf9895b192eace7f7d5 P 2807a439752a4b0ab39c82cdef37f499: Time spent bootstrapping tablet: real 0.017s	user 0.014s	sys 0.001s
I20260709 12:25:50.906785 25070 tablet_bootstrap.cc:654] T f51f7a5450c04dc9a93b0efbda38b7ea P 0b1901a1549f476696340f2fc61af194: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:50.893625 25071 ts_tablet_manager.cc:1434] T f379878d1a0c474aab696ddb355df430 P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:25:50.908963 25071 tablet_bootstrap.cc:492] T 65f6dd3a57e04c6a880d0fecbd65401f P 6f825b5c64844f7fb020b60f13e6fdb4: Bootstrap starting.
I20260709 12:25:50.909510 25072 raft_consensus.cc:359] T 804a1b4315914cf9895b192eace7f7d5 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:50.910696 25072 raft_consensus.cc:385] T 804a1b4315914cf9895b192eace7f7d5 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:50.911195 25072 raft_consensus.cc:740] T 804a1b4315914cf9895b192eace7f7d5 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Initialized, Role: FOLLOWER
I20260709 12:25:50.912176 25072 consensus_queue.cc:260] T 804a1b4315914cf9895b192eace7f7d5 P 2807a439752a4b0ab39c82cdef37f499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:50.917547 25072 ts_tablet_manager.cc:1434] T 804a1b4315914cf9895b192eace7f7d5 P 2807a439752a4b0ab39c82cdef37f499: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:25:50.919312 25072 tablet_bootstrap.cc:492] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499: Bootstrap starting.
I20260709 12:25:50.921429 25070 tablet_bootstrap.cc:492] T f51f7a5450c04dc9a93b0efbda38b7ea P 0b1901a1549f476696340f2fc61af194: No bootstrap required, opened a new log
I20260709 12:25:50.922317 25070 ts_tablet_manager.cc:1403] T f51f7a5450c04dc9a93b0efbda38b7ea P 0b1901a1549f476696340f2fc61af194: Time spent bootstrapping tablet: real 0.024s	user 0.017s	sys 0.003s
I20260709 12:25:50.925931 25072 tablet_bootstrap.cc:654] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:50.925966 25070 raft_consensus.cc:359] T f51f7a5450c04dc9a93b0efbda38b7ea P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:50.927193 25070 raft_consensus.cc:385] T f51f7a5450c04dc9a93b0efbda38b7ea P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:50.927742 25070 raft_consensus.cc:740] T f51f7a5450c04dc9a93b0efbda38b7ea P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Initialized, Role: FOLLOWER
I20260709 12:25:50.928604 25070 consensus_queue.cc:260] T f51f7a5450c04dc9a93b0efbda38b7ea P 0b1901a1549f476696340f2fc61af194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:50.932055 25070 ts_tablet_manager.cc:1434] T f51f7a5450c04dc9a93b0efbda38b7ea P 0b1901a1549f476696340f2fc61af194: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.007s
I20260709 12:25:50.932669 25072 tablet_bootstrap.cc:492] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499: No bootstrap required, opened a new log
I20260709 12:25:50.933091 25071 tablet_bootstrap.cc:654] T 65f6dd3a57e04c6a880d0fecbd65401f P 6f825b5c64844f7fb020b60f13e6fdb4: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:50.933235 25070 tablet_bootstrap.cc:492] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194: Bootstrap starting.
I20260709 12:25:50.933243 25072 ts_tablet_manager.cc:1403] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499: Time spent bootstrapping tablet: real 0.014s	user 0.012s	sys 0.001s
I20260709 12:25:50.939256 25072 raft_consensus.cc:359] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:50.940215 25072 raft_consensus.cc:385] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:50.940589 25072 raft_consensus.cc:740] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Initialized, Role: FOLLOWER
I20260709 12:25:50.941488 25072 consensus_queue.cc:260] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:50.942842 25071 tablet_bootstrap.cc:492] T 65f6dd3a57e04c6a880d0fecbd65401f P 6f825b5c64844f7fb020b60f13e6fdb4: No bootstrap required, opened a new log
I20260709 12:25:50.943470 25071 ts_tablet_manager.cc:1403] T 65f6dd3a57e04c6a880d0fecbd65401f P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent bootstrapping tablet: real 0.035s	user 0.011s	sys 0.004s
I20260709 12:25:50.943536 25070 tablet_bootstrap.cc:654] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:50.943924 25072 ts_tablet_manager.cc:1434] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 12:25:50.945119 25072 tablet_bootstrap.cc:492] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499: Bootstrap starting.
I20260709 12:25:50.946978 25071 raft_consensus.cc:359] T 65f6dd3a57e04c6a880d0fecbd65401f P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:50.947803 25071 raft_consensus.cc:385] T 65f6dd3a57e04c6a880d0fecbd65401f P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:50.948155 25071 raft_consensus.cc:740] T 65f6dd3a57e04c6a880d0fecbd65401f P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Initialized, Role: FOLLOWER
I20260709 12:25:50.948995 25071 consensus_queue.cc:260] T 65f6dd3a57e04c6a880d0fecbd65401f P 6f825b5c64844f7fb020b60f13e6fdb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:50.955138 25072 tablet_bootstrap.cc:654] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:50.962013 25071 ts_tablet_manager.cc:1434] T 65f6dd3a57e04c6a880d0fecbd65401f P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:25:50.962141 25072 tablet_bootstrap.cc:492] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499: No bootstrap required, opened a new log
I20260709 12:25:50.963469 25072 ts_tablet_manager.cc:1403] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.009s
I20260709 12:25:50.965696 25070 tablet_bootstrap.cc:492] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194: No bootstrap required, opened a new log
I20260709 12:25:50.966300 25070 ts_tablet_manager.cc:1403] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194: Time spent bootstrapping tablet: real 0.033s	user 0.012s	sys 0.016s
I20260709 12:25:50.963596 25071 tablet_bootstrap.cc:492] T f51f7a5450c04dc9a93b0efbda38b7ea P 6f825b5c64844f7fb020b60f13e6fdb4: Bootstrap starting.
I20260709 12:25:50.969434 25070 raft_consensus.cc:359] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:50.970276 25070 raft_consensus.cc:385] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:50.970588 25070 raft_consensus.cc:740] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Initialized, Role: FOLLOWER
I20260709 12:25:50.970924 25072 raft_consensus.cc:359] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:50.971768 25072 raft_consensus.cc:385] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:50.971499 25070 consensus_queue.cc:260] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:50.972106 25072 raft_consensus.cc:740] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Initialized, Role: FOLLOWER
I20260709 12:25:50.973577 25071 tablet_bootstrap.cc:654] T f51f7a5450c04dc9a93b0efbda38b7ea P 6f825b5c64844f7fb020b60f13e6fdb4: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:50.973541 25072 consensus_queue.cc:260] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:50.974400 25070 ts_tablet_manager.cc:1434] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 12:25:50.977779 25070 tablet_bootstrap.cc:492] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194: Bootstrap starting.
I20260709 12:25:50.976357 25072 ts_tablet_manager.cc:1434] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.001s
I20260709 12:25:50.984424 25070 tablet_bootstrap.cc:654] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:50.996024 25072 tablet_bootstrap.cc:492] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499: Bootstrap starting.
I20260709 12:25:50.997135 25070 tablet_bootstrap.cc:492] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194: No bootstrap required, opened a new log
I20260709 12:25:51.000372 25070 ts_tablet_manager.cc:1403] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.008s
I20260709 12:25:51.002805 25071 tablet_bootstrap.cc:492] T f51f7a5450c04dc9a93b0efbda38b7ea P 6f825b5c64844f7fb020b60f13e6fdb4: No bootstrap required, opened a new log
I20260709 12:25:51.003407 25071 ts_tablet_manager.cc:1403] T f51f7a5450c04dc9a93b0efbda38b7ea P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent bootstrapping tablet: real 0.040s	user 0.023s	sys 0.013s
I20260709 12:25:51.003809 25072 tablet_bootstrap.cc:654] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:51.003808 25070 raft_consensus.cc:359] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.004818 25070 raft_consensus.cc:385] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:51.005203 25070 raft_consensus.cc:740] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Initialized, Role: FOLLOWER
I20260709 12:25:51.005971 25070 consensus_queue.cc:260] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.006767 25071 raft_consensus.cc:359] T f51f7a5450c04dc9a93b0efbda38b7ea P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.007840 25071 raft_consensus.cc:385] T f51f7a5450c04dc9a93b0efbda38b7ea P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:51.008163 25071 raft_consensus.cc:740] T f51f7a5450c04dc9a93b0efbda38b7ea P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Initialized, Role: FOLLOWER
I20260709 12:25:51.008621 25070 ts_tablet_manager.cc:1434] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:25:51.009006 25071 consensus_queue.cc:260] T f51f7a5450c04dc9a93b0efbda38b7ea P 6f825b5c64844f7fb020b60f13e6fdb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.009778 25070 tablet_bootstrap.cc:492] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194: Bootstrap starting.
I20260709 12:25:51.011672 25071 ts_tablet_manager.cc:1434] T f51f7a5450c04dc9a93b0efbda38b7ea P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:25:51.013041 25071 tablet_bootstrap.cc:492] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4: Bootstrap starting.
I20260709 12:25:51.015777 25072 tablet_bootstrap.cc:492] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499: No bootstrap required, opened a new log
I20260709 12:25:51.019632 25072 ts_tablet_manager.cc:1403] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499: Time spent bootstrapping tablet: real 0.032s	user 0.012s	sys 0.007s
I20260709 12:25:51.021572 25056 raft_consensus.cc:493] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.022100 25056 raft_consensus.cc:515] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.022991 25072 raft_consensus.cc:359] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.023700 25072 raft_consensus.cc:385] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:51.023999 25072 raft_consensus.cc:740] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Initialized, Role: FOLLOWER
I20260709 12:25:51.017233 25070 tablet_bootstrap.cc:654] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:51.024278 25056 leader_election.cc:290] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:25:51.024690 25071 tablet_bootstrap.cc:654] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:51.027715 25072 consensus_queue.cc:260] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.030447 25072 ts_tablet_manager.cc:1434] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.003s
I20260709 12:25:51.031464 25072 tablet_bootstrap.cc:492] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499: Bootstrap starting.
I20260709 12:25:51.038578 25072 tablet_bootstrap.cc:654] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:51.039810 25076 raft_consensus.cc:493] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.040692 25076 raft_consensus.cc:515] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.044790 25076 leader_election.cc:290] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:51.046396 24920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb7e15e532104619bfc9dfcc15baceea" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499" is_pre_election: true
I20260709 12:25:51.047348 24920 raft_consensus.cc:2468] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b1901a1549f476696340f2fc61af194 in term 0.
I20260709 12:25:51.045033 24772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb7e15e532104619bfc9dfcc15baceea" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" is_pre_election: true
I20260709 12:25:51.048702 25071 tablet_bootstrap.cc:492] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4: No bootstrap required, opened a new log
I20260709 12:25:51.048991 24810 leader_election.cc:304] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 2807a439752a4b0ab39c82cdef37f499; no voters: 
I20260709 12:25:51.049345 25071 ts_tablet_manager.cc:1403] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent bootstrapping tablet: real 0.037s	user 0.016s	sys 0.010s
W20260709 12:25:51.049942 24813 leader_election.cc:343] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:51.050014 25076 raft_consensus.cc:2804] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:25:51.050525 25076 raft_consensus.cc:493] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.050906 25076 raft_consensus.cc:3060] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:51.053242 24772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f51f7a5450c04dc9a93b0efbda38b7ea" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" is_pre_election: true
I20260709 12:25:51.053385 24846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f51f7a5450c04dc9a93b0efbda38b7ea" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194" is_pre_election: true
I20260709 12:25:51.053926 24772 raft_consensus.cc:2468] T f51f7a5450c04dc9a93b0efbda38b7ea P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 0.
I20260709 12:25:51.054152 24846 raft_consensus.cc:2468] T f51f7a5450c04dc9a93b0efbda38b7ea P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 0.
I20260709 12:25:51.054821 25056 raft_consensus.cc:493] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.055320 25056 raft_consensus.cc:515] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.054176 25071 raft_consensus.cc:359] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.056051 25070 tablet_bootstrap.cc:492] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194: No bootstrap required, opened a new log
I20260709 12:25:51.056375 25071 raft_consensus.cc:385] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:51.056667 25070 ts_tablet_manager.cc:1403] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194: Time spent bootstrapping tablet: real 0.047s	user 0.015s	sys 0.001s
I20260709 12:25:51.056809 25071 raft_consensus.cc:740] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Initialized, Role: FOLLOWER
I20260709 12:25:51.057451 25056 leader_election.cc:290] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:25:51.057833 25071 consensus_queue.cc:260] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.060146 24887 leader_election.cc:304] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 2807a439752a4b0ab39c82cdef37f499; no voters: 
I20260709 12:25:51.060386 25070 raft_consensus.cc:359] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.061517 25056 raft_consensus.cc:2804] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:25:51.062180 24847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65f6dd3a57e04c6a880d0fecbd65401f" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194" is_pre_election: true
I20260709 12:25:51.070237 25070 raft_consensus.cc:385] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:51.070283 25056 raft_consensus.cc:493] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.070518 24847 raft_consensus.cc:2468] T 65f6dd3a57e04c6a880d0fecbd65401f P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 0.
I20260709 12:25:51.070784 25070 raft_consensus.cc:740] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Initialized, Role: FOLLOWER
I20260709 12:25:51.070955 25056 raft_consensus.cc:3060] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:51.071483 25070 consensus_queue.cc:260] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.071908 24887 leader_election.cc:304] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 2807a439752a4b0ab39c82cdef37f499; no voters: 
I20260709 12:25:51.074069 25070 ts_tablet_manager.cc:1434] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:25:51.067157 24772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65f6dd3a57e04c6a880d0fecbd65401f" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" is_pre_election: true
I20260709 12:25:51.075266 24772 raft_consensus.cc:2468] T 65f6dd3a57e04c6a880d0fecbd65401f P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 0.
I20260709 12:25:51.075593 25070 tablet_bootstrap.cc:492] T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194: Bootstrap starting.
I20260709 12:25:51.077471 25087 raft_consensus.cc:2804] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:25:51.077929 25087 raft_consensus.cc:493] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.078270 25087 raft_consensus.cc:3060] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:51.085976 25070 tablet_bootstrap.cc:654] T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:51.069350 25076 raft_consensus.cc:515] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.086529 25087 raft_consensus.cc:515] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.088512 25076 leader_election.cc:290] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 election: Requested vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:51.089219 24772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb7e15e532104619bfc9dfcc15baceea" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
I20260709 12:25:51.089852 24772 raft_consensus.cc:3060] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:51.090446 25056 raft_consensus.cc:515] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.092976 25056 leader_election.cc:290] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 election: Requested vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:25:51.069530 25071 ts_tablet_manager.cc:1434] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.000s
I20260709 12:25:51.096421 25071 tablet_bootstrap.cc:492] T 804a1b4315914cf9895b192eace7f7d5 P 6f825b5c64844f7fb020b60f13e6fdb4: Bootstrap starting.
I20260709 12:25:51.103010 25071 tablet_bootstrap.cc:654] T 804a1b4315914cf9895b192eace7f7d5 P 6f825b5c64844f7fb020b60f13e6fdb4: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:51.095062 25087 leader_election.cc:290] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 election: Requested vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:25:51.115160 25070 tablet_bootstrap.cc:492] T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194: No bootstrap required, opened a new log
I20260709 12:25:51.115821 25070 ts_tablet_manager.cc:1403] T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194: Time spent bootstrapping tablet: real 0.041s	user 0.027s	sys 0.008s
I20260709 12:25:51.119292 25070 raft_consensus.cc:359] T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
W20260709 12:25:51.123386 24887 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:44285, user_credentials={real_user=slave}} blocked reactor thread for 48679.4us
I20260709 12:25:51.123667 25070 raft_consensus.cc:385] T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:51.124425 25070 raft_consensus.cc:740] T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Initialized, Role: FOLLOWER
I20260709 12:25:51.125519 25070 consensus_queue.cc:260] T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.126366 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f51f7a5450c04dc9a93b0efbda38b7ea" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
I20260709 12:25:51.127223 24771 raft_consensus.cc:3060] T f51f7a5450c04dc9a93b0efbda38b7ea P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:51.122962 25071 tablet_bootstrap.cc:492] T 804a1b4315914cf9895b192eace7f7d5 P 6f825b5c64844f7fb020b60f13e6fdb4: No bootstrap required, opened a new log
I20260709 12:25:51.128839 25071 ts_tablet_manager.cc:1403] T 804a1b4315914cf9895b192eace7f7d5 P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent bootstrapping tablet: real 0.033s	user 0.009s	sys 0.004s
I20260709 12:25:51.133304 25071 raft_consensus.cc:359] T 804a1b4315914cf9895b192eace7f7d5 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.134186 25071 raft_consensus.cc:385] T 804a1b4315914cf9895b192eace7f7d5 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:51.134575 25071 raft_consensus.cc:740] T 804a1b4315914cf9895b192eace7f7d5 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Initialized, Role: FOLLOWER
I20260709 12:25:51.136626 25071 consensus_queue.cc:260] T 804a1b4315914cf9895b192eace7f7d5 P 6f825b5c64844f7fb020b60f13e6fdb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.135746 25070 ts_tablet_manager.cc:1434] T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194: Time spent starting tablet: real 0.020s	user 0.002s	sys 0.007s
I20260709 12:25:51.138374 24847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f51f7a5450c04dc9a93b0efbda38b7ea" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194"
I20260709 12:25:51.139281 24847 raft_consensus.cc:3060] T f51f7a5450c04dc9a93b0efbda38b7ea P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:51.141109 25071 ts_tablet_manager.cc:1434] T 804a1b4315914cf9895b192eace7f7d5 P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent starting tablet: real 0.012s	user 0.000s	sys 0.009s
I20260709 12:25:51.142109 24772 raft_consensus.cc:2468] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b1901a1549f476696340f2fc61af194 in term 1.
I20260709 12:25:51.142354 25071 tablet_bootstrap.cc:492] T 959e5d187b7445d4ad05d5ddd448d101 P 6f825b5c64844f7fb020b60f13e6fdb4: Bootstrap starting.
I20260709 12:25:51.147568 24813 leader_election.cc:304] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:51.138733 25070 tablet_bootstrap.cc:492] T 8484b63eb29b4275b24ccd0cd68b87ce P 0b1901a1549f476696340f2fc61af194: Bootstrap starting.
I20260709 12:25:51.148950 25076 raft_consensus.cc:2804] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:51.149624 25076 raft_consensus.cc:697] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [term 1 LEADER]: Becoming Leader. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Running, Role: LEADER
I20260709 12:25:51.150748 25076 consensus_queue.cc:237] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.156317 25070 tablet_bootstrap.cc:654] T 8484b63eb29b4275b24ccd0cd68b87ce P 0b1901a1549f476696340f2fc61af194: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:51.163036 24771 raft_consensus.cc:2468] T f51f7a5450c04dc9a93b0efbda38b7ea P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 1.
I20260709 12:25:51.165840 24846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65f6dd3a57e04c6a880d0fecbd65401f" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194"
I20260709 12:25:51.166565 24846 raft_consensus.cc:3060] T 65f6dd3a57e04c6a880d0fecbd65401f P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:51.166798 25072 tablet_bootstrap.cc:492] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499: No bootstrap required, opened a new log
I20260709 12:25:51.167315 25072 ts_tablet_manager.cc:1403] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499: Time spent bootstrapping tablet: real 0.136s	user 0.034s	sys 0.040s
I20260709 12:25:51.170603 25072 raft_consensus.cc:359] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.171306 25072 raft_consensus.cc:385] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:51.171595 25072 raft_consensus.cc:740] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Initialized, Role: FOLLOWER
I20260709 12:25:51.173820 24847 raft_consensus.cc:2468] T f51f7a5450c04dc9a93b0efbda38b7ea P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 1.
I20260709 12:25:51.172792 24996 raft_consensus.cc:493] T f379878d1a0c474aab696ddb355df430 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.174686 24996 raft_consensus.cc:515] T f379878d1a0c474aab696ddb355df430 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.176414 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65f6dd3a57e04c6a880d0fecbd65401f" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
I20260709 12:25:51.177017 24771 raft_consensus.cc:3060] T 65f6dd3a57e04c6a880d0fecbd65401f P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:51.185974 24771 raft_consensus.cc:2468] T 65f6dd3a57e04c6a880d0fecbd65401f P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 1.
I20260709 12:25:51.172741 25072 consensus_queue.cc:260] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.180742 24996 leader_election.cc:290] T f379878d1a0c474aab696ddb355df430 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:51.187508 24920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379878d1a0c474aab696ddb355df430" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499" is_pre_election: true
I20260709 12:25:51.188311 24920 raft_consensus.cc:2468] T f379878d1a0c474aab696ddb355df430 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f825b5c64844f7fb020b60f13e6fdb4 in term 0.
W20260709 12:25:51.179205 24810 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:45247, user_credentials={real_user=slave}} blocked reactor thread for 105627us
I20260709 12:25:51.187546 24970 raft_consensus.cc:493] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.190153 24887 leader_election.cc:304] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2807a439752a4b0ab39c82cdef37f499, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:51.190531 24735 leader_election.cc:304] T f379878d1a0c474aab696ddb355df430 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2807a439752a4b0ab39c82cdef37f499, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:51.188922 25071 tablet_bootstrap.cc:654] T 959e5d187b7445d4ad05d5ddd448d101 P 6f825b5c64844f7fb020b60f13e6fdb4: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:51.191103 25072 ts_tablet_manager.cc:1434] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499: Time spent starting tablet: real 0.023s	user 0.010s	sys 0.000s
I20260709 12:25:51.191461 25056 raft_consensus.cc:2804] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:51.192054 25056 raft_consensus.cc:697] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Becoming Leader. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Running, Role: LEADER
I20260709 12:25:51.192612 25072 tablet_bootstrap.cc:492] T 959e5d187b7445d4ad05d5ddd448d101 P 2807a439752a4b0ab39c82cdef37f499: Bootstrap starting.
I20260709 12:25:51.192991 25056 consensus_queue.cc:237] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.195302 24920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb7e15e532104619bfc9dfcc15baceea" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
I20260709 12:25:51.195576 24996 raft_consensus.cc:2804] T f379878d1a0c474aab696ddb355df430 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:25:51.191129 24887 leader_election.cc:304] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2807a439752a4b0ab39c82cdef37f499, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:51.195946 24920 raft_consensus.cc:3060] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:51.196040 24996 raft_consensus.cc:493] T f379878d1a0c474aab696ddb355df430 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.196549 24996 raft_consensus.cc:3060] T f379878d1a0c474aab696ddb355df430 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:51.197155 25070 tablet_bootstrap.cc:492] T 8484b63eb29b4275b24ccd0cd68b87ce P 0b1901a1549f476696340f2fc61af194: No bootstrap required, opened a new log
I20260709 12:25:51.197659 25070 ts_tablet_manager.cc:1403] T 8484b63eb29b4275b24ccd0cd68b87ce P 0b1901a1549f476696340f2fc61af194: Time spent bootstrapping tablet: real 0.059s	user 0.013s	sys 0.012s
I20260709 12:25:51.199231 24846 raft_consensus.cc:2468] T 65f6dd3a57e04c6a880d0fecbd65401f P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 1.
I20260709 12:25:51.190199 24970 raft_consensus.cc:515] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.200531 25070 raft_consensus.cc:359] T 8484b63eb29b4275b24ccd0cd68b87ce P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.201287 25070 raft_consensus.cc:385] T 8484b63eb29b4275b24ccd0cd68b87ce P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:51.201646 25070 raft_consensus.cc:740] T 8484b63eb29b4275b24ccd0cd68b87ce P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Initialized, Role: FOLLOWER
I20260709 12:25:51.202716 25070 consensus_queue.cc:260] T 8484b63eb29b4275b24ccd0cd68b87ce P 0b1901a1549f476696340f2fc61af194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.203768 24970 leader_election.cc:290] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:51.204903 25070 ts_tablet_manager.cc:1434] T 8484b63eb29b4275b24ccd0cd68b87ce P 0b1901a1549f476696340f2fc61af194: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 12:25:51.205900 25070 tablet_bootstrap.cc:492] T ebfd2dfa84b74747bd5e4c3cf31902bf P 0b1901a1549f476696340f2fc61af194: Bootstrap starting.
I20260709 12:25:51.208673 24920 raft_consensus.cc:2468] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b1901a1549f476696340f2fc61af194 in term 1.
I20260709 12:25:51.210309 25087 raft_consensus.cc:2804] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:51.210855 25087 raft_consensus.cc:697] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Becoming Leader. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Running, Role: LEADER
I20260709 12:25:51.211732 25087 consensus_queue.cc:237] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.212649 24846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379878d1a0c474aab696ddb355df430" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194" is_pre_election: true
I20260709 12:25:51.213780 24846 raft_consensus.cc:2468] T f379878d1a0c474aab696ddb355df430 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f825b5c64844f7fb020b60f13e6fdb4 in term 0.
I20260709 12:25:51.215574 25072 tablet_bootstrap.cc:654] T 959e5d187b7445d4ad05d5ddd448d101 P 2807a439752a4b0ab39c82cdef37f499: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:51.216820 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "804a1b4315914cf9895b192eace7f7d5" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" is_pre_election: true
I20260709 12:25:51.217537 24771 raft_consensus.cc:2468] T 804a1b4315914cf9895b192eace7f7d5 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b1901a1549f476696340f2fc61af194 in term 0.
I20260709 12:25:51.218921 25070 tablet_bootstrap.cc:654] T ebfd2dfa84b74747bd5e4c3cf31902bf P 0b1901a1549f476696340f2fc61af194: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:51.218724 24813 leader_election.cc:304] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:51.222362 25076 raft_consensus.cc:2804] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:25:51.222842 25076 raft_consensus.cc:493] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.223271 25076 raft_consensus.cc:3060] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:51.225638 24996 raft_consensus.cc:515] T f379878d1a0c474aab696ddb355df430 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.225946 24669 catalog_manager.cc:5697] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b1901a1549f476696340f2fc61af194 (127.5.58.2). New cstate: current_term: 1 leader_uuid: "0b1901a1549f476696340f2fc61af194" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:51.227958 24920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "804a1b4315914cf9895b192eace7f7d5" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499" is_pre_election: true
I20260709 12:25:51.234661 24846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379878d1a0c474aab696ddb355df430" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194"
I20260709 12:25:51.234972 24996 leader_election.cc:290] T f379878d1a0c474aab696ddb355df430 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 1 election: Requested vote from peers 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:51.235417 24846 raft_consensus.cc:3060] T f379878d1a0c474aab696ddb355df430 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:51.234333 24920 raft_consensus.cc:2468] T 804a1b4315914cf9895b192eace7f7d5 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b1901a1549f476696340f2fc61af194 in term 0.
I20260709 12:25:51.245151 25071 tablet_bootstrap.cc:492] T 959e5d187b7445d4ad05d5ddd448d101 P 6f825b5c64844f7fb020b60f13e6fdb4: No bootstrap required, opened a new log
I20260709 12:25:51.246258 25071 ts_tablet_manager.cc:1403] T 959e5d187b7445d4ad05d5ddd448d101 P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent bootstrapping tablet: real 0.104s	user 0.019s	sys 0.008s
I20260709 12:25:51.248157 24920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f379878d1a0c474aab696ddb355df430" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
I20260709 12:25:51.248996 24920 raft_consensus.cc:3060] T f379878d1a0c474aab696ddb355df430 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:51.249547 25071 raft_consensus.cc:359] T 959e5d187b7445d4ad05d5ddd448d101 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.250317 25071 raft_consensus.cc:385] T 959e5d187b7445d4ad05d5ddd448d101 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:51.250595 25071 raft_consensus.cc:740] T 959e5d187b7445d4ad05d5ddd448d101 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Initialized, Role: FOLLOWER
I20260709 12:25:51.251011 25072 tablet_bootstrap.cc:492] T 959e5d187b7445d4ad05d5ddd448d101 P 2807a439752a4b0ab39c82cdef37f499: No bootstrap required, opened a new log
I20260709 12:25:51.251417 25071 consensus_queue.cc:260] T 959e5d187b7445d4ad05d5ddd448d101 P 6f825b5c64844f7fb020b60f13e6fdb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.257596 25072 ts_tablet_manager.cc:1403] T 959e5d187b7445d4ad05d5ddd448d101 P 2807a439752a4b0ab39c82cdef37f499: Time spent bootstrapping tablet: real 0.065s	user 0.016s	sys 0.006s
I20260709 12:25:51.267405 25072 raft_consensus.cc:359] T 959e5d187b7445d4ad05d5ddd448d101 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.268229 25072 raft_consensus.cc:385] T 959e5d187b7445d4ad05d5ddd448d101 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:51.268600 25072 raft_consensus.cc:740] T 959e5d187b7445d4ad05d5ddd448d101 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Initialized, Role: FOLLOWER
I20260709 12:25:51.255352 25076 raft_consensus.cc:515] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.269425 25072 consensus_queue.cc:260] T 959e5d187b7445d4ad05d5ddd448d101 P 2807a439752a4b0ab39c82cdef37f499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.272567 25070 tablet_bootstrap.cc:492] T ebfd2dfa84b74747bd5e4c3cf31902bf P 0b1901a1549f476696340f2fc61af194: No bootstrap required, opened a new log
I20260709 12:25:51.273339 25070 ts_tablet_manager.cc:1403] T ebfd2dfa84b74747bd5e4c3cf31902bf P 0b1901a1549f476696340f2fc61af194: Time spent bootstrapping tablet: real 0.068s	user 0.011s	sys 0.001s
I20260709 12:25:51.274004 25072 ts_tablet_manager.cc:1434] T 959e5d187b7445d4ad05d5ddd448d101 P 2807a439752a4b0ab39c82cdef37f499: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:25:51.274972 25072 tablet_bootstrap.cc:492] T a55055527e0941eeb03b4f81ea2a6777 P 2807a439752a4b0ab39c82cdef37f499: Bootstrap starting.
I20260709 12:25:51.267253 24668 catalog_manager.cc:5697] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "2807a439752a4b0ab39c82cdef37f499" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:51.260541 24920 raft_consensus.cc:2468] T f379878d1a0c474aab696ddb355df430 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f825b5c64844f7fb020b60f13e6fdb4 in term 1.
I20260709 12:25:51.277559 24735 leader_election.cc:304] T f379878d1a0c474aab696ddb355df430 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2807a439752a4b0ab39c82cdef37f499, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:51.278481 24996 raft_consensus.cc:2804] T f379878d1a0c474aab696ddb355df430 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:51.279011 25076 leader_election.cc:290] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 election: Requested vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:51.280020 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "804a1b4315914cf9895b192eace7f7d5" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
I20260709 12:25:51.280730 24771 raft_consensus.cc:3060] T 804a1b4315914cf9895b192eace7f7d5 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:51.281186 24996 raft_consensus.cc:697] T f379878d1a0c474aab696ddb355df430 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 LEADER]: Becoming Leader. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Running, Role: LEADER
I20260709 12:25:51.282280 24996 consensus_queue.cc:237] T f379878d1a0c474aab696ddb355df430 P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.280730 24921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "804a1b4315914cf9895b192eace7f7d5" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
I20260709 12:25:51.285353 24921 raft_consensus.cc:3060] T 804a1b4315914cf9895b192eace7f7d5 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:51.287829 25072 tablet_bootstrap.cc:654] T a55055527e0941eeb03b4f81ea2a6777 P 2807a439752a4b0ab39c82cdef37f499: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:51.288694 25056 raft_consensus.cc:493] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.289235 25056 raft_consensus.cc:515] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.291563 25056 leader_election.cc:290] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:25:51.291842 25098 raft_consensus.cc:493] T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.292644 25098 raft_consensus.cc:515] T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.262982 24846 raft_consensus.cc:2468] T f379878d1a0c474aab696ddb355df430 P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f825b5c64844f7fb020b60f13e6fdb4 in term 1.
I20260709 12:25:51.263353 25071 ts_tablet_manager.cc:1434] T 959e5d187b7445d4ad05d5ddd448d101 P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:25:51.295737 24771 raft_consensus.cc:2468] T 804a1b4315914cf9895b192eace7f7d5 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b1901a1549f476696340f2fc61af194 in term 1.
I20260709 12:25:51.277623 25070 raft_consensus.cc:359] T ebfd2dfa84b74747bd5e4c3cf31902bf P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.297688 25070 raft_consensus.cc:385] T ebfd2dfa84b74747bd5e4c3cf31902bf P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:51.297950 25070 raft_consensus.cc:740] T ebfd2dfa84b74747bd5e4c3cf31902bf P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Initialized, Role: FOLLOWER
I20260709 12:25:51.277658 24668 catalog_manager.cc:5697] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "2807a439752a4b0ab39c82cdef37f499" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:51.298671 24847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebfd2dfa84b74747bd5e4c3cf31902bf" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194" is_pre_election: true
I20260709 12:25:51.299010 25098 leader_election.cc:290] T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:51.298620 25070 consensus_queue.cc:260] T ebfd2dfa84b74747bd5e4c3cf31902bf P 0b1901a1549f476696340f2fc61af194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.300503 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebfd2dfa84b74747bd5e4c3cf31902bf" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" is_pre_election: true
I20260709 12:25:51.301066 25071 tablet_bootstrap.cc:492] T a55055527e0941eeb03b4f81ea2a6777 P 6f825b5c64844f7fb020b60f13e6fdb4: Bootstrap starting.
I20260709 12:25:51.303218 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55055527e0941eeb03b4f81ea2a6777" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" is_pre_election: true
I20260709 12:25:51.301429 24847 raft_consensus.cc:2468] T ebfd2dfa84b74747bd5e4c3cf31902bf P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 0.
I20260709 12:25:51.301394 24813 leader_election.cc:304] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
W20260709 12:25:51.304251 24887 leader_election.cc:343] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:51.307346 25098 raft_consensus.cc:2804] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:51.309514 25070 ts_tablet_manager.cc:1434] T ebfd2dfa84b74747bd5e4c3cf31902bf P 0b1901a1549f476696340f2fc61af194: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.001s
I20260709 12:25:51.309573 24887 leader_election.cc:304] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 2807a439752a4b0ab39c82cdef37f499; no voters: 6f825b5c64844f7fb020b60f13e6fdb4
W20260709 12:25:51.313750 24813 leader_election.cc:343] T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:51.308099 25098 raft_consensus.cc:697] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194 [term 1 LEADER]: Becoming Leader. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Running, Role: LEADER
I20260709 12:25:51.314814 25072 tablet_bootstrap.cc:492] T a55055527e0941eeb03b4f81ea2a6777 P 2807a439752a4b0ab39c82cdef37f499: No bootstrap required, opened a new log
I20260709 12:25:51.315315 25072 ts_tablet_manager.cc:1403] T a55055527e0941eeb03b4f81ea2a6777 P 2807a439752a4b0ab39c82cdef37f499: Time spent bootstrapping tablet: real 0.041s	user 0.012s	sys 0.007s
I20260709 12:25:51.314973 25098 consensus_queue.cc:237] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.317507 25056 raft_consensus.cc:2804] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:25:51.317847 25056 raft_consensus.cc:493] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.318115 25056 raft_consensus.cc:3060] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:51.317987 25072 raft_consensus.cc:359] T a55055527e0941eeb03b4f81ea2a6777 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.318743 25072 raft_consensus.cc:385] T a55055527e0941eeb03b4f81ea2a6777 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:51.318982 25072 raft_consensus.cc:740] T a55055527e0941eeb03b4f81ea2a6777 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Initialized, Role: FOLLOWER
I20260709 12:25:51.319708 25072 consensus_queue.cc:260] T a55055527e0941eeb03b4f81ea2a6777 P 2807a439752a4b0ab39c82cdef37f499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.322017 25072 ts_tablet_manager.cc:1434] T a55055527e0941eeb03b4f81ea2a6777 P 2807a439752a4b0ab39c82cdef37f499: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:25:51.322994 25072 tablet_bootstrap.cc:492] T 8484b63eb29b4275b24ccd0cd68b87ce P 2807a439752a4b0ab39c82cdef37f499: Bootstrap starting.
I20260709 12:25:51.326303 25071 tablet_bootstrap.cc:654] T a55055527e0941eeb03b4f81ea2a6777 P 6f825b5c64844f7fb020b60f13e6fdb4: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:51.328938 25072 tablet_bootstrap.cc:654] T 8484b63eb29b4275b24ccd0cd68b87ce P 2807a439752a4b0ab39c82cdef37f499: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:51.331121 25056 raft_consensus.cc:515] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.332882 24920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55055527e0941eeb03b4f81ea2a6777" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499" is_pre_election: true
I20260709 12:25:51.324156 24921 raft_consensus.cc:2468] T 804a1b4315914cf9895b192eace7f7d5 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b1901a1549f476696340f2fc61af194 in term 1.
I20260709 12:25:51.334805 25056 leader_election.cc:290] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 election: Requested vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:25:51.333812 24920 raft_consensus.cc:2468] T a55055527e0941eeb03b4f81ea2a6777 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b1901a1549f476696340f2fc61af194 in term 0.
I20260709 12:25:51.339568 24810 leader_election.cc:304] T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 2807a439752a4b0ab39c82cdef37f499; no voters: 6f825b5c64844f7fb020b60f13e6fdb4
I20260709 12:25:51.340792 25076 raft_consensus.cc:2804] T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:25:51.341177 25076 raft_consensus.cc:493] T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.341545 25076 raft_consensus.cc:3060] T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:51.342803 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebfd2dfa84b74747bd5e4c3cf31902bf" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
W20260709 12:25:51.344604 24887 leader_election.cc:343] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:51.347874 24847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebfd2dfa84b74747bd5e4c3cf31902bf" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194"
I20260709 12:25:51.348529 24847 raft_consensus.cc:3060] T ebfd2dfa84b74747bd5e4c3cf31902bf P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:51.345775 24670 catalog_manager.cc:5697] T f379878d1a0c474aab696ddb355df430 P 6f825b5c64844f7fb020b60f13e6fdb4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:51.353233 24666 catalog_manager.cc:5697] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b1901a1549f476696340f2fc61af194 (127.5.58.2). New cstate: current_term: 1 leader_uuid: "0b1901a1549f476696340f2fc61af194" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:51.356009 24847 raft_consensus.cc:2468] T ebfd2dfa84b74747bd5e4c3cf31902bf P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 1.
I20260709 12:25:51.357421 24887 leader_election.cc:304] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 2807a439752a4b0ab39c82cdef37f499; no voters: 6f825b5c64844f7fb020b60f13e6fdb4
I20260709 12:25:51.358552 25056 raft_consensus.cc:2804] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:51.359023 25056 raft_consensus.cc:697] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Becoming Leader. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Running, Role: LEADER
I20260709 12:25:51.359845 25056 consensus_queue.cc:237] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.360419 25076 raft_consensus.cc:515] T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.363723 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55055527e0941eeb03b4f81ea2a6777" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
W20260709 12:25:51.369714 24813 leader_election.cc:343] T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:25:51.371413 25072 tablet_bootstrap.cc:492] T 8484b63eb29b4275b24ccd0cd68b87ce P 2807a439752a4b0ab39c82cdef37f499: No bootstrap required, opened a new log
I20260709 12:25:51.372022 25072 ts_tablet_manager.cc:1403] T 8484b63eb29b4275b24ccd0cd68b87ce P 2807a439752a4b0ab39c82cdef37f499: Time spent bootstrapping tablet: real 0.049s	user 0.011s	sys 0.000s
I20260709 12:25:51.373916 25076 leader_election.cc:290] T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 election: Requested vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:51.375545 25072 raft_consensus.cc:359] T 8484b63eb29b4275b24ccd0cd68b87ce P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.376437 25072 raft_consensus.cc:385] T 8484b63eb29b4275b24ccd0cd68b87ce P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:51.376782 25072 raft_consensus.cc:740] T 8484b63eb29b4275b24ccd0cd68b87ce P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Initialized, Role: FOLLOWER
I20260709 12:25:51.377352 25071 tablet_bootstrap.cc:492] T a55055527e0941eeb03b4f81ea2a6777 P 6f825b5c64844f7fb020b60f13e6fdb4: No bootstrap required, opened a new log
I20260709 12:25:51.377974 25071 ts_tablet_manager.cc:1403] T a55055527e0941eeb03b4f81ea2a6777 P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent bootstrapping tablet: real 0.077s	user 0.010s	sys 0.022s
I20260709 12:25:51.377827 25072 consensus_queue.cc:260] T 8484b63eb29b4275b24ccd0cd68b87ce P 2807a439752a4b0ab39c82cdef37f499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.379509 24920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55055527e0941eeb03b4f81ea2a6777" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
I20260709 12:25:51.380440 24920 raft_consensus.cc:3060] T a55055527e0941eeb03b4f81ea2a6777 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:51.380669 25072 ts_tablet_manager.cc:1434] T 8484b63eb29b4275b24ccd0cd68b87ce P 2807a439752a4b0ab39c82cdef37f499: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:25:51.381765 25071 raft_consensus.cc:359] T a55055527e0941eeb03b4f81ea2a6777 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.382346 25071 raft_consensus.cc:385] T a55055527e0941eeb03b4f81ea2a6777 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:51.382547 25071 raft_consensus.cc:740] T a55055527e0941eeb03b4f81ea2a6777 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Initialized, Role: FOLLOWER
I20260709 12:25:51.383432 25071 consensus_queue.cc:260] T a55055527e0941eeb03b4f81ea2a6777 P 6f825b5c64844f7fb020b60f13e6fdb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.385514 25071 ts_tablet_manager.cc:1434] T a55055527e0941eeb03b4f81ea2a6777 P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 12:25:51.386482 25071 tablet_bootstrap.cc:492] T ebfd2dfa84b74747bd5e4c3cf31902bf P 6f825b5c64844f7fb020b60f13e6fdb4: Bootstrap starting.
I20260709 12:25:51.390470 24920 raft_consensus.cc:2468] T a55055527e0941eeb03b4f81ea2a6777 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b1901a1549f476696340f2fc61af194 in term 1.
I20260709 12:25:51.391855 24810 leader_election.cc:304] T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 2807a439752a4b0ab39c82cdef37f499; no voters: 6f825b5c64844f7fb020b60f13e6fdb4
I20260709 12:25:51.392735 25076 raft_consensus.cc:2804] T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:51.393200 25076 raft_consensus.cc:697] T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194 [term 1 LEADER]: Becoming Leader. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Running, Role: LEADER
I20260709 12:25:51.393294 25071 tablet_bootstrap.cc:654] T ebfd2dfa84b74747bd5e4c3cf31902bf P 6f825b5c64844f7fb020b60f13e6fdb4: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:51.394004 25076 consensus_queue.cc:237] T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.406767 25071 tablet_bootstrap.cc:492] T ebfd2dfa84b74747bd5e4c3cf31902bf P 6f825b5c64844f7fb020b60f13e6fdb4: No bootstrap required, opened a new log
I20260709 12:25:51.407393 25071 ts_tablet_manager.cc:1403] T ebfd2dfa84b74747bd5e4c3cf31902bf P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent bootstrapping tablet: real 0.021s	user 0.004s	sys 0.015s
I20260709 12:25:51.410570 25071 raft_consensus.cc:359] T ebfd2dfa84b74747bd5e4c3cf31902bf P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.411048 24666 catalog_manager.cc:5697] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "2807a439752a4b0ab39c82cdef37f499" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:51.411715 25071 raft_consensus.cc:385] T ebfd2dfa84b74747bd5e4c3cf31902bf P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:51.412096 25071 raft_consensus.cc:740] T ebfd2dfa84b74747bd5e4c3cf31902bf P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Initialized, Role: FOLLOWER
I20260709 12:25:51.412860 25071 consensus_queue.cc:260] T ebfd2dfa84b74747bd5e4c3cf31902bf P 6f825b5c64844f7fb020b60f13e6fdb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.415088 25071 ts_tablet_manager.cc:1434] T ebfd2dfa84b74747bd5e4c3cf31902bf P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 12:25:51.414258 24668 catalog_manager.cc:5697] T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b1901a1549f476696340f2fc61af194 (127.5.58.2). New cstate: current_term: 1 leader_uuid: "0b1901a1549f476696340f2fc61af194" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:51.416429 25071 tablet_bootstrap.cc:492] T 8484b63eb29b4275b24ccd0cd68b87ce P 6f825b5c64844f7fb020b60f13e6fdb4: Bootstrap starting.
I20260709 12:25:51.423278 25071 tablet_bootstrap.cc:654] T 8484b63eb29b4275b24ccd0cd68b87ce P 6f825b5c64844f7fb020b60f13e6fdb4: Neither blocks nor log segments found. Creating new log.
I20260709 12:25:51.429312 25071 tablet_bootstrap.cc:492] T 8484b63eb29b4275b24ccd0cd68b87ce P 6f825b5c64844f7fb020b60f13e6fdb4: No bootstrap required, opened a new log
I20260709 12:25:51.429771 25071 ts_tablet_manager.cc:1403] T 8484b63eb29b4275b24ccd0cd68b87ce P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent bootstrapping tablet: real 0.014s	user 0.010s	sys 0.001s
I20260709 12:25:51.432103 25071 raft_consensus.cc:359] T 8484b63eb29b4275b24ccd0cd68b87ce P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.432675 25071 raft_consensus.cc:385] T 8484b63eb29b4275b24ccd0cd68b87ce P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:25:51.432879 25071 raft_consensus.cc:740] T 8484b63eb29b4275b24ccd0cd68b87ce P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Initialized, Role: FOLLOWER
I20260709 12:25:51.433429 25071 consensus_queue.cc:260] T 8484b63eb29b4275b24ccd0cd68b87ce P 6f825b5c64844f7fb020b60f13e6fdb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.435462 25071 ts_tablet_manager.cc:1434] T 8484b63eb29b4275b24ccd0cd68b87ce P 6f825b5c64844f7fb020b60f13e6fdb4: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:25:51.442373 25076 raft_consensus.cc:493] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.442955 25076 raft_consensus.cc:515] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.445333 25076 leader_election.cc:290] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:51.446178 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "959e5d187b7445d4ad05d5ddd448d101" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" is_pre_election: true
I20260709 12:25:51.446411 25076 raft_consensus.cc:493] T 8484b63eb29b4275b24ccd0cd68b87ce P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.446482 24920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "959e5d187b7445d4ad05d5ddd448d101" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499" is_pre_election: true
I20260709 12:25:51.446944 24771 raft_consensus.cc:2468] T 959e5d187b7445d4ad05d5ddd448d101 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b1901a1549f476696340f2fc61af194 in term 0.
I20260709 12:25:51.447310 24920 raft_consensus.cc:2468] T 959e5d187b7445d4ad05d5ddd448d101 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b1901a1549f476696340f2fc61af194 in term 0.
I20260709 12:25:51.447088 25076 raft_consensus.cc:515] T 8484b63eb29b4275b24ccd0cd68b87ce P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.448081 24813 leader_election.cc:304] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:51.449033 25098 raft_consensus.cc:2804] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:25:51.449396 25098 raft_consensus.cc:493] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.449782 25098 raft_consensus.cc:3060] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:51.450083 25076 leader_election.cc:290] T 8484b63eb29b4275b24ccd0cd68b87ce P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:51.450688 24920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8484b63eb29b4275b24ccd0cd68b87ce" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499" is_pre_election: true
I20260709 12:25:51.451285 24920 raft_consensus.cc:2468] T 8484b63eb29b4275b24ccd0cd68b87ce P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b1901a1549f476696340f2fc61af194 in term 0.
I20260709 12:25:51.451448 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8484b63eb29b4275b24ccd0cd68b87ce" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" is_pre_election: true
I20260709 12:25:51.452106 24771 raft_consensus.cc:2468] T 8484b63eb29b4275b24ccd0cd68b87ce P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b1901a1549f476696340f2fc61af194 in term 0.
I20260709 12:25:51.452669 24810 leader_election.cc:304] T 8484b63eb29b4275b24ccd0cd68b87ce P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 2807a439752a4b0ab39c82cdef37f499; no voters: 
I20260709 12:25:51.454037 25076 raft_consensus.cc:2804] T 8484b63eb29b4275b24ccd0cd68b87ce P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:25:51.454468 25076 raft_consensus.cc:493] T 8484b63eb29b4275b24ccd0cd68b87ce P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.454833 25076 raft_consensus.cc:3060] T 8484b63eb29b4275b24ccd0cd68b87ce P 0b1901a1549f476696340f2fc61af194 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:51.457032 25098 raft_consensus.cc:515] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.459712 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "959e5d187b7445d4ad05d5ddd448d101" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
I20260709 12:25:51.460108 24921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "959e5d187b7445d4ad05d5ddd448d101" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
I20260709 12:25:51.460824 24921 raft_consensus.cc:3060] T 959e5d187b7445d4ad05d5ddd448d101 P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:51.462257 25076 raft_consensus.cc:515] T 8484b63eb29b4275b24ccd0cd68b87ce P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.460284 24771 raft_consensus.cc:3060] T 959e5d187b7445d4ad05d5ddd448d101 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:51.464221 25076 leader_election.cc:290] T 8484b63eb29b4275b24ccd0cd68b87ce P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 election: Requested vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:51.464936 25098 leader_election.cc:290] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 election: Requested vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:51.464958 24772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8484b63eb29b4275b24ccd0cd68b87ce" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
I20260709 12:25:51.465809 24772 raft_consensus.cc:3060] T 8484b63eb29b4275b24ccd0cd68b87ce P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:51.465960 24920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8484b63eb29b4275b24ccd0cd68b87ce" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
I20260709 12:25:51.466591 24920 raft_consensus.cc:3060] T 8484b63eb29b4275b24ccd0cd68b87ce P 2807a439752a4b0ab39c82cdef37f499 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:51.471233 24921 raft_consensus.cc:2468] T 959e5d187b7445d4ad05d5ddd448d101 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b1901a1549f476696340f2fc61af194 in term 1.
I20260709 12:25:51.472193 24771 raft_consensus.cc:2468] T 959e5d187b7445d4ad05d5ddd448d101 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b1901a1549f476696340f2fc61af194 in term 1.
I20260709 12:25:51.472834 24810 leader_election.cc:304] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 2807a439752a4b0ab39c82cdef37f499; no voters: 
I20260709 12:25:51.473740 24772 raft_consensus.cc:2468] T 8484b63eb29b4275b24ccd0cd68b87ce P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b1901a1549f476696340f2fc61af194 in term 1.
I20260709 12:25:51.473834 24920 raft_consensus.cc:2468] T 8484b63eb29b4275b24ccd0cd68b87ce P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b1901a1549f476696340f2fc61af194 in term 1.
I20260709 12:25:51.474985 25098 raft_consensus.cc:2804] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:51.475762 24810 leader_election.cc:304] T 8484b63eb29b4275b24ccd0cd68b87ce P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 2807a439752a4b0ab39c82cdef37f499; no voters: 
I20260709 12:25:51.476601 25076 raft_consensus.cc:2804] T 8484b63eb29b4275b24ccd0cd68b87ce P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:25:51.476248 25098 raft_consensus.cc:697] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194 [term 1 LEADER]: Becoming Leader. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Running, Role: LEADER
I20260709 12:25:51.477113 25076 raft_consensus.cc:697] T 8484b63eb29b4275b24ccd0cd68b87ce P 0b1901a1549f476696340f2fc61af194 [term 1 LEADER]: Becoming Leader. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Running, Role: LEADER
I20260709 12:25:51.478101 25098 consensus_queue.cc:237] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.478122 25076 consensus_queue.cc:237] T 8484b63eb29b4275b24ccd0cd68b87ce P 0b1901a1549f476696340f2fc61af194 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.488884 24668 catalog_manager.cc:5697] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b1901a1549f476696340f2fc61af194 (127.5.58.2). New cstate: current_term: 1 leader_uuid: "0b1901a1549f476696340f2fc61af194" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:51.501987 24668 catalog_manager.cc:5697] T 8484b63eb29b4275b24ccd0cd68b87ce P 0b1901a1549f476696340f2fc61af194 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b1901a1549f476696340f2fc61af194 (127.5.58.2). New cstate: current_term: 1 leader_uuid: "0b1901a1549f476696340f2fc61af194" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:51.592775 25107 raft_consensus.cc:493] T a55055527e0941eeb03b4f81ea2a6777 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.593329 25107 raft_consensus.cc:515] T a55055527e0941eeb03b4f81ea2a6777 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.595170 25107 leader_election.cc:290] T a55055527e0941eeb03b4f81ea2a6777 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:51.595898 24847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55055527e0941eeb03b4f81ea2a6777" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194" is_pre_election: true
I20260709 12:25:51.596375 24920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55055527e0941eeb03b4f81ea2a6777" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499" is_pre_election: true
I20260709 12:25:51.597172 24920 raft_consensus.cc:2393] T a55055527e0941eeb03b4f81ea2a6777 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6f825b5c64844f7fb020b60f13e6fdb4 in current term 1: Already voted for candidate 0b1901a1549f476696340f2fc61af194 in this term.
I20260709 12:25:51.598183 24735 leader_election.cc:304] T a55055527e0941eeb03b4f81ea2a6777 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 0b1901a1549f476696340f2fc61af194, 2807a439752a4b0ab39c82cdef37f499
I20260709 12:25:51.598908 25107 raft_consensus.cc:3060] T a55055527e0941eeb03b4f81ea2a6777 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:51.606300 25107 raft_consensus.cc:2749] T a55055527e0941eeb03b4f81ea2a6777 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:25:51.616495 25098 consensus_queue.cc:1048] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:51.625164 25098 consensus_queue.cc:1048] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:51.662525 24920 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "65f6dd3a57e04c6a880d0fecbd65401f"
dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
mode: GRACEFUL
new_leader_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
 from {username='slave'} at 127.0.0.1:57276
I20260709 12:25:51.663106 24920 raft_consensus.cc:606] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Received request to transfer leadership to TS 6f825b5c64844f7fb020b60f13e6fdb4
I20260709 12:25:51.682127 24847 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb7e15e532104619bfc9dfcc15baceea"
dest_uuid: "0b1901a1549f476696340f2fc61af194"
mode: GRACEFUL
new_leader_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
 from {username='slave'} at 127.0.0.1:47940
I20260709 12:25:51.682627 24847 raft_consensus.cc:606] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [term 1 LEADER]: Received request to transfer leadership to TS 6f825b5c64844f7fb020b60f13e6fdb4
I20260709 12:25:51.691516 24847 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "804a1b4315914cf9895b192eace7f7d5"
dest_uuid: "0b1901a1549f476696340f2fc61af194"
mode: GRACEFUL
new_leader_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
 from {username='slave'} at 127.0.0.1:47940
I20260709 12:25:51.691977 24847 raft_consensus.cc:606] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194 [term 1 LEADER]: Received request to transfer leadership to TS 6f825b5c64844f7fb020b60f13e6fdb4
I20260709 12:25:51.697470 24920 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f51f7a5450c04dc9a93b0efbda38b7ea"
dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
mode: GRACEFUL
new_leader_uuid: "0b1901a1549f476696340f2fc61af194"
 from {username='slave'} at 127.0.0.1:57276
I20260709 12:25:51.697749 25056 consensus_queue.cc:1048] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:51.698606 24920 raft_consensus.cc:606] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Received request to transfer leadership to TS 0b1901a1549f476696340f2fc61af194
I20260709 12:25:51.703540 25098 consensus_queue.cc:1048] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:25:51.703881 25107 consensus_queue.cc:1048] T f379878d1a0c474aab696ddb355df430 P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:25:51.710759 25093 raft_consensus.cc:993] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499: : Instructing follower 0b1901a1549f476696340f2fc61af194 to start an election
I20260709 12:25:51.711339 25093 raft_consensus.cc:1081] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Signalling peer 0b1901a1549f476696340f2fc61af194 to start an election
I20260709 12:25:51.714257 24847 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f51f7a5450c04dc9a93b0efbda38b7ea"
dest_uuid: "0b1901a1549f476696340f2fc61af194"
 from {username='slave'} at 127.0.0.1:47934
I20260709 12:25:51.714838 24847 raft_consensus.cc:493] T f51f7a5450c04dc9a93b0efbda38b7ea P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:51.715413 24847 raft_consensus.cc:3060] T f51f7a5450c04dc9a93b0efbda38b7ea P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:51.721539 25092 consensus_queue.cc:1048] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:25:51.722949 24847 raft_consensus.cc:515] T f51f7a5450c04dc9a93b0efbda38b7ea P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.724892 24847 leader_election.cc:290] T f51f7a5450c04dc9a93b0efbda38b7ea P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 2 election: Requested vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:51.725729 24770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f51f7a5450c04dc9a93b0efbda38b7ea" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
I20260709 12:25:51.730002 24921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f51f7a5450c04dc9a93b0efbda38b7ea" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
I20260709 12:25:51.730664 24921 raft_consensus.cc:3055] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:51.730988 24921 raft_consensus.cc:740] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Running, Role: LEADER
I20260709 12:25:51.731015 25098 consensus_queue.cc:1048] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:25:51.731945 24921 consensus_queue.cc:260] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.736059 24921 raft_consensus.cc:3060] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:51.733354 24847 raft_consensus.cc:1240] T f51f7a5450c04dc9a93b0efbda38b7ea P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Rejecting Update request from peer 2807a439752a4b0ab39c82cdef37f499 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:51.747747 24921 raft_consensus.cc:2468] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b1901a1549f476696340f2fc61af194 in term 2.
I20260709 12:25:51.748885 24810 leader_election.cc:304] T f51f7a5450c04dc9a93b0efbda38b7ea P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 2807a439752a4b0ab39c82cdef37f499; no voters: 6f825b5c64844f7fb020b60f13e6fdb4
I20260709 12:25:51.750283 25098 raft_consensus.cc:2804] T f51f7a5450c04dc9a93b0efbda38b7ea P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:51.750792 25098 raft_consensus.cc:697] T f51f7a5450c04dc9a93b0efbda38b7ea P 0b1901a1549f476696340f2fc61af194 [term 2 LEADER]: Becoming Leader. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Running, Role: LEADER
I20260709 12:25:51.751317 25135 consensus_queue.cc:1048] T f379878d1a0c474aab696ddb355df430 P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:25:51.751693 25098 consensus_queue.cc:237] T f51f7a5450c04dc9a93b0efbda38b7ea P 0b1901a1549f476696340f2fc61af194 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.754024 25076 raft_consensus.cc:993] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194: : Instructing follower 6f825b5c64844f7fb020b60f13e6fdb4 to start an election
I20260709 12:25:51.755990 25098 raft_consensus.cc:1081] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194 [term 1 LEADER]: Signalling peer 6f825b5c64844f7fb020b60f13e6fdb4 to start an election
I20260709 12:25:51.762552 24666 catalog_manager.cc:5697] T f51f7a5450c04dc9a93b0efbda38b7ea P 0b1901a1549f476696340f2fc61af194 reported cstate change: term changed from 1 to 2, leader changed from 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3) to 0b1901a1549f476696340f2fc61af194 (127.5.58.2). New cstate: current_term: 2 leader_uuid: "0b1901a1549f476696340f2fc61af194" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:51.779137 24772 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "804a1b4315914cf9895b192eace7f7d5"
dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
 from {username='slave'} at 127.0.0.1:41276
I20260709 12:25:51.779863 24772 raft_consensus.cc:493] T 804a1b4315914cf9895b192eace7f7d5 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:51.780217 24772 raft_consensus.cc:3060] T 804a1b4315914cf9895b192eace7f7d5 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:51.790158 25056 consensus_queue.cc:1048] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:51.790472 24772 raft_consensus.cc:515] T 804a1b4315914cf9895b192eace7f7d5 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.791751 24771 raft_consensus.cc:1240] T 804a1b4315914cf9895b192eace7f7d5 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Rejecting Update request from peer 0b1901a1549f476696340f2fc61af194 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:51.793462 24846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "804a1b4315914cf9895b192eace7f7d5" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0b1901a1549f476696340f2fc61af194"
I20260709 12:25:51.794064 24846 raft_consensus.cc:3055] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:51.794328 24846 raft_consensus.cc:740] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Running, Role: LEADER
I20260709 12:25:51.795763 24846 consensus_queue.cc:260] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.797394 24846 raft_consensus.cc:3060] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:51.805914 25099 consensus_queue.cc:1048] T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:25:51.806758 24846 raft_consensus.cc:2468] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f825b5c64844f7fb020b60f13e6fdb4 in term 2.
I20260709 12:25:51.807497 25087 consensus_queue.cc:1048] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:51.808059 24772 leader_election.cc:290] T 804a1b4315914cf9895b192eace7f7d5 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 2 election: Requested vote from peers 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:51.808425 24738 leader_election.cc:304] T 804a1b4315914cf9895b192eace7f7d5 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:51.809407 25133 raft_consensus.cc:2804] T 804a1b4315914cf9895b192eace7f7d5 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:51.809849 25133 raft_consensus.cc:697] T 804a1b4315914cf9895b192eace7f7d5 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 LEADER]: Becoming Leader. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Running, Role: LEADER
I20260709 12:25:51.810845 25133 consensus_queue.cc:237] T 804a1b4315914cf9895b192eace7f7d5 P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.814002 24919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "804a1b4315914cf9895b192eace7f7d5" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
I20260709 12:25:51.814693 24919 raft_consensus.cc:3060] T 804a1b4315914cf9895b192eace7f7d5 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:51.828765 25076 consensus_queue.cc:1048] T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:25:51.831863 24919 raft_consensus.cc:2468] T 804a1b4315914cf9895b192eace7f7d5 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f825b5c64844f7fb020b60f13e6fdb4 in term 2.
I20260709 12:25:51.864583 25087 raft_consensus.cc:993] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499: : Instructing follower 6f825b5c64844f7fb020b60f13e6fdb4 to start an election
I20260709 12:25:51.863763 24668 catalog_manager.cc:5697] T 804a1b4315914cf9895b192eace7f7d5 P 6f825b5c64844f7fb020b60f13e6fdb4 reported cstate change: term changed from 1 to 2, leader changed from 0b1901a1549f476696340f2fc61af194 (127.5.58.2) to 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1). New cstate: current_term: 2 leader_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:51.865125 25087 raft_consensus.cc:1081] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Signalling peer 6f825b5c64844f7fb020b60f13e6fdb4 to start an election
I20260709 12:25:51.867802 24772 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "65f6dd3a57e04c6a880d0fecbd65401f"
dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
 from {username='slave'} at 127.0.0.1:41260
I20260709 12:25:51.868489 24772 raft_consensus.cc:493] T 65f6dd3a57e04c6a880d0fecbd65401f P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:51.868919 24772 raft_consensus.cc:3060] T 65f6dd3a57e04c6a880d0fecbd65401f P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:51.875233 24772 raft_consensus.cc:515] T 65f6dd3a57e04c6a880d0fecbd65401f P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.877710 24772 leader_election.cc:290] T 65f6dd3a57e04c6a880d0fecbd65401f P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 2 election: Requested vote from peers 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:51.878509 24846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65f6dd3a57e04c6a880d0fecbd65401f" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0b1901a1549f476696340f2fc61af194"
I20260709 12:25:51.879098 24846 raft_consensus.cc:3060] T 65f6dd3a57e04c6a880d0fecbd65401f P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:51.879056 24920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65f6dd3a57e04c6a880d0fecbd65401f" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
I20260709 12:25:51.879794 24920 raft_consensus.cc:3055] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:51.880084 24920 raft_consensus.cc:740] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Running, Role: LEADER
I20260709 12:25:51.880841 24920 consensus_queue.cc:260] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.882050 24920 raft_consensus.cc:3060] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:51.886547 25133 raft_consensus.cc:493] T ebfd2dfa84b74747bd5e4c3cf31902bf P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.887116 25133 raft_consensus.cc:515] T ebfd2dfa84b74747bd5e4c3cf31902bf P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.887367 24846 raft_consensus.cc:2468] T 65f6dd3a57e04c6a880d0fecbd65401f P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f825b5c64844f7fb020b60f13e6fdb4 in term 2.
I20260709 12:25:51.889048 24738 leader_election.cc:304] T 65f6dd3a57e04c6a880d0fecbd65401f P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:51.890231 24996 raft_consensus.cc:2804] T 65f6dd3a57e04c6a880d0fecbd65401f P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:51.890403 24919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebfd2dfa84b74747bd5e4c3cf31902bf" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499" is_pre_election: true
I20260709 12:25:51.890712 24996 raft_consensus.cc:697] T 65f6dd3a57e04c6a880d0fecbd65401f P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 LEADER]: Becoming Leader. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Running, Role: LEADER
I20260709 12:25:51.891579 24996 consensus_queue.cc:237] T 65f6dd3a57e04c6a880d0fecbd65401f P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:51.889258 25133 leader_election.cc:290] T ebfd2dfa84b74747bd5e4c3cf31902bf P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:51.893288 24847 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebfd2dfa84b74747bd5e4c3cf31902bf" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194" is_pre_election: true
I20260709 12:25:51.894052 24847 raft_consensus.cc:2393] T ebfd2dfa84b74747bd5e4c3cf31902bf P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6f825b5c64844f7fb020b60f13e6fdb4 in current term 1: Already voted for candidate 2807a439752a4b0ab39c82cdef37f499 in this term.
I20260709 12:25:51.896173 24738 leader_election.cc:304] T ebfd2dfa84b74747bd5e4c3cf31902bf P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 0b1901a1549f476696340f2fc61af194, 2807a439752a4b0ab39c82cdef37f499
I20260709 12:25:51.897367 24996 raft_consensus.cc:3060] T ebfd2dfa84b74747bd5e4c3cf31902bf P 6f825b5c64844f7fb020b60f13e6fdb4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:25:51.898965 24920 raft_consensus.cc:2468] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f825b5c64844f7fb020b60f13e6fdb4 in term 2.
I20260709 12:25:51.902717 24669 catalog_manager.cc:5697] T 65f6dd3a57e04c6a880d0fecbd65401f P 6f825b5c64844f7fb020b60f13e6fdb4 reported cstate change: term changed from 1 to 2, leader changed from 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3) to 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1). New cstate: current_term: 2 leader_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:51.905407 24996 raft_consensus.cc:2749] T ebfd2dfa84b74747bd5e4c3cf31902bf P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:25:51.911233 25076 consensus_queue.cc:1048] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:51.918025 25056 consensus_queue.cc:1048] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:51.927234 25099 consensus_queue.cc:1048] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:51.934544 25099 consensus_queue.cc:1048] T 8484b63eb29b4275b24ccd0cd68b87ce P 0b1901a1549f476696340f2fc61af194 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:51.948427 25056 consensus_queue.cc:1048] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:25:51.952559 25136 consensus_queue.cc:1048] T 8484b63eb29b4275b24ccd0cd68b87ce P 0b1901a1549f476696340f2fc61af194 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:25:52.013986 25099 raft_consensus.cc:993] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194: : Instructing follower 6f825b5c64844f7fb020b60f13e6fdb4 to start an election
I20260709 12:25:52.014518 25136 raft_consensus.cc:1081] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [term 1 LEADER]: Signalling peer 6f825b5c64844f7fb020b60f13e6fdb4 to start an election
I20260709 12:25:52.016105 24772 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bb7e15e532104619bfc9dfcc15baceea"
dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
 from {username='slave'} at 127.0.0.1:41276
I20260709 12:25:52.016820 24772 raft_consensus.cc:493] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:52.017225 24772 raft_consensus.cc:3060] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:52.023025 24772 raft_consensus.cc:515] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:52.025082 24772 leader_election.cc:290] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 2 election: Requested vote from peers 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:25:52.025959 24846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb7e15e532104619bfc9dfcc15baceea" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0b1901a1549f476696340f2fc61af194"
I20260709 12:25:52.027501 24846 raft_consensus.cc:3055] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:52.027842 24846 raft_consensus.cc:740] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Running, Role: LEADER
I20260709 12:25:52.026854 24919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb7e15e532104619bfc9dfcc15baceea" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
I20260709 12:25:52.028703 24919 raft_consensus.cc:3060] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:52.028818 24846 consensus_queue.cc:260] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:52.030567 24846 raft_consensus.cc:3060] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:52.037429 24919 raft_consensus.cc:2468] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f825b5c64844f7fb020b60f13e6fdb4 in term 2.
I20260709 12:25:52.037961 24846 raft_consensus.cc:2468] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f825b5c64844f7fb020b60f13e6fdb4 in term 2.
I20260709 12:25:52.039003 24735 leader_election.cc:304] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2807a439752a4b0ab39c82cdef37f499, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:52.039968 24996 raft_consensus.cc:2804] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:52.040480 24996 raft_consensus.cc:697] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 LEADER]: Becoming Leader. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Running, Role: LEADER
I20260709 12:25:52.041359 24996 consensus_queue.cc:237] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:52.050158 24669 catalog_manager.cc:5697] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4 reported cstate change: term changed from 1 to 2, leader changed from 0b1901a1549f476696340f2fc61af194 (127.5.58.2) to 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1). New cstate: current_term: 2 leader_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:52.159108 24919 raft_consensus.cc:1275] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Refusing update from remote peer 0b1901a1549f476696340f2fc61af194: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:52.160714 25136 consensus_queue.cc:1048] T f51f7a5450c04dc9a93b0efbda38b7ea P 0b1901a1549f476696340f2fc61af194 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:52.169705 24772 raft_consensus.cc:3060] T f51f7a5450c04dc9a93b0efbda38b7ea P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:52.178377 24772 raft_consensus.cc:1275] T f51f7a5450c04dc9a93b0efbda38b7ea P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Refusing update from remote peer 0b1901a1549f476696340f2fc61af194: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:52.180431 25136 consensus_queue.cc:1048] T f51f7a5450c04dc9a93b0efbda38b7ea P 0b1901a1549f476696340f2fc61af194 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:52.214756 24846 raft_consensus.cc:1275] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Refusing update from remote peer 6f825b5c64844f7fb020b60f13e6fdb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:52.216187 24996 consensus_queue.cc:1048] T 804a1b4315914cf9895b192eace7f7d5 P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:52.225229 24919 raft_consensus.cc:1275] T 804a1b4315914cf9895b192eace7f7d5 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Refusing update from remote peer 6f825b5c64844f7fb020b60f13e6fdb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:52.226997 25133 consensus_queue.cc:1048] T 804a1b4315914cf9895b192eace7f7d5 P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:52.331800 24919 raft_consensus.cc:1275] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Refusing update from remote peer 6f825b5c64844f7fb020b60f13e6fdb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:52.333421 25135 consensus_queue.cc:1048] T 65f6dd3a57e04c6a880d0fecbd65401f P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:52.342125 24846 raft_consensus.cc:1275] T 65f6dd3a57e04c6a880d0fecbd65401f P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Refusing update from remote peer 6f825b5c64844f7fb020b60f13e6fdb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:52.343756 25135 consensus_queue.cc:1048] T 65f6dd3a57e04c6a880d0fecbd65401f P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:52.488581 24846 raft_consensus.cc:1275] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Refusing update from remote peer 6f825b5c64844f7fb020b60f13e6fdb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:52.489961 25135 consensus_queue.cc:1048] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:52.498167 24919 raft_consensus.cc:1275] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Refusing update from remote peer 6f825b5c64844f7fb020b60f13e6fdb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:52.499804 25107 consensus_queue.cc:1048] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:54.725176 24670 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42322:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:25:54.753304 24670 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:42322:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:25:54.838348  5352 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
W20260709 12:25:55.079327 25187 log.cc:927] Time spent T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194: Append to log took a long time: real 0.064s	user 0.002s	sys 0.000s
W20260709 12:25:55.143199 25176 log.cc:927] Time spent T 8484b63eb29b4275b24ccd0cd68b87ce P 0b1901a1549f476696340f2fc61af194: Append to log took a long time: real 0.083s	user 0.001s	sys 0.001s
W20260709 12:25:55.145915 24738 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.2:38549, user_credentials={real_user=slave}} blocked reactor thread for 69737.5us
I20260709 12:25:55.209126 25211 tablet.cc:1724] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:25:55.209175 25204 tablet.cc:1724] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:25:55.242362 25202 log.cc:927] Time spent T ebfd2dfa84b74747bd5e4c3cf31902bf P 6f825b5c64844f7fb020b60f13e6fdb4: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:25:55.243245 25203 tablet.cc:1724] T 65f6dd3a57e04c6a880d0fecbd65401f P 6f825b5c64844f7fb020b60f13e6fdb4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:25:55.246369 25207 tablet.cc:1724] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:25:55.280244 25209 tablet.cc:1724] T f379878d1a0c474aab696ddb355df430 P 6f825b5c64844f7fb020b60f13e6fdb4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:25:55.289024 24798 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:65f6dd3a57e04c6a880d0fecbd65401f)
W20260709 12:25:55.290445 24798 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:65f6dd3a57e04c6a880d0fecbd65401f)
I20260709 12:25:55.299412 25208 tablet.cc:1724] T ebfd2dfa84b74747bd5e4c3cf31902bf P 0b1901a1549f476696340f2fc61af194: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:25:55.318307 25201 log.cc:927] Time spent T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499: Append to log took a long time: real 0.122s	user 0.000s	sys 0.002s
I20260709 12:25:55.365466 25213 tablet.cc:1724] T f379878d1a0c474aab696ddb355df430 P 2807a439752a4b0ab39c82cdef37f499: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:25:55.392014 24846 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "959e5d187b7445d4ad05d5ddd448d101"
dest_uuid: "0b1901a1549f476696340f2fc61af194"
mode: GRACEFUL
new_leader_uuid: "2807a439752a4b0ab39c82cdef37f499"
 from {username='slave'} at 127.0.0.1:35014
I20260709 12:25:55.404995 25222 tablet.cc:1724] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:25:55.378940 25221 tablet.cc:1724] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:25:55.417186 25219 tablet.cc:1724] T ebfd2dfa84b74747bd5e4c3cf31902bf P 6f825b5c64844f7fb020b60f13e6fdb4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:25:55.429597 25218 log.cc:927] Time spent T 804a1b4315914cf9895b192eace7f7d5 P 2807a439752a4b0ab39c82cdef37f499: Append to log took a long time: real 0.121s	user 0.000s	sys 0.002s
I20260709 12:25:55.428874 25217 tablet.cc:1724] T f51f7a5450c04dc9a93b0efbda38b7ea P 0b1901a1549f476696340f2fc61af194: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:25:55.444923 25220 tablet.cc:1724] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:25:55.449836 25214 log.cc:927] Time spent T f51f7a5450c04dc9a93b0efbda38b7ea P 6f825b5c64844f7fb020b60f13e6fdb4: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 12:25:55.442215 24846 raft_consensus.cc:606] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194 [term 1 LEADER]: Received request to transfer leadership to TS 2807a439752a4b0ab39c82cdef37f499
I20260709 12:25:55.464949 25209 tablet.cc:1724] T 959e5d187b7445d4ad05d5ddd448d101 P 6f825b5c64844f7fb020b60f13e6fdb4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:25:55.478819 25204 tablet.cc:1724] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:25:55.478070 25213 tablet.cc:1724] T 8484b63eb29b4275b24ccd0cd68b87ce P 2807a439752a4b0ab39c82cdef37f499: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:25:55.485522 25203 tablet.cc:1724] T a55055527e0941eeb03b4f81ea2a6777 P 6f825b5c64844f7fb020b60f13e6fdb4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:25:55.508167 25208 tablet.cc:1724] T 65f6dd3a57e04c6a880d0fecbd65401f P 0b1901a1549f476696340f2fc61af194: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:25:55.497128 25209 tablet.cc:1724] T 804a1b4315914cf9895b192eace7f7d5 P 6f825b5c64844f7fb020b60f13e6fdb4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:25:55.499174 25220 tablet.cc:1724] T 8484b63eb29b4275b24ccd0cd68b87ce P 0b1901a1549f476696340f2fc61af194: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:25:55.513062 25211 tablet.cc:1724] T f51f7a5450c04dc9a93b0efbda38b7ea P 6f825b5c64844f7fb020b60f13e6fdb4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:25:55.502142 25221 tablet.cc:1724] T a55055527e0941eeb03b4f81ea2a6777 P 2807a439752a4b0ab39c82cdef37f499: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:25:55.523108 25222 tablet.cc:1724] T f379878d1a0c474aab696ddb355df430 P 0b1901a1549f476696340f2fc61af194: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:25:55.527521 25217 tablet.cc:1724] T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:25:55.504724 25207 tablet.cc:1724] T 959e5d187b7445d4ad05d5ddd448d101 P 2807a439752a4b0ab39c82cdef37f499: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:25:55.545758 25211 tablet.cc:1724] T 8484b63eb29b4275b24ccd0cd68b87ce P 6f825b5c64844f7fb020b60f13e6fdb4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:25:55.546630 24798 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:65f6dd3a57e04c6a880d0fecbd65401f)
I20260709 12:25:55.561797 25207 tablet.cc:1724] T 804a1b4315914cf9895b192eace7f7d5 P 2807a439752a4b0ab39c82cdef37f499: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:25:55.578881 25220 tablet.cc:1724] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:25:55.597832 24771 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb7e15e532104619bfc9dfcc15baceea"
dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
mode: GRACEFUL
new_leader_uuid: "2807a439752a4b0ab39c82cdef37f499"
 from {username='slave'} at 127.0.0.1:56084
I20260709 12:25:55.598506 24771 raft_consensus.cc:606] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 LEADER]: Received request to transfer leadership to TS 2807a439752a4b0ab39c82cdef37f499
I20260709 12:25:55.600144  5352 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 12:25:55.601173  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:55.612718 24771 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3f69b65186ed4f6c8fe7c50f6b0133d3"
dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
mode: GRACEFUL
new_leader_uuid: "2807a439752a4b0ab39c82cdef37f499"
 from {username='slave'} at 127.0.0.1:56084
I20260709 12:25:55.613389 24771 raft_consensus.cc:606] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 LEADER]: Received request to transfer leadership to TS 2807a439752a4b0ab39c82cdef37f499
I20260709 12:25:55.615551  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:25:55.615986  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:55.626385 25168 raft_consensus.cc:993] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4: : Instructing follower 2807a439752a4b0ab39c82cdef37f499 to start an election
I20260709 12:25:55.627182 25195 raft_consensus.cc:1081] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 LEADER]: Signalling peer 2807a439752a4b0ab39c82cdef37f499 to start an election
I20260709 12:25:55.629403 24918 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bb7e15e532104619bfc9dfcc15baceea"
dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
 from {username='slave'} at 127.0.0.1:57264
I20260709 12:25:55.630065 24918 raft_consensus.cc:493] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:55.630472 24918 raft_consensus.cc:3060] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:55.638295 24918 raft_consensus.cc:515] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:55.636868 24669 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 54ac5f2af4044ee5ac6d51afda1bade2: soft_deleted_table [id=c3590c8078c04b8c80ea4d2e16f47c35] alter complete (version 1)
I20260709 12:25:55.641590 24918 leader_election.cc:290] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 3 election: Requested vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:25:55.642729 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb7e15e532104619bfc9dfcc15baceea" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
I20260709 12:25:55.643540 24771 raft_consensus.cc:3055] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:55.643903 24771 raft_consensus.cc:740] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Running, Role: LEADER
I20260709 12:25:55.644778 24771 consensus_queue.cc:260] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:55.644897 24845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb7e15e532104619bfc9dfcc15baceea" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "0b1901a1549f476696340f2fc61af194"
I20260709 12:25:55.645582 24845 raft_consensus.cc:3060] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:55.646023 24771 raft_consensus.cc:3060] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:55.652108 24845 raft_consensus.cc:2468] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 3.
I20260709 12:25:55.652007 24771 raft_consensus.cc:2468] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 3.
I20260709 12:25:55.655202 24887 leader_election.cc:304] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2807a439752a4b0ab39c82cdef37f499, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:55.656319 25087 raft_consensus.cc:2804] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:55.656857 25087 raft_consensus.cc:697] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499 [term 3 LEADER]: Becoming Leader. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Running, Role: LEADER
I20260709 12:25:55.657773 25087 consensus_queue.cc:237] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:55.667064 24669 catalog_manager.cc:5697] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499 reported cstate change: term changed from 2 to 3, leader changed from 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1) to 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3). New cstate: current_term: 3 leader_uuid: "2807a439752a4b0ab39c82cdef37f499" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:55.859807 25168 raft_consensus.cc:993] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4: : Instructing follower 2807a439752a4b0ab39c82cdef37f499 to start an election
I20260709 12:25:55.860321 25195 raft_consensus.cc:1081] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 LEADER]: Signalling peer 2807a439752a4b0ab39c82cdef37f499 to start an election
I20260709 12:25:55.862082 24917 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3f69b65186ed4f6c8fe7c50f6b0133d3"
dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
 from {username='slave'} at 127.0.0.1:57264
I20260709 12:25:55.862591 24917 raft_consensus.cc:493] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:55.862903 24917 raft_consensus.cc:3060] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:55.869187 24917 raft_consensus.cc:515] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } }
I20260709 12:25:55.871173 24917 leader_election.cc:290] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 3 election: Requested vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:25:55.872164 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f69b65186ed4f6c8fe7c50f6b0133d3" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
I20260709 12:25:55.872306 24845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f69b65186ed4f6c8fe7c50f6b0133d3" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0b1901a1549f476696340f2fc61af194"
I20260709 12:25:55.872958 24771 raft_consensus.cc:3055] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:55.873066 24845 raft_consensus.cc:3060] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:55.873333 24771 raft_consensus.cc:740] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Running, Role: LEADER
I20260709 12:25:55.874274 24771 consensus_queue.cc:260] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } }
I20260709 12:25:55.875521 24771 raft_consensus.cc:3060] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:55.880808 24845 raft_consensus.cc:2468] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 0b1901a1549f476696340f2fc61af194 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 3.
I20260709 12:25:55.880968 24771 raft_consensus.cc:2468] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 3.
I20260709 12:25:55.882078 24887 leader_election.cc:304] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2807a439752a4b0ab39c82cdef37f499, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:55.882822 25087 raft_consensus.cc:2804] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:55.883235 25087 raft_consensus.cc:697] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 3 LEADER]: Becoming Leader. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Running, Role: LEADER
I20260709 12:25:55.884169 25087 consensus_queue.cc:237] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } }
I20260709 12:25:55.894496 24669 catalog_manager.cc:5697] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 reported cstate change: term changed from 2 to 3, leader changed from 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1) to 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3). New cstate: current_term: 3 leader_uuid: "2807a439752a4b0ab39c82cdef37f499" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:55.922183 25185 raft_consensus.cc:993] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194: : Instructing follower 2807a439752a4b0ab39c82cdef37f499 to start an election
I20260709 12:25:55.922860 25149 raft_consensus.cc:1081] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194 [term 1 LEADER]: Signalling peer 2807a439752a4b0ab39c82cdef37f499 to start an election
I20260709 12:25:55.925132 24917 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "959e5d187b7445d4ad05d5ddd448d101"
dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
 from {username='slave'} at 127.0.0.1:57250
I20260709 12:25:55.925769 24917 raft_consensus.cc:493] T 959e5d187b7445d4ad05d5ddd448d101 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:55.926193 24917 raft_consensus.cc:3060] T 959e5d187b7445d4ad05d5ddd448d101 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:55.931614 24917 raft_consensus.cc:515] T 959e5d187b7445d4ad05d5ddd448d101 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:55.933399 24917 leader_election.cc:290] T 959e5d187b7445d4ad05d5ddd448d101 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 2 election: Requested vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:25:55.934315 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "959e5d187b7445d4ad05d5ddd448d101" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
I20260709 12:25:55.934639 24845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "959e5d187b7445d4ad05d5ddd448d101" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "0b1901a1549f476696340f2fc61af194"
I20260709 12:25:55.934988 24771 raft_consensus.cc:3060] T 959e5d187b7445d4ad05d5ddd448d101 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:55.935225 24845 raft_consensus.cc:3055] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:55.935570 24845 raft_consensus.cc:740] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0b1901a1549f476696340f2fc61af194, State: Running, Role: LEADER
I20260709 12:25:55.936476 24845 consensus_queue.cc:260] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:55.938024 24845 raft_consensus.cc:3060] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:55.943337 24771 raft_consensus.cc:2468] T 959e5d187b7445d4ad05d5ddd448d101 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 2.
I20260709 12:25:55.944885 24887 leader_election.cc:304] T 959e5d187b7445d4ad05d5ddd448d101 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2807a439752a4b0ab39c82cdef37f499, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:25:55.945899 25087 raft_consensus.cc:2804] T 959e5d187b7445d4ad05d5ddd448d101 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:55.946370 25087 raft_consensus.cc:697] T 959e5d187b7445d4ad05d5ddd448d101 P 2807a439752a4b0ab39c82cdef37f499 [term 2 LEADER]: Becoming Leader. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Running, Role: LEADER
I20260709 12:25:55.946909 24845 raft_consensus.cc:2468] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 2.
I20260709 12:25:55.947371 25087 consensus_queue.cc:237] T 959e5d187b7445d4ad05d5ddd448d101 P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:55.955623 24669 catalog_manager.cc:5697] T 959e5d187b7445d4ad05d5ddd448d101 P 2807a439752a4b0ab39c82cdef37f499 reported cstate change: term changed from 1 to 2, leader changed from 0b1901a1549f476696340f2fc61af194 (127.5.58.2) to 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3). New cstate: current_term: 2 leader_uuid: "2807a439752a4b0ab39c82cdef37f499" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:56.165051 24845 raft_consensus.cc:1275] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [term 3 FOLLOWER]: Refusing update from remote peer 2807a439752a4b0ab39c82cdef37f499: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:25:56.166527 25087 consensus_queue.cc:1048] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:56.175385 24771 raft_consensus.cc:1275] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4 [term 3 FOLLOWER]: Refusing update from remote peer 2807a439752a4b0ab39c82cdef37f499: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:25:56.176733 25087 consensus_queue.cc:1048] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:56.267220 24845 raft_consensus.cc:1275] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 0b1901a1549f476696340f2fc61af194 [term 3 FOLLOWER]: Refusing update from remote peer 2807a439752a4b0ab39c82cdef37f499: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:56.268712 25087 consensus_queue.cc:1048] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:56.279654 24771 raft_consensus.cc:1275] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 3 FOLLOWER]: Refusing update from remote peer 2807a439752a4b0ab39c82cdef37f499: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:56.281890 25198 consensus_queue.cc:1048] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:56.420503 24845 raft_consensus.cc:1275] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Refusing update from remote peer 2807a439752a4b0ab39c82cdef37f499: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:25:56.422333 25198 consensus_queue.cc:1048] T 959e5d187b7445d4ad05d5ddd448d101 P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:56.431669 24771 raft_consensus.cc:1275] T 959e5d187b7445d4ad05d5ddd448d101 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Refusing update from remote peer 2807a439752a4b0ab39c82cdef37f499: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:25:56.433681 25236 consensus_queue.cc:1048] T 959e5d187b7445d4ad05d5ddd448d101 P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:56.616981  5352 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:25:56.621898  5352 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:25:56.622351  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:56.629583  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:25:56.630218  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:57.631369  5352 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:25:57.638298  5352 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:25:57.638775  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:57.643537  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:25:57.643916  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:58.645215  5352 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:25:58.650671  5352 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:25:58.651224  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:58.658480  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:25:58.659062  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:59.741093  5352 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:25:59.796128  5352 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:25:59.834149  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:59.842301  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
W20260709 12:25:59.897109 24793 debug-util.cc:398] Leaking SignalData structure 0x7b08001b89a0 after lost signal to thread 5355
I20260709 12:25:59.898478 25087 raft_consensus.cc:493] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0b1901a1549f476696340f2fc61af194)
I20260709 12:26:00.034739  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:00.035190 25087 raft_consensus.cc:515] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:25:59.906257 25242 raft_consensus.cc:493] T 959e5d187b7445d4ad05d5ddd448d101 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2807a439752a4b0ab39c82cdef37f499)
I20260709 12:26:00.038293 25242 raft_consensus.cc:515] T 959e5d187b7445d4ad05d5ddd448d101 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:26:00.044879 25087 leader_election.cc:290] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:25:59.990008 25243 raft_consensus.cc:493] T a55055527e0941eeb03b4f81ea2a6777 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0b1901a1549f476696340f2fc61af194)
I20260709 12:26:00.045907 25243 raft_consensus.cc:515] T a55055527e0941eeb03b4f81ea2a6777 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:26:00.051782 25242 leader_election.cc:290] T 959e5d187b7445d4ad05d5ddd448d101 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:26:00.053956 25243 leader_election.cc:290] T a55055527e0941eeb03b4f81ea2a6777 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:26:00.024178 25241 raft_consensus.cc:493] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2807a439752a4b0ab39c82cdef37f499)
I20260709 12:25:59.929153 25244 raft_consensus.cc:493] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6f825b5c64844f7fb020b60f13e6fdb4)
I20260709 12:26:00.060562 25244 raft_consensus.cc:515] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:26:00.062613 25240 raft_consensus.cc:493] T 3f803a6e48f74d26b826aa57f9233ee9 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6f825b5c64844f7fb020b60f13e6fdb4)
I20260709 12:26:00.063253 25244 leader_election.cc:290] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:26:00.063824 25244 raft_consensus.cc:493] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6f825b5c64844f7fb020b60f13e6fdb4)
I20260709 12:26:00.064658 25241 raft_consensus.cc:515] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } }
I20260709 12:26:00.064280 25244 raft_consensus.cc:515] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:26:00.067837 25244 leader_election.cc:290] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:26:00.068509 25246 raft_consensus.cc:493] T 357a8c2b04ca46349006b7487ed1409e P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2807a439752a4b0ab39c82cdef37f499)
I20260709 12:26:00.069108 25246 raft_consensus.cc:515] T 357a8c2b04ca46349006b7487ed1409e P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:26:00.063313 25240 raft_consensus.cc:515] T 3f803a6e48f74d26b826aa57f9233ee9 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:26:00.071417 25246 leader_election.cc:290] T 357a8c2b04ca46349006b7487ed1409e P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:26:00.072053 25246 raft_consensus.cc:493] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6f825b5c64844f7fb020b60f13e6fdb4)
I20260709 12:26:00.068053 25237 raft_consensus.cc:493] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2807a439752a4b0ab39c82cdef37f499)
I20260709 12:26:00.075459 25237 raft_consensus.cc:515] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
W20260709 12:26:00.076476 24736 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.023s	user 0.022s	sys 0.013s
W20260709 12:26:00.077060 24736 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.023s	user 0.022s	sys 0.013s
I20260709 12:26:00.072283 25240 leader_election.cc:290] T 3f803a6e48f74d26b826aa57f9233ee9 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:26:00.077528 25237 leader_election.cc:290] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:26:00.078658 25246 raft_consensus.cc:515] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
W20260709 12:26:00.079809 24793 debug-util.cc:398] Leaking SignalData structure 0x7b0800113520 after lost signal to thread 24704
I20260709 12:26:00.074849 25241 leader_election.cc:290] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
W20260709 12:26:00.096729 24887 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.1:44285, user_credentials={real_user=slave}} blocked reactor thread for 167103us
I20260709 12:26:00.100149 24770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "357a8c2b04ca46349006b7487ed1409e" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" is_pre_election: true
I20260709 12:26:00.102455 24770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df8831adbf14c3594dea358a98bfff3" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" is_pre_election: true
I20260709 12:26:00.102958 25246 leader_election.cc:290] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:26:00.103832 24845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "959e5d187b7445d4ad05d5ddd448d101" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194" is_pre_election: true
I20260709 12:26:00.108421 24915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "959e5d187b7445d4ad05d5ddd448d101" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499" is_pre_election: true
I20260709 12:26:00.111888 24920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f69b65186ed4f6c8fe7c50f6b0133d3" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499" is_pre_election: true
W20260709 12:26:00.112722 24793 debug-util.cc:398] Leaking SignalData structure 0x7b080009b5a0 after lost signal to thread 24796
I20260709 12:26:00.112689 24770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a81d7f4ccb459ab2e722f09c10d9a1" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" is_pre_election: true
W20260709 12:26:00.114195 24793 debug-util.cc:398] Leaking SignalData structure 0x7b08000d14a0 after lost signal to thread 24871
I20260709 12:26:00.114431 24768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1d7d9ea11c24a428a3ca7a20b11e81c" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" is_pre_election: true
I20260709 12:26:00.111178 24914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55055527e0941eeb03b4f81ea2a6777" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499" is_pre_election: true
W20260709 12:26:00.115423 24793 debug-util.cc:398] Leaking SignalData structure 0x7b080016a8c0 after lost signal to thread 24945
I20260709 12:26:00.116981 24770 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f803a6e48f74d26b826aa57f9233ee9" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" is_pre_election: true
I20260709 12:26:00.119958 24845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55055527e0941eeb03b4f81ea2a6777" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194" is_pre_election: true
I20260709 12:26:00.122673 24845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f69b65186ed4f6c8fe7c50f6b0133d3" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194" is_pre_election: true
I20260709 12:26:00.123356 24845 raft_consensus.cc:2468] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 0b1901a1549f476696340f2fc61af194 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f825b5c64844f7fb020b60f13e6fdb4 in term 3.
I20260709 12:26:00.124418 25250 raft_consensus.cc:493] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 0b1901a1549f476696340f2fc61af194 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2807a439752a4b0ab39c82cdef37f499)
I20260709 12:26:00.125106 25250 raft_consensus.cc:515] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 0b1901a1549f476696340f2fc61af194 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } }
I20260709 12:26:00.127565 24738 leader_election.cc:304] T 959e5d187b7445d4ad05d5ddd448d101 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 0b1901a1549f476696340f2fc61af194, 2807a439752a4b0ab39c82cdef37f499
I20260709 12:26:00.129345 24844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df8831adbf14c3594dea358a98bfff3" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194" is_pre_election: true
I20260709 12:26:00.129603 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36fd425a2ae24bba8cc5934237f465b6" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" is_pre_election: true
I20260709 12:26:00.130312 24738 leader_election.cc:304] T a55055527e0941eeb03b4f81ea2a6777 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 0b1901a1549f476696340f2fc61af194, 2807a439752a4b0ab39c82cdef37f499
I20260709 12:26:00.131189 25195 raft_consensus.cc:2749] T 959e5d187b7445d4ad05d5ddd448d101 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:26:00.131853 25242 raft_consensus.cc:2749] T a55055527e0941eeb03b4f81ea2a6777 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:26:00.132055 24738 leader_election.cc:304] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 2807a439752a4b0ab39c82cdef37f499
I20260709 12:26:00.132907 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a81d7f4ccb459ab2e722f09c10d9a1" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" is_pre_election: true
I20260709 12:26:00.133605 25243 raft_consensus.cc:2804] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:26:00.133867 24846 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1d7d9ea11c24a428a3ca7a20b11e81c" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194" is_pre_election: true
I20260709 12:26:00.134078 25243 raft_consensus.cc:493] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 2807a439752a4b0ab39c82cdef37f499)
I20260709 12:26:00.134558 25243 raft_consensus.cc:3060] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:00.138484 24845 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f803a6e48f74d26b826aa57f9233ee9" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194" is_pre_election: true
I20260709 12:26:00.131779 24844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a81d7f4ccb459ab2e722f09c10d9a1" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194" is_pre_election: true
I20260709 12:26:00.152470 25250 leader_election.cc:290] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247)
I20260709 12:26:00.149264 25243 raft_consensus.cc:515] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } }
I20260709 12:26:00.154953 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f69b65186ed4f6c8fe7c50f6b0133d3" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" is_pre_election: true
I20260709 12:26:00.156399 24887 leader_election.cc:304] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2807a439752a4b0ab39c82cdef37f499; no voters: 0b1901a1549f476696340f2fc61af194, 6f825b5c64844f7fb020b60f13e6fdb4
I20260709 12:26:00.156937 25243 leader_election.cc:290] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 4 election: Requested vote from peers 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:26:00.156771 24843 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f69b65186ed4f6c8fe7c50f6b0133d3" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194"
I20260709 12:26:00.157392 25249 raft_consensus.cc:2749] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:26:00.156744 24914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "357a8c2b04ca46349006b7487ed1409e" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499" is_pre_election: true
I20260709 12:26:00.159816 24887 leader_election.cc:304] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2807a439752a4b0ab39c82cdef37f499; no voters: 0b1901a1549f476696340f2fc61af194, 6f825b5c64844f7fb020b60f13e6fdb4
I20260709 12:26:00.159956 24920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36fd425a2ae24bba8cc5934237f465b6" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499" is_pre_election: true
I20260709 12:26:00.160708 25249 raft_consensus.cc:2749] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:26:00.162555 24914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f69b65186ed4f6c8fe7c50f6b0133d3" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
I20260709 12:26:00.163251 24887 leader_election.cc:304] T 3f803a6e48f74d26b826aa57f9233ee9 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2807a439752a4b0ab39c82cdef37f499; no voters: 0b1901a1549f476696340f2fc61af194, 6f825b5c64844f7fb020b60f13e6fdb4
I20260709 12:26:00.164139 25248 raft_consensus.cc:2749] T 3f803a6e48f74d26b826aa57f9233ee9 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:26:00.165369 24887 leader_election.cc:304] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2807a439752a4b0ab39c82cdef37f499; no voters: 0b1901a1549f476696340f2fc61af194, 6f825b5c64844f7fb020b60f13e6fdb4
I20260709 12:26:00.165902 24735 leader_election.cc:304] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 0b1901a1549f476696340f2fc61af194, 2807a439752a4b0ab39c82cdef37f499
I20260709 12:26:00.166141 25248 raft_consensus.cc:2749] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:26:00.171334 24810 leader_election.cc:304] T 357a8c2b04ca46349006b7487ed1409e P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0b1901a1549f476696340f2fc61af194; no voters: 2807a439752a4b0ab39c82cdef37f499, 6f825b5c64844f7fb020b60f13e6fdb4
I20260709 12:26:00.167152 24920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a81d7f4ccb459ab2e722f09c10d9a1" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499" is_pre_election: true
I20260709 12:26:00.168999 24918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f69b65186ed4f6c8fe7c50f6b0133d3" candidate_uuid: "0b1901a1549f476696340f2fc61af194" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499" is_pre_election: true
I20260709 12:26:00.172678 24810 leader_election.cc:304] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0b1901a1549f476696340f2fc61af194; no voters: 2807a439752a4b0ab39c82cdef37f499, 6f825b5c64844f7fb020b60f13e6fdb4
I20260709 12:26:00.173353 25243 raft_consensus.cc:2749] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:26:00.173487 25251 raft_consensus.cc:2749] T 357a8c2b04ca46349006b7487ed1409e P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:26:00.174364 25237 raft_consensus.cc:2749] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:00.175626 24810 leader_election.cc:304] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0b1901a1549f476696340f2fc61af194; no voters: 2807a439752a4b0ab39c82cdef37f499, 6f825b5c64844f7fb020b60f13e6fdb4
I20260709 12:26:00.176625 25250 raft_consensus.cc:2749] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:26:00.176888 24810 leader_election.cc:304] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 0b1901a1549f476696340f2fc61af194 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0b1901a1549f476696340f2fc61af194; no voters: 2807a439752a4b0ab39c82cdef37f499, 6f825b5c64844f7fb020b60f13e6fdb4
I20260709 12:26:00.177973 25250 raft_consensus.cc:2749] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 0b1901a1549f476696340f2fc61af194 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:26:00.573455 24771 raft_consensus.cc:1240] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 4 FOLLOWER]: Rejecting Update request from peer 2807a439752a4b0ab39c82cdef37f499 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:00.575428 25248 consensus_queue.cc:1059] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:00.576709 25087 raft_consensus.cc:3055] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:00.577090 25087 raft_consensus.cc:740] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Running, Role: LEADER
I20260709 12:26:00.577845 25087 consensus_queue.cc:260] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } }
I20260709 12:26:00.578940 25087 raft_consensus.cc:3060] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:01.037051  5352 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:26:01.043572  5352 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:26:01.044095  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:01.051079  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:26:01.051548  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:01.806782 25255 raft_consensus.cc:493] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.807286 25255 raft_consensus.cc:515] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } }
I20260709 12:26:01.809263 25255 leader_election.cc:290] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:26:01.810284 24918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f69b65186ed4f6c8fe7c50f6b0133d3" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499" is_pre_election: true
I20260709 12:26:01.812016 24844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f69b65186ed4f6c8fe7c50f6b0133d3" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194" is_pre_election: true
I20260709 12:26:01.812147 24918 raft_consensus.cc:2468] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f825b5c64844f7fb020b60f13e6fdb4 in term 4.
I20260709 12:26:01.812803 24844 raft_consensus.cc:2468] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 0b1901a1549f476696340f2fc61af194 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f825b5c64844f7fb020b60f13e6fdb4 in term 3.
I20260709 12:26:01.814383 24738 leader_election.cc:304] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:26:01.815227 25255 raft_consensus.cc:2804] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:01.815649 25255 raft_consensus.cc:493] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:01.816308 25255 raft_consensus.cc:3060] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:01.823670 25255 raft_consensus.cc:515] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } }
I20260709 12:26:01.825516 25255 leader_election.cc:290] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 5 election: Requested vote from peers 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3:45247), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:26:01.826303 24918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f69b65186ed4f6c8fe7c50f6b0133d3" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
I20260709 12:26:01.826576 24844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f69b65186ed4f6c8fe7c50f6b0133d3" candidate_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0b1901a1549f476696340f2fc61af194"
I20260709 12:26:01.827040 24918 raft_consensus.cc:3060] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:01.827243 24844 raft_consensus.cc:3060] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 0b1901a1549f476696340f2fc61af194 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:26:01.833549 24844 raft_consensus.cc:2468] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 0b1901a1549f476696340f2fc61af194 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f825b5c64844f7fb020b60f13e6fdb4 in term 5.
I20260709 12:26:01.834595 24738 leader_election.cc:304] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1901a1549f476696340f2fc61af194, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:26:01.835258 25255 raft_consensus.cc:2804] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:01.835691 25255 raft_consensus.cc:697] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 5 LEADER]: Becoming Leader. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Running, Role: LEADER
I20260709 12:26:01.835587 24918 raft_consensus.cc:2468] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f825b5c64844f7fb020b60f13e6fdb4 in term 5.
I20260709 12:26:01.836611 25255 consensus_queue.cc:237] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } }
I20260709 12:26:01.852713 24668 catalog_manager.cc:5697] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 reported cstate change: term changed from 3 to 5, leader changed from 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3) to 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1). New cstate: current_term: 5 leader_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:02.052836  5352 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:26:02.058175  5352 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:26:02.058629  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:02.065994 24771 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97a81d7f4ccb459ab2e722f09c10d9a1"
dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
mode: GRACEFUL
new_leader_uuid: "2807a439752a4b0ab39c82cdef37f499"
 from {username='slave'} at 127.0.0.1:56084
I20260709 12:26:02.066509 24771 raft_consensus.cc:606] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 LEADER]: Received request to transfer leadership to TS 2807a439752a4b0ab39c82cdef37f499
I20260709 12:26:02.067727  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:26:02.068233  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:02.268621 24918 raft_consensus.cc:1275] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 5 FOLLOWER]: Refusing update from remote peer 6f825b5c64844f7fb020b60f13e6fdb4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:26:02.270043 25255 consensus_queue.cc:1048] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:26:02.282138 24844 raft_consensus.cc:1275] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 0b1901a1549f476696340f2fc61af194 [term 5 FOLLOWER]: Refusing update from remote peer 6f825b5c64844f7fb020b60f13e6fdb4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:26:02.284102 25255 consensus_queue.cc:1048] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:26:02.388273 25253 raft_consensus.cc:993] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4: : Instructing follower 2807a439752a4b0ab39c82cdef37f499 to start an election
I20260709 12:26:02.389014 25255 raft_consensus.cc:1081] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 LEADER]: Signalling peer 2807a439752a4b0ab39c82cdef37f499 to start an election
I20260709 12:26:02.391001 24918 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "97a81d7f4ccb459ab2e722f09c10d9a1"
dest_uuid: "2807a439752a4b0ab39c82cdef37f499"
 from {username='slave'} at 127.0.0.1:57264
I20260709 12:26:02.391680 24918 raft_consensus.cc:493] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:02.392032 24918 raft_consensus.cc:3060] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:02.398000 24918 raft_consensus.cc:515] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:26:02.401003 24918 leader_election.cc:290] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 3 election: Requested vote from peers 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285), 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549)
I20260709 12:26:02.401803 24771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a81d7f4ccb459ab2e722f09c10d9a1" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6f825b5c64844f7fb020b60f13e6fdb4"
I20260709 12:26:02.402149 24844 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97a81d7f4ccb459ab2e722f09c10d9a1" candidate_uuid: "2807a439752a4b0ab39c82cdef37f499" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0b1901a1549f476696340f2fc61af194"
I20260709 12:26:02.402519 24771 raft_consensus.cc:3055] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:26:02.402884 24771 raft_consensus.cc:740] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6f825b5c64844f7fb020b60f13e6fdb4, State: Running, Role: LEADER
I20260709 12:26:02.403216 24844 raft_consensus.cc:3060] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:02.403663 24771 consensus_queue.cc:260] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:26:02.404690 24771 raft_consensus.cc:3060] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:02.411182 24771 raft_consensus.cc:2468] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 3.
I20260709 12:26:02.412545 24844 raft_consensus.cc:2468] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 0b1901a1549f476696340f2fc61af194 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2807a439752a4b0ab39c82cdef37f499 in term 3.
I20260709 12:26:02.413897 24887 leader_election.cc:304] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2807a439752a4b0ab39c82cdef37f499, 6f825b5c64844f7fb020b60f13e6fdb4; no voters: 
I20260709 12:26:02.415558 25254 raft_consensus.cc:2804] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:26:02.417820 25254 raft_consensus.cc:697] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [term 3 LEADER]: Becoming Leader. State: Replica: 2807a439752a4b0ab39c82cdef37f499, State: Running, Role: LEADER
I20260709 12:26:02.418879 25254 consensus_queue.cc:237] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } }
I20260709 12:26:02.429291 24669 catalog_manager.cc:5697] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 reported cstate change: term changed from 2 to 3, leader changed from 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1) to 2807a439752a4b0ab39c82cdef37f499 (127.5.58.3). New cstate: current_term: 3 leader_uuid: "2807a439752a4b0ab39c82cdef37f499" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2807a439752a4b0ab39c82cdef37f499" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 45247 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:02.800273 24844 raft_consensus.cc:1275] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 0b1901a1549f476696340f2fc61af194 [term 3 FOLLOWER]: Refusing update from remote peer 2807a439752a4b0ab39c82cdef37f499: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:26:02.801872 25254 consensus_queue.cc:1048] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1901a1549f476696340f2fc61af194" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 38549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:26:02.813657 24771 raft_consensus.cc:1275] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 3 FOLLOWER]: Refusing update from remote peer 2807a439752a4b0ab39c82cdef37f499: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:26:02.816538 25267 consensus_queue.cc:1048] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f825b5c64844f7fb020b60f13e6fdb4" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 44285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:26:03.069267  5352 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:26:03.074343  5352 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:26:03.074802  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:03.079593  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:26:03.080005  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:04.081877  5352 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:26:04.089951  5352 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:26:04.090446  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:04.096683  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:26:04.097139  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:05.098577  5352 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:26:05.105134  5352 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:26:05.105664  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:05.110230  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:26:05.110649  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:06.182602  5352 tablet_server.cc:179] TabletServer@127.5.58.1:0 shutting down...
W20260709 12:26:06.192667 24887 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 24 similar messages]
W20260709 12:26:06.197032 24887 consensus_peers.cc:597] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 -> Peer 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285): Couldn't send request to peer 6f825b5c64844f7fb020b60f13e6fdb4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:06.212392 24887 consensus_peers.cc:597] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499 -> Peer 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285): Couldn't send request to peer 6f825b5c64844f7fb020b60f13e6fdb4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:06.237607  5352 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:26:06.238869  5352 tablet_replica.cc:333] T a55055527e0941eeb03b4f81ea2a6777 P 6f825b5c64844f7fb020b60f13e6fdb4: stopping tablet replica
I20260709 12:26:06.240761  5352 raft_consensus.cc:2243] T a55055527e0941eeb03b4f81ea2a6777 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.241819  5352 raft_consensus.cc:2272] T a55055527e0941eeb03b4f81ea2a6777 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.246975  5352 tablet_replica.cc:333] T 959e5d187b7445d4ad05d5ddd448d101 P 6f825b5c64844f7fb020b60f13e6fdb4: stopping tablet replica
I20260709 12:26:06.248045  5352 raft_consensus.cc:2243] T 959e5d187b7445d4ad05d5ddd448d101 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.249128  5352 raft_consensus.cc:2272] T 959e5d187b7445d4ad05d5ddd448d101 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:06.250847 24813 consensus_peers.cc:597] T f51f7a5450c04dc9a93b0efbda38b7ea P 0b1901a1549f476696340f2fc61af194 -> Peer 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285): Couldn't send request to peer 6f825b5c64844f7fb020b60f13e6fdb4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:06.253010  5352 tablet_replica.cc:333] T 804a1b4315914cf9895b192eace7f7d5 P 6f825b5c64844f7fb020b60f13e6fdb4: stopping tablet replica
I20260709 12:26:06.254184  5352 raft_consensus.cc:2243] T 804a1b4315914cf9895b192eace7f7d5 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:26:06.255626  5352 raft_consensus.cc:2272] T 804a1b4315914cf9895b192eace7f7d5 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:06.256374 24813 consensus_peers.cc:597] T 89fd2ebc3fe5461c93eac233d095a6df P 0b1901a1549f476696340f2fc61af194 -> Peer 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285): Couldn't send request to peer 6f825b5c64844f7fb020b60f13e6fdb4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:06.258865  5352 tablet_replica.cc:333] T f51f7a5450c04dc9a93b0efbda38b7ea P 6f825b5c64844f7fb020b60f13e6fdb4: stopping tablet replica
I20260709 12:26:06.259905  5352 raft_consensus.cc:2243] T f51f7a5450c04dc9a93b0efbda38b7ea P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.260690  5352 raft_consensus.cc:2272] T f51f7a5450c04dc9a93b0efbda38b7ea P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.263695  5352 tablet_replica.cc:333] T 8484b63eb29b4275b24ccd0cd68b87ce P 6f825b5c64844f7fb020b60f13e6fdb4: stopping tablet replica
I20260709 12:26:06.264456  5352 raft_consensus.cc:2243] T 8484b63eb29b4275b24ccd0cd68b87ce P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.265004  5352 raft_consensus.cc:2272] T 8484b63eb29b4275b24ccd0cd68b87ce P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.268698  5352 tablet_replica.cc:333] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4: stopping tablet replica
I20260709 12:26:06.269555  5352 raft_consensus.cc:2243] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 5 LEADER]: Raft consensus shutting down.
W20260709 12:26:06.270018 24813 consensus_peers.cc:597] T 8484b63eb29b4275b24ccd0cd68b87ce P 0b1901a1549f476696340f2fc61af194 -> Peer 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285): Couldn't send request to peer 6f825b5c64844f7fb020b60f13e6fdb4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:06.270741  5352 raft_consensus.cc:2272] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.273753  5352 tablet_replica.cc:333] T f379878d1a0c474aab696ddb355df430 P 6f825b5c64844f7fb020b60f13e6fdb4: stopping tablet replica
I20260709 12:26:06.274472  5352 raft_consensus.cc:2243] T f379878d1a0c474aab696ddb355df430 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:26:06.275496  5352 raft_consensus.cc:2272] T f379878d1a0c474aab696ddb355df430 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.278457  5352 tablet_replica.cc:333] T ebfd2dfa84b74747bd5e4c3cf31902bf P 6f825b5c64844f7fb020b60f13e6fdb4: stopping tablet replica
I20260709 12:26:06.279199  5352 raft_consensus.cc:2243] T ebfd2dfa84b74747bd5e4c3cf31902bf P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.279857  5352 raft_consensus.cc:2272] T ebfd2dfa84b74747bd5e4c3cf31902bf P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.282857  5352 tablet_replica.cc:333] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4: stopping tablet replica
I20260709 12:26:06.283521  5352 raft_consensus.cc:2243] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.284097  5352 raft_consensus.cc:2272] T 8f0eec1401e842eeb0541c2dd6621e58 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.286665  5352 tablet_replica.cc:333] T 357a8c2b04ca46349006b7487ed1409e P 6f825b5c64844f7fb020b60f13e6fdb4: stopping tablet replica
I20260709 12:26:06.287364  5352 raft_consensus.cc:2243] T 357a8c2b04ca46349006b7487ed1409e P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.287954  5352 raft_consensus.cc:2272] T 357a8c2b04ca46349006b7487ed1409e P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.290840  5352 tablet_replica.cc:333] T f1d7d9ea11c24a428a3ca7a20b11e81c P 6f825b5c64844f7fb020b60f13e6fdb4: stopping tablet replica
I20260709 12:26:06.291455  5352 raft_consensus.cc:2243] T f1d7d9ea11c24a428a3ca7a20b11e81c P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:26:06.292532  5352 raft_consensus.cc:2272] T f1d7d9ea11c24a428a3ca7a20b11e81c P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.295331  5352 tablet_replica.cc:333] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4: stopping tablet replica
I20260709 12:26:06.295974  5352 raft_consensus.cc:2243] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.296511  5352 raft_consensus.cc:2272] T bb7e15e532104619bfc9dfcc15baceea P 6f825b5c64844f7fb020b60f13e6fdb4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.299517  5352 tablet_replica.cc:333] T 1df8831adbf14c3594dea358a98bfff3 P 6f825b5c64844f7fb020b60f13e6fdb4: stopping tablet replica
I20260709 12:26:06.300509  5352 raft_consensus.cc:2243] T 1df8831adbf14c3594dea358a98bfff3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.301425  5352 raft_consensus.cc:2272] T 1df8831adbf14c3594dea358a98bfff3 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.304898  5352 tablet_replica.cc:333] T 3f803a6e48f74d26b826aa57f9233ee9 P 6f825b5c64844f7fb020b60f13e6fdb4: stopping tablet replica
I20260709 12:26:06.306021  5352 raft_consensus.cc:2243] T 3f803a6e48f74d26b826aa57f9233ee9 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:26:06.307235  5352 raft_consensus.cc:2272] T 3f803a6e48f74d26b826aa57f9233ee9 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:06.310271 24887 consensus_peers.cc:597] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 -> Peer 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285): Couldn't send request to peer 6f825b5c64844f7fb020b60f13e6fdb4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:06.311256  5352 tablet_replica.cc:333] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4: stopping tablet replica
I20260709 12:26:06.312033  5352 raft_consensus.cc:2243] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.312728  5352 raft_consensus.cc:2272] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.316066  5352 tablet_replica.cc:333] T 65f6dd3a57e04c6a880d0fecbd65401f P 6f825b5c64844f7fb020b60f13e6fdb4: stopping tablet replica
I20260709 12:26:06.316661  5352 raft_consensus.cc:2243] T 65f6dd3a57e04c6a880d0fecbd65401f P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:26:06.317525  5352 raft_consensus.cc:2272] T 65f6dd3a57e04c6a880d0fecbd65401f P 6f825b5c64844f7fb020b60f13e6fdb4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.320155  5352 tablet_replica.cc:333] T 89fd2ebc3fe5461c93eac233d095a6df P 6f825b5c64844f7fb020b60f13e6fdb4: stopping tablet replica
I20260709 12:26:06.320750  5352 raft_consensus.cc:2243] T 89fd2ebc3fe5461c93eac233d095a6df P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.321421  5352 raft_consensus.cc:2272] T 89fd2ebc3fe5461c93eac233d095a6df P 6f825b5c64844f7fb020b60f13e6fdb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.324311  5352 tablet_replica.cc:333] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4: stopping tablet replica
I20260709 12:26:06.324944  5352 raft_consensus.cc:2243] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.325507  5352 raft_consensus.cc:2272] T 36fd425a2ae24bba8cc5934237f465b6 P 6f825b5c64844f7fb020b60f13e6fdb4 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:06.340194 24887 consensus_peers.cc:597] T 959e5d187b7445d4ad05d5ddd448d101 P 2807a439752a4b0ab39c82cdef37f499 -> Peer 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285): Couldn't send request to peer 6f825b5c64844f7fb020b60f13e6fdb4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:06.368692 24813 consensus_peers.cc:597] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194 -> Peer 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285): Couldn't send request to peer 6f825b5c64844f7fb020b60f13e6fdb4. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:44285: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:06.385778 24813 consensus_peers.cc:597] T 8f0eec1401e842eeb0541c2dd6621e58 P 0b1901a1549f476696340f2fc61af194 -> Peer 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285): Couldn't send request to peer 6f825b5c64844f7fb020b60f13e6fdb4. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:44285: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:06.389086  5352 tablet_server.cc:196] TabletServer@127.5.58.1:0 shutdown complete.
I20260709 12:26:06.436175  5352 tablet_server.cc:179] TabletServer@127.5.58.2:0 shutting down...
W20260709 12:26:06.453994 24887 consensus_peers.cc:597] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 -> Peer 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549): Couldn't send request to peer 0b1901a1549f476696340f2fc61af194. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:06.461054 24887 consensus_peers.cc:597] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 -> Peer 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549): Couldn't send request to peer 0b1901a1549f476696340f2fc61af194. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:06.471464 24887 consensus_peers.cc:597] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499 -> Peer 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285): Couldn't send request to peer 6f825b5c64844f7fb020b60f13e6fdb4. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:44285: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:06.482362  5352 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:26:06.483319  5352 tablet_replica.cc:333] T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194: stopping tablet replica
I20260709 12:26:06.484616  5352 raft_consensus.cc:2243] T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:26:06.485811  5352 raft_consensus.cc:2272] T a55055527e0941eeb03b4f81ea2a6777 P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.489351  5352 tablet_replica.cc:333] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194: stopping tablet replica
I20260709 12:26:06.490124  5352 raft_consensus.cc:2243] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.490658  5352 raft_consensus.cc:2272] T 959e5d187b7445d4ad05d5ddd448d101 P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.493046  5352 tablet_replica.cc:333] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194: stopping tablet replica
I20260709 12:26:06.493729  5352 raft_consensus.cc:2243] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.494300  5352 raft_consensus.cc:2272] T 804a1b4315914cf9895b192eace7f7d5 P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.496666  5352 tablet_replica.cc:333] T f51f7a5450c04dc9a93b0efbda38b7ea P 0b1901a1549f476696340f2fc61af194: stopping tablet replica
I20260709 12:26:06.497311  5352 raft_consensus.cc:2243] T f51f7a5450c04dc9a93b0efbda38b7ea P 0b1901a1549f476696340f2fc61af194 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:26:06.498162  5352 raft_consensus.cc:2272] T f51f7a5450c04dc9a93b0efbda38b7ea P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.501124  5352 tablet_replica.cc:333] T 8484b63eb29b4275b24ccd0cd68b87ce P 0b1901a1549f476696340f2fc61af194: stopping tablet replica
I20260709 12:26:06.501721  5352 raft_consensus.cc:2243] T 8484b63eb29b4275b24ccd0cd68b87ce P 0b1901a1549f476696340f2fc61af194 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:26:06.502626  5352 raft_consensus.cc:2272] T 8484b63eb29b4275b24ccd0cd68b87ce P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.505155  5352 tablet_replica.cc:333] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 0b1901a1549f476696340f2fc61af194: stopping tablet replica
I20260709 12:26:06.505784  5352 raft_consensus.cc:2243] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 0b1901a1549f476696340f2fc61af194 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.506412  5352 raft_consensus.cc:2272] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 0b1901a1549f476696340f2fc61af194 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.509680  5352 tablet_replica.cc:333] T f379878d1a0c474aab696ddb355df430 P 0b1901a1549f476696340f2fc61af194: stopping tablet replica
I20260709 12:26:06.510541  5352 raft_consensus.cc:2243] T f379878d1a0c474aab696ddb355df430 P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.511163  5352 raft_consensus.cc:2272] T f379878d1a0c474aab696ddb355df430 P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:06.512066 24887 consensus_peers.cc:597] T 959e5d187b7445d4ad05d5ddd448d101 P 2807a439752a4b0ab39c82cdef37f499 -> Peer 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549): Couldn't send request to peer 0b1901a1549f476696340f2fc61af194. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:06.514197  5352 tablet_replica.cc:333] T 357a8c2b04ca46349006b7487ed1409e P 0b1901a1549f476696340f2fc61af194: stopping tablet replica
I20260709 12:26:06.515034  5352 raft_consensus.cc:2243] T 357a8c2b04ca46349006b7487ed1409e P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.515590  5352 raft_consensus.cc:2272] T 357a8c2b04ca46349006b7487ed1409e P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.518054  5352 tablet_replica.cc:333] T ebfd2dfa84b74747bd5e4c3cf31902bf P 0b1901a1549f476696340f2fc61af194: stopping tablet replica
I20260709 12:26:06.518657  5352 raft_consensus.cc:2243] T ebfd2dfa84b74747bd5e4c3cf31902bf P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.519167  5352 raft_consensus.cc:2272] T ebfd2dfa84b74747bd5e4c3cf31902bf P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.521765  5352 tablet_replica.cc:333] T 8f0eec1401e842eeb0541c2dd6621e58 P 0b1901a1549f476696340f2fc61af194: stopping tablet replica
I20260709 12:26:06.522346  5352 raft_consensus.cc:2243] T 8f0eec1401e842eeb0541c2dd6621e58 P 0b1901a1549f476696340f2fc61af194 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:26:06.523265  5352 raft_consensus.cc:2272] T 8f0eec1401e842eeb0541c2dd6621e58 P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.525877  5352 tablet_replica.cc:333] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 0b1901a1549f476696340f2fc61af194: stopping tablet replica
I20260709 12:26:06.526455  5352 raft_consensus.cc:2243] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 0b1901a1549f476696340f2fc61af194 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.526999  5352 raft_consensus.cc:2272] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 0b1901a1549f476696340f2fc61af194 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.529810  5352 tablet_replica.cc:333] T 3f803a6e48f74d26b826aa57f9233ee9 P 0b1901a1549f476696340f2fc61af194: stopping tablet replica
I20260709 12:26:06.530512  5352 raft_consensus.cc:2243] T 3f803a6e48f74d26b826aa57f9233ee9 P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.531205  5352 raft_consensus.cc:2272] T 3f803a6e48f74d26b826aa57f9233ee9 P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.533867  5352 tablet_replica.cc:333] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194: stopping tablet replica
I20260709 12:26:06.534525  5352 raft_consensus.cc:2243] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.535064  5352 raft_consensus.cc:2272] T 36fd425a2ae24bba8cc5934237f465b6 P 0b1901a1549f476696340f2fc61af194 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.537629  5352 tablet_replica.cc:333] T 65f6dd3a57e04c6a880d0fecbd65401f P 0b1901a1549f476696340f2fc61af194: stopping tablet replica
I20260709 12:26:06.538246  5352 raft_consensus.cc:2243] T 65f6dd3a57e04c6a880d0fecbd65401f P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.538818  5352 raft_consensus.cc:2272] T 65f6dd3a57e04c6a880d0fecbd65401f P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.541581  5352 tablet_replica.cc:333] T 89fd2ebc3fe5461c93eac233d095a6df P 0b1901a1549f476696340f2fc61af194: stopping tablet replica
I20260709 12:26:06.542176  5352 raft_consensus.cc:2243] T 89fd2ebc3fe5461c93eac233d095a6df P 0b1901a1549f476696340f2fc61af194 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:26:06.543066  5352 raft_consensus.cc:2272] T 89fd2ebc3fe5461c93eac233d095a6df P 0b1901a1549f476696340f2fc61af194 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.545404  5352 tablet_replica.cc:333] T f1d7d9ea11c24a428a3ca7a20b11e81c P 0b1901a1549f476696340f2fc61af194: stopping tablet replica
I20260709 12:26:06.545979  5352 raft_consensus.cc:2243] T f1d7d9ea11c24a428a3ca7a20b11e81c P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.546494  5352 raft_consensus.cc:2272] T f1d7d9ea11c24a428a3ca7a20b11e81c P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.549043  5352 tablet_replica.cc:333] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194: stopping tablet replica
I20260709 12:26:06.549827  5352 raft_consensus.cc:2243] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.550647  5352 raft_consensus.cc:2272] T bb7e15e532104619bfc9dfcc15baceea P 0b1901a1549f476696340f2fc61af194 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:06.552597 24887 consensus_peers.cc:597] T 357a8c2b04ca46349006b7487ed1409e P 2807a439752a4b0ab39c82cdef37f499 -> Peer 6f825b5c64844f7fb020b60f13e6fdb4 (127.5.58.1:44285): Couldn't send request to peer 6f825b5c64844f7fb020b60f13e6fdb4. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:44285: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:06.554139  5352 tablet_replica.cc:333] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194: stopping tablet replica
W20260709 12:26:06.554180 24887 consensus_peers.cc:597] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499 -> Peer 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549): Couldn't send request to peer 0b1901a1549f476696340f2fc61af194. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:06.554886  5352 raft_consensus.cc:2243] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:26:06.555866  5352 raft_consensus.cc:2272] T 1df8831adbf14c3594dea358a98bfff3 P 0b1901a1549f476696340f2fc61af194 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.606292  5352 tablet_server.cc:196] TabletServer@127.5.58.2:0 shutdown complete.
I20260709 12:26:06.652158  5352 tablet_server.cc:179] TabletServer@127.5.58.3:0 shutting down...
I20260709 12:26:06.705144  5352 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:26:06.706365  5352 tablet_replica.cc:333] T a55055527e0941eeb03b4f81ea2a6777 P 2807a439752a4b0ab39c82cdef37f499: stopping tablet replica
W20260709 12:26:06.707129 24887 consensus_peers.cc:597] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499 -> Peer 0b1901a1549f476696340f2fc61af194 (127.5.58.2:38549): Couldn't send request to peer 0b1901a1549f476696340f2fc61af194. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:38549: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:06.708313  5352 raft_consensus.cc:2243] T a55055527e0941eeb03b4f81ea2a6777 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.709195  5352 raft_consensus.cc:2272] T a55055527e0941eeb03b4f81ea2a6777 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.712558  5352 tablet_replica.cc:333] T 959e5d187b7445d4ad05d5ddd448d101 P 2807a439752a4b0ab39c82cdef37f499: stopping tablet replica
I20260709 12:26:06.713352  5352 raft_consensus.cc:2243] T 959e5d187b7445d4ad05d5ddd448d101 P 2807a439752a4b0ab39c82cdef37f499 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:26:06.714510  5352 raft_consensus.cc:2272] T 959e5d187b7445d4ad05d5ddd448d101 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.717471  5352 tablet_replica.cc:333] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499: stopping tablet replica
I20260709 12:26:06.718006  5352 raft_consensus.cc:2243] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.718490  5352 raft_consensus.cc:2272] T 65f6dd3a57e04c6a880d0fecbd65401f P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.720592  5352 tablet_replica.cc:333] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499: stopping tablet replica
I20260709 12:26:06.721134  5352 raft_consensus.cc:2243] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.721629  5352 raft_consensus.cc:2272] T f51f7a5450c04dc9a93b0efbda38b7ea P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.723958  5352 tablet_replica.cc:333] T 804a1b4315914cf9895b192eace7f7d5 P 2807a439752a4b0ab39c82cdef37f499: stopping tablet replica
I20260709 12:26:06.724676  5352 raft_consensus.cc:2243] T 804a1b4315914cf9895b192eace7f7d5 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.725208  5352 raft_consensus.cc:2272] T 804a1b4315914cf9895b192eace7f7d5 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.727425  5352 tablet_replica.cc:333] T 8484b63eb29b4275b24ccd0cd68b87ce P 2807a439752a4b0ab39c82cdef37f499: stopping tablet replica
I20260709 12:26:06.728062  5352 raft_consensus.cc:2243] T 8484b63eb29b4275b24ccd0cd68b87ce P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.728906  5352 raft_consensus.cc:2272] T 8484b63eb29b4275b24ccd0cd68b87ce P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.731748  5352 tablet_replica.cc:333] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499: stopping tablet replica
I20260709 12:26:06.732522  5352 raft_consensus.cc:2243] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.733060  5352 raft_consensus.cc:2272] T 3f69b65186ed4f6c8fe7c50f6b0133d3 P 2807a439752a4b0ab39c82cdef37f499 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.735409  5352 tablet_replica.cc:333] T f379878d1a0c474aab696ddb355df430 P 2807a439752a4b0ab39c82cdef37f499: stopping tablet replica
I20260709 12:26:06.735952  5352 raft_consensus.cc:2243] T f379878d1a0c474aab696ddb355df430 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.736506  5352 raft_consensus.cc:2272] T f379878d1a0c474aab696ddb355df430 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.739132  5352 tablet_replica.cc:333] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499: stopping tablet replica
I20260709 12:26:06.739696  5352 raft_consensus.cc:2243] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:26:06.740878  5352 raft_consensus.cc:2272] T ebfd2dfa84b74747bd5e4c3cf31902bf P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.743587  5352 tablet_replica.cc:333] T 8f0eec1401e842eeb0541c2dd6621e58 P 2807a439752a4b0ab39c82cdef37f499: stopping tablet replica
I20260709 12:26:06.744171  5352 raft_consensus.cc:2243] T 8f0eec1401e842eeb0541c2dd6621e58 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.744700  5352 raft_consensus.cc:2272] T 8f0eec1401e842eeb0541c2dd6621e58 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.747252  5352 tablet_replica.cc:333] T 357a8c2b04ca46349006b7487ed1409e P 2807a439752a4b0ab39c82cdef37f499: stopping tablet replica
I20260709 12:26:06.747862  5352 raft_consensus.cc:2243] T 357a8c2b04ca46349006b7487ed1409e P 2807a439752a4b0ab39c82cdef37f499 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:26:06.748788  5352 raft_consensus.cc:2272] T 357a8c2b04ca46349006b7487ed1409e P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.751533  5352 tablet_replica.cc:333] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499: stopping tablet replica
I20260709 12:26:06.752161  5352 raft_consensus.cc:2243] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:26:06.753156  5352 raft_consensus.cc:2272] T 36fd425a2ae24bba8cc5934237f465b6 P 2807a439752a4b0ab39c82cdef37f499 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.755690  5352 tablet_replica.cc:333] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499: stopping tablet replica
I20260709 12:26:06.756321  5352 raft_consensus.cc:2243] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:26:06.757405  5352 raft_consensus.cc:2272] T 97a81d7f4ccb459ab2e722f09c10d9a1 P 2807a439752a4b0ab39c82cdef37f499 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.759743  5352 tablet_replica.cc:333] T 3f803a6e48f74d26b826aa57f9233ee9 P 2807a439752a4b0ab39c82cdef37f499: stopping tablet replica
I20260709 12:26:06.760262  5352 raft_consensus.cc:2243] T 3f803a6e48f74d26b826aa57f9233ee9 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.760795  5352 raft_consensus.cc:2272] T 3f803a6e48f74d26b826aa57f9233ee9 P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.763171  5352 tablet_replica.cc:333] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499: stopping tablet replica
I20260709 12:26:06.763708  5352 raft_consensus.cc:2243] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:26:06.764662  5352 raft_consensus.cc:2272] T bb7e15e532104619bfc9dfcc15baceea P 2807a439752a4b0ab39c82cdef37f499 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.767045  5352 tablet_replica.cc:333] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499: stopping tablet replica
I20260709 12:26:06.767529  5352 raft_consensus.cc:2243] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.767985  5352 raft_consensus.cc:2272] T 1df8831adbf14c3594dea358a98bfff3 P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.770165  5352 tablet_replica.cc:333] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499: stopping tablet replica
I20260709 12:26:06.770658  5352 raft_consensus.cc:2243] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.771135  5352 raft_consensus.cc:2272] T f1d7d9ea11c24a428a3ca7a20b11e81c P 2807a439752a4b0ab39c82cdef37f499 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.773303  5352 tablet_replica.cc:333] T 89fd2ebc3fe5461c93eac233d095a6df P 2807a439752a4b0ab39c82cdef37f499: stopping tablet replica
I20260709 12:26:06.773768  5352 raft_consensus.cc:2243] T 89fd2ebc3fe5461c93eac233d095a6df P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:06.774148  5352 raft_consensus.cc:2272] T 89fd2ebc3fe5461c93eac233d095a6df P 2807a439752a4b0ab39c82cdef37f499 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.818091  5352 tablet_server.cc:196] TabletServer@127.5.58.3:0 shutdown complete.
I20260709 12:26:06.856549  5352 master.cc:562] Master@127.5.58.62:37581 shutting down...
I20260709 12:26:06.881534  5352 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 54ac5f2af4044ee5ac6d51afda1bade2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:26:06.882216  5352 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 54ac5f2af4044ee5ac6d51afda1bade2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:06.882638  5352 tablet_replica.cc:333] T 00000000000000000000000000000000 P 54ac5f2af4044ee5ac6d51afda1bade2: stopping tablet replica
I20260709 12:26:06.904731  5352 master.cc:584] Master@127.5.58.62:37581 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (21386 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 12:26:06.969180  5352 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.58.62:35569
I20260709 12:26:06.970535  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:26:06.977319 25290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:06.977543 25289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:06.977850 25292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:26:06.980482  5352 server_base.cc:1061] running on GCE node
I20260709 12:26:06.981462  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:26:06.981668  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:26:06.981796  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599966981784 us; error 0 us; skew 500 ppm
I20260709 12:26:06.982321  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:26:06.984853  5352 webserver.cc:533] Webserver started at http://127.5.58.62:34329/ using document root <none> and password file <none>
I20260709 12:26:06.985289  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:26:06.985442  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:26:06.985649  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:26:06.986739  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599341860723-5352-0/minicluster-data/master-0-root/instance:
uuid: "c9754062669049dbb745218f0c39bff4"
format_stamp: "Formatted at 2026-07-09 12:26:06 on dist-test-slave-xs2d"
I20260709 12:26:06.992180  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.003s
I20260709 12:26:06.996763 25297 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:06.997776  5352 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:26:06.998162  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599341860723-5352-0/minicluster-data/master-0-root
uuid: "c9754062669049dbb745218f0c39bff4"
format_stamp: "Formatted at 2026-07-09 12:26:06 on dist-test-slave-xs2d"
I20260709 12:26:06.998554  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599341860723-5352-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599341860723-5352-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599341860723-5352-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:26:07.015782  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:26:07.017107  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:26:07.062587  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.62:35569
I20260709 12:26:07.062686 25348 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.62:35569 every 8 connection(s)
I20260709 12:26:07.066903 25349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:07.082384 25349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9754062669049dbb745218f0c39bff4: Bootstrap starting.
I20260709 12:26:07.087513 25349 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9754062669049dbb745218f0c39bff4: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:07.091898 25349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9754062669049dbb745218f0c39bff4: No bootstrap required, opened a new log
I20260709 12:26:07.094288 25349 raft_consensus.cc:359] T 00000000000000000000000000000000 P c9754062669049dbb745218f0c39bff4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9754062669049dbb745218f0c39bff4" member_type: VOTER }
I20260709 12:26:07.094738 25349 raft_consensus.cc:385] T 00000000000000000000000000000000 P c9754062669049dbb745218f0c39bff4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:07.094962 25349 raft_consensus.cc:740] T 00000000000000000000000000000000 P c9754062669049dbb745218f0c39bff4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9754062669049dbb745218f0c39bff4, State: Initialized, Role: FOLLOWER
I20260709 12:26:07.095602 25349 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9754062669049dbb745218f0c39bff4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9754062669049dbb745218f0c39bff4" member_type: VOTER }
I20260709 12:26:07.096125 25349 raft_consensus.cc:399] T 00000000000000000000000000000000 P c9754062669049dbb745218f0c39bff4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:26:07.096431 25349 raft_consensus.cc:493] T 00000000000000000000000000000000 P c9754062669049dbb745218f0c39bff4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:26:07.096719 25349 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c9754062669049dbb745218f0c39bff4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:07.101689 25349 raft_consensus.cc:515] T 00000000000000000000000000000000 P c9754062669049dbb745218f0c39bff4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9754062669049dbb745218f0c39bff4" member_type: VOTER }
I20260709 12:26:07.102229 25349 leader_election.cc:304] T 00000000000000000000000000000000 P c9754062669049dbb745218f0c39bff4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9754062669049dbb745218f0c39bff4; no voters: 
I20260709 12:26:07.103420 25349 leader_election.cc:290] T 00000000000000000000000000000000 P c9754062669049dbb745218f0c39bff4 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:26:07.103847 25352 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c9754062669049dbb745218f0c39bff4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:07.105400 25352 raft_consensus.cc:697] T 00000000000000000000000000000000 P c9754062669049dbb745218f0c39bff4 [term 1 LEADER]: Becoming Leader. State: Replica: c9754062669049dbb745218f0c39bff4, State: Running, Role: LEADER
I20260709 12:26:07.106160 25352 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9754062669049dbb745218f0c39bff4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9754062669049dbb745218f0c39bff4" member_type: VOTER }
I20260709 12:26:07.106709 25349 sys_catalog.cc:565] T 00000000000000000000000000000000 P c9754062669049dbb745218f0c39bff4 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:26:07.111716 25354 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9754062669049dbb745218f0c39bff4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9754062669049dbb745218f0c39bff4. Latest consensus state: current_term: 1 leader_uuid: "c9754062669049dbb745218f0c39bff4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9754062669049dbb745218f0c39bff4" member_type: VOTER } }
I20260709 12:26:07.111851 25353 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9754062669049dbb745218f0c39bff4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9754062669049dbb745218f0c39bff4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9754062669049dbb745218f0c39bff4" member_type: VOTER } }
I20260709 12:26:07.112701 25354 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9754062669049dbb745218f0c39bff4 [sys.catalog]: This master's current role is: LEADER
I20260709 12:26:07.112792 25353 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9754062669049dbb745218f0c39bff4 [sys.catalog]: This master's current role is: LEADER
I20260709 12:26:07.120033 25359 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:26:07.125630 25359 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:26:07.126914  5352 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:26:07.135790 25359 catalog_manager.cc:1383] Generated new cluster ID: ae397cbb0b6e4dab80b2710e18f39d75
I20260709 12:26:07.136080 25359 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:26:07.153090 25359 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:26:07.154423 25359 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:26:07.169055 25359 catalog_manager.cc:6070] T 00000000000000000000000000000000 P c9754062669049dbb745218f0c39bff4: Generated new TSK 0
I20260709 12:26:07.169688 25359 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:26:07.193663  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:26:07.199998 25370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:07.200975 25371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:07.202692 25373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:26:07.202786  5352 server_base.cc:1061] running on GCE node
I20260709 12:26:07.204099  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:26:07.204298  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:26:07.204586  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599967204551 us; error 0 us; skew 500 ppm
I20260709 12:26:07.205183  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:26:07.207679  5352 webserver.cc:533] Webserver started at http://127.5.58.1:35427/ using document root <none> and password file <none>
I20260709 12:26:07.208187  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:26:07.208482  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:26:07.208767  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:26:07.209936  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599341860723-5352-0/minicluster-data/ts-0-root/instance:
uuid: "997fbd9d26514ae18ff1347929d8fa12"
format_stamp: "Formatted at 2026-07-09 12:26:07 on dist-test-slave-xs2d"
I20260709 12:26:07.214558  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.005s
I20260709 12:26:07.218137 25378 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:07.219108  5352 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:26:07.219491  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599341860723-5352-0/minicluster-data/ts-0-root
uuid: "997fbd9d26514ae18ff1347929d8fa12"
format_stamp: "Formatted at 2026-07-09 12:26:07 on dist-test-slave-xs2d"
I20260709 12:26:07.219820  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599341860723-5352-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599341860723-5352-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599341860723-5352-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:26:07.229311  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:26:07.230511  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:26:07.231949  5352 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:26:07.234290  5352 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:26:07.234504  5352 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:07.234745  5352 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:26:07.234946  5352 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:07.281153  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.1:33821
I20260709 12:26:07.281275 25440 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.1:33821 every 8 connection(s)
I20260709 12:26:07.287599  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:26:07.295276 25445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:07.297178 25446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:26:07.299297  5352 server_base.cc:1061] running on GCE node
W20260709 12:26:07.302062 25448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:26:07.303161  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:26:07.303475  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:26:07.303655  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599967303636 us; error 0 us; skew 500 ppm
I20260709 12:26:07.303916 25441 heartbeater.cc:344] Connected to a master server at 127.5.58.62:35569
I20260709 12:26:07.304352 25441 heartbeater.cc:461] Registering TS with master...
I20260709 12:26:07.304394  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:26:07.305377 25441 heartbeater.cc:507] Master 127.5.58.62:35569 requested a full tablet report, sending...
I20260709 12:26:07.307315  5352 webserver.cc:533] Webserver started at http://127.5.58.2:41747/ using document root <none> and password file <none>
I20260709 12:26:07.308437  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:26:07.308535 25314 ts_manager.cc:194] Registered new tserver with Master: 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821)
I20260709 12:26:07.308965  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:26:07.309708  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:26:07.311347  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599341860723-5352-0/minicluster-data/ts-1-root/instance:
uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
format_stamp: "Formatted at 2026-07-09 12:26:07 on dist-test-slave-xs2d"
I20260709 12:26:07.311586 25314 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37598
I20260709 12:26:07.316979  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.004s
I20260709 12:26:07.321151 25453 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:07.322161  5352 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 12:26:07.322485  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599341860723-5352-0/minicluster-data/ts-1-root
uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
format_stamp: "Formatted at 2026-07-09 12:26:07 on dist-test-slave-xs2d"
I20260709 12:26:07.322742  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599341860723-5352-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599341860723-5352-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599341860723-5352-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:26:07.341490  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:26:07.342697  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:26:07.344151  5352 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:26:07.346521  5352 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:26:07.346800  5352 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:07.347046  5352 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:26:07.347214  5352 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:07.389351  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.2:45595
I20260709 12:26:07.389497 25515 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.2:45595 every 8 connection(s)
I20260709 12:26:07.394315  5352 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:26:07.404143 25519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:26:07.405031 25520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:26:07.406744  5352 server_base.cc:1061] running on GCE node
W20260709 12:26:07.407071 25522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:26:07.407892 25516 heartbeater.cc:344] Connected to a master server at 127.5.58.62:35569
I20260709 12:26:07.408126  5352 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:26:07.408371 25516 heartbeater.cc:461] Registering TS with master...
W20260709 12:26:07.408520  5352 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:26:07.408893  5352 hybrid_clock.cc:648] HybridClock initialized: now 1783599967408872 us; error 0 us; skew 500 ppm
I20260709 12:26:07.409547 25516 heartbeater.cc:507] Master 127.5.58.62:35569 requested a full tablet report, sending...
I20260709 12:26:07.409909  5352 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:26:07.411842 25314 ts_manager.cc:194] Registered new tserver with Master: e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595)
I20260709 12:26:07.412596  5352 webserver.cc:533] Webserver started at http://127.5.58.3:46041/ using document root <none> and password file <none>
I20260709 12:26:07.413130  5352 fs_manager.cc:362] Metadata directory not provided
I20260709 12:26:07.413337  5352 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:26:07.413275 25314 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37604
I20260709 12:26:07.413812  5352 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:26:07.415184  5352 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599341860723-5352-0/minicluster-data/ts-2-root/instance:
uuid: "2128ac3598974a23ad037af1cd5003f4"
format_stamp: "Formatted at 2026-07-09 12:26:07 on dist-test-slave-xs2d"
I20260709 12:26:07.419922  5352 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.003s
I20260709 12:26:07.423949 25527 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:07.425129  5352 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:26:07.425534  5352 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599341860723-5352-0/minicluster-data/ts-2-root
uuid: "2128ac3598974a23ad037af1cd5003f4"
format_stamp: "Formatted at 2026-07-09 12:26:07 on dist-test-slave-xs2d"
I20260709 12:26:07.425796  5352 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599341860723-5352-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599341860723-5352-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskS7LhV5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599341860723-5352-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:26:07.440366  5352 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:26:07.441620  5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:26:07.443246  5352 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:26:07.445739  5352 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:26:07.445973  5352 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:07.446219  5352 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:26:07.446403  5352 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:26:07.496079  5352 rpc_server.cc:307] RPC server started. Bound to: 127.5.58.3:37109
I20260709 12:26:07.496304 25589 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.58.3:37109 every 8 connection(s)
I20260709 12:26:07.512423 25590 heartbeater.cc:344] Connected to a master server at 127.5.58.62:35569
I20260709 12:26:07.512980 25590 heartbeater.cc:461] Registering TS with master...
I20260709 12:26:07.513756 25590 heartbeater.cc:507] Master 127.5.58.62:35569 requested a full tablet report, sending...
I20260709 12:26:07.516265 25314 ts_manager.cc:194] Registered new tserver with Master: 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:07.516450  5352 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016757109s
I20260709 12:26:07.518080 25314 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37612
I20260709 12:26:07.548780 25314 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37626:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:26:07.551813 25314 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:26:07.644449 25481 tablet_service.cc:1511] Processing CreateTablet for tablet 3189aaf08cab4d40b1057fbe7f1dc5a0 (DEFAULT_TABLE table=test-workload [id=f75ec85a6ae84ed29011a8343134109a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:26:07.646037 25481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3189aaf08cab4d40b1057fbe7f1dc5a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:07.653326 25480 tablet_service.cc:1511] Processing CreateTablet for tablet d23a8c8d8bbd4694b5a7346938ed345c (DEFAULT_TABLE table=test-workload [id=f75ec85a6ae84ed29011a8343134109a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:26:07.653939 25479 tablet_service.cc:1511] Processing CreateTablet for tablet 15b68b570a91456ab85bdab93a936fdf (DEFAULT_TABLE table=test-workload [id=f75ec85a6ae84ed29011a8343134109a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:26:07.654938 25480 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d23a8c8d8bbd4694b5a7346938ed345c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:07.655256 25477 tablet_service.cc:1511] Processing CreateTablet for tablet 4ac7b3f40fee4f25935d149993ed14fe (DEFAULT_TABLE table=test-workload [id=f75ec85a6ae84ed29011a8343134109a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:26:07.656647 25478 tablet_service.cc:1511] Processing CreateTablet for tablet 20fa2b8266614739b49be1cc8c6ee07f (DEFAULT_TABLE table=test-workload [id=f75ec85a6ae84ed29011a8343134109a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:26:07.657797 25476 tablet_service.cc:1511] Processing CreateTablet for tablet 7fa678a733d149f0b6aaa02a76eeb81c (DEFAULT_TABLE table=test-workload [id=f75ec85a6ae84ed29011a8343134109a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:26:07.660081 25405 tablet_service.cc:1511] Processing CreateTablet for tablet d23a8c8d8bbd4694b5a7346938ed345c (DEFAULT_TABLE table=test-workload [id=f75ec85a6ae84ed29011a8343134109a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:26:07.661330 25474 tablet_service.cc:1511] Processing CreateTablet for tablet f19c0845761542bab632f9298955bbec (DEFAULT_TABLE table=test-workload [id=f75ec85a6ae84ed29011a8343134109a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:26:07.661908 25405 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d23a8c8d8bbd4694b5a7346938ed345c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:07.663614 25406 tablet_service.cc:1511] Processing CreateTablet for tablet 3189aaf08cab4d40b1057fbe7f1dc5a0 (DEFAULT_TABLE table=test-workload [id=f75ec85a6ae84ed29011a8343134109a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:26:07.664891 25406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3189aaf08cab4d40b1057fbe7f1dc5a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:07.660084 25475 tablet_service.cc:1511] Processing CreateTablet for tablet 920bc30698554542ba1e1cead26a05d3 (DEFAULT_TABLE table=test-workload [id=f75ec85a6ae84ed29011a8343134109a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:26:07.671257 25404 tablet_service.cc:1511] Processing CreateTablet for tablet 15b68b570a91456ab85bdab93a936fdf (DEFAULT_TABLE table=test-workload [id=f75ec85a6ae84ed29011a8343134109a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:26:07.673549 25403 tablet_service.cc:1511] Processing CreateTablet for tablet 20fa2b8266614739b49be1cc8c6ee07f (DEFAULT_TABLE table=test-workload [id=f75ec85a6ae84ed29011a8343134109a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:26:07.674775 25398 tablet_service.cc:1511] Processing CreateTablet for tablet f1da56cb6b614019a612688d93a953dd (DEFAULT_TABLE table=test-workload [id=f75ec85a6ae84ed29011a8343134109a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:26:07.677579 25401 tablet_service.cc:1511] Processing CreateTablet for tablet 7fa678a733d149f0b6aaa02a76eeb81c (DEFAULT_TABLE table=test-workload [id=f75ec85a6ae84ed29011a8343134109a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:26:07.678808 25552 tablet_service.cc:1511] Processing CreateTablet for tablet 20fa2b8266614739b49be1cc8c6ee07f (DEFAULT_TABLE table=test-workload [id=f75ec85a6ae84ed29011a8343134109a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:26:07.679714 25400 tablet_service.cc:1511] Processing CreateTablet for tablet 920bc30698554542ba1e1cead26a05d3 (DEFAULT_TABLE table=test-workload [id=f75ec85a6ae84ed29011a8343134109a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:26:07.680886 25548 tablet_service.cc:1511] Processing CreateTablet for tablet f19c0845761542bab632f9298955bbec (DEFAULT_TABLE table=test-workload [id=f75ec85a6ae84ed29011a8343134109a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:26:07.681656 25551 tablet_service.cc:1511] Processing CreateTablet for tablet 4ac7b3f40fee4f25935d149993ed14fe (DEFAULT_TABLE table=test-workload [id=f75ec85a6ae84ed29011a8343134109a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:26:07.681785 25399 tablet_service.cc:1511] Processing CreateTablet for tablet f19c0845761542bab632f9298955bbec (DEFAULT_TABLE table=test-workload [id=f75ec85a6ae84ed29011a8343134109a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:26:07.675705 25402 tablet_service.cc:1511] Processing CreateTablet for tablet 4ac7b3f40fee4f25935d149993ed14fe (DEFAULT_TABLE table=test-workload [id=f75ec85a6ae84ed29011a8343134109a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:26:07.685674 25473 tablet_service.cc:1511] Processing CreateTablet for tablet f1da56cb6b614019a612688d93a953dd (DEFAULT_TABLE table=test-workload [id=f75ec85a6ae84ed29011a8343134109a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:26:07.684103 25404 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15b68b570a91456ab85bdab93a936fdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:07.687443 25554 tablet_service.cc:1511] Processing CreateTablet for tablet d23a8c8d8bbd4694b5a7346938ed345c (DEFAULT_TABLE table=test-workload [id=f75ec85a6ae84ed29011a8343134109a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:26:07.685837 25549 tablet_service.cc:1511] Processing CreateTablet for tablet 920bc30698554542ba1e1cead26a05d3 (DEFAULT_TABLE table=test-workload [id=f75ec85a6ae84ed29011a8343134109a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:26:07.687775 25547 tablet_service.cc:1511] Processing CreateTablet for tablet f1da56cb6b614019a612688d93a953dd (DEFAULT_TABLE table=test-workload [id=f75ec85a6ae84ed29011a8343134109a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:26:07.656618 25477 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ac7b3f40fee4f25935d149993ed14fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:07.689528 25555 tablet_service.cc:1511] Processing CreateTablet for tablet 3189aaf08cab4d40b1057fbe7f1dc5a0 (DEFAULT_TABLE table=test-workload [id=f75ec85a6ae84ed29011a8343134109a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:26:07.690538 25473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1da56cb6b614019a612688d93a953dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:07.677582 25553 tablet_service.cc:1511] Processing CreateTablet for tablet 15b68b570a91456ab85bdab93a936fdf (DEFAULT_TABLE table=test-workload [id=f75ec85a6ae84ed29011a8343134109a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:26:07.691097 25550 tablet_service.cc:1511] Processing CreateTablet for tablet 7fa678a733d149f0b6aaa02a76eeb81c (DEFAULT_TABLE table=test-workload [id=f75ec85a6ae84ed29011a8343134109a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:26:07.692245 25478 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20fa2b8266614739b49be1cc8c6ee07f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:07.692621 25552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20fa2b8266614739b49be1cc8c6ee07f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:07.693025 25479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15b68b570a91456ab85bdab93a936fdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:07.694000 25402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ac7b3f40fee4f25935d149993ed14fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:07.695012 25401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fa678a733d149f0b6aaa02a76eeb81c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:07.695158 25548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f19c0845761542bab632f9298955bbec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:07.696172 25553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15b68b570a91456ab85bdab93a936fdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:07.696697 25399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f19c0845761542bab632f9298955bbec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:07.696864 25554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d23a8c8d8bbd4694b5a7346938ed345c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:07.696907 25474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f19c0845761542bab632f9298955bbec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:07.698026 25555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3189aaf08cab4d40b1057fbe7f1dc5a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:07.698376 25400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 920bc30698554542ba1e1cead26a05d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:07.698760 25547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1da56cb6b614019a612688d93a953dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:07.700067 25403 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20fa2b8266614739b49be1cc8c6ee07f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:07.698030 25476 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fa678a733d149f0b6aaa02a76eeb81c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:07.701768 25549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 920bc30698554542ba1e1cead26a05d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:07.704691 25475 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 920bc30698554542ba1e1cead26a05d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:07.708721 25550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fa678a733d149f0b6aaa02a76eeb81c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:07.710047 25551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ac7b3f40fee4f25935d149993ed14fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:07.751355 25398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1da56cb6b614019a612688d93a953dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:07.792052 25607 tablet_bootstrap.cc:492] T d23a8c8d8bbd4694b5a7346938ed345c P 997fbd9d26514ae18ff1347929d8fa12: Bootstrap starting.
I20260709 12:26:07.817315 25606 tablet_bootstrap.cc:492] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4: Bootstrap starting.
I20260709 12:26:07.824400 25605 tablet_bootstrap.cc:492] T d23a8c8d8bbd4694b5a7346938ed345c P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Bootstrap starting.
I20260709 12:26:07.836694 25605 tablet_bootstrap.cc:654] T d23a8c8d8bbd4694b5a7346938ed345c P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:07.839213 25606 tablet_bootstrap.cc:654] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:07.855439 25606 tablet_bootstrap.cc:492] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4: No bootstrap required, opened a new log
I20260709 12:26:07.857204 25606 ts_tablet_manager.cc:1403] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4: Time spent bootstrapping tablet: real 0.040s	user 0.012s	sys 0.010s
I20260709 12:26:07.861322 25606 raft_consensus.cc:359] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:07.862775 25606 raft_consensus.cc:385] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:07.863397 25606 raft_consensus.cc:740] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2128ac3598974a23ad037af1cd5003f4, State: Initialized, Role: FOLLOWER
I20260709 12:26:07.864699 25606 consensus_queue.cc:260] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:07.867794 25607 tablet_bootstrap.cc:654] T d23a8c8d8bbd4694b5a7346938ed345c P 997fbd9d26514ae18ff1347929d8fa12: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:07.887197 25605 tablet_bootstrap.cc:492] T d23a8c8d8bbd4694b5a7346938ed345c P e6f5fa5b38734da2a3ef8a7ce4fc6dee: No bootstrap required, opened a new log
I20260709 12:26:07.887773 25605 ts_tablet_manager.cc:1403] T d23a8c8d8bbd4694b5a7346938ed345c P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent bootstrapping tablet: real 0.064s	user 0.010s	sys 0.043s
I20260709 12:26:07.890998 25605 raft_consensus.cc:359] T d23a8c8d8bbd4694b5a7346938ed345c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:07.891667 25605 raft_consensus.cc:385] T d23a8c8d8bbd4694b5a7346938ed345c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:07.891989 25605 raft_consensus.cc:740] T d23a8c8d8bbd4694b5a7346938ed345c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Initialized, Role: FOLLOWER
I20260709 12:26:07.893690 25605 consensus_queue.cc:260] T d23a8c8d8bbd4694b5a7346938ed345c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:07.895162 25590 heartbeater.cc:499] Master 127.5.58.62:35569 was elected leader, sending a full tablet report...
I20260709 12:26:07.895944 25606 ts_tablet_manager.cc:1434] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4: Time spent starting tablet: real 0.038s	user 0.019s	sys 0.018s
I20260709 12:26:07.896515 25607 tablet_bootstrap.cc:492] T d23a8c8d8bbd4694b5a7346938ed345c P 997fbd9d26514ae18ff1347929d8fa12: No bootstrap required, opened a new log
I20260709 12:26:07.897081 25607 ts_tablet_manager.cc:1403] T d23a8c8d8bbd4694b5a7346938ed345c P 997fbd9d26514ae18ff1347929d8fa12: Time spent bootstrapping tablet: real 0.105s	user 0.014s	sys 0.006s
I20260709 12:26:07.897403 25606 tablet_bootstrap.cc:492] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4: Bootstrap starting.
I20260709 12:26:07.900826 25607 raft_consensus.cc:359] T d23a8c8d8bbd4694b5a7346938ed345c P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:07.902470 25607 raft_consensus.cc:385] T d23a8c8d8bbd4694b5a7346938ed345c P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:07.902787 25607 raft_consensus.cc:740] T d23a8c8d8bbd4694b5a7346938ed345c P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Initialized, Role: FOLLOWER
I20260709 12:26:07.903497 25605 ts_tablet_manager.cc:1434] T d23a8c8d8bbd4694b5a7346938ed345c P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent starting tablet: real 0.015s	user 0.011s	sys 0.003s
I20260709 12:26:07.903764 25607 consensus_queue.cc:260] T d23a8c8d8bbd4694b5a7346938ed345c P 997fbd9d26514ae18ff1347929d8fa12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:07.904384 25516 heartbeater.cc:499] Master 127.5.58.62:35569 was elected leader, sending a full tablet report...
I20260709 12:26:07.905539 25606 tablet_bootstrap.cc:654] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:07.915747 25605 tablet_bootstrap.cc:492] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Bootstrap starting.
I20260709 12:26:07.926383 25605 tablet_bootstrap.cc:654] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:07.944693 25441 heartbeater.cc:499] Master 127.5.58.62:35569 was elected leader, sending a full tablet report...
I20260709 12:26:07.945432 25607 ts_tablet_manager.cc:1434] T d23a8c8d8bbd4694b5a7346938ed345c P 997fbd9d26514ae18ff1347929d8fa12: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.033s
I20260709 12:26:07.945693 25606 tablet_bootstrap.cc:492] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4: No bootstrap required, opened a new log
I20260709 12:26:07.946368 25606 ts_tablet_manager.cc:1403] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4: Time spent bootstrapping tablet: real 0.049s	user 0.038s	sys 0.009s
I20260709 12:26:07.946403 25605 tablet_bootstrap.cc:492] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee: No bootstrap required, opened a new log
I20260709 12:26:07.946647 25607 tablet_bootstrap.cc:492] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 997fbd9d26514ae18ff1347929d8fa12: Bootstrap starting.
I20260709 12:26:07.947326 25605 ts_tablet_manager.cc:1403] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent bootstrapping tablet: real 0.032s	user 0.008s	sys 0.013s
I20260709 12:26:07.950168 25606 raft_consensus.cc:359] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:07.950452 25605 raft_consensus.cc:359] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:07.951386 25606 raft_consensus.cc:385] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:07.951440 25605 raft_consensus.cc:385] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:07.952009 25606 raft_consensus.cc:740] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2128ac3598974a23ad037af1cd5003f4, State: Initialized, Role: FOLLOWER
I20260709 12:26:07.952128 25605 raft_consensus.cc:740] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Initialized, Role: FOLLOWER
I20260709 12:26:07.953215 25606 consensus_queue.cc:260] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:07.953521 25605 consensus_queue.cc:260] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:07.954301 25607 tablet_bootstrap.cc:654] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 997fbd9d26514ae18ff1347929d8fa12: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:07.956259 25606 ts_tablet_manager.cc:1434] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:26:07.957617 25606 tablet_bootstrap.cc:492] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 2128ac3598974a23ad037af1cd5003f4: Bootstrap starting.
I20260709 12:26:07.960619 25605 ts_tablet_manager.cc:1434] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.004s
I20260709 12:26:07.961755 25605 tablet_bootstrap.cc:492] T 15b68b570a91456ab85bdab93a936fdf P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Bootstrap starting.
I20260709 12:26:07.966703 25606 tablet_bootstrap.cc:654] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 2128ac3598974a23ad037af1cd5003f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:07.966768 25607 tablet_bootstrap.cc:492] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 997fbd9d26514ae18ff1347929d8fa12: No bootstrap required, opened a new log
I20260709 12:26:07.967571 25607 ts_tablet_manager.cc:1403] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 997fbd9d26514ae18ff1347929d8fa12: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.005s
I20260709 12:26:07.968941 25605 tablet_bootstrap.cc:654] T 15b68b570a91456ab85bdab93a936fdf P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:07.971328 25607 raft_consensus.cc:359] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } } peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } }
I20260709 12:26:07.972520 25607 raft_consensus.cc:385] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:07.972941 25607 raft_consensus.cc:740] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Initialized, Role: FOLLOWER
I20260709 12:26:07.974289 25607 consensus_queue.cc:260] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 997fbd9d26514ae18ff1347929d8fa12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } } peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } }
I20260709 12:26:07.978121 25607 ts_tablet_manager.cc:1434] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 997fbd9d26514ae18ff1347929d8fa12: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:26:07.979545 25607 tablet_bootstrap.cc:492] T f19c0845761542bab632f9298955bbec P 997fbd9d26514ae18ff1347929d8fa12: Bootstrap starting.
I20260709 12:26:07.980118 25610 raft_consensus.cc:493] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.980705 25610 raft_consensus.cc:515] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:07.983474 25610 leader_election.cc:290] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595)
I20260709 12:26:07.990514 25607 tablet_bootstrap.cc:654] T f19c0845761542bab632f9298955bbec P 997fbd9d26514ae18ff1347929d8fa12: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:07.995121 25606 tablet_bootstrap.cc:492] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 2128ac3598974a23ad037af1cd5003f4: No bootstrap required, opened a new log
I20260709 12:26:07.997968 25606 ts_tablet_manager.cc:1403] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 2128ac3598974a23ad037af1cd5003f4: Time spent bootstrapping tablet: real 0.041s	user 0.010s	sys 0.014s
W20260709 12:26:08.000934 25591 tablet.cc:2406] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:26:08.004443 25606 raft_consensus.cc:359] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } } peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } }
I20260709 12:26:08.005280 25606 raft_consensus.cc:385] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:08.005620 25606 raft_consensus.cc:740] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2128ac3598974a23ad037af1cd5003f4, State: Initialized, Role: FOLLOWER
I20260709 12:26:08.006585 25606 consensus_queue.cc:260] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 2128ac3598974a23ad037af1cd5003f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } } peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } }
I20260709 12:26:08.009145 25606 ts_tablet_manager.cc:1434] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 2128ac3598974a23ad037af1cd5003f4: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:26:08.013352 25605 tablet_bootstrap.cc:492] T 15b68b570a91456ab85bdab93a936fdf P e6f5fa5b38734da2a3ef8a7ce4fc6dee: No bootstrap required, opened a new log
I20260709 12:26:08.013587 25607 tablet_bootstrap.cc:492] T f19c0845761542bab632f9298955bbec P 997fbd9d26514ae18ff1347929d8fa12: No bootstrap required, opened a new log
I20260709 12:26:08.013986 25605 ts_tablet_manager.cc:1403] T 15b68b570a91456ab85bdab93a936fdf P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent bootstrapping tablet: real 0.052s	user 0.015s	sys 0.010s
I20260709 12:26:08.014230 25607 ts_tablet_manager.cc:1403] T f19c0845761542bab632f9298955bbec P 997fbd9d26514ae18ff1347929d8fa12: Time spent bootstrapping tablet: real 0.035s	user 0.018s	sys 0.011s
I20260709 12:26:08.017666 25605 raft_consensus.cc:359] T 15b68b570a91456ab85bdab93a936fdf P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.018505 25605 raft_consensus.cc:385] T 15b68b570a91456ab85bdab93a936fdf P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:08.018132 25607 raft_consensus.cc:359] T f19c0845761542bab632f9298955bbec P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.018865 25605 raft_consensus.cc:740] T 15b68b570a91456ab85bdab93a936fdf P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Initialized, Role: FOLLOWER
I20260709 12:26:08.019047 25607 raft_consensus.cc:385] T f19c0845761542bab632f9298955bbec P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:08.019455 25607 raft_consensus.cc:740] T f19c0845761542bab632f9298955bbec P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Initialized, Role: FOLLOWER
I20260709 12:26:08.019819 25605 consensus_queue.cc:260] T 15b68b570a91456ab85bdab93a936fdf P e6f5fa5b38734da2a3ef8a7ce4fc6dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.020491 25607 consensus_queue.cc:260] T f19c0845761542bab632f9298955bbec P 997fbd9d26514ae18ff1347929d8fa12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.023283 25607 ts_tablet_manager.cc:1434] T f19c0845761542bab632f9298955bbec P 997fbd9d26514ae18ff1347929d8fa12: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:26:08.025396 25607 tablet_bootstrap.cc:492] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12: Bootstrap starting.
I20260709 12:26:08.026288 25605 ts_tablet_manager.cc:1434] T 15b68b570a91456ab85bdab93a936fdf P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.004s
I20260709 12:26:08.027459 25605 tablet_bootstrap.cc:492] T f1da56cb6b614019a612688d93a953dd P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Bootstrap starting.
I20260709 12:26:08.029021 25491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d23a8c8d8bbd4694b5a7346938ed345c" candidate_uuid: "2128ac3598974a23ad037af1cd5003f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" is_pre_election: true
I20260709 12:26:08.029845 25491 raft_consensus.cc:2468] T d23a8c8d8bbd4694b5a7346938ed345c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2128ac3598974a23ad037af1cd5003f4 in term 0.
I20260709 12:26:08.032927 25607 tablet_bootstrap.cc:654] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:08.032481 25416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d23a8c8d8bbd4694b5a7346938ed345c" candidate_uuid: "2128ac3598974a23ad037af1cd5003f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997fbd9d26514ae18ff1347929d8fa12" is_pre_election: true
I20260709 12:26:08.033655 25416 raft_consensus.cc:2468] T d23a8c8d8bbd4694b5a7346938ed345c P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2128ac3598974a23ad037af1cd5003f4 in term 0.
I20260709 12:26:08.034634 25605 tablet_bootstrap.cc:654] T f1da56cb6b614019a612688d93a953dd P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:08.024303 25606 tablet_bootstrap.cc:492] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4: Bootstrap starting.
I20260709 12:26:08.046835 25607 tablet_bootstrap.cc:492] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12: No bootstrap required, opened a new log
I20260709 12:26:08.046918 25605 tablet_bootstrap.cc:492] T f1da56cb6b614019a612688d93a953dd P e6f5fa5b38734da2a3ef8a7ce4fc6dee: No bootstrap required, opened a new log
I20260709 12:26:08.047546 25607 ts_tablet_manager.cc:1403] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12: Time spent bootstrapping tablet: real 0.022s	user 0.008s	sys 0.008s
I20260709 12:26:08.048121 25605 ts_tablet_manager.cc:1403] T f1da56cb6b614019a612688d93a953dd P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent bootstrapping tablet: real 0.021s	user 0.001s	sys 0.014s
I20260709 12:26:08.042795 25531 leader_election.cc:304] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:08.049481 25606 tablet_bootstrap.cc:654] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:08.049569 25610 raft_consensus.cc:2804] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:08.050127 25610 raft_consensus.cc:493] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.050479 25610 raft_consensus.cc:3060] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:08.050894 25607 raft_consensus.cc:359] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.051829 25607 raft_consensus.cc:385] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:08.052163 25607 raft_consensus.cc:740] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Initialized, Role: FOLLOWER
I20260709 12:26:08.053186 25607 consensus_queue.cc:260] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.056293 25607 ts_tablet_manager.cc:1434] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 12:26:08.057209 25605 raft_consensus.cc:359] T f1da56cb6b614019a612688d93a953dd P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.057740 25607 tablet_bootstrap.cc:492] T 4ac7b3f40fee4f25935d149993ed14fe P 997fbd9d26514ae18ff1347929d8fa12: Bootstrap starting.
I20260709 12:26:08.058254 25605 raft_consensus.cc:385] T f1da56cb6b614019a612688d93a953dd P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:08.058723 25605 raft_consensus.cc:740] T f1da56cb6b614019a612688d93a953dd P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Initialized, Role: FOLLOWER
I20260709 12:26:08.059722 25605 consensus_queue.cc:260] T f1da56cb6b614019a612688d93a953dd P e6f5fa5b38734da2a3ef8a7ce4fc6dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.062530 25605 ts_tablet_manager.cc:1434] T f1da56cb6b614019a612688d93a953dd P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:26:08.065448 25605 tablet_bootstrap.cc:492] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Bootstrap starting.
I20260709 12:26:08.075841 25605 tablet_bootstrap.cc:654] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:08.077109 25607 tablet_bootstrap.cc:654] T 4ac7b3f40fee4f25935d149993ed14fe P 997fbd9d26514ae18ff1347929d8fa12: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:08.088515 25607 tablet_bootstrap.cc:492] T 4ac7b3f40fee4f25935d149993ed14fe P 997fbd9d26514ae18ff1347929d8fa12: No bootstrap required, opened a new log
I20260709 12:26:08.089211 25607 ts_tablet_manager.cc:1403] T 4ac7b3f40fee4f25935d149993ed14fe P 997fbd9d26514ae18ff1347929d8fa12: Time spent bootstrapping tablet: real 0.032s	user 0.019s	sys 0.002s
W20260709 12:26:08.089084 25531 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45595, user_credentials={real_user=slave}} blocked reactor thread for 53697.4us
I20260709 12:26:08.090924 25605 tablet_bootstrap.cc:492] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: No bootstrap required, opened a new log
I20260709 12:26:08.088192 25610 raft_consensus.cc:515] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.091638 25605 ts_tablet_manager.cc:1403] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent bootstrapping tablet: real 0.028s	user 0.004s	sys 0.015s
I20260709 12:26:08.094079 25416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d23a8c8d8bbd4694b5a7346938ed345c" candidate_uuid: "2128ac3598974a23ad037af1cd5003f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997fbd9d26514ae18ff1347929d8fa12"
I20260709 12:26:08.093906 25607 raft_consensus.cc:359] T 4ac7b3f40fee4f25935d149993ed14fe P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.095058 25416 raft_consensus.cc:3060] T d23a8c8d8bbd4694b5a7346938ed345c P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:08.095486 25607 raft_consensus.cc:385] T 4ac7b3f40fee4f25935d149993ed14fe P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:08.095450 25605 raft_consensus.cc:359] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.096604 25605 raft_consensus.cc:385] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:08.096546 25607 raft_consensus.cc:740] T 4ac7b3f40fee4f25935d149993ed14fe P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Initialized, Role: FOLLOWER
I20260709 12:26:08.097316 25605 raft_consensus.cc:740] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Initialized, Role: FOLLOWER
I20260709 12:26:08.098490 25605 consensus_queue.cc:260] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.098455 25607 consensus_queue.cc:260] T 4ac7b3f40fee4f25935d149993ed14fe P 997fbd9d26514ae18ff1347929d8fa12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.104894 25612 raft_consensus.cc:493] T f1da56cb6b614019a612688d93a953dd P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.105108 25607 ts_tablet_manager.cc:1434] T 4ac7b3f40fee4f25935d149993ed14fe P 997fbd9d26514ae18ff1347929d8fa12: Time spent starting tablet: real 0.015s	user 0.012s	sys 0.001s
I20260709 12:26:08.105473 25612 raft_consensus.cc:515] T f1da56cb6b614019a612688d93a953dd P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.106549 25607 tablet_bootstrap.cc:492] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12: Bootstrap starting.
I20260709 12:26:08.109407 25491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d23a8c8d8bbd4694b5a7346938ed345c" candidate_uuid: "2128ac3598974a23ad037af1cd5003f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
I20260709 12:26:08.110250 25491 raft_consensus.cc:3060] T d23a8c8d8bbd4694b5a7346938ed345c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:08.111532 25612 leader_election.cc:290] T f1da56cb6b614019a612688d93a953dd P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:08.116796 25607 tablet_bootstrap.cc:654] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:08.116783 25605 ts_tablet_manager.cc:1434] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.004s
I20260709 12:26:08.120993 25610 leader_election.cc:290] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4 [CANDIDATE]: Term 1 election: Requested vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595)
I20260709 12:26:08.122818 25605 tablet_bootstrap.cc:492] T 4ac7b3f40fee4f25935d149993ed14fe P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Bootstrap starting.
W20260709 12:26:08.123910 25362 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:26:08.131613 25491 raft_consensus.cc:2468] T d23a8c8d8bbd4694b5a7346938ed345c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2128ac3598974a23ad037af1cd5003f4 in term 1.
I20260709 12:26:08.133406 25416 raft_consensus.cc:2468] T d23a8c8d8bbd4694b5a7346938ed345c P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2128ac3598974a23ad037af1cd5003f4 in term 1.
I20260709 12:26:08.133945 25605 tablet_bootstrap.cc:654] T 4ac7b3f40fee4f25935d149993ed14fe P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:08.134895 25531 leader_election.cc:304] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, 997fbd9d26514ae18ff1347929d8fa12; no voters: 
I20260709 12:26:08.136011 25610 raft_consensus.cc:2804] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:08.138173 25607 tablet_bootstrap.cc:492] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12: No bootstrap required, opened a new log
I20260709 12:26:08.138751 25607 ts_tablet_manager.cc:1403] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12: Time spent bootstrapping tablet: real 0.032s	user 0.017s	sys 0.005s
I20260709 12:26:08.140054 25606 tablet_bootstrap.cc:492] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4: No bootstrap required, opened a new log
I20260709 12:26:08.140657 25606 ts_tablet_manager.cc:1403] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4: Time spent bootstrapping tablet: real 0.131s	user 0.046s	sys 0.035s
I20260709 12:26:08.140803 25610 raft_consensus.cc:697] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4 [term 1 LEADER]: Becoming Leader. State: Replica: 2128ac3598974a23ad037af1cd5003f4, State: Running, Role: LEADER
I20260709 12:26:08.142190 25610 consensus_queue.cc:237] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.143556 25607 raft_consensus.cc:359] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.143847 25606 raft_consensus.cc:359] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.144398 25607 raft_consensus.cc:385] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:08.144779 25606 raft_consensus.cc:385] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:08.145661 25606 raft_consensus.cc:740] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2128ac3598974a23ad037af1cd5003f4, State: Initialized, Role: FOLLOWER
I20260709 12:26:08.144779 25607 raft_consensus.cc:740] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Initialized, Role: FOLLOWER
I20260709 12:26:08.146690 25606 consensus_queue.cc:260] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.147624 25607 consensus_queue.cc:260] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.149257 25606 ts_tablet_manager.cc:1434] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:26:08.151479 25606 tablet_bootstrap.cc:492] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4: Bootstrap starting.
I20260709 12:26:08.153005 25607 ts_tablet_manager.cc:1434] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 12:26:08.154094 25607 tablet_bootstrap.cc:492] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12: Bootstrap starting.
I20260709 12:26:08.157651 25605 tablet_bootstrap.cc:492] T 4ac7b3f40fee4f25935d149993ed14fe P e6f5fa5b38734da2a3ef8a7ce4fc6dee: No bootstrap required, opened a new log
I20260709 12:26:08.158270 25605 ts_tablet_manager.cc:1403] T 4ac7b3f40fee4f25935d149993ed14fe P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent bootstrapping tablet: real 0.036s	user 0.014s	sys 0.004s
I20260709 12:26:08.158607 25606 tablet_bootstrap.cc:654] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:08.160969 25607 tablet_bootstrap.cc:654] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:08.161731 25605 raft_consensus.cc:359] T 4ac7b3f40fee4f25935d149993ed14fe P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.162539 25605 raft_consensus.cc:385] T 4ac7b3f40fee4f25935d149993ed14fe P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:08.163265 25605 raft_consensus.cc:740] T 4ac7b3f40fee4f25935d149993ed14fe P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Initialized, Role: FOLLOWER
I20260709 12:26:08.165982 25605 consensus_queue.cc:260] T 4ac7b3f40fee4f25935d149993ed14fe P e6f5fa5b38734da2a3ef8a7ce4fc6dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.174715 25416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1da56cb6b614019a612688d93a953dd" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997fbd9d26514ae18ff1347929d8fa12" is_pre_election: true
I20260709 12:26:08.177696 25565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1da56cb6b614019a612688d93a953dd" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4" is_pre_election: true
I20260709 12:26:08.179771 25605 ts_tablet_manager.cc:1434] T 4ac7b3f40fee4f25935d149993ed14fe P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.006s
W20260709 12:26:08.180084 25457 leader_election.cc:343] T f1da56cb6b614019a612688d93a953dd P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:08.181088 25605 tablet_bootstrap.cc:492] T f19c0845761542bab632f9298955bbec P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Bootstrap starting.
W20260709 12:26:08.183598 25457 leader_election.cc:343] T f1da56cb6b614019a612688d93a953dd P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:08.184158 25457 leader_election.cc:304] T f1da56cb6b614019a612688d93a953dd P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 2128ac3598974a23ad037af1cd5003f4, 997fbd9d26514ae18ff1347929d8fa12
I20260709 12:26:08.185488 25607 tablet_bootstrap.cc:492] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12: No bootstrap required, opened a new log
I20260709 12:26:08.185634 25612 raft_consensus.cc:2749] T f1da56cb6b614019a612688d93a953dd P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:08.186026 25607 ts_tablet_manager.cc:1403] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12: Time spent bootstrapping tablet: real 0.032s	user 0.009s	sys 0.001s
I20260709 12:26:08.188481 25606 tablet_bootstrap.cc:492] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4: No bootstrap required, opened a new log
I20260709 12:26:08.189080 25606 ts_tablet_manager.cc:1403] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4: Time spent bootstrapping tablet: real 0.038s	user 0.019s	sys 0.009s
I20260709 12:26:08.189193 25605 tablet_bootstrap.cc:654] T f19c0845761542bab632f9298955bbec P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:08.189230 25607 raft_consensus.cc:359] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.190256 25607 raft_consensus.cc:385] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:08.190570 25607 raft_consensus.cc:740] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Initialized, Role: FOLLOWER
I20260709 12:26:08.191427 25607 consensus_queue.cc:260] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.196362 25607 ts_tablet_manager.cc:1434] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.003s
I20260709 12:26:08.196094 25606 raft_consensus.cc:359] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.197114 25606 raft_consensus.cc:385] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:08.197521 25607 tablet_bootstrap.cc:492] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12: Bootstrap starting.
I20260709 12:26:08.197477 25606 raft_consensus.cc:740] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2128ac3598974a23ad037af1cd5003f4, State: Initialized, Role: FOLLOWER
I20260709 12:26:08.198530 25606 consensus_queue.cc:260] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.200205 25612 raft_consensus.cc:493] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.201038 25612 raft_consensus.cc:515] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.201625 25606 ts_tablet_manager.cc:1434] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:26:08.200417 25312 catalog_manager.cc:5697] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "2128ac3598974a23ad037af1cd5003f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:08.203414 25606 tablet_bootstrap.cc:492] T 7fa678a733d149f0b6aaa02a76eeb81c P 2128ac3598974a23ad037af1cd5003f4: Bootstrap starting.
I20260709 12:26:08.208070 25565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fa2b8266614739b49be1cc8c6ee07f" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4" is_pre_election: true
I20260709 12:26:08.208691 25607 tablet_bootstrap.cc:654] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:08.209168 25565 raft_consensus.cc:2468] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in term 0.
I20260709 12:26:08.205333 25416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fa2b8266614739b49be1cc8c6ee07f" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997fbd9d26514ae18ff1347929d8fa12" is_pre_election: true
I20260709 12:26:08.207399 25605 tablet_bootstrap.cc:492] T f19c0845761542bab632f9298955bbec P e6f5fa5b38734da2a3ef8a7ce4fc6dee: No bootstrap required, opened a new log
I20260709 12:26:08.210736 25605 ts_tablet_manager.cc:1403] T f19c0845761542bab632f9298955bbec P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.000s
I20260709 12:26:08.211943 25457 leader_election.cc:304] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:08.213241 25613 raft_consensus.cc:493] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.213323 25612 leader_election.cc:290] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:08.214103 25612 raft_consensus.cc:2804] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:08.213989 25613 raft_consensus.cc:515] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.214601 25612 raft_consensus.cc:493] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.215273 25612 raft_consensus.cc:3060] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:08.217286 25606 tablet_bootstrap.cc:654] T 7fa678a733d149f0b6aaa02a76eeb81c P 2128ac3598974a23ad037af1cd5003f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:08.217684 25613 leader_election.cc:290] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:08.214538 25605 raft_consensus.cc:359] T f19c0845761542bab632f9298955bbec P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.223517 25605 raft_consensus.cc:385] T f19c0845761542bab632f9298955bbec P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:08.223973 25605 raft_consensus.cc:740] T f19c0845761542bab632f9298955bbec P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Initialized, Role: FOLLOWER
I20260709 12:26:08.225117 25605 consensus_queue.cc:260] T f19c0845761542bab632f9298955bbec P e6f5fa5b38734da2a3ef8a7ce4fc6dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.227653 25613 raft_consensus.cc:493] T f19c0845761542bab632f9298955bbec P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.228286 25613 raft_consensus.cc:515] T f19c0845761542bab632f9298955bbec P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
W20260709 12:26:08.229640 25457 leader_election.cc:343] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:08.230640 25613 leader_election.cc:290] T f19c0845761542bab632f9298955bbec P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:08.234747 25605 ts_tablet_manager.cc:1434] T f19c0845761542bab632f9298955bbec P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent starting tablet: real 0.023s	user 0.004s	sys 0.004s
I20260709 12:26:08.236212 25605 tablet_bootstrap.cc:492] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Bootstrap starting.
I20260709 12:26:08.237146 25610 raft_consensus.cc:493] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.237782 25610 raft_consensus.cc:515] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.237680 25612 raft_consensus.cc:515] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.242470 25610 leader_election.cc:290] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595)
I20260709 12:26:08.251783 25491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19c0845761542bab632f9298955bbec" candidate_uuid: "2128ac3598974a23ad037af1cd5003f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" is_pre_election: true
I20260709 12:26:08.252525 25607 tablet_bootstrap.cc:492] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12: No bootstrap required, opened a new log
I20260709 12:26:08.255280 25606 tablet_bootstrap.cc:492] T 7fa678a733d149f0b6aaa02a76eeb81c P 2128ac3598974a23ad037af1cd5003f4: No bootstrap required, opened a new log
I20260709 12:26:08.256767 25607 ts_tablet_manager.cc:1403] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12: Time spent bootstrapping tablet: real 0.059s	user 0.009s	sys 0.012s
I20260709 12:26:08.258134 25606 ts_tablet_manager.cc:1403] T 7fa678a733d149f0b6aaa02a76eeb81c P 2128ac3598974a23ad037af1cd5003f4: Time spent bootstrapping tablet: real 0.055s	user 0.011s	sys 0.004s
I20260709 12:26:08.259368 25605 tablet_bootstrap.cc:654] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:08.261680 25606 raft_consensus.cc:359] T 7fa678a733d149f0b6aaa02a76eeb81c P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.262547 25606 raft_consensus.cc:385] T 7fa678a733d149f0b6aaa02a76eeb81c P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:08.262840 25606 raft_consensus.cc:740] T 7fa678a733d149f0b6aaa02a76eeb81c P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2128ac3598974a23ad037af1cd5003f4, State: Initialized, Role: FOLLOWER
I20260709 12:26:08.263955 25613 raft_consensus.cc:493] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.264631 25612 leader_election.cc:290] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 election: Requested vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:08.264822 25613 raft_consensus.cc:515] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } } peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } }
I20260709 12:26:08.265836 25606 consensus_queue.cc:260] T 7fa678a733d149f0b6aaa02a76eeb81c P 2128ac3598974a23ad037af1cd5003f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.264438 25491 raft_consensus.cc:2468] T f19c0845761542bab632f9298955bbec P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2128ac3598974a23ad037af1cd5003f4 in term 0.
I20260709 12:26:08.268731 25531 leader_election.cc:304] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:08.269541 25606 ts_tablet_manager.cc:1434] T 7fa678a733d149f0b6aaa02a76eeb81c P 2128ac3598974a23ad037af1cd5003f4: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 12:26:08.269809 25610 raft_consensus.cc:2804] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:08.270252 25610 raft_consensus.cc:493] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.270601 25610 raft_consensus.cc:3060] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:08.270756 25606 tablet_bootstrap.cc:492] T 4ac7b3f40fee4f25935d149993ed14fe P 2128ac3598974a23ad037af1cd5003f4: Bootstrap starting.
I20260709 12:26:08.271050 25416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fa2b8266614739b49be1cc8c6ee07f" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997fbd9d26514ae18ff1347929d8fa12"
I20260709 12:26:08.272755 25565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fa2b8266614739b49be1cc8c6ee07f" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
I20260709 12:26:08.273799 25565 raft_consensus.cc:3060] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:08.268127 25613 leader_election.cc:290] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:08.279651 25606 tablet_bootstrap.cc:654] T 4ac7b3f40fee4f25935d149993ed14fe P 2128ac3598974a23ad037af1cd5003f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:08.281744 25613 raft_consensus.cc:493] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.282328 25613 raft_consensus.cc:515] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.285492 25613 leader_election.cc:290] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:08.286259 25610 raft_consensus.cc:515] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.287274 25628 raft_consensus.cc:493] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.289093 25628 raft_consensus.cc:515] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.291148 25607 raft_consensus.cc:359] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.292027 25607 raft_consensus.cc:385] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:08.292402 25607 raft_consensus.cc:740] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Initialized, Role: FOLLOWER
I20260709 12:26:08.292486 25415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19c0845761542bab632f9298955bbec" candidate_uuid: "2128ac3598974a23ad037af1cd5003f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997fbd9d26514ae18ff1347929d8fa12" is_pre_election: true
I20260709 12:26:08.293288 25415 raft_consensus.cc:2468] T f19c0845761542bab632f9298955bbec P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2128ac3598974a23ad037af1cd5003f4 in term 0.
I20260709 12:26:08.293195 25607 consensus_queue.cc:260] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.293519 25491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b68b570a91456ab85bdab93a936fdf" candidate_uuid: "2128ac3598974a23ad037af1cd5003f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" is_pre_election: true
I20260709 12:26:08.294474 25491 raft_consensus.cc:2468] T 15b68b570a91456ab85bdab93a936fdf P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2128ac3598974a23ad037af1cd5003f4 in term 0.
I20260709 12:26:08.295818 25607 ts_tablet_manager.cc:1434] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12: Time spent starting tablet: real 0.039s	user 0.004s	sys 0.001s
I20260709 12:26:08.296553 25531 leader_election.cc:304] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:08.291548 25628 leader_election.cc:290] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595)
I20260709 12:26:08.297959 25628 raft_consensus.cc:2804] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:08.294534 25416 raft_consensus.cc:3060] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:08.298350 25628 raft_consensus.cc:493] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.301057 25610 leader_election.cc:290] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4 [CANDIDATE]: Term 1 election: Requested vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595)
I20260709 12:26:08.302748 25607 tablet_bootstrap.cc:492] T 920bc30698554542ba1e1cead26a05d3 P 997fbd9d26514ae18ff1347929d8fa12: Bootstrap starting.
I20260709 12:26:08.302820 25628 raft_consensus.cc:3060] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:08.304188 25491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b68b570a91456ab85bdab93a936fdf" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" is_pre_election: true
I20260709 12:26:08.304878 25491 raft_consensus.cc:2468] T 15b68b570a91456ab85bdab93a936fdf P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 997fbd9d26514ae18ff1347929d8fa12 in term 0.
I20260709 12:26:08.305497 25415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b68b570a91456ab85bdab93a936fdf" candidate_uuid: "2128ac3598974a23ad037af1cd5003f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997fbd9d26514ae18ff1347929d8fa12" is_pre_election: true
I20260709 12:26:08.306172 25415 raft_consensus.cc:2468] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2128ac3598974a23ad037af1cd5003f4 in term 0.
I20260709 12:26:08.307888 25491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19c0845761542bab632f9298955bbec" candidate_uuid: "2128ac3598974a23ad037af1cd5003f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
I20260709 12:26:08.308585 25491 raft_consensus.cc:3060] T f19c0845761542bab632f9298955bbec P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:08.310484 25416 raft_consensus.cc:2468] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in term 1.
I20260709 12:26:08.312443 25490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19c0845761542bab632f9298955bbec" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" is_pre_election: true
I20260709 12:26:08.315392 25628 raft_consensus.cc:515] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.317936 25628 leader_election.cc:290] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4 [CANDIDATE]: Term 1 election: Requested vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595)
I20260709 12:26:08.320917 25489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fa678a733d149f0b6aaa02a76eeb81c" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" is_pre_election: true
I20260709 12:26:08.321010 25564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b68b570a91456ab85bdab93a936fdf" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4" is_pre_election: true
I20260709 12:26:08.322095 25564 raft_consensus.cc:2393] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 997fbd9d26514ae18ff1347929d8fa12 in current term 1: Already voted for candidate 2128ac3598974a23ad037af1cd5003f4 in this term.
I20260709 12:26:08.321357 25607 tablet_bootstrap.cc:654] T 920bc30698554542ba1e1cead26a05d3 P 997fbd9d26514ae18ff1347929d8fa12: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:08.323277 25563 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19c0845761542bab632f9298955bbec" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4" is_pre_election: true
I20260709 12:26:08.324276 25490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3189aaf08cab4d40b1057fbe7f1dc5a0" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" is_pre_election: true
I20260709 12:26:08.324856 25563 raft_consensus.cc:2393] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 997fbd9d26514ae18ff1347929d8fa12 in current term 1: Already voted for candidate 2128ac3598974a23ad037af1cd5003f4 in this term.
I20260709 12:26:08.325224 25562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fa678a733d149f0b6aaa02a76eeb81c" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4" is_pre_election: true
I20260709 12:26:08.323618 25565 raft_consensus.cc:2468] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in term 1.
I20260709 12:26:08.326032 25415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19c0845761542bab632f9298955bbec" candidate_uuid: "2128ac3598974a23ad037af1cd5003f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997fbd9d26514ae18ff1347929d8fa12"
I20260709 12:26:08.323102 25489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b68b570a91456ab85bdab93a936fdf" candidate_uuid: "2128ac3598974a23ad037af1cd5003f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
I20260709 12:26:08.328490 25382 leader_election.cc:304] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 997fbd9d26514ae18ff1347929d8fa12, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:08.329164 25457 leader_election.cc:304] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:08.329715 25415 raft_consensus.cc:3060] T f19c0845761542bab632f9298955bbec P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:08.338989 25612 raft_consensus.cc:2804] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:08.330857 25606 tablet_bootstrap.cc:492] T 4ac7b3f40fee4f25935d149993ed14fe P 2128ac3598974a23ad037af1cd5003f4: No bootstrap required, opened a new log
I20260709 12:26:08.339504 25612 raft_consensus.cc:697] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 LEADER]: Becoming Leader. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Running, Role: LEADER
I20260709 12:26:08.339965 25606 ts_tablet_manager.cc:1403] T 4ac7b3f40fee4f25935d149993ed14fe P 2128ac3598974a23ad037af1cd5003f4: Time spent bootstrapping tablet: real 0.069s	user 0.019s	sys 0.000s
I20260709 12:26:08.340399 25612 consensus_queue.cc:237] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.343312 25606 raft_consensus.cc:359] T 4ac7b3f40fee4f25935d149993ed14fe P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.326061 25562 raft_consensus.cc:2468] T 7fa678a733d149f0b6aaa02a76eeb81c P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 997fbd9d26514ae18ff1347929d8fa12 in term 0.
I20260709 12:26:08.323186 25491 raft_consensus.cc:2468] T f19c0845761542bab632f9298955bbec P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2128ac3598974a23ad037af1cd5003f4 in term 1.
I20260709 12:26:08.344743 25613 raft_consensus.cc:2804] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:08.344933 25489 raft_consensus.cc:3060] T 15b68b570a91456ab85bdab93a936fdf P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:08.345166 25613 raft_consensus.cc:493] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.345543 25613 raft_consensus.cc:3060] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:08.326097 25564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3189aaf08cab4d40b1057fbe7f1dc5a0" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4" is_pre_election: true
I20260709 12:26:08.352036 25605 tablet_bootstrap.cc:492] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: No bootstrap required, opened a new log
I20260709 12:26:08.353021 25605 ts_tablet_manager.cc:1403] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent bootstrapping tablet: real 0.117s	user 0.016s	sys 0.002s
I20260709 12:26:08.351552 25606 raft_consensus.cc:385] T 4ac7b3f40fee4f25935d149993ed14fe P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:08.351231 25531 leader_election.cc:304] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:08.353969 25564 raft_consensus.cc:2468] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 997fbd9d26514ae18ff1347929d8fa12 in term 0.
I20260709 12:26:08.356253 25628 raft_consensus.cc:2804] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:08.356850 25628 raft_consensus.cc:697] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4 [term 1 LEADER]: Becoming Leader. State: Replica: 2128ac3598974a23ad037af1cd5003f4, State: Running, Role: LEADER
I20260709 12:26:08.356874 25489 raft_consensus.cc:2468] T 15b68b570a91456ab85bdab93a936fdf P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2128ac3598974a23ad037af1cd5003f4 in term 1.
I20260709 12:26:08.358233 25415 raft_consensus.cc:2468] T f19c0845761542bab632f9298955bbec P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2128ac3598974a23ad037af1cd5003f4 in term 1.
I20260709 12:26:08.358063 25628 consensus_queue.cc:237] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.359089 25531 leader_election.cc:304] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:08.360129 25643 raft_consensus.cc:2804] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:08.360596 25643 raft_consensus.cc:697] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4 [term 1 LEADER]: Becoming Leader. State: Replica: 2128ac3598974a23ad037af1cd5003f4, State: Running, Role: LEADER
W20260709 12:26:08.358371 25382 leader_election.cc:343] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:08.361184 25606 raft_consensus.cc:740] T 4ac7b3f40fee4f25935d149993ed14fe P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2128ac3598974a23ad037af1cd5003f4, State: Initialized, Role: FOLLOWER
W20260709 12:26:08.363683 25382 leader_election.cc:343] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:08.365232 25643 consensus_queue.cc:237] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.366838 25382 leader_election.cc:304] T f19c0845761542bab632f9298955bbec P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 997fbd9d26514ae18ff1347929d8fa12; no voters: 2128ac3598974a23ad037af1cd5003f4, e6f5fa5b38734da2a3ef8a7ce4fc6dee
I20260709 12:26:08.367163 25613 raft_consensus.cc:515] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.364111 25606 consensus_queue.cc:260] T 4ac7b3f40fee4f25935d149993ed14fe P 2128ac3598974a23ad037af1cd5003f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.374927 25607 tablet_bootstrap.cc:492] T 920bc30698554542ba1e1cead26a05d3 P 997fbd9d26514ae18ff1347929d8fa12: No bootstrap required, opened a new log
I20260709 12:26:08.375483 25607 ts_tablet_manager.cc:1403] T 920bc30698554542ba1e1cead26a05d3 P 997fbd9d26514ae18ff1347929d8fa12: Time spent bootstrapping tablet: real 0.073s	user 0.014s	sys 0.021s
I20260709 12:26:08.376300 25606 ts_tablet_manager.cc:1434] T 4ac7b3f40fee4f25935d149993ed14fe P 2128ac3598974a23ad037af1cd5003f4: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.000s
I20260709 12:26:08.377964 25606 tablet_bootstrap.cc:492] T f1da56cb6b614019a612688d93a953dd P 2128ac3598974a23ad037af1cd5003f4: Bootstrap starting.
I20260709 12:26:08.378286 25607 raft_consensus.cc:359] T 920bc30698554542ba1e1cead26a05d3 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.379024 25607 raft_consensus.cc:385] T 920bc30698554542ba1e1cead26a05d3 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:08.379292 25607 raft_consensus.cc:740] T 920bc30698554542ba1e1cead26a05d3 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Initialized, Role: FOLLOWER
I20260709 12:26:08.379920 25607 consensus_queue.cc:260] T 920bc30698554542ba1e1cead26a05d3 P 997fbd9d26514ae18ff1347929d8fa12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.381954 25605 raft_consensus.cc:359] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } } peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } }
I20260709 12:26:08.382608 25605 raft_consensus.cc:385] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:08.382901 25605 raft_consensus.cc:740] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Initialized, Role: FOLLOWER
I20260709 12:26:08.383114 25312 catalog_manager.cc:5697] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee reported cstate change: term changed from 0 to 1, leader changed from <none> to e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2). New cstate: current_term: 1 leader_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:08.385825 25605 consensus_queue.cc:260] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } } peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } }
I20260709 12:26:08.388712 25650 raft_consensus.cc:2749] T f19c0845761542bab632f9298955bbec P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:08.389065 25612 raft_consensus.cc:493] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.389667 25612 raft_consensus.cc:515] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.381536 25613 leader_election.cc:290] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 election: Requested vote from peers e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:08.392740 25415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "920bc30698554542ba1e1cead26a05d3" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997fbd9d26514ae18ff1347929d8fa12" is_pre_election: true
I20260709 12:26:08.394834 25612 leader_election.cc:290] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:08.394914 25648 raft_consensus.cc:493] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.395717 25648 raft_consensus.cc:515] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.396550 25564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "920bc30698554542ba1e1cead26a05d3" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4" is_pre_election: true
I20260709 12:26:08.396888 25607 ts_tablet_manager.cc:1434] T 920bc30698554542ba1e1cead26a05d3 P 997fbd9d26514ae18ff1347929d8fa12: Time spent starting tablet: real 0.021s	user 0.013s	sys 0.008s
I20260709 12:26:08.398042 25648 leader_election.cc:290] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
W20260709 12:26:08.400132 25457 leader_election.cc:343] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:08.393355 25415 raft_consensus.cc:2468] T 920bc30698554542ba1e1cead26a05d3 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in term 0.
I20260709 12:26:08.409503 25605 ts_tablet_manager.cc:1434] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent starting tablet: real 0.056s	user 0.008s	sys 0.000s
I20260709 12:26:08.410382 25605 tablet_bootstrap.cc:492] T 7fa678a733d149f0b6aaa02a76eeb81c P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Bootstrap starting.
I20260709 12:26:08.410820 25382 leader_election.cc:304] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, 997fbd9d26514ae18ff1347929d8fa12; no voters: e6f5fa5b38734da2a3ef8a7ce4fc6dee
I20260709 12:26:08.412462 25382 leader_election.cc:304] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, 997fbd9d26514ae18ff1347929d8fa12; no voters: e6f5fa5b38734da2a3ef8a7ce4fc6dee
I20260709 12:26:08.413355 25651 raft_consensus.cc:2804] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:08.413789 25651 raft_consensus.cc:493] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.414140 25651 raft_consensus.cc:3060] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:08.415031 25489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b68b570a91456ab85bdab93a936fdf" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
I20260709 12:26:08.415925 25489 raft_consensus.cc:2393] T 15b68b570a91456ab85bdab93a936fdf P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 997fbd9d26514ae18ff1347929d8fa12 in current term 1: Already voted for candidate 2128ac3598974a23ad037af1cd5003f4 in this term.
I20260709 12:26:08.416802 25457 leader_election.cc:304] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 997fbd9d26514ae18ff1347929d8fa12, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 2128ac3598974a23ad037af1cd5003f4
I20260709 12:26:08.415802 25612 raft_consensus.cc:493] T 4ac7b3f40fee4f25935d149993ed14fe P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.417721 25612 raft_consensus.cc:515] T 4ac7b3f40fee4f25935d149993ed14fe P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.418553 25637 raft_consensus.cc:2804] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:08.419003 25637 raft_consensus.cc:493] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.419353 25637 raft_consensus.cc:3060] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:08.420104 25612 leader_election.cc:290] T 4ac7b3f40fee4f25935d149993ed14fe P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:08.420159 25650 raft_consensus.cc:2804] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:08.421083 25491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1da56cb6b614019a612688d93a953dd" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" is_pre_election: true
I20260709 12:26:08.421455 25650 raft_consensus.cc:493] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.421846 25491 raft_consensus.cc:2468] T f1da56cb6b614019a612688d93a953dd P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 997fbd9d26514ae18ff1347929d8fa12 in term 0.
I20260709 12:26:08.421999 25650 raft_consensus.cc:3060] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:08.422981 25651 raft_consensus.cc:515] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.425179 25651 leader_election.cc:290] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 election: Requested vote from peers e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:08.425361 25564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b68b570a91456ab85bdab93a936fdf" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
I20260709 12:26:08.425792 25562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1da56cb6b614019a612688d93a953dd" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4" is_pre_election: true
I20260709 12:26:08.426607 25491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fa678a733d149f0b6aaa02a76eeb81c" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
I20260709 12:26:08.426491 25382 leader_election.cc:304] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 997fbd9d26514ae18ff1347929d8fa12, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:08.428601 25562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fa678a733d149f0b6aaa02a76eeb81c" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
I20260709 12:26:08.428821 25651 raft_consensus.cc:2804] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:08.429280 25651 raft_consensus.cc:493] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.429783 25651 raft_consensus.cc:3060] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:08.431524 25415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b68b570a91456ab85bdab93a936fdf" candidate_uuid: "2128ac3598974a23ad037af1cd5003f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997fbd9d26514ae18ff1347929d8fa12"
I20260709 12:26:08.432596 25415 raft_consensus.cc:2393] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2128ac3598974a23ad037af1cd5003f4 in current term 1: Already voted for candidate 997fbd9d26514ae18ff1347929d8fa12 in this term.
I20260709 12:26:08.433009 25605 tablet_bootstrap.cc:654] T 7fa678a733d149f0b6aaa02a76eeb81c P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:08.433130 25564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac7b3f40fee4f25935d149993ed14fe" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4" is_pre_election: true
I20260709 12:26:08.433853 25564 raft_consensus.cc:2468] T 4ac7b3f40fee4f25935d149993ed14fe P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in term 0.
I20260709 12:26:08.431635 25416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac7b3f40fee4f25935d149993ed14fe" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997fbd9d26514ae18ff1347929d8fa12" is_pre_election: true
I20260709 12:26:08.434862 25416 raft_consensus.cc:2468] T 4ac7b3f40fee4f25935d149993ed14fe P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in term 0.
I20260709 12:26:08.408458 25606 tablet_bootstrap.cc:654] T f1da56cb6b614019a612688d93a953dd P 2128ac3598974a23ad037af1cd5003f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:08.429236 25562 raft_consensus.cc:3060] T 7fa678a733d149f0b6aaa02a76eeb81c P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:08.436125 25457 leader_election.cc:304] T 4ac7b3f40fee4f25935d149993ed14fe P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 997fbd9d26514ae18ff1347929d8fa12, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:08.437043 25612 raft_consensus.cc:2804] T 4ac7b3f40fee4f25935d149993ed14fe P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:08.437417 25612 raft_consensus.cc:493] T 4ac7b3f40fee4f25935d149993ed14fe P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.437748 25612 raft_consensus.cc:3060] T 4ac7b3f40fee4f25935d149993ed14fe P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:08.438364 25651 raft_consensus.cc:515] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
W20260709 12:26:08.428635 25382 leader_election.cc:343] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:08.441038 25651 leader_election.cc:290] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 election: Requested vote from peers e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:08.441900 25382 leader_election.cc:304] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 997fbd9d26514ae18ff1347929d8fa12; no voters: 2128ac3598974a23ad037af1cd5003f4, e6f5fa5b38734da2a3ef8a7ce4fc6dee
I20260709 12:26:08.443112 25651 raft_consensus.cc:2749] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:26:08.438931 25637 raft_consensus.cc:515] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.443604 25562 raft_consensus.cc:2468] T 7fa678a733d149f0b6aaa02a76eeb81c P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 997fbd9d26514ae18ff1347929d8fa12 in term 1.
I20260709 12:26:08.444773 25311 catalog_manager.cc:5697] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "2128ac3598974a23ad037af1cd5003f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:08.448442 25606 tablet_bootstrap.cc:492] T f1da56cb6b614019a612688d93a953dd P 2128ac3598974a23ad037af1cd5003f4: No bootstrap required, opened a new log
I20260709 12:26:08.449266 25606 ts_tablet_manager.cc:1403] T f1da56cb6b614019a612688d93a953dd P 2128ac3598974a23ad037af1cd5003f4: Time spent bootstrapping tablet: real 0.072s	user 0.013s	sys 0.000s
I20260709 12:26:08.448522 25311 catalog_manager.cc:5697] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "2128ac3598974a23ad037af1cd5003f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:08.450407 25605 tablet_bootstrap.cc:492] T 7fa678a733d149f0b6aaa02a76eeb81c P e6f5fa5b38734da2a3ef8a7ce4fc6dee: No bootstrap required, opened a new log
I20260709 12:26:08.450258 25416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "920bc30698554542ba1e1cead26a05d3" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997fbd9d26514ae18ff1347929d8fa12"
I20260709 12:26:08.450875 25605 ts_tablet_manager.cc:1403] T 7fa678a733d149f0b6aaa02a76eeb81c P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent bootstrapping tablet: real 0.041s	user 0.009s	sys 0.000s
I20260709 12:26:08.450917 25416 raft_consensus.cc:3060] T 920bc30698554542ba1e1cead26a05d3 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:08.446703 25612 raft_consensus.cc:515] T 4ac7b3f40fee4f25935d149993ed14fe P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.452044 25606 raft_consensus.cc:359] T f1da56cb6b614019a612688d93a953dd P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.458112 25606 raft_consensus.cc:385] T f1da56cb6b614019a612688d93a953dd P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:08.458434 25606 raft_consensus.cc:740] T f1da56cb6b614019a612688d93a953dd P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2128ac3598974a23ad037af1cd5003f4, State: Initialized, Role: FOLLOWER
I20260709 12:26:08.459197 25606 consensus_queue.cc:260] T f1da56cb6b614019a612688d93a953dd P 2128ac3598974a23ad037af1cd5003f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.459520 25612 leader_election.cc:290] T 4ac7b3f40fee4f25935d149993ed14fe P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 election: Requested vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:08.459929 25605 raft_consensus.cc:359] T 7fa678a733d149f0b6aaa02a76eeb81c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.461036 25605 raft_consensus.cc:385] T 7fa678a733d149f0b6aaa02a76eeb81c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:08.461359 25605 raft_consensus.cc:740] T 7fa678a733d149f0b6aaa02a76eeb81c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Initialized, Role: FOLLOWER
I20260709 12:26:08.461560 25606 ts_tablet_manager.cc:1434] T f1da56cb6b614019a612688d93a953dd P 2128ac3598974a23ad037af1cd5003f4: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 12:26:08.462786 25606 tablet_bootstrap.cc:492] T 920bc30698554542ba1e1cead26a05d3 P 2128ac3598974a23ad037af1cd5003f4: Bootstrap starting.
I20260709 12:26:08.462371 25605 consensus_queue.cc:260] T 7fa678a733d149f0b6aaa02a76eeb81c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.464630 25605 ts_tablet_manager.cc:1434] T 7fa678a733d149f0b6aaa02a76eeb81c P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.002s
I20260709 12:26:08.464417 25650 raft_consensus.cc:515] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } } peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } }
I20260709 12:26:08.466238 25562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac7b3f40fee4f25935d149993ed14fe" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
I20260709 12:26:08.466609 25650 leader_election.cc:290] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 election: Requested vote from peers e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:08.466907 25562 raft_consensus.cc:3060] T 4ac7b3f40fee4f25935d149993ed14fe P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:26:08.448468 25382 leader_election.cc:343] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:08.469381 25382 leader_election.cc:304] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, 997fbd9d26514ae18ff1347929d8fa12; no voters: e6f5fa5b38734da2a3ef8a7ce4fc6dee
I20260709 12:26:08.470281 25606 tablet_bootstrap.cc:654] T 920bc30698554542ba1e1cead26a05d3 P 2128ac3598974a23ad037af1cd5003f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:08.470542 25565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3189aaf08cab4d40b1057fbe7f1dc5a0" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
I20260709 12:26:08.470810 25564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1da56cb6b614019a612688d93a953dd" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
I20260709 12:26:08.471300 25565 raft_consensus.cc:3060] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:08.471680 25564 raft_consensus.cc:3060] T f1da56cb6b614019a612688d93a953dd P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:08.473526 25491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1da56cb6b614019a612688d93a953dd" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
I20260709 12:26:08.474166 25491 raft_consensus.cc:3060] T f1da56cb6b614019a612688d93a953dd P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:08.474951 25637 leader_election.cc:290] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 election: Requested vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:08.473553 25489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3189aaf08cab4d40b1057fbe7f1dc5a0" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
I20260709 12:26:08.478571 25489 raft_consensus.cc:3060] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:08.484570 25564 raft_consensus.cc:2468] T f1da56cb6b614019a612688d93a953dd P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 997fbd9d26514ae18ff1347929d8fa12 in term 1.
I20260709 12:26:08.487560 25382 leader_election.cc:304] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, 997fbd9d26514ae18ff1347929d8fa12; no voters: 
I20260709 12:26:08.490041 25651 raft_consensus.cc:2804] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:08.461963 25415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac7b3f40fee4f25935d149993ed14fe" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997fbd9d26514ae18ff1347929d8fa12"
I20260709 12:26:08.490527 25651 raft_consensus.cc:697] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 [term 1 LEADER]: Becoming Leader. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Running, Role: LEADER
I20260709 12:26:08.490809 25415 raft_consensus.cc:3060] T 4ac7b3f40fee4f25935d149993ed14fe P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:08.491660 25651 consensus_queue.cc:237] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.490899 25648 raft_consensus.cc:2804] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:08.496858 25648 raft_consensus.cc:697] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12 [term 1 LEADER]: Becoming Leader. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Running, Role: LEADER
I20260709 12:26:08.497978 25648 consensus_queue.cc:237] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.504709 25562 raft_consensus.cc:2468] T 4ac7b3f40fee4f25935d149993ed14fe P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in term 1.
I20260709 12:26:08.509963 25457 leader_election.cc:304] T 4ac7b3f40fee4f25935d149993ed14fe P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:08.477046 25416 raft_consensus.cc:2468] T 920bc30698554542ba1e1cead26a05d3 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in term 1.
I20260709 12:26:08.515691 25457 leader_election.cc:304] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 997fbd9d26514ae18ff1347929d8fa12, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:08.516775 25612 raft_consensus.cc:2804] T 4ac7b3f40fee4f25935d149993ed14fe P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:08.508288 25491 raft_consensus.cc:2468] T f1da56cb6b614019a612688d93a953dd P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 997fbd9d26514ae18ff1347929d8fa12 in term 1.
I20260709 12:26:08.518258 25489 raft_consensus.cc:2468] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 997fbd9d26514ae18ff1347929d8fa12 in term 1.
I20260709 12:26:08.522173 25382 leader_election.cc:304] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 997fbd9d26514ae18ff1347929d8fa12, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:08.523121 25650 raft_consensus.cc:2804] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:08.523228 25415 raft_consensus.cc:2468] T 4ac7b3f40fee4f25935d149993ed14fe P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in term 1.
I20260709 12:26:08.524664 25650 raft_consensus.cc:697] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 LEADER]: Becoming Leader. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Running, Role: LEADER
I20260709 12:26:08.508684 25565 raft_consensus.cc:2468] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 997fbd9d26514ae18ff1347929d8fa12 in term 1.
I20260709 12:26:08.525991 25650 consensus_queue.cc:237] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } } peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } }
I20260709 12:26:08.526798 25564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "920bc30698554542ba1e1cead26a05d3" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
I20260709 12:26:08.519946 25637 raft_consensus.cc:2804] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:26:08.532102 25457 leader_election.cc:343] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:08.537559 25612 raft_consensus.cc:697] T 4ac7b3f40fee4f25935d149993ed14fe P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 LEADER]: Becoming Leader. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Running, Role: LEADER
I20260709 12:26:08.539239 25612 consensus_queue.cc:237] T 4ac7b3f40fee4f25935d149993ed14fe P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.541435 25606 tablet_bootstrap.cc:492] T 920bc30698554542ba1e1cead26a05d3 P 2128ac3598974a23ad037af1cd5003f4: No bootstrap required, opened a new log
I20260709 12:26:08.542158 25606 ts_tablet_manager.cc:1403] T 920bc30698554542ba1e1cead26a05d3 P 2128ac3598974a23ad037af1cd5003f4: Time spent bootstrapping tablet: real 0.080s	user 0.033s	sys 0.033s
I20260709 12:26:08.541365 25313 catalog_manager.cc:5697] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "997fbd9d26514ae18ff1347929d8fa12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:08.543692 25313 catalog_manager.cc:5697] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "997fbd9d26514ae18ff1347929d8fa12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:08.548597 25606 raft_consensus.cc:359] T 920bc30698554542ba1e1cead26a05d3 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.549422 25606 raft_consensus.cc:385] T 920bc30698554542ba1e1cead26a05d3 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:08.549732 25606 raft_consensus.cc:740] T 920bc30698554542ba1e1cead26a05d3 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2128ac3598974a23ad037af1cd5003f4, State: Initialized, Role: FOLLOWER
I20260709 12:26:08.550683 25606 consensus_queue.cc:260] T 920bc30698554542ba1e1cead26a05d3 P 2128ac3598974a23ad037af1cd5003f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.555943 25606 ts_tablet_manager.cc:1434] T 920bc30698554542ba1e1cead26a05d3 P 2128ac3598974a23ad037af1cd5003f4: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:26:08.564277 25311 catalog_manager.cc:5697] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 997fbd9d26514ae18ff1347929d8fa12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "997fbd9d26514ae18ff1347929d8fa12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:08.564886 25643 consensus_queue.cc:1048] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:08.580973 25637 raft_consensus.cc:697] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 LEADER]: Becoming Leader. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Running, Role: LEADER
I20260709 12:26:08.582059 25637 consensus_queue.cc:237] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.591912 25311 catalog_manager.cc:5697] T 4ac7b3f40fee4f25935d149993ed14fe P e6f5fa5b38734da2a3ef8a7ce4fc6dee reported cstate change: term changed from 0 to 1, leader changed from <none> to e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2). New cstate: current_term: 1 leader_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:08.593921 25311 catalog_manager.cc:5697] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee reported cstate change: term changed from 0 to 1, leader changed from <none> to e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2). New cstate: current_term: 1 leader_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:08.599067 25643 consensus_queue.cc:1048] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:08.758520 25564 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d23a8c8d8bbd4694b5a7346938ed345c"
dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
mode: GRACEFUL
new_leader_uuid: "997fbd9d26514ae18ff1347929d8fa12"
 from {username='slave'} at 127.0.0.1:45852
I20260709 12:26:08.759111 25564 raft_consensus.cc:606] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4 [term 1 LEADER]: Received request to transfer leadership to TS 997fbd9d26514ae18ff1347929d8fa12
I20260709 12:26:08.764743 25564 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15b68b570a91456ab85bdab93a936fdf"
dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
mode: GRACEFUL
new_leader_uuid: "997fbd9d26514ae18ff1347929d8fa12"
 from {username='slave'} at 127.0.0.1:45852
I20260709 12:26:08.765357 25564 raft_consensus.cc:606] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4 [term 1 LEADER]: Received request to transfer leadership to TS 997fbd9d26514ae18ff1347929d8fa12
I20260709 12:26:08.770726 25643 consensus_queue.cc:1048] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:08.783450 25491 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20fa2b8266614739b49be1cc8c6ee07f"
dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
mode: GRACEFUL
new_leader_uuid: "997fbd9d26514ae18ff1347929d8fa12"
 from {username='slave'} at 127.0.0.1:49282
I20260709 12:26:08.784045 25643 consensus_queue.cc:1048] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:08.784117 25491 raft_consensus.cc:606] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 LEADER]: Received request to transfer leadership to TS 997fbd9d26514ae18ff1347929d8fa12
I20260709 12:26:08.791147 25659 consensus_queue.cc:1048] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:08.795547 25659 consensus_queue.cc:1048] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:08.797358 25643 raft_consensus.cc:993] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4: : Instructing follower 997fbd9d26514ae18ff1347929d8fa12 to start an election
I20260709 12:26:08.798055 25628 raft_consensus.cc:1081] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4 [term 1 LEADER]: Signalling peer 997fbd9d26514ae18ff1347929d8fa12 to start an election
I20260709 12:26:08.800210 25416 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "15b68b570a91456ab85bdab93a936fdf"
dest_uuid: "997fbd9d26514ae18ff1347929d8fa12"
 from {username='slave'} at 127.0.0.1:43580
I20260709 12:26:08.801560 25416 raft_consensus.cc:493] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:08.802084 25416 raft_consensus.cc:3060] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:08.807775 25643 consensus_queue.cc:1048] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:08.808516 25637 raft_consensus.cc:993] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee: : Instructing follower 997fbd9d26514ae18ff1347929d8fa12 to start an election
I20260709 12:26:08.809038 25637 raft_consensus.cc:1081] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 LEADER]: Signalling peer 997fbd9d26514ae18ff1347929d8fa12 to start an election
I20260709 12:26:08.813092 25414 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "20fa2b8266614739b49be1cc8c6ee07f"
dest_uuid: "997fbd9d26514ae18ff1347929d8fa12"
 from {username='slave'} at 127.0.0.1:43588
I20260709 12:26:08.813707 25414 raft_consensus.cc:493] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:08.814105 25414 raft_consensus.cc:3060] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:08.820621 25416 raft_consensus.cc:515] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.825470 25414 raft_consensus.cc:515] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.831012 25637 raft_consensus.cc:493] T 7fa678a733d149f0b6aaa02a76eeb81c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.831607 25637 raft_consensus.cc:515] T 7fa678a733d149f0b6aaa02a76eeb81c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.834787 25412 raft_consensus.cc:1240] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Rejecting Update request from peer e6f5fa5b38734da2a3ef8a7ce4fc6dee for earlier term 1. Current term is 2. Ops: []
I20260709 12:26:08.833109 25413 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7fa678a733d149f0b6aaa02a76eeb81c"
dest_uuid: "997fbd9d26514ae18ff1347929d8fa12"
mode: GRACEFUL
new_leader_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
 from {username='slave'} at 127.0.0.1:43598
I20260709 12:26:08.833348 25489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b68b570a91456ab85bdab93a936fdf" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
I20260709 12:26:08.836159 25413 raft_consensus.cc:606] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 [term 1 LEADER]: Received request to transfer leadership to TS e6f5fa5b38734da2a3ef8a7ce4fc6dee
I20260709 12:26:08.836220 25489 raft_consensus.cc:3060] T 15b68b570a91456ab85bdab93a936fdf P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:08.835328 25564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15b68b570a91456ab85bdab93a936fdf" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
I20260709 12:26:08.837924 25564 raft_consensus.cc:3055] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:08.838228 25564 raft_consensus.cc:740] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2128ac3598974a23ad037af1cd5003f4, State: Running, Role: LEADER
I20260709 12:26:08.839270 25628 raft_consensus.cc:493] T 920bc30698554542ba1e1cead26a05d3 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.839756 25628 raft_consensus.cc:515] T 920bc30698554542ba1e1cead26a05d3 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.840515 25612 consensus_queue.cc:1059] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:08.842336 25628 leader_election.cc:290] T 920bc30698554542ba1e1cead26a05d3 P 2128ac3598974a23ad037af1cd5003f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595)
I20260709 12:26:08.842552 25658 raft_consensus.cc:3055] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:08.843205 25658 raft_consensus.cc:740] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Running, Role: LEADER
I20260709 12:26:08.844359 25658 consensus_queue.cc:260] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.845135 25491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "920bc30698554542ba1e1cead26a05d3" candidate_uuid: "2128ac3598974a23ad037af1cd5003f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" is_pre_election: true
I20260709 12:26:08.845613 25413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "920bc30698554542ba1e1cead26a05d3" candidate_uuid: "2128ac3598974a23ad037af1cd5003f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997fbd9d26514ae18ff1347929d8fa12" is_pre_election: true
I20260709 12:26:08.846525 25413 raft_consensus.cc:2393] T 920bc30698554542ba1e1cead26a05d3 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2128ac3598974a23ad037af1cd5003f4 in current term 1: Already voted for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in this term.
I20260709 12:26:08.847594 25531 leader_election.cc:304] T 920bc30698554542ba1e1cead26a05d3 P 2128ac3598974a23ad037af1cd5003f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4; no voters: 997fbd9d26514ae18ff1347929d8fa12, e6f5fa5b38734da2a3ef8a7ce4fc6dee
I20260709 12:26:08.847613 25643 consensus_queue.cc:1048] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:08.848645 25658 raft_consensus.cc:3060] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:08.849303 25565 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f19c0845761542bab632f9298955bbec"
dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
mode: GRACEFUL
new_leader_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
 from {username='slave'} at 127.0.0.1:45852
I20260709 12:26:08.850365 25414 leader_election.cc:290] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 2 election: Requested vote from peers e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:08.850594 25565 raft_consensus.cc:606] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4 [term 1 LEADER]: Received request to transfer leadership to TS e6f5fa5b38734da2a3ef8a7ce4fc6dee
I20260709 12:26:08.852861 25628 raft_consensus.cc:3060] T 920bc30698554542ba1e1cead26a05d3 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:08.853060 25491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fa2b8266614739b49be1cc8c6ee07f" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
I20260709 12:26:08.854264 25637 leader_election.cc:290] T 7fa678a733d149f0b6aaa02a76eeb81c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:08.854549 25562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fa2b8266614739b49be1cc8c6ee07f" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
I20260709 12:26:08.855161 25562 raft_consensus.cc:3060] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:08.845049 25489 raft_consensus.cc:2468] T 15b68b570a91456ab85bdab93a936fdf P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 997fbd9d26514ae18ff1347929d8fa12 in term 2.
I20260709 12:26:08.856184 25564 consensus_queue.cc:260] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.857447 25564 raft_consensus.cc:3060] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:08.864460 25416 leader_election.cc:290] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 2 election: Requested vote from peers e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:08.870714 25564 raft_consensus.cc:2468] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 997fbd9d26514ae18ff1347929d8fa12 in term 2.
I20260709 12:26:08.864555 25382 leader_election.cc:304] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 997fbd9d26514ae18ff1347929d8fa12, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:08.873692 25415 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f1da56cb6b614019a612688d93a953dd"
dest_uuid: "997fbd9d26514ae18ff1347929d8fa12"
mode: GRACEFUL
new_leader_uuid: "2128ac3598974a23ad037af1cd5003f4"
 from {username='slave'} at 127.0.0.1:43598
I20260709 12:26:08.874333 25650 raft_consensus.cc:2804] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:08.874351 25415 raft_consensus.cc:606] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12 [term 1 LEADER]: Received request to transfer leadership to TS 2128ac3598974a23ad037af1cd5003f4
I20260709 12:26:08.874490 25628 raft_consensus.cc:2749] T 920bc30698554542ba1e1cead26a05d3 P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:08.874898 25413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fa678a733d149f0b6aaa02a76eeb81c" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997fbd9d26514ae18ff1347929d8fa12" is_pre_election: true
I20260709 12:26:08.875718 25650 raft_consensus.cc:697] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12 [term 2 LEADER]: Becoming Leader. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Running, Role: LEADER
I20260709 12:26:08.876358 25565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fa678a733d149f0b6aaa02a76eeb81c" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4" is_pre_election: true
I20260709 12:26:08.876624 25650 consensus_queue.cc:237] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.877219 25565 raft_consensus.cc:2393] T 7fa678a733d149f0b6aaa02a76eeb81c P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in current term 1: Already voted for candidate 997fbd9d26514ae18ff1347929d8fa12 in this term.
I20260709 12:26:08.878381 25457 leader_election.cc:304] T 7fa678a733d149f0b6aaa02a76eeb81c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 2128ac3598974a23ad037af1cd5003f4, 997fbd9d26514ae18ff1347929d8fa12
I20260709 12:26:08.879678 25637 raft_consensus.cc:3060] T 7fa678a733d149f0b6aaa02a76eeb81c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:08.884866 25312 catalog_manager.cc:5697] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12 reported cstate change: term changed from 1 to 2, leader changed from 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3) to 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1). New cstate: current_term: 2 leader_uuid: "997fbd9d26514ae18ff1347929d8fa12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:08.887979 25562 raft_consensus.cc:2468] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 997fbd9d26514ae18ff1347929d8fa12 in term 2.
I20260709 12:26:08.888011 25637 raft_consensus.cc:2749] T 7fa678a733d149f0b6aaa02a76eeb81c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:08.883948 25491 raft_consensus.cc:2468] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 997fbd9d26514ae18ff1347929d8fa12 in term 2.
I20260709 12:26:08.893110 25382 leader_election.cc:304] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, 997fbd9d26514ae18ff1347929d8fa12; no voters: 
I20260709 12:26:08.894292 25648 raft_consensus.cc:2804] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:08.895035 25648 raft_consensus.cc:697] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12 [term 2 LEADER]: Becoming Leader. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Running, Role: LEADER
I20260709 12:26:08.895927 25648 consensus_queue.cc:237] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.907052 25311 catalog_manager.cc:5697] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12 reported cstate change: term changed from 1 to 2, leader changed from e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2) to 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1). New cstate: current_term: 2 leader_uuid: "997fbd9d26514ae18ff1347929d8fa12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:08.909032 25643 raft_consensus.cc:993] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4: : Instructing follower e6f5fa5b38734da2a3ef8a7ce4fc6dee to start an election
I20260709 12:26:08.909583 25643 raft_consensus.cc:1081] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4 [term 1 LEADER]: Signalling peer e6f5fa5b38734da2a3ef8a7ce4fc6dee to start an election
I20260709 12:26:08.912102 25491 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f19c0845761542bab632f9298955bbec"
dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
 from {username='slave'} at 127.0.0.1:49264
I20260709 12:26:08.913501 25491 raft_consensus.cc:493] T f19c0845761542bab632f9298955bbec P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:08.913913 25491 raft_consensus.cc:3060] T f19c0845761542bab632f9298955bbec P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:08.923698 25491 raft_consensus.cc:515] T f19c0845761542bab632f9298955bbec P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.925931 25491 leader_election.cc:290] T f19c0845761542bab632f9298955bbec P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 2 election: Requested vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:08.927209 25562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19c0845761542bab632f9298955bbec" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
I20260709 12:26:08.927974 25562 raft_consensus.cc:3055] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:08.928386 25562 raft_consensus.cc:740] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2128ac3598974a23ad037af1cd5003f4, State: Running, Role: LEADER
I20260709 12:26:08.927791 25415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f19c0845761542bab632f9298955bbec" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "997fbd9d26514ae18ff1347929d8fa12"
I20260709 12:26:08.929529 25415 raft_consensus.cc:3060] T f19c0845761542bab632f9298955bbec P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:08.929193 25562 consensus_queue.cc:260] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.930591 25562 raft_consensus.cc:3060] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:08.931493 25648 consensus_queue.cc:1048] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:08.939845 25415 raft_consensus.cc:2468] T f19c0845761542bab632f9298955bbec P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in term 2.
I20260709 12:26:08.941177 25457 leader_election.cc:304] T f19c0845761542bab632f9298955bbec P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 997fbd9d26514ae18ff1347929d8fa12, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:08.941198 25562 raft_consensus.cc:2468] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in term 2.
I20260709 12:26:08.942286 25637 raft_consensus.cc:2804] T f19c0845761542bab632f9298955bbec P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:08.942793 25637 raft_consensus.cc:697] T f19c0845761542bab632f9298955bbec P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 LEADER]: Becoming Leader. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Running, Role: LEADER
I20260709 12:26:08.943571 25637 consensus_queue.cc:237] T f19c0845761542bab632f9298955bbec P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.946597 25648 raft_consensus.cc:993] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12: : Instructing follower 2128ac3598974a23ad037af1cd5003f4 to start an election
I20260709 12:26:08.947103 25648 raft_consensus.cc:1081] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12 [term 1 LEADER]: Signalling peer 2128ac3598974a23ad037af1cd5003f4 to start an election
I20260709 12:26:08.951171 25648 consensus_queue.cc:1048] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:08.951687 25650 consensus_queue.cc:1048] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:08.952050 25562 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f1da56cb6b614019a612688d93a953dd"
dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
 from {username='slave'} at 127.0.0.1:45838
I20260709 12:26:08.954504 25562 raft_consensus.cc:493] T f1da56cb6b614019a612688d93a953dd P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:08.954900 25562 raft_consensus.cc:3060] T f1da56cb6b614019a612688d93a953dd P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:08.958973 25313 catalog_manager.cc:5697] T f19c0845761542bab632f9298955bbec P e6f5fa5b38734da2a3ef8a7ce4fc6dee reported cstate change: term changed from 1 to 2, leader changed from 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3) to e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2). New cstate: current_term: 2 leader_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:08.963218 25562 raft_consensus.cc:515] T f1da56cb6b614019a612688d93a953dd P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.968360 25415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1da56cb6b614019a612688d93a953dd" candidate_uuid: "2128ac3598974a23ad037af1cd5003f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "997fbd9d26514ae18ff1347929d8fa12"
I20260709 12:26:08.969054 25415 raft_consensus.cc:3055] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:08.969369 25415 raft_consensus.cc:740] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Running, Role: LEADER
I20260709 12:26:08.970170 25415 consensus_queue.cc:260] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.970650 25650 consensus_queue.cc:1048] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:08.971189 25415 raft_consensus.cc:3060] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:08.975246 25650 consensus_queue.cc:1048] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:08.979437 25415 raft_consensus.cc:2468] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2128ac3598974a23ad037af1cd5003f4 in term 2.
I20260709 12:26:08.980651 25531 leader_election.cc:304] T f1da56cb6b614019a612688d93a953dd P 2128ac3598974a23ad037af1cd5003f4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, 997fbd9d26514ae18ff1347929d8fa12; no voters: 
I20260709 12:26:08.981626 25610 raft_consensus.cc:2804] T f1da56cb6b614019a612688d93a953dd P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:08.982439 25610 raft_consensus.cc:697] T f1da56cb6b614019a612688d93a953dd P 2128ac3598974a23ad037af1cd5003f4 [term 2 LEADER]: Becoming Leader. State: Replica: 2128ac3598974a23ad037af1cd5003f4, State: Running, Role: LEADER
I20260709 12:26:08.983354 25610 consensus_queue.cc:237] T f1da56cb6b614019a612688d93a953dd P 2128ac3598974a23ad037af1cd5003f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:08.986356 25489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1da56cb6b614019a612688d93a953dd" candidate_uuid: "2128ac3598974a23ad037af1cd5003f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
I20260709 12:26:08.994942 25637 consensus_queue.cc:1048] T 4ac7b3f40fee4f25935d149993ed14fe P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:08.997793 25648 consensus_queue.cc:1048] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:09.006810 25648 raft_consensus.cc:993] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12: : Instructing follower e6f5fa5b38734da2a3ef8a7ce4fc6dee to start an election
I20260709 12:26:09.007275 25648 raft_consensus.cc:1081] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 [term 1 LEADER]: Signalling peer e6f5fa5b38734da2a3ef8a7ce4fc6dee to start an election
I20260709 12:26:09.009482 25637 consensus_queue.cc:1048] T 4ac7b3f40fee4f25935d149993ed14fe P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:09.012718 25562 leader_election.cc:290] T f1da56cb6b614019a612688d93a953dd P 2128ac3598974a23ad037af1cd5003f4 [CANDIDATE]: Term 2 election: Requested vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595)
I20260709 12:26:09.038153 25489 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7fa678a733d149f0b6aaa02a76eeb81c"
dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
 from {username='slave'} at 127.0.0.1:49270
I20260709 12:26:09.038225 25610 raft_consensus.cc:993] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4: : Instructing follower 997fbd9d26514ae18ff1347929d8fa12 to start an election
I20260709 12:26:09.038908 25489 raft_consensus.cc:493] T 7fa678a733d149f0b6aaa02a76eeb81c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:09.039228 25610 raft_consensus.cc:1081] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4 [term 1 LEADER]: Signalling peer 997fbd9d26514ae18ff1347929d8fa12 to start an election
I20260709 12:26:09.039405 25489 raft_consensus.cc:3060] T 7fa678a733d149f0b6aaa02a76eeb81c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:09.041491 25415 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d23a8c8d8bbd4694b5a7346938ed345c"
dest_uuid: "997fbd9d26514ae18ff1347929d8fa12"
 from {username='slave'} at 127.0.0.1:43580
I20260709 12:26:09.042155 25415 raft_consensus.cc:493] T d23a8c8d8bbd4694b5a7346938ed345c P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:09.042555 25415 raft_consensus.cc:3060] T d23a8c8d8bbd4694b5a7346938ed345c P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:09.048189 25489 raft_consensus.cc:515] T 7fa678a733d149f0b6aaa02a76eeb81c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:09.059702 25415 raft_consensus.cc:515] T d23a8c8d8bbd4694b5a7346938ed345c P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:09.063825 25489 leader_election.cc:290] T 7fa678a733d149f0b6aaa02a76eeb81c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 2 election: Requested vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:09.064898 25565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fa678a733d149f0b6aaa02a76eeb81c" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
I20260709 12:26:09.065588 25565 raft_consensus.cc:3060] T 7fa678a733d149f0b6aaa02a76eeb81c P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:09.065878 25415 leader_election.cc:290] T d23a8c8d8bbd4694b5a7346938ed345c P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 2 election: Requested vote from peers e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:09.067044 25562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d23a8c8d8bbd4694b5a7346938ed345c" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
I20260709 12:26:09.067723 25562 raft_consensus.cc:3055] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:09.068051 25562 raft_consensus.cc:740] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2128ac3598974a23ad037af1cd5003f4, State: Running, Role: LEADER
I20260709 12:26:09.068953 25562 consensus_queue.cc:260] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:09.069269 25312 catalog_manager.cc:5697] T f1da56cb6b614019a612688d93a953dd P 2128ac3598974a23ad037af1cd5003f4 reported cstate change: term changed from 1 to 2, leader changed from 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1) to 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3). New cstate: current_term: 2 leader_uuid: "2128ac3598974a23ad037af1cd5003f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:09.070129 25562 raft_consensus.cc:3060] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:09.073365 25565 raft_consensus.cc:2468] T 7fa678a733d149f0b6aaa02a76eeb81c P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in term 2.
I20260709 12:26:09.073472 25413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fa678a733d149f0b6aaa02a76eeb81c" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "997fbd9d26514ae18ff1347929d8fa12"
I20260709 12:26:09.074671 25457 leader_election.cc:304] T 7fa678a733d149f0b6aaa02a76eeb81c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:09.074213 25413 raft_consensus.cc:3055] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:09.075883 25413 raft_consensus.cc:740] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Running, Role: LEADER
I20260709 12:26:09.075513 25658 raft_consensus.cc:2804] T 7fa678a733d149f0b6aaa02a76eeb81c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:09.076525 25658 raft_consensus.cc:697] T 7fa678a733d149f0b6aaa02a76eeb81c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 LEADER]: Becoming Leader. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Running, Role: LEADER
I20260709 12:26:09.076771 25413 consensus_queue.cc:260] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:09.077384 25658 consensus_queue.cc:237] T 7fa678a733d149f0b6aaa02a76eeb81c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:09.077924 25413 raft_consensus.cc:3060] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:09.082140 25562 raft_consensus.cc:2468] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 997fbd9d26514ae18ff1347929d8fa12 in term 2.
I20260709 12:26:09.080863 25489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d23a8c8d8bbd4694b5a7346938ed345c" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
I20260709 12:26:09.083444 25382 leader_election.cc:304] T d23a8c8d8bbd4694b5a7346938ed345c P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, 997fbd9d26514ae18ff1347929d8fa12; no voters: 
I20260709 12:26:09.084050 25489 raft_consensus.cc:3060] T d23a8c8d8bbd4694b5a7346938ed345c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:09.084371 25651 raft_consensus.cc:2804] T d23a8c8d8bbd4694b5a7346938ed345c P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:09.084890 25651 raft_consensus.cc:697] T d23a8c8d8bbd4694b5a7346938ed345c P 997fbd9d26514ae18ff1347929d8fa12 [term 2 LEADER]: Becoming Leader. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Running, Role: LEADER
I20260709 12:26:09.085014 25413 raft_consensus.cc:2468] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in term 2.
I20260709 12:26:09.085783 25651 consensus_queue.cc:237] T d23a8c8d8bbd4694b5a7346938ed345c P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:09.091317 25489 raft_consensus.cc:2468] T d23a8c8d8bbd4694b5a7346938ed345c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 997fbd9d26514ae18ff1347929d8fa12 in term 2.
I20260709 12:26:09.098592 25312 catalog_manager.cc:5697] T d23a8c8d8bbd4694b5a7346938ed345c P 997fbd9d26514ae18ff1347929d8fa12 reported cstate change: term changed from 1 to 2, leader changed from 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3) to 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1). New cstate: current_term: 2 leader_uuid: "997fbd9d26514ae18ff1347929d8fa12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:09.101315 25311 catalog_manager.cc:5697] T 7fa678a733d149f0b6aaa02a76eeb81c P e6f5fa5b38734da2a3ef8a7ce4fc6dee reported cstate change: term changed from 1 to 2, leader changed from 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1) to e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2). New cstate: current_term: 2 leader_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:09.173468 25612 consensus_queue.cc:1048] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:09.185490 25612 consensus_queue.cc:1048] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:09.279717 25562 raft_consensus.cc:1275] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Refusing update from remote peer 997fbd9d26514ae18ff1347929d8fa12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:09.281237 25651 consensus_queue.cc:1048] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:09.291054 25489 raft_consensus.cc:1275] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Refusing update from remote peer 997fbd9d26514ae18ff1347929d8fa12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:09.292727 25613 consensus_queue.cc:1048] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:09.317049 25489 raft_consensus.cc:1275] T 15b68b570a91456ab85bdab93a936fdf P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Refusing update from remote peer 997fbd9d26514ae18ff1347929d8fa12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:09.318723 25613 consensus_queue.cc:1048] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:09.332693 25562 raft_consensus.cc:1275] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Refusing update from remote peer 997fbd9d26514ae18ff1347929d8fa12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:09.334182 25651 consensus_queue.cc:1048] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:09.401126 25413 raft_consensus.cc:1275] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Refusing update from remote peer 2128ac3598974a23ad037af1cd5003f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:09.402483 25643 consensus_queue.cc:1048] T f1da56cb6b614019a612688d93a953dd P 2128ac3598974a23ad037af1cd5003f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:09.411268 25489 raft_consensus.cc:3060] T f1da56cb6b614019a612688d93a953dd P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:09.422252 25489 raft_consensus.cc:1275] T f1da56cb6b614019a612688d93a953dd P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Refusing update from remote peer 2128ac3598974a23ad037af1cd5003f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:09.424266 25628 consensus_queue.cc:1048] T f1da56cb6b614019a612688d93a953dd P 2128ac3598974a23ad037af1cd5003f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:09.425482 25562 raft_consensus.cc:1275] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Refusing update from remote peer e6f5fa5b38734da2a3ef8a7ce4fc6dee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:09.426803 25612 consensus_queue.cc:1048] T f19c0845761542bab632f9298955bbec P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:09.435786 25413 raft_consensus.cc:1275] T f19c0845761542bab632f9298955bbec P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Refusing update from remote peer e6f5fa5b38734da2a3ef8a7ce4fc6dee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:09.439251 25637 consensus_queue.cc:1048] T f19c0845761542bab632f9298955bbec P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:09.527612 25562 raft_consensus.cc:1275] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Refusing update from remote peer 997fbd9d26514ae18ff1347929d8fa12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:09.529168 25613 consensus_queue.cc:1048] T d23a8c8d8bbd4694b5a7346938ed345c P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:09.540316 25489 raft_consensus.cc:1275] T d23a8c8d8bbd4694b5a7346938ed345c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Refusing update from remote peer 997fbd9d26514ae18ff1347929d8fa12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:09.540571 25413 raft_consensus.cc:1275] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Refusing update from remote peer e6f5fa5b38734da2a3ef8a7ce4fc6dee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:09.542187 25637 consensus_queue.cc:1048] T 7fa678a733d149f0b6aaa02a76eeb81c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:09.542498 25651 consensus_queue.cc:1048] T d23a8c8d8bbd4694b5a7346938ed345c P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:09.554340 25562 raft_consensus.cc:1275] T 7fa678a733d149f0b6aaa02a76eeb81c P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Refusing update from remote peer e6f5fa5b38734da2a3ef8a7ce4fc6dee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:09.557683 25637 consensus_queue.cc:1048] T 7fa678a733d149f0b6aaa02a76eeb81c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:11.914400 25312 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37640:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:26:11.916956 25312 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:26:11.973060 25404 tablet_service.cc:1511] Processing CreateTablet for tablet ca1ffddacb9444a09baf22ab222ff39c (DEFAULT_TABLE table=soft_deleted_table [id=80e824e1119c4a348c9bef53ccb1d5be]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:26:11.974255 25404 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca1ffddacb9444a09baf22ab222ff39c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:11.975572 25476 tablet_service.cc:1511] Processing CreateTablet for tablet ca1ffddacb9444a09baf22ab222ff39c (DEFAULT_TABLE table=soft_deleted_table [id=80e824e1119c4a348c9bef53ccb1d5be]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:26:11.976531 25549 tablet_service.cc:1511] Processing CreateTablet for tablet ca1ffddacb9444a09baf22ab222ff39c (DEFAULT_TABLE table=soft_deleted_table [id=80e824e1119c4a348c9bef53ccb1d5be]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:26:11.976966 25476 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca1ffddacb9444a09baf22ab222ff39c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:11.977828 25549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca1ffddacb9444a09baf22ab222ff39c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:11.997040 25400 tablet_service.cc:1511] Processing CreateTablet for tablet 01c700f42bf34944ad7cad704c5f5f36 (DEFAULT_TABLE table=soft_deleted_table [id=80e824e1119c4a348c9bef53ccb1d5be]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:26:11.998493 25400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01c700f42bf34944ad7cad704c5f5f36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:11.998003 25481 tablet_service.cc:1511] Processing CreateTablet for tablet 01c700f42bf34944ad7cad704c5f5f36 (DEFAULT_TABLE table=soft_deleted_table [id=80e824e1119c4a348c9bef53ccb1d5be]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:26:11.999552 25481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01c700f42bf34944ad7cad704c5f5f36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:11.998003 25547 tablet_service.cc:1511] Processing CreateTablet for tablet 01c700f42bf34944ad7cad704c5f5f36 (DEFAULT_TABLE table=soft_deleted_table [id=80e824e1119c4a348c9bef53ccb1d5be]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:26:12.002032 25547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01c700f42bf34944ad7cad704c5f5f36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:12.011291 25551 tablet_service.cc:1511] Processing CreateTablet for tablet ed736971b10844fbada6e03f39434740 (DEFAULT_TABLE table=soft_deleted_table [id=80e824e1119c4a348c9bef53ccb1d5be]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:26:12.012758 25551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed736971b10844fbada6e03f39434740. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:12.013105 25398 tablet_service.cc:1511] Processing CreateTablet for tablet c02e9bd829f8498b9af872415fd82d93 (DEFAULT_TABLE table=soft_deleted_table [id=80e824e1119c4a348c9bef53ccb1d5be]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:26:12.014662 25398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c02e9bd829f8498b9af872415fd82d93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:12.015300 25473 tablet_service.cc:1511] Processing CreateTablet for tablet 651a9ec9672842b6b0a0a9edb451ff33 (DEFAULT_TABLE table=soft_deleted_table [id=80e824e1119c4a348c9bef53ccb1d5be]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:26:12.011368 25474 tablet_service.cc:1511] Processing CreateTablet for tablet ed736971b10844fbada6e03f39434740 (DEFAULT_TABLE table=soft_deleted_table [id=80e824e1119c4a348c9bef53ccb1d5be]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:26:12.016067 25403 tablet_service.cc:1511] Processing CreateTablet for tablet ed736971b10844fbada6e03f39434740 (DEFAULT_TABLE table=soft_deleted_table [id=80e824e1119c4a348c9bef53ccb1d5be]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:26:12.016986 25473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 651a9ec9672842b6b0a0a9edb451ff33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:12.017314 25403 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed736971b10844fbada6e03f39434740. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:12.017020 25477 tablet_service.cc:1511] Processing CreateTablet for tablet c02e9bd829f8498b9af872415fd82d93 (DEFAULT_TABLE table=soft_deleted_table [id=80e824e1119c4a348c9bef53ccb1d5be]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:26:12.018599 25477 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c02e9bd829f8498b9af872415fd82d93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:12.014343 25475 tablet_service.cc:1511] Processing CreateTablet for tablet 18f5ed022d284752a9a35292d6adecb2 (DEFAULT_TABLE table=soft_deleted_table [id=80e824e1119c4a348c9bef53ccb1d5be]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:26:12.024168 25550 tablet_service.cc:1511] Processing CreateTablet for tablet c02e9bd829f8498b9af872415fd82d93 (DEFAULT_TABLE table=soft_deleted_table [id=80e824e1119c4a348c9bef53ccb1d5be]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:26:12.024441 25554 tablet_service.cc:1511] Processing CreateTablet for tablet 18f5ed022d284752a9a35292d6adecb2 (DEFAULT_TABLE table=soft_deleted_table [id=80e824e1119c4a348c9bef53ccb1d5be]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:26:12.026357 25548 tablet_service.cc:1511] Processing CreateTablet for tablet 651a9ec9672842b6b0a0a9edb451ff33 (DEFAULT_TABLE table=soft_deleted_table [id=80e824e1119c4a348c9bef53ccb1d5be]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:26:12.027832 25550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c02e9bd829f8498b9af872415fd82d93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:12.028127 25401 tablet_service.cc:1511] Processing CreateTablet for tablet 651a9ec9672842b6b0a0a9edb451ff33 (DEFAULT_TABLE table=soft_deleted_table [id=80e824e1119c4a348c9bef53ccb1d5be]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:26:12.029630 25401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 651a9ec9672842b6b0a0a9edb451ff33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:12.024806 25474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed736971b10844fbada6e03f39434740. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:12.032991 25548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 651a9ec9672842b6b0a0a9edb451ff33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:12.026780 25402 tablet_service.cc:1511] Processing CreateTablet for tablet 18f5ed022d284752a9a35292d6adecb2 (DEFAULT_TABLE table=soft_deleted_table [id=80e824e1119c4a348c9bef53ccb1d5be]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:26:12.034861 25402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18f5ed022d284752a9a35292d6adecb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:12.038187 25554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18f5ed022d284752a9a35292d6adecb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:12.033036 25475 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18f5ed022d284752a9a35292d6adecb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:12.046427 25399 tablet_service.cc:1511] Processing CreateTablet for tablet 61d18172d49042bb9ec71317b75eb72c (DEFAULT_TABLE table=soft_deleted_table [id=80e824e1119c4a348c9bef53ccb1d5be]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:26:12.047987 25399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61d18172d49042bb9ec71317b75eb72c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:12.069470 25406 tablet_service.cc:1511] Processing CreateTablet for tablet b3e8f6a25e95488b966e3229c04e03ef (DEFAULT_TABLE table=soft_deleted_table [id=80e824e1119c4a348c9bef53ccb1d5be]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:26:12.070271 25479 tablet_service.cc:1511] Processing CreateTablet for tablet b3e8f6a25e95488b966e3229c04e03ef (DEFAULT_TABLE table=soft_deleted_table [id=80e824e1119c4a348c9bef53ccb1d5be]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:26:12.070734 25480 tablet_service.cc:1511] Processing CreateTablet for tablet 9f66d125d0b24ecf8b2485004114dcd2 (DEFAULT_TABLE table=soft_deleted_table [id=80e824e1119c4a348c9bef53ccb1d5be]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:26:12.071461 25406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3e8f6a25e95488b966e3229c04e03ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:12.073927 25553 tablet_service.cc:1511] Processing CreateTablet for tablet 61d18172d49042bb9ec71317b75eb72c (DEFAULT_TABLE table=soft_deleted_table [id=80e824e1119c4a348c9bef53ccb1d5be]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:26:12.073604 25405 tablet_service.cc:1511] Processing CreateTablet for tablet 9f66d125d0b24ecf8b2485004114dcd2 (DEFAULT_TABLE table=soft_deleted_table [id=80e824e1119c4a348c9bef53ccb1d5be]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:26:12.074374 25552 tablet_service.cc:1511] Processing CreateTablet for tablet 9f66d125d0b24ecf8b2485004114dcd2 (DEFAULT_TABLE table=soft_deleted_table [id=80e824e1119c4a348c9bef53ccb1d5be]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:26:12.075750 25553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61d18172d49042bb9ec71317b75eb72c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:12.075829 25405 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f66d125d0b24ecf8b2485004114dcd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:12.077510 25555 tablet_service.cc:1511] Processing CreateTablet for tablet b3e8f6a25e95488b966e3229c04e03ef (DEFAULT_TABLE table=soft_deleted_table [id=80e824e1119c4a348c9bef53ccb1d5be]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:26:12.078445 25479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3e8f6a25e95488b966e3229c04e03ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:12.079531 25480 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f66d125d0b24ecf8b2485004114dcd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:12.089298 25555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3e8f6a25e95488b966e3229c04e03ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:12.091934 25552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f66d125d0b24ecf8b2485004114dcd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:12.092536 25477 tablet_service.cc:1511] Processing CreateTablet for tablet 61d18172d49042bb9ec71317b75eb72c (DEFAULT_TABLE table=soft_deleted_table [id=80e824e1119c4a348c9bef53ccb1d5be]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:26:12.137619 25477 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61d18172d49042bb9ec71317b75eb72c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:12.148324 25699 tablet_bootstrap.cc:492] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Bootstrap starting.
W20260709 12:26:12.169193 25362 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:26:12.178702 25700 tablet_bootstrap.cc:492] T 01c700f42bf34944ad7cad704c5f5f36 P 997fbd9d26514ae18ff1347929d8fa12: Bootstrap starting.
I20260709 12:26:12.183724 25699 tablet_bootstrap.cc:654] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:12.194834 25700 tablet_bootstrap.cc:654] T 01c700f42bf34944ad7cad704c5f5f36 P 997fbd9d26514ae18ff1347929d8fa12: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:12.199903 25701 tablet_bootstrap.cc:492] T ca1ffddacb9444a09baf22ab222ff39c P 2128ac3598974a23ad037af1cd5003f4: Bootstrap starting.
I20260709 12:26:12.208635 25699 tablet_bootstrap.cc:492] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee: No bootstrap required, opened a new log
I20260709 12:26:12.209079 25700 tablet_bootstrap.cc:492] T 01c700f42bf34944ad7cad704c5f5f36 P 997fbd9d26514ae18ff1347929d8fa12: No bootstrap required, opened a new log
I20260709 12:26:12.209329 25699 ts_tablet_manager.cc:1403] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent bootstrapping tablet: real 0.061s	user 0.008s	sys 0.018s
I20260709 12:26:12.209661 25700 ts_tablet_manager.cc:1403] T 01c700f42bf34944ad7cad704c5f5f36 P 997fbd9d26514ae18ff1347929d8fa12: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.004s
I20260709 12:26:12.209903 25701 tablet_bootstrap.cc:654] T ca1ffddacb9444a09baf22ab222ff39c P 2128ac3598974a23ad037af1cd5003f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:12.212174 25700 raft_consensus.cc:359] T 01c700f42bf34944ad7cad704c5f5f36 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.213061 25700 raft_consensus.cc:385] T 01c700f42bf34944ad7cad704c5f5f36 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:12.212621 25699 raft_consensus.cc:359] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.213372 25700 raft_consensus.cc:740] T 01c700f42bf34944ad7cad704c5f5f36 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Initialized, Role: FOLLOWER
I20260709 12:26:12.213706 25699 raft_consensus.cc:385] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:12.214221 25699 raft_consensus.cc:740] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Initialized, Role: FOLLOWER
I20260709 12:26:12.214207 25700 consensus_queue.cc:260] T 01c700f42bf34944ad7cad704c5f5f36 P 997fbd9d26514ae18ff1347929d8fa12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.215394 25699 consensus_queue.cc:260] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.216807 25700 ts_tablet_manager.cc:1434] T 01c700f42bf34944ad7cad704c5f5f36 P 997fbd9d26514ae18ff1347929d8fa12: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 12:26:12.218088 25700 tablet_bootstrap.cc:492] T ed736971b10844fbada6e03f39434740 P 997fbd9d26514ae18ff1347929d8fa12: Bootstrap starting.
I20260709 12:26:12.218781 25699 ts_tablet_manager.cc:1434] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.006s
I20260709 12:26:12.219961 25699 tablet_bootstrap.cc:492] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Bootstrap starting.
I20260709 12:26:12.225180 25700 tablet_bootstrap.cc:654] T ed736971b10844fbada6e03f39434740 P 997fbd9d26514ae18ff1347929d8fa12: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:12.226478 25699 tablet_bootstrap.cc:654] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:12.229549 25696 raft_consensus.cc:493] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.230355 25696 raft_consensus.cc:515] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.234359 25413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca1ffddacb9444a09baf22ab222ff39c" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997fbd9d26514ae18ff1347929d8fa12" is_pre_election: true
W20260709 12:26:12.236275 25457 leader_election.cc:343] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:12.238775 25699 tablet_bootstrap.cc:492] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: No bootstrap required, opened a new log
I20260709 12:26:12.238850 25700 tablet_bootstrap.cc:492] T ed736971b10844fbada6e03f39434740 P 997fbd9d26514ae18ff1347929d8fa12: No bootstrap required, opened a new log
I20260709 12:26:12.239490 25699 ts_tablet_manager.cc:1403] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.005s
I20260709 12:26:12.239655 25700 ts_tablet_manager.cc:1403] T ed736971b10844fbada6e03f39434740 P 997fbd9d26514ae18ff1347929d8fa12: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.004s
I20260709 12:26:12.240430 25701 tablet_bootstrap.cc:492] T ca1ffddacb9444a09baf22ab222ff39c P 2128ac3598974a23ad037af1cd5003f4: No bootstrap required, opened a new log
I20260709 12:26:12.241019 25701 ts_tablet_manager.cc:1403] T ca1ffddacb9444a09baf22ab222ff39c P 2128ac3598974a23ad037af1cd5003f4: Time spent bootstrapping tablet: real 0.041s	user 0.012s	sys 0.008s
I20260709 12:26:12.242794 25699 raft_consensus.cc:359] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.243299 25700 raft_consensus.cc:359] T ed736971b10844fbada6e03f39434740 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.243813 25699 raft_consensus.cc:385] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:12.244227 25700 raft_consensus.cc:385] T ed736971b10844fbada6e03f39434740 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:12.244403 25699 raft_consensus.cc:740] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Initialized, Role: FOLLOWER
I20260709 12:26:12.244061 25701 raft_consensus.cc:359] T ca1ffddacb9444a09baf22ab222ff39c P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.245307 25701 raft_consensus.cc:385] T ca1ffddacb9444a09baf22ab222ff39c P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:12.245677 25701 raft_consensus.cc:740] T ca1ffddacb9444a09baf22ab222ff39c P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2128ac3598974a23ad037af1cd5003f4, State: Initialized, Role: FOLLOWER
I20260709 12:26:12.245057 25700 raft_consensus.cc:740] T ed736971b10844fbada6e03f39434740 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Initialized, Role: FOLLOWER
I20260709 12:26:12.245648 25699 consensus_queue.cc:260] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.247391 25696 leader_election.cc:290] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:12.247192 25701 consensus_queue.cc:260] T ca1ffddacb9444a09baf22ab222ff39c P 2128ac3598974a23ad037af1cd5003f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.249126 25699 ts_tablet_manager.cc:1434] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:26:12.250419 25699 tablet_bootstrap.cc:492] T ed736971b10844fbada6e03f39434740 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Bootstrap starting.
I20260709 12:26:12.247040 25700 consensus_queue.cc:260] T ed736971b10844fbada6e03f39434740 P 997fbd9d26514ae18ff1347929d8fa12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.253119 25700 ts_tablet_manager.cc:1434] T ed736971b10844fbada6e03f39434740 P 997fbd9d26514ae18ff1347929d8fa12: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:26:12.254195 25700 tablet_bootstrap.cc:492] T ca1ffddacb9444a09baf22ab222ff39c P 997fbd9d26514ae18ff1347929d8fa12: Bootstrap starting.
I20260709 12:26:12.257097 25564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca1ffddacb9444a09baf22ab222ff39c" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4" is_pre_election: true
I20260709 12:26:12.257831 25564 raft_consensus.cc:2468] T ca1ffddacb9444a09baf22ab222ff39c P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in term 0.
I20260709 12:26:12.258172 25699 tablet_bootstrap.cc:654] T ed736971b10844fbada6e03f39434740 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:12.259609 25457 leader_election.cc:304] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 997fbd9d26514ae18ff1347929d8fa12
I20260709 12:26:12.260414 25700 tablet_bootstrap.cc:654] T ca1ffddacb9444a09baf22ab222ff39c P 997fbd9d26514ae18ff1347929d8fa12: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:12.263703 25699 tablet_bootstrap.cc:492] T ed736971b10844fbada6e03f39434740 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: No bootstrap required, opened a new log
I20260709 12:26:12.264806 25699 ts_tablet_manager.cc:1403] T ed736971b10844fbada6e03f39434740 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent bootstrapping tablet: real 0.015s	user 0.013s	sys 0.000s
I20260709 12:26:12.264875 25696 raft_consensus.cc:2804] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:12.265309 25696 raft_consensus.cc:493] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.265673 25696 raft_consensus.cc:3060] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:12.268226 25699 raft_consensus.cc:359] T ed736971b10844fbada6e03f39434740 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.269111 25699 raft_consensus.cc:385] T ed736971b10844fbada6e03f39434740 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:12.269451 25699 raft_consensus.cc:740] T ed736971b10844fbada6e03f39434740 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Initialized, Role: FOLLOWER
I20260709 12:26:12.250144 25701 ts_tablet_manager.cc:1434] T ca1ffddacb9444a09baf22ab222ff39c P 2128ac3598974a23ad037af1cd5003f4: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 12:26:12.271314 25701 tablet_bootstrap.cc:492] T 01c700f42bf34944ad7cad704c5f5f36 P 2128ac3598974a23ad037af1cd5003f4: Bootstrap starting.
I20260709 12:26:12.270272 25699 consensus_queue.cc:260] T ed736971b10844fbada6e03f39434740 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.277685 25699 ts_tablet_manager.cc:1434] T ed736971b10844fbada6e03f39434740 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:26:12.278756 25699 tablet_bootstrap.cc:492] T 61d18172d49042bb9ec71317b75eb72c P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Bootstrap starting.
I20260709 12:26:12.274001 25700 tablet_bootstrap.cc:492] T ca1ffddacb9444a09baf22ab222ff39c P 997fbd9d26514ae18ff1347929d8fa12: No bootstrap required, opened a new log
I20260709 12:26:12.278611 25651 raft_consensus.cc:493] T ed736971b10844fbada6e03f39434740 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.280138 25651 raft_consensus.cc:515] T ed736971b10844fbada6e03f39434740 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.282877 25651 leader_election.cc:290] T ed736971b10844fbada6e03f39434740 P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:12.283851 25701 tablet_bootstrap.cc:654] T 01c700f42bf34944ad7cad704c5f5f36 P 2128ac3598974a23ad037af1cd5003f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:12.280164 25700 ts_tablet_manager.cc:1403] T ca1ffddacb9444a09baf22ab222ff39c P 997fbd9d26514ae18ff1347929d8fa12: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.005s
I20260709 12:26:12.285717 25489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed736971b10844fbada6e03f39434740" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" is_pre_election: true
I20260709 12:26:12.286583 25699 tablet_bootstrap.cc:654] T 61d18172d49042bb9ec71317b75eb72c P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:12.275092 25696 raft_consensus.cc:515] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.286926 25489 raft_consensus.cc:2468] T ed736971b10844fbada6e03f39434740 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 997fbd9d26514ae18ff1347929d8fa12 in term 0.
I20260709 12:26:12.289407 25700 raft_consensus.cc:359] T ca1ffddacb9444a09baf22ab222ff39c P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.290370 25700 raft_consensus.cc:385] T ca1ffddacb9444a09baf22ab222ff39c P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:12.289574 25696 leader_election.cc:290] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 election: Requested vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:12.290772 25700 raft_consensus.cc:740] T ca1ffddacb9444a09baf22ab222ff39c P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Initialized, Role: FOLLOWER
I20260709 12:26:12.291513 25413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca1ffddacb9444a09baf22ab222ff39c" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997fbd9d26514ae18ff1347929d8fa12"
I20260709 12:26:12.292143 25700 consensus_queue.cc:260] T ca1ffddacb9444a09baf22ab222ff39c P 997fbd9d26514ae18ff1347929d8fa12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.292928 25382 leader_election.cc:304] T ed736971b10844fbada6e03f39434740 P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 997fbd9d26514ae18ff1347929d8fa12, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:12.294142 25413 raft_consensus.cc:3060] T ca1ffddacb9444a09baf22ab222ff39c P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:12.295992 25700 ts_tablet_manager.cc:1434] T ca1ffddacb9444a09baf22ab222ff39c P 997fbd9d26514ae18ff1347929d8fa12: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:26:12.298869 25700 tablet_bootstrap.cc:492] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12: Bootstrap starting.
I20260709 12:26:12.299636 25701 tablet_bootstrap.cc:492] T 01c700f42bf34944ad7cad704c5f5f36 P 2128ac3598974a23ad037af1cd5003f4: No bootstrap required, opened a new log
I20260709 12:26:12.299995 25699 tablet_bootstrap.cc:492] T 61d18172d49042bb9ec71317b75eb72c P e6f5fa5b38734da2a3ef8a7ce4fc6dee: No bootstrap required, opened a new log
I20260709 12:26:12.300243 25701 ts_tablet_manager.cc:1403] T 01c700f42bf34944ad7cad704c5f5f36 P 2128ac3598974a23ad037af1cd5003f4: Time spent bootstrapping tablet: real 0.029s	user 0.010s	sys 0.001s
I20260709 12:26:12.300694 25699 ts_tablet_manager.cc:1403] T 61d18172d49042bb9ec71317b75eb72c P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.011s
I20260709 12:26:12.301831 25564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca1ffddacb9444a09baf22ab222ff39c" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
I20260709 12:26:12.302549 25564 raft_consensus.cc:3060] T ca1ffddacb9444a09baf22ab222ff39c P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:12.304036 25701 raft_consensus.cc:359] T 01c700f42bf34944ad7cad704c5f5f36 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.305040 25701 raft_consensus.cc:385] T 01c700f42bf34944ad7cad704c5f5f36 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:12.304668 25699 raft_consensus.cc:359] T 61d18172d49042bb9ec71317b75eb72c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.305572 25701 raft_consensus.cc:740] T 01c700f42bf34944ad7cad704c5f5f36 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2128ac3598974a23ad037af1cd5003f4, State: Initialized, Role: FOLLOWER
I20260709 12:26:12.306005 25699 raft_consensus.cc:385] T 61d18172d49042bb9ec71317b75eb72c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:12.306417 25699 raft_consensus.cc:740] T 61d18172d49042bb9ec71317b75eb72c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Initialized, Role: FOLLOWER
I20260709 12:26:12.307003 25701 consensus_queue.cc:260] T 01c700f42bf34944ad7cad704c5f5f36 P 2128ac3598974a23ad037af1cd5003f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.305359 25651 raft_consensus.cc:2804] T ed736971b10844fbada6e03f39434740 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:12.307360 25699 consensus_queue.cc:260] T 61d18172d49042bb9ec71317b75eb72c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.308359 25700 tablet_bootstrap.cc:654] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:12.308575 25651 raft_consensus.cc:493] T ed736971b10844fbada6e03f39434740 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.309098 25651 raft_consensus.cc:3060] T ed736971b10844fbada6e03f39434740 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:12.310757 25699 ts_tablet_manager.cc:1434] T 61d18172d49042bb9ec71317b75eb72c P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 12:26:12.311995 25699 tablet_bootstrap.cc:492] T 01c700f42bf34944ad7cad704c5f5f36 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Bootstrap starting.
I20260709 12:26:12.313230 25701 ts_tablet_manager.cc:1434] T 01c700f42bf34944ad7cad704c5f5f36 P 2128ac3598974a23ad037af1cd5003f4: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:26:12.314720 25701 tablet_bootstrap.cc:492] T 18f5ed022d284752a9a35292d6adecb2 P 2128ac3598974a23ad037af1cd5003f4: Bootstrap starting.
I20260709 12:26:12.319772 25699 tablet_bootstrap.cc:654] T 01c700f42bf34944ad7cad704c5f5f36 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:12.323238 25701 tablet_bootstrap.cc:654] T 18f5ed022d284752a9a35292d6adecb2 P 2128ac3598974a23ad037af1cd5003f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:12.304754 25562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed736971b10844fbada6e03f39434740" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4" is_pre_election: true
W20260709 12:26:12.335210 25382 leader_election.cc:343] T ed736971b10844fbada6e03f39434740 P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:12.344812 25701 tablet_bootstrap.cc:492] T 18f5ed022d284752a9a35292d6adecb2 P 2128ac3598974a23ad037af1cd5003f4: No bootstrap required, opened a new log
I20260709 12:26:12.345789 25701 ts_tablet_manager.cc:1403] T 18f5ed022d284752a9a35292d6adecb2 P 2128ac3598974a23ad037af1cd5003f4: Time spent bootstrapping tablet: real 0.031s	user 0.018s	sys 0.000s
I20260709 12:26:12.346501 25413 raft_consensus.cc:2468] T ca1ffddacb9444a09baf22ab222ff39c P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in term 1.
I20260709 12:26:12.347959 25564 raft_consensus.cc:2468] T ca1ffddacb9444a09baf22ab222ff39c P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in term 1.
I20260709 12:26:12.349220 25457 leader_election.cc:304] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:12.351754 25699 tablet_bootstrap.cc:492] T 01c700f42bf34944ad7cad704c5f5f36 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: No bootstrap required, opened a new log
I20260709 12:26:12.352347 25699 ts_tablet_manager.cc:1403] T 01c700f42bf34944ad7cad704c5f5f36 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent bootstrapping tablet: real 0.041s	user 0.010s	sys 0.007s
I20260709 12:26:12.353715 25696 raft_consensus.cc:2804] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:12.354382 25696 raft_consensus.cc:697] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 LEADER]: Becoming Leader. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Running, Role: LEADER
I20260709 12:26:12.355283 25696 consensus_queue.cc:237] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.355800 25701 raft_consensus.cc:359] T 18f5ed022d284752a9a35292d6adecb2 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.357443 25701 raft_consensus.cc:385] T 18f5ed022d284752a9a35292d6adecb2 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:12.357909 25701 raft_consensus.cc:740] T 18f5ed022d284752a9a35292d6adecb2 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2128ac3598974a23ad037af1cd5003f4, State: Initialized, Role: FOLLOWER
I20260709 12:26:12.359064 25701 consensus_queue.cc:260] T 18f5ed022d284752a9a35292d6adecb2 P 2128ac3598974a23ad037af1cd5003f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.353562 25651 raft_consensus.cc:515] T ed736971b10844fbada6e03f39434740 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.361528 25699 raft_consensus.cc:359] T 01c700f42bf34944ad7cad704c5f5f36 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.362421 25699 raft_consensus.cc:385] T 01c700f42bf34944ad7cad704c5f5f36 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:12.362738 25699 raft_consensus.cc:740] T 01c700f42bf34944ad7cad704c5f5f36 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Initialized, Role: FOLLOWER
I20260709 12:26:12.364195 25699 consensus_queue.cc:260] T 01c700f42bf34944ad7cad704c5f5f36 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.367084 25699 ts_tablet_manager.cc:1434] T 01c700f42bf34944ad7cad704c5f5f36 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:26:12.368209 25699 tablet_bootstrap.cc:492] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Bootstrap starting.
I20260709 12:26:12.374468 25651 leader_election.cc:290] T ed736971b10844fbada6e03f39434740 P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 election: Requested vote from peers e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:12.375130 25489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed736971b10844fbada6e03f39434740" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
I20260709 12:26:12.381263 25489 raft_consensus.cc:3060] T ed736971b10844fbada6e03f39434740 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:12.385782 25699 tablet_bootstrap.cc:654] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:12.386729 25701 ts_tablet_manager.cc:1434] T 18f5ed022d284752a9a35292d6adecb2 P 2128ac3598974a23ad037af1cd5003f4: Time spent starting tablet: real 0.040s	user 0.009s	sys 0.000s
I20260709 12:26:12.389191 25700 tablet_bootstrap.cc:492] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12: No bootstrap required, opened a new log
I20260709 12:26:12.389839 25700 ts_tablet_manager.cc:1403] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12: Time spent bootstrapping tablet: real 0.091s	user 0.032s	sys 0.044s
I20260709 12:26:12.394112 25700 raft_consensus.cc:359] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.394829 25700 raft_consensus.cc:385] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:12.395123 25700 raft_consensus.cc:740] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Initialized, Role: FOLLOWER
I20260709 12:26:12.376816 25564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed736971b10844fbada6e03f39434740" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
I20260709 12:26:12.395871 25700 consensus_queue.cc:260] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.397181 25701 tablet_bootstrap.cc:492] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4: Bootstrap starting.
W20260709 12:26:12.397377 25382 leader_election.cc:343] T ed736971b10844fbada6e03f39434740 P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:12.398343 25700 ts_tablet_manager.cc:1434] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 12:26:12.399600 25700 tablet_bootstrap.cc:492] T 18f5ed022d284752a9a35292d6adecb2 P 997fbd9d26514ae18ff1347929d8fa12: Bootstrap starting.
I20260709 12:26:12.408157 25489 raft_consensus.cc:2468] T ed736971b10844fbada6e03f39434740 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 997fbd9d26514ae18ff1347929d8fa12 in term 1.
I20260709 12:26:12.409632 25382 leader_election.cc:304] T ed736971b10844fbada6e03f39434740 P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 997fbd9d26514ae18ff1347929d8fa12, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 2128ac3598974a23ad037af1cd5003f4
I20260709 12:26:12.410669 25699 tablet_bootstrap.cc:492] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: No bootstrap required, opened a new log
I20260709 12:26:12.410854 25651 raft_consensus.cc:2804] T ed736971b10844fbada6e03f39434740 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:12.411391 25699 ts_tablet_manager.cc:1403] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent bootstrapping tablet: real 0.043s	user 0.019s	sys 0.008s
I20260709 12:26:12.411533 25651 raft_consensus.cc:697] T ed736971b10844fbada6e03f39434740 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 LEADER]: Becoming Leader. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Running, Role: LEADER
I20260709 12:26:12.412923 25651 consensus_queue.cc:237] T ed736971b10844fbada6e03f39434740 P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.416162 25701 tablet_bootstrap.cc:654] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:12.417192 25700 tablet_bootstrap.cc:654] T 18f5ed022d284752a9a35292d6adecb2 P 997fbd9d26514ae18ff1347929d8fa12: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:12.419283 25313 catalog_manager.cc:5697] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee reported cstate change: term changed from 0 to 1, leader changed from <none> to e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2). New cstate: current_term: 1 leader_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:12.422734 25699 raft_consensus.cc:359] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.423753 25699 raft_consensus.cc:385] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:12.424135 25699 raft_consensus.cc:740] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Initialized, Role: FOLLOWER
I20260709 12:26:12.425159 25699 consensus_queue.cc:260] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.428457 25699 ts_tablet_manager.cc:1434] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent starting tablet: real 0.016s	user 0.002s	sys 0.007s
I20260709 12:26:12.429288 25696 raft_consensus.cc:493] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.429813 25699 tablet_bootstrap.cc:492] T b3e8f6a25e95488b966e3229c04e03ef P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Bootstrap starting.
I20260709 12:26:12.429875 25696 raft_consensus.cc:515] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.433378 25696 leader_election.cc:290] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:12.434713 25413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651a9ec9672842b6b0a0a9edb451ff33" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997fbd9d26514ae18ff1347929d8fa12" is_pre_election: true
I20260709 12:26:12.434913 25562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651a9ec9672842b6b0a0a9edb451ff33" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4" is_pre_election: true
W20260709 12:26:12.436636 25457 leader_election.cc:343] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:26:12.437379 25457 leader_election.cc:343] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:12.437829 25457 leader_election.cc:304] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 2128ac3598974a23ad037af1cd5003f4, 997fbd9d26514ae18ff1347929d8fa12
I20260709 12:26:12.438550 25699 tablet_bootstrap.cc:654] T b3e8f6a25e95488b966e3229c04e03ef P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:12.439021 25696 raft_consensus.cc:2749] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:12.446126 25700 tablet_bootstrap.cc:492] T 18f5ed022d284752a9a35292d6adecb2 P 997fbd9d26514ae18ff1347929d8fa12: No bootstrap required, opened a new log
I20260709 12:26:12.446730 25700 ts_tablet_manager.cc:1403] T 18f5ed022d284752a9a35292d6adecb2 P 997fbd9d26514ae18ff1347929d8fa12: Time spent bootstrapping tablet: real 0.047s	user 0.006s	sys 0.010s
I20260709 12:26:12.449914 25700 raft_consensus.cc:359] T 18f5ed022d284752a9a35292d6adecb2 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.450724 25700 raft_consensus.cc:385] T 18f5ed022d284752a9a35292d6adecb2 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:12.451076 25700 raft_consensus.cc:740] T 18f5ed022d284752a9a35292d6adecb2 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Initialized, Role: FOLLOWER
I20260709 12:26:12.451752 25700 consensus_queue.cc:260] T 18f5ed022d284752a9a35292d6adecb2 P 997fbd9d26514ae18ff1347929d8fa12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.454429 25700 ts_tablet_manager.cc:1434] T 18f5ed022d284752a9a35292d6adecb2 P 997fbd9d26514ae18ff1347929d8fa12: Time spent starting tablet: real 0.007s	user 0.009s	sys 0.001s
I20260709 12:26:12.454830 25701 tablet_bootstrap.cc:492] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4: No bootstrap required, opened a new log
I20260709 12:26:12.455462 25701 ts_tablet_manager.cc:1403] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4: Time spent bootstrapping tablet: real 0.059s	user 0.013s	sys 0.025s
I20260709 12:26:12.456539 25699 tablet_bootstrap.cc:492] T b3e8f6a25e95488b966e3229c04e03ef P e6f5fa5b38734da2a3ef8a7ce4fc6dee: No bootstrap required, opened a new log
I20260709 12:26:12.457181 25699 ts_tablet_manager.cc:1403] T b3e8f6a25e95488b966e3229c04e03ef P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.009s
I20260709 12:26:12.460086 25699 raft_consensus.cc:359] T b3e8f6a25e95488b966e3229c04e03ef P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.459793 25311 catalog_manager.cc:5697] T ed736971b10844fbada6e03f39434740 P 997fbd9d26514ae18ff1347929d8fa12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "997fbd9d26514ae18ff1347929d8fa12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:12.460855 25699 raft_consensus.cc:385] T b3e8f6a25e95488b966e3229c04e03ef P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:12.461184 25699 raft_consensus.cc:740] T b3e8f6a25e95488b966e3229c04e03ef P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Initialized, Role: FOLLOWER
I20260709 12:26:12.461892 25699 consensus_queue.cc:260] T b3e8f6a25e95488b966e3229c04e03ef P e6f5fa5b38734da2a3ef8a7ce4fc6dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.463567 25699 ts_tablet_manager.cc:1434] T b3e8f6a25e95488b966e3229c04e03ef P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:26:12.465149 25700 tablet_bootstrap.cc:492] T c02e9bd829f8498b9af872415fd82d93 P 997fbd9d26514ae18ff1347929d8fa12: Bootstrap starting.
I20260709 12:26:12.466635 25701 raft_consensus.cc:359] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.467669 25701 raft_consensus.cc:385] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:12.468030 25701 raft_consensus.cc:740] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2128ac3598974a23ad037af1cd5003f4, State: Initialized, Role: FOLLOWER
I20260709 12:26:12.469134 25701 consensus_queue.cc:260] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.471904 25699 tablet_bootstrap.cc:492] T 18f5ed022d284752a9a35292d6adecb2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Bootstrap starting.
I20260709 12:26:12.472617 25700 tablet_bootstrap.cc:654] T c02e9bd829f8498b9af872415fd82d93 P 997fbd9d26514ae18ff1347929d8fa12: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:12.479794 25701 ts_tablet_manager.cc:1434] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.001s
I20260709 12:26:12.480847 25699 tablet_bootstrap.cc:654] T 18f5ed022d284752a9a35292d6adecb2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:12.487392 25701 tablet_bootstrap.cc:492] T ed736971b10844fbada6e03f39434740 P 2128ac3598974a23ad037af1cd5003f4: Bootstrap starting.
I20260709 12:26:12.494447 25700 tablet_bootstrap.cc:492] T c02e9bd829f8498b9af872415fd82d93 P 997fbd9d26514ae18ff1347929d8fa12: No bootstrap required, opened a new log
I20260709 12:26:12.494913 25699 tablet_bootstrap.cc:492] T 18f5ed022d284752a9a35292d6adecb2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: No bootstrap required, opened a new log
I20260709 12:26:12.494976 25701 tablet_bootstrap.cc:654] T ed736971b10844fbada6e03f39434740 P 2128ac3598974a23ad037af1cd5003f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:12.495033 25700 ts_tablet_manager.cc:1403] T c02e9bd829f8498b9af872415fd82d93 P 997fbd9d26514ae18ff1347929d8fa12: Time spent bootstrapping tablet: real 0.030s	user 0.007s	sys 0.021s
I20260709 12:26:12.495651 25699 ts_tablet_manager.cc:1403] T 18f5ed022d284752a9a35292d6adecb2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent bootstrapping tablet: real 0.031s	user 0.014s	sys 0.007s
I20260709 12:26:12.497822 25700 raft_consensus.cc:359] T c02e9bd829f8498b9af872415fd82d93 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.498435 25700 raft_consensus.cc:385] T c02e9bd829f8498b9af872415fd82d93 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:12.498677 25700 raft_consensus.cc:740] T c02e9bd829f8498b9af872415fd82d93 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Initialized, Role: FOLLOWER
I20260709 12:26:12.499234 25699 raft_consensus.cc:359] T 18f5ed022d284752a9a35292d6adecb2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.499559 25700 consensus_queue.cc:260] T c02e9bd829f8498b9af872415fd82d93 P 997fbd9d26514ae18ff1347929d8fa12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.499969 25699 raft_consensus.cc:385] T 18f5ed022d284752a9a35292d6adecb2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:12.500455 25699 raft_consensus.cc:740] T 18f5ed022d284752a9a35292d6adecb2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Initialized, Role: FOLLOWER
I20260709 12:26:12.501384 25699 consensus_queue.cc:260] T 18f5ed022d284752a9a35292d6adecb2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.501859 25701 tablet_bootstrap.cc:492] T ed736971b10844fbada6e03f39434740 P 2128ac3598974a23ad037af1cd5003f4: No bootstrap required, opened a new log
I20260709 12:26:12.502069 25700 ts_tablet_manager.cc:1434] T c02e9bd829f8498b9af872415fd82d93 P 997fbd9d26514ae18ff1347929d8fa12: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.002s
I20260709 12:26:12.503189 25700 tablet_bootstrap.cc:492] T 61d18172d49042bb9ec71317b75eb72c P 997fbd9d26514ae18ff1347929d8fa12: Bootstrap starting.
I20260709 12:26:12.503643 25699 ts_tablet_manager.cc:1434] T 18f5ed022d284752a9a35292d6adecb2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent starting tablet: real 0.008s	user 0.000s	sys 0.005s
I20260709 12:26:12.504263 25696 raft_consensus.cc:493] T 18f5ed022d284752a9a35292d6adecb2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.504531 25651 raft_consensus.cc:493] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.504907 25699 tablet_bootstrap.cc:492] T 9f66d125d0b24ecf8b2485004114dcd2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Bootstrap starting.
I20260709 12:26:12.504784 25696 raft_consensus.cc:515] T 18f5ed022d284752a9a35292d6adecb2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.505060 25651 raft_consensus.cc:515] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.506976 25696 leader_election.cc:290] T 18f5ed022d284752a9a35292d6adecb2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:12.507212 25651 leader_election.cc:290] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:12.509148 25489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3e8f6a25e95488b966e3229c04e03ef" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" is_pre_election: true
I20260709 12:26:12.509812 25413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18f5ed022d284752a9a35292d6adecb2" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997fbd9d26514ae18ff1347929d8fa12" is_pre_election: true
I20260709 12:26:12.510511 25413 raft_consensus.cc:2468] T 18f5ed022d284752a9a35292d6adecb2 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in term 0.
I20260709 12:26:12.511574 25562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3e8f6a25e95488b966e3229c04e03ef" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4" is_pre_election: true
I20260709 12:26:12.511586 25699 tablet_bootstrap.cc:654] T 9f66d125d0b24ecf8b2485004114dcd2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:12.512872 25564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18f5ed022d284752a9a35292d6adecb2" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4" is_pre_election: true
I20260709 12:26:12.513577 25564 raft_consensus.cc:2468] T 18f5ed022d284752a9a35292d6adecb2 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in term 0.
W20260709 12:26:12.515676 25382 leader_election.cc:343] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:12.516788 25457 leader_election.cc:304] T 18f5ed022d284752a9a35292d6adecb2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:12.518208 25700 tablet_bootstrap.cc:654] T 61d18172d49042bb9ec71317b75eb72c P 997fbd9d26514ae18ff1347929d8fa12: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:12.509855 25489 raft_consensus.cc:2468] T b3e8f6a25e95488b966e3229c04e03ef P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 997fbd9d26514ae18ff1347929d8fa12 in term 0.
I20260709 12:26:12.520463 25382 leader_election.cc:304] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 997fbd9d26514ae18ff1347929d8fa12, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 2128ac3598974a23ad037af1cd5003f4
I20260709 12:26:12.522133 25651 raft_consensus.cc:2804] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:12.522612 25651 raft_consensus.cc:493] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.522897 25696 raft_consensus.cc:2804] T 18f5ed022d284752a9a35292d6adecb2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:12.523631 25696 raft_consensus.cc:493] T 18f5ed022d284752a9a35292d6adecb2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.524151 25696 raft_consensus.cc:3060] T 18f5ed022d284752a9a35292d6adecb2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:12.523052 25651 raft_consensus.cc:3060] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:12.527256 25701 ts_tablet_manager.cc:1403] T ed736971b10844fbada6e03f39434740 P 2128ac3598974a23ad037af1cd5003f4: Time spent bootstrapping tablet: real 0.040s	user 0.011s	sys 0.004s
I20260709 12:26:12.531121 25701 raft_consensus.cc:359] T ed736971b10844fbada6e03f39434740 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.541810 25701 raft_consensus.cc:385] T ed736971b10844fbada6e03f39434740 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:12.542344 25701 raft_consensus.cc:740] T ed736971b10844fbada6e03f39434740 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2128ac3598974a23ad037af1cd5003f4, State: Initialized, Role: FOLLOWER
I20260709 12:26:12.543572 25701 consensus_queue.cc:260] T ed736971b10844fbada6e03f39434740 P 2128ac3598974a23ad037af1cd5003f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.540673 25696 raft_consensus.cc:515] T 18f5ed022d284752a9a35292d6adecb2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.544628 25651 raft_consensus.cc:515] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.548986 25489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3e8f6a25e95488b966e3229c04e03ef" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
I20260709 12:26:12.549537 25489 raft_consensus.cc:3060] T b3e8f6a25e95488b966e3229c04e03ef P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:12.550038 25701 ts_tablet_manager.cc:1434] T ed736971b10844fbada6e03f39434740 P 2128ac3598974a23ad037af1cd5003f4: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.003s
I20260709 12:26:12.550901 25701 tablet_bootstrap.cc:492] T c02e9bd829f8498b9af872415fd82d93 P 2128ac3598974a23ad037af1cd5003f4: Bootstrap starting.
I20260709 12:26:12.553900 25696 leader_election.cc:290] T 18f5ed022d284752a9a35292d6adecb2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 election: Requested vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:12.557411 25700 tablet_bootstrap.cc:492] T 61d18172d49042bb9ec71317b75eb72c P 997fbd9d26514ae18ff1347929d8fa12: No bootstrap required, opened a new log
I20260709 12:26:12.555433 25714 raft_consensus.cc:493] T 01c700f42bf34944ad7cad704c5f5f36 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.557811 25701 tablet_bootstrap.cc:654] T c02e9bd829f8498b9af872415fd82d93 P 2128ac3598974a23ad037af1cd5003f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:12.558243 25700 ts_tablet_manager.cc:1403] T 61d18172d49042bb9ec71317b75eb72c P 997fbd9d26514ae18ff1347929d8fa12: Time spent bootstrapping tablet: real 0.055s	user 0.011s	sys 0.027s
I20260709 12:26:12.558125 25714 raft_consensus.cc:515] T 01c700f42bf34944ad7cad704c5f5f36 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.560302 25714 leader_election.cc:290] T 01c700f42bf34944ad7cad704c5f5f36 P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:12.556495 25564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18f5ed022d284752a9a35292d6adecb2" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
I20260709 12:26:12.557492 25699 tablet_bootstrap.cc:492] T 9f66d125d0b24ecf8b2485004114dcd2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: No bootstrap required, opened a new log
I20260709 12:26:12.554517 25413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18f5ed022d284752a9a35292d6adecb2" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997fbd9d26514ae18ff1347929d8fa12"
I20260709 12:26:12.562387 25699 ts_tablet_manager.cc:1403] T 9f66d125d0b24ecf8b2485004114dcd2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent bootstrapping tablet: real 0.058s	user 0.025s	sys 0.020s
I20260709 12:26:12.562494 25413 raft_consensus.cc:3060] T 18f5ed022d284752a9a35292d6adecb2 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:12.561774 25491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c700f42bf34944ad7cad704c5f5f36" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" is_pre_election: true
I20260709 12:26:12.564772 25491 raft_consensus.cc:2468] T 01c700f42bf34944ad7cad704c5f5f36 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 997fbd9d26514ae18ff1347929d8fa12 in term 0.
I20260709 12:26:12.562885 25562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c700f42bf34944ad7cad704c5f5f36" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4" is_pre_election: true
I20260709 12:26:12.565246 25699 raft_consensus.cc:359] T 9f66d125d0b24ecf8b2485004114dcd2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.565964 25699 raft_consensus.cc:385] T 9f66d125d0b24ecf8b2485004114dcd2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:12.566045 25562 raft_consensus.cc:2468] T 01c700f42bf34944ad7cad704c5f5f36 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 997fbd9d26514ae18ff1347929d8fa12 in term 0.
I20260709 12:26:12.561754 25564 raft_consensus.cc:3060] T 18f5ed022d284752a9a35292d6adecb2 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:12.566273 25699 raft_consensus.cc:740] T 9f66d125d0b24ecf8b2485004114dcd2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Initialized, Role: FOLLOWER
I20260709 12:26:12.566754 25637 raft_consensus.cc:493] T 01c700f42bf34944ad7cad704c5f5f36 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.567049 25699 consensus_queue.cc:260] T 9f66d125d0b24ecf8b2485004114dcd2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.567281 25637 raft_consensus.cc:515] T 01c700f42bf34944ad7cad704c5f5f36 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.567396 25382 leader_election.cc:304] T 01c700f42bf34944ad7cad704c5f5f36 P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 997fbd9d26514ae18ff1347929d8fa12, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:12.569407 25637 leader_election.cc:290] T 01c700f42bf34944ad7cad704c5f5f36 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:12.569890 25637 raft_consensus.cc:493] T 61d18172d49042bb9ec71317b75eb72c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.570099 25714 raft_consensus.cc:2804] T 01c700f42bf34944ad7cad704c5f5f36 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:12.570338 25637 raft_consensus.cc:515] T 61d18172d49042bb9ec71317b75eb72c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.570729 25714 raft_consensus.cc:493] T 01c700f42bf34944ad7cad704c5f5f36 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.571226 25714 raft_consensus.cc:3060] T 01c700f42bf34944ad7cad704c5f5f36 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:12.573392 25562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c700f42bf34944ad7cad704c5f5f36" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4" is_pre_election: true
I20260709 12:26:12.574098 25562 raft_consensus.cc:2468] T 01c700f42bf34944ad7cad704c5f5f36 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in term 0.
I20260709 12:26:12.575381 25457 leader_election.cc:304] T 01c700f42bf34944ad7cad704c5f5f36 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:12.576997 25637 leader_election.cc:290] T 61d18172d49042bb9ec71317b75eb72c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:12.579599 25562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d18172d49042bb9ec71317b75eb72c" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4" is_pre_election: true
W20260709 12:26:12.584400 25457 leader_election.cc:343] T 61d18172d49042bb9ec71317b75eb72c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:12.586760 25564 raft_consensus.cc:2468] T 18f5ed022d284752a9a35292d6adecb2 P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in term 1.
I20260709 12:26:12.588609 25457 leader_election.cc:304] T 18f5ed022d284752a9a35292d6adecb2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:12.589658 25637 raft_consensus.cc:2804] T 18f5ed022d284752a9a35292d6adecb2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:12.590794 25637 raft_consensus.cc:697] T 18f5ed022d284752a9a35292d6adecb2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 LEADER]: Becoming Leader. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Running, Role: LEADER
I20260709 12:26:12.580109 25489 raft_consensus.cc:2468] T b3e8f6a25e95488b966e3229c04e03ef P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 997fbd9d26514ae18ff1347929d8fa12 in term 1.
I20260709 12:26:12.592568 25564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3e8f6a25e95488b966e3229c04e03ef" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
I20260709 12:26:12.592401 25637 consensus_queue.cc:237] T 18f5ed022d284752a9a35292d6adecb2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.584556 25696 raft_consensus.cc:2804] T 01c700f42bf34944ad7cad704c5f5f36 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:12.595824 25696 raft_consensus.cc:493] T 01c700f42bf34944ad7cad704c5f5f36 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.595530 25700 raft_consensus.cc:359] T 61d18172d49042bb9ec71317b75eb72c P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.596342 25696 raft_consensus.cc:3060] T 01c700f42bf34944ad7cad704c5f5f36 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:12.593761 25382 leader_election.cc:304] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 997fbd9d26514ae18ff1347929d8fa12, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:12.596396 25700 raft_consensus.cc:385] T 61d18172d49042bb9ec71317b75eb72c P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:12.598304 25700 raft_consensus.cc:740] T 61d18172d49042bb9ec71317b75eb72c P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Initialized, Role: FOLLOWER
I20260709 12:26:12.591171 25651 leader_election.cc:290] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 election: Requested vote from peers e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:12.590068 25413 raft_consensus.cc:2468] T 18f5ed022d284752a9a35292d6adecb2 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in term 1.
I20260709 12:26:12.603008 25651 raft_consensus.cc:2804] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:12.603549 25651 raft_consensus.cc:697] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 [term 1 LEADER]: Becoming Leader. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Running, Role: LEADER
I20260709 12:26:12.604677 25651 consensus_queue.cc:237] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.606537 25714 raft_consensus.cc:515] T 01c700f42bf34944ad7cad704c5f5f36 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.612691 25700 consensus_queue.cc:260] T 61d18172d49042bb9ec71317b75eb72c P 997fbd9d26514ae18ff1347929d8fa12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.614984 25714 leader_election.cc:290] T 01c700f42bf34944ad7cad704c5f5f36 P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 election: Requested vote from peers e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:12.615237 25700 ts_tablet_manager.cc:1434] T 61d18172d49042bb9ec71317b75eb72c P 997fbd9d26514ae18ff1347929d8fa12: Time spent starting tablet: real 0.056s	user 0.005s	sys 0.001s
I20260709 12:26:12.611588 25413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c700f42bf34944ad7cad704c5f5f36" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997fbd9d26514ae18ff1347929d8fa12" is_pre_election: true
I20260709 12:26:12.611661 25415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d18172d49042bb9ec71317b75eb72c" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997fbd9d26514ae18ff1347929d8fa12" is_pre_election: true
I20260709 12:26:12.616482 25415 raft_consensus.cc:2468] T 61d18172d49042bb9ec71317b75eb72c P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in term 0.
I20260709 12:26:12.617316 25637 raft_consensus.cc:493] T 9f66d125d0b24ecf8b2485004114dcd2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.617815 25637 raft_consensus.cc:515] T 9f66d125d0b24ecf8b2485004114dcd2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.619808 25637 leader_election.cc:290] T 9f66d125d0b24ecf8b2485004114dcd2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:12.621619 25700 tablet_bootstrap.cc:492] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12: Bootstrap starting.
I20260709 12:26:12.621918 25564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f66d125d0b24ecf8b2485004114dcd2" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4" is_pre_election: true
I20260709 12:26:12.622833 25415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f66d125d0b24ecf8b2485004114dcd2" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997fbd9d26514ae18ff1347929d8fa12" is_pre_election: true
W20260709 12:26:12.624662 25457 leader_election.cc:343] T 9f66d125d0b24ecf8b2485004114dcd2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:12.624769 25413 raft_consensus.cc:2393] T 01c700f42bf34944ad7cad704c5f5f36 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in current term 1: Already voted for candidate 997fbd9d26514ae18ff1347929d8fa12 in this term.
I20260709 12:26:12.625741 25457 leader_election.cc:304] T 61d18172d49042bb9ec71317b75eb72c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 997fbd9d26514ae18ff1347929d8fa12, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 2128ac3598974a23ad037af1cd5003f4
W20260709 12:26:12.627955 25457 leader_election.cc:343] T 9f66d125d0b24ecf8b2485004114dcd2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:12.628469 25457 leader_election.cc:304] T 9f66d125d0b24ecf8b2485004114dcd2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 2128ac3598974a23ad037af1cd5003f4, 997fbd9d26514ae18ff1347929d8fa12
I20260709 12:26:12.630544 25637 raft_consensus.cc:2804] T 61d18172d49042bb9ec71317b75eb72c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:12.631044 25637 raft_consensus.cc:493] T 61d18172d49042bb9ec71317b75eb72c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.631443 25637 raft_consensus.cc:3060] T 61d18172d49042bb9ec71317b75eb72c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:12.632419 25733 raft_consensus.cc:2749] T 9f66d125d0b24ecf8b2485004114dcd2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:12.639344 25311 catalog_manager.cc:5697] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "997fbd9d26514ae18ff1347929d8fa12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:12.641959 25700 tablet_bootstrap.cc:654] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:12.647624 25637 raft_consensus.cc:515] T 61d18172d49042bb9ec71317b75eb72c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.641618 25696 raft_consensus.cc:515] T 01c700f42bf34944ad7cad704c5f5f36 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
W20260709 12:26:12.639362 25382 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.58.2:45595, user_credentials={real_user=slave}} blocked reactor thread for 45701.5us
I20260709 12:26:12.653793 25696 leader_election.cc:290] T 01c700f42bf34944ad7cad704c5f5f36 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 election: Requested vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:12.657230 25415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d18172d49042bb9ec71317b75eb72c" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997fbd9d26514ae18ff1347929d8fa12"
I20260709 12:26:12.657953 25415 raft_consensus.cc:3060] T 61d18172d49042bb9ec71317b75eb72c P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:12.660185 25700 tablet_bootstrap.cc:492] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12: No bootstrap required, opened a new log
I20260709 12:26:12.661823 25700 ts_tablet_manager.cc:1403] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12: Time spent bootstrapping tablet: real 0.045s	user 0.012s	sys 0.003s
I20260709 12:26:12.663805 25562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c700f42bf34944ad7cad704c5f5f36" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
I20260709 12:26:12.664739 25562 raft_consensus.cc:3060] T 01c700f42bf34944ad7cad704c5f5f36 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:12.666617 25415 raft_consensus.cc:2468] T 61d18172d49042bb9ec71317b75eb72c P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in term 1.
I20260709 12:26:12.670887 25565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d18172d49042bb9ec71317b75eb72c" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
I20260709 12:26:12.671696 25457 leader_election.cc:304] T 61d18172d49042bb9ec71317b75eb72c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 997fbd9d26514ae18ff1347929d8fa12, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:12.673277 25696 raft_consensus.cc:2804] T 61d18172d49042bb9ec71317b75eb72c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:12.674355 25696 raft_consensus.cc:697] T 61d18172d49042bb9ec71317b75eb72c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 LEADER]: Becoming Leader. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Running, Role: LEADER
I20260709 12:26:12.675655 25696 consensus_queue.cc:237] T 61d18172d49042bb9ec71317b75eb72c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
W20260709 12:26:12.676873 25457 leader_election.cc:343] T 61d18172d49042bb9ec71317b75eb72c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:12.667753 25413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c700f42bf34944ad7cad704c5f5f36" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997fbd9d26514ae18ff1347929d8fa12"
I20260709 12:26:12.664723 25700 raft_consensus.cc:359] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.679567 25700 raft_consensus.cc:385] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:12.679890 25700 raft_consensus.cc:740] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Initialized, Role: FOLLOWER
I20260709 12:26:12.680655 25700 consensus_queue.cc:260] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.681375 25413 raft_consensus.cc:2393] T 01c700f42bf34944ad7cad704c5f5f36 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in current term 1: Already voted for candidate 997fbd9d26514ae18ff1347929d8fa12 in this term.
I20260709 12:26:12.682300 25700 ts_tablet_manager.cc:1434] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.000s
I20260709 12:26:12.679957 25637 leader_election.cc:290] T 61d18172d49042bb9ec71317b75eb72c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 election: Requested vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:12.683629 25700 tablet_bootstrap.cc:492] T 651a9ec9672842b6b0a0a9edb451ff33 P 997fbd9d26514ae18ff1347929d8fa12: Bootstrap starting.
W20260709 12:26:12.689574 25382 leader_election.cc:343] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:26:12.691088 25700 tablet_bootstrap.cc:654] T 651a9ec9672842b6b0a0a9edb451ff33 P 997fbd9d26514ae18ff1347929d8fa12: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:12.696815 25564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c700f42bf34944ad7cad704c5f5f36" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
I20260709 12:26:12.698704 25457 leader_election.cc:304] T 01c700f42bf34944ad7cad704c5f5f36 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 2128ac3598974a23ad037af1cd5003f4, 997fbd9d26514ae18ff1347929d8fa12
I20260709 12:26:12.701052 25696 raft_consensus.cc:2749] T 01c700f42bf34944ad7cad704c5f5f36 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:26:12.704074 25489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c700f42bf34944ad7cad704c5f5f36" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
I20260709 12:26:12.705394 25489 raft_consensus.cc:2393] T 01c700f42bf34944ad7cad704c5f5f36 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 997fbd9d26514ae18ff1347929d8fa12 in current term 1: Already voted for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in this term.
I20260709 12:26:12.706364 25700 tablet_bootstrap.cc:492] T 651a9ec9672842b6b0a0a9edb451ff33 P 997fbd9d26514ae18ff1347929d8fa12: No bootstrap required, opened a new log
I20260709 12:26:12.707106 25700 ts_tablet_manager.cc:1403] T 651a9ec9672842b6b0a0a9edb451ff33 P 997fbd9d26514ae18ff1347929d8fa12: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.004s
I20260709 12:26:12.710976 25700 raft_consensus.cc:359] T 651a9ec9672842b6b0a0a9edb451ff33 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.711876 25700 raft_consensus.cc:385] T 651a9ec9672842b6b0a0a9edb451ff33 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:12.711819 25313 catalog_manager.cc:5697] T 18f5ed022d284752a9a35292d6adecb2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee reported cstate change: term changed from 0 to 1, leader changed from <none> to e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2). New cstate: current_term: 1 leader_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:12.712986 25700 raft_consensus.cc:740] T 651a9ec9672842b6b0a0a9edb451ff33 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Initialized, Role: FOLLOWER
I20260709 12:26:12.713984 25700 consensus_queue.cc:260] T 651a9ec9672842b6b0a0a9edb451ff33 P 997fbd9d26514ae18ff1347929d8fa12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.716874 25700 ts_tablet_manager.cc:1434] T 651a9ec9672842b6b0a0a9edb451ff33 P 997fbd9d26514ae18ff1347929d8fa12: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.001s
I20260709 12:26:12.723167 25562 raft_consensus.cc:2468] T 01c700f42bf34944ad7cad704c5f5f36 P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 997fbd9d26514ae18ff1347929d8fa12 in term 1.
I20260709 12:26:12.724470 25382 leader_election.cc:304] T 01c700f42bf34944ad7cad704c5f5f36 P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, 997fbd9d26514ae18ff1347929d8fa12; no voters: e6f5fa5b38734da2a3ef8a7ce4fc6dee
I20260709 12:26:12.725250 25731 raft_consensus.cc:2804] T 01c700f42bf34944ad7cad704c5f5f36 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:12.725310 25701 tablet_bootstrap.cc:492] T c02e9bd829f8498b9af872415fd82d93 P 2128ac3598974a23ad037af1cd5003f4: No bootstrap required, opened a new log
I20260709 12:26:12.725770 25731 raft_consensus.cc:697] T 01c700f42bf34944ad7cad704c5f5f36 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 LEADER]: Becoming Leader. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Running, Role: LEADER
I20260709 12:26:12.725989 25701 ts_tablet_manager.cc:1403] T c02e9bd829f8498b9af872415fd82d93 P 2128ac3598974a23ad037af1cd5003f4: Time spent bootstrapping tablet: real 0.175s	user 0.041s	sys 0.032s
I20260709 12:26:12.726608 25731 consensus_queue.cc:237] T 01c700f42bf34944ad7cad704c5f5f36 P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.729321 25701 raft_consensus.cc:359] T c02e9bd829f8498b9af872415fd82d93 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.730103 25701 raft_consensus.cc:385] T c02e9bd829f8498b9af872415fd82d93 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:12.730408 25701 raft_consensus.cc:740] T c02e9bd829f8498b9af872415fd82d93 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2128ac3598974a23ad037af1cd5003f4, State: Initialized, Role: FOLLOWER
I20260709 12:26:12.730778 25729 raft_consensus.cc:493] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.731227 25701 consensus_queue.cc:260] T c02e9bd829f8498b9af872415fd82d93 P 2128ac3598974a23ad037af1cd5003f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.731292 25729 raft_consensus.cc:515] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.731315 25313 catalog_manager.cc:5697] T 61d18172d49042bb9ec71317b75eb72c P e6f5fa5b38734da2a3ef8a7ce4fc6dee reported cstate change: term changed from 0 to 1, leader changed from <none> to e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2). New cstate: current_term: 1 leader_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:12.729794 25699 ts_tablet_manager.cc:1434] T 9f66d125d0b24ecf8b2485004114dcd2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Time spent starting tablet: real 0.167s	user 0.039s	sys 0.085s
I20260709 12:26:12.734489 25729 leader_election.cc:290] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:12.736065 25413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c02e9bd829f8498b9af872415fd82d93" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997fbd9d26514ae18ff1347929d8fa12" is_pre_election: true
I20260709 12:26:12.736305 25562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c02e9bd829f8498b9af872415fd82d93" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4" is_pre_election: true
I20260709 12:26:12.736774 25413 raft_consensus.cc:2468] T c02e9bd829f8498b9af872415fd82d93 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in term 0.
I20260709 12:26:12.737083 25701 ts_tablet_manager.cc:1434] T c02e9bd829f8498b9af872415fd82d93 P 2128ac3598974a23ad037af1cd5003f4: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.004s
I20260709 12:26:12.737021 25562 raft_consensus.cc:2468] T c02e9bd829f8498b9af872415fd82d93 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in term 0.
I20260709 12:26:12.738265 25701 tablet_bootstrap.cc:492] T b3e8f6a25e95488b966e3229c04e03ef P 2128ac3598974a23ad037af1cd5003f4: Bootstrap starting.
I20260709 12:26:12.738256 25457 leader_election.cc:304] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:12.741952 25729 raft_consensus.cc:2804] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:12.742401 25729 raft_consensus.cc:493] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.742781 25729 raft_consensus.cc:3060] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:12.742858 25311 catalog_manager.cc:5697] T 01c700f42bf34944ad7cad704c5f5f36 P 997fbd9d26514ae18ff1347929d8fa12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "997fbd9d26514ae18ff1347929d8fa12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:12.745872 25701 tablet_bootstrap.cc:654] T b3e8f6a25e95488b966e3229c04e03ef P 2128ac3598974a23ad037af1cd5003f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:12.750928 25729 raft_consensus.cc:515] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.753480 25729 leader_election.cc:290] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 election: Requested vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:12.754415 25413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c02e9bd829f8498b9af872415fd82d93" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997fbd9d26514ae18ff1347929d8fa12"
I20260709 12:26:12.755093 25413 raft_consensus.cc:3060] T c02e9bd829f8498b9af872415fd82d93 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:12.756599 25562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c02e9bd829f8498b9af872415fd82d93" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
I20260709 12:26:12.757321 25562 raft_consensus.cc:3060] T c02e9bd829f8498b9af872415fd82d93 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:12.763370 25701 tablet_bootstrap.cc:492] T b3e8f6a25e95488b966e3229c04e03ef P 2128ac3598974a23ad037af1cd5003f4: No bootstrap required, opened a new log
I20260709 12:26:12.763990 25701 ts_tablet_manager.cc:1403] T b3e8f6a25e95488b966e3229c04e03ef P 2128ac3598974a23ad037af1cd5003f4: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.005s
I20260709 12:26:12.766822 25701 raft_consensus.cc:359] T b3e8f6a25e95488b966e3229c04e03ef P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.767373 25701 raft_consensus.cc:385] T b3e8f6a25e95488b966e3229c04e03ef P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:12.767623 25701 raft_consensus.cc:740] T b3e8f6a25e95488b966e3229c04e03ef P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2128ac3598974a23ad037af1cd5003f4, State: Initialized, Role: FOLLOWER
I20260709 12:26:12.768278 25701 consensus_queue.cc:260] T b3e8f6a25e95488b966e3229c04e03ef P 2128ac3598974a23ad037af1cd5003f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.763221 25413 raft_consensus.cc:2468] T c02e9bd829f8498b9af872415fd82d93 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in term 1.
I20260709 12:26:12.770326 25562 raft_consensus.cc:2468] T c02e9bd829f8498b9af872415fd82d93 P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in term 1.
I20260709 12:26:12.770694 25457 leader_election.cc:304] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 997fbd9d26514ae18ff1347929d8fa12, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:12.771215 25701 ts_tablet_manager.cc:1434] T b3e8f6a25e95488b966e3229c04e03ef P 2128ac3598974a23ad037af1cd5003f4: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:26:12.771967 25696 raft_consensus.cc:2804] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:12.772318 25701 tablet_bootstrap.cc:492] T 61d18172d49042bb9ec71317b75eb72c P 2128ac3598974a23ad037af1cd5003f4: Bootstrap starting.
I20260709 12:26:12.772539 25696 raft_consensus.cc:697] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 LEADER]: Becoming Leader. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Running, Role: LEADER
I20260709 12:26:12.773528 25696 consensus_queue.cc:237] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.780052 25701 tablet_bootstrap.cc:654] T 61d18172d49042bb9ec71317b75eb72c P 2128ac3598974a23ad037af1cd5003f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:12.784212 25311 catalog_manager.cc:5697] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee reported cstate change: term changed from 0 to 1, leader changed from <none> to e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2). New cstate: current_term: 1 leader_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:12.785673 25716 raft_consensus.cc:493] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.786103 25716 raft_consensus.cc:515] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.786954 25701 tablet_bootstrap.cc:492] T 61d18172d49042bb9ec71317b75eb72c P 2128ac3598974a23ad037af1cd5003f4: No bootstrap required, opened a new log
I20260709 12:26:12.787468 25701 ts_tablet_manager.cc:1403] T 61d18172d49042bb9ec71317b75eb72c P 2128ac3598974a23ad037af1cd5003f4: Time spent bootstrapping tablet: real 0.015s	user 0.005s	sys 0.007s
I20260709 12:26:12.787827 25716 leader_election.cc:290] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595)
I20260709 12:26:12.789549 25489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651a9ec9672842b6b0a0a9edb451ff33" candidate_uuid: "2128ac3598974a23ad037af1cd5003f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" is_pre_election: true
I20260709 12:26:12.789549 25413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651a9ec9672842b6b0a0a9edb451ff33" candidate_uuid: "2128ac3598974a23ad037af1cd5003f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997fbd9d26514ae18ff1347929d8fa12" is_pre_election: true
I20260709 12:26:12.790268 25701 raft_consensus.cc:359] T 61d18172d49042bb9ec71317b75eb72c P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.790333 25489 raft_consensus.cc:2468] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2128ac3598974a23ad037af1cd5003f4 in term 0.
I20260709 12:26:12.791121 25701 raft_consensus.cc:385] T 61d18172d49042bb9ec71317b75eb72c P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:12.791484 25701 raft_consensus.cc:740] T 61d18172d49042bb9ec71317b75eb72c P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2128ac3598974a23ad037af1cd5003f4, State: Initialized, Role: FOLLOWER
I20260709 12:26:12.792408 25531 leader_election.cc:304] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:12.792189 25701 consensus_queue.cc:260] T 61d18172d49042bb9ec71317b75eb72c P 2128ac3598974a23ad037af1cd5003f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.793320 25628 raft_consensus.cc:2804] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:12.793663 25628 raft_consensus.cc:493] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.794003 25628 raft_consensus.cc:3060] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:12.794435 25701 ts_tablet_manager.cc:1434] T 61d18172d49042bb9ec71317b75eb72c P 2128ac3598974a23ad037af1cd5003f4: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:26:12.795464 25701 tablet_bootstrap.cc:492] T 9f66d125d0b24ecf8b2485004114dcd2 P 2128ac3598974a23ad037af1cd5003f4: Bootstrap starting.
I20260709 12:26:12.790468 25413 raft_consensus.cc:2468] T 651a9ec9672842b6b0a0a9edb451ff33 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2128ac3598974a23ad037af1cd5003f4 in term 0.
I20260709 12:26:12.801968 25701 tablet_bootstrap.cc:654] T 9f66d125d0b24ecf8b2485004114dcd2 P 2128ac3598974a23ad037af1cd5003f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:26:12.802835 25628 raft_consensus.cc:515] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.805137 25628 leader_election.cc:290] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4 [CANDIDATE]: Term 1 election: Requested vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595)
I20260709 12:26:12.806301 25413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651a9ec9672842b6b0a0a9edb451ff33" candidate_uuid: "2128ac3598974a23ad037af1cd5003f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997fbd9d26514ae18ff1347929d8fa12"
I20260709 12:26:12.806644 25701 tablet_bootstrap.cc:492] T 9f66d125d0b24ecf8b2485004114dcd2 P 2128ac3598974a23ad037af1cd5003f4: No bootstrap required, opened a new log
I20260709 12:26:12.806840 25489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651a9ec9672842b6b0a0a9edb451ff33" candidate_uuid: "2128ac3598974a23ad037af1cd5003f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
I20260709 12:26:12.807123 25701 ts_tablet_manager.cc:1403] T 9f66d125d0b24ecf8b2485004114dcd2 P 2128ac3598974a23ad037af1cd5003f4: Time spent bootstrapping tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 12:26:12.806959 25413 raft_consensus.cc:3060] T 651a9ec9672842b6b0a0a9edb451ff33 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:12.807703 25489 raft_consensus.cc:3060] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:12.807749 25562 raft_consensus.cc:3060] T ed736971b10844fbada6e03f39434740 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:12.810381 25701 raft_consensus.cc:359] T 9f66d125d0b24ecf8b2485004114dcd2 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.811093 25701 raft_consensus.cc:385] T 9f66d125d0b24ecf8b2485004114dcd2 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:26:12.811367 25701 raft_consensus.cc:740] T 9f66d125d0b24ecf8b2485004114dcd2 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2128ac3598974a23ad037af1cd5003f4, State: Initialized, Role: FOLLOWER
I20260709 12:26:12.812119 25701 consensus_queue.cc:260] T 9f66d125d0b24ecf8b2485004114dcd2 P 2128ac3598974a23ad037af1cd5003f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.814400 25701 ts_tablet_manager.cc:1434] T 9f66d125d0b24ecf8b2485004114dcd2 P 2128ac3598974a23ad037af1cd5003f4: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:26:12.815486 25489 raft_consensus.cc:2468] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2128ac3598974a23ad037af1cd5003f4 in term 1.
I20260709 12:26:12.816704 25413 raft_consensus.cc:2468] T 651a9ec9672842b6b0a0a9edb451ff33 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2128ac3598974a23ad037af1cd5003f4 in term 1.
I20260709 12:26:12.816983 25531 leader_election.cc:304] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:12.817831 25731 consensus_queue.cc:1048] T ed736971b10844fbada6e03f39434740 P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:12.818096 25628 raft_consensus.cc:2804] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:12.819293 25628 raft_consensus.cc:697] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4 [term 1 LEADER]: Becoming Leader. State: Replica: 2128ac3598974a23ad037af1cd5003f4, State: Running, Role: LEADER
I20260709 12:26:12.820180 25628 consensus_queue.cc:237] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.832962 25311 catalog_manager.cc:5697] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3). New cstate: current_term: 1 leader_uuid: "2128ac3598974a23ad037af1cd5003f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:12.834666 25731 consensus_queue.cc:1048] T ed736971b10844fbada6e03f39434740 P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:12.899233 25628 raft_consensus.cc:493] T b3e8f6a25e95488b966e3229c04e03ef P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.899899 25628 raft_consensus.cc:515] T b3e8f6a25e95488b966e3229c04e03ef P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:12.901911 25628 leader_election.cc:290] T b3e8f6a25e95488b966e3229c04e03ef P 2128ac3598974a23ad037af1cd5003f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595)
I20260709 12:26:12.902990 25489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3e8f6a25e95488b966e3229c04e03ef" candidate_uuid: "2128ac3598974a23ad037af1cd5003f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" is_pre_election: true
I20260709 12:26:12.902819 25413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3e8f6a25e95488b966e3229c04e03ef" candidate_uuid: "2128ac3598974a23ad037af1cd5003f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997fbd9d26514ae18ff1347929d8fa12" is_pre_election: true
I20260709 12:26:12.903872 25489 raft_consensus.cc:2393] T b3e8f6a25e95488b966e3229c04e03ef P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2128ac3598974a23ad037af1cd5003f4 in current term 1: Already voted for candidate 997fbd9d26514ae18ff1347929d8fa12 in this term.
I20260709 12:26:12.905577 25531 leader_election.cc:304] T b3e8f6a25e95488b966e3229c04e03ef P 2128ac3598974a23ad037af1cd5003f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4; no voters: 997fbd9d26514ae18ff1347929d8fa12, e6f5fa5b38734da2a3ef8a7ce4fc6dee
I20260709 12:26:12.906716 25628 raft_consensus.cc:3060] T b3e8f6a25e95488b966e3229c04e03ef P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:12.914176 25628 raft_consensus.cc:2749] T b3e8f6a25e95488b966e3229c04e03ef P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:12.914647 25696 consensus_queue.cc:1048] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:12.926412 25733 consensus_queue.cc:1048] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:13.084246 25728 consensus_queue.cc:1048] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:13.095096 25730 consensus_queue.cc:1048] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:13.104195 25733 consensus_queue.cc:1048] T 18f5ed022d284752a9a35292d6adecb2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:13.116888 25628 raft_consensus.cc:493] T 61d18172d49042bb9ec71317b75eb72c P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.117550 25628 raft_consensus.cc:515] T 61d18172d49042bb9ec71317b75eb72c P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:13.119767 25628 leader_election.cc:290] T 61d18172d49042bb9ec71317b75eb72c P 2128ac3598974a23ad037af1cd5003f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595)
I20260709 12:26:13.121770 25489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d18172d49042bb9ec71317b75eb72c" candidate_uuid: "2128ac3598974a23ad037af1cd5003f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" is_pre_election: true
I20260709 12:26:13.122066 25733 consensus_queue.cc:1048] T 18f5ed022d284752a9a35292d6adecb2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:13.121873 25413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61d18172d49042bb9ec71317b75eb72c" candidate_uuid: "2128ac3598974a23ad037af1cd5003f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "997fbd9d26514ae18ff1347929d8fa12" is_pre_election: true
I20260709 12:26:13.124795 25413 raft_consensus.cc:2393] T 61d18172d49042bb9ec71317b75eb72c P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2128ac3598974a23ad037af1cd5003f4 in current term 1: Already voted for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in this term.
I20260709 12:26:13.126041 25531 leader_election.cc:304] T 61d18172d49042bb9ec71317b75eb72c P 2128ac3598974a23ad037af1cd5003f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4; no voters: 997fbd9d26514ae18ff1347929d8fa12, e6f5fa5b38734da2a3ef8a7ce4fc6dee
I20260709 12:26:13.127518 25628 raft_consensus.cc:3060] T 61d18172d49042bb9ec71317b75eb72c P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:13.136708 25628 raft_consensus.cc:2749] T 61d18172d49042bb9ec71317b75eb72c P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:26:13.148427 25730 raft_consensus.cc:493] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.148965 25730 raft_consensus.cc:515] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:13.150918 25730 leader_election.cc:290] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:13.151389 25489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f66d125d0b24ecf8b2485004114dcd2" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" is_pre_election: true
I20260709 12:26:13.151875 25562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f66d125d0b24ecf8b2485004114dcd2" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4" is_pre_election: true
I20260709 12:26:13.152053 25489 raft_consensus.cc:2468] T 9f66d125d0b24ecf8b2485004114dcd2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 997fbd9d26514ae18ff1347929d8fa12 in term 0.
I20260709 12:26:13.152524 25562 raft_consensus.cc:2468] T 9f66d125d0b24ecf8b2485004114dcd2 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 997fbd9d26514ae18ff1347929d8fa12 in term 0.
I20260709 12:26:13.153116 25382 leader_election.cc:304] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 997fbd9d26514ae18ff1347929d8fa12, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:13.153813 25730 raft_consensus.cc:2804] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:26:13.154160 25730 raft_consensus.cc:493] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.154466 25730 raft_consensus.cc:3060] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:13.159257 25733 consensus_queue.cc:1048] T 61d18172d49042bb9ec71317b75eb72c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:13.160421 25728 consensus_queue.cc:1048] T 01c700f42bf34944ad7cad704c5f5f36 P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:13.163014 25730 raft_consensus.cc:515] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:13.165783 25730 leader_election.cc:290] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 election: Requested vote from peers e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:13.166422 25491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f66d125d0b24ecf8b2485004114dcd2" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
I20260709 12:26:13.167148 25491 raft_consensus.cc:3060] T 9f66d125d0b24ecf8b2485004114dcd2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:13.176048 25491 raft_consensus.cc:2468] T 9f66d125d0b24ecf8b2485004114dcd2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 997fbd9d26514ae18ff1347929d8fa12 in term 1.
I20260709 12:26:13.177166 25564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f66d125d0b24ecf8b2485004114dcd2" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
I20260709 12:26:13.177436 25382 leader_election.cc:304] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 997fbd9d26514ae18ff1347929d8fa12, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:13.177870 25564 raft_consensus.cc:3060] T 9f66d125d0b24ecf8b2485004114dcd2 P 2128ac3598974a23ad037af1cd5003f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:26:13.178505 25730 raft_consensus.cc:2804] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:26:13.179006 25730 raft_consensus.cc:697] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 LEADER]: Becoming Leader. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Running, Role: LEADER
W20260709 12:26:13.180053 25362 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:26:13.179980 25730 consensus_queue.cc:237] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:13.185551 25564 raft_consensus.cc:2468] T 9f66d125d0b24ecf8b2485004114dcd2 P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 997fbd9d26514ae18ff1347929d8fa12 in term 1.
I20260709 12:26:13.189841 25730 consensus_queue.cc:1048] T 01c700f42bf34944ad7cad704c5f5f36 P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:13.192627 25313 catalog_manager.cc:5697] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1). New cstate: current_term: 1 leader_uuid: "997fbd9d26514ae18ff1347929d8fa12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:13.194204 25733 consensus_queue.cc:1048] T 61d18172d49042bb9ec71317b75eb72c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:13.242523 25733 consensus_queue.cc:1048] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:13.244944 25733 consensus_queue.cc:1048] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:13.253988 25489 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca1ffddacb9444a09baf22ab222ff39c"
dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
mode: GRACEFUL
new_leader_uuid: "997fbd9d26514ae18ff1347929d8fa12"
 from {username='slave'} at 127.0.0.1:49282
I20260709 12:26:13.254837 25489 raft_consensus.cc:606] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 LEADER]: Received request to transfer leadership to TS 997fbd9d26514ae18ff1347929d8fa12
I20260709 12:26:13.273810 25489 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c02e9bd829f8498b9af872415fd82d93"
dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
mode: GRACEFUL
new_leader_uuid: "997fbd9d26514ae18ff1347929d8fa12"
 from {username='slave'} at 127.0.0.1:49282
I20260709 12:26:13.274415 25489 raft_consensus.cc:606] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 LEADER]: Received request to transfer leadership to TS 997fbd9d26514ae18ff1347929d8fa12
I20260709 12:26:13.283399 25562 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "651a9ec9672842b6b0a0a9edb451ff33"
dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
mode: GRACEFUL
new_leader_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
 from {username='slave'} at 127.0.0.1:45852
I20260709 12:26:13.283939 25562 raft_consensus.cc:606] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4 [term 1 LEADER]: Received request to transfer leadership to TS e6f5fa5b38734da2a3ef8a7ce4fc6dee
I20260709 12:26:13.287927 25628 consensus_queue.cc:1048] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:13.295064 25415 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3e8f6a25e95488b966e3229c04e03ef"
dest_uuid: "997fbd9d26514ae18ff1347929d8fa12"
mode: GRACEFUL
new_leader_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
 from {username='slave'} at 127.0.0.1:43598
I20260709 12:26:13.295704 25415 raft_consensus.cc:606] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 [term 1 LEADER]: Received request to transfer leadership to TS e6f5fa5b38734da2a3ef8a7ce4fc6dee
I20260709 12:26:13.301462 25716 consensus_queue.cc:1048] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:13.310745 25415 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f66d125d0b24ecf8b2485004114dcd2"
dest_uuid: "997fbd9d26514ae18ff1347929d8fa12"
mode: GRACEFUL
new_leader_uuid: "2128ac3598974a23ad037af1cd5003f4"
 from {username='slave'} at 127.0.0.1:43598
I20260709 12:26:13.311291 25415 raft_consensus.cc:606] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 LEADER]: Received request to transfer leadership to TS 2128ac3598974a23ad037af1cd5003f4
I20260709 12:26:13.313104 25716 raft_consensus.cc:993] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4: : Instructing follower e6f5fa5b38734da2a3ef8a7ce4fc6dee to start an election
I20260709 12:26:13.313607 25743 raft_consensus.cc:1081] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4 [term 1 LEADER]: Signalling peer e6f5fa5b38734da2a3ef8a7ce4fc6dee to start an election
I20260709 12:26:13.315210 25489 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "651a9ec9672842b6b0a0a9edb451ff33"
dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
 from {username='slave'} at 127.0.0.1:49264
I20260709 12:26:13.315809 25489 raft_consensus.cc:493] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:13.316184 25489 raft_consensus.cc:3060] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:13.321695 25489 raft_consensus.cc:515] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:13.323493 25489 leader_election.cc:290] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 2 election: Requested vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:13.324982 25415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651a9ec9672842b6b0a0a9edb451ff33" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "997fbd9d26514ae18ff1347929d8fa12"
I20260709 12:26:13.325770 25415 raft_consensus.cc:3060] T 651a9ec9672842b6b0a0a9edb451ff33 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:13.331856 25415 raft_consensus.cc:2468] T 651a9ec9672842b6b0a0a9edb451ff33 P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in term 2.
I20260709 12:26:13.333299 25457 leader_election.cc:304] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 997fbd9d26514ae18ff1347929d8fa12, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:13.333580 25562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "651a9ec9672842b6b0a0a9edb451ff33" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
I20260709 12:26:13.334075 25733 raft_consensus.cc:2804] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:13.334236 25562 raft_consensus.cc:3055] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:13.334576 25733 raft_consensus.cc:697] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 LEADER]: Becoming Leader. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Running, Role: LEADER
I20260709 12:26:13.334602 25562 raft_consensus.cc:740] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2128ac3598974a23ad037af1cd5003f4, State: Running, Role: LEADER
I20260709 12:26:13.335718 25562 consensus_queue.cc:260] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:13.335670 25733 consensus_queue.cc:237] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:13.337379 25562 raft_consensus.cc:3060] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:13.344136 25562 raft_consensus.cc:2468] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in term 2.
I20260709 12:26:13.346289 25312 catalog_manager.cc:5697] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee reported cstate change: term changed from 1 to 2, leader changed from 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3) to e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2). New cstate: current_term: 2 leader_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:13.535624 25637 raft_consensus.cc:993] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee: : Instructing follower 997fbd9d26514ae18ff1347929d8fa12 to start an election
I20260709 12:26:13.536125 25733 raft_consensus.cc:1081] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 LEADER]: Signalling peer 997fbd9d26514ae18ff1347929d8fa12 to start an election
I20260709 12:26:13.537577 25415 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ca1ffddacb9444a09baf22ab222ff39c"
dest_uuid: "997fbd9d26514ae18ff1347929d8fa12"
 from {username='slave'} at 127.0.0.1:43588
I20260709 12:26:13.538007 25415 raft_consensus.cc:493] T ca1ffddacb9444a09baf22ab222ff39c P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:13.538290 25415 raft_consensus.cc:3060] T ca1ffddacb9444a09baf22ab222ff39c P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:13.543795 25415 raft_consensus.cc:515] T ca1ffddacb9444a09baf22ab222ff39c P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:13.545764 25415 leader_election.cc:290] T ca1ffddacb9444a09baf22ab222ff39c P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 2 election: Requested vote from peers e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:13.546960 25489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca1ffddacb9444a09baf22ab222ff39c" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
I20260709 12:26:13.547086 25562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca1ffddacb9444a09baf22ab222ff39c" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
I20260709 12:26:13.547523 25489 raft_consensus.cc:3055] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:13.547612 25562 raft_consensus.cc:3060] T ca1ffddacb9444a09baf22ab222ff39c P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:13.547850 25489 raft_consensus.cc:740] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Running, Role: LEADER
I20260709 12:26:13.548626 25489 consensus_queue.cc:260] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:13.549535 25489 raft_consensus.cc:3060] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:13.552486 25562 raft_consensus.cc:2468] T ca1ffddacb9444a09baf22ab222ff39c P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 997fbd9d26514ae18ff1347929d8fa12 in term 2.
I20260709 12:26:13.553674 25382 leader_election.cc:304] T ca1ffddacb9444a09baf22ab222ff39c P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, 997fbd9d26514ae18ff1347929d8fa12; no voters: 
I20260709 12:26:13.554111 25489 raft_consensus.cc:2468] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 997fbd9d26514ae18ff1347929d8fa12 in term 2.
I20260709 12:26:13.554559 25728 raft_consensus.cc:2804] T ca1ffddacb9444a09baf22ab222ff39c P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:13.555083 25728 raft_consensus.cc:697] T ca1ffddacb9444a09baf22ab222ff39c P 997fbd9d26514ae18ff1347929d8fa12 [term 2 LEADER]: Becoming Leader. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Running, Role: LEADER
I20260709 12:26:13.555900 25728 consensus_queue.cc:237] T ca1ffddacb9444a09baf22ab222ff39c P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:13.562922 25312 catalog_manager.cc:5697] T ca1ffddacb9444a09baf22ab222ff39c P 997fbd9d26514ae18ff1347929d8fa12 reported cstate change: term changed from 1 to 2, leader changed from e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2) to 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1). New cstate: current_term: 2 leader_uuid: "997fbd9d26514ae18ff1347929d8fa12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:13.626416 25728 raft_consensus.cc:993] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12: : Instructing follower e6f5fa5b38734da2a3ef8a7ce4fc6dee to start an election
I20260709 12:26:13.626878 25730 raft_consensus.cc:1081] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 [term 1 LEADER]: Signalling peer e6f5fa5b38734da2a3ef8a7ce4fc6dee to start an election
I20260709 12:26:13.628471 25489 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b3e8f6a25e95488b966e3229c04e03ef"
dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
 from {username='slave'} at 127.0.0.1:49270
I20260709 12:26:13.629128 25489 raft_consensus.cc:493] T b3e8f6a25e95488b966e3229c04e03ef P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:13.629494 25489 raft_consensus.cc:3060] T b3e8f6a25e95488b966e3229c04e03ef P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:13.632040 25730 consensus_queue.cc:1048] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:13.635732 25489 raft_consensus.cc:515] T b3e8f6a25e95488b966e3229c04e03ef P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:13.637677 25489 leader_election.cc:290] T b3e8f6a25e95488b966e3229c04e03ef P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 2 election: Requested vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:13.638813 25415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3e8f6a25e95488b966e3229c04e03ef" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "997fbd9d26514ae18ff1347929d8fa12"
I20260709 12:26:13.639541 25415 raft_consensus.cc:3055] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:13.639734 25728 raft_consensus.cc:993] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12: : Instructing follower 2128ac3598974a23ad037af1cd5003f4 to start an election
I20260709 12:26:13.639906 25415 raft_consensus.cc:740] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Running, Role: LEADER
I20260709 12:26:13.640300 25731 raft_consensus.cc:1081] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 LEADER]: Signalling peer 2128ac3598974a23ad037af1cd5003f4 to start an election
I20260709 12:26:13.641016 25564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3e8f6a25e95488b966e3229c04e03ef" candidate_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
I20260709 12:26:13.643680 25564 raft_consensus.cc:3060] T b3e8f6a25e95488b966e3229c04e03ef P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:13.640878 25415 consensus_queue.cc:260] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:13.647197 25415 raft_consensus.cc:3060] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:13.642509 25562 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9f66d125d0b24ecf8b2485004114dcd2"
dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
 from {username='slave'} at 127.0.0.1:45838
I20260709 12:26:13.648627 25562 raft_consensus.cc:493] T 9f66d125d0b24ecf8b2485004114dcd2 P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:13.648996 25562 raft_consensus.cc:3060] T 9f66d125d0b24ecf8b2485004114dcd2 P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:13.651436 25564 raft_consensus.cc:2468] T b3e8f6a25e95488b966e3229c04e03ef P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in term 2.
I20260709 12:26:13.655542 25562 raft_consensus.cc:515] T 9f66d125d0b24ecf8b2485004114dcd2 P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:13.656711 25565 raft_consensus.cc:1240] T 9f66d125d0b24ecf8b2485004114dcd2 P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Rejecting Update request from peer 997fbd9d26514ae18ff1347929d8fa12 for earlier term 1. Current term is 2. Ops: []
I20260709 12:26:13.658618 25415 raft_consensus.cc:2468] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6f5fa5b38734da2a3ef8a7ce4fc6dee in term 2.
I20260709 12:26:13.658752 25731 consensus_queue.cc:1048] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:26:13.660307 25731 consensus_queue.cc:1059] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:26:13.658670 25457 leader_election.cc:304] T b3e8f6a25e95488b966e3229c04e03ef P e6f5fa5b38734da2a3ef8a7ce4fc6dee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:13.661372 25731 raft_consensus.cc:3055] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:13.661824 25731 raft_consensus.cc:740] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Running, Role: LEADER
I20260709 12:26:13.662212 25729 raft_consensus.cc:2804] T b3e8f6a25e95488b966e3229c04e03ef P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:13.662170 25415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f66d125d0b24ecf8b2485004114dcd2" candidate_uuid: "2128ac3598974a23ad037af1cd5003f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "997fbd9d26514ae18ff1347929d8fa12"
I20260709 12:26:13.662907 25729 raft_consensus.cc:697] T b3e8f6a25e95488b966e3229c04e03ef P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 LEADER]: Becoming Leader. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Running, Role: LEADER
I20260709 12:26:13.662717 25731 consensus_queue.cc:260] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:13.663718 25731 raft_consensus.cc:3060] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:13.663800 25729 consensus_queue.cc:237] T b3e8f6a25e95488b966e3229c04e03ef P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:13.664587 25490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f66d125d0b24ecf8b2485004114dcd2" candidate_uuid: "2128ac3598974a23ad037af1cd5003f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
I20260709 12:26:13.666441 25562 leader_election.cc:290] T 9f66d125d0b24ecf8b2485004114dcd2 P 2128ac3598974a23ad037af1cd5003f4 [CANDIDATE]: Term 2 election: Requested vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595)
I20260709 12:26:13.677868 25311 catalog_manager.cc:5697] T b3e8f6a25e95488b966e3229c04e03ef P e6f5fa5b38734da2a3ef8a7ce4fc6dee reported cstate change: term changed from 1 to 2, leader changed from 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1) to e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2). New cstate: current_term: 2 leader_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:13.678900 25415 raft_consensus.cc:2468] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2128ac3598974a23ad037af1cd5003f4 in term 2.
I20260709 12:26:13.680960 25531 leader_election.cc:304] T 9f66d125d0b24ecf8b2485004114dcd2 P 2128ac3598974a23ad037af1cd5003f4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, 997fbd9d26514ae18ff1347929d8fa12; no voters: e6f5fa5b38734da2a3ef8a7ce4fc6dee
I20260709 12:26:13.682008 25743 raft_consensus.cc:2804] T 9f66d125d0b24ecf8b2485004114dcd2 P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:13.682516 25743 raft_consensus.cc:697] T 9f66d125d0b24ecf8b2485004114dcd2 P 2128ac3598974a23ad037af1cd5003f4 [term 2 LEADER]: Becoming Leader. State: Replica: 2128ac3598974a23ad037af1cd5003f4, State: Running, Role: LEADER
I20260709 12:26:13.683265 25743 consensus_queue.cc:237] T 9f66d125d0b24ecf8b2485004114dcd2 P 2128ac3598974a23ad037af1cd5003f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:13.692103 25312 catalog_manager.cc:5697] T 9f66d125d0b24ecf8b2485004114dcd2 P 2128ac3598974a23ad037af1cd5003f4 reported cstate change: term changed from 1 to 2, leader changed from 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1) to 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3). New cstate: current_term: 2 leader_uuid: "2128ac3598974a23ad037af1cd5003f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:13.893679 25696 raft_consensus.cc:993] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: : Instructing follower 997fbd9d26514ae18ff1347929d8fa12 to start an election
I20260709 12:26:13.894428 25696 raft_consensus.cc:1081] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 LEADER]: Signalling peer 997fbd9d26514ae18ff1347929d8fa12 to start an election
I20260709 12:26:13.896270 25415 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c02e9bd829f8498b9af872415fd82d93"
dest_uuid: "997fbd9d26514ae18ff1347929d8fa12"
 from {username='slave'} at 127.0.0.1:43588
I20260709 12:26:13.896843 25415 raft_consensus.cc:493] T c02e9bd829f8498b9af872415fd82d93 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:13.897157 25415 raft_consensus.cc:3060] T c02e9bd829f8498b9af872415fd82d93 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:13.902532 25415 raft_consensus.cc:515] T c02e9bd829f8498b9af872415fd82d93 P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:13.904219 25415 leader_election.cc:290] T c02e9bd829f8498b9af872415fd82d93 P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 2 election: Requested vote from peers e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595), 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3:37109)
I20260709 12:26:13.905563 25491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c02e9bd829f8498b9af872415fd82d93" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
I20260709 12:26:13.905769 25562 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c02e9bd829f8498b9af872415fd82d93" candidate_uuid: "997fbd9d26514ae18ff1347929d8fa12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
I20260709 12:26:13.906373 25491 raft_consensus.cc:3055] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:13.906418 25562 raft_consensus.cc:3060] T c02e9bd829f8498b9af872415fd82d93 P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:13.906793 25491 raft_consensus.cc:740] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Running, Role: LEADER
I20260709 12:26:13.907790 25491 consensus_queue.cc:260] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:13.909179 25491 raft_consensus.cc:3060] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:13.912959 25562 raft_consensus.cc:2468] T c02e9bd829f8498b9af872415fd82d93 P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 997fbd9d26514ae18ff1347929d8fa12 in term 2.
I20260709 12:26:13.914018 25382 leader_election.cc:304] T c02e9bd829f8498b9af872415fd82d93 P 997fbd9d26514ae18ff1347929d8fa12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, 997fbd9d26514ae18ff1347929d8fa12; no voters: 
I20260709 12:26:13.914781 25731 raft_consensus.cc:2804] T c02e9bd829f8498b9af872415fd82d93 P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:13.915315 25731 raft_consensus.cc:697] T c02e9bd829f8498b9af872415fd82d93 P 997fbd9d26514ae18ff1347929d8fa12 [term 2 LEADER]: Becoming Leader. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Running, Role: LEADER
I20260709 12:26:13.915318 25491 raft_consensus.cc:2468] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 997fbd9d26514ae18ff1347929d8fa12 in term 2.
I20260709 12:26:13.916229 25731 consensus_queue.cc:237] T c02e9bd829f8498b9af872415fd82d93 P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:13.917060 25562 raft_consensus.cc:1275] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Refusing update from remote peer e6f5fa5b38734da2a3ef8a7ce4fc6dee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:13.918258 25696 consensus_queue.cc:1048] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:13.924963 25313 catalog_manager.cc:5697] T c02e9bd829f8498b9af872415fd82d93 P 997fbd9d26514ae18ff1347929d8fa12 reported cstate change: term changed from 1 to 2, leader changed from e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2) to 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1). New cstate: current_term: 2 leader_uuid: "997fbd9d26514ae18ff1347929d8fa12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:13.931793 25415 raft_consensus.cc:1275] T 651a9ec9672842b6b0a0a9edb451ff33 P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Refusing update from remote peer e6f5fa5b38734da2a3ef8a7ce4fc6dee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:13.933871 25637 consensus_queue.cc:1048] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:14.073421 25491 raft_consensus.cc:1275] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Refusing update from remote peer 997fbd9d26514ae18ff1347929d8fa12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:14.076974 25730 consensus_queue.cc:1048] T ca1ffddacb9444a09baf22ab222ff39c P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:14.095237 25562 raft_consensus.cc:1275] T ca1ffddacb9444a09baf22ab222ff39c P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Refusing update from remote peer 997fbd9d26514ae18ff1347929d8fa12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:14.097342 25731 consensus_queue.cc:1048] T ca1ffddacb9444a09baf22ab222ff39c P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:14.155562 25415 raft_consensus.cc:1275] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Refusing update from remote peer 2128ac3598974a23ad037af1cd5003f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:14.157500 25743 consensus_queue.cc:1048] T 9f66d125d0b24ecf8b2485004114dcd2 P 2128ac3598974a23ad037af1cd5003f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:14.170847 25491 raft_consensus.cc:3060] T 9f66d125d0b24ecf8b2485004114dcd2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:14.183184 25491 raft_consensus.cc:1275] T 9f66d125d0b24ecf8b2485004114dcd2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Refusing update from remote peer 2128ac3598974a23ad037af1cd5003f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:14.184864 25772 consensus_queue.cc:1048] T 9f66d125d0b24ecf8b2485004114dcd2 P 2128ac3598974a23ad037af1cd5003f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:14.187215 25562 raft_consensus.cc:1275] T b3e8f6a25e95488b966e3229c04e03ef P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Refusing update from remote peer e6f5fa5b38734da2a3ef8a7ce4fc6dee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:14.189246 25637 consensus_queue.cc:1048] T b3e8f6a25e95488b966e3229c04e03ef P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:14.217322 25413 raft_consensus.cc:1275] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Refusing update from remote peer e6f5fa5b38734da2a3ef8a7ce4fc6dee: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:14.219205 25637 consensus_queue.cc:1048] T b3e8f6a25e95488b966e3229c04e03ef P e6f5fa5b38734da2a3ef8a7ce4fc6dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:14.420823 25562 raft_consensus.cc:1275] T c02e9bd829f8498b9af872415fd82d93 P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Refusing update from remote peer 997fbd9d26514ae18ff1347929d8fa12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:14.422667 25731 consensus_queue.cc:1048] T c02e9bd829f8498b9af872415fd82d93 P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:14.438025 25491 raft_consensus.cc:1275] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Refusing update from remote peer 997fbd9d26514ae18ff1347929d8fa12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:14.439970 25730 consensus_queue.cc:1048] T c02e9bd829f8498b9af872415fd82d93 P 997fbd9d26514ae18ff1347929d8fa12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:16.315282 25312 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37640:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:26:16.340727 25312 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37640:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:26:16.496222  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:16.556721 25804 tablet.cc:1724] T ca1ffddacb9444a09baf22ab222ff39c P 997fbd9d26514ae18ff1347929d8fa12: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:16.617266 25814 tablet.cc:1724] T 18f5ed022d284752a9a35292d6adecb2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:16.643200 25820 tablet.cc:1724] T ca1ffddacb9444a09baf22ab222ff39c P 2128ac3598974a23ad037af1cd5003f4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:26:16.626037 25457 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.58.3:37109, user_credentials={real_user=slave}} blocked reactor thread for 65379.7us
I20260709 12:26:16.761629 25829 tablet.cc:1724] T ed736971b10844fbada6e03f39434740 P 2128ac3598974a23ad037af1cd5003f4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:16.752559 25822 tablet.cc:1724] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:16.837668 25829 tablet.cc:1724] T 01c700f42bf34944ad7cad704c5f5f36 P 2128ac3598974a23ad037af1cd5003f4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:16.839536 25832 tablet.cc:1724] T 01c700f42bf34944ad7cad704c5f5f36 P 997fbd9d26514ae18ff1347929d8fa12: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:16.906908 25830 tablet.cc:1724] T ed736971b10844fbada6e03f39434740 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:16.918365 25843 tablet.cc:1724] T 18f5ed022d284752a9a35292d6adecb2 P 997fbd9d26514ae18ff1347929d8fa12: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:16.918735 25842 tablet.cc:1724] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:16.938416 25837 tablet.cc:1724] T 18f5ed022d284752a9a35292d6adecb2 P 2128ac3598974a23ad037af1cd5003f4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:16.940927 25835 tablet.cc:1724] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:16.942567 25820 tablet.cc:1724] T 9f66d125d0b24ecf8b2485004114dcd2 P 2128ac3598974a23ad037af1cd5003f4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:16.954999 25413 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20fa2b8266614739b49be1cc8c6ee07f"
dest_uuid: "997fbd9d26514ae18ff1347929d8fa12"
mode: GRACEFUL
new_leader_uuid: "2128ac3598974a23ad037af1cd5003f4"
 from {username='slave'} at 127.0.0.1:35576
I20260709 12:26:16.955971 25413 raft_consensus.cc:606] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12 [term 2 LEADER]: Received request to transfer leadership to TS 2128ac3598974a23ad037af1cd5003f4
I20260709 12:26:16.968887 25814 tablet.cc:1724] T b3e8f6a25e95488b966e3229c04e03ef P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:26:16.983696 25840 log.cc:927] Time spent T b3e8f6a25e95488b966e3229c04e03ef P 2128ac3598974a23ad037af1cd5003f4: Append to log took a long time: real 0.078s	user 0.000s	sys 0.002s
I20260709 12:26:16.984977 25804 tablet.cc:1724] T c02e9bd829f8498b9af872415fd82d93 P 997fbd9d26514ae18ff1347929d8fa12: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:16.991250 25837 tablet.cc:1724] T 61d18172d49042bb9ec71317b75eb72c P 2128ac3598974a23ad037af1cd5003f4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:16.984561 25841 tablet.cc:1724] T ed736971b10844fbada6e03f39434740 P 997fbd9d26514ae18ff1347929d8fa12: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:16.986640 25820 tablet.cc:1724] T c02e9bd829f8498b9af872415fd82d93 P 2128ac3598974a23ad037af1cd5003f4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:17.007745 25835 tablet.cc:1724] T b3e8f6a25e95488b966e3229c04e03ef P 2128ac3598974a23ad037af1cd5003f4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:17.009003 25832 tablet.cc:1724] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:17.010179 25830 tablet.cc:1724] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:17.004632 25822 tablet.cc:1724] T 9f66d125d0b24ecf8b2485004114dcd2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:17.025117 25488 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "920bc30698554542ba1e1cead26a05d3"
dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
mode: GRACEFUL
new_leader_uuid: "2128ac3598974a23ad037af1cd5003f4"
 from {username='slave'} at 127.0.0.1:60892
I20260709 12:26:17.025835 25488 raft_consensus.cc:606] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 LEADER]: Received request to transfer leadership to TS 2128ac3598974a23ad037af1cd5003f4
I20260709 12:26:17.028548  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:26:17.029188  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:17.037717 25843 tablet.cc:1724] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:17.038914 25814 tablet.cc:1724] T 61d18172d49042bb9ec71317b75eb72c P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:17.049592 25804 tablet.cc:1724] T 651a9ec9672842b6b0a0a9edb451ff33 P 997fbd9d26514ae18ff1347929d8fa12: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:17.059718 25830 tablet.cc:1724] T 01c700f42bf34944ad7cad704c5f5f36 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:17.068902 25832 tablet.cc:1724] T 61d18172d49042bb9ec71317b75eb72c P 997fbd9d26514ae18ff1347929d8fa12: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:26:17.087984 25313 catalog_manager.cc:6242] T 00000000000000000000000000000000 P c9754062669049dbb745218f0c39bff4: soft_deleted_table [id=80e824e1119c4a348c9bef53ccb1d5be] alter complete (version 1)
I20260709 12:26:17.232960 25729 raft_consensus.cc:993] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: : Instructing follower 2128ac3598974a23ad037af1cd5003f4 to start an election
I20260709 12:26:17.233662 25696 raft_consensus.cc:1081] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 LEADER]: Signalling peer 2128ac3598974a23ad037af1cd5003f4 to start an election
I20260709 12:26:17.235396 25565 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "920bc30698554542ba1e1cead26a05d3"
dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
 from {username='slave'} at 127.0.0.1:45832
I20260709 12:26:17.235932 25565 raft_consensus.cc:493] T 920bc30698554542ba1e1cead26a05d3 P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:17.236253 25565 raft_consensus.cc:3060] T 920bc30698554542ba1e1cead26a05d3 P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:17.242277 25565 raft_consensus.cc:515] T 920bc30698554542ba1e1cead26a05d3 P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:17.243952 25565 leader_election.cc:290] T 920bc30698554542ba1e1cead26a05d3 P 2128ac3598974a23ad037af1cd5003f4 [CANDIDATE]: Term 2 election: Requested vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595)
I20260709 12:26:17.244974 25413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "920bc30698554542ba1e1cead26a05d3" candidate_uuid: "2128ac3598974a23ad037af1cd5003f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "997fbd9d26514ae18ff1347929d8fa12"
I20260709 12:26:17.245262 25488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "920bc30698554542ba1e1cead26a05d3" candidate_uuid: "2128ac3598974a23ad037af1cd5003f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
I20260709 12:26:17.245642 25413 raft_consensus.cc:3060] T 920bc30698554542ba1e1cead26a05d3 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:17.245930 25488 raft_consensus.cc:3055] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:26:17.246240 25488 raft_consensus.cc:740] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e6f5fa5b38734da2a3ef8a7ce4fc6dee, State: Running, Role: LEADER
I20260709 12:26:17.247052 25488 consensus_queue.cc:260] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:17.248108 25488 raft_consensus.cc:3060] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:26:17.251006 25413 raft_consensus.cc:2468] T 920bc30698554542ba1e1cead26a05d3 P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2128ac3598974a23ad037af1cd5003f4 in term 2.
I20260709 12:26:17.252516 25531 leader_election.cc:304] T 920bc30698554542ba1e1cead26a05d3 P 2128ac3598974a23ad037af1cd5003f4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, 997fbd9d26514ae18ff1347929d8fa12; no voters: 
I20260709 12:26:17.253407 25772 raft_consensus.cc:2804] T 920bc30698554542ba1e1cead26a05d3 P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:26:17.253911 25772 raft_consensus.cc:697] T 920bc30698554542ba1e1cead26a05d3 P 2128ac3598974a23ad037af1cd5003f4 [term 2 LEADER]: Becoming Leader. State: Replica: 2128ac3598974a23ad037af1cd5003f4, State: Running, Role: LEADER
I20260709 12:26:17.254842 25772 consensus_queue.cc:237] T 920bc30698554542ba1e1cead26a05d3 P 2128ac3598974a23ad037af1cd5003f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:17.255493 25488 raft_consensus.cc:2468] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2128ac3598974a23ad037af1cd5003f4 in term 2.
I20260709 12:26:17.263739 25311 catalog_manager.cc:5697] T 920bc30698554542ba1e1cead26a05d3 P 2128ac3598974a23ad037af1cd5003f4 reported cstate change: term changed from 1 to 2, leader changed from e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2) to 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3). New cstate: current_term: 2 leader_uuid: "2128ac3598974a23ad037af1cd5003f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:17.411144 25731 raft_consensus.cc:993] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12: : Instructing follower 2128ac3598974a23ad037af1cd5003f4 to start an election
I20260709 12:26:17.411624 25731 raft_consensus.cc:1081] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12 [term 2 LEADER]: Signalling peer 2128ac3598974a23ad037af1cd5003f4 to start an election
I20260709 12:26:17.413360 25565 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "20fa2b8266614739b49be1cc8c6ee07f"
dest_uuid: "2128ac3598974a23ad037af1cd5003f4"
 from {username='slave'} at 127.0.0.1:45838
I20260709 12:26:17.413977 25565 raft_consensus.cc:493] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:17.414335 25565 raft_consensus.cc:3060] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:17.424324 25565 raft_consensus.cc:515] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:17.426270 25565 leader_election.cc:290] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4 [CANDIDATE]: Term 3 election: Requested vote from peers 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821), e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595)
I20260709 12:26:17.427214 25413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fa2b8266614739b49be1cc8c6ee07f" candidate_uuid: "2128ac3598974a23ad037af1cd5003f4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "997fbd9d26514ae18ff1347929d8fa12"
I20260709 12:26:17.427593 25488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fa2b8266614739b49be1cc8c6ee07f" candidate_uuid: "2128ac3598974a23ad037af1cd5003f4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee"
I20260709 12:26:17.428166 25413 raft_consensus.cc:3055] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:26:17.428521 25488 raft_consensus.cc:3060] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:17.428571 25413 raft_consensus.cc:740] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 997fbd9d26514ae18ff1347929d8fa12, State: Running, Role: LEADER
I20260709 12:26:17.429365 25413 consensus_queue.cc:260] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:17.430644 25413 raft_consensus.cc:3060] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:26:17.436040 25488 raft_consensus.cc:2468] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2128ac3598974a23ad037af1cd5003f4 in term 3.
I20260709 12:26:17.436655 25413 raft_consensus.cc:2468] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2128ac3598974a23ad037af1cd5003f4 in term 3.
I20260709 12:26:17.437564 25531 leader_election.cc:304] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2128ac3598974a23ad037af1cd5003f4, e6f5fa5b38734da2a3ef8a7ce4fc6dee; no voters: 
I20260709 12:26:17.438265 25772 raft_consensus.cc:2804] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:26:17.438779 25772 raft_consensus.cc:697] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4 [term 3 LEADER]: Becoming Leader. State: Replica: 2128ac3598974a23ad037af1cd5003f4, State: Running, Role: LEADER
I20260709 12:26:17.439659 25772 consensus_queue.cc:237] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } }
I20260709 12:26:17.447227 25311 catalog_manager.cc:5697] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4 reported cstate change: term changed from 2 to 3, leader changed from 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1) to 2128ac3598974a23ad037af1cd5003f4 (127.5.58.3). New cstate: current_term: 3 leader_uuid: "2128ac3598974a23ad037af1cd5003f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2128ac3598974a23ad037af1cd5003f4" member_type: VOTER last_known_addr { host: "127.5.58.3" port: 37109 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:17.714234 25488 raft_consensus.cc:1275] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Refusing update from remote peer 2128ac3598974a23ad037af1cd5003f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:17.715562 25772 consensus_queue.cc:1048] T 920bc30698554542ba1e1cead26a05d3 P 2128ac3598974a23ad037af1cd5003f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:17.733098 25413 raft_consensus.cc:1275] T 920bc30698554542ba1e1cead26a05d3 P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Refusing update from remote peer 2128ac3598974a23ad037af1cd5003f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:26:17.735957 25833 consensus_queue.cc:1048] T 920bc30698554542ba1e1cead26a05d3 P 2128ac3598974a23ad037af1cd5003f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:26:17.940052 25413 raft_consensus.cc:1275] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12 [term 3 FOLLOWER]: Refusing update from remote peer 2128ac3598974a23ad037af1cd5003f4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:26:17.941567 25833 consensus_queue.cc:1048] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "997fbd9d26514ae18ff1347929d8fa12" member_type: VOTER last_known_addr { host: "127.5.58.1" port: 33821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:26:17.955951 25488 raft_consensus.cc:1275] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 3 FOLLOWER]: Refusing update from remote peer 2128ac3598974a23ad037af1cd5003f4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:26:17.960175 25856 consensus_queue.cc:1048] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6f5fa5b38734da2a3ef8a7ce4fc6dee" member_type: VOTER last_known_addr { host: "127.5.58.2" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:26:18.030589  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:18.036715  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:26:18.037086  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:19.038170  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:19.044956  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:26:19.045382  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:20.046481  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:20.054088  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:26:20.054476  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:21.055567  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:21.060580  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:26:21.061000  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:22.062052  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:22.067129  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:26:22.067577  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:23.068670  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:23.073778  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:26:23.074275  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:24.075461  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:24.081305  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:26:24.081779  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:25.083043  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:25.088516  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:26:25.088884  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:26.090006  5352 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:26.095796  5352 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:26:26.096225  5352 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:27.163259  5352 tablet_server.cc:179] TabletServer@127.5.58.1:0 shutting down...
I20260709 12:26:27.208736  5352 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:26:27.210165  5352 tablet_replica.cc:333] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12: stopping tablet replica
I20260709 12:26:27.211231  5352 raft_consensus.cc:2243] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.212141  5352 raft_consensus.cc:2272] T 9f66d125d0b24ecf8b2485004114dcd2 P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.215366  5352 tablet_replica.cc:333] T 61d18172d49042bb9ec71317b75eb72c P 997fbd9d26514ae18ff1347929d8fa12: stopping tablet replica
I20260709 12:26:27.216193  5352 raft_consensus.cc:2243] T 61d18172d49042bb9ec71317b75eb72c P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.216913  5352 raft_consensus.cc:2272] T 61d18172d49042bb9ec71317b75eb72c P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.219825  5352 tablet_replica.cc:333] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12: stopping tablet replica
I20260709 12:26:27.220515  5352 raft_consensus.cc:2243] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.221356  5352 raft_consensus.cc:2272] T b3e8f6a25e95488b966e3229c04e03ef P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.224547  5352 tablet_replica.cc:333] T 651a9ec9672842b6b0a0a9edb451ff33 P 997fbd9d26514ae18ff1347929d8fa12: stopping tablet replica
I20260709 12:26:27.225265  5352 raft_consensus.cc:2243] T 651a9ec9672842b6b0a0a9edb451ff33 P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.226059  5352 raft_consensus.cc:2272] T 651a9ec9672842b6b0a0a9edb451ff33 P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.229966  5352 tablet_replica.cc:333] T ca1ffddacb9444a09baf22ab222ff39c P 997fbd9d26514ae18ff1347929d8fa12: stopping tablet replica
I20260709 12:26:27.230667  5352 raft_consensus.cc:2243] T ca1ffddacb9444a09baf22ab222ff39c P 997fbd9d26514ae18ff1347929d8fa12 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:26:27.231750  5352 raft_consensus.cc:2272] T ca1ffddacb9444a09baf22ab222ff39c P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.234743  5352 tablet_replica.cc:333] T 920bc30698554542ba1e1cead26a05d3 P 997fbd9d26514ae18ff1347929d8fa12: stopping tablet replica
I20260709 12:26:27.235445  5352 raft_consensus.cc:2243] T 920bc30698554542ba1e1cead26a05d3 P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.236009  5352 raft_consensus.cc:2272] T 920bc30698554542ba1e1cead26a05d3 P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.239130  5352 tablet_replica.cc:333] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12: stopping tablet replica
I20260709 12:26:27.239919  5352 raft_consensus.cc:2243] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.240613  5352 raft_consensus.cc:2272] T f1da56cb6b614019a612688d93a953dd P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.243543  5352 tablet_replica.cc:333] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12: stopping tablet replica
I20260709 12:26:27.244311  5352 raft_consensus.cc:2243] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:26:27.245394  5352 raft_consensus.cc:2272] T 15b68b570a91456ab85bdab93a936fdf P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.248091  5352 tablet_replica.cc:333] T 4ac7b3f40fee4f25935d149993ed14fe P 997fbd9d26514ae18ff1347929d8fa12: stopping tablet replica
I20260709 12:26:27.248797  5352 raft_consensus.cc:2243] T 4ac7b3f40fee4f25935d149993ed14fe P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.249408  5352 raft_consensus.cc:2272] T 4ac7b3f40fee4f25935d149993ed14fe P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.251983  5352 tablet_replica.cc:333] T 01c700f42bf34944ad7cad704c5f5f36 P 997fbd9d26514ae18ff1347929d8fa12: stopping tablet replica
I20260709 12:26:27.252648  5352 raft_consensus.cc:2243] T 01c700f42bf34944ad7cad704c5f5f36 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:26:27.253672  5352 raft_consensus.cc:2272] T 01c700f42bf34944ad7cad704c5f5f36 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.256505  5352 tablet_replica.cc:333] T ed736971b10844fbada6e03f39434740 P 997fbd9d26514ae18ff1347929d8fa12: stopping tablet replica
I20260709 12:26:27.257195  5352 raft_consensus.cc:2243] T ed736971b10844fbada6e03f39434740 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:26:27.258172  5352 raft_consensus.cc:2272] T ed736971b10844fbada6e03f39434740 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.260964  5352 tablet_replica.cc:333] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 997fbd9d26514ae18ff1347929d8fa12: stopping tablet replica
I20260709 12:26:27.261695  5352 raft_consensus.cc:2243] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:26:27.262986  5352 raft_consensus.cc:2272] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:27.264094 25457 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 15 similar messages]
I20260709 12:26:27.266728  5352 tablet_replica.cc:333] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12: stopping tablet replica
I20260709 12:26:27.267959  5352 raft_consensus.cc:2243] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.268893  5352 raft_consensus.cc:2272] T 7fa678a733d149f0b6aaa02a76eeb81c P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:27.270251 25457 consensus_peers.cc:597] T b3e8f6a25e95488b966e3229c04e03ef P e6f5fa5b38734da2a3ef8a7ce4fc6dee -> Peer 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821): Couldn't send request to peer 997fbd9d26514ae18ff1347929d8fa12. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:27.272157  5352 tablet_replica.cc:333] T f19c0845761542bab632f9298955bbec P 997fbd9d26514ae18ff1347929d8fa12: stopping tablet replica
I20260709 12:26:27.272936  5352 raft_consensus.cc:2243] T f19c0845761542bab632f9298955bbec P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.273492  5352 raft_consensus.cc:2272] T f19c0845761542bab632f9298955bbec P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.275960  5352 tablet_replica.cc:333] T c02e9bd829f8498b9af872415fd82d93 P 997fbd9d26514ae18ff1347929d8fa12: stopping tablet replica
I20260709 12:26:27.276540  5352 raft_consensus.cc:2243] T c02e9bd829f8498b9af872415fd82d93 P 997fbd9d26514ae18ff1347929d8fa12 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:26:27.277519  5352 raft_consensus.cc:2272] T c02e9bd829f8498b9af872415fd82d93 P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.280644  5352 tablet_replica.cc:333] T 18f5ed022d284752a9a35292d6adecb2 P 997fbd9d26514ae18ff1347929d8fa12: stopping tablet replica
I20260709 12:26:27.281302  5352 raft_consensus.cc:2243] T 18f5ed022d284752a9a35292d6adecb2 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.282259  5352 raft_consensus.cc:2272] T 18f5ed022d284752a9a35292d6adecb2 P 997fbd9d26514ae18ff1347929d8fa12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.285919  5352 tablet_replica.cc:333] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12: stopping tablet replica
I20260709 12:26:27.286543  5352 raft_consensus.cc:2243] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.287216  5352 raft_consensus.cc:2272] T 20fa2b8266614739b49be1cc8c6ee07f P 997fbd9d26514ae18ff1347929d8fa12 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.289973  5352 tablet_replica.cc:333] T d23a8c8d8bbd4694b5a7346938ed345c P 997fbd9d26514ae18ff1347929d8fa12: stopping tablet replica
I20260709 12:26:27.290599  5352 raft_consensus.cc:2243] T d23a8c8d8bbd4694b5a7346938ed345c P 997fbd9d26514ae18ff1347929d8fa12 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:26:27.291615  5352 raft_consensus.cc:2272] T d23a8c8d8bbd4694b5a7346938ed345c P 997fbd9d26514ae18ff1347929d8fa12 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:27.334712 25382 connection.cc:466] server connection from 127.0.0.1:43580 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43580 (request call id 137) could send its response
W20260709 12:26:27.336894 25531 consensus_peers.cc:597] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4 -> Peer 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821): Couldn't send request to peer 997fbd9d26514ae18ff1347929d8fa12. Status: Network error: recv got EOF from 127.5.58.1:33821 (error 108). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:27.351575 25531 consensus_peers.cc:597] T f1da56cb6b614019a612688d93a953dd P 2128ac3598974a23ad037af1cd5003f4 -> Peer 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821): Couldn't send request to peer 997fbd9d26514ae18ff1347929d8fa12. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:33821: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:27.353214  5352 tablet_server.cc:196] TabletServer@127.5.58.1:0 shutdown complete.
W20260709 12:26:27.353358 25457 consensus_peers.cc:597] T 18f5ed022d284752a9a35292d6adecb2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee -> Peer 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821): Couldn't send request to peer 997fbd9d26514ae18ff1347929d8fa12. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:33821: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:27.358013 25531 consensus_peers.cc:597] T 920bc30698554542ba1e1cead26a05d3 P 2128ac3598974a23ad037af1cd5003f4 -> Peer 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821): Couldn't send request to peer 997fbd9d26514ae18ff1347929d8fa12. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:33821: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:27.402881  5352 tablet_server.cc:179] TabletServer@127.5.58.2:0 shutting down...
W20260709 12:26:27.419828 25457 consensus_peers.cc:597] T f19c0845761542bab632f9298955bbec P e6f5fa5b38734da2a3ef8a7ce4fc6dee -> Peer 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821): Couldn't send request to peer 997fbd9d26514ae18ff1347929d8fa12. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:33821: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:27.440199  5352 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:26:27.441502  5352 tablet_replica.cc:333] T 9f66d125d0b24ecf8b2485004114dcd2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: stopping tablet replica
I20260709 12:26:27.442677  5352 raft_consensus.cc:2243] T 9f66d125d0b24ecf8b2485004114dcd2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.443351  5352 raft_consensus.cc:2272] T 9f66d125d0b24ecf8b2485004114dcd2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.445993  5352 tablet_replica.cc:333] T b3e8f6a25e95488b966e3229c04e03ef P e6f5fa5b38734da2a3ef8a7ce4fc6dee: stopping tablet replica
I20260709 12:26:27.446782  5352 raft_consensus.cc:2243] T b3e8f6a25e95488b966e3229c04e03ef P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:26:27.447921  5352 raft_consensus.cc:2272] T b3e8f6a25e95488b966e3229c04e03ef P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.451243  5352 tablet_replica.cc:333] T 61d18172d49042bb9ec71317b75eb72c P e6f5fa5b38734da2a3ef8a7ce4fc6dee: stopping tablet replica
I20260709 12:26:27.451997  5352 raft_consensus.cc:2243] T 61d18172d49042bb9ec71317b75eb72c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:26:27.452412 25457 consensus_peers.cc:597] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee -> Peer 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821): Couldn't send request to peer 997fbd9d26514ae18ff1347929d8fa12. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:33821: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:27.453423  5352 raft_consensus.cc:2272] T 61d18172d49042bb9ec71317b75eb72c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.457180  5352 tablet_replica.cc:333] T 01c700f42bf34944ad7cad704c5f5f36 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: stopping tablet replica
W20260709 12:26:27.457805 25457 consensus_peers.cc:597] T 7fa678a733d149f0b6aaa02a76eeb81c P e6f5fa5b38734da2a3ef8a7ce4fc6dee -> Peer 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821): Couldn't send request to peer 997fbd9d26514ae18ff1347929d8fa12. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:33821: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:27.458202  5352 raft_consensus.cc:2243] T 01c700f42bf34944ad7cad704c5f5f36 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.458879  5352 raft_consensus.cc:2272] T 01c700f42bf34944ad7cad704c5f5f36 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.461408  5352 tablet_replica.cc:333] T ed736971b10844fbada6e03f39434740 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: stopping tablet replica
I20260709 12:26:27.462361  5352 raft_consensus.cc:2243] T ed736971b10844fbada6e03f39434740 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.462893  5352 raft_consensus.cc:2272] T ed736971b10844fbada6e03f39434740 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.465468  5352 tablet_replica.cc:333] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: stopping tablet replica
I20260709 12:26:27.465987  5352 raft_consensus.cc:2243] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:26:27.466845  5352 raft_consensus.cc:2272] T 651a9ec9672842b6b0a0a9edb451ff33 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.469241  5352 tablet_replica.cc:333] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee: stopping tablet replica
I20260709 12:26:27.469820  5352 raft_consensus.cc:2243] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.470328  5352 raft_consensus.cc:2272] T ca1ffddacb9444a09baf22ab222ff39c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.472946  5352 tablet_replica.cc:333] T 7fa678a733d149f0b6aaa02a76eeb81c P e6f5fa5b38734da2a3ef8a7ce4fc6dee: stopping tablet replica
I20260709 12:26:27.473824  5352 raft_consensus.cc:2243] T 7fa678a733d149f0b6aaa02a76eeb81c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:26:27.475150  5352 raft_consensus.cc:2272] T 7fa678a733d149f0b6aaa02a76eeb81c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.478598  5352 tablet_replica.cc:333] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: stopping tablet replica
W20260709 12:26:27.478818 25531 consensus_peers.cc:597] T 9f66d125d0b24ecf8b2485004114dcd2 P 2128ac3598974a23ad037af1cd5003f4 -> Peer 997fbd9d26514ae18ff1347929d8fa12 (127.5.58.1:33821): Couldn't send request to peer 997fbd9d26514ae18ff1347929d8fa12. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.1:33821: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:27.479542  5352 raft_consensus.cc:2243] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.480134  5352 raft_consensus.cc:2272] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.482772  5352 tablet_replica.cc:333] T 18f5ed022d284752a9a35292d6adecb2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: stopping tablet replica
I20260709 12:26:27.483629  5352 raft_consensus.cc:2243] T 18f5ed022d284752a9a35292d6adecb2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:26:27.484840  5352 raft_consensus.cc:2272] T 18f5ed022d284752a9a35292d6adecb2 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.487916  5352 tablet_replica.cc:333] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee: stopping tablet replica
I20260709 12:26:27.488700  5352 raft_consensus.cc:2243] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.489360  5352 raft_consensus.cc:2272] T 20fa2b8266614739b49be1cc8c6ee07f P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.492827  5352 tablet_replica.cc:333] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: stopping tablet replica
I20260709 12:26:27.493643  5352 raft_consensus.cc:2243] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.494264  5352 raft_consensus.cc:2272] T c02e9bd829f8498b9af872415fd82d93 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.497299  5352 tablet_replica.cc:333] T f19c0845761542bab632f9298955bbec P e6f5fa5b38734da2a3ef8a7ce4fc6dee: stopping tablet replica
I20260709 12:26:27.498072  5352 raft_consensus.cc:2243] T f19c0845761542bab632f9298955bbec P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:26:27.499282  5352 raft_consensus.cc:2272] T f19c0845761542bab632f9298955bbec P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:27.501065 25531 consensus_peers.cc:597] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4 -> Peer e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595): Couldn't send request to peer e6f5fa5b38734da2a3ef8a7ce4fc6dee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:27.503054  5352 tablet_replica.cc:333] T 15b68b570a91456ab85bdab93a936fdf P e6f5fa5b38734da2a3ef8a7ce4fc6dee: stopping tablet replica
I20260709 12:26:27.503795  5352 raft_consensus.cc:2243] T 15b68b570a91456ab85bdab93a936fdf P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.504618  5352 raft_consensus.cc:2272] T 15b68b570a91456ab85bdab93a936fdf P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.507725  5352 tablet_replica.cc:333] T 4ac7b3f40fee4f25935d149993ed14fe P e6f5fa5b38734da2a3ef8a7ce4fc6dee: stopping tablet replica
I20260709 12:26:27.508335  5352 raft_consensus.cc:2243] T 4ac7b3f40fee4f25935d149993ed14fe P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:26:27.509342  5352 raft_consensus.cc:2272] T 4ac7b3f40fee4f25935d149993ed14fe P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.511889  5352 tablet_replica.cc:333] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee: stopping tablet replica
I20260709 12:26:27.512523  5352 raft_consensus.cc:2243] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.513037  5352 raft_consensus.cc:2272] T 920bc30698554542ba1e1cead26a05d3 P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.515628  5352 tablet_replica.cc:333] T f1da56cb6b614019a612688d93a953dd P e6f5fa5b38734da2a3ef8a7ce4fc6dee: stopping tablet replica
I20260709 12:26:27.516134  5352 raft_consensus.cc:2243] T f1da56cb6b614019a612688d93a953dd P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.516661  5352 raft_consensus.cc:2272] T f1da56cb6b614019a612688d93a953dd P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.519068  5352 tablet_replica.cc:333] T d23a8c8d8bbd4694b5a7346938ed345c P e6f5fa5b38734da2a3ef8a7ce4fc6dee: stopping tablet replica
I20260709 12:26:27.519536  5352 raft_consensus.cc:2243] T d23a8c8d8bbd4694b5a7346938ed345c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.519989  5352 raft_consensus.cc:2272] T d23a8c8d8bbd4694b5a7346938ed345c P e6f5fa5b38734da2a3ef8a7ce4fc6dee [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.568462  5352 tablet_server.cc:196] TabletServer@127.5.58.2:0 shutdown complete.
I20260709 12:26:27.614063  5352 tablet_server.cc:179] TabletServer@127.5.58.3:0 shutting down...
W20260709 12:26:27.614091 25531 consensus_peers.cc:597] T 920bc30698554542ba1e1cead26a05d3 P 2128ac3598974a23ad037af1cd5003f4 -> Peer e6f5fa5b38734da2a3ef8a7ce4fc6dee (127.5.58.2:45595): Couldn't send request to peer e6f5fa5b38734da2a3ef8a7ce4fc6dee. Status: Network error: Client connection negotiation failed: client connection to 127.5.58.2:45595: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:27.655373  5352 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:26:27.656209  5352 tablet_replica.cc:333] T 9f66d125d0b24ecf8b2485004114dcd2 P 2128ac3598974a23ad037af1cd5003f4: stopping tablet replica
I20260709 12:26:27.656977  5352 raft_consensus.cc:2243] T 9f66d125d0b24ecf8b2485004114dcd2 P 2128ac3598974a23ad037af1cd5003f4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:26:27.657898  5352 raft_consensus.cc:2272] T 9f66d125d0b24ecf8b2485004114dcd2 P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.660184  5352 tablet_replica.cc:333] T 61d18172d49042bb9ec71317b75eb72c P 2128ac3598974a23ad037af1cd5003f4: stopping tablet replica
I20260709 12:26:27.660867  5352 raft_consensus.cc:2243] T 61d18172d49042bb9ec71317b75eb72c P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.661335  5352 raft_consensus.cc:2272] T 61d18172d49042bb9ec71317b75eb72c P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.663420  5352 tablet_replica.cc:333] T b3e8f6a25e95488b966e3229c04e03ef P 2128ac3598974a23ad037af1cd5003f4: stopping tablet replica
I20260709 12:26:27.663937  5352 raft_consensus.cc:2243] T b3e8f6a25e95488b966e3229c04e03ef P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.664474  5352 raft_consensus.cc:2272] T b3e8f6a25e95488b966e3229c04e03ef P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.666596  5352 tablet_replica.cc:333] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4: stopping tablet replica
I20260709 12:26:27.667120  5352 raft_consensus.cc:2243] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.667601  5352 raft_consensus.cc:2272] T 651a9ec9672842b6b0a0a9edb451ff33 P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.669876  5352 tablet_replica.cc:333] T ca1ffddacb9444a09baf22ab222ff39c P 2128ac3598974a23ad037af1cd5003f4: stopping tablet replica
I20260709 12:26:27.670506  5352 raft_consensus.cc:2243] T ca1ffddacb9444a09baf22ab222ff39c P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.671013  5352 raft_consensus.cc:2272] T ca1ffddacb9444a09baf22ab222ff39c P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.673314  5352 tablet_replica.cc:333] T f1da56cb6b614019a612688d93a953dd P 2128ac3598974a23ad037af1cd5003f4: stopping tablet replica
I20260709 12:26:27.673920  5352 raft_consensus.cc:2243] T f1da56cb6b614019a612688d93a953dd P 2128ac3598974a23ad037af1cd5003f4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:26:27.674746  5352 raft_consensus.cc:2272] T f1da56cb6b614019a612688d93a953dd P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.677115  5352 tablet_replica.cc:333] T 920bc30698554542ba1e1cead26a05d3 P 2128ac3598974a23ad037af1cd5003f4: stopping tablet replica
I20260709 12:26:27.677764  5352 raft_consensus.cc:2243] T 920bc30698554542ba1e1cead26a05d3 P 2128ac3598974a23ad037af1cd5003f4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:26:27.678673  5352 raft_consensus.cc:2272] T 920bc30698554542ba1e1cead26a05d3 P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.680898  5352 tablet_replica.cc:333] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4: stopping tablet replica
I20260709 12:26:27.681476  5352 raft_consensus.cc:2243] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.681948  5352 raft_consensus.cc:2272] T 15b68b570a91456ab85bdab93a936fdf P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.684057  5352 tablet_replica.cc:333] T 4ac7b3f40fee4f25935d149993ed14fe P 2128ac3598974a23ad037af1cd5003f4: stopping tablet replica
I20260709 12:26:27.684741  5352 raft_consensus.cc:2243] T 4ac7b3f40fee4f25935d149993ed14fe P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.685254  5352 raft_consensus.cc:2272] T 4ac7b3f40fee4f25935d149993ed14fe P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.687288  5352 tablet_replica.cc:333] T ed736971b10844fbada6e03f39434740 P 2128ac3598974a23ad037af1cd5003f4: stopping tablet replica
I20260709 12:26:27.687872  5352 raft_consensus.cc:2243] T ed736971b10844fbada6e03f39434740 P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.688448  5352 raft_consensus.cc:2272] T ed736971b10844fbada6e03f39434740 P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.690507  5352 tablet_replica.cc:333] T 01c700f42bf34944ad7cad704c5f5f36 P 2128ac3598974a23ad037af1cd5003f4: stopping tablet replica
I20260709 12:26:27.691061  5352 raft_consensus.cc:2243] T 01c700f42bf34944ad7cad704c5f5f36 P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.692185  5352 raft_consensus.cc:2272] T 01c700f42bf34944ad7cad704c5f5f36 P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.694541  5352 tablet_replica.cc:333] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4: stopping tablet replica
I20260709 12:26:27.695102  5352 raft_consensus.cc:2243] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.695659  5352 raft_consensus.cc:2272] T f19c0845761542bab632f9298955bbec P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.697901  5352 tablet_replica.cc:333] T 7fa678a733d149f0b6aaa02a76eeb81c P 2128ac3598974a23ad037af1cd5003f4: stopping tablet replica
I20260709 12:26:27.698455  5352 raft_consensus.cc:2243] T 7fa678a733d149f0b6aaa02a76eeb81c P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.698939  5352 raft_consensus.cc:2272] T 7fa678a733d149f0b6aaa02a76eeb81c P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.701274  5352 tablet_replica.cc:333] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 2128ac3598974a23ad037af1cd5003f4: stopping tablet replica
I20260709 12:26:27.702000  5352 raft_consensus.cc:2243] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.702877  5352 raft_consensus.cc:2272] T 3189aaf08cab4d40b1057fbe7f1dc5a0 P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.706312  5352 tablet_replica.cc:333] T c02e9bd829f8498b9af872415fd82d93 P 2128ac3598974a23ad037af1cd5003f4: stopping tablet replica
I20260709 12:26:27.706959  5352 raft_consensus.cc:2243] T c02e9bd829f8498b9af872415fd82d93 P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.707496  5352 raft_consensus.cc:2272] T c02e9bd829f8498b9af872415fd82d93 P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.710085  5352 tablet_replica.cc:333] T 18f5ed022d284752a9a35292d6adecb2 P 2128ac3598974a23ad037af1cd5003f4: stopping tablet replica
I20260709 12:26:27.710686  5352 raft_consensus.cc:2243] T 18f5ed022d284752a9a35292d6adecb2 P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.711603  5352 raft_consensus.cc:2272] T 18f5ed022d284752a9a35292d6adecb2 P 2128ac3598974a23ad037af1cd5003f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.714644  5352 tablet_replica.cc:333] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4: stopping tablet replica
I20260709 12:26:27.715477  5352 raft_consensus.cc:2243] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:26:27.717286  5352 raft_consensus.cc:2272] T 20fa2b8266614739b49be1cc8c6ee07f P 2128ac3598974a23ad037af1cd5003f4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.719696  5352 tablet_replica.cc:333] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4: stopping tablet replica
I20260709 12:26:27.720221  5352 raft_consensus.cc:2243] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:27.720852  5352 raft_consensus.cc:2272] T d23a8c8d8bbd4694b5a7346938ed345c P 2128ac3598974a23ad037af1cd5003f4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.770005  5352 tablet_server.cc:196] TabletServer@127.5.58.3:0 shutdown complete.
I20260709 12:26:27.813426  5352 master.cc:562] Master@127.5.58.62:35569 shutting down...
I20260709 12:26:27.844244  5352 raft_consensus.cc:2243] T 00000000000000000000000000000000 P c9754062669049dbb745218f0c39bff4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:26:27.844914  5352 raft_consensus.cc:2272] T 00000000000000000000000000000000 P c9754062669049dbb745218f0c39bff4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:27.845196  5352 tablet_replica.cc:333] T 00000000000000000000000000000000 P c9754062669049dbb745218f0c39bff4: stopping tablet replica
I20260709 12:26:27.869132  5352 master.cc:584] Master@127.5.58.62:35569 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (20970 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (42356 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (645906 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.RestartTserver

 1 FAILED TEST
I20260709 12:26:27.944576  5352 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 2180 messages since previous log ~32 seconds ago
I20260709 12:26:27.944880  5352 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 2188 messages since previous log ~32 seconds ago
I20260709 12:26:27.945104  5352 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 2188 messages since previous log ~32 seconds ago
I20260709 12:26:27.945343  5352 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~55 seconds ago
I20260709 12:26:27.945576  5352 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~55 seconds ago
I20260709 12:26:27.945843  5352 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 11 messages since previous log ~151 seconds ago
I20260709 12:26:27.946084  5352 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/master/catalog_manager.cc:5227: suppressed but not reported on 3 messages since previous log ~327 seconds ago
I20260709 12:26:27.946295  5352 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/consensus/raft_consensus.cc:670: suppressed but not reported on 1 messages since previous log ~525 seconds ago
I20260709 12:26:27.946602  5352 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 10 messages since previous log ~0 seconds ago
I20260709 12:26:27.946952  5352 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~19 seconds ago
